In this post:
Transactions properties
Transactions constraints
Examples:
- mongo shell
- Pure PHP
- Symfony + Doctrine:
- TransationalDocumentManager step-by-step guide
In MongoDB, regardless of its version, an operation on a single document is atomic. Because you can use embedded documents and arrays to capture relationships between data in a single document structure instead of normalizing across multiple documents and collections, this single-document atomicity obviates the need for multi-document transactions for many practical use cases.
With the release of version 4.0, we now have multi-document ACID transactions in MongoDB.
In MongoDB 4.0 however (including current - at the time of writing this post - version 4.0.9) transactions work across a replica set only, and MongoDB 4.2 will extend support to transactions across a sharded deployment.
Which means that they are not supported in standalone server installations.
Q&A: How do transactions work in MongoDB?
Through snapshot isolation, transactions provide a consistent view of data, and enforce all-or-nothing execution to maintain data integrity. Transactions can apply to operations against multiple documents contained in one, or in many, collections and databases. The changes to MongoDB that enable multi-document transactions do not impact performance for workloads that don't require them.
During its execution, a transaction is able to read its own uncommitted writes, but none of its uncommitted writes will be seen by other operations outside of the transaction. Uncommitted writes are not replicated to secondary nodes until the transaction is committed to the database. Once the transaction has been committed, it is replicated and applied atomically to all secondary replicas.
An application specifies write concern in the transaction options to state how many nodes should commit the changes before the server acknowledges the success to the client. All uncommitted writes live on the primary exclusively.
Taking advantage of the transactions infrastructure introduced in MongoDB 4.0, the new snapshot read concern ensures queries and aggregations executed within a read-only transaction will operate against a single, isolated snapshot on the primary replica. As a result, a consistent view of the data is returned to the client, irrespective of whether that data is being simultaneously modified by concurrent operations. Snapshot reads are especially useful for operations that return data in batches with the getMore command.
Even before MongoDB 4.0, typical MongoDB queries leveraged WiredTiger snapshots. The distinction between typical MongoDB queries and snapshot reads in transactions is that snapshot reads use the same snapshot throughout the duration of the query. Whereas typical MongoDB queries may switch to a more current snapshot during yield points.
Read more in MongoDB Multi-Document ACID Transactions are GA blog post and MongoDB documentation for multi-document transactions
Transaction properties
- Multi-document transactions can be used across multiple operations, collections, databases, and documents.
- Multi-document transactions provide an “all-or-nothing” proposition.
- When a transaction commits, all data changes made in the transaction are saved.
- If any operation in the transaction fails, the transaction aborts and all data changes made in the transaction are discarded without ever becoming visible.
- Until a transaction commits, no write operations in the transaction are visible outside the transaction.
Transaction constraints
- You can execute CRUD operations only on existing collections.
- A multi-document transaction cannot include an insert operation that would result in the creation of a new collection.
- Transactions always are associated with a session.
- At any given time, you can have at most one open transaction for a session.
- To associate read and write operations with an open transaction, you need to pass the session to the operation.
- Transactions with write conflicts are aborted.
Q&A: What are transactions best practices?
By default, MongoDB will automatically abort any multi-document transaction that runs for more than 60 seconds. Note that if write volumes to the server are low, you have the flexibility to tune your transactions for a longer execution time. To address timeouts, the transaction should be broken into smaller parts that allow execution within the configured time limit. You should also ensure your query patterns are properly optimized with the appropriate index coverage to allow fast data access within the transaction.
There are no hard limits to the number of documents that can be read within a transaction. As a best practice, no more than 1,000 documents should be modified within a transaction. For operations that need to modify more than 1,000 documents, developers should break the transaction into separate parts that process documents in batches.
In MongoDB 4.0, a transaction is represented in a single oplog entry, therefore must be within the 16MB document size limit. While an update operation only stores the deltas of the update (i.e., what has changed), an insert will store the entire document. As a result, the combination of oplog descriptions for all statements in the transaction must be less than 16MB. If this limit is exceeded, the transaction will be aborted and fully rolled back. The transaction should therefore be decomposed into a smaller set of operations that can be represented in 16MB or less.
When a transaction aborts, an exception is returned to the driver and the transaction is fully rolled back. Developers should add application logic that can catch and retry a transaction that aborts due to temporary exceptions, such as a transient network failure or a primary replica election. With retryable writes, the MongoDB drivers will automatically retry the commit statement of the transaction.
DDL operations, like creating an index or dropping a database, block behind active running transactions on the namespace. All transactions that try to newly access the namespace while DDL operations are pending, will not be able to obtain locks, aborting the new transactions.
Read more in MongoDB Multi-Document ACID Transactions are GA blog post and MongoDB documentation for multi-document transactions
Examples
mongo shell
The following mongo shell example highlights the key components of using transactions:
// Start a session.
session = db.getMongo().startSession( { readPreference: { mode: "primary" } } );
employeesCollection = session.getDatabase("hr").employees;
eventsCollection = session.getDatabase("reporting").events;
// Start a transaction
session.startTransaction( { readConcern: { level: "snapshot" }, writeConcern: { w: "majority" } } );
// Operations inside the transaction
try {
employeesCollection.updateOne( { employee: 3 }, { $set: { status: "Inactive" } } );
eventsCollection.insertOne( { employee: 3, status: { new: "Inactive", old: "Active" } } );
} catch (error) {
// Abort transaction on error
session.abortTransaction();
throw error;
}
// Commit the transaction using write concern set at transaction start
session.commitTransaction();
session.endSession();
Pure PHP
If you're using the PHP library that wraps the driver, after creating an instance of Client
e.g. called $client
, you can do the following:
$session = $client->startSession();
$session->startTransaction();
try {
// Perform actions.
$session->commitTransaction();
} catch(Exception $e) {
$session->abortTransaction();
}
Symfony with Doctrine and DoctrineMongoDBBundle
This example is based on Symfony 4 with DoctrineMongoDBBundle (doctrine/mongodb-odm-bundle
) installed,
which integrates Doctrine2 MongoDB Object Document Mapper (ODM) library (doctrine/mongodb-odm
) into Symfony.
composer require doctrine/mongodb-odm-bundle alcaeus/mongo-php-adapter
The problem with this ODM is that it does not support multi-document transaction out of the box:
As per the documentation, MongoDB write operations are "atomic on the level of a single document".
Even when updating multiple documents within a single write operation, though the modification of each document is atomic, the operation as a whole is not and other operations may interleave.
As stated in the FAQ, "MongoDB does not support multi-document transactions" and neither does Doctrine MongoDB ODM.
But even with transaction support missing from Doctrine MongoDB ODM library, we can still use MongoDB multi-document transactions with few little tricks.
Step-by-step guide
Because Doctrine MongoDB ODM library does not support transactions, will will use native PHP MongoDB driver instead. First though we need to dig it out of the depths of Doctrine.
Let's start with creating new TransactionalDocumentManager
which will extend Doctrine ODM's native DocumentManager
:
// src/Manager/MongoDB/TransactionalDocumentManager.php
namespace App\Manager\MongoDB;
use Doctrine\ODM\MongoDB\DocumentManager;
/**
* Class TransactionalDocumentManager
*
* Extends Doctrine2 MongoDB Object Document Mapper (ODM) library
* to add support for sessions and multi-document transactions.
*
* @link https://www.doctrine-project.org/projects/mongodb-odm.html
* @link https://docs.mongodb.com/master/core/transactions/
*/
class TransactionalDocumentManager extends DocumentManager
{
}
What we will need to be able to use sessions and transactions is native PHP's \MongoDB\Client
, which we can easily dig out of Doctrine:
/** @var \Doctrine\MongoDB\Connection $mongoConnection */
$mongoConnection = $this->getConnection();
/** @var \MongoClient $mongoClient */
$mongoClient = $mongoConnection->getMongoClient();
/** @var \MongoDB\Client $phpClient */
$phpClient = $mongoClient->getClient();
Converting it into constructor for our TransactionalDocumentManager
class:
// src/Manager/MongoDB/TransactionalDocumentManager.php
class TransactionalDocumentManager extends DocumentManager
{
/**
* @var \MongoDB\Client
*/
protected $phpClient;
/**
* {@inheritDoc}
*/
public function __construct(
Connection $conn = null,
Configuration $config = null,
EventManager $eventManager = null
) {
parent::__construct($conn, $config, $eventManager);
$this->phpClient = $this->getConnection()
->getMongoClient()
->getClient();
}
}
To have all services injected into our TransactionalDocumentManager
we need to define them in the services.yaml
:
// config/services.yaml
services:
App\Manager\MongoDB\TransactionalDocumentManager:
public: true
arguments:
- "@doctrine_mongodb.odm.default_connection"
- "@doctrine_mongodb.odm.default_configuration"
- "@doctrine_mongodb.odm.default_connection.event_manager"
Now, having access to \MongoDB\Client
we can start a \MongoDB\Driver\Session
:
use MongoDB\Driver\ReadPreference;
/** @var \MongoDB\Driver\Session $session */
$session = $this->phpClient->startSession([
'readPreference' => new ReadPreference(ReadPreference::RP_PRIMARY),
]);
Q&A: What are MongoDB sessions?
MongoDB’s server sessions, or logical sessions, are the underlying framework used by client sessions to support Causal Consistency and retryable writes. Server sessions are available for replica sets and sharded clusters only.
Applications use client sessions to interface with server sessions.
Read more in MongoDB documentation for Server Sessions and Client Sessions
Q&A: What is readPreference
?
Read preference describes how MongoDB clients route read operations to the members of a replica set.
primary
is the default mode. All operations read from the current replica set primary.
Multi-document transactions that contain read operations must use read preference primary.
All operations in a given transaction must route to the same member.
Read more in MongoDB documentation for Read Preference
And having session we can start a new transaction:
use MongoDB\Driver\ReadConcern;
use MongoDB\Driver\WriteConcern;
$session->startTransaction([
'readConcern' => new ReadConcern('snapshot'),
'writeConcern' => new WriteConcern(WriteConcern::MAJORITY),
]);
Q&A: What is readConcern
?
The readConcern option allows you to control the consistency and isolation properties of the data read from replica sets and replica set shards.
Through the effective use of write concerns and read concerns, you can adjust the level of consistency and availability guarantees as appropriate, such as waiting for stronger consistency guarantees, or loosening consistency requirements to provide higher availability.
Read concern "snapshot" is available only for operations within multi-document transactions.
- If the transaction is not part of a causally consistent session, upon transaction commit with write concern "majority", the transaction operations are guaranteed to have read from a snapshot of majority-committed data.
- If the transaction is part of a causally consistent session, upon transaction commit with write concern "majority", the transaction operations are guaranteed to have read from a snapshot of majority-committed data that provides causal consistency with the operation immediately preceding the transaction start.
Read more in MongoDB documentation for Read Concern
Q&A: What is writeConcern
?
Write concern describes the level of acknowledgment requested from MongoDB for write operations to a standalone mongod or to replica sets or to sharded clusters. In sharded clusters, mongos instances will pass the write concern on to the shards.
For multi-document transactions, you set the write concern at the transaction level, not at the individual operation level. Do not explicitly set the write concern for individual write operations in a transaction.
Write concern "majority" requests acknowledgment that write operations have propagated to the majority (M) of data-bearing voting members. The majority (M) is calculated as the majority of all voting members, but the write operation returns acknowledgement after propagating to M-number of data-bearing voting members (primary and secondaries with members[n].votes greater than 0).
For example, consider a replica set with 3 voting members, Primary-Secondary-Secondary (P-S-S). For this replica set, M is two [1], and the write must propagate specifically to the primary and one secondary to acknowledge the write concern to the client.
Read more in MongoDB documentation for Write Concern
Adding all that (and few extra helper methods) to our TransactionalDocumentManager
class we get its final version:
TransactionalDocumentManager
// src/Manager/MongoDB/TransactionalDocumentManager.php
namespace App\Manager\MongoDB;
use Doctrine\Common\EventManager;
use Doctrine\MongoDB\Connection;
use Doctrine\ODM\MongoDB\Configuration;
use Doctrine\ODM\MongoDB\DocumentManager;
use MongoDB\Driver\Exception\RuntimeException;
use MongoDB\Driver\ReadConcern;
use MongoDB\Driver\ReadPreference;
use MongoDB\Driver\Session;
use MongoDB\Driver\WriteConcern;
/**
* Class TransactionalDocumentManager
*
* Extends Doctrine2 MongoDB Object Document Mapper (ODM) library
* to add support for sessions and multi-document transactions.
*
* @link https://www.doctrine-project.org/projects/mongodb-odm.html
* @link https://docs.mongodb.com/master/core/transactions/
*/
class TransactionalDocumentManager extends DocumentManager
{
/**
* @var \MongoDB\Client
*/
protected $phpClient;
/**
* @var \MongoDB\Driver\Session
*/
private $session;
/**
* {@inheritDoc}
*/
public function __construct(
Connection $conn = null,
Configuration $config = null,
EventManager $eventManager = null
) {
parent::__construct($conn, $config, $eventManager);
$this->phpClient = $this->getConnection()
->getMongoClient()
->getClient();
}
/**
* Start a transaction.
*
* @link http://php.net/manual/en/mongodb-driver-session.starttransaction.php
* @link https://docs.mongodb.com/manual/reference/method/Session.startTransaction/
* @link https://docs.mongodb.com/master/core/transactions/
* @link https://docs.mongodb.com/manual/reference/read-concern-snapshot/
* @link https://docs.mongodb.com/manual/reference/write-concern/#writeconcern._dq_majority_dq_
*
* @param array $options
*
* @return void
*/
public function startTransaction($options = [])
{
$this->startSession();
$options = [
'readConcern' => new ReadConcern('snapshot'),
'writeConcern' => new WriteConcern(WriteConcern::MAJORITY),
] + $options;
$this->session->startTransaction($options);
}
/**
* Commit current transaction.
*
* @link http://php.net/manual/en/mongodb-driver-session.committransaction.php
* @link https://docs.mongodb.com/manual/reference/method/Session.commitTransaction/
* @link https://docs.mongodb.com/manual/reference/command/commitTransaction/
* @link https://docs.mongodb.com/master/core/transactions/
*
* @return void
*/
public function commitTransaction()
{
$this->session->commitTransaction();
}
/**
* Abort current transaction.
*
* @link http://php.net/manual/en/mongodb-driver-session.aborttransaction.php
* @link https://docs.mongodb.com/manual/reference/method/Session.abortTransaction/
* @link https://docs.mongodb.com/manual/reference/command/abortTransaction/
* @link https://docs.mongodb.com/master/core/transactions/
*
* @return void
*/
public function abortTransaction()
{
$this->session->abortTransaction();
}
/**
* Start a new client session.
*
* @link http://php.net/manual/en/mongodb-driver-manager.startsession.php
* @link https://docs.mongodb.com/manual/reference/server-sessions/
* @link https://docs.mongodb.com/manual/reference/method/Session/
* @link https://docs.mongodb.com/manual/reference/read-preference/#primary
*
* @param array $options
*
* @return void
*/
public function startSession($options = [])
{
if (!$this->session instanceof Session) {
$options = [
'readPreference' => new ReadPreference(ReadPreference::RP_PRIMARY),
] + $options;
$this->session = $this->phpClient->startSession($options);
} else {
throw new RuntimeException('Session already started.');
}
}
/**
* Get current session.
*
* @link https://docs.mongodb.com/manual/reference/server-sessions/
* @link https://docs.mongodb.com/manual/reference/method/Session/
*
* @return \MongoDB\Driver\Session
*/
public function getSession()
{
return $this->session;
}
/**
* End current session.
*
* @link http://php.net/manual/en/mongodb-driver-session.endsession.php
* @link https://docs.mongodb.com/manual/reference/server-sessions/
* @link https://docs.mongodb.com/manual/reference/method/Session/
*
* @return void
*/
public function endSession()
{
if ($this->session instanceof Session) {
$this->session->endSession();
} else {
throw new RuntimeException('Session not found.');
}
}
}
Now we just need to use it.
ProductManager
Let's say in our fictional ProductManager
we have a method publishing all unpublished products. Before introducing our new TransactionalDocumentManager
it might have looked something like this:
// src/Manager/ProductManager.php
namespace App\Manager;
use App\Document\Product;
use Doctrine\ODM\MongoDB\DocumentManager;
/**
* Class ProductManager
*
* @package App\Manager
*/
class ProductManager
{
/**
* @var \Doctrine\ODM\MongoDB\DocumentManager
*/
protected $documentManager;
/**
* @var \Doctrine\Common\Persistence\ObjectRepository
*/
protected $productRepository;
/**
* ProductManager constructor.
*
* @param \Doctrine\ODM\MongoDB\DocumentManager $documentManager
*/
public function __construct(DocumentManager $documentManager)
{
$this->documentManager = $documentManager;
$this->productRepository = $documentManager->getRepository(Product::class);
}
/**
* Publish all unpublished products.
*/
public function publishProducts()
{
/** @var \App\Document\Product[] $products */
$products = $this->productRepository->findBy(['published' => false]);
foreach ($products as $product) {
$product->setPublished(true);
}
$this->documentManager->flush();
}
}
In this scenario each update is atomic, meaning that if we have 5 unpublished products (so 5 documents to update), and the process fails on the 3rd product for whatever reason, we are left with 2 updated products and 3 that are not updated. Not good at all.
Our new TransactionalDocumentManager
to the rescue!
After switching from native DocumentManager
to our new TransactionalDocumentManager
we can add all new shiny transaction-related code, which will make our ProductManager
look something like this:
// src/Manager/ProductManager.php
namespace App\Manager;
use App\Document\Product;
use App\Manager\MongoDB\TransactionalDocumentManager;
/**
* Class ProductManager
*
* @package App\Manager
*/
class ProductManager
{
/**
* @var \App\Manager\MongoDB\TransactionalDocumentManager
*/
protected $documentManager;
/**
* @var \Doctrine\Common\Persistence\ObjectRepository
*/
protected $productRepository;
/**
* ProductManager constructor.
*
* @param \App\Manager\MongoDB\TransactionalDocumentManager $documentManager
*/
public function __construct(TransactionalDocumentManager $documentManager)
{
$this->documentManager = $documentManager;
$this->productRepository = $documentManager->getRepository(Product::class);
}
/**
* Publish all unpublished products.
*/
public function publishProducts()
{
/** @var \App\Document\Product[] $products */
$products = $this->productRepository->findBy(['published' => false]);
$this->documentManager->startTransaction();
foreach ($products as $product) {
$product->setPublished(true);
}
$this->documentManager->flush(null, [
'session' => $this->documentManager->getSession(),
]);
$this->documentManager->commitTransaction();
}
}
In this case if the update fails at any point during the process, the transaction will be automatically aborted and all changes made in the transaction discarded without ever becoming visible.
We might also wrap that publishProducts()
code in a try-catch
block and call $this->documentManager->abortTransaction();
in its catch
section, but if a transaction is not committed, it will be automatically aborted (rolled back), so there is no real need for that here.
Sources: