C# How to ambient transaction work under the hood

In previous article, we looked at how TransactionScope (ambient transaction) simplifies transactional consistency. But how does it do this? How does such a simple and convenient mechanism solve this problem? How exactly is a transaction passed between threads is asynchronous communication? And how does the process of rollback and commit work?
In this article we will try to figure it out with you.

Structure:

1. TransactionScope and Transaction.Current
2. How exactly does ExecutionContext 'know' about ContextKey through AsyncLocal and what happens when await occurs
3. How Transaction is passed to Npgsql
4. How do Commit() and Rollback() work
5. Conclusion

1. TransactionScope and Transaction.Current

sources:

1. transaction scope class - source
2. System.Transaction class - source

The TransactionScope class in .NET is used to automatically manage transactions through the ambient transactions mechanism. You can read more about its application here

Now let's try to figure out what happens when we create a TransactionScope.
Exp:

using var scope = new TransactionScope();

// our repo methods.

scope.Complete();

For understanding, I suggest you open the TransactionScope.cs source code in parallel - source.
NOTE: For easy of reading, find the creation of the public TransactionScope(...) instance.
At the time of writing this is line 53.

Steps:

1. Validate and set AsyncFlow in ValidateAndSetAsyncFlowOption
2. Trying to see if there is a transaction already created or if we have to create a new one in NeedToCreateTransaction (the name might be a bit weird, that's ok (-: )
   1. In the CommonInitialize method using Transaction.GetCurrentTransactionAndScope(...) we try to get the current transaction and scope:
      1. We look at the current transaction in current thread
         1. If there is one we return otherwise null.
   2. Call ValidateAsyncFlowOptionAndESInteropOption
      1. In short, it is forbidden to use AsyncFlow in single-threaded mode.
3. Check scope option, if TransactionScope.Required
   1. We return false if there is a transaction, or true if we need to create one.
4. Creation is done via CommitableTransaction
   1. This is the actual managed transaction object that will be either Commit() or Rollback() in the end. (TransactionScope automatically completes the transaction only if Complete() has been called and there are no exceptions or other processed errors. Otherwise, the transaction is rolled back when leaving the using scope)
5. Make a Clone() transaction for subsequent transfer to Transaction.Current
   1. Why? The Clone() method returns a transaction object without the ability to do Commit(). Isolation and single point control mechanism.
6. Set TransactionScope and specify the transaction as ambient available via Transaction.Current in PushScope()
   1. Here we are interested in the CallContextCurrentData.CreateOrGetCurrentData method, it is operation is as follows:
      1. AsyncLocal<ContextKey> - responsible for 'where we are'.
      2. ConditionalWeakTable<ContextKey, ContextData - stores the actual state of the transaction.
      3. => we add our ContextKey transaction on these objects.
      4. Now ExecutionContext via AsyncLocal 'knows' which ContextKey is active and we can get the state from it.
         1. See p.2 for exactly how this works.
   2. SetCurrent - set the current transaction.

2. How exactly does ExecutionContext 'know' about ContextKey through AsyncLocal and what happens when await occurs

ExecutionContext - This is the container of the logical execution context, which includes:

• AsyncLocal<T> values
• CallContext
• HttpContext
• SecurityContext
• and other 'logical' thread data

Its main task:
Automatically copy and restore all values associated with the current execution context when you switch between threads, tasks, etc.

Where is AsyncLocal in this?

Example from Transaction.cs

private static readonly AsyncLocal<ContextKey?> s_currentTransaction = new AsyncLocal<ContextKey?>();

When we do:

• s_currentTransaction.Value = someContextKey;

Its mean, that:

1. The current ExecutionContext is saved
2. Then, when continuing (on another thread/task), it will be restored
3. And s_currentTransaction.Value will be equal to someContextKey again.

What does AsyncLocal do under the hood?

• AsyncLocal<T> is registered in the ExecutionContext via the .NET infrastructure
• When any await, ThreadPool.QueueUserWorkItem, Task.Run():
  • Copying of ExecutionContext occurs
  • Along with it - all AsyncLocal are copied

3. How Transaction is passed to Npgsql

This is where we have to touch on ADO.NET
ADO.NET is a low-level data access platform in .NET that:

• Allow working with databases (SQL Server, PostgreSQL, Oracle, etc.)
• Supports connection, executing SQL queries, reading results, transaction read Microsoft documentation for more details - here

Npgsql is an ADO.NET provider for PostgreSQL. In this article we will consider it, but the principle of transaction retrieval itself should be almost the same for other providers.

When you work with TransactionScope or Transaction.Current and open SqlConnection, NpgsqlConnection and etc, the following happens:

Inside Npgsql when you call NpgsqlConnection.Open(), Transaction.Current is checked, and it it is not null, the driver itself call EnlistTransaction(...), thus attaching the connection to the transaction.

NOTE: What happens if there is no transaction? - Then your request will work in normal autocommit mode. So be careful when creating TransactionScope to avoid making this situation

Steps:

1. You call async method inside TransactionScope

using var scope = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled))

await _repo.DoSmth();_

scope.Complete();

1. Inside the _repo.DoSmth() method you open a connection to the database
2. When the connection is opened, the binding to the transaction takes place:
   1. NpgsqlConnection source
   2. In the OpenAsync() method, we define var enlistToTransaction = Settings.Enlist ? Transaction.Current : null
   3. Settings.Enlist - answers whether to try to connect to the current transaction or not.
      1. By default this value = true. Documentation - here
   4. => we take Transaction.Current
      1. Where the next chain of calls will be:
         1. Transaction.Current => GetCurrentTransactionAndScope(...) => LookupContextData(...) => TryGetCurrentData(...) => s_currentTransaction.Value

FINISH: Congratulations, we have figured out how a transaction works within TransactionScope.

But that is not all. Let's try to understand what IDBTransaction has to do with it and how exactly Commit and Rollback will work.

4. How do Commit() and Rollback() work

We have worked out that:

• Transaction.Current - is simply a global reference to the current active transaction in the thread or AsyncLocal
• It does not commit or rollback
• It provides you a transaction that you can:
  • EnlistTransaction(tx) is a method that:
    • Remember Transaction.Current as the active transaction
    • Creates a real BEGIN at the PostgreSQL level
    • Waits for a Commit() or Rollback() command from .NET and does not do the COMMIT itself.

Actually Commit() is only called manually by the user via scope.Complete().
If scope.Complete() has not been called, then Rollback() will be executed when scope.Dispose() is called automatically or manually.

What does IDbTransaction have to do with this?

In brief, it has nothing to do with:

• IDbTransaction is an ADO.NET interface for manual managing
• It allows you to manually manage a transaction
• has Commit() and Rollback() like TransactionScope()

Despite this TransactionScope and IDBTransaction do not overlap directly, only relatively.

How then does TransactionScope work without IDBTransaction?

When you use TransactionScope with NpgsqlConnection Npgsql registers a special VilatileResourceManager object in Transaction.Current via the

transaction.EnlistVolatile(volatileResourceManager, EnlistmentOptions.None);

This VilatileResourceManager implements:

• ISinglePhaseNotification (for an one-phase commit)
• IEnlistmentNotification (because ISinglePhaseNotification inherits it) (for a two-phase commit)

When we call scope.Complete() = just sets the flag _complete = true
scope.Dispose() => call Commit() from CommitableTransaction

Where resource.SinglePhaseCommit(enlistment) will subsequently be called.
On the Npgsql side, the SinglePhaseCommit implementation will call _connector.ExecuteInternal(“Commit”) where _connector = NpgsqlConnector;

The NpgsqConnector itself does the following:

• Formats the SQL-query into binary or text PostgreSQL protocol
• Sends the command via TCP (via Socket, Stream)
• Processes the response (e.g., CommandComplete)
• Updates the internal transaction state in connector.TrasactionStatus

NOTE: This article only considers the single-phase commit scenario, where only one resource in invilved in a transaction.

Conclusion

As a result, we have seen how exactly TransactionScope works under the hood and what makes it possible to write such simple and elegant code.

Since this article serves as a companion to C# Ambient Transactions: What They Are and Why They Matter
I’ve added transaction logging at each step in the demo project, which you can see via the console. The code is here

console log exp.

[Endpoint] Start: Thread: 7
[Endpoint] Start: Transaction.Current:
 => [TxFactory] Creating TransactionScope
 => [TxFactory] Thread: 7
 => [TxFactory] TransactionScope created. Transaction.Current ID: c37ac104-0513-4445-b2cd-ae4687fb5598:1
 => => [DB] UpdateBalanca: UserId: 1. Before OpenAsync. Thread: 7
 => => [DB] UpdateBalanca: UserId: 1. Transaction.Current: c37ac104-0513-4445-b2cd-ae4687fb5598:1
 => => [DB] UpdateBalanca: UserId: UserId: 1. After OpenAsync. Still in Transaction: c37ac104-0513-4445-b2cd-ae4687fb5598:1
 => => [DB] UpdateBalanca: UserId: 2. Before OpenAsync. Thread: 7
 => => [DB] UpdateBalanca: UserId: 2. Transaction.Current: c37ac104-0513-4445-b2cd-ae4687fb5598:1
 => => [DB] UpdateBalanca: UserId: UserId: 2. After OpenAsync. Still in Transaction: c37ac104-0513-4445-b2cd-ae4687fb5598:1
 => => [DB] AddLog: UserFrom: 1 -> UserTo: 2. Before OpenAsync. Thread: 7
 => => [DB] AddLog: UserFrom: 1 -> UserTo: 2. Transaction.Current: c37ac104-0513-4445-b2cd-ae4687fb5598:1
 => => [DB] AddLog: UserFrom: 1 -> UserTo: 2. After OpenAsync. Still in Transaction: c37ac104-0513-4445-b2cd-ae4687fb5598:1
[Endpoint] Stop: After [TxFactory].Dispose: Thread: 7
[Endpoint] Stop: After [TxFactory].Dispose: Transaction.Current:

Thank you for your attention.