.NET Learning Notes: Custom In-Memory Provider(2) - In-Memory Database Runtime

CustomMemoryEFProvider Database Runtime GitHub

This section describes the runtime architecture of the in-memory storage layer, independent from EF Core’s translation and compilation pipeline.

MemoryDatabaseRoot (Store Registry / Database Name Isolation)

MemoryDatabaseRoot is the global registry of in-memory stores. It ensures that DbContext instances using the same database name share the same underlying store, while different names result in isolated databases.

It is registered as a singleton and caches MemoryDatabase instances by databaseName, providing name-based store isolation consistent with EF Core’s in-memory behavior.

IMemoryDatabase / MemoryDatabase (Table Registry + Transaction Routing)

MemoryDatabase acts as the coordination layer between entity tables and transactions. It maintains a registry of tables keyed by CLR type, ensuring that each entity type corresponds to a single in-memory table instance.

MemoryDatabase itself does not implement storage logic. Its responsibility is orchestration—table resolution and transaction routing—while the actual data storage behavior is delegated to MemoryTable<T>.

IMemoryTable / MemoryTable — Snapshot Rows and Change Merging

MemoryTable is designed as a snapshot-based storage unit. It does not store entity instances directly. Instead, each row is represented as a SnapshotRow, which contains a primary key and a ScalarSnapshot. This keeps the storage layer independent from tracked entity objects and prevents object graph leakage into the persistence layer.

The reason for introducing SnapshotRow — and exposing IQueryable<SnapshotRow> via QueryRows — is to clearly separate storage representation from entity instances, while allowing EF Core to fully control materialization, tracking, and identity resolution.

Instead of storing entity objects directly, the table stores ScalarSnapshot data. This avoids persisting full object graphs, navigation properties, collections, and circular references. By limiting storage to scalar values only, the provider keeps the storage layer deterministic, lightweight, and easy to clone during transactions. Snapshot-based storage also makes transaction isolation much simpler, since copy-on-write operates on row data rather than entity instances.

SnapshotRow encapsulates both the primary key (Key) and the scalar data (Snapshot). The key provides a stable identity for the row, which is essential for tracking and identity resolution. The snapshot represents the raw column values. This structure mirrors how relational providers conceptually operate: rows are independent data records, and entity instances are created later during materialization.

Returning IQueryable<SnapshotRow> preserves deferred execution and composability. EF Core can continue building expression trees on top of it (Where, Select, Join, Include, etc.), and the provider participates in the normal translation and compilation pipeline. Only during the shaped query compilation phase are rows materialized into entity instances. This ensures the provider does not prematurely instantiate entities or bypass EF Core’s tracking pipeline.

Disabling direct IQueryable<TEntity> access enforces this design. All queries must go through QueryRows, ensuring that entity materialization, identity resolution, and tracking remain under EF Core’s control rather than leaking storage-level objects directly.

In short, SnapshotRow establishes a clean row-level storage contract. It enables clear separation of concerns, simplifies transaction isolation, and maximizes reuse of EF Core’s existing tracking and identity resolution mechanisms.**

Internally, the table maintains two logical states: committed data and pending changes. Committed data represents the stable, persisted state of the table. Pending changes record modifications before SaveChanges is called, and each entry is marked as Added, Modified, or Deleted.

Only scalar properties are included in ScalarSnapshot. Navigation properties, collections, and complex object graphs are intentionally excluded. This avoids circular references, deep cloning problems, and recursive graph persistence. The table remains a flat, value-based storage model, while relationship fix-up and identity resolution are handled later in the EF Core query pipeline rather than inside the storage layer.

In ExtractSnapshot, scalar property values are stored in a dictionary keyed by property name rather than by positional index. This design avoids relying on property ordering and instead preserves a stable name-based mapping. EF Core’s query pipeline does not guarantee that property ordering during materialization will always match CLR reflection order. By storing values keyed by property name, snapshot reconstruction becomes resilient to ordering differences and aligns naturally with EF Core’s metadata-driven property resolution model. In other words, the snapshot format is name-oriented rather than position-oriented, which makes materialization deterministic and decoupled from reflection order assumptions.

Transactions and Isolation Model — MemoryTransaction and Transactional Clone

The transaction model in this provider follows a simplified isolation strategy based on table-level cloning. When a transaction begins, the system creates transactional tables by cloning the current visible state of each base table. The cloned state is not a raw object copy of entity instances, but a snapshot-level copy based on the merged result of committed data and pending changes as exposed by QueryRows. This ensures that the transaction starts from a consistent logical view of the database.

Once a transaction is active, all table access is routed to the transactional tables. Changes are applied only to these transactional copies, leaving the base tables untouched during the transaction scope.

On Commit, the transactional tables replace the corresponding base tables. In practice, this means clearing the base table and re-importing all rows from the transactional table, effectively promoting the transactional state to committed state.

On Rollback, the transactional tables are simply discarded. Because base tables were never modified during the transaction, no additional undo logic is required.

This design represents a single-transaction, table-level copy-on-write model. It does not support nested transactions or fine-grained row versioning. Instead, it prioritizes architectural clarity and predictable isolation behavior suitable for a demonstration provider.

    [DbContext] 

--> [MemoryDatabaseRoot]

--> [MemoryDatabase]

--> [MemoryTable<T>]

--> [SnapshotRow]

--> [ScalarSnapshot]