DEV Community: Alex

.NET Learning Notes:Deploying a Microservices Application to VPS with Docker, Nginx, and CD

Alex — Thu, 19 Mar 2026 04:09:22 +0000

1.Environment Setup(VPS Preparation)

Before deploying the application, I prepared a clean Ubuntu-based VPS environment.
Connect to VPS

ssh root@your_vps_ip

System Update

apt update && apt upgrade -y

During the upgrade process:

Kept the existing SSH configuration
Restarted services using default options This ensures system packages are up to date without breaking remote access.

Install Docker
To containerize the microservices, Docker and Docker Compose were installed:

apt install -y ca-certificates curl gnupg

install -m 0755 -d /etc/apt/keyrings

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg

chmod a+r /etc/apt/keyrings/docker.gpg

echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo $VERSION_CODENAME) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null

apt update

apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Verify Installation

docker run hello-world

This confirms Docker is correctly installed and functioning.

2. Application Deployment & Infrastructure Setup

After preparing the VPS environment, I deployed the application using Docker and configured domain routing, reverse proxy, and HTTPS.
Deploy Application with Docker Compose
The project is structured as a microservices system and deployed via Docker Compose:

git clone <your-repo>
cd <repo>
docker compose up -d --build
docker ps

This starts all services.

Handling Database Initialization Issue
During the first deployment, a common issue occurred:

The application attempted to run database migrations before MySQL was fully ready.

This caused startup failures. To address this, I enabled retry logic in the database configuration, allowing the service to wait and retry until the database becomes available.

Cloudflare Proxy & SSL Configuration
A custom domain was purchased and configured via Cloudflare.
DNS records:

chenlis.com → frontend
admin.chenlis.com → admin panel
api.chenlis.com → API gateway

Cloudflare proxy (orange cloud) was enabled to introduce an additional layer between users and the origin server:

User → Cloudflare → VPS

This provides several important benefits:

CDN acceleration: Static assets can be cached at Cloudflare edge nodes, reducing latency and improving load times for users in different regions. It also reduces direct traffic to the VPS.
Basic DDoS protection: Incoming traffic is filtered by Cloudflare before reaching the server. This helps mitigate simple flooding or abnormal request patterns that could otherwise overwhelm a small VPS.
Hiding the origin server IP:The public IP of the VPS is no longer exposed. Requests are routed through Cloudflare, making it harder for attackers to directly target the server.
TLS optimization: Cloudflare handles TLS negotiation at the edge, improving connection performance and reducing cryptographic overhead on the origin server.

SSL Mode: Full (Strict)
The SSL/TLS mode was configured to Full (Strict).
This ensures:

User → HTTPS → Cloudflare → HTTPS → Origin Server

with certificate validation on both sides.

In Full mode, Cloudflare encrypts traffic to the server but does not validate the certificate.
In Full (Strict) mode, Cloudflare requires a valid certificate (e.g., Let’s Encrypt), ensuring both encryption and trust. Since a valid certificate was already issued via Certbot, switching to Full (Strict) guarantees end-to-end secure communication.

Reverse Proxy with Nginx
Nginx was installed and configured as a reverse proxy:

sudo apt update
sudo apt install nginx -y

sudo systemctl start nginx
sudo systemctl enable nginx

Configuration file:

server {
    listen 80;
    server_name chenlis.com;

    location / {
        proxy_pass http://localhost:3001;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

server {
    listen 80;
    server_name admin.chenlis.com;

    location / {
        proxy_pass http://localhost:3000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

server {
    listen 80;
    server_name api.chenlis.com;

    location / {
        proxy_pass http://localhost:5193;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

Enable the configuration:

sudo ln -s /etc/nginx/sites-available/fintrack /etc/nginx/sites-enabled/
sudo rm /etc/nginx/sites-enabled/default
sudo nginx -t
sudo systemctl reload nginx

Enable HTTPS with Certbot
To secure all endpoints, HTTPS was configured using Let’s Encrypt:

sudo apt update
sudo apt install certbot python3-certbot-nginx -y

sudo certbot --nginx \
  -d chenlis.com \
  -d admin.chenlis.com \
  -d api.chenlis.com

This automatically:

Issues SSL certificates
Configures Nginx
Enables auto-renewal

3. CI/CD Pipeline with GitHub Actions

To automate deployment, I implemented a simple CI/CD pipeline using GitHub Actions.
This allows the application to be deployed automatically to the VPS whenever changes are merged into the main branch.

Deployment Strategy
The workflow follows a straightforward approach:

Code is developed in feature branches
Changes are merged into main via pull requests
Deployment is triggered only on push to main This avoids deploying unverified code and keeps the production environment stable.

GitHub Actions Workflow
The deployment is handled via SSH using a GitHub Actions workflow:

name: Deploy to VPS

on:
  push:
    branches:
      - main

jobs:
  deploy:
    runs-on: ubuntu-latest

    steps:
      - name: Deploy via SSH
        uses: appleboy/ssh-action@v1.0.3
        with:
          host: ${{ secrets.VPS_HOST }}
          username: ${{ secrets.VPS_USERNAME }}
          key: ${{ secrets.VPS_SSH_KEY }}
          port: ${{ secrets.VPS_PORT }}
          script: |
            set -e
            cd ~/FinTrack-Microservices
            git pull
            docker compose up -d --build
            docker image prune -f

Why SSH-Based Deployment
Instead of using a full CI/CD platform, I chose a lightweight SSH-based approach:

Simple to set up
No additional infrastructure required
Full control over deployment commands This is sufficient for small-to-medium scale systems and personal projects.

Deployment Flow
Once configured, the deployment flow becomes:

 git push → GitHub Actions → SSH to VPS → git pull → docker rebuild → services updated

This eliminates the need for manual deployment and ensures consistency between local and production environments.

.NET Learning Notes: Configure docker for project

Alex — Tue, 10 Mar 2026 02:45:19 +0000

Project Background

The project discussed in this article is available here: FinTrack

FinTrack is a microservices-based backend system. The services are separated into multiple independent components and communicate through APIs and message queues.

During early development, everything was started manually. Each service needed to run in its own terminal, and several external services were also required, such as databases and message brokers.

This quickly became inconvenient.

For example, starting the system locally often required opening multiple terminals and running different commands for each service. In addition, the project depended on several infrastructure components. Managing and starting all of them manually was tedious and error-prone.

Another problem appeared when demonstrating the project. Launching each service one by one was not an ideal workflow.

This is where Docker becomes useful.

By containerizing the services and their dependencies, the entire system can be started in a consistent way. With tools such as Docker Compose, multiple services can be launched together with a single command.

For a microservices project, this greatly simplifies both local development and project demonstration.

Common Docker Compose Commands

In daily development, only a few Docker Compose commands are used frequently. Most of the time, managing containers revolves around starting, stopping, and rebuilding services.

docker compose up The most common command is:

docker compose up

This starts the services defined in the docker-compose.yml file.
Two options are commonly used:

docker compose up -d --build

-d runs the containers in detached mode (in the background).
--build forces Docker to rebuild images before starting the containers.

This is useful when the source code or Dockerfile has changed.

docker compose down

docker compose down

This stops and removes the containers, networks, and default resources created by the Compose project.

Sometimes volumes are also removed:

docker compose down -v

The -v option removes the volumes associated with the containers.

docker compose start

docker compose start

This command starts containers that already exist but are currently stopped.

It does not rebuild images or recreate containers.

docker compose stop

docker compose stop

This stops the running containers but does not remove them. The containers can later be restarted using docker compose start.

docker compose -f

Sometimes multiple compose files are used. In that case, the -f option allows specifying the compose file manually.

Example:

docker compose -f docker-compose.dev.yml up -d

This tells Docker Compose to use a specific configuration file instead of the default docker-compose.yml.

Dockerfile Example

Below is the Dockerfile used to build the IdentityService.

# ---------- build stage ----------
# Use the .NET SDK image as the build environment
FROM mcr.microsoft.com/dotnet/sdk:9.0 AS build

# Set the working directory inside the container to /src
# After this line, all following commands (COPY, RUN, etc.)
# will use /src as their base directory
WORKDIR /src

# Copy the solution file into the container build environment
# The destination "./" means the current WORKDIR, which is /src
COPY FinTrack.sln ./

# Copy project files (csproj) into the container
# These paths are relative to the current WORKDIR (/src)
COPY src/Services/IdentityService/IdentityService.csproj src/Services/IdentityService/
COPY src/Shared/SharedKernel/SharedKernel.csproj src/Shared/SharedKernel/

# Restore NuGet dependencies for the project
RUN dotnet restore src/Services/IdentityService/IdentityService.csproj

# Copy the full source code into the container
# "." means the build context root on the host
# "." as the destination means the current WORKDIR (/src)
COPY . .

# Publish the application
# The compiled output will be placed in /app/publish inside the container
RUN dotnet publish src/Services/IdentityService/IdentityService.csproj \
    -c Release \
    -o /app/publish \
    /p:UseAppHost=false


# ---------- runtime stage ----------
# Use a lighter ASP.NET runtime image for running the application
FROM mcr.microsoft.com/dotnet/aspnet:9.0

# Set the runtime working directory inside the container
# The application will run from /app
WORKDIR /app

# Copy the published output from the build stage
# "/app/publish" refers to the path inside the build stage container
# "." means the current WORKDIR (/app) in this runtime stage
COPY --from=build /app/publish .

# Configure ASP.NET Core to listen on port 8080 inside the container
ENV ASPNETCORE_URLS=http://+:8080

# Document that the container exposes port 8080
EXPOSE 8080

# When the container starts, run the service
ENTRYPOINT ["dotnet", "IdentityService.dll"]

Docker Compose File example

version: '3.9'
services:

  # ========================
  # Infrastructure services
  # These are shared components used by backend services
  # ========================

  mysql:
    image: mysql:8.0
    container_name: fintrack-mysql
    environment:
      MYSQL_ROOT_PASSWORD: 123456
      MYSQL_DATABASE: fintrack_identity
    ports:
      - "3307:3306" # host:container
    volumes:
      - mysql_data:/var/lib/mysql # persist database data outside container
    healthcheck:
      # wait until MySQL is ready before dependent services start
      test: ["CMD","mysqladmin","ping","-h","localhost","-p123456"]
      interval: 10s
      timeout: 5s
      retries: 10


  # ========================
  # Backend microservices
  # Each service is built from its own Dockerfile
  # ========================

  notification-service:
    build:
      context: .
      dockerfile: src/Services/NotificationService/Dockerfile
    container_name: fintrack-notification
    depends_on:
      rabbitmq:
        condition: service_healthy
      mailhog:
        condition: service_started
    environment:
      ASPNETCORE_ENVIRONMENT: Development
      ASPNETCORE_URLS: http://+:8080
      RabbitMQ__Host: rabbitmq
      SMTP__Host: mailhog
      SMTP__Port: 1025
    ports:
      - "5103:8080"


  # ========================
  # API Gateway
  # Acts as the entry point for backend APIs
  # ========================

  gateway:
    build:
      context: .
      dockerfile: src/Services/GatewayService/Dockerfile
    container_name: fintrack-gateway
    depends_on:
      identity-service:
        condition: service_started
      transaction-service:
        condition: service_started
      auditlog-service:
        condition: service_started
      notification-service:
        condition: service_started
      redis:
        condition: service_started
    environment:
      ASPNETCORE_ENVIRONMENT: Development
      ASPNETCORE_URLS: http://+:8080

      # Redis used for caching
      ConnectionStrings__Redis: redis:6379

      # Reverse proxy routing configuration
      ReverseProxy__Clusters__identityCluster__Destinations__identity__Address: http://identity-service:8080/
      ReverseProxy__Clusters__transactionCluster__Destinations__transaction__Address: http://transaction-service:8080/
      ReverseProxy__Clusters__auditlogCluster__Destinations__destination1__Address: http://auditlog-service:8080/
    ports:
      - "5193:8080"


  # ========================
  # Frontend applications
  # ========================

  web-portal:
    build:
      context: ./apps/web-portal
      dockerfile: Dockerfile
    container_name: fintrack-web-portal
    depends_on:
      gateway:
        condition: service_started
    ports:
      - "3000:80"

# ========================
# Named volumes
# Used to persist database data
# ========================

volumes:
  mysql_data:

Some Practical Notes

These concepts are simple, but understanding their boundaries helps avoid many common Docker misunderstandings.

When using Docker in a microservice project, it is helpful to clearly understand the scope of several concepts: images, containers, and networks.

Image vs Container

An image is a packaged environment that contains the application and everything it needs to run.

A container is a running instance of that image.

In other words:

image → blueprint

container → running process

Multiple containers can be created from the same image.

Container Network

When using Docker Compose, all services are placed on the same internal Docker network.

Inside this network, containers can communicate using service names as hostnames.

For example:

identity-service can reach MySQL using:

mysql:3306

This works because Docker Compose automatically creates an internal network and provides DNS resolution between services.

Container vs Host

Containers run in an isolated environment.

Services inside containers are not automatically accessible from the host machine unless a port is exposed.

For example:

ports:

"5100:8080"

This means:

host:5100 → container:8080

After this mapping, the service can be accessed from the host machine using:

http://localhost:5100

Container vs Server

In development, containers run on the local machine.

In production, the same images can run on a remote server or cloud environment.

Since the application and its dependencies are packaged inside the image, the runtime environment remains consistent.

.NET Learning Notes: DelegatingHandler and Harmony (HTTPMataki logging library)

Alex — Tue, 03 Mar 2026 09:45:33 +0000

Exploring Runtime HTTP Logging in .NET

Recently, I experimented with an automatic HTTP request logging library:HttpMataki.NET.

This library can capture HTTP request and response information without modifying the original application code, which makes it especially useful during development and debugging. Being able to inspect request and response details transparently is extremely convenient when diagnosing issues or researching process.

While reviewing its implementation, I noticed that it relies on two interesting .NET runtime techniques. I took this opportunity to explore and better understand how these mechanisms work under the hood.

DelegatingHandler

Before diving into more advanced runtime techniques, the first concept worth clarifying is DelegatingHandler.

A natural question comes up:

If we can log requests using middleware, why do we need DelegatingHandler at all?

To answer that, we need to distinguish between inbound requests and outbound requests in a .NET web application.

Inbound Requests: Handled by Kestrel

When an application receives an HTTP request:
Client → Kestrel → Middleware Pipeline → Controller

The request is processed by the web server (typically Kestrel) and flows through the middleware pipeline.

This is where we commonly implement:

Logging
Authentication
Exception handling
Metrics collection

Middleware operates on incoming requests.

Outbound Requests: Handled by HttpClient

However, when our application calls an external service:
Our App → HttpClient → External API

This is a completely different pipeline.

Middleware does not participate in outbound HTTP calls.

Instead, outbound requests are handled by HttpClient, and this is where DelegatingHandler comes in.

What is DelegatingHandler?

DelegatingHandler acts as a message handler in the HttpClient pipeline. It forms a responsibility chain around outbound HTTP requests.

You can think of it as:

A wrapper around outgoing requests
A way to intercept and modify requests
A way to inspect responses

It allows you to log, modify, or short-circuit outbound HTTP traffic.

This is how the logging library is able to inspect:

Requests sent via HttpClient
Responses received from external services

Without modifying application business logic.

For detailed implementation, you can refer to the official documentation, but conceptually it behaves like a chain-of-responsibility pattern for outbound HTTP traffic.

Simple Demonstration

I created a minimal example to demonstrate where middleware and DelegatingHandler sit in the request lifecycle:

These examples clearly show the difference in execution position between inbound middleware processing and outbound HttpClient interception.

Harmony: Runtime Method Interception in .NET

The second technique I explored is Harmony, a runtime method patching library.

Harmony allows developers to modify the behavior of existing methods at runtime without changing the original source code. It works by intercepting method execution and injecting custom logic before, after, or even in place of the original implementation.

However, it is important to understand its scope and limitations.

Harmony can only operate within the same process. It does not:

Inject code into other processes
Modify unmanaged/native applications
Change type structure (such as adding fields or extending enums)

In other words, Harmony modifies behavior — not structure.

A Simple Example

To better understand how it works, I created a minimal example demonstrating:example

Overall, Harmony is quite straightforward to use once you understand its parameter binding conventions and runtime patching model.

In this case, I encountered Harmony while reviewing the implementation of the Mataki library and took the opportunity to explore how it works under the hood.

While it is not a tool I would reach for in everyday application development, understanding its mechanics provides valuable insight into how runtime interception works in .NET.

.NET Learning Notes: Custom In-Memory Provider(5) - Include & ThenInclude — Navigation Loading and Fix-Up

Alex — Thu, 19 Feb 2026 02:51:35 +0000

In EF Core, Include and ThenInclude are not just “query operators that add extra columns.” They are a loading contract: “when the outer entities are materialized, also load the related entities and make the relationship consistent in the tracked graph.” Relational providers often implement this by producing SQL that happens to look like LEFT JOIN, but the real goal is not the join itself. The real goal is: load related rows and run relationship fix-up so that navigation properties on both sides point to the correct tracked instances.

This is why Include feels deceptively simple at the API level but becomes difficult at provider level. It sits right at the boundary between query execution, entity materialization, tracking/identity resolution, and navigation fix-up. If you get any of those wrong, you either populate navigations with duplicate instances, break EF Core’s identity map guarantees, or end up with “loaded but not marked loaded” behaviors that cause EF Core to issue unexpected additional loads later.

Reference Include vs Collection Include vs ThenInclude

A reference include populates a single navigation property such as Order.Customer. Conceptually it is “1:0..1” or “many-to-one”: each outer entity points to at most one related entity. The fix-up is straightforward: once both entities are materialized and tracked, the outer reference can be assigned, and the inverse navigation (if present) can also be set.

A collection include populates a collection navigation such as Customer.Orders. This is “1:N”. The key difference is that collection include is not just “attach one inner row.” It is “attach potentially many related rows,” and that immediately introduces grouping semantics. A join-shaped result naturally duplicates outer rows; EF Core must prevent that duplication from turning into duplicated outer instances and duplicated navigation items. This is why collection includes are harder: the provider must either group inner rows per outer key or build a correlated subquery per outer row, and then apply fix-up consistently.

ThenInclude is not a third separate mechanic; it is simply “Include on an entity that is itself included.” In a reference chain, it looks like Blog -> Owner -> Address. In a collection chain, it looks like Blog -> Posts -> Comments. The difficulty is that it composes include paths and requires the provider to apply fix-up in the correct sequence across multiple relationship edges, while still respecting identity resolution.

How EF Core Represents Include Internally

At the surface level, Include looks simple. You write:

context.Customers.Include(c => c.Orders)

and expect the related entities to be loaded. But internally, EF Core does not treat Include as a normal LINQ operator like Where or Select. So internally, Include lives inside the shaping layer, not the filtering layer.

reference navigation

For a reference navigation such as:

context.Orders.Include(o => o.Customer)

EF Core translates the include into a LEFT JOIN during the translation phase. Internally, the query is rewritten so that the related entity is brought into scope through a join operation. The translation visitor typically calls into TranslateLeftJoin, producing a join expression that temporarily carries both the outer and inner elements.

Conceptually, the transformation looks like this:

Orders
    .LeftJoin(
        Customers,
        o => o.CustomerId,
        c => c.Id,
        (o, c) => new TransparentIdentifier<Order, Customer>(o, c)
    )

The TransparentIdentifier<Outer, Inner> is an internal carrier that keeps both sides of the join available for the next projection step.

After the join, EF Core applies a Select projection that restores the final result shape to the outer entity type. At this stage, the query element type becomes Order again, not a pair of (Order, Customer). The join is used only to retrieve the related row.

The important part is how Include itself is represented. It is not treated as a normal executable LINQ operator. Instead, EF Core inserts an IncludeExpression (or equivalent marker) into the ShaperExpression, not the main query expression. The shaper is responsible for materialization. During execution, the shaper uses the joined inner entity to populate the navigation:

order.Customer = customer;

If tracking is enabled, EF Core’s identity resolution and relationship fix-up mechanisms are triggered automatically.

This is the essential pattern for reference include: LEFT JOIN to fetch the related entity, projection back to the outer type, and navigation assignment handled during shaping and tracking.

collection navigation & ThenInclude

In this provider, collection navigation and ThenInclude are implemented using the same underlying rewriting mechanism. Instead of relying on EF Core’s relational join-based pipeline, the provider intercepts IncludeExpression during the compilation stage and replaces it with executable logic that performs a correlated subquery and explicit navigation fix-up.

For collection navigation (for example, Blog.Include(b => b.Posts)), EF Core represents the include inside the shaper as an IncludeExpression. When the navigation is a collection, the NavigationExpression is typically a MaterializeCollectionNavigationExpression, which contains a subquery describing how to load the related entities. This subquery is not directly executable and cannot be left in the final expression tree. It must be transformed.

During compilation, the provider scans Queryable.Select calls and detects whether the selector body contains an IncludeExpression. If found, it rewrites the selector. For collection includes, the subquery stored in MaterializeCollectionNavigationExpression. Subquery is first rewritten into a provider-executable query expression. This step ensures that any ShapedQueryExpression, query roots, or provider-specific expressions are converted into an expression that returns an IEnumerable<TElement>.

Instead of embedding the subquery expression directly into the final tree, the provider compiles it into a delegate of type Func<QueryContext, TOuter, IEnumerable<TElement>>. At runtime, this delegate is invoked to execute the correlated subquery for each outer entity instance. The result is materialized as a list.

The original IncludeExpression is then replaced with a call to a helper method such as LoadCollection. This method receives the current entity instance, the navigation metadata, and the materialized related entities. It retrieves or creates the navigation collection on the entity, clears any existing contents, adds the loaded elements, and optionally sets IsLoaded = true on the navigation entry. In this way, the navigation is populated explicitly during materialization.

ThenInclude does not require a separate mechanism. It works naturally through recursion. When an include subquery itself contains further includes, the same rewriting logic is applied to the nested query during compilation. Because subqueries are rewritten into executable delegates before being invoked, any nested IncludeExpression encountered inside them is also transformed. In effect, ThenInclude is just another include applied within a deeper subquery scope, and the same rewrite-and-fix-up process is reused.

This approach avoids join-based row duplication and does not depend on TransparentIdentifier shapes. Instead of flattening data through joins and reconstructing object graphs from duplicated rows, the provider executes a correlated subquery per outer entity and performs fix-up directly on entity instances. The final query shape remains a flat sequence of outer entities, with collection navigations populated after materialization.

The main limitation of this strategy is that it relies on LINQ-to-Objects execution of subqueries and explicit fix-up logic, rather than fully integrating with EF Core’s internal include pipeline. To implement include in a fully “native” EF Core way, the provider would need to participate in navigation expansion, entity key resolution factories, include metadata binding, and shaper-based fix-up integration. That would require significantly more work and much deeper alignment with EF Core’s internal query pipeline.

Why Include/ThenInclude Is Difficult for a Custom Provider

The core difficulty is that Include spans multiple EF Core subsystems at once.

First, it depends on correct identity resolution. If the included inner entity is materialized as a new object each time it appears, the graph becomes inconsistent immediately.

Second, it depends on navigation metadata and FK matching. For reference include, you need to locate the inner entity by foreign key from the outer entity. For collection include, you need to locate many inner entities by matching their FK to the outer PK. EF Core has metadata for this (keys, foreign keys, navigations, inverse navigations), but providers must wire enough services so that EF can ask for entity finders and key factories consistently.

Finally, ThenInclude multiplies the complexity because the provider must apply include logic repeatedly across the included entities, not only the root query entity, while keeping graph identity stable.

Why LINQ-to-Objects Was the Practical Choice

In your current approach, you treat Include as “metadata in the shaper,” not as something you execute directly. The core pipeline still compiles the main query (QueryRows → TrackFromRow → replay steps → apply terminal). For Include, you extract the navigation names from the shaper before stripping the Include markers, then you run fix-up outside the formal translation/compilation of Include itself.

That is exactly what this part of your code implies:

var includeNavs = ExtractIncludeNavigationNames(shapedQueryExpression.ShaperExpression);
executor = CompileQueryRowsPipeline(q);
// executor = new IncludeStrippingVisitor().Visit(executor)!;
executor = ApplyMarkLoadedWrapper(executor, QueryCompilationContext.QueryContextParameter, includeNavArrayExpr);

The sequence matters. You extract include information while it still exists in the shaper. Then you compile your own executable pipeline that does not rely on EF Core’s IncludeExpression being executable. Then you apply a wrapper to ensure EF Core sees navigations as loaded (and/or you apply your own fix-up step).

Chosing LINQ-to-Objects because it’s the shortest path to a correct demonstration: you can run the main query normally, materialize and track root entities correctly, then for each root entity perform additional in-memory lookups against your IMemoryDatabase tables to populate navigations. This avoids having to fully model EF Core’s internal include shaping contracts while still producing correct final graphs for a demo provider.

What “Formal EF Core Pipeline Include” Would Require

If we wanted to implement Include using EF Core’s formal pipeline rather than rewriting it into LINQ-to-Objects logic, the provider would need to participate much deeper in EF Core’s shaping and materialization infrastructure.

First, the provider would need to fully support EF Core’s IncludeExpression inside the shaper expression. That means not stripping or rewriting it, but allowing it to flow into the shaped query compilation stage. The provider’s IShapedQueryCompilingExpressionVisitor would then be responsible for generating materialization code that cooperates with EF Core’s navigation fix-up system rather than performing manual assignment.

Second, reference include would require proper join translation support. The translation visitor would need to implement TranslateLeftJoin (and possibly other join patterns) so that EF Core can express reference navigations as join-based query shapes. The provider must then handle TransparentIdentifier-style intermediate projections correctly and ensure that both outer and inner entities are materialized in a coordinated way.

Third, collection include would require correlated subquery or grouping support at the query model level. Instead of executing a compiled delegate per outer entity, the provider would need to preserve EF Core’s subquery representation and let the shaped query compiler build the correct collection materialization pipeline. This involves supporting MaterializeCollectionNavigationExpression and ensuring that collection shapers cooperate with tracking and fix-up.

Fourth, the provider would need to integrate fully with EF Core’s tracking hooks. That includes correct usage of key factories, entity finder services, navigation metadata, and tracking behavior flags. Fix-up must occur through the StateManager automatically during materialization rather than through explicit property assignment.

Finally, the provider would need to guarantee that the final expression tree produced by compilation contains only reducible, executable nodes. IncludeExpression and other internal markers must be handled in the shaper phase, not left in the executable pipeline.

In short, a formal pipeline implementation would require full join translation support, collection shaping support, proper handling of EF Core’s internal include expressions, and tight integration with the StateManager’s fix-up system. It is significantly more complex than a LINQ-to-Objects rewriting approach because it requires the provider to implement EF Core’s relational-style shaping semantics rather than delegating navigation loading to post-processing logic.

.NET Learning Notes: Custom In-Memory Provider(4) - ReadPath - From IQueryable to Result Execution

Alex — Thu, 19 Feb 2026 00:56:46 +0000

The read path in this provider follows EF Core’s standard query pipeline: expression construction, translation, compilation, and execution. The provider does not implement individual LINQ operators as standalone execution logic. Instead, it integrates into EF Core’s existing pipeline and participates at the appropriate extension points.

When a query starts, no execution occurs. Methods such as Where, Select, OrderBy, ThenBy, Skip, and Take are non-terminal operations. They only build or transform expression trees. Each call composes additional structure onto the existing query expression. At this stage, nothing is executed and no entities are materialized.

Translation

The provider integrates during the translation phase through IQueryableMethodTranslatingExpressionVisitor. EF Core parses the LINQ method calls and invokes this visitor to produce a ShapedQueryExpression. In this implementation, the provider does not translate LINQ into a custom DSL or separate query representation. Instead, it constructs a ShapedQueryExpression that carries two things: the underlying row-level query expression and the associated shaper expression. The structure of the query records each translated function's parameters.

It is important to understand that translation is not initiated by the provider. EF Core drives the entire process. LINQ method calls are first parsed into method-call expressions, and EF Core then invokes the appropriate TranslateXXX methods on the provider’s visitor. The provider does not “scan” or interpret LINQ directly — it simply responds to EF Core’s translation callbacks and records the query semantics accordingly.

QueryRoot and ShapedQuery

A QueryRoot represents the starting point of a query. When application code writes:

context.Set<TestEntity>()

EF Core does not interpret this as “give me a collection of TestEntity”. Instead, it treats it as a declarative query origin: “This query starts from the entity type TestEntity.” Internally, EF Core represents this as a query root expression associated with the entity type. At this stage: No data is accessed, No enumeration occurs, No filtering or projection happens. A QueryRoot is purely symbolic. It describes what is being queried, not how to retrieve it. Every EF Core query—no matter how simple—must begin with a QueryRoot.

A Shaped Query is the executable form of a query. It represents the result of translating a QueryRoot (and any LINQ operators) into something EF Core can actually run. A shaped query combines two essential components:

Query Expression: Describes how to retrieve raw data (rows, objects, records) from the data source.
Shaper Expression: Describes how to transform each retrieved item into the final result type returned to the user. Together, they define: “How the query executes” and “What shape the results take.”

This is why EF Core uses the term “shaped”—the query not only fetches data, but also defines how that data is molded into entities, projections, or scalar values.

/// <summary>
/// Entry point from QueryRoot to ShapedQuery.
/// For a full table scan:
/// QueryExpression = CustomMemoryQueryExpression(entityType)
/// ShaperExpression = CustomMemoryEntityShaperExpression(entityType)
/// </summary>
protected override ShapedQueryExpression? CreateShapedQueryExpression(IEntityType entityType)  
{  
    if (entityType == null) throw new ArgumentNullException(nameof(entityType));  
    var queryExpression = new CustomMemoryQueryExpression(entityType);  
    var shaperExpression = new CustomMemoryEntityShaperExpression(entityType);  
    return new ShapedQueryExpression(queryExpression, shaperExpression);  
}

Recording Query Semantics

In this provider, the translation stage is intentionally “record-only”. EF Core parses a LINQ query into method-call expressions and invokes CustomMemoryQueryableMethodTranslatingExpressionVisitor to translate each operator, but the provider does not execute anything at this point. Instead, it stores the translated intent inside CustomMemoryQueryExpression, and passes that forward to the compilation stage.

CustomMemoryQueryExpression acts as a lightweight query model owned by the provider. It does not try to become a DSL, and it does not attempt to replace EF Core’s pipeline. Its job is simpler: capture the entity type being queried and record every non-terminal operator in the exact order it appears in the original LINQ chain. The ordering requirement is the key reason this type exists. Operators like OrderBy, Skip, and Take are not commutative, and their meaning depends on sequencing. OrderBy().Skip(10).Take(5) is a different query from Skip(10).Take(5).OrderBy(). If the provider only stored “there is a skip and a take somewhere” without preserving their positions, it would not be able to reproduce correct behavior during execution.

This is why CustomMemoryQueryExpression keeps Steps as an ordered list rather than a set of flags. Each translation method appends a new step and returns a new CustomMemoryQueryExpression instance, so the expression remains immutable and composable while still preserving the original operator sequence.

For example, when EF Core encounters a Where(...) call, visitor handles it by reading the existing CustomMemoryQueryExpression from the shaped query, appending a WhereStep(predicate), and returning a new ShapedQueryExpression that carries the updated query expression:

protected override ShapedQueryExpression? TranslateWhere(
    ShapedQueryExpression source, 
    LambdaExpression predicate)
{
    if (source.QueryExpression is not CustomMemoryQueryExpression q)
    {
        return null;
    }

    var q2 = q.AddStep(new WhereStep(predicate));

    return new ShapedQueryExpression(q2, source.ShaperExpression);
}

A concrete example makes the “record in order” design easier to see. Suppose the user writes:

var query =
    context.Set<Product>()
        .Where(p => p.Price > 1000)
        .OrderBy(p => p.Id)
        .Skip(10)
        .Take(5);

protected override ShapedQueryExpression? TranslateWhere(ShapedQueryExpression source, LambdaExpression predicate)  
{  
    if (source.QueryExpression is not CustomMemoryQueryExpression q)  
    {        return null;  
    }  
    var q2 = q.AddStep(new WhereStep(predicate));  

    return new ShapedQueryExpression(q2, source.ShaperExpression);  
}

protected override ShapedQueryExpression? TranslateOrderBy(ShapedQueryExpression source,  
    LambdaExpression keySelector, bool ascending)  
{  
    if (source.QueryExpression is not CustomMemoryQueryExpression q)  
        return null;  
    var kind = ascending ? CustomMemoryQueryStepKind.OrderBy : CustomMemoryQueryStepKind.OrderByDescending;  
    // OrderBy: descending = !ascending, thenBy = false  
    var q2 = q.AddStep(new OrderStep(kind, keySelector));  

    return new ShapedQueryExpression(q2, source.ShaperExpression);  
}

protected override ShapedQueryExpression? TranslateSkip(ShapedQueryExpression source, Expression count)  
{  
    if (source.QueryExpression is not CustomMemoryQueryExpression q)  
        return null;  

    // record skip  
    var q2 = q.AddStep(new SkipStep(count));  

    return new ShapedQueryExpression(q2, source.ShaperExpression);  
}

protected override ShapedQueryExpression? TranslateTake(ShapedQueryExpression source, Expression count)  
{  
    if (source.QueryExpression is not CustomMemoryQueryExpression q)  
        return null;  

    var q2 = q.AddStep(new TakeStep(count));  

    return new ShapedQueryExpression(q2, source.ShaperExpression);  
}

During translation, this does not run a scan, does not apply filters, and does not materialize entities. Instead, the visitor transforms the query into a single CustomMemoryQueryExpression whose Steps sequence matches the LINQ chain: first a WhereStep(Price > 1000), then an OrderByStep(Id), then a SkipStep(10), then a TakeStep(5). That ordered list is the only thing your compilation stage needs in order to replay the semantics correctly over the underlying row source. In other words, translation is where the provider captures “what the query means”, and compilation is where it later decides “how to execute it”.

This also explains why your translation methods return new ShapedQueryExpression(q2, source.ShaperExpression) rather than producing results. The shaped query is the carrier EF Core expects at the boundary between translation and compilation. Your provider simply plugs its recorded query model (CustomMemoryQueryExpression) into that carrier, so the next phase can compile it into executable logic without losing any of the original LINQ intent.

In short, translation is not a provider-controlled stage but an EF Core-driven callback sequence. The provider participates by responding to translation hooks and capturing semantics, rather than independently parsing or executing LINQ.

Compilation Stage and Identity Resolution Integration

The compilation stage transforms the recorded query semantics into an executable expression tree. At this point, translation has already produced a ShapedQueryExpression containing a CustomMemoryQueryExpression. That object records the entity type, the ordered non-terminal steps, and any terminal operator. No execution has occurred yet.

A ShapedQueryExpression consists of two parts: a query expression and a shaper expression. The query expression describes how raw data should be retrieved. The shaper expression describes how each element should be shaped into the expected result type. Translation constructs both pieces, but neither is executed during that phase.

When a terminal operation such as ToList, First, Single, Count, or Any is invoked, EF Core enters the compilation phase and calls the provider’s IShapedQueryCompilingExpressionVisitor. The responsibility of this phase is to construct an expression tree that EF Core will later compile into a delegate. Execution only happens after that delegate is produced.

Compilation begins by constructing the row-level source:

IMemoryTable<TEntity>.QueryRows

This yields an IQueryable<SnapshotRow>, representing the storage-level rows. No entity instances exist at this stage.

The provider then builds a projection that converts each SnapshotRow into a tracked entity instance. This is done by inserting a Queryable.Select call whose selector invokes a helper method such as:

TrackFromRow<TEntity>(...)

The resulting pipeline conceptually resembles:

QueryRows
    .Select(row => TrackFromRow<TEntity>(...))
    .Where(...)
    .OrderBy(...)
    ...

The non-terminal steps recorded during translation are replayed in their original order. Each recorded step is translated into the corresponding Queryable method call expression and appended to the expression tree. For example, a recorded WhereStep becomes a Queryable.Where call over the current source expression. Terminal operators are appended as the final node, shaping the overall result form.

The final result of compilation is a complete expression tree describing the entire query. EF Core compiles this tree into a delegate and executes it.

Reusing EF Core Identity Resolution and Fix-Up

Identity management is not implemented by the provider. Instead, the provider integrates with EF Core’s tracking infrastructure at materialization time.

Inside TrackFromRow<TEntity>, the provider first ensures that the QueryContext has an initialized state manager. It then attempts identity resolution using the entity’s primary key:

var entry = qc.TryGetEntry(
    primaryKey,
    row.Key,
    throwOnNullKey: false,
    out var hasNullKey);

If a tracked instance already exists for that key, it is returned immediately. This guarantees that within a single DbContext, only one object instance exists per primary key.

If no tracked entry is found, the provider materializes a new instance using EF Core’s IEntityMaterializerSource. The snapshot is converted into a ValueBuffer, and EF Core’s materializer is invoked to construct the entity. After materialization, the provider calls:

qc.StartTracking(entityType, instance, originalValues);

This inserts the entity into EF Core’s identity map. From that point forward, identity resolution, navigation fix-up, and relationship consistency are handled entirely by EF Core.

The provider does not maintain its own identity map and does not implement fix-up logic. It supplies stable row identity and defers object identity semantics to EF Core. Because tracking is injected at materialization time, the provider remains fully compatible with change tracking and relationship fix-up without reimplementing those mechanisms.

Architectural Separation

The responsibilities remain clearly separated.

The storage layer provides SnapshotRow and guarantees stable row identity.
The compilation layer constructs the executable query pipeline.
The tracking layer, including identity resolution and fix-up, is fully managed by EF Core.

By delegating object identity to EF Core and keeping the storage model snapshot-based, the provider preserves EF Core’s semantics while maintaining a clean separation between row storage and entity instantiation.

.NET Learning Notes: Custom In-Memory Provider(3) - Storage Write Model and Key-Based Retrieval

Alex — Wed, 18 Feb 2026 04:44:12 +0000

Write Model

var e = new TestEntity { Name = "SmokeTest User" };  
ctx.TestEntities.Add(e);
var addCount = ctx.SaveChanges();

EF Core does not immediately write changes to the database when Add, Update, or Remove is called. These operations only modify the internal ChangeTracker. Every tracked entity is assigned an EntityState, such as Added, Modified, or Deleted. The StateManager accumulates these transitions in memory and maintains a consistent view of the unit of work.

When SaveChanges() is invoked, EF Core collects all pending changes from the ChangeTracker, transforms them into a list of IUpdateEntry instances, and forwards that list to the provider through Database(public class CustomMemoryEfDatabase : Database)'s SaveChanges(IList<IUpdateEntry> entries).

At this moment, control shifts from EF Core to the provider.

In the custom in-memory provider, the CustomMemoryEfDatabase class receives the accumulated entries and iterates through them. For each entry, the provider inspects EntityState and routes the entity to the appropriate table-level operation (Add, Update, or Remove). After all entries are processed, the provider commits those pending table changes to the underlying storage by calling IMemoryDatabase.SaveChanges().

All of detecting changes logic is delegated to EF Core. The provider’s responsibility begins only when EF Core decides to persist changes. EF Core owns state management and change detection. The provider owns storage persistence. SaveChanges() becomes the clear write boundary between these two layers.

public class CustomMemoryEfDatabase : Database
{
    public override int SaveChanges(IList<IUpdateEntry> entries)  
    {  
        var affected = 0;  
        foreach (var entry in entries)  
        {        // 先只做 Added，跑通闭环  
            if (entry.EntityState is not  
                (Microsoft.EntityFrameworkCore.EntityState.Added  
                or Microsoft.EntityFrameworkCore.EntityState.Modified  
                or Microsoft.EntityFrameworkCore.EntityState.Deleted))  
                continue;  

            var entityEntry = entry.ToEntityEntry();  
            var entity  = entityEntry.Entity;  
            var runtimeType = entity.GetType();  

            switch (entry.EntityState)  
            {            
                case Microsoft.EntityFrameworkCore.EntityState.Added:  
                    InvokeTableMethod(runtimeType, "Add", entity);  
                    affected++;  
                    break;  
                case Microsoft.EntityFrameworkCore.EntityState.Modified:  
                    InvokeTableMethod(runtimeType, "Update", entity);  
                    affected++;
                    break;  
                case Microsoft.EntityFrameworkCore.EntityState.Deleted:  
                    InvokeTableMethod(runtimeType, "Remove", entity);  
                    affected++; 
                    break;  
            }    
        }  
        _memoryDatabase.SaveChanges();  
        return affected;  
    }
}

Key-Based Retrieval (Find)

var e = new TestEntity { Name = "FinderTest" };  
ctx1.Add(e);  
ctx1.SaveChanges();  

id = e.Id;  
Console.WriteLine($"CTX1: Added entity id={id}");  

// Same context: Find twice (2nd time should be TRACKED HIT)  
var f1 = ctx1.Set<TestEntity>().Find(id);

In this provider, Find is not merely a dictionary lookup. It acts as a controlled bridge between the storage layer and EF Core’s tracking system. The operation consists of two conceptual steps: resolving a row by primary key and ensuring the resulting entity participates in EF Core’s identity map.

When Find is invoked, the provider first validates the supplied key values against the entity’s primary key metadata. It then checks the StateManager to determine whether an entity with the same key is already tracked. If so, that tracked instance is returned directly. This preserves EF Core’s guarantee that only one entity instance per key exists within a DbContext.

If the entity is not tracked, the provider performs a storage-level lookup. In this implementation, the in-memory table already materializes entities when retrieving rows, so table.Find(...) returns an entity instance rather than a raw snapshot. In other words, materialization happens inside the storage layer rather than through the query pipeline.

However, returning the instance alone would bypass EF Core’s tracking guarantees. Therefore, the provider explicitly registers the materialized entity with the StateManager, marking it as Unchanged with acceptChanges: true. This ensures the entity becomes part of EF Core’s identity map and behaves consistently with other tracked entities.

It is important to emphasize that Find does not implement identity resolution itself. Identity resolution is enforced by EF Core’s tracking infrastructure. The provider’s responsibility is to supply a stable key-based lookup and to integrate the resulting entity into the tracking system.

Unlike LINQ queries, Find bypasses the query translation pipeline entirely. It is a specialized, key-based retrieval path that integrates directly with the storage layer while still honoring EF Core’s tracking model.

public sealed class CustomMemoryEntityFinder<TEntity> : IEntityFinder<TEntity> where TEntity : class
{
    public object? Find(object?[]? keyValues) => FindCore(keyValues);

    private TEntity? FindCore(object?[]? keyValues)  
    {  
        if (keyValues is null) return null;  

        var pk = _entityType.FindPrimaryKey()  
             ?? throw new InvalidOperationException($"Entity type '{_entityType.Name}' has no primary key.");  

        // 1) tracked hit  
        var tracked = _stateManager.TryGetEntry(pk, keyValues);  
        if (tracked != null)  
        return tracked.EntityState == Microsoft.EntityFrameworkCore.EntityState.Deleted  
            ? null  
            : (TEntity)tracked.Entity;  

        // 2) store lookup  
        var table = _db.GetTable(_entityType.ClrType);  
        var foundObj = table.Find(ToObjectArray(keyValues));  
        if (foundObj is null)  
        {        
        return null;  
        }  
        var found = (TEntity)foundObj;  

        // 3) attach  
        var internalEntry = _stateManager.GetOrCreateEntry(found, _entityType);  

        internalEntry.SetEntityState(EntityState.Unchanged, acceptChanges: true);  
        return found;  
    }
}

In EF Core, Find exists as a dedicated identity-based retrieval path rather than as part of the normal query pipeline. Its purpose is fundamentally different from LINQ-based queries.

A standard query is expression-driven. It flows through translation, compilation, and shaped execution before entities are materialized. Find, by contrast, is key-driven. Its semantic contract is simple and strict: retrieve an entity by its primary key while preserving identity consistency within the current DbContext.

Performance is another reason for the separation. Primary key lookup is one of the most common operations in data access. It does not require expression tree translation or query compilation. Treating it as a specialized path avoids unnecessary overhead.

In this provider, the distinction is explicit. LINQ queries operate over IQueryable<SnapshotRow> and participate fully in translation and compilation. Find bypasses that mechanism and directly performs key-based lookup, then integrates the result into EF Core’s tracking system. The storage layer supplies row identity; the tracking layer enforces object identity.

In short, Find is intentionally independent from the query pipeline because it serves a different architectural role. It is not expression-based retrieval — it is identity-based retrieval.

                 ┌──────────────────────┐
                 │      DbContext       │
                 └─────────┬────────────┘
                           │
         ┌─────────────────┴─────────────────┐
         │                                   │
     Write Path                            Find
 (Add/Update/Delete)                   (Key-Based Lookup)
         │                                   │
         ▼                                   ▼
   ChangeTracker                       StateManager
 (EntityState entries)             (Identity Map check)
         │                                   │
         ▼                                   ▼
  SaveChanges() call                   Store Lookup
         │                                   │
         ▼                                   ▼
 CustomMemoryEfDatabase                MemoryTable.Find
         │                                   │
         ▼                                   ▼
     MemoryTable                        Entity Attach
 (Snapshot + pending)               (SetEntityState = Unchanged)

.NET Learning Notes: Custom In-Memory Provider(2) - In-Memory Database Runtime

Alex — Mon, 16 Feb 2026 08:55:35 +0000

CustomMemoryEFProvider Database Runtime GitHub

This section describes the runtime architecture of the in-memory storage layer, independent from EF Core’s translation and compilation pipeline.

MemoryDatabaseRoot (Store Registry / Database Name Isolation)

MemoryDatabaseRoot is the global registry of in-memory stores. It ensures that DbContext instances using the same database name share the same underlying store, while different names result in isolated databases.

It is registered as a singleton and caches MemoryDatabase instances by databaseName, providing name-based store isolation consistent with EF Core’s in-memory behavior.

IMemoryDatabase / MemoryDatabase (Table Registry + Transaction Routing)

MemoryDatabase acts as the coordination layer between entity tables and transactions. It maintains a registry of tables keyed by CLR type, ensuring that each entity type corresponds to a single in-memory table instance.

MemoryDatabase itself does not implement storage logic. Its responsibility is orchestration—table resolution and transaction routing—while the actual data storage behavior is delegated to MemoryTable<T>.

IMemoryTable / MemoryTable — Snapshot Rows and Change Merging

MemoryTable is designed as a snapshot-based storage unit. It does not store entity instances directly. Instead, each row is represented as a SnapshotRow, which contains a primary key and a ScalarSnapshot. This keeps the storage layer independent from tracked entity objects and prevents object graph leakage into the persistence layer.

The reason for introducing SnapshotRow — and exposing IQueryable<SnapshotRow> via QueryRows — is to clearly separate storage representation from entity instances, while allowing EF Core to fully control materialization, tracking, and identity resolution.

Instead of storing entity objects directly, the table stores ScalarSnapshot data. This avoids persisting full object graphs, navigation properties, collections, and circular references. By limiting storage to scalar values only, the provider keeps the storage layer deterministic, lightweight, and easy to clone during transactions. Snapshot-based storage also makes transaction isolation much simpler, since copy-on-write operates on row data rather than entity instances.

SnapshotRow encapsulates both the primary key (Key) and the scalar data (Snapshot). The key provides a stable identity for the row, which is essential for tracking and identity resolution. The snapshot represents the raw column values. This structure mirrors how relational providers conceptually operate: rows are independent data records, and entity instances are created later during materialization.

Returning IQueryable<SnapshotRow> preserves deferred execution and composability. EF Core can continue building expression trees on top of it (Where, Select, Join, Include, etc.), and the provider participates in the normal translation and compilation pipeline. Only during the shaped query compilation phase are rows materialized into entity instances. This ensures the provider does not prematurely instantiate entities or bypass EF Core’s tracking pipeline.

Disabling direct IQueryable<TEntity> access enforces this design. All queries must go through QueryRows, ensuring that entity materialization, identity resolution, and tracking remain under EF Core’s control rather than leaking storage-level objects directly.

In short, SnapshotRow establishes a clean row-level storage contract. It enables clear separation of concerns, simplifies transaction isolation, and maximizes reuse of EF Core’s existing tracking and identity resolution mechanisms.**

Internally, the table maintains two logical states: committed data and pending changes. Committed data represents the stable, persisted state of the table. Pending changes record modifications before SaveChanges is called, and each entry is marked as Added, Modified, or Deleted.

Only scalar properties are included in ScalarSnapshot. Navigation properties, collections, and complex object graphs are intentionally excluded. This avoids circular references, deep cloning problems, and recursive graph persistence. The table remains a flat, value-based storage model, while relationship fix-up and identity resolution are handled later in the EF Core query pipeline rather than inside the storage layer.

In ExtractSnapshot, scalar property values are stored in a dictionary keyed by property name rather than by positional index. This design avoids relying on property ordering and instead preserves a stable name-based mapping. EF Core’s query pipeline does not guarantee that property ordering during materialization will always match CLR reflection order. By storing values keyed by property name, snapshot reconstruction becomes resilient to ordering differences and aligns naturally with EF Core’s metadata-driven property resolution model. In other words, the snapshot format is name-oriented rather than position-oriented, which makes materialization deterministic and decoupled from reflection order assumptions.

Transactions and Isolation Model — MemoryTransaction and Transactional Clone

The transaction model in this provider follows a simplified isolation strategy based on table-level cloning. When a transaction begins, the system creates transactional tables by cloning the current visible state of each base table. The cloned state is not a raw object copy of entity instances, but a snapshot-level copy based on the merged result of committed data and pending changes as exposed by QueryRows. This ensures that the transaction starts from a consistent logical view of the database.

Once a transaction is active, all table access is routed to the transactional tables. Changes are applied only to these transactional copies, leaving the base tables untouched during the transaction scope.

On Commit, the transactional tables replace the corresponding base tables. In practice, this means clearing the base table and re-importing all rows from the transactional table, effectively promoting the transactional state to committed state.

On Rollback, the transactional tables are simply discarded. Because base tables were never modified during the transaction, no additional undo logic is required.

This design represents a single-transaction, table-level copy-on-write model. It does not support nested transactions or fine-grained row versioning. Instead, it prioritizes architectural clarity and predictable isolation behavior suitable for a demonstration provider.

    [DbContext] 

--> [MemoryDatabaseRoot]

--> [MemoryDatabase]

--> [MemoryTable<T>]

--> [SnapshotRow]

--> [ScalarSnapshot]

.NET Learning Notes: Custom In-Memory Provider(1) - Registration and Discovery

Alex — Fri, 13 Feb 2026 04:28:39 +0000

1. Custom In-Memory Provider Registration and Discovery in EF Core

Core Goal: The objective of this section is to establish a proper EF Core provider registration entry point, allowing EF Core to discover the custom provider, integrate it into the options pipeline, and later invoke its service registration logic during DbContext initialization.

Key Implementations

1.1 Provider Extension Entry

I implement EF Core-style extension methods as the entry point for the custom provider. Following EF Core's naming conventions: Use Use[ProviderName] (e.g., UseCustomMemoryDb)—the standard convention for EF Core provider activation methods.

// Configures the DbContext to use the custom in-memory database provider
public static DbContextOptionsBuilder UseCustomMemoryDb( 
    this DbContextOptionsBuilder builder, 
    string databaseName, 
    bool clearOnCreate = false) 
{ 
    if (builder == null)  
    {  
        throw new ArgumentNullException(nameof(builder), "DbContextOptionsBuilder cannot be null.");  
    }  

    // Create extension instance with user-provided configuration  
    var extension = builder.Options.FindExtension<CustomMemoryDbContextOptionsExtension>()   
    ?? new CustomMemoryDbContextOptionsExtension(databaseName, clearOnCreate);

    // Add or update the extension in EF Core's options (EF Core internal API)  
    ((IDbContextOptionsBuilderInfrastructure)builder).AddOrUpdateExtension(extension);

    return builder;
}

AddOrUpdateExtension injects the custom IDbContextOptionsExtension into EF Core’s immutable options model.

During service provider construction, EF Core scans registered extensions and invokes their ApplyServices method. This is the moment where provider-specific services (query compilation, execution pipeline, database abstractions, etc.) are wired into the internal DI container.

And we can use it by:

services.AddDbContext<AppDbContext>(
    options =>
        options.UseCustomMemoryDb("TestDatabase"));

This step only integrates the extension into EF core's configuration pipeline. The actual behavior is defined later in the IDbContextOptionsExtension.ApplyServices implementation.

1.2 Provider Extension Class

The CustomMemoryDbContextOptionsExtension class implements IDbContextOptionsExtension. This extension is the provider's configuration carrier and the formal signal EF Core uses to discover and activate a database provider. EF Core recognizes it as a database provider via the DbContextOptionsExtensionInfo metadata (e.g. IsDatabaseProvider => true), and later calls ApplyServices as the entry hook to register provider services.

ApplyServices runs during EF Core’s internal service-provider construction for a given DbContextOptions instance. This is important: provider services are registered into EF Core’s internal service collection for that context (a scoped “internal container”), not the application’s global DI container. In other words, the provider wiring is isolated to the DbContext’s internal dependency graph.

public void ApplyServices(IServiceCollection services)  
{  
    // Register configuration as singleton (available to all provider services)  
    services.AddSingleton(new CustomMemoryDbConfig(_databaseName, _clearOnCreate));  
    services.AddEntityFrameworkCustomMemoryDatabase();  
}

public static IServiceCollection AddEntityFrameworkCustomMemoryDatabase(  
    this IServiceCollection serviceCollection)  
{  
    // Validate input (align with official provider patterns)
    ArgumentNullException.ThrowIfNull(serviceCollection, nameof(serviceCollection));  
    // NEW: register SnapshotValueBufferFactory for compiling visitor factory  
    serviceCollection.TryAddSingleton<SnapshotValueBufferFactory>();  
    
    // Step 1: register EF Core framework-facing services (core pipeline slots)
    var builder = new EntityFrameworkServicesBuilder(serviceCollection);  

    // These are EF Core "framework services" that must be present for a database provider
    builder.TryAdd<ITypeMappingSource, CustomMemoryTypeMappingSource>();  
    builder.TryAdd<LoggingDefinitions, CustomMemoryLoggingDefinitions>();  
    builder.TryAdd<IQueryableMethodTranslatingExpressionVisitorFactory,  
        CustomMemoryQueryableMethodTranslatingExpressionVisitorFactory>();  
    builder.TryAdd<IShapedQueryCompilingExpressionVisitorFactory,  
        CustomMemoryShapedQueryCompilingExpressionVisitorFactory>();  
    builder.TryAdd<IValueGeneratorSelector, CustomMemoryValueGeneratorSelector>();  
    builder.TryAdd<IDatabaseProvider, CustomMemoryDatabaseProvider>();
    // Required: provider must supply an IDatabase implementation 
    builder.TryAdd<IDatabase, CustomMemoryEfDatabase>(); 
    builder.TryAdd<IQueryContextFactory, CustomMemoryQueryContextFactory>();

    // Register EF Core core services (fills remaining defaults) 
    builder.TryAddCoreServices();

    // Override specific EF services when default behavior is not compatible with this provider
    serviceCollection.Replace(  
        ServiceDescriptor.Scoped<IEntityFinderSource, CustomMemoryEntityFinderSource>()  
    );

    // Step 2: provider-specific extension services (provider-only abstractions)
    builder.TryAddProviderSpecificServices(p =>  
    {  
        p.TryAddScoped<IEntityFinderFactory, CustomMemoryEntityFinderFactory>();  
        p.TryAddScoped<ICustomMemoryDatabaseProvider, CustomMemoryDatabaseProvider>();  
    });

    // Step 3: provider storage services (actual in-memory database implementation)
    serviceCollection.TryAddSingleton(new MemoryDatabaseRoot());  
    serviceCollection.TryAddScoped<IMemoryDatabase>(sp =>  
    {  
        var cfg = sp.GetRequiredService<CustomMemoryDbConfig>();  
        var root = sp.GetRequiredService<MemoryDatabaseRoot>();  
        var db = root.GetOrAdd(cfg.DatabaseName);  

        if (cfg.ClearOnCreate)  
        {            db.ClearAllTables();  
        }        return db;  
    });    serviceCollection.TryAddScoped(typeof(IMemoryTable<>), typeof(MemoryTable<>));  
    return serviceCollection;      
}

Provider registers config as singleton (services.AddSingleton(new CustomMemoryDbConfig(_databaseName, _clearOnCreate));). Because EF Core can cache or reuse the internal service provider based on the options fingerprint.

EF Core assumes your provider fully participates in the internal pipeline. That pipeline has specific service slots (translation, compilation, execution, tracking helpers, value generation, type mapping, etc.). These slots are not optional in practice: EF Core will hit them during normal operations such as translating LINQ, compiling shaped queries, creating query contexts, generating keys, and materializing results.

TryAddCoreServices() fills many of those slots with EF Core defaults. That’s useful, but it also means you must register your provider implementations before calling TryAddCoreServices() when you intend to override defaults (or when defaults are provider-incompatible). Otherwise, EF Core may wire up default services that either do not work for your provider, or silently bypass your implementation.

IQueryableMethodTranslatingExpressionVisitorFactory

This is the core LINQ-to-provider translation entry. EF Core parses LINQ into expression trees, then uses this factory to create a translator visitor that converts LINQ method calls (Where/Select/Join/etc.) into a provider-specific query representation. If you want to control what LINQ patterns are supported and how they are interpreted, this is one of the most important extension points.
IShapedQueryCompilingExpressionVisitorFactory

After translation, EF Core still needs to compile a “shaped query”: a runnable delegate that materializes results into entity instances (or projections), handles tracking, identity resolution, and include fix-up. This factory is where a provider plugs into the compilation phase and controls how snapshots/rows become materialized objects.
IValueGeneratorSelector

the function of this Selector is value generation. By default, EF Core’s built-in ValueGeneratorSelector only knows how to generate: Guid, string, byte[]. It does not generate int identities unless a provider supplies that behavior.

.NET Learning Notes: YARP

Alex — Wed, 05 Nov 2025 20:54:56 +0000

A reverse proxy is a server that sits in fronts of one or more backend servers and handles requests from clients on behalf of those servers.

A reverse proxy provides several benefits:

Routing
Load balancing
Scalability: By distributing traffic across multiple serves, a reverse proxy helps scale apps to handle more users and higher loads.
SSL/TLS Termination
Incoming URLs can be transformed before forwarding to the backend. Internal service endpoints can change without affecting external URLs.
Security: Internal service endpoints can be hidden from external exposure.
Caching # YARP ## 1. Load Balancing Load balancing policies are registered in the DI container via the AddLoadBalancingPolicies() method, which is automatically called by AddReverseProxy(). The middleware is added with UseLoadBalancing, which is included by default in the parameterless MapReverseProxy method. ### Cluster Configuration The algorithm used to determine the destination can be configured by setting the LoadBalancingPolicy. If no policy is specified, PowerOfTwoChoices will be used. Built-in policies:
FirstAlphabetical
Random
PowerOfTwoChoices (default)
RoundRobin
LeastRequests

"ReverseProxy": {
  "Clusters": {
    "cluster1": {
      "LoadBalancingPolicy": "RoundRobin",
      "Destinations": {
        "cluster1/destination1": {
          "Address": "https://localhost:10000/"
        },
        "cluster1/destination2": {
          "Address": "https://localhost:10010/"
        }
      }
    }
  }
}

2. Session Affinity

Session affinity is a mechanism to bind a causally related request sequence to the destination that handled the first request when the load is balanced among several destinations.

What is an Affinity Key

In YARP, an Affinity Key is simply the identifier that ties a client's subsequent requests to the same backend destination.

The key itself is just a container name.
The value stored inside that key is generated dynamically by YARP at runtime and is different for each client or session.
This value encodes the target destination ID - or sometimes a group of destinations - so that the proxy knows where to route the next request. ### Affinity failure policy
Redistribute (default)- tries to establish a new affinity to one of available healthy destinations by skipping the affinity lookup step and passing all healthy destination to the load balancer the same way it is done for a request without any affinity.
Return503Error - send a 503 response back to the client and request processing is terminated. ## 3. Destination health checks ### Active health checks YARP can proactively monitor destination health by sending periodic probing requests to designated health endpoints and analyzing responses.

"Clusters": {
  "cluster1": {
    "HealthCheck": {
      "Active": {
        "Enabled": "true",
        "Interval": "00:00:10",
        "Timeout": "00:00:10",
        "Policy": "ConsecutiveFailures",
        "Path": "/api/health",
        "Query": "?foo=bar"
      }
    },
    "Metadata": {
      "ConsecutiveFailuresHealthPolicy.Threshold": "3"
    },
    "Destinations": {
      "cluster1/destination1": {
        "Address": "https://localhost:10000/"
      },
      "cluster1/destination2": {
        "Address": "http://localhost:10010/",
        "Health": "http://localhost:10020/"
      }
    }
  }
}

Passive health checks

YARP can passively watch for successes and failures in client request proxying to reactively evaluate destination health states.

"Clusters": {
  "cluster1": {
    "HealthCheck": {
      "Passive": {
        "Enabled": "true",
        "Policy": "TransportFailureRate",
        "ReactivationPeriod": "00:02:00"
      }
    },
    "Metadata": {
      "TransportFailureRateHealthPolicy.RateLimit": "0.5"
    },
    "Destinations": {
      "cluster1/destination1": {
        "Address": "https://localhost:10000/"
      },
      "cluster1/destination2": {
        "Address": "http://localhost:10010/"
      }
    }
  }
}

When a destination gets marked as unhealthy, it stops receiving new requests until it gets reactivated after a configured period.

4. Rate Limiting

The reverse proxy can be used to rate-limit requests before they are proxied to the destination servers. This can reduce load on the destination servers, add a layer of protection, and ensure consistent policies are implemented across your applications.

Global Limiter

globalLimiter is an illustration here, we need to implement it by ourself.

services.AddRateLimiter(options => options.GlobalLimiter = globalLimiter);

Route Limiter

Rate limiter policies can be specified per route.

{
  "ReverseProxy": {
    "Routes": {
      "route1" : {
        "ClusterId": "cluster1",
        "RateLimiterPolicy": "customPolicy",
        "Match": {
          "Hosts": [ "localhost" ]
        }
      }
    },
    "Clusters": {
      "cluster1": {
        "Destinations": {
          "cluster1/destination1": {
            "Address": "https://localhost:10001/"
          }
        }
      }
    }
  }
}

RateLimiter policies can be configured in services as follows:

services.AddRateLimiter(options =>
{
    options.AddFixedWindowLimiter("customPolicy", opt =>
    {
        opt.PermitLimit = 4;
        opt.Window = TimeSpan.FromSeconds(12);
        opt.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
        opt.QueueLimit = 2;
    });
});

Then add the RateLimiter middleware:

app.UseRateLimiter();

app.MapReverseProxy();

Disable Rate Limiting

Specifying the value disable in a route's RateLimiterPolicy parameter means the rate limiter middleware will not apply any policies to this route, even the default policy.

.NET Learning Notes: How to use Redis in ASP.NET

Alex — Mon, 04 Aug 2025 08:23:39 +0000

1. Install NuGet package

dotnet add package StackExchange.Redis

2. Configuration connection string

Here, I config it in appsettings.json

  "ConnectionStrings": {
    "Redis": "localhost:6379"
  }

3. Configure connectiong string in Program.cs

builder.Services.AddSingleton<IConnectionMultiplexer>(sp =>
{
    var redisCon = builder.Configuration.GetConnectionString("Redis")!;
    var configuration = ConfigurationOptions.Parse(redisCon, true);
    configuration.ResolveDns = true;
    return ConnectionMultiplexer.Connect(configuration);
});

4. DI IConnectionMultiplexer to service

public class BasicJwtTokenValidationMiddleware
{
    private readonly RequestDelegate _next;
    private readonly List<string> _authWhiteList;
    private readonly IConnectionMultiplexer _redis;

    public BasicJwtTokenValidationMiddleware(RequestDelegate next, 
        IOptions<AuthenticationOptions> authOptions,
        IConnectionMultiplexer redis)
    {
        _next = next;
        _authWhiteList = authOptions.Value.AuthWhiteList;
        _redis = redis;
    }

    public Task Invoke(HttpContext context)
    {
      // *****
      var redisVersion = await _redis.GetDatabase().StringGetAsync($"jwt:version:{userPublicId}");

      // ***
    }
}

5. How to mock it in unit test (use AutoFixture + Moq)

[Theory, AutoMoqData]
public async Task IsValidVersionAsync_ReturnsTrue_WhenVersionMatches(
    [Frozen] Mock<IConnectionMultiplexer> redisMock,
    [Frozen] Mock<IDatabase> dbMock)
{
    // Arrange
    var userId = "user-123";
    var version = "3";

    dbMock.Setup(db => db.StringGetAsync($"jwt:version:{userId}", It.IsAny<CommandFlags>()))
        .ReturnsAsync(version);

    redisMock.Setup(r => r.GetDatabase(It.IsAny<int>(), null))
        .Returns(dbMock.Object);
}

Why do we mock GetDatabase with parameters even though we call it without parameters in code?

Answer: Because GetDatabase uses default parameter values in C#;
Method definition of GetDatabase:

IDatabase GetDatabase(int db = -1, object? asyncState = null);

So, if we mock with our parameters like:redisMock.Setup(c => c.GetDatabase()). It won't match the actual call to GetDatabase(-1, null)

.NET Learning Notes: Using MailHog with .NET for Email Testing

Alex — Sun, 06 Jul 2025 10:09:54 +0000

MailHog is a simple and powerful tool for capturing and viewing emails sent from your application during development or testing. It acts as a local SMTP server and provides a web interface for viewing the captured messages. This blog explains how to use MailHog with a .NET project for email sending and testing, and how to integrate it into CI pipelines.

Why Use MailHog?

Captures emails sent from development apps.
Web UI for viewing emails at http://localhost:8025.
Excellent for integration testing.
No real emails sent — safe for all environments.
Free!!

Running MailHog with Docker

The easiest way to start MailHog is with Docker:

docker run -d -p 1025:1025 -p 8025:8025 mailhog/mailhog

SMTP Server: localhost:1025
Web UI: http://localhost:8025

Or with a docker-compose.mailhog.yml file:

version: '3.8'
services:
  mailhog:
    image: mailhog/mailhog
    ports:
      - "1025:1025"
      - "8025:8025"

Then run:

docker-compose -f docker-compose.mailhog.yml up -d

Sending Email with .NET and MailHog

Here’s how you can configure and use MailHog to send test emails in a .NET service.

1. SMTP Options

You can configure MailHog with no credentials and unencrypted SMTP:

// appsettings.Development.json
{
  "Smtp": {
    "Host": "localhost",
    "Port": 1025,
    "User": "",
    "Password": ""
  }
}

2. The Email Service Class

This service uses MailKit and MimeKit to send email messages:

public class SmtpEmailService : IEmailService
{
    private readonly SmtpOptions _smtpOptions;

    public SmtpEmailService(IOptions<SmtpOptions> smtpOptions, ILogger<SmtpEmailService> logger)
    {
        _smtpOptions = smtpOptions.Value;
    }

    public async Task SendEmailAsync(EmailSendRequestedEvent emailEvent)
    {
        var message = new MimeMessage();
        message.From.Add(new MailboxAddress("FinTrack", "no-reply@fintrack.com"));
        message.To.Add(new MailboxAddress(emailEvent.ToName ?? emailEvent.To, emailEvent.To));
        message.Subject = emailEvent.Subject;
        message.Body = emailEvent.IsHtml
            ? new TextPart("html") { Text = emailEvent.Body }
            : new TextPart("plain") { Text = emailEvent.Body };

        using var client = new SmtpClient();
        await client.ConnectAsync(_smtpOptions.Host, _smtpOptions.Port, SecureSocketOptions.None);
        await client.AuthenticateAsync(_smtpOptions.User, _smtpOptions.Password);
        await client.SendAsync(message);
        await client.DisconnectAsync(true);
    }
}

Use SecureSocketOptions.None because MailHog does not support SSL or STARTTLS.

3. Unit Test Example

Use mock IOptions and ILogger, then run a real test that talks to MailHog:

[Fact]
public async Task SendAsync_WithValidInput_SendsEmailWithoutException()
{
    var smtpOptions = new SmtpOptions
    {
        Host = "localhost",
        Port = 1025,
        User = "",
        Password = ""
    };

    var optionsMock = new Mock<IOptions<SmtpOptions>>();
    optionsMock.Setup(o => o.Value).Returns(smtpOptions);

    var loggerMock = new Mock<ILogger<SmtpEmailService>>();
    var emailService = new SmtpEmailService(optionsMock.Object, loggerMock.Object);

    var emailEvent = new EmailSendRequestedEvent
    {
        From = "from@test.com",
        To = "to@test.com",
        ToName = "Receiver",
        Subject = "Test Subject",
        Body = "Test Body",
        IsHtml = false
    };

    await emailService.SendEmailAsync(emailEvent);
}

Once this test runs, visit http://localhost:8025 and you’ll see the message inside MailHog's web UI. No real emails will be sent — perfect for development and CI.

Can You Set Password for MailHog?

MailHog does not support SMTP authentication or TLS encryption. It’s intentionally insecure for local use only.

CI Integration

You can easily run MailHog inside GitHub Actions with this snippet:

services:
  mailhog:
    image: mailhog/mailhog
    ports:
      - 1025:1025
      - 8025:8025

Then point your tests to localhost:1025, and run them just like local.

MailHog is a powerful and easy-to-use tool for safely testing email logic without sending real emails. It's perfect for local development, integration testing, and even CI environments.

.NET Learning Notes: Implementing EventBus with CAP and RabbitMQ in Microservices

Alex — Sun, 29 Jun 2025 09:19:11 +0000

CAP Doc

1. What Is CAP and Why Do We Need It?

In microservices architectures, services often need to communicate with each other asynchronously to remain decoupled, scalable, and fault-tolerant. Event-driven communication using message queues like RabbitMQ or Kafka is a popular solution — but it comes with its own set of challenges, especially around reliability and distributed transaction consistency.

This is where CAP (Distributed Transaction Solution in .NET Core) comes in.

CAP is an open-source .NET library designed to help developers manage reliable event publishing and consumption across services, while also handling eventual consistency in database transactions. It follows the Outbox Pattern, allowing you to store events in your local database as part of a transaction and forward them to a message queue only after the transaction succeeds.

Problems CAP Solves

Distributed Transaction Complexity: Ensures that data changes and event publishing happen atomically, without requiring a distributed transaction coordinator.
Message Loss Prevention: Persist events locally before sending, so that messages are not lost even if the broker or consumer service is temporarily down.
Retry & Monitoring: Built-in retry logic and a web dashboard for tracking sent and received messages.

With just a few lines of configuration, CAP lets your services publish and consume events reliably, while abstracting away the complexities of queue connection, message delivery, error handling, and storage.

2. How to Integrate CAP in a .NET Microservice

In this section, we'll cover how to add CAP to your .NET microservice, including the required NuGet packages, how to configure RabbitMQ and the database, and the meaning of the DefaultGroup setting for subscribers.

2.1 Install Required Packages

You need to install the following NuGet packages:

For EF Core + RabbitMQ setup:

dotnet add package DotNetCore.CAP
dotnet add package DotNetCore.CAP.RabbitMQ
dotnet add package DotNetCore.CAP.EntityFrameworkCore

2.2 Configure CAP in Program.cs

First, make sure your DbContext is already registered with EF Core:

builder.Services.AddDbContext<AppDbContext>(options =>
    options.UseMySql(builder.Configuration.GetConnectionString("DefaultConnection"),
        ServerVersion.AutoDetect(builder.Configuration.GetConnectionString("DefaultConnection"))));

builder.Services.AddCap(x =>
{
    x.UseEntityFramework<AppDbContext>(); // Use EF Core to store CAP events

    x.UseRabbitMQ(cfg =>
    {
        cfg.HostName = builder.Configuration["CAP:RabbitMQ:HostName"];
        cfg.UserName = builder.Configuration["CAP:RabbitMQ:UserName"];
        cfg.Password = builder.Configuration["CAP:RabbitMQ:Password"];
    });
});

2.3 Configuration File

You can store the CAP configuration in appsettings.json:

"CAP": {
  "DefaultGroup": "order.service",
  "RabbitMQ": {
    "HostName": "localhost",
    "UserName": "guest",
    "Password": "guest"
  }
}

2.4 What Is DefaultGroup?

The DefaultGroup defines the consumer group name for this service. It's used to distinguish multiple subscribers of the same topic in different services. CAP ensures that only one subscriber within the same group will consume the message.

Services with different group names can receive the same event (fan-out).
Services with the same group name will compete to consume the event (load balancing).

2.5 Do Publisher and Subscriber Use the Same Configuration?

Yes, both the sender and receiver services use the same basic CAP configuration. The only difference is that:

The publisher uses ICapPublisher to send messages.
The subscriber uses the [CapSubscribe] attribute to receive messages.

Both sides must be properly configured with the same transport (e.g., RabbitMQ) and the same storage (e.g., MySQL via EF Core) to work together.

3. Running RabbitMQ with Docker (Local Setup)

To enable CAP to work properly, we need a running instance of a message broker. RabbitMQ is one of the supported brokers and works well for local development and testing.

Step 1: Pull the Official RabbitMQ Image

docker pull rabbitmq:3-management

This image includes the RabbitMQ server and the Management UI plugin.

Step 2: Run the RabbitMQ Container

docker run -d \
  --hostname rabbitmq \
  --name rabbitmq \
  -p 5672:5672 \
  -p 15672:15672 \
  rabbitmq:3-management

Explanation:

--name rabbitmq: Names the container so you can manage it easily.
--hostname rabbitmq: Sets the hostname inside the container (optional).
-p 5672:5672: Exposes the RabbitMQ AMQP port.
-p 15672:15672: Exposes the RabbitMQ Management UI port.
-d: Runs the container in detached mode.

Step 3: Access the RabbitMQ Management UI

Open your browser and go to: http://localhost:15672
Default login credentials:
- Username: guest
- Password: guest

Summary

With just two commands (pull and run), you’ll have a fully functional RabbitMQ server running locally, along with a powerful UI for inspecting exchanges, queues, and messages — ideal for working with CAP.

4. Publish and Subscribe with CAP

In the sender service, inject ICapPublisher and use it to publish an event:

public class OrderService
{
    private readonly ICapPublisher _capBus;

    public OrderService(ICapPublisher capBus)
    {
        _capBus = capBus;
    }

    public async Task PlaceOrderAsync()
    {
        await _capBus.PublishAsync("order.created", new { OrderId = 123, Amount = 99.9 });
    }
}

In the receiver service, add a method with [CapSubscribe] to handle the event:

public class OrderEventHandler
{
    [CapSubscribe("order.created")]
    public void HandleOrderCreated(dynamic data)
    {
        Console.WriteLine($"Order received: {data.OrderId}, Amount: {data.Amount}");
    }
}

Both services need to be connected to the same message broker and configured with CAP.

5. CAP Message Tables

CAP relies on two key tables in your database to ensure message reliability and traceability:

cap.published: This table stores all the messages that have been published. When a message is sent via ICapPublisher, it first gets recorded here. CAP will attempt to deliver this message based on the retry policy if the subscriber is not ready.
cap.received: Once a message is successfully consumed by a subscriber, it will be recorded in this table. This acts as a log of all successfully handled messages.

If you're using in-memory storage instead of a real database, these records will not be persisted after a service restart.

CAP ensures that messages are reliably transferred and tracked between microservices using this storage mechanism.

6. Why CAP Is Powerful

One of the most powerful features of CAP is its simplicity.

You only need to configure the database and message broker once, and CAP will automatically handle:
- Reliable message delivery
- Retry mechanisms
- Failure tracking
- Distributed transaction support (Outbox pattern)
No extra queue management or manual retry logic is needed.
With just [CapSubscribe] and ICapPublisher, microservices can communicate reliably and efficiently.

CAP abstracts away much of the complexity of building a robust EventBus system. Once configured, your services can send and receive events without worrying about infrastructure details.