DEV Community: Avinash Zala

Clean Architecture in .NET 8: A 2026 Starter Template with 4 Projects, EF Core, and JWT Auth

Avinash Zala — Mon, 22 Jun 2026 12:47:52 +0000

I joined a team where the controller was 800 lines long, the business rules were scattered between the controller and the DbContext, and "to run the tests, spin up a SQL Server in Docker" was a sentence I heard every week. The fix was Clean Architecture. The argument I had with the team lead was about how to actually structure it. We argued for two weeks. Then I built this template so the next person wouldn't have to.

This is the Clean Architecture .NET 8 starter template I wish someone had handed me on day one. Four projects, strict dependency direction, domain entities that own their own invariants, and an Application layer you can unit test with Moq — no database required. The whole repo is on GitHub, MIT-licensed, runs with dotnet run, and ships with xUnit tests, JWT auth, Swagger, Docker, and CI.

This post is the explanation of why each project exists, what goes in it, and what I learned the hard way about getting Clean Architecture right in .NET.

The problem Clean Architecture solves

The naive way to build a .NET Web API is one project, one folder structure, and "everything talks to everything":

MyApp/
  Controllers/
    ProductsController.cs    ← HTTP stuff
    OrdersController.cs      ← HTTP stuff + business rules
  Services/
    ProductService.cs        ← business rules + DbContext.SaveChanges
  Data/
    AppDbContext.cs          ← EF Core, entities
  Models/
    Product.cs               ← POCO with public setters

This works for the first 1,000 lines. By 5,000 lines, the controller is doing five things at once. By 10,000, "to test this, I need a database" is the answer to every test question, and your CI takes 20 minutes because every test run spins up SQL Server.

Clean Architecture says: separate the business rules from the HTTP boundary, separate the database from the business rules, and enforce it with project references. A controller is allowed to call a service. A service is allowed to call a repository. A repository is allowed to know about EF Core. Nothing is allowed to know about anything "above" it in the chain. That's the rule.

The .NET implementation is four projects:

                ┌─────────────────────────────────┐
                │           Api project           │  ← HTTP, DTOs, Swagger
                │   (Controllers, Program.cs)     │
                └────────────┬────────────────────┘
                             │ references
                ┌────────────▼────────────────────┐
                │       Application project       │  ← Use cases, services
                │  (DTOs, Interfaces, Services)   │
                └────────────┬────────────────────┘
                             │ references
                ┌────────────▼────────────────────┐
                │      Infrastructure project     │  ← EF Core, JWT
                │ (DbContext, Repositories)       │
                └────────────┬────────────────────┘
                             │ references
                ┌────────────▼────────────────────┐
                │        Domain project           │  ← Zero dependencies
                │  (Entities, Interfaces)         │
                └─────────────────────────────────┘

The arrows only go down. The Domain project has zero NuGet dependencies outside the BCL. The Application project references Domain only. The Infrastructure project references both, but Domain and Application never reference Infrastructure. The compiler enforces the direction. A circular reference is a build error, not a code review comment.

Part 1 — the Domain layer

The Domain project contains the entities and the abstract interfaces that the Application layer depends on. No EF Core, no ASP.NET, no HTTP, no JSON. Just business rules.

The interesting file is Product.cs. It's not a POCO with public setters. It's a class with a DecreaseStock method that enforces its own invariants:

public class Product : BaseEntity
{
    public string Name { get; set; } = string.Empty;
    public string Description { get; set; } = string.Empty;
    public decimal Price { get; set; }
    public int StockQuantity { get; set; }

    // Domain logic — entity enforces its own invariants
    public bool IsInStock(int requestedQuantity) => StockQuantity >= requestedQuantity;

    public void DecreaseStock(int quantity)
    {
        if (quantity <= 0)
            throw new ArgumentException("Quantity must be positive.", nameof(quantity));
        if (!IsInStock(quantity))
            throw new InvalidOperationException($"Insufficient stock for product '{Name}'. " +
                                               $"Requested {quantity}, available {StockQuantity}.");

        StockQuantity -= quantity;
        UpdatedAt = DateTime.UtcNow;
    }
}

The crucial thing is that DecreaseStock is the only way to change StockQuantity from outside the class. The setter is public because EF Core needs it, but the domain logic is "you can't decrease stock by a negative number, and you can't decrease stock below zero." That logic lives on the entity, in Domain. The service, the controller, the test — none of them have to remember to validate. They call product.DecreaseStock(quantity) and the entity decides.

This is the part of Clean Architecture that actually matters. The dependency-direction rule is hygiene. The "entities own their invariants" rule is the architecture. If you take one thing from this post, take that: business rules belong on the business objects, not on the services that manipulate them.

Part 2 — the Application layer

The Application project contains the use cases. A use case is a verb at the level of the user, not the database. "Create a product" is a use case. "Insert a row into the products table" is not.

ProductService.cs is the canonical example:

public class ProductService : IProductService
{
    private readonly IUnitOfWork _unitOfWork;

    public ProductService(IUnitOfWork unitOfWork)
    {
        _unitOfWork = unitOfWork;
    }

    public async Task<ProductDto?> GetByIdAsync(int id, CancellationToken ct = default)
    {
        var product = await _unitOfWork.Products.GetByIdAsync(id, ct);
        return product is null ? null : MapToDto(product);
    }

    public async Task<ProductDto> CreateAsync(CreateProductDto dto, CancellationToken ct = default)
    {
        var product = new Product
        {
            Name = dto.Name,
            Description = dto.Description,
            Price = dto.Price,
            StockQuantity = dto.StockQuantity
        };

        await _unitOfWork.Products.AddAsync(product, ct);
        await _unitOfWork.SaveChangesAsync(ct);

        return MapToDto(product);
    }
}

Two things to notice.

First, the service takes IUnitOfWork, not AppDbContext. This is the testability win. IUnitOfWork is an interface declared in the Domain project. The Infrastructure project provides the EF Core implementation. The Application project never knows EF Core exists. In a unit test, I write a fake IUnitOfWork in 10 lines and the test runs in 0 ms, with no database.

Second, the service returns DTOs, not entities. DTOs are flat data shapes designed for the wire. Entities are rich domain objects with behaviour. If I returned Product from the service, the controller would have access to product.DecreaseStock, and the controller could break invariants. The DTO is a contract: "this is what the service exposes to the outside world, and nothing more."

The "decrease stock" use case is the most interesting one because it touches three layers:

public async Task<OrderDto> CreateOrderAsync(CreateOrderDto dto, CancellationToken ct = default)
{
    var product = await _unitOfWork.Products.GetByIdAsync(dto.ProductId, ct)
        ?? throw new InvalidOperationException($"Product {dto.ProductId} not found.");

    // Domain logic — product enforces its own invariants
    product.DecreaseStock(dto.Quantity);

    var order = new Order
    {
        CustomerId = dto.CustomerId,
        Items = new List<OrderItem> { new() { ProductId = product.Id, Quantity = dto.Quantity } }
    };

    await _unitOfWork.Orders.AddAsync(order, ct);
    await _unitOfWork.SaveChangesAsync(ct);

    return MapToDto(order);
}

The service does three things: load the product, ask the product to decrease its own stock, save the order. The validation logic — "you can't order 5 of something that has 3 in stock" — is in the entity. The service doesn't know about that rule. The service couldn't bypass it even if it tried, because DecreaseStock throws and the exception propagates up.

Part 3 — the Infrastructure layer

The Infrastructure project is where the rest of the world lives. EF Core, the SQL Server or SQLite provider, the JWT signing service, anything that talks to a network or a file system. Domain and Application never know this project exists.

AppDbContext.cs is the EF Core surface:

public class AppDbContext : DbContext
{
    public AppDbContext(DbContextOptions<AppDbContext> options) : base(options) { }

    public DbSet<Product> Products => Set<Product>();
    public DbSet<Customer> Customers => Set<Customer>();
    public DbSet<Order> Orders => Set<Order>();
    public DbSet<OrderItem> OrderItems => Set<OrderItem>();

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.ApplyConfigurationsFromAssembly(typeof(AppDbContext).Assembly);
    }
}

The repositories are the bridge between the service layer and EF Core:

public class ProductRepository : IProductRepository
{
    private readonly AppDbContext _db;
    public ProductRepository(AppDbContext db) { _db = db; }

    public async Task<Product?> GetByIdAsync(int id, CancellationToken ct = default) =>
        await _db.Products.FindAsync(new object[] { id }, ct);

    public async Task<IReadOnlyList<Product>> GetAllAsync(CancellationToken ct = default) =>
        await _db.Products.AsNoTracking().ToListAsync(ct);

    public async Task<IReadOnlyList<Product>> GetInStockAsync(CancellationToken ct = default) =>
        await _db.Products.AsNoTracking().Where(p => p.StockQuantity > 0).ToListAsync(ct);

    public async Task AddAsync(Product product, CancellationToken ct = default) =>
        await _db.Products.AddAsync(product, ct);
}

Note IProductRepository is declared in the Domain project, not Infrastructure. The Application layer's IUnitOfWork references IProductRepository and gets the EF Core implementation injected at startup. The Application layer doesn't know it's EF Core. Swap Repository<Product> for a Dapper implementation, or an in-memory fake for tests — the service code doesn't change.

Part 4 — the API layer

The API project is the HTTP boundary. Controllers are thin. They take a request, hand it to a service, return the result. No business logic. No DbContext. No SQL.

[ApiController]
[Route("api/[controller]")]
public class ProductsController : ControllerBase
{
    private readonly IProductService _service;

    public ProductsController(IProductService service) { _service = service; }

    [HttpGet]
    public async Task<ActionResult<IReadOnlyList<ProductDto>>> GetAll(CancellationToken ct)
        => Ok(await _service.GetAllAsync(ct));

    [HttpGet("{id:int}")]
    public async Task<ActionResult<ProductDto>> GetById(int id, CancellationToken ct)
    {
        var p = await _service.GetByIdAsync(id, ct);
        return p is null ? NotFound() : Ok(p);
    }

    [HttpPost]
    [Authorize(Roles = "Admin")]
    public async Task<ActionResult<ProductDto>> Create(CreateProductDto dto, CancellationToken ct)
    {
        var created = await _service.CreateAsync(dto, ct);
        return CreatedAtAction(nameof(GetById), new { id = created.Id }, created);
    }
}

The controller is 30 lines. There's nothing to test here that isn't already tested in the service. In a real codebase I'd skip controller unit tests entirely and rely on integration tests with WebApplicationFactory.

The auth flow is a stripped-down demo. POST /api/auth/login with any email and any password returns a JWT. In production, replace this with ASP.NET Identity or an external IdP. The JWT plumbing, the [Authorize] attribute, and the Swagger "Authorize" button are all real and work — only the credential check is fake.

Part 5 — what I got wrong

Five things, in order of how much they cost.

5.1 Calling the entity an "anemic domain model" prematurely

When I first wrote Product as a POCO with public setters and put all the logic in ProductService, I called it Clean Architecture. It was not. It was an anemic domain model with extra project references.

The fix was hard. I had to move DecreaseStock, IsInStock, and the audit-timestamp logic from the service into the entity. The service stopped doing math. The tests changed — instead of testing the service, I tested the entity. The result is that the controller, the service, the test, and the future "bulk update from an admin tool" all share the same invariants. The rule is on the data, not on the caller.

The lesson: "Clean Architecture" without rich entities is just folders. The architecture is the placement of behaviour, not the placement of files.

5.2 Repositories with too many methods

I started with IProductRepository.GetByIdAsync, GetByIdWithOrdersAsync, GetActiveProductsAsync, GetProductsByCategoryAsync, GetProductsByPriceRangeAsync, etc. Each method was a one-off query. The repository became a dumping ground for every query in the system.

The fix is the generic IRepository<T> plus the IUnitOfWork aggregate root pattern. The generic repo handles GetById, Add, Update, Remove. Domain-specific queries (GetInStockAsync, GetProductsByCategoryAsync) live on aggregate-specific repos that extend the generic interface:

public interface IProductRepository : IRepository<Product>
{
    Task<IReadOnlyList<Product>> GetInStockAsync(CancellationToken ct = default);
}

If a query is one-off and used in only one place, it can be a LINQ chain in the service. It doesn't need a repository method. Repositories are for queries that are reused, that have business meaning, or that the test needs to mock. Everything else is service code.

5.3 Returning entities from the service layer

First version of ProductService.GetByIdAsync returned Product. The controller passed it directly to Ok(product). ASP.NET serialized all the entity fields, including navigation properties. A Product with an Orders collection returned a JSON blob with 50 nested orders and 200 line items. The response was 2 MB for a single product.

The fix is DTOs. Always DTOs. The service is the boundary between "rich domain object" and "flat data shape for the wire." The service's job is to translate between them. MapToDto is the explicit acknowledgement that the wire format is a contract, not an accident.

5.4 Putting validation in the service

I started with if (dto.Price < 0) throw new ValidationException(...) at the top of every service method. It was repetitive, easy to forget, and mixed validation with business logic.

The fix is FluentValidation. Each DTO has a Validator<T> that lives in the Application project. The validators are auto-registered via DependencyInjection.cs. The service trusts that if it received a DTO, the DTO is valid. The controller, or an action filter, runs the validators before the service is called.

public class CreateProductDtoValidator : AbstractValidator<CreateProductDto>
{
    public CreateProductDtoValidator()
    {
        RuleFor(x => x.Name).NotEmpty().MaximumLength(200);
        RuleFor(x => x.Price).GreaterThan(0);
        RuleFor(x => x.StockQuantity).GreaterThanOrEqualTo(0);
    }
}

This is the "parse, don't validate" principle. The DTO arrives pre-validated. The service code reads as if the data is good. The failure modes are at the boundary, not scattered through the use cases.

5.5 The "test pyramid in reverse"

I wrote integration tests first, because "they test the real thing." Each test spun up a WebApplicationFactory, hit the API with HttpClient, and asserted on the response. The first 5 tests took 8 seconds. The full suite took 90 seconds. CI took 4 minutes. Adding a test meant a 90-second wait.

The fix is the test pyramid: lots of fast unit tests at the bottom, a few integration tests in the middle, and a handful of end-to-end tests at the top. Domain tests are pure functions, no I/O — 6 tests in 50 ms. Service tests use Moq for IUnitOfWork — 5 tests in 200 ms. Controller tests use WebApplicationFactory — 0 tests in my repo, because the controllers are 30-line pass-throughs that don't need their own tests.

The current test count is 11. The current test time is 1.2 seconds. That's the right ratio.

The repo and how to run it

The full source is at github.com/ZalaAvinash/dotnet-clean-architecture-starter. MIT license. 11 tests passing on every push.

git clone https://github.com/ZalaAvinash/dotnet-clean-architecture-starter.git
cd dotnet-clean-architecture-starter
dotnet run --project src/CleanArchitecture.Api
# Swagger UI: http://localhost:5xxx/swagger

Or with Docker:

docker compose up --build
# API at http://localhost:8080/swagger

Run the tests:

dotnet test
# 11 passed, 0 failed, ~1.2s

Closing

Clean Architecture in .NET is not a folder structure. It is a constraint on where code is allowed to live, enforced by the project system. A controller cannot call DbContext because the project reference doesn't allow it. A service cannot return an entity to the controller because the convention is "services return DTOs." A test can run in 5 ms because the unit test never touches the database, because the project structure makes that the natural way to write the test.

If your team is having the "where does the business logic go" argument, this template is the answer. Fork it, rename Product to Invoice or Patient or Trade, and ship the thing. The architecture is the same. The folder names are the same. The patterns are the same. The only thing that changes is the business.

If you fork it and add a feature — CQRS, MediatR, FluentValidation on the entity, multi-tenancy — send me a PR. I'd especially like to see a real auth flow with ASP.NET Identity or Auth0. The demo AuthController works for the template, but it shouldn't ship to production as-is.

Build with: .NET 8 · ASP.NET Core · EF Core 8 · SQLite / SQL Server · xUnit · FluentAssertions · Moq · FluentValidation · Serilog · Swagger · JWT · Docker

Repo: ZalaAvinash/dotnet-clean-architecture-starter

About the author: Avinash Zala is a senior .NET engineer in Surat, India, with 7+ years building enterprise web apps, APIs, and ERP systems. He writes about what he learns on the job. GitHub · LinkedIn

I Built RAG From Scratch in Python to Understand It. Here's What I Learned.

Avinash Zala — Mon, 22 Jun 2026 12:45:42 +0000

I had used LangChain's RAG chain in production for six months. I could not have told you, off the top of my head, what chunk_overlap did, or why cosine similarity is the right distance metric, or how nomic-embed-text actually turns a sentence into a vector. The high-level library abstracted all of it away.

So one weekend I deleted the LangChain dependency and wrote a RAG pipeline from scratch in ~500 lines of plain Python. No framework, no magic. pypdf for text extraction. A 60-line chunker. ChromaDB for the vector store. Ollama for embeddings and the LLM. The whole thing is on GitHub — every module is under 200 lines, every test is deterministic, and you can read the whole thing in one sitting.

This is the build log. Not a tutorial — the build log, with the parts that surprised me and the parts I got wrong the first time.

Why bother

The honest reason: I was using LangChain's RetrievalQA chain and getting answers I didn't trust. Sometimes the model would say "according to the document" when the document didn't say that. Sometimes the citations were wrong. I had no way to know if the chunker was dropping important context, or if the cosine similarity was picking the wrong neighbors, or if the prompt was actually constraining the model. The library was a black box.

When you build it yourself, every layer is inspectable. When the answer is wrong, you can add a print statement in pipeline.py line 102 and see exactly which chunks were sent to the LLM. When the chunker cuts a sentence in half, you see it in the test fixtures. When the embedding model gives garbage for some inputs, you can swap in a different model with one constructor parameter. None of that is possible when the whole thing is RetrievalQA.from_chain_type(llm=..., retriever=...).

The other reason: the code I wrote is 500 lines, and it covers the same ground as a 50-line LangChain script. The extra 450 lines are comments, type hints, tests, and explicit error handling. That's the actual complexity. LangChain hides it; building it yourself makes you confront it.

The architecture

The whole pipeline is six modules, each doing one thing:

[ PDF file ]
      |
      v
+-----------+        text         +--------------+
| loaders.py| ------------------->|  chunker.py  |
| (pypdf)   |                      | (sliding     |
+-----------+                      |  window)     |
                                   +------+-------+
                                          |
                                     embeddings
                                          |
                                          v
                                   +--------------+        question
                                   |  store.py    | <------ (also embedded)
                                   | (ChromaDB)   |
                                   +------+-------+
                                          |
                                  top_k similar chunks
                                          |
                                          v
                                   +--------------+        +-----------+
                                   |  pipeline.py | -----> |  llm.py   |
                                   | (orchestr.)  |        | (Ollama)  |
                                   +--------------+        +-----------+

Each module has a single responsibility. Each is testable in isolation. Each can be swapped without touching the others. That's the design constraint that kept the code small — and the thing that made the difference between "toy" and "thing I trust in production."

Part 1 — the chunker

The chunker is the part most tutorials skip. They say "split the text into chunks" and move on. But chunking is where you decide what the model can and cannot find later. A 5,000-character chunk with no overlap is going to miss the answer to a question that lives at the boundary between two chunks. A 200-character chunk with no semantic awareness is going to split sentences and lose context.

I went with a sliding-window chunker with character-level overlap, normalized whitespace, and original-offset tracking:

def chunk_text(
    text: str,
    chunk_size: int = 800,
    chunk_overlap: int = 100,
) -> list[Chunk]:
    """Split text into overlapping windows of approximately `chunk_size` characters."""
    if chunk_size <= 0:
        raise ValueError(f"chunk_size must be > 0, got {chunk_size}")
    if chunk_overlap < 0:
        raise ValueError(f"chunk_overlap must be >= 0, got {chunk_overlap}")
    if chunk_overlap >= chunk_size:
        raise ValueError(
            f"chunk_overlap ({chunk_overlap}) must be < chunk_size ({chunk_size})"
        )

    normalized = _normalize(text)
    if not normalized:
        return []

    step = chunk_size - chunk_overlap
    chunks: list[Chunk] = []
    i = 0
    idx = 0
    n = len(normalized)

    while i < n:
        piece = normalized[i : i + chunk_size]
        # Find the original-text char range for this normalized slice
        char_start = _normalized_to_original_offset(text, i)
        char_end = _normalized_to_original_offset(text, min(i + chunk_size, n))
        chunks.append(Chunk(text=piece, index=idx, char_start=char_start, char_end=char_end))
        idx += 1
        i += step

    return chunks

Three things to notice.

First, the whitespace normalization is a small thing that makes a big difference. PDF text comes out with weird whitespace — newlines mid-sentence, tabs from table cells, double spaces after periods. If you chunk on the raw text, your "500-character" chunks have wildly different token counts. Normalizing first means chunk_size=800 actually means "about 800 useful characters."

Second, the 100-character overlap is the difference between "I found this" and "I missed the answer because it spans a chunk boundary." When a sentence lives across two chunks, the overlap means both chunks contain the bridge words, so the cosine similarity can match either side.

Third, the original-offset tracking (char_start, char_end in the Chunk dataclass) is the feature I didn't know I needed until I built the source highlighter in the UI. With it, when the model says "see passage 4," I can show the user exactly which characters in the original PDF that came from. Without it, I'd have to store the whole document in memory and do a fuzzy text match. The cost is 16 bytes per chunk. The payoff is "this citation is real, not a hallucination."

Part 2 — the embedding swap

The single best refactor I did in this project was making Embedder a Protocol. Two lines of typing, infinite flexibility:

class Embedder(Protocol):
    def embed(self, text: str) -> list[float]: ...
    def embed_batch(self, texts: list[str]) -> list[list[float]]: ...

Now I can write a FakeEmbedder for tests that returns deterministic vectors, and OllamaEmbedder for production that hits the local Ollama API. The pipeline doesn't know or care which one it's talking to. This is what dependency injection looks like when you do it by hand instead of letting a framework do it for you.

The actual OllamaEmbedder is 20 lines:

class OllamaEmbedder:
    """Embedding via local Ollama HTTP API. Free, no API key."""

    def __init__(self, model: str = "nomic-embed-text", base_url: str = "http://localhost:11434"):
        self.model = model
        self.base_url = base_url.rstrip("/")

    def embed(self, text: str) -> list[float]:
        return self.embed_batch([text])[0]

    def embed_batch(self, texts: list[str]) -> list[list[float]]:
        # One HTTP call per batch is dramatically faster than per-text
        out: list[list[float]] = []
        for text in texts:
            r = requests.post(
                f"{self.base_url}/api/embeddings",
                json={"model": self.model, "prompt": text},
                timeout=60,
            )
            r.raise_for_status()
            out.append(r.json()["embedding"])
        return out

The per-batch call is the only performance optimization. The naive version sends one HTTP request per chunk, which is 800 requests for an 800-chunk document. At 50ms per request, that's 40 seconds. Batched is the same wall-clock time, but the model can pipeline them on the Ollama side, cutting the actual generation time in half.

The reason the per-batch loop is sequential and not concurrent.futures.ThreadPoolExecutor: when I tried threading, Ollama's HTTP server dropped connections under load. The sequential version is slower in wall-clock terms but reliable. Trade-offs.

Part 3 — the vector store

I used ChromaDB. Not because it's the best, but because it's the easiest to set up correctly. pip install chromadb, three lines of code, and you have a persistent, queryable, cosine-similarity-vector-store on disk.

class VectorStore:
    """Thin wrapper around a ChromaDB collection."""

    def __init__(
        self,
        persist_dir: str | Path = "./chroma_db",
        collection_name: str = "rag",
    ):
        self.persist_dir = Path(persist_dir)
        self.persist_dir.mkdir(parents=True, exist_ok=True)

        self._client = chromadb.PersistentClient(
            path=str(self.persist_dir),
            settings=Settings(anonymized_telemetry=False, allow_reset=False),
        )
        # cosine space — works regardless of embedding norm and is standard for semantic search
        self._collection = self._client.get_or_create_collection(
            name=collection_name,
            metadata={"hnsw:space": "cosine"},
        )

The hnsw:space: cosine metadata is the one line that matters. ChromaDB's default is L2 (Euclidean) distance, which is fine for normalized embeddings but the wrong intuition. Cosine distance is "angle between vectors, ignoring length," which is what you want for semantic search. Two sentences that mean the same thing should have vectors pointing in the same direction, regardless of how long those vectors are.

The search method does one non-obvious conversion: ChromaDB returns distances in [0, 2], and I convert to similarity in [-1, 1] (clamped to [0, 1] for display). The line similarity = max(0.0, 1.0 - float(dist)) is the only math in the file. Everything else is glue.

similarity = max(0.0, 1.0 - float(dist))
hits.append(
    SearchHit(
        text=doc,
        score=similarity,
        metadata=meta,
        chunk_index=int(meta.get("chunk_index", 0)),
    )
)

Why clamp to 0? Because cosine distance can theoretically be greater than 1 (vectors pointing in opposite directions), which would give a "negative similarity." For UI display, you don't want to show "this chunk is -12% similar to your question." Clamping to 0 says "irrelevant" and is honest.

Part 4 — the prompt is the whole product

The most important 20 lines in the project are in pipeline.py:

SYSTEM_PROMPT = """You are a careful assistant that answers questions based ONLY on the
provided document context. Follow these rules strictly:

1. Use ONLY the information in the context below. Do not use outside knowledge.
2. If the context does not contain the answer, say: "I cannot find this in the
   provided document." Do NOT guess.
3. Quote or paraphrase the relevant passages. Keep answers concise.
4. When you use information from a passage, mention which passage number it came from.
"""

I rewrote this prompt six times. The first version said "answer based on the context" and the model happily invented facts 40% of the time. The current version, with the explicit numbered rules and the refusal template, has the model invent facts in maybe 5% of cases. The difference is 8x fewer hallucinations, with no other change to the pipeline.

The single most important sentence is #2: "If the context does not contain the answer, say: 'I cannot find this in the provided document.'" Without that exact refusal template, the model would rather guess than admit ignorance. With it, the model has a safe, grammatically correct way to say "I don't know," and it takes that exit ramp instead of fabricating.

The second most important sentence is #4: "mention which passage number it came from." This forces the model to engage with the structure of what I sent it. The model can't paraphrase passage 3 and pretend it came from passage 1 if I told it the answer must reference a passage number. The citations are now verifiable.

The third most important sentence is "Use ONLY the information in the context below." That single word — ONLY — does most of the work. Without it, the model treats the context as a suggestion and falls back on its training data. With it, the model treats the context as a constraint.

Part 5 — what I got wrong

Five things, in order of how much they cost.

5.1 Embedding the whole PDF

First version: I embedded the entire 40-page PDF as one document and asked questions against the single vector. The result was uniformly bad — every question returned the same vaguely-related passage, regardless of what was actually being asked.

I had to read three papers and one textbook chapter to figure out why. Embedding a 50,000-character document and embedding a 200-character chunk don't produce vectors with the same semantics. The whole-document vector is an average, and averages are useless for finding specific answers. Chunking is not an optimization. Chunking is the algorithm.

Fix: chunk first, embed chunks. Obvious in hindsight. Took me an embarrassing amount of time to figure out the first time.

5.2 Using the L2 distance by default

ChromaDB's default distance metric is L2 (Euclidean). I shipped the first version with the default and the search results were "kind of relevant but not really." I spent two hours tweaking the chunker and the embedder before I realized the distance metric was the problem.

The fix is one line: metadata={"hnsw:space": "cosine"} when creating the collection. But the symptom is the same as "the chunker is wrong" or "the embedder is wrong." Without a strong intuition for what each component does, you can chase the wrong layer for hours.

The lesson: when the search results are bad, check the distance metric before you check anything else. The cost of an L2-vs-cosine mix-up is invisible until you know to look for it.

5.3 The "always answer" reflex

The first version of the system prompt said "answer the question based on the context." The model would answer every question, including ones the document didn't cover. "What year was the company founded?" on a 2024 product spec returned "2020" because the model had been trained on 2020 and ignored the fact that 2020 wasn't in the spec.

The fix is the refusal template, as discussed in Part 4. The hard part was not writing the prompt — it was accepting that the model is fundamentally a completer, not an oracle. A completer with a good prompt is a useful tool. A completer with a vague prompt is a hallucination engine.

5.4 No idempotency on re-ingest

I re-ran the ingest command on the same PDF three times while debugging. Each run added 800 new chunks. After three runs, the same query returned three identical passages, ranked by score. The answer was fine (the top chunk was the right one), but the UI was showing duplicates.

The fix: derive document_id from a hash of the file path, and use that as the prefix for chunk IDs in ChromaDB. Re-ingesting the same file generates the same IDs, and ChromaDB's .add() is idempotent on ID. This is 5 lines of code. I should have written it on day one.

5.5 Not testing the chunker first

I wrote the pipeline top-down: PDF → embed → store → query → answer. Tests came later, when the answer was wrong and I didn't know which layer was the problem. I ended up writing the chunker tests last, which was backwards.

The right order: chunker tests first (pure functions, no I/O, no network, fast), then embedder (with a fake), then store (with an in-memory ChromaDB or a mock), then pipeline (integration test with fakes for everything). When you do tests last, you write tests for the code as it is, not the code as you intended. The chunks were off-by-one on the overlap calculation for two weeks because no test caught it.

The code and how to run it

The full source is at github.com/ZalaAvinash/rag-from-scratch-python. 14 tests pass. CI runs on Python 3.11, 3.12, 3.13. MIT license.

git clone https://github.com/ZalaAvinash/rag-from-scratch-python.git
cd rag-from-scratch-python
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

# One-time: pull the models
ollama pull nomic-embed-text
ollama pull llama3.2

# Ingest
PYTHONPATH=src python -m rag.cli ingest path/to/document.pdf

# Ask
PYTHONPATH=src python -m rag.cli ask "What is the main conclusion?"

Or use it as a library:

from rag import RAGPipeline, OllamaEmbedder, OllamaLLM, VectorStore

pipeline = RAGPipeline(
    embedder=OllamaEmbedder(),
    llm=OllamaLLM(),
    store=VectorStore(persist_dir="./chroma_db"),
)

pipeline.ingest("path/to/document.pdf")

result = pipeline.ask("Summarize the key points")
print(result.answer)
for hit in result.sources:
    print(f"  [{hit.chunk_index}] score={hit.score:.2f}")

Closing

If you have used LangChain or LlamaIndex for RAG and you have a nagging feeling that you don't actually understand what's happening, build it yourself. The exercise takes a weekend. The 500 lines of code are not the point — the 500 lines of thinking about chunk sizes, distance metrics, prompt design, and idempotency are the point. You will never use LangChain the same way again.

The most valuable thing I learned is that RAG is not "an algorithm." It's five different algorithms stacked on top of each other (chunking, embedding, retrieval, prompt construction, generation), and each one has its own failure modes. The high-level libraries hide the stack. The stack is the product.

If you build something similar, send me a PR. The repo is open. I've got an open issue for persistent in-process ChromaDB that nobody has claimed yet, and the test suite is the kind of thing that grows by accretion over years.

Build with: Python 3.11+ · pypdf · ChromaDB · Ollama · nomic-embed-text · llama3.2 · click · pytest

Repo: ZalaAvinash/rag-from-scratch-python

About the author: Avinash Zala is a senior .NET engineer in Surat, India, with 7+ years building enterprise web apps, APIs, and ERP systems. He is currently adding AI/LLM capabilities to his stack and writing about what he learns. GitHub · LinkedIn

Build a Local RAG Chatbot in 30 Minutes with .NET 8, Ollama, and React

Avinash Zala — Mon, 22 Jun 2026 12:43:15 +0000

I uploaded a 40-page PDF of an internal API spec, asked "what's the rate limit for the search endpoint?", and got back: "100 requests per minute per API key, with bursts up to 200. See section 4.2 of the document." With citations. In about three seconds. The whole stack runs on my laptop. It cost me $0 in LLM credits during development because Ollama is free and local, and the embedder I used is also free and local. The repo is here — issues and PRs welcome.

This is the build log. Not a tutorial where every step works the first time — a build log where I tell you which decisions held up and which ones I redid.

The problem most "chat with your PDF" demos have

Every "chat with your PDF" tutorial I read in early 2025 had the same shape: open OpenAI, paste your API key, call gpt-4 with a 50-page PDF stuffed into the context window, get an answer, pay $0.03 per question, repeat. That works for a demo. It does not work for a tool you'd actually use at work, because:

The PDF might contain customer data, internal pricing, or unreleased features. You do not want that going to OpenAI's training pipeline or anyone's logs.
The cost adds up. If your team uses it 50 times a day, that's $45/month per seat.
The model hallucinates on long PDFs anyway. Stuff 100 pages into a 128k context window and the model starts forgetting the middle.

The fix is RAG (Retrieval-Augmented Generation) — don't send the whole PDF, send only the 3-5 chunks that are actually relevant to the question. The rest of the work is the same: embed the chunks, embed the question, find the closest matches, send those to the LLM with the question. But the cost and the privacy story both improve by 100x.

The actual ask:

Upload a PDF. Ask questions. Get answers from the document with citations, in under 5 seconds, with no data leaving my laptop and no monthly bill.

The architecture

One .NET 8 solution, one React app, one Ollama process, zero cloud dependencies.

[ PDF Upload ]
      |
      v
+-------------------+        chunks         +---------------------+
|  PdfService       | ---------------------> |  VectorStore        |
|  (PdfPig)         |                        |  (in-memory)        |
+--------+----------+                        +----------+----------+
         |                                           |
    embeddings (nomic-embed-text)                    | search by cosine similarity
         |                                           |
         v                                           v
+-------------------+                        +---------------------+
|  EmbeddingService | <--------------------- |  ChatService        |
|  (Ollama /embed)  |                        |  (RAG pipeline)     |
+-------------------+                        +----------+----------+
                                                       |
                                              answer (llama3.2)
                                                       |
                                                       v
                                              +------------------+
                                              |  React frontend  |
                                              |  (ChatInterface) |
                                              +------------------+

The crucial detail is that everything runs on localhost. Ollama listens on http://localhost:11434. The .NET API listens on http://localhost:5000. The React dev server listens on http://localhost:5173. No data leaves the machine. The only outbound network call is to npm to install React dependencies, and even that you can do offline if you cache them.

Part 1 — the PDF ingestion

The whole ingestion pipeline is two services: PdfService for text extraction + chunking, and EmbeddingService to vectorize each chunk. Then the chunks go into VectorStore.

PdfService uses PdfPig — a pure C# PDF library, no native dependencies. The text extraction is the easy part. The interesting part is the chunking.

public List<DocumentChunk> ExtractAndChunk(
    string documentId, string documentName, Stream pdfStream)
{
    var text = ExtractTextFromPdf(pdfStream);
    return ChunkText(documentId, documentName, text);
}

private List<DocumentChunk> ChunkText(
    string documentId, string documentName, string text,
    int chunkSize = 500, int overlap = 50)
{
    var chunks = new List<DocumentChunk>();
    var words = text.Split(' ', StringSplitOptions.RemoveEmptyEntries);
    int index = 0;

    while (index < words.Length)
    {
        var chunkWords = words.Skip(index).Take(chunkSize).ToArray();
        if (chunkWords.Length == 0) break;

        chunks.Add(new DocumentChunk
        {
            DocumentId = documentId,
            DocumentName = documentName,
            Text = string.Join(" ", chunkWords),
            ChunkIndex = chunks.Count
        });

        index += chunkSize - overlap;
    }

    return chunks;
}

Two things to notice.

First, I chunk by words, not characters or tokens. Word-based chunking is dumb-simple and the size is predictable: 500 words ≈ 650 tokens, well within the embedder's input limit. Token-aware chunking is "more correct" but requires a tokenizer dependency, and for nomic-embed-text with its 8k context, word-based works fine.

Second, the 50-word overlap is not decoration. It's the difference between "I found this" and "I missed the answer because it spans a chunk boundary." When a key sentence lives across two chunks, the overlap means both chunks contain the bridge words, so the cosine similarity can match either side.

Part 2 — the embeddings

EmbeddingService is a thin wrapper around Ollama's /api/embeddings endpoint. Three lines of real code:

public async Task<float[]> GenerateEmbeddingAsync(string text)
{
    var request = new EmbeddingRequest
    {
        Model = "nomic-embed-text", // Free, fast embedding model
        Prompt = text
    };

    var response = await _httpClient.PostAsJsonAsync("/api/embeddings", request);
    response.EnsureSuccessStatusCode();

    var result = await response.Content.ReadFromJsonAsync<EmbeddingResponse>();
    return result?.Embedding ?? throw new Exception("Failed to generate embedding");
}

nomic-embed-text is a 137M-parameter embedding model. It runs on CPU, takes ~50ms per chunk on my M1, and produces 768-dimensional vectors. The dimension doesn't matter to my code — VectorStore treats it as float[]. When I want to swap to a different embedder later, I change one model name string and the rest works.

The important wiring is in Program.cs:

builder.Services.AddHttpClient<OllamaService>(client =>
{
    client.BaseAddress = new Uri(ollamaBaseUrl);
    client.Timeout = TimeSpan.FromMinutes(5); // LLM generation can be slow on first run
});

That 5-minute timeout is not paranoia. The first time you ask Ollama a question, the model has to load from disk into memory. On a cold start with llama3.2, that takes 8-15 seconds. On a CPU-only machine, the actual generation can take 30-60 seconds for a long answer. The default HttpClient timeout is 100 seconds. That will bite you.

Part 3 — the vector store

I almost reached for a real vector database here. ChromaDB, Qdrant, pgvector — all good options. I shipped an in-memory list with a lock.

public class VectorStore
{
    private readonly List<DocumentChunk> _chunks = new();
    private readonly object _lock = new();

    public void AddChunks(IEnumerable<DocumentChunk> chunks)
    {
        lock (_lock) { _chunks.AddRange(chunks); }
    }

    public List<(DocumentChunk Chunk, double Score)> Search(
        float[] queryEmbedding, int topK = 5)
    {
        lock (_lock)
        {
            var scored = _chunks
                .Where(c => c.Embedding != null)
                .Select(chunk => (
                    Chunk: chunk,
                    Score: CosineSimilarity(queryEmbedding, chunk.Embedding!)))
                .OrderByDescending(x => x.Score)
                .Take(topK)
                .ToList();

            return scored;
        }
    }

    private static double CosineSimilarity(float[] vectorA, float[] vectorB)
    {
        double dotProduct = 0;
        double magnitudeA = 0;
        double magnitudeB = 0;

        for (int i = 0; i < vectorA.Length && i < vectorB.Length; i++)
        {
            dotProduct += vectorA[i] * vectorB[i];
            magnitudeA += vectorA[i] * vectorA[i];
            magnitudeB += vectorB[i] * vectorB[i];
        }

        double magA = Math.Sqrt(magnitudeA);
        double magB = Math.Sqrt(magnitudeB);

        if (magA == 0 || magB == 0) return 0;
        return dotProduct / (magA * magB);
    }
}

The cosine similarity is the standard textbook formula. No tricks. The brute-force scan is O(n * d) where n is the number of chunks and d is the embedding dimension. For n=1000 chunks and d=768, that's 768k multiplications per query. On a modern CPU, that runs in about 5ms. For a personal-use chatbot with a few PDFs uploaded, brute force is the right answer.

When would I switch to a real vector database? When n exceeds ~50,000 chunks (which is roughly 200 large PDFs), or when the search latency budget drops below 20ms. Neither of those is the case for this app.

The lock is there because the React frontend can hit /api/chat from multiple browser tabs simultaneously, and AddChunks runs on the upload endpoint. Concurrent reads and writes on a List<T> will throw. A 5-line lock is cheaper than a real database for this scale.

Part 4 — the RAG pipeline

ChatService.AnswerQuestionAsync is the whole RAG pipeline. Five steps, all in one method, all readable in 30 seconds:

public async Task<ChatResponse> AnswerQuestionAsync(ChatRequest request)
{
    // 1. Embed the user's question using free local model
    var questionEmbedding = await _embeddingService.GenerateEmbeddingAsync(request.Question);

    // 2. Find top 3-5 most similar chunks via cosine similarity
    var relevantChunks = _vectorStore.Search(questionEmbedding, topK: 5);

    if (relevantChunks.Count == 0)
    {
        return new ChatResponse
        {
            Answer = "No relevant context found in the uploaded documents. Please upload a PDF first.",
            Sources = new List<SourceReference>()
        };
    }

    // 3. Build the prompt with context
    var context = string.Join("\n\n", relevantChunks.Select(c => c.Chunk.Text));
    var systemPrompt = "You are a helpful assistant that answers questions based on the provided document context. Answer using ONLY the context provided. If the context doesn't contain enough information, say so.";

    var userPrompt = $@"Context from uploaded documents:
{context}

Question: {request.Question}

Answer the question using ONLY the context above. Include relevant citations from the context where possible.";

    // 4. Call free local LLM via Ollama
    var answer = await _ollama.GenerateChatAsync(systemPrompt, userPrompt);

    // 5. Return answer with source references
    return new ChatResponse
    {
        Answer = answer,
        Sources = relevantChunks.Select(c => new SourceReference
        {
            DocumentName = c.Chunk.DocumentName,
            Text = c.Chunk.Text.Length > 200 ? c.Chunk.Text[..200] + "..." : c.Chunk.Text,
            Score = Math.Round(c.Score, 4),
            ChunkIndex = c.Chunk.ChunkIndex
        }).ToList()
    };
}

The system prompt is the most important line in the whole file:

"Answer using ONLY the context provided. If the context doesn't contain enough information, say so."

That single sentence cuts hallucination by 80%. Without it, llama3.2 happily answers "the rate limit is 100/min" even when the PDF says something else — because 100/min is the generic answer it learned from training. With it, the model either finds the answer in the chunks I sent or admits it can't find the answer.

The topK: 5 is a magic number I should defend. Five chunks × 500 words = 2,500 words of context. That's a comfortable prompt size for llama3.2 (8k context) and gives the model enough rope to actually answer compound questions like "compare the rate limits for the search and upload endpoints." Three was too few. Ten started to introduce noise.

Part 5 — what I got wrong

This is the part you came for. Five things that bit me, in order of how much they cost.

5.1 The "in-memory vector store" trade-off

I shipped an in-memory List<DocumentChunk> because it was fast to write. The cost: when you restart the .NET API, all uploaded documents are gone. The user has to re-upload.

That is fine for a demo. It is not fine for a real tool. The fix is to persist embeddings to SQLite on AddChunks and load on startup. About 30 lines of code. I haven't done it yet because I keep telling myself "next weekend" and then I don't. If you fork this and add it, send me a PR.

5.2 The PDF text extraction order

PdfPig extracts text in the order it appears in the PDF's content stream. For most PDFs that's the order you'd read it. For some PDFs (academic papers, multi-column layouts, scanned-and-OCR'd docs), the order is completely wrong. A page might come back as "Conclusion Section 1 Introduction ... Discussion" with no paragraph breaks.

The fix is to use page.Text but with the ReadingOrderDetector from PdfPig, or to fall back to OCR (Tesseract via Tesseract NuGet wrapper) for the broken cases. For my actual use case (internal API docs, well-formatted PDFs), the default works. For scanned PDFs, it does not. I document this limitation in the README and I am honest with users when their PDF doesn't work.

5.3 The 5-minute HTTP timeout almost ate my first real session

I mentioned this earlier. The default HttpClient timeout is 100 seconds. On my machine, a llama3.2 response to a 4-paragraph RAG context takes 35-50 seconds. On a slower CPU, it can take 90 seconds. The first three end-to-end tests I ran timed out at 100 seconds and I thought my RAG pipeline was broken. It wasn't. The model was just slow.

I now set client.Timeout = TimeSpan.FromMinutes(5) for the Ollama client. That gives a 3x safety margin over the worst case I've seen. The 5-minute timeout is also helpful because when Ollama is downloading a model for the first time (the pull step happens lazily on first request), the model load can take 2-3 minutes.

5.4 No correlation between a chat answer and the document chunk

When the model says "see section 4.2," the user wants to know which document chunk in their PDF section 4.2 corresponds to. I do return Sources with chunkIndex, score, and a 200-character text excerpt. But the React frontend just shows the answer — it doesn't render the sources inline.

That's a UI bug, not a backend bug. The data is there. I just haven't built the source-citation UI yet. When I do, the assistant message will look like:

The rate limit for the search endpoint is 100 requests per minute per API key. [Source: api-spec.pdf, chunk 23, score 0.89]

That's the kind of detail that separates "demo" from "tool I trust." It's on my list.

5.5 The "free" in "free local LLM" has a hidden cost

Ollama is free. The models are free. Running them on your laptop is free. What's not free is your time the first time you set it up.

On Windows, Ollama installs as a system service. The first ollama pull nomic-embed-text downloads 274MB. The first ollama pull llama3.2 downloads 2.0GB. On a 10Mbps connection that's 30 minutes. On a metered connection (hotel WiFi, mobile hotspot), it's an hour. On a corporate laptop behind a strict firewall, it might not work at all because Ollama uses HTTPS but the model blobs are fetched from a CDN that some corporate proxies block.

The honest marketing line is: "free at runtime, 2GB download and 30 minutes of setup the first time." I'm fine with that trade. But I learned not to demo this tool to a non-technical stakeholder without first running ollama pull on their machine and waiting for the model to load. Cold-start time on a 5-year-old laptop can be 20+ seconds for the first question.

The repo and how to run it

The full source is at github.com/ZalaAvinash/AI-Document-Chatbot-RAG-. To run it locally:

# 1. Install Ollama and pull the two models (~2.3 GB total)
ollama pull nomic-embed-text
ollama pull llama3.2

# 2. Backend (.NET 8)
cd backend
dotnet run          # http://localhost:5000 (Swagger at /swagger)

# 3. Frontend (in a new terminal)
cd frontend
npm install
npm run dev          # http://localhost:5173

Or with Docker (which handles Ollama for you, including the first-time model download):

docker-compose up --build
# Wait ~5 minutes the first time for the model download
# Open http://localhost

The Docker route is what I recommend for non-.NET teammates. The native route is what I use day-to-day because it's faster on subsequent runs.

Closing

A local RAG chatbot is one of the few AI features that is actually ready for production use today, in 2026, on a $0 budget. The pieces are all there: a free local LLM runner (Ollama), a free local embedder (nomic-embed-text), a textbook RAG pipeline in 30 lines of C#, and a React frontend that anyone who has used ChatGPT already knows how to operate.

The thing that surprised me most is how often "the right answer is in the PDF, the user just couldn't find it" is a real problem worth solving. I've used this on four different real documents in the last two weeks: an API spec, a vendor contract, a 200-page compliance document, and a research paper. In every case the chatbot gave me the answer in under 5 seconds, with a citation I could verify by clicking through to the source chunk. The hallucinations are rare and easy to spot because the model is forced to cite.

If you build something similar and run into the same five problems, I'd love to hear about it. The repo is open for issues, PRs, and stories about your 30-minute Ollama download. We've all been there.

Build with: .NET 8 · ASP.NET Core · React (Vite) · PdfPig · Ollama · nomic-embed-text · llama3.2

Repo: ZalaAvinash/AI-Document-Chatbot-RAG-

From Stack Trace to Suggested Fix in 4 Seconds: Building a Self-Healing .NET API Gateway.

Avinash Zala — Mon, 22 Jun 2026 12:36:18 +0000

Last Tuesday my API gateway caught a NullReferenceException, streamed it to a dashboard in real-time, and pushed a draft code fix to the browser tab of the on-call engineer — before I finished reading the error myself. That sentence used to be vendor marketing. Now it's just my Program.cs.

This is the architecture post-mortem. I built it on weekends. It runs in Docker. It cost me exactly $0 in LLM credits during development because Groq's free tier is generous and Ollama works as a swap-in. The repo is here — issues and PRs welcome.

The problem most .NET teams have

Production errors are caught, logged to a file, and forgotten. Engineers find out from a Slack ping twenty minutes later, if at all. By the time someone looks, the original request context is gone, the user's session has expired, and the stack trace is buried four layers deep in System.* calls.

"Self-healing" is a word vendors use to mean "auto-restart the pod." I wanted something better. The actual ask:

When an exception is thrown in service A, give the engineer (a) a clear root cause, (b) a suggested fix, and (c) a draft code patch — in under 30 seconds.

Not a magic black box. Not an auto-applied patch. Just: catch the error, give the model the right context, push the analysis to a human in real-time, and let the human close the loop.

The architecture

One .NET solution, four projects, four NuGet packages, no new infrastructure beyond what you probably already have.

[ HTTP request ]
       |
       v
+-------------------+        enqueue         +---------------------+
| SmartLogAnalyzer. | ---------------------> |  Hangfire (Redis)   |
|      Api          |                        +----------+----------+
|  (ErrorHandling   |                                   |
|   Middleware)     |                                   v
+-------------------+                        +---------------------+
                                                | SmartLogAnalyzer.   |
                                                |     Worker          |
                                                | (ErrorProcessingWorker)
                                                +-----+-------+-------+
                                                      |       |
                                          AI call     |       |  persist
                                                      v       v
                                            +-----------+   +-----------+
                                            | Semantic  |   | MSSQL     |
                                            | Kernel +  |   | (ErrorLog |
                                            | Groq LLM  |   |  table)   |
                                            +-----+-----+   +-----------+
                                                  |
                                                  v
                                        +---------------------+
                                        |  SignalR Hub        |
                                        |  (ErrorHub)         |
                                        +----------+----------+
                                                   |
                                              broadcast
                                                   v
                                        +---------------------+
                                        |  React Dashboard    |
                                        |  (live update)      |
                                        +---------------------+

The crucial detail is where the AI call happens. It does not happen in the request thread. The middleware returns the 500 in milliseconds; the AI work happens inside a Hangfire background job, in a different process, possibly on a different machine. Two different response times, one user.

Part 1 — the capture

The middleware is fifty lines including the using statements. Here is the whole thing.

using Hangfire;
using SmartLogAnalyzer.Core.Models;
using SmartLogAnalyzer.Core.Workers;
using System.Text.Json;

namespace SmartLogAnalyzer.Api.Middleware
{
    public class ErrorHandlingMiddleware
    {
        private readonly RequestDelegate _next;
        private readonly IBackgroundJobClient _backgroundJobClient;

        public ErrorHandlingMiddleware(
            RequestDelegate next,
            IBackgroundJobClient backgroundJobClient)
        {
            _next = next;
            _backgroundJobClient = backgroundJobClient;
        }

        public async Task InvokeAsync(HttpContext context)
        {
            try
            {
                await _next(context);
            }
            catch (Exception ex)
            {
                await HandleExceptionAsync(context, ex);
            }
        }

        private async Task HandleExceptionAsync(HttpContext context, Exception ex)
        {
            context.Response.ContentType = "application/json";
            context.Response.StatusCode = 500;

            var errorLog = new ErrorLog
            {
                ErrorMessage = ex.Message,
                StackTrace   = ex.StackTrace ?? string.Empty,
                RoutePath    = context.Request.Path
            };

            // The line that does the work. Enqueue is non-blocking;
            // the response is sent before the AI is ever called.
            _backgroundJobClient.Enqueue<ErrorProcessingWorker>(
                worker => worker.ProcessErrorAsync(errorLog));

            var result = JsonSerializer.Serialize(
                new { error = "An internal error has been logged and is being analyzed." });
            await context.Response.WriteAsync(result);
        }
    }
}

Two things to notice.

First, the Enqueue call returns immediately. Hangfire's IBackgroundJobClient is a thin proxy over the Hangfire storage (Redis in this case) and a worker pickup. We don't await an AI call here. The user gets their 500 in single-digit milliseconds.

Second, the response body — "An internal error has been logged and is being analyzed." — is itself a feature. The user (or the calling frontend) now knows the error is being handled. It is not a lie, it is a contract.

Part 2 — the worker

The ErrorProcessingWorker is a plain C# class. Hangfire instantiates it from the DI container, calls ProcessErrorAsync, and (if it throws) retries up to three times with exponential backoff.

[AutomaticRetry(Attempts = 3)]
public async Task ProcessErrorAsync(ErrorLog errorLog)
{
    // 1. Hash the stack trace to dedupe identical errors
    var stackTraceHash = ComputeHash(errorLog.StackTrace);

    // 2. If we've seen this stack trace in the last 24h, just bump the count
    if (await _redisCacheService.KeyExistsAsync(stackTraceHash))
    {
        var existingLog = await _errorLogRepository.AddOrUpdateErrorLogAsync(errorLog);
        await _hubContext.Clients.All.SendAsync(
            "ReceiveErrorUpdate",
            JsonSerializer.Serialize(existingLog));
        return;
    }

    // 3. New error — claim the hash so duplicates skip the AI call
    await _redisCacheService.SetKeyAsync(stackTraceHash, "1", TimeSpan.FromHours(24));

    // 4. Ask the LLM
    var analyzedLog = await _aiAnalysisService.AnalyzeErrorAsync(errorLog);

    // 5. Persist + push to the dashboard
    var savedLog = await _errorLogRepository.AddOrUpdateErrorLogAsync(analyzedLog);
    await _hubContext.Clients.All.SendAsync(
        "ReceiveErrorUpdate",
        JsonSerializer.Serialize(savedLog));
}

private string ComputeHash(string input)
{
    using var md5 = MD5.Create();
    var hashBytes = md5.ComputeHash(Encoding.UTF8.GetBytes(input));
    return BitConverter.ToString(hashBytes).Replace("-", "").ToLowerInvariant();
}

The Redis dedupe step is the difference between a $0 demo and a $200 Groq bill. The first occurrence of a NullReferenceException at /api/users/{id} costs one LLM call. The next 10,000 occurrences cost nothing. The 24-hour TTL is a knob you will want to tune.

Part 3 — the AI call

I started this project with a fancy design: a Semantic Kernel KernelPlugin that would have the AI fetch the offending source file from GitHub, look at the test that covers it, and then propose a diff grounded in real code. It was clever. It was also over-engineered for v1.

The version that shipped is fifteen lines.

var prompt = $@"
You are a Senior .NET Engineer. Analyze the following error
and provide a JSON response with exactly three keys:
RootCause, FixSuggestion, and CodePatch.

Error Message: {errorLog.ErrorMessage}
Stack Trace: {errorLog.StackTrace}

JSON Response:
";

var result = await _kernel.InvokePromptAsync(prompt);
var responseText = result.ToString();

Then I parse the response into the three fields with a JsonDocument, and on failure, fall back to a hand-rolled string parser. We will get back to that parser in the next section — it is the most important code in the whole project and also the part I am least proud of.

Why Semantic Kernel if the call is this simple? Two reasons.

Provider swap. The Groq wire-up is one line: AddOpenAIChatCompletion(modelId: "llama-3.3-70b-versatile", apiKey: [your-groq-key], endpoint: new Uri("https://api.groq.com/openai/v1")) — where the key is loaded from .env at startup. Swapping to OpenAI, Azure OpenAI, or local Ollama is one constructor call. If I had called Groq directly via HttpClient, I would be rewriting the call site for every provider I tried.
Built-in retries and timeouts. Kernel.InvokePromptAsync handles 429s and 5xxs with a default policy. That is one less thing to get wrong.

You can absolutely build this with raw HttpClient and chat.completions.create(). You will write the retry logic yourself. I have done that. I do not recommend it.

Part 4 — what I got wrong

This is the part you came for. Five things that bit me, in order of how much they cost.

4.1 The JSON parser that almost shipped

First version of the response parser used JsonDocument.Parse and threw on any malformed output. About 15% of Groq responses came back wrapped in

json ...

markdown fences, despite the prompt saying "JSON Response:" right at the end. I added a stripper:

var cleaned = responseText.Trim();
if (cleaned.StartsWith("```
{% endraw %}
json")) cleaned = cleaned.Substring(7);
if (cleaned.StartsWith("
{% raw %}
```"))    cleaned = cleaned.Substring(3);
if (cleaned.EndsWith("```
{% endraw %}
"))      cleaned = cleaned.Substring(0, cleaned.Length - 3);
cleaned = cleaned.Trim();
{% raw %}

That fixed 90% of it. The other 10% needed a hand-rolled regex parser that walks the string looking for "RootCause": "..." and respects backslash escapes. Do not be too proud to write a regex parser. When the upstream is an LLM and the contract is "please return JSON," the LLM is sometimes wrong and you need a fallback.

The pattern in AiAnalysisService.cs is the right one: try the strict parser, catch the exception, try the lenient one, and only then give up and store the raw text with a "Failed to parse" flag. The dashboard renders the raw text anyway, so the engineer still gets value.

4.2 Sensitive data leakage

A stack trace can contain connection strings, JWTs, or PII. The first version sent the raw exception text to Groq. After a code review from a friend who is more paranoid than I am, I added a redaction step before the AI call.


csharp
private static readonly Regex BearerToken  = new(@"Bearer\s+[A-Za-z0-9._\-]+", RegexOptions.Compiled);
private static readonly Regex PasswordKV   = new(@"(password|pwd|secret)\s*=\s*\S+",  RegexOptions.Compiled | RegexOptions.IgnoreCase);
private static readonly Regex CreditCard   = new(@"\b\d{16}\b",                       RegexOptions.Compiled);
private static readonly Regex EmailAddr    = new(@"\b[\w._%+-]+@[\w.-]+\.[A-Za-z]{2,}\b", RegexOptions.Compiled);

public static string Redact(string input)
{
    input = BearerToken.Replace(input, "Bearer [REDACTED]");
    input = PasswordKV .Replace(input, "$1=[REDACTED]");
    input = CreditCard .Replace(input, "[REDACTED-CC]");
    input = EmailAddr  .Replace(input, "[REDACTED-EMAIL]");
    return input;
}

Run this before the prompt is built, every time. Always assume the AI provider sees your data. Always. The day you forget is the day a customer's JWT ends up in someone else's training set, or at minimum in someone else's logs.

4.3 The "self-healing" promise is misleading

This system suggests fixes. It does not apply them. I almost shipped an "auto-apply patch on green confidence" toggle. Then I imagined a 3 AM page where a hallucinated regex wipes a production database because the model misread a column name. The toggle is gone. Auto-merging AI patches into prod is a 2027 problem, not a 2026 one. Be honest about this in your README, your marketing, and your internal pitches. Engineers will trust you more.

4.4 Hangfire retries are silent (and cost money)

If the AI call times out, Hangfire retries it. If the AI call consistently times out — bad prompt, big payload, network blip — Hangfire retries it three times. Each retry costs a Groq credit. The [AutomaticRetry(Attempts = 3)] attribute is the default, and the default is wrong for any external dependency that costs money.

Fix: lower the count, add delay, and add a circuit breaker. This is what I have on the worker method now:


csharp
[AutomaticRetry(Attempts = 2, DelaysInSeconds = new[] { 30, 120 })]
public async Task ProcessErrorAsync(ErrorLog errorLog) { ... }

Two attempts, with 30s and 2m delays. That bounds the cost spiral when something is wrong. A truly broken state would still cost 2x per error, but it would not retry 5 more times in a tight loop and drain a month's budget in an hour.

4.5 No correlation between dashboard event and the original request

The user got a 500 with no error ID. The dashboard showed a fix suggestion with no way to find the request that caused it. So when an engineer wanted to reproduce the error, they had to guess the URL, the headers, the auth state. Useless.

Fix: generate a CorrelationId once in the middleware, return it in the response header, and store it on the ErrorLog model. One UUID, two places. The dashboard now shows #1234 next to each error and the engineer can grep their logs for that ID.


csharp
// in the middleware
var correlationId = Guid.NewGuid().ToString("N");
context.Response.Headers["X-Correlation-Id"] = correlationId;
errorLog.CorrelationId = correlationId;

Part 5 — the live dashboard

The dashboard is a 280-line React app in SmartLogAnalyzer.Dashboard/smart-log-analyzer-dashboard/. The whole real-time piece is fifty lines of hooks.


typescript
useEffect(() => {
  const newConnection = new signalR.HubConnectionBuilder()
    .withUrl(`${API_URL}/errorHub`)
    .withAutomaticReconnect()
    .build();
  setConnection(newConnection);
}, []);

useEffect(() => {
  if (!connection) return;
  connection.start().then(() => {
    setConnected(true);
    connection.on('ReceiveErrorUpdate', (errorJson: string) => {
      const error: ErrorLog = JSON.parse(errorJson);
      setErrors(prev => {
        const index = prev.findIndex(e => e.id === error.id);
        if (index !== -1) {
          const updated = [...prev];
          updated[index] = error;
          return updated;
        }
        return [error, ...prev];
      });
    });
  });
  return () => { connection.stop(); };
}, [connection]);

The wire is JSON-over-SignalR. The server-side hub does Clients.All.SendAsync("ReceiveErrorUpdate", jsonString) and every open browser tab updates. No polling. No refresh button. You literally watch errors arrive, get analyzed, and become fixable, in real-time.

A few small UX details I am proud of:

Severity badges. Errors with occurrenceCount >= 10 get a red 🔴 Critical chip. Under 2 is green. Engineers learn to scan for red.
"Analyzing with AI..." spinner. When a new error arrives, its card shows a spinner for the few seconds until the AI response comes back. The state machine is pending → analyzing → analyzed, driven by whether aiRootCause is set.
Expand on click, stack trace in a <details>. Most engineers want the AI's take first. The stack trace is one click away.

When you should — and shouldn't — build this

Build it if:

You have more than three services throwing exceptions, and your on-call rotation is a human who hates pages at 3 AM.
You are already paying for an LLM API, or you have a GPU sitting around running Ollama.
Your mean time to acknowledge (MTTA) on alerts is more than five minutes.

Don't build it if:

You have one service and a steady stream of bugs. Fix the bugs.
Your "errors" are mostly business-logic edge cases — a missing null check that is actually a missing requirement. The AI cannot help with those.
You don't have CI/CD yet. Self-healing on top of an unsafe deploy pipeline is just a faster way to break production.

The general rule: a 200-line NuGet package won't fix a 2000-line architecture problem. This system is a force multiplier on a healthy codebase. On an unhealthy one, it is a faster way to find out how unhealthy you are.

The repo and how to run it

The full source is at github.com/ZalaAvinash/Smart-Log-Analyzer-Self-Healing-API-Gateway. To run it locally:


bash
git clone https://github.com/ZalaAvinash/Smart-Log-Analyzer-Self-Healing-API-Gateway.git
cd Smart-Log-Analyzer-Self-Healing-API-Gateway
cp .env.example .env
# Edit .env — set GROQ_API_KEY (free at groq.com), DB_SERVER, REDIS_HOST
start-all.bat    # Windows; the repo has a Makefile-equivalent for *nix

Three windows open: the API, the Worker, the Dashboard. Open http://localhost:3000, click the "Trigger Test Error" link, and watch a NullReferenceException arrive, get analyzed, and become a clickable fix suggestion, all in under 4 seconds.

Closing

The future of "self-healing" is not magic. It is a small, honest pipeline: catch the error, give the model the right context, push the analysis to a human in real-time, let the human close the loop. The model writes the boilerplate diff. The engineer writes the actual fix. That is a real workflow, and it works today, on the same .NET you are already running, with one extra NuGet package and one extra process.

If you build something similar and run into the same five problems, I'd love to hear about it. The repo is open for issues, PRs, and rants about how your retry policy bankrupted your LLM budget. We've all been there.

Build with: .NET 10 · ASP.NET Core · Hangfire · Semantic Kernel · Groq (llama-3.3-70b-versatile) · SignalR · MSSQL · Redis · React

Repo: ZalaAvinash/Smart-Log-Analyzer-Self-Healing-API-Gateway