DEV Community: Ben Witt

Stop Loading Your Entire Instruction System Into Every Session

Ben Witt — Wed, 17 Jun 2026 08:00:00 +0000

Most people talk about better prompts. Hardly anyone talks about what happens before every prompt: the instructions the assistant loads into the context before the actual work begins.

Depending on the system, you pay for that in different ways: input tokens, latency, reduced available context, or simply more noise in the assistant's active instructions. Even if the financial cost is partly reduced through prompt caching, the cognitive cost remains: the assistant still has to operate inside a larger instruction environment.

At some point, my setup had become one single, constantly growing instruction file. System structure, assistant personality, workflows, session rules, special cases: everything was in one file. And everything was loaded into the context on every interaction, no matter whether I was solving a complex task or just asking a quick question.

That is roughly like starting every phone call by reading the entire employee handbook before getting to the actual topic.

The Actual Problem

A monolithic instruction file has two costs that become unpleasant when combined:

The baseline gets expensive.

Most of the file is irrelevant to the concrete task. Still, it sits in the active context. Depending on the system, that means token cost, latency, less room for the real task, or all of them at once.
The signal-to-noise ratio drops.

The more rules and special cases you add, the more the currently relevant part gets diluted. More context does not automatically mean more competence.

Both scale in the wrong direction: the more mature your setup becomes, the heavier and less precise it gets, as long as everything lives in one file.

The Insight

Not all instructions are needed all the time.

I always need the assistant's personality and basic operating principles. I only need the exact structure of my project system when I actually navigate through it. I only need the session-end rules at the end of a session, and never before that.

A writing task does not need filesystem navigation rules.

A quick reasoning task does not need session-close workflows.

A debugging session does not need publishing guidelines.

If that is true, it makes no sense to keep everything loaded permanently.

The Architecture

I split the one large file into a lean entry point plus specialized modules:

.config/
├── instructions.md   -> compact entry point, always loaded
├── persona.md        -> personality, tone, behavior
├── structure.md      -> system structure, only relevant for navigation
└── workflows.md      -> session workflows, only relevant when needed

The main instruction file is now intentionally small. It contains the minimum that really has to be present in every session, plus clear references: which module is responsible for what, and when it should be loaded.

The detail modules are not active by default. They are accessible, but they only become part of the context when the task requires them.

That distinction matters. The full instruction set is not magically present for free. It is only available if the assistant knows that a module exists, recognizes that it is relevant, and loads it at the right moment.

So modularization does not mean: same context, lower cost.

It means: smaller baseline, with more responsibility placed on routing and loading.

How Loading Works

In my setup, the entry point acts as a router. It does not contain all detailed rules. It contains short loading rules such as:

If the task involves navigating the project system,
load structure.md before answering.

If the task involves ending or reviewing a session,
load workflows.md before making recommendations.

If the task is a quick standalone question,
do not load additional modules unless needed.

This is simple, but it is also the fragile part of the system. If the entry point is vague, the assistant may fail to load the right module. If it is too broad, it loads too much again and the benefit disappears.

The quality of the entry point determines the quality of the whole architecture.

The Result

In my setup, the baseline token load per session dropped by around 60-80%.

I measured this by comparing the files that were previously loaded unconditionally at session start with the files that are now loaded unconditionally. The important number is not the total size of all available instructions. It is the size of the always-loaded baseline.

Before modularization:

Always loaded:
instructions.md
persona.md
structure.md
workflows.md

Baseline load:
~4,800 tokens

After modularization:

Always loaded:
instructions.md
persona.md

Baseline load:
~1,450 tokens

Reduction:
69.8%

The full instruction set still exists, but it is no longer active by default. It becomes active only when needed.

Why This Works

The trick is not compressing individual instructions. The trick is separating baseline load from on-demand load.

Baseline load: what is loaded in every session. This is where the savings matter most, because this cost is paid repeatedly.
On-demand load: what is only relevant in specific situations. This can be large and detailed, as long as it is loaded only when it actually matters.

So you are not optimizing the total size of your instructions. You are optimizing which part of them must always be present. And that is surprisingly little.

Prompt caching can reduce the financial cost of repeated baseline instructions in some systems. But it does not remove the context-budget cost, the latency implications in every environment, or the signal-to-noise problem. A cached irrelevant instruction is still an irrelevant instruction in the active instruction set.

What It Costs

This is not a free lunch.

Indirection:
The assistant sometimes has to take an extra step to load the right module. That is slightly slower and creates the risk that the right module is not loaded.
Routing errors:
If the assistant does not recognize that a task requires a module, it may answer with incomplete instructions. This is the main operational risk.
Maintenance:
More files mean more places that can drift apart. If the entry point promises something that no longer exists in a module, you have a silent consistency problem.
Rule conflicts:
Modules can contradict each other or the entry point. You need a precedence rule: general instructions define the default, specialized modules override them only within their domain, and explicit user instructions still have to be handled according to the system's hierarchy.
Onboarding:
An outsider first has to understand the loading logic before the system becomes readable. A single file is trivial to understand.

The real trade-off is this:

You reduce baseline cost, but you give up permanent availability. You move complexity from runtime context into structure, routing, and maintenance.

When It Is Worth It

It is worth it if:

your instruction set is large and has grown over time
you start sessions frequently and baseline cost is noticeable
clearly separable situations exist, such as navigation, session-end handling, publishing, coding, or special workflows
your environment allows the assistant to load additional instruction files reliably

It is not worth it if your entire setup fits into a few hundred tokens anyway. In that case, modularization is premature optimization: you trade real simplicity for imagined efficiency.

It is also not worth it if the assistant cannot reliably access the modules when needed. A small always-loaded file plus inaccessible detail files is not an architecture. It is missing context with extra steps.

Conclusion

The biggest lever for instruction cost is rarely a better prompt. It is the question of what you force the assistant to carry into every interaction, and what it should only load when needed.

Separate baseline load from on-demand load. Keep the entry point small and turn it into a precise router. Leave the details where they only create cost when they are actually needed.

In my case, that meant 60-80% less baseline load. But the important part is not just the savings. It is the trade-off: less permanent context, more deliberate loading.

That is the architecture I actually want. Not less instruction, but less unnecessary instruction in the room at the wrong time.

The Most Dangerous Bias of Your AI Assistant Is That It Agrees With You

Ben Witt — Wed, 10 Jun 2026 08:00:00 +0000

We talk a lot about hallucinations. But there is another failure mode we should take just as seriously: AI assistants are optimized to be helpful, polite, and cooperative. Over a longer session, that can quietly turn into agreeableness.

In a system that is supposed to help you think, this is not a cosmetic problem. It is the core problem. An assistant that agrees with you simply because you are the one asking is worthless as a sparring partner. It confirms your bad ideas just as readily as your good ones.

I call this sycophancy drift, and I have added a reflective layer to my knowledge system to detect it.

What Sycophancy Drift Is

At the beginning of a session, the assistant still pushes back. You make a suggestion, and it gives you three reasons against it. Good.

Twenty messages later, the tone is different. You suggest something, and suddenly everything is “a good point,” “absolutely reasonable,” or “a strong idea.” Not because your ideas have necessarily become better, but because the conversation context has increasingly conditioned the assistant toward agreement.

To be precise: the model is not learning in the training sense. Its weights are not changing during the session. But the transcript becomes part of the active context, and that context can reward a pattern. If agreement keeps the conversation moving, the assistant may drift toward more agreement.

That is the drift: a gradual shift from honest evaluation to confirmation.

This is not speculation. Anthropic’s own research (Perez et al. 2022; Sharma et al. 2023) shows that RLHF training systematically rewards agreement: both human raters and preference models prefer convincingly written sycophantic answers over correct ones a non-negligible fraction of the time. The disposition is built in at training time. A long conversation does not create it - it amplifies it.

Why You Do Not Notice It

Because it feels good. That is the whole trick.

Hallucinations stand out because they are wrong. Sycophancy does not stand out because it is pleasant. You feel confirmed, you move forward, the session feels productive, and nobody tells you that the quality of disagreement has quietly dropped.

Message 5:
Assistant:
"I see three major risks."

Message 35:
Assistant:
"That's a very strong idea."

Drift warning:
Criticism frequency dropped from 4 objections
per proposal to 0.8 objections per proposal.

(Schematic illustration; the actual reports are qualitative)

This is exactly why the assistant cannot reliably correct this in real time by itself. It is inside the same conversational pressure. You need an outside view, or at least a retrospective one.

The Idea: A Reflective Layer at the End of the Session

In my setup, the assistant does not rewrite its own rules automatically. Instead, I use a dedicated reflective layer after a session has ended.

The process is intentionally simple:

At the end of a session, the analysis layer is explicitly triggered.
It reads the transcript of the entire session.
It compares the session against an explicit, versioned rule set.
It writes a structured proposal with four sections:

New rules: what this session produced as lessons.
Confirmed rules: what proved useful again.
Drift warnings: where the assistant agreed too much or weakened its criticism.
Recommendation: what should be added to the rule set.

The decisive point is this: it is only a proposal. It is written for human review. Not a single rule changes automatically.

What the Layer Actually Flags

The layer does not look for politeness. Politeness is not the problem.

It looks for disappearing resistance:

fewer objections over time
softer risk language
repeated validation phrases
missing alternative paths
ignored counterarguments
praise replacing evaluation
earlier rules being quietly weakened
decisions accepted without checking trade-offs

The goal is not to make the assistant negative. The goal is to preserve useful friction.

A good thinking partner should not disagree for sport. But it should notice when the session has become too smooth.

Three Design Decisions That Make the Difference

1. Human in the loop, not out of caution, but out of logic.

It would be tempting to let the analysis layer write its findings directly into the rule set. That would be the mistake. You would let a system that is prone to conversational drift rewrite its own anti-drift rules.

Separating proposal from adoption is not a comfort feature. It is the safety mechanism that makes the whole concept meaningful.

There is also a simple technical reality in my setup: the assistant has no autonomous background process and no unchecked memory update across sessions. Every change is conscious, triggered, and reviewable. This is not a limitation I want to bypass. It is the property that keeps the system honest.

2. Importance and frequency are two different axes.

Every rule has two independent dimensions: an importance value and a frequency counter.

A rule can occur rarely and still be critical. If I mix both dimensions together, the rare but important rule will eventually disappear because it looks statistically insignificant. That is why critical rules are protected from being archived, no matter how rarely they are triggered.

3. A maximum of five new rules per session.

Without a limit, the system overfits to a single conversation. One intense session could flood the rule set with special cases that may never matter again.

The upper limit forces selection: what was truly a lesson, and what was just noise?

Five is not a magic number. It is calibrated to my review capacity: a proposal I cannot review in five minutes is a proposal I will eventually stop reading and an unreviewed proposal pipeline is worse than none.

The Honest Part: The Recursion Problem

The obvious weakness is this: I am using the same kind of model that drifted to detect its own drift. Can an assistant that tends to agree really expose its own agreement reliably?

The honest answer is: not perfectly.

A retrospective layer can itself become performative. It may learn to produce the kind of criticism the user expects, without actually identifying meaningful drift. It may become another agreeable ritual: “Here are your drift warnings,” even when the analysis is shallow.

But the approach is still useful for two reasons.

First, checking a finished transcript against an explicit checklist is a narrower task than generating helpful answers in the middle of a live conversation. Evaluation is not the same as participation.

Second, the result is not trusted blindly. A human reviews it. The layer does not have to detect drift perfectly. It only has to make the pattern visible enough to question.

Additional costs:

No real-time feedback. Drift during the current session is detected afterwards, for the next run.
Review effort. Reading proposals and deciding what to adopt is work.
False confidence. A reflective layer can look more objective than it really is.

That last point matters. The layer is not a guarantee. It is a mirror.

The industry attacks the same problem one level deeper: Anthropic’s persona-vector research identifies activation patterns associated with sycophancy and steers models away from them during training. That is the right fix at the model level. But it cannot reach the workflow level — no lab can review whether your assistant pushed back on your architecture decision yesterday. That layer is yours to build.

When It Is Worth It

If you use the assistant as a thinking tool, where honest disagreement is part of the value, then drift detection is not a nice extra. It is one of the conditions under which the tool works at all.

If you only use the assistant for clearly defined execution tasks, where agreement does not matter much, you probably do not need this.

But if the assistant helps shape your decisions, your architecture, your writing, your strategy, or your beliefs, then you should care about how much resistance disappears over time.

Conclusion

The labs are working on this at the training level, evals, steering, character training. What is largely missing is the workflow level: per-user, per-session drift detection that you control.

The most pleasant bias may also be the most dangerous one: an assistant that agrees with you feels like a good assistant while it slowly stops being useful as a thinking partner.

The solution is not simply a better model version. It is a layer that looks back after the work is done, names the drift, and leaves the decision with you.

The assistant should not only help you move faster. It should also preserve the friction that keeps your thinking honesty.

Part II will follow later

Transitive Dependency – Challenges and Approaches in .NET

Ben Witt — Wed, 12 Feb 2025 09:00:00 +0000

Imagine you are developing a .NET application that runs perfectly at first. Suddenly, however, unexpected runtime errors occur—even though your main project does not explicitly include any additional packages. How can this phenomenon be explained? Often, the cause is a transitive dependency: a library such as Entity Framework Core is silently included in your project via another NuGet package. In complex application scenarios, this issue can lead to version conflicts or even hard-to-diagnose runtime errors like the MissingMethodException. Without targeted management of these dependencies, the effort required for error resolution can become significant.

1. Basics and Problem Statement

1.1 Transitive References vs. Transitive Dependencies

Transitive References:

In a multi-layered solution structure, for example, Project A directly references Project B, which in turn uses Project C. In this way, Project A has an indirect (transitive) reference to Project C—even if it is not explicitly listed in the .csproj file.
Transitive Dependencies:

With external NuGet packages, the problem appears when they are brought in by a referenced package and do not show up in the main project’s .csproj file. For instance, if Project A uses a package (e.g., PackageX) that internally references EF Core in a specific version, then EF Core is included transitively—even though Project A never explicitly added it.

Isn’t it remarkable how quickly such subtle dependency chains can emerge in large projects without being noticed?

1.2 A Typical Scenario in .NET

In many .NET applications, Entity Framework Core (EF Core) is used as the data access library. Even if the main project does not directly reference EF Core, it can be silently included as a transitive dependency via another utility or data access package. Initially, everything seems to work smoothly—EF Core provides a stable API. However, when updating the .NET framework or integrating new features, version conflicts may occur that eventually lead to hard-to-diagnose runtime errors.

2. Detailed Solution Approaches

To prevent the unwanted propagation of packages, two main approaches are available: the targeted restriction of individual packages using PrivateAssets="all" or the complete deactivation of transitive project references with <DisableTransitiveProjectReferences>true</DisableTransitiveProjectReferences>. Both approaches are critically examined below.

2.1 `PrivateAssets="all"`

By adding the attribute PrivateAssets="all" in the PackageReference, you prevent the respective library from being passed on to other projects in the dependency hierarchy. This way, the package—and its transitive dependencies—remain available only within the current project.

Advantages:

Granularity: You decide on a package-by-package basis whether a dependency should be inherited.
Avoidance of Side Effects: Problematic libraries are not automatically transferred to subordinate projects.

Disadvantages:

Increased Configuration Effort: If another project needs the same library, it must be explicitly referenced again.

Example:

<ItemGroup>
  <PackageReference Include="SomeDataAccessPackage" Version="1.2.3" PrivateAssets="all" />
</ItemGroup>

Could it be that in complex environments the administrative effort for such package-specific configuration outweighs the benefits?

2.2 DisableTransitiveProjectReferences

With the global setting true in the .csproj file, you prevent the automatic propagation of all transitive project references. Every project must then explicitly list all the required packages.

Advantages:
• Increased Transparency: All dependencies are explicitly visible and controllable.
• Prevention of Unintended Inclusions: No “hidden” libraries are automatically added to the project.

Disadvantages:
• Higher Maintenance Effort: Explicitly listing all dependencies can create a significant administrative overhead, especially in larger projects.
• Risk of Omissions: Essential dependencies might accidentally be overlooked, leading to compile-time or runtime errors.

Example:


<PropertyGroup>
  <DisableTransitiveProjectReferences>true</DisableTransitiveProjectReferences>
</PropertyGroup>

Given the additional effort, is it really practical to explicitly declare all dependencies?

2.3 Comparison of Both Approaches

Criterion	PrivateAssets="all"	DisableTransitiveProjectReferences
Granularity	Package-specific control	Global deactivation – all dependencies must be explicitly referenced
Maintenance Effort	Requires individual adjustments per project	Higher maintenance effort due to separate management of all references
Clarity	Can lead to opaque dependency chains in complex structures	Clear separation, but increases the risk of overlooking essential packages
Flexibility	High flexibility in isolated control of individual packages	Strict control, but less adaptability

3. Practical Example: Abstraction and Interface Encapsulation

Consider a specific application: a .NET Standard library named Common encapsulates various helper methods and tools and internally includes a data access package that brings EF Core as a transitive dependency. In this case, it is advisable not to expose EF Core-specific classes in the public API; instead, these should be abstracted through interfaces.

Example Implementation:


// In Common
public interface IDataService
{
    IEnumerable<string> GetData();
}

internal class EfDataService : IDataService
{
    public IEnumerable<string> GetData()
    {
        // Internal use of EF Core
        return new List<string> { "Data from EF Core" };
    }
}

public static class DataServiceFactory
{
    public static IDataService CreateService()
    {
        return new EfDataService();
    }
}

Configuration in Common.csproj:


<ItemGroup>
  <PackageReference Include="SomeDataAccessPackage" Version="1.2.3" PrivateAssets="all" />
</ItemGroup>

This approach ensures that EF Core remains exclusively available internally—the main application only interacts with the interface without needing to know the underlying implementation.

4. Advanced Strategies and Alternative Approaches

In addition to the configuration options discussed above, other strategies can help minimize version conflicts and the issues of transitive dependencies:
• Binding Redirects:
Especially in older .NET Framework applications, binding redirects can help resolve conflicts between different versions of the same library.
• Modular Architectures:
Separating the application into clearly defined modules allows for targeted isolation of dependencies. This makes it easier to control unwanted side effects.

Should we not always question whether a focus solely on .csproj configurations in highly complex systems is truly the optimal solution?

5. Summary and Outlook

Transitive dependencies represent a serious issue in complex .NET projects. Seemingly insignificant libraries like EF Core can—when included via other NuGet packages—lead to significant version conflicts and runtime errors. Consistent separation of dependencies and the targeted use of measures such as PrivateAssets="all" or offer practical solutions. It is essential to weigh whether the administrative effort is justified by the benefits and how long-term maintainability can be ensured.

By using abstractions, explicit references, and additional measures such as binding redirects, developers can not only create a more stable codebase but also reduce ongoing maintenance efforts. Regularly reviewing dependency hierarchies using tools like dotnet list package --include-transitive or NDepend is indispensable.

Is it not ultimately the task of every developer to continually question whether the current architecture still meets the growing demands of modern applications?

I think so….!

Efficient Debugging and Precise Logging in C#: Using Caller Attributes

Ben Witt — Wed, 29 Jan 2025 17:01:53 +0000

How often do you wish, while debugging a complex application, to instantly see where in the code an error was triggered? Often, you are left with basic error messages that lack valuable context or exact code lines. This is where the C# attributes [CallerMemberName], [CallerFilePath], and [CallerLineNumber] come into play.

These attributes automatically provide information about the calling method’s name, the physical file path, and the line number in the source code when a method is called. This enables precise error tracking and logging, which is useful both during debugging and in production logging. Especially when troubleshooting large codebases, they prove to be indispensable tools for quick and targeted analysis.

Basics of the Attributes

The attributes [CallerMemberName], [CallerFilePath], and [CallerLineNumber] are applied directly to method parameters. The compiler automatically replaces the arguments with the corresponding values from the call context without any extra effort from the developer:
• [CallerMemberName] provides the name of the calling method.
• [CallerFilePath] gives the full path to the file where the method call is located.
• [CallerLineNumber] provides the line number of the call within that file.

The syntax is straightforward. A classic example:

public void LogInfo(
    string message, 
    [CallerMemberName] string memberName = "",
    [CallerFilePath] string filePath = "", 
    [CallerLineNumber] int lineNumber = 0)
{
    Console.WriteLine($"[{memberName}] {message} (File: {filePath}, Line: {lineNumber})");
}

Here, the parameters must have default values (e.g., = "" or = 0) so that the compiler can insert the values automatically. When you call the method without specifying these optional parameters, they are substituted automatically.

Example Method for Capturing Call Stack Information


public void ProcessData([CallerMemberName] string caller = "",
                        [CallerFilePath] string path = "",
                        [CallerLineNumber] int line = 0)
{
    Console.WriteLine($"Called by: {caller}, File: {path}, Line: {line}");
    // Further data processing code...
}

In this simple scenario, you can already gain initial insights into the call context and source code position. This meta-information is extremely valuable both for debugging and for later evaluations in operation.

Code Examples for Practical Applications

a) Simple Examples

Imagine you are running a library or an API to manage borrowing processes. Everywhere important actions take place—such as registering a book, registering new users, or handling requests—you want to create log entries. You could use a helper class for this:

public static class Logger
{
    public static void LogOperation(
        string operation, 
        [CallerMemberName] string memberName = "",
        [CallerFilePath] string filePath = "",
        [CallerLineNumber] int lineNumber = 0)
    {
        // For example, log to a file or a database
        Console.WriteLine($"Operation: {operation} in {memberName}, File: {filePath}, Line: {lineNumber}");
    }
}

Every time you call Logger.LogOperation("BookBorrowed"), the log provides much more information than if you had to add it manually.

b) More Complex Example with a Base Class

In larger projects, it is advisable to centralize logging and error handling routines in an abstract base class. The following example shows an abstract BaseRepository that catches errors and automatically uses the attributes for meaningful error messages:


public abstract class BaseRepository
{
    protected void LogError(Exception ex, 
                            [CallerMemberName] string memberName = "",
                            [CallerFilePath] string filePath = "", 
                            [CallerLineNumber] int lineNumber = 0)
    {
        Console.WriteLine($"Error in {memberName} at {filePath} (Line {lineNumber}): {ex.Message}");
    }
}

public class BookRepository : BaseRepository
{
    public void AddBook(Book book)
    {
        try
        {
            // Simulated database operation
            throw new InvalidOperationException("Database error while adding a book.");
        }
        catch (Exception ex)
        {
            LogError(ex);
            throw;  // Re-throw the error to continue the stack
        }
    }
}

public class LibraryService
{
    private readonly BookRepository _repository = new BookRepository();

    public void Execute()
    {
        try
        {
            _repository.AddBook(new Book { Title = "C# Deep Dive" });
        }
        catch (Exception ex)
        {
            Console.WriteLine("Error in the service layer: " + ex.Message);
            throw;
        }
    }
}

// Application
var service = new LibraryService();
service.Execute();

This approach allows complete tracking of the call stack. When an error is triggered in BookRepository, the LogError method precisely logs the context: the member name (AddBook), the file path, and the line number. At the same time, the error is re-thrown so that the higher layer (LibraryService) can perform further actions, such as separate logging or sending an alert.

c) Tracking a Complete Stack

In complex multi-layered architectures—consisting of service layers, repository layers, and database connections—it is crucial to trace the source of an error without gaps. The above example demonstrates this flow:
1. The service layer calls AddBook in BookRepository.
2. The repository layer throws an exception due to a database error, logs details using the caller attributes, and re-throws the error.
3. The service layer catches the error and can handle it accordingly.

With this technique, precise logging of each relevant layer is achieved, ensuring that no information about the actual origin is lost.

Practical Application with a Library Management System

In library software, where daily borrowings, reservations, returns, or new registrations are logged, the caller attributes are used optimally:
1. Logging API Errors: If an HTTP request fails, the log immediately provides information about the error location in the code (method, file, line).
2. Logging Database Operations: In dynamic environments, faulty SQL queries or transaction conflicts can quickly lead to confusing errors. Thanks to the caller attributes, it is possible to determine exactly where these conflicts were triggered, whether in the repository or service layer.

Such practical insights into the system not only shorten debugging times but also improve the overall maintainability of the application.

Limitations and Best Practices

Although logging file and line details is very helpful during development, it is important to consider potential drawbacks:
• Performance Considerations: With frequent log calls, excessive capturing of caller information can quickly bloat logs and impact application performance. Therefore, in production environments, it should be carefully considered where detailed logs are truly necessary.
• Security Aspects: Revealing file paths may be undesirable in some cases, such as in security-critical applications or externally shared logs.
• Reusability: To keep maintenance effort low, it is recommended to encapsulate the attributes in base classes or utility methods, as shown in the example (BaseRepository). This avoids redundant code and ensures consistent error messages across the entire project.

With a sensible logging concept and thoughtful filtering mechanisms, the level of detail can be flexibly controlled depending on the environment (development, staging, production).

Conclusion

The attributes [CallerMemberName], [CallerFilePath], and [CallerLineNumber] offer significant benefits for debugging, logging, and error tracing in C# projects. Their versatile applications—from simple logs in the development environment to complex error tracking in multi-layered architectures—make them essential tools in modern application development.

So why not start implementing more precise error logging and more efficient debugging in your own projects right away? The implementation is straightforward, while the benefits are enormous—especially when errors in production systems need to be quickly isolated and resolved. With targeted use, thoughtful structuring, and a bit of pragmatism, a new level of transparency in error analysis is achieved, significantly easing the developer’s daily work.

A Critical Look at Cancellation Management in .NET Applications

Ben Witt — Wed, 29 Jan 2025 17:01:36 +0000

The CancellationToken and the related types provided in .NET offer a central and effective foundation for carefully canceling tasks, threads, or asynchronous processes. However, the question arises whether this concept truly provides sufficient flexibility and stability in all application scenarios. But how often is this feature actually used in practice, and what risks come with improper handling? This article explains how the CancellationToken works and offers concrete recommendations for its efficient use. Additionally, potential pitfalls are highlighted and critically examined.

Definition and Purpose of CancellationToken

The CancellationToken is a structure from the Task Parallel Library (TPL) that is used to signal an ongoing operation that a cancellation request has been made. In a software architecture increasingly based on asynchronous and parallel processes, the CancellationToken plays a central role: it allows computationally intensive processes or long-running tasks to be terminated early, conserving resources and ensuring an improved user experience.

Why is this construct indispensable? In a modern .NET application, numerous operations can run in parallel or asynchronously. Without coordinated cancellation and resource release, there is a risk that certain tasks continue running and block system resources, even though they are no longer needed.

How It Works and Basic Concepts

Cancellation control is managed by the CancellationTokenSource class. It acts as the central control unit and provides methods such as Cancel(), which sends a cancellation signal to all linked CancellationTokens.

Resource Management with IDisposable

A frequently underestimated aspect is that CancellationTokenSource implements the IDisposable interface. If this is not taken into account, it can easily lead to resource leaks—especially in scenarios where a large number of CancellationTokenSource objects are created in parallel. Why should one neglect implementing clean resource management?

The recommendation is to always use the using keyword to enforce automatic resource release:

using (var cts = new CancellationTokenSource())
{
    // Code to execute the operation
}

Failing to call the Dispose() method can leave unmanaged resources in memory, ultimately affecting the application’s stability.

Methods to Cancel an Operation

There are several approaches to sending a cancellation request to an ongoing operation:
1. Synchronous Cancellation
The cancellation request is triggered immediately. This means the current method actively checks the CancellationToken and reacts right away.
2. Asynchronous Cancellation
The request is performed in the background without immediately blocking the calling code’s flow.
3. Delayed Cancellation
A predefined time span is used after which the operation is automatically terminated. This is especially helpful when a user does not want to wait further, such as in network or I/O-intensive tasks.

A concrete example of this is setting a timeout:

var cts = new CancellationTokenSource(TimeSpan.FromSeconds(10));

After this time span elapses, a cancellation request is triggered, ensuring that an operation does not run beyond the set limits. But is it always sensible to set a rigid timeout without being able to flexibly respond to external events?

Monitoring Cancellation Requests

Polling

A common practice is to regularly check the CancellationToken:

while (!token.IsCancellationRequested)
{
    // Continue the operation
}

When such loops are implemented in computationally intensive areas, the frequency of checks can significantly affect performance. The question arises: How often should the check be performed to recognize a cancellation promptly without excessively burdening execution time?

Alternatively, the ThrowIfCancellationRequested() method can be used, which throws an OperationCanceledException if a cancellation request has been made:

token.ThrowIfCancellationRequested();

This approach is usually cleaner but can also lead to unexpected side effects if the exception is not caught appropriately.

Callback Registration

In scenarios where regular polling is impractical or resource-intensive, a callback can be registered to execute automatically when a cancellation request arrives. For example, if a user is downloading a large file and cancels the process, not only should the download threads stop, but any temporary files created should also be deleted to free up space and avoid inconsistencies.

token.Register(() => 
{
    Console.WriteLine("Operation canceled.");
    // Additional cleanup, e.g., deleting temporary files
});

This method has the advantage of calling cancellation actions specifically without the ongoing operation having to constantly check for the cancellation signal. However, callback registration can lead to messy code flow in complex scenarios when multiple cancellation cases and cleanup actions need to be considered.

Linking Multiple CancellationTokens

What happens when an application needs to meet several potential cancellation conditions at the same time? For example, an operation might need to respond to both a user event (like closing a window) and a timeout simultaneously. The CancellationTokenSource class provides the CreateLinkedTokenSource method for this purpose:

var linkedTokenSource = CancellationTokenSource.CreateLinkedTokenSource(token1, token2);

The created instance combines multiple CancellationTokens into a new token. Once any of the involved tokens is canceled, the combined token is also triggered. This simplifies the implementation of complex cancellation logic and improves readability. However, caution is necessary: incorrect linking can cause operations to be canceled too early or not at all if the relationships between tokens are not clearly defined.

Best Practices for Handling CancellationToken

To ensure smooth cancellation management, the following points should be considered:
1. Respect the Passed Token
Avoid ignoring the CancellationToken. In every relevant method, actively check it or use an appropriate mechanism (e.g., callback).
2. Properly Release Resources
Always call the Dispose() method on CancellationTokenSource, preferably using the using block to prevent memory leaks.
3. Avoid Unnecessary Sources
Only create a new CancellationTokenSource if passing an existing CancellationToken is insufficient. Otherwise, you risk an unclear codebase and unnecessary resource consumption.
4. Proper Error Handling
Catch the OperationCanceledException and handle it meaningfully. Silently swallowing the exception can lead to hard-to-trace error patterns, as cancellation operations behave “invisibly.”

Additional Aspects and Critical Consideration

Use Cases for Timeouts

While the use of a timeout has already been mentioned, in a highly connected world, variable external factors such as fluctuating network bandwidths or different devices can quickly render a rigid timeout obsolete. Wouldn’t it make sense to take a closer look at adaptive timeout management?

Consider scenarios where the system responds to environmental parameters and dynamically adjusts timeouts. For example, a microservice that reduces wait times for certain requests under high load or grants more generous timeouts with a good connection could significantly enhance user satisfaction. Load balancers and reverse proxies also play an important role here, as they can capture both the context and the load of the respective services.

In short, a purely time-based approach without context can lead to misjudgments and, at worst, prematurely cancel important operations. In the age of cloud computing and highly variable environments, it’s worthwhile to consider context-dependent timeout management with circuit breakers or retry mechanisms.

Depth of Example Implementations

Although the article uses concise code snippets (such as the using statement), a more in-depth presentation of more complex application examples would be helpful. Especially in microservice architectures or when accessing databases, the layered cancellation logic often becomes apparent in practice.

A realistic example could cover the following aspects:
• Multiple CancellationTokens originating from different sources (e.g., user inputs, timeouts, overload signals).
• A database-heavy operation accessing an external resource that, upon interruption, must not only reset local but also database transactions.
• Logical separation of cancellation handling at the service level and repository level to ensure a clear separation of concerns and minimize side effects.

Such comprehensive implementations would allow developers to recognize and avoid common pitfalls—such as forgetting to call Dispose() or improperly catching OperationCanceledException—in a realistic context.

More Comprehensive Error Handling

The article already emphasizes the importance of catching OperationCanceledException. But how specifically should the error situation be handled in complex systems where multiple components respond to the same token?

A possible scenario could look like this: A user-initiated cancellation request is recognized, but a downstream sub-component does not respond in time or blocks while requesting resources. Partial failures occur, which might only appear late in the logs and are difficult to attribute to a specific cause. In such cases, structured exception handling is needed, considering the following points:
• Targeted Logging: Every cancellation request should appear in the logs, ideally with context information (e.g., which sub-component responded, which actions were interrupted).
• Separation of Regular Exceptions and Cancellation Cases: An OperationCanceledException does not indicate a classic error state but a deliberate cancellation. However, the interplay of multiple CancellationTokens can create unforeseen side effects, making a consistent and well-documented error strategy essential.
• Fallback Mechanisms: In safety-critical or highly available environments, a cancellation should not lead to an unstable overall system but support orderly fallback procedures (e.g., graceful shutdown).

Especially in distributed architectures where multiple services and applications interact, a well-documented error handling and logging process is indispensable to make cancellation processes both traceable and controlled.

Conclusion

The CancellationToken is a central component in the development of modern, reactive, and resource-efficient .NET applications. However, its correct use requires a certain level of attention: underestimating the importance of clean resource management or ignoring cancellation signals in the code can lead to errors and performance problems almost by default.

Is it really acceptable to forgo this central cancellation mechanism or implement it only half-heartedly? Those who value scalability and user satisfaction will clearly answer no. Consistently following the described best practices leads to robust and controlled cancellation management, enhances application performance, and improves the user experience equally. At the same time, a critical look at advanced concepts such as adaptive timeout management, comprehensive implementation examples, and structured error handling is recommended to make the use of CancellationToken even more effective and secure.

Asynchronous HTTP Requests – Explained Simply

Ben Witt — Wed, 29 Jan 2025 17:01:17 +0000

Imagine a waiter taking your order and waiting in the kitchen until your food is ready. During this time, the waiter cannot serve other guests. This is how a synchronous HTTP request works – it blocks the process.

An asynchronous HTTP request, on the other hand, allows the waiter to serve other guests while your food is being prepared. Once your dish is ready, the waiter brings it to you without others being delayed. Sounds efficient, right?

Why Are Asynchronous Requests So Important?

In daily life, we encounter asynchronous HTTP requests all the time, for example:
• WhatsApp: Messages are loaded while you’re using other functions.
• Instagram: Images and videos are loaded in the background.
• Netflix: The next episode is preloaded while you’re still watching.
• Online Shopping: Products are filtered and sorted in the background.

What Does Idempotence Mean in This Context?

Idempotence is an essential concept in web development, particularly for HTTP requests. It describes the property of an operation that delivers the same result no matter how many times it is performed. This principle plays an important role in the reliability and repeatability of requests – especially with asynchronous HTTP requests, as they might be sent multiple times due to network failures or timeouts.

Examples of idempotent HTTP methods:
• GET: Repeatedly fetching the same resource always returns the same data (unless the resource changes).
• PUT: Overwriting a resource with the same data does not change the final state.
• DELETE: Deleting a resource has no effect if the resource has already been removed.

In contrast, POST is not idempotent because it often creates new resources or changes states, which can lead to inconsistent behavior when executed multiple times.

A Simple Introduction in C#

Here is an example of fetching weather data in C#, which also considers idempotence:

public class WetterService
{
    private readonly HttpClient _httpClient;

    public WetterService()
    {
        _httpClient = new HttpClient();
        _httpClient.BaseAddress = new Uri("https://api.wetter.de/");
    }

    public async Task<WetterDaten> HoleWetterAsync(string stadt)
    {
        try
        {
            var antwort = await _httpClient.GetAsync($"wetter/{stadt}");

            if (!antwort.IsSuccessStatusCode)
            {
                return new WetterDaten { Fehler = "Ups! Da ist etwas schiefgelaufen." };
            }

            var json = await antwort.Content.ReadAsStringAsync();
            return JsonSerializer.Deserialize<WetterDaten>(json);
        }
        catch (Exception)
        {
            return new WetterDaten { Fehler = "Sorry, wir konnten das Wetter nicht abrufen." };
        }
    }
}

Note: The use of the HTTP-GET method ensures idempotence. Even with multiple requests, the server’s state remains unchanged.

Practical Example: Online Shop

A typical controller processing asynchronous and idempotent requests:

[ApiController]
[Route("api/produkte")]
public class ProduktController : ControllerBase
{
    private readonly IProduktService _produktService;

    public ProduktController(IProduktService produktService)
    {
        _produktService = produktService;
    }

    [HttpGet]
    public async Task<ActionResult<List<Produkt>>> HoleAlleProdukte()
    {
        var produkte = await _produktService.HoleAlleProdukteAsync();
        return Ok(produkte);
    }

    [HttpGet("{id}")]
    public async Task<ActionResult<Produkt>> HoleProdukt(int id)
    {
        var produkt = await _produktService.HoleProduktAsync(id);

        if (produkt == null)
        {
            return NotFound("Produkt nicht gefunden");
        }

        return Ok(produkt);
    }

    [HttpDelete("{id}")]
    public async Task<IActionResult> LoescheProdukt(int id)
    {
        var erfolgreich = await _produktService.LoescheProduktAsync(id);

        if (!erfolgreich)
        {
            return NotFound("Produkt nicht gefunden oder bereits gelöscht.");
        }

        return NoContent();
    }
}

Idempotence in detail: Both GET and DELETE are idempotent here. Even with multiple executions, the results remain consistent, enhancing the system’s stability.

Benefits of Asynchronous and Idempotent HTTP Requests
1. Better User Experience: The app remains responsive and does not freeze.
2. More Efficient Resource Utilization: The server can handle other tasks during wait times.
3. Increased Reliability: Repeated requests do not lead to inconsistent behavior thanks to idempotence.
4. Scalability: Asynchronous processing reduces bottlenecks, especially with parallel requests.

Advanced Use Case: Parallel Loading

Sometimes it is necessary to load multiple resources simultaneously:

public class DashboardService
{
    private readonly IBenutzerService _benutzerService;
    private readonly IBestellungService _bestellungService;

    public async Task<DashboardDaten> HoleDashboardDatenAsync()
    {
        var benutzerTask = _benutzerService.HoleStatistikenAsync();
        var bestellungenTask = _bestellungService.HoleStatistikenAsync();

        await Task.WhenAll(benutzerTask, bestellungenTask);

        return new DashboardDaten
        {
            BenutzerStats = await benutzerTask,
            BestellungStats = await bestellungenTask
        };
    }
}

Common Pitfalls and Solutions
1. Forgotten await: Without await, the code remains blocking and loses its asynchronous behavior.
2. Neglecting Error Handling: Network issues are common, so plan for appropriate error handling.
3. Not Setting Timeouts: Avoid endless wait times by implementing timeout rules.
4. Ignoring Idempotence: Non-idempotent requests can lead to hard-to-trace errors.
5. Lack of Tests: Asynchronous code is prone to subtle bugs and should be thoroughly tested.

Conclusion

Asynchronous HTTP requests are the foundation of modern, responsive applications. By adhering to idempotence principles, repeated requests do not produce unexpected results. Together, they not only enhance the user experience but also create robust and scalable systems. However, their proper implementation requires careful planning and execution.

WPF Application with Plugin Architecture

Ben Witt — Tue, 15 Oct 2024 08:00:00 +0000

In modern software architectures, plug-in systems offer a flexible and modular way of extending software without changing the core of the application. A specific WPF project will be used to show how a plug-in architecture can be implemented to create loosely coupled and extensible components. The focus is on the interaction between the main application, plug-ins and the use of a central event aggregator for communication between the components.

Overview over the structure

The plug-in interface

Central to any plug-in system is the definition of a clear interface that all plug-ins must implement. This creates consistency and ensures that the main application and the plug-ins can interact seamlessly with each other. In our example, this is achieved through the IPlugin interface:

namespace MyPluginInterface
{
  public interface IPlugin
  {
    string Name { get; }
    IEnumerable<Type> ViewTypes { get; } 
    IEnumerable<Type> ViewModelTypes { get; }
  }
}

This interface specifies that each plug-in provides a name, a list of views (ViewTypes) and the associated ViewModels (ViewModelTypes). This ensures that all plug-ins can be uniformly integrated into the application.

Implementation of a plug-in

A typical plug-in in this system implements the IPlugin interface. In the following example, the plug-in provides two views and the corresponding ViewModels:

using MyFirstPlugin.ViewModels;
using MyFirstPlugin.Views;
using MyPluginInterface;

namespace MyFirstPlugin
{
  public class MyFirstPlugin : IPlugin
  {
    public string Name => "My First Plugin with Multiple Views";

    public IEnumerable<Type> ViewTypes => new List<Type>
    {
        typeof(MyPluginView),
        typeof(MyPlugin1View)
    };

    public IEnumerable<Type> ViewModelTypes => new List<Type>
    {
        typeof(MyPluginViewModel),
        typeof(MyPlugin1ViewModel)
    };
  }
}

This plug-in defines two views (MyPluginView, MyPlugin1View) and their associated ViewModels (MyPluginViewModel, MyPlugin1ViewModel). This makes it possible to define various UI components within the plug-in that can be integrated into the main application.

Dynamic loading and registration of plug-ins

One of the central functions of a plug-in system is the dynamic loading of plug-ins at runtime. This is done by searching for and loading assemblies that implement the plug-in interface. The following code in the main application shows how this is achieved:

private void LoadPlugins(ContainerBuilder builder)
{
  var pluginPath = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "Plugins");
  var pluginFiles = Directory.GetFiles(pluginPath, "*.dll");

  foreach (var pluginFile in pluginFiles)
  {
    var assembly = Assembly.LoadFrom(pluginFile);
    var pluginTypes = assembly.GetTypes().Where(t => typeof(IPlugin).IsAssignableFrom(t) && !t.IsInterface && !t.IsAbstract);

    foreach (var pluginType in pluginTypes)
    {
      var plugin = (IPlugin)Activator.CreateInstance(pluginType);

      foreach (var viewType in plugin.ViewTypes)
      {
        builder.RegisterType(viewType).AsSelf().InstancePerDependency();
      }

      foreach (var viewModelType in plugin.ViewModelTypes)
      {
        builder.RegisterType(viewModelType).AsSelf().InstancePerDependency();
      }

      builder.RegisterInstance(plugin).As<IPlugin>();

      // Registers the views and ViewModels
      builder.RegisterViewsAndViewModels(assembly);
    }
  }
}

Here, a directory is searched for plug-ins that are stored as DLLs. Each plug-in is loaded dynamically and the views and ViewModels defined in it are registered with Autofac in the dependency injection container.

Automatic registration of views and ViewModels with Autofac

We use an extended function of Autofac for efficient registration of views and ViewModels:

public static class AutofacExtensions
{
  public static void RegisterViewsAndViewModels(this ContainerBuilder builder, Assembly assembly)
  {
    builder.RegisterAssemblyTypes(assembly)
           .Where(t => t.Name.EndsWith("View") && typeof(FrameworkElement).IsAssignableFrom(t))
           .OnActivated(args =>
           {
             var viewType = args.Instance.GetType(); 
             var viewModelTypeName = viewType.Name.Replace("View", "ViewModel");
             var viewModelType = viewType.Assembly.GetType(viewType.Namespace + "." + viewModelTypeName); 

             if (viewModelType == null)
             {
               viewModelType = assembly.GetTypes().FirstOrDefault(t => t.Name == viewModelTypeName);
             }

             if (viewModelType != null)
             {
               var viewModel = args.Context.Resolve(viewModelType);
               ((FrameworkElement)args.Instance).DataContext = viewModel;
             }
           })
           .AsSelf()
           .InstancePerDependency();

    builder.RegisterAssemblyTypes(assembly)
           .Where(t => t.Name.EndsWith("ViewModel"))
           .AsSelf()
           .InstancePerDependency();
  }
}

This method registers all Views and ViewModels of a plug-in based on naming conventions. Views that end with 'View' are registered and the associated ViewModels are also identified and loaded into the DI container.

Communication between plug-ins via the EventAggregator

In complex applications, the plug-ins must communicate with each other without creating direct dependencies in order to ensure modularity. In our project, this is made possible by the EventAggregator. Here is a concrete example from the MyFirstPlugin:
Plugin 1: Send message:

public class MyPlugin1ViewModel
{
    private readonly IEventAggregator _eventAggregator;

    public ICommand MyCommand { get; }

    public MyPlugin1ViewModel(IEventAggregator eventAggregator)
    {
      _eventAggregator = eventAggregator;
      MyCommand = new RelayCommand(_ => OnButtonClick());
    }

    private void OnButtonClick()
    {
      var msg = new SpecialEvent("Hello from the other view internal my first plugin");
      _eventAggregator.Publish(msg);
    }
}

In this example, the MyPlugin1ViewModel sends a special event, SpecialEvent, with a message when the user presses a button.

Plugin 2: Receive message
The second plug-in can subscribe to this event and process the message:

public class MySecondPluginViewModel : IHandle<SpecialEvent>
{
    private readonly IEventAggregator _eventAggregator;

    public MySecondPluginViewModel(IEventAggregator eventAggregator)
    {
        _eventAggregator = eventAggregator;
        _eventAggregator.Subscribe(this);
    }

    public void Handle(SpecialEvent message)
    {
        // Process message
        Console.WriteLine(message.Content);
    }
}

In this example, MySecondPluginViewModel receives the SpecialEvent and processes its content in the handle method. This ensures that the two plug-ins can communicate via the EventAggregator without being directly dependent on each other.

Conclusion

A plug-in architecture offers a flexible and scalable solution for extending an application. In our example, this architecture is realised through the combination of Autofac for dependency injection and an event aggregator for communication between the components. Such systems are ideal for scenarios in which an application needs to be dynamically extended with new functionalities without changing the core of the application.

plugIn architecture

HealthChecks

Ben Witt — Tue, 08 Oct 2024 08:00:00 +0000

From time to time it happens that the connection to the database fails or the services or containers crash. This can lead to a considerable investigation effort to find the reasons why these problems occur, which can be facilitated by the use of “HealthChecks”.

ASP.NET provides a convenient and effective approach to implementing this functionality.

To achieve this, the following nuggets need to be installed:

To include the following items in the Program.cs file:

builder.Services.AddHealthChecks();  
...  
...  
//Now the HealthChecks should be mapped to the current endpoint  
app.MapHealthChecks("/_health");

This would provide us with the foundation to check the health of our application. Currently, no checks are performed, so our application is considered to be in a healthy state.

Upon launching the app, initially, only the Swagger site is loaded and no other content is visible. Let’s simply update the address to:

To extend our HealthCheck with a custom check, we will create a new class called “MyFirstHealthCheck” with the following content:

public class MyFirstHealthCheck : IHealthCheck  
{    
  public Task<HealthCheckResult> CheckHealthAsync(HealthCheckContext context, CancellationToken cancellationToken = default)    
  {      
    return Task.FromResult(HealthCheckResult.Healthy("This could be a service"));    
  }  
}

This class serves the purpose of returning a “HealthCheckResult” with the status “Healthy”.

We have reached a point where we need to include this check in our Program.cs file. When we call this check, the situation looks a little different. Initially, the appearance remains unchanged. However, if we set a breakpoint in the source code, it will be triggered when we refresh the page.

Instead of using the term “healthy”, which is not very meaningful, it would be more helpful to convey a specific message. For example, we could use a more descriptive message to communicate the status or result of the test.

To accomplish this, we need to ensure that the NuGet package “AspNetCore.HealthChecks.UI.Client” is installed. After that, we can proceed to extend the following functionality:

In the Program.cs file, we need to incorporate a ResponseWriter into the mapper of the HealthCheck. This will enable us to customize the output or response generated by the HealthCheck module.

Let’s restart the program and see what comes out of it.

To enable easy retrieval from a list, we can assign a tag to this check. Implementing the tag in the service would facilitate the process.

Now let’s execute this request once more.

Let’s create another check:

It can be a bit overwhelming at first.

To create a dashboard that lists all our checks using the NuGet package “AspNetCore.HealthChecks.UI.Client," we need to make three additions to the Program.cs file:

1. Add a new service:

This new dashboard requires persistent storage, so we need to add the NuGet package “AspNetCore.HealthChecks.UI.InMemory.Storage."

2. New Mapping

Below the previous AddHealthChecks() method call, we can add a new method call AddHealthChecksUI() that uses the in-memory store.

To map this new user interface, we can add a route to the Configure() method in the Startup.cs file. The route should point to the Health Checks UI endpoint.

3.) AppSettings.json

To configure the dashboard to forward requests to the appropriate address, you can modify the appsettings.json file. Within the file, you will need to update the relevant settings to specify the desired address for request forwarding.

We start the application and change the address:

Now our dashboard opens:

By utilizing the dashboard we have created, we can now have a comprehensive overview of the checks we have implemented for our application. The dashboard not only displays the current status of the checks but also provides a history section that gives us insights into the events that have occurred from the application’s start until now. To access more detailed information, we can explore the details section within the dashboard.

What could a HealthCheck that monitors the connection to the database look like?

We create a new object that checks the connection to the database.

public class DatabaseConnectionCheck : IHealthCheck
{
  public async Task<HealthCheckResult> CheckHealthAsync(HealthCheckContext context, CancellationToken cancellationToken = default)
  {
    try
    {
      using var connection = new SqlConnection("Data Source=.;Initial Catalog=MyDatabase;Integrated Security=True;Connect Timeout=30;Encrypt=False;Trust Server Certificate=False;Application Intent=ReadWrite;Multi Subnet Failover=False");
      await connection.OpenAsync(cancellationToken).ConfigureAwait(false);
      using var command = connection.CreateCommand();
      command.CommandText = "Select 1";
      object result = await command.ExecuteScalarAsync(cancellationToken).ConfigureAwait(false);

      return HealthCheckResult.Healthy("The status of the connection is open");
    }
    catch (Exception ex)
    {
      return HealthCheckResult.Unhealthy("The status of the connection is close");
    }

  }
}

Here only the connection to the database is established and by the select 1 it is checked whether the query can be sent or not. Accordingly, a result is returned which is healthy or unhealthy.

Now we only have to announce the service:

Now we have a comprehensive overview of all the checks that have been configured in our application. This provides a clear and organized display of the various checks and their respective statuses, making it easier to monitor and track the health of our application.

these checks are queried every 5 seconds as we set, now if the connection to the database is closed or interrupted, this dashboard will react to it:

you will find a number of nuget dealing with some topics:

Github: HealthChecks

DBConcurrency with EF Core

Ben Witt — Tue, 01 Oct 2024 08:00:00 +0000

What is concurrency in the database?

Concurrency is the simultaneous processing of data by multiple users or processes. In a database environment, this can lead to conflicts if two transactions attempt to access the same data at the same time.

Why is concurrency control important?

Concurrency control is important to ensure that multiple processes or threads can run simultaneously in a system without causing unexpected problems. This is important for efficient resource utilisation, data consistency and avoiding race conditions and deadlocks. Effective concurrency control improves the overall performance of systems, especially in multi-core environments, and contributes to a stable and user-friendly experience.

Conflict detection in Entity Framework Core

Entity Framework Core offers mechanisms for recognising conflicts.

In Entity Framework Core, conflict detection is often activated by the “Timestamp” attribute or the “ConcurrencyCheck” attribute.

„Timestamp” attribute:

Purpose: The “Timestamp” attribute is used to detect conflicts with concurrent updates in Entity Framework Core.
How it works: When the Timestamp attribute is applied to a property, Entity Framework Core automatically creates a database column of type RowVersion. This column is automatically updated with every update. If two updates occur simultaneously, the system recognises a conflict based on this column.

public class Employee
{
  public int Id { get; set; }
  public string FirstName { get; set; } = string.Empty;
  public string LastName { get; set; } = string.Empty;

  public string Fullname => $"{FirstName} {LastName}";

  public string Department { get; set; } = string.Empty;
  public int EntryYear { get; set; }

  [Timestamp]
  public byte[] Version { get; set; }
}

„ConcurrencyCheck“ attribute:

Purpose: The “ConcurrencyCheck” attribute is also used to detect conflicts during updates by ensuring that no changes were made to certain properties during the update.
How it works: When the ConcurrencyCheck attribute is applied to a property, Entity Framework Core compares the value of that property during the update with the value stored in the database. If differences are detected, a conflict is recognised.

public class Employee
{
  public int Id { get; set; }
  public string FirstName { get; set; } = string.Empty;
  public string LastName { get; set; } = string.Empty;

  public string Fullname => $"{FirstName} {LastName}";

  public string Department { get; set; } = string.Empty;
  public int EntryYear { get; set; }

  [ConcurrencyCheck]
  public byte[] RowVersion { get; set; }
}

Both approaches serve to ensure that database updates can be carried out consistently and without conflict, even if several users access the database at the same time.

Strategies for conflict resolution

When resolving conflicts in an application, there are various strategies to ensure that simultaneous updates of data are consistent and without data inconsistencies. Here are some common strategies:

Last writer wins (lww):

Description: the last update written to the database is considered the valid version.
Implementation: If a conflict occurs, the update with the most recent timestamp or transaction ID is favoured.

Extended conflict resolution (Custom Conflict Resolution):

Description: custom logic is implemented to resolve conflicts based on application logic or user preferences.
Implementation: Developers determine how conflicts should be handled, possibly through user interaction or specific business rules.

Optimistic conflict resolution:

Description: It is assumed that conflicts rarely occur, so that the data is updated without prior blocking. Conflicts are only checked when data is written to the database.
Implementation: Use of mechanisms such as Timestamp or ConcurrencyCheck attributes. In the event of a conflict, an exception is raised and the application can react.

Pessimistic conflict resolution:

Description: data is locked throughout the update process to ensure that no other updates can be performed at the same time.
Implementation: Use of explicit locks or transaction isolation levels to ensure that no other operations can access the locked data at the same time.

Automatic merging:

Description: In the event of conflicts, changes are automatically merged if this is possible. This is often used in distributed version control systems or collaborative tools.
Implementation: Automatic merging algorithms are used to recognise and automatically resolve conflicts.

The choice of the appropriate strategy depends on the requirements of the application, the type of data and the expected usage patterns. It is important to choose a strategy that ensures data consistency and at the same time does not impair the performance and user-friendliness of the application.

Implementation of concurrency control in a sample application

Example:

Assume you have an “Employee” entity class with a “LastName” property and a timestamp attribute for concurrency control:

public class Employee
{
public int Id { get; set; }
public string FirstName { get; set; } = string.Empty;
public string LastName { get; set; } = string.Empty;

public string Fullname => $"{FirstName} {LastName}";

public string Department { get; set; } = string.Empty;
public int EntryYear { get; set; }
[Timestamp]
public byte[] Timestamp { get; set; } // Timestamp-Eigenschaft für Concurrency
}

And here’s how you can use the ConcurrencyCheck attribute for concurrency control in an entity class:

public class Employee
{
  public int Id { get; set; }
  public string FirstName { get; set; } = string.Empty;
  public string LastName { get; set; } = string.Empty;

  public string Fullname => $"{FirstName} {LastName}";

  public string Department { get; set; } = string.Empty;
  public int EntryYear { get; set; }

  [ConcurrencyCheck]
  public byte[] RowVersion { get; set; }
}

In your DbContext, you can then specify that the timestamp row acts as RowVersion:

In our example, we select the “ConcurrencyCheck”

public void Configure(EntityTypeBuilder<Employee> builder)
 {
   builder.HasKey(c => c.Id);
   builder.Property(c => c.Id).ValueGeneratedOnAdd();
   builder.Property(rv => rv.RowVersion).IsRowVersion();
 }

Conflict simulation:

To simulate a concurrency conflict, you can perform the following steps:

Assume you already have an employee in your database:

var loadedEmployee = dbContext1.Employees.Find(id);

Now we simulate a conflict by updating the lastname both in the database and in the application:

// Changes in the database by another process

var existingEmployeeInDatabase = dbContext2.Employees.Find(id);


if (loadedEmployee != null && existingEmployeeInDatabase != null)
 {
   // Change only loadedEmployee
   loadedEmployee.LastName = "NewLastName1";

   try
   {
     dbContext1.SaveChanges();  // Save changes in LoadedEmployee
   }
   catch (DbUpdateConcurrencyException ex)
   {
     Console.WriteLine($"Concurrency conflict on LoadedEmployee: {ex.Message}");
     // Here you can implement conflict handling, e.g. retry or report errors to the user interface.
   }

   // Change existingEmployeeInDatabase now
   existingEmployeeInDatabase.LastName = "NewLastName2";

   try
   {
     dbContext2.SaveChanges();  // Try to save changes in existingEmployeeInDatabase, which should throw a DbUpdateConcurrencyException
     Console.WriteLine("Records updated successfully.");
   }
   catch (DbUpdateConcurrencyException ex)
   {
     Console.WriteLine($"Concurrency conflict on existingEmployeeInDatabase: {ex.Message}");
     // Here you can implement conflict handling, e.g. retry or report errors to the user interface.
   }
 }

In this example, dbContext2.SaveChanges() will throw a DbUpdateConcurrencyException because the database version of the product no longer matches the one in your application.

You can now implement the conflict resolution logic to decide how to deal with this conflict. This can include applying strategies such as “client wins”, “database wins” or customised conflict resolution.

Here is an example of the client wins strategy:

using (var dbContext1 = Create.Context())
 using (var dbContext2 = Create.Context())
 {
   var loadedEmployee = dbContext1.Employees.Find(id);
   var existingEmployeeInDatabase = dbContext2.Employees.Find(id);

   if (loadedEmployee != null && existingEmployeeInDatabase != null)
   {
     // Change only loadedEmployee
     loadedEmployee.LastName = "NewLastName1";

     try
     {
       dbContext1.SaveChanges();  // Save changes in LoadedEmployee
     }
     catch (DbUpdateConcurrencyException ex)
     {
       Console.WriteLine($"Concurrency conflict on LoadedEmployee: {ex.Message}");

       ex.Entries.Single().Reload();
       dbContext1.SaveChanges(); 
       Console.WriteLine("Client Wins: Records updated successfully.");
     }

     // Change existingEmployeeInDatabase now
     existingEmployeeInDatabase.LastName = "NewLastName2";

     try
     {
       dbContext2.SaveChanges();  // Try to save changes in existingEmployeeInDatabase, which should throw a DbUpdateConcurrencyException
       Console.WriteLine("Records updated successfully.");
     }
     catch (DbUpdateConcurrencyException ex)
     {
       Console.WriteLine($"Concurrency conflict on existingEmployeeInDatabase: {ex.Message}");
       // Here you can implement conflict handling, e.g. retry or report errors to the user interface.
     }
   }
 }

Concurrency proves to be essential by enabling simultaneous access to database resources. Precise handling of DBConcurrency minimises potential conflicts, ensures data consistency and guarantees integrity in dynamic environments. Clever implementation of mechanisms such as transaction control and locking procedures not only optimises synchronisation, but also increases performance. In complex applications that access shared databases, the accurate handling of concurrency is crucial for a robust system architecture and smooth functioning under intensive load.
GitHub: DBConcurrency

Basics of Clean Architecture with C#

Ben Witt — Tue, 24 Sep 2024 08:00:00 +0000

This article covers the basics of Clean Architecture with C#. Readers will be empowered to develop solid, well-structured code that excels in maintainability, extensibility, and testability.

Introduction to Clean Architecture

What is Clean Architecture?
Clean Architecture, proposed by Robert C. Martin or “Uncle Bob,” is an architectural pattern for software projects aimed at keeping the code clean, modular, and independent of external influences. It isolates the business logic of the application from external details such as user interfaces, databases, and frameworks.

Why is Clean Architecture important?
Clean Architecture offers a variety of advantages:

High Maintainability: The clear separation of business logic and technical details facilitates understanding of the code and enhances its maintainability.

2.** Extensibility**: Thanks to the well-structured architecture, new features can be added or existing ones modified easily without jeopardizing the integrity of the overall application.

Testability: The clean separation of components simplifies the writing of automated tests, thereby improving the quality of the software.
Independence from external influences: By using abstractions and interfaces, the application remains flexible and is not tied to specific technologies or frameworks.

The benefits of Clean Architecture include:

Clear Separation of Responsibilities: The clear structure defines the responsibilities of the various components, leading to better-organized code.
Minimization of Dependencies: Clean Architecture minimizes dependencies between different parts of the application, enhancing flexibility and maintainability.
Facilitation of Testing: The clean separation of components allows tests to focus on individual parts of the application, improving testability and software quality.
Support for Scalability and Extensibility: The modular structure of Clean Architecture makes it easier to scale the application and add new features without redesigning the entire architecture.

The Principles of Clean Architecture

Separation of Concerns:
This principle demands that various aspects of a software application, such as user interface, business logic, and data persistence, be separated from each other. This leads to better-structured and more maintainable code.

Dependency Rule:
The dependency rule states that dependencies within an application should always point towards stable layers. For example, the business logic layer should not have direct dependencies on the user interface or databases.

Stable Dependencies Principle:
This principle dictates that dependent components should be more stable than the components they depend on. This means that high-level modules should not depend on low-level modules.

Stable Abstractions Principle:
Here, the abstraction should be more stable than the concrete implementation. Abstractions should be independent of concrete implementations.

These principles form the backbone of Clean Architecture and contribute to creating a robust and flexible software architecture. Now, let’s take a look at the components of Clean Architecture.

The Components of Clean Architecture

Clean Architecture comprises various components designed to divide the application into clearly defined layers. Here are the main components:

1. Dependency Inversion Principle: This principle states that dependencies should depend on abstract interfaces, not concrete implementations. This decouples the layers and increases the flexibility and testability of the application.

2. Entities: Entities represent the core objects of the application, representing the state and behavior of the business logic. They are independent of the database or other external systems.

3. Use Cases: Use Cases contain the application logic and define how user interactions are handled. They are independent of the user interface and other external systems.

4. Interface Adapters: These adapters connect the application to external systems such as databases, APIs, or user interfaces. They convert data and calls into formats that can be processed by the core application.

5. Frameworks & Drivers: This layer contains external frameworks and drivers used by the application, such as web frameworks, database drivers, or user interface libraries.

By clearly separating these components, changes can be made in one area of the application without affecting other areas. This makes the application flexible and easier to maintain.

Implementation of Clean Architecture in C# (Continued)

Implementing the Project Structure:

1. Creating the Core Layer:

Define Entities and Use Cases.
Create a folder named “Core.”
Within this folder, classes for Entities such as “User.cs” or “Product.cs” are created.
Create interfaces or abstract classes for Use Cases like “IUserService.cs” or “IProductService.cs”.

2. Creating the Application Layer:
Implement Use Cases.
Create a folder named “Application.”
Implement Use Cases as classes that inherit from the interfaces defined in the Core layer.
For example: “UserRegistrationService.cs”, “ProductManagementService.cs”.

3. Creating the Infrastructure Layer:
Implement Interface Adapters.
Create a folder named “Infrastructure.”
Implement concrete classes for the interfaces defined in the Core layer.
For example: “UserRepository.cs” for “IUserRepository”.

4. Creating the Presentation Layer:
Implement user interface or API controllers.
Create a folder named “Presentation.”
Implement controllers, such as ASP.NET MVC controllers or WebAPI controllers.

Implementing Entities:

Entities represent the core objects of the application and include the state and behavior of the business logic. Here’s an example of a “User” Entity:

namespace CleanArchitectureExample.Core.Entities;

public class User
{
  public int Id { get; set; }
  public string Username { get; set; }
  public string Email { get; set; }
}

With this structure, we lay the foundation for a clean and modular architecture that is easily maintainable and extensible.

Implementing Use Cases:

Use Cases contain the application logic and define how user interactions are processed. Here’s an example of a “UserRegistration” Use Case:

namespace CleanArchitectureExample.Core.UseCases;

public interface IUserRegistrationUseCase
{
  void RegisterUser(string username, string email);
}

namespace CleanArchitectureExample.Application.Services;

public class UserService : IUserRegistrationUseCase
{
  private readonly IUserRepository _userRepository;

  public UserService(IUserRepository userRepository)
  {
    _userRepository = userRepository;
  }

  public IUserRepository GetUserRepository()
  {
    return _userRepository;
  }

  public void RegisterUser(string username, string email)
  {
    // The user registration logic is implemented here
    var newUser = new User { Username = username, Email = email };

    _userRepository.AddUser(newUser);
    Console.WriteLine($"User {username} with the email {email} was successfully registered.");
  }
}

Implementing Interface Adapters:

Interface Adapters connect the application to external systems such as databases or APIs. Here’s an example of a UserRepository implementing the database access interface:

namespace CleanArchitectureExample.Infrastructure;

public interface IUserRepository
{
  void AddUser(User user);
}

public class UserRepository : IUserRepository
{
  public void AddUser(User user)
  {
    // This is where the logic for adding a user to the database would be implemented
    Console.WriteLine($"User {user.Username} has been added to the database.");
  }
}

Now, we will proceed to manage dependencies between components using Dependency Injection.

Implementation of Clean Architecture in C

Setting up Dependency Injection:
In this step, we will set up Dependency Injection (DI) in our C# project to manage dependencies between different components.

First, we need to configure a DI container in our project. We can use the built-in DI container library provided by .NET for this purpose. Here’s an example of how we can do this:

namespace CleanArchitectureExample;

class Program
{
  static void Main(string[] args)
  {
    // Configuration of the DI container
    var services = new ServiceCollection();
    services.AddTransient<IUserRepository, UserRepository>();
    services.AddTransient<IUserRegistrationUseCase, UserService>();

    // Create the DI container
    var serviceProvider = services.BuildServiceProvider();

    // Use the dependent components
    var userService = serviceProvider.GetService<IUserRegistrationUseCase>();
    userService.RegisterUser("Max Mustermann", "max@example.com");

    Console.ReadLine(); // Wait for user input so that the console application is not closed immediately
  }
}

In this example, we use the ServiceCollection class to register dependencies. With AddTransient, we specify that a new instance of the dependent class should be created each time it’s requested.

Once we have configured our DI container, we can use the dependent components in our classes. The DI container takes care of resolving and providing the dependent instances.

For example, the UserRegistrationService can access the IUserRepository like this:

namespace CleanArchitectureExample.Application.Services;

public class UserService : IUserRegistrationUseCase
{
  private readonly IUserRepository _userRepository;

  public UserService(IUserRepository userRepository)
  {
    _userRepository = userRepository;
  }

  public IUserRepository GetUserRepository()
  {
    return _userRepository;
  }

  public void RegisterUser(string username, string email)
  {
    // The user registration logic is implemented here
    var newUser = new User { Username = username, Email = email };

    _userRepository.AddUser(newUser);
    Console.WriteLine($"User {username} with the email {email} was successfully registered.");
  }
}

By using Dependency Injection, the components are loosely coupled, which improves the maintainability, extensibility, and testability of our application.

Testing in Clean Architecture

Testing is an integral part of Clean Architecture, ensuring that our application functions correctly and that changes do not introduce unwanted side effects. In Clean Architecture, we use various types of tests:

1. Unit Tests: These test individual components of our application in isolation. Mock objects can be used to simulate external dependencies.

2. Integration Tests: These test the collaboration of multiple components of our application to ensure they interact properly.

3. End-to-End Tests: These test the entire application from an external perspective, such as by executing user interactions through the user interface.

Test-Driven Development (TDD) Approach:

In the test-driven development approach, we first write tests that define the expected behavior of the application, and then implement the code that passes these tests. This approach promotes thorough test coverage and often leads to better code quality and structure.

Example Tests for Various Components:

Here is an example of a unit test for the UserRegistrationService:

[Fact]
  public void RegisterUser_Should_Register_User()
  {
    // Arrange
    var mockUserRepository = new Mock<IUserRepository>();
    var userService = new UserService(mockUserRepository.Object);
    var username = "John Smith";
    var email = "john@example.com";

    // Act
    userService.RegisterUser(username, email);

    // Assert
    mockUserRepository.Verify(repo => repo.AddUser(It.IsAny<User>()), Times.Once);
  }

In this test, we use a mock object for IUserRepository to simulate database access. We verify that the AddUser method of the repository was called once after registering a user.

Conclusion:

In this article, we’ve provided a comprehensive overview of the fundamentals of Clean Architecture with C#, covering everything from basic principles to practical implementation and testing. Clean Architecture offers a multitude of benefits for software project development, including improved maintainability, extensibility, and testability. By clearly separating responsibilities and minimizing dependencies, the code becomes more structured and easier to maintain. The use of Dependency Injection allows for flexible configuration of the application and facilitates testing by isolating components. Overall, Clean Architecture provides a robust approach to developing high-quality software that has proven itself in a variety of application domains.

Clean architecture

HybridCache in a console application with Redis

Ben Witt — Tue, 17 Sep 2024 08:00:00 +0000

In modern applications, the efficient management of data and the avoidance of unnecessary database queries is essential. An effective way to achieve this is to use a HybridCache, which utilises both a MemoryCache and a Distributed Cache such as Redis. In this article, I will present an implementation of a HybridCache in a console-only application and explain when to query which cache to achieve optimal performance. In addition, I will discuss the importance of cache invalidation and explain how this can be solved in the implementation.

What is a HybridCache?

A HybridCache combines several cache types in order to increase performance. In our case, we first access the fast, memory-based MemoryCache, and if no data is available there, we fall back on a distributed cache — in this case Redis. Only if neither cache returns a hit is a query sent to the database. This approach helps to minimise database queries and significantly reduce latency times.

Implementation of the HybridCache

I have created a console application that implements the HybridCache with Microsoft.Extensions.Caching.Memory and Redis. The following code shows the basic implementation where cache invalidation plays a role:

public class HybridCache : IHybridCache
{
    private readonly IMemoryCache _memoryCache;
    private readonly IDistributedCache _distributedCache;
    private static readonly object _lock = new object();  // Lock for thread-safety

    public HybridCache(IMemoryCache memoryCache, IDistributedCache distributedCache)
    {
        _memoryCache = memoryCache;
        _distributedCache = distributedCache;
    }

    public async Task<T> GetOrSetAsync<T>(string key, Func<Task<T>> factory, TimeSpan duration)
    {
        T value;

        // Check MemoryCache first
        lock (_lock)
        {
            if (_memoryCache.TryGetValue(key, out value))
            {
                string valueString = value.ToString();
                valueString = ReplaceSourceInfo(valueString, "MemoryCache");
                value = (T)Convert.ChangeType(valueString, typeof(T));
                return value;
            }
        }

        // Check Distributed Cache (Redis)
        var cachedData = await _distributedCache.GetStringAsync(key);
        if (cachedData != null)
        {
            value = System.Text.Json.JsonSerializer.Deserialize<T>(cachedData);

            string valueString = value.ToString();
            valueString = ReplaceSourceInfo(valueString, "Distributed Cache");
            value = (T)Convert.ChangeType(valueString, typeof(T));

            // Set the value in MemoryCache
            lock (_lock)
            {
                _memoryCache.Set(key, value, absoluteExpirationRelativeToNow: duration);
            }

            return value;
        }

        // Query the database if not found in any cache
        value = await factory();

        // Save the data to both MemoryCache and Redis
        string newValue = ReplaceSourceInfo(value.ToString(), "Database");
        value = (T)Convert.ChangeType(newValue, typeof(T));

        var serializedValue = System.Text.Json.JsonSerializer.Serialize(value);
        await _distributedCache.SetStringAsync(key, serializedValue, new DistributedCacheEntryOptions
        {
            AbsoluteExpirationRelativeToNow = duration  // Set expiration time for Redis
        });

        lock (_lock)
        {
            _memoryCache.Set(key, value, duration);  // Set expiration time for MemoryCache
        }

        return value;
    }

    public void InvalidateCache(string key)
    {
        // Remove key from MemoryCache
        lock (_lock)
        {
            _memoryCache.Remove(key);
        }

        // Remove key from Distributed Cache (Redis)
        _distributedCache.Remove(key);
    }

    private string ReplaceSourceInfo(string originalValue, string source)
    {
        string patternToRemove = @"\(Source:.*?\)";
        string updatedValue = System.Text.RegularExpressions.Regex.Replace(originalValue, patternToRemove, "");
        return $"{updatedValue} (Source: {source})";
    }

    public async Task<T> GetAsync<T>(string key)
    {
        // Try getting the value from MemoryCache first
        lock (_lock)
        {
            if (_memoryCache.TryGetValue(key, out T value))
            {
                return value;
            }
        }

        // If not found, get the value from Distributed Cache
        var cachedData = await _distributedCache.GetStringAsync(key);
        return cachedData != null ? System.Text.Json.JsonSerializer.Deserialize<T>(cachedData) : default;
    }
}

Cache invalidation

In the above implementation, a simple time-based expiration is used to ensure that the cache entries in both the MemoryCache and the Redis Cache become invalid after a certain time. This is controlled by the AbsoluteExpirationRelativeToNow method, which determines when an entry expires automatically.
In addition, I have added an InvalidateCache method that enables manual invalidation of the cache entries. With this method, the cache entry can be explicitly deleted from both the MemoryCache and Redis if, for example, the underlying data in the database has been changed.

When is which cache used?

In this implementation, the HybridCache ensures that queries are made according to a clear hierarchy. MemoryCache is queried first, as it is faster. If no data is found there, the cache checks Redis. Redis is favoured in distributed systems as it can serve multiple instances of the application and therefore provides a central source for the cache data. Finally, if both caches fail, the database is queried.

Extended strategies for cache invalidation

manual/explicit invalidation: As shown in the code, there is an InvalidateCache method that can be used to manually remove cache entries. This is useful when data in the database changes and the cache contains outdated entries.
event-based invalidation: In distributed systems, an event-based model could be used to invalidate cache entries. If the data changes, a message could be sent (e.g. via a message queue system) that invalidates the cache entry in several instances simultaneously.
sliding expiration: Another possible strategy is sliding expiration, in which the expiration time of a cache entry is extended with each access. This ensures that frequently used data remains in the cache for longer, while unused data expires after a certain time. An example of sliding expiration could look like this:

var cacheEntryOptions = new DistributedCacheEntryOptions
{
    SlidingExpiration = TimeSpan.FromMinutes(5)  // Expiration is reset after each access
};

await _distributedCache.SetStringAsync(key, serializedValue, cacheEntryOptions);

Install Redis - locally or via Docker

Redis is used as a distributed cache for this application. Redis can be installed in various ways. An easy way to install Redis locally is to download the Redis binaries from the official website (https://redis.io/download). Once Redis is installed, it can be executed on the local host on the standard port 6379.
Alternatively, Redis can also be run via Docker.

Other options - e.g. with MS SQL Server

In addition to Redis, there are also other options for implementing a distributed cache. One widespread option is the use of an SQL-based cache, e.g. with Microsoft SQL Server. This solution is often used in environments where there is already a strong integration with SQL Server-based systems.

Conclusion

The implementation of a HybridCache in a console application offers numerous advantages, especially when it comes to reducing latency times and database queries. By combining a fast memory-based cache such as MemoryCache and a distributed cache such as Redis, application performance can be significantly increased. The introduction of thread safety and proper synchronisation ensures that parallel requests do not cause inconsistencies. Furthermore, cache invalidation is an important aspect to ensure that only up-to-date data is used and should be customised to the needs of the application.
With Redis as a locally installed cache - or even as a Docker container - and the right cache hierarchy, scalable and high-performance applications can be implemented.

DEMO:

Github: HybridCache

Implementing the Cached Repository Pattern in C#

Ben Witt — Tue, 23 Jul 2024 08:00:00 +0000

Introduction to the Concept of a Cached Repository

A cached repository is a design pattern aimed at enhancing application performance by storing data in a fast-access memory area known as a cache. This reduces the number of database accesses, thereby improving response times and the application's scalability. A repository abstracts data access and provides uniform interfaces for CRUD operations (Create, Read, Update, Delete). Combining these concepts offers a powerful method for optimizing data access patterns in modern applications.

Importance and Benefits of Using Cached Repositories in C# for Advanced Developers

For advanced developers, cached repositories offer several advantages:

Performance Improvement: Reducing database accesses significantly enhances response times.
Scalability: Lower database load facilitates better application scalability.
Cost Reduction: Fewer database queries translate to lower costs, especially with cloud services billed per query.
Consistency and Abstraction: Using a uniform repository ensures consistent data access and allows for easy abstraction and testing.

Detailed Implementation of a Cached Repository in C# Using the Decorator Pattern and EF Core

Implementing a cached repository can be effectively achieved through the decorator pattern. This pattern allows additional functionality to be added to an object without altering its structure.

Define the Repository Interface

public interface IProductRepository
{
    Task<Product> GetProductByIdAsync(int id);
    Task<IEnumerable<Product>> GetAllProductsAsync();
    Task AddProductAsync(Product product);
    Task UpdateProductAsync(Product product);
    Task DeleteProductAsync(int id);
}

Implement the Base Repository with EF Core

public class ProductRepository : IProductRepository
{
    private readonly ApplicationDbContext _context;

    public ProductRepository(ApplicationDbContext context)
    {
        _context = context;
    }

    public async Task<Product> GetProductByIdAsync(int id)
    {
        return await _context.Products.FirstOrDefaultAsync(x => x.Id == id);
    }

    public async Task<IEnumerable<Product>> GetAllProductsAsync()
    {
        return await _context.Products.ToListAsync();
    }

    public async Task AddProductAsync(Product product)
    {
        await _context.Products.AddAsync(product);
        await _context.SaveChangesAsync();
    }

    public async Task UpdateProductAsync(Product product)
    {
        _context.Products.Update(product);
        await _context.SaveChangesAsync();
    }

    public async Task DeleteProductAsync(int id)
    {
        var product = await _context.Products.FindAsync(id);
        if (product != null)
        {
            _context.Products.Remove(product);
            await _context.SaveChangesAsync();
        }
    }
}

Implement the Cached Repository

public class CachedProductRepository : IProductRepository
{
    private readonly IProductRepository _repository;
    private readonly IMemoryCache _cache;
    private readonly TimeSpan _cacheDuration = TimeSpan.FromMinutes(5);

    public CachedProductRepository(IProductRepository repository, IMemoryCache cache)
    {
        _repository = repository;
        _cache = cache;
    }

    public async Task<Product> GetProductByIdAsync(int id)
    {
        string cacheKey = $"Product_{id}";
        if (!_cache.TryGetValue(cacheKey, out Product product))
        {
            product = await _repository.GetProductByIdAsync(id);
            if (product != null)
            {
                _cache.Set(cacheKey, product, _cacheDuration);
            }
        }
        return product;
    }

    public async Task<IEnumerable<Product>> GetAllProductsAsync()
    {
        const string cacheKey = "AllProducts";
        if (!_cache.TryGetValue(cacheKey, out IEnumerable<Product> products))
        {
            products = await _repository.GetAllProductsAsync();
            _cache.Set(cacheKey, products, _cacheDuration);
        }
        return products;
    }

    public async Task AddProductAsync(Product product)
    {
        await _repository.AddProductAsync(product);
        InvalidateCache();
    }

    public async Task UpdateProductAsync(Product product)
    {
        await _repository.UpdateProductAsync(product);
        InvalidateCache();
    }

    public async Task DeleteProductAsync(int id)
    {
        await _repository.DeleteProductAsync(id);
        InvalidateCache();
    }

    private void InvalidateCache()
    {
        _cache.Remove("AllProducts");
    }
}

Best Practices and Potential Pitfalls in Using Cached Repositories in C#

Best Practices:

Cache Invalidation: Ensure the cache is invalidated after write operations (Add, Update, Delete) to maintain consistency.
Cache Duration: Choose an appropriate cache duration to balance freshness and performance.
Memory Management: Avoid overloading the cache, especially in memory-intensive applications.

Potential Pitfalls:

Stale Data: Cached data can become outdated, leading to inconsistencies.
Complexity: Implementing and managing cached repositories increases codebase complexity.
Memory Consumption: Excessive caching can lead to high memory usage and potential out-of-memory issues.

Comparison with Other Caching Strategies and Their Applications

In addition to the decorator pattern for cached repositories, there are several other caching strategies:

In-Memory Caching: Direct use of in-memory data stores like IMemoryCache or ConcurrentDictionary. Ideal for short-term, small data sets.
Distributed Caching: Use of distributed caches like Redis or Memcached. Suitable for applications with high scalability requirements.
HTTP Caching: Use of HTTP headers to cache web resources. Ideal for high-traffic web applications.

Each strategy has specific use cases and challenges that need to be carefully evaluated.

Advanced Topics: Cache Invalidation and Synchronization Between Cache and Database

Cache invalidation and synchronization are complex topics that require special attention:

Cache Invalidation:

Time-to-Live (TTL): Set a TTL for cache entries to ensure automatic invalidation.
Event-Based Invalidation: Use events or message queues to synchronize cache invalidations in distributed systems.

Synchronization Between Cache and Database:

Write-Through Caching: Write operations are performed on both the database and the cache, ensuring consistency.
Write-Behind Caching: Write operations are initially performed on the cache and later synchronized with the database. This can improve performance but carries the risk of data inconsistency in the event of a crash.
Cache Priming: Preload frequently accessed data into the cache at application startup to avoid initial latencies.

A comprehensive understanding and correct implementation of these techniques are crucial for successfully leveraging cached repositories in demanding applications.

In summary, cached repositories, combined with the decorator pattern and Entity Framework Core, offer an effective method for optimizing data access patterns. They provide significant performance benefits but require careful implementation and management to avoid potential pitfalls.

Github:
https://github.com/WittBen/CachedRepository

DEV Community: Ben Witt

Stop Loading Your Entire Instruction System Into Every Session

The Actual Problem

The Insight

The Architecture

How Loading Works

The Result

Why This Works

What It Costs

When It Is Worth It

Conclusion

The Most Dangerous Bias of Your AI Assistant Is That It Agrees With You

What Sycophancy Drift Is

Why You Do Not Notice It

The Idea: A Reflective Layer at the End of the Session

What the Layer Actually Flags

Three Design Decisions That Make the Difference

The Honest Part: The Recursion Problem

When It Is Worth It

Conclusion

Transitive Dependency – Challenges and Approaches in .NET

1. Basics and Problem Statement

1.1 Transitive References vs. Transitive Dependencies

1.2 A Typical Scenario in .NET

2. Detailed Solution Approaches

2.1 PrivateAssets="all"

2.2 DisableTransitiveProjectReferences

2.3 Comparison of Both Approaches

3. Practical Example: Abstraction and Interface Encapsulation

4. Advanced Strategies and Alternative Approaches

5. Summary and Outlook

Efficient Debugging and Precise Logging in C#: Using Caller Attributes

Basics of the Attributes

Code Examples for Practical Applications

Practical Application with a Library Management System

Limitations and Best Practices

Conclusion

A Critical Look at Cancellation Management in .NET Applications

Asynchronous HTTP Requests – Explained Simply

WPF Application with Plugin Architecture

Overview over the structure

The plug-in interface

Implementation of a plug-in

Dynamic loading and registration of plug-ins

Automatic registration of views and ViewModels with Autofac

Communication between plug-ins via the EventAggregator

Conclusion

HealthChecks

DBConcurrency with EF Core

What is concurrency in the database?

Why is concurrency control important?

Conflict detection in Entity Framework Core

„Timestamp” attribute:

„ConcurrencyCheck“ attribute:

Strategies for conflict resolution

Implementation of concurrency control in a sample application

Conflict simulation:

Basics of Clean Architecture with C#

Introduction to Clean Architecture

The Principles of Clean Architecture

The Components of Clean Architecture

Implementing the Project Structure:

Implementing Entities:

Implementing Use Cases:

Implementing Interface Adapters:

Implementation of Clean Architecture in C

Testing in Clean Architecture

Test-Driven Development (TDD) Approach:

Conclusion:

HybridCache in a console application with Redis

What is a HybridCache?

Implementation of the HybridCache

Cache invalidation

When is which cache used?

Extended strategies for cache invalidation

Install Redis - locally or via Docker

Other options - e.g. with MS SQL Server

Conclusion

DEMO:

Implementing the Cached Repository Pattern in C#

2.1 `PrivateAssets="all"`