Designing a Local AI Gateway on .NET: Isolating Unmanaged Dependencies and DDD Patterns

#ai #dotnet #architecture #cpp

Tools like Ollama and LM Studio have made local LLM hosting incredibly accessible for developer environments and rapid prototyping. However, building a similar production-ready inference gateway on .NET for isolated corporate perimeters presents a major engineering challenge.

You must directly integrate rapidly evolving, low-level C/C++ libraries (like llama.cpp and ggml) while maintaining strict architectural stability. This is exactly the problem solved by InstantAIGate, a standardized local inference gateway developed on .NET 10 for enterprise ecosystems.

In this article, we will examine the approach to designing the unmanaged code interaction layer using Clean Architecture and Domain-Driven Design (DDD) principles. We will explore how to isolate business logic from the implementation details of C++ bindings, ensuring deployment predictability and protecting the upper layers of the system from breaking changes in third-party libraries.

The Architectural Challenge: Fragility of Transitive Dependencies

The top priority for infrastructure software is the stability of production environments and the predictability of updates. However, the open-source AI library ecosystem updates daily. Function signatures change, data structures are rebuilt, and memory allocation logic is rewritten.

If the upper layers of the application (for example, REST API controllers implementing the OpenAI API specification) directly call llama.cpp methods, any library update will lead to a cascading failure of the codebase.

To resolve this problem, InstantAIGate employs the Anti-Corruption Layer (ACL) pattern, which physically and logically separates low-level calls (P/Invoke) and domain abstractions.

Layer 1: Deterministic P/Invoke (NativeMethods)

The first layer of the system is a strictly isolated static class, NativeMethods, whose sole responsibility is to declare the signatures of unmanaged functions, structures, and enumerations. This layer contains zero business logic.

For example, enumerations and structures exactly mirror the C header files:

Logging levels ggml_log_level are defined (from GGML_LOG_LEVEL_NONE to GGML_LOG_LEVEL_CONT).
Quantization types are described in ggml_type (e.g., GGML_TYPE_F16, GGML_TYPE_Q4_K, etc.).
The llama_model_params structure is declared, containing device pointers, layer distribution parameters (n_gpu_layers), and flags like use_mlock and use_mmap.
The llama_context_params structure is defined for context configuration (batch size n_batch, thread count n_threads, attention and pooling types).

Functions are imported via the [DllImport] attribute:

[DllImport("llama", CallingConvention = CallingConvention.Cdecl,
 EntryPoint = "llama_model_load_from_file", CharSet = CharSet.Ansi)]
public static extern IntPtr llama_model_load_from_file(
    [MarshalAs(UnmanagedType.LPUTF8Str)] string path_model,
    llama_model_params @params);

The entire responsibility of this layer is to guarantee correct memory marshaling between the .NET managed environment and the unmanaged core memory.

Layer 2: Transformation Layer (NativeLlamaApi)

To ensure the upper layers remain completely unaware of the existence of NativeMethods, we introduce the INativeLlamaApi interface and its single implementation, the NativeLlamaApi class. This class acts as a facade that converts C# domain types into the structures expected by the llama.cpp library.

Let's look at the model loading process:

public IntPtr LoadModel(string path, int gpuLayers, int mainGpu,
 bool useMlock, bool useMmap, NativeLlamaSplitMode splitMode)
{
    var p = NativeMethods.llama_model_default_params(); 
    p.n_gpu_layers = gpuLayers; 
    p.main_gpu = mainGpu; 
    p.use_mlock = useMlock; 
    p.use_mmap = useMmap; 
    p.split_mode = (NativeMethods.llama_split_mode)splitMode; 

    return NativeMethods.llama_model_load_from_file(path, p); 
}

Here, NativeLlamaApi receives safe .NET primitives and the internal NativeLlamaSplitMode enum, requests default model parameters via llama_model_default_params(), maps them onto the unmanaged llama_model_params structure, and executes the load.

The same principle applies to context creation: the class forms the llama_context_params structure, filling in n_ctx, n_batch, embeddings, and translating domain enumerations into llama_flash_attn_type and ggml_type before calling llama_init_from_model.

Change Management: Adapting to Library Updates

Suppose in a new version of llama.cpp, developers decide to optimize the sampling process. They might remove the llama_sampler_init_temp function and replace it with a single, complex initialization function with a different signature.

In a classic monolithic architecture, this would require refactoring all controllers and text generation services. In the InstantAIGate architecture, the upper layers and microservices interact only with the INativeLlamaApi abstraction via Dependency Injection.

The process of adapting to Breaking Changes comes down to two local steps:

Updating the abstraction: In the NativeMethods class, we update the DllImport signature according to the new C++ header file.
Adjusting the mapping: In the NativeLlamaApi class, we rewrite the method logic (e.g., SamplerInitTemp) so that it formats parameters for the new unmanaged function.

The INativeLlamaApi interface remains unchanged. Domain logic, queue orchestration, and REST API endpoints remain unaware that any changes have occurred. This allows for seamless updates and maintains the long-term stability (LTS) of enterprise systems without the need to rewrite application code.

Conclusion

Strict adherence to Clean Architecture principles and the isolation of unmanaged dependencies via interface facades is not just a tribute to academic patterns, but a necessary standard for enterprise-level systems. This approach ensures predictable memory profiling, minimizes risks when updating computational cores, and guarantees the stable operation of a standalone native executable in isolated environments.

In upcoming articles, we will dive deeper into unmanaged memory management and explore the use of SafeHandle to make the codebase even more performant while guaranteeing deterministic resource cleanup.

Bonus: AI Prompt Template for .NET/C++ Interop

If you are building your own infrastructure and need to integrate a rapidly changing C/C++ library into a .NET ecosystem, creating a robust Anti-Corruption Layer manually can be tedious and prone to memory leaks.

You can use the following prompt in ChatGPT, Claude, or Gemini to automatically generate a safe, DDD-compliant interop architecture based on any C/C++ header file.

Copy and paste this prompt into your AI assistant:

System Role: You are a Principal .NET Solution Architect specializing in unmanaged memory, P/Invoke, and Clean Architecture.

Task: I will provide you with a C/C++ header file (.h). Your goal is to design a secure, production-ready C# integration layer using the Anti-Corruption Layer (ACL) pattern.

Strict Requirements:

NativeMethods Class: Create an internal static class named NativeMethods. Translate all C-structs using [StructLayout(LayoutKind.Sequential)] and map all functions using [DllImport]. Do not put any business or orchestration logic in this class.

Domain Abstraction: Create a clean C# interface (e.g., INativeEngineFacade) that uses only safe .NET primitives (strings, enums, standard arrays) instead of raw pointers (IntPtr).

The Facade Implementation: Implement the interface in a Facade class. This class must handle all data marshaling, pointer conversions, and memory boundary crossing between the safe C# domain types and the unmanaged NativeMethods.

Safety & Stability: Emphasize memory leak prevention, correct data type mapping (e.g., C++ bool vs C# bool marshaling), and deterministic execution.

Input: Here is the C/C++ header code:
[INSERT YOUR C/C++ HEADER CODE HERE]