DEV Community: Instancian

Designing a Local AI Gateway on .NET: Isolating Unmanaged Dependencies and DDD Patterns

Instancian — Sat, 11 Jul 2026 12:13:29 +0000

Tools like Ollama and LM Studio have made local LLM hosting incredibly accessible for developer environments and rapid prototyping. However, building a similar production-ready inference gateway on .NET for isolated corporate perimeters presents a major engineering challenge.

You must directly integrate rapidly evolving, low-level C/C++ libraries (like llama.cpp and ggml) while maintaining strict architectural stability. This is exactly the problem solved by InstantAIGate, a standardized local inference gateway developed on .NET 10 for enterprise ecosystems.

In this article, we will examine the approach to designing the unmanaged code interaction layer using Clean Architecture and Domain-Driven Design (DDD) principles. We will explore how to isolate business logic from the implementation details of C++ bindings, ensuring deployment predictability and protecting the upper layers of the system from breaking changes in third-party libraries.

The Architectural Challenge: Fragility of Transitive Dependencies

The top priority for infrastructure software is the stability of production environments and the predictability of updates. However, the open-source AI library ecosystem updates daily. Function signatures change, data structures are rebuilt, and memory allocation logic is rewritten.

If the upper layers of the application (for example, REST API controllers implementing the OpenAI API specification) directly call llama.cpp methods, any library update will lead to a cascading failure of the codebase.

To resolve this problem, InstantAIGate employs the Anti-Corruption Layer (ACL) pattern, which physically and logically separates low-level calls (P/Invoke) and domain abstractions.

Layer 1: Deterministic P/Invoke (NativeMethods)

The first layer of the system is a strictly isolated static class, NativeMethods, whose sole responsibility is to declare the signatures of unmanaged functions, structures, and enumerations. This layer contains zero business logic.

For example, enumerations and structures exactly mirror the C header files:

Logging levels ggml_log_level are defined (from GGML_LOG_LEVEL_NONE to GGML_LOG_LEVEL_CONT).
Quantization types are described in ggml_type (e.g., GGML_TYPE_F16, GGML_TYPE_Q4_K, etc.).
The llama_model_params structure is declared, containing device pointers, layer distribution parameters (n_gpu_layers), and flags like use_mlock and use_mmap.
The llama_context_params structure is defined for context configuration (batch size n_batch, thread count n_threads, attention and pooling types).

Functions are imported via the [DllImport] attribute:

[DllImport("llama", CallingConvention = CallingConvention.Cdecl,
 EntryPoint = "llama_model_load_from_file", CharSet = CharSet.Ansi)]
public static extern IntPtr llama_model_load_from_file(
    [MarshalAs(UnmanagedType.LPUTF8Str)] string path_model,
    llama_model_params @params);

The entire responsibility of this layer is to guarantee correct memory marshaling between the .NET managed environment and the unmanaged core memory.

Layer 2: Transformation Layer (NativeLlamaApi)

To ensure the upper layers remain completely unaware of the existence of NativeMethods, we introduce the INativeLlamaApi interface and its single implementation, the NativeLlamaApi class. This class acts as a facade that converts C# domain types into the structures expected by the llama.cpp library.

Let's look at the model loading process:

public IntPtr LoadModel(string path, int gpuLayers, int mainGpu,
 bool useMlock, bool useMmap, NativeLlamaSplitMode splitMode)
{
    var p = NativeMethods.llama_model_default_params(); 
    p.n_gpu_layers = gpuLayers; 
    p.main_gpu = mainGpu; 
    p.use_mlock = useMlock; 
    p.use_mmap = useMmap; 
    p.split_mode = (NativeMethods.llama_split_mode)splitMode; 

    return NativeMethods.llama_model_load_from_file(path, p); 
}

Here, NativeLlamaApi receives safe .NET primitives and the internal NativeLlamaSplitMode enum, requests default model parameters via llama_model_default_params(), maps them onto the unmanaged llama_model_params structure, and executes the load.

The same principle applies to context creation: the class forms the llama_context_params structure, filling in n_ctx, n_batch, embeddings, and translating domain enumerations into llama_flash_attn_type and ggml_type before calling llama_init_from_model.

Change Management: Adapting to Library Updates

Suppose in a new version of llama.cpp, developers decide to optimize the sampling process. They might remove the llama_sampler_init_temp function and replace it with a single, complex initialization function with a different signature.

In a classic monolithic architecture, this would require refactoring all controllers and text generation services. In the InstantAIGate architecture, the upper layers and microservices interact only with the INativeLlamaApi abstraction via Dependency Injection.

The process of adapting to Breaking Changes comes down to two local steps:

Updating the abstraction: In the NativeMethods class, we update the DllImport signature according to the new C++ header file.
Adjusting the mapping: In the NativeLlamaApi class, we rewrite the method logic (e.g., SamplerInitTemp) so that it formats parameters for the new unmanaged function.

The INativeLlamaApi interface remains unchanged. Domain logic, queue orchestration, and REST API endpoints remain unaware that any changes have occurred. This allows for seamless updates and maintains the long-term stability (LTS) of enterprise systems without the need to rewrite application code.

Conclusion

Strict adherence to Clean Architecture principles and the isolation of unmanaged dependencies via interface facades is not just a tribute to academic patterns, but a necessary standard for enterprise-level systems. This approach ensures predictable memory profiling, minimizes risks when updating computational cores, and guarantees the stable operation of a standalone native executable in isolated environments.

In upcoming articles, we will dive deeper into unmanaged memory management and explore the use of SafeHandle to make the codebase even more performant while guaranteeing deterministic resource cleanup.

Bonus: AI Prompt Template for .NET/C++ Interop

If you are building your own infrastructure and need to integrate a rapidly changing C/C++ library into a .NET ecosystem, creating a robust Anti-Corruption Layer manually can be tedious and prone to memory leaks.

You can use the following prompt in ChatGPT, Claude, or Gemini to automatically generate a safe, DDD-compliant interop architecture based on any C/C++ header file.

Copy and paste this prompt into your AI assistant:

System Role: You are a Principal .NET Solution Architect specializing in unmanaged memory, P/Invoke, and Clean Architecture.

Task: I will provide you with a C/C++ header file (.h). Your goal is to design a secure, production-ready C# integration layer using the Anti-Corruption Layer (ACL) pattern.

Strict Requirements:

NativeMethods Class: Create an internal static class named NativeMethods. Translate all C-structs using [StructLayout(LayoutKind.Sequential)] and map all functions using [DllImport]. Do not put any business or orchestration logic in this class.

Domain Abstraction: Create a clean C# interface (e.g., INativeEngineFacade) that uses only safe .NET primitives (strings, enums, standard arrays) instead of raw pointers (IntPtr).

The Facade Implementation: Implement the interface in a Facade class. This class must handle all data marshaling, pointer conversions, and memory boundary crossing between the safe C# domain types and the unmanaged NativeMethods.

Safety & Stability: Emphasize memory leak prevention, correct data type mapping (e.g., C++ bool vs C# bool marshaling), and deterministic execution.

Input: Here is the C/C++ header code:
[INSERT YOUR C/C++ HEADER CODE HERE]

Vectors in AI: A Bridge Between Code and Business

Instancian — Thu, 14 Aug 2025 18:58:34 +0000

Vectors in AI: A Bridge Between Code and Business

Note from the series “About AI for Yourself and Others”

Whether you’re shaping a digital strategy, integrating AI into your workflow, or just curious about what makes an assistant respond intelligently — understanding one simple idea can change the way you think about AI.

That idea is the vector — a compact way AI represents meaning in words, documents, questions, even emotions.

Knowing how vectors work will help you:

Find information without exact keywords
Train AI on your company’s internal knowledge
Automate processes in HR, customer support, and beyond

Let’s see why vectors are at the heart of modern AI — and why they matter for your business.

Example: Vectors in HR — From Filtering to Team Building

In HR, AI has moved far beyond scanning resumes — now it can help you build stronger, more aligned teams. With vector representations, we can work with people in smarter ways — from building stronger teams to checking cultural fit and mapping career growth.

Here’s how it works:

📄 Resumes as vectors — AI transforms each resume into a set of numbers capturing the candidate’s skills, experience, and professional style. For example, someone with extensive project management and team motivation experience will have higher values on those dimensions.
🧾 Job descriptions as vectors — Each job description is also converted into a vector, reflecting tasks, goals, and company culture.
🤖 Semantic matching — AI compares resume and job vectors to see how well they align in meaning and context, not just by shared keywords.

What this gives to HR and business:

⚡ Faster team assembly for projects, using past roles and shared values as a guide.
🧠 Better candidate discovery, even if resumes lack the “right” keywords.
🧭 Cultural fit insights — AI can estimate a candidate’s communication style, preferences, and certain soft skills by analyzing the choice of words and phrasing in their resume.

The Resume of the Future

Not long ago, candidates were advised to keep resumes short and scannable — because recruiters often spent just 6–8 seconds deciding whether to read further or move on.

Today, we’re moving toward rich, detailed project histories — and that’s a good thing.

With vector analysis, AI can go far beyond job titles. It can understand:

📂 Projects a candidate worked on
🏷️ Roles they played in each project
🛠️ Problem-solving approaches they used
📈 Skill growth over time

This means format matters less — AI focuses on the meaning of someone’s experience, even if it’s not in the usual one-page format.

Technical Basics: What Is a Vector?

If you strip away all the AI buzzwords, a vector is simply a way to represent meaning as numbers.

It’s a list of coordinates that describes something — like text, an image, or a question — in a space with hundreds or even thousands of dimensions.

Example: the word “leadership”
After processing by an AI model, it becomes a vector like this:
[0.134, -0.278, 0.045, 0.892, -0.116, 0.003, 0.721, -0.443, 0.056, ...]

You can think of a vector as a GPS coordinate in a city of meanings. Each word, sentence, or document is a location on this map. Words with similar meanings cluster together in neighborhoods, and streets connect related ideas. For example, the word leadership would be located in a “Neighborhood of Leadership,” surrounded by streets and blocks where related words live — management, motivation, and teamwork. Nearby streets connect to other neighborhoods, like “Communication” or “Strategy,” allowing AI to navigate from one concept to another, finding words or ideas that are semantically close, even if the exact words differ.

In short, AI doesn’t look for exact words — it looks for similar meanings, navigating the “city of words” to understand context and relationships.

These vectors don’t appear out of nowhere. They are created by specially trained AI models, most commonly variants of BERT and its successors. The model reads text (or other data) and assigns each word, sentence, or document a position in the “city of meanings” we described. You could also use other models, like Gemma3, but for most business applications this can be overhead — BERT-based models or their lighter variants usually provide a good balance between quality and efficiency.

Models That Create Vectors

Model	Year	Developer	Notes
RoBERTa	2019	Facebook	Trained on more data than BERT
ALBERT	2019	Google	Smaller size, similar accuracy
DistilBERT	2019	HuggingFace	Faster and lighter, good quality
E5	2022	Microsoft	Made for search and RAG tasks
OpenAI Ada v2/v3	2022–24	OpenAI	Works well with large vector databases
InstructorXL	2023	HuggingFace/Alibaba	Adapts to specific tasks

How It Works in a Business App (5 Steps)

Text → Model → Vector

A document or query is transformed into a vector by an AI model. This vector captures the semantic meaning of the text in numerical form.
Save in Database

The vector is stored alongside the original text — for example, in PostgreSQL using pgvector.
User Query → Vector

When a user asks a question, the query is also transformed into a vector by the same model, ensuring that both documents and queries are in the same semantic space.
Compare Vectors

The system compares the query vector with stored vectors to find the closest matches:
- Using SQL — e.g., ORDER BY embedding <-> query_vector LIMIT 5 in PostgreSQL with pgvector. This returns the nearest vectors based on numerical similarity.
- Using RAG (Retrieval-Augmented Generation) — an AI assistant first retrieves the most relevant documents via vector similarity, then generates answers based on them.
Return Result

The system delivers the most relevant matches to the user — for example, a list of similar documents, products, or candidates.
If connected to an AI assistant, these results can be used as context to generate a detailed, personalized answer.

This approach allows AI to retrieve documents or generate answers based on meaning, not just exact keyword matches. Advanced topics like indexing, normalization, and fine-tuning can improve efficiency and accuracy, but these five steps cover the core idea.

HR is a familiar use case for vector-based AI — and yes, perhaps a bit overused. Still, it offers a clear starting point. In future articles, we’ll move beyond the resume and dive into more original, business-critical applications: semantic email processing, the design of executive assistants with real accountability, and the improvement of workflows.

Want to build your own semantic assistant? Let’s explore it together in the next article.

Programmer’s Demons — and How to Defeat Them

Instancian — Thu, 31 Jul 2025 12:58:44 +0000

Introduction: Thought Patterns That Interfere With Architectural Decisions

When working in system architecture without external guidance, decision-making depends entirely on your ability to stay clear-minded and focused. Yet clarity is constantly challenged. Certain thought patterns — recurring, subtle, and often embedded in team culture — can undermine productive action.

They don’t announce themselves. Instead, they show up as familiar hesitations, internal arguments, or inherited habits:

“What if we need this later?” — defers important clean-up decisions and feeds technical debt.
“Abstraction for abstraction’s sake” — adds complexity without increasing clarity or value.
“We must do it the right way” — slows momentum under the weight of imagined ideals.

This article is a practical snapshot of recurring thought patterns encountered by architects working in high-autonomy environments. In such contexts, clarity isn’t pre-approved — it’s earned through thoughtful, independent decision-making. Identifying these patterns helps build stronger judgment, reduce friction, and support more confident architectural choices.

Demon 1: “What if we need it later?”

The main supplier of useless layers, DI wrappers, and microservices.

What it says:

“Make it flexible now. Otherwise, we’ll rewrite everything later.”
“What if we scale to millions of requests?”
“What if we have 7 teams next year?”

🛡️ How to fight it:

Ask: “What do we need this month?”
Motto: “When ‘later’ comes — we’ll build it. For now — go straight.”

Demon 2: Abstraction for abstraction’s sake

“Add an interface, just in case the class changes.”

What it says:

“You can’t call a class directly — abstract it.”
“Add a layer on top of the layer, just in case.”

🛡️ How to fight it:

Practice: “Use interfaces only when there are real alternatives.”
Principle: “Keep it simple until it hurts.”

Demon 3: “This is how enterprise does it”

“If Google does it — we must too.”

What it says:

“Clean Architecture is the only way.”
“Without CQRS, DI, EventBus — you’re not a real architect.”

🛡️ How to fight it:

Realize: “Their scale ≠ your scale.”
Practice: “Write for your reality, not for a mythical industry.”

Demon 4: “Let’s make it perfect — then release”

The guardian of eternal refactoring and dead MVPs.

What it says:

“Just a bit more — then it’ll be perfect.”
“It’s embarrassing to show it like this.”
“We need to rethink the architecture…”

🛡️ How to fight it:

Test: “Can you show this to a user tomorrow?”
Motto: “Done > Perfect”

Demon 5: “We must do it right, no mistakes”

The source of decision paralysis.

🧟 What it says:

“What if this is the wrong approach?”
“If it’s not SOLID, they’ll laugh.”
“Check 4 more options before choosing.”

🎯 How to fight it:

Mindset: “Decisions are not sacred — you can change them.”
Phrase: “Wrong is okay. Useless is not.”
Clarification: You always have the right to make a mistake. The real issue is repeating the same mistake three times.

Practices of Liberation

1. Morning Focus Protocol

Before diving into code — remind yourself why you’re doing this.

Why am I writing this? (Who is it for? What problem does it solve?)
What can I do simply today?
Where can I stop overthinking?

🧭 Shift from “engineer of engineering” to “creator of meaning.”

2. Task Mirror

Before adding a layer — ask:

Does this abstraction solve a real pain — or a hypothetical one?
Am I complicating out of fear or necessity?
Can I solve this right now in a simpler way?

📌 Put this in your IDE as a snippet or sticky note.

3. Release Pressure from Perfectionism

“Code doesn’t need to last forever. It needs to work now — clearly and usefully.”

⛔ Not Needed	✅ Enough for Today
Perfect	Useful
Architecturally fancy	Architecturally clear

🧭 Designing for eternity is a trap. In fast-moving environments — where frameworks shift, APIs deprecate, and teams evolve — even well-crafted code can become legacy within 2–3 years. Your product doesn’t need to last forever. It needs to be clear today, useful tomorrow, and replaceable when the world moves on.

4. Log of Useful Traces

At the end of the day — not “what I did,” but “what became useful.”

Examples:

Simplified generator template — less confusion
Added examples to README — easier to use
Removed extra DI layer — easier to read

🧠 Why it's useful to log what became useful: You’re training your brain to notice value — not just effort. By shifting focus from “what I did” to “what helped,” you build mental patterns that prioritize clarity, contribution, and self-trust. This reduces stress in high-pressure situations. You’re not just working — you’re learning. Not just how to deliver outcomes, but how to shape your way of being. Each log is proof that progress can be simple, even when circumstances aren’t.