Using Microsoft.Extensions.AI to Build Provider-Agnostic AI Applications in .NET

Introduction

In the previous article, we built a secure AI-ready chat API using ASP.NET Core. We used a custom interface called IAiChatService and implemented a local MockAiChatService so the project could run without Azure OpenAI quota.

That was a good first step.

In this article, we will take the next step and use Microsoft’s official AI abstraction approach with Microsoft.Extensions.AI.

The main goal is simple:

Build AI applications where the business code does not directly depend on one specific AI provider SDK.

Instead of writing application code that is tightly coupled to Azure OpenAI, OpenAI, Ollama, GitHub Models, or any other provider, we can build against a common abstraction such as IChatClient.

This makes the application cleaner, easier to test, and easier to change later.

Why Provider-Agnostic AI Design Matters

In many enterprise applications, the AI provider may change over time.

Today, the application may use:

Mock AI service for local development

Tomorrow, it may use:

Azure OpenAI

Later, it may use:

OpenAI
Local model
GitHub Models
Another OpenAI-compatible endpoint

If the controller or business service directly depends on one specific SDK, changing providers becomes difficult.

Bad design:

ChatController
|
v
Azure OpenAI SDK directly

Better design:

ChatController
|
v
IChatClient
|
v
AI Provider Implementation

This is where Microsoft.Extensions.AI becomes useful.

What Is Microsoft.Extensions.AI?

Microsoft.Extensions.AI provides common abstractions for AI services in .NET applications.

It includes abstractions such as:

IChatClient
IEmbeddingGenerator
ChatMessage
ChatResponse
ChatOptions

The Microsoft.Extensions.AI.Abstractions package provides core exchange types, including IChatClient and IEmbeddingGenerator. Any .NET library that provides an LLM client can implement IChatClient so consuming code can integrate with it more easily.

The important idea is:

Application code should depend on abstractions, not directly on provider-specific SDKs.

What We Will Build

We will build an ASP.NET Core Web API that uses IChatClient.

For local development, we will create a custom MockChatClient.

The flow looks like this:

The controller does not need to know which provider is behind IChatClient.

Project Setup

Create a new ASP.NET Core Web API project.

dotnet new webapi -n ProviderAgnosticAi.Api
cd ProviderAgnosticAi.Api

Add the required packages:

dotnet add package Microsoft.Extensions.AI.Abstractions
dotnet add package Swashbuckle.AspNetCore

For a future OpenAI-compatible implementation, you can also add:

_dotnet add package Microsoft.Extensions.AI.OpenAI
dotnet add package Azure.AI.OpenAI
dotnet add package Azure.Identity
_
For this article, we will keep the project runnable locally using a mock IChatClient.

Recommended Project Structure

ProviderAgnosticAi.Api
│
├── Controllers
│ └── ChatController.cs
│
├── Models
│ ├── ChatRequestDto.cs
│ └── ChatResponseDto.cs
│
├── Services
│ ├── MockChatClient.cs
│ ├── IProviderAgnosticChatService.cs
│ └── ProviderAgnosticChatService.cs
│
└── Program.cs

Create the Request Model

Create a file:

Models/ChatRequestDto.cs

namespace ProviderAgnosticAi.Api.Models;

public class ChatRequestDto
{
    public string UserMessage { get; set; } = string.Empty;
}

This model represents the user input.

Create the Response Model

Create a file:

Models/ChatResponseDto.cs

namespace ProviderAgnosticAi.Api.Models;

public class ChatResponseDto
{
    public bool Success { get; set; }

    public string Answer { get; set; } = string.Empty;

    public string Provider { get; set; } = string.Empty;

    public string? ErrorMessage { get; set; }
}

The Provider field helps us identify which implementation generated the response.

For local development, it will show:

MockChatClient

Later, it can show an Azure OpenAI or OpenAI-backed client.

Create a Mock IChatClient

Now create a mock implementation of Microsoft’s IChatClient.

Create a file:

Services/MockChatClient.cs

using Microsoft.Extensions.AI;

namespace ProviderAgnosticAi.Api.Services;

public class MockChatClient : IChatClient
{
    public Task<ChatResponse> GetResponseAsync(
        IEnumerable<ChatMessage> messages,
        ChatOptions? options = null,
        CancellationToken cancellationToken = default)
    {
        var userMessage = messages
            .LastOrDefault(m => m.Role == ChatRole.User)?
            .Text ?? string.Empty;

        string answer;

        if (userMessage.Contains("provider", StringComparison.OrdinalIgnoreCase))
        {
            answer =
                "Provider-agnostic AI design means your application depends on an abstraction like IChatClient, " +
                "instead of directly depending on one AI provider SDK.";
        }
        else if (userMessage.Contains("Microsoft.Extensions.AI", StringComparison.OrdinalIgnoreCase))
        {
            answer =
                "Microsoft.Extensions.AI provides common abstractions such as IChatClient and IEmbeddingGenerator " +
                "so .NET applications can integrate AI services using familiar dependency injection patterns.";
        }
        else if (userMessage.Contains("azure", StringComparison.OrdinalIgnoreCase))
        {
            answer =
                "Azure OpenAI can be used behind the same IChatClient abstraction when a deployment and quota are available.";
        }
        else
        {
            answer =
                "This is a local mock response using IChatClient. The controller depends on Microsoft.Extensions.AI abstractions, " +
                "so the implementation can later be replaced with Azure OpenAI, OpenAI, or another compatible provider.";
        }

        var response = new ChatResponse(
            new ChatMessage(ChatRole.Assistant, answer));

        return Task.FromResult(response);
    }

    public IAsyncEnumerable<ChatResponseUpdate> GetStreamingResponseAsync(
        IEnumerable<ChatMessage> messages,
        ChatOptions? options = null,
        CancellationToken cancellationToken = default)
    {
        throw new NotImplementedException(
            "Streaming is not implemented in the mock client for this demo.");
    }

    public object? GetService(Type serviceType, object? serviceKey = null)
    {
        return null;
    }

    public void Dispose()
    {
    }
}

This mock client lets us test the application without Azure OpenAI quota.

Create Application Service Interface

Create a file:

Services/IProviderAgnosticChatService.cs

using ProviderAgnosticAi.Api.Models;

namespace ProviderAgnosticAi.Api.Services;

public interface IProviderAgnosticChatService
{
    Task<ChatResponseDto> AskAsync(
        ChatRequestDto request,
        CancellationToken cancellationToken);
}

This service keeps the controller clean.

Create Application Service Implementation

Create a file:

Services/ProviderAgnosticChatService.cs

using Microsoft.Extensions.AI;
using ProviderAgnosticAi.Api.Models;

namespace ProviderAgnosticAi.Api.Services;

public class ProviderAgnosticChatService : IProviderAgnosticChatService
{
    private readonly IChatClient _chatClient;
    private readonly ILogger<ProviderAgnosticChatService> _logger;

    public ProviderAgnosticChatService(
        IChatClient chatClient,
        ILogger<ProviderAgnosticChatService> logger)
    {
        _chatClient = chatClient;
        _logger = logger;
    }

    public async Task<ChatResponseDto> AskAsync(
        ChatRequestDto request,
        CancellationToken cancellationToken)
    {
        try
        {
            if (string.IsNullOrWhiteSpace(request.UserMessage))
            {
                return new ChatResponseDto
                {
                    Success = false,
                    ErrorMessage = "User message is required.",
                    Provider = "IChatClient"
                };
            }

            var messages = new List<ChatMessage>
            {
                new(ChatRole.System,
                    "You are a helpful enterprise AI assistant. " +
                    "Explain concepts clearly and professionally."),

                new(ChatRole.User, request.UserMessage)
            };

            ChatResponse response =
                await _chatClient.GetResponseAsync(
                    messages,
                    cancellationToken: cancellationToken);

            return new ChatResponseDto
            {
                Success = true,
                Answer = response.Text,
                Provider = _chatClient.GetType().Name,
                ErrorMessage = null
            };
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Provider-agnostic AI chat failed.");

            return new ChatResponseDto
            {
                Success = false,
                Provider = _chatClient.GetType().Name,
                ErrorMessage = "Unable to generate a response at this time."
            };
        }
    }
}

This service depends on IChatClient.

It does not care whether the implementation is:

MockChatClient
Azure OpenAI client
OpenAI client
Local model client

That is the main benefit of provider-agnostic design.

Create the API Controller

Create a file:

Controllers/ChatController.cs

using Microsoft.AspNetCore.Mvc;
using ProviderAgnosticAi.Api.Models;
using ProviderAgnosticAi.Api.Services;

namespace ProviderAgnosticAi.Api.Controllers;

[ApiController]
[Route("api/provider-agnostic-chat")]
public class ChatController : ControllerBase
{
    private readonly IProviderAgnosticChatService _chatService;

    public ChatController(IProviderAgnosticChatService chatService)
    {
        _chatService = chatService;
    }

    [HttpPost]
    public async Task<IActionResult> Ask(
        [FromBody] ChatRequestDto request,
        CancellationToken cancellationToken)
    {
        ChatResponseDto response =
            await _chatService.AskAsync(request, cancellationToken);

        if (!response.Success)
        {
            return BadRequest(response);
        }

        return Ok(response);
    }
}

The controller only talks to the application service.

The application service talks to IChatClient.

This creates a clean separation of concerns.

Update Program.cs

Replace Program.cs with the following:

using Microsoft.Extensions.AI;
using ProviderAgnosticAi.Api.Services;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddControllers();

builder.Services.AddSingleton<IChatClient, MockChatClient>();
builder.Services.AddScoped<IProviderAgnosticChatService, ProviderAgnosticChatService>();

builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();

var app = builder.Build();

app.UseSwagger();
app.UseSwaggerUI();

app.UseHttpsRedirection();

app.MapControllers();

app.Run();

Run and Test the API

Run the application:

dotnet run

Open Swagger:

https://localhost:/swagger

Test the endpoint:

POST /api/provider-agnostic-chat

Request:

{
"userMessage": "What is provider-agnostic AI design in .NET?"
}

Expected response:

{
"success": true,
"answer": "Provider-agnostic AI design means your application depends on an abstraction like IChatClient, instead of directly depending on one AI provider SDK.",
"provider": "MockChatClient",
"errorMessage": null
}

Test another request:

{
"userMessage": "What is Microsoft.Extensions.AI?"
}

Expected response:

{
"success": true,
"answer": "Microsoft.Extensions.AI provides common abstractions such as IChatClient and IEmbeddingGenerator so .NET applications can integrate AI services using familiar dependency injection patterns.",
"provider": "MockChatClient",
"errorMessage": null
}

Where Microsoft.Extensions.AI Fits in Enterprise Architecture

A provider-agnostic AI architecture can look like this:

This pattern is useful when:

Azure quota is not available during development
Teams want local testing
The organization may change model providers later
You want to mock AI responses in unit tests
You want to centralize logging and telemetry
You want clean dependency injection

Security and Enterprise Considerations

Even when using abstractions, we still need enterprise controls.

Recommended practices:

Do not call AI providers directly from the frontend.
Keep AI access behind ASP.NET Core APIs.
Do not store provider keys in source code.
Use local mock providers for development and testing.
Use Azure Key Vault or Managed Identity for production secrets.
Validate user input before calling the model.
Avoid logging full prompts if they contain sensitive information.
Track model usage, latency, and errors.
Add rate limiting to control cost.
Use RAG for company-specific answers.

Microsoft.Extensions.AI helps with abstraction, but architecture, security, and governance still need to be designed carefully.

Conclusion

In this article, we built a provider-agnostic AI API using ASP.NET Core and Microsoft.Extensions.AI.

The main design principle is:

Application code should depend on AI abstractions, not directly on one provider SDK.

By using IChatClient, our application becomes easier to test, easier to extend, and easier to switch to another provider later.

For local development, we used:

MockChatClient

For production, we can later replace it with an Azure OpenAI-backed implementation.

This approach is useful for enterprise developers because it supports clean architecture, dependency injection, local testing, provider flexibility, and long-term maintainability.

Github Link : https://github.com/indirakumar710/ProviderAgnosticAi.Api