Reliable AI Outputs: Function Calling, JSON Mode, and Structured Generation in C#

LLMs are impressive text generators, but production applications need more than prose. You need structured data—JSON objects you can deserialize, decisions you can act on, and outputs that fit your domain models.

This is where function calling and structured outputs transform LLMs from chatbots into programmable decision engines.

The Reliability Problem

Ask an LLM to extract product information from a review, and you might get:

The product seems good. Rating: probably 4 stars. Pros include durability.

Or sometimes:

{"rating": 4, "pros": ["durability"]}

Or even:

Here is the JSON you requested:
{"rating": "four", "pros": "durable"}

This inconsistency breaks production code. You need guarantees—not hope.

Function Calling: Turning LLMs into Decision Engines

Function calling lets you define actions the LLM can invoke. Instead of generating text that describes what to do, the model returns structured function calls that your code executes.

Defining Functions with Attributes

Microsoft.Extensions.AI uses attributes to define callable functions:

public class OrderFunctions
{
    private readonly IOrderRepository _orders;
    private readonly IShippingService _shipping;

    public OrderFunctions(IOrderRepository orders, IShippingService shipping)
    {
        _orders = orders;
        _shipping = shipping;
    }

    [Description("Get the current status and details of an order")]
    public async Task<OrderInfo> GetOrderStatusAsync(
        [Description("The order ID (e.g., ORD-12345)")] string orderId)
    {
        var order = await _orders.GetByIdAsync(orderId);
        if (order == null)
            return new OrderInfo { Found = false, Message = $"Order {orderId} not found" };

        return new OrderInfo
        {
            Found = true,
            OrderId = order.Id,
            Status = order.Status.ToString(),
            Items = order.Items.Select(i => i.Name).ToList(),
            Total = order.Total,
            EstimatedDelivery = order.EstimatedDelivery
        };
    }

    [Description("Initiate a return for an order")]
    public async Task<ReturnResult> InitiateReturnAsync(
        [Description("The order ID to return")] string orderId,
        [Description("Reason for the return")] string reason,
        [Description("Items to return (empty = all items)")] List<string>? items = null)
    {
        var order = await _orders.GetByIdAsync(orderId);
        if (order == null)
            return new ReturnResult { Success = false, Message = "Order not found" };

        if (order.Status == OrderStatus.Delivered && 
            order.DeliveryDate < DateTime.UtcNow.AddDays(-30))
        {
            return new ReturnResult 
            { 
                Success = false, 
                Message = "Return window has expired (30 days)" 
            };
        }

        var returnId = await _orders.CreateReturnAsync(orderId, reason, items);

        return new ReturnResult
        {
            Success = true,
            ReturnId = returnId,
            Message = "Return initiated. You will receive a shipping label via email."
        };
    }

    [Description("Get shipping rates for an address")]
    public async Task<ShippingRates> GetShippingRatesAsync(
        [Description("Destination ZIP code")] string zipCode,
        [Description("Package weight in pounds")] double weightLbs)
    {
        var rates = await _shipping.GetRatesAsync(zipCode, weightLbs);
        return new ShippingRates
        {
            Standard = rates.Standard,
            Express = rates.Express,
            Overnight = rates.Overnight
        };
    }
}

Registering Functions with ChatOptions

Convert your function class into AI tools:

var orderFunctions = new OrderFunctions(orderRepo, shippingService);

var options = new ChatOptions
{
    Tools = AIFunctionFactory.CreateFromInstance(orderFunctions)
};

You can also create functions from static methods or individual delegates:

// From a type (static methods)
var tools = AIFunctionFactory.CreateFromType<StaticFunctions>();

// From a specific method
var tool = AIFunctionFactory.Create(
    (string city) => weatherService.GetWeatherAsync(city),
    "get_weather",
    "Get current weather for a city");

The Function Calling Loop

When you send a message with tools, the model may respond with function calls instead of text. You need to execute them and feed results back:

public class FunctionCallingChatService
{
    private readonly IChatClient _chatClient;
    private readonly ChatOptions _options;

    public FunctionCallingChatService(
        IChatClient chatClient, 
        OrderFunctions orderFunctions)
    {
        _chatClient = chatClient;
        _options = new ChatOptions
        {
            Tools = AIFunctionFactory.CreateFromInstance(orderFunctions),
            ToolMode = ChatToolMode.Auto // Let model decide when to use tools
        };
    }

    public async Task<string> ProcessAsync(string userMessage)
    {
        var messages = new List<ChatMessage>
        {
            new(ChatRole.System, """
                You are a customer support agent for an e-commerce store.
                Use the available tools to help customers with their orders.
                Always verify order information before taking actions.
                Be concise and helpful.
                """),
            new(ChatRole.User, userMessage)
        };

        const int maxIterations = 10; // Prevent infinite loops

        for (int i = 0; i < maxIterations; i++)
        {
            var response = await _chatClient.CompleteAsync(messages, _options);
            messages.Add(response.Message);

            // Extract function calls from the response
            var functionCalls = response.Message.Contents
                .OfType<FunctionCallContent>()
                .ToList();

            // If no function calls, we have our final answer
            if (functionCalls.Count == 0)
            {
                return response.Message.Text ?? string.Empty;
            }

            // Execute each function call
            foreach (var call in functionCalls)
            {
                object? result;
                try
                {
                    result = await call.InvokeAsync();
                }
                catch (Exception ex)
                {
                    result = new { error = ex.Message };
                }

                // Add the result back to the conversation
                messages.Add(new ChatMessage(ChatRole.Tool, new[]
                {
                    new FunctionResultContent(call.CallId, call.Name, result)
                }));
            }
        }

        throw new InvalidOperationException("Max iterations exceeded");
    }
}

Controlling Tool Behavior

Use ChatToolMode to control when tools are used:

// Let the model decide
options.ToolMode = ChatToolMode.Auto;

// Force a specific tool
options.ToolMode = ChatToolMode.RequireSpecific("get_order_status");

// Require some tool to be called (model chooses which)
options.ToolMode = ChatToolMode.RequireAny;

// Never use tools this turn
options.ToolMode = ChatToolMode.None;

Structured Outputs with JSON Mode

Function calling is powerful for actions, but sometimes you just need structured data extraction. JSON mode forces the model to output valid JSON matching a schema.

Basic JSON Mode

public record ProductReview(
    string Summary,
    int Rating,
    List<string> Pros,
    List<string> Cons,
    bool Recommended,
    string Sentiment);

public async Task<ProductReview?> AnalyzeReviewAsync(string reviewText)
{
    var options = new ChatOptions
    {
        ResponseFormat = ChatResponseFormat.ForJsonSchema<ProductReview>()
    };

    var messages = new List<ChatMessage>
    {
        new(ChatRole.System, """
            Analyze the product review and extract structured information.
            Rating should be 1-5.
            Sentiment should be: positive, negative, or mixed.
            """),
        new(ChatRole.User, reviewText)
    };

    var response = await _chatClient.CompleteAsync(messages, options);

    return JsonSerializer.Deserialize<ProductReview>(response.Message.Text!);
}

Complex Nested Schemas

JSON mode handles complex, nested types:

public record Invoice(
    string InvoiceNumber,
    DateTime Date,
    Customer Customer,
    List<LineItem> Items,
    decimal Subtotal,
    decimal Tax,
    decimal Total,
    PaymentTerms Terms);

public record Customer(
    string Name,
    string Email,
    Address BillingAddress);

public record Address(
    string Street,
    string City,
    string State,
    string ZipCode,
    string Country);

public record LineItem(
    string Description,
    int Quantity,
    decimal UnitPrice,
    decimal Total);

public enum PaymentTerms { Net30, Net60, DueOnReceipt }

public async Task<Invoice?> ExtractInvoiceAsync(string invoiceText)
{
    var options = new ChatOptions
    {
        ResponseFormat = ChatResponseFormat.ForJsonSchema<Invoice>()
    };

    var response = await _chatClient.CompleteAsync(
        new ChatMessage(ChatRole.User, $"Extract invoice data:\n\n{invoiceText}"),
        options);

    return JsonSerializer.Deserialize<Invoice>(
        response.Message.Text!,
        new JsonSerializerOptions { PropertyNameCaseInsensitive = true });
}

Schema Generation for AOT

For ahead-of-time compilation or when you need explicit schema control:

[JsonSerializable(typeof(ProductReview))]
[JsonSerializable(typeof(Invoice))]
public partial class AppJsonContext : JsonSerializerContext { }

// Use with explicit schema
var schema = JsonSchema.FromType<ProductReview>(new JsonSchemaOptions
{
    SerializerContext = AppJsonContext.Default
});

var options = new ChatOptions
{
    ResponseFormat = ChatResponseFormat.ForJsonSchema(schema, "ProductReview")
};

Validation and Error Handling

LLM outputs can still fail validation even with structured output. Build robust handling:

public class StructuredOutputService<T> where T : class
{
    private readonly IChatClient _chatClient;
    private readonly ILogger<StructuredOutputService<T>> _logger;
    private readonly JsonSerializerOptions _jsonOptions;

    public StructuredOutputService(
        IChatClient chatClient,
        ILogger<StructuredOutputService<T>> logger)
    {
        _chatClient = chatClient;
        _logger = logger;
        _jsonOptions = new JsonSerializerOptions
        {
            PropertyNameCaseInsensitive = true,
            Converters = { new JsonStringEnumConverter() }
        };
    }

    public async Task<Result<T>> ExtractAsync(
        string prompt,
        string content,
        int maxRetries = 3)
    {
        var options = new ChatOptions
        {
            ResponseFormat = ChatResponseFormat.ForJsonSchema<T>()
        };

        List<string> errors = new();

        for (int attempt = 1; attempt <= maxRetries; attempt++)
        {
            try
            {
                var messages = new List<ChatMessage>
                {
                    new(ChatRole.System, prompt)
                };

                // On retry, include previous errors
                if (errors.Count > 0)
                {
                    messages.Add(new(ChatRole.System, 
                        $"Previous attempts failed with: {string.Join("; ", errors)}. " +
                        "Please fix these issues."));
                }

                messages.Add(new(ChatRole.User, content));

                var response = await _chatClient.CompleteAsync(messages, options);
                var text = response.Message.Text;

                if (string.IsNullOrWhiteSpace(text))
                {
                    errors.Add("Empty response");
                    continue;
                }

                var result = JsonSerializer.Deserialize<T>(text, _jsonOptions);

                if (result == null)
                {
                    errors.Add("Deserialization returned null");
                    continue;
                }

                // Custom validation
                var validationErrors = Validate(result);
                if (validationErrors.Any())
                {
                    errors.AddRange(validationErrors);
                    continue;
                }

                return Result<T>.Ok(result);
            }
            catch (JsonException ex)
            {
                _logger.LogWarning(ex, 
                    "JSON parsing failed on attempt {Attempt}", attempt);
                errors.Add($"JSON error: {ex.Message}");
            }
            catch (Exception ex)
            {
                _logger.LogError(ex, 
                    "Unexpected error on attempt {Attempt}", attempt);
                throw;
            }
        }

        return Result<T>.Fail($"Failed after {maxRetries} attempts: {string.Join("; ", errors)}");
    }

    private IEnumerable<string> Validate(T result)
    {
        var context = new ValidationContext(result);
        var results = new List<ValidationResult>();

        if (!Validator.TryValidateObject(result, context, results, true))
        {
            return results.Select(r => r.ErrorMessage ?? "Validation failed");
        }

        return Enumerable.Empty<string>();
    }
}

public record Result<T>
{
    public bool Success { get; init; }
    public T? Value { get; init; }
    public string? Error { get; init; }

    public static Result<T> Ok(T value) => new() { Success = true, Value = value };
    public static Result<T> Fail(string error) => new() { Success = false, Error = error };
}

Token-Efficient Function Definitions

Function definitions consume tokens. Optimize them for cost:

// ❌ BAD: Overly verbose
[Description("This function retrieves the complete and comprehensive shipping " +
    "status including the full tracking number, the name of the carrier company, " +
    "the estimated delivery date and time, the current geographic location of the " +
    "package, and all historical tracking events that have occurred for the " +
    "specified order identified by the order ID parameter")]
public async Task<ShippingStatus> GetShippingStatusVerbose(string orderId)

// ✅ GOOD: Concise but clear
[Description("Get shipping status and tracking for an order")]
public async Task<ShippingStatus> GetShippingStatus(
    [Description("Order ID (e.g., ORD-12345)")] string orderId)

Minimize Parameter Descriptions

// ❌ Redundant - the type says it all
[Description("A string containing the email address of the user")]
string email

// ✅ Only add what the type doesn't tell you
[Description("User email")]
string email

// ✅ Or when format matters
[Description("Email (must be verified)")] 
string email

Use Enums for Fixed Options

// ❌ String with description
[Description("The priority level: can be low, medium, high, or urgent")]
string priority

// ✅ Enum - self-documenting and validated
public enum Priority { Low, Medium, High, Urgent }
public async Task CreateTicket(Priority priority)

Parallel Function Calls

Modern models can request multiple function calls simultaneously. Handle them efficiently:

public async Task<string> ProcessWithParallelCallsAsync(string userMessage)
{
    var messages = new List<ChatMessage>
    {
        new(ChatRole.System, "You are a helpful assistant. " +
            "You can call multiple functions in parallel when appropriate."),
        new(ChatRole.User, userMessage)
    };

    while (true)
    {
        var response = await _chatClient.CompleteAsync(messages, _options);
        messages.Add(response.Message);

        var calls = response.Message.Contents
            .OfType<FunctionCallContent>()
            .ToList();

        if (calls.Count == 0)
            return response.Message.Text ?? "";

        // Execute all calls in parallel
        var tasks = calls.Select(async call =>
        {
            try
            {
                var result = await call.InvokeAsync();
                return new FunctionResultContent(call.CallId, call.Name, result);
            }
            catch (Exception ex)
            {
                return new FunctionResultContent(
                    call.CallId, call.Name, new { error = ex.Message });
            }
        });

        var results = await Task.WhenAll(tasks);

        // Add all results in a single message
        messages.Add(new ChatMessage(ChatRole.Tool, results));
    }
}

Combining Functions and Structured Output

Use functions for actions and structured output for the final response:

public record SupportResponse(
    string Summary,
    List<ActionTaken> Actions,
    string NextSteps,
    bool EscalationNeeded);

public record ActionTaken(string Action, bool Success, string Details);

public async Task<SupportResponse> HandleSupportRequestAsync(string request)
{
    // Phase 1: Let the model use tools to gather info and take actions
    var conversationResult = await ProcessWithToolsAsync(request);

    // Phase 2: Get a structured summary
    var summaryOptions = new ChatOptions
    {
        ResponseFormat = ChatResponseFormat.ForJsonSchema<SupportResponse>()
    };

    var summaryMessages = new List<ChatMessage>
    {
        new(ChatRole.System, """
            Summarize the support interaction in structured format.
            Include all actions taken and their outcomes.
            Determine if escalation to a human agent is needed.
            """),
        new(ChatRole.User, $"Original request: {request}\n\n" +
            $"Conversation and actions:\n{conversationResult}")
    };

    var response = await _chatClient.CompleteAsync(summaryMessages, summaryOptions);

    return JsonSerializer.Deserialize<SupportResponse>(response.Message.Text!)!;
}

Best Practices Summary

Function Calling

1. Keep functions focused - Single responsibility per function
2. Return actionable data - Include success/failure status and messages
3. Handle errors gracefully - Catch exceptions and return error info
4. Set iteration limits - Prevent infinite loops in the calling loop
5. Use enums over strings - For parameters with fixed options

Structured Output

1. Design for the model - Keep schemas simple and well-documented
2. Validate extensively - Don't trust the output blindly
3. Implement retries - Models occasionally fail; retry with error feedback
4. Use nullable wisely - Mark optional fields as nullable
5. Consider partial extraction - Sometimes you don't need all fields

Token Efficiency

1. Be concise - Every token in function definitions costs money
2. Use defaults - Let parameters have sensible defaults
3. Batch when possible - One function returning a list vs. many calls
4. Cache schemas - Generate JSON schemas once at startup

What's Next

In Part 3, we'll tackle conversation patterns—managing chat history, context windows, and building stateful conversational experiences that maintain coherence across multiple turns.

We'll cover:

• Conversation state management strategies
• Context window management and summarization
• System prompt design patterns
• Multi-turn conversation best practices
• Session isolation and security

---

This is Part 2 of the "Generative AI Patterns in C#" series. Function calling and structured outputs are the foundation of reliable AI applications—master these, and you'll build systems that actually work in production.