DEV Community

Cover image for Building AI Agents for Automated Trading Systems Using .NET C# Semantic Kernel, Azure AI Services, and Azure Functions
Paulo Torres
Paulo Torres

Posted on

Building AI Agents for Automated Trading Systems Using .NET C# Semantic Kernel, Azure AI Services, and Azure Functions

#ai #machinelearning #azure #python

In the dynamic world of financial trading, leveraging Artificial Intelligence (AI) to automate trading strategies has become a game-changer. Integrating AI agents into trading systems allows for real-time data analysis, predictive modeling, and autonomous decision-making. This comprehensive guide explores how to build a real-world AI agent for automated trading using .NET C# Semantic Kernel, .NET Core C# 8, ASP.NET Core Web API, Azure AI Services, Azure Functions, Azure Key Vault, Azure Cosmos DB (MongoDB API), Azure Kubernetes Service (AKS), Python, and other relevant technologies.

Table of Contents

  1. Introduction
  2. Prerequisites
  3. Architecture Overview
  4. Setting Up the Development Environment
  5. Developing the AI Model with Python and Azure Machine Learning
  6. Utilizing .NET C# Semantic Kernel for AI Agents
  7. Creating the .NET Core Web API
  8. Integrating the AI Model into the .NET Core Application
  9. Implementing Data Storage with Azure Cosmos DB
  10. Securing Secrets with Azure Key Vault
  11. Using Azure Functions for Event-Driven Processing
  12. Containerizing the Application with Docker
  13. Deploying to Azure Kubernetes Service (AKS)
  14. Monitoring and Logging
  15. Conclusion

Introduction

Automated trading systems utilize algorithms and AI models to execute trades based on predefined strategies without human intervention. By incorporating AI agents, these systems can learn from historical data, predict market trends, and make autonomous decisions. This guide demonstrates how to build such an AI-powered trading agent using cutting-edge technologies and cloud services, including the .NET C# Semantic Kernel, which facilitates the integration of AI models into .NET applications.

Prerequisites

  • Azure Subscription: Access to Microsoft Azure services.
  • Development Tools:
    • Visual Studio 2022 or Visual Studio Code with .NET Core SDK installed.
    • Python 3.8+ with necessary libraries.
  • Azure CLI: For managing Azure resources from the command line.
  • Docker: Installed on your development machine.
  • Azure Kubernetes Service (AKS): Basic understanding.
  • .NET C# Semantic Kernel: Installed and configured.

Architecture Overview

The architecture consists of the following components:

  • AI Model: Developed in Python using Azure Machine Learning.
  • .NET C# Semantic Kernel: Integrates AI capabilities into the .NET Core application.
  • ASP.NET Core Web API: Serves as the backend, interacting with the AI agent and handling trade executions.
  • Azure Cosmos DB: Stores trading data and model predictions.
  • Azure Key Vault: Secures sensitive information like API keys and connection strings.
  • Azure Functions: Handles event-driven processes such as real-time data ingestion.
  • Azure Kubernetes Service (AKS): Hosts the containerized application for scalability and high availability.
  • Azure AI Services: Provides additional AI capabilities if needed.

Setting Up the Development Environment

Install .NET Core SDK and Tools

  • Download and install the latest .NET Core SDK.
  • Install Visual Studio 2022 or Visual Studio Code.

Install .NET C# Semantic Kernel

  • Clone the Semantic Kernel repository: 
  git clone https://github.com/microsoft/semantic-kernel.git
Enter fullscreen mode Exit fullscreen mode
  • Build and install the Semantic Kernel library.

Install Python and Libraries

  • Download and install Python 3.8+.
  • Install necessary Python libraries: 
  pip install pandas numpy scikit-learn joblib azureml-sdk
Enter fullscreen mode Exit fullscreen mode

Install Azure CLI

Install Docker

Developing the AI Model with Python and Azure Machine Learning

Step 1: Define the Trading Strategy

Determine the AI model's purpose, such as predicting stock price movements or classifying market trends.

Step 2: Set Up Azure Machine Learning Workspace

  1. Create a Resource Group: 
   az group create --name TradingAI_RG --location eastus
Enter fullscreen mode Exit fullscreen mode
  1. Create an Azure Machine Learning Workspace: 
   az ml workspace create -w TradingAI_WS -g TradingAI_RG
Enter fullscreen mode Exit fullscreen mode

Step 3: Develop the AI Model

Create a Python script train_model.py for training the model.

import pandas as pd
import numpy as np
from sklearn.ensemble import RandomForestClassifier
import joblib
from azureml.core import Workspace, Dataset

# Connect to Azure ML Workspace
ws = Workspace.from_config()

# Load Dataset
dataset = Dataset.get_by_name(ws, name='historical_stock_data')
df = dataset.to_pandas_dataframe()

# Data Preprocessing
# ... (cleaning, feature engineering)

# Split Data
X = df.drop('target', axis=1)
y = df['target']

# Train Model
model = RandomForestClassifier(n_estimators=100)
model.fit(X, y)

# Save Model
joblib.dump(model, 'model.joblib')
Enter fullscreen mode Exit fullscreen mode

Step 4: Register the Model in Azure ML 

from azureml.core.model import Model

model = Model.register(workspace=ws,
                       model_path='model.joblib',
                       model_name='TradingAIModel')
Enter fullscreen mode Exit fullscreen mode

Utilizing .NET C# Semantic Kernel for AI Agents

The Semantic Kernel is an open-source SDK that allows .NET developers to integrate AI models, including Large Language Models (LLMs), into their applications.

Step 1: Install the Semantic Kernel NuGet Package 

dotnet add package Microsoft.SemanticKernel
Enter fullscreen mode Exit fullscreen mode

Step 2: Integrate the AI Model into the Semantic Kernel

Create a new class TradingAgentKernel.cs to define the AI agent's capabilities.

using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Orchestration;
using Microsoft.SemanticKernel.SkillDefinition;

public class TradingAgentKernel
{
    private readonly IKernel _kernel;

    public TradingAgentKernel()
    {
        _kernel = Kernel.Builder.Build();

        // Register AI skills or functions
        _kernel.RegisterSemanticFunction("PredictMarketTrend", PredictMarketTrend);
    }

    [SKFunction]
    public async Task<SKContext> PredictMarketTrend(SKContext context)
    {
        // Implement logic to call the AI model
        var features = context.Variables["features"];
        var prediction = await CallAIModelAsync(features);
        context.Variables["prediction"] = prediction.ToString();
        return context;
    }

    private async Task<int> CallAIModelAsync(string features)
    {
        // Logic to call the AI model (e.g., via REST API)
        // ...
        return prediction;
    }

    public async Task<string> ExecuteAsync(string input)
    {
        var context = await _kernel.RunAsync(input, "PredictMarketTrend");
        return context.Variables["prediction"];
    }
}
Enter fullscreen mode Exit fullscreen mode

Step 3: Leverage Azure OpenAI Service (Optional)

If you want to integrate LLMs like GPT-3 into your agent:

_kernel.Config.AddAzureOpenAICompletionBackend(
    "azureOpenAI",
    "deploymentName",
    "https://your-openai-endpoint.openai.azure.com/",
    "your-api-key"
);
Enter fullscreen mode Exit fullscreen mode

Creating the .NET Core Web API

Step 1: Initialize the Project 

dotnet new webapi -n TradingAIAPI
cd TradingAIAPI
Enter fullscreen mode Exit fullscreen mode

Step 2: Install Necessary NuGet Packages 

dotnet add package Microsoft.Azure.Cosmos
dotnet add package Azure.Security.KeyVault.Secrets
dotnet add package Microsoft.Extensions.Configuration.AzureKeyVault
dotnet add package Microsoft.Azure.Services.AppAuthentication
dotnet add package Microsoft.SemanticKernel
Enter fullscreen mode Exit fullscreen mode

Step 3: Set Up Project Structure

  • Controllers: For API endpoints.
  • Services: For business logic.
  • Models: For data structures.

Step 4: Configure App Settings

Create appsettings.json with placeholders.

{
  "Azure": {
    "KeyVault": "https://your-keyvault-name.vault.azure.net/",
    "CosmosDb": {
      "Endpoint": "your-cosmosdb-endpoint",
      "Database": "TradingDB",
      "Container": "Trades"
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Integrating the AI Model into the .NET Core Application

Step 1: Use Semantic Kernel in the Application

TradingAgentKernel.cs (as defined earlier) will be used in the application.

Step 2: Implement the Controller

TradesController.cs

[ApiController]
[Route("[controller]")]
public class TradesController : ControllerBase
{
    private readonly TradingAgentKernel _tradingAgentKernel;
    private readonly TradeRepository _tradeRepository;

    public TradesController(TradingAgentKernel tradingAgentKernel, TradeRepository tradeRepository)
    {
        _tradingAgentKernel = tradingAgentKernel;
        _tradeRepository = tradeRepository;
    }

    [HttpPost("execute")]
    public async Task<IActionResult> ExecuteTrade([FromBody] TradeRequest request)
    {
        // Use Semantic Kernel to get prediction
        var prediction = await _tradingAgentKernel.ExecuteAsync(request.FeaturesJson);

        if (int.Parse(prediction) == 1)
        {
            // Logic to execute buy order
            await _tradeRepository.AddTradeAsync(new Trade
            {
                Symbol = request.Symbol,
                Price = request.Price,
                Timestamp = DateTime.UtcNow,
                Action = "Buy"
            });
        }
        else
        {
            // Logic to execute sell order
            await _tradeRepository.AddTradeAsync(new Trade
            {
                Symbol = request.Symbol,
                Price = request.Price,
                Timestamp = DateTime.UtcNow,
                Action = "Sell"
            });
        }

        return Ok(new { Success = true });
    }
}
Enter fullscreen mode Exit fullscreen mode

TradeRequest Model

public class TradeRequest
{
    public string Symbol { get; set; }
    public double Price { get; set; }
    public string FeaturesJson { get; set; } // JSON string of features
}
Enter fullscreen mode Exit fullscreen mode

Implementing Data Storage with Azure Cosmos DB

Follow the same steps as before, using the MongoDB Driver for .NET Core.

TradeRepository.cs remains unchanged.

Securing Secrets with Azure Key Vault

Step 1: Create an Azure Key Vault 

az keyvault create --name TradingAIKeyVault --resource-group TradingAI_RG --location eastus
Enter fullscreen mode Exit fullscreen mode

Step 2: Store Secrets 

az keyvault secret set --vault-name TradingAIKeyVault --name "CosmosDbConnectionString" --value "your-connection-string"
Enter fullscreen mode Exit fullscreen mode

Step 3: Configure the Application to Use Key Vault

Install Packages

dotnet add package Azure.Extensions.AspNetCore.Configuration.Secrets
dotnet add package Azure.Identity
Enter fullscreen mode Exit fullscreen mode

Modify Program.cs

using Azure.Identity;

public class Program
{
    public static void Main(string[] args)
    {
        var host = CreateHostBuilder(args).Build();
        host.Run();
    }

    public static IHostBuilder CreateHostBuilder(string[] args) =>
        Host.CreateDefaultBuilder(args)
            .ConfigureAppConfiguration((context, config) =>
            {
                var builtConfig = config.Build();
                var keyVaultName = builtConfig["Azure:KeyVault"];
                if (!string.IsNullOrEmpty(keyVaultName))
                {
                    var keyVaultUri = new Uri(keyVaultName);
                    config.AddAzureKeyVault(keyVaultUri, new DefaultAzureCredential());
                }
            })
            .ConfigureWebHostDefaults(webBuilder =>
            {
                webBuilder.UseStartup<Startup>();
            });
}
Enter fullscreen mode Exit fullscreen mode

Using Azure Functions for Event-Driven Processing

Azure Functions can be used to handle real-time data ingestion or to trigger trading actions based on specific events.

Step 1: Create an Azure Function Project 

func init TradingAIFunctions --dotnet
cd TradingAIFunctions
func new --name MarketDataIngestion --template "HttpTrigger" --authlevel "Function"
Enter fullscreen mode Exit fullscreen mode

Step 2: Implement the Function

MarketDataIngestion.cs

using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Logging;

public static class MarketDataIngestion
{
    [FunctionName("MarketDataIngestion")]
    public static async Task<IActionResult> Run(
        [HttpTrigger(AuthorizationLevel.Function, "post", Route = null)] HttpRequest req,
        ILogger log)
    {
        // Parse incoming market data
        var requestBody = await new StreamReader(req.Body).ReadToEndAsync();
        var marketData = JsonConvert.DeserializeObject<MarketData>(requestBody);

        // Logic to process data or trigger trading actions
        // ...

        return new OkResult();
    }
}

public class MarketData
{
    public string Symbol { get; set; }
    public double Price { get; set; }
    // Other relevant fields
}
Enter fullscreen mode Exit fullscreen mode

Step 3: Deploy the Function to Azure 

func azure functionapp publish TradingAIFunctionsApp
Enter fullscreen mode Exit fullscreen mode

Step 4: Integrate with the Main Application

  • Use Azure Event Grid or Azure Service Bus to communicate between Azure Functions and the main application.

Containerizing the Application with Docker

Follow the same steps as before, ensuring that the Dockerfile includes the necessary configurations for the Semantic Kernel and any dependencies.

Dockerfile

# Base image
FROM mcr.microsoft.com/dotnet/aspnet:8.0 AS base
WORKDIR /app

# Copy files and build
FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build
WORKDIR /src
COPY ["TradingAIAPI.csproj", "./"]
RUN dotnet restore "./TradingAIAPI.csproj"
COPY . .
RUN dotnet build "TradingAIAPI.csproj" -c Release -o /app/build

# Publish
FROM build AS publish
RUN dotnet publish "TradingAIAPI.csproj" -c Release -o /app/publish

# Final stage
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "TradingAIAPI.dll"]
Enter fullscreen mode Exit fullscreen mode

Deploying to Azure Kubernetes Service (AKS)

Follow the same steps as before to create an AKS cluster, push Docker images to Azure Container Registry, and deploy the application.

Monitoring and Logging

Step 1: Enable Azure Monitor for Containers 

az aks enable-addons -a monitoring -n TradingAICluster -g TradingAI_RG
Enter fullscreen mode Exit fullscreen mode

Step 2: Use Application Insights

Install Package

dotnet add package Microsoft.ApplicationInsights.AspNetCore
Enter fullscreen mode Exit fullscreen mode

Configure in Startup.cs

public void ConfigureServices(IServiceCollection services)
{
    services.AddApplicationInsightsTelemetry("Your_Instrumentation_Key");
    services.AddControllers();
}
Enter fullscreen mode Exit fullscreen mode

Step 3: View Logs and Metrics

  • Access Azure Portal to view logs in Azure Monitor.
  • Use Application Insights for application-level telemetry.

Conclusion

By integrating .NET C# Semantic Kernel and Azure Functions into your automated trading system, you enhance the AI capabilities and enable event-driven processing, which is crucial for real-time trading applications. The combination of these technologies with Azure AI Services, Azure Key Vault, Azure Cosmos DB, AKS, and Python allows you to build a sophisticated, scalable, and secure AI agent for automated trading.

About the Author

As a Senior Software Engineer and Senior AI Engineer specializing in .NET C#, Azure, and AI, I bring extensive experience in developing enterprise-grade applications and AI solutions. This article combines practical knowledge and best practices to help professionals build advanced automated trading systems.

References

Top comments (0)

Read next

fabbuilder profile image

Code Generated Architecture Diagram using Azure DevOps

FAB Builder | Code Generation -

mohammadpooladi profile image

Synchronous Applications

Mohammad Pooladi -

abheeshta profile image

🧠Generative AI - 3

Abheeshta P -

fkkarakurt profile image

Be Careful When Using YAML in Python! There May Be Security Vulnerabilities

Fatih Küçükkarakurt -