DEV Community

loading...

F# App Stub for AKS hosting (with WebJobs but without Azure Functions fluff)

deyanp profile image Deyan Petrov Updated on ・6 min read

TLDR; Use a standard .NET 5 host which gives you full control (incl. upgradability!) and still allows for WebJobs syntactic sugar, instead of the fluffy and "magical" Azure Functions host.

Introduction

In a previous post I listed a number of reasons why you'd better migrate away from Azure Functions and host your .NET (F# recommended!) apps in AKS. In this post I will try to give you as much code as you need to be able to start with this alternative approach in a matter of minutes instead of hours/days.

I will create a fully-blown stub including all the functionality you may need - WebJobs (EventHubTrigger, QueueTrigger, TimerTrigger), Web Server, custom Background Services, App Insights integration, Console Log Formatting, so it will look as close to Azure Functions as possible, but with full control.

Note: Azure Functions support for .NET 5 (the so-called Isolated Host) has actually taken a similar approach, but there is still too much fluff/magic for my liking ..

Design Approach/Principles

  1. Mimic Azure Functions setup - including the usage of WebJobs (used by Azure Functions themselves), Application Insights integration, even the same or slightly better console log formatting!
  2. "Close to the metal" or as basic as possible - just use Asp.Net Core, no framework like Saturn, Giraffe etc. required for just a few PUT/POST/PATCH/GET operations.
  3. Utilize standard HostBuilder functionality like running additional HostedServices or BackgroundServices.

Program.fs

If you don't have prior knowledge about HostBuilder then in a nutshell it is just a configurable way of spawning several threads/application services, e.g. a web server, some background services used by the WebJobs SDK for listening to Azure Storage Queues or Event Hubs for example, some other custom background processes of yours (e.g. for watching a MongoDB collection) or even some startup code which runs when the whole application starts (e.g. for filling an in-memory cache from a db table).

The main bootstrapping code is in the main function and may look like this:

[<EntryPoint>]
let main argv =
    let builder =
        Host.CreateDefaultBuilder(argv)
        |> Framework.Hosting.HostBuilder.configureLogging
        |> Framework.Hosting.HostBuilder.configureAppInsights
        |> configureWorker1
        |> configureWorker2
        |> Framework.Hosting.HostBuilder.configureWebHost configureEndpoints
        |> configureWebJobs

    use tokenSource = new CancellationTokenSource()    
    use host = builder.Build()
    host.RunAsync(tokenSource.Token) |> Async.AwaitTask |> Async.RunSynchronously

    0 // return an integer exit code
Enter fullscreen mode Exit fullscreen mode

As you see from above, we are configuring logging, App Insights integration, then 2 background workers (IHostedServices), the web server with 3 "HttpTriggers", and web jobs which actually means EventHubTriggers, QueueTriggers, TimerTriggers etc.

WebJobs Configuration

WebJobs configuration is extremely straightforward:

let configureWebJobs (builder:IHostBuilder) = 
    builder.ConfigureWebJobs(fun b ->
        b.AddAzureStorageCoreServices() |> ignore
        b.AddEventHubs() |> ignore
        b.AddAzureStorageQueues() |> ignore
        b.AddTimers() |> ignore)
Enter fullscreen mode Exit fullscreen mode

Additional services can be configured as well, e.g. b.AddSignalR() |> ignore, etc.

Web Server Configuration

Standard Kestrel configuration ... basic mapping of GET/PUT/POST/PATCH to functions, with manual injection of dependencies (e.g. logger) as a function parameter (even partial application not required):

let configureEndpoints (app:IApplicationBuilder) (endpoints:IEndpointRouteBuilder) =
    let hostedServices = Framework.BackgroundService.Hosted.findAll app.ApplicationServices

    let worker1 = Framework.BackgroundService.Hosted.find hostedServices "Worker1"
    endpoints.MapGet("api/v1/background-services/worker1/status", fun context ->
        worker1.GetProcessingStatusHttp()
        |> MappedHttpResponse.toHttpResponse context
        :> Task) |> ignore

    let worker2 = Framework.BackgroundService.Hosted.find hostedServices "Worker2"
    endpoints.MapGet("api/v1/background-services/worker2/status", fun context ->
        worker2.GetProcessingStatusHttp()
        |> MappedHttpResponse.toHttpResponse context
        :> Task) |> ignore

    endpoints.MapGet("api/v1/webFunction1", fun context ->
        let logger = app.ApplicationServices.GetService<ILogger<Object>>()

        WebApi.webFunction1 logger context.Request
        |> Async.StartAsTask
        |> Task.bind (MappedHttpResponse.toHttpResponse context)
        :> Task) |> ignore

let configureWebHost configureEndpoints (builder:IHostBuilder) : IHostBuilder =
    builder.ConfigureServices(fun context services ->
        if context.HostingEnvironment.IsDevelopment() then
            services.AddCors(fun options ->
                options.AddDefaultPolicy(fun b ->
                    b
                        .AllowCredentials()
                        .WithOrigins("http://localhost:8080")
                        .WithHeaders("authorization","content-type","if-match","etag","content-disposition","x-requested-with")
                        .WithExposedHeaders("authorization","content-type","if-match","etag","content-disposition")
                        .WithMethods("GET","POST","PUT","PATCH")
                    |> ignore
                )
            ) |> ignore
        ) |> ignore

    builder.ConfigureWebHostDefaults(fun b ->
        b.Configure(fun context app ->
            app.UseRouting() |> ignore

            if context.HostingEnvironment.IsDevelopment() then        
                app.UseCors() |> ignore

            app.UseEndpoints (fun endpoints -> configureEndpoints app endpoints) |> ignore

        ) |> ignore
    )

Enter fullscreen mode Exit fullscreen mode

Bonus: As you see above there are 2 status web functions which report the status of the threads started by the background services (see next section).

Custom Background Services Configuration

You can do anything in a HostedService/BackgroundService, even get rid of the WebJobs SDK and simply listen to event hubs using EventHubProcessor for example by yourself (I have always regarded XxxTrigger as a non-mandatory syntactic sugar ...):

let startedEvent1 = new ManualResetEvent(false) 

let configureWorker1 (builder:IHostBuilder) : IHostBuilder =
    builder.ConfigureServices(fun context services ->
        services.AddSingleton<IHostedService>(
            fun serviceProvider -> 
                let logger = serviceProvider.GetService<ILogger<Object>>()
                Framework.BackgroundService.Hosted.create
                    "Worker1"
                    (BackgroundServices.worker1 logger startedEvent1) :> IHostedService) |> ignore
        )

let configureWorker2 (builder:IHostBuilder) : IHostBuilder =
    builder.ConfigureServices(fun context services ->
        services.AddSingleton<IHostedService>(
            fun serviceProvider -> 
                let logger = serviceProvider.GetService<ILogger<Object>>()
                Framework.BackgroundService.Hosted.create
                    "Worker2"
                    (BackgroundServices.worker2 logger startedEvent1) :> IHostedService) |> ignore
        )
Enter fullscreen mode Exit fullscreen mode

Logging Configuration, including Console Log Formatting

To simulate the Azure Functions logger formatting I had to unfortunately implement a full-blown ConsoleFormatter ... but also made the log output a bit more-concise on the way ;)

let configureLogging (builder:IHostBuilder) : IHostBuilder =
    builder.ConfigureLogging(fun context b ->
//        b.ClearProviders() |> ignore  // commented out because otherwise Application Insights does not collect logger.Information ...
        b.AddConsole(fun options -> options.FormatterName <- "CustomConsoleFormatter")
            .AddConsoleFormatter<Framework.Logging.Console.CustomConsoleFormatter, Framework.Logging.Console.CustomConsoleFormatterOptions>(fun options ->
                options.Excludes <- Framework.Logging.Console.defaultExcludes
                options.SingleLinePerCategory <- ["Microsoft.Azure.WebJobs.Hosting.OptionsLoggingService", false; "Host.Triggers.Timer", false] |> dict
                options.SingleLine <- true
                options.ColorBehavior <- LoggerColorBehavior.Enabled)
        |> ignore
    )
Enter fullscreen mode Exit fullscreen mode

Application Insights Integration

Application Insights works here as well, note that I am injecting a custom initializer and processor to suppress some http status codes etc.

let configureAppInsights (builder:IHostBuilder) : IHostBuilder =    
    builder.ConfigureServices(fun context services ->
        let aiOptions = ApplicationInsightsServiceOptions(EnableAdaptiveSampling = false)   // disable sampling (turned on by default, see https://docs.microsoft.com/en-us/azure/azure-monitor/app/sampling#brief-summary)
        services.AddApplicationInsightsTelemetry(aiOptions) |> ignore
        services.AddSingleton<ITelemetryInitializer>(
            Framework.Logging.Telemetry.Configuration.CustomTelemetryInitializer(
                Environment.GetEnvironmentVariable("WEBSITE_SITE_NAME", true),
                Framework.Logging.Telemetry.Configuration.defaultSuppressedStatusCodes))
        |> ignore

        services.AddSingleton<Framework.Logging.Telemetry.Configuration.CustomTelemetryProcessorConfig>(Framework.Logging.Telemetry.Configuration.defaultCustomTelemetryProcessorConfig) |> ignore
        services.AddApplicationInsightsTelemetryProcessor<Framework.Logging.Telemetry.Configuration.CustomTelemetryProcessor>() |> ignore
    )
Enter fullscreen mode Exit fullscreen mode

Custom Functions

And the last bit are the user-defined functions (="Azure Functions):



module BackgroundServices =
    let worker1 (logger:ILogger) (startedEvent: ManualResetEvent) (token:CancellationToken) : Async<unit> =
        async {
            startedEvent.Set() |> ignore
            while true do
                do logger.LogInformation("Hello from worker1")
                Thread.Sleep(5000)
        }        
    let worker2 (logger:ILogger) (startedEvent: ManualResetEvent) (token:CancellationToken) : Async<unit> =
        async {
            startedEvent.Set() |> ignore
            while true do
                do logger.LogInformation("Hello from worker2")
                Thread.Sleep(5000)
        }        

module WebApi =
    let webFunction1 (logger:ILogger) (req: HttpRequest) : Async<MappedHttpResponse> =
        async {
            do logger.LogInformation("webFunction1")
            return {
                StatusCode = 200
                Content = "Some response" |> MappedHttpResponseContent.Json
                Headers = List.empty
            }
        }

type WebJobs(telemetryConfiguration: TelemetryConfiguration) =
    let telemetryClient = TelemetryClient(telemetryConfiguration)

    [<FunctionName("HandleEventHubMessage")>]
    member _.HandleEventHubMessage
        ([<EventHubTrigger("", Connection = "EventHubConnectionString", ConsumerGroup = "test-cg")>]    // path to event hub is in the connection string
        msg: EventData,
        enqueuedTimeUtc: DateTime,
        sequenceNumber: Int64,
        offset: string,
        logger: ILogger)
        =
        async {
            do logger.LogInformation($"HandleEventHubMessage: {Encoding.UTF8.GetString(msg.Body.ToArray())}")
        } |> Async.StartAsTask

    [<FunctionName("HandleQueueMessage")>]
    member _.HandleQueueMessage
        ([<QueueTrigger("test-queue", Connection = "StorageQueueConnectionString")>] msg: string,
        logger:ILogger)
        =
        async {
            do logger.LogInformation($"HandleQueueMessage: {msg}")
        } |> Async.StartAsTask

    [<FunctionName("HandleTimerEvent")>]
    member this.HandleTimerEvent
        ([<TimerTrigger("0 0 0 1 * *", RunOnStartup = true)>] timer: TimerInfo,
        logger: ILogger)
        =
        async {
            do logger.LogInformation("HandleTimerEvent")
        } |> Async.StartAsTask
Enter fullscreen mode Exit fullscreen mode

As you see, not much difference compared to "real" Azure Functions ...

Migration from Azure Functions

  1. Add Program.fs if not already existing, and copy over the host builder pipeline (including the required WebJobs AddXxxx services)
  2. Configure WebJobs (no change required to corresponding XxxTriggers)
  3. Implement configureEndpoints, and remove HttpTrigger attributes from the http-based azure functions (no change required for the non-http azure functions, attributes remain)

Conclusion

With greater control comes bigger responsibility, however you can see from the above that the required code is not only relatively easy to grasp and maintain, but it offers much more! For example, background services can be injected nicely, and a lot of additional HostBuilder functionality can be used - there are tons of Q&As and documentations online. Last but not least, after migrating from Azure Functions v3 to the above "custom .NET host" we experienced a 50% drop in memory consumption of the AKS pods which I believe is attributable to the removed Azure Functions bloat, so that was another bonus for us.

P.S. Working project can be found at https://github.com/deyanp/FSharpAKSAppStub.

Discussion (0)

Forem Open with the Forem app