DEV Community: Matthew MacFarland

Understanding map and bind with F# options

Matthew MacFarland — Sat, 25 Nov 2023 17:51:39 +0000

The option is a very frequently used type in F# programs. We use options to represent values that may (Some) or may not (None) be present. When reading in data from an external source we can't be sure that every field is filled in with a value so we'll represent these as options. Some of the functions we try to perform on values may not be able to produce an instance of the desired type and so we'll return an option of the type. For example, we can't be sure that the string we want to convert to an int is valid for that conversion.

It's simple to work with options using the match expression. When learning F# it can be helpful to start with match expressions since they're easy to understand, especially for the simple option type. However, when the program starts to grow it can become very tedious to write a match expression for every interaction with an option. The other problem with match expressions is that they're not helpful when we want to use a pipeline to perform a series of steps.

The option module in F# has several pipeline friendly functions that be used instead of match expressions, they include Option.map, bind, and defaultValue. It took a while for me to grasp how best to use these functions. They're quite simple at first glance but it just wasn't obvious how to incorporate them into my programs.

Let's look a few examples where these Option module function can make working with options much easier.

For the first example, we'll read in CSV data, parse it, and create an instance of a record type for each line. This common scenario can lead to many places where we'll need to work with options. We'll also see the problem that arises when we start trying to compose functions that return options with other functions that don't expect options as input.

Here's the first try to perform two of the steps. Split up the CSV data into an array of lines and then split each line into an array of the fields. So, we'd like to end up with an array of arrays.

let productsRawData =
    "id,description,rating,price
123,Product 1,3,99.95
345,Product 2,,23,95
456,Product 3,3,
567,Product 4,,
678,Product 5,2,23.9a
789,  ,4,25"

let csvToLines (csvData:string) : string array option =
    if csvData.Length <= 1 then
        None
    else
        Some (csvData.Split("\r\n") |> Array.skip 1)

let lineToFields (line:string) : string array =
    let fields = line.Split(",")
    fields

let results =
    productsRawData
    |> csvToLines
    |> Array.map lineToFields // Won't compile 

printfn $"%A{results}"

The csvToLines function returns a string array option because the CSV data may be empty or have only a header. In those cases we'll return None. Once we have the array of lines we want to call lineToFields for each of them using Array.map. This can't compile however because Array.map expects an array and we're trying to pass it an array option.

Let's fix this first using a match expression followed by the alternative Option.map

let lines =
    productsRawData
    |> csvToLines

let results =
    match lines with
    | Some l -> Some(Array.map lineToFields l)
    | None -> None

printfn $"%A{results}"

Some
  [|[|"123"; "Product 1"; "3"; "99.95"|];
    [|"345"; "Product 2"; ""; "23"; "95"|];
    [|"456"; "Product 3"; "3"; ""|];
    [|"567"; "Product 4"; ""; ""|];
    [|"678"; "Product 5"; "2"; "23.9a"|];
    [|"789"; "  "; "4"; "25"|]|]

This works! However, we lost the nice concise pipeline syntax and had to break the problem up into an intermediate step. Let's try with Option.map.

let results =
    productsRawData
    |> csvToLines
    |> Option.map (Array.map lineToFields)

printfn $"%A{results}"

Some
  [|[|"123"; "Product 1"; "3"; "99.95"|];
    [|"345"; "Product 2"; ""; "23"; "95"|];
    [|"456"; "Product 3"; "3"; ""|];
    [|"567"; "Product 4"; ""; ""|];
    [|"678"; "Product 5"; "2"; "23.9a"|];
    [|"789"; "  "; "4"; "25"|]|]

Great! We're getting the exact same result, we get to keep the pipeline of functions going, and we can still handle the option. Why does this work? Take a look at the source code for Option.map next to the source for our match expression solution.

match lines with
    | Some l -> Some(Array.map lineToFields l)
    | None -> None

// Option.map source
let map mapping option =
    match option with
    | None -> None
    | Some x -> Some(mapping x)

Very close. Option.map is performing the exact same matching logic that we were doing inline. The difference is that the function we need to call is passed in as the mapping parameter and Option.map calls it for us if the option has a value. Notice this bit Some(mapping x). Because the mapping function we pass in doesn't return an option type, we have to convert it to a option to balance with None case. The code wouldn't compile otherwise.

Let's finish the remaining steps of parsing the fields and converting each array of fields into a Product record. Along the way there are a few more cases where the Option module functions can make this easier. We'll use both Option.map and Option.defaultValue here.

let parseStringField (field: string) : string option =
    match field with
    | field when String.IsNullOrWhiteSpace(field) -> None
    | _ -> Some(field.Trim())

let parseIntField (field: string) : int option =
    let success, value = Int32.TryParse(field)
    if success then Some value else None

let parseDecimalField (field: string) : decimal option =
    let success, value = Decimal.TryParse(field)
    if success then Some value else None

let createRecord (fields: string array) : Product =
    if fields.Length < 4 then
        None
    else
        Some
            { Id =
                fields[0]
                |> parseIntField
                |> Option.defaultValue 0
              Description =
                fields[1]
                |> parseStringField
                |> Option.defaultValue ""
              Rating =
                fields[2]
                |> parseIntField
                |> Option.defaultValue 0
              Price =
                fields[3]
                |> parseDecimalField
                |> Option.defaultValue 0m }
    |> Option.defaultValue Product.Default

let results =
    productsRawData
    |> csvToLines
    |> Option.map (Array.map lineToFields)
    |> Option.map (Array.map createRecord)
    |> Option.map (Array.filter (fun x -> x.Id > 0)) // Exclude default records
    |> Option.defaultValue [||]

So, the pipeline now includes several steps. The functions we want to use including Array.map and our custom functions don't expect options as inputs nor do they return options. But, using Option.map we can bridge that without breaking up the flow of the pipeline. If we attempted all the steps with inline match expressions this would get really long and hard to follow. The Option.defaultValue function is another pipeline friendly function that was very helpful for converting our option values back to regular values.

The pattern to look for where Option.map can be helpful is when you're trying to compose functions where the first function returns an option but the second function takes a non-option input and returns a non-option output.

What about functions that take a regular input value and return an option. These can also be composed in a pipeline and the Option.bind function is used in that case. Option.bind is very similar to Option.map. Look at the source code for them side by side.

let bind binder option =
    match option with
    | None -> None
    | Some x -> binder x

let map mapping option =
    match option with
    | None -> None
    | Some x -> Some(mapping x)

Map performs the one extra step of wrapping the results from the mapping function into a Some. Bind omits that step since the binder function already returns an option. That's the reason that we select map for functions that don't return an option and bind for functions that do.

Let's look at a simple example where we want to perform a couple of parsing steps based on a product description. The first function has to return an option because the value we're trying to extract may not be present. The next function will take the product code we extract and map it to a category. Again, we can't just wire these up directly in a pipeline because the option output from the first function is not compatible with the string input expected by the second function.

type ProductCategory =
    | Electronics
    | Timepieces
    | Kitchen

let tryGetProductCode (description:string) : string option =
    let options = (RegexOptions.Multiline ||| RegexOptions.IgnoreCase)
    let mtch = Regex.Match(description, "(\w{1}-\d{4}-\d{2})", options)
    if mtch.Success then
        Some mtch.Groups[0].Value
    else
        None

let tryGetCategory (productCode:string) : ProductCategory option =
    match productCode.Substring(0,1) with
    | "K" -> Some Kitchen
    | "T" -> Some Timepieces
    | "P" -> Some Pets
    | "E" -> Some Electronics
    | _ -> None

let someCategory =
    "Citizen Men's Eco-Drive Sport
Luxury Endeavor Watch: T-1933-32"
    |> tryGetProductCode
    |> Option.bind tryGetCategory

printfn $"%A{someCategory}"
// Some Timepieces

let noCategory =
    "SAMSUNG 43-Inch Class QLED 4K Q60C Series Quantum HDR
Dual LED, Object Tracking Sound Lite, Q-Symphony,
Motion Xcelerator, Gaming Hub"
    |> tryGetProductCode
    |> Option.bind tryGetCategory

printfn $"%A{noCategory}"
// None

The Option.map, bind, and defaultValue functions are very helpful in many day to day programming tasks. I'm finding that I use them quite frequently and rely much less on inline match expressions. I also find code written with a combination of pipeline steps and these option functions is much easier to read. The intention of the program doesn't become lost in the details of manually managing all the options.

Managing I/O in F# Web API applications

Matthew MacFarland — Thu, 23 Nov 2023 21:28:03 +0000

Full working source code for this article is available on github.

In this article, we'll consider a method for handling I/O operations in an F# application. There are several methods available for separating I/O in F# apps. These range from using the built-in dependency injection and interfaces to more abstract functional styles. We'll look at a simple example using an interface based on the weather API template.

Dedicated interface for each controller

In .NET applications, it's common to inject several service interfaces into each controller. These interfaces may each have any number of methods and the controllers often don't use them all. In order to be more explicit about what I/O operations the controller needs to perform, we can create a dedicated interface for each controller with only the methods it needs defined.

An alternative to interfaces that can be used with F#, is a record of functions. This works well and is mostly a difference in style. The use of interfaces is the recommended approach found in the F# guidelines for passing a group of functions and it also leaves open the possibility of using standard mocking libraries like NSubstitute.

Here's an example of our interface with 3 I/O operations defined.

type ErrorMessage = string
type MessageTemplate = string
type Latitude = float
type Longitude = float

// I/O operations interface
type IWeatherForecastIO =
    abstract CallWeatherServiceAsync: 
        Latitude * Longitude * CancellationToken -> 
            Task<Result<WeatherForecast, ErrorMessage>>
    abstract LogError: MessageTemplate * obj array -> unit
    abstract LogInformation: MessageTemplate * obj array -> unit

Using tuple style arguments for interface members

There is a gotcha to watch out for with interfaces in F#. Even if we define an interface member with curried style parameters, the compiler will convert these methods to tuple style automatically. So, when defining and calling interface members in F#, using tuple style throughout may reduce any confusion.

Note: Curried style parameters are defined in function type signatures as FunctionType: Arg1 -> Arg2 -> ReturnType whereas tuple style parameters are defined as shown in the example FunctionType: Arg1 * Arg2 -> ReturnType

See Brian's detailed explanation in the following Stackoverflow Q&A

How are F# interface members implemented with object expressions?

Module functions to implement each I/O method

While the interface members are written using an object oriented style, the functions that implement each of these members can use a functional style and curried parameters. The implementing function will be connected up to each of the members in the interface when we create the object instances that implement the interface.

Here is how the object that implements the interface is defined and added to the service collection in Program.fs

    ...
    builder.Services.AddSingleton<IWeatherForecastIO>(fun sp -> 
        let httpClientFactory = 
            sp.GetRequiredService<IHttpClientFactory>()
        let logger = 
            sp.GetRequiredService<ILogger<WeatherForecastController>>() 
        { new IWeatherForecastIO with
            member _.CallWeatherServiceAsync(latitude, longitude, token) =
                OpenMeteoWeatherService.getForecastAsync
                    httpClientFactory latitude longitude token
            member _.LogError(error, args) = 
                logger.LogError(error, args)
            member _.LogInformation(error, args) =
                logger.LogInformation(error, args) }
    )
    ...

The dependencies httpClientFactory and logger were defined previously and are pulled in here for use in the new object. The function we've created to implement the CallWeatherServiceAsync interface member calls out to the free Open Meteo service to get the weather data. The getForecastAsync function needs access to the httpClientFactory to make the API call so we pass that in as one of the parameters. Our production code that uses the interface won't need to interact with the HTTP client at all using this setup. Creating a mock for the CallWeatherServiceAsync interface member is now much simpler because only the simple data criteria and return types need to be considered.

It is very helpful to keep low level dependencies like database, cloud, and HTTP API concealed this way. It also reduces the number of places that changes need to be made when switching I/O sources between different databases and cloud services. As long as the simple public API of the interface remains unchanged such infrastructure changes won't spill over beyond this setup point.

Note: Keep in mind the the dependency libraries often define types for input and return data as well. To maintain full isolation, create custom types within the application and map inputs and outputs to these within the implementing module functions. For example, the client for AWS queues define SendMessageResponse and ReceiveMessageResponse types. If these types are used at the interface level then all the consuming code downstream will have to take a reference on the AWS client. If 3rd party types remain isolated only within the module that interacts with that resource it can lead to much easier maintenance.

Using the DI container

The standard DI container that comes with ASP.NET is an easy default choice for many applications. In our example above the finished implementation of IWeatherForecastIO is added to the container as a singleton. So now it's simple to use this within the controller.

[<ApiController>]
[<Route("[controller]")>]
type WeatherForecastController (io: IWeatherForecastIO) =
    inherit ControllerBase()

    let _io = io

    [<HttpGet>]
    member this.GetAsync(location:string) : Task<IActionResult> =
        task{
            let cts = new CancellationTokenSource(5000)

            let coord = Coordinates.ofString location

            let! weatherForecast =
                match coord with
                | Some { Latitude = lat; Longitude = long } ->
                    _io.CallWeatherServiceAsync(lat, long, cts.Token)
                | None ->
                    Task.FromResult(Result.Error "Invalid location.")

            match weatherForecast with
            | Ok res ->
                _io.LogInformation(
                    "Successfully retrieved forecast for {location}", [| location |] )
                return ObjectResult(res) :> IActionResult
            | Error err ->
                _io.LogError(
                    "Error {error} retrieving forecast for {location}", [| err; location |] )
                return this.BadRequest(err) :> IActionResult
        }

Testing with standard mocking tools

Because we selected an interface for the I/O operations, we can make use of any of the standard .NET mocking tool libraries. In our example we'll use NSubstitute. It's also very common to create custom mock functions in F#, and that option remains available. In fact, if the mock will only be used to return specific a canned response (it's really stub in that case) then it's really just as simple to use a custom mock function. If we want to test interactions with a mock function like counting how many times it was called, then the mocking library can be more helpful. Let's look at it both ways.

In the first example test we'll use NSubstitute.

[<Fact>]
member this.``GetAsync should return a list of weather forecasts``() =
    task {
        // Arrange
        let (location: Location) = "41.8755616,-87.624421"
        let today = DateTime.Today

        let (expectedForecastResult: Result<WeatherForecast, string>) =
            Ok
                { TemperatureCurrentC = 7
                  TemperatureCurrentF = 45
                  DailyForecast =
                    [| DailyForecast.Create today 
                        7 45 12 55 "Overcast"
                       DailyForecast.Create
                        (today.AddDays(1)) 3 38 17 62 "Drizzle"
                       DailyForecast.Create
                        (today.AddDays(2)) 3 38 17 62 "Drizzle"
                       DailyForecast.Create
                        (today.AddDays(3)) 9 48 17 62 "Clear"
                       DailyForecast.Create
                        (today.AddDays(4)) 9 48 20 68 "Overcast"
                       DailyForecast.Create
                        (today.AddDays(5)) 14 57 23 74 "Overcast"
                       DailyForecast.Create
                        (today.AddDays(6)) 19 67 29 85 "Partly cloudy" |] }

        let mockWeatherForecastIO = 
            Substitute.For<IWeatherForecastIO>()
        mockWeatherForecastIO
            .CallWeatherServiceAsync(Arg.Any(), Arg.Any(), Arg.Any())
            .Returns(expectedForecastResult)
        |> ignore

        let ctrl = WeatherForecastController(mockWeatherForecastIO)

        // Act
        let! result = ctrl.GetAsync location

        // Assert
        Assert.NotNull(result)

        Assert.IsAssignableFrom<ObjectResult>(result)
        |> ignore

        let objectValue = 
            (result :?> ObjectResult).Value :?> WeatherForecast
        Assert.Equal(7, objectValue.TemperatureCurrentC)
    }

In this test we setup some expected data to be returned by the API call. Create here is a simple static member on the DailyForecast record type to make this creation operation more concise. Review the full code sample on github if you'd like to see the details.

The mockWeatherForecastIO instance is quite simple to setup. Since we're not trying to assert anything about what the log functions are doing in this test, we just skip setting them up and go with whatever the mocking library does as default. The hardest part of writing this test by far was figuring out how to get at the value inside the ObjectResult type.

let objectValue = (result :?> ObjectResult).Value :?> WeatherForecast

Here is an alternative implementation of the mock that doesn't use NSubstitute.

let mockWeatherForecastIOCustom =
    { new IWeatherForecastIO with
        member _.CallWeatherServiceAsync(_, _, _) =
            task{ return expectedForecastResult }
        member _.LogError(error, args) = ()
        member _.LogInformation(error, args) = () }

This implementation uses a very nice F# feature called an object expression to create an object that implements the interface inline. If you want to track how many times the custom CallWeatherServiceAsync gets called by the controller, a mutable value can be used like this.

let mutable timesCalled = 0 
let mockWeatherForecastIOCustom =
    { new IWeatherForecastIO with
        member _.CallWeatherServiceAsync(_, _, _) =
            task{
                timesCalled <- timesCalled + 1
                return expectedForecastResult
            }
        member _.LogError(error, args) = ()
        member _.LogInformation(error, args) = () }

Then, simply inspect the value with an assertion.

Alternative approaches

I found a lot of help and inspiration when looking into this topic from these articles.