Single Flight in Gleam: Managing Processes with OTP

🔗 Links

• SingleFlight repository
• SingleFlight README
• SingleFlight OTP design notes
• Public API implementation
• Coordinator actor
• Flight actor
• Concurrency and cache tests
• Gleam documentation
• Erlang/OTP design principles
• Go singleflight package

The idea

I wanted to play with Gleam processes and OTP by building something more involved than a basic server. Single flight was a good fit: small enough to understand, but rich enough to explore actors, monitoring, supervision, retries, caching, and Erlang's “let it crash” approach.

If 1,000 requests ask for the same expensive resource at the same time, the application should not make 1,000 identical calls. The first caller starts the work. Everyone else with the same operation and key joins it; synchronous callers receive the same outcome.

1,000 concurrent calls with the same operation and key
                    ↓
             one callback execution
                    ↓
  1,000 call(...) callers receive the same outcome

This is not quite caching. A cache reuses completed work; single flight joins work that is still running. Different keys can still run concurrently, and two different operations do not collide even if they produce the same string key.

I built SingleFlight as a typed Gleam library for one BEAM node. The public API is small; most of the project is about deciding which process owns each piece of state and what happens when a process dies.

A complete example

This is the smallest end-to-end shape, adapted from the tests:

type Operations {
  Operations(
    get_user: single_flight.OperationDefinition(Int, String),
  )
}

pub fn main() -> Nil {
  let operations =
    Operations(
      get_user: single_flight.operation(
        key: int.to_string,
        run: fn(user_id) { "user-" <> int.to_string(user_id) },
      ),
    )

  let assert Ok(instance) =
    single_flight.new()
    |> single_flight.with_operations(operations)
    |> single_flight.start

  let assert Ok("user-7") =
    instance
    |> single_flight.use_operation(fn(operations) { operations.get_user })
    |> single_flight.call(7)

  Nil
}

Concurrent calls to get_user with 7 join one flight. A call with another user id gets a separate flight and can run at the same time.

The BEAM mental model

BEAM processes do not share mutable state. Each process owns its state and receives messages through a mailbox. That changed the design question from “which lock protects this object?” to “which process owns this state?”:

Process	Responsibility
Coordinator	Owns the map of active and cached keys.
Flight actor	Owns one key, its waiters, attempts, and retained result.
Callback worker	Runs one attempt of user code.
Supervisors	Own the runtime infrastructure and flight lifecycles.

Gleam exposes these Erlang/OTP concepts through typed APIs. A process.Subject(Message) is a typed address, an actor is a process with state and a message loop, a monitor reports process death as a message, and a supervisor defines recovery boundaries.

Why operations live in an application record

The shape of the API came from one requirement: selecting an operation must preserve its exact parameter and result types all the way to call and send.

A regular List or Dict is homogeneous, so a straightforward runtime registry would either require every operation to share the same parameter and result types or erase them behind Dynamic and cast later. Neither option gives the call site the contract I wanted.

Instead, the application defines a record that acts like a typed operation interface:

type Operations {
  Operations(
    get_user: single_flight.OperationDefinition(Int, String),
    get_report: single_flight.OperationDefinition(Nil, Int),
  )
}

Each field has its own OperationDefinition(param, result). It binds together:

• the parameter accepted by both key and run;
• the result returned by run;
• a private identity used to scope concurrent work.

The implementation is correspondingly small:

pub opaque type SingleFlight(operations) {
  SingleFlight(config: Config, operations: operations)
}

pub opaque type OperationDefinition(param, result) {
  OperationDefinition(
    id: reference.Reference,
    key: fn(param) -> String,
    run: fn(param) -> result,
  )
}

pub opaque type Operation(param, result) {
  Operation(
    id: reference.Reference,
    coordinator_name: coordinator.Name,
    flight_supervisor_name: flight_supervisor.Name,
    key: fn(param) -> String,
    run: fn(param) -> result,
    settings: settings.Settings,
  )
}

pub fn operation(
  key key: fn(param) -> String,
  run run: fn(param) -> result,
) -> OperationDefinition(param, result) {
  OperationDefinition(id: reference.new(), key:, run:)
}

The definition is opaque and process-free; creating one does not start an actor. The fields may have unrelated types: get_user is Int → String, while get_report is Nil → Int. The record gives them source-level names without forcing a common result type or storing application values as Dynamic.

The runtime itself carries the record type as SingleFlight(operations). use_operation accepts a selector from that exact record and returns Operation(param, result):

let get_user =
  instance
  |> single_flight.use_operation(fn(operations) { operations.get_user })

let result = single_flight.call(get_user, 7)

The selector is the important part. When it returns operations.get_user, Gleam infers both Int and String. The bound operation then makes call accept only an Int and return Result(String, single_flight/error.Error); no string operation name or result cast is involved.

The public implementation shows that type flow directly:

pub fn use_operation(
  instance instance: SingleFlight(operations),
  using select: fn(operations) -> OperationDefinition(param, result),
) -> Operation(param, result) {
  let SingleFlight(config:, operations:) = instance
  let OperationDefinition(id:, key:, run:) = select(operations)

  Operation(
    id:,
    coordinator_name: config.coordinator_name,
    flight_supervisor_name: config.flight_supervisor_name,
    key:,
    run:,
    settings: config.settings,
  )
}

pub fn call(
  operation operation: Operation(param, result),
  with param: param,
) -> Result(result, error.Error) {
  call_resolved(operation, param, operation.settings)
}

Operation definitions should be created once and reused. Each call to single_flight.operation also creates a new internal reference, so recreating an otherwise identical definition creates a separate namespace.

The key defines equivalent work

The key function defines when two calls may share one result:

single_flight.operation(
  key: fn(user_id) { "github:user:" <> int.to_string(user_id) },
  run: github.get_user,
)

If the key omits a relevant input, unrelated requests may be collapsed. If it includes irrelevant changing data, useful deduplication is lost.

Internally, the coordinator scopes the string key by the operation's private reference:

type ScopedKey =
  #(reference.Reference, String)

That is why two operations can both produce "same-key" without sharing a flight or even sharing a result type.

The process topology

The runtime has two long-lived children and one temporary actor for every active or retained key:

                    OneForAll supervisor
                    / \
       FlightFactorySupervisor Coordinator
                 | |
          temporary children scoped key → flight
                 |
             FlightActor
                 |
       monitored callback worker

Within that tree, one acquisition flows like this:

caller
  │ Acquire(operation id, key)
  ▼
Coordinator
  ├─ missing key → start a FlightActor
  └─ existing key → deliver another Run message
                              │
                              ▼
                         FlightActor
                          ├─ first Run → start one worker
                          └─ later Run → add a waiter

The coordinator

The coordinator owns routing state, not user work. Its mailbox serializes Acquire, settlement, expiry, and child-down messages. That is the atomicity boundary: two callers cannot both observe a missing scoped key and create two owners for it.

Its actor message loop stays small and delegates each state transition:

fn handle_message(
  state state: State,
  message message: protocol.CoordinatorMessage,
) {
  case message {
    protocol.Acquire(key, operation_id, start, deliver, reply_to) ->
      handle_acquire(state, key, operation_id, start, deliver, reply_to)

    protocol.Settled(key, operation_id, pid, retention) ->
      handle_settled(state, key, operation_id, pid, retention)

    protocol.Expire(key, operation_id, pid) ->
      handle_expire(state, key, operation_id, pid)

    protocol.ChildDown(down) -> handle_child_down(state, down)
  }
  |> actor.continue
}

The central acquisition branch is regular actor state handling:

case dict.get(state.entries, scoped_key) {
  Error(Nil) -> start_flight(state, scoped_key, start, deliver, reply_to)

  Ok(entry) -> {
    let protocol.FlightHandle(pid:, ..) = entry.handle

    case process.is_alive(pid) {
      False ->
        state
        |> remove_entry(scoped_key)
        |> start_flight(scoped_key, start, deliver, reply_to)

      True ->
        case deliver(pid) {
          Ok(Nil) -> {
            process.send(reply_to, Ok(pid))
            touch_entry(state, scoped_key, entry)
          }
          Error(error.FlightUnavailable) -> {
            stop(entry.handle)
            state
            |> remove_entry(scoped_key)
            |> start_flight(scoped_key, start, deliver, reply_to)
          }
          Error(delivery_error) -> {
            process.send(reply_to, Error(delivery_error))
            state
          }
        }
    }
  }
}

On a miss, start_flight enforces the active-flight limit and asks the factory supervisor to start a child. On a hit, the coordinator sends the new waiter to the existing flight. touch_entry updates LRU order when that entry is cached.

Checking is_alive is not enough by itself: the process can die before accepting Run. Delivery therefore waits for an acknowledgement from the flight. If it returns FlightUnavailable, the coordinator removes the stale entry and retries acquisition with a fresh flight.

The coordinator also monitors every flight actor and removes its scoped entry when that child exits.

The flight supervisor

The factory supervisor has a smaller job: start and own flight actors. Its complete child configuration is short:

pub fn supervised(name name: Name) {
  factory_supervisor.worker_child(fn(start_child) { start_child() })
  |> factory_supervisor.named(name)
  |> factory_supervisor.restart_strategy(supervision.Temporary)
  |> factory_supervisor.supervised
}

Temporary is essential. A flight contains callbacks, waiters, attempts, and timers for one scoped key. Restarting it with empty state would not recover that work, so a crashed flight is removed and a later acquisition creates a new one.

Four invariants keep the design understandable:

1. Only the coordinator assigns a scoped key to a flight.
2. One flight owns all attempts, waiters, and the retained result for that key.
3. User code runs in a separate worker, never inside the coordinator or flight actor.
4. Coordinator messages that remove or expire an entry must still refer to the same flight pid.

The last check matters because monitor notifications and expiry timers may arrive after a replacement flight has claimed the same logical key.

One flight is a state machine

State	Event	Effect
Idle	First Run	Start one monitored worker and timeout timer.
Running	Another Run	Add a waiter without starting more work.
Running	Result	Ask the coordinator whether to retain or drop it.
Running	Crash or timeout	Retry, or settle one shared error.
RetryPending	Retry timer	Start the next attempt.
Finalizing	Retained or Stop	Reply to waiters, then keep or stop the actor.
Completed	Another Run	Return the retained outcome immediately.

Every attempt gets an id. Completion, timeout, and retry messages are accepted only when their id matches the current attempt, so a late message cannot settle newer work.

The finalizing handshake is also deliberate. The flight waits until the coordinator has registered the retained entry or removed the dropped one before replying to callers. A new acquisition therefore cannot arrive while the result is published but key ownership is undecided.

Monitoring, linking, and “let it crash”

User callbacks run in unlinked processes:

let worker =
  process.spawn_unlinked(fn() {
    let result = operation()
    process.send(state.subject, OperationCompleted(id:, result:))
  })

let monitor = process.monitor(worker)
let timer =
  process.send_after(state.subject, timeout, OperationTimedOut(id:))

This is where “let it crash” becomes a design tool rather than a slogan. A broken callback is allowed to die, but its failure is contained. Because the worker is not linked to the flight, it cannot take the waiter-owning actor down with it. The monitor turns the death into a WorkerDown message, which the flight can handle as OperationFailed or retry.

There are three outcomes:

• completion: cancel the timer and settle the result;
• crash: cancel the timer and handle OperationFailed;
• timeout: stop monitoring, kill the worker, and handle Timeout.

The ownership boundary matters more than whether a process crashes: unsafe user code is disposable, while the process coordinating callers stays alive long enough to give them one shared outcome.

Retries belong to the flight

Retries cannot belong to individual callers. If five waiters retried independently, the implementation would stop being single flight. One flight owns one attempt sequence:

let retrying =
  settings.new()
  |> settings.retry(
    max_retries: 2,
    delay: 100,
    on: settings.FailuresAndTimeouts,
  )
  |> single_flight.operation_settings(instance)

max_retries: 2 means at most three attempts including the first. Retries may target worker failures, timeouts, or both. All waiters remain attached to the same flight. A successful value is sent to all of them; a final runtime error is delivered only to checked waiters created by call.

This does not guarantee exactly-once execution. A callback can complete an external side effect and then crash, or time out while external work continues. Side-effecting callbacks still need idempotency.

Supervision and recovery boundaries

The coordinator and the factory supervisor run under OneForAll:

static_supervisor.new(static_supervisor.OneForAll)
|> static_supervisor.add(flight_supervisor.supervised(
  config.flight_supervisor_name,
))
|> static_supervisor.add(coordinator.supervised(
  config.coordinator_name,
  config.flight_supervisor_name,
  config.settings,
))

They form one consistency boundary. If only the coordinator restarted, it would forget live flights. If only the flight supervisor restarted, the coordinator could retain dead entries. Restarting both restores a consistent empty runtime.

Together with the Temporary flight policy shown earlier, this gives a clear recovery boundary: durable infrastructure restarts together, while disposable keyed work is recreated only when a caller asks for it again.

An application with its own supervision tree can use supervised and connect after the parent has started the returned child specification:

let assert Ok(#(pending, single_flight_child)) =
  single_flight.new()
  |> single_flight.with_operations(operations)
  |> single_flight.supervised

let assert Ok(_) =
  static_supervisor.new(static_supervisor.OneForOne)
  |> static_supervisor.add(single_flight_child)
  |> static_supervisor.start

let assert Ok(instance) = single_flight.connect(pending)

call and send

call waits forResult(result, single_flight/error.Error). It monitors the flight while waiting, so a dead flight becomes FlightUnavailable instead of an endless receive.

send accepts a subject and returns after the caller has joined the flight. The subject receives successful values only; runtime errors are not delivered to it. Use call when the caller must observe failures such as Timeout or TooManyWaiters.

The library treats an application's own Result as an ordinary successful value. If a callback returns Result(User, ApiError), SingleFlight errors still describe only the runtime around that callback.

The first caller for a scoped key also defines that flight's callback and operation settings. Later callers join the active flight; they do not replace its retry, cache, timeout, or waiter policy.

Each synchronous caller still has its own receive deadline. A caller may return Timeout while a longer shared flight continues for other waiters. Version 1 has no caller cancellation, so its reply subject remains attached until that flight settles.

Caching without erasing result types

Active deduplication and completed-result caching share the same flight actor but solve different problems:

• the coordinator owns cache metadata, TTL timers, and LRU order;
• the retained flight owns the typed result in Completed(result) or CompletedError(error.Error).

This keeps heterogeneous results out of a dynamic container. The coordinator only needs a pid, approximate size, expiry timer, and access order.

settings.new()
|> settings.cache(
  success: settings.CacheFor(500),
  error: settings.NoCache,
)

The policies are NoCache, CacheFor(milliseconds), and CacheUntilEvicted. Instance-wide item and byte limits bound retained state. The byte size comes from Erlang external-term encoding, so it is an approximation rather than process-heap usage.

Successes and SingleFlight runtime errors have separate retention policies. Nil is cached like any other successful value.

LRU and TTL eviction send Stop to the retained flight. Expiry messages include both the operation identity and pid, preventing an old timer from deleting a replacement flight for the same key.

Backpressure

BEAM can run many lightweight processes, so these settings are application limits rather than the VM's process limit. They make overload visible and predictable before memory, scheduler time, sockets, ports, or the downstream service become the real bottleneck.

A slow key may collect many waiting subjects, while many distinct keys create a flight actor and callback worker each. The runtime therefore has separate controls:

Setting	Bounds
max_waiters	Callers attached to one active flight.
settings.max_flights	Distinct keys executing at once.
Cache item and byte limits	Completed state retained by the runtime.

When the active-flight limit is reached, a new key receives TooManyFlights, while a duplicate key may still join its existing flight.

The useful number is not “how many processes can BEAM theoretically open?” It is how much concurrent work this application and its dependencies can handle. TooManyWaiters and TooManyFlights provide explicit failure paths instead of letting load grow until another resource fails.

Per-operation handles may change timeout, retry, cache, and waiter policies. Active-flight and cache resource limits belong to the already running instance, so creating another handle cannot resize shared runtime capacity.

The current implementation coordinates one BEAM node. A future distributed version could route a scoped key to one owner node and spread different keys across the cluster. That requires real distributed ownership and failure handling; simply hashing a key or increasing the worker count would not prevent duplicate execution during node failures or inconsistent cluster views.

Summary

Working with several processes and supervisors required a better understanding of how the BEAM actually behaves: who owns state, how messages are ordered, what links and monitors do, and which process a supervisor should restart.

“Let it crash” does not mean ignoring failures. Here it means running user code in an unlinked worker, monitoring it, and keeping the flight actor alive to handle the result, timeout, crash, or retry for its callers. The coordinator and supervisors then handle recovery at the level where state can be rebuilt safely.

Single flight was a useful exercise because the public idea is small, while the implementation touches most of these process boundaries without needing a large application around it.