Shuttle: Small, Type-Safe Composition Primitives for Go

Sorting a slice by one field is easy. Filtering one collection is easy. Returning (T, bool) is idiomatic. So is writing a nested loop.

The friction appears when the same ordering must be shared by a stable sort and an extrema operation, a filter must be reused across several APIs, or a nested traversal grows into four nearly identical loops. At that point, the code is still simple locally, but the semantics are scattered across call sites.

Shuttle is an attempt to give those semantics small, typed values. It is not a general-purpose functional programming framework, and it is not a port of Java Stream. Its scope is four focused abstractions: comparators, predicates, optional values, and lazy streams.

What Shuttle is

Shuttle is one Go module containing four packages:

• comparator defines Func[T], a named func(T, T) int for reusable three-way orderings.
• predicate defines Func[T], a named func(T) bool with short-circuiting composition.
• optional defines an eager Optional[T] whose presence bit is independent of the value of T.
• stream defines a lazy, ordered, sequential Stream[T] over iter.Seq[T].

The types compose through ordinary Go assignability. A predicate.Func[T] can be passed directly to Optional.Filter or Stream.Filter; a comparator.Func[T] can be passed directly to slices.SortStableFunc, Stream.SortedFunc, or the Stream extrema terminals. The consuming packages do not need to import the descriptor packages to make that work.

The module has no third-party runtime dependencies. It deliberately does not include a root shuttle package, an error-carrying stream, parallel operators, I/O sources, or a collectors framework.

A realistic nested-data example

The repository includes an executable examples/animals program. Its data model contains orders, families, species, subspecies, and animals. The core traversal is a direct adaptation of that example:

func animalsFromOrders(orders []AnimalOrder) stream.Stream[Animal] {
  return stream.FromSlice(orders).
    FlatMapSlice(func(order AnimalOrder) []AnimalFamily {
      return order.Families
    }).
    FlatMapSlice(func(family AnimalFamily) []AnimalSpecies {
      return family.Species
    }).
    FlatMapSlice(func(species AnimalSpecies) []AnimalSubspecies {
      return species.Subspecies
    }).
    FlatMapSlice(func(subspecies AnimalSubspecies) []Animal {
      return subspecies.Animals
    })
}

func adultForestAnimals(orders []AnimalOrder) []Animal {
  adult := predicate.On(
    func(animal Animal) int { return animal.Age },
    predicate.Func[int](func(age int) bool { return age >= 3 }),
  )
  inForest := predicate.On(
    func(animal Animal) string { return animal.Habitat },
    predicate.Equal("forest"),
  )

  byAgeDescendingThenName := comparator.
    ByDescending(func(animal Animal) int { return animal.Age }).
    ThenBy(func(animal Animal) string { return animal.Name })

  return animalsFromOrders(orders).
    Filter(adult.And(inForest)).
    SortedFunc(byAgeDescendingThenName).
    Collect()
}

func oldestAnimalName(orders []AnimalOrder) string {
  return animalsFromOrders(orders).
    MaxBy(func(animal Animal) int { return animal.Age }).
    Map(func(animal Animal) string { return animal.Name }).
    OrElse("none")
}

FlatMapSlice preserves both outer and inner encounter order. It does not defensively copy or cache the slices returned by the callback. The filter reuses two projected predicates; the comparator expresses one descending level followed by an ascending tie-breaker.

Nothing is traversed while these Stream pipelines are being assembled. Collect and MaxBy start traversal. SortedFunc is lazy at construction, but once traversed it is a barrier: it must collect the complete finite input before emitting a stably sorted value. MaxBy is also a full-consumption terminal, and its result is an Optional[Animal] that is mapped eagerly to a name.

The checked-in example produces eligible animals in this order:

[Shere Khan Koko Binti]

Comparators as values

comparator.Func[T] has the same underlying function type used by the standard library:

type Func[T any] func(left, right T) int

Only the sign of the result matters. The comparator package does not validate ordering laws, own or sort a collection, or cache projected keys.

For a mixed ordering, the descriptor stays separate from the consumer:

type Result struct {
  Name  string
  Score int
}

input := []Result{
  {Name: "beta", Score: 2},
  {Name: "alpha", Score: 1},
  {Name: "gamma", Score: 1},
}

byScoreThenNameDescending := comparator.
  By(func(value Result) int { return value.Score }).
  ThenByDescending(func(value Result) string { return value.Name })

standard := slices.Clone(input)
slices.SortStableFunc(standard, byScoreThenNameDescending)

streamed := stream.FromSlice(input).
  SortedFunc(byScoreThenNameDescending).
  Collect()

Lexicographic levels run from left to right and stop at the first nonzero result. ThenByDescending reverses only the appended level; Reverse reverses the complete ordering built so far. That distinction matters as soon as primary keys differ.

This is useful when the ordering is a domain descriptor that belongs in more than one operation. For a one-off comparison, an inline function is still shorter. Also note that projections run again for every reached comparison. If computing a key is expensive, precomputing keys in an explicit loop may be the better design.

Composable predicates

predicate.Func[T] is similarly small:

type Func[T any] func(T) bool

Its methods are And, Or, and Not. And stops at the first false result; Or stops at the first true result. Evaluation is synchronous, serial, and left to right, with the same reached-versus-skipped behavior as && and ||. Panics are not recovered.

The animals example uses predicate.On to project an Animal to an age or habitat before evaluating a predicate over that field. Helpers include Equal, EqualFunc, Always, IsNil, and IsNotNil. Reflection is confined to the two nil helpers so they can recognize typed nils stored in interfaces; ordinary equality and composition are not reflective.

An inline expression such as animal.Age >= 3 && animal.Habitat == "forest" is often clearest. A named predicate earns its keep when the rule must be passed around, tested independently, projected onto another type, or assembled differently by multiple callers.

Optional without treating zero values as absence

The zero value of Optional[T] is None, but the zero value of T can be present. Some(0), Some(""), and Some(false) are all present.

Pointers expose the distinction most clearly:

none := optional.None[*Animal]()
presentNil := optional.Some[*Animal](nil)
presentValue := optional.Some(&Animal{Name: "Koko"})

nilPayload, ok := presentNil.Value()

fmt.Println(none.IsNone())         // true
fmt.Println(ok, nilPayload == nil) // true true
fmt.Println(presentValue.IsSome()) // true

Logically, Optional still has two states: absent or present with one T. When T is a pointer, the payload can itself be nil, so an application can distinguish absent, present-nil, and present-non-nil when that distinction is meaningful.

Operations such as Map, FlatMap, Filter, and Match are eager and invoke only the selected branch. Value() converts back to Go's (T, bool) shape, while Of(value, ok) adapts from it.

This is not a claim that Optional should replace (T, bool) or (T, error). Those forms are usually the best API boundary in Go. Optional[T] is useful when absence itself needs to participate in a longer value transformation, as with MaxBy(...).Map(...).OrElse(...) above.

There is one important JSON caveat: both None[*T]() and Some[*T](nil) encode as null, and decoding null produces None. The presence bit of a present nil therefore does not round-trip through JSON.

Lazy, ordered Stream

Stream[T] is a descriptor around iter.Seq[T], not a stored collection. Its intermediate operations are lazy at construction and, unless documented otherwise, preserve encounter order.

The repository's infinite-source example shows the demand model:

values := stream.Iterate(1, func(value int) int { return value + 1 }).
  Filter(func(value int) bool { return value%2 == 0 }).
  Map(func(value int) int { return value * value }).
  Take(5).
  Collect()

fmt.Println(values) // [4 16 36 64 100]

Construction invokes none of those callbacks. Collect starts traversal; Take(5) propagates termination as soon as five post-filter values have been accepted. Incremental operators such as Map, Filter, FlatMapSlice, Take, Chunk, and Window process demand as it arrives. SortedFunc and Reverse are different: they are construction-lazy finite-input barriers and cannot emit before the source ends.

Shuttle adds no worker pool, hidden error channel, automatic cache, or replay buffer. It does not silently make a single-use iterator reusable. FromSeq preserves the source's replay and cleanup behavior, while built-in sources such as Of, FromSlice, and Range are reusable. Copying a Stream copies only its descriptor.

Errors remain explicit. There is a short-circuiting ForEachErr terminal, but the Stream itself never stores a latent error. Errorful transformations need an element type chosen by the caller; I/O and channel sources are outside the current scope because cancellation and ownership require additional contracts.

Design constraints, not just fluent syntax

The most useful part of the project may be its written contract. DESIGN.md explains the choices, while API_SPEC.md specifies callback order, short-circuiting, ownership, nilness, replay, barriers, and infinite-input behavior.

A few constraints shape the implementation:

• Optional is eager; Stream intermediates are lazy. Laziness is used where there is a sequence to defer, not applied everywhere as a style.
• A zero Optional is None, and a zero Stream is empty. A zero comparator or predicate is an ordinary nil function, not an invented identity.
• Encounter order is preserved unless an operation explicitly changes it. SortedFunc is stable, DistinctBy retains first occurrences, and GroupBy orders groups by first key encounter.
• Of shallow-snapshots its variadic slice; FromSlice is the explicit zero-copy view. Chunk and window results have independently owned backing arrays.
• Shuttle starts no worker goroutines, catches no panics, and adds no synchronization around caller-owned state.
• Runtime packages use only the standard library.

Allocation behavior is treated as part of review rather than a slogan. The repository has tests requiring zero allocation per comparator or predicate evaluation after construction when callbacks do not allocate. Its allocation tests also check that selected stateless Stream pipelines and FlatMapSlice do not add allocations that grow with element count. Stateful maps, sorting buffers, collected results, chunks, and windows allocate according to their semantics. The design explicitly says Shuttle does not promise to beat a hand-written loop; BENCHMARKS.md defines how regressions are reviewed instead of publishing context-free numbers.

Go 1.27 generic methods make fluent type-changing calls such as Stream[T].Map[R] possible. They are also why Shuttle requires Go 1.27. Some constrained or structurally expanding operations remain package functions—such as stream.Sorted, stream.Chunk, and stream.Zip—because of Go's receiver constraints and a compiler limitation validated against Go 1.27.0.

Why not just write loops?

Often, you should.

A loop is direct, familiar, easy to debug, and gives precise control over allocation, errors, cancellation, and resource lifetime. For a single transformation or a performance-critical path, adding a Stream pipeline can make the code less obvious rather than more obvious.

Shuttle becomes more interesting when the operation itself is reusable: an ordering shared by slices and Stream terminals, a predicate assembled from independently tested rules, an optional result transformed without repeatedly unpacking it, or a nested iterator pipeline whose demand and encounter order matter.

That is a tradeoff, not a universal upgrade. The useful evaluation question is whether the descriptor makes an important rule more visible at its call sites. If readers must mentally translate the fluent chain back into a loop every time, the loop probably wins.

Current status: v0.1.0, before v1

The current release is v0.1.0, the first public pre-v1 release. It requires Go 1.27 or newer, with Go 1.27.0 as the documented development and release-validation baseline. The project is licensed under MIT, and its runtime packages have only standard-library dependencies.

This is not a stability claim. The current release is intended for evaluation and API review. Incompatible changes may still occur before v1.0.0 and should be expected if review finds a better boundary or a semantic correction.

Useful feedback includes whether the names and method-versus-function split feel natural in Go, whether the iterator and ownership contracts cover real sources, whether Optional earns its cost at call sites, and whether any proposed addition fits the deliberately narrow scope.

To try the tagged release in a Go 1.27 module:

go get github.com/imbrooklyn/shuttle@v0.1.0

After cloning the repository, run the complete example and tests:

go run ./examples/animals
go test ./...

The code, design document, API specification, examples, and issue tracker are at:

https://github.com/imbrooklyn/shuttle

If these abstractions fit a real problem in your codebase, try them and open an issue where the semantics or API feel wrong. Review and design feedback before v1 is especially valuable; contributions are welcome, and a star is useful if you want to follow the project.