DEV Community: Attila Večerek

Effective Pragmatism: Observability

Attila Večerek — Thu, 06 Mar 2025 09:58:03 +0000

Just like in the previous post, let's start with the definition.

Observability is the ability to understand the internal state or condition of a complex system based solely on knowledge of its external outputs, specifically its telemetry.¹

There are a few key terms we need to understand:

internal state or condition: is a particular service up and running, are the requests being processed, queued, or dropped, is the service operating as expected, etc?
external outputs and telemetry: logs, metrics, and traces are the three pillars of observability and the main types of telemetry. They are considered external outputs because applications emit these data, for example, by writing them to the file system or sending them elsewhere over the network.

Observability provides businesses with deep visibility into their systems' inner workings and overall state. The importance of observability varies depending on a company's stage of development, with different priorities emerging at each phase.

Startup: Observability is moderately important at this stage. The primary focus is on rapid development, but it's still crucial to identify and resolve critical errors quickly to maintain early customer trust. Observability tools and manual code instrumentation help engineers debug issues in production efficiently.
Scale-up: As the company grows, rapid scaling becomes the top priority. In addition to quick debugging, observability tools help prevent production issues as more customers are onboarded. They also assist in detecting scaling challenges, such as slow database queries. The ability to generate telemetry data easily from application code becomes essential.
Corporate, enterprise, and mega-corp: At this level, observability is a mission-critical software quality. Downtime can cost millions of dollars per hour, leading to financial loss, reputational damage, and eroded customer trust. Observability helps prevent downtime by providing key signals for automated recovery actions (self-healing capabilities). Additionally, it enhances cost efficiency by offering insights into resource utilization (CPU, RAM, network bandwidth, etc.) across individual workloads.

Observability presents one major challenge for businesses: cost. The key is finding the right balance between cost and insight.

A common mistake companies make is becoming overly reliant on a single vendor, which can lead to vendor lock-in². Depending on the observability provider, businesses may have limited control over their expenses. For example, if all logs are sent directly to a vendor, charges will apply for every log line and trace ingested into their system. In such cases, the only available cost-control mechanism may be adjusting the sampling rate, which reduces indexing costs but does not eliminate ingestion fees.

OpenTelemetry³ is a vendor-neutral framework for emitting metrics, traces, and logs. It supports head sampling⁴, allowing us to decide whether to sample or discard a trace or span before the data is ingested by a vendor.

Effect natively integrates with OpenTelemetry, providing built-in APIs for emitting logs⁵, metrics⁶, and traces⁷.

In the industry, it is common to use separate libraries for each telemetry type. Additionally, different dependencies within a project may rely on different sub-dependencies for emitting telemetry data. Each dependency may also configure telemetry output differently — for example, some may emit unstructured logs, while others produce structured logs in various formats. Inconsistent telemetry data is more difficult to analyze. Furthermore, applications can end up with multiple redundant dependencies scattered throughout the dependency tree. This leads to:

An increased attack surface, making the system more vulnerable to security risks.
Additional boilerplate, complicating seamless telemetry integration.
A higher maintenance burden, adding unnecessary complexity and overhead.

The following is a list of popular libraries for producing telemetry:

Logs: pino, winston, bunyan
Traces: dd-trace-js, sentry, opentelemetry-api
Metrics: hot-shots, opentelemetry-sdk-metrics

In this article, I will focus on tracing to highlight the benefits of using Effect but the same benefits apply across all types of telemetry.

Tracing

In my opinion, traces are the most universal form of telemetry. A single trace can reveal:

How long an operation takes
Which processes are executed within the operation
Which process is the bottleneck
Which process failed
And, thanks to trace-log correlation⁸, possibly why

Traces can also serve as the foundation for derived metrics, allowing us to visualize request and error rates, latency percentiles, and other key performance indicators. Given their depth of insight, it's no surprise that traces are often at the forefront of a company's observability strategy and the primary telemetry type used in Application Performance Monitoring (APM)⁹.

Let's revisit our example from the previous post about creating an article. First, let's see how we would add manual instrumentation using a conventional tracing client.

const createArticle = (input: unknown, currentUser: User, tracer: Tracer) => {
  return tracer.trace("createArticle", { tags: { input, currentUser } }, async (span) => {
    try {
      authorize(currentUser, ["READ_CATEGORY", "WRITE_ARTICLE"], tracer)
      const { categoryId, ...article } = parseInput(input)
      await enforceCategoryExists(categoryId, tracer)
      await enforceArticleLimitInCategory(categoryId, tracer)
      const persistedArticle = await persistArticle(article, tracer)

      return { status: 201, article: persistedArticle }
    } catch (e) {
      if (e instanceof /* ??? */) {}
    }
  })
}

These are the key issues with this approach:

Nested callbacks reduce readability
- Wrapping the entire function body inside tracer.trace introduces deep nesting.
- When overused, this reduces code readability, which is a crucial software quality that directly impacts developer productivity.
Manual propagation of the tracer instance
- We must explicitly pass the tracer instance to the instrumented function.
- If we want all underlying actions to be instrumented, we must continue passing it down. In frontend development, this pattern is often called prop drilling¹⁰.

While this may not seem like a major issue in an isolated example, consider a large-scale codebase with hundreds or thousands of files. Repetitive manual instrumentation like this compounds over time, leading to reduced engineering productivity.

There’s a better way — the Effect way. 🚀

const createArticle = (input: unknown, currentUser: User) => Effect.gen(function* () {
  // happy path
}).pipe(
  Effect.withSpan("createArticle", { attributes: { input, currentUser } })
)

In Effect, manual instrumentation works through composition. We simply add a call to Effect.withSpan¹¹ within the pipeline, following the happy path logic. Additionally, there's no need to manually pass a reference to a tracer — Effect maintains a reference to the tracer client for us, eliminating unnecessary boilerplate.

Let's visualize¹² an example trace of executing createArticle.

Notice that input and currentUser are listed under the attributes of the createArticle span. But what if we wanted currentUser to be added to all spans, not just the one being annotated? With the conventional approach, we would need to pass the tracer reference alongside every annotation. However, with Effect, we can simply use Effect.annotateSpans to achieve this seamlessly.

const createArticle = (input: unknown, currentUser: User) => Effect.gen(function* () {
  // happy path
}).pipe(
  Effect.withSpan("createArticle", { attributes: { input } }),
  Effect.annotateSpans({ currentUser }),
)

Run example

Here's the result after this change:

As you can see, currentUser is now present in all spans.

Why is this useful? Observability platforms index spans and their attributes, making them easy to search and filter. By annotating all spans with a user ID, for example, we can track user-specific patterns, identify performance bottlenecks affecting specific users, and uncover valuable clues while troubleshooting various issues.

Exporting Traces

In the previous examples, we demonstrated how traces are generated. The next step is to show how to export¹³ them.

Using the conventional approach, a tracer must be instantiated and passed down to all functions that require manual instrumentation. This often comes with critical constraints. For example, Datadog's Node.js tracing library must be imported and initialized before any other module for its auto-instrumentation feature to work¹⁴.

Almost no one gets this right on the first try, leading to long debugging sessions and a significant hit on developer productivity when setting up telemetry. This requirement also impacts project structure.

// tracer.ts
import tracer from "dd-trace"

export default tracer.init()

// index.ts
import tracer from "./tracer.js"

The tracer must be initialized in a separate file to prevent hoisting and ensure the correct load order. Then, in index.ts, the tracer must be imported before any other module.

An alternative approach is to initialize the tracer inside the main function and dynamically import the rest of the project, but this adds even more complexity and overhead.

Unlike conventional approaches, Effect simplifies the setup process.

Exporting traces to the IDE

If you're a VS Code user, you can even view traces directly within your IDE. All you need to do is install the Effect DevTools¹⁵ extension and provide it as a dependency when executing your program.

import { DevTools } from "@effect/experimental"
import { NodeRuntime, NodeSocket } from "@effect/platform-node"
import { Effect, Layer } from "effect"

const DevToolsLayer = DevTools.layerWebSocket().pipe(
  Layer.provide(NodeSocket.layerWebSocketConstructor)
)

const program = Effect.void

program.pipe(
  Effect.provide(DevToolsLayer),
  NodeRuntime.runMain
)

The above example shows the simplest program (Effect.void) being run. The focal points of this example are:

the construction of the DevToolsLayer dependency,
its injection into the program using Effect.provide.

We will explore dependency injection in more detail in the upcoming post on testability.

The main takeaway is that Effect allows complete flexibility in structuring our projects. However, it is important to note that this is not a direct comparison to conventional approaches — Effect does not attempt to auto-instrument third-party libraries. Instead, we achieve the same benefits by embracing Effect-first libraries that generate telemetry for us.

This marks a fundamental shift in the JavaScript ecosystem and how libraries are designed. Traditionally, observability has been treated as an afterthought, leaving the responsibility of instrumentation to tracing libraries. This creates friction, as tracing libraries must continuously adapt to changes in the libraries they instrument. For full coverage, every library would require a dedicated auto-instrumentation module for each tracing client — Datadog, Sentry, OpenTelemetry, and others. This approach is difficult to scale and demands significant ongoing maintenance.

Effect challenges the status quo by flipping this responsibility. Instead of relying on external tracing libraries, each library takes full ownership of its own telemetry. Effect then provides a unified platform, ensuring that all telemetry data — whether traces, logs, or metrics — is exported consistently according to the application's configuration.

This approach is clearly beneficial for application developers, but what about library authors? Why should they take ownership of their library’s telemetry?

For one, it lowers the barrier to contribution. Open-source libraries used within commercial products often face challenges when debugging issues, as reproducing certain problems may require replicating part of the commercial environment — including sensitive customer data. However, if the library owns its telemetry, contributors can provide relevant telemetry samples in issue descriptions without exposing sensitive information. This can:

Help pinpoint the root cause more quickly.
Provide insights into how to reproduce the issue.
Reveal instrumentation gaps for further investigation.

Additionally, when a library owns its telemetry, users can more easily locate relevant sections of the source code for debugging, modification, or further exploration. This enhances transparency and improves the overall experience of contributing back to libraries.

One obvious challenge with this approach is cost control. Applications should have full control over which libraries emit telemetry and when. With Effect’s built-in configuration management, libraries can easily regulate their telemetry emission. Even if a library does not explicitly provide such configuration options, OpenTelemetry's head sampling can serve as a fallback solution, ensuring cost efficiency without sacrificing observability.

Exporting traces to a collector

Exporting traces to an OpenTelemetry collector is just as straightforward. Simply replace DevTools with NodeSdk and use the official OpenTelemetry libraries to configure the span processor.

import { NodeSdk } from "@effect/opentelemetry"
import { NodeRuntime } from "@effect/platform-node"
import { BatchSpanProcessor } from "@opentelemetry/sdk-trace-base"
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http"
import { Effect, Layer } from "effect"

const NodeSdkLayer = NodeSdk.layer(() => ({
  resource: { serviceName: "example" },
  spanProcessor: new BatchSpanProcessor(new OTLPTraceExporter())
}))

const program = Effect.void

program.pipe(
  Effect.provide(NodeSdkLayer),
  NodeRuntime.runMain
)

Additionally, we can switch between the two approaches based on the runtime environment:

// imports, DevToolsLayer, NodeSdkLayer

const RuntimeEnvConfig = Config.literal("development", "staging", "production")

const TelemetryLayer = Effect.gen(function* () {
  const env = yield* RuntimeEnvConfig("RUNTIME_ENV")

  return env === "development"
    ? DevToolsLayer
    : NodeSdkLayer
}).pipe(Layer.unwrapEffect)

// program

program.pipe(
  Effect.provide(TelemetryLayer),
  NodeRuntime.runMain
)

Summary

Let's recap the key takeaways about observability:

Systems generate external outputs — telemetry — such as traces, logs, and metrics.
Telemetry provides observability, allowing us to understand system behavior and state.
Observability helps businesses debug production issues, detect errors and performance bottlenecks before customers do, assist engineers in cost reduction efforts, and prevent costly downtime.
Businesses often generate large volumes of telemetry data, which can be expensive.
A common challenge is vendor lock-in, which exacerbates costs by making it difficult, time-consuming, and expensive to switch providers.
OpenTelemetry introduces vendor-neutral observability.
Effect natively integrates with OpenTelemetry, improving readability and eliminating prop drilling.
In the current JavaScript ecosystem, popular libraries are instrumented separately by each tracing client. This approach does not scale and creates maintenance overhead.
With Effect, libraries take full ownership of their telemetry, while Effect provides a unified platform that ensures consistent telemetry production across all dependencies. This eliminates the scalability and maintenance challenges in both the ecosystem and our applications.
Effect enables developers to visualize telemetry data directly within the IDE. This improves developer productivity because telemetry data can be verified before changes are deployed to an environment with an OTEL collector.

Observability is a core feature of Effect, not an afterthought. Combined with its robust error management capabilities, it provides a comprehensive approach to error detection and prevention.

In the previous post, I briefly touched on dependency management. In this post, I explained how Effect maintains an internal reference to the tracer, which is closely tied to its dependency management system. In the next post, we’ll explore testability, diving deeper into how Effect handles dependencies and the advantages this approach offers to both application and library developers.

What is observability? ↩
Vendor lock-in ↩
OpenTelemetry ↩
Head sampling ↩
Effect logging ↩
Effect metrics ↩
Effect tracing ↩
OpenTelemetry log correlation ↩
Application Performance Monitoring ↩
What is prop drilling in React? ↩
Effect.withSpan ↩
Visualizing traces ↩
Trace exporters ↩
Datadog: Import and initialize the tracer ↩
Effect DevTools ↩

Effective Pragmatism: Reliability

Attila Večerek — Thu, 06 Mar 2025 09:57:19 +0000

Let's start with my chosen definition of reliability. It is quite academic but don't worry, we will dissect and explain it in a human and business-centric way.

Reliability is the probability of failure-free software operation for a specified period of time in a specified environment.¹

Reliability is fundamentally about correctness. Time and environment are considered for practical reasons — to make reliability quantifiable and measurable. Companies typically assess reliability based on the environment (development, staging, production) and specific timeframes (e.g., the last 7, 30, or 90 days). It is important to note that reliability is not the same as availability — software can function correctly yet still be unavailable.

Now that we know what reliability is, we can discuss why it is important to the business even at the risk of stating the obvious. When people buy software products and services, they place their expectations on them. These expectations are influenced by the provider's claims of quality. If these expectations are not met, they may walk back on their purchase, ask for a refund, and eventually, turn to the competition for alternatives. A low level of reliability may be one of the reasons for not meeting their expectations. Other reasons include availability issues, low performance, poor user experience, and so on.

How does unreliable software lead to customer churn? When people use our software, they pursue a goal. If they cannot achieve their goal, there is no point in using the software in the first place. To achieve these goals, the software must work correctly as expected by the user. Often incorrectness manifests itself in obvious ways such as internal server errors. If these errors occur intermittently, it is not necessarily a reason to leave for the competition. People have learned to live with occasional issues and intuitively refresh the page when they occur. However, if it happens often enough, it becomes frustrating, compounds over time, and leads to customer churn. The good news is that software providers can easily detect and measure these errors. At other times, incorrectness is very subtle and difficult to spot. For example, mistakes in financial data analytics. People base some serious decisions on this data. If the software presents the wrong numbers, it may have a catastrophic impact on the company using the software.

Does this mean that every company aims to develop equally reliable software? No. While reliability is important for all companies, its significance varies depending on their stage of development.

Startup: Reliability is moderately important. While the highest priority is rapid iteration, a base level of reliability is essential to gain customer trust. The company must ensure that core features function as expected, critical bugs are identified and fixed quickly, and third-party vendors are reliable to minimize infrastructure risks.
Scale-Up: At this stage, poor reliability can stall growth. The challenge is to scale rapidly while minimizing reliability issues. The more reliability suffers, the longer it will take to reach the growth targets. Conversely, excessive focus on reliability can also slow growth. Striking the right balance is crucial. Key investments should include automated alerting and monitoring to detect and resolve failures quickly, as well as redundancy and failover mechanisms to handle traffic spikes.
Corporate: Reliability now directly impacts customer retention, operational efficiency, and revenue. The primary goal is to maintain customer trust. Key investment areas include high availability, multi-regional architecture, strict SLAs to uphold reliability commitments, and service incident post-mortems to prevent recurring failures.
Enterprise and Mega-Corp: At this scale, mission-critical failures can lead to financial, legal, and operational crises. The main objective is to mitigate risks. Key investment areas include disaster recovery plans, predictive analytics, DDoS protection, and a zero-trust security architecture.

Okay, we know what reliability is and what it means to the business. How does Effect improve the reliability of the produced software? Two major features of this framework directly impact reliability: Error Management² and Configuration Management³. I want to focus specifically on these two features for a reason. From my experience in the industry, escaped defects (colloquially known as bugs) and configuration errors cause over 40% of all incidents. Improvements in these two areas only can yield up to twice as reliable software. Can Effect help us get rid of all the bugs? Absolutely not. Can it significantly reduce the surface area of possible bugs? Absolutely. Let me show you how.

Error Management

Error management can be divided into two main concerns:

Error discovery
Error handling

Error discovery

In conventional programming, where errors are not represented as values and are not statically typed and hence known at the time of development, error discovery is a painful and lengthy process. Generally speaking, there are a few approaches to it:

Reading: the engineer would inspect the implementation of each action to an arbitrary level of depth. Remember, in the real world, our programs tend to be very deep. This is because business domains tend to get more complex as time passes, as well as the increased utilization of internal/external libraries to reap some productivity benefits. It is unrealistic to expect all possible errors to be discovered this way.
Trial: the engineer would try different inputs, and simulate different states of the environment (network failures, permutations of stored data, etc.) to discover how the program can fail. It is unrealistic to expect all possible errors to be discovered this way, either.
LLM: the engineer turns to their favorite LLM for advice. The machine activates all its artificial neurons to ask humanity's collective consciousness what to tell the poor soul. Still, we cannot realistically expect to discover all possible errors this way, either; especially in large codebases consisting of hundreds and thousands of files.

As demonstrated, conventional techniques are either labor-intensive or limited in effectiveness. A common approach is to identify obvious errors early while allowing unforeseen issues to surface in production — essentially "taxing the early adopters," who are often paying customers. Some may see this as a disservice.

TypeScript and Effect offer a superior alternative — automating the error discovery phase using the compiler. This process takes only an instant to a few seconds.

type CreateArticle = Effect<Success, Failure, Requirements>

Both success and failure can be encoded in the type system. The industry calls this a "result type"⁴. The Effect type⁵ extends this concept by also encoding dependencies — more on that in a future post.

Error handling

To demonstrate error handling, let's implement a function called createArticle that takes two arguments:

input: The request body data for creating an article.
currentUser: The user making the request.

The function must meet the following requirements:

Only users with READ_CATEGORY and WRITE_ARTICLE permissions can create an article.
The input has an unknown structure and must be parsed.
The article must belong to an existing category.
A category has a limit on the number of articles it can contain.

We’ll explore three different approaches to error handling.

Try-catch approach

const createArticle = async (input: unknown, currentUser: User) => {
  try {
    authorize(currentUser, ["READ_CATEGORY", "WRITE_ARTICLE"])
    const { categoryId, ...article } = parseInput(input)
    await enforceCategoryExists(categoryId)
    await enforceArticleLimitInCategory(categoryId)
    const persistedArticle = await pRetry(
      () => persistArticle(article),
      { retries: 1 }
    )

    return { status: 201, article: persistedArticle }
  } catch (e) {
    if (e instanceof /* ??? */) {
    //               ^^^^^^^^^
    //               no knowledge of what can go wrong
    }
  }
}

The try-catch block is readable and easy to understand but ineffective for error discovery. Since the caught error is typed as unknown, the IDE cannot provide intelligent hints about possible error types. Even with LSP⁶ assistance, there is no guarantee the listed errors will occur at runtime.

Go-style approach

const createArticle = async (input: unknown, currentUser: User) => {
  const [_, unauthorizedError] = authorize(currentUser, ["READ_CATEGORY", "WRITE_ARTICLE"])
  if (unAuthorizedError) {
    return { status: 403 }
  }
  const [parseResult, parseError] = parseInput(input)
  if (parseError) {
    return { status: 400, errors: [{ id: parseError.name, details: parseError.message }] }
  }
  const { categoryId, ...article } = parseResult

  // etc.

  return { status: 201, article: persistedArticle }
}

This approach requires functions to return tuples, ensuring errors are explicitly handled. For example, parseInput could be typed as:

type ParseInput = (input: unknown) =>
  | [CreateArticleInput, false]
  | [void, ParseError]

This forces error handling in most cases. However, since authorize returns void regardless of success or failure, the compiler cannot enforce error handling for it. Additionally, this approach is verbose and negatively impacts code readability compared to try-catch.

Result type approach

Another alternative is to use a basic result type with a tagged union⁷.

type ParseInput = (input: unknown) =>
  | { _tag: "success", value: CreateArticleInput }
  | { _tag: "failure", error: ParseError }

Some developers prefer the explicitness of this type. However, it makes the resulting code even more verbose than the Go-style error-handling approach.

const createArticle = async (input: unknown, currentUser: User) => {
  const authResult = authorize(currentUser, ["READ_CATEGORY", "WRITE_ARTICLE"])
  if (authResult._tag === "failure") {
    return { status: 403 }
  }
  const parseResult = parseInput(input)
  if (parseResult._tag === "failure") {
    return { status: 400, errors: [{ id: parseResult.error.name, details: parseResult.error.message }] }
  }
  const { categoryId, ...article } = parseResult.value

  // etc.

  return { status: 201, article: persistedArticle }
}

Compared to the previous approach, the reliability gains of this method are minimal at best.

Effect approach

Is there a way to combine the readability of try-catch with the reliability of Go's style of error handling? Yes. Effect’s error handling is just as readable as try-catch while providing even stronger reliability guarantees than the Go approach.

Don't worry if the following code seems complex — we'll break it down step by step in the next section.

import { Effect, Schema } from "effect"
import { ArticleBody, ArticleTitle, authorize, CategoryID, enforceArticleLimitInCategory, enforceCategoryExists, persistArticle, type User } from "./domain.js"
import { exhaustiveErrorCheck } from "./helpers.js"

class Input extends Schema.Class<Input>("Input")({
  categoryId: CategoryID,
  title: ArticleTitle,
  body: ArticleBody
}) {}

const parseInput = Schema.decodeUnknown(Input)

const createArticle = (input: unknown, currentUser: User) => Effect.gen(function*() {
  yield* authorize(currentUser, ["READ", "WRITE"])
  const { categoryId, ...article } = yield* parseInput(input)
  yield* enforceCategoryExists(categoryId)
  yield* enforceArticleLimitInCategory(categoryId)
  const persistedArticle = yield* persistArticle(article).pipe(
    Effect.retry({ times: 1 })
  )

  return { status: 201, article: persistedArticle }
}).pipe(
  Effect.catchTags({
    "ArticleLimitExceededError": ({ name: id }) => Effect.succeed({ status: 400, errors: [{ id }] }),
    "CategoryNotFoundError": ({ name: id }) => Effect.succeed({ status: 400, errors: [{ id }] }),
    "ParseError": (e) => Effect.succeed({ status: 400, errors: [{ id: e.name, details: e.message }]}),
    "UserUnauthorizedError": () => Effect.succeed({ status: 403 }),
    "SqlError": (e) => Effect.die(e),
  }),
  exhaustiveErrorCheck
)

Run example

Let's focus on the business logic first. The program executes the following sequence of actions, any of which can fail in some way and short-circuit the flow:

authorize: makes sure that the user has the specified permissions needed to create an article. This action may fail with UserUnauthorizedError.
parseInput: takes some data of an unknown shape and parses it into data of an expected shape. I chose to type the input as unknown to model the system boundary. The request body of an incoming HTTP request is usually a stream of characters. When this function receives the input, though, we expect it was already parsed partially, e.g. using JSON.parse which usually returns an unsafe any type. Typing it as unknown leads to better reliability by forcing us to properly parse the data. This action may fail with ParseError.
enforceCategoryExists: makes sure that categoryId points to an existing category. Assume that our domain requires all articles to be created under a specific category. This action may fail with CategoryNotFoundError.
enforceArticleLimitInCategory: assume that our domain places a limit on how many articles can be placed in a given category. This function makes sure that this limit has not yet been reached before continuing with the rest of the flow. This action may fail with ArticleLimitExceededError.
persistArticle: attempts to save the article in our persistence layer. This action may be retried exactly once in case of failure. It may fail with a rather obscure SqlError. In practice, this error could be modeled as a union of specific SQL errors to distinguish between retriable and non-retriable errors.

const createArticle = (input: unknown, currentUser: User) => Effect.gen(function*() {
  yield* authorize(currentUser, ["READ", "WRITE"])
  const { categoryId, ...article } = yield* parseInput(input)
  yield* enforceCategoryExists(categoryId)
  yield* enforceArticleLimitInCategory(categoryId)
  const persistedArticle = yield* persistArticle(article).pipe(
    Effect.retry({ times: 1 })
  )

  return { status: 201, article: persistedArticle }
})

The above excerpt of our earlier code example only deals with the happy path. In Effect, this would be a valid program that compiles. TypeScipt is fully capable of inferring the type of this program thanks to the use of generators⁸. The signature would look similar to the following:

type Success = { status: 201; article: Article }

type Failure = 
  | ArticleLimitExceededError
  | CategoryNotFoundError
  | ParseError
  | UserUnauthorizedError
  | SqlError

type CreateArticle =
  (input: unknown, currentUser: User) => Effect<Success, Failure>

As we can see, Effect is extremely composable. TypeScript's inference capabilities coupled with Effect's composability help us write reliable software. It also provides flexibility in error handling. It does not force us to handle all expected errors. We have the freedom to handle as many or as few of the errors as we see fit. It even lets us enforce exhaustive error handling. To do just that, we can implement a simple helper function.

import { type Effect } from "effect"
import { satisfies } from "effect/Function"

export const exhaustiveErrorCheck = satisfies<Effect.Effect<any, never, any>>()

The above is an identity function ((a) => a) that enforces that the input is an effect that can never fail. Let's see how to use this in practice.

pipe(
  Effect.catchTags({
    "ArticleLimitExceededError": ({ name: id }) => Effect.succeed({ status: 400, errors: [{ id }] }),
    "CategoryNotFoundError": ({ name: id }) => Effect.succeed({ status: 400, errors: [{ id }] }),
    "ParseError": (e) => Effect.succeed({ status: 400, errors: [{ id: e.name, details: e.message }]}),
    "UserUnauthorizedError": () => Effect.succeed({ status: 403 }),
    "SqlError": (e) => Effect.die(e),
  }),
  exhaustiveErrorCheck
)

This excerpt from our original code example demonstrates error handling which is visually and logically separated from the happy path with a pipeline⁹. Effect.catchTags¹⁰ is a way to define multiple error handlers at once. exhaustiveErrorCheck makes sure that all expected errors have a defined handler. If we were to remove the SqlError handler, the program would compile with the following error:

Argument of type '<Success, Requirements, E extends Effect.Effect<Success, never, Requirements>>(effect: E) => E' is not assignable to parameter of type '(_: Effect<{ status: number; article: { title: string & Brand<"ArticleTitle">; body: string & Brand<"ArticleBody">; id: string & Brand<"ArticleID">; }; } | { status: number; } | { ...; } | { ...; } | { ...; }, SqlError, never>) => Effect<...>'.
  Types of parameters 'effect' and '_' are incompatible.
    Type 'Effect<{ status: number; article: { title: string & Brand<"ArticleTitle">; body: string & Brand<"ArticleBody">; id: string & Brand<"ArticleID">; }; } | { status: number; } | { ...; } | { ...; } | { ...; }, SqlError, never>' is not assignable to type 'Effect<unknown, never, unknown>' with 'exactOptionalPropertyTypes: true'. Consider adding 'undefined' to the types of the target's properties.
      Type 'SqlError' is not assignable to type 'never'.(2345)

As with most TypeScript errors, the key information is found at the bottom of the message: Type 'SqlError' is not assignable to type 'never'. This means we did not handle the case of SqlError. Additionally, exhaustive error checking also helps when new error cases are added, for example, by augmenting the persistArticle action to also emit a Kafka message and possibly fail with KafkaProducerError.

Notice that all errors, except for SqlError, are handled by resolving the effect successfully using Effect.succeed¹¹. That does not mean that the article was created successfully. It simply means that the operation of creating an article is executed according to expectations. That is, when the user is not authorized to create an article, the function successfully prevents them from creating one.

The handler for SqlError resolves the effect using Effect.die¹². If the code execution gets to this point, it means the retry of persistArticle failed. We could retry more times, add an exponential backoff schedule¹³ with jitter, etc. but if all attempts fail, we would still have to handle the no longer retriable SqlError. The only thing left for us to do at that point is to handle the case as an unrecoverable error¹⁴.

After we handle all expected errors, either with Effect.succeed or Effect.die, TypeScript's inference understands that the handled errors should no longer appear in the inferred type of the effect. The new signature of the createArticle function would look similar to the following:

type Success =
  | { status: 201; article: Article }
  | { status: 400; errors: ArticleError[] }
  | { status: 403 }

type CreateArticle =
  (input: unknown, currentUser: User) => Effect<Success, never>

TypeScript and Effect provide a world-class developer experience in terms of error handling. The IDE¹⁵ is our friend and suggests what error cases to handle as we implement the block of code in Effect.catchTags.

Configuration Management

All programs run in a certain environment. Configuration can be viewed as the API between the environment and the program. The program declares its requirements, and the environment tries to satisfy them. This establishes a consumer-provider relationship between the program and its environment. This fact has a serious reliability implication in practice. A program and its environment can change independently of each other. Since the requirements are defined in the program but the configuration values live in the environment, the possibility of the two drifting, especially in complex systems, is very high. That can, and in practice often does, cause service incidents. To prevent that from happening, we have to carefully design our software as well as the deployment pipeline. Should the environment not fully satisfy the program's requirements, its deployment must fail; keeping the healthy version of the software and its environment running. Effect can help us design such software.

Effect is fully aware of this dual nature of managing configuration. This is why it models it as such. The Config module implements the tools necessary to declare the configuration to be consumed by the program, while the ConfigProvider module implements the tools necessary to provide, or extract, the configuration from the environment (environment variables, file system, network,...).

The following example is a simple program declaring some configuration it requires to run correctly.

import { Config, Effect } from "effect"
import { MysqlConfig, RuntimeEnvConfig } from "./config"

const program = Effect.gen(function*() {
  const config = yield* Config.all({
    mysql: MysqlConfig,
    port: Config.number("PORT").pipe(
      Config.withDefault(3000)
    ),
    runtimeEnv: RuntimeEnvConfig("RUNTIME_ENV")
  })

  yield* Effect.log(config)
})

Run example

Notice, that the code in the example does not say anything about how to extract the configuration from the environment. It does not say if they should be read from the environment variables, the file system, or some other place. That is the responsibility of the ConfigProvider. Let's see how that fits in the picture by demonstrating how to run this program.

import { NodeRuntime } from "@effect/platform-node"
import { program } from "./main.js"

NodeRuntime.runMain(program)

By default, Effect uses its built-in ConfigProvider that extracts the configuration from the environment variables. We can fully modify this behavior, and we'll do so very soon. But first, let's take a look at the errors produced by this program when the environment does not satisfy the requirements.

Error: (Missing data at MYSQL.USERNAME: "Expected MYSQL_USERNAME to exist in the process context")
and (Missing data at MYSQL.PASSWORD: "Expected MYSQL_PASSWORD to exist in the process context")
and (Missing data at MYSQL.HOST: "Expected MYSQL_HOST to exist in the process context")
and (Missing data at MYSQL.PORT: "Expected MYSQL_PORT to exist in the process context")
and (Missing data at RUNTIME_ENV: "Expected RUNTIME_ENV to exist in the process context")

I formatted the above message to slightly improve its readability. As we can see, the following environment variables are missing: MYSQL_USERNAME, MYSQL_PASSWORD, MYSQL_HOST, MYSQL_PORT, RUNTIME_ENV. Notice that Effect collects all unsatisfied configuration errors, instead of failing fast. This massively improves the developer experience when faced with multiple missing environment variables. What we can also notice is that the program does not complain about the PORT configuration because it was instructed to fall back to a default value in case it was missing.

So, what if we wanted some of the configuration to be read from the file system instead of the environment variables for the reasons of enhanced security? We can do so by providing an additional ConfigProvider that will read the configuration from the file system if it's not available in the form of an environment variable.

import { PlatformConfigProvider } from "@effect/platform"
import { NodeContext, NodeRuntime } from "@effect/platform-node"
import { Effect } from "effect"
import { program } from "./main.js"

program.pipe(
  Effect.provide(PlatformConfigProvider.layerFileTreeAdd({
    rootDirectory: "/secrets"
  })),
  Effect.provide(NodeContext.layer),
  NodeRuntime.runMain
)

If we run the program again without satisfying the requirements first, we'll see the following error.

Error: (Missing data at MYSQL.USERNAME: "Expected MYSQL_USERNAME to exist in the process context")
or (Missing data at MYSQL.USERNAME: "Path /secrets/MYSQL/USERNAME not found")
and (Missing data at MYSQL.PASSWORD: "Expected MYSQL_PASSWORD to exist in the process context")
or (Missing data at MYSQL.PASSWORD: "Path /secrets/MYSQL/PASSWORD not found")
and (Missing data at MYSQL.HOST: "Expected MYSQL_HOST to exist in the process context")
or (Missing data at MYSQL.HOST: "Path /secrets/MYSQL/HOST not found") 
and (Missing data at MYSQL.PORT: "Expected MYSQL_PORT to exist in the process context")
or (Missing data at MYSQL.PORT: "Path /secrets/MYSQL/PORT not found") 
and (Missing data at RUNTIME_ENV: "Expected RUNTIME_ENV to exist in the process context")
or (Missing data at RUNTIME_ENV: "Path /secrets/RUNTIME_ENV not found")

Notice that the error is more verbose now. We can see that it tried looking up the configuration both in the process context (environment variables) and the file system.

Let's see what the output looks like when we do satisfy all the requirements. There is one small problem, though. The example runs in a closed sandbox environment. We cannot provide any environment variables, nor can we write to the file system. Don't worry because Effect got us covered! We can provide the configuration explicitly from the code itself using ConfigProvider.fromMap.

import { PlatformConfigProvider } from "@effect/platform"
import { NodeContext, NodeRuntime } from "@effect/platform-node"
import { Effect } from "effect"
import { program } from "./main.js"

program.pipe(
  Effect.provide(PlatformConfigProvider.layerFileTreeAdd({
    rootDirectory: "/secrets"
  })),
  Effect.provide(NodeContext.layer),
  Effect.withConfigProvider(
    ConfigProvider.fromMap(
      new Map([
        ["MYSQL_USERNAME", "abcde"],
        ["MYSQL_PASSWORD", "12345"],
        ["MYSQL_HOST", "dummy host"],
        ["MYSQL_PORT", "3306"],
        ["RUNTIME_ENV", "development"],
      ]),
      { pathDelim: "_" }
    )
  ),
  NodeRuntime.runMain
)

I might be a bit ahead of myself but this way of providing configuration significantly improves the testability of all application code written in Effect. Notice how all the values supplied are strings. The Config module knows exactly how to parse and validate configuration of any complexity. This is what the output looks like:

{
  mysql: {
    username: 'abcde',
    password: <redacted>,
    host: 'dummy host',
    port: 3306
  },
  port: 3000,
  runtimeEnv: 'development'
}

Summary

Let's review the key points we learned about reliability:

Reliability is a measure of correctness.
Correctness is crucial for software users, as it enables them to achieve their objectives.
When software operates correctly (as expected), it reduces customer churn, leading to more customers with higher lifetime value¹⁶.
Escaped defects and configuration errors are the primary causes of service incidents.
Effect's error management and configuration management features help build more reliable software.
The unique combination of TypeScript’s inference and Effect’s composability provides a clear view of edge cases in any part of the program and equips us with the tools to handle them efficiently.
Configuration errors arise due to the inherent disconnect between where configuration values reside (the environment) and where configuration requirements are defined (the program).
Effect’s configuration management is designed to address this disconnect.

Even with Effect’s enhanced reliability guarantees, escaped defects can still occur. These may result from unexpected errors not captured in our type system or from a misalignment between our assumptions about customer expectations and reality. That’s why robust observability practices are essential — they provide visibility into these issues.

In the next post, we’ll explore observability and how it helps monitor both errors and software performance. See you there!

Software Reliability, J.Pan - Carnegie Mellon University ↩
Error Management ↩
Configuration Management ↩
Result type ↩
The Effect Type ↩
Language Server Protocol ↩
Tagged union ↩
Effect.gen ↩
Building pipelines ↩
Effect.catchTags ↩
Effect.succeed ↩
Effect.die ↩
Exponential backoff schedule ↩
Unrecoverable errors ↩
Integrated Development Environment ↩
Customer Lifetime Value ↩

Effective Pragmatism: Introduction

Attila Večerek — Thu, 06 Mar 2025 09:55:31 +0000

Welcome to my series on Effect. Throughout this series, I will explore why I believe Effect is among the most pragmatic technology choices for software companies today. In March 2025, I presented a concise version of these ideas at Effect Days — if you are interested, check out the recorded session.

Before we start talking about the technology itself, we have to establish what all software companies seek. How do their needs change based on their size and the economic environment in which they operate?

In a utopia, all software companies would produce high-quality software that costs them nothing to develop and maintain, and ship it immediately to their customers. This is not possible in the real world. Companies often need to weigh their options and settle for a mix of trade-offs that suits them the most in a given moment. There are many things to be considered. The following is a non-exhaustive list of examples:

Timeline: does the project need to be shipped before a certain event (board meeting, launch event, etc.)?
Resources: are there enough resources with the right qualifications to deliver the project?
Cost: is the current operating budget enough for the delivery and long-term maintenance of the project?
Regulation: privacy policy, cookies, GDPR, HIPAA, SOC2, FedRAMP, WCAG, etc.

Typically, on one end of the spectrum, we have startups prioritizing faster time-to-market over anything else just so they can prove their product fits the market. The long-term viability and software maintenance are of secondary importance to them. Once they succeed, they can hire engineers who can fix any mistake. At least that is what they think. On the other end of the spectrum, we have large enterprises prioritizing software quality above everything else to protect their existing customer base from experiencing service disruptions. Remember, the bigger they are, the harder they fall. This rule also applies to vast and complex systems with large codebases. One small change in one part of such a system can have unintended, unpredictable, and potentially cascading effects in other parts of the same system. These companies trade the pace of innovation in favor of stability. It is very rare to see large companies that can innovate fast. I mean true innovation here, not a yearly announcement of minor feature improvements in their product catalog.

Let's take a quick look at the various development stages of companies and what their business goals are for the given stage:

Startup: The primary goal is to validate product ideas and achieve product-market fit as quickly as possible.
Scale-Up: The focus is on accelerating growth. This requires hiring talented people rapidly, making popular technologies advantageous due to the larger pool of available engineers. The company must also develop the capability to release new features at an incredible pace—any productivity bottleneck could jeopardize its growth targets.
Corporate: After a period of substantial growth, the company shifts its focus to optimizing systems and processes. Cost efficiency and operational effectiveness become major priorities. With established revenue streams, stakeholders now expect increasing profit margins.
Enterprise: Having streamlined operations and improved profit margins, the company is ready to scale even further. Instead of hiring individual engineers, it can acquire entire companies. Success depends on selecting the right acquisitions and integrating them quickly. The right technologies and processes play a crucial role in achieving operational efficiency at scale.
Mega-Corp (or conglomerate): As a well-established enterprise capable of seamlessly integrating acquired companies, the focus shifts to diversification to mitigate risk. Expanding into new industries and markets becomes essential, with an emphasis on establishing new revenue streams.

The global economy also plays a role in these considerations. In the past, when inflation and interest rates were low, and VC funding was abundant¹, cost was rarely a consideration. A company would throw more engineers or hardware at a given problem to solve it faster and better. Engineers would not experience pressure from the business to optimize their workloads for cost efficiency. These days are very different. In the pursuit of sustainable growth, companies are obsessed with achieving the principle called "Rule of 40"², and mass layoffs have become the norm. With these macroeconomic headwinds in play, software companies are now forced to figure out how to deliver high-quality software not only fast but also cheap (i.e. efficiently).

The industry is facing a serious dilemma. Companies want conventional technologies (languages, frameworks, libraries, etc.) to be used to write their software. The idea is that freshly hired engineers can immediately be productive and, in case of turnover, the market can easily provide substitutes. If the supply of engineers who can work with a specific technology is low, employers would have to pay higher wages or invest time and money in upskilling and educating their workforce. The problem with conventional tech such as Ruby on Rails, React, etc. is that they are designed to get one started fast. Building high-quality software with such tools requires years of experience, which often ends up negating the productivity benefits these tools were designed to offer. To offset the shortfalls of these technologies, engineers often utilize advanced design patterns and clever abstractions. This ends up hurting the companies because new hires have to spend time understanding and learning how to work with these technologies in the context of the specific company. Thus, the idea of hiring an engineer who can be immediately productive is implausible. Furthermore, once we commit to a specific design pattern, it has to be adhered to. Otherwise, we would end up with a Big Ball of Mud³. This takes discipline and time, further lowering the overall engineering productivity.

Some engineering managers I talk to about this topic like pointing out that a lack of discipline is a "people problem" and bringing more technology will not fix it. I agree, but that is not the point. We do not apply technology to fix people. We apply technology to make people's lives and work easier. This is what humanity has been doing for millennia. We invent and improve things to solve a specific set of problems. Yes, this often introduces a new set of problems. However, if the invention is good, the new set of problems is more enjoyable to deal with than the old one until a new invention comes along and does the same, ad infinitum.

Other people think that AI could be the answer. Imagine a future where fallible humans could be taken out, almost fully, of the software production loop. Machines are generally better at work that requires discipline, aren’t they? This way of thinking is an open admission of the human incapability to produce high-quality software fast.

Okay, but we are still quite far away from a future like that. Today's LLMs produce code of average quality at best because, guess what, they have been trained on publicly available code produced by humans. What shall we do until we reach the amazing future, though?

One thing that comes to mind is the use of cutting-edge technologies specifically designed to solve the difficult problems one faces at scale; technologies that give us the productivity of Ruby on Rails and the reliability of Rust. Technologies that teach us about advanced programming concepts while using them, and not the other way around where we have to learn the concepts first to use them. Utilizing such technologies could also be considered an investment in our future software-producing AI systems that are very much in need of high-quality training data. Sounds too good to be true? Well, I have some good news. This technology exists and it is called Effect.

Effect is a general-purpose programming framework written in TypeScript. It is designed to provide end-to-end type safety⁴, immutable data structures⁵, formalized dependency injection⁶, structured concurrency⁷, and so much more. It fills in the gap left behind by the missing standard library in the TypeScript ecosystem. Its main goal is to empower engineers to ship high-quality software fast.

That is enough of marketing talk. These are pretty strong claims that need to be backed up. The rest of this series is dedicated to doing just that. We dive deep into software quality: reliability, security, observability, maintainability, testability, performance, and so on. We explore how each of these quality traits supports the business and why it is important for the produced software to possess these traits if one wants to build a lasting product. Last but not least, we discuss how Effect helps us achieve these traits efficiently.

Effect is designed to help us tackle the difficult parts of engineering. Therefore, throughout the entire series, I often demonstrate its capabilities through fairly complex code examples. Simple and short examples that could convey the same message are tough to devise but don't worry! I will do my best to break the complexity down and explain the examples bit by bit on a higher level. Ignore the details. The goal is not to learn how Effect works internally or how to use it. The goal is to demonstrate its benefits as clearly as possible.

The end of 0% interest rates: what it means for tech startups and the industry ↩
What Is The Rule Of 40? ↩
Big Ball of Mud ↩
Fully Typed Web Apps ↩
Immutable Data Structures ↩
Dependency Injection ↩
Structured Concurrency ↩

Reader monads

Attila Večerek — Sun, 04 Jun 2023 12:53:15 +0000

At the end of the previous post, there was a code example that could be improved in several ways. One of them was using dependency injection to improve its testability. In this post, we learn how to do just that using the reader monads.

The reader monad represents a computation that can read values from an environment, or context, and return a value. The Reader interface is declared the following way^[1]:

interface Reader<R, A> {
  (r: R): A
}

R represents the environment and A is the return value. In this post, we use the following terms to refer to R: "R type", "environment", "dependencies".

Similar to how dependency injection is used in object-oriented programming, the Reader monad serves as a mechanism for managing and accessing dependencies in functional programming. It abstracts away the manual handling of dependencies by implicitly threading them through the computation. Let's demonstrate this on a small example.

import { pipe } from "fp-ts/lib/function.js";

interface Logger {
  debug: (msg: string) => void;
}

const logger: Logger = {
  debug: (msg) => console.debug(msg),
};

const addWithLogging = (a: number) => ({ debug }: Logger) => (b: number): number => {
  debug(`Adding ${b} to ${a}`);

  return a + b;
}

const res = pipe(
  39,
  addWithLogging(1)(logger),
  addWithLogging(2)(logger),
);

console.log({ res });

Notice the code duplication inside the pipe and how we manually pass down the logger to both calls of addWithLogging. This can be solved by the adoption of the Reader monad and its functions like so:

import { reader } from "fp-ts";
import { pipe } from "fp-ts/lib/function.js";

interface Logger {
  debug: (msg: string) => void;
}

const logger: Logger = {
  debug: (msg) => console.debug(msg),
};

const addWithLogging = (a: number) => (b: number): reader.Reader<Logger, number> => ({ debug }) => {
  debug(`Adding ${b} to ${a}`);

  return a + b;
}

const res = pipe(
  39,
  addWithLogging(1),
  reader.chain(addWithLogging(2)),
)(logger);

console.log({ res });

Notice how we rearranged the order of arguments in addWithLogging to help us make the function more composable using reader.chain. With this, we can pass the logger to the computation in a single place only.

Types of Reader monads

Depending on the type of computation, there are different "flavors" of the Reader monad:

Reader: represents a synchronous operation that depends on a certain environment.
ReaderTask: represents an asynchronous operation that does not fail and depends on a certain environment. Syntactic sugar over Reader<R, Task<A>>. Basically, it is a Task with a declared dependency.
ReaderTaskEither: represents an asynchronous operation that can fail and depends on a certain environment. Syntactic sugar over Reader<R, TaskEither<E, A>>. Basically, it is a TaskEither with a declared dependency.
etc.

Conversions

It is possible to convert one Reader monad to another. The following table shows how to perform a couple of such conversions:

From	To	Converter
Task	ReaderTask	`readerTask.fromTask`
Reader	ReaderTask	`readerTask.fromReader`
Option	ReaderTaskEither	`readerTaskEither.fromOption`
Either	ReaderTaskEither	`readerTaskEither.fromEither`
Task	ReaderTaskEither	`readerTaskEither.fromTask`
TaskEither	ReaderTaskEither	`readerTaskEither.fromTaskEither`
Reader	ReaderTaskEither	`readerTaskEither.fromReader`

Reader-specific functions

The family of Reader monads implements a couple of functions that no other monad does. In this section, we take a brief look at some of them.

asks

reader.asks lets us tap into the environment and describe (not execute) a computation. It is defined as²:

import { reader } from "fp-ts";

export declare const asks: <R, A>(
  f: (r: R) => A
  ) => reader.Reader<R, A>;

asks basically takes a function of Reader<R, A> and returns a Reader<R, A>. It may seem as if it returned itself. However, in practice, we may use this to tap into the environment, perform a computation, and return the result as a reader. This becomes particularly useful in combination with the Do notation and binding values.

The Do notation is basically just a pipe that starts with reader.Do which merely initializes an empty object for us to be the target of the bind operations that follow. Several monads implement the Do notation, it is not specific to the Reader monad.

import { random as _random, reader } from "fp-ts";
import { pipe } from "fp-ts/lib/function.js";

interface Random {
  next: () => number;
}

interface Env {
  random: Random;
}

const nextRandom: reader.Reader<Env, number> = ({
  random
}) => random.next();

export const chanceButLower: reader.Reader<Env, number> = pipe(
  reader.Do, // returns reader.Reader<unknown, {}>
  reader.bind("a", () => reader.asks(nextRandom)),
  reader.bind("b", () => reader.asks(nextRandom)),
  reader.map(({ a, b }) => a * b)
);

// Example usage
console.log({
  result: chanceButLower({ random: { next: _random.random } }),
});

In the above example, the chanceButLower function depends on an environment containing a random number generator service. It generates two random numbers between 0 and 1, multiplies them, and returns the result.

local

Normally, we can only "read" the dependencies from within a reader. However, sometimes we may run into situations where we need to modify the R value before executing a chained computation. An example situation is when we want to compose two readers that have completely different dependencies. This is where reader.local comes into play. It is defined as³:

import type { reader } from "fp-ts";

export declare const local: <R2, R1>(
  f: (r2: R2) => R1
) => <A>(ma: reader.Reader<R1, A>) => reader.Reader<R2, A>;

It takes two arguments: (1) a function that maps one R value to another, and (2) a reader of the R value returned by the first argument. The function itself returns a reader of the initial R value. The easiest way to think about this function is that it modifies the environment (context) for the local execution of its second argument. The ability of changing the environment for local function executions can help us compose readers of different R types.

import { random as _random, reader } from "fp-ts";
import { flow, pipe } from "fp-ts/lib/function.js";

interface Random {
  next: () => number;
}

const chanceButLower: reader.Reader<Random, number> = ({
  next
}) => next() * next();

type Result = "WIN" | "LOSE";

interface SlotMachine {
  result: (chance: number) => Result;
}

const predict = (
  chance: number
): reader.Reader<SlotMachine, Result> => ({
  result
}) => result(chance);

interface Deps {
  random: Random;
  slotMachine: SlotMachine;
}

const tryMyLuck: reader.Reader<Deps, Result> = pipe(
  chanceButLower,
  reader.local((deps: Deps) => deps.random),
  reader.chain(
    flow(
      predict,
      reader.local((deps: Deps) => deps.slotMachine)
    )
  )
)

// Example usage
console.log({
  result: tryMyLuck({
    random: { next: _random.random },
    slotMachine: { result: (chance) => chance >= 0.99 ? "WIN" : "LOSE" }
  })
});

In the above example, we have two functions that each depend on a different R type. The chanceButLower function depends on a Random service, while the predict function does so on a SlotMachine service. The tryMyLuck function composes these two functions by using reader.local to map the environment for each function accordingly. This way, tryMyLuck depends on a combined environment of the two composed functions.

However, in this particular case, we can completely eliminate the need to use reader.local, and make the composition significantly simpler. To achieve that, we need to adopt a simple rule: all R types representing dependencies must be interface types, even if they contain just a single dependency.

import { random as _random, reader } from "fp-ts";
import { pipe } from "fp-ts/lib/function.js";

interface Random {
  next: () => number;
}

const chanceButLower: reader.Reader<{ random: Random }, number> = ({
  random
}) => random.next() * random.next();

type Result = "WIN" | "LOSE";

interface SlotMachine {
  result: (chance: number) => Result;
}

const predict = (
  chance: number
): reader.Reader<{ slotMachine: SlotMachine }, Result> => ({
  slotMachine
}) => slotMachine.result(chance);

interface Deps {
  random: Random;
  slotMachine: SlotMachine;
}

const tryMyLuck: reader.Reader<Deps, Result> = pipe(
  chanceButLower,
  reader.chainW(predict)
)

// Example usage
console.log({
  result: tryMyLuck({
    random: { next: _random.random },
    slotMachine: { result: (chance) => chance >= 0.99 ? "WIN" : "LOSE" }
  })
});

Notice how much simpler the tryMyLuck function becomes compared to the previous code example. Because of reader's chainW the first function's environment is widened to the final (combined) environment.

As long as we follow the "R type convention" for dependencies, and use the same key to refer to the same dependency across the project, functions returning a Reader become trivial to compose.

Practical code example

Now that we know how to manage dependencies using the Reader monad, we are ready to fix the shortcomings of the example code from the previous post. Let's start by updating the Project domain file.

// domains/project.ts

import type { either, readerTaskEither } from "fp-ts";
import type * as db from "./db.js";
import type * as kafka from "./kafka.js";

export interface Project {
  id: string;
  name: string;
  description: string;
  organizationId: string;
}

export type ProjectInput = Pick<
  Project,
  "name" | "description" | "organizationId"
>;

export class ParseInputError {
  readonly _tag = "ParseInputError";
  constructor(readonly error: Error) {}
}

export class ProjectNameUnavailableError {
  readonly _tag = "ProjectNameUnavailableError";
  constructor(readonly error: Error) {}
}

export type ValidateAvailabilityError =
  | ProjectNameUnavailableError
  | db.QueryError;

export class ProjectLimitReachedError {
  readonly _tag = "ProjectLimitReachedError";
  constructor(readonly error: Error) {}
}

export type EnforceLimitError = ProjectLimitReachedError | db.QueryError;

export class MessageEncodingError {
  readonly _tag = "MessageEncodingError";
  constructor(readonly error: Error) {}
}

export type EmitEntityError = MessageEncodingError | kafka.ProducerError;

/**
 * A synchronous operation that accepts an unknown object
 * and parses it. The result is an `Either`.
 */
export type ParseInput = (
  input: unknown
) => either.Either<ParseInputError, ProjectInput>;
export declare const parseInput: ParseInput;

export interface DbEnv {
  db: db.Service
}

/**
 * A function that accepts an object representing
 * the input data of a project and returns a ReaderTaskEither
 * describing an asynchronous operation which queries
 * the database to check whether the project name
 * is still available. Project names across an organization
 * must be unique. This operation may fail due to different
 * reasons, such as network errors, database connection
 * errors, SQL syntax errors, etc. The database client
 * is the only dependency of this function.
 */
export type ValidateAvailability = (
  input: ProjectInput
) => readerTaskEither.ReaderTaskEither<DbEnv, ValidateAvailabilityError, void>;
export declare const validateAvailability: ValidateAvailability;

/**
 * A task describing an asynchronous operation that
 * queries the database for the number of existing
 * projects for a given organization. There is a
 * product limit for how many projects can be created
 * by an organization. This operation fails if the limit
 * is reached or any other network or database error occurs.
 * The database client is the only dependency of this function.
 */
export type EnforceLimit = readerTaskEither.ReaderTaskEither<DbEnv, EnforceLimitError, void>;
export declare const enforceLimit: EnforceLimit;

/**
 * A function that accepts a project object and returns
 * a task describing an asynchronous operation that
 * persists this object in the database. This operation
 * fails if any network or database error occurs.
 * The database client is the only dependency of this function.
 */
export type Create = (
  project: Project
) => readerTaskEither.ReaderTaskEither<DbEnv, db.QueryError, void>;
export declare const create: Create;

/**
 * A function that accepts a project object and returns
 * a task describing an asynchronous operation that encodes
 * this object and produces a Kafka message. This operation
 * fails if the encoding fails, or any other network or broker
 * error occurs. The database client is the only dependency
 * of this function.
 */
export type EmitEntity = (
  project: Project
) => readerTaskEither.ReaderTaskEither<DbEnv, EmitEntityError, void>;
export declare const emitEntity: EmitEntity;

As we can see, all we did was we replaced taskEither.TaskEither with readerTaskEither.ReaderTaskEither, declared a type for the database dependency, and referenced it from all the necessary places. Now, we're ready to update the Project repository file containing the composition of the domain functions.

// repositories/project.ts

import { readerTaskEither } from "fp-ts";
import { flow } from "fp-ts/lib/function.js";
import { ulid } from "ulid";
import * as project from "../domains/project.js";

export const create = flow(
  project.parseInput,
  readerTaskEither.fromEither,
  readerTaskEither.chainFirstW(project.validateAvailability),
  readerTaskEither.chainFirstW(() => project.enforceLimit),
  readerTaskEither.bind("id", () => readerTaskEither.right(ulid())),
  readerTaskEither.chainFirstW(project.create),
  readerTaskEither.chainFirstW(project.emitEntity)
);

As we can see, the update here was just as simple as the one above if not even simpler.

Let's take a look at another shortcoming, the lack of transaction control. To address that, we need to implement a withTransaction function that takes a computation and makes sure that it starts a transaction before the computation is executed and that it commits and rolls back the transaction according to the result of the computation. After the implementation, this is how the file implementing our database service looks like:

import { either, readerTaskEither, taskEither } from "fp-ts";
import { constVoid, pipe } from "fp-ts/lib/function.js";

export class QueryError {
  readonly _tag = "QueryError";
  constructor(readonly error: Error) {}
}

export interface Service {
  query: (sql: string) => (values: unknown[]) => taskEither.TaskEither<QueryError, unknown>;
}

export class StartTransactionFailedError {
  readonly _tag = "StartTransactionFailedError";
  constructor(readonly error: Error) {}
}

export class RollbackFailedError {
  readonly _tag = "RollbackFailedError";
  constructor(readonly error: Error) {}
}

export class CommitFailedError {
  readonly _tag = "RollbackFailedError";
  constructor(readonly error: Error) {}
}

export type WithTransactionError = CommitFailedError | QueryError | RollbackFailedError | StartTransactionFailedError;

export interface WithTransaction {
  <R, E, A>(
    use: readerTaskEither.ReaderTaskEither<R, E, A>
  ): readerTaskEither.ReaderTaskEither<R & { db: Service }, E | WithTransactionError, A>
}
export const withTransaction: WithTransaction = (use) => (env) => taskEither.bracketW(
  pipe(
    [],
    env.db.query("START TRANSACTION"),
    taskEither.mapLeft(({ error }) => new StartTransactionFailedError(error))
  ),
  () => use(env),
  (_, res) =>
    pipe(
      res,
      either.match(
        () =>
          pipe(
            [],
            env.db.query("ROLLBACK"),
            taskEither.mapLeft(({ error }) => new RollbackFailedError(error)),
            taskEither.map(constVoid)
          ),
        () =>
          pipe(
            [],
            env.db.query("COMMIT"),
            taskEither.mapLeft(({ error }) => new CommitFailedError(error)),
            taskEither.map(constVoid)
          )
      )
    )
);

The withTransaction function takes a single argument (use) representing an async computation that can fail and depends on a certain environment. The return value of this function is another async computation that can also fail and depends on a combined environment of the use function and its own environment ({ db: Service }). It is implemented using taskEither.bracket. We can think of bracket as a resource manager that works in three steps:

It acquires a resource. We simply execute a START TRANSACTION query.
It performs a computation, optionally by using the acquired resource. In this particular case, we throw the value of the resource away.
It releases the resource. In this step, bracket provides the acquired resource and the result of the computation from the previous step to the caller. We inspect the result:
- in the case of a failure, we execute the ROLLBACK query,
- otherwise, we execute the COMMIT query.

To wrap our composed project.create function in a transaction, all we need to do is to call withTransaction as the last step of the pipeline like so:

// repositories/project.ts

import { readerTaskEither } from "fp-ts";
import { flow } from "fp-ts/lib/function.js";
import { ulid } from "ulid";
import * as project from "../domains/project.js";
import * as db from "../db.js";

export const create = flow(
  project.parseInput,
  readerTaskEither.fromEither,
  readerTaskEither.chainFirstW(project.validateAvailability),
  readerTaskEither.chainFirstW(() => project.enforceLimit),
  readerTaskEither.bind("id", () => readerTaskEither.right(ulid())),
  readerTaskEither.chainFirstW(project.create),
  readerTaskEither.chainFirstW(project.emitEntity),
  db.withTransaction
);

Just like in the previous post, the type of the create function gets widened again. This time by the additional db.WithTransactionError error type:

import type { readerTaskEither } from "fp-ts";
import type * as project from "../domains/project.js";
import type * as db from "../db.js";

export interface Create {
  (input: unknown): readerTaskEither.ReaderTaskEither<
    project.DbEnv,
    | db.WithTransactionError
    | db.QueryError
    | project.ParseInputError
    | project.ProjectNameUnavailableError
    | project.ProjectLimitReachedError
    | project.EmitEntityError,
    project.Project
  >;
}

Wrap-up

The Reader monad serves as a mechanism for managing and accessing dependencies.
reader.asks allows us to access the dependencies, perform a computation, and return its result as another reader of the same dependencies.
reader.local allows us to compose readers of different dependencies.
Reader monads compose much easier when we adopt the convention of representing dependencies through an interface type (even if the reader only declares a single dependency) and we give our dependencies a consistent key across the whole project.
Depending on the result type, there can be different "flavors" of the Reader monad, for example ReaderEither for readers returning an Either.
It is easy to convert between instances of different "flavors" of reader monads.
It is easier to reason about programs using Reader monads because it is immediately clear what dependencies must be satisfied by the caller.
It is easier to test code that uses Reader monads the same way it is easier to test object-oriented code that uses dependency injection.

The next post of this series is centered around runtime type-systems and how to use them to build programs with end-to-end type safety.

Extra resources

Task monads

Attila Večerek — Sun, 18 Dec 2022 15:41:37 +0000

Up until now, all code examples in the series were centered around synchronous computation. In reality, we need to be able to combine both synchronous and asynchronous computations. Some examples of async functions are: database lookups, HTTP calls, reading data from the filesystem, etc. In this post, we take a look at how such functions can be composed.

The pitfalls of working with Promises

Promise is JavaScript's native language construct representing the result of an async operation^[1]. However, it may be quite inconvenient to work with for multiple reasons.

First of all, once we instantiate a promise, it starts evaluating immediately. Canceling one is non-trivial, so it is best not to instantiate a promise unless absolutely necessary. This requires us to have a tight control over the order of execution of async functions. This may be difficult to achieve in large and complex codebases. However, this control is crucial when it comes to fixing pesky bugs or optimizing for performance or resource utilization.

Secondly, as seen from its type signature (Promise<A>), a promise only communicates the type of its resolved value. However, it can also reject with any value. This requires us to handle unknown edge cases of an any type which is basically giving up on type safety. However, since TypeScript 4.4 we can use the useUnknownInCatchVariables compiler option to get type-safety back at the cost of reduced ergonomics:

class SomeCustomError extends Error {}

try {
  // call something that may throw
} catch (err: unknown) {
  if (err instanceof SomeCustomError) {
    // handle some custom error
  } else if (err instanceof Error) {
    // handle a generic error
  } else {
    throw new Error(String(err));
  }
}

Lastly, the type system does not have a mechanism to enforce the handling of possibly rejected promises. Basically, whether we wrap some code in a try-catch block or not it still produces a valid program. This contributes to bugs caused by human error. People may simply forget to handle errors.

Tasks to the rescue

Although more complex, Task monads help us deal with the shortcomings of Promise. fp-ts provides the following four task monads:

Task<A>: represents an async computation that never fails.
TaskEither<E,A>: represents an async computation that may fail.
TaskOption<A>: represents an async computation that never fails and may or may not return a value.
TaskThese<E,A>: represents an async computation that may fail, succeed, or do both at the same time.

The main advantages of tasks as opposed to using only promises can be summarized as:

We retain control over when tasks are executed. Hence, it becomes easier to fix bugs and optimize our code.
We can clearly see from the type signature what effects a task has. This not only saves us time learning about possible edge cases to handle but the compiler actually forces us to handle them. This helps us write more correct business logic. This is fairly similar to the advantages of using the Either monad for synchronous computations that may fail.

This post only describes the Task and TaskEither monads in more detail. Understanding these two is enough to understand the other ones.

Task

A task describes an asynchronous computation that never fails. It is defined as follows ^[2]:

export interface Task<A> {
  (): Promise<A>;
}

It is a thunk - basically a lazily executed Promise. Here's an example of an async function that cannot fail:

import * as fs from "node:fs/promises";

export const safeReadFile =
  (fallback: string) =>
  async (path: string): Promise<string> => {
    try {
      return fs.readFile(path, "utf8");
    } catch (_err) {
      return fallback;
    }
  };

safeReadFile wraps the native promise-based readFile function and handles any error by returning a configurable string as the fallback value. Hence, calling safeReadFile never fails.

Constructors

The simplest way to construct a task is by using the of function:

import { task } from "fp-ts";

export const myTask = task.of(42);
// Value of myTask is () => Promise.resolve(42)
// Type of myTask is Task<number>

However, the most common way to construct one is to just return an async thunk like so:

import { task } from "fp-ts";
import * as fs from "node:fs/promises";

export const safeReadFile =
  (fallback: string) =>
  (path: string): task.Task<string> =>
  async () => {
    try {
      return fs.readFile(path, "utf8");
    } catch (_err) {
      return fallback;
    }
  };

To get the value wrapped by the task, it needs to be first eliminated by calling the task. After that, its result needs to be awaited in an async context like so:

import { task } from "fp-ts";
import assert from "node:assert";

const myTask = task.of(42);

(async () => {
  const value = await myTask();
  assert.equal(value, 42);
  // OK, 42 == 42
})();

TaskEither

A TaskEither describes an async computation that may fail. It is defined as follows ^[3]:

import type { either, task } from "fp-ts";

export interface TaskEither<E, A> extends task.Task<either.Either<E, A>> {}

We can think of the TaskEither monad as a syntactic sugar over the Task monad that wraps an Either. Task<Either<E, A>> would be fairly inconvenient to work with. The map, chain, fold, etc. functions of the Task monad would only unwrap the Task instance. Unwrapping the instance of an Either would be the caller's responsibility. This can be demonstrated by the following code example.

import { either, task } from "fp-ts";
import { pipe } from "fp-ts/lib/function.js";
import { db } from "./db.js";

const query =
  (sql: string): task.Task<either.Either<Error, unknown>> =>
  async () => {
    try {
      const res = await db.query(sql);
      return either.right(res);
    } catch (error) {
      return pipe(error, either.toError, either.left);
    }
  };

export const testDbConnection: task.Task<boolean> = pipe(
  query("SELECT 1"),
  task.map(
    // unwrapping the task here
    either.fold(
      // unwrapping the either here
      () => false,
      () => true
    )
  )
);

Assume we implemented a db module that knows how to authenticate against the database and fetch a connection. The above code implements query as a functional wrapper over querying the database that throws an error in case of connection issues, syntax errors in the SQL statement, etc. It also implements testDbConnection which returns true if the test query succeeds and false otherwise. It is common to see implementations like this for diagnostic endpoints that are regularly called by an internal service for monitoring purposes. Using the TaskEither monad simplifies our example from above:

import { either, task, taskEither } from "fp-ts";
import { pipe } from "fp-ts/lib/function.js";
import { db } from "./db.js";

const query = (sql: string): taskEither.TaskEither<Error, unknown> =>
  taskEither.tryCatch(() => db.query(sql), either.toError);

export const testDbConnection: task.Task<boolean> = pipe(
  query("SELECT 1"),
  taskEither.fold(
    () => task.of(false),
    () => task.of(true)
  )
);

The query function becomes less verbose. taskEither.tryCatch handles the wrapping of the success/failure values in their respective instances of Either for us. It also reduces the level of nesting in testDbConection by one. Notice, the functions passed to fold must return a Task. That is because once we operate on an asynchronous computation, we cannot turn it into a synchronous one.

Constructors

The simplest way to construct a TaskEither is by using the right and left functions:

import { taskEither } from "fp-ts";

export const asyncSuccess = taskEither.right(42);
// The value is () => Promise.resolve({ _tag: "Right", value: 42 })
// The type is TaskEither<never, number>

export const asyncFailure = taskEither.left(new Error("Something went wrong"));
// The value is () => Promise.reject({ _tag: "Left", value: new Error("Something went wrong" }))
// The type is TaskEither<Error, never>

Another way to construct a TaskEither is to return an async thunk that returns an Either:

import { either } from "fp-ts";
import * as fs from "node:fs/promises";

export const safeReadFile = (path: string) => async () => {
  try {
    return either.right(fs.readFile(path, "utf8"));
  } catch (reason) {
    return either.left(either.toError(reason));
  }
};

Last but not least, we can use taskEither.tryCatch to wrap an async function that may throw an error:

import { either, taskEither } from "fp-ts";
import * as fs from "node:fs/promises";

export const safeReadFile = (path: string) =>
  taskEither.tryCatch(
    () => fs.readFile(path, "utf8"),
    either.toError
  );

Just like in the case of the Option monad, tryCatch is most useful when wrapping third-party library or native functions written in a non-functional style.

Conversions

It is possible to convert one Task monad to another. The following table shows how to perform a couple of such conversions:

From	To	Converter
Either	TaskEither	`taskEither.fromEither`
Task	TaskEither	`taskEither.fromTask`
TaskOption	TaskEither	`taskEither.fromTaskOption`
TaskEither	Task	`taskEither.fold`
Either	TaskOption	`taskOption.fromEither`
Task	TaskOption	`taskOption.fromTask`
TaskEither	TaskOption	`taskOption.fromTaskEither`
TaskOption	Task	`taskOption.fold`

Practical code example

Assume we worked on a software for tracking issues and the codebase follows the onion architecture. The core layer represents the different domains and implements functions that store and retrieve data in a database, emit Kafka messages, etc. The layer above that represent the repositories that compose the different functions of the domain layer to implement the business logic. The last layer is an API layer that exposes all the functionality to the clients of the service. It could consist of REST endpoints, GraphQL resolvers, workflow orchestrators, and console roles implementing REPL-like debugging interfaces in production, etc.

In this example, we only describe the two innermost layers of the architecture. Assume a project domain that implements the following data structures and functions:

// domains/project.ts

import type { either, taskEither } from "fp-ts";
import type * as db from "./db.js";
import type * as kafka from "./kafka.js";

export interface Project {
  id: string;
  name: string;
  description: string;
  organizationId: string;
}

export type ProjectInput = Pick<
  Project,
  "name" | "description" | "organizationId"
>;

export interface ParseInputError {
  _tag: "ParseInputError";
  value: Error;
}

export interface ProjectNameUnavailableError {
  _tag: "ProjectNameUnavailableError";
  value: Error;
}

export type ValidateAvailabilityError =
  | ProjectNameUnavailableError
  | db.QueryError;

export interface ProjectLimitReachedError {
  _tag: "ProjectLimitReachedError";
  value: Error;
}

export type EnforceLimitError = ProjectLimitReachedError | db.QueryError;

export interface MessageEncodingError {
  _tag: "MessageEncodingError";
  value: Error;
}

export type EmitEntityError = MessageEncodingError | kafka.ProducerError;

/**
 * A synchronous operation that accepts an unknown object
 * and parses it. The result is an `Either`.
 */
export type ParseInput = (
  input: unknown
) => either.Either<ParseInputError, ProjectInput>;
export declare const parseInput: ParseInput;

/**
 * A function that accepts an object representing
 * the input data of a project and returns a task
 * describing an asynchronous operation which queries
 * the database to check whether the project name
 * is still available. Project names across an organization
 * must be unique. This operation may fail due to different
 * reasons, such as network errors, database connection
 * errors, SQL syntax errors, etc.
 */
export type ValidateAvailability = (
  input: ProjectInput
) => taskEither.TaskEither<ValidateAvailabilityError, void>;
export declare const validateAvailability: ValidateAvailability;

/**
 * A task describing an asynchronous operation that
 * queries the database for the number of existing
 * projects for a given organization. There is a
 * product limit for how many projects can be created
 * by an organization. This operation fails if the limit
 * is reached or any other network or database error occurs.
 */
export type EnforceLimit = taskEither.TaskEither<EnforceLimitError, void>;
export declare const enforceLimit: EnforceLimit;

/**
 * A function that accepts a project object and returns
 * a task describing an asynchronous operation that
 * persists this object in the database. This operation
 * fails if any network or database error occurs.
 */
export type Create = (
  project: Project
) => taskEither.TaskEither<db.QueryError, void>;
export declare const create: Create;

/**
 * A function that accepts a project object and returns
 * a task describing an asynchronous operation that encodes
 * this object and produces a Kafka message. This operation
 * fails if the encoding fails, or any other network or broker
 * error occurs.
 */
export type EmitEntity = (
  project: Project
) => taskEither.TaskEither<EmitEntityError, void>;
export declare const emitEntity: EmitEntity;

With the project domain implemented, we can demonstrate how the corresponding repository implements a create action by composing the domain functions together:

// repositories/project.ts

import { taskEither } from "fp-ts";
import { flow } from "fp-ts/lib/function.js";
import { ulid } from "ulid";
import * as project from "../domains/project.js";

export const create = flow(
  project.parseInput,
  taskEither.fromEither,
  taskEither.chainFirstW(project.validateAvailability),
  taskEither.chainFirstW(() => project.enforceLimit),
  taskEither.bind("id", () => taskEither.right(ulid())),
  taskEither.chainFirstW(project.create),
  taskEither.chainFirstW(project.emitEntity)
);

First, the input data is parsed. Next, we perform a small set of fail-fast validations. We start by checking if the project name is available. Since this validation is a task that may fail, we need to first lift the Either produced by parseInput to a TaskEither. We use chainFirstW for two main reasons:

To call validateAvailability in the pipeline so that we can ignore its return value and keep on passing the ProjectInput object down the pipeline (flow). Note that chainFirstW unwraps the taskEither and passes its value down to the supplied function (validateAvailability).
To widen the failure type of TaskEither, thus extending the number of distinct errors that can cause the whole pipeline to fail.

The next validation is enforceLimit which is composed using chainFirstW for the same reasons. However, here we pass an anonymous function () => project.enforceLimit to chainFirstW. That is because enforceLimit is a task as opposed to a function returning a task as it was the case with validateAvailability.

Once all the validation is done, we generate a unique ID for the project and bind it to the ProjectInput object under the key id. This creates an object with the shape of the Project interface and allows us to continue in the create pipeline of the project repository.

Next, we chainFirstW the project.create function to persist the project object in the database. At last, we chainFirstW the project.emitEntity function that produces a Kafka message to a compacted topic letting other microservices perform asynchronous data aggregation and replication as necessary.

If any of the steps in the pipeline fail, for whatever reason, the failure is propagated through the rest of the pipeline without executing any of the remaining tasks. The type of the create action is widened to the following final type that documents all possible failure cases that an engineer would now be aware of and can be handled at the call site:

import type { taskEither } from "fp-ts";
import type * as project from "../domains/project.js";
import type * as db from "../db.js";

export interface Create {
  (input: unknown): taskEither.TaskEither<
    | project.ProjectLimitReachedError
    | db.QueryError
    | project.ParseInputError
    | project.ProjectNameUnavailableError
    | project.EmitEntityError,
    project.Project
  >;
}

Concurrency

We could optimize the code and perform the projects.create and project.emitEntity actions concurrently. To do that, we'll use taskEither.sequenceArray which is an equivalent of Promise.all:

// repositories/project.ts

import { taskEither } from "fp-ts";
import { flow } from "fp-ts/lib/function.js";
import { ulid } from "ulid";
import * as project from "./13-domains-project.js";
import type * as db from "./db.js";

const createAndEmitProject = (proj: project.Project) =>
  taskEither.sequenceArray<void, db.QueryError | project.EmitEntityError>([
    project.create(proj),
    project.emitEntity(proj),
  ]);

export const create = flow(
  project.parseInput,
  taskEither.fromEither,
  taskEither.chainFirstW(project.validateAvailability),
  taskEither.chainFirstW(() => project.enforceLimit),
  taskEither.bind("id", () => taskEither.right(ulid())),
  taskEither.chainFirstW(createAndEmitProject)
);

We can create a helper function called createAndEmitProject. Since projects.create and projects.emitEntity have different failure types, taskEither.sequenceArray can't infer the resulting type properly. Thus, we have to override the success and failure types manually. With that done, we can simply replace the two calls to chainFirstW with a single one in the create pipeline.

The shortcomings of this example

Although this example looks fairly complete, it ignores several aspects of real life software engineering:

The database operations may need to be performed in a single transaction to prevent data from entering an inconsistent state.
These functions may rely on one or more dependencies such as a database client, a logger, a tracer, etc.

I hope to address these shortcomings in the next parts of this series once we learn about the Reader monads for managing dependencies.

Wrap-up

Task describes an async computation that never fails.
TaskEither describes an async computation that may fail.
TaskOption describes an async computation that never fails and returns an Option.
TaskThese describes an async computation that may fail, succeed, or do both.
All tasks are thunks that return a Promise.
Tasks are more complex to work with but superior to promises in every other way:
- We retain better control over the execution of the underlying Promise of a task.
- Failures and edge cases are visible from the type signature.
- The type system forces us to handle the failure cases.
Once we have a task, it stays a task until eliminated (by calling it and awaiting its result).
It is easy to convert between instances of different types of task monads.
To create a task that performs other tasks concurrently, we can use the sequenceArray monadic function.

The next post of this series delves into the Reader monads and how they can help us with dependency injection.

Extra resources

The Option monad

Attila Večerek — Tue, 06 Dec 2022 10:05:22 +0000

Just like the Either monad in the previous post, the Option monad also specifies an Option data type and the functions operating on top of it. In this post, we'll also learn that once we know how to work with one monad, we pretty much know how to work with all of them.

The Option data type represents a value that may not be present. That's also why in some functional languages this monad is called the Maybe monad. Practically speaking, it allows us to get rid of the use of A | null and A | undefined kind of types. Consequently, it also removes the need for constant null checks in our code. This is how fp-ts defines Option ^[1]:

interface None {
  readonly _tag: "None";
}

interface Some<A> {
  readonly _tag: "Some";
  readonly value: A;
}

export type Option<A> = None | Some<A>;

Option is a tagged union consisting of the None and Some types. None represents the missing value, whereas Some wraps the value of type A. In a sense, it is a specialized form of Either<undefined, A>. This type is usually returned from lookup functions:

accessing an index of an array,
accessing a property of an object,
looking up a specific record in a database.

Constructors

The Option monad has two basic constructors, one for each tag:

import { option } from "fp-ts";

export const missing = option.none;
export const present = option.some(42);

It can also construct a new Option from a nullable type (A | null | undefined). Very useful in the inevitable case of working with some popular libraries that do return such values.

import { option } from "fp-ts";

export const missing = option.fromNullable(null); // None
export const alsoMissing = option.fromNullable(undefined); // None
export const present = option.fromNullable(42); // Some(42)

Mapping

As opposed to Either, the Option monad implements a single map function. The None type does not wrap any value, hence there's nothing to map from. However, there is an exception to this when lifting Option to Either. The concept of lifting is discussed in more details in a later section of this post.

import { option, record } from "fp-ts";
import { flow } from "fp-ts/lib/function.js";

interface LookupCount {
  (_: Record<string, number>): option.Option<{ count: number }>;
}

const lookupCount: LookupCount = flow(
  record.lookup("total"),
  option.map((count) => ({ count }))
);

export const res = lookupCount({ total: 42 });
// Value of res is { _tag: "Some", value: { count: 42 } }

export const res2 = lookupCount({});
// Value of res2 is { _tag: "None" }
// Type of both res and res2 is Option<{ count: number }>

In the above example, we define a function called lookupCount that takes a Record, checks if it contains a property called total and maps it to an object of type { count: number } returning a Some. If the property does not exist, it returns a None.

Our previous examples did not contain manual typings and relied on type inference instead. Unfortunately, in this example, explicitly typing the lookupCount function is necessary. It is likely that TypeScript cannot infer the type of the looked up property correctly ^[2] when record.lookup is used in a flow. Nevertheless, I've reported this issue in fp-ts.

Without using the Option monad, the example code may look like this:

interface LookupCount {
  (_: Record<string, number>): { count: number } | null;
}

const lookupCount: LookupCount = (r) => {
  if (r["total"]) {
    return { count: r["total"] };
  } else {
    return null;
  }
};

export const res = lookupCount({ total: 42 });
// Value of res is { count: 42 };

export const res2 = lookupCount({});
// Value of res2 is null
// Type of both res and res2 is { count: 42 } | null

Even though the implementation looks fairly neat, the disadvantage of this approach is incurred at the call site. Working with the result down the line requires us to check whether the value is null every single time until a fallback value is defined or an error is thrown. Using the Option monad, we can simply keep on mapping and chaining functions completely removing the need for null checks.

Chaining

option.chain works exactly the same way as either.chain does.

import { option, record } from "fp-ts";
import { pipe } from "fp-ts/lib/function.js";

const makeCount = (x: string, y: string) => (r: Record<string, number>) =>
  pipe(
    record.lookup(x, r),
    option.chain((a) =>
      pipe(
        record.lookup(y, r),
        option.map((b) => ({ count: a + b }))
      )
    )
  );

const countDogsAndCats = makeCount("dogs", "cats");

export const res = countDogsAndCats({ dogs: 21, cats: 21 });
// Value of res is { _tag: "Right", value: { count: 42 } }

export const res2 = countDogsAndCats({});
// Value of res2 is { _tag: "None" }
// Type of both res and res2 is Option<{ count: number }>

In the above example, the code performs two record lookups. Only if both lookups result in a hit, the values are added together and wrapped in an object of type { count: number }. This implementation isn't perfect, though. Imagine we'd need to perform five lookups and additions in total. This approach would yield deeply nested code. We'll demonstrate how to improve this in the bind section.

Not using the Option monad, we arrive at a simpler implementation of makeCount. However, the approach suffers from the same disadvantages as the imperative code example under mapping.

const makeCount = (x: string, y: string) => (r: Record<string, number>) => {
  if (r[x] && r[y]) {
    // @ts-expect-error TS still thinks r[x] and r[y] may possibly be undefined
    return r[x] + r[y];
  }

  return null;
};

const countDogsAndCats = makeCount("dogs", "cats");

export const res = countDogsAndCats({ dogs: 21, cats: 21 });
// Value of res is { count: 42 }

export const res2 = countDogsAndCats({});
// Value of res2 is null
// Type of both res and res2 is { count: number } | null

Other functions

Similarly to Either, the Option monad also implements other functions. In fact, it implements the same functions described in the previous post: getOrElse, getOrElseW, fold, chainFirst, chainFirstW, and some more (except for getValidation, which is Either-specific). These functions work the exact same way in every monad that implements them. Hence, there is no point in going through them again.

In the rest of this section, we'll demonstrate a new set of functions that are also implemented by Either and more or less all the monads explored in the rest of this series.

bind

bind combines chain with an assignment to a readonly property of a typed object. It returns a corresponding instance of Option.

import { option, record } from "fp-ts";
import { pipe } from "fp-ts/lib/function.js";

const makeLookupCount = (x: string) => (r: Record<string, number>) =>
  pipe(
    option.some({}),
    option.bind("count", () => record.lookup(x, r))
  );

const lookupCount = makeLookupCount("total");

export const res = lookupCount({ total: 42 });
// Value of res is { _tag: "Some", value: { count: 42 } }

export const res2 = lookupCount({});
// Value of res2 is { _tag: "None" }
// Type of both res and res2 is Option<{ readonly count: number }>

bindTo

bindTo creates a new typed object and assigns the Some value under a readonly key of our choice. It also returns a corresponding instance of Option.

import { option, record } from "fp-ts";
import { pipe } from "fp-ts/lib/function.js";

export const makeLookupCount = (x: string) => (r: Record<string, number>) =>
  pipe(record.lookup(x, r), option.bindTo("count"));
// makeLookupCount returns Option<{ readonly count: number }>
// after the full application of its parameters

With bind and bindTo, we can improve makeCount's implementation from our earlier example for chaining:

import { option, record } from "fp-ts";
import { pipe } from "fp-ts/lib/function.js";

export const makeCount =
  (x: string, y: string) => (r: Record<string, number>) =>
    pipe(
      record.lookup(x, r), // Option<number>
      option.bindTo("a"), // Option<{ a: number }>
      option.bind("b", () => record.lookup(y, r)), // Option<{ a: number; b: number }>
      option.map(({ a, b }) => ({ count: a + b })) // Option<{ count: number }>
    );

With this approach, we avoid deeply nested structures making our code more readable and maintainable.

foldW

foldW is similar to fold from the previous post. The difference is that the None and Some handler functions no longer have to return the same type. This results in a widened return type:

import { option } from "fp-ts";
import { pipe } from "fp-ts/lib/function.js";

export const res = pipe(
  option.some(42),
  option.foldW(
    () => ({ _tag: "Fallback" as const, value: 0 }),
    (value) => ({ _tag: "Derived" as const, value })
  )
);
// Value of res is { _tag: "Derived", value: 42 }
// Type of res is { _tag: "Derived", value: number } | { _tag: "Fallback", value: number }

tryCatch

tryCatch creates a functional wrapper around functions performing a synchronous computation that can fail by throwing an exception. We can use this to our advantage when working with popular libraries or native functions written in a non-functional style.

import { option } from "fp-ts";
import * as fs from "fs";

export const res = option.tryCatch(() => fs.readFileSync("/path/to/your/file"));
// Type of res is Option<Buffer>

In the above example, if readFileSync throws an error - e.g. the file does not exist - the value of res will be an instance of option.None. This is a good solution if the missing file can be handled down the line. Should we want to keep the original error around, e.g. for reporting reasons, we may want to use either.tryCatch instead.

Lifting

Now that we know two monads, we can delve into the concept of lifting. Up until now, all our examples were composing functions within the same monad. Oftentimes, we have to compose functions across different monads. In fp-ts, we can achieve this by "lifting one monad to another" by applying a function that takes a value of one type of a monad and returns a value of another:

Assume we had a value of type Either<Error, Record<string, number>>, and we wanted to lookup the value of a key in case it was a Right. One option would be to extract that value and pipe it into record.lookup.

import { either, record } from "fp-ts";
import { pipe } from "fp-ts/lib/function.js";

export const res = pipe(
  either.right({ total: 42 } as Record<string, number>),
  either.getOrElse(() => ({} as Record<string, number>)),
  record.lookup("total")
);
// Value of res is { _tag: "Some", value: 42 }
// Type of res is Option<number>

However, in that case, we would have to handle the Left type by specifying some kind of a fallback logic. We might not want to do that just yet. Therefor, the other option is to lift the Either to an Option and chain record.lookup:

import { either, option, record } from "fp-ts";
import { pipe } from "fp-ts/lib/function.js";

export const res = pipe(
  either.right({ total: 42 } as Record<string, number>),
  option.fromEither,
  option.chain(record.lookup("total"))
);
// Value of res is { _tag: "Some", value: 42 }
// Type of res is Option<number>

This way, we don't need to handle the Left value ourselves since it is mapped to a corresponding None value by option.fromEither.

Even though the name of the concept may suggest so, lifting is not necessarily unidirectional. We can lift an Option to an Either almost as easily:

import { either, option } from "fp-ts";
import { pipe } from "fp-ts/lib/function.js";

export const res = pipe(
  option.some(42), // Option<number>
  either.fromOption(() => "Missing value") // Either<string, number>
);
// Value of res is { _tag: "Right", value: 42 }

The only difference is that we have to pass in a function to specify how to lift the None value to a Left<E>. Remember, the None value does not hold any actual value, there's none. This is not the case with Left, which is a generic type.

Wrap-up

Option represents optionality or nullability.
- None stands for a missing value.
- Some stands for a present value.
We use:
- bindTo: to create a typed object and assign a given Option's value to a specific key of that object. This returns an Option wrapping that typed object.
- bind: to chain a computation and assign the resulting Option's value to a specific key of a typed object. It often follows a call to bindTo and solves the issue of deeply nested chaining.
- foldW: to extract a single value from an Option while applying transformations to both the none and some values while widening the Some type.
- tryCatch: to create a functional wrapper around synchronous computations that may fail by throwing an exception. It is usually used over functions coming from third-party libraries or native functions written in a non-functional style.
Lifting helps us compose functions across different types of monads by taking one monad, such as Either, and transforming it into another, for example an Option.

In the next post, we’ll take a look at how to write asynchronous code that composes well using the Task monad.

Extra resources

The Either monad

Attila Večerek — Mon, 28 Nov 2022 01:23:07 +0000

The Either monad specifies the Either data type as well as several functions that operate on top of it. The Either data type represents the result of a computation that may fail. This is how fp-ts defines it ^[1]:

interface Left<E> {
  readonly _tag: "Left";
  readonly left: E;
}

interface Right<A> {
  readonly _tag: "Right";
  readonly right: A;
}

export type Either<E, A> = Left<E> | Right<A>;

As we can see, it is a simple tagged union with two distinct tags: Left and Right. Left represents the failure while right represents the success of a computation. A computation may fail for different reasons. In such cases, we can use another tagged union to represent the left type, e.g.:

// parse-int.ts
import type { either } from "fp-ts";

interface NaNError {
  readonly _tag: "NaNError";
  readonly value: number;
}

interface OutOfRangeError {
  readonly _tag: "OutOfRangeError";
  readonly value: number;
}

export type ParseIntError = NaNError | OutOfRangeError;

export type ParseIntResult = either.Either<ParseIntError, number>;

export type ParseInt = (x: string) => ParseIntResult;
export declare const parseInt: ParseInt;

In the above example, we just created a new type signature for the (not so?) well known parseInt function. The function signature of the original parseInt function is the following:

(string: string, radix?: number | undefined) => number

It returns NaN in case it cannot parse the string as an integer. It returns Infinity in case the parsed number is outside of the safe range, i.e. too big or too small ^[2]. The function signature does not indicate these edge cases because both NaN and Infinity are still typed as number. They can only be distinguished from valid numbers at runtime using the isNaN and isFinite functions, respectively.

Let's be honest, how many times do we actually remember to handle such edge cases? parseInt is not the only example of such functions. There are many that may fail but their type signature does not indicate that: JSON.parse, new Date(), new Array(), BigInt, etc.

In my opinion, encoding all possible outcomes of a computation in the type signature is generally better. Life is too short to read possibly nonexistent documentation. Also, there are more important things to keep in mind when our IDEs could help us make sure that these edge cases are handled. For example, by enabling the noImplicitReturns compiler option:

import { ParseIntError } from "./parse-int.js";

// @ts-expect-error TS2366: Function lacks ending return statement and return type does not include 'undefined'.
export const handleParseIntError = (err: ParseIntError): number => {
  if (err._tag === "NaNError") {
    return 0;
  }
};

Or, by implementing and using an assertUnreachable function.

Now that we know why Either is useful, let's see how to actually use it.

Constructors

In the previous post, we learnt that every monad needs to be constructible, mappable, and chainable. The Either monad has two constructors, one for each tag:

import { either } from "fp-ts";

export const success = either.right(42);
export const failure = either.left({ _tag: "NaNError", value: NaN });

Mapping

The Either monad offers two map functions: map and mapLeft. Each maps the encapsulated value of the corresponding tag. We may map the right/left values to a new value of the same type, or a completely different one, e.g.: turning Either<Error, number> into Either<Error, { count: number }> or Either<string, number>.

For the following code examples, assume we have implemented the parseInt function. We can then use map/mapLeft to evolve the success and failure paths, respectively:

import { either } from "fp-ts";
import { flow } from "fp-ts/lib/function.js";
import { parseInt } from "./parse-int.js";

const parseCount = flow(
  parseInt,
  either.map((count) => ({ count })),
  either.mapLeft(({ value }) => `"${value}" could not be parsed as an integer`)
);

export const res = parseCount("42");
// Value of res is { _tag: "Right", right: 42 }

export const res2 = parseCount("abc");
// Value of res2 is { _tag: "Left", left: '"42" could not be parsed as an integer' }
// Type of both res and res2 is Either<string, { count: number }>

Without using the Either monad (assume parseInt returns ParseIntError | number instead), the above code may look like this:

import { ParseIntError } from "./parse-int.js";

declare const parseInt: (_: string) => ParseIntError | number;

const parseCount = (x: string) => {
  const parsedInt = parseInt(x);

  if (typeof parsedInt === "number") {
    return { count: parsedInt };
  } else {
    return `"${parsedInt.value} could not be parsed as an integer`;
  }
};

export const res = parseCount("42");
// Value of res is 42

export const res2 = parseCount("abc");
// Value of res2 is '"42" could not be parsed as an integer'
// Type of both res and res2 is number | string

This code follows more of an imperative programming paradigm. It requires us to:

store the intermediate result in a variable,
figure out how to name that variable,
fork the logic based on the return type.

With the Either monad code, we only need to declare what the different results should map to.

Chaining

chain is kind of similar to map. The difference is that the function passed into chain must return another Either. We can think of it as performing a mapping that may fail.

import { either } from "fp-ts";
import { pipe } from "fp-ts/lib/function.js";
import { parseInt } from "./parse-int.js";

const add = (x: string, y: string) =>
  pipe(
    parseInt(x),
    either.chain(
      (a) =>
        pipe(
          parseInt(y),
          either.map((b) => a + b)
        ) // `Either<ParseIntError, number>` is returned by this pipe
    )
  );

export const res = add("42", "24");
// Value of res is { _tag: "Right", right: 66 }

export const res2 = add("abc", "24");
// Value of res2 is { _tag: "Left", left: { _tag: "NaNError", value: NaN } }
// Type of both res and res2 is Either<ParseIntError, number>

In the above code, we first parse "42" as an integer. Then, we parse "24" as an integer but only if the first call to parseInt succeeds. Finally, if the second call to parseInt succeeds, we add the results together. Should any of the parsing steps fail, the corresponding left result is returned.

Without using the Either monad, the above code may look like this:

import { ParseIntError } from "./parse-int.js";

declare const parseInt: (_: string) => ParseIntError | number;

const add = (x: string, y: string) => {
  const parsedIntX = parseInt(x);
  const parsedIntY = parseInt(y);

  if (typeof parsedIntX === "number") {
    if (typeof parsedIntY === "number") {
      return parsedIntX + parsedIntY;
    } else {
      return parsedIntY;
    }
  } else {
    return parsedIntX;
  }
};

export const res = add("42", "24");
// Value of res is 66

export const res2 = add("abc", "24");
// Value of res2 is { _tag: "NaNError", value: NaN }
// Type of both res and res2 is number | ParseIntError

This code shares the same properties with the imperative code example in the Mapping section. On top of that, it is less readable and maintainable because of the nested if statements. Imagine we added three more computations (not necessarily the same parseInt steps) each of which could fail. The level of nesting would be three levels deeper making it more likely to introduce bugs. chain allows us to flatten otherwise nested if statements like so:

import { either } from "fp-ts";
import { pipe } from "fp-ts/lib/function.js";
import { parseInt } from "./parse-int.js";

// either.right represents an arbitrary computation that may fail in this example

pipe(
  parseInt("42"),
  either.chain((x) => either.right(x)),
  either.chain((x) => either.right(x)),
  either.chain((x) => either.right(x)),
  either.chain((x) => either.right(x))
);

Other functions

Besides constructors, mappers, and chainers, monads may implement any additional interfaces. This section describes some of the other functions implemented by the Either monad.

getOrElse

The map and mapLeft functions, as demonstrated in the Mapping section, produce an instance of an Either. What if we wanted to extract its value? That's when we make use of getOrElse, which forces us to define a fallback value in case of a failure.

import { either } from "fp-ts";
import { flow } from "fp-ts/lib/function.js";
import { parseInt } from "./parse-int.js";

const myParseInt = flow(
  parseInt,
  either.getOrElse(() => 0)
);

export const res = myParseInt("42");
// Value of res is 42, not a Right

export const res2 = myParseInt("abc");
// Value of res2 is 0, not a Left
// Type of both res and res2 is number

We can use getOrElseW if we want to widen - that's what the W at the end of the function name stands for - the right type of the Either:

import { either } from "fp-ts";
import { flow } from "fp-ts/lib/function.js";
import { parseInt } from "./parse-int.js";

const myParseInt = flow(
  parseInt,
  either.getOrElseW(() => "Zero" as const)
);

export const res = myParseInt("42");
// Value of res is 42

export const res2 = myParseInt("abc");
// Value of res2 is "Zero"
// Type of both res and res2 is number | "Zero"

In the unlikely scenario that the failure is unrecoverable and we want our program to crash, we could also use getOrElseW to throw an error:

import { either } from "fp-ts";
import { flow } from "fp-ts/lib/function.js";
import { parseInt } from "./parse-int.js";

const throwError = (e: Error) => {
  throw e;
};

const myParseInt = flow(
  parseInt,
  either.getOrElseW(flow(either.toError, throwError))
);

export const res = myParseInt("42");
// Value of res is 42
// Type of res is number

export const res2 = myParseInt("abc");
// res2 never gets assigned a value because an error is thrown

either.toError takes any value and does a best effort conversion to an Error object. throwError takes an Error object and throws it. Functions that don't return anything and just throw, have a return type of never. Hence, getOrElseW in this example widens the type of number by never. However, number | never is the same type as number.

fold

fold is a function that allows us to transform both tags of an Either at the same time. It expects two functions as the first set of arguments:

A function that takes a value of the left type of the Either and returns a value of type A (it can be anything).
A function that takes a value of the right type of the Either and also returns a value of type A.

With an Either value provided as the second argument, fold executes one of the two provided functions based on the tag and returns a value of type A. It can be used to extract the encapsulated value of the Either as well as perform mappings at the same time:

import { either } from "fp-ts";
import { flow, pipe } from "fp-ts/lib/function.js";
import { parseInt } from "./parse-int.js";

const throwError = (e: Error) => {
  throw e;
};

export const res = pipe(
  parseInt("42"),
  either.fold(flow(either.toError, throwError), (count) => ({ count }))
);

// Type of res is { count: number }

// Functionally the same as writing the below code
// but using fewer steps (1 instead of 3):

export const res2 = pipe(
  parseInt("42"),
  either.mapLeft(either.toError),
  either.map((count) => ({ count })),
  either.getOrElseW(throwError)
);

It is still possible to throw an error using our throwError function from the previous example because any value is assignable to the type of never.

chainFirst

chainFirst is similar to chain. It takes a function that has to return an Either. If that function returns a Right value, its result is thrown away. In such case, the result of the previous computation is returned, i.e. the argument passed into the provided function. In the opposite case, the Left value is returned. I tend to use it for logging intermediate values or setting breakpoints.

import { either } from "fp-ts";
import { pipe } from "fp-ts/lib/function.js";
import { parseInt } from "./parse-int.js";

export const res = pipe(
  parseInt("42"),
  either.chainFirst((count) => {
    console.log({ count });

    return either.right(undefined);
  }),
  either.getOrElse(() => 0)
);

// Type of res is number

There's also a chainFirstW alternative, which may be useful for fail-fast type of validations that keep widening the Left type with new failure types:

import { either } from "fp-ts";
import { flow } from "fp-ts/lib/function.js";
import { parseInt } from "./parse-int.js";

interface IncorrectAnswerError {
  readonly _tag: "IncorrectAnswerError";
  readonly value: number;
}

const myParseInt = flow(
  parseInt,
  either.chainFirstW((count) =>
    count === 42
      ? either.right(undefined)
      : either.left({
          _tag: "IncorrectAnswerError",
          value: count,
        } as IncorrectAnswerError)
  )
);

export const res = myParseInt("42");
// Value of res is { _tag: "Right", right: 42 }

export const res2 = myParseInt("abc");
// Value of res2 is { _tag: "Left", left: { _tag: "NaNError", value: NaN } }

export const res3 = myParseInt("41");
// Value of res3 is { _tag: "Left", left: { _tag: "IncorrectAnswerError", value: 41 } }
// Type of res, res2, res3 is the same: Either<ParseIntError | IncorrectAnswerError, number>

Assume we were to validate a form with multiple input fields and we wanted to present all the errors at once. We'd have to somehow collect multiple possible failures instead of returning on the first encountered one. In such cases, we should use either.getValidation. There's already a great article written about it by the author of fp-ts himself. I highly recommend checking it out.

Wrap-up

Either represents the result of a computation that may fail.
- Left stands for failure.
- Right stands for success.
We use:
- map: to transform the right value into a different value and this transformation cannot fail.
- mapLeft: to transform the left value into a different value and this transformation cannot fail.
- getOrElse: to extract the right value while setting a default value in case of a Left.
- getOrElseW: to extract the right value while widening its type, or throwing an error on an unrecoverable Left.
- fold: to extract a single value from Either while applying transformations to both the left and right values.
- chain: to transform the right value into a different value and this transformation can fail.
- chainFirst: to log intermediate values or set breakpoints for debugging.
- chainFirstW: to perform one or more fail-fast type of validations.
- getValidation: to perform validations that can be collected.

If I missed a function from the Either monad that you often use or find particularly interesting, or I didn't cover a use case for the functions described in this post, please let me know in the comments 🙏

In the next post of this series, we will look into the Option monad.

Extra resources

Functor and Monad

Attila Večerek — Wed, 02 Nov 2022 01:52:26 +0000

Functors and monads are concepts that are often misunderstood and over-explained at the same time. Many tutorials and blog posts focus on why these concepts are useful instead of what they actually are. This is understandable since it is actually not required to know what these concepts are in order to take advantage of them.

In this post, we first look into what a functor and a monad are. After that, we switch over to what we can achieve by using them.

Functor

The word functor is the name of a specific interface which describes the following property: when we have something of type a, and we have a way to take a and map it to b, we can have something of type b ^[1]. Something needs to be a source of a value of some type. The word "source" is chosen deliberately to represent an abstract thing because it can be pretty much anything. Let's take a look at a couple of examples of a source of type string:

// the values are accessed by iteration
export type ArraySource = Array<string>;

// the value is accessed through property `a`
export interface RecordSource {
  a: string;
}

// the value is accessed through a function call
export interface FunctionSource {
  (): string;
}

All of the above are sources of type string because there is a way to access the value. Array, Record, and Function are functors because they specify a way to map their enclosed values to a potentially different type. We can also think of functors as type classes specifying some kind of a map function. That function does not necessarily have to be called map but needs to behave like one. For example, the type Array is a functor because it defines a map function.

// source of values `a` (Array<number>)
const numbers = [1, 2, 3];

// mapping of `a` (number) to `b` (string)
const toString = (a: number): string => String(a);

// source of values `b` (Array<string>)
export const result = numbers.map(toString);

Array#map has the ability to take an a (of any type) from an instance of Array by iterating over its values, apply a mapping function on each a, and produce a new instance of Array holding values of b of type that is dictated by the return type of our mapping function.

How would a Function#map look like?

interface Fn<A> {
  (): A;
}

interface FnMap {
  <A, B>(fn: (a: A) => B): (a: Fn<A>) => Fn<B>;
}

const functionMap: FnMap = (fn) => (a) => {
  return () => fn(a());
};

// source of values `a` (Fn<number>)
const makeNumber = () => 42;

// mapping of `a` (number) to `b` (string)
const toString = (a: number): string => String(a);

// source of values `b` (Function<string>)
export const result = functionMap(toString)(makeNumber);

functionMap has the ability to take an a (of any type) from an instance of type Function by calling it, apply a mapping function on its return value, and produce a new instance of Function holding a value of b of type that is dictated by the return type of our mapping function.

In both examples, we use the exact same mapping function (toString) without having to change anything about it. The same function can then be applied to both functors: Array and Function. In fact, we can supply that mapping function to any functor's map function exactly because the term functor describes a property, and not a noun. Reusing code without additional modifications is extremely powerful.

Monad

Just like functor, the word monad is also the name of a specific interface. A monad is something that fulfils the following:

it is a functor, i.e. implements a map function
it can be chained, i.e. implements a flatMap function
it can be constructed, i.e. implements an of function
it abides by the Monad laws

Chainability

The definition of chainability is very similar to functor's mappability. The difference is highlighted in bold letters: when we have something of type a, and a way to map a onto the same type of something of type b, we can have something of type b ^[1]. Same as with functor, something needs to be a source of a value of some type. We can also think of monads as type classes specifying some kind of a flatMap function. That function does not necessarily have to be called flatMap but needs to behave like one. For example, the type Array, besides being a functor, is also a monad because it defines a flatMap function.

// source of values `a` (Array<string>)
const expressions = ["Functors and monads", "are easy"];

// mapping of `a` (string) to the same type of source (Array) of `b` (Array<string>)
const splitByWords = (a: string): string[] => a.split(" ");

// source of values `b` (Array<string>)
export const words = expressions.flatMap(splitByWords);
// ["Functors", "and", "monads", "are", "easy"]

So, how does our example with flatMap fit the definition? Let's break it down:

"when we have something of type a": expressions - the type of the source is Array<string>.
"and a way to map a onto the same type of something of type b": splitByWords maps string onto Array<string> (matches the type from above).
"we can have something of type b": expressions.flatMap(splitByWords) produces Array<string> because flatMap knows two things:
1. how to provide a to the mapping function,
2. how to flatten the intermediate results produced by the mapping function.

Evaluating expressions.map(splitByWords) would produce a value of type Array<Array<string>>. Hence, things that implement a function like map but don't implement a function like flatMap cannot be called monads.

Constructability

A monad also needs to implement a function that may take a value and returns an instance of the monad encapsulating that value. Many times these functions will be called of, as in Task.of(42), but can also be called anything else, e.g.: left, right, some, none, etc.

Monad laws

The explanation of monad laws is beyond the scope of this series. However, if you're interested, you can read Monad laws for regular developers written by Miklós Martin. It provides a nice explanation using both Scala and JavaScript examples.

Why are functors and monads useful?

In my opinion, they are useful because of their following three innate characteristics:

They are polymorphic.
They abstract away some boilerplate code such as iteration and control flow logic.
They are composable.

Polymorphism

Polymorphism just means that an object can take up multiple forms. For example, the type Array can be of strings, numbers or any other type.

Functions like map and flatMap don't care what the type of the value is. That is the responsibility of the mapping function. With map and flatMap, we can easily go from an Array<number> to an Array<string>.

const numbers = [1, 2, 3];
export const strings = numbers.map(String);
// ["1", "2", "3"]

This is a great property to have because we don't have to implement map and flatMap functions for every possible type of Array out there.

Useful abstractions

Functors and monads abstract away some common boilerplate code such as iteration and control flow logic. By using functors and monads, we repeat less code. Also, we tend to write more declarative code instead of imperative.

Imperative code is more prescriptive, i.e. we say how a certain thing should be implemented.

Declarative code is more descriptive, i.e. we say what a certain thing should do. People who write code are humans, and humans make mistakes. So, the less we have to describe how things should be implemented, the more of the following may apply to our code:

has fewer bugs,
runs faster,
is more secure.

Obviously, this applies only if the functor/monad implementation is well tested, optimized for speed and implemented with security in mind.

Iteration and Control flow

The previously shown code examples demonstrate how map and flatMap both hide the iteration logic. To demonstrate the control flow logic being abstracted away, we'll need to assume something about how Array's map function is implemented.

export const map =
  <A, B>(fn: (a: A) => B) =>
  (x: A[]): B[] => {
    if (x.length === 0) return [];

    // pretend to be the rest of map's implementation
    return x.map(fn);
  };

This is a simple and not very useful optimization but it serves its purpose well.

[]
  .map((x) => x + 1)
  .map((x) => x * 2)
  .map((x) => x % 2);

// []

Every time map is called, an empty array is returned early skipping possibly expensive and unnecessary computations. The real value of abstracting the control flow logic will be more apparent when we'll look into the Either and Option monads in the upcoming chapters.

Function composition

Because of how functor and monad is defined, it lends itself to function composition very well. Remember how Array#map and Array#flatMap work? They both return an instance of Array. This means, we can keep calling map and flatMap in succession:

const splitByWords = (a: string): string[] => a.split(" ");
const len = (a: string): number => a.length;
const double = (a: number): number => a * 2;

const expressions = ["Functors and monads", "are easy"];
export const result = expressions.flatMap(splitByWords).map(len).map(double);
// [ 16, 6, 12, 6, 8 ]

Function composition in this example is achieved by method chaining. If we want, we can create our own map with a slightly different interface and use pipe from the previous chapter:

import { pipe } from "fp-ts/lib/function.js";

const map =
  <A, B>(f: (_: A) => B) =>
  (a: A[]): B[] =>
    a.map(f);
const len = (a: string): number => a.length;
const double = (a: number): number => a * 2;

export const result = pipe(
  ["Functors", "and", "monads", "are", "easy"],
  map(len),
  map(double)
);
// [ 16, 6, 12, 6, 8 ]

The advantage of redefining map's interface is that we can "chain" functions other than map and flatMap as well. Luckily for us, fp-ts already did that for both map and flatMap, so we could have written our example just as:

import { array } from "fp-ts";
import { pipe } from "fp-ts/lib/function.js";

const splitByWords = (a: string): string[] => a.split(" ");
const len = (a: string): number => a.length;
const double = (a: number): number => a * 2;

export const result = pipe(
  ["Functors and monads", "are easy"],
  array.chain(splitByWords), // flatMap is called chain in fp-ts
  array.map(len),
  array.map(double)
);
// [ 16, 6, 12, 6, 8 ]

Wrap-up

Functors and monads can be thought of as interfaces.
Functor is an interface specifying a function similar to Array#map.
Monad is a functor that also:
- specifies a function similar to Array#flatMap,
- has a constructor: Array(),
- abides by the Monad laws.
Functors and monads are useful because they are:
- polymorphic,
- reduce code repetition,
- compose well.

The next part of this series delves into a very specific type of monad implemented in fp-ts: the Either monad.

Extra resources

Function composition

Attila Večerek — Wed, 02 Nov 2022 01:28:18 +0000

Function composition is just a fancy term describing the act of passing the return value of one function as an argument to another. We say that functions compose well if the result of one function can be directly passed as an argument to another function.

// Good example
const toNumber = (a: string): number => Number(a);
const double = (a: number): number => a * 2;
const toArray = <A>(a: A): A[] => [a];

export const result = toArray(double(toNumber("21")));

// Bad example
export const badMultiply = (input: number, by: number): number =>
  input * by;

export const result2 = toArray(badMultiply(toNumber("21"), 2));

In the first example, the functions compose well because the result of each function matches the expected input of the next function. In the second example, it is no longer enough to just pass down the result of toNumber because badMultiply expects two arguments.

In functional programming, we strive to write functions that compose well. In TypeScript, it is not possible to return multiple values. Hence, the rule of thumb is to write functions of arity 1, i.e. functions that take a single argument. However, it is possible to "rewrite" functions so they do compose well using a technique called currying.

Currying is a transformation of functions that translates a function from callable as f(a, b, c) into callable as f(a)(b)(c)

export const multiply = (by: number) => (input: number) =>
  badMultiply(input, by);

The above function takes a single argument and returns another function that also takes a single argument. For the sake of simplicity, going forward we refer to this as a function with two arguments. The order of arguments matters. The section called pipe provides more details on that.

Another technique worth mentioning is partial application. If we have a function of arity n and partially apply the first m arguments, the result is a function of arity n - m. This is extremely helpful for creating specialized functions that hold a certain state. For example:

// http.ts
type Method = "GET" | "POST" | "PUT" | "DELETE";
type Headers = Record<string, string>;
type Body =
  | Blob
  | Buffer
  | URLSearchParams
  | FormData
  | string
  | null
  | undefined;
type Request = {
  method: Method;
  url: URL;
  headers: Headers;
  body: Body;
};

// a function of arity 2
export const request =
  (method: Method) =>
  (url: URL): Request => ({
    method,
    url: url,
    headers: {},
    body: undefined,
  });

// functions of arity 1
export const get = request("GET");
export const post = request("POST");
export const put = request("PUT");
export const del = request("DELETE");

In the above example, request is a generic request building function of arity 2. The two arguments it takes are method and url. We can partially apply every distinct value of method to the request function resulting in four specialized request functions, each of arity 1. Now, whenever we want to build a get request we have a shortcut to do so:

import { get, request } from "./http.js";

// instead of
export const myGetRequest = request("GET")(new URL("http://google.com"));

// we can just do
export const myGetRequest2 = get(new URL("http://google.com"));

`pipe`

Code such as toArray(double(toNumber("21"))) reads opposite to the order of execution but can still be considered as fairly readable. Now, imagine our program consisted of 10 or more functions. Code written in such style could fairly quickly become unreadable. fp-ts provides some tools to make function composition more readable. The example could be re-written using pipe the following way:

import { pipe } from "fp-ts/lib/function.js";

const toNumber = (a: string): number => Number(a);
const double = (a: number): number => a * 2;
const toArray = <A>(a: A): A[] => [a];

export const result = pipe(
  "21",
  toNumber,
  double,
  toArray
);

The above code reads in the same order it is executed. It demonstrates pipe with functions that all expect a single argument. What about functions that take more arguments?

import { pipe } from "fp-ts/lib/function.js";

const toNumber = (a: string): number => Number(a);
const multiplyBy = (by: number) => (input: number) => input * by;
const toArray = <A>(a: A): A[] => [a];

export const result = pipe(
  "21",
  toNumber,
  multiplyBy(2),
  toArray
);

The previous section mentions that the order of arguments in function composition matters. We can see why in the above example. If input was the first argument, the above code would have to be written as follows:

import { pipe } from "fp-ts/lib/function.js";

const toNumber = (a: string): number => Number(a);
const multiplyBy = (input: number) => (by: number) => input * by;
const toArray = <A>(a: A): A[] => [a];

export const result = pipe(
  "21",
  toNumber,
  multiplyBy,
  (by) => by(2),
  toArray
);

The above code is not only longer but it also reads bad. This is how much impact the order of arguments can have on writing functional code.

`apply`

If the order of arguments prevents the function being simply piped into another and we don't want to wrap it, we could use apply. The previous code example could be rewritten as:

import { apply, pipe } from "fp-ts/lib/function.js";

const toNumber = (a: string): number => Number(a);
const multiplyBy = (input: number) => (by: number) => input * by;
const toArray = <A>(a: A): A[] => [a];

export const result = pipe(
  "21",
  toNumber,
  multiplyBy,
  apply(2),
  toArray
);

`flow`

What if we wanted to reuse the code in pipe and call the function calculate?

import { pipe } from "fp-ts/lib/function.js";

const toNumber = (a: string): number => Number(a);
const multiplyBy = (by: number) => (input: number) => input * by;
const toArray = <A>(a: A): A[] => [a];

const calculate = (a: string) => pipe(
  a,
  toNumber,
  multiplyBy(2),
  toArray
);

export const result = calculate("21");

The code does not read terribly bad but fp-ts provides another tool to solve the problem of code-reuse more elegantly. We can think of flow as a reusable pipe:

import { flow } from "fp-ts/lib/function.js";

const toNumber = (a: string): number => Number(a);
const double = (a: number): number => a * 2;
const toArray = <A>(a: A): A[] => [a];

const calculate = flow(toNumber, double, toArray);

export const result = calculate("21");

pipe's first argument is always a value and is followed by functions and returns a value. flow expects only function arguments and returns a function. The type annotation of calculate from the above example could be written as:

export type Calculate = (_: string) => number[];

Wrap-up

Function composition is all about passing the result of one function to another as its input.
In TypeScript, two functions compose well when they accept a single argument and their respective "input" and "output" types match.
Functions that don't compose well can be transformed into functions that do by currying.
Partial application allows us to create functions that store state and thus become specialized versions of a generic function that we partially applied the arguments on.
pipe allows us to read composed functions in the order of execution.
apply allows us to partially apply an argument to a function or work around suboptimal ordering of function arguments.
flow allows us to re-use composed functions more easily.

Next up, we look into the commonly misunderstood concepts of functor and monad.

Extra resources

Introduction to FP

Attila Večerek — Wed, 02 Nov 2022 01:27:45 +0000

Welcome to the series of blog posts about functional programming in TypeScript. In this series, we explore the basic concepts of function composition and monads. The code examples in this series are built using the fp-ts library and TypeScript's native features.

I have chosen fp-ts since it is the tool I am the most familiar with but there are others out there, for example effect-ts.

Target audience

Function composition and monads are concepts that can sound scary to people coming from a non-functional programming background. It certainly did sound scary to me at first. In this series, I want to explain these concepts using very simple terms and examples. I am not trying to be exhaustive, nor completely accurate in my explanations. If you want to learn about the mathematical background of functional programming and category theory, this isn't the series you're looking for.

Structure

Each chapter consists of the following:

explanations centered around a single concept,
code examples (ESM),
additional learning materials to offset the incompleteness and low accuracy of my explanations,
a short wrap-up and a brief introduction to the next chapter.

The chapters build on top of each other. Hence, it is strongly recommended to read them in the given order.

Wrap-up

The aim of the series is to explain a few basic functional programming concepts in simple terms.
The series is targeted at software engineers not comfortable with functional programming concepts who have at least a basic understanding of TypeScript or any other typed language.

In the next chapter, we learn about function composition. See you there!👋