DEV Community: Eric D Johnson

Is this code deterministic?

Eric D Johnson — Fri, 16 Jan 2026 18:20:30 +0000

I recently posted a small code snippet in a LinkedIn poll and asked what sounded like a simple question:

Is this code deterministic?

Those are usually the dangerous questions.

I asked on purpose. I’ve been spending time talking with folks much smarter than me, reading docs, and honestly leaning on code assistants to sanity-check my thinking as I go. Durable execution has a way of surfacing edge cases you don’t normally think about, and I wanted to learn in public—right alongside everyone else.

The discussion that followed (in the original post) was excellent. It also showed how easy it is to mix together concepts like determinism, replay, retries, and idempotency. This post is my attempt to slow things down and separate those ideas, using the original example and AWS’s guidance on deterministic code in AWS Lambda durable functions.

Here’s the code that started it all:

import { withDurableExecution, DurableContext } from '@aws/durable-execution-sdk-js';

export const handler = withDurableExecution(
  async (event: any, context: DurableContext) => {
    const orders = event.orders.sort((a, b) => a.priority - b.priority);

    const results = [];
    for (const order of orders) {
      const result = await context.step(`process-${order.id}`, async () => {
        return processOrder(order);
      });
      results.push(result);
    }

    return { processed: results.length, timestamp: Date.now() };
  }
);

Most people voted “No, non-deterministic.”

That’s the correct answer—but not always for the reasons people first reach for.

Let’s walk through it.

Problem 1: Equal-priority ordering is under-specified

What’s happening

event.orders.sort((a, b) => a.priority - b.priority);

Modern JavaScript engines (ES2019+) guarantee that Array.prototype.sort() is stable. If two orders have the same priority, their relative order is preserved.

So no, JavaScript isn’t secretly reordering your data.

Why it still matters (and why this one is subtle)

I’ll be honest: this one felt nit-picky to me at first. If the input is the same and the sort is stable, it feels like everything should be fine.

The important realization is this: a stable sort preserves whatever order the input already had—but it doesn’t explain why that order exists.

In this code, the implicit rule becomes:

“If priorities are equal, keep whatever order the input arrived in.”

If that order is intentional and guaranteed, great. Nothing wrong here.

But if it’s incidental—maybe merged upstream, aggregated from multiple sources, or simply not meant to be meaningful—then the workflow’s step ordering now depends on an accident of the input.

Nothing is broken. But you may have just encoded behavior you didn’t mean to encode.

One more thing worth saying out loud: if the order in which steps are created doesn’t matter, you may not need to sort at all. Sorting only makes sense if you’re enforcing a real business rule like “highest priority goes first.”

Why not just wrap the ordering in a step?

This is a very common reaction—and a reasonable one.

Yes, you could wrap the sort in a step and checkpoint it. That would make the ordering fully durable and replay-stable.

But steps are not free.

They add latency. They cost money. They count toward operation limits. And they exist primarily to protect work that is slow, expensive, or has side effects.

Pure, fast, in-memory logic like sorting is already replayable. Re-running a sort of 10 items during replay is usually far cheaper than checkpointing it. Even with larger lists, the trade-off depends on size, cost, and intent.

The rule of thumb I like is this:

If the logic is pure, fast, and deterministic, don’t rush to wrap it.

If you can’t make it deterministic, or replaying it is expensive, that’s when a step makes sense.

Fix

If ordering matters, make it explicit and deterministic, without mutating the input:

const orders = [...event.orders].sort(
  (a, b) => a.priority - b.priority || a.id.localeCompare(b.id)
);

Problem 2: `Date.now()` is non-deterministic

What’s happening

timestamp: Date.now()

This value is computed at runtime, so every execution produces a different number.

Why it matters

In this handler, the timestamp is just part of the returned response. It doesn’t affect control flow or step scheduling, so it’s harmless today.

But time-based APIs are explicitly called out in the durable execution docs as a common source of non-determinism. If this value later gets stored as workflow state, passed into a step, or used in a conditional, replay behavior can change in ways that are very hard to reason about.

This is less “this is wrong” and more “this is easy to trip over later.”

Fix

If the timestamp actually matters, capture it once inside a step so it replays consistently:

const timestamp = await context.step("timestamp", async () => Date.now());

Problem 3: Side effects hidden inside a single step

Ben Kehoe correctly points out a subtle but important issue in this code.

What’s happening

await context.step(`process-${order.id}`, async () => {
  return processOrder(order);
});

A durable step can fail and be retried. Once a step completes, it won’t be re-run on replay—but retries can re-execute the step body.

If processOrder performs multiple side effects, a failure partway through can cause those side effects to run again.

Why it matters

This is not a determinism problem.

It’s not a replay problem either.

This is a retry safety problem.

If a step body can’t safely run more than once, retries can produce duplicate effects unless everything inside the step is idempotent.

Fix

Be intentional about retry boundaries and align steps with retry-safe work.

Problematic version:

await context.step(`process-${order.id}`, async () => {
  await chargeCard(order);
  await writeAuditRecord(order);
  await sendConfirmation(order);
});

If this fails after chargeCard, a retry may re-run everything.

Safer version:

await context.step(`charge-${order.id}`, async () => {
  return chargeCard(order, { idempotencyKey: order.id });
});

await context.step(`audit-${order.id}`, async () => {
  return writeAuditRecord(order);
});

await context.step(`notify-${order.id}`, async () => {
  return sendConfirmation(order, { idempotencyKey: order.id });
});

This doesn’t magically make things idempotent. It just limits the blast radius when retries happen.

Step semantics and retry intent

AWS Lambda durable functions also give you control over how steps retry.

By default, steps use AtLeastOncePerRetry semantics. If a step fails or the Lambda is interrupted, the runtime may re-execute the step body. In this mode, the retry count acts as a lower bound on executions.

If you have a step that must never run more than once, you can use StepSemantics.AtMostOncePerRetry with zero retries. In that case, a failure surfaces as an error instead of re-running the step.

Put simply:

AtLeastOncePerRetry → max attempts is a lower bound
AtMostOncePerRetry → max attempts is an upper bound

Neither is “better.” They just encode different assumptions.

Putting it all together

Once you make ordering explicit, keep non-deterministic values under control, and think carefully about retry boundaries, the handler becomes much easier to reason about.

Here are two durable-safe ways to structure it, depending on how independent your work items are and how much concurrency you want.

Durable-safe handler (step by step)

import { withDurableExecution, DurableContext } from '@aws/durable-execution-sdk-js';

export const handler = withDurableExecution(
  async (event: any, context: DurableContext) => {
    const orders = [...event.orders].sort(
      (a, b) => a.priority - b.priority || a.id.localeCompare(b.id)
    );

    for (const order of orders) {
      await context.step(`validate-${order.id}`, async () => {
        if (!order.id) throw new Error("Missing order id");
      });

      await context.step(`charge-${order.id}`, async () => {
        return chargeCard(order, { idempotencyKey: order.id });
      });

      await context.step(`notify-${order.id}`, async () => {
        return sendConfirmation(order, { idempotencyKey: order.id });
      });
    }

    return { processed: orders.length };
  }
);

Durable-safe handler using `context.map()`

context.map() changes the shape of the problem a bit. Each item becomes its own durable unit of work.

That matters because:

Failures are isolated to a single item
Completed items don’t get re-run because something else failed
Concurrency becomes a first-class knob (maxConcurrency)

The trade-offs are real too:

Large lists can emit a lot of steps quickly
Strict sequencing is harder to express

import { withDurableExecution, DurableContext } from '@aws/durable-execution-sdk-js';

export const handler = withDurableExecution(
  async (event: any, context: DurableContext) => {
    const orders = [...event.orders].sort(
      (a, b) => a.priority - b.priority || a.id.localeCompare(b.id)
    );

    const mapResult = await context.map(
      "process-orders",
      orders,
      async (ctx: DurableContext, order: any) => {
        await ctx.step(`validate-${order.id}`, async () => {
          if (!order.id) throw new Error("Missing order id");
        });

        await ctx.step(`charge-${order.id}`, async () => {
          return chargeCard(order, { idempotencyKey: order.id });
        });

        await ctx.step(`notify-${order.id}`, async () => {
          return sendConfirmation(order, { idempotencyKey: order.id });
        });

        return { orderId: order.id, status: "ok" };
      },
      { maxConcurrency: 5 }
    );

    const results = mapResult.getResults();
    return { processed: results.length, results };
  }
);

Final takeaway

Durable execution encourages you to slow down just a bit and be explicit about ordering, retries, idempotency, and where work can safely be repeated.

That’s exactly why this question was worth asking—and why the conversation around it was worth having.

AWS Lambda Durable Functions: Build Workflows That Last

Eric D Johnson — Wed, 03 Dec 2025 21:33:02 +0000

Long-running workflows without managing infrastructure

Your Lambda function needs to wait for a human approval. Or retry a failed API call with exponential backoff. Or orchestrate multiple steps that span hours. How do you build that without managing servers or databases?

AWS Lambda Durable Functions solve this. Write your workflow in your programming language—Node.js, TypeScript, Python, with more coming—using straightforward async code. Lambda handles the rest: checkpointing state, resuming after waits, retrying issues, and scaling automatically. Workflows can run for up to a year, and you only pay for actual execution time—not while waiting.

What Are Durable Functions?

Durable functions are Lambda functions that can pause and resume. When your function waits for a callback or sleeps for an hour, Lambda checkpoints its state and stops execution. When it's time to continue, Lambda resumes exactly where it left off—with all variables and context intact.

This isn't a new compute model. It's regular Lambda with automatic state management. You write normal async/await code. Lambda makes it durable.

A Simple Example

Here's a workflow that creates an order, waits 5 minutes, then sends a notification:

import { DurableContext, withDurableExecution } from '@aws/durable-execution-sdk-js';

export const handler = withDurableExecution(
  async (event: any, context: DurableContext) => {
    const order = await context.step('create-order', async () => {
      return createOrder(event.items);
    });

    await context.wait({ seconds: 300 });

    await context.step('send-notification', async () => {
      return sendEmail(order.customerId, order.id);
    });

    return { orderId: order.id, status: 'completed' };
  }
);

That's it. No state machines to configure, no databases to manage, no polling loops. The function pauses during the wait, costs nothing while idle, and resumes automatically after 5 minutes.

Key Capabilities

Long execution times - Functions can run for up to 1 year. Individual invocations are still limited to 15 minutes, but the workflow continues across multiple invocations.

Automatic checkpointing - Lambda saves your function's state at each step. If something fails, the function resumes from the last checkpoint—not from the beginning.

Built-in retries - Configure retry strategies with exponential backoff. Lambda handles the retry logic and timing automatically.

Wait for callbacks - Pause execution until an external event arrives. Perfect for human approvals, webhook responses, or async API results.

Parallel execution - Run multiple operations concurrently and wait for all to complete. Lambda manages the coordination.

Nested workflows - Invoke other durable functions and compose complex workflows from simple building blocks.

How It Works: The Replay Model

Durable functions use a replay-based execution model. When your function resumes, Lambda replays it from the start—but instead of re-executing operations, it uses checkpointed results.

Here's what happens:

First invocation - Your function runs, executing each step and checkpointing results
Wait or callback - Function pauses, Lambda saves state and stops execution
Resume - Lambda invokes your function again, replaying from the start
Replay - Operations return checkpointed results instantly instead of re-executing
Continue - Function continues past the wait with all context intact

This model ensures your function always sees consistent state, even across issues and restarts. Operations are deterministic—they execute once and replay with the same result.

Learn more: Understanding the Replay Model explains how replay works, why operations must be deterministic, and how to handle non-deterministic code safely.

Common Use Cases

Approval workflows - Wait for human approval before proceeding. The function pauses until someone clicks approve or reject.

Saga patterns - Coordinate distributed transactions with compensating actions. If a step fails, automatically roll back previous steps.

Scheduled tasks - Wait for specific times or intervals. Process data at midnight, send reminders after 24 hours, or retry every 5 minutes.

API orchestration - Call multiple APIs with retries and error handling. Coordinate responses and handle partial issues gracefully.

Data processing pipelines - Process large datasets in stages with checkpoints. Resume from the last successful stage if something fails.

Event-driven workflows - React to external events like webhooks, IoT signals, or user actions. Wait for events and continue processing when they arrive.

Testing Your Workflows

Testing long-running workflows doesn't mean waiting hours. The Durable Execution SDK includes a testing library that runs your functions locally in milliseconds:

import { LocalDurableTestRunner } from '@aws/durable-execution-sdk-js-testing';

const runner = new LocalDurableTestRunner({
  handlerFunction: handler,
});

const execution = await runner.run();

expect(execution.getStatus()).toBe('SUCCEEDED');
expect(execution.getResult()).toEqual({ orderId: '123', status: 'completed' });

The test runner simulates checkpoints, skips time-based waits, and lets you inspect every operation. You can test callbacks, retries, and issues without deploying to AWS.

Learn more: Testing Durable Functions covers local testing, cloud integration tests, debugging techniques, and best practices.

Deploying with AWS SAM

Deploy durable functions using AWS SAM with a few key configurations:

Resources:
  OrderProcessorFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: src/order-processor
      Handler: index.handler
      Runtime: nodejs22.x
      DurableConfig:
        ExecutionTimeout: 900
        RetentionPeriodInDays: 7
    Metadata:
      BuildMethod: esbuild
      BuildProperties:
        EntryPoints:
          - index.ts

The DurableConfig property enables durable execution and sets the workflow timeout. SAM automatically handles IAM permissions for checkpointing and state management.

Learn more: Deploying Durable Functions with SAM covers template configuration, permissions, build settings, and deployment best practices.

When to Use Durable Functions

Your workflow spans multiple steps with waits or callbacks
You need automatic retries with exponential backoff
You want to coordinate multiple async operations
Your process requires human approval or external events
You need to handle long-running tasks without managing state
You prefer writing workflows as code rather than configuration

Getting Started

Install the SDK: npm install @aws/durable-execution-sdk-js
Write your function: Wrap your handler with withDurableExecution()
Use durable operations: context.step(), context.wait(), context.waitForCallback()
Test locally: Use LocalDurableTestRunner for fast iteration
Deploy with SAM: Add DurableConfig to your template
Monitor execution: Use Amazon CloudWatch and AWS X-Ray for observability

Learn More

Understanding the Replay Model - Deep dive into how durable functions work under the hood
Testing Durable Functions - Comprehensive guide to testing workflows locally and in the cloud
Deploying with AWS SAM - Complete deployment guide with templates and best practices

Summary

AWS Lambda Durable Functions let you build long-running workflows without managing infrastructure. Write straightforward async code, and Lambda handles state management, retries, and resumption. Your functions can wait for callbacks, retry issues, and run for up to a year—all while paying only for execution time.

Start with simple workflows, test locally for fast iteration, and deploy with confidence knowing Lambda manages the complexity of distributed state.

Developing AWS Lambda Durable Functions with AWS SAM

Eric D Johnson — Wed, 03 Dec 2025 21:25:35 +0000

How to configure, build, and deploy long-running workflows using SAM templates

You've written a durable function that orchestrates a multi-step workflow. Now you may want to deploy it. AWS Serverless Application Model (AWS SAM) makes this straightforward with specific configurations in your template. Let's walk through how to set up durable functions effectively.

Prerequisites

You'll need AWS SAM CLI version 1.150.1 or greater. Check your version:

sam --version

If you need to upgrade, follow the AWS SAM installation guide.

The Basics: Enabling Durable Functions

To make an AWS Lambda function durable, add the DurableConfig property to your AWS SAM template:

Resources:
  OrderProcessorFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: src/order-processor
      Handler: index.handler
      Runtime: nodejs22.x
      Architectures:
        - arm64
      Timeout: 900                # Function timeout: 15 minutes
      DurableConfig:
        ExecutionTimeout: 3600    # Execution timeout: 1 hour
        RetentionPeriodInDays: 7  # Keep execution history for 7 days

Understanding the two timeout settings is crucial:

Function Timeout (Timeout) controls how long each individual Lambda invocation can run. This is still capped at 15 minutes (900 seconds), just like regular Lambda functions. Each time your durable function checkpoints and resumes, it's a new invocation with its own 15-minute window.

Execution Timeout (ExecutionTimeout) controls how long the entire workflow can run across all invocations. This can be up to 1 year. Your workflow can pause, wait for callbacks, and resume many times, as long as the total elapsed time doesn't exceed this limit.

For long-running workflows where the execution timeout exceeds the function timeout, you must invoke the function asynchronously. Synchronous invocations will fail with a validation error if the execution timeout is longer than the function timeout.

Installing the SDK

Add the durable execution SDK to your function's dependency manifest. SAM will automatically install it during the build process.

TypeScript/JavaScript - Add to package.json:

{
  "dependencies": {
    "@aws/durable-execution-sdk-js": "^1.0.0"
  }
}

Python - Add to requirements.txt:

aws-durable-execution-sdk-python

For client functions that send callbacks or query execution state, add the AWS SDK:

TypeScript/JavaScript:

{
  "dependencies": {
    "@aws-sdk/client-lambda": "^3.0.0"
  }
}

Python:

boto3

Build Configuration

Configure SAM to build your TypeScript functions with esbuild:

Resources:
  OrderProcessorFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: src/order-processor
      Handler: index.handler
      Runtime: nodejs22.x
      Architectures:
        - arm64
      DurableConfig:
        ExecutionTimeout: 3600
        RetentionPeriodInDays: 7
      Metadata:
        BuildMethod: esbuild
        BuildProperties:
          EntryPoints:
            - index.ts

SAM will automatically compile your TypeScript code and bundle dependencies into a deployment package.

IAM Permissions

AWS SAM automatically grants the necessary permissions for durable functions to checkpoint and manage their execution state. You don't need to explicitly configure these permissions.

However, you may want to configure permissions for client functions that interact with durable executions. For functions that send callbacks:

BaristaCallbackFunction:
  Type: AWS::Serverless::Function
  Properties:
    CodeUri: src/barista-callback
    Handler: index.handler
    Policies:
      - Statement:
          - Effect: Allow
            Action:
              - lambda:SendDurableExecutionCallbackSuccess
              - lambda:SendDurableExecutionCallbackFailure
              - lambda:SendDurableExecutionCallbackHeartbeat
            Resource: !Sub 'arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:${CoffeeOrderFunction}'

For functions that monitor execution state or stop executions:

MonitoringFunction:
  Type: AWS::Serverless::Function
  Properties:
    CodeUri: src/monitoring
    Handler: index.handler
    Policies:
      - Statement:
          - Effect: Allow
            Action:
              - lambda:GetDurableExecution
              - lambda:GetDurableExecutionHistory
              - lambda:ListDurableExecutionsByFunction
              - lambda:SendDurableExecutionCallbackSuccess
              - lambda:SendDurableExecutionCallbackHeartbeat
              - lambda:SendDurableExecutionCallbackFailure
              - lambda:StopDurableExecution
            Resource: !Sub 'arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:${CoffeeOrderFunction}'

These permissions allow client functions to interact with your durable workflows by sending callbacks, monitoring execution state, or stopping running executions.

Using Globals for Common Configuration

AWS SAM's Globals section reduces repetition when you have multiple functions with shared settings:

Globals:
  Function:
    Runtime: nodejs22.x
    Architectures:
      - arm64
    Timeout: 30
    MemorySize: 512
    Tracing: Active

Resources:
  OrderProcessorFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: src/order-processor
      Handler: index.handler
      DurableConfig:
        ExecutionTimeout: 3600
        RetentionPeriodInDays: 7

  CallbackHandlerFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: src/callback-handler
      Handler: index.handler
      # No DurableConfig - this is a regular Lambda function

You can set common runtime, architecture, and memory settings at the global level. For DurableConfig, configure it at the function level to make it explicit which functions are durable. Functions without DurableConfig are regular Lambda functions.

A Complete Example

Here's a full AWS SAM template for a coffee ordering system with durable workflows:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Coffee Ordering System with Durable Functions

Globals:
  Function:
    Runtime: nodejs22.x
    Architectures:
      - arm64
    Timeout: 30
    MemorySize: 512
    Tracing: Active

Resources:
  # ========== AMAZON DYNAMODB ==========

  OrdersTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: CoffeeOrders
      BillingMode: PAY_PER_REQUEST
      AttributeDefinitions:
        - AttributeName: orderId
          AttributeType: S
      KeySchema:
        - AttributeName: orderId
          KeyType: HASH

  # ========== FUNCTIONS ==========

  CoffeeOrderFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: src/coffee-order
      Handler: index.handler
      AutoPublishAlias: Live
      DurableConfig:
        ExecutionTimeout: 3600
        RetentionPeriodInDays: 7
      Environment:
        Variables:
          ORDERS_TABLE: !Ref OrdersTable
      Policies:
        - DynamoDBCrudPolicy:
            TableName: !Ref OrdersTable
      Events:
        ApiEvent:
          Type: Api
          Properties:
            Path: /orders
            Method: post
      Metadata:
        BuildMethod: esbuild
        BuildProperties:
          EntryPoints:
            - index.ts

  BaristaCallbackFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: src/barista-callback
      Handler: index.handler
      Policies:
        - Statement:
            - Effect: Allow
              Action:
                - lambda:SendDurableExecutionCallbackSuccess
                - lambda:SendDurableExecutionCallbackFailure
              Resource: !Sub 'arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:${CoffeeOrderFunction}'
      Events:
        ApiEvent:
          Type: Api
          Properties:
            Path: /barista/accept/{orderId}
            Method: post
      Metadata:
        BuildMethod: esbuild
        BuildProperties:
          EntryPoints:
            - index.ts

Outputs:
  ApiUrl:
    Description: API Gateway endpoint URL
    Value: !Sub 'https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod'

  CoffeeOrderFunctionArn:
    Description: Coffee Order Function ARN
    Value: !GetAtt CoffeeOrderFunction.Arn

Building and Deploying

Build your application with AWS SAM:

sam build

This builds your functions and layers. AWS SAM automatically runs the Makefiles for your layers and uses esbuild for your TypeScript functions.

Deploy to AWS:

sam deploy --guided

The --guided flag walks you through configuration options. After the first deployment, AWS SAM saves your settings in samconfig.toml, so subsequent deploys are just:

sam deploy

Local Testing

Test your durable function locally before deploying:

sam local invoke CoffeeOrderFunction --event events/order.json

The event file contains your test payload:

{
  "orderId": "test-123",
  "attendeeId": "user-456",
  "orderDetails": {
    "drinkType": "Latte",
    "size": "Grande"
  }
}

Accessing Cloud Resources Locally

When testing locally, you often need to access deployed AWS resources like Amazon DynamoDB tables or Amazon EventBridge buses. Use the --env-vars flag with a JSON file to provide environment variables and credentials:

sam local invoke CoffeeOrderFunction \
  --event events/order.json \
  --env-vars locals.json

Create a locals.json file with environment variables for each function:

{
  "CoffeeOrderFunction": {
    "AWS_REGION": "us-east-1",
    "ORDERS_TABLE_NAME": "CoffeeOrders",
    "EVENT_BUS_NAME": "CoffeeOrderingEventBus"
  },
  "BaristaCallbackFunction": {
    "AWS_REGION": "us-east-1",
    "ORDERS_TABLE_NAME": "CoffeeOrders"
  }
}

This allows your local function to interact with deployed resources in AWS, assuming your AWS credentials have the necessary permissions. Your function code can access these variables through process.env:

const tableName = process.env.ORDERS_TABLE_NAME;
const region = process.env.AWS_REGION;

Tracking Execution State

For durable functions, you can track execution state:

# Get execution details
sam local execution get $EXECUTION_ARN

# View execution history
sam local execution history $EXECUTION_ARN

# Stop a running execution
sam local execution stop $EXECUTION_ARN

Test callbacks locally:

# Send success callback
sam local callback succeed $CALLBACK_ID --result '{"status": "accepted"}'

# Send failure callback
sam local callback fail $CALLBACK_ID --error '{"message": "Rejected"}'

# Send heartbeat
sam local callback heartbeat $CALLBACK_ID

Remote Testing

After deploying, test against your live functions:

# Invoke the function (automatically uses $LATEST qualifier)
sam remote invoke CoffeeOrderFunction --event events/order.json

# Invoke a specific qualifier/alias
sam remote invoke CoffeeOrderFunction --event events/order.json --parameter Qualifier=prod

# Get execution details
sam remote execution get $EXECUTION_ARN

# View execution history
sam remote execution history $EXECUTION_ARN

By default, sam remote invoke uses the $LATEST qualifier. You can override this with --parameter Qualifier=<your-qualifier> to test against a specific version or alias.

The execution history shows every step, checkpoint, and state transition. It's invaluable for debugging workflows in production.

Configuration Best Practices

ExecutionTimeout can be up to 1 year (31,536,000 seconds) and controls how long your entire workflow can run across all invocations. For a coffee order that waits up to 5 minutes for barista acceptance, consider setting it to 600 seconds (10 minutes) to allow some buffer. For document processing that might take hours or days, you can set it much higher - up to the 1-year maximum.

Function Timeout is still capped at 15 minutes (900 seconds) and controls how long each individual invocation can run. Set this based on how long your function needs between checkpoints. If your steps typically complete in seconds, starting with 30 seconds is reasonable. If you have longer-running operations, consider increasing it up to the 15-minute maximum.

Async vs Sync Invocation: If your execution timeout exceeds your function timeout, you must use asynchronous invocation. Synchronous invocations will fail validation when the execution timeout is longer than the function timeout. Configure your event sources (API Gateway, EventBridge, etc.) to invoke asynchronously for long-running workflows.

RetentionPeriodInDays determines how long execution history is kept. Consider using 7-30 days for development environments where you're actively debugging. For production environments where you might need to investigate issues weeks later, consider using 90+ days.

Memory affects both performance and cost. Starting with 512 MB and adjusting based on your function's needs is a common approach. Durable functions checkpoint frequently, so memory usage is typically lower than equivalent non-durable functions.

Monitoring and Observability

Enable AWS X-Ray tracing to see how your workflow executes:

Globals:
  Function:
    Tracing: Active

This traces each invocation and shows you the complete execution path across multiple invocations.

Add Amazon CloudWatch Logs Insights queries to analyze execution patterns:

fields @timestamp, @message
| filter @message like /CHECKPOINT/
| sort @timestamp desc
| limit 100

Create CloudWatch alarms for execution failures:

ExecutionFailureAlarm:
  Type: AWS::CloudWatch::Alarm
  Properties:
    AlarmName: CoffeeOrderExecutionFailures
    MetricName: Errors
    Namespace: AWS/Lambda
    Statistic: Sum
    Period: 300
    EvaluationPeriods: 1
    Threshold: 5
    ComparisonOperator: GreaterThanThreshold
    Dimensions:
      - Name: FunctionName
        Value: !Ref CoffeeOrderFunction

Common Patterns

Event-driven workflows triggered by API Gateway, EventBridge, or Amazon Simple Queue Service (Amazon SQS) - useful for order processing, approval workflows, and asynchronous task execution:

CoffeeOrderFunction:
  Type: AWS::Serverless::Function
  Properties:
    # ... other properties
    Events:
      ApiTrigger:
        Type: Api
        Properties:
          Path: /orders
          Method: post
      EventBridgeTrigger:
        Type: EventBridgeRule
        Properties:
          Pattern:
            source:
              - coffee.orders
            detail-type:
              - OrderPlaced

For long-running workflows, configure API Gateway to invoke asynchronously:

CoffeeOrderFunction:
  Type: AWS::Serverless::Function
  Properties:
    # ... other properties
    Timeout: 900  # 15 minutes
    DurableConfig:
      ExecutionTimeout: 86400  # 24 hours
    Events:
      ApiTrigger:
        Type: Api
        Properties:
          Path: /orders
          Method: post
          RequestParameters:
            - method.request.header.X-Amz-Invocation-Type:
                Required: false
                Caching: false

Scheduled workflows that run on a timer - useful for daily reports, periodic data processing, or scheduled maintenance tasks:

DailyReportFunction:
  Type: AWS::Serverless::Function
  Properties:
    # ... other properties
    Events:
      DailySchedule:
        Type: Schedule
        Properties:
          Schedule: cron(0 9 * * ? *)  # 9 AM daily

Fan-out workflows that process items in parallel - useful for batch processing, data transformation, or concurrent API calls:

BatchProcessorFunction:
  Type: AWS::Serverless::Function
  Properties:
    # ... other properties
    ReservedConcurrentExecutions: 10  # Limit parallel executions

Troubleshooting

Validation error on invoke: If you get a validation error saying "Execution timeout must be within the function timeout," you're trying to synchronously invoke a function where the execution timeout exceeds the function timeout. Use asynchronous invocation for long-running workflows.

Individual invocation timeout: If your function times out during a single invocation, consider increasing the Timeout property. Start by doubling the current value, up to the maximum of 900 seconds. This is separate from the workflow's total ExecutionTimeout.

Workflow timeout: If your entire workflow times out, consider increasing the ExecutionTimeout in DurableConfig. This can be up to 1 year (31,536,000 seconds).

Permission denied for callbacks: Ensure client functions have the correct IAM permissions to interact with durable executions. Functions sending callbacks need SendDurableExecutionCallback* permissions with the target durable function as the resource.

Summary

Deploying durable functions with AWS SAM involves several key configurations: enable DurableConfig on your function to set execution timeout and retention period, install the durable execution SDK as a dependency, and configure esbuild for TypeScript compilation. AWS SAM facilitates the IAM permissions needed for durable execution operations. For client functions that interact with durable executions, consider explicitly configuring permissions for callback and monitoring operations.

Use Globals to reduce repetition across functions, set appropriate timeouts for both individual invocations and total workflow duration, and enable tracing for observability. Test locally with sam local invoke and sam local execution commands before deploying to AWS.

With these patterns in place, you can deploy long-running workflows that span hours or days, handle failures gracefully, and provide full visibility into execution state.

The Replay Model: How AWS Lambda Durable Functions Actually Work

Eric D Johnson — Wed, 03 Dec 2025 00:27:54 +0000

Understanding the checkpoint-based execution that makes long-running workflows possible

You write an AWS Lambda function that looks like it runs continuously for hours. But Lambda functions can only run for 15 minutes. How does this work?

The answer is replay - a checkpoint-based execution model that makes your function restart from the beginning on every invocation, but skip the work it's already done. It's elegant, efficient, and once you understand it, surprisingly intuitive.

The Core Principle

Here's the fundamental truth about durable functions:

Your handler function re-executes from the beginning on every invocation, but completed operations return cached results from checkpoints instead of re-executing.

Let's see this in action with a simple workflow:

async function processOrder(event: any, ctx: DurableContext) {
  const order = await ctx.step('create-order', async () => {
    console.log('Creating order...');
    return { orderId: '123', total: 50 };
  });

  const payment = await ctx.step('process-payment', async () => {
    console.log('Processing payment...');
    return { transactionId: 'txn-456', status: 'success' };
  });

  await ctx.wait({ seconds: 300 }); // Wait 5 minutes

  const notification = await ctx.step('send-notification', async () => {
    console.log('Sending notification...');
    return { sent: true };
  });

  return { order, payment, notification };
}

Here's what actually happens across two separate Lambda invocations:

Invocation 1 (t=0s):

Creating order...
Processing payment...
[Checkpoint: create-order completed]
[Checkpoint: process-payment completed]
[Function terminates - waiting 5 minutes]

Invocation 2 (t=300s, after wait completes):

[REPLAY MODE: Skipping create-order - returning cached result]
[REPLAY MODE: Skipping process-payment - returning cached result]
[EXECUTION MODE: Running send-notification]
Sending notification...
[Checkpoint: send-notification completed]
[Function completes]

Notice what happened: The function started from the beginning both times, but on the second invocation, create-order and process-payment didn't re-execute. The logs only appeared once, even though the code ran twice. The function seamlessly continued from where it left off.

This is replay.

Execution Modes: The Secret Sauce

The SDK operates in two modes that automatically switch based on what's happening.

ExecutionMode is when the function is executing operations for the first time. Operations execute normally, results are saved to checkpoints, logs are emitted, and side effects happen.

ReplayMode is when the function is replaying previously completed operations. Operations return cached results instantly without actual execution, logs are suppressed, and no side effects occur.

The SDK automatically transitions from ReplayMode to ExecutionMode when it reaches an operation that hasn't been completed yet.

How Checkpoints Work

Every operation creates a checkpoint that stores:

{
  operationId: "2",           // Sequential ID
  operationType: "STEP",      // STEP, WAIT, INVOKE, etc.
  operationName: "process-payment",
  status: "SUCCEEDED",        // STARTED, SUCCEEDED, FAILED, PENDING
  result: {                   // The actual return value
    transactionId: "txn-456",
    status: "success"
  }
}

When your function restarts, the SDK loads all checkpoints from storage, indexes them by operation ID, returns cached results for completed operations, and executes new operations normally.

The Determinism Requirement

For replay to work, your code must be deterministic - the same sequence of operations must happen in the same order every time.

What Breaks Determinism

// ❌ Random control flow
if (Math.random() > 0.5) {
  await ctx.step('optional-step', async () => doSomething());
}
// First run: random = 0.7, step executes, checkpoint created
// Second run: random = 0.3, step skipped
// Error: Expected operation 'optional-step' at position 2, not found!

// ❌ Time-based branching
const isWeekend = new Date().getDay() >= 5;
if (isWeekend) {
  await ctx.step('weekend-task', async () => doWeekendWork());
}
// First run: Friday (day 5), step executes
// Second run: Monday (day 1), step skipped
// Error: Replay consistency violation!

// ❌ External state
let counter = 0;
await ctx.step('step1', async () => {
  counter++; // Won't increment during replay!
  return counter;
});

How to Write Deterministic Code

The rule is simple: capture non-deterministic values inside steps.

// ✅ Capture random values in steps
const randomId = await ctx.step('generate-id', async () => {
  return crypto.randomUUID(); // Executed once, cached on replay
});

// ✅ Capture timestamps in steps
const timestamp = await ctx.step('get-timestamp', async () => {
  return Date.now(); // Same timestamp on every replay
});

// ✅ Use event data for control flow
if (event.shouldProcess) { // Deterministic - same event every time
  await ctx.step('process', async () => doWork());
}

// ✅ Capture time-based decisions in steps
const isWeekend = await ctx.step('check-day', async () => {
  return new Date().getDay() >= 5;
});
if (isWeekend) {
  await ctx.step('weekend-task', async () => doWeekendWork());
}

Replay Consistency Validation

The SDK validates that operations occur in the same order on every invocation:

// What gets validated:
// 1. Operation type (STEP, WAIT, INVOKE)
// 2. Operation name (your identifier)
// 3. Operation position (sequential order)

// Example validation error:
// "Replay consistency violation: Expected operation 'process-payment' 
//  of type STEP at position 2, but found operation 'send-email' of type STEP"

This catches bugs early. If your code's execution path changes between invocations, you'll know immediately.

A Complete Example: Order Processing with Replay

Let's examine a potential workflow through multiple invocations:

async function processOrder(event: any, ctx: DurableContext) {
  ctx.logger.info('Order processing started', { orderId: event.orderId });

  // Step 1: Validate inventory
  const inventory = await ctx.step('check-inventory', async () => {
    ctx.logger.info('Checking inventory');
    const response = await fetch(`https://api.inventory.com/check`, {
      method: 'POST',
      body: JSON.stringify({ items: event.items })
    });
    return response.json();
  });

  if (!inventory.available) {
    ctx.logger.warn('Out of stock', { missing: inventory.missing });
    return { status: 'out-of-stock' };
  }

  // Step 2: Process payment
  const payment = await ctx.step('process-payment', async () => {
    ctx.logger.info('Processing payment', { amount: inventory.total });
    const response = await fetch(`https://api.payments.com/charge`, {
      method: 'POST',
      body: JSON.stringify({ 
        customerId: event.customerId, 
        amount: inventory.total 
      })
    });
    return response.json();
  });

  // Step 3: Wait for warehouse confirmation (5 minute timeout)
  ctx.logger.info('Waiting for warehouse confirmation');
  const confirmation = await ctx.waitForCallback(
    'warehouse-confirm',
    async (callbackId) => {
      // Send callback ID to warehouse system
      await fetch(`https://api.warehouse.com/notify`, {
        method: 'POST',
        body: JSON.stringify({ orderId: order.id, callbackId })
      });
    },
    { timeout: { seconds: 300 } }
  );

  // Step 4: Send notification
  await ctx.step('notify-customer', async () => {
    ctx.logger.info('Sending customer notification');
    await fetch(`https://api.notifications.com/send`, {
      method: 'POST',
      body: JSON.stringify({
        customerId: event.customerId,
        message: 'Your order is confirmed!'
      })
    });
  });

  ctx.logger.info('Order processing completed');
  return { status: 'completed', orderId: payment.orderId };
}

Invocation Timeline

Invocation 1 (t=0s) runs in ExecutionMode. The logs show "Order processing started", "Checking inventory", "Processing payment", and "Waiting for warehouse confirmation". Checkpoints are created for check-inventory and process-payment, both marked as SUCCEEDED. The function then enters a waiting state for the callback 'warehouse-confirm', creating a checkpoint to persist the callbackId and set the timer for the timeout.

Invocation 2 (t=120s, warehouse confirms) starts in ReplayMode and transitions to ExecutionMode. During the replay phase, "Order processing started", "Checking inventory", "Processing payment", and "Waiting for warehouse confirmation" are all suppressed - the inventory and payment steps return their cached results without re-executing. Once the function reaches new operations, it switches to ExecutionMode, checkpoints the callback result, and logs "Sending customer notification" and "Order processing completed". A checkpoint is created for notify-customer marked as SUCCEEDED, and the function completes.

Notice how the function ran from the beginning both times, but the inventory and payment APIs were only called once. Logs only appeared once with no duplicates, and the function seamlessly continued after the callback.

Operation IDs: The Replay Index

Operations are identified by sequential IDs that determine replay order:

// Root context operations
await ctx.step('step1', ...);  // ID: "1"
await ctx.step('step2', ...);  // ID: "2"
await ctx.step('step3', ...);  // ID: "3"

// Child context operations (from ctx.runInChildContext)
await ctx.runInChildContext(async (childCtx) => {  // This operation gets ID: "4"
  await childCtx.step('child-step1', ...);  // ID: "4-1"
  await childCtx.step('child-step2', ...);  // ID: "4-2"
});

IDs are deterministic - they're based on execution order, not operation names. This is why operation order must be consistent.

Common Pitfalls and Solutions

1. Non-Deterministic Control Flow

// ❌ BAD: Random branching
if (Math.random() > 0.5) {
  await ctx.step('optional', async () => doWork());
}

// ✅ GOOD: Event-based branching
if (event.shouldDoWork) {
  await ctx.step('optional', async () => doWork());
}

// ✅ GOOD: Capture decision in step
const shouldDoWork = await ctx.step('decide', async () => {
  return Math.random() > 0.5;
});
if (shouldDoWork) {
  await ctx.step('optional', async () => doWork());
}

2. Mutating Closure Variables

// ❌ BAD: External mutation
let total = 0;
await ctx.step('add-items', async () => {
  total += 10; // Won't happen during replay!
});

// ✅ GOOD: Return values
const total = await ctx.step('calculate-total', async () => {
  return 10;
});

3. Side Effects Outside Steps

// ❌ BAD: Direct API calls
const data = await fetch('https://api.example.com/data');
await ctx.step('process', async () => processData(data));
// API called on every replay!

// ✅ GOOD: API calls inside steps
const data = await ctx.step('fetch-data', async () => {
  return await fetch('https://api.example.com/data');
});
await ctx.step('process', async () => processData(data));

Debugging Replay Issues

When replay goes wrong, use the execution history:

# View execution history
sam remote execution history $EXECUTION_ARN

# See detailed operation data
sam remote execution history $EXECUTION_ARN --format json

The history shows every operation that executed, the order they ran in, their results or errors, and when mode transitions occurred. Look for operations appearing in different orders, missing or extra operations, or operations with different names at the same position.

Best Practices for Replay-Safe Code

Wrap all non-deterministic operations in steps - random numbers, timestamps, API calls, and database queries should always be inside ctx.step(). Use event data for control flow rather than runtime-generated values, and never mutate closure variables - return values from steps instead. Keep operation order consistent so the same sequence happens every time. Test with multiple invocations to verify replay behavior locally, and check execution history to debug replay issues quickly.

Summary

The replay model is what makes durable functions possible. Your function restarts from the beginning on every invocation, but completed operations return cached results without re-executing. The SDK automatically switches between ReplayMode and ExecutionMode, and your code must be deterministic for replay to work correctly.

Once you internalize these principles, writing durable functions becomes natural. You write straightforward procedural code, and the SDK handles all the complexity of checkpointing, replay, and state management. The result? Long-running workflows that look like simple functions. That's the magic of replay.

Testing AWS Lambda Durable Functions in TypeScript

Eric D Johnson — Tue, 02 Dec 2025 23:46:01 +0000

How to write fast, reliable tests for workflows that span hours

Testing long-running workflows is tricky. Your function waits for callbacks, retries failed operations, and spans multiple invocations. How do you test that without actually waiting hours or deploying to AWS?

The AWS Durable Execution SDK includes a testing library that solves this. You can run your durable functions locally, skip time-based waits, inspect every operation, and verify behavior without touching AWS. Let's see how.

The Testing Library

The SDK provides testing tools for different scenarios:

LocalDurableTestRunner runs your function in-process with a simulated checkpoint server. Tests execute in milliseconds, even for workflows that would normally take hours. This is what you'll use for most testing.

CloudDurableTestRunner tests against deployed Lambda functions in AWS. Use this for integration tests or when you need to verify behavior in the actual AWS environment.

run-durable CLI provides quick command-line testing without writing test code. Perfect for rapid iteration and debugging during development.

We'll focus on local testing since that's where you'll spend most of your time.

Your First Test

Let's start with a simple durable function:

// order-processor.ts
import { DurableContext, withDurableExecution } from '@aws/durable-execution-sdk-js';

export const handler = withDurableExecution(
  async (event: any, context: DurableContext) => {
    const order = await context.step('create-order', async () => {
      return { orderId: '123', total: 50 };
    });

    await context.wait({ seconds: 300 }); // Wait 5 minutes

    const notification = await context.step('send-notification', async () => {
      return { sent: true };
    });

    return { order, notification };
  }
);

Here's how to test it:

// order-processor.test.ts
import { LocalDurableTestRunner } from '@aws/durable-execution-sdk-js-testing';
import { handler } from './order-processor';

describe('Order Processor', () => {
  beforeAll(async () => {
    await LocalDurableTestRunner.setupTestEnvironment({ skipTime: true });
  });

  afterAll(async () => {
    await LocalDurableTestRunner.teardownTestEnvironment();
  });

  it('should process order successfully', async () => {
    const runner = new LocalDurableTestRunner({
      handlerFunction: handler,
    });

    const execution = await runner.run();

    expect(execution.getStatus()).toBe('SUCCEEDED');
    expect(execution.getResult()).toEqual({
      order: { orderId: '123', total: 50 },
      notification: { sent: true }
    });
  });
});

That's it. The test runs in milliseconds, even though the function has a 5-minute wait. The skipTime: true option tells the test runner to skip over time-based operations instantly.

Setup and Teardown

Every test suite needs setup and teardown:

beforeAll(async () => {
  await LocalDurableTestRunner.setupTestEnvironment({ skipTime: true });
});

afterAll(async () => {
  await LocalDurableTestRunner.teardownTestEnvironment();
});

setupTestEnvironment() starts a local checkpoint server and optionally installs fake timers for time skipping. teardownTestEnvironment() cleans everything up. Call these once per test file in beforeAll and afterAll.

The skipTime option is crucial for fast tests. When enabled, operations like context.wait(), setTimeout, and retry delays complete instantly. Without it, your tests would actually wait for the specified durations.

Inspecting Operations

The real power of the testing library is inspecting what your function did:

it('should execute operations in correct order', async () => {
  const runner = new LocalDurableTestRunner({
    handlerFunction: handler,
  });

  const execution = await runner.run();

  // Get all operations
  const operations = execution.getOperations();
  expect(operations).toHaveLength(3); // create-order, wait, send-notification

  // Get specific operation by index
  const createOrder = runner.getOperationByIndex(0);
  expect(createOrder.getType()).toBe('STEP');
  expect(createOrder.getStatus()).toBe('SUCCEEDED');
  expect(createOrder.getStepDetails()?.result).toEqual({
    orderId: '123',
    total: 50
  });

  // Get wait operation
  const waitOp = runner.getOperationByIndex(1);
  expect(waitOp.getType()).toBe('WAIT');
  expect(waitOp.getWaitDetails()?.waitSeconds).toBe(300);

  // Get notification operation
  const notification = runner.getOperationByIndex(2);
  expect(notification.getStepDetails()?.result).toEqual({ sent: true });
});

You can access operations by index, by name, or by ID. Each operation exposes its type, status, and type-specific details like step results or wait durations.

Testing Failures and Retries

Real workflows fail. Let's test retry behavior:

// payment-processor.ts
import { DurableContext, withDurableExecution, createRetryStrategy } from '@aws/durable-execution-sdk-js';

export const handler = withDurableExecution(
  async (event: any, context: DurableContext) => {
    const payment = await context.step('process-payment', async () => {
      const response = await fetch('https://api.payments.com/charge', {
        method: 'POST',
        body: JSON.stringify({ amount: event.amount })
      });
      if (!response.ok) throw new Error('Payment failed');
      return response.json();
    }, {
      retryStrategy: createRetryStrategy({
        maxAttempts: 3,
        backoffRate: 2,
        initialInterval: 1000
      })
    });

    return payment;
  }
);

Test it with mocks:

// payment-processor.test.ts
import { LocalDurableTestRunner } from '@aws/durable-execution-sdk-js-testing';
import { handler } from './payment-processor';

// Store original fetch
const originalFetch = global.fetch;

describe('Payment Processor', () => {
  beforeAll(async () => {
    await LocalDurableTestRunner.setupTestEnvironment({ skipTime: true });
  });

  afterAll(async () => {
    await LocalDurableTestRunner.teardownTestEnvironment();
  });

  beforeEach(() => {
    // Mock fetch for external APIs only
    global.fetch = jest.fn((url: string | URL | Request, ...args) => {
      const urlString = url.toString();
      // Let checkpoint server calls through
      if (urlString.includes('127.0.0.1') || urlString.includes('localhost')) {
        return originalFetch(url as any, ...args);
      }
      // Mock external API calls
      return Promise.reject(new Error('Unmocked fetch call'));
    }) as any;
  });

  afterEach(() => {
    global.fetch = originalFetch;
  });

  it('should succeed on first attempt', async () => {
    (global.fetch as jest.Mock).mockImplementation((url: string | URL | Request, ...args) => {
      const urlString = url.toString();
      if (urlString.includes('127.0.0.1') || urlString.includes('localhost')) {
        return originalFetch(url as any, ...args);
      }
      if (urlString.includes('api.payments.com')) {
        return Promise.resolve({
          ok: true,
          json: async () => ({ transactionId: 'txn-123', status: 'success' })
        });
      }
      return Promise.reject(new Error('Unmocked fetch'));
    });

    const runner = new LocalDurableTestRunner({
      handlerFunction: handler,
    });

    const execution = await runner.run({ payload: { amount: 100 } });

    expect(execution.getStatus()).toBe('SUCCEEDED');
    expect(execution.getResult()).toEqual({
      transactionId: 'txn-123',
      status: 'success'
    });
  });

  it('should retry on failure and eventually succeed', async () => {
    let callCount = 0;
    (global.fetch as jest.Mock).mockImplementation((url: string | URL | Request, ...args) => {
      const urlString = url.toString();
      if (urlString.includes('127.0.0.1') || urlString.includes('localhost')) {
        return originalFetch(url as any, ...args);
      }
      if (urlString.includes('api.payments.com')) {
        callCount++;
        if (callCount === 1) {
          return Promise.reject(new Error('Network error'));
        }
        return Promise.resolve({
          ok: true,
          json: async () => ({ transactionId: 'txn-456', status: 'success' })
        });
      }
      return Promise.reject(new Error('Unmocked fetch'));
    });

    const runner = new LocalDurableTestRunner({
      handlerFunction: handler,
    });

    const execution = await runner.run({ payload: { amount: 100 } });

    expect(execution.getStatus()).toBe('SUCCEEDED');
    expect(execution.getResult()).toEqual({
      transactionId: 'txn-456',
      status: 'success'
    });
  });

  it('should fail after exhausting retries', async () => {
    (global.fetch as jest.Mock).mockImplementation((url: string | URL | Request, ...args) => {
      const urlString = url.toString();
      if (urlString.includes('127.0.0.1') || urlString.includes('localhost')) {
        return originalFetch(url as any, ...args);
      }
      if (urlString.includes('api.payments.com')) {
        return Promise.reject(new Error('Persistent failure'));
      }
      return Promise.reject(new Error('Unmocked fetch'));
    });

    const runner = new LocalDurableTestRunner({
      handlerFunction: handler,
    });

    const execution = await runner.run({ payload: { amount: 100 } });

    expect(execution.getStatus()).toBe('FAILED');
    const error = execution.getError();
    expect(error?.errorMessage).toBe('Persistent failure');
    expect(error?.errorType).toBe('StepError');
  });
});

With skipTime: true, the retry delays happen instantly. The test verifies retry behavior without actually waiting for backoff intervals.

Important: When mocking fetch, preserve calls to localhost/127.0.0.1 so the checkpoint server can communicate. Only mock external API calls.

Testing Callbacks

Durable functions often wait for external events. Here's how to test that:

// approval-workflow.ts
import { DurableContext, withDurableExecution } from '@aws/durable-execution-sdk-js';

export const handler = withDurableExecution(
  async (event: any, context: DurableContext) => {
    const request = await context.step('create-request', async () => {
      return { requestId: event.requestId, status: 'pending' };
    });

    // Wait up to 24 hours for approval
    try {
      const approval = await context.waitForCallback('approval', 86400);
      return { status: 'approved', approvedBy: approval.userId };
    } catch (error) {
      return { status: 'timeout' };
    }
  }
);

Test with callback simulation:

// approval-workflow.test.ts
import { LocalDurableTestRunner } from '@aws/durable-execution-sdk-js-testing';
import { handler } from './approval-workflow';

describe('Approval Workflow', () => {
  beforeAll(async () => {
    await LocalDurableTestRunner.setupTestEnvironment({ skipTime: true });
  });

  afterAll(async () => {
    await LocalDurableTestRunner.teardownTestEnvironment();
  });

  it('should complete when approval is received', async () => {
    const runner = new LocalDurableTestRunner({
      handlerFunction: handler,
    });

    const execution = runner.run({ payload: { requestId: '123' } });

    // Get the callback operation
    const callbackOp = runner.getOperation('approval');

    // Simulate approval callback
    await callbackOp.sendCallbackSuccess({ userId: 'user-456' });

    const result = await execution;

    expect(result.getStatus()).toBe('SUCCEEDED');
    expect(result.getResult()).toEqual({
      status: 'approved',
      approvedBy: 'user-456'
    });
  });

  it('should timeout when no approval is received', async () => {
    const runner = new LocalDurableTestRunner({
      handlerFunction: handler,
    });

    // Don't send callback - let it timeout
    const execution = await runner.run({ payload: { requestId: '123' } });

    expect(execution.getStatus()).toBe('SUCCEEDED');
    expect(execution.getResult()).toEqual({ status: 'timeout' });
  });
});

The getOperation() method returns an operation handle that lets you send callbacks using sendCallbackSuccess(). With skipTime: true, the timeout happens instantly if no callback is sent.

Testing Parallel Operations

Durable functions can run operations in parallel:

// batch-processor.ts
import { DurableContext, withDurableExecution } from '@aws/durable-execution-sdk-js';

export const handler = withDurableExecution(
  async (event: any, context: DurableContext) => {
    await context.parallel(
      'process-items',
      event.items.map((item: any, index: number) =>
        async (childContext: DurableContext) => {
          return await childContext.step(`process-item-${index}`, async () => {
            return { id: item.id, processed: true };
          });
        }
      )
    );

    return {
      total: event.items.length,
      successful: event.items.length
    };
  }
);

Test parallel execution:

// batch-processor.test.ts
import { LocalDurableTestRunner } from '@aws/durable-execution-sdk-js-testing';
import { handler } from './batch-processor';

describe('Batch Processor', () => {
  beforeAll(async () => {
    await LocalDurableTestRunner.setupTestEnvironment({ skipTime: true });
  });

  afterAll(async () => {
    await LocalDurableTestRunner.teardownTestEnvironment();
  });

  it('should process all items in parallel', async () => {
    const runner = new LocalDurableTestRunner({
      handlerFunction: handler,
    });

    const execution = await runner.run({
      payload: {
        items: [
          { id: '1' },
          { id: '2' },
          { id: '3' }
        ]
      }
    });

    expect(execution.getStatus()).toBe('SUCCEEDED');
    expect(execution.getResult()).toEqual({
      total: 3,
      successful: 3
    });

    // Verify operations were tracked
    const operations = execution.getOperations();
    expect(operations.length).toBeGreaterThan(0);
  });
});

The context.parallel() method takes a name and an array of functions that each receive a child context. Each parallel operation is tracked independently.

Testing Nested Functions

Durable functions can invoke other durable functions:

// main-workflow.ts
import { DurableContext, withDurableExecution } from '@aws/durable-execution-sdk-js';

export const childHandler = withDurableExecution(
  async (event: any, context: DurableContext) => {
    const result = await context.step('child-step', async () => {
      return { processed: event.data };
    });
    return result;
  }
);

export const mainHandler = withDurableExecution(
  async (event: any, context: DurableContext) => {
    const childResult = await context.invoke('child-function', {
      data: event.input
    });

    return { main: 'completed', child: childResult };
  }
);

// main-workflow.test.ts
import { LocalDurableTestRunner } from '@aws/durable-execution-sdk-js-testing';
import { mainHandler, childHandler } from './main-workflow';

describe('Main Workflow', () => {
  beforeAll(async () => {
    await LocalDurableTestRunner.setupTestEnvironment({ skipTime: true });
  });

  afterAll(async () => {
    await LocalDurableTestRunner.teardownTestEnvironment();
  });

  it('should invoke child function successfully', async () => {
    const runner = new LocalDurableTestRunner({
      handlerFunction: mainHandler,
    });

    // Register the child function
    runner.registerDurableFunction('child-function', childHandler);

    const execution = await runner.run({ payload: { input: 'test-data' } });

    expect(execution.getStatus()).toBe('SUCCEEDED');
    expect(execution.getResult()).toEqual({
      main: 'completed',
      child: { processed: 'test-data' }
    });
  });
});

The registerDurableFunction() method tells the test runner how to handle context.invoke() calls. Without registration, invocations would fail.

Quick Testing with the CLI

For rapid iteration without writing test code, use the run-durable CLI:

npm run run-durable -- path/to/your-function.ts

The CLI runs your function locally and displays an operations table automatically. Useful options:

no-skip-time - Actually wait for delays (default skips time)
verbose - Show detailed execution logs
show-history - Display history events table

# Run with verbose logging and history
npm run run-durable -- src/order-processor.ts verbose show-history

The CLI is perfect for quick debugging and verifying function behavior without setting up a full test suite.

Debugging with Execution History

When tests fail, the execution history helps you understand what happened:

it('should provide detailed execution history', async () => {
  const runner = new LocalDurableTestRunner({
    handlerFunction: handler,
  });

  const execution = await runner.run();

  // Print operations table to console
  execution.print();

  // Get detailed history events
  const history = execution.getHistoryEvents();
  console.log('History:', JSON.stringify(history, null, 2));

  // Get all invocations
  const invocations = execution.getInvocations();
  console.log('Invocations:', invocations.length);

  // Filter operations by status
  const succeeded = execution.getOperations({ status: 'SUCCEEDED' });
  const failed = execution.getOperations({ status: 'FAILED' });
});

The print() method outputs a formatted table of all operations. The history events show every checkpoint, invocation, and state transition. You can filter operations by status to focus on specific outcomes.

Additional operation access methods:

getOperationByNameAndIndex(name, index) - Get specific occurrence of a named operation
getOperationById(id) - Get operation by its unique ID

Testing Without Time Skipping

Sometimes you want to test actual timing behavior:

it('should respect actual wait durations', async () => {
  // Setup without skipTime
  await LocalDurableTestRunner.setupTestEnvironment({ skipTime: false });

  const runner = new LocalDurableTestRunner({
    handlerFunction: handler,
  });

  const startTime = Date.now();
  await runner.run();
  const duration = Date.now() - startTime;

  // Should have actually waited ~2 seconds
  expect(duration).toBeGreaterThanOrEqual(2000);

  await LocalDurableTestRunner.teardownTestEnvironment();
});

Without skipTime, waits and retries happen in real time. Use this sparingly - most tests should skip time for speed.

Cloud Testing

For integration tests against deployed functions:

import { CloudDurableTestRunner } from '@aws/durable-execution-sdk-js-testing';

describe('Deployed Function Integration Tests', () => {
  it('should execute in AWS', async () => {
    const runner = new CloudDurableTestRunner({
      functionName: 'OrderProcessorFunction:$LATEST',
      region: 'us-east-1',
    });

    const execution = await runner.run({ payload: {} });

    expect(execution.getStatus()).toBe('SUCCEEDED');
  }, 600000); // 10 minute timeout for cloud tests
});

Cloud testing invokes your actual Lambda function and waits for completion. Important notes:

Qualifier required: You must specify a qualifier (:$LATEST, :prod, etc.) when invoking durable functions
Execution timeout constraint: For synchronous invocation, the function's ExecutionTimeout in DurableConfig must not exceed 900 seconds (15 minutes)
Test timeout: Set a Jest timeout that accounts for your workflow duration - cloud tests run in real time
Real waits: Unlike local tests, cloud tests actually wait for the specified durations (e.g., 5 minutes for a 300-second wait)
AWS credentials: Ensure AWS credentials are configured. You can set the AWS_PROFILE environment variable or use other AWS credential providers

Cloud testing is slower than local testing but verifies behavior in the real AWS environment with actual checkpointing and state management.

Best Practices

Use local testing for most tests. It's fast, doesn't require AWS credentials, and gives you full control over timing and callbacks.

Skip time by default. Set skipTime: true in setupTestEnvironment() unless you specifically need to test timing behavior.

Mock external dependencies. Use Jest mocks for API calls, database operations, and other external services. This keeps tests fast and deterministic.

Test failure scenarios. Don't just test the happy path. Verify retry behavior, timeout handling, and error propagation.

Inspect operations, not just results. Use getOperation() and getOperations() to verify that your function executed the right steps in the right order.

Register nested functions. If your function uses context.invoke(), register the child functions with registerDurableFunction().

Use cloud testing sparingly. Reserve CloudDurableTestRunner for integration tests that need to verify behavior in AWS.

Common Patterns

Testing with dynamic operation names:

it('should handle dynamic operation names', async () => {
  const runner = new LocalDurableTestRunner({
    handlerFunction: handler,
  });

  const execution = await runner.run({ payload: { userId: '123' } });

  // Get operation by name pattern
  const userOp = runner.getOperation('process-user-123');
  expect(userOp.getStatus()).toBe('SUCCEEDED');
});

Testing error propagation:

it('should propagate errors correctly', async () => {
  const runner = new LocalDurableTestRunner({
    handlerFunction: handler,
  });

  const execution = await runner.run();

  expect(execution.getStatus()).toBe('FAILED');
  const error = execution.getError();
  expect(error?.errorType).toBe('StepError');
  expect(error?.errorMessage).toContain('expected error message');
});

Testing with multiple callbacks:

it('should handle multiple callbacks', async () => {
  const runner = new LocalDurableTestRunner({
    handlerFunction: handler,
  });

  const execution = runner.run();

  const callback1 = runner.getOperation('callback-1');
  const callback2 = runner.getOperation('callback-2');

  await callback1.sendCallback({ data: 'first' });
  await callback2.sendCallback({ data: 'second' });

  const result = await execution;
  expect(result.getStatus()).toBe('SUCCEEDED');
});

Summary

The Durable Execution testing library makes it easy to test long-running workflows. Run functions locally with simulated checkpoints, skip time-based operations for fast tests, inspect every operation, and verify behavior without deploying to AWS.

Use LocalDurableTestRunner for fast unit tests with full control over timing and callbacks. Use CloudDurableTestRunner for integration tests against deployed functions. Mock external dependencies, test failure scenarios, and inspect operations to verify correct behavior.

With these tools, testing durable functions is as straightforward as testing any other code. Write tests first, iterate quickly, and ship workflows with confidence.

DEV Community: Eric D Johnson

Is this code deterministic?

Problem 1: Equal-priority ordering is under-specified

What’s happening

Why it still matters (and why this one is subtle)

Why not just wrap the ordering in a step?

Fix

Problem 2: Date.now() is non-deterministic

What’s happening

Why it matters

Fix

Problem 3: Side effects hidden inside a single step

What’s happening

Why it matters

Fix

Step semantics and retry intent

Putting it all together

Durable-safe handler (step by step)

Durable-safe handler using context.map()

Final takeaway

AWS Lambda Durable Functions: Build Workflows That Last

What Are Durable Functions?

A Simple Example

Key Capabilities

How It Works: The Replay Model

Common Use Cases

Testing Your Workflows

Deploying with AWS SAM

When to Use Durable Functions

Getting Started

Learn More

Summary

Developing AWS Lambda Durable Functions with AWS SAM

Prerequisites

The Basics: Enabling Durable Functions

Installing the SDK

Build Configuration

IAM Permissions

Using Globals for Common Configuration

A Complete Example

Building and Deploying

Local Testing

Accessing Cloud Resources Locally

Tracking Execution State

Remote Testing

Configuration Best Practices

Monitoring and Observability

Common Patterns

Troubleshooting

Summary

The Replay Model: How AWS Lambda Durable Functions Actually Work

The Core Principle

Execution Modes: The Secret Sauce

How Checkpoints Work

The Determinism Requirement

What Breaks Determinism

How to Write Deterministic Code

Replay Consistency Validation

A Complete Example: Order Processing with Replay

Invocation Timeline

Operation IDs: The Replay Index

Common Pitfalls and Solutions

1. Non-Deterministic Control Flow

2. Mutating Closure Variables

3. Side Effects Outside Steps

Debugging Replay Issues

Best Practices for Replay-Safe Code

Summary

Testing AWS Lambda Durable Functions in TypeScript

The Testing Library

Your First Test

Setup and Teardown

Inspecting Operations

Testing Failures and Retries

Testing Callbacks

Testing Parallel Operations

Testing Nested Functions

Quick Testing with the CLI

Debugging with Execution History

Testing Without Time Skipping

Problem 2: `Date.now()` is non-deterministic

Durable-safe handler using `context.map()`