DEV Community: Ali Shaikh

Meet EkkoJS: The JavaScript Runtime That Doesn't Apologise for Starting Fresh

Ali Shaikh — Wed, 17 Jun 2026 06:29:01 +0000

There's a moment in every developer's day where they type npm install and watch hundreds of packages materialise from the internet. You didn't ask for most of them. A few are transitive dependencies you'll never audit. One or two probably have CVEs filed against them right now. And somehow, this became normal.

EkkoJS is a bet that it doesn't have to be.

Built by the team at Ampla Network and open-sourced under MIT, EkkoJS is a JavaScript and TypeScript runtime that trades backward compatibility for a cleaner slate. No CommonJS. No node_modules. No URL imports. Just ES modules, TypeScript that runs without a build step, security baked in at the permission level, and a standard library big enough that you won't spend the first hour of a new project hunting for packages.

It's v0.8.3 and labelled a Technology Preview as of June 2026. That means it's not ready for production systems yet. But it's already interesting enough to warrant a proper look.

What's actually running your code

The architecture is the first thing that sets EkkoJS apart. Instead of sitting on top of a single runtime engine, it layers three technologies together.

At the outermost layer is a Rust host running on the Tokio async loop. Rust handles the CLI, the process lifecycle, the permission enforcement, and the FFI bridge. It's fast and has no garbage collector, which keeps latency predictable. Inside that, V8 executes your JavaScript (and your TypeScript, after SWC transpiles it in-process). Below V8, native operations (file I/O, cryptography, HTTP) cross the FFI boundary into a .NET 10 AOT-compiled layer. Ahead-of-time compilation means that layer is native machine code, not JIT-compiled at startup.

The result: your script starts quickly, and the path from your TypeScript source to native system calls is about as short as it can get without writing Rust directly.

Here's the thing: you don't feel any of this complexity as a user. You just point ekko at a TypeScript file and it runs.

The philosophy behind it

Francois, the creator of EkkoJS, has been writing JavaScript since 2001. In his own words from the EkkoJS site: "I never felt fully comfortable with the direction JavaScript runtimes took. I kept dreaming of something different."

That frustration shapes every design decision in the runtime. Three stand out.

ESM only, local only. Every import in an EkkoJS program resolves to either a local file or a declared package. URL imports (the kind you might have seen in Deno or CDN-style scripts) are a compile-time error. This isn't just aesthetic; it closes a whole category of supply chain risk. An attacker can't slip a malicious CDN URL into your dependency graph if the runtime refuses to fetch code from the internet at import time.

Permissions as a first-class concern. Programs start with zero access to the file system, network, or environment variables. You grant what you need at the command line. Running a server that reads one database file looks like this:

ekko run --allow=net,fs:./app.db server.ts

If the server tries to read /etc/passwd or make a request to an unexpected host, it throws. The attack surface shrinks to exactly what you declared.

TypeScript without the ceremony. No tsconfig.json, no separate compile step, no output directory to manage. SWC transpiles inline, the V8 isolate picks it up, and your .ts file just runs. For small scripts and prototypes this alone saves a disproportionate amount of setup friction.

Batteries genuinely included

Over thirty modules live under the ekko: namespace. HTTP server and client. A SQL database with an ORM that supports PostgreSQL, MySQL, MSSQL, and MongoDB. Authentication with RBAC, JWT, OAuth, and TOTP. GraphQL. WebSockets. Cron and job queues. Image processing. Compression. A full-stack React framework called Rune. Even desktop and TUI app toolkits.

The HTTP server, for instance, is backed by ASP.NET Kestrel (that .NET AOT layer doing work) and comes with Express-style routing, rate limiting, CORS, and security headers built in:

import { createServer, cors, helmet, rateLimit } from "ekko:web";

const server = createServer({ port: 3000, compression: true });

server.use(cors());
server.use(helmet());
server.use(rateLimit({ max: 100, window: 60_000 }));

server.get("/users/:id", (req, res) => {
  res.json({ userId: req.params.id });
});

server.start();

No Express install. No Helmet install. No cors package. All of it ships with the runtime.

How does it perform?

Honestly, it's competitive but not dominant. And the team is transparent about that. In their own benchmark table (run on an AMD EPYC 7451, 48 threads):

EkkoJS finishes second overall across 35 benchmarks, behind Bun but ahead of Node.js and Deno
It wins the concurrency and threading category outright , with thread spawn overhead of 265/s versus Node.js at 23/s and Bun at 103/s
8-way parallel scaling hits 638 ops/s against Node.js at 532 and Deno at 202
File read performance beats Node.js for binary reads, and Deno across the board
Bun leads raw single-threaded throughput, especially on maths and regex

The honest note at the bottom of the benchmarks page: "EkkoJS is a Technology Preview. We do not pretend to compete yet with Node.js, Bun, or Deno, which are production-grade, stable, and have been battle-tested for years. The comparison is a compass for our own progress."

That kind of candour is rare and worth noting.

A real example: talking to S3 via LocalStack

Here's where things get practical. Let's build a small script that uses EkkoJS to interact with an S3 bucket through LocalStack, the local AWS emulator. This demo creates a bucket, uploads a JSON document, then retrieves and logs it.

EkkoJS's ekko:crypto module provides HMAC-SHA256, which is all you need for AWS Signature Version 4. No AWS SDK. No node_modules. Just built-in modules and a handful of lines of TypeScript.

Prerequisites: Docker running with LocalStack (docker run -p 4566:4566 localstack/localstack), and EkkoJS installed. If LocalStack is new to you, I covered it in depth across a 9-part build series. Part 1 starts with S3 and presigned URLs, which maps nicely to what we're doing here.

`s3-demo.ts`

import { fetch } from "ekko:web";
import { hmac, hash } from "ekko:crypto";

// LocalStack defaults. Swap these for real AWS credentials and endpoint in prod.
const REGION = "us-east-1";
const ENDPOINT = "http://localhost:4566";
const ACCESS_KEY = "test";
const SECRET_KEY = "test";
const BUCKET = "ekko-demo";

// ─── AWS Signature V4 helpers ─────────────────────────────────────────────────

function toHex(bytes: Uint8Array): string {
  return Array.from(bytes)
    .map((b) => b.toString(16).padStart(2, "0"))
    .join("");
}

async function sha256Hex(data: string): Promise<string> {
  return toHex(await hash("sha256", data));
}

async function hmacSha256(key: Uint8Array | string, data: string): Promise<Uint8Array> {
  return hmac("sha256", key, data);
}

async function signedHeaders(
  method: string,
  path: string,
  service: string,
  body: string,
  extraHeaders: Record<string, string> = {}
): Promise<Record<string, string>> {
  const now = new Date();
  const date = now.toISOString().slice(0, 10).replace(/-/g, ""); // YYYYMMDD
  const datetime = now.toISOString().replace(/[:-]|\.\d+/g, "").slice(0, 15) + "Z";

  const payloadHash = await sha256Hex(body);
  const host = ENDPOINT.replace(/^https?:\/\//, "").split("/")[0];

  const headers: Record<string, string> = {
    host,
    "x-amz-date": datetime,
    "x-amz-content-sha256": payloadHash,
    ...extraHeaders,
  };

  const signedHeaderNames = Object.keys(headers).sort().join(";");
  const canonicalHeaders = Object.keys(headers)
    .sort()
    .map((k) => `${k}:${headers[k]}`)
    .join("\n") + "\n";

  const canonicalRequest = [
    method,
    path,
    "",
    canonicalHeaders,
    signedHeaderNames,
    payloadHash,
  ].join("\n");

  const credentialScope = `${date}/${REGION}/${service}/aws4_request`;
  const stringToSign = [
    "AWS4-HMAC-SHA256",
    datetime,
    credentialScope,
    await sha256Hex(canonicalRequest),
  ].join("\n");

  const signingKey = await (async () => {
    const k1 = await hmacSha256(`AWS4${SECRET_KEY}`, date);
    const k2 = await hmacSha256(k1, REGION);
    const k3 = await hmacSha256(k2, service);
    return hmacSha256(k3, "aws4_request");
  })();

  const signature = toHex(await hmacSha256(signingKey, stringToSign));

  return {
    ...headers,
    Authorization: [
      `AWS4-HMAC-SHA256 Credential=${ACCESS_KEY}/${credentialScope}`,
      `SignedHeaders=${signedHeaderNames}`,
      `Signature=${signature}`,
    ].join(", "),
  };
}

// ─── S3 operations ────────────────────────────────────────────────────────────

async function createBucket(bucket: string): Promise<void> {
  const path = `/${bucket}`;
  const headers = await signedHeaders("PUT", path, "s3", "");
  const res = await fetch(`${ENDPOINT}${path}`, { method: "PUT", headers });
  console.log(`Create bucket: ${res.status}`);
}

async function putObject(bucket: string, key: string, body: string): Promise<void> {
  const path = `/${bucket}/${key}`;
  const headers = await signedHeaders("PUT", path, "s3", body, {
    "content-type": "application/json",
  });
  const res = await fetch(`${ENDPOINT}${path}`, {
    method: "PUT",
    headers,
    body,
  });
  console.log(`Upload "${key}": ${res.status}`);
}

async function getObject(bucket: string, key: string): Promise<string> {
  const path = `/${bucket}/${key}`;
  const headers = await signedHeaders("GET", path, "s3", "");
  const res = await fetch(`${ENDPOINT}${path}`, { method: "GET", headers });
  return res.text();
}

// ─── Run ──────────────────────────────────────────────────────────────────────

await createBucket(BUCKET);

const payload = JSON.stringify({ runtime: "EkkoJS", version: "0.8.3", hello: "world" });
await putObject(BUCKET, "hello.json", payload);

const retrieved = await getObject(BUCKET, "hello.json");
console.log("Retrieved:", retrieved);

Run it with:

ekko run --allow=crypto,net s3-demo.ts

That --allow=crypto,net is EkkoJS's permission model at work: crypto gates the cryptography module, and net grants outbound network access. What's worth noting is how you discover these flags. The runtime tells you exactly what's missing. Forget --allow=crypto and you get PermissionError: crypto access denied. Run with --allow=crypto. Add it and forget --allow=net and you get PermissionError: net access denied for 'http://localhost:4566/...'. Each error names the flag and the resource it blocked. You build up the permission set incrementally, with no guessing. The script simply cannot reach anything outside what you declared.

The output:

Create bucket: 200
Upload "hello.json": 200
Retrieved: {"runtime":"EkkoJS","version":"0.8.3","hello":"world"}

Ekko Terminal Commands

No packages installed. No package.json. No build output folder. Just a TypeScript file and a runtime.

S3 Bucket view in Cloud Sprocket

The wider ecosystem

EkkoJS doesn't stand alone. Three companion projects round it out:

Rune (rune.ekkojs.com) is the full-stack React framework built into the runtime. Server rendering by default, client-side navigation after the first load, and Mimir for state that survives a full browser refresh.

Bifrost (bifrost.ekkojs.com) is the package registry. Because EkkoJS refuses URL imports, it needs its own distribution channel. Bifrost is that channel. Think npm, but scoped to EkkoJS-compatible ESM packages.

Asgard (asgard.ekkojs.com) is the official component suite for Rune — 30+ accessible, themeable UI components that are server-rendered out of the box. Add it to any EkkoJS project with ekko add @ekko/asgard, wrap your app in a ThemeProvider, and you have a full component library without touching npm.

Should you use it now?

Probably not in production. The team is explicit about that: v0.8.x is a Technology Preview, and breaking changes are expected before the first production release later in 2026.

But for experimenting? For internal tooling where you control the environment? For understanding where JavaScript runtimes might go? Yes, absolutely. EkkoJS has ideas worth spending time with.

The permission model is something I wish Node.js had shipped with. The ESM-only stance closes supply chain vectors that the rest of the ecosystem still works around. The .NET AOT layer for native operations is an unusual and clever choice. And the honest benchmark page, including the numbers EkkoJS doesn't win, is a refreshing signal about the team behind it.

It's early. The echo is just starting to resonate.

EkkoJS is open source under the MIT licence. Find the runtime at ekkojs.com and the source at github.com/e-mc2-dev/ekkojs.

Run AWS on Your Laptop: A 9-Part LocalStack Build Series (Part 5 - SQS, SNS, and a Dead-Letter Queue)

Ali Shaikh — Tue, 26 May 2026 19:41:38 +0000

Every real app eventually needs background jobs: sending welcome emails, processing notifications, retrying webhooks. We'll build the canonical AWS pattern on LocalStack: an SNS topic for fan-out, an SQS queue for the job, a dead-letter queue for poison messages, and a redrive policy that ties them together.

What you'll need: Part 0 setup, Python 3.x with boto3, and 30 minutes.

What we're building

The three components work together like this:

producer (CLI/app)
        │
        ▼
   SNS topic ─── fan-out ───┬───▶ SQS welcome-emails queue
                            │ │
                            │ ▼
                            │ consumer (poll → process → delete)
                            │ │
                            │ │ fails 3 times
                            │ ▼
                            │ SQS welcome-emails-dlq
                            │
                            └───▶ (other subscribers - analytics, audit, etc.)

The shape that maps onto pretty much every real app:

SNS topic for fan-out - many subscribers can listen to the same event without the producer caring.
SQS queue for the job - durable, at-least-once delivery with retries and a DLQ.
Dead-letter queue for poison messages - anything that fails repeatedly lands here for inspection rather than blocking the queue forever.

In a real app, the producer might be your signup API publishing a user.created event, or an image-processing pipeline publishing photo.ready. The consumer might send a welcome email, create a CRM contact, fire a webhook to another system, or queue a follow-up job for indexing and search. We'll build the polling worker because it's the easiest way to watch the full retry and dead-letter flow without extra moving parts.

Why SNS and SQS, not just one?

Quick answer if you've not used either at scale:

SQS alone works fine for one producer talking to one consumer. The moment you want a second consumer (analytics, audit log, notifications system) listening to the same events, you'd have to fan out manually or put a copy on each queue.
SNS alone fans out messages to subscribers, but it doesn't give you queue buffering or consumer-side retry behaviour.
SNS → SQS → consumer is the combination. SNS handles fan-out, SQS gives you durability and retries, the DLQ catches what slips through.

It's three pieces of infrastructure and most beginners skip straight to "I'll just call my consumer directly". Six months later, that's usually the choice they wish they hadn't made.

Step 1: Project folder

cd ~/projects/localstack-series
mkdir part5-queues
cd part5-queues

Command style in this part

This part uses awslocal in the main examples because it's shorter. If you prefer the plain AWS CLI, use the same command with aws --endpoint-url=http://localhost:4566 ... instead.

# awslocal
awslocal sqs list-queues

# aws cli
aws --endpoint-url=http://localhost:4566 sqs list-queues

If your LocalStack box lives on another IP, swap localhost for that host in the --endpoint-url value.

Step 2: Create the dead-letter queue first

You need the DLQ's ARN before you can create the main queue (the redrive policy references it).

awslocal sqs create-queue --queue-name welcome-emails-dlq

DLQ_URL=$(awslocal sqs get-queue-url --queue-name welcome-emails-dlq \
  --query QueueUrl --output text)

DLQ_ARN=$(awslocal sqs get-queue-attributes --queue-url $DLQ_URL \
  --attribute-names QueueArn --query 'Attributes.QueueArn' --output text)

echo $DLQ_ARN
# arn:aws:sqs:us-east-1:000000000000:welcome-emails-dlq

Step 3: Create the main queue with a redrive policy

awslocal sqs create-queue --queue-name welcome-emails \
  --attributes "{\"RedrivePolicy\":\"{\\\"deadLetterTargetArn\\\":\\\"$DLQ_ARN\\\",\\\"maxReceiveCount\\\":\\\"3\\\"}\",\"VisibilityTimeout\":\"5\"}"

QUEUE_URL=$(awslocal sqs get-queue-url --queue-name welcome-emails \
  --query QueueUrl --output text)

QUEUE_ARN=$(awslocal sqs get-queue-attributes --queue-url $QUEUE_URL \
  --attribute-names QueueArn --query 'Attributes.QueueArn' --output text)

Two attributes worth understanding:

maxReceiveCount: 3 - after three receives without a successful delete, the message gets moved to the DLQ. The retry budget. Three is a sane default; raise it if your downstream is flaky.
VisibilityTimeout: 5 seconds - when a consumer receives a message, it becomes invisible to other consumers for this long. If the consumer doesn't delete the message in that window, it becomes eligible to be received again. We're setting it short for testing; in production it should be comfortably longer than your longest expected processing time.

The escaping on that JSON-inside-JSON --attributes parameter is annoying - it's the AWS CLI's way of life. A cleaner alternative is --attributes file://attributes.json with a properly-quoted file.

Step 4: Create the SNS topic and subscribe the queue

TOPIC_ARN=$(awslocal sns create-topic --name user-events \
  --query 'TopicArn' --output text)

SUB_ARN=$(awslocal sns subscribe \
  --topic-arn $TOPIC_ARN \
  --protocol sqs \
  --notification-endpoint $QUEUE_ARN \
  --query 'SubscriptionArn' --output text)

Set raw message delivery so SQS receives just the message body, not the full SNS envelope:

awslocal sns set-subscription-attributes \
  --subscription-arn $SUB_ARN \
  --attribute-name RawMessageDelivery \
  --attribute-value true

Without RawMessageDelivery=true, the consumer would have to parse SNS's JSON wrapper ({ "Type": "Notification", "MessageId": "...", "Message": "<your actual message>" }) on every receive. With it on, the consumer just gets your message body. Almost always what you want.

Step 5: The consumer

Create consumer.py:

The key behaviour to notice before you read the code: sqs.delete_message(...) is only called on success. If the consumer raises, no delete happens, the visibility timeout expires, the message becomes eligible to be received again, and the receive count ticks up. After three receives without a successful delete, SQS moves it to the DLQ.

import json
import os
import sys

import boto3

ENDPOINT = os.environ.get("AWS_ENDPOINT_URL") or "http://localhost:4566"
QUEUE_URL = os.environ["QUEUE_URL"]

sqs = boto3.client("sqs", endpoint_url=ENDPOINT, region_name="us-east-1")

def process(message_body):
    payload = json.loads(message_body)
    if payload.get("email") == "fail@example.com":
        raise RuntimeError("Simulated downstream failure")
    print(f" ✓ Sent welcome email to {payload['email']} (user_id={payload.get('user_id')})")

def poll_once(max_messages=10, wait_seconds=2):
    out = sqs.receive_message(
        QueueUrl=QUEUE_URL,
        MaxNumberOfMessages=max_messages,
        WaitTimeSeconds=wait_seconds,
    )
    messages = out.get("Messages", [])
    print(f"Polled - {len(messages)} message(s)")

    for msg in messages:
        try:
            process(msg["Body"])
        except Exception as e:
            print(f" ✗ {e} - leaving message in queue for retry")
            continue
        sqs.delete_message(QueueUrl=QUEUE_URL, ReceiptHandle=msg["ReceiptHandle"])

if __name__ == " __main__":
    poll_once(wait_seconds=int(sys.argv[1]) if len(sys.argv) > 1 else 2)

WaitTimeSeconds enables long polling - the consumer waits up to N seconds for a message rather than returning empty immediately. Long polling drops API costs and improves latency. Use it.

If your LocalStack box lives on another IP, export AWS_ENDPOINT_URL=http://<that-ip>:4566 before running the consumer. The default localhost:4566 is correct only when the worker and LocalStack are on the same machine.

Step 6: Test the happy path and the poison path together

Publish three messages - two clean, one deliberately failing:

awslocal sns publish --topic-arn $TOPIC_ARN \
  --message '{"user_id":"u-1","email":"alice@example.com"}'

awslocal sns publish --topic-arn $TOPIC_ARN \
  --message '{"user_id":"u-2","email":"fail@example.com"}'

awslocal sns publish --topic-arn $TOPIC_ARN \
  --message '{"user_id":"u-3","email":"bob@example.com"}'

Run the consumer four times, with a 6-second pause between each (long enough for the visibility timeout to release the poison message back into the queue):

QUEUE_URL=$QUEUE_URL python3 consumer.py
# Pass 1
Polled - 3 message(s)
  ✓ Sent welcome email to alice@example.com (user_id=u-1)
  ✗ Simulated downstream failure - leaving message in queue for retry
  ✓ Sent welcome email to bob@example.com (user_id=u-3)

sleep 6 && QUEUE_URL=$QUEUE_URL python3 consumer.py
# Pass 2
Polled - 1 message(s)
  ✗ Simulated downstream failure - leaving message in queue for retry

sleep 6 && QUEUE_URL=$QUEUE_URL python3 consumer.py
# Pass 3
Polled - 1 message(s)
  ✗ Simulated downstream failure - leaving message in queue for retry

sleep 6 && QUEUE_URL=$QUEUE_URL python3 consumer.py
# Pass 4
Polled - 0 message(s)

The poison message has been moved to the DLQ:

awslocal sqs receive-message --queue-url $DLQ_URL \
  --max-number-of-messages 5 --wait-time-seconds 2 \
  --query 'Messages[*].Body' --output text
# {"user_id":"u-2","email":"fail@example.com"}

There it is. The two well-formed messages drained on the first pass. The poison message reached maxReceiveCount=3 without a successful delete, disappeared from the main queue after the third failed receive, and showed up in the DLQ by the time the fourth poll ran.

What to actually do with messages in the DLQ

Don't just leave them there forever. The DLQ is an inbox, not a graveyard. Two patterns worth knowing:

Inspect and replay. Pull the messages, fix whatever broke (a bug in the consumer, a third-party outage), then either re-publish them to the topic or move them back to the main queue with aws sqs start-message-move-task. That's the normal replay API on AWS. On the 2026-05-26 homelab LocalStack rerun, the command completed but did not actually move the message, so re-publishing to the topic is the reliable local fallback there.
Alarm on depth. A non-zero DLQ depth means someone needs to look at this. Wire a CloudWatch alarm on ApproximateNumberOfMessagesVisible and route it to PagerDuty / Opsgenie / your phone. We'll touch alarms in a later article.

Wiring a Lambda consumer instead (the production pattern)

What we built above polls in a loop, which is perfect for learning and fine for a small service. In production, most teams use a Lambda triggered by SQS via an "event source mapping". Same redrive behaviour, no Python worker to keep alive.

# (Build a Lambda using the deployment flow from [Part 3](https://alishaikh.me/run-aws-on-your-laptop-a-9-part-localstack-build-series-part-3-lambda-s3-thumbnailer-pipeline/) or [Part 4](https://alishaikh.me/run-aws-on-your-laptop-a-9-part-localstack-build-series-part-4-api-gateway-lambda-and-jwt-auth/), then:)
awslocal lambda create-event-source-mapping \
  --function-name welcome-mailer \
  --event-source-arn $QUEUE_ARN \
  --batch-size 10 \
  --maximum-batching-window-in-seconds 5

Now Lambda polls SQS, invokes your function with batches of up to 10 messages, and the same queue visibility timeout and DLQ policy still apply. Failures bubble up the same way.

The polling consumer is the right thing to build first because every step is observable. The Lambda version is the right thing to ship.

Common pitfalls

Message lands in the queue but the consumer never sees it. You forgot RawMessageDelivery=true on the subscription, and your json.loads(message_body) is choking on the SNS envelope. Either set the flag or json.loads(json.loads(body)["Message"]).
Poison message never goes to the DLQ. maxReceiveCount is too high, or the consumer is silently swallowing the exception and deleting the message. Print failures and don't catch broadly.
Messages that should be dead-lettered come back forever. The visibility timeout is shorter than your processing time. The consumer is still working when SQS thinks it's hung, so the message comes back. Make VisibilityTimeout comfortably longer than your worst-case job.
InvalidParameterValueException on create-queue. The redrive policy JSON is malformed (the escaping on the CLI is fiddly). Use a --attributes file://attrs.json instead.
Queue policy permission errors when SNS tries to deliver. LocalStack's Hobby tier doesn't enforce IAM strictly, so this is rare on LocalStack but bites in real AWS - set the queue policy to allow sns.amazonaws.com to send.

Cleanup commands worth knowing

# Drain all messages
awslocal sqs purge-queue --queue-url $QUEUE_URL
awslocal sqs purge-queue --queue-url $DLQ_URL

# Remove the queues and topic
awslocal sqs delete-queue --queue-url $QUEUE_URL
awslocal sqs delete-queue --queue-url $DLQ_URL
awslocal sns delete-topic --topic-arn $TOPIC_ARN

If you're going to Part 6, keep the topic. Step Functions can publish to it.

Save this as a checkpoint

The full queue topology, DLQ, main queue with redrive policy, topic, and the raw-delivery subscription, re-creates itself on every container start.

Save as init/ready.d/05-part5-queues.sh:

The names in this script are fixed on purpose. That keeps the bootstrap predictable across restarts and avoids creating a fresh queue/topic pair every time the container comes back up.

#!/usr/bin/env bash
# Part 5 checkpoint - SQS welcome-emails + DLQ + SNS user-events topic
awslocal sqs create-queue --queue-name welcome-emails-dlq 2>/dev/null || true

DLQ_URL=$(awslocal sqs get-queue-url --queue-name welcome-emails-dlq --query QueueUrl --output text 2>/dev/null)
DLQ_ARN=$(awslocal sqs get-queue-attributes --queue-url "$DLQ_URL" \
  --attribute-names QueueArn --query 'Attributes.QueueArn' --output text 2>/dev/null)

awslocal sqs create-queue --queue-name welcome-emails \
  --attributes "{\"RedrivePolicy\":\"{\\\"deadLetterTargetArn\\\":\\\"$DLQ_ARN\\\",\\\"maxReceiveCount\\\":\\\"3\\\"}\",\"VisibilityTimeout\":\"5\"}" 2>/dev/null || true

awslocal sns create-topic --name user-events 2>/dev/null || true

QUEUE_ARN="arn:aws:sqs:us-east-1:000000000000:welcome-emails"
TOPIC_ARN="arn:aws:sns:us-east-1:000000000000:user-events"

# Avoid creating duplicate subscriptions on each restart
EXISTING=$(awslocal sns list-subscriptions-by-topic --topic-arn "$TOPIC_ARN" \
  --query "Subscriptions[?Endpoint=='$QUEUE_ARN'].SubscriptionArn" --output text 2>/dev/null)
if [-z "$EXISTING"] || ["$EXISTING" = "None"]; then
  SUB_ARN=$(awslocal sns subscribe --topic-arn "$TOPIC_ARN" --protocol sqs \
    --notification-endpoint "$QUEUE_ARN" --query 'SubscriptionArn' --output text 2>/dev/null)
  awslocal sns set-subscription-attributes --subscription-arn "$SUB_ARN" \
    --attribute-name RawMessageDelivery --attribute-value true 2>/dev/null || true
fi

echo "[bootstrap] part 5 - queues + topic + subscription ready"


chmod +x init/ready.d/05-part5-queues.sh

Jumping in at Part 5 from scratch? You only need Part 0 setup for the queue topology itself. If you want to test the "producer from the shorten Lambda" story later, keep Part 4 around too.

What we'll wire up next

You've got fan-out, queueing, retries, and dead-lettering - the building blocks of every real async pipeline. The next part graduates from "send one message" to orchestration: an EventBridge rule starts a Step Functions state machine that fans across multiple Lambdas (resize → tag → notify → archive). The kind of workflow that's too gnarly to express in plain queue messaging and is exactly what Step Functions exists for.

The full series

Part 0 - Start here: series intro and installing LocalStack
Part 1 - S3 locally: buckets, presigned URLs, and a tiny photo uploader
Part 2 - DynamoDB locally: building a URL shortener data layer
Part 3 - Lambda + S3 events: an image thumbnailer pipeline
Part 4 - API Gateway + Lambda + JWT auth: a real HTTP API
Part 5 - SQS + SNS: a background job queue with a dead-letter queue (this article)
Part 6 - EventBridge + Step Functions: orchestrating a photo-processing workflow (next)
Part 7 - Secrets Manager + KMS: handling secrets and encryption locally
Part 8 - Terraform (tflocal) + GitHub Actions: integration tests against LocalStack

Sources

Related on alishaikh.me

LocalStack: Run AWS Services Locally for Free

Run AWS on Your Laptop: A 9-Part LocalStack Build Series (Part 4 - API Gateway, Lambda, and JWT Auth)

Ali Shaikh — Sun, 17 May 2026 13:55:09 +0000

The URL shortener from Part 2 - DynamoDB URL Shortener Data Layer needs an HTTP front door. We'll put API Gateway in front of it, back it with two Lambdas, protect POST /shorten with a third Lambda authoriser, and pass the JWT subject all the way through to DynamoDB. Commands and outputs below were verified against a real LocalStack instance.

What you'll need: Part 0 setup and Part 2 finished (we'll reuse the shortlinks table), Python 3.12, Docker, curl, and 60 minutes.

What we're building

                       ┌─────────────────────┐
client (curl/browser) ─▶ POST /dev/shorten │
                       │ with JWT in header │
                       └──────────┬──────────┘
                                  │
                       ┌──────────▼──────────┐
                       │ authorizer Lambda │
                       │ (validates JWT) │
                       └──────────┬──────────┘
                                  │ isAuthorized: true
                       ┌──────────▼──────────┐ ┌────────────────┐
                       │ shorten Lambda │───▶│ DynamoDB │
                       └─────────────────────┘ │ shortlinks │
                                                  │ │
client ──────────────▶ GET /dev/r/{code} ────▶│ redirect │
                                                  │ Lambda │
                                                  └────────────────┘
                              302 redirect ◀──────────┘

Three pieces of code. One HTTP API gluing them together. By the end of this article you can curl -X POST a shortener running on LocalStack from your terminal - same shape as a real production API.

Why HTTP API rather than REST API

API Gateway has two flavours: REST API (v1) and HTTP API (v2). Both work on LocalStack. Use HTTP API unless you have a specific reason not to:

HTTP API has lower latency, much simpler config, and ~70% lower cost in real AWS.
REST API is older and has more features (request validators, SDK generation, usage plans). If you don't need those, skip it.
Lambda authorisers in HTTP API have a "simple response" mode ({ "isAuthorized": true }) that REST API doesn't. Cleaner to write.

Everything in this article uses HTTP API.

Step 1: Project layout

cd ~/projects/localstack-series
mkdir part4-api
cd part4-api
mkdir shorten redirect authorizer

Three folders, one Lambda each. Plus a token-signing helper at the project root.

Step 2: The shorten Lambda

Create shorten/handler.py:

import json
import os
import random
import string

import boto3
from botocore.exceptions import ClientError

ENDPOINT = os.environ.get("AWS_ENDPOINT_URL") or "http://localhost.localstack.cloud:4566"
TABLE = os.environ.get("TABLE", "shortlinks")

ddb = boto3.client("dynamodb", endpoint_url=ENDPOINT, region_name="us-east-1")

def random_code(n=6):
    return "".join(random.choices(string.ascii_lowercase + string.digits, k=n))

def handler(event, context):
    try:
        body = json.loads(event.get("body") or "{}")
    except json.JSONDecodeError:
        return _resp(400, {"error": "invalid JSON body"})

    long_url = body.get("long_url")
    code = body.get("code") or random_code()
    if not long_url:
        return _resp(400, {"error": "long_url is required"})

    user_id = (event.get("requestContext", {}).get("authorizer", {}).get("lambda", {}) or {}).get("userId", "anonymous")

    try:
        ddb.put_item(
            TableName=TABLE,
            Item={
                "code": {"S": code},
                "long_url": {"S": long_url},
                "owner_id": {"S": user_id},
                "click_count": {"N": "0"},
            },
            ConditionExpression="attribute_not_exists(code)",
        )
    except ClientError as e:
        if e.response["Error"]["Code"] == "ConditionalCheckFailedException":
            return _resp(409, {"error": f"code '{code}' already taken"})
        raise

    return _resp(201, {"code": code, "long_url": long_url, "owner_id": user_id})

def _resp(status, body):
    return {
        "statusCode": status,
        "headers": {"Content-Type": "application/json"},
        "body": json.dumps(body),
    }

The line worth noticing: event["requestContext"]["authorizer"]["lambda"]["userId"]. That's where the authoriser's context payload lands. Whatever the authoriser puts in context shows up here, available to your handler. We'll use this to record which user owns each shortlink.

Step 3: The redirect Lambda

Create redirect/handler.py:

import json
import os

import boto3
from botocore.exceptions import ClientError

ENDPOINT = os.environ.get("AWS_ENDPOINT_URL") or "http://localhost.localstack.cloud:4566"
TABLE = os.environ.get("TABLE", "shortlinks")

ddb = boto3.client("dynamodb", endpoint_url=ENDPOINT, region_name="us-east-1")

def handler(event, context):
    code = event.get("pathParameters", {}).get("code")
    if not code:
        return {"statusCode": 400, "body": json.dumps({"error": "missing code"})}

    try:
        out = ddb.update_item(
            TableName=TABLE,
            Key={"code": {"S": code}},
            UpdateExpression="SET click_count = if_not_exists(click_count, :zero) + :one",
            ConditionExpression="attribute_exists(code)",
            ExpressionAttributeValues={":one": {"N": "1"}, ":zero": {"N": "0"}},
            ReturnValues="ALL_NEW",
        )
    except ClientError as e:
        if e.response["Error"]["Code"] == "ConditionalCheckFailedException":
            return {
                "statusCode": 404,
                "headers": {"Content-Type": "application/json"},
                "body": json.dumps({"error": "not found"}),
            }
        raise

    return {
        "statusCode": 302,
        "headers": {"Location": out["Attributes"]["long_url"]["S"]},
        "body": "",
    }

statusCode: 302 plus a Location header is all it takes for an API Gateway response to be a redirect. The browser follows it transparently. The ConditionExpression="attribute_exists(code)" matters too: UpdateItem can create an item if you leave it unguarded, so this keeps a missing short code from turning into a ghost row with a counter but no URL.

Step 4: The JWT authoriser Lambda

Create authorizer/handler.py:

import base64
import hashlib
import hmac
import json
import os
import time

JWT_SECRET = os.environ.get("JWT_SECRET", "dev-secret-change-me")

def handler(event, context):
    token = _extract_token(event)
    payload = _verify(token, JWT_SECRET) if token else None

    if not payload:
        return {"isAuthorized": False}

    return {
        "isAuthorized": True,
        "context": {"userId": payload.get("sub", "unknown")},
    }

def _extract_token(event):
    headers = event.get("headers") or {}
    auth = headers.get("authorization") or headers.get("Authorization") or ""
    if auth.lower().startswith("bearer "):
        return auth.split(None, 1)[1].strip()
    return None

def _verify(token, secret):
    parts = token.split(".")
    if len(parts) != 3:
        return None
    header_b64, payload_b64, sig_b64 = parts

    expected = _sign(f"{header_b64}.{payload_b64}", secret)
    if not hmac.compare_digest(expected, sig_b64):
        return None

    try:
        payload = json.loads(_b64url_decode(payload_b64))
    except (ValueError, json.JSONDecodeError):
        return None

    if "exp" in payload and time.time() > payload["exp"]:
        return None

    return payload

def _sign(data, secret):
    sig = hmac.new(secret.encode(), data.encode(), hashlib.sha256).digest()
    return _b64url_encode(sig)

def _b64url_encode(b):
    return base64.urlsafe_b64encode(b).decode().rstrip("=")

def _b64url_decode(s):
    pad = "=" * ((4 - len(s) % 4) % 4)
    return base64.urlsafe_b64decode(s + pad)

This is HS256 JWT verification in pure stdlib - no pyjwt dependency, no zip bundling, the whole authoriser is one file. The shape of the response is what HTTP API expects: { "isAuthorized": true|false, "context": {...} }. The context object becomes available to downstream Lambdas via event.requestContext.authorizer.lambda.

One small event-shape detail: HTTP API payload format 2.0 lowercases header names in the Lambda event. Checking both authorization and Authorization is harmlessly defensive here, but authorization is the key you should expect to see.

A note on production: HS256 with a shared secret is fine for learning and internal services. Public APIs typically use RS256 with a JWKS endpoint so you can rotate keys without redeploying. The verification flow looks similar, but you would switch from HMAC to the cryptography library's asymmetric signature verification methods.

Step 5: A token-signing helper

So we can produce tokens for testing. Create sign-token.py at the project root:

import base64, hashlib, hmac, json, sys, time

def b64url(b):
    return base64.urlsafe_b64encode(b).decode().rstrip("=")

def sign(payload, secret):
    header = b64url(json.dumps({"alg": "HS256", "typ": "JWT"}, separators=(",", ":")).encode())
    body = b64url(json.dumps(payload, separators=(",", ":")).encode())
    sig = hmac.new(secret.encode(), f"{header}.{body}".encode(), hashlib.sha256).digest()
    return f"{header}.{body}.{b64url(sig)}"

if __name__ == " __main__":
    user = sys.argv[1] if len(sys.argv) > 1 else "user-1"
    secret = sys.argv[2] if len(sys.argv) > 2 else "dev-secret-change-me"
    print(sign({"sub": user, "exp": int(time.time()) + 3600}, secret))


$ python3 sign-token.py user-1
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ1c2VyLTEi...

That's a valid JWT for user-1, expiring in an hour, signed with the same secret the authoriser will check against.

Step 6: Bundle and deploy the Lambdas

for fn in shorten redirect authorizer; do
  cd $fn && zip -q ../$fn.zip handler.py && cd ..
done

Deploy them:

for fn in shorten redirect; do
  awslocal lambda create-function \
    --function-name $fn \
    --runtime python3.12 \
    --role arn:aws:iam::000000000000:role/lambda-role \
    --handler handler.handler \
    --zip-file fileb://$fn.zip \
    --timeout 10 \
    --environment 'Variables={TABLE=shortlinks}'
done

awslocal lambda create-function \
  --function-name authorizer \
  --runtime python3.12 \
  --role arn:aws:iam::000000000000:role/lambda-role \
  --handler handler.handler \
  --zip-file fileb://authorizer.zip \
  --timeout 10 \
  --environment 'Variables={JWT_SECRET=dev-secret-change-me}'

for fn in shorten redirect authorizer; do
  awslocal lambda wait function-active-v2 --function-name $fn
done

Wait until all three are ACTIVE before going on.

Step 7: Create the HTTP API

API_ID=$(awslocal apigatewayv2 create-api \
  --name shortener --protocol-type HTTP \
  --query 'ApiId' --output text)
echo "API_ID=$API_ID"

Step 8: Lambda integrations

Each route needs a Lambda integration that connects API Gateway to the function:

SHORTEN_INT=$(awslocal apigatewayv2 create-integration \
  --api-id $API_ID --integration-type AWS_PROXY \
  --integration-uri arn:aws:lambda:us-east-1:000000000000:function:shorten \
  --payload-format-version 2.0 \
  --query 'IntegrationId' --output text)

REDIRECT_INT=$(awslocal apigatewayv2 create-integration \
  --api-id $API_ID --integration-type AWS_PROXY \
  --integration-uri arn:aws:lambda:us-east-1:000000000000:function:redirect \
  --payload-format-version 2.0 \
  --query 'IntegrationId' --output text)

AWS_PROXY is the integration type that ships the raw HTTP request to the Lambda and expects an HTTP response back. --payload-format-version 2.0 is what HTTP API uses; REST API still uses 1.0.

Step 9: The Lambda authoriser

AUTH_ID=$(awslocal apigatewayv2 create-authorizer \
  --api-id $API_ID \
  --name jwt-auth \
  --authorizer-type REQUEST \
  --identity-source '$request.header.Authorization' \
  --authorizer-uri arn:aws:apigateway:us-east-1:lambda:path/2015-03-31/functions/arn:aws:lambda:us-east-1:000000000000:function:authorizer/invocations \
  --authorizer-payload-format-version 2.0 \
  --enable-simple-responses \
  --query 'AuthorizerId' --output text)

Three flags worth understanding:

--identity-source '$request.header.Authorization' tells API Gateway which header carries the credential. If that identity source is missing on a request, API Gateway returns 401 Unauthorized instead of invoking the authoriser Lambda.
--authorizer-payload-format-version 2.0 - must match the integration version.
--enable-simple-responses switches to the { isAuthorized: true|false } shape rather than the full IAM policy response. Cleaner.

Step 10: Routes

# Auth-protected
awslocal apigatewayv2 create-route \
  --api-id $API_ID \
  --route-key 'POST /shorten' \
  --target integrations/$SHORTEN_INT \
  --authorization-type CUSTOM \
  --authorizer-id $AUTH_ID

# Public
awslocal apigatewayv2 create-route \
  --api-id $API_ID \
  --route-key 'GET /r/{code}' \
  --target integrations/$REDIRECT_INT

{code} is a path parameter. It arrives at the redirect Lambda as event.pathParameters.code.

Step 11: Stage with auto-deploy

awslocal apigatewayv2 create-stage \
  --api-id $API_ID \
  --stage-name dev \
  --auto-deploy

--auto-deploy means any subsequent route changes go live without an explicit deploy. Convenient for development.

Step 12: Permissions for API Gateway to invoke the Lambdas

for fn in shorten redirect; do
  awslocal lambda add-permission --function-name $fn \
    --statement-id apigw-invoke \
    --action lambda:InvokeFunction \
    --principal apigateway.amazonaws.com \
    --source-arn "arn:aws:execute-api:us-east-1:000000000000:$API_ID/*"
done

awslocal lambda add-permission \
  --function-name authorizer \
  --statement-id apigw-authorizer-invoke \
  --action lambda:InvokeFunction \
  --principal apigateway.amazonaws.com \
  --source-arn "arn:aws:execute-api:us-east-1:000000000000:$API_ID/authorizers/$AUTH_ID"

Without these, API Gateway can't invoke your Lambdas and you get a generic 500 with no useful error. The route handlers can use the broad api-id/* form for a learning setup; the authoriser permission above uses the narrower authorizer ARN shape AWS documents.

Step 13: Test it

The API endpoint URL is http://<API_ID>.execute-api.localhost.localstack.cloud:4566/dev when LocalStack is running on the same machine as your client. If your LocalStack box lives on another IP, send the request to that IP and set the Host header to <API_ID>.execute-api.localhost.localstack.cloud:4566.

Generate a token:

TOKEN=$(python3 sign-token.py user-1)
BASE="http://$API_ID.execute-api.localhost.localstack.cloud:4566/dev"

Test the four scenarios:

# 1. POST /shorten without token (expect 401)
curl -s -i -X POST "$BASE/shorten" \
  -H 'Content-Type: application/json' \
  -d '{"long_url":"https://anthropic.com"}'
# HTTP/1.1 401 Unauthorized
# {"message":"Unauthorized"}

# 2. POST /shorten with valid JWT (expect 201)
curl -s -i -X POST "$BASE/shorten" \
  -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' \
  -d '{"long_url":"https://anthropic.com","code":"anth"}'
# HTTP/1.1 201 Created
# {"code": "anth", "long_url": "https://anthropic.com", "owner_id": "user-1"}

# 3. GET /r/anth (expect 302 redirect)
curl -s -D - -o /dev/null "$BASE/r/anth"
# HTTP/1.1 302 FOUND
# Location: https://anthropic.com

# 4. GET /r/missing (expect 404)
curl -s -i "$BASE/r/missing"
# HTTP/1.1 404 Not Found
# {"error": "not found"}

The interesting bit: owner_id: "user-1" came from the JWT subject claim, flowing through the authoriser context into the shorten Lambda and onto the DynamoDB row. Auth context moved all the way through the request, with no app code involved beyond reading event.requestContext.authorizer.lambda.userId.

Hit the redirect a few more times and check the click counter:

curl -s -o /dev/null "$BASE/r/anth"
curl -s -o /dev/null "$BASE/r/anth"
awslocal dynamodb get-item --table-name shortlinks \
  --key '{"code":{"S":"anth"}}' \
  --query 'Item.click_count.N' --output text
# 3

Atomic increment from Part 2, wired through HTTP API and a Lambda. No race conditions.

Common pitfalls

Every request returns 500 with no useful error. Check that you ran add-permission for each Lambda after creating the API. API Gateway can call IAM-restricted Lambdas only with those statements. On real AWS, the authoriser permission should use the api-id/authorizers/authorizer-id source ARN shape.
The very first request after wiring everything returns 500, then the retry works. Fresh API Gateway resources on LocalStack can take a second to settle after route, stage, and permission creation. Give it a moment and try once more before changing code.
401 on every request, even with a valid token. Confirm the authoriser's JWT_SECRET matches what sign-token.py is using. Print both and eyeball them.
Could not connect to localstack:4566 in Lambda logs. Same hostname gotcha as Part 3. This usually means your code or env var points at localhost:4566 from inside the Lambda container. Use localhost.localstack.cloud:4566 instead.
403 with "Forbidden" body on the authoriser path. The identity-source doesn't match the header your client is sending, or the route is wired to the wrong authoriser. Clients can send either header casing, but the HTTP API event passed to Lambda lowercases header names.
CORS errors when calling from a browser. HTTP API doesn't enable CORS by default. Add --cors-configuration '{"AllowOrigins":["*"],"AllowMethods":["GET","POST"],"AllowHeaders":["*"]}' to your create-api call (or update the API after the fact).

About JWT in production

What we built is a working educational example. To take this to production:

Swap HS256 for RS256. Public APIs sign with a private key and verify with a public key fetched from a JWKS endpoint. That way you can rotate signing keys without distributing a new shared secret.
Validate iss, aud, nbf on top of exp. Issuer, audience, "not before" - all standard JWT claims worth checking.
Use a real IdP like Cognito, Auth0, Clerk, or a self-hosted Keycloak. They handle key rotation, user management, MFA, and password resets so you don't have to.
Cache authoriser results. API Gateway can cache an authoriser response for up to an hour, which slashes Lambda invocations on a busy endpoint. Set --authorizer-result-ttl-in-seconds 300 on the create-authorizer call.

Cognito is a tempting target here, and yes, LocalStack supports it on the Pro tier. We're staying on the simpler stack in this series, but it'll be a future quick-win article.

Cleanup commands worth knowing

# Tear down the API (removes routes, integrations, authorisers, stage)
awslocal apigatewayv2 delete-api --api-id $API_ID

# Remove the Lambdas
for fn in shorten redirect authorizer; do
  awslocal lambda delete-function --function-name $fn
done

Don't actually delete shortlinks from DynamoDB if you're going on to Part 5 - we'll keep it.

Save this as a checkpoint

This part is almost entirely Lambdas and API Gateway resources, both of which depend on .zip files we just built. Trying to bootstrap the API automatically from an init hook ends up more complex than just walking through the article body again - the Lambda zips need to live somewhere the container can read them, the integrations reference Lambda ARNs, and so on.

So Part 4's checkpoint is intentionally a no-op. It exists for completeness and to remind you that this part is rebuilt manually from the article steps above.

Save as init/ready.d/04-part4-api.sh:

#!/usr/bin/env bash
# Part 4 checkpoint - no automated bootstrap.
# The HTTP API, three Lambdas, and authoriser are deployed by the steps in
# Part 4's body. Re-run the build + create-function + create-api flow manually
# after a restart.
echo "[bootstrap] part 4 - manual deployment required (see article)"


chmod +x init/ready.d/04-part4-api.sh

Jumping in at Part 4 from scratch? Make sure 02-part2-dynamodb.sh is in your init/ready.d/ first. Part 4 reads from and updates the shortlinks table from Part 2. Then walk through Steps 2-13 above.

What we'll wire up next

You've got an authenticated HTTP API in front of a real data layer. The next part builds the asynchronous side: an SQS queue with a dead-letter queue, fed by an SNS topic, consumed by a Lambda. The use case is "send a welcome email" / "notify a user that their photo has been processed" - the kind of background job that has no business living on the request path.

The full series

Part 0 - Start here: series intro and installing LocalStack
Part 1 - S3 locally: buckets, presigned URLs, and a tiny photo uploader
Part 2 - DynamoDB locally: building a URL shortener data layer
Part 3 - Lambda + S3 events: an image thumbnailer pipeline
Part 4 - API Gateway + Lambda + JWT auth: a real HTTP API (this article)
Part 5 - SQS + SNS: a background job queue with a dead-letter queue (next)
Part 6 - EventBridge + Step Functions: orchestrating a photo-processing workflow
Part 7 - Secrets Manager + KMS: handling secrets and encryption locally
Part 8 - Terraform (tflocal) + GitHub Actions: integration tests against LocalStack

Sources

Related on alishaikh.me

LocalStack: Run AWS Services Locally for Free

Run AWS on Your Laptop: A 9-Part LocalStack Build Series (Part 3 - Lambda S3 Thumbnailer Pipeline)

Ali Shaikh — Sat, 16 May 2026 15:13:31 +0000

If you've already worked through Part 2 - DynamoDB URL Shortener Data Layer , this is where the series turns event-driven. Upload a photo to S3, a Lambda fires automatically, and a thumbnail appears in another bucket. By the end you'll have a working pipeline you can adapt for any "react to upload" use case - virus scanning, OCR, watermarking, you name it.

What you'll need: Part 0 setup, Part 1 finished (we'll reuse the photos bucket convention), Python 3.12, Docker on the machine running LocalStack (because LocalStack executes Lambda invocations through Docker containers), and 45 minutes.

What we're building

Upload to s3://photos/foo.jpg
        │
        ▼
S3 ObjectCreated event
        │
        ▼
   Lambda fires
        │
        ├── reads s3://photos/foo.jpg
        ├── resizes to 256×N (aspect preserved)
        └── writes s3://thumbnails/thumb-foo.jpg

The same trigger pattern AWS customers use for image-heavy SaaS (think Imgur, photo sharing apps) and for "do something on upload" pipelines (virus scanning, transcoding, redaction). The event pattern carries across directly to AWS, but a production version usually adds stricter IAM, idempotency, input validation, retries, object filtering, and monitoring around the same basic Lambda shape.

Why this is genuinely useful for learning

LocalStack's Lambda runtime spawns a sibling Docker container per invocation, runs your code in it, then tears it down. Same as real AWS conceptually. Same event payload shape. A familiar CloudWatch Logs workflow too, via LocalStack's emulation.

The two things that differ from real AWS - and are worth knowing about up front - are:

You need Docker on the host running LocalStack. Lambda needs the Docker daemon socket. Part 0's compose file mounts /var/run/docker.sock for exactly this.
From inside a Lambda, localhost:4566 doesn't reach LocalStack. Lambdas run in a sibling container, so localhost is the Lambda itself. Use localhost.localstack.cloud:4566 instead. LocalStack documents that this resolves to the LocalStack container for LocalStack-managed compute environments such as Lambda. This is the gotcha that costs everyone an hour the first time.

Step 1: Make the buckets

cd ~/projects/localstack-series
mkdir part3-lambda
cd part3-lambda

awslocal s3 mb s3://photos # if you didn't keep it from Part 1
awslocal s3 mb s3://thumbnails

Two buckets: source uploads land in photos, thumbnails go into thumbnails. Splitting them is the standard pattern - it stops the Lambda triggering recursively on its own output, and it lets you set different lifecycle rules per bucket.

Step 2: Write the Lambda

Create thumbnailer.py:

import io
import os
import urllib.parse

import boto3
from PIL import Image

ENDPOINT = os.environ.get("AWS_ENDPOINT_URL") or "http://localhost.localstack.cloud:4566"
THUMBNAIL_BUCKET = os.environ.get("THUMBNAIL_BUCKET", "thumbnails")
THUMBNAIL_SIZE = (256, 256)

s3 = boto3.client("s3", endpoint_url=ENDPOINT, region_name="us-east-1")

def handler(event, context):
    results = []
    for record in event.get("Records", []):
        bucket = record["s3"]["bucket"]["name"]
        key = urllib.parse.unquote_plus(record["s3"]["object"]["key"])

        obj = s3.get_object(Bucket=bucket, Key=key)
        img = Image.open(io.BytesIO(obj["Body"].read()))
        img.thumbnail(THUMBNAIL_SIZE)

        buf = io.BytesIO()
        fmt = (img.format or "JPEG").upper()
        if fmt == "JPEG" and img.mode in ("RGBA", "P"):
            img = img.convert("RGB")
        img.save(buf, format=fmt)
        buf.seek(0)

        s3.put_object(
            Bucket=THUMBNAIL_BUCKET,
            Key=f"thumb-{key}",
            Body=buf.getvalue(),
            ContentType=obj.get("ContentType", "image/jpeg"),
        )
        results.append({"source": f"{bucket}/{key}", "thumb": f"{THUMBNAIL_BUCKET}/thumb-{key}", "size": img.size})
    return {"thumbnails": results}

A few line-by-line notes:

urllib.parse.unquote_plus - S3 event keys are URL-encoded (a file called my photo.jpg arrives as my+photo.jpg). Decoding is the kind of thing every "first Lambda" gets bitten by.
Image.thumbnail(size) - Pillow's thumbnail resizes in place and preserves aspect ratio. For a 1200×800 source with (256, 256) you get 256×171 out, not 256×256. That's almost always what you want.
The RGBA/P→RGB conversion - JPEG can't store transparency. Without this, PNG-with-alpha sources crash on save. Quietly converting is the simplest fix.
endpoint_url uses localhost.localstack.cloud:4566 by default, with AWS_ENDPOINT_URL as the override. For LocalStack-managed Lambda containers, that hostname resolves back to the LocalStack instance. Recent AWS SDK endpoint settings can also honour AWS_ENDPOINT_URL directly, but keeping the fallback explicit here makes the LocalStack hostname obvious in the sample. The default is enough for the standard LocalStack-on-Docker setup; only set the env var in Step 4 if you need to override it.

Step 3: Bundle Pillow for the Lambda runtime

This is the macOS-developer gotcha. This walkthrough targets the LocalStack Python 3.12 Lambda environment we verified, which expects Linux-compatible native wheels. Pillow ships native binaries, so pip will install macOS-arm64 wheels on your laptop unless you tell it otherwise, and those won't work in the Lambda container.

The fix is pip install --platform manylinux2014_x86_64:

mkdir package
pip install Pillow \
  --platform manylinux2014_x86_64 \
  --target ./package \
  --only-binary=:all: \
  --python-version 3.12

cp thumbnailer.py package/

cd package && zip -qr ../thumbnailer.zip . && cd ..

Resulting zip is about 8MB - well under Lambda's 50MB direct upload limit. For larger deps you'd push to S3 first or use a Lambda layer; we'll do layers in a future article.

If you're already on Linux x86_64, drop the --platform flag and just pip install Pillow --target ./package.

Step 4: Deploy the Lambda

awslocal lambda create-function \
  --function-name thumbnailer \
  --runtime python3.12 \
  --role arn:aws:iam::000000000000:role/lambda-role \
  --handler thumbnailer.handler \
  --zip-file fileb://thumbnailer.zip \
  --timeout 30 \
  --memory-size 512 \
  --environment 'Variables={THUMBNAIL_BUCKET=thumbnails}'

Three knobs worth noticing:

--timeout 30 - default is 3 seconds, which is enough for the smoke test but won't survive a 4MP photo on a cold start. 30 seconds is comfortable.
--memory-size 512 - Pillow's image decoding allocates buffers proportional to image dimensions. 512MB is a safe minimum.
--role - LocalStack doesn't validate IAM by default on the Hobby tier, so any well-formed role ARN works. In real AWS you'd point at a role with s3:GetObject and s3:PutObject policies.

Wait for it to be ready:

awslocal lambda wait function-active-v2 --function-name thumbnailer

Step 5: Wire the S3 trigger

Two pieces: a permission on the Lambda saying "S3 is allowed to invoke me", and a notification on the bucket saying "send ObjectCreated events here".

# 1. Lambda permission for S3 to invoke
awslocal lambda add-permission \
  --function-name thumbnailer \
  --statement-id s3-invoke \
  --action lambda:InvokeFunction \
  --principal s3.amazonaws.com \
  --source-arn arn:aws:s3:::photos

Then the bucket notification config - save this as notification.json:

{
  "LambdaFunctionConfigurations": [
    {
      "Id": "thumb-on-upload",
      "LambdaFunctionArn": "arn:aws:lambda:us-east-1:000000000000:function:thumbnailer",
      "Events": ["s3:ObjectCreated:*"]
    }
  ]
}

Apply it:

awslocal s3api put-bucket-notification-configuration \
  --bucket photos \
  --notification-configuration file://notification.json

Step 6: Test it end to end

Generate a test image (or use any JPEG you have lying around):

python3 -c "
from PIL import Image
img = Image.new('RGB', (1200, 800), color='steelblue')
img.save('test.jpg', 'JPEG', quality=85)
print('Created 1200x800 test.jpg')
"

Upload it:

awslocal s3 cp test.jpg s3://photos/test.jpg
sleep 5 # demo pause; S3 notifications usually arrive in seconds but can sometimes take longer
awslocal s3 ls s3://thumbnails/

You should see:

2026-05-12 00:16:20 1332 thumb-test.jpg

Pull it back and check the dimensions:

awslocal s3 cp s3://thumbnails/thumb-test.jpg ./result.jpg
python3 -c "from PIL import Image; print(Image.open('result.jpg').size)"


(256, 171)

The 1200×800 source thumbnailed to 256×171 (aspect preserved). On a homelab box this usually completes in a couple of seconds, but S3 event notifications are asynchronous and designed for at-least-once delivery, so treat the demo sleep 5 as a convenience rather than a guarantee.

Step 7: Reading the logs when something goes wrong

This is the bit that saves your evening. Every Lambda invocation produces logs you can inspect through LocalStack's CloudWatch Logs emulation, which is close enough to the AWS workflow to make local debugging feel familiar.

# What log groups exist?
awslocal logs describe-log-groups --query 'logGroups[*].logGroupName' --output text

# Latest log stream for our Lambda
LATEST=$(awslocal logs describe-log-streams \
  --log-group-name /aws/lambda/thumbnailer \
  --order-by LastEventTime --descending --max-items 1 \
  --query 'logStreams[0].logStreamName' --output text)

# Read the events
awslocal logs get-log-events \
  --log-group-name /aws/lambda/thumbnailer \
  --log-stream-name "$LATEST" \
  --query 'events[*].message' --output text

You'll see the START, END, and REPORT lines you're used to from real Lambda, plus any print() or logging calls from your code.

The first time I ran this, the Lambda fired but bombed with EndpointConnectionError: Could not connect to "http://localstack:4566/photos/test.jpg" - that's the hostname gotcha I mentioned at the top. The fix was the localhost.localstack.cloud:4566 line in the Python source. The logs showed it instantly.

The networking gotcha, in detail

When LocalStack invokes a Lambda, it spawns a fresh Docker container on the Docker host. From inside that container:

localhost is the Lambda container itself - useless for talking back to LocalStack.
localstack would work if the LocalStack container is named that and on a Docker network the Lambda joins. Often it isn't.
localhost.localstack.cloud is the default hostname to use for LocalStack-managed Lambda containers, because localhost inside the function points back at the function container itself.

If you're running LocalStack on a remote homelab box and developing on your laptop, the Lambda environment variable for AWS_ENDPOINT_URL should still be http://localhost.localstack.cloud:4566 - it's resolved at Lambda runtime, inside the homelab's Docker network, not from your laptop.

Common pitfalls

Lambda fires but produces no thumbnail. Check the CloudWatch logs first. In LocalStack it's usually the hostname issue or a Pillow image-format issue. In real AWS, also check the execution role for missing s3:GetObject or s3:PutObject permissions.
Unable to import module 'thumbnailer': No module named 'PIL'. The Pillow native binaries didn't make it into the zip, or you skipped the --platform manylinux2014_x86_64 flag and bundled the Mac wheels.
Lambda invocations hang for 30+ seconds and time out. Docker socket mount missing on the host running LocalStack. Add /var/run/docker.sock:/var/run/docker.sock to the compose file and restart.
InvalidArgument: ... Configuration is ambiguously defined. You're applying a second notification config without clearing the first. Use --notification-configuration '{}' to wipe it, or fix the JSON to merge.
The thumbnail is bigger than the source. Pillow's thumbnail() only shrinks. If your source is 200×100, the "thumbnail" stays 200×100. That's correct behaviour.

Cleanup commands worth knowing

# Remove the Lambda
awslocal lambda delete-function --function-name thumbnailer

# Remove the bucket notification (the Lambda link)
awslocal s3api put-bucket-notification-configuration \
  --bucket photos --notification-configuration '{}'

# Wipe the thumbnails bucket
awslocal s3 rm s3://thumbnails --recursive

Don't actually delete the Lambda or the buckets if you're carrying on to Part 4 - we'll layer API Gateway on top of these.

Save this as a checkpoint

The supporting resources for the thumbnailer pipeline (the thumbnails bucket) can sit in your init hooks folder. The Lambda itself is built from a .zip of your code, which is awkward to redeploy from a one-shot shell script - so we'll skip the Lambda in the checkpoint and rely on the manual deploy steps above.

Save as init/ready.d/03-part3-lambda.sh:

#!/usr/bin/env bash
# Part 3 checkpoint - supporting resources for the thumbnailer pipeline
# (Lambda deployment + S3 trigger wiring is article-driven; redeploy via the
# steps in the article body. This script just ensures the buckets exist.)
awslocal s3 mb s3://thumbnails 2>/dev/null || true
echo "[bootstrap] part 3 - thumbnails bucket ready (deploy Lambda manually)"


chmod +x init/ready.d/03-part3-lambda.sh

Jumping in at Part 3 from scratch? You'll need scripts 01-part1-s3.sh and 02-part2-dynamodb.sh from the previous articles in your init/ready.d/ too. Then follow the Lambda build + trigger wiring above to get the dynamic part going.

What we'll wire up next

You've got an event-driven pipeline running locally. The next part puts a real HTTP API in front of the URL shortener from Part 2 - API Gateway with a Lambda backend, plus a JWT authoriser so only signed requests get through. Same Lambda mechanics as today, with HTTP routing and auth in front.

The full series

Part 0 - Start here: series intro and installing LocalStack
Part 1 - S3 locally: buckets, presigned URLs, and a tiny photo uploader
Part 2 - DynamoDB locally: building a URL shortener data layer
Part 3 - Lambda + S3 events: an image thumbnailer pipeline (this article)
Part 4 - API Gateway + Lambda + JWT auth: a real HTTP API (next)
Part 5 - SQS + SNS: a background job queue with a dead-letter queue
Part 6 - EventBridge + Step Functions: orchestrating a photo-processing workflow
Part 7 - Secrets Manager + KMS: handling secrets and encryption locally
Part 8 - Terraform (tflocal) + GitHub Actions: integration tests against LocalStack

Sources

Related on alishaikh.me

Run AWS on Your Laptop: A 9-Part LocalStack Build Series (Part 2 - DynamoDB URL Shortener Data Layer)

Ali Shaikh — Mon, 11 May 2026 17:35:09 +0000

If you've already worked through Part 1 - S3 Photo Uploader with Presigned URLs , this is where the app gets a data layer. By the end of this article you'll have a DynamoDB table on LocalStack, a small Node CLI that handles shortening, resolving, and atomic click counting, plus a fixture loader you can reuse later. Part 3 detours into Lambda + S3 events, and Part 4 will wrap this shortener in an HTTP API.

Why DynamoDB fits this job

A URL shortener is one of those workloads DynamoDB fits neatly:

The hot path is simple: look up code → long_url, then bump a click counter. That's still a tiny, key-based access pattern - exactly the sort of thing DynamoDB is good at.
Click counting wants atomic increments. DynamoDB supports SET attr = attr + :n natively - no read-modify-write race conditions.
Writes are point inserts with a uniqueness check. DynamoDB's ConditionExpression makes "fail if this code already exists" a one-line addition.

In practice, Bitly, TinyURL, and plenty of internal "go links" tools use this same shape: short code in, long URL out, plus a counter on the side.

Step 1: Create the shortlinks table

cd ~/projects/localstack-series
mkdir part2-dynamodb
cd part2-dynamodb

Create the table:

awslocal dynamodb create-table \
  --table-name shortlinks \
  --attribute-definitions AttributeName=code,AttributeType=S \
  --key-schema AttributeName=code,KeyType=HASH \
  --billing-mode PAY_PER_REQUEST

A few things worth understanding:

AttributeName=code,KeyType=HASH - code is the partition key. The CLI still uses the legacy HASH / RANGE terminology. For a URL shortener it's a 6-character random string like abc123.
PAY_PER_REQUEST - billing mode. The other option is PROVISIONED where you pre-allocate read/write capacity. PAY_PER_REQUEST (also called "on-demand") is what you want for unpredictable workloads, and what most new tables use.
Why is long_url not in --attribute-definitions? DynamoDB only requires you to declare attributes used in keys (primary or secondary indexes). Everything else is freeform.

Confirm it's there:

awslocal dynamodb describe-table \
  --table-name shortlinks \
  --query 'Table.{Name:TableName,Status:TableStatus,Keys:KeySchema}'


{
    "Name": "shortlinks",
    "Status": "ACTIVE",
    "Keys": [
        { "AttributeName": "code", "KeyType": "HASH" }
    ]
}

Step 2: Set up the Node project

npm init -y
npm install @aws-sdk/client-dynamodb @aws-sdk/lib-dynamodb

Set "type": "module" in package.json:

{
  "name": "part2-dynamodb",
  "type": "module",
  "dependencies": {
    "@aws-sdk/client-dynamodb": "^3.600.0",
    "@aws-sdk/lib-dynamodb": "^3.600.0"
  }
}

A note on the two packages: @aws-sdk/client-dynamodb is the low-level client that speaks the raw DynamoDB JSON wire format (where strings are wrapped as { S: "..." } and numbers as { N: "..." }). @aws-sdk/lib-dynamodb is a small wrapper that lets you work with plain JavaScript objects. Use the wrapper for application code; you almost never want the raw form.

The exact minor version will move over time. Any current AWS SDK v3 release is fine here, the pattern is the thing that matters.

Step 3: Write the shortlinks module

Create shortlinks.js:

import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
import {
  DynamoDBDocumentClient,
  PutCommand,
  ScanCommand,
  UpdateCommand,
  DeleteCommand,
  BatchWriteCommand,
} from '@aws-sdk/lib-dynamodb';
import { setTimeout as sleep } from 'node:timers/promises';

const client = new DynamoDBClient({
  endpoint: 'http://localhost:4566',
  region: 'us-east-1',
  credentials: { accessKeyId: 'test', secretAccessKey: 'test' },
});

const ddb = DynamoDBDocumentClient.from(client);
const TABLE = 'shortlinks';

function randomCode(len = 6) {
  const chars = 'abcdefghijklmnopqrstuvwxyz0123456789';
  let out = '';
  for (let i = 0; i < len; i++) out += chars[Math.floor(Math.random() * chars.length)];
  return out;
}

Now the operations, one at a time.

Shorten - write a new entry, fail if the code is taken

export async function shorten(longUrl, code = randomCode()) {
  await ddb.send(new PutCommand({
    TableName: TABLE,
    Item: {
      code,
      long_url: longUrl,
      created_at: new Date().toISOString(),
      click_count: 0,
    },
    ConditionExpression: 'attribute_not_exists(code)',
  }));
  return code;
}

The line that's worth understanding: ConditionExpression: 'attribute_not_exists(code)' makes the write conditional on the code not already being in the table. Two requests racing to claim the same code - only one wins. The other gets ConditionalCheckFailedException, which the caller can catch and retry with a new code.

This is genuinely how production URL shorteners avoid double-claiming. In a real app, that failure path usually triggers "generate a new code and try again" rather than surfacing the exception directly.

Resolve - increment the counter and return the updated item

export async function resolve(code) {
  try {
    const updated = await ddb.send(new UpdateCommand({
      TableName: TABLE,
      Key: { code },
      ConditionExpression: 'attribute_exists(code)',
      UpdateExpression: 'SET click_count = if_not_exists(click_count, :zero) + :one',
      ExpressionAttributeValues: {
        ':zero': 0,
        ':one': 1,
      },
      ReturnValues: 'ALL_NEW',
    }));
    return updated.Attributes;
  } catch (err) {
    if (err.name === 'ConditionalCheckFailedException') return null;
    throw err;
  }
}

This version is closer to what you'd actually ship:

attribute_exists(code) stops DynamoDB from creating a brand-new row when the short code doesn't exist.
if_not_exists(click_count, :zero) + :one keeps the counter safe even if an older row is missing click_count.
ReturnValues: 'ALL_NEW' gives you the updated item back in the same call, so the read path stays compact.

The key point is the atomic increment. Even with lots of concurrent resolves, every click is counted exactly once. No read-modify-write loop, no transaction needed. The attribute_exists(code) guard matters because UpdateItem can create an item if you let it.

List, remove

export async function list(limit = 25) {
  const out = await ddb.send(new ScanCommand({
    TableName: TABLE,
    Limit: limit,
  }));
  return (out.Items || []).sort((a, b) => a.code.localeCompare(b.code));
}

export async function remove(code) {
  await ddb.send(new DeleteCommand({
    TableName: TABLE,
    Key: { code },
  }));
}

ScanCommand is fine for tens or hundreds of items but reads every item in the table. It also does not return a meaningful order, so the small in-memory sort is the only reason the CLI output looks stable here. We'll talk about the bigger tradeoffs in a moment.

Fixture loading with BatchWrite

export async function loadFixtures(items) {
  // BatchWrite is capped at 25 items per request.
  for (let i = 0; i < items.length; i += 25) {
    let pending = items.slice(i, i + 25).map(item => ({
      PutRequest: {
        Item: {
          code: item.code,
          long_url: item.long_url,
          created_at: item.created_at || new Date().toISOString(),
          click_count: item.click_count || 0,
        },
      },
    }));

    while (pending.length > 0) {
      const out = await ddb.send(new BatchWriteCommand({
        RequestItems: {
          [TABLE]: pending,
        },
      }));

      pending = out.UnprocessedItems?.[TABLE] || [];
      if (pending.length > 0) await sleep(200);
    }
  }
}

The 25-per-request cap is a real DynamoDB limit, not a LocalStack one. The other detail that matters is UnprocessedItems: DynamoDB can accept part of a batch and ask you to retry the rest. That's why the loop keeps retrying until the chunk drains. The fixed 200ms pause is perfectly fine for a local fixture script like this; for production retry loops, exponential backoff is the usual guidance. One more wrinkle: BatchWrite only supports puts and deletes, not conditional writes or partial updates, which is why fixtures use it but the live shorten/resolve path doesn't.

CLI wrapper at the bottom

if (import.meta.url === `file://${process.argv[1]}`) {
  const [, , cmd, ...args] = process.argv;

  switch (cmd) {
    case 'shorten':
      console.log(`Shortened to: ${await shorten(args[0], args[1])}`);
      break;
    case 'resolve': {
      const item = await resolve(args[0]);
      console.log(item ? JSON.stringify(item, null, 2) : 'Not found');
      break;
    }
    case 'list': {
      const items = await list();
      console.table(items.map(({ code, long_url, click_count }) =>
        ({ code, long_url, click_count })));
      break;
    }
    case 'remove':
      await remove(args[0]);
      console.log(`Removed: ${args[0]}`);
      break;
    case 'fixtures': {
      await loadFixtures([
        { code: 'gh', long_url: 'https://github.com' },
        { code: 'np', long_url: 'https://npmjs.com' },
        { code: 'mdn', long_url: 'https://developer.mozilla.org' },
        { code: 'aws', long_url: 'https://docs.aws.amazon.com' },
      ]);
      console.log('Loaded 4 fixtures');
      break;
    }
    default:
      console.error('Usage: node shortlinks.js <shorten|resolve|list|remove|fixtures> [args...]');
      process.exit(1);
  }
}

Step 4: Try it out

$ node shortlinks.js fixtures
Loaded 4 fixtures

$ node shortlinks.js list
┌─────────┬───────┬─────────────────────────────────┬─────────────┐
│ (index) │ code │ long_url │ click_count │
├─────────┼───────┼─────────────────────────────────┼─────────────┤
│ 0 │ 'aws' │ 'https://docs.aws.amazon.com' │ 0 │
│ 1 │ 'gh' │ 'https://github.com' │ 0 │
│ 2 │ 'mdn' │ 'https://developer.mozilla.org' │ 0 │
│ 3 │ 'np' │ 'https://npmjs.com' │ 0 │
└─────────┴───────┴─────────────────────────────────┴─────────────┘

$ node shortlinks.js resolve gh
{
  "click_count": 1,
  "created_at": "2026-05-11T17:15:33.184Z",
  "code": "gh",
  "long_url": "https://github.com"
}

$ node shortlinks.js resolve gh
{
  "click_count": 2,
  "created_at": "2026-05-11T17:15:33.184Z",
  "code": "gh",
  "long_url": "https://github.com"
}

$ node shortlinks.js shorten "https://anthropic.com" claude
Shortened to: claude

$ node shortlinks.js shorten "https://example.com" claude
... ConditionalCheckFailedException

The atomic counter ticked from 1 to 2 across two separate processes. The second shorten for claude failed cleanly because the code was already taken. In a production shortener, that's where you'd generate another code and retry.

Scan vs Query, before this grows up

Scan walks the table rather than doing a key-targeted lookup - fine for tens of items, painful at thousands, ruinous at millions. The two patterns you'll actually want in production:

Query - point reads by partition key, plus sorted reads when you have a sort key. The thing DynamoDB is fastest at.
Global Secondary Index (GSI) - a second key on a different attribute. For instance, if you wanted to look up shortlinks by long_url ("has anyone shortened this before?"), you'd add a GSI with long_url as its partition key.

For Part 4 of this series we won't need a GSI - the API only ever looks up by code. If you later want a "my shortlinks" listing per user, add a GSI on owner_id and Query it. We'll wire that up properly when we hit the auth article.

Single-table design - when you'd actually need it

You'll see "single-table design" mentioned in DynamoDB tutorials a lot. The idea is simple enough: instead of one table per entity (users, posts, comments), you put everything in one table with composed keys (USER#123, POST#456, COMMENT#789#REPLY#1). It's powerful and saves money at scale.

For a URL shortener with one entity, it's overkill. The moment you start adding owners, tags, click events, or shared lists, the single-table approach starts paying off. We're keeping it simple here. If this app grows up in your own homelab, that's the point where you'd revisit the table design. Alex DeBrie's DynamoDB Book is the canonical reference.

Common pitfalls

ResourceNotFoundException when running the script. Forgot to create the table, or you typed the table name wrong.
ConditionalCheckFailedException on resolve for a code you expected to exist. The item is missing, or you wrote it into a different table or endpoint.
Empty Items array on Scan even after writes succeeded. You're scanning a different table, or pointing at a different LocalStack instance. Run awslocal dynamodb list-tables to confirm.
UnrecognizedClientException when using the SDK. You forgot to set endpoint and credentials on the client - it's trying to hit real AWS.

Cleanup commands worth knowing

# Drop the table entirely
awslocal dynamodb delete-table --table-name shortlinks

# Recreate it when you want a clean slate again
awslocal dynamodb create-table \
  --table-name shortlinks \
  --attribute-definitions AttributeName=code,AttributeType=S \
  --key-schema AttributeName=code,KeyType=HASH \
  --billing-mode PAY_PER_REQUEST

Drop-and-recreate is what you'll usually reach for during development. Fewer moving parts, less shell glue, less chance of deleting the wrong thing.

Save this as a checkpoint

Add this to your init/ready.d/ folder so the table comes back automatically next time you docker compose up. (Pattern set up in Part 0.)

Save as init/ready.d/02-part2-dynamodb.sh:

#!/usr/bin/env bash
# Part 2 checkpoint - DynamoDB shortlinks table
awslocal dynamodb create-table \
  --table-name shortlinks \
  --attribute-definitions AttributeName=code,AttributeType=S \
  --key-schema AttributeName=code,KeyType=HASH \
  --billing-mode PAY_PER_REQUEST 2>/dev/null || true

awslocal dynamodb wait table-exists --table-name shortlinks 2>/dev/null || true
echo "[bootstrap] part 2 - shortlinks table ready"


chmod +x init/ready.d/02-part2-dynamodb.sh

Jumping in at Part 2 from scratch? Drop in 01-part1-s3.sh from Part 1 alongside this one. Strictly speaking, Part 2 doesn't depend on Part 1 yet. Still, keeping the scripts in numeric order now makes the later parts much less messy.

What we'll wire up next

You've got a working data layer with atomic counters and collision-safe writes. The next part brings Lambda into the mix: a Python function that listens for S3 upload events from Part 1's photo bucket and writes a thumbnail to a second bucket. Then Part 4 will wrap our shortener (this article's table) and the photo flow (Part 1 + 3) into a real HTTP API with JWT auth.

The full series

Part 0 - Start here: series intro and installing LocalStack
Part 1 - S3 locally: buckets, presigned URLs, and a tiny photo uploader
Part 2 - DynamoDB locally: building a URL shortener data layer (this article)
Part 3 - Lambda + S3 events: an image thumbnailer pipeline (next)
Part 4 - API Gateway + Lambda + JWT auth: a real HTTP API
Part 5 - SQS + SNS: a background job queue with a dead-letter queue
Part 6 - EventBridge + Step Functions: orchestrating a photo-processing workflow
Part 7 - Secrets Manager + KMS: handling secrets and encryption locally
Part 8 - Terraform (tflocal) + GitHub Actions: integration tests against LocalStack

Sources

Related on alishaikh.me

Run AWS on Your Laptop: A 9-Part LocalStack Build Series (Part 1 - S3 Photo Uploader with Presigned URLs)

Ali Shaikh — Sat, 09 May 2026 19:50:51 +0000

If you've already worked through Part 0 - Setup , this is where the series starts feeling real. By the end of this article you'll have a working photo uploader where the browser sends files straight to an S3 bucket through a presigned URL . Same production pattern, just running on LocalStack on your laptop.

What we're building

Browser ─┐
          │ 1. GET /sign?key=<filename>
          ▼
        Backend (Node) ── 2. presigned URL ──▶ Browser
                                                  │
                                                  │ 3. PUT file directly to S3
                                                  ▼
                                              LocalStack S3
                                              (bucket: photos)

A two-component app. The Node backend issues short-lived presigned URLs. The browser uses those URLs to upload files straight to S3 - the backend never sees the file bytes.

Why this pattern actually matters

If you're new to presigned URLs, this is the part where it clicks. Most beginner tutorials show you how to upload a file by streaming it through your backend: browser → backend → S3. That works for tiny files. It falls apart fast at scale because:

Your backend handles every byte. A 50MB photo upload ties up a server worker for several seconds.
Bandwidth is doubled (in to your backend, out to S3).
Memory pressure climbs the moment you have a few concurrent uploads.

Presigned URLs flip this. Your backend signs a short-lived URL that authorises a single upload, then hands the URL to the browser. The browser uploads directly to S3. The backend is back to handling JSON-sized requests in milliseconds.

You'll see this pattern all over modern SaaS. Worth learning properly.

Step 1: Make a project folder for Part 1

We'll keep each part's code in its own subfolder under the series root from Part 0:

cd ~/projects/localstack-series
mkdir part1-s3-uploader
cd part1-s3-uploader

LocalStack should already be running. Quick check:

docker compose ps
awslocal s3 ls

If you see your hello-localstack bucket from Part 0, you're good to go.

Step 2: Create the photos bucket

awslocal s3 mb s3://photos

That's the storage half done. Next we need to tell the bucket it's allowed to accept uploads from a browser.

Step 3: Configure CORS on the bucket

The browser will be making a cross-origin PUT from http://localhost:3000 (our backend) to http://localhost:4566 (LocalStack S3). Without CORS configured, the browser will refuse the upload.

Create cors.json:

{
  "CORSRules": [
    {
      "AllowedHeaders": ["*"],
      "AllowedMethods": ["GET", "PUT"],
      "AllowedOrigins": ["*"],
      "ExposeHeaders": ["ETag"]
    }
  ]
}

Apply it:

awslocal s3api put-bucket-cors --bucket photos --cors-configuration file://cors.json

AllowedOrigins: ["*"] is fine for local dev. In production you'd lock this down to your real frontend origin.

Step 4: The backend - issuing presigned URLs

Set up the Node project:

npm init -y
npm install @aws-sdk/client-s3 @aws-sdk/s3-request-presigner

Then in package.json, set the type to module:

{
  "name": "part1-s3-uploader",
  "type": "module",
  "dependencies": {
    "@aws-sdk/client-s3": "^3.600.0",
    "@aws-sdk/s3-request-presigner": "^3.600.0"
  }
}

Create server.js:

import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3';
import { getSignedUrl } from '@aws-sdk/s3-request-presigner';
import http from 'node:http';
import { readFileSync } from 'node:fs';

const s3 = new S3Client({
  endpoint: 'http://localhost:4566',
  region: 'us-east-1',
  forcePathStyle: true,
  credentials: { accessKeyId: 'test', secretAccessKey: 'test' },
});

const html = readFileSync('./index.html', 'utf8');

const server = http.createServer(async (req, res) => {
  if (req.url === '/') {
    res.setHeader('Content-Type', 'text/html');
    res.end(html);
    return;
  }

  if (req.url.startsWith('/sign')) {
    const url = new URL(req.url, 'http://localhost');
    const key = url.searchParams.get('key');
    const type = url.searchParams.get('type') || 'application/octet-stream';

    if (!key) {
      res.statusCode = 400;
      res.setHeader('Content-Type', 'application/json');
      res.end(JSON.stringify({ error: 'key is required' }));
      return;
    }

    const cmd = new PutObjectCommand({
      Bucket: 'photos',
      Key: key,
      ContentType: type,
    });
    const signedUrl = await getSignedUrl(s3, cmd, { expiresIn: 60 });

    res.setHeader('Content-Type', 'application/json');
    res.end(JSON.stringify({ url: signedUrl, key }));
    return;
  }

  res.statusCode = 404;
  res.end('not found');
});

server.listen(3000, () => console.log('Listening on http://localhost:3000'));

A few things worth understanding line by line:

endpoint: 'http://localhost:4566' - points the SDK at LocalStack instead of real AWS.
forcePathStyle: true - the small but important LocalStack gotcha. As the LocalStack S3 docs explain, path-style requests are the safe default for local S3 work. Without this, the SDK builds URLs like https://photos.s3.amazonaws.com/... which won't resolve locally.
credentials: { accessKeyId: 'test', secretAccessKey: 'test' } - LocalStack accepts any credentials by default. The strings can be anything.
expiresIn: 60 - the presigned URL is valid for 60 seconds. Long enough for a slow connection, short enough that a leaked URL isn't a long-term problem.

Step 5: The frontend - a tiny upload form

Create index.html next to server.js:

<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <title>Photo Uploader (LocalStack)</title>
  <style>
    body { font-family: system-ui; max-width: 480px; margin: 4rem auto; padding: 0 1rem; }
    button { padding: .5rem 1rem; }
    .ok { color: #137333; }
    .err { color: #c5221f; }
  </style>
</head>
<body>
  <h1>Photo Uploader</h1>
  <p>Pick an image and upload it straight to LocalStack S3.</p>
  <input type="file" id="file" accept="image/*">
  <button id="upload">Upload</button>
  <p id="status"></p>

  <script>
    const $file = document.getElementById('file');
    const $upload = document.getElementById('upload');
    const $status = document.getElementById('status');

    $upload.addEventListener('click', async () => {
      const file = $file.files[0];
      if (!file) return;

      const key = `${Date.now()}-${crypto.randomUUID()}-${file.name}`;
      const type = file.type || 'application/octet-stream';
      $status.className = '';
      $status.textContent = 'Getting presigned URL...';

      const params = new URLSearchParams({ key, type });
      const r = await fetch(`/sign?${params}`);
      const { url } = await r.json();

      $status.textContent = 'Uploading to S3...';
      const upload = await fetch(url, {
        method: 'PUT',
        body: file,
        headers: { 'Content-Type': type },
      });

      if (upload.ok) {
        $status.className = 'ok';
        $status.textContent = `Uploaded as ${key}`;
      } else {
        $status.className = 'err';
        $status.textContent = `Upload failed: ${upload.status}`;
      }
    });
  </script>
</body>
</html>

Two small but important details:

The browser PUTs the file directly to the URL returned by the backend. Note the request goes to localhost:4566, not localhost:3000 - the backend is out of the picture once the URL is signed.
The Content-Type header on the PUT must match what the backend signed. If they don't match, S3 returns a 403 with a "signature does not match" error. The code above uses the same type variable for both.

Step 6: Run it

node server.js

Then open http://localhost:3000 in a browser, pick a photo, and click Upload. You should see "Uploaded as 1714...-..." within a second.

Step 7: Verify the file in S3

awslocal s3 ls s3://photos/

Your uploaded file should be there with a Date.now()-prefixed key. To download it back and check:

awslocal s3 cp s3://photos/<your-key> ./downloaded.jpg

Or grab a presigned GET URL and view it in a browser:

awslocal s3 presign s3://photos/<your-key> --expires-in 60

Open the printed URL - your photo loads.

A real-world touch: auto-expiring uploads with a lifecycle rule

S3 lifecycle rules let you tell the bucket "delete anything older than N days". Genuinely useful for user uploads where you don't want to store everything forever.

Create lifecycle.json:

{
  "Rules": [
    {
      "ID": "expire-uploads-after-30-days",
      "Status": "Enabled",
      "Filter": { "Prefix": "" },
      "Expiration": { "Days": 30 }
    }
  ]
}

Apply:

awslocal s3api put-bucket-lifecycle-configuration \
  --bucket photos \
  --lifecycle-configuration file://lifecycle.json

In real AWS, S3 runs lifecycle expiration asynchronously, not the instant the clock flips. For this article, the useful bit is the bucket configuration itself: you can keep the same rule in local dev and production without changing your app code.

Common pitfalls

CORS errors in the browser console. Re-check that you ran the put-bucket-cors command after creating the bucket. The error message looks like "blocked by CORS policy".
403 SignatureDoesNotMatch on upload. The Content-Type you sent in the PUT didn't match what the backend signed. Print both, line them up.
getaddrinfo ENOTFOUND photos.s3.amazonaws.com. You forgot forcePathStyle: true on the S3 client.
Presigned URL works for a few seconds then 403s. It's expired - you set expiresIn too low for a slow upload. Bump it to 300 (five minutes) for testing.
Server can't read index.html. Run node server.js from the part1-s3-uploader/ folder, not from a parent.

Cleanup commands worth knowing

# Empty the bucket
awslocal s3 rm s3://photos --recursive

# Delete the bucket entirely
awslocal s3 rb s3://photos --force

# Or just stop the Node server with Ctrl+C - LocalStack keeps running

Don't actually delete the bucket if you plan to do Part 3 (the Lambda thumbnailer reads from this same bucket).

Save this as a checkpoint

State doesn't survive a docker compose down on the free Hobby tier, so let's bottle this part up as a script that recreates everything next time LocalStack starts. (Part 0 set up the init/ready.d/ folder and the docker-compose mount; we just drop a new file in it.)

Save as init/ready.d/01-part1-s3.sh:

#!/usr/bin/env bash
# Part 1 checkpoint - S3 photo uploader (bucket + CORS + lifecycle)
awslocal s3 mb s3://photos 2>/dev/null || true

awslocal s3api put-bucket-cors --bucket photos --cors-configuration '{
  "CORSRules":[{"AllowedHeaders":["*"],"AllowedMethods":["GET","PUT"],"AllowedOrigins":["*"],"ExposeHeaders":["ETag"]}]
}' 2>/dev/null || true

awslocal s3api put-bucket-lifecycle-configuration --bucket photos --lifecycle-configuration '{
  "Rules":[{"ID":"expire-uploads-after-30-days","Status":"Enabled","Filter":{"Prefix":""},"Expiration":{"Days":30}}]
}' 2>/dev/null || true

echo "[bootstrap] part 1 - photos bucket + CORS + lifecycle ready"


chmod +x init/ready.d/01-part1-s3.sh

Next docker compose up and your Part 1 setup is back. The 2>/dev/null || true keeps each command idempotent - re-running on an already-set-up bucket is harmless.

Jumping in at Part 1 from a fresh LocalStack? This script is all you need (Part 0 has no resources of its own beyond the smoke-test bucket).

What we'll wire up next

You've got a real upload flow running locally - same pattern as production SaaS apps, in 50-odd lines of code. The next part builds the data layer for our URL shortener using DynamoDB: single-table design, fixture loading, and the awslocal dynamodb commands you'll keep coming back to. Part 4 will wire it all to an HTTP API.

The full series

Part 0 - Start here: series intro and installing LocalStack
Part 1 - S3 locally: buckets, presigned URLs, and a tiny photo uploader (this article)
Part 2 - DynamoDB locally: building a URL shortener data layer (next)
Part 3 - Lambda + S3 events: an image thumbnailer pipeline
Part 4 - API Gateway + Lambda + JWT auth: a real HTTP API
Part 5 - SQS + SNS: a background job queue with a dead-letter queue
Part 6 - EventBridge + Step Functions: orchestrating a photo-processing workflow
Part 7 - Secrets Manager + KMS: handling secrets and encryption locally
Part 8 - Terraform (tflocal) + GitHub Actions: integration tests against LocalStack

Sources

Related Posts

Docker's MCP Catalog and Toolkit: What It Is and Why It Matters

Ali Shaikh — Tue, 05 May 2026 17:03:02 +0000

So Docker has shipped its MCP Toolkit. If you've been wiring up Claude or Cursor to talk to outside tools and felt like you were duct-taping JSON config files together at 2 am, you're not the only one. The Model Context Protocol crowd has grown fast, and the plumbing has stayed messy. Docker's answer: package the servers as containers, list them in a catalogue, and put a single gateway in front of the lot.

Let me explain what's actually new, what it does, and where it fits.

The mess MCP was creating

Model Context Protocol is the open spec that lets a model talk to external tools, files, and APIs in a structured way. It works. But every server tends to have its own install steps, its own runtime, and its own quirks. Want GitHub, Atlassian, and a search tool wired into your editor? You'd be cloning three repos, juggling Node and Python versions, and stuffing API keys into config files that almost certainly end up in someone's screenshare by accident.

Docker's pitch is straightforward: containers already solved this for web apps a decade ago, so do the same for MCP servers (Docker Docs).

What the MCP Toolkit actually is

The MCP Toolkit is a management surface inside Docker Desktop (4.62 and up) that lets you browse, launch, and configure containerised MCP servers, then point your AI client at them (Docker Docs). Underneath, every server is a regular container image, so you don't need Node, Python, or some random binary on your host to run them.

The Toolkit comes with three pieces worth knowing about:

The Catalog : a curated list of over 200 verified MCP servers, packaged as signed images with SBOMs, provenance, and security updates (Docker Docs).
The Gateway : a single endpoint your client connects to, which fronts every server you've enabled.
Profiles : named bundles of servers and their config, so your "data analysis" stack doesn't bleed into your "infra ops" stack.

I'll come back to each.

The Catalog: a registry, but for AI tools

If Docker Hub is where you go for a Postgres image, the MCP Catalog is where you go for a YouTube transcript reader, a Brave Search server, a Notion connector, an Atlassian bridge, and so on. Images live under the mcp/ namespace, are signed by Docker, and ship with a Software Bill of Materials so you can see exactly what's inside (Docker Docs).

The supply-chain story matters here. MCP servers run with privileges your model gets to use, which means a poisoned image is a really bad day. Signed images, provenance checks, and Scout integration push that risk closer to the floor.

The Gateway: one socket to rule them all

Here's where it gets clever. Instead of every client opening a connection to every server, the Gateway sits in the middle and aggregates them (Docker blog). Your client (Claude Desktop, Cursor, VS Code, Windsurf, Goose, continue.dev, and OpenAI's Codex CLI) connects once. The Gateway:

starts and stops the underlying containers,
proxies trusted remote servers when you'd rather not run them locally,
handles OAuth flows so you don't paste tokens into config files,
and mounts secrets only into the container that needs them.

You can run more than one Gateway, each with a different profile attached. So a "chess tools" Gateway and a "cloud ops" Gateway can sit side by side without stepping on each other.

Profiles and the March 2026 templates

Profiles are just named collections: pick a few servers, set their config, save the bundle. They're version-controllable, which is the bit teams care about. Compose-first means a profile can move from a laptop into Cloud Run or Azure Container Apps without much rework (Docker blog).

In March 2026, Docker Desktop 4.67 added Profile Templates: a starter-pack idea applied to MCP, surfaced through the Profiles tab and the docker mcp profile create CLI (Doolpa for the release date). Pick a card on the Profiles tab, get a pre-configured bundle for web work (think GitHub plus Playwright), data analysis, or cloud infra, and you're running in seconds rather than fiddling with five separate setup guides.

Security, but the sort that doesn't make you cry

A few things make the security model nicer than the DIY route:

Image signing and provenance on everything in the catalogue.
OAuth handled in-browser , with credentials stored by the Toolkit rather than scattered across YAML.
Per-container secret mounts so your GitHub token isn't visible to the Notion server.
Scoped tokens for services like GitHub, so the Toolkit hands the model the narrowest credential it can get away with rather than a blanket personal access token (Docker blog).

None of this is magic. It's the same supply-chain hygiene Docker already pushed for application images, just pointed at a new audience.

Using it from the CLI

If you live in your terminal, the Toolkit ships with a docker mcp command set so you can list servers, enable them, and start the Gateway without ever opening Desktop's UI (Docker Docs). Handy for scripts and CI, and frankly nicer when you're already in flow.

A first-time run looks roughly like this:

Update Docker Desktop to 4.62 or later.
Open the MCP Toolkit panel, browse the Catalog, click into a server you want.
Authenticate where needed (OAuth pops a browser window).
Add the server to a profile.
Start the Gateway.
Point Claude, Cursor, or VS Code at it.
Run mcp list from the client and watch the tools appear.

That's the loop. No virtualenvs, no Node version managers, no "works on my machine" tickets.

Wiring it into Codex

Codex deserves its own note because Docker has done the integration work properly. OpenAI's Codex CLI and IDE extension share an MCP config at ~/.codex/config.toml, and Docker ships a one-shot helper that fills it in for you (Docker blog):

docker mcp client configure codex

Run that once, and any server you've enabled in your Toolkit profile shows up inside Codex on the next launch. The same trick works for Claude Desktop, Cursor, and the others, but the Codex story is the freshest one and the easiest to recommend if you're already in the OpenAI camp (OpenAI Developers).

Who actually benefits

A few honest answers:

Solo developers who've been dabbling with MCP and want to try ten servers without breaking their machine.
Teams that need shared, reproducible tool stacks. Compose files give you a real source of truth.
Security-conscious shops that won't run unsigned binaries from random GitHub repos. Now they don't have to.
Anyone moving from prototype to production. The same profile that runs locally can ship to Cloud Run, Azure, or wherever your container platform lives.

If you're not using MCP yet, the Toolkit lowers the trial cost so much that there's no real reason not to poke at it for an afternoon.

Where it goes from here

Docker is treating MCP the way it treated the early days of containers: turn the messy install dance into a one-click experience, build a catalogue, then add governance on top. The Gateway being open source means the community can extend it, which is the bit that'll matter long-term (Docker blog).

Worth a look if you've spent any time wiring AI tools by hand. You'll get that hour back, plus a few more.

Sources

Run AWS on Your Laptop: A 9-Part LocalStack Build Series (Part 0 — Setup)

Ali Shaikh — Sun, 03 May 2026 13:42:56 +0000

Welcome to a 9-part hands-on series on building real things with LocalStack. We'll build a small but real backend on your laptop — photo uploads, image processing, a URL shortener with JWT auth, async job queues, scheduled workflows, encrypted secrets, the lot — all running locally, all without an AWS bill. New parts land as I finish them. By the end you'll have a working stack provisioned with Terraform and tested in GitHub Actions, plus the kind of feel for AWS that only comes from actually wiring it up rather than reading docs.

This first article is the "start here" piece. By the end of the next 30 minutes, you'll have LocalStack running on your laptop, the AWS CLI pointed at the right endpoint, and a smoke-tested S3 bucket — ready for the next part.

What you'll need: Docker (Docker Desktop on Mac/Windows, or Docker Engine on Linux), about 30 minutes, and a free LocalStack account.

What is LocalStack, in one paragraph

LocalStack is an emulator that runs a fake AWS on your laptop. You point your AWS CLI or SDK at http://localhost:4566 instead of the real AWS endpoints, and the rest of your code stays the same. Most of the AWS services you'd actually use day to day are covered — S3, Lambda, DynamoDB, SQS, SNS, EventBridge, API Gateway, Step Functions, Secrets Manager, KMS, and more. The exact count varies by plan and release; the free Hobby plan advertises 30+ services, paid tiers more. If you want the longer conceptual intro, my older post LocalStack: Run AWS Services Locally for Free is the place. It pre-dates the March 2026 packaging change (which we'll handle in a moment) but the model still holds.

What we're building over the series

One coherent project, threaded through the next nine weeks. By the end of week nine you'll have:

A photo upload flow (S3 + presigned URLs) where the browser uploads directly to a bucket
An image processing pipeline (Lambda fired by S3 events) that creates thumbnails
A URL shortener API (DynamoDB + API Gateway + Lambda + JWT auth) so users can share short links
A background job queue (SQS + SNS + dead-letter queue) for things like welcome emails
A scheduled workflow (EventBridge + Step Functions) running nightly cleanup and reprocessing
Encrypted secrets (Secrets Manager + KMS) for API keys and third-party credentials
The whole stack provisioned with Terraform (via tflocal) and tested in GitHub Actions CI

Same architectural patterns most small SaaS apps use. The same patterns you'd use against real AWS — the only difference is the endpoint URL.

Who this series is for

You'll get the most out of it if:

You write code in Node, Python, Go, or any language with an AWS SDK
You've used AWS before, even briefly, but don't want to keep paying to learn
You'd like to write integration tests that actually exercise S3, Lambda, and DynamoDB
You're a homelabber who wants the AWS patterns running on your own hardware
You're sizing up AWS for a side project and want to prototype before committing

If you've never opened the AWS console, the older intro post is the friendlier starting point.

Heads up: the 2026 packaging change

Quick footnote because it'll catch you out otherwise. In the March 2026 release (2026.03.0), LocalStack consolidated its two Docker images into one and folded the previous free Community Edition into a free Hobby plan for non-commercial use. The image is still free — you just register an account and pass an auth token to the container. Step 1 below walks through it.

Step 1: Grab your auth token

Sign up at app.localstack.cloud, confirm the email, and copy the auth token from your account page. Add it to your shell config so every future shell picks it up:

# ~/.zshrc or ~/.bashrc
export LOCALSTACK_AUTH_TOKEN=ls-...your-token-here...

Reload your shell (source ~/.zshrc) before moving on.

Step 2: Set up the project folder

Everything in this series will live under one folder. Create it now:

mkdir -p ~/projects/localstack-series
cd ~/projects/localstack-series

We'll add subdirectories per article as the series goes on (part1-s3-uploader/, part2-dynamodb/, and so on).

Step 3: The docker-compose.yml

Drop this in the project root. It's the file you'll keep coming back to:

# docker-compose.yml
services:
  localstack:
    container_name: localstack
    image: localstack/localstack:latest
    ports:
      - "127.0.0.1:4566:4566"
      - "127.0.0.1:4510-4559:4510-4559"
    environment:
      - LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN}
      - DEBUG=0
      - PERSISTENCE=1
      - SERVICES=s3,lambda,dynamodb,sqs,sns,events,iam,sts,apigateway,secretsmanager,kms,stepfunctions,logs
    volumes:
      - "./init:/etc/localstack/init"
      - "./.localstack:/var/lib/localstack"
      - "/var/run/docker.sock:/var/run/docker.sock"

A few choices worth explaining:

localstack/localstack:latest — the simplest "always current" pin. The image moves with each monthly release; if you'd rather lock to a specific version once you're up and running, swap latest for the full version tag (2026.04.0 or whatever's current at app.localstack.cloud).
Port 4566 is the LocalStack Gateway — every emulated AWS service is reachable through it. The 4510-4559 range is the external services port range that some services use for their own endpoints.
SERVICES restricts LocalStack to only the listed services — anything not in the list is disabled. Useful for keeping the memory footprint small and for failing loudly if your code accidentally calls something you didn't intend to test. By default LocalStack lazy-loads services on first use, so unused ones cost nothing — but disabling them entirely is cleaner.
PERSISTENCE=1 plus the .localstack volume is the configuration that saves state between restarts on the paid Base and Ultimate plans. On the free Hobby tier the env var is silently unsupported — state wipes on every docker compose down. I've left the variable in the compose file because it's harmless when ignored and starts working automatically the moment you upgrade. For Hobby readers, plan to recreate resources at the start of each article in the series — none of it is heavy, but it's worth knowing up front.
The Docker socket mount is needed for Lambda — LocalStack runs Lambda functions in sibling Docker containers, so it needs to talk to the host's Docker daemon. Without this mount, Lambda invocations fail with "Docker not available".

Add a .gitignore while you're here (the init/ folder is not ignored — it'll hold our checkpoint scripts and we want those committed):

.localstack/
*.zip
node_modules/
__pycache__ /

Step 4: Bring it up

docker compose up -d
docker compose logs -f localstack

You're looking for output that ends with:

LocalStack version: 2026.04.x
LocalStack build date: 2026-...
Ready.

Hit Ctrl+C to stop following logs — the container keeps running. If "Ready." doesn't appear within 30 seconds, jump to the troubleshooting section a bit further down.

Step 5: Install the AWS CLI and point it at LocalStack

The official AWS CLI talks to LocalStack the same way it talks to real AWS — you just point it at LocalStack's endpoint URL.

Install it if you don't already have it:

brew install awscli # macOS
sudo apt install awscli # Debian/Ubuntu
# Windows: installer from https://aws.amazon.com/cli/

Configure it once for LocalStack. Add these to your shell profile (~/.zshrc, ~/.bashrc, or PowerShell $PROFILE):

export AWS_ENDPOINT_URL=http://localhost:4566
export AWS_ACCESS_KEY_ID=test
export AWS_SECRET_ACCESS_KEY=test
export AWS_DEFAULT_REGION=us-east-1

PowerShell equivalent:

# Clear any persistent AWS_PROFILE so the env vars below take precedence
Remove-Item Env:AWS_PROFILE -ErrorAction SilentlyContinue

$env:AWS_ENDPOINT_URL = "http://localhost:4566"
$env:AWS_ACCESS_KEY_ID = "test"
$env:AWS_SECRET_ACCESS_KEY = "test"
$env:AWS_DEFAULT_REGION = "us-east-1"

The Remove-Item Env:AWS_PROFILE line matters on Windows: if you've previously set AWS_PROFILE=production (or anything else) at the user level, the AWS CLI will keep using that profile and ignore the env vars we just set. Clearing it first makes the LocalStack config win.

Reload your shell. Now aws s3 ls calls LocalStack instead of real AWS — same CLI, different endpoint.

Or use a dedicated `localstack` AWS profile (cleaner for anyone with day-job AWS)

The LocalStack AWS CLI integration docs recommend splitting the config across the two AWS files — keys in ~/.aws/credentials, region/output/endpoint in ~/.aws/config:

# ~/.aws/credentials
[localstack]
aws_access_key_id = test
aws_secret_access_key = test


# ~/.aws/config
[profile localstack]
region = us-east-1
output = json
endpoint_url = http://localhost.localstack.cloud:4566

Then either pin per command (aws --profile localstack s3 ls) or set AWS_PROFILE=localstack for the project folder. Your real-AWS profiles stay untouched.

The hostname localhost.localstack.cloud is the one LocalStack recommends — it resolves to 127.0.0.1 from the host and to the LocalStack container from inside Docker networks, so the same config works everywhere.

Optional shortcuts for `awslocal`

If you want to type awslocal s3 ls instead of aws s3 ls, two options — pick whichever is less hassle for you.

Option A — install the Python wrapper:

awslocal is a small Python package that sets the endpoint URL and placeholder credentials for you.

pipx install awscli-local # or: pip install awscli-local

Option B — define a shell function (no install):

Drop into ~/.zshrc or ~/.bashrc:

awslocal() {
  AWS_PROFILE= \
  AWS_ACCESS_KEY_ID=test \
  AWS_SECRET_ACCESS_KEY=test \
  AWS_DEFAULT_REGION=us-east-1 \
  aws --endpoint-url=http://localhost:4566 "$@"
}

The empty AWS_PROFILE= makes sure your day-job profile doesn't leak in. PowerShell equivalent:

function awslocal {
    $env:AWS_PROFILE = ""
    $env:AWS_ACCESS_KEY_ID = "test"
    $env:AWS_SECRET_ACCESS_KEY = "test"
    $env:AWS_DEFAULT_REGION = "us-east-1"
    aws --endpoint-url=http://localhost:4566 @args
}

This series uses awslocal in code samples because it's shorter. Every command works the same way with plain aws once you've exported the env vars above. Use whichever you prefer.

Step 6: Smoke test

Create your first bucket:

aws s3 mb s3://hello-localstack

You should see:

make_bucket: hello-localstack

Confirm it's there:

aws s3 ls


2026-05-03 12:34:56 hello-localstack

Put a file in it:

echo "hello from localstack" > hello.txt
aws s3 cp hello.txt s3://hello-localstack/
aws s3 ls s3://hello-localstack/

If your file shows up, LocalStack is healthy and you're ready for Week 1.

(If you went with the awslocal shortcut from Step 5, every aws here works the same way as awslocal — interchangeable.)

Step 7: Set up the checkpoint mechanism (init hooks)

State doesn't survive a restart on the free Hobby tier — persistence is paid-only. The good news: there's a free workaround that turns out to be even better for a tutorial series. Init hooks let LocalStack run shell scripts every time the container starts. Instead of saving state, we save a script that recreates state in seconds.

Init hooks are listed as "Included in Plans: Hobby, Base, Ultimate", so they work on the free tier.

The pattern across this series:

Each article ends with a small "save this as a checkpoint" script
Drop the scripts into your init/ready.d/ folder, named with a numeric prefix per article (01-part1-s3.sh, 02-part2-dynamodb.sh, ...)
Every docker compose up re-runs them in order — your resources rebuild themselves
Want to jump in at Part 4 without doing the previous ones? Add scripts 01 through 04 to your init/ready.d/ and you're set up. They're additive

The mount in our compose file (./init:/etc/localstack/init) is already there. Make the directory:

mkdir -p init/ready.d

Try it with a tiny example. Save this as init/ready.d/00-part0-smoke.sh:

#!/usr/bin/env bash
aws --profile localstack s3 mb s3://hello-localstack 2>/dev/null || true
echo "[bootstrap] part 0 — smoke bucket ready"

Make it executable:

chmod +x init/ready.d/00-part0-smoke.sh

Restart LocalStack to pick up the new mount and the script:

docker compose down
docker compose up -d
sleep 5

Confirm:

aws s3 ls
# 2026-05-03 12:34:56 hello-localstack

The bucket came back automatically. The 2>/dev/null || true makes the command idempotent — if a future run finds the bucket already there, the error is swallowed and the script keeps going.

That's the whole pattern. Each part of the series will give you one or two more lines to paste into the same folder.

Caveat for Lambda steps: Lambda functions need a .zip of your code, which is awkward to deploy through a one-off shell script. Articles that introduce Lambda (Parts 3 and 4) will walk through deployment manually, and the checkpoint scripts will skip the Lambda parts — only the supporting resources (buckets, tables, queues, IAM roles) get bootstrapped automatically.

Common pitfalls

A short list of things that have caught readers before:

"Auth token not set" or services refusing to start. Re-export LOCALSTACK_AUTH_TOKEN and run docker compose up -d --force-recreate. The variable has to be in the shell that runs docker compose, not just in the container.
Port 4566 already in use. Another LocalStack container is still running. Run docker ps -a | grep localstack and kill the duplicate.
aws: Could not connect to the endpoint URL. The AWS_ENDPOINT_URL env var isn't set in this shell. Re-source your profile (source ~/.zshrc / . $PROFILE) or re-export the four env vars from Step 5.
AWS CLI says "Unable to locate credentials". Same root cause — the four env vars from Step 5 aren't in scope. Re-source the profile or set them inline for one command. If you went with the awslocal shortcut, pipx ensurepath (then a new shell) usually fixes "command not found" for that route.
State doesn't survive a restart. Expected on the free Hobby tier (persistence is paid-only). Use the init-hooks pattern from Step 7 to recreate resources automatically.
Init script doesn't run. Check that the file is executable (chmod +x) and that init/ready.d/ is mounted at /etc/localstack/init/ready.d inside the container. docker compose exec localstack ls /etc/localstack/init/ready.d confirms.
Lambda invocations hang. The Docker socket mount is missing. Add /var/run/docker.sock:/var/run/docker.sock and recreate the container.
Slow first request after a cold start. Normal — LocalStack lazy-loads services on first request by default. The second call is instant.

Cleanup commands worth knowing

# Stop the container, keep state
docker compose stop

# Stop and remove the container, keep state on disk
docker compose down

# Stop, remove the container, AND wipe all state
docker compose down && rm -rf .localstack

The third one is the "I've got it into a weird state, start fresh" command. You'll want it.

A small habit worth picking up

Drop these in your shell config so you never start a session forgetting the token:

# ~/.zshrc or ~/.bashrc
ls-up() { (cd ~/projects/localstack-series && docker compose up -d); }
ls-down() { (cd ~/projects/localstack-series && docker compose down); }
ls-logs() { (cd ~/projects/localstack-series && docker compose logs -f localstack); }

Then ls-up, ls-logs, ls-down from anywhere on the machine.

What we'll wire up next

You've got LocalStack running, the AWS CLI pointed at it, a smoke-tested S3 bucket, and the init-hooks checkpoint pattern in place. The next part uses this setup for something real: a tiny photo uploader where the browser uploads files directly to S3 via a presigned URL — the same pattern Slack, Imgur, and most SaaS apps use for user-generated content. By the end of Part 1, you'll have a working endpoint and a single-page upload form you'll keep around for the rest of the series.

The full series

Part 0 — Start here: series intro and installing LocalStack (this article)
Part 1 — S3 locally: buckets, presigned URLs, and a tiny photo uploader (next)
Part 2 — DynamoDB locally: building a URL shortener data layer
Part 3 — Lambda + S3 events: an image thumbnailer pipeline
Part 4 — API Gateway + Lambda + JWT auth: a real HTTP API
Part 5 — SQS + SNS: a background job queue with a dead-letter queue
Part 6 — EventBridge + Step Functions: orchestrating a photo-processing workflow
Part 7 — Secrets Manager + KMS: handling secrets and encryption locally
Part 8 — Terraform (tflocal) + GitHub Actions: integration tests against LocalStack

Sources

Fedora 44 Is Out: GNOME 50, GCC 16, Ruby 4.0, and Wine That Just Works

Ali Shaikh — Wed, 29 Apr 2026 17:45:01 +0000

Fedora 44 dropped on 28 April 2026 after a couple of slip dates, and it's a chunkier release than the version number suggests. Major language toolchain bumps, a kernel that auto-enables NTSYNC for Wine and Steam, GNOME 50, KDE Plasma 6.6, and a switch to DNF5 as the PackageKit backend. Here's what actually matters, what might bite you on upgrade day, and the commands to get from 43 to 44 without surprises.

I've been running it in a Proxmox VM since the beta. The short version: it feels stable, the toolchain refresh is bigger than the headline suggests, and if you write code in Ruby or anything that touches LLVM, you'll want to read the toolchain section before you upgrade your work machine.

Let's go through what's new.

The Headline Versions

Quick rundown of the version bumps you'll notice straight away:

Linux kernel 6.19.14
GNOME 50 (Workstation default)
KDE Plasma 6.6 (with a new Plasma Login Manager)
GCC 16, with binutils 2.46, glibc 2.43, gdb 16
LLVM 22 across all sub-projects
Go 1.26
Ruby 4.0 (yes, a major version jump from 3.4)
CMake 4.x with ninja as the new default generator
MariaDB 11.8 as the default

A lot of those are "ah, fine" upgrades that just keep you current. Two of them - Ruby and CMake - are big enough that they deserve their own sections later.

Wine and Steam Got a Quiet Win

This is the change I think most users will actually feel, even if they don't know what NTSYNC is.

Linux kernel 6.14 introduced NTSYNC, a module that mimics Windows NT thread-synchronisation primitives natively in the kernel. Wine and Proton already had esync and fsync to do similar work, but NTSYNC sticks closer to how Windows actually behaves, which means fewer compatibility quirks and, in some cases, better framerates.

Fedora 44 wires it up so most users never have to think about it. When you install a package that recommends wine-ntsync (Wine itself, or the Steam package from RPM Fusion - both pull wine-ntsync in via RPM Recommends), Fedora drops a config into /usr/lib/modules-load.d/ntsync.conf. After reboot, the module is loaded. No modprobe, no editing config files, no chasing down a forum post.

You can check it's loaded with:

lsmod | grep ntsync

If you see output, you're good. If you don't, install Wine (or Steam from RPM Fusion) and reboot:

# Wine from the main repo
sudo dnf install wine

# Steam usually comes from RPM Fusion (enable it first if you haven't)
sudo dnf install steam

sudo systemctl reboot

Don't expect a 50% framerate jump in everything you run. Proton's existing esync/fsync paths cover most cases. What you will notice is that games which previously needed manual tweaks to work now run out of the box. That's a real quality-of-life win.

GNOME 50: Polish, Not Revolution

GNOME 50 isn't trying to reinvent itself. The big themes are accessibility, colour management, and remote desktop, and most of the visible changes feel like sanding down rough edges rather than throwing furniture around.

The accessibility work matters more than headlines suggest. Screen-reader behaviour is more consistent across apps, keyboard navigation in Settings has fewer dead ends, and the magnifier handles fractional scaling cleanly. If you support users who rely on these tools, the upgrade is a clean win.

Colour management gained proper per-monitor profile support, which is going to make photographers and designers very happy. Remote desktop now negotiates pixel formats more intelligently when you're connecting from a different display profile.

If you're on KDE instead, Plasma 6.6 is the headline. The new Plasma Login Manager replaces SDDM in the default install, and the first-run setup wizard is genuinely shorter than it used to be.

For Developers: The Toolchain Bumps

Here's where you need to pay attention if Fedora is your daily driver for work.

GCC 16 and LLVM 22

GCC 16 brings the usual mix of new C++26 features, better diagnostics, and improved optimisation passes. LLVM 22 ships across clang, lld, lldb, and the rest of the family. If you build native code, neither of these should break anything, but you'll want to rebuild any pre-compiled artifacts that ship with a Fedora 43 toolchain stamp.

Quick sanity check after upgrade:

gcc --version
clang --version

Both should report their new major versions. If you've got a project that pins to an exact GCC version in CI, mirror that in your local Containerfile rather than relying on the system compiler.

Ruby 4.0: The Big Jump

Ruby skipped from 3.4 straight to 4.0 in this cycle. The Fedora project's own change proposal calls it "the superior Ruby development platform" - their words, not mine, but the upgrade itself is the thing to plan for.

Most Ruby code that worked on 3.4 will work on 4.0, but a few rough edges to watch:

The soname bumped, so some C-extension gems need a rebuild against the new ABI. bundle install should handle this for most gems, but native gems with vendored binaries can stumble.
A handful of long-deprecated stdlib methods are gone. If your project still has Ruby 1.9-era code lurking, run your test suite before you ship.
Paths have shifted to fit the new version layout, so anything hard-coding /usr/lib64/ruby/3.4 or similar will need updating.

If you maintain a Rails app, the safest play is:

# In your project directory
rm -rf vendor/bundle
bundle install
bundle exec rspec # or your test command of choice

Don't blindly dnf upgrade your work machine if you've got a Ruby project with deadlines this week. Spin up a Toolbox container first:

toolbox create --release 44 ruby4-test
toolbox enter ruby4-test
cd ~/projects/my-rails-app
bundle install

That gives you Ruby 4.0 in an isolated environment without touching the rest of your system.

Go 1.26

Quieter upgrade. Most Go code is forward-compatible by design. The standard library has the usual round of additions, and the toolchain is faster on cold builds. Nothing here should bite you.

CMake 4.x with Ninja by Default

This one is small in code but worth flagging. Fedora 44 ships CMake 4.x, and the default generator is now ninja instead of make.

If your build scripts assume make is the generator (looking for Makefile, parsing make output, anything like that), you've got two options:

# Option 1: explicitly request make
cmake -G "Unix Makefiles" .

# Option 2: just use ninja and update your scripts
cmake .
ninja

Ninja is faster, especially on incremental builds, so most projects will benefit from making the switch permanent. But if your CI runs scripts that grep make output, fix them now rather than at 2am next Tuesday.

DNF5 Is the New Backend (For PackageKit Too)

Fedora 41 made DNF5 the default command-line tool. Fedora 44 finishes the job: PackageKit, the abstraction layer that GUI software updaters sit on top of, now uses DNF5 as its backend through libdnf5.

For most users this is invisible. GNOME Software still works, KDE Discover still works, the GUI updater on your aunt's laptop still works. What it means for you, if you're a developer or sysadmin, is that the legacy DNF4 code paths are now genuinely on borrowed time. If you've got internal tooling that shells out to dnf and parses output, this is a good week to make sure you're using the DNF5 syntax.

A few command differences worth noting:

# DNF4-style (still works for backward compatibility, but deprecated)
dnf list installed

# DNF5-style (preferred)
dnf list --installed

Most everyday commands (install, remove, upgrade, search) work identically. The differences are in the more specialised flags - dnf list installed is the classic example, superseded by dnf list --installed in DNF5. Old scripts won't break overnight, but if you're writing new tooling, follow the DNF5 conventions. The manual is genuinely well-written - man dnf5 is worth ten minutes if you live in this tooling.

Other Things Worth Knowing

A few smaller changes that don't deserve their own section but are worth a paragraph each.

MariaDB 11.8 is the new default. If you upgrade a server, run mariadb-upgrade afterwards to update the system tables. Don't skip this and then wonder why connections behave oddly.

Fedora Cloud images now use Btrfs subvolumes for /boot. Mostly invisible, but if you've got automation that assumes /boot is a separate ext4 partition, test before you redeploy.

OpenSSL added directory-hash support for ca-certificates. Faster certificate loading on systems with a lot of trust roots, no behaviour change for normal use.

Anaconda (the installer) now creates network profiles only for devices that are actually configured during install, instead of generating empty profiles for every NIC it sees. Cleaner, less to clean up.

QEMU 32-bit host builds are gone. Upstream deprecated them, Fedora followed. If you're running a 32-bit host (rare in 2026), this is your nudge.

Bootupd phase 1 begins. Boot loader updates start migrating to a unified mechanism. You won't notice unless something goes wrong, in which case the new tooling makes recovery cleaner.

How to Upgrade

If you're on Fedora 43, the upgrade path hasn't changed. Two ways to do it.

GUI: GNOME Software

Open Software, look for the "Fedora 44 is now available" banner, click the button, reboot when prompted. Easy mode. Works for the vast majority of desktops.

Terminal: dnf-plugin-system-upgrade

The reliable, scriptable path:

# 1. Make sure your current system is fully patched
sudo dnf upgrade --refresh
sudo systemctl reboot # only if there are kernel/glibc updates

# 2. Install the system-upgrade plugin if you don't have it
sudo dnf install dnf-plugin-system-upgrade

# 3. Download the upgrade
sudo dnf system-upgrade download --releasever=44

# 4. Trigger the upgrade reboot
sudo dnf system-upgrade reboot

Step 3 takes a while - it's downloading a few GB of packages. Step 4 reboots into a special offline upgrade mode that runs the actual transaction. Coffee break, come back to a Fedora 44 login screen.

If something goes wrong during step 4, you'll boot back into your old system. That's a feature, not a bug.

Should You Upgrade?

For most desktops, yes, and there's no reason to wait. The release cycle stability has been good and the headline features (NTSYNC, GNOME 50, KDE 6.6) are user-facing wins.

For work machines, give it a week. Not because Fedora 44 is risky in itself, but because your toolchain might have rough edges with the new compiler versions. If you run a Ruby shop, run your test suite in a Toolbox container before you commit.

For servers, give it longer. Fedora is a leading-edge distribution by design. If you need long support windows and conservative changes, this is what CentOS Stream and RHEL are for. Test in staging, run for a fortnight, then upgrade prod.

For my own laptop, I'm pulling the trigger this evening.

Try This Before You Upgrade

If you want to kick the tyres without committing, the easiest path is a live USB. Download the Workstation ISO from getfedora.org, write it to a USB stick, boot from it, and have a poke around. Nothing touches your existing install.

The simplest path is Fedora Media Writer - it handles the download and the writing in one GUI flow, and it works on Linux, macOS, and Windows. If you'd rather use the terminal:

# Replace /dev/sdX with your USB device - check with lsblk first!
# status=progress and oflag=direct are optional; they just give you feedback and force direct I/O
sudo dd if=Fedora-Workstation-Live-x86_64-44-1.5.iso of=/dev/sdX bs=4M status=progress oflag=direct

You can run NTSYNC checks, test GNOME 50, and confirm your hardware works without committing to the upgrade. Five minutes well spent.

That's Fedora 44. Solid release, decent toolchain refresh, real gains for anyone running Wine or Steam. Worth the upgrade for most users, worth the planning for developers, and worth the staging-environment patience for servers. Same Fedora cadence, same predictable spring release, same weeks of polish ahead.

Secure Self-Hosted OpenClaw AI Assistant: Step-by-Step Proxmox Template Guide with Tailscale Integration

Ali Shaikh — Sun, 01 Feb 2026 16:15:50 +0000

For the last few weeks, you could not escape seeing "Clawd" everywhere - the viral AI assistant that went from zero to 100,000 GitHub stars in just two months. Originally called Clawdbot, now rebranded as OpenClaw, it's an open-source AI that actually does things instead of just chatting. So this weekend, I set it up on my Proxmox homelab.

Now I have a digital assistant that I can communicate with through WhatsApp and Telegram. I asked it to choose its own name and persona, and now it is called Lyra. I gave it a dedicated email and GitHub account.

This guide walks through creating a reusable Proxmox template with all dependencies pre-installed. Clone the template, boot it, configure OpenClaw and Tailscale, and you're done.

What you get:

Debian 13 VM template with Node.js 24, OpenClaw CLI, and Tailscale pre-installed
Automatic installation via cloud-init (no manual package installs)
Secure HTTPS access via Tailscale Serve (no public exposure)
Progress indicators and status helpers baked in
~15 minutes from template creation to running AI assistant

GitHub: OpenClaw Proxmox Template Script

What is OpenClaw?

OpenClaw is an autonomous AI assistant that actually executes tasks:

Controls browsers (fill forms, scrape data, automate workflows)
Manages files on your system
Integrates with WhatsApp, Telegram, Discord
Runs terminal commands
All through a web UI or chat interfaces

Think of it as the plumbing layer that connects AI models (Claude, GPT-4o, Gemini) to real-world actions.

The catch: It's powerful, which means it needs proper isolation. Hence the dedicated VM approach.

Architecture

Your Browser
    │
    │ HTTPS (Tailscale Serve)
    ▼
OpenClaw VM (Proxmox)
    │
    ├─ Tailscale Serve → http://127.0.0.1:18789
    │
    ├─ OpenClaw Gateway
    │ ├─ Control UI
    │ └─ AI agent + channels
    │
    └─ Internet (outbound only)
        └─ WhatsApp/Telegram/AI APIs

Key design:

Gateway binds to loopback (127.0.0.1) only
Tailscale Serve proxies HTTPS from your Tailnet
No public exposure, no firewall rules needed
Messaging platforms connect over regular internet
If using Tailscale, the server is only accessible through your Tailscale network, ensuring secure, private access

Prerequisites

You'll need:

Proxmox VE 8.x (ensure it's updated: apt update && apt full-upgrade via SSH or web console)
At least 4GB RAM and 20GB storage available per VM (adjust based on workload)
SSH access to Proxmox (or use the web console as a fallback)
A Tailscale account (free tier works)
Your SSH public key
Cloud-init enabled in Proxmox (check under Datacenter > Options; if not, enable it)

Quick Tip: If you're new to Tailscale, start with their quickstart guide: Tailscale Docs.

Part 1: Create the Template

Download the Script

SSH into Proxmox and download the script directly to avoid copy-paste errors:

ssh root@your-proxmox-host
wget https://raw.githubusercontent.com/Ali-Shaikh/proxmox-toolbox/main/templates/openclaw/debian-13-openclaw-ready-template.sh -O debian-13-openclaw-ready-template.sh

Configure It

Open the script and modify these values:

nano debian-13-openclaw-ready-template.sh

Change these:

VMID=10010 # Template ID (pick any unused ID; check with `qm list`)
STORAGE="local-lvm" # Your Proxmox storage pool (verify with `pvesm list`)
MEMORY=4096 # 4GB RAM minimum
CORES=2 # CPU cores

CI_USER="debian"
CI_PASSWORD="$(openssl passwd -6 $(pwgen -s 16 1))" # Generate a strong password with pwgen or similar; install pwgen if needed: apt install pwgen

# Replace with YOUR SSH public key
echo "ssh-rsa AAAA... your-email@example.com" > ~/ssh.pub

Note: Homebrew (Linuxbrew) is installed via cloud-init for certain dependencies; it's not native to Debian but works fine for this setup.

Run It

chmod +x debian-13-openclaw-ready-template.sh
./debian-13-openclaw-ready-template.sh

The script will:

Download Debian 13 cloud image
Create a VM with UEFI support
Configure cloud-init to install Node.js, OpenClaw, Tailscale, pnpm, Homebrew
Convert it to a template

Time: ~2-3 minutes

Part 2: Deploy a VM

Clone the template (replace IDs if needed; check for conflicts with qm list):

qm clone 10010 201 --name openclaw-prod --full
qm start 201

Wait 5-7 minutes. Cloud-init is installing everything. You can watch progress:

# Get the VM's IP (once it boots)
qm guest cmd 201 network-get-interfaces

# SSH in and watch the install
ssh debian@<vm-ip>
sudo tail -f /var/log/openclaw-install.log

When you see "Installation Complete", you're ready.

Verify Installation

# Quick check
check-install-status

# Or check versions manually
cat /root/install-versions.txt

You should see Node.js 24, npm, pnpm, OpenClaw, Tailscale, and Homebrew all installed.

Part 3: Configure OpenClaw

Connect to Tailscale

Get an auth key from https://login.tailscale.com/admin/settings/keys (set to expire in 30-90 days for security).

sudo tailscale up --authkey=tskey-auth-YOUR_KEY_HERE
tailscale status # Verify connection

Set Up pnpm (Required for Skills)

pnpm setup
source ~/.bashrc

Run OpenClaw Onboarding

openclaw onboard --install-daemon

Critical configuration choices:

Question	Answer	Why?
Setup type	Local gateway (this machine)	Keeps everything self-contained.
Bind address	Loopback (127.0.0.1) - MUST BE LOOPBACK	Prevents direct exposure to LAN or internet; required for secure Tailscale proxying.
Authentication	Token (Recommended)	Stronger than alternatives for access control.
Expose via	Tailscale Serve	Secure Tailnet-only access without public exposure.

Important:

Never select "LAN (0.0.0.0)" for bind address
Do not select "Funnel" (exposes to public internet)
If you get a Tailscale binary warning, choose "No" and continue

Save Your Token

The wizard will display your gateway token. Save it:

cat ~/.openclaw/openclaw.json | jq -r '.gateway.auth.token'

Output: claw_abc123def456...

Write this down. You'll need it to access the UI.

Verify Tailscale Serve

tailscale serve status

Expected output:

https://openclaw-host.tail-scale.ts.net/ (Tailscale Serve)
|-- / proxy http://127.0.0.1:18789

If empty, configure manually:

sudo tailscale serve --bg --https=443 http://127.0.0.1:18789

Check Gateway Status

openclaw status
openclaw health
openclaw gateway logs # Watch for errors

Part 4: Access the UI

From any device on your Tailscale network:

# Get all access info in one command
openclaw-access-info

This shows your Tailscale URL and gateway token. Open the URL in your browser.

Example: https://openclaw-host.tail-scale.ts.net/

Enter your gateway token when prompted. If "allowTailscale": true is enabled in your config, you'll be auto-authenticated - see Configuration File for trade-offs.

Important: Based on common troubleshooting (though not explicitly in OpenClaw docs), when using Tailscale Serve, you may need to add "controlUi": { "allowInsecureAuth": true } to your config file to avoid "pairing required" WebSocket errors. This skips device pairing for compatibility but reduces security slightly; use only if needed and combine with strong token auth.

Configuration File

Your OpenClaw config lives at ~/.openclaw/openclaw.json:

{
  "gateway": {
    "port": 18789,
    "bind": "127.0.0.1",
    "tailscale": {
      "mode": "serve"
    },
    "auth": {
      "mode": "token",
      "token": "your-gateway-token-here",
      "allowTailscale": false // Set to true for auto-auth via Tailscale identity (convenience trade-off: trust your Tailnet fully or use token-only for stricter security)
    },
    "controlUi": {
      "allowInsecureAuth": true // May be needed for Tailscale Serve to fix WebSocket errors; skips device pairing (use with caution)
    }
  }
}

Key settings:

bind: "127.0.0.1" - Gateway only listens on loopback (required for Tailscale Serve)
tailscale.mode: "serve" - Tailnet-only access
auth.allowTailscale: false - Prefer token auth; enable true only if you fully trust your Tailnet (trade-off: easier access vs. potential risk if Tailnet is compromised)
controlUi.allowInsecureAuth: true - May be needed for Tailscale Serve - Skips device pairing to prevent "pairing required" WebSocket errors (trade-off: fixes compatibility but weakens pairing-based security; not explicitly required in OpenClaw docs but helps with common issues)

View your config:

cat ~/.openclaw/openclaw.json | jq

Add Chat Integrations

openclaw channels login
# Scan QR code with WhatsApp > Settings > Linked Devices (note: WhatsApp limits linked devices; check your account limits)

openclaw configure --section channels.telegram
# Add bot token from @BotFather

Discord

openclaw configure --section channels.discord
# Add bot token from Discord Developer Portal

Note: For LLM integrations (e.g., Claude, GPT-4o keys), add them securely via openclaw configure --section models. Store keys in environment variables or a secrets manager for best practices.

Troubleshooting

Tailscale Serve Not Working

Check Tailscale status:

tailscale status
sudo systemctl status tailscaled

Manually configure Serve:

sudo tailscale serve --bg --https=443 http://127.0.0.1:18789
tailscale serve status

Ensure HTTPS is enabled for your Tailnet:

Visit https://login.tailscale.com/admin/dns
Enable HTTPS if prompted

Can't Access from Remote Device

Verify both devices are on Tailscale:

tailscale status # Run on both devices

Test connectivity:

ping openclaw-host
curl -k https://openclaw-host.tail-scale.ts.net/health

WebSocket Errors: "pairing required" (1008)

Edit your config:

nano ~/.openclaw/openclaw.json

Add under the gateway section:

"controlUi": {
  "allowInsecureAuth": true
}

Restart the gateway:

systemctl --user restart openclaw-gateway

This skips device pairing for Tailscale compatibility.

Installation Seems Stuck

# Check cloud-init progress
sudo tail -100 /var/log/cloud-init-output.log

# Check OpenClaw install progress
sudo tail -100 /var/log/openclaw-install.log

# Current step
cat /var/run/openclaw-install-progress

pnpm Global Bin Error

If you see ERR_PNPM_NO_GLOBAL_BIN_DIR when installing skills:

pnpm setup
source ~/.bashrc
openclaw onboard --install-daemon # Retry

Proxmox-Specific Issues

VM won't boot: Check UEFI settings in Proxmox VM hardware tab; ensure BIOS is set to OVMF (UEFI).
No IP assigned: Verify DHCP in your Proxmox network bridge; restart VM if needed.

Reset Everything

Start over if needed:

openclaw gateway stop
rm -rf ~/.openclaw/
openclaw onboard --install-daemon

Diagnostic Commands

openclaw doctor # Comprehensive health check
openclaw status --all # Deep status
openclaw health # Quick health probe
openclaw gateway logs -f # Follow gateway logs

Security Considerations

OpenClaw can execute commands and access your system. Here's how to secure it:

Gateway Security

1. Loopback binding only

{ "gateway": { "bind": "127.0.0.1" } }

2. Token authentication

{
  "gateway": {
    "auth": {
      "mode": "token",
      "allowTailscale": false // Token-only is stricter; enable true only if Tailnet is highly trusted
    }
  }
}

3. Rotate tokens regularly (every 90 days)

openclaw configure --section gateway.auth

4. Monitor access logs

openclaw gateway logs | grep -i "error\|unauthorised\|failed"

Tailscale Security

Use Serve (Tailnet-only), not Funnel (public).
Generate auth keys with 30-90 day expiration; rotate them.
Enable firewall (e.g., UFW: sudo ufw enable and allow only necessary outbound traffic).

System Security

Regular updates:

sudo apt update && sudo apt upgrade -y
openclaw update

Consider automating via cron: crontab -e and add 0 2 * * * /usr/bin/apt update && /usr/bin/apt upgrade -y.

Backups:

Take Proxmox snapshots before changes: qm snapshot 201 pre-config --description 'Before updates'.

Also, backup config:

tar -czf openclaw-backup-$(date +%Y%m%d).tar.gz ~/.openclaw/

Best Practices:

Use token authentication
Enable Tailscale Serve (not Funnel)
Regular backups and updates
Monitor logs for issues
Rotate credentials periodically
Never expose gateway on 0.0.0.0
Never commit tokens to git
Install monitoring tools like Fail2Ban for SSH: sudo apt install fail2ban
For least privilege (e.g., running OpenClaw as non-root user), we'll cover it in a future article when we deep dive into securing it.
Consider sandboxing with Docker inside the VM for extra isolation.

Testing and Validation

After setup, run a smoke test:

Access the UI via Tailscale URL.
Send a test message via WhatsApp/Telegram: "Hello, what's the weather in Dubai?" (assuming LLM integration).
Verify browser control: Ask it to open a safe site and scrape non-sensitive data.
Check logs for errors: openclaw gateway logs.

If issues arise, use the troubleshooting section.

The Template Script (Full Code)

Here is the full code for reference (you can copy-paste this into a file if the wget fails, but using the raw URL is recommended):

#!/bin/bash

# ===== PRODUCTION-READY OPENCLAW PROXMOX TEMPLATE =====
# This script creates a Debian 13 VM template with OpenClaw pre-installed
#
# Author: Ali Shaikh <alishaikh.me>
#
# Features:
# - Cloud-init progress indication
# - Login warnings during initialization
# - Automatic pnpm setup
# - Status check commands
# - Access info helper command
# - Proper error handling
#
# ===== CONFIGURABLE PARAMETERS =====

VMID=10010
VM_NAME="debian-13-openclaw-ready-template"
STORAGE="local-lvm"
MEMORY=4096
SOCKETS=1
CORES=2
DISK_ADDITIONAL_SIZE="+15G"

IMAGE_URL="https://cloud.debian.org/images/cloud/trixie/latest/debian-13-generic-amd64.qcow2"
IMAGE_PATH="/var/lib/vz/template/iso/$(basename ${IMAGE_URL})"

CI_USER="debian"
CI_PASSWORD="$(openssl passwd -6 debian)" # For production, use a stronger password

# Replace with your actual SSH public key
echo "ssh-rsa " > ~/ssh.pub

# Download Debian 13 Cloud Image
if [! -f "${IMAGE_PATH}"]; then
  echo "Downloading Debian 13 Cloud Image..."
  wget -P /var/lib/vz/template/iso/ "${IMAGE_URL}"
else
  echo "Image already exists at ${IMAGE_PATH}"
fi

set -x

# Destroy existing VM if it exists
qm destroy $VMID 2>/dev/null || true

# Create VM
qm create ${VMID} \
  --name "${VM_NAME}" \
  --ostype l26 \
  --memory ${MEMORY} \
  --cores ${CORES} \
  --agent 1 \
  --bios ovmf --machine q35 --efidisk0 ${STORAGE}:0,pre-enrolled-keys=0 \
  --cpu host --socket ${SOCKETS} --cores ${CORES} \
  --vga serial0 --serial0 socket \
  --net0 virtio,bridge=vmbr0

# Import disk
qm importdisk ${VMID} "${IMAGE_PATH}" ${STORAGE}
qm set $VMID --scsihw virtio-scsi-pci --virtio0 $STORAGE:vm-${VMID}-disk-1,discard=on
qm resize ${VMID} virtio0 ${DISK_ADDITIONAL_SIZE}
qm set $VMID --boot order=virtio0
qm set $VMID --scsi1 $STORAGE:cloudinit

# Create snippets directory
mkdir -p /var/lib/vz/snippets/

# Create Cloud-Init configuration with proper YAML formatting
cat << 'CLOUDCONFIG' | tee /var/lib/vz/snippets/debian13_openclaw_ready.yaml
#cloud-config

package_update: true
package_upgrade: true

packages:
  - curl
  - wget
  - git
  - build-essential
  - procps
  - file
  - ca-certificates
  - gnupg
  - qemu-guest-agent
  - htop
  - vim
  - tmux
  - jq
  - cron

write_files:
  # Login status checker - runs on every login to show progress
  - path: /etc/profile.d/cloud-init-status.sh
    permissions: '0755'
    owner: root:root
    content: |
      #!/bin/bash
      # Check cloud-init and installation status on login

      RED='\033[0;31m'
      GREEN='\033[0;32m'
      YELLOW='\033[1;33m'
      BLUE='\033[0;34m'
      NC='\033[0m' # No Color

      # Check if cloud-init is still running
      if cloud-init status 2>/dev/null | grep -q "running"; then
        echo ""
        echo -e "${YELLOW}========================================${NC}"
        echo -e "${YELLOW} SYSTEM INITIALISATION IN PROGRESS${NC}"
        echo -e "${YELLOW}========================================${NC}"
        echo ""
        echo -e "${BLUE}Cloud-init is still running. Please wait...${NC}"
        echo ""
        echo "Monitor progress:"
        echo " sudo tail -f /var/log/cloud-init-output.log"
        echo " sudo tail -f /var/log/openclaw-install.log"
        echo ""
        echo "Check status:"
        echo " cloud-init status"
        echo " cat /var/run/openclaw-install-progress"
        echo ""
        echo -e "${YELLOW}Some commands may not be available yet!${NC}"
        echo ""
      elif [-f /var/run/openclaw-install-progress]; then
        PROGRESS=$(cat /var/run/openclaw-install-progress 2>/dev/null)
        if ["$PROGRESS" != "COMPLETE"]; then
          echo ""
          echo -e "${YELLOW}Installation in progress: $PROGRESS${NC}"
          echo "Run: sudo tail -f /var/log/openclaw-install.log"
          echo ""
        fi
      elif [-f /root/.openclaw-ready]; then
        # Only show ready message once per session
        if [-z "$OPENCLAW_READY_SHOWN"]; then
          echo ""
          echo -e "${GREEN}System is ready! Run: setup-openclaw.sh${NC}"
          echo ""
          export OPENCLAW_READY_SHOWN=1
        fi
      fi

  - path: /usr/local/bin/openclaw-access-info
    permissions: '0755'
    owner: root:root
    content: |
      #!/bin/bash
      echo "==========================================="
      echo "OpenClaw Access Information"
      echo "==========================================="
      echo ""
      echo "Tailscale Serve URL:"
      tailscale serve status 2>/dev/null | grep -o 'https://[^]*' || echo " Not configured - run: openclaw onboard --install-daemon"
      echo ""
      if [-f ~/.openclaw/openclaw.json]; then
        echo "Gateway Token:"
        cat ~/.openclaw/openclaw.json | jq -r '.gateway.auth.token' 2>/dev/null || echo " Not found"
        echo ""
        echo "Allow Tailscale Auth:"
        cat ~/.openclaw/openclaw.json | jq -r '.gateway.auth.allowTailscale' 2>/dev/null || echo " Not configured"
      else
        echo "OpenClaw not configured yet"
        echo "Run: openclaw onboard --install-daemon"
      fi
      echo "==========================================="

  - path: /usr/local/bin/setup-openclaw.sh
    permissions: '0755'
    owner: root:root
    content: |
      #!/bin/bash
      echo "======================================"
      echo "OpenClaw Setup Helper"
      echo "======================================"
      echo ""
      echo "Installed versions:"
      echo " Node.js: $(node --version 2>/dev/null || echo 'Not found')"
      echo " npm: $(npm --version 2>/dev/null || echo 'Not found')"
      echo " pnpm: $(pnpm --version 2>/dev/null || echo 'Not installed')"
      echo " OpenClaw: $(openclaw --version 2>/dev/null || echo 'Not installed - run: npm install -g openclaw@latest')"
      echo " Tailscale: $(tailscale version 2>/dev/null || echo 'Not found')"
      echo " Homebrew: $(brew --version 2>/dev/null | head -n1 || echo 'Not found - source ~/.bashrc first')"
      echo ""
      echo "Quick Start Guide:"
      echo "=================="
      echo ""
      echo "1. Connect to Tailscale:"
      echo " Get auth key: https://login.tailscale.com/admin/settings/keys"
      echo " sudo tailscale up --authkey=tskey-auth-YOUR_KEY"
      echo ""
      echo "2. Configure pnpm (required for skills):"
      echo " pnpm setup && source ~/.bashrc"
      echo ""
      echo "3. Configure OpenClaw:"
      echo " openclaw onboard --install-daemon"
      echo ""
      echo " Configuration choices:"
      echo " - Setup: Local gateway (this machine)"
      echo " - Bind: Loopback (127.0.0.1) <- REQUIRED"
      echo " - Auth: Token (Recommended)"
      echo " - Tailscale: Serve (for HTTPS)"
      echo ""
      echo "4. Get access info:"
      echo " openclaw-access-info"
      echo ""
      echo "5. Verify:"
      echo " openclaw status"
      echo " openclaw health"
      echo " tailscale serve status"
      echo ""
      echo "Docs: https://docs.openclaw.ai"
      echo ""

  # Progress checker command
  - path: /usr/local/bin/check-install-status
    permissions: '0755'
    owner: root:root
    content: |
      #!/bin/bash
      echo "========================================"
      echo "Installation Status Check"
      echo "========================================"
      echo ""

      # Cloud-init status
      echo "Cloud-init status:"
      cloud-init status 2>/dev/null || echo " Unable to check"
      echo ""

      # Progress file (cleared after reboot, so may not exist)
      if [-f /var/run/openclaw-install-progress]; then
        echo "Current step: $(cat /var/run/openclaw-install-progress)"
      fi

      # Ready marker (use sudo to check /root)
      if sudo test -f /root/.openclaw-ready 2>/dev/null; then
        echo "Status: READY"
      else
        echo "Status: IN PROGRESS (or checking permissions...)"
      fi
      echo ""

      # Installed versions (use sudo to read /root)
      if sudo test -f /root/install-versions.txt 2>/dev/null; then
        echo "Installed versions:"
        sudo cat /root/install-versions.txt | sed 's/^/ /'
      fi
      echo ""

      echo "Logs:"
      echo " sudo tail -f /var/log/openclaw-install.log"

  - path: /root/install-openclaw.sh
    permissions: '0755'
    owner: root:root
    content: |
      #!/bin/bash
      # Don't use set -e - we want to continue even if some commands fail
      exec > /var/log/openclaw-install.log 2>&1

      # Progress update function
      update_progress() {
        echo "$1" > /var/run/openclaw-install-progress
        echo ">>> PROGRESS: $1"
      }

      echo "=== OpenClaw Installation Started at $(date) ==="
      update_progress "Starting installation..."

      # Install Tailscale
      update_progress "Installing Tailscale..."
      curl -fsSL https://tailscale.com/install.sh | sh
      if command -v tailscale &> /dev/null; then
        echo "Tailscale version: $(tailscale version)"
      else
        echo "WARNING: Tailscale installation may have failed"
      fi

      # Install Node.js 24
      update_progress "Installing Node.js 24..."
      curl -fsSL https://deb.nodesource.com/setup_24.x | bash -
      apt-get install -y nodejs
      echo "Node.js version: $(node --version)"
      echo "npm version: $(npm --version)"

      # Install pnpm globally
      update_progress "Installing pnpm..."
      npm install -g pnpm@latest
      echo "pnpm version: $(pnpm --version)"

      # Setup pnpm global bin directory (required for OpenClaw skills)
      pnpm setup
      source /root/.bashrc 2>/dev/null || true
      # Also setup for debian user
      sudo -u debian bash -c 'pnpm setup' 2>/dev/null || true

      # Install OpenClaw via npm
      update_progress "Installing OpenClaw..."
      npm install -g openclaw@latest

      # Verify installation and add to PATH if needed
      if command -v openclaw &> /dev/null; then
        echo "OpenClaw version: $(openclaw --version)"
      else
        echo "OpenClaw not in PATH, checking npm global bin..."
        NPM_BIN=$(npm bin -g)
        if [-f "$NPM_BIN/openclaw"]; then
          echo "Found at $NPM_BIN/openclaw, creating symlink..."
          ln -sf "$NPM_BIN/openclaw" /usr/local/bin/openclaw
          echo "OpenClaw version: $(openclaw --version)"
        else
          echo "ERROR: OpenClaw installation failed"
          echo "NPM global bin: $NPM_BIN"
          ls -la "$NPM_BIN/" 2>/dev/null || echo "Cannot list npm bin directory"
        fi
      fi

      # Install Homebrew for debian user (for plugins)
      update_progress "Installing Homebrew (this may take a while)..."
      # Create the debian user's home if it doesn't exist
      mkdir -p /home/debian
      chown debian:debian /home/debian

      # Install Homebrew as debian user
      sudo -u debian bash -c 'NONINTERACTIVE=1 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"' || {
        echo "WARNING: Homebrew installation failed"
      }

      # Add Homebrew to debian user's PATH
      if [-d "/home/linuxbrew/.linuxbrew"]; then
        sudo -u debian bash -c 'echo "# Homebrew" >> ~/.bashrc'
        sudo -u debian bash -c 'echo "eval \"$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)\"" >> ~/.bashrc'
        echo "Homebrew installed successfully"
      else
        echo "WARNING: Homebrew directory not found after installation"
      fi

      update_progress "Finalising..."

      echo "=== Installation Complete at $(date) ==="
      echo "Node.js: $(node --version 2>/dev/null || echo 'FAILED')" > /root/install-versions.txt
      echo "npm: $(npm --version 2>/dev/null || echo 'FAILED')" >> /root/install-versions.txt
      echo "pnpm: $(pnpm --version 2>/dev/null || echo 'FAILED')" >> /root/install-versions.txt
      echo "OpenClaw: $(openclaw --version 2>/dev/null || echo 'FAILED')" >> /root/install-versions.txt
      echo "Tailscale: $(tailscale version 2>/dev/null || echo 'FAILED')" >> /root/install-versions.txt
      echo "Homebrew: $(sudo -u debian /home/linuxbrew/.linuxbrew/bin/brew --version 2>/dev/null | head -n1 || echo 'FAILED')" >> /root/install-versions.txt

      # Mark installation as complete
      update_progress "COMPLETE"
      touch /root/.openclaw-ready

      echo ""
      echo "Check /root/install-versions.txt for results"

runcmd:
  - systemctl enable qemu-guest-agent
  - systemctl start qemu-guest-agent
  - echo "INITIALISING" > /var/run/openclaw-install-progress
  - chmod +x /root/install-openclaw.sh
  - /root/install-openclaw.sh
  - chown debian:debian /usr/local/bin/setup-openclaw.sh
  - |
    cat >> /etc/motd << 'MOTD'

    ========================================
    OpenClaw Ready Template
    ========================================
    Created by Ali Shaikh
    https://alishaikh.me

    Pre-installed (globally):
      - Node.js 24 (NodeSource)
      - pnpm (latest)
      - OpenClaw CLI
      - Tailscale (official)
      - Homebrew (for plugins)

    Check installation: cat /root/install-versions.txt
    Check status: check-install-status
    Setup guide: setup-openclaw.sh
    Access info: openclaw-access-info (after configuration)

    MOTD
  - apt-get clean
  - apt-get autoremove -y

power_state:
  mode: reboot
  message: "Rebooting after OpenClaw installation"
  timeout: 30
  condition: True

CLOUDCONFIG

# Apply Cloud-Init configuration
qm set $VMID --cicustom "vendor=local:snippets/debian13_openclaw_ready.yaml"
qm set $VMID --tags debian-template,openclaw-ready,nodejs24,production
qm set ${VMID} --ciuser ${CI_USER} --cipassword "${CI_PASSWORD}"
qm set $VMID --sshkeys ~/ssh.pub
qm set $VMID --ipconfig0 ip=dhcp

# Convert to template
qm template ${VMID}

echo ""
echo "=========================================="
echo "PRODUCTION TEMPLATE CREATED SUCCESSFULLY"
echo "=========================================="
echo "Template ID: ${VMID}"
echo "Template Name: ${VM_NAME}"
echo ""
echo "Globally installed:"
echo " - Node.js 24 (NodeSource)"
echo " - pnpm (latest)"
echo " - OpenClaw CLI"
echo " - Tailscale (official)"
echo " - Homebrew (for plugins)"
echo ""
echo "DEPLOYMENT:"
echo " qm clone ${VMID} 201 --name openclaw-prod --full"
echo " qm start 201"
echo ""
echo "IMPORTANT: Wait 5-7 minutes for installation to complete"
echo ""
echo "CHECK INSTALLATION STATUS:"
echo " ssh debian@<vm-ip>"
echo " check-install-status # Quick status check"
echo " tail -f /var/log/openclaw-install.log"
echo " cat /root/install-versions.txt"
echo ""

Conclusion

You now have a production-ready AI assistant running on your own infrastructure, accessible from anywhere via WhatsApp, Telegram, or your browser, with zero public exposure. Your conversations and command history stay on your server, your files remain under your control, and you decide exactly what the AI can access. However, be clear about the privacy boundary: when you use cloud models like Claude, GPT-4o, or Gemini, your prompts and responses still go to Anthropic, OpenAI, or Google. Self-hosting OpenClaw gives you infrastructure control and data residency for everything except the LLM API calls. For true end-to-end privacy, you'd need to run local models via Ollama or LM Studio, though at the cost of performance compared to cloud models.

Before you start feeding your assistant sensitive data, remember what you've built: a system that can execute arbitrary commands, access files, and interact with external services. The security measures in this guide (loopback binding, token authentication, Tailscale isolation) are foundational, not exhaustive. Treat this like any other privileged service: rotate credentials regularly, monitor access logs, keep the system patched, and never expose the gateway publicly. The Proxmox snapshot feature is your friend - take one before major changes so you can roll back easily.

The Proxmox template makes this repeatable: clone it whenever you need a fresh instance, whether for testing new configurations or deploying assistants for different use cases. OpenClaw is still evolving rapidly (remember, it went from zero to 100k stars in two months), so expect frequent updates and new capabilities. Consider this deployment a learning platform first, a production tool second. Test destructively in clones, understand the permission model, and gradually increase what you trust it to do. Report issues or contribute on the GitHub repo.

Self-hosting means self-responsibility, but it also means you're not at the mercy of a vendor's API changes or pricing decisions. Join the community, experiment with custom skills, contribute back what you learn, and most importantly, keep your gateway token secure. Stay vigilant, stay curious, and enjoy having an AI that actually works for you. Happy automating! 🦞

References

Festive STEM Adventure: Building the Acebott ESP32 QD001 Robot Car with My Daughter

Ali Shaikh — Tue, 30 Dec 2025 20:43:59 +0000

Remember that traffic light controller I built for my daughter? Then the proximity sensor upgrade that turned it into a peek-a-boo game? Well, this Christmas, I thought: why not go bigger?

Enter the Acebott QD001 ESP32 4WD Robot Kit. Four wheels, motors, sensors, and an ESP32 brain. A proper robot car that she could actually drive around the living room. The kind of thing that would've cost hundreds a few years ago, now sitting at a fraction of that price.

I'll be honest—I was curious. Would a ready-made kit be as satisfying as building something from scratch? Would the quality hold up? And more importantly, could my 2-year-old actually get involved in putting this together?

Turns out, yes to all three.

What You Get in the Box

The Acebott QD001 arrives as a complete starter kit. No hunting for compatible parts across three different websites. No "oh, I forgot to order the motor driver" moments at 11 PM.

Here's what's included:

Electronics: ESP32 MAX V1.0 development board and L298N motor driver shield
Sensors: HC-SR04 ultrasonic sensor, line tracking sensors, and IR receiver
Mechanical: 4WD chassis with motors and wheels, acrylic chassis plates
Accessories: IR remote control, LED headlights, buzzer, battery holder (18650 batteries not included), screws, standoffs, and cables

The documentation is solid too. Clear assembly diagrams, wiring schematics, and example code for different functionalities. You can find it all on their documentation site. They also have YouTube videos for assembly.

Putting It Together

This is where it got fun. The assembly is genuinely straightforward—no ambiguous instructions or mystery screws left over at the end. The chassis uses an acrylic sandwich design: bottom plate, motors mounted with brackets, middle plate for the electronics, top plate for the sensors.

My daughter's contribution? She handed me screws (sometimes the right ones), held the screwdriver when I needed both hands, plugged in a few of the larger connectors, and helped push the wheels onto the motor shafts. That last bit required a bit of force, and she was genuinely proud when they clicked into place.

The motors mount with simple brackets. The motor driver board sits on standoffs in the middle. Wiring is colour-coded. Takes maybe 2-3 hours if you're going slow and letting a toddler "help."

The only slightly fiddly bit was routing all the wires neatly so they wouldn't get caught in the wheels. Small zip ties helped there.

The Upload Problem (and Solution)

Right, so the robot's built. Time to upload some code and make it move.

Except... it wouldn't upload. At all.

Arduino IDE kept throwing errors: "Failed to connect with the device" and "Wrong boot mode detected (0x13)." The ESP32 was booting normally instead of entering download mode to accept new firmware.

Here's the thing about the Acebott ESP32 MAX V1.0—it doesn't have a dedicated BOOT button like most development boards. When you want to upload code, you need to manually force it into download mode.

The fix: connect GPIO0 (labelled "00" on the board) to GND with a jumper wire, then press the RST button. This pulls GPIO0 low during reset, which is what triggers the bootloader.

I wrote up the full solution here if you run into the same problem. The short version:

Use a jumper wire to connect GPIO0 to GND
Press RST
Start your upload
Keep the jumper connected until upload completes
Remove jumper, press RST again to boot normally

Annoying? Yes. But once you know the trick, it's fine. Takes ten extra seconds per upload.

Making It Work Better

The kit comes with separate example programs—one for IR remote control, another for app control via WiFi. Both worked fine, but I wanted both control methods available at the same time. Why choose?

Combining them was straightforward. The IR receiver and WiFi stack don't interfere with each other. I merged the two examples into a single program that listens for commands from either source. Whichever one sends a signal first wins. Simple.

One limitation with the app control is that you need to connect to the car's WiFi network to control it. It works, but it's not ideal—you lose internet connectivity on your phone whilst connected. I'll explore creating a custom app in the future that might handle this better, perhaps using a different network architecture.

The other change I made was moving some of the larger constant arrays into PROGMEM. The example code had a bunch of lookup tables and configuration data sitting in RAM. On an ESP32, you've got plenty of flash storage—why waste precious RAM?

Not a massive difference for a project this size, but it's good practice. Keeps RAM free for runtime variables and buffers.

Troubleshooting the IR Remote

Getting the IR remote working turned into an interesting learning exercise. The remote uses infrared signals to communicate with the receiver on the robot—essentially flashing an IR LED in specific patterns that encode commands.

The protocol is fairly simple once you understand it. When you press a button, the remote sends a burst of IR pulses at 38kHz (the carrier frequency). The pulses are modulated to encode data: different combinations of short and long pulses represent different commands. The IR receiver on the robot demodulates this signal and outputs the raw timing data.

The Arduino IR libraries (like IRremote) handle the decoding. They measure the pulse widths and decode them into command codes. Each button on the remote has a unique code—forward might be 0xFF18E7, backward 0xFF4AB5, and so on.

My first issue was that pin 4 on the shield wasn't responding at all. No matter what I tried, the IR receiver wouldn't pick up signals. After checking the wiring multiple times, I moved the IR receiver connection to GPIO 27 instead. Worked immediately. Sometimes it's just a dodgy connection point on the board.

The other thing I wanted was to control the robot's lights with the remote—not just movement. To do this, I needed to know the exact codes each button sends. I flashed a simple debug sketch first that just printed out the raw IR codes to the Serial Monitor. Pressed each button on the remote, noted down the hex codes, then added those to the main program. The * button now toggles the headlights on and off. I captured the other button codes as well and stored them in the code for future upgrades—having those codes ready means I can easily map new functions later without running the debug sketch again.

It's a nice reminder that even with kits, you often need to go off-script to get exactly what you want.

The complete code will be available on my GitHub if you want to see the changes. Nothing ground-breaking—just sensible tweaks that made the robot more flexible.

Why Kits Like This Matter

There's something to be said for building everything from scratch. I still enjoy that. Picking components, writing custom code, troubleshooting weird hardware interactions.

But kits like the Acebott QD001 serve a different purpose. They lower the barrier to entry. Someone who's never touched robotics can open this box, follow the instructions, and have a working robot in an afternoon. That's powerful.

The cost is reasonable too. Around £50-60 depending where you buy it. That includes everything except batteries. Compare that to buying motors, chassis, driver boards, sensors, and a microcontroller separately—you'd spend more and have to figure out compatibility yourself.

And for kids? There's real value in seeing a pile of parts transform into something that moves and responds. My daughter doesn't understand motor drivers or PWM signals. But she absolutely understands that when she presses the button on the remote, the robot goes forward. Cause and effect. That's where learning starts.

These kits also teach you the fundamentals without getting lost in the weeds. You learn how motor control works, how sensors provide feedback, how wireless communication lets you control things remotely. The principles transfer to other projects.

Could I have built something similar from scratch? Sure. Would it have been as polished? Probably not. Would my daughter have cared? Definitely not.

Watching Her Play

The first time she made the robot move, her face lit up. Same expression as when she discovered the traffic light responded to the proximity sensor.

She's still working out the controls—forward is intuitive, turning less so. But she's figuring it out. Sometimes the robot crashes into furniture. Sometimes she just drives it in circles. Doesn't matter.

What matters is that she's experimenting. Testing what happens when she presses different buttons. Learning that her actions have predictable results. That's the foundation of computational thinking, even if she won't call it that for another decade.

And when she gets bored of just driving it around? The kit has line-following sensors and obstacle avoidance capabilities built in. Though I'll admit, the obstacle avoidance mode isn't particularly accurate—it's a bit hit and miss. That's something I'll explore improving, along with finding better control methods that don't require connecting to the car's WiFi. There's room to grow.

What's Next: Expansion Packs

One thing I appreciate about the Acebott ecosystem is the availability of expansion packs. The QD001 is designed to be modular—you're not stuck with what comes in the box.

Acebott sells several add-ons: robot arm packs, tank treads, shooting mechanisms, even solar panels and GPS modules. I've already picked up the QD002 Camera Expansion Pack for the next upgrade.

The QD002 adds an ESP32-CAM module to the robot, giving it vision capabilities. Real-time video streaming to your phone, image recognition, wireless image transmission—all the things that turn a simple remote-control car into something that can "see" its environment. It's designed for beginners to learn about camera integration and wireless streaming without getting overwhelmed by the hardware side.

I'll probably tackle that upgrade in a future article. Adding a camera opens up possibilities for autonomous navigation, line-following with visual feedback, or even facial recognition experiments. The robot's already got the processing power with the ESP32—might as well put it to use.

Final Thoughts

The Acebott QD001 isn't trying to be everything to everyone. It's a starter kit. It gives you a solid foundation and lets you build from there.

The quality is good. The motors have decent torque. The chassis is sturdy enough to survive a two-year-old's driving. The documentation is clear. And when you inevitably want to modify it—add more sensors, change the behaviour, integrate it with other systems—it's all ESP32-based. You're not locked into proprietary hardware.

Is it perfect? No. The lack of a BOOT button is annoying. The battery holder could be more secure. But these are minor gripes.

For anyone wanting to learn robotics, or teach it to someone else, or just have a fun weekend project, this kit delivers. It's accessible, educational, and actually works.

Plus, my living room now has a small yellow robot roaming around. That's worth something.

References & Resources

Product & Documentation:

Troubleshooting & Guides:

How to Enter Download Mode on Acebott ESP32 MAX V1.0

Related Projects:

More parent-engineer adventures to come.

Kubernetes 1.35: In-Place Pod Vertical Scaling Reaches GA

Ali Shaikh — Mon, 29 Dec 2025 20:48:39 +0000

Kubernetes v1.35, released on 17 December 2025, brings the long-awaited in-place pod vertical scaling feature to general availability (GA). This feature, officially called 'In-Place Pod Resize', allows administrators to adjust CPU and memory resources for running containers without recreating pods.

The Traditional Approach to Resource Scaling

Before this feature, scaling pod resources required deleting and recreating the pod:

1. Update deployment/pod specification with new resource values
2. Kubernetes terminates the existing pod
3. Scheduler creates a new pod with updated resources
4. Container runtime starts the new pod
5. Application initialises and becomes ready

This approach causes:

Dropped network connections
Loss of in-memory application state
Service disruption during pod restart
Extended downtime for applications with long startup times

What is In-Place Pod Vertical Scaling?

In-Place Pod Vertical Scaling enables runtime modification of CPU and memory resource requests and limits without pod recreation. The Kubelet adjusts cgroup limits for running containers, allowing resource changes with minimal disruption.

Development Timeline

v1.27 (April 2023): Alpha release
v1.33 (May 2025): Beta release
v1.35 (December 2025): General availability (stable)

The feature required 6+ years of development to solve complex challenges, including container runtime coordination, scheduler synchronisation, and memory safety guarantees.

Key Features in v1.35

1. Memory Limit Decrease

Previous versions prohibited memory limit decreases due to out-of-memory (OOM) concerns. Version 1.35 implements best-effort memory decrease with safety checks:

Kubelet verifies current memory usage is below the new limit
Resize fails gracefully if usage exceeds the new limit
Not guaranteed to prevent OOM, but significantly safer than forced decrease

Example:

# Previously prohibited, now allowed
resources:
  requests:
    memory: "512Mi" # Decreased from 1Gi
  limits:
    memory: "1Gi" # Decreased from 2Gi

2. Prioritised Resize Queue

When node capacity is insufficient for all resize requests, Kubernetes prioritises them by:

PriorityClass value (higher first)
QoS class (Guaranteed > Burstable > BestEffort)
Duration deferred (oldest first)

This ensures critical workloads receive resources before lower-priority pods.

3. Enhanced Observability

New metrics and events improve resize operation tracking:

Kubelet Metrics:

pod_resource_resize_requests_total
pod_resource_resize_failures_total
pod_resource_resize_duration_seconds

Pod Conditions:

PodResizePending: Request cannot be immediately granted
PodResizeInProgress: Kubelet is applying changes

4. VPA Integration

Vertical Pod Autoscaler (VPA) InPlaceOrRecreate update mode graduated to beta, enabling automatic resource adjustment using in-place resize when possible.

How It Works

Resize Policies

Containers specify restart behaviour for each resource type:

apiVersion: v1
kind: Pod
metadata:
  name: example
spec:
  containers:
  - name: app
    image: nginx:1.27
    resources:
      requests:
        cpu: "500m"
        memory: "512Mi"
      limits:
        cpu: "1000m"
        memory: "1Gi"
    resizePolicy:
    - resourceName: cpu
      restartPolicy: NotRequired # No restart for CPU changes
    - resourceName: memory
      restartPolicy: RestartContainer # Restart required for memory

Policy Options:

NotRequired: Apply changes without container restart (default for CPU)
RestartContainer: Restart container to apply changes (often needed for memory)

Applying Resource Changes

Use the --subresource=resize flag with kubectl (requires kubectl v1.32+):

kubectl patch pod example --subresource=resize --patch '
{
  "spec": {
    "containers": [
      {
        "name": "app",
        "resources": {
          "requests": {"cpu": "800m"},
          "limits": {"cpu": "1600m"}
        }
      }
    ]
  }
}'

Monitoring Resize Status

Check desired vs actual resources:

# Desired (spec)
kubectl get pod example -o jsonpath='{.spec.containers[0].resources}'

# Actual (status)
kubectl get pod example -o jsonpath='{.status.containerStatuses[0].resources}'

View resize conditions:

kubectl get pod example -o jsonpath='{.status.conditions[?(@.type=="PodResizeInProgress")]}'
kubectl describe pod example | grep -A 10 Events

Use Cases

1. Peak Hour Resource Scaling

Scale resources to match daily traffic patterns without service disruption. For example, scale up CPU/memory at 08:55 for business hours, then scale down at 18:05 after hours, eliminating the need for pod recreation.

2. Database Maintenance Operations

Temporarily increase CPU allocation for intensive maintenance tasks like VACUUM or REINDEX operations, then restore normal allocation once complete. This avoids pod restart and preserves warm database caches.

3. JIT Compilation Warmup

Provide additional CPU during application startup for JIT compilation warmup, then reduce allocation once the application reaches steady state. Particularly beneficial for Java applications with large codebases.

4. Cost Optimisation

Dynamically right-size resources to reduce waste by scaling down during low-traffic periods, reducing over-provisioning based on actual usage patterns, and implementing time-based scaling without pod recreation overhead.

Limitations and Constraints

Resource Support:

Only CPU and memory can be resized (GPU, ephemeral storage, hugepages remain immutable)
Init and ephemeral containers cannot be resized
QoS class cannot change during resize

Platform Requirements:

Not supported on Windows nodes
Requires compatible container runtime (containerd v2.0+, CRI-O v1.25+)
Cannot resize with static CPU or Memory manager policies

Important Behaviours:

Most runtimes (Java, Python, Node.js) require restart for memory changes
Updating Deployment specs does not auto-resize existing pods
Cannot remove requests or limits entirely, only modify values

Upgrading to Kubernetes v1.35

Prerequisites

Kubernetes v1.34.x cluster
kubectl v1.32+ client
Compatible container runtime

Upgrade Process

1. Add v1.35 repository to all nodes:

# On each node
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.35/deb/Release.key | \
  sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-1.35-apt-keyring.gpg

echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-1.35-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.35/deb/ /' | \
  sudo tee /etc/apt/sources.list.d/kubernetes-1.35.list

sudo apt update

2. Upgrade control plane:

sudo apt-mark unhold kubeadm
sudo apt install -y kubeadm=1.35.0-1.1
sudo kubeadm upgrade apply v1.35.0 -y
sudo apt-mark unhold kubelet kubectl
sudo apt install -y kubelet=1.35.0-1.1 kubectl=1.35.0-1.1
sudo systemctl restart kubelet

3. Upgrade workers (one at a time):

kubectl drain <worker-name> --ignore-daemonsets --delete-emptydir-data --force

ssh <worker-node>
sudo apt-mark unhold kubeadm kubelet kubectl
sudo apt install -y kubeadm=1.35.0-1.1 kubelet=1.35.0-1.1 kubectl=1.35.0-1.1
sudo kubeadm upgrade node
sudo systemctl restart kubelet

kubectl uncordon <worker-name>

4. Verify upgrade:

kubectl get nodes
# All nodes should show v1.35.0

Testing In-Place Resize

Basic CPU Resize Test

apiVersion: v1
kind: Pod
metadata:
  name: cpu-test
spec:
  containers:
  - name: nginx
    image: nginx:1.27
    resources:
      requests:
        cpu: "250m"
      limits:
        cpu: "500m"
    resizePolicy:
    - resourceName: cpu
      restartPolicy: NotRequired

Create pod and verify baseline:

kubectl apply -f cpu-test.yaml
kubectl get pod cpu-test -o jsonpath='{.status.containerStatuses[0].restartCount}'
# Output: 0

Resize CPU:

kubectl patch pod cpu-test --subresource=resize --patch '
{
  "spec": {
    "containers": [{
      "name": "nginx",
      "resources": {
        "requests": {"cpu": "800m"},
        "limits": {"cpu": "800m"}
      }
    }]
  }
}'

Verify no restart occurred:

kubectl get pod cpu-test -o jsonpath='{.status.containerStatuses[0].restartCount}'
# Output: 0 (unchanged)

kubectl get pod cpu-test -o jsonpath='{.spec.containers[0].resources.requests.cpu}'
# Output: 800m (updated)

Memory Decrease Test

# Create pod with high memory
kubectl apply -f memory-test.yaml

# Increase memory first
kubectl patch pod memory-test --subresource=resize --patch '
{
  "spec": {
    "containers": [{
      "name": "app",
      "resources": {
        "requests": {"memory": "1Gi"},
        "limits": {"memory": "2Gi"}
      }
    }]
  }
}'

# Decrease memory (new in v1.35)
kubectl patch pod memory-test --subresource=resize --patch '
{
  "spec": {
    "containers": [{
      "name": "app",
      "resources": {
        "requests": {"memory": "512Mi"},
        "limits": {"memory": "1Gi"}
      }
    }]
  }
}'

Best Practices

Testing: Verify application behaviour in non-production before enabling in production.

Resize Policies: Use NotRequired for CPU (usually safe), RestartContainer for memory (often requires restart).

Monitoring: Track Kubelet metrics for resize operations, alert on persistent PodResizePending conditions, monitor OOM events after decreases.

Resource Changes: Make incremental adjustments, avoid large jumps, verify current usage before decreasing limits.

Priority: Use PriorityClasses for critical workloads to ensure resize priority during capacity constraints.

Troubleshooting

Resize Rejected (stuck in PodResizePending): Check node capacity (kubectl describe node), verify QoS class compatibility, review kubelet configuration for static resource managers, ensure resize policy allows the change.

Unexpected Restarts: Memory changes often require restart regardless of policy. Use RestartContainer for memory, test application behaviour, monitor OOM events.

OOM After Memory Decrease: Verify current usage first (kubectl top pod), make gradual decreases, ensure application can release memory under pressure, consider using RestartContainer to force release.

Anti-Patterns: When NOT to Use In-Place Resize

Important: In-place pod resize is a powerful feature that can become an anti-pattern if misused. It should be reserved for specific edge cases, not used as a general scaling strategy.

Why It Can Be an Anti-Pattern

1. Violates Immutable Infrastructure: Kubernetes follows "cattle not pets" philosophy with ephemeral, replaceable pods. In-place resize makes pods mutable and long-lived, contradicting this design principle and reducing system resilience.

2. Breaks GitOps: Manual pod patching creates drift between Git repository and cluster reality. Next deployment overwrites manual changes, cluster state doesn't match version control, and you cannot reproduce environments from Git.

3. Configuration Drift: Pods in the same Deployment can have different resource allocations, causing inconsistent behaviour across replicas, difficult debugging, and unpredictable load distribution.

4. Reduces Failure Isolation: Encourages keeping pods alive longer rather than quick replacement, which can hide underlying issues like memory leaks and delay necessary restarts.

5. Better Alternatives Exist: Traffic spikes should use HPA (scale out), resource changes should update Deployments (declarative), and environment differences should use Kustomize/Helm.

Some Valid Use Cases

1. Stateful Applications with Expensive Startup: Databases with 15+ minute startup times (cache warming, index loading). Example: Scale up CPU for PostgreSQL VACUUM/ANALYZE operations, then scale down. Justified because recreation cost exceeds configuration drift cost.

2. Single-Instance Workloads: Legacy applications with shared file locks, non-distributed state, or connection pooling limitations. Justified because horizontal scaling isn't possible without full rewrite.

3. Emergency Production Incidents: Immediate relief for OOM conditions at 3am when proper solutions require hours of testing. Must be temporary (fix properly within 24 hours) and documented.

4. Predictable Temporary Spikes: Monthly report generation requiring 4x CPU for 2 hours. Automate with CronJobs to scale up/down on schedule. Justified because running 4x resources 24/7 wastes 96% of allocation.

5. Testing and Capacity Planning: Non-production experimentation to measure performance at different resource levels and determine optimal production sizing.

Future Enhancements

Kubernetes SIG-Node is working on:

Removing static CPU/Memory manager restrictions
Support for additional resource types
Improved safety for memory decreases (runtime-level checks)
Resource pre-emption (evict lower-priority pods for high-priority resizes)
Better integration with horizontal autoscaling

Conclusion

In-Place Pod Vertical Scaling addresses a long-standing Kubernetes limitation, enabling resource adjustments without service disruption. The v1.35 GA release brings production-ready capability with important improvements including memory decrease support, prioritised queuing, and enhanced observability.

While this feature enables dynamic resource management for stateful workloads and emergency scenarios, it should complement rather than replace traditional scaling patterns. Use HPA for traffic-based scaling, update Deployments for permanent changes, and reserve in-place resize for the specific edge cases where pod recreation cost genuinely exceeds operational complexity.