DEV Community: Alexander Smirnoff

From 444 Sessions to a Useful Shortlist: How I Used ChatGPT to Filter and Favorite AWS re:Invent Sessions

Alexander Smirnoff — Mon, 20 Jul 2026 20:08:43 +0000

Conference catalogs are useful until they become another backlog.

When I checked the AWS re:Invent 2026 catalog, it contained 444 sessions. I wanted every Level 300 and 400 session relevant to my work, but I did not want to read hundreds of cards and click the favorite button multiple times.

I used ChatGPT to turn the catalog into a profile-based shortlist and then add the selected sessions to my AWS event favorites. This article documents the complete workflow, including the prompts, validation checks, browser automation, and the part that initially failed.

The result in my case was:

444 sessions in the catalog
412 sessions after applying the Level 300 and 400 filter
89 sessions relevant to my work
30 sessions ranked as the highest priority

The numbers are only a snapshot from July 18, 2026. The catalog will change. The workflow is the reusable part.

Also, all of my favorites are not an agenda. They are a searchable shortlist. I will make the real agenda later, once AWS publishes the full schedule and session conflicts become visible.

Why a personal relevance profile comes first

The obvious first prompt would be: "Pick the best sessions for me."

That is too vague. ChatGPT may know your role from earlier conversations, but some details may be outdated, inferred, or simply wrong. If the model starts with a weak profile, it can produce a polished but generic list.

I first made ChatGPT state what it knew about me and explain how each fact should affect session selection. This separated confirmed information from assumptions before the catalog influenced the answer.

Use this as the first prompt in a new chat:

Before inspecting any conference catalog, validate what you currently know about me and create a Conference Relevance Profile.

Consult the personal context, memories, files, and conversation history that are legitimately available to you. Do not invent details or turn assumptions into facts.

Cover:

1. My current role and responsibilities
2. My strongest technical skills
3. Technologies, platforms, and programming languages I use
4. Architecture areas relevant to my work
5. My level of experience
6. Projects or problems I appear to be working on
7. Technologies I want to learn
8. Strategic interests that could affect conference-session selection
9. Topics that are probably irrelevant to me
10. Any preferences or constraints that should influence the selection

For every statement, label it as:

- Confirmed
- Reasonably inferred
- Unknown

Explain briefly how each confirmed or inferred item affects session relevance.

If important information is missing, ask no more than five focused questions. Finish with a concise Session Selection Profile that I can edit and approve.

Do not inspect the conference catalog yet. Wait for me to approve or correct the profile.

Review the result. Correct wrong assumptions and add missing interests. Do not skip this step just because the profile looks mostly right.

My approved profile prioritized:

AWS Serverless and event-driven architecture
Agentic AI and production AI engineering
Platform engineering and developer experience
Amazon ECS and container operations
Streaming, Kafka, Flink, DynamoDB, and real-time data
Media and sports workloads
Architecture trade-offs, resilience, security, and cost

I also made the exclusions explicit. Generic migrations, introductory sessions, SAP, Oracle, VMware, model training hardware, and narrow industry sessions were out unless they had a direct connection to my work.

Open the catalog and let the user handle authentication

Next, open the official AWS re:Invent event catalog.

If your ChatGPT version provides an integrated browser, tell it to use that browser. If AWS asks you to sign in, take control and complete the login yourself. Never give ChatGPT your password, MFA code, session cookie, or access token.

This prompt makes the boundary clear:

Use my approved Conference Relevance Profile for this task.

Open the official AWS re:Invent event catalog in the integrated ChatGPT browser. You are authorized to open the catalog, read its session content, apply catalog filters, and navigate public catalog pages.

If authentication is required, pause and let me sign in manually in the opened browser. Do not ask for or handle my password, MFA code, cookies, or tokens. Continue only after I confirm that login is complete.

Do not select or favorite anything yet.

Not every ChatGPT surface supports browser control or authenticated browsing. If yours does not, open the catalog in your normal browser and save or upload the HTML page. That route still works, but it needs an additional completeness check.

Verify that the catalog data is complete

This was the most important technical check in my run.

The saved HTML reported 444 events, but it contained only the first 50 rendered session cards. The site loads more cards on the client side, so parsing the file without checking the count would have silently ignored almost 90 percent of the catalog.

After the Level 300 and 400 filter was applied, the live catalog exposed 412 matching sessions. ChatGPT loaded all results and compared the extracted count with the number shown by the catalog.

Use this prompt with either the live catalog or an uploaded HTML file:

Inspect the complete conference catalog and verify the input before selecting sessions.

Requirements:

- Read the total session count shown by the catalog.
- Apply only Level 300 and Level 400 as mandatory filters.
- Load all results, including lazy-loaded or paginated cards.
- Compare the number of extracted cards with the number reported by the UI.
- If an uploaded HTML file contains only the initially rendered cards, do not treat it as the complete catalog.
- Use the integrated browser and the catalog's public client-rendered data where available.
- Report any count mismatch or visibility gap explicitly.
- Do not infer missing sessions and do not claim complete coverage unless the counts can be reconciled.

Stop before relevance scoring and show me the validation result.

Do not accept an answer such as "I reviewed the catalog" without counts. For dynamic web applications, the count is the evidence that the model actually reached the full result set.

Filter by hard requirements, then score relevance

Once the input was validated, I asked ChatGPT to apply two different kinds of criteria:

Hard filters: Level 300 or 400 only.
Relevance scoring: how directly each session matched my approved profile.

Keeping these separate matters. A relevant Level 200 session still fails the acceptance criteria. A Level 400 session is not automatically relevant just because it is advanced.

Here is the selection prompt:

Using the validated catalog and my approved Conference Relevance Profile, select all sessions relevant to me.

Acceptance criteria:

- Include only Level 300 or Level 400 sessions.
- Evaluate every session that passes the level filter.
- Use the title and full description, not keyword matching alone.
- Include a session only when you can explain its direct relevance to my role, current work, or stated learning goals.
- Exclude generic, introductory, vendor-marketing, and unrelated industry sessions.
- Keep borderline sessions out unless there is a concrete technical reason to include them.
- Do not recommend EKS merely because it is a platform-engineering topic. Include an EKS session only if it provides a useful comparison with my ECS work or covers a capability I need to evaluate.
- Deduplicate by session code.

For each selected session return:

- Session code
- Level
- Format
- Exact title
- Relevance category
- One short reason for inclusion

Then split the result into:

1. Reserve first: the strongest matches
2. Also relevant: useful sessions with lower priority

Finally report:

- Total catalog entries
- Entries passing the level filter
- Selected sessions
- Reserve-first sessions
- Excluded topic groups

Do not add favorites yet.

In my case, ChatGPT selected 89 sessions and put 30 into the first-priority group. It retained a few EKS sessions for an honest ECS-versus-EKS platform comparison, not because every container workload needs Kubernetes.

That distinction is useful. A recommendation system should help you test your current architecture decisions, not just mirror your existing preferences.

Review the shortlist before changing anything

Before touching the AWS event account, I reviewed the result by category:

Architecture, serverless, and event-driven systems
Agentic AI engineering
Platform engineering and ECS
Streaming, data, and media
Resilience, isolation, security, and cost

I also checked that every item had a session code. Titles can change slightly, and some titles look similar. The session code is the stable allow-list for the next step.

Use this validation prompt:

Validate the selected session list before any browser changes.

- Confirm that every item has one unique session code.
- Report duplicate codes, missing codes, or malformed codes.
- Confirm that every item is Level 300 or 400.
- Return a plain, space-separated allow-list of session codes.
- State the exact number of unique codes.

Do not open or modify my favorites yet.

Save this allow-list. It prevents the browser step from turning into an open-ended "click anything relevant" instruction.

Add the sessions with the integrated browser

Favorites modify your event account, even though the action is reversible. I used a separate prompt with a strict sequence: discover, validate, confirm, mutate, verify.

Open the official AWS re:Invent event catalog in the integrated ChatGPT browser.

If login is required, pause and let me sign in manually. Never request or handle my credentials, MFA code, cookies, or tokens.

Use the approved session-code allow-list from the previous step. Do not add any session that is not in that list.

Before making changes:

1. Load all catalog results.
2. Locate every selected session code.
3. Read the current favorite state for each one.
4. Report any missing or ambiguous code.
5. Make no changes if at least one code cannot be resolved.
6. Ask for one final confirmation and state the number of sessions that will be added.

After I confirm:

- Add only sessions from the allow-list.
- Skip sessions that are already favorites.
- Change one session at a time.
- Wait until the UI confirms the favorite state before continuing.
- Stop on the first action that cannot be confirmed.
- Do not unfavorite anything.

At the end, report four counts:

- Newly added
- Already favorited
- Missing
- Failed or unconfirmed

If the browser can perform the actions, keep the tab open until it reports completion. Do not interrupt a running batch just because the UI looks idle for a few seconds.

Fallback: run a reviewed script in your signed-in browser

My ChatGPT environment could read the catalog but could not operate the authenticated AWS session. The fallback was a small script executed in the developer console of my already signed-in browser.

This script does not need your password or call a private API directly. It uses the same favorite buttons shown in the page. It first loads all results, verifies that every requested session exists, asks for confirmation, skips existing favorites, and stops if a click cannot be confirmed.

Open the official catalog, sign in, apply the Level 300 and 400 filters, then open Developer Tools and select Console. Replace the example codes below with the allow-list generated for you.

Review the script before running it. Only run it on registration.awsevents.com. Browser console code has the same access to the page as you do.

(async () => {
  const codes = `
    ARC306 ARC314 ARC312
  `.trim().split(/\s+/);

  if (location.hostname !== "registration.awsevents.com") {
    throw new Error("Open the official AWS event catalog before running this script.");
  }

  const uniqueCodes = [...new Set(codes)];
  if (uniqueCodes.length !== codes.length) {
    throw new Error("The session-code list contains duplicates.");
  }

  const pause = ms => new Promise(resolve => setTimeout(resolve, ms));
  const cards = () =>
    [...document.querySelectorAll("li.catalog-result.session-result")];

  // Load every lazy-loaded result.
  for (let round = 0; round < 50; round++) {
    const more = [...document.querySelectorAll("button.show-more-btn")]
      .find(button => !button.disabled && button.offsetParent !== null);

    if (!more) break;

    const before = cards().length;
    more.click();

    for (let attempt = 0; attempt < 60; attempt++) {
      await pause(250);
      if (cards().length > before) break;
    }

    if (cards().length === before) {
      throw new Error("The catalog stopped loading additional sessions.");
    }
  }

  const cardFor = code =>
    document.getElementById(code.toLowerCase())
      ?.closest("li.catalog-result.session-result");

  const buttonFor = code =>
    cardFor(code)?.querySelector("button.favorite-session");

  const isFavorite = button => Boolean(
    button && (
      button.getAttribute("aria-pressed") === "true" ||
      /unfavorite/i.test(button.getAttribute("aria-label") || "")
    )
  );

  const missing = uniqueCodes.filter(code => !cardFor(code));
  if (missing.length) {
    throw new Error(`No changes made. Missing sessions: ${missing.join(", ")}`);
  }

  const toAdd = uniqueCodes.filter(code => !isFavorite(buttonFor(code)));

  if (!confirm(
    `Found all ${uniqueCodes.length} sessions. ` +
    `${toAdd.length} will be added and ` +
    `${uniqueCodes.length - toAdd.length} are already favorites. Continue?`
  )) {
    return;
  }

  const results = [];

  for (const code of uniqueCodes) {
    let button = buttonFor(code);

    if (!button) {
      results.push({ code, status: "favorite button missing" });
      break;
    }

    if (isFavorite(button)) {
      results.push({ code, status: "already favorite" });
      continue;
    }

    button.click();

    let confirmed = false;
    for (let attempt = 0; attempt < 25; attempt++) {
      await pause(200);
      button = buttonFor(code);
      if (isFavorite(button)) {
        confirmed = true;
        break;
      }
    }

    results.push({
      code,
      status: confirmed ? "added" : "not confirmed; stopped"
    });

    if (!confirmed) break;

    // Avoid sending a burst of UI actions.
    await pause(350);
  }

  console.table(results);

  const added = results.filter(result => result.status === "added").length;
  const existing = results.filter(
    result => result.status === "already favorite"
  ).length;
  const failed = results.filter(
    result => !["added", "already favorite"].includes(result.status)
  ).length;

  console.log({
    requested: uniqueCodes.length,
    added,
    alreadyFavorite: existing,
    failed,
    confirmedFavorite: added + existing
  });
})();

The selectors matched the re:Invent 2026 catalog on July 18, 2026. AWS can change the page structure at any time. If that happens, the script should stop rather than guess, but you will need to update the selectors.

The script is safe to rerun for the same list because it skips sessions that are already favorites.

Verify the final state

Do not treat a successful click as a successful batch. Reload the catalog, switch to the favorites view if available, and verify the session codes again.

You can ask ChatGPT to perform or guide the read-only verification:

Verify the final favorite state against my approved session-code allow-list.

- Reload the official catalog.
- Do not change any favorite state.
- Check every code in the allow-list.
- Report codes confirmed as favorites.
- Report codes that are missing or not favorited.
- Compare the confirmed total with the expected unique-code count.

Do not claim completion if the counts differ.

For large lists, export or copy the final table from the console. It gives you a useful audit trail if the browser loses its session halfway through.

What I would keep from this workflow

The useful part was not simply asking an AI to recommend sessions. Four controls made the result trustworthy:

The relevance profile was reviewed before the catalog was opened.
The number of extracted cards was checked against the catalog count.
Session codes became a fixed allow-list before account changes began.
Every favorite action had to be confirmed by the UI.

The catalog will grow, session titles will change, and AWS may update the UI. Rerun the selection when the schedule is closer to final. Keep the profile and prompts, but regenerate the shortlist from the current catalog rather than assuming an old list is still complete.

This turns several hours of catalog work into a repeatable review process while keeping the final decisions, authentication, and account changes under your control.

Amazon S3 Files: from Kafka to S3 via NFS

Alexander Smirnoff — Mon, 13 Apr 2026 16:24:24 +0000

AWS recently announced Amazon S3 Files and the obvious question was whether it actually makes day-to-day code simpler.

The interesting part is not the marketing line. The interesting part is this: can I take Kafka events, write them with normal filesystem calls like open() or write_bytes(), and still end up with objects in S3 without pushing SDK calls into the application code?

I built a small demo to check exactly that. The setup is simple: Confluent Cloud Kafka, AWS Lambda, AWS CDK, and an Amazon S3 Files mount. This page shows how the demo is wired, where CDK is still rough, and what the read cache looks like when you run the same workload twice.

What matters here

Amazon S3

S3 is still just object storage. It is durable, cheap, and scales well, but objects are not files. You do not get normal filesystem behavior. You cannot treat a bucket like a mounted directory and expect things like os.listdir(), append-style writes, or file-style access patterns to behave the way they would on a POSIX filesystem.

For a lot of workloads that is fine. For some, it is just friction.

Amazon EFS and the caching layer

Amazon EFS gives you a shared POSIX-compatible filesystem over NFS. Lambda, EC2, and ECS can all mount it and use plain filesystem calls.

The useful bit in this demo is the cache behavior. If a file is not already warm, EFS may need to pull it from backing storage first. Once it is in the SSD-backed cache, later reads are much faster.

Amazon S3 Files

Amazon S3 Files sits between those two worlds. You mount it over NFS, but the backing store is an S3 bucket. From the application point of view, you write files. Under the hood, those files are synced to S3. When you read them back, the first access may come from S3 and later accesses can come from the EFS cache.

That gives you a pretty nice mix:

S3 pricing and durability
normal filesystem calls instead of SDK-heavy application code
compatibility with the rest of the S3 ecosystem
cached reads for data that gets reused

There is one behavior that matters a lot: the cache import threshold. You can define rules per prefix so only files below a configured size are imported into the fast cache. Files above that threshold are still in S3, but they are not treated the same way on read. That threshold was the main thing I wanted to test.

What the demo does

The Confluent Cloud topic gets two kinds of synthetic events:

Class	Size	S3 Files prefix	Cache rule
`small`	~4 KB	`small/`	`sizeLessThan: 131072` (128 KB) → cached
`large`	~920 KB	`large/`	`sizeLessThan: 2097152` (2 MB) → cached

A consumer Lambda is triggered by Lambda Event Source Mapping (ESM) and writes each record to the S3 Files NFS mount at /mnt/events. A replayer Lambda, invoked on-demand, walks the mount, reads every file, and re-publishes events to a replay Kafka topic. It also returns the average read latency so the cache effect is visible.

The simplest experiment is to run the replayer twice against the same prefix:

first run -> cold read, files are fetched from S3
second run -> warm read, files come from cache

That is enough to see whether S3 Files behaves the way the docs imply.

Architecture

Infrastructure with AWS CDK

The stack is CDK v2 in TypeScript. The main limitation right now is that there are no L2 constructs for S3 Files yet. CloudFormation already has the resource types, so the practical approach is to use L1 CfnResource for the S3 Files pieces and move on.

The S3 bucket

S3 Files requires versioning. The rest is ordinary bucket setup.

const eventsBucket = new s3.Bucket(this, "EventsBucket", {
  encryption: s3.BucketEncryption.S3_MANAGED,
  blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL,
  versioned: true,           // required by S3 Files
  removalPolicy: RemovalPolicy.DESTROY,
  autoDeleteObjects: true,
  lifecycleRules: [{ expiration: Duration.days(90) }],
});

The S3 Files service role

S3 Files needs a role so it can read and write the bucket on your behalf. The trust relationship is a bit odd the first time you see it, so it is worth calling out.

const s3FilesServiceRole = new iam.Role(this, "S3FilesServiceRole", {
  assumedBy: new iam.ServicePrincipal("elasticfilesystem.amazonaws.com", {
    conditions: {
      StringEquals: { "aws:SourceAccount": this.account },
      ArnLike: {
        "aws:SourceArn": `arn:aws:s3files:${this.region}:${this.account}:file-system/*`,
      },
    },
  }),
});
s3FilesServiceRole.addToPrincipalPolicy(
  new iam.PolicyStatement({
    actions: ["s3:*"],
    resources: [eventsBucket.bucketArn, `${eventsBucket.bucketArn}/*`],
  }),
);

The S3 Files filesystem, mount targets, and access point

Since there is no higher-level CDK support yet, this part uses raw CloudFormation resources.

const s3FilesFs = new CfnResource(this, "S3FilesFileSystem", {
  type: "AWS::S3Files::FileSystem",
  properties: {
    Bucket: eventsBucket.bucketArn,
    RoleArn: s3FilesServiceRole.roleArn,
    AcceptBucketWarning: true,   // required: S3 Files warns about versioning costs
  },
});

const fileSystemId = s3FilesFs.getAtt("FileSystemId").toString();

// One mount target per AZ – Lambda must be in the same subnet.
const mountTargets = mountSubnets.map((subnet, i) => {
  const mt = new CfnResource(this, `S3FilesMountTarget${i}`, {
    type: "AWS::S3Files::MountTarget",
    properties: {
      FileSystemId: fileSystemId,
      SubnetId: subnet.subnetId,
      SecurityGroups: [mountTargetSg.securityGroupId],
    },
  });
  mt.addDependency(s3FilesFs);
  return mt;
});

// Access point: UID/GID 1000 matches the Lambda runtime user.
const s3FilesAccessPoint = new CfnResource(this, "S3FilesAccessPoint", {
  type: "AWS::S3Files::AccessPoint",
  properties: {
    FileSystemId: fileSystemId,
    PosixUser: { Uid: "1000", Gid: "1000" },
    RootDirectory: {
      Path: "/events",
      CreationPermissions: { OwnerUid: "1000", OwnerGid: "1000", Permissions: "755" },
    },
  },
});

Attaching the mount to a Lambda function

This is where the current CDK story is still rough.

const attachS3FilesMount = (fn: lambda.Function, write: boolean): void => {
  // FileSystemConfigs escape-hatch (see pitfalls section below)
  const cfnFn = fn.node.defaultChild as lambda.CfnFunction;
  cfnFn.addPropertyOverride("FileSystemConfigs", [
    {
      Arn: s3FilesAccessPoint.getAtt("AccessPointArn"),
      LocalMountPath: "/mnt/events",
    },
  ]);

  // Lambda must not start until mount targets are ready.
  for (const mt of mountTargets) cfnFn.addDependency(mt);

  // IAM: s3files permissions.
  const actions = ["s3files:ClientMount", "s3files:ClientRootAccess"];
  if (write) actions.push("s3files:ClientWrite");
  fn.addToRolePolicy(new iam.PolicyStatement({
    actions,
    resources: [`arn:aws:s3files:${this.region}:${this.account}:file-system/*`],
  }));

  // Secrets Manager + KMS for the Confluent credentials.
  confluentSecret.grantRead(fn);
  fn.addToRolePolicy(new iam.PolicyStatement({
    actions: ["kms:Decrypt", "kms:DescribeKey"],
    resources: ["*"],
  }));
};

The consumer Lambda with ESM

The consumer Lambda gets Kafka records from Lambda ESM and writes them straight to the mounted path.

const consumerFn = new lambda.Function(this, "ConsumerFn", {
  runtime: lambda.Runtime.PYTHON_3_12,
  handler: "handler.handler",
  code: bundle("./lambda/consumer"),
  vpc,
  vpcSubnets: { subnets: mountSubnets },
  securityGroups: [lambdaSg],
  timeout: Duration.minutes(5),
  environment: {
    MOUNT_PATH: "/mnt/events",
    SMALL_PREFIX: "small/",
    LARGE_PREFIX: "large/",
    SMALL_SIZE_THRESHOLD: String(128 * 1024),
  },
});
attachS3FilesMount(consumerFn, true);

// ESM with BASIC_AUTH (SASL/PLAIN) – no confluent-kafka library needed in Lambda.
new lambda.EventSourceMapping(this, "LambdaConsumer", {
  target: consumerFn,
  kafkaBootstrapServers: [kafkaBootstrapServer],
  kafkaTopic: KAFKA_TOPIC,
  batchSize: 100,
  startingPosition: lambda.StartingPosition.LATEST,
  provisionedPollerConfig: {
    minimumPollers: 1,
    maximumPollers: 5,
  },
  sourceAccessConfigurations: [
    {
      type: lambda.SourceAccessConfigurationType.BASIC_AUTH,
      uri: confluentSecret.secretArn,    // {username: api_key, password: api_secret}
    },
  ],
});

The consumer handler

Because ESM already handles polling, batching, and offset management, the Lambda code itself is boring in a good way. It decodes the record value, decides whether the payload is small or large, and writes a file to the mounted path.

import base64, logging, os, uuid
from datetime import datetime, timezone
from pathlib import Path

MOUNT_PATH      = os.environ.get("MOUNT_PATH", "/mnt/events")
SMALL_PREFIX    = os.environ.get("SMALL_PREFIX", "small/")
LARGE_PREFIX    = os.environ.get("LARGE_PREFIX", "large/")
SMALL_THRESHOLD = int(os.environ.get("SMALL_SIZE_THRESHOLD", str(128 * 1024)))


def _write(prefix: str, value: bytes) -> None:
    now      = datetime.now(timezone.utc)
    dir_path = Path(MOUNT_PATH) / prefix / now.strftime("%Y/%m/%d/%H")
    dir_path.mkdir(parents=True, exist_ok=True)
    (dir_path / f"evt-{uuid.uuid4().hex}.json").write_bytes(value)


def handler(event: dict, context):
    small = large = 0
    for partition_records in event.get("records", {}).values():
        for rec in partition_records:
            value = base64.b64decode(rec["value"])
            if len(value) < SMALL_THRESHOLD:
                _write(SMALL_PREFIX, value)
                small += 1
            else:
                _write(LARGE_PREFIX, value)
                large += 1

    return {"written_small": small, "written_large": large}

That is the whole point of the demo. The application code does not know anything about S3 APIs. It just writes bytes to a path.

The replayer handler

The replayer is equally simple. It walks the mounted directory, measures read_bytes(), and optionally pushes the content back to a replay topic. The latency number in the response is enough to see whether you are reading from cold storage or a warm cache.

def handler(event: dict, context):
    source_prefix = event.get("source_prefix", "")
    dry_run       = event.get("dry_run", True)

    root     = Path(MOUNT_PATH) / source_prefix if source_prefix else Path(MOUNT_PATH)
    producer = None if dry_run else _make_producer(_get_credentials())

    files_read = published = errors = 0
    total_read_ms = 0.0

    for fpath in sorted(root.rglob("*")):
        if not fpath.is_file() or fpath.suffix.lower() not in (".json", ".ndjson"):
            continue

        t0      = time.monotonic()
        data    = fpath.read_bytes()          # ← this is the measured operation
        read_ms = (time.monotonic() - t0) * 1000
        total_read_ms += read_ms
        files_read += 1

        if producer:
            for evt in _parse(fpath.suffix, data):
                producer.produce(REPLAY_TOPIC, value=evt)
                published += 1
        producer and producer.poll(0)

    producer and producer.flush()
    return {
        "files_read":  files_read,
        "published":   published,
        "avg_read_ms": round(total_read_ms / files_read if files_read else 0, 2),
        "dry_run":     dry_run,
    }

Run it twice against the same prefix and the difference is obvious.

# First run – cold cache
aws lambda invoke --function-name <ReplayerFnArn> \
  --cli-binary-format raw-in-base64-out \
  --payload '{"source_prefix": "small/2026/04/13/", "dry_run": true}' \
  /tmp/response.json && cat /tmp/response.json
# → {"avg_read_ms": 87.4, ...}

# Second run – warm cache
aws lambda invoke --function-name <ReplayerFnArn> \
  --cli-binary-format raw-in-base64-out \
  --payload '{"source_prefix": "small/2026/04/13/", "dry_run": true}' \
  /tmp/response.json && cat /tmp/response.json
# → {"avg_read_ms": 1.2, ...}

The annoying part: CDK L2 does not understand S3 Files yet

This was the part that ate the most time.

CDK already has the nice lambda.FileSystem abstraction, so at first glance it looks like mounting a filesystem into Lambda should be trivial. In practice it is only built for EFS access points. S3 Files access points use a different ARN namespace, so CDK rejects them before you even get to deployment.

My first attempt was the obvious one:

// ❌ This does not compile – FileSystemConfig expects an EFS ARN type
new lambda.Function(this, "ConsumerFn", {
  filesystem: lambda.FileSystem.fromEfsAccessPoint(
    s3FilesAccessPoint,   // CfnResource – wrong type
    "/mnt/events",
  ),
  ...
});

That path goes nowhere.

The working solution was to stop fighting the L2 and set FileSystemConfigs directly on the underlying CfnFunction. It is not pretty, but it works and still stays dynamic because the access point ARN is resolved at deploy time.

const cfnFn = fn.node.defaultChild as lambda.CfnFunction;
cfnFn.addPropertyOverride("FileSystemConfigs", [
  {
    Arn: s3FilesAccessPoint.getAtt("AccessPointArn"),   // IResolvable – resolved at deploy time
    LocalMountPath: "/mnt/events",
  },
]);

Once you bypass the L2, there is one more thing to remember: you also lose the automatic dependency wiring that CDK normally adds for EFS mounts. Without an explicit dependency on the mount targets, CloudFormation can create the Lambda before the NFS endpoint is ready. When that happens, the function fails during init with an NFS connection error.

for (const mt of mountTargets) cfnFn.addDependency(mt);

So yes, the escape hatch works, but it also means you need to handle the lifecycle details yourself.

Apply the sync rules after deployment

Another rough edge: ImportDataRules and ExpirationDataRules are not CloudFormation properties on AWS::S3Files::FileSystem. You cannot finish the full setup in one CDK deploy. The sync rules need to be applied after the stack is up.

FS_ID=$(aws cloudformation describe-stacks \
  --stack-name AmazonS3FilesAndLambdaStack \
  --query "Stacks[0].Outputs[?OutputKey=='S3FilesFileSystemId'].OutputValue" \
  --output text)

aws s3files put-synchronization-configuration \
  --file-system-id "${FS_ID}" \
  --import-data-rules '[
    {"prefix":"small/","trigger":"ON_FILE_ACCESS","sizeLessThan":131072},
    {"prefix":"large/","trigger":"ON_FILE_ACCESS","sizeLessThan":2097152}
  ]' \
  --expiration-data-rules '[{"daysAfterLastAccess":30}]'

This config does three things:

files under small/ smaller than 128 KB are imported into cache on first access
files under large/ smaller than 2 MB are also imported into cache on first access
cached data that has not been touched for 30 days is eligible for eviction

That second step is easy enough, but it is still a separate step, which is worth knowing before you try to wrap this in a clean IaC story.

Deploying

# Clone the repo
git clone https://github.com/<your-org>/<your-repo>
cd amazon-s3-files-kafka-lambda-demo

# Install CDK dependencies
npm install

# Bootstrap your account/region (once per account)
npx cdk bootstrap

# Deploy – provide your VPC name, secret ARN, and Confluent bootstrap server
npx cdk deploy \
  --context vpcName=my-vpc \
  --context confluentSecretArn=arn:aws:secretsmanager:eu-central-1:123456789012:secret:confluent/dev/api-key/my-cluster \
  --context kafkaBootstrapServer=pkc-xxxxx.eu-central-1.aws.confluent.cloud:9092

The Confluent Secrets Manager secret must look like this:

{
  "endpoint":   "pkc-xxxxx.eu-central-1.aws.confluent.cloud:9092",
  "username":   "YOUR_API_KEY",
  "password":   "YOUR_API_SECRET"
}

Producing test events

I used a synthetic producer so the demo can be run without any real source system.

export BOOTSTRAP_SERVERS=pkc-xxxxx.eu-central-1.aws.confluent.cloud:9092
export API_KEY=<your-api-key>
export API_SECRET=<your-api-secret>
export NUM_SMALL=1000  # ~4 KB each → routes to small/
export NUM_LARGE=1000  # ~920 KB each → routes to large/

uv run ./scripts/produce-demo-events.py

The payload sizes are deliberate. Small events are far below the 128 KB threshold. Large events are above 128 KB but below 2 MB, so they still match the larger cache rule.

Benchmarking

The numbers below came from a real run in eu-central-1. The producer sent 1 000 small events at around 4 KB and 1 000 large events at around 920 KB. The consumer Lambda wrote them to the mounted S3 Files path in batches of up to 100 records through Lambda ESM.

NFS write latency in the consumer Lambda

Write latency was measured around each write_bytes() call and published to CloudWatch as StatisticValues.

Size class	Payload	Avg write latency	Implied NFS throughput
`small`	~4 KB	11–15 ms	~0.3 MB/s (latency-bound – RTT dominates)
`large`	~920 KB	62–64 ms	~14 MB/s (throughput-bound – single NFS stream)

The small-file case is mostly round-trip overhead. Payload size barely matters there. The large-file case looks more like what you would expect from a normal throughput-bound transfer over NFS.

The CloudWatch metric definition:

_cw.put_metric_data(
    Namespace="S3FilesDemo",
    MetricData=[{
        "MetricName": "WriteLatencyMs",
        "Dimensions": [{"Name": "SizeClass", "Value": "small"}],
        "StatisticValues": {
            "SampleCount": float(len(latencies)),
            "Sum":         sum(latencies),
            "Minimum":     min(latencies),
            "Maximum":     max(latencies),
        },
        "Unit": "Milliseconds",
    }]
)

Using StatisticValues instead of individual data points lets you see p50/p99 alarms on accumulated batches without any CloudWatch custom metric quota concerns.

NFS read latency in the replayer Lambda - cold vs warm

The replayer reads each file with read_bytes() and records the latency. Run it twice against the same prefix and you get the cold-read vs warm-read comparison immediately.

# Cold read – files not yet in EFS SSD cache
aws lambda invoke --function-name <ReplayerFnArn> \
  --cli-binary-format raw-in-base64-out \
  --payload '{"source_prefix": "small/2026/04/13/", "dry_run": true}' \
  /tmp/r1.json && jq '{avg_small: .avg_read_ms_small, avg_large: .avg_read_ms_large}' /tmp/r1.json

# Warm read – same files, now cached in EFS
aws lambda invoke --function-name <ReplayerFnArn> \
  --cli-binary-format raw-in-base64-out \
  --payload '{"source_prefix": "small/2026/04/13/", "dry_run": true}' \
  /tmp/r2.json && jq '{avg_small: .avg_read_ms_small, avg_large: .avg_read_ms_large}' /tmp/r2.json

The response includes per-class averages:

// First invocation (cold)
{ "avg_read_ms_small": 94.3, "avg_read_ms_large": 187.6 }

// Second invocation (warm)
{ "avg_read_ms_small": 1.1,  "avg_read_ms_large": 4.8 }

Numbers above are representative; your latency depends on VPC-to-S3 distance, mount target AZ alignment, and EFS cache state at invocation time.

The important part is not the exact number. It is the shape of the result. The first read pays the fetch cost. The second read is dramatically faster because the working set is already warm.

What to look at in CloudWatch

Open CloudWatch → Metrics → S3FilesDemo and look at these four metrics together:

Metric	Stat	What to look for
`WriteLatencyMs`	p99	Tail latency spikes during large ESM batches
`ReadLatencyMs`	Average	Clear drop between cold and warm replayer runs
`FilesWritten`	Sum	Total files written in the selected time window
`FilesRead`	Sum	Confirms the replayer actually walked the whole prefix

S3 Files Pricing

	Price
High-performance storage*	$0.36/GB-mo
Data access
File reads from high-performance storage	$0.04/GB
File reads directly from S3 bucket**	FREE
File writes	$0.07/GB

* Pricing is quoted monthly and prorated by the hour.

Takeaways

S3 Files is useful when you want file semantics on top of S3. The main value here is not magic performance. It is simpler application code. The ingest path can just write bytes to a mounted path and let the backing service deal with S3.
Lambda ESM works well with Confluent Cloud in this setup. BASIC_AUTH was enough for SASL/PLAIN, and the consumer Lambda did not need to embed a Kafka client just to read records.
CDK support is still immature. You can make this work today, but expect to use L1 resources and escape hatches. This is not a clean all-L2 CDK experience yet.
Cold and warm reads are very different. That is the behavior to understand before using S3 Files in a real workload. For archive and replay style flows, that can be perfectly fine. For anything latency-sensitive, cache behavior is not a detail - it is the design constraint.

The full source code is available at https://github.com/dobeerman/amazon-s3-files-kafka-lambda-demo

Pause your Lambda: Building a Slack approval workflow with AWS Durable functions

Alexander Smirnoff — Sun, 18 Jan 2026 15:53:57 +0000

1. The "Stateful" problem in serverless

We all love AWS Lambda. It scales to zero, handles massive traffic spikes without breaking a sweat, and best of all, you only pay for the milliseconds your code runs. It is the perfect tool for event-driven tasks: "When this happens, do that."

But there is one scenario where standard Lambda functions struggle: Waiting.

Imagine you are building an approval workflow. A user asks for permission to deploy to production. You send a message to a Slack channel asking an admin to click "Approve" or "Deny."

Now, what does your code do?

In the traditional serverless world, you had two main options:

The "Bad" Way: You keep the Lambda running with a while loop, checking a database every few seconds to see if the admin clicked the button.
- Result: You are paying for a server to sit idle. If the admin takes an hour (or goes to lunch), your Lambda times out.
The "Hard" Way: You use AWS Step Functions. You create a state machine (in JSON or YAML) that handles the wait and triggers a different Lambda when the task is done.
- Result: It works well, but your logic is split. Half is in your TypeScript code, and half is in infrastructure configuration. It adds cognitive load and makes testing harder.

Enter AWS Lambda Durable functions.

AWS now allows us to write "Durable" execution logic directly in our code. We can tell a Lambda function to pause execution, save its state, and go to sleep. When an event happens (like a button click) hours or days later, it wakes up exactly where it left off.

No idle server costs. No complex JSON state machines. Just clean, readable TypeScript.

2. The project: A Slack approval bot

To demonstrate this power, I built a simple Email Approval Bot for Slack.

It’s a classic "Human-in-the-Loop" workflow that usually requires complex infrastructure. With Durable Functions, it becomes a single, readable story.

The Flow

Here is exactly what happens when you run the demo:

Initiation: A user types a slash command in Slack: /approve user@example.com.
Validation: A generic Lambda (Slack proxy) verifies the request came from Slack and kicks off our Durable Function.
The Durable logic: Our core function (Approval processor) starts running:
- It validates the email format.
- It generates a unique correlation ID.
- It sends a message to a secure Slack channel using the Slack API, adding "Approve" and "Deny" buttons.
The "Magic" pause: The function hits a line of code that says waitForCallback. It stops running. It costs $0 while waiting.
The interaction: An admin sees the message in Slack and clicks "Approve."
The resume: Slack sends the click event to our API. A tiny helper Lambda receives the click and signals our paused function.
Completion: The Durable Function "wakes up", remembers the original email and correlation ID, handles the approval, and sends a final confirmation message to Slack.

The beauty of this is that steps 3 through 7 are defined in one single file, reading from top to bottom.

Let’s look at how we build it.

3. Crucial concept 1: The "Durable" wrapper

The first thing you'll notice is that this isn't a complex framework with a dozen configuration files. It looks like standard TypeScript.

The only "magic" is a higher-order function that wraps your handler.

src/approval-processor.ts

import { type DurableContext, withDurableExecution } from "@aws/durable-execution-sdk-js";

const run = async (event: LambdaEvent, ctx: DurableContext) => {
    // Your logic goes here
};

// This is the key line!
export const handler = withDurableExecution(run);

By wrapping your run function with withDurableExecution, you are telling the AWS SDK: "Hey, handle the state for me. If I pause, save my spot. if I resume, load my variables."

4. Crucial concept 2: The "Save points" (`ctx.step`)

In a normal Lambda, if the function stops, your variables die. disappears into the void.

In a Durable function, we use ctx.step. Think of this as a "Save point" in a video game.

// Validation logic wrapped in a step
const validationResult = await ctx.step("validate-email", async () => {
    const email = event.text?.trim() || "";
    // ... validation logic ...
    return { success: true, data: email };
});

const correlationId = await ctx.step("get-correlation-id", async () => `corr-${randomUUID()}`);

If this function runs, pauses for 3 days, and then wakes up:

It skips the code inside the async () => { ... } block because it already has the result saved.
It re-assigns the saved validationResult and correlationId variables instantly.

This guarantees that your random IDs don't change and your heavy computations don't run twice.

5. Crucial concept 3: The magic pause (`ctx.waitForCallback`)

This is the superpower. We want to send a message to Slack and then stop existing until someone clicks a button.

const slackActionResult = await ctx.waitForCallback(
    "approval",
    async (callbackId, _ctx) => {
        // This code runs JUST BEFORE pausing.
        // We attach the 'callbackId' to the Slack button.
        await slackModule.postMessage(event.response_url, {
            blocks: [
                {
                    type: "button",
                    action_id: "approve_button",
                    value: callbackId, // <--- THE KEY!
                    text: { type: "plain_text", text: "Approve" }
                }
            ]
        });
    },
    { timeout: { minutes: 5 } } // Auto-fail if no one clicks in 5 mins
);

What happens here?

callbackId: The SDK generates a secret, unique ID for this specific pause.
Send: We send that ID to Slack (hidden in the button's value property).
Halt: The function exits. The Lambda stops billing. The state is saved to the backend storage.

6. Crucial concept 4: Waking up (The action handler)

So, the Lambda is asleep. How do we wake it up?

When the admin clicks "Approve" in Slack, Slack calls our API Gateway. We need a tiny, standard Lambda to receive that webhook and forward the signal.

src/approval-action-handler.ts

import { LambdaClient, SendDurableExecutionCallbackSuccessCommand } from "@aws-sdk/client-lambda";

// ... inside the handler receiving the Slack webhook ...

// 1. Extract the callback ID we sent earlier
const payload = JSON.parse(body.payload);
const action = payload.actions[0];
const callbackId = action.value; 

// 2. Tell AWS Lambda: "Resume the function waiting for this ID!"
await lambda.send(
    new SendDurableExecutionCallbackSuccessCommand({
        CallbackId: callbackId,
        Result: JSON.stringify({ action_id: "approve_button" }),
    }),
);

That's it!

The standard Lambda gets the click.
It calls SendDurableExecutionCallbackSuccessCommand.
Our "Durable" Lambda wakes up, the await ctx.waitForCallback line finishes, and it returns the result we passed ().

7. Why this matters to you

If you have ever built this workflow using Step Functions, you know the pain:

You need a CDK/SAM definition for the State Machine.
You need one Lambda to start the task.
You need another Lambda to handle the logic.
You need another Lambda to receive the callback.
"Passing data" between them involves wrestling with JSON paths.

With Durable functions:

It's one file.
It's standard TypeScript.
It handles state, retries, and waiting natively.

This is a massive leap forward for developer experience in the AWS ecosystem. It brings the power of "Workflow as Code" (popularized by tools like Temporal) directly into the native Lambda environment.

Ready to try it?
Clone the repo and deploy it to your own AWS account to see the magic in action.

GitHub repository

Happy Coding!