DEV Community: Guillaume Marchand

Identifying speakers by voice in live streaming with AWS

Guillaume Marchand — Thu, 06 Aug 2026 17:41:37 +0000

Introduction

During a televised debate, a false claim travels faster than its correction. Newsrooms that analyze live need two things at once: what was said, and which speaker said it. Amazon Transcribe answers the first question well. Identifying the speaker by voice is harder than it looks, and it is the subject of this post.

Live processing imposes constraints that batch processing ignores. Amazon Transcribe finalizes a speech segment about every five seconds, while an automated analysis runs several AI agents for two to five minutes. Naming the author of each segment has to fit in a budget of a few hundred milliseconds. Speaker turns sometimes last under a second.

Three failures are possible, and they do not cost the same. Not knowing who speaks is inconvenient. Answering too late is useless. Attributing a sentence to the wrong person is worse than attributing nothing.

Attributing a claim to someone

A verified claim without an author loses most of its journalistic value. “Unemployment fell by three points” carries different weight depending on whether a candidate, a minister, or an invited economist says it. The journalist needs to attach each claim to a person. They also need to follow that person from one broadcast to the next, to measure how consistent their statements are.

Amazon Transcribe provides diarization: the service segments the stream and labels speaker turns. That information is useful, and it is not sufficient for this use case.

Three limits of diarization alone

Diarization assigns local labels. It tells you that two passages come from two different people within one stream. It does not tell you who they are. Nothing makes a given speaker keep the same label for the whole stream either, since a label can be reassigned or split mid-session. And those labels do not survive the end of the stream, because they are renumbered on the next one.

The three limits compound. A participant becomes “speaker 1” on Monday and “speaker 3” on Thursday, with nothing linking the two. Comparing what one person said across several shows then takes manual reconciliation, which is what live processing rules out.

The first approach: inferring names from text

The first version of the identification named speakers without analyzing their voices. An AI agent “Context” on Bedrock AgentCore Runtime kept every segment of the session in Amazon Bedrock AgentCore Memory. It exploited a regularity of talk shows: at the opening, each participant introduces themselves or is introduced by the host.

From those sentences, the “Context” agent linked the anonymous diarization label to a name, then propagated that link across the rest of the broadcast. The mechanism is still readable in that sub-agent prompt. It records facts of the form “speaker 0 in session S is the first candidate”, then tries to relate any new label to the facts already stored.

That approach worked often, and that is the problem. The name came from a language model inference over text, not from a measurement on the signal. Nothing made the same conclusion reproducible from one run to the next, and no value quantified the confidence.

Failure was also silent. A missing introduction, an ambiguous phrasing, or a guest arriving mid-show, and the agent produced a plausible name rather than no name. A journalist cannot publish an attribution on that basis, because they cannot know which one to verify. The decision needed to be reproducible, to carry a score, and to be computed on the voice itself.

Solution overview

A voice fingerprint answers all those limits. It describes the timbre as a vector, independent of the show where it was computed. It is the principle of a fingerprint, applied to the speech signal. Two nearby vectors designate the same voice, across shows and across weeks. The platform compares each finalized speech segment against a registry of known voices, and attaches either a name, a stable anonymous identifier, or an explicit abstention.

That choice moves the problem rather than removing it. It introduces persistent biometric data, with the obligations that come with it, and a new risk: attributing a sentence to the wrong person. For a newsroom, that error is worse than no attribution, and the two rules in the next section come from it.

Two design rules before any optimization

The system prefers silence to error. When the resemblance to a known voice is insufficient or ambiguous, it assigns “no name”. It produces a stable anonymous label instead, such as “voice 12”, which stays the same for the whole show and for later shows. The journalist sees that the same person is speaking, without being told who. An operator can name that voice afterwards, and the history already produced becomes named.

Recognition never blocks verification. If identification fails, exceeds its time budget, or could not start, the segment still goes to the AI agents. It then carries an identity marked as degraded, along with the reason. A pipeline whose optional link can interrupt the main link is not usable live.

The architecture

The diagram that follows traces one audio stream from the sender through the streaming server to the registry and the verification agents. Read it left to right: the sender feeds a WebSocket connection, the server splits the audio between transcription and identification, and the identity joins the segment before dispatch.

Figure 1. Speaker identification architecture, from audio capture to the fingerprint registry.

An FFmpeg process decodes the source and emits signed 16-bit little-endian pulse-code modulation (PCM) audio, 16 kHz mono. It is pushed over a WebSocket connection in 100 millisecond frames.

The streaming server is a container running on AWS Fargate, in an Amazon Elastic Container Service (Amazon ECS) cluster. That container does four things. It relays the audio to Amazon Transcribe Streaming and receives partial then finalized results, with their native time boundaries. It keeps recent audio in a 40 second rolling buffer, one per session. It runs two Open Neural Network Exchange (ONNX) models on CPU, and it compares the resulting fingerprint to the registry of known voices.

The buffer holds 40 seconds for a reason. That covers the 30 second cap observed on finalized results, plus the two second widening described in the section on going from signal to identity, plus delivery lag. The cost is bounded: 40 seconds of 16 kHz 16-bit mono audio is 1.28 MB per session.

Amazon Transcribe Streaming is used, and it is the only transcription engine in the system. It produces the text, the partial results displayed to the journalist immediately, the segmentation of speaker turns, and their time boundaries. What the system does not use is the diarization the same service offers alongside transcription.

The platform needs to follow a speaker from one broadcast to the next and to attach a name. Diarization labels, whether from Amazon Transcribe or from any other model, are anonymous identifiers local to a single inference. They distinguish speakers within one stream but do not name them and do not persist across streams. The fingerprint registry solves both: it carries names, and it survives sessions.

Two models run inside the container, and neither provides identity. Segmentation uses pyannote segmentation-3.0 under the MIT license. It signals where there is voice, where the speaker changes, and where two people speak at once. Those are segmentation signals: they tell the fingerprint extractor where to cut, not who speaks. The pyannote speaker classes are just as anonymous and local as the Transcribe spk_N labels, and the system reads none of them. The fingerprint itself uses WeSpeaker ResNet34-LM under the Apache 2.0 license, which produces a 256 dimension vector. Identity comes from comparing that vector against the registry, and from nowhere else.

Both artifacts are published to a versioned Amazon Simple Storage Service (Amazon S3) bucket, and the model version is recorded with every registry entry. A fingerprint computed by one model is comparable only to fingerprints from the same model, and that constraint has to stay verifiable months later.

Running inference inside the container avoids an extra network hop on a time-constrained path. Inference is CPU-bound, so it runs in a dedicated thread pool, never on the event loop that serves the WebSocket connections.

The registry lives in Amazon DynamoDB, in an infrastructure stack separate from the server. That separation is deliberate: biometric data has its own lifecycle, independent of redeployments of the compute that produces it. An AWS Lambda function handles enrolment of reference voices.

There is no synchronization, and that is the design

One question follows from the architecture. The ONNX models run inside the container, and Amazon Transcribe is a remote service. The identity has to land exactly on the boundaries of the text. How are the two clocks kept in step?

They are not, because there are not two streams to align. There is one stream of PCM audio, and both consumers count the same samples. A single call site pushes each frame to the transcription queue and to the rolling buffer, in that order, from one coroutine.

The two clocks are in fact the same clock. The rolling buffer is addressed in bytes, not in time. Its position in milliseconds is the number of bytes received divided by 32, because 16 kHz at two bytes per sample gives 32 bytes per millisecond. Its origin advances only when the buffer trims, so it is a sample counter rather than a wall clock.

Amazon Transcribe returns start and end times in seconds since the beginning of its own stream. That stream was fed by those same bytes. Byte N therefore sits at the same millisecond on both sides. The audio is the clock, nothing needs reconciling, and no correlation between concurrent tasks is required.

In practice, a finalized result arrives with boundaries such as 78787 to 85500 milliseconds. The server cuts its own buffer at those indices and runs the ONNX models on those bytes. The remote service never returns audio.

The principle is worth stating once. The remote automatic speech recognition (ASR) service is used as a function from audio to text plus time coordinates. The reference audio stays local, always.

From signal to identity

The next diagram shows the decision path a single finalized segment travels. It runs from the Transcribe boundaries through the quality tiers to one of three outcomes: a name, an anonymous group, or a degraded identity.

Figure 2. From audio signal to speaker identity, including every abstention path.

Each segment finalized by Amazon Transcribe triggers the same sequence, and the two models play distinct roles. Amazon Transcribe supplies the segment time boundaries, which trigger the processing and frame the window cut from the buffer.

The pyannote segmentation gives voice activity at millisecond resolution, which measures the speech actually usable in the window rather than its raw duration. It gives the instants where the speaker changes, and the passages where two people speak at once. The Transcribe boundaries say when a sentence starts and ends, but not where the silence sits inside it, or where the voice changes.

A short sentence supplies little material, so the system widens the window by up to two seconds before and after. Two rules bound that widening. It never crosses a speaker change boundary that the segmentation detected, otherwise the fingerprint would mix two voices. And it uses only audio already in the buffer, never waiting for audio still to come, which would hold up the verification.

The usable speech duration then determines what the system allows itself to do. Below 30 seconds it extracts no fingerprint and returns a degraded identity.

Between 30 and 75 seconds it extracts a low quality fingerprint. That fingerprint can attach the segment to an already known voice, but it never creates a new voice and never enriches an existing one.

Above 75 seconds the fingerprint is high quality and does both.

This asymmetry protects the registry. A fingerprint computed on a speech fragment is noisy. Using it to label one segment costs at most one wrong label, visible with its score. Using it to create or update a registry entry makes that noise permanent, and every later comparison inherits it. The system therefore lets a weak fingerprint read the registry, never write to it.

Two distinct thresholds govern what follows. Above 0.60 cosine similarity, the segment attaches to the nearest voice and inherits its stable identifier. Above 0.70, and only if that voice already carries a name, the identity is named. Between the two, the segment joins the group without carrying a name.

A third safeguard applies. If the two highest-scoring candidates are separated by less than 0.05, no name is assigned. Two voices too close to tell apart produce an abstention, not a bet. Every branch converges on sending the segment to the verification agents, including the degraded branches.

Concurrency: two invariants to hold

Several Fargate tasks can process sessions in parallel. Two properties of the registry have to hold under that concurrency, and neither comes from a naive write.

one new voice must never create two entries. A lock held in a DynamoDB item serializes the critical section that compares then creates. The lock is acquired by conditional write, and carries a time to live that releases it if its holder disappears. The holder rereads the registry before deciding, and therefore sees any entry created in the meantime.
two simultaneous enrichments of the same voice must both be applied. The aggregate fingerprint of a voice is therefore not a vector rewritten on every contribution, which would lose one write in two. Each contribution is an immutable item on a distinct sort key, and the aggregate fingerprint is a derived value recomputed from the full set of contributions. Two concurrent writes land on two different keys and cannot overwrite each other.

That recomputation happens off the critical path. It cost 95 to 153 milliseconds per segment when it ran synchronously, for a value that is only a cache of contributions already made durable.

Keeping up with live: a session round robin

The dispatch design follows from one piece of arithmetic, which the next diagram lays out alongside the pool it produces.

Figure 3. Session round robin across Amazon Bedrock AgentCore session identifiers.

Amazon Transcribe finalizes a segment about every five seconds, while a full verification runs multiple sub-agents and takes minutes. Processing segments one after another therefore opens a backlog that never closes.

The first version of the POC framed that imbalance with a semaphore: one segment at a time, the others dropped. It lost about 80 percent of the stream. A debate verified at 20 percent has no editorial value.

Amazon Bedrock AgentCore Runtime offers the lever that solves this. It isolates execution by session identifier, so two calls carrying two distinct identifiers run in parallel, in separate environments. Having K identifiers is therefore enough to obtain K concurrent verifications, with no containers to provision or manage.

The server builds that pool at the start of each broadcast. The K identifiers are derived deterministically from the broadcast identifier, which keeps them stable across reconnections, and AgentCore requires each one to be at least 33 characters long. Segments are then distributed across the pool in round robin. The sizing follows from the arithmetic above: the segment rate multiplied by the worst case latency.

The call is made without waiting for the response, and that is the second point holding the whole thing together. The streaming server does not need the verdict. The agent publishes it to AWS AppSync thanks the hook feature of Strands Agent SDK, and the journalist interface receives it through a GraphQL subscription. The server drops the segment on a thread pool and moves to the next one.

This inversion removes backpressure instead of managing it. There is no semaphore, no reading of the response stream, and no segment dropped for lack of room. The server sends 12 segments per minute whatever a verification costs.

The pool is prewarmed. When the broadcast opens, the server sends K warmup calls in parallel, so the first real segments meet environments that are already active rather than a cold start.

Two properties of the system make giving up session affinity acceptable. The AI sub-agents are stateless from one segment to the next. And the facts accumulated about speakers live in the long-term memory of Amazon Bedrock AgentCore Memory, whose namespace is partitioned by broadcast identifier rather than by session identifier. The executions therefore read and write the same set of facts, while short-term memory stays per session.

Two tradeoffs come with this choice. Verdicts arrive out of order, because verifications started in sequence do not finish in the same sequence, so the interface reorders them on their original timestamp. And the shared facts are eventually consistent: a fact written by one execution can take a few seconds to become visible to another. That is acceptable here because a speaker identity stays stable during a show.

One implementation detail is worth calling out, because it connects this section to the concurrency invariants. The container uses two distinct thread pools. One is sized on K for the calls to the agents, which wait on the network. The other is limited to four threads for model inference, which consumes CPU. Mixing them would place voice identification behind 24 in-flight network calls, and would blow its time budget on every segment.

Security and GDPR

A voice fingerprint vector used to uniquely identify a person is biometric data. GDPR places it in the special categories, whose processing is prohibited unless an explicit exception applies. What follows describes technical measures, not a legal qualification. Security on AWS rests on a shared responsibility model. AWS is responsible for security of the cloud, and provides tools such as encryption and fine-grained access management. The customer remains responsible for security in the cloud: configuration, access control, and the compliance of their own processing. AWS helps customers work toward their compliance goals, and no AWS service by itself makes a GDPR processing activity compliant.

Raw audio is never retained. The buffer holds 40 seconds of signal, is trimmed continuously, and is released in full when the session ends or the connection drops. Only the 256 dimension vector crosses the boundary into durable storage. That is data minimization applied as close as possible: the system keeps what it needs to compare, and nothing more.

The registry is encrypted at rest with an AWS managed key, and point-in-time recovery is enabled. Access is restricted by AWS Identity and Access Management policies to the Fargate task role and the enrolment function alone. No other component of the platform reads that data.

Every entry carries its traceability : provenance, indicating whether an operator enrolled the voice or the system built it automatically, creation date, last update date, and model version. That traceability serves accountability under Article 5, and makes it possible to invalidate a batch of entries if the model changes.

Deletion is a tested code path , not an intention. A voice can be deleted by identifier or by person name, which erases the aggregate entry, all of its contributions, and the references to its enrolment clips. After deletion, the system no longer produces the corresponding name or identifier.

What this prototype does not handle deserves to be stated as plainly. It does not manage consent collection or its traceability. It applies no automatic retention policy, so a voice stays in the registry until explicit deletion. It produces no record of processing activities. Those elements are required for a production deployment and sit outside the technical scope described here.

Measured results, and what is not achieved

On an extract of a televised debate used as a demonstration dataset, the system processed 24 speaker turns and told the two speakers apart, with no wrong attribution observed. On a measurement set of 11 annotated segments, 8 are named correctly and 3 stay anonymous because those voices are absent from the registry, which is the expected behavior. These samples are small, and they validate behavior rather than establish general performance.

Measurement on Fargate gives about 470 milliseconds on average and 991 milliseconds at the 95th percentile.

The breakdown explains why, and it is the most useful lesson of this work. The time splits into three parts of comparable weight: inference of the two models, signal preparation ahead of the models, and round trips to DynamoDB. There is no isolated hot spot. The fingerprint comparison itself, which you might suspect first, costs a tenth of a millisecond, because the registry holds a few hundred entries and the vectors are normalized.

I removed what could be removed without a tradeoff. The aggregate fingerprint recomputation moved off the critical path, and the read cache is no longer invalidated in full after each write. That gained about 25 percent on the average, and it is where the cheap wins stopped.

Conclusion

A newsroom that analyzes live can now attach each verified claim to a named speaker, and follow that speaker from one broadcast to the next. That is the benefit the voice fingerprint delivers over the diarization labels the platform started from, and the measurement in this post shows where the two diverge.

Recognizing a voice in a real-time stream asks less for algorithmic sophistication than for rigor on edge cases. The decisions that mattered most are not the choice of models. They are three tradeoffs: preferring abstention to error, preventing identification from blocking processing, and forbidding low quality fingerprints from writing to the registry.

The sensitivity of the data imposes a framework from the design stage. Keeping only the vector, encrypting it, restricting access, tracing provenance, and making deletion effective are measures to build in from the start, not to add afterwards.

To go further, read the Amazon Transcribe Developer Guide, and the Amazon DynamoDB Developer Guide on the conditional writes that make the concurrency invariants holdable. The Amazon Bedrock AgentCore documentation covers the session isolation the round robin relies on.

If you work on an adjacent problem, the first question to settle is not technical: what does your system do when it is not sure?

Accelerate Step Functions Development with LocalStack: Testing Workflows Locally While Connecting…

Guillaume Marchand — Tue, 03 Feb 2026 08:28:09 +0000

Accelerate Step Functions Development with LocalStack: Testing Workflows Locally While Connecting to Real AWS Services

Developing complex Step Functions workflows often creates a frustrating bottleneck for development teams. Every code change requires a full deployment to AWS, turning what should be a quick iteration into a 30-minute wait. This slow feedback loop hampers productivity and makes debugging workflows unnecessarily difficult.

I want to share how you can accelerate your Step Functions development by using LocalStack to test workflows locally while maintaining connections to real AWS services. This approach reduces your feedback loop from 30 minutes to just few minutes, enabling rapid iteration without sacrificing the reliability of testing against real AWS services.

The Traditional Development Challenge

Step Functions workflows orchestrate multiple AWS services through complex state machines. A typical data ingestion workflow might coordinate multiple different steps using parallel states, map operations, and Lambda integrations with services like Amazon DynamoDB, Amazon Bedrock, and Amazon S3.

Testing these workflows traditionally requires deploying the entire stack to AWS. This creates several problems. Development cycles become slow and expensive. Debugging requires sifting through CloudWatch logs across multiple services. Team members often step on each other when sharing development environments.

The core challenge lies in balancing local development speed with testing authenticity. You want the rapid feedback of local testing, but you also need confidence that your workflow will behave the same way in production when interacting with real AWS services.

Solving the LocalStack Credentials Challenge

LocalStack provides an excellent foundation for local AWS service emulation, but it presents a specific challenge when you want to connect to real AWS services. LocalStack automatically injects fake AWS credentials into Lambda containers, preventing them from accessing actual AWS resources.

When LocalStack starts Lambda containers, it injects environment variables that override your real AWS credentials. Variables like AWS_ACCESS_KEY_ID receive fake values, AWS_ENDPOINT_URL gets redirected to LocalStack's internal endpoints, and AWS_SESSION_TOKEN contains invalid tokens. These injected values prevent your Lambda functions from connecting to real AWS services.

The solution involves understanding how LocalStack manages Lambda execution and configuring it to allow real AWS access. LocalStack runs Lambda containers with a specific user account called sbx_user1051, not the root user. This detail becomes crucial when mounting AWS credentials files.

Implementing the Solution

The implementation requires careful configuration of both LocalStack and your Lambda functions. You configure LocalStack using Docker Compose with specific environment variables that control credential injection and container behavior.

Setting network_mode to host allows LocalStack to mount volumes from the host system. The DISABLE_TRANSPARENT_ENDPOINT_INJECTION flag partially disables LocalStack's automatic endpoint redirection. Most importantly, LAMBDA_DOCKER_FLAGS mounts your AWS credentials file into the correct location within Lambda containers.

The critical insight involves mounting credentials to /home/sbx_user1051/.aws rather than /root/.awsbecause LocalStack's Lambda containers run with the sbx_user1051 user account. This ensures that when your Lambda code calls boto3, it can find and use your real AWS credentials.

Your Lambda functions need modification to handle both local and AWS execution environments. You create a session management module that detects LocalStack execution and removes the injected fake credentials. This allows boto3 to fall back to reading credentials from the mounted file.

The session management code checks for the LOCALSTACK_HOSTNAME environment variable to detect local execution. When running locally, it removes AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN, and AWS_ENDPOINT_URL from the environment. This forces boto3 to reload credentials from the mounted AWS credentials file.

Results and Trade-offs

This approach delivers significant productivity improvements. Your development feedback loop accelerates from 10s minutes to some minutes. You test real Lambda code against actual AWS services, ensuring high confidence in your testing.

The solution does involve some trade-offs. You still incur costs for AWS services like Amazon Bedrock and DynamoDB during testing. Temporary credentials require periodic regeneration during development sessions. The LocalStack user account path could potentially change between versions, though this rarely occurs in practice.

Conclusion

LocalStack provides a powerful foundation for accelerating Step Functions development when configured properly to work with real AWS services. By understanding and overcoming the credential injection challenges, you can achieve rapid local iteration while maintaining confidence through testing against production services.

This approach transforms Step Functions development from a slow, deployment-heavy process into a fast, iterative experience. Teams report significant productivity gains and reduced debugging time when implementing this local testing strategy.

The investment in setting up this local development environment pays dividends quickly through faster iteration cycles and more reliable deployments. Your development team can focus on building great workflows rather than waiting for deployments and debugging in production environments.

Standardizing AI-Developer Collaboration

Guillaume Marchand — Mon, 06 Oct 2025 10:09:56 +0000

Development teams across industries adopt AI coding assistants (Amazon Q Developer, Cursor, Kiro.dev, Cline, RooCode) to accelerate productivity and improve code quality. However, maintaining consistent standards between AI assistants and human developers presents significant challenges. Teams struggle with varying coding standards, undocumented architectural decisions, and complex knowledge transfer processes as their development efforts scale.

This article introduces a comprehensive sample that addresses these collaboration challenges by establishing standardized development practices, implementing proven architecture patterns, and integrating AI coding assistant configuration. This approach creates a unified development experience that benefits both AI assistants and human developers across all projects.

Addressing AI Development Complexity

Developers want to focus on creative aspects of building applications while staying in their workflow without compromising quality or dealing with repetitive boilerplate code. AI coding assistants promise increased flow, better productivity, and a more enjoyable development experience. As these assistants evolve toward more agentic workflows, developers need effective ways to provide appropriate context to these AI tools.

Many developers express excitement about emerging approaches like “vibe coding,” where they chat with AI agents that guide them step by step through application development. These approaches work well for small prototypes but often break down as projects increase in complexity. Developers frequently report that vibe-coding works for side projects but not for professional work where code quality and correctness are paramount.

Current agentic development approaches often require developers to spend significant time guiding the agents and fixing problems — sometimes as much time as writing code from scratch. As teams leverage agents for more complex tasks, they need to provide more precise project plans to reduce ambiguity and ensure the agent’s work meets quality standards.

Introducing Kiro Integration

Kiro integrates four key features that create structured workflows to maintain quality standards while enabling AI automation. Spec-driven development provides a formalized approach to building features through iterative design and implementation processes. Agent hooks enable automatic AI execution when specific development events occur, operating autonomously based on developer-defined prompts to maintain quality practices.

Agent steering provides additional context and instructions that influence AI assistant behavior throughout development interactions through specialized steering files. Model Context Protocol (MCP) servers extend AI assistant capabilities with specialized tools for various development tasks, making the assistants more capable and contextually aware.

Establishing Shared Development Standards

The sample creates shared understanding between AI assistants and development teams through standardized development rules across projects. This unified approach enables AI assistants to access the same project knowledge that guides human developers, ensuring consistent architectural decisions and coding practices throughout the development lifecycle.

The sample includes pre-configured Model Context Protocol servers for AWS services, documentation generation, diagram creation, and code analysis. Comprehensive development rules guide both AI assistants and developers, eliminating knowledge gaps and ensuring consistent implementation patterns across your projects.

These standardized rules cover architecture patterns, coding standards, testing strategies, and development practices. This approach creates a shared vocabulary between AI assistants and development teams, improving collaboration effectiveness and maintaining consistency as projects evolve.

Implementing Enterprise-Grade Architecture

The sample implements a hexagonal architecture pattern that ensures clean separation between domain logic, ports, and adapters. This pattern makes applications more resilient to changes in external dependencies while enabling AI assistants to understand clear boundaries between business logic and infrastructure concerns.

The hexagonal approach allows your teams to focus on core business logic while keeping implementation details at the edges of your application. This separation of concerns makes your applications easier to test, maintain, and evolve over time, providing long-term benefits for both development velocity and code quality.

Scaling Across Enterprise Teams

At enterprise scale, the template maintains consistency across multiple teams and projects through standardized approaches including design patterns, coding rules, software lifecycle management, and testing strategies. Lead developers can initialize projects with proven patterns while teams receive updates to maintain consistency with evolving standards.

The template uses Cruft and Cookiecutter for project generation and ongoing synchronization with upstream improvements. This approach ensures projects generated from the template receive updates to development standards, security improvements, and new features without manual intervention.

Your teams benefit from centralized standard management while maintaining autonomy in their specific implementations. The template approach scales organizational knowledge and best practices across all your development efforts, creating consistency without sacrificing team flexibility.

Getting Started with Immediate Benefits

Using the template requires minimal setup and provides immediate productivity gains.

Prerequisites

Install the required tools:

Install Task for task automation
Install uv for Python environment management
Install Cruft for template management:

pip install cruft

Creating a New Project

Generate a new project with a single command:

cruft create https://github.com/aws-samples/sample-ai-coding-standards-template.git --directory template/

This command creates a complete project structure with:

Hexagonal architecture implementation
AWS CDK infrastructure setup
Integration tests with real AWS resources
AI assistant configurations for multiple AI Coding Assistants
Comprehensive documentation system
Build and deployment automation

Project Setup and Development

Once your project is generated, follow these steps:

Set up development environment :

task setup

2. Set up infrastructure environment and build Lambda functions :

task cdk:setup 
task cdk:build

3. Deploy to AWS :

task cdk:deploy

Set up test environment and run tests :

task test:setup 
task test:all

AI Assistant Integration

Each generated project includes pre-configured AI assistant support:

Amazon Q Developer
Roo Cline
Kiro AI
Cursor AI
Cline

The steering files and hooks configure AI agent behavior for specific project contexts, providing comprehensive organizational context that enables AI assistants to understand architectural patterns, coding standards, and specific requirements without extensive human guidance.

Template Synchronization

Every project includes automated template update capabilities:

# One command for seamless template updates
task cruft:update

This automation provides automatic conflict resolution, requires no user interaction, maintains a clean workspace, and never blocks your workflow. The system applies updates where possible and gracefully handles conflicts.

Documentation System

Projects include a comprehensive documentation system built with MkDocs:

# Set up and serve documentation locally
task docs:setup
task docs:build
task docs:serve

The documentation system automatically generates API documentation from Python docstrings, organized by hexagonal architecture layers, and includes examples demonstrating proper usage patterns.

Transforming Development Team Productivity

This standardized approach delivers significant advantages for teams building applications with AI assistance. AI coding assistants receive comprehensive organizational context, enabling them to understand architectural patterns, coding standards, and specific requirements without extensive human explanation. Structured development rules and steering files provide clear guidance on implementation patterns, making AI suggestions more accurate and contextually appropriate for your projects.

Developers eliminate the cognitive overhead of maintaining consistency between AI-generated code and established project standards. Teams can trust AI suggestions because they follow the same architectural patterns and coding conventions that guide human development decisions. Comprehensive documentation and reference implementations enable developers to quickly understand and extend AI-generated code while maintaining project consistency.

The hexagonal architecture creates loosely coupled systems where application components can be tested independently, with no dependencies on data stores or user interfaces. This pattern helps prevent technology lock-in while providing a clear structure for both AI assistants and human developers to follow.

AI assistants become more effective contributors through better project context understanding. Developers spend less time reviewing and correcting AI-generated code because it follows established standards from creation. Standardized patterns simplify team member onboarding and maintain consistency across projects and contributors.

The combination of proven architectural patterns, comprehensive automation, and shared standards creates a powerful foundation for modern cloud native development. Your teams can leverage AI assistance effectively while maintaining the consistency and quality standards required for successful applications, accelerating development velocity while ensuring code quality and architectural integrity across your organization.

Process millions of media assets with FFmpeg on AWS

Guillaume Marchand — Wed, 03 Sep 2025 07:36:52 +0000

Process millions of media assets with FFmpeg on AWS

I need to process over 3 million multi-modal files for training a large language model (LLM) that can understand and generate audio in order to launch generative artificial intelligence based customer experiences. Training an audio LLM requires massive amounts of high-quality audio data to learn and understand acoustic patterns. The team has access to millions of audio files stored in Amazon S3, but processing them sequentially on an Amazon EC2 instance does not scale.

To efficiently process the audio for training LLMs , I improved the AWS Batch with FFmpeg sample code, an open audio/video processing sample code using AWS Batch and an Open Source tool FFmpeg.

In this article, I provide technical details on building a reliable, scalable processing workflow using AWS Step Functions and AWS Batch. The workflow utilizes the open source tool “FFmpeg”, to process large volumes of media assets. I describe how to configure AWS Step Functions to orchestrate AWS Batch jobs, handle job failures gracefully, and work within service limits. This architecture shows how you can leverage AWS services like Step Functions and Batch together with open source tools like FFmpeg to create a robust and managed processing pipeline.

The architecture

At AWS re:Invent 2022, AWS announced the availability of a distributed map for AWS Step Functions. This new state type extended support for orchestrating large-scale parallel workloads.

This state is ideal for processing workflows, where many assets can be processed in parallel. I can compose any AWS service API supported by Step Functions into the workflow. In our use case, AWS Batch is invoked directly from the Amazon States Language to parallel process assets without writing new code. This is achieved through Step Functions Service Integrations which allow users to call supported services directly in the Resource field of a Task state.

The following code describes a Step Function state submitting a new job to Batch:

"SubmitJob": {
            "Type": "Task",
            "Resource": "arn:aws:states:::batch:submitJob.sync",
            "Parameters": {
              "JobName.$": "$.name",
              "JobDefinition.$": "States.Format('arn:aws:batch:<region>:<account>:job-definition/batch-ffmpeg-job-definition-{}',$.compute)",
              "JobQueue.$": "States.Format('arn:aws:batch:<region>:<account>:job-queue/batch-ffmpeg-job-queue-{}',$.compute)",
              "Parameters.$": "$"
            },
            "End": true,
          }

I build upon the existing “AWS Batch and FFmpeg“ sample by encapsulating it in a Step Functions state machine. The state machine uses a distributed Map state task optimized for Amazon S3 inputs. By configuring the S3 bucket and prefix directly in the map, the state machine processes assets in parallel.

For each map task, the state machine invokes a Batch job to leverage FFmpeg tool and perform audio encoding.

The following design shows how AWS Batch handles compute provisioning and scheduling, while Step Functions orchestrates the workflow — all in a serverless model.

Beyond the limit

Retry mechanism

Following the Step Functions Quota documentation, the Step Functions distributed map supports a maximum concurrency of up to 10,000 executions in parallel, which exceeds the concurrency limits of AWS Batch. When integrating Step Functions with other services, I must consider the downstream service’s quotas and limits, to avoid errors, as you can read in the following screenshot.

AWS Batch has a quota of 1 million jobs in Submitted state and a limit of 50 transactions per second (TPS) for SubmitJob API calls. To avoid exceeding the TPS limit, I could configure the Step Functions distributed map’s “maximum concurrency” to 50. However, at this rate, it would take over 16 hours to submit 1 million jobs, excluding processing time.

A better solution is to use Step Functions’ “enhanced error handling” capabilities. This allows us to set a max limit on retry intervals to prevent excessive delays. Adding jitter introduces randomness into the retries, avoiding a retry storm that could overwhelm Batch. The combined error handling controls retry rates appropriately during failures while still allowing the high concurrency of distributed maps for normal operation.

When configuring AWS Step Functions, it’s important to set the maximum concurrency appropriately for the workload. Here, I set it to 5,000. Instead of retrying immediately and aggressively, the Step Functions waits some amount of time between tries. The most common pattern is an exponential backoff, where the wait time (IntervalSeconds = 180 sec.) is increased exponentially (BackoffRate = 3) after every attempt. Exponential backoff can lead to very long backoff times, because exponential functions grow quickly. To avoid retrying for too long, implementations typically cap their backoff to a maximum value ( MaxAttempts = 10). This is called, predictably, “capped exponential backoff “, the blog post ”Timeouts, retries, and backoff with jitter“ from Amazon Builders’ Library explains in detail the concept . If all the failed calls back off to the same time, they cause contention or overload again when they are retried, Jitter adds some amount of randomness to the backoff to spread the retries around in time (JitterStrategy = “FULL”).

The following code describes a Step Function state part with this retry mechanism:

"Retry": [
              {
                "ErrorEquals": [
                  "States.ALL"
                ],
                "BackoffRate": 3,
                "IntervalSeconds": 180,
                "MaxAttempts": 10,
                "Comment": "retry because of AWS Batch Quotas Issue",
                "MaxDelaySeconds": 300,
                "JitterStrategy": "FULL"
              }
            ]

When the Step Functions workflow starts, the initial burst of AWS Batch SubmitJob API calls throttles due to exceeding TPS limits. But the built-in retry policy with capped exponential backoff and jitter allows all jobs to eventually succeed without failing. The backoff provides time for the throttling to clear, while jitter spreads out the retries to avoid more throttling. This shows how Step Functions’ retry policies can gracefully handle temporary throttling or failures.

Application state data

AWS Step Functions store application state data for each workflow invocation. The maximum size limit for this application state data is 256 kilobytes per workflow invocation. This means the total size of all data loaded into the state machine and passed across transitions must be less than 256KB for each invocation. Exceeding this 256KB limit will result in an exception and aborted execution as described in the following screenshot.

Fortunately, AWS Step Functions provide a solution to consolidate large amounts of data from child workflow executions. It aggregates all child workflow execution data, including execution inputs, outputs, and status. Step Functions export executions with the same status to their respective files in the specified Amazon S3 location. The “ResultWriter” field specifies the S3 bucket and prefix where Step Functions will write the aggregated results of all child workflows started by a Distributed Map state.

The following code describes a Step Function state part with this application state data export configuration:

"ResultWriter": {
        "Resource": "arn:aws:states:::s3:putObject",
        "Parameters": {
          "Bucket.$": "$.input.s3_bucket",
          "Prefix": "batch-ffmpeg-state-machine/results-output/"
        }

How to use it

Prerequisites

You will need the following prerequisites to set up the solution:

An AWS account
Latest version of AWS Cloud Development Kit (AWS CDK) with bootstrapping already done
Latest version of Task
Latest version of Docker
Latest version of Python 3.

Deploy the sample code

Deploy the “AWS Batch with FFMPEG“ sample code following the README file in the GitHub repository : https://github.com/aws-samples/aws-batch-with-ffmpeg#deploy-the-solution-with-aws-cdk

Use the solution

A Step Functions execution is triggered with a JSON file as an input. In our case, here is the JSON “input.json” designed for the solution:

{
  "name": "pytest-sdk-audio",
  "compute": "intel",
  "input": {
    "s3_bucket": "my-input-bucket",
    "s3_prefix": "media-assets/",
    "file_options": "null"
  },
  "output": {
    "s3_bucket": "my-output-bucket",
    "s3_prefix": "output/",
    "s3_suffix": "",
    "file_options": "-ac 1 -ar 48000"
  },
  "global": {
    "options": "null"
  }
}

Parameters of this JSON Step Function Execution input are:

$.name: metadata of this job for observability.
$.compute: Instances family used to compute the media asset : intel, arm, amd, nvidia, xilinx.
$.input.s3_bucket and $.input.s3_prefix: List of Amazon S3 Objects to be processed by FFmpeg.
$.input.file_options: FFmpeg input file options described in the FFmpeg official documentation.
$.output.s3_bucket and $.output.s3_prefix: S3 bucket and prefix where all processed media assets will be stored.
$.output.s3_suffix : Suffix to add to all processed media assets which will be stored on a Amazon S3 Bucket
$.output.file_options: FFmpeg output file options described in the official documentation.
$.global.options: FFmpeg global options described in the official documentation.

I submit the Step Function execution with the AWS CLI

aws stepfunctions start-execution — state-machine-arn arn:aws:states:.<region>:<account_id>:stateMachine:batch-ffmpeg-state-machine —name <execution-name> —input "$(jq -R . input.json —raw-output)

When the execution of this Step Functions completes, the processed media assets become available within the S3 bucket configured as “output”. The S3 path to access these media files is: s3://{$.output.s3_bucket}{$.output.s3_suffix}{Input S3 object key}{$.output.s3_suffix}.

Now I have access to millions of properly processed audio files and can proceed to train its audio LLM (Large Language Model).

Cost

AWS Batch enables optimizing compute costs by only paying for the resources used. Leveraging Spot instances allows customers to take advantage of unused EC2 capacity to achieve significant cost savings compared to On-Demand instances. It’s important to benchmark different instance types and sizes to find the optimal configuration for the workload. Testing options like GPU vs CPU helps strike the right balance between performance and cost as described in the following blog post “Optimizing video encoding with FFmpeg using NVIDIA GPU-based Amazon EC2 instances”.

Clean up

To avoid incurring unnecessary charges after testing this solution, I have to clean up the resources I created by following these steps:

Delete all objects in the Amazon S3 bucket used for testing. Remove these objects from the S3 console by selecting all objects and clicking “Delete.”
Destroy the AWS CDK stack that was deployed for testing. Open a terminal in the Git repository and run: task cdk:destroy
Verify that all resources have been removed by checking the AWS console. This ensures no resources are accidentally left running, which would lead to unexpected charges.

Summary

I leveraged an Open Source tool FFmpeg and multiple AWS services (AWS Batch, AWS Step Functions, and Amazon S3) to process millions of audio files in parallel. This serverless architecture overcame scalability and service quota challenges by combining AWS services with an open source technology:

AWS Step Functions’ distributed map enabled large-scale parallel processing of assets stored in S3.
Integrating AWS Batch into the Step Functions workflow provided scalable compute while Step Functions handled orchestration.
Error handling strategies like retries and jitter in Step Functions helped avoid overloading downstream AWS Batch when executing high volumes of jobs.

In summary, the combination of AWS Batch, Step Functions, S3, and Open Source tool like FFmpeg allowed efficient, scalable, parallel processing of millions of assets.

The following screenshot illustrates the item status processing of 2 million audio files accomplished by the team in nearly 2 days. A sequential execution would have taken several weeks to complete the same task.

Automate Cloud Resource Management for Scheduled Events

Guillaume Marchand — Tue, 02 Sep 2025 09:58:26 +0000

Organizations across industries face operational challenges when managing planned high-traffic events. Business teams must create IT support tickets for each event, requiring DevOps teams to manually provision and scale resources. This process creates bottlenecks that impact customer experience and operational efficiency.

The "Event Scheduling on AWS" implementation sample addresses these challenges by automating resource provisioning and scaling for planned events. This sample enables organizations to deliver exceptional customer experiences during high-demand periods while reducing operational costs and eliminating manual coordination between business and technical teams.

Understanding the Challenge

Enterprises across multiple industries face similar operational obstacles when managing planned events. A media company planning a live sports broadcast must coordinate with DevOps teams weeks in advance, creating support tickets and requiring technical staff presence during events for manual scaling operations.

Retail organizations preparing flash sales encounter comparable challenges. E-commerce platforms must provision additional compute capacity, configure content delivery networks (CDNs), and scale database resources to handle sudden traffic spikes during promotional events. Manual coordination between marketing and technical teams creates delays and increases the risk of failures.

Gaming companies launching new titles or hosting e-sports tournaments face infrastructure scaling complexities. They must coordinate server provisioning across multiple regions, configure matchmaking services, and ensure backend systems can handle concurrent player loads. The manual nature of these operations often results in poor player experiences during peak gaming events.

Financial services organizations managing trading platform events encounter similar operational bottlenecks. Market events, earnings announcements, and regulatory changes require rapid infrastructure adjustments to handle increased trading volumes. Manual provisioning processes create delays that can impact trading performance and customer satisfaction.

These manual approaches increase operational overhead and extend planning timelines across industries. The risks of human error increase, IT resource utilization becomes inefficient, and organizations struggle to scale their operations effectively. These factors directly impact service quality during business-critical moments.

Solution Architecture

The "Event Scheduling on AWS" implementation sample uses an event-driven architecture built on AWS serverless technologies. AWS Step Functions orchestrates workflows through preparation, provisioning, configuration, and cleanup phases. Amazon EventBridge manages event scheduling and message coordination.

The Step Functions workflow implements a complete event lifecycle through preroll and postroll phases. The preroll phase executes before the event, managing resource provisioning, configuration validation, and system preparation tasks. During this phase, the platform deploys AWS Service Catalog products or executes Systems Manager documents for infrastructure scaling, and performs checks to ensure readiness.

The postroll phase activates after event completion, managing resource cleanup, and operational reporting. This phase terminates temporary resources, reduces infrastructure to baseline levels. The preroll and postroll approach ensures consistent event execution while optimizing resource utilization and operational costs.

AWS Service Catalog manages infrastructure deployments while AWS Systems Manager executes automation workflows. AWS AppSync provides the GraphQL API layer and Amazon CloudWatch delivers comprehensive monitoring and alerting capabilities.

The platform integrates with existing AWS services through standardized tags and IAM policies. Resources tagged with application=event-scheduling-platform become available for orchestration, enabling seamless integration with current infrastructure.

Key Organizational Benefits

Business teams gain self-service capabilities to schedule events without dependencies on IT teams. They work with familiar business metrics such as audience size and performance targets while tracking event status in real-time. This approach removes bottlenecks and accelerates event planning cycles.

Operational teams maintain visibility and control despite increased business team independence. The integration with AWS Chatbot delivers real-time notifications to Microsoft Teams and Slack channels, ensuring operational staff remain informed of all event activities. Teams receive alerts for event scheduling, resource provisioning status, execution progress, and completion notifications.

This notification system allows operational teams to monitor business-initiated events without requiring direct involvement in routine operations. They can respond quickly to issues while enabling business teams to operate independently for standard event scenarios. The integration preserves operational oversight while eliminating manual coordination bottlenecks.

DevOps teams create reusable infrastructure templates through AWS Service Catalog products and Systems Manager documents. Automation reduces manual intervention, allowing technical personnel to focus on platform improvements rather than routine operational tasks.

The implementation sample optimizes costs through automatic resource cleanup after events.

Open Architecture with AWS Service Integration

The "Event Scheduling" implementation sample provides an open framework that packages industry-specific use cases through established AWS services. This approach ensures organizations can leverage existing AWS capabilities while maintaining operational consistency and security standards.

AWS Systems Manager Integration

AWS Systems Manager documents encapsulate operational procedures for different industries. Gaming companies can create documents that scale Amazon EC2 Auto Scaling groups and configure Amazon ElastiCache clusters for tournament events. Financial services organizations can develop documents that adjust Amazon RDS replicas and modify AWS Lambda concurrency limits during trading events.

Each Systems Manager document includes built-in audit capabilities through AWS CloudTrail integration. Execution history, parameter changes, and resource modifications are automatically logged, providing complete traceability for compliance requirements. The service manages document versioning, rollback capabilities, ensuring reliable automation across distributed environments.

Kubernetes Integration for Containerized Workloads

Organizations running containerized applications can implement SSM documents that integrate with Amazon EKS clusters. These documents address the technical challenge of authenticating with EKS clusters and interacting with the Kubernetes API from SSM automation workflows.

The SSM document would enable resource pre-warming via Kubernetes Horizontal Pod Autoscaler (HPA) or Deployment for existing EKS clusters. This document would support both pre-roll actions that scale deployments before events and post-roll actions that reduce resources afterward. The integration leverages AWS Lambda functions within the SSM document to execute Kubernetes client operations, ensuring secure authentication and reliable API interactions.

AWS Service Catalog Product Packaging

AWS Service Catalog products package complete infrastructure solutions for specific event types. Service Catalog provides governance through launch constraints, template constraints, and notification constraints. Product portfolios enable different organizational units to access appropriate infrastructure templates while preventing unauthorized resource creation.

Live Video Streaming with SRT Source

Media organizations could deploy complete live video streaming infrastructure using a Service Catalog product that provisions AWS Media services. This product would address the technical challenge of provisioning live video workflows with predefined SRT input endpoints and HLS/DASH output endpoints that integrate with existing information systems.

The product would include MediaConnect for SRT source ingestion, MediaLive for video processing, MediaPackage for content packaging, and CloudFront for content delivery. Route 53 would provide predictable domain names while AWS Certificate Manager would manage TLS certificates tied to CloudFront distributions. This comprehensive approach could ensure reliable video delivery with minimal manual configuration.

Existing AWS Solutions Integration

Organizations can leverage existing AWS architectures by integrating CloudFormation or Terraform templates into Service Catalog products.

This approach allows organizations to build upon proven AWS architectures while adding custom configurations and governance controls. The integration maintains the benefits of AWS architectures while providing the automation and scheduling capabilities of the "Event Scheduling" implementation sample.

Operational Reliability

AWS Step Functions provides reliable workflow orchestration with built-in error handling, retry logic, and state management. The service automatically manages transient failures and provides detailed execution history for troubleshooting. The integration with Amazon EventBridge ensures event scheduling remains accurate even during service interruptions.

Amazon CloudWatch monitors all platform components with custom metrics, alarms, and dashboards. Organizations can track event success rates, resource provisioning times, and cost optimization metrics. AWS X-Ray provides distributed tracing capabilities for complex multi-service event workflows.

Getting Started

Follow the README.md in the GitHub project for implementation guidance.

Resource Integration

Existing AWS Service Catalog products and Systems Manager documents integrate with the platform through proper tagging. Resources must include the application=event-scheduling-platform tag to become discoverable by the orchestration system.

The platform provides automated registration scripts that discover and register properly tagged resources. This eliminates manual registration requirements and ensures consistent integration across existing infrastructure.

An audit tool validates configuration and identifies common integration issues. The tool checks IAM permissions, Service Catalog configuration, Systems Manager configuration, and resource tagging to ensure proper platform operation.

Cost Considerations

The solution uses pay-per-use AWS services including DynamoDB, Lambda, Step Functions, AppSync, and CloudWatch. Costs scale with usage patterns and event frequency rather than requiring fixed infrastructure investments.

Automatic resource cleanup after events prevents unnecessary charges from orphaned resources. The architecture design minimizes state transitions and optimizes service usage to control operational expenses.

Next Steps

The "Event Scheduling" implementation sample enables organizations to transform their event management operations by eliminating manual processes and reducing operational overhead. Organizations benefit from faster event deployment cycles, improved resource utilization, and enhanced customer experiences during business-critical moments.

Explore the open-source implementation to understand integration patterns and architectural decisions. Comprehensive documentation includes deployment guides, troubleshooting resources, and extension samples.

Start with the sample resources to understand platform capabilities, then progressively integrate existing infrastructure through proper tagging and registration processes. Audit tools help ensure successful integration and identify optimization opportunities.