DEV Community: Jamie Cole

Building Reliable AI Generation Workflows: The API Call Is the Easy Part

Jamie Cole — Thu, 20 Aug 2026 14:11:25 +0000

Over the past month, I have been building an AI generation product with asynchronous video tasks.

I originally expected model integration to be the difficult part.

It was not.

Sending a request to an AI provider and receiving a task ID is relatively straightforward. The harder engineering work begins when the provider is slow, the network fails at the wrong moment, the user refreshes the page, or money has already changed hands.

This article covers the architecture lessons I learned while turning an AI API into a workflow that can recover from uncertainty.

The basic workflow

At a high level, an asynchronous generation request looks simple:

Client
  → Generation API
  → Local task record
  → AI provider
  → Polling or webhook
  → Result reconciliation
  → Persistent result

The happy path is easy:

Validate the request.
Calculate the cost.
Deduct credits.
Submit the provider task.
Poll until it succeeds.
Store the result.

Production systems rarely stay on the happy path.

The interesting problems appear between those steps.

1. Treat submission as a state machine

A generation task is not simply pending, success, or failed.

There is another important state:

submission_unknown

Imagine sending a POST request to a provider. The provider accepts it and starts the generation, but the response is lost because a gateway times out.

From the application's perspective, two realities are possible:

the provider never accepted the request;
the provider accepted it, but the application did not receive confirmation.

Automatically retrying the request may create a second paid generation.

Treating it as an ordinary failure may refund the user while an expensive provider task continues running.

The system therefore needs to distinguish a confirmed rejection from an uncertain submission.

A conceptual state model might look like this:

type GenerationState =
  | 'created'
  | 'submitting'
  | 'pending'
  | 'succeeded'
  | 'failed'
  | 'submission_unknown';

The exact names are not important. The important part is preserving uncertainty instead of forcing every outcome into success or failure.

2. Give every submission a stable idempotency key

Users double-click buttons. Browsers retry requests. Mobile connections disconnect. UI state gets restored after a refresh.

The generation endpoint must assume that the same logical submission can arrive more than once.

Each client submission should carry a stable idempotency key:

type GenerateRequest = {
  prompt: string;
  model: string;
  idempotencyKey: string;
};

On the server, that key should resolve to one local task for one user:

const existingTask = await findTaskByIdempotencyKey({
  userId,
  idempotencyKey,
});

if (existingTask) {
  return existingTask;
}

Database uniqueness is still necessary because two requests may pass the initial lookup concurrently.

A safe flow is:

Check for an existing task.
Attempt to create the task with a unique key.
If the insert loses a race, load and return the winning task.
Never deduct credits twice for the same logical submission.

Idempotency is not only protection against double-clicks. It is what makes recovery possible across the entire workflow.

3. Do not confuse an observation timeout with a task timeout

Polling is an observation mechanism. It is not the task itself.

A browser may stop polling after thirty seconds, but the provider task can still be running correctly.

This distinction matters:

Polling window ended ≠ Generation task failed

If the client treats every polling timeout as a terminal failure, it may submit another task when the user tries again.

A better approach is to preserve the original task ID and resume observation later:

async function resumeTask(taskId: string) {
  const localTask = await loadTask(taskId);

  if (isTerminal(localTask.status)) {
    return localTask;
  }

  return observeExistingTask(taskId);
}

Refreshing the page, reopening the application, or returning from another device should continue tracking the same task whenever possible.

The UI can say that observation has paused without claiming that generation has failed.

4. Make reconciliation shared and idempotent

A task may reach a terminal result through several paths:

foreground polling;
a provider webhook;
scheduled recovery;
an admin repair operation.

These paths should not contain separate versions of the completion logic.

They should all call the same reconciliation function:

async function reconcileTask(input: {
  taskId: string;
  providerStatus: ProviderStatus;
  source: 'poll' | 'webhook' | 'recovery';
}) {
  // Load the current task.
  // Ignore an already-finalized task.
  // Persist the result or failure once.
  // Refund credits at most once.
  // Return the canonical local state.
}

Without shared reconciliation, race conditions become likely.

A webhook may complete the task while foreground polling is still active. Both paths may attempt to save the result or issue a refund.

Terminal transitions must therefore be idempotent:

if (task.finalizedAt) {
  return task;
}

In practice, a simple application-level check may not be enough. Database constraints or transactional updates should enforce the invariant.

The rule is:

Many observers may discover the outcome, but only one transition should finalize it.

5. Calculate billing on the server

The frontend needs to display the expected credit cost, but it must not decide the final charge.

Generation cost may depend on:

model;
duration;
resolution;
output count;
generation mode;
optional capabilities.

The server should parse and validate the real request parameters, then calculate the authoritative cost:

const options = parseGenerationOptions(request.body);

const creditCost = calculateCreditCost({
  model: options.model,
  duration: options.duration,
  resolution: options.resolution,
  outputCount: options.outputCount,
});

The frontend should consume the same shared pricing definitions when possible, but the server must still recompute the charge.

Otherwise, stale clients or modified requests can create a gap between displayed pricing and actual billing.

Refunds need the same discipline.

A refund should be tied to a unique task and transaction reason so that repeated recovery attempts cannot refund the same task twice.

6. Persist enough information to recover the user experience

Task recovery is not only a backend problem.

After a refresh, the frontend needs enough information to reconstruct what the user was doing:

the original prompt;
selected model;
input assets;
generation settings;
local task ID;
current status;
result metadata.

Persisting only the provider task ID is not enough.

The provider knows how to generate the output, but it does not necessarily know how your product should present the user's intent.

A local task record should remain the application's source of truth. Provider state is external evidence used to update that record.

This separation also prevents internal provider identifiers and implementation details from leaking into the user interface.

What changed in my thinking

Before building this workflow, I thought the central engineering problem was model integration.

Now I see three different layers:

Provider transport — submitting and querying external tasks.
Product workflow — task identity, recovery, billing, and reconciliation.
User experience — explaining progress, uncertainty, failure, and recovery clearly.

The provider API only solves the first layer.

The product is built in the second and third.

AI tools can make implementation faster, but they also make it easier to add features before the underlying state model is reliable.

The most valuable work was not adding another model. It was defining what must remain true when several things fail at once.

Final takeaway

For asynchronous AI products, reliability comes from preserving identity and uncertainty:

one logical submission maps to one task;
an unknown outcome is not silently treated as failure;
polling timeouts do not recreate work;
every completion path shares one reconciliation contract;
billing is authoritative on the server;
refunds and terminal transitions are idempotent;
the UI resumes existing work instead of submitting again.

The API call may be the visible feature.

The recovery model is the real product.

Disclosure: This article is based on my own implementation experience and was reviewed against the system's actual behavior. AI was used to help organize the structure and edit the English draft.

How to Keep Characters Consistent in AI Videos: A Reference-First Workflow

Jamie Cole — Mon, 10 Aug 2026 07:18:10 +0000

Shot one looks right. Shot two is polished too, but the lead character now has different eyes, a wider jaw, and a jacket that changed color. Each clip works on its own. Put them together, and the story falls apart.

Character consistency in AI video is not mainly a matter of writing a longer prompt. It is a workflow problem. If every shot begins with a fresh text description, the model has room to invent a fresh person. A reference-first workflow reduces that room before motion generation begins.

The Short Answer

To keep a character recognizable across AI video shots:

Define the identity details that must not change.
Approve one clean master reference before generating video.
Add only the extra angles or expressions a shot genuinely needs.
Use image or reference inputs instead of rebuilding the character from text.
Write prompts about action, camera, and environment rather than redesigning the person.
Generate short, controlled shots and review identity before moving on.
Save successful frames as tested anchors for later scenes.

The goal is not perfect pixel-level duplication. The goal is a character your audience recognizes from shot to shot without stopping to wonder whether the actor changed.

Why Characters Drift Between AI Video Shots

A text prompt describes a type of person. A reference image shows a specific person.

That difference matters. If you prompt for “a woman with short dark hair in a navy jacket” five times, the model may satisfy the sentence with five different faces. Even a detailed description leaves many visual decisions open: jaw shape, eye spacing, hairline, proportions, fabric details, and how those features look from another angle.

Motion makes the problem harder. During a head turn, fast gesture, close interaction, or dramatic camera move, the model must preserve identity while also inventing unseen views and maintaining temporal continuity. The more constraints competing for attention, the easier it is for facial features, clothing, or body proportions to drift.

This is why “add more appearance details” often disappoints. The better first move is to give the generation a stronger visual anchor.

A consistent character shown across changing scenes.

Build a Character Anchor Kit Before You Animate

A useful anchor kit is small. You do not need a folder of nearly identical portraits. You need a few approved images in which the character is easy to read.

Start with one master reference

Choose a clear front or three-quarter view with:

Sharp facial details
Even lighting
An unobstructed face and hairline
A simple background
One visible character
The exact hairstyle, outfit, and signature details you want to preserve

Avoid using a dramatic action frame as the master reference. Motion blur, a hidden eye, a hand across the face, or colored stage lighting may look cinematic, but each one removes information the model needs later.

If the character does not exist yet, create several candidates with the Text to Image AI Generator, select one design, and stop redesigning it. If you already have the right person but need a cleaner angle, background, or wardrobe treatment, use Image to Image AI to create controlled variations from the approved source.

Add views because the shot needs them

A second or third reference is useful when the planned camera angle reveals information the master image does not contain. For example:

Add a side view before a profile shot.
Add a full-body view before a walking scene.
Add a close portrait before an emotional close-up.
Add a tested outfit reference when clothing continuity matters.

More references are not automatically better. Two images with different hairstyles, apparent ages, lighting, or facial proportions create a conflict instead of a stronger identity. Every image in the kit should look like evidence of the same character on the same production.

A three-view reference layout that keeps the face, clothing, and silhouette visible on one canvas.

Record the non-negotiables

Write a short identity lock beside the images. Keep it limited to details the audience would notice immediately if they changed:

Face and hairstyle
Signature clothing or accessory
Approximate age and build
Art direction, such as photorealistic or stylized 3D
Details that must stay fixed, such as a scar, glasses, or an ear cuff

This is not a second giant prompt. It is a review checklist for deciding whether a generated shot belongs in the sequence.

Choose the Right Visual Control for Each Shot

Reference images, a starting frame, and start/end frames solve related but different problems.

Image to Video AI separates these controls into focused browser workflows. You can begin with an approved still, provide visual references, or guide a shot with start and end frames, then choose the available model and video settings, generate the clip, and review the MP4 result. The useful question is not which mode has the most inputs. It is which input gives the current shot the control it actually needs.

One neutral character anchor reused across different environments and camera conditions.

Use Reference to Video for identity across new scenes

Choose Reference to Video when the character should remain recognizable but the new shot can begin in a different composition, location, or pose. The reference supplies identity and visual context; the prompt describes what happens next.

This is a strong fit for a recurring character moving between scenes: a presenter in a studio, then at a product launch, then walking through a city. The number of reference images available depends on the selected model, so use the model limit as a ceiling, not a target.

Your browser does not support embedded video playback.

Example of a recurring character moving through multiple European locations.

Use Image to Video when the opening composition is already correct

Choose Image to Video AI when you have approved the exact still that should become the first frame. This is often the simplest route to a stable single shot because the face, pose, wardrobe, lighting, and composition are already established before motion begins.

It works especially well for portrait motion, product demonstrations, subtle expressions, and scenes where the first frame matters more than matching a separate reference sheet.

Your browser does not support embedded video playback.

A restrained portrait-motion example generated from a reference image.

Use Frames to Video when the shot needs a controlled destination

Choose Frames to Video when both the beginning and ending composition matter. A start frame and an end frame can guide a reveal, transformation, camera move, or planned transition.

This gives you endpoint control, but the two frames must agree. If the face, outfit, or proportions differ between them, the model has to reconcile the conflict during the shot, often by morphing. Build both frames from the same approved character kit before asking the video model to connect them.

First and last frames define the two visual endpoints the model must connect.

A Reference-First Workflow for Consistent AI Video Characters

The following process is deliberately sequential. Do not generate ten finished scenes and inspect identity at the end. Validate the character at each stage, when mistakes are still cheap to replace.

Step 1: Define one shot, not the whole film

Write down the job of the next clip in one sentence:

A medium tracking shot of the character walking through a rainy street, glancing toward the camera, and stopping beneath a streetlight.

This sentence defines the subject, action, camera relationship, environment, and endpoint. If it contains three locations, several costume changes, and four camera moves, split it into separate shots.

Step 2: Match the reference to the camera angle

For the rainy street example, a three-quarter or full-body reference is more useful than an extreme facial close-up. If the camera will orbit behind the character, add a rear or side reference rather than asking the model to invent every unseen detail.

Check the reference at the size viewers will actually see. A face that looks clear when zoomed in may become ambiguous in a full vertical frame.

Step 3: Separate identity from motion in the prompt

Let the image establish who the character is. Use the prompt to direct what changes.

A weak prompt keeps redesigning the subject:

Beautiful young woman with dark hair, cinematic face, perfect features, stylish clothes, walking in a city, dramatic movement, dynamic camera, highly detailed.

The adjectives are broad, and several of them invite a new interpretation of the person.

A stronger motion prompt is easier to execute:

The referenced character walks slowly through a rain-soaked street, glances toward the camera once, and stops beneath a warm streetlight. Medium tracking shot, gentle handheld movement, natural body motion, shallow depth of field. Preserve the same face, short black hair, silver ear cuff, and navy jacket. No outfit change and no additional people.

The prompt has one action sequence, one camera idea, one environment, and a short identity lock.

Step 4: Keep the first motion test simple

Test a restrained version before adding a fast turn, spinning camera, crowd, flying fabric, or physical interaction. A slow walk, small head movement, or seated gesture tells you whether the reference and prompt work together.

When the simple version holds the character, add one layer of complexity at a time. That makes failures diagnosable. If identity breaks only after adding a fast orbit, the camera move is the variable to change—not the character design.

Your browser does not support embedded video playback.

A more demanding spinning-camera example for reviewing face, outfit, and silhouette stability during motion.

Step 5: Generate a short shot and inspect specific frames

Do not judge consistency from a looping thumbnail. Pause at the beginning, middle, and end. Then inspect the moment with the largest head turn or body movement.

Check:

Do the eyes, nose, jaw, and hairline still belong to the same person?
Did the outfit color, neckline, sleeves, or accessories change?
Did body proportions shift during movement?
Did fingers, hair, or another character merge into the face?
Does the final frame remain usable as an edit point?

Approve identity first. A beautiful camera move is not worth keeping if it breaks the character between shots.

Step 6: Save successful outputs as production assets

When a shot works, save the prompt, model, aspect ratio, reference set, and a clean frame from the result. That frame is now tested visual evidence of the character in a new angle or environment.

For the next shot, begin from the closest tested asset instead of returning to the oldest portrait every time. The reference library should grow from approved results, not from random variations.

How to Fix Common Character Consistency Problems

The face changes as soon as the character turns

The model may not have enough information about the side of the face. Add a matching three-quarter or profile reference, reduce the speed of the turn, or change the camera so it does not demand a full unseen angle in one motion.

The outfit changes color or shape

Keep the outfit visible in the reference and name only its defining details in the identity lock. Remove style words that conflict with it. “Futuristic fashion,” for example, may encourage the model to redesign a simple jacket even if the reference shows exactly what you want.

Two characters blend together

Establish each character separately before generating interaction shots. Make their silhouettes, clothing colors, and signature features visually distinct. Begin with simple staging, such as standing side by side, before attempting an embrace, fight, or fast dance where limbs and faces overlap.

The first frame is right but the face degrades over time

Reduce simultaneous demands. Shorten the action, simplify the background, slow the camera, or divide the sequence into two clips. A single generation that must preserve identity, perform complex choreography, reveal a new environment, and execute a dramatic camera move is carrying too many jobs.

Every generation is a different kind of wrong

Return to the anchor kit. If the references disagree, prompts cannot reliably repair them. Pick one master identity, rebuild only the necessary angles from it, and run a neutral motion test before returning to the production scene.

A Practical Prompt Formula

Use this order when writing a character video prompt:

Referenced character + one action + environment + camera framing/movement + lighting/mood + short identity lock + exclusions

For example:

The referenced character sits at a cafe table and closes a sketchbook, then looks through the window. Medium close-up with a slow push-in, soft morning light, quiet reflective mood. Preserve the same face, bob haircut, round glasses, and green cardigan. No speaking, no extra people, no wardrobe change.

The formula is a diagnostic tool, not a reason to make every prompt long. If the result drifts, remove optional details until the model can preserve the character and perform the central action. Then add detail back carefully.

Final Consistency Checklist

Before generating:

One approved master reference
Additional views only when required by the shot
Matching identity, outfit, age, and visual style across references
One clear action and one primary camera idea
A short list of identity details that must not change

Before accepting a clip:

Same recognizable face at the start, middle, and end
Stable hair, clothing, accessories, and body proportions
No unwanted character merging or new people
Motion that does not hide a severe identity break
A final frame that can cut cleanly into the next shot
Saved prompt, settings, references, and approved output

Consistency Comes From the Pipeline

The recurring mistake is treating every clip as a separate prompt-writing contest. A sequence becomes more reliable when each shot inherits approved visual information from the one before it.

Start with one character, one clean reference, and one simple action. Use Reference to Video when identity must travel into a new scene, use Image to Video AI when the opening still is already correct, and use Frames to Video when the destination frame matters too.

Lock the character first. Direct the motion second. Review before you scale. That order will do more for continuity than another paragraph of adjectives.

Validating Paths in a Browser-Based Word Puzzle Game

Jamie Cole — Sat, 13 Jun 2026 05:42:22 +0000

Word puzzle games often look simple: players connect letters, form words, and finish the board.

The implementation is less simple.

A browser-based word puzzle needs to answer a few important questions:

Is the selected path legal?
Are the selected tiles adjacent?
Is any tile reused?
Does the selected path form an accepted word?
Can the final board be completed fairly?

This post focuses on the validation layer behind a small word path puzzle game.

Representing a tile

A tile needs more than just a letter. It also needs a position.

const tile = {
  row: 1,
  col: 2,
  letter: "R"
}

The row and col values are important because every movement rule depends on position.

Without stable coordinates, the game cannot reliably check whether two tiles are connected.

Checking adjacency

The first rule is movement.

If diagonal movement is allowed, a tile can move to any of the eight neighboring tiles.

function isAdjacent(a, b) {
  const dx = Math.abs(a.row - b.row)
  const dy = Math.abs(a.col - b.col)

  return dx <= 1 && dy <= 1 && !(dx === 0 && dy === 0)
}

This function rejects the same tile and accepts horizontal, vertical, and diagonal neighbors.

If the puzzle should only allow horizontal and vertical movement, the rule can be changed:

function isOrthogonallyAdjacent(a, b) {
  const dx = Math.abs(a.row - b.row)
  const dy = Math.abs(a.col - b.col)

  return dx + dy === 1
}

This small difference changes the entire feel of the puzzle.

Diagonal movement gives players more freedom. Orthogonal movement creates stricter paths and often makes puzzles easier to analyze.

Preventing reused tiles

A common rule in word path puzzles is that a tile cannot be reused inside the same path.

The simplest way to check this is to store each tile coordinate in a set.

function hasDuplicateTiles(path) {
  const seen = new Set()

  for (const tile of path) {
    const key = `${tile.row},${tile.col}`

    if (seen.has(key)) {
      return true
    }

    seen.add(key)
  }

  return false
}

This prevents loops where the player returns to a tile that has already been used.

Validating the full path

Once adjacency and duplicate checks exist, a full path validator is simple.

function isValidPath(path) {
  if (path.length === 0) {
    return false
  }

  if (hasDuplicateTiles(path)) {
    return false
  }

  for (let i = 1; i < path.length; i++) {
    if (!isAdjacent(path[i - 1], path[i])) {
      return false
    }
  }

  return true
}

This only validates movement.

It does not decide whether the path forms a real word. That should be handled separately.

Converting a path into a word

A path can be converted into a word by joining the tile letters.

function pathToWord(path) {
  return path.map(tile => tile.letter).join("")
}

Then it can be checked against an accepted word list.

function isAcceptedWord(path, acceptedWords) {
  const word = pathToWord(path).toLowerCase()
  return acceptedWords.has(word)
}

For puzzle games, I usually prefer a curated answer list instead of a large dictionary.

A large dictionary may create many accidental valid words. A curated list gives the designer more control over the intended solution.

Checking board coverage

Some word path puzzles require the final solution to cover every required tile.

That means the game needs to know how many unique tiles are covered by all solved paths.

function getCoveredTiles(paths) {
  const covered = new Set()

  for (const path of paths) {
    for (const tile of path) {
      covered.add(`${tile.row},${tile.col}`)
    }
  }

  return covered
}

Then the board can be checked like this:

function isBoardComplete(paths, requiredTileCount) {
  return getCoveredTiles(paths).size === requiredTileCount
}

This is useful because a puzzle can contain valid words but still be incomplete.

A fair board should not leave impossible or confusing leftover tiles.

Avoiding overlapping solution paths

If each tile should belong to only one final answer, paths must not overlap.

function pathsOverlap(paths) {
  const used = new Set()

  for (const path of paths) {
    for (const tile of path) {
      const key = `${tile.row},${tile.col}`

      if (used.has(key)) {
        return true
      }

      used.add(key)
    }
  }

  return false
}

This check becomes important when generating puzzles automatically.

A generator can accidentally create boards where multiple solution paths fight for the same tile.

A practical generation approach

For a first version, I would not start with fully random letters.

Random boards are easy to create but hard to make fair.

A more controlled approach is:

choose a board size
choose a small curated word list
place each word as a legal path
reject overlapping paths
check board coverage
review difficulty before publishing

This is not a perfect generator, but it is a useful starting point.

For browser games, a small reliable generator is better than a large unreliable one.

Daily puzzles are easier to control

An infinite random mode sounds exciting, but quality control becomes harder.

Daily puzzles are easier because the game only needs one good board per day.

That allows extra checks:

path legality
word validity
board coverage
duplicate tile prevention
difficulty review

For a small V0, daily mode is often the better choice.

Final thoughts

The hard part of a word path puzzle is not rendering the grid.

The hard part is validation.

Before adding accounts, leaderboards, animations, or sharing features, the core rules should be solid:

legal movement
no reused tiles
accepted words only
no invalid overlap
complete board coverage

Once those rules work, the game becomes much easier to improve.

I used this approach while experimenting with a small independent browser puzzle project. The main lesson was simple: build the validator first, then build the game around it.

How I Built a Performant Logic Puzzle Game with Next.js and a Web Worker

Jamie Cole — Tue, 19 Aug 2025 02:05:11 +0000

Hey DEV community!

I've always been a huge fan of logic puzzles. Recently, I got hooked on the Queens puzzle game but felt the classic rules were a bit too simple. This sparked an idea: what if I could create a more modern, more challenging version?

Today, I'm excited to share the result of that idea: Enhanced Queens Game, a free-to-play web game I built from scratch.

🎮 You can play it here: queens game

This post is a quick breakdown of the journey, the tech stack, and one interesting technical challenge I solved.

The Core Idea: Adding a "Third Dimension"
The biggest change from the classic puzzle is the addition of colored regions. On top of having only one queen per row and column, you must also have only one per colored block. This simple twist turns a 2D puzzle into a much deeper 3D spatial reasoning challenge.

![Insert a GIF of the gameplay, showing the colored regions and a queen turning red]

The Tech Stack
I chose a modern stack to build this, focusing on performance and developer experience:

Framework: Next.js (for great SEO and fast performance)

Language: TypeScript (a must for managing complex game logic)

Styling: Tailwind CSS (for building the clean UI quickly)

The Coolest Technical Challenge: Real-time Validation without Lag
A key feature is that a queen turns red instantly if it breaks a rule. On larger 7x7 boards, checking all conflicts (row, column, region, and adjacency) on every single move could potentially lag the main UI thread.

The solution? A Web Worker.

I offloaded the most intensive validation logic to a separate background thread using a Web Worker (solveQueensWorker.ts in the repo). The main thread sends the current board state to the worker, the worker runs the heavy computations, and then sends the results (any conflicts) back.

This ensures the UI remains buttery smooth and responsive, even while complex calculations are happening in the background. It was a perfect use case for a worker and a fun problem to solve.

Feedback Welcome!
This has been an incredibly rewarding personal project. I'd be honored if you'd take a look. I'm open to any and all feedback—whether it's on the game's difficulty, the UI/UX, or the code itself.

Thanks for reading!