DEV Community: Super Lewis

Your AI Eraser Redraws the Whole Photo: Inside a Label-Then-Edit Object Remover

Super Lewis — Thu, 22 Oct 2026 13:55:15 +0000

This week two of our own help pages contradicted each other. MagicRemover's passersby page promised that only the area you brush changes; the people page said the whole image may shift slightly. Only one could be true, so I read our pipeline end to end instead of trusting either page.

The people page was right. An instruction-based eraser doesn't fill a hole: it re-renders the entire photo from a text instruction, and your brush stroke only decides what that instruction says. This post walks through how that pipeline works in production, why it was worth switching to, and what it costs the pixels you never touched.

Disclosure: I build MagicRemover, the browser-based AI eraser this post is about. The draft was written with AI assistance and every code excerpt and number was checked against the repository and production logs.

Two ways to build an eraser

There are two broad designs, and they fail in opposite ways.

Mask inpainting takes the image plus a binary mask. The model generates pixels only inside the mask, and the pipeline copies everything outside it straight from the original. Outside the brush, the result is pixel-identical to your upload. The weakness is context: if you brush a person but not their shadow, the shadow stays, because the model was only allowed to touch the mask.

Instruction editing takes the image plus a sentence such as "remove the man in the blue shirt". The model returns a complete new image. Because it understands the scene, it can remove the shadow and the reflection you didn't brush. The cost is that every pixel in the output was generated, including the ones you wanted left alone.

MagicRemover's photo tools ran on mask inpainting until May 2026. In early May we added a labelling step; in June we replaced the backend with an instruction-following image editor that takes only an image and a prompt. Fills got noticeably more coherent, especially around shadows. What we didn't do was update every sentence on the site that still described the old behaviour.

How a brush stroke becomes a sentence

Here is the path a removal takes today.

1. The browser shrinks the photo to about 1 megapixel

Every upload goes through one helper before anything leaves the browser:

/** Cap uploads at ~1 megapixel (matches the model's ~1MP working resolution). */
export const MAX_PIXELS = 1_000_000;
/** Secondary edge cap for very wide panoramas. */
export const MAX_EDGE = 2048;

export function fitTarget(w: number, h: number) {
  let scale = 1;
  const pixels = w * h;
  if (pixels > MAX_PIXELS) scale = Math.sqrt(MAX_PIXELS / pixels);
  const longest = Math.max(w, h) * scale;
  if (longest > MAX_EDGE) scale *= MAX_EDGE / longest;
  return {
    width: Math.max(1, Math.round(w * scale)),
    height: Math.max(1, Math.round(h * scale)),
  };
}

A 4000×3000 phone photo becomes 1155×866. There is no point sending more: the editing model works at roughly one megapixel and returns an image at that size, so extra pixels would only make the request slower and bigger.

2. The browser sends two copies

The editor posts the clean, downscaled original and a second copy with the user's brush strokes filled solid red. The red copy is never edited; it exists so the next step can see what was marked.

3. A vision model names what is under the red

A vision-language model receives both images and a tool-specific instruction. The people tool's instruction starts like this:

Identify the person or people inside the red-marked region of image 2.
Your reply MUST uniquely identify ONLY the red-marked subjects so they can
be deleted without affecting any similar-looking person ...

It answers with a short label, along the lines of "the man in the blue shirt on the left". In the nine removals logged since our latest deploy, all nine got a label, and the labelling step took about 2.5 seconds on average. That's a small sample, but it matches what we see day to day.

4. The editor gets the clean photo and a sentence

The label is dropped into a per-tool template. The people tool uses:

Completely remove {label} from the photo, leaving no trace, shadow, or
outline, and realistically reconstruct the background and scene behind
them. Keep everyone and everything else exactly unchanged.

The editor receives the clean original, never the red copy, so there's no risk of red pixels leaking into the result.

5. If labelling fails, fall back to the red copy

When the vision model can't produce a label, the editor gets the red-marked copy and a generic instruction instead:

Completely remove whatever the red area covers, then remove the red marking
itself and fill the space with background that naturally matches the
surroundings. Leave no trace, shadow, or outline. Keep everything outside
the red area exactly unchanged.

That keeps the tool working when the labeller is down, at the cost of slightly worse results, because "the red area" is a vaguer target than "the man in the blue shirt".

Why the whole photo changes

Notice the last sentence of both prompts: "Keep everything else exactly unchanged." It's a request, not a guarantee. The model regenerates the full frame, and nothing in the pipeline copies the original pixels back.

In practice the untouched parts of the photo look the same, but they aren't bit-identical. The things most likely to drift are fine repeating textures (fabric, foliage, gravel), small text away from the brushed area, and faces in the background. Colour can shift slightly across the whole image too.

That's acceptable for the use most people have: a holiday photo going to Instagram or a chat. It's not acceptable for an e-commerce product shot that has to match its colour profile, or a scanned document where every other character must survive untouched. So the pages now say it plainly: changes concentrate in the brushed area, the rest looks the same but isn't pixel-identical.

Why we don't paste the original back (yet)

The obvious fix is to composite: keep the edited pixels inside the brushed area, take the original's pixels everywhere else. It's about twenty lines with sharp. This is an example, not production code; it's tested with sharp 0.34.5 on Node 24:

import sharp from "sharp";

const { width, height } = await sharp("original.png").metadata();

// 1. Grow the brushed area so the seam lands outside the object's edge:
//    blur the mask, then turn every non-black pixel white.
const blurred = await sharp("mask.png").resize(width, height).blur(8).toBuffer();
const grown = await sharp(blurred).threshold(1).toBuffer();

// 2. Feather the grown edge and keep a single raw channel to use as alpha.
const alpha = await sharp(grown).blur(6).extractChannel(0).raw().toBuffer();

// 3. The edited image at the original's size, with that alpha channel.
const rgb = await sharp("edited.png").resize(width, height).removeAlpha().toBuffer();
const patch = await sharp(rgb)
  .joinChannel(alpha, { raw: { width, height, channels: 1 } })
  .png()
  .toBuffer();

// 4. Lay the patch over the untouched original.
await sharp("original.png").composite([{ input: patch }]).toFile("result.png");

Two gotchas cost me time while testing it. sharp applies operations in a fixed internal order, not in the order you chain them, so blur, threshold and blur again only behave as written when each runs in its own pipeline. And removeAlpha() chained after joinChannel() quietly strips the channel you just joined, which is why step 3 builds the RGB buffer first.

So why isn't this in production? Because it gives back what instruction editing bought us.

Shadows come back. The editor removes a person's shadow even if the user brushed only the person. Composite outside the mask and the shadow is pasted right back from the original. Growing the mask helps a little; it can't know where a long evening shadow ends.
Seams appear. The editor relights the area it rebuilt. Where its version meets the original, a feathered edge turns a sharp seam into a soft halo, but on a flat sky or a white wall you can still see the step in brightness.
Resolution stays the same. Compositing keeps the original pixels outside the mask, but if you composite onto the downscaled original, the output is still about one megapixel. To get full resolution back you'd upscale the patch onto the full-size upload, and a 1 MP patch dropped into a 12 MP photo looks soft next to its neighbours.

A hybrid is on our list: composite only when the brushed area is small and far from strong shadows, and leave the full regeneration otherwise. Until then, the honest move was to change the copy rather than the pipeline.

When an instruction editor is the wrong tool

You need everything outside the mask to stay bit-identical: product photos with strict colour, documents, anything used as a record. Use mask inpainting with compositing, or Photoshop's Remove Tool on the full-resolution file.
You need print resolution. Any eraser that works at about one megapixel, MagicRemover included, will hand you back a smaller image.
The image is dense with text you want to keep. Regenerating the frame can subtly alter characters far from the brush.

Where MagicRemover fits

MagicRemover is a browser-based AI eraser: you upload a photo, brush over a person, object, text, sticker or watermark, and it returns the photo with that thing removed, usually in well under a minute. It's built for photos headed to the web and social media, with 2 free removals a day without an account and 2 more after Google sign-in. Don't use it when you need full resolution or pixel-exact output outside the brushed area; for those jobs a desktop editor on the original file is the right call.

The tool walkthroughs, including the people remover discussed above, are at magicremover.org.

If you've shipped an eraser yourself: do you composite the original back, and how do you deal with shadows that fall outside the user's mask?

FLUX 3 Max Keyframes Explained: 24 fps Frame Math, Validation Rules and a Node.js Helper

Super Lewis — Wed, 21 Oct 2026 01:30:12 +0000

FLUX 3 Max is a fast video generation model by Black Forest Labs (a latency-optimized variant of FLUX 3 Video). Its keyframes-to-video mode lets you pin 1 to 10 images to exact positions in a 5 to 20 second clip and have the model generate the motion between them. This post is about that mode only: how positions are counted, which requests get rejected and why, and a small Node.js helper that turns a list of images into valid keyframes.

Disclosure: I work on apimodels.app, a multi-model API where FLUX 3 Max is available as flux-3-max. The request shapes and error messages below are the ones that API returns; I tested them on 20 October 2026.

Positions are frame numbers at 24 fps

A keyframe is an image plus a position, and the position is a frame number in the 24 fps output, not a timestamp:

{ "image_url": "https://example.com/a.jpg", "frame_index": 72 }

The rules that follow from that:

Frame 0 is the first frame of the video.
The last frame is duration × 24. A 5-second clip ends at frame 120, a 10-second clip at 240, the 20-second maximum at 480.
A position must be a whole number in that range, and two keyframes cannot share one.
Order in the array does not matter; they are sorted by position before generation.

Converting from seconds is Math.round(seconds × 24), so 2.5 s is frame 60 and 3 s is frame 72. apimodels.app accepts { image_url, time } with time in seconds and does that conversion for you, which is handy when your storyboard is written in seconds. You can mix both forms in one array.

Keyframes replace first and last frame

FLUX 3 Max has a separate image-to-video mode driven by first_frame_url and an optional last_frame_url. The two modes do not combine: if a request carries both a keyframes array and a first or last frame, the API returns 400 rather than guessing which one you meant.

To lock both ends and something in the middle, put all of them in keyframes:

"keyframes": [
  { "image_url": "https://example.com/start.jpg", "time": 0 },
  { "image_url": "https://example.com/middle.jpg", "time": 3 },
  { "image_url": "https://example.com/end.jpg", "time": 6 }
]

with "duration": 6, so the last image sits on frame 144, the final frame.

The 400s you will meet

Every one of these is checked before the task is created, so a rejected request costs nothing. The messages are what apimodels.app returns:

You sent	Message
`keyframes: []`	keyframes must be a non-empty array of {image_url, frame_index} (or {image_url, time} in seconds)
11 or more keyframes	keyframes accepts at most 10 images (got 11)
`time: 7` with `duration: 6`	keyframes[2] frame position must be a whole frame from 0 to 144 (duration × 24 fps; got 7)
Two keyframes at frame 72	keyframes[1] repeats frame 72; each keyframe needs its own position
A keyframe with no position	keyframes[0] needs frame_index (24 fps frame number) or time (seconds)
`keyframes` plus `first_frame_url`	Send either keyframes or first_frame_url / last_frame_url, not both
Several images in `images`	takes one first frame (plus an optional last frame) or a keyframes array, not 3 reference images

The out-of-range one is the most common: you shorten duration from 10 to 6 and forget that a keyframe at 8 seconds no longer fits. The fix is to compute positions from the duration instead of hardcoding them.

A helper that spaces images evenly

Most storyboards are "these images, in this order, evenly across the clip". This helper does that and stays inside the rules: first image on frame 0, last image on the final frame, the rest spread between them, no duplicates.

const FPS = 24;

// images: array of URLs in story order (1 to 10)
// duration: whole seconds, 5 to 20
function evenKeyframes(images, duration) {
  if (images.length < 1 || images.length > 10) throw new Error("1 to 10 images");
  const last = duration * FPS;
  if (images.length === 1) return [{ image_url: images[0], frame_index: 0 }];
  return images.map((url, i) => ({
    image_url: url,
    frame_index: Math.round((i * last) / (images.length - 1)),
  }));
}

console.log(evenKeyframes(["a.jpg", "b.jpg", "c.jpg", "d.jpg"], 6));
// frames 0, 48, 96, 144

Even spacing is a starting point, not a rule. If a shot needs to linger, move its keyframe later; the model has more frames to work with on that stretch.

Full request with polling

The API is asynchronous: create a task, then poll the same endpoint with the task id (or pass callback_url to get the result POSTed to you).

const API = "https://api.apimodels.app/v1/video/generations";
const headers = {
  Authorization: `Bearer ${process.env.APIMODELS_API_KEY}`,
  "Content-Type": "application/json",
};

const duration = 6;
const create = await fetch(API, {
  method: "POST",
  headers,
  body: JSON.stringify({
    model: "flux-3-max",
    prompt:
      "A smooth cinematic transition from a casual kitchen snapshot to a premium " +
      "studio product shot, light sweeps across the product",
    duration,
    resolution: "720p",
    keyframes: evenKeyframes(
      ["https://example.com/start.jpg", "https://example.com/mid.jpg", "https://example.com/end.jpg"],
      duration
    ),
    generate_audio: false,
  }),
}).then((r) => r.json());

if (create.code !== 200) throw new Error(create.msg); // the 400s above land here

let task;
do {
  await new Promise((r) => setTimeout(r, 3000));
  task = (await fetch(`${API}?task_id=${create.data.taskId}`, { headers }).then((r) => r.json())).data;
} while (task.state === "pending");

if (task.state === "failed") throw new Error(task.failMsg);
console.log(task.resultUrls[0]);

In my run (two square product images at 0 s and 5 s, 5 seconds, 720p, audio off, reasoning_level: "low") the video was ready 19 seconds after the create request, at 960×960. One run is a spot check, not a benchmark. With aspect_ratio left at auto the output follows the first image, so square inputs give a square video.

Prompting between keyframes

The keyframes fix what the frames look like; the prompt is where you describe the motion that connects them. A few things that help:

Describe the transition, not the images. "Light sweeps across as the camera pushes in" tells the model something the images cannot.
Keep subject, lighting and lens consistent across keyframe images when you want continuity. Very different images force a transition the model has to invent.
Leave generate_audio on and describe the sound ("soft whoosh, quiet room tone") if you want audio that follows the motion; turn it off if you will score the video yourself.

When keyframes are the wrong tool

You want the opening image to drive the whole clip and do not care about intermediate frames: use plain image-to-video with first_frame_url, which is a simpler request.
You need the same character across many separate shots, or more than 20 seconds in one pass: FLUX 3 Max takes at most 10 images and 20 seconds. Seedance 2.5 accepts up to 50 reference assets and 30 seconds.
You need to transfer motion from a reference video: FLUX 3 Max does not accept reference videos.

Cost

On apimodels.app FLUX 3 Max is billed per second of output, and keyframes cost the same as the other modes: $0.221/s at 720p and $0.377/s at 1080p, so the 6-second 720p example is $1.33. Until 7 November 2026 (00:00 UTC) there is a special price of $0.085/s and $0.145/s. Failed generations and rejected requests are not charged.

Parameters, pricing and a comparison with Seedance 2.5, Kling V3 and Veo 3.1 are in the FLUX 3 Max API guide; the playground with a keyframe uploader is on the FLUX 3 Max model page.

Written with AI assistance for structure and wording; the code, error messages and timing come from my own test runs.

Build a Grok Voice 3.0 Voice Agent in Node.js: Mic In, Speaker Out, Live Transcripts

Super Lewis — Sat, 17 Oct 2026 15:46:39 +0000

You open a WebSocket to a speech-to-speech model, start streaming your microphone, and three things go wrong at once: the model answers itself because it hears its own voice through your speakers, your transcript prints the same sentence a dozen times, and when you ask "who are you?" it describes its origin wrong. All three have small, specific fixes, and this tutorial builds them in from the start.

Grok Voice 3.0 is xAI's speech-to-speech model for realtime voice agents, released on 14 October 2026: you stream 16-bit PCM audio over one WebSocket and the same model answers with audio, using events that xAI describes as compatible with the OpenAI Realtime API. By the end you will have a terminal voice agent in Node.js that listens to your mic, plays replies, prints live transcripts and calls a function, plus the browser variant that never exposes your API key.

Disclosure: I work on apimodels.app, the API gateway the examples connect to, and the relay behaviour described below comes from its source code.

What you need

Node.js 18 or newer and the ws package 8.x (pnpm add ws or npm install ws)
SoX on your PATH for microphone and speaker I/O: brew install sox on macOS, sudo apt install sox on Debian or Ubuntu. The script uses its rec and play commands.
An apimodels.app API key in the APIMODELS_API_KEY environment variable, with at least one minute of balance ($0.104), which the server checks before it accepts the connection
Headphones. With open speakers the model hears its own voice and keeps answering itself.

How a Grok Voice session flows

A Grok Voice 3.0 session is a single WebSocket that carries JSON events in both directions, with audio inside the events as base64. You connect, send one session.update to set the voice, instructions and turn detection, then keep appending microphone audio with input_audio_buffer.append. With turn_detection: {"type": "server_vad"} the server decides when you stopped talking and starts answering on its own; you never send a "done" signal.

The answer arrives as a stream: response.output_audio.delta events carry base64 PCM chunks to play in order, response.output_audio_transcript.delta events carry the text of what the model is saying, and response.done closes the turn. Your own speech comes back as conversation.item.input_audio_transcription.updated.

That last event is where the "same sentence a dozen times" bug comes from. xAI's input transcription is cumulative: each updated event carries the whole transcript so far, which may correct earlier words, not a delta to append. Code written for OpenAI-style incremental deltas prints every partial version.

The default audio format is 16-bit PCM, 24 kHz, mono, little-endian, in both directions, so 100 ms of audio is 2,400 samples or 4,800 bytes.

Step 1: Connect and surface handshake errors

Save the four blocks in Steps 1 to 4 into one file, voice-agent.mjs, in order. This first block opens the socket with your key in the Authorization header and prints the server's reason if the handshake is rejected.

// voice-agent.mjs (example). Node 18+, ws 8.x, SoX on PATH. Use headphones.
import { spawn } from "node:child_process";
import WebSocket from "ws";

const URL = "wss://api.apimodels.app/v1/realtime?model=grok-voice-3.0";
const SOX = ["-q", "-t", "raw", "-r", "24000", "-e", "signed-integer", "-b", "16", "-c", "1", "-"];

const ws = new WebSocket(URL, {
  headers: { Authorization: `Bearer ${process.env.APIMODELS_API_KEY}` },
});
const send = (event) => ws.readyState === WebSocket.OPEN && ws.send(JSON.stringify(event));

// A rejected handshake is a plain HTTP response with a JSON body:
// 401 bad key, 400 wrong model, 402 less than one minute of balance, 429 more than 5 sessions.
ws.on("unexpected-response", (_req, res) => {
  let body = "";
  res.on("data", (c) => (body += c));
  res.on("end", () => {
    console.error(`handshake rejected: ${res.statusCode} ${body}`);
    process.exit(1);
  });
});

With a wrong or missing key you see handshake rejected: 401 {"error":{"type":"invalid_api_key","code":"invalid_api_key","message":"Invalid or missing API key / client secret."}} and nothing is charged.

Step 2: Configure the session, identity included

As soon as the socket opens, send session.update. The voice, instructions and turn_detection fields sit directly under session.

ws.on("open", () => {
  send({
    type: "session.update",
    session: {
      voice: "eve", // one of 28 built-in voices; each one speaks every supported language
      instructions:
        "You are Grok Voice 3.0, a realtime voice assistant built by xAI, " +
        "answering for the Acme help desk. Keep answers under three sentences.",
      turn_detection: { type: "server_vad" },
      audio: {
        input: { format: { type: "audio/pcm", rate: 24000 } },
        output: { format: { type: "audio/pcm", rate: 24000 } },
      },
      tools: [
        {
          type: "function",
          name: "get_order_status",
          description: "Look up an order by its id",
          parameters: {
            type: "object",
            properties: { order_id: { type: "string" } },
            required: ["order_id"],
          },
        },
      ],
    },
  });
});

The first sentence of instructions is the fix for the third bug. Setting custom instructions replaces the model's default persona, and without an identity in your prompt the model may describe its origin incorrectly when a caller asks who it is. If your product has its own persona ("Mia from Acme"), write that instead; just write something.

Step 3: Stream the microphone in 100 ms frames

Start the mic only after the server confirms the session with session.updated, so your first words are not processed with default settings. SoX's rec writes raw PCM to stdout; the code cuts it into 4,800-byte frames so no 16-bit sample is ever split across two events.

let mic = null;
function startMic() {
  mic = spawn("rec", SOX, { stdio: ["ignore", "pipe", "ignore"] }); // raw 16-bit PCM, 24 kHz, mono
  let pending = Buffer.alloc(0);
  mic.stdout.on("data", (chunk) => {
    pending = Buffer.concat([pending, chunk]);
    while (pending.length >= 4800) { // 100 ms = 2,400 samples x 2 bytes
      send({ type: "input_audio_buffer.append", audio: pending.subarray(0, 4800).toString("base64") });
      pending = pending.subarray(4800);
    }
  });
}

function newSpeaker() {
  const p = spawn("play", SOX, { stdio: ["pipe", "ignore", "ignore"] }); // reads raw PCM on stdin
  p.stdin.on("error", () => {}); // ignore EPIPE after we kill it for a barge-in
  return p;
}
let speaker = newSpeaker();

Step 4: Play replies, print transcripts, answer function calls

One message handler does the rest. Note the transcription case: it overwrites the current line instead of appending, because each event already holds the full sentence.

ws.on("message", (raw) => {
  const ev = JSON.parse(raw.toString());
  switch (ev.type) {
    case "session.updated":
      if (!mic) { startMic(); console.log("listening... (Ctrl+C to hang up)"); }
      break;

    case "input_audio_buffer.speech_started":
      // Caller started talking over the model: drop queued audio.
      // (Our browser playground handles this event the same way.)
      speaker.kill();
      speaker = newSpeaker();
      break;

    case "conversation.item.input_audio_transcription.updated":
    case "conversation.item.input_audio_transcription.completed":
      // Cumulative: overwrite the line, never append.
      process.stdout.write(`\r\x1b[2Kyou:  ${ev.transcript ?? ""}`);
      break;

    case "response.created":
      process.stdout.write("\ngrok: ");
      break;
    case "response.output_audio_transcript.delta":
      process.stdout.write(ev.delta);
      break;
    case "response.output_audio.delta":
      speaker.stdin.write(Buffer.from(ev.delta, "base64"));
      break;
    case "response.done":
      process.stdout.write("\n");
      break;

    case "response.function_call_arguments.done": {
      const args = JSON.parse(ev.arguments || "{}");
      const result = { order_id: args.order_id, status: "shipped" }; // replace with your lookup
      send({
        type: "conversation.item.create",
        item: { type: "function_call_output", call_id: ev.call_id, output: JSON.stringify(result) },
      });
      send({ type: "response.create" }); // ask the model to speak the result
      break;
    }

    case "error":
      console.error(`\n[error] ${ev.error?.code}: ${ev.error?.message}`);
      break;
  }
});

function hangUp() { mic?.kill(); speaker.kill(); ws.close(); }
process.on("SIGINT", hangUp);
ws.on("close", (code, reason) => {
  console.log(`\nsession closed (${code} ${reason})`);
  mic?.kill(); speaker.kill();
});

Closing the socket is not cleanup trivia: it is what stops the billing clock. Sessions are billed by connected time, and the relay only treats a session as idle after 10 minutes with no client events at all. A process that keeps streaming microphone audio, even silence, is never idle and runs until the 60-minute cap.

Verify it works

Run it with headphones on:

APIMODELS_API_KEY=your_key_here node voice-agent.mjs

You should see listening..., then your own words appear on a you: line that updates in place while you speak. Ask "Who are you, and where is order A-1001?" The reply should introduce itself as Grok Voice 3.0 built by xAI (from your instructions), and the function call should come back as a spoken "shipped". Press Ctrl+C; the session appears as one record in your console call history with its duration and cost.

Do the same from a browser

Browsers cannot set headers on a WebSocket, so never put the API key in front-end code. Your server mints a short-lived token and the page passes it as a subprotocol.

// On your server (example): exchange the key for a token valid 10 to 600 seconds (default 60)
export async function mintRealtimeToken() {
  const res = await fetch("https://api.apimodels.app/v1/realtime/client_secrets", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.APIMODELS_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ expires_after: { seconds: 60 } }),
  });
  return res.json(); // { value, expires_at }
}

// In the browser (example), from a click handler so audio is allowed to start
const { value } = await (await fetch("/api/realtime-token", { method: "POST" })).json();
const ws = new WebSocket(
  "wss://api.apimodels.app/v1/realtime?model=grok-voice-3.0",
  ["xai-client-secret." + value],
);

The token only governs when a connection can be opened; an open session keeps running after it expires. The rest of the event handling is the same as in Node. For microphone capture, use an AudioWorklet and resample to 24 kHz yourself: Firefox refuses to connect a microphone to an AudioContext created at a non-default sample rate, which is why our own browser playground captures at the device rate and resamples in JavaScript. The Grok Voice 3.0 API docs on apimodels.app have a complete browser example.

Where apimodels.app fits

apimodels.app is a multi-model API gateway: one API key and OpenAI-compatible endpoints for 140+ image, video, audio and language models. Most mainstream models there have multiple upstream channels with automatic failover, failed requests are not charged, long jobs run asynchronously with webhooks, and any request's charge can be looked up with GET /v1/records/{id}. Grok Voice 3.0 itself has a single upstream route, so there is no failover for it; a session that cannot reach the model is not charged.

For this tutorial it supplied the endpoint, the key and the billing: $0.104 per minute of connected session, billed per second (a 90-second call costs $0.156), $0.0052 per typed text message, and no extra fee for function calls. You can also talk to the model with your microphone on the Grok Voice 3.0 model page before writing any code.

Do not route through it if you only use xAI, already have direct xAI access and want the lowest per-minute price: connect to xAI directly, where the same events work. The same goes if you need sessions longer than 60 minutes, more than 5 concurrent sessions per account, or xAI's server-side tools (web_search, x_search, file_search, mcp) inside the voice session, which the relay rejects.

Common errors

handshake rejected: 401 ... invalid_api_key: missing or wrong key, or a browser trying to send the key as a header. Use the Authorization header from Node, or a client secret from a browser.
handshake rejected: 400 ... "Model gpt-realtime is not available for realtime. Use grok-voice-3.0.": the model query parameter is wrong. Use grok-voice-3.0 (or grok-voice-latest).
handshake rejected: 404 ... Unknown API endpoint: GET /api/v1/realtime: you connected to apimodels.app/api/v1/realtime. The endpoint only exists on api.apimodels.app, because the main site's hosting layer cannot proxy WebSockets.
402 insufficient_balance at connect, or an error event with that code mid-call: balance below one minute. Each minute is charged when it starts, so the session closes when the next minute cannot be covered.
429 too_many_sessions: more than 5 open sessions on one account. Close sockets you no longer use.
[error] unsupported_tool: your session.update included web_search, x_search, file_search or mcp. Only function tools are accepted.
The model keeps answering itself: speaker echo. Use headphones, or echo cancellation in the browser.
The same sentence printed many times: you appended cumulative transcription events. Overwrite instead.

Limits

This script is an example built from the documented event protocol; run it once with headphones before you build on it. SoX device handling differs by OS, and the SoX arguments above are the only platform-specific part.
Barge-in via input_audio_buffer.speech_started is what our browser playground uses; xAI's documentation page does not list that event, so treat it as best-effort.
Sessions end at 60 minutes with a session_limit error; reconnect to continue.

What did you use for audio I/O in Node: SoX, a native addon, or a browser-only setup? I'd like to know what held up in production.

This post was written with AI assistance from the apimodels.app source code and public docs, and reviewed before publishing.

How to Call Gemini 4 Argon with the OpenAI SDK: Streaming, reasoning_effort and the Real Cost of Each Request

Super Lewis — Sat, 17 Oct 2026 15:27:14 +0000

You send Gemini 4 Argon a one-line question, get a one-word answer, and the usage block says you paid for about 70 output tokens. That is not a bug. Argon thinks before every answer, and thinking is billed as output.

Gemini 4 Argon (gemini-4-argon) is Google's flagship reasoning model, available on the paid tier of the Gemini API since 13 October 2026, with a 1M-token context window shared by input and output and up to 65,536 output tokens per response. This tutorial calls it through an OpenAI-compatible endpoint with the official openai Python package, streams long answers, sets the thinking level with reasoning_effort, and reads back exactly what each request cost.

Disclosure up front: the endpoint in the examples is apimodels.app, the API gateway I work on. The same code runs against Google's own OpenAI-compatible endpoint if you swap the base URL; I note the differences where they matter. This post was drafted with AI assistance from our docs and test notes, and every command was checked against those sources.

What you need

Python 3.9 or newer and the OpenAI SDK: pip install openai
curl for the first check
An API key, stored as an environment variable (placeholder below, never a real key in code):

export APIMODELS_API_KEY="sk_your_key_here"

Base URL: https://api.apimodels.app/v1 (with Google directly, it is https://generativelanguage.googleapis.com/v1beta/openai/ and a Gemini API key)

Why Argon costs more than the text you see

Argon always reasons before it answers, and Google bills those thinking tokens at the output rate. Google's pricing page says it plainly: "Output price (including thinking tokens)". Google's thinking guide lists three levels for Argon (low, medium, high), sets medium as the default, and thinking cannot be turned off.

Our own measurements show the effect. On 14 October 2026, asking Argon to reply with just "OK" used 66 thinking tokens. On 15 October, the same prompt used 105 thinking tokens at low and 302 at high. On one hard reasoning prompt, high ran for 8 to 10 minutes. So two habits matter from the first line of code: set the thinking level on purpose, and read the thinking count from every response.

Step 1: make the first call with curl

This request asks a short question at the lowest thinking level, which keeps the first test fast.

curl https://api.apimodels.app/v1/chat/completions \
  -H "Authorization: Bearer $APIMODELS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini-4-argon",
    "messages": [{"role": "user", "content": "In one sentence: what does a database index trade off?"}],
    "reasoning_effort": "low",
    "max_tokens": 4000
  }'

You get a standard chat-completion object. Look at usage: completion_tokens already includes thinking, and completion_tokens_details.reasoning_tokens tells you how much of it was thinking.

Step 2: call it from Python and read the usage

The OpenAI SDK needs only a different base_url. Using with_raw_response keeps the response headers, which carry the request ID and, for non-streaming calls on apimodels.app, the amount charged.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["APIMODELS_API_KEY"],
    base_url="https://api.apimodels.app/v1",
    timeout=900,  # high-effort runs can take minutes
)

raw = client.chat.completions.with_raw_response.create(
    model="gemini-4-argon",
    messages=[{"role": "user", "content": "Explain the N+1 query problem in two sentences."}],
    reasoning_effort="low",
    max_tokens=4000,
)
resp = raw.parse()
print(resp.choices[0].message.content)

u = resp.usage
thinking = getattr(u.completion_tokens_details, "reasoning_tokens", 0) or 0
print("input:", u.prompt_tokens, "| output incl. thinking:", u.completion_tokens, "| thinking:", thinking)
print("request id:", raw.headers.get("x-apimodels-request-id"))
print("charged (USD):", raw.headers.get("x-apimodels-cost"))

The last two lines are apimodels.app-specific headers; with Google directly they are absent and you calculate cost from usage yourself.

Step 3: stream long answers and keep the usage

Streaming is the safe default for Argon, because a long, high-effort answer can take minutes. Ask for usage in the final chunk with stream_options.

stream = client.chat.completions.create(
    model="gemini-4-argon",
    messages=[{"role": "user", "content": "Write a step-by-step plan to migrate a Flask app from SQLAlchemy 1.4 to 2.0."}],
    reasoning_effort="medium",
    stream=True,
    stream_options={"include_usage": True},
)

usage = None
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:
        usage = chunk.usage

print("\n\nusage:", usage)

In one test on 15 October 2026, a stream of 16,599 output tokens took 144 seconds, with the first token after 8.7 seconds, about 120 tokens per second after that. On api.apimodels.app a streamed Argon response can run for up to 30 minutes; a non-streaming request is cut off at 13 minutes (limits as of 17 October 2026). Use the api.apimodels.app host for long jobs.

Step 4: pick the thinking level with reasoning_effort

reasoning_effort maps to Argon's three thinking levels. apimodels.app sends none and minimal as low (Argon cannot turn thinking off), medium as medium, and high, xhigh and max as high; any other value is dropped and the model's default (medium) applies. Google's own OpenAI-compatible endpoint maps minimal to low for Argon as well.

A practical split:

low: extraction, short answers, formatting, anything a user waits on
medium: everyday coding and analysis
high: hard multi-step problems, run in the background and streamed

Do not use max_tokens as your cost control. Google's docs warn that when the limit is hit during reasoning, the answer can come back truncated or empty while the thinking tokens are still billed. Lower the thinking level instead, and keep max_tokens generous. The ceiling for a normal request is 65,536; anything above that returns a 400 error.

Step 5: pass tools without the $schema key

Tool calling works through the same endpoint: in our tests on 15 October 2026, single tools, three parallel tools, streaming and multi-turn tool conversations all passed. One schema detail trips people up. Some JSON Schema generators add a top-level "$schema" key, and Gemini rejects it:

Invalid JSON payload received. Unknown name "$schema" at 'tools[0].function_declarations[0].parameters'

Strip it before you send the tool definition:

def clean_parameters(schema: dict) -> dict:
    schema = dict(schema)
    schema.pop("$schema", None)
    return schema

Verify it works: look up what the request cost

Every successful LLM response from apimodels.app carries an x-apimodels-request-id header, streaming included. Pass that ID to the records endpoint to get the settled amount.

curl https://api.apimodels.app/v1/records/<request-id> \
  -H "Authorization: Bearer $APIMODELS_API_KEY"

When data.settled is true, data.credits is the final USD amount taken from your balance, and data.usage holds the token counts. A failed request always shows 0, and the lookup itself is free.

Where apimodels.app fits, and when to skip it

apimodels.app is a multi-model API gateway: one API key and an OpenAI-compatible endpoint, plus native Gemini and Anthropic formats, for more than 140 image, video, audio and language models. For Argon that means more than one upstream channel with automatic failover, no charge for failed requests, and the per-request lookup above. As of 17 October 2026 Argon costs $1.10 per million input tokens and $5.50 per million output there, 55% of Google's $2 / $10 list. Google's list rises to $4 / $20 on 1 December 2026, and apimodels.app moves to $2.20 / $11.00 at 00:00 UTC that day, still 55%, with no code change. Argon supports Google's implicit caching only: explicit cache objects are not supported, and whether a repeated prefix hits is decided on Google's side and is not guaranteed. Cached tokens bill at $0.055 per million ($0.11 from 1 December), but budget as if nothing is cached. Details are on the Gemini 4 Argon model page.

Skip it if you need more than 65,536 output tokens in one answer: that is Google's long-decode mode, which needs Google's background execution API, and apimodels.app does not expose it. Skip it too if you need a contract, compliance terms or provisioned throughput directly with Google.

Common errors

Invalid JSON payload received. Unknown name "$schema": your tool parameters contain a $schema key. Remove it (Step 5).
Empty content with finish_reason: "length": thinking used up max_tokens. Raise max_tokens or lower reasoning_effort.
400 when max_tokens is above 65,536: that is Google's synchronous output cap for Argon.
401: the key is missing or wrong. The OpenAI-format endpoint expects Authorization: Bearer <key>.
Client timeout on a long answer: switch to stream=True, or raise the SDK timeout for non-streaming calls.

FAQ

Does reasoning_effort work with Gemini 4 Argon? Yes. It maps to Argon's thinking levels low, medium and high, and it changes the bill: the same prompt used 105 thinking tokens at low and 302 at high in our test. Argon cannot turn thinking off, so none behaves like low. Set it per request rather than globally, and use high only for work that runs in the background.

How do I know what a single Argon request cost? Read usage.completion_tokens (thinking included) and usage.prompt_tokens from the response. On apimodels.app, non-streaming responses also carry the charged amount in x-apimodels-cost, and GET /v1/records/{id} returns the settled amount for any request, streaming or not.

Limits of this walkthrough

Timings are single runs from our tests on 14 and 15 October 2026; your prompts and context sizes will differ.
Google's long-decode mode, search grounding and code execution are not covered.
Prices change on 1 December 2026. Recheck them before you plan spend.

What do you set reasoning_effort to for coding agents: one global level, or per task?

This post was written with AI assistance from apimodels.app's docs and our test notes, and reviewed before publishing. I work on apimodels.app.

Claude Sonnet 5.5 and Haiku 5.5 in Python: Two SDKs, Prompt Caching and Haiku's 100K Pricing Cliff

Super Lewis — Sat, 17 Oct 2026 15:24:52 +0000

Short answer: call claude-haiku-5-5 for classification, extraction, routing and summaries, and claude-sonnet-5-5 for multi-step coding and long tool loops. Both work through the Anthropic SDK and, via a gateway, the OpenAI SDK. Keep each Haiku 5.5 prompt at or under 100,000 tokens, because above that every rate on the request is billed at five times.

Claude Sonnet 5.5 (released 28 September 2026) and Claude Haiku 5.5 (released 7 October 2026) are Anthropic's mid and small tiers of the Claude 5.5 family. Both have a 1M-token context window, 128K max output, text and image input, and adaptive thinking that is on by default. Anthropic's list prices are $2 / $10 per million input / output tokens for Sonnet 5.5 and $0.10 / $0.50 for Haiku 5.5.

Disclosure: the base URL in the examples belongs to apimodels.app, the API gateway I work on. Both models there run on more than one upstream channel with automatic failover, and failed requests are not charged. The snippets follow the request shapes in Anthropic's documentation, but I did not run them for this post, so check the output on your side.

What you need

Python 3.10 or later
pip install anthropic openai requests (any recent versions; newer request fields go through extra_body, so you do not depend on a specific SDK release)
An API key in APIMODELS_API_KEY. If you call Anthropic directly instead, drop the base_url lines and use your Anthropic key.

Which model for which call

Job	Model	Why
Ticket or intent classification, routing	Haiku 5.5	Short prompts, huge volume, 20x lower list price
Field extraction, summaries, context compaction	Haiku 5.5	Anthropic's stated home ground for Haiku 5.5
Subagent steps inside a coding agent	Haiku 5.5	Anthropic: "pairs well with Opus 5.5 and Sonnet 5.5 as a subagent"
Multi-file coding, debugging, long tool loops	Sonnet 5.5	Terminal-Bench 4.0: 70.6% against Haiku 5.5's 39.2% (Anthropic's Haiku launch table)
Prompts above 100K tokens	Either	Haiku bills them at 5x, but still a quarter of Sonnet's list price per token; decide on quality

Step 1: Call Haiku 5.5 with the Anthropic SDK

This is a ticket classifier. Effort is set to low because the job needs a label, not a chain of reasoning.

import os
import anthropic

client = anthropic.Anthropic(
    base_url="https://api.apimodels.app",          # no /v1 here; the SDK adds /v1/messages
    api_key=os.environ["APIMODELS_API_KEY"],
)

msg = client.messages.create(
    model="claude-haiku-5-5",
    max_tokens=4000,                                 # thinking tokens count toward this
    system="Classify the ticket as billing, bug, feature or other. Reply with the label only.",
    messages=[{"role": "user", "content": "I was charged twice for the same top-up."}],
    extra_body={"output_config": {"effort": "low"}},
)

# Haiku 5.5 can open with thinking blocks even if you never asked for thinking,
# so pick text blocks by type, never content[0].
label = "".join(b.text for b in msg.content if b.type == "text").strip()
print(label, msg.usage.input_tokens, msg.usage.output_tokens)

Three details matter here. There is no temperature: on Haiku 5.5 Anthropic only accepts temperature 1 or top_p 0.99, and any top_k is a 400. max_tokens is generous because thinking tokens count toward it, and a small value can stop the response after the thinking block with no text at all. And the request ends with a user turn: Haiku 5.5 rejects an assistant prefill even with thinking off.

Step 2: Call Sonnet 5.5 with the OpenAI SDK

If your codebase is built on the OpenAI SDK, point it at the gateway's OpenAI-compatible endpoint and change only the model string.

from openai import OpenAI

oai = OpenAI(
    base_url="https://api.apimodels.app/v1",
    api_key=os.environ["APIMODELS_API_KEY"],
)

resp = oai.chat.completions.create(
    model="claude-sonnet-5-5",
    reasoning_effort="medium",                       # mapped to Anthropic's effort setting
    messages=[
        {"role": "system", "content": "You are a senior Python reviewer."},
        {"role": "user", "content": "Find the race condition in this function:\n..."},
    ],
)
print(resp.choices[0].message.content)

On this endpoint the gateway does three translations worth knowing. If you leave out max_tokens, it fills in the model's own output ceiling instead of a small default; until 28 September it used 4,096, which cut long code files off mid-function. It drops temperature, top_p and top_k for both 5.5 models, so clients that send a default temperature do not hit Anthropic's 400. And reasoning_effort becomes Anthropic's effort level.

For tools on Sonnet 5.5, keep tool_choice at auto and set strict: true on the function. Anthropic rejects forced tool use on this model with tool_choice: type "tool" and "any" are not supported for this model., and its migration guide recommends auto plus strict instead.

Step 3: Cache the prefix you send every time

Prompt caching pays off on anything you repeat: a long system prompt, a policy document, a repository summary. The minimum cacheable prompt on both models is 512 tokens.

POLICY = open("refund_policy.md").read()   # a few thousand tokens, identical on every call

def ask(question: str):
    return client.messages.create(
        model="claude-sonnet-5-5",
        max_tokens=8000,
        system=[{"type": "text", "text": POLICY, "cache_control": {"type": "ephemeral"}}],
        messages=[{"role": "user", "content": question}],
    )

for q in ["Can a used top-up be refunded?", "How long does a refund take?"]:
    u = ask(q).usage
    print("write:", u.cache_creation_input_tokens, "read:", u.cache_read_input_tokens, "new:", u.input_tokens)

Expect the first call to report a cache write and the second, within five minutes, a cache read. When we tested this pattern on Sonnet 5.5 at launch, three calls with the same prefix gave one write and two reads.

Through apimodels.app, a cache read costs a tenth of fresh input on both models: $0.12 per million on Sonnet 5.5 (input $1.20, cache write $1.50) and $0.009 on Haiku 5.5 (input $0.09, cache write $0.1125). Every repeated prefix that hits the cache is billed at that rate instead of the input rate.

Step 4: Price the 100K cliff before it prices you

Anthropic's rule for Haiku 5.5: if a request's prompt is over 100,000 tokens, counting new input, cache reads and cache writes, every rate on that request is multiplied by five, output included. Sonnet 5.5 has no such step anywhere in its 1M window.

This function reproduces the rule with apimodels.app prices as of 17 October 2026 (swap in Anthropic's list prices if you call them directly; the shape is identical):

# USD per 1M tokens, apimodels.app, as of 2026-10-17
PRICES = {
    "claude-sonnet-5-5": {"in": 1.20, "out": 6.00, "cache_read": 0.12,  "cache_write": 1.50},
    "claude-haiku-5-5":  {"in": 0.09, "out": 0.45, "cache_read": 0.009, "cache_write": 0.1125},
}

def request_cost(model, fresh_in, out, cache_read=0, cache_write=0):
    p = PRICES[model]
    prompt = fresh_in + cache_read + cache_write
    mult = 5 if model == "claude-haiku-5-5" and prompt > 100_000 else 1
    return mult * (fresh_in * p["in"] + out * p["out"]
                   + cache_read * p["cache_read"] + cache_write * p["cache_write"]) / 1_000_000

Running it on a few request shapes gives:

Request	Haiku 5.5	Sonnet 5.5
99K prompt, 1K output	$0.00936	$0.12480
101K prompt, 1K output	$0.04770	$0.12720
Same 101K split into two 50.5K requests	$0.01000	not needed
150K prompt (140K cache read + 10K new), 2K output	$0.01530	$0.04080

Read it in three ways. Two thousand extra tokens multiply the Haiku request by 5.1. Splitting the job into two prompts under the line cuts it by about 79% (you pay for a merge step on top). And caching does not get you back under the threshold, because cache reads count toward it.

What the table does not say is "switch to Sonnet". Over the cliff, Haiku 5.5 is still 37.5% of Sonnet 5.5's price for the same request here, and a quarter of it at Anthropic's list. Move a long-prompt job to Sonnet when Haiku's answers are not good enough, not to save money.

To catch the cliff in production, log the prompt size from every response:

def prompt_tokens(u):
    return u.input_tokens + (u.cache_read_input_tokens or 0) + (u.cache_creation_input_tokens or 0)

if prompt_tokens(msg.usage) > 100_000:
    print("billed at 5x:", prompt_tokens(msg.usage), "prompt tokens")

Verify what you were billed

Every response from the gateway carries an x-apimodels-request-id header, and non-streaming responses also carry x-apimodels-cost. The record endpoint returns the settled amount and the token usage behind it.

import requests

raw = client.messages.with_raw_response.create(
    model="claude-haiku-5-5",
    max_tokens=1000,
    messages=[{"role": "user", "content": "Summarise: ..."}],
)
msg = raw.parse()
rid = raw.headers.get("x-apimodels-request-id")
print("charged now:", raw.headers.get("x-apimodels-cost"))

rec = requests.get(
    f"https://api.apimodels.app/v1/records/{rid}",
    headers={"Authorization": f"Bearer {os.environ['APIMODELS_API_KEY']}"},
).json()["data"]
print(rec["settled"], rec["credits"], rec["currency"], rec["usage"])

If the cost does not match your own request_cost, compare rec["usage"] with what you assumed: thinking tokens in the output and the cache split are the usual differences.

Common errors

tool_choice: type "tool" and "any" are not supported for this model. Sonnet 5.5 does not allow forced tool use. Use auto with strict: true, and say in the prompt when the tool applies.
400 on thinking: {"type": "disabled"} (Sonnet 5.5). Send {"type": "between_tools"} instead, at high effort or below.
400 on temperature, top_p or top_k on the native endpoint. Remove them; steer with the prompt.
400 on a trailing assistant message (Haiku 5.5). Prefill is gone; end with a user turn and use structured outputs for format.
stop_reason: "max_tokens" with no text. Thinking used the whole limit. Raise max_tokens or lower effort.
IndexError or empty string from content[0].text. The first block is a thinking block. Filter by type.
prompt is too long: ~3,062,816 input tokens (estimated) exceeds the 1,000,000-token context window for claude-haiku-5-5. The gateway rejects prompts it estimates are beyond the 1M window before sending them. Split the input first.
Streaming UI goes silent between tool calls (Sonnet 5.5). Notes between tool calls arrive as thinking blocks whose text is empty by default. Set thinking.display or use between_tools.

Where apimodels.app fits

apimodels.app is a multi-model API gateway: one API key, a native Anthropic Messages endpoint and an OpenAI-compatible endpoint for more than 140 image, video, audio and language models. Sonnet 5.5 and Haiku 5.5 each have more than one upstream channel with automatic failover, failed requests are not charged, and every call can be reconciled with GET /v1/records/{id} as in the last step. Prices as of 17 October 2026 are $1.20 / $6.00 for Sonnet 5.5 (60% of list) and $0.09 / $0.45 for Haiku 5.5 (90% of list). The records API docs show the full response.

Skip it if you need Anthropic's Message Batches API (there is no batch endpoint), or if you need a contract or data processing agreement with Anthropic itself.

Limits

The code is written to Anthropic's documented request shapes and was not executed for this post.
Prices and behaviour were checked on 17 October 2026. Re-check Anthropic's pricing page and the gateway's model pages before you plan a quarter's spend on these numbers.

Which of your calls would you move to Haiku 5.5 first, and how are you keeping prompts under the 100K line?

Written with AI assistance from Anthropic's documentation and apimodels.app's published prices, then checked against both before publishing.

How to Generate MiniMax H3 RH Enhanced Videos via API: Text, Frames and Multi-Reference, With Polling or a Webhook

Super Lewis — Sat, 17 Oct 2026 15:08:18 +0000

A four-second 1080P clip from MiniMax H3 RH Enhanced took 218 seconds in our test. If your code holds an HTTP request open until the video exists, it will time out long before the video does. And because one model id covers three input modes, mixing the wrong fields gets you a 400 before anything renders.

This tutorial shows the full loop for MiniMax H3 RH Enhanced: create a task, wait for it with polling or a webhook, download the MP4, verify it, and confirm what you were billed. MiniMax H3 RH Enhanced is an enhanced build of MiniMax's open-weight H3 video model that generates 4-to-15-second clips with an audio track at 480P, 768P or 1080P, from text, from a first and/or last frame, or from up to 9 reference images and 3 reference videos, optionally driven by your own audio.

Disclosure: I work on apimodels.app, the API used in every example below, and the timings come from its test logs. I wrote this with AI assistance (DEV's "AI-Assisted" tier): the structure and prose were drafted with an AI, the request fields and error messages were checked against the API source, and the code is example code written against the documented contract.

What you need

Python 3.10+ and requests (pip install requests); Flask only if you want the webhook in Step 4.
curl and, for verification, ffprobe from FFmpeg.
An API key in an environment variable: export APIMODELS_KEY="sk_your_key_here".
Media as public HTTPS URLs. Images can also be sent as base64.
For webhooks: a publicly reachable HTTPS URL. A tunnel such as ngrok works for local development; localhost and private addresses are rejected.

Why one model id behaves like three

The mode is picked from the fields you send, in this order:

Any of reference_image_urls, reference_video_urls or reference_video_audio_urls → multi-reference.
Otherwise, first_frame_url, last_frame_url or reference_audio_urls (voice references) → image-to-video.
Otherwise → text-to-video, and prompt is required.

Frames and references live in different pipelines, so a request with both is rejected at create time rather than having half its inputs silently ignored. Every mode also accepts audio_url (a driving track for lip sync) and audio_mode.

The fields you will use most:

Field	Required	Values
`model`	yes	`minimax-h3-rh-enhanced`
`prompt`	text-to-video	up to 20,000 characters
`duration`	yes	4–15 (seconds; billed for this number)
`resolution`	no	`480p`, `768p`, `1080p` (default `1080p`)
`aspect_ratio`	no	`1:1` `2:3` `3:2` `3:4` `4:3` `9:16` `16:9` `21:9`
`first_frame_url`, `last_frame_url`	no	URL or base64
`reference_image_urls`	no	up to 9
`reference_video_urls`	no	up to 3
`reference_video_audio_urls`	no	up to 2 (audio of the reference videos)
`audio_url`	no	driving audio for lip sync
`reference_audio_urls`	no	up to 3 voice references
`audio_mode`	no	`native` (default), `lock_source`, `remix_source`, `reference_only`
`callback_url`	no	public URL we POST the result to

Step 1: Create a text-to-video task

Start at 480P and 4 seconds while you test; it is the fastest and smallest combination.

curl -s https://api.apimodels.app/v1/video/generations \
  -H "Authorization: Bearer $APIMODELS_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "minimax-h3-rh-enhanced",
    "prompt": "A potter shapes a clay bowl on a spinning wheel, she looks up and smiles at the camera, warm window light",
    "duration": 4,
    "resolution": "480p",
    "aspect_ratio": "16:9"
  }'

The response comes back immediately with a task ID and state: "pending":

{"code":200,"msg":"success","data":{"taskId":"<task_id>","state":"pending","model":"minimax-h3-rh-enhanced"}}

Step 2: Wait by polling

The status endpoint returns state as pending, processing, completed or failed. On completed you get resultUrls; on failed you get failMsg and a retryable flag. Poll every 10 to 15 seconds.

# h3e.py — example client
import json, os, time, requests

BASE = "https://api.apimodels.app/v1"
HEADERS = {"Authorization": f"Bearer {os.environ['APIMODELS_KEY']}"}

def create(payload: dict) -> str:
    r = requests.post(f"{BASE}/video/generations", headers=HEADERS, json=payload, timeout=60)
    body = r.json()
    if r.status_code != 200:
        raise RuntimeError(f"{r.status_code}: {body.get('msg')}")
    return body["data"]["taskId"]

def wait(task_id: str, every: int = 15, give_up_after: int = 1800) -> dict:
    deadline = time.time() + give_up_after
    while time.time() < deadline:
        r = requests.get(f"{BASE}/video/generations", headers=HEADERS,
                         params={"task_id": task_id}, timeout=30)
        data = r.json()["data"]
        if data["state"] == "completed":
            return data
        if data["state"] == "failed":
            raise RuntimeError(f"failed: {data.get('failMsg')} (retryable={data.get('retryable')})")
        time.sleep(every)
    raise TimeoutError(f"{task_id} still running after {give_up_after}s")

Step 3: Download the result

Result URLs point to MP4 files that are deleted after 30 days, so copy them to your own storage.

def download(url: str, path: str) -> None:
    with requests.get(url, stream=True, timeout=300) as r:
        r.raise_for_status()
        with open(path, "wb") as f:
            for chunk in r.iter_content(1 << 20):
                f.write(chunk)

if __name__ == "__main__":
    task_id = create({
        "model": "minimax-h3-rh-enhanced",
        "prompt": "A potter shapes a clay bowl on a spinning wheel, she looks up and smiles at the camera",
        "duration": 4,
        "resolution": "480p",
        "aspect_ratio": "16:9",
    })
    print("task", task_id)
    data = wait(task_id)
    download(data["resultUrls"][0], f"{task_id}.mp4")
    print("saved", f"{task_id}.mp4", "billed USD:", data["credits"])

Step 4: Or let the API call you

Add callback_url to the create body and the API POSTs to it when the task finishes, successful or not. The result URLs sit inside resultJson, which is a JSON string, so parse it. Return a 2xx quickly: each delivery attempt times out after 10 seconds and is tried up to three times.

# webhook.py — example receiver (Flask)
import json
from flask import Flask, request

app = Flask(__name__)

@app.post("/apimodels/callback")
def callback():
    data = request.get_json(force=True)["data"]
    urls = json.loads(data.get("resultJson") or "{}").get("resultUrls", [])
    print(data["taskId"], data["state"], urls, data.get("credits"), data.get("failMsg"))
    # enqueue the download here; don't do slow work inside the request
    return "", 200

Keep polling as a backstop for the rare callback your server misses.

Step 5: Image-to-video with a driving audio track

Same client, different body. A first frame plus a voice line turns a still portrait into a speaking shot. In our end-to-end test this exact shape at 480P and 4 seconds finished in 50 seconds.

task_id = create({
    "model": "minimax-h3-rh-enhanced",
    "prompt": "She keeps working and talks to the camera with a warm smile",
    "first_frame_url": "https://example.com/first.jpg",
    "audio_url": "https://example.com/voice.mp3",
    "duration": 6,
    "resolution": "480p",
    "aspect_ratio": "3:4",
})

Step 6: Multi-reference with your soundtrack locked

Reference images set who and where, a reference video sets the movement, and audio_mode: "lock_source" keeps your song untouched. This is the slowest shape: a 6.6-second reference video pushed a 4-second 480P test to 346 seconds, so it is a good place to use the webhook.

task_id = create({
    "model": "minimax-h3-rh-enhanced",
    "prompt": "The singer from image 1 performs on the stage from image 2, moving like the dancer in the reference video",
    "reference_image_urls": ["https://example.com/singer.jpg", "https://example.com/stage.jpg"],
    "reference_video_urls": ["https://example.com/dance.mp4"],
    "audio_url": "https://example.com/song.mp3",
    "audio_mode": "lock_source",
    "duration": 10,
    "resolution": "768p",
    "aspect_ratio": "9:16",
    "callback_url": "https://your-app.example.com/apimodels/callback",
})

Verify it works

Check the file's streams and length:

ffprobe -v error -show_entries stream=codec_type,width,height,r_frame_rate:format=duration \
  -of default=noprint_wrappers=1 <task_id>.mp4

What our 2026-10-10 tests produced, so you know what "correct" looks like: a video stream at 24 fps plus an audio stream; 864×480 for 480P at 16:9, 544×768 for 480P at 3:4, 1920×1088 for 1080P at 16:9; and a duration about 0.46 seconds longer than requested.

Then confirm the charge for that one task:

curl -s https://api.apimodels.app/v1/records/<task_id> -H "Authorization: Bearer $APIMODELS_KEY"

data.settled is true once the amount is final and data.credits is the USD taken from your balance, for example 0.2 for 480P × 4 seconds before any account discount, and 0 for a failed task.

Where apimodels.app fits

apimodels.app is a multi-model API gateway: one API key and one OpenAI-compatible endpoint for more than 140 image, video, audio and language models. For most mainstream models it keeps more than one upstream route with automatic failover; failed requests are not charged; long jobs run asynchronously with the webhook shown above; and GET /v1/records/{id} returns the exact charge for any single request. This model does not have a failover route yet: a failed job is refunded, and resubmitting is up to you (check retryable).

Pricing for minimax-h3-rh-enhanced is per requested second: $0.05 at 480P, $0.07 at 768P and $0.10 at 1080P. Reference images, reference videos and audio inputs are free. The MiniMax H3 RH Enhanced API reference lists every field and alias.

Do not use it if you need a 1080P clip back in under a minute, need 2K or 4K, need clips longer than 15 seconds, or need a guaranteed native 1080P master: we have not verified whether 768P and 1080P are rendered natively or upscaled.

Common errors

All of these come back as HTTP 400 at create time. No task is created and nothing is charged.

duration is required (4-15 seconds): there is no default length. Send duration.
duration must be between 4 and 15 seconds (got "20"): split longer scenes into several clips.
prompt is required for text-to-video (or pass first_frame_url / reference_image_urls): an empty prompt with no frames or references.
first_frame_url / last_frame_url cannot be combined with reference_image_urls / reference_video_urls — use one mode per request: choose frames or references for each shot.
resolution must be one of 480p / 768p / 1080p (got "720p"): there is no 720p tier.
aspect_ratio must be one of 1:1 / 2:3 / 3:2 / 3:4 / 4:3 / 9:16 / 16:9 / 21:9 (got "5:4").
at most 9 reference images allowed (got 10): the same pattern applies to 3 videos, 3 voice references and 2 reference-video audio tracks.
callback_url host "localhost" is not reachable from our servers …: use a public URL or a tunnel.

One failure arrives later, through failMsg: Content did not pass the safety review. Please change the reference media or prompt and try again. It is not charged.

Limits

Resolution tops out at 1080P, and the 1080P file is 1920×1088, not 1920×1080. Crop if your pipeline is strict.
Omitting resolution renders at 1080P, the most expensive and slowest tier.
The image-to-video and multi-reference timings above come from a handful of our own tests.
We have not measured lip-sync accuracy or voice similarity.

What turnaround time does your current video pipeline assume, and would a 218-second wait for a four-second 1080P clip break it?

I build apimodels.app, one API key for 140+ image, video, audio and language models.

Nano Banana 2.1 vs Nano Banana Pro: Same Prompts, Real Latency Data, and When Pro Is Worth It

Super Lewis — Thu, 08 Oct 2026 02:59:03 +0000

Short answer: for most product images, banners and text-heavy graphics, Nano Banana 2.1 does the job at roughly half to a third of Nano Banana Pro's price and about twice the speed. Pick Pro when the scene needs more world knowledge, richer detail or more character references (5 versus 4).

Nano Banana 2.1 is Google's Flash-tier image generation and editing model (Gemini API id gemini-nano-banana-2.1, released on 6 October 2026, built on Gemini 3.6 Flash). Nano Banana Pro is the Pro tier (gemini-3-pro-image). Google's own docs call 2.1 "the more efficient counterpart to Gemini 3 Pro Image".

Disclosure up front: the production numbers below come from apimodels.app, the API platform I work on, and the test images were generated through it.

How I compared them

I fixed the criteria before looking at any output:

Price per image at 1K, 2K and 4K, from Google's pricing page (updated 7 October 2026, standard tier).
Documented capabilities: reference-image limits, thinking controls, grounding, aspect ratios.
Same-prompt output at 2K, 16:9: one text-heavy infographic, one edit from a reference image.
Real-world latency and success rate from one week of production traffic (1–8 October 2026).

At a glance

	Nano Banana 2.1	Nano Banana Pro
Gemini API id	`gemini-nano-banana-2.1`	`gemini-3-pro-image`
Google price, 1K / 2K / 4K	$0.0336 / $0.0504 / $0.113	$0.134 / $0.134 / $0.24
Object references	up to 10	up to 6
Character references	up to 4	up to 5
Style references	up to 3	not documented
Thinking	minimal / medium / high (default medium)	always on
Google Search grounding	Web and Image Search	Web Search
Extra wide ratios	1:4, 4:1, 1:8, 8:1	no
Our median time, 2K	about 25 s	about 40 s
Best for	volume, banners, infographics, many object refs	complex scenes, brand-exact work, more character refs

At 2K, Pro costs 2.7 times as much as 2.1 on Google's list; at 1K the gap is 4 times.

Test 1: a text-heavy infographic

Prompt (same for both, 2K, 16:9):

A clean flat-design infographic poster, 16:9, titled "How an image API request works". Four numbered steps left to right, each with an icon and a short label: "1. Send prompt", "2. Queue task", "3. Render image", "4. Download result". A small footer line reads "Average time: 25 seconds". White background, navy and coral accents, crisp legible typography.

Both models spelled every string correctly: the title, the four labels and the footer. Pro drew richer icons inside numbered circles; 2.1 went flatter and turned the footer into a navy bar. 2.1 finished in 25 seconds, Pro in 57. For text-heavy graphics at this size, I could not tell you which one is "better" without a style preference.

Test 2: editing with a reference image

The reference was a 1280×720 title card from our own site (that is why the brand name appears on the screen). Prompt:

Use the reference image as the screen content. Show it playing on a large modern TV mounted on a living-room wall at night, warm lamp light, a plant on the left, a soft reflection on the floor. Keep the on-screen poster recognisable.

Scored against the prompt:

Wall-mounted TV: 2.1 yes; Pro placed it on a cabinet.
Plant on the left, warm lamp: both.
Reflection on the floor: Pro yes; 2.1 put it on a foreground table (with the letters correctly mirrored).
On-screen text kept exact: both.

Each model missed one instruction. 2.1 took 28 seconds, Pro 49.

A week of production timings

Two prompts prove nothing about reliability, so here is what real traffic looked like on apimodels.app from 1 to 8 October 2026:

Model and job	Requests	Median	90th percentile
Pro, 4K with references	617	61 s	252 s
Pro, 2K with references	203	40 s	76 s
Pro, 2K text only	83	38 s	143 s
2.1, 4K with references	16	45 s	50 s
2.1, 2K (all)	6	about 25 s	35 s
2.1, 1K text only	2	13 s	15 s

Success rates over the same week: Pro 98.9% (971 requests; all 11 failures were content-moderation refusals), 2.1 97.6% (41 requests since it went live on 7 October; one upstream failure). The 2.1 sample is small, so treat its percentiles as a first look. The striking number is Pro's 4K tail: one request in ten took over four minutes.

Reproduce it

Node 18+ (uses the built-in fetch). The same script runs either model; swap the id.

// compare.mjs — APIMODELS_API_KEY in your environment
const BASE = 'https://api.apimodels.app/v1'
const H = { Authorization: `Bearer ${process.env.APIMODELS_API_KEY}`, 'Content-Type': 'application/json' }
const prompt = 'A clean flat-design infographic poster, 16:9, titled "How an image API request works" ...'

async function run(model) {
  const t0 = Date.now()
  const created = await fetch(`${BASE}/images/generations`, {
    method: 'POST', headers: H,
    body: JSON.stringify({ model, prompt, aspect_ratio: '16:9', resolution: '2K' }),
  }).then(r => r.json())
  const id = created.data.taskId
  for (;;) {
    await new Promise(r => setTimeout(r, 4000))
    const task = await fetch(`${BASE}/images/generations?task_id=${id}`, { headers: H }).then(r => r.json())
    if (!['pending', 'processing'].includes(task.data.state)) {
      return { model, state: task.data.state, seconds: Math.round((Date.now() - t0) / 1000), urls: task.data.resultUrls }
    }
  }
}

console.log(await Promise.all([run('nano-banana-2-1'), run('gemini-3-pro-image')]))

To edit instead of generating, add image_urls: ['https://…/your-reference.jpg'] to the body (2.1 accepts up to 10 references here).

Which one should you pick

High-volume banners, thumbnails, product shots → 2.1. About half the time per image and a third to a half of the price.
Infographics and labelled diagrams at 2K or 4K → 2.1 first. It matched Pro on spelling in my test, and Google's model card reports infographic factuality of 0.521 for 2.1 versus 0.179 for Nano Banana 2.
Very wide banners (4:1, 8:1) → 2.1. Pro does not offer those ratios.
Scenes that lean on world knowledge, exact brand assets or five recurring characters → Pro. That is where Google positions it, and its extra detail showed in the icons.
Small text at 1K or long paragraphs → neither. Google lists blurry small text at 1K as a known 2.1 limitation; render text at 2K or add it in post.

Where the numbers came from, and when not to use apimodels.app

apimodels.app is a multi-model API gateway: one API key and OpenAI- and Anthropic-compatible endpoints for about 150 image, video, audio and language models. Over the week above, Pro and 2.1 ran there at 98.9% and 97.6% success, failed calls are not charged, and the per-image prices are $0.024 / $0.04 / $0.064 for 2.1 and $0.08 / $0.08 / $0.13 for Pro, below Google's standard list.

Do not route through it if you need 2.1's thinking-level control, Google Search grounding or all 14 reference images: our endpoint does not expose those yet, and Google's API does. The same applies if you only use Gemini and can wait for Google's batch tier, which is half the standard price. Model pages: Nano Banana 2.1 and Nano Banana Pro.

Limits of this comparison

Two prompts, one run each, at 2K only. Image models vary run to run, so a single miss (Pro's cabinet, 2.1's table reflection) is an anecdote, not a rate. The latency data is real traffic but skewed toward 4K jobs with references for Pro.

Which job would you trust to 2.1 and which would you still send to Pro? I'm curious where others draw the line.

This post was written with AI assistance from the test data above and reviewed before publishing. The comparison images are AI-generated by the two models being compared.

How to Render a Motion Graphics Video with Claude Opus 5.5: Prompt, Frame Renderer, ffmpeg and Real Cost

Super Lewis — Wed, 07 Oct 2026 05:39:15 +0000

Claude Opus 5.5 cannot output a single pixel, yet people keep posting motion-graphics videos "made by Opus 5.5". The trick is that the model writes a program that draws every frame, and you render that program into an MP4. This post is the full, reproducible pipeline we used for a 12-second 1280x720 promo: the exact prompt, the API call, a 25-line frame renderer, the ffmpeg command, and what it actually cost.

Disclosure up front: we ran this through apimodels.app, the API gateway I work on. apimodels.app is a multi-model API gateway: one API key and an OpenAI-compatible endpoint for about 150 image, video, audio and language models, including claude-opus-5-5. Every step below works the same against any OpenAI-compatible endpoint that serves Opus 5.5; only the base URL changes.

The idea in one sentence

Ask Opus 5.5 for an HTML page with a deterministic renderFrame(t) function, not for "a video", then let a headless browser call that function once per frame.

Opus 5.5 outputs text. A page that animates itself with requestAnimationFrame looks fine in a browser, but you cannot export it frame-accurately, because what you see depends on timing. A page whose every frame is a pure function of t can be rendered frame 0, frame 1 … frame 359 in any order, and each call returns the same picture. That is exactly what a video encoder needs.

What you need

An API key for an endpoint that serves claude-opus-5-5 (OpenAI chat-completions format).
Node.js 20+ and puppeteer-core (npm i puppeteer-core works; we use the Chrome already installed on the machine).
ffmpeg 6+.
About seven minutes of patience for the model call. More on that below.

Step 1: write the prompt as a frame contract

The rules in the first half of this prompt are what make the output renderable. The story comes second, the style last. This is the prompt we sent, unedited:

You are writing a program that renders a video. Output one complete, self-contained HTML file and nothing else.

Rules for the program (these make it renderable frame by frame):
1. A single <canvas> of exactly 1280x720, no CSS scaling, black page background.
2. Expose a global function window.renderFrame(t) that draws the frame at time t (seconds, 0 <= t < 12) from scratch. Everything on screen must be a pure function of t: no Date.now, no performance.now, no Math.random (use a seeded PRNG if you need noise), no accumulated state between calls, no requestAnimationFrame loop.
3. Also expose window.DURATION = 12 and window.FPS = 30. When the page is opened normally, play it once in a loop with requestAnimationFrame by calling renderFrame, so a human can preview it.
4. No external assets, fonts, images or libraries. Use system-ui for text.
5. The rhythm is 120 BPM: a beat every 0.5 s. Put every major cut, text entrance and impact on a beat time (a multiple of 0.5). List the beat plan as a comment at the top of the script.

The story (tell this, then decide the visuals):
A 12-second promo for "APIMODELS", an API gateway. Problem, then turn, then payoff:
- 0-3 s: a developer's screen is cluttered with many different API keys and SDK logos flying in from every side, overlapping, getting chaotic (draw them as simple rounded labels such as "image", "video", "LLM", "audio", "key_1", "key_2", "sdk", not real company logos).
- 3-4.5 s: everything snaps together on a beat and collapses into one glowing key.
- 4.5-9 s: from that one key, lines fan out to a grid of model cards that light up one per beat (label them "145 models", "image", "video", "LLM", "audio", "one endpoint").
- 9-12 s: clean end card: the word APIMODELS, the line "One key. Every model.", and "apimodels.app" small underneath; hold still for the last second.

Style: dark background, one accent colour (electric violet #7c5cff) plus white, motion with easing (easeOutCubic / easeInOutQuad written by you), subtle depth (scale and blur fall-off), no clutter in the final 3 seconds.

Before the code, do not explain. Return only the HTML.

Three details mattered more than the wording:

Ban every source of non-determinism by name. Date.now, performance.now, Math.random and accumulated state are the usual culprits. The model followed all four.
Fix the rhythm in numbers. "120 BPM, cuts on multiples of 0.5 s, write the beat plan as a comment" produced a 24-beat plan, and every cut landed on one.
Say what must not appear. We asked for plain rounded labels instead of real company logos, which keeps the result usable.

Step 2: call Opus 5.5 with streaming and no small max_tokens

curl https://api.apimodels.app/v1/chat/completions \
  -H "Authorization: Bearer $APIMODELS_API_KEY" \
  -H "Content-Type: application/json" \
  -d "$(jq -n --rawfile p prompt.md \
        '{model:"claude-opus-5-5", stream:true, messages:[{role:"user", content:$p}]}')" \
  --no-buffer > stream.txt

Two things will bite you here if you skip them:

Opus 5.5 thinks before it writes, and thinking counts as output. On this prompt the first character of the answer arrived after 110 seconds, and the full reply took 6 minutes 37 seconds. Stream the request so the connection stays alive. A client with a 60-second timeout will give up long before the first token.
Do not send a small max_tokens. The reply used 31,786 output tokens for about 16,000 characters of HTML, most of it thinking. A max_tokens: 4096 cap cuts the file off mid-script. If you omit the field, apimodels.app uses the model's own ceiling instead of inventing a small default.

Join the delta.content pieces from stream.txt and save the result as promo.html. Open it in a browser first: it should play its own preview loop.

Step 3: render every frame with headless Chrome

// render.mjs: pnpm add puppeteer-core (or npm i puppeteer-core)
import puppeteer from 'puppeteer-core'
import { mkdirSync, writeFileSync } from 'node:fs'

mkdirSync('frames', { recursive: true })
const browser = await puppeteer.launch({
  executablePath: '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
})
const page = await browser.newPage()
await page.setViewport({ width: 1280, height: 720 })
await page.goto('file://' + process.cwd() + '/promo.html')
await page.evaluate(() => { window.requestAnimationFrame = () => 0 }) // stop the preview loop
const { DURATION, FPS } = await page.evaluate(() => ({ DURATION, FPS }))
for (let i = 0; i < DURATION * FPS; i++) {
  const png = await page.evaluate((t) => {
    window.renderFrame(t)
    return document.querySelector('canvas').toDataURL('image/png')
  }, i / FPS)
  writeFileSync(`frames/f${String(i).padStart(4, '0')}.png`, Buffer.from(png.split(',')[1], 'base64'))
}
await browser.close()

On Linux, point executablePath at your Chromium binary. 360 frames took 14 seconds on a laptop, and the page threw no errors.

Step 4: encode with ffmpeg

ffmpeg -framerate 30 -i frames/f%04d.png -c:v libx264 -pix_fmt yuv420p -crf 20 -movflags +faststart promo.mp4
# optional soundtrack, keep the BPM you gave the model:
# ffmpeg -i promo.mp4 -i beat.mp3 -c:v copy -shortest promo-with-audio.mp4

yuv420p and faststart are what make the file play everywhere, including in browsers and on phones. The result was a 1.8 MB, 12-second, 30 fps MP4.

Step 5: check frames at the story beats

Before judging the video, pull frames at the moments the brief describes and put them side by side: 2.3 s (the clutter), 3.5 s (one key), 7.2 s (cards lighting up) and 11.3 s (end card).

All four matched on the first attempt. To change something, ask by timecode and beat ("at 7.5 s the sixth card should light up with a pulse") and ask for the whole file back, so the frame contract stays intact.

What it cost

Item	Value
Model	`claude-opus-5-5`
First token / total time	110 s / 6 min 37 s
Output tokens	31,786 (mostly thinking)
Cost at Anthropic list price ($4 / $20 per 1M tokens)	about $0.64
Cost at apimodels.app list price ($2.40 / $12 per 1M tokens)	about $0.38
Rendering and encoding	local, free

Output tokens dominate. A clip that needs 30,000 to 80,000 tokens of thinking and code costs roughly $0.36 to $0.96 per attempt at the lower rate, so budget for two or three attempts per finished clip.

When this approach is the wrong tool

Code-rendered video is good at kinetic typography, UI walkthroughs, animated charts, logo reveals, explainers and social templates you can re-render with new text in seconds. It is the wrong tool for anything that has to look filmed: photoreal people, natural camera motion, skin, fabric, physics. For those shots use a text-to-video model and let Opus 5.5 write the shot list and prompts instead.

It also has no audio (add it with ffmpeg), and the model never runs its own page. Treat the HTML as untested code: render it and look at the frames before you trust it.

Where to go from here

The model page for Claude Opus 5.5 on apimodels.app has the API parameters and this run's files.
We collected 60+ Opus 5.5 video prompts with the original posts and methods, sorted into motion design, product promos, explainers, music videos and 3D.

What would you render first with a renderFrame(t) contract: a chart, a logo reveal, or a UI walkthrough? I'm curious which kind of clip breaks the approach.

This post was written from our own run's code and numbers; AI helped with structure and editing.

Photoshop's Pen Tool Is Done: How AI Splits a Flat Image into Layers

Super Lewis — Tue, 06 Oct 2026 13:27:39 +0000

Originally published on the LayerGrab blog. Disclosure: I build LayerGrab, an AI tool that splits images into layers; this post was drafted with AI assistance and edited by me.

Separating a flat image into layers used to be the one job you could not hand off: Pen tool, Layer via Copy, Content-Aware Fill, repeat for every element. Image layer decomposition is the AI task that does it in one pass: a model takes a flat RGB image and returns a stack of RGBA layers, one per element, with the regions behind each element filled in so the layers recompose into the original.

Here is how it works, what we measured on a real poster today, and where it breaks.

What the model has to do

Three things at once, which is why this was hard until recently:

Decide what the elements are. Not pixels by colour, but "headline", "badge", "cup".
Cut each one out as RGBA with a usable alpha edge.
Invent what was behind each element, because the camera or designer never stored it. Without this, moving a layer leaves a hole.

Segmentation models solve step 2 and inpainting models solve step 3. A decomposition model does all three, which is what makes the output a real layered file instead of cutouts on a broken background.

Running the open model: what we measured

Qwen-Image-Layered (Apache 2.0, weights released December 2025) is the open model for this. We ran it through fal's hosted API on an 880 × 1184 coffee shop poster on 2026-10-06. Example request, from fal's public docs:

# Example (fal queue API). FAL_KEY is your own key.
curl -X POST https://queue.fal.run/fal-ai/qwen-image-layered \
  -H "Authorization: Key $FAL_KEY" \
  -H "Content-Type: application/json" \
  -d '{"image_url": "https://example.com/poster.png", "num_layers": 4, "output_format": "png"}'

What came back with num_layers: 4:

4 PNGs, each 544 × 736, the full canvas size: the model works at around 640 px on the long side, and its README recommends that resolution.
Layer 1 was the fully opaque background; layers 2–4 were transparent everywhere except their element.
Text, the discount badge and three small leaves shared one layer; the cup got its own; one large leaf got its own.
Stacked back together, the layers matched the downscaled original with a 1.5% mean pixel difference.
18.6 s of inference, 65 s end to end including the queue.

With num_layers: 2, the cup and headline stayed in the background layer, and the second layer held only the small decorations.

Pitfalls that decide whether the layers are usable

You choose the layer count, and it groups by depth, not by object. Too few layers and the main subject stays in the background. Start at 4–6.
Resolution drops. Expect ~640 px output. Upscale afterwards if you need print size.
VRAM. Users in the model's GitHub issues report failures on 16 GB cards and trouble on 24 GB; ComfyUI's guide measured a 45 GB peak at 1024 px.
Text is pixels. You get the headline on its own layer, not as editable type. Hide it and retype.
Hosted APIs bill per output image. fal charges per image returned, so 4 layers cost 4×.

When you don't want to run it yourself

LayerGrab is an AI tool that splits one image into up to 16 separate transparent layers, one per element, names each layer from what it shows, fills the background behind them, and exports PNG, ZIP or a layered PSD. On the same poster it returned 10 named layers in 100 seconds at the original resolution. It also runs Qwen-Image-Layered itself, with a free 2-layer try, if you want to compare: layergrab.com/qwen-image-layered-free.

Don't use it if your images must stay on your own machine: run the open model locally instead.

Is Photoshop dead?

As a finishing tool, no: we even ship a Photoshop plugin. As the place you go to cut a flat image apart by hand, yes. That job is now a model call.

Have you tried any of these models on real client work? I'm curious where the output broke for you.

Claude Opus 5.5 vs GPT-6.1 Sol: which one to call, and how to measure it on your own prompts

Super Lewis — Thu, 01 Oct 2026 14:51:37 +0000

Claude Opus 5.5 and GPT-6.1 Sol are the two models most teams are choosing between this week. Opus 5.5 is Anthropic's everyday flagship, released on 22 September 2026. GPT-6.1 Sol is OpenAI's upgraded mid-tier model, released at DevDay on 29 September 2026. The short answer: pick GPT-6.1 Sol for high-volume and cost-sensitive work, and Claude Opus 5.5 for long agentic coding and tasks where a wrong answer costs more than the tokens. The longer answer depends on reasoning effort, and you can check it on your own prompts in about ten minutes.

Disclosure up front: I work on apimodels.app, a multi-model API gateway that serves both models. Prices for it below are its public prices on 1 October 2026. This post was drafted with AI assistance and checked against the vendors' pages.

How I compared them

List price per 1M tokens, from the vendors' pages on 1 October 2026.
Specs that change your code: context window, max output, reasoning controls.
Benchmarks both models were run on in the same table. The labs mostly publish different suites, so I only use rows where both appear, and I name who ran them.
Cost per call for a realistic agent turn, computed from the prices above.

At a glance

	Claude Opus 5.5	GPT-6.1 Sol
Released	22 Sep 2026 (Anthropic)	29 Sep 2026 (OpenAI DevDay)
List price, input / output	$4 / $20	$2 / $10
Cached input (list)	$0.20 read, $5 write	$0.10 read, $2.50 write
Context / max output	1M / 128K	1.05M / 128K
Knowledge cutoff	June 2026	30 April 2026
Reasoning control	effort low to max, default medium, always thinks	reasoning_effort low to max, default medium, no `none`
Long context surcharge	—	above 272K input: input and cache 2×, output 1.5×
Best for	long agent sessions, review, careful writing	volume, extraction, cost-capped agents
Not for	high-volume bulk classification	code that relies on `reasoning_effort: none`

The benchmarks they share

OpenAI's GPT-6.1 Sol launch included Opus 5.5 in several rows. These are OpenAI's runs and OpenAI's choice of benchmarks, so read them as a vendor's claim:

Benchmark (OpenAI launch)	Effort	GPT-6.1 Sol	Claude Opus 5.5
AutomationBench	medium	31.7%	29.5%
AutomationBench	max	36.1% ($0.30/task)	42.5% ($1.44/task)
GDP.pdf	medium	30.0% ($0.34/task)	25.6% ($0.80/task)
Terminal-Bench Science 0.1	max	57.0% ($5.47/task)	63.3% ($23.21/task)

The pattern is consistent: at medium effort GPT-6.1 Sol edges ahead, at max effort Opus 5.5 pulls ahead, and Sol costs less per task in every row. Independent aggregates point the same way on capability. Reported max-effort figures put Opus 5.5 at 58 and GPT-6.1 Sol at 52 on the Artificial Analysis Intelligence Index, with cost per index task of $5.98 and $0.72.

Anthropic's own launch numbers for Opus 5.5 (66.4% on Terminal-Bench 4.0, 57.8% on CursorBench 4.0) have no GPT-6.1 Sol row, so they don't settle the head-to-head.

What one agent turn costs

Take a coding-agent turn that re-sends 50,000 input tokens, 40,000 of them cached, and returns 2,000 output tokens:

	Claude Opus 5.5	GPT-6.1 Sol
At list price	$0.088	$0.044
On apimodels.app	$0.053	$0.022

Two things move these numbers in practice. Opus 5.5 always thinks, so its reasoning tokens land in the output bill even on short answers. GPT-6.1 Sol at low effort spends far fewer reasoning tokens, which is where most of its cost-per-task advantage comes from.

Measure it on your own prompts

Benchmarks are someone else's tasks. The fastest way to decide is to send the same twenty prompts to both models and compare answers and cost. Both are OpenAI-compatible on apimodels.app, so one client covers both, and every response carries the billed amount in an x-apimodels-cost header.

Tested with Python 3.11 and openai 1.x:

import os, time
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["APIMODELS_API_KEY"],
    base_url="https://api.apimodels.app/v1",
)

MODELS = ["claude-opus-5-5", "gpt-6.1-sol"]

def ask(model: str, prompt: str) -> dict:
    t0 = time.time()
    raw = client.chat.completions.with_raw_response.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
        max_tokens=8000,
    )
    r = raw.parse()
    return {
        "model": model,
        "seconds": round(time.time() - t0, 1),
        "in": r.usage.prompt_tokens,
        "out": r.usage.completion_tokens,
        "billed_usd": raw.headers.get("x-apimodels-cost"),  # what you were actually charged
        "answer": r.choices[0].message.content,
    }

prompts = [line for line in open("prompts.txt") if line.strip()]
for p in prompts:
    for m in MODELS:
        res = ask(m, p)
        print(f'{res["model"]:16} {res["seconds"]:>6}s  in={res["in"]:>6} out={res["out"]:>6}  ${res["billed_usd"]}')

Score the answers yourself, or with a third model as judge, then divide the total billed cost by the number of answers you would ship. That number is the one that matters.

Gotchas when switching

reasoning_effort: "none" is not on GPT-6.1 Sol's list. OpenAI's model page lists low, medium, high, xhigh and max. Code written for GPT-6 Sol that sends none should send low.
Small max_tokens can return an empty answer. Reasoning counts against the budget on both models. If finish_reason is length and the content is empty, raise max_tokens or lower the effort.
Long prompts change Sol's price. Above 272K input tokens the whole request is billed at 2× input and 1.5× output.
Opus 5.5 can't be made to cost less by turning thinking off. If a task is simple enough that thinking is waste, it is probably a Sol task.

Which one should you pick

Classification, extraction, routing, bulk summarisation → GPT-6.1 Sol at low effort.
Agents with a hard cost cap per task → GPT-6.1 Sol at medium effort.
Long agentic coding sessions, code review, careful long-form writing → Claude Opus 5.5.
The hardest reasoning tasks → run both at max effort on twenty real examples before you commit.
You need a direct contract with OpenAI or Anthropic, such as zero data retention or a regulated-industry agreement → call the vendor directly, not a gateway.

About apimodels.app

apimodels.app is a multi-model API gateway: one key and one OpenAI-compatible endpoint for 140+ image, video, audio and language models. When an upstream channel fails, requests switch to another channel automatically, failed calls are not charged, and long-running jobs can report back through async callbacks. On price, on 1 October 2026 it lists Claude Opus 5.5 at $2.40 / $12 per 1M tokens, 40% below Anthropic's list, and GPT-6.1 Sol at $1 / $5, half of OpenAI's list, billed per token. It is not the right choice if you only ever call one model and already have the vendor's SDK and contract in place.

Which prompts in your own workload flip between the two models? I'd like to hear where your results disagree with the launch tables.

We Replaced a 2,300-Character Image Prompt With One Sentence

Super Lewis — Sun, 27 Sep 2026 02:02:56 +0000

Our prompt for making AI images look real used to be 2,300 characters long. It listed every AI tell to remove, every camera detail to add, and rules for colour, shadows and lens optics. It produced faces covered in freckles and skin so drained it looked like a corpse. The version in production today is one sentence.

The lesson generalises to any image-editing prompt: a modern image model already knows what a phone photo looks like, so naming a familiar camera does the work of a page of instructions, and every extra clause is something the model can over-apply.

sparkpix.ai's AI Image Humanizer is a web tool that re-renders an AI-generated image so it reads like a real phone-camera photo, keeping pose and framing. Disclosure: I build sparkpix.ai. This post was drafted with AI assistance for structure and wording; the prompt text, comments and code are from the sparkpix codebase (Next.js App Router on Vercel, calling GPT Image 2 through apimodels.app, September 2026).

Long prompts fail by arguing with themselves

Every clause in the long prompt was added to fix a problem the previous clause created. "Add natural skin texture" produced freckles; "don't add freckles" produced flat skin; "keep healthy colour" fought "remove the saturated AI grade". The model obeyed every clause a little too hard, and the clauses pulled against each other.

What replaced it, tested directly against GPT Image 2 and preferred by eye on 2026-07-30:

export const DE_AI_PROMPT =
  'Remove the AI-generated look from this image so it looks natural — for a ' +
  'photographic image, like it was taken with an iPhone 12 camera. Keep the ' +
  'original art style: an illustration or anime image must stay an ' +
  'illustration, not become a photograph.'

"Like it was taken with an iPhone 12 camera" carries the sensor noise, the colour science, the skin rendering and the slightly imperfect light, all at once, because the model has seen millions of those photos. There is no clause left to over-apply.

One word can change the subject instead of the camera

We tried "selfie" in that sentence, and it re-posed people. "Selfie" does not only describe a camera; it describes a photo genre: arm raised, head tilted, shot close and from above. The model read it as an instruction about the person and changed the pose the user had already chosen.

The rule we took from it: name the device, never the genre. "iPhone 12 camera" is about the lens. "Selfie", "portrait shot", "street photo" are about the picture, and the model will happily give you a new picture.

Scope a style instruction, don't negate it

The unscoped sentence, "make it look like an iPhone photo", is also an instruction to convert the medium, and anime uploads started coming back as photographs. The model was doing exactly what it was told.

The tempting fix is to append "do not change the art style". That recreates the original failure: two clauses arguing. Instead, the camera reference is scoped to photographic input ("for a photographic image, like it was taken with…"), and the second sentence adds a constraint rather than a contradiction.

Honest status: the scoped version has not been through the same side-by-side test as the one-sentence version. It is in production because it fixed the anime-to-photo reports, not because it was benchmarked.

Keep the prompt in exactly one place

The first time we shortened the prompt, nothing changed for real users. The wording lived in three places: two tool defaults on the server, and a pre-filled textarea on each landing page. The landing-page copy is what actually reached the model, because the editor sends its textarea as prompt and the server only falls back to its default when prompt is empty. We had edited the fallback.

Now there is one exported constant, imported by both the tool config and the pages, and the old long prompt is not kept as a "fallback", because an unused second copy is how the drift started. It is in git history if anyone needs it.

Failures must refund exactly once

A humanizer call can fail after credits are taken: the model's safety filter refuses the image, or the upstream API errors. The refund has to happen once, even though a failure can be observed by several code paths (the background job, a client poll, a cleanup cron). The pattern is to let the database decide who gets to refund:

const failAndRefund = async (errorMessage: string) => {
  const result = await query(
    `UPDATE generations SET status = 'failed', error_message = $1
     WHERE id = $2 AND status = 'processing'`,
    [errorMessage.substring(0, 500), generationId],
  )
  if (result.rowCount && result.rowCount > 0) {
    await addCredits(userId, creditsCharged)
  }
}

Only the caller whose UPDATE actually flips the row from processing to failed sees rowCount > 0, so only that caller refunds. A SELECT followed by an UPDATE looks equivalent and is not: two pollers can both read processing before either writes. We learned that the expensive way, with a polling endpoint that refunded the same failure on every poll.

Where this approach falls short

A one-sentence prompt gives the user no dial. If you need to control grain strength or skin detail per image, a slider-based tool is the better fit. A generative re-render can also shift very fine details, and GPT Image 2's safety filter refused about 1 in 20 of these requests in early September 2026, which is why the refund path matters.

FAQ

Why an iPhone 12 and not the latest model? Nothing magic; it is a very common, well-represented camera. Users do edit it: we see iPhone 13 to 17 in their prompts.

Does naming a camera work with other models? We only tested GPT Image 2. The idea, naming a familiar device instead of describing its output, should transfer, but test it.

Why not just add grain in post? Grain fixes the surface only. Plastic lighting, colour grades and over-smooth skin are structural, and a texture overlay cannot touch them.

The tool is at sparkpix.ai/ai-image-humanizer, 5 credits per image with the first image free. For how it compares with nine other tools, see Best AI Image Humanizers in 2026.

How to Read C2PA and IPTC AI Labels From Image Bytes in Node.js

Super Lewis — Sat, 26 Sep 2026 14:42:19 +0000

Most AI images carry a note from the tool that made them, and you can read it with about 60 lines of TypeScript. The hard part is not parsing it; it is knowing what its absence means, which is nothing at all.

sparkpix.ai's AI Image Detector is a web tool that estimates whether an image was AI-generated by reading provenance labels in the file first and then scoring the pixels. This post walks through how the first layer works in Node.js, how the two layers combine into a verdict, and the one upload bug that silently deletes the evidence.

Disclosure: I build sparkpix.ai and apimodels.app, and both appear below. This post was drafted with AI assistance for structure and wording; the code is from the sparkpix codebase (Next.js App Router, Node 24, September 2026).

Two kinds of label live inside AI images

AI tools write provenance into the file in two standard ways, and a detector should look for both.

C2PA Content Credentials. A signed manifest, defined by the Coalition for Content Provenance and Authenticity, that records which tool produced the file and what was done to it. OpenAI, Adobe Firefly and Microsoft embed one. It sits in a JUMBF box: JPEG APP11 segments, a caBX chunk in PNG, a C2PA chunk in WebP.
IPTC Digital Source Type. An XMP field whose value trainedAlgorithmicMedia means "created by a generative model" and compositeWithTrainedAlgorithmicMedia means "edited with one". Google writes it into Gemini and Imagen images, and C2PA action assertions reuse the same vocabulary.

When either is present, the question is settled: the maker declared it.

A byte scan is enough to detect presence

You do not need a full C2PA validator to answer "does this file claim to be AI?". We deliberately scan bytes and report presence plus the claimed generator, not signature validity:

export function readMetadataSignals(buf: Buffer): MetadataSignals {
  const latin = buf.toString('latin1')
  const c2pa =
    (latin.includes('jumb') && latin.includes('c2pa')) ||
    latin.includes('caBX') ||
    /C2PA[\s\S]{0,8}jumb/.test(latin)
  const composite = latin.includes('compositeWithTrainedAlgorithmicMedia')
  const generated = !composite && latin.includes('trainedAlgorithmicMedia')
  let generator = c2pa ? claimGenerator(buf) : null
  if (!generator && /Made with Google AI/i.test(latin)) generator = 'Google AI'
  return { c2pa, generator, aiSource: composite ? 'composite' : generated ? 'generated' : null }
}

Note the order of the two IPTC checks: compositeWithTrainedAlgorithmicMedia contains trainedAlgorithmicMedia as a substring, so the composite test has to run first or every AI-edited photo reads as fully generated.

The generator name comes from the C2PA claim, which is CBOR. Newer (v2) claims store it as claim_generator_info: [{ name: "ChatGPT" }]; older (v1) claims use a single claim_generator string such as Adobe_Photoshop/25.0 adobe_c2pa/0.7.6. Reading a CBOR text string by hand is a few lines, because the first byte tells you the length:

function readCborText(buf: Buffer, at: number): string | null {
  const b = buf[at]
  let len = -1
  let start = at + 1
  if (b >= 0x60 && b <= 0x77) len = b - 0x60      // length in the type byte
  else if (b === 0x78) { len = buf[at + 1]; start = at + 2 }            // 1-byte length
  else if (b === 0x79) { len = buf.readUInt16BE(at + 1); start = at + 3 } // 2-byte length
  if (len <= 0 || len > 200 || start + len > buf.length) return null
  return buf.toString('utf8', start, start + len)
}

What this does not do: it does not verify the signature, so a forged manifest would pass. For "is this probably AI?" that is an acceptable trade; for legal provenance, use the official c2pa-rs or c2pa-node libraries.

Your upload pipeline may be deleting the evidence

The most common reason a detector finds no label is that your own code stripped it. Our site compresses every upload in the browser and re-encodes it to WebP before sending it to storage, which is right for an image editor and fatal for a detector: re-encoding drops the metadata segments where both labels live. The detector's upload path skips that optimizer for any file up to 10 MB and stores it byte-for-byte; larger files still get compressed, and the user is told the metadata check may miss.

Social platforms and chat apps do the same thing to everyone, and a screenshot creates a brand-new file. That is why a missing label proves nothing, and why a second layer is needed.

The second layer scores the pixels

When there is no label, a classifier looks at texture, noise and structure. We call an external model for this, apimodels.app's ai-image-detector (POST /v1/images/detections, synchronous, about $0.015 per image), which also reads C2PA itself and returns a 0-1 score:

Upload is stored unmodified, then two checks run.
Byte scan for C2PA and IPTC labels. If IPTC says generated, the verdict is AI Generated (99%+); if it says composite, Digitally Edited.
Otherwise the pixel classifier's score picks the band: 85-100 AI Generated, 60-84 Likely AI, 40-59 Uncertain, 16-39 Likely Real, 0-15 Real Photo.

A metadata declaration always wins over the pixel score, because it is the maker's own statement. Without either signal, the endpoint refuses to guess: if the classifier is down and the file has no label, the user gets an error and their credits back, not a made-up "Uncertain".

How accurate is it?

On our own set of 28 images, 14 AI images from GPT Image 2 and Gemini plus the 14 real photos they were modelled on, the combined detector got 27 right: no real photo was flagged and one AI image passed as real. Calls took 1.6 to 3.1 seconds. The classifier returned near-binary scores (0.99 or 0.001), and it named the generator on none of the 28, so attribution is best-effort.

That test is small and covers two generators. Don't read it as a benchmark.

When not to build this

If you need provenance that stands up in court or in a newsroom's publishing workflow, a byte scan is the wrong tool: use a real C2PA validator that checks signatures and the trust list. And if your images come from social media, skip the metadata layer's expectations entirely; almost everything will arrive stripped.

FAQ

Can I detect AI images without calling any API? Partly. The metadata layer is free and local, and it is decisive when a label exists. It answers nothing for stripped files, which is most images shared online.

Does SynthID show up in this scan? No. Google's SynthID is an invisible watermark in the pixels, not a metadata field, and there is no public API to read it.

Why is 40-59% labelled "Uncertain" instead of a lean? Because a classifier score near the middle carries almost no information, and presenting it as "slightly AI" invites people to act on noise.

Try the detector at sparkpix.ai/ai-image-detector. For the non-code version, including reverse image search and visual checks, see How to Tell If an Image Is AI-Generated.