DEV Community: zheng

GIF to sprite sheet: the pipeline I use for Phaser and Godot, and where it breaks

zheng — Sat, 26 Sep 2026 11:30:55 +0000

An animated GIF is the easiest way to share a sprite animation and one of the worst ways to ship
one. It has no alpha channel worth the name, it re-renders every frame from a delta-encoded stream, and
your engine can't index a frame in it. So the first thing I do with any GIF that contains animation I
actually want in a game is convert it into a sprite sheet plus an atlas file.

This is the pipeline I use, the parts that bit me, and the code for both Phaser and Godot.

What a GIF actually gives you

"Just extract the frames" hides three problems:

Frame disposal. Each frame is a delta against the previous one, with a disposal method saying what to do with the previous frame's pixels (keep, restore to background, restore to previous). Extract frames by reading byte offsets naively and you get smeared, partially updated images. You need a real decoder that composites each frame onto the running canvas — the browser gives you this for free via ImageDecoder or by drawing an <img> that points at the GIF and reading frames off a canvas.
1-bit transparency. GIF pixels are either fully opaque or fully transparent. No partial alpha. So anti-aliased edges that looked fine on a flat background turn into a hard fringe the moment you put the sprite on a different background. If the source was a game animation, check whether the GIF was exported from a PNG sequence — the original PNGs are always better input than the GIF.
Palette quantization. 256 colours per frame, usually with dithering. Your character's smooth gradient is now noise. The sprite may be 2x the size as a sheet but that's a good trade.

Practical consequence: the GIF is fine as a source of frame order and timing, and poor as a source of
colour. If you have the original frames, use those.

Step 1 — get the frames out

Command line, if you're already in a pipeline:

# one PNG per GIF frame, keeping the composited result
ffmpeg -i run.gif -vsync 0 -f image2 frame_%04d.png

-vsync 0 matters: without it ffmpeg duplicates or drops frames to hit a constant rate and your walk
cycle gains or loses a frame.

In the browser, with no upload, the same job is a few lines against a canvas — decode the GIF, draw each
frame, toBlob() it. That's the path I use when the asset isn't mine to send to a server, and it's the
reason the tool I maintain, GIF to sprite sheet,
runs entirely client-side (createImageBitmap per frame, pack, canvas.toBlob, ZIP in memory).

Either way, name the frames after the action, not after the index you happened to extract them in:

idle0001.png  idle0002.png  idle0003.png  idle0004.png

Four digits, zero-padded, action name first. This is not cosmetic — the FNF/Flixel family derives the
animation from the name prefix, and your own importer will thank you when walk and walk_back both
exist.

Step 2 — pack them

Two layouts, and picking wrong costs you either bytes or correctness:

Grid — every cell the same size, frame N at (col, row). Simple, and the metadata is tiny. Right for an animation whose frames are all the same canvas (most character animations). Wastes space when frames vary in size.
Auto-pack — tight rows around differently sized rectangles. Saves texture space, but frame positions are now irregular, so the atlas metadata stops being optional. If you ever see a sheet in the wild with no JSON next to it, that's a grid.

Two settings that are not optional in practice:

Padding — empty pixels around the whole sheet.
Spacing — empty pixels between frames.

Both exist to stop texture bleeding: when the GPU samples a sprite with linear filtering or mipmaps at
a non-integer position, it reads a fraction of the neighbouring texel. On a tightly packed sheet that
fraction is the next frame's elbow. A 2 px gutter makes the artifact disappear. If your sprites still
show a one-pixel bright edge after adding spacing, you're probably seeing premultiplied-alpha rounding,
not bleeding — export as PNG-32 and draw at integer coordinates.

One thing worth knowing when you compare tools: rotated frames. Some packers rotate frames 90° to save
space. That's fine for a generic engine but breaks frame-name/atlas conventions that assume unrotated
frames. If the pipeline feeds an engine with a naming convention, keep rotation off.

Step 3 — wire it into the engine

Phaser 3

The atlas JSON is TexturePacker-shaped, so the built-in loader takes it:

function preload() {
  // sheet.png + sheet.json produced from the GIF
  this.load.atlas("hero", "assets/hero.png", "assets/hero.json");
}

function create() {
  // frame names came from the extraction step: idle0001, idle0002, ...
  const frames = this.textures.get("hero").getFrameNames();       // ["idle0001", ...]
  const idle = frames.filter((n) => n.startsWith("idle")).sort();

  this.anims.create({
    key: "idle",
    frames: idle.map((frame) => ({ key: "hero", frame })),
    frameRate: 12,
    repeat: -1
  });

  this.add.sprite(160, 120, "hero").play("idle");
}

The frames array must be sorted explicitly — the loader hands them back in atlas order, which for an
auto-packed sheet is whatever order the packer placed them in, not animation order. This is the single
most common bug I hit: the animation plays, the poses are correct, and they arrive shuffled.

Godot 4

Godot wants a SpriteFrames resource. Either import the atlas with an editor plugin, or build the
resource directly — a .tres with one animation per action:

var sf := SpriteFrames.new()
sf.remove_animation("default")
sf.add_animation("idle")
sf.set_animation_speed("idle", 12.0)
sf.set_animation_loop("idle", true)

var sheet := load("res://assets/hero.png") as Texture2D
for x in range(4):                     # 4 columns of a grid sheet
    var at := AtlasTexture.new()
    at.atlas = sheet
    at.region = Rect2(x * 64, 0, 64, 64)
    sf.add_frame("idle", at)

AtlasTexture.region is exact pixels — no trimming, no rotation. That's why the grid layout is the
friendlier one for hand-written import code.

Unity

Sprite Atlas with multiple sprites, or the texture's Sprite Mode set to Multiple with the atlas JSON
imported as sprite metadata. Unity's convention is {x, y, width, height} while Phaser's frame records
use {x, y, w, h} — a copy-paste of a loader between the two silently produces zero-sized frames
rather than an error.

The gotchas, in one list

Sort frames by name before building the animation. Atlas order ≠ animation order.
Spacing ≥ 2 px if you use linear filtering or mipmaps; 0 is fine only for nearest-neighbour pixel art drawn at integer positions.
Keep rotation off when a naming convention is involved.
Watch the sheet size limit. 4096×4096 is the common engine/GPU ceiling; a long GIF will blow past it and you want the packer to tell you, not the driver.
GIF in, PNG out. Never ship the GIF itself: no alpha, no mipmaps, no atlas.
Triple-check the transparent background. If the GIF was recorded against a checkerboard, that checkerboard is now part of your sprite.

Sanity check before you export

Play the sheet back at the intended frame rate before it reaches the engine. A missing pose, a reversed
frame or a duplicated one is instant to spot in a preview and annoying to debug inside a game loop. Any
decent packer has a preview for exactly this reason; the engine guides
section covers the per-engine import steps in more detail.

If you want to try the pipeline above without installing anything, the GIF → sheet step is a browser tool:
drop the GIF in, it decodes the frames locally, you reorder if needed, and you get a PNG plus JSON or XML
out. Nothing is uploaded, and the free path needs no account.

Shipping a polyphonic audio-to-MIDI converter that runs entirely in the browser

zheng — Mon, 21 Sep 2026 13:17:54 +0000

Almost every "audio to MIDI" tool follows the same shape: upload the file, wait, download a .mid. That is
fine until you notice what you just handed over — a demo, an unfinished song, a client's reference track.

So the constraint I set for Tonera was: the standard conversion must never upload the audio. Everything
below is what that constraint costs, and what I got for paying it.

What actually runs in the page

The transcription model is Spotify's Basic Pitch — an open-source
polyphonic pitch-detection model. It does not need a GPU cluster; it needs a fixed input shape and a bounded
amount of memory, which makes it a candidate for running client-side instead of behind an API.

The pipeline in the browser is deliberately boring:

decode the file the user dropped (AudioContext / WebAudio),
downmix to 22.05 kHz mono — the sample rate the model expects,
run the model, which returns pitch/onset/contour estimates,
turn those estimates into notes the user can inspect and edit,
export.

Steps 1, 2, 4 and 5 are ordinary deterministic code. Only step 3 is a model. Keeping that line sharp matters:
it means the editable part of the result (note boundaries, quantisation, notation) can be recomputed
instantly and reproducibly, while the model runs exactly once per file.

The ceiling nobody advertises: memory

The honest reason there is a ten-minute cap is RAM, not licensing. Decoded 22.05 kHz mono audio plus the
model's intermediate tensors have to fit in a browser tab; a phone and a desktop are very different budgets.
Instead of silently failing on a long rehearsal recording, the limit is stated up front.

The practical workaround is the same one audio engineers already use: split around musical sections and
convert the passage you actually need. You rarely want MIDI for the four minutes of tuning and talking at
the start of a rehearsal take.

Where transcription actually goes wrong

Not where people assume. It is not "MP3 bad, WAV good" — converting an MP3 to WAV before uploading changes
the container and restores nothing that the encoder already discarded.

What breaks transcription is evidence, not bitrate:

compression artefacts — short, blurred attacks in the high end look like note onsets;
reverb and room sound — tails smear note boundaries;
overlapping harmonics — a dense mix asks the model to separate instruments it was never told to separate;
quiet grace notes — the first thing an encoder spends fewer bits on.

That is why the result view is not a download button. It is a note map plus five controls: onset threshold,
frame threshold, minimum note length, and a low/high frequency range — with the discarded candidates still
visible, so raising a threshold is a decision you can see rather than trust.

Post-processing does the rest of the cleanup: quantise to a grid at the detected tempo, split into left/right
hand, smooth velocities, transpose. All of it re-runs instantly and free, because none of it touches the model.

Exports are where the tool earns its place in a workflow

One detection pass, several outputs, because the next step differs by person:

MIDI — straight into a DAW;
MusicXML — into notation software with real engraving;
PDF — for reading, printing, handing to a player;
ABC notation — for text-based / folk workflows;
chord chart — for the "I just need the changes" case.

MusicXML→MIDI, MIDI preview and WAV synthesis are deterministic converts, not generative: if you ask for the
same file twice you get byte-comparable behaviour, not a new opinion.

When server-side work is justified

Dense mixes are a genuine hard case, and pretending otherwise would be dishonest. So there is an optional
high-precision mode that runs HTDemucs stem separation before transcription — the audio is uploaded for
that path, it is labelled before it happens, and it costs credits because it consumes real compute.

Retention is stated rather than implied: temporary source objects are covered by a one-day lifecycle rule and
generated stems by a seven-day rule. If a workflow does not need that mode, nothing leaves the device at all.

Four things I would tell anyone building this

Do not promise one accuracy number. Results depend on instrumentation, room, compression and overlap. Publish the limits next to the tool; it is the difference between a demo and something a musician trusts.
Test on real material — phone memos, 128 kbps downloads, full mixes. A clean studio solo makes every model look good.
Expose the uncertainty. Discarded notes are as informative as kept ones; a threshold without visible consequences is just a slider.
Keep the model at one end of the pipeline. The moment transcription output is frozen into notes, the rest should be cheap, fast and reversible.

Tonera is where I put this together — it runs in the browser, the standard conversion is free and needs no
account, and the accuracy trade-offs are documented rather than buried:
https://tonera.app · method and limits: https://tonera.app/blog/basic-pitch-accuracy-guide

If you try it on something and the result is bad, that is useful information — the failure modes above are
exactly what I want to hear about.