DEV Community: Tony Yum

How full is a fighter's brain? A network-size experiment

Tony Yum — Sat, 31 Oct 2026 04:15:00 +0000

In short. The AI fighter’s whole repertoire, eighteen techniques plus walking, blocking and the dash, is one neural network with two hidden layers of 512 neurons: about half a million weights. Lately every new skill had cost an old one, which is what a full network looks like, but it is also what teachers pulling against each other look like. To tell the two apart I grew the network three ways, wider (1024 × 2), deeper (512 × 3) and both (1024 × 3), in a way that leaves its behaviour exactly as it was, and trained each for the same 200 million steps beside an unchanged 512 × 2 control. Closeness to the teachers: the control improved by 7 % and flattened; the grown networks improved by 20 %, 24 % and 41 %, on every one of the sixteen teachers, and the largest was still improving at the end. By that measure the small network is at its capacity. Fight tests: across 21 tests, none of the three bigger networks was clearly better or worse than the control on any of them. Closer to the teachers has not yet meant a better fighter. I kept the largest anyway, for the work that comes next.

A network that trades skills

The fighter is a policy: fifty times a second it reads 465 numbers about its body, its balance and which technique button is pressed, and outputs 48, a target angle for every joint motor. One network, the student, does everything on the try page. It learns by policy distillation: for each technique there is a teacher, a separate network trained on that one technique, and the student is trained to act like the right teacher whenever that technique runs, and like the base teacher (guard18, the stance, punches, walking and blocks) the rest of the time. In October that meant sixteen teachers poured into one student.

The last few distillations had a pattern. distill12, the student before this experiment, came out with better blocks and kicks than the one before it, and worse knees (their gate score down from 11 to 8) and a worse left elbow (52 down to 49). Earlier students had shown the same kind of trade. A network that can only learn one thing by forgetting another is the textbook sign of a full one (catastrophic forgetting is the extreme case). But there are at least two other explanations: the training budget (each run is a few hours; maybe the student just needed longer), and the teachers themselves, which were trained separately and can ask for incompatible things in similar situations.

Neural networks don’t come with a gauge. There is no reading that says “83 % full”. So the question has to be answered by experiment: give the same student more room, change nothing else, and see whether it does better.

The design

Four networks, one starting point. The obvious experiment, training a bigger network from scratch, would change two things at once: the size and everything the student had already learnt. Instead, every network started as the same function: the kept student distill12-196.6M, grown by function-preserving growth.

Wider. Each hidden layer gets 512 new neurons. Their incoming weights are random, like any new neuron’s; their outgoing weights are all zero. Whatever the new neurons compute, nothing downstream reads it yet, so the network’s outputs are exactly the old ones. Training then gives the new neurons a use. This is the widening of Net2Net (Chen et al., 2016), with zeros instead of a copy.
Deeper. A new hidden layer is added just before the output. Its weights are the identity matrix and its bias is +1. The network’s activation function, the ELU, is the identity for positive inputs, and the layer before always outputs more than −1, so after the +1 every input to the new layer is positive and passes through unchanged. The output layer’s bias subtracts the extra 1 again:

Figure 1. How the networks were grown, as weight matrices (rows are a layer's neurons, columns its inputs). Widening adds rows of random weights for the new neurons and columns of zeros where later layers would read them, so no output changes. Deepening inserts an identity layer with a bias of +1, and the output layer's bias takes the 1 back.

Before launching, I checked this on the real student with 4096 random inputs: the widened policy’s outputs were identical to the original’s (largest difference 0.0), and the deepened one’s differed by 4 × 10⁻⁵, the size of floating-point rounding. Any difference between the four after training is therefore due to their size and nothing else.

Figure 2. A rendering of the grown brain at the moment it starts training. The left and right blocks are the two original layers, their 512 trained neurons recording the student's real activity; the new neurons around them are silent (dark). The pale column in the middle is a new layer that only mirrors what reaches it.

A 2 × 2 design. Width and depth were varied separately and together, so their effects can be told apart:

network	hidden layers	policy parameters	relative size
512 × 2 (control, the original)	2 of 512	525,872	1.0
1024 × 2 (wider)	2 of 1024	1,575,984	3.0
512 × 3 (deeper)	3 of 512	788,528	1.5
1024 × 3 (both)	3 of 1024	2,625,584	5.0

(The critic, a second network of the same shape that estimates how well things are going, grows the same way and isn’t counted here.)

The control is the original network trained exactly like the others. It separates “more training helps” from “more room helps”: if the control improves as much as the grown networks, the student simply needed more steps.

Equal steps. Every network trained for exactly 200 million environment steps with distill12’s settings unchanged: the same environments, the same sixteen teachers with the same weight, the same learning rate and the same mix of practice. A bigger network is slower per step, so equal steps means unequal hours; the cost is reported below rather than levelled out.

Two measures. The first is how close the student gets to its teachers, logged throughout training for each teacher separately. The second is the gate, the same fight tests every candidate student goes through, on each network’s final checkpoint.

Distance to the teachers

During training, before every update, the trainer takes the states the student actually visited in its last rollouts and asks the teacher in charge of each state what it would have done. The distance is the mean squared difference between the student’s action and the teacher’s, averaged over the 48 joint commands, and kept separately per teacher:

where is the set of visited states in which teacher is in charge, the student’s action (the mean of its action distribution) and the teacher’s. Measuring on the student’s own states matters: a student that drifts into situations its teachers never saw is judged there, not on the teachers’ home ground. Below, distances are multiplied by 1000.

Result 1: bigger networks get closer to their teachers

Figure 3. The mean distance to all sixteen teachers over training, each point averaged over 6.5 M steps. The 512 × 2 control (dashed) improves by 7 % and flattens. Wider (1024 × 2) and deeper (512 × 3) improve by 20 % and 24 %, close to each other. Both together (1024 × 3) improves by 41 % and is still falling at 200 M. Hover for the values.

steps	512 × 2	1024 × 2	512 × 3	1024 × 3
0–25 M	4.60	4.66	4.87	4.91
50–75 M	4.48	4.37	4.42	3.98
100–125 M	4.43	4.03	4.03	3.33
175–200 M	4.28	3.75	3.71	2.88
change	−7 %	−20 %	−24 %	−41 %

Three things stand out.

The control is near its limit. With another 200 M steps of the same training, the original network improved by only 7 %, and its curve is close to flat by the end. More training alone would not have closed the gap.

Width and depth helped about equally, and together most. 1024 × 2 has twice the parameters of 512 × 3 and ended in the same place (3.75 against 3.71). Per parameter, the extra layer was the better buy. The two together did better than either: 1024 × 3’s improvement (41 %) is close to the sum of the other two (20 % + 24 %).

The grown networks start slightly behind. In the first 25 M steps the bigger networks are a little further from their teachers than the control (4.87–4.91 against 4.60), although they started as exactly the same function. The new weights are trained from their first update, and with the Adam optimiser, which scales each weight’s step to its own gradient history, the new ones move as fast as the old. That disturbs the inherited behaviour for a while before it pays off. This is my reading, not something I measured separately; it costs about 10–20 M steps.

Figure 4. Each teacher's distance over the last 25 M steps, for the four networks, sorted by the control's distance. The bar runs from the control to 1024 × 3; the right column is 1024 × 3's change against the control. Every teacher shows the same gap: 30–35 % for almost all of them, 20 % for the dash and 30 % for the base.

The gap is everywhere. If one or two teachers had been in conflict, the extra room would have helped those techniques and left the rest alone. Instead, 1024 × 3 is 30–35 % closer than the control on fifteen of the sixteen teachers, from the kicks, the hardest to imitate (distance 5–7), to the knees and the body straight (2–4). That includes the base, the stance, punches, walking and blocks that were in the network first (0.30 to 0.21). The dash, already the easiest after the base, improves the least (20 %). An even gap across every teacher is what a shortage of room looks like, not a quarrel between two teachers.

Result 2: in the fight tests, no clear difference

Distance to a teacher is a training measure. What matters is whether the fighter fights better, and that is what the gate tests: every technique at aim points in the air and at a heavy bag, with six random seeds; 120 punches; 40 walking goals; 32 blocks; 160 punches from the opponent to defend; punches at 60 set distances; the body straight. Each row counts successes out of a fixed number of tries, played without exploration noise.

For each row I took the difference between a network’s success rate and the control’s, with a 95 % confidence interval (the Agresti–Caffo interval, which behaves sensibly when a row is at 0 or at every try). Rows where all four networks scored the same, such as every kick in the air at 30 of 30, are left out: 21 rows remain.

Figure 5. The gate at 200 M steps: 1024 × 3's success rate minus the control's on each row, with its 95 % interval. Every interval crosses zero. The counts on the right are the control's and 1024 × 3's successes. A row with only 6 tries has an interval more than 60 points wide.

against 512 × 2	better	worse	tied	clearly different	sign test
1024 × 2	9	10	2	1 (right uppercut at the bag, 3 → 11 of 54)	p = 1.0
512 × 3	8	8	5	0	p = 1.0
1024 × 3	8	10	3	0	p = 0.82

Not one of the 1024 × 3 rows is clearly different from the control. It is ahead on the punches (61 against 56 of 120), the pull counter in the air (6 against 3 of 6) and the mid kicks and left low kick at the bag; it is behind on the blocks (24 against 26 of 32), the knees in the air, the right elbow and the left high kick in the air (23 against 30 of 36). Counting rows instead of looking at each, it is better on 8 and worse on 10: a sign test gives p = 0.82, no evidence either way. The one “clear” row for 1024 × 2 is what chance alone would produce: with 21 rows and 95 % intervals, about one should fall outside by luck.

Why the two measures disagree

A network 41 % closer to its teachers that fights no better needs explaining. Four reasons, from the most to the least certain:

The gate can’t see differences this size. Its rows have 6 to 160 tries. With 30 tries at a success rate near 70 %, a 95 % interval on a difference is about ±23 points wide; with 6 tries, more than ±50. To detect a real 10-point improvement on a single row with the usual 80 % power would take about 330 tries per network. The gate is built to catch a student that has broken something, which it does well, not to rank four students that are all good.

Squared distances exaggerate. The distance is a mean of squares. Its square root, a typical size of the difference in one joint command, falls from 0.065 to 0.054, 18 %, not 41 %. Most of the gain is the bigger network matching the teachers’ larger, rarer actions better, and those may matter less to a hit than to the distance.

The teachers are not perfect either. The fight tests score the student’s outcome: did the knee land within 5 cm at speed? A student that copies its teacher more closely inherits the teacher’s misses too. Once the student is close enough to land what its teacher lands, getting closer still adds little.

200 M steps may be early. 1024 × 3’s distance was still falling steeply when the run stopped. A capacity advantage might show in the fight tests only once the gap in closeness is larger still.

So the honest summary is two-sided. The small network is at its capacity: by the measure it is trained on, more training stopped helping and more room helped on every skill. Whether that capacity is what limits the fighter’s techniques today, the experiment cannot say. Within what the gate can resolve, it isn’t.

What it cost and which machine

Each network trained on its own rented RTX 5090 on vast.ai, all four at once. The bigger networks were slower per step, but much less than their size: the simulation of the body, not the network, takes most of each step.

network	steps per second	hours for 200 M steps
512 × 2	25,400	2.19
1024 × 2	23,900	2.33
512 × 3	24,100	2.30
1024 × 3	21,900 (−14 %)	2.54

Five times the parameters cost 14 % in training speed. The gate is where the size shows: it runs the policy on ordinary CPUs, and 1024 × 3’s gate took 49 minutes on 12 cores, against 30–45 minutes for the others on 16–32. Seven more boxes were rented and never came up (no connection after 4 minutes, or still loading after 10); they were destroyed within minutes and are counted in the cost at the top. Every machine is listed in the work log below.

The decision

I kept 1024 × 3, cap1024x3-200.0M, as the parent of everything that follows. The rule I set before seeing the gate was simple: if nothing in it is clearly against the biggest network, take it. Nothing was. The reason is what comes next. The plan for the coming weeks adds to what this one network has to hold: aiming at a moving opponent, a queue for the next technique so strikes can be joined into combinations, and seeing where the opponent is. Each of those is new behaviour in the same weights. A network that has already shown it has room to spare on every current skill is the better place to put them.

Figure 6. The kept network, 1024 × 3, throwing a rear knee. Its brain panel is wider and has three columns, one per hidden layer, each neuron showing its real activity in this knee.

There were bigger changes on the table if size alone hadn’t been enough: a mixture of experts, where a small network learns to blend several specialised ones (Peng et al., 2019), or a learned space of skills that a higher-level controller picks from (Peng et al., 2022). Both are much larger changes than adding neurons. This experiment says they aren’t needed yet.

Figure 7. The four networks side by side, each panel drawn to its network's size and showing that network's real activity while standing. (The four bodies all replay 1024 × 3's motion; standing, the four are hard to tell apart.)

Limitations

One run per network. Each size was trained once. The curves are smooth and the gaps large compared to their wobble, so I trust the ordering of the distances; I can’t put an error bar on them.
Different machines. The four runs were on four hosts (France, Norway, Vietnam, Canada). The steps-per-second figures mix network size with hardware.
Equal steps, not equal compute. A fair comparison per hour would give the small network about 16 % more steps. Given how flat its curve was, that would not have closed a 33 % gap.
Growth, not training from scratch. The result says how much extra room helps a student that has already learnt. A network trained at 1024 × 3 from the beginning might end up elsewhere.

What I think, as a coach

This experiment was one of several I asked to run in parallel on 31 October, before the bigger job of the week, the combo-aiming plan: “launch a bunch of steps that can run in parallel, for example … whether we need to increase the network size or more layers”. Before the fight tests came back I had already said “if nothing obviously against it, let’s go 1024x3”, and that is what happened.

When I made the short film at the top of this page, I wanted it to open on the question I care about, “how do we know when the AI’s brain is full”, and to end on what the answer is for: now that the brain is ready for more training, more Muay Thai techniques are coming.

References

Agresti, A. and Caffo, B. (2000). Simple and effective confidence intervals for proportions and differences of proportions result from adding two successes and two failures. The American Statistician 54(4).
Chen, T., Goodfellow, I. and Shlens, J. (2016). Net2Net: accelerating learning via knowledge transfer. ICLR.
Peng, X. B., Chang, M., Zhang, G., Abbeel, P. and Levine, S. (2019). MCP: learning composable hierarchical control with multiplicative compositional policies. NeurIPS.
Peng, X. B., Guo, Y., Halper, L., Levine, S. and Fidler, S. (2022). ASE: large-scale reusable adversarial skill embeddings for physically simulated characters. ACM Transactions on Graphics 41(4).
Rusu, A. A. et al. (2016). Policy distillation. ICLR.

The work logs

Work log	Runs	Hours	Cost
Students: the network-size experiment31 October 2026 · The student grown from 2 × 512 to 1024 × 2, 512 × 3 and 1024 × 3 without changing what it does, and trained beside a 512 × 2 control for 200 M steps each: the distances to the teachers, the gate, and the four boxes.	4	14	$7.65
Total	4	14	$7.65

Read it on ai.kakuto.io, with the interactive charts and the glossary.

Torque and neural visualisation of Muay Thai techniques

Tony Yum — Thu, 29 Oct 2026 11:21:00 +0000

Watch the video on YouTube

In short. The AI fighter’s brain is a small neural network: 465 numbers in, two layers of 512 neurons, 48 joint commands out, 50 times a second. I recorded everything inside it, and every joint’s torque, while it stood, took a 260 N shove from four directions, was knocked over by a 900 N one, and threw fourteen techniques. Then I drew the two side by side: the joints glow yellow with how hard their motor pushes, and a panel shows the neurons fire. Standing , no neuron is far from its resting level. A shove wakes 98–146 neurons within 0.16 s, and the fighter steps back into balance every time. The knockdown : the network’s own forecast of how things will go halves 40 ms after the push; the body hits the floor 1.94 s later. A press of a technique’s button lights 108–140 neurons on the very next tick, before the body has moved. Fingerprints : a cross and a rear hook share 28 of their 40 most responsive neurons, a jab and a left high kick share 2. Force : in almost every punch the busiest joints are the hips, not the arms; in the knees and kicks the feet take over.

Why look inside

Every other post judges the fighter from the outside: did the punch land, did it fall, does it look like Muay Thai. On 29 October I asked for something different: a film that shows “how these neural networks are interpreted in terms of what techniques being used, which neurons activated”, because “seeing through the brain of the fighter actually helps our understanding of techniques”. I wanted the torques of the joints too, and to watch the brain react as the fighter goes from balance to being knocked over, gets hit from various directions and has to rebalance.

Looking inside a trained network to see what its parts do is called interpretability. This post is a first, simple pass at it: no new training, just one fighter, recorded carefully, and a way to draw what it does.

The brain being looked at

The fighter is the current student, distill9-196.6M: one policy that has every technique on the try page, distilled from the teachers in earlier posts. Fifty times a second it reads its observation, 465 numbers: the angles and speeds of its joints, how its body is moving, where its feet are, the phase of the stance’s rhythm, and which technique button is pressed. Those pass through two hidden layers of 512 neurons each (an MLP with ELU activations) and come out as 48 numbers: a target angle for every joint’s servo. A second network, the critic, reads the same 465 numbers and gives one: its estimate of how much reward the next few seconds will bring. In the pictures that is the forecast.

Figure 1. The fighter's brain: 465 inputs, two layers of 512 neurons, 48 commands, one per joint motor, 50 times a second. The dots light up as a signal passes through; the lines are a sample of the connections (each layer has 512 × 512 of them).

Two ways to look

Joint force. Each joint motor has a maximum torque, set so the fighter is no stronger than a person. Every physics step I recorded how hard each one pushed, as a percentage of its own maximum. A joint with several motors (the hip turns three ways) shows its strongest. On the body, each joint glows yellow with that percentage; in the panel’s heatmap each row is a joint and time runs left to right. A percentage of each motor’s own maximum is how close the joint is to its limit, not how much force it makes: the hip’s strongest motor has a maximum of 220 N·m, the elbow’s 90 and the toes’ 30.

Neurons. For the second layer’s 512 neurons I first recorded three seconds of quiet standing, to learn each neuron’s resting level and how much it wanders. A neuron’s response at any moment is how far it is from rest, in units of its own wandering, a z-score:

where is the neuron’s output, and its mean and spread while standing (0.05 keeps a neuron that never moves from dividing by nearly zero). Over 3, I call it lit. In the panel each neuron is a row, sorted by when it peaks, so a wave of activity reads as a diagonal sweep. Bright means far from rest.

Every segment below started from the same reset, in the same stance, and was recorded once at 60 frames a second on the MuJoCo simulation. The robot body is drawn in Blender over the simulated one.

Standing: never still

Figure 2. The stance. The hips hold about a third of their strength (left 34 %, right 26 %) and the right knee 22 %. In the second layer, nothing is lit: the raster is the neurons' quiet wandering.

Standing is never still. The stance has a rhythm, a gentle bounce on the lead foot, and the legs carry the body’s weight through it, so the hips and knees are always working. The neurons hum but never stray: in the quiet seconds not one is lit.

A push from four sides

Then a steel block hits the chest: 260 N for 0.16 s, from in front, from behind, from the left and from the right, each from a fresh start.

Figure 3. Shoved from the front, 1 s after the push. The left hip is at 85 % of its maximum, the right hip at 76 %. In the raster, a sweep of neurons lights up after the push line.

Within 0.16 s of the block landing, 98–146 neurons are lit, depending on the direction (front 98, behind 130, left 146, right 144); at their peak 161–174. Over the second after the push, both hips average 60–64 % of their maximum when pushed from the front, and three to five joints touch 95 % or more. The fighter steps and is balanced again every time.

In all four the environment switched on survive mode. It comes on when the body’s capture point, where its momentum is carrying it, heads past the edge of its feet, and goes off once it is calm again. In survive mode the legs may move further from the stance and the only reward is staying up. The network sees the mode as one of its inputs; the recovery itself is learnt.

Do different pushes use different neurons? For each direction I took the 40 neurons that responded most in the second after the push and counted how many each pair of directions share:

	behind	left	right
front	18	18	25
behind		30	18
left			16

Figure 4. The 512 second-layer neurons, grouped by the push they answer most; bright = strong. Pushes from the left and the right share only 16 of their 40 strongest neurons; from behind and from the left, 30.

Left and right are told apart most clearly, which makes sense: they need opposite answers. The surprise is that a push from behind and one from the left look almost the same to it. My guess, not something I measured: in its side-on stance, both tip it the same way, over its lead side. Nobody designed this map; it grew out of the millions of shoves in its training.

Knocked over

The same push at 900 N is too big to hold.

Figure 5. The 900 N push. Top: 0.45 s after it, falling, with the left ankle and toes at 100 %; the red forecast line at the bottom of the panel has already dropped. Below: on the floor 2.4 s after the push, still pushing with its toes.

The forecast before the push is 555 (the reward it expects over its horizon). Two ticks after the push, 40 ms, it is already below half. It bottoms out at −92 a second later. The body hits the floor at 1.94 s. The network knew it would fall long before it did, and kept fighting anyway: 187 neurons lit at 0.16 s, nine joints at 95 % or more of their maximum. On the ground, it keeps trying.

Pressing a technique

A technique starts with a button: one of the 465 inputs flips from 0 to 1.

Figure 6. A jab, 0.75 s after the press. The busiest joint is the left hip (44 % at this moment); 161 neurons are lit. The vertical line in both panels is the press.

On the very next tick, 20 ms later, 108–140 neurons are lit, across all fourteen techniques, before the body has moved at all: the press itself is the cause, not anything the body felt. Then the wave spreads, peaking at 179–232 neurons: up to 45 % of the layer for one technique.

Three techniques switched survive mode on for a moment: the rear hook, the switch knee and the pull counter. For a short time each of them put the capture point past the edge of the feet, so close to falling that the balance rules took over.

Figure 7. The pull counter, 0.88 s after the press: lean back, then come forward with the counter. The right toes are at 97 %, the right ankle at 92 %.

Neural fingerprints

Each push and technique leaves a pattern: every neuron’s average response over the second after it. I scaled each pattern to the same size (otherwise the loud techniques all look alike just for being loud), took away the average pattern, and compared each pair with cosine similarity: 1 for the same neurons in the same proportions, 0 for unrelated, negative for opposite.

Figure 8. How alike each pair of moves' neuron responses are: red = the same neurons, blue = different. The pushes form one block, the straight punches and hooks another; the uppercuts, elbows, knees and kicks each mostly stand alone.

The straight punches and hooks share much of their brain. A cross and a rear hook: similarity 0.75, and 28 of their 40 most responsive neurons in common. Jab and cross 0.69 (23 of 40), the two hooks 26 of 40.
The pushes are a family of their own. Behind and left are the most alike pair of anything: 0.93.
The rest are individuals. A jab and a left high kick share 2 of 40. The two high kicks share 9, the rear and switch knees 11. Each knee and kick has neurons of its own.
A shared core. Eleven neurons are in the top 60 of at least 70 % of the techniques: something every technique uses, whatever it is.

The uppercuts don’t join the punches’ block: in the grid they sit with the elbows, knees and kicks, each mostly on its own.

Where the force comes from

The joints tell what each technique asks of the body. For each, the three joints that worked hardest on average in the second after the press:

Figure 9. Each technique's three hardest-working joints: mean torque as a percentage of the joint's maximum, in the second after the press. Hips (and in the kicks, ankles and toes) dominate.

Punches: the hips are the top two joints in every punch: the jab (left hip 56 %, right 54 %), the cross (61 %, 49 %), both hooks and both uppercuts. In the rear hook, the rear uppercut and the cross to the body the lower back is next (38–43 %). The arm joints never make the top three, apart from the lead hook’s shoulder girdle (29 %).
Knees and kicks: the feet take over. The switch knee’s left toes 82 %; the right high kick’s toes 65–66 % on both feet and its right ankle 64 %; the left high kick’s left ankle 69 % and right hip 68 %. The standing foot works as hard as the kicking one: it is the only thing holding the body up.

What this does and doesn’t show

It shows what goes together, not what causes what. A neuron that lights up during a hook might drive the hook, or only notice it. The next step is to switch neurons off, or hold them at rest, and see which techniques break.
One take each. Every push and technique is one recording from one start. The counts would shift a little with another start, and with the threshold (3) I chose for “lit”.
The second layer only. The first layer and the critic’s own neurons aren’t drawn.
Percent of each motor’s own maximum. Hips and ankles have far stronger motors than elbows, so the same percentage is not the same force.

No one told these neurons what to mean, or the fighter where a punch’s force should come from. Both were found by trial and error in training, and looking inside is a way to read them back out.

Read it on ai.kakuto.io, with the interactive charts and the glossary.

Muscles instead of motors

Tony Yum — Wed, 28 Oct 2026 05:04:00 +0000

Watch the video on YouTube

In short. Every fighter so far moves with a motor in each joint that drives it to a target angle. I asked what happens with muscles instead: pull-only, slow to switch on, weaker away from their best length, and stiff only when opposing muscles both work. Three stages, each learning the Muay Thai stance from scratch on a rented GPU, against the motor body on the same settings. Joint muscles (86, one pair per joint): solved in 2.1–2.9 times the training steps. Real leg muscles from MyoSuite on their anatomical paths (26 of them, then all 80): 2.6–3.9 and 4.9–5.8 times, once three fixes were in. The GPU simulation was no slower; all the extra is learning. Then the punches : the motor body learnt them on its own (hits 4 % → 9 %), the muscle body never punched fast enough to score. Its arm muscles braked every punch: forced on, the jab reaches 8.6 m/s with the opposing muscles off and 3.0 m/s with them half on (a hit needs 3.5). Relaxing the arms fixed the braking, and the fighter landed its first muscle punch, then settled on slow punches that keep it on its feet.

Motors in the joints

The fighter in every other post is a MuJoCo body with 43 moving joints, each with a motor. The motors are position servos: the policy gives each joint a target angle, as a correction to the reference motion, and the servo pushes the joint there. An action of zero already follows the reference. Holding a pose is the default; the policy only has to learn what to change.

A real body works differently. A muscle can only pull, so every joint needs at least two working against each other. It doesn’t respond instantly: the nerve signal (the excitation) takes about 10 ms to switch it on and 40 ms to let it go. Its force depends on how stretched it is and how fast it is moving. And a joint is stiff only when the muscles on both sides of it pull at once. On 27 October I asked: if the fighter were driven by muscles, how much longer would training take, is it possible at all, and how much of the existing work could be reused?

Stage 1: a pair of muscles on every joint

The cleanest first test changes only the drive. Same skeleton, masses, contacts, reference motions and rewards; each joint’s motor is replaced by two MuJoCo muscles, one pulling each way, as strong as the motor at their best length. That’s 86 muscles. MuJoCo’s muscles have the three properties that make muscle control hard: activation that lags the command, a force that depends on length and speed, and pulling only. What this stage leaves out is anatomy: each muscle acts on one joint with a fixed lever.

The policy now outputs one excitation per muscle (86 numbers instead of 43), and gets nothing for free: at zero, every muscle is at half excitation, which turns no joint and holds no posture. Left alone, the servo body stands for about 2 s on its servos; the muscle body folds in 0.6 s.

Every run learnt the stance env from scratch: hold the stance, then sway with its rhythm, then take shoves, until it passes the trainer’s test (falls at most 5 % and nearly full-length episodes, over 20 iterations at the full rhythm). Each ran on one rented GPU (RunPod, RTX 4090 or 5090), simulating 1,024 fighters at once with MuJoCo Warp.

The muscles cost nothing on the GPU. The muscle body simulated at 45.5k steps a second against the servo body’s 36.6k; the contacts dominate the cost, not the muscles.
On the servo body’s settings it never solved. Two seeds plateaued at 7–9 % falls; one ran on and collapsed to 100 % falls at 185 M steps. No muscle was at full strength before any fall. The problem was timing: the policy’s actions pass through a smoothing filter that adds about 80 ms of lag, on top of the muscles’ own.
With a lighter filter (about 20 ms) it solved: 37.8 M and 26.6 M steps on two seeds. The servo body on the same filter: 12.9 M and 12.7 M. 2.1–2.9 times the steps , seed for seed, and 1.8–2.6 times the time (10–15 minutes against 5–6).
A servo policy as a teacher (its joint torques turned into target excitations) made the start 3–4 times faster, but once the teacher faded the run went back to the same plateau. It isn’t a substitute for learning on the muscle body.

Everything except the trained models carried over unchanged: body, references, rewards, the GPU trainer, the cloud tooling, the evaluation tools.

Stage 2: real leg muscles

Joint muscles are a simplification. Real leg muscles run along paths over the bones, so their lever on a joint changes with the angle, and several cross two joints (the hamstrings bend the knee and extend the hip). I took the leg muscles of MyoSuite, an open-source musculoskeletal model built for this kind of research, and mapped their paths onto the fighter’s legs, scaled to his bones. Stage 2a used MyoLeg26 (13 muscles a leg, 26 paths in all); 2b the full MyoLeg (40 a leg, wrapping round the bones). Everything above the hips kept stage 1’s joint pairs, so the full body was 96 and then 142 muscles. I checked every muscle’s lever arm against MyoSuite’s own model at every joint it crosses: within 2.2 cm everywhere, most within 1 cm.

It took three fixes before the 2a body learnt the stance, and none of them was needed by the servo body.

A neutral zero action. At half excitation, real leg muscles don’t cancel: the hip extensors, for one, are much stronger than the flexors, and the leg was pushed up to about 180 N m off the stance. From there the policy never found the stance at all: it stood tall and straight-legged with its arms out. I set each leg muscle’s resting excitation to the values that hold the still stance, so a zero action means “stand”, as it does for the servo body.

A cost for resting on joint limits. With that, both seeds passed the trainer’s test, but in the wrong posture: the rear leg straight and swung back, the trunk bent to make up for it (Figure 1, middle). My first guess was a weak muscle: the stance turns the rear hip out 45°, where the hip abductors lose half their lever, in MyoSuite’s own models too. Making them 1.5 and 2 times stronger changed nothing, to the degree. The real cause: the rear leg was pressed against its hip and knee joint limits, with its muscles idle. A straight leg on its limits is held up by the simulated bone, for free. The policy found that and kept it. A small cost for resting within 10° of a limit took the free support away: the rear knee went to 33–35° (the reference: 43°), and the servo body didn’t notice the cost at all.

Smaller policy updates. In the right posture, the balance plateaued at 7–15 % falls and then collapsed. With a third of the learning rate and half the PPO passes over each batch, it solved within 6.6 M more steps.

Figure 1. The still stance after 3 s, deterministic policy, no shoves, seen from the fighter's left: the servo body (mbase-s0-11.7M), stage 1 (muscle1f-37.8M), stage 2a with its rear leg on its hip and knee limits (muscle2n-46.2M), stage 2a with the joint-limit cost (muscle2S-33.1M) and stage 2b (muscle2b-63.3M). The red lines on the legs of the last three are the muscle paths. Rendered in MuJoCo from the checkpoints.

With all three from the start, the 2a body solved from scratch in 33.1 M and 49.6 M steps, and the full 2b body in 63.3 M and 73.5 M, first time, with nothing body-specific added. Figure 2 puts the four bodies side by side.

Figure 2. Steps to pass the trainer's test, learning the stance from scratch; mean of two seeds, the range above each bar. All on the same action filter and reward; stages 2a and 2b also have the joint-limit cost and smaller PPO updates, which leave the servo body's result unchanged (12.3 M and 12.1 M steps). RunPod RTX 4090, 5090 and RTX PRO 6000 GPUs, 1,024 fighters at once.

So: possible, yes, at 2–6 times the training , and the more anatomy, the more training. On wall time the muscle bodies took 10 to 37 minutes on one GPU against about 6 for the servos. The muscle stances are a little looser (pose score 0.86–0.89 against the servos’ 0.95), and stage 1’s held still more smoothly than the servos (0.04 joint reversals a second against 0.15).

Stage 3: the punches

Standing is one thing; the point of the fighter is to fight. The punch env is the one the five punches (jab, cross, both hooks, the uppercut) were learnt in: punches pressed on command at targets in the air and at a heavy bag, with shoves in between. A punch counts as a hit when the fist passes within 5 cm of the target at 3.5 m/s or faster. Each body started from its own stance, on the same settings.

The servo body learnt the punches without help: within 10 M steps the punch left the guard, and over 145 M steps its hits rose from 4 % to 9 % and its misses shrank from 25 to 10 cm, with under 3 % falls. The 2a muscle body never landed one. In the first runs its arm barely left the guard (misses of 51–68 cm), and the shoves took up the whole run.

A stretch reflex

The servo body gets a rough punch for free: its targets follow the punch’s reference, so the arm goes out even before the policy has learnt anything. The muscle body has to find the punch from 96 excitations and the reward alone. In a person, the spinal cord does some of this work: a stretch reflex excites a muscle when it is stretched, which pulls a limb back toward where it was going. I gave each muscle one: extra excitation in proportion to how far it is stretched past its length at the target pose, plus a little for how fast. The arm came out at once (misses 28–31 cm, where the servo body started).

It had to be there from the start, though. Switched on under a stance trained without it, the body fell in 97 % of episodes at first and 25–30 % after 30 M steps. A stance trained with the reflex from scratch solved in 56.0 M steps, with the tidiest pose of all (0.93), but three times the jitter (0.30 joint reversals a second against 0.10). The punches from that stance: still no hits.

Why it didn’t score

Tested without exploration noise, the muscle fighter didn’t fall at all, and some punches came close: uppercuts 3–10 cm from the target, left hooks 10–16 cm. But its fist moved at 0.5–2.9 m/s. The servo body’s: 3 to 11. The punches weren’t short so much as slow.

Recording the arm muscles through a jab showed why. Both muscles of the shoulder’s and the elbow’s pairs sat between 0.3 and 0.7 excitation the whole time: a lot of co-contraction. A muscle that is switched on while being stretched resists more than it can pull, so the opposing muscles were braking every punch. To check it was that and not a lack of strength, I forced the jab’s muscles (the shoulder’s forward muscle and the elbow’s straightener) fully on from the stance, open loop, with everything else the same:

Figure 3. Peak fist speed when the jab's muscles are forced fully on from the stance, with the opposing muscles (shoulder back, elbow bend) switched off or held at half excitation, where the trained policy kept them. Stage 2a body with the reflex (jab3R at 19 M steps), CPU MuJoCo, 0.5 s. The dashed line is the speed a hit needs.

The body is strong enough: 8.6 m/s with the brakes off. With them at half, as the policy held them, 3.0 m/s, under what a hit needs. And the policy never learnt to switch them off: half excitation was the resting point of every arm muscle, and moving off it in the right muscles at the right moment is a lot to find by trial and error.

Relaxed arms

So I changed the arms’ resting point: every arm muscle rests at 10 % instead of 50 %, with the policy’s action still covering the whole range from off to full on. That’s a different starting point, not a different body: the same muscles and strengths. The stance on it solved in 47.1 M steps (pose 0.92). In the punches, the opposing muscles now switch off (the reflex pushes them to zero while the punching muscle goes to full), and the fighter landed its first muscle-body hit within 4 M steps: an uppercut at 4.3 m/s.

Then it stopped improving. Within about 10 M steps the policy settled on slow punches that keep it on its feet (a local optimum), 1.5 m/s for the jab, and it stayed there for another 120 M steps (Figure 4). With exploration noise on, as in training, it fell in about a quarter of episodes even with the shoves at their weakest; without it, never. With a third of the noise, punches came out sooner (2 % hits early) but it fell in nearly every episode and never recovered. Fast punches knock this body over. The servo body’s legs absorb them.

Figure 4. The punches on the servo body (jab3s, from its stance) and the muscle body with relaxed arms and the reflex (jab3A, from its stance), same punch env and settings, both on RunPod GPUs. Means over 10 M training steps, with exploration noise and shoves, as in training. The servo body improves throughout; the muscle body stays at about a quarter of episodes falling and almost no hits.

Figure 5. The jab at its furthest reach, deterministic policy, no shoves: the servo body after 145 M steps of punch training (jab3s) and the relaxed-arm muscle body after 133 M (jab3A). The servo arm is straight and on the line; the muscle body keeps its elbow bent and its weight back. Rendered in MuJoCo from the final checkpoints.

I stopped there. The next steps are bigger pieces of work, not more steps: a teacher that shows the muscle arms the servo body’s punch, or a hit-speed bar that starts low and rises as the fighter gets better.

What it means

Possible: yes, for the stance. A body with 142 muscles, real leg anatomy among them, learns the Muay Thai stance, its rhythm and shoves, from scratch, in under 40 minutes on one GPU.
How much longer: 2–3 times with joint muscles, 5–6 times with the full leg , measured against the servo body on the same settings. The simulation isn’t slower; the extra is all learning.
What carries over: everything except the trained models: the body, the references, the rewards, the training code and the tooling. A servo policy as a teacher helps only at the start.
What a muscle body needs that a motor body doesn’t: a resting point that holds the pose; a reason not to hang on its joint limits; smaller learning steps near its balance limit; for the arms, a resting point low enough to let them move fast.
What it hasn’t learnt: to punch fast and stay on its feet. The servo body gets the punch’s shape for free and its legs absorb the swing; the muscle body has to find both. Its arms can move fast enough. Putting speed and balance together is the open problem.

What I think, as a coach

My question was a simple one: if this fighter had muscles instead of motors, how much longer would he take to learn, could he learn at all, and how much of what we’ve built would still be useful? I let the experiment run on its own recommendations (“go with all your recommendation”), and from the second day I told it to make the decisions itself on a $50 budget and report back. I watched the videos of each new stance as they came in.

So far we’ve used servo motors for the body because they’re convenient and quick to learn. But the goal really is to fight Muay Thai as a human. So it’s very important to understand how muscles move, and eventually the energy those movements cost.

This is a very early experiment. It doesn’t cover all the techniques, and we didn’t try much on redundancy, or on injury to muscles and hyperextension. Eventually we’ll do Brazilian Jiu-Jitsu, where a joint can be hyperextended and a ligament broken. So this is literally my first experiment in understanding how a servo robot translates to muscles and a more human-like structure.

The work logs

The runs behind this post, stage by stage. The costs are each RunPod pod’s billed hours times its hourly price, from the RunPod ledger, so they are estimates, not the bill. They include the short GPU checks that the simulation of each new body matched the CPU one.

Work log	Runs	Hours	Cost
Muscles, stage 3: the punches28 October 2026 · The five punches on the muscle body, against the servo body on the same settings. The muscle body never punched fast enough to score; why, a stretch reflex and relaxed arms, and where it stopped.	10	9	$11.18
Muscles, stage 2: real leg muscles27–28 October 2026 · MyoSuite's leg muscles on their real paths, mapped onto the fighter: 26 of them, then all 80. A torque-neutral zero action, a cost for resting on joint limits and smaller PPO updates got both to the stance.	15	6.1	$9.69
Muscles, stage 1: joint muscles instead of motors27 October 2026 · An experiment I asked for: drive the fighter with 86 joint muscles instead of servos, and measure how much longer the stance takes to learn, if it can be learnt at all.	8	4.9	$5.75
Total	33	20	$26.62

Read it on ai.kakuto.io, with the interactive charts and the glossary.

Correcting the knee

Tony Yum — Mon, 26 Oct 2026 12:43:00 +0000

Watch the video on YouTube

In short. Every new technique now starts as motion capture of me, filmed with two phones and reconstructed in 3D with open-source software on a GPU. It then goes through six more stages before the fighter can use it: retargeting onto the fighter’s body, a table of variations, a one-technique teacher, an evaluation gate, distillation into the student, and a final check. I review the result between every pair of stages, because each one can be wrong in a way its numbers don’t show. The second half of this post is one such case. The rear knee to the body passed every test: it landed, it didn’t fall, and it followed my motion. But after the knee came down the fighter arched backwards for about half a second. The reward allowed it; an opponent wouldn’t. A reference that returns to upright, plus a one-sided penalty on leaning back past it, reduced the lean past the reference from 12.4° to 1.3° in the air and from 9.7° to 0.9° at the bag. Every bag knee still landed and none fell. The fix survived distillation into the student: 1.4° and 1.2°.

From describing to capturing

When I started, I taught the fighter the way I’d teach a beginner over the phone. I described the technique in words, sent photos of myself from a few angles, and corrected what I saw: tuck the elbow in more, stretch further, turn the hip. The stance and balance and the jab were built that way, as poses designed on the simulated body itself, and it worked well enough for them.

It doesn’t work for bigger movements. A knee or a kick moves the whole body through large angles, quickly, and the order of events matters: which foot leaves the floor first, when the hips turn, where the arms go to balance. Describing all of that in words gets you something that moves, but not something that looks like Muay Thai. So I moved to motion capture.

The pipeline

I don’t use a commercial motion-capture system. Everything is open source, mainly Pose2Sim, which means the heavy parts can run on whatever GPU machine is cheapest. Figure 1 shows the stages. The purple ones end with me looking at the result before the next stage starts.

Film me, two phones

A calibration board on the floor, photographed once from each phone's spot. Then the board is taken away and both phones film the technique at 60 fps from 60–90° apart, without moving.

2D keypoints GPU

A pose estimator (RTMPose, with heels and toes) finds the joints in every frame of both videos: about two minutes for a 30-second take.

Sync and triangulate GPU / CPU

The two videos are lined up by their sound, each joint is triangulated in 3D from the two views, and the paths are smoothed.

Fit my skeleton CPU

A skeleton scaled to my measurements is fitted to the 3D points, giving my joint angles frame by frame.

Retarget CPU

My joint directions and turns are moved onto the fighter's body, and a small inverse-kinematics step keeps planted feet planted.

I check the sim beside my video, through each phone, frame by frame. I mark what's wrong and choose between copying me exactly and what works on his body.

Table of variations CPU

The technique is re-solved for targets a little higher, lower, left and right, so he can strike from wherever he is.

I check the corners and the in-betweens, where knees twist and feet lift.

Teacher rented GPU

A copy of the latest model learns only this technique (plus its stance and balance) by reinforcement learning against the table.

I check short progress clips sent to me through the day.

Gate CPU boxes

Deterministic tests: hits in the air and at the bag, falls, feet, facing, lean.

I check the final video, slowed down.

Distil into the student rented GPU

One student learns every technique from its teacher while keeping all the old skills.

I check the student's gate and videos. A fault here is fixed in the teacher, then distilled again.

Figure 1. From a filmed technique to a skill in the student. Turquoise steps run unattended; purple steps end with my review, and a technique goes back a step (or several) when I find something wrong. Steps 2–3 run on a GPU; steps 7 and 9 on rented GPUs (vast.ai or RunPod RTX 5090s, about 30,000 simulation steps a second).

Filming and reconstruction

Two phones see the same moment from two directions, so the depth of every joint is measured by triangulation instead of guessed. My first attempt used one phone, guessed depth from bone lengths, and at one point put the legs the wrong way round in the stance while every picture-based check passed. For triangulation to work, each phone’s lens and position must be known exactly. That is camera calibration, done with a printed board of known size on the floor.

A pose estimator finds my joints in each frame; Pose2Sim lines up the two videos, triangulates the joints, smooths them with a low-pass filter (12 Hz, not the 6 Hz default for walking, which would round off a kick), and fits a skeleton scaled to me using inverse kinematics.

Where it goes wrong

The reconstruction is good, but not good enough to use without checking. The errors I’ve seen fall into three kinds.

Wrong estimates. A foot judged to be below the floor gets pushed back up, and the body pops up and down for a few frames. A limb pointing straight at a camera collapses onto its joint; a sole facing a camera drags the knee onto the foot; left and right swap when the body turns side-on. In the knee take used below, for 0.06 s after the knee the reconstruction throws my pelvis back 26 cm and my rear foot jumps, so the end of the clip had to be replaced.
Different bodies. The fighter’s legs are about 25 % longer than mine relative to his trunk. Copying my joint positions folds his elbows into his head and bends his knees oddly, so retargeting copies directions and turns instead, and fixes only what has to touch something. Even then his body has limits mine doesn’t: I plant my rear foot about 31 cm behind my hips after the knee, past his hip’s range, so his lands 8 cm nearer.
Estimates of what can’t be seen. Forearm roll, wrists and collarbones aren’t visible to the cameras and are held at the stance.

So step 5 is a review. I watch my video beside the simulated fighter rendered through each phone’s view, slowed down, and point out the differences. Each one is a choice: copy me more exactly, or accept a difference that works better on his body. The corrections go into a layer of eased offsets on named joints.

A table of variations

A fighter can’t only strike from one recorded position, because in a fight he and his opponent are both moving. So each technique becomes a table: the same strike re-solved for a grid of targets a little higher, lower, left and right (for the kicks ±10 cm each way). In training the target can be anywhere inside the grid, and the reference motion is interpolated between its nearest entries.

A perfect reference can still produce bad entries in the table. In the low and mid kicks, the foot’s inverse kinematics flicked the knee inwards by about 40° over 4 frames during a switch hop. Nothing measured caught it; I saw it on the preview. So step 6 is another review.

Teacher, gate, student

A teacher is a copy of the latest model that practises only this technique, with every episode a press of it and shoves in between to test balance. It learns by reinforcement learning (PPO), rewarded for following the reference, for landing on the target, and for not falling. It doesn’t copy the reference exactly, and it can’t. The reference has no physics in it: my capture doesn’t know the bag pushes back, or that the fighter’s mass is spread differently from mine. A policy that copied it exactly would fall. During training the GPU box sends me short clips, and I watch them through the day.

The gate then runs deterministic tests: knees in the air and at a heavy bag, six seeds each, with hits, falls, where the feet end, which way he faces and how far he leans. Teachers that pass are distilled into the student (policy distillation, described in the shadow-clones post). The student is checked again, because distillation is a second round of training and things can change in it.

Even after all of that, the result can be wrong.

The case: the right knee

The rear knee to the body comes from my take of 23 October: the lead foot steps in about 28 cm, the rear foot leaves the floor, the hips turn about 70° and the right knee drives up and forward into the opponent’s body, 1.14 m high at impact. The knee comes down, the rear foot lands ahead of where it left, and the fighter steps back into his stance. Its teacher took three runs (work log): the first landed every knee but finished in a southpaw stance 0.6 m forward, the second fixed the feet, and the third, kneeR3, made the steps home. kneeR3 passed the gate: in the air 5/6, at the bag 6/6, feet back in the stance, no falls. It went into the student, distill4.

What the training objective allowed

Leaving out the details, the teacher maximises, at each step ,

following the reference, landing the knee (within 5 cm of the target and fast), keeping the feet where the reference has them, and not falling. None of these terms says anything about the shape of the trunk after the knee lands, as long as he doesn’t fall and the pose stays close enough to the reference.

And close enough it was. After the impact, my capture leans back a little: my pelvis 5–9° back and my chest 10–16°, while my stance is near 0° and +1.5°. kneeR3 followed that lean and pushed it further, tipping his pelvis up to about 8° further back than the reference. He bent his lower back about 20° to keep his chest on the reference: hips forward, chest back. That’s an arch. Then he recovered, sometimes with a small hop, and stood in his stance again. Every term was satisfied.

Why that is wrong

I noticed it in the student’s video: “after the knee and the feet land back on the ground the fighter arches its back”. From a reward’s point of view this is a perfectly good knee. From a fighter’s point of view it isn’t. For the half-second he is arched, his hips are pushed forward and his shoulders hang back over them, his feet are busy catching him instead of being set, and he can’t step, block or strike back (Figures 4 and 5). That’s when an opponent hits him. The technique isn’t finished when the knee lands; it’s finished when he’s ready again.

Measuring it

To fix it I first needed a number for it. The trunk lean of a body part is the angle of its “up” axis along the direction the hips face on the floor, :

computed for the pelvis and the chest. The gate’s new row is the most either one leans back past the reference after the hit:

where + = \max(x, 0), so leaning forward of the reference counts as zero, and {\text{up}} is the start of the upright window below. Against the corrected reference, kneeR3 scores = 12.4° in the air and 9.7° at the bag.

The fix

Two changes, one to the target and one to the reward.

The reference. From 7.05 s of the capture, just after the hip thrust, to 7.35 s, the pelvis’s lean and the spine’s bend are eased to the stance’s and held there to the end. Before 7.05 s the reference is unchanged, including the impact at 6.933 s. I approved the preview before any training. This is one of those trade-offs between copying me and what works: my own lean back after a knee is small and I recover from it without thinking, but on his body, with the policy free to exaggerate it, it turned into an arch.

A one-sided reward term. After the hit window, every step pays

where is the weight the technique’s terms have at that moment. This is a Gaussian kernel on the lean back past the reference: 1.5 when he’s at or forward of the reference, about 0.9 at 5° back, about 0.2 at 10° and almost nothing past 15°. Leaning forward is free, because leaning forward after a knee into the body is what a fighter does anyway.

The new teacher, kneeR4, was warm-started from kneeR3 on one rented RTX 5090 for its 2-hour cap (207.9 M steps, about $1.29). The lean dropped almost at once: by 13 M steps the bag row was down from 9.2° to 1.8°.

Results

Figure 2 shows the lean of the pelvis and the chest through one knee at the bag, step by step, before and after the fix, in the student. Each line is the median of six seeds and the band their range.

Figure 2. Pelvis and chest lean through one rear knee at the heavy bag, in the student before the fix (distill4-131.1M, kneeR3's knee) and after it (distill7-245.8M, kneeR4's), time from the press, 50 steps a second; median of 6 seeds, band = their range; dashed = the corrected reference. Both lean far back at the impact: that is the hip thrust, and the reference does it too. The difference is after the landing. From about 0.9 s to 1.9 s the pelvis before the fix sits 12° behind his stance on average (20° at worst); after it, 5° (7° at worst). In his stance the pelvis tips about 10° forward; the reference holds it at 0° after its upright window, so after the fix he ends between the two.

The picture is the same in the air and for the teachers. Figure 3 compares the two numbers that matter for all four models, and the table has the rest of the gate.

Figure 3. Before and after the fix, teachers (kneeR3-183.5M, kneeR4-183.5M) and students (distill4-131.1M, distill7-245.8M), median of 6 seeds. Top: the gate's lean row, the most the pelvis or chest leans back past the reference after the upright window starts. Bottom: how far the pelvis sits behind his own stance, on average, from the end of the upright window until the knee's reference ends (about 0.9–1.9 s after the press).

model	lean past reference, air / bag	hits, air / bag	feet at the end (lead, rear; stance 0.37, 0 m)
kneeR3-183.5M (before)	12.4° / 9.7°	5/6, 6/6	0.42–0.44, −0.01–0.02
kneeR4-183.5M (after)	1.3° / 0.9°	0/6, 6/6	0.40–0.41, −0.01
distill4-131.1M (student before)	11.4° / 10.7°	in the gate: 11/12 overall
distill7-245.8M (student after)	1.4° / 1.2°	2/6, 6/6

Figure 4. The student at the heavy bag, seed 1, seen from behind his right shoulder, with the camera turning with his hips, so the line of his back shows side-on: before the fix (top, distill4-131.1M) and after it (bottom, distill7-245.8M), 1.02, 1.14, 1.26, 1.38 and 1.50 s after the press, the rear foot just down. Before the fix the hips are pushed forward under a trunk that hangs back: the pelvis is tipped 7° and then 11° back in the first two frames and only reaches his stance by the fourth. After it the pelvis stays 4–6° forward the whole time and the trunk is stacked over the feet. Rendered in MuJoCo from the checkpoints, deterministic policy.

The arch is the part I pointed at, but what worried me more was the balance. Seen straight from behind (Figure 5), the student before the fix spends the time after the landing catching himself with his feet. After the rear foot comes down it comes up again, 5–11 cm across the six seeds, where after the fix it rises 4–5 cm as he settles it. In seed 1 both feet are off their resting height at once for 0.44 s, a small hop to stay up. After the fix no seed does that. His centre of mass stays between his feet in both models, about two-thirds of the way from the rear foot to the lead foot, so he was never close to falling. He doesn’t fall, but he isn’t set either, and a fighter who isn’t set can’t fight.

Figure 5. The same knee, seed 1, seen from straight behind him, the camera again turning with his hips: before the fix (top, distill4-131.1M) and after it (bottom, distill7-245.8M), at the same five moments. Before the fix he is still catching himself after the landing: his rear foot comes back up 8 cm and his lead foot 6 cm, both at once for 0.44 s. After it the rear foot rises 4 cm as he settles it, the lead foot stays down, and he is back in his guard. Rendered in MuJoCo from the checkpoints, deterministic policy.

What the fix cost, and what it didn’t fix

Knee hits in the air. kneeR4 hits 0/6 in the air, where the kept kneeR3 snapshot hit 5/6. The fix may play a part, but the air row was unstable before it too: kneeR3’s later snapshots hit 2/6 and 0/6. The air test counts a hit only at 3 m/s within 5 cm of the target, a fist’s speed. The reference knee crosses the target plane at under 1 m/s, so a knee that follows my capture rarely counts. At the bag, which is three quarters of the presses and is scored by the push, every knee lands.
The hips’ bob. The hips still rise and fall about 4–5 cm as he comes out of the knee, about the same before and after in the student. This was not the complaint, and the fix doesn’t target it.
The stance mismatch. The upright window brings the pelvis to the reference’s stance, which is about 10° more upright than the guard stance the base model actually stands in. The term only punishes leaning back past the reference, so he ends between the two (Figure 2). A cleaner fix would take the reference’s stance from the base model.
Not every snapshot. The fix holds across kneeR4’s snapshots (about 1–5° in the air, 0–1.8° at the bag), but the last one, at 206 M steps, leaned 10.8° in the air on one seed. 183.5 M was kept.
Small samples. Six seeds per row, deterministic. That’s enough to see a 10° effect clearly, and not enough to rank differences of a degree.

Into the student

The fix was made in the teacher, not the student, because the student’s knee is only as good as the teacher it copies. distill7 was warm-started from distill4 and distilled again with kneeR4 as the knee’s teacher (along with two new uppercut teachers), on a RunPod RTX 5090 for 2.5 hours (work log). The distance between the student’s and kneeR4’s actions fell from 0.025 to 0.0027 (mean squared error). The student’s lean past the reference went from 11.4° / 10.7° to 1.4° / 1.2°, the same as the teacher, with no falls, the bag knees 6/6 and the old skills at about distill4’s level. I watched its videos and said: “Looks good.”

What I think, as a coach

On paper, the knee was already good. The policy maximises the knee’s impact on the target, it’s punished if it falls, and it tries to follow my reference, but not exactly, because the physics is different: the bag’s weight, the recoil, his body against mine. If it copied me exactly it would fall over. So when it staggered back after the knee, arched a little and then recovered, that was perfectly fine by everything we had told it.

But when I looked at it, all I could see was this: while he’s off balance, he’s going to get hit. In the gym, a knee you can’t recover from straight away is a bad knee, however hard it lands. Nobody counts that in a score; you just see it, the same way you see a fighter who drops his hands after a punch.

That’s why I don’t trust any stage of this pipeline on its numbers alone. The motion capture can be wrong, the table can twist a knee, the teacher can find a way to satisfy every term and still look wrong, and the student can lose something in distillation. Every time, the fix has been the same: watch it, say what’s wrong in plain words, turn that into something the training can measure, and check again. The knee went from “fine according to all the policy” to something I’d be happy to see in the ring.

The work logs

The runs behind this post are in these work logs: the teacher that fixed the knee, and the evaluation and render boxes that made this post’s figures (EC2 spot, about ten minutes each, because my small development server is throttled on CPU). The earlier knee teachers and the student distill7 are counted in the shadow-clones post and their own work log. The costs are each machine’s billed hours times its hourly price, from the vast.ai ledger and the EC2 spot price, so they are estimates, not the bill.

Work log	Runs	Hours	Cost
Evaluation: the rear knee's lean, step by step28 October 2026 · Four short EC2 boxes that traced the trunk's lean through the rear knee, before and after the upright fix, and rendered the stills for the post.	0	0.4	$0.11
Teacher: the rear knee, landing upright26 October 2026 · One more rear-knee teacher, so the knee lands upright instead of arched back.	1	2.1	$1.29
Total	1	2.5	$1.40

Read it on ai.kakuto.io, with the interactive charts and the glossary.

Shadow clones: four Muay Thai techniques learnt in parallel

Tony Yum — Sat, 24 Oct 2026 20:30:00 +0000

Watch the video on YouTube

In short. Each new technique is trained in its own copy of the latest model, a teacher, on its own rented GPU. A teacher only has to learn its technique and keep the stance and balance. The teachers that pass are then distilled into one student: it keeps training with PPO, and is also pulled towards whichever teacher owns the technique it is doing at that moment. Four teachers passed: the rear straight to the body, the rear knee, the switch knee and the pull counter. The kept student, distill4-131.1M, has the first three and still has every earlier skill. The pull counter only half came back. Several clones finished training, scored well on the numbers and were still wrong, because of the motion they were copying or because of how training rewarded them. This post covers about $80 of training in eight work logs. It is the first part of a longer story.

This post is written in the first person. Earlier posts talked about “Tony” and “the coach”. From now on it’s just me.

The Naruto idea

In Naruto Shippuden there is a scene I kept thinking about while planning this. Naruto has to master a technique in far less time than it should take. Kakashi’s solution is the shadow clone. Naruto makes a crowd of copies of himself, they all train at once, and when a clone is dispelled, everything it learnt goes back to the original. A week of practice fits into a day. There is a catch: the clones’ fatigue comes back too.

That is how I wanted to add techniques to the fighter. Until now every new skill was fine-tuned into one model, one after another, with the older skills held in place by a teacher. Each new technique waited for the one before it, and each one risked the others (catastrophic forgetting). Training techniques separately is close to embarrassingly parallel. A knee doesn’t need to wait for a body straight, so they can run on different machines at the same time.

So the plan was:

Take the latest model (the base) and make a clone of it for each new technique.
Train each clone on its own rented GPU. It only has to learn its technique and keep the stance and balance: “all it needs to know is the stance and the balance and just the specific technique”.
Bring the experience back: distil the clones that pass into one student, which keeps everything the base could do.

Figure 1. Parallel training as shadow clones. The base (guard18-124.5M, every earlier skill on the new high guard) is copied into one-technique teachers. Each trains on its own rented GPU (vast.ai RTX 5090, or RunPod for the pull counter and the uppercuts). The four that passed (solid) were distilled back into the student. Dashed: clones that finished training and were judged failures, or are not distilled yet.

The rest of this post describes how a clone is made, how the experience is brought back, and what went wrong. Every run is in the work logs.

Making a clone: the one-technique teacher

A teacher starts as a copy of the base: a multi-layer perceptron with two hidden layers of 512 units. It reads 448 inputs and drives 48 motor outputs. Three things turn the copy into a teacher.

A new input, added without changing the network. Each technique gets its own input, “this technique is running”, placed just before the last input (which is the balance gate). The new input’s weights into the first layer start at zero. If is the first layer’s weight matrix, its columns are split as and the grown network uses

so on the first step the grown network behaves exactly like the base (function-preserving growth). It can only start using the input through training. Every teacher reads the same input layout the final student will have, so the student can later learn from all of them on the same observations.

Practice only that technique. Every episode is a punch episode, and every press is the teacher’s technique, with shoves in between so the balance is still tested.

The parent holds everything else. A frozen copy of the base stays on as a teacher everywhere except while the new technique runs (see policy distillation). The teacher is free to change its motion during the technique, and is pulled back to the base’s behaviour the rest of the time. That keeps the stance and the recovery from shoves, which the technique has to start from and end in.

Each technique’s motion comes from my own motion capture, the reference motion the teacher is rewarded for following. For example, the rear straight to the body is a 1.45 s window of one of my takes.

Running the clones side by side

The trainer runs on one GPU: 2,048 copies of the world (parallel environments) for 16 steps each, 32,768 steps per iteration. On a rented RTX 5090 that is about 25,000 to 34,000 steps a second. Each clone gets its own machine. Two knee teachers trained side by side, then their fixes ran side by side. Four evasion teachers started within a minute of each other. Two uppercut teachers ran on two RunPod machines at once.

technique	teacher kept	runs	from the parent	what it took
rear straight to the body	bodycross4-118.0M	4	guard14 (guard18 holding the rest in the last run)	a “no low blows” line, the bag moved to the impact point, then retrained on the base’s slower elbows
rear knee	kneeR3-183.5M	3	guard14	the feet kept where the capture has them, then the steps home at the capture’s pace
switch knee	kneeL3-214.0M	3	guard14	the same
pull counter	pull1-209.7M	1	distill1-282.5M	8/8 in the air and 8/8 at the bag, first try

Training each teacher takes about two hours. Most of the time and money goes into the fixes: several short runs, each warm-started from the last good checkpoint with one change. Because the clones are independent, a failed fix on one technique costs nothing on the others.

Bringing the experience back: distillation

The student starts as the base, grown to read every teacher’s input. It keeps training on the full mix of episodes (punches, blocks, walking, the opponent, shoves and the new techniques). Its loss is PPO’s plus a pull towards the right teacher.

The student’s policy is a Gaussian, (see stochastic policy). There are technique teachers plus the base . Each state is routed to one of them by its inputs:

and every minibatch of the student’s own experience is trained on

The first three terms are the usual PPO training: the clipped policy objective, the value error and a smoothness penalty on the actions (CAPS). The last is a squared error between the student’s mean action and the routed teacher’s mean action, with weight (2 in the first runs, 4 in the kept one). A few details matter:

The states are the student’s own. The teachers label the states the student actually visits (on-policy), not states from their own runs. Copying a teacher from its own recordings (behaviour cloning) breaks down once the student drifts off the teacher’s path, because it has never seen those states (covariate shift). Asking the teacher what it would do in the student’s states is the DAgger fix.
Each teacher sees what it was trained on. A teacher reads its own slice of the inputs, normalised with its own running statistics. An earlier bug, a teacher seeing inputs scaled by the student’s statistics, had made its advice slightly wrong.
The rewards stay on. The student still earns every task reward, so it can’t just copy a teacher into a fall.
One mean, many teachers. Exactly one teacher is in charge of each state, so the targets never conflict. The student has to fit all of them in one network.

The trainer logs one number per teacher to watch: the student’s mean squared error against that teacher’s action means, over the states routed to it, before each update. If the error stalls high on the new techniques while the base’s stays low, the network has run out of room (capacity), and I would widen it. It never stalled. In the first run the base’s error stayed at 0.002 throughout. The body straight fell from 0.028 to 0.005 by 39 M steps, the rear knee from 0.054 to 0.007 and the switch knee from 0.059 to 0.008. They were still falling when the run ended at 282.5 M. Three techniques fit in 512 units with room to spare.

What came back

Every candidate is judged by the gate: deterministic tests of every skill, the old ones and the new ones. Six distillation runs went through it. Figure 2 compares the base with three students, and the table has the full rows.

Figure 2. The gate for the base (guard18-124.5M) and three students (new techniques: – where the gate had no row for them). distill1 has both knees but not the body straight (its teacher was trained on faster elbows than the base's). distill2 adds the body straight from the retrained teacher but loses the cross in the air. distill4 (kept, at 131.1 M steps) has all three new techniques and is at or above the base on punches, walking, the opponent and distances. It is 3 short on blocks.

	punches /120	cross in the air /12	walk /40	blocks /32	opponent /160	distances /60	body straight /24	knees R, L /12
guard18 (base)	59	3	39	27	114	34	–	–
distill1 282.5 M	53	0	39	25	109	36	(not in yet)	6/8, 5/8 at 40 M
distill2 52.4 M	54	0	38	29	122	38	21	10, 12
distill4 131.1 M	60	3	39	24	117	39	20	11, 9

Figure 3. The kept student, distill4-131.1M, throwing the three techniques it learnt from its clones, each at the heavy bag at the moment of impact, seen from its right side: the rear straight to the body (left), the rear knee (middle) and the switch knee (right). Rendered in MuJoCo from the checkpoint, deterministic policy.

The student was picked from snapshots, not from the end of the run. The training averages hardly moved, but the deterministic gate swung from one snapshot to the next (blocks between 21 and 29 in distill3). distill4’s five snapshots were all gated, and the one at 131.1 M was kept. What made distill4 better than distill2 was a stronger pull ( from 2 to 4) and more practice for the cross. That brought the cross in the air back, from 0/12 to the base’s 3/12, and tightened every teacher’s match. The cost is 3 blocks, and putting the block share back (distill5) didn’t recover them.

Not everything comes back

distill6 added the fourth teacher, the pull counter (lean back, then a rear straight to the head). The teacher hits it 8/8 in the air and 8/8 at the bag. The student learnt it at the bag (6/6) but not in the air (1–2/6, about 10 cm short), and none of its snapshots beat distill4 overall. So for now the pull counter lives in its clone, not in the fighter. The next distillation gives it more presses and a stronger pull.

Figure 4. The pull-counter teacher, pull1-209.7M, at the heavy bag: the lean back away from the punch with the rear fist wound back (left), then the rear straight to the head (right). This one lives in its clone for now: the student learnt it only at the bag.

It is a reminder that distillation is not like Naruto dispelling a clone. The experience doesn’t arrive whole. It has to be trained in, alongside everything else the student must keep, and where two skills compete for the same output the result is a trade.

Clones that finished and were still wrong

The worse surprise was clones that finished training, passed or nearly passed the gate, and were wrong when I watched them. The gate says a technique landed. It doesn’t say the technique looked like the one I asked for. These are the main cases. The details, with the numbers, are in the work logs.

The reference. The evasion teachers (slips, rolls, lean back) learnt from my motion capture, but they ran it up to twice as fast. The punch clock, which lets the policy speed up a punch, had been built for punches and was still on for these. A roll squeezed that much becomes a drop. When I asked for a wider, U-shaped head path and the reference itself was widened, the trunk bent sideways to reach it, and the teachers learnt the bend. Every roll and lean-back teacher was judged a failure on video. The slip uppercut is kept only until a better one exists (“looks like a big uppercut [more] than a slip”). They all start again from the original capture.

The training policy. The lead uppercut over distance landed 40/54 in the air, but after it the fighter turned 90° to his right and stayed there. The cause was not the reference, which ends square at every depth. Training rewarded a home heading that was reset at every press and every recovery from a shove, and the opponent turned to face wherever the fighter faced. Nothing paid for facing one direction. Both uppercuts are now retraining with the heading fixed per episode, and the gate now measures the turn after every technique. Smaller cases of the same kind:

A knee that landed in a southpaw stance 0.6 m forward, fixed with terms for the feet.
A body straight that went below the belt, fixed with a line it may not cross.
A body-straight teacher trained on faster elbows than the student has, which missed once distilled and had to be retrained.

Start again, or restart from a checkpoint? Most fixes warm-start from the last checkpoint that was good where it mattered. A teacher whose problem is in its reference has to start again from a checkpoint before it learnt the wrong motion. And the first stepping attempt was given up altogether: the long step never learnt to push off the back leg, and those references will be rebuilt from a better motion capture.

What I think, as a coach

The shadow-clone idea worked the way I hoped where it worked. Three new techniques arrived in one model without breaking any of the old ones, and none of the clones had to wait for another. It also changes how I coach the fighter. I can look at one technique at a time, in a clone that does nothing else, and correct it before it goes into the student.

The failures were mostly the kind a coach knows from people. A student who rushes a slip because nobody told them the timing matters. A fighter who finishes a combination facing the wrong way because their drills never asked them to stay square. My comments on the videos were all about shape and timing, things the numbers didn’t catch:

The roll should not just be a drop, it should roll to the side like a U shape.

Slip and roll are all wrong. It’s going sideways but not by bending the torso.

Keep to my original reference, like the one from motion capture.

After the uppercut the fighter turns 90 degree to the right. It needs to keep facing the opponent.

The lesson I take from these is that the motion capture has to stay mine. Every time the reference was edited to get a cleaner number, the movement got less like Muay Thai.

The work logs

Everything behind this post is in eight work logs, one per area. Each says why every run was started, what changed, what the gate and the videos showed and what it cost. Together they cover every run since the last post, and they add up to the cost at the top of this page and its row on the stats page. The costs are from the vast.ai and RunPod ledgers, each machine’s billed hours times its hourly price. Machines that never came up and RunPod refusals are in them too.

Work log	Runs	Hours	Cost
Teachers: the uppercuts over distance25 October 2026 · Two uppercut teachers trained in parallel on RunPod. The rear one was kept; the lead one turned 90° after the punch, which traced back to how training rewarded facing.	2	10.1	$7.81
Teachers: slips, rolls and the lean back25 October 2026 · Nine evasion teachers in two rounds of four parallel boxes. They passed the gate and failed on video: rushed by the punch clock, then bent by an edited reference.	9	20	$12.97
Distillation: six runs to the student distill424–25 October 2026 · Distilling the body straight, both knees and the pull counter into the base: what each run changed, how the gate picked the student, and what it cost to evaluate.	6	22.9	$16.27
Teacher: the pull counter24–25 October 2026 · One run on RunPod after four vast.ai boxes never loaded: a lean back and a rear straight to the head, 8/8 in the air and at the bag.	1	3.1	$3.45
Teacher: the rear straight to the body24 October 2026 · Four runs to a body-straight teacher: a no-low-blow line, the bag at the impact point, then a retrain on the base's slower elbows.	4	5.7	$4.01
Teachers: the rear knee and the switch knee24 October 2026 · Two knee teachers trained side by side, three runs each: the knees landed first time, the feet and the steps home took two more.	6	9.2	$6
The higher guard: retraining every skill onto a new stance22–24 October 2026 · Eighteen runs that moved every skill onto a higher guard, fixed the lead hook and the jab, and produced guard18-124.5M, the base every clone starts from.	18	24.5	$21.04
Fast stepping and the step-jab (abandoned)22 October 2026 · Twenty-six short GPU runs on fast steps in and out of range and a jab that steps in. The long step never pushed off the back leg, and the work was stopped to redo the motion capture.	25	12.6	$9.49
Total	71	108.1	$81.04

Two of them are not about this post’s techniques but cost money in the same days. The higher guard is the retraining that produced the base every clone started from. Fast stepping is the abandoned attempt mentioned above.

To be continued

This is the first part. The uppercut teachers are retraining with the facing fixed. The evasions start again from my original capture, with the punch clock held to the capture’s pace. The pull counter goes into the next distillation with the uppercuts. Kicks are waiting for better motion capture. Each new technique will be one more clone, and each distillation one more round of bringing the experience back.

Read it on ai.kakuto.io, with the interactive charts and the glossary.

Two fighters and vision: GPU against CPU, optimised

Tony Yum — Fri, 23 Oct 2026 05:08:00 +0000

Watch the video on YouTube

Training on a rented GPU compared nine GPUs against EC2 spot machines on the first skill the fighter learnt: standing in the stance while being shoved. A rented card came out about 4.5 times cheaper. Since then the training has moved on to blocking an opponent, a much heavier job. Tony Yum, the coach:

We’ve already done an analysis on various GPU on stance and balance, but now we have a more complex model with two fighters and vision, so we want to redo the analysis. But just to reduce the scope of the GPU choices, we’ve just chosen RTX 5090 because that’s proven to be the most effective in the last training. But now we’re going to keep optimizing both the CPU usage on EC2 spots versus the RTX GPU and see how far we go, and again test the price and speed comparison between CPU and GPU.

So this post compares one GPU, the RTX 5090 rented on vast.ai, against one CPU machine, a 96-vCPU EC2 spot machine. It measures both as they were, optimises each until there was little left to gain, and compares them again. As before, the models trained here were thrown away.

What makes this training heavier

The stance test trained one body and one network. The blocking training (the environment from the blocking post, now with every skill mixed in) has much more in every step:

Two fighters. Ours, the muscular body, and an opponent who throws recorded punches at it.
Vision. The fighter doesn’t get the opponent’s position for free. It sees his keypoints from its own eyes: 32 rays cast to the points of his body, and any point behind its own gloves is hidden (occlusion), with a tenth of a second of delay.
The punch’s score. When a punch arrives, 19 paths from the glove to the head say how much of it got through the guard.
Everything mixed. Some episodes face the opponent; the rest are blocks on command, walking to a spot, punches at a bag, or recovering from shoves. Each block press fits the arms to a pose with inverse kinematics.
A second network. The walking model from earlier posts runs beside the policy as a teacher, pulling its legs toward a good walk (policy distillation).

Figure 1. The opponent environment, from the RTX 5090 run trained in this post: our fighter in its guard, and the opponent drawn as the keypoints the fighter's vision gets (green: seen; grey: hidden behind its own gloves), throwing a jab, a left hook and an uppercut.

The test

Both machines start from the same model (guard5, the current best blocker) with the same settings as real training, and train for 15 minutes.

The unit of work is one training iteration of PPO: collect about 30,000 steps from many copies of the world at once (parallel environments), then update the network on them, four passes in minibatches of 2,048. The stance test showed that how fast the fighter learns is set by the number of iterations, not the number of steps. So the speed here is seconds per iteration , and the cost dollars per 1,000 iterations.

CPU: a 96-vCPU EC2 spot machine with 72 worker processes, each stepping 6 copies of the world for 64 steps (27,648 steps an iteration), and two learner processes of 12 threads updating the network.
GPU: an RTX 5090 stepping 2,048 copies at once for 16 steps (32,768 steps), all of it, physics and network, on the card with MuJoCo Warp.

	EC2 96 vCPU spot	RTX 5090, vast.ai
price	$1.48/h (c8a), $2.58/h (m8a, the one available)	$0.67/h
seconds per iteration	1.90	2.00
steps per second	14,600	16,300
$ per 1,000 iterations	0.78 (c8a), 1.36 (m8a)	0.37

Unoptimised, the two took the same time, and the GPU was 2.1 times cheaper. On the stance it had been 4.5 times. Two fighters and vision narrowed the gap.

How busy was the CPU?

Tony’s first question once the numbers were in: “How well optimised is the CPU? I.e. what CPU utilisation is it?”

About three quarters. The trainer’s profile splits every iteration in two. The workers collect the next batch while the learners update on the last one (decoupled PPO), so an iteration takes as long as the slower of the two.

The learners’ update took 1.90 s, the whole iteration. They were the bottleneck.
The 72 workers stepped the physics for 1.34 s and then waited 0.56 s for the learners, idle 30 % of the time.

So 24 of the 96 vCPUs were always busy and 72 were busy 70 % of the time: about 77 % in all.

Optimising the GPU

Where the step went. The profile of one step with 2,048 copies counted 13,700 kernels, the small programs the GPU runs. About 1,800 were MuJoCo Warp’s physics, rays and arm fitting. The other 11,700 were tiny arithmetic kernels from the rewards, observations and resets, written line by line in PyTorch. Each did almost no work, and each had a fixed start-up cost. The GPU spent much of the step waiting between them.

Fusing them. torch.compile reads that PyTorch code and merges chains of small operations into a few bigger kernels (kernel fusion). The physics calls were kept out of it and run as before. A step with 2,048 copies went from 79.5 to 60.2 ms (Figure 2).

Figure 2. Time for one environment step of the opponent environment on an RTX 5090, as written (eager) and compiled with torch.compile, for 1,024, 2,048 and 4,096 copies of the world. Compiling saves about 18–20 ms a step whatever the number of copies: the start-up cost of the kernels it merged.

Two things had to be fixed on the way. The compiled code at first came out in pieces, because some table offsets were numpy integers, which the compiler traced as data instead of constants. Then some outputs came out in 64-bit instead of 32-bit (floating point): the project’s settings are numpy numbers like the 4-degree tolerance np.radians(4.0), and the compiler turned them into 64-bit arrays where plain PyTorch treats them as ordinary numbers. The step’s outputs are now converted back to 32-bit. Against the uncompiled step from the same start and the same actions, the rewards matched to 3 in 100,000 for the first steps, and then drifted apart the way any two 32-bit runs of contact physics do, with the same averages.

The update as one recording. The network update was the other 0.53 s of the iteration. Run as written, its hundreds of small kernels are sent one by one by the host’s CPU, and most rented 5090s sit behind slow server processors: the same update took 0.13 s on a box with a fast desktop processor and about 0.5 s on the usual EPYC servers. Recording one minibatch step once and replaying it (a CUDA graph) takes the CPU out: 0.08 s.

That recording already existed but had been switched off, because the first rollout after it crashed the GPU with an illegal memory access, on every vast.ai box. A test with the same pieces didn’t crash, and adding the trainer’s extras back one by one didn’t either. What did work: recording the update before the environment’s own recording, not after. Two runs of 538 and 792 iterations ran clean. The underlying cause is still unknown; most likely the two recordings’ memory.

The result: 1.19 s an iteration , 27,500 steps a second, against 2.00 s. The environment step is now 1.09 s of that and the update 0.08. A 20-minute run on an ordinary EPYC-hosted 5090 matched the baseline’s training figures iteration by iteration: the same returns, falls, punches, goals, blocks and opponent passes. Only the time changed.

Optimising the CPU

The learners. At 12 threads, one learner’s update runs at about 1 TFLOPS, about half the cores’ theoretical peak, which is typical for networks this small. Six ways of splitting the 96 vCPUs between workers and learners were tried: one learner with 24 threads, two with 8, 12 or 16, three with 8 or 12. Most barely moved the iteration. The best was 60 workers with 7 copies each and three learners of 12 threads : 1.91 to 1.74 s an iteration on the same machine, 9 % faster.

More learners would lose. Each extra learner process adds to the all-reduce, the step where the processes add up their gradients so every copy of the network stays the same, and with three it already took 0.38 s an iteration.

Fewer bits. Running the network’s passes in bfloat16, 16-bit numbers that the AMD processor multiplies faster, nearly halved the update: 0.88 to 0.48 s at 12 threads. The iteration went to 1.43 s, 25 % faster than at the start. It changes the numbers slightly, and it hasn’t been checked yet that a policy trained this way learns as well, so it is listed separately below.

The workers. One copy of the environment took 4.4 ms a step, of which MuJoCo itself was about 1.6 ms. The rest was the project’s own Python and numpy around it: numbers clipped one at a time, arrays joined with slow helpers, table lookups. Rewriting the worst of it made one environment about 20 % faster, with every observation and reward identical to the last bit over 6,000 steps. On the 96-vCPU machine, where 72 processes share the memory, it saved 7–11 % of the workers’ time. But the learners still set the pace, so on its own it didn’t shorten the iteration.

The machine. The same work on an Intel c8i machine took 1.45 times as long in the physics as on AMD’s c8a or m8a, which have a whole core per vCPU. The AMD machines are the ones to use.

Keeping the comparison fair

Midway through, the GPU’s training figures changed: about 200 blocks an iteration instead of 1,150, and far fewer punches. It looked like the compiled code was broken. It wasn’t: the same trainer without compiling showed the same drop. That morning, separate work on kicks had widened the range of the fighter’s ankles, and every reference motion had been rebuilt for the new body. With the old ankles shipped to the machine, the blocks came back.

So the final numbers use the same body as the first test, on both machines, and the CPU’s before and after ran back to back on one machine.

Optimised: the comparison

	CPU, 32-bit	CPU, bfloat16 (marked)	RTX 5090
seconds per iteration	1.91 → 1.74	1.43	2.00 → 1.19
steps per second	14,500 → 15,300	17,800	16,300 → 27,500
$ per 1,000 iterations	0.78 → 0.71	0.59	0.37 → 0.22
$ per million steps	0.027	0.023	0.0068
paid set-up before training	about 2 min	about 2 min	about 10 min (~$0.11)

CPU prices at c8a spot ($1.48/h), the cheapest 96-vCPU AMD machine; the measurements are from an m8a with the same processor. GPU at $0.67/h.

Figure 3. Seconds per training iteration and dollars per 1,000 iterations for the opponent environment, before and after optimising, on a 96-vCPU EC2 spot machine (priced at c8a) and an RTX 5090 on vast.ai. CPU bf16 runs the learner's network passes in 16-bit numbers, which changes the learning slightly and hasn't been checked yet.

What it shows:

The RTX 5090 is now about 3.2 times cheaper per iteration than the best 32-bit setup on EC2, and faster : 1.19 s an iteration against 1.74. Against bfloat16 learners on the CPU it is 2.7 times cheaper. Per step, about four times.
The GPU gained more from optimising : 41 % off the iteration against 9 % (25 % with bfloat16). Its waste was in starting work (thousands of tiny kernels, a CPU sending them one by one), and that can be removed. The CPU’s learner was already doing its arithmetic at a fair rate.
The cheapest 5090s should now be as good as any. With the update recorded, the CPU next to the card only replays recordings, so a $0.60–0.67 server box should run as fast as a $0.91 one with a fast desktop processor. The final run, on an old EPYC server, reached 1.19 s.
The GPU costs more to start : about 10 minutes from renting to the first iteration in these runs (booting, rebuilding the reference tables, compiling the physics and then the fused kernels), about 11 cents. For runs of an hour or more it hardly matters.

What Coach Tony thinks

Tony framed this one as a follow-up rather than a new survey: keep the GPU choice to the RTX 5090, “proven to be the most effective in the last training”, and “keep optimizing both the CPU usage on EC2 spots versus the RTX GPU and see how far we go”. When the first comparison came back, his question was about the CPU side, how much of the machine was actually being used, and his instruction afterwards was to keep optimising both “until you think there is pretty well optimised”, then see how cost and speed compare.

What’s next

The GPU’s step. About half of the 1.09 s is MuJoCo Warp’s own physics, the rest the environment’s kernels and a reset that runs on every copy every step. More copies cost little more time (4,096 take 99 ms a step against 60 ms for 2,048), so 4,096 copies for 8 steps would make the same batch in about 0.8 s. That shortens each copy’s run between updates, which changes the learning, so it needs a learning test first.
The start-up. The compiling could be kept between machines instead of redone each time.
bfloat16 on the CPU would need a learning test before it is used, but with the GPU this far ahead, the CPU is now the backup.

What it cost

Most of the money went on short GPU and CPU runs to measure each change, and on chasing the GPU crash. The vast.ai figures come from each machine’s billed time and hourly price; the EC2 figures are estimates from each run’s length and the zone’s spot price.

item	machines	machine-hours	cost
the first comparison: 15 minutes on each	1 RTX 5090, 1 EC2 m8a.24xlarge	about 0.7	about $1.06
GPU optimising: benchmarks, the crash hunt, test and final runs	15 RTX 5090 (two never answered)	about 3.2	about $2.65
CPU optimising: benchmarks and the final comparison	7 EC2 spot (c8i, c7a, c8a, m8a, 96 vCPU)	about 1.4	about $3.00
total	24	about 5.3	about $6.70

Read it on ai.kakuto.io, with the interactive charts and the glossary.

Training on a rented GPU: four and a half times cheaper

Tony Yum — Wed, 21 Oct 2026 07:54:00 +0000

Watch the video on YouTube

Every post so far has trained the fighter on Amazon EC2 spot machines with 96 or 192 vCPUs. Tony Yum, the coach, on where that was going:

As I’m provisioning more and more virtual CPUs on EC2 spot instances, the cost is racking up. And this is only the beginning. There are a lot more training to do, possibly bigger networks. So it’s only going to get more expensive. So let’s think of a way to save money.

GPU is one option, but there are some complications with the number of contacts and the way the body is built. So some experiments are needed. Yes, in order to save costs, I’ll have to throw even more money at an experiment, but it’ll be a good investment, I believe.

So this post isn’t about a new skill. It is about the machine the skills are learnt on: whether the training is cheaper on a rented GPU, what the code had to change to get there, and which GPU is cheapest. The models trained here were thrown away; only the timings and the bills mattered.

It covers one morning, 04:05 to 07:54 UTC on 21 October.

Why a GPU wasn’t the obvious answer

In the blocking post a rented GPU was tried for a few minutes and set aside: training waits on the physics, and the physics ran on the CPU. A GPU only helped the small network.

The way round that is to move the physics too. MuJoCo Warp is a version of MuJoCo that runs on the GPU and simulates thousands of copies of the world at once, all in step. That has two catches for this fighter.

The body. The muscular body has 89 degrees of freedom in one tree, from the pelvis out to every finger joint, and 63 egg-shaped muscles on top. GPU simulators are built for robots with a few dozen joints. The simple body (Figure 1, right) has 43.
The contacts. On a CPU each copy of the world keeps as many contacts as it needs. On the GPU every copy gets a fixed number of slots, set before training starts. After a fall, a muscular body lying on the floor reaches 32 contacts and 260 constraint rows, the equations the physics solves to keep things from passing through each other. Smaller slots overflowed and crashed the run; the slots that work leave most of the memory empty most of the time.

Figure 1. The two bodies the test trained, each after solving the stance with shoves on: muscular-fast (left pair) and simple (right pair), in the stance and 0.1 s into a shove (red arrow; 55 N and 32 N). These are the EC2 runs of the test; the GPU runs solved the same task, but their checkpoints went with the rented machines.

The test

The task was the first one the fighter ever learnt: stand in the stance, keep the rhythm, and stay up while being shoved (the balance post). Each machine trained it from nothing until it was solved: amplitude at full, at most 5 % of episodes ending in a fall, and episodes lasting the full 10 s. Both bodies were tested, muscular-fast and simple.

Two figures were kept for every machine: time to solve , and cost per solve , which is that time times the machine’s price per hour.

Both sides train many copies of the world at once, parallel environments. The CPU side was two ordinary EC2 spot runs with the usual trainer: 72 worker processes with 6 copies each.

machine	body	price	time to solve	cost
c8i.24xlarge spot, 96 vCPU	muscular-fast	$1.45/h	714 s	$0.29
c8a.24xlarge spot, 96 vCPU	simple	$2.02/h	404 s	$0.23

Step 1: tuning the code for one GPU

The first GPU was an RTX 5090, the newest consumer card, rented on vast.ai for $0.74 an hour. vast.ai is a market where people and small data centres rent out their GPU machines by the second.

The GPU trainer was written two days earlier and tested on an older card. On the 5090 it ran 44,000–47,000 steps a second with 4,096 copies of the world, the GPU at 97 %. The physics took nine tenths of the time.

Finding the slow part. Profiling timed every small program the GPU ran (each one a kernel). One took 53 % of the time: the factorisation of the mass matrix, the table that says how hard each joint is to move given everything attached to it. MuJoCo Warp has a fast method for bodies up to 64 degrees of freedom. The muscular body’s 89, all in one tree, fell back to a slow general method: 24 separate launches per factorisation, one for each level of the tree, twice in every physics step.

A replacement. The new version (warp_factor.py) gives each copy of the world a group of 64 GPU threads that do the whole factorisation in one launch, waiting for each other between levels instead of starting a new kernel. Long sums are split into chunks of 8 so no thread is left doing all the work. The factorisation went from 38.6 to 8.9 ms of GPU time per step, and the physics got 28 % faster: 76,000 steps a second at 8,192 copies. It is the same physics: on the same episode with the same actions, the joint angles matched the stock method to a millionth of a radian, as close as 32-bit numbers allow.

The simple body didn’t need this. Its 43 degrees of freedom fit MuJoCo Warp’s fast method.

Then the surprise: fewer copies of the world trained faster. With the new factorisation and 8,192 copies, the GPU trained at 61,000 steps a second, five times the CPU machine. After 12 minutes the fighter had reached only 60 % of the full stance. The CPU machine had solved it in less.

The reason is the curriculum. Each time the stance gets wider, it waits a fixed number of training iterations, about two episodes for every copy of the world, before it can rise again. Each iteration steps every copy 32 times. So a solve takes about 350–500 iterations whatever the machine, and the time to solve is that number times the seconds per iteration. More copies only make each iteration take longer. Training is fastest with as few copies as keep the GPU busy, and on the 5090 that was 1,024 (Figure 2).

Figure 2. Training the muscular-fast body from nothing on the RTX 5090 with different numbers of copies of the world (parallel environments), against the 96-vCPU CPU machine. More copies give more steps a second, but each training iteration takes longer, and the solve needs about the same number of iterations, so 8,192 copies hadn't solved it after 722 s. 1,024 copies solved it in 386 s on this seed; a second seed took 529 s.

At 1,024 copies the GPU is no longer short of work to do; it is waiting. Each of the thousands of small kernels in a physics step has a fixed start-up cost, and with 1,024 copies that cost sets the pace, not the arithmetic. So the next change cut the waiting.

No questions to the CPU in the middle of a step. Code like “did any copy fall?” makes the CPU stop and wait for the GPU’s answer. Falls, resets, shoves and foot forces were rewritten to work on every copy at once with a mask, and the trainer reads its figures from the GPU once an iteration.

Muscular-fast at 1,024 copies stayed at 31,000–32,000 steps a second on the 5090. On that machine the physics was the limit. The waiting mattered on the next machines.

Step 2: nine GPUs

A script (vast_bench.py) rented the cheapest offer of each GPU type at the same time, copied the code over, ran the same speed tests and two minutes of training per body, and handed the machine back. Nine types, from a 2018 data-centre T4 at $0.15 an hour to an RTX PRO 6000 at $1.00.

The first pass was a disappointment. Every machine except the 5090’s took 0.7–1.4 s per iteration to update the network, where the 5090 took 0.2 s. The 5090 had happened to sit in a desktop with a fast new processor. Most rented GPUs sit in servers with slower, older processors, and the trainer was waiting on them. An L40S, a card nearly as fast as the 5090 here, trained at 13,000 steps a second.

So the CPU was taken out of the loop:

The network update as a CUDA graph. One training step on a minibatch (run the network, work out the gradients, clip them, update the weights) is recorded once and replayed as a single unit, instead of the CPU sending each of its hundreds of kernels one at a time. 0.6–0.7 s an iteration became 0.2 s.
The whole environment step as a CUDA graph , the physics included. Only 5 % on its own.
One lookup instead of eight. The reference motion (where each joint should be in the stance’s rhythm) was 8 tables looked up one by one, three times a step: 8 of the 21 ms the step spent outside the physics. Now the tables sit side by side and one lookup reads them all, and the non-physics work fell to 8.7 ms. With that and the contact slots halved for the simple body (the foot-force check reads every slot), the simple body went from 41,000 to 48,000 steps a second on an RTX 3090.

Two bugs turned up on the way. The first step of every run had been simulated twice, once to prepare the GPU code and once for real (harmless, now fixed). And once, a heap of fallen muscular bodies ran out of constraint slots, the second catch above. The slots went up from 320 to 512 rows, at no measurable cost, and the trainer now prints OVERFLOW when any copy runs out.

With the new code, a second pass of the survey (Figure 3) gave cards behind ordinary server processors their real speed. A second test on the two cheapest, on different machines in other countries, gave the same seconds per iteration within a few percent.

Figure 3. Cost and time to solve the stance with the muscular-fast body, per machine, including the paid set-up before training (about 1.5 minutes). GPUs are rented on vast.ai at that morning's offer, with 1,024 copies of the world on the final code; times come from the measured seconds per iteration times 426 iterations. g6.xlarge is EC2's own L4 machine, priced with the L4's speed measured on vast.ai (an estimate). c8a 96 and c8a 192 are the EC2 CPU spot machines; the 192-vCPU time is estimated from its measured speed-up. A single solve varies by about 30 % from seed to seed.

Step 3: GPU against EC2

The full list, cost per solve with set-up included. ”~” marks an estimated time.

machine	where	$/h	muscular-fast	$	simple	$
RTX 5080	vast.ai	0.39	515 s	0.065	230 s	0.034
L40S	vast.ai	0.44	452 s	0.065	227 s	0.038
RTX 3090	vast.ai	0.20	1163 s	0.070	451 s	0.030
RTX 4090	vast.ai	0.51	436 s	0.074	217 s	0.043
RTX 5090 (fastest)	vast.ai	0.69	377 s	0.089	185 s	0.053
T4	vast.ai	0.15	2326 s	0.100	1025 s	0.046
L4	vast.ai	0.36	1074 s	0.117	389 s	0.048
A100	vast.ai	0.46	931 s	0.129	403 s	0.062
g6.xlarge (L4)	EC2 spot	0.48	~1074 s	~0.167	~389 s	~0.076
g6e.xlarge (L40S)	EC2 spot	1.47	~452 s	~0.259	~227 s	~0.167
c8a.24xlarge, 96 vCPU	EC2 spot	1.34	714 s	0.299	404 s	0.184
c8a.48xlarge, 192 vCPU	EC2 spot	3.65	~529 s	~0.627	~299 s	~0.394

What it shows:

A rented consumer GPU solves the stance for less than a quarter of the price : $0.065 on an RTX 5080 or L40S against $0.30 on the 96-vCPU spot machine for muscular-fast, $0.03–0.04 against $0.18 for simple. And it is faster too: the 5080, L40S and 4090 take 55–70 % of the CPU machine’s time, the 5090 about half.
The cheapest cards are all within about 15 % of each other : RTX 5080, L40S, RTX 3090 and RTX 4090. The 3090 is the cheapest by the hour but more than twice as slow. The 5090 is the fastest at about 30 % more per solve, worth it when time matters more than money.
Data-centre cards are poor value here. The A100, L4 and T4 are built for large blocks of matrix maths and lots of memory, but this physics is ordinary 32-bit arithmetic in thousands of small steps. It wants clock speed, not memory or tensor cores.
EC2’s own GPU machines don’t pay. The same L40S costs 3.4 times as much on EC2 as on vast.ai. The best of them, g6.xlarge with an L4, costs about half the CPU machine per solve but is slower than it.
The simple body costs about half as much on a GPU : it runs 1.7 times as many steps a second and needs about the same number of iterations. On the CPU the gap is only 1.3 times.

Figure 4. Dollars per solve for the two bodies on the cheapest GPUs and the 96-vCPU EC2 machine, set-up included.

Getting a machine ready

Speed per iteration isn’t the whole bill. A rented machine is paid for from the moment it starts, before any training. On vast.ai the time spent downloading the software image is free; copying the code over, installing the physics libraries and preparing the GPU code is paid at the full rate.

At first the copy was slow. The script sent every file in the project, 57 MB, 51 MB of it a table only the walking environment uses, and many cheap hosts have slow incoming connections: the A100 in California took 17 minutes to receive it, then 26 minutes to install the libraries over the same line. Now the script sends 1.4 MB and asks for hosts with at least 500 Mbit/s. On a good host the paid set-up is about 1.5 minutes, one or two cents, about what an EC2 CPU machine takes from launch to the first iteration (75–93 s).

The real risk is a bad host. An RTX 4090 in Thailand ran a tenth as fast as another 4090 on both passes. An RTX PRO 6000 started, never answered, and cost $0.51 for nothing. An RTX 5080 had a driver too old for the GPU code and couldn’t see its own card. The script now asks for a recent driver and gives up on any machine that doesn’t answer within 5 minutes.

What Coach Tony thinks

Tony’s brief for the morning was to optimise the GPU code for the machine first, then compare: both bodies, other GPU machines to find the cheapest, and EC2. He called the experiment money spent to save money, “a good investment”. The models could be thrown away.

When the first survey came back, he asked why the uploads were so slow and wanted the set-up costs in the comparison, not just the training time. That is where the 57 MB upload and the bad-host losses came to light.

What’s next

Only the stance runs on the GPU so far. Punching, walking, blocking and the opponent environment each need their own GPU version before they can move off EC2. On these numbers that work pays for itself quickly: the last post’s 20 blocking runs cost about $33 on EC2.

What it cost

Most of the cost was renting GPUs to measure them, plus two CPU runs on EC2 to compare against, each left running a few minutes past its solve. The vast.ai figure comes from vast.ai’s charge records; the EC2 figure is an estimate from each run’s hours and spot price.

item	machine-hours	cost
RTX 5090 on vast.ai: profiling, the new factorisation, the copies-of-the-world tests, the simple body	about 1.7	about $1.21
RTX 3090 on vast.ai: the CPU-out-of-the-loop work and two runs from nothing	about 1.1	about $0.22
the survey: about 17 GPU machines over two passes and a timed re-run	about 5.5 (estimate)	about $2.61
2 EC2 spot runs (c8i.24xlarge and c8a.24xlarge, 96 vCPU)	about 0.8	about $1.38
total	about 9	about $5.40

Read it on ai.kakuto.io, with the interactive charts and the glossary.

Blocking: reading a punch you see a tenth of a second late

Tony Yum — Tue, 20 Oct 2026 17:43:00 +0000

Watch the video on YouTube

By the end of the footwork post, the fighter could stand in its stance, throw five punches and walk to a spot. Every target so far had been a point in the air or a bag. None of them hit back.

This post is about defence. Tony Yum, the coach, explains why it is hard:

Blocking is a complicated thing. Firstly, there’s a reaction time. What you see and process is actually behind reality. And you don’t know what your opponent is actually thinking, whether it’s throwing a jab or a hook. All you have is the visual clues, like the turning of the hip, the movement of the shoulder or elbow. That can also become a feint. So it’s a very interesting chess game.

And what this first step leaves out:

In the real combat fight, you’re thinking about distancing. You wouldn’t allow your opponent to stand in striking distance and just stand there. Typically, you would go in and out of the pocket. You might bob your head to avoid jabs. You might be prepared to move, or you might trade. You might think: that quick jab was a bait, and you’re actually thinking to move out of the way and do a big cross. So all that is in consideration. But one baby step at a time.

So: one punch at a time, from an opponent standing in range, and the fighter only allowed to block. No slipping, no stepping back, no countering. It sees the opponent a tenth of a second late, and it has to tell from his first movements which punch is coming.

It covers 20 training runs over about 18 hours, from 23:43 UTC on 19 October to 18:04 UTC on 20 October. The model that was kept finished training at 17:43 UTC on 20 October.

Four blocks

Tony sent photos of the blocks he wanted, and four poses were built from them (Figure 1):

Left high block , against the opponent’s right hook: the left elbow up in front of the face at forehead height, the forearm back along the temple, chin tucked.
Right high block , the same on the right, against the left hook.
Front block , against the jab and cross: both forearms upright and pressed together in front of the face.
Low front block , against the uppercut: the front block with the elbows down at the chest, so the forearms close the gap under the chin.

The fighter fights bare-fisted. Gloves were tried and dropped the same day: they covered the eyes in the guard, and the fighter will grapple later.

Figure 1. The guard and the four blocks, from the front, as posed (no physics). Left to right: the stance guard, left high, right high, front, low front. Share of punch paths each one stops before they reach the head: left high 79 % of the left side, right high 73 % of the right side, front 95 % of the face straight on, low front 93 % of the chin from below; the guard alone 5–25 %.

Step 1: blocks on command

Option one was to put an opponent in front of the fighter, let it get hit, and wait for it to discover blocking. With only “getting hit is bad” to go on, a reinforcement learning agent is more likely to invent an odd flinch than a Muay Thai block. So training came in two stages. First, blocks on command: a button says which block, and the policy, the neural network that drives the body, learns to raise it fast and hold it without falling over. Then the button goes, an opponent comes in, and the same policy decides which block and when.

While the button is held, the blocking arm’s motor targets switch to the block’s pose. The rewards pay for speed (a bonus for reaching 90 % of the pose’s cover, in full within 0.15 s), for holding still, for keeping the elbow bent on the way up (Tony’s rule against a wide, telegraphed swing), and for a smooth return to the guard. The legs and trunk keep their stance rewards.

The fighter must not forget what it already knows. Tony’s rule: “make sure that none of the previous techniques are ever forgotten. So it still knows how to balance and it still knows how to do all the punches and walk.” Half the training episodes are blocks; the other half are the walking environment’s exactly, with walking and punches. A frozen copy of the walking model acts as a teacher (policy distillation) whenever no block is held. And no model is kept until it passes a skills gate: at least 58 of 120 test punches, at least 39 of 40 walking targets with no falls, in no more than about 8.3 s each.

Three of the four blocks came at once. The left high block never covered, in 15 M steps. Three runs found out why:

The poses were worked out on the reference stance, but the trained fighter stands with its chest turned about 8° further and leaning 7° further back. With every joint exactly on the pose’s angle on that trunk, the block covered only 82 % of its design. So at each press, the arm is now fitted to where the head actually is, with inverse kinematics.
That alone didn’t move it. The policy adds its own small offsets to every motor target, and its guard habit held the left shoulder about 4° low and 8° in: the arm stayed 3.9 cm off the pose. A reward for pulling the elbow and wrist to the pose didn’t break the habit in 10 M steps either.
What worked was turning the habit down: while a block is up, the policy’s offsets on the blocking arm count only 30 %. In 4.6 M steps all four blocks covered, and every old skill passed the gate.

Then the high blocks were too slow: 0.26 s for the left, 0.21 s for the right. The fighter’s left shoulder swings from 53° to 129° in that block. Each joint is driven by a position servo, and its damping, the part that resists speed, cancelled the pull 20 ms into the swing. No amount of training could fix that. Softening the damping to a quarter while a high block rises did: 0.12 s for the left, 0.08 s for the right, 0.10 s for the front and 0.07 s for the low front block, every press covered, with punches and walking still passing.

Is there time to block?

Before the opponent came in, one question needed numbers: can a block arrive in time at all?

The opponent is the fighter itself. Its own fast punches (jab, cross, both hooks and the uppercut) were recorded once and are replayed, without physics, from in front of it. From those recordings: when is the first visible sign of each punch, and when does the fist land?

The fighter doesn’t see him as an image. It sees 16 keypoints (head, neck, shoulders, elbows, wrists, fists, hips, knees, ankles), each as a 3D vector from its own head, with velocities and a short history of 40 and 80 ms back (frame stacking). It sees them late. Tony chose a vision delay of 100 ms, about a fast professional’s. And it can be blind. Rays go from both eyes to each keypoint, and a point is hidden if the fighter’s own arms are in the way (occlusion).

punch	first moves	lands	warning	cover at landing*
jab	elbow and fist, 0.083 s	0.187 s	0.10 s	0.50 (guard: 0.48)
cross	elbow, 0.079 s	0.220 s	0.14 s	0.68
left hook	fist, 0.110 s	0.342 s	0.23 s	0.82
right hook	elbow, 0.106 s	0.343 s	0.24 s	0.69
uppercut	fist and elbow, 0.081 s	0.235 s	0.15 s	0.79

* The share of the block’s cover in place when the fist lands, reacting at the first movement plus 100 ms. Times are from the opponent’s decision to punch, which the fighter can’t see. With 100 ms of delay, the jab’s 0.10 s of warning is used up before the fighter even knows it has started. Reacting to what it sees, the fighter can’t add anything to its guard against a full-speed jab. Against the hooks there is time for most of a block.

One surprise: in every punch the hips move 20–60 ms after the arm. Tony reads the hip turn; in these punches, the first sign is the elbow or the fist.

From that table Tony set a pass mark per punch, the share of the punch that must be stopped at full speed: jab 0.50 (keep the guard: beating the jab needs anticipation or distance), cross 0.65, hooks 0.70, uppercut 0.75. Damage counts partially: the share of the fist’s path that reaches the head or neck before it meets an arm.

Step 2: the opponent

Now the button went, and the policy got four new outputs of its own: one per block. Punches come one after another, with half a second to two seconds of the opponent moving in his stance in between. Tony calls these slow combos: “it’s one shot after another, so it’s not a real true combo.” The opponent’s speed rises with a curriculum: punches start at 50–60 % speed, and the top speed goes up as the fighter passes.

Getting the fighter to block for itself at all took three runs. Its exploration (random noise on its outputs) flipped a button up and down within a step or two, so a random press dropped the block before the arm moved. Presses now hold their block for at least 0.25 s. Then a smoothness term in training (CAPS) was flattening the buttons, since a press is exactly a sudden jump in an output. The buttons were freed from it. In the third run, the fighter blocked.

It raised the low front block against everything, and it raised it before the opponent punched.

The giveaways

The fighter wasn’t reading punches. It had found a cheaper cue (shortcut learning), and it would do so four times:

He came alive before each punch. Between punches the opponent stood frozen, and each recorded punch began 0.18 s before the press with his stance’s movement. So he twitched just before every punch. The opponent was re-recorded with 2 s of stance motion before each punch, so he never stands still.
Each punch had its own distance. Tony spotted this one: the hooks stood closer. Head to head, the jab and cross came from 0.78 m, the right hook 0.62 m, the left hook 0.52 m, the uppercut 0.46 m. Telling the distance is easier than telling the punch.
The overlap wasn’t enough. Varying each punch’s distance only partly hid it. One distance range (0.57–0.73 m) for every punch fixed it.
The uppercut stood him lower. Each recording had its target at its own height on his body, so lining the uppercut up with the fighter’s head stood him 4–5 cm lower. His head is now at one height for every punch.

Two other things went wrong in between, neither of them the fighter’s fault. The network scales its inputs using running averages of what it has seen (observation normalisation). Those averages share one sample count for every input, and by then it was 1.2 billion. The opponent’s inputs, added later, kept their starting values (mean 0, spread 1) when their real spread was 0.29, some 0.016. For five runs the fighter had been all but blind to the opponent. Its buttons pulsed every 0.75 s on its own rhythm, whatever he did. The averages were re-estimated, and still nothing changed: a reward for pressing the suitable button (a dense shaping reward) was too weak a signal.

What worked was telling it directly. The opponent is scripted, so the training knows which punch is coming. An auxiliary loss pulls the right button’s output up while a punch can be seen, and the others down. The damage still decides whether blocking pays. Within 5 M steps, the fighter raised different blocks for different punches, and right away it found giveaways 3 and 4.

With all four gone, in the ninth run against the opponent, the fighter waited. No block before he punched. Then the front block against the jab and cross, the right high block against the left hook, the left high block against the right hook and the low front block against the uppercut, with the high blocks rising 0.24–0.28 s into his punch. The check that proves it reads the punch: move his punch later, and the fighter’s block moves with it.

Figure 2. The kept model at the moment each punch is scored, full speed, side view. The opponent is drawn as his 16 keypoints: green where the fighter can see them, grey where its own arms hide them. Left to right: jab (low front block, 32 % of the punch through, passes), cross (front block, 32 %, passes), left hook (right high block, 0 %), right hook (left high block, raised 0.26 s into the punch, 79 % through: fails), uppercut (front block, 0 %).

Full speed

Reading the punch was one thing. Doing it at full speed was another. The curriculum only raised the speed once 80 % of punches at the top speed passed, and it never got there: for 23 M steps nothing faster than 60 % was trained. Lowering the bar to 60 % moved it, and full speed reached 21 of 30 test throws. Then it stuck at 13–19 for the next three runs, including one where a quarter of all punches came at full speed.

What was left was the jab, which the time budget says can’t be read in time, and the right hook. The fighter’s left high block came about 0.1 s after the oracle’s, a script that presses the right block at the opponent’s first movement plus the 100 ms delay. Tony’s first idea was a faster left block. Measured first, it wasn’t slow: softer damping and more torque barely changed the damage. The cause was the timing of its guess. Before the punch can be told apart (about 0.22 s in), the right high block’s button hovered half up, a guess at a left hook. In 3 of 12 right hooks it pressed that guess just before the left button rose, and the 0.25 s hold kept the wrong block up through the hook. So a block may now be swapped for another during its hold; only dropping it waits. Without any training, that alone took the right hooks passed from 3 of 12 to 5.

The last run asked which block actually works at each distance. Holding every block up early against every punch at five distances showed that the rule “one block per punch” was wrong in two places. Against the uppercut, the front block lets only 7–12 % through at 0.59–0.65 m, where the low front block lets 43–68 % through. Against a jab from 0.68 m or more, the left high block stops it better than the front block. The rewards now teach the measured best block at each distance. At 14.5 M steps full speed reached 24 of 30, the best yet, with the jab 6 of 6 and the uppercut 6 of 6.

Figure 3. The opponent runs from the first without giveaways (opp4) to the kept model (opp15), each at its best test. Top: punches passed of 30 at full speed (5 punches × 6 throws). Bottom: passed of 120 (the same at 50, 70, 90 and 100 % speed). Six throws per punch make the scores noisy by a few points. opp7 and opp8 were stopped before their tests.

Where it stands

The kept model, on the skills gate’s tests (8 throws per punch at each speed):

Against the opponent: 117 of 160 passed, 31 of 40 at full speed, with no falls. The gated model before it scored 78 of 160.
Blocks on command still work: all 32 presses covered (left high 0.12 s, right high 0.08 s, front 0.11 s, low front 0.07 s), also with shoves.
Old skills kept: 61 of 120 punches, 40 of 40 walking targets with no falls. But walking got slower: 8.45 s per target, just over the 8.3 s mark. The model before it, which passes every mark, is kept as a fallback.

What it can’t do yet:

The far right hook. Held up from before the punch, the left high block still lets about 0.44 of a right hook from 0.73 m through. Those hooks start almost straight ahead and slip between the elbow and the fist onto the temple. A search for a better pose within Tony’s shape found almost nothing (0.77 to 0.79 of the paths stopped): the elbow is at the end of its reach. It needs a different shape, which is Tony’s call.
The mid-range uppercut. The front block at 0.61–0.65 m is only half learnt.
The jab, mostly. At training distances, 7 of 12 jabs pass.

What Coach Tony thinks

Tony’s most interesting point is about the front block:

One very interesting point is that if the agent always holds a front block, like the forearm blocking the face, he actually gets penalised, not by the reinforcement learning punishment, but rather by the fact that he won’t be able to see if the hands are in front of the face.

That is how the fighter’s vision was built, and it was a deliberate choice. With both forearms in front of its face, the fighter sees none of the 13 points tested on an opponent ahead (Figure 1’s fourth pose): no fists, no elbows, no shoulders. The stance guard sees 10, the low front block all 13 between its fists. There is no reward term for being blind. The cost is meant to be what it costs a real fighter: the next punch comes, and you didn’t see which one. A “can’t see” penalty would first have taught the fighter not to block at all. (There is one small cost for holding any block while no punch is coming, so that it doesn’t keep one up all the time.)

He also sees how far this is from a fight. The punches come one at a time, from an opponent who stands in range and never feints. In his own thinking out loud during the runs: stay out of jab range so the opponent has to step in, and the step is the tell; move the head in the rhythm so a fast jab can miss; learn an opponent’s habits, his rhythm, his favourite combinations. None of that is in this model. But it now sees an opponent, late and partly blind, and picks a block from what he does, not from where he stands.

The path, in numbers

Figure 4. The 20 runs on one clock (UTC, 19–20 October). Filled bars are runs whose model passed the skills gate and was kept. block1–5 are blocks on command; opp1–15 are against the opponent. The gap at 03:30–05:20 is the punch recording and the time-budget table.

run	what changed	outcome
block1	blocks on command; half the episodes walking and punching; the walker as teacher	three blocks work; the left high block never covers
block2	the arm fitted to the head at each press	left still 5–13 % covered
block3	a reward pulling the elbow and wrist to the pose	the guard habit stays
block4	the policy’s arm offsets at 30 % during a block	all four covered, every skill kept
block5	soft damping while a high block rises	left 0.12 s (from 0.26), right 0.08 s
opp1	the opponent, keypoints, 100 ms delay, real blindness; the fighter’s own buttons	no block ever kept up
opp2	presses hold for 0.25 s	the smoothness term flattens the buttons
opp3	buttons free of the smoothness term	blocks for itself, but from giveaways 1 and 2
opp4	he never stands still; overlapping distances	giveaways gone; guard only
opp5	a reward for the suitable button	no change: found the input-scaling bug
opp6	the opponent inputs rescaled	no change
opp7	an auxiliary loss on the buttons	a block per punch, read from his distance
opp8	one distance range	the right block per punch, the uppercut read from his height
opp9	one head height	reads the punch ; never trained past 60 % speed
opp10	the curriculum’s bar 80 % → 60 %	full speed 21/30
opp11	a bigger training batch (a speed test)	the same learning; then a trainer bug, discarded
opp12	opp11 again from its last clean model	full speed flat, 13–15/30
opp13	a quarter of the punches at full speed	full speed flat, 15–17/30
opp14	a block may be swapped during its hold	19/30; kept
opp15	the rewards teach the block that works at each distance	24/30, 93/120: kept

Making training faster and cheaper

The blocking runs were also a day of tuning the training machines. A training run has two sides working at the same time (decoupled PPO). Worker processes simulate the fighter in hundreds of copies of the environment and collect experience. Learner processes use the last batch of experience to update the network. Whichever side is slower sets the pace, and the other side’s processors wait.

The muscular body was wasting time on its own fists. Profiling a punching run counted 38 contacts per physics step, and 31.6 of them were the body touching itself, almost all fingers on fingers inside the fists. The fingers aren’t driven by the policy: motors hold them in a fist, in a pose that overlaps the finger shapes slightly, so the physics resolved about 25 contacts every step for nothing. Turning off contacts between the parts of each hand took that to 12.5 contacts per step and made a physics step 1.8 times faster (742 to 412 µs). The punches still land as before (fist speed 4.06 against 4.12 m/s). This “muscular-fast” body is the default for every run from the fourth opponent run on. It can’t be used for grappling later, when the hand has to open and grip.

A GPU doesn’t help yet. A rented GPU (a GTX 1080 for about $0.06 an hour, on vast.ai) rendered the fighter at 1920×1080 in 0.4 s, so it is useful for videos. But training waits on the physics, which runs on the CPU, not on the small network. Moving the physics to the GPU would mean rewriting every environment, and the muscular body’s 63 egg-shaped muscles are a bad fit for GPU simulators. vast.ai’s large CPU machines were no cheaper per step than the c8a spot machines on EC2.

Bigger machines, then faster code. In the morning the spot quota was raised to 192 vCPUs, and the ninth opponent run moved to a c8a.48xlarge: 192 cores for $3.83 an hour. It ran 1.4 times the steps of the 96-vCPU machine for 2.4 times the price. About 70 % of the time its workers were idle, waiting for the learners. Then the code changed, one thing at a time:

Shared memory. The network’s weights and the collected experience had been sent between processes through pipes, copied and repacked by one core. Now every process reads them from the same memory. 44 → 50 M steps an hour.
Four learners instead of two , with 144 workers. 60 M an hour. A test of seven layouts found all of them limited by the learners: more learners do the maths faster, but spend as much extra time agreeing on each update.
A training batch twice the size. Fewer of those agreements per update. It changes how the network learns, so Tony agreed to test it on its own first: 8 % faster, and the fighter learnt about as well per step.
Each learner prepares its own part of the batch , instead of one core preparing it all and sending it out. 70 M an hour, 59 % faster than the morning on the same machine. The first version of this had a bug: three learners prepared their part from a buffer the workers were already refilling, so three quarters of each batch was wrong. 9 M steps of training were thrown away, and every change to the trainer is now checked on its first update.

Which machine is cheapest? Faster isn’t cheaper. With the new code, a 96-vCPU c8a.24xlarge ran 50.5 M steps an hour with its workers 85 % busy, a balanced machine. At $1.87 an hour, that is $0.037 per million steps. The 192-vCPU c8a was 1.35 times faster but $0.055 per million, because half its cores still waited for the learners. An m8a with 192 cores did 64 M an hour, nearly as fast as the c8a. The c8i machines split each core into two hardware threads, so a “vCPU” there is half a core: a c8i simulated about 1.45 times slower than a c8a with the same vCPU count. Tony chose cost over speed, and the last two runs went back to 96 vCPUs. Capacity mattered too: the 192-vCPU machines were taken back three times in two hours that afternoon.

Figure 5. Training speed and cost on the opponent environment, 20 October, in the order things were tried. 192: c8a.48xlarge at $3.83 an hour; m8a: m8a.48xlarge (192 vCPU) at $3.33; 96: c8a.24xlarge at $1.87; c8i: c8i.24xlarge (96 vCPU, two threads per core) at about $1.44. Cost per million steps is the spot price divided by the speed. The 50 and 60 M figures are from a benchmark; the rest from live runs. The c8i figure is from a later run with the batch back at its normal size.

What it cost

Most training runs used a c8a.24xlarge on Amazon EC2, a machine with 96 vCPUs, rented as a spot instance in us-east-1 at about $1.4–1.9 an hour. When it had no spare capacity, the launch script fell back to the same size from c8i or c7a. Five runs in the middle used the 192-vCPU machine (above). Amazon took the machine back 13 times. Each run saves a checkpoint often, so it carried on where it stopped on a new machine. In the late afternoon spot capacity ran out altogether, and the last two runs finished on on-demand machines at about $4.15 an hour.

item	machine-hours	cost
20 training runs	about 13	about $30
9 skills gates, punch recordings, the block-by-distance table and a baseline test, on smaller machines	about 3.5	about $2
one speed test of the training layout on 192 vCPUs	about 0.2	about $0.80
total	about 17	about $33

These are estimates from each run’s start and stop times and the price at the time, not from the bill.

What’s next

The fighter can see an opponent and block single punches it can read. The next things it needs are the ones Tony listed. A second, faster punch after the first, so that covering the eyes costs something. Feints. A memory longer than a tenth of a second, to learn an opponent’s rhythm and habits. And above all, distance: stepping out of range, so the opponent has to step in, and the step is the tell.

Read it on ai.kakuto.io, with the interactive charts and the glossary.

Footwork: walking to a spot in a Muay Thai stance

Tony Yum — Mon, 19 Oct 2026 15:44:00 +0000

Watch the video on YouTube

By the end of the faster punches post, the fighter could throw five punches from its guard, quickly and on command. But it threw them all from the same spot. A fighter who can’t move can’t choose when to be in range.

Tony Yum, the coach, puts it like this:

Controlling the distance is one of the most important facets of striking martial arts. You want to be close enough that you are still engaged, but not close enough that the opponent can hit you. Every striking technique has slightly different range, and your opponent has different range. And so whoever understands the distance best has a huge advantage.

This post is about the first step towards that: footwork. Not the sudden step in to land a punch, or the quick step out to avoid one. That comes later. This is the slow movement at the start of a round, when two fighters are sussing each other out: moving around in a good Muay Thai stance, forward, back, sideways and diagonally, and pivoting to turn.

It covers 22 training runs over 25 hours, from 17:34 UTC on 18 October to the evening of 19 October. The model that was kept finished training at 15:44 UTC on 19 October.

The task: a spot and a facing

The fighter is given a target: a spot on the floor for the ball of its lead foot, and a direction to face. It has to get there, in its stance, and settle. The spot can be anywhere: ahead, behind, to the side, or behind it with an order to turn and face it.

As with the punches, the movement starts as a reference motion, an animation with no physics in it. This one was built from a 28-second video of Tony walking in his stance: forward, backward, a pivot to face the camera, forward and back again, and pivots back. From the video came a library of single steps: 16 directions around the body, each at any size up to a full step (26 cm forward, 22 cm back, 20 cm to the side), plus pivots to turn left and right. Going forward, the lead foot steps first and lands on its ball; the rear foot follows, low. Going back, the rear foot steps first. Each step is timed to one beat of the fighter’s rhythm, the gentle weight shift it makes in its stance, so the walk keeps the same pulse as everything else it does. The foot and hip positions are scripted, and inverse kinematics works out the joint angles.

Something has to decide which step to take next. That is a simple planner, like the ones that move characters in a video game. At the start of each beat, it looks at where the target is. If the facing is more than 30° off, it pivots. Otherwise it steps straight towards the spot, in whatever direction that is, sized to land on it. The policy, the neural network that drives the body, has to turn each planned step into a real one, with real muscles and gravity, while the fighter is being shoved now and then.

And it must not forget its punches. A frozen copy of the best punching model, the one from the faster punches post, acts as a teacher (policy distillation): whenever the fighter isn’t walking, the new policy is pulled towards the teacher’s actions. Walking is free to change; the punches should not.

Lesson 1: a pivot can’t twist a loaded foot

Each run continued from the model before it (fine-tuning), starting from the punching model. The first run learnt to step, but in 8.8 M steps of training it never once turned the body in a pivot. Eight pivots in a row, the hips turned 0° of 45°.

The reference had spun the lead foot on its ball while the weight was on it. Real feet can do that on a canvas, but the simulated foot’s contact with the floor doesn’t slip when twisted. And the lead hip can’t rotate 45° over a planted foot anyway. The reference was asking for something the body couldn’t do.

The second run used a different pivot: all the weight onto the flat lead foot, the rear foot lifts and swings round while the hips turn over the planted foot, the rear foot lands, and then the lead foot lifts a little and re-aims. 30° per beat. Within 6 M steps, pivots turned 21–23° each. But now the steps had stopped working: the lead foot hardly moved. The rewards for walking were simply too small. Getting 26 cm nearer the target paid about 0.5 per beat, against about 50 per beat for holding a good stance. The third run made the walking rewards several times stronger, and the steps came back, slowly.

Lesson 2: tell the muscles in advance

Steps were landing, but short: a forward step covered 4 % of the planned distance, a sideways one under 20 %. Two changes fixed most of it.

Each joint is driven by a position servo, a motor that pulls the joint towards a target angle. A servo lags behind a target that keeps moving. So the walk’s servo targets now lead the reference by that lag, a kind of feed-forward: the muscles are told where the leg will need to be, not where it is now. And 30 % of training episodes now started partway through a step, in the reference’s pose and speed (reference state initialisation), so the policy practised the middle of a step and not only its start.

One run later, steps covered 92 % of the plan going back, 58 % forward and 56 % sideways. Six of eight test targets were reached. The two misses circled round the spot, 10–15 cm away, taking small steps that overshot. The planner was changed so that a spot, once reached, stays reached. From then on, every test target was reached, 8 out of 8, with no falls.

Lesson 3: don’t forget the punches

By then the fighter had walked for about 100 M steps, and the punch test said it had lost something. The punching model scored 38 of 60 presses on its own test; the walker scored 20. The teacher’s pull (weight 2) had let the punches drift.

Raising the teacher’s weight to 10 didn’t help, and the reason was a bug. A neural network sees its inputs scaled to a standard range, using running averages of what it has seen so far (observation normalisation). The teacher was being fed inputs scaled with the student’s averages, which had drifted as walking data came in. The teacher’s advice was off by about 0.007 per motor, sometimes 0.025. That is small, but it was the same size as the differences that made the punches miss. Pulling harder towards slightly wrong advice can’t fix anything. With the teacher on its own scaling, the punches came back.

The test itself also had a flaw. Each test pressed every punch six times, at six moments in the fighter’s rhythm. But the walking environment drew its random numbers in a different order, so its six presses fell at different moments, and six presses swing the score by a lot. The punching model itself scored 19 of 60 in the walking environment, against 38 in its own. With the moments fixed and 20 presses per punch, the punching model scored 112 of 200, and the walker after the fix 107. The loss before the fix had been real, but about 10 points, not 30.

Lesson 4: steer with the steps, not with the rewards

The walk reached every target, but slowly: about 7.5 s per target. Measuring each step showed where the time went. The hips drifted 3–10° per step, always the same way for a given direction, until the facing was so far off that a pivot was needed, and pivots moved the lead foot 5–13 cm.

The first fix asked the fighter to steer up to 10° per step through the rewards alone. It didn’t learn it, and got slower: the reference’s joint angles were still those of a straight step, so the rewards for matching the pose held the hips straight. The second fix put the steering into the reference: 32 new steps, each a normal step that also turns 10° left or right while the foot is in the air. That worked at once. Forward steps reached their full length (27–28 cm, against 17–22), and a target took 6.7 s instead of 7.4.

Tony asked for one more thing here: make sure training includes turning all the way round, to face a target behind. It already did (about 11 % of training targets needed a turn of more than 150°), but turning round now took the longest: five or six pivots of about 33°. So the left pivot was made to turn 60°, the hips turning half over the planted lead foot and half over the rear foot once it lands. The right pivot had to stay at 30°: turning right, the rear hip runs out of rotation as soon as the foot swings past the hips. The planner turns the way that needs fewer pivots. A 180° turn on the spot went from 5.9 s to 3.5 s (Figure 2).

Figure 1. The kept model walking forward 1.2 m to the green spot (green line: the facing), seen from the side, at 0, 1, 2, 3, 4 and 5.4 s after the target is set. It arrives at 5.4 s, about 10 cm from the spot. No shoves and no exploration noise; with shoves this target takes 7.6 s on average.

Figure 2. Turning 180° on the spot, at 0, 0.72, 1.44, 2.16, 2.88 and 3.76 s. Three 60° left pivots: the weight goes onto the lead foot, the rear foot swings round in the air, and the hips turn half over each foot. Done at 3.76 s, facing the other way (green line).

Lesson 5: Tony’s eye for style

At this point, the walk worked. It reached all 40 test targets (ten kinds of target, four times each, with shoves) in 6.09 s on average, landing 5.4 cm from the spot, and the punches held. Then Tony watched it as a coach.

After each step it goes back to Muay Thai stance … make the walking stay in walking mode and not flip between stance and walking.

He was right. Every step in the reference started and ended at rest in the stance, and about a quarter of each step had no foot moving. Each step was split into four versions: moving off, flowing through, arriving, and the single step. Walking on, the fighter now passed through the stance instead of stopping in it. The hips still slowed to about a quarter of walking speed at each stance, so the next run paid for the hips’ speed over the floor. That didn’t move it either. Tony saw why first:

After every step it goes into the stance again … almost like there’s 70% weight on the back leg and 30 at the front … ideally, there should be no stance at all, just from step to step.

The reference itself returned to the stance’s weight, 69 % on the rear foot, between every two steps. So the stance was taken out of the middle of a walk: the weight at 50/50 between flowing steps. Then came more of Tony’s notes, one at a time:

“The back leg often drags and that even sometimes trips up the model.” The feet were lifted higher, with a reward for clearing the floor and a cost for scraping it.
“First the heel lift, but then the toes should follow so that in mid-air the foot is flat rather than having the toes hanging down.” The rear foot now levels off in the air. The ankle can’t bend far enough to hold it fully flat behind the body, so it was lifted higher to get close.
“I still see the heavy back foot weight … and the front leg with the heel off the floor.” The lead heel now stays down between steps.
“The target destination of the walk doesn’t need to be so accurate … the punch itself should be able to adjust a little bit of distance.” A target now counts as reached within 15 cm and 15°, not 8 cm and 8°.

Each change made the walk look more like Tony’s. And each one made it harder. The higher rear foot made every step a balancing act, and the policy answered by taking shorter steps. Then:

It looks like he’s limping … he tilts almost to the front leg, his body even tilts to the left.

I still see a lot of limping, like it’s bowing diagonally forward, so the torso is bending.

One of the new rewards paid for the right share of weight on each foot. The fighter’s way of loading the front foot was to lean over it: a bow, 8.2° beyond the stance’s own lean on average and up to 16°. That is a textbook case of reward hacking: it did what the numbers asked, not what was meant. The next run removed that reward and added one for keeping the trunk upright. The bow went (4° on average), but the fighter had been using it to get its weight across. Without it, it couldn’t step left:

The diagonal step is completely wrong. It keeps stepping forward and back. It doesn’t actually move forward.

Steps to the left now covered 9–15 cm of the planned 20, with 90 % or more of the weight left on the rear foot. The fighter stalled about 28 cm from the spot, just outside the 25 cm within which the planner would have called it close enough, and shuffled its lead foot back and forth. One more reward, for the hips following the reference’s path, made it worse: 17 of 40 targets.

Figure 3. Bowing and upright, both walking forward to the same target, seen from the front, each at its moment of greatest trunk lean. Left: the 'limp' model, leaning 11.6° beyond its stance over its front leg. Right: the kept model, 3.1°.

Function first

Tony made the call:

Okay let’s remove all the requirement for the looks. Walking to the target is the most important. Except it should not bow.

The next run went back to the best walker, the one from before the style work (40 of 40, 6.09 s per target), with its original reference. Every look reward came off: hip speed, foot height, scraping, the weight share. Two things stayed: the reward for keeping the trunk upright, and the loose 15 cm arrival.

At 20.7 M steps it reached all 40 targets, with no falls, in 6.70 s on average. Its forward bow averaged 2.8° beyond the stance, at most 6.9°. Its punches scored 60 of 120 on the 12-press test, the same as every walking run since the teacher fix. That is the model this post keeps.

Figure 4. The style series, every run tested on the same 40 targets (10 kinds × 4, with shoves). Top: targets reached. Bottom: mean seconds per target reached (the dashed line is the 7 s bar Tony set for the final model). 'walks on' is the model before the style work, and 'function first' went back to it. 'hips' reached only 17 targets, so its time covers only the easy ones.

A last run tried to fix the weak spot: steps sideways. The upright reward had counted any lean as a bow, including the small sideways lean a sideways step needs. So that run allowed sideways lean and banned only forward lean. Its steps got shorter everywhere, and it reached 38 of 40. The sideways lean wasn’t what held the steps back.

What Coach Tony thinks

Tony’s own summary:

It turns out it’s very difficult to learn how to walk stylistically. It went from lifting the leg at every step, like Muay Thai stance, to limping, putting too much weight on the front leg, dragging the back feet. But the more constraints [I] added on stylistically, the worse the walking becomes. So there’s a balance between style and actual practical movements.

That is what the numbers show too (Figure 4). Every one of his notes was a fair coaching point, and the fighter did change in the way he asked. But every new rule was one more thing to satisfy at once, on a body that was already busy staying upright under shoves and keeping its punches. Each rule cost a little speed or a little reach, and two of them broke the walk.

For now, function wins. The fighter gets to the spot, upright, ready to punch. Tony also pointed out that the arrival doesn’t need to be exact: the punch itself can adjust a little for distance and direction. The style can come back later, one rule at a time, on a walk that works.

Where it stands

The kept model, tested on the 40 targets with shoves on:

Every target reached, no falls: 6.70 s per target on average, landing 7.8–13 cm from the spot (counted as arrived within 15 cm).
Turning round is quick: 180° on the spot in about 3.7 s; a target 1 m behind, turn and face it, in about 8.5 s.
Upright: forward or backward lean beyond the stance 2.8° on average, never over 6.9°, against the limping model’s 8.2° average and 16.3° peak.
Punches kept: 60 of 120, where the punching model it started from scores about 56 %.

The weak spot is stepping sideways. A sideways step covers 10–11 cm of the planned 20, so it takes more of them: 7.3–8.3 s for a target 0.8 m to the side, about as long as 1.2 m forward (7.6 s). It is also still the walk Tony criticised: it drops back towards the stance between steps, and the trailing foot swings low, heel up.

The path, in numbers

Figure 5. The 22 runs on one clock (UTC, 18–19 October). Filled bars are runs whose model was kept (frozen). The gap overnight is a pause between runs. 'walk19' was frozen but not used; 'walk22' was frozen but did worse than 'walk21', the model kept.

run	what changed	outcome
walk1	walk to a spot and a facing; the punching model as teacher	steps work; pivots turn 0°: the reference twisted a loaded foot
walk2	pivot over a planted lead foot, 30° per beat	pivots learnt; steps stop
walk3	walking rewards several times stronger	steps start landing, slowly
walk4	servo feed-forward; 30 % of episodes start mid-step	steps reach 56–92 % of plan; 6/8 targets
walk5	a reached spot stays reached (no circling)	8/8 targets, no falls; punches drifting
walk6	lower learning rate, tighter landing	8/8 in 9–11 s, 5–8 cm
walk7	teacher weight 2 → 10	stopped: the teacher used the wrong input scaling
walk8	teacher on its own scaling	punches back (107/200 against 112); 7.4 s per target
walk9	steer through the rewards	not learnt, slower
walk10	steering steps in the reference	6.7 s per target, full-length steps
walk11	60° left pivots	180° turn in 3.5 s; 6.3 s per target
walk12	walk on through the stance	40/40, 6.09 s; hips still slow in the stance
walk13	reward the hips’ speed	no change: the reference still stopped in the stance
walk14	no stance between steps; feet lifted, scraping costs	flowing walk; feet still scrape
walk15	flat rear foot in the air, lifted 7 cm	shorter steps, 7.1 s
walk16	lead heel down between steps	7.4 s; sits back on the rear foot
walk17	arrival within 15 cm; weight-share reward	6.7 s, but limps and bows
walk18	trunk-upright reward; weight share off	no bow; can’t step left, 32/40
walk19	reward the hips’ path	worse: 17/40
walk20	the old sideways steps back	stopped after 5 minutes for walk21
walk21	walk12’s walk, no look rewards except upright	40/40, 6.70 s, no bow: kept
walk22	allow sideways lean	shorter steps, 38/40

What it cost

Every training run used one c8a.24xlarge on Amazon EC2, a machine with 96 vCPUs, rented as a spot instance in us-east-1 at $1.48–1.80 an hour. When that type had no spare capacity, the launch script fell back to the same size from the same generation (c8i), and once to the generation before (c7a). It ran 33–38 M simulation steps an hour. Amazon took the machine back mid-run five times. Every run saves a checkpoint once a minute, so each time the run carried on from where it stopped on a new machine, losing a few minutes.

item	machine-hours	cost
22 training runs	about 19	about $29
about 30 test and table-building jobs on smaller machines	about 5	about $2
two speed tests of the training layout (one hung for 2 h)	about 2.4	about $3.50
total	about 26	about $35

The tests ran on separate, smaller machines, so the training machine never stopped to evaluate. One of the speed tests found that splitting the training work in two ran 4 % faster, which every run since has used. These are estimates from each run’s start and stop times and the spot price at the time, not from the bill.

What’s next

The walk gets the fighter where it needs to be. It doesn’t yet look like Tony walking, and it is slow sideways. Two ideas not yet tried: plan shorter sideways steps, so the plan matches what the fighter actually does, or allow more steps per target.

The next skill is already building on this model. Blocking training started from it, with the walk, the stance and the punches all held by the same kind of teacher, so the fighter can move, punch and defend in one body.

Read it on ai.kakuto.io, with the interactive charts and the glossary.

Faster punches: from trained amateur towards professional

Tony Yum — Sun, 18 Oct 2026 16:27:00 +0000

Watch the video on YouTube

At the end of the five punches post, the fighter could throw a jab, a cross, two hooks and an uppercut, into the air or a 35 kg heavy bag, without falling over. Tony Yum, the coach, was happy with it, but not finished.

His verdict had two parts. First, the punches were okay, but some still fell a little short, above all the cross. Second, the speed. For an average person who trains, it was not bad. At a professional level he expects more: a faster fist at the impact, and the whole punch , from the guard to the target and back, completed faster.

This post covers the afternoon of 18 October: seven training runs, from 11:45 to 16:27 UTC. Some of the fixes were changes to how the fighter is trained and rewarded. One was a change to its body: Tony allowed the elbow a stronger, faster motor.

The video at the top of this post shows the result. The figures below are stills from the MuJoCo simulation.

Measure first

Before changing anything, the first step was to measure. Was the fighter slower than its reference motion, the animation each punch is learnt from? Then something in the training was holding it back. Or was it exactly as fast as the animation? Then the animation itself was the ceiling.

It was the second. Every punch reached the target on the animation’s own schedule, within a few hundredths of a second: the jab in 0.22 s from the button press, the cross in 0.26 s, the hooks in about half a second, the uppercut in 0.36 s. The reason was built into the setup. The policy doesn’t move the joints directly. It adds a small correction to the animation’s pose, and motors pull each joint towards the sum. The animation ran on its own clock, so the fighter could bend the shape of a punch a little, but not its timing.

And the reward didn’t ask for more. Nothing paid for reaching the target sooner, and the pay for fist speed stopped growing once the fist matched the animation’s speed. The fighter was doing exactly what it was paid for.

Lesson 1: let the fighter set the pace

The first change gave the policy control of the clock. It got one more output, the speed of the punch’s clock: anything from the animation’s pace to twice as fast. The animation still shows the shape of the punch, but the policy now decides how fast to play it. A new reward paid, once per punch, for reaching the target sooner than the animation, and the pay for fist speed was allowed to grow to one and a half times its old top.

The first run did the opposite of what was intended. Within 11 M steps, every punch had got slower : the jab took 0.36 s to reach the target instead of 0.22, the left hook 0.92 s instead of 0.52. The clock was being turned down, not up.

The reason was in the reward’s arithmetic. Most of the style terms, how well the hips turn, where the hands are, how upright the body is, are paid on every step, 50 times a second. A punch that takes twice as long collects them twice. That was worth more than the new bonus for being quick, so the policy slowed down to collect. It is a textbook case of reward hacking: the policy did what the numbers said, not what was meant.

The fix was to make the length of a punch irrelevant to its style pay. The clock could now only speed up (one to two times the animation’s pace), and while a punch runs, every per-step reward is multiplied by the clock’s speed. A punch played at double speed collects its style pay at double the rate, for half as long: the same total either way. Only the quickness and the impact can tell a fast punch from a slow one.

The second run worked straight away for the pivot punches. The hooks reached the target in 0.36–0.39 s instead of 0.48–0.52, and the uppercut in 0.24 s instead of 0.36. They were hitting at 5.5–6.3 m/s. Figure 1 shows the difference on the left hook: by the time the old fighter’s fist reaches the target, the new one has hit and is back in its guard.

Figure 1. The left hook in the air, before (top: the five punches post's final model) and after (bottom: the final model of this post), at the same times after the button press: 0.12, 0.24, 0.34, 0.44 and 0.52 s. The green dot is the target. After: the fist reaches it at 0.34 s and is back in the guard by 0.44 s. Before: it arrives at 0.52 s.

Lesson 2: stopping short shouldn’t pay

The straight punches, the jab and cross, didn’t follow. Both now ran up to about 10 m/s, and then braked and stopped about 2.6 cm short of the target.

Again, the reward explained it. In the five punches work, a punch that fell short was scored by its top speed on the way, so a fist that stopped 3 cm short still learnt something (Tony’s “don’t make it so binary”). With the speed pay raised to one and a half times, stopping 2.6 cm short at 10 m/s now paid more than arriving at 2.4 m/s. A cut fixed it: a punch that stops short is now scored at 0.4 times its top speed, and never above 6 m/s. Stopping short at 10 m/s scores like arriving at 4 m/s, and it earns no quickness pay.

That run’s jab and cross went through the target again, but they arrived slowly, at about 3 m/s. They stayed that way for 50 M steps: the training numbers in Figure 7 below are flat through the whole run.

Lesson 3: an elbow that can straighten

The obvious guess was the hips: perhaps the body wasn’t driving far enough forward for the arm to reach. Measuring showed otherwise. The hips were about where the animation put them. The arm was the problem: the punching elbow never straightened past 37–44°, where the animation reaches 20–25°.

Was that the policy’s choice? A test answered it. The policy’s elbow output was overridden during the punch, and forced to ask for a fully straight arm. The elbow still stopped at 37–40°. So the policy wasn’t choosing a bent arm. The motor couldn’t straighten it in time.

Each joint is driven by a position servo. It pulls towards a target angle like a spring, with a damper that resists motion so the joint doesn’t wobble, and its total torque is capped. On a slow movement the damper hardly matters. On a fast elbow extension it took the whole torque budget, and the animation holds the arm straight for only 0.01 s before it starts back. The fist ran out of time. This was very likely the old “can’t reach” problem of the cross all along.

Tony’s question was a practical one:

If I allow the elbow to straighten faster would that help? I could continue to use the existing neural network right?

Yes on both counts. The network’s inputs and outputs don’t change, so training could carry on from the same model. The elbow motors got a 1.5 times higher torque cap, 90 to 135 N m, and a quarter of the damping, 36 to 13.5 N m s per radian. This is a judgement call about the body, not the training: the coach decided the fighter’s elbow should be able to snap straight like a trained athlete’s, and the simulated arm is now stronger there than before. Figure 2 shows the effect.

Figure 2. The cross from the side, before and after the elbow change, at 0.18, 0.20, 0.22 and 0.24 s after the press. Top: the old elbow motor; the arm stays bent and the fist stops short of the target (green). Bottom: the stronger, less damped elbow motor, one training run later; the arm reaches full length at the target.

The elbow straightened at once: 16–17° at full extension, against 37–44° before. The training hit rate jumped from about 18 % to 27 %, and full-reach jabs hit 41–49 % of the time in training, against 10–14 %. It is the largest single step in Figure 7.

Lesson 4: the shoulder has to come through

The jab was now reaching. The cross was still a centimetre or two short. Measuring along the punch line at full extension showed why: the cross’s hips were in the right place, but its chest and rear shoulder were 6–8.5 cm behind the animation’s. The trunk wasn’t turning the rear shoulder through. A cross is thrown from the back foot through the hips and the trunk to the fist, a kinetic chain, and here the chain broke at the shoulder.

Was the trunk too weak? No: the spine and shoulder motors never reached their torque caps during the cross. They simply weren’t being asked. So a new reward term asked them: it pays, on every step of a straight punch, for the punching shoulder being no further behind the animation’s than about 4 cm.

The first version did nothing. The lag opened only in the last one or two steps before the impact, and carried on after it, where the term no longer counted. So it paid almost nothing either way. The second version made it nearly three times stronger (4 per step), narrower (3 cm) and counted it on through the impact. Within 10 M steps the air cross started hitting, for the first time in this whole series. The cross’s shoulder lag at its worst fell from 7–9 cm to 2–6.5 cm, and back to about 0–1 cm by the end of the hit.

After about 15 M steps the training hit rate levelled off at about 33 %. Ten more million steps changed nothing, so the run was stopped at a plateau, at 16:27 UTC.

Where it stands

The final model was tested the same way as in the five punches post: no exploration noise (deterministic), six presses per punch at different moments of the fighter’s rhythm, in the air and against the heavy targets. Figure 3 shows its punches at the moment they reach the target.

Figure 3. The final model's five punches in the air, each at the moment it reaches the target (green): jab at 0.18 s after the press, cross at 0.22 s, left and right hooks at 0.34 s, uppercut at 0.24 s. All five hit.

Every punch now reaches the target sooner than before, from 0.04 s sooner for the jab to 0.19 s for the left hook (Figure 4). Every one of them is also sooner than the animation it learnt from. The cross, the punch Tony singled out, changed most: in the air it used to stop short in four presses out of six, at about 1 m/s. Now it lands in five out of six, at 4.4 m/s. Figure 5 has it frame by frame.

Figure 4. Before (the five punches post's final model) and after (this post's), in the air, deterministic, six presses per punch; medians. Top: time from the button press to the fist reaching the target; the dashed marks either side are the reference animation's time. Bottom: the fist's speed along the punch line as it reaches the target. Before, the cross stopped short in four presses of six.

Figure 5. The cross in the air, before (top) and after (bottom), at 0.06, 0.12, 0.18, 0.22 and 0.26 s after the press, same rhythm state. After: the fist is at the target (green) at 0.22 s, at 5.5 m/s, and already on its way back at 0.26 s. Before: it is still travelling at 0.26 s and arrives at 1.4 m/s, too slow to count as a hit.

In the full test it scored 38 hits out of 60, against 34 before, and it never fell (Figure 6).

Figure 6. The final model's test, six presses per punch, deterministic. Top: hits in the air (within 10 cm at 3 m/s or more). Bottom: hits on the heavy targets (the 35 kg bag; for the uppercut the 25 kg ball). Total: 38 of 60, no falls (before: 34 of 60). The cross went from 0 to 5 in the air; the jab lost one press on the bag.

Not everything got better:

The hooks and the uppercut on the heavy targets still connect light: 10–15 N s of momentum into the bag, under its bar of about 21 N s, as before. The left hook slows from about 4 m/s in the air to about 1 m/s on the bag.
The left hook arrives sooner but a little slower : 4.2 m/s at the target, against 5.3 before. The uppercut is about the same. The quickness came mostly from cutting out wasted time, not from a faster fist.
The straight punches meet the target at the very end of the arm’s reach. They peak at about 8.5 m/s on the way and slow to 4.5–5 m/s by the target. In some rhythm states they still stop just short. A professional punches through the target, with the hips driving or a step in, so the fist is still accelerating at the impact.

The path, in numbers

Every run except the first continued from the model before it (fine-tuning). A frozen copy of an earlier model acted as a teacher for the stance and the rhythm, as in the five punches post (policy distillation), but it no longer had any say over the punches, whose timing was the thing to change.

Figure 7. Training, five runs end to end (the first, the slow-down run, was dropped and isn't shown). Teal: the share of all punches in training that hit. Purple: the share of full-reach crosses that hit. Means per million steps, with exploration noise on. The short-fist run is flat for 50 M steps; the stronger elbow and then the stronger shoulder term are the two jumps.

Figure 8. The seven runs on one clock (UTC, 18 October). Filled bars are the runs whose model was kept (frozen). The 'shoulder' run's first launch crashed at start-up and was relaunched two minutes later. Short gaps between runs were Amazon's spot request for the previous machine still closing.

run	what changed	outcome
own clock	the policy sets the punch’s speed (half to twice the animation’s), a reward for arriving sooner	every punch slower: per-step style pay rewarded long punches; dropped
clock ×1–2	the clock only speeds up; per-step pay scaled by the clock’s speed	hooks and uppercut 25–33 % sooner and hitting; jab and cross stop 2.6 cm short
short fist	a punch that stops short scores at 0.4 times its top speed	jab and cross reach, but slowly; the elbow can’t straighten; flat for 50 M steps
elbows	elbow torque cap ×1.5, damping ×0.25	the elbow straightens; training hit rate 18 % → 27 %
shoulder	reward for the punching shoulder keeping up	too weak: the lag opened after the term stopped counting
shoulder ×4	the shoulder term stronger, narrower, through the impact	the air cross lands; 38/60 in the test; final model

What it cost

All seven runs used the same machine as the end of the five punches work: a c8a.24xlarge on Amazon EC2, with 96 vCPUs, rented as a spot instance in us-east-1d at $1.80 an hour. The same machine on demand costs about $4.90. It ran about 37 M simulation steps an hour; the long “short fist” run managed about 25 M, because the progress videos were being rendered on the same machine.

Every run saved a checkpoint once a minute, and a new run could start from any earlier run’s, so a stopped run’s training was never lost. The one crash (an attribute set too late when a training episode started mid-punch) cost three minutes and 9 cents.

run	machine-hours	cost
own clock	0.38	$0.69
clock ×1–2	0.58	$1.05
short fist	1.6	$2.88
elbows	0.62	$1.11
shoulder (crashed launch)	0.05	$0.09
shoulder	0.4	$0.72
shoulder ×4	0.85	$1.53
total	about 4.5	about $8

These are estimates from each run’s start and stop times and the zone’s spot price at the time, not from the bill. About 160 M simulation steps in all, about 5 cents per million. The same work on demand would have cost about $22.

What’s next

Tony’s two points were the cross falling short and the punches not being fast enough for a professional. The first is mostly fixed: the cross lands in the air and on the bag. The second is half done. Every punch now completes sooner, faster than the animation it learnt from, but the fist isn’t yet hitting harder.

That is the next step. The straight punches need to go through the target at speed rather than reach it at full stretch, which means more drive from the hips, or a step in. The same step-in is needed for targets further away, where a real fighter lunges. And the hooks and the uppercut have to put real weight into the bag, not just touch it.

Read it on ai.kakuto.io, with the interactive charts and the glossary.

Five punches: the cross, the hooks and the uppercut

Tony Yum — Sun, 18 Oct 2026 10:00:00 +0000

Watch the video on YouTube

The jab took two days. Once it was fast, accurate and stepping in, Tony Yum wanted the rest of the basic Muay Thai punches: the cross, the left and right hooks, and the lead uppercut. All five in one fighter, on command, from the same guard.

This post covers the next 18 hours: ten training runs, from the afternoon of 17 October to 10:00 UTC on 18 October, when the last one finished. The fighter went from punching the air to punching a 35 kg heavy bag, and from one punch to five.

Tony’s verdict at the end, in short: it isn’t perfect, and he is happy with it. A couple of punches still fall a little short. The rest look close to how an ordinary person who trains would throw them. The next step is to make them look like an athlete’s, and that is already under way.

Four new punches

Each new punch started, like the jab, as a reference motion: an animation built pose by pose from Tony’s description and photos, with no physics in it. His specs read like a coach’s notes:

The cross (the rear straight). The right arm shoots out, the fist turning over so the palm faces the floor. The chest turns left, the lead foot steps in, and the rear foot pivots on its ball with the heel up. The left glove stays on the cheek.
The hooks. “The power comes from the legs, hips and core, not the arm.” The arm takes the hook shape, elbow bent about 90° at shoulder height, and then stays locked to the chest. The body’s turn carries the fist through the target. A small wind-up the other way comes first.
The uppercut. The load is a turn, not a squat. The hips and chest turn left over the lead leg, and the fist drops to chest height. Then the turn reverses from the feet up, and the fist drives straight up into the chin.

The fighter then has to do all of this in a physics simulation, with weight, momentum and feet that can slip. One policy runs all five punches. It sees which button was pressed and where the target is, and it gets a reward for following that punch’s reference and hitting the target fast. Figure 1 shows where it ended up.

Figure 1. The five punches of the final model (18 October, 10:00 UTC), in the air. Top: just before the impact. Bottom: the moment the fist is nearest the target (green dot). Left to right: jab, cross, left hook, right hook, uppercut. The rear foot comes up onto its ball in the cross and the right hook, and the uppercut's fist arrives from below.

Air, then a pad, then a heavy bag

Tony’s brief had one more part. Real fighters don’t only shadow box, so the fighter shouldn’t either:

Train from hitting air, then a light target, then a heavy bag, so it learns the impact and the recoil.

So the targets became part of a curriculum (Figure 2). At level 0 the fighter punches a point in the air, and only speed and accuracy count. From level 0 to 1 a pad appears, the size of a focus mitt, on springs, and grows from 0.75 to 3 kg. From level 1 to 2 it becomes a heavy bag on a rope, from 12.5 up to 35 kg. The uppercut gets its own targets, because hitting a hanging bag from below makes no sense: a pad facing down, then a heavy ball hanging at chin height. That was Tony’s idea too.

Figure 2. The target curriculum. The level rises by 0.1 when, over 20 training iterations, every punch hits at least 40 % of the time with at most 20 % falls. Above level 0, three presses in four get a target and one stays in the air, so the fighter keeps practising both. The green dot is the aim point: from the second morning it sits inside the target, so the punch has to go through the surface.

Lesson 1: don’t forget the jab

The first run started from the best jab and added the four new punches. Within 2.7 M steps, about seven minutes, the jab was falling apart. Its hits dropped from 28 % to 2–7 %, and its speed from 2.7 to 1.3 m/s. The fighter was still on its feet, but it had bought its balance with a slower jab. This is catastrophic forgetting, and an earlier attempt with several buttons had lost the jab in the same way and never got it back.

The fix was to give the old policy a job: teacher. A frozen copy of the best jab policy watches the new one during training. Whenever no new punch is running, in the stance, the rhythm and the jab, the new policy pays a penalty for doing anything different from the teacher. While a cross, hook or uppercut runs, it is free to learn. This is a form of policy distillation, and it worked at once. The jab stayed at 5–6 m/s for the next 36 M steps while everything else was learnt around it.

The hooks and the uppercut were quick. They hit within about 10 M steps. They are pivot punches: the body turns, the arm is carried, and the reference had already worked out the shape. The curriculum then climbed from the air to the 35 kg bag in about 18 M steps. After about an hour and a half of training, four punches were landing on the heavy bag with no falls. The fists arrived a little slower on the bag than in the air, 3–4 m/s against 5–6, as if bracing for the contact.

The fifth punch was the cross, and it was 9–12 cm short.

Lesson 2: the cross that rode the rhythm

The cross was the jab’s old problem again. A straight punch from the rear hand has further to go. To reach, the hips have to drive forward about 19 cm in 0.16 s, at up to 1.75 m/s. The policy’s hips stayed 9–11 cm behind the reference’s, so the arm ran out of reach and the fist stopped short.

The next run gave the cross 40 % of the button presses, a reward for the hips keeping up with the reference, and a target that started 10 cm nearer and stepped back 2 cm whenever the cross hit often enough. The teacher held the other four punches in place. Over 103 M steps the cross went from 17 cm short to reaching its full-reach target.

But the test videos had been flattering it. Each grid of test videos showed three presses per punch, and those three happened to come at good moments. A wider test, six presses per punch at different points in the fighter’s rhythm, found the cross 0 for 12. Tony’s fighter stands in a Muay Thai rhythm, rocking gently between the feet. The cross only worked when the rock was big. From a big rock the fist arrived at 2.6–4.1 m/s. From a near-still stance it stopped 3–7 cm short, at 0 m/s, in 9 presses out of 10. The policy was borrowing the rock’s momentum for its hip drive, not making its own.

Two more runs went at this:

Press more crosses from a still stance. No better: still 3–5 cm short from still, and worse from a rock. Then Amazon took the machine back (more on that below).
Pay for the hips’ forward speed and turn directly. The turn got fixed: 5–8° short became spot on. The hips moved about 1 cm nearer. After 61 M steps the cross was no better, and slower from a rock.

Three runs had now failed to make the body drive its hips 19 cm in 0.16 s from a dead stop. That was the point to stop changing the reward, and to ask the coach.

Lesson 3: through the target, not to it

Tony was watching the grids on his phone, and he saw something else first:

That green dot, the target should be inside the heavy bag and inside the small target. Otherwise, you’ll end up hitting the surface. You need to go through. So put the target in the middle.

He was right, and it was a basic mistake. The target’s surface had been placed at the aim point, so every punch was trained to arrive at the surface and stop. The aim point moved inside the target, first at the bag’s centre, then (after “the middle of the bag’s too central”) 7.5 cm in. A hit was now scored where the knuckle meets the surface, and the punch was asked to carry on through it.

That raised a second question: what should a heavy bag reward? The first answer was distance, how far the bag swings. It plateaued at about 18 cm. The fist stayed on the bag for only 0.04–0.08 s, because the reference pulls the arm straight back, so every punch was a single knock. Then Tony ruled distance out entirely:

Never push the bag. It’s all about impact. Impact and snap back. Don’t push into the bag. Impact and pull back.

So the heavy targets now pay for two things. The impact is how fast the bag moves in the first 0.06 s after the fist reaches it, which measures the punch’s momentum. The snap back pays for the fist leaving the bag within 0.2 s. The jab learnt both straight away: 31–34 N s into the bag, and off it in 0.06–0.08 s. The cross and the hooks stayed light.

Lesson 4: make it easier to get right

After the cross had stalled through several runs, Tony looked at it as a coach would look at a student who keeps missing:

The target should be a little bit closer. And also whether it’s considered a hit or not, let’s not make it so binary. You get more reward the closer you are to the target.

Three changes followed in the same run:

The cross’s target moved 6 cm nearer. A cross that stops short of a target at full stretch is mostly a reach problem, and a real fighter would simply stand a little closer.
A punch that falls short now scores by its top speed on the way, not zero. Before, the reward jumped from almost nothing to a lot at the surface, and a fist stopping 3 cm short learnt nothing about stopping 2 cm short. Now each centimetre and each m/s counts. This is reward shaping.
The bars came down. The target for a hit got bigger, 10 cm instead of 5. A test showed the misses were nearly all within 5 cm already, and they missed on speed or impact. So the speed for a hit in the air came down from 3.5 to 3 m/s, and on the heavy bag to about 21 N s from 31. Tony’s answer to that question was two words: “Lower the bars.”

The cross started landing. Full-reach crosses in training went from 0 % to 34–42 % hits, and it hit the bag in three test presses out of three.

Lesson 5: what only a coach sees

With the punches landing, Tony’s notes turned to things no hit rate shows.

A step back before the punch. “Why does it always take a step back?” On the jab, the fighter shifted its weight back, lifted the lead foot and moved it about 5 cm back, then stepped in. On the cross, the rear foot slid back as the heel came up. The reference does neither: the rear foot stays planted and the lead foot steps straight in. A new reward term paid for neither foot going behind its starting spot until the punch returns. The lead foot’s step back went from up to 4 cm to almost none.

The head. “The head’s really tilted to the side.” It was, by about 30–35° in the guard. None of the references tilts it. Tracing back through every saved model found where it came from: a jab run on 17 October that trained against much harder shoves. Before that run the tilt was 5–9°. After it, 20–30°, and the teacher had faithfully passed it on to every run since.

A reward for keeping the head near the reference did nothing in 6 M steps: the head was a small part of a large reward. So the next test was blunt. The neck and head were locked to the reference, and the punches were re-tested. They scored 15 hits out of 40 locked, against 16 out of 40 free, with the same speeds and no falls. The tilt had been doing nothing useful. The neck stays locked, and the head is level (Figure 3).

Figure 3. The guard before and after the head fix, from the front, two seconds into the same rhythm. Left: the model at 08:12 UTC on 18 October, head tilted about 30° to the side, a habit learnt under hard shoves the day before. Right: the final model, with the neck and head locked to the reference, head level. The punches scored the same either way.

Where it stands

The final model is from the last run, frozen at 46 M steps. It was tested with no exploration noise (deterministic), six presses per punch at different moments of the rhythm, in the air and against the heavy targets. It never fell. Figure 4 shows the bag row, and Figure 5 the scores.

Figure 4. The final model against the heavy targets: the 35 kg bag, and for the uppercut the 25 kg ball hanging at chin height. Top: just before the impact. Bottom: the impact. The aim point is hidden 7.5 cm inside the surface, so the fist has to go through it.

Figure 5. The final model's test, six presses per punch, deterministic. Top: hits in the air (within 10 cm at 3 m/s or more). Middle: hits on the heavy targets. Bottom: the momentum given to the target in the first 0.06 s after contact; the dashed line is the bag's hit bar, about 21 N s. The uppercut's ball is lighter and its bar is about 17 N s. Total: 34 hits out of 60, no falls.

In words:

The jab is the best of them: 11 of 12, and 29–33 N s into the 35 kg bag.
The cross lands on the bag, 5 of 6. In the air it is fast, about 7 m/s, but passes 2–5 cm off or stops short. This is the punch that is still “a little short”.
The hooks and the uppercut hit every time in the air, 18 of 18, at 5–6 m/s. On the heavy targets they connect, but light: 11–15 N s on the bag, under its bar.

From the coach’s side, that is a fighter with a real combination. The stance holds, the head is up, the feet don’t shuffle back before a punch, and each punch has the right shape: the hips turn, the rear heel comes up on the cross, the hooks are carried by the body. It looks like someone who has trained for a while. Not yet like a fighter.

The path, in numbers

Figure 6. The ten runs on one clock (UTC, 17–18 October). Each run started from the best or latest model of an earlier one. Filled bars are the runs whose model was kept (frozen); outlined ones were stopped or superseded, and their lessons carried forward. The gap before midnight began with a reclaimed spot machine that went unnoticed for an hour.

run	what changed	outcome
five buttons	four new punches on top of the best jab	the jab decayed within 2.7 M steps; stopped
teacher	the jab policy anchors everything except the new punches	four punches on the 35 kg bag in 36 M steps; the cross 9–12 cm short
cross	cross 40 % of presses, hip-drive reward, nearer target that steps back	the cross reaches, but only from a big rhythm rock; 0/12 in the wider test
still	most crosses from a near-still stance	no better from still; spot machine reclaimed at 32 M
push	pay for the hips’ forward speed and turn	turn fixed, hips 1 cm nearer; not kept
aim inside	the aim point inside the pad and the bag	punches go through the surface; stopped for the next idea
push the bag	pay for how far the bag swings; uppercut pad and ball	plateaued at ~18 cm swing; Tony: “never push the bag”
snap back	pay for impact in 0.06 s and leaving the bag	the jab learns it; cross and hooks light
nearer, graded	cross 6 cm nearer, graded short punches, bigger target, lower bars	the cross lands: 34–42 % at full reach in training
feet and head	no step back before the punch; head level	the final model: 34/60, no falls

What it cost

The fighter trains in the cloud on Amazon EC2 spot instances, spare capacity sold at a discount that Amazon can take back at two minutes’ notice. Most of this work ran on a c8a.24xlarge: 96 vCPUs, at $1.47–1.80 an hour depending on the zone, against about $4.90 on demand.

The machine changed at the start of this work. On the usual 64-vCPU machine, the simulation wasn’t the bottleneck any more; the learning update was, and the simulator cores sat idle about 40 % of the time. Moving to 96 vCPUs, with bigger batches and fewer passes over each batch, took training from about 22 M to about 40 M steps an hour: 1.8 times the speed for about 1.5 times the price.

Spot had its costs. Amazon reclaimed a machine three times in these 18 hours. Twice the run resumed within minutes from its last checkpoint. Once, late on 17 October, the watcher kept showing the last status line and nobody noticed for an hour, which is the gap in Figure 6.

run	machine-hours	cost
five buttons	0.15	$0.15
teacher	1.5	$1.87
cross	2.8	$4.10
still	0.9	$1.57
push	1.8	$2.91
aim inside	1.0	$1.50
push the bag	1.7	$2.85
snap back	1.3	$2.10
nearer, graded	1.4	$2.15
feet and head	1.7	$2.75
total	about 14	about $22

These are estimates from each run’s start and stop times and the spot price of its zone at the time, not from the bill. About 470 M simulation steps in all, a little under 5 cents per million.

What’s next: from trained to professional

Tony is pleased with this. The fighter throws five recognisable punches from a Muay Thai guard, on command, into the air or a heavy bag, without falling over. But he wants more than a trained amateur.

Measuring the final model against the animations showed why it looks the way it does. Every punch reaches the target on the reference’s own schedule: the jab in 0.22 s, the cross in 0.26 s, the hooks in about half a second. Nothing in the reward paid for getting there sooner. The hooks’ fists peak at 5.6–5.8 m/s, against 8.3–8.8 m/s in the animation, and the cross arrives slowly or short.

A professional’s punch differs in exactly these places. It gets to the target sooner, hits harder at impact, and comes back to the guard faster. So the next runs let the fighter set its own pace, and pay for quickness and for speed at the target. That work started an hour and three quarters after this run finished. It will be the next post, with a video of all five punches.

Read it on ai.kakuto.io, with the interactive charts and the glossary.

The jab: the simplest punch, and a long lesson

Tony Yum — Sun, 18 Oct 2026 00:00:00 +0000

Watch the video on YouTube

The jab is the first punch anyone learns. Lead hand out, lead hand back. Tony Yum has taught it to beginners for years, so he expected it to be the easy one for the AI fighter too.

It wasn’t. Over two days and 23 training runs, the jab turned out to hold most of the lessons of the project so far. This post follows that path from the coach’s side. The previous posts cover how the fighter learnt to stand and how the coaching works.

What a jab really is

To a beginner, a jab is an arm movement. To a coach, the arm is the last link of a kinetic chain that starts at the feet. Tony’s spec for the reference motion read like a checklist he would give a student:

The lead foot steps forward a little (16 cm) as the fist goes out, and turns with the hips.
The hips turn about 45° to the right, so the left shoulder goes forward and the right shoulder back.
The fist turns over on the way, so the palm faces the floor at full extension.
The weight stays about 50/50 between the feet. The jab doesn’t fall onto the front leg.
The chin stays tucked behind the lead shoulder. The rear glove comes up beside the right cheek.
After the snap the fist comes straight back to the guard, and the lead foot steps back into the stance.

The fist goes out in 0.14 seconds and is back in the guard 0.14 seconds later. The video at the top of this post shows the result.

There is a catch. The reference is an animation. It was built pose by pose with inverse kinematics, shaped by Tony’s corrections until it looked right. Nothing in it obeys physics. It has no mass, no momentum and no feet that slip. It is, in Tony’s words, vibe-created. Figure 1 shows its key moments, with a line down from its centre of mass to show where the weight sits.

Figure 1. The jab reference in MuJoCo, from the front three-quarter view (top) and the side (bottom). Left to right: the stance, the fist on its way out, full extension 0.14 s after the start, the snap back, and the lead foot back in the stance. The red line drops from the body's centre of mass to the floor, so it shows where the weight sits between the feet. This is the animation only: no physics yet.

The fighter then has to do the same movement in a physics simulation, where everything has weight. A reward function scores how well it does. The policy tries millions of jabs and keeps whatever scores best. So everything below is about one question: when the animation and physics disagree, which parts of the animation does the coach insist on, and which does he let go?

Lesson 1: reaching out and touching

The first run copied the reference and paid a reward for hitting a target at full reach. The fighter kept its balance and stretched its arm out, but never stepped. Every jab landed 15 cm short.

The reason was simple arithmetic. A hit paid about 2 % of what a whole episode paid. Standing still in a good stance paid the rest. So the safe choice was to keep the feet planted. With the hit reward made four times bigger, the fighter started to step and hit. After target distances were added as a curriculum, it hit within 5 cm on 93 % of full-reach jabs.

Tony watched the videos and wasn’t impressed. “It’s reaching out and touching.” There was no speed and no power.

This was the first clash between the animation and physics. The reference fist slowed to zero on the target and held there for 0.04 seconds, like a finger pressing a doorbell. The reward paid for every moment the fist stayed near the target, so arriving gently and staying there scored best. The policy did exactly what it was paid for, a mild case of reward hacking.

So the reference changed: no hold, and a quicker jab. The target moved 7 cm inside the point where the reference’s fist stops, so the punch goes through it, as a real jab goes through the target. The reward changed too. One payment per jab, multiplied by the fist’s speed when it crosses the target.

The fist’s top speed rose from 5.5 to 7.4 m/s. Jabs at nearer targets now hit at 4.4 to 5.4 m/s, against about 1 m/s before.

Lesson 2: the step that never came back

Then Tony saw something in the grid of test videos. After each jab, the fighter stayed where it had stepped. A second later it faced 29 to 39° to the right, with both feet 10 to 20 cm from where they had started.

Every part of the stance reward was measured in the body’s own frame. A stance a little forward and turned to the side scored just as well as the original one. The fighter had no reason to go back, and no way to see where it had started.

The obvious fix was a “home” reward: after each punch, face forward again with the feet back in place. The fighter learnt that. But it stopped stepping into the jab at all. Its hips ended up 15 to 18 cm behind the reference at impact, and the punch lost its power.

Was the home reward stopping the step? Two runs went side by side to find out, one with the home reward and one without. Neither stepped. Both lost the jab’s speed, from 1.5 m/s at the target down to 0.4 to 0.7 m/s. Meanwhile their total scores went up, from the stance terms. Not stepping had become a local optimum. Stepping in and back would have scored more. But it is a long, coordinated sequence the policy had never found, and every small step towards it scored worse at first.

Here Tony made a coaching call:

The speed is important and so is the style before and on impact. But the step back can be more relaxed stylistically, to make it easier to recover.

So the reward split the jab in two. Up to the impact it asked for style and speed: the reference’s turn, the foot placement, and a fist moving at least as fast as the reference’s. After the impact, the stance reward eased off to simply “stay up and keep your guard” for half a second. The home target also got twice as forgiving, 20° and 12 cm.

It worked straight away. The new run hit on 54 % of its training jabs at 4.4 m/s. The old reward, over the same time, managed about 0 % at 0.6 to 1.2 m/s.

Lesson 3: the hips, and how much rotation to ask for

The arm was now fast, but the hips stayed 11 cm behind the reference at impact. That is an arm punch, the beginner’s mistake of jabbing without the body behind it. In the reward, the hips were worth about 3 points per jab against about 45 for the fist’s speed and the hit. The policy had worked out that the hips didn’t matter much.

The next run multiplied the impact payment by how close the hips were to the reference’s at impact. Hits dropped to 17 % at first, because the impact suddenly paid less. Then they recovered. By the end the hips were 6 cm behind, with 57 % hits at 4.0 m/s. Most of the gap had been the step’s length, not the hips’ turn.

Tony then gave the body some freedom:

If the hip movement, that rotation, the hip and the feet, is costing the jab speed and accuracy, focus on the top half of the body. The step is still important, stepping forward is important, but the actual rotation can be relaxed a bit.

So the hip score now counted only how far the hips lagged forward, not their sideways offset or their turn. The turn’s tolerance widened from 8° to 20°. A new term asked the arms and shoulders to follow the reference up to the impact. The jab improved again: 66 % hits at 4.5 m/s, hips 5.7 cm behind.

Lesson 4: the bow

Freedom has a price. Tony spotted it in the next grid:

After the jab and the recovery, the character wobbles forward like a little bow.

Half a second after each jab, the chest pitched 13 to 16° forward, then straightened up. The relaxed return asked only “stay up and keep your guard”, and a bow does both. The body had found the cheapest way to soak up the punch’s momentum.

The fix was to give the trunk back one rule during the relaxed return: keep the chest within 4° of the reference’s tilt. The legs kept their freedom. The bow went from about 15° to under 4°. The jab got better too: 76 % hits at 4.6 m/s, and the hips 4.9 cm behind, inside the 5 cm goal for the first time. Figure 2 shows one of its jabs, now under physics.

Figure 2. One jab by the trained fighter, from the run that fixed the bow (11 M steps), at a target (green ball) level and straight ahead. Left to right: the guard, the fist on its way, the hit, the snap back, and the guard again, with the chest upright. Compare the reference in Figure 1.

It had a side effect. With the chest held only after the punch, the fighter’s ready stance drifted back to a 5° lean. So the chest rule was extended to cover every moment except the punch’s drive itself. The stance stood upright again, with no bow, and the fighter hit 9 out of 9 test targets at every aim, up to 15 cm off centre.

Lesson 5: distance

A real opponent isn’t always at the perfect distance, so the targets moved nearer and farther. Two more coaching calls came out of it.

Too close. A target 25 cm nearer forced a cramped jab: a bent arm and no step. Tony’s verdict: it “looks really bad”, and at that range a fighter should throw an elbow or a knee anyway. The close jab was dropped.
Too far. For targets beyond full reach, the hips stayed behind again, 9 to 10 cm, and the jab slowed down. A reward for the hips’ position at the moment the fist crosses the target helped full-reach jabs, from about 4 to about 5 m/s. The far jab still didn’t get fast.

That run gave the most accurate jab so far: 85 % hits on a fixed test of 54 jabs, at a median of 4.7 m/s. Figure 3 is the grid Tony got on his phone from that version. The near and mid jabs hit hard. At full reach they still hit, a little slower. The far jabs get there, but slowly. The far jab is waiting for a new reference. Tony will film himself throwing a normal jab and a long one, side on, so the longer step can be copied from a real body.

Figure 3. A test grid from the most accurate jab (10 M steps), caught at the moment of impact. Each panel is one jab at a fixed target (green ball). Rows: the target 12 cm nearer than full reach, 6 cm nearer, at full reach and 5 cm beyond it. Columns: to the fighter's right, centre and left. Under each panel: how close the fist came to the target and how fast it was going there, green for a hit (within 5 cm at 3.5 m/s or more). The far jabs reach the target at only 1–3 m/s.

The path, in numbers

Figure 4. The five jab runs of the early morning of 17 October (UTC), each continuing from the one before. Top: the share of training jabs that hit within 5 cm at 3.5 m/s or more. Middle: the fist's speed at the target. Bottom: how far the hips were behind the reference's at impact (lower is better). Training numbers, with exploration noise on.

The same numbers as a table:

run	what changed	hits	fist speed	hips behind at impact
home reward	come back to the stance after each jab	~0 %	0.6–1.2 m/s	12–14 cm
style	style and speed up to impact, relaxed return	54 %	4.4 m/s	10.9 cm
hips	impact pays less with the hips behind	57 %	4.0 m/s	6 cm
upper body	arms first, hip turn relaxed	66 %	4.5 m/s	5.7 cm
upright	no bow in the return	76 %	4.6 m/s	4.9 cm

Hits and speeds are from training, with the policy’s exploration noise on. Each run continued from the one before (fine-tuning), so the skill built up.

One machine quirk is worth knowing. In several later runs the training numbers held steady while the noise-free, deterministic jab slowly got worse after about 10 M steps. So runs now save a checkpoint every couple of million steps, and the best one is picked by testing the jab itself.

What it cost, and which machine

The fighter trains in the cloud on Amazon EC2, as described in the balance post. A physics simulator like MuJoCo runs on the processor, one copy of the fighter per core, so the number of vCPUs is what buys speed. A graphics card adds very little.

The default machine is a c8a.16xlarge: 64 vCPUs, about 29 M simulation steps an hour. Rented as a spot instance, from Amazon’s spare capacity, it cost about $0.97 an hour in us-east-1 over these two days. The same machine on demand costs $3.28. On that machine a typical jab run of 15 M steps took 40 to 50 minutes, so each experiment cost well under a dollar.

Three decisions shaped the bill:

Spot, and a fallback when spot fails. Amazon can take a spot machine back at two minutes’ notice. Training saves a checkpoint every minute and a watcher relaunches the run, so a reclaim costs a few minutes. On the afternoon of 16 October one run was reclaimed six times in 90 minutes. It moved to an on-demand machine for its last 35 minutes, about $2, to finish. When no c8a was free, a c7a.16xlarge filled in: about 30 % slower, but from a different pool of spare machines.
Two smaller machines for a side-by-side test. The account could run 64 spot vCPUs at a time. For the home-reward test, two runs had to start together with the same settings. So they each ran on a c8a.8xlarge: half the cores, at about $0.68 an hour. That is about 40 % more per core, but it gave a fair comparison in the same half hour.
Short runs, often stopped early. Each run had a deadline and a step budget, and the grids of test videos showed problems within 5 M steps. When a run clearly wasn’t going anywhere, it was stopped and the next idea started.

Adding up the start and stop times of the 23 jab and punch runs from 16 and 17 October gives about 16 machine-hours. At the spot prices of those days that comes to roughly $16, including the on-demand finish. Two days of a coach’s ideas, tested, for the price of a lunch.

What the coach learnt

Each fix moved the problem somewhere else.

Pay for a hit, and the fighter reaches and touches.
Pay for speed, and it steps in and never comes back.
Make it come back, and it stops stepping, and the jab goes weak.
Relax the way back, and it bows.
Hold the chest after the punch, and the stance leans back.

None of this is a bug in the AI. Each time, the policy did exactly what the reward asked. The reference was made without physics, so it can’t all be copied. The coach has to choose which details are the jab and which are decoration.

For Tony the answer came out in an order he recognises from the gym. The speed and the shape up to the impact come first. The step forward comes next, because that is where the power comes from. The chest stays upright, and the head stays up. The hip turn and the legs on the way back can be looser, as long as the fighter ends up back in its stance, guard up.

It is also the order in which he fixes a beginner’s jab. A student told to “get back to your guard” often stops stepping in. A student told to “turn your hips” sometimes turns everything and loses the target. Coaching means asking for one thing at a time, and noticing what the student gives up to get it. The fighter just does it faster, and on a phone screen.

Read it on ai.kakuto.io, with the interactive charts and the glossary.