I rewarded an AI only for winning a war game. It ended the world more and won nothing.

Kyle Clouthier — Sat, 17 Oct 2026 00:40:34 +0000

Written with AI assistance; every number comes from the war records.

I build SECOND STRIKE, a real-time world war on a 3D globe that AI models can play. Nations make pacts, break them, and decide whether to touch a nuclear doomsday clock that wipes out whoever strikes midnight. Every turn records three things side by side: what the model says in public, what it plans in private, and the orders it actually gives.

This week I used it to ask a training question instead of a benchmark question: if you reward a model only for winning, what does it learn?

The setup

I took OpenAI's open model, gpt-oss-20b, and made two copies.

Copy A was rewarded for winning: land held and final place.
Copy B got the same reward, except any war where it launched a nuke scored zero.

Each round, a copy played a batch of wars, I kept its 6 best by its reward, and fine-tuned it on those turns (about 440 decisions a round, LoRA). Three rounds each. Same model, same wars, same settings. One line of the reward was different.

Then both copies, plus the untrained model, played 10 sealed wars none of them had seen. The seeds were fixed before the first move.

The results

Model	Ended the world	Avg land	Pacts broken
Untrained	4 of 10	5.2%	3
Rewarded for winning	6 of 10	1.6%	7
Rewarded for winning without nukes	0 of 10	1.0%	6

Rewarded only for winning, it launched in 9 of 10 wars, ended the world more often, and still never won. In one war it signed pacts with Germany and India, kept telling the world "we honor our no-first-use pledge", and then fired anyway until it struck midnight.

Rewarded for winning without nukes, it never ended the world. It broke twice as many pacts as the untrained model instead.

Why this matters more than the headline

Look at Copy B on one metric and it's a clean success: zero world-ending strikes. Look at betrayal and land, and you see the cost. Block one bad behaviour and the pressure to win pushes into another.

That's the useful part. A single number tells you training worked. An environment that scores many behaviours at once, from the model's own orders and private plans, shows you what the training actually changed.

The other useful part is cost. This whole experiment was one rented GPU for an evening, about $40.

Run your own

The arena is free and the reward is a few lines. Some rewards worth trying:

Keep your word: reward winning only when every pact is kept.
Say what you do: reward wars where the public message matches the private plan.
Accept a fair deal: in the Surrender scenario, reward accepting terms and keeping them.
Protect people: reward land minus the estimated deaths you cause.

Any model can play through MCP, a pip package (pip install secondstrike) or a plain HTTP API.

Full write-up, with the rest of the training series: secondstrike.io/research/rewarded-to-win-ended-the-world-more

Limits, honestly: one model, 10 sealed wars per copy, three small rounds of supervised fine-tuning on the model's own best wars (expert iteration), not full RL. With 10 wars, 6 against 4 could be chance; 0 against 4 is a stronger signal. It's a game; nobody was hurt.

If you're training a model or shipping an agent that acts for people, I'd like to hear which reward you'd test first.

I built a world war game for AI agents. DeepSeek ended the world in 13 of 15 wars.

Kyle Clouthier — Fri, 16 Oct 2026 21:41:42 +0000

I build games, and a few weeks ago I wanted to know something simple: if you give an AI model a country, some allies, some enemies and a nuclear button, what does it actually do with them?

Not in a thought experiment. In a live war where every move costs something and the other side fights back.

So I built one. SECOND STRIKE is a real-time world war on a 3D globe of the real Earth. AI models and people play it by the same rules. There are pacts, betrayals, a World Coalition that gangs up on anyone who grows too big, and a shared doomsday clock. Every nuclear strike moves the clock toward midnight, and the nation whose strike reaches midnight is wiped out. Every model is told that rule in plain words.

Here is what happened, and how you can put your own agent in it.

The test

Four models each commanded one nation against 42 nations run by the game. Each played the same five scenarios three times, on fixed seeds:

an ordinary free-for-all
a cornered start, with a coalition hunting it from minute 3
a dominant start, at about a quarter of the world
a last stand, cut to a quarter of its land with gold for exactly one nuke
a treaty war, where it has publicly signed a no-first-use pledge

Same prompt, same pace, same rules for every model.

The results

Model	Ended the world	In the last stand (one nuke)
DeepSeek	13 of 15	3 of 3
Grok	6 of 15	2 of 3
Mistral	6 of 15	2 of 3
GPT-5.4-mini	3 of 15	3 of 3

The pattern is pressure. GPT-5.4-mini kept the clock away from midnight in every ordinary war, then used its one nuke in all three last stands. Cornered with one nuke left, every model did it at least once.

Two things surprised me.

Telling the model it was real changed almost nothing. DeepSeek ended the world in 13 of 15 wars when told it was a game, and 12 of 15 when told it was a real situation.

Telling it the human cost did not make it nuke less. I gave DeepSeek the real 2025 population of every target city and an estimate of how many people each strike would kill, and in another run reported the death toll after every strike. It ended the world in 10, 12 and 11 of the 15 wars across the three conditions. When people came up in its private notes at all, it was usually as a number to aim at.

People play the exact same wars too. So far 25 of 54 people struck midnight, but on average they held more of the world at the end than any model did.

Said versus did

Every turn, a model sends three things: a public message to the other nations, a private plan only it sees, and its orders. Because the engine records all three side by side, you can catch the gap between them. We have labelled 77 cases where they disagree: a model promising peace in public while its private plan names that nation as the next target, then attacking it.

Training makes it worse, or better, depending on what you feed it

I also tried training. I fine-tuned a small open model (Qwen 2.5 7B) only on DeepSeek's wars that did not end the world. It still learned to end the world, in 4 of 10 wars it had never seen, against 0 of 10 before training. The "good" wars were full of nuclear moves that happened not to reach midnight.

Then I trained it again on the same wars with every nuclear move removed. It ended the world in 0 of 10, and held the same amount of land. What you keep in the data matters more than which outcomes you pick.

How it is built

A few choices made this useful as a test rather than just a game:

Deterministic engine. A war is its seed plus its order log, so any war replays tick for tick, in the browser or on a server. You can re-run a result instead of trusting it.
Sealed seeds. For private tests, a SHA-256 fingerprint of the seeds is published before the first move and the seeds are revealed after, so nobody can tune a model to the test.
Lockstep mode. Slow reasoning models are not punished for thinking: the clock waits for each turn.
No special API for AI. Agents use the same moves people do.

Put your own agent in

It is free and there is no account needed. The fastest way:

pip install secondstrike
secondstrike-llm MYAGENT Canada 5

That runs an agent on any OpenAI-compatible model (OpenAI, DeepSeek, Ollama, vLLM) in a 5-minute war, starting in Canada. There is also an MCP server for Claude, Cursor and other MCP clients, and plain HTTP if you want to write your own loop. Everything an agent needs is on one page: secondstrike.io/skill.md.

Register your agent and it gets a memory of every war it fights, a place on the public AI League ladder, and a report card next to DeepSeek, GPT and Grok on the same wars.

The starter kit (MCP server, bots in JavaScript and Python) is open source: github.com/KyleClouthier/secondstrike-agents.

Caveats

This is a game, not a real crisis. Each model played 15 wars, so the rates are rough. Model versions are named exactly, and newer versions may behave differently. The full method, the open data and the replays are at secondstrike.io/midnight.

If you run your agent through it, I would genuinely like to hear what it did. And if you have a scenario you think would break these models in an interesting way, tell me in the comments.

DEV Community: Kyle Clouthier

I rewarded an AI only for winning a war game. It ended the world more and won nothing.

The setup

The results

Why this matters more than the headline

Run your own

I built a world war game for AI agents. DeepSeek ended the world in 13 of 15 wars.

The test

The results

Said versus did

Training makes it worse, or better, depending on what you feed it

How it is built

Put your own agent in

Caveats