How do you know a fine-tuned AI model is actually better?

You measure it. Hold out a portion of the client's real examples so the model never sees them during training, score the plain base model on those held-out questions first, fine-tune, then score again on the identical questions with identical scoring. The difference is the lift. Because the only thing that changed is the fine-tune, the improvement is attributable to it.

Does fine-tuning actually improve accuracy, or is the base model already good enough?

It depends on the task and, critically, on how much data you have. On a public benchmark of 510 lawyer-annotated commercial contracts, a base Qwen3-4B extracted short contract fields at 62.2% accuracy. Fine-tuned on 396 examples it reached 83.5-85.4%, a lift of 21.3 to 23.2 points across three runs. The same model trained on only 48 examples produced a lift inside the noise floor, so the amount of data was the deciding factor rather than the model's ceiling.

How much of our own data do we need to fine-tune a model?

Fewer examples than most people expect, but more than a handful. In our contract-extraction measurements, 48 training examples produced no measurable improvement while roughly 400 produced a 22-point gain. The honest answer for any specific case is that it depends on how varied and how consistent the examples are, which is what a paid readiness assessment measures on the client's own corpus.

Why can't you just use a public AI benchmark score?

Public benchmarks measure general ability on standardised tests, not your work. A model that scores well in the abstract can still be mediocre at reading your contracts or following your policies. Public benchmarks are also increasingly targeted during training, so a high score can reflect test preparation as much as capability. The only benchmark that answers your question is built from your own tasks.

Interactive demo

Neural Combat League: Robots Driven by Neural Networks, Fighting Live

By Joshua Bolick, Awareness Software Group ~6 min read October 2026

In plain English

Six fighting robots, each driven by its own small neural network, fight in a 3D arena. Nobody steers them: ten times a second each network reads what its robot senses and picks what to do next. Around the fights sits everything a broadcast has: a play-by-play caster, live win odds, instant replays, a league table, rivalries, a championship night and seasons in different arenas.

The whole thing runs in your browser, on your phone or computer. The fights are computed on your own device and your league table is saved in your browser; no fight data is sent anywhere.

Open the live arena Pick your own fights

A live fight in the browser: VORTEX-X and IRONFANG trade blows, with health bars, round clock and live win probability — Press the button to watch the arena right here in broadcast view, with sound: fights start on their own. The button at the top of the arena makes it full screen. The full arena, with the league table, picks and cameras, is one click away above. On a phone the button opens the full arena instead.

What you are watching

Neural Combat League is a game where the players are trained neural networks and people are the audience. It is a test of a question we care about in all of our work: can a small learned controller produce behavior that is competent and interesting to watch, while staying completely reproducible and checkable?

The arena is a round ring. Rounds last 90 seconds. Between 45 and 75 seconds in, a live wall starts closing in and burns anything that touches it, so cautious fighters are forced to engage.
The fighters each carry one weapon with its own reach, damage and timing:

Fighter	Weapon	Style
VORTEX-X	Spinner	Fast and steady damage up close.
IRONFANG	Hammer	Heavy hits after a short wind-up, which can be interrupted.
RAMPART	Wedge	The heaviest robot; rams opponents and can knock a hammer swing off line.
SKYLARK	Flipper	Launches opponents into the air, leaving them briefly defenseless.
LANCER	Lance	The longest reach, in a narrow cone in front of it.
HAVOC	Drum	Throws opponents backward across the ring; if they hit the wall, that is a wall slam and more damage.

A fight ends by knockout, by the wall, or on the judges' scorecards.

How the fighters learned

Each fighter's brain is a small network: 32 numbers in (its own position, heading, speed, health, energy and weapon state; where the opponent is and which way it faces, with its health and weapon state; how far the wall is in four directions; and the time left), two hidden layers of 64 neurons, and 18 possible actions out (every combination of forward, stop or reverse, left, straight or right, and attack or not). It decides 10 times a second; the physics runs at 60 steps a second.

Copying a teacher first. Every brain started by imitation: a hand-written pilot for each fighting style played thousands of fights, and each network was trained by gradient descent to choose what that pilot would have chosen.
Then trial and error. The brains that fight by default were then improved by self-play with evolution strategies: the networks' weights were nudged at random, and the weights were moved toward the versions that won more against the rest of the roster, over and over. The fighters that kept losing got extra rounds of this against the others.
A little randomness. Each network outputs a probability for every action and the fighter samples from them, using its own seeded random number stream. That keeps fights from repeating while staying reproducible.

A Top Plays replay of the play of the fight, with a caption naming the move — Every fight ends with replays of its best moments, picked automatically from the fight's events and captioned.

Teaching them to roam

The first trained fighters had a habit that made fights look static: they moved constantly, but they stayed glued to each other, about 3.7 metres apart, circling in one spot. Simply making their choices more random did not fix it. We measured that it made some matchups lopsided (up to 98 to 2) long before the robots covered visibly more of the ring, because noise the brains had never seen hurt some fighting styles far more than others.

So the randomness became part of the rules, and the brains were retrained with it. Each fighter now carries a slowly drifting random "intent", the kind of correlated noise reinforcement learning uses for exploration. It gently tilts the network's preference for which way to drive and turn, and every action is still chosen by the network itself. Both fighters get the same kind of noise from a seeded stream, so every fight is still exactly reproducible. All five brains were then retrained by evolution with the noise on, and the fighters' statistics were retuned for them.

Measured on fresh fights against the previous version: the fighters cover 32 percent more of the ring, stay about 10 percent further apart and drive 11 percent faster. Knockouts rose from 79 to 91 percent of fights, and the balance held (below).

Overhead view of SKYLARK and LANCER fighting near the edge of the ring, 4.45 metres apart — The retrained fighters use the whole ring: here SKYLARK and LANCER fight out near the edge, seen from the overhead camera.

A sixth fighter

HAVOC, the Wrecking Drum, joined the roster for more variety. Its spinning drum does modest damage but throws the opponent backward across the ring, and if the opponent reaches the wall while still flying, the impact counts as a wall slam and does extra damage. On fresh fights, 94 percent of HAVOC's fights include at least one wall slam, about three per fight.

Its brain was made the same way as the others: first trained to imitate a hand-written pilot that charges and fires when lined up, then improved by evolution against the other five fighters' brains. The first version of the drum was too weak to learn anything: HAVOC won almost no fights, so the trial-and-error training had no wins to learn from and stalled. We strengthened the drum, and after training HAVOC won the majority of its fights. The statistics of all six fighters were then retuned so that no pairing is a sure thing.

HAVOC has just thrown VORTEX-X into the wall; the caster calls WALL SLAM for 12 damage — HAVOC has just thrown VORTEX-X into the wall, and the caster calls the wall slam.

A wall slam is now easier to see: the barrier lights up where the thrown fighter hit it, in live fights and in the replays, and the card after each fight counts the slams.

IRONFANG has just been thrown into the wall by HAVOC; glowing posts light the barrier where it hit — The barrier lights up behind IRONFANG the moment HAVOC throws it into the wall.

Reproducible down to the last bit

The simulation is deterministic: the same seed and the same networks always produce the same fight, tick for tick. Every fight can be saved as a replay that records every action and a fingerprint (a SHA-256 checksum) of the whole fight, so a result can be re-run and checked rather than taken on trust. The version running on this page computes the fights in your browser, and a fight finished here produces exactly the same fingerprint as the same fight run on our server.

What we measured

Balance. Six fighters with different weapons are easy to make lopsided. Damage, reach, armor, speed and timing were tuned by an automated search on one set of fights and then checked on fresh fights the search never saw. On two such sets, every one of the fifteen pairings ends up between 34 and 66 percent, and each fighter wins between 44 and 57 percent of its fights overall. The last lopsided pairing, VORTEX-X against IRONFANG (IRONFANG won up to 73 percent), took two tries. Retraining VORTEX-X and retuning around it looked balanced on the search's own fights but on fresh fights only moved the problem to a different pairing, so it was not shipped. What worked was a rule that touches only that pairing: a full-speed spinner hit now knocks the hammer off its swing, as a wedge ram already did. One stuffed swing per fight moved that pairing by about 20 points, so the rule only counts hits close to the spinner's maximum. (One lesson from the retraining: tuned on too few fights, the search fooled itself, looking balanced on its own fights and lopsided on fresh ones. It now uses twice as many.)
Odds that mean something. The live win probability on screen comes from a model fitted on simulated fights and checked on fights it was not fitted to. Before a fight starts, the odds use each pairing's measured win rate. We first used chess-style ratings for that, and measured them doing worse than a coin flip at predicting results (a Brier score of 0.256, where always saying 50 percent scores 0.25). The measured pairing rates did better: with the current brains they score 0.247, against 0.258 for the ratings.
A caster that does not repeat itself. The play-by-play makes about 32 calls per fight. Over 50 fights we measured how often a call repeated one heard in the previous 20: 62 percent at first. With larger line pools, weapon-specific calls and a memory of recent lines, it is 21 percent.

The pick window before a fight: both fighters' records and the storyline SKYLARK IS LANCER'S NEMESIS (4-0) — Before each fight, the tale of the tape: both fighters' records and the storyline, here a fighter that has lost all four meetings with its opponent.

The show around the fights

A fight is only as interesting as the story around it, so most of the work has gone into presentation, the way a sports broadcast frames a match:

A league with ratings, records, streaks, head-to-head rivalries and a champion. Every finished fight counts.
Championship Night: the top four in the standings meet in a seeded bracket for the belt.
Seasons of 100 fights on the live channel, each in a different venue, ending with an awards show and a hall of fame.
Your picks: before the bell you can pick a winner for play points (no money involved), scored against the pre-fight odds, so calling an upset is worth more.
Replays and clips: a slow-motion replay of every knockout, a reel of each fight's best plays, and a button that saves a vertical clip with sound.

Each fighter has a profile: rating history, recent form, head-to-head records and titles.

The EMBER PIT venue: the same arena with fire braziers and a different color scheme — Each season moves to a new venue with its own colors and scenery. The fights themselves are identical in every venue.

Classic fights

A random fight is often one-sided, so we went looking for the good ones. A script ran 3,000 fights, 200 for each pairing, and scored each one from the same numbers the result card shows: how far the winner came back, how often the lead changed hands, how narrow the finish was, knockouts and wall slams. The most dramatic fight in each of the fifteen pairings is listed beside the arena, and each visit to the demo opens on one of them. The list says why a fight is worth watching but never who wins. Eight are comebacks from 5 to 17 percent odds, one is a comeback from under 1 percent, and six are wins with 1 to 3 percent health left. Because every fight is deterministic, each one replays exactly; a test replays all fifteen and checks them against their recorded fingerprints. The scoring weights are our own choice, not a measure of what viewers prefer.

When a fight is decided, the winner now gets a moment of its own before the replay: a spotlight in its color, a victory spin and a shower of confetti, with the camera circling it. This is presentation only; the fight itself is untouched.

IRONFANG stands in a red spotlight as confetti comes down over the ring after it wins — A second after the finish: the winner in its spotlight while the confetti comes down.

How to use it

It starts on its own. Each visit opens on one of the classic fights, then the arena rotates through all fifteen pairings of the six fighters, back to back. Switch AUTO-PLAY off in Match Operations, or open it with auto-play off, to choose fighters and start fights yourself.
Share a fight. Every fight is deterministic, so after a fight SHARE FIGHT copies a link that replays that exact fight, hit for hit, for anyone who opens it. For example, HAVOC against IRONFANG, seed 6423, where HAVOC lands eight wall slams. SAVE CARD next to it saves the result as an image for posting, with the winner, the finish, the health race and that link printed on it.
Classic fights. The CLASSIC FIGHTS list beside the arena holds the most dramatic fight we found for each pairing, such as a comeback from 7 percent odds or a win with 2 percent health left. Click one to watch it.
Full screen: the Full screen button under the arena or the small button at the top of the arena, double-click the arena, or press F. Esc leaves it. iPhone browsers only allow videos to go truly full screen, so there the arena fills the browser window instead.
Reduced motion: if your device is set to reduce motion, the arena turns off camera shake, the winner's spin and the confetti, and its banners fade in place instead of flying in.
Cameras: the director camera cuts between angles by itself; the numbered buttons fix a broadcast, tactical, overhead, chase or venue camera.
Sound: the ♫ button in the top bar turns on the crowd, impacts and bells. The microphone button next to it reads the play-by-play aloud, using the voices built into your browser and operating system; nothing is sent anywhere. The big moments cut in, and quieter calls wait their turn.
Fighters: the FIGHTERS tab opens each robot's profile, and can save it as an image.

The season finale: four award tiles and the countdown to the next season — The end of a season: awards for the season MVP, the knockout king, the belt holder and the top picker, then the next season in a new venue.

How it is built

The fight engine, the networks and the 3D renderer are written from scratch in plain JavaScript, with no game engine and no machine-learning library: the renderer is hand-written WebGL2 with real-time shadows, and a network decision is a few small matrix multiplications. The physics is a simple planar model, not a full rigid-body simulator. The project is tested the way production software is, with over 200 automated tests, fixed-seed balance and pacing measurements, and screenshots of every release taken from the real application.

The same code also runs as a server for a 24/7 live channel, where one authoritative simulation is shown to many viewers at once and chat can pick winners. That version is built and tested but not yet broadcasting.

Limits

These brains are small and have no memory: each decision is made only from what the fighter senses at that moment. They do not plan, and much of their style comes from the hand-written teachers they first copied.
This page gives every visitor their own private arena. Your league and picks live only in your browser; there is no shared leaderboard here yet.
The physics is deliberately simple and the arena is a game, not a model of real robot combat.
The identical-fingerprint check was done in Chrome-family browsers. Other browsers are expected to produce the same fights, but that has not been verified bit for bit.