https://awarenesssoftwaregroup.com/research/ Open research on how AI models actually learn. One finding per entry, in plain English first and in full technical detail underneath, including the approaches that turned out not to work. en Tue, 29 Sep 2026 03:20:12 +0000 https://awarenesssoftwaregroup.com/research/a-delay-by-most-measures/ https://awarenesssoftwaregroup.com/research/a-delay-by-most-measures/ The easy-sum warm-up that looked like a trap mostly delayed learning: given longer, most runs learned the hard sum, several thousand steps later than Tue, 29 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-learning-rate-warm-up-does-the-job/ https://awarenesssoftwaregroup.com/research/a-learning-rate-warm-up-does-the-job/ Starting training with small steps and ramping up did as well as our best easy-data warm-up, and better than the easy-task mix. Tue, 29 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-sweet-spot-not-the-nearest/ https://awarenesssoftwaregroup.com/research/a-sweet-spot-not-the-nearest/ Before a hard memory task, the best single warm-up was a version easier by a clear margin. Tue, 29 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-task-battery-with-designed-headroom/ https://awarenesssoftwaregroup.com/research/a-task-battery-with-designed-headroom/ Our quality checks all passed because the benchmark scored 99% and nothing could fail. Tue, 29 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/against-the-best-plain-run-not-established/ https://awarenesssoftwaregroup.com/research/against-the-best-plain-run-not-established/ On the second task, starting with easier sums solved the hard sum on 11 of 16 runs against 6 for the best ordinary recipe, but the accuracy difference is Tue, 29 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/easier-data-adds-nothing-to-a-rate-warm-up/ https://awarenesssoftwaregroup.com/research/easier-data-adds-nothing-to-a-rate-warm-up/ Combined with a standard learning-rate warm-up, our best easy-data warm-up added nothing. The best result of the whole study needed no special data at all. Tue, 29 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/it-replicates-on-a-second-target/ https://awarenesssoftwaregroup.com/research/it-replicates-on-a-second-target/ On a different hard sum, the easy-sum mix again beat the best ordinary recipe on fresh runs, solving it on 25 of 32 runs against 2. Tue, 29 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/no-other-claim-ran-hot/ https://awarenesssoftwaregroup.com/research/no-other-claim-ran-hot/ We checked every standing speed-up result in the archive for the mistake that undid our warm-up results. Only the warm-up results themselves had made it. Tue, 29 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/on-fresh-seeds-the-mixture-wins/ https://awarenesssoftwaregroup.com/research/on-fresh-seeds-the-mixture-wins/ On the adding task, starting with a mix of easier sums beat the best ordinary recipe, learning-rate warm-up included, on 32 new runs: 91 percent against Tue, 29 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/one-nearby-version-does-most-of-it/ https://awarenesssoftwaregroup.com/research/one-nearby-version-does-most-of-it/ Warming up on a single easier version close to the hard task gave most of the benefit, on every run. Mixing several easier versions added at most a little. Tue, 29 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/order-can-make-the-warm-up-harmful/ https://awarenesssoftwaregroup.com/research/order-can-make-the-warm-up-harmful/ The same three easier sums helped a small model when shuffled and stopped it learning the hard sum at all when given from hardest to easiest. Tue, 29 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/order-did-not-help-mixing-might/ https://awarenesssoftwaregroup.com/research/order-did-not-help-mixing-might/ Working up from easy to hard did not speed learning at matched compute; a random mix of easier tasks early ended clearly higher, which we are now testing Tue, 29 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/questions/ https://awarenesssoftwaregroup.com/research/questions/ The 10 questions our public AI training research is organised around, each with its current answer and the published results behind it. Tue, 29 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/slower-than-tuned-not-hotter/ https://awarenesssoftwaregroup.com/research/slower-than-tuned-not-hotter/ A check on one earlier result: its task learns fastest at twice the learning rate it was run at, so it did not have the too-large-rate problem. Tue, 29 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/the-formula-could-have-failed/ https://awarenesssoftwaregroup.com/research/the-formula-could-have-failed/ Our formula for how long to train a model you borrow from had never faced a test it could fail. We built one. Tue, 29 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/the-mixture-gets-there-sooner/ https://awarenesssoftwaregroup.com/research/the-mixture-gets-there-sooner/ Given twice as long, ordinary training on the adding task mostly caught up. What the easy-sum mix really did was solve it in about a third of the steps. Tue, 29 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/tuned-on-its-own-target-plain-never-solves-it/ https://awarenesssoftwaregroup.com/research/tuned-on-its-own-target-plain-never-solves-it/ Given four ordinary settings tuned on the second hard sum itself, plain training solved it on none of 16 runs; the easy-sum mix solved it on 12. Tue, 29 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/two-easy-sums-are-traps/ https://awarenesssoftwaregroup.com/research/two-easy-sums-are-traps/ Warming up on either of the two easiest sums alone stopped a small model learning the hard sum at all. Tue, 29 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-clock-not-a-warning/ https://awarenesssoftwaregroup.com/research/a-clock-not-a-warning/ Moving the moment a model learns, its approach to the edge of chaos moved too, but always a fixed fraction of the way there: it tracks training progress, Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-higher-ceiling-not-a-head-start/ https://awarenesssoftwaregroup.com/research/a-higher-ceiling-not-a-head-start/ Given more training, models that started on a random mix of easier tasks reached 94 percent on the hard task while models trained on it alone stalled at Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-short-warm-up-is-enough/ https://awarenesssoftwaregroup.com/research/a-short-warm-up-is-enough/ Spending only the first 6 percent of training on a random mix of easier tasks lifted a small model from 58 to 83 percent on a hard one. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-speed-accuracy-frontier/ https://awarenesssoftwaregroup.com/research/a-speed-accuracy-frontier/ Faster learning rates made small models learn sooner but end less accurate. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/at-width-96-the-gap-holds-steady/ https://awarenesssoftwaregroup.com/research/at-width-96-the-gap-holds-steady/ In a model twice as wide, trained 2.5 times longer, the easy-task warm-up stayed about five points ahead but did not pull further away, unlike in the Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/being-choosy-about-data-costs-more-than-it-saves/ https://awarenesssoftwaregroup.com/research/being-choosy-about-data-costs-more-than-it-saves/ Inspect four batches, train only on the most informative. It sounds efficient. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/bigger-batch-higher-limit/ https://awarenesssoftwaregroup.com/research/bigger-batch-higher-limit/ The fastest learning rate a small model can tolerate roughly triples from batch 16 to batch 256, consistent with the common square-root rule. Mon, 28 Sep 2026 00:00:00 +0000