https://awarenesssoftwaregroup.com/research/ Open research on how AI models actually learn. One finding per entry, in plain English first and in full technical detail underneath, including the approaches that turned out not to work. en Mon, 28 Sep 2026 01:29:39 +0000 https://awarenesssoftwaregroup.com/research/boost-then-decay/ https://awarenesssoftwaregroup.com/research/boost-then-decay/ A learning-rate schedule that speeds up early and slows down late made small models learn as fast as our boost and end as accurate as the standard Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/fast-or-accurate/ https://awarenesssoftwaregroup.com/research/fast-or-accurate/ Our short early learning-rate boost made small models learn fastest; the standard warm-up-and-decay schedule made them end the most accurate. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/grow-the-model-save-a-quarter/ https://awarenesssoftwaregroup.com/research/grow-the-model-save-a-quarter/ Starting a model small and doubling it twice reached the target with 26 percent less compute than training at full size, and when to grow barely mattered. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/growth-did-not-travel/ https://awarenesssoftwaregroup.com/research/growth-did-not-travel/ The model-growth recipe that saved a quarter of the compute on one task rarely reached the target on a second, slower task, because its timings were set Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/memorised-never-generalised/ https://awarenesssoftwaregroup.com/research/memorised-never-generalised/ Trained repeatedly on a small fixed set of addition problems, a small model memorised them in a hundred steps and never solved a new one in ten thousand. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/no-delay-at-the-edge/ https://awarenesssoftwaregroup.com/research/no-delay-at-the-edge/ With enough training data to learn the rule at all, a small model learned it while fitting its data, never long after. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/only-as-good-as-its-forecast/ https://awarenesssoftwaregroup.com/research/only-as-good-as-its-forecast/ A free formula timed a training intervention nearly as well as watching the run, for model sizes inside its range, and no better than chance at the edges. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/questions/ https://awarenesssoftwaregroup.com/research/questions/ The 10 questions our public AI training research is organised around, each with its current answer and the published results behind it. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/the-frame-decides-the-sign/ https://awarenesssoftwaregroup.com/research/the-frame-decides-the-sign/ Whether bigger models reorganise less while learning depends on which of two standard measures you use: one rises with size, the other falls. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/the-scoring-made-the-steps/ https://awarenesssoftwaregroup.com/research/the-scoring-made-the-steps/ We thought our way of keeping score was blurring distinct stages of learning together. It turned out to be creating them. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/the-training-curve-cannot-tell-you/ https://awarenesssoftwaregroup.com/research/the-training-curve-cannot-tell-you/ Two models with near-identical training curves, one building machinery and one polishing machinery it was handed. Only looking inside separates them. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-critical-amount-per-step/ https://awarenesssoftwaregroup.com/research/a-critical-amount-per-step/ Learning time follows one curve in material per training step, and the standard critical batch size formula fits it within 2.2 percent across an 86-fold Sun, 27 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-rise-not-a-peak/ https://awarenesssoftwaregroup.com/research/a-rise-not-a-peak/ Five independently trained copies of a public language model all reorganise their internal state as they learn to copy, then level off. Sun, 27 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-speed-accuracy-frontier/ https://awarenesssoftwaregroup.com/research/a-speed-accuracy-frontier/ Faster learning rates made small models learn sooner but end less accurate. Sun, 27 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-task-battery-with-designed-headroom/ https://awarenesssoftwaregroup.com/research/a-task-battery-with-designed-headroom/ Our quality checks all passed because the benchmark scored 99% and nothing could fail. Sun, 27 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-third-was-model-size/ https://awarenesssoftwaregroup.com/research/a-third-was-model-size/ With every model built the same size, a bigger vocabulary still slows learning, but about a third less than an earlier estimate that let model size change Sun, 27 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/batch-and-length-are-substitutes/ https://awarenesssoftwaregroup.com/research/batch-and-length-are-substitutes/ Bigger batches and longer sequences both speed learning, but each helps less when the other is large. A law that treats them as independent misses that. Sun, 27 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/flat-through-width-256/ https://awarenesssoftwaregroup.com/research/flat-through-width-256/ Across a fourfold range of model width, an unneeded skill costs the other skills the same share of their learning time. Sun, 27 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/four-agree-one-took-a-shortcut/ https://awarenesssoftwaregroup.com/research/four-agree-one-took-a-shortcut/ Four ways of packing the same material into a training step took the same time to learn. Sun, 27 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/is-learning-a-lucky-accident/ https://awarenesssoftwaregroup.com/research/is-learning-a-lucky-accident/ Testing whether sudden learning is randomness knocking a model out of a rut. Sun, 27 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/it-sees-the-damage-not-the-repair/ https://awarenesssoftwaregroup.com/research/it-sees-the-damage-not-the-repair/ Rounding a model's weights to fewer bits damages it. Measuring its internal shape spots the damage, but a repaired model works again with a changed shape. Sun, 27 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/it-was-the-rate-not-the-width/ https://awarenesssoftwaregroup.com/research/it-was-the-rate-not-the-width/ The easing of competition between skills that seemed to come with a wider model came from its slower learning rate. Sun, 27 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/less-competition-at-width-512/ https://awarenesssoftwaregroup.com/research/less-competition-at-width-512/ With its learning rate tuned, the widest model paid a smaller share for an unneeded skill, but its rate changed along with its width, so which one Sun, 27 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/longer-sequences-did-not-help/ https://awarenesssoftwaregroup.com/research/longer-sequences-did-not-help/ Training on sequences eight times longer barely changed when a small model learned, because the extra length only repeated what each sequence already held. Sun, 27 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/most-of-it-is-pull/ https://awarenesssoftwaregroup.com/research/most-of-it-is-pull/ Teaching an unneeded skill half as often removed about half of what it cost a small model's other skills: the competition is mostly the pull of training, Sun, 27 Sep 2026 00:00:00 +0000