https://awarenesssoftwaregroup.com/research/ Open research on how AI models actually learn. One finding per entry, in plain English first and in full technical detail underneath, including the approaches that turned out not to work. en Mon, 28 Sep 2026 02:57:23 +0000 https://awarenesssoftwaregroup.com/research/a-clock-not-a-warning/ https://awarenesssoftwaregroup.com/research/a-clock-not-a-warning/ Moving the moment a model learns, its approach to the edge of chaos moved too, but always a fixed fraction of the way there: it tracks training progress, Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/boost-then-decay/ https://awarenesssoftwaregroup.com/research/boost-then-decay/ A learning-rate schedule that speeds up early and slows down late made small models learn as fast as our boost and end as accurate as the standard Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/early-towards-the-edge/ https://awarenesssoftwaregroup.com/research/early-towards-the-edge/ A small model's recurrence moves from very stable towards the edge of chaos early in training, half-way at the same step in every run, long before it Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/fast-or-accurate/ https://awarenesssoftwaregroup.com/research/fast-or-accurate/ Our short early learning-rate boost made small models learn fastest; the standard warm-up-and-decay schedule made them end the most accurate. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/grow-the-model-save-a-quarter/ https://awarenesssoftwaregroup.com/research/grow-the-model-save-a-quarter/ Starting a model small and doubling it twice reached the target with 26 percent less compute than training at full size, and when to grow barely mattered. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/growth-did-not-travel/ https://awarenesssoftwaregroup.com/research/growth-did-not-travel/ The model-growth recipe that saved a quarter of the compute on one task rarely reached the target on a second, slower task, because its timings were set Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/is-learning-a-lucky-accident/ https://awarenesssoftwaregroup.com/research/is-learning-a-lucky-accident/ Testing whether sudden learning is randomness knocking a model out of a rut. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/it-tries-the-wrong-rule-first/ https://awarenesssoftwaregroup.com/research/it-tries-the-wrong-rule-first/ Instead of asking how often the model is right, we asked what it is doing. Early on it behaves like a simpler rule that does not work. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/late-not-delayed/ https://awarenesssoftwaregroup.com/research/late-not-delayed/ Run three times longer with little data, a small model learned the rule late but always while fitting its examples, never after. No grokking. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/learning-it-again-is-not-like-learning-it/ https://awarenesssoftwaregroup.com/research/learning-it-again-is-not-like-learning-it/ We changed the task underneath a trained model and watched it learn again. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/memorised-never-generalised/ https://awarenesssoftwaregroup.com/research/memorised-never-generalised/ Trained repeatedly on a small fixed set of addition problems, a small model memorised them in a hundred steps and never solved a new one in ten thousand. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/no-delay-at-the-edge/ https://awarenesssoftwaregroup.com/research/no-delay-at-the-edge/ With enough training data to learn the rule at all, a small model learned it while fitting its data, never long after. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/only-as-good-as-its-forecast/ https://awarenesssoftwaregroup.com/research/only-as-good-as-its-forecast/ A free formula timed a training intervention nearly as well as watching the run, for model sizes inside its range, and no better than chance at the edges. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/questions/ https://awarenesssoftwaregroup.com/research/questions/ The 10 questions our public AI training research is organised around, each with its current answer and the published results behind it. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/the-frame-decides-the-sign/ https://awarenesssoftwaregroup.com/research/the-frame-decides-the-sign/ Whether bigger models reorganise less while learning depends on which of two standard measures you use: one rises with size, the other falls. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/the-habit-it-did-not-need/ https://awarenesssoftwaregroup.com/research/the-habit-it-did-not-need/ Early on the model falls into a simple wrong habit. We took the habit away. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/the-scoring-made-the-steps/ https://awarenesssoftwaregroup.com/research/the-scoring-made-the-steps/ We thought our way of keeping score was blurring distinct stages of learning together. It turned out to be creating them. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/the-training-curve-cannot-tell-you/ https://awarenesssoftwaregroup.com/research/the-training-curve-cannot-tell-you/ Two models with near-identical training curves, one building machinery and one polishing machinery it was handed. Only looking inside separates them. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/we-gave-it-the-answer-anyway/ https://awarenesssoftwaregroup.com/research/we-gave-it-the-answer-anyway/ We wired a working shortcut straight into the model. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-critical-amount-per-step/ https://awarenesssoftwaregroup.com/research/a-critical-amount-per-step/ Learning time follows one curve in material per training step, and the standard critical batch size formula fits it within 2.2 percent across an 86-fold Sun, 27 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-rise-not-a-peak/ https://awarenesssoftwaregroup.com/research/a-rise-not-a-peak/ Five independently trained copies of a public language model all reorganise their internal state as they learn to copy, then level off. Sun, 27 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-speed-accuracy-frontier/ https://awarenesssoftwaregroup.com/research/a-speed-accuracy-frontier/ Faster learning rates made small models learn sooner but end less accurate. Sun, 27 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-task-battery-with-designed-headroom/ https://awarenesssoftwaregroup.com/research/a-task-battery-with-designed-headroom/ Our quality checks all passed because the benchmark scored 99% and nothing could fail. Sun, 27 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-third-was-model-size/ https://awarenesssoftwaregroup.com/research/a-third-was-model-size/ With every model built the same size, a bigger vocabulary still slows learning, but about a third less than an earlier estimate that let model size change Sun, 27 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/batch-and-length-are-substitutes/ https://awarenesssoftwaregroup.com/research/batch-and-length-are-substitutes/ Bigger batches and longer sequences both speed learning, but each helps less when the other is large. A law that treats them as independent misses that. Sun, 27 Sep 2026 00:00:00 +0000