https://awarenesssoftwaregroup.com/research/ Open research on how AI models actually learn. One finding per entry, in plain English first and in full technical detail underneath, including the approaches that turned out not to work. en Mon, 28 Sep 2026 13:30:48 +0000 https://awarenesssoftwaregroup.com/research/a-clock-not-a-warning/ https://awarenesssoftwaregroup.com/research/a-clock-not-a-warning/ Moving the moment a model learns, its approach to the edge of chaos moved too, but always a fixed fraction of the way there: it tracks training progress, Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-higher-ceiling-not-a-head-start/ https://awarenesssoftwaregroup.com/research/a-higher-ceiling-not-a-head-start/ Given more training, models that started on a random mix of easier tasks reached 94 percent on the hard task while models trained on it alone stalled at Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-short-warm-up-is-enough/ https://awarenesssoftwaregroup.com/research/a-short-warm-up-is-enough/ Spending only the first 6 percent of training on a random mix of easier tasks lifted a small model from 58 to 83 percent on a hard one. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-speed-accuracy-frontier/ https://awarenesssoftwaregroup.com/research/a-speed-accuracy-frontier/ Faster learning rates made small models learn sooner but end less accurate. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-task-battery-with-designed-headroom/ https://awarenesssoftwaregroup.com/research/a-task-battery-with-designed-headroom/ Our quality checks all passed because the benchmark scored 99% and nothing could fail. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/being-choosy-about-data-costs-more-than-it-saves/ https://awarenesssoftwaregroup.com/research/being-choosy-about-data-costs-more-than-it-saves/ Inspect four batches, train only on the most informative. It sounds efficient. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/bigger-batch-higher-limit/ https://awarenesssoftwaregroup.com/research/bigger-batch-higher-limit/ The fastest learning rate a small model can tolerate roughly triples from batch 16 to batch 256, consistent with the common square-root rule. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/boost-then-decay/ https://awarenesssoftwaregroup.com/research/boost-then-decay/ A learning-rate schedule that speeds up early and slows down late made small models learn as fast as our boost and end as accurate as the standard Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/early-towards-the-edge/ https://awarenesssoftwaregroup.com/research/early-towards-the-edge/ A small model's recurrence moves from very stable towards the edge of chaos early in training, half-way at the same step in every run, long before it Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/fast-or-accurate/ https://awarenesssoftwaregroup.com/research/fast-or-accurate/ Our short early learning-rate boost made small models learn fastest; the standard warm-up-and-decay schedule made them end the most accurate. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/grow-the-model-save-a-quarter/ https://awarenesssoftwaregroup.com/research/grow-the-model-save-a-quarter/ Starting a model small and doubling it twice reached the target with 26 percent less compute than training at full size, and when to grow barely mattered. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/growth-did-not-travel/ https://awarenesssoftwaregroup.com/research/growth-did-not-travel/ The model-growth recipe that saved a quarter of the compute on one task rarely reached the target on a second, slower task, because its timings were set Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/growth-saves-nothing-here/ https://awarenesssoftwaregroup.com/research/growth-saves-nothing-here/ With its timing rescaled, the model-growth recipe still saved nothing on a second task: a medium fixed-size model was as cheap and more reliable. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/is-learning-a-lucky-accident/ https://awarenesssoftwaregroup.com/research/is-learning-a-lucky-accident/ Testing whether sudden learning is randomness knocking a model out of a rut. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/it-already-knew-the-order/ https://awarenesssoftwaregroup.com/research/it-already-knew-the-order/ We found the order in which a model learns three related skills, then tried teaching them in that order. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/it-tries-the-wrong-rule-first/ https://awarenesssoftwaregroup.com/research/it-tries-the-wrong-rule-first/ Instead of asking how often the model is right, we asked what it is doing. Early on it behaves like a simpler rule that does not work. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/late-not-delayed/ https://awarenesssoftwaregroup.com/research/late-not-delayed/ Run three times longer with little data, a small model learned the rule late but always while fitting its examples, never after. No grokking. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/learning-it-again-is-not-like-learning-it/ https://awarenesssoftwaregroup.com/research/learning-it-again-is-not-like-learning-it/ We changed the task underneath a trained model and watched it learn again. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/longer-memory-lower-limit/ https://awarenesssoftwaregroup.com/research/longer-memory-lower-limit/ The fastest learning rate at which a small recurrent model still learns falls as a power of how far back it must remember, as a 2025 paper predicts. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/memorised-never-generalised/ https://awarenesssoftwaregroup.com/research/memorised-never-generalised/ Trained repeatedly on a small fixed set of addition problems, a small model memorised them in a hundred steps and never solved a new one in ten thousand. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/mixing-first-lifts-the-ceiling/ https://awarenesssoftwaregroup.com/research/mixing-first-lifts-the-ceiling/ Spending the first quarter of training on a random mix of easier tasks left a small model 21 points more accurate on the hard task, confirmed on fresh Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/mixing-first-makes-it-learnable/ https://awarenesssoftwaregroup.com/research/mixing-first-makes-it-learnable/ On a second task, a random mix of easier versions early made a hard task learnable: six runs in eight learned it, against one in eight trained on it Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/no-delay-at-the-edge/ https://awarenesssoftwaregroup.com/research/no-delay-at-the-edge/ With enough training data to learn the rule at all, a small model learned it while fitting its data, never long after. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/one-hundred-steps-is-enough/ https://awarenesssoftwaregroup.com/research/one-hundred-steps-is-enough/ A burst of easier examples for just the first 100 of 6,000 training steps lifted a small model from 58 to 82 percent on a hard task, on every run. Mon, 28 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/one-jump-or-several/ https://awarenesssoftwaregroup.com/research/one-jump-or-several/ Averaging accuracy across positions hid four separate events. Mon, 28 Sep 2026 00:00:00 +0000