https://awarenesssoftwaregroup.com/research/ Open research on how AI models actually learn. One finding per entry, in plain English first and in full technical detail underneath, including the approaches that turned out not to work. en Wed, 30 Sep 2026 11:11:58 +0000 https://awarenesssoftwaregroup.com/research/a-faster-rate-and-still-slower/ https://awarenesssoftwaregroup.com/research/a-faster-rate-and-still-slower/ Tuning the easy-start recipe's own learning rate made it faster on the random lookup table, but it stayed about 1,600 steps behind ordinary training. Wed, 30 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-harder-maximum-and-it-helps-again/ https://awarenesssoftwaregroup.com/research/a-harder-maximum-and-it-helps-again/ On a harder version of the maximum task, starting on easier versions saved about a quarter of the steps, correcting our earlier reading that the speed-up Wed, 30 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-random-table-and-it-slows/ https://awarenesssoftwaregroup.com/research/a-random-table-and-it-slows/ On a random lookup table as hard as tasks where it halved the steps, starting on easier versions slowed learning instead: the easy start needs a rule to Wed, 30 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-sweet-spot-not-the-nearest/ https://awarenesssoftwaregroup.com/research/a-sweet-spot-not-the-nearest/ Before a hard memory task, the best single warm-up was a version easier by a clear margin. Wed, 30 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/differences-help-too/ https://awarenesssoftwaregroup.com/research/differences-help-too/ Warming up on easy subtraction problems sped up learning a hard addition problem almost as much as easy addition did. Wed, 30 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/for-the-maximum-it-slows-learning/ https://awarenesssoftwaregroup.com/research/for-the-maximum-it-slows-learning/ When the hard task was to output the larger of two remembered symbols, starting on easier versions slowed learning by about 1,300 steps instead of Wed, 30 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/kept-in-they-get-in-the-way/ https://awarenesssoftwaregroup.com/research/kept-in-they-get-in-the-way/ Even with the hard sum three quarters of the training data, keeping easy sums in the mix stopped every run learning it, worse than no easy sums at all. Wed, 30 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/lstm-best-rate-found/ https://awarenesssoftwaregroup.com/research/lstm-best-rate-found/ Tuned to its best learning rate, ordinary training of a small LSTM still took about 3,800 more steps than starting on easier sums, and solved the task Wed, 30 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/on-a-transformer-no-speed-up/ https://awarenesssoftwaregroup.com/research/on-a-transformer-no-speed-up/ Given enough runs and time, a small transformer learned the hard sum more often and sooner without the easy-sum warm-up. Wed, 30 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/on-a-transformer-not-established/ https://awarenesssoftwaregroup.com/research/on-a-transformer-not-established/ On a small transformer, the easy-sum start got more runs to the target, but most runs of both recipes had not got there by the end, so the result is not Wed, 30 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/on-an-lstm-it-holds/ https://awarenesssoftwaregroup.com/research/on-an-lstm-it-holds/ On a second kind of recurrent network, an LSTM, the easy-sum start solved the hard sum on 15 of 16 runs; ordinary training on 5. Wed, 30 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/predicting-the-cost-from-the-task/ https://awarenesssoftwaregroup.com/research/predicting-the-cost-from-the-task/ What it costs to take something away from a model turns out to be set by the structure of the exercise, not by the model. Change the exercise and it moves. Wed, 30 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/questions/ https://awarenesssoftwaregroup.com/research/questions/ The 10 questions our public AI training research is organised around, each with its current answer and the published results behind it. Wed, 30 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/subtraction-too/ https://awarenesssoftwaregroup.com/research/subtraction-too/ With subtraction instead of addition as the hard task, starting on easier subtractions still roughly halved the training steps needed. Wed, 30 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/the-first-piece-is-the-load-bearing-one/ https://awarenesssoftwaregroup.com/research/the-first-piece-is-the-load-bearing-one/ We expected the hardest part of a task to be the one you cannot take away from a model. It is the easiest part, the one it learns first. Wed, 30 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/the-operation-and-the-component/ https://awarenesssoftwaregroup.com/research/the-operation-and-the-component/ Easy sums sharing nothing with the hard sum still sped up learning, and easy sums sharing a part of it sped it up about as much again. Wed, 30 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/tuned-on-its-own-target-plain-never-solves-it/ https://awarenesssoftwaregroup.com/research/tuned-on-its-own-target-plain-never-solves-it/ Given four ordinary settings tuned on the second hard sum itself, plain training solved it on none of 16 runs; the easy-sum mix solved it on 12. Wed, 30 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/xor-halves-the-steps-too/ https://awarenesssoftwaregroup.com/research/xor-halves-the-steps-too/ With bitwise XOR of two remembered symbols as the task, starting on easier versions reached the solution in about half the steps of ordinary training. Wed, 30 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/xor-tuned-and-it-still-halves/ https://awarenesssoftwaregroup.com/research/xor-tuned-and-it-still-halves/ Ordinary training on the XOR task was tuned to its best learning rate, and starting on easier versions still reached the solution in about half the steps. Wed, 30 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-delay-by-most-measures/ https://awarenesssoftwaregroup.com/research/a-delay-by-most-measures/ The easy-sum warm-up that looked like a trap mostly delayed learning: given longer, most runs learned the hard sum, several thousand steps later than Tue, 29 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-higher-ceiling-not-a-head-start/ https://awarenesssoftwaregroup.com/research/a-higher-ceiling-not-a-head-start/ Given more training, models that started on a random mix of easier tasks reached 94 percent on the hard task while models trained on it alone stalled at Tue, 29 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-learning-rate-warm-up-does-the-job/ https://awarenesssoftwaregroup.com/research/a-learning-rate-warm-up-does-the-job/ Starting training with small steps and ramping up did as well as our best easy-data warm-up, and better than the easy-task mix. Tue, 29 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-task-battery-with-designed-headroom/ https://awarenesssoftwaregroup.com/research/a-task-battery-with-designed-headroom/ Our quality checks all passed because the benchmark scored 99% and nothing could fail. Tue, 29 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/against-the-best-plain-run-not-established/ https://awarenesssoftwaregroup.com/research/against-the-best-plain-run-not-established/ On the second task, starting with easier sums solved the hard sum on 11 of 16 runs against 6 for the best ordinary recipe, but the accuracy difference is Tue, 29 Sep 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/at-width-24-undecided/ https://awarenesssoftwaregroup.com/research/at-width-24-undecided/ In a model half the size, the easy-sum start leaned the same way, needing about three quarters of the steps, but the difference was not clear of the noise. Tue, 29 Sep 2026 00:00:00 +0000