https://awarenesssoftwaregroup.com/research/ Open research on how AI models actually learn. One finding per entry, in plain English first and in full technical detail underneath, including the approaches that turned out not to work. en Thu, 01 Oct 2026 15:35:46 +0000 https://awarenesssoftwaregroup.com/research/a-faster-rate-and-still-slower/ https://awarenesssoftwaregroup.com/research/a-faster-rate-and-still-slower/ Tuning the easy-start recipe's own learning rate made it faster on the random lookup table, but it stayed about 1,600 steps behind ordinary training. Thu, 01 Oct 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-harder-maximum-and-it-helps-again/ https://awarenesssoftwaregroup.com/research/a-harder-maximum-and-it-helps-again/ On a harder version of the maximum task, starting on easier versions saved about a quarter of the steps, correcting our earlier reading that the speed-up Thu, 01 Oct 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-harder-table-and-still-slower/ https://awarenesssoftwaregroup.com/research/a-harder-table-and-still-slower/ On a harder random lookup table, starting on easier versions was still about 3,000 steps slower: difficulty alone does not make the easy start pay. Thu, 01 Oct 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-random-table-and-it-slows/ https://awarenesssoftwaregroup.com/research/a-random-table-and-it-slows/ On a random lookup table as hard as tasks where it halved the steps, starting on easier versions slowed learning instead: the easy start needs a rule to Thu, 01 Oct 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-text-curriculum-on-a-small-model/ https://awarenesssoftwaregroup.com/research/a-text-curriculum-on-a-small-model/ Our first test on real text: neither an easy-first curriculum nor a learning-rate warm-up beat a constant learning rate, and the curriculum's easy Thu, 01 Oct 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-window-that-parks/ https://awarenesssoftwaregroup.com/research/a-window-that-parks/ We wrote down the practice plan an adaptive teacher discovered, as a two-setting rule. Thu, 01 Oct 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/both-tuned-and-still-slower/ https://awarenesssoftwaregroup.com/research/both-tuned-and-still-slower/ With both recipes tuned to their best learning rate, starting on easier versions of a random lookup table still took about 2,000 more steps than ordinary Thu, 01 Oct 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/differences-help-too/ https://awarenesssoftwaregroup.com/research/differences-help-too/ Warming up on easy subtraction problems sped up learning a hard addition problem almost as much as easy addition did. Thu, 01 Oct 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/for-the-maximum-it-slows-learning/ https://awarenesssoftwaregroup.com/research/for-the-maximum-it-slows-learning/ When the hard task was to output the larger of two remembered symbols, starting on easier versions slowed learning by about 1,300 steps instead of Thu, 01 Oct 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/it-already-knew-the-order/ https://awarenesssoftwaregroup.com/research/it-already-knew-the-order/ We found the order in which a model learns three related skills, then tried teaching them in that order. Thu, 01 Oct 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/kept-in-they-get-in-the-way/ https://awarenesssoftwaregroup.com/research/kept-in-they-get-in-the-way/ Even with the hard sum three quarters of the training data, keeping easy sums in the mix stopped every run learning it, worse than no easy sums at all. Thu, 01 Oct 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/lstm-best-rate-found/ https://awarenesssoftwaregroup.com/research/lstm-best-rate-found/ Tuned to its best learning rate, ordinary training of a small LSTM still took about 3,800 more steps than starting on easier sums, and solved the task Thu, 01 Oct 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/matched-and-the-sum-still-halves/ https://awarenesssoftwaregroup.com/research/matched-and-the-sum-still-halves/ With the same number of symbols and the same model as the random lookup table, the sum still reached the solution in about half the steps with the easy Thu, 01 Oct 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/on-a-transformer-not-established/ https://awarenesssoftwaregroup.com/research/on-a-transformer-not-established/ On a small transformer, the easy-sum start got more runs to the target, but most runs of both recipes had not got there by the end, so the result is not Thu, 01 Oct 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/predicting-the-cost-from-the-task/ https://awarenesssoftwaregroup.com/research/predicting-the-cost-from-the-task/ What it costs to take something away from a model turns out to be set by the structure of the exercise, not by the model. Change the exercise and it moves. Thu, 01 Oct 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/questions/ https://awarenesssoftwaregroup.com/research/questions/ The 10 questions our public AI training research is organised around, each with its current answer and the published results behind it. Thu, 01 Oct 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/revisiting-rescues-the-easy-tasks/ https://awarenesssoftwaregroup.com/research/revisiting-rescues-the-easy-tasks/ Adding one rule to a hand-written practice plan, go back to whatever slipped, closes half the gap to an adaptive teacher. Thu, 01 Oct 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/the-first-piece-is-the-load-bearing-one/ https://awarenesssoftwaregroup.com/research/the-first-piece-is-the-load-bearing-one/ We expected the hardest part of a task to be the one you cannot take away from a model. It is the easiest part, the one it learns first. Thu, 01 Oct 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/the-harder-the-task-the-bigger-the-saving/ https://awarenesssoftwaregroup.com/research/the-harder-the-task-the-bigger-the-saving/ Across five versions of one task, starting on easier versions moved smoothly from costing a quarter of the steps to saving 40% as the task got harder. Thu, 01 Oct 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/the-operation-and-the-component/ https://awarenesssoftwaregroup.com/research/the-operation-and-the-component/ Easy sums sharing nothing with the hard sum still sped up learning, and easy sums sharing a part of it sped it up about as much again. Thu, 01 Oct 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/the-sum-baseline-tuned-both-ways/ https://awarenesssoftwaregroup.com/research/the-sum-baseline-tuned-both-ways/ Ordinary training on the sum was tried at lower learning rates too; its best stayed in the middle of the range, and the easy start still took about half Thu, 01 Oct 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/the-sum-never-crosses/ https://awarenesssoftwaregroup.com/research/the-sum-never-crosses/ Every version of the sum task was already harder than the point where the easy start begins to pay, so it was ahead on all of them and the crossing could Thu, 01 Oct 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/the-teacher-found-a-plan/ https://awarenesssoftwaregroup.com/research/the-teacher-found-a-plan/ An adaptive teacher that picks what a model practises beat every fixed plan we wrote. Then replaying its choices, blind, did just as well. Thu, 01 Oct 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/xor-halves-the-steps-too/ https://awarenesssoftwaregroup.com/research/xor-halves-the-steps-too/ With bitwise XOR of two remembered symbols as the task, starting on easier versions reached the solution in about half the steps of ordinary training. Thu, 01 Oct 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-sweet-spot-not-the-nearest/ https://awarenesssoftwaregroup.com/research/a-sweet-spot-not-the-nearest/ Before a hard memory task, the best single warm-up was a version easier by a clear margin. Wed, 30 Sep 2026 00:00:00 +0000