https://awarenesssoftwaregroup.com/research/ Open research on how AI models actually learn. One finding per entry, in plain English first and in full technical detail underneath, including the experiments that went against us. en Wed, 26 Aug 2026 15:22:51 +0000 https://awarenesssoftwaregroup.com/research/a-dial-not-a-switch/ https://awarenesssoftwaregroup.com/research/a-dial-not-a-switch/ We knew this internal change needs a model to be learning something. Now we know it scales with how much there is to learn, almost exactly. Wed, 26 Aug 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-rule-from-1990/ https://awarenesssoftwaregroup.com/research/a-rule-from-1990/ We finally asked what a pre-neural predictor scores on the task most of our results were measured on. It scores perfectly, and the network never catches it. Wed, 26 Aug 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/a-task-battery-with-designed-headroom/ https://awarenesssoftwaregroup.com/research/a-task-battery-with-designed-headroom/ Our quality checks all passed because the benchmark scored 99% and nothing could fail. Four candidate replacements, an acceptance rule fixed in advance, and two adoptions. Wed, 26 Aug 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/an-average-is-not-a-guarantee/ https://awarenesssoftwaregroup.com/research/an-average-is-not-a-guarantee/ Two warning signals that fire before a model learns, and why neither can be relied on for any particular training run. Also a correction to our own previous result. Wed, 26 Aug 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/being-choosy-about-data-costs-more-than-it-saves/ https://awarenesssoftwaregroup.com/research/being-choosy-about-data-costs-more-than-it-saves/ Inspect four batches, train only on the most informative. It sounds efficient. Even given perfect information for free, choosing well beat choosing arbitrarily by half a step out of ninety-four. Wed, 26 Aug 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/consolidation-is-a-measurement-frame-effect/ https://awarenesssoftwaregroup.com/research/consolidation-is-a-measurement-frame-effect/ Freezing the reference frame reverses a result we had reported for a year. The internal spread we thought was settling after learning is in fact still growing. Wed, 26 Aug 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/depth-does-not-add-stages/ https://awarenesssoftwaregroup.com/research/depth-does-not-add-stages/ In a two-layer model the layers learn at almost the same moment, so training them on separate schedules has no basis. But the reorganisation we study nearly doubles in the deeper layer. Wed, 26 Aug 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/does-the-early-signal-generalise/ https://awarenesssoftwaregroup.com/research/does-the-early-signal-generalise/ Two findings that looked independent turn out to have identical boundaries. The early-warning gap appears exactly where the internal reorganisation does. Wed, 26 Aug 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/does-the-effect-survive-the-optimizer/ https://awarenesssoftwaregroup.com/research/does-the-effect-survive-the-optimizer/ Every result we had used one training algorithm. Changing it at matched steps-to-solve keeps the effect at full size and takes away its timing. Wed, 26 Aug 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/does-the-transition-create-features/ https://awarenesssoftwaregroup.com/research/does-the-transition-create-features/ Freezing a model's memory at its random starting values and training only the two ends. It learns, slowly and partially, and never abruptly. Wed, 26 Aug 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/does-the-warning-cry-wolf/ https://awarenesssoftwaregroup.com/research/does-the-warning-cry-wolf/ Every reliability figure we have published was measured on runs that succeeded. This asks what our early warning does when nothing is coming. Wed, 26 Aug 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/eight-runs-is-not-enough/ https://awarenesssoftwaregroup.com/research/eight-runs-is-not-enough/ A free experiment produced two statistically significant results pointing in opposite directions. Our first analysis reported one of them as a discovery. Three times the runs later, we have the real answer. Wed, 26 Aug 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/how-many-of-our-results-are-close-calls/ https://awarenesssoftwaregroup.com/research/how-many-of-our-results-are-close-calls/ One of our results was overturned just by running it more times. So we measured how close every other published result sits to the line that decides it. Wed, 26 Aug 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/how-you-cut-a-model-matters/ https://awarenesssoftwaregroup.com/research/how-you-cut-a-model-matters/ Delete three quarters of a model's internal connections two different ways, keeping exactly the same number, and one way costs twice as much as the other. Wed, 26 Aug 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/is-learning-a-lucky-accident/ https://awarenesssoftwaregroup.com/research/is-learning-a-lucky-accident/ Testing whether sudden learning is randomness knocking a model out of a rut. Injected noise makes the plateau longer and the transition blurrier, so randomness is friction rather than fuel. Wed, 26 Aug 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/is-transition-timing-controllable/ https://awarenesssoftwaregroup.com/research/is-transition-timing-controllable/ A nudge of one part in a hundred million delays learning by 21 steps. A nudge ten million times larger delays it by 24. There is no dial, and this closed a research line. Wed, 26 Aug 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/it-is-not-how-much-you-remove/ https://awarenesssoftwaregroup.com/research/it-is-not-how-much-you-remove/ Removing one direction from a model's training signal at the right moment causes most of the damage that stopping the whole component does. Removing three times as much at the wrong moment causes a third as much. Wed, 26 Aug 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/it-tries-the-wrong-rule-first/ https://awarenesssoftwaregroup.com/research/it-tries-the-wrong-rule-first/ Instead of asking how often the model is right, we asked what it is doing. Early on it behaves like a simpler rule that does not work. Wed, 26 Aug 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/latent-knowledge-before-behaviour/ https://awarenesssoftwaregroup.com/research/latent-knowledge-before-behaviour/ The answer is linearly readable from a model's internal state about 23 steps before the model can produce it. An earlier version of the same experiment reported the opposite. Wed, 26 Aug 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/learning-it-again-is-not-like-learning-it/ https://awarenesssoftwaregroup.com/research/learning-it-again-is-not-like-learning-it/ We changed the task underneath a trained model and watched it learn again. It did, but the internal signature we have been studying was about a sixth the size the second time. Wed, 26 Aug 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/looking-in-the-right-place-first/ https://awarenesssoftwaregroup.com/research/looking-in-the-right-place-first/ By changing one earlier word at a time, we can see what a prediction depends on. The model finds the right word long before it can use it. Wed, 26 Aug 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/no-part-of-the-model-is-special/ https://awarenesssoftwaregroup.com/research/no-part-of-the-model-is-special/ We cut a model's memory into separate compartments to see whether the effect we study concentrates in one of them. It does not, at any split we tried, which closes a line of work we had been holding open. Wed, 26 Aug 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/one-jump-or-several/ https://awarenesssoftwaregroup.com/research/one-jump-or-several/ Averaging accuracy across positions hid four separate events. The model learns a repeating pattern one element at a time, in order, and the order repeats across runs. Wed, 26 Aug 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/our-early-warning-gets-worse-as-models-grow/ https://awarenesssoftwaregroup.com/research/our-early-warning-gets-worse-as-models-grow/ The one genuinely useful instrument we have built stops being useful in bigger models, and not for the reason anyone would guess. Wed, 26 Aug 2026 00:00:00 +0000 https://awarenesssoftwaregroup.com/research/predicting-the-cost-from-the-task/ https://awarenesssoftwaregroup.com/research/predicting-the-cost-from-the-task/ What it costs to take something away from a model turns out to be set by the structure of the exercise, not by the model. Change the exercise and it moves. Wed, 26 Aug 2026 00:00:00 +0000