In an oven, as temperature rises, molecules move faster and disorder increases. With language models like ChatGPT, it's the opposite. When developers increase the 'temperature' parameter, the model gets access to a wider choice of words, but its long texts suddenly start looping.
A team of researchers tested 11 different AIs by generating 12,000 text continuations. At high temperatures, responses passed through a peak of diversity, then fell into 'population inversion'—a state where certain word choices repeat over and over. Frozen phrases, cycles, and bursts of disorder alternating with strict order emerged.
The authors discovered a hidden coordinate that controls this process, like a tuning knob. Knowing its position, one can accurately predict whether the model will start repeating itself. This approach transforms language models from 'stochastic parrots' into controllable nonlinear systems whose internal dynamics can be measured and directed.
The effect of order from noise is known in physics as 'stochastic resonance': sometimes adding chaos helps a system find a stable state, rather than destroying it.
In Stanislaw Lem's novel 'His Master's Voice,' scientists encounter a signal that, as energy increases, doesn't become more complex but degenerates into a repeating code, as if an alien thought freezes from overheating.