A Machine That Comes Up with Theories About Thinking express
A closed loop: the machine proposes an idea, tests it on people, sees the failures, and spawns a new, more accurate guess. That's how AutoCog works — an automatic cognitive scientist. Previously, in the science of thinking (cognitive science), the hardest part, coming up with a theory, was done only by humans. Data collection was automated, models were computed by computers, but the creative step remained with humans. AutoCog closes this loop entirely for the first time.
The system is structured as a debate among several agents based on large language models. Each agent proposes its own decision-making model — not just text, but a working program. Then AutoCog designs the experiment that best adjudicates the debate, recruits volunteers online, and collects their responses. Theories are evaluated by how well they predict human behavior. Losing models analyze their errors and are reborn as improved versions. The cycle repeats.
This finding was not pre-programmed. When the system was run on simulated data, it recovered even unusual strategies that contradicted the language model's "common sense." This means AutoCog truly listens to data rather than repeating learned patterns. In experiments with real people, its theories proved more accurate than established ones and worked across two different tasks. The final test was passed by the new theory: its predictions were confirmed in a preregistered experiment with new participants.
The circle is closed. AutoCog turns theory-building about thinking from a vague art into a transparent, reproducible, and cumulative science.
Original abstract
as written by the authors arXiv:2606.26448
Across the sciences, autonomous systems are increasingly being used in closed-loop discovery, proposing new theories and designing and running experiments to test them. This approach is yet to be applied in the field of cognitive science, where the central bottleneck is theory-building: the creative step of turning the accumulated failures of existing models into better ones. Theory generation has remained manual even as data collection, modeling, and experiment design have been automated. We present the Automated Cognitive Scientist (AutoCog), a fully autonomous agentic-AI system that closes this loop. Large-language-model agents advocate competing theories, each expressed as an executable cognitive model, design experiments that best discriminate them, collect behavioral data from participants recruited online, score theories against collected data based on their generative performance, diagnose why they fail, and synthesize a better successor. Repeating this cycle allows them to search the space of theories, models, and experiments. In the domain of decision-making, AutoCog recovered known decision-making strategies from simulated behavior, including unconventional ones, showing that its discoveries are ultimately driven by the data rather than strictly bound by the priors of the underlying language models. When run with human participants, it produced theories that outperformed the established theories it was seeded with and generalized to held-out studies in two different experimental settings. It also surfaced a novel theory of multi-cue decision-making in which choices show diminishing sensitivity to feature values. The distinctive predictions of this theory were confirmed in a preregistered study with new participants. AutoCog demonstrates how an automated discovery system can be used to turn cognitive theory-building into an explicit, executable, and cumulative science.
The AutoCog system is structured as a debate: several language models compete to see whose theory best explains human behavior, and the losers then "mutate" into new versions.
A similar approach is described in Isaac Asimov's short story "The Last Question," where a machine iteratively improves its answers until it achieves a breakthrough.