Collective motion in nature, from bird flocks to bacterial swarms, is usually explained by direct social interactions: agents align their directions or approach neighbors. The Vicsek model and its successors have cemented this logic. When learning is introduced into such models, agents are explicitly rewarded for velocity alignment. However, animals rarely receive a reward for grouping per se; they seek food, mates, or shelter, and social structure emerges as a byproduct. Whether collective order can arise without a direct incentive has remained an open question. This work sits alongside studies of collective search and swarm learning but differs in that it does not encode group behavior into the objective function.
The authors considered a population of 50 agents in a two-dimensional square domain of side 50 with periodic boundaries. Agents take steps of unit length and after each step either continue in their current direction or turn to a random heading. The environment contains 100 stationary targets of radius 0.5; after capture, a target is depleted for a time τ. Agents do not see targets; they only perceive conspecifics within a vision cone with half-angle π/8 and radius $r_v$. The visual signal is coarse-grained into three states: none present, an unrewarded agent present, or a rewarded agent present (one that recently captured a target). Each agent learns independently via projective simulation, a reinforcement learning algorithm without a model, with reward +1 for each captured target. Two scenarios are considered: competitive (a target is unavailable to all for τ) and cooperative (a target is unavailable only to the capturer).
At small visual radii, the efficiency (average reward per step) matches that of a solitary agent without vision. Then, at a certain $r_v$, efficiency rises sharply and reaches a plateau. Analysis of the learned strategies reveals two regimes: at small $r_v$, agents use scale-adaptive search tuned to the characteristic lengths of the environment; at large $r_v$, they switch to a collective strategy, turning almost every step until they see a rewarded conspecific, then moving ballistically toward it. The transition is accompanied by spatial aggregation: the Clark–Evans index R (the ratio of the mean nearest-neighbor distance to that expected for a Poisson process) drops below 1, indicating clustering. Just before the transition, overdispersion is observed (R>1). The competitive scenario delays the transition to aggregation to larger $r_v$. Finite-size analysis shows that the transition is a sharp crossover, not a thermodynamic phase transition: the width of the transition region does not vanish as the system grows.
This work shows that collective phenomena can arise from an indirect source, maximizing individual target capture, without direct reward for grouping. This contrasts with the standard approach where order is imposed by a direct social term. Importantly, the same learning scheme that reproduces optimal solitary search discovers a dynamic crossover without any hints. This makes reinforcement learning a tool for discovering emergent physics in active systems, not just an optimizer. Compared to neighboring work on collective search, this is the first demonstration that aggregation can be a byproduct of optimal foraging under purely indirect reward.
A natural direction is heterogeneous learning, where only part of the population adapts while others follow fixed strategies. This raises questions about how learned strategies adjust to the presence of agents with fixed behavior and what fraction of learners is needed for aggregation to emerge. Another direction is extending to heterogeneous populations, mobile or spatially structured resources, and richer sensory channels, bringing the model closer to real organisms.
The results could influence our understanding of collective animal behavior and the design of robot swarms, where group behavior is desirable without explicitly programming social rules.
Test how heterogeneity among learning agents affects the emergence of aggregation. Investigate the effects of moving targets and more realistic sensory signals.
In the statistical physics of active systems, the question of which mechanisms drive phase transitions in collective motion remains open. This work links the emergence of aggregation to the optimization of individual search, offering an alternative to direct social forces and highlighting the role of indirect signals in shaping macroscopic order.
This section is addressed to the author of the paper and their colleagues. Neighbouring works were found by meaning-based search across our archive, not by word overlap. We do not review: we show the field around the work.
What the work does
The authors trained reinforcement-learning agents to seek replenishing targets without seeing the targets themselves, only by observing their peers, and showed that as the agents' visual range increases, they sharply switch from an individual to a collective strategy, coinciding with the onset of spatial aggregation. An analytical model of first achievement reproduces this transition as a crossover between two search strategies.
Where its strength is
This work introduces an indirect mechanism for the emergence of collective motion through individual optimization of resource search, without imposing group behavior in the objective function, and supports it with an analytical model that reproduces the transition.
Where it could go next
Worth trying
Why this may matter
If collective order can arise as a byproduct of individual search optimization, this changes our understanding of the evolution of social behavior and may suggest new principles for designing robot swarms where group interaction need not be explicitly programmed.
Nearby works in our archive
The archive is sparse around this work: the closest review is only 0.54, against a usual 0.64. It looks like few have drilled nearby.
The Clark–Evans index, used here to quantify aggregation, was proposed in 1954 by ecologists to analyze the spatial distribution of plants and animals.
The idea that complex collective behavior can emerge from simple individual rules without central control echoes the concept of the 'swarm' in Stanisław Lem's novel 'The Invincible', where microscopic machines form destructive structures.