Calibrating a Virtual City: The SIVE Experiment ✛

The 120 residents of Montelago are not people but digital characters. Each, besides a name and biography, has three hidden coordinates: trust in municipal authorities, assessment of source credibility, perceived adequacy of information. These numbers are known to the creators but not to the system itself. The system uses a Large Language Models based on transformers and attention mechanisms. It generates character reactions to external messages, relying only on their textual histories.

Such a synthetic population can be compared to a scientific instrument. Before measuring the unknown, you calibrate the instrument — apply signals of known strength and observe the response. The SIVE experiment does exactly that. Researchers created seven versions of news about Montelago's water supply network: from very good (pipe replacement, transparent monitoring) to very bad (accidents, lack of funds, murky water). They added a placebo, a harmless message about a city festival, and a control group that received no news. Then, after each stimulus, the experiment recorded shifts in characters' responses to trust questions.

The seven pre-announced criteria passed the test at three generation temperatures (0.2, 0.5, 0.7)—a parameter controlling the probabilistic randomness of answers. Hidden trust and its measured manifestation correlated with a strength of 0,89–0,91. Before stimulus presentation, scores remained stable. Instrument noise, measured by presenting the same positive stimulus twice, was about 1,4 points on a ten-point scale across different characters and dropped to 0,77 when fixing the identity, like a voltmeter needle trembling within a division.

But the most interesting moment was the flaw. Initially designed as a weakly positive stimulus, the message about partial improvement caused a drop in trust. This was not a model error but a defect in the message itself: analysis of the text revealed that the wording emphasized problems rather than successes. After rewriting the stimulus, order was restored. Thus calibration uncovered and helped fix a non-obvious mistake.

The SIVE approach introduces a mandatory procedure into work with LLM simulations — a test of controllability. Just as physicists calibrate detectors on sources of known intensity, developers of digital twins can now calibrate their instrument before trusting its predictions about real people. For city administrations planning risk communication, whether soil contamination in Brescia or renovation plans, this is a chance to test messages on a realistic yet controllable model for the first time. The calibrated instrument from Montelago is ready for real measurements.