Cairo Rosetta vs. Frontier LLMs: Predicting What a Specific Person Will Actually Choose

Predicting what people do on average is a solved problem for most teams. Predicting what one specific person will choose — given who they are and the situation in front of them — is much harder. It is also the question behind almost every people decision: how will this customer react to a price, this employee to a new policy, this voter to a message?

Large language models are increasingly used to simulate human behavior, and they work reasonably well in aggregate. But at the individual level the evidence is thin, limited by the scarcity of high-quality public behavioral data. So at Cairo Labs we built Rosetta Arena, a benchmarking environment that tests different AI systems on one question: given a person's profile and a closed set of options, which one did they actually pick?

This article summarizes the first experiment, published as a preliminary technical report (downloadable below). We want to be explicit about what it shows — and what it does not.

Why individual choice is hard

Aggregate simulation rewards models that capture population averages. Individual prediction punishes that shortcut: two people with identical demographics routinely make opposite decisions in the same situation. The signal that separates them lives in psychometric profile, context, and stake — exactly the information a specialized model must learn to use, and exactly what a frequency-based guess cannot see.

That distinction shapes the whole design of Rosetta Arena. Every competitor receives the same standardized cognitive and personality profile, the same situation description, and the same shuffled catalog of 2 to 12 discrete options. The person's real decision stays hidden until scoring.

The competitors

Five systems faced the same 30 cases:

• Cairo Rosetta — our specialized behavior model (a Large Behavior Model). It receives the structured person-and-situation description and returns the catalog option that best matches the profile. Evaluated as a black box under the same input conditions as everyone else.
• Claude Sonnet 5.5, GPT-5.6 Terra, and Gemini 3.8 Flash — three general-purpose frontier LLMs accessed through OpenRouter, each given an identical structured prompt and required to answer in JSON.
• A frequency baseline — a deterministic lookup that always predicts the option people historically chose most often for that event. If a system cannot beat this, it is extracting nothing useful from the individual's profile.

Results

Rosetta achieved the highest point accuracy, followed closely by Claude Sonnet 5.5. But the confidence intervals are wide — roughly ±16.5 percentage points — and they overlap heavily. Rosetta's edge over the best LLM amounts to a single case out of thirty.

The honest reading is competitive parity with a directional advantage for Rosetta, not demonstrated superiority. The larger gaps (20 points over the baseline, 23 over Gemini) also fall short of significance in an unpaired test; a case-level paired analysis may change that as the sample grows.

Evidence that Rosetta reads the individual

The baseline enables a sharper reading. Its 12 hits correspond exactly to the cases where the person chose the historically most common option. In the other 18 cases, the individual departed from the mode — and those are the decisions you can only get right by using information about the person.

By construction the baseline gets none of them. Rosetta correctly anticipated at least 6 of those 18 atypical decisions (a third or more); the corresponding bounds are 5 for Claude Sonnet 5.5, 3 for GPT-5.6 Terra, and 0 for Gemini 3.8 Flash — which, notably, finished below the baseline. Being a frontier model does not guarantee extracting useful signal from a person's profile.

Where the gap is clearest: efficiency

Accuracy differences hinge on a few cases. The latency and cost differences are large and structural — they follow from dedicated inference versus general-purpose models served through an API.

At small scale these figures are pocket change. At the scale Rosetta is designed for — simulating thousands or millions of profiles across many scenarios — they decide whether behavioral simulation is economically viable at all, and whether results arrive at interactive speed. On point estimates, Rosetta sits ahead of all three LLMs on accuracy, latency, and direct cost simultaneously; the accuracy part carries the uncertainty above, while the latency and cost parts are large differences.

What this run does not tell us

We would rather list the limits ourselves:

• Small sample. Thirty decisions give about ±16.5 points of uncertainty per accuracy figure. Detecting a 10-point difference with 80% power would take roughly 390 cases per competitor in an unpaired design.
• Home-field advantage. Rosetta and the baseline drew training or reference data from the same sources as the test cases (in separate partitions), while the LLMs ran zero-shot. Generalization to unseen data sources is still untested.
• Possible LLM contamination. Both datasets are public and may sit inside the LLMs' pretraining data — which would favor the LLMs, not Rosetta.
• One prompt. The LLMs received a single structured prompt; few-shot examples, extended reasoning, or tuned sampling could shift their accuracy, latency, and cost.
• Infrastructure differences. LLM latency includes OpenRouter routing and provider networks, while Rosetta runs on dedicated infrastructure. Part of the speed gap is infrastructure, not only the model.
• One run, one seed. We have not yet measured variability across seeds or LLM non-determinism, and with 2 to 12 options per case, chance-level accuracy differs case by case.

What comes next

The next iteration of Rosetta Arena is designed to answer exactly these questions: scale to several hundred cases stratified by dilemma type; publish case-by-case results with paired McNemar tests and accuracy split between modal and non-modal decisions; add probabilistic metrics (log-loss, Brier score, calibration); test the LLMs under stronger prompting protocols; measure Rosetta on data sources it has never seen; and run an ablation where every competitor receives the cases without the individual profile, to quantify how much accuracy truly comes from psychometric information.

The bottom line

This first run shows competitive accuracy with a directional edge for Rosetta, and a clear, large advantage in speed and cost. It does not show that a specialized behavior model beats frontier LLMs on accuracy — that claim needs a bigger, paired evaluation, and that is the work in front of us.

What the experiment already validates is the arena itself: a reproducible, leak-free environment where we can keep testing Rosetta in public as it evolves.