Large language model “personas” have two parts: stable average traits and a frame-dependent pattern of coordination
This paper asks whether the personality we read from a large language model (LLM) is a single, fixed thing or two different kinds of signals. The authors show that when you probe an LLM with a standard personality questionnaire, you get both simple average tendencies (like mean scores on the Big Five personality dimensions) and a separate geometric pattern of how answers hang together across questions. The geometric pattern is not fixed: it depends a lot on the temporal frame set by the question order, while the average scores are more robust to that framing.
To test this, the researchers prompted GPT-4o to simulate American and Chinese‑American personas and then gave it the 50‑item IPIP personality inventory adapted into first‑person statements. They ran experiments under three question‑ordering conditions: a fixed order (frame aligned), a random order (frame misaligned), and a random order with a bootstrap procedure that re‑aligns frames across instances. For each model run they built an “item‑dimension” correlation matrix that records how different questionnaire dimensions co‑vary during the model’s sequential response. Those matrices live on the set of symmetric positive definite (SPD) matrices — in plain language, a mathematical space for describing patterns of pairwise relationships — and the authors used standard tools to compare them while respecting that geometry.
The results show a clear two‑part pattern. When questions are shuffled, average measures such as Big Five scores fall modestly (about a 21% drop under content randomization), but they remain mostly stable to frame misalignment. By contrast, geometric features that capture the pattern of correlations between items collapse under a misaligned frame (about a 42% drop). Crucially, when the researchers restored a shared temporal frame across instances, those geometric features largely recovered — to roughly 84% of their original signal — producing a V‑shaped collapse‑then‑recovery pattern across the three conditions. That pattern fits the authors’ frame‑dependence hypothesis and not alternative ideas that the geometry is either wholly intrinsic or a pure measurement artifact.