Researchers tested whether general-purpose large language models (LLMs) can judge which 3D arrangements of the same molecule (called conformers) are more stable. They gave models raw XYZ coordinates for 30 conformers of each molecule and asked the models to rank them by stability. The benchmark, called LLMConfBench, used 810 geometries from 27 small organic molecules and used r2SCAN-3c density functional theory as the reference energy to score the rankings. To avoid memorization, each conformer was randomly rotated so only the shape, not absolute coordinates, mattered.

Many modern frontier LLMs showed an unexpected ability to rank conformers. The best LLMs — GPT-6 Astra (mean reliable-pair Kendall’s = 0.67) and Claude Opus 5 (0.61) — came close to the performance of a recent physics-based force field called GFN-FF (0.72). Other recent models such as Gemini 3.6 Flash (0.43), GPT-5.6 Sol (0.40), and Kimi K3 (0.38) also outperformed the older Universal Force Field (UFF, 0.23). By contrast, force-field methods vary: a classical force field MMFF94 scored very high (0.93) on this small-molecule test.

The authors examined model explanations for their rankings and found a pattern. Models did best when stability depended on well-described intramolecular interactions, notably hydrogen bonds. These interactions are frequently and precisely discussed in scientific writing, which the models were trained on. Models struggled more with stability driven by loosely defined chemical concepts, such as ring strain, where traditional force fields often do better. This suggests the models’ geometric reasoning may be tied to how clearly a concept appears in text rather than to a deep physical simulation.

The study also found that a model’s success at this task correlates with its performance on scientific, coding, and abstract-reasoning benchmarks. That implies the ability may have emerged unintentionally as a side effect of broad training on related tasks, not from explicit training to read or predict 3D chemistry. The benchmark uses a “reliable-pair” metric that only scores conformer pairs differing by more than 2 kcal/mol, to avoid rewarding guesses on nearly degenerate structures.