Benchmark of 23 machine‑learning interatomic potentials shows big models give small accuracy gains but much slower speeds
This paper tests whether modern machine‑learning interatomic potentials (MLIPs) are truly practical for everyday atom simulations. The authors find a clear trade‑off: the largest, state‑of‑the‑art models give only about 3–5 meV per atom better accuracy, but they are one to three orders of magnitude slower than much smaller models. Lightweight MLIPs instead sit on a practical “Pareto frontier” of good-enough accuracy and high speed on ordinary hardware.
The team ran a unified benchmark of 23 open‑source MLIPs on a modest GPU workstation (an NVIDIA DGX Spark with 128 GB native GPU memory, capped at 80 GB to mimic typical lab hardware). All runs used the Atomic Simulation Environment (ASE) with default public model configurations and weights. They measured three things: predictive accuracy (using a phonon thermal‑conductivity metric called κ_SRME from the Matbench Discovery protocol), molecular‑dynamics (MD) throughput (integration steps per second), and hardware scalability (the largest system size manageable under the 80 GB limit). A small cross‑check on a different runtime (TorchSim) changed speeds by only 1.11–1.4×, supporting the ASE results.
The results show three tiers. Lightweight models (roughly 0.5–5 million parameters) use little memory and run hundreds to thousands of times faster than a DFT reference, while retaining competitive accuracy. Medium models (≈5–10 million parameters) trade some speed for a modest accuracy boost. Large models (≈10–730 million parameters) achieve the best static scores but only improve accuracy by a few meV per atom compared with lightweight models. That small gain is below typical room‑temperature thermal noise and, in the authors’ tests, often comes at the cost of collapsing throughput so that some large models run less than twice as fast as the density functional theory (DFT) baseline.