This paper introduces ClinFusion, a multimodal large language model (MLLM) designed to understand medical images the way clinicians do. The team built a vision-centered system that natively handles both 2D images (like X‑rays and pathology slides) and native 3D volumes (like CT and MRI scans). They also created new evaluation tools that try to judge model outputs the same way radiologists assess reports, by focusing on clinically relevant regions and factual correctness rather than just overall text similarity.

To process different kinds of medical images, the researchers replaced the usual single vision encoder with a compositional design: a base, well-aligned vision transformer plus several specialist 2D encoders and a dedicated 3D encoder. These parts are combined by a Cascade Spatial-Aware Locality (CaSL) Fusion operator. In plain terms, CaSL lets the specialist 2D and 3D components exchange information in stages so the model keeps an aligned core representation while progressively adding modality-specific detail. For 3D scans, they use a 2D‑anchored, depth-aware version of this fusion so the volume information can be linked back to familiar 2D features.

The paper also tackles evaluation, which the authors argue is often misaligned with clinical practice. They introduce MedIF-Bench to measure whether models can follow clinical instructions, and they propose a region-of-interest (RoI) grounded method to evaluate report generation. The RoI approach conditions the model on patient-specific clinical context (which drives attention to particular anatomical areas) and uses an LLM to break diagnostic claims into matched, missed, and hallucinated findings. This aims to measure factual accuracy and clinical relevance rather than just surface text overlap.

On a broad set of tests, ClinFusion reportedly sets new state-of-the-art results across many 2D and 3D medical benchmarks. According to the paper excerpt, it outperformed leading open-source medical MLLMs (for example, Hulu‑Med and Lingshu) on 20 of 24 benchmarks, and beat several proprietary models (such as GPT‑5.2 and Gemini‑3‑Flash) on 13 of 16 multimodal benchmarks. The authors also ran a blinded study in which six board-certified radiologists independently ranked reports from ClinFusion and other models across 300 CT and X‑ray cases; ClinFusion’s reports were ranked highest and their RoI-grounded metric showed the strongest correlation with expert judgment among the automatic metrics tested.