This paper studies how foundation models—large pre-trained AI models used for image segmentation—perform when asked to outline lung nodules in CT scans. Segmentation means marking which pixels belong to a nodule so doctors can measure size and shape. The authors show that these models are sensitive to the form and placement of input “prompts,” such as a point inside the nodule or a bounding box around it. Small changes to those prompts can change the result a lot.

The team ran experiments with several recent foundation models, including SAM and MedSAM and their 3D variants (SAM2 and MedSAM2). They tested three kinds of prompts: a single point, a bounding box, and a point plus box. They used the LUNA16 dataset, which comes from the LIDC-IDRI collection and contains 888 chest CT scans with 1,186 annotated nodules larger than 3 mm. To measure mask quality they report the Dice coefficient, a common overlap score where higher means a better match to the radiologists’ masks.

They also tested robustness by perturbing prompts. For point prompts they shifted the point by a random radius. For boxes they changed box size by adding or subtracting pixels. These tests confirmed that bounding boxes generally give better results than single points, and that some models are more robust to such changes than others. For example, SAM2 achieved a Dice of about 0.89 with bounding boxes in one configuration, and several 3D model configurations reached about 0.88. Point-only prompts performed poorly in many cases (Dice below 0.1) except for MedSAM2 which reached above 0.7 in the experiments.

To reduce reliance on manual prompts, the authors built a lightweight synthetic prompt generator. It reuses the foundation model’s image encoder to produce dense features, then trains a small decoder (based on a feature pyramid network) to predict a coarse nodule mask and to output a suggested point and bounding box. The synthetic prompts are fed back into the foundation model and the final mask is compared to the ground truth during training. Loss terms include Dice and binary cross-entropy for masks, L1 losses for point and box coordinates, and a generalized Intersection-over-Union loss for boxes. This pipeline reached a Dice of 0.85, suggesting automatic prompt generation can be a promising way to run prompt-based models without manual input.