DOI: 10.1177/20552076261505195 ISSN: 2055-2076

SVPTS: Sparse visual prompt-guided tumor segmentation with efficient cross-attention feature amplification

Syed Muntasin Fayaz, F M Javed Mehedi Shamrat, Debabrata Mallick, Md. Asraf Ali
Show PDF Cite

Objective

Accurate brain tumor segmentation from magnetic resonance imaging (MRI) is essential for treatment planning, but conventional networks impose high computational cost and prompt-based foundation models such as the Segment Anything Model (SAM) rely on large encoders with decoder-only prompt conditioning. This study develops a computationally efficient interactive framework in which sparse point prompts condition multi-scale encoder features.

Methods

SVPTS (Sparse Visual Prompt Tumor Segmentation) receives three to five simulated, ground-truth-derived point clicks and injects them into the stage-3 and stage-4 encoder feature maps through a Prompt-Guided Feature Amplification (PGFA) module. PGFA applies cross-attention between image tokens and Transformer-derived prompt embeddings, followed by gated residual fusion and FiLM-style conditioning, and is zero-initialized as an identity mapping. Evaluation on a public brain MRI dataset uses one fixed image-level split, exact surface metrics, paired and equivalence tests, three seeds, a duplicate-free subset, information-matched baselines, and prompt-robustness, interactive-click and calibration analyses.

Results

SVPTS achieved a Dice similarity coefficient (DSC) of 0.7294, with no significant difference from U-Net (0.7388; Wilcoxon p = 0.076) and equivalence to U-Net++ within 0.02 DSC (TOST p = 0.030). It required 7.38 GFLOPs (93.3% fewer than U-Net), with 56.9% lower GPU latency, 79.2% lower CPU latency and 68.0% lower peak memory, and was best calibrated (calibration error 0.0139 versus 0.0720). Bypassing PGFA reduced DSC by 0.0434 (95% CI 0.0240 to 0.0633), whereas perturbing the simulated prompts changed DSC by at most 0.0034. SVPTS exceeded zero-shot SAM by 0.2240 DSC.

Conclusion

Token-based prompt conditioning yields a compact interactive model whose DSC is not significantly different from U-Net and statistically equivalent to U-Net++, at a fraction of their computational cost. Tolerance to perturbed simulated prompts does not imply responsiveness to corrective clicks, which remained limited. Clinical deployment requires validation with real user interaction, subject-level data and external cohorts.

More from our Archive