通过局部感知扫描和重要性重采样,释放整个幻灯片图像的状态空间模型的力量。
Unleash the Power of State Space Model for Whole Slide Image with Local Aware Scanning and Importance Resampling.
发表日期:2024 Oct 07
作者:
Yanyan Huang, Weiqin Zhao, Yu Fu, Lingting Zhu, Lequan Yu
来源:
IEEE TRANSACTIONS ON MEDICAL IMAGING
摘要:
全幻灯片图像 (WSI) 分析在医学成像领域越来越受到重视。然而,由于 WSI 的大小为十亿像素,以前的方法通常无法有效地处理整个 WSI。受状态空间模型最新发展的启发,本文引入了新的 Pathology Mamba (PAM),以实现更准确、更稳健的 WSI 分析。 PAM 包括三个精心设计的组件,以应对巨大图像尺寸、本地和分层信息的利用以及 WSI 分析期间训练和测试的特征分布之间不匹配的挑战。具体来说,我们设计了一种双向 Mamba 编码器来有效且高效地处理 WSI 中存在的大量斑块,它可以处理大规模病理图像,同时实现高性能和准确性。为了进一步利用 WSI 的本地信息和固有的分层结构,我们引入了一种新颖的本地感知扫描模块,该模块采用本地感知机制和分层扫描来熟练地捕获 WSI 内的本地信息和总体结构。此外,为了缓解训练和测试之间的补丁特征分布不一致,我们提出了测试时重要性重采样模块来进行测试补丁重采样,以确保训练和测试阶段之间特征分布的一致性,从而增强模型预测。对包含癌症亚型和生存预测任务的 9 个 WSI 数据集进行的广泛评估表明,PAM 优于当前最先进的方法,并且其在 WSI 内的判别区域建模方面的能力也得到了增强。源代码可在 https://github.com/HKU-MedAI/PAM 获取。
Whole slide image (WSI) analysis is gaining prominence within the medical imaging field. However, previous methods often fall short of efficiently processing entire WSIs due to their gigapixel size. Inspired by recent developments in state space models, this paper introduces a new Pathology Mamba (PAM) for more accurate and robust WSI analysis. PAM includes three carefully designed components to tackle the challenges of enormous image size, the utilization of local and hierarchical information, and the mismatch between the feature distributions of training and testing during WSI analysis. Specifically, we design a Bi-directional Mamba Encoder to process the extensive patches present in WSIs effectively and efficiently, which can handle large-scale pathological images while achieving high performance and accuracy. To further harness the local information and inherent hierarchical structure of WSI, we introduce a novel Local-aware Scanning module, which employs a local-aware mechanism alongside hierarchical scanning to adeptly capture both the local information and the overarching structure within WSIs. Moreover, to alleviate the patch feature distribution misalignment between training and testing, we propose a Test-time Importance Resampling module to conduct testing patch resampling to ensure consistency of feature distribution between the training and testing phases, and thus enhance model prediction. Extensive evaluation on nine WSI datasets with cancer subtyping and survival prediction tasks demonstrates that PAM outperforms current state-of-the-art methods and also its enhanced capability in modeling discriminative areas within WSIs. The source code is available at https://github.com/HKU-MedAI/PAM.