研究动态
Articles below are published ahead of final publication in an issue. Please cite articles in the following format: authors, (year), title, journal, DOI.

利用 2D 深度学习 ImageNet 训练的模型进行本机 3D 医学图像分析。

Leveraging 2D Deep Learning ImageNet-trained models for Native 3D Medical Image Analysis.

发表日期:2023
作者: Bhakti Baheti, Sarthak Pati, Bjoern Menze, Spyridon Bakas
来源: BIOMEDICINE & PHARMACOTHERAPY

摘要:

由于大量 2D 训练数据的可用性,卷积神经网络 (CNN) 在各种 2D 计算机视觉任务中表现出了良好的性能。相反,医学成像处理 3D 数据,通常缺乏用于开发 AI 模型的同等数据范围和多样性。迁移学习提供了一种方法,可以使用为一个应用程序训练的模型作为另一个应用程序的起点。在这项工作中,我们通过探索轴向-冠状-矢状 (ACS) 卷积的概念,利用 2D 预训练模型作为 3D 医学应用的起点。我们将 ACS 作为原生 3D 卷积的替代方案纳入 General Nuanced 深度学习框架 (GaNDLF),提供各种完善且最先进的网络架构,并可使用 2D 数据进行预训练的编码器。我们对脑肿瘤患者 3D MRI 数据进行 i) 肿瘤分割和 ii) 放射基因组分类的实验评估结果显示,模型大小减少了约 22%,验证准确性提高了约 33%。我们的研究结果支持了预训练 2D CNN 中的 ACS 卷积相对于无需预训练的 3D CNN 的优势,适用于 3D 分割和分类任务,使在前所未有的规模的数据集中训练的现有模型民主化,并在医疗保健领域显示出前景。
Convolutional neural networks (CNNs) have shown promising performance in various 2D computer vision tasks due to availability of large amounts of 2D training data. Contrarily, medical imaging deals with 3D data and usually lacks the equivalent extent and diversity of data, for developing AI models. Transfer learning provides the means to use models trained for one application as a starting point to another application. In this work, we leverage 2D pre-trained models as a starting point in 3D medical applications by exploring the concept of Axial-Coronal-Sagittal (ACS) convolutions. We have incorporated ACS as an alternative of native 3D convolutions in the Generally Nuanced Deep Learning Framework (GaNDLF), providing various well-established and state-of-the-art network architectures with the availability of pre-trained encoders from 2D data. Results of our experimental evaluation on 3D MRI data of brain tumor patients for i) tumor segmentation and ii) radiogenomic classification, show model size reduction by ~22% and improvement in validation accuracy by ~33%. Our findings support the advantage of ACS convolutions in pre-trained 2D CNNs over 3D CNN without pre-training, for 3D segmentation and classification tasks, democratizing existing models trained in datasets of unprecedented size and showing promise in the field of healthcare.