研究动态
Articles below are published ahead of final publication in an issue. Please cite articles in the following format: authors, (year), title, journal, DOI.

人工智能在外科课程设计中的应用及模拟培训技术能力的意外结果。

AI in Surgical Curriculum Design and Unintended Outcomes for Technical Competencies in Simulation Training.

发表日期:2023 Sep 05
作者: Ali M Fazlollahi, Recai Yilmaz, Alexander Winkler-Schwartz, Nykan Mirchi, Nicole Ledwos, Mohamad Bakhaidar, Ahmad Alsayegh, Rolando F Del Maestro
来源: JAMA Network Open

摘要:

为了更好地阐明人工智能(AI)在外科技能培训中的作用,需要调查隐藏课程的潜在存在。评估AI选择的技术能力在外科模拟训练中的教育价值及其延伸效果。本队列研究是对加拿大蒙特利尔脑科学研究所脑外科模拟与人工智能学习中心进行的一项随机临床试验的跟踪研究,研究对象为来自麦吉尔大学蒙特利尔分校的医学生。比较接受AI增强型培训课程的医学生与未接受反馈的对照组参与者以及专家水平指标之间的外科表现指标。横断面数据收集时间为2021年1月至4月,医学生数据;2015年3月至2016年5月,专家数据。本次跟踪副分析的时间为2022年6月至9月。研究对象包括干预队列中的医学生(本科0-2年级)和神经外科医生以建立专家水平的基准。通过智能导师在模拟训练期间对4个AI选择的学习目标进行绩效评估和个性化反馈。感兴趣的结果是非预期的绩效结果,通过对干预队列的270个绩效指标与基线之间的显著内部参与者差异进行测量,对照组中这种差异未观察到。本研究共纳入46名医学生(年龄中位数[范围]为22 [18-27]岁;女性27名[59%])和14名外科医生(年龄中位数[范围]为45 [35-59]岁;男性14名[100%])。所有参与者都未失访。接受AI增强型课程的参与者在整个手术过程中的32个绩效指标和在肿瘤摘除过程中的20个绩效指标中,与对照组相比,观察到了额外的绩效变化。接受AI增强型课程的参与者在安全绩效方面表现出显着改善,比如减少健康组织切除的速率(平均差异,每20毫秒减少7.05 x 10-5 [95%置信区间,-1.09 x 10-4至-3.14 x 10-5] mm3;P < 0.001)并保持对手术领域的双手控制(最大仪器散度平均差异,-4.99 [95%置信区间,-8.48至-1.49] mm;P = 0.006)。然而,还观察到了负面的无意义效应,包括主导手的速度和加速度显著降低(速度:平均差异,每20毫秒减少0.13 [95%置信区间,-0.17至-0.09] mm;P < 0.001;加速度:平均差异,每20毫秒2减少2.25 x 10-2 [95%置信区间,-3.20 x 10-2至-1.31 x 10-2] mm;P < 0.001)以及肿瘤摘除速率显著降低(平均差异,每20毫秒减少4.85 x 10-5 [95%置信区间,-7.22 x 10-5至-2.48 x 10-5] mm3;P < 0.001)。这些意外结果使得学生的运动和效率绩效指标偏离了专家水平的基准。在医学生的这项队列研究中,AI增强型双手外科技能课程导致了改善安全性但对一些效率指标产生了负面影响的意外改变。将AI纳入课程设计需要进行持续评估,以保持透明度并培养基于证据的学习目标。
To better elucidate the role of artificial intelligence (AI) in surgical skills training requires investigations in the potential existence of a hidden curriculum.To assess the pedagogical value of AI-selected technical competencies and their extended effects in surgical simulation training.This cohort study was a follow-up of a randomized clinical trial conducted at the Neurosurgical Simulation and Artificial Intelligence Learning Centre at the Montreal Neurological Institute, McGill University, Montreal, Canada. Surgical performance metrics of medical students exposed to an AI-enhanced training curriculum were compared with a control group of participants who received no feedback and with expert benchmarks. Cross-sectional data were collected from January to April 2021 from medical students and from March 2015 to May 2016 from experts. This follow-up secondary analysis was conducted from June to September 2022. Participants included medical students (undergraduate year 0-2) in the intervention cohorts and neurosurgeons to establish expertise benchmarks.Performance assessment and personalized feedback by an intelligent tutor on 4 AI-selected learning objectives during simulation training.Outcomes of interest were unintended performance outcomes, measured by significant within-participant difference from baseline in 270 performance metrics in the intervention cohort that was not observed in the control cohort.A total of 46 medical students (median [range] age, 22 [18-27] years; 27 [59%] women) and 14 surgeons (median [range] age, 45 [35-59] years; 14 [100%] men) were included in this study, and no participant was lost to follow-up. Feedback on 4 AI-selected technical competencies was associated with additional performance change in 32 metrics over the entire procedure and 20 metrics during tumor removal that was not observed in the control group. Participants exposed to the AI-enhanced curriculum demonstrated significant improvement in safety metrics, such as reducing the rate of healthy tissue removal (mean difference, -7.05 × 10-5 [95% CI, -1.09 × 10-4 to -3.14 × 10-5] mm3 per 20 ms; P < .001) and maintaining a focused bimanual control of the operative field (mean difference in maximum instrument divergence, -4.99 [95% CI, -8.48 to -1.49] mm, P = .006) compared with the control group. However, negative unintended effects were also observed. These included a significantly lower velocity and acceleration in the dominant hand (velocity: mean difference, -0.13 [95% CI, -0.17 to -0.09] mm per 20 ms; P < .001; acceleration: mean difference, -2.25 × 10-2 [95% CI, -3.20 × 10-2 to -1.31 × 10-2] mm per 20 ms2; P < .001) and a significant reduction in the rate of tumor removal (mean difference, -4.85 × 10-5 [95% CI, -7.22 × 10-5 to -2.48 × 10-5] mm3 per 20 ms; P < .001) compared with control. These unintended outcomes diverged students' movement and efficiency performance metrics away from the expertise benchmarks.In this cohort study of medical students, an AI-enhanced curriculum for bimanual surgical skills resulted in unintended changes that improved performance in safety but negatively affected some efficiency metrics. Incorporating AI in course design requires ongoing assessment to maintain transparency and foster evidence-based learning objectives.