Artificial Intelligence and Machine Learning in Minimally Invasive Upper GI Cancer Surgery: Workflow Analysis and Intraoperative Event Prediction towards Risk Mitigation

• Applicant: Dr. Jennifer Aylin Eckhoff
• Subject Area: General and Visceral Surgery
• Term: since 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 520324645

Artificial Intelligence (AI) in minimally invasive surgery holds the potential to transform surgical teaching, provide intraoperative decision support and reduce complication rates. Yet this translational science is waiting to be transferred to complex, oncological procedures to reveal its true risk mitigation potential. Machine Learning (ML) and Computer Vision (CV) models can provide predictive analyses of temporal, spatial, and conceptual aspects of surgery. Algorithms are trained to detect and predict clinically-relevant target features in surgical video data, previously assessed by advanced surgeons in the form of annotations. However, existing applications merely focus on highly standardized, low-risk procedures such as laparoscopic cholecystectomy. This research proposal offers advanced approaches for applying AI for automated comprehension of surgical workflow to complex, oncological upper gastrointestinal (GI) surgery. The first goal of this project is the assessment, validation, and implementation of a standardized framework for surgical video annotation, data acquisition, and management specific to video data of foregut surgery. This framework will provide a comprehensive infrastructure and set the foundation for future AI research and the following aims of this proposal. Secondly, the applicant will develop ML models for the detection and prediction of surgical phases, tools, and tissues as well as tool-tissue-action dependencies in robotic-assisted minimally invasive esophagectomy (RAMIE). This will result in a spatial and temporal map of RAMIE, scalable to various research problems. Thirdly, the applicant will use the established methodologies to perform a CV-based analysis of ICG positivity for delineation of ideal resection areas in lymphadenectomy during RAMIE. The expertise of the Surgical Artificial Intelligence and Innovation Laboratory (SAIIL), a collaborative working group between the Massachusetts Institute of Technology and Harvard Medical School, will provide her with the necessary prerequisites to achieve the proposed aims and advance surgical AI. Besides the profound computational skills, the applicant acquired during her research fellowship at SAIIL thus far, the support and mentoring of her clinical and technical colleagues at SAIIL and her home institution will enable her to fulfill the described methodologies. This research proposal lays the foundation for AI-based risk mitigation and intraoperative decision support in complex oncological upper GI surgery.

• DFG Programme: WBP Fellowship
• International Connection: USA
• Host: Professor Ozanan Meireles