Robotic ultrasound guidance for real-time motion compensated radiation therapy (RobUST), Phase II
Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
Epidemiology and Medical Biometry/Statistics
Final Report Abstract
The research project focuses on the integration of robotic ultrasound (US) systems in medical applications, particularly in the context of radiation therapy treatment. The project encompasses multiple phases and explores various aspects, from data acquisition and analysis to the development of novel algorithms and methodologies for treatment planning. In the initial project phase, first, a long-term volunteer study was conducted using a robotic arm holding a transabdominal probe and an intercostal liver probe. The study revealed challenges with the accuracy of the force estimation model, leading to the implementation of a deep neural network to determine contact forces. This novel approach significantly reduced errors in measuring contact forces. Motion analysis of US probes in different anatomical viewports highlighted varying motion patterns in the liver and prostate. To optimize robotic US image acquisition, the project explored factors affecting tracking accuracy, such as imaging resolution and probe orientation. Also a neural network was trained to adapt the probe orientation based on 3D US images and force values, demonstrating a low error in a phantom study. Real-time processing of volumetric US data was addressed, and machine learning-based approaches were developed for target tracking and deformation estimation. These methods proved to be real-time capable with high accuracy, offering promising solutions for tissue tracking during medical procedures. The project also delved into calibration algorithms for robot-US probe alignment and investigated the impact of the US probe's weight and cable on joint torque measurements. The second part of the project focused on fundamental aspects of integrating robotic ultrasound with a robotic radiation therapy system, i.e., considering the robot carrying the ultrasound probe (US robot) during treatment planning. Furthermore, we studied how motion and deformation detected by real-time ultrasound imaging could be used for online treatment optimization. First, we studied the impact of robotic ultrasound on the plan quality, illustrating that particularly the US robot’s kinematic redundancy allows minimizing adverse effects. Second, we showed that a careful patient specific optimization of the setup is feasible and relevant and we demonstrated that coordinating the US robot and the treatment robot results in reduced treatment time. Third, we proposed a novel approach using machine learning methods for direct aperture optimization, significantly improving target coverage while reducing computational runtime. Forth, we investigated the impact of organ motion and deformation on the treatment plan quality, leading to the development of a new method for fast constraint re-planning after partial delivery of a treatment plan. Overall, this research project contributes valuable insights into the challenges and solutions associated with integrating robotic ultrasound systems into medical applications, particularly in the context of radiation therapy treatment planning. The study showcases advancements in technology, algorithm development, and optimization techniques, laying the foundation for enhanced medical procedures and patient-specific treatment planning.
Publications
-
Robotic ultrasound-guided SBRT of the prostate: feasibility with respect to plan quality. International Journal of Computer Assisted Radiology and Surgery, 12(1), 149-159.
Gerlach, Stefan; Kuhlemann, Ivo; Jauer, Philipp; Bruder, Ralf; Ernst, Floris; Fürweger, Christoph & Schlaefer, Alexander
-
Robots with seven degrees of freedom: Is the additional DoF worth it?. 2016 2nd International Conference on Control, Automation and Robotics (ICCAR), 80-84. IEEE.
Kuhlemann, I.; Jauer, P.; Ernst, F. & Schweikard, A.
-
SU‐G‐JeP3‐03: Effect of Robot Pose On Beam Blocking for Ultrasound Guided SBRT of the Prostate. Medical Physics, 43(6Part27), 3670-3671.
Gerlach, S.; Kuhlemann, I.; Ernst, F.; Fuerweger, C. & Schlaefer, A.
-
Impact of robotic ultrasound image guidance on plan quality in SBRT of the prostate. The British Journal of Radiology, 90(1078).
Gerlach, Stefan; Kuhlemann, Ivo; Ernst, Floris; Fürweger, Christoph & Schlaefer, Alexander
-
Analysis and optimization of the robot setup for robotic-ultrasound-guided radiation therapy. International Journal of Computer Assisted Radiology and Surgery, 14(8), 1379-1387.
Schlüter, Matthias; Gerlach, Stefan; Fürweger, Christoph & Schlaefer, Alexander
-
Methods for Quasi Static Tasks with Redundant Manipulators – Advances in Kinematics, Dexterity and Sensitivity
I. Kuhlemann
-
Optimizing Configurations for 7-DoF Robotic Ultrasound Guidance in Radiotherapy of the Prostate. 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 6983-6986. IEEE.
Schluter, Matthias; Furweger, Christoph & Schlaefer, Alexander
-
Optimizing robot motion for robotic ultrasound-guided radiation therapy. Physics in Medicine & Biology, 64(19), 195012.
Schlüter, Matthias; Fürweger, Christoph & Schlaefer, Alexander
-
Efficient Body Registration Using Single-View Range Imaging and Generic Shape Templates. Current Directions in Biomedical Engineering, 6(3), 119-122.
Çallar, Tolga-Can; Rueckert, Elmar & Böttger, Sven
-
Feasibility and analysis of CNN‐based candidate beam generation for robotic radiosurgery. Medical Physics, 47(9), 3806-3815.
Gerlach, Stefan; Fürweger, Christoph; Hofmann, Theresa & Schlaefer, Alexander
-
Learning Local Feature Descriptions in 3D Ultrasound. 2020 IEEE 20th International Conference on Bioinformatics and Bioengineering (BIBE), 323-330. IEEE.
Wulff, Daniel; Hagenah, Jannis; Ipsen, Svenja & Ernst, Floris
-
Multicriterial CNN based beam generation for robotic radiosurgery of the prostate. Current Directions in Biomedical Engineering, 6(1).
Gerlach, Stefan; Fürweger, Christoph; Hofmann, Theresa & Schlaefer, Alexander
-
“A 3D Slicer module for calibration of spatially tracked 3D ultrasound probes,” Int. J. Comput. Assist. Ra diol. Surg., vol. 15, no. S1, pp. 14–16, Jun. 2020. (OA)
F. von Haxthausen, S. Ipsen, H. Schwegmann, R. Bruder, F. Ernst & V. García-Vázquez
-
Analysis of ultrasound and optical coherence tomography for markerless volumetric image guidance in robotic radiosurgery
M. Schlüter
-
Towards automated ultrasound imaging—robotic image acquisition in liver and prostate for long-term motion monitoring. Physics in Medicine & Biology, 66(9), 094002.
Ipsen, Svenja; Wulff, Daniel; Kuhlemann, Ivo; Schweikard, Achim & Ernst, Floris
-
“Comparison of Representation Learning Techniques for Tracking in time resolved 3D Ultrasound,” in Proceedings of Medical Imaging with Deep Learning, 2021. (OA)
D. Wulff, J. Hagenah & F. Ernst
-
AI-based optimization for US-guided radiation therapy of the prostate. International Journal of Computer Assisted Radiology and Surgery, 17(11), 2023–2032.
Gerlach, Stefan; Hofmann, Theresa; Fürweger, Christoph & Schlaefer, Alexander
-
Generalized Automatic Probe Alignment based on 3D Ultrasound. Current Directions in Biomedical Engineering, 8(1), 58-61.
Osburg, Jonas; Wulff, Daniel & Ernst, Floris
-
Landmark tracking in 4D ultrasound using generalized representation learning. International Journal of Computer Assisted Radiology and Surgery.
Wulff, Daniel; Hagenah, Jannis & Ernst, Floris
-
Robotic Systems in Radiotherapy and Radiosurgery. Current Robotics Reports, 3(1), 9-19.
Gerlach, Stefan & Schlaefer, Alexander
-
Systematic analysis of volumetric ultrasound parameters for markerless 4D motion tracking. International Journal of Computer Assisted Radiology and Surgery, 17(11), 2131-2139.
Sprenger, Johanna; Bengs, Marcel; Gerlach, Stefan; Neidhardt, Maximilian & Schlaefer, Alexander
-
Using Deep Neural Networks to Improve Contact Wrench Estimation of Serial Robotic Manipulators in Static Tasks. Frontiers in Robotics and AI, 9.
Osburg, Jonas; Kuhlemann, Ivo; Hagenah, Jannis & Ernst, Floris
-
“TH-B-206-02: Fast Adaptive Replanning by Constrained Reoptimization for Intra-Fractional Non-Periodic Motion During SBRT of the Prostate,” Med. Phys., vol. 49, no. 6, Jun. 2022
S. Gerlach, T. Hofmann, C. Fuerweger & A. Schlaefer
-
Real-Time Motion Analysis With 4D Deep Learning for Ultrasound-Guided Radiotherapy. IEEE Transactions on Biomedical Engineering, 70(9), 2690-2699.
Bengs, Marcel; Sprenger, Johanna; Gerlach, Stefan; Neidhardt, Maximilian & Schlaefer, Alexander
-
Target Tracking in 4D US Based on Template Matching and Target Forecasting Using Spatio-Temporal Autoencoders. 2023 IEEE 23rd International Conference on Bioinformatics and Bioengineering (BIBE), 113-120. IEEE.
Wulff, Daniel; Sarau, Ricardo & Ernst, Floris
-
Towards fast adaptive replanning by constrained reoptimization for intra‐fractional non‐periodic motion during robotic SBRT. Medical Physics, 50(7), 4613-4622.
Gerlach, Stefan; Hofmann, Theresa; Fürweger, Christoph & Schlaefer, Alexander
-
Towards Realistic 3D Ultrasound Synthesis: Deformable Augmentation using Conditional Variational Autoencoders. 2023 IEEE 36th International Symposium on Computer-Based Medical Systems (CBMS), 821-826. IEEE.
Wulff, Daniel; Dohnke, Timon; Nguyen, Ngoc Thinh & Ernst, Floris
-
Feature Description using Autoencoders for Fast 3D Ultrasound Tracking. Current Directions in Biomedical Engineering, 10(2), 21-24.
Wulff, Daniel & Ernst, Floris
-
Target Tracking in 4D Ultrasound using Localization Networks. Current Directions in Biomedical Engineering, 10(2), 29-32.
Krause, Cassandra; Wulff, Daniel & Ernst, Floris
