3D+t Terabyte Image Analysis

• Applicants: Professor Dr.-Ing. Ralf Mikut; Professor Dr. Peter Sanders
• Subject Area: Theoretical Computer Science ; Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: from 2015 to 2019
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 268484869

• Final Report Year: 2019

Final Report Abstract

In the scope of the present project, we developed methods for processing high resolution 3D+t images with an application focus on time-resolved 3D microscopy data. The goal was to obtain very high performance both with respect to result quality and computational efficiency on modern hardware. The exemplary application domain was detection, segmentation and tracking of objects (e.g., cell nuclei and cell plasma membranes) in 3D+t microscopy images where different object classes are labeled with particular fluorescent dyes. These tracking problems are highly important in biological research and challenging because the known computational approaches can only scratch the surface of the wealth of information available in the huge datasets currently being acquired. Our strategy was to couple high-performance algorithms for 3D image processing with uncertainty information for an improved detection quality while having a negligible impact on processing times. Fuzzy set theory was used as an overarching framework for uncertainty estimation and propagation. We used the new framework for uncertainty modeling and propagation to extend detection, segmentation and tracking algorithms by uncertainty treatment. During the project duration, deep learning approaches became more and more popular and we also decided to exploit powerful new approaches like convolutional neural networks for improved segmentation correction heuristics, which was initially not aspired in the project proposal. To quantitatively assess the result quality of the developed algorithms under realistic imaging conditions, we developed a large-scale, realistic, simulated benchmark dataset including ground truth and with varying levels of noise and blur. All developed algorithms are incorporated into the new open source tools XPIWIT and EmbryoMiner that have been made publicly available for the scientific community. Taken together, these two tools span the entire portfolio of methods required for analyzing large-scale 3D+t data, e.g., originating from projects in developmental biology. XPIWIT implements all imagebased algorithms that were developed for fast object detection and segmentation in 3D and can be executed even on computing clusters. The MATLAB toolbox EmbryoMiner subsequently can perform 3D object tracking and enables both interactive and automatic analyses of the data including access to various preexisting data mining methods. We have demonstrated the performance of our methods in multiple international cooperations in basic biological research and analyzed large real-world inputs of developing embryos generated by 3D confocal and 3D+t light-sheet microscopy (including analyses where a single-embryo experiment easily reached more than 10 terabytes of data).

Publications

• Generating semi-synthetic validation benchmarks for embryomics, in: Proc., IEEE International Symposium on Biomedical Imaging: From Nano to Macro, 2016
  J. Stegmaier, J. Arz, B. Schott, J. C. Otte, A. Kobitski, G. U. Nienhaus, U. Strähle, P. Sanders, R. Mikut
  
• XPIWIT - an XML pipeline wrapper for the insight toolkit, Bioinformatics 32(2) (2016) 315–317
  A. Bartschat, E. Hübner, M. Reischl, R. Mikut, J. Stegmaier
  
• 3D cell nuclei segmentation with balanced graph partitioning
  J. Arz, P. Sanders, J. Stegmaier, R. Mikut
  
• An objective comparison of cell-tracking algorithms, Nature Methods 14 (12) (2017) 1141–1152
  V. Ulman, M. Maska, K. E. G. Magnusson, O. Ronneberger, C. Haubold, N. Harder, P. Matula, P. Matula, D. Svoboda, M. Radojevic, I. Smal, K. Rohr, J. Jalden, H. M. Blau, O. Dzyubachyk, B. Lelieveldt, P. Xiao, Y. Li, S.-Y. Cho, A. C. Dufour, J.-C. Olivo- Marin, C. C. Reyes-Aldasoro, J. A. Solis-Lemus, R. Bensch, T. Brox, J. Stegmaier, R. Mikut, S. Wolf, F. A. Hamprecht, T. Esteves, P. Quelhas, m. Demirel, L. Malmstrom, F. Jug, P. Tomancak, E. Meijering, A. Munoz-Barrutia, M. Kozubek, C. Ortizde Solorzano
  
• Fuzzy-based propagation of prior knowledge to improve large-scale image analysis pipelines, PLoS ONE 12 (11) (2017) e0187535
  J. Stegmaier, R. Mikut
  
• EmbryoMiner: A new framework for interactive knowledge discovery in large-scale cell tracking data of developing embryos, PLoS Computational Biology 14(4) (2018) e1006128
  B. Schott, M. Traub, C. Schlagenhauf, M. Takamiya, T. Antritter, A. Bartschat, K. Löffler, D. Blessing, J. C. Otte, A. Y. Kobitski, G. U. Nienhaus, U. Strähle, R. Mikut, J. Stegmaier