Improving cell tracking by jointly handling missed detections, false detections, cell divisions and higher-order motion models

• Applicant: Professor Dr. Fred A. Hamprecht
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: from 2016 to 2020
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 286502966

• Final Report Year: 2020

Final Report Abstract

In this project, we set out to improve an already sophisticated computational procedure to track (biological) objects, that is, to follow them over time in a (microscopic) video. It turned out that our previous model already did very well in an international competition, but that its main source of errors was not the tracking model itself, but rather the segmentation procedure used. “Segmentation” is the process of delineating an object (such as a cell or a roundworm) in an image. In response, we have mainly concentrated on improving the quality of the segmentation itself. Our first contribution is a procedure that finds multiple plausible segmentation candidates, by finding not just the very best, but several good and yet diverse solutions to energy functions describing the interaction of elements that are arranged in a treeshaped manner. Such energy functions comprise e.g. nested segmentation hypotheses, or the medial axis of roundworms. Our second major contribution is a scheme to take into account non-local cues in difficult segmentation problems. The third contribution has characterized the optimization problem inherent in deep neural networks. These architectures have drastically improved the achievable (segmentation) accuracy in recent years, but it is unclear just why these methods perform so well, and why they can be trained with relative ease. Indeed, a modern neural network has millions of parameters that need to be configured, and it is surprising that it is possible to do so, successfully, with limited training data and compute power. We have made a rather surprising discovery, namely that such networks do not have distinct minima in the function that needs to be optimized at train time! Instead, the minimum seems to be a single, connected manifold. It is fair to say that this work has caused a bit of a stir in the theoretical community, which is now trying to understand this phenomenon. Finally, we have put a lot of effort into making our methods available in open-source software. All methods developed as part of this project are publicly available as part of open-source software packages and some of them have been published in the most competitive international conferences. In ongoing work, we are trying to develop a neural network that can solve the segmentation and tracking problem jointly. Two of the cooperation partners in this project have in the meantime become group leaders at EMBL and EBI, two leading biological research institutes. They are at the forefront of a new generation of successful female scientists that push the boundary of what is possible today in bioimage analysis.

Publications

• “Diverse M-best solutions by dynamic programming.” In German Conference on Pattern Recognition, pp. 255-267. Springer (2017)
  Haubold, Uhlmann, Unser, Hamprecht
  
• “DiversePathsJ: diverse shortest paths for bioimage analysis.” Bioinformatics 34, no. 3 (2018)
  Uhlmann, Haubold, Hamprecht, Unser
  
• “Essentially No Barriers in Neural Network Energy Landscape” International Conference on Machine Learning (2018)
  Draxler, Veschgini, Salmhofer, Hamprecht
  
• “ilastik: Interactive machine learning for (bio) image analysis.” Nature Methods (2019)
  Berg, Kutra, Kroeger, Straehle, Kausler, Haubold, Hamprecht, Kreshuk
  
• “Leveraging Domain Knowledge to Improve Microscopy Image Segmentation with Lifted Multicuts.” Frontiers in Computer Science 1 (2019)
  Pape, Matskevych, Wolny, Hennies, Mizzon, Louveaux, Musser, Maizel, Arendt, Kreshuk