LF4D - Four-dimensional light field microscopy
Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
Optics, Quantum Optics and Physics of Atoms, Molecules and Plasmas
Final Report Abstract
A systems-level understanding of biomedical processes in intact tissue necessitates fast, three-dimensional videography of dynamic processes with cellular resolution. Capturing a visual scene simultaneously from several perspectives (its light field) allows extracting depth information from single acquisitions without the need for time-consuming scanning procedures. With the advent of sufficiently large camera sensors and precision micromanufacturing of multi-lens arrays, light field microscopy can be realized in very compact imaging instrumentation operating without moving parts. However, there are substantial challenges to develop efficient and fast reconstruction algorithms that yield accurate and robust results in realistic biological samples. In this research project, we thus joined forces between an expert team focused on image reconstruction, and a laboratory specialized on molecular imaging of zebrafish and human organoids to enable 4D light field microscopy for high-throughput microscopic analysis of dynamic molecular processes. We reconstructed light field data from progressively more difficult biological model systems in four steps from static samples with constant signal intensities to moving samples with time-varying signal intensities. In this project we developed new forward models for light field microscopy and the newer Fourier light field microscopy, together with novel reconstruction algorithms, that come in both classic, model-based flavors as well as in new deep learning based flavors, enabling biological laboratories worldwide to utilize four-dimensional light field microscopy to tackle biological questions related to high-throughput 3D gene expression screening, in-vivo biomechanics, and synchronicity of neuronal activity patterns.
Publications
-
Artifact-free deconvolution in light field microscopy. Optics Express, 27(22), 31644.
Stefanoiu, Anca; Page, Josue; Symvoulidis, Panagiotis; Westmeyer, Gil G. & Lasser, Tobias
-
oLaF: A flexible 3D reconstruction framework for light field microscopy. TUM Technical Report TUMI-1978, 2019.
A. Stefanoiu; J. Page & T. Lasser
-
Deconvolution in Fourier integral microscopy. Computational Imaging V, 18. SPIE.
Stefanoiu, Anca; Scrofani, Gabriele; Saavedra, Genaro; Martínez-Corral, Manuel & Lasser, Tobias
-
What about computational super-resolution in fluorescence Fourier light field microscopy?. Optics Express, 28(11), 16554.
Stefanoiu, Anca; Scrofani, Gabriele; Saavedra, Genaro; Martínez-Corral, Manuel & Lasser, Tobias
-
Learning to Reconstruct Confocal Microscopy Stacks From Single Light Field Images. IEEE Transactions on Computational Imaging, 7, 775-788.
Vizcaíno, Josué Page; Saltarin, Federico; Belyaev, Yury; Lyck, Ruth; Lasser, Tobias & Favaro, Paolo
-
Real-Time Light Field 3D Microscopy via Sparsity-Driven Learned Deconvolution. 2021 IEEE International Conference on Computational Photography (ICCP), 1-11. IEEE.
Vizcaino, Josue Page; Wang, Zeguan; Symvoulidis, Panagiotis; Favaro, Paolo; Guner-Ataman, Burcu; Boyden, Edward S. & Lasser, Tobias
-
Immobilized fluorescently stained zebrafish through the eXtended Field of view Light Field Microscope 2D-3D dataset. Zenodo, June 2023.
J. Page Vizcaíno; P. Symvoulidis; Z. Wang; J. Jelten; P. Favaro; E. Boyden & T. Lasser
-
Fast light-field 3D microscopy with out-of-distribution detection and adaptation through conditional normalizing flows. Biomedical Optics Express, 15(2), 1219.
Page, Vizcaíno Josué; Symvoulidis, Panagiotis; Wang, Zeguan; Jelten, Jonas; Favaro, Paolo; Boyden, Edward S. & Lasser, Tobias
