Model-based mesh-to-grid image resampling with application to robust object detection, recognition and tracking

• Applicant: Professor Dr.-Ing. André Kaup
• Subject Area: Electronic Semiconductors, Components and Circuits, Integrated Systems, Sensor Technology, Theoretical Electrical Engineering
• Term: from 2018 to 2021
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 402837983

Detection, recognition and tracking of objects are common tasks in video processing and they are widely used in various fields ranging from surveillance and medical imaging to automotive and entertainment industry. These tasks are, however, highly sensitive to image degradations such as low resolution input data, noise or optical and perspective distortions. In fact, when such common image conditions are faced, the performance of detection, recognition and tracking is severely deteriorated and it may also completely fail. To tackle this problem, input frames are typically preprocessed in order to counteract the effects of these adverse image conditions. First, input frames are registered. This registration typically operates with sub-pixel accuracy so the registered samples are located at arbitrary non-integer positions, called mesh. Hence, in a second step, mesh-to-grid resampling is applied. It consists of reconstructing the pixels on the regular 2D grid from the available mesh of samples and it conditions the resulting quality. Current solutions for resampling show significant low-pass behaviour and tend to introduce various visual artefacts which produce considerable and even irrecoverable errors during detection, recognition and tracking. Therefore, the goal of this project is to develop a robust high-quality mesh-to-grid resampling technique in order to leverage object detection, recognition and tracking. We take advantage of the so called frequency selective mesh-to-grid resampling algorithm which is an iterative procedure that generates an image model using a set of suitable basis functions. This technique exhibits an excellent performance in an ideal error-free environment. However, the performance of this algorithm drops drastically in real world scenarios where estimation errors during registration are involved. These scenarios shall be analysed by theoretical considerations in order to develop mechanisms to deal with these errors, to stabilise the output and to make it robust against noisy input data. The developed mesh-to-grid resampling technique will be tested for various detection, recognition and tracking applications. We expect to boost the performance of detection, recognition and tracking and make them correctly function in scenarios where other resamplers currently lead to failure. Finally, the obtained algorithms as well as the simulation framework will be made available to the scientific community in form of a software toolbox.

• DFG Programme: Research Grants