Unsupervised audio-visual geometry calibration of distributed microphone arrays

• Applicant: Professor Dr.-Ing. Reinhold Häb-Umbach
• Subject Area: Acoustics
• Term: from 2010 to 2014
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 166631756

The overall goal of the project is to simplify the installation of audio-visual sensor networks. This is achieved by developing algorithms for automatically determining the position of the distributed microphone arrays from reverberant speech input. Then correspondences between events sensed by the acoustic sensor network and those sensed by a sensor network of another modality are established. This other modality is given by a multi-camera-network, where the camera positions are assumed to be known. Once the correspondences have been detected the relative geometries of the acoustic sensor network can be mapped to the given coordinate system of the camera network.In the first project phase we have developed algorithms for microphone array posiltion self calibration in 2D, which are characterized by immunity to reverberation and which can operate on speech input, not requiring artificial calibration signals.The requested project extension is devoted to the following objectives:1) A correspondence will be developed between the relative geometries of the acoustic sensor network and a global coordinate system, which is given by a multi-camera system. This shall be achieved by mapping trajectories of speakers obtained from the acoustic subsystem to trajectories of the visual correlates, i.e., faces or persons.2) The developed geometry self calibration system will be embedded in an ambient communication system for evaluation purposes.In the ambient communication system an acoustic beamformer extracts the speech signal, while the active camera system tracks the speaker. In this test bed the interaction between user and system can be studied in a real-time environment and under realistic environmental conditions, eventually leading to an optimization of the calibration system.

• DFG Programme: Research Grants
• Participating Person: Professor Dr.-Ing. Gernot A. Fink