Anticipating Human Motion and Activities (P3)

• Applicant: Professor Dr. Jürgen Gall
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: from 2017 to 2021
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 313421352

The goal of the project is to anticipate human behavior at different granularities, which can be the future activities of a person, but also detailed human motion. Instead of forecasting both granularities independently, it is beneficial to anticipate them jointly. First, detailed human motion can only be forecast for a short time horizon of very few seconds if the intention of the human is unknown. We hypothesize that by anticipating activities, and thus the intention of the person, it will be possible to forecast detailed human motion also for longer time horizons. Second, detailed human motion provides a better visualization and interpretation of the forecast activities. Instead of just having a forecast activity, the model also forecasts the human motion that will be performed to execute the activity. We will in particular focus on modeling the uncertainty. In fact, the future is ambiguous and multiple scenarios can happen in the future. For example, given a video snippet that shows an unknown person taking a cup from the cupboard, we cannot be sure if the person will make tea or coffee. Our goal is therefore to forecast multiple sequences of activities and human motion that reflect the diversity of possible future scenarios. The uncertainty of the future, however, can be reduced if additional information is available. For instance, if we know that the person prefers tea over coffee it is more likely that the person makes tea. Even if the person has not been observed before, external factors like the country, date, time, or the present objects provide additional background information that reduces the uncertainty in the prediction. While coffee is very popular for breakfast in Germany, tea is preferred in the UK and if we know that no coffee is available, making coffee can even be excluded. In order to reduce the forecast uncertainty, we therefore aim to model external factors and condition the models for anticipating human motion and activities on these external factors.

• DFG Programme: Research Units
• Subproject of: FOR 2535: Anticipating Human Behavior