PERIAPT: Joint Person Detection, Re-Identification and Pose Tracking in Video

• Applicant: Professor Dr. Jürgen Gall
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: from 2019 to 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 410904267

• Final Report Year: 2023

Final Report Abstract

For many applications like robotics, autonomous driving, or smart factories, machines have to be aware of humans in their vicinity. This requires to know the location and pose of the present humans and track them continuously. Although person detection, person re-identification and pose tracking are highly correlated, the three tasks have been previously studied independently due to lack of a dataset that contains annotations for all three tasks. We therefore created a large-scale dataset, called PoseTrack21, which closes this gap. The dataset contains over 400,000 annotated bounding boxes, over 170,000 annotated human poses with occlusion flags, and annotated IDs for tracking and person search. Furthermore, we investigated how the three tasks can assist each other to improve the accuracy in particular in case of occlusions.

Publications

• Hierarchical Online Instance Matching for Person Search. Proceedings of the AAAI Conference on Artificial Intelligence, 34(07), 10518-10525.
  Chen, Di; Zhang, Shanshan; Ouyang, Wanli; Yang, Jian & Schiele, Bernt
  
• Norm-Aware Embedding for Efficient Person Search. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE.
  Chen, Di; Zhang, Shanshan; Yang, Jian & Schiele, Bernt
  
• Self-supervised Keypoint Correspondences for Multi-person Pose Estimation and Tracking in Videos. Lecture Notes in Computer Science, 36-52. Springer International Publishing.
  Rafi, Umer; Doering, Andreas; Leibe, Bastian & Gall, Juergen
  
• Guided Attention in CNNs for Occluded Pedestrian Detection and Re-identification. International Journal of Computer Vision, 129(6), 1875-1892.
  Zhang, Shanshan; Chen, Di; Yang, Jian & Schiele, Bernt
  
• Hierarchical Information Passing Based Noise-Tolerant Hybrid Learning for Semi-Supervised Human Parsing. Proceedings of the AAAI Conference on Artificial Intelligence, 35(3), 2207-2215.
  Liu, Yunan; Zhang, Shanshan; Yang, Jian & Yuen, PongChi
  
• Improving Pedestrian Detection from a Long-tailed Domain Perspective. Proceedings of the 29th ACM International Conference on Multimedia, 2918-2926. ACM.
  Ding, Mengyuan; Zhang, Shanshan & Yang, Jian
  
• Learning a Dynamic High-Resolution Network for Multi-Scale Pedestrian Detection. 2020 25th International Conference on Pattern Recognition (ICPR), 9076-9082. IEEE.
  Ding, Mengyuan; Zhang, Shanshan & Yang, Jian
  
• Learning Scale-Adaptive Representations for Point-Level LiDAR Semantic Segmentation. 2021 International Conference on 3D Vision (3DV), 920-929. IEEE.
  Zhang, Tongfeng; Yang, Kaizhi & Chen, Xuejin
  
• Nighttime Pedestrian Detection Based on Feature Attention and Transformation. 2020 25th International Conference on Pattern Recognition (ICPR), 9180-9187. IEEE.
  Li, Gang; Zhang, Shanshan & Yang, Jian
  
• Norm-Aware Embedding for Efficient Person Search and Tracking. International Journal of Computer Vision, 129(11), 3154-3168.
  Chen, Di; Zhang, Shanshan; Yang, Jian & Schiele, Bernt
  
• Recursive Bayesian Filtering for Multiple Human Pose Tracking from Multiple Cameras. Lecture Notes in Computer Science, 438-453. Springer International Publishing.
  Kwon, Oh-Hun; Tanke, Julian & Gall, Juergen
  
• Unified Density-Aware Image Dehazing and Object Detection in Real-World Hazy Scenes. Lecture Notes in Computer Science, 119-135. Springer International Publishing.
  Zhang, Zhengxi; Zhao, Liang; Liu, Yunan; Zhang, Shanshan & Yang, Jian
  
• Keypoint Message Passing for Video-Based Person Re-identification. Proceedings of the AAAI Conference on Artificial Intelligence, 36(1), 239-247.
  Chen, Di; Doering, Andreas; Zhang, Shanshan; Yang, Jian; Gall, Juergen & Schiele, Bernt
  
• Knowledge Distillation for Object Detection via Rank Mimicking and Prediction-Guided Feature Imitation. Proceedings of the AAAI Conference on Artificial Intelligence, 36(2), 1306-1313.
  Li, Gang; Li, Xiang; Wang, Yujie; Zhang, Shanshan; Wu, Yichao & Liang, Ding
  
• PoseTrack21: A Dataset for Person Search, Multi-Object Tracking and Multi-Person Pose Tracking. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 20931-20940. IEEE.
  Doering, Andreas; Chen, Di; Zhang, Shanshan; Schiele, Bernt & Gall, Juergen
  
• A Dual-Source Attention Transformer for Multi-Person Pose Tracking, 2023
  Andreas Doering & Juergen Gall