Exploring the Design Space of Situated Analytics

• Applicant: Professor Dr. Michael Sedlmair
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: since 2022
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 495135767

This project will study situated analytics (SitA), that is, analytic work and sense-making that is embedded into the spatial environment of a user. Using augmented reality (AR) displays, visualizations will be spatially registered to referents, i.e., physical objects around a user. SitA will allow to support analytical questions that need to combine physical (objects) and digital (data) information.At the moment, it remains largely unclear how to properly leverage the physical context provided by spatial referents to aid a user in sense-making. Important research questions that will be addressed are if/how analytic work can be interleaved with physical activities, how users prefer to arrange their visualizations near or around referents, and how to make the visualizations not only respond to explicit actions of the user, but also implicitly to arbitrary changes in the environment. A final research question concerns situated authoring. Rather than letting users analyze data using a finite set of visual representations and interactions, users are encouraged to create novel content embedded in their physical environment. We strongly believe that augmented reality (AR) is now mature enough to fully support SitA. However, to move beyond the current capabilities afforded by immersive analytics (which mostly deals with virtual reality), we first require a dedicated toolkit for building SitA specifically for AR platforms. We propose to build such a toolkit for SitA and use it for experiments on the nature of interaction between visualizations and referents. The toolkit must provide features that are currently not available. First, it must be able to embed interactive visualizations with referents based on a dynamic physical-virtual model (akin to a digital twin). Second, it must be aware of the real world context, such as of active devices used as referents. Third, it must allow situated authoring of visualizations and not merely authoring of situated visualizations. We will then use the toolkit to implement an evaluation testbed for the empirical studies of the research questions summarized above.In contrast to immersive analytics, which is now a strong trend, research on SitA has been scarce so far. Referents add substantial complexity to already complex research issues in visualization and analytics, and the lack of toolkits specifically targeting the need for SitA means that even basic questions about referents, such as the ones mentioned above, have barely been addressed yet. Our proposal lays out a plan to change this and lay groundwork in this new field. Michael Sedlmair is full professor at University of Stuttgart, Germany, and has extensive experience in visualization, visual analytics, and human-computer interaction. Dieter Schmalstieg is full professor at Graz University of Technology and is well-known for his work in augmented and virtual reality.

• DFG Programme: Research Grants
• International Connection: Austria
• Partner Organisation: Fonds zur Förderung der wissenschaftlichen Forschung (FWF)
• Cooperation Partner: Professor Dr. Dieter Schmalstieg