Servicenavigation

Project Details

Back

Projekt Print View

LIVE: Empirical Studies on the Effects of Liveness on Programming

Subject Area Software Engineering and Programming Languages
Term from 2020 to 2024
Project identifier Deutsche Forschungsgemeinschaft (DFG) - Project number 449591262
 
Final Report Year 2024

Final Report Abstract

Liveness in programming tools is the impression of changing a program while it is running. Various tools support liveness, including commercial programming systems, such as MS Excel and Jupyter Notebooks. Tool designers assume that liveness improves the programming experience, but this assumption has insufficient and inconclusive empirical backing. This lack of evidence might lead to the promotion of liveness in unsuitable settings and the neglect of important settings, which would waste design and implementation efforts. In this project, we investigated the effects of live tools on debugging. In two controlled experiments we studied the influence of task complexity and delayed interactions on the effects of live tools. Compared to previous experiments on liveness, the participants in our experiments had considerable experience with live tools. In our first experiment we tested whether the influence of live tools on debugging time differs for simple and complex tasks. We found that live tools significantly shorten the time needed to debug defects. At the same time, we could not confirm our main hypothesis that task complexity moderates this effect. However, our results indicate that task complexity indeed influences the effect, but less than suggested by the pilot. For programming tool researchers and designers, our results show that programmers can benefit from live tools, but that they need to consider task complexity and participants’ experience with liveness when preparing studies or building tools. With our second experiment, we aimed to better understand the first experiment’s observations. Based on Information Foraging Theory, we assumed that live tools reduce the perceived cost of obtaining dynamic information so that programmers consult it more often when helpful. Therefore, we tested whether programmers use live tools less frequently if access to them is delayed. The experiment did not yield sufficiently enough results for a thorough analysis, but the collected data shows no clear decline in live tool usage. Yet, an ongoing post hoc analysis using edit-run cycles suggests that participants’ workflows changed. During the first experiment, we found that it is a great challenge to operationalize the complexity of maintenance tasks in programming tool studies. Thus, we conducted a survey to curate a collection of factors from related studies that can help shape the complexity of such tasks. With this collection, researchers can deliberately decide on the complexity level for their studies’ tasks. This project also resulted in a novel concept for teaching debugging through contests and improved setups for related studies on liveness conducted in our group.

Publications

 
 

Additional Information

© 2026 DFG Disclaimer / Copyright / Privacy Policy

Textvergrößerung und Kontrastanpassung