Diagnosing alternative conceptions (DIALOC) — Assessing the quality of diagnosis and diagnostic activities of (prospective) teachers using an interactive assessment tool

• Applicants: Privatdozent Dr. Sascha Bernholt; Dr. Nina Minkley
• Subject Area: General and Domain-Specific Teaching and Learning
• Term: since 2025
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 566880195

The conceptions of students about natural scientific phenomena often lead to misconceptions and significantly impact the process of understanding. Therefore, they are of great importance for academic learning, which is why they should be explicitly considered in lesson planning, implementation, and reflection. Many alternative conceptions (AC), are, however, persistent to instructional attempts at change and pose a continuous challenge for (prospective) teachers. The latter may be attributed, among other things, to a lack of systematic learning opportunities, as, for instance, academic courses may provide limited interactive engagement with AC, while they arise unsystematically in complex teaching scenarios and may be overshadowed by other occurrences. Accordingly, existing assessment and practice tools (e.g., text and video vignettes, questionnaires) provide the advantage of complexity reduction to systematically examine the diagnostic process but lack interactivity. The present project aims to systematically investigate the diagnostic activities and diagnostic quality of prospective teachers in dealing with (alternative) student conceptions in an interactive setting that enables a discursive diagnostic process. Diagnostic quality is operationalized in terms of accuracy (recognized vs. not recognized) and the level of diagnostic judgment (describe, evaluate, explain), while diagnostic activities are classified along the main categories of perceiving, interpreting, and judging. Additionally, the pattern of diagnostic activities is recorded (e.g., frequency, proportion of questions with high diagnostic potential, timing of judgment). Therefore, in Study 1 (qualitative), the diagnostic quality and diagnostic activities used by (prospective) teachers to recognize, evaluate, and explain AC are examined. In Study 2 (quantitative), the relationship between diagnostic quality and situational and personal characteristics, as well as diagnostic activities, is investigated. In the data collection, participants interact with various chatbots based on a generative model, responding to questions in alignment with a specific AC or a scientifically correct conception (virtual student) to create an interactive diagnostic situation. Therefore, the participants are asked to determine the conception related to the given theme, followed by describing, evaluating, and explaining it. The independent variable context (situation characteristic) is systematically varied (Biology, Chemistry, and interdisciplinary) and recorded along with personal characteristics (e.g., content knowledge, pedagogical content knowledge, attitude). The dependent variables diagnostic quality and diagnostic activities are systematically assessed based on chat logs, written judgments, and thinking aloud.

• DFG Programme: Research Grants