Deep cognitive diagnosis in intelligent tutoring systems in the framework of logic programming and meta-level reasoning
Final Report Abstract
• We have shown that our innovative adaptation of algorithmic debugging is an effective and valuable tool to identify errors in learners’ problem solving. • We have demonstrated that automatic program perturbation, i.e., the automatic introduction of bugs into expert models, is an effective mechanism to reproduce the erroneous procedures learners are following. • We have initial results that show that inductive logic programming can be successfully used to generate generic clauses from observed learner behaviour. • We have combined algorithmic debugging, code perturbation and inductive logic programming to perform deep cognitive analyses of learner errors. • We have applied the techniques in multiple domains. • From an educational point of view, van Lehn’s theory of impasses and repairs was found to be particularly well-suited to our logic and meta-level reasoning approach. Errors that were manually reproduced by van Lehn and others were reproduced by our methods automatically. • From an application point of view, multiple, prototypical intelligent tutoring systems based upon the technology have been built and made available via browser-based web interfaces. We have studied, devised and implemented a domain-independent and reusable inventory of three methods to reconstruct students’ erroneous procedures from observed learner behaviour and expert procedures modelling ideal problem solving behaviour. While none of the three methods, taken in isolation, is truly new, at least in the declarative programming community, their innovative adaptation and combined practical use in the tutoring context has shed new light on their nature, and has initiated a promising and more systematic approach for the deep analysis of student input in intelligent tutoring systems. There is good evidence that a sound methodology of cognitive diagnosis can be realised in the framework of logic programming. Its declarative aspect, its view that program equals data, the substantial work in areas such as meta-level interpreters, partial evaluation, reasoning about programs, algorithmic debugging and inductive logic programming shows that there is ample potential to harness such techniques to support cognitive diagnosis in the context of intelligent tutoring systems. Our project advanced cognitive diagnosis in this direction. To our knowledge, it was the first project of a kind that aims at systematically developing, adapting and studying various methods to effectively perform cognitive diagnosis in realistic educational settings. While our research touched many issues related to the processes of learning, it was our primary interest to develop an innovative and solid arsenal of diagnostic models and to exploit them by building software to support learners and teachers.
Publications
- Program analysis and manipulation to reproduce learner’s erroneous reasoning. In E. Albert, editor, 22nd Intl. Symp. of Logic-Based Program Synthesis and Transformation, volume 7844 of LNCS, pages 228–243. Springer, 2012
C. Zinn
(See online at https://doi.org/10.1007/978-3-642-38197-3_15) - Algorithmic debugging for intelligent tutoring: How to use multiple models and improve diagnosis. In I. J. Timm, editor, KI-2013: From Research to Innovation and Practical Applications, volume 8077 of LNAI, pages 272-283, Springer, 2013
C. Zinn
(See online at https://doi.org/10.1007/978-3-642-40942-4_24) - Heuristic Search over Program Transformations. In M. Hanus and R. Rocha, editors, WFLP 2013 – Functional and (Constraint) Logic Programming, volume 8439 of LNAI, pages 234-249, Springer, 2013
C. Zinn
(See online at https://doi.org/10.1007/978-3-319-08909-6_15) - Algorithmic Debugging and Literate Programming to Generate Feedback in Intelligent Tutoring Systems. In C. Lutz and M. Thielscher, editors, KI 2014, volume 8736 of LNAI, pages 37–48. Springer, 2014
C. Zinn
(See online at https://doi.org/10.1007/978-3-319-11206-0_4)
