Confidence-based learning: establishing a novel form of learning without feedback

• Applicant: Dr. Marcus Rothkirch, since 2/2022
• Subject Area: Human Cognitive and Systems Neuroscience ; Biological Psychiatry
• Term: from 2018 to 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 403630675

Theories of reinforcement learning explain fundamental characteristics of human learning and behaviour, but also how aberrancies in learning can lead to the manifestation of dysfunctional behavioural patterns and psychiatric diseases. However, a critical challenge for reinforcement learning accounts is that many forms of learning occur in the absence of external feedback and important aspects of psychiatric diseases are best characterized by self-reinforcing processes that do not depend on external feedback. Recently, theoretical progress and empirical findings in the field of perceptual learning have made the case for an intriguing extension of reinforcement learning, proposing that internal confidence in one’s own actions serves as a teaching signal when no external feedback is available. Moreover, it has been found that the computational principles and the neural underpinnings of such learning are highly similar to those of normative reinforcement learning based on external feedback. On this basis, the overarching goal of the present proposal is to 1) establish the generality of this novel form of learning at the behavioural and neurobiological level, and 2) to investigate interindividual differences in such learning in the paradigmatic case of proneness to anhedonia.In a first work package, we will investigate in two functional magnetic resonance imaging experiments whether the observed parallel between normative and confidence-based learning generalizes to the two most generic forms of reinforcement learning − classical and instrumental conditioning. The first experiment will examine whether the repeated pairing of a stimulus with induced levels of confidence in a distractor task leads to ‘confidence-based classical conditioning’, and thus to the behavioural, physiological and neurobiological effects known from normative classical conditioning. The second experiment will investigate confidence-based conditioning in an instrumental learning task, in which participants receive or do not receive external feedback. Using a novel confidence-based reinforcement learning model, we will test the hypothesis that confidence affects value representations by means of neurocomputational mechanisms known from normative instrumental conditioning.In a second work package we will investigate interindividual differences in confidence-based learning in a large general population sample recruited from the online marketplace Amazon Mechanical Turk. Adhering to a continuum view of psychiatric disease and using model-based mechanistic markers of confidence-based learning, we will test the hypothesis of a link between reduced confidence-based learning and proneness to anhedonia.Overall, the proposed project is expected to provide major insights into the fundamental behavioural and neurobiological nature of confidence-based learning and its significance for a key psychiatric symptom dimension, anhedonia.

• DFG Programme: Research Grants
• Co-Investigator: Professor Dr. Philipp Sterzer, Ph.D.
• Ehemaliger Antragsteller: Dr. Matthias Guggenmos, until 2/2022