Confidence-based learning: establishing a novel form of learning without feedback

• Applicant: Dr. Marcus Rothkirch, since 2/2022
• Subject Area: Human Cognitive and Systems Neuroscience ; Biological Psychiatry
• Term: from 2018 to 2024
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 403630675

• Final Report Year: 2024

Final Report Abstract

Learning is a crucial feature to adapt and improve in every-changing dynamic world. Most theories of learning focus on the mechanisms of how we learn through external feedback. Yet, in many instances humans learn in the absence of external feedback. For example, when we privately practice a musical instrument, the feedback is not provided by an external teacher, but by ourselves. This self-evaluatory feedback is a mechanism which researchers refer to as metacognition, or more specifically, when such internal evaluation concerns the correctness of our actions, confidence. The key proposal of this project is that there is a fundamental parallel between external rewardbased feedback and internal confidence-based feedback. We studied this hypothesis in two fundamental forms of learning – instrumental and classical conditioning. Instrumental conditioning refers to instances of learning in which feedback changes the relative value of actions that an individual can choose from. An example would be preferring a walk over a public transport for a certain connection if one repeatedly had a bad experience with the latter. Does confidence affect the value of choice options similar to external feedback? To find out we tested participants in reward experiments in which they would first receive reward reinforcement about a set of choice options, but then entered a phase without reward reinforcement in which the only feedback available was internal (i.e. choice confidence). We found that the values in this phase were still subject to change and that these changes was best explained by a “confidence prediction error” signal – the difference between predicted confidence and actual confidence. Classical conditioning is most famously associated with Pavlov’s dog and refers to instances in which a previously neutral stimulus becomes valuable through repeated pairing with a form of reinforcement. If confidence is a form of internally generated reward, it should lead to similar reinforcement effects when systematically paired with neutral stimuli. We tested this prediction in a paradigm in which neutral sounds were paired with either high or low confidence in a perceptual decision-making task. We found that this novel reinforcement scheme resulted in behavioural and physiological conditioning effects similar to external reward-based reinforcement – including extinction effects when reinforcement was stopped. Finally, to better understand the computations of such metacognitive learning signals, this project developed the computational modelling toolbox ReMeta which allows estimating metacognitive biases and inefficiencies from confidence data.

Publications

• Sustained effects of corrupted feedback on perceptual inference. Scientific Reports, 9(1).
  Varrier, R. S.; Stuke, H.; Guggenmos, M. & Sterzer, P.
  
• A multimodal neuroimaging classifier for alcohol dependence. Scientific Reports, 10(1).
  Guggenmos, Matthias; Schmack, Katharina; Veer, Ilya M.; Lett, Tristram; Sekutowicz, Maria; Sebold, Miriam; Garbusow, Maria; Sommer, Christian; Wittchen, Hans-Ulrich; Zimmermann, Ulrich S.; Smolka, Michael N.; Walter, Henrik; Heinz, Andreas & Sterzer, Philipp
  
• No evidence for mnemonic modulation of interocularly suppressed visual input. NeuroImage, 215, 116801.
  Gayet, Surya; Guggenmos, Matthias; Christophel, Thomas B.; Haynes, John-Dylan; Paffen, Chris L.E.; Sterzer, Philipp & Van, der Stigchel Stefan
  
• Unreliable feedback deteriorates information processing in primary visual cortex. NeuroImage, 214, 116701.
  Varrier, Rekha S.; Rothkirch, Marcus; Stuke, Heiner; Guggenmos, Matthias & Sterzer, Philipp
  
• Measuring metacognitive performance: type 1 performance dependence and test-retest reliability. Neuroscience of Consciousness, 2021(1).
  Guggenmos, Matthias
  
• Reverse engineering of metacognition. eLife, 11.
  Guggenmos, Matthias
  
• The value of confidence: Confidence prediction errors drive value-based learning in the absence of external feedback. PLOS Computational Biology, 18(10), e1010580.
  Ptasczynski, Lena Esther; Steinecker, Isa; Sterzer, Philipp & Guggenmos, Matthias
  
• Cross-Modality Evidence for Reduced Choice History Biases in Psychosis-Prone Individuals. Schizophrenia Bulletin, 49(2), 397-406.
  Eckert, Anna-Lena; Gounitski, Yael; Guggenmos, Matthias & Sterzer, Philipp