Studies on the role of subcortical dopaminergic networks in decision-making

• Applicant: Professor Dr. Jan Peters
• Subject Area: Human Cognitive and Systems Neuroscience
• Term: from 2014 to 2022
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 257204944

• Final Report Year: 2023

Final Report Abstract

This project examined the role of dopamine neurotransmission in the regulation of learning and decision-making processes in humans. A particular focus was on the impact of appetitive cue exposure on human choice behavior, and on the dopaminergic regulation of reinforcement learning, exploration and temporal discounting. Further projects examined alterations in learning and decision-making in individuals suffering from disordered gambling, as well as modulatory effects of gambling environment exposure (both real and virtual) on learning and decision-making in this group. The projects leveraged hierarchical Bayesian learning and decision-making models, including drift diffusion temporal discounting and reinforcement learning models, to delineate the underlying processes. Pharmacological modulation of dopamine neurotransmission impacted upon a range of both lower-level (e.g. non-decision time) and higher-level (e.g. temporal discounting, decision threshold adjustment, directed exploration) processes, which was also confirmed in catecholamine supplementation work. To examine a causal role of the human nucleus accumbens region in temporal discounting, we longitudinally followed patients receiving deepbrain stimulation (DBS) to this structure. This revealed a disruption in preferences following DBS, highlighting a subtle but striking causal contribution of this circuit. Effects of appetitive cues on temporal discounting in healthy volunteers were much more variable than predicted based on the state of the literature at the time of project application. For example, across a number of studies, substantial physiological arousal and reward circuit activity elicited by appetitive cues was insufficient to drive reliable increases in temporal discounting. Robust effects were limited to block-wise cue exposure designs and more subtle sequential sampling model-based effects (starting point in the drift diffusion model). In contrast, gambling environment exposure effects in individuals suffering from disordered gambling revealed a large effect on temporal discounting, replicating previous work. Here we also discovered an unexpected improvement in model-based control during gambling environment exposure in gambling disorder, at odds with habit theories of addiction. Following extensive validation work on the application of decision tasks in virtual reality (VR) environments, it was then tested whether virtual gambling environment exposure elicits similar effects on model-based control and temporal discounting in gambling disorder as real exposure. This was not the case, although the predicted general group differences replicated in VR. The more general work on learning and decision-making in disordered gambling then revealed patterns that largely complemented our pharmacological work. Generally, processes that were reduced during pharmacological enhancement of dopamine transmission (e.g. directed exploration, decision thresholds) tended to be impaired or reduced in individuals suffering from disordered gambling, in line with the view of disordered gambling as a hyperdopaminergic state. Reduced model-based control during reinforcement learning in disordered gambling was replicated across two separate data sets (virtual reality gambling environment exposure and an fMRI study on reinforcement learning in gambling disorder).

Publications

• Dopamine and Risky Decision-Making in Gambling Disorder. eneuro, 7(3), ENEURO.0461-19.2020.
  Peters, Jan; Vega, Taylor; Weinstein, Dawn; Mitchell, Jennifer & Kayser, Andrew
  
• Dopaminergic Modulation of Human Intertemporal Choice: A Diffusion Model Analysis Using the D2-Receptor Antagonist Haloperidol. The Journal of Neuroscience, 40(41), 7936-7948.
  Wagner, Ben; Clos, Mareike; Sommer, Tobias & Peters, Jan
  
• Dopaminergic modulation of the exploration/exploitation trade-off in human decision-making. eLife, 9.
  Chakroun, Karima; Mathar, David; Wiehler, Antonius; Ganzer, Florian & Peters, Jan
  
• The drift diffusion model as the choice rule in inter-temporal and risky choice: A case study in medial orbitofrontal cortex lesion patients and controls. PLOS Computational Biology, 16(4), e1007615.
  Peters, Jan & D’Esposito, Mark
  
• Attenuated Directed Exploration during Reinforcement Learning in Gambling Disorder. The Journal of Neuroscience, 41(11), 2512-2522.
  Wiehler, A.; Chakroun, K. & Peters, J.
  
• Increased temporal discounting and reduced model-based control in problem gambling are not substantially modulated by exposure to virtual gambling environments
  Bruder, L.; Wagner, B.; Mathar, D. & Peters, J.
  
• Reliability assessment of temporal discounting measures in virtual reality environments. Scientific Reports, 11(1).
  Bruder, Luca R.; Scharer, Lisa & Peters, Jan
  
• Gambling Environment Exposure Increases Temporal Discounting but Improves Model-Based Control in Regular Slot-Machine Gamblers. Computational Psychiatry, 6(1), 142-165.
  Wagner, Ben; Mathar, David & Peters, Jan
  
• Motor response vigour and visual fixation patterns reflect subjective valuation during intertemporal choice. PLOS Computational Biology, 18(6), e1010096.
  Smith, Elke & Peters, Jan
  
• Parameter and Model Recovery of Reinforcement Learning Models for Restless Bandit Problems. Computational Brain & Behavior, 5(4), 547-563.
  Danwitz, Ludwig; Mathar, David; Smith, Elke; Tuzsus, Deniz & Peters, Jan
  
• The catecholamine precursor Tyrosine reduces autonomic arousal and decreases decision thresholds in reinforcement learning and temporal discounting. PLOS Computational Biology, 18(12), e1010785.
  Mathar, David; Erfanian, Abdoust Mani; Marrenbach, Tobias; Tuzsus, Deniz & Peters, Jan
  
• Trial‐wise exposure to visual emotional cues increases physiological arousal but not temporal discounting. Psychophysiology, 59(4).
  Knauth, Kilian & Peters, Jan
  
• Chronic Deep Brain Stimulation of the Human Nucleus Accumbens Region Disrupts the Stability of Intertemporal Preferences. The Journal of Neuroscience, 43(43), 7175-7185.
  Wagner, Ben J.; Schüller, Canan B.; Schüller, Thomas; Baldermann, Juan C.; Kohl, Sina; Visser-Vandewalle, Veerle; Huys, Daniel; Marx, Milena; Kuhn, Jens & Peters, Jan
  
• Dopamine regulates decision thresholds in human reinforcement learning in males. Nature Communications, 14(1).
  Chakroun, Karima; Wiehler, Antonius; Wagner, Ben; Mathar, David; Ganzer, Florian; van Eimeren, Thilo; Sommer, Tobias & Peters, Jan
  
• Erotic cue exposure increases neural reward responses without modulating temporal discounting. Imaging Neuroscience, 1, 1-25.
  Knauth, Kilian; Mathar, David; Kuzmanovic, Bojana; Tittgemeyer, Marc & Peters, Jan
  
• Erotic cue exposure increases physiological arousal, biases choices toward immediate rewards, and attenuates model‐based reinforcement learning. Psychophysiology, 60(12).
  Mathar, David; Wiebe, Annika; Tuzsus, Deniz; Knauth, Kilian & Peters, Jan