Mapping anhedonia onto reinforcement learning: a behavioural meta-analysis

Biology of Mood & Anxiety Disorders

Quentin JM Huys⁴  Diego A Pizzagalli²  Ryan Bogdan¹  Peter Dayan³ 

[1] Department of Psychology and Neuroscience, Duke University, NC, USA

[2] Department of Psychiatry, Harvard Medical School, MA, USA

[3] Gatsby Computational Neuroscience Unit, UCL, London, UK

[4] Department of Psychiatry, Psychotherapy and Psychosomatics, University Hospital of Psychiatry, Zurich, Switzerland

关键词: Learning rate;    Reward sensitivity;    Meta-analysis;    Computational;    Prediction error;    Reward learning;    Reinforcement learning;    Depression;    Major depressive disorder;    Anhedonia;   

Others  :  791930
DOI  :  10.1186/2045-5380-3-12

 received in 2012-04-17, accepted in 2013-05-09,  发布年份 2013

【 摘 要 】

Background

Depression is characterised partly by blunted reactions to reward. However, tasks probing this deficiency have not distinguished insensitivity to reward from insensitivity to the prediction errors for reward that determine learning and are putatively reported by the phasic activity of dopamine neurons. We attempted to disentangle these factors with respect to anhedonia in the context of stress, Major Depressive Disorder (MDD), Bipolar Disorder (BPD) and a dopaminergic challenge.

Methods

Six behavioural datasets involving 392 experimental sessions were subjected to a model-based, Bayesian meta-analysis. Participants across all six studies performed a probabilistic reward task that used an asymmetric reinforcement schedule to assess reward learning. Healthy controls were tested under baseline conditions, stress or after receiving the dopamine D₂ agonist pramipexole. In addition, participants with current or past MDD or BPD were evaluated. Reinforcement learning models isolated the contributions of variation in reward sensitivity and learning rate.

Results

MDD and anhedonia reduced reward sensitivity more than they affected the learning rate, while a low dose of the dopamine D₂ agonist pramipexole showed the opposite pattern. Stress led to a pattern consistent with a mixed effect on reward sensitivity and learning rate.

Conclusion

Reward-related learning reflected at least two partially separable contributions. The first related to phasic prediction error signalling, and was preferentially modulated by a low dose of the dopamine agonist pramipexole. The second related directly to reward sensitivity, and was preferentially reduced in MDD and anhedonia. Stress altered both components. Collectively, these findings highlight the contribution of model-based reinforcement learning meta-analysis for dissecting anhedonic behavior.

【 授权许可】

   
2013 Huys et al.; licensee BioMed Central Ltd.

【 图 表 】

Figure 1.

Figure 2.

Figure 3.

Figure 4.

【 参考文献 】

• [1]American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders. American Psychiatric Association Press; 1994.
• [2]World Health Organization: International Classification of Diseases. World Health Organization Press; 1990.
• [3]Myin-Germeys I, Peeters F, Havermans R, Nicolson NA, DeVries MW, Delespaul P, Os JV: Emotional reactivity to daily life stress in psychosis and affective disorder: an experience sampling study. Acta Psychiatr Scand 2003, 107(2):124-131.
• [4]Costello CG: Depression: Loss of reinforcers or loss of reinforcer effectiveness? Behav Ther 1972, 3:240-247.
• [5]Akiskal HS, McKinney WT: Depressive disorders: toward a unified hypothesis. Science 1973, 182(107):20-29.
• [6]Blaney PH: Contemporary theories of depression: critique and comparison. J Abnorm Psychol 1977, 86(3):203-223.
• [7]Nelson RE, Craighead WE: Selective recall of positive and negative feedback, self-control behaviors, and depression. J Abnorm Psychol 1977, 86(4):379-388.
• [8]Henriques JB, Glowacki JM, Davidson RJ: Reward fails to alter response bias in depression. J Abnorm Psychol 1994, 103(3):460-466.
• [9]Henriques JB, Davidson RJ: Decreased Responsiveness to reward in depression. Cogn Emotion 2000, 14(5):711-724.
• [10]Pizzagalli DA, Jahn AL, O’Shea JP: Toward an objective characterization of an anhedonic phenotype: a signal-detection approach. Biol Psychiatry 2005, 57(4):319-327. [http://dx.doi.org/10.1016/j.biopsych.2004.11.026 webcite]
• [11]Steele JD, Kumar P, Ebmeier KP: Blunted response to feedback information in depressive illness. Brain 2007, 130(Pt 9):2367-2374. [http://dx.doi.org/10.1093/brain/awm150 webcite]
• [12]Huys QJM: Reinforcers and control. Towards a computational ætiology of depression. PhD thesis. Gatsby Computational Neuroscience Unit, UCL, University of London 2007, [http://www.gatsby.ucl.ac.uk/qhuys/pub.html webcite]
• [13]Huys QJM, Vogelstein J, Dayan P, Bottou L: Psychiatry: Insights into depression through normative decision-making models. In Advances in Neural Information Processing Systems 21. Edited by Schuurmans D, Koller D, Bengio Y. MIT Press; 2009:729-736.
• [14]Chase HW, Frank MJ, Michael A, Bullmore ET, Sahakian BJ, Robbins TW: Approach and avoidance learning in patients with major depression and healthy controls: relation to anhedonia. Psychol Med 2010, 40(3):433-440. [http://dx.doi.org/10.1017/S0033291709990468 webcite]
• [15]Chase HW, Michael A, Bullmore ET, Sahakian BJ, Robbins TW: Paradoxical enhancement of choice reaction time performance in patients with major depression. J Psychopharmacol 2010, 24(4):471-479. [http://dx.doi.org/10.1177/0269881109104883 webcite]
• [16]Beck A, Steer R, Brown G: Manual for the Beck Depression Inventory-II. San Antonio: Psychological Corporation; 1996.
• [17]Watson D, Weber K, Assenheimer JS, Clark LA, Strauss ME, McCormick RA: Testing a tripartite model: I. Evaluating the convergent and discriminant validity of anxiety and depression symptom scales. J Abnorm Psychol 1995, 104:3-144.
• [18]Treadway MT, Zald DH: Reconsidering anhedonia in depression: lessons from translational neuroscience. Neurosci Biobehav Rev 2011, 35(3):537-555. [http://dx.doi.org/10.1016/j.neubiorev.2010.06.006 webcite]
• [19]Papp M, Klimek V, Willner P: Parallel changes in dopamine D2 receptor binding in limbic forebrain associated with chronic mild stress-induced anhedonia and its reversal by imipramine. Psychopharmacology 1994, 115(4):441-446.
• [20]Ichikawa J, Meltzer HY: Effect of antidepressants on striatal and accumbens extracellular dopamine levels. Eur J Pharmacol 1995, 281(3):255-261.
• [21]D’Aquila PS, Collu M, Gessa GL, Serra G: The role of dopamine in the mechanism of action of antidepressant drugs. Eur J Pharmacol 2000, 405(1-3):365-373.
• [22]Barr AM, Markou A, Phillips AG: A ‘crash’ course on psychostimulant withdrawal as a model of depression. Trends Pharmacol Sci 2002, 23(10):475-482.
• [23]Montague PR, Dayan P, Sejnowski TJ: A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 1996, 16(5):1936-1947.
• [24]O’Doherty JP, Dayan P, Friston K, Critchley H, Dolan RJ: Temporal difference models and reward-related learning in the human brain. Neuron 2003, 38(2):329-337.
• [25]Bayer HM, Glimcher PW: Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 2005, 47:129-141. [http://dx.doi.org/10.1016/j.neuron.2005.05.020 webcite]
• [26]Waelti P, Dickinson A, Schultz W: Dopamine responses comply with basic assumptions of formal learning theory. Nature 2001, 412(6842):43-48. [http://dx.doi.org/10.1038/35083500 webcite]
• [27]D’Ardenne K, McClure SM, Nystrom LE, Cohen JD: BOLD responses reflecting dopaminergic signals in the human ventral tegmental area. Science 2008, 319(5867):1264-1267. [http://dx.doi.org/10.1126/science.1150605 webcite]
• [28]Kapur S, Mann JJ: Role of the dopaminergic system in depression. Biol Psychiatry 1992, 32:1-17. [http://dx.doi.org/10.1016/0006-3223(92)90137 webcite-O]
• [29]Nutt DJ: The role of dopamine and norepinephrine in depression and antidepressant treatment. J Clin Psychiatry 2006, 67(Suppl 6):3-8.
• [30]Nestler EJ, Carlezon WA: The mesolimbic dopamine reward circuit in depression. Biol Psychiatry 2006, 59(12):1151-1159. [http://dx.doi.org/10.1016/j.biopsych.2005.09.018 webcite]
• [31]Dunlop BW, Nemeroff CB: The role of dopamine in the pathophysiology of depression. Arch Gen Psychiatry 2007, 64(3):327-337. [http://dx.doi.org/10.1001/archpsyc.64.3.327 webcite]
• [32]Parker G: Defining melancholia: the primacy of psychomotor disturbance. Acta Psychiatr Scand 2007, 115 (Suppl 433)(2):21-30.
• [33]Mathew SJ, Manji HK, Charney DS: Novel drugs and therapeutic targets for severe mood disorders. Neuropsychopharmacology 2008, 33(9):2080-2092. [http://dx.doi.org/10.1038/sj.npp.1301652 webcite]
• [34]Schlaepfer TE, Cohen MX, Frick C, Kosel M, Brodesser D, Axmacher N, Joe AY, Kreft M, Lenartz D, Sturm V: Deep brain stimulation to reward circuitry alleviates anhedonia in refractory major depression. Neuropsychopharmacology 2008, 33(2):368-377. [http://dx.doi.org/10.1038/sj.npp.1301408 webcite]
• [35]Bewernick BH, Hurlemann R, Matusch A, Kayser S, Grubert C, Hadrysiewicz B, Axmacher N, Lemke M, Cooper-Mahkorn D, Cohen MX, Brockmann H, Lenartz D, Sturm V, Schlaepfer TE: Nucleus accumbens deep brain stimulation decreases ratings of depression and anxiety in treatment-resistant depression. Biol Psychiatry 2010, 67(2):110-116. [http://dx.doi.org/10.1016/j.biopsych.2009.09.013 webcite]
• [36]Rescorla R, Wagner A: A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In Classical Conditioning II: Current Research and Theory. Edited by Black AH, Prokasy WF. Appleton-Century-Crofts; 1972:64-99.
• [37]Widrow B, Hoff M: Adaptive switching circuits. In WESCON Convention Report, Volume 4. Institute of, Radio Engineers; 1960:96-104.
• [38]Sutton R, Barto A, et al.: Toward a modern theory of adaptive networks Expectation and prediction. Psychol Rev 1981, 88(2):135-170.
• [39]Sutton RS, Barto AG: Reinforcement Learning: An Introduction. Cambridge: MIT Press; 1998. [http://www.cs.ualberta.ca/ webcite~sutton/book/the-book.html]
• [40]Berridge KC, Robinson TE: What is the role of dopamine in reward hedonic impact, reward learning, or incentive salience? Brain Res Rev 1998, 28(3):209-269.
• [41]Smith A, Li M, Becker S, Kapur S: A model of antipsychotic action in conditioned avoidance: a computational approach. Neuropsychopharm 2004, 29(6):1040-1049.
• [42]Daw N: Decision Making, Affect, and Learning: Attention and Performance XXIII. Edited by Delgado MR, Phelps EA, Robbins TW. OUP; 2009.
• [43]Kass R, Raftery A: Bayes factors. J Am Stat Assoc 1995, 90(430):773-795.
• [44]Gelman A, Carlin J, Stern H, Rubin D: Bayesian Data Analysis. Chapman and Hall/CRC Press; 2004.
• [45]Stephan KE, Penny WD, Daunizeau J, Moran RJ, Friston KJ: Bayesian model selection for group studies. Neuroimage 2009, 46(4):1004-1017. [http://dx.doi.org/10.1016/j.neuroimage.2009.03.025 webcite]
• [46]Huys QJM, Moutoussis M, Williams J: Are computational models of any use to psychiatry? Neural Netw 2011, 24(6):544-551. [http://dx.doi.org/10.1016/j.neunet.2011.03.001 webcite]
• [47]Bogdan R, Santesso DL, Fagerness J, Perlis RH, Pizzagalli DA: Corticotropin-releasing hormone receptor type 1 (CRHR1) genetic variation and stress interact to influence reward learning. J Neurosci 1324, 31(37):6-13254. [http://dx.doi.org/10.1523/JNEUROSCI.2661-11.2011 webcite]
• [48]Pizzagalli DA, Iosifescu D, Hallett LA, Ratner KG, Fava M: Reduced hedonic capacity in major depressive disorder: evidence from a probabilistic reward task. J Psychiatr Res 2008, 43:76-87. [http://dx.doi.org/10.1016/j.jpsychires.2008.03.001 webcite]
• [49]Dutra S, Brooks N, Lempert K, Guardado A, Goetz E, Pizzagalli D: Reward responsiveness in a remitted depressed sample: Effects of gender and trait negative affect. In 23rd Annual Meeting of the Society for Research in Psychopathology. Minneapolis: ; 2009.
• [50]Pizzagalli DA, Goetz E, Ostacher M, Iosifescu DV, Perlis RH: Euthymic patients with bipolar disorder show decreased reward learning in a probabilistic reward task. Biol Psychiatry 2008, 64(2):162-168. [http://dx.doi.org/10.1016/j.biopsych.2007.12.001 webcite]
• [51]Young RC, Biggs JT, Ziegler VE, Meyer DA: A rating scale for mania reliability, validity and sensitivity. Br J Psychiatry 1978, 133:429-435.
• [52]Hamilton M: A rating scale for depression. J Neurol Neurosurg Psychiatry 1960, 23:56-62.
• [53]Pizzagalli DA, Evins AE, Schetter EC, Frank MJ, Pajtas PE, Santesso DL, Culhane M: Single dose of a dopamine agonist impairs reinforcement learning in humans: behavioral evidence from a laboratory-based measure of reward responsiveness. Psychopharmacol (Berl) 2008, 196(2):221-232. [http://dx.doi.org/10.1007/s00213-007-0957-y webcite]
• [54]Bogdan R, Pizzagalli DA: Acute stress reduces reward responsiveness: implications for depression. Biol Psychiatry 2006, 60(10):1147-1154. [http://dx.doi.org/10.1016/j.biopsych.2006.03.037 webcite]
• [55]Tripp G, Alsop B: Sensitivity to reward frequency in boys with attention deficit hyperactivity disorder. J Clin Child Psychol 1999, 28(3):366-375. [http://dx.doi.org/10.1207/S15374424jccp280309 webcite]
• [56]Santesso DL, Evins AE, Frank MJ, Schetter EC, Bogdan R, Pizzagalli DA: Single dose of a dopamine agonist impairs reinforcement learning in humans: evidence from event-related potentials and computational modeling of striatal-cortical function. Hum Brain Mapp 2009, 30(7):1963-1976. [http://dx.doi.org/10.1002/hbm.20642 webcite]
• [57]Green DM, Swets JA, et al.: Signal Detection Theory and Psychophysics, Volume 1974. New York: Wiley; 1966.
• [58]Huys QJM, Cools R, Gölzer M, Friedel E, Heinz A, Dolan RJ, Dayan P: Disentangling the roles of approach, activation and valence in instrumental and pavlovian responding. PLoS Comput Biol 2011, 7(4):e1002028. [http://dx.doi.org/10.1371/journal.pcbi.1002028 webcite]
• [59]MacKay DJ: Information Theory, Inference and Learning Algorithms. Cambridge: CUP; 2003.
• [60]Huys QJM, Eshel N, O’Nions E, Sheridan L, Dayan P, Roiser JP: Bonsai trees in your head: how the Pavlovian system sculpts goal-directed choices by pruning decision trees. PLoS Comput Biol 2012, 8(3):e1002410. [http://dx.doi.org/10.1371/journal.pcbi.1002410 webcite]
• [61]Berlin I, Givry-Steiner L, Lecrubier Y, Puech AJ: Measures of anhedonia and hedonic responses to sucrose in depressive and schizophrenic patients in comparison with healthy subjects. Eur Psychiatry 1998, 13(6):303-309. [http://dx.doi.org/10.1016/S0924-9338(98)80048 webcite-5]
• [62]Dichter GS, Smoski MJ, Kampov-Polevoy AB, Gallop R, Garbutt JC: Unipolar depression does not moderate responses to the Sweet Taste Test. Depress Anxiety 2010, 27(9):859-863. [http://dx.doi.org/10.1002/da.20690 webcite]
• [63]Willner P, Towell A, Sampson D, Sophokleous S, Muscat R: Reduction of sucrose preference by chronic unpredictable mild stress, and its restoration by a tricyclic antidepressant. Psychopharmacology 1987, 93(3):358-364. [http://dx.doi.org/10.1007/BF00187257 webcite]
• [64]Bylsma LM, Morris BH, Rottenberg J: A meta-analysis of emotional reactivity in major depressive disorder. Clin Psychol Rev 2008, 28(4):676-691. [http://dx.doi.org/10.1016/j.cpr.2007.10.001 webcite]
• [65]Smith KS, Berridge KC: The ventral pallidum and hedonic reward neurochemical maps of sucrose “liking” and food intake. J Neurosci 2005, 25(38):8637-8649.
• [66]Pecina S, Berridge KC: Hedonic hot spot in nucleus accumbens shell where do mu-opioids cause increased hedonic impact of sweetness? J Neurosci 1177, 25(50):7-11786.
• [67]Berridge KC: ‘Liking’ and ‘wanting’ food rewards: brain substrates and roles in eating disorders. Physiol Behav 2009, 97(5):537-550. [http://dx.doi.org/10.1016/j.physbeh.2009.02.044 webcite]
• [68]Kennedy SE, Koeppe RA, Young EA, Zubieta JK: Dysregulation of endogenous opioid emotion regulation circuitry in major depression in women. Arch Gen Psychiatry 2006, 63(11):1199-1208. [http://dx.doi.org/10.1001/archpsyc.63.11.1199 webcite]
• [69]Garriock HA, Tanowitz M, Kraft JB, Dang VC, Peters EJ, Jenkins GD, Reinalda MS, McGrath PJ, von Zastrow M, Slager SL, Hamilton SP: Association of mu-opioid receptor variants and response to citalopram treatment in major depressive disorder. Am J Psychiatry 2010, 167(5):565-573. [http://dx.doi.org/10.1176/appi.ajp.2009.08081167 webcite]
• [70]Grant BF: Comorbidity between DSM-IV drug use disorders and major depression: results of a national survey of adults. J Subst Abuse 1995, 7(4):481-497.
• [71]Swendsen J, Conway KP, Degenhardt L, Glantz M, Jin R, Merikangas KR, Sampson N, Kessler RC: Mental disorders as risk factors for substance use, abuse and dependence: results from the 10-year follow-up of the National Comorbidity Survey. Addiction 2010, 105(6):1117-1128. [http://dx.doi.org/10.1111/j.1360-0443.2010.02902.x webcite]
• [72]Samuels ER, Hou RH, Langley RW, Szabadi E, Bradshaw CM: Comparison of pramipexole and amisulpride on alertness, autonomic and endocrine functions in healthy volunteers. Psychopharmacology (Berl) 2006, 187(4):498-510. [http://dx.doi.org/10.1007/s00213-006-0443-y webcite]
• [73]Samuels ER, Hou RH, Langley RW, Szabadi E, Bradshaw CM: Comparison of pramipexole and modafinil on arousal, autonomic, and endocrine functions in healthy volunteers. J Psychopharmacol 2006, 20(6):756-770. [http://dx.doi.org/10.1177/0269881106060770 webcite]
• [74]Samuels ER, Hou RH, Langley RW, Szabadi E, Bradshaw CM: Comparison of pramipexole with and without domperidone co-administration on alertness, autonomic, and endocrine functions in healthy volunteers. Br J Clin Pharmacol 2007, 64(5):591-602. [http://dx.doi.org/10.1111/j.1365-2125.2007.02938.x webcite]
• [75]Frank MJ, O’Reilly RC: A mechanistic account of striatal dopamine function in human cognition: psychopharmacological studies with cabergoline and haloperidol. Behav Neurosci 2006, 120(3):497-517. [http://dx.doi.org/10.1037/0735-7044.120.3.497 webcite]
• [76]Grace AA: Phasic versus tonic dopamine release and the modulation of dopamine system responsivity: a hypothesis for the etiology of schizophrenia. Neuroscience 1991, 41:1-24.
• [77]Cooper J, Bloom F, Roth R: The Biochemical Basis of Neuropharmacology. USA: Oxford University Press; 2003.
• [78]Baudry M, Martres MP, Schwartz JC: In vivo binding of 3H-pimozide in mouse striatum: effects of dopamine agonists and antagonists. Life Sci 1977, 21(8):1163-1170.
• [79]Fuller RW, Clemens JA, Hynes MD: Degree of selectivity of pergolide as an agonist at presynaptic versus postsynaptic dopamine receptors implications for prevention or treatment of tardive dyskinesia. J Clin Psychopharmacol 1982, 2(6):371-375.
• [80]Tissari AH, Rossetti ZL, Meloni M, Frau MI, Gessa GL: Autoreceptors mediate the inhibition of dopamine synthesis by bromocriptine and lisuride in rats. Eur J Pharmacol 1983, 91(4):463-468.
• [81]Sumners C, de Vries JB, Horn AS: Behavioural and neurochemical studies on apomorphine-induced hypomotility in mice. Neuropharmacology 1981, 20(12A):1203-1208.
• [82]Schmitz Y, Benoit-Marand M, Gonon F, Sulzer D: Presynaptic regulation of dopaminergic neurotransmission. J Neurochem 2003, 87(2):273-289.
• [83]Piercey MF, Hoffmann WE, Smith MW, Hyslop DK: Inhibition of dopamine neuron firing by pramipexole, a dopamine D3 receptor-preferring agonist comparison to other dopamine receptor agonists. Eur J Pharmacol 1996, 312:35-44.
• [84]Chowdhury R, Guitart-Masip M, Dayan P, Huys QJM, Düzel E, Dolan RJ, Lambert: Dopamine restores reward prediction errors in older age. Nat Neurosci 2013. in press
• [85]Tremblay LK, Naranjo CA, Cardenas L, Herrmann N, Busto UE: Probing brain reward system function in major depressive disorder: altered response to dextroamphetamine. Arch Gen Psych 2002, 59(5):209-215.
• [86]Berton O, Nestler EJ: New approaches to antidepressant drug discovery: beyond monoamines. Nat Rev Neurosci 2006, 7(2):137-151. [http://dx.doi.org/10.1038/nrn1846 webcite]
• [87]Krishnan V, Han MH, Graham DL, Berton O, Renthal W, Russo SJ, Laplant Q, Graham A, Lutter M, Lagace DC, Ghose S, Reister R, Tannous P, Green TA, Neve RL, Chakravarty S, Kumar A, Eisch AJ, Self DW, Lee FS, Tamminga CA, Cooper DC, Gershenfeld HK, Nestler EJ: Molecular adaptations underlying susceptibility and resistance to social defeat in brain reward regions. Cell 2007, 131(2):391-404. [http://dx.doi.org/10.1016/j.cell.2007.09.018 webcite]
• [88]Vialou V, Robison AJ, Laplant QC, Covington HE, Dietz DM, Ohnishi YN, Mouzon E, Rush AJ, Watts EL, Wallace DL, Iñiguez SD, Ohnishi YH, Steiner MA, Warren BL, Krishnan V, BolaÃśos CA, Neve RL, Ghose S, Berton O, Tamminga CA, Nestler EJ: DeltaFosB in brain reward circuits mediates resilience to stress and antidepressant responses. Nat Neurosci 2010, 13(6):745-752. [http://dx.doi.org/10.1038/nn.2551 webcite]
• [89]Pitchot W, Hansenne M, Ansseau M: Role of dopamine in non-depressed patients with a history of suicide attempts. Eur Psychiatry 2001, 16(7):424-427.
• [90]Pitchot W, Reggers J, Pinto E, Hansenne M, Fuchs S, Pirard S, Ansseau M: Reduced dopaminergic activity in depressed suicides. Psychoneuroendocrinology 2001, 26(3):331-335.
• [91]Tandberg E, Larsen JP, Aarsland D, Cummings JL: The occurrence of depression in Parkinson’s disease. A community-based study. Arch Neurol 1996, 53(2):175-179.
• [92]Gelder M, Harrison P, Cowen P: Shorter Oxford Textbook of Psychiatry. Oxford: Oxford University Press; 2006.
• [93]Floresco SB, West AR, Ash B, Moore H, Grace AA: Afferent modulation of dopamine neuron firing differentially regulates tonic and phasic dopamine transmission. Nat Neurosci 2003, 6(9):968-973. [http://dx.doi.org/10.1038/nn1103 webcite]
• [94]Goto Y, Grace AA: Dopaminergic modulation of limbic and cortical drive of nucleus accumbens in goal-directed behavior. Nat Neurosci 2005, 8(6):805-812. [http://dx.doi.org/10.1038/nn1471 webcite]
• [95]Salamone JD, Correa M, Farrar AM, Nunes EJ, Pardo M: Dopamine behavioral economics, and effort. Front Behav Neurosci 2009., 3(13) [http://dx.doi.org/10.3389/neuro.08.013.2009 webcite]
• [96]Niv Y, Daw ND, Joel D, Dayan P: Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacol(Berl) 2007, 191(3):507-520. [http://dx.doi.org/10.1007/s00213-006-0502-4 webcite]
• [97]Gard D, Gard M, Kring A, John O: Anticipatory and consummatory components of the experience of pleasure: a scale development study. J Res Pers 2006, 40(6):1086-1102.
• [98]Satoh T, Nakai S, Sato T, Kimura M: Correlated coding of motivation and outcome of decision by dopamine neurons. J Neurosci 2003, 23(30):9913-9923.
• [99]Guitart-Masip M, Fuentemilla L, Bach DR, Huys QJM, Dayan P, Dolan RJ, Duzel E: Action dominates valence in anticipatory representations in the human striatum and dopaminergic midbrain. J Neurosci 2011, 31(21):7867-7875. [http://dx.doi.org/10.1523/JNEUROSCI.6376-10.2011 webcite]
• [100]Parkinson JA, Olmstead MC, Burns LH, Robbins TW, Everitt BJ: Dissociation in effects of lesions of the nucleus accumbens core and shell on appetitive pavlovian approach behavior and the potentiation of conditioned reinforcement and locomotor activity by D-amphetamine. J Neurosci 1999, 19(6):2401-2411.
• [101]Dickinson A, Smith J, Mirenowicz J: Dissociation of Pavlovian and instrumental incentive learning under dopamine antagonists. Behav Neurosci 2000, 114(3):468-483.
• [102]Parkinson JA, Dalley JW, Cardinal RN, Bamford A, Fehnert B, Lachenal G, Rudarakanchana N, Halkerston KM, Robbins TW, Everitt BJ: Nucleus accumbens dopamine depletion impairs both acquisition and performance of appetitive Pavlovian approach behaviour: implications for mesoaccumbens dopamine function. Behav Brain Res 2002, 137(1-2):149-163.
• [103]Di Ciano P, Cardinal RN, Cowell RA, Little SJ, Everitt BJ: Differential involvement of NMDA, AMPA/kainate, and dopamine receptors in the nucleus accumbens core in the acquisition and performance of pavlovian approach behavior. J Neurosci 2001, 21(23):9471-9477.
• [104]Murschall A, Hauber W: Inactivation of the ventral tegmental area abolished the general excitatory influence of Pavlovian cues on instrumental performance. Learn Mem 2006, 13(2):123-126. [http://dx.doi.org/10.1101/lm.127106 webcite]
• [105]Lex B, Hauber W: The role of nucleus accumbens dopamine in outcome encoding in instrumental and Pavlovian conditioning. Neurobiol Learn Mem 2010, 93(2):283-290. [http://dx.doi.org/10.1016/j.nlm.2009.11.002 webcite]
• [106]Sasaki-Adams DM, Kelley AE: Serotonin-Dopamine interactions in the control of conditioned reinforcement and motor behaviour. Neuropsychopharm 2001, 25(3):440-452.
• [107]Iiguez SD, Warren BL, Bolaos-Guzmn CA: Short- and long-term functional consequences of fluoxetine exposure during adolescence in male rats. Biol Psychiatry 2010, 67(11):1057-1066. [http://dx.doi.org/10.1016/j.biopsych.2009.12.033 webcite]
• [108]Flagel SB, Clark JJ, Robinson TE, Mayo L, Czuj A, Willuhn I, Akers CA, Clinton SM, Phillips PEM, Akil H: A selective role for dopamine in stimulus-reward learning. Nature 2011, 469(7328):53-57. [http://dx.doi.org/10.1038/nature09588 webcite]
• [109]Elliott R, Sahakian BJ, Michael A, Paykel ES, Dolan RJ: Abnormal neural response to feedback on planning and guessing tasks in patients with unipolar depression. Psychol Med 1998, 28(3):559-71.
• [110]Steele JD, Meyer M, Ebmeier KP: Neural predictive error signal correlates with depressive illness severity in a game paradigm. Neuroimage 2004, 23:269-2680.
• [111]Knutson B, Bhanji JP, Cooney RE, Atlas LY, Gotlib IH: Neural responses to monetary incentives in major depression. Biol Psychiatry 2008, 63(7):686-692. [http://dx.doi.org/10.1016/j.biopsych.2007.07.023 webcite]
• [112]Kumar P, Waiter G, Ahearn T, Milders M, Reid I, Steele JD: Abnormal temporal difference reward-learning signals in major depression. Brain 2008, 131(Pt 8):2084-2093. [http://dx.doi.org/10.1093/brain/awn136 webcite]
• [113]Pizzagalli DA, Holmes AJ, Dillon DG, Goetz EL, Birk JL, Bogdan R, Dougherty DD, Iosifescu DV, Rauch SL, Fava M: Reduced caudate and nucleus accumbens response to rewards in unmedicated individuals with major depressive disorder. Am J Psychiatry 2009, 166(6):702-710. [http://dx.doi.org/10.1176/appi.ajp.2008.08081201 webcite]
• [114]Smoski MJ, Felder J, Bizzell J, Green SR, Ernst M, Lynch TR, Dichter GS: fMRI of alterations in reward selection, anticipation, and feedback in major depressive disorder. J Affect Disord 2009, 118(1-3):69-78. [http://dx.doi.org/10.1016/j.jad.2009.01.034 webcite]
• [115]Dempster A, Laird N, Rubin D: Maximum likelihood from incomplete data via the EM algorithm. J Royal Stat Soc Series B (Methodological) 1977, 39:1-38.
• [116]Sahani M, Linden JF: How linear are auditory cortical responses? In Advances in Neural Information Processing Systems, Volume 15. Edited by Becker S, Thrun S, Obermayer K. Cambridge: MIT Press; 2003:109-116.
• [117]Dickinson A, Balleine B: The role of learning in the operation of motivational systems. In Stevens’ Handbook of Experimental Psychology, Volume 3. Edited by Gallistel R. New York: Wiley; 2002:497-534.
• [118]Daw ND, Niv Y, Dayan P: Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat Neurosci 2005, 8(12):1704-1711. [http://dx.doi.org/10.1038/nn1560 webcite]
• [119]Sutton R: Integrated architectures for learning, planning, and reacting based on approximating dynamic programming. In Proceedings of the Seventh International Conference on Machine Learning, Volume 216. ; 1990:224-224.
• [120]Dayan P, Kakade S, Montague PR: Learning and selective attention. Nat Neurosci 2000, 3 Suppl:1218-1223. [http://dx.doi.org/10.1038/81504 webcite]
• [121]Behrens TEJ, Woolrich MW, Walton ME, Rushworth MFS: Learning the value of information in an uncertain world. Nat Neurosci 2007, 10(9):1214-1221. [http://dx.doi.org/10.1038/nn1954 webcite]
• [122]Baxter MG, Chiba AA: Cognitive functions of the basal forebrain. Curr Opin Neurobiol 1999, 9(2):178-183.
• [123]Yu AJ, Dayan P: Uncertainty, neuromodulation, and attention. Neuron 2005, 46(4):681-692. [http://dx.doi.org/10.1016/j.neuron.2005.04.026 webcite]
• [124]Hasselmo: Neuromodulation: acetylcholine and memory consolidation. Trends Cogn Sci 1999, 3(9):351-359.
• [125]Aston-Jones G, Cohen JD: Adaptive gain and the role of the locus coeruleus-norepinephrine system in optimal performance. J Comp Neurol 2005, 493:99-110. [http://dx.doi.org/10.1002/cne.20723 webcite]
• [126]Bylsma LM, Taylor-Clift A, Rottenberg J: Emotional reactivity to daily events in major and minor depression. J Abnorm Psychol 2011, 120:155-167. [http://dx.doi.org/10.1037/a0021662 webcite]
• [127]Haeffel GJ, Abramson LY, Brazy PC, Shah JY, Teachman BA, Nosek BA: Explicit and implicit cognition: a preliminary test of a dual-process theory of cognitive vulnerability to depression. Behav Res Ther 2007, 45(6):1155-1167. [http://dx.doi.org/10.1016/j.brat.2006.09.003 webcite]
• [128]Eshel N, Roiser JP: Reward and punishment processing in depression. Biol Psychiatry 2010, 68(2):118-124. [http://dx.doi.org/10.1016/j.biopsych.2010.01.027 webcite]