Handbook of emotion regulation ( PDFDrive )

Neurobiological Model of Implicit and Explicit Emotion Regulation 81 of emotional conflict (i.e., vACC and amyg- alter the rate of habituation to nonemotional dala). Specifically, the vACC and adjacent stimuli (e.g., tones; Gruzelier & Eves, 1987), vmPFC reduce fear responses through inter- suggesting that there are both implicit and connections with the amygdala and hippo- explicit processes at play in habituation. campus during extinction learning and recall of extinction memories (Ehrlich et al., 2009; Studies have shown that the signal from Milad & Quirk, 2002; Phelps, Delgado, the amygdala is reliably reduced as partici- Nearing, & LeDoux, 2004). More success- pants repeatedly encounter a salient but emo- ful extinction is associated with decreased tionally nonrewarding or punishing stimu- conditioned stimulus (CS)-driven activa- lus (Fisher et al., 2009; Hare et al., 2008). tion in the dorsal anterior cingulate cortex The extent of habituation, however, appears (dACC) in human (Milad et al., 2009) and to be a function of the engagement of medial animal models (Livneh & Paz, 2012). This PFC circuitry that overlaps with the circuit is consistent with a recent review of the role associated with regulation of emotional of the dACC in evaluation and expression of conflict (i.e., vACC). For example, during negative emotion (Etkin, Egner, & Kalisch, habituation, engagement of the medial PFC 2011) and suggest the presence of a second is related to lower amygdala responses (Hare pathway between the dACC and the amyg- et al., 2008). A recent fMRI–PET (positron dala where activation might be actually det- emission tomography) study also showed rimental to extinction of fear memories. that higher density of serotonergic neurons in the vACC—considered an indicator of the Clinical models document impaired capacity to modulate emotional arousal—­is extinction behaviorally in posttraumatic related to better habituation to negative faces stress disorder (PTSD; Norrholm et al., and lower amygdala responses (Fisher et al., 2011; Wessa & Flor, 2007; for a review, see 2009). Failure of habituation of the amyg- Lissek et al., 2005). Behavioral symptoms dala has also been documented in severely in GAD and specific anxiety disorders also inhibited individuals (Blackford, Allen, indicate an inability to inhibit dysfunctional Cowan, & Avery, 2013) and in high trait fear responses. In addition, the presence of anxiety (Hare et al., 2008) as well as PTSD safety and avoidance behaviors in these dis- (Shin et al., 2005). Furthermore, impaired orders might function to prevent the uncou- habituation of auditory startle responses (a pling of fear responses (Graham & Milad, process mediated through the amygdala) 2011; Milad et al., 2009). Consistent with was found to be a marker of later develop- this, heightened amygdala activation was ment of PTSD in a longitudinal study (Pole documented in PTSD during fear extinction et al., 2009). following fear conditioning (Milad et al., 2009). However, there were no differences Anticipatory Responding in behavioral or neural patterns of extinc- to Emotional Stimuli tion as a function of social anxiety (Pejic, Anticipation of negative emotional events Hermann, Vaitl, & Stark, 2011), which prepares the individual to respond adaptively might suggest preservations of extinction in and flexibly to actual emotional challenges social anxiety. as they arise. This process has typically been studied in experimental paradigms by Habituation is the adaptive reduction of presenting cues that with some probability the initial response to emotional stimuli that predict the occurrence of a threatening or carries no immediate reward or punishment negative event (e.g., electric shock, thermal value to the individual. A typical paradigm pain, negative image) (Drabant et al., 2011; used in studying the neurobiological sub- Nitschke et al., 2009; Straube, Schmidt, strates of habituation in humans is to repeat- Weiss, Mentzel, & Miltner, 2009). Anticipa- edly present emotionally salient stimuli (e.g., tory responding fits our definition of emotion emotional faces) over several runs of the scan regulation, because it allows the individual and look at the time course of the neuronal to optimize and prepare future responding signal change from early to late trials (Hare in the face of an emotional challenge. The et al., 2008) within regions typically asso- conscious anticipatory component gives a ciated with processing the stimuli. Behav- more explicit flavor to these paradigms, but ioral manipulations of attention to different aspects of the stimulus have been shown to

82 BIOLOGICAL BASES in the absence of explicit instruction to regu- instructed to label the emotion displayed late the response, the implicit component is on a face (e.g., indicate whether the face is retained. angry or afraid), match the affect shown on the face to another face showing the same More importantly, these paradigms have emotion, or match the gender of the face proved to be useful in mapping the neu- (Hariri, Bookheimer, & Mazziotta, 2000; ral circuits associated with anticipatory Lieberman et al., 2007; Lieberman, Hariri, responses and associated anxiety. These Jarcho, Eisenberger, & Bookheimer, 2005). results show the involvement of the vACC There are important similarities between and the broader ACC with typically higher verbal labeling as done in this task and cre- activation during anticipation (Butler et ation of a verbal narrative that is supposed al., 2005; Nitschke et al., 2009; Ploghaus, to lessen emotional arousal seen in explicit Becerra, Borras, & Borsook, 2003; regulatory tasks. Because of these parallels, Ploghaus et al., 1999; Straube et al., 2009), the task fits our definition of emotion regu- and often tracking concomitant anxiety. lation, as it results in the regulation of emo- Sarinopoulos et al. (2010) reported that tional arousal in a goal-­directed fashion. anticipatory activation in the vACC follow- Semantically labeling emotional stimuli, or ing uncertain cues was related to systematic “putting feelings into words,” as these para- overestimation of the frequency with which digms are typically described, versus match- the cue was predictive of aversive images. ing the emotion on another face or labeling In terms of the relationship to anticipa- the gender has a marked explicit compo- tory responding, informative moderation nent, yet the regulation still happens in the effects have been found. Specifically, dur- absence of an explicit goal to do so. ing moderate levels of threat (mildly pain- ful electric shock), higher vACC activation At the neural level, matching emotional was related to higher anticipatory prepared- expressions on faces with words that label ness, whereas during high levels of threat affect results in lower amygdala activation, of electric shocks, higher vACC activation as compared to matching expressions to was related to lower anticipatory prepared- other expressions, matching the gender to ness (Straube et al., 2009), suggesting that gender labels, or simply viewing the face. higher vACC activation indices anticipa- Moreover, labeling the emotion of facial tory preparedness, whereas deactivation expressions or focusing on physical rather might track disengagement from the task, than emotional features of an emotionally a perhaps more explicit form of regulation. evocative scene results in increased activa- vACC activation was also documented in tion in the lateral and medial prefrontal cor- a virtual chasing game while participants tices, despite the fact that emotion regulation expected painful levels of electric shocks, in this paradigm is unintentional or implicit and this activation increased as a function (Hariri et al., 2000; Lieberman et al., 2005). of anticipated pain (Mobbs et al., 2007). Importantly, Lieberman and colleagues (2007) documented that a vmPFC activation Clinically, abnormalities in activation mediated the relationship between lateral to anticipatory cues preceding negative PFC and decreased amygdala activity, sug- anxiety-p­ rovoking images have been docu- gesting that similar to implicit emotion regu- mented in GAD (Nitschke et al., 2009). Sim- lation, it may be the medial PFC–amygdala ilarly, patients with unipolar major depres- pathway that plays a critical role in dampen- sive disorder (Abler, Erk, Herwig, & Walter, ing emotional reactivity in these tasks (see 2007) also showed higher amygdala acti- Figure 5.1). vation compared to healthy controls while anticipating negative pictures. Variants of the affect-l­abeling task have been used in pediatric anxiety and mood Incidental Regulation disorder populations with or at-risk for psy- through Language chopathology (Beesdo et al., 2009; Monk Goal-d­irected engagement of higher order et al., 2007; Taylor, Eisenberger, Saxbe, cognitive processes, such as language, in Lehman, & Lieberman, 2006). These stud- emotional contexts can have “acciden- ies show that among at-risk adolescents, tal” emotion regulatory consequences. In focusing attention on the physical features experimental paradigms, participants are of an emotional face (nose width) or label- ing one’s subjective emotions while viewing

Neurobiological Model of Implicit and Explicit Emotion Regulation 83 an emotional face, normalized concomitant having engaged in emotion regulation). neural activation in the PFC and amygdala. Often-s­tudied forms of explicit regulation Similarly, McClure and colleagues (2007) are reappraisal, in which participants are found that the performance of children instructed to attempt to change the way they with GAD did not differ from that of their think about the emotional event so that they healthy counterparts in rating physical fea- feel less negative emotions, and suppression, tures of negative emotional faces, and that in which participants are instructed not to they recruited the same amygdala, dACC, show how they feel. and vmPFC networks as their healthy ado- lescent counterparts. However, in another Neuroimaging studies show that reap- study, Taylor et al. (2006) found that chil- praisal results in a dynamic interchange dren from high-s­tress family environments between frontal lobe areas implicated in showed higher amygdala and dlPFC activa- cognitive control and executive function, tion during an affect labeling of faces task and emotion reactivity areas (for reviews, see than children from low-risk families, who Ochsner & Gross, 2005; Ochsner & Gross, showed lower amygdala activation during this volume; Kalisch, 2009). Specifically, affect labeling. Low-risk children did not imaging studies indicate that attempts to differ in dlPFC activation. Taken together, reappraise negative stimuli result in increased these result shows that disordered and at- activation in ventrolateral (vlPFC), dlPFC, risk children are capable of labeling as a dACC—areas traditionally implicated in form of regulation behaviorally, and they non-e­motional forms of cognitive control might do so largely through the normative (Kalisch, 2009). This frontal lobe activa- circuitry. tion in turn is related to reduced activation in emotional reactivity-r­elated areas (amyg- Instructed Regulation dala and insula), suggesting that engage- of Emotional Responses ment of control-r­elated circuitry dampens A paradigmatic case of explicit regulation reactivity in these critical emotion reactiv- may be found in work by James Gross and ity regions (Banks, Eddy, Angstadt, Nathan, colleagues (e.g., Gross & Levenson, 1993; & Phan, 2007; Goldin, McRae, Ramel, & McRae et al., 2010; Ochsner, Bunge, Gross, Gross, 2008; Ochsner et al., 2002; Wager, & Gabrieli, 2002). As reviewed by Ochsner Davidson, Hughes, Lindquist, & Ochsner, and Gross (this volume), in a typical test of 2008). Furthermore, there appears to be a explicit emotion regulation, participants are temporal specificity in the engagement of presented with a task that involves process- frontal lobe regions as a function of imple- ing stimuli under two different conditions—­ mentation stage (Paret et al., 2011). Specifi- one in which participants are instructed to cally, Kalisch and colleagues (Kalisch, 2009; react naturally (reactivity trial), and another Paret et al., 2011) propose that instructed in which they are instructed to regulate reappraisal can be partitioned into two their emotional responses (regulation trial) temporarily distinct phases: an early phase using a strategy specified by the researcher of implementation that comprises strategy that participants have had ample opportu- selection and retrieval of reappraisal mate- nity to practice beforehand. Explicit emo- rial into working memory, and a later phase tion regulation performance is indexed of maintenance of performance-­monitoring by contrasting emotional responses in the processes. Corresponding shifts in the reactivity and regulation trials. These tasks recruitment of the lateral PFC are observed fit our definition of explicit emotion regu- as a function of these phases from posterior lation, because the process is effortful and to anterior parts of the lateral frontal cortex carried out with considerable awareness (Paret et al., 2011). and monitoring. Specifically, individuals are (1) aware of the cues that elicit emotional Assessment of instructed emotion regula- responses (i.e., images, films), (2) aware of tory abilities in clinical populations, how- the emotion itself (i.e., they report reduc- ever, has produced mixed results. There tion in accompanying feelings), and (3) appears to be agreement that behavior- aware of the effect of the regulation on their ally, patients with mood and anxiety dis- behavior (i.e., if prompted, they can report orders are not deficient in using regulation when instructed to do so in the laboratory (Ehring, Tuschen-C­ affier, Schnülle, Fischer, & Gross, 2010; Goldin, Manber, Hakimi,

84 BIOLOGICAL BASES Canli, & Gross, 2009; Goldin, Manber- regulation, highlighting and expanding our Ball, Werner, Heimberg, & Gross, 2009), prior model of an implicit–­explicit emotion but they might spontaneously use certain regulatory spectrum. We have argued sev- beneficial forms of regulation such as reap- eral key points. praisal less frequently, or report feeling less successful in regulating their emotions as First, types of emotion regulation vary in a result (Gruber, Harvey, & Gross, 2012). the degree to which they are implicit versus By contrast, a variety of neural deficits have explicit. We demonstrated that the rela- been found in different patient groups. For tive degree of implicit or explicit features example, currently depressed patients acti- depends on multiple factors, including the vated bilateral PFC during reappraisal of stimulus, task, context, individual differ- negative scenes, while healthy subjects did ences, psychopathology, and time. Com- so only unilaterally (Johnstone et al., 2007). pounding this complexity, we showed that in another study, there was a lack of dlPFC many existing experimental paradigms activation in patients with major depressive involve various admixtures of implicit and disorder altogether, but activated dACC, explicit types of emotion regulation. Greater amygdala, and insular cortices (Beauregard, attention should therefore be paid to the Paquette, & Lévesque, 2006). Patients with implications of these design features in light social anxiety disorder showed delayed but of a conceptualization of implicit regulation. overall enhanced engagement of dlPFC and Studies need to address the relative degree to dACC when instructed to reappraise nega- which regulation is at any time implicit or tive self-­relevant thoughts (Goldin, Man- explicit in both existing and new paradigms. ber-Ball, et al., 2009). By contrast, during Probing awareness about regulation is one reappraisal of angry/contemptuous faces, way to assess implicitness versus explicit- patients with social anxiety disorder failed ness. Other factors that influence whether to activate dACC and anterior medial fron- a particular regulatory task is implicit or tal cortex (neither dlPFC nor vlPFC were explicit are personality features, such as low involved in this task; Goldin, Manber-Ball, trait rumination, which might predispose et al., 2009). Patients with GAD and social individuals to have more automatic tenden- phobia both failed to activate the dACC cies to use reappraisal, and they might do so when reappraising either positive or negative in a more implicit, automatic fashion (Ray et pictures (no dlPFC or vlPFC was involved in al., 2005). Another example is pathological this task; Blair et al., 2012). Finally, trauma worry in GAD (Borkovec, Alcaine, & Behar, exposed patients with borderline personality 2004), which might parallel explicit forms disorder hypoactivated lateral and medial of regulation by virtue of engaging effort- PFC more during reappraisal of negative ful resources (e.g., prefrontal areas). This scenes, relative to trauma-­exposed healthy in turn might influence the ability of people controls. Taken together, these results with GAD engage explicit regulation in a show a rather mixed set of abnormalities in goal-d­ irected fashion. instructed emotion regulation, with differ- ences likely due to different tasks, stimuli, Additionally, several researchers have or instructions. Regardless, explicit regula- pointed out the inherent difficulties involved tion at the behavioral level does not appear in differentiating emotional regulation from to be perturbed in psychiatric disorder, even emotional reactivity, with some questioning if patients recruit different neural circuits. the utility of emotion regulation as a scien- Impairments may exist, however, in the tific construct altogether (Campos, Frankel, spontaneous and potentially more implicit & Camras, 2004; Cole, Martin, & Dennis, engagement of these effortful forms of regu- 2004). Neuroimaging studies get around lation. this problem and routinely include a con- trasting control condition when the effects Implications and Future of emotion regulation are compared to those Research Directions of canonical emotional reactivity trials (i.e., Our goal in this review was to provide a adaptation to emotional conflict is a contrast unifying conceptual framework for emotion between iI and cI conditions; reappraisal is a contrast between uninstructed “watch” trials and instructed regulation). However, studies that purportedly measure the effects of spontaneous use of effortful regulation

Neurobiological Model of Implicit and Explicit Emotion Regulation 85 often do not include a contrasting reactiv- will allow us to ground implicit and explicit ity condition and do not carefully control regulation in neurobiology and further chart for the effects of reactivity. Future research the interactive and dynamic processes by has to address emotional reactivity by sys- which the brain accomplishes regulation. tematically including a baseline unregulated condition, in addition to assessing implicit Our third point is that in striving for a and explicit features. Future studies examin- neurobiological understanding of emotion ing the explicit–i­mplicit dimension system- regulation and developing these new para- atically should also test the separability and digms, it will be important to refine the interactions between these circuits. circuitry underlying implicit and explicit regulation. The human brain is likely to Second, we showed that there appears accomplish emotion regulation through an broadly to be a medial–­lateral differentiation economical process that relies on a distrib- between implicit and explicit regulatory pro- uted network of brain areas. Three brain cesses (see Figure 5.1). That is, paradigmatic areas highlighted here as underlying implicit forms of implicit emotion regulation (e.g., and explicit forms of regulation (medial emotional conflict regulation) involve ven- PFC and dlPFC/vlPFC, respectively) have tromedial prefrontal regions in regulation, been documented to be involved in a wide while paradigmatic forms of explicit regula- array of processes besides emotion regula- tion (e.g., reappraisal) involve mainly dlPFC tion. In this vein, further research will likely and vlPFC for regulation. Other forms of broaden our understanding of the relevant regulation that contain a mixture of implicit circuitry and how these functions relate to and explicit processes (e.g., language-b­ ased the variety of implicit and explicit regula- incidental regulation) involve roles for both tory mechanisms. Moreover, most studies lateral PFC and vmPFC. While we do not focused on corticoamygdalar circuitry, but consider this medial–l­ateral gradient to be less attention has been paid to the role of the an absolute rule, neither would we be con- insula and midbrain structures essential for fident in interpreting medial or lateral acti- emotional processing (e.g., periaqueductal vation differences in patients as necessarily gray). The role of these nodes in the emotion reflecting deficits in implicit or explicit regu- regulatory circuitry with respect to implicit lation, respectively; this model provides a and explicit features remains to be explored. useful framework for describing the range of emotion regulation processes, anchored in Another important element that has thus relevant neurocircuitry. Moreover, we would far been unmapped in terms of implicit readily concede that our framework is useful and explicit regulation is the modulation for anchoring our theoretical understanding of approach tendencies and positive emo- of emotion regulation, but it is unlikely that tions (e.g., reward). There is evidence that implicit and explicit modules per se exist at approach might also be regulated in an the level of the brain. Future research might implicit and explicit fashion through some- parse the operations of implicit and explicit what dissociable circuitry (Kable & Glim- regulation into circuit-­level components cher, 2007; McClure, Laibson, Loewenstein, (e.g., working memory, interference resolu- & Cohen, 2004). For example, value-based tion) and our understanding of the implicit–­ decision making about food while focusing explicit anchors might transform as our on the health aspect of the stimuli modu- understanding of the interactions between lates activation in medial prefrontal regions participating brain areas develops. Our that overlap with the vACC region mapped goal was to illustrate the fluid boundaries to types of implicit regulation, such as emo- between implicit and explicit forms of regu- tional conflict adaptation, and this modula- lation, and move away from characterizing tion seems to be orchestrated by the dlPFC, implicit and explicit regulation in a dichoto- which is involved in more explicit forms mous fashion. New paradigms in which the of regulation (Hare, Malmaud, & Ran- level of explicitness and implicitness is para- gel, 2011). Additionally, explicit regulation metrically manipulated (e.g., in the form of of monetary rewards in a group of self-­ increased explicit regulatory effort in the assessed, successful regulators was related context of an otherwise implicit regulatory to lower striatal activation and fewer risky process or by repetitively training explicit behavioral choices (Martin & Delgado, regulation until it becomes automatized) 2011), but without medial prefrontal or dlPFC involvement. These findings in the

86 BIOLOGICAL BASES realm of positive approach related will need basic and translational affective neurosci- to be integrated into our explicit–i­mplicit ence fields. framework through systematic studies that manipulate explicitness and implicitness in References the context of positive emotions. Abler, B., Erk, S., Herwig, U., & Walter, H. Our fourth point is that better definition (2007). Anticipation of aversive stimuli acti- of implicit and explicit tasks at the process vates extended amygdala in unipolar depres- level, and a refined understanding of the sion. Journal of Psychiatric Research, 41(6), neural circuits involved, will help concret- 511–522. ize how we can best improve these functions using targeted, neurobiologically derived, Banks, S. J., Eddy, K. T., Angstadt, M., Nathan, customized, and adaptive training meth- P. J., & Phan, K. L. (2007). Amygdala–­frontal ods (e.g., Vinogradov, Fisher, & de Villers-­ connectivity during emotion regulation. Social Sidani, 2011). One question is whether tar- Cognitive and Affective Neuroscience, 2(4), geted training aimed at the dlPFC and vlPFC 303–312. will impart emotion regulatory benefits, and whether these benefits will be evident in Bargh, J. A. (1994). The four horsemen of auto- implicit or explicit regulatory tasks. maticity: Awareness, intention, efficiency, and control in social cognition. In R. S. Wyer & As our fifth point, we presented evidence T. K. Srull (Eds.), Handbook of social cogni- that difficulties in emotion regulation in tion (2nd ed., Vol. 1, pp. 1–40). Hillsdale, NJ: psychiatric disorders cut across implicit and Erlbaum. explicit types. Specifically, we found behav- ioral and neural deficits in implicit forms of Beauregard, M., Paquette, V., & Lévesque, J. regulation: emotional adaptation; habitua- (2006). Dysfunction in the neural circuitry of tion; and documented, more subtle, largely emotional self-­regulation in major depressive neural deficits in more explicit forms of reg- disorder. NeuroReport, 17(8), 843–846. ulation (affect labeling and instructed reap- praisal). These results support clinical obser- Beesdo, K., Lau, J. Y. F., Guyer, A. E., McClure- vations of pervasive emotion regulatory Tone, E. B., Monk, C. S., et al. (2009). Common symptoms in psychopathology and provide and distinct amygdala-­function perturbations neurobiological anchors for the observed in depressed vs. anxious adolescents. Archives deficits that might track with treatment. of General Psychiatry, 66(3), 275–285. Conclusions Blackford, J. U., Allen, A. H., Cowan, R. L., Strategies that individuals use to alter their & Avery, S. N. (2013). Amygdale and hip- emotions affect not only their current emo- pocampus fail to habituate to faces in indi- tional experience but also broader emo- viduals with an inhibited temperament. Social tional, cognitive, and interpersonal func- Cognitive and Affective Neuroscience, 8(2), tioning. Thus, emotion regulation is of clear 143–150. importance for both basic and applied trans- lational investigations. Implicit processes Blair, K. S., Geraci, M., Smith, B. W., Hollon, N., are hidden for measurement reasons but DeVido, J., Otero, M., et al. (2012). Reduced are now more tractable, thanks to neurosci- dorsal anterior cingulate cortical activity dur- ence methods. We have reached an exciting ing emotional regulation and top-down atten- point at which neurobiologically anchored tional control in generalized social phobia, understanding of both implicit and explicit generalized anxiety disorder, and comorbid processes is rapidly advancing. Implicit and generalized social phobia/generalized anxiety explicit processes arose to help us navigate disorder. Biological Psychiatry, 72(6), 476– our social and emotional worlds, and it is 482. important that we conduct systematic stud- ies on this domain. Our hope is that a uni- Borkovec, T. D., Alcaine, O. M., & Behar, E. fying framework will facilitate this process (2004). Avoidance theory of worry and gener- and result in research advances for both alized anxiety disorder. In R. G. Heimberg, C. L. Turk, & D. S. Mennin (Eds.), Generalized anxiety disorder: Advances in research and practice (pp. 77–108). New York: Guilford Press. Botvinick, M., Nystrom, L. E., Fissell, K., Carter, C. S., & Cohen, J. D. (1999). Conflict moni-

Neurobiological Model of Implicit and Explicit Emotion Regulation 87 toring versus selection-­for-­action in anterior cingulate activation and connectivity with the cingulate cortex. Nature, 402(6758), 179–181. amygdala during implicit regulation of emo- Butler, T., Pan, H., Epstein, J., Protopopescu, tional processing in generalized anxiety disor- X., Tuescher, O., Goldstein, M., et al. (2005). der. American Journal of Psychiatry, 167(5), Fear-r­ elated activity in subgenual anterior cin- 545–554. gulate differs between men and women. Neu- Etkin, A., & Schatzberg, A. F. (2011). Common roReport, 16(11), 1233–1236. abnormalities and disorder-s­pecific compen- Campos, J. J., Frankel, C. B., & Camras, L. sation during implicit regulation of emotional (2004). On the nature of emotion regulation. processing in generalized anxiety and major Child Development, 75(2), 377–394. depressive disorders. American Journal of Cole, P. M., Martin, S. E., & Dennis, T. A. Psychiatry, 168(9), 968–978. (2004). Emotion regulation as a scientific con- Fisher, P. M., Meltzer, C. C., Price, J. C., Cole- struct: Methodological challenges and direc- man, R. L., Ziolko, S. K., Becker, C., et al. tions for child development research. Child (2009). Medial prefrontal cortex 5-HT2A Development, 75(2), 317–333. density is correlated with amygdala reactiv- Delgado, M. R., Nearing, K. I., LeDoux, J. ity, response habituation, and functional cou- E., & Phelps, E. A. (2008). Neural circuitry pling. Cerebral Cortex, 19(11), 2499–2507. underlying the regulation of conditioned fear Frijda, N. H. (1987). The emotions. Cambridge, and its relation to extinction. Neuron, 59(5), UK: Cambridge University Press. 829–838. Goldin, P. R., Manber, T., Hakimi, S., Canli, T., Dijksterhuis, A., & Smith, P. K. (2002). Affective & Gross, J. J. (2009). Neural bases of social habituation: Subliminal exposure to extreme anxiety disorder: Emotional reactivity and stimuli decreases their extremity. Emotion, cognitive regulation during social and physical 2(3), 203–214. threat. Archives of General Psychiatry, 66(2), Drabant, E. M., Kuo, J. R., Ramel, W., Blechert, 170 –180. J., Edge, M. D., Cooper, J. R., et al. (2011). Goldin, P. R., Manber-Ball, T., Werner, K., Experiential, autonomic, and neural responses Heimberg, R., & Gross, J. J. (2009). Neural during threat anticipation vary as a function mechanisms of cognitive reappraisal of nega- of threat intensity and neuroticism. NeuroIm- tive self-b­ eliefs in social anxiety disorder. Bio- age, 55(1), 401–410. logical Psychiatry, 66(12), 1091–1099. Egner, T., Etkin, A., Gale, S., & Hirsch, J. (2008). Goldin, P. R., McRae, K., Ramel, W., & Gross, Dissociable neural systems resolve conflict J. J. (2008). The neural bases of emotion regu- from emotional versus nonemotional distract- lation: Reappraisal and suppression of nega- ers. Cerebral Cortex, 18(6), 1475–1484. tive emotion. Biological Psychiatry, 63(6), Ehring, T., Tuschen-C­affier, B., Schnülle, J., 577–586. Fischer, S., & Gross, J. J. (2010). Emotion Graham, B. M., & Milad, M. R. (2011). The regulation and vulnerability to depression: study of fear extinction: Implications for anxi- Spontaneous versus instructed use of emotion ety disorders. American Journal of Psychia- suppression and reappraisal. Emotion, 10(4), try, 168(12), 1255–1265. 563–572. Gratton, G., Coles, M. G., & Donchin, E. (1992). Ehrlich, I., Humeau, Y., Grenier, F., Ciocchi, Optimizing the use of information: Strategic S., Herry, C., & Lüthi, A. (2009). Amygdala control of activation of responses. Journal of inhibitory circuits and the control of fear Experimental Psychology. General, 121(4), memory. Neuron, 62(6), 757–771. 480 –506. Etkin, A., Egner, T., & Kalisch, R. (2011). Emo- Gross, J. J. (this volume, 2014). Emotion regula- tional processing in anterior cingulate and tion: Conceptual and empirical foundations. medial prefrontal cortex. Trends in Cognitive In J. J. Gross (Ed.), Handbook of emotion Sciences, 15(2), 85–93. regulation (2nd ed., pp. 3–20). New York: Etkin, A., Egner, T., Peraza, D. M., Kandel, E. Guilford Press. R., & Hirsch, J. (2006). Resolving emotional Gross, J. J, & Levenson, R. W. (1993). Emo- conflict: A role for the rostral anterior cingu- tional suppression: Physiology, self-­report, late cortex in modulating activity in the amyg- and expressive behavior. Journal of Personal- dala. Neuron, 51(6), 871–882. ity and Social Psychology, 64(6), 970–986. Etkin, A., Prater, K. E., Hoeft, F., Menon, V., Gross, J. J., & Thompson, R. A. (2007). Emo- & Schatzberg, A. F. (2010). Failure of anterior tion regulation: Conceptual foundations. In J.

88 BIOLOGICAL BASES J. Gross (Ed.), Handbook of emotion regula- LeDoux, J. (2012). Rethinking the emotional tion (pp. 3–24). New York: Guilford Press. brain. Neuron, 73(4), 653–676. Gruber, J., Harvey, A. G., & Gross, J. J. (2012). When trying is not enough: Emotion regula- Levenson, R. W. (1999). The intrapersonal func- tion and the effort-­success gap in bipolar dis- tions of emotion. Cognition and Emotion, order. Emotion, 12(5), 997–1003. 13(5), 481–504. Gruzelier, J., & Eves, F. (1987). Rate of habitu- ation of electrodermal orienting responses: A Levenson, R. W. (2003). Blood, sweat, and comparison of instructions to stop respond- fears: The autonomic architecture of emotion. ing, count stimuli, or relax and remain indif- Annals of the New York Academy of Sciences, ferent. International Journal of Psychophysi- 1000, 348–366. ology, 4(4), 289–291. Gyurak, A., Gross, J. J., & Etkin, A. (2011). Lieberman, M. D., Eisenberger, N. I., Crockett, Explicit and implicit emotion regulation: A M. J., Tom, S. M., Pfeifer, J. H., et al. (2007). dual-p­ rocess framework. Cognition and Emo- Putting feelings into words: Affect label- tion, 25(3), 400–412. ing disrupts amygdala activity in response to Hare, T. A., Malmaud, J., & Rangel, A. (2011). affective stimuli. Psychological Science, 18(5), Focusing attention on the health aspects of 421– 428. foods changes value signals in vmPFC and improves dietary choice. Journal of Neurosci- Lieberman, M. D., Hariri, A. R., Jarcho, J. ence, 31(30), 11077–11087. M., Eisenberger, N. I., & Bookheimer, S. Y. Hare, T. A., Tottenham, N., Galvan, A., Voss, H. (2005). An fMRI investigation of race-­related U., Glover, G. H., & Casey, B. J. (2008). Bio- amygdala activity in African-A­ merican and logical substrates of emotional reactivity and Caucasian-A­ merican individuals. Nature regulation in adolescence during an emotional Neuroscience, 8(6), 720–722. go–no-go task. Biological Psychiatry, 63(10), 927–934. Lissek, S., Powers, A. S., McClure, E. B., Phelps, Hariri, A. R., Bookheimer, S. Y., & Mazziotta, E. A., Woldehawariat, G., Grillon, C., et al. J. C. (2000). Modulating emotional responses: (2005). Classical fear conditioning in the Effects of a neocortical network on the limbic anxiety disorders: A meta-a­ nalysis. Behaviour system. NeuroRreport, 11(1), 43–48. Research and Therapy, 43(11), 1391–1424. James, W. (1884). What is an emotion? Mind, 9, 188–205. Livneh, U., & Paz, R. (2012). Amygdala–­ Johnstone, T., Van Reekum, C. M., Urry, H. prefrontal synchronization underlies resis- L., Kalin, N. H., & Davidson, R. J. (2007). tance to extinction of aversive memories. Neu- Failure to regulate: Counterproductive recruit- ron, 75(1), 133–142. ment of top-down prefrontal–­subcortical cir- cuitry in major depression. Journal of Neuro- Lovibond, P. F. (2004). Cognitive processes in science, 27(33), 8877–8884. extinction. Learning and Memory, 11(5), Kable, J. W., & Glimcher, P. W. (2007). The 495–500. neural correlates of subjective value during intertemporal choice. Nature Neuroscience, Maier, M. E., & di Pellegrino, G. (2012). 10(12), 1625–1633. Impaired conflict adaptation in an emotional Kalisch, R. (2009). The functional neuroanat- task context following rostral anterior cingu- omy of reappraisal: Time matters. Neurosci- late cortex lesions in humans. Journal of Cog- ence and Biobehavioral Reviews, 33(8), 1215– nitive Neuroscience, 24(10), 2070–2079. 1226. Keltner, D., & Gross, J. J. (1999). Functional Martin, L. N., & Delgado, M. R. (2011). The accounts of emotions. Cognition and Emo- influence of emotion regulation on decision-­ tion, 13(5), 467–480. making under risk. Journal of Cognitive Neu- Keltner, D., & Haidt, J. (2001). Social functions roscience, 23(9), 2569–2581. of emotions at four levels of analysis. In W. G. Parrott (Ed.), Emotions in social psychology: Mauss, I. B., Levenson, R. W., McCarter, L., Essential readings (pp. 175–184). New York: Wilhelm, F. H., & Gross, J. J. (2005). The tie Psychology Press. that binds? Coherence among emotion expe- rience, behavior, and physiology. Emotion, 5, 175–190. McClure, S. M., Laibson, D. I., Loewenstein, G., & Cohen, J. D. (2004). Separate neural sys- tems value immediate and delayed monetary rewards. Science, 306(5695), 503–507. McClure, E. B., Monk, C. S., Nelson, E. E., Parrish, J. M., Adler, A., Blair, R. J. R., et al. (2007). Abnormal attention modulation of fear circuit function in pediatric generalized

Neurobiological Model of Implicit and Explicit Emotion Regulation 89 anxiety disorder. Archives of General Psychia- Ochsner, K. N., & Gross, J. J. (this volume, try, 64(1), 97–106. 2014). The neural bases of emotion and emo- McRae, K., Hughes, B., Chopra, S., Gabrieli, J. tion regulation: A valuation perspective. In J. D. E., Gross, J. J., & Ochsner, K. N. (2010). J. Gross (Ed.), Handbook of emotion regula- The neural bases of distraction and reap- tion (2nd ed., pp. 23–42). New York: Guilford praisal. Journal of Cognitive Neuroscience, Press. 22(2), 248–262. Milad, M. R., Pitman, R. K., Ellis, C. B., Gold, A. Paret, C., Brenninkmeyer, J., Meyer, B., Yuen, L., Shin, L. M., Lasko, N. B., et al. (2009). Neu- K. S. L., Gartmann, N., Mechias, M.-L., robiological basis of failure to recall extinction et al. (2011). A test for the implementation–­ memory in posttraumatic stress disorder. Bio- maintenance model of reappraisal. Frontiers logical Psychiatry, 66(12), 1075–1082. in Psychology, 2, 216. [E-publication ahead of Milad, M. R., & Quirk, G. J. (2002). Neurons print] in medial prefrontal cortex signal memory for fear extinction. Nature, 420(6911), 70–74. Pejic, T., Hermann, A., Vaitl, D., & Stark, R. Mobbs, D., Petrovic, P., Marchant, J. L., Has- (2011). Social anxiety modulates amygdala sabis, D., Weiskopf, N., Seymour, B., et al. activation during social conditioning. Social (2007). When fear is near: Threat imminence Cognitive and Affective Neuroscience, 8(3), elicits prefrontal–p­ eriaqueductal gray shifts in 267–276. humans. Science, 317(5841), 1079–1083. Monk, C. S., Klein, R. G., Telzer, E. H., Schroth, Pessoa, L. (2010). Emergent processes in E. A., Mannuzza, S., Moulton, J. L., et al. cognitive–­emotional interactions. Dialogues (2007). Amygdala and nucleus accumbens in Clinical Neuroscience, 12(4), 433–448. activation to emotional facial expressions in children and adolescents at risk for major Phelps, E. A., Delgado, M. R., Nearing, K. I., & depression. American Journal of Psychiatry, LeDoux, J. E. (2004). Extinction learning in 165(1), 90–98. humans: Role of the amygdala and vmPFC. Moors, A., & De Houwer, J. (2006). Automa- Neuron, 43(6), 897–905. ticity: A theoretical and conceptual analysis. Psychological Bulletin, 132(2), 297–326. Ploghaus, A., Becerra, L., Borras, C., & Bor- Myers-S­ chulz, B., & Koenigs, M. (2012). Func- sook, D. (2003). Neural circuitry underly- tional anatomy of ventromedial prefrontal ing pain modulation: Expectation, hypnosis, cortex: Implications for mood and anxiety dis- placebo. Trends in Cognitive Sciences, 7(5), orders. Molecular Psychiatry, 17(2), 132–141. 197–200. Nitschke, J. B., Sarinopoulos, I., Oathes, D. J., Johnstone, T., Whalen, P. J., Davidson, R. J., Ploghaus, A., Tracey, I., Gati, J. S., Clare, S., et al. (2009). Anticipatory activation in the Menon, R. S., Matthews, P. M., et al. (1999). amygdala and anterior cingulate in general- Dissociating pain from its anticipation in the ized anxiety disorder and prediction of treat- human brain. Science, 284(5422), 1979–1981. ment response. American Journal of Psychia- try, 166(3), 302–310. Pole, N., Neylan, T. C., Otte, C., Henn-Hasse, Norrholm, S. D., Jovanovic, T., Olin, I. W., C., Metzler, T. J., & Marmar, C. R. (2009). Sands, L. A., Karapanou, I., Bradley, B., et al. Prospective prediction of posttraumatic stress (2011). Fear extinction in traumatized civil- disorder symptoms using fear potentiated ians with posttraumatic stress disorder: Rela- auditory startle responses. Biological Psychia- tion to symptom severity. Biological Psychia- try, 65(3), 235–240. try, 69(6), 556–563. Ochsner, K. N., Bunge, S. A., Gross, J. J., & Ray, R. D., Ochsner, K. N., Cooper, J. C., Rob- Gabrieli, J. D. E. (2002). Rethinking feelings: ertson, E. R., Gabrieli, J. D., & Gross, J. J. An FMRI study of the cognitive regulation of (2005). Individual differences in trait rumina- emotion. Journal of Cognitive Neuroscience, tion and the neural systems supporting cog- 14(8), 1215–1229. nitive reappraisal. Cognitive, Affective, and Ochsner, K. N., & Gross, J. J. (2005). The cogni- Behavioral Neuroscience, 5(2), 156–168. tive control of emotion. Trends in Cognitive Sciences, 9(5), 242–249. Sarinopoulos, I., Grupe, D. W., Mackiewicz, K. L., Herrington, J. D., Lor, M., Steege, E. E., et al. (2010). Uncertainty during anticipa- tion modulates neural responses to aversion in human insula and amygdala. Cerebral Cortex, 20(4), 929–940. Shin, L. M., Wright, C. I., Cannistraro, P. A., Wedig, M. M., McMullin, K., Martis, B., et al. (2005). A functional magnetic resonance imaging study of amygdala and medial pre-

90 BIOLOGICAL BASES frontal cortex responses to overtly presented E. (2011). Cognitive training for impaired neu- fearful faces in posttraumatic stress disorder. ral systems in neuropsychiatric illness. Neuro- Archives of General Psychiatry, 62(3), 273– psychopharmacology, 37(1), 43–76. 281. Wager, T. D., Davidson, M. L., Hughes, B. L., Straube, T., Schmidt, S., Weiss, T., Mentzel, Lindquist, M. A., & Ochsner, K. N. (2008). H.-J., & Miltner, W. H. R. (2009). Dynamic Prefrontal–s­ubcortical pathways mediating activation of the anterior cingulate cortex dur- successful emotion regulation. Neuron, 59(6), ing anticipatory anxiety. NeuroImage, 44(3), 1037–1050. 975–981. Wessa, M., & Flor, H. (2007). Failure of extinc- Stroop, J. R. (1935). Studies of interference in tion of fear responses in posttraumatic stress serial verbal reactions. Journal of Experimen- disorder: Evidence from second-­order con- tal Psychology, 18, 643–662. ditioning. American Journal of Psychiatry, Taylor, S. E., Eisenberger, N. I., Saxbe, D., 164(11), 1684–1692. Lehman, B. J., & Lieberman, M. D. (2006). Williams, L. M., Liddell, B. J., Kemp, A. H., Neural responses to emotional stimuli are Bryant, R. A., Meares, R. A., Peduto, A. S., et associated with childhood family stress. Bio- al. (2006). Amygdala–p­ refrontal dissociation logical Psychiatry, 60(3), 296–301. of subliminal and supraliminal fear. Human Vinogradov, S., Fisher, M., & de Villers-­Sidani, Brain Mapping, 27(8), 652–661.

Part III Cognitive Approaches



Chapter 6 Delay Discounting: A Two‑Systems Perspective Eric M. Miller Christian Rodriguez Bokyung Kim Samuel M. McClure Throughout our daily lives, we confront functions, which describe mathematically decisions and situations that require self-­ how subjective value declines as a function control. Generally, these are of relatively of delay. minor consequence, such as when one resists the urge to order dessert or spend money on There have been tremendous advances in an expensive electronic gadget. However, research on delay discounting over the past failure to control one’s impulses can also 10 years, spurred in part by experiments have serious ramifications, as with eating using brain imaging as people engage in disorders, substance abuse, and compulsive intertemporal choices. To a first approxima- gambling (Dawe & Loxton, 2004). In this tion, the processes involved in evaluating chapter, we discuss the cognition–­emotion outcomes have been distinguished by the interactions that occur with temptations extent to which they rely on “emotion” ver- and self-c­ ontrol as they are understood and sus “reason” (e.g., McClure, Laibson, Loew- have been studied in psychology and neuro- enstein, & Cohen, 2004; Hare, Camerer, science. & Rangel, 2009; Figner et al., 2010). This dichotomy roughly corresponds to the trad- Most often this work focuses on a set of eoff between satisfying immediate desires phenomena that together comprise delay and adhering to abstract, long-term goals. discounting and the class of choices that The terms used to describe these processes require selecting between rewards available have been diverse: hot and cold; automatic at different points in time, known as inter- and controlled; impulsive and deliberative; temporal choices. Delay discounting refers passions and reason; visceral and abstract; to the simple fact that people tend to prefer and doer and planner, among others (Posner to receive rewards sooner rather than later. & Snyder, 1975; Shiffrin & Schneider, 1977; In an intertemporal choice, the subjective Thaler & Shefrin, 1981; Chaiken & Trope, value of each outcome is discounted in pro- 1999; Lieberman, Gaunt, Gilbert, & Trope, portion to the delay until its receipt, through 2002; Mischel, Ayduk, & Mendoza-­Denton, the process of delay discounting. One of the 2003; Loewenstein, 1996). advantages of focusing on delay discounting is that it is possible to analyze how discount- The interaction between cognition and ing takes place with the use of discount emotion is common to emotion regulation generally; delay discounting can therefore be 93

94 COGNITIVE APPROACHES thought of as one class of behavior in which Progress along these lines has come in part emotion regulation strategies are employed. from investigations of how different decision Indeed, the self-­regulation strategies used in processes interact in guiding behavior, high- the context of intertemporal choice can be lighting important functions that seem to be interpreted through the regulatory process played by regions of the medial prefrontal model proposed by Gross (1998). Specifi- cortex. Finally, we return to emotion regu- cally, regulation can occur at several points lation, and discuss some of the intentional in the decision-­making process. First, we can strategies that we can (and do) employ to intervene before the decision takes place, overcome temptations, described from the avoiding situations and circumstances that perspective of the neural systems involved in are likely to require self-­control. In inter- delay discounting. temporal choice, one such strategy referred to as precommitment involves committing Neural Systems Involved to one course of behavior ahead of time so in Delay Discounting as to avoid unwanted temptations (Strotz, Our argument is that multiple systems con- 1956). Second, we can regulate our valua- tribute to delay discounting. Although this tion processes at the time of decision mak- model provides a reasonable account of how ing, shifting our attention or reappraising decision making might be thought to occur, our perception of an alluring stimulus (e.g., an important question should be addressed Hutcherson, Plassman, Gross, & Rangel, at the outset: Is it necessary to posit the exis- 2012). Third, following the process of valu- tence of multiple separate systems? Many ation, we can attempt to suppress impulsive economic theories indicate that decision choice tendencies by exerting top-down cog- making can be wholly explained by the max- nitive regulation, a mechanism commonly imization of a unitary function of expected related to self-­control (Hare et al., 2009). utility. Given that a single utility function Choice regulation by self-­control implies can reasonably account for decision making that we select an option that is consistent in most contexts, why not settle for a more with long-term goals even while we expe- parsimonious, single-s­ystem account? Our rience the temptation to choose a differ- answer is two pronged. First, as will be dis- ent option (cf. Figner et al., 2010). In each cussed, the variability in delay discounting of these regulatory strategies, the interplay behavior observed within and between indi- between valuation and cognitive regula- viduals is difficult to capture with a single-­ tion aligns closely with the functions of the system account, but it follows naturally from “hot” and “cold” systems described earlier. a multiple systems model (van den Bos & As such, studying intertemporal choices as McClure, 2013). Second, to examine deci- a combination of these two factors provides sion making with techniques such as func- a useful framework for understanding the tional magnetic resonance imaging (fMRI) mechanisms by which we regulate our deci- provides strong evidence for qualitatively sions. different neural systems involved in delay discounting (although some controversy still In the remainder of this chapter we con- remains; McClure et al., 2004; McClure, sider the various cognitive and neural pro- Ericson, Laibson, Loewenstein, & Cohen, cesses that together compose delay dis- 2007; Kable & Glimcher, 2007; Peters & counting, and emphasize the regulatory Büchel, 2011). In many aspects of their func- mechanisms involved in controlling our tion and relationship to behavior, these neu- decision making. We begin with the brain, ral systems appear to be specialized for the and describe the large-scale distinction that qualitatively different purposes of valuation is commonly made between brain networks and top-down cognitive control. linked to the behaviorally defined constructs of valuation and self-c­ ontrol. We then sum- Valuation marize available evidence suggesting that We face a tremendous number of choices these processes account for many aspects of every day. Occasionally, these choices are delay discounting behavior. Recent research has been expanding on this framework, with the goal of identifying the diversity of brain processes involved in intertemporal choice.

Delay Discounting 95 very significant, such as when we select obscure to our explicit understanding. The which college to attend or decide how to subjective experiences generated by immedi- invest our money. Such decisions warrant ate outcomes share these properties. careful deliberation and integration of the various factors that contribute to the value At the neural level, a complex network of each course of action. However, for the of brain structures, both cortical and sub- vast majority of the choices that we face, cortical, underlies these automatic valua- careful deliberation would be prohibitive. tion processes (Figure 6.1). A wealth of data If we stopped to think about all of the fac- now indicate that positive motivational and tors that determine what we should eat for affective signals are encoded in and com- breakfast, which clothes to wear, and which municated by regions of the brain that are direction to drive to work, it would be diffi- targeted by midbrain dopamine neurons, cult to accomplish anything during the day. including the nucleus accumbens (NAcc) and portions of the ventromedial prefrontal cor- We refer to valuation processes as the tex (vmPFC; e.g., Knutson, Fong, Bennett, set of relatively automatic processes that Adams, & Hommer, 2003). Other regions, guide our behavior outside of our delibera- including the amygdala and insular cortex, tive control, facilitating the myriad choices are also involved in affective processing, we face every day. In many instances, such particularly with regard to signaling salient automatic judgments are accompanied by an or aversive events (LaBar, Gatenby, Gore, affective response (Kahneman, 2003). This LeDoux, & Phelps, 1998; Wicker et al., appears to be particularly true for delay 2003). The NAcc and amygdala, core struc- discounting. An immediate reward can be tures in these circuits, are evolutionarily extremely tempting, producing affectively quite old and exist with relatively conserved charged, high-a­ rousal approach tendencies. function in many animal species. Due partly Influential theories of emotions posit that to this fact, the motivational processes sup- one of the functions of these processes is to ported by this network of brain regions are signal assessments, such as danger, so as to thought to form a system that is separable influence behavior without necessarily mak- from deliberative thought and reasoning ing us fully aware of the source of the threat that is likely to be specific to humans. (LeDoux, 2000). Generally, we make these assessments reflexively, without intention or Dopaminergic projections (i.e., neuronal deliberate control, and in a manner that is connections that rely on the neurotransmit- ter dopamine) into the NAcc are known to posterior parietal posterior cingulate dorsal anterior cortex cortex cingulate cortex lateral PFC substantia nigra/ ventromedial VTA PFC ventral striatum FIGURE 6.1.  Two systems in delay discounting. Reward prediction, affect, and automatic behavioral responses are associated with the ventral striatum (nucleus accumbens), ventromedial PFC, and pos- terior cingulate cortex. These functions are components of automatic valuation. By contrast, regions in the dorsolateral and dorsomedial PFC, as well as the posterior parietal cortex, are associated with cognitive control processes and self-c­ ontrol.

96 COGNITIVE APPROACHES be an especially crucial motivational sig- value and guide our behavior on the basis of nal. Some of the most compelling evidence these predictions (Schultz, Dayan, & Mon- for this comes from studies in which rats tague, 1997). However, when these pro- were given the opportunity to perform an cesses go awry, as is the case in addiction, action to self-s­timulate different regions of exaggerated value signals for things such as their brains (Olds, 1977; Gallistel, Shizgal, drug-r­elated cues can lead us to overvalue & Yeomans, 1981; Ikemoto & Panksepp, and seek out maladaptive outcomes (Redish, 1999). Rats in these studies are especially 2004). motivated to self-­stimulate regions of the midbrain that evoke dopamine release into The brain contains a separate network the NAcc (Ikemoto & Panksepp, 1999). In for encoding aversive signals and detecting fact, self-­stimulation to these brain regions outcomes in our environment that ought is so reinforcing that rats will self-­stimulate to be avoided. The amygdala has been the continuously (i.e., 2,000 times an hour, or brain structure most clearly implicated with roughly once every 2 seconds), to the exclu- learning and detecting negative stimuli and sion of all other activities, until they are environmental states (although recent evi- too physically exhausted to continue (Olds, dence suggests that, at the spatial resolu- 1977). Rats also exert significant physical tion afforded by fMRI, the amygdala may effort and endure pain (electric shock) to respond more generally to salient events, receive this stimulation (Olds, 1977). They irrespective of valence; Hamann, Ely, Hoff- even prefer brain stimulation to rewards man, & Kilts, 2002; Phelps, 2006). For such as food, when the stimulation is strong example, presenting human or animal sub- enough (Shizgal & Conover, 1996). jects with a stimulus that predicts an aver- sive event, such as a shock, is associated Dopamine release in the NAcc seems to be with increased amygdala activation, even if the crucial mediator of this self-­stimulating the shock does not actually occur (Phillips behavior. Pharmacologically inhibiting the & LeDoux, 1992; Phelps et al., 2001). Simi- effects of dopamine release in the NAcc larly, amygdala lesions in animals have been greatly diminishes the rewarding proper- shown to produce a number of odd behav- ties of this stimulation, and direct release of ioral effects, including increased approach dopamine into the NAcc can substitute for behaviors in monkeys toward objects that electrical stimulation (Wise, 1982). These they would normally fear (Blanchard & reward effects of dopamine also seem to Blanchard, 1972). As might be expected, apply in humans. This is clearly demon- although aversive signals from the amygdala strated by many drugs of abuse that directly are generally necessary for adaptive behav- increase dopamine availability in the brain, ior, they can also go awry, as is the case in such as cocaine and methamphetamine social phobias (Phan, Fitzgerald, Nathan, & (Koob & Bloom, 1988). Together, these data Tancer, 2006). underscore the importance of dopamine and the NAcc in motivating behavior (Ikemoto A clear feature of both NAcc and amyg- & Panksepp, 1999; Berridge, Robinson, & dala function is their automaticity. Amyg- Aldridge, 2009). dala responses to fearful or salient stimuli occur even when participants are unaware Functional neuroimaging experiments in of seeing them, such as when images are humans have also demonstrated the involve- shown so fast that they are only perceived ment of dopamine-­associated brain regions subliminally (Whalen et al., 1998). Sublimi- such as the NAcc and vmPFC in track- nal responses are also found with respect ing value and signaling the expectation of to rewards in the NAcc (Pessiglione et al., rewarding outcomes (Knutson et al., 2003; 2008). Additionally, the NAcc is a com- McClure, Berns, & Montague, 2003; Del- ponent of the basal ganglia, a subcortical gado, 2007). Neural activity in these regions motor circuit that underlies the development can be measured using fMRI, and such mea- of habits (e.g., Jog, Kubota, Connolly, Hill- surements have been shown to scale propor- egaart, & Graybiel, 1999). As such, it is well tionally with the magnitude of expected positioned to facilitate the development of reward (Knutson et al., 2003). In most situ- new automatic behaviors. ations, these dopamine-­based value signals are quite helpful, as they allow us to predict Thus, there is strong evidence that both the amygdala and NAcc underlie the auto-

Delay Discounting 97 matic and stereotyped behaviors that are ink; Cohen, Dunbar, & McClelland, 1990). often accompanied by affective responses. Without explicitly controlling behavior to It is easy to see why such a system might arrive at the appropriate response, people have evolved: The automatic generation of are likely to commit errors by automatically value and behaviors allows quick and effort- reading the word. less formulation of responses to rewards or threats in the environment. These systems The brain structures associated with cog- are therefore believed to allow us to get on nitive control are distinct from those that with our day, and, perhaps just as impor- we previously attributed to valuation. These tantly, once helped us in ancestral environ- structures include dorsolateral and anterior ments to respond quickly enough to avoid (frontopolar) regions of the prefrontal cor- predators and obtain fleeting rewards. tex, regions within the dorsomedial prefron- tal cortex, as well as the posterior parietal Although rapid, effortless, adaptive cortex (PPC). These brain regions are consis- responses are a key benefit of this system, tently observed to be involved in a variety of it is at the same time limited by its reliance cognitive processes, such as working mem- on stereotyped behaviors. In the absence of ory (e.g., Cohen et al., 1997), abstract rea- regulatory influences, our valuation system soning (e.g., Kroger et al., 2002), and general seems to overgeneralize responses, leading problem solving (e.g., Duncan et al., 2000). to behaviors that are inappropriate in some There is broad consensus that these systems circumstances (e.g., Hershberger, 1986). are central to the brain’s ability to respond Moreover, in the domain of delay discount- flexibly to rapidly changing task demands, ing, automatic valuation processes are inad- as well as the pursuit of longer-t­erm, goal-­ equate for long-term planning. The system is directed behaviors, especially when faced hypothesized to learn to produce behaviors with competition from more salient stimuli that are appropriate given what is occurring or automatic responses (Miller & Cohen, in the immediate environment. For goals 2001). Although our brain’s cognitive con- that require planning over extended peri- trol system involves an extensive network ods of time (weeks or years) valuation pro- of interconnected structures, it is possible cesses are simply not equipped to determine to illustrate some of its mechanistic features appropriate behaviors. Instead, in these by restricting our focus to two regions: the cases, decision making requires careful con- dorsolateral prefrontal cortex (dlPFC) and sideration of goals and simulation of future dorsal anterior cingulate cortex (dACC). states. These goal-r­ elated functions are com- monly referred to as cognitive control. The dlPFC seems to be crucial for success- ful planning and other executive functions. Cognitive Control Lesions of the dlPFC are associated with dys- The top-down cognitive control system is function in organization, planning, working defined by deliberative, rule-based consider- memory, and attention (Shallice & Burgess, ation of different courses of action (Kahne- 1991; Stuss & Levine, 2002). Similarly, man, 2003). By definition, the computations brain imaging experiments in healthy adults involved in cognitive control occur within have shown that the completion of tasks our conscious awareness; this is the kind that require planning and problem solving of reasoning that we think about when we is associated with greater activity in this consider our subjective mental life. These region (e.g., Baker et al., 1996). The dlPFC capacities are particularly important when is also associated with the maintenance and we face unfamiliar circumstances for which manipulation of information stored in work- we do not have enough experience to know ing memory, consistent with its hypoth- the appropriate (automatic) response. We esized role in integrating and manipulating also need to rely on cognitive control when stored mental representations in rational there is a high chance for error. Consider as problem solving (Owen, McMillan, Laird, an example the famous Stroop task in which & Bullmore, 2005). Finally, the dlPFC is you have to name the color in which a word associated with the control of actions during is written, while you suppress reading the tasks that require people to override auto- word (think of the word red written in green matic responses and instead respond based on a higher-­level rule (MacDonald, Cohen, Stenger, & Carter, 2000).

98 COGNITIVE APPROACHES The dACC also plays an important role in to be gained is worth the wait. That is, as in any value-based decision, the desirability of high-level cognition. In particular, the dACC the available rewards must be determined, has been shown to be especially important and a choice must be made on the basis of for the online monitoring of performance, as these relative values (Rangel, Camerer, & well as conflict and error detection (Carter Montague, 2008). In intertemporal choice, et al., 1995). According to a dominant the- the desirability of each reward depends on ory of dACC function, conflict detection in its magnitude, as well as the delay until its the dACC acts as a gating mechanism that receipt. signals when additional control is necessary (Botvinick, Braver, Barch, Carter, & Cohen, Although everyone would prefer to 2001). According to this theory, the dACC receive reward sooner and in greater mag- detects when our automatic processes are nitude when all else is kept equal, people leading us awry and modulates activity in differ substantially in how they discount the the dlPFC, leading to changes in behavior value of future compensation. For exam- (i.e., so that we respond appropriately in cir- ple, people with chronic deficits in impulse cumstances in which errors are likely). The control, such as pathological gamblers and dACC’s purported roles in conflict monitor- cocaine addicts, tend to discount the value ing and dlPFC modulation are evident in of delayed reward far more than control tasks that span a variety of domains, rang- subjects (Bickel, Jarmolowicz, Mueller, Kof- ing from simple motor responses to moral farnus, & Gatchalian, 2012). People suf- and social decision making (MacDonald et fering from attention-­deficit/hyperactivity al., 2000; Greene, Nystrom, Engell, Darley, disorder (ADHD), a condition character- & Cohen, 2004; Sanfey, Rilling, Aronson, ized by deficits in self-­control, also discount Nystrom, & Cohen, 2003). rewards more steeply than healthy controls (Barkley, Edwards, Lanieri, Fletcher, & The dlPFC and dACC are two regions in Metevia, 2001). Discounting rates also tend a network of structures that encode contex- to decrease as people age from adolescence tual information and facilitate deliberative into adulthood, consistent with the general processing and planning. These are pre- improvements in self-c­ ontrol that occur over cisely the functions ascribed to cognitive the course of development (Green, Fry, & control. A critical limitation of this system Myerson, 1994). is that information processing is slow and computationally expensive relative to auto- As already stated, from a mechanis- matic valuation. Moreover, we can only tic point of view, making intertemporal explicitly consider one thing at a time, so choices requires discounting the value of processing in this system is serial (Kahne- outcomes based on their delay, and choos- man, 2003). These constraints limit the set ing the option with greatest discounted val- of the ­decisions that are best addressed by ues. Rationally, preferences generated in this this system. way should be consistent across time. If you prefer outcome A over an outcome B that is Intertemporal Choice to occur some fixed amount of time after A To this point, we have considered valuation (say a week), then you should always prefer and cognitive control in relative isolation. A to B, whether A occurs today (and B in a In many situations, we are presented with week) or A occurs in a year (and B in a year choices for which we must arbitrate between and a week). If this were not the case, then following our impulses to achieve immediate there would be some critical point in time at gratification and choosing the actions that which preferences must switch, so that you we know will be more beneficial in the long initially prefer B but then reverse to prefer run. Intertemporal choice experiments tar- A some time thereafter. Mathematically, get precisely this dilemma, asking people to the only discount function that ensures con- decide between smaller rewards (most com- sistency of preference is one that declines monly money) that are available sooner in exponentially with delay (Frederick, Leow- time and larger rewards that are available enstein, & O’Donoghue, 2003). This sounds after a longer delay. In such cases, it must be reasonable enough, but neither humans nor determined whether the additional amount animals demonstrate such consistency. In an often-cited example, when deciding between

Delay Discounting 99 one apple available in a year and two apples arousing, as is commonly the case with available in a year and a day, people gener- money and food (Loewenstein, 1996). Simi- ally prefer the latter. However, when the larly, we tend to be impulsive when we are choice is between one apple available today aroused, either because of some exogenous and two apples available tomorrow, people emotionally charged event (e.g., Li, 2008) or generally prefer the former (Thaler, 1981). because of internal reasons such as hunger Such preference reversals indicate that (Giordano et al., 2002; Wang & Dvorak, humans (and other animals) discount more 2010). While people appear to exhibit hyper- steeply over the near term than over the lon- bolic discounting under all of these condi- ger term; that is, people have a tendency to tions, the nature of the discount function respond impulsively to tempting rewards varies substantially by circumstance (van that are immediately available. den Bos & McClure, 2013). Time-­dependent preference reversals are An alternative approach to describ- most often explained by positing that delay ing intertemporal choice views discount- discounting follows a hyperbolic discount ing behavior as resulting from the engage- function (e.g., Ainslie, 1975; Kirby, 1997; ment of separate evaluative systems, each see Figure 6.2 for more detail). Mathemati- of which uses a different discount function cally, the hyperbolic function captures the (e.g., Loewenstein, 1996; van den Bos & observation that discount rates decline with McClure, 2013). The combined effect of time. However, this framework does little multiple distinct discount functions can pro- to explain the broad range of discount rates duce hyperbolic-­like behavior. The simplest that people exhibit for different goods under version of this account suggests that there different circumstances. In a well-known are two types of discounting systems: one example, it was pointed out that it is hard that values only goods that are immediately to imagine people being impulsive for writ- available, and another system that discounts ing paper or gasoline (Hoch & Loewen- more modestly over time (Laibson, 1997). stein, 1991). Instead, hyperbolic discounting This corresponds well to the two processing seems to apply most when we are dealing systems discussed earlier, with the automatic with goods that are somehow viscerally valuation system exhibiting steep discount- A Exponential B Hyperbolic Smaller, sooner Larger, later Value Value Time Time FIGURE 6.2.  Value and preference reversal for smaller, sooner (SS) and larger, later (LL) reward. As time progresses, and the delay until receipt of SS and LL decreases, delay discounting also decreases and subjective value grows. (A) With exponential discounting, if LL is preferred to SS initially (solid line greater than dashed line at Time 0), then this ordinal preference is necessarily maintained. (B) However, with hyperbolic discounting, preference can reverse as the time to the SS reward approaches. The gray bar indicates the period of time, just prior to the availability of SS, when preferences favor the SS outcome.

100 COGNITIVE APPROACHES ing and the cognitive control system placing In other words, when structures in the auto- more equal weight on immediate and future matic valuation system were more active, rewards (McClure et al., 2004, 2007). people tended to choose more impulsively, but when structures in the cognitive control A tremendous advantage of the two-­ system were more active, people were more system model of discounting is that it pro- likely to choose the larger, delayed option. vides a simple explanation for why people are impulsive for certain goods and not oth- Subsequent fMRI studies have further ers (e.g., writing paper) for which there is elaborated and confirmed the dissociation no associated affective response. Moreover, of these neural systems. For example, the it suggests alternative routes to meliorat- results from the experiment just described ing impulsive tendencies. One can work to were replicated in an experiment in which dampen the emotional impact of a stimu- thirsty subjects had to make choices between lus, exert deliberative control for the sake of quantities of juice delivered at different peri- satisfying longer-­term goals, or undertake ods of time, demonstrating that this type of behaviors to avoid situations in which temp- dual processing is not specific to monetary tations hold sway. Many of these approaches rewards (McClure et al., 2007). Moreover, have been investigated in recent research, when deciding between foods, dlPFC activ- and they coincide with different strategies ity tends to support the selection of healthier common to instances of emotion regulation. options via suppressing responses in vmPFC (Hare et al., 2009; discussed in greater Neuroscientific studies of intertemporal detail below). There are important indi- choice provide support for the dissociation vidual differences in discounting that are between automatic and controlled systems well explained by this two-s­ystem frame- involved in the decision making. In one of work as well. Hariri and colleagues (2006) our early fMRI studies, people were pre- showed that individual differences in NAcc sented with two types of monetary choices: responses to winning money predict dis- a choice between an immediately available count rates. In other words, the more that option or a larger, delayed option, and a the automatic valuation system activates for choice in which both options were available the receipt of rewards, the more impulsively only after delays (McClure et al., 2004). one tends to seek them. By contrast, individ- According to the dual-p­rocess model of ual differences in cognitive control predict discounting, one would expect that choices reduced discounting. People who perform involving an immediately available reward better on a working memory task and show would be more affectively arousing in greater responses in anterior parts of the lat- nature and therefore partially reflect the eral prefrontal cortex also tend to be more function of the impulsive valuation system. patient in intertemporal choices (Shamosh et By contrast, choices involving only delayed al., 2008). rewards would not have this property, and would instead rely more heavily on delib- One important limitation of the results erative processes to form judgments. The that we have discussed up to this point is that brain activation that was observed while functional brain imaging provides data that subjects performed these two types of tri- are correlational in nature. In other words, als was consistent with this proposed for- fMRI provides information regarding which mulation. A number of valuation-a­ ssociated regions of the brain are associated with a brain structures, including the NAcc and particular cognitive process but reveals little vmPFC, were activated during decisions that about whether these brain regions actually involved choices with an immediately avail- cause the observed differences in behavior. able option. In contrast, structures involved To address this limitation, two recent stud- in deliberative processing, including the ies assessed the causal importance of these dACC and dlPFC, were more engaged dur- structures through more direct manipula- ing decisions that involved two delayed out- tions. The first manipulated intertemporal comes. Additionally, for choices that pit an decision making using repetitive transcra- immediate reward against a delayed one, nial magnetic stimulation (rTMS; Figner et relative brain activation in the regions that al., 2010). In rTMS, electromagnetic pulses comprise these two systems was predictive are produced to disrupt activity in particular of the choice that people ultimately made. regions of an otherwise typically functioning

Delay Discounting 101 brain. In this experiment, rTMS was admin- that has received very little attention in the istered the left and right dlPFC. After rTMS literature on delay discounting—­the hippo- administration, subjects completed a set of campus—­in making intertemporal choices intertemporal choice questions to assess the (Peters & Büchel, 2011). As a whole, these extent to which the procedure would disrupt findings indicate the need to understand the decision-­making process. rTMS to the how automatic and controlled processes are left lateral prefrontal cortex caused people to integrated to produce coherent behavior, make more impatient choices, demonstrat- and how this integration might be expanded ing that activation in this region causally to include additional brain systems. promotes goal-d­ irected decision making. A second study produced a similar effect by Value Integration in the vmPFC enhancing dopamine function pharmaco- A recent fMRI study dissociated the contri- logically by giving participants drugs that butions of the brain systems we have linked increase dopaminergic activity (Pine, Shiner, to valuation and self-c­ontrol in the context Seymour, & Dolan, 2010). Consistent with of decision making relative to unhealthy the two-s­ ystem framework, this study found foods (Hare et al., 2009). In this study, sub- that discount rates were elevated as well. jects were scanned as they chose between food options that varied in health and taste Toward Multiple Interacting Systems qualities. For example, potato chips are low The two-­system model we discussed earlier on health value but high on taste, and the was primarily derived from behavioral stud- opposite is true for broccoli. Subjects were ies. We now know that many of the prop- classified as “self-­controllers” or “non-self-­ erties of this two-­system model correspond controllers” based on their food preferences well with what is known about brain reward as expressed during the task. Self-c­ ontrollers systems (i.e., dopamine and the NAcc) and were people who made their decisions based regions linked to executive functions (i.e., on both health and taste, whereas non- dlPFC and dACC). However, additional self-c­ontrollers made their decisions based details of brain function urge a more com- solely on taste. Activation in the vmPFC plex model, with more precise functions was shown to be associated with the total ascribed to individual brain areas, inclusion subjective value assigned to individual food of additional systems in the delay discount- items, such that activity was correlated ing processes, as well as a more complete with subjects’ rated preference for the item theory for how brain systems interact dur- at the time of choice. Additionally, for self-­ ing choice. controllers, vmPFC activation at the time of choice was correlated with both the health At least three lines of research support and the taste of the items, whereas for non- a more refined, multiple-­systems model of self-c­ontrollers, vmPFC activation was cor- delay discounting. First, work by Bechara, related only with taste. Thus, this study indi- Damasio, Damasio, and Lee (1999) exam- cates that the vmPFC encodes the total value ining the behavioral deficits associated with assigned to individual food items. lesions to the vmPFC indicates a role for this structure in integrating cognitive and Importantly, the separate factors that con- emotional influences on behavior, suggest- tribute to total value could also be identified ing that this structure’s function is more in this study. The dlPFC showed elevated nuanced than allowed for in a two-­system activation when participants successfully model. Second, although the available evi- engaged self-c­ontrol, so that dlPFC activity dence suggests that the dACC plays a criti- was greatest when subjects chose to reject cal role in regulating the relative influence liked but unhealthy foods. Additionally, of automatic and controlled processes in comparison of functional activity in the decision making, no studies have clari- dlPFC between self-­controllers and non-self-­ fied exactly how this regulatory function is controllers on these trials indicated that self-­ implemented in complex scenarios such as controllers displayed greater activation in intertemporal choice. Finally, recent studies this region at the time of choice, providing suggest an important role for a brain region confirmatory evidence that activity in this region facilitates controlled decision making.

102 COGNITIVE APPROACHES Finally, the investigators found a relation- study the menu, considering prices and the ship between activation in the vmPFC and different types of food available. He would dlPFC for decisions on liked but unhealthy even drive to the restaurants to assess their foods (i.e., those that should require the most ambiance. Despite this effort, he would still self-c­ontrol). In particular, greater activity be unable to decide. On open-ended neuro- in the dlPFC was associated with reduced psychiatric tests, Elliot could provide many activity in the vmPFC on these trials. These reasonable solutions to complex problems, results indicate a means by which valuation only to exclaim at the end “and after all this, and self-c­ontrol processes can interact dur- I still wouldn’t know what to do!” (Dama- ing decision making to facilitate successful sio, 1994, p. 49). Elliot’s problem lay in self-­control. Critically, the vmPFC seems to linking automatic valuation and regulatory function as the site of interaction. control when necessary to make a decision. Specifically, when decisions become too These findings fit into a larger body of complex to be handled by the limited capac- data indicating that the vmPFC is impor- ity of our cognitive control system, we seem tant for integrating cognitive and emotional to rely on the combination of affective and influences on behavior. Some of the most cognitive judgments—s­omething like a gut compelling evidence for this function comes feeling—­to make the decision. This seems to from studies of patients with lesions to the be the function of the vmPFC. vmPFC. These patients have normal execu- tive functioning, with normal IQ and over- Valuation and Control Interactions all performance on tests of cognitive abili- Involving the dACC–dlPFC Network ties. They also express normal emotional The dlPFC and dACC, key components of responses in tasks such as fear condition- the cognitive control system discussed ear- ing, in which an initially neutral stimulus lier, have been extensively studied in cogni- (e.g., a visual stimulus) is paired with some- tive neuroscience and are primary regions thing aversive (e.g., an unpleasant sound) involved in deliberated choices. They are until participants learn to fear the stimulus critical nodes in a general-p­urpose cogni- (Bechara et al., 1999). Where patients with tive control network that guides behavior in vmPFC lesions show striking deficits is in a wide variety of tasks, including intertem- situations in which cognition and emotion poral decision making (Ridderinkhof et al., must be combined to behave appropriately. 2004). We understand the most about the Damasio (1994) summarizes this by saying interactions between the dlPFC and dACC that reason and emotion “intersect” at the from their function in perceptual decision-­ vmPFC. making tasks. Although these perceptual tasks are mechanistically different from This intersection is clearly illustrated by intertemporal choice, they provide useful behavioral deficits expressed by Elliot, a insight regarding the role these regions may famous patient with vmPFC impairment play in more complex decision scenarios. (Eslinger & Damasio, 1985). Elliot had a tumor surgically removed from his vmPFC, Two experimental paradigms, the Erik- compromising the function of neighboring sen flanker task and the Stroop task, have cortical areas. The consequences were disas- revealed a wealth of mechanistic detail about trous. By all accounts, Elliot seemed normal dACC and dlPFC function. In both tasks, when in conversations. His IQ was signifi- successful performance requires participants cantly above normal, he was appropriately to override a prepotent motor response in emotionally expressive, and he passed all order to respond accurately to the stimulus manner of neuropsychological tests. Despite presented. We discussed the Stroop task ear- this, he made terrible financial decisions, lier, and why deliberation and control are squandering all his money in poor business necessary for appropriate behavior under the decisions. He was unable to hold a job, and conditions of this experiment. The flanker he interacted poorly with others, eventually task presents similar requirements. In this leading to his wife divorcing him. Elliot’s task, participants are asked to identify the behavior seemed generally normal, but he central item among a set of distracter stim- had trouble with specific types of choices. uli (e.g., respond “right” to > > > > > and For example, choosing a restaurant for din- ner could be an onerous chore. He would

Delay Discounting 103 “left” to > > < > >). In incongruent trials, in increased activity, which then activates which the flanking stimuli support a differ- the dlPFC (Kerns et al., 2004). Consistent ent response than the target stimulus (i.e., > with this account, increased activity in this > < > >), cognitive control becomes essential. dACC–dlPFC network is commonly associ- People have a tendency to emit the incorrect ated with improved performance during per- response, as evidenced by an elevated error ceptual decision making. rate (Gratton, Coles, & Donchin, 1992). Furthermore, even when people are able to An analogous mechanism seems likely suppress the erroneous response, they are to be at play in delay discounting. Imag- slower to respond than when the flanking ine, for example, that you are asked to stimuli support the correct answer. Thus, as choose between two options that seem in the Stroop task, successful performance equally appealing (say, $10 today or $20 in on incongruent trials requires inhibition of a month). Response conflict is inevitable in the automatic tendency to respond to the this circumstance, and dACC activity may predominant stimulus attribute. be anticipated to increase. Indeed, this has been found in several studies of intertem- Imaging and neurophysiological studies poral choice (e.g., Marco-­Pallarés, Moham- indicate that the type of control required by madi, Samii, & Münte, 2010). In our own these tasks is primarily facilitated by con- work we find evidence that the dACC sig- tributions from the dlPFC and the dACC nals conflict as well. For example, Figure (Yeung, Botvinick, & Cohen, 2004; Kerns 6.3 shows an analysis (McClure et al., 2004) et al., 2004; Botvinick et al., 2001). As dis- that identifies the dACC as a region whose cussed earlier, these regions play comple- activity scales with reaction time (a surro- mentary roles. The dACC is believed to gate for choice difficulty, and hence conflict). monitor for the coactivation of incompatible responses, essentially serving as a detector The consequences of increased dlPFC for situations in which errors are likely and activity following response conflict in delay control is necessary. The dlPFC is thought discounting remain somewhat unresolved. to maintain the appropriate task goals On the one hand, a bulk of evidence suggests and to direct action selection accordingly. that increased dlPFC activity promotes the When conflict is detected, the dACC shows selection of larger, later rewards. In choices between immediate and delayed rewards, A B 0.5 dACC Short 0.4 Intermediate % Signal Change Long 0.3 0.2 0.1 0.0 -0.1 04 8 -4 Time (s) FIGURE 6.3.  dACC activity in intertemporal choice. (A) Reanalysis of the data from McClure et al. (2004) reveals that dACC responses scale with choice difficulty, as approximated by choice response time. (B) Trials were split into three groups based on reaction time. Mean hemodynamic responses in the dACC scale linearly with response time. Time of zero seconds is when participants submitted their responses. The lag to the peak of the hemodynamic responses (~5–6s) is typical for this measurement modality.

104 COGNITIVE APPROACHES increased dlPFC activity, particularly rela- campal damage are unable to reason about tive to NAcc responses, predicts selection of hypothetical future events, perhaps because the latter alternative (McClure et al., 2004, people rely on past experiences in order to 2007). Furthermore, disruption of the dlPFC conceive of probable future occurrences. by rTMS increases the proportion of choices for the immediate rewards, particularly in For delay discounting, this function of the conditions where conflict is anticipated to be hippocampus turns out to be very impor- high (Figner et al., 2010). These results sup- tant. Intuitively, bringing past experiences to port the hypothesis that when conflict arises, mind and envisioning future occurrences is the dACC may modulate the dlPFC, which an integral part of controlled decision mak- in turn implements self-­control by directly ing. Directed memory recall is guided in a biasing choices toward future-­oriented goals goal-d­irected manner by dlPFC inputs to or by modulating the integrated valuation the hippocampus (Levy & Anderson, 2002). processes occurring in the vmPFC. Consequently, rewards promised at future times linked to specific events (e.g., a $25 On the other hand, recent evidence from payment on December 23, during the holi- perceptual decision-­making experiments days) elicit increased responses in the dlPFC (e.g., Philiastides, Auksztulewicz, Heekeren, and hippocampus (Peters & Büchel, 2010). & Blankenburg, 2011), as well as theoreti- Such tangible future episodes have profound cal proposals (Kable & Glimcher, 2009), effects on delay discounting; the greater the suggests that the dlPFC might instead facili- recruitment of the hippocampus for future tate the selection of the option assigned the dates, the less the discounting of future highest discounted value by the valuation rewards. Similarly, when people are given system. This evidence supports an alterna- identical intertemporal choices, they are tive hypothesis in which the function of more patient when future times are linked to cognitive control is not necessarily linked concrete events (e.g., a holiday) than when to future-­oriented goals independent of they are expressed only with respect to the automatic valuation processes. Instead, this delay until receipt (e.g., in 3 months; Peters hypothesis suggests that automatic valua- & Büchel, 2010). This suggests that con- tion and controlled processes may interact trolled processes interact not only with valu- more generally to facilitate decision making. ation but also with cognitive functions such However, both of these proposals are consis- as memory, underscoring the need to expand tent with the notion that the dACC–dlPFC existing two-­system models of intertemporal network is the basis of deliberative decision choice. making, guiding choices toward options that maximize value, either specifically for Emotion Regulation the long-term or generally. In either case, the in Intertemporal Choice dACC–dlPFC network appears to influence We face intertemporal dilemmas every day behavior through interactions with valua- and employ a host of strategies to overcome tion regions, particularly the vmPFC (Hare temptations. In this section we discuss a few et al., 2009). of those strategies and relate them to the brain systems and cognitive processes on Controlled Decisions and Memory which they impinge. in the Hippocampus The hippocampus is well known in cognitive The most famous strategy for avoiding neuroscience primarily because of its role in temptations is termed “precommitment,” remembering specific life events, known as in which one commits to a course of action episodic memories (Burgess, Maguire, & prior to the actual time of choice to avoid O’Keefe, 2002). This is best exemplified responding impulsively. One famous exam- by patients such as HM, who are unable ple of precommitment involves a product to form new memories after losing hippo- that banks used to offer, known as “Christ- campal function (Scoville & Milner, 1957). mas Clubs” (Strotz, 1956). Christmas Clubs Episodic memory is also very important for were savings accounts that offered no inter- conceiving of the future (Schacter, Addis, est and forbade withdrawals until just before & Buckner, 2008); patients with hippo- Christmas. They were popular; people will-

Delay Discounting 105 fully signed up to have access to their money reward now and receiving two rewards at restricted for months at a time, with no some indeterminate time in the future. This compensation, so that they could afford to delay is ordinarily very difficult for children buy presents at the holidays. Analogous pro- to endure; however, Mischel et al. found that grams for precommitment continue to exist instructing children to think of the abstract today. For example, people pay money to qualities of the rewards (thereby reducing join weight loss programs that restrict their their desirability) dramatically improved calorie intake, and they join the military to their ability to wait for the better outcome. improve self-d­iscipline. Furthermore, there Furthermore, children who were worse at now exist websites where one can sign up delaying gratification expressed greater to be punished financially for not accom- activation in reward-­related regions when plishing some goal (e.g., stickk.com and attempting to suppress responses to an allur- j­oesgoals.com). ing cue (when studied again as adults; Casey et al., 2011). This provides further evidence Precommitment devices are extremely that suppression of reward-­related activation effective. For example, people ordinarily is an important factor in controlled decision perform abysmally at saving for retirement. making during intertemporal choice. However, savings rates increase dramatically when retirement plans include a precommit- A number of other strategies have been ment to increase savings progressively by shown to help regulate decision making dur- committing a portion of future raises (Thaler ing intertemporal choice. For example, based & Benartzi, 2004). Additionally, when we on the work by Peters and Büchel (2010) understand that we have “weaknesses,” we discussed earlier, thinking of the future in often take actions in anticipation of this fact. a more concrete manner can increase the Since precommitment requires anticipation appeal of obtaining a delayed reward. Such a and planning about future situations, we change in the way future reward is construed suspect that precommitment depends on the may compensate for the fact that future ben- involvement of the dlPFC and hippocampus, efits tend to be thought of in very general because concretely contemplating the future terms that do not have the same appeal as depends on these structures. the much more precisely considered imme- diate outcomes (Trope & Liberman, 2003). In the absence of precommitment, when Indeed, priming people to think more con- we confront tempting stimuli, a number of cretely also makes them more patient (Fujita strategies can be applied to curb our emo- & Han, 2009). tions and help us respond appropriately. For example, in a recent study, people were Another strategy that has been shown scanned while they attempted cognitively to regulate decision making in intertempo- to regulate their cravings and to make deci- ral choice is focusing one’s attention on the sions about whether to eat unhealthy foods future consequences of an action. Consider (Hutcherson et al., 2012). As expected, the manipulation discovered by Magen, participants made more healthy decisions Dweck, and Gross (2008). Usually, inter- when actively regulating their cravings, and temporal choices are presented as decisions they showed less controlled decision mak- between smaller, more immediate rewards ing when they focused on the pleasurable (i.e., $10 available right now) and larger, aspects of the food options. Furthermore, delayed rewards (i.e., $20 available in a active regulation of cravings was associated month). Of course, it is equivalent to reex- with decreased activity in regions of the press this choice as either $10 now and noth- brain associated with automatic valuation, ing in a month or nothing now but $20 in a whereas value-r­elated activation increased month. In this “explicit zero” framing, peo- when these cravings were indulged. Thus, ple are significantly more patient. Recently, active regulation of our emotional respond- we examined the mechanisms underlying ing can facilitate controlled decision making this effect. Based on our studies, we believe by suppressing the brain’s automatic value that expressing the future consequences of responses. immediate reward forces people to think about the future, thereby reducing a natural Similarly, in one particularly revealing present bias that leads to underconsideration study by Mischel, Shoda, and Rodriguez of future desires. In fact, priming people to (1989), children chose between receiving one

106 COGNITIVE APPROACHES think about the future is sufficient to reduce impulsivity in intertemporal choice (Radu, controlled decision making, respectively, Yi, Bickel, Gross, & McClure, 2011). based on analyses that sought to identify qualities of these hypothesized cognitive There are undoubtedly many ways that processes. Subsequent work has refined our one can reframe alluring immediate rewards understanding of these structures. We now to reduce impulsivity. Furthermore, there are have more detailed understanding of how numerous known influences on delay dis- self-c­ontrol works (via dACC–dlPFC inter- count rates that remain unexplained, such actions) and how valuation and self-­control as the “date-delay effect,” in which people interact (in part, via the vmPFC). Our sus- are more patient when future times are picion is that over the next several years the expressed as exact dates (i.e., July 25 vs. 30 nature of these kinds of processes will be days from now), or the fact that people are even better understood, and our conceptu- more patient when deciding between larger alization of the overall architecture support- rather than smaller rewards (e.g., Thaler, ing intertemporal decision making will be 1981; Read, Frederick, Orsel, & Rahman, expanded to include additional brain sys- 2005). We suspect that these effects work tems and cognitive processes. through a common pathway that ultimately suppresses reward-r­elated activation in the Although we have a basic understanding NAcc, as meliorating this myopic and impul- of how emotion regulation impacts neu- sive signal could facilitate far-s­ ighted behav- ral activation during intertemporal choice, ior. Dampening of NAcc responses can further research is necessary to refine our occur either exogenously (i.e., by presenting understanding of how different regulatory rewards in different ways) or endogenously strategies function with respect to the brain. (i.e., by deliberatively reconstruing a tempt- This will give us a better understanding of ing outcome). Identifying the precise mecha- how known regulatory strategies work, nisms by which these manipulations work allow us to understand better individual to reduce impulsivity is an exciting future differences in impulsivity, and provide a direction in this research domain. framework for developing ever more effec- tive strategies to overcome problems with Conclusions self-c­ontrol. Addiction, obesity, and other Delay discounting encompasses a spectrum self-c­ ontrol problems are rife in our society. of important behaviors in which emotion They can all be conceptualized through the regulation strategies have important con- framework of intertemporal choice, and a sequences, particularly for meliorating more detailed mechanistic understanding of impulsivity. Our aim has been to illustrate delay discounting can only facilitate efforts cognitive approaches for understanding how to combat these problems. delay discounting occurs, with a particular emphasis on two-s­ystems models. These References models posit the existence of an automatic, affect-l­aden set of processes that interact Ainslie, G. (1975). Specious reward: A behavioral with a more flexible, deliberate system to theory of impulsiveness and impulse control. govern our behavior. Psychological Bulletin, 82, 463–496. Studying delay discounting from the pur- Baker, S. C., Rogers, R. D., Owen, A. M., Frith, view of neuroscience is exciting for a number C. D., Dolan, R. J., Frackowiak, R. S. J., et al. of reasons. First, neuroscience offers a com- (1996). Neural systems engaged by planning: plementary empirical approach for establish- A PET study of the Tower of London task. ing the function of constructs such as valu- Neuropsychologia, 34, 515–526. ation and self-c­ontrol in decision making. Following on this, neuroscience contributes Barkley, R. A., Edwards, G., Laneri, M., Fletcher, additional details to a mechanistic under- K., & Metevia, L. (2001). Executive function- standing of delay discounting. For example, ing, temporal discounting, and sense of time in we originally identified vmPFC and dACC adolescents with attention deficit hyperactiv- as components of automatic valuation and ity disorder (ADHD) and oppositional defiant disorder (ODD). Journal of Abnormal Child Psychology, 29, 541–556. Bechara, A., Damasio, H., Damasio, A. R., &

Delay Discounting 107 Lee, G. P. (1999). Different contributions of Damasio, A. (1994). Descartes’ error. New York: the human amygdala and ventromedial pre- Penguin Books. frontal cortex to decision-m­ aking. Journal of Neuroscience, 19, 5473–5481. Dawe, S., & Loxton, N. J. (2004). The role of Bergh, B. V. D., Dewitte, S., & Warlop, L. (2008). impulsivity in the development of substance Bikinis instigate generalized impatience in use and eating disorders. Neuroscience and intertemporal choice. Journal of Consumer Biobehavioral Reviews, 28(3), 343–351. Research, 35, 85–97. Berridge, K. C., Robinson, T. E., & Aldridge, J. Delgado, M. R. (2007). Reward-­related responses W. (2009). Dissecting components of reward: in the human striatum. Annals of the New “Liking,” “wanting,” and learning. Current York Academy of Sciences, 1104, 70–88. Opinion in Pharmacology, 9, 65–73. Bickel, W. K., Jarmolowicz, D. P., Mueller, E. Duncan, J., Seitz, R. J., Kolodny, J., Bor, D., T., Koffarnus, M. N., & Gatchalian, K. M. Herzog, H., Ahmed, A., et al. (2000). A neu- (2012). Excessive discounting of delayed rein- ral basis for general intelligence. Science, 289, forcers as a trans-d­ isease process contributing 457– 460. to addiction and other disease-­related vulner- abilities: Emerging evidence. Pharmacology Eslinger, P. J., & Damasio, A. R. (1985). Severe and Therapeutics, 134, 287–297. disturbance of higher cognition after bilateral Blanchard, D. C., & Blanchard, R. J. (1972). frontal lobe ablation: Patient EVR. Neurol- Innate and conditioned reactions to threat in ogy, 35, 1731–1741. rats with amygdaloid lesions. Journal of Com- parative and Physiological Psychology, 81, Figner, B., Knoch, D., Johnson, E. J., Krosch, 281–290. A. R., Lisanby, S. H., Fehr, E., & Weber, E. Botvinick, M. M., Braver, T. S., Barch, D. M., U. (2010). Lateral prefrontal cortex and self-­ Carter, C. S., & Cohen, J. D. (2001). Conflict control in intertemporal choice. Nature Neu- monitoring and cognitive control. Psychologi- roscience, 13, 538–539. cal Review, 108, 624–652. Burgess, N., Maguire, E. A., & O’Keefe, J. Frederick, S., Loewenstein, G., & O’Donoghue, (2002). The human hippocampus and spatial T. (2003). Time discounting and time prefer- and episodic memory. Neuron, 35, 625–641. ence: A critical review. In G. Loewenstein, D. Carter, C. S., Braver, T. S., Barch, D. M., Botv- Read, & R. Baumeister (Eds.), Decision and inick, M. M., Noll, D., & Cohen, J. D. (1995). time (pp. 13–86). New York: Russell Sage Anterior cingulate cortex, error detection, and Foundation. the online monitoring of performance. Sci- ence, 280, 747–749. Fujita, K., & Han, H. (2009). Moving beyond Casey, B. J., Somerville, L. H., Gotlib, I. H., deliberative control of impulses: The effect of Ayduk, O., Franklin, N. T., Askren, M. K., et construal levels on evaluative associations in al. (2011). Behavioral and neural correlates of self-­control conflicts. Psychological Science, delay of gratification 40 years later. Proceed- 20, 799–804. ings of the National Academy of Sciences, Gallistel, C. R., Shizgal, P., & Yeomans, J. S. 108(36), 14998–15003. (1981). A portrait of the substrate for self-­ Chaiken, S., & Trope, Y. (Eds.). (1999). Dual-­ stimulation. Psychological Review, 88, 228– process theories in social psychology. New 273. York: Guilford Press. Cohen, J. D., Dunbar, K., & McClelland, J. L. Giordano, L. A., Bickel, W. K., Loewenstein, (1990). On the control of automatic processes: G., Jacobs, E. A., Marsch, L., & Badger, G. J. A parallel distributed processing account of (2002). Opioid deprivation affects how opioid-­ the Stroop effect. Psychological Review, 97, dependent outpatients discount the value of 332–361. delayed heroin and money. Psychopharmacol- Cohen, J. D., Perlstein, W. M., Braver, T. S., ogy, 163, 174–182. Nystrom, L. E., Noll, D. C., Jonides, J., et al. (1997). Temporal dynamics of brain activation Gratton, G., Coles, M. G., & Donchin, E. (1992). during a working memory task. Nature, 386, Optimizing the use of information: Strategic 604–608. control of activation of responses. Journal of Experimental Psychology: General, 121, 480 –506. Green, L., Fry, A., & Myerson, J. (1994). Dis- counting of delayed rewards: A life-span com- parison. Psychological Science, 5, 33–36. Greene, J. D., Nystrom, L. E., Engell, A. D., Dar- ley, J. M., & Cohen, J. D. (2004). The neu- ral bases of cognitive conflict and control in moral judgment. Neuron, 44, 389–400.

108 COGNITIVE APPROACHES Gross, J. J. (1998). Antecedent- and response-­ (2004). Anterior cingulate conflict monitor- focused emotion regulation: Divergent con- ing and adjustments in control. Science, 303, sequences for experience, expression, and 1023–1026. physiology. Journal of Personality and Social Kirby, K. N. (1997). Bidding on the future: Psychology, 74, 224–237. Evidence against normative discounting of delayed rewards. Journal of Experimental Hamann, S. B., Ely, T. D., Hoffman, J. M., & Psychology: General, 126, 54–70. Kilts, C. D. (2002). Ecstasy and agony: Acti- Knutson, B., Adams, C. M., Fong, G. W., & vation of the human amygdala in positive Hommer, D. (2001). Anticipation of increas- and negative emotion. Psychological Sci- ing monetary reward selectively recruits ence, 13(2), 135–141. nucleus accumbens. Journal of Neuroscience, 21, RC159. Hare, T. A., Camerer, C. F., & Rangel, A. (2009). Knutson, B., Fong, G. W., Bennett, S. M., Self-­control in decision-m­ aking involves mod- Adams, C. M., & Hommer, D. (2003). A ulation of the VMPFC valuation system. Sci- region of mesial prefrontal cortex tracks mon- ence, 324, 646–648. etarily rewarding outcomes: Characterization with rapid event-­related fMRI. NeuroImage, Hariri, A. R., Brown, S. M., Williamson, D. 18, 263–272. E., Flory, J. D., de Wit, H., & Manuck, S. B. Koob, G. F., & Bloom, F. E. (1988). Cellular and (2006). Preference for immediate over delayed molecular mechanisms of drug dependence. rewards is associated with magnitude of ven- Science, 242, 715–723. tral striatal activity. Journal of Neuroscience, Kroger, J. K., Sabb, F. W., Fales, C. L., 26, 13213–13217. Bookheimer, S. Y., Cohen, M. S., & Holyoak, K. J. (2002). Recruitment of anterior dorso- Hershberger, W. A. (1986). An approach through lateral prefrontal cortex in human reasoning: the looking glass. Animal Learning and A parametric study of relational complexity. Behavior, 14, 443–451. Cerebral Cortex, 12, 477–485. LaBar, K. S., Gatenby, J. C., Gore, J. C., LeDoux, Hoch, S. J., & Loewenstein, G. F. (1991). Time-­ J. E., & Phelps, E. A. (1998). Human amyg- inconsistent preferences and consumer self-­ dala activation during conditioned fear acqui- control. Journal of Consumer Research, 17, sition and extinction: A mixed-trial fMRI 492–507. study. Neuron, 20, 937–945. Laibson, D. I. (1997). Golden eggs and hyper- Hutcherson, C. A., Plassmann, H., Gross, J. J., bolic discounting. Quarterly Journal of Eco- & Rangel, A. (2012). Cognitive regulation nomics, 42, 861–871. during decision-­making shifts behavioral con- LeDoux, J. E. (2000). Emotion circuits in the trol between ventromedial and dorsolateral brain. Annual Review of Neuroscience, 23(1), prefrontal value systems. Journal of Neurosci- 155–184. ence, 32(39), 13543–13554. Levy, B. J., & Anderson, M. C. (2002). Inhibitory processes and the control of memory retrieval. Ikemoto, S., & Panksepp, J. (1999). The role of Trends in Cognitive Sciences, 6, 299–305. nucleus accumbens dopamine in motivated Li, X. (2008). The effects of appetitive stimuli on behavior: A unifying interpretation with out-of-domain consumption impatience. Jour- special reference to reward-­seeking. Brain nal of Consumer Research, 34, 649–656. Research Reviews, 31, 6–41. Lieberman, M. D., Gaunt, R., Gilbert, D. T., Trope, Y. (2002). Reflection and reflexion: Jog, M. S., Kubota, Y., Connolly, C. I., Hill- A social cognitive neuroscience approach to egaart, V., & Graybiel, A. M. (1999). Building attributional inference. Advances in Experi- neural representations of habits. Science, 286, mental Social Psychology, 34, 199–249. 1745–1749. Loewenstein, G. (1996). Out of control: Vis- ceral influences on behavior. Organizational Kable, J. W., & Glimcher, P. W. (2007). The neu- Behavior and Human Decision Processes, 65, ral correlates of subjective value during inter- 272–293. temporal choice. Nature Neuroscience, 10, Loewenstein, G., & Prelec, D. (1992). Anoma- 1625–1633. Kable, J. W., & Glimcher, P. W. (2009). The neu- robiology of decision: Consensus and contro- versy. Neuron, 63(6), 733–745. Kahneman, D. (2003). Maps of bounded ratio- nality: Psychology for behavioral economics. American Economic Review, 93, 1449–1475. Kerns, J. G., Cohen, J. D., MacDonald, A. W., Cho, R. Y., Stenger, V. A., & Carter, C. S.

Delay Discounting 109 lies in intertemporal choice: Evidence and an Pessiglione, M., Petrovic, P., Daunizeau, J., interpretation. Quarterly Journal of Econom- Palminteri, S., Dolan, R. J., & Frith, C. D. ics, 107, 573–597. (2008). Subliminal instrumental conditioning MacDonald, A. W., Cohen, J. D., Stenger, V. demonstrated in the human brain. Neuron, A., & Carter, C. S. (2000). Dissociating the 59, 561–567. role of the dorsolateral prefrontal and anterior cingulate cortex in cognitive control. Science, Peters, J., & Büchel, C. (2010). Episodic future 288, 1835–1838. thinking reduces reward delay discount- MacLean, P. D. (1990). The triune brain in evo- ing through an enhancement of prefrontal–Â

110 COGNITIVE APPROACHES Read, D., Frederick, S., Orsel, B., & Rahman, J. dynamic utility maximization. Review of Eco- (2005). Four Score and Seven Years from Now: nomic Studies, 23, 165–180. The date/delay effect in temporal discounting. Stuss, D. T., & Levine, B. (2002). Adult clinical Management Science, 51(9), 1326–1335. neuropsychology: Lessons from studies of the frontal lobes. Annual Review in Psychology, Redish, A. D. (2004). Addiction as a computa- 53, 401–433. tional process gone awry. Science, 306, 1944– Thaler, R. H. (1981). Some empirical evidence on 1947. time inconsistency. Review of Economic Stud- ies, 23, 165–180. Ridderinkhof, K. R. (2004). The role of the Thaler, R. H., & Benartzi, S. (2004). Save More medial frontal cortex in cognitive control. Sci- TomorrowTM: Using behavioral economics to ence, 306(5695), 443–447. increase employee saving. Journal of Political Economy, 112, S164–S187. Sanfey, A. G., Rilling, J. K., Aronson, J. A., Nys- Thaler, R. H., & Shefrin, H. (1981). An eco- trom, L. E., & Cohen, J. D. (2003). The neural nomic theory of self-­control. Journal of Politi- basis of economic decision-m­ aking in the ulti- cal Economy, 89, 392–406. matum game. Science, 300, 1755–1758. Trope, Y., & Liberman, N. (2003). Temporal con- strual. Psychological Review, 110, 403–421. Schacter, D. L., Addis, D. R., & Buckner, R. L. van den Bos, W., & McClure, S. M. (2013). (2008). Remembering the past to imagine the Towards a general model of delay discount- future: The prospective brain. Nature Reviews ing. Journal of the Experimental Analysis of Neuroscience, 8, 657–661. Behavior, 99, 58–73. Wang, X. T., & Dvorak, R. D. (2010). Sweet Schultz, W., Dayan, P., & Montague, P. R. future: Fluctuating blood glucose levels affect (1997). A neural substrate of prediction and future discounting. Psychological Science, 21, reward. Science, 275, 1593–1599. 183–188. Whalen, P. J., Rauch, S. L., Etcoff, N. L., McIn- Scoville, W. B., & Milner, B. (1957). Loss of erney, S. C., Lee, M. B., & Jenike, M. A. recent memory after bilateral hippocampal (1998). Masked presentations of emotional lesions. Journal of Neurology, Neurosurgery, facial expressions modulate amygdala activity and Psychiatry, 20, 11–21. without explicit knowledge. Journal of Neuro- science, 18, 411–418. Shallice, T., & Burgess, P. W. (1991). Deficits Wicker, B., Keysers, C., Plailly, J., Royet, J. P., in strategy application following frontal lobe Gallese, V., & Rizzolatti, G. (2003). Both of damage in man. Brain, 114, 727–741. us disgusted in my insula: The common neu- ral basis of seeing and feeling disgust. Neuron, Shamosh, N. A., DeYoung, C. G., Green, A. E., 40, 655–664. Reis, D. L., Johnson, M. R., Conway, A. R. A., Wise, R. A. (1982). Neuroleptics and operant et al. (2008). Individual differences in delay behavior: The anhedonia hypothesis. Behav- discounting: Relation to intelligence, working ioral and Brain Sciences, 5, 39–87. memory, and anterior prefrontal cortex. Psy- Yeung, N., Botvinick, M. M., & Cohen, J. D. chological Science, 19, 904–911. (2004). The neural basis of error detection: Conflict monitoring and the error-­related Shiffrin, R. M., & Snyder, W. (1977). Controlled negativity. Psychological Review, 111(4), 931– and automatic processing: II. Perceptual learn- 959. ing, automatic attending, and a general theory. Psychological Review, 84, 127–190. Shizgal, P., & Conover, K. (1996). On the neural computation of utility. Current Directions in Psychological Science, 5, 37–43. Simon, H. A. (1959). Theories of decision-­ making in economics and behavioral science. American Economic Review, 49, 253–283. Strotz, R. (1956). Myopia and inconsistency in

Chapter 7 The Role of Emotion and Emotion Regulation in the Ability to Delay Gratification Anna Luerssen Ozlem Ayduk You would be hard pressed to find a psy- ing rewards (e.g., Green, Fry, & Myerson, chologist, professional or budding, who is 1994). The bulk of this work, however, has not familiar with research on delay of grati- most often described and discussed DG as a fication (DG)—the ability to put off a small, form of behavioral self-c­ ontrol and focused but immediate, reward in favor of receiving on its cognitive and attentional underpin- a larger reward later (e.g., Mischel & Ayduk, nings. 2011). In fact, more and more, this work is being written about in the popular media More recently, there has been a growing (e.g., Brooks, 2006; Lehrer, 2009), and thus body of research linking DG to affective likely discussed in nonacademic circles, such processes—e­motional experience and emo- as parents, educators, and clinicians. What tion regulation. In this chapter, we examine makes DG, as a topic, so interesting? This these findings. To do so, we first provide fascination is likely due to the longitudinal a brief overview of DG, including its mea- studies linking performance on the classic surement across different age groups and a DG paradigm, during which a child tries description of the most studied mechanism to wait for two treats instead of having one thought to underlie performance. Subse- right away, to a variety of critical outcomes quently, we review four bodies of research in later life, including drug use, social com- on DG and emotion that in many ways over- petence, and even Scholastic Aptitude Test lap. First, we discuss research on the devel- (SAT) scores (e.g., Mischel, Shoda, & Rodri- opmental precursors of DG ability, focus- guez, 1989). ing on how caregivers’ emotion expression and responsivity to child distress impact the Although this early longitudinal research development of this competency in young popularized DG for the general public, children (e.g., Rodriguez et al., 2005). Sec- researchers have certainly expanded beyond ond, we describe how an individual’s own it. For example, research has linked DG to affective state impacts DG—more particu- other forms of cognitive control (e.g., Casey larly, the damaging effect of negative emo- et al., 2011; Eigsti et al., 2006), identified tion on one’s ability to put off long-term mechanisms that underlie DG ability (e.g., rewards (e.g., Tice, Baumeister, & Zhang, Rodriguez, Mischel, & Shoda, 1989), and 2004). Third, we draw on literature sug- examined how these processes play out in gesting that DG ability may serve an impor- adults, with whom treats are no longer entic- tant role in the regulation of emotion—­as 111

112 COGNITIVE APPROACHES a resource for individuals to utilize in emo- rary researchers using the classic paradigm tionally taxing situations (e.g., Ayduk et al., often measure both wait time and attention 2000). Finally, we describe neuroimaging deployment in their evaluation of DG per- research that has provided clues about the formance. links between affective processes and DG (e.g., McClure, Laibson, Loewenstein, & Although the classic paradigm is the most Cohen, 2004). studied DG measure, it was designed for pre- school children and expanded for children Review of DG up to 11 years of age (Rodriguez et al., 1989). Early research on DG was conducted pri- Prior to preschool (before approximately age marily with preschool-­age children using 4), children typically have neither the cogni- what has become the classic DG paradigm, tive capacity to understand the contingencies referred to fondly as the “marshmallow of the task nor the ability to wait extended task” (see Mischel & Ayduk, 2011, for periods of time alone. Alternatively, for ado- review). First, the experimenter asks the lescents and adults, cookies and marshmal- child whether they would prefer a smaller lows may not be tempting in the way they or larger reward (e.g., one marshmallow or are for children. Moreover, the experimental two marshmallows). Most children prefer situation with a capped wait time may not the larger option, and once they say so, the tax adults’ self-r­egulatory competencies suf- experimenter goes on to explain the task con- ficiently. For these reasons the classic task is tingencies. If, indeed, the child would like to unlikely to be diagnostic of individual differ- have the larger reward, he or she must wait ences in DG ability after childhood. Given until the experimenter returns to the room, that researchers have wanted to study DG in ostensibly after setting up another task. The other age groups, they developed a variety of child can have the smaller reward anytime, tasks better suited to these populations. however, by ringing a bell and bringing the experimenter back. The experimenter leaves Researchers evaluating DG ability in very the child alone with their options sitting on young children often employ variations of a plate in front of them (e.g., one marshmal- either the gift delay or snack delay tasks, low on one side of the plate, two marshmal- both developed by Kochanska and Knaack lows on the other side) and the amount of (2003). These tasks require the child to time the child can wait is recorded—­the pri- exert impulse control before obtaining a mary measure of DG ability. desired reward. In a standard gift delay task, for example, the experimenter pres- Longer wait-time scores on this task have ents a wrapped gift to the child, then has to been linked to varied long-term outcomes leave the room and asks the child to wait to such as higher SAT scores, lower drug use, open it until he or she returns. The child’s and better social-­cognitive competence behavior is scored with codes ranging from (e.g., Mischel et al., 1989; Shoda, Mischel, not touching or peeking inside the bag to & Peake, 1990). As such, researchers have opening the gift completely. In a typical sought to understand what mechanisms snack delay task, children have to wait for underlie the ability of children to wait (e.g., the experimenter to ring a bell before tak- Rodriguez et al., 1989). Experimental and ing M&M’s “hidden” under a clear cup. individual-d­ifferences research has found There are a series of trials that vary in the that attention deployment during the wait- time delay before the experimenter rings the ing period is an important predictor of wait bell (e.g., 10–30 seconds). Halfway through time. The more time a child spends looking the delay, the experimenter lifts the cup but at the rewards and the bell (i.e., hot-f­ocused does not ring the bell. The child’s behavior attention), the harder it is for him or her to is observed and coded as waiting, touching, wait. Alternatively, the more time a child or eating the M&M’s before the bell rings, spends looking away from the rewards and with the assumption that these behaviors bell and instead focuses on other things in reflect decreasing levels of DG ability. the room (i.e., cool-f­ocused attention), the easier it is to wait. In this way, contempo- Choice delay tasks are used with adoles- cents and adults, though many studies also employ these tasks with children. In a stan- dard choice delay task, the participant is asked to make a series of choices between

Delay and Emotion 113 a smaller, immediate reward and a larger, performance on age-­appropriate DG tasks, delayed reward (e.g., $5 today vs. $10 in 1 including the snack and gift delay tasks. week) (e.g., Seeman & Schwarz, 1974; Wert- Children of parents who displayed lower lev- heim & Schwarz, 1983). The sizes of the els of positive affect were worse at DG 1 year small and large options, as well as the delay later. In a similar study, maternal responsive- intervals, are varied between trials. In some ness to a child’s verbal communications was studies, researchers calculate participants’ coded during home visits when the child was temporal or delay discounting rate—how age 24 months. The more responsive his or quickly larger rewards become unappeal- her mother was, the more patient and task-­ ing when a delay is required to obtain them. oriented the child was on a gift delay task For more on choice delay tasks, see Miller, 1 year later (Olson, Bates, & Bayles, 1990). Rodriguez, Kim, and McClure (this volume). Houck and Lecuyer-­Maus (2004) had While we do not claim that these tasks similar results when evaluating the associa- perfectly parallel the classic paradigm, in tion between maternal responsivity and DG this review we include research using these behavior in a narrower context—d­uring measures, with the assumption that there is child limit setting. During laboratory ses- at least a moderate degree of overlap in the sions at child ages 12, 24, and 36 months, psychological conditions between them. In parents were asked to prevent their child summary, researchers have developed a vari- from touching and playing with an attractive ety of tasks to evaluate DG ability. In child- toy. The researchers coded four limit-s­ etting hood, the classic paradigm or “marshmal- approaches, which varied in the degree of low task” is the predominant measure, with parent responsivity to the child’s emotion. both wait time and attention deployment Children of parents who predominantly scored. Other measures have been developed used a power-based limit-s­etting approach, for young children and for use with adoles- characterized by little responsivity and high cents and adults. In the subsequent review, negative affect, waited the least amount of we link performance on these measures to time during the DG task, significantly less emotion and emotion regulation. than children whose parents’ approaches were more responsive to the child’s feelings. Caregiver Emotion and Responsivity Given the connection between DG perfor- Rodriguez and colleagues (2005) also mance and such varied and critical long- evaluated the role of maternal responsiv- term outcomes, the question of how this ity in their study linking behaviors during ability develops becomes important. Of par- the Strange Situation paradigm (used to ticular relevance for our discussion of DG measure caregiver–c­hild attachment) when and emotion is whether there is an associa- the child was 18 months old to DG ability tion between affective environments in early at age 4 (Rodriguez et al., 2005). Mater- childhood and the successful development of nal responsiveness during the free-play and DG ability later on. reunion episodes was coded using facial and vocal expression, position and body contact, Researchers have found that caregiver and expression of affection, among other variables, including caregiver emotion and variables. In addition, during the reunion caregiver responsivity to children’s emotion, episode, the researchers identified instances are important correlates of subsequent DG when a mother disengaged from her child development. These variables are often inter- specifically following the child’s expression related or measured interchangeably—w­ ith of negative affect. Children of unresponsive the nature of caregivers’ expressed emotion and disengaged mothers waited marginally serving as a common signal of whether the less time for a larger reward, and spent more caregiver is responsive to the child’s emo- time directing their attention toward the tion. In one such study, Eiden, Edwards, and rewards and bell during the waiting period, Leonard (2007) coded parental affect from which, again, is an important mechanism videos of the parent and child interacting underlying DG ability (Rodriguez et al., at home when the child was age 2. DG was 1989). measured 1 year later with a composite of In fact, children’s attachment style with their caregivers, a more general reflection of caregiver emotion and responsivity (Shaver

114 COGNITIVE APPROACHES & Mikulincer, 2011), has also been cor- the DG task, including the snack and gift related with DG performance. For exam- delay tasks. ple, in one study, children who were more securely attached to their caregivers waited Collectively, these studies suggest that the longest on the classic DG task (Jacob- caregiver emotion and responsivity are asso- sen, Huss, Fendrich, Kruesi, & Ziegenhain, ciated with the development of DG ability 1997). The greatest difference in wait time in childhood. But why exactly is a home life was found between children who displayed characterized by caregiver positive affect, a secure attachment pattern and those who limited hostility, and clear awareness and displayed a disorganized attachment pat- sensitivity to a child’s emotion related to tern (characterized by fearful and disorga- superior DG? How do these affective vari- nized responses during the reunion phase). ables relate to the development of the ability For more on attachment and emotion, see to put off immediate gratification in order Shaver and Mikulincer (this volume). to achieve long-term goals? Though there have been little to no studies exploring this A variety of studies have corroborated question of mechanism, researchers have put the finding that a caregiver’s hostile, over- forth a few hypotheses, as described below. controlling behavior is associated with poor DG ability. Jacobsen (1998) coded expressed Children’s negative affect, emerging emotion during a 5-minute speech in which in the context of stress or frustration, is mothers were asked to describe what their likely short-c­ircuited by responsive caregiv- child was like. Children of mothers who ers. These caregivers attend to their child’s expressed criticism (resentment of the child’s emotional needs and may provide support, behavior or characteristics) waited less time for example, through soothing or distract- on the classic DG paradigm. In another ing behavior (e.g., Garner, 2006; Jahromi study, researchers coded caregivers’ behav- & Stifter, 2007). This external source of ior for directives (commands) during a free- emotion regulation may serve dual func- play time, along with take-overs (completing tions. In the immediate, it helps to mitigate the game themselves) and contingent posi- children’s negative affective states. As fur- tive affect (responding to the child’s correct ther described in the next section, negative response with affect signaling approval) dur- emotion can destabilize DG (e.g., Mischel, ing a teaching–­learning task (Silverman & Ebbesen, & Zeiss, 1972). Ippolito, 1995). Children of mothers who used directives and engaged in take-overs At a more global level, the external forms performed worse on age-a­ppropriate ver- of regulation provided by responsive care- sions of the DG task, including the snack givers serve an important modeling func- and gift delay tasks, 6 months later. Chil- tion. Children of responsive caregivers may dren whose mothers engaged in contingent have more opportunities to observe and positive affect were better at DG. learn successful emotion regulation strate- gies that they themselves can employ at an Other studies have provided evidence that appropriate age (e.g., Eisenberg, Fabes, & a caregiver’s approach to emotion more gen- Murphy, 1996; Garner, 2006). These skills erally relates to DG development. Brophy- may be useful for successful DG, which is Herb, Stansbury, Bocknek, and Horodyn- in itself a frustrating task. In contrast, chil- ski (2011) evaluated the relation between dren who lack such regulatory models may children’s DG and a variety of parental not develop these skills, therefore becoming emotion-r­elated socialization behaviors more vulnerable to DG failure. (ERSBs)—parenting behaviors thought to be critical for a child’s socioemotional devel- Caregivers’ own affective profiles may opment. ERSBs were scored during parent–­ serve to intensify these processes. Whereas child interactions and with questionnaires. warm caregivers who score high in posi- These included positive emotional respon- tive affect likewise engender positive affect sivity and general expressivity in the home, in their children, caregivers who score high among other measures. Higher scores on in negative affect regularly respond to their the ERSB composite factor (more emotion-­ children’s emotions and behaviors in nega- related socialization) positively predicted tive ways, thereby maintaining or even performance on age-a­ ppropriate versions of escalating their children’s negative affect (Malatesta, Culver, Tesman, & Shepard, 1989). Over time, a caregiver’s character-

Delay and Emotion 115 istic affective tone serves as a model for the classic DG task will help the child wait the development of a child’s own affective for the larger reward (Mischel et al., 1972). profile—­teaching the child which emotional Indeed, children directed to think “fun” responses are appropriate across a range of thoughts during the task (e.g., playing with situations (e.g., Eisenberg et al., 1999, 2003; toys) did wait longer. However, the con- Garner, 1995). Children with more chronic tent of the distraction was found to matter. negative affect may be regularly diverting Directing children to think “sad” thoughts resources from DG goals, toward the regula- (e.g., crying with no one to help you) actu- tion of their negative affect. ally hurt wait time. Children performed at the same poor level when directed to think In summary, warm and responsive care- sad thoughts, as when they were directed to givers may raise children with high posi- focus their attention on the rewards them- tive affect and low negative affect, who are selves, that is, hot attentional focus. armed with a host of regulation strategies, whereas cold and distant caregivers may Similar results have been found with raise children with low positive affect and choice delay tasks (e.g., Schwarz & Pollack, high negative affect, who have fewer regu- 1977). For example, in one study, 9-year-old latory strategies. Future research should participants drew a picture, and then went evaluate these hypotheses more directly, for through either a positive affect induction—­ example, by linking parental emotion and receiving feedback that the drawing was regulatory profiles to the types of affective good and would be featured in an art reactions and regulatory strategies employed exhibit, or a negative affect induction—­ by children during the DG task. For more on receiving feedback that the drawing was the role of family on emotion, see Thompson not good and would not be featured (See- (this volume). man & Schwarz, 1974). Subsequently, they were asked a series of choice delay questions. Affective State and DG Participants in the negative affect condition Does an individual’s prior affective state chose large, delayed rewards significantly impact his or her DG ability? For example, less often than participants in the positive how might children perform if they started affect condition. In a similar study, partici- the classic DG task right after learning that pants, ages 3 to 5 years, directed to think they received a failing grade on a test? What sad thoughts were more likely to choose an if they had earned the top score in their immediate, mediocre reward over a desir- class? As briefly mentioned in the previous able, delayed reward than were children in section, researchers have found that negative either neutral or positive affect conditions affect, in particular, is associated with poor (Moore, Clyburn, & Underwood, 1976). performance on both laboratory measures Similar results have been found with adults of DG and during real-world DG scenarios, (Gray, 1999). In one study, participants in such as dietary restraint. The evidence on a negative affective state (exposed to aver- positive affect is more mixed: Some studies sive pictures; in a state of heightened stress indicate that positive affect helps DG; others about upcoming midterms) showed a prefer- find that it hurts DG; and still others find ence for short-term gains at the expense of that it does not exert an impact all that dif- long-term gains, thus performing worse on ferent from neutral emotional states. the task overall. Negative Affect Individual-­difference studies have cor- Analyses conducted with various laboratory roborated this body of experimental work. measures have collectively supported the For example, developmental researchers prediction that negative emotion is associ- have argued that temperament (biologically ated with poor DG performance. In an early based personality differences; see Rothbart, study, researchers hypothesized that behav- Sheese, & Posner, 2007, for review) is com- iors, or even thoughts, that successfully dis- prised of three factors: arousal, emotion, and tract the child from the frustrating nature of self-r­ egulation. These researchers found that temperamental levels of negative affectivity are inversely related to temperamental levels of self-r­egulation (Derryberry & Rothbart, 1988). Relatedly, in studies with toddlers,

116 COGNITIVE APPROACHES more frequent negative affect during the plete an ostensible taste test study in which separation (Rodriguez et al., 2005; Sethi, they tried different crackers and rated their Mischel, Aber, Shoda, & Rodriguez, 2000) flavor preferences. Among restrained eat- and reunion (Rodriguez et al., 2005) phases ers, participants who received failure feed- of the Strange Situation procedure is associ- back ate significantly more crackers than ated with more hot attentional focus (look- those who received success feedback. In ing toward the rewards and bell) during the contrast, nonrestrained eaters ate margin- classic DG task at ages 4 and 5. Moreover, ally fewer crackers after failure feedback in a choice delay task, mild and moderately than after success feedback. This pattern depressed adults were more likely to choose of results held for both obese and nonobese immediate, less desirable rewards over large, participants. The link between negative delayed rewards than individuals who were affect and weakened dietary restraint was not depressed (Wertheim & Schwarz, 1983). also found among participants experiencing clinical depression (characterized by strong Similar effects have been found in more negative affect; Polivy & Herman, 1976). real-world DG scenarios. For example, In this study, restrained eaters reported a dietary restraint, like laboratory measures of considerable weight gain since the onset of DG, requires an individual to put off short- their depression (~6 lbs on average), while term gratification (e.g., the pleasure derived nonrestrained eaters reported a considerable from eating delicious foods), for the sake of weight loss (~5 lbs on average), with the dif- a long-term goal (e.g., losing weight, being ference between them highly significant. healthy). Because restrained eaters are con- stantly trying to delay gratification to obtain Collectively, both experimental and a thinner and healthier body, how their eating individual-­difference studies, involving changes in response to their current affective classic laboratory measures and more eco- state can inform our understanding of how logically valid study designs, suggest that affective states impact DG performance more negative affect is associated with worse DG generally. According to the restraint theory performance. A variety of theories have been of Herman and Polivy (1975), negative affect posited to explain this association. The most increases eating in restrained eaters (the rea- common account argues that negative affec- sons for which we evaluate below). tive states are incredibly distressing, and as such, relief from them may take precedence In an experimental evaluation of this over long-term goals (e.g., Heatherton & prediction, Frost, Goolkasian, Ely, and Baumeister, 1991). Energy or resources ini- Blanchard (1982) had participants, who var- tially directed toward putting off immedi- ied on degree of dietary restraint, go through ate gratification may be diverted and used either a negative, positive, or neutral affect toward emotion regulation instead. With induction. During this affect induction, a fewer resources aimed specifically at the DG bowl of M&M’s was placed in front of par- attempt, the ability to wait thus suffers (see ticipants, and they were told to eat freely Baumeister, Vohs, & Tice, 2007, for review). from the bowl. Restrained eaters in the nega- For example, restrained eaters, when put into tive affect condition ate more M&M’s than a negative affective state, may need to divert nonrestrained eaters in the same condition, the self-­regulatory resources they typically and ate more than restrained eaters in both use to restrict their caloric intake, to regu- the positive and neutral affect conditions. late their emotions, resulting in more eating. Though there was not a significant differ- Note, however, that some researchers have ence in the eating behavior of nonrestrained argued that individuals who report dieting eaters across conditions, they appeared to eat may in fact be worse at self-r­egulation more the least when in a negative affective state. generally (see Pratt & Wardle, 2012, for review). Baucom and Aiken (1981) had similar results. Their participants, obese and non- Moreover, in some DG scenarios a proxi- obese, restrained and nonrestrained eaters, mal source of affect relief is available—­the were asked to solve a task and received either smaller, but immediately available, option. success feedback (a positive affect induction) For example, a child might experience a or failure feedback (a negative affect induc- reduction in his or her sad feelings as a tion). Participants were then asked to com-

Delay and Emotion 117 result of thinking sad thoughts by enjoy- often than those in the neutral affect condi- ing an immediately available, albeit small, tion, though these results were not statisti- treat. At a more global level, this suggests cally significant (Moore et al., 1976). Stud- that negative affect may change the subjec- ies have also shown that putting restrained tive reinforcement values of immediate and eaters in a positive or neutral affective state, delayed rewards (e.g., Baucom & Aiken, in comparison to a negative affective state, is 1981; Schwarz & Pollack, 1977; Seeman & associated with greater DG ability (Baucom Schwarz, 1974). For individuals in a negative & Aiken, 1981; Frost et al., 1982). Positive affective state, the value of the more immedi- affective states did not consistently improve ate option, though objectively smaller, may DG performance more than neutral states, increase, because it has the potential to ame- however (Frost et al., 1982). liorate negative affect. On the other hand, the value of the long-term goal, or delayed In summary, the majority of evidence sug- option, is likely to decrease, because it serves gests that being in a positive affective state less of a purpose in removing the negative does not hurt DG the way being in a nega- affect and may even exacerbate it by adding tive affective state does, and in some cases it additional frustration. may even help DG. Although some research- ers’ findings implied that there are specific Positive Affect conditions under which positive affect may Although the link between DG and nega- thwart DG, we suspect that positive affect tive affect seems well established, the find- is likely to serve a facilitative function under ings on positive affect are more mixed. Some most conditions. This assumption is con- researchers find that, like negative affect, sistent with studies linking positive affect positive affect is associated with poor DG to improved self-r­egulation more generally performance. For example, Cools, Schotte, (e.g., Shmueli & Prochaska, 2012; Tice, and McNally (1992) exposed participants to Baumeister, Shmueli, & Muraven, 2007; a neutral film, a comedy (i.e., positive affect Tice et al., 2004). induction), or a horror film (i.e., negative affect induction). Participants were given a Although there is need for more research bag of popcorn during the movie and told to firmly establish the facilitative impact of to eat as much as they liked. In the neutral positive affect on DG, what mechanisms condition, participants higher in dietary might explain such an effect? As previously restraint ate less popcorn than those lower described, positive affect has been found in dietary restraint. In both the comedy to restore self-­regulatory resources (Tice and horror film conditions, however, par- et al., 2007), suggesting that individuals ticipants higher in dietary restraint ate more in a positive affective state may have addi- popcorn. tional resources available for DG. More- over, research in support of Fredrickson’s In many other studies, however, partici- broaden-a­nd-build theory of positive emo- pants in a positive affective state performed tion, has found that positive affect increases better at DG tasks than those in a negative one’s attentional focus, which may help affective state. As previously described, chil- transcend the here and now, and thus DG dren who received positive feedback regard- (see Fredrickson, 2001, for review; cf. Gable ing the quality of their drawings chose & Harmon-Jones, 2011). Relatedly, positive larger, delayed rewards more often than affect appears to mitigate the adverse effects those who received negative feedback (See- of negative affect (e.g., Fredrickson, 2001; man & Schwarz, 1974). Similarly, children Fredrickson & Levenson, 1998), and per- completing the classic DG task performed haps by extension the damaging effect nega- better when asked to think fun, distract- tive affect has on DG performance. Finally, ing thoughts than when they were directed it is possible that positive affect has informa- to think about the rewards (Mischel et al., tional value—affirming that things are safe 1972). On a choice delay task, children or certain (see Schwarz, 2011, for review), in the positive affect condition tended to which may lead individuals to be more will- choose larger, delayed rewards even more ing to wait for the delayed rewards during a DG task. Future work should examine these hypotheses directly.

118 COGNITIVE APPROACHES Emotion Regulation and DG not regulating their negative emotion; rather Since the initial studies, researchers have they were less emotionally reactive—a­nd readily assumed that DG tasks induce emo- this is why they displayed fewer expressions tion, and thus by extension, require emotion of anger and sadness. These possibilities regulation—t­ he process of changing the way need to be disentangled in future research. we think, feel, and express emotions (e.g., Gross & Thompson, 2007). Does the way Although research that examines emotion an individual deals with these task-i­nduced regulation during the DG task is scarce, a emotions impact his or her ability to wait for growing body of work shows that individu- the larger reward? What about the reverse? als strong in DG ability are better able to Is there an association between an individu- regulate their emotions across a variety of al’s DG ability and how adept he or she is at situations. In fact, this literature suggests regulating emotions? Indeed, a large body of that individuals at risk for emotion regu- research demonstrates that one’s DG ability latory difficulties may be protected from is predictive of emotion regulation behaviors negative life outcomes if they are also good in a variety of emotionally taxing situations. at DG. To illustrate, in one set of studies, researchers evaluated the link between DG Whether, and how, children regulate their and emotion regulation outcomes in people emotions during DG tasks likely impacts per- with high rejection sensitivity (RS; Ayduk et formance. Here we refer to emotions stimu- al., 2000). These are individuals who anx- lated by the task itself, above and beyond iously expect, readily perceive, and overreact affective states that precede task onset (as to social rejection (e.g., Downey & Feldman, described in the previous section). For exam- 1996). Prior research indicates that individu- ple, if a child feels frustration and anger that als with high RS respond to perceived social he or she cannot have the two marshmallows rejection with dysregulated emotion, includ- immediately, does down-­regulating negative ing jealousy (Downey & Feldman, 1996), affect help or hurt efforts to wait? On the depressed affect (Ayduk, Downey, & Kim, flip side, if a child’s predominant affective 2001), and hostility directed toward the reaction to the DG situation is eagerness and source of rejection (Ayduk, Downey, Testa, excitement, does down-r­ egulating such posi- Yen, & Shoda, 1999). tive emotions facilitate or undermine DG? Ayduk and colleagues (2000) hypoth- Unfortunately, there has been little to no esized that individuals with high RS and research that provides clear-cut answers strong DG ability would be better able to to these questions. This shortage may be focus on their longer-­term goals, such as due, in part, to the difficulty of disentan- relationship maintenance, during perceived gling emotional reactivity during DG tasks rejection, and thus be more likely to regu- (e.g., how frustrated–­excited the individual late their reactive emotional responses. In feels) and emotion regulation (e.g., how support of this prediction, inner-city middle well an individual can regulate frustration–­ school students with high RS were rated by excitement if it indeed emerges). One of the their teachers as more aggressive and less few studies evaluating emotion during DG socially accepted than those with low RS, demonstrates this conundrum. Researchers but only if they had poor DG ability, as found that children, ages 4–7, who expressed measured with the classic paradigm 2 years more anger and sadness during three age-­ earlier. In fact, participants with high RS appropriate DG tasks, including the gift and good DG were rated as marginally less delay task and a modified version of the clas- aggressive and better liked than their peers sic task, also focused their attention more with low RS. Correspondingly, in a sample on the rewards (an important mechanism of middle-a­ ged participants, those with high underlying poor DG performance; Santucci RS were rated by their parents as worse at et al., 2008). If more expressed negative emotion regulation than were those with low emotion reflects less emotion regulation, the RS (e.g., more likely to go to pieces under results would suggest that regulating nega- stress), but again, only if they performed tive emotion during the DG task should help poorly on the classic DG task as preschool- performance. It is also possible, however, ers (Ayduk et al., 2000). that children who performed better were A similar pattern of results was found with respect to borderline personality disor-

Delay and Emotion 119 der (BPD) symptoms (Ayduk et al., 2008). A etary reward available after a short, fixed core feature of this disorder is extreme affec- delay and a large monetary reward avail- tive lability that bears a resemblance to that able after a longer delay (Cherek, Moeller, of high RS, including intense hostility and Dougherty, & Rhoades, 1997). The number depression. Ayduk and colleagues hypoth- of times parolees chose the immediate, small esized, and found, that although there was reward was correlated with the number of a positive association between RS and BPD aggressive responses they made in another symptoms, this association was weaker in paradigm. In a follow-­up study, parolees individuals with good DG ability, as mea- who were imprisoned for violent crimes (e.g., sured with the classic DG task in childhood. aggravated assault, manslaughter) displayed a preference for small, immediate rewards A host of other studies corroborate these significantly more than parolees imprisoned findings—­that individuals with good DG for nonviolent crimes (e.g., theft, forgery) ability seem better able to regulate their (Cherek & Lane, 1999). emotions in a wide range of situations. For example, researchers have found that good Collectively these studies show that there DG ability is related to lower verbal and is a strong association between DG and emo- physical aggression (Mischel et al., 1989), tion regulation, and they may suggest that along with less cheating behavior (Mischel both forms of regulation are drawing on a & Gilligan, 1964). Another study found that domain-­general regulatory competence or adolescents rated by observers as less likely resource that can be utilized in a wide vari- to wait for a larger, but delayed, payment for ety of situations. In the next section we pres- study participation were also rated higher in ent evidence in support of this thesis with hostility (Funder & Block, 1989). Kochan- work evaluating the neural underpinnings of ska, Murray, and Harlan (2000) found that both emotion regulation and DG processes. among toddlers (i.e., 22–33 months olds), effortful control (of which DG is a com- Emotion Regulation and DG ponent) was related to better regulation of in the Brain anger, both concurrently and prospectively. Why does an individual’s DG ability relate to Relatedly, inhibitory control ability (similar how well they can regulate their emotions? to effortful control) among preschoolers, One theory suggests that DG and emotion in part measured by a gift delay task, posi- regulation are related because both reflect tively correlated with emotion regulation in an individual’s more general self-r­egulation controlling both positive and negative affect ability (e.g., Cohen & Lieberman, 2010; (Carlson & Wang, 2007). For more infor- Heatherton & Wagner, 2011)—how well mation on emotion regulation and effortful–­ the individual is able to overcome prepotent, inhibitory control, see Eisenberg, Hofer, automatic, or stimulus-d­riven responses Sulik, and Spinrad (this volume). (e.g., Metcalfe & Mischel, 1999). Accord- ing to this theory, the ability to overcome an Studies with clinical populations yield automatic response, be it to grab a marsh- similar results. In one study, early adoles- mallow (in the case of DG) or to become cents with externalizing disorder (character- angry and lash out (in the case of emotion ized by hostility and aggression) performed regulation), is served by a regulatory “con- worse on a modified DG task that involved trol center” that varies among individuals a series of choices between pressing a button in its efficiency or strength. Performance immediately, which provided a 40% chance on tasks diagnostic of self-­regulatory ability of winning a nickel, or waiting to press a in one domain (e.g., DG) are therefore pre- button, which provided an 80% chance of dictive of self-r­egulatory ability in another winning a nickel (Krueger, Caspi, Moffitt, domain (e.g., emotion regulation), in that White, & Stouthamer-L­ oeber, 1996). Math- both utilize and reflect the power of this ias and colleagues (2011) found that adoles- more general regulatory ability. cent girls’ preference for small, immediate rewards on a choice delay task was associ- This control center may emerge at a ated with higher self-r­eported depression, neural level, with common brain regions aggression, impulsivity, and suicide intent. or networks serving self-r­egulation of all In another study, male parolees were pre- sented with a choice between a small mon-

120 COGNITIVE APPROACHES types, including, but not limited to, DG naturalistic study, cigarette smokers viewed and emotion regulation. From this perspec- pictures of cigarettes known to induce crav- tive, individual differences in self-­regulation ings (Kober et al., 2010). During the NOW reflect the relative or effective recruitment of condition, participants were directed to these regions. Neuroimaging work support- focus on the immediate consequences of ing this thesis has found that dorsolateral consuming the substance (how great that (dlPFC) and ventrolateral (vlPFC) regions of cigarette would taste; i.e., immediate gratifi- the lateral prefrontal cortex (lPFC) are acti- cation), while during the LATER condition, vated during both DG and emotion regula- participants were directed to focus on the tion tasks (e.g., Cohen & Lieberman, 2010; long-term consequences (risk for heart dis- Kober et al., 2010; Ochsner, Bunge, Gross, ease; i.e., delayed gratification). There was & Gabrieli, 2002). This lPFC activation greater activation of the dlPFC and vlPFC is associated with the down-r­egulation of during LATER trials than during NOW tri- activity in subcortical regions, such as those als. Moreover, dlPFC activation was associ- implicated in emotional reactivity in the case ated with decreased activation in the ventral of emotion regulation (e.g., amygdala; Kober striatum. In fact, ventral striatal activation et al., 2010), and reward-r­ elated reactivity in mediated the relationship between dlPFC the case of DG (e.g., ventral striatum; Och- activation and self-r­eported craving. This sner, Bunge, Gross, & Gabrieli, 2002). suggests that the dlPFC may down-­regulate reward-­related activity in the ventral stria- For example, in one study participants pre- tum, which then may decrease the subjective sented with aversive pictures were directed craving experience. to regulate their negative emotion by either increasing or decreasing it (Ochsner et al., Together these studies show that the lPFC, 2004). When engaging in both forms of including both dorsolateral and ventrolat- emotion regulation, the dlPFC and vlPFC eral subregions, serve an important role became active. Moreover, up-r­egulation in both DG and emotion regulation tasks. of negative emotion was associated with These results suggest that the lPFC may increases in amygdala activation, while be a critical component of a general self-­ down-r­egulation of negative emotion was regulation control center, or network, in the associated with decreases in amygdala acti- brain. A recent review conducted by Cohen vation. In another study, participants viewed and Lieberman (2010) supports this theory. negative pictures and were instructed to cog- They found that vlPFC activity, in particu- nitively reappraise (i.e., emotion regulation), lar, was associated with self-­regulation in a or just attend to the pictures (Ochsner et al., variety of domains, including emotion regu- 2002). During reappraisal, the lPFC became lation and DG, as well as motor response more active and the amygdala became less inhibition, suppressing risky behavior, mem- active. In fact, there was an inverse corre- ory inhibition, and thought suppression. lation between vlPFC and amygdala activa- tion. These results suggest that, during reap- Imaging studies also have the poten- praisal, vlPFC may down-r­egulate activity tial to clarify the relative contributions of in the amygdala, a region, again, known for both emotional reactivity and regulation its role in emotion processing. For further to behavioral indices of DG. For example, review on the neural bases of emotion regu- Casey and colleagues (2011) found that lation, see Ochsner and Gross (this volume). participants with good DG ability showed greater right lPFC activation when inhibit- DG studies have similarly found the ing responses to appetitive stimuli (happy lPFC is important for self-­regulation. For faces). During these same trials, those with example, in choice delay tasks, both dlPFC good DG also showed less ventral striatum (Christakou, Brammer, & Rubia, 2011; activation. Importantly, there was no rela- McClure et al., 2004) and vlPFC (McClure tionship between the lPFC and the ventral et al., 2004) activation is associated with striatum, suggesting that activation in each delayed, rather than immediate, choices. region independently related to DG. These Additionally, participants who showed low results imply that DG ability may as much levels of dlPFC activation more quickly dis- reflect diminished emotional reactivity to counted delayed rewards (i.e., worse DG rewards, here social rewards in the form of performance; Hariri et al., 2006). In a more smiling faces, as regulatory control. Future

Delay and Emotion 121 work is needed, however, to more fully dis- suggest that performance on both emotion entangle how and when these regions work regulation and DG tasks may reflect an indi- independently versus inversely in predicting vidual’s domain-Âg

122 COGNITIVE APPROACHES mental research is needed to answer ques- Casey, B. J., Somerville, L. H., Gotlib, I. H., tions about the causal connection between Ayduk, O., Franklin, N. T., Askren, M. K., et emotion, emotion regulation, and DG. al. (2011). Behavioral and neural correlates of delay of gratification 40 years later. Proceed- References ings of the National Academy of Sciences USA, 108(36), 14998–15003. Ayduk, O., Downey, G., & Kim, M. (2001). Rejection sensitivity and depressive symptoms Cherek, D. R., & Lane, S. D. (1999). Laboratory in women. Personality and Social Psychology and psychometric measurements of impul- Bulletin, 27(7), 868–877. sivity among violent and nonviolent female parolees. Biological Psychiatry, 46(2), 273– Ayduk, O., Downey, G., Testa, A., Yen, Y., & 280. Shoda, Y. (1999). Does rejection elicit hostil- ity in rejection sensitive women? Social Cogni- Cherek, D. R., Moeller, F. G., Dougherty, D. tion, 17(2), 245–271. M., & Rhoades, H. (1997). Studies of violent and nonviolent male parolees: II. Laboratory Ayduk, O., Mendoza-D­ enton, R., Mischel, W., and psychometric measurements of impulsiv- Downey, G., Peake, P. K., & Rodriguez, M. ity. Biological Psychiatry, 41(5), 523–529. (2000). Regulating the interpersonal self: Stra- tegic self-r­egulation for coping with rejection Christakou, A., Brammer, M., & Rubia, K. sensitivity. Journal of Personality and Social (2011). Maturation of limbic corticostriatal Psychology, 79(5), 776–792. activation and connectivity associated with developmental changes in temporal discount- Ayduk, O., Zayas, V., Downey, G., Cole, A. B., ing. NeuroImage, 54(2), 1344–1354. Shoda, Y., & Mischel, W. (2008). Rejection sensitivity and executive control: Joint predic- Cohen, J. R., & Lieberman, M. D. (2010). The tors of borderline personality features. Journal common neural basis of exerting self-­control of Research in Personality, 42(1), 151–168. in multiple domains. In R. R. Hassin, K. N. Ochsner, Y. Trope, R. R. Hassin, K. N. Och- Baucom, D. H., & Aiken, P. A. (1981). Effect of sner, & Y. Trope (Eds.), Self control in society, depressed mood on eating among obese and mind, and brain (pp. 141–160). New York: nonobese dieting and nondieting persons. Oxford University Press. Journal of Personality and Social Psychology, 41(3), 577–585. Cools, J., Schotte, D. E., & McNally, R. J. (1992). Emotional arousal and overeating in Baumeister, R. F., Vohs, K. D., & Tice, D. M. restrained eaters. Journal of Abnormal Psy- (2007). The strength model of self-c­ontrol. chology, 101(2), 348–351. Current Directions in Psychological Science, 16(6), 351–355. Derryberry, D., & Rothbart, M. K. (1988). Arousal, affect, and attention as components Brooks, D. (2006, May 7). Marshmallows and of temperament. Journal of Personality and public policy. Retrieved from www.nytimes. Social Psychology, 55(6), 958–966. c o m / 2 0 0 6 / 05/ 0 7/o pi n io n / 0 7b rook s . ht ml?n =top%2fopinion%2feditorials%20and%20 Downey, G., & Feldman, S. I. (1996). Implica- op%2ded%2fop%2ded%2fcolumnists%2fda tions of rejection sensitivity for intimate rela- vid%20brooks. tionships. Journal of Personality and Social Psychology, 70(6), 1327–1343. Brophy-Herb, H. E., Stansbury, K., Bocknek, E., & Horodynski, M. A. (2011). Modeling Eiden, R. D., Edwards, E. P., & Leonard, K. maternal emotion-r­elated socialization behav- E. (2007). A conceptual model for the devel- iors in a low-­income sample: Relations with opment of externalizing behavior problems toddlers’ self-­regulation. Early Childhood among kindergarten children of alcoholic Research Quarterly, 27, 352–364. families: Role of parenting and children’s self-­ regulation. Developmental Psychology, 43(5), Carlson, S. M., & Wang, T. S. (2007). Inhibitory 1187–1201. control and emotion regulation in preschool children. Cognitive Development, 22(4), 489– Eigsti, I., Zayas, V., Mischel, W., Shoda, Y., 510. Ayduk, O., Dadlani, M. B., et al. (2006). Pre- dicting cognitive control from preschool to Casey, B. J., Galvan, A., & Hare, T. A. (2005). late adolescence and young adulthood. Psy- Changes in cerebral functional organization chological Science, 17(6), 478–484. during cognitive development. Current Opin- ion in Neurobiology, 15(2), 239–244. Eisenberg, N., Fabes, R. A., & Murphy, B. C. (1996). Parents’ reactions to children’s nega- tive emotions: Relations to children’s social

Delay and Emotion 123 competence and comforting behavior. Child counting of delayed rewards: A life-span com- Development, 67(5), 2227–2247. parison. Psychological Science, 5(1), 33–36. Eisenberg, N., Fabes, R. A., Shepard, S. A., Guth- Gross, J. J., & Thompson, R. A. (2007). Emotion rie, I. K., Murphy, B. C., & Reiser, M. (1999). regulation: Conceptual foundations. In J. J. Parental reactions to children’s negative emo- Gross (Ed.), Handbook of emotion regulation tions: Longitudinal relations to quality of chil- (pp. 3–24). New York: Guilford Press. dren’s social functioning. Child Development, Hariri, A. R., Brown, S. M., Williamson, D. 70(2), 513–534. E., Flory, J. D., de Wit, H., & Manuck, S. B. Eisenberg, N., Hofer, C., Sulik, J. & Spinrad, T. (2006). Preference for immediate over delayed L. (this volume, 2014). Self-­regulation, effort- rewards is associated with magnitude of ven- ful control, and their socioemotional corre- tral striatal activity. Journal of Neuroscience, lates. In J. J. Gross (Ed.), Handbook of emo- 26(51), 13213–13217. tion regulation (2nd ed., pp. 157–172). New Hariri, A. R., Mattay, V. S., Tessitore, A., Fera, York: Guilford Press. F., & Weinberger, D. R. (2003). Neocortical Eisenberg, N., Valiente, C., Morris, A. S., Fabes, modulation of the amygdala response to fear- R. A., Cumberland, A., Reiser, M., et al. ful stimuli. Biological Psychiatry, 53, 494– (2003). Longitudinal relations among parental 501. emotional expressivity, children’s regulation, Heatherton, T. F., & Baumeister, R. F. (1991). and quality of socioemotional functioning. Binge eating as escape from self-a­wareness. Developmental Psychology, 39(1), 3–19. Psychological Bulletin, 110(1), 86–108. Fredrickson, B. L. (2001). The role of positive Heatherton, T. F., & Wagner, D. D. (2011). Cog- emotions in positive psychology: The broaden-­ nitive neuroscience of self-r­egulation failure. and-build theory of positive emotions. Ameri- Trends in Cognitive Sciences, 15(3), 132–139. can Psychologist, 56(3), 218–226. Herman, C. P., & Polivy, J. (1975). Anxiety, Fredrickson, B. L., & Levenson, R. W. (1998). restraint, and eating behavior. Journal of Positive emotions speed recovery from the Abnormal Psychology, 84(6), 666–672. cardiovascular sequelae of negative emo- Houck, G. M., & Lecuyer-­Maus, E. A. (2004). tions. Cognition and Emotion, 12(2), 191– Maternal limit setting during toddlerhood, 220. delay of gratification, and behavior problems Frost, R. O., Goolkasian, G. A., Ely, R. J., & at age five. Infant Mental Health Journal, Blanchard, F. A. (1982). Depression, restraint, 25(1), 28–46. and eating behavior. Behaviour Research and Jacobsen, T. (1998). Delay behavior at age six: Therapy, 20(2), 113–121. Links to maternal expressed emotion. Journal Funder, D. C., & Block, J. (1989). The role of of Genetic Psychology, 159(1), 117–120. ego-­control, ego-r­esiliency, and IQ in delay of Jacobsen, T., Huss, M., Fendrich, M., Kruesi, M. gratification in adolescence. Journal of Per- J. P., & Ziegenhain, U. (1997). Children’s abil- sonality and Social Psychology, 57(6), 1041– ity to delay gratification: Longitudinal rela- 1050. tions to mother–c­hild attachment. Journal of Gable, P. A., & Harmon-Jones, E. (2011). Atten- Genetic Psychology, 158(4), 411–426. tional consequences of pregoal and postgoal Jahromi, L. B., & Stifter, C. A. (2007). Individ- positive affects. Emotion, 11(6), 1358–1367. ual differences in the contribution of maternal Garner, P. W. (1995). Toddlers’ emotion regula- soothing to infant distress reduction. Infancy, tion behaviors: The roles of social context and 11(3), 255–269. family expressiveness. Journal of Genetic Psy- Kidd, C., Palmeri, H., & Aslin, R. N. (2013). chology, 156(4), 417–430. Rational snacking: Young children’s decision-­ Garner, P. W. (2006). Prediction of prosocial and making on the marshmallow task is moder- emotional competence from maternal behav- ated by beliefs about environmental reliability. ior in African American preschoolers. Cul- Cognition, 126, 109–114. Kober, H., Mende-S­iedlecki, P., Kross, E. F., tural Diversity and Ethnic Minority Psychol- Weber, J., Mischel, W., Hart, C. L., et al. ogy, 12(2), 179–198. (2010). Prefrontal–s­triatal pathway underlies Gray, J. R. (1999). A bias toward short-term cognitive regulation of craving. Proceedings thinking in threat-­related negative emotional states. Personality and Social Psychology Bul- of the National Academy of Sciences USA, letin, 25(1), 65–75. 107(33), 14811–14816. Green, L., Fry, A. F., & Myerson, J. (1994). Dis- Kochanska, G., & Knaack, A. (2003). Effort-

124 COGNITIVE APPROACHES ful control as a personality characteristic of a cognitive-a­ffective processing system: The young children: Antecedents, correlates, and dynamics of delay of gratification. In K. D. consequences. Journal of Personality, 71(6), Vohs & R. F. Baumeister (Eds.), Handbook of 1087–1112. self-r­ egulation: Research, theory, and applica- Kochanska, G., Murray, K. T., & Harlan, E. T. tions (2nd ed., pp. 83–105). New York: Guil- (2000). Effortful control in early childhood: ford Press. Continuity and change, antecedents, and Mischel, W., Ebbesen, E., & Zeiss, A. (1972). implications for social development. Develop- Cognitive and attentional mechanisms in mental Psychology, 36(2), 220–232. delay of gratification. Journal of Personality Krueger, R. F., Caspi, A., Moffitt, T. E., White, and Social Psychology, 21(2), 204–218. J., & Stouthamer-­Loeber, M. (1996). Delay of Mischel, W., & Gilligan, C. (1964). Delay of grat- gratification, psychopathology, and personal- ification, motivation for the prohibited gratifi- ity: Is low self-­control specific to externaliz- cation, and responses to temptation. Journal ing problems? Journal of Personality, 64(1), of Abnormal and Social Psychology, 69(4), 107–129. 411– 417. Lehrer, J. (2009, May 18). Don’t!: The Mischel, W., Shoda, Y., & Rodriguez, M. L. secret of self-c­ontrol. The New Yorker. (1989). Delay of gratification in children. Sci- Retrieved from www.newyorker.com/ ence, 244(4907), 933–938. reporting/ 20 09/ 05/18/ 090518fa _ fact _ lehrer. Moore, B. S., Clyburn, A., & Underwood, B. Leroy, V., Grégoire, J., Magen, E., Gross, J. J., (1976). The role of affect in delay of gratifica- & Mikolajczak, M. (2012). Resisting the tion. Child Development, 47(1), 273–276. sirens of temptation while studying: Using Ochsner, K. N., Bunge, S. A., Gross, J. J., & reappraisal to increase focus, enthusiasm, and Gabrieli, J. D. (2002). Rethinking feelings: performance. Learning and Individual Differ- An fMRI study of the cognitive regulation of ences, 22(2), 263–268. emotion. Journal of Cognitive Neuroscience, Magen, E., & Gross, J. J. (2007). Harnessing 14(8), 1215–1229. the need for immediate gratification: Cogni- Ochsner, K. N., & Gross, J. J. (this volume, tive reconstrual modulates the reward value of 2014). The neural bases of emotion and emo- temptations. Emotion, 7(2), 415–428. tion regulation: A valuation perspective. In J. Malatesta, C. Z., Culver, C., Tesman, J. R., & J. Gross (Ed.), Handbook of emotion regula- Shepard, B. (1989). The development of emo- tion (2nd ed., pp. 23–42). New York: Guilford tion expression during the first two years of Press. life. Monographs of the Society for Research Ochsner, K. N., Ray, R. D., Cooper, J. C., Rob- in Child Development, 54(1–2), 1–104. ertson, E. R., Chopra, S., Gabrieli, J. D., et al. Mathias, C. W., Dougherty, D. M., James, L. (2004). For better or for worse: Neural sys- M., Richard, D. M., Dawes, M. A., Ache- tems supporting the cognitive down-and up-­ son, A., et al. (2011). Intolerance to delayed regulation of negative emotion. NeuroImage, reward in girls with multiple suicide attempts. 23, 483–499. Suicide and Life-­Threatening Behavior, 41(3), Olson, S. L., Bates, J. E., & Bayles, K. (1990). 277–286. Early antecedents of childhood impulsivity: McClure, S. M., Laibson, D. I., Loewenstein, G., The role of parent–­child interaction, cogni- & Cohen, J. D. (2004). Separate neural sys- tive competence, and temperament. Journal of tems value immediate and delayed monetary Abnormal Child Psychology, 18(3), 317–334. rewards. Science, 306(5695), 503–507. Polivy, J., & Herman, C. P. (1976). Clinical Metcalfe, J., & Mischel, W. (1999). A hot/ depression and weight change: A complex rela- cool-­system analysis of delay of gratifica- tion. Journal of Abnormal Psychology, 85(3), tion: Dynamics of willpower. Psychological 338–340. Review, 106(1), 3–19. Pratt, F. J., & Wardle, J. (2012). Dietary restraint Miller, E. M., Rodriguez, C., Kim, B., & and self-r­egulation in eating behavior. Inter- McClure, S. M. (this volume, 2014). Delay national Journal of Obesity, 36, 665–674. discounting: A two-­systems perspective. In J. Rodriguez, M., Ayduk, O., Aber, J. L., Mischel, J. Gross (Ed.), Handbook of emotion regula- W., Sethi, A., & Shoda, Y. (2005). A con- tion (2nd ed., pp. 93–110). New York: Guil- textual approach to the development of self-­ ford Press. regulatory competencies: The role of maternal Mischel, W., & Ayduk, O. (2011). Willpower in unresponsivity and toddlers’ negative affect

Delay and Emotion 125 in stressful situations. Social Development, Kruglanski, & E. T. Higgins (Eds.), Handbook 14(1), 136–157. of theories of social psychology (pp.€160–179). Rodriguez, M. L., Mischel, W., & Shoda, Y. Thousand Oaks, CA: Sage. (1989). Cognitive person variables in the delay Shmueli, D., & Prochaska, J. J. (2012). A test of gratification of older children at risk. Jour- of positive affect induction for countering self-cÂ

Chapter 8 Emotion Regulation Choice: Theory and Findings Gal Sheppes A few months ago, I had to take a long, tive lack of evidence on implicit emotion reg- 12-hour international flight in order to get ulation, I limit this chapter to deliberate and home. Having more than enough time to explicit forms of emotion regulation choice. spend, I started looking around me. While some people were sleeping (in extremely cre- Can we consistently predict the regulatory ative body postures), or watching movies choices individuals make to deal with their (with delight that they finally had enough emotions? Are there specific emotional, time to watch Tolkien’s full trilogy), others cognitive, and motivational influences that appeared quite alert and fearful, especially systematically affect these regulatory prefer- when the captain announced that the plane ences? What are the major factors underly- had started its final descent. While appear- ing the regulatory choice process? And what ing fearful, luckily these individuals were are the broad implications of regulatory able to choose between different regulatory choices? In this chapter I try to answer these options to control their fear. For example, important questions. Specifically, I begin by they could choose whether they want to deal introducing the importance of emotion reg- with their flight phobia by disengagement, ulation choice and its relative lack of empiri- closing their window shades while immers- cal investigation. I then introduce a recent ing themselves in unrelated yet demand- conceptual model to explain dominant emo- ing conversations. Alternatively, they could tional, cognitive, and motivational determi- choose to look out the window while nants, and underlying mechanisms of emo- explaining some facts, including a decline in tion regulation choice between dominant crash statistics since autopilots took charge. cognitive emotion regulation categories. I close by describing broad implications and Emotion regulatory choices are an inte- future directions. gral part of our daily lives in a dynamically changing affective world. By emotion regu- The Importance of Emotion lation choice I mean the choices individuals Regulation Choice make as to how they should regulate their The different ways individuals can go about emotions in a particular context when regu- controlling their emotions has attracted lation is warranted and more than one regu- scholars for centuries. Nevertheless, emo- latory option is active. Furthermore, given tion regulation has become an independent the abundance of knowledge on explicit emotion regulation strategies and the rela- 126

Emotion Regulation Choice 127 field only recently (Gross, 1998, 2007, 2010; sion was shown to be beneficial in extremely Koole, 2009; Tamir, 2011). From its early adverse situations (e.g., Bonanno & Kelt- days a central question in this field has been ner, 1997). At the same time, the ostensibly whether different forms of emotion regula- adaptive strategy of distraction was found to tion have different consequences. However, be maladaptive when long-term adjustment in a first generation of studies, different is required (Kross & Ayduk, 2008), and the forms of emotion regulation have generally ostensibly adaptive strategy of reappraisal been considered to be either adaptive or mal- was found to be less effective when dealing adaptive. with particularly high-i­ntensity emotional situations (Sheppes, Catran, & Meiran, Of two well-known examples, consider 2009; Sheppes & Meiran, 2007, 2008). first Nolen-H­ oeksema’s influential response styles theory contrasting maladaptive rumi- The main conclusion derived from the nation with distraction (Nolen-­Hoeksema, second generation of studies is that emotion 1991; see Nolen-H­ oeksema, Wisco, & Lyu- regulation strategies have different conse- bomirsky, 2008, for reviews). Multiple stud- quences in different contexts. This means ies have widely established that ruminating that healthy adaptation is the result of flex- on negative attributes of one’s self rather ibly choosing between regulation strategies than distracting attention away from them to adapt to differing situational demands is strongly related to the development, main- (e.g., Bonanno, 2005; Kashdan & Rotten- tenance, and recurrence of depressive epi- berg, 2010; see Troy & Mauss, 2011, for sodes. As a second example, consider Gross’s reviews). To illustrate, Kashdan and Rot- canonical process model and supporting evi- tenberg (2010) argue that different forms dence from multiple studies showing the rel- of psychopathology can be characterized by ative superiority of reappraising the mean- different ways in which flexible regulation ing of negative events over suppression of choice breaks down, and Troy and Mauss’s affective expressions, with respect to a wide (2011) and Bonanno’s (2005) influential range of affective, cognitive, and social indi- accounts highlight how flexible regulation cators of adaptive functioning (Gross, 2002; choice promotes resilience in the face of see Gross & Thompson, 2007, for reviews). stress and trauma. The centrality of the alleged dichotomy between “adaptive” and “maladaptive” Although emotion regulation choice is now forms of emotion regulation is captured by viewed as a crucial element in healthy adap- a recent meta-a­nalysis that summarizes a tation, until recently it has only been studied decade of work on the relationship between indirectly. Two forms of important yet indi- some regulation strategies (rumination, sup- rect evidence come from correlational stud- pression) and psychopathology, and other ies involving self-­report questionnaires that strategies (reappraisal, problem solving) and assess individual differences in people’s fre- resilience (Aldao, Nolen-­Hoeksema, & Sch- quency of use of different regulatory strate- weizer, 2010). gies across situations (e.g., Aldao & Nolen-­ Hoeksema, 2012a; Garnefski et al., 2001; There is no doubt that the first generation Gross & John, 2003; John & Gross, 2007) of studies has enormously advanced the field and from laboratory experiments that evalu- of emotion regulation. However, with its ate spontaneous use of emotion regulation rapid growth a second generation of studies strategies in emotion-­inducing situations to emerge has begun to find inconsistencies (e.g., Aldao & Nolen-H­ oeksema, 2012b; in the formerly unconditional maladaptive–­ Ehring, Tuschen-­Caffier, Schnülle, Fischer, adaptive label given to different strategies. & Gross, 2010; Gruber, Harvey, & Gross, For example, the ostensibly maladaptive 2012). While showing important links to strategy of rumination was found to have well-being and various forms of psychopa- both adaptive and maladaptive forms (Wat- thology (see Aldao et al., 2010, for a review), kins, 2008). For example, rumination was these studies do not assess the factors that found to be advantageous in situations in influence individuals predominantly to pre- which a single goal needs to be maintained fer using one particular regulatory strategy in the face of distractors (Altamirano, over another. Even more conventional stud- Miyake, & Whitmer, 2010). Similarly, the ies of emotion regulation have not exam- ostensibly maladaptive strategy of suppres- ined which regulation strategies are chosen

128 COGNITIVE APPROACHES in different emotional contexts, because engagement–­disengagement dimension rela- the experimental design involved directly tive to other potent factors, such as cogni- instruct participants to use regulation strat- tive effort. To help us better appreciate the egies in different situations. Consider, for broader context of this emotion regulation example, the most direct and convincing evi- choice conceptual framework I elaborate it dence of the importance of flexible regula- below. tion patterns, which has shown that the abil- ity to alternate flexibly between enhancing The starting point of this conceptual and suppressing emotion strongly predicts framework is that individuals’ limited cog- healthy adaptation (Bonanno, Papa, Lal- nitive capacity poses permanent processing ande, Westphal, & Coifman, 2004) over an constraints. These constraints result in a extended time period (Westphal, Seivert, & constant competition between emotion gen- Bonanno, 2010), and that flexible regulation eration and emotion regulation processes can protect people from complicated grief (Gross, Sheppes, & Urry, 2011a, 2011b) patterns in bereavement (Gupta & Bonanno, for dominance over behavior. Our concep- 2011). In these and other studies, the regu- tual account draws on major information-­ lation strategies employed by participants processing theories (e.g., Hübner, Stein- were determined by the experimenter, leav- hauser, & Lehle, 2010; Pashler, 1998) and ing the important topic of determinants and the process model of emotion regulation consequences of emotion regulation choice (Gross & Thompson, 2007) to suggest that unexplored. regulating one’s emotions involves recruit- ing deliberate executive control mechanisms Conceptualizing Emotion that try to modify the nature of emotional Regulation Choice information processing at two major cogni- Recently my colleagues and I developed a tive stages (Sheppes & Gross, 2011, 2012), conceptual framework to explain (1) major which include an early attentional selection determinants and (2) underlying mecha- stage and a late semantic meaning stage. nisms of emotion regulation choice (Sheppes, Specifically, regulation can be achieved via Scheibe, Suri, Radu, Blechert, & Gross, early disengagement from emotional pro- 2012). A central working hypothesis in this cessing at the attention selection stage, or via framework is that healthy individuals would an engagement with emotional processing be sensitive to the central costs and benefits that is modulated at a late semantic meaning associated with the implementation of each stage (e.g., Johnston & Heinz, 1978; Lehle regulatory option under different contexts. & Hübner, 2008). This conceptual frame- With this working hypothesis in mind, cer- work focuses on two particular regulatory tain emotional, cognitive, and motivational strategies that are frequently used in every- contextual factors are likely to bias regula- day life and have their major influence in tory choices in ways that are congruent with each of these two cognitive stages of infor- the differential consequences of implement- mation processing. ing these strategies under these conditions. In illuminating the underlying mechanisms Incoming emotional information can of emotion regulation choice a further argu- be regulated at an early attentional selec- ment suggests that healthy regulation choice tion processing stage by disengaging from should require in some contexts an ability to emotional information processing before recruit deliberate executive control processes it undergoes elaborated processing. A clas- that can override contrasting associative sic early selection strategy is distraction, emotional processes. This interplay of higher which involves disengaging attention from control processes that can override opposing emotional processing before it is represented associative processes is at the heart of many in working memory by producing neutral models of self-­regulation (e.g., Muraven & thoughts that are independent from and not Baumeister, 2000). Moreover, the regula- in conflict with emotional information (e.g., tion choice process gives a central weight to van Dillen & Koole, 2007). differences between strategies’ underlying Engagement with incoming emotional information that passes the early atten- tional selection stage can still be regulated at a late semantic meaning processing stage before it affects behavior. A classic late

Emotion Regulation Choice 129 selection regulation strategy is reappraisal, from behavioral and electrophysiological which involves engaging with and elaborat- studies. Specifically, several behavioral stud- ing emotional information prior to changing ies showed that employing early disengage- its meaning in a late processing stage (e.g., ment distraction in high-­sadness emotion- Gross, 2002). In reappraisal, the neutral ally intense situations resulted in successful reinterpretation is semantically dependent regulation (Sheppes & Meiran, 2007), and and in direct conflict with the original emo- did not result in an increased expenditure tional information. According to our con- of cognitive resources (Sheppes et al., 2009; ceptual framework, these underlying char- Sheppes & Meiran, 2008). At the same time, acteristics of disengagement distraction and distraction’s lack of emotional processing engagement reappraisal result in a differen- was demonstrated in an impaired memory tial cost–b­ enefit tradeoff (Sheppes & Gross, for emotional information (Sheppes & Mei- 2012). Specifically, the benefits of block- ran, 2007, 2008), and distraction’s motiva- ing emotional information early, before it tional costs were evident in a lack of long- gathers force via distraction, are that emo- term attenuation of the intensity or quality tionally high-i­ntensity information can be of important autobiographical, negative successfully modulated. Cognitively, this emotional events (Kross & Ayduk, 2008). successful modulation involves relatively By contrast, these studies showed that while simple processes, because the generation of employing late engagement reappraisal in regulatory neutral thoughts in distraction low-s­ adness emotionally intense situations is independent from and does not conflict was successful, high-­sadness emotionally with the original emotional information. intensity situations resulted in less success- Nevertheless, the major cost of distraction ful modulation and in an increased expendi- is that because it does not allow processing, ture of cognitive resources. The elaborated evaluating, and remembering of emotional emotional processing was demonstrated in information, motivationally it is less benefi- intact memory for emotional information cial for one’s long-term goals and adaptation (see also Dillon, Ritchey, Johnson, & LaBar, (see Wilson & Gilbert, 2008, for a review). 2007; Richards & Gross, 1999, 2000), and Specifically, distraction does not allow one reappraisal’s motivational benefits evinced to attend to emotional events that are being in adaptation to distressing events that are repeatedly encountered and be provided important for one’s long-term goals and with adequate explanation, a requirement functioning. that is at the heart of many long-term goals in which an individual has to face difficul- In two recent electrophysiological stud- ties in order to adapt. ies, my colleagues and I took advantage of the excellent temporal resolution of electro- The underlying characteristics of engage- encephalography (EEG) and event-­related ment reappraisal result in a different set of potentials (ERPs) to provide further sup- costs and benefits. Specifically, the elabo- port for the differential underlying cognitive rated semantic processing that occurs prior mechanisms and consequences of employ- to late modulation should be emotionally ing distraction and reappraisal (Blechert, costly, because it can less successfully block Sheppes, Di Tella, Williams, & Gross, 2012; high-i­ntensity emotional information. Cog- Thiruchselvam, Blechert, Sheppes, Ryd- nitively, reappraisal engages relatively com- strom, & Gross, 2011). Recent ERP stud- plex processes, because the generation of ies in emotion regulation showed that dis- alternative construals is dependent on and traction (e.g., Dunning & Hajcak, 2009; in conflict with the original emotional infor- Hajcak, Dunning, & Foti, 2009) and reap- mation. Nevertheless, the major benefit of praisal (e.g., Foti & Hajcak, 2008; Hajcak engaging with emotional information is that & Nieuwenhuis, 2006) modulate the late motivationally it allows one to process, eval- positive potential (LPP)—an electrocortical uate, and remember emotional information, component that is enhanced during emo- which are crucial for long-term goals and for tionally arousing viewing and that reflects adaptation. enhanced attentional and semantic meaning processing of emotionally salient informa- Direct empirical support for the costs and tion (Hajcak, MacNamara, & Olvet, 2010). benefits of employing (rather than freely choosing) distraction and reappraisal comes Consistent with the framework, we found

130 COGNITIVE APPROACHES that implementing distraction resulted in differences between distraction and reap- a strong modulation of an initial phase of praisal, as well as how to implement each. In the LPP that represents early disengage- the actual task, participants are exposed to ment of emotional information before it is series of trials that involve a brief presenta- represented in working memory, and reap- tion of an emotional stimulus followed by a praisal only modulated the late phase of the choice screen, where they decide their pre- LPP representing engagement and elabo- ferred regulatory strategy. Following a short rated meaning prior to late modulation preparation period, the emotional stimulus (Thiruchselvam et al., 2011). In that same reappears and participants are instructed study, we also tested our prediction that, to implement their chosen strategy and to motivationally, distraction relative to reap- rate how they feel. In general, participants praisal cannot be in accord with major long- are instructed to choose freely between dis- term goals. Specifically, distraction does not traction and reappraisal, and to base their allow attending to and explaining emotional choice on the strategy they think will make information, which result in subsequent them feel less negative. The main dependent attenuation of emotional responses relative measure is the proportion of choice of each to novel emotional situations (Wilson & regulatory option. To evaluate adherence of Gilbert, 2008). To that end, our participants participants’ reports of their chosen regula- were reexposed to emotional materials they tory strategies (achieved via which button had previously distracted or reappraised. they pressed during the choice screen), we Consistent with our prediction, we found either asked participants to talk out loud that emotional materials with a distraction about their chosen strategies during strat- but not reappraisal history demonstrated egy implementation or we administered a an enhanced LPP during reexposure, which surprise memory test for emotional materi- represents an extended influence of negative als that were presented (see Sheppes et al., emotional processing beyond the regulatory 2011). These two methods have proven to episode. Prolonged influence of emotional be satisfactory. Specifically, adherence levels events is considered maladaptive, with many were evident in high agreement between par- long-term goals that require dealing with ticipants’ button responses and their talk- emotional events that are repeatedly encoun- out-loud protocols, and in the finding that tered (see also MacNamara, Ochsner, & in trials on which participants indicated that Hajcak, 2011). In a similar vein, we recently they chose to distract, memory was worse showed that repeated reappraisal efforts (and not significantly different from chance) with biologically significant emotional stim- than that in trials of participants who indi- uli (i.e., angry facial expressions) resulted cated that they chose to reappraise (Sheppes in a gradual change to the basic evaluation et al., 2011). and thus representation of these emotional stimuli (Blechert et al., 2012). The first determinant of regulation choice examined is emotional intensity, which is a Emotional, Cognitive, key dimension of variation across emotional and Motivational Determinants contexts (Sheppes et al., 2011). Based on the of Emotion Regulation Choice emotion regulation choice conceptual frame- In a recent set of studies my colleagues and I work, we predicted that under low-n­ egative have evaluated central determinants of emo- emotionally intense situations, individuals tion regulation choice (Sheppes, Scheibe, would prefer to choose late selection engage- Suri, & Gross, 2011; Sheppes et al., 2012). ment reappraisal over early selection disen- Prior to discussing these studies I wish gagement distraction, because reappraisal briefly to describe the novel paradigm we can both successfully modulate emotional developed to enable the evaluation of emo- responding and provide long-term affec- tion regulation choice. In short, partici- tive adaptation. However, we predicted that pants that come to the laboratory initially under high-­negative intensity situations, undergo a learning phase, followed by a participants would switch, to prefer choos- training phase in which they are taught the ing early disengagement distraction over reappraisal, because only distraction can successfully block emotional information early before it gathers force.