Trudne Zagadki Logiczne

496 Ian McLaren and Frederick Verbruggen “overexpectation” first demonstrated by Rescorla (1970). If X and Y are both trained individually (X+ Y+) and then trained in compound with the same reinforcer (XY+), the result is that at test, X and Y will both elicit less responding in the animal than after the initial training involving the individual stimuli. Thus, a reduction in associative strength is deemed to have taken place as a result of the two stimuli “overpredicting” the US when offered in compound. Kremer predicted that if BXY+ was trained after the X+ Y+ pretraining, the overexpectation effect should confer inhibitory status on the initially neutral B. Kremer observed exactly this. Our point is that at no stage in this procedure does the outcome (delivery of the same US) change, making any interference account of this phenomenon hard to sustain. This is not to say that interference may not play a role in some demonstrations of what is termed “inhibition,” but we do not believe that it can be the full story. This point will take on added significance when we review some of the human data in a later section. Which brings us back to the question: Which of the associative architectures shown in Figure  19.1 is to be preferred? The evidence that tends to favor the direct link shown in the left‐hand panel of Figure 19.1 is that involving CS–CS associations, such as the Espinet effect. To understand this, it is necessary to realize that the role of A in the figure is being played by the common element X (lemon in this case), the role of B by saline, and the role of the US by sucrose. Thus, a preexposure trial involving sucrose + lemon leads to an association between their representations forming as shown between A and the US in the figure. Now, a trial following this in which saline + lemon is presented will allow the representation of lemon (A) to activate the represen- tation of sucrose (US), so that the representation of saline (B) forms an inhibitory link to that representation of sucrose (which is not physically present). We have already explained why the effect is thought to be mediated via the ability of saline, say, to inhibit the representation of sucrose after experience of sucrose + lemon/saline + lemon exposure. Clearly, it makes little sense to talk of saline exciting an aversive center when both the sucrose and saline solutions are essentially neutral prior to con- ditioning (the rats have a mild liking for both at the concentrations used). We are forced to the conclusion that either the No‐US representation has to be very specific (i.e., in this case “No‐Sucrose”), or an inhibitory link to the sucrose represen- tation itself is required. Both structures amount to much the same thing once we realize that the “No‐Sucrose” structure is effectively an implementation of the direct inhibitory link that gets around the need for relatively long‐distance pathways for inhibition (see above). Hence, we are proposing an excitatory link to some local inter- neuron that then inhibits (locally) the representation of sucrose. Clearly, we would also need to postulate some resting activation of this sucrose representation in order for this inhibition mediated via activation of some representation of saline to be ­effective and to give us the Espinet effect. The type of evidence that tends to favor the mutually inhibitory appetitive/aversive centers structure draws on studies of trans‐reinforcer blocking. Dickinson and Dearing (1979) were able to show that training B to be an inhibitor for a food US enabled it to successfully block learning involving a shock US. That is, once the A+ AB– training was completed using the food US, the next phase was CB+ where the + now denotes shock. Compared with controls, this group learned less about the association between C and shock, suggesting that the prior training of B was, to some extent, blocking acquisition for C. A result of this type fits in well with the idea that the “No‐US” center could indeed be some general appetitive or aversive motivational representation,

Association, Inhibition, and Action 497 such that a stimulus that came to predict the absence of food that was otherwise expected could itself acquire aversive properties. It is difficult to see how a result of this type could be generated with the architecture shown in the left‐hand panel of Figure 19.1. For a review of motivational conditioning and interactions between the appetitive and aversive system, see Dickinson and Balleine (2002). Our final position, then, is that there is evidence for (1) a general form of inhibition mediated via excitatory connections to appetitive/aversive centers that mutually inhibit one another and (2) a more specific form of inhibition that is equivalent to a direct inhibitory link to the stimulus representation (be it CS or US) in question. The first mechanism relates more strongly to the motivationally significant stimuli (USs) used in conditioning, the second to structures in what might be termed associative memory. Basic Phenomena II: Conditioned Inhibitory Control All our examples of inhibition so far relate to what is called Pavlovian or classical con- ditioning where associations are formed between representations of events that occur in the environment. But this is simply one form of what Dickinson calls event–event learning (Dickinson, 1980). Now we turn to the issue of inhibition in an instrumental context, where the task is to withhold or cancel a thought or action rather than detect the unexpected absence of an event. To do this, we will focus on human experiments that investigate the role of inhibition in executive control. Our review of this area will conclude that in many cases, it is unnecessary to appeal to inhibition to explain performance, But there are some circumstances where the case for inhibition seems to be strong, and we will focus on these once we have identified them. In the last few decades, “inhibition” has become a central concept in many theories of attentional and executive control. The general tenet is that humans need inhibitory mechanisms to suppress irrelevant stimuli, thoughts, actions, and emotions to deal effectively with the constant inflow of information and multitude of response options. Within the executive control domain, inhibition is not regarded as a unitary c­ onstruct, and several taxonomies have been proposed. Nigg (2000) distinguished between (1) cognitive inhibition, which refers to the suppression of irrelevant thoughts and information in working memory; (2) interference control, which refers to suppression of irrelevant stimuli; (3) behavioral or motor inhibition, which refers to the suppres- sion of automatic, prepared, or cued responses; and (4) oculomotor inhibition, which refers to the effortful suppression of reflexive saccades. Similar taxonomies and dis- tinctions between cognitive and behavioral (or motor) inhibition have been proposed by Friedman and Miyake (2004) and Harnishfeger (1995), among others. The case for cognitive inhibition is weak (see, e.g., Raaijmakers & Jakab, 2013; MacLeod et al., 2003). Therefore, we will focus on the inhibition of responses. Top‐down response inhibition in interference tasks The role of inhibition in interference control or congruency tasks, such as the Eriksen flanker task or the Stroop task, is still disputed. Popular dual‐route models (e.g., Kornblum, Hasbroucq, & Osman, 1990) assume that responses in congru- ency tasks are activated via a direct activation route and an indirect activation route.

498 Ian McLaren and Frederick Verbruggen Activation via the direct route is unconditional and automatic, independent of the task instructions. By contrast, activation of the response via the indirect route is deliberate and controlled. Inhibitory accounts state that conflict or interference is resolved by strengthening the processing of relevant information via the indirect route and by selectively inhibiting irrelevant information and responses that were activated via the direct route (e.g., Ridderinkhof, 2002). Some have argued that inhibition is required to suppress all motor responses globally when conflict bet- ween alternative actions is detected (Frank, 2006; Wiecki & Frank, 2013). This would effectively allow the system to prevent premature responses and to select the appropriate response. In recent years, evidence both in favor and against inhibitory accounts of interfer- ence control has been forthcoming. First, several studies have demonstrated that t­op‐down inhibition may not be required to resolve interference, as this can be achieved by top‐down enhancement of relevant information alone. Several computa- tional models of interference control assume that task demand units or representa- tions of the relevant categories will bias processing in the subordinate pathways, enhancing the processing of task‐relevant information (e.g., Cohen, Dunbar, & McClelland, 1990; Herd, Banich & O’Reilly, 2006). It may be that activation of task‐ relevant information leads to inhibition of competing task‐irrelevant processing via lateral inhibitory connections. But it is important to stress that this inhibition is achieved locally and not via top‐down inhibitory connections. Thus, inhibition of task‐irrelevant information would be a local “side‐effect” of top‐down excitation of task‐relevant information. Again, this would help to get around the need for relatively long‐distance pathways for inhibition. But the top‐down response‐inhibition account has also received support, primarily from neuroscience studies (but see also Ridderinkhof, 2002). For example, a recent study tested the response inhibition account using motor‐evoked potentials (MEPs) elicited by transcranial magnetic stimulation of the right motor cortex (Klein, Petitjean, Olivier, & Duque, 2014). The authors found reduced MEPs for trials on which the distractors were mapped onto a left response. This suggests that suppres- sion of motor excitability is a component of interference control (see also van den Wildenberg et al., 2010). It is possible that interference and competition caused by irrelevant stimuli is resolved by activating relevant features and stimulus processing, whereas response competition is resolved by activating the relevant response and selectively suppressing the irrelevant response via separate Go and NoGo pathways between prefrontal cortex and the basal ganglia (e.g., Frank, 2005). More specifically, the relevant response can be activated via activation of “Go” cells in the striatum that inhibit the internal segment of the globus pallidus (GPi); this reduces inhibition of the thalamus, leading to the execution of a motor response (the direct cortical–­ subcortical pathway; Nambu, Tokuno, & Takada, 2002).1 Irrelevant responses can be suppressed via activation of “Nogo” striatal cells, which inhibit the external segment of the globus pallidus (GPe); this reduces tonic inhibition between GPe and the GPi, resulting in increased activity in GPi and, consequently, increased inhibition of the thalamus (the indirect cortical–subcortical pathway; Nambu et al., 2002). Note that global suppression of all motor output, as postulated by Frank and colleagues, could be achieved via a third pathway, namely the hyperdirect pathway. This involves activation of the subthalamic nucleus, which has in turn a broad effect on GPi, leading

Association, Inhibition, and Action 499 to global suppression of the thalamus. Prefrontal areas, such as the presupplementary motor area and the right inferior frontal gyrus, are thought to activate the Nogo cells in the striatum or the subthalamic nucleus. Aftereffects of top‐down inhibition: negative priming After a stimulus has appeared as a distractor in congruency tasks such as a picture‐ naming task or an Eriksen flanker task, responding to it on the next trial is usually impaired. This finding is referred to as “negative priming.” The dominant inhibition account of negative priming assumes that when an item is a distractor, its representa- tion or the process linking the representation with the response becomes suppressed, and that residual inhibition impairs responding to the item on the following trial (e.g., Tipper, 2001). However, this impairment could be caused by the retrieval of stimulus‐ and response information from the previous trial (e.g., Neill, Valdes, Terry, & Gorfein, 1992; Rothermund, Wentura, & De Houwer, 2005). For example, Neill and colleagues proposed that a distractor becomes associated with a do‐not‐respond ­representation; when it is repeated on the next trial as a target, the do‐not‐respond association is activated via associative retrieval, and this will interfere with responding. By contrast, Rothermund et al. (2005) suggested that the distractor becomes associated with the response to the target on the prime trial; retrieval of this response association will interfere with responding on the current probe trial because the retrieved information is usually inconsistent with the currently relevant response (see Jones, Wills, & McLaren, 1998, for an example of how this type of response interference might be implemented). Mayr and Buchner (2007) reviewed the negative priming literature, and argued that the available data generally favor the memory account over the distractor‐inhibition account. There is a parallel to draw between the memory retrieval accounts of negative priming and the conditioned inhibition accounts discussed in the section “Basic Phenomena I: Conditioned Inhibition”. The response‐interference account of nega- tive priming is akin to the interference account of conditioned inhibition that assumes US–US interference. As discussed above, interference between CS or US representa- tions may contribute to conditioned inhibition, but it seems unlikely that it is the only mechanism responsible for the effects we have covered. Similarly, Rothermund et al. (2005, p. 493) noted that “stimulus–response retrieval is not the only mechanism that produces negative priming, it is one of the underlying mechanisms.” One of the other mechanisms could be the establishment of a link between the stimulus and a “do not respond” or “no response” representation, similar to a “no‐US” representation in conditioned inhibition paradigms. This “no‐response” representation could be specific (e.g., “no left response,” akin to a “no‐A” representation) or more general. Consistent with the latter option, Frings, Moeller, and Rothermund (2013) have argued that both stimuli and responses may be represented by abstract conceptual codes; for example, responses would be coded in terms of approach or avoidance. In the context of negative priming, this would imply that distractors are linked to a gen- eral “avoid/aversive” representation. Indeed, several recent studies suggest that conflict is aversive (e.g., Fritz & Dreisbach, 2013; van Steenbergen, Band, & Hommel, 2009; see also Botvinick, 2007). Furthermore, work by Raymond and colleagues sug- gests that ignoring a distractor could lead to its devaluation (e.g., Raymond, Fenske, &

500 Ian McLaren and Frederick Verbruggen Tavassoli, 2003). Again, this is consistent with the idea that stimuli can be linked with general appetitive/approach and aversive/avoidance centers, which mutually inhibit each other. Later on, we will argue that there is good reason to suppose the existence of both mutually inhibitory appetitive/aversive centers and separate approach/avoid- ance centers, which we will refer to as “go” and “stop” centers. Top‐down inhibition of behavior The idea that responses or motor actions can be inhibited in a top‐down fashion receives the strongest support from paradigms such as the go/no‐go paradigm and the stop‐signal paradigm. Therefore, we will focus on these two paradigms in the remainder of this chapter. In the go/no‐go paradigm, subjects are presented with a series of stimuli and are told to respond when a go stimulus is presented and to with- hold their response when a no‐go stimulus is presented (e.g., press the response key for a square, but do not press the response key for a diamond; Figure 19.2, left panel). One could argue that the go/no‐go task corresponds to an AX+ | BX– design, with A and B as the go stimulus and the no‐go stimulus, respectively, and X as the task context. In the stop‐signal paradigm, subjects usually perform a choice reaction task on no‐signal trials (e.g., press the left response key for a square and press the right response key for a diamond; Figure 19.2, right panel). On a random selection of the trials (stop‐signal trials), a stop signal (e.g., an auditory tone or a visual cue, such as the outline of the go stimulus turning bold) is presented after a variable delay (stop‐ signal delay; SSD), which instructs subjects to withhold the response to the go ­stimulus on those trials. This corresponds to an A+ | AB– design, with A corresponding to the go stimuli, and B the stop signal. Behaviorally, performance in both paradigms can be modeled as an independent race between a go process, which is triggered by the presentation of a go stimulus, and a stop process, which is triggered by the presentation of the no‐go stimulus or the stop signal (Logan & Cowan, 1984; Logan, Van Zandt, Verbruggen, & Wagenmakers, (A) go nogo go (B) right stop go left left FIX MAX FIX MAX FIX MAX RT RT RT FIX SSD MAX RT –SSD Figure 19.2  Schematic illustration of the go/no‐go and stop‐signal paradigms. FIX = dura- tion of the fixation interval; MAX RT = maximum response latency; SSD = variable stop‐signal delay in the stop‐signal paradigm.

Association, Inhibition, and Action 501 2014; Verbruggen & Logan, 2009). When the stop process finishes before the go process, response inhibition is successful, and no response is emitted (signal‐inhibit); when the go process finishes before the stop process, response inhibition is unsuc- cessful, and the response is incorrectly emitted (signal‐respond). In the stop‐signal task, the covert latency of the stop process (stop‐signal reaction time or SSRT) can be estimated from the independent race model (Logan & Cowan, 1984). SSRT has proven to be an important measure of the cognitive control processes that are involved in stopping. For recent reviews of studies of response inhibition in cognitive p­ sychology, cognitive neuroscience, developmental science, and psychopathology, see, for example, Bari and Robbins (2013), Chambers, Garavan, and Bellgrove (2009), and Verbruggen and Logan (2008c). Neurally, response inhibition processes primarily engage a fronto‐basal‐ganglia inhibition network, which includes the right (and possibly left) inferior frontal gyrus, the presupplementary motor area, the anterior cingulate cortex, the dorsolateral ­prefrontal cortex, parietal regions, and basal ganglia (Aron, Robbins, & Poldrack, 2014; Bari & Robbins, 2013; Chambers et al., 2009; Swick, Ashley, & Turken, 2011).2 On go trials, activation in frontal and parietal areas could lead to activation of a go response via the direct fronto‐basal ganglia pathway (see above). In the case of response inhibition, activation in prefrontal areas could lead to a suppression of motor output via the hyperdirect fronto‐basal ganglia pathway (see above), resulting in fast and global suppression of motor output. This might affect all response tendencies including activation in muscles that are irrelevant to the task (Badry et al., 2009; Greenhouse, Oldenkamp, & Aron, 2011; Majid, Cai, George, Verbruggen, & Aron, 2012). More selective inhibition of a specific response could potentially be achieved via activation of the indirect fronto‐basal pathway (Majid et al., 2012; Smittenaar, Guitart‐Masip, Lutti, & Dolan, 2013). The exact cognitive role of the frontal regions is debatable, partly because a detailed processing framework is lacking in many neuro- science studies (McLaren, Verbruggen, & Chambers, 2014). Moreover, the ­prefrontal areas that are involved in top‐down response inhibition are generally recruited by tasks that require selection of competing actions (Bunge, 2004; Duncan & Owen, 2000) and reprogramming or updating actions (Buch, Mars, Boorman, & Rushworth, 2010; Verbruggen, Aron, Stevens, & Chambers, 2010). Thus, response selection and  response inhibition may be two sides of the same coin (see also Mostofsky & Simmonds, 2008), relying on overlapping prefrontal brain areas that bias processing in subordinate systems in a context‐dependent fashion. The independent race model of Logan and Cowan (1984) assumes stochastic independence between the go and stop processes. However, the cognitive neurosci- ence of stopping indicates that go and stop processes interact to produce controlled movements (see also the discussion of the basal ganglia pathways above). To address this “paradox,” Boucher, Palmeri, Logan, and Schall (2007) proposed an interactive model. In their model, the go process is initiated by the go stimulus, and a go ­representation is activated after an afferent delay. The stop process is initiated by the stop signal, and a stop representation is also activated after an afferent delay. Once the  stop representation is activated, it inhibits go processing strongly and quickly. In this interactive model, SSRT primarily reflects the period before the stop unit is activated, during which stop and go processing are independent, so its predictions correspond to those of the independent model (Logan & Cowan, 1984).

502 Ian McLaren and Frederick Verbruggen Conditioned inhibitory control? Performance in response‐inhibition paradigms is usually attributed to a top‐down act of control (Verbruggen & Logan, 2008b; McLaren, Verbruggen, & Chambers, 2014). However, in recent years, several studies have examined both the short‐term and long‐term aftereffects of stopping a response. This work suggests that stop repre- sentations may be activated via the retrieval of stimulus–stop associations. Eventually, this could lead to automaticity of stopping (Logan, 1988; Verbruggen & Logan, 2008b). In other words, inhibitory control may become conditioned. Several stop‐signal studies have observed that response latencies on no‐signal trials increase after both successful and unsuccessful stopping. This response slowing has been attributed to strategic control adjustments: Subjects must try to find a balance between responding quickly on no‐signal trials (speed) and stopping on stop‐signal trials (caution); this balance would be adjusted in favor of caution after a stop‐signal trial (Bissett & Logan, 2011). However, the slowing is more pronounced when the stimulus or stimulus category of the previous trial is repeated (Bissett & Logan, 2011; Enticott, Bradshaw, Bellgrove, Upton, & Ogloff, 2009; Oldenburg, Roger, Assecondi, Verbruggen, & Fias, 2012; Rieger & Gauggel, 1999; Verbruggen & Logan, 2008a; Verbruggen, Logan, Liefooghe, & Vandierendonck, 2008). This analysis suggests some contribution of memory retrieval. Logan (1988) argued that every time people respond to a stimulus, processing episodes are stored as instances in memory. These episodes consist of the stimulus (e.g., a shape), the interpretation given to a stimulus (e.g., “square”), the task goal (“shape judgment”), and the response (“left”). When the stimulus is repeated, previous processing episodes are retrieved, facilitating performance if the retrieved information is consistent with the currently relevant information but impairing performance if the retrieved information is inconsistent. On a stop‐signal trial, the go stimulus or stimulus category becomes associated with stopping; when the stimulus (or category) is repeated, the stimulus–stop association is retrieved, and this interferes with responding on no‐signal trials. The idea here, then, is that the go response/goal and the stop response/goal are mutually inhibitory (cf. Boucher et al., 2007) in much the way that Dickinson and Dearing (1979) pos- tulate appetitive and aversive stimuli are. This stimulus–stop association account is related to the “do‐not‐respond tag” account of the negative priming effect, ­mentioned earlier (Neill & Valdes, 1992; Neill et al., 1992); of course this is no coincidence because both accounts are based on the Instance Theory of Logan (1988). The stimulus– stop effects are observed up to 20 trials after the presentation of the stop signal (Verbruggen & Logan, 2008a). Similar long‐term effects have been observed in task‐ switching studies, suggesting that stimuli can become associated with tasks or task goals (Waszak, Hommel, & Allport, 2003, 2004, 2005). Theoretically, repetition priming effects can be viewed as the first step toward autom- atization (Logan, 1990). According to Logan, automatization involves a transition from performance based on cognitive algorithms or rules to performance based on memory retrieval. Therefore, the observation that a stimulus could prime stopping after a signal trial raises the question whether inhibitory control may become a bottom‐ up act of control, driven by retrieval of stimulus–stop associations from memory, instead of a top‐down act of control. In a series of experiments, we examined the b­ ottom‐up idea (Verbruggen & Logan, 2008b). Initially, we used go/no‐go tasks in

Association, Inhibition, and Action 503 which the stimulus category defined whether subjects had to respond (e.g., natural objects = go) or not (e.g., man‐made objects = no‐go). We trained subjects to stop their response to a specific stimulus, and then reversed the go/no‐go mappings in a test phase. In this test phase, subjects were slower to respond to that stimulus com- pared with stimuli that they had not seen before (Verbruggen & Logan, 2008b, Experiment 1). Furthermore, learning the new go association was slowed, so one could argue that it passes a retardation test for inhibition. The response slowing was still observed when the tasks changed from training to test: Subjects made natural/ man‐made judgments in training but large/small judgments in test (or vice versa; Experiment 2), and RTs were longer for inconsistent items (i.e., no-go in one task but go in the other task) than for consistent items (i.e., go in both tasks). This last is a result akin to that obtained in summation tests for inhibition if training for a given stimulus in one category was natural + stimulus = no-go, then on test small + stimulus = go; the inhibition derived from training has transferred to the novel test situation in a manner analogous to combining an inhibitor with a novel excitor. We also demonstrated (Experiment 3) that the effect was not entirely category driven, as stimulus‐specific slowing was observed when the category‐stop mappings were inconsistent in training: Here, the go/no‐go mappings changed every block (e.g., natural = go and man‐made = no‐go, vs. natural = no‐go, man‐made = go), but we used different words for each go/ no‐go rule (resulting in consistent stimulus–stop mappings). Based on these findings, we proposed the automatic inhibition hypothesis: “automatic inhibition” occurs when old no‐go stimuli retrieve the stop goal when they are repeated, and this interferes with go processing (Verbruggen & Logan, 2008b). The stimulus–stop mapping is typically consistent in the go/no‐go paradigm, so automatic inhibition is likely to occur. However, automatic inhibition can also occur in the stop‐signal task when the mapping is manipulated (Verbruggen & Logan, 2008b; experiment 5). The experiments of Verbruggen and Logan demonstrated behaviorally that response inhibition is not always an effortful or deliberate act of control. A follow‐up neuroim- aging study showed that the right inferior frontal gyrus, which is part of the fronto‐ basal‐ganglia network that supports deliberate response inhibition (see above), was also activated when stimuli previously associated with stopping were presented in a stop‐signal task (Lenartowicz, Verbruggen, Logan, & Poldrack, 2011). Thus, at least part of the top‐down inhibition network was activated in the absence of external stop signals. However, the rIFG has been associated with a multitude of roles (e.g., atten- tional reorientation, context monitoring, response selection, reversal learning), and so this finding does not necessarily allow strong inferences about the underlying cognitive mechanisms. What is learned during conditioning of inhibitory control? What is learned during go/no‐go and stop‐signal tasks is still unclear. Based on Logan’s (1988) Instance Theory of Automatization, we hypothesized that stimuli became associated with a stop goal or stop representation in training, which impaired responding to them at test (Verbruggen et al., 2008; Verbruggen & Logan, 2008b). Like “No‐US” representations (section “Basic Phenomena I: Conditioned Inhibition”), stop representations can be interpreted in different ways. First, the stop

504 Ian McLaren and Frederick Verbruggen representation could be response specific. When a cue or stimulus is trained with s­ topping a left manual response, the stop representation would be “stop left response” (or, to be even more specific, “stop left‐hand response”); but when the stimulus is trained with stopping a right response, the stop representation would be “stop right response.” Second, the stop representation could be more general. Previously, we have argued that in stop‐signal tasks, a stimulus becomes associated with an abstract and general representation of going or stopping; in other words, it does not specify which specific response or motor program has to be executed or stopped (Verbruggen & Logan, 2008b). The study of Giesen and Rothermund (2014) provides direct support for this general representation idea. These authors demonstrated that responding to a stimulus that was previously associated with stopping was delayed even when the expected go response had changed. More specifically, the color of a letter indicated whether subjects had to execute a left or right response; the identity of the letter (“D” or “L”) was irrelevant. They found that responding to a letter was slowed down if a stop signal was presented on the previous trial, regardless of the ­“to‐be‐executed” or “to‐be‐stopped” response (e.g., a green D on the prime, followed by a red D). This suggests that the stimulus–stop associations are general. Note that the “general stop representation” idea is also indirectly supported by the observation that stopping often has general effects on the motor system (see above). Recent work on stopping to motivationally salient stimuli suggests a third interpre- tation. Several studies have found that consistent pairing of food‐related pictures to stopping in a go/no‐go or stop‐signal‐paradigm reduced subsequent food consump- tion (e.g., Houben, 2011; Houben & Jansen, 2011; Lawrence, Verbruggen, Adams, & Chambers, 2013; Veling, Aarts, & Papies, 2011; Veling, Aarts, & Stroebe, 2012). Furthermore, a similar procedure with alcohol‐related stimuli reduced alcohol intake in the laboratory (Jones & Field, 2013) and even self‐reported weekly alcohol intake of heavy drinking students (Houben, Havermans, Nederkoorn, & Jansen, 2012; but see Jones & Field, 2013). These effects could be mediated by devaluation of the stimuli that were associated with stopping (e.g., Houben et al., 2012; Kiss, Raymond, Westoby, Nobre, & Eimer, 2008; Veling, Holland, and van Knippenberg, 2008). Ferrey, Frischen, and Fenske (2012) showed that stop associations impact not only on the hedonic value of the stimuli associated with stopping but also on their behavioral incentive. They paired sexually attractive images with either going or stopping in a training phase, and then asked subjects to rate the attractiveness of the images. They found that the no-go (stop) images were rated less positively than the go images. This is similar to the findings of Raymond et al., who showed that ignoring a distractor leads to its devaluation. In a second study, Ferrey et al. showed that subjects were less willing to work to see the erotic images that were paired with stopping. Thus, conditioned inhibitory control may impact on the motivational value of stimuli, ­perhaps via creating links between the stimuli and the appetitive/aversive centers ­postulated by Dickinson and Dearing (1979). Central to the “conditioned inhibitory control” idea is the notion that the retrieval of stop representations will impair responding. However, such impairments could arise in at least two different processing stages: action selection and action execution.3 First, in go/no‐go and stop‐signal tasks, subjects must select an action on each trial (Gomez, Ratcliff, & Perea, 2007; Logan et al., 2014). The retrieval of stop information could interfere with selecting the appropriate “go” action. This would be akin to “central” interference between two competing go responses when selecting a response. This also

Association, Inhibition, and Action 505 implies that conditioned inhibitory control could be achieved via lateral local inhibitory connections between competing action options. This interference or conflict account receives some support from short‐term aftereffect studies, which demonstrated that stopping on the previous trial affected the stimulus‐locked parietal P300, but only when the stimulus was repeated (Oldenburg et al., 2012). Response‐locked motor compo- nents were not influenced, arguing against a motor locus for the effect (see also Enticott et al., 2009). Second, the retrieval of the stimulus–stop association could serve as a conditioned stop “signal,” activating the indirect or hyperdirect pathways that suppress motor output. This would be more similar to the direct, unconditional, automatic activation of an incorrect go response in interference tasks. Consistent with the motor suppression idea, Chiu, Aron, and Verbruggen (2012) showed that motor excitability was suppressed a mere 100 ms after the presentation of stimuli that were previously associated with stopping, but now required going. Of course, the two options are not exclusive. They may even rely on overlapping neural structures. The detection of conflict (defined as the competition between response options) could trigger a braking mecha- nism via the No‐go cells of the indirect pathway or the hyperdirect pathway (see above; Frank, 2006; Ratcliff & Frank, 2012). If conflict between go and stop representations is detected early enough, this braking mechanism could account for the reduced motor excitability observed in Chiu et al. (2012). Thus, the main difference between the “automatic suppression” account and the “conflict” account is the trigger of the b­ raking or stopping mechanism: the stimulus itself or the conflict caused by the retrieved information, respectively. Future work is required to determine how exactly stop repre- sentations influence responding in various situations. In combination, the work above suggests that inhibitory control can be conditioned or become “automatized.” Dickinson and Dearing (1979) made a strong case for motivational influences and an appetitive–aversive interaction in Pavlovian condi- tioning. The work on conditioned inhibitory control suggests that very similar mech- anisms might operate in instrumental inhibitory conditioning, despite the fact that Pavlovian and instrumental conditioning differ in many other ways (cf. Dickinson & Balleine, 2002). In the next section, we will focus on integrating these findings and develop a theory of how “conditioned” or “automatic” inhibition might operate. Integration: Inhibition and Association Here, we ask if it is possible to bring these two very different areas (animal condi- tioning and human cognitive psychology) together and arrive at a unified treatment of “inhibition” that would make sense in both domains. Our (somewhat tentative) answer is that it may be possible to develop an integrated approach that captures an emerging consensus in the two separate areas. This consensus revolves more around the associative structures that need to be posited to capture the notion of inhibition than the particular learning algorithms needed to operate within those structures, and so our treatment will mostly focus on the general architecture of inhibition at this point rather than exactly how it develops within this architecture (though the two issues are clearly not independent of one another). To recap, the work reviewed in the section “Basic Phenomena I: Conditioned Inhibition” suggests that there is a general form of inhibition mediated via excitatory connections to appetitive/aversive centers that mutually inhibit one another, and a

506 Ian McLaren and Frederick Verbruggen more specific form of inhibition that is equivalent to a direct inhibitory link to the stimulus representation (be it CS or US) in question. Both will contribute to learning, and task contexts might determine the relative contribution of the two. The work reviewed in the section “Basic Phenomena II: Conditioned Inhibitory Control” s­uggests that inhibition of responses is an integral part of executive control, but in many situations, this top‐down response inhibition can become “automatized.” Recent work suggests that subjects learn a general form of response inhibition, which transfers between tasks. This could be mediated by the same excitatory connections to the appetitive and aversive centers that are a key component of Pavlovian condi- tioning. Indeed, learning to stop or not to respond to a certain stimulus not only slows responding to it (e.g., Lenartowicz et al., 2011; Neill et al., 1992; Verbruggen & Logan, 2008b) but also reduces its hedonic value and motivational incentive (e.g., Ferrey et al., 2012; Houben et al., 2012; Kiss et al., 2008; Raymond et al., 2003; Veling et al., 2008). Our interpretation of this is that when a distractor or no‐go/stop stimulus becomes associated with an avoidance/aversive center, presentation of it will directly activate the avoidance/aversive center, which in turn will suppress activation of the approach/appetitive center (cf. Dickinson & Balleine, 2002). This could explain both the slower responding in an RT task and the lower hedonic values in a stimulus evaluation task using ratings. In a sense, then, we are arguing that “Go” and “Stop” are the instrumental equiv- alents of the Pavlovian “Good” and “Bad,” and a scheme that implements this idea is shown in outline in Figure 19.3. Of course, Pavlovian and instrumental conditioning should not be equated entirely, as they appear to be influenced in different ways by Associative system Response Go Stop system Motivational Appetitive Aversive system system system Figure  19.3  Model integrating associative and motivational subsystems that would enable implementation of our proposals for conditioned inhibition. The associative system contains both an auto‐associative network and recurrence, giving it the ability to capture statistical reg- ularities in the environment and between actions and outcomes. The motivational and response systems are a synthesis of Dickinson and Balleine’s (2002) implementation of Konorski’s pro- posal with an instrumental Stop/Go system along the lines proposed by Boucher et al. (2007). “Direct” conditioned inhibition takes place within the associative system, and is outcome specific. “General” conditioned inhibition takes place via links from the associative system to the other systems either at the Stop/Go instrumental level or at the Appetitive/Aversive Pavlovian level.

Association, Inhibition, and Action 507 manipulations of contexts and omission schedules (Dickinson & Balleine, 2002), and they are supported by different corticostriatal loops (for a short review, see Guitart‐ Masip, Duzel, Dolan, & Dayan, 2014). Nevertheless, recent work suggests that Pavlovian and instrumental conditioning interact in a go/no‐go task (Guitart‐Masip et al., 2014). For example, in a study by Guitart‐Masip et al. (2012), subjects had to learn stimulus‐go/no‐go contingencies. They learned them faster when correct go responses were rewarded and incorrect no‐go responses were punished, than the other way around. This was attributed to a hard‐wired Pavlovian equivalence between reward/punishment and approach/avoidance, respectively. The Konorskian model, as discussed in Dickinson and Balleine (2002), also links the aversive system with avoidance (withdrawal, suppression) and the appetitive system with approach (go). Therefore, it seems plausible to suggest that when subjects always have to stop their response to a specific stimulus, a link between this stimulus and the aversive/avoid- ance system will be created. Despite the seemingly overwhelming evidence for a strong link between go and appetite/reward and between no‐go and aversion/punishment, a few findings appear inconsistent with the no‐go/aversion account. For example, some studies have shown that response inhibition might be impaired rather than enhanced when negative emo- tional or threatening stimuli are presented (e.g., De Houwer & Tibboel, 2010; Pessoa, Padmala, Kenzer, & Bauer, 2012; Verbruggen & De Houwer, 2007). Because these studies showed similar impairments when positive stimuli were presented, the effect of emotional and threatening stimuli has been attributed to arousal (rather than valence): Arousing stimuli tend to attract attention (and are processed centrally when they high in threat), causing “dual‐task” interference. In other words, effects of arousal (attention) may have counteracted or dominated the effects of valence (positive/negative). Perhaps this is not very surprising given recent work that sug- gests that most of the stopping latency is occupied by afferent or sensory processes (Boucher et al., 2007; Salinas & Stanford, 2013); in other words, activation of the avoidance/aversive center may only have a small influence on the overall SSRT, com- pared with the effect of arousal, because of the different time courses for the processes involved. In the study by Pessoa et al. (2012; Experiment 2) in particular, the latency for activation of any aversive center due to associations between the stimulus and some motivationally significant outcome may have been too long for it to have much effect on stopping in the stop‐signal task, making any effect entirely dependent on a more cognitive appraisal of the stimulus. So far, we have focused mostly on the link between conditioned inhibitory control and appetitive/aversive valence. But our discussion of the conditioned inhibition ­literature suggests that performance cannot be explained using a single inhibitory mechanism. Apart from the direct link between the CS and the appetitive/aversive centers, there is the more specific link between the CS and US (or another CS). In the case of conditioned inhibition, this link will be inhibitory. Of course, in many other situations, this link will be excitatory (as in the original work of Pavlov). It seems likely that in the context of conditioned inhibitory control, subjects can also learn associa- tions between the representation of the go stimulus and the representation of the stop signal (Verbruggen et al., 2014). Factors such as the number and kind of stop or ­no‐go signals could determine the relative contribution of stimulus–stimulus associa- tions versus stimulus–approach/avoidance associations.

508 Ian McLaren and Frederick Verbruggen Conclusion: Inhibition in Cognitive Control and Associative Learning We have tried to provide a modern approach to the problem of inhibition that draws on many of the classic studies in the animal learning tradition that exemplify the c­ontribution that experimental psychology can make to current issues in cognitive neuroscience. We hope that this integration of the old and the new will prove fruitful in providing a framework for future research on behavioral inhibition. Notes 1 Note that the cortico‐basal‐ganglia pathways do not directly map on to the direct and indirect routes discussed in dual‐route frameworks. 2 Inhibition of eye movements may recruit a different network. Single‐cell studies ­indicate that it relies primarily on the activation of movement‐ and fixation‐related neurons in frontal eye fields in dorsolateral prefrontal cortex and superior colliculus in midbrain (for a review, see Schall & Godlove, 2012). 3 In Verbruggen, Best, Bowditch, Stevens, and McLaren (2014), we discuss a third ­possibility, namely that attention and signal detection become conditioned. References Aron, A. R., Robbins, T. W., & Poldrack, R. A. (2014). Inhibition and the right inferior frontal cortex: One decade on. Trends in Cognitive Sciences, 18, 177–185. Baddeley, A. (1996). Exploring the central executive. Quarterly Journal of Experimental Psychology, 49A, 5–28 Badry, R., Mima, T., Aso, T., Nakatsuka, M., Abe, M., Fathi, D., . . . Fukuyama, H. (2009). Suppression of human cortico‐motoneuronal excitability during the stop‐signal task. Clinical Neurophysiology, 120, 1717–1723. Bari, A., & Robbins, T. W. (2013). Inhibition and impulsivity: Behavioral and neural basis of response control. Progress in Neurobiology, 108, 44–79. Bennett, C. H., Scahill, V. L., Griffiths, D. P., & Mackintosh, N. J. (1999). The role of inhib- itory associations in perceptual learning. Learning & Behavior, 27, 333–345. Bissett, P. G., & Logan, G. D. (2011). Balancing cognitive demands: Control adjustments in the stop‐signal paradigm. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37, 392–404. Botvinick, M. M. (2007). Conflict monitoring and decision making: reconciling two perspectives on anterior cingulate function. Cognitive, Affective, & Behavioral Neuroscience, 7, 356–366. Boucher, L., Palmeri, T. J., Logan, G. D., & Schall, J. D. (2007). Inhibitory control in mind and brain: an interactive race model of countermanding saccades. Psychological Review, 114, 376–397. Buch, E. R., Mars, R. B., Boorman, E. D., & Rushworth, M. F. (2010). A network centered on ventral premotor cortex exerts both facilitatory and inhibitory control over primary motor cortex during action reprogramming. The Journal of Neuroscience, 30, 1395–1401.

Association, Inhibition, and Action 509 Bunge, S. A. (2004). How we use rules to select actions: A review of evidence from cognitive neuroscience. Cognitive, Affective & Behavioral Neuroscience, 4, 564–579. Chambers, C. D., Garavan, H., & Bellgrove, M. A. (2009). Insights into the neural basis of response inhibition from cognitive and clinical neuroscience. Neuroscience and Biobehavioral Reviews, 33, 631–646. Chapman, G. B. (1991). Trial‐order affect s cue interaction in contingency judgement. Journal of Experimental Psychology: Learning, Memory and Cognition, 17, 837–854. Chiu, Y. C., Aron, A. R., & Verbruggen, F. (2012). Response suppression by automatic retrieval of stimulus–stop association: Evidence from transcranial magnetic stimulation. Journal of Cognitive Neuroscience, 24, 1908–1918. Cohen, J. D., Dunbar, K., & McClelland, J. L. (1990). On the control of automatic processes: A parallel distributed processing account of the Stroop effect. Psychological Review, 97, 332–361. Cotton, M. M., Goodall, G., & Mackintosh, N. J. (1982). Inhibitory conditioning resulting from a reduction in the magnitude of reinforcement. Quarterly Journal of Experimental Psychology, 34B, 163–180. De Houwer, J., & Tibboel, H. (2010). Stop what you are not doing! Emotional pictures inter- fere with the task not to respond. Psychonomic Bulletin & Review, 17, 699–703. Dickinson, A. (1980). Contemporary animal learning theory. Cambridge, UK: CUP. Dickinson, A., & Balleine, B. (2002). The role of learning in the operation of motivational ­systems. In R. Gallistel (Ed.), Stevens’ handbook of experimental psychology (3rd ed., Vol. 3, pp. 497–534). New York, NY: John Wiley & Sons. Dickinson, A., & Burke, J. (1996). Within‐compound associations mediate the retrospective reval- uation of causality judgements. Quarterly Journal of Experimental Psychology, 49B, 60–80. Dickinson, A., & Dearing, M. F. (1979). Appetitive–aversive interactions and inhibitory processes. In A. Dickinson & R. A. Boakes (Eds.), Mechanisms of learning and motivation (pp. 203–232). Englewood Cliffs, NJ: Erlbaum. Duncan, J., & Owen, A. M. (2000). Common regions of the human frontal lobe recruited by diverse cognitive demands. Trends in Neurosciences, 23, 475–483. Enticott, P. G., Bradshaw, J. L., Bellgrove, M. A., Upton, D. J., & Ogloff, J. R. (2009). Stop task after‐effects: The extent of slowing during the preparation and execution of movement. Experimental Psychology, 56, 247–251. Espinet, A., Iraola, J. A., Bennett, C. H., & Mackintosh, N. J. (1995). Inhibitory associations between neutral stimuli in flavor‐aversion conditioning. Animal Learning & Behavior, 23, 361–368. Ferrey, A. E., Frischen, A., & Fenske, M. J. (2012). Hot or not: response inhibition reduces the hedonic value and motivational incentive of sexual stimuli. Frontiers in Psychology, 3. Frank, M. J. (2005). Dynamic dopamine modulation in the basal ganglia: a neurocomputa- tional account of cognitive deficits in medicated and nonmedicated Parkinsonism. Journal of Cognitive Neuroscience, 17, 51–72. Frank, M. J. (2006). Hold your horses: A dynamic computational role for the subthalamic nucleus in decision making. Neural Networks, 19, 1120–1136. Friedman, N. P., & Miyake, A. (2004). The relations among inhibition and interference control functions: A latent‐variable analysis. Journal of Experimental Psychology. General, 133, 101–135. Frings, C., Moeller, B., & Rothermund, K. (2013). Retrieval of event files can be conceptually mediated. Attention, Perception, & Psychophysics, 75, 700–709. Fritz, J., & Dreisbach, G. (2013). Conflicts as aversive signals: Conflict priming increases neg- ative judgments for neutral stimuli. Cognitive, Affective & Behavioral Neuroscience, 13, 311–317. Giesen, C., & Rothermund, K. (2014). You better stop! Binding “stop” tags to irrelevant stim- ulus features. The Quarterly Journal of Experimental Psychology, 67, 809–832.

510 Ian McLaren and Frederick Verbruggen Gomez, P., Ratcliff, R., & Perea, M. (2007). A model of the go/no‐go task. Journal of Experimental Psychology. General, 136, 389–413. Graham, S. (1999). Retrospective revaluation and inhibitory associations: does perceptual learning modulate our perception of the contingencies between events? Quarterly Journal of Experimental Psychology, 52B, 159–185. Graham, S., Jie, H. L., Minn, C. H., McLaren, I. P. L., & Wills, A. J. (2011). Simultaneous backward conditioned inhibition and mediated conditioning. Journal of Experimental Psychology: Animal Behavior Processes, 37, 241–245. Greenhouse, I., Oldenkamp, C. L., & Aron, A. R. (2011). Stopping a response has global or non‐global effects on the motor system depending on preparation. Journal of Neurophysiology, 107, 384–392. Guitart‐Masip, M., Duzel, E., Dolan, R., & Dayan, P. (2014). Action versus valence in decision making. Trends in Cognitive Sciences, 18, 194–202 Guitart‐Masip, M., Huys, Q. J., Fuentemilla, L., Dayan, P., Duzel, E., & Dolan, R. J. (2012). Go and no‐go learning in reward and punishment: interactions between affect and effect. Neuroimage, 62, 154–166. Hall, G., & Pearce, J. M. (1979). Latent inhibition of a CS during CS–US parings. Journal of Experimental Psychology: Animal Behaviour Processes, 5, 31–43. Harris, J. A., Kwok, D. W. S., & Andrew, B. J. (2014). Conditioned inhibition and reinforce- ment rate. Journal of Experimental Psychology: Animal Learning & Cognition, 40, 335–354. Harnishfeger, K. K. (1995). The development of cognitive inhibition: Theories, definitions, and research evidence. In F. N. Dempster & C. J. Brainerd (Eds.), Interference and inhi- bition in cognition (pp. 175–204). San Diego, CA: Academic Press. Herd, S. A., Banich, M. T., & O’Reilly, R. C. (2006). Neural mechanisms of cognitive control: An integrative model of stroop task performance and FMRI data. Journal of Cognitive Neuroscience, 18, 22–32. Houben, K. (2011). Overcoming the urge to splurge: Influencing eating behavior by manipu- lating inhibitory control. Journal of Behavior Therapy and Experimental Psychiatry, 42, 384–388. Houben, K., Havermans, R. C., Nederkoorn, C., & Jansen, A. (2012). Beer à no‐go: Learning to stop responding to alcohol cues reduces alcohol intake via reduced affective associations rather than increased response inhibition. Addiction, 107, 1280–1287. Houben, K., & Jansen, A. (2011). Training inhibitory control. A recipe for resisting sweet temptations. Appetite, 56, 345–349. Jones, A., & Field, M. (2013). The effects of cue‐specific inhibition training on alcohol c­ onsumption in heavy social drinkers. Experimental and Clinical Psychopharmacology, 21, 8–16. Jones, F., Wills, A. J., & McLaren, I. P. L. (1998). Perceptual categorisation: connectionist modelling and decision rules. Quarterly Journal of Experimental Psychology, 51B, 33–58. Kiss, M., Raymond, J. E., Westoby, N., Nobre, A. C., & Eimer, M. (2008). Response inhibi- tion is linked to emotional devaluation: Behavioural and electrophysiological evidence. Frontiers in Human Neuroscience, 2, 13. Klein, P. A., Petitjean, C., Olivier, E., & Duque, J. (2014). Top‐down suppression of incom- patible motor activations during response selection under conflict. NeuroImage, 86, 138–149. Konorski, J. (1948). Conditioned reflexes and neuron organisation. Cambridge, UK: Cambridge University Press. Konorski, J. (1967). Integrative activity of the brain: An interdisciplinary approach. Chicago, IL: University of Chicago Press.

Association, Inhibition, and Action 511 Kornblum, S., Hasbroucq, T., & Osman, A. (1990). Dimensional overlap: Cognitive basis for stimulus–response compatibility – a model and taxonomy. Psychological Review, 97, 253–270. Kremer, E. F. (1978). The Rescorla–Wagner model: Losses of associative strength in compound conditioned stimuli. Journal of Experimental Psychology: Animal Behavior Processes, 4, 22–36. Lawrence, N. S., Verbruggen, F., Adams, R. C., & Chambers, C. D. (2013). Stopping to food cues reduces snack food intake: Effects of cue specificity, control conditions and individual d­ ifferences. Manuscript in preparation. Lenartowicz, A., Verbruggen, F., Logan, G. D., & Poldrack, R. A. (2011). Inhibition‐related activation in the right inferior frontal gyrus in the absence of inhibitory cues. Journal of Cognitive Neuroscience, 23, 3388–3399. Le Pelley, M. E. (2004). The role of associative history in models of associative learning: A selective review and a hybrid model. Quarterly Journal of Experimental Psychology, 57B, 193–243. Le Pelley, M. E., Cutler, D. L., & McLaren, I. P. L. (2000). Retrospective effects in human causality judgment. In Proceedings of the Twenty‐Second Annual Conference of the Cognitive Science Society. Englewood Cliffs, NJ: Erlbaum. Le Pelley, M. E., & McLaren, I. P. L. (2001). Retrospective revaluation in humans: Learning or memory? Quarterly Journal of Experimental Psychology, 54B, 311–352. Logan, G. D. (1985). Executive control of thought and action. Acta Psychologica, 60, 193–210. Logan, G. D. (1988). Toward an instance theory of automatization. Psychological Review, 95, 492–527. Logan, G. D. (1990). Repetition priming and automaticity: Common underlying mechanisms? Cognitive Psychology, 22, 1–35. Logan, G. D., & Cowan, W. B. (1984). On the ability to inhibit thought and action: A theory of an act of control. Psychological Review, 91, 295–327. Logan, G. D., Van Zandt, T., Verbruggen, F., & Wagenmakers, E. J. (2014). On the ability to inhibit thought and action: General and special theories of an act of control. Psychological Review, 121, 66–95. Lotz, A., Uengoer, M., Koenig, S., Pearce, J. M., & Lachnit, H. (2012). An exploration of the feature‐positive effect in adult humans. Learning & Behavior, 40, 222–230. Mackintosh, N. J. (1974). The psychology of animal learning. London, UK: Academic Press. MacLeod, C. M., Dodd, M. D., Sheard, E. D., Wilson, D. E., & Bibi, U. (2003). In opposition to inhibition. In Psychology of learning and motivation: Advances in research and theory (Vol. 43, pp. 163–214). London, UK: Academic Press. Majid, D. S., Cai, W., George, J. S., Verbruggen, F., & Aron, A. R. (2012). Transcranial magnetic stimulation reveals dissociable mechanisms for global versus selective corticomo- tor suppression underlying the stopping of action. Cerebral Cortex, 22, 363–371. Mayr, S., & Buchner, A. (2007). Negative priming as a memory phenomenon. Zeitschrift Für Psychologie/Journal of Psychology, 215, 35–51. McLaren, I. P. L., & Dickinson, A. (1990). The conditioning connection. Philosophical Transactions of the Royal Society B: Biological Sciences, 329, 179–186. McLaren, I. P. L., Forrest, C. L., & McLaren, R. P. (2012). Elemental representation and con- figural mappings: Combining elemental and configural theories of associative learning. Learning and Behavior, 40, 320–333. McLaren, I. P. L., Kaye, H., & Mackintosh, N. J. (1989). An associative theory of the repre- sentation of stimuli: applications to perceptual learning and latent inhibition. In R. G. M. Morris (Ed.), Parallel distributed processing – implications for psychology and neurobiology. Oxford, UK: Oxford University Press.

512 Ian McLaren and Frederick Verbruggen McLaren, I. P. L., & Mackintosh, N. J. (2000). An elemental model of associative learning: I. Latent inhibition and perceptual learning. Animal Learning and Behavior, 38, 211–246. McLaren, I. P. L., & Mackintosh, N. J. (2002).Associative learning and elemental representa- tion: II. Generalization and discrimination. Animal Learning & Behavior, 30, 3, 177–200. Miller, R. R., & Schachtman, T. R. (1985). Conditioning context as an associative baseline: Implications for response generation and the nature of conditioned inhibition. In R. R. Miller & N. E. Spear (Eds.), Information processing in animals: Conditioned inhibition (pp. 51–88). Englewood Cliffs, NJ: Erlbaum. Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. D. (2000). The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis. Cognitive Psychology, 41, 49–100. Mostofsky, S. H., & Simmonds, D. J. (2008). Response inhibition and response selection: Two sides of the same coin. Journal of Cognitive Neuroscience, 20, 751–761. Mundy, M. E., Dwyer, D. M., & Honey, R. C. (2006). Inhibitory associations contribute to perceptual learning in humans. Journal of Experimental Psychology: Animal Behavior Processes, 32, 178–184. Nambu, A., Tokuno, H., & Takada, M. (2002). Functional significance of the cortico–­ subthalamo–pallidal ‘hyperdirect’pathway. Neuroscience Research, 43, 111–117. Neill, W. T., & Valdes, L. A. (1992). Persistence of negative priming – steady‐state or decay. Journal of Experimental Psychology: Learning Memory and Cognition, 18, 565–576. Neill, W. T., Valdes, L. A., Terry, K. M., & Gorfein, D. S. (1992). Persistence of negative priming: II. Evidence for episodic trace retrieval. Journal of Experimental Psychology: Learning Memory and Cognition, 18, 993–1000. Nigg, J. T. (2000). On inhibition/disinhibition in developmental psychopathology: Views from cognitive and personality psychology and a working inhibition taxonomy. Psychological Bulletin, 126, 220–246. Oldenburg, J. F., Roger, C., Assecondi, S., Verbruggen, F., & Fias, W. (2012). Repetition priming in the stop signal task: The electrophysiology of sequential effects of stopping. Neuropsychologia, 50, 2860–2868. Pavlov, I. P. (1927). Conditioned reflexes. Oxford, UK: Oxford University Press. Pearce, J. M., & Hall, G. (1980). A model for Pavlovian conditioning: Variations in the effective- ness of conditioned but not of unconditioned stimuli. Psychological Review, 87, 532–552. Pessoa, L., Padmala, S., Kenzer, A., & Bauer, A. (2012). Interactions between cognition and emotion during response inhibition. Emotion, 12, 192–197. Raaijmakers, J. G., & Jakab, E. (2013). Is forgetting caused by inhibition? Current Directions in Psychological Science, 22, 205–209. Ratcliff, R., & Frank, M. J. (2012). Reinforcement‐based decision making in corticostriatal circuits: mutual constraints by neurocomputational and diffusion models. Neural Computation, 24, 1186–1229. Raymond, J. E., Fenske, M. J., & Tavassoli, N. T. (2003). Selective attention determines emo- tional responses to novel visual stimuli. Psychological Science, 14, 537–542. Rescorla, R. A. (1969). Conditioned inhibition of fear resulting from negative CS–US contin- gencies. Journal of Comparative and Physiological Psychology, 67, 504–509. Rescorla, R. A. (1970). Reduction in the effectiveness of reinforcement after prior excitatory conditioning. Learning and Motivation, 1, 372–381. Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and non‐reinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II: Current research and theory (pp. 64–99). New York, NY: Appleton‐Century‐Crofts.

Association, Inhibition, and Action 513 Ridderinkhof, K. R. (2002). Activation and suppression in conflict tasks: Empirical clarification through distributional analyses. In Attention and performance XIX: Common mechanisms in perception and action (pp. 494–519). Oxford: Oxford University Press. Rieger, M., & Gauggel, S. (1999). Inhibitory after‐effects in the stop signal paradigm. British Journal of Psychology, 90, 509–518. Rothermund, K., Wentura, D., & De Houwer, J. (2005). Retrieval of incidental stimulus– response associations as a source of negative priming. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31, 482–495. Salinas, E., & Stanford, T. R. (2013). The countermanding task revisited: fast stimulus detec- tion is a key determinant of psychophysical performance. The Journal of Neuroscience, 33, 5668–5685. Schall, J. D., & Godlove, D. C. (2012). Current advances and pressing problems in studies of stop- ping. Current Opinion in Neurobiology, 22, 1012–1021. Smittenaar, P., Guitart‐Masip, M., Lutti, A., & Dolan, R. J. (2013). Preparing for selective inhibition within frontostriatal loops. The Journal of Neuroscience, 33, 18087–18097. Swick, D., Ashley, V., & Turken, A. U. (2011). Are the neural correlates of stopping and not going identical? Quantitative meta‐analysis of two response inhibition tasks. NeuroImage, 56, 1655–1665. Tipper, S. P. (2001). Does negative priming reflect inhibitory mechanisms? A review and integration of conflicting views. The Quarterly Journal of Experimental Psychology: Section A, 54, 321–343. van den Wildenberg, W. P. M., Wylie, S. A., Forstmann, B. U., Burle, B., Hasbroucq, T., & Ridderinkhof, K. R. (2010). To head or to heed? Beyond the surface of selective action inhi- bition: A review. Frontiers in Human Neuroscience, 4, 222. Van Hamme, L. J., & Wasserman, E. A. (1994). Cue competition in causality judgements: The role of nonpresentation of compound stimulus elements. Learning and Motivation, 25, 127–151. van Steenbergen, H., Band, G. P., & Hommel, B. (2009). Reward counteracts conflict adaptation. Evidence for a role of affect in executive control. Psychological Science, 20, 1473–1477. Veling, H., Aarts, H., & Papies, E. K. (2011). Using stop signals to inhibit chronic dieters’ responses toward palatable foods. Behaviour Research and Therapy, 49, 771–780. Veling, H., Aarts, H., & Stroebe, W. (2012). Using stop signals to reduce impulsive choices for palatable unhealthy foods. British Journal of Health Psychology, 18, 354–368. Veling, H., Holland, R. W., & van Knippenberg, A. (2008). When approach motivation and behavioral inhibition collide: Behavior regulation through stimulus devaluation. Journal of Experimental Social Psychology, 44, 1013–1019. Verbruggen, F., Aron, A. R., Stevens, M. A., & Chambers, C. D. (2010). Theta burst stimula- tion dissociates attention and action updating in human inferior frontal cortex. Proceedings of the National Academy of Sciences of the United States of America, 107, 13966–13971. Verbruggen, F., Best, M., Bowditch, W., Stevens, T., McLaren, I. P. L. (2014). The inhibitory control reflex. Neuropsychologia, 65, 263–278. Verbruggen, F., & De Houwer, J. (2007). Do emotional stimuli interfere with response inhi- bition? Evidence from the stop signal paradigm. Cognition & Emotion, 21, 391–403. Verbruggen, F., & Logan, G. D. (2008a). Long‐term aftereffects of response inhibition: Memory retrieval, task goals, and cognitive control. Journal of Experimental Psychology: Human Perception and Performance, 34, 1229–1235. Verbruggen, F., & Logan, G. D. (2008b). Automatic and controlled response inhibition: Associative learning in the go/no‐go and stop‐signal paradigms. Journal of Experimental Psychology: General, 137, 649–672.

514 Ian McLaren and Frederick Verbruggen Verbruggen, F., & Logan, G. D. (2008c). Response inhibition in the stop‐signal paradigm. Trends in Cognitive Sciences, 12, 418–424. Verbruggen, F., & Logan, G. D. (2009). Models of response inhibition in the stop‐signal and stop‐change paradigms. Neuroscience and Biobehavioral Reviews, 33, 647–661. Verbruggen, F., Logan, G. D., Liefooghe, B., & Vandierendonck, A. (2008). Short‐term after- effects of response inhibition: Repetition priming or between‐trial control adjustments? Journal of Experimental Psychology: Human Perception and Performance, 34, 413–426. Verbruggen, F., McLaren, I. P. L., & Chambers, C. D. (2014). Banishing the control homun- culi in studies of action control and behaviour change. Perspectives on Psychological Science, 9, 497–524. Waszak, F., Hommel, B., & Allport, A. (2003). Task‐switching and long‐term priming: Role of episodic stimulus‐task bindings in task‐shift costs. Cognitive Psychology, 46, 361–413. Waszak, F., Hommel, B., & Allport, A. (2004). Semantic generalization of stimulus‐task ­bindings. Psychonomic Bulletin & Review, 11, 1027–1033. Waszak, F., Hommel, B., & Allport, A. (2005). Interaction of task readiness and automatic retrieval in task switching: Negative priming and competitor priming. Memory & Cognition, 33, 595–610. Wiecki, T. V., & Frank, M. J. (2013). A computational model of inhibitory control in frontal cortex and basal ganglia. Psychological Review, 120, 329–355. Zimmer‐Hart, C. L., & Rescorla, R. A. (1974). Extinction of Pavlovian conditioned inhibition. Journal of Comparative and Physiological Psychology, 86, 837–845.

20 Mirror Neurons from Associative Learning Caroline Catmur, Clare Press, and Cecilia Heyes Associative learning theory has typically been used to explain the behavior of whole animals; to understand why organisms make particular kinds of responses to focal stimuli and contextual cues. In this chapter, we use research on associative learning in a slightly different way, in an attempt to explain the behavior – the firing patterns – of individual neurons, rather than whole animals. The neurons in question are known as “mirror neurons” (MNs), and the behavior that has made MNs famous is their ten- dency to fire not only when a macaque performs an action, but also when the macaque passively observes a similar action performed by another. Neurons with this capacity to match observed and executed actions were originally found in area F5 of the ven- tral premotor cortex (PMC; di Pellegrino, Fadiga, Fogassi, Gallese, & Rizzolatti, 1992) and subsequently in the inferior parietal lobule (IPL; Fogassi et al., 2005) of the macaque brain. A substantial body of evidence now suggests that MNs are also present in the human brain (Molenberghs, Cunnington, & Mattingley, 2012). A variety of functions have been ascribed to MNs. Popular suggestions relate to action understanding (Gallese & Sinigaglia, 2011; Rizzolatti, Fadiga, Gallese, & Fogassi, 1996), imitation (Iacoboni et al., 1999), and language processing (Rizzolatti & Arbib, 1998). A great deal of interest has also been generated in the wider scientific and public media: MNs have been hailed as “cells that read minds” (Blakesee, 2006), “the neurons that shaped civilization” (Ramachandran, 2009), and a “revolution” in understanding social behavior (Iacoboni, 2008). Whereas much research has focused on theorizing and speculation about MN functions, this chapter’s primary focus is the origin of MNs. We ask not “What are MNs for?,” but “What is the process that gives MNs their ‘mirrorness’; their fasci- nating capacity to match observed with executed actions?” The standard answer to this question (e.g., Rizzolatti & Craighero, 2004) is evolution. The “adaptation account” assumes that the mirrorness of MNs was produced by natural selection act- ing on genetic variation. In contrast, we will argue that the balance of evidence sup- ports the “associative account” (Catmur, Press, Cook, Bird, & Heyes, 2014; Cook, Bird, Catmur, Press, & Heyes, 2014; Heyes, 2010); it suggests that the mirrorness of MNs is produced in the course of individual development by sensorimotor associative The Wiley Handbook on the Cognitive Neuroscience of Learning, First Edition. Edited by Robin A. Murphy and Robert C. Honey. © 2016 John Wiley & Sons, Ltd. Published 2016 by John Wiley & Sons, Ltd.

516 Caroline Catmur, Clare Press, and Cecilia Heyes learning. We will also argue that the associative model has major methodological implications for research investigating the functions of MNs. The first section outlines key background information regarding MNs in macaques and humans. Next, we present the adaptation and associative accounts. In the follow- ing section, we introduce four kinds of evidence that have the potential to favor one of these hypotheses over the other, and discuss each of these types of evidence in turn. Finally, we examine the implications of the associative account for future research investigating the functions of MNs. MN Background Information Where are they found, and what qualifies as an MN? In the macaque, “classical” MN areas include ventral PMC and IPL (see Figure 20.1). However, MNs have also been found in “nonclassical” areas, including primary motor cortex and dorsal PMC (Dushanova & Donoghue, 2010; Tkach, Reimer, & Hatsopoulos, 2007). In humans, there is evidence at both the single‐cell and population level of neurons with sensorimotor matching properties. These have been found both in “classical” MN areas, including inferior frontal gyrus (IFG; considered the human homolog of macaque F5) (Kilner, Neal, Weiskopf, Friston, & Frith, 2009) and inferior parietal cortex (Chong, Cunnington, Williams, Kanwisher, & Mattingley, 2008), and in nonclassical areas, including dorsal PMC, superior parietal lobule, and cerebellum (Molenberghs et al., 2012), occipitotemporal cortex (Oosterhof, Tipper, & Downing, 2012), supplementary motor area, and medial temporal lobe (Mukamel, Ekstrom, Kaplan, Iacoboni, & Fried, 2010). Although some researchers only refer to neurons found in classical areas as MNs (e.g., Molenberghs et al., 2012), many others, like us, use the term “MN” to refer to neurons in both classical and nonclassical areas (Gallese & Sinigaglia, 2011; Keysers & Gazzola, 2010). Functional definitions of what constitutes a MN also vary. In some cases, the term “MN” is used to refer to any neuron that fires during both the execu- tion and observation of action, regardless of whether the executed and observed actions are similar to one another (Gallese, Fadiga, Fogassi, & Rizzolatti, 1996; Rizzolatti & Craighero, 2004). In contrast, and following the majority of researchers in the field, we consider that MNs’ “mirrorness” is defined by the fact that they respond to observation and execution of similar actions. However, following common usage, we also refer to “logically related” MNs (see following subsection), which fire during observation and execution of dissimilar actions that have some functional relation, as “MNs.” MN response properties in the macaque Macaque MNs have been broadly divided into three types (Figure 20.2), based on their field properties, the sensory and motoric conditions in which they fire: “Strictly congruent” MNs discharge during observation and execution of the same action, for example, a “precision” grip made with thumb and index finger. “Broadly congruent” MNs fire during the execution of one action (e.g., precision grip) and during the

Mirror Neurons from Associative Learning 517 (A) (B) Human brain, lateral view Macaque brain, lateral view Dorsal PMC Superior parietal Primary motor cortex lobule Dorsal PMC IPL IPL Ventral PMC IFG Occipitotemporal Cerebellum cortex (C) Human brain, medial view Supplementary motor area Medial temporal lobe Figure 20.1  MN areas in (A) the macaque and (B, C) the human brain. These are areas in which there is evidence at the single‐cell or population level of neurons with sensorimotor matching properties. IFG = inferior frontal gyrus; IPL = inferior parietal lobule; PMC = premotor cortex. observation of one or more similar, but not identical, actions (e.g., only power grip; or multiple actions e.g. precision grip, power grip, and grasping with the mouth). So‐called “logically related” MNs (di Pellegrino et al., 1992) respond to different actions in observe and execute conditions. For example, they fire during the observa- tion of an experimenter placing food in front of the monkey, and when the monkey executes a grasp on the food in order to eat it (it is likely that cells with these prop- erties were dubbed “logically related,” not because there is a formal relationship b­ etween their eliciting conditions, but to acknowledge that, unlike other MNs, they do not match or “mirror” observed and executed actions). MNs do not respond to the presentation of objects alone (di Pellegrino et al., 1992). However, “canonical neurons,” which are active during object observation and also during execution of an action that is commonly performed on that object, are located alongside MNs in both

518 Caroline Catmur, Clare Press, and Cecilia Heyes Motor properties Sensory properties Strictly congruent Mirror neurons Broadly congruent Or Or Logically-related Canonical neurons Figure 20.2  Types of MN in the macaque. Typical sensory properties of four different types of sensorimotor neuron are shown; for simplicity, the same motor property (a precision grip) is shown for each MN type. premotor and parietal areas (Murata et al., 1997; Murata, Gallese, Luppino, Kaseda, & Sakata, 2000). Macaque MNs fire during execution and observation of a broad range of hand and mouth actions. The hand actions include grasping, placing, manipulating with the fingers, and holding (di Pellegrino et al., 1992). The mouth actions include ingestive behaviors, such as breaking food items, chewing, and sucking; and communicative gestures, such as lip‐smacking, lip protrusion, and tongue protrusion (Ferrari, Gallese, Rizzolatti, & Fogassi, 2003). MNs in humans Only one study offers single‐cell recording evidence of MNs in the human brain (Mukamel et al., 2010). However, a considerable body of evidence from neuroimag- ing, TMS, and behavioral studies, summarized in the following subsections, suggests that human brains contain Mirror Neurons or comparable “mirror mechanisms” (Glenberg, 2011; referred to throughout this chapter as “MNs”).

Mirror Neurons from Associative Learning 519 Functional magnetic resonance imaging (fMRI) has identified regions of PMC and inferior parietal areas that respond during both action observation and execution (Gazzola & Keysers, 2009; Iacoboni et al., 1999; Vogt et al., 2007). More recently, “repetition suppression” effects, whereby the neural response is reduced when events activating the same neuronal population are repeated (Grill‐Spector, Henson, & Martin, 2006), provide further evidence for the presence of “mirror” neuronal popu- lations. Action observation followed by execution of the same action, or vice versa, elicits a suppressed response in inferior parietal regions (Chong et al., 2008; Lingnau, Gesierich, & Caramazza, 2009) and in PMC (Kilner et al., 2009; Lingnau et al., 2009), indicating that the same neuronal population is active when observing and executing the same action. Multivariate pattern analysis has also revealed cross‐modal action‐specific representations consistent with the presence of “mirror” neuronal populations (Oosterhof et al., 2012): A “classifier” program trained to discriminate neural responses to the execution of different actions can subsequently, when tested with neural responses to the observation of those actions, detect which action was observed, suggesting that the same neural representations encode action observation and execution. “Mirror” patterns of MEPs further suggest a human mirror mechanism (Fadiga, Fogassi, Pavesi, & Rizzolatti, 1995). When TMS is applied to M1 during passive action observation, the amplitude of MEPs recorded from the muscles required to execute that action increases. For example, observing index and little finger move- ments selectively facilitates the amplitude of MEPs recorded from the muscles respon- sible for index and little finger movements (Catmur, Mars, Rushworth, & Heyes, 2011). That action observation selectively increases corticospinal excitability to action relevant muscles is suggestive of “mirror” sensorimotor connectivity. Behaviorally, automatic imitation occurs when observation of an action involun- tarily facilitates performance of a topographically similar action and/or interferes with performance of a topographically dissimilar action (Brass, Bekkering, & Prinz, 2001; Stürmer, Aschersleben, & Prinz, 2000). Humans show robust automatic imitation when they observe hand, arm, foot, and mouth movements (Heyes, 2011). This is regarded by many researchers as evidence of a human mirror mechanism (Ferrari, Bonini, & Fogassi, 2009; Iacoboni, 2009; Kilner, Paulignan, & Blakemore, 2003). MNs’ “mirrorness”: Adaptation or association? Here, we outline the standard, adaptation account of the origin of MNs, and the alternative associative account. Both accounts assume that genetic information and experience contribute to the development of MNs. They differ in the roles they assign to genetic evolution and to learning in producing MNs’ characteristic matching properties. The adaptation account suggests that the matching properties of MNs are an adaptation for action understanding and/or related social cognitive abilities (the term “adaptation” is used here to describe a phenotypic characteristic that is genetically inherited, and that was favored by natural selection to fulfill a particular function or “purpose”; Williams, 1966). Specifically, the adaptation account assumes that among common ancestors of macaques and humans, some individuals had a stronger genetic predisposition to develop MNs with matching properties, and that these individuals

520 Caroline Catmur, Clare Press, and Cecilia Heyes were more reproductively successful than those with a weaker genetic predisposition because the development of MNs enhanced their capacity to understand others’ actions. Consequently, a genetic predisposition to develop MNs became universal, or nearly universal, in macaques and humans. The adaptation account further suggests that motor experience (executing actions) and/or visual experience (observing actions) plays a facilitative or “triggering” (Gottlieb, 1976; Ariew, 2006) role in the development of MNs, but their “mirror,” sensory‐to‐motor matching properties are due to this genetic predisposition. The adaptation account has largely been set out in discussions of the “evolution” of MNs (Gallese & Goldman, 1998; Rizzolatti & Arbib, 1998; Rizzolatti & Craighero, 2004; Rochat et al., 2010). For example, it was suggested that “the mirror neuron mechanism is a mechanism of great evolutionary importance through which primates understand actions done by their conspecifics” (Rizzolatti & Craighero, 2004, p. 172). A number of discussions have also suggested that MNs are present at birth (Ferrari et al., 2009; Gallese et al., 2009; Lepage & Theoret, 2007; Rizzolatti & Fadiga, 1998), a feature commonly associated with adaptations (Mameli & Bateson, 2006). In contrast, the associative account suggests that the matching properties of MNs are not a product of a specific genetic predisposition, but instead result from domain‐ general processes of associative learning (Catmur et al., 2014; Cook et al., 2014; Heyes, 2010). Associative learning is found in a wide range of vertebrate and inverte- brate species, indicating that it is an evolutionarily ancient and highly conserved adaptation for tracking predictive relationships between events (Heyes, 2012; Schultz & Dickinson, 2000). Figure 20.3 represents a theory (Heyes, 2010; Heyes & Ray, 2000) of how MNs might acquire their matching properties through sensorimotor associative learning. Before associative learning, sensory neurons responsive to different high‐level visual properties of observed action (Oram & Perrett, 1994, 1996) are weakly connected, directly or indirectly, to motor neurons in parietal cortex (Gallese et al., 2002) and PMC (Rizzolatti et al., 1988). Although some of these connections may be stronger than others, the links between sensory and motor neurons coding similar actions are not consistently stronger than other, nonmatching links. Correlated (i.e., contiguous and contingent) excitation of sensory and motor neurons that code similar actions produces MNs. For example, when an adult imitates an infant’s facial movements, there might be correlated excitation of neurons that are responsive to the observa- tion and execution of lip protrusion. Correlated excitation of the sensory and motor neurons increases the strength of the connection between them, so that subsequent excitation of the sensory neuron propagates to the motor neuron. Thereafter, the motor neuron fires, not only during execution of lip protrusion, but also during observation of lip protrusion, via its connection with the sensory neuron; what was originally a motor neuron has become a lip protrusion MN. In humans, there are many possible sources of correlated excitation of sensory and motor neurons encod- ing the same action. It  occurs not only when we are imitated, but also when we observe our own actions – directly or using an optical mirror; when we observe others during synchronous activities – for example, in sports and dance training; and via “acquired equivalence” experience, for example, when the same sound (a word, or a sound produced by an action, e.g. lip‐smacking) is paired sometimes with obser- vation of an action and s­ometimes with its execution (Ray & Heyes, 2011). In all of

Mirror Neurons from Associative Learning 521 (A) Before learning S1 S2 Sn c M1 M2 Mn (B) During learning a b S1 S2 Sn M1 M2 Mn d (C) After learning Sn S1 S2 M1 M2 Mirrorn Figure  20.3  MNs from associative learning. (A) Before learning, sensory neurons encoding visual descriptions of observed action are not systematically connected to motor neurons in parietal and premotor areas involved in the production of similar actions. (B, a–d) Through social interaction and self‐observation in the course of typical development, agents receive correlated sensorimotor experience; they see and do the same action at about the same time (contiguity), with one event predicting the other (contingency). This experience produces correlated activation of sensory and motor neurons coding similar actions, and, through associative learning, (C) strengthens connections between these neurons. Owing to these connections, neurons that were once involved only in the execution of action will also discharge during observation of a similar action; motor neurons become MNs. Figure reproduced with permission from Heyes (2010). these situations, motor activity is not initiated by, but it is correlated with, observa- tion of matching actions. Thus, the associative account identifies sources in everyday life of the kind of ­correlated sensorimotor experience necessary for MN development, and many of these sources are sociocultural; to a large extent, MNs are built through social interaction.

522 Caroline Catmur, Clare Press, and Cecilia Heyes Another important point to note about the associative account is its emphasis on contingency. Following contemporary associative learning theory, it anticipates that the mature properties of MNs will covary, not only with the number of occasions on which observation of an action has been paired with its execution (contiguity), but also, as a result of context blocking, with the relative predictiveness of observation for execution, or vice versa (contingency; Cook, Press, Dickinson, & Heyes, 2010). Experiments testing the associative account are discussed below. In summary: The associative account implies that the characteristic, matching prop- erties of MNs result from a genetically evolved process, associative learning, but that this process was not “designed” by genetic evolution specifically to produce matching MNs. It just happens to produce matching MNs when the developing system receives correlated experience of observing and executing similar actions. When the system receives correlated experience of observing objects and executing actions, the same associative process produces canonical neurons. When the system receives correlated experience of observing one action and executing a different action, the same associative process produces logically related MNs. Thus, the adaptation account says that genetic evolution has played a specific and decisive role, and learning plays a merely facilitative role, in the development of matching MNs. In contrast, the associative account says that evolution has played a nonspecific background role, and that the characteristic matching properties of MNs are forged or “induced” (Gottlieb, 1976) by sensorimotor learning. Distinguishing the Adaptation and Associative Accounts Here, we present the four evidence‐based arguments that aid in distinguishing ­between the adaptation and associative accounts. The first argument provides the foundation for the adaptation account. It suggests that examination of the field prop- erties of MNs – and, in particular, their “goal” coding – forces the conclusion that MNs are “designed” (Williams, 1966) for action understanding. In the following subsection, we examine the field properties of MNs and suggest that this argument is not compelling. The second argument suggests that research using conditioning procedures shows associative learning to be the right kind of learning to produce MNs. Specifically, the ways in which associative learning tracks contingent relationships, and enables contex- tual modulation of these connections, make it apt to produce MNs (and nonmatching visuomotor neurons) in typical developmental environments. We then draw on research examining the development of MNs and their modifica- tion through sensorimotor experience. First, we discuss research with infants and adults that has been used to support a “poverty of the stimulus” argument (Chomsky, 1975); to suggest that MNs emerge too early in development, after too little sensori- motor experience, to have been forged by associative learning. In contrast, we offer a “wealth of the stimulus” argument. Finally, we focus on evidence that, even in adulthood, the properties of MNs can be changed in radical ways by relatively brief periods of sensorimotor experience. This evidence supports the associative account in two ways: It confirms novel predictions

Mirror Neurons from Associative Learning 523 of the associative account and indicates that the development of MNs is not buffered or protected from perturbation in the way one would expect if MNs were an adaptation for action understanding. Do MNs encode the “goal” of an action? Supporters of the adaptation account (e.g., Rizzolatti & Sinigaglia, 2010) argue that examination of the field properties of MNs indicates that they encode “goals.” They further argue that this property suggests that MNs evolved to mediate action under- standing. We first, therefore, consider how well the neurophysiological data accord with this view. The term “goal” has numerous interpretations (Hickok, 2009). We will consider two commonly adopted definitions, assuming that MNs encode “goals” if they encode (1) object‐directed actions or (2) high‐level action intentions. Early descriptions of MN field properties reported that intransitive, that is nonobject‐­directed, actions (e.g., tongue protrusion) and pantomimed actions (e.g., miming a precision grip without an object) did not elicit MN responses (di Pellegrino et al., 1992; Gallese et al., 1996). In contrast, robust responses were reported when monkeys observed object‐directed actions. This pattern raises the possibility that MNs encode “goals” in the sense that they are responsive only to object‐directed actions. However, a close reading of the single‐cell data suggests that only a small subset of MNs appear to encode action goals in these terms. A subset of the MNs described in the early reports continued to respond, albeit less strongly, to panto- mimed or intransitive actions (di Pellegrino et al., 1992; Gallese et al., 1996, figure  5b). Subsequent studies confirmed that sizable proportions, perhaps the majority, of MNs exhibit robust responses to the observation of object‐free body movements, such as lip‐smacking, lip protrusion, and tongue protrusion (Ferrari et al., 2003). Also, as reported by Kraskov, Dancause, Quallo, Shepherd, and Lemon (2009), 73% of MN responses modulated by observation of object‐directed grasping showed similar modulation during observation of pantomimed grasping. As well as referring to the object of an action, the term “goal” has also been used to refer to what the actor intends to achieve – for example, “grasp in order to eat” (Fogassi et al., 2005) or “taking possession of an object” (Rochat et al., 2010). Rizzolatti and Sinigaglia (2010, p. 269) stated: “only those [neurons] that can encode the goal of the motor behavior of another individual with the greatest degree of gen- erality can be considered to be crucial for action understanding.” The suggestion that MNs encode high‐level action intentions is consistent with reports that some broadly congruent MNs respond to the observation of multiple actions; for example any “grasping” action executed with the hand or mouth (Gallese et al., 1996). It is also made plausible by reports that MN responses to grasping can be modulated by the final outcome of the motor sequence (Bonini et al., 2010; Fogassi et al., 2005). However, the single‐cell data again suggest that relatively few MNs have the field properties one would expect of a system designed to represent high‐level action inten- tions. For example, Gallese et al. (1996) reported that during action observation, 37.5% of MNs responded differently depending on whether the action was executed with the left or right hand, and 64% showed direction sensitivity, preferring either left‐to‐right or right‐to‐left grasping actions. Similarly, many MNs (53%) respond

524 Caroline Catmur, Clare Press, and Cecilia Heyes selectively to the observation of actions executed within (“peripersonal” MNs) or beyond (“extrapersonal” MNs), not the actor’s, but the observing monkey’s reach (Caggiano, Fogassi, Rizzolatti, Thier, & Casile, 2009). The majority (74%) of MNs also exhibit view‐dependent responses; some MNs are tuned to egocentric (first‐ person) presentation, while others respond maximally to allocentric (third‐person) perspectives (Caggiano et al., 2011). Each of these classes of MN is sensitive to f­eatures of action that fall well below the “greatest degree of generality,” and of inten- tions such as “grasping in order to eat” or “taking possession of an object.” Associative learning: the right kind of learning to generate MN field properties? The previous subsection suggested that many MNs have field properties incompatible with the hypothesis that they were designed by evolution to mediate action understanding via goal coding. Here, in complementary fashion, we argue that research on the roles of contingency and contextual modulation in associative learning enables the associative account to provide a unified explanation of all MN field properties reported to date. Associative learning depends not only on contiguity – events occurring close together in space and time – but also on contingency: the degree to which one event reliably predicts the other (Elsner & Hommel, 2004; Rescorla, 1968; Schultz & Dickinson, 2000). The associative account therefore anticipates that MNs will acquire sensorimotor matching properties only when an individual experiences systematic contingencies between sensory events and performed actions (Cooper, Cook, Dickinson, & Heyes, 2013). This feature of associative learning ensures that the matching properties of MNs reflect sensorimotor relationships that occur reliably in the individual’s environment, rather than chance co‐occurrences. Cook, Press, Dickinson, and Heyes (2010) described evidence that the human mirror mechanism is modified by contingent, but not by noncontingent, sensorimotor experience. Sensitivity to contingency explains the mix of strictly congruent MNs, sensitive to the low‐level features of observed actions (type of grip, effector used, direction of movement, viewpoint, proximity to the observer), and broadly congruent MNs, responsive to multiple related actions irrespective of the manner of their execution. Both visual and motor systems are known to be organized hierarchically (Jeannerod, 1994; Perrett et al., 1989), comprising different populations encoding relatively low‐ level (e.g., descriptions of particular “precision” or “power” grips) and more abstract representations (e.g., descriptions of “grasping”). Crucially, contingencies can be experienced between both low‐ and high‐level sensory and motor representations. When a monkey observes itself performing a precision grip, the excitations of sensory and motor populations encoding a specific grip (low‐level) are correlated. However, during group feeding, a monkey might observe and perform a range of grasping actions, thereby causing correlated excitation of higher‐level visual and motoric descriptions of grasping. Contingency sensitivity therefore explains the existence of both strictly congruent MNs, tuned to a particular sensory representation (e.g., a right‐to‐left precision grip executed with the right hand viewed allocentrically in extrapersonal space), and broadly congruent MNs, responsive to the observation of a number of related actions (see Figure 20.4).

Mirror Neurons from Associative Learning 525 Situation Performed Observed Subsequent effective visual Type of mirror neuron A. Self-observation action action produced input for neuron to fire Strictly congruent B. Group feeding and / or or Broadly congruent C. Laboratory-based training: and / or or i. No pot present: grasp-to-eat Strictly congruent ii. Pot present: grasp-to-place Context-dependent and Figure  20.4  Examples of contingencies that would produce (A) strictly congruent, (B)  broadly congruent, and (C) context‐dependent (“grasp‐to‐place”) MNs. (A) When a monkey watches its own actions while feeding, alone or in a group, the probability of seeing a particular grip (e.g., a precision grip) while performing exactly the same grip is high. (B) When a monkey watches the actions of others during group feeding, the probability of seeing a range of grasping actions while performing a particular (e.g., precision) grip is also high (and, cru- cially, it is higher than the probability of seeing an unrelated action, e.g. a kick). (C) Before testing for the presence of “grasp‐to‐place” MNs, monkeys are trained: (i) when a pot is not present, food items should be eaten, but (ii) when a pot is present, food items should be placed in the pot (in return for a higher‐value food reward). Self‐observation during this training ensures that, in the presence of a pot, the probability of seeing a grasp‐to‐place action while performing a grasp‐to‐place action is high. Subsequently, in the presence of a pot, the sight of a grasping action activates grasp‐to‐place (rather than grasp‐to‐eat) motor commands. Contingency sensitivity also explains other MN properties. According to the associative account, MNs acquire sensorimotor properties whenever individuals e­xperience a contingency between “seeing” and “doing.” Crucially, there is no requirement that contingencies be between action execution and observation of the same action. Both monkeys and humans frequently experience nonmatching sensori- motor contingencies, where the observation of one action predicts the execution of another; for example, you release, and I grasp (Newman‐Norlund, van Schie, van Zuijlen, & Bekkering, 2007). The associative account therefore explains the existence of logically related MNs that respond to different actions in observe and execute c­onditions. Equally, there is no requirement that contingencies be between action execution and the perception of “natural” action‐related stimuli, such as the sight of animate motion or sounds that could have been heard by ancestors of contemporary monkeys. Thus, the associative account explains why “tool‐use” MNs (Ferrari, Rozzi, & Fogassi, 2005) develop when action execution (e.g., grasping a food item) is reliably predicted by the sight of actions performed with tools (e.g., seeing food items being gripped with pliers) and why “audiovisual” MNs (Keysers et al., 2003; Kohler et al., 2002) develop when action performance predicts characteristic action sounds (e.g., paper tearing or plastic crumpling; Cook, 2012): There is a high contingency ­between the sight of the experimenter gripping food with pliers and the subsequent execution of a grasp by the macaque; and between the sound of paper tearing and the execution of the ripping action that produces that sound.

526 Caroline Catmur, Clare Press, and Cecilia Heyes Studies of conditioning that have supported the role of contingency indicate that learned responses acquired under contingency control are often also subject to ­contextual control; if a stimulus is associated with two responses, each in a different context, then the context determines which association, representing a response–­ outcome contingency, is cued by the stimulus (Bouton, 1993, 1994; Peck & Bouton, 1990). For example, Peck and Bouton (1990) initially placed rats in a conditioning chamber with a distinctive scent (e.g., coconut) where they learned to expect electric shock following a tone. The rats were then transferred to a second chamber with a different scent (e.g., aniseed) where the same tone predicted the delivery of food. The rats quickly learned the new contingency, and conditioned foraging responses replaced conditioned freezing. However, learning in the second phase was context dependent. When returned to the first chamber, or transferred to a third chamber with a novel scent, the tone once again elicited freezing. The associative account of MN properties draws on the components of associative learning theory that explain this kind of effect. Using associative learning theory in this way, several findings from the MN litera- ture can be interpreted in terms of contextual modulation of MN firing (Cook, Dickinson, & Heyes, 2012). For example, some MNs show stronger visual responses to object‐directed grasping than to pantomimed grasping in object‐absent contexts (Gallese et al., 1996), and in some cases, the modulating influence of the object ­context can be seen even when the target object is occluded prior to contact with the hand (Umilta et al., 2001). Similarly, MN responses during the observation of grasp- ing may be modulated by the type of object being grasped (Caggiano et al., 2012), with some MNs responding strongly in the presence of high‐value objects (food, non- food objects predictive of reward), and some in the presence of low‐value objects (nonfood objects not associated with reward). In the clearest example, the same motor act, grasping with a precision grip, elicits different MN responses dependent on whether the action is observed in the presence (“grasp to place”) or absence (“grasp to eat”) of a plastic cup (Bonini et al., 2010; Fogassi et al., 2005). Rather than the plastic cup providing a cue to the actor’s intention, it may act as a cue modu- lating the operation of two associations. In the same way that the sound of the tone elicited different behaviors when presented in the coconut and aniseed contexts (Peck & Bouton, 1990), observing a precision grip may excite different MNs in the cup‐ present and cup‐absent contexts (see Figure  20.4). Thus, while many of the field properties described above are frequently cited as e­ vidence of goal (intention) coding by MNs, they are equally consistent with contextual modulation within an associative framework. Sufficient opportunity for learning before MNs emerge? MNs have not been measured directly in neonates. However, other research involving infants has been used to support a “poverty of the stimulus” (Chomsky, 1975) argument suggesting that MNs emerge too early in development, after too little sensorimotor experience, to have been forged by associative learning. Specifically, it has been claimed that imitation is mediated by MNs, and that both human and macaque infants are able to imitate when they have had minimal oppor- tunity for sensorimotor learning. However, the evidence supporting the second claim is not compelling. Building on previous analyses (e.g., Anisfeld, 1996), a

Mirror Neurons from Associative Learning 527 recent review found evidence that human neonates “copy” only one action – tongue protrusion – and that, since tongue protrusion occurs in response to a range of arousing stimuli, this “copying” does not show the specificity that is characteristic of imitation or of MNs (Ray & Heyes, 2011). Turning to macaque infants, Ferrari et al. (2006) reported immediate imitation of tongue protrusion and lip‐smacking in 3‐day‐old macaques. However, the effects were not present on days 1, 7, and 14 postpartum, and it is not clear whether they were rep- licated in a subsequent study (Paukner, Ferrari, & Suomi, 2011). The later study did report imitation of lip‐smacking in macaques less than 1 week old, but this effect seems to have been due to a low frequency of lip‐smacking in the control condition, rather than to an elevated frequency of lip‐smacking when the infants were observing l­ip‐smacking. Therefore, in common with the data from human infants, studies of imi- tation in newborn macaques do not currently support the conclusion that infants can imitate before they have had the opportunity for relevant sensorimotor learning. A related argument has suggested that the associative account must be wrong because suppression of electroencephalographic (EEG) activity in the alpha frequency range (~6–13 Hz) during action observation (and execution) reflects the operation of MNs; and that both human and macaque infants show alpha suppression when they have had minimal opportunity for sensorimotor learning. In this case, both of the claims are weak. Alpha suppression is found over central cortical regions when observing and executing actions, but it may not reflect the activity of MNs. First, the functional significance of lower band EEG activity is poorly understood, even in adults, and is yet more difficult to interpret in infants where, for example, less information is available about the source (Marshall & Meltzoff, 2011). Alpha suppression in other locations is interpreted differently (e.g., as evidence of increased visual processing), and the only neonatal action observation study (Ferrari et al., 2012) has insufficient spatial resolu- tion to provide source information. Second, adult studies have traced the likely source of alpha suppression during action execution to the somatosensory cortex (Hari & Salmelin, 1997), suggesting that alpha suppression during action observation may not index motor processing (and thus MNs) at all (Coll, Bird, Catmur, & Press, 2015). Third, even if alpha suppression does index motor processing, it does not show that the motor activation matches or mirrors the observed actions (Marshall & Meltzoff, 2011). Thus, alpha suppression during observation of lip‐smacking, which has been reported in neonatal monkeys (Ferrari et al., 2012), may reflect a generalized readiness to act, or motor activation of tongue protrusion or hand movement, rather than motor activation of lip‐smacking. Furthermore, it has not been shown that alpha suppression occurs when infants have had insufficient correlated sensorimotor experience to build MNs through associative learning. Indeed, studies of human infants suggest an age‐ related trend consistent with the associative account: For example, Nyström (2008) found no evidence of alpha suppression when 6‐month‐old infants observed actions, but effects have been obtained at 9 and 14 months (Marshall, Young, & Meltzoff, 2011; Southgate, Johnson, El Karoui, & Csibra, 2010). It is important to note that although MN activity in newborns would be incon- sistent with the associative model, the associative account is predicated on a “wealth of the stimulus” argument, and therefore anticipates MN activity in young infants following sufficient correlated sensorimotor experience (Ray & Heyes, 2011). This “wealth argument” points out that typical human developmental environments

528 Caroline Catmur, Clare Press, and Cecilia Heyes contain multiple sources of the kind of correlated sensorimotor experience necessary to build MNs; that each of these sources is rich; and that the mechanisms of associative learning can make swift and efficient use of these sources. The range of sources available to young human infants includes self‐observation, being imi- tated by adults, being rewarded by adults for imitation, and acquired equivalence experience in which, for example, the infant hears the same tapping sound when she hits an object herself and when she sees the object hit by another person. A common misconception about associative learning is that it always occurs slowly. On the con- trary, when contingency is high, infants can learn action–effect associations in just a few trials (Paulus, Hunnius, van Elk, & Bekkering, 2012; Verschoor, Weidema, Biro, & Hommel, 2010) and human adults demonstrate rapid learning even with complex contingencies (e.g., Baker, Vallée‐Tourangeau, & Murphy, 2000). Influence of sensorimotor learning The associative account has been explicitly tested in experiments examining the effects of laboratory‐based sensorimotor training on MNs in human adults. Building on the results of more naturalistic studies (Calvo‐Merino, Glaser, Grezes, Passingham, & Haggard, 2005; Calvo‐Merino, Grezes, Glaser, Passingham, & Haggard, 2006; Ferrari et al., 2005; Vogt et al., 2007), these experiments have isolated the effects of sensorimotor experience from those of purely visual and purely motor experience. Using all the measures of MN activity commonly applied to humans (imitation, motor evoked potentials, and fMRI measures including repetition suppression), they have shown that relatively brief periods of sensorimotor experience can enhance (Press, Gillmeister, & Heyes, 2007), abolish (Cook et al., 2010, 2012; Gillmeister, Catmur, Liepelt, Brass, & Heyes, 2008; Heyes, Bird, Johnson, & Haggard, 2005; Wiggett, Hudson, Tipper, & Downing, 2011), reverse (Catmur et al., 2008, 2011; Catmur, Walsh, & Heyes, 2007; Cavallo, Heyes, Becchio, Bird, & Catmur, 2014), and induce (Landmann, Landi, Grafton, & Della‐Maggiore, 2011; Petroni, Baguear, & Della‐Maggiore, 2010; Press et al., 2012) MN activity (details below). These ­findings reveal the kind of flexibility one would expect if MNs are forged by sensorimotor associative learning. In contrast, this kind of flexibility is hard to reconcile with the adaptation account. If MNs were a genetic adaptation, one would expect their development to be protected or “buffered” against environmental perturbations that were occurring when MNs evolved and that could interfere with their adaptive function (Cosmides & Tooby, 1994; Pinker, 1997). Thus, if MNs are indeed an adaptation for “action understanding,” their development should be buffered to ­prevent them from coding stimulus–response and response–outcome relationships that could interfere with that function. For example, MNs should be prevented from coding inanimate, rather than action, stimuli; and from coding dissimilar, rather than similar, observed and executed actions. Evidence that MNs are not resistant to coding inanimate stimuli comes from studies showing that arbitrary sound, color and shape stimuli can induce mirror motor evoked potentials (D’Ausilio, Altenmüller, Olivetti Belardinelli, & Lotze, 2006; Petroni et al., 2010), fMRI responses (Landmann et al., 2011; Press et al., 2012), and behavioral effects (Press et al., 2007) following sensorimotor training. For example, Press and colleagues (2007) gave participants approximately 50 min of sensorimotor training in

Mirror Neurons from Associative Learning 529 which they repeatedly opened their hand when seeing a robotic pincer open, and closed their hand when seeing the robotic pincer close. Prior to this training, the pincer movement elicited less automatic imitation than human hand movement, but 24 hr after training, the automatic imitation effect was as strong for the pincer movement as for the human hand. Evidence that MNs are not resistant to coding dissimilar actions comes from studies showing that nonmatching (or “counter‐mirror”) sensorimotor training abolishes automatic imitation (Cook et al., 2010, 2012; Gillmeister et al., 2008; Heyes et al., 2005; Wiggett et al., 2011) and reverses both fMRI (Catmur et al., 2008) and MEP mirror responses (Catmur et al., 2007). For example, Catmur and colleagues (2007) gave participants approximately 90 min of nonmatching sensorimotor training in which they repeatedly made an index‐finger movement while observing a little‐finger movement, and vice versa. Before this training, they showed mirror MEP responses. That is, observation of index‐finger movement elicited more activity in an index‐ finger muscle than observation of little‐finger movement, and vice versa for the little‐ finger muscle. After training, this pattern was reversed. For example, observation of index‐finger movement elicited more activity in the little‐finger muscle than observa- tion of little‐finger movement. Similarly, following sensorimotor training in which observation of hand actions was paired with execution of foot actions and vice versa, fMRI responses to action observation were reversed: Premotor and parietal areas n­ ormally more responsive to the sight of hand actions now showed stronger responses to observation of foot actions (Catmur et al., 2008). Thus, a substantial body of evidence from studies of training and expertise has ­confirmed the predictions of the associative account, showing that mirror responses can be changed in radical ways by sensorimotor learning. In particular, these studies suggest that MNs are not buffered or protected against sensorimotor experience of a kind that makes them code inanimate stimuli and dissimilar actions. Investigating the contribution of MNs to social behavior The associative account suggests that MNs do not have a specific biological purpose or “adaptive function,” distinct from that of other neurons with visuomotor properties. However, the associative account leaves open the possibility that MNs are recruited in the course of development to contribute to one or more “psychological functions.” They could be useful – possibly they could contribute to a variety of social functions – without having been designed by evolution for a particular use. Thus, the associative account is functionally permissive; however, it implies that a radically new approach is required to find out what, if anything, MNs contribute to social behavior. Theories relating to MN function have mainly been inspired by “reflection” on the field properties of MNs found in a sample of laboratory monkeys with unreported (and usually unknown) developmental histories. This method asks what neurons with these field properties might enable the animal to do. For example, early reports that MNs discharged when monkeys saw and produced object‐directed actions inspired the theory that MNs mediate action understanding via “motor resonance.” Even now, opposition to the idea that MNs mediate action understanding tends to be answered by focusing on the conditions in which they fire (Gallese et al., 2011). The associative account suggests that the “reflection” method needs to be changed and

530 Caroline Catmur, Clare Press, and Cecilia Heyes extended by embedding MN research in system‐level theories of social behavior, by considering individuals’ developmental history, and by carrying out experimental investigation of MN function. If MNs were an adaptation, one could argue that new categories of psychological functioning – such as “action understanding” and “motor resonance” – are necessary to characterize what they do. In contrast, by showing that established psychological theory – associative learning theory – can cast light on the origin of MNs, the associative account underlines the value of embedding research on MN function within system‐level psychological and computational theories of how the brain pro- duces behavior. This implies that hypotheses about MN function should specify a part in a process – a process that goes all the way from peripheral sensory input to overt motor output – that MNs are thought to fulfill. The name assigned to this part is not important in itself. What is important is that the hypothetical function of MNs is dis- tinguished clearly from other components of the same overall process. For example, in this kind of system‐level, theory‐guided approach, “action understanding” would be distinguished from components that are more purely perceptual (which might be called “action perception” or “action recognition”) or more purely motoric (e.g., “action execution”), or constitute a higher level of “understanding” (e.g., mentaliz- ing). This approach would also make it clear whether the hypothetical function is thought to be optional or obligatory; whether it can be, or must be, done by MNs. The kind of system‐level theoretical approach required in research on the functions of MNs is exemplified by studies of their role in speech perception (Lotto, Hickok, & Holt, 2009; Scott, McGettigan, & Eisner, 2009). Regarding MN development, if MNs were an adaptation, it is likely that their prop- erties would be relatively invariant across developmental environments. Therefore, it would be possible to make valid inferences about species‐typical properties of MNs based on a relatively small and developmentally atypical sample of individuals. If MNs are instead a product of associative learning, this kind of inference is not valid. Whether or not an individual has MNs, which actions are encoded by their MNs, and at what level of abstraction, will all depend on the types of sensorimotor experience received by the individual in the course of their development. Therefore, the associative account implies that it is crucial for studies of laboratory monkeys to report, and ide- ally to control, the animals’ developmental history; the kinds of sensorimotor experi- ence to which they have been exposed. A corollary of this is that we cannot assume that the mirror mechanisms found in the members of one human culture are repre- sentative of the whole human species. With its emphasis on the role of social practices in driving the development of MNs, the associative account provides specific, theory‐ driven motivation for cross‐cultural studies of mirroring. In terms of function, a system‐level theoretical approach would overcome a problem that has haunted discussions of the action understanding hypothesis since MNs were discovered: Is this hypothesis claiming that MN activity causes or constitutes action understanding? The former is an empirically testable hypothesis suggesting that there is a distinctive behavioral competence, called action understanding, to which the activity of MNs contributes. The latter implies that the firing of MNs during action observation is, in itself, a form of action understanding; it does not need to have further consequences in order to qualify as action understanding. This claim is not sub- ject to empirical evaluation; it is true, or otherwise, by virtue of the meanings of words.

Mirror Neurons from Associative Learning 531 Empirical (rather than constitutive) claims about the function of MNs need to be tested by experiments looking for, at a minimum, covariation between MN activity and behavioral competence, and, ideally, testing for effects on behavioral competence of interventions that change MN activity. At present, this research faces two major challenges. First, because the hypothetical functions of MNs typically are not defined in the context of a system‐level theory, it is difficult to design appropriate control tasks. For example, if an experiment is testing the hypothesis that MNs play a causal role in action understanding, should it control for the possibility that they instead play some role in action perception? If so, what kind of behavioral competence is indicative of action perception rather than action understanding? To date, only a small number of studies (e.g., Pobric & Hamilton, 2006) have made a serious attempt to tackle this problem. The second challenge is that, with rare exceptions (Mukamel et al., 2010), MN activity cannot be localized precisely within the human brain. Consequently, many studies assume that activity in the ventral PMC and IPL – areas homologous to those in which MNs have been found in macaques – is MN activity, and that behavioral changes brought about through interference with the functioning of these areas are due to interference with MNs. This is unsatisfactory because, in macaques, it is likely that fewer than 20% of the neurons in these classical mirror areas are actually MNs, and because there is evidence of MNs in nonclassical areas in both macaques and humans (see first section). Techniques such as fMRI repetition suppression, multivar- iate pattern analysis, and TMS adaptation (Cattaneo et al., 2011; Silvanto, Muggleton, Cowey, & Walsh, 2007) hold some promise as means of overcoming the localization problem with human participants, by isolating behavioral effects to specific popula- tions of neurons. Guided by system‐level theory, future studies could use these tech- niques with a range of tasks to isolate the processes in which MNs are involved. Alongside such future studies with human participants, animal studies could be con- ducted, not only to document the field properties of MNs, but to examine how those properties relate to behavioral competence. For example, are animals with MNs for actions X and Y better than other animals of the same species at behavioral discrimination of X and Y, or at imitating X and Y? Studies of this kind have been dismissed as imprac- tical on the assumption that they would have to involve monkeys, and that between‐ group variation in MN activity would have to be induced via lesions or disruptive TMS. However, the associative account suggests that between‐group variation in the number and type of MNs could be induced using sensorimotor training, either in monkeys or by establishing a rodent model. If the associative account is correct, rodents, birds, and other animals are likely to have the potential to develop MNs because they are capable of associative learning. Whether or not they receive in the course of typical development the sensorimotor experience necessary to realize this potential, relevant sensorimotor training could be provided in the laboratory. Conclusion The associative account of the origin of MN properties paves the way for an alternative approach to MN research. It acknowledges that MNs were a fascinating discovery and is open to the possibility that they play one or more important roles in social

532 Caroline Catmur, Clare Press, and Cecilia Heyes interaction. It differs from the adaptation account in suggesting that sensorimotor learning plays a crucial, inductive role in the development of MNs, and, because of this, we will obtain reliable information about the function of MNs only by applying an approach based on system‐level theory, developmental history, and experimenta- tion. These methodological implications underline the fact that, relative to the adaptation account, the associative account shifts the balance of explanatory power from MNs themselves to the environments in which they develop. In some ways, this is inconvenient because developmental environments are much harder to study in the laboratory, but there are significant potential payoffs. As a rich source of testable predictions about when, where, and how MNs develop, associative learning theory can provide clear guidance for future research on the taxonomic distribution, typical properties, and functional roles of MNs. Acknowledgments CC is supported by the ESRC (ES/K00140X/1). We are also very grateful to Richard Cook and Geoff Bird with whom we developed the associative account of MNs. References Anisfeld, M. (1996). Only tongue protrusion modeling is matched by neonates. Developmental Review, 16, 149–161. Ariew, A. (2006). Innateness. In M. Matthen & C. Stevens (Eds.), Handbook of the philosophy of science (Vol. 3, pp. 567–584). Oxford, UK: Elsevier. Baker, A. G., Vallée‐Tourangeau, F., & Murphy, R. A. (2000). Asymptotic judgment of cause in a relative validity paradigm. Memory & Cognition, 28, 466–479. Blakesee, S. (2006). Cells that read minds. The New York Times, January 10. Bonini, L., Rozzi, S., Serventi, F. U., Simone, L., Ferrari, P. F., & Fogassi, L. (2010). Ventral premotor and inferior parietal cortices make distinct contribution to action organization and intention understanding. Cerebral Cortex, 20, 1372–1385. Bouton, M. E. (1993). Context, time, and memory retrieval in the interference paradigms of Pavlovian learning. Psychological Bulletin, 114, 80–99. Bouton, M. E. (1994). Context, ambiguity, and classical‐conditioning. Current Directions in Psychological Science, 3, 49–53. Brass, M., Bekkering, H., & Prinz, W. (2001). Movement observation affects movement exe- cution in a simple response task. Acta Psychologica, 106, 3–22. Caggiano, V., Fogassi, L., Rizzolatti, G., Casile, A., Giese, M. A., & Thier, P. (2012). Mirror neurons encode the subjective value of an observed action. Proceedings of the National Academy of Sciences of the United States of America, 109, 11848–11853. Caggiano, V., Fogassi, L., Rizzolatti, G., Pomper, J. K., Thier, P., Giese, M. A., & Casile, A. (2011). View‐based encoding of actions in mirror neurons of area f5 in macaque premotor cortex. Current Biology, 21, 144–148. Caggiano, V., Fogassi, L., Rizzolatti, G., Thier, P., & Casile, A. (2009). Mirror neurons differ- entially encode the peripersonal and extrapersonal space of monkeys. Science, 324, 403–406.

Mirror Neurons from Associative Learning 533 Calvo‐Merino, B., Glaser, D. E., Grezes, J., Passingham, R. E., & Haggard, P. (2005). Action observation and acquired motor skills: An fMRI study with expert dancers. Cerebral Cortex, 15, 1243–1249. Calvo‐Merino, B., Grezes, J., Glaser, D. E., Passingham, R. E., & Haggard, P. (2006). Seeing or doing? Influence of visual and motor familiarity in action observation. Current Biology, 16, 1905–1910. Catmur, C., Gillmeister, H., Bird, G., Liepelt, R., Brass, M., & Heyes, C. (2008). Through the looking glass: counter‐mirror activation following incompatible sensorimotor learning. European Journal of Neuroscience, 28, 1208–1215. Catmur, C., Mars, R. B., Rushworth, M. F., & Heyes, C. (2011). Making mirrors: Premotor cortex stimulation enhances mirror and counter‐mirror motor facilitation. Journal of Cognitive Neuroscience, 23, 2352–2362. Catmur, C., Press, C., Cook, R., Bird, G., & Heyes, C. M. (2014) Mirror neurons: tests and testability. Behavioral and Brain Sciences, 37, 221–241. Catmur, C., Walsh, V., & Heyes, C. (2007). Sensorimotor learning configures the human mirror system. Current Biology, 17, 1527–1531. Cattaneo, L., Barchiesi, G., Tabarelli, D., Arfeller, C., Sato, M., & Glenberg, A. M. (2011). One’s motor performance predictably modulates the understanding of others’ actions through adaptation of premotor visuo‐motor neurons. Social Cognitive and Affective Neuroscience, 6, 301–310. Cavallo, A., Heyes, C., Becchio, C., Bird, G., & Catmur, C. (2014). Timecourse of mirror and counter‐mirror effects measured with transcranial magnetic stimulation. Social Cognitive and Affective Neuroscience, 9, 1082–1088. Chomsky, N. (1975). Reflections on language. New York, NY: Pantheon Books. Chong, T. T. J., Cunnington, R., Williams, M. A., Kanwisher, N., & Mattingley, J. B. (2008). fMRI adaptation reveals mirror neurons in human inferior parietal cortex. Current Biology, 18, 1576–1580. Coll, M.‐P., Bird, G., Catmur, C., & Press, C. (2015). Cross‐modal repetition effects in the mu rhythm indicate tactile mirroring during action observation. Cortex, 63, 121–131. Cook, R. (2012). The ontogenetic origins of mirror neurons: evidence from “tool‐use” and “audiovisual” mirror neurons. Biology Letters, 8, 856–859. Cook, R., Bird, G., Catmur, C., Press, C., & Heyes, C. M. (2014). Mirror neurons: from origin to function. Behavioral and Brain Sciences, 37, 177–192. Cook, R., Dickinson, A., & Heyes, C. (2012). Contextual modulation of mirror and coun- termirror sensorimotor associations. Journal of Experimental Psychology: General, 141, 774–87. Cook, R., Press, C., Dickinson, A., & Heyes, C. (2010). Acquisition of automatic imitation is sensitive to sensorimotor contingency. Journal of Experimental Psychology: Human Perception and Performance, 36, 840–852. Cooper, R., Cook, R., Dickinson, A., & Heyes, C. (2013). Associative (not Hebbian) learning and the mirror neuron system. Neuroscience Letters, 540, 28–36. Cosmides, L., & Tooby, J. (1994). Beyond intuition and instinct blindness: toward an evolutionary rigorous cognitive science. Cognition, 50, 41–77. D’Ausilio, A., Altenmüller, E., Olivetti Belardinelli, M., & Lotze, M. (2006). Cross‐modal plasticity of the motor cortex while listening to a rehearsed musical piece. European Journal of Neuroscience, 24, 955–958. di Pellegrino, G., Fadiga, L., Fogassi, L., Gallese, V., & Rizzolatti, G. (1992). Understanding motor events: a neurophysiological study. Experimental Brain Research, 91, 176–180. Dushanova, J., & Donoghue, J. (2010). Neurons in primary motor cortex engaged during action observation. European Journal of Neuroscience, 31, 386–398.

534 Caroline Catmur, Clare Press, and Cecilia Heyes Elsner, B., & Hommel, B. (2004). Contiguity and contingency in action–effect learning. Psychological Research, 68, 138–154. Fadiga, L., Fogassi, L., Pavesi, G., & Rizzolatti, G. (1995). Motor facilitation during action observation: a magnetic stimulation study. Journal of Neurophysiology, 73, 2608–2611. Ferrari, P. F., Bonini, L., & Fogassi, L. (2009). From monkey mirror neurons to primate behaviours: possible “direct” and “indirect” pathways. Philosophical Transactions of the Royal Society B: Biological Sciences, 364, 2311–2323. Ferrari, P. F., Gallese, V., Rizzolatti, G., & Fogassi, L. (2003). Mirror neurons responding to the  observation of ingestive and communicative mouth actions in the monkey ventral p­ remotor cortex. European Journal of Neuroscience, 17, 1703–1714. Ferrari, P. F., Rozzi, S., & Fogassi, L. (2005). Mirror neurons responding to observation of actions made with tools in monkey ventral premotor cortex. Journal of Cognitive Neuroscience, 17, 212–226. Ferrari, P. F., Vanderwert, R. E., Paukner, A., Bower, S., Suomi, S. J., & Fox, N. A. (2012). Distinct EEG amplitude suppression to facial gestures as evidence for a mirror mechanism in newborn monkeys. Journal of Cognitive Neuroscience, 24, 1165–1172. Ferrari, P. F., Visalberghi, E., Paukner, A., Fogassi, L., Ruggiero, A., & Suomi, S. J. (2006). Neonatal imitation in rhesus macaques. PLoS Biology, 4(e302). Fogassi, L., Ferrari, P. F., Gesierich, B., Rozzi, S., Chersi, F., & Rizzolatti, G. (2005). Parietal lobe: from action organization to intention understanding. Science, 308, 662–667. Gallese, V., Fadiga, L., Fogassi, L., & Rizzolatti, G. (1996). Action recognition in the premo- tor cortex. Brain, 119, 593–609. Gallese, V., Fadiga, L., Fogassi, L., & Rizzolatti, G. (2002). Action representation and the inferior parietal lobule. In W. Prinz & B. Hommel (Eds.), Common mechanisms in per- ception and action: attention and performance XIX. Oxford, UK: Oxford University Press. Gallese, V., Gernsbacher, M., Hickok, G., Heyes, C., & Iacoboni, M. (2011). Mirror neuron forum. Perspectives on Psychological Science, 6, 369–407. Gallese, V., & Goldman, A. (1998). Mirror neurons and the simulation theory of mind‐reading. Trends in Cognitive Sciences, 2, 493–501. Gallese, V., Rochat, M., Cossu, G., & Sinigaglia, C. (2009). Motor cognition and its role in the phylogeny and ontogeny of action understanding. Developmental Psychology, 45, 103–113. Gallese, V., & Sinigaglia, C. (2011). What is so special about embodied simulation? Trends in Cognitive Sciences, 15, 512–519. Gazzola, V., & Keysers, C. (2009). The observation and execution of actions share motor and somatosensory voxels in all tested subjects: single‐subject analyses of unsmoothed fMRI data. Cerebral Cortex, 19, 1239–1255. Gillmeister, H., Catmur, C., Liepelt, R., Brass, M., & Heyes, C. (2008). Experience‐based priming of body parts: A study of action imitation. Brain Research, 1217, 157–170. Glenberg, A. (2011). Introduction to the Mirror Neuron Forum. Perspectives on Psychological Science, 8, 363–368. Gottlieb, G. (1976). The roles of experience in the development of behavior and the nervous system. In G. Gottlieb (Ed.), Neural and behavioral plasticity (pp. 24–54). New York, NY: Academic Press. Grill‐Spector, K., Henson, R., & Martin, A. (2006). Repetition and the brain: neural models of stimulus‐specific effects. Trends in Cognitive Sciences, 10, 14–23. Hari, R., & Salmelin, R. (1997). Human cortical oscillations: A view through the skull. Trends in Neuroscience, 20, 44–49. Heyes, C. (2010). Where do mirror neurons come from? Neuroscience and Biobehavioral Reviews, 34, 575–583.

Mirror Neurons from Associative Learning 535 Heyes, C. (2011). Automatic imitation. Psychological Bulletin, 137, 463–483. Heyes, C. (2012). Simple minds: A qualified defence of associative learning. Philosophical Transactions of the Royal Society B: Biological Sciences, 367, 2695–703. Heyes, C., Bird, G., Johnson, H., & Haggard, P. (2005). Experience modulates automatic imita- tion. Cognitive Brain Research, 22, 233–240. Heyes, C. M., & Ray, E. (2000). What is the significance of imitation in animals? Advances in the Study of Behavior, 29, 215–245. Hickok, G. (2009). Eight problems for the mirror neuron theory of action understanding in monkeys and humans. Journal of Cognitive Neuroscience, 21, 1229–1243. Iacoboni, M. (2008). The mirror neuron revolution: Explaining what makes humans social. Scientific American, July 1. Iacoboni, M. (2009). Imitation, empathy, and mirror neurons. Annual Review of Psychology, 60, 653–670. Iacoboni, M., Woods, R. P., Brass, M., Bekkering, H., Mazziotta, J. C., & Rizzolatti, G. (1999). Cortical mechanisms of human imitation. Science, 286, 2526–2528. Jeannerod, M. (1994). The representing brain. Neural correlates of motor intention and imagery. Behavioral and Brain Sciences, 17, 187–245. Keysers, C., & Gazzola, V. (2010). Social neuroscience: mirror neurons recorded in humans. Current Biology, 20, R353–R354. Keysers, C., Kohler, E., Umilta, M. A., Nanetti, L., Fogassi, L., & Gallese, V. (2003). Audiovisual mirror neurons and action recognition. Experimental Brain Research, 153, 628–636. Kilner, J. M., Neal, A., Weiskopf, N., Friston, K. J., & Frith, C. D. (2009). Evidence of mirror neurons in human inferior frontal gyrus. Journal of Neuroscience, 29, 10153–10159. Kilner, J. M., Paulignan, Y., & Blakemore, S. J. (2003). An interference effect of observed biological movement on action. Current Biology, 13, 522–525. Kohler, E., Keysers, C., Umilta, M. A., Fogassi, L., Gallese, V., & Rizzolatti, G. (2002). Hearing sounds, understanding actions: action representation in mirror neurons. Science, 297, 846–848. Kraskov, A., Dancause, N., Quallo, M. M., Shepherd, S., & Lemon, R. N. (2009). Corticospinal neurons in macaque ventral premotor cortex with mirror properties: a potential mecha- nism for action suppression? Neuron, 64, 922–930. Landmann, C., Landi, S. M., Grafton, S. T., & Della‐Maggiore, V. (2011). fMRI supports the sensorimotor theory of motor resonance. PLoS One, 6, 1–8. Lepage, J. F., & Theoret, H. (2007). The mirror neuron system: grasping others’ actions from birth? Developmental Science, 10, 513–523. Lingnau, A., Gesierich, B., & Caramazza, A. (2009). Asymmetric fMRI adaptation reveals no evidence for mirror neurons in humans. Proceedings of the National Academy of Sciences of the United States of America, 106, 9925–9930. Lotto, A. J., Hickok, G. S., & Holt, L. L. (2009). Reflections on mirror neurons and speech perception. Trends in Cognitive Sciences, 13, 110–114. Mameli, M., & Bateson, P. (2006). Innateness and the sciences. Biology and Philosophy, 21, 155–188. Marshall, P. J., & Meltzoff, A. N. (2011). Neural mirroring systems: Exploring the EEG mu rhythm in human infancy. Developmental Cognitive Neuroscience, 1, 110–123. Marshall, P. J., Young, T., & Meltzoff, A. N. (2011). Neural correlates of action observa- tion and execution in 14‐month‐old infants: An event‐related EEG desynchroniza- tion study. Developmental Science, 14, 474–480. Molenberghs, P., Cunnington, R., & Mattingley, J. B. (2012). Brain regions with mirror prop- erties: a meta‐analysis of 125 human fMRI studies. Neuroscience and Biobehavioral Reviews, 36, 341–349.

536 Caroline Catmur, Clare Press, and Cecilia Heyes Mukamel, R., Ekstrom, A. D., Kaplan, J., Iacoboni, M., & Fried, I. (2010). Single‐neuron responses in humans during execution and observation of actions. Current Biology, 20, 750–756. Murata, A., Fadiga, L., Fogassi, L., Gallese, V., Raos, V., & Rizzolatti, G. (1997). Object rep- resentation in the ventral premotor cortex (area F5) of the monkey. Journal of Neurophysiology, 78, 2226–2230. Murata, A., Gallese, V., Luppino, G., Kaseda, M., & Sakata, H. (2000). Selectivity for the shape, size, and orientation of objects for grasping in neurons of monkey parietal area AIP. Journal of Neurophysiology, 83, 2580–2601. Newman‐Norlund, R. D., van Schie, H. T., van Zuijlen, A. M. J., & Bekkering, H. (2007). The mirror neuron system is more active during complementary compared with imitative action. Nature Neuroscience, 10, 817–818. Nyström, P. (2008). The infant mirror neuron system studied with high density EEG. Social Neuroscience, 3, 334–347. Oosterhof, N. N., Tipper, S. P., & Downing, P. E. (2012). Viewpoint (in)dependence of action representations: an MVPA study. Journal of Cognitive Neuroscience, 24, 975–989. Oram, M. W., & Perrett, D. I. (1994). Responses of anterior superior temporal polysensory (STPa) neurons to biological motion stimuli. Journal of Cognitive Neuroscience, 6, 99–116. Oram, M. W., & Perrett, D. I. (1996). Integration of form and motion in the anterior superior temporal polysensory area (STPa) of the macaque monkey. Journal of Neurophysiology, 76, 109–129. Paukner, A., Ferrari, P. F., & Suomi, S. J. (2011). Delayed imitation of lipsmacking gestures by infant rhesus macaques (Macaca mulatta). PLoS ONE, 6, e28848. Paulus, M., Hunnius, S., van Elk, M., & Bekkering, H. (2012). How learning to shake a rattle affects 8‐month‐old infants’ perception of the rattle’s sound: Electrophysiological ­evidence for action–effect binding in infancy. Developmental Cognitive Neuroscience, 2, 90–96. Peck, C. A., & Bouton, M. E. (1990). Context and performance in aversive‐to‐appetitive and appetitive‐to‐aversive transfer. Learning and Motivation, 21, 1–31. Perrett, D. I., Harries, M. H., Bevan, R., Thomas, S., Benson, P. J., Mistlin, A. J., … Ortega, J. E. (1989). Frameworks of analysis for the neural representation of animate objects and actions. Journal of Experimental Biology, 146, 87–113. Petroni, A., Baguear, F., & Della‐Maggiore, V. (2010). Motor resonance may originate from sensorimotor experience. Journal of Neurophysiology, 104, 1867–1871. Pinker, S. (1997). How the mind works. London, UK: The Penguin Press. Pobric, G., & Hamilton, A. F. (2006). Action understanding requires the left inferior frontal cortex. Current Biology, 16, 524–529. Press, C., Catmur, C., Cook, R., Widmann, H., Heyes, C., & Bird, G. (2012). fMRI evidence of “mirror” responses to geometric shapes. PLoS One, 7, e51934. Press, C., Gillmeister, H., & Heyes, C. (2007). Sensorimotor experience enhances automatic imitation of robotic action. Proceedings of the Royal Society B: Biological Sciences, 274, 2509–2514. Ramachandran, V. S. (2009). The neurons that shaped civilization. TED, from http://www. ted.com/talks/vs_ramachandran_the_neurons_that_shaped_civilization.html Ray, E., & Heyes, C. (2011). Imitation in infancy: the wealth of the stimulus. Developmental Science, 14, 92–105. Rescorla, R. A. (1968). Probability of shock in the presence and absence of CS in fear condi- tioning. Journal of Comparative and Physiological Psychology, 66, 1–5. Rizzolatti, G., & Arbib, M. A. (1998). Language within our grasp. Trends in Neurosciences, 21, 188–194. Rizzolatti, G., Camarda, R., Fogassi, L., Gentilucci, M., Luppino, G., & Matelli, M. (1988). Functional organization of inferior area 6 in the macaque monkey. II. Area F5 and the control of distal movements. Experimental Brain Research, 71, 491–507.

Mirror Neurons from Associative Learning 537 Rizzolatti, G., & Craighero, L. (2004). The mirror‐neuron system. Annual Review of Neuroscience, 27, 169–192. Rizzolatti, G., & Fadiga, L. (1998). Grasping objects and grasping action meanings: the dual role of monkey rostroventral premotor cortex (area F5). Novartis Foundation Symposium, 218, 81–95; discussion 95–103. Rizzolatti, G., Fadiga, L., Gallese, V., & Fogassi, L. (1996). Premotor cortex and the recognition of motor actions. Cognitive Brain Research, 3, 131–141. Rizzolatti, G., & Sinigaglia, C. (2010). The functional role of the parieto‐frontal mirror circuit: interpretations and misinterpretations. Nature Reviews Neuroscience, 11, 264–274. Rochat, M. J., Caruana, F., Jezzini, A., Escola, L., Intskirveli, I., Grammont, F., … Umiltà, M. A. (2010). Responses of mirror neurons in area F5 to hand and tool grasping observation. Experimental Brain Research, 204, 605–616. Schultz, W., & Dickinson, A. (2000). Neuronal coding of prediction errors. Annual Review of Neuroscience, 23, 473–500. Scott, S. K., McGettigan, C., & Eisner, F. (2009). A little more conversation, a little less action‐ candidate roles for the motor cortex in speech perception. Nature Reviews Neuroscience, 10, 295–302. Silvanto, J., Muggleton, N. G., Cowey, A., & Walsh, V. (2007). Neural activation state deter- mines behavioral susceptibility to modified theta burst transcranial magnetic stimulation. European Journal of Neuroscience, 26, 523–528. Southgate, V., Johnson, M. H., El Karoui, I., & Csibra, G. (2010). Motor system activation reveals infants’ online prediction of others’ goals. Psychological Science, 21, 355–359. Stürmer, B., Aschersleben, G., & Prinz, W. (2000). Correspondence effects with manual ges- tures and postures: a study of imitation. Journal of Experimental Psychology: Human Perception and Performance, 26, 1746–1759. Tkach, D., Reimer, J., & Hatsopoulos, N. G. (2007). Congruent activity during action and action observation in motor cortex. Journal of Neuroscience, 27, 13241–13250. Umilta, M. A., Kohler, E., Gallese, V., Fogassi, L., Fadiga, L., Keysers, C., & Rizzolatti G. (2001). I know what you are doing: A neurophysiological study. Neuron, 31, 155–165. Verschoor, S. A., Weidema, M., Biro, S., & Hommel, B. (2010). Where do action goals come from? Evidence for spontaneous action–effect binding in infants. Frontiers in Psychology, 1, 1:6. Vogt, S., Buccino, G., Wohlschlager, A. M., Canessa, N., Shah, N. J., Zilles, K., … Fink G. R. (2007). Prefrontal involvement in imitation learning of hand actions: effects of practice and expertise. Neuroimage, 37, 1371–1383. Wiggett, A. J., Hudson, M., Tipper, S. P., & Downing, P. E. (2011). Learning associations between action and perception: effects of incompatible training on body part and spatial priming. Brain and Cognition, 76, 87–96. Williams, G. C. (1966). Adaptation and natural selection. Princeton, NJ: Princeton University Press.

21 Associative Approaches to Lexical Development Kim Plunkett Associative or What? An associative approach to lexical acquisition assumes that the principles of associative learning are adequate to account for the representations and processes underlying the mature use of words. Theoretical constructs available to contemporary associative learning theory are powerful and varied. They include processes such as classical and instrumental conditioning, discrimination learning, blocking, extinction, and so forth, and do not shy away from using constructs such as attention, representation, categorization and memory, which will be familiar to modern‐day cognitivists (see Dickinson, 1980; Pearce, 2008, for overviews). Nevertheless, it is still commonly assumed by many developmental psycholinguists that associative approaches to lan- guage acquisition became obsolete with Chomsky’s (1959) critique of Skinner’s (1957) Verbal Behavior. For example, a common interpretation of associative learning among many developmentalists is that associations can only be formed between stimuli that are present in the organism’s immediate environment. Yet associative learning theory can readily account for the formation of associations between a stim- ulus and a memory representation of another stimulus not present in the current environment (see Holland, 1990; Chapter 4). Cognitivists might object that admission of theoretical constructs, such as attention and representation, transform associative theory into a cognitive one. Similar argu- ments have been put forward in criticisms of connectionist modeling of cognitive processes, where some of the elementary processing units might themselves have a symbolic character (Fodor & Pylyshyn, 1988; Lachter & Bever, 1988). The validity of this critique depends much upon the manner in which the constituent theoretical constructs are used. If, for example, standard cognitive machinery is needed to get the associative explanation to work, then clearly the associative account fails. In this chapter, I consider whether cognitive machinery is needed to explain early lexical development by entertaining the possibility that associative mechanisms are sufficient to account for some of the important findings in the field. My strategy is to apply ­constructs taken from associative learning theory, including those implemented in contemporary connectionist learning models, to erstwhile cognitive explanations of The Wiley Handbook on the Cognitive Neuroscience of Learning, First Edition. Edited by Robin A. Murphy and Robert C. Honey. © 2016 John Wiley & Sons, Ltd. Published 2016 by John Wiley & Sons, Ltd.

Associative Approaches to Lexical Development 539 lexical development. I should acknowledge at the outset that this strategy may fall foul of the criticism of being “merely implementational” (Pinker & Prince, 1988). However, it is offered in the spirit of the connectionist insight that some cognitive explanations may merely be “descriptive conveniences” (Rumelhart & McClelland, 1986), and that the associative approach provides a closer view of the mechanisms at work. The Problem A central question in early lexical development is how infants learn to understand the meaning of words. In a typical labeling situation, the caregiver points at an object (Fido the dog) and says “Look, this is a dog!” The infant has then to rule out a huge number of possible meanings in order to decipher the intended meaning: Do the words refer to the size, to the shape, to the color, to the individual Fido or to the intended meaning: “Dog” is a label that can be used for this dog and for all dogs. An influential solution to this conundrum was introduced over 50 years ago: Language learners make use of linguistic constraints in order to narrow the hypothesis space in order to assign meaning to words (Quine, 1960). Three such constraints have proved particularly influential in informing cognitive approaches to lexical development (Markman, 1990, p. 57): Whole object constraint: toddlers interpret novel terms as labels for objects – not parts, substances, or other properties of objects; Taxonomic constraint: toddlers consider labels as referring to objects of like kind, rather than to objects that are thematically related; Mutual exclusivity: toddlers expect each object to have only one label. These constraints “guide children’s initial hypotheses and eliminate numerous hypotheses from consideration and thereby help them solve the inductive problem posed by word‐learning” (Markman, 1990, p. 75). A common interpretation of these claims is that infants are already in possession of knowledge underlying the use of these word‐learning constraints at the outset of lexical development. This raises the question as to whether such knowledge is learnable and/or whether lexical learning contributes to the establishment of such constraints. Phrased more bluntly, are Markman’s word‐learning constraints an emergent property of linguistic and cognitive development, or are they innate? I will consider each constraint in turn and attempt to demonstrate that each can emerge from associative learning processes. Whole Object Constraint The whole object constraint (WOC) strikes at the heart of the Quinean conundrum, solving it at a stroke by stipulating that toddlers interpret novel labels as names for whole objects. The constraint takes it as given that toddlers can readily identify novel labels in the speech stream and segment objects in the visual scene. These are not trivial capacities and have been the focus of intensive programs of research for decades.

540 Kim Plunkett However, I will assume along with Markman (1990) that the infant’s perceptual system delivers words and objects as feature packages for further processing. I will also assume that there are situational characteristics, such as joint attention, that facilitate the operation of the constraint. Of course, all of these assumptions require additional explanation to be rendered amenable to an associative account. For the moment, the problem is to account for the WOC whereby a package of linguistic features consti- tuting a novel label is preferentially associated with a package of visual features defining the whole object rather than some subset (or superset) of these features. For the pre- sent purposes, I will assume that a whole object is represented as a bundle of features defining shape, coloring, texture, and spatio‐temporal location. Likewise, I will assume that novel words are represented as bundles of phonetic or phonological features.1 A simple associative implementation of the WOC might exploit an auto‐associator. An auto‐associator (Figure 21.1) consists of a set of units (represented by circles) with incoming and outgoing connections (represented by arrows). Each unit possesses a set of connections to every other unit in the network (represented by small black circles). Activity entering the network along the input lines initiates a buildup of activity in the units that is passed forward along the output lines and to the other units in the net- work. A reverberating cycle of activation is thereby launched in the network. If the strength of the connections in the network is suitably chosen, the auto‐associator will eventually stabilize to a state of equilibrium in the activity of the units. Usually, the pattern of activation achieved by the auto‐associator is just the same pattern of activity that was used to initiate the cycle, hence the term auto‐association. It may seem strange to build a network that just replicates the pattern of activity to which it is exposed. However, there are a several desirable properties associated with networks of this type: 1 The network can act as a store for many input patterns simultaneously, thereby functioning as a memory system. 2 The network can be trained to reproduce new patterns by adapting the connec- tions using a simple Hebbian learning algorithm. 3 If the auto‐associator is presented with a noisy version of one of the patterns in its memory, the final stable state of the network will look more like the original pattern than the noisy input. The auto‐associator performs pattern completion (sometimes described as clean‐up). Figure 21.1  An auto‐associator.

Associative Approaches to Lexical Development 541 There is a substantial body of evidence indicating that neural networks in the h­ippocampus store episodic memories and that their computational/architectural structure resembles that of an auto‐associator (e.g., Treves & Rolls, 1994). Of particular relevance to a discussion of the WOC is the auto‐associator’s capacity to compute correlations (positive or negative) in the activity of different components of the input signal, and adjust the connections appropriately (excitatory or inhibitory). Assume for the moment that input to the auto‐associator is a compound audio‐visual stimulus such as a visual scene depicting a dog and somebody pointing and saying dog. Since each component unit is connected to every other unit, the pattern of correla- tions encoded in the device can be complex (many‐to‐one and one‐to‐many). The WOC can then be construed as encoding a pattern of correlations between a package of linguistic features and a package of visual features in a compound audio‐visual stim- ulus. Spurious correlations (such as whether a dog is moving or standing still, or indeed the breed of the dog, when the word dog is uttered) between the activities of the units of auto‐associator will eventually be weeded out by the Hebbian learning algorithm with subsequent occurrences of similar events, leaving the essential ingre- dients of the word–object association in place: If the trained auto‐associator is then presented with just auditory input (even noisy auditory input), it will reactivate just that pattern of visual activity with which it correlates, namely the bundle of visual f­eatures that were consistently presented with the auditory stimulus. For example, hearing dog will activate a visual representation of dogs, and vice versa. This implementation of the WOC comes at a price: Multiple exposures to a particular object–word pairing are required for identification of the appropriate set of visual features for a given package of linguistic features. The learner cannot know at the outset which visual features (or, for that matter, which linguistic features) are r­elevant – the original Quinean conundrum. Consequently, initial solutions to the conundrum will have a global or holistic character whereby a broader range of visual features will be associated with the auditory label than is necessary. Furthermore, f­requent exposures to identical (or highly similar) auditory‐visual pairings will bias the learner to highlight certain features over others, even though they may not be central to the correct association. In fact, there is empirical evidence that early word meanings capture a broader context compared with adult meanings (e.g., Barrett, Harris, & Chasin, 1991; Kuczaj & Barrett, 1986; Meints, Plunkett, Harris, & Dimmock, 2004) and that these meanings are gradually de‐contextualized to their core conceptual components. Similarly, highly frequent objectword pairings can be mastered surpris- ingly early by infants, as early as 6–9 months according to Bergelson and Swingley (2012) and Tincoff and Jusczyk (1999), suggesting that the WOC emerges incre- mentally rather than all at once. It might be objected that this solution loses the force of the WOC. After all, on this associative account, the learner may include background context, in addition to the whole object, as part of the meaning of the word until subsequent experience teaches otherwise. A one‐shot application of the WOC is not guaranteed in infants or adults. Several exposures, at least, are required to learn the meaning of words from an associative perspective. As much has been shown for toddlers: Horst and Samuelson (2008) have demonstrated the time‐dependent and incremental nature of word learning in 24‐month‐olds. Of course, previous learning may fine‐tune the learner’s attention to specific features, such as the shape of an object, when a labeling

542 Kim Plunkett event occurs (Landau, Smith, & Jones, 1988). When processing novel word–object associations, the connections in the auto‐associator associated with these specific f­eatures may already be strong enough to highlight their role in the association, or equivalently, the learning algorithm may adapt to strengthen such connections more quickly than others (Kruschke, 1992). Taxonomic Constraint The taxonomic constraint (TC) assumes that “labels refer to objects of like kind rather than objects that are thematically related” (Markman, 1992, p. 57). An equivalent formulation is that labels refer to objects that belong to the same taxonomic category, where a category can be defined in terms of visual features (visible or hidden) or functional relations (dynamic or abstract). Importantly, labels do not refer to groups of objects that merely co‐occur, either by virtue of their presence in the same event (e.g., dog–bone) or because they are mentioned in the same utterance. This does not exclude the possibility that taxonomically related objects occur in the same event or are mentioned in the same utterance, but that such co‐occurrence is insufficient. Markman and Hutchinson (1984) initially introduced a “weak” form of the TC as infants’ relative preference for taxonomic over thematic extension of labels. They pre- sented one group of young children (2–3‐year‐olds) with an object (say, a toy dog) that was labeled with a novel word (e.g., “dax”). The children were subsequently asked to find “another dax” from a pair of stimuli consisting of a taxonomic alternative (e.g., a cat) and a thematic alternative (e.g., a bone). For other children, the object was not labeled. Instead, they were asked to find “another one” from the same pair of stimuli (the taxonomic and the thematic alternatives). Children were more likely to pick the taxonomic alternative (the cat) when asked to find “another dax” than when asked for “another one,” suggesting that children take novel words to refer to taxo- nomic categories and not to groups of objects defined by thematic relations. In its strong form, the TC assumes that: “when infants embark upon the process of lexical acquisition, they are initially biased to interpret a word applied to an object as referring to that object and to other members of its kind” (Waxman & Markow, 1995, p. 257). In other words, from a single labeling event, the infant infers that every object that belongs to the same category is called by the same name. In this form, the TC equips the infant with a powerful communication tool, since she can now refer to objects she has never seen before, provided they belong to known categories. At first glance, the TC might seem difficult to implement in an associative mecha- nism, since it involves one‐to‐many associations: The label dog is used to refer to many different types of dogs. If the target referent is sufficiently different from the initial referent, then stimulus generalization2 (Moore, 1972) will fail. For example, if dog is used initially to refer to a German Shepherd, then dog may fail to be used appropri- ately to identify a Chihuahua. A simple way to fix this problem from an associative perspective is to ensure that all objects belonging to the category Dog are labeled dog. Of course, this is not a realistic solution, as the child will often encounter dogs she has never seen before and still name them correctly. However, if she is exposed to Dog

Associative Approaches to Lexical Development 543 labeling events across a representative sample of the species, then stimulus generalization can fill in the gaps allowing appropriate usage in the presence of novel exemplars, provided they fit in the space of possible dogs as defined by the child’s experience. Solution 1 An example of an associative vocabulary learner of this kind was proposed by Plunkett, Sinha, Møller, and Strandsby (1992). The learner is an auto‐encoder network that has computational properties similar to that of the auto‐associator shown in Figure 21.1, in that its task is to reproduce its input at the output. Unlike the auto‐associator, which is fully recurrent, the auto‐encoder is a structured network with intermediate layers of hidden units between the input and output. The hidden units function as information bottlenecks, forcing the network to encode the input patterns into a more compact, abstract representation that can then be decoded to reproduce the original input at the output. The auto‐encoder network architecture consists of two partially merging subnet- works: a visual subnetwork and a linguistic subnetwork (see Figure 21.2). The visual pathway is presented with random dot images (Posner & Keele, 1968) that are pre- processed by input receptors with Gaussian receptive fields. The second input pathway processes linguistic input corresponding to the names of the random dot patterns. Thirty two categories, each containing eight objects derived from a different random dot prototype, and 32 labels are presented to the auto‐encoder network in a three‐ phase training cycle involving the object alone, the label alone, and the objectlabel pair, aimed at capturing the attention switching process to the label, the object, and finally the object–label pair. The performance of the network is evaluated by analyzing the network’s ability to produce the correct label when only an image is presented (analogous to production) and to produce the correct image when only a label is presented (analogous to comprehension). The model successfully captures the well‐ known vocabulary spurt, patterns of over‐ and underextension errors, prototype effects, and the comprehension production asymmetry observed during infant vocab- ulary development. In particular, this associative learner behaves taxonomically: After Retinal units Label units Hidden units Hidden units Hidden units Retinal units Label units Figure 21.2  Auto‐encoder used by Plunkett et al. (1992). Reproduced with permission from Taylor & Francis.

544 Kim Plunkett training, the network is able to assign the correct label to images that it has never seen before, just so long as those images are taken from the space of random dot patterns that span the category associated with the label. Prototypes of each category are responded to more robustly than peripheral members of the category, though this discrepancy decreases with training. The prototype effect is evident, even though the network has never been trained on the original pattern. Again, this implementation of TC comes at a price: During the early stages of training, the network only responds appropriately to the range of examples presented in the training set. If the sampling of this space is not representative of the category, the network will underextend the assignment of the label. In fact, there is empirical evidence that infants respond in this way, too. Meints, Plunkett, and Harris (1999) showed that 12‐month‐old infants only correctly identify typical members of a cate- gory (GERMAN SHEPHERDS for dog) in a preferential looking task. Atypical mem- bers of categories (such as CHIHUAHUAS) are not identified as appropriate referents until around 18 months. Similar findings have been reported for action words (Meints, Plunkett, & Harris, 2008) and location terms (Meints, Plunkett, & Harris, 2002). On this account, the TC is an emergent property of the associative learner’s exposure to a representative sample of object–label pairings. If these pairings sample the full space of the target category, then the learner will be able to interpolate to novel object tokens and hence respond taxonomically. However, this explanation of taxonomic responding falls short of the stronger version of the TC described earlier: From a single labeling event, the infant infers that every object that belongs to the same category is called by the same name. The associative learner described by Plunkett et al. (1992) requires exposure to multiple object–label tokens. In order for this kind of one‐shot learning and generalization to occur, the learner must have prior knowledge of the category boundaries. Solution 2 Mayor and Plunkett (2010) addressed this problem by separating the visual and auditory pathways in the network and by using self‐organizing maps (SOMs; Kohonen, 1984) to extract category representations from the auditory and object tokens pre- sented in the training set (Figure 21.3). SOMs are associative learners that offer an efficient computational method for forming categories in a complex input space. They extract statistical regularities from the input and form categories of similar objects without explicit supervision. They achieve this result through dimensionality reduction and self‐organization around topological maps. SOMs are thereby able to capture the natural clustering of objects that share properties with each other. At the end of the process of self‐organization, similar objects activate neighboring neurones in the map. The model consists of two separate SOMs, visual and auditory, that receive visual input and input from acoustic tokens, respectively. Again, it is assumed that infants have already developed the ability to segment objects out of complex visual scenes (e.g., Kellman, Spelke, & Short, 1986) and labels from continuous speech (e.g., Jusczyk & Aslin, 1995) by the time they start forming categories of objects and word types. Through the separate presentation of multiple object and label tokens, both SOMs form categories based on the similarity of the complex set of input patterns.

Associative Approaches to Lexical Development 545 Dog ! Figure 21.3  Sketch of the network in a joint attentional event. When a dog is presented to the visual map, a coherent activity pattern emerges. Similarly, when an acoustic token of the label dog is presented to the auditory map, a selection of neurones will be activated. Synapses connecting the two maps are modulated according to the Hebb rule. The reinforcement of synapses originating from neurones neighboring the maximally active neurone is a key element in generalizing single associations and therefore taxonomic responding. The organization of the stimuli on the maps mimics the infant’s perceptual refinement of her sensory cortices, based on the unsupervised experience of seeing different objects and hearing different speech fragments. By the time infants are able to engage in joint attentional activities with their caregivers (usually toward the end of the first year), their perceptual systems are already well organized. Both the visual and auditory maps have undergone self‐organization, so that when a joint attentional event occurs, effective associations between the preestablished categories can be formed. Joint attentional events, such as the caregiver looking at a dog and saying dog at the same time as the infant is paying attention to the dog, are mimicked through the simultaneous presentation of objects and their labels, and constitute the supervised component of word learning that is essential for learning the arbitrary mappings bet- ween labels and objects. Synapses connecting active neurones on both maps are reinforced through Hebbian learning (Hebb, 1949) as shown in Figure 21.3 (see also Chapter 3). Owing to the topographical organization of the maps that takes place in early development, many neighboring neurones on each map will be activated by the presentation of an object and its corresponding sound pattern. Crossmodal Hebbian learning will then take place for neighboring neurones on each map. Therefore, the association between the paired object and its corresponding sound pattern will be generalized, automatically building associations between all objects in its category to all sound patterns of the appropriate type. A single labeling event is thereby able to induce a taxonomic response with the label extended to all objects of like type: The novel word is learned. Within this framework, the TC emerges from architectural constraints built into the network: The topographic organization of the SOM ensures that similar objects