What causes the behavior weakens and what causes the behavior strengthen?
|
Prev
Med. Author manuscript; available in PMC 2015 Nov 1. Published in final edited form as: PMCID: PMC4287360 NIHMSID: NIHMS605603 Unhealthy behavior is responsible for much human disease, and a common goal of contemporary preventive medicine is therefore to encourage behavior change. However, while behavior
change often seems easy in the short run, it can be difficult to sustain. This article provides a selective review of research from the basic learning and behavior laboratory that provides some insight into why. The research suggests that methods used to create behavior change (including extinction, counterconditioning, punishment, reinforcement of alternative behavior, and abstinence reinforcement) tend to inhibit, rather than erase, the original behavior. Importantly, the inhibition, and thus
behavior change more generally, is often specific to the “context” in which it is learned. In support of this view, the article discusses a number of lapse and relapse phenomena that occur after behavior has been changed (renewal, spontaneous recovery, reinstatement, rapid reacquisition, and resurgence). The findings suggest that changing a behavior can be an inherently unstable and unsteady process; frequent lapses and relapse should be expected to occur. In the long run, behavior-change
therapies might benefit from paying attention to the context in which behavior change occurs. Keywords: behavior change, contingency management, behavioral inhibition, context, relapse Behavior causes a surprising amount of human disease. For example, an estimated 40% of premature deaths in the U.S. can be attributed to unhealthy behaviors, such as smoking and inactivity (e.g.,
Schroeder, 2007). Eliminating such behaviors, and replacing them with healthier ones, is therefore one of the most important strategies for improving U.S. population health. But a persistent challenge to the field is that sustaining behavior change is not easy. Classic data suggest that roughly 70% of individuals who successfully quit illicit drug use, cigarette smoking, or
problem drinking return to their old behaviors within a year (Hunt, Barnett, & Branch, 1971). More recent data suggest similar outcomes (e.g., Hughes, Keely, & Naud, 2004;
Kirshenbaum, Olsen, & Bickel, 2009). Even patients who enter an incentive-based “contingency-management” treatment that explicitly reinforces healthy behavior with vouchers or prizes (e.g., Higgins, Silverman, & Heil, 2008;
Higgins, Silverman, Sigmon, & Naito, 2012; Fisher, Green, Calvert, & Glasgow, 2011) often return to their unwanted behaviors over time. That is, once contingency management stops, and the reinforcers are discontinued, many individuals return to the original behavior (e.g.,
John, Loewenstein, Troxel, Norton, Fassbender, & Volpp, 2011; Silverman, DeFulio, & Sigurdsson, 2012). Despite the fact that contingency management is one of the most successful behavioral intervention strategies, for the case of drug dependence, “the
development of more enduring solutions to sustain abstinence over years and lifetimes is perhaps the greatest challenge facing the substance abuse treatment research community today” (Silverman et al., 2012, p. S47). The purpose of the present article is to present some research from the basic behavioral laboratory that might shed light on why it is so difficult to
sustain behavior change. The issue has been discussed in other papers (e.g., Bouton, 2000, 2002); the current article focuses on behavior change in general with an emphasis on recent work addressing instrumental (operant) learning. Roughly three decades of basic
research on behavior change suggests two main conclusions. First, changing or replacing an old behavior with a new behavior does not erase the original one. Second, behavior change can be remarkably specific to the “context” in which it occurs. Both of these features of behavior change appear to be general across different treatment strategies for creating change. They might provide some insight into why behavior change can be so difficult to maintain. Behavior change can be studied in the laboratory with variations of two well-known behavioral methods. In the first, organisms like rats or pigeons learn to perform specific behaviors (such as pressing a lever or pecking at a disk) to obtain food, water, or drug reinforcers. The study of such operant conditioning provides a method that allows behavioral scientists to study how “free” or “voluntary” behavior is influenced by its consequences. In the
second method, Pavlovian or classical conditioning, the organism learns to associate a signal (such as presentation of a tone or light) with upcoming reinforcers or punishers (e.g., food, water, drugs, or a mild shock). This kind of learning in turn allows the organism to adapt to significant events in the environment by making anticipatory responses in the presence of the signal. Both Pavlovian and operant learning are widely represented in human experience and provide the building
blocks of many complex behaviors and actions (e.g., Baldwin & Baldwin, 2001). In either type of learning, behavior change can be studied by altering the relationship between the action or the signal and the reinforcing or punishing outcome. In extinction, perhaps the most basic form of behavior change, the strength or rate of the behavior declines when the reinforcing
outcome is eliminated. The behavior eventually goes away, and is said to be “extinguished.” Extinction is a reliable way to reduce a learned behavior, and it is thought to be the mechanism behind various cognitive behavior therapies that eliminate unwanted behaviors, thoughts, or emotions by repeatedly exposing the patient to the cues or situations that trigger them (e.g., Craske,
Kircanski, Zelikowsky, Mystkowski, Chowdhury, & Baker, 2008). It is tempting to conclude that extinction erases or destroys the original learning. But the evidence suggests that extinction is best thought of as producing a kind of behavioral inhibition. That is, the original behavior is still in the brain or memory system, but is inhibited and ready to return to performance under certain conditions. Learning theorists have long emphasized a distinction between learning and
performance. Just because a behavior is not manifest in performance does not mean that its underlying basis is gone. It is potentially available to produce lapse or relapse. Since the 1970s, extinction has been studied extensively with Pavlovian methods. As noted above, when the significant event is no longer presented, anticipatory responses to the signal go away. However, the extinguished response can readily return with any of several experimental manipulations (see
Bouton, 2004; Bouton & Woods, 2008, for more extensive discussions). These are summarized in Table 1. In what is probably the
most fundamental example, the renewal effect, extinguished responding to the signal (the conditioned stimulus or “CS”) returns if the CS is simply tested in a different context (e.g., Bouton & Bolles, 1979; Bouton & King, 1983;
Bouton & Peck, 1989). (In the animal laboratory, “contexts” are usually provided by the Skinner boxes in which learning and testing occur; they usually differ in their visual, olfactory, tactile, and spatial respects.) In spontaneous recovery, the extinguished response can return if the CS is tested again after some time has elapsed after extinction. The phenomenon can be viewed as
another example of the renewal effect in which extinction is shown to be specific to its temporal context (e.g., Bouton, 1988). In reinstatement (e.g., Rescorla & Heth, 1975), mere exposure to the significant event (the unconditioned stimulus or “US”) again after extinction can make responding return to the CS. Importantly, the reinstating effect of
presenting the US alone is also a context effect. For example, in Pavlovian learning, presentation of the US must occur in the context in which testing will take place in order for the response to return (Bouton, 1984; Bouton & Bolles, 1979;
Bouton & King, 1983; Bouton & Peck, 1989; see also Westbrook, Iordanova, Harris, McNally, & Richardson, 2002). The picture that emerges is that behavior
after extinction is quite sensitive to the current context. When the trigger cue is returned to the acquisition context, when the context is merely changed, or when the context is associated with the reinforcer again, the cue (CS) can readily trigger responding again. Context, lapse, and relapse effects that can interfere with lasting behavior change
A fourth phenomenon is rapid reacquisition. In this case, when CS-US pairings are resumed after extinction, the return of responding can be very rapid (Napier, Macrae, & Kehoe, 1992; Ricker & Bouton, 1996). Rapid reacquisition may be especially relevant to behavior change in the natural world, because the US or reinforcer is usually presented whenever a lapsing drug user or over-eater consumes the drug or junk food again. The evidence suggests that reacquisition is rapid because the reinforced trials were part of the “context” of original conditioning (Bouton, Woods, & Pineno, 2004; Ricker & Bouton, 1996). Thus, when the CS and US are paired again, the organism is returned to the original context, and responding recovers because it is a form of an ABA renewal effect. Once again, performance after extinction depends on context. And the meaning of “context” can be very broad and include not only the physical background, but recent events, mood states, drug states, deprivation states, and time (e.g., see Bouton, 1991, 2002). It is important to note that what we know about extinction also applies to other Pavlovian behavior-change procedures (Bouton, 1993). For example, in counterconditioning, the CS is paired with a new US in Phase 2 instead of simply being presented alone. Here we also find little evidence for erasure and a lot for the role of context. For example, when CS-shock pairings are followed by CS-food pairings, renewal of fear occurs after a context change (Peck & Bouton, 1990), spontaneous recovery occurs after the passage of time (Bouton & Peck, 1992), and reinstatement of fear to the tone occurs if shock is presented alone again (Brooks, Hale, Nelson, & Bouton, 1995). Renewal and spontaneous recovery of appetitive behavior can also occur when tone-shock follows tone-food (Peck & Bouton, 1990; Bouton & Peck, 1992). We have also seen renewal and spontaneous recovery after discrimination reversal learning in which tone-shock and light-no shock were followed by tone-no shock and light-shock (Bouton & Brooks, 1992). And when an inhibitory CS that signals “no reinforcer” is converted into an excitor that now signals that the reinforcer will occur, the original inhibitory meaning can return upon return to the original inhibitory conditioning context (Peck, 1995; see also Fiori, Barnet, & Miller, 1994). All of these findings suggest that extinction can be viewed as a representative form of retroactive inhibition in which new learning replaces the old (Bouton, 1993). Learning something new about a stimulus does not necessarily erase the earlier learning. It involves inhibition that is sensitive to context change. The variety of different lapse and relapse effects suggests that behavior change can be an intrinsically unsteady affair. Given the many possible context changes that can occur in the natural world after a behavior is inhibited, repeated lapses should always be expected. One rule of thumb is that after extinction the signal has had a history of two associations with the US (CS-US learned in conditioning and CS-no US learned in extinction). Its meaning is therefore ambiguous. And like the current meaning of an ambiguous word (or the verbal response it evokes), the current response evoked by the trigger cue depends on the current context. More detailed reviews of extinction in Pavlovian conditioning with an eye toward making it more enduring can be found in Bouton and Woods (2008) and Laborda, McConnell, and Miller (2011). Extinction and inhibition of voluntary behaviorMore recent research has asked whether similar principles apply to extinction in operant learning. As noted above, operant learning may be especially relevant for understanding factors that influence voluntary behaviors, such as over-eating, smoking, problem drinking, and illicit drug use. Here again, extinction can be created (and behavior “eliminated”) by allowing the organism to make the response without the reinforcer. And although the procedure makes the behavior go away, it does not necessarily erase it. For example, recent experiments have demonstrated the ABA, ABC, and AAB renewal effects after operant extinction (Bouton, Todd, Vurbic, & Winterbauer, 2011). ABA and AAB renewal have also been demonstrated in discriminated operant learning, where the response is only reinforced in the presence of a discriminative stimulus (SD), such as a tone or a light, which consequently sets the occasion for the response (ABA renewal: Nakajima, Tanaka, Urushihara, & Imada, 2000; Vurbic, Gold, & Bouton, 2011; Todd, Vurbic, & Bouton, 2014; AAB renewal: Todd et al., 2014). The discriminated operant situation may be especially relevant to problematic human behavior, because so much of the latter takes place in the presence of cues that regularly set the occasion for them (for example, overeating takes place in the presence of stimuli, such as a bag of chips or a bucket of fried chicken, that set the occasion for eating). Moreover, reinstatement, spontaneous recovery, and rapid reacquisition have also been demonstrated in operant learning. They have all occurred in drug self-administration experiments in which animals are reinforced for responding with drugs of abuse, such as heroin, cocaine, or alcohol (see Bouton, Winterbauer, & Vurbic, 2013 for a review). Thus, as in Pavlovian conditioning, operant extinction depends on inhibition that is specific to the context in which it is learned. Recent evidence from my laboratory further suggests that the organism learns something very specific when an operant behavior is extinguished: It learns not to make a specific response in a specific context (e.g., Todd, 2013; Todd et al., 2014). We know this, for example, because extinction of one behavior (e.g., pressing a lever) in the presence of an SD (e.g., a tone) can prevent renewal of the same response occasioned by other SDs (e.g., a light) in the same context (Todd et al., 2014, Experiment 3). In contrast, extinction of a different behavior (e.g., pulling a chain) is not as effective. Similarly, renewal is also not prevented by simple unreinforced exposure to the context without allowing the animal to make the response (Bouton et al., 2011, Experiment 4). This result is consistent with the finding that simple exposure to contextual cues might not weaken drug taking (Conklin & Tiffany, 2002). Our evidence suggests that extinction of operant behavior may require that the individual be given an opportunity to learn to inhibit the response directly. Interestingly, at least one effective extinction-based treatment of overeating in children required the children to make the response (eating) at least a little in the presence of food cues during cue exposure (Boutelle, Zucker, Peterson, Rydell, Cafri, & Harnack, 2011). We also know that our understanding of extinction applies to other retroactive interference arrangements in the operant paradigm. For example, Marchant, Khuc, Pickens, Bonci, and Shaham (2013) reported that alcohol seeking in rats could be suppressed by punishment. In their experiment, lever pressing was first trained by reinforcing it with alcohol. In a second phase, responses also produced a mild footshock, which reduced the behavior’s probability to near zero. However, when punishment occurred in Context B after the original training occurred in Context A, the response returned (was renewed) when it was tested in Context A. We recently examined renewal after punishment in more detail (Bouton & Schepers, 2014). We found that the context-specificity of punishment was not merely due to the subject associating the shock with Context B (which could have suppressed behavior on its own), and that renewal also occurred if testing was conducted in a neutral Context C (i.e., we observed ABC renewal). We also found that, paralleling research on operant extinction (Todd, 2013), training one response (R1, e.g., chain pulling) in Context A and a different response (R2, e.g., lever pressing) in Context B and then punishing each in the opposite context (i.e., R1 in Context B and R2 in Context A) allowed a renewal of responding when the responses were tested in their original training contexts. The test context also affected choice of R1 vs. R2 when both were made available at the same time. In punishment, as in extinction, the organism thus learns not to make a specific response in a specific context. Interestingly, re-exposure to a drug reinforcer can also reinstate operant drug self-administration behavior after it has been punished (Panlilo, Thorndike, & Schindler, 2003; see also Panlilio, Thorndike, & Schindler, 2005). The fact that what we know about extinction might also apply to punishment is important because the knowledge of aversive consequences of a behavior is another reason why humans might stop over-eating or drug-taking. Renewal of instrumental behavior occurs after still other forms of behavioral inhibition. For example, instead of studying extinction, Nakajima, Urushihara, and Masaka (2002) introduced a negative contingency between the operant response and getting the reinforcer in Phase 2 (see also Kearns & Weiss, 2007). After first training rats to lever press for food pellets, they made pressing the lever postpone a reinforcer that otherwise occurred freely. This suppressed responding, of course. But when the rats were returned to Context A after the negative contingency training had occurred in Context B, responding was renewed. These results, like the punishment results, suggest that other forms of behavioral inhibition besides extinction create a context-dependent form of inhibition—and not erasure or unlearning. Resurgence after behavior changeThese ideas come together further in a paradigm that may have an especially direct connection to the contingency management or incentivized treatments mentioned at the start of this article (e.g., Higgins et al., 2008; Fisher et al., 2011). Replacing an operant behavior with a new behavior while the first is being extinguished can still allow relapse to occur when the replacement behavior is itself extinguished (e.g., Leitenberg, Rawson, & Bath, 1970). The basic method is as follows. In an initial phase, pressing a lever (R1) is reinforced. Then, in a response-elimination phase, R1 is extinguished (it no longer produces the reinforcer) at the same time an alternative behavior (pressing a second lever, R2) is reinforced. At the end of the response elimination phase, R2 has replaced R1. But when R2 is no longer reinforced, the animal returns to and makes a number of responses on R1. R1 is said to have “resurged.” Once again, extinction (of R1) did not erase it. This phenomenon, the last one listed in Table 1, may be a more direct laboratory model of what occurs in either therapy or the natural world when a problem behavior is replaced with a healthier one. The field has recently begun to study resurgence in some detail (e.g., Bouton & Schepers, 2014; Cançado & Lattal, 2011; Lieving & Lattal, 2003; Shahan & Sweeney, 2011; Sweeney & Shahan, 2013; Winterbauer & Bouton, 2010, 2012; Winterbauer, Lucke, & Bouton, 2013). My colleagues and I have shown that under certain conditions resurgence may survive very extensive response elimination training. As illustrated in Figure 1, 36 sessions of response elimination did nothing to decrease resurgence beyond what we observed after 4 or 12 sessions (Winterbauer et al., 2013; but see Leitenberg, Rawson, & Mulick, 1975). In other experiments, we found that introducing an “abstinence” contingency into the response elimination phase did not abolish the effect either (Bouton & Schepers, 2014; see Figure 2). In this case, instead of merely extinguishing R1 while R2 was being reinforced, a reinforcer was delivered for R2 only if R1 had not been emitted for a minimum period of time (e.g., 45, 90, or 135 s). The addition of this abstinence contingency weakened the final resurgence effect, but it did not eliminate it. This result may not be surprising based on the evidence, reviewed above, that an original behavior can survive many different retroactive interference treatments.  Resurgence of extinguished operant behavior. Left to right, the top panels illustrate the rate at which subjects performed the target first behavior (L1 for Lever 1) during the acquisition phase, response elimination (extinction) phase, and the resurgence test phase (respectively). The bottom panels illustrate the rate of the replacement behavior (L2 for Lever 2) during the response elimination and resurgence testing phases. Groups that received 4 (Ext 4), 12 (Ext 12), or 36 (Ext 36) sessions of response elimination training showed reliable resurgence of L1 responding that did not differ significantly (upper right). From Winterbauer et al. (2013). Resurgence of operant behavior that was suppressed by an abstinence contingency. Left to right, the top panels show the rate at which subjects performed the target first behavior (R1 for Response 1) during acquisition, response elimination (Extinction), and resurgence testing. The bottom panels illustrate the acquisition of the replacement behavior (R2) during response elimination and testing. The groups differed in their treatment during response elimination: Group Extinction received R1 extinction while R2 was reinforced; Group 45 s Negative Contingency was required to abstain from R1 for at least 45 s if an R2 response was to be reinforced; subjects in Group Yoked received reinforcement at the same points in time as a subject in the Negative Contingency group but without being required to abstain from R1. Note that the latter groups showed the same reduced, but not eliminated, resurgence (upper right). From Bouton and Schepers (2014). My colleagues and I have argued that resurgence may simply be another example of the context-specificity of extinction. That is, the organism might learn to inhibit its performance of R1 in the “context” of a second behavior (R2) being reinforced. Then, when R2 reinforcers are themselves discontinued, the context changes—and R1 responding returns in the form of an ABC renewal effect (Bouton et al., 2011; Todd, 2013; Todd et al., 2012). Consistent with this idea, resurgence can be reduced, and possibly eliminated, if we allow the animal to learn extinction of R1 in a “context” that is more similar to the one that prevails during resurgence testing. For example, resurgence is reduced if the rate at which reinforcers are delivered for R2 is gradually decreased or “thinned” over the course of the response elimination phase (Winterbauer & Bouton, 2012; see also Sweeney & Shahan, 2013). Thinner schedules of reinforcement may allow the rat to learn not to make R1 in the context of fewer and fewer reinforcers—a context more like the one that prevails during resurgence testing. In related, yet-to-be-published work, Sydney Trask and I found that resurgence can be eliminated if very thin schedules of reinforcement are used to reinforce R2 from the beginning (see also Leitenberg, Rawson, & Mulick, 1975; Sweeney & Shahan, 2013). Finally, our experiments on the effects of adding an abstinence contingency (Bouton & Schepers, 2014) discovered that the reason why an abstinence contingency between R1 and the reinforcer weakened resurgence had nothing to do with the abstinence contingency per se. Instead, requiring abstinence from R1 made it difficult to earn reinforcers for R2 and reduced their frequency. It therefore gave the subject the opportunity to learn that R1 was extinguished during prolonged periods without a reinforcer. We know this because rats that were allowed to earn the reinforcer at the same rate as an abstinence group, but without an actual abstinence contingency, showed the same reduced, but not eliminated, level of resurgence (Figure 2). Across experiments, then, a reliable way to reduce resurgence is to give the subject an opportunity to learn not to perform R1 in the absence of frequent reinforcement. All in all, one of the messages of our work on resurgence is that response-elimination therapies might benefit from encouraging generalization from the behavior-change context to new contexts that might otherwise allow relapse in the form of the renewal effect. Another possibility, of course, would be to maintain abstinence reinforcement indefinitely, as suggested by Silverman et al. (2012), who demonstrated prolonged abstinence among cocaine users when they were given prolonged abstinence-contingent employment. The general context-dependence of operant behaviorEncouraging generalization to new contexts is also important for another reason. Our recent work on operant behavior has further discovered that operant behaviors are always context-dependent to some extent. That is, if a rat is reinforced for pressing a lever or pulling a chain in one context, merely testing the response in a second context consistently seems to weaken it. This effect of changing the context after operant learning appears to occur regardless of reinforcement schedule, the amount of training, whether the behavior is a discriminated or a non-discriminated operant, and whether the changed-to context is equally associated with reinforcers or the training of a different operant response (Bouton et al., 2011; Bouton, Todd, & León, 2014; Thrailkill & Bouton, submitted; Todd, 2013). The context thus appears to play a rather general role in enabling operant behavior. This idea has been a revelation to us because unlike operant responses, Pavlovian responses (fear or appetitive behaviors triggered by signals for shock or food) are often not weakened by changing the context (e.g., Bouton, Frohardt, Waddell, Sunsay, & Morris, 2008; Bouton & King, 1983; Bouton & Peck, 1989). Recent research thus suggests that there may be something especially important about the context in supporting voluntary, operant behavior. The revelation is worth mentioning in a discussion of behavior change because it suggests that any new and healthy behavior that a patient might learn might also be disrupted by a change of context. Thus, in addition to the effects summarized in Table 1 (which can make first-learned unhealthy behaviors return), merely changing the context on a healthy behavior may be another factor that will weaken it. This is another reason to encourage generalization to new contexts, perhaps by training healthy behavior in the contexts where the patient or client will most need it. Another possibility is to reinforce the new behavior in multiple contexts (cf. Gunther, Denniston, & Miller, 2008; Wasserman & Bhatt, 1992). From a theoretical perspective, practice in multiple contexts might help, because contexts are made up of many “stimulus elements,” and generalization to a new context might depend on the number of elements it has in common (e.g., Estes, 1955). By training a behavior in multiple contexts, one increases the breadth of elements that can occasion the behavior, thereby increasing the likelihood that a new context will contain an already-treated element. Behavior change can be made more permanent by being aware of the operant behavior’s inherent context dependence. The context-specificity of operant behavior makes ABC and AAB renewal after extinction especially interesting (e.g., Bouton et al., 2011; Todd, 2013; Todd et al., 2014). In both the ABC and AAB situations, inhibited responding returns in a context that is different from both the conditioning context and the extinction context. The fact that responding returns tells us that, despite the fact that operant behavior is at least somewhat context-specific, its inhibition is even more so. Therefore, the unhealthy first-learned behaviors that we may want to get rid of may still generalize better to new contexts than their inhibition will. This, coupled with the fact that behavior change does not cause erasure, provides the familiar imbalance that can make sustained behavior change so difficult. ConclusionsTo summarize this brief review of basic research on behavior change, successful learning of a new behavior does not permanently replace an earlier one. Behavior change does not equal unlearning; just because an old behavior has achieved a zero value does not mean that it has been erased. Second, the new behavior may be easy to disrupt because it may be especially context-dependent. Although our recent results suggest that all operant behavior may be inherently context-specific, the second-learned behavior appears to be more so (e.g., Nelson, 2002). Third, the fact that context change can disrupt the new behavior (and cause a lapse of the original one) underscores the importance of finding ways to make new behaviors generalize. For contingency management/incentives interventions, if prolonged or “lifetime” continuation of abstinence reinforcement (e.g., Silverman et al., 2012) is not possible, our resurgence results suggest that giving the client practice inhibiting her unhealthy behavior in the absence of explicit reinforcement (e.g., by thinning the rate at which abstinence is reinforced) might help behavior change to persist. Another possibility is to make sure the new behaviors are practiced in the contexts where lapse and relapse are most likely—including the contexts in which the first behavior was learned. A final tool for encouraging generalization to new contexts would be to practice the new behavior in multiple contexts. The point is that therapies must be designed to anticipate the possible deleterious consequences of context change. There may be other ways of encouraging generalization between the treatment and relapse contexts. One method involves presenting cues during lapse or relapse testing that can remind the organism of extinction. Brooks and Bouton (1993, 1994) found that a discrete cue that was presented intermittently during extinction could reduce spontaneous recovery or renewal if it was presented just before the test (see also, e.g., Brooks, 2000; Collins & Brandon, 2002; Mystkowski, Craske, Echiverri, & Labus, 2006). Reminder cues can also reduce renewal after the extinction of operant alcohol-seeking (Willcocks & McNally, 2014). In another method of encouraging generalization between the treatment and relapse contexts, Woods and Bouton (2007) modified an operant extinction procedure so that the response was occasionally reinforced after gradually lengthening intervals (see also Bouton, Woods, & Pineno, 2004). When the response was paired with the reinforcer again during a reacquisition phase, the rats given this treatment responded less than those given a traditional extinction treatment after each of the new response-reinforcer pairings. Woods and Bouton argued that the rat had learned that reinforced responses signaled more extinction instead of more imminent response-reinforcer pairings, and this allowed extinction to generalize more effectively to the reacquisition test. The method may be related to a smoking-reduction procedure introduced by Cinciripini et al. (1994, 1995) in which smokers slowly reduce their cigarette consumption by smoking only at predetermined intervals. From our point of view, occasional but distributed cigarettes (response-reinforcer pairings) may reduce the tendency of a single smoke to set the occasion for another one. Can other new methods be developed that will help promote behavior change? Several possibilities have been proposed. First, since response elimination involves new learning, drug compounds that can facilitate the learning process may be able to facilitate it. One example is d-cycloserine, a partial agonist of the NMDA receptor that is involved in long-term potentiation, a cellular model of learning. Although administering DCS during extinction can facilitate the rate at which Pavlovian fear extinction is learned (e.g., Walker, Ressler, Lu, & Davis, 2002), it does not necessarily reduce extinction’s context-dependency (Bouton, Vurbic, & Woods, 2008; Woods & Bouton, 2006). That is, renewal can still occur when the extinguished fear signal is tested in the original fear-conditioning context. There is also evidence that DCS can facilitate operant extinction learning (e.g., Leslie, Norwood, Kennedy, Begley, & Shaw, 2012), although our own attempts to produce such an effect have been unsuccessful (Vurbic et al., 2011). Vurbic et al. suggested that the drug may be mostly effective in Pavlovian extinction, which can be a part of operant procedures that involve explicit extinction of conditioned reinforcers (see Thanos et al., 2011). A second possibility for promoting permanent behavior change is reconsolidation (e.g., Nader, Schafe, & LeDoux, 2000). The idea here is that when a memory is retrieved, it becomes temporarily vulnerable to disruption by administration of certain drugs before it is reconsolidated (made permanent again) (e.g., Nader et al., 2000; Kindt, Soeter, & Vervliet, 2009). The argument is that the disrupted memory is at least partly erased. There is evidence that the process is most effective with weak (undertrained) or old memories (Wang, de Oliveira Alvares, & Nader, 2009). There is also evidence that extinction conducted soon after a memory is retrieved can also interfere with reconsolidation (Monfils, Cowansage, Klann, & LeDoux, 2009; Xue et al., 2012; but see Chan, Leung, Westbrook, & McNally, 2010; Kindt & Soeter, 2013; Soeter & Kindt, 2011). We are a long way from understanding the latter phenomenon, however; for example, it is not clear why the first trial of any extinction procedure does not retrieve the memory and produce the same effect. At this point in time, we do not know enough about the conditions that permit reconsolidation to take place. Until we do, the safest approach to promoting behavior change may be to assume that lapse and relapse can potentially occur, especially with a change of context. The animal research reviewed here encourages a very broad definition of “context.” Although exteroceptive apparatus or room cues support both animal and human memory performance (e.g., see Smith & Vila, 2001), the results reviewed in the present article suggest that it is useful to think that time, recent reinforcers, and recent signal-reinforcer or response-reinforcer pairings can also serve (as in spontaneous recovery, resurgence, and rapid reacquisition, respectively). As noted earlier, previous reviews of the literature (e.g., Bouton, 1991, 2002) have suggested that drug states, hormonal states, mood states, and deprivation states can also play the role of context. At this point in time, research from the basic behavior laboratory mainly provides ideas and principles. More research with humans in applied settings will be necessary to provide more specific information about the kinds of contextual cues that may be most important to people who are undertaking behavior change. Highlights
AcknowledgmentsSupported by Grant RO1 DA033123 from the National Institute on Drug Abuse. I thank Stephen Higgins, Scott Schepers, Eric Thrailkill, and Sydney Trask for their comments and discussion of the manuscript. FootnotesPublisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. References
How do you strengthen your behavior?With positive reinforcement, behavior is strengthened by presenting a pleas- ing stimulus (a reward) after the desired behavior occurs; with negative reinforcement, behavior is strengthened by removing an aversive stimulus after the desired behavior occurs.
What strengthens behavior in operant conditioning?Positive reinforcement is a term described by B. F. Skinner in his theory of operant conditioning. In positive reinforcement, a response or behavior is strengthened by rewards, leading to the repetition of desired behavior. The reward is a reinforcing stimulus.
Does negative reinforcement weaken or strengthen behavior?Negative reinforcement can be an effective way to strengthen the desired behavior. However, it is most effective when reinforcers are presented immediately following a behavior. When a long period elapses between the behavior and the reinforcer, the response is likely to be weaker.
What process weakens operant behavior?There are five basic processes in operant conditioning: positive and negative reinforcement strengthen behavior; punishment, response cost, and extinction weaken behavior.
|
