According to operant conditioning, what is the term for a stimulus that cues a reward contingency?

Abstract

The operant conditioning has been less studied than the classical conditioning as a mechanism of placebo-like effect, and two distinct learning mechanisms have never been compared to each other in terms of their neural activities. Twenty-one participants completed cue-learning based pain rating tasks while their brain responses were measured using functional magnetic resonance imaging. After choosing [instrumental] or viewing [classical] one of three predictive cues [low- and high-pain cues with different level of certainty], they received painful stimuli according to the selected cues. Participants completed the same task during the test session, except that they received only a high pain stimulus regardless of the selected cues to identify the effects of two learning paradigms. While receiving a high pain stimulation, low-pain cue significantly reduced pain ratings compared to high-pain cue, and the overall ratings were significantly lower under operant than under classical conditioning. Operant behavior activated the temporoparietal junction significantly more than the passive behavior did, and neural activity in the primary somatosensory cortex was significantly reduced during pain in instrumental as compared with classical conditioning trials. The results suggest that pain modulation can be induced by classical and operant conditioning, and mechanisms of attention and context change are involved in instrumental learning.

Subject terms: Neuroscience, Psychology

Introduction

The placebo effect, which occurs when an inactive treatment leads to positive outcomes, is a genuine example of mind–body interaction, as it demonstrates conscious or unconscious modulation of our brain, behavior, and physiological responses. Among various mechanisms underlying the placebo effect, including expectation, conditioning with our without conscious learning, and reward learning, the most extensively studied mechanisms are expectation [e.g., expectation of clinical improvement] and conditioning [e.g., conditioned therapeutic effect induced by placebo administration]1–3. The role of expectation in the placebo effect is mainly evident in verbally induced expectancy manipulation paradigms, such as suggestion of positive treatment effects4–6, and open-hidden administration of drugs7–10. While verbally manipulated expectations modulate human behaviors explicitly and consciously, classical Pavlovian conditioning changes a neutral stimulus into a conditioned stimulus, which subsequently elicits involuntary [conditioned] responses after repeated pairings with the unconditioned stimulus. Classical conditioning does not solely produce unconscious physiological changes [e.g., drug-like responses by administration of inert vehicle or by conditioned cue stimulus after repeated drug administration1,11,12], on the other hand, it also modulates our expectations for the environment. Previous studies have suggested that the conditioning procedure generates or alters expectation, and expectation changes the effect of conditioning on the placebo effect3,13,14. In summary, evidence suggests that learning, either conscious or unconscious, is a crucial mechanism underlying the placebo effect induced by a classical conditioning paradigm15,16.

Classical conditioning requires active processing of the predictive value of a conditioned stimulus, which elicits expectations and results in the acquisition of a conditioned response17. Operant conditioning [also called instrumental conditioning] is another type of learning procedure that involves a reinforcer or punisher, which increases or reduces, respectively, the frequency of a voluntary behavior. The classical conditioning paradigm has been extensively studied as a mechanism and means of generating the placebo effect since early stages of placebo research18–21, whereas studies of other types of learning were initiated later. Fordyce suggested that pain behaviors might occur as a reinforcement [e.g., avoiding aversive events], eventually leading to chronic pain22. In a series of studies by Hölzl and Becker, implicit learning of pain relief and pain increase was combined with operant conditioning [temperature-lowering behavior was rewarded, and temperature-increasing behavior was punished] in healthy23,24 and chronic pain patients25. The results showed that intrinsic operant reinforcers were effective in changing sensitization and habituation behaviors. Recently, placebo analgesia was successfully induced by operant conditioning paradigm applying the contingency between pain responses and reward/punishers26. As previous studies have demonstrated, pain relief is a reward and serves as a reinforcer27–29. Operant conditioning using extrinsic or intrinsic reinforcers/punishers leads to changes in pain perception through reward learning.

Although learning processes are related to the classical and operant conditioning-induced placebo effects, overlapping and distinct mechanisms underlying expectations and various types of learning are still under debate. Only few studies have investigated the neural substrates of operant conditioning of pain modulation30–32, and it still remains unrevealed how our brain is working during different types of learning tasks and placebo analgesic phase. Functional neuroimaging techniques, such as functional magnetic resonance imaging [fMRI] and positron emission tomography [PET], have revealed brain regions involved in the effect of placebo on pain perception in humans, such as anterior cingulate cortex, prefrontal cortex, and periaqueductal gray33,34. However, to the best of our knowledge, no studies have yet been conducted investigating disparate neural activities mediating the pain modulation effect via operant and classical learning processes.

In the current study, we aimed to test the validity of pain relief as a reinforcer of instrumental behavior in a pain modulation paradigm, and identify brain regions which are likely to have different roles in operant and classical learning processes. We hypothesized that brain activities through operant conditioning might be different from brain activities through classical conditioning, as the former requires intentional exploration of the environment and instrumental behavior. However, we did not limit our analysis to a specific region due to the lack of evidence. To test our hypothesis, we implied fMRI during operant and classical learning tasks and asked participants to choose actively [operant conditioning] or to see passively [classical conditioning] predictive visual cues, which suggest the amount of pain relief, and rate the intensity of painful mechanical stimuli. We measured their learning behavior using a number of cue choices in the operant conditioning session, and the placebo-like effect was defined by the influence of visual cues [one implied that they would receive high pain stimulus while two other cues implied that they would receive low pain stimulus] on subjective pain ratings to the same noxious stimuli in both operant and classical conditioning session. Functional activities during the cue choice/viewing tasks and pain perception were compared between the operant and classical test sessions.

Results

Operant learning of pain relief cues

In this study, we measured participants’ learning behavior using the number of trials in which they selected each visual cue. To evaluate each participant’s choices of visual cues—the uncertain low pain [UL] cue, the certain low pain [CL] cue, and the certain high pain [CH] cue- during the cue-selection task in the operant conditioning session, we conducted a repeated-measure ANOVA test, which revealed the significant effect of visual cues on the cue choice task [p = 1.888e−13; Cohen’s f = 1.21, 95% confidence interval [0.78, 1.6]]. Post hoc testing with the Bonferroni correction showed that both low-pain cues were significantly more frequently selected than the high-pain cue [UL vs. CL: t = −1.36, p = 0.54; UL vs. CH: t = −7.20, p = 3.4e−09; CL vs. CH: t = −5.84, p = 6.7e−07; Fig. 1].

Operant learning of visual cues. Error bars represent between-subject standard errors. Participants preferred cues predicting low pain to those predicting high pain regardless of the certainty of the cues, suggesting that the participants learned the association between visual cues and pain levels through operant conditioning [UL vs. CL: t = −1.36, p = 0.54; UL vs. CH: t = −7.20, p = 3.4e−09; CL vs. CH: t = −5.84, p = 6.7e−07].

Intensity ratings of the second pain

Participants were asked to rate the relative magnitude of pain induced by the second painful stimulus [various pinprick needles adjusted according to the selected cues] relative to the first painful stimulus [512 mN pinprick needle] which serves as a reference pain intensity. We tested this method in our previous study, and it allowed us to minimize sensitized or habituated responses to repeated painful stimuli35.

For overall pain ratings, two-way ANOVA showed significant main effects of type of learning [p = 0.04; Cohen’s f = 0.17, 95% confidence interval [0.03, 0.30]] and visual cues [p = 2e−16; Cohen’s f = 0.63, 95% confidence interval [0.48, 0.77]]. The post hoc t-tests with Bonferroni correction showed that the pain ratings were significantly lower in operant than in classical learning trials [operant vs. classical learning: t = −2.011, p = 0.045]. The pain intensity ratings following the UL and CL cues were significantly lower than those following the CH cue, which we defined as placebo-like effect driven by learning the associations between cues and pain intensities [UL vs. CH: t = 7.573, p = 4.7e−09; CL vs. CH: t = 8.588, p = 3.8e−11]. There was no significant difference in the intensity of pain between the UL and CL trials [t = −1.043; p = 0.55].

Conditioning session

Pain intensity ratings for the second pain were 22.3 ± 3.8 [mean ± standard error; active UL], 18.5 ± 2.6 [active CL], 57.6 ± 5.7 [active CH], 24.1 ± 3.2 [passive UL], 21.7 ± 3.6 [passive CL], 64.6 ± 4.6 [passive CH] during the conditioning session. Two-way ANOVA revealed no significant, but marginal main effect of conditioning type [operant vs. classical conditioning, p = 0.059; Cohen’s f = 0.21, 95% confidence interval [0.00, 0.14]], but found a significant main effect of cues on the pain ratings [p = 2e−16; Cohen’s f = 1.85, 95% confidence interval [1.52, 2.16]] during the conditioning session on day 2. A post hoc test showed that the subjective intensities of the painful stimulus following the UL and CL cues were significantly lower than the ratings following the CH cue [UL vs. CH: t = −15.46, p = 2.3e−16; CL vs. CH: t = −16.70, p = 2e−16]. The intensities of subjective pain did not differ significantly between the UL and CL trials [UL vs. CL: t = 1.26, p = 0.42; Fig. 2].

Pain ratings during the conditioning and test sessions on day 2. Bars represent the average relative pain ratings in response to the second pain stimulus relative to the reference pain, and error bars represent between-subject standard errors. Significant main effects of type of learning [p = 0.04; Cohen’s f = 0.17, 95% confidence interval [0.03, 0.30]] and visual cues [p = 2e−16; Cohen’s f = 0.63, 95% confidence interval [0.48, 0.77]] were determined by two-way ANOVA. The post hoc t-tests with Bonferroni correction showed that the pain ratings were significantly lower in operant than in classical learning trials [operant vs. classical learning: t = −2.011, p = 0.045]. Additionally, the intensity ratings of pain experienced with the 256 mN pinprick needle following the UL and CL cues were significantly lower than those for the same painful stimulus [the 256 mN needle] following the CH cue [UL vs. CH: t = 7.573, p = 4.7e−09; CL vs. CH: t = 8.588, p = 3.8e−11]. There was no significant difference in the intensity of pain between the UL and CL trials [t = −1.043; p = 0.55].

Test session

Pain intensity ratings for the second pain were 56.0 ± 5.2 [active UL], 51.7 ± 4.9 [active CL], 69.1 ± 4.0 [active CH], 62.4 ± 4.7 [passive UL], 89.7 ± 5.7 [passive CL], 73.2 ± 4.3 [passive CH] during the test session. With respect to the intensity of pain reported during the test session on day 2, two-way ANOVA revealed significant main effects of type of learning [p = 0.008; Cohen’s f = 0.32, 95% confidence interval [0.12, 0.53]] and visual cue [p = 1.26e−05; Cohen’s f = 0.49, 95% confidence interval [0.26, 0.69]]. A post hoc test showed that the pain ratings were significantly lower in operant learning trials than in classical, thus the stimulus was perceived as less painful with operant learning than with classical conditioning [active vs. passive: t = −2.67, p = 0.009]. The intensities of pain following the UL and CL cues were significantly lower than those following the CH cue [UL vs. CH: t = −3.50, p = 0.04; CL vs. CH: t = −4.61, p = 0.006], as determined by the post hoc with Bonferroni correction test. The subjective pain intensities did not differ significantly between the UL and CL trials [t = 1.18; p = 0.46; Fig. 2].

We compared the conditioning induced placebo analgesic effects by two types of learning using paired t-test for a [pain ratings in high-cue trials—those in low-cue trials] contrast of active vs passive condition, however, the pain modulation effect did not differ significantly between the operant and classical learning conditions [t = −0.047, p = 0.96].

Brain activity during cue-selection tasks [test session]

To identify the neural mechanisms underlying the operant choice behavior, we investigated brain activity related to the cue-selection task in active learning trials compared to the activity related to the passive selection task during classical learning trials when participants clicked the cue previously determined by the computer. During the test session, the active cue-choice task evoked significantly greater activation in the left inferior parietal gyrus and superior temporal sulcus, the regions comprising the temporoparietal junction [TPJ], than did the passive selection task [peak coordinate: X = −45.5, Y = −60, Z = 56.5, 38 voxels, Z score = 2.89, cluster-level FWE corrected p < 0.05; Fig. 3a].

Brain responses during cue selection [a] and painful stimulation [b] [operant vs. classical learning]. [a] Voxels in the left inferior parietal gyrus and superior temporal sulcus showed significantly greater activity during the voluntary choice of cues based on the anticipation of pain relief formed through instrumental learning compared to a simple motor task in classical conditioning [clicking the cue selected by the computer; peak coordinate: X = −45.5, Y = −60, Z = 56.5, 38 voxels, cluster-level FWE corrected p < 0.05]. [b] Perception of pain following the instrumental learning cues, compared with identical painful stimulation following the classical conditioning cues, was associated with decreased activity in the left primary somatosensory cortex, primary motor cortex, and parietal lobe [peak coordinate: X = −52.5, Y = −18, Z = 60, 94 voxels, cluster-level FWE corrected p < 0.05]. [c] Pearson’s correlation analysis was performed to analyze the correlation between the parameter estimates of the left SI and subjective pain ratings. We found significant correlation between the reduced left SI activities and decreased pain ratings [r = 0.50, p = 0.024]. SI, primary somatosensory cortex; TPJ, temporoparietal junction.

Brain activity during pain perception [test session]

We modeled the brain responses to the second pain in the test session regardless of the type of learning using fMRI] and found that the left primary and secondary somatosensory cortices, primary motor cortex, right superior parietal gyrus, bilateral superior frontal gyrus, anterior cingulate cortex, thalamus, caudate, and anterior-middle insula were significantly activated, while activity in the bilateral lingual and fusiform gyrus was significantly reduced in response to the painful mechanical stimulus [Table ​1].

Table 1

Brain regions showing increased or decreased activation in response to painful mechanical stimulation.