Can Surprising Nonreward and Adjunctive Behavior Influence Each Other ?

Adjunctive or schedule-induced behavior can be defined as an increase in the frequency of a behavior under conditions of intermittent reinforcement. Adjunctive behavior has been explained as developing from reinforcement contingencies and emotional responses of frustration due to periods of reward omission, among other accounts. Two experiments with rats were designed to test the hypothesis that frustration mediates adjunctive behavior. According to this hypothesis, extensive training under conditions involving both reward and reward omission (e.g., partial reinforcement and intermittent reinforcement) promote frustration counterconditioning, a mechanism activated when reward omissions are followed by reward. Frustration counterconditioning reduces avoidance and enhances approach, inducing behavioral persistence that transfers across situations inducing frustration—a mechanism yielding transsituational transfer. In Experiment 1, Phase 1 involved runway training under continuous vs. partial food reinforcement, whereas Phase 2 involved exposure to a fixed-time 60-s schedule of intermittent food reinforcement with free access to a wheel. Counterconditioning of goal approach in Phase 1 led only to a modest increase in wheel running in Phase 2. In Experiment 2, Phase 1 involved groups exposed to either intermittent reinforcement or massed-food control, both with access to a wheel, followed in Phase 2 by runway training under continuous reinforcement and extinction. No support was found for a transfer effect. The results are discussed in relation to accounts of adjunctive behavior based on Amsel’s frustration theory and Killeen-Pellón’s reinforcement theory of adjunctive behavior.

reported that rats exposed to a variable-interval food reinforcement schedule with concurrent access to water drank excessively, more than it was expected as a result of the food-induced prandial thirst.Similar excessive behaviors were also reported under intermittent food schedules (see Killeen & Pellón, 2013;Wetherington, 1982).The fact that such excessive behaviors were not contingent with food delivery, as well as the apparent lack of contiguity with the reinforcer, led to conceptualize these behaviors as different from reinforced behavior.Their generally excessive nature also prevented their classification as unconditioned responses to the reinforcer.However, their progressive increase across sessions suggested that their emergence was linked to training under intermittent reinforcement conditions.Accordingly, they were called adjunctive behaviors (Falk, 1971) or schedule-induced behaviors (Staddon, 1977).
Understanding the mechanisms underlying the production of these behaviors is important to progress in the treatment of certain compulsive or addictive behaviors that concur with certain psychopathologies in humans (Falk, 1981;Gilbert, 1978).Different explanations of adjunctive behavior have been offered (see Baum, 2012;Boakes, Patterson, Kendig, & Harris, 2015;Killeen & Pellón, 2013;Pellón & Killeen, 2015).Such hypotheses can be grouped into three classes.First, some emphasize peripheral factors.For example, excessive drinking could result from mouth dryness produced by eating food pellets (Stein, 1964), but other adjunctive behaviors could not be explained in these terms (e.g., food-induced wheel running; Levitsky & Collier, 1968).Although peripheral factors may contribute to the exaggerated nature of adjunctive behaviors, there seems to be no unique factor that would apply to the wide variety of behaviors affected by intermittent reinforcement.
Second, some theories claim that adjunctive behaviors are controlled by reinforcement, despite the relatively long temporal gap between their peak occurrence and reinforcer delivery.Clark (1962) posed that adjunctive behaviors could be subject to adventitious reinforcement.Killeen and Pellón (2013) noted the lack of contiguity between adjunctive behaviors and reinforcement, but still suggested that they are regulated by the same mechanisms underlying operant behavior.According to this view, preexisting behaviors are reinforced according to their proximity to reinforcement and therefore compete for expression.Different responses acquire dominance at different levels of temporal proximity with respect to reinforcement.Consistent with this hypothesis, there are parallels between adjunctive and operant behaviors.For example, both appear to be affected by previous history, deprivation state, reinforcer magnitude, signaling procedures, and length of the inter-food interval, among others factors (Flory, 1971;Roper, 1978).Thus, the main difference between these behaviors lies in the degree of temporal delay required for their reinforcement (Pellón, Íbias, & Killeen, in press;Pellón & Pérez-Padilla, 2013).
Third, several theories focus on motivational and emotional processes induced by surprising nonreward derived from the intermittent reinforcement characterizing adjunctive behavior training.In these sessions, excessive behaviors tend to occur during the early portions of the inter-reinforcement interval (IRI), when the probability of reinforcement is low (Lashley & Rosellini, 1980).Surprising nonreward refers to the omission, reduction in magnitude, or quality degradation of an appetitive reinforcer that occurs in the presence of signals previously paired with a reward of greater incentive value than the current one (Papini & Dudley, 1997).According to Papini and Dudley (1997), nonreward is surprising provided that it occurs in an excitatory context, a hypothesis consistent with the low levels of adjunctive behavior when long IRIs are used (Falk, 1966a).Surprising nonreward would then induce an internal aversive emotional state of frustration, which provides a source of motivation invigorating adjunctive behavior.Thus, engaging in adjunctive behaviors would allow the animal to interfere, reduce, or even eliminate the aversive emotional state in a way that would be functionally analogous to an escape response.Thus, frustration induced by surprising nonreward derived from intermittent reinforcement would provide an internal source of reinforcement for any activity supported by the environment that reduces frustration (e.g., drinking if there is water available, wheel running if there is an accessible wheel, etc.).Papini and Dudley (1997) based their view in three findings about schedule-induced drinking.First, drinking is facilitated by large, or more preferred, reinforcers (Rosellini & Burdette, 1980;Rosellini & Lashley, 1982; but see Reid & Dale, 1983), when periods of omission would be particularly frustrating.Second, the termination of a meal would be especially frustrating when the context is strongly associated with food delivery.According to this view, schedule-induced drinking is minimal with short IRIs, it increases as the IRI increases, but it declines again with very long IRIs (Falk, 1966b).Similarly, adjunctive drinking is reduced when food delivery is scheduled according to a random-interval schedule, yielding food at a constant probability in time (Millenson, Allen, & Pinker, 1977; but see Plonsky, Driscoll, Warren, & Rosellini, 1984).Interestingly, inserting a minimum postreinforcement interval during which food was not available increase drinking in a random-interval schedule (Shurtleff, Delamater, & Riley, 1983).Third, adjunctive behavior reduces corticosterone activation triggered by intermittent reinforcement: Providing access to water or nonnutritious substances that can be chewed reduced corticosterone levels during sessions of intermittent reinforcement (Dantzer, Gonyou, Curtis, & Kelley, 1987;Levine, Weinberg, & Brett, 1979;Osborne, Sivakumaran, & Black, 1979).
The involvement of frustration in adjunctive behavior was tested directly by Thomka and Rosellini (1975), pointing out the analogy between the paradigms that produce an emotional response of frustration and the intermittent reinforcement schedules in which the occurrence of induced behaviors is consistently observed.They found evidence that a number of frustrating conditions (a reduction from 6 to 2 food pellets per trial in a conditioning chamber; partial reinforcement training in a runway) influenced excessive drinking behavior, suggesting a role of frustration in adjunctive behavior.
In the present study we sought additional evidence, based on transfer designs, for the hypothesis that frustration induced by intermittent reinforcement provides the motivation for adjunctive behavior.The rationale for these transfer designs is based on the mechanisms of counterconditioning and persistence outlined by Amsel (1992).Counterconditioning occurs during pairings between stimuli of opposite hedonic value, such as pairing electric shock with food delivery, as first reported by Pavlov (1927).In Pavlov's experiment, dogs eventually salivated to the delivery of a shock.Similarly, in Amsel's theory, pairings of anticipatory frustration (aversive) with food anticipation (appetitive), as it is bound to occur during partial reinforcement (PR) training, provide the basis for frustration counterconditioning.Since anticipatory frustration promotes avoidance and anticipatory reward promotes approach, counterconditioning facilitates the development of approach to stimuli that evoke anticipatory frustration, thus generating persistence.Interestingly, persistence that has been learned in one particular situation can transfer to a different situation whenever frustration (a common element between these situations) is evoked, even if frustration counterconditioning has not been specifically trained in the new situation.Based on the hypothesis that both PR and intermittent reinforcement that sustains adjunctive behavior induce frustration (common element), we predicted that counterconditioning generated during PR training in a runway would transfer to the intermittent reinforcement situation resulting in higher levels of wheel running relative to continuous reinforcement (CR) training (tested in Experiment 1).Vice versa, increased wheel running generated during intermittent reinforcement training would transfer to appetitive extinction in the runway (after CR training) by increasing resistance to extinction relative to a massed-food control (tested in Experiment 2).

Experiment 1
In Experiment 1, animals were exposed to either PR or CR training in a runway (Phase 1), followed by exposure to intermittent food reinforcement in a conditioning chamber equipped with a running wheel (Phase 2).We were expecting that persistence acquired during PR training in the runway during Phase 1 would transfer to adjunctive behavior in Phase 2, according to the mechanisms derived from Amsel's (1992) theory, as outlined above.Ross (1964) provided compelling evidence supporting this prediction.In Phase 1, rats received training in one context, under food deprivation, and requiring a running response to obtain food under either PR or CR.In Phase 2, animals received CR training in a different context, under water deprivation, and requiring a running response to obtain water, and finally were given extinction training.Animals that had received PR training in Phase 1 exhibited more resistance to extinction in Phase 2 than animals that had received CR training.The implication was that the final extinction phase had induced frustration, which in PR animals (but not in CR animals) was counterconditioned and tended to induce approach behavior, rather than avoidance.Similar transfer effects were reported in anxiety-prone Roman low-avoidance rats in transfer between consummatory and instrumental situations involving downshifts in reinforcer magnitude (Cuenya et al., 2015).
Whereas Thomka and Rosellini (1975) exposed rats to sessions of PR vs. CR training and intermittent reinforcement with access to water in the same day, in the present experiment animals were exposed to runway running training in Phase 1 and, once concluded, to intermittent food reinforcement with access to wheel running in Phase 2. If adjunctive wheel running induces frustration in Phase 2, then previously counter-conditioned frustration during PR training in Phase 1 should increase adjunctive wheel running relative to previous CR training.This prediction is based on the assumed compatibility of running in the runway and in the wheel (see Ross, 1964).

Method
Subjects.The subjects were 16 male Wistar rats, about 75 days old at the start of the experiment, and experimentally naïve.The mean ad lib weight was 327.5 g.Animals were food deprived until their weight was 80-85% of their ad lib weight.Animals were individually housed in home cages measuring 21 x 45 x 24 cm (L x W x D), in a room with constant humidity (45%) and temperature (21 ºC), and with a 12-hr light-dark cycle (lights on at 08:00).Animals were fed daily approximately 30 min after finishing the daily session to minimize possible interfering with task behavior.
Apparatus.A straight runway was used during Phase 1 training.The runway was made of 0.7 mm thick Plexiglas sheets, translucent on the top and black in walls and floor.The runway was 245 x 12 x 12 cm (L x W x D).It was divided into three sections: start box, runway, and goal box.The start and goal boxes were 20 cm long; the opening to the runway had a guillotine door made of a 0.3 mm thick black Plexiglas sheet.The guillotine door in the start box was kept closed when introducing the animal and it was manually raised when the trial started.The guillotine door in the goal compartment was kept open while the animal was in the runway and was closed once it had its four legs inside the goal box.All times were manually recorded with Ipso JS-307 stop watches (Extech,model 365510,Madrid,Spain).
Eight conditioning chambers were used, each measuring 29 x 24.5 x 35.5 cm (H x W x D).The walls were made of black Plexiglas, except for two walls.One had an entrance to the cage, which was made of translucent Plexiglas, and the other contained the feeder, which was made of aluminum plates.The ceiling was also made of translucent Plexiglas and the floor was made with metal rods located over a removable sawdust tray.The feeder was located in the center of the wall and connected to a food pellet dispenser.Opposite to the feeder wall was a round opening with access to an activity wheel.This wheel was 34 cm in diameter and 9 cm in width.A light located on top of the chamber provided diffuse illumination (25 W) and the ventilation system provided masking noise (60 dB).Wheel turns were recorded automatically with MED-PC-IV software.
Procedure.Subjects were randomly assigned to two groups (n = 8).Group PR/W received 50% PR training in Phase 1 and intermittent reinforcement with wheel access (W) in Phase 2. Group CR/W received CR in Phase 1 and the same training as the other group in Phase 2 (W).Prior to acquisition sessions in the runway, all animals received 3 sessions of habituation to the runway.In the first two sessions, 5 trials were administered with 12 food pellets distributed on the floor while animals were allowed to move about the runway with the two guillotine doors open.During the third habituation session, three trials were administered; the first two trials followed the same procedure described above, whereas in the third trial, animals were locked in the goal box where they found the 12 food pellets.In addition, after this last habituation session, 12 food pellets were also spread on the floor of the home cage.
Phase 1 involved acquisition training in the runway and it started the day after the last habituation session.There were 10 acquisition sessions, each including 5 trials, with a 10 min intertrial interval.Trials started with the animal being placed in the start box; the guillotine door was opened when the rat oriented itself towards the door.Once the door was raised, a stopwatch was started and stopped when the animal had its four legs inside the goal box.This was the running latency, measured in seconds.In reinforced trials for both groups, animals were kept in the goal box until all food pellets were eaten or a maximum of 30 s elapsed.In nonreinforced trials for Group PR/W, the same procedure was followed, except that the trial ended with a 30 s detention in the goal box without food.There were 5 trials per session; in Group PR/W, there were 3 reinforced / 2 nonreinforced trials or 2 reinforced / 3 nonreinforded trials alternating across days.Overall, animals in Group PR/W received 25 trials ending with access to 12 food pellets and 25 trials ending with an empty goal box (i.e., 50% PR schedule).Animals in Group CR/W received 50 trials, each one ending with access to 12 food pellets.Animals entered the training room in squads of eight.
Phase 2 started a day after completion of Phase 1.All animals received training in a fixed-time (FT) 60 s food schedule.Each cycle ended with the delivery of 1 food pellet and each session lasted 60 min.Sessions began by introducing the animal into the chamber, after which the houselight was turned on and the session started.The first food pellet was administered after the first 60 s.The houselight was turned off with the delivery of the 60th food pellet.Running in the activity wheel and the entries into the feeder, were recorded with MED-PC software.During the course of the FT schedule, animals had free access to the activity wheel through a hole in the conditioning chamber.
The data (latencies, wheel turns, and goal entries) were analyzed using IBM SPSS Statistics 24 software.When required, the source of significant interactions was determined via LSD pairwise tests derived from the main analysis.In all cases, alpha was set at the .05level.Latencies in seconds were transformed to the log10 to improve normality and allow for parametric statistics.

Results and Discussion
Figure 1, top panel, shows the results of Phase 1 training in the runway.There were nonsignificant effects for the group by session interaction, F(9, 126) = 1.74, p > .08,and no differences between groups, F < 1.However, there was a steady reduction in latency across sessions that reached significance, F(9, 126) = 76.01,p < .01.Thus, there was no behavioral evidence that PR and CR caused differential acquisition of the running response in the runway.
Figure 1, middle panel, depicts distance scores obtained from wheel running during Phase 2. There was a trend toward increased wheel running in rats that had previously received PR training in the runway, but the effect was not significant either in terms of a group or a group by session interaction, Fs < 1.13, ps > .10.However, there was a significant increase in wheel running across sessions, F(29, 406) = 4.32, p < .01.With such an extensive training period it is possible that a small effect could be masked by a relatively large number of sessions in which the groups did not differ.Indeed, there was modest evidence that the performance of Group PR/W increased above that of CR/W during the middle portion of training.We calculated group by session effects independently for sessions 1-10, 11-20, and 21-30.On sessions 1-10, there was only an increase across sessions, F(9, 126) = 5.55, p < .01,but no group or group by session effects, Fs < 1.29, ps > .25.On sessions 11-20, however, there was a significant interaction effect, F(9, 126) = 2.07, p < .04.Follow-up pairwise LSD tests derived from the main analysis indicated that the source of the interaction was significantly higher wheel running in Group PR/W relative to CR/W on session 20, F(1, 14) = 5.60, p < .04.Differences in the same direction fell short of significant for sessions 14, F(1, 14) = 4.55, p = .05,and 19, F(1, 14) = 4.07, p = .06.Other pairwise comparisons were not significant, Fs(1, 14) < 1.43, ps > .25.A similar analysis for sessions 21-30 yielded nonsignificant effects, Fs < 1.
Figure 1, bottom panel, depicts goal entries during Phase 2. There was a tendency for Group PR/W to exhibit somewhat higher frequency of goal entries toward the end of this phase, but this trend was clearly inconclusive.An overall analysis only found a significant increase across sessions, F(29, 406) = 4.92, p < .01;other effects were nonsignificant, Fs < 1.32, ps > .27.
Figure 2 plots wheel-running data in 2-s bins to map this adjunctive behavior in terms of its distribution along the IRI.These three panel show performance during the early (sessions 1-4), middle (sessions 14-17), and late (sessions 27-30) portions of Phase 2 training.Each plot represents the mean performance for each group collapsed from all 59 IRIs within each of four sessions.Two properties of these functions are worth noting.First, initially the functions are almost flat, but a peak develops and is already present in the middle portion of training.Second, that peak occurred during the first half of the IRI interval.There was a tendency for animals that had received PR training to show increase wheel running in the early and middle portions of training, but not late in training.However, group by 2-s bin analyses for each of the three sets of data revealed only significant bin effects, Fs(29, 406) > 6.73, ps < .01.The main group effect and the interaction were nonsignificant in the three analyses computed, Fs < 1.02, ps > .45.
These results provided only weak evidence supporting predictions derived from Amsel's (1992) theory for situations involving transfer effects based on procedures that induce counterconditioning of frustration, such as PR training.They are also weakly consistent with previous results showing transfer across situations involving exposure to unexpected reward omissions (e.g., Cuenya et al., 2015;Ross, 1964).Their implications and limitations are considered in the General Discussion.

Rick et al. 146 Experiment 2
Animals in Experiment 2 were first exposed to a FT 60 s schedule of food delivery or to a massed-food control condition during Phase 1, and then given CR training followed by extinction in a runway during Phase 2. Massed food has been used as a control in adjunctive behavior experiments (Roper, 1981).In this situation, all the food pellets are delivered at the beginning of the experimental session while the animals are kept in that situation for the same duration as those exposed to intermittent reinforcement.Massed-control conditions match groups in terms of food amount and session length, but eliminate the intermittent aspect that induces excessive behavior.Thus, Experiment 2 looked for evidence of transfer by administering the same tasks, but in opposite order relative to Experiment 1.
If intermittent reinforcement (but not massed-food delivery) induces counterconditioned frustration due to pairings between wheel running and food reinforcement (given their relative temporal proximity; see Killeen & Pellón, 2013), then animals exposed to intermittent reinforcement in Phase 1 should exhibit increased resistance to extinction in Phase 2 relative to massed-food controls.Notice that Phase 2 training does not involve counterconditioning since animals are trained under CR, but it does involve frustration when animals are shifted to extinction.

Method
Subjects and apparatus.Sixteen male experimentally naïve Wistar rats, 10 weeks old at the start of the experiment, served as subjects.Their mean ad lib weight was 307.5 g.Before to the start of this experiment, their weights were reduced to 80-85% of the individual ad lib weight through food restriction.Housing, maintenance, and apparatus were the same described in Experiment 1.
Procedure.Subjects were randomly assigned to two groups (n = 8): W/CR and C/CR.In Phase 1, Group W/CR was treated identically to Group CR/W in Phase 2 of Experiment 1 (i.e., intermittent reinforcement with wheel access).In Group C/CR, animals found 60 food pellets placed in the conditioning chamber and remained there with wheel access for 60 min with the houselight on, just as it was the case for Group W/CR, except for the simultaneous presentation of food pellets, rather than intermittent presentation.All other aspects were as described in Experiment 1.
In Phase 2, both groups received habituation training and acquisition training as described for Group CR/W during Phase 1 in Experiment 1.The day following the last acquisition session, animals were exposed to 7 extinction sessions equal in all respects to acquisition sessions, except that no food was delivered in the goal box.Once an animal reached the goal compartment, it was kept there for 30 s.
All other procedural aspects were as described in Experiment 1.

Results and Discussion
Figure 3, top panel, shows the mean wheel running measured in meters during Phase 1. Group W/CR ran consistently more than Group C/CR.An analysis indicated that the session increase in wheel running was statistically significant, F(29, 406) = 2.22, p < .01.No statistically significant groups by session interaction, F(29, 406) = 1.05, p > .40,or group effect, F < 1, were observed.The lack of a statistically significant effect between groups was entirely due to one subject in Group C/CR that produced a high level of running; the mean wheel running over the 30 sessions in Phase 1 for this animal was 402.6 m, whereas for the rest of the group, it was 148.9 m (±15.8 m).To avoid failing to identify an effect because of a single outlier, we calculated a similar analysis without this rat.In this case, there was a significant group difference, F(1, 13) = 7.76, p < .02,and also a significant increase across sessions, F(29, 377) = 1.54, p < .04.The interaction was not significant, F(29, 377) = 1.15, p > .27.Thus, intermittent vs. simultaneous food delivery appeared to have been effective in all but one control animal.
Figure 3, middle panel, shows magazine entries per session for both groups.Whereas groups produced similar scores during the initial three sessions, there was a clear differentiation across sessions, with rats in the massed-food control condition exhibiting substantially fewer magazine entries than animals in the intermittent reinforcement condition.An analysis yielded significant effects for the group, session, and interaction effects, Fs > 3.03, ps < .01.LSD pairwise tests derived from this analysis indicated that group W/CR displayed a higher level of goal entries than Group C/CR in all sessions, except the initial two, Fs(1, 154) > 16.81, ps < .01. Figure 3, bottom panel, shows the results of runway performance in Phase 2 during acquisition (1-10), extinction (11-17), and reacquisition (18-19) sessions.The relatively higher variability of data in Group C/CR was due to the same animal that performed an unusually high level of wheel running during Phase 1.Still, with this animal included, there was only a significant acquisition effect, F(9, 126) = 6.17, p < .01,but no interaction or group effects, Fs < 1.53, ps > .14.Extinction was also significant across sessions, F(6, 84) = 2.79, p < .02,but again the group and group by session interaction effects were not significant, Fs < 1.05, ps > .40.Finally, there was a significant reacquisition effect, F(1, 14) = 14.26, p < .01,but again nonsignificant group or group by sessions interaction effects, Fs < 1.
Figure 4 shows the same within IRI analysis performed for data in Experiment 1 and the results for Group W/CR were similar to those for CR/W in the previous experiment (compare Figures 2 and 4).Adjunctive wheel running developed gradually, but rapidly across sessions in animals exposed to intermittent reinforcement, but not at all in massed-food controls, which exhibited a flat response distribution.Separate analyses were computed for each data panel in Figure 4.The overall group by session analysis provided significant interaction effect for all three periods examined, Fs > 8.48, ps < .01.These analyses also showed significant performance changes across sessions in all three cases, Fs > 6.41, ps < .01,but none of them revealed a significant group effect, Fs < 1. Pairwise analysis using the LSD test indicated that adjunctive behavior changed the most toward the end of the IRI.Thus, for example, early in training (Figure 4, top panel) Group W/CR performed below C/CR on bins 2, 3, 6, and 8-10, Fs(1, 14) > 4.82, ps < .05.Notice the absence of a group effect late in the IRI.In the middle and late sessions (Figure 4, middle and bottom), groups differed, in different directions, early, in the middle, and late in the IRI.As an example, the average of sessions 27-30 yielded group differences on bins 1-3, 6-14, and 24-30, Fs(1, 14) > 4.84, ps < .05.
Although Experiment 2 provided good evidence of differential performance in Groups W/CR vs. C/CR, this experience provided no evidence that it affected runway extinction.It also did not affect acquisition and reacquisition performance in the runway, but the key prediction was concerned with extinction performance.

General Discussion
The present work intended to discern whether PR vs. CR training in Phase 1 affected adjunctive behavior in Phase 2 (Experiment 1) and whether intermittent-vs.massed-food deliveries in Phase 1 affected runway extinction in Phase 2 (Experiment 2).Predictions derived from Amsel's (1992) theory suggested that surprising nonreward resulting from both PR and intermittent reinforcement would lead to the counterconditioning of running performance, whether in the runway or the conditioning chamber (Papini & Dudley, 1997).Such counterconditioning was expected to increase running behavior leading to higher wheel running in Experiment 1, Phase 2, and increase persistence in extinction in Experiment 2, Phase 2. Of these two predictions, the results provided only weak confirmation for the first one.Therefore, if one takes these results at face value, it could be argued that transfer between these two tasks was asymmetrical, working in one direction (PR to intermittent reinforcement) better than in the opposite direction (intermittent reinforcement to extinction).These results are discussed below in terms of two main topics: First, assuming Amsel's (1992) theory is correct, how could these transfer effects be enhanced?Second, assuming that there is no transfer between these two types of running responses, what other theories would be consistent with these results?In these experiments it seems unlikely that failures to observe transfer were due to inappropriate training in the intermittent reinforcement phase since good evidence for the development of adjunctive behavior was observed.By contrast, there was no evidence that PR training produced any effect on behavior in Experiment 1, and also extinction was relatively fast in Experiment 2. However, lack of a PR effect on acquisition in Experiment 1 does not necessarily imply that PR did not affect subsequent behavior.Compared to extinction, acquisition performance is often insensitive to PR training.For example, Boughner and Papini (2008) trained rats in PR vs. CR autoshaping acquisition, followed by extinction.Although PR training had no impact on acquisition relative to CR, extinction was slower after PR than CR-a partial reinforcement extinction effect.Gómez et al. (2008) and Cuenya et al. (2012) reported similar findings with runway running.In any case, future experiments may reconsider the parameters of runway training to determine if enhancing frustration counterconditioning would promote transfer across situations.Such transfer would suggest that internal reinforcement derived from frustration reduction is involved in excessive wheel running.For example, provided that the strength of frustration counterconditioning is directly related to the length of PR training (Amsel, 1992), the weak transfer effect evident in these data (Figures 1 and 2) could be potentiated by a more extensive acquisition training in the runway.In addition, introducing a delay of reinforcement in the goal box would also encourage the counterconditioning of frustration (Shanab & Ferrell, 1975).Having PR combined with delay of reinforcement and a large reinforcer may yield a strong level of frustration counterconditioning sufficient to affect adjunctive behavior.Experiment 2 tested the hypothesis that frustration counterconditioning present during adjunctive training could transfer to appetitive extinction in the runway.In this case, lengthening acquisition training may also help detect this effect if this change were to slow down extinction without inducing counterconditioning.
An interesting aspect of Amsel's (1992) theory is that it identifies a source of behavioral control that transcends current conditions, thus allowing for transfer across situations that are otherwise very different (e.g., Ross, 1964).For such a transfer to occur, there must be a common element between situations.In this case, that common element is the internal response of frustration induced by surprising nonreward and the corresponding reinforcement failure.It is possible that the weak, at best, effects observed in the present experiments simply reflect that frustration is not a prominent feature in the adjustment to intermittent reinforcement schedules, unlike it was suggested by Papini and Dudley (1997).This would leave us to account for Thomka and Rosellini's (1975) results.In their Experiment 2, rats received runway training under PR and CR.When the runway performance of both groups was equivalent, about 30-90 min after each session, rats were exposed to intermittent reinforcement in a schedule similar to that used in the present experiments, except that water, rather than a wheel, was freely available during the session.Under these conditions, rats that received PR training in the runway later that day consumed less water than animals that received CR training.The authors' interpretation of these results was that having learned to tolerate frustration in the runway, animals also tolerated frustration in the intermittent reinforcement situation.The implication is that one function of adjunctive behavior is to reduce frustration induced by periods of reward failure.
The differences between Thomka and Rosellini's (1975) results and the present results in Experiment 1 may relate to the type of adjunctive behavior measured in each case.To illustrate this point, consider groups that receive PR training for either running or climbing, and are tested for transfer in terms of running (Ross, 1964).Consistency between responses in animals trained and tested with running produced positive transfer (i.e., increased persistence in extinction), whereas incompatible responses in the case of climbing vs. running yielded negative transfer (i.e., reduced persistence in extinction).Similarly, runway running and wheel running compatibility suggests that frustration counterconditioning should enhance adjunctive behavior (positive transfer), whereas runway running and drinking incompatibility suggests that adjunctive behavior should be reduced (negative transfer).This interpretation is undermined by recent data showing positive transfer between running and drinking in tasks involving a downshift on reinforcer magnitude (Cuenya et al., 2015).Thomka and Rossellini's (1975) results could depend on differences in IRI distributions for various responses.Just as adjunctive drinking has been shown to increase immediately after food delivery, adjunctive running seems to be more widely distributed throughout the IRI.Such a difference has implications for theories that pose that adjunctive behavior is also the result of instrumental reinforcement (Killeen & Pellón, 2013).If PR training in the runway induced a sort of intermittent goal approach in Thomka and Rosellini's experiment, this could have interfered with the distribution of other behaviors, thus leading to intermittent drinking.Thus, lower drinking for PR animals could be the result of a more degraded drinking-food contingency in PR animals.In addition, the fact that in Thomka and Rosellini's (1975) experiment runway and conditioning chamber sessions were administered daily may have maintained response patterns throughout the experiment.By contrast, the present design used a sequential arrangement of runway and conditioning chamber sessions, something that would have favored an adjustment of wheel running to the temporal distribution food, thus diminishing the influence of the previous runway experience.In the absence of a strong influence from prior experience, the relevant controlling factor for wheel running seems to be the instrumental contingency with the food reinforcer, as modeled by Killeen and Pellón (2013).If one assumes that there was no transfer between the two types of running responses used in the present study, then the theory of adjunctive behavior based on external reinforcement could be consistent with present results.Specifically, the absence of a transfer effect in Experiment 1 can be explained by the fact that both PR and CR training yielded a similar degree of (reinforced) runway running at the end of Phase 1.In Experiment 2, Phase 2, the absence of transfer can be explained by the relatively rapid development of runway running in both groups (compare the bottom panel of Figure 3 with the top panel of Figure 1).Notice that, according to Killeen and Pellón (2013), running in Group W/CR should have been reinforced during Phase 1 of Experiment 2 to a greater extent (in principle) than in Group C/CR.Thus, the fact that partial reinforcement training only subtly affected the subsequent development of the adjunctive behavior of wheel running (Experiment 1), as well as the absence of transfer obtained in Experiment 2 during extinction, could indicate a lack of a frustration component during the intermittent reinforcement training.The present weak support for a mediational role of frustration counterconditioning to account for transfer to and from adjunctive behavior becomes meaningful if one considers adjunctive behavior as resulting from food reinforcement, in agreement with past (Killeen & Pellón, 2013;Pellón & Killeen, 2015) and current evidence (e.g., Álvarez, Íbias, & Pellón, 2016).Frustration counterconditioning should be expected to play a significant role if one looks at behavior as a stream of organized activities (adjunctive behavior) aiming at reward seeking, rather than considering the reduction of frustration as the source of reinforcement for adjunctive behavior.These activities will be initially induced by reward delivery and be subsequently reinforced by the consistent occurrence of reinforcement at the end of the behavioral chain (see Ruiz, López-Tolsa, & Pellón, 2016).This integrative theoretical view makes no fundamental distinction between the different kinds of activities that occur during intermittent reinforcement schedules.

Figure 1 .
Figure 1.Means (±SEMs) running latencies (in log10 seconds) for partial reinforcement (PR/W) and continuous reinforcement (CR/W) groups during runway acquisition (top panel), meters ran in the activity wheel during FT 60 s in the conditioning chambers (middle panel), and magazine entries during FT 60 s in the conditioning chambers (bottom panel).PR: partial reinforcement; CR: continuous reinforcement; W: access to a wheel during intermittent reinforcement.Error bars were always included, but they are difficult to visualize in some cases.

Figure 2 .
Figure 2. Mean (±SEMs) distribution of wheel running (in meters) within inter-reinforcement intervals plotted in 2 s bins for the initial, intermediate, and final sessions of exposure to the FT 60 s food delivery schedule, for the partial reinforcement (PR/W) and continuous reinforcement (CR/W) groups.PR: partial reinforcement; CR: continuous reinforcement; W: access to a wheel during intermittent reinforcement.Error bars were always included, but they are difficult to visualize in some cases.

Figure 3 .
Figure 3. Means (±SEMs) ran in the activity wheel (in meters) for the experimental (W/CR) and control (C/CR) groups (top panel) and magazine entries (middle panel) during initial training with FT 60 s in the conditioning chambers, and running latency (in log10 seconds) during acquisition, extinction, and reacquisition in the runway (bottom panel).W: access to a wheel during intermittent reinforcement; C: massed-food control; CR: continuous reinforcement.Error bars were always included, but they are difficult to visualize in some cases.

Figure 4 .
Figure 4. Mean (±SEMs) distribution of wheel running (in meters) within inter-reinforcement intervals plotted in 2-s bins for the initial, intermediate, and final sessions of exposure to the FT 60 s food delivery schedule, for the experimental (W/CR) and control (C/CR) groups.W: access to a wheel during intermittent reinforcement; C: massed-food control; CR: continuous reinforcement.Error bars were always included, but they are difficult to visualize in some cases.