Plasticity of a Respondent Behavior

The marine mollusc Aplysia has provided several well-known examples of aversive and appetitive classical conditioning that have been studied at the cellular level (Walters et al., 1979, 1981; Carew et al., 1981; Colwill et al., 1988, 1997; Lechner et al., 2000a). In a typical aversive procedure, the animal is trained to produce a strong protective reflex of its respiratory apparatus—a gill and an exhalent cutaneous fold (the siphon)—in response to the application of a light, tactile stimulus. In naive unconditioned animals, a brief and light touch to the siphon elicits a weak gill and siphon contraction. During a training period, this CS is associated with an US that consists of an aversive electrical shock delivered to the animal’s tail. This latter stimulus alone is sufficient to elicit a defensive reaction that includes a powerful gill and siphon contraction (Carew et al., 1981). Following repetition of such a CS-US pairing, the CS alone becomes capable of eliciting a strong gill and siphon withdrawal. This conditioned response is expressed for several days after its induction, and it is extinguished under repeated CS alone. It was only weakly expressed after a training protocol in which the CS and US are presented randomly or alone. Moreover, conditioning fails when the onset of the CS precedes that of the US with a time interval longer than 1 second, or with a backward conditioning in which the US precedes the CS (Hawkins et al., 1986). Finally, a decrease in contingency of the stimuli by adding supplemental US not associated with the CS in the training procedure no longer induces the conditioned response. The requirement of a strong temporal correlation between the stimuli in learning and retention/extinction of the conditioned response thus indicated that conditioning of the gill and siphon withdrawal reflex in Aplysia shares essential features of Pavlovian conditioning as found in vertebrates (Rescorla, 1967).

Additional features of conditioning in the gill and siphon withdrawal reflex provide information about neuronal changes involved in learning. First, the reflex can be modified by differential conditioning in which two different CS are delivered in alternation to different sites on the animal skin (Carew et al., 1983). One of them is closely paired with the US after a short delay. The other stimulus is expressly unpaired, terminating long before the US onset. Only the CS explicitly paired with the US subsequently elicited a conditioned response. This stimulus specificity, in which two separate CS-related sensory inputs that are known to activate a common group of interneurons and motor neurons, indicated that learning-induced plasticity may be partly determined by the sensory neurons themselves. Furthermore, learning was found to (p. 539) transform a preexisting response to a CS into a conditioned response that resembles the UR (Hawkins et al., 1989; Walters, 1989). The CS can initially elicit a normal forward contraction of the siphon. After its pairing with an electrical shock, this CS alone comes to evoke a backward contraction that resembles the US response. In contrast, after pairing this CS with an electrical shock to the mantle covering the gill, which alone triggers a forward contraction, the CS alone becomes able to elicit such a forward contraction. Thus, since two distinct classes of motor neurons produce backward and forward bending of the siphon (Frost et al., 1988), this response specificity suggests that learning may also modify specific motor neurons and/or presynaptic interneuronal pathways.

Neuronal Organization of the Sensory-Motor Pathways

At least 100 neurons, including sensory, interneurons, and motor neurons, contribute to the CS-elicited siphon withdrawal behavior of Aplysia (Cleary et al., 1995; Frost & Kandel, 1995). In the absence of sensory stimulation, most of these circuit neurons express no relevant or patterned electrical activity, but only produce coordinated activity in response to the siphon stimulation. Clusters of mechanosensory neurons with receptive fields over the siphon make conventional monosynaptic excitatory connections with clusters of siphon motor neurons in the abdominal ganglion. Among these sensory neurons, the LE neurons contribute up to 60% of the monosynaptic excitation of the LFS motor neurons and determine 30% of the siphon contraction amplitude (Antonov et al., 1999). The sensory neurons also activate the motor neurons via polysynaptic interneuronal pathways that include excitatory, inhibitory, and modulatory interneurons. These interneuronal pathways increase the contribution made by the monosynaptic circuitry to siphon contraction. They also distribute information to other neuronal circuits outside of the reflex pathway, such as in the network responsible for spontaneous respiratory pumping, in order to coordinate the motor responses (Hawkins et al., 1981).

The unconditioned tail and siphon contraction is mediated by sensory neurons and interneurons that are essentially distinct from the CS-elicited pathway (Cleary et al., 1995). Nevertheless, the US-activated tail sensory neurons also excite polysynaptic interneuronal pathways that belong to or converge onto the sensory, interneurons, and motor neurons of the CS-elicited reflex pathway. Interestingly, this interneuronal pathway activated by the US contains modulatory interneurons, such as the serotoninergic CB1 neurons and the putative peptidergic/nitrergic L29 neurons, which interact and modify the functional properties of the neurons in the CS pathway (Mackey et al., 1989; Antonov et al., 2007).

Neuronal Plasticity Underlying Conditioned Respondent Behavior

Learning-induced neuronal plasticity has been investigated by making intracellular recordings in reduced preparations that consist of neuronal ganglia connected via their peripheral nerves to the tail and siphon (Antonov et al., 2001). In preparations from naïve animals, a CS elicits a transient burst of action potentials in the sensory-motor CS pathway that leads to a weak, short-lasting siphon contraction (Hawkins & Schacher, 1989; Frost et al., 1997; Antonov et al., 2001). Repeated pairing of this CS with the US that reproduces classical conditioning in vivo gradually increases the firing properties in the CS pathway. For several tens of minutes after this in vitro training procedure, the CS elicits a stronger burst of action potentials in the sensory and motor neurons and a powerful contraction of the siphon, like the conditioned response in vivo. This long-term and pairing-specific change in neuronal firing is associated with an increase in excitability of the sensory LE neurons, a facilitation of their monosynaptic connection with the motor neurons, and changes in the interneuronal inputs to the motor neurons (Antonov et al., 2001).

The neuronal plasticity in the CS pathway is thought to be induced by modulatory transmitters, such as serotonin (5-HT), nitric oxide (NO), small cardioactive peptide (SCP), and by conjoint electrical activity in the CS and US pathways (Antonov et al., 2003, 2007; Roberts & Glanzman, 2003). In reduced preparations consisting of the isolated neural ganglia attached to the tail, an association of the tail shock (US) to a brief intracellular depolarization of sensory neurons, which mimics their response to the CS, reproduced the prolonged and pairing-specific facilitation of the sensory-motor synapses (Hawkins et al., 1983; Walters & Byrne, 1983; Buonomano & Byrne, 1990). The US activates several serotoninergic neurons, including CB1, and nitrergic/peptidergic neurons, including L29. Experimental intracellular depolarization of these cells reproduces aspects of the learning-induced synaptic plasticity (Hawkins & Schacher, 1989; Mackey et al., 1989; Marinesco & Carew, (p. 540) 2002). Moreover, bath application of exogenous 5-HT or NO mimics the effects of the US in reduced preparations. Also, in isolated clusters of sensory neurons or cultures of sensory and motor neurons, a pairing of electrical activity elicited in the sensory neurons to mimic the CS reproduces the long-lasting and pairing-specific changes in bioelectrical properties of the sensory neurons and a facilitation of their synapses with postsynaptic motor neurons (Abrams, 1985; Ocorr et al., 1985; Eliot et al., 1994; Antonov et al. 2007). Thus, an activity-dependent modulation in the CS pathway that increases the excitability of the sensory neurons and their excitation of the motor neurons contributes to the associative process that establishes the conditioned response in a stimulus-specific manner.

Induction of the conditioned response also implicates a Hebbian-type plasticity that develops through coincident activation of presynaptic sensory and postsynaptic motor neurons in the CS pathway. Early data showing a role for postsynaptic motor neuronal activity were obtained in co-cultures of sensory and motor neurons. A long-term potentiation (LTP) in in vitro that re-formed sensory-motor synapses was induced by pairing electrical activity in the sensory neuron with a depolarization of the motor neuron. This plasticity, mediated by postsynaptic NMDA/glutamate receptors, was blocked by intracellular injection of a hyperpolarizing current or a calcium chelator into the postsynaptic motor neuron (Lin & Glanzman, 1994a, b; Bao et al., 1998). Such Hebbian LTP at the siphon sensory-motor synapse was also induced in a reduced neuronal preparation in which the CS was mimicked by a brief intracellular depolarization of a siphon sensory neuron, and the US by an electrical stimulation of the tail nerves (Murphy & Glanzman, 1997). Finally, in a preparation that permits simultaneous behavioral and electrophysiological investigations of classical conditioning of the siphon withdrawal reflex, intracellular injection of the calcium chelator into the postsynaptic LFS motor neurons drastically reduced the learning-induced enhancement of the monosynaptic sensory-motor connections in the CS pathway (Antonov et al., 2003). These results therefore indicate that the postsynaptic motor neuron serves as a second site of convergence for the US and CS, a contribution that may account for the response specificity of conditioning.

Interestingly, the non-Hebbian and Hebbian mechanisms that underlie the associative process in classical conditioning may not act independently of each other (for discussion, see Lechner & Byrne, 1998; Roberts & Glanzman, 2003). Data suggest that the Hebbian plasticity and calcium entry into the postsynaptic neurons may stimulate a retrograde transsynaptic messenger that in combination with 5-HT, SCP, and NO contributes to the non-Hebbian activity-dependent presynaptic modulation (Antonov et al., 2003).

Learning From an Operant Emission and Its Associated Outcome

Invertebrates do not only learn passively from the association between sensory stimuli but also from the association between active emissions of an operant and the reinforcing or punishing consequences. In operant conditioning, the animal itself is a crucial contributor to the learning process because it actively generates the operant that determines the occurrence of a positive or a negative behavioral outcome. Cellular analysis of both appetitive and aversive operant conditioning has been developed in Aplysia, and more extensively in feeding behavior (Susswein & Schwarz, 1983; Susswein et al., 1986; Cook & Carew, 1986, 1989a, 1989b; Schwarz et al., 1988; Brembs et al., 2002; Lyons et al., 2005; Hawkins et al., 2006; Nargeot et al., 2007). In this goal-directed behavior, the animal’s internal drives largely contribute to the active emission of behaviors such as locomotion, head waving, and buccal movement cycles. These latter movements are classified as ingestion or egestion and are easily quantifiable by the all-or-none behavioral emergence of a tongue-like organ, the radula (Kupfermann, 1974). For this reason, and because the neuronal network that generates buccal movements has been at least partly identified, Aplysia’s cyclic radula movements have served as an operant in several learning procedures.

In an aversive form of operant conditioning, animals learn to stop their attempts to ingest an appetitive, but inedible, food substance (Susswein et al., 1986; Lyons et al., 2005). This was shown by wrapping food in an inedible rough net with a hole that released food-related stimuli to arouse the animal, which eventually attempted to bite and swallow the netted food. However, the repeated inability to succeed provides a punishing influence that gradually decreases emission of biting movements until its eventual cessation. These behavioral changes, which are likely to occur in nature, are retained for several days after the conditioning procedure (Schwarz (p. 541) et al., 1991). Furthermore, they are specifically due to the action/punishment association that is characteristic of operant conditioning, since they were not induced by a procedure in which the netted food was withdrawn after emission of a bite, thereby dissociating the operant and the punishment.

Appetitive forms of operant conditioning also modify aspects of Aplysia’s food-seeking behavior (Susswein et al., 1986; Brembs et al., 2002; Nargeot et al., 2007). In one procedure, a food reward was delivered in association with each ingestion movement cycle that is actively although erratically expressed by an aroused animal. This action/reward contingency strongly increased the likelihood and regularity of emission of the rewarded action (Nargeot et al., 2007). This learning depended on the action/reward contingency that characterizes operant conditioning because it was not induced when the food reward was delivered periodically without temporal association with spontaneous operant emissions, or when no food reward was delivered. Moreover, the increase in frequency and regularity of action selectively involved the rewarded action, which then became prevalent over other none-rewarded food-seeking actions such as head waving and locomotion. Finally, the learning depended on the appetitive value of the reward itself because it no longer occurred when the food was devalued or consisted of nonpalatable elements.

Similar behavioral changes of Aplysia have been described from use of an experimental paradigm analogous to self-stimulation that underlies operant-reward learning in vertebrates (Brembs et al., 2002). In this procedure, electrical stimulation of a dopaminergic input nerve, which reproduced its activity recorded during actual food ingestion, was made contingent with emission of ingestion movement cycles. This association strongly increased the frequency of subsequent emissions of the rewarded action. This plasticity extinguished several hours to a day after training, and it was not induced with a yoked control paradigm, in which the stimulation was delivered in one animal depending on behavior emitted by another “yoked” animal, or with a control protocol in which no stimulation was delivered.

All these features—including changes in the likelihood and regularity of active operant emissions, sensitivity to action/outcome contingency, sensitivity to reinforcing/punishing stimuli, retention and extinction of the learned behavior—indicate that operant conditioning in Aplysia’s feeding behavior shares similar features with those described in vertebrates (Skinner, 1938).

A Dopaminergic Reinforcing Pathway

The reinforcing stimuli for the appetitive operant learning are conveyed to the central nervous system by the bilateral esophageal nerves (En.). These input nerves are rich in dopamine-containing fibers and make monosynaptic connections, both inhibitory and excitatory, with several key neurons of the central network that generate the motor patterns underlying each cycle of radula movements (Kabotyanski et al., 1998; Nargeot et al., 1999c; Martinez-Rubio et al., 2009; Bédécarrats et al., 2013). Cutting these nerves prevents learning (Schwarz & Susswein, 1986), whereas their electrical stimulation can reproduce the behavioral and neuronal plasticity induced by operant-reward learning in the behaving animal. Such features led to the development of in vitro analogs of appetitive operant conditioning in the isolated buccal ganglia (Nargeot et al. 1997, 1999a, 1999b; Bédécarrats et al. 2013). These ganglia continue spontaneously to generate cyclic radula motor patterns that are similar to those recorded in vivo during actual feeding. These patterns can thus be used as a “fictive” operant, while phasic electrical stimulation of the esophageal nerve can be employed to reproduce food reward-elicited activity. En. stimulation contingent with active emission of a buccal motor pattern in vitro reproduces essential features of operant conditioning in vivo. Specifically, it induces a contingent-dependent increase in both the frequency and regularity of emissions of the reinforced motor patterns. This motor plasticity is retained for several hours and is not induced by noncontingent stimulation paradigms. Moreover, induction of the plasticity is impaired by bath application of dopamine receptors antagonists. Thus, coordinate activity in the buccal ganglia and dopaminergic input fibers is essential to elaborate an internal representation of the action/reward association required in the operant learning. The neural pathways that convey punishment are less understood. Nevertheless, existing evidence indicates that this outcome leads to the release of NO and histamine in the cerebral ganglion, which in turn exerts a modulatory control over the buccal neural network operation (Katzoff et al., 2002, 2006, 2009).

Plasticity in a Decision-Making Network

The appetitive operant conditioning-induced plasticity of Aplysia’s feeding behavior has been further investigated in the neuronal network of buccal ganglia isolated from contingent, noncontingent, and control animals. These (p. 542) ganglia contain the central pattern generating network (CPG) that autonomously produces the radula motor patterns (Cropper et al., 2004; Nargeot & Simmers, 2012). This CPG contains several decision-making neurons that drive and select expression of the radula motor patterns on a cycle-by-cycle basis. Among these cells, a kernel of electrically coupled neurons, the B63/B65/B30 cells, “decides” through their irregular pacemaker properties, when to initiate each motor pattern. These cells are the first active in the CPG. In neuronal preparations from naïve animals, they generate repetitive bursts of action potentials that drive the radula motor patterns at a low frequency and irregular time intervals as seen with the radula movement cycles in these animals. Different CPG network cells contribute to the type of motor pattern selected. For example, the occurrence of a burst of action potentials in the B51 cell during motor pattern genesis triggers the characteristic pattern responsible for ingestion movement. Alternatively, a spontaneous absence or experimental suppression of this activity in B51 leads to generation of an egestion pattern (Nargeot et al., 1999b). Thus, the discharge (or not) of this cell represents a decision maker for the type of motor pattern and resulting radula movement expressed.

Operant learning modifies the dynamic properties of these decision-making neurons. Bursting activity in B63/B65/B30 is generated by irregular, endogenous oscillations of their membrane potential (Nargeot et al., 2009). Learning increases the regularity of these oscillatory properties, increases membrane excitability and input resistance, and strengthens the electrical synapses within this neuronal subset. Dynamic-clamp controlled manipulations of the excitability and electrical synapses of the decision neurons indicate that these parameters are critical for the expression of the learning-induced increase in frequency and regularity of the motor pattern genesis, respectively (Sieling et al., 2014). Moreover, in operantly trained animals with contingent dopaminergic nerve stimulation, training increased the likelihood of bursting activity occurrences in the ingestion pattern-selecting B51 neuron. This contingency-specific plasticity is associated with a decrease in the cell’s threshold for generating plateau potentials and an increase in its input resistance (Brembs et al., 2002).

Similar plasticity in the endogenous and dynamic properties of the decision neurons B63/B65/B30 and B51 can be induced by an analog of operant conditioning in isolated buccal ganglia (Nargeot et al., 1999a; Bédécarrats et al., 2013). Moreover, learning is also replicated in in situ or in cell culture preparations in which plateau potentials in B51 is elicited by injected intracellular current pulses in association with either stimulation of the dopamine input nerve or a puff ejection of dopamine. These procedures also decrease the plateau potential threshold and increase the input resistance of B51 (Nargeot et al., 1999b; Brembs et al., 2002; Lorenzetti et al., 2008).

Thus, these data indicated that the decision-making neurons in the buccal CPG of Aplysia are loci for the functional plasticity induced by operant learning. Specifically, this plasticity involves a modification in the dynamic membrane properties and electrical coupling of this neuronal subset, which in turn explains the main features of appetitive operant conditioning of feeding that include a contingency-dependent increase in the frequency and regularity of the rewarded ingestion pattern. Interestingly, feeding in Aplysia also provides a unique model for the comparison of plasticity induced by operant and classical conditioning in the same identified decision neuron (see section on “Plasticity in Synaptic and Intrinsic Membrane Properties Induced by Classical and Operant Conditioning in a Single Neuronal Circuit”).

From the aforementioned examples it can be concluded that classical and operant conditioning change very different behaviors as has been suggested in vertebrates (Skinner, 1938). Whereas classical conditioning modifies respondent behaviors that are elicited by specific stimuli, operant conditioning modifies the ongoing emissions of a spontaneous action that are organized by the inherent dynamic properties of central neuronal networks. Consequently, classical and operant conditioning might be reasonably expected to implicate different learning processes that depend on different neuronal organizations and/or different contributions of sensory processing and motor output genesis. To this end, comparison of learning in a variety of invertebrate models and their behaviors helps to pinpoint general principles of the neuronal organization and plasticity that contribute to classical and operant conditioning (Tables 23.1 and 23.2).

Classical Conditioning and Sensory Processing

Drosophila and Apis can be trained to discriminate odors (CS) depending on the latter’s (p. 543) association with appetitive or aversive US. Drosophila learns to avoid an odor that was previously paired with an electrical chock or an aversive odor (Quinn et al., 1974). In flies and bees, moreover, odor paired with sucrose contacting the tarsi and proboscis increases the animal’s motion toward that odor or elicits a proboscis-extension response (Takeda, 1961; Bitterman et al., 1983; Kim et al., 2007; Colomb et al., 2009). Depending on massed or spaced repetitions of the CS/US pairing, short-, intermediate-, and long-term memories can be induced. Correlates of these learning responses were found at several levels of the conditioned pathway that contributes to odor processing (for review, see Davis, 2005, 2011; Giurfa & Sandoz, 2012; Menzel, 2012; Guven-Ozkan & Davis, 2014). This pathway is basically composed of a three-order neuron tract to the mushroom bodies, a higher order center of sensory processing onto which converges information from different sensory modalities. Odors are initially detected by olfactory receptor neurons in the antenna. Their axons connect to local and projection neurons in the antenna lobes. The latter send their axons to the lateral protocerebral lobe and to the dentritic arbors of Kenyon cells in the calyces of the mushroom bodies. The axons of Kenyon cells in turn bifurcate to terminate in the mushroom body output lobes. The neuronal pathways and synaptic connections between neurons in the mushroom bodies and structures generating the motor responses are less well understood. They may partly involve pre- and motor neurons in the subesophageal ganglion. Appetitive unconditioned stimuli activate octopaminergic and/or dopaminergic modulatory neurons located in the subesophageal and mushroom bodies, respectively (Hammer, 1993; Liu et al., 2012; Qin et al. 2012). A different subset of dopaminergic neurons in the mushroom bodies contributes to aversive US. An identified octopamine neuron projects its axon to the main synaptic structures of the conditioned pathway, including the antennal lobe and the mushroom bodies. Dopaminergic neurons also project to a restricted and dedicated target area in the mushroom bodies.

Experimental approaches that block neuronal activity or synaptic transmission at specific levels of the CS pathway and optical imaging of calcium activity have revealed that learning involves the antennal lobes, their projection neurons, and Kenyon cells in the mushroom bodies, essential sites for the CS-US association (see “Cellular Mechanisms Underlying Coincidence Detection and Memory”; Hammer, 1997; McGuire et al., 2001; Yu et al., 2004; Carcaud et al., 2016). Thus, several sites that contribute to sensory processing are implicated in classical conditioning (Table 23.1).

In the terrestrial mollusc Limax, aversive olfactory learning develops when a natural food odor (CS) is paired with an aversive chemical or electrical stimulus (US) (Gelperin, 1975; Sahley et al., 1981; for review, see Watanabe et al., 2008). This simple classical conditioning and related higher order conditioning were monitored through the expression of behavioral choice or the latency in response to the CS. The CS pathway is composed of primary sensory cells in the tentacle epithelium that project to a tentacle ganglion. This ganglion contains local and projection neurons that send their axons to the procerebral lobe, a region specialized in olfactory processing (Ratté & Chase, 2000) where modulatory transmitters, including 5-HT and NO, mediate aversive and appetitive signals, respectively (Inoue et al. 2001; Yabumoto et al., 2008). Staining, imaging, and molecular data have suggested that learning and associated modulatory transmitters modify interneuronal activity in the procerebral lobe and local oscillatory field potentials that contribute to olfactory processing (Kimura et al., 1998; Nikitin & Balaban, 2000; Nakaya et al., 2001; Sekiguchi et al., 2010).

In the marine mollusc Hermissenda, phototaxic locomotion is reduced by pairing light (CS) with turbulence rotating or shaking of the animal (US) (Crow & Alkon, 1978; Crow & Jin, 2013). Before training, light elicits a forward movement of the animal, whereas a perturbation (US) produced by a rotating table to mimic turbulence of the animal’s natural environment elicits a foot contraction and shortening. After pairing light with the disturbance, an animal learns to contract its foot in response to light alone. Light activates five photoreceptors (two A types, three B types) in each eye. Through mono- and polysynaptic connections, these sensory cells excite and inhibit several groups of interneurons in the cerebropleural ganglia, which themselves elicit synaptic responses in pedal motor neurons. The US is conveyed by hair cells located in the statocyst organs. These sensory cells connect through polysynaptic tracts and different groups of interneurons to the pedal motor neurons, which mediate the unconditioned response (Crow & Tian, 2004). Interestingly, the hairs cells are connected to the photoreceptors via both a monosynaptic GABAergic connection and polysynaptic serotoninergic pathways (Alkon et al., 1993). (p. 544) (p. 545) Accordingly, the photoreceptors appear to be the primary site for convergence of the CS and US and are key loci of memory storage (Crow & Alkon, 1980). Thus, classical conditioning, or its analog in which a depolarization of the photoreceptors is paired with activation of the statocyst with an appropriate temporal relationship, causes a long-lasting increase and decrease in the intrinsic excitability of the type B and A photoreceptors, respectively (Farley et al., 1983). Learning also increases the inhibitory synaptic connection from the medial type B to type A photoreceptors (Frysztak & Crow, 1997). Furthermore, recent evidence has indicated that a second site of convergence of the CS/US is provided by identified interneurons in the unconditioned pathway that receives monosynaptic inputs from the sensory cells of the photoreceptors and hairs. Stimulus pairing-induced plasticity of intrinsic excitability in these neurons may also contribute to expression of the conditioned response (Crow & Tian, 2003; Crow & Jin, 2013). Thus, learning modifies sensory processing through changes in the pattern of light-elicited activity in the primary photoreceptors, and also in the interneuronal information processing that transforms the sensory activity into a motor response.

Classical Conditioning–Induced Plasticity in the Interneuronal Pathways Driving Motor Output

The marine mollusc Pleurobrancheae learns to avoid food stimuli after a training period in which this CS is associated with an electrical shock (US). This suppression of feeding is not expressed in control animals that receive the CS and US alternately rather than in temporal association (Mpitsos & Collins, 1975). Food stimuli applied to the oral veil elicits strong excitation of the PC and PSE cerebropleural to buccal interneurons (CBIs). Activity in these command neurons is critical for initiating feeding. It induces synaptic excitation of a half-center oscillator in the buccal ganglia, which is composed of the protraction and retraction neurons of the radula. Activity of the command neurons can be inhibited by two major classes of cerebropleural interneurons (I1, I2) through polysynaptic (I2) and monosynaptic (I1) connections (Jing & Gillette, 2000). Stimulus pairing-specific plasticity was found in the I2 neurons (London & Gillette, 1986). In semi-intact preparations from trained as compared to control animals, a food stimulus elicits a stronger and longer depolarization of the I2 neurons. This enhanced neuronal response causes a stronger inhibition of the command PC neurons, thereby leaving the radula in a retracted position. This pairing-specific plasticity in I2 results from a change in the neuron’s intrinsic excitability that in turn modifies I2’s response to CS-elicited presynaptic inputs.

Classical conditioning of feeding in the freshwater pond snail Lymnaea has clearly demonstrated that learning-induced plasticity can develop at multiple sites in a sensory-motor circuit with a critical contribution of bioelectrical changes in cerebrobuccal command interneurons (Benjamin et al., 2000). In an appetitive form of conditioning, animals learn from the association of a neutral tactile or chemical stimulus (CS) with a sucrose (US) application to the lips (Alexander et al., 1984; Kemenes & Benjamin, 1989). After repeated associations, and subsequently for several days, the CS alone elicits a pattern of feeding movements that is not expressed by control animals. The neuronal organization of the CS and US pathways has been described (Benjamin, 2012). Lip sensory neurons are connected to higher order command interneurons in the cerebral ganglia, including the serotoninergic CGC and CV1 neurons. These modulatory CBIs make synaptic connections within the buccal ganglia to the motor pattern generating network that is comprised of three interconnected cells (N1, N2, N3).

In naive animals, a touch to the lips (CS) fails to trigger any radula motor pattern emission. After training, the same stimulation elicits a bout of rhythmic and triphasic radula motor output. Both semi-intact preparations and an in vitro analog of conditioning were used to investigate the cellular basis of this learning. Pairing a tactile CS with the US produces an increase in the CS-elicited activity in the sensory inputs associated with an increase in their synaptic drives to the CBI neurons. Moreover, these high-order command neurons express a persistent depolarization that makes the cells more responsive to the CS. Finally, learning is correlated with an increase in the CPG-driven excitation of the B3 motor neuron (Staras et al., 1999; Jones et al., 2003). Similarly, pairing an appetitive chemical CS with the US is associated with a change in the resting membrane potential of the CGC and CV1 command neurons and a decrease in tonic firing of the inhibitory CPG neuron N3. All of these neuronal correlates might not be crucial for the expression of the conditioned response; nevertheless, experimental manipulations of the CV1’s resting membrane potential indicated that its plasticity alone is necessary and sufficient (Jones et al., 2003). (p. 546) Experimental changes in this resting membrane potential rendered the cell more or less responsive to the CS and were able to reproduce or block expression of the conditioned tactile response. Thus, these findings indicate that neuronal correlates of classical conditioning can be found at several levels in a distributed sensory-motor circuit and can involve both synaptic and passive membrane properties in the circuit’s constituent neurons (Table 23.1; Nikitin et al., 2008, 2013).

Operant Conditioning and Motor Systems

Fewer animal models have been developed to analyze operant than classical conditioning. Consequently, the available data may provide an incomplete picture of the neuronal plasticity induced by this form of associative learning. Nevertheless, converging findings in insects and molluscs can be seen to further confirm the idea that the inherent dynamics of neuronal circuits and neurons that actively generate operants play a fundamental role in the plasticity underlying operant learning (Table 23.2).

Whole and headless locusts, grasshoppers, and cockroaches can learn to maintain a leg position away from the preferred one after pairing the leg position with a positive (food reward) or negative (sound or electrical stimulus) reinforcement. A continuous negative reinforcement can reflexively elicit and maintain a moderate leg flexion. This reinforcement ceases when the leg is actively extended or more strongly flexed. In this operant conditioning procedure, therefore, an animal learns by its own action to escape from an aversive stimulus (Horridge, 1962; Hoyle, 1979, 1980; Harris, 1991). A similar training procedure was developed in a semi-intact preparation in which an identified motor neuron controlling leg position was recorded intracellularly. Computer-controlled electrical shocks to this leg were delivered in association to either a spontaneous increase (up-learning) or decrease (down-learning) in the motor neuron’s own discharge. Relative to prestimulus levels, the frequency of this spontaneous impulse activity increased or decreased after up- or down-learning, respectively. This plasticity was recorded in normal saline as well as in a solution blocking synaptic inputs and, thus, the plasticity appeared to involve the intrinsic “pacemaker” properties of the motor neuron itself, rather than the cell’s synaptic inputs (Woollacott & Hoyle, 1977; Hoyle, 1979).

Respiratory behavior in Lymnaea has provided a more extensive cellular analysis of aversive operant conditioning. In a hypoxic environment, this bimodal (cutaneous and lung) breathing mollusc performs aerial breathing by moving to the water surface and opening its respiratory apparatus, the pneumostome. During operant conditioning, each attempt at pneumostome opening was associated with delivery of a punishing tactile stimulation, which triggers pneumostome closure. Repeated associations between the operant and punishment reduced both the number of aerial breaths and the total breathing time. Yoked-control animals that received the tactile stimulus independently of their own behavior, but in temporal correlation with breathing in a trained snail, did not change their behavior. Furthermore, intermediate- and long-term memories developed, depending on the interval between training sessions (Lukowiak et al., 1996, 2003).

Rhythmic pneumostome opening and closing are generated by a CPG network composed of three main interneurons: a high-order command-like RPeD1 neuron and the IP3 and VD4 interneurons, which through their synapses promote activity in opener and closer motor neurons, respectively. The reciprocal inhibitory and/or excitatory connections between these three CPG neurons and their intrinsic firing properties, particularly postinhibitory rebound activation, are sufficient to generate the respiratory rhythm and pattern (Syed et al., 1990). In isolated nervous preparations from trained animals, and in in vitro semi-intact preparations, operant conditioning was found to reduce respiratory CPG activity by decreasing the spontaneous electrical activity of IP3. Additionally, learning reduced spontaneous activity in the higher order RPeD1 neurons and their ability to trigger patterned bursting activity in IP3 (Spencer et al., 1999; McComb et al., 2005). These changes were not observed in preparations from yoked control animals. Moreover, contingent-dependent plasticity in RPeD1 was associated with a reduction in the cell’s intrinsic excitability and input resistance, thereby indicating that this CPG neuron is one of the primary sites for the learning-induced plasticity (Braun & Lukowiak, 2011). This cell is also critically involved in long-term memory formation and storage, since the selective ablation of its soma prior to learning impaired long-term memory formation (Scheibenstock et al., 2002).

A particularly fruitful approach to determine if classical and operant conditioning derives from the (p. 547) same or distinct neuronal processes is to compare the plasticity induced by these distinct paradigms in a same neuronal network. Different animal models, Drosophila and Aplysia, have allowed such an approach in which the same behavior, locomotion and feeding respectively, was modified by classical and operant learning procedures (Brembs & Heisenberg, 2000; Brembs et al., 2004; Baxter & Byrne, 2006). Because of the accessibility to individual neurons for intracellular recording and the resultant identification of underlying neuronal circuitry, Aplysia in particular has provided the basis for a comparative analysis of the functional plasticity involved in associative learning.

Feeding behavior in this animal can be modified by appetitive operant conditioning (as detailed earlier) and by appetitive classical conditioning (Colwill et al., 1997; Lechner et al., 2000a). In the latter case, a tactile stimulus (CS) to the lips was paired with delivery of food (US). After repeated CS/US associations, the number of ingestion movement cycles of the radula triggered by the tactile stimulus alone is strongly increased. It is important to note, however, that no change in the spontaneous emissions of these movement cycles occurred after learning. The conditioned response was retained for at least 24 hours and was not induced by unpaired stimuli or by the US alone (Lechner et al., 2000a). Thus, the same motor act, the ingestion movements of the radula, and thereby the same central circuit as that modified by operant conditioning can also be altered by classical (p. 548) conditioning. In addition, the same reinforcing and dopaminergic pathway was implicated in both conditioning procedures. Food ingestion rather than a simple contact of food on the lips provided the reinforcement in classical conditioning. Thus, a pairing procedure in which food (US) was withdrawn before it was ingested prevented Pavlovian learning. In accordance, an experimental lesion of the dopaminergic esophageal nerves, which convey sensory information about successful food intake, but which did not hamper the ability of animal to actually ingest food, also impaired learning (Lechner et al., 2000b). By taking advantage of the identification of this reinforcing pathway, an in vitro analog of classical conditioning was developed in which a brief electrical stimulation of a mechanoafferent lip nerve (CS) was paired with an electrical stimulation of the dopaminergic input nerve (US). This paradigm in turn reproduced the pairing-specific and dopamine-dependent increase in occurrences of the CS-elicited ingestion motor pattern (Mozzachiodi et al., 2003; Reyes et al., 2005).

Decision-Making CPG Neurons Are Common Loci for Plasticity in Classical and Operant Conditioning

The lip mechanosensory neurons that convey the CS excite higher order cerebrobuccal interneurons (CBIs). These command neurons are connected by mono- and polysynaptic pathways to buccal CPG neurons that generate the ingestion and egestion motor patterns. Neuronal correlates of classical conditioning have been investigated in isolated cerebral-buccal ganglia preparations from trained and control animals. Pairing-specific changes induced by classical conditioning were monitored in two types of decision neurons in the buccal CPG (Lechner et al., 2000b; Mozzachiodi et al., 2003): first, in B31/32 neurons, a neuronal pair that through their powerful monosynaptic excitatory synapses with B63 neurons contribute to the decision-= process that can spontaneously initiate the radula motor patterns (Susswein et al., 2002; Hurwitz et al., 2008); and secondly, in the decision-making neuron B51, a key CPG element that is involved in motor pattern selection leading to expression of the ingestion pattern (Nargeot et al., 1999a, b). Classical conditioning learning enhances the complex excitatory synaptic drive that is elicited in both these neurons in response to a transient stimulation of the lip mechanosensory input nerve. This synaptic plasticity recorded in B31/32 is not associated with any change in the cell’s intrinsic membrane properties. In contrast, in B51 the synaptic plasticity was associated with a pairing-specific reduction in its intrinsic excitability. Although no change in resting membrane potential and input resistance was found in B51, the change in excitability resulted from an increase in threshold for eliciting plateau potentials (Lorenzetti et al., 2006). An identical plasticity in B31/32 and B51 was also found in in vitro neuronal analogs of classical conditioning. Two other sites of plasticity have been identified after classical conditioning and in its in vitro analog. One site concerns the B4/5 cells in the buccal CPG, whereas the other site is a high-order CBI neuron in the cerebral ganglia. These cellular loci fail to exhibit intrinsic changes following classical conditioning. Nevertheless, in vitro CS-US pairing was found to increase in the CS-evoked excitatory synaptic drive to CBI, which consequently may contribute to the increased synaptic drive to B31/32.

Thus, decision-making neurons in the same motor pattern-generating network can express changes evoked by either classical or operant conditioning. The target neurons implicated are key CPG elements for both the generation and designation of motor patterns, and hence any changes in their synaptic inputs and/or intrinsic excitability can drastically alter the network’s output. Therefore, either in response to a transient CS in classical conditioning or to the autonomous process of motor pattern genesis in operant conditioning, appetitive learning increases the genesis and expression of a specific motor act, namely that responsible for ingestion.

Differential Contribution of Dynamic Membrane Properties to Classical and Operant Learning

A particularly interesting aspect of a comparative study of classical and operant conditioning of Aplysia’s feeding behavior is that a unique decision neuron (B51) in the buccal CPG network is able to express a pairing-specific plasticity induced by either learning procedure. Intriguingly, this plasticity, which involves the cell’s intrinsic membrane properties, differs according to the conditioning procedure and even though both constitute appetitive forms of learning (Lorenzetti et al., 2006). Plateau potential generation in B51 produces an extended retraction and closure of the radula, thereby promoting the act of food ingestion (Nargeot et al., 1999a, b). Operant conditioning enhances this capability (Brembs et al., 2002), whereas classical conditioning decreases the cell’s capability to generate plateau potentials. Here, however, the increase (p. 549) in ingestion responses is associated with stronger CS-elicited synaptic input to B51 (Mozzachiodi et al. 2003). Thus, appetitive classical conditioning appears to favor synapse-driven responses of the neuron, whereas appetitive operant conditioning favors expression of an endogenous membrane property that contributes to the autonomous process of motor pattern genesis.

Classical and operant learning in Aplysia and in the animal models described earlier has been related to plasticity in synaptic and intrinsic membrane properties (Mozzachiodi & Byrne, 2010) that determine the respondent or intrinsic dynamic behavior of neurons. Chemical synaptic transmission and intrinsic excitability contribute to the electrical activity of neurons by imparting proportionality (linearity) in their response to triggering input information. In contrast, plateau potentials, pacemaker or oscillatory membrane properties, and electrical synapses contribute to a neuron’s spontaneous or dynamic (nonlinear) electrical behavior that is not, or only partially, determined by external information. In this context, it is noticeable from the studies described earlier that classical conditioning primarily modifies the respondent bioelectrical behavior of neurons, whereas operant conditioning is largely but not exclusively associated with changes in their inherent dynamic properties.

In associative learning, it is fundamental to understand how the temporal association that is essential for learning is encoded by central nervous systems. The earlier descriptions of pairing-specific plasticity in identified cells suggest that such an encoding might occur at the level of individual neurons (Abrams & Kandel, 1988). This hypothesis is further supported by the development of single-cell analogs of classical and operant conditioning that reproduces the essential features of a given learning-induced plasticity. These analogs have illustrated that two main elementary mechanisms, activity-dependent modulation and Hebbian-like plasticity, acting separately or in combination, contribute to the coincidence detection necessary for learning and memory (Tables 23.1 and 23.2; Bailey et al., 2000; Lechner & Byrne, 1998; Roberts & Glanzman, 2003).

Activity-Dependent Modulation in Classical Conditioning

A single-cell analog of classical conditioning of Aplysia’s gill-withdrawal reflex was developed in isolated sensory-motor neurons placed in co-culture. In this analog system, brief electrical excitations of a sensory neuron used to reproduce the CS was associated with puff applications of 5-HT analogous to the US (Eliot et al., 1994; Bao et al., 1998). This protocol reproduced the essential characteristics of the plasticity induced by learning in vivo, including the pairing-specific facilitation of the sensory-motor synapse. The facilitation was not associated with a long-lasting change in the frequency and amplitude of spontaneous miniature excitatory postsynaptic potentials, suggesting that it was related to a change in the sensory neuron’s spike-elicited release of transmitter. The sensory neuron spiking activity (CS) led to calcium entry and binding to the protein calmodulin in the presynaptic terminals. Serotonin application (US) to the same sensory neuron activated G protein–coupled receptor, cyclic adenosine monophosphate (cAMP) and protein kinase A (PKA) cascades. The temporal association between this electrical activity and serotonergic modulation in the single cell allows Ca²⁺/calmodulin binding to amplify the 5-HT-induced activity of a type I adenylyl cyclase (Abrams et al., 1991, 1998; Bao et al., 1998). This activity-dependent modulation, which implicates a dual activation of adenylyl cyclase, leads to a strong increase in intracellular cAMP levels compared to neurons subjected to a nonassociative procedure (Ocorr et al., 1985). Thereby, the synaptic plasticity induced by an associative learning paradigm appears to involve an amplification of heterosynaptic facilitation, a form of presynaptic plasticity that develops in nonassociative learning (Byrne & Kandel, 1996; Byrne & Hawkins, 2015; Yu & Rankin, 2017). In its short-term form, this facilitation leads to an increase in PKA and a reduction of K⁺ currents, with resultant spike broadening and an increased excitability of the sensory neuron. These bioelectrical changes thus favor action potential-evoked calcium influx and transmitter release. Mechanism for long-term memory in classical conditioning has not been investigated in Aplysia, yet. However, classical conditioning of feeding in Lymnaea recruits a PKA pathway that activates the transcription factor cAMP response element-binding protein (CREB) and contributes to different phases of long-term memory consolidation (Ribeiro et al., 2003; Kemenes et al., 2006; Michel et al., 2008).

In Drosophila and Apis, activity-dependent modulation by dopamine or octopamine as principal neuromodulators and the involvement of calcium-sensitive adenylyl cyclase as a molecular (p. 550) coincidence detector were also found to contribute to appetitive and aversive olfactory classical conditioning (Tsydzik & Wright, 2009; Tomchik & Davis, 2009; Gervasi et al., 2010; Guven-Ozkan & Davis, 2014; Matsumoto et al., 2014). Genetic analysis first identified two learning mutants, Rutabaga and Dunce, which led to abnormally weak memory after classical conditioning in which odorants were associated with electrical shock stimulation. These mutations specifically affected the expression of a type I adenylyl cyclase and a cAMP-specific phosphodiesterase in Rutabaga and Dunce, respectively (Dudai et al., 1976; Livingstone et al., 1984; Chen et al., 1986; Levin et al., 1992). In addition, genetic manipulations that overexpressed peptide inhibiting PKA activity also disrupted associative learning (Drain et al., 1991). These proteins are mainly expressed in neurons of the mushroom bodies, which play a critical role in olfactory learning (Nighorn et al., 1991; Han et al., 1992; Skoulakis et al., 1993). In an analog of classical conditioning, bath application of either dopamine or octopamine, the monoamines released in the mushroom bodies in response to aversive and/or appetitive stimuli, was used as a US. Application of acetylcholine, a transmitter released in the same neuronal structure in response to odors, was used as a CS (Gervasi et al., 2010). Dopamine or octopamine increased PKA activity in specific lobes or in all parts of the mushroom bodies, respectively, and required the contribution of adenylyl cyclase activity. Acetylcholine increased intracellular calcium concentration in all regions of the mushroom bodies, although this transmitter alone has no effect on PKA activity. However, conjoint application of acetylcholine (CS) with either dopamine or octopamine (US) led to a much stronger PKA activity than the monoaminergic modulation alone. This pairing-specific increase in PKA activity was not induced in Rutabaga flies, thus indicating that this associative effect is critically dependent on adenylyl cyclase activity. Memory traces, partly in relationship to their duration and type of learning, were found in various neuronal structures, including different lobes and neurons in the mushroom bodies. Furthermore, short-term memory tended to be correlated with the recruitment of new synapses, whereas long-term memory was found to be generally dependent on proteins synthesis and activity of CREB (Davis, 2011).

Classical conditioning of phototaxic behavior in Hermissenda revealed that activity-dependant modulation induced by CS/US pairing can also implicate protein kinase C (PKC) signaling pathways (Acosta-Urquidi et al., 1984). In in vitro preparations, light (CS), as well as photoreceptor depolarization, leads to intracellular calcium elevation and translocation of PKC from cytosol to the neuronal membrane (Muzzio et al., 1997). In contrast, hair cell activity (US) and the release of GABA acting via GABA_B receptors activate phospholipase A2 (PLA2) in the B photoreceptors. A persistent activation of PKC requires the conjoint involvement of a calcium/diacylglycerol elevation induced by the CS, and activation of PLA2 and production of arachidonic acids in response to the US (Matzel et al., 1998; Blackwell, 2006). This persistent activation of PKC phosphorylates different proteins including potassium and calcium channels that in turn increase B receptor excitability (Alkon et al., 1982; Alkon, 1984; Farley & Auerbach, 1986). Long-term memory depends on protein synthesis (Crow & Forrester, 1990).

Activity-Dependent Modulation in Operant Conditioning

A cellular analog of appetitive operant conditioning of feeding behavior in Aplysia indicated that adenylyl cyclase can also serve as a molecular coincident detector in this form of associative learning (Lorenzetti et al., 2008). In this analog, plateau potentials can be experimentally triggered in the B51 decision neurons placed in cell culture, and serve as an analog of the operant, whereas a puff of dopamine acts as an analog of positive reinforcement. Pairing expression of plateau potential with a dopamine pulse reproduces the changes in the intrinsic membrane properties of B51 that are recorded after in vivo operant conditioning. Dopamine activates a D1-like receptor, which recruits cAMP/PKA signaling pathways. Dopamine alone did not raise cAMP to a level sufficient to induce the cellular plasticity. However, when electrical activity in B51 was repetitively paired with a puff of dopamine to reproduce the operant training procedure, activity in B51 produced a calcium influx and a subsequent activation of calcium-dependent PKC that was necessary to modify the intrinsic membrane properties of B51. A convergent site for the effect of dopamine and electrical activity-induced activation of calcium-sensitive PKC was localized upstream to cAMP production, probably at the level of the adenylyl cyclase complex. These data therefore suggest that mechanisms of coincidence detection in operant conditioning may be very similar to (p. 551) those described in classical conditioning, although the target molecules of the signaling cascade in the appetitive form of operant conditioning remain to be investigated. In the aversive form, PKA/PKC contributes differently to short- and long-term memory (Michel et al., 2010). Moreover, this latter requires protein synthesis and an activation of polyADP-ribose-polymerase proteins (Cohen-Armon et al., 2004; Michel et al., 2012).

PKA and PKC similarly contribute to operant learning in the respiratory behavior of Lymnaea (Rosenegger & Lukowiak, 2010; Takigami et al., 2014). In Drosophila, PKA and PKC signaling contributes differently to classical and operant conditioning (Brembs & Plendl, 2008). Thus, Rutabaga mutants that are deficient in type I adenylyl cyclase and are poor learners in classical conditioning express an unaltered operant conditioning of turning behavior. However, this latter form of learning is impaired in mutants expressing an inhibitor protein for PKC. Interestingly, evidence suggests that the action of PKC is required in motor neurons (Colomb & Brembs, 2016).

Hebbian Synaptic Plasticity

In addition to activity-dependent modulation, there is evidence to indicate that classical conditioning of gill- and siphon-withdrawal reflexes in Aplysia involves a Hebbian-type plasticity (see earlier; Lin & Glanzman, 1994a, b; Roberts & Glanzman, 2003; Antonov et al., 2003). This plasticity, which potentiates sensory-motor synaptic transmission in a long-lasting manner, develops from the coincident activation of pre- and postsynaptic neurons. It depends on specific properties of the NMDA subtype of glutamate receptors, the permeability of which requires both glutamate binding and a concomitant membrane depolarization. Thus, NMDA receptors can serve as a second molecular coincidence detector in which presynaptic glutamate release is elicited by the CS, and postsynaptic neuron depolarization is produced by the US. This dual activation of NMDA receptors leads to calcium influx that through an activation of kinases, and long-lasting modifications of AMPA receptor-mediated responses and trafficking, enhances synaptic transmission (Murphy & Glanzman, 1996). Interestingly, a contribution of NMDA receptors in classical conditioning was also described in Drosophila (Xia et al., 2005; Miyashita et al., 2012; Ueno et al., 2017), as well as for intermediate and long-term memory formation following respiratory operant conditioning in Lymnaea (Rosenegger & Lukowiak, 2010).