Show Summary Details

Page of

PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). © Oxford University Press, 2018. All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in Oxford Handbooks Online for personal use (for details see Privacy Policy and Legal Notice).

date: 21 May 2018

Associative Learning in Invertebrates

Abstract and Keywords

Behaviors of invertebrates can be modified by associative learning in a similar manner to those of vertebrates. Two simple forms of associative learning, Pavlovian and operant conditioning, allow animals to establish a predictive relationship between two events. Here we summarize five decades of studies of behavioral, cellular, and subcellular changes that are induced by these two learning paradigms in different invertebrate animal models. A comparative description of circuitry, neuronal elements, and properties that contribute to these conditioning procedures will be drawn to decipher common and distinguishing features of the learning processes. We will illustrate that similar circuits, synaptic and neuronal membrane plasticity, and similar molecular sites of detection of association are implicated in both forms of conditioning. However, evidence will also suggest that passively responding and endogenous dynamic properties of central networks and/or their constituent neurons might differentially contribute to Pavlovian and operant learning.

Keywords: Pavlovian conditioning, operant conditioning, learning and memory, networks, central pattern generator, intrinsic membrane properties, plasticity, serotonin, dopamine, NMDA receptors

Associative learning allows living animals to establish a predictive relationship between event occurrences that include sensory stimuli and/or motor acts. Based on the associated events, two simple procedures of associative learning can be distinguished: Pavlovian, also called classical or respondent conditioning, and operant or instrumental conditioning. In Pavlovian conditioning, a conditioned stimulus (CS) is delivered in correlation with an appetitive or aversive stimulus, a so-called unconditioned stimulus (US). In Pavlov’s original experiment with dogs, after a repeated association of the CS (a bell tone) with the US (a meat powder), the CS delivered alone elicits a conditioned response (CR) that resembles the unconditioned response (UR, i.e., salivation), which initially is elicited by the US alone (Pavlov, 1927). This process suggested that an animal learns that the CS predicts the US and consequently anticipates the behavioral response to a CS presentation alone. In operant conditioning, under a specific motivational state, the emission of a behavior—the so-called operant—is associated after a short delay with resulting reinforcement or punishment that, respectively, increases or decreases the frequency of the ongoing behavior. In this form of conditioning, by actively controlling the emission of the operant, the animal anticipates the consequences of the particular behavior. This conditioning process is well illustrated by delivery of a positive reinforcement (or reward) in a Skinner box (Skinner, 1938). In this original experiment with pigeons, the act of picking on a lever (operant) is associated with the delivery of seeds (reward). For a hungry bird, this association increases the likelihood and regularity of further emissions of the reinforced action.

There is a long-standing historical debate as to whether these different conditionings implicate distinct or common learning processes (Rescorla & Solomon, 1967; Rescorla, 1987). Despite the operational distinction based on stimulus/stimulus (in Pavlovian conditioning) or action/stimulus contingency (in operant conditioning), the same basic elements are required in learning. These include (1) an antecedent stimulus that initially has no or only a weak effect on the expression of the behavior (i.e., the CS or a specific motivational state such as hunger); (2) a distinct behavior (i.e., the CR or the operant); and (3) an appetitive or aversive stimulus (i.e., the US or the reinforcement/punishment). Moreover, both learning procedures are similarly sensitive to variables such as the amount, delay, and contingency of reinforcement. On this basis, therefore, learning may implicate a single underlying process that could be viewed as an appetitive/aversive stimulus-induced change in the capability of an antecedent input to elicit a specific behavior. However, Pavlovian and operant conditioning also have very different characteristics. For example, conditioned responses in Pavlovian learning are respondent behaviors. Their expression is strictly dependent upon triggering by simple or complex preceding stimuli. In operant conditioning, the operant is not strictly a stimulus-triggered response in the sense that a CS elicits the CR. Rather, this behavior is actively generated by animals through central processes of decision making, which to a large extent are independent of explicit sensory cues.

Analysis of the neuronal basis of simple forms of Pavlovian and operant conditioning has provided new insights to this debate, in which invertebrate animal models have been found to be particularly useful (Byrne, 1987; Hawkins & Byrne, 2015). These animals express relatively simple behaviors that can be modified by several forms of associative learning procedures, similar to those employed by vertebrates. Moreover, the neuronal networks that produce these behaviors are composed of small numbers of neurons that are individually identifiable and amenable to cellular analysis. In this chapter, we will review classical and operant conditioning processes in several invertebrate animal models with an aim to identifying common and distinguishing neuronal mechanisms that underlie learning.

Classical Conditioning of the Defensive-Withdrawal Reflex in Aplysia

Plasticity of a Respondent Behavior

The marine mollusk Aplysia has provided several well-known examples of aversive and appetitive classical conditioning that have been studied at the cellular level (Walters et al., 1979, 1981; Carew et al., 1981; Colwill et al., 1988, 1997; Lechner et al., 2000a). In a typical aversive procedure, the animal is trained to produce a strong protective reflex of its respiratory apparatus—a gill and an exhalent cutaneous fold (the siphon)—in response to the application of a light, tactile stimulus. In naive unconditioned animals, a brief and light touch to the siphon elicits a weak gill and siphon contraction. During a training period, this CS is associated with an US that consists of an aversive electrical shock delivered to the animal’s tail. This latter stimulus alone is sufficient to elicit a defensive reaction that includes a powerful gill and siphon contraction (Carew et al., 1981). Following repetition of such a CS-US pairing, the CS alone becomes capable of eliciting a strong gill and siphon withdrawal. This conditioned response is expressed for several days after its induction, and it is extinguished under repeated CS alone. It was only weakly expressed after a training protocol in which the CS and US are presented randomly or alone. Moreover, conditioning fails when the onset of the CS precedes that of the US with a time interval longer than 1 second, or with a backward conditioning in which the US precedes the CS (Hawkins et al., 1986). Finally, a decrease in contingency of the stimuli by adding supplemental US not associated with the CS in the training procedure no longer induces the conditioned response. The requirement of a strong temporal correlation between the stimuli in learning and retention/extinction of the conditioned response thus indicated that conditioning of the gill and siphon withdrawal reflex in Aplysia shares essential features of Pavlovian conditioning as found in vertebrates (Rescorla, 1967).

Additional features of conditioning in the gill and siphon withdrawal reflex provide information about neuronal changes involved in learning. First, the reflex can be modified by differential conditioning in which two different CS are delivered in alternation to different sites on the animal skin (Carew et al., 1983). One of them is closely paired with the US after a short delay. The other stimulus is expressly unpaired, terminating long before the US onset. Only the CS explicitly paired with the US subsequently elicited a conditioned response. This stimulus specificity, in which two separate CS-related sensory inputs that are known to activate a common group of interneurons and motor neurons, indicated that learning-induced plasticity may be partly determined by the sensory neurons themselves. Furthermore, learning was found to transform a preexisting response to a CS into a conditioned response that resembles the UR (Hawkins et al., 1989; Walters, 1989). The CS can initially elicit a normal forward contraction of the siphon. After its pairing with an electrical shock, this CS alone comes to evoke a backward contraction that resembles the US response. In contrast, after pairing this CS with an electrical shock to the mantle covering the gill, which alone triggers a forward contraction, the CS alone becomes able to elicit such a forward contraction. Thus, since two distinct classes of motor neurons produce backward and forward bending of the siphon (Frost et al., 1988), this response specificity suggests that learning may also modify specific motor neurons and/or presynaptic interneuronal pathways.

Neuronal Organization of the Sensory-Motor Pathways

At least 100 neurons, including sensory, interneurons, and motor neurons, contribute to the CS-elicited siphon withdrawal behavior of Aplysia (Cleary et al., 1995; Frost & Kandel, 1995). In the absence of sensory stimulation, most of these circuit neurons express no relevant or patterned electrical activity, but only produce coordinated activity in response to the siphon stimulation. Clusters of mechanosensory neurons with receptive fields over the siphon make conventional monosynaptic excitatory connections with clusters of siphon motor neurons in the abdominal ganglion. Among these sensory neurons, the LE neurons contribute up to 60% of the monosynaptic excitation of the LFS motor neurons and determine 30% of the siphon contraction amplitude (Antonov et al., 1999). The sensory neurons also activate the motor neurons via polysynaptic interneuronal pathways that include excitatory, inhibitory, and modulatory interneurons. These interneuronal pathways increase the contribution made by the monosynaptic circuitry to siphon contraction. They also distribute information to other neuronal circuits outside of the reflex pathway, such as in the network responsible for spontaneous respiratory pumping, in order to coordinate the motor responses (Hawkins et al., 1981).

The unconditioned tail and siphon contraction is mediated by sensory, interneurons, and motor neurons that are essentially distinct from the CS-elicited pathway (Cleary et al., 1995). Nevertheless, the US-activated tail sensory neurons also excite polysynaptic interneuronal pathways that belong to or converge onto the sensory, interneurons, and motor neurons of the CS-elicited reflex pathway. Interestingly, this interneuronal pathway activated by the US contains modulatory interneurons, such as the serotoninergic CB1 neurons and the putative peptidergic/nitrergic L29 neurons, which interact and modify the functional properties of the neurons in the CS pathway (Mackey et al., 1989; Antonov et al., 2007).

Neuronal Plasticity Underlying Conditioned Respondent Behavior

Learning-induced neuronal plasticity has been investigated by making intracellular recordings in reduced preparations that consist of neuronal ganglia connected via their peripheral nerves to the tail and siphon (Antonov et al., 2001). In preparations from naïve animals, a CS elicits a transient burst of action potentials in the sensory-motor CS pathway that leads to a weak, short-lasting siphon contraction (Hawkins & Schacher, 1989; Frost et al., 1997; Antonov et al., 2001). Repeated pairing of this CS with the US that reproduces classical conditioning in vivo gradually increases the firing properties in the CS pathway. For several tens of minutes after this in vitro training procedure, the CS elicits a stronger burst of action potentials in the sensory and motor neurons and a powerful contraction of the siphon, like the conditioned response in vivo. This long-term and pairing-specific change in neuronal firing is associated with an increase in excitability of the sensory LE neurons, a facilitation of their monosynaptic connection with the motor neurons, and changes in the interneuronal inputs to the motor neurons (Antonov et al., 2001).

The neuronal plasticity in the CS pathway is thought to be induced by modulatory transmitters, such as serotonin (5-HT), nitric oxide (NO), small cardioactive peptide (SCP), and by conjoint electrical activity in the CS and US pathways (Antonov et al., 2003, 2007; Roberts & Glanzman, 2003). In reduced preparations consisting of the isolated neural ganglia attached to the tail, an association of the tail shock (US) to a brief intracellular depolarization of sensory neurons, which mimics their response to the CS, reproduced the prolonged and pairing-specific facilitation of the sensory-motor synapses (Hawkins et al., 1983; Walters & Byrne, 1983; Buonomano & Byrne, 1990). The US activates several serotoninergic neurons, including CB1, and nitrergic/peptidergic neurons, including L29. Experimental intracellular depolarization of these cells reproduces aspects of the learning-induced synaptic plasticity (Hawkins & Schacher, 1989; Mackey et al., 1989; Marinesco & Carew, 2002). Moreover, bath application of exogenous 5-HT or NO mimics the effects of the US in reduced preparations. Also, in isolated clusters of sensory neurons or cultures of sensory and motor neurons, a pairing of electrical activity elicited in the sensory neurons to mimic the CS reproduces the long-lasting and pairing-specific changes in bioelectrical properties of the sensory neurons and a facilitation of their synapses with postsynaptic motor neurons (Abrams, 1985; Ocorr et al., 1985; Eliot et al., 1994; Antonov et al. 2007). Thus, an activity-dependent modulation in the CS pathway that increases the excitability of the sensory neurons and their excitation of the motor neurons contributes to the associative process that establishes the conditioned response in a stimulus-specific manner.

Induction of the conditioned response also implicates a Hebbian-type plasticity that develops through coincident activation of presynaptic sensory and postsynaptic motor neurons in the CS pathway. Early data showing a role for postsynaptic motor neuronal activity were obtained in co-cultures of sensory and motor neurons. A long-term potentiation (LTP) in in vitro that re-formed sensory-motor synapses was induced by pairing electrical activity in the sensory neuron with a depolarization of the motor neuron. This plasticity, mediated by postsynaptic NMDA/glutamate receptors, was blocked by intracellular injection of a hyperpolarizing current or a calcium chelator into the postsynaptic motor neuron (Lin & Glanzman, 1994a, b; Bao et al., 1998). Such Hebbian LTP at the siphon sensory-motor synapse was also induced in a reduced neuronal preparation in which the CS was mimicked by a brief intracellular depolarization of a siphon sensory neuron, and the US by an electrical stimulation of the tail nerves (Murphy & Glanzman, 1997). Finally, in a preparation that permits simultaneous behavioral and electrophysiological investigations of classical conditioning of the siphon withdrawal reflex, intracellular injection of the calcium chelator into the postsynaptic LFS motor neurons drastically reduced the learning-induced enhancement of the monosynaptic sensory-motor connections in the CS pathway (Antonov et al., 2003). These results therefore indicate that the postsynaptic motor neuron serves as a second site of convergence for the US and CS, a contribution that may account for the response specificity of conditioning.

Interestingly, the non-Hebbian and Hebbian mechanisms that underlie the associative process in classical conditioning may not act independently of each other (for discussion, see Lechner & Byrne, 1998; Roberts & Glanzman, 2003). Data suggest that the Hebbian plasticity and calcium entry into the postsynaptic neurons may stimulate a retrograde transsynaptic messenger that in combination with 5-HT, SCP, and NO contributes to the non-Hebbian activity-dependent presynaptic modulation (Antonov et al., 2003).

Operant Conditioning of Food-Seeking Behavior in Aplysia

Learning From an Operant Emission and Its Associated Outcome

Invertebrates do not only learn passively from the association between sensory stimuli but also from the association between active emissions of an operant and the reinforcing or punishing consequences. In operant conditioning, the animal itself is a crucial contributor to the learning process because it actively generates the operant that determines the occurrence of a positive or a negative behavioral outcome. Cellular analysis of both appetitive and aversive operant conditioning has been developed in Aplysia, and more extensively in feeding behavior (Susswein & Schwarz, 1983; Susswein et al., 1986; Cook & Carew, 1986, 1989a, 1989b; Schwarz et al., 1988; Brembs et al., 2002; Lyons et al., 2005; Hawkins et al., 2006; Nargeot et al., 2007). In this goal-directed behavior, the animal’s internal drives largely contribute to the active emission of behaviors such as locomotion, head waving, and buccal movement cycles. These latter movements are classified as ingestion or egestion and are easily quantifiable by the all-or-none behavioral emergence of a tongue-like organ, the radula (Kupfermann, 1974). For this reason, and because the neuronal network that generates buccal movements has been at least partly identified, Aplysia’s cyclic radula movements have served as an operant in several learning procedures.

In an aversive form of operant conditioning, animals learn to stop their attempts to ingest an appetitive, but inedible, food substance (Susswein et al., 1986; Lyons et al., 2005). This was shown by wrapping food in an inedible rough net with a hole that released food-related stimuli to arouse the animal, which eventually attempted to bite and swallow the netted food. However, the repeated inability to succeed provides a punishing influence that gradually decreases emission of biting movements until its eventual cessation. These behavioral changes, which are likely to occur in nature, are retained for several days after the conditioning procedure (Schwarz et al., 1991). Furthermore, they are specifically due to the action/punishment association that is characteristic of operant conditioning, since they were not induced by a procedure in which the netted food was withdrawn after emission of a bite, thereby dissociating the operant and the punishment.

Appetitive forms of operant conditioning also modify aspects of Aplysia’s food-seeking behavior (Susswein et al., 1986; Brembs et al., 2002; Nargeot et al., 2007). In one procedure, a food reward was delivered in association with each ingestion movement cycle that is actively although erratically expressed by an aroused animal. This action/reward contingency strongly increased the likelihood and regularity of emission of the rewarded action (Nargeot et al., 2007). This learning depended on the action/reward contingency that characterizes operant conditioning because it was not induced when the food reward was delivered periodically without temporal association with spontaneous operant emissions, or when no food reward was delivered. Moreover, the increase in frequency and regularity of action selectively involved the rewarded action, which then became prevalent over other none-rewarded food-seeking actions such as head waving and locomotion. Finally, the learning depended on the appetitive value of the reward itself because it no longer occurred when the food was devalued or consisted of nonpalatable elements.

Similar behavioral changes of Aplysia have been described from use of an experimental paradigm analogous to self-stimulation that underlies operant-reward learning in vertebrates (Brembs et al., 2002). In this procedure, electrical stimulation of a dopaminergic input nerve, which reproduced its activity recorded during actual food ingestion, was made contingent with emission of ingestion movement cycles. This association strongly increased the frequency of subsequent emissions of the rewarded action. This plasticity extinguished several hours to a day after training, and it was not induced with a yoked control paradigm, in which the stimulation was delivered in one animal depending on behavior emitted by another “yoked” animal, or with a control protocol in which no stimulation was delivered.

All these features—including changes in the likelihood and regularity of active operant emissions, sensitivity to action/outcome contingency, sensitivity to reinforcing/punishing stimuli, retention and extinction of the learned behavior—indicate that operant conditioning in Aplysia’s feeding behavior shares similar features with those described in vertebrates (Skinner, 1938).

A Dopaminergic Reinforcing Pathway

The reinforcing stimuli for the appetitive operant learning are conveyed to the central nervous system by the bilateral esophageal nerves (En.). These input nerves are rich in dopamine-containing fibers and make monosynaptic connections, both inhibitory and excitatory, with several key neurons of the central network that generate the motor patterns underlying each cycle of radula movements (Kabotyanski et al., 1998; Nargeot et al., 1999c; Martinez-Rubio et al., 2009; Bédécarrats et al., 2013). Cutting these nerves prevents learning (Schwarz & Susswein, 1986), whereas their electrical stimulation can reproduce the behavioral and neuronal plasticity induced by operant-reward learning in the behaving animal. Such features led to the development of in vitro analogs of appetitive operant conditioning in the isolated buccal ganglia (Nargeot et al. 1997, 1999a, 1999b; Bédécarrats et al. 2013). These ganglia continue spontaneously to generate cyclic radula motor patterns that are similar to those recorded in vivo during actual feeding. These patterns can thus be used as a “fictive” operant, while phasic electrical stimulation of the esophageal nerve can be employed to reproduce food reward-elicited activity. En. stimulation contingent with active emission of a buccal motor pattern in vitro reproduces essential features of operant conditioning in vivo. Specifically, it induces a contingent-dependent increase in both the frequency and regularity of emissions of the reinforced motor patterns. This motor plasticity is retained for several hours and is not induced by noncontingent stimulation paradigms. Moreover, induction of the plasticity is impaired by bath application of dopamine receptors antagonists. Thus, coordinate activity in the buccal ganglia and dopaminergic input fibers is essential to elaborate an internal representation of the action/reward association required in the operant learning. The neural pathways that convey punishment are less understood. Nevertheless, existing evidence indicates that this outcome leads to the release of NO and histamine in the cerebral ganglion, which in turn exerts a modulatory control over the buccal neural network operation (Katzoff et al., 2002, 2006, 2009).

Plasticity in a Decision-Making Network

The appetitive operant conditioning-induced plasticity of Aplysia’s feeding behavior has been further investigated in the neuronal network of buccal ganglia isolated from contingent, noncontingent, and control animals. These ganglia contain the central pattern generating network (CPG) that autonomously produces the radula motor patterns (Cropper et al., 2004; Nargeot & Simmers, 2012). This CPG contains several decision-making neurons that drive and select expression of the radula motor patterns on a cycle-by-cycle basis. Among these cells, a kernel of electrically coupled neurons, the B63/B65/B30 cells, “decides” through their irregular pacemaker properties, when to initiate each motor pattern. These cells are the first active in the CPG. In neuronal preparations from naïve animals, they generate repetitive bursts of action potentials that drive the radula motor patterns at a low frequency and irregular time intervals as seen with the radula movement cycles in these animals. Different CPG network cells contribute to the type of motor pattern selected. For example, the occurrence of a burst of action potentials in the B51 cell during motor pattern genesis triggers the characteristic pattern responsible for ingestion movement. Alternatively, a spontaneous absence or experimental suppression of this activity in B51 leads to generation of an egestion pattern (Nargeot et al., 1999b). Thus, the discharge (or not) of this cell represents a decision maker for the type of motor pattern and resulting radula movement expressed.

Operant learning modifies the dynamic properties of these decision-making neurons. Bursting activity in B63/B65/B30 is generated by irregular, endogenous oscillations of their membrane potential (Nargeot et al., 2009). Learning increases the regularity of these oscillatory properties, increases membrane excitability and input resistance, and strengthens the electrical synapses within this neuronal subset. Dynamic-clamp controlled manipulations of the excitability and electrical synapses of the decision neurons indicate that these parameters are critical for the expression of the learning-induced increase in frequency and regularity of the motor pattern genesis, respectively (Sieling et al., 2014). Moreover, in operantly trained animals with contingent dopaminergic nerve stimulation, training increased the likelihood of bursting activity occurrences in the ingestion pattern-selecting B51 neuron. This contingency-specific plasticity is associated with a decrease in the cell’s threshold for generating plateau potentials and an increase in its input resistance (Brembs et al., 2002).

Similar plasticity in the endogenous and dynamic properties of the decision neurons B63/B65/B30 and B51 can be induced by an analog of operant conditioning in isolated buccal ganglia (Nargeot et al., 1999a; Bédécarrats et al., 2013). Moreover, learning is also replicated in in situ or in cell culture preparations in which plateau potentials in B51 is elicited by injected intracellular current pulses in association with either stimulation of the dopamine input nerve or a puff ejection of dopamine. These procedures also decrease the plateau potential threshold and increase the input resistance of B51 (Nargeot et al., 1999b; Brembs et al., 2002; Lorenzetti et al., 2008).

Thus, these data indicated that the decision-making neurons in the buccal CPG of Aplysia are loci for the functional plasticity induced by operant learning. Specifically, this plasticity involves a modification in the dynamic membrane properties and electrical coupling of this neuronal subset, which in turn explains the main features of appetitive operant conditioning of feeding that include a contingency-dependent increase in the frequency and regularity of the rewarded ingestion pattern. Interestingly, feeding in Aplysia also provides a unique model for the comparison of plasticity induced by operant and classical conditioning in the same identified decision neuron (see section on “Plasticity in Synaptic and Intrinsic Membrane Properties Induced by Classical and Operant Conditioning in a Single Neuronal Circuit”).

Neuronal Organization and Learning

From the aforementioned examples it can be concluded that classical and operant conditioning change very different behaviors as has been suggested in vertebrates (Skinner, 1938). Whereas classical conditioning modifies respondent behaviors that are elicited by specific stimuli, operant conditioning modifies the ongoing emissions of a spontaneous action that are organized by the inherent dynamic properties of central neuronal networks. Consequently, classical and operant conditioning might be reasonably expected to implicate different learning processes that depend on different neuronal organizations and/or different contributions of sensory processing and motor output genesis. To this end, comparison of learning in a variety of invertebrate models and their behaviors helps to pinpoint general principles of the neuronal organization and plasticity that contribute to classical and operant conditioning (Tables 1 and 2).

Classical Conditioning and Sensory Processing

Drosophila and Apis can be trained to discriminate odors (CS) depending on the latter’s association with appetitive or aversive US. Drosophila learns to avoid an odor that was previously paired with an electrical chock or an aversive odor (Quinn et al., 1974). In flies and bees, moreover, odor paired with sucrose contacting the tarsi and proboscis increases the animal’s motion toward that odor or elicits a proboscis-extension response (Takeda, 1961; Bitterman et al., 1983; Kim et al., 2007; Colomb et al., 2009). Depending on massed or spaced repetitions of the CS/US pairing, short-, intermediate-, and long-term memories can be induced. Correlates of these learning responses were found at several levels of the conditioned pathway that contributes to odor processing (for review, see Davis, 2005, 2011; Giurfa & Sandoz, 2012; Menzel, 2012; Guven-Ozkan & Davis, 2014). This pathway is basically composed of a three-order neuron tract to the mushroom bodies, a higher order center of sensory processing onto which converges information from different sensory modalities. Odors are initially detected by olfactory receptor neurons in the antenna. Their axons connect to local and projection neurons in the antenna lobes. The latter send their axons to the lateral protocerebral lobe and to the dentritic arbors of Kenyon cells in the calyces of the mushroom bodies. The axons of Kenyon cells in turn bifurcate to terminate in the mushroom body output lobes. The neuronal pathways and synaptic connections between neurons in the mushroom bodies and structures generating the motor responses are less well understood. They may partly involve pre- and motor neurons in the subesophageal ganglion. Appetitive unconditioned stimuli activate octopaminergic and/or dopaminergic modulatory neurons located in the subesophageal and mushroom bodies, respectively (Hammer, 1993; Liu et al., 2012; Qin et al. 2012). A different subset of dopaminergic neurons in the mushroom bodies contributes to aversive US. An identified octopamine neuron projects its axon to the main synaptic structures of the conditioned pathway, including the antennal lobe and the mushroom bodies. Dopaminergic neurons also project to a restricted and dedicated target area in the mushroom bodies.

Experimental approaches that block neuronal activity or synaptic transmission at specific levels of the CS pathway and optical imaging of calcium activity have revealed that learning involves the antennal lobes, their projection neurons, and Kenyon cells in the mushroom bodies, essential sites for the CS-US association (see “Cellular Mechanisms Underlying Coincidence Detection and Memory”; Hammer, 1997; McGuire et al., 2001; Yu et al., 2004; Carcaud et al., 2016). Thus, several sites that contribute to sensory processing are implicated in classical conditioning (Table 1).

Table 1 Summary of Classical Conditioning

Animal

Learning Paradigm

Behavior

Site(s) of Plasticity

Modulatory Transmitter

Neuronal Plasticity

Subcellular Processes

Apis

Appetitive

Proboscis extension reflex

Sensory processing in antenna lobes and mushroom bodies

Octopamine

Neuronal recruitment; change in neuronal activity patterns

NMDA receptors; Ca2+

Aplysia

Aversive

Gill and siphon withdrawal reflexes

Sensory, inter-and motor neurons in abdominal ganglion

5-HT, SCP, NO, glutamate

Presynaptic facilitation; intrinsic excitability increase

Type I adenylyl cyclase; Ca2+, cAMP/PKA signaling pathways; K+ channels

Aplysia

Appetitive

Feeding

CPG neurons and their synaptic inputs in buccal ganglia

Dopamine

Enhancement of presynaptic inputs; plateau potential threshold increase

D2 receptors

Drosophila

Appetitive/

Locomotion

Sensory processing in antenna lobes and mushroom bodies

Octopamine Dopamine

D1 receptors Type I adenylyl cyclase; Ca2+, cAMP/PKA pathways

Aversive

Dopamine

Drosophila

Aversive

Turning

Mushroom bodies

cAMP/PKA pathways

Hermissenda

Aversive

Phototaxic

Photoreceptors; Interneurons in cerebropleural ganglia

GABA, 5-HT

Intrinsic excitability increase; strengthening of inhibitory synapses

GABAB receptors; Ca2+, PKC pathways; K+ and Ca2+ channels

Limax

Aversive

Feeding

Sensory processing in procerebral lobes

5-HT

Sensory elicited increase in proximal-to-distal ratio of oscillatory field potential

Appetitive

NO

Lymnaea

Appetitive

Feeding

Sensory and command interneurons in cerebral ganglion, and motor neurons in buccal ganglia

Change in synaptic inputs; resting membrane potential increase; intrinsic excitability increase

cAMP/PKA pathways; AMPA receptor levels; K+ and Na+ channels

Pleurobranchaea

Aversive

Feeding

Interneurons and command neurons in cerebropleural ganglia

Intrinsic excitability increase

In the terrestrial mollusk Limax, aversive olfactory learning develops when a natural food odor (CS) is paired with an aversive chemical or electrical stimulus (US) (Gelperin, 1975; Sahley et al., 1981; for review, see Watanabe et al., 2008). This simple classical conditioning and related higher order conditioning were monitored through the expression of behavioral choice or the latency in response to the CS. The CS pathway is composed of primary sensory cells in the tentacle epithelium that project to a tentacle ganglion. This ganglion contains local and projection neurons that send their axons to the procerebral lobe, a region specialized in olfactory processing (Ratté & Chase, 2000) where modulatory transmitters, including 5-HT and NO, mediate aversive and appetitive signals, respectively (Inoue et al. 2001; Yabumoto et al., 2008). Staining, imaging, and molecular data have suggested that learning and associated modulatory transmitters modify interneuronal activity in the procerebral lobe and local oscillatory field potentials that contribute to olfactory processing (Kimura et al., 1998; Nikitin & Balaban, 2000; Nakaya et al., 2001; Sekiguchi et al., 2010).

In the marine mollusk Hermissenda, phototaxic locomotion is reduced by pairing light (CS) with turbulence rotating or shaking of the animal (US) (Crow & Alkon, 1978; Crow & Jin, 2013). Before training, light elicits a forward movement of the animal, whereas a perturbation (US) produced by a rotating table to mimic turbulence of the animal’s natural environment elicits a foot contraction and shortening. After pairing light with the disturbance, an animal learns to contract its foot in response to light alone. Light activates five photoreceptors (two A types, three B types) in each eye. Through mono- and polysynaptic connections, these sensory cells excite and inhibit several groups of interneurons in the cerebropleural ganglia, which themselves elicit synaptic responses in pedal motor neurons. The US is conveyed by hair cells located in the statocyst organs. These sensory cells connect through polysynaptic tracts and different groups of interneurons to the pedal motor neurons, which mediate the unconditioned response (Crow & Tian, 2004). Interestingly, the hairs cells are connected to the photoreceptors via both a monosynaptic GABAergic connection and polysynaptic serotoninergic pathways (Alkon et al., 1993). Accordingly, the photoreceptors appear to be the primary site for convergence of the CS and US and are key loci of memory storage (Crow & Alkon, 1980). Thus, classical conditioning, or its analog in which a depolarization of the photoreceptors is paired with activation of the statocyst with an appropriate temporal relationship, causes a long-lasting increase and decrease in the intrinsic excitability of the type B and A photoreceptors, respectively (Farley et al., 1983). Learning also increases the inhibitory synaptic connection from the medial type B to type A photoreceptors (Frysztak & Crow, 1997). Furthermore, recent evidence has indicated that a second site of convergence of the CS/US is provided by identified interneurons in the unconditioned pathway that receives monosynaptic inputs from the sensory cells of the photoreceptors and hairs. Stimulus pairing-induced plasticity of intrinsic excitability in these neurons may also contribute to expression of the conditioned response (Crow & Tian, 2003; Crow & Jin, 2013). Thus, learning modifies sensory processing through changes in the pattern of light-elicited activity in the primary photoreceptors, and also in the interneuronal information processing that transforms the sensory activity into a motor response.

Classical Conditioning–Induced Plasticity in the Interneuronal Pathways Driving Motor Output

The marine mollusk Pleurobrancheae learns to avoid food stimuli after a training period in which this CS is associated with an electrical shock (US). This suppression of feeding is not expressed in control animals that receive the CS and US alternately rather than in temporal association (Mpitsos & Collins, 1975). Food stimuli applied to the oral veil elicits strong excitation of the PC and PSE cerebropleural to buccal interneurons (CBIs). Activity in these command neurons is critical for initiating feeding. It induces synaptic excitation of a half-center oscillator in the buccal ganglia, which is composed of the protraction and retraction neurons of the radula. Activity of the command neurons can be inhibited by two major classes of cerebropleural interneurons (I1, I2) through polysynaptic (I2) and monosynaptic (I1) connections (Jing & Gillette, 2000). Stimulus pairing-specific plasticity was found in the I2 neurons (London & Gillette, 1986). In semi-intact preparations from trained as compared to control animals, a food stimulus elicits a stronger and longer depolarization of the I2 neurons. This enhanced neuronal response causes a stronger inhibition of the command PC neurons, thereby leaving the radula in a retracted position. This pairing-specific plasticity in I2 results from a change in the neuron’s intrinsic excitability that in turn modifies I2’s response to CS-elicited presynaptic inputs.

Classical conditioning of feeding in the freshwater pond snail Lymnaea has clearly demonstrated that learning-induced plasticity can develop at multiple sites in a sensory-motor circuit with a critical contribution of bioelectrical changes in cerebrobuccal command interneurons (Benjamin et al., 2000). In an appetitive form of conditioning, animals learn from the association of a neutral tactile or chemical stimulus (CS) with a sucrose (US) application to the lips (Alexander et al., 1984; Kemenes & Benjamin, 1989). After repeated associations, and subsequently for several days, the CS alone elicits a pattern of feeding movements that is not expressed by control animals. The neuronal organization of the CS and US pathways has been described (Benjamin, 2012). Lip sensory neurons are connected to higher order command interneurons in the cerebral ganglia, including the serotoninergic CGC and CV1 neurons. These modulatory CBIs make synaptic connections within the buccal ganglia to the motor pattern generating network that is comprised of three interconnected cells (N1, N2, N3).

In naive animals, a touch to the lips (CS) fails to trigger any radula motor pattern emission. After training, the same stimulation elicits a bout of rhythmic and triphasic radula motor output. Both semi-intact preparations and an in vitro analog of conditioning were used to investigate the cellular basis of this learning. Pairing a tactile CS with the US produces an increase in the CS-elicited activity in the sensory inputs associated with an increase in their synaptic drives to the CBI neurons. Moreover, these high-order command neurons express a persistent depolarization that makes the cells more responsive to the CS. Finally, learning is correlated with an increase in the CPG-driven excitation of the B3 motor neuron (Staras et al., 1999; Jones et al., 2003). Similarly, pairing an appetitive chemical CS with the US is associated with a change in the resting membrane potential of the CGC and CV1 command neurons and a decrease in tonic firing of the inhibitory CPG neuron N3. All of these neuronal correlates might not be crucial for the expression of the conditioned response; nevertheless, experimental manipulations of the CV1’s resting membrane potential indicated that its plasticity alone is necessary and sufficient (Jones et al., 2003). Experimental changes in this resting membrane potential rendered the cell more or less responsive to the CS and were able to reproduce or block expression of the conditioned tactile response. Thus, these findings indicate that neuronal correlates of classical conditioning can be found at several levels in a distributed sensory-motor circuit and can involve both synaptic and passive membrane properties in the circuit’s constituent neurons (Table 1; Nikitin et al., 2008, 2013).

Operant Conditioning and Motor Systems

Fewer animal models have been developed to analyze operant than classical conditioning. Consequently, the available data may provide an incomplete picture of the neuronal plasticity induced by this form of associative learning. Nevertheless, converging findings in insects and mollusks can be seen to further confirm the idea that the inherent dynamics of neuronal circuits and neurons that actively generate operants play a fundamental role in the plasticity underlying operant learning (Table 2).

Table 2 Summary of Operant Conditioning

Animal

Learning Paradigm

Behavior

Site(s) of Plasticity

Modulatory Transmitter

Neuronal Plasticity

Subcellular Processes

Aplysia

Appetitive

Feeding

CPG neurons in buccal ganglia

Dopamine

Plateau potential threshold decrease; regularization of oscillatory membrane properties; intrinsic excitability increase; strengthening of electrical synapses

D1 receptors; Type II adenylyl cyclase Ca2+, PKC and cAMP/PKA pathways; CREB levels

Aversive

NO

PKA and PKC pathways; PolyADP-ribosepolymerase proteins

Cockroach/Grasshopper/Locust

Aversive

Leg positioning

Motor neurons in thoracic ganglia

Pacemaker properties

Drosophila

Aversive

Turning

Motor neurons

PKC pathways; transcription factors

Lymnaea

Aversive

Respiration

CPG neurons in pedal, parietal, and abdominal ganglia

Spontaneous activity decrease; intrinsic excitability decrease

PKC pathways; mRNA and protein synthesis; NMDA receptors

Whole and headless locusts, grasshoppers, and cockroaches can learn to maintain a leg position away from the preferred one after pairing the leg position with a positive (food reward) or negative (sound or electrical stimulus) reinforcement. A continuous negative reinforcement can reflexively elicit and maintain a moderate leg flexion. This reinforcement ceases when the leg is actively extended or more strongly flexed. In this operant conditioning procedure, therefore, an animal learns by its own action to escape from an aversive stimulus (Horridge, 1962; Hoyle, 1979, 1980; Harris, 1991). A similar training procedure was developed in a semi-intact preparation in which an identified motor neuron controlling leg position was recorded intracellularly. Computer-controlled electrical shocks to this leg were delivered in association to either a spontaneous increase (up-learning) or decrease (down-learning) in the motor neuron’s own discharge. Relative to prestimulus levels, the frequency of this spontaneous impulse activity increased or decreased after up- or down-learning, respectively. This plasticity was recorded in normal saline as well as in a solution blocking synaptic inputs and, thus, the plasticity appeared to involve the intrinsic “pacemaker” properties of the motor neuron itself, rather than the cell’s synaptic inputs (Woollacott & Hoyle, 1977; Hoyle, 1979).

Respiratory behavior in Lymnaea has provided a more extensive cellular analysis of aversive operant conditioning. In a hypoxic environment, this bimodal (cutaneous and lung) breathing mollusk performs aerial breathing by moving to the water surface and opening its respiratory apparatus, the pneumostome. During operant conditioning, each attempt at pneumostome opening was associated with delivery of a punishing tactile stimulation, which triggers pneumostome closure. Repeated associations between the operant and punishment reduced both the number of aerial breaths and the total breathing time. Yoked-control animals that received the tactile stimulus independently of their own behavior, but in temporal correlation with breathing in a trained snail, did not change their behavior. Furthermore, intermediate- and long-term memories developed, depending on the interval between training sessions (Lukowiak et al., 1996, 2003).

Rhythmic pneumostome opening and closing are generated by a CPG network composed of three main interneurons: a high-order command-like RPeD1 neuron and the IP3 and VD4 interneurons, which through their synapses promote activity in opener and closer motor neurons, respectively. The reciprocal inhibitory and/or excitatory connections between these three CPG neurons and their intrinsic firing properties, particularly postinhibitory rebound activation, are sufficient to generate the respiratory rhythm and pattern (Syed et al., 1990). In isolated nervous preparations from trained animals, and in in vitro semi-intact preparations, operant conditioning was found to reduce respiratory CPG activity by decreasing the spontaneous electrical activity of IP3. Additionally, learning reduced spontaneous activity in the higher order RPeD1 neurons and their ability to trigger patterned bursting activity in IP3 (Spencer et al., 1999; McComb et al., 2005). These changes were not observed in preparations from yoked control animals. Moreover, contingent-dependent plasticity in RPeD1 was associated with a reduction in the cell’s intrinsic excitability and input resistance, thereby indicating that this CPG neuron is one of the primary sites for the learning-induced plasticity (Braun & Lukowiak, 2011). This cell is also critically involved in long-term memory formation and storage, since the selective ablation of its soma prior to learning impaired long-term memory formation (Scheibenstock et al., 2002).

Plasticity in Synaptic and Intrinsic Membrane Properties Induced by Classical and Operant Conditioning in a Single Neuronal Circuit

A particularly fruitful approach to determine if classical and operant conditioning derives from the same or distinct neuronal processes is to compare the plasticity induced by these distinct paradigms in a same neuronal network. Different animal models, Drosophila and Aplysia, have allowed such an approach in which the same behavior, locomotion and feeding respectively, was modified by classical and operant learning procedures (Brembs & Heisenberg, 2000; Brembs et al., 2004; Baxter & Byrne, 2006). Because of the accessibility to individual neurons for intracellular recording and the resultant identification of underlying neuronal circuitry, Aplysia in particular has provided the basis for a comparative analysis of the functional plasticity involved in associative learning.

Feeding behavior in this animal can be modified by appetitive operant conditioning (as detailed earlier) and by appetitive classical conditioning (Colwill et al., 1997; Lechner et al., 2000a). In the latter case, a tactile stimulus (CS) to the lips was paired with delivery of food (US). After repeated CS/US associations, the number of ingestion movement cycles of the radula triggered by the tactile stimulus alone is strongly increased. It is important to note, however, that no change in the spontaneous emissions of these movement cycles occurred after learning. The conditioned response was retained for at least 24 hours and was not induced by unpaired stimuli or by the US alone (Lechner et al., 2000a). Thus, the same motor act, the ingestion movements of the radula, and thereby the same central circuit as that modified by operant conditioning can also be altered by classical conditioning. In addition, the same reinforcing and dopaminergic pathway was implicated in both conditioning procedures. Food ingestion rather than a simple contact of food on the lips provided the reinforcement in classical conditioning. Thus, a pairing procedure in which food (US) was withdrawn before it was ingested prevented Pavlovian learning. In accordance, an experimental lesion of the dopaminergic esophageal nerves, which convey sensory information about successful food intake, but which did not hamper the ability of animal to actually ingest food, also impaired learning (Lechner et al., 2000b). By taking advantage of the identification of this reinforcing pathway, an in vitro analog of classical conditioning was developed in which a brief electrical stimulation of a mechanoafferent lip nerve (CS) was paired with an electrical stimulation of the dopaminergic input nerve (US). This paradigm in turn reproduced the pairing-specific and dopamine-dependent increase in occurrences of the CS-elicited ingestion motor pattern (Mozzachiodi et al., 2003; Reyes et al., 2005).

Decision-Making CPG Neurons Are Common Loci for Plasticity in Classical and Operant Conditioning

The lip mechanosensory neurons that convey the CS excite higher order cerebrobuccal interneurons (CBIs). These command neurons are connected by mono- and polysynaptic pathways to buccal CPG neurons that generate the ingestion and egestion motor patterns. Neuronal correlates of classical conditioning have been investigated in isolated cerebral-buccal ganglia preparations from trained and control animals. Pairing-specific changes induced by classical conditioning were monitored in two types of decision neurons in the buccal CPG (Lechner et al., 2000b; Mozzachiodi et al., 2003): first, in B31/32 neurons, a neuronal pair that through their powerful monosynaptic excitatory synapses with B63 neurons contribute to the decision-= process that can spontaneously initiate the radula motor patterns (Susswein et al., 2002; Hurwitz et al., 2008); and secondly, in the decision-making neuron B51, a key CPG element that is involved in motor pattern selection leading to expression of the ingestion pattern (Nargeot et al., 1999a, b). Classical conditioning learning enhances the complex excitatory synaptic drive that is elicited in both these neurons in response to a transient stimulation of the lip mechanosensory input nerve. This synaptic plasticity recorded in B31/32 is not associated with any change in the cell’s intrinsic membrane properties. In contrast, in B51 the synaptic plasticity was associated with a pairing-specific reduction in its intrinsic excitability. Although no change in resting membrane potential and input resistance was found in B51, the change in excitability resulted from an increase in threshold for eliciting plateau potentials (Lorenzetti et al., 2006). An identical plasticity in B31/32 and B51 was also found in in vitro neuronal analogs of classical conditioning. Two other sites of plasticity have been identified after classical conditioning and in its in vitro analog. One site concerns the B4/5 cells in the buccal CPG, whereas the other site is a high-order CBI neuron in the cerebral ganglia. These cellular loci fail to exhibit intrinsic changes following classical conditioning. Nevertheless, in vitro CS-US pairing was found to increase in the CS-evoked excitatory synaptic drive to CBI, which consequently may contribute to the increased synaptic drive to B31/32.

Thus, decision-making neurons in the same motor pattern-generating network can express changes evoked by either classical or operant conditioning. The target neurons implicated are key CPG elements for both the generation and designation of motor patterns, and hence any changes in their synaptic inputs and/or intrinsic excitability can drastically alter the network’s output. Therefore, either in response to a transient CS in classical conditioning or to the autonomous process of motor pattern genesis in operant conditioning, appetitive learning increases the genesis and expression of a specific motor act, namely that responsible for ingestion.

Differential Contribution of Dynamic Membrane Properties to Classical and Operant Learning

A particularly interesting aspect of a comparative study of classical and operant conditioning of Aplysia’s feeding behavior is that a unique decision neuron (B51) in the buccal CPG network is able to express a pairing-specific plasticity induced by either learning procedure. Intriguingly, this plasticity, which involves the cell’s intrinsic membrane properties, differs according to the conditioning procedure and even though both constitute appetitive forms of learning (Lorenzetti et al., 2006). Plateau potential generation in B51 produces an extended retraction and closure of the radula, thereby promoting the act of food ingestion (Nargeot et al., 1999a, b). Operant conditioning enhances this capability (Brembs et al., 2002), whereas classical conditioning decreases the cell’s capability to generate plateau potentials. Here, however, the increase in ingestion responses is associated with stronger CS-elicited synaptic input to B51 (Mozzachiodi et al. 2003). Thus, appetitive classical conditioning appears to favor synapse-driven responses of the neuron, whereas appetitive operant conditioning favors expression of an endogenous membrane property that contributes to the autonomous process of motor pattern genesis.

Classical and operant learning in Aplysia and in the animal models described earlier has been related to plasticity in synaptic and intrinsic membrane properties (Mozzachiodi & Byrne, 2010) that determine the respondent or intrinsic dynamic behavior of neurons. Chemical synaptic transmission and intrinsic excitability contribute to the electrical activity of neurons by imparting proportionality (linearity) in their response to triggering input information. In contrast, plateau potentials, pacemaker or oscillatory membrane properties, and electrical synapses contribute to a neuron’s spontaneous or dynamic (nonlinear) electrical behavior that is not, or only partially, determined by external information. In this context, it is noticeable from the studies described earlier that classical conditioning primarily modifies the respondent bioelectrical behavior of neurons, whereas operant conditioning is largely but not exclusively associated with changes in their inherent dynamic properties.

Cellular Mechanisms Underlying Coincidence Detection and Memory

In associative learning, it is fundamental to understand how the temporal association that is essential for learning is encoded by central nervous systems. The earlier descriptions of pairing-specific plasticity in identified cells suggest that such an encoding might occur at the level of individual neurons (Abrams & Kandel, 1988). This hypothesis is further supported by the development of single-cell analogs of classical and operant conditioning that reproduces the essential features of a given learning-induced plasticity. These analogs have illustrated that two main elementary mechanisms, activity-dependent modulation and Hebbian-like plasticity, acting separately or in combination, contribute to the coincidence detection necessary for learning and memory (Tables 1 and 2; Bailey et al., 2000; Lechner & Byrne, 1998; Roberts & Glanzman, 2003).

Activity-Dependent Modulation in Classical Conditioning

A single-cell analog of classical conditioning of Aplysia’s gill-withdrawal reflex was developed in isolated sensory-motor neurons placed in co-culture. In this analog system, brief electrical excitations of a sensory neuron used to reproduce the CS was associated with puff applications of 5-HT analogous to the US (Eliot et al., 1994; Bao et al., 1998). This protocol reproduced the essential characteristics of the plasticity induced by learning in vivo, including the pairing-specific facilitation of the sensory-motor synapse. The facilitation was not associated with a long-lasting change in the frequency and amplitude of spontaneous miniature excitatory postsynaptic potentials, suggesting that it was related to a change in the sensory neuron’s spike-elicited release of transmitter. The sensory neuron spiking activity (CS) led to calcium entry and binding to the protein calmodulin in the presynaptic terminals. Serotonin application (US) to the same sensory neuron activated G protein–coupled receptor, cyclic adenosine monophosphate (cAMP) and protein kinase A (PKA) cascades. The temporal association between this electrical activity and serotonergic modulation in the single cell allows Ca2+/calmodulin binding to amplify the 5-HT-induced activity of a type I adenylyl cyclase (Abrams et al., 1991, 1998; Bao et al., 1998). This activity-dependent modulation, which implicates a dual activation of adenylyl cyclase, leads to a strong increase in intracellular cAMP levels compared to neurons subjected to a nonassociative procedure (Ocorr et al., 1985). Thereby, the synaptic plasticity induced by an associative learning paradigm appears to involve an amplification of heterosynaptic facilitation, a form of presynaptic plasticity that develops in nonassociative learning (Byrne & Kandel, 1996; Byrne & Hawkins, 2015; Yu & Rankin, 2017). In its short-term form, this facilitation leads to an increase in PKA and a reduction of K+ currents, with resultant spike broadening and an increased excitability of the sensory neuron. These bioelectrical changes thus favor action potential-evoked calcium influx and transmitter release. Mechanism for long-term memory in classical conditioning has not been investigated in Aplysia, yet. However, classical conditioning of feeding in Lymnaea recruits a PKA pathway that activates the transcription factor cAMP response element-binding protein (CREB) and contributes to different phases of long-term memory consolidation (Ribeiro et al., 2003; Kemenes et al., 2006; Michel et al., 2008).

In Drosophila and Apis, activity-dependent modulation by dopamine or octopamine as principal neuromodulators and the involvement of calcium-sensitive adenylyl cyclase as a molecular coincidence detector were also found to contribute to appetitive and aversive olfactory classical conditioning (Tsydzik & Wright, 2009; Tomchik & Davis, 2009; Gervasi et al., 2010; Guven-Ozkan & Davis, 2014; Matsumoto et al., 2014). Genetic analysis first identified two learning mutants, Rutabaga and Dunce, which led to abnormally weak memory after classical conditioning in which odorants were associated with electrical shock stimulation. These mutations specifically affected the expression of a type I adenylyl cyclase and a cAMP-specific phosphodiesterase in Rutabaga and Dunce, respectively (Dudai et al., 1976; Livingstone et al., 1984; Chen et al., 1986; Levin et al., 1992). In addition, genetic manipulations that overexpressed peptide inhibiting PKA activity also disrupted associative learning (Drain et al., 1991). These proteins are mainly expressed in neurons of the mushroom bodies, which play a critical role in olfactory learning (Nighorn et al., 1991; Han et al., 1992; Skoulakis et al., 1993). In an analog of classical conditioning, bath application of either dopamine or octopamine, the monoamines released in the mushroom bodies in response to aversive and/or appetitive stimuli, was used as a US. Application of acetylcholine, a transmitter released in the same neuronal structure in response to odors, was used as a CS (Gervasi et al., 2010). Dopamine or octopamine increased PKA activity in specific lobes or in all parts of the mushroom bodies, respectively, and required the contribution of adenylyl cyclase activity. Acetylcholine increased intracellular calcium concentration in all regions of the mushroom bodies, although this transmitter alone has no effect on PKA activity. However, conjoint application of acetylcholine (CS) with either dopamine or octopamine (US) led to a much stronger PKA activity than the monoaminergic modulation alone. This pairing-specific increase in PKA activity was not induced in Rutabaga flies, thus indicating that this associative effect is critically dependent on adenylyl cyclase activity. Memory traces, partly in relationship to their duration and type of learning, were found in various neuronal structures, including different lobes and neurons in the mushroom bodies. Furthermore, short-term memory tended to be correlated with the recruitment of new synapses, whereas long-term memory was found to be generally dependent on proteins synthesis and activity of CREB (Davis, 2011).

Classical conditioning of phototaxic behavior in Hermissenda revealed that activity-dependant modulation induced by CS/US pairing can also implicate protein kinase C (PKC) signaling pathways (Acosta-Urquidi et al., 1984). In in vitro preparations, light (CS), as well as photoreceptor depolarization, leads to intracellular calcium elevation and translocation of PKC from cytosol to the neuronal membrane (Muzzio et al., 1997). In contrast, hair cell activity (US) and the release of GABA acting via GABAB receptors activate phospholipase A2 (PLA2) in the B photoreceptors. A persistent activation of PKC requires the conjoint involvement of a calcium/diacylglycerol elevation induced by the CS, and activation of PLA2 and production of arachidonic acids in response to the US (Matzel et al., 1998; Blackwell, 2006). This persistent activation of PKC phosphorylates different proteins including potassium and calcium channels that in turn increase B receptor excitability (Alkon et al., 1982; Alkon, 1984; Farley & Auerbach, 1986). Long-term memory depends on protein synthesis (Crow & Forrester, 1990).

Activity-Dependent Modulation in Operant Conditioning

A cellular analog of appetitive operant conditioning of feeding behavior in Aplysia indicated that adenylyl cyclase can also serve as a molecular coincident detector in this form of associative learning (Lorenzetti et al., 2008). In this analog, plateau potentials can be experimentally triggered in the B51 decision neurons placed in cell culture, and serve as an analog of the operant, whereas a puff of dopamine acts as an analog of positive reinforcement. Pairing expression of plateau potential with a dopamine pulse reproduces the changes in the intrinsic membrane properties of B51 that are recorded after in vivo operant conditioning. Dopamine activates a D1-like receptor, which recruits cAMP/PKA signaling pathways. Dopamine alone did not raise cAMP to a level sufficient to induce the cellular plasticity. However, when electrical activity in B51 was repetitively paired with a puff of dopamine to reproduce the operant training procedure, activity in B51 produced a calcium influx and a subsequent activation of calcium-dependent PKC that was necessary to modify the intrinsic membrane properties of B51. A convergent site for the effect of dopamine and electrical activity-induced activation of calcium-sensitive PKC was localized upstream to cAMP production, probably at the level of the adenylyl cyclase complex. These data therefore suggest that mechanisms of coincidence detection in operant conditioning may be very similar to those described in classical conditioning, although the target molecules of the signaling cascade in the appetitive form of operant conditioning remain to be investigated. In the aversive form, PKA/PKC contributes differently to short- and long-term memory (Michel et al., 2010). Moreover, this latter requires protein synthesis and an activation of polyADP-ribose-polymerase proteins (Cohen-Armon et al., 2004; Michel et al., 2012).

PKA and PKC similarly contribute to operant learning in the respiratory behavior of Lymnaea (Rosenegger & Lukowiak, 2010; Takigami et al., 2014). In Drosophila, PKA and PKC signaling contributes differently to classical and operant conditioning (Brembs & Plendl, 2008). Thus, Rutabaga mutants that are deficient in type I adenylyl cyclase and are poor learners in classical conditioning express an unaltered operant conditioning of turning behavior. However, this latter form of learning is impaired in mutants expressing an inhibitor protein for PKC. Interestingly, evidence suggests that the action of PKC is required in motor neurons (Colomb & Brembs, 2016).

Hebbian Synaptic Plasticity

In addition to activity-dependent modulation, there is evidence to indicate that classical conditioning of gill- and siphon-withdrawal reflexes in Aplysia involves a Hebbian-type plasticity (see earlier; Lin & Glanzman, 1994a, b; Roberts & Glanzman, 2003; Antonov et al., 2003). This plasticity, which potentiates sensory-motor synaptic transmission in a long-lasting manner, develops from the coincident activation of pre- and postsynaptic neurons. It depends on specific properties of the NMDA subtype of glutamate receptors, the permeability of which requires both glutamate binding and a concomitant membrane depolarization. Thus, NMDA receptors can serve as a second molecular coincidence detector in which presynaptic glutamate release is elicited by the CS, and postsynaptic neuron depolarization is produced by the US. This dual activation of NMDA receptors leads to calcium influx that through an activation of kinases, and long-lasting modifications of AMPA receptor-mediated responses and trafficking, enhances synaptic transmission (Murphy & Glanzman, 1996). Interestingly, a contribution of NMDA receptors in classical conditioning was also described in Drosophila (Xia et al., 2005; Miyashita et al., 2012; Ueno et al., 2017), as well as for intermediate and long-term memory formation following respiratory operant conditioning in Lymnaea (Rosenegger & Lukowiak, 2010).

Conclusion

Associative learning, including classical and operant conditioning, has been extensively investigated in invertebrates. The accessibility for experimental analysis of the neuronal networks that generate behaviors in these animals has allowed many fundamental concepts of the neural substrates of learning and memory to be established. This chapter has attempted to summarize this knowledge with a special emphasis placed on characterizing common and distinguishing features of these learning processes in a variety of invertebrate model systems, from the behavioral through to cellular and subcellular levels.

Learning in classical conditioning depends on an association between sensory stimuli that are received regardless of the animal’s behavior, whereas in operant conditioning, the learning process develops through an association between an emitted motor act and a sensory-derived outcome for that behavior. Despite this procedural difference, cellular analyses of associative learning in invertebrates indicate that both forms of learning implicate a number of important common features. First, both learning processes are correlated with plasticity of synapses and membrane properties that can occur in multiple constituent neurons of a circuit. Second, in both cases this neuronal plasticity is mediated by an appetitive or aversive stimulus and the release of modulatory transmitters, such as monoamines. Third, adenylyl cyclases serve as a molecular site of convergence for events in both types of learning. Fourth, at the cellular level, classical and operant conditioning produce activation of second messenger cascades, implicating calcium and various protein kinases, which in turn lead to posttranscriptional changes of pre-existing proteins in short-term memory and de novo protein synthesis in long-term memory.

However, several major distinguishing processes in classical and operant conditioning must be underlined. First, learning in classical conditioning is determined by changes in sensory processing and the transmission of sensory information to target motor systems. In contrast, according to present knowledge, the induction of the neuronal changes in operant conditioning appears to depend more on motor pattern generation or decision-making processes. Second, a corollary of this observation is that dynamic and nonlinear membrane properties of neurons, such as oscillatory, pacemaker, and plateau properties, appear to play a key role in the induction and expression of plasticity in operant conditioning. This is in contrast to classical conditioning, where the resultant neuronal plasticity depends essentially on the conveyance of sensory-elicited activity that is governed by the linear characteristics of synaptic transmission and neuronal intrinsic excitability. Third, unlike classical conditioning, no changes in actual sensory processing have thus far been identified in operant conditioning. Fourth, the strengthening of synaptic transmission within a circuit by Hebbian plasticity has not (yet, at least) been found to be critical for learning in operant conditioning.

Neuronal analyses of associative learning in invertebrates are still progressing and undoubtedly will further evolve and improve these conclusions. In particular, the contribution of changes in sensory processing in operant conditioning and a possible role for dynamic membrane properties of decision neurons in classical conditioning should now be investigated. Then, it would be interesting to determine which of these processes are critical to the induction and expression of the plasticity in both forms of learning and which ones are not. Finally, it is important to realize that the fundamental concepts that have emerged from the cellular analysis of associative learning in invertebrates are to a large extent shared with vertebrate learning. Thus, future developments of investigations in invertebrates are likely to provide essential information on the similarities and specificities of learning in the animal kingdom in general.

Acknowledgments

We thank Dr. John Simmers and Victoria Normand for helpful discussions and corrections of the manuscript. The preparation of this chapter was supported by grants ANImE ANR-13-BSV5-0014-01, BRAIN ANR-10-LABX-43 and ANR-10-IDEX-03-02.

References

Abrams, T. W. (1985). Activity-dependent presynaptic facilitation: an associative mechanism in Aplysia. Cellular and Molecular Neurobiology, 5, 123–145.Find this resource:

Abrams, T. W., & Kandel, E. R. (1988). Is contiguity detection in classical conditioning a system or a cellular property? Learning in Aplysia suggests a possible molecular site. Trends in Neuroscience, 11, 128–135.Find this resource:

Abrams, T. W., Karl, K. A., & Kandel, E. R. (1991). Biochemical studies of stimulus convergence during classical conditioning in Aplysia: Dual regulation of adenylate cyclase by Ca2+/calmodulin and transmitter. Journal of Neuroscience, 11, 2655–2665.Find this resource:

Abrams, T. W., Yovell, Y., Onyike, C. U., Cohen, J. E., & Jarrard, H. E. (1998). Analysis of sequence-dependent interactions between transient calcium and transmitter stimuli in activating adenylyl cyclase in Aplysia: Possible contribution to CS—US sequence requirement during conditioning. Learning and Memory, 4, 496–509.Find this resource:

Acosta-Urquidi, J., Alkon, D. L., & Neary, J. T. (1984). Ca2+-dependent protein kinase injection in a photoreceptor mimics biophysical effects of associative learning. Science, 224, 1254–1257.Find this resource:

Alexander, J., Audesirk, T. E., & Audesirk, G. J. (1984). One-trial reward learning in the snail Lymnea stagnalis. Journal of Neurobiology, 15, 67–72.Find this resource:

Alkon, D. L. (1984). Calcium-mediated reduction of ionic currents: A biophysical memory trace. Science, 226, 1037–1045.Find this resource:

Alkon, D. L., Anderson, M. J., Kuzirian, A. J., Rogers, D. F., Fass, D. M., Collin, C., … Matzel, L. D. (1993). GABA-mediated synaptic interaction between the visual and vestibular pathways of Hermissenda. Journal of Neurochemistry, 61, 556–566.Find this resource:

Alkon, D. L., Lederhendler, I., & Shoukimas, J. J. (1982). Primary changes of membrane currents during retention of associative learning. Science, 215, 693–695.Find this resource:

Antonov, I., Antonova, I., Kandel, E. R., & Hawkins, R. D. (2001). The contribution of activity-dependent synaptic plasticity to classical conditioning in Aplysia. Journal of Neuroscience, 21, 6413–6422.Find this resource:

Antonov, I., Antonova, I., Kandel, E. R., & Hawkins, R. D. (2003). Activity-dependent presynaptic facilitation and Hebbian LTP are both required and interact during classical conditioning in Aplysia. Neuron, 37, 135–147.Find this resource:

Antonov, I., Ha, T., Antonova, I., Moroz, L. L., & Hawkins, R. D. (2007). Role of nitric oxide in classical conditioning of siphon withdrawal in Aplysia. Journal of Neuroscience, 27, 10993–11002.Find this resource:

Antonov, I., Kandel, E. R., & Hawkins, R. D. (1999). The contribution of facilitation of monosynaptic PSPs to dishabituation and sensitization of the Aplysia siphon withdrawal reflex. Journal of Neuroscience, 19, 10438–10450.Find this resource:

Bailey, C. H., Giustetto, M., Huang, Y. Y., Hawkins, R. D., & Kandel, E. R. (2000). Is heterosynaptic modulation essential for stabilizing Hebbian plasticity and memory? Nature Review in Neuroscience, 1, 11–20.Find this resource:

Bao, J. X., Kandel, E. R., & Hawkins, R. D. (1998). Involvement of presynaptic and postsynaptic mechanisms in a cellular analog of classical conditioning at Aplysia sensory-motor neuron synapses in isolated cell culture. Journal of Neuroscience, 18, 458–466.Find this resource:

Baxter, D. A., & Byrne, J. H. (2006). Feeding behavior of Aplysia: A model system for comparing cellular mechanisms of classical and operant conditioning. Learning and Memory, 13, 669–680.Find this resource:

Bédécarrats, A., Cornet, C., Simmers, J., & Nargeot, R. (2013). Implication of dopaminergic modulation in operant reward learning and the induction of compulsive-like feeding behavior in Aplysia. Learning and Memory, 20, 318–327.Find this resource:

Benjamin, P. R. (2012). Distributed network organization underlying feeding behavior in the mollusk Lymnaea. Neural Systems and Circuits, 2, 4.Find this resource:

Benjamin, P. R., Staras, K., & Kemenes, G. (2000). A systems approach to the cellular analysis of associative learning in the pond snail Lymnaea. Learning and Memory, 7, 124–131.Find this resource:

Bitterman, M. E., Menzel, R., Fietz, A., & Schäfer, S. (1983). Classical conditioning of proboscis extension in honeybees (Apis mellifera). Journal of Comparative Psychology, 97, 107–119.Find this resource:

Blackwell, K. T. (2006). Subcellular, cellular, & circuit mechanisms underlying classical conditioning in Hermissenda crassicornis. Anatatomical Record B New Anatomist, 289, 25–37.Find this resource:

Buonomano, D. V., & Byrne, J. H. (1990). Long-term synaptic changes produced by a cellular analog of classical conditioning in Aplysia. Science, 249, 420–423.Find this resource:

Braun, M. H., & Lukowiak, K. (2011). Intermediate and long-term memory are different at the neuronal level in Lymnaea stagnalis (L.). Neurobiology Learning and Memory, 96, 403–416.Find this resource:

Brembs, B., Baxter, D. A., & Byrne, J. H. (2004). Extending in vitro conditioning in Aplysia to analyze operant and classical processes in the same preparation. Learning and Memory, 11, 412–420.Find this resource:

Brembs, B., & Heisenberg, M. (2000). The operant and the classical in conditioned orientation of Drosophila melanogaster at the flight simulator. Learning and Memory, 7, 104–115.Find this resource:

Brembs, B., Lorenzetti, F. D., Reyes, F. D., Baxter, D. A., & Byrne, J. H. (2002). Operant reward learning in Aplysia: Neuronal correlates and mechanisms. Science, 296, 1706–1709.Find this resource:

Brembs, B., & Plendl, W. (2008). Double dissociation of PKC and AC manipulations on operant and classical learning in Drosophila. Current Biology, 18, 1168–1171.Find this resource:

Byrne, J. H. (1987). Cellular analysis of associative learning. Physiology Review, 67, 329–439.Find this resource:

Byrne, J. H., & Hawkins, R. D. (2015). Nonassociative learning in invertebrates. Cold Spring Harbor Perspectives in Biology, 7, a021675.Find this resource:

Byrne, J. H., & Kandel, E. R. (1996). Presynaptic facilitation revisited: State and time dependence. Journal of Neuroscience, 16, 425–435.Find this resource:

Carcaud, J., Giurfa, M., & Sandoz, J. C. (2016). Parallel olfactory processing in the honey bee brain: Odor learning and generalization under selective lesion of a projection neuron tract. Frontiers in Integrative Neuroscience, 9, 75.Find this resource:

Carew, T. J., Hawkins, R. D., & Kandel, E. R. (1983). Differential classical conditioning of a defensive withdrawal reflex in Aplysia californica. Science, 219, 397–400.Find this resource:

Carew, T. J., Walters, E. T., & Kandel, E. R. (1981). Classical conditioning in a simple withdrawal reflex in Aplysia californica. Journal of Neuroscience, 12, 1426–1437.Find this resource:

Chen, C. N., Denome, S., & Davis, R. L. (1986). Molecular analysis of cDNA clones and the corresponding genomic coding sequences of the Drosophila dunce+ gene, the structural gene for cAMP phosphodiesterase. Proceedings of the National Academy of Sciences USA, 83, 9313–9317.Find this resource:

Cleary, L. J., Byrne, J. H., & Frost, W. N. (1995). Role of interneurons in defensive withdrawal reflexes in Aplysia. Learning and Memory, 2, 133–151.Find this resource:

Cohen-Armon, M., Visochek, L., Katzoff, A., Levitan, D., Susswein, A. J., Klein, R., Valbrun, M., & Schwartz, J. H. (2004). Long-term memory requires polyADP-ribosylation. Science, 304, 1820–1822.Find this resource:

Colomb, J., & Brembs, B. (2016). PKC in motorneurons underlies self-learning, a form of motor learning in Drosophila. Peer Journal, 4, e1971.Find this resource:

Colomb, J., Kaiser, L., Chabaud, M. A., & Preat, T. (2009). Parametric and genetic analysis of Drosophila appetitive long-term memory and sugar motivation. Genes Brain Behavior, 8, 407–415.Find this resource:

Colwill, R. M., Absher, R. A., & Roberts, M. L. (1988). Context-US learning in Aplysia californica. Journal of Neuroscience, 8, 4434–4439.Find this resource:

Colwill, R. M., Goodrum, K., & Martin, A. (1997). Pavlovian appetitive discriminative conditioning in Aplysia californica. Animal Learning and Behavior, 25, 268–276.Find this resource:

Cook, D. G., & Carew, T. J. (1986). Operant conditioning of head-waving in Aplysia. Proceedings of the National Academy of Sciences USA, 83, 1120–1124.Find this resource:

Cook, D. G., & Carew, T. J. (1989a). Operant conditioning of head-waving in Aplysia. II. Contingent modification of electromyographic activity in identified muscles. Journal of Neuroscience, 9, 3107–3114.Find this resource:

Cook, D. G., & Carew, T. J. (1989b). Operant conditioning of head-waving in Aplysia. III. Cellular analysis of possible reinforcement pathways. Journal of Neuroscience, 9, 3115–3122.Find this resource:

Cropper, E. C., Evans, C. G., Hurwitz, I., Jing J., Proekt, A., Romero, A., & Rosen, S. C. (2004). Feeding neural networks in the mollusc Aplysia. Neurosignals, 3, 70–86.Find this resource:

Crow, T., & Alkon, D. L. (1978). Retention of an associative behavioral change in Hermissenda. Science, 201, 1240–1241.Find this resource:

Crow, T., & Alkon, D. L. (1980). Associative behavioral modification in Hermissenda: Cellular correlates. Science, 209, 412–414.Find this resource:

Crow, T., & Forrester, J. (1990). Inhibition of protein synthesis blocks long-term enhancement of generator potentials produced by one-trial in vivo conditioning in Hermissenda. Proceedings of the National Academy of Sciences USA, 87, 4490–4494.Find this resource:

Crow, T., & Jin, N. G. (2013). Multiple cellular and synaptic mechanisms in Hermissenda Pavlovian conditioning. In R. Menzel & P. R. Benjamin (Eds.), Invertebrate learning and memory (pp. 236–250). San Diego, CA: Academic.Find this resource:

Crow, T., & Tian, L. M. (2003). Neural correlates of Pavlovian conditioning in components of the neural network supporting ciliary locomotion in Hermissenda. Learning and Memory, 10, 209–216.Find this resource:

Crow, T., & Tian, L. M. (2004). Statocyst hair cell activation of identified interneurons and foot contraction motor neurons in Hermissenda. Journal of Neurophysiology, 91, 2874–2883.Find this resource:

Davis, R. L. (2005). Olfactory memory formation in Drosophila: From molecular to systems neuroscience. Annual Review in Neuroscience, 28, 275–302.Find this resource:

Davis, R. L. (2011). Traces of Drosophila memory. Neuron, 70, 8–19.Find this resource:

Drain, P., Folkers, E., & Quinn, W. G. (1991). cAMP-dependent protein kinase and the disruption of learning in transgenic flies. Neuron, 6, 71–82.Find this resource:

Dudai, Y., Jan, Y. N., Byers, D., Quinn, W. G., & Benzer, S. (1976). Dunce, a mutant of Drosophila deficient in learning. Proceedings of the National Academy of Sciences USA, 73, 1684–1688.Find this resource:

Eliot, L. S., Hawkins, R. D., Kandel, E. R., Schacher, S. (1994). Pairing-specific, activity-dependent presynaptic facilitation of Aplysia sensory-motor neuron synapses in isolated cell culture. Journal of Neuroscience, 14, 368–383.Find this resource:

Farley, J., & Auerbach, S. (1986). Protein kinase C activation induces conductance changes in Hermissenda photoreceptors like those seen in associative learning. Nature, 319, 220–223.Find this resource:

Farley, J., Richards, W. G., Ling, L. J., Liman, E., & Alkon, D. L. (1983). Membrane changes in a single photoreceptor cause associative learning in Hermissenda. Science, 221, 1201–1203.Find this resource:

Frost, W. N., Clark, G. A., & Kandel, E. R. (1988). Parallel processing of short-term memory for sensitization in Aplysia. Journal of Neurobiology, 19, 297–334.Find this resource:

Frost, W. N., & Kandel, E. R. (1995). Structure of the network mediating siphon-elicited siphon withdrawal in Aplysia. Journal of Neurophysiology, 73, 2413–2427.Find this resource:

Frost, L., Kaplan, S. W., Cohen, T. E., Henzi, V., Kandel, E. R., & Hawkins, R. D. (1997). A simplified preparation for relating cellular events to behavior: Contribution of LE and unidentified siphon sensory neurons to mediation and habituation of the Aplysia gill-and siphon-withdrawal reflex. Journal of Neuroscience, 17, 2900–2913.Find this resource:

Frysztak, R. J., & Crow, T. (1997). Synaptic enhancement and enhanced excitability in presynaptic and postsynaptic neurons in the conditioned stimulus pathway of Hermissenda. Journal of Neuroscience, 17, 4426–4433.Find this resource:

Gelperin, A. (1975). Rapid food-aversion learning by a terrestrial mollusk. Science, 189, 567–570.Find this resource:

Gervasi, N., Tchénio, P., & Preat, T. (2010). PKA dynamics in a Drosophila learning center: Coincidence detection by rutabaga adenylyl cyclase and spatial regulation by dunce phosphodiesterase. Neuron, 65, 516–529.Find this resource:

Giurfa, M., & Sandoz, J. C. (2012). Invertebrate learning and memory: Fifty years of olfactory conditioning of the proboscis extension response in honeybees. Learning and Memory, 19, 54–66.Find this resource:

Guven-Ozkan, T., & Davis, R. L. (2014). Functional neuroanatomy of Drosophila olfactory memory formation. Learning and Memory, 21, 519–526.Find this resource:

Hammer, M. (1993). An identified neuron mediates the unconditioned stimulus in associative olfactory learning in honeybees. Nature, 366, 59–63.Find this resource:

Hammer, M. (1997). The neural basis of associative reward learning in honeybees. Trends in Neuroscience, 20, 245–252.Find this resource:

Han, P.-L., Levin, L. R., Reed, R. R., & Davis, R. L. (1992). Preferential expression of the Drosophila rutabaga gene in mushroom bodies, neural centers for learning in insects. Neuron, 9, 619–627.Find this resource:

Harris, C. L. (1991). An improved Horridge procedure for studying leg-position learning in cockroaches. Physiology and Behavior, 49, 543–548.Find this resource:

Hawkins, R. D., Abrams, T. W., Carew, T. J., & Kandel, E. R. (1983). A cellular mechanism of classical conditioning in Aplysia: Activity-dependent amplification of presynaptic facilitation. Science, 219, 400–404.Find this resource:

Hawkins, R. D., & Byrne, J. H. (2015). Associative learning in invertebrates. Cold Spring Harbor Perspectives in Biology, 7, a021709.Find this resource:

Hawkins, R. D., Carew, T. J., & Kandel, E. R. (1986). Effects of interstimulus interval and contingency on classical conditioning of the Aplysia siphon withdrawal reflex. Journal of Neuroscience, 6, 1695–1701.Find this resource:

Hawkins, R. D., Castellucci, V. F., & Kandel, E. R. (1981). Interneurons involved in mediation and modulation of gill-withdrawal reflex in Aplysia. I. Identification and characterization. Journal of Neurophysiology, 45, 304–314.Find this resource:

Hawkins, R. D., Clark, G. A., & Kandel, E. R. (2006). Operant conditioning of gill withdrawal in Aplysia. Journal of Neuroscience, 26, 2443–2448.Find this resource:

Hawkins, R. D., Lalevic, N., Clark, G. A., & Kandel, E. R. (1989). Classical conditioning of the Aplysia siphon-withdrawal reflex exhibits response specificity. Proceedings of the National Academy of Sciences USA, 86, 7620–7624.Find this resource:

Hawkins, R. D., & Schacher, S. (1989). Identified facilitator neurons L29 and L28 are excited by cutaneous stimuli used in dishabituation, sensitization, and classical conditioning of Aplysia. Journal of Neuroscience, 9, 4236–4245.Find this resource:

Horridge, G. A. (1962). Learning of leg position by headless insects. Nature, 193, 697–698.Find this resource:

Hoyle, G. (1979). Mechanisms of simple motor learning. Trends in Neuroscience, 2, 153–155.Find this resource:

Hoyle, G. (1980). Learning, using natural reinforcements, in insect preparations that permit cellular neuronal analysis. Journal of Neurobiology, 11, 323–354.Find this resource:

Hurwitz, I., Ophir, A., Korngreen, A., Koester, J., & Susswein, A. J. (2008). Currents contributing to decision making in neurons B31/B32 of Aplysia. Journal of Neurophysiology, 99, 814–830.Find this resource:

Inoue, T., Watanabe, S., & Kirino, Y. (2001). Serotonin and NO complementarily regulate generation of oscillatory activity in the olfactory CNS of a terrestrial mollusk. Journal of Neurophysiology, 85, 2634–2638.Find this resource:

Jing, J., & Gillette, R. (2000). Escape swim network interneurons have diverse roles in behavioral switching and putative arousal in Pleurobranchaea. Journal of Neurophysiology, 83, 1346–1355.Find this resource:

Jones, N. G., Kemenes, I., Kemenes, G., & Benjamin, P. R. (2003). A persistent cellular change in a single modulatory interneuron contributes to associative long term memory. Current Biology, 13, 1064–1069.Find this resource:

Kabotyanski, E. A., Baxter, D. A., & Byrne, J. H. (1998). Identification and characterization of catecholaminergic neuron B65, which initiates and modifies patterned activity in the buccal ganglia of Aplysia. Journal of Neurophysiology, 79, 605–621.Find this resource:

Katzoff, A., Ben-Gedalya, T., Hurwitz, I., Miller, N., Susswein, Y. Z., & Susswein, A. J. (2006). Nitric oxide signals that Aplysia have attempted to eat, a necessary component of memory formation after learning that food is inedible. Journal of Neurophysiology, 96, 1247–1257.Find this resource:

Katzoff, A., Ben-Gedalya, T., & Susswein, A. J. (2002). Nitric oxide is necessary for multiple memory processes after learning that a food is inedible in Aplysia. Journal of Neuroscience, 22, 9581–9594.Find this resource:

Katzoff, A., Miller, N., & Susswein, A. J. (2009). Nitric oxide and histamine signal attempts to swallow: A component of learning that food is inedible in Aplysia. Learning and Memory, 17, 50–62.Find this resource:

Kemenes, G., & Benjamin, P. R. (1989). Appetitive learning in snails shows characteristics of conditioning in vertebrates. Brain Research, 489, 163–166.Find this resource:

Kemenes, G., Kemenes, I., Michel, M., Papp, A., & Müller, U. (2006). Phase-dependent molecular requirements for memory reconsolidation: Differential roles for protein synthesis and protein kinase A activity. Journal of Neuroscience, 26, 6298–6302.Find this resource:

Kim, Y. C., Lee, H. G., & Han, K. A. (2007). Classical reward conditioning in Drosophila melanogaster. Genes Brain Behavior, 6, 201–207.Find this resource:

Kimura, T., Suzuki, H., Kono, E., & Sekiguchi, T. (1998). Mapping of interneurons that contribute to food aversion conditioning in the slug brain. Learning and Memory, 4, 376–388.Find this resource:

Kupfermann, I. (1974) Feeding behavior in Aplysia: A simple system for the study of motivation. Behavior Biology, 10, 1–26.Find this resource:

Lechner, H. A., Baxter, D. A., & Byrne, J. H. (2000a). Classical conditioning of feeding in Aplysia: I. Behavioral analysis. Journal of Neuroscience, 20, 3369–3376.Find this resource:

Lechner, H. A., Baxter, D. A., & Byrne, J. H. (2000b). Classical conditioning of feeding in Aplysia: II. Neurophysiological correlates. Journal of Neuroscience, 20, 3377–3386.Find this resource:

Lechner, H. A., & Byrne, J. H. (1998). New perspectives on classical conditioning: A synthesis of Hebbian and non-Hebbian mechanisms. Neuron, 20, 355–358.Find this resource:

Levin, L. R., Han, P. L., Hwang, P. M., Feinstein, P. G., Davis, R. L., & Reed, R. R. (1992). The Drosophila learning and memory gene rutabaga encodes a Ca2+/Calmodulinresponsive adenylyl cyclase. Cell, 68, 479–489.Find this resource:

Lin, X. Y., & Glanzman, D. L. (1994a). Long-term potentiation of Aplysia sensorimotor synapses in cell culture: Regulation by postsynaptic voltage. Proceedings of the Academy of Biology Sciences, 255, 113–118.Find this resource:

Lin, X. Y., & Glanzman, D. L. (1994b). Hebbian induction of long-term potentiation of Aplysia sensorimotor synapses: Partial requirement for activation of an NMDA-related receptor. Proceedings of the Academy of Biology Sciences, 255, 215–221.Find this resource:

Liu, C., Plaçais, P. Y., Yamagata, N., Pfeiffer, B. D., Aso, Y., Friedrich, A. B., … Tanimoto, H. (2012). A subset of dopamine neurons signals reward for odour memory in Drosophila. Nature, 488, 512–516.Find this resource:

Livingstone, M. S., Sziber, P. P., & Quinn, W. G. (1984). Loss of calcium/calmodulin responsiveness in adenylate cyclase of rutabaga, a Drosophila learning mutant. Cell, 37, 205–215.Find this resource:

London, J. A., & Gillette, R. (1986). Mechanism for food avoidance learning in the central pattern generator of feeding behavior of Pleurobranchae californica. Proceedings of the National Academy of Sciences USA, 83, 4058–4062.Find this resource:

Lorenzetti, F. D., Baxter, D. A., & Byrne, J. H. (2008). Molecular mechanisms underlying a cellular analog of operant reward learning. Neuron, 59, 815–828.Find this resource:

Lorenzetti, F. D., Mozzachiodi, R., Baxter, D. A., & Byrne, J. H. (2006). Classical and operant conditioning differentially modify the intrinsic properties of an identified neuron. Nature Neuroscience, 9, 17–19.Find this resource:

Lukowiak, K., Ringseis, E., Spencer, G., Wildering, W., & Syed, N. (1996). Operant conditioning of aerial respiratory behaviour in Lymnaea stagnalis. Journal of Experimental Biology, 199, 683–691.Find this resource:

Lukowiak, K., Sangha, S., Scheibenstock, A., Parvez, K., McComb, C., Rosenegger, D., Varshney, N., & Sadamoto, H. (2003). A molluscan model system in the search for the engram. Journal of Physiology (Paris), 97, 69–76.Find this resource:

Lyons, L. C., Rawashdeh, O., Katzoff, A., Susswein, A. J., & Eskin, A. (2005). Circadian modulation of complex learning in diurnal and nocturnal Aplysia. Proceedings of the National Academy of Sciences USA, 102, 12589–12594.Find this resource:

Mackey, S. L., Kandel, E. R., & Hawkins, R. D. (1989). Identified serotonergic neurons LCB1 and RCB1 in the cerebral ganglia of Aplysia produce presynaptic facilitation of siphon sensory neurons. Journal of Neuroscience, 9, 4227–4235.Find this resource:

Marinesco, S., & Carew, T. J. (2002). Serotonin release evoked by tail nerve stimulation in the CNS of Aplysia: Characterization and relationship to heterosynaptic plasticity. Journal of Neuroscience, 22, 2299–2312.Find this resource:

Martinez-Rubio, C., Serrano, G. E., & Miller, M. W. (2009). Localization of biogenic amines in the foregut of Aplysia californica: Catecholaminergic and serotoninergic innervation. Journal of Comparative Neurology, 514, 329–342.Find this resource:

Matsumoto, Y., Sandoz, J. C., Devaud, J. M., Lormant, F., Mizunami, M., & Giurfa, M. (2014). Cyclic nucleotide-gated channels, calmodulin, adenylyl cyclase, & calcium/calmodulin-dependent protein kinase II are required for late, but not early, long-term memory formation in the honeybee. Learning and Memory, 21, 272–286.Find this resource:

Matzel, L. D., Talk, A. C., Muzzio, I. A., & Rogers, R. F. (1998). Ubiquitous molecular substrates for associative learning and activity-dependent neuronal facilitation. Review in Neuroscience, 9, 129–167.Find this resource:

McComb, C., Rosenegger, D., Varshney, N., Kwok, H. Y., & Lukowiak, K. (2005). Operant conditioning of an in vitro CNS-pneumostome preparation of Lymnaea. Neurobiology Learning and Memory, 84, 9–24.Find this resource:

McGuire, S. E., Le, P. T., & Davis, R. L. (2001). The role of Drosophila mushroom body signaling in olfactory memory. Science, 293, 1330–1333.Find this resource:

Menzel, R. (2012). The honeybee as a model for understanding the basis of cognition. Nature Review in Neuroscience, 13, 758–768.Find this resource:

Michel, M., Green, C. L., Gardner, J. S., Organ, C. L., & Lyons, L. C. (2012). Massed training-induced intermediate-term operant memory in Aplysia requires protein synthesis and multiple persistent kinase cascades. Journal of Neuroscience, 32, 4581–4591.Find this resource:

Michel, M., Green, C. L., & Lyons, L. C. (2010). PKA and PKC are required for long-term but not short-term in vivo operant memory in Aplysia. Learning and Memory, 18, 19–23.Find this resource:

Michel, M., Kemenes, I., Müller, U., & Kemenes, G. (2008). Different phases of long-term memory require distinct temporal patterns of PKA activity after single-trial classical conditioning. Learning and Memory, 15, 694–702.Find this resource:

Miyashita, T., Oda, Y., Horiuchi, J., Yin, J. C., Morimoto, T., & Saitoe, M. (2012). Mg2+ block of Drosophila NMDA receptors is required for long-term memory formation and CREB-dependent gene expression. Neuron, 74, 887–898.Find this resource:

Mozzachiodi, R., & Byrne, J. H. (2010). More than synaptic plasticity: Role of nonsynaptic plasticity in learning and memory. Trends in Neuroscience, 33, 17–26.Find this resource:

Mozzachiodi, R., Lechner, H. A., Baxter, D. A., & Byrne, J. H. (2003). In vitro analog of classical conditioning of feeding behavior in Aplysia. Learning and Memory, 10, 478–494.Find this resource:

Mpitsos, G. J., & Collins, S. D. (1975). Learning: Rapid aversive conditioning in the gastropod mollusk Pleurobranchaea. Science, 188, 954–957.Find this resource:

Murphy, G. G., & Glanzman, D. L. (1996). Enhancement of sensorimotor connections by conditioning-related stimulation in Aplysia depends upon postsynaptic Ca2+. Proceedings of the National Academy of Sciences USA, 93, 9931–9936.Find this resource:

Murphy, G. G., & Glanzman, D. L. (1997). Mediation of classical conditioning in Aplysia californica by long-term potentiation of sensorimotor synapses. Science, 278, 467–471.Find this resource:

Muzzio, I. A., Talk, A. C., & Matzel, L. D. (1997). Incremental redistribution of protein kinase C underlies the acquisition curve during in vitro associative conditioning in Hermissenda. Behavioral Neuroscience, 111, 739–753.Find this resource:

Nakaya, T., Kawahara, S., Watanabe, S., Lee, D.-S., Suzuki, T., & Kirino, Y. (2001). Identification and expression of a novel gene in odour-taste associative learning in the terrestrial slug. Genes to Cells, 6, 43–56.Find this resource:

Nargeot, R., Baxter, D. A., & Byrne, J. H. (1997). Contingent-dependent enhancement of rhythmic motor patterns: An in vitro analog of operant conditioning. Journal of Neuroscience, 17, 8093–8105.Find this resource:

Nargeot, R., Baxter, D. A., & Byrne, J. H. (1999a). In vitro analog of operant conditioning in Aplysia. I. Contingent reinforcement modifies the functional dynamics of an identified neuron. Journal of Neuroscience, 19, 2247–2260.Find this resource:

Nargeot, R., Baxter, D. A., & Byrne, J. H. (1999b). In vitro analog of operant conditioning in Aplysia. II. Modifications of the functional dynamics of an identified neuron contribute to motor pattern selection. Journal of Neuroscience, 19, 2261–2272.Find this resource:

Nargeot, R., Baxter, D. A., Patterson, G. W., & Byrne, J. H. (1999c). Dopaminergic synapses mediate neuronal changes in an analogue of operant conditioning. Journal of Neurophysiology, 81, 1983–1987.Find this resource:

Nargeot, R., Le Bon-Jego, M., & Simmers, J. (2009). Cellular and network mechanisms of operant learning-induced compulsive behavior in Aplysia. Current Biology, 19, 975–984.Find this resource:

Nargeot, R., Petrissans, C., & Simmers, J. (2007). Behavioral and in vitro correlates of compulsive-like food-seeking induced by operant conditioning in Aplysia. Journal of Neuroscience, 27, 8059–8070.Find this resource:

Nargeot, R., & Simmers, J. (2012). Functional organization and adaptability of a decision-making network in Aplysia. Frontiers in Neuroscience, 6, 113.Find this resource:

Nighorn, A., Healy, M. J., & Davis, R. L. (1991). The cyclic AMP phosphodiesterase encoded by the Drosophila dunce gene is concentrated in the mushroom body neuropil. Neuron, 6, 455–467.Find this resource:

Nikitin, E. S., & Balaban, P. M. (2000). Optical recording of odor-evoked responses in the olfactory brain of the naïve and aversively trained terrestrial snails. Learning and Memory, 7, 422–432.Find this resource:

Nikitin, E. S., Balaban, P. M., & Kemenes, G. (2013). Nonsynaptic plasticity underlies a compartmentalized increase in synaptic efficacy after classical conditioning. Current Biology, 23, 614–619.Find this resource:

Nikitin, E. S., Vavoulis, D. V., Kemenes, I., Marra, V., Pirger, Z., Michel, M., … Kemenes, G. (2008). Persistent sodium current is a nonsynaptic substrate for long-term associative memory. Current Biology, 18, 1221–1226.Find this resource:

Ocorr, K. A., Walters, E. T., & Byrne, J. H. (1985). Associative conditioning analog selectively increases cAMP levels of tail sensory neurons in Aplysia. Proceedings of the National Academy of Sciences USA, 82, 2548–2552.Find this resource:

Pavlov, I. P. (1927). Conditioned reflexes. London, UK: Oxford University Press.Find this resource:

Qin, H., Cressy, M., Li, W., Coravos, J. S., Izzi, S. A., & Dubnau, J. (2012). Gamma neurons mediate dopaminergic input during aversive olfactory memory formation in Drosophila. Current Biology, 22, 608–614.Find this resource:

Quinn, W. G., Harris, W. A., & Benzer, S. (1974). Conditioned behavior in Drosophila melanogaster. Proceedings of the National Academy of Sciences USA, 71, 708–712.Find this resource:

Ratté, S., & Chase, R. (2000). Synapse distribution of olfactory interneurons in the procerebrum of the snail Helix aspersa. Journal of Comparative Neurology, 417, 366–384.Find this resource:

Rescorla, R. A. (1967). Pavlovian conditioning and its proper control procedures. Psychology Review, 74, 71–80.Find this resource:

Rescorla, R. A. (1987). A Pavlovian analysis of goal-directed behavior. American Psychologist, 42, 119–129.Find this resource:

Rescorla, R. A., & Solomon, R. L. (1967). Two-process learning theory: Relationships between Pavlovian conditioning and instrumental learning. Psychology Review, 74, 151–182.Find this resource:

Reyes, F. D., Mozzachiodi, R., Baxter, D. A., & Byrne, J. H. (2005). Reinforcement in an in vitro analog of appetitive classical conditioning of feeding behavior in Aplysia: Blockade by a dopamine antagonist. Learning and Memory, 12, 216–220.Find this resource:

Ribeiro, M. J., Serfozo, Z., Papp, A., Kemenes, I., O'Shea, M., Yin, J. C., Benjamin, P. R., & Kemenes, G. (2003). Cyclic AMP response element-binding (CREB)-like proteins in a molluscan brain: Cellular localization and learning-induced phosphorylation. European Journal of Neuroscience, 18, 1223–1234.Find this resource:

Roberts, A. C., & Glanzman, D. L. (2003). Learning in Aplysia: Looking at synaptic plasticity from both sides. Trends in Neuroscience, 26, 662–670.Find this resource:

Rosenegger, D., & Lukowiak, K. (2010). The participation of NMDA receptors, PKC, & MAPK in the formation of memory following operant conditioning in Lymnaea. Molecular Brain, 3, 24.Find this resource:

Sahley, C., Gelperin, A., & Rudy, J. W. (1981). One-trial associative learning modifies food odor preferences of a terrestrial mollusc. Proceedings of the National Academy of Sciences, 78, 640–642.Find this resource:

Scheibenstock, A., Krygier, D., Haque, Z., Syed, N., & Lukowiak, K. (2002). The Soma of RPeD1 must be present for long-term memory formation of associative learning in Lymnaea. Journal of Neurophysiology, 88, 1584–1591.Find this resource:

Schwarz, M., Feldman, E., & Susswein, A. J. (1991). Variables affecting long-term memory of learning that a food is inedible in Aplysia. Behavioral Neuroscience, 105, 193–201.Find this resource:

Schwarz, M., Markovich, S., & Susswein, A. J. (1988). Parametric features of inhibition of feeding in Aplysia by associative learning, satiation, and sustained lip stimulation. Behavioral Neuroscience, 102, 124–133.Find this resource:

Schwarz, M., & Susswein, A. J. (1986). Identification of the neural pathway for reinforcement of feeding when Aplysia learn that food is inedible. Journal of Neuroscience, 6, 1528–1536.Find this resource:

Sekiguchi, T., Furudate, H., & Kimura, T. (2010). Internal representation and memory formation of odor preference based on oscillatory activities in a terrestrial slug. Learning and Memory, 17, 372–380.Find this resource:

Sieling, F., Bédécarrats, A., Simmers, J., Prinz, A. A., & Nargeot, R. (2014). Differential role of nonsynaptic and synaptic plasticity in operant conditioning-induced compulsive behavior. Current Biology, 24, 941–950.Find this resource:

Skinner, B. F. (1938). The behavior of organisms. R. M. Elliott (Ed.), New York, NY: Appleton-Century-Crofts, Inc.Find this resource:

Skoulakis, E. M., Kalderon, D., & Davis, R. L. (1993). Preferential expression in mushroom bodies of the catalytic subunit of protein kinase A and its role in learning and memory. Neuron, 11, 197–208.Find this resource:

Spencer, G. E., Syed, N. I., & Lukowiak, K. (1999). Neural changes after operant conditioning of the aerial respiratory behavior in Lymnaea stagnalis. Journal of Neuroscience, 19, 1836–1843.Find this resource:

Staras, K., Kemenes, G., & Benjamin, P. R. (1999). Cellular traces of behavioral classical conditioning can be recorded at several specific sites in a simple nervous system. Journal of Neuroscience, 19, 347–357.Find this resource:

Susswein, A. J., Hurwitz, I., Thorne, R., Byrne, J. H., & Baxter, D. A. (2002). Mechanisms underlying fictive feeding in Aplysia: Coupling between a large neuron with plateau potentials activity and a spiking neuron. Journal of Neurophysiology, 87, 2307–2323.Find this resource:

Susswein, A. J., & Schwarz, M. (1983). A learned change of response to inedible food in Aplysia. Behavioral and Neural Biology, 39, 1–6.Find this resource:

Susswein, A. J., Schwarz, M., & Feldman, E. (1986). Learned changes of feeding behavior in Aplysia in response to edible and inedible foods. Journal of Neuroscience, 6, 1513–1527.Find this resource:

Syed, N. I., Bulloch, A. G., & Lukowiak, K. (1990). In vitro reconstruction of the respiratory central pattern generator of the mollusk Lymnaea. Science, 250, 282–285.Find this resource:

Takeda, K. (1961). Classical conditioned response in the honey bee. Journal of Insect Physiology, 6, 168–179.Find this resource:

Takigami, S., Sunada, H., Lukowiak, K., Kuzirian, A. M., Alkon, D. L., & Sakakibara, M. (2014). Protein kinase C mediates memory consolidation of taste avoidance conditioning in Lymnaea stagnalis. Neurobiology Learning and Memory, 111, 9–18.Find this resource:

Tomchik, S. M., & Davis, R. L. (2009). Dynamics of learning-related cAMP signaling and stimulus integration in the Drosophila olfactory pathway. Neuron, 64, 510–521.Find this resource:

Tsydzik, V., & Wright, N. J. (2009). Dopamine modulation of the in vivo acetylcholine response in the Drosophila mushroom body. Developmental Neurobiology, 69, 705–714.Find this resource:

Ueno, K., Suzuki, E., Naganos, S., Ofusa, K., Horiuchi, J., & Saitoe, M. (2017). Coincident postsynaptic activity gates presynaptic dopamine release to induce plasticity in Drosophila mushroom bodies. Elife, 6.Find this resource:

Walters, E. T., & Byrne, J. H. (1983). Associative conditioning of single sensory neurons suggests a cellular mechanism for learning. Science, 219, 405–408.Find this resource:

Walters, E. T., Carew, T. J., & Kandel, E. R. (1979). Classical conditioning in Aplysia californica. Proceedings of the National Academy of Sciences USA, 76, 6675–6679.Find this resource:

Walters, E. T., Carew, T. J., & Kandel, E. R. (1981). Associative learning in Aplysia: Evidence for conditioned fear in an invertebrate. Science, 11, 504–506.Find this resource:

Walters, E. T. (1989). Transformation of siphon responses during conditioning of Aplysia suggests a model of primitive stimulus-response association. Proceedings of the National Academy of Sciences USA, 86, 7616–7619.Find this resource:

Watanabe, S., Kirino, Y. & Gelperin, A. (2008). Neural and molecular mechanisms of microcognition in Limax. Learning and Memory, 15, 633–642.Find this resource:

Woollacott, M., & Hoyle, G. (1977). Neural events underlying learning in insects: Changes in pacemaker. Proceedings of the Royal Society of London B: Biology Sciences, 195, 395–415.Find this resource:

Xia, S., Miyashita, T., Fu, T. F., Lin, W. Y., Wu, C. L., Pyzocha, L., … Chiang, A. S. (2005). NMDA receptors mediate olfactory learning and memory in Drosophila. Current Biology, 15, 603–615.Find this resource:

Yabumoto, T., Takanashi, F., Kirino, Y., & Watanabe, S. (2008). Nitric oxide is involved in appetitive but not aversive olfactory learning in the land mollusk Limax valentianus. Learning and Memory, 15, 229–232.Find this resource:

Yu, A. J, & Rankin, C. H. (2017). Non-associative learning in Invertebrates. In J. H. Byrne (Ed.), The Oxford handbook of invertebrate neurobiology. New York, NY: Oxford University Press.Find this resource:

Yu, D., Ponomarev, A., & Davis, R. L. (2004). Altered representation of the spatial code for odors after olfactory classical conditioning; memory trace formation by synaptic recruitment. Neuron, 42, 437–449.Find this resource: