Context Dependence

An important source of behavioral richness, flexibility, and adaptiveness is context dependence. Animals express different behavioral patterns under (p. 665) different internal and external contexts. These behavioral patterns can be either rather stereotyped innate sensory-motor routines (as, for example, in the starvation/satiation distinction in Drosophila; Su & Wang, 2014) or involve learned complex behaviors. Contexts control sensory processing, prepare for adapted motor expression, promote attentional selection, and induce goal directedness by selective expectation. In this sense, context dependence lies at the heart of intentionality in a broad sense. Insects do not differ in this respect from other animals. In the honeybee context, dependence includes age dependence, social conditions, and individual experiences. Foraging for food is an interesting case. Bees easily learn different combinations of time of the day, location, color, and odor. In the natural foraging cycle, honeybees do not need to be hungry to search for nectar or pollen and to be rewarded by food (Menzel & Giurfa, 2001).

In the laboratory, proboscis extension response (PER) conditioning is applied to test how the response to the conditioned odor (the cue) is changed when it is announced by a learned visual context (Gerber & Smith, 1998). Bees learn context-dependent cue rewarding rather well. Interestingly, bees do not learn to respond with proboscis extension to the visual stimulus alone, but neural responses are predictive for the learned visual context (see later). Bees also solve a transwitching task and a cue/context reversal task (Hussaini & Menzel, 2013). In these problems, an animal is trained differentially with two olfactory stimuli, A and B, and with two different visual contexts, C1 and C2. When C1 is available, stimulus A is rewarded while stimulus B is not (A+ vs. B–), while it is the opposite (A– vs. B+) with C2. Focusing on the elements alone does not allow solving the problem because each element (A, B) appears equally as often rewarded and nonrewarded. Each context (C1, C2) is, in the same way, simultaneously rewarded and nonrewarded, depending on its association with A or B. Animals have, therefore, to learn that C1 and C2 define the valid contingency.

An example is given in Fig. 28.1. Differential conditioning was applied with C+O+ (color context C+, odor cue O+) as the rewarded condition, and a C–O– as the unrewarded condition. C+ versus C– was conditioned alone prior to the context/cue conditioning. In the tests the various combinations as well as the single stimuli were tested, and both the response probability and the response latency were evaluated (Filla & Menzel, 2015). The C+O+ combination had the highest probability and most rapid responding, indicating that a learned context leads to an expectation of the learned cue.

Internal context conditions are not as well controlled experimentally, but levels of satiation are well known to influence reward-dependent learning, attentiveness, sensory sensitivity, and motor performance.

Contexts and cues will be processed independently in the periphery if they belong to different sensory modalities and need to converge in some form for referenced encoding. A major part of neural convergence of highly processed sensory information is the mushroom body (MB) in the insect brain. It is thus likely that intrinsic and extrinsic neurons of the MB will be involved in processing of context and cue. Furthermore, the MB receives ascending input from both body-related sensory input and modulatory networks most likely being involved in encoding body states. Therefore, neural activity of the MB and its extrinsic neurons likely represents the substrate for context-dependent attentional selection, goal directedness, and expectation. Neurophysiological studies supporting this view will be presented next.

Selective Attention

Selective attention requires the ability to focus perceptual mechanisms on a particular stimulus and to actively process this information while ignoring nonrelevant stimuli (Zentall, 2005). Innate or learned search images facilitate detection of relevant stimuli in the environment, focusing perception on specific stimulus configurations as a way to more efficiently forage and avoid predation. Naïve bees exhibit innate preferences for biologically relevant floral cues such as colors (Giurfa et al., 1995; Gumbert, 2000) or bilateral symmetry (Rodriguez et al., 2004). In discriminative learning, selective attention is cued by the nonrelevant stimulus leading to progressively better discrimination of, for example, a trained color from colors that are perceptually close. It is relatively easy to train honeybee foragers using a single or a set of colored targets and to test their response to stimuli they had not experienced during the course of training (Martínez-Harms et al., 2014) took advantage of the quantitative model of color vision in honeybees and applied the paradigm of peak shift in color discrimination. They found that bees preferred the rewarded color (S+) more when the response toward S+ was tested against a novel color whose locus was located closer to S– than S+. In contrast, bees tested with (p. 666) a novel color located farther away from S– relative to S+ shifted their response away from S– toward the novel color. These results can be interpreted along the selective attention hypothesis such that the bees gradually learned to attend to the spectral dimension in which the discriminative stimuli differed. An attentional account also applies to pattern discrimination experiments in which bees do or do not discriminate between the same two patterns depending on the kind of training used, that is, absolute or differential conditioning (Giurfa et al., 1999). No neural correlates have yet been identified for these forms of attentional effects of sensory discrimination.

Click to view larger

Figure 28.1 Context-dependent proboscis extension response (PER) conditioning. (A) Bees were harnessed in tubes and differentially conditioned to two different olfactory cues during two different color contexts. (B) During the first part of the experiment (upper part of B) the visual contexts (C+ and C–) were conditioned alone by forward pairing of C+ with sucrose reward and C– without reward. The animals responded neither to the C+ nor to the C–. In the second part the odor cue O+ was paired with reward during C+, and the odor cue O– was not warded during C–. In the following extinction tests all combinations and single stimuli were presented without reward and the probability of PER evaluated. C and D give the results of the extinction tests for the probability of PER (C) and the latency between stimulus onset and PER.

(From Filla & Menzel, 2015.)

Spatial cues predicting the location of a target are commonly used as an experimental manipulation of visual attention. Response times are reduced or accuracy increased when the target appears at the cued location. Eckstein et al. (2013) compared spatial cueing effects in humans, monkeys, and honeybees. A simple modified two-alternative, forced-choice cueing paradigm was used for all three species. The target appeared in one of two locations with equal probability and a highly visible cue that appeared randomly at one of the two locations indicating 75% validity of the location of the target. Monkeys and humans indicated their decision by making an eye movement toward the target location. (p. 667) Honeybees’ selection was based on the bee circling close to the box, resting or entering one of two box choices. Humans showed a larger cueing effect than monkeys and bees showed the smallest effect. It was concluded that the cueing effect of the honeybees allows rejecting models in which the bees ignored the cue, followed the cue, or adopted a probability matching strategy. Again, no neural substrate has been identified so far.

Rule Learning

Rule learning requires attention to the stimuli representing the common feature of a series of targets. A frequently applied paradigm is delayed matching to sample (DMTS) and the delayed nonmatching to sample (DNMTS). In DMTS, animals are presented with a sample and then with a set of stimuli, one of which is similar to the sample and which is then reinforced. Honeybees foraging in a Y-maze learn both rules (Giurfa et al., 2001). Bees trained with colors and presented in transfer tests with gratings that they had not experienced before solved the problem and chose the grating identical to the sample at the entrance of the maze. Similarly, bees trained with the gratings and tested with colors in transfer tests also solved the problem and chose the novel color corresponding to that of the sample grating at the maze entrance. Transfer was not limited to different visual stimuli (pattern vs. color), but it could also operate between different sensory domains (olfaction and vision). Furthermore, bees also mastered a DNMTS task, thus showing that they also learned a principle of difference between stimuli. These results were verified more recently. Furthermore, the working memory necessary for the comparison between sample and cue was found to be around 5 seconds (Zhang et al., 2005).

Another paradigm used to study rule learning in olfactory learning takes advantage of PER conditioning, an experimental condition that lends itself to neurophysiological studies. Stimulating the two antennae separately leads to side-specific olfactory learning (Sandoz & Menzel, 2001). Bees were differentially conditioned with an odor A+ versus an odor B– on one antenna and with A– versus B+ on the other. This discrimination can also be seen as a case of contextual learning, since the context of each antennal side (left vs. right) determines the contingency of the stimuli. This and two other paradigms were used to test MB-ablated bees and to determine whether intact mushroom bodies are necessary to solve nonelemental olfactory discriminations (Komischke et al., 2005). Bees with unilateral mushroom body lesions could not solve an unambiguous double discrimination (A+, B– on one antenna, C+, D– on the other; A+B–/C+D–), whereas they could solve at least one of both discriminations of an ambiguous problem (A+, B– on one antenna, A–, B+ on the other; A+B–/A–B+). In the latter case, they solved the discrimination proposed to their intact brain side. Nonablated bees could learn both side-specific discriminations. When odorants were delivered simultaneously to both antennae (A+B–C+D–), ablated bees learned slower than normal bees. Thus, in all three cases, the unilateral loss of a median calyx affected olfactory learning. It was proposed that MB ablations could have an effect on the information exchange between the brain hemispheres.

Bees extract the change of reward if they experience a rising or falling trend of reward quantity indicating an extraction of a rule that governs long-term reward expectations (Gil et al., 2007). The term expectation refers here to reward “incentive,” which can be related to an internal representation determined by previous reward experiences. The incentive values are learned, and they modulate current performances. Specifically, five experimental series were performed with increasing, decreasing, or constant volumes of sugar solution throughout all visits at multiple feeding palaces in a patch. Twenty-four and 48 hours after training, the bees’ foraging behavior at the patch was tested without reward. Bees from the increasing series assigned more time to flower inspection than those from the decreasing series. They did so neither because they had more strongly associated the related predicting signals nor because they were fed more or faster. These results document that honeybees develop reward expectations in the long term. A neural correlate of reward expectation will be discussed later in the context of the reward neuron VUMmx1.

Navigation

Before embarking on foraging flights, honeybees explore the environment around the hive. They learn the immediate and more distant surroundings of the hive, and they calibrate their sun compass and their distance measurement system (Degen et al., 2015). They use the memories formed during this process to perform subsequent foraging flights effectively and to communicate important locations in the terrain (Degen et al., in press). We used a harmonic radar system to follow the experimental bee’s flight path (Menzel et al., 2005). Bees start foraging only after they had systematically explored (p. 668) the landscape around their hive. Foragers trained to a feeding site and then transported to a distant release site within the explored area initially fly along a vector they would have taken had they not been transported (vector flight), and they then performed searching flights (search flight) before flying directly back to the hive (return flight) or to the feeder. These return flights were performed over distances that rule out the possibility that the bee was able to see the goal when setting out on its return flight. They also could not use the skyline of the horizon for aiming into the hive or the feeder. Furthermore, the return flights were performed all around the hive and not just from a specific direction. The return flights fulfill the requirement of a novel shortcut to the intended goal (Tolman, 1948). Thus, it has been concluded that exploratory and foraging flights lead to map-like memory structure that allows the animal to select goals, choose between them, and then fly toward them without having sensory access to the goal (cognitive map) (Menzel & Greggers, 2015) (see also the controversy about this interpretation: Cheung et al., 2014; Cheeseman et al., 2014a, 2014b).

Because bees communicate the outbound vector toward a feeding place or a new nest site, one can ask next whether an experienced feeding site and a dance indicated place are represented in a common navigational memory. The experiments were as follows (Menzel et al., 2011): Experienced foragers were trained to the location FD (Fig. 28.2). After 2–3 days they no longer received food at that location. Because these experiments are performed in autumn and additional natural feeding sites were not present in the landscape, after several unsuccessful visits to the location, the trained bees stayed in the hive. There they followed a dancer indicating the location FT. When these animals left the hive, their flight behavior was observed by radar. Some animals flew first to FD, others to FT. This depended on the number of waggle runs they have observed. If they followed many (>15) waggle runs, then they flew first to FD; if they followed only a few, then they flew first to FT. Fig. 28.2B shows three exemplary flight paths. For distances of 300 m between the hive and FD or FT, both for the 30-degree and for the 60-degree angle between the routes “hive—FD” and “hive—FT,” the bees flew from FD to FT or from FT to FD, depending on where they arrived first. For distances of 650 m between the hive and FD or FT, they flew this direct route only with an angle of 30 degrees between the routes “hive—FD” and “hive—FT.” So, if the distance between the locations FD and FT was the same as the distance back to the hive, then they flew this route only for shorter distances and not for longer distances. Note that this route from FD to FT (or vice versa from FT to FD) was not flown previously by the animals nor was it communicated in the dance; thus, it was a novel shortcut. Bees that flew first to the trained location after following only a few dance rounds still could perform a shortcut to the dance-indicated location. The information for the dance-indicated location was sufficient for the decision to aim for it when the animal was out in the field (and experienced that there was no food available anymore) but not to motivate it to fly directly to this location. The decision to steer toward one of the two locations obviously depended on the expected value at the respective locations: Initially this value was high for FT but then it was degraded by experience and became lower than that of FD. These results not only support the view that learned and dance-communicated locations are embedded in the same spatial memory structure but more important that bees make decisions with respect to the expected outcome without access to the stimuli emanating from the respective locations, taking into account the current state of the respective memories.

Social Communication

Karl von Frisch discovered that bees perform a waggle dance that communicates to the hive mates the direct flight path to the indicated goal (a feeding site or a new nesting site) (von Frisch, 1967). Thus, the waggle dance encodes the direct flight path in a symbolically communicated vector if such a direct flight is possible to fly. If not, then the indicated vector encodes the direction toward the goal but the actual length of the flown distance. The flight direction in relation to the current position of the sun is given in the angle of the waggle phase relative to gravity on the vertical comb in the dark hive. The distance from the hive to the indicated location is measured visually and encoded in the number of waggle movements (and other associated parameters such as waggle time or the time taken to complete a rotation). It is remarkable that this kind of symbolic communication of a location is not known to exist in any other animal. In our context, it is interesting to ask what is being communicated: Is it the outbound flight vector (as I have cautiously called it previously) or rather the location? In the first case, the attending bee only receives information about the direct flight path from the hive to the communicated goal and would have to follow this (p. 669) (p. 670) instruction without referring to its own memory of the landscape. This would mean that a flight instruction is given. In the second case, the bee would communicate a place which the attending bee recalls from its memory of the landscape during the communication process. A place is defined not only by its spatial location but also by a series of characteristics that both the dancing bee and the attending bee must have stored previously. These include, for instance, the spatial reference to landmarks, the route to the place (as shown by Karl von Frisch, the bee might have to fly around a mountain spur), the fact that the attending bee might already have experienced, or indeed not have experienced, the place, that it has (or does not have) experience of the odor and/or taste communicated during the dance, and other items of information that rely on whether the attending bee introduces specific experiences of its own into the communication process. If the attending bee has gained experience of the indicated location (e.g., how rich the feeding site is, what odor it has, how the flower is to be manipulated, whether nectar or pollen is to be collected, etc.), then it will make its decision about whether to follow the dance information dependent on that experience. In such a case, the attending bee would have certain expectations not only regarding the place but also regarding its properties and the landscape characteristics it can expect to encounter en route to and around that location. From these contrasting formulations it becomes clear that completely different assumptions can be made about the cognitive processes involved in the communication process, decision making, and subsequent navigation. It is difficult to avoid an assumption of intentionality involved in this communication process because practically all the signals attached to the performance of the flight are not accessible during the communication process inside the dark hive. A striking example of evaluation by the receiving bee is the stop signal produced by scouts in a swarm. Scouts that had discovered a superior nest side inhibit another scout bee’s efforts to continue advertising for an inferior nest site without actually inspecting that inferior nest site (Seeley et al., 2012). The evaluation is based merely on the symbolic communication process, suggesting a mental state in which a comparison is made between transformed own experience and that of another animal.

Click to view larger

Figure 28.2 Novel shortcut flights between learned location and dance indicated location. (A) Design of the experiment. The animals were first trained to FT either 600 m away from the hive (FT30°, FT60°, outer circle 600 m) or 300 m away from the hive (FT30°, FT60°, inner circle 300 m). The two bees danced for the locations FD. These were either 30° or 60° off from the trained locations as seen from the hive for both distances (300 m, 600 m). (B) Three representative examples for novel shortcuts. The two figures on the left give the result for the 300 m distance (30° and 60° angular distance between FT and FD), and the right figure gives the result for the 600 m distance (30°).

Memory Reactivation During Sleep

During sleep the brain is disconnected from the external world both at the sensory and motor side. This is the stage in which the brain somehow talks to itself, and the effects of internal processing might become particularly obvious. Bees, like other insects, sleep (Kaiser & Steiner-Kaiser, 1983). Since then, the list of other invertebrate species in sleep research has grown and now includes Drosophila melanogaster, scorpions, cockroaches, and Caenorhabditis elegans (Hendricks et al., 2000; Shaw et al., 2000; Sauer et al., 2003). In honeybees, retention scores improve if sleep is not disturbed, while memory formation after extinction learning—but not after acquisition learning—is selectively reduced if animals are prevented from sleeping (Hussaini et al., 2009). In sleep-deprived honeybees, waggle-dance precision is impaired (Klein et al., 2010) and newly acquired navigation memory is compromised when night sleep is interrupted (Beyaert et al., 2012). We found recently that this consolidation process can be enhanced by stimulating sleeping bees with the context odor to which they were exposed when they learned to associate a temperature stimulus with reward (Zwaka et al., 2015). Presentation of the context odor during wake phases or novel odors during sleep does not enhance memory retention the next day. Furthermore, single-trial learning that does not lead to 24-hour memory is converted to stable memory if the context stimulus is applied during sleep phases. In humans, memory consolidation can be triggered during phases of slow-wave sleep by presenting a context odor during slow-wave sleep that had been present during learning (Vorster & Born, 2015). These results reveal that phases in honeybees have the potential to prompt memory consolidation, just as they do in humans. Memory consolidation in mammals during sleep is suggested to function via reactivation of a recently learned memory trace (Wilson & McNaughton, 1994; Peigneux et al., 2004; Ji & Wilson, 2007). Sleep supports consolidation of memory in the bee, but whether reactivation of the memory trace is essential is not known yet. The discovery of such a process would be convincing evidence for internal processing of the brain. Our attempts to collect support for this interpretation by recording from mushroom body output neurons have provided only hints so far.

Models of Intentionality

Concepts about how the brain may accomplish tasks requiring some form of intentionality (p. 671) clearly differ markedly between elementary and cognitive accounts. In the first case (Fig. 28.3), sensory-motor routines are assumed, which each work in isolation. They link certain levels of sensory integration to special motor programs, with each of these links equipped with a specialized memory. Elementary explanations consist of lists of separate submechanisms. Here, the term tool box has become established in the academic literature (Wehner, 1987), which suggests that, in a certain behavioral situation, the animal has only one neural mechanism at its disposal to perform the behavior, and successive behaviors activate the respective tools. Hence, the challenge for the neurobiologist searching for elementary tasks is to detect the relevant submechanisms. Surprisingly, so far, no evidence for such sensory-motor routines has been found in the insect brain. Even an “isolated” subfunction such as release of motor patterns induced by the cricket songs (Hedwig, 2006) or orientation to the pattern of linearly polarized light, which was considered a prime example of a solution to the problem already on the level of the structure of the complex eye (Wehner, 1992), could not be traced back to such a tool box neural mechanism (Homberg et al., 2011; Trager & Homberg, 2011).

Click to view larger

Figure 28.3 Box models of brain functions. (A) The matched filter concept as applied to elementary cognitive functions in the insect brain. (B) A box model of cognitive functions of the brain.

(From Carruthers, 2006.)

The cognitive explanation postulates that the various sensory mechanisms that contribute to (p. 672) the particular behavioral phenomenon feed into a common neuronal representation of the experienced world (Fig. 28.4), and that it is this entity which controls intentional states. Neural correlates supporting such a concept of brain function exist only in rather limited form (see later discussion), because no neural processes in the insect brain have yet been identified that could be interpreted as mechanisms for decision making between imagined outcomes of own behavior. In all cases so far studied, the stimuli retrieving memory have been presented as in classical and operant conditioning. Furthermore, no neural correlates of, for example, observational learning have even been searched for, a behavioral condition in which neural mechanisms underlying expectation of future events without sensory input from these events should become particularly obvious.

The situation is different in studies of the mammalian brain. The hippocampus stores a cognitive map that results from exploratory learning, and the processes underlying space representation and planning of future paths are intensively studied, although not yet fully understood. The moment a relevant neural substrate is found for intentional states, the debate is decided, as it is, for example, with respect to the cognitive map concept when head direction cells, place cells, grid cells, and sequence cells are recorded in the hippocampus of the rat (O’Keefe & Nadel, 1978; Moser et al., 2008). In this respect, the navigation problem does not differ from other topics of research in behavioral biology, such as the question as to whether insects have expectations, make decisions, plan, pursue intentions, recognize themselves as individuals in a social community, judge other members of the social community in terms of their reliability, or whether they reflect on themselves and their own actions (Allen & Bekoff, 1997).

Click to view larger

Figure 28.4 The mushroom body in the honeybee brain and its global connections with other areas of the brain. AL, antennal lobe; CC, central complex; LC, lateral calyx; LO, lobula; MC, medial calyx; ME, medulla; PL, protocerebral lobe; SOG, subesophageal ganglion; α, alpha lobe; β, beta lobe.

The structural requirements for the processing of such cognitive faculties are multifold: convergence of multisensory inputs both from the external world and the body, control by modulatory networks, remoteness from premotor commands, multiple connections to other parts of the brain, including feedback loops to sensory integration centers and to its own input, a common memory store for multiple forms of experience. The mushroom body meets these requirements (Fig. 28.4). It lies in a parallel pathway, receives highly processed sensory information from all sensory modalities, is innervated by multiple modulatory networks, is necessary for adaptive behavior, stores memory traces, and is equipped with feedback loops to other brain sites, including its own input site. The direct pathways run from the sensory inputs via sensory neuropiles, the central complex (CC), or the lateral protocerebrum (lPC) to the descending premotor (p. 673) pathways (Menzel, 2013). This wiring principle suggests that the direct pathways are responsible for rapid, primarily stereotypical functions, and especially for innate functions and those that maintain the organism’s homeostasis. Based on these considerations, the search for the neural correlates of cognitive behavior in the honeybee should concentrate on the mushroom body and its ring neuropil.

Reward Expectation in the Reward Neuron VUMmx1

A particularly striking neuron in the bee brain is VUMmx1 (ventral unpaired median neuron of the maxillary neuromere 1), which serves the function of an appetitive value system for olfactory learning (Hammer, 1993). The VUMmx1 neuron belongs to a group of 15 ventral unpaired median neurons of the suboesophageal ganglion (Schroter et al., 2007). Most of the 15 neurons differ in the structure of their dendritic arborization, but one other neuron, VUMmd1, has the same branching pattern as VUMmx1. All 15 neurons appear to receive input in the suboesophageal ganglion from sucrose receptors both at the antennae and the proboscis. The dendrites of VUMmx1 (and VUMmd1) arborize symmetrically in the brain and converge with the olfactory pathway at three sites, the primary olfactory center, the antennal lobe (AL), the secondary olfactory integration area, the lip region of the mushroom bodies (MB), and the output region of the brain, the lateral horn (LH). VUMmx1 responds to sucrose solution (the US in conditioning) both at the antenna and the proboscis with long-lasting spike activity and to various visual, olfactory, and mechanosensory stimuli with low-frequency spike activity. Differential conditioning leads to an enhanced response of VUMmx1 to CS+ (conditioned stimulus, the forward paired odor) but not to CS– (the backward paired odor). This property could support second-order conditioning, a phenomenon well documented in PER conditioning. In this case, if a new CS is followed by the learned CS+, it will be associated transitively with VUMmx1 activation. Replacing the sucrose reward by intracellular depolarizing VUMmx1 shortly after the presentation of an odor leads to behavioral learning, but depolarization before the presentation of an odor does not, documenting that VUMmx1 excitation following the CS is sufficient for CS learning. Most important, if the US follows the presentation of the CS+, the response of VUMmx1 to the US is greatly reduced, and even inhibited. In contrast, the response of VUMmx1 to the US after the presentation of the CS– remains normal, indicating that an expected US blocks US-induced excitation (Hammer, 1997; Menzel & Giurfa, 2001). Thus, VUMmx1 codes the prediction error similarly to dopamine neurons in the ventral tegmentum of mammals (Schultz, 2006). This property requires an input from neurons coding the CS+, which in turn leads to prolonged inhibition. Such an input may come from inhibitory PCT neurons that terminate in the lip region of the MB calyx on dendrites of VUMmx1 (see later discussion). PCT neurons are octopamine immunoreactive and VUMmx1 dendrites contain octopamine receptors (Sinakevitch et al., 2013; Zwaka et al., submitted).

The property of VUMmx1 to code the expectation of reward may be the neural substrate for a prediction error as postulated by Rescorla and Wagner (1972). A critical test for the role of the prediction error is the phenomenon of blocking. When animals are conditioned to a mixture of two stimuli (AB+) containing a previously conditioned stimulus (A+), then their response to the second stimulus (B), when presented alone, is reduced compared to that of animals which had been conditioned to the mixture (AB+) alone and were subsequently also presented with B. Learning about the first stimulus (A+) “blocks” learning about the second stimulus (B+) during compound conditioning. These results demonstrate that the single identified neuron VUMmx1 is not only a sufficient neural substrate for the reinforcing function of the unconditioned stimulus sucrose in olfactory conditioning but also implements properties that allow explaining central features of associative learning, including second-order conditioning and blocking.

Memory Encoding in the Mushroom Body and Readout

The mushroom body as a whole appears to act as a recoding device, converting sensory information to value-based information (Menzel, 2012). The calyx comprises the dendrites of two major classes of intrinsic neurons (Kenyon cells [KC]) receiving input from second and higher-order sensory neurons, in the lip from olfactory and in the collar from visual, and in the basal ring from both olfactory and mechanosensory neurons (Fig. 28.5) (Mobbs, 1982; Rybak & Menzel, 1993; Gronenberg, 2001). A small and less separated region receives input from gustatory neurons (Schröter & Menzel, 2003). The approximately140,000 KC synapse onto a smaller number of extrinsic neurons (MBENs, few hundred). Overall, the connectivity at the input (p. 674) site (calyx) is dominated by massive divergence. At the level of the single KC, however, multiple olfactory, visual, mechanosensory, and gustatory neurons converge. Both divergence and convergence form a matrix-like pattern of connectivity that codes multiple aspects of the sensory world in a highly dimensional way (Fig. 28.5). Synaptic contacts are partly understood for the lip region where approximately 8–12 spines of KCs are postsynaptic to one presynaptic bouton from olfactory projection neurons and putative inhibitory inputs from recurrent neurons of the protoceribral calycal tract (PCT). The VUMmx1 neuron is likely to be pre- and postsynaptic to PCT neurons, but possibly also to KCs. KC require coincident input from several sensory neurons to produce a small number of action potentials (Szyszka et al., 2005; Farkhooi et al., 2013). Associative plasticity of these synaptic contacts stores the memory trace in stimulus-specific changes of synaptic efficiency (Szyszka et al., 2008; Yamagata and Haenicke, personal communication).

Click to view larger

Figure 28.5 A model of the mushroom body (MB) in the honeybee brain. Highly processed sensory input from the visual (Vis.), olfactory (Olf.) and mechanosensory (Mec.) (also gustatory, not shown) modalities reach the input site, the Calyx, of the MB and both diverge and converge on a large number of intrinsic neurons, the Kenyon cells (KC). The connectivity is characterized by the high dimensionality of multiple sensory components and by a matrix-like memory. The calyx receives input also from the reward neurons VUMmx1 (VUM) and multiple modulatory inputs (not shown). Furthermore, inhibitory recurrent neurons in the protoceribral calycal tract (PCT) project into the calyx. MB extrinsic neurons (MB ENs) originate predominantly in the lobes (here only one lobe, the alpha lobe is shown) and project to other regions within the brain (see also Fig. 28.4). ENs differ with respect to the compartments from which they receive input from the KCs. Connectivity in the lobes is characterized by high convergence on to the rather small number of ENs. One identified EN is the PE1 neuron.

Each axon of KCs splits into two collaterals halfway along the peduncle forming the alpha and the beta lobe. The lobes are the major output regions of the MB. Here the large number of densely packed KCs converges on a rather small number (a few hundred) of MB extrinsic neurons (ENs) (Fig. 28.5). ENs are categories according to the location of their somata (Fig. 28.6A). Most of these groups contain approximately 70 neurons as judged by the number of somata (one exception: A5 comprises only four neurons). Four main dendritic target areas were found: (1) the ring neuropil of the α lobe to which all α lobe ENs project at least with parts of their dendrites, (2) the lateral horn (LH) (A4 and PE1) and optical tubercle (A5 and A7), (3) the contralateral protocerebrum (A6 and A7), and (4) the feedback neurons to the calyx (A3). The multiplicity of connections established by these ENs makes it very likely that each group serves a different function. Since many of these ENs receive input across the modality-specific regions of the MB (Rybak & Menzel, 1993), it is not surprising that they respond to a large range of sensory stimuli indicating a different coding scheme than the highly specific combinatorial sensory code at the input of the MB (Menzel, 2013). The pattern of convergence on ENs is not understood, but a separation of compartments within the lobes as seen in the Drosophilaγ-lobe has not been detected. One large EN, the pedunculus extrinsic neuron #1 (PE1, Fig. 28.6B), offers the unique possibility to repeatedly record from the same identified neuron during olfactory PER conditioning (see later discussion). (p. 675)

Click to view larger

Figure 28.6 Mushroom body extrinsic neurons. (A) Schematic depiction of the α lobe ENs showing their respective clusters of somata (black circles with numbers A1–A7) and their dendritic branching areas.

(From Rybak & Menzel, 1993.)

(B) The neuron PE1, an individually identified EN of the alpha and beta lobe. The arrows point to the midline of the brains (m) and the dorsal direction also termed frontal direction of the insect body (d).

(p. 676)

Neural Correlates of Context-Dependent Learning

The context in which a cue is learned needs both to be attached to the cue and distinguished from the cue. Rescorla and Wagner (1972) argued that all stimuli present at the occurrence of the reinforcer are associated separately with the reinforcer, whereas Pearce (1994) favored the idea that co-occurring stimuli should be treated as unique combinations or configurations that are distinct from the elements. Elucidation of neural processing of cues and contexts may help to resolve this issue and to clarify the cognitive components of neural context/cue coding. We have asked how context and cue are coded in MB ENs by recording from three kinds of ENs, the PE1 neuron, the AE ENS (PCT neurons), and the A1/A2 ENs. In all three studies the PER conditioning paradigm was applied (Fig. 28.1).

PE1 Neuron

This unique neuron receives excitatory input across the whole peduncle of the MB from KCs, also indicated by its multimodal response property (Mauelshagen, 1993; Iwama & Shibuya, 1998; Rybak & Menzel, 1998), and most likely inhibitory input presumably from GABA-ir A3 neurons of the PCT (Okada et al., 2007). As Fig. 28.7 shows, the firing rate of PE1 is enhanced for rewarded visual contexts and reduced for rewarded olfactory cues (Hussaini & Menzel, 2013). Thus, context and cue are coded in categorically different ways, allowing for separation between the stimuli involved in one learning situation. These results support the interpretation that the two stimuli develop their specific associative strengths separately, a potential neural correlate of the Rescorla and Wagner theory.

PCT Neurons

In these experiments, bees were trained with a combined visual/olfactory paradigm while the PCT neurons were recorded (see Fig. 28.1; Filla & Menzel, 2015). PCT neurons are GABA-ir A3 feedback neurons providing an inhibitory recurrent feedback loop from the MB to its input. Ca²⁺ imaging experiments revealed significant increased neuronal responses to CS+ after training but also reduced responses to CS+ that were less frequent. These neuronal changes were linked to the behavioral changes, as seen in retention tests on the first (Haehnel & Menzel, 2010) or the second (Haehnel & Menzel, 2012) day. PCT neurons with their output within the lobes are expected to locally inhibit other ENs in the lobes in a learning-related fashion and thus may act as the source of learning-related CS+ response reduction as documented for the PE1 neuron (Okada et al., 2007). In the context/cue training experiments, the total number of neurons responding to the four combinations of cue and context (C+O+, C–O+, C+O–, C–O–) after training was highest for C+O+, followed by C–O+, then by C+O–, and lowest for C–O– (Fig. 28.8; Filla & Menzel, 2015).

Two groups of neurons were distinguished according to their responses to context and cue alone before training (Fig. 28.9: naïve responding units and naïve nonresponding units). When tested with the separate stimuli (context: C+, C–, Ctr, cue: O+, O–, Ctr) after context/cue compound condition, one could estimate the learning-related plasticity of each neuron. Most of the naïve responding units changed their responses during conditioning either not at all or (more rarely) developed selective responses to the separate stimuli (e.g., #5 stronger enhancement to C+ than to C– or the control visual context Ctr, or #8 stronger enhancement to O+ than to O– and the control odor cue Ctr). Naïve nonresponding units, however, showed a dominance of enhancement to C+ and O+ with selective responses to C+ and O+ in six out of eight units. The selective responses are more frequently an enhancement for C+ or O+ (five out of eight). In each case only one unit gave a higher response to C– or O–. Thus, increasing, decreasing, or not changing responses was found for both the context and cue, and these changes are often not the same in a particular unit for context and cue (Fig. 28.9). Some of the neurons developed similar associative plasticity for the visual context and the olfactory cue, indicating a neural correlate of Pearce’s (1994) concept that co-occurring stimuli will be treated as a unique configuration. Other neurons changed their responses differently for context and cue similar to the PE1 neuron (see earlier). In the cases of increase or decrease, they often expressed the inverse rate change to the nonreinforced stimuli (context and cue).

The predictive value of the context became particularly clear when test trials were evaluated in which the animal made an error (Fig. 28.10). When the animals responded correctly, the correct combination of context and cue C+O+ led to the highest activity in PCT neurons, followed by activity to C–O+ and then to O+ after no preceding context. Lowest activity was seen to C–O–. When the animal made an error and responded to the incorrect context and cue, this response pattern was reversed, (p. 677) (p. 678) giving the highest activity to C–O– and the lowest to O+ without preceding context.

Click to view larger

Figure 28.7 PE1 recording during context-dependent PER conditioning. Contexts 1 and 2 are visual stimuli (blue or yellow); stimuli A, B, and C are olfactory stimuli. (A) Context-dependent learning. Preconditioning phase: bees were presented with context 1 followed by odors A, B, and C. Context 1 was turned off and context 2 was presented followed by odors A, B, and C after which context 2 was turned off. This comprised one preconditioning trial, and the bees were subjected to two such trials. Conditioning phase: bees were presented with context 1 and odor A was presented together with the sugar reward (US), whereas odor B was presented without any US. Context 1 was turned off, and context 2 was presented, after which the two odors A and B were presented without any US. Conditioning trial ended when context 2 was turned off. Bees were subjected to five conditioning trials. Postconditioning phase: same as preconditioning, but each trial was presented once at 15, 60, and 120 min after conditioning. (B) Change in neuronal response of the PE1 neuron. Spike firing rate (Δ SFR) is expressed as the quotient between (postconditioning rate –preconditioning rate)/(postconditioning rate + preconditioning rate) during the particular stimulation. The panels below give the particular test stimulations. Δ SFR toward odor A in context 1 (rewarded) was significantly reduced at 60 and 120 min after conditioning compared with the other two odors B and C. There was no difference in firing rate between odors A, B, and C in context 2. Change in neuronal response toward context 1 increased at 15, 60, and 120 min after conditioning, whereas change in neuronal response toward context 2 remained unchanged. Plots show average Δ SFR for each group, and error bars represent SEM. (C) Plots showing percentage PER at 15, 60, and 120 min after conditioning. First group of bars show percentage PER toward rewarded odor A only in context 1 but not context 2; 40% of the bees learned the simple context learning rule. Second and third groups show percentage PER toward unrewarded odor B and neutral odor C, respectively, in either context 1 or context 2. Less than 10% of the bees showed a preference toward odors B and C.

(From Hussaini & Menzel, 2013.)

Click to view larger

Figure 28.8 Neural odor responses after conditioning to visual context C+ and odor cue O+ (see also Fig. 28.1). (A) Averaged normalized firing rates of PCT neurons to the conditioned odor (O+ black bars; O– gray bars) alone and during the visual contexts (C+ green outline; C– orange outline). Normalized firing rate is expressed in Z scores as defined by Filla and Menzel (2015). The learned visual contexts modulate of olfactory responses in two directions: the rewarded visual context (C+) enhances the response to the rewarded odor cue O+, whereas the unrewarded visual context reduce the response to the unrewarded odor O–. (B) The relative number of active units in response to the four compound stimuli whose rate responses are outlined as in A (together with single-odor responses). Of a total of 116 recorded units, 21% were active when the unrewarded odor O– was presented during the unrewarded context C–, and twice as many units (41%) were active when the same odor O– was presented during the rewarded context C+. Fifty-four percent of the recorded units fired in response to the rewarded odor O+ during the unrewarded context C–, and again, twice as many (97%) fired in response to O+ during the rewarded context C+.

(From Filla & Menzel, 2015.)

The learned visual context acts as a modulator of learned odor responses in PCT neurons. The C⁺ enhances the O⁺ response and subsequent reward expectation when color and odor are tested in the trained sequence. Most important, PCT neurons changed their firing pattern according to the value associated with all stimuli only in test trials in which the animal responded correctly. The neurons perfectly integrate the outcome-related value associated with both color and odor, indicating attention selective value coding that predicts behavioral action selection. Attentional control should prioritize neural processing of the most relevant stimuli in a given situation in reference to the expected outcome, and PCT neurons appear to participate in such a prioritizing process.

Enhanced learning-related inhibition via PCT neurons in the input site of the MB might reduce further strengthening of synaptic transmission for already learned stimuli. In contrast, lower inhibition via PCT neurons would favor the induction of synaptic plasticity for novel stimuli. Evidence supporting this hypothesis arises from the GABAergic anterior paired lateral neuron (APL) of Drosophila, which has striking morphological similarities with the A3-v cluster of the PCT (Liu & Davis, 2009). The APL neuron suppresses and is suppressed by olfactory learning, suggesting that reduced inhibition promotes learning.

Memory Processing and Expectation

Most EN neurons develop a response change to the learned stimuli over hours and days, indicating endogenous memory processing. Some of the A1/A2 neurons responded initially only to the US and after conditioning only to the CS+. Other A1/A2 ENs changed from excitatory CS+ responses to transient CS+ inhibitory responses. About half of the ENs recorded changed their responses to the reinforced stimuli. Interestingly most change their responses not during the acquisition process but after a consolidation phase of a few hours (Strube-Bloss et al., 2011). The delayed expression of associative plasticity in these ENs could reflect memory consolidation that depends on prolonged neural activities because consolidation in the MB can be blocked by cooling (Masuhr & Menzel, 1972). A1/A2 ENs develop also fully new properties over time, for example, which input side in the paired olfactory system retrieves the memory trace. Unilateral olfactory conditioning leads to a stable memory that can be initially retrieved only via the antenna that was stimulated during training. Several hours later the association can be behaviorally recalled via the contralateral antenna (Sandoz & Menzel, 2001), indicating across brain side interactions. Lateral (p. 679) integration might be necessary for solving complex forms of learning and memory formation. As pointed out earlier, nonelemental learning tasks are performed well when both antennae and MBs are involved (Chandra & Smith, 1998; Deisig et al., 2002). But if only one antenna is stimulated (Komischke et al., 2003) or only one MB is functioning, bees are no longer able to solve side-spanning learning tasks, do not solve contradictory information during differential conditioning, and do not learn negative and positive patterning tasks (Komischke et al., 2005). In addition, blocking between odorants in binary mixtures after pre-exposure to one element of the mixture depends on input from both antennae (Smith & Cobey, 1994). Removing the input from one antenna eliminates also the blocking of one odor by the other (Thorn & Smith, 1997). Recently, Strube-Bloss et al. (2011) found that A1/A2 ENs not responding initially to contralateral olfactory stimulation developed a unique and stable representation of the rewarded compound stimulus (odor plus contralateral stimulation side) after 3 hours predicting its value during subsequent memory retention tests. In parallel, the animals established a stable conditioned response behavior. The retrieval of the consolidated compound stimulus is delayed by about 50 ms, indicating an increased computation time for the readout after lateral integration. The PCT neurons are differently involved in encoding the learned context/cue association over the periods of days. Some of these neurons change their response properties already on the first day of training, some only after the second, and some after the third day of training (Filla & Menzel, unpublished data).

Click to view larger

Figure 28.9 Response changes of PCT neurons in the course of condition as tested for the separate stimuli involved in visual context and olfactory cue conditioning. Z scores (see legend for Fig. 28.8) were calculated for the respective stimulus conditions before conditioning (Z_before) and after conditioning (Z_after). The color code refers to the quotient Z_after/Z_before. Thus, values >0 indicate increased responses in the course of conditioning, and values <0 indicate decreases. Notice the different false color coding scales for naïve responding units and naïve nonresponding units. The two upper patterns display the response change to the visual context, and the two lower ones that for the odor cue. Numbers (columns) refer to different units, and the rows give the respective stimuli. Only separate stimuli (context or cue) were tested here, not context cue combinations, in order to test the learning related changes to the separate stimuli. Notice that color context learning leads to neural changes, although the animals did not respond with PER to the trained color context (see text).

Memory processing during sleep is facilitated if the context stimulus is applied during sleep phases at night (see earlier discussion). Preliminary recordings from A1/A2 ENs indicated enhanced activity to the thermal stimulus (CS+) after pairing with reward. A similar enhanced response was seen during sleep after context stimulation at the time window when the thermal stimulus would have occurred, indicating a reactivation of the CS+-dependent memory trace (Zwaka et al., unpublished data). It is (p. 680) thus possible that reactivation of the CS+ memory by the context stimulus leads to the stronger memory observed the next day in behavioral tests.

Click to view larger

Figure 28.10 Visual context modulation of olfactory processing and reward expectation during stimulus selection trials. Averaged normalized firing rates of all recorded neurons during stimulus selection trials to the conditioned odors (O+ in black, O– in gray) separately and together with the trained or reversed conditioned color contexts (C+ in green, C– in orange). For simplicity, only nonsignificant (ns.) differences are depicted as most of the pairwise comparisons were significantly different. (A) O+ Odor responses were enhanced during the high-value context (for the trained and previously rewarded compound [m1], C+O+, green and black bar) in comparison with the reversed counterpart compared to the nonreinforced context (C–O+, orange and black bar), O– responses were reduced in the nonreinforced context (C–O–, orange and gray bar), while the odor response of the trained and previously not rewarded compound. C–O– (orange and gray bar) was reduced opposed to its reversed counterpart compared to the high-value context (C+O–, green and gray bar). However, only the C+ context effectively enhanced the O+ response in comparison with odor O+ presentation alone (black bar). Odor responses of both C–O– (orange and gray) and C+O– (green and gray) compounds were not different from O– presentation alone (gray bar). The response to the C– (orange bar) was effectively reduced around odor omission (during color tests, no odor was presented) opposed to the C+ presentation (green bar). (B) All rate differences during the odor response window shown in A outlasted throughout the reward omission window. Additionally, the C+ effectively enhanced O– responses during the C+O– compound condition (green and gray bar), and the C– reduced O+ responses during the reversed C–O+ compound (orange and black bar) both in comparison with odor presentation alone.