Adaptive Behavior and Learning/contents/Chapter 2

page type::transcription thing type::book chapter URL/outdated::http://psychweb.psych.duke.edu/department/jers/abl/Chapter02.pdf]

Orb-web spiders have devised a most efficient net for catching flying insects, yet we can trace no history of trial and error in the life of an individual spider that could explain the excellence of the web's design. Spiders don't learn how to weave good webs; no spider tries different designs and discards all but the most efficient. It is this instant perfection that sustains mystical beliefs in the power of instinct and fortifies disbelievers in evolution. Yet there surely was trial and error, in the form of ancestral spiders that built webs with varying efficiencies. Those that built most efficiently were better nourished and had most progeny. If variation in web-building ability is partly inherited, then the ability to build better webs evolved not by selection of good webs by individual spiders, but by selection of spiders who made better webs. Web building depends on history, but more on the history of the race than on the history of the individual spider.

But in higher animals more and more behavior is selected in the first way, in the life of the individual, not by differential reproduction of individuals with different innate talents. This book is about the ways that individual history determines adaptive behavior.

The simpler the animal, the more we know about the processes that select one behavior over others. This chapter has two objectives: First, to illustrate the processes of variation and selection of individual behavior in some simple cases – orientation mechanisms in plants and protists. Second, to describe elementary processes such as habituation, adaptation, and the "laws" of reflex action, that are some of the ingredients of adaptive behavior in higher animals. The next chapter introduces the notion of feedback in connection with direct (taxic) orientation.

SIMPLE ORIENTING MECHANISMS
Finding the proper habitat, a place not too hot or cold, nor too dry or wet, safe from predators and with a supply of food, is a major behavioral problem for all animals. For simple, single-celled animals such as protists and primitive invertebrates it is the main problem. Since these organisms possess either no, or only the most rudimentary, nervous system, they must get nourishment and stay safe using very simple means. Hence, simple orientation mechanisms exhibit the properties of adaptive behavior in their clearest form.

The movement of climbing plants provides a good illustration of the orienta­tion problem, and of a simple solution to it. An important objective for any green plant is to gain maximum access to sunlight. Obstacles are presented by other plants with similar requirements. Where the need for light is not outweighed by other considerations, such as avoidance of predation or extremes of temperature or the effects of wind, plants therefore grow vertically and seek the highest, best-lighted point.

Charles Darwin identified rotation of the growing plant tip (he called it circumnutation) as a key element in direct plant growth. He describes circumnutation thus: When the shoot of a hop (Humulus lupulus) rises from the ground, the two or three first-formed, whilst very young, may be seen to bend to one side and to travel slowly round towards all points of the compass... From seven observations made during August ... the average rate during hot weather and during the day is 2 hrs. 8 m. for each revolution... The revolving movement continues as long as the plant continues to grow; but each separate internode, as it becomes old, ceases to move. &mdash;(1875b. pp. 2-3) Figure 2.1 shows Darwin's record of the changing position of the tip of a Trifolium plant, with points plotted every half hour.

Two forces are at work here, one directed and one undirected. The upward growth of the plant is a directed movement, a taxis; the direction is opposite to the force of gravity (negative geotropism). The way taxes work is described in the next chapter. But the turning movement is undirected – until the moving tip encounters a vertical obstacle such as a stick or the stem of another plant. Once such an obstacle is encountered, the turning movement, together with upward growth, ensures that the plant will twine about it and be lifted to a higher (and usually more satisfactory) location.

Most plants also possess a second taxic mechanism, phototropism. This combines with upward growth to direct the tip of the plant toward the lightest part of the sky. The efficient light-seeking of the plant can, therefore, be explained by the combined effects of three separate adaptive mechanisms: negative geotropism, circumnutation, and positive phototropism.

This example is very simple. There is no real need to talk about variation and selection here because all the ingredients are open to view. Nevertheless, circumnutation plays the role of variation: The plant tip sweeps out an irregular arc‚ "searching" for some vertical support. The environment, in the form of vertical obstacles, then plays the role of selection by blocking the moving tip and constraining it to a particular place. As we get to more complicated behavior, the mechanisms that underlie variation and selection are less well understood, although both variation in behavior, and the selective elimination of variants, can always be observed.

Light-seeking by a plant illustrates two aspects of the study of adaptive behavior: analysis as the route to understanding, and the link between physiology and behavior. The analysis of behavior into component processes, such as the three just described, is a major theme of this book. In chemistry the aim is to describe all compounds by the rules of combination of a few elements. In similar fashion, the aim of behavioral analysis is to explain all the behavior of an organism in terms of a limited number of fundamental processes and their rules of interaction. We are closest to achieving this with simple orientation mechanisms. We still have a way to go in understanding the adaptive behavior of most vertebrates.

The second aspect of the study of adaptive behavior is its relation to physiology: Once an adaptive mechanism has been identified at the purely behavioral level, it becomes profitable to look for its physiological basis; that is, for the structures and physico-chemical processes that underlie it. In the behavior of higher animals, the best understood processes tend to be those that involve the peripheral parts of the nervous system: sensory processes, simple perception, and some aspects of the organization of motor behavior, such as simple reflexes. It is no surprise, therefore, that it is in just these areas that our understanding of the physiological basis for behavior is most advanced.

Following this pattern, the study of plant movement by Darwin and others led eventually to the discovery of auxins, powerful regulators of the differential growth that underlies movement in plants. The movement of bacteria, the next example, is also being traced to complexes of reversible chemical reactions.

Indirect orientation (kineses)
All living creatures behave; not even a virus is completely inert. Even supposedly primitive animals often show extraordinarily clever behavior. For example, in the l880s the German biologists Engelmann and Pfeffer discovered bacterial chemotaxis (actually a kinesis in the terminology I am using): the ability of bacteria such as  to move up or down a chemical gradient. By inserting a capillary tube containing a suspension of attractive substance into a medium containing bacteria they showed that more bacteria entered the capillary than would be expected if they simply moved randomly. This is extraordinary behavior for an organism only about 2 &micro;m long, with only the most primitive sensory and motor apparatus, and no nervous system at all.

Bacteria have no distance receptors; they cannot detect the source of a chemical in the way that we can detect an illumination source, from a distance, by looking at it. (Mammals have no distance receptors for chemical gradients either, of course, nor is such a thing possible. Hence, dogs follow a scent trail in much the same ways as moths and bacteria.) The only way to find the source of a diffusing chemical is by means of a process called  (sometimes also called gradient descent/ascent – depending one whether the source is good or bad), where the hill to be climbed is the chemical gradient. Hill climbing can be explained by the simile of a blind man trying to find his way to the top of a hill. His simplest course is just to sample a direction: If it is downhill or level, sample again, but if it is uphill, follow it for one step, then sample again, and so on. A more efficient type of hill climbing is the method of steepest ascent: Here the man samples different directions from his current position and picks the one that gives the biggest increment. The logic of these two strategies is illustrated by flowcharts in Figure 2.2. Both require a small memory, sufficient to store a single slope value, but in addition the steepest-ascent method requires a stopping rule to tell the man when to cease sampling at each step. Still more complex, and more efficient, variants of hill climbing are possible (indeed, all adaptive behavior can be thought of as a sort of hill climbing).

The essential process in hill climbing is comparison of two heights, which define the gradient. There are only two ways a bacterium can detect a chemical gradient. One is by a comparison of chemical concentrations across the minuscule length of the body (simultaneous comparison), which would involve detection of concentration differences of as little as one part in 10,000, a formidable task. This mechanism would allow for directed movement up the gradient, a  in the classification of Fraenkel and Gunn (1940), whereas the observed movements look almost random with respect to the gradient. The second possibility is for comparison across time (successive comparison), allowing modulation of a largely undirected pattern of movement by increases and decreases in concentration. This kind of orientation mechanism is termed a kinesis in the Fraenkel and Gunn scheme.

Macnab and Koshland (1972) showed that temporal comparison was the key in ingenious experiments with two species of bacteria: Salmonella typhimurium, the agent of common food poisoning, and Escherichia coli, a resident of the human gut much used in molecular biology. First, they showed that the movement pattern of the bacteria was the same at different absolute concentrations of an attractant. Then they subjected the bacteria to a sudden, "step" change in concentration, and looked at them immediately after mixing was complete. If the organism's behavior is guided by comparison between the concentrations at its "head" and "tail" there should be no effect of the sudden change. But if their movement was determined by a difference across time, then a sudden drop in concentration should produce the same effect as swimming down a gradient.

And so it proved. These organisms show essentially only two modes of movement: straight-line swimming, and "tumbling." Bacterial tumbling increased dramatically if attractant concentration decreased, as it would when swimming down a gradient; tumbling decreased when attractant concentration increased, as when swimming up a gradient.

Thus, the bacteria have reduced a most complex problem in three-dimensional orientation to simple on-off control of a random pattern: When things are getting worse, tumble; when they begin to improve, swim straight. Bacteria find their way up the attractant gradient by means of the simple hill-climbing strategy in Figure 2.2: Straight swimming corresponds to "move forward" in Figure 2.2, and tumbling corresponds to "sample direction." The tumbling performs the role of a random search of the environment; when the search turns up a slight improvement, the organism follows that direction: "By taking giant steps in the right direction and small steps in the wrong direction. it biases its walk very effectively in the direction which aids its survival" (Koshland, 1977, p. 1057).

These experiments show that the bacteria are able to detect a gradient by making successive, rather than simultaneous comparisons. Over how much distance can they compare? This is determined by a process of adaptation that is, in effect, a simple memory (recall that some memory is essential to hill climb­ing). It can be measured as follows: In the attractant-increase experiment the initial effect is an increase in smooth swimming, and a decrease in tumbling. As time elapses, however, fewer and fewer bacteria swim smoothly and eventually all resume tumbling. This almost-constant tumbling is consistent with the initial observation that under constant conditions, whether attractant concentration is high or low, the bacteria show the same pattern of movement. Figure 2.3 shows a graph of the adaptation process. It shows the proportion of bacteria swimming smoothly at different times after mixing; the numbers were obtained from photographs like those in the figure. The curve is declining, showing that fewer and fewer bacteria are swimming smoothly as time goes by. The form of the curve is exponential, following an initial period of slower decline.

The adaptation mechanism allows these organisms to make comparisons across a distance of between 20 and 100 body lengths, which reduces the analytical problem from 10,000 to 1 to between 100 and 1,000 to 1: difficult, but better than simultaneous comparison.

The hill-climbing process used by these bacteria is so simple that Koshland has suggested that it may be caused by a relatively simple chemical process. His proposal is shown in Figure 2.4. X is a "response regulator," a substance whose concentra­tion determines the type of response: For example, if X suppresses tumbling, then when its concentration reaches a threshold value, tumbling is suppressed and the bacterium swims smoothly; but when the level of X is below threshold, the organism tumbles. Thus X is like the temperature input to an on-off thermostat. The rate of adaptation is determined by the rate, Vf, at which X is formed from the precursor W relative to the rate at which it is decomposed, Vd. In this chemical system, Vd is directly related to the concentration of X, so that the system is inherently stable: An increase in X caused by a rise in Vf will eventually be compensated for by an increase in Vd, so that the level of X will return to its initial level.

If Vf increases more than Vd when the bacterium travels up a gradient, more X will be produced, further suppressing tumbling and allowing further progress. If the level of attractant ceases to increase, however, Vd will catch up with Vf, the level of X will decrease, and the organism will begin to tumble, slowing its forward progress. On going down a gradient the opposite would occur: Vd will gain on Vf, tumbling will occur, and progress will be arrested.

The quantitative and biochemical details of this system remain to be worked out. Key quantitative elements are the value of the threshold and the rate of adaptation, since these will determine the range of gradient slopes where the system will function adaptively: For any choice of threshold and adaptation rate there will be some gradients where the system will fail to climb or may even descend the gradient.

Orienting behavior of the type shown so clearly by bacteria is common in simple animals that need to find stimuli that (a) cannot be sensed at a distance, such as temperature, chemical and humidity gradients (direct [taxic] orientation is more usual in illumination gradients); and (b) are likely to be gradual. For an organism the size of a bacterium or a Paramecium, or even a planarian, sharp spatial changes in temperature or chemical concentration are rarely encountered.

Tumbling in bacteria responds symmetrically to attractants and repellants: An increase in attractant concentration tends to suppress tumbling, as does a decrease in the concentration of a repellant. This relatively simple process thus provides the animal with a complete motivational system, able to deal adaptively with both positive and negative events.

As we move from bacteria to more complex single-celled animals, kinesic orientation becomes correspondingly more complex. For example, Jennings (1906) in a classic account described how ciliates such as Paramecium and Oxytricha avoid a region that is unfavorable (e.g., because of its temperature or pH). Figure 2.5 is taken from his book and shows the reaction of a single animal to a heated zone at the top of the figure. When it first enters the unfavorable region the animal stops, backs, then turns to the right and starts to move in a new direction; if the bad area is encountered once again, the process is repeated until a successful direction is found. The direction of the turn each time is unrelated to the actual direction of the boundary. This trial-and-error mechanism is obviously similar in many ways to the initiation of tumbling of bacteria when circumstances change for the worse; but just as obviously it is more efficient, because the first reaction of the animal is actually to reverse its direction of movement. It would be even more efficient if the turn were reliably away from the boundary, but this requires localization of the point of maximal stimulation on the animal's body and is presumably beyond the sensory capacities of Oxytricha.

Paramecium and many other ciliates show essentially the same pattern as Oxytricha, reacting to negative gradients by retreating and turning. As with the bacteria, this process serves both to avoid bad regions and seek good ones. If a bad region surrounds a good one, the process keeps the animal within the good region.

REFLEX MECHANISMS
The beauty of simple orienting mechanisms is that we can capture in one or two simple processes the entire behavioral repertoire of the animal. The familiar psychological categories of motivation, cognition, sensation, and perception are all fused in a harmonious combination of mechanisms that initiate action, respond to stimulation, and guide the animal toward things that will aid its survival and reproduction. Behavior such as the avoiding reaction of Paramecium – reversal of movement in response to a sudden "change for the worse" – makes sense as part of a system of reactions that combine to produce adaptive behavior. With animals like birds or mammals, on the other hand, it is more difficult to see how things fit together. Because of this complexity, researchers often succumb to the natural tendency to look for some particular type of behavior – like the "reflex" or the "operant" – as the key to behavior in general, when the real key is no behavior in particular, but rather how all behaviors fit together to yield adaptation to a niche. The behavior of mammals and birds is so varied and so dependent on individual history, and their ecological niches are so poorly understood, that we cannot grasp as a whole the functional relations among the various elements in their repertoire.

Paramecium can vary only its rate and direction of movement. Its rather rich sensitivity to a variety of chemical, photic, and thermal stimuli must, therefore, be funneled into these limited modes of action. "Higher" animals are not so limited. Not only can they sense more aspects of the physical environment, they can do many more things in response. The avoiding reaction of Paramecium is one extreme of a continuum of reactions that serve the function of bringing the animal into a congenial environment. The reflexes of higher animals serve a similar function – most reflexes avoid, escape from, or minimize the effect of, noxious stimuli – but rather than being modulated by the environment, like the avoiding reaction, they are simply replaced by quite different kinds of behavior when circumstances demand it. The boxer reflexly blinks and ducks as his opponent's fist approaches his face, but if he avoids successfully, his next reaction is likely to be a planned offensive strategy, not an automatic reaction.

Thus reflexes are a part, actually a rather small part, of the adaptive repertoire of higher organisms. Nevertheless, they have had historically a special role because of the apparent simplicity of the relation between stimulus and response. The immediacy of reflexes, and their relative independence of the animal's past history, make them easy to study. Moreover, the obvious adaptiveness of reflex properties, and the similarity between reflex properties and some aspects of the properties of "higher" behaviors, suggested to many that a complete psychology might be built on deep understanding of reflexes. This hope has not panned out. Nevertheless, for historical reasons and reasons of simplicity, a discussion of reflexes is a natural preliminary to discussion of more complex kinds of behavior.

The reflex idea has a long history, beginning in the modern era with the French philosopher Ren&eacute; Descartes (1596-1650), who was perhaps the first to propose that all animal, and much human, behavior can be explained mechanistically, rather than by reference to a "soul" that directs the animal from within. This idea was elaborated by the Russian physiologist I. M. Sechenov (1829-1905) who also carried out experimental work of the modern type on "spinal reflexes," that is, the built-in protective and integrative responses that can be demonstrated in animals in which the higher brain centers have been severed from the spinal cord (so-called 'spinal preparations'). This work was brought to its highest pitch by the British physiologist C. S. Sherrington and is summarized in his influential book The Integrative Action of the Nervous System (1906), based on a course of lectures delivered at Yale University.

Sherrington's work followed a familiar pattern. Rather than attempt to study sensory and motor integration as a whole, a confusing and impossible task, he attempted, instead, to reduce it to its simplest elements. Instead of working with an intact dog or cat, he operated surgically on the animal so as to disable the higher brain centers. These "spinal" animals cannot learn, so behave in an automatic way that can be easily studied. In his characteristic literary style (somewhat archaic, even in his own day), Sherrington describes the properties of his experimental preparation as follows:

Experiment today put[s] within reach of the observer a puppet-animal which conforms largely with Descartes' assumptions. In the more organized animals of the vertebrate type the shape of the central nerve-organ [i.e., the brain] allows a simple operation to reduce the animals to the Descartes condition. An overlying outgrowth of the central nerve-organ in the head can be removed under anaesthesia, and on the narcosis passing off the animal is found to be a Cartesian puppet: it can execute certain acts but is devoid of mind... Thoughts, feeling, memory, percepts, conations, etc.; of these no evidence is forthcoming or to be elicited. Yet the animal remains a motor mechanism which can be touched into action in certain ways so as to exhibit pieces of its behavior. &mdash;(1947. p. xi)

With this reduced, decerebrate animal it is possible to study in full quantitative detail the simplest level of reflex, sensorimotor integration, free of the complications introduced by spontaneity and "volition."

Reflexes are automatic but not stupid: they serve a clear function in the normal life of the animal. Sherrington continues with a number of examples:

The movements are not meaningless; they carry each of them an obvious meaning. The scope commonly agrees with some act which the normal animal under like circumstances would do. Thus the cat set upright... on a "floor" moving backward under its feet walks, runs or gallops according to the speed given the floorway. Again in the dog a feeble electric current ("electric flea") applied by a minute entomological pin set lightly in the hair-bulb layer of the skin of the shoulder brings the hind paw of that side to the place, and with unsheathed claws the foot performs a rhythmic grooming of the hairy coat there. If the point lie forward at the ear, the foot is directed thither, if far back in the loin the foot goes thither, and similarly at any intermediate spot. The list of such purposive movements is impressive. If a foot tread on a thorn that foot is held up from the ground while the other legs limp away. Milk placed in the mouth is swallowed; acid solution is rejected. Let fall, inverted, the reflex cat alights on its feet. The dog shakes its coat dry after immersion in water. A fly settling on the ear is instantly flung off by the ear. Water entering the ear is thrown out by violent shaking of the head. An exhaustive list would be much larger than that given here... But when all is said, if we compare such a list with the range of situations to which the normal dog or cat reacts appropriately, the list is extremely poverty stricken... It contains no social reactions. It evidences hunger by restlessness and brisker knee-jerks; but it fails to recognize food as food: it shows no memory, it cannot be trained or learn: it cannot be taught its name. &mdash;(1947, pp. xi-xiii)

The deficiencies of the reflex animal are in the simplicity of the stimuli to which it can respond and in the absence of any but the briefest memory. It can neither learn new things nor recall past experiences. It is just this absence of history that makes the preparation of such analytical interest. The assurance that what is observed now can be traced to causes in the environment that are either present now, or no more than a minute or two in the past, much simplifies the scientist’s task. The absence of spontaneous movement completes the picture. How different is the normal animal, whose behavior now may reflect experiences months or years ago and whose ability to adapt to new situations demands the expression of a variety of novel, spontaneous behaviors.

Sherrington was a physiologist and his concern was with the functioning of the nervous system. He defined a reflex as a sensorimotor (stimulus-response) relation involving at least two neurons between receptor and effector, that is, at least one synapse, the then hypothetical (now directly observed and much-studied) point of contact between communicating nerve cells. Most of his experiments on reflex function were purely behavioral, however, and the reflex properties that emerged from them turn up even in organisms lacking a nervous system. The same properties – habituation, spatial and temporal summation, thresholds, refractory period, “momentum,” and others – are shown by many reactions of intact higher animals. The properties of the reflex seem to reflect evolutionary convergence; that is, a set of similar adaptations to similar environmental features.

Sherrington’s concept of the reflex is far from the simple, inflexible, push-button caricature sometimes encountered in introductory textbooks. To be sure, there is always a stimulus and a response; but the ability of the stimulus to produce the response depends on the reflex threshold, and the threshold of each reflex depends not only on the state of many other reflexes but also (in the intact animal) on higher centers, which retain the effects of an extensive past history. The function of reflexes is the integration of behavior, which would be impossible without well-defined rules of interaction.

Reflex properties can be divided into two classes: the properties of reflexes considered in isolation, and the rules governing interactions between reflexes. I consider each class in turn.

Individual reflexes
A stimulus, a sound, a light, a touch, is presented and, after a brief delay, a response – a movement of the ear, contraction of the pupil, a scratching movement of the paw – ensues. Reflex is the name for the properties of this relation between stimulus and response. Stimuli have intensity, quality, duration, location, and perhaps other properties; responses have similar properties and, in addition, follow the stimulus with a certain delay (latency) and with a certain reliability (probability). The properties of the reflex are the relations between these stimulus and response properties.

Underlying all is the theoretical notion of reflex strength, the tendency for the reflex response to occur. Reflex strength cannot be measured directly, but can be estimated by, for example, the intensity of stimulus required to elicit the response, by the degree of variation from the optimal stimulus that is still sufficient to produce the response, and by the latency between stimulus and response: The weaker or more different from optimal the stimulus, and the more rapid the response, the stronger the reflex.

There are seven main reflex properties:

Threshold
The stimulus for a reflex must be above a certain minimum level if it is to elicit a response. This is the absolute threshold for the response. The threshold is not fixed, but depends on a number of factors. In the intact animal, the threshold depends on, for example, the stimulus situation, the animal’s state of attention, its motivational state, and its past history – both immediate and remote. In the spinal, or decerebrate animal, the threshold is mainly affected by immediate past history and the state of other reflexes.

Thresholds are typically measured in the following way. Weak stimuli (in the neighborhood of the threshold) are presented in a random order, well spaced in time, and occurrences of the reflex response that exceed some criterion are counted. A stimulus close to threshold will sometimes elicit the response and sometimes not. The proportion of times that each stimulus is effective is plotted as a function of stimulus intensity, and the result is usually an S-shaped (sigmoidal) function, as shown in Figure 2.6. The threshold is taken, by convention, to be the stimulus intensity that elicits the response 50% of the time. Since the location of the function, and hence of the threshold, along the stimulus-intensity axis depends both on features of the experimental situation and on the criteria chosen, the threshold function is not of much interest in isolation; rather it is used as a dependent variable. The experimenter is usually interested in the change in the threshold caused by some other experimental manipulation, such as earlier elicitation of the same or an antagonistic reflex, a change in the animal’s motivational state, or some other factor that may affect reflex strength.

The gradual slope of the threshold function reflects the contribution of noise – unpredictable fluctuations in threshold – traceable to lack of perfect constancy in the physiological state of the system. If noise were to be completely eliminated (a physical impossibility), the threshold function would be a straight, vertical line, a “step” change from 0% to 100% response at the threshold intensity. In most cases, threshold variability is “planned” (rather than a reflection of poor construction) in the sense that some variation in threshold is usually adaptive. When variation makes no adaptive sense, the system shows very little variability: Under optimal conditions, human visual sensitivity is close to the limits imposed by the quantal nature of light, for example.

Latency
The time between stimulus onset and the occurrence of the reflex response is one of the most useful measures of the state (strength) of the reflex. Like threshold (and for the same reasons), latency measures show some variability; the same stimulus will produce responses with slightly different latencies on successive presentations. Even when temporal interactions (to be discussed subsequently) are well controlled.

Typical latency distributions are shown in Figure 2.7. Latency distributions are usually of the form to be expected from a relatively simple stochastic, that is, probabilistic-in-time, process. Here is one way to look at it. Suppose that in the absence of a stimulus, the reflex response occurs at a very low, random rate, but after the stimulus, the response occurs almost at once. The dog scratches once in a while just for the heck of it, but if you tickle his ear, he scratches at once.

How might we model this? Well, a “random” rate of response corresponds to tossing a biased coin, whose probability of “heads,” P, is equal to the probability a response will occur at the end of every small time increment &delta;t. If heads comes up, the response occurs. If P is constant, the resulting distribution of interresponse times will be exponential, like the red line in Figure 2.7. What might be the effect of the stimulus for the reflex? The simplest is to say that the stimulus has an effect on the value of P, the response probability in each &delta;t. If P is close to zero, very few random responses will occur in the absence of a stimulus. If the stimulus elevates the value of P, then a response latency is likely to occur soon after the stimulus is presented. In this case, the latency distribution, the distribution of times between stimulus onset and the random first response, will also be exponential. The combination of the two, randomly varying latency (dead time) plus random response hereafter, might look something like the blue line in Figure 2.7. The virtue of these simple stochastic models, in addition to economy of description, is that things that affect latency often have selective effects on just one of the parameters of the stochastic model.

There are two general messages from this brief discussion of latency distributions. The first is that a process that is random in time yields an exponential latency distribution (not one that is flat or bell shaped). Second, that quite simple processes may underlie apparently complicated distributions.

Latency depends on the same extrinsic factors as threshold, and the two measures tend to covary: a low threshold being associated with a short latency, a high threshold with a long latency. Latency also depends on stimulus intensity; usually a more intense stimulus elicits a more rapid response. These characteristics are consistent with the protective function of most reflexes. A more intense stimulus is likely to be more dangerous and to justify a more rapid and vigorous response. Thus the strength, as well as the speed, of most reflex responses is directly related to stimulus intensity.

Refractory period
After a reflex response has occurred, the threshold of the reflex may be elevated for a brief refractory period, a post-stimulus time when the probability of response is zero, no matter how strong the stimulus. The refractory period is intrinsic to the physiology of nerve conduction, is common in spinal reflexes, but may or may not occur in stimulus-response relations in the intact organism.

Reflexes are either tonic (i.e., maintained by continuous stimulation, as in postural reflexes), or phasic (i.e., transient or repetitive, as in the scratch reflex). A refractory period is obviously essential to phasic reflexes, else continued stimulation would lead to a continual excitation of the response, as in tonic reflexes. The function of the scratch reflex, for example, is to produce repeated, back-and-forth limb movements in response to a continuous Ã¢â‚¬Å“tickleÃ¢â‚¬Â stimulus. Time for limb withdrawal (flexion) is provided by the refractory period of the active (extension) phase of the reflex: After the limb has extended, continued stimulation can produce no further extension. Withdrawal occurs because of reciprocal inhibition between flexors and extensors, which is discussed later. Flexor and extensor muscles are so connected that facilitation of one produces inhibition of the other, and vice versa. Hence, inhibition of the extensor in the refractory period of the scratch reflex automatically yields activation of the flexor, and, thus, withdrawal of the limb ready for the next stroke. The process is cumbersome to describe in words, but can be expressed economically in the formal language of oscillatory networks (an attempt to do this can be found in Staddon, 2001b, Chapter 5). Reciprocal inhibition and successive induction are properties of such networks.

Temporal summation
Two subthreshold stimuli, spaced closely in time, may excite a reflex when either alone would be ineffective; this is temporal summaÃ‚Â­tion. For example, the so-called orientation reflex is a set of coordinated moveÃ‚Â­ments by which an animal focuses on a novel stimulus, such as an unexpected sound. A cat will turn toward the sound, looking at the direction of the source with ears pricked forward, pupils dilated, and muscles tensed. The animal is in the optimal state both to receive any new information and to act on it immediately. It is a common observation that one weak stimulus may not elicit this response (i.e., Ã¢â‚¬Å“get the animalÃ¢â‚¬â„¢s attention,Ã¢â‚¬Â as it is usually expressed), but two in close succession may. For example, the prey-catching response of frogs toward small moving objects can be facilitated by small movements of the same stimulus a few seconds earlier, even though the initial movements seldom trigger the response (Ingle, 1975). Spatial summation

Somatosensory reflexes (such as the scratch reflex) each have a specific region of the skin within which a stimulus can excite a response. This region is the receptive field of the reflex (see Figure 2.8). Two stimuli, each individually subthreshold, may excite the response if presented together: This is spatial summation. The degree of summation, in general, depends on the degree of proximity of the two stimulated regions within the receptive field; the closer the stimuli, the more they summate.

Temporal and spatial summation interact, as graphically described by Sherrington:

Momentum (after-discharge)
The excitation of most phasic reflexes will generÃ‚Â­ally outlast the stimulus that produced it. SherringtonÃ¢â‚¬â„¢s discussion of the scratch reflex provides an example (Ã¢â‚¬Å“...a vigorous reflex that continues and outlasts the application of the stimulusÃ¢â‚¬Â). Under restricted conditions, repeated elicitation of a reflex may become easier and easier, and this may also be related to momentum. But the usual effect of a weak stimulus is habituation. Habituation

Repeated elicitation of any reflex response eventually leads to a decrease in vigor and loss of stability (irregularity and tremor) and finally to cessation of the response. Habituation to a weak stimulus occurs sooner than to a strong one, presumably because the reflex threshold rises above the level where a weak stimulus is effective before it comes to exceed the level of a stronger stimulus. Thus, a reflex fatigued by a weak stimulus can be elicited once again by a stronger one, and does not persist when the stimulus intensity is reduced again. Most important, reflex habituation is not due to muscle fatigue: Flexor muscles that cease to act as a consequence of habituation of the flexion reflex (withdrawal of a limb to painful stimulation) are still readily excited by the scratch reflex; a scratch reflex habituated by repeated stimulation at a point in its receptive field is readily excited again by stimulation at a new place within the field a little distance away from the old. Reflex habituation, like habituation in the intact animal, is stimulus specific. Habituation dissipates with time: If a habituated reflex is left unstimulated for a time, it will recover so that the same stimulus applied after a delay is once again effective.11

Reflex interaction
Very little of the behavior of intact, higher animals is purely reflexive: The same stimulus doesnÃ¢â‚¬â„¢t invariably elicit the same response; the effective stimulus (i.e., the set of all physical stimuli adequate to produce the response) is often complex and hard to define in physical terms, and the organismÃ¢â‚¬â„¢s history (for more than just a few minutes past) affects responsiveness. The intact animal, unlike the spinal preparation, cannot be restored to some constant, Ã¢â‚¬Å“groundÃ¢â‚¬Â state simply by the lapse of time; stimulus effects are not completely reversible.

Nevertheless, the properties of reflexes are of interest to students of the norÃ‚Â­mal, intact animal because reflexes exhibit in clear and relatively simple form the essential features of any behavioral unit: temporal properties (dynamics), the effects of stimulus intensity and frequency, self-exciting (momentum), and self-limiting (habituation) effects. These properties of reflexes are useful dimensions for the analysis of more complex behaviors.

Some of the reflex properties already described are not characteristic even of all reflexes, much less of the behavior of intact animals. For example, learned stimulus-response relations, or operants, which form the major topic for this book, donÃ¢â‚¬â„¢t always show the direct relation between stimulus intensity and response vigor that is shown by protective reÃ‚Â­flexes: Yelled instructions arenÃ¢â‚¬â„¢t necessarily more effective than instructions delivered in more normal tones. The latency measure is not appropriate for many operants where time is part of the effective stimulus. On the other hand, many phasic operants, such as pressing a bar or pecking at a fixed stimulus, show properties akin to habituation and momentum. The phenomena of temporal and spatial summation have a parallel in stimulus generalization.

The constraints that determine reflex interaction apply to all behavior, however. There are two: (a) The constraint that the organism can do only one, or at most a few, things at a time (this corresponds to SherringtonÃ¢â‚¬â„¢s final common path for reflex action).12 And (b) The adaptiveness, or lack of adaptiveness, of intermediate courses of action. Most commonly, perhaps, Ã¢â‚¬Å“he who hesitates is lostÃ¢â‚¬Â Ã¢â‚¬â€ a state of activity intermediate between two opposed courses of action is less adaptive than either. When confronted with a stimulus that has both attractive and frightening features, flight or fight are both likely to be better than hesitation; orientation to either of two simultaneous novel stimuli coming from different quarters is better than looking between them. On the other hand, conflicting postural demands are best resolved by an intermediate solution, rather than either extreme.

There are three principles of reflex interaction.

Reciprocal inhibition (competition)
Since incompatible reflexes must compete for the same final common path to the effectors, the primary principle of reflex interaction is reciprocal inhibition: Facilitation of reflex A inhibits reflex B; and, conversely, inhibition of A facilitates B. As we have seen, this rule holds for incompatible behavioral units at any level of complexity.

This principle was particularly obvious to reflexologists because of the reciprocal innervation of flexor and extensor muscles that is essential to the proper coordination of movement. The reflex principle of reciprocal antagonism applies to individual responses, but a similar principle of antithesis seems also to apply to whole systems of muscles that are affected by positive or negative Ã¢â‚¬Å“moodsÃ¢â‚¬Â of an animal.

Darwin (1872) in his book on psychology, The Expression of the Emotions in Man and Animals, noticed that each emotional Ã¢â‚¬Å“state of mindÃ¢â‚¬Â tends to be associated with a stereotyped set of reactions. For example, a person who is perplexed may scratch his head, roll his eyes, or cough; someone who disagrees with a proposition is likely to avert or close his eyes and shake his head, whereas if he agrees, he will look straight with open eyes, and nod his head. In greeting his master a dog typically takes up the submissive posture shown in the upper panel of Figure 2.9: The forelegs are bent, the coat smooth, and the tail wagging. The dogÃ¢â‚¬â„¢s mood when confronting a potential adversary is in a sense opposite to his mood when facing a well-loved individual. Darwin pointed out that the active muscles are also opposite: if a flexor is tensed in the first case, the extensor will be in the second. The bottom panel of Figure 2.9 illustrates such a threat posture: The dogÃ¢â‚¬â„¢s head is erect, his ears pricked, his legs are straight, and his fur and tail raised Ã¢â‚¬â€ all actions opposite to the submissive posture in the upper panel.

Many expressions of emotion function as communications, and are, therefore, subject to special constraints. In particular, messages requiring very different actions of the receiver should also be very different in form, so that the inevitable confusions are more likely to occur between messages similar in meaning than between messages of very different meaning. Thus the property of reciprocal inhibition may have quite a different functional basis in reflexes and in emotional expression, although in both it can, ultimately, be traced to the adaptive imporÃ‚Â­tance of decisive action.

Most reflexes compete, but some combine synergistically. The second propeÃ‚Â­rty of reflex interaction is therefore...

Cooperation
Reflexes that share response components may facilitate one anÃ‚Â­other (Ã¢â‚¬Å“alliedÃ¢â‚¬Â reflexes), or when simultaneously excited may yield a response whose character is intermediate (blending). For example, subthreshold stimuli for the flexor and scratch reflexes may summate to elicit limb withdrawal. This is an example of mutual facilitation (also termed immediate induction) by two reflexes sharing a common response. The same reflex excited by stimuli from widely different parts of its receptive field behaves in many ways like two separate reflexes (for example, habituation of the reflex to one stimulus may have little effect in the threshold for the other); thus, spatial summation is a phenomenon closely related to cooperation.

Because posture must be maintained at all times, postural reflexes always combine synergistically with other reflexes to reach a compromise that is usually adaptive. Blending is also a common mode of interaction between action tendenÃ‚Â­cies guided by some of the orientation mechanisms discussed in Chapter 3. Successive induction

An important effect that derives from reciprocal inhibition (competition) and the temporal properties of reflexes is successive induction (also termed Ã¢â‚¬Å“spinal contrastÃ¢â‚¬Â). In SherringtonÃ¢â‚¬â„¢s words: Ã¢â‚¬Å“...the extension-reflex predisposes to and may actually induce a flexion-reflex, and conversely the flexion-reflex predisposes to and may actually induce an extension-reflexÃ¢â‚¬Â (Sherrington, 1947, p. 213). The role of this mechanism in recurrent reflexes (e.g., scratch reflex) has already been touched on, and its usefulness in alternating movements, such as those involved in walking – extension of the left leg induces flexion in the right, and vice versa Ã¢â‚¬â€ is obvious.

An example of successive induction is shown in Figure 2.10, which is taken from SherringtonÃ¢â‚¬â„¢s (1906) book. It shows the magnitude of the extension reflex of the hind limb of a spinal dog (measured by means of a pen arrangement connected to the limb and writing on a rotating cylinder) both before (panel A) and after (panels B-F) strong excitation of the opposed flexor reflex. The extensor reflex is strongly facilitated by the prior excitation of its opponent (panel B). This effect depends both on the reciprocal inhibition between flexor and extensor reflexes, and on the time between stimulation of the flexor and subsequent elicitation of the extensor: the longer the time between, the less the successive induction effect. Most of the decrease in the extensor reflex shown in panels B-F is simply due to the passage of time, rather than to habituation of the extensor reflex due to repeated elicitation.

The term contrast is often used for successive induction because of the obviÃ‚Â­ous resemblance to perceptual effects: A bright disk surrounded by a dark annulus appears brighter than the same area surrounded by a light annulus (simultaneous brightness contrast); similarly, an illuminated area viewed after a period of darkness appears brighter than following a period of illumination (successive brightness contrast).

Contrast effects turn up in all sensory modalities, in the behavior of most, if not all, animals, and even in experiments on reward and punishment (behavioral contrast; see Chapter 11). Several interesting examples are discussed by the entomologist Kennedy (1965) who has repeatedly emphasized the importance of temporal interaction between behaviors as a source of Ã¢â‚¬Å“motivatedÃ¢â‚¬Â behavior.

KennedyÃ¢â‚¬â„¢s most extensive series of experiments involves the aphid Aphis fabae. This animal has two main modes of behavior, Ã¢â‚¬Å“settling,Ã¢â‚¬Â which comprises several related actions (probing the leaf, feeding, excretion, and reproduction), and Ã¢â‚¬Å“flight,Ã¢â‚¬Â which involves locomotion (by walking or flight) between feeding sites. There is reciprocal inhibition between these two systems, and independent stimulus control over each: The stimulus that tends to excite flight is light, and the stimulus for settling is the surface of the host plant. This system of interactions can conveniently be represented as in Figure 2.11, which shows the two activities, S and F, their facilitating stimuli, P and L, and the reciprocal inhibiÃ‚Â­tion between activities.

The temporal interactions between settling and flight in this animal can become quite complicated. However, it is possible to distinguish two relatively simple modes. Because each activity has its own controlling stimulus, each can be elicited separately. If the aphid has been flying for some time (and this can be arranged with the aid of a vertical wind tunnel, so that the animal maintains a constant air speed without moving relative to its surroundings), presentation of an appropriate surface will elicit settling, which, therefore, inhibits flight. ConÃ‚Â­versely, if the animal is settled, turning on the overhead light will usually elicit upward flight, which, therefore, inhibits settling.

In addition to these direct inhibitory effects there are also aftereffects. The details have been best worked out for flight as it is affected by prior settling: If the tendency to flight is already strong (i.e., the animal shows a high air speed) when it is suppressed by settling, then removal of the settling stimulus causes a rebound of the tendency to fly above its level when the settling stimulus came on. This is an example of successive induction of excitation. Conversely, if the tendency to fly is low when the settling stimulus is presented (and by implication, the tendency to settle is relatively high), then flying may be depressed (relative to the preinhibition level) after the settling stimulus is withdrawn: This is successive induction of inhibition. The balance between these two effects, poststimulus excitation versus poststimulus inhibition, is unstable and shifts from moment to moment. This instability is probably not accidental, but represents a useful source of variability in behavior. Since adaptive behavior involves the selection of adaptive variants by an unpredictable environment, a residue of behavioral variation is often essential if an animal is not to become Ã¢â‚¬Ëœlocked inÃ¢â‚¬Â to maladaptive modes of behavior.

A second example, from a study of walking in locusts by Moorhouse, Fosbrooke, and Kennedy (1978), shows how periodic changes in the stimulating environÃ‚Â­ment can interact with the successive-induction process to produce paradoxical effects. Young locusts were positioned in the treadmill apparatus shown in Figure 2.12, which allowed their walking and turning movements to be moniÃ‚Â­tored automatically. Moorhouse et al. divided the behavior of the insects into two mutually exclusive categories: Ã¢â‚¬Å“walking,Ã¢â‚¬Â which entails forward progression at or above a minimum speed, and Ã¢â‚¬Å“nonwalking,Ã¢â‚¬Â which is everything else (backward progression, slow forward progression, grooming, biting, Ã¢â‚¬Å“peering,Ã¢â‚¬Â and complete immobility). The animals were subjected to three experimental treatments: continuous light, continuous darkness, and alternating light and darkness. The pattern of behavior following light onset is shown in the three-dimensional graph in Figure 2.13. The graph shows the range of walking speeds in 10-min segments following light onset. For the first 10 min, the animal walks little, but the amount of walking gradually picks up over time, reaching a peak after about 60 min and declining slowly thereafter. The pattern of walking in continuous darkness was different in two main respects: The onset of walking was almost immediate, so that the animals walked substantially even during the First 10 min, but the rate of walking peaked sooner, after about 30 min, and declined thereafter to a low level. Overall, the animals walked more in the light than in the dark.

The third treatment involved continuous darkness followed by alternating 10-min periods of light and dark. Here, surprisingly, the animals walked more in dark periods than in light, walked much less in the light than in continuous light (the first treatment), and walked more in the alternating periods even than in continuous light. The results of the three treatments are compared in Figure 2.14.

These results are readily explained as successive induction of excitation. The slow buildup of running in the light suggests that following light onset, activities antagonistic to running are strongly, but transiently, excited. When light and dark periods alternate, only these nonwalking activities have time to occur during each light period. However, since these activities inhibit walking, and since they are not excited by dark (indicated by the immediate onset of walking in the dark), their offset in the dark allows walking to rebound to a level above its level in continuous light.

INHIBITION AND REFLEX STRENGTH
Inhibition and reflex strength have already appeared in earlier discusÃ‚Â­sion, but they deserve separate treatment. Strength is a useful but hard-to-define term that refers to the various measures associated with the Ã¢â‚¬Å“elicitabilityÃ¢â‚¬Â of a reflex: threshold, latency, vigor, susceptibility to habituation, and probability of elicitation, all considered in relation to the adequacy of the stimulus: a weak response to a poor stimulus may represent the same reflex strength as a strong response to a good stimulus. For most protective reflexes, under most conditions, these measures are correlated: A short latency goes along with a low threshold and a vigorous response elicited with high probability. The correlation tends to break down for other (e.g., postural, autonomic) reflexes and for learned behaviors. The concept of strength is nevertheless useful, even though some discretion must be exercised in picking appropriate measures in each particular case. For many recurrent habitual behaviors (e.g., lever pressing, runÃ‚Â­ning, pecking, etc.) the rate of occurrence of the activity (i.e., instances per unit time), or the proportion of time spent engaging in it, are useful measures of strength (more on this in later chapters).

Part of the difficulty in defining strength comes from inhibitory effects. For example, an animal may have a relatively strong tendency to engage in an activity such as walking, but the activity may nevertheless fail to occur because of inhibition by some other activity, as in the locust experiments just described: The high strength of walking was apparent as soon as the inhibiting activity ceased, since the inhibited activity then rebounded (successive induction). Evidently, a suppressed response may have considerable strength even though it is not occurring.

The first direct demonstration of inhibition in the nervous system was Eduard and Ernst WeberÃ¢â‚¬â„¢s discovery in 1845 that stimulation of the vagus nerve suppresses the heartbeat. This is straightforward enough: An ongoing activity is weakened by the addition of some factor. Nevertheless, the concept of inhibition has a history of muddle and mystification. Perhaps the simplest way to think about it is that inhibition refers to one of the ways in which causal factors may combine to produce an effect. In the simplest case (and even quite complex cases can be reduced to this one by appropriate scale changes and other manipulations) causal factors combine additively, so that inhibitory factors are just those that add in with negative sign. A concrete illustration is provided by the physical arrangement shown in Figure 2.15. The dependent variable (Ã¢â‚¬Å“level of responseÃ¢â‚¬Â: y) is the pointer reading, which can range from 0-10). (Note that there are no negative numbers, since the measured level of a response must be nonnegative.) The causal factors affecting y are simply weights that may be added to pans A and B.  If there is no friction and the spring is perfectly elastic, then obviously, y, the pointer displacement, is proportional to the net weight acting against the restorÃ‚Â­ing spring, so that:			y =  C(a1 + a2 + ... - b1 - b2 - Ã¢â‚¬Â¦) or, using summation notation:										(2.1) where ai (i =  1, 2,Ã¢â‚¬Â¦    ,m) denotes one of m weights added to pan A, bj (j = 1, 2 Ã¢â‚¬Â¦n) denotes one of n weights added to pan B, and C is a constant of proportionality that depends on the strength of the spring.  Thus the B-factors are inhibitory with respect to y and the A-factors are excitatory.

Figure 2.15 also illustrates three other characteristics of Ã¢â‚¬Å“realÃ¢â‚¬Â behaviors. (a) If the B-weights are large, the pointer will be off the scale to the left (the behavior will not be occurring). Yet obviously the number of A-weights necessary to bring the pointer on-scale will not be constant but will depend on the number of B-weights (this is sometimes called inhibition below zero; if there are always a certain number of B-weights, the resulting minimum A-weight necessary to bring the pointer on-scale corresponds to an absolute threshold). Thus, when a behavior is not occurring we cannot know what its real strength is. (b) Similarly, if there are many weights in pan A, the pointer may be off the scale to the right (we can consider this to be a pointer reading of 10). This corresponds to the physiological factors that limit the actual level (force, rate of occurrence) of any behavior (this is often called a ceiling effect). (c) Real balÃ‚Â­ances are subject to friction, which means that small changes in the level of the A- or B-weights may have no effect. Friction is analogous to a differential threshold, and can produce hysteresis, that is, a dependence of the pointer reading not just on the levels of the A- and B-weights, but also on the way in which those levels were arrived at.13 These factors may limit the proportionality between causal factors and the observed level of a behavior.

These three factors, ceiling effects and differential and absolute thresholds, mean that a given response measure may not be linearly related (proportional) to response strength. There are two main types of nonlinearity: One tends to limit the effects of causal factors, the other to exaggerate them. The model in Figure 2.15 illustrates the limiting type, in which extreme causal-factor values are limited by a ceiling. This effect is usually gradual, rather than abrupt: After the absolute threshold is exceeded, fixed increments in causal factors produce smaller and smaller increments in the level of behavior (this is called a negatively accelerated function.  The solid line in Figure 2.16 shows an example). The effects behavior of external factors, such as stimuli and reinforcement, generally follow negatively accelerated functions; in this way the unlimited range of physical stimuli is funneled into the limited range of physiological response.

The inhibition between competing reflexes usually follows the exaggerating, positively accelerated, kind of nonlinearity (the dashed line in Figure 2.16), however, because this ensures that either one or other, but not both, can occur. Elimination of the spring from Figure 2.15 leaves a model of this kind of nonlinearity in its most extreme form. Without the spring, the pointer has only two stable positions, 0 or 10, depending which pan is heavier.

SUMMARY
Reflexes and kineses are both limited in a critical way: They depend only on events in the present or the immediate past. Reflexes are limited in an additional way, in that they are ballistic, that is, they occur in response to an eliciting stimulus and are not readily modified by their consequences: they are almost independent of feedback.

This difference between reflexes and orienting mechanisms simply reflects their different functions. Kinesic orientation mechanisms have a large random component because simple organisms are in no position to predict important features of their environment. Consequently, a large pool of behavioral variation is necessary if the right variant (location) is to be found. Conversely, reflexes have evolved to deal with highly predictable contingencies: Limb withdrawal is almost certainly the best response to painful stimulation of an extremity; limb flexion is certainly best accompanied by relaxation of the extensor, and so on.

Later chapters deal with behavior that shows progressively more and more dependence on past history. The next chapter takes up the general topic of feedback and its role in orientation.

Edit Notes

 * 2005-08-14 Main text entered and broken into sections and paragraphs; To do:
 * Figures
 * Formatting (italics, bold, superscripts on footnotes, etc.)
 * Equations
 * 2012-12-11 substantial cleanup that was done for the temp HTYP has now been imported back into this copy