NEURAL SELECTION AND CONTROL OF VISUALLY GUIDED EYE MOVEMENTS

Jeffrey D. Schall and Kirk G. Thompson

Vanderbilt Vision Research Center, Department of Psychology, Vanderbilt University,

Nashville, TN 37240

ABSTRACT 1

INTRODUCTION 1

Scanning Eye Movements 1

NEURAL CORRELATES OF SACCADE TARGET SELECTION 2

Neural Correlates of Visual Selection 2

Frontal Eye Field 2

Target Selection in Frontal Eye Field During Visual Search 3

Target Selection in Frontal Eye Field During Visual Masking 3

Effects of Experience on Target Selection 4

Timecourse of Target Discrimination 4

NEURAL CONTROL OF SACCADE INITIATION 5

Rise to Threshold Mechanism for Reaction Time 5

The Countermanding Paradigm 6

Gaze Control Signals in Frontal Eye Field 6

CONCLUSION 7

Literature Cited 7

ABSTRACT

We review neural correlates of perceptual and motor decisions, examining whether the time they occupy explains the duration and variability of behavioral reaction times. The location of a salient target is identified through a spatiotemporal evolution of visually-evoked activation throughout the visual system. Selection of the target leads to stochastic growth of movement-related activity toward a fixed threshold to generate the gaze shift. For a given image, the neural concomitants of perceptual processing occupy a relatively constant interval so that stochastic variability in response generation introduces additional variability in reaction times.

INTRODUCTION

Nervous systems evolved to make decisions when faced with alternatives. The decisions leading to complex actions such as choosing to read this review are difficult to investigate at present. However, the decisions leading to basic actions such as eye movements are accessible experimentally. Each eye movement represents the outcome of two basic decisions -- where to look and when to look. The decision where to look entails the interaction of visual processing and cognitive guidance, and the decision of when to look entails the control of gaze. The neural systems responsible for visual processing and gaze control have been described (Wurtz & Goldberg 1989; Carpenter 1991; Colby & Duhamel 1991; Merigan & Maunsell 1993) as have neural correlates of visual perception (Parker & Newsome 1998) and attention (Desimone & Duncan 1995; Maunsell 1995; Schall 1995; Schall & Bichot 1998). We will emphasize new information about the selection processes that complete perceptual processing to guide action and new insights into how cortical neural processes regulate the initiation of eye movements.

Scanning Eye Movements

The function of the visuomotor system must be understood in reference to natural behaviors. Figure 1 shows the pattern of eye movements produced by a monkey inspecting a simple visual scene to locate a specific target. The rapid shifts of gaze that redirect the fovea of the retina, which provides high acuity vision, onto a new point in the image are called saccades. Saccades tend to direct gaze to conspicuous, apparently informative features in the scene. If a particular cognitive strategy is employed, then gaze focuses primarily on appropriate elements in the image; otherwise, gaze is dispersed (e.g., Yarbus 1967; Viviani 1990). Thus, normal visual behavior is accomplished through a cycle of fixation and visual analysis interrupted by saccadic eye movements. Figure 1B illustrates the variability of fixation durations over time. In this short period including just nine saccades between the stimuli, the fixation durations ranged from 85 to 320 msec. Similar fixation durations have been observed in humans performing a scanning visual search task (Hooge & Erkelens 1996), but when viewing more naturally engaging scenes fixation durations range from 200 to as much as 800 msec (Viviani 1990). Why are the fixation durations so long and variable?

While fixating a point in an image at least two processes take place. First, perceptual processing analyzes the object in the fovea to ascertain its identity and the image in the periphery to locate the target for the next saccade. Second, response preparation programs the saccade. These two processes occupy time (Figure 1C). Perceptual processing requires at least 100 msec (e.g., Salthouse et al 1981), and saccade programming, around 100-150 ms (Becker & Jurgens 1979; Lisberger, et al 1975; Hanes & Schall 1995). These estimates of minimal perceptual and response preparation delays do not account for the variability and length of natural fixation durations. Carpenter (1988) has called attention to this apparent procrastination, noting that much of the delay and variability of fixation durations is likely due to decision processes that intervene between the afferent and efferent delays. The main goal of this review is to provide some account of this decision delay by examining how neural correlates of perceptual and motor stages of processing relate to saccade latency.

NEURAL CORRELATES OF SACCADE TARGET SELECTION

Neural Correlates of Visual Selection

Many studies have now demonstrated visual selection processes in visual cortical areas. For instance, recent studies have demonstrated that neurons in visual cortex are sensitive to local stimulus irregularities (Knierem & Van Essen 1992) and texture gradients (Olavarria et al 1992; Zipser et al 1996). This process would serve to locate conspicuous points in the image. Other recent studies have shown how instructions of what stimulus property or visual field location to attend for an operant response modulate the responses of neurons in area V4 (Motter 1994; Luck et al 1997; Connor et al 1997), areas MT and MST (Treue & Maunsell 1996), inferior temporal cortex (Chelazzi et al 1993) and posterior parietal cortex (Robinson et al 1995; Steinmetz & Constantinidis 1995; Shadlen & Newsome 1996; Platt & Glimcher 1997; Gottlieb et al 1998).

How are signals in different visual areas of different streams combined to guide action? Models of visual attention incorporate a saliency map, a topographic representation of the visual field in which the locations of potential targets are registered (e.g., Treisman 1988; Cave & Wolfe 1990). The activation in the saliency map is derived in part from bottom-up processes that identify conspicuous elements in the image defined by the range of elementary features, e.g., color, form, motion, stereo. The activation in the saliency map also reflects top-down influences derived from goals and expectations. Anatomical and physiological data suggest that the frontal eye field (FEF) may be regarded as a saliency map.

Frontal Eye Field

FEF, located in the rostral bank of the arcuate sulcus in frontal cortex of macaques, participates in the transformation of visual computations guided by cognitive processes into saccade motor commands (reviewed by Schall 1998). FEF is heavily interconnected in a topographic fashion with areas in both the dorsal and ventral streams of extrastriate visual cortex (e.g., Schall et al 1995b). Ventrolateral FEF, which is responsible for generating short amplitude saccades, receives visual afferents from the foveal representation in retinotopically organized areas such as MT and V4, from areas that represent central vision in inferotemporal cortex such as TEO and caudal TE and from areas in parietal cortex having little retinotopic order such as LIP. In contrast, dorsomedial FEF, which is responsible for generating longer amplitude saccades, is innervated by the peripheral visual field representation of retinotopically organized areas, from areas that emphasize peripheral vision such as MSTd and PO, as well as from LIP. FEF is also innervated by areas in prefrontal cortex (Stanton et al 1993).

Physiological recordings in the FEF of monkeys trained in visual tracking tasks have found that roughly half of the neurons have visual responses (e.g., Mohler et al 1973; Bruce & Goldberg 1985; Schall 1991). The receptive fields are large, emphasizing the contralateral hemifield but occasionally extending into the ipsilateral. As previously observed in the superior colliculus, the response of the visual cells is enhanced if the stimulus is the target for a saccade (Goldberg & Bushnell 1981).

FEF plays a direct role in producing saccadic eye movements. Saccades are elicited by low intensity intracortical microstimulation of FEF (e.g., Bruce et al 1985). This direct influence is mediated by a subpopulation of neuron in FEF that discharges specifically before and during saccades (Bruce & Goldberg 1985; Hanes & Schall 1996) and innervate the superior colliculus (Segraves & Goldberg 1987) and the brainstem saccade generating circuit (Segraves 1992). Saccade amplitude is topographically mapped in FEF; shorter saccades are represented ventrolaterally with progressively longer saccades represented dorsomedially (Bruce et al 1985). Recent work has demonstrated that reversible inactivation of FEF impairs monkeys' ability to make saccades (Dias et al 1995; Sommer & Tehovnik 1997) and complements earlier observations that ablation of FEF causes an initial severe impairment in saccade production that recovers over time (e.g., Schiller et al 1987).

Target Selection in Frontal Eye Field During Visual Search

The visual search paradigm has been used extensively to investigate visual selection and attention (Treisman 1988; Egeth & Yantis 1997). The results of many experiments distinguish two modes of visual search. One mode is the effortless (pop-out) visual search for, say, a white spot among several black spots (Figure 2). The second mode is the effortful search for, say, a randomly oriented T among randomly oriented L's (Figure 1). To investigate how the brain selects targets for visually guided saccades, we have recorded the activity of neurons in the FEF of monkeys trained to shift gaze to the oddball target in either of two complementary pop-out visual search arrays (Figure 3A) (Schall et al 1995a; Bichot et al 1996; Thompson et al 1996 1997). Most visually responsive cells in FEF responded initially indiscriminately to the target or the distractor of the search array in their receptive field (Figure 3B). The absence of a feature selective response in FEF during visual search is consistent with earlier work (Mohler et al 1973). However, before saccades were generated, a discrimination process proceeded by which most visually responsive cells in FEF ultimately signaled the location of the oddball target stimulus. Thus, the activity of FEF visual neurons indexes if not participates in the evolution of the visual selection process. The movement-related activity in FEF was the same immediately before saccades to the target presented alone or with distractors (Hanes et al 1995; Schall et al 1995a). But this should not be surprising, because after all the same saccade is generated in both conditions. Similar observations have been made in monkeys scanning complex images (Burman & Segraves 1994).

The visual selection process observed in FEF had a number of interesting properties. First, the initial response to an array of stimuli was consistently attenuated relative to the initial response to a single stimulus in the receptive field; this has also been observed now in the superior colliculus (Basso & Wurtz 1997). The attenuation of the initial response may have been due to suppressive interactions arising from stimuli surrounding the receptive field or to a failure of enhancement due to uncertainty about the location of the target. Second, the selection occurred regardless of the visual feature (color or form) that distinguished the target from the distractors. Third, the target selection was achieved by a suppression or decay of the responses evoked by the distractor associated with a maintenance or enhancement of responses evoked by the target. Human attention studies have provided evidence for both distractor suppression and target facilitation (e.g., Maljkovic & Nakayama 1994; Cepeda et al 1998). Fourth, the magnitude of the distractor suppression in some cells reflected the proximity of the target to the receptive field, there being more suppression of the response to the distractor centered in the receptive field if the target flanked the receptive field. This center-surround organization of the visual selection observed in FEF is reminiscent of the receptive field organization observed in other cortical areas and subcortical structures (e.g., Allman et al 1985) and indicates that a primitive mechanism of sensory coding seems to be utilized in the guidance of eye movements in complex images.

Target Selection in Frontal Eye Field During Visual Masking

In the visual search paradigm selection of the target for the saccade is based on the contrast between the visual features of stimuli. To investigate whether the selection signals in FEF depend on salient visual differences, we recorded in monkeys making saccades to physically identical targets that were made intermittently perceptible by backward masking (Thompson & Schall 1998).

It is generally thought that visual responses in frontal cortex are rather specifically associated with action and awareness (e.g., Crick & Koch 1996; Schall 1998). Hence, we were surprised to find that visual neurons in FEF responded to undetected masked stimuli. This unexpected result challenges the view that neuronal responses to undetected stimuli that are masked by light do not get out of the retina (Breitmeyer 1984) and indicates that a psychophysical response criterion must be reached for target detection. In fact, we observed that monkeys shifted gaze to the masked stimulus when the initial visual response was slightly stronger than otherwise.

We also found that the saccade to the masked stimulus was preceded by prolonged selective activation of many FEF neurons. The selective activation of movement neurons but not visual neurons was related to programming the saccade. The selective pattern of activation of visual neurons preceding the saccade to a detected masked stimulus resembled the selective activation preceding the saccade to a visual search target. However, in visual neurons tested in both conditions we observed target selection activity during visual search but not during the visual masking task (Thompson & Schall, unpublished observation). This dissociation indicates that FEF neurons reflect different types of selection, one form of selection is apparently derived from visual salience and the other appears to be associated with selection of the behavioral response.

Effects of Experience on Target Selection

Many studies have demonstrated the powerful influence of cognitive strategies on eye movements (e.g., Yarbus 1967; Nodine et al 1996). We have made a serendipitous discovery that the neural selection process in FEF can be modified profoundly by differences in monkeys' search strategy (Bichot et al 1996). The correlates of target selection in FEF during visual search illustrated in Figure 2B were observed in monkeys trained to search for an oddball stimulus regardless of the particular visual feature that defined it (e.g., both a red target among green distractors and a green target among red distractors). The monkeys had learned to generalize a strategy of searching for the oddball stimulus. When monkeys are given exclusive experience with one visual search array (e.g., only red among green), they adopt the strategy of ignoring stimuli that were distractors, even when those same stimuli become the target when the array was switched (Figure 2C). We found that in monkeys at this stage of training, half of the visually responsive cells in FEF exhibited a suppressed response to the learned distractor as soon as the neurons responded (Figure 2D). In other words, a subpopulation of visual neurons exhibited feature selectivity in their initial responses unlike what had been observed before in FEF.

The time at which visual responses in frontal cortex are modulated in this and other experiments (e.g., Boussaoud & Wise 1993; Sakagami & Niki 1994) imposes constraints on possible mechanisms. For the initial response of FEF neurons to be modulated according to whether the target or a distractor was in the receptive field requires that the color of the stimulus in the receptive field be discriminated before the FEF neuron responds. The visual latencies at which we observed the selective modulation in FEF ranged from 60-80 msec which are comparable to the visual response latencies of color-selective neurons in visual cortex (Nowak & Bullier 1998; Schmolesky et al 1998). However, when monkeys search for a target among arrays of 5 to 8 elements, neurons in V4 exhibit attentional modulation no earlier than 100 ms (Motter 1994; Luck et al 1997); this is too late to influence the initial visual response of FEF neurons. If an on-line attentional process is too slow to mediate the induced visual selectivity of FEF neurons for the learned target, then the modulation may be a result of reduced efficacy of synapses conveying the original response to the distractor stimuli. In other words, the experience-induced visual feature selectivity in FEF may represent a modified representation in a feature map similar to the experience-dependent expansions of representations in topographic maps consequent to selective stimulation of a part of the sensory surface or particular motor effectors (Weinberger 1995). Further work is needed to examine the neural responses in visual areas conveying visual signals to FEF to determine at what stage and time this learning effect is manifest.

Timecourse of Target Discrimination

Reaction time is one of the basic measures of experimental psychology. The capriciousness of reaction times was noted by its earliest investigators. "Everyone who makes reaction-time experiments for the first time is surprised to find how little he is master of his own movements... Not only does their energy lie, as it were, outside the field of choice, but even the time in which the movement occurs depends only partly upon ourselves." (Exner 1873). The variability of reaction time is not a laboratory artifact, for as described above, fixation durations are variable during natural scanning eye movements. Nor is reaction time variability unique to complex nervous systems; even the primitive jellyfish exhibits variable reaction times (Yerkes 1903).

A fundamental premise is that behavioral response times are composed of stages of processing (Donders 1868; Sternberg 1969). Most simply, a perceptual stage identifies and encodes stimuli while a response stage prepares and executes movements. Distinguishing separate stages and determining how their duration relates to reaction time has been a central problem for decades (e.g., Posner 1978; Luce 1986; Meyers et al 1988). Unfortunately, the conclusions derived from measures of performance alone must be inferential because they lack markers for the end of one stage and the beginning of another. Physiological measures can provide such markers. For instance, the P300 component of event-related potentials (ERP) represents the termination of perceptual processing, and the lateralized readiness potential (LRP) represents response preparation (e.g., Coles et al 1995). However, without knowledge of the neural generators of ERPs the conclusions drawn from these studies are somewhat limited. Psychological stages of processing can be investigated with the highest spatial and temporal resolution by recording the activity of single neurons.

Having found that saccade target selection by FEF neurons occurred over time in the monkeys trained on the general visual search task (Figure 2B), we performed an analysis to determine when the selection was accomplished (Thompson et al 1996). By partitioning the reaction time of monkeys this way, we could examine the sources of the variability of saccade reaction times. We found that most FEF neurons with sustained visual activity selected the target for the saccade a fairly constant interval after stimulus presentation (Figure 3). In other words, the time when the target of the saccade was identified did not predict the time when the eyes moved. In fact, not only does the visual selection in FEF occur independently of when a saccade is initiated, the selection occurs even if no saccade is planned (Thompson et al 1997).

The observation that the target discrimination of FEF neurons can occur independently of saccade production indicates, in general terms, that the activity of most visually responsive FEF cells signals the outcome of perceptual processing. If this is so, then the timecourse of this neural discrimination process would seem to correspond to the timecourse of perceptual processing. The fact that the time of saccade initiation was not predicted by the time of target discrimination by FEF neurons indicates that reaction time is not dictated by the duration of perceptual processing. From this we conclude that a significant fraction of the variability in reaction time arises in post-perceptual response preparation (see also Coles et al 1995). Therefore, to understand the variability in reaction time and saccade fixation durations, we now consider the response preparation process.

NEURAL CONTROL OF SACCADE INITIATION

Rise to Threshold Mechanism for Reaction Time

Over the years many models have been developed to explain the stochastic variability of reaction time (Luce 1986), only a fraction of which are amenable to evaluation in relation to brain function. One such class of models supposes that in response to a stimulus, a signal in the brain grows until it reaches a threshold at which point a movement is produced. In models of this sort there are at least two sources for stochastic variability in reaction time. According to one type of accumulator model the variability in reaction time arises from randomness in the level of the trigger threshold (e.g., Grice et al 1982). According to another type of accumulator model the threshold is constant, but the rate of growth of the accumulator is random across trials (e.g., Carpenter 1988; Ratcliff et al 1998). Both architectures can account for reaction time data distributions under a range of conditions (e.g., Nazir & Jacobs 1991; Carpenter & Williams 1995; Ratcliff et al 1998); indeed, it has been shown mathematically that random accumulator and random threshold models generate equivalent predictions (Dzhafarov 1993).

We have recently examined movement-related activity recorded in FEF to evaluate the alternative models of reaction time (Hanes & Schall 1996). We found clear evidence that saccades were initiated when movement-related activity in FEF reached a particular level, and this level did not vary with saccade latency (Figure 4). The same conclusion has been drawn from a similar analysis of LRP amplitude (Gratton et al 1988). In the FEF data the variability in reaction time was accounted for by variation in the rate of growth of the premovement activity, which began at a fairly constant interval after target presentation, towards the threshold. Accordingly, the movement-related neural activity in FEF corresponds to an accumulator model architecture with variable growth to a fixed threshold and directly contradicts the architecture with a fixed growth process and random threshold.

Why is the growth of movement-related activity variable? Does it simply reflect the best a nervous system can do? Perhaps there has been no evolutionary selection pressure to react nonreflexively in a faster and more reliable fashion; after all, over the life of a primate what difference would it make whether gaze shifts 50 msec sooner or later? On the other hand, the world is an ever-changing place; an action chosen at one instant may in the next become a bad choice. Variability in the response production system can permit the perceptual system to naturally countermand a planned movement that has become maladaptive. Hence, the variability in behavioral responding may have been selected for through evolution to permit flexible, adaptive, creative behavior.

What is the logical relationship between the movement activity of neurons in FEF and saccade production? It is well known that many neurons across multiple structures contribute to each movement (Lee et al 1988; Georgopoulos 1996). Therefore, the activity of a single movement neuron is not necessary for movement production. Nevertheless, the data do indicate that the activity of single neurons in FEF are very accurate indices of the state of preparation of the whole oculomotor system. In fact, the distribution of saccade latencies produced by monkeys while a single movement cell was recorded could be reconstructed with a simple linear rise to threshold simulation run with parameters derived from that single neuron. Therefore, the activity of a single movement neuron appears to be sufficient to account for when movements are produced.

Does the same relationship between the growth of movement-related neural activity and the time of movement initiation hold for neurons in other motor structures? Recordings in motor cortex (Lecas et al 1986) and superior colliculus (Sparks 1978; Dorris et al 1997) indicate probably so. From this, we can infer that each neuron that contributes to a given movement may have an idiosyncratic threshold, but the rate of growth of activity to reach that threshold is correlated within and across structures so that there is temporal agreement on when to move. While this is a matter for empirical verification, this inference has important implications for the functional organization of the neural motor system, chief among them involves the question of how coordinated growth of movement-related activity across the brain might be achieved. One possibility is that interactions between oculomotor structures coordinate the growth of movement activation such that neurons that lag behind are accelerated and those that speed ahead are decelerated. Another possibility is that an external source such as the cholinergic or catecholaminergic systems influence the state of activation of the whole system.

The foregoing speculations rest on the validity of the hypothesis that movements are produced when motor activity reaches a fixed threshold. Further research has tested the validity of the threshold conception by comparing neural activity when saccades were either made or withheld after different degrees of preparation.

The Countermanding Paradigm

To investigate the neural control of movement initiation, we have implemented a behavioral paradigm with behaving monkeys, referred to as the countermanding paradigm that was originally developed to investigate human performance (reviewed by Logan 1994). The countermanding paradigm probes a subject's ability to control the production of movements in a reaction time task that infrequently presents an imperative stop signal. In the oculomotor version, monkeys were trained to make a saccade to a peripheral target unless an stop signal was presented in which case they must withhold the movement; the stop signal was the reappearance of the fixation spot (Hanes & Schall 1995). Performance on this task can be accounted for by a race between a process that generates the movement and a process that inhibits the movement (Logan 1994). This race model provides an estimate of the time needed to cancel the planned movement which is referred to as the stop signal reaction time. Oculomotor stop signal reaction times average around 100 ms in monkeys (Hanes & Schall 1995) and are slightly longer in humans (Hanes & Carpenter 1998). The stop signal reaction time corresponds theoretically and quantitatively to estimates of the time needed to reprogram a saccade in double-step saccade tasks (Lisberger et al 1975; Becker & Jurgens 1979).

Gaze Control Signals in Frontal Eye Field

The chief virtue of the countermanding paradigm is that one can determine whether single neurons generate signals that are logically sufficient to control the production of movements. The logic of the countermanding paradigm establishes two criteria a neuron must meet to play a direct role in the control of movement. First and most obviously the neuron must discharge differently when a saccade is initiated versus when a saccade is withheld. Second and most importantly, the difference in activity must occur by the time that the movement is canceled, i.e., the stop-signal reaction time.

We applied this approach to neural activity recorded in FEF (Hanes et al 1998). The main finding was that movement-related activity in FEF, which began to grow toward the trigger threshold, failed to reach the threshold activation when movements were canceled but instead decreased rapidly after the stop signal was presented (Figure 5). Moreover, the movement-related activity differentiated between execution and inhibition of the movement before the stop signal reaction time had elapsed. Therefore, according to the logic of the countermanding paradigm, the activity of this single neuron was logically sufficient to specify whether or not the saccade would be produced. This pattern of results was observed in all cells with movement-related or fixation-related activity but was not observed in neurons with only visual responses. The different results for the different functional classes of neurons is entirely consistent with the fact that movement and fixation neurons in FEF directly innervate efferent oculomotor structures but visual neurons do not (Segraves & Goldberg 1987; Segraves 1992).

The speed of the stopping process in this task deserves comment. The latency of the visual response to the foveal stop signal is no less than 50 ms. Given the 100 ms stop signal reaction time, this means that the movement was actually canceled in just 50 ms which amounts to just 5 action potentials in a neuron firing 100 spikes/sec. The potency of the stop signal is probably because it was a flash of a light in the fovea that directly activated the gaze fixation system (e.g., Munoz & Wurtz 1993). Further research with nonfoveal or nonvisual stop signals is clearly warranted.

These results demonstrate that the countermanding paradigm can provide more refined insights into the neural control of gaze. Now that the validity of the countermanding paradigm has been established, it can be applied to investigate brain areas in which the relation of neural activity to gaze control is not quite as clear, such as posterior parietal cortex (e.g., Colby et al 1996; Mazzoni et al 1996) or the supplementary eye field, a second area in frontal cortex that is involved in saccade production (Schall 1998). Preliminary data collected with the countermanding paradigm in the supplementary eye field indicates that unlike FEF the neurons do not generate signals sufficient to control gaze but instead appear to monitor performance, registering correct or errant responses (Patterson & Schall 1997).

CONCLUSION

A goal of this review was to examine how perceptual and motor decision processes contribute to the variability in reaction time. Two hypotheses linking neural and cognitive processes were suggested. The visual target selection process observed in FEF corresponds to the perceptual processing stage. The growth of activity of FEF movement neurons corresponds to the response preparation stage. The identification of neural concomitant of these processing stages in FEF led to the conclusion that while the perceptual processing stage contributes to the duration of reaction time, the response preparation stage is responsible for much of the variability.

The premise that reaction times are occupied by successive stages of processing has led to a fundamental question; does information flow discretely or continuously from one stage to the next (Meyer et al 1988)? If the linking hypotheses we propose are correct, then this question can be addressed empirically by recording from multiple single neurons. With continued work recording neural activity in a range of brain areas using comparable tasks and displays, we can anticipate a fuller understanding of the brain processes responsible for simple decision making. We hope that the mechanisms thus revealed may be applicable to more interesting and demanding decisions.

Acknowledgments: We thank N. Bichot, D. Hanes, S.C. Rao and T. Taylor for their contributions to this work. Our research has been supported by R01-EY08890, R01-MH55806, P30-EY08126, the McDonnell-Pew Program in Cognitive Neuroscience and by the McKnight Endowment Fund for Neuroscience. JDS is a Kennedy Center Investigator.

Literature Cited

Allman JM, Miezin F, McGuinness E. 1985. Stimulus specific responses from beyond the classical receptive field: Neurophysiological mechanisms for local-global comparisons of visual motion. Annu. Rev. Neurosci. 3:532-48

Basso MA, Wurtz RH. 1997. Modulation of neural activity by target uncertainty. Nature 389:66-9

Becker W, Jurgens R. 1979. An analysis of the saccadic system by means of double step stimuli. Vis. Res. 19:967-83

Bichot NP, Schall JD, Thompson KG. 1996. Visual feature selectivity in frontal eye fields induced by experience in mature macaques. Nature 381:697-9

Boussaoud D, Wise SP. 1993. Primate frontal cortex: neuronal activity following attentional versus intentional cues. Exp. Brain. Res. 95:15-27

Breitmeyer BG. 1984. Visual Masking: An Integrative Approach New York: Oxford University Press

Bruce CJ, Goldberg ME. 1985. Primate frontal eye fields. I. Single neurons discharging before saccades. J. Neurophysiol. 53:603-35

Bruce CJ, Goldberg ME, Bushnell C, Stanton GB. 1985. Primate frontal eye fields. II. Physiological and anatomical correlates of electrically evoked eye movements. J. Neurophysiol. 54:714-34

Burman DD, Segraves MA. 1994. Primate frontal eye field activity during natural scanning eye movements. J. Neurophysiol. 71:1266-71

Bushnell MC, Goldberg ME, Robinson DL. 1981. Behavioral enhancement of visual responses in monkey cerebral cortex. I. Modulation in posterior parietal cortex related to selective visual attention. J. Neurophysiol. 46:755-72

Carpenter RHS. 1988. Movements of the Eyes. London: Pion

Carpenter RHS. 1991. Eye Movements. London: Macmillan

Carpenter RHS, Williams MLL. 1995. Neural computation of log likelihood in the control of saccadic eye movements. Nature 377:59-62

Cave KR, Wolfe JM. 1990. Modeling the role of parallel processing in visual search. Cog. Psychol. 22:225-71

Cepeda NJ, Cave KR, Bichot NP, Kim M-S. 1998. Spatial selection via feature-driven inhibition of distractor locations. Percept. Psychophys. In press

Chelazzi L, Miller EK, Duncan J, Desimone R. 1993. A neural basis for visual search in inferior temporal cortex. Nature 363:345-7

Colby CL, Duhamel, J-R, Goldberg ME. 1996. Visual, presaccadic and cognitive activation of single neurons in monkey lateral intraparietal area. J. Neurophysiol. 76:2841-52

Colby CL, Duhamel J-R. 1991. Heterogeneity of extrastriate visual areas and multiple parietal areas in the macaque monkey. Neuropsychologia 29:517-37

Coles MGH, Smid HGOM, Scheffers MK, Otten LJ. 1995. Mental chronometry and the study of human information processing. In Electrophysiology of Mind: Event-Related Brain Potentials and Cognition, ed. MD Rugg, MGH Coles, pp 86-131. Oxford: Oxford University Press

Connor CE, Preddie DC, Gallant JL, Van Essen DC. 1997. Spatial attention effects in macaque area V4. J. Neurosci. 17:3201-14

Crick F, Koch C. 1995. Are we aware of neural activity in primary visual cortex? Nature 375:121-3

Desimone R, Duncan J. 1995. Neural mechanisms of selective visual attention. Ann. Rev. Neurosci. 18:193-222

Dias EC, Kiesau M, Segraves MA. 1995. Acute activation and inactivation of macaque frontal eye field with GABA-related drugs. J. Neurophysiol. 74:2744-8

Donders FC. 1868. On the speed of mental processes. Trans. WG Koster, 1969, in Attention and Performance II pp. 412-431. Amstterdam:L North-Holland Publishing Co.

Dorris MC, Paré M, Munoz DP. 1997. Neuronal activity in monkey superior colliculus related to the initiation of saccadic eye movements. J. Neurosci. 17:8566-79

Dzhafarov, EN. 1993. Grice-representability of response time distribution families. Psychometrika 58:281-314

Egeth HE, Yantis S. 1997. Visual attention: Control, representation, and time course. Ann. Rev. Psychol. 48:269-97

Exner S. 1873. Experimentelle Untersuchung der einfachsten psychischen Processe. Pflüger's Archiv. für gesamte Physiologie 7:601-660. Transl. W. James, 1890/1950 The Principles of Psychology p. 92. New York: Dover Publications, Inc.

Georgopoulos AP. 1996. Arm movements in monkeys: behavior and neurophysiology. J. Comp. Physiol. [A] 179:603-12

Goldberg ME, Bushnell MC. 1981. Behavioral enhancement of visual responses in monkey cerebral cortex. II. Modulation in frontal eye fields specifically related to saccades. J. Neurophysiol. 46:773-87

Gottlieb JP, Kusunoki M, Goldberg ME. 1998. The representation of visual salience in monkey parietal cortex. Nature 391:481-3

Gratton G, Coles MGH, Sirevaag EJ, Eriksen CJ, Donchin E. 1988. Pre- and poststimulus activation of response channels: A psychophysiological analysis. J. Exp. Psychol.: Hum. Percept. Perform. 14:331-44

Grice GR, Nullmeyer R, Spiker VA. 1982. Human reaction time: Toward a general theory. J. Exp. Psych.: Gen. 111:135-53

Hanes DP, Carpenter RHS. 1998. Countermanding saccades in humans: Evidence for a race-to-threshold process. Vision Res. In press

Hanes DP, Schall JD. 1995. Countermanding saccades in macaque. Vis. Neurosci. 12:929-37

Hanes DP, Schall JD. 1996. Neural control of voluntary movement initiation. Science 274:427-30

Hanes DP, Patterson WF, Schall JD. 1998. The role of frontal eye field in countermanding saccades: Visual, movement and fixation activity. J. Neurophysiol. In press

Hanes DP, Thompson KG, Schall JD. 1995. Relationship of presaccadic activity in frontal eye field and supplementary eye field to saccade initiation in macaque: Poisson spike train analysis. Exp. Brain Res. 103:85-96

Hooge ITC, Erkelens CJ. 1996. Control of fixation duration in a simple search task. Percept. Psychophys. 58:969-76

Knierim JJ, Van Essen DC. 1992. Neuronal responses to static texture patterns in area V1 of the alert macaque monkey. J. Neurophysiol. 67:961-80

Lecas J-C, Requin J, Anger C, Vitton N. 1986. Changes in neuronal activity of the monkey precentral cortex during preparation for movement. J. Neurophysiol. 56:1680-1702

Lee C, Rohrer WH, Sparks DL. 1988. Population coding of saccadic eye movements by neurons in the superior colliculus. Nature 332:357-60

Lisberger SG, Fuchs AF, King WM, Evinger LC. 1975. Effect of mean reaction time on saccadic responses to two-step stimuli with horizontal and vertical components. Vis. Res. 15:1021-5

Logan GD. 1994. On the ability to inhibit thought and action: A users' guide to the stop signal paradigm. In: Inhibitory Processes in Attention, Memory and Language, ed. D Dagenbach, TH Carr, pp 189-239. San Diego: Academic Press

Luce RD. 1986. Response Times: Their Role in Inferring Elementary Mental Organization. Oxford: Oxford University Press

Luck SJ, Chelazzi L, Hillyard SA, Desimone R. 1997. Neural mechanisms of spatial selective attention in areas V1, V2, and V4 of macaque visual cortex. J. Neurophysiol. 77:24-42

Maljkovic V, Nakayama K. 1994. Priming of pop-out: I. Role of features. Memory & Cognition 22:657-72

Maunsell JHR. 1995. The brain's visual world: Representations of visual targets in cerebral cortex. Science 270:764-68

Mazzoni P, Bracewell RM, Barash S, Andersen RA. 1996. Motor intention activity in the macaque's lateral intraparietal area. I. Dissociation of motor plan from sensory memory. J. Neurophysiol. 76:1439-56

Merigan WH, Maunsell JHR. 1993. How parallel are the primate visual pathways? Annu. Rev. Neurosci. 16:369-402

Meyer DE, Osman AM, Irwin DE, Yantis S. 1988. Modern mental chronometry. Biol Psychol 26:3-67

Mohler CW, Goldberg ME, Wurtz RH. 1973. Visual receptive fields of frontal eye field neurons. Brain Res. 61:385-9

Motter BC. 1994. Neural correlates of attentive selection for color or luminance in extrastriate area V4. J. Neurosci. 14:2178-89

Munoz DP, Wurtz RH. 1993. Fixation cells in monkey superior colliculus. I. Characteristics of cell discharge. J. Neurophysiol. 70:559-575

Nazir TA, Jacobs AM. 1991. The effects of target discriminability and retinal eccentricity on saccade latencies: An analysis in terms of variable-criterion theory. Psychol. Res. 53:281-9

Nodine CF, Kundel HL, Lauver SC, Toto LC. 1996. Nature of expertise in searching mammograms for breast masses. Acad. Radiol. 3:1000-6

Nowak, Bullier. 1998. The timing of information transfer in the visual system. in Extrastriate Cortex of Primates, Volume 12 of Cerebral Cortex, ed. K Rockland, A Peters, J Kaas, pp 205-41. New York: Plenum

Olavarria JF, DeYoe EA, Knierim JJ, Fox JM, Van Essen DC. 1992. Neural responses to visual texture patterns in middle temporal area of the macaque monkey. J. Neurophysiol. 68:164-81

Parker AJ, Newsome WT 1998. Sense and the single neuron: Probing the physiology of perception. Annu. Rev. Neurosci. 21:000-000

Patterson WF, Schall JD. 1997. Supplementary eye field studied with the countermanding paradigm. Soc. Neurosci. Abstr. 23:474

Platt ML, Glimcher PW. 1997. Responses of intraparietal neurons to saccadic targets and visual distractors. J. Neurophysiol. 78:1574-89

Posner MI. 1978. Chronometric Explorations of Mind, Hillsdale, NJ: L. Erlbaum Associates.

Ratcliff R, Van Zandt T, McKoon G. 1998. Connectionist and diffusion models of reaction time. Psychol. Rev. In press

Robinson DL, Bowman EM, Kertzman C. 1995. Covert orienting of attention in macaques. II. Contributions of parietal cortex. J. Neurophysiol. 74:698-712

Salthouse TA, Ellis CL, Diener DC, Somberg BL. 1981. Stimulus processing during eye fixations. J. Exp. Psychol.: Human Percept. Perform. 7:611-23

Sakagami M, Niki H. 1994. Encoding of behavioral significance of visual stimuli by primate prefrontal neurons: relation to relevant task conditions. Exp. Brain Res. 97:423-36

Schall JD. 1991a. Neuronal activity related to visually guided saccades in the frontal eye fields of rhesus monkeys: Comparison with supplementary eye fields. J. Neurophysiol. 66:559-79

Schall JD. 1991b. Neuronal basis of saccadic eye movements. In The Neural Basis of Visual Function, ed. AG Leventhal, pp 388-442. London: Macmillan Press

Schall JD. 1995. Neural basis of saccade target selection. Rev. Neurosci.. 6:63-85

Schall JD. 1998. Visuomotor areas of the frontal lobe. In Extrastriate Cortex of Primates, Volume 12 of Cerebral Cortex, ed. K Rockland, A Peters, J Kaas, pp 527-638 New York: Plenum

Schall JD, Bichot NP. 1998. Neural correlates of visual and motor decision processes. Curr. Opin. Neurobiol. In press

Schall JD, Hanes DP, Thompson KG, King DJ. 1995a. Saccade target selection in frontal eye field of macaque. I. Visual and premovement activation. J. Neurosci. 15:6905-18

Schall JD, Morel A, King DJ, Bullier J. 1995b. Topography of visual cortical afferents to frontal eye field in macaque: Convergence and segregation of processing streams. J. Neurosci. 15:4464-87

Schiller PH, Sandell JH, Maunsell JHR. 1987. The effect of frontal eye field and superior colliculus lesions on saccadic latencies in the rhesus monkey. J. Neurophysiol. 57:1033-49

Schmolesky MT, Wang YC, Hanes DP, Thompson KG, Leutgeb S, Schall JD, Leventhal AG. 1998. Signal timing across the macaque visual system. J. Neurophysiol. In press

Segraves MA. 1992. Activity of monkey frontal eye field neurons projecting to oculomotor regions of the pons. J. Neurophysiol. 68:1967-85

Segraves MA, Goldberg ME. 1987. Functional properties of corticotectal neurons in the monkey's frontal eye fields. J. Neurophysiol. 58:1387-1419

Sommer MA, Tehovnik EJ. 1997. Reversible inactivation of macaque frontal eye field. Exp. Brain Res. 116:229-49

Shadlen MN, Newsome WT. 1996. Motion perception: Seeing and deciding. Proc. Natl. Acad. Sci. USA 93:628-33

Sparks DL. 1978. Functional properties of neurons in the monkey superior colliculus: Coupling of neuronal activity and saccade onset. Brain Res. 156:1-16

Stanton GB, Bruce CJ, Goldberg ME. 1993. Topography of projections to the frontal lobe from the macaque frontal eye fields. J. Comp. Neurol. 330:286-301

Steinmetz MA, Constantinidis C. Neurophysiological evidence for a role of posterior parietal cortex in redirecting visual attention. Cerebral Cortex 5:448-56

Sternberg S. 1969. The discovery of processing stages: Extensions of Donders' method. Acta Psychologica 30:276-315

Thompson KG, Schall JD. 1998. Selection for action in macaque prefrontal cortex during visual masking. Science In press

Thompson KG, Hanes DP, Bichot NP, Schall JD. 1996 Perceptual and motor processing stages identified in the activity of macaque frontal eye field neurons during visual search. J. Neurophysiol. 76:4040-55

Thompson KG, Bichot NP, Schall JD. 1997. Dissociation of target selection from saccade planning in macaque frontal eye field. J. Neurophysiol. 77:1046-50

Treisman A. 1988. Features and objects: The Fourteenth Bartlett Memorial Lecture. Q. J. Exp. Psychol. 40A:201-37

Treue S, Maunsell JHR. 1996. Attentional modulation of visual motion processing in cortical areas MT and MST. Nature 382:539-41

Viviani P. 1990. Eye movements in visual search: Cognitive, perceptual and motor control aspects. In Eye Movements and their Role in Visual and Cognitive Processes, ed. E Kowler, pp 353-93. New York: Elsevier

Weinberger N. 1995. Dynamic regulation of receptive fields and maps in the adult sensory cortex. Annu. Rev. Neurosci. 18:129-58

Wurtz RH, Goldberg ME, eds. 1989. The Neurobiology of Saccadic Eye Movements. New York: Elsevier

Yarbus AL. 1967. Eye Movements and Vision. New York: Plenum

Yerkes RM. 1904. Variability of reaction-time. Psych. Bull. 1:137-46

Zipser K, Lamme VAF, Schiller PH. 1996. Contextual modulation in primary visual cortex. J. Neurosci. 16:7376-89

FIGURE LEGENDS

Figure 1. A. Pattern of gaze shifts made by a monkey searching for a randomly oriented T among L's. The T among L array appeared after the monkey was fixating the central spot. On this trial the monkey's first saccade was to the left, followed by a sequence of eye movements around the perimeter of the array. B. The same sequence of saccades plotted as horizontal (thick) and vertical (thin) eye position as a function of time. C. A segment of the scanpath including the first saccade to the target is expanded to illustrate the timecourse of perceptual processing and of response preparation. These processes precede each saccade.

Figure 2. Relation of FEF target selection to cognitive strategy. Illustration of the saccades made by two monkeys in response to complementary visual search arrays. The performance shown in A was collected from a monkey that had been trained on both versions of the visual search array. The performance shown in B was collected from a monkey that had been trained on just one instance of the visual search array. When presented an array in which the target and distractor colors were switched, instead of looking at the salient oddball stimulus, this monkey looked at a distractor that was the same color as the target in the complementary array. The timecourse of activation of single FEF neurons during the visual search task when the target was in the receptive field (solid line) and when distractors were in the receptive field (dotted line) is shown for a monkey trained to search for the salient oddball (C) and from a monkey that directed gaze only to stimuli of a particular color (D). The arrows indicate the period in which saccades were produced. (Modified from Bichot et al, 1996 and Thompson et al, 1996)

Figure 3. Relationship between the time of target discrimination in a FEF neuron (lower panels) and the time of saccade initiation (upper panel). The timecourse of activation of a single FEF neuron during the visual search task when the target was in the receptive field (solid line) and when distractors were in the receptive field (dotted line) is shown for subsets of trials in which saccade latency was short (middle panel) or long (bottom). The times of target discrimination (solid arrows) were approximately the same in both subsets of trials and therefore does not account for the range of saccade latencies.

Figure 4. Relationship between movement-related FEF activity and saccade initiation. Timecourse of activation of a single movement-related FEF neuron is shown for three subsets of trials having different saccade latencies. Plots are aligned on target presentation and stop at saccade initiation; eye movement traces are illustrated for each group. The level of activity at which the saccade is triggered (gray bar) is fairly constant across saccade latencies. Variability in saccade latency is accounted for by the time taken by the neuron to reach the threshold activation.

Figure 5. Movement-related FEF activity when saccades are initiated or canceled. The countermanding task consists of many No Stop Signal trials in which the fixation disappearance and target step is followed by a visually guided saccade. Stop Signal trials occur at random when the target step is followed by the reappearance of the fixation spot (solid vertical line). Stop signal reaction time is the time needed cancel the planned movement in reaction to the stop signal (dashed vertical line). Activity on trials when no stop signal was presented (thin line) is compared with activity on trials when the planned saccade was canceled because the stop signal appeared (thick line). The activity when the movement was canceled decayed precipitously immediately before the stop signal reaction time. (Modified from Hanes et al, 1998)