Shape from Temporal Synchrony or From Static Luminance Cues?

S.H-Lee & R. Blake

Is shape from temporal synchrony produced by luminance cues created by temporal filtering? This is the conjecture offered by Adelson and Farid in their Technical Comment on our recently published paper "Visual form created solely from temporal structure"? The animation sequences on this page show versions of our temporal synchrony animations and the strength of the putative luminance cue produced by filtering individual frames of those sequences through the temporal integration filter described by Adelson and Farid.

Original  Animation in which Gabor elements in the "figure" all change direction irregularly but synchronously over time; Gabor elements in the background also change irregularly and synchronously. The points in time at which changes occur are uncorrelated within figure and background. This corresponds to the 100/100 condition in our original paper.

The images shown below were derived from six successive frames of the original temporal synchrony animation above. Each image shown below was produced by convolving a frame from the "original" with a lowpass temporal filter that integrates pixel luminance values over multiple, successive frames; filter parameters are identical to those used byAdelson and Farid in their Technical Comment. These static images confirm that "runs" and "jitter" can - in principle - create luminance-defined shape in individual frames of the sequence. These images do not constitute evidence that observers use this cue nor, for that matter, that human vision creates it in the first place.

The series of animations shown below were created to minimize the incidence of "runs" and "jitters" (pruned sequence), to mask the presence of the luminance cue (randomized contrast and randomized luminance) and, finally, to combine pruning with randomization to produce a sequence in which static luminance is completely eliminated as a potential cue. It should be stressed that these manipulations of our original sequence do not eliminate temporal synchrony as a cue for grouping, but they do introduce conflicting grouping cues (contrast and luminance randomization) and, in the case of the pruned sequence, they reduce the entropy (richness of temporal structure) in the new sequences. Despite conflicting cues and reduced entropy, temporal synchrony still creates shape.

Pruned Animation Sequence  Animation in which "runs" (sequences of "no change") and "jitter" (sequences of repetitive change) have been pruned from a temporal synchrony animation. Shape from temporal synchrony remains visible.

The sequence of six images shown below were produced by filtering the "pruned" sequence seen above through the same lowpass temporal filter that "uncovered" the luminance artifacts in the original sequences. These six images, the first in the sequence, are representative of the filter's output with "pruned" inputs.

Randomized Contrast Animation Sequence  Animation in which the contrast of each Gabor element is randomized from frame to frame of an unpruned temporal synchrony animation. Shape from temporal synchrony remains visible.

The images shown below were produced by filtering the "contrast randomized" sequence seen above through the same lowpass temporal filter that "uncovered" the luminance artifacts in the original sequences. These six images, the first in the sequence, are representative of the filter's output with "randomized contrast" inputs.

Randomized Luminance Animation Sequence  Unpruned animation in which the average luminance of each Gabor element is set to a randomly determined value and maintained at that value throughout the sequence. Shape from temporal synchrony remains visible.

The images shown below were produced by filtering the "luminance randomized" sequence seen above through the same lowpass temporal filter that "uncovered" the luminance artifacts in the original sequences. These six images, the first in the sequence, are representative of the filter's output with "randomized luminance" inputs.

Pruned & Randomized Animation Sequence  Pruned sequence with contrast and luminance randomized. Shape from temporal synchrony remains visible.

The images shown below were produced by filtering the "pruned/randomized" sequence seen above through the same lowpass temporal filter that "uncovered" the luminance artifacts in the original sequences. These successive images confirm that eliminating "runs" and "jitter" and randomizing contrast and luminance preclude recovery of shape from luminance in individual frames of the animation.

These demonstrations undermine the conjecture that temporal integration alone can "explain the perception of form" in these stochastic displays. Instead, the demonstrations add further evidence that human vision contains mechanisms that preserve the temporal fine structure in dynamic events, structure that operates in the interests of spatial grouping.

We thank Adelson and Farid for their thoughtful comments and for sharing their filtering code with us. Their stimulating reactions led us to pursue ideas that we might have otherwise overlooked.