PHENOMENAL REPORTS

Faces behind bars: illusory eye movements induced by gratings

Benjamin Balas^*

Department of Psychology, Center for Visual and Cognitive Neuroscience, North Dakota State University, Fargo, ND, USA

Abstract

I describe a novel illusion in which perceived eye movements are induced by moving a vertical grating across a single image of a forward-looking face. By varying properties of the grating, a wide range of illusory eye movements can be generated including nystagmus, a ‘swirling’ motion of the eyes, and vertical scanning/blinking. I suggest that the phenomenon is closely related to the footsteps illusion, but reveals the role that object shape and grating spatial frequency together play in determining the direction of illusory motion that observers perceive. I also discuss the relationship between the current illusion, the footsteps illusion, and Moire pattern animations.

Keywords: Face perception; eye movements; Footsteps illusion; motion perception; spatial frequency

Edited by: Dejan Todorovic´ University of Belgrade, Serbia

Reviewed by: Takahiro Kawabe NTT Communication Science Laboratories, Japan

Akiyoshi Kitaoka Ritsumeikan University, Japan

In Barrier-Grid animation (also sometimes referred to as ‘Picket-Fence animation’), a single static image is turned into a short, looping animated sequence by moving an overlaid grating across the image (Hopwood, 1899). This technique, most commonly marketed as ‘Scanimation’, (Seder, 2007) typically depends on constructing the base image by interlacing multiple images that make up the animated sequence. Specifically, by sampling each frame of the animation with vertical stripes at a spatial frequency consistent with the overlaid grating, the samples from each frame can be offset in phase such that frame visibility depends on the position of the overlaid grating and that each frame becomes visible in a fixed sequence. The result is that the sampling of the interlaced image by the grating reveals new frames as the grating moves, leading to a simple animated sequence (Fig. 1).

Fig. 1.  A transparent overlay of vertical stripes applied to this image and moved left-to-right will yield an animation of a spinning cube. The periodic vertical sampling of the constituent frames in this image is evident in the striped structure of this base image. (Attribution: Pixelmaniac pictures, Public domain, via Wikimedia Commons).

I present an illusion that I suggest is related both to this phenomenon and the Foosteps illusion (#REF), but also differs in interesting ways that reveal new properties of illusory motion perception. The current illusion involves animating a single static image via an overlaid grating, but does not require the careful interlacing of multiple images into a single composite as in Barrier-Grid animation. Instead, the spatial properties of a single natural image are used as the basis for a range of different animated sequences by varying properties of the overlaid grating.

The basic parameters of the illusion are illustrated in Fig. 2. The static image that serves as the basis for the effect is a single forward-looking face. I’ve selected an average face for this demonstration, but the effect is robust to the choice of specific face image. As in most Barrier-grid animations, the overlaid grating is a set of vertical stripes. By moving this grating across the face (Movie 1 – gratingFaceIllusion_Nystagmus.mp4), we see that nystagmus-like eye movements (horizontal scanning) are induced in the face, even though there are no explicit frames depicting the eyes in different positions included in the base image. Further, by changing properties of the grating, different illusory eye movements can be generated. For example, in Movie 2 (gratingFaceIllusion_Swirling.mp4) I have applied a vertical grating that has a slightly different spatial frequency and also has closer spacing of the black stripes in the grating. The result is a sort of ‘swirling’ pattern of movement in which the eyes appear to roll around as the grating moves from left to right. Finally, in Movie 3 (gratingFaceIllusion_VerticalScanning.mp4), I use still another spatial frequency (this time including thicker black stripes) to induce eye movements that are either perceived as vertical scanning or as blinking.

Fig. 2.  The basic stimulus configuration for the illusory eye-movement effect: A vertical grating (which may vary in spatial frequency properties) is moved across a non-interlaced, forward-looking face. The result is perceived movement of the eyes, which varies with the properties of the grating.

There are other eye movements that can be achieved with this technique by further varying the properties of the grating. In particular, we have observed that some gratings lead to unyoked illusory movements of the eyes in which the grating induces nystagmus-like scanning movements that are out of phase across the two eyes. In extreme cases, this can lead to perceived crossing and uncrossing of the eyes. The illusion is also sensitive to spatial and temporal parameters of the entire display (base image and grating). We have found that as the size of the illusion increases the perception of eye movements is reduced, appearing more like flicker than like movement of the pupil and iris. Moving the grating too quickly or too slowly also reduces the strength of the effect.

The illusion clearly depends on the grating-like structure of the sclera and the darker region composed of the pupil and the iris. While face recognition tasks generally depend on horizontal orientation energy (Goffaux & Greenwood, 2016), vertical orientation energy has been shown to be of critical importance for tasks involving the estimation of gaze and the perception of face properties that depend on eye appearance (Goffaux, 2019), which is consistent with the observed impact of a vertical grating on eye movement perception in this illusion. The high contrast between the sclera and the dark portion of the eye also support simple rules for gaze estimation based on low-level measurements of local contrast, elegantly demonstrated in Sinha’s Bogart Illusion (Sinha, 2000). In this illusion, contrast negation is used to reverse the polarity of the dark and light regions in a face image, which subsequently reverses the direction of perceived gaze. This phenomenon thus results from the natural spatial properties of the eyes as they appear in natural images and an interference effect that is similar to the emergent structure in Moiré patterns.

However, this effect is also clearly related to Anstis’ ‘Footsteps Illusion’ (2001), in which the contrast between the leading edge of a rectangle and a background grating leads to variation in perceived speed as the rectangle moves across the background. The current effect is most similar to Howe et al.’s ‘inverted footsteps illusion’ (2006) in which the rectangles themselves are stationary and the background grating moves behind them, but also shares some aspects of the ‘Kickback Illusion’ (Howe et al., 2006) insofar as the perceived direction of motion alternates as the grating is moved across the face. The Footsteps Illusion and its associated variants encompass a wide variety of movement patterns (see Kitaoka & Anstis, 2021 for a comprehensive recent review), and similar mechanisms likely underlie the diverse perceived eye movements we are able to generate in this illusory effect by varying properties of the overlaid grating. One of the potentially unique features of the current illusion is the observation of swirling and vertical movements of the target shape, which have not been previously reported in the literature describing the Footsteps illusion. However, during the review process, Kitaoka evaluated the novelty of this phenomenon by creating additional demonstrations of the effects of target shape and grating spatial frequency in the Footsteps Illusion. These are depicted in Movie 4 and Movie 5, which demonstrate how changing the target shape can lead to both swirling and vertical movements with an appropriate choice of grating width. Thus, the current illusion clearly depends on similar mechanisms as the Foosteps illusion, with the shape of the dark and light portions of the eye supporting multiple movement patterns as a function of grating width.

Movie 4. Swirling movement is observed in the Footsteps illusion when tapered shapes translate across the grid instead of rectangles. (Movie supplied by Akiyoshi Kitaoka)

Movie 5. Swirling movement is also observed in the Inverted Footsteps illusion, replicating this property of the “Faces behind bars” illusion. (Movie supplied by Akiyoshi Kitaoka)

To further examine how low-level properties of the dark and light portions of the eye affect perceived illusory motion in this illusion, I developed several variants of the illusion by 1) Elongating the dark portion of the eye horizontally and vertically, and 2) Varying the width of the light portion of the eye. These manipulations demonstrate the the spatial frequency imposed by the dark and light portions of the eye, as well as any enclosing dark border surrounding the light portion of the eye, influence the perceived movement of the eyes. Specifically, while varying the elongation of the dark portion of the eye appears to mainly affect the magnitude, but not the direction, of illusory movement, the width of the light portion of the eye can profoundly change the perceived motion. In the Supplemental file containing the images associated with these variations on the original illusion, one can observe how the absence of a bounding contour for the light part of the eye leads to a ‘pulsing’ effect in which the dark shape appears to change shape or appear and disappear as the stripes move across the pattern. With the imposition of a bounding contour for the light part of the eye, the original horizontal scanning pattern is observed so long as the light portion of the eye is not too large in area relative to the dark portion of the eye. Again, these demonstrations point to low-level mechanisms that drive the illusory eye movement effect, closely related to those that support the Footsteps Illusion.

I suggest that there is potentially at least some mid- to high-level contribution to this effect, however, as evidenced by Movie 6 (gratingFaceIllusion_Nystagmus_Negated.mp4) below in which I present a contrast-negated face with a white grating moving across it. Though the low-level contrast relationships supporting the original effect are largely retained in this configuration, the eye movements in this version of the illusion are somewhat weaker (though still apparent). This may indicate that eye- or face-specific mechanisms for perceiving gaze (which are disrupted by contrast negation (Ricciardelli et al., 2009) also play a role in the original illusion in conjunction with low-level mechanisms.

Having begun the presentation of this illusion by referencing Barrier-Grid animation, it is interesting to consider the relationship between these animated images and the Foosteps illusion. First, it is worthwhile to point out that the current effect differs somewhat from how typical Barrier-Grid animation works while sharing some of its properties. In those stimuli, the perceived animation depends on completion: Each periodically-sampled constituent frame is amodally completed behind the occluding grid, leading to the sequential presentation of a series of different completed images. The current illusory effect is arguably not as directly about amodal completion, however, given that the correct completed image does not differ as the grid changes position. Instead, the perception of eye movement depends on interference of the grid itself with the underlying image and the sequential presentation of different interference patterns that are then perceived as a coherent eye movement. As such, the illusion more closely resembles a Moiré animation (Lincoln, 2020; Spillmann, 1993), but implemented using images with different spatial properties. To what extent is this similar to what is happening in the Footsteps illusion? That is, do both of these phenomena depend on similar low-level mechanisms, but expressed in the context of different local patterns? This illusory phenomenon does not allow us to draw a firm conclusion about this possibility, but I think it does reveal interesting and novel results related to motion perception, in particular the interaction of object shape and spatial frequency in the Footsteps illusion and perhaps a broad class of related illusory motion phenomena that depend on the application of moving gratings.

Ultimately, this illusion demonstrates how some complex natural images include specific low-level features that are used fairly directly for higher-level inferences about object appearance. By combining a single static face image with various vertical gratings, we can create a range of plausible and implausible eye movemements based on these inferences.

Acknowledgements

Special thanks to Erin Conwell and Blaise Balas for their observations of early versions of the illusion. This research was supported by NIGMS P30 GM114748, which supports the Center for Visual and Cognitive Neuroscience at North Dakota State University.

References

Anstis, S. M. (2001). Footsteps and inchworms: Illusions show that contrast modulates motion salience. Perception, 30, 785–794. doi: 10.1068/p3211

Goffaux, V. (2019). Fixed or flexible? Orientation preference in identity and gaze processing in humans. PLoS One, 14(1), e0210503. doi: 10.1371/journal.pone.0210503

Goffaux, V., & Greenwood, J. A. (2016). The orientation selectivity of face identification. Scientific Reports, 6, 34204. doi: 10.1038/srep34204

Hopwood, H. V. (1899). Living pictures; Their history, photo-production and practical working: With a digest of British patents and annotated bibliography. London, UK: Forgotten Books (reprint).

Howe, P. D. L., Thompson, P. G., Anstis, S., Sagreiya, H., & Livingstone, M. S. (2006). Explaining the footsteps, belly dancer, Wenceslas and kickback illusions. Journal of Vision, 6, 1396–1405. doi: 10.1167/6.12.5

Kitaoka, A., & Anstis, S. (2021). A review of the footsteps illusion. Journal of Illusion, 2, 5612. doi: 10.47691/joi.v2.5612

Lincoln, J. (2020). Digital scrolling and Moiré pattern animations. The Physics Teacher, 58, 684. doi: 10.1119/10.0002748

Ricciardelli, P., Betta, E., Pruner, S., & Turatto, M. (2009). Is there a direct link between gaze perception and joint attention behaviours? Effects of gaze contrast polarity on oculomotor behaviour. Experimental Brain Research, 194(3), 347–357. doi: 10.1007/s00221-009-1706-8

Seder, R. B. (2007). Gallop! A Scanimation book. New York, NY USA: Workman Publishing Company.

Sinha, P. (2000). Here’s looking at you, kid. Perception, 29, 1005–1008. doi: 10.1068/p2908no

Spillmann, L. (1993). The perception of movement and depth in moiré patterns. Perception, 22(3), 287–308. doi: 10.1068/p220287