ABSTRACT. Sensing stereoscopic depth requires that image points be binocularly matched. Therein lies the correspondence problem: how are true matches distinguished from false ones? Conventional algorithms select true matches on the basis of correlated features and adherence to natural statistics, while rejecting false matches as noise. We propose here an alternative that uses the signals present in false matches to delineate the true solution. When visualized in a Keplerian array, binocular matches are symmetrically reflected about an axis that represents a potential solution. Surface properties such as slant and curvature are encoded the transformation that describes how one-half of the matches reflects across the symmetry axis onto the other.

To implement this strategy, we convolved left and right images with Gabor kernels of various standard deviations (spatial frequencies). We then produced Keplerian arrays by comparing filter responses across left and right spatial-frequency combinations. Responses that are minimally different across the eyes gave rise to regions of high symmetry; response position within the Keplerian array gave the location of a solution in space. Solutions possessing natural surface regularities consistently showed minimal differences for one left : right spatial frequency ratio, which is correlated with the local surface slant. As a result, combining responses within particular ratio families can distinguish true matches from false ones. True matches tend to be elongated and smoothly contoured, with symmetry preserved across all members of a ratio family from low to high spatial-frequency combinations.

This approach is efficient; preprocessing is minimal since no feature extraction is involved. It can be implemented in machine vision to solve the correspondence problem for depth sensing algorithms. It is robust when tested against perfectly camouflaged surfaces in random dot stereograms and consistent with physiological data showing that false match signals are propagated to higher cortical areas along the dorsal pathway.

ABSTRACT. For the visual system, it is one of the fundamental tasks to separate between glossiness (specular highlight) and pigmentation (the surface texture) on object’s surface in the world. We investigated whether the higher-order image statistics has the potential to deal with this separating task. Images of bumpy surfaces were randomly generated from 3D-spherical objects, and created 1,800 images per different four categories (matte, gloss, textured-matte and textured-gloss). All of them were normalized to possess the same mean and variance pixel intensities, and then 744 of PS statistics (Portilla & Simoncelli, 2000) were computed from them. Next, we developed the classifier trained those generated images to statistically categorize the images into four categories by canonical discriminant analysis. We found out that the classifier can obviously classify the generated images, and some specific statistics termed Linear cross position and Energy cross position (Okazawa et al., 2014) were highly used. We also conducted psychophysical experiment that subjects asked to rate perceived glossiness and pigmentation from the generated images. Comparing the classifier’s result and the psychophysical result, it turned out that the former is highly correlated to the latter in short-time presenting. In addition, we tested whether both can equally detect the highlight inconsistency to be based on the separation mechanism between glossiness and pigmentation. The classifier that only trained glossiness and pigment factors could differentiate the highlight-consistent/inconsistent images, and this result corresponded with human perception by psychophysics when the surface texture of the highlight component had high or low local contrasts, however, left the gap at mid-level local contrasts. Thus it was suggested that the classifier is possible to detect the highlight inconsistency using the higher-order image statistics that are not completely same statistics employed by the visual system.

ABSTRACT. Facial expression analysis is an active research field in vision (studying the neural pathways that encode and represent facial expressions in the brain), psychophysics (studying the viewing patterns and the perception of facial expressions by human observers) and computer vision (building and training automatic systems that can analyze facial expressions from images). The common ground between these disciplines are image datasets of people exhibiting various facial expressions, that are either shown to human observers or used to train machine learning algorithms.

Although there are many available facial expression datasets, most of them are limited to 7 prototypical expressions (angry, sad, happy, surprised, disgusted, afraid and neutral) with a single intensity instance per expression. Among them, even fewer include validation studies which provide measures for Arousal, Valence, Intensity or Genuineness.

Based on Russell’s ‘circumplex model of affect’, and a dataset with prototypical expressions, we introduce an augmentation process that employs sophisticated morphing algorithms in order to produce computer generated variations of the original parent expressions. Two types of variations are included; variations across intensity, modelled as transitions between the neutral and a peak-expression face, and variations across peak-expressions. From an initial set of 7 prototypical parent expressions per subject, the final result is an augmented set of 120 natural looking morphed expressions, between 7° to 207° (with a 15°-step) in the Arousal-Valence plane, and 10 intensity increments.

We demonstrate our approach for 3 existing datasets; Radboud, Karolinska, and Warsaw and use the augmented dataset for the training of a computer vision system, exhibiting promising results for the prediction of Arousal Valence and Intensity, in unseen images. Additionally, the dataset is also used in psychophysical eye-tracking studies in order to analyze the differences in observer viewing patterns for high and low intensity expressions.

ABSTRACT. Perceptual experience can be distorted by sensory coding. Some examples - prolonged exposure (adaptation) to a rightward tilted stimulus can make previously vertical looking inputs seem left tilted, adaptation to an elongated shape can make previously circular looking forms seem squished, and adaptation to a male face can make previously androgynous looking faces seem feminine. In all these contexts it has been argued that distorted form perceptions can either be exclusively or partially driven by re-normalization. According to such proposals inputs are encoded in terms of how they differ from a perceptual ‘norm’ that is updated to better reflect the prevailing environment. Exposure to a male face, for example, could shift ones impression of what constitutes an average, androgynous looking, face toward more masculine physical inputs. This would produce a uni-directional aftereffect. Previously androgynous looking faces would look feminine as there would be a greater female-signed difference between these and the updated ‘norm’, and previously masculine looking faces would look more feminine as there would be a lesser male-signed difference between these and the updated norm. Similar explanations have been offered to explain distorted orientation and shape perceptions. Another possibility is that such distortions happen because sensory coding exaggerates differences between adaptors and subsequent inputs, the critical prediction being that such aftereffects should be bi-directional. Adaptation to a somewhat masculine face, for instance, should make less masculine faces seem more feminine and more masculine faces even more masculine. In this talk I will argue there is no compelling evidence for perceptual re-normalization of tilt, shape or face perception, and that bi-directional aftereffects can be demonstrated in each context. Consequently I will argue that adaptation induced distortions of human form perception are generally explicable in terms of neural-population codes that exaggerate differences between successive inputs.

ABSTRACT. A large portion of our perception relies on the discrepancy between the representation of the world we learnt and what we effectively observe. In order to learn what to predict in a given situation, the brain very efficiently extracts the multiple regularities present in the sensory-motor streams. Within the visual modality, those regularities are anchored in space with clustered features that allow us to segment and identify objects. As proprioception provides precise estimation of the visual target in egocentric space, a saccade can be seen as a transformation of the visual information given a motor action. This transformation itself is a form of regularity that can be learnt. We want to study the mechanisms for extracting rules from statistical regularities, and model these mechanisms computationally. Since human adults are already highly familiar with the natural world, we construct a procedurally-generated artificial world based on three types of rules: i) gist (semantically-related elements are spatially grouped into clusters); ii) semantic (the meaning of an element defines the cluster it belongs to); iii) syntactic (the local organization of elements follows a specific grammar). These three rules mimic natural regularities in the real world, while being completely controllable. We then compare how different algorithms (e.g. Deep Belief Network, Self-Organizing Map, Look-up Table) behave when presented with this world and tested on tasks such as pattern completion, visual search and predicting the perceptual consequences of a saccade. Preliminary results on the pattern completion task suggest that the three rules can be learned by auto-associative memory models, therefore providing the necessary capability for the more complex tasks of visual search and estimation of saccade’s effect. Finally, we discuss the feasibility of comparing human and algorithm performance on the same tasks, as a way to benchmark the general rule extraction capability of an artificial system.

ABSTRACT. Accumulating evidence suggests that the medial frontal region is involved in performance monitoring and behavioral adjustment. However, the neural mechanisms underlying these functions remain unclear. To address this issue, I will present one of recent studies performed in our laboratory. The topic covers “surprise neurons” in the supplementary eye field (SEF). The SEF is located in the dorsomedial frontal cortex lateral to the preSMA. The SEF is a higher oculomotor area closely connected to the frontal eye field (FEF) and parietal oculomotor-related areas. To investigate SEF involvement in performance monitoring and adjustment, we introduced a visual search task that required switching between exploration (searching a valid pair) and exploitation (maintaining alternating gaze shift between a correct pair). Our study demonstrated that the SEF was involved in detecting unexpected errors or correct feedback, “surprise signals,” and switching between exploitation and exploration during serial visual search tasks. These studies suggested that medial frontal cortices execute performance monitoring and behavioral adjustment by detecting unexpected situations and then generating a top-down signal to switch behavioral modes.

ABSTRACT. Prevailing views of the visual processing is that visual information is decomposed into many visual attributes, such as shape, color, and motion, in the hierarchical visual processing, and these attributes are unified for various brain computations for perception and action. We may therefore tend to think that each visual attribute analysis is commonly used for different brain functions. In fact, several studies reported that similar forms of visual feature detection were found between oculomotor control and perception or among different motor systems. On the other hand, some dissociations of visual processing have been also reported, which could be occurred in the high-level or distinct levels of visual attribute processing. However, dissociation in visual analysis extracting a particular visual attribute had not yet been examined well. Recently, we have provided quantitative evidence that a single visual attribute coding, first-order visual motion, is differently formed according to particular downstream motor functions, i.e., quick hand and eye controls. As shown in previous studies (Saijo et al 2005; Gomi, 2008; Miles et al., 1986), surrounding visual motion elicits ultra-short latency manual flowing response (MFR) and ocular following response (OFR), which are considered to function in reducing corresponding errors in interaction with environments. Interestingly, while the MFR has gain-tuning specificities similar with those of the OFR for changes in image luminance contrast (Gomi et al., 2006), it was newly found that response specificities of MFR and OFR for the stimulus size and location were dramatically distinct from each other (Gomi et al. 2013). Since the contrasts cannot be explained by any difference in motor dynamics or coordinates, the results indicate multiple visual motion analyses for different motor functions. For a deeper understanding, we would need to reconsider the overly simplified representation of visual attributes in the brain.

ABSTRACT. Following a transition to a new scene, there is a systematic shift of eye movements to the center in the first saccades. Previous accounts of this initial centering response are based on the idea that the scene has changed and the new scene needs to be processed. Yet, scene transitions that introduce changes to the scene also produce a global luminance transient, whose effects have never been examined. In this study, we investigated how changes in the scene and the global transient contribute to the initial centering response. In three experiments, we presented pairs of scene images. Each image in a pair was presented for several seconds before it transitioned to the other. The magnitude of scene change between the two images was manipulated. To test the effect of a global transient, a blank mask was presented very briefly (15ms) between the two images. Our results show that the initial centering response was triggered by the global transient, and was further increased by the magnitude of scene change. Even when the global transient was presented without any changes in the scene, the initial centering response was significant. Our results suggest that the initial centering response comprises of a low-level reflexive component, and is modulated by scene change. 

ABSTRACT. When the right and left eye’s images of a visual object are fused and perceived as a single object, these images are presented on the empirical corresponding points (or neighboring areas) in the retinas. Empirical horopter defines the locus in the visual space in which objects produce images on the corresponding points in both eyes. Because the corresponding points are the reference points for detecting binocular disparity, it is beneficial for the visual system if the binocular images of objects are presented to these points. It is suggested that some unconscious actions such as head movements and hand manipulations are performed to move the retinal images to these points. The shape of the empirical horopter surface in the frontal direction has been well defined based on measurements of the corresponding points，however, for eccentric visual directions, its shape has not yet been accurately identified from direct measurements. We measured the binocular corresponding points using the criterion of minimal apparent interocular motion and estimated the shape of the empirical horopter for horizontal and vertical eccentric gaze directions for several participants. The results showed that the loci of the empirical horopter for horizontal eccentric gaze directions deviated from the locus of the Vieth-Muller circle and those for upper and lower eccentric gaze direction were more inclined (top-away) and less inclined about the horizontal axis, respectively. These results were inconsistent with the prediction made from the measurements of corresponding points for the frontal gaze direction, but could be explained by considering the distortion of eye shape and cyclovergence. From these results, we can specify the shape of the empirical horopter in the wide visual field. We will discuss the relation of the shape of the empirical horopter with the unconscious action we perform and the positions of systems to display visual information.

ABSTRACT. Attention and awareness are two tightly coupled processes that have been the subject of the same enduring debate: Are they allocated in a discrete or in a graded fashion? Using the attentional blink paradigm with a psychophysics approach, we show that awareness arises at central stages of information processing in an all-or-none manner. Manipulating the temporal delay between two targets affected subjects’ likelihood of consciously perceiving the second target, but did not affect relative contrast judgments between the two targets. Conversely, when the same stimuli were used in a spatial attention cueing paradigm (a replication of Carrasco, Ling, & Reid, 2004), cueing did affect relative contrast judgments. The findings distinguish the fundamental contributions of attention and awareness at central stages of visual cognition: Conscious perception emerges in a quantal manner, with attention serving to modulate the probability that representations reach awareness.

ABSTRACT. Background: We perceive “instantaneous” events, even though they actually have some amount of physical time. The psychological instant has previously been indicated to occur below 125 ms of duration. The present study explored neural bases of psychological instant and continuance when matching event-durations to attentional templates of duration, by recording event-related potentials (ERPs). Method: EEGs were recorded from 12 healthy participants. Stimuli were 4 geometric shapes (1 target; 3 non-targets) with durations of 83 ms (short) or 133 ms (long), which were sequentially presented with equi-probability and in random order. Inter-stimulus intervals were 900-1200 ms (7 steps). The participants were instructed to respond to the short or long targets in particular blocks. Regarding task relevancy, ERPs in response to the targets and non-targets were classified into two classes, respectively: those with relevant duration or irrelevant duration. The order of to-be-responded durations was counter-balanced across the participants. Results: For the long non-targets, stimuli with relevant duration elicited more positive ERPs than those with irrelevant duration in 330-430 ms post-stimulus at a right parieto-occipital site (PO4). On the other hand, the short targets elicited more negative ERPs when they had relevant duration in 220-340 ms at a left parieto-occipital site (PO3). Discussion: As our knowledge, these results are the first report of ERP manifestations of matching processes of attentional time templates and stimulus durations. Particularly, the ERP relevance effect for long non-targets may reflect pure duration-matching processes because the procedure minimized effects of motor preparation or decision-making processes. Interestingly, the short-duration stimuli did not cause this effect; instead, the relevancy effect was found only for targets. This indicates that the matching of shapes preceded that of duration. In conclusion, the present study shows that the matching processes in working memory are different between psychological instant and continuance.

ABSTRACT. The physical motion of an object is always affected by the gravity on the earth. Therefore, the perception of motion is considered to be affected by gravity information. It has been shown that the perceived acceleration of an object’s motion in a frontal plane is affected by the direction of the motion in relation to gravity (Kaneko et al., APCV 2010).In this study, we investigated the effects of gravity information from vision and of body orientation on the perception of motion in depth. We measured the magnitude of the acceleration of an approaching object that appeared to move with a constant velocity using the method of constant stimuli. To manipulate gravity information from the vestibular and somatosensory systems, we used three conditions of observer’s body orientation: upright, supine, and prone states. In addition, we manipulated gravity information from vision by presenting the natural 3D pictures behind the approaching object taken toward the three directions: upward, downward, and horizontal. The results showed that observers perceived motion in depth with a constant velocity when the stimulus had physical deceleration for all conditions of body orientation and background. The magnitude of deceleration was greater for the prone and supine states than for the upright state. However, the effect of gravity information from vision on the motion perception was not systematic among conditions and was inconsistent across observers. The effects of visual information seem to depend on the contents of the picture, and the perceptual bias from the vestibular and somatosensory systems seems to be dominant when visual information for gravity is not evident.

ABSTRACT. When observing a moving object in the frame sequential presentation (one of the 3D image presentation methods), the time difference between binocular images is considered to affect the detection of binocular disparity. In this study, we intended to reveal the depth perception of moving objects with various pattern of motion profile in this method, and discuss the mechanism of extracting binocular disparity from the images presented asynchronously. In experiment one, the binocular images of a moving disc that should be seen simultaneously in natural condition were presented alternately as seen in the frame sequential 3D method. As a result, the perceived depth corresponded to the disparity extracted from a real image in one-eye and a spatially averaged image from temporally adjacent two images in another eye. Moreover, two oppositely moving stimuli showed as the same way were perceived in the planes with opposite depth. These facts are consistent with the idea that the disparity is detected on the moving coordinate which based on the interpolated position of the moving stimulus with intermitted presentation. In experiment two, depth perception were measured while observing the moving object having constant acceleration, that having constant acceleration with a fixed deviation for one eye or that having constant acceleration with random deviation in the frame sequential presentation. Results showed that the depth separation was constant as long as the deviation was the same independent of the magnitude of acceleration. From the facts, we presume that human can interpolate the position of moving stimulus with intermitted presentation at least under the uniform acceleration condition for detecting binocular disparity.

ABSTRACT. When we compare the subjective duration of a moving object with that of a static object, the former is perceived as being longer than the latter. Previous research revealed that the presentation duration of a running man represented by a realistic stationary figure is also perceived as being longer than that of a standing man. In this study, we investigated whether the perceived duration is also affected by pictograms made of simple components and lengthened by an increase of the implied speed in the pictograms. We created four types of pictograms representing a standing, a walking, a running and a rapidly running man. The four pictograms were used as stimuli. Participants fixated on a cross presented on a gray background and pressed a key. After a random duration of blank, one of the four stimuli was presented. The presentation duration of the stimulus was 150, 300, 450, 600, 750 or 900 ms. Ten participants were asked to evaluate the motion speed implied in the stimulus by a numerical number. Another fifteen participants were asked to reproduce the duration by pressing a key to make it subjectively equal to the duration of the presented stimulus. It was confirmed that the evaluated value of the speed was 0 when the standing man was presented and increased in the following order: the walking, the running and the rapidly running man. The results of the reproduction show that the presented duration of the rapidly running man was perceived to be longer than that of the standing and the walking man. These results indicate that the motion implied by simple figures can also affect the perceived duration and suggest the possibility that the cognitive speed modify the perceived duration.

ABSTRACT. Optokinetic nystagmus (OKN) is elicited by motion in a large part of the visual field, and attention modulates the frequency of OKN. For example, OKN is facilitated by paying attention to the motion of the large peripheral visual field in the case of no fixation point. However, the detailed properties of attentional modulation to OKN and of the fixation point’s effect are not completely understood. OKN might be facilitated by attention to a small area or to an object with motion because the function of OKN is to stabilize a moving object on the retina. This study investigated the influence of spatial attention to motion in peripheral visual field on OKN. OKN was measured when observers attended to a motion stimulus presented to a specific area in the peripheral visual field with the fixation point at the center of the display. The stimulus consisted of random dots or periodic stripes and during the spatial and temporal frequency of the stimulus was manipulated. The direction of motion in the attended area and that in the unattended area were always opposite each other. Observers responded the orientation of a target superimposed on the motion stimulus to maintain attention on the specified area with motion. OKN was also measured during conditions of direct viewing and without attention. The results indicated that OKN occurred when attending to a motion stimulus although the frequency was less than that in the direct viewing condition. OKN being consistent with the motion in the unattended area did not occur . In addition, analysis of small eye movements during the experiment revealed that OKN was not elicited by directly viewing the target. These results suggest that attention enhances the occurrence of OKN to stabilize the retinal image and acquire information relevant to the observer.

ABSTRACT. When observers view a large visual stimulus that moves uniformly, they often perceive illusory self-motion in the direction opposite to the stimulus motion. This phenomenon is called vection. A vection-inducing stimulus is also known to induce postural responses in the same direction as the inducing stimulus and, therefore, many studies have used visually evoked postural responses (VEPRs) as objective measures of vection. In the present study, to further investigate the relationship between vection and VEPRs, we measured vection and center of foot pressure (COP) to a vertically or horizontally moving random-dot pattern. In an experiment, ten standing participants viewed the inducing stimulus for 30 s, while holding a mouse in the right hand. The participants were asked to report vection by pressing the mouse button whenever they felt vection during stimulus presentation. After each trial, they were also asked to rate vection magnitude (from 0 to 100). COP was measured in the lateral and anterior-posterior directions during a 40-s interval from 5 s before the inducing stimulus presentation to 5 s after it. Vection magnitude was stronger to vertically moving stimuli than to horizontally moving stimuli. There was stronger vection for upward motion than for downward motion. COP started to move in the inducing stimulus direction immediately after the onset of the inducing stimulus, and its magnitude (difference from the baseline, i.e., COP data before the stimulus presentation) gradually became larger during the stimulus presentation. The mean COP during 1-s intervals before and after the onset of vection showed larger COP after the vection onset than before it. Taken together, the present results suggest that, at least to some degree, mechanisms underlying vection and VEPRs are related.

ABSTRACT. Previous study revealed that perceived duration is distorted by several factors. Filled duration illusion is known as showing that the occurrence of additional sensory information (so called ‘divider’) during interval of signals increases perceived duration of this interval. It was confirmed that this effect is occurred when target and divider were visual interval with auditory dividers (Mitsudo et al., 2012). However, the effect of divider’s modality is largely unclear. In this study, we investigated the effect of divider’s modalities, on auditory and visual, on the filled duration illusion. An interval of 700ms was fixed as a standard interval which presented to be A-A (two auditory signals) or V-V (two visual signals), and it was compared its perceived duration with that of another target interval which duration was varied by participant’s answer. Interval of targets in a trial was consisted of either audio or visual, and the modality of target signals was always consistent with standard interval. Except control target conditions (A-A and V-V), there were four divided conditions (AaA, AvA, VaV, and VvV) where target signals were interrupted by two brief auditory or visual dividers. Standard and target intervals were randomly assigned to the 1st or the 2nd temporal order. The PSE was measured with staircase procedure at 50% correct level. As a result, in divided condition, mean PSE increased on both visual and auditory targets compared to control condition, showing that filled duration illusion had occurred in both cases. Filled duration illusion was much enhanced when dividers’ modality was identical with target modality. Moreover, visual targets were more influenced by dividers than auditory target signals. This result suggests that the visual processing of the time is not only less accurate than auditory processing but also strongly influenced by temporal interruptions.

ABSTRACT. Purpose: We investigated the effects of dominant eye and contextual background on predominance during binocular rivalry. Methods: 10 subjects with normal or corrected-to-normal vision were recruited for the present study. Dominant eye was determined using the hole-in-the-card test. In experiment 1, subjects viewed the stimuli through anaglyph filters and reported the color. The subject’s responses were compared with the color on the dominant eye. To rule out color dominance, we conducted the experiment with switched-anaglyph filters. We designed experiment 1-2 to investigate the influence of contextual color on binocular rivalry. In experiment 2, the subject viewed the stimuli through the polarized filters and reported the orientation. The subject's responses were compared with the grating on dominant eye. To rule out the effect of stimulus size, we conducted the experiment with a smaller target. We designed the additional experiment to investigate the influence of contextual grating information on binocular rivalry. Results: In experiment 1, 8 of 10 subjects reported that eye preference was highly correlated with dominant eye. This finding is significant without reference to color. In experiment 2, 7 of 10 subjects reported that eye preference was highly correlated with dominant eye. This finding is significant without reference to size. In experiment 1-2 and 2-2, all subjects reported that predominance of context-contradictory target. Conclusion: We found the relationship between dominant eye and eye preference. We suggested that intrinsic factors, such as dominant eye affect stimulus strength. Experiment 1-2 and 2-2 showed that contradictory context increases target predominance during binocular rivalry. Overall, our results indicate that the contextual background reduce the stimulus strength of the context-consistent target; it would correspond to an increase in the dominance duration of the context-contradictory target.

ABSTRACT. The Cafe Wall illusion is an illusion that the orientation of the gray line (mortar line) between displaced rows of alternating black and white blocks is perceived to be tilted. We conducted two experiments to investigate the effect of binocular disparity information on the Cafe Wall illusion. In Experiment 1, the Cafe Wall stimuli were vertically placed in three different conditions: the condition in which there was no disparity, the condition in which mortar lines were in different depth planes, and the condition in which mortar lines and middle column were in different depth planes. In Experiment 2, the effect of binocular disparity was tested in the Cafe Wall evoked by horizontally-oriented stimuli. The results showed that the degree of perceived tilt (magnitude of illusion) and confidence of illusionary perception was generally similar. The magnitude of illusion was the largest in the no-disparity condition. In contrast, when different depth information was added, by which the mortar lines and middle blocks seemed to appear in front, the illusion decreased. These results that binocular disparity affected the illusion evoked by physically identical stimuli indicate that Cafe Wall illusion might occur primarily in a stage before depth processing.

ABSTRACT. In this study, we describe a new speed optical illusion that makes fall of snow appear faster than its actual speed, when viewed through a window blind with horizontal slats. We simulated this natural phenomenon using a computer. The simulated stimulus was presented in a dark room as dots that represented snow and an occluder that represented a window blind with horizontal slats. We studied the basic features of this speed optical illusion with two experiments. First, we focused on the slit width. Variations in slit width change the perception time when viewing the dots through the slits. Several previous studies have reported the speed perception for a given perception time. We compared the illusion magnitudes for vision fixation with a standard speed (2.0 deg/s) using various slit widths and an adjustment method. The illusion magnitude was calculated as the standard speed minus a point of subjective equality. The illusion magnitudes were approximately 8 deg/s, 11 deg/s, and 12 deg/s with 0.4 deg, 0.2 deg, and 0.1 deg of slit width, respectively. Thus, the illusion magnitude increased with decreasing slit width. Second, we focused on the direction of motion of the dots. Previous studies have reported the features of vertical or horizontal speed perception. We compared the illusion magnitudes for vision fixation using a standard speed (2.0 deg/s), 0.2 deg of slit width, and a constant method with upward or downward motion at the first onset. The illusion magnitudes were about 0.7 deg/s and 0.8 deg/s with upward and downward motion, respectively. Thus, the illusion magnitudes were not affected by the vertical direction of motion. In the future, we plan to study the illusion magnitudes for horizontal directions and elucidate the mechanism underlying this new speed optical illusion.

ABSTRACT. Both shading and cast shadows are used as depth cues in our visual system. Several studies have shown that we use the light-from-above (or light-from-left in a few studies) assumption in interpreting the shape-from-shading displays. However, for cast shadows, few studies have shown such a perceptual bias. Moreover, perceptual interactions between shading and cast shadows have not been demonstrated. We investigated whether the light-from-above (or light-from-left) assumption played a role in interpreting cast shadows and whether shading information had an influence on that. There were two stimulus orientation (vertical and horizontal) conditions. In vertical conditions, a stimulus image included disks and dark blurred “shadows” that were arranged in a vertical line. There were three disk surface (two gradation and one plain grey) conditions. In gradation conditions, the disk had a vertical luminance gradient (white-on-top or black-on-top). In horizontal conditions, the stimuli were rotated by 90 deg. Sixteen subjects participated in the experiment. They observed the stimuli with one eye and chose one of the cast shadows presented on both sides of a disk, e.g. above and below, that perceptually corresponded to the disk. Varying the location of the cast shadows, we measured strength in perceptual correspondence between the disk and the cast shadows. A method of constant stimuli was used. The results of the plain grey condition showed a perceptual preference toward the shadow below the disk while no bias was found in a horizontal dimension. Thus, the light-from-above assumption is used in interpreting cast shadows while the light-from-left assumption is not. The effect of luminance gradient on the directional bias of preferred cast shadows was found in the white-on-top, white-on-left, and white-on-right conditions. In each condition, the darker side of the luminance gradient corresponded to the preferred cast-shadow location. Therefore, shading information could be used in processing cast shadows.

ABSTRACT. Continuous flash suppression (CFS) is a potent form of binocular rivalry where a rapidly changing pattern viewed in one eye suppresses the target in the other eye for several seconds. Although the effectiveness of CFS has been linked to reduced retinotopic neural adaptation from the fast transients in the masker, recent studies have shown that the amount of suppression depends on the similarity in spatial properties between the masker and target. The current study examines the role of temporal components in CFS suppression by varying the temporal frequencies of spatially matched targets and maskers. Our preliminary results show that tuning curves for temporal frequencies under interocular suppression are similar to binocular temporal tuning curves reported by Cass and Alais (2006), providing further support for interactions between different temporal channels and underscoring the role of early visual areas in CFS.

ABSTRACT. Inhibition of return (IOR) is an orienting bias against recently attended locations. Early work by Posner and Cohen (1984) and Maylor and Hockey (1985) showed that IOR is coded in spatiotopic or environment-centered coordinates. Recent studies, however, have reported IOR effects in both spatiotopic and retinotopic coordinates. One potential methodological confound of all previous studies is that the spatial gradient of IOR was not considered when selecting the baseline for calculating spatiotopic and retinotopic IOR effects. This issue was addressed with the classic Maylor and Hockey (1985) task, with the addition of trials without onset peripheral cues. In this task, the participants shifted their gaze in response to a displacement of the fixation stimulus. On 50% of the trials, an uninformative peripheral cue was presented before the gaze shift. After the gaze shift a target could appear at the cued spatial location, a location corresponded to the cued retinal locus, or one of two control locations. Immediately following the gaze shift, the IOR effects at the cued spatial and retinal locations were 55 ms and 43 ms, respectively (Exp. 1). The 43-ms IOR effect was much stronger than that of a control location equally distant to the cued spatial location, suggests that this effect was not part of the spatiotopic IOR gradient centered at the cued spatial location. About 1400 ms following the gaze shift (Exp. 2), IOR effects at the cued spatial and retinal locations reduced to 10 ms and 15 ms and those at the control locations were no longer distinguishable from zero. Together, these results provide unambiguous evidence that IOR is coded in both spatiotopic and retiontopic coordinates.

ABSTRACT. We compared the strength of self-motion perception (vection) with and without a decoy who reported very strong or very weak vection. The decoys were two 24-year old male graduate students. They were not known by the experimental participants, and they did not know the true purpose of the experiment. One decoy was used in the strong vection condition (strong decoy), and the other was used in the weak vection condition (weak decoy). In the vection task, participants verbally reported the subjective vection strength. We then determined whether a conformity effect could be obtained when participants made their verbal reports in the presence of a decoy. The experiment had three conditions with two types of decoys: strong decoy, weak decoy, and no decoy (control). Strong decoys always reported very strong vection and weak decoys always reported very weak vection. When the decoy reported strong vection, participants reported stronger vection that they did when the decoy reported weak vection. Vection latency was shortest in the strong vection condition and the longest in the weak vection condition, while vection duration was longest in the strong condition and the shortest in the weak condition. Subjective vection magnitude was lowest in the weak vection decoy condition and highest in the strong decoy condition. In the control, condition, these measurements each fell somewhere in between the two decoy conditions. These results strongly indicate that participants’ perception conformed to the strength of vection reported by the decoys. When the decoy reported strong vection, significantly stronger vection was reported by the participants, and when the decoy reported weak vection, significantly weaker vection was reported by the participants. This is the first report demonstrating that vection perception can be altered by conformity to decoys. We should examine other social effects on vection in future.

ABSTRACT. It has been observed that a visual stimulus rotating globally along an observer’s line of sight can induce the illusory perception of self-rotation in the roll axis toward the opposite direction (roll vection). Psychophysical experiments were conducted to examine the effects of local rotations of visual elements of the stimulus that were manipulated independently of the global rotation. The results indicated that the addition of local rotations inconsistent with the global rotation, which can be assumed as the primary inducer of roll vection, generally decreased the strength of perceived self-rotation. The uniformity of orientation of the elements composing the global visual pattern and the visual polarities assigned to each visual element, i.e., intrinsic directionality concerning up and down, were observed to function as modulators of the effects of the local rotation. These results suggested that local motion signals arising from independent rotations assigned to each element of a visual object, which can be assumed to be irrelevant with integration of the global rotation, cannot be ignored in the perceptual mechanism underlying roll vection.

ABSTRACT. Attentional modulation on visual processing is not limited to the focus of attention, but found within a range in the visual field as the spotlight metaphor suggests. We developed a method to estimate spatial tuning of visual attention with steady state visual evoked potential (SSVEP) and compared to the estimate with event related potential (ERP). SSVEPs are oscillatory the electroencephalogram (EEG) potentials that occur synchronously to flickering visual stimuli and amplitude of each temporal frequency reflects attentional modulation at the stimulus flickering at the temporal frequency. Monitoring SSVEP for disks flickering at different temporal frequencies provide attentional modulation around the focus of attention. The amplitude is expected to be high at the temporal frequency, corresponding to the flicker of the disk where attention is focused on. The amplitude would reduce as the attention focus moves away from the disk. Using circularly arranged disks around the point of fixation, we succeeded to estimate spatial tuning of visual attention around the focus of visual attention. The tuning is broad and attentional modulation was found even 90 degrees or more in rotation angle (several degree of visual angle) apart from the attention focus. Another measure, ERP is the EEG potential synchronized to the target presentation. Since target was presented at one of the disks, spatial tuning of attention can be estimated from ERP amplitude at different locations. P3 component of ERP showed large amplitude only at the focus of attention, showing a narrow spatial tuning. We suggest that the different spatial effects of attention reflect attention modulation at different stages of visual processing.

ABSTRACT. Temporal regularities in the environment are thought to guide the allocation of attention in time. Recent studies suggest that rhythmic entrainment of ongoing brain oscillations may support this phenomemon. In this talk, I will present a recent study in which we investigated whether rhythmic entrainment can optimize neural processes across modalities. Participants viewed a series of images in silence, or in-synchrony or out-of-synchrony with a slow auditory rhythm. We examined scalp-recorded evoked oscillations in occipital regions before image presentation, and event-related potentials (ERP) after image presentation.The phase alignment of beta oscillations differed between in-synchrony and out-of-syncrhony presentation. ERPs revealed a enhanced anticipatory potential and enhanced N1 in the in-synchrony as compared to the out-of synchrony condition. In addition, beta phase differences between conditions predicted amplitude enhancement in the ERP responses. These results indicate that rhythmic entrainment extends across modality and is supported by the reorganization of neural oscillations. This reorganization reflects global, instead of local, rhythmic cues, and can affect frequencies higher than external rhythms. These findings support the idea that neuronal oscillations constitute a general mechanism through which the brain uses predictive elements in the environment to optimize processing resources and perception across modalities.

ABSTRACT. Steady state visual evoked potentials (SSVEPs), which are elicited by flickering stimuli with tagged frequency, have been widely noticed as an attention status extraction method. This paper describes our attempts to explore how and to what extent the SSVEPs can be used for extracting the attentional status in two cases: tracking multiple moving objects and estimating the useful field of view (UFOV). First, we try to identify the multiple attended objects from SSVEPs. In this experiment, three objects, which move clockwise, were displayed with different flickering tagged frequencies (12, 13.33, and 15Hz). Participants were instructed to track two moving targets among three (i.e. three attentional states). Peak amplitude of each single-trial SSVEPs was analyzed by canonical correlation analysis (CCA) to identify the attended multiple objects. The results showed that (1) SSVEP amplitudes differed depending on attentional states; (2) identification accuracy was 54% in average of three states (50, 54, 60% for each state) and was above chance, meaning that it is possible to extract the attentional states to even for the multiple moving objects using SSVEPs. In the second experiment, we investigated the relation between SSVEPs and UFOV, which is defined as the visual area from which one can extract visual information at glance without eye or head movement. For each participant, standard UFOV performance test was performed to score the performance to divide attention between central and peripheral stimuli. Then, we recorded the SSVEPs using similar display as the UFOV test with flickering squares at center and peripheral. It turned out that the total SSVEP power (center + peripheral) was negatively correlated with the UFOV performance, suggesting that the participants with better UFOV performance require less attentional load due to their efficient attentional allocation skill.

ABSTRACT. Spatial attention typically causes amplitude modulations of sequential event-related potentials (ERPs), such as P1, N1, and P2, over occipito-temporal scalp sites within about 300 ms after onsets of visual stimuli, which may reflect gating mechanisms in the flow of visual processing. However, it remains unclear what types of visual processing are manifested by each ERP modulation. Computational views suggest that our visual system is specialized to extract objects from fragmented retinal images, which requires long-range spatial interactions in the visual cortical hierarchy. Previous research has shown that N1 enhancement for unilateral probes at attended versus unattended locations is decreased when the probes are embedded in pre-viewed large objects. This suggests that neural activations by spatial attention spread over object representations in the lateral occipital cortex. In our laboratory, we have applied another classical paradigm of spatial attention with bilateral stimulus arrays and found the following. 1) Attention-spreading for object onsets was also associated with N1. 2) The N1 attention-spreading effect occurred irrespective of types of visual grouping, such as connectedness, achromatic-color similarity, shape similarity, and amodal completion. 3) A P2 attention-spreading effect, rather than the N1 effect, was observed for unfilled line objects. These results suggest that the N1 is engaged in enhanced neural representations of figural regions against the background, while P2 is engaged in the individuation of objects with more abstract representations. Furthermore, in recent experiments with the unilateral-probe paradigm, we found a P1 attention-spreading effect for objects in front of other objects, but not for objects in the back of the surroundings. In conclusion, depth cues in three-dimensional world may enable rapid selection or formation of perceptual objects, while two-dimensional forms may require later stages of processing to resolve figure/ground ambiguities in early visual cortices.

ABSTRACT. When a new object appears unexpectedly in the display, an orienting response (OR) is elicited.  We interpret this as attentional capture by the novel stimulus.  But, when the same stimulus is presented again, the OR habituates, and the stimulus, no longer novel, fails to capture attention.  The surprise paradigm (Horstmann, 2002) is perhaps the best example of attentional capture disappearing after a single presentation of a novel, salient stimulus.  However, in contingent capture – where the putative capture stimulus shares the target's diagnostic feature, but in all other respects (e.g., its form) is clearly a non-target, attention capture persists.  Here, we ask whether attention recapture occurs in the contingent capture scenario.  In these experiments, the target is a letter with a specific hue (e.g., red).  The paradigm allowed attention to be captured by initially a color singleton (e.g., four blue spots).  This was followed 150 ms later by a second set of color spots.  The question is whether this second irrelevant stimulus was capable of recapturing attention.  The results show that if this second stimulus succeeds in recapturing attention only if it possesses the target's diagnostic feature.  The suggestion here is that the OR that does not habituate is probably a voluntary OR (e.g., Maltzman, 1979) distinct from the involuntary OR to a surprise stimulus.

ABSTRACT. Spatial frequency plays an important role in face perception (Wu et al., 2009). Low spatial frequency was found to indicate holistic face processing (Goffaux & Rossion, 2006). However, its effect on face adaptation and awareness of emotion is largely unknown. In particular, does spatial frequency affect facial expression adaptation and awareness of emotion? We designed two experiments, the first was to test the adaptation effect by manipulating the spatial frequency of the adapting faces, and the second was to test the disciminability of these faces.In the first experiment, we found that adapting to normal- or low-spatial frequency faces generated singificant facial expression aftereffects. However, adaptint to high-spatial frequency face did not generate significant facial expression aftereffect. In the second experiment, we examined the awareness of emotions of these adapting faces. We found that the participants could always accurately differentiate facial emotions for both normal- and low-spatial frequency faces, but the accuracy of high-spatial frequenty faces was at chance level (50%). These findings indicate the holistic nature of facial expression adaptation, and this effect of spatial frequency is likely to be influenced by the awareness of the adapting faces’ emotion. Therefore, our study sheds light on the holistic nature of face processing and awareness of emotion during adaptation.

ABSTRACT. Detection and recognition of multi-modal emotion is important for social development and attention allocation in infants. Studies have shown infants are capable to match visual-audio emotion in faces and video-taped whole body movements. However, it is unknown if this ability also applies to biological motion, which requires higher level holistic processing to detect the configuration. Although ERP responses in 8-month-old infants differed when fearful and happy point-walkers were presented, no study was available to confirm infants’ behavioural discriminability in biological walkers. Current study fills in this research gap by assessing 8- to 10-month infants’ capacity to pair consistent audio emotions to biological point-walkers.

Audio stimuli included female-recorded (1) laughter and (2) cry, (3) happy and (4) sad speeches cut from an audiobook. Laughter and cry lasted for 4s, and speeches were of 7s each. In each trial, one type of audio stimulus was paired with one happy point-light walker displayed side-by-side with another sad point-light walker (adopted from the Biomotion Lab). Each infant watched each condition twice (total 8 trials) while Tobii eye tracker recorded their gazes. Any looking preference toward congruent pairs was an indicator of bimodal emotion recognition.

Interestingly, infants looked significantly longer toward happy biological point-walkers regardless of the audio emotion, demonstrating their capacity to discriminate emotions contained in biological motion. It also implies that the audio-visual emotion carried in our stimuli may be too complicated for the developing brains, or that our chosen audio stimuli did not match with the action of walking. This is the first behavioural evidence that infants can detect emotion from biological motion, and future studies are called to identify the mechanisms to account for infants’ preference toward happy walkers.

ABSTRACT. Working memory and attention have been described as two distinct cognitive constructs. Recent research, however, has proposed a unified framework of working memory and attention, by stating that working memory as internal attention, shares the same resource with the external attention operating over scene. However, few empirical studies, have directly examined this unified framework. We examine two fundamental questions in the current study: (1) Do working memory and external attention share one resource? (2) If they share a single resource, is it domain-general or domain-specific? We explored these questions by adding an attention-demanding secondary task during the maintenance phase of a working memory task, where participants were required to remember three colors, shapes, or color-shape bindings. Sharing a domain-specific resource predicted a novel resource-release: In a dual-task setting, the resource used in the secondary task should be released after its completion, and received by working memory. This would lead to a performance-recovery when sufficient time was given. We tested this prediction in three experiments. Experiments 1 and 2 added a transparent-motion task and an object-based feature-report task, respectively, to consume the object-based attention. Both experiments revealed a larger impairment for binding than for the constituent features with a short recovery-time after completing the secondary task. Critically, by giving 900 ms of additional time, a significant binding performance-recovery was demonstrated. However, such selective binding impairment and recovery vanished when a space-based visual-search task was added (Experiment 3). Together, these results suggest that working memory and external attention share an object-based resource when processing feature bindings, providing critical evidence to the “working memory as internal attention” view.

ABSTRACT. The episodic buffer is a limited storage system that initially set to actively bind features across perceptual dimensions and between working and long-term memory, using the domain-general central executive resource. However, studies in the last decade implied that the episodic buffer was just a passive storage buffer receiving bindings constructed elsewhere. The current study hypothesized that the episodic buffer is an active buffer, yet it is fueled by object-based attention. We tested this hypothesis by testing the role of object-based attention in three types of bindings which theoretically should be constructed within the episodic buffer: The two constituent features of binding were spatially separated (cross-space binding; Experiment 1), were from different modalities (cross-modality binding; Experiment 2), and were from different working memory modules (cross-module binding; Experiment 3). Moreover, we added an object-based feature report task during the maintenance phase of a working memory task, to compete the object-based attention with the to-be-memorized stimuli. In line with the prediction of object-based attention hypothesis, Experiments 1–3 consistently revealed a significant impairment for binding than for constituent single features, suggesting that object-based attention plays a core role in retaining binding representations in the episodic buffer. Theoretical implications to the episodic buffer are discussed.

ABSTRACT. Although the mechanisms of visual working memory (VWM) have been studied extensively in recent years, the active property of VWM has received less attention. In the current study, we examined how VWM integrates sequentially presented stimuli by focusing on the role of Gestalt principles, which are important organizing principles in perceptual integration. We manipulated the level of Gestalt cues among three or four sequentially presented objects that were memorized. The Gestalt principle could not emerge unless all the objects appeared together. We distinguished two hypotheses: a perception-alike hypothesis and an encoding-specificity hypothesis. The former predicts that the Gestalt cue will play a role in information integration within VWM; the latter predicts that the Gestalt cue will not operate within VWM. In three experiments, we demonstrated that collinearity (Experiment 1), closure (Experiment 2), and color similarity cues significantly improved VWM performance. These findings together suggest that VWM realizes and uses potential Gestalt principles within the stored representations, supporting a perception-alike hypothesis.

ABSTRACT. Background: The frontal eye fields (FEF) and the dorsolateral prefrontal cortex (dl-PFC) are two prefrontal regions that share direct reciprocal anatomical connections, and are thought to be involved in saccade generation and the deployment of voluntary visual attention. Both regions contain neurons that respond either to visual stimuli, around the time of eye movements, and/or to spatial locations stored in working memory. While the single-cell behavior is well understood, less is known about the functional connectivity between these two regions. Methods: We recorded populations of neurons in the behaving monkey (Macaca fusicularis) using chronically implanted electrodes while the animal was engaged in a delayed memory saccade task with a distractor. We binned the neural responses aligned to target onset using 20 ms bins. We then computed the transfer entropy as the difference in entropy between one cell's responses conditioned on its own history, and the entropy of the same cell's responses conditioned on both its own history and the history of a second cell.

Results: We found that cells in the FEF tended to initiate the transfer of information to other cells, both in the FEF and the dl-PFC early in the trial compared to the information transfer from cells in the dl-PFC. We also found that connections originating from the FEF tended to be stronger than those from the dl-PFC. When comparing correct to aborted trials, we found that connections were significantly delayed in aborted trials. This was true for connections originating from both FEF and dl-PFC. Connections during aborted trials also tended to be weaker compared to correct trials.

Conclusion: We found evidence of canonical circuitry between the FEF and the dl-PFC, in which FEF encodes the target, then passes the information about the target on to the dl-PFC, which then maintains the memory throughout the delay period.

ABSTRACT. What are the sensory drivers of audio-visual sound symbolism? Morton (1994) has suggested that in animals, lower fundamental frequencies are linked to larger shapes, as lower pitch naturally signals larger body plans. Ohala (1994) has suggested that the resonant frequencies of speech also code information about size, due to relative shortening/lengthening of the vocal tract when the lips are retracted (e.g., 'ee') or lengthened (e.g., 'oo'), as reflected in the frequency of the 3rd harmonic (3rd formant). However, few experimental studies have attempted to test the combination of these factors empirically. We asked participants to select a shape from a grid of 11 different shapes, at 11 different sizes, while listening to a range of sounds. The shapes ranged from a highly convoluted shape (high edge complexity), to a simple circle (low edge complexity). Sounds were a high and low note from the natural range of 6 instruments from 3 different families: strings (violin, cello), double-reed (oboe, bassoon) and brass (trumpet, tuba), along with artificial since waves, and human voices. For the instruments, lower notes were matched to rounder shapes than higher notes (F(1,27)=20.53, p<.001), but the precise shape selection differed between instrument families (F(1, 27)=17.69, p<.001), between instruments within a family (F(1, 27)=63.93, p<.001), and all three factors interacted (F(1, 81)=15.36, p<.001), producing unique patterns for individual sounds. A similar pattern was observed for male (low) and female (high) voices articulating the /i/ sound in 'feet', the /u/ sound in 'shoes', the /a/ sound in 'umbrella', and /y/, “ü”). Interestingly, shape (edge complexity) was modulated more than size, undermining the suggestion that audio-visual matching is primarily driven by the mapping of pitch to size. Instead, these findings suggest a mapping of spatial-to-temporal frequency, with higher pitches and higher harmonics corresponding to higher spatial frequencies.