ABSTRACT. What do safe driving, healthy eating, and understanding someone else’s perspective all have in common? They are all underscored by attentional control - the need for attentional control in our everyday lives is ubiquitous, and individual differences in it have implications for our health, safety, and relationships. Developing behavioural measures that can reliably capture the full richness and diversity of attentional control in our lives has proved challenging. In this talk, I will discuss the insights offered from an alternative approach: subjective measures of attentional control. I will focus on the Cognitive Failures Questionnaire (CFQ), which quantifies individual differences in the extent to which people experience failures of attentional control in everyday life, such as forgetting appointments and daydreaming when supposed to be listening. I will present research from my lab about how CFQ scores relate to objective performance on important tasks that require attentional control such as low prevalence visual search, and how they relate to different aspects of empathy and negative affect. Finally, I will discuss our development and validation of the new “Cognitive Failures Questionnaire 2.0” and show how it explains greater variance in objective attentional control performance than the original CFQ. This highlights the insights offered by subjective measures of attentional control and the role attentional control plays across cognitive, social, and emotional domains.

ABSTRACT. Rapidly orienting towards salient objects in a crowded environment has played a crucial role in human evolution as it permits us to quickly identify potential prey, mates or predators. This inherent human trait also makes us vulnerable as our attention can be captured by salient objects around us. For instance, we are often distracted by our phone's buzzer when focusing on a task at hand, giving rise to the phenomenon of attentional capture. Despite this, it remains unclear when and how salient distractors interact with our goals and our knowledge on the neural mechanisms responsible for attentional capture is limited to a few brain regions recorded from non-human primates. To explore this, we examined intracranial electroencephalography recordings of 18 epilepsy patients performing an additional singleton task. In this task, participants searched for a unique shape (target) among search elements while keeping fixation at the center and responded to the orientation of the line segment inside the target. In the distractor present condition, a uniquely colored distractor was randomly present. We conducted a multivariate analysis on human intracranial signals and successfully dissociated distractor-specific representations from target-arousal signals in the high-frequency (60-100 Hz) activity. We found that salient distractors were processed rapidly around 220 ms, while target-tuning attention was attenuated simultaneously, supporting initial capture by distractors. Notably, neuronal activity specific to the distractor representation was strongest in superior and middle temporal gyrus, amygdala, and anterior cingulate cortex, while there were smaller contributions from parietal and frontal cortices. These results provide neural evidence for attentional capture by salient distractors engaging a much larger network than previously appreciated. Theories on the network that contributes to salient distractor processing need to be revised to embrace the role of the ACC communicating with temporal lobe structures.

ABSTRACT. Stimuli that engage attention often enter awareness. Attentional engagement also has a dark side: Unattended stimuli are frequently missed. Non-spatial contingent capture (Folk et al., 2007) and surprise-induced blindness (Asplund et al., 2010) illustrate detection failures following attentional capture. In contingent capture, a distractor with a target-defining feature disrupts detection of a target that follows several hundred milliseconds later. In surprise-induced blindness, a relatively unexpected, task-irrelevant distractor disrupts target detection for a similar duration. Both effects habituate across trials, though the latter’s habituation is faster and more complete (Liaw et al., 2020). Given the similarities in their evoking conditions (abrupt distractor onsets) and effects, we tested whether contingent and surprise capture may jointly contribute to observed deficits in contingent capture paradigms; we also sought to better dissociate their effects. In four online experiments, participants searched for a color target in a rapid serial visual presentation of letters surrounded by a white box. On a minority of trials (15%), the box transiently became a critical distractor whose color was the same, similar, or opposite the target color. When critical distractors and targets were separated by two intervening stimuli (Lag 3; pilot, n=108, and Expt. 1, N=121), the critical distractors induced a deficit proportional to their target similarity, demonstrating contingent capture. The deficits habituated across trials, suggesting a surprise capture contribution as well. With no intervening stimuli (Lag 1; Expt. 2, N=78), however, the deficits were larger and showed no sign of habituation. Similar effects were observed even when the critical distractors appeared during every trial (Expt. 3, N=151). Taken together, the results suggest that abrupt onsets capture attention regardless of their task relevance, but only some distractors cause prolonged attentional engagement and subsequent awareness deficits (Theeuwes, 2021). Such engagement can be due to target relevance (contingent) or contextual novelty (surprise).

ABSTRACT. Introduction: Our prior event-related potential (ERP) studies (Qu, Hillyard & Ding, 2017; Hu, Ding & Qu, 2019) showed that, after extensive training over days as a searching target, a non-salient shape (which could be considered as a conjunction within shape dimension) could capture attention (indexed by N2pc) when serving as a task-irrelevant distractor. However, it remains unclear whether such learning-induced capture mechanism also applies to cross-dimension conjunctions(Ding,Li & Qu, 2023). The main purpose of the present study is to investigate whether a color-orientation conjunction could obtain an ability to capture attention after perceptual learning. Methods: Two types of color-orientation conjunctions were adopted: one does not contain spatial configuration of features (Exp. 1), and the other does (Exp. 2). Like our previous studies, participants were first trained with a hard conjunction search task for several days until their performance were relatively stable. Then, in the post-training ERP sessions, the previously-trained conjunction was designed to be a distractor in a conjunction search task or in a central rapid serial visual presentation task. We are interested in whether the trained color-orientation conjunction could induce an N2pc when serving as a task-irrelevant distractor. Results: In both experiments, searching performance increased dramatically through perceptual learning. However, distinct patterns of ERP results were obtained. In Exp. 1, the trained color-orientation conjunction (with no spatial configuration) could not induce distractor-N2pc in either tasks. By contrast, in Exp.2, distractor-N2pc was induced by the trained conjunction (with spatial configuration), indicating that it captured attention successfully. Conclusion: Different from simple color-orientation conjunction, a well-practiced configural conjunction could capture attention involuntarily. Our findings support that, spatial configuration plays an essential role in the process of converting different features into a new functional unit.

ABSTRACT. Explicit goals and implicit learning bias attentional control. These influences form a control state that drives saliency computations on spatial and feature gain control (Luck et al., 2021). Although studies of value-driven (or value-modulated) attentional capture have characterized how reward learning influences spatial gain control, less is known about non-spatial feature gain control. In this talk, I will present five systematic experiments on temporal attention using the RSVP paradigm, pairing value feedback with specific target or distractor colour features. The findings taken together show that learned value signals can influence attention through non-spatial feature gain control, but likely only when rewarded features are directly target relevant. Distraction from learned value signals may happen under more specific contexts in the absence of strong goal-directed influences, and likely to be partially distinct from spatial gain control at least.

ABSTRACT. Introduction: It is well known that the ability of distractor suppression has a close relation to visual working memory (VWM). Recent event-related potential (ERP) studies showed that individuals with high working memory capacity could call on an early suppression (indexed by Pd component) to salient-but-irrelevant distractors whereas those low-capacity ones could not. However, it remains unclear whether such an early suppression mechanism applies to nonsalient distractors that possess the target-defining feature. The main purpose of the present study is to investigate the relation between the VWM capacity and the attentional process of feature-matched distractors. Methods: Fifty-one healthy young adults participated in this study. Like previous studies, individual VWM capacity was measured by the K-score of change detection task. In the ERP experiment, we adopted a central rapid serial visual presentation (RSVP) task in which distractors with or without the target-defining feature were presented peripherally. Participants were informed to identify a digit of a specified color in the central RSVP stream. We focus on two distractor-elicited ERP components (i.e., N2pc and Pd) that would reflect two distractor-related attentional processes (i.e., attentional capture and attentional suppression), respectively. Results: Surprisingly, we revealed that high-capacity individuals would be captured more attention (reflected by a larger distractor-N2pc emerging in less than 200 ms) compared to low-capacity ones, meaning that they are less able to ignore such distractors in early visual processing. Although feature-matched distractors captured more attention for high-capacity individuals, they received stronger suppression soon afterwards, indexed by a larger distractor-Pd. Conclusion: Compared to low-capacity individuals, high-capacity ones would first be captured more attention by feature-matched distractors, then enact more inhibition to those distractors. Our findings support that, high-capacity individuals could exhibit more flexible ways of attentional processing when dealing with different kinds of distractors.

ABSTRACT. Synthetic datasets are crucial for enhancing the capabilities of multi-modal models and embodied agents in visually grounded referencing tasks. We present two studies highlighting the pivotal role of synthetic data in improving object referencing, particularly focusing on natural language concepts such as determiners and prepositions. In the first study, we introduce DetermiNet, a diagnostic dataset designed to capture the nuances of referencing using 25 determiners. Determiners, like "my," "either," and "those," are vital in natural language, specifying the reference to or the quantity of a noun. However, existing datasets often neglect determiners, hindering the precision of vision-language models in referencing objects. DetermiNet, comprising 250,000 images and captions, challenges models to predict bounding boxes based on determiner semantics, underlining the need for datasets covering the full linguistic complexity. In the second study, we investigate the utility of synthetic datasets on enhancing compositional understanding in embodied agents. Reinforcement learning agents are trained to navigate to specific object groups based on natural language instructions, which specifies the noun and attributes with eight determiners or prepositions. Determiners, crucial for object referencing, and prepositions, essential for conveying spatial relationships between objects, present unique challenges. While agents excel in grounding determiner concepts, they encounter difficulties with preposition concepts like "behind" and "beside." Through curriculum learning strategies, agents become proficient in grounding these concepts to decompose held-out test instructions and solve for unseen visual object combinations. While synthetic datasets offer significant advancements, their limitations, particularly in replicating real-world complexity, persist. Nevertheless, they represent a crucial step towards grounding vision models on understudied natural language concepts.

ABSTRACT. Our previous study (Li & Nagai, Research Square, 2023) suggested that the color category boundaries are involved in the sensitivity to suprathreshold color differences, and their contribution (the category effect) increases with the magnitude of target color differences. According to the categorical facilitation hypothesis (Witzel, 2019), observers are expected to spontaneously pay attention to linguistic categories of stimulus colors when judging their color differences, potentially affecting their judgments. In this study, we aimed to discover how attention to linguistic information affects the judgment of large color differences and how it influences color category effects. We psychophysically investigated whether the Stroop task conducted for colored strings at the center of the display, which is expected to use attentional resources for color terms, interferes with a color difference judgment task at peripheral stimuli. Observers performed the color difference judgment tasks based on maximum likelihood difference scaling (MLDS) for three square stimuli with smaller or larger color differences under conditions with or without the Stroop task. Observers’ sensitivities to color differences were analyzed separately for smaller and larger color differences and then compared between the with- and without-Stroop task conditions. The results showed that judgments of color differences became more difficult (i.e., sensitivities to color differences decreased) under the Stroop task condition. Furthermore, this interference effect was particularly pronounced for larger color differences but not for smaller color differences. Additionally, the Stroop task increased reaction times only for larger color differences. These results suggest that attention plays an important role in judgments of large color differences, which is not contradictory to the categorical facilitation hypothesis. However, even with the Stroop task, color category effects on color difference perception persisted. That means we do not have evidence that attention to linguistic categories is essential for color difference judgments.

ABSTRACT. When observing a visual stimulus presented on a large visual field and moves in a uniform direction, observer occasionally experiences illusory self-motion perception in the direction opposite to the visual inducer. This perceptual phenomenon is called visually induced self-motion perception, or vection. Previous vection studies have indicated that visual stimuli with an additional acceleration component, such as jitter or oscillation, can induce a stronger self-motion perception than visual stimuli moving at a constant speed. However, the underlying mechanism concerning the facilitative effects has not been clarified yet. In this study, the effects of the additional acceleration on vection were investigated employing visual stimulus which was composed of a random-dot pattern moving horizontally at a constant speed with/without vertical oscillation. The vertical visual oscillation was accomplished by two ways; the first one was a real motion of the dots, and the second was an apparent motion illusion. In the apparent motion condition, four stroke apparent motion (4SAM) was employed to render the vertical oscillation. In the 4SAM, the visual pattern contains a luminance defined-motion energy and observer perceives a unidirectional visual motion but without a global visual displacement (each visual element flickered on the spot). The results of the psychophysical experiments indicated that visual oscillation rendered by the 4SAM was still effective to facilitate vection strength, although its potential was significantly weaker than the oscillation rendered by the real motion. These attempts would be worthy in order to understand the perceptual mechanism underlying the facilitative effects of the additional accelerating motion on vection.

ABSTRACT. Attention plays a significant role in visual perception and is an indicator of high-level visual processing. The visual saltation illusion (VSI)—where the second flash is mislocalized to occur at a point between its true location and the location of a third flash—only arises in peripheral vision. In this experiment, prediction was used as a factor to observe the impact of attention on the VSI, hypothesizing that suggestion may induce a stronger illusion. Each trial consisted of three flashes; the second flash was presented in the same position as the first or third flash, or at the midpoint between the two. Stimuli were white circles perceived as flashes, measuring 4.1 deg in diameter, and presented¬ on a gray background. Subjects reported where the second flash was located relative to the first and third flashes. Subjects fixated on a cross on a screen aligned with their vision while the stimuli occurred 28.8 degrees below. The first flash was vertically aligned with the fixation point, while the second and third flashes followed in the left or right direction. Prompts randomly appeared above the area where the flashes would occur. Subjects underwent an illusion block with a short stimulus duration (50 ms) and a control block with longer stimulus duration (950 ms), both with the same interstimulus interval (ISI) of 50 ms. In control conditions, with and without prompts, the illusion did not occur. However, under illusion conditions the second flash was perceived to occur at the midpoint with and without prompts; the illusion was slightly stronger with a prompt. When attention is induced, the strength of the VSI can increase, although the effect is not large. This may suggest that the interpretation of the VSI may take place at least partly in a higher level of visual processing.

ABSTRACT. Color assimilation is a perceptual phenomenon that the color appearance of center region is induced similar color of surrounding inducer. This phenomenon is driven by both optical and neural factors. The optical factor is the smoothing effect on the retinal image due to chromatic aberration and light scattering, while neural factors is the spatial summation of visual responses after the photoreceptor. These factors are mixed, however the specific contributions of each factor to color assimilation remain unclear. To elucidate this, we conducted psychophysical experiments and optical simulations of the eye. In the psychophysics, we measured the effect of inducer hue and background luminance on a color assimilation, imitating a watercolor effect. Observers viewed horizontally aligned illusion images with varying background luminance and reported whether a color assimilation was perceived. Color assimilation typically occurred when the luminance contrast between the inducer and background was low, although hue dependency was observed. Specifically, a relatively high luminance backgrounds were preferable for red inducers, while low luminance backgrounds were preferable for blue. Moreover, red inducers often elicited null color assimilation responses. The optical simulations, based on Marimont’s model, were conducted on the same images used in the psychophysical experiments. Obtained retinal images were then analyzed for luminance contrast and color contrast between inducer and center region. We found that the luminance contrast was minimized with relatively high background luminance for red inducer, and low luminance for blue. Color differences were small for blue or yellow inducers, but pronounced for red. These hue dependencies were highly correlated with the psychophysical experiments. These results suggest that optical factor predominantly influence hue and luminance dependencies in the occurrence of color assimilation, at least within the stimulus composition of this study.

ABSTRACT. When the horizontal line is removed from the vertical arrangement of the inward and outward figures of the Müller-Lyer illusion, the position of the apexes of the center angles are perceived to be shifted in the opposite direction from the direction expected from the illusion caused by the length perception of the line segment in the Müller-Lyer illusion. This is one of the paradoxes in displacement (Morinaga, 1941; Morinaga & Ikeda, 1965) reported by Morinaga, and it is also called Morinaga illusion (Day et al. 1983). In this study, we examined whether the same paradox in displacement occur in illusions other than Müller-Lyer illusion. When we checked whether Morinaga's paradox occur in illusions of the same type as Müller-Lyer illusion, we found that there were illusions that occur paradox and illusions that do not. The relationship between the Giovanelli illusion (Giovanelli, 1966) and the gravity lens illusion (Naito, 1994) could be understood in the framework of Morinaga's paradox in the same way as the above two illusions. It was suggested that Morinaga's paradox occurred when the polarity of the inducer was opposite, and Morinaga's paradox did not occur when the polarity was the same.

ABSTRACT. Humans possess a remarkable capacity for abstraction, simplifying complex learning tasks by utilizing composite feature cues. This feature-based cognitive structure facilitates learning and generalization across arbitrary associations. For instance, evaluating candidates' competence pairwise is resource-intensive, whereas leveraging features such as education and work experience allows for more efficient inference. However, our research uncovers a surprising tendency for humans to over-rely on feature-based inference, showing persistent leverage of features even when they fail to predict outcomes. In our experiments, participants took on the role of recruiters tasked with evaluating the competencies of alien candidates, each distinguished by unique shapes and colors. Using the classical transitive inference task, participants were exposed to local comparisons (e.g., A > B; B > C) and were required to infer the hidden ordinal structure of the aliens' competence (e.g., A > B > C). Two sets of alien candidates were presented: a "predictive" set, where features (i.e., color and shape) were correlated with rank, and a "non-predictive" set, where features were entirely irrelevant, making feature-based inference erroneous. Surprisingly, we found that participants consistently used features to estimate rank order, even though they received ground-truth reward feedback about the alien pairs during the learning phase. This tendency persisted even under stricter conditions designed to discourage feature-based inference, where participants were required to achieve 80% accuracy during learning. Computational models were developed to recover the behavioral data, again revealing a weighted combination of individual-level and feature-level inference, with the feature inference weight inversely correlated with participants’ performance. Our findings offer intriguing empirical insights into the persistence of superstitious beliefs in everyday life, even in the absence of conditioned reinforcement.

ABSTRACT. Alphonse Mucha (1860-1939) is a leading Art Nouveau artist whose graphic artworks are well known worldwide. In particular, his commercial posters, which are his masterpieces, are painted in such a way that they convey a message by inducing the viewer's gaze. It has also been stated by the artist himself that the external ocular muscles that move the eyeballs contract and relax harmoniously and rhythmically, creating a sense of comfort and, ultimately, a sense of beauty (Mucha, 1975). In this study, we attempted to verify the gaze induction effect incorporated in Mucha's paintings. As stimuli for the experiment, we used six images: four pictures from his commercial graphic artworks depicting a woman and her gaze, and two pictures of similar scenes by other artists of the same period. Fourteen undergraduate students (8 male and 6 female) participated in the experiment as observers. They wore a eyeglass-type eye tracker (NAC EMR-9) and, with their face fixed on a chin rest, freely observed the stimulus images presented on an 85-inch 4K monitor at a distance of 145 cm. The vertical size of the images was set to the vertical size of the monitor, that corresponds to 39 degrees in visual angle. The gaze induction assumed when looking at the pictures was generally as follows for each stimulus: woman's face - the point of her gaze - string of poster title - curved lines of plants or design figures appearing in the poster - string of final message. Here, we compared the similarity between the assumed induction curves and the actual eye tracker measurement results to confirm the high similarity in Mucha's paintings. And also, frequency analysis of eye movement trajectories was used to analyze components that might produce a sense of "comfort."

ABSTRACT. Executive control is a crucial component of cognition, allowing individuals to modify their ongoing thoughts to achieve behavioral goals. Executive control has been studied with 2 visual paradigms: the n-back and visual attentional blink (VAB). A working memory control limitation is a hypothesized cause for visual awareness failures in both tasks. Here we used Connectome-based Predictive Models (CPMs), which associate individual differences in task performance with fMRI-based functional connectivity (FC) patterns, to better understand the processes involved in the VAB and n-back tasks. For our comparisons, we used Wu et al. (2020)’s vabCPM (n=73), which could successfully account for VAB performance from FC data (rs=0.31, p=.011). We also constructed a CPM from Chen, Tam, et al. (2022)’s analysis of the Adolescent Brain Cognitive Development Study’s 2back task (n=2252). This 2backCPM was also successful (rs=0.48, p<.001). To test for prediction generalizability, we first applied the vabCPM to the 2-back FC and behavioral data, which yielded a small but significant positive correlation (rs=0.05, p=.014). Applying the 2backCPM to the VAB data produced a non-significant correlation (rs=0.13, p=.29), perhaps due to insufficient power. To account for cross-task generalization, edges across each predictive network were compared. The most positively predictive 10% of all edges overlapped significantly above chance across each task’s CPM (17.7%), as did the most negatively predictive 10% of all edges (12.4%). Surprisingly, the edges that predicted both better VAB and 2-back performance were inter-network connections between the default and either somatomotor or salience attention regions. Conversely, worse performance was predicted by greater FC within dorsal attention, and between dorsal attention and either somatomotor or salience attention regions. Contrary to what we hypothesized, successful performance in each paradigm was due to broad coordination or induced brain states involving the default mode network, not due to executive control network function per se.

ABSTRACT. Working memory (WM), the capacity to hold and manipulate information over short periods, is fundamental to various cognitive functions. The neural population activities in the prefrontal cortices, notably the dorsolateral prefrontal cortex (dlPFC) and the frontal eye field (FEF), are believed to play a critical role in WM. In the real world, WM often needs to sustain encoded targets amidst external distractions while also allowing updates with new, relevant information, but the mechanisms that reconcile the stability with adaptability of WM remain elusive. To address the questions, two monkeys were trained on a 2-items delayed-saccade task. Two spatially distinct stimuli were sequentially presented with intervening delays in each trial, with the first stimulus always served as a target, while the second be either an irrelevant distractor or a new target. Monkeys were tasked with responding only to the latest target location at the end of trial. Results revealed stable codes of target locations information in persistent firings of dlPFC and FEF cell populations during the delays. In the retarget condition, the code of target information was replaced faster and more radical in FEF than in dlPFC, indicating potential functional dissociation between the two regions. Further analysis found a choice subspace in both regions where the coding and replacement target information took place. Finally, computational models were built and replicated the hallmark behavioral and neuronal findings observed in monkeys.

ABSTRACT. Boundary extension is a phenomenon where one recalls a scene with a broader area than originally seen. Recent studies suggest that not only boundary extension but also boundary contraction are common phenomena and that scene properties influence these boundary transformations. The cognitive mechanisms underlying how scene properties influence boundary transformation remain unclear. Previous studies have suggested that object-oriented images that typically contain an object in the middle of the scene induce boundary extension. In contrast, scene-oriented images that usually have objects distributed throughout the scene induce boundary contraction. We speculated that object- or scene-oriented images induce boundary transformation by guiding visual attention to the center or periphery of the scene. To test this, in the current study, we manipulated the deployment of visual attention by presenting salient cues at the center or edges of the scenes when displaying object-oriented and scene-oriented images. The results replicated the effects of scene properties on boundary transformation. Furthermore, when salient cues were presented at the center of the scenes boundary extension was more pronounced while cues presented at the edges led to more boundary contraction. These findings indicate that visual attention influences boundary transformation and suggest that the process by which scene properties induce boundary transformation can be partially explained by the mechanisms of visual attention.

ABSTRACT. The dorsal premotor cortex (PMd) is known to play a role in the preparation of saccadic eye movements. However, it is unclear how the population activity of this region unfolds if the timing of a saccade is uncertain. Do oculomotor preparation signals continue to ramp until the onset of an external movement cue, or do they anticipate the earliest possible movement timing? To investigate this, we recorded PMd neurons of non-human primate (NHP) subjects, who performed a delayed saccade working memory task with variable go-cue timings. We found that while the neuron population activity peaks upon saccade movement, there was also anticipatory ramping activity around the time of the earliest possible go-cue, regardless of the onset of the actual go-cue. Similar anticipatory activity was also observed in error trials where subjects make impulsive saccades before the go-cue onset. PMd oculomotor preparation signals therefore not only encode timing of saccadic movements, but also timing information related to anticipation.

ABSTRACT. In daily life, the process of shifting visual attention is often self-initiated without direct influence of external information. This is different from experiments to investigate the effect of attention in laboratory. To investigate the neural mechanisms underlying self-initiated (or willed) attention, we developed a new type of visual search experiment. There were four discs on a display with a RSVP of letter stimuli. The task was to identify the discs with target presentations in the RSVP sequence. Participants searched targets at each disc to judge whether targets were included in the sequence at each disc. Participants shifted gaze to another disc when they found a target at the disc, which was currently being watched, or when they decided to go to another disc, judging the disc without a target or giving up waiting. We also had free-looking and restricted visual search conditions. In the free looking condition, there was no task and participants were instructed to view the discs freely, In the restricted visual search condition, instruction was to shift gaze after finding a target at a disc to the diagonal position of the current disc. We have four types of gaze shifts: free looking, self-initiated shift, spatial-only shift, and instructed shift. We simultaneously recorded EEG and gaze data while participants overtly shifted attention. We analyzed EEG data locking to the saccade onset for shifting gaze and compared theta band in this presentation among different types of gaze shifts. The analysis revealed that differences in frontal theta (4-8Hz) power are statistically significant between for the period between -0.4s and 0s before saccade onset. Brain activity shows that there are differences between self-initiated attention shifts and instructed attention shifts at several hundred milliseconds before the shift of attention. This suggests distinct neural processes underlying different attentional strategies.

ABSTRACT. Vision tends to dominant over audition in adults, as seen in the Colavita visual dominance effect, where a tone is sometimes undetected when paired with a flash, even though the tone is well detected when presented alone. We investigated whether a recent perceptual event modulates visual dominance, and whether this serial dependence occurs at the perceptual or response level. In a simple discrimination task (Experiment 1), participants were required to press a key for flashes, another key for tones, and both keys for combined flash-tone detections (i.e., a bimodal target). The visual dominance, defined as the higher percentage of visual-only errors than auditory-only errors for bimodal targets, was compared according to the target modality in the previous trial. A visual dominance effect was more pronounced when the previous trial consisted of an auditory target (8.05 %) compared to a visual or bimodal target (3.21% and 4.03 %). In Experiment 2, in each trial, a visual, auditory, or bimodal cue that is uninformative to the target modality and needless to respond was presented 1.05 seconds preceding the target. The visual dominance effect was more pronounced when the cue was auditory (5.95 %) compared to when it was visual or bimodal (3.33% and 3.89 %). In both experiments, the more pronounced visual dominance effect was attributable to increased visual-only errors. Hence, people’s intersensory dominance is modulated by recent experience: an auditory event subsequently induced a stronger visual dominance, while a visual event did not biased intersensory dominance, compared to a bimodal event. Such repulsive and asymmetrical serial dependence effect occurs at the perceptual level rather than response level. Our results resonate with a general attentional bias toward vision proposed by Posner et al.(1976), and attention appears to acutely rebound to vision subsequent to being drawn towards another sensory modality, such as audition.

ABSTRACT. Many previous studies on cross-modal interference have reported visual dominance, in which accuracy in visual processing tends to be preferentially maintained when attention is directed simultaneously to visual and auditory stimuli. On the other hand, it has been observed that reaction times to stimuli show an auditory dominance in target detection and discrimination tasks that require speeded responses (Robinson et al., 2016 & 2018). Specifically, they reported that reaction times to visual stimuli were slower in bimodal-presentation than in unimodal-presentation, but reaction times to auditory stimuli depended on task conditions. In the present study, we examined whether auditory dominance in the target detection and discrimination tasks is replicated using common visual/auditory stimuli (5 shapes and tones) and presentation time (200 ms). The results showed an interaction between modality (visual/auditory) and presentation (unimodal/bimodal) in the target detection task. The auditory response time did not change between the unimodal and the bimodal conditions, but the visual response time was slower in the bimodal condition compared to the unimodal condition. On the other hand, there was no interaction between modality and presentation in the discrimination task, and reaction times were slower in the bimodal condition for both vision and audition. It is suggested that the auditory dominance in reaction time is less robust in the discrimination task than in the detection task.

ABSTRACT. We investigated how head orientation affects perceived eye gaze direction in children aged 4-to-6, 7-to-9, and 10-to-16 years. Participants performed a left vs. right gaze discrimination task, with shifts in points of subjectively direct gaze between the head orientations calculated for each age group. In Experiment 1, we used Wollaston face images, where identical eyes were placed in different facial orientation contexts. We observed a greater shift in perceived gaze direction towards head orientation in 4-to-6-year-olds compared to 10-to-16-year-olds (p < .01). Notably, 10-to-16-year-olds did not exhibit a significant shift in perceived gaze direction, unlike the two younger age groups (ps < .01). The results demonstrate the perception of the Wollaston illusion (Wollaston, 1824) in the two younger age groups but not in 10-to-16-year-olds, suggesting a greater attractive influence of head orientation in the younger children. In Experiment 2, we used Normal face images, in which head rotation is accompanied by a shift in the relative position of the iris/pupil within the eyes, typically leading to a repulsive shift in perceived gaze direction. We found a significant shift in the perceived gaze direction opposite to head orientation in all age groups (ps < .01). However, this repulsive shift was significantly smaller in the 4-to-6-year-olds, indicating moderation by a greater attractive influence of head orientation in the younger children as in Experiment 1. Reaction time analysis of both experiments provided further support for this. Additionally, in 7-to-9-year-olds (Experiment 3), we found that the attractive influence of head orientation increased with smaller image sizes (p < .01), indicating flexible weightings of head and eye information depending on available information. In summary, our findings demonstrate age-related changes in the influence of head orientation on the perceived gaze direction, with some flexibility in the degree of influence within the age.

ABSTRACT. Since J. J. Gibson’s treatise on the Ground Theory of Space Perception (1950), much empirical evidence reveals the human visual system uses a representation of the ground surface as a reference frame for space perception in the intermediate distance range (~2-25 m). Here, we present a framework of the perceptual processes for representing the ground surface as a spatial reference frame. (1) When no external ground surface information is visible, such as in the dark, the visual system uses an internal knowledge of the ground surface to locate targets. We refer to this default ground representation as the intrinsic bias. (2) The elliptical-like shape of the intrinsic bias is modulated by individual differences in eye height. (3) When visible ground surface information is available, the visual system integrates the intrinsic bias with depth information on the ground to form the ground surface representation whose shape tends to be more veridical. (4) The accuracy of the ground surface representation is governed by a sequential surface integration process (SSIP) that integrates the texture gradient information on the ground from near the observer’s feet forward to the farther ground extent. (5) The attention mechanism plays an important role in selecting the information for surface integration. (6) The ground surface is coded with an allocentric coordinate system. In preparation for a translational movement, the intrinsic bias is anchored at a ground location and remains at the same location during motion through a path-integration process. The anchored intrinsic bias integrates with depth cues to form the allocentric ground surface representation. This allocentric ground surface reference frame helps render our experience of environmental stability while navigating. Overall, our theoretical framework underscores the fundamental role of the intrinsic bias for representing the ground surface both in the static and dynamic environments.

ABSTRACT. The kinetic Orbison illusion is an illusion of the motion trajectory of a dot moving on a square path on a background of concentric circles. When the dot is tracked by eyes, the sides of the square trajectory are perceived to bend toward the center of the circles. Here, we present the illusion with various kinds of backgrounds. Ten observers participated in the experiment. When dashed circles, radial lines, and random texture are used as backgrounds and rotated in the same direction as the dot’s movement, the bending of the sides of the square trajectory was strongly perceived. The observers drew the perceived trajectory shape with a stylus pen. The illusory curvature of the trajectory shape reached 22.2 - 30.0% toward the center of the square. However, when the backgrounds rotated in a direction that was opposite to the dot’s movement, the illusion diminished. We conclude that when the illusion occurred, the dot motion on a square path was decomposed in two components: a circular motion component and a radial motion component. The former was common with the background rotation and the latter was relative to the background’s motion. The radial motion component causes a perceptual bias in motion direction of the dot toward an inward or outward radial direction around the corners of the square trajectory, and perceptually sharpens the angles. Due to the perceptual sharpening of the square trajectory corners, each side between the corners may appear to bend. This may cause or enhance the illusion of motion trajectory of the dot in the kinetic Orbison illusion and its variants.

ABSTRACT. We have been studying eye-catching three-dimensional signboards that use the reverse perspective illusion. The reverse perspective illusion is also used in Hughes' paintings (Wade & Hughes, 1999), and Takeuchi et al. (Takeuchi, 2018) have proposed a method to observe it using an HMD. In this study, we present a method to verify the strength of the illusion using a VR simulator during the production stage in order to reduce the effort of modifying the signs after they are produced. The three-dimensional letters of the lowercase alphabet and hiragana signboards were composed of two planes as the visible surface (letter surface, side of the three-dimensional letter) when they were installed by rotating them 45 degrees around the y-axis. Then, based on the characteristics of the figures, they were classified into five types, and a VR simulator was developed to measure the strength of these illusions. Subjects wore HMDs and measured the distance of the illusion produced by gradually backing away from the simulated signboard and the position where the illusion disappears when approaching. The experimental results showed that the critical distance in the approach and recession conditions tended to be farther in the recession condition. This is the same trend as that observed by Papathomas (2002), who measured the critical distance for the reverse perspective illusion in real space, and therefore, it can be said that VR is as effective as verifying the strength of the illusion in reality. Although there was no significant difference in the critical distance for most of the characters, the hiragana characters "Ku" and "Tsu" showed a significant trend (p<0.1) compared to the other characters, suggesting that the design needs to be improved. In the future, we would like to study the effects of the surrounding environment on three-dimensional signboards.

ABSTRACT. The brain employs intrinsic dynamics to predict observed sensory inputs, especially under perceptual ambiguity. Despite rigorous investigations, the underlying mechanisms of multistable perception remain unclear. The bistable apparent motion in the Ternus display serves as an example of this, where subjective perception spontaneously alternates between element motion (EM) and group motion (GM) percepts. In the visual domain, it has been suggested that the pre-stimulus phase of low-frequency brain oscillations has been related to trial-by-trial fluctuations in threshold-level perception. It implies that good phases are more efficient, leading to an optimal perceptual outcome, while bad phases are less efficient, leading to a suboptimal perceptual outcome. Based on this hypothesis, we expected to observe significant differences in the pre-stimulus low-frequency phase between the EM and GM percepts. Using intracranial electroencephalography recordings in 4 patients with epilepsy during the Ternus paradigm, we showed significant pre-stimulus low-frequency phase differences between EM and GM percepts, particularly in the early visual areas. Intriguingly, the probability of GM percepts decreased to a minimum at the opposite phase angle, confirming that the low-frequency phase preceding stimulus presentation influences subsequent perceptual outcomes.

ABSTRACT. The typical Poggendorff illusion consists of a two-dimensional figure with two parallel lines and diagonal lines on each side of the parallel lines. There is a hypothesis that the Poggendorff illusion arises from the ambiguous depth relationship between the parallel and diagonal lines in three-dimensional space. This study investigates how the depth relationship between a diagonal line and a three-dimensional cuboid affects the Poggendorff illusion. The changes in the illusion were examined when the diagonal line intersected with the edge of the front face, the center of the side face, and the edge of the back face. The width of the cuboid was categorized into three levels: narrow, medium, and wide. When the diagonal line intersected with the back edge of the narrow cuboid, the side of the medium cuboid, and the front edge of the wide cuboid, the two-dimensional gap between the diagonal lines remained the same, but they were perceived as farther apart in three-dimensional space the farther back they were located. This setup allowed for a comparison the effects of two-dimensional and three-dimensional spacing. The experimental results showed that the illusion was weakest when the diagonal line intersected with the front edge of the cuboid and strongest when it intersected with the back edge. The illusion increased as the diagonal line intersected closer to the back of the cuboid. This tendency persisted even when the two-dimensional gap between the diagonal lines was controlled. The results indicate that the Poggendorff illusion becomes stronger as the perceived distance between the diagonal lines increases in three-dimensional space. Therefore, the Poggendorff illusion is related to depth processing, suggesting that higher-level visual information processing influences its occurrence.

ABSTRACT. It is known that there exists a blind area in human vision where optic information cannot be received due to the absence of photoreceptors. Although we do not notice the existence of the blind spot in our daily life, we may find that during monocular observation we cannot recognize some visual images due to this blind spot. Previous studies have shown that the blind spot could be perceptually filled in by its surrounding visual information. On the other hand, it has been reported that even when a visual image on the retina can be received by photoreceptors, perceptual fading would sometimes occur due to filling-in (e.g., Troxler fading, contrast decrement fading, and transient-induced fading). The present study then aims to investigate how visual information located in the blind spot area is filled in by surrounding visual information when perceptual fading simultaneously occurs around the blind spot. While blocking one eye of a subject, a green disk that covered the subject’s blind spot (colored red) was presented on a gray background. The blind spot area was perceived as green through the filling-in of surrounding information. Then, when the surrounding green area perceptually faded, we confirmed that the entire visual stimuli was seen as gray, including the blind spot area. Our results showed that the filling-in process of the blind spot could be induced by its surrounding information, even when the surrounding itself was further filled-in by the background. The underlying mechanism may be worth discussing.

ABSTRACT. Memorability refers to an intrinsic property of images that determines how likely they will be remembered or forgotten. Recent research has shown that memorability is highly consistent across individuals and cultural groups. Despite this high consistency, previous studies found that participants' subjective evaluations of stimulus memorability do not match their memory performance. This discrepancy may be due to subjective evaluations being biased towards consciously accessible properties that do not directly relate to memorability. Our study used an indirect behavioral measure to predict image memorability to avoid this bias. Participants were asked to press a keyboard key when they had finished encoding an image, then moved on to the following image. We expected that encoding response time (RT) could reflect the encoding efficiency of images and thus predict their memorability. Consistent with our prediction, results indicated that memorable images were encoded faster than forgettable ones. Additionally, we reanalyzed data from a previous study that used similar experimental paradigms to replicate and generalize the current finding. The results showed that the efficiency of the encoding process, measured by spontaneous encoding RT, reliably predicts memorability. This finding was consistent across different stimulus categories, as well as among participants from various cultural backgrounds, providing converging evidence for our findings. We propose that encoding RT can serve as an effective indirect behavioral measure for predicting memorability.

ABSTRACT. The roles of the dorsolateral prefrontal cortex (DLPFC) and the frontal eye fields (FEF) in working memory and saccadic movements respectively have been well established. The present research aims to study the interplay between the two regions in the context of a delayed oculomotor task. DLPFC and FEF activity was recorded in a non-human primate model during the performance of the task and statistical analysis, including principal component analysis and linear decoders, was performed. Correlated activity between the DLPFC and FEF regions together with the decoding of spatial information revealed the transfer of spatial working memory information encoded in the DLPFC to the FEF during the delay epoch for subsequent saccade preparation.

ABSTRACT. This study investigated the effect of top-down interventions on a bottom-up attentional phenomenon: emotion-induced blindness (EIB). A previous study found that the distraction from an emotional image can be reduced by providing a warning about the distractor category. We tested the benefits of image-specific or categorical warnings. Participants performed an RSVP task with erotic or neutral distractor images, with or without warnings. Three pre-registered experiments tested different warnings: a preview of the upcoming distractor image (Experiment 1, N = 24), a different image from the same distractor category (Experiment 2, N = 24), or a text warning identifying the distractor category (Experiment 3, N = 48). We also included erotic images of males and females and an equal number of male and female participants to test for the potential interaction between image gender and participant gender on the distraction effect. In all experiments, warnings decreased the distraction caused by erotic images. Previewing the distractor image provided much more benefit (68% reduction) than presenting a same-category image (36.2% reduction) or text warning (31.6% reduction). We suggest that the preview images either primed the processing of distractor images or inhibited processing of distractors by providing a negative template. The benefits from other warnings demonstrate that knowing the distractor category is sufficient for some reduction in the EIB effect. The pooled gender analysis from three experiments found no interaction between image gender and participant gender on the distraction effect. Our results show that the EIB effect can be reduced by both image-specific and categorical foreknowledge. Our results also show that the EIB effect does not depend on image gender and participant gender.

ABSTRACT. Schizophrenia is a complex neuropsychiatric disorder characterized by positive symptoms such as hallucinations and delusions, as well as negative symptoms including cognitive impairments. In this study, we aimed to enhance the visual working memory (VWM) performance of schizophrenia patients by applying theta (6 Hz) transcranial alternating current stimulation (tACS)—a technique previously shown to improve VWM in low-performing healthy individuals—over the right frontoparietal pathway. Additionally, we modified the VWM change detection task by introducing arrow cues at different stages to reduce memory load during early encoding, late encoding, or retrieval stages. In low-performing individuals, we observed significant effects of memory cues and tACS (without interaction), whereas no effect was observed in high-performing individuals. Specifically, low-performers benefited from both early and late maintenance cues, but not from retrieval cues, suggesting that memory consolidation issues were the primary source of their memory difficulties. Low-performers also benefited from frontoparietal theta tACS, which nonselectively boosted VWM performance across all memory stages. Together, we conclude that poor VWM in low-performing schizophrenia patients can be traced to the maintenance stage due to poor memory consolidation, and such deficit can be ameliorated both with memory cues and frontoparietal theta tACS.

ABSTRACT. Introduction: Previously, it has been shown that colour vision test scores improved by use of specific colour vision aids. However, there is knowledge gap on the performance of colour vision aids in real-world tasks. The aim of this study is to investigate the effect of colour vision aids in a visual search task. Methods: Three male subjects (aged 22-36 years) with congenital colour vision deficiency (identified based on the screening using HRR plates) were tested on Colour Assessment and Diagnosis (CAD) test to quantify the severity and type of colour vision defect. Written informed consent was obtained from participants before participating in the study. Visual search task was then performed on natural images (such as flowers, birds, animals, and sceneries etc) which has been chosen for specific colour combination pairs. The task was performed with and without colour vision aids (VINO, EnChroma). The order of testing was randomized. The outcome variables obtained were reaction time, accuracy, and performance index (product of speed and accuracy). A total of 60 images were presented in a random order using Psychopy software and maximum duration of 3 seconds was given to identify the target object. Instructions for the ‘target object’ was provided 3 seconds prior to image presentation. Results: The average R-G CAD test score was 27.24 ± 0.72 CAD units (1 deutan and 2 protans). The overall average reaction time with EnChroma (Indoor universal lens) improved (decreased 1.33 ± 0.17s) significantly compared to the baseline (1.54 ± 0.07s). Similarly, average performance index also improved (62.80 ± 14.18 %correct*s-1) with the EnChroma relative to baseline (51.97± 7.13 %correct*s-1). Conclusion: Preliminary results indicate the reaction times improve with EnChroma lenses relative to VINO and baseline. However, the type and severity of the defect may play a key role in determining the efficacy of these aids.

ABSTRACT. What kind of social network service (SNS) advertisements promote viewers' click behavior is essential for understanding advertising effectiveness online. Previous studies have revealed that the category for website can be recognized even with short presentations using the Rapid Serial Visual Presentation and the Scene Recognition tasks. However, it is not clear whether different categories of advertisements cause differences in click behavior. The present study examined prepotent motor responses in a Go/No-go task using SNS advertisements for functional cosmetics as a case study. Nineteen university students (8 females, 11 males; 22.47 ± 1.72 years old) participated in the two types of go/no-go tasks: (a) they pressed the key when an advertisement for acne was presented [Go] and did not respond to advertisements for moisturizing [No-go]; (b) they responded to the advertisement for moisturizing [Go] and did not respond to the advertisement for acne [No-go]. Participants observed the smartphone-sized pictures on color-calibrated monitors in a dark room. Stimuli were 26 pictures with products and the Japanese words of "acne (nikibi)" or "moisturizing (hoshitsu)”, selecting from a SNS advertising library. They conducted total of 312 trials (13 pictures × 2 types of advertising × 6 repetitions × 2 types of task). A paired t-test was used to compare the performance of the go/no-go task for the two types of advertising. The results showed that there was significant on sensitivity index d’ [t(18)＝ 6.72, p < .001, Cohen’s d = 2.31], however, response bias index c was no significant difference [t(18)＝ 1.32, p = .202, Cohen’s d = 0.35]. D-prime for the advertising of acne (d’ = 4.13) was higher than that of moisturizing (d’ = 2.98). The results indicated that SNS advertisements for the acne may be clicked on with lower impulse control than moisturizing.

ABSTRACT. Crossmodal correspondences refer to the matching tendency between certain features in one sensory modality and other features in another. Studies have demonstrated that people often match angular shapes with yellow hue or higher brightness, and rounded shape with red hue, particularly within visual contexts. However, this matching tendency shifts when people experience these shapes through touch alone, as angular shapes become associated with red hue or lower brightness, and rounded shapes with blue hue or higher brightness. Previous studies mostly used geometric shapes like circle and triangle, which may carry linguistic and social connotations. We used complex 3D-printed complex objects to systematically investigate how angularity and complexity of the objects influenced color matching tendencies across visual and haptic modalities. Nine objects and a color table with 10 hues * 3 brightness levels and 6 achromatic colors were used. We recruited 845 participants who either viewed or touched each object and selected the color best matched their perception of the shape. Our results showed that in the visual-only condition, participants tended to match angular shapes with yellow/gray hue, less angular shapes with pink hue, and rounded shapes with blue/brown hues. The influence of complexity was revealed in the most angular shapes with lower complexity associating with yellow hue. Conversely, participants in haptic-only condition tended to match angular shapes with red hue and lower brightness, while rounded shapes with blue/green hues and higher brightness. The influence of complexity was found in most angular shapes with higher complexity associating with lower brightness. These findings suggested different color matching tendencies may be linked to connotative meaning derived from the unique experiences of each modality. Visual shape–color associations may relate with evaluation (e.g., unpleasant/pleasant) or linguistic/social meanings (e.g., flower-like object), while haptic shape–color associations may be linked with evaluation or potency (e.g., hard/soft).

ABSTRACT. The advancement of autonomous driving technology and sophisticated safety features present significant potential for mitigating accidents. However, vehicles equipped with these supportive functions being widely accessible is years away. Hence, it is imperative to concurrently prioritize accident prevention measures alongside the widespread adoption of this technology. Recently, driving support systems utilizing extended reality have emerged. Conventional driving simulators face challenges in effectively assessing the efficacy of these new driving support systems. Therefore, the objective of this research was to develop a driving simulator capable of evaluating the effectiveness of such systems. To accomplish this goal, we designed a driving simulator that integrates virtual reality and a head-mounted display to monitor eye movements. Additionally, we conducted experiments to analyze driver eye movements during intersection navigation. By comparing these findings with real-world vehicle experiments, we demonstrated the effectiveness of the developed system. Furthermore, given the widespread integration of autonomous driving technology and advanced safety features, accurately evaluating the effectiveness of driving assistance systems and ensuring driver safety has become increasingly crucial. Hence, we aim to precisely measure the effectiveness of new driving assistance systems and establish criteria for promptly addressing any areas requiring improvement.

ABSTRACT. Our ability to act flexibly in response to environmental events is crucial in everyday life. However, there is limited understanding of how cognitive flexibility changes while our bodies are in motion. This study aimed to explore cognitive flexibility during walking and running using an anti-saccade task. The anti-saccade task allows us to investigate flexible behaviors and requires participants to either look at the peripheral target (pro-saccade) or suppress the stimulus-driven response and look in the opposite direction (anti-saccade). The experimental setup consisted of a treadmill in front of a large screen, with participants wearing eye tracker glasses. The stimulus sequence involved presenting a colored cross fixation point (1 deg) at the center of the screen for 1 second, followed by a circular target (1.2 deg) presented to the left or right (eccentricity: 5, 10 and 15 deg). Participants were instructed to move their eyes as quickly and accurately as possible in the same direction as the target if the fixation point was red (pro-saccade task) or in the opposite direction if it was green (anti-saccade task). There were three conditions: standing, walking, and running. In the walking condition, body sway was smaller than in the running condition, but the exercise load was increased by matching the walking speed to the running speed (7.5 km/h). Results showed that latencies for anti-saccades were longer than for pro-saccades under all conditions, with the difference being significantly larger during walking and running compared to standing. Furthermore, anti-saccade amplitude was reduced in the running condition compared to standing, and varied among individuals in the walking condition. These findings suggest that anti-saccade latency and amplitude reflect a decrease in cognitive flexibility in different ways. Amplitude appears to be more sensitive in indicating the subjective difficulty of body movements.

ABSTRACT. Humans move their heads and eyes in a coordinated manner to explore their surroundings. In our previous study (Fang et al., 2015), 20 participants watched a 15-minute movie on an ultrahigh-definition display (189 × 110 cm; 7,680 x 4,320 pixels) while freely moving their heads. The study revealed an eye-position bias toward the direction of the head during fixation. However, it remains an open question whether there is similar coordination of the head and eyes while shifting gaze. The present study aimed to investigate the relative movement of the head and eyes during gaze shifts, including saccades and pursuits. By analyzing the eye movement data of the previous study, we found that their movements depended on the amplitude and velocity of the gaze shift. When the amplitude of saccade or pursuit was large (≥ 10 deg.), the head and eye tended to move in the same direction as in the case of fixation. For pursuits with small amplitudes (3-10 deg.), the head and eyes tend to move in opposite directions. For saccades, in contrast, the head and eyes tend to move in the same direction. Additionally, we analyzed the vestibulo-ocular reflex (VOR), which is responsible for smooth eye movements with head movement to keep gaze at a location. The correlation coefficients between eye and head velocity were smaller than around -0.6 for both horizontal and vertical directions, suggesting that eye movements were able to compensate for head movements to a certain extent. These results demonstrate that eye-head coordination while watching natural scene movies is a complex process that depends on the amplitude and velocity of the gaze.

ABSTRACT. PURPOSE: We aim to explore the spatiotemporal characteristics of binocular interaction in unilateral amblyopia. METHODS: Six anisometropic amblyopes (26.2 ± 3.2 years) and 5 normal-sighted participants (24.0 ± 1.2 years) were recruited in this study. The stimulus (mask) was a sequence of 9 frames, each consisting of 11 vertical, spatially adjacent bars. The luminance value of each bar was sampled randomly from a uniform distribution spanning ± 10.5 cd/m2 around background luminance and scaled in amblyopes. In half of the trials, a brighter bar with incremental luminance (target) appeared in the middle of the central frame in the stimulus. Viewing the stimuli dichoptically, the participants were asked to detect whether the target appeared. Each participant completed the task with dichoptic settings in two separate sessions, i.e. mask at the dominant eye and target at the non-dominant eye and the other way around. By classification image technique, the mask images were classified into four types (i.e. hit, false alarm, miss, correct reject), and the binocular spatiotemporal interactive field (BSIF) of each participant was obtained by averaging the classified images. RESULTS: Each BSIF had a positive peak at the target location spatiotemporally, and negative at spatial flanks. The normal participants had a similar BSIF no matter to which eye the target was presented dichoptically. For amblyopic participants, however, the BSIFs of their two eyes were asymmetrical. Specifically, amblyopic eye had attenuated response with advanced and broader temporal window and expanded negative spatial flanks, while its fellow eye had a sharper response accurately tuned to the target. CONCLUSIONS: The BSIF of the amblyopic visual system showed abnormality in both spatial and temporal domains. The asymmetrical spatiotemporal processing measured in amblyopia can provide new insight to the dynamics of the binocular suppression in amblyopia.

ABSTRACT. Background: Amblyopia is a neurodevelopmental disorder that arises because of binocularly decorrelated visual experience during the critical period of visual development. Fixation stability is known to be poor in the amblyopic eye than the dominant eye. Therefore, one could hypothesize that the fixation stability should improve in the amblyopic eye with anti-suppression treatment. We investigated this in a patient.

Case description: A 28-year-old male, previously operated for congenital esotropia reported to our clinic wanting to improve the vision in his amblyopic right eye. His right and left eye acuity (measured using a thresholding protocol available in COMPlog visual acuity software) were 0.50 and 0.0 logMAR respectively. Stereoacuity was worse than 400” measured with Randot stereo test. Right eye showed a suppression scotoma of 5.2°, measured using VTS4 (Vision Therapy System 4). VTS4 presents dichoptic target to each eye, the size of which can be reduced. The size at which the amblyopic eye’s target disappears, gives the measure of the suppression scotoma.

Fixation stability: was measured using Eyelink 1000 eye tracker (500Hz) when the patient viewed a fixation target (0.65°) for 1-minute. A bivariate contour ellipse area (BCEA) was fitted and calculated from the fixational eye movements. BCEA was measured pre-, during, and post-vision therapy exercises. Lower values denote better fixation stability.

Outcomes: The patient underwent anti-suppression vision therapy exercises for 3 months. BCEA for fixation stability improved in the post-therapy visit (0.73 deg-sqr) when compared to the pre-therapy visit (1.56 deg-sqr) in the amblyopic eye. Visual acuity improved to 0.38 logMAR. Suppression scotoma size reduced to 0.5°. No improvement in stereoacuity was observed.

Conclusion: Fixation stability as measured with BCEA can be an additional marker to monitor the amblyopic eye with vision therapy. Further studies are needed to correlate fixation stability with other measured visual outcomes in patients with amblyopia.

ABSTRACT. Most forms of induced and inherited retinal degeneration involve induction of local inflammation that is characterized by changes in retinal Müller glia and microglia, and at least some of these changes accelerate vision loss in advanced blinding diseases. The specific triggers, hierarchy of molecular steps, and cellular interactions that promote these changes initially are still largely obscure. However, it is generally thought that Müller glia and microglia activation is secondary to stress signals and cellular debris due to programmed cell death of photoreceptors or other retinal neurons. Here, we report early changes in gene and protein expression in acutely induced or inherited, chronic mouse and rat models of retinal pigment epithelium (RPE) injury or dysfunction. We identify patterns of cytokine communication originating in the RPE. We also find that RPE cytosolic inflammatory signaling is dependent on the Tyro3/Axl/MERTK (TAM) family of receptor tyrosine kinases. RPE-based inflammation is sufficient to activate resident Müller glia and microglia cells of the retina. RPE inflammation is detectable early postnatally in mutant murine retina, at ages prior to full retinal maturity and visual function when there is no evidence of neural retina distress let alone neuronal death. Finally, we show that either reducing microglia activation pharmacologically or cytokine reduction through gene knockout is sufficient to delay photoreceptor degeneration. Altogether, our results suggest that targeted pharmacological interference with the early inflammatory crosstalk of RPE, Müller glia and microglia that precedes neuronal death may be an effective therapeutic approach for retinal degenerations that are yet without cure.

ABSTRACT. Our previous studies have shown that our perceptual learning technology successfully improved functional vision in amblyopia, presbyopia and processing speed in young participants. Our recent data demonstrates that brain processing can be significantly improved beyond the expected saturation levels and overcome compromised (blurred) visual input regardless of the age and the impaired visual condition. Trainees trained on a dedicated application pre-installed on a tablet device of short sessions (7-10 minutes). We trained 3 different eye conditions and age groups. Results are presented as mean ±STD and VA in logMAR. 1) Low myopia subjects (N= 18, 21-30 years), (0.00-1.25, spherical equivalent -0.49±0.43). Uncorrected Distance VA improved by 0.16±0.03 after 8 sessions. 2) Presbyopia subjects (N= 19, 59.4±6.1 years), initiated primary training at the average age of 52±6 years, and followed with 1-2 sessions of boosting/maintenance monthly for an average of 7 years (range 5-12). At current age, measured near addition was +2.15±0.36 diopters (expected near VA to be at 0.7±0.1), while near VA was 0.18 ±0.12, demonstrating long-term 0.52 lines of improvement attributed to improved blur processing in the brain. 3) Dry AMD patients with Geographic Atrophy (GA) (N=7, 77.2 ±5.3 years). There is no known treatment nor drug that improves near VA in subjects with GA, and they lose an average of 3-5 letters per year. The patients initiated primary training of 24 sessions. Average near VA remarkably improved by 0.19±0.12, potentially indicating this specific perceptual training method as the only solution to prevent degradation of VA in AMD patients. Our results show that our technology enables to enhance abnormal retinal input in a wide range of age groups with varies condition of the visual deficit. The remarkable amount of long-term improvement in presbyopia may suggest a parallel outcome in low myopia that will be tested in the future.

ABSTRACT. Stochastic resonance (SR), a phenomenon wherein optimal amounts of external noise can enhance performance, has predominantly been investigated in the context of binocular vision. However, the potential impacts of SR on monocular vision have yet to be explored comprehensively. Previous research has employed the double-receptor design (noise and signal presented to different eyes) to infer where the signals interact, and found SR persisted. This finding suggests that signal and noise interact at an interocular level. However, this method does not reveal whether SR can occur at a strictly monocular level. For our approach, we employed the single-receptor design (signal and noise presented to only one eye, and an eye-patch over the other eye) to examine whether SR occurs at a monocular level. Overall, our findings indicate an absence of SR for monocular vision. We retested this on a small sample of participants who previously achieved SR in the binocular condition, and they also did not show SR in the monocular condition. In conclusion, our study indicates that SR does not occur at strictly monocular levels, and requires interocular interactions, perhaps depending on binocular summation.

ABSTRACT. The implicit nature of learning the statistical structure in the sensory environment has been repeatedly demonstrated. On the other hand, we seem to be able to explicitly track the co- occurrences of sensory objects and utilize such information for survival. Here we tested if implicit/unconscious probability tracking occurred in an exogenous cueing paradigm. When both the cue (a disc) and the target (a Gabor patch) were visible, how well the cue predicted the target location (cue validity) was initially unbeknown to the observer. A canonical cueing effect was found: target reaction time decreased when the target appeared in the cued location, as compared to when it did not. This cueing effect scaled with the actual validity, but subjective awareness of the validity did not alter the effect. This was shown both with ascending and descending validity changes. We further directly examined the role of explicit knowledge in this cueing effect. While keeping the validity constant (70%), we asked the participants to estimate the validity or gave them an explicit prime of the validity. In both experiments, the cueing effect persisted but was irrelevant to the reported or primed validity, suggesting that the effect was immune to the explicit perception of validity. Furthermore, when the cue was masked and made invisible, the same cueing effect was found. Intriguingly, the direction of the cueing effect in the later test blocks was determined by the cue-target probabilistic relationship in the preceding training blocks. Together, these results lend support to the dominant role of implicit gathering of the probabilistic structure of the sensory objects.

ABSTRACT. For survival, the brain builds up an internal model to track the statistics of our surroundings. Recent evidence shows that the locus-coeruleus (LC) – norepinephrine (NE) neuromodulatory system plays a crucial role in such processes, revealed by pupillary responses. In two studies, we demonstrated that (1) tonic baseline pupil size correlates with auditory bistable percept (i.e., the ABA sequence) and (2) phasic pupil dilation response tracks unexpected uncertainty in rapid tone- pip sequences. In the first study, we found that tonic baseline pupil size, an index of temporal fluctuation of arousal level of the LC-NE system, monotonically increased with an increasing number of perceptual alternations and its occurrence probability. The prediction of the baseline pupil size to the perceptual alternations was observed at least 35 s before the behavioral response and maintained over a sustained time window of 45 s at minimum. In the second study, we looked into the phasic pupil dilation response (PDR), which arose immediately following the critical event and peaked within 2 s. Participants listened to an auditory sequence containing a transition from a repeating, regular (REG) frequency structure to a random (RAND) structure or vice versa. The unexpected uncertainty was defined when the model resetting was required, i.e., REG-to-RAND but not RAND-to-REG. Despite being task-irrelevant, phasic PDR specifically tracked the transition in the REG-to-RAND condition but not RAND-to-REG. The overall results indicate that pupil-linked arousals track auditory information extraction at different timescales, presumably capturing different characteristics of the LC-NE system modulations.

ABSTRACT. Visual statistical learning (VSL) occurs when participants are exposed to spatially or temporally ordered stimuli and become increasingly sensitive to them without explicitly realizing the hidden regularities. The measurement of VSL, however, can be quite tricky and often relies on participants guessing the correct answer above chance level. To address this issue in the temporal domain of VSL, some researchers have utilized reaction time in a rapid serial visual presentation (RSVP) task to provide a more objective measurement of implicit learning. Here, we aim to demonstrate that previous implementations of the VSL-RSVP task may have been biased due to a lack of counterbalanced design of stimuli order. Indeed, the previously reported VSL effect in RSVP is eliminated when the RSVP stream is fully counterbalanced (Experiment 1), but resurfaces when the RSVP stream returns to its original pseudorandomized design (Experiment 2), even in the absence of any implicit learning (Experiment 4). Importantly, implicit learning is still observable when we switch back to the traditional familiarity-based measure (Experiment 3). Therefore, although the traditional familiarity-based measure relies to some extent on participants’ subjective reports, it is currently still the best measurement we have for temporal VSL. Together, these results suggest that implicit learning is real but can be easily masked by task demand from the dependent measure. Future studies should pay attention to the design of the dependent measure as they strive to uncover the presence of implicit processing and learning.

ABSTRACT. The current view of perceptual decision-making suggests that once the decision is made, a single motor program associated with the decision is carried out, irrespective of the uncertainty involved in the decision-making. As opposed to this view, we show that multiple motor programs can be acquired based on the preceding uncertainty of the decision, indicating that decision uncertainty works as a contextual cue for motor memory. Participants judged the direction (left or right) of a visual motion (random-dot motion) presented on the screen. They were asked to reach towards the target of the perceived motion direction by moving a manipulandum with their right hand. The reaching movement following the decision was performed under a velocity-dependent force-field, which required the participants to learn and compensate for the perturbation to make a straight reaching movement. Uncertainty of the decision was manipulated by changing the coherence level of dots motion. Actions learned following certain (uncertain) decisions only partially transferred to uncertain (certain) decisions. Participants were able to form distinct motor memories for the apparently same movement based on the preceding decision uncertainty. Crucially, such contextual effect generalized to novel stimuli with matched uncertainty levels, demonstrating that the decision uncertainty itself is the contextual cue. Our findings broaden our understanding of contextual inference for motor memory, emphasizing that it extends beyond the direct motor control cues to encompass the state of the decision-making process.

ABSTRACT. Walking is a very frequent voluntary behaviour yet the consequences of locomotion upon perceptual and cognitive function remain largely unknown. Recent work has highlighted that although walking feels smooth and continuous, critical phases exist within each step for the successful coordination of perceptual and motor function. Here, we test whether these phasic demands impact upon perception by assessing performance on visual and auditory tasks during natural walking in a wireless, motion-tracked, virtual reality (VR) environment. We finely sampled perceptual performance over the stride-cycle as participants walked along a smooth linear path at a comfortable speed by using head-height data to define the start and end of each stride so that stimulus presentation could be defined relative to the stride cycle. Our analyses uncovered a previously unknown link between perception and walking. At the group-level, accuracy and reaction times both showed strong oscillations at approximately 2 cycles-per-stride (~2 Hz) for both visual and auditory stimuli and the phase of the performance oscillation was locked to the gait cycle. For vision, optimal performance occurred during the swing phase between each footfall, while for audition optimal performance had the opposite phase and was best at footfall. There were also oscillations of behaviour (button press and saccades) at ~2 Hz which peaked at footfall, revealing an entrainment of the timing of motor actions. These findings reveal a new link between perception and action and underscore that perception in an active environment may differ from traditional lab-based experiments.

ABSTRACT. A recent finding is that New World flying squirrels (Genus Glaucomys) fluoresce under ultraviolet light, suggesting that these nocturnal mammals might use UV-induced fluorescence as a signal. Other mammals may also fluoresce under UV, particularly some marsupials. We explored whether there was convergent evolution between New World flying squirrels and Southeast Asia’s gliding Sunda colugos, Galeopterus variegatus, by testing colugos for fluorescence under UV light, along with an outgroup (tree shrews; Genus Tupaia) and Southeast Asian flying squirrels (Subfamily Sciurinae; Tribe Pteromyini). We found little evidence of UV-induced fluorescence in the mammals we tested. We discuss the difficulties of testing fluorescence rigorously. We also discuss the functional significance, if any, of nocturnal mammals that fluoresce under UV light. Many of the mammals that apparently fluoresce under UV can see well into the UV range. We pose the question, What function might UV-induced fluorescence serve, given the visual sensitivity of the mammals in question? Are these mammals using UV-induced fluorescence as signals? And how could elucidate what function, if any, UV-induced fluoresence has?

ABSTRACT. Stereoscopic vision refers to the ability to extract depth from the data captured by the two eyes. Here, I will present how spiking neural networks (SNNs) can be used to better understand stereoscopic vision in biological systems and to develop neuromorphic approaches for depth perception. I will first describe a recent computational model based on an SNN equipped with a bio-inspired plasticity rule (spike-timing dependent plasticity or ‘STDP’) in order to simulate the emergence of stereoscopic vision in primates (Chauhan et al., 2018; 2020). When trained with stereo images of natural scenes, this model not only converge to realistic single-cells selective to binocular disparity, but also demonstrates how known biases in natural statistics may influence population encoding and downstream correlates of behavior. The model can also be used to characterize the deficit of binocular vision observed in strabismic amblyopia. In a second section, I will describe how a new SNN with a deep architecture (‘StereoSpike’, Rancon et al., 2022) and trained with the surrogate gradient descent can extract precise and dense depth predictions from the data collected by two event-based cameras. Through regularization, this SNN can reach a very low firing rate (<5%), with a minimal cost in accuracy. StereoSpike can thus be efficiently implemented on neuromorphic chips, opening the door to precise depth estimation on low power and real-time bio-inspired embedded systems.

References:

- Chauhan, T., Masquelier, T., Montlibert, A., & Cottereau, B. R. (2018). Emergence of binocular disparity selectivity through Hebbian learning. Journal of Neuroscience, 38(44), 9563-9578.

- Chauhan, T., Héjja-Brichard, Y., & Cottereau, B. R. (2020). Modelling binocular disparity processing from statistics in natural scenes. Vision Research, 176, 27-39.

- Rançon, U., Cuadrado-Anibarro, J., Cottereau, B. R., & Masquelier, T. (2022). Stereospike: Depth learning with a spiking neural network. IEEE Access, 10, 127428-127439.

ABSTRACT. It is well known that the intrinsically photosensitive retinal ganglion cells (ipRGCs), which include the photopigment melanopsin are known to perform functions of circadian photoentrainment and control of the pupillary light reflex. However, their contribution to human spatial vision remains largely unknown. This study has focused on a contribution of ipRGC to spatial contrast sensitivity function (sCSF) in the luminance pathway, a fundamental function of pattern vision. A multi-primary stimulation system with a silent substitution technique was used to isolate melanopsin stimulation. We varied the amount of background melanopsin stimulation and measured spatial contrast sensitivity functions at difference eccentricities. The results showed that increased background melanopsin stimulation enhanced contrast sensitivity. Furthermore, the receptive field analysis based on the DoG model, Difference of Gaussians, indicated that contrast sensitivity is detected by the spatially center-surround opponent receptive fields. The receptive field center size corresponds to the dendritic field size of retinal parasol ganglion cells, even at different eccentricities. This suggests that contrast sensitivity is detected by retinal parasol ganglion cells, which is consistent with previous studies. Increased background melanopsin stimulation resulted in facilitated excitatory-center and inhibitory-surround of the spatially opponent center-surround receptive field of parasol retinal ganglion cells. These results indicated that changing the amount of melanopsin stimulation in the background can modify the structure of the receptive fields of the parasol retinal ganglion cells. In other words, the parasol retinal ganglion cells receive signals from ipRGCs to modulate the receptive field structure.

ABSTRACT. Why don’t we see everything that we want to see or that we think we should see? From typos to tumors to gorillas in Introductory Psychology demos, humans manage to routinely miss targets that are "right in front of our eyes". In the psychologically most interesting cases, these are instances where observers fail to respond to stimuli that are clearly visible. The observers may even directly fixate on those stimuli. Some of these “Looked But Failed to See” (LBFTS) errors are life-threatening (e.g. missing that tumor or a weapon in carry-on baggage). Nevertheless, such errors are common enough that we can describe them as “normal blindness". If you are a radiologist in the US, these errors are also the sort of errors that end up in court (“Doctor, would you say that this spot that I am pointing at is a sign of cancer?” If the answer is “yes”, does that mean that it is “negligent” to have missed the diagnosis?). I will give a four-part account of LBFTS errors: Our vision and attention are 1) limited, 2) misguided, 3) incomplete, and 4) misunderstood. All four of these “problems” have evolved to allow us to move through the world, but they are virtually guaranteed to cause us to miss some significant stimuli. I will illustrate with some demonstrations and with an emphasis on radiology. If all goes well, you will look, you will fail to see, and you will gain insight into why that happens.