ABSTRACT. Noninvasive brain stimulation (NIBS) has gained lots of attention from both academics and clinical usage. Its curative effect shows positive feedback in different kinds of neurological and ophthalmological disorders. Stroke is one of them that could benefit from this new technology. However, the unknown underlying mechanism of brain stimulation hinders our further exploration of brain recovery. Studies on the prediction of possible recovery rates with brain network features are rare. This study proposes a hybrid graph convolutional autoencoder (HGCAE) predict stroke recovery potential after electrical stimulation therapy. Twenty-four occipital stroke patients were randomly assigned to one of three groups, receiving different NIBS interventions. After two months, we identified the responders based on visual performance. The results show that using HGCAE based on brain network measures achieved an overall sensitivity of 91% of predicting recovery following NIBS intervention. This result may help predict the potential outcome of neuronal modulation in stroke patients and allows us to gain more insight into clinical interventions using neuromodulation.

ABSTRACT. In the rapidly evolving world of remote work, mobile telepresence robots (MTRs) have become increasingly popular, providing new avenues for people to actively engage in activities at a distance. The existing studies indicate, however, that remote navigation around humans in dense environments can be challenging for humans, resulting in a decreased level of satisfaction. Work on shared autonomy for navigation has generally addressed static environments or situations where only one pedestrian interacts with the robot. In this paper, we present our ongoing work on SocNavAssist, a haptic shared autonomy framework for navigation assistance of mobile telepresence robots in human-populated environments. It uses a modified approach of reciprocal velocity obstacles to consider social constraints in dynamic collision avoidance. We also provide visualization of system intent via predicted trajectories on an augmented visual feedback interface to enhance transparency and cooperation. In addition, we outline the proposed experiment to be used in future work to evaluate the proposed framework.

ABSTRACT. The critical role of unconscious processing in problem solving had been proved by previous empirical evidence. However, we still could not understand how unconscious mind process information differently from the conscious mind. One previous study utilized an event-related potential approach to demonstrate the different processes we went through during processing unconscious cue and conscious cue. In addition to this sequential and localized processing view, this study reanalyzed the data from that study and aimed to investigate how the default mode network and attention network reacted to and engaged in the processing of an unconscious cue. Functional connectivity was measured and graph theory was utilized to obtain the network dynamics. The results showed that compared to the no cue condition, the conscious cue condition did not exhibit a different communication pattern within the default mode network and the attention network. But after the presentation of the unconscious cue, the default mode network and attention network both reacted with enhanced communication within the network. It suggests that if a stimulus is presented below the perceptual threshold, it will activate the default mode network, which might support the divergent activation of the cue. The attention network might be engaged to monitor the default mode network and inhibit some unwanted divergent activation.

ABSTRACT. This study presents a novel approach for familiar and unfamiliar face classification based on electroencephalography (EEG). Firstly, the raw EEG epoch is temporally split into three overlapped segments, and each segment is decomposed into multiple sub-bands by band-pass filters. Then, differential entropy is employed to extract discriminative EEG features. Finally, the obtained features are concatenated and classified with the support vector machine (SVM). The results yielded on our database indicate that the proposed method can achieve a mean accuracy of 76.2% over five participants. This work primarily demonstrates that differential entropy is an effective feature for EEG-based familiar and unfamiliar face classification, and has the potential to be applied to other EEG-based visual task analyses.

ABSTRACT. Driving distraction is a main human factor of traffic accidents. Distraction would seriously affect the drivers’ cognitive process, inducing the inability to fully perceive the surrounding environment, make the correct judgments and perform the proper operations in time. It is important to identify the drivers’ attentional state accurately and quickly during the driving process. The objective of this study was to develop a novel driving distraction detection method based on electroencephalographic (EEG) signals. A simultaneous driving and distraction experiment was designed, in which the alternative secondary task with a 2-back paradigm were utilized to induce the visual or auditory distraction. The EEG signals of 22 subjects were analysed to distinguish the focused state of the driver from distraction. Results indicated that, the proposed method based on EEGNet and long short-term memory (LSTM) provided an average classification accuracy of 71.1% in three-class classification. Reducing the number of the electrodes from 63 to 14 would not significantly reduce the accuracy, so that a higher model efficiency could be obtained.

ABSTRACT. The goal of this research is to utilize joint angle tracking from depth sensors to design a real-time, powered ankle-foot prosthesis assistive algorithm. These sensors are used primarily for computer vision tasks surrounding rehabilitation; however, their use in motor feedback control is limited. By extracting joint positions from a depth map, the joint angle will be calculated and along with angular velocity collected from an external IMU, are used to train a neural network. This will be deployed alongside human-in-the-loop optimization to determine how the most effective prosthetic regimen can be found in the most efficient manner. The 3D joint angle method is evaluated with EMG data and metabolic cost from prosthetic use and is compared to current control methods.

ABSTRACT. Understanding crowd behavior has attracted tremendous attention from researchers over the years. In this work, we propose an unsupervised approach for crowd scene modeling and anomaly detection using association rules mining. Using object tracklets, we identify different paths/routes, i.e., the distinct \textit{events} taking place at various locations in the scene. Interval-based frequent temporal patterns characterizing the scene model are mined using a temporal mining algorithm using Allen's interval-based temporal logic. The resulting frequent patterns are used to generate temporal association rules, which convey the semantic information contained in the scene. Our overall aim is to generate rules that govern dynamics of the scene. Finally the anomalies, both spatial and spatio-temporal, are found by considering behavioral interactions among different objects. We apply the proposed approach on a publicly available dataset and demonstrate its efficient use.

ABSTRACT. The concerns over radiation-related health risks associated with the increasing use of computed tomography (CT) have accelerated the development of low-dose strategies. There is a higher need for low dosage in interventional applications as repeated scanning is performed. However, using the noisier and undersampled low-dose datasets, the standard reconstruction algorithms produce low-resolution images with severe streaking artifacts. This adversely affects the CT assisted interventions. Recently, variational autoencoders (VAEs) have achieved state-of-the-art results for the reconstruction of high fidelity images. The existing VAE approaches typically use mean squared error (MSE) as the loss, because it is convex and differentiable. However, pixel-wise MSE does not capture the perceptual quality difference between the target and model predictions. In this work, we propose two simple but effective MSE based perception-aware losses, which facilitate a better reconstruction quality. The proposed losses are motivated by perceptual fidelity measures used in image quality assessment. One of the losses involves calculation of the MSE in the spectral domain. The other involves calculation of the MSE in the pixel space and the Laplacian of Gaussian transformed domain. We use a hierarchical vector-quantized VAE equipped with the perception-aware losses for the artifact removal task. The best performing perception-aware loss improves the structural similarity index measure (SSIM) from 0.74 to 0.80. Further, we provide an analysis of the role of the pertinent components of the architecture in the denoising and artifact removal task.

ABSTRACT. Social robots are widely applied in various contexts to provide human-like assistance and facilitate service experience. Prior research considered a variety of design features to explore the influences in human-robot relationships, while a robot’s manner of assisting in interaction and its consequent effects are rarely discussed. This study aims to investigate the relationship between a robot’s communication design and a human’s perceived intimacy in the human- humanoid robot interaction. Different levels of service proactivity (proactive vs. reactive) and types of expressive behaviors (neutral vs. intimate) are developed and empirically validated through an online survey. The findings indicate that the manipulations designed for each experimental condition can be recognized by the participants. In addition, the perception of intimacy is significantly affected when interacting with different robots accompanying different types of behaviors.

ABSTRACT. Anomaly detectors are used to distinguish the difference between normal and abnormal data, which are usually implemented by evaluating and ranking anomaly scores of each instance. Static unsupervised anomaly detectors can be difficult to adjust anomaly score calculation for streaming data. In real scenarios, anomaly detection often needs to be regulated by human feedback, which benefits to adjust anomaly detectors. In this paper, we propose a human-machine interactive anomaly detection method, named ISPForest, which can be adaptively updated under the guidance of human feedback. In particular, the feedback will be used to adjust the anomaly score calculation and structure of the tree-based detector, ideally attaining more accurate anomaly scores in the future. Our main contribution is to improve the tree model that can be dynamically updated from perspectives of anomaly score calculation and the model’s structure. Our approach is instantiated for the powerful class of tree-based anomaly detectors, and we conduct experiments on a range of benchmark datasets. The results demonstrate that human expert feedback is helpful to improve the accuracy of anomaly detectors.

ABSTRACT. Interactive Information Visualization and Human Computer Interaction provides useful support for inexperienced user and experts as well. On one hand visualization provides an informative overview of data and on the other hand interaction encourages users for exploration within it. We chose the challenge of a specialized/expert scenario to build a pipeline that provides interactive visualizations to encourage users for further exploration. We chose the complex task of a phylogenetic analysis of SARS-CoV-2 genomes for mutation tracking. The proposed pipeline hides the mathematical details while providing complex information visually and intuitively. In our proof-of-concept we analyzed four variants of concern and identified mutations in the spike glycoprotein with more than 70% precision and 77% recall in reference to the reports of the Centers for Disease Control and Prevention.

ABSTRACT. Depth perception is a common issue in augmented reality (AR). Projective AR, where the spatial relations between the projection surface and displayed virtual contents need to be represented properly, is particularly affected. This is crucial in the medical domain, e.g., for the distances between the patient's skin and projected inner anatomical structures, but not much research was conducted in this context before. To this end, this work investigates the applicability of surface visualization techniques to support the perception of spatial relations in projective AR. Four methods previously explored in different domains were combined with the projection of inner anatomical structures on a human torso phantom. They were evaluated in a comparative user study (n=21) with respect to a distance estimation and a sorting task. Measures included Task completion time, accuracy, total Head movement and Confidence of the participants. Consistent results across variables show advantages of more occluding surface visualizations for the distance estimation task. Opposite results were obtained for the sorting task. This suggests that the amount of needed surface preservation depends on the use case and individual occlusion compromises need to be explored in future work.

ABSTRACT. Flexible production sites will become increasingly important in the future. In order to open up this path for SMEs as well as for larger companies, solutions are needed that do not rely on expensive special machines and can be used universally in a wide variety of application areas such as separation or sorting. For a highly flexible gripper with sensory capabilities, which is currently under development by the Fraunhofer IFF, suitable drives have to be identified that would meet all the required specifications and at the same time allow a low price of the final product. This paper describes an universal test setup that can be used to evaluate combinations of motors and gearboxes from different manufacturers and product lines for their suitability for use in a robotic gripper. The measurement procedure is divided into three parts: determination of the motor constant, torque accuracy and the positioning accuracy of the complete drive. The method as well as the results from the investigation of a motor-gearbox combination are presented.

ABSTRACT. Collaborative robotics are increasingly finding use beyond the traditional domain of manufacturing, in areas such as healthcare, rehabilitation, agriculture and logistics. This development greatly increases the size and variations in the level of expertise of cobot stakeholders. This become particularly critical considering the role of human safety for collaborative robotics applications. In order to support the wide range of cobot stakeholders, the EU-funded project COVR “Being safe around collaborative and versatile robots in shared spaces” has developed a freely available, web-based Toolkit that offers support to understand how to consider the safety of cobot applications. This paper describes the state of the art for ensuring safety across various life cycle phases in the development and implementation of collaborative robotics applications and highlights how the Toolkit provides practical support during these tasks. The Toolkit aims to be the most comprehensive resource for supporting cobot stakeholders in ensuring the safety of their applications.

ABSTRACT. Optical sensor systems are widely used in academic and research with the aim of providing safety of humans at human-robot collaboration (HRC). For using these mainly 2D- or 3D- cameras as speed and separation monitoring systems in industrial HRC settings several requirements need to be fulfilled. Beside the overall functional safety of the sensor, the monitoring system have to deal with changing environmental influences as well. This includes not only natural illumination changes of ambient light (sunlight) but also high frequency pulsation of flashlights (stroboscopic effects). In recent years, an innovative optical sensor system based on camera and projector techniques was introduced. The implemented working principle of this HRC monitoring system offers high potential to meet the standardized safety requirements. In this paper, we present the extension of the pulse-modulated light principle to a space-time approach that is capable of handling sudden illumination changes (e.g. flashlights) without falling to danger.

ABSTRACT. This paper presents RoSA, the Robot System Assistant, a concept for intuitive human-machine-interaction based on speech, facial, and gesture recognition. The interaction modalities were found and reviewed through a preceding wizard-of-oz study showing high impact for speech and pointing gestures. The system's framework is based on the Robot Operating System (ROS), allowing modularity and extendibility. This contactless concept also includes ideas for multi-device and multi-user implementation, working at different workstations.

NOTE TO EDITORS // REVIEWERS We have updated this paper (original: Submission 19) regarding the remarks made by editors and reviewers. We were asked by the chairs to upload the submission to the “Interactive Robotics” track. This is the updated version as a short paper (4 pages). By the time we present the paper, we will have completed the study and can share the first results.

ABSTRACT. In the first step, a 1-DOF power assist robotic system (PARS) was developed for object manipulation with it, and the dynamics for human-robot co-manipulation of objects was derived reflecting human cognition (weight perception). Then, an admittance control scheme with position feedback and velocity controller was derived from the weight-perception-based dynamics. In a user study, human subjects lifted objects with the system. An evaluation scheme was developed to evaluate human-robot interaction (HRI) and co-manipulation performance. A reinforcement learning method was implemented to learn the admittance control parameters resulting in satisfactory HRI and manipulation performance. The results showed that inclusion of weight perception in the dynamics and the learning control were effective to produce satisfactory HRI and performance. In the second step, a novel variable admittance feedforward adaptive control algorithm was proposed, which helped further improve the HRI and manipulation performance. Then, effectiveness of the adaptive feedforward learning control method was validated using a multi-DOF PARS for manipulating heavy objects.

ABSTRACT. Brain-computer interfaces (BCIs) provide a direct pathway of communication between humans and computers. There are three major BCI paradigms that are commonly employed: motor-imagery (MI), event-related potential (ERP), and steady-state visually evoked potential (SSVEP). In our study we sought to expand this by focusing on “Inner Speech” paradigm using EEG signals. Inner Speech refers to the internalized process of imagining one’s own “voice”. Using a 2D Convolutional Neural Network (CNN) based on the EEGNet architecture, we classified the EEG signals from eight subjects when they internally thought about four different words. Our results showed an average accuracy of 28.5% for word recognition, which is slightly above chance. We discuss the limitations and provide suggestions for future research.

ABSTRACT. Considering the use of a multi-modal framework to enhance emotion recognition, we propose to combine electroencephalography (EEG) and electrooculogram (EOG) through decision level fusion(DLF) and feature level fusion(FLF) for emotion recognition. By using different temporal window sizes to segment the signal, we explore the duration of the emotion of the EOG signal and the EEG signal. Then, some temporal window sizes that are friendly to both EOG signal and EEG signal are selected for segmentation and emotion recognition. According to the different degree of dependence of subjects, the accuracy of the proposed algorithm on subject-dependent and subject-independent is verified on the DEAP dataset. For subject-dependent, using feature level fusion strategy with a window size of 6 seconds, the accuracy is 0.9562 in terms of arousal, and 0.9558 in terms of valence. For subject-independent, using feature level fusion strategy with a window size of 5 seconds, the accuracy is 0.8638 in terms of arousal, and 0.8542 in terms of valence. The experimental results show that the proposed algorithm can better enhance emotion recognition.

ABSTRACT. With the increased collaboration between humans and robots in daily life, face recognition becomes one of the essential aspects of human-robot interaction (HRI). The robot requires a highly accurate face recognition system to be able to operate in different environments and conditions. The role of achieving a high accuracy face recognition is the enhancement of learning discriminative feature representation, which is almost entirely around minimizing the intra-class distance and maximizing the inter-class distance. The loss function is used on the deep Convolutional Neural Networks (CNNs) to enhance this discriminative power of the deeply learned features. Softmax loss is one of the most used loss functions in many CNNs. However, Softmax loss didn't have the sufficient discriminative power needed by face recognition. Recently, many researchers work on developing novel loss functions to improve discriminatory power mainly, the intra-class distance of deep features. This survey paper's main objective is to compare the multiple loss functions used for deep face recognition networks showing the weakness for each loss function.

ABSTRACT. The purpose of this paper is to describe a novel fusion algorithm for multimodal biometric identification. In this paper we describe the fusion of fingerprints and voice. This combination of biometrics is rarely used in verification systems although this biometric pair is simple to use and not too invasive. A framework for the combination of several data fusion algorithms is described. In this paper we use only two types of data fusion techniques, namely weighted sum and fuzzy system. Two independent identity decisions can be thus obtained, and from them two beliefs that the identity is verified can be derived. The two beliefs are combined using Dempster Shafer’s approach to obtain the final decision. The results are reported by ROC curves.

ABSTRACT. Adaptive information retrieval systems could provide users individual support for their current search activity. For that a clear detection and understanding of such activities are a necessity. In this paper we analyze reading strategies, such as Scanning, Skimming and ’hard’ Reading, given two search activities, namely Fact-Finding and Exploratory search activities. We analyzed the eye tracking data of a lab experiment and identified a positive correlated trend of ’hard’ Reading towards Exploratory searches and Skimming towards Fact-Finding searches. Using the interpretation of these reading strategies, we argue that we are able to draw conclusions about the possible search intent during individual search activities.

ABSTRACT. With human-agent teams beginning to enter the workforce, it is important that humans are well equipped to lead their future teams. Due to the addition of artificial intelligence to teams, the behavioral functions of leaders need to be critically examined to determine their fit with the future of human-agent teamwork. This paper identifies these functional behaviors as resource management behaviors and information behaviors based on past research in teamwork. These behaviors are reviewed within the context of human-human teamwork to define human-oriented leadership behaviors. Based on the review of human-human teamwork along with recent research in human-agent teamwork, an adaptable framework is created for leadership behaviors that will help guide human leaders in human-agent teams. This framework provides a foundation for future human-agent teams to empower and guide human leaders of human-agent teams who need to mediate the integration of agents alongside humans.

ABSTRACT. The management of thermal comfort in a building is a challenging and multi-faced problem because it requires considering both objective and subjective parameters that are often in contrast. Subjective parameters are tied to reaching and maintaining an adequate user comfort by considering human preferences and behaviours, while objective parameters can be related to other important aspects like the reduction of energy consumption. This paper exploits cognitive technologies, based on Deep Reinforcement Learning (DRL), for automatically learning how to control the HVAC system in an office. The goal is to develop a cyber-controller able to minimize both the perceived thermal discomfort and the needed energy. The learning process is driven through the definition of a cumulative reward, which includes and combines two reward components that consider, respectively, user comfort and energy consumption. Simulation experiments show that the adopted approach is able to affect the behaviour of the DRL controller and the learning process and therefore to balance the two objectives by weighing the two components of the reward.