ABSTRACT. Conic sections are extensively encountered in a wide range of disciplines, including optics, physics, and various other fields. Consequently, the geometric algebra community is actively engaged in developing frameworks that enable efficient support and manipulation of conic sections. Conic-conic intersection objects are known and supported by algebras specialized in conic sections representation, but there is yet no elegant formula to extract the intersection points from them. This paper proposes a method for point extraction from an conic intersection through the concept of pencils. It will be based on QC2GA, the 2D version of QCGA, that we also prove to be equivalent to GAC.

ABSTRACT. The detection of concavity and convexity of vertices and edges of three-dimensional (3D) geometric objects is a classic problem in the field of computer graphics. As the foundation of other related graphics algorithms and operations, scholars have proposed many algorithms for determining the concavity and convexity of vertices and edges. However, existing concavity and convexity detection algorithms mainly focus on vertices and lack research on concavity and convexity detection methods for edges of 3D geometric objects. On the other hand, existing algorithms often require different detection methods when dealing with two-dimensional (2D) planar geometric objects and 3D spatial geometric objects. This means that the algorithm structure of those algorithms becomes very complex when dealing with concavity and convexity judgments involving both planar polygon vertices and 3D geometric object edges. To solve the above problems, this paper design a multi-dimensional unified concave convex detection algorithm framework for geometric objects taking advantages of geometric algebra in multi-dimensional unified expression and calculation. The method proposed in this article can not only achieve concavity and convexity detection of planar polygon vertices and 3D geometric object vertices based on unified rules, but also further achieve concavity and convexity detection of 3D geometric object edges on this basis. By unifying the framework and detection rules of different dimensional geometric object concavity detection algorithms, the complexity of synchronous detection algorithms for planar polygon vertices and 3D geometric object vertices and edges concavity can be effectively simplified.

ABSTRACT. The quality of multi-focus microscopic image fusion hinges upon the precision of the image registration technology. However, algorithms for registration tailored specifically for multifocal microscopic images are lacking. Due to the presence of fuzzy regions and weak textures of multi-focus microscope images, the registration of patches is suboptimal. For these problems, this paper formulates a hybrid supervised deep learning model. It can improve the accuracy of registration and fusion. The generalization ability of the model to the actual deformation field enhance by the artificial deformation field. A step of patch movement simulation is employed to blur the multi-focus microscopic images and make synthetic flow, thus emulating distinct fuzzy regions in the two images to be registered, consequently enhancing the model's generalization ability. The experiments demonstrate that our proposed approach is superior to the existing registration algorithms and improves the accuracy of image fusion.

ABSTRACT. Recently, vision transformers have become outstanding segmentation structures for their remarkable global modeling capability. In current transformer-based models for medical image segmentation, convolutional layers are often replaced by transformers, or transformers are added to the deepest layer of the encoder to learn the global context. However, for the extracted multi-scale feature information, most existing methods tend to ignore the multi-scale dependencies, which leads to inadequate feature learning and fails to produce rich feature representations. In this paper, we propose ScaleNet from the perspective of feature interaction at different scales that can alleviate mentioned problems. Specifically, our approach consists of two multi-scale feature interaction modules: the spatial scale interaction (SSI) and the channel scale interaction (CSI). SSI uses a transformer to aggregate patches from different scale features to enhance the feature representations at the spatial scale. CSI uses a 1D convolutional layer and a fully connected layer to perform a global fusion of multi-level features at the channel scale. The combination of CSI and SSI enables ScaleNet to emphasize multi-scale dependencies and effectively resolve complex scale variations. Extensive experiments on multi-organ (Synapse, ACDC) and skin lesion segmentation tasks (ISIC 2018) present the superior performance of ScaleNet compared to previous works.

ABSTRACT. In a foundational expository paper geometric algebra objects are rigorously established paying particular attention to complications made the presence of null vectors.

ABSTRACT. Considering geometric algebra’s status as the unified language of mathematics, physics, and engineering in the 21st century, which coincides with the era of arti-ficial intelligence, the utilization of a Large Language Model (LLM) can greatly benefit the learning and application of geometric algebra. This article aims to ex-plore the fusion of geometric algebra and a large-scale language model from mul-tiple perspectives, including concepts, operators, equations, and computer pro-gramming. This study developed a representative system based on the ggml-ggml-nous-gpt4-vicuna-13b model. Collecting 20,711 papers and books from sources such as arXiv and AACA extensively explores key terms in geometric algebra, such as affine space, homogeneous space, and conformal space, estab-lishing a repository of geometric algebra knowledge. Through the process of data extraction and transformation, mathematical and code languages are converted in-to text suitable for model learning, thus establishing a knowledge base for geo-metric algebra. Furthermore, this system has the capability of iterative refinement, strengthening its understanding and reasoning of geometric algebra knowledge. It has accomplished the textual summarization of research content, methods, inno-vations, and conclusions. It offers interactive question-and-answer sessions about geometric algebra concepts, such as vectors, points, lines, planes, stations, projec-tions, and operations, including inner and outer products. Additionally, it facili-tates the development of tailored learning plans for students from diverse fields to acquire knowledge of geometric algebra in their respective domains.

ABSTRACT. We describe an example of using a Geometric Algebra algorithm to compute motion in a game physics engine, we optimise the Geometric Algebra algorithm using GAALOP and utilise RISC-V Vector Extension (RVV) operations to perform computations on vectors, we combine this with vectors used to represent a number of Fourier series to model x, y and z components of gravity, wind and surface friction.

ABSTRACT. Based on the quaternion domain Fourier transform (QDFT) of 2016 and the quadratic-phase Fourier transform of 2018, we introduce the quadratic-phase quaternion domain Fourier transform (QPQDFT) and study some of its properties, like its representation in terms of the QDFT, linearity, Riemann-Lebesgue lemma, shift and modulation, scaling, inversion, Parseval type identity, Plancherel theorem, directional uncertainty principle, and the (direction-independent) uncertainty principle. The generalization thus achieved includes the special cases of QDFT, a quaternion domain (QD) fractional Fourier transform, and a QD linear canonical transform.

ABSTRACT. In this work, an improved end-to-end U-Net structure, a hierarchical multi-scale interconnection network (HMINet), is proposed to make full use of the information contained in different feature maps in encoders and decoders to improve the accuracy of medical image segmentation. The network consists of two main components: a multi-scale fusion unit (MSF) and a multi-head feature enhancement unit (MFE). In the encoder part, the multi-scale fusion unit is used to fuse the information between the feature maps of different scales. By using convolution at different levels, a wider range of context information can be captured and fused into a more comprehensive representation of features. This helps to reduce information loss and improve the accuracy of medical image segmentation tasks. In the decoder part, multiple feature enhancement units can fully pay attention to the coordinates and channel information between feature maps, and then splice the encoded feature maps step by step to maximize the use of information from different feature maps. These feature maps are joined by a well-designed skip connection mechanism to retain more feature information and minimize information loss. The proposed method is tested on four public medical datasets and compared with other classical image segmentation models. The results show that HMINet can significantly improve the accuracy of medical image segmentation tasks and exceed the performance of other models in most cases.

ABSTRACT. Cough is a common symptom of respiratory disease, which produces a specific sound. Cough detection has great significance to prevent, assess, and control epidemics. This paper proposes CASCO (Cough Analysis System using Short-Time Fourier Transform (STFT) and Convolutional Neural Networks (CNN) in the WeChat mini Program), a cough detection system capable of quantifying the number of coughs through an audio division algorithm. This system combines STFT with CNN, achieving accuracy, precision, recall, and F1-score with 97.0\%, 95.6\%, 98.7\%, and 0.97 respectively in cough detection. The model is embedded into the WeChat mini program to make it feasible to apply cough detection on smartphones and realize large-scale and contactless cough screening. Future research can combine audio and video signals to further improve the accuracy of large-scale cough screening.

ABSTRACT. Point cloud analysis is a technique that performs analysis and processing of point cloud data. In the medical field, point cloud analysis has been widely used. However, the existing common neighbor aggregation module in point cloud analysis networks can only aggregate some of the neighbor features, which will lead to the omission of valid information and affect the performance of point cloud analysis, which may lead to serious consequences in the medical diagnosis process. In this paper, we improve the ability of point cloud analysis networks to extract complex biological structures by improving the neighbor aggregation module in point cloud analysis. Specifically, we enable the module to efficiently extract more adequate information by softening the max pooling function commonly used in the neighbor aggregation module. In particular, we improve 2.18\% IoU on the IntrA dataset compared to the previous state-of-the-art method, and we also surpass the previous state-of-the-art method on the S3DIS dataset.

ABSTRACT. Glaucoma is one of the leading causes of irreversible blindness worldwide. Numerous studies have shown that a larger vertical Cup-to-Disc Ratio (CDR) is closely associated with the glaucoma diagnosis. CDR is highly useful in the clinical practice and evaluation of glaucoma. However, the determination of CDR varies among clinicians and is highly dependent on the doctor's subjectivity. Existing methods only segment the cup and disc features without considering the nearby vascular information. Based on guidance and criteria from experienced clinicians in diagnosing glaucoma, we incorporate segmented essential vascular information to constrain CDR segmentation. We add key vessel information to the network as the prior knowledge to better guide the model to distinguish the boundary of the optic cup. The effectiveness of incorporating essential vascular information has been demonstrated through experiments conducted on the public dataset REFUGE as well as the home-made dataset. The home-made dataset consists of high-quality CDR images and remade CDR images, corresponding to the diagnosis scenario and the screening scenario in which the patient needs to upload the fundus image by taking photos. The model is deployed on the Wechat mini-program for practical glaucoma diagnostic and screening applications.

ABSTRACT. At present, face 3D reconstruction has broad application prospects in various fields, but the research on it is still in the development stage. In this paper, we hope to achieve better face 3D reconstruction quality by combining multi-view training framework with face parametric model FLAME, propose a multi-view training and testing model MFNet(Multi-view FLAME Network). We build a self-supervised training framework and implement constraints such as multi-view optical flow loss function and face landmark loss, and finally obtain a complete MFNet. We propose innovative implementations of multi-view optical flow loss and the covisible mask. We test our model on AFLW and facescape datasets and also take pictures of our faces to reconstruct 3D faces while simulating actual scenarios as much as possible, which achieves good results. Our work mainly addresses the problem of combining parametric models of faces with multi-view face 3D reconstruction and explores the implementation of a FLAME-based multi-view training and testing framework for contributing to the field of face 3D reconstruction.

ABSTRACT. In this work, we focus on the label layout problem: specifying the positions of overlaid virtual annotations in Virtual/Augmented Reality scenarios. Designing a layout of labels that does not violate domain-specific design requirements, while at the same time satisfying aesthetic and functional principles of good design, can be a daunting task even for skilled visual designers. Presenting the annotations in 3D object space instead of projection space, allows for the preservation of spatial and depth cues. This results in stable layouts in dynamic environments, since the annotations are anchored in 3D space. In this paper we make two major contributions. First, we propose a technique for managing the layout and rendering of annotations in Virtual/Augmented Reality scenarios by manipulating the annotations directly in 3D space. For this, we make use of Artificial Potential Fields and use 3D geometric constraints to adapt them in 3D space. Second, we introduce PartLabeling: an open source platform in the form of a web application that acts as a much-needed generic framework allowing to easily add labeling algorithms and 3D models. This serves as a catalyst for researchers in this field to make their algorithms and implementations publicly available, as well as ensure research reproducibility. The PartLabeling framework relies on a dataset that we generate as a subset of the original PartNet dataset consisting of models suitable for the label management task. The dataset consists of 1,000 3D models with part annotations.

ABSTRACT. Opening design is a major consideration in architectural buildings during early structural layout specification. Decisions regarding the geometric characteristics of windows, skylights, hatches, etc., greatly impact the overall energy efficiency, airflow and appearance of a building, both internally and externally. In this work, we employ a goal-based, illumination-driven approach to opening design using a Bayesian Optimization approach, based on Gaussian Processes. A method is proposed that allows a designer to easily set lighting intentions along with qualitative and quantitative characteristics of desired openings. All parameters are optimized within a cost minimization framework to calculate geometrically feasible, architecturally admissible and aesthetically pleasing openings of any desired shape, while respecting the designer's lighting constraints.

ABSTRACT. The demand for interactive photorealistic 3D environments has increased in recent years and in various fields such as architecture, engineering and entertainment. Nevertheless, achieving a balance between quality and performance for high-performance 3D applications and Virtual Reality (VR) remains a challenge. This paper addresses this issue by revisiting and extending view interpolation for image-based rendering, enabling the exploration of spacious open environments in 3D and VR. Therefore, we introduce multi-morphing, a novel rendering method based on a spatial data structure of 2D image patches, called the image graph. With this approach, novel views can be rendered with up to six degrees of freedom using only a sparse set of views. The rendering process does not require 3D reconstruction of geometry, nor per-pixel depth information: All relevant data for output is extracted from local morphing cells of the image graph. Detection of parallax image regions during preprocessing reduces rendering artifacts by extrapolating image patches from adjacent cells in real-time. Additionally, a GPU-based solution to resolve exposure inconsistencies within a dataset is presented, enabling seamless transitions of brightness when moving between areas with varying light intensities. Experiments on multiple real-world and synthetic scenes demonstrate that the presented method achieves high ``VR-compatible'' frame rates, even on mid-range and legacy hardware, respectively.

ABSTRACT. One of the aspects often missing from real-time character animation is change in color of the skin due to emotions (e.g., reddening due to embarrassment) or exerted pressure (e.g., whitening due to pressure on the skin, or stretching of skin during motion). These effects, while subtle, are important for creating truly convincing virtual characters. These changes in skin color are due to a change in blood volume. Hence, modeling skin color in animations can be done using a vascular network that models the subcutaneous blood vessels which then determines the resulting skin color. This paper presents a method that allows real-time, realistic modeling of skin color due to blood volume change, providing a next step towards realistic virtual characters. Our method includes a procedure to dynamically generate blood vessel networks based on medically accurate models, and an algorithm to render these as a set of textures accepted by modern (real-time) rendering engines. Moreover, these textures can be altered in real-time. This allows for the simulation of change in skin color due to blood volume changes, all while maintaining biological plausibility.

ABSTRACT. Digital watermarking technology plays an essential role in the work of anti-counterfeiting and traceability. However, image watermarking algorithms are weak against hybrid attacks, especially geometric attacks, such as cropping attacks, rotation attacks, etc. We propose a robust blind image watermarking algorithm that combines stable interest points and deep learning networks to improve the robustness of the watermarking algorithm further. First, to extract more sparse and stable interest points, we use the Superpoint algorithm for generation and design two steps to perform the screening procedure. We first keep the points with the highest possibility in a given region to ensure the sparsity of the points and then filter the robust interest points by hybrid attacks to ensure high stability. The message is embedded in sub-blocks centered on stable interest points using a deep learning-based framework. Different kinds of attacks and simulated noise are added to the adversarial training to guarantee the robustness of embedded blocks. We use the ConvNext network for watermark extraction and determine the division threshold based on the decoded values of the unembedded sub-blocks. Through extensive experimental results, we demonstrate that our proposed algorithm can improve the accuracy of the network in extracting information while ensuring high invisibility between the embedded image and the original cover image. Comparison with previous SOTA work reveals that our algorithm can achieve better visual and numerical results on hybrid and geometric attacks.

ABSTRACT. External knowledge representations play an essential role in knowledge-based visual question and answering to better understand complex scenarios in the open world. Recent entity-relationship embedding approaches are deficient in some of representing complex relations, resulting in a lack of topic-related knowledge but the redundancy of topic-irrelevant information. To this end, we propose MKEAH to represent Multimodal Knowledge Extraction and Accumulation on Hyperplanes. To ensure that the length of the feature vectors projected to the hyperplane compares equally and to filter out enough topic-irrelevant information, two losses are proposed to learn the triplet representations from the complementary views: range loss and orthogonal loss. In order to interpret the capability of extracting topic-related knowledge, we present Topic Similarity (TS) between topic and entity-relation. Experimental results demonstrate the effectiveness of hyperplane embedding for knowledge representation in knowledge-based visual question answering. Our model outperforms the state-of-the-art methods by 2.12% and 3.24%, respectively, on two challenging knowledge-required datasets: OK-VQA and KRVQA. The obvious advantages of our model on TS shows that using hyperplane embedding to represent multimodal knowledge can improve the ability of the model to extract topic-related knowledge.

ABSTRACT. We present a novel method for dense shape correspondence that combines spatial information transformed by neural networks with their projection on spectral maps. This approach builds on the proven efficiency of the original "functional mapping" method, but addresses a major challenge faced by all such methods: a "Chicken or the egg" scenario, where poor spatial features lead to inadequate spectral alignment and vice versa during training. This often results in slow convergence, high computational costs, and failure to learn, particularly when working with small datasets.

To overcome this challenge, we propose a new method that selectively samples only those points with high confidence in their alignment. These points then participate in the alignment and spectral loss terms, boosting training and accelerating convergence by a factor of 5. To ensure full, unsupervised learning, we use the \textit{Gromov Hausdorff distance metric} to choose the best set of confident points with the maximal alignment score.

Our approach offers significant advantages over current methods, including faster convergence, improved accuracy, and reduced computational costs. We demonstrate the effectiveness of our approach on several benchmark datasets and report superior results compared to spectral and spatial based methods. Overall, our method provides a promising new approach to dense shape correspondence that addresses key challenges in the field.

ABSTRACT. With the development of information technology, network traffic logs mixed with various kinds of cyber-attacks have grown explosively. Traditional intrusion detection systems (IDS) have limited ability to discover new inconstant patterns and identify malicious traffic traces in real-time. It is urgent to implement more effective intrusion detection technologies to protect computer security. In this paper, we design a hybrid IDS, combining our incremental learning model (KAN-SOINN) and active learning, to learn new log patterns and detect various network anomalies in real-time. The experimental results on the NSLKDD dataset show that the KAN-SOINN can be improved continuously and detect malicious logs more effectively. Meanwhile, the comparative experiments prove that using a hybrid query strategy in active learning can improve the model learning efficiency.

ABSTRACT. The versors of the homogeneous-point-based version R(d,0,1) (dubbed HGA) are related to the basic operations in geometric paraxial optics. Odd versors represent reflections in spherical mirrors (be they concave or convex) and even versors implement the lens equation. We extend the results to arbitrarily positioned optical elements by embedding R(d,0,1) into CGA R(d+1,1). The total transformation through a paraxial optical system now consists of successive teleportation (by CGA dot and outer product) to the next optical center, and then applying its local versors.

The result is a straightforward sequence of operations that implements a total system of arbitrarily placed paraxial lenses and mirrors in 3D (or any dimension), parameterized by the CGA tangent vectors (from each optical center to the corresponding focal point) for each optical component. This can be used to compile the homogeneous transformation matrices of a total paraxial system in terms of those geometric parameters.

ABSTRACT. We study the geometry and kinematics of surveillance of static objects with moving cameras both in 2D and 3D space via geometric algebras. The former case is easily handled using polar representation of complex numbers, while for the latter we resort on dual projective quaternions and Pl\"{u}cker line geometry. Hence, a brief preliminary section is provided, after which we approach the task naively, searching for exact solutions in the case of pre-determined trajectory. The infinitesimal setting is of particular interest for practical reasons, so we pay attention to it as well. Finally, some ideas for generalizations are given, along with curious examples.

ABSTRACT. Octonions are 8-dimensional hypercomplex numbers which form the largest normed division algebra (in the wider sense of admitting non-associativity) over the real numbers. Motivated by applications in theoretical physics, continuous octonionic analysis has become a very active area of research during the recent years. Looking at possible practical applications, it is beneficial to work directly with discrete structures rather than approximating continuous objects. Therefore, in previous papers, we have proposed some ideas towards the development of a discrete octonionic analysis. It is well known that there are several possibilities to discretise the continuous setting. The Weyl calculus approach, which is typically used in the associative discrete Clifford analysis setting, has not been studied in the octonionc setting yet. The aim of this paper is to close this gap. We succeed in presenting the discretisation of octonionic analysis based on the Weyl calculus.

ABSTRACT. Clifford algebras provide the natural generalizations of complex, dual numbers and quaternions into non-commutative Clifford numbers. The paper demonstrates an algorithm for the computation of inverses of such numbers in a non-degenerate Clifford algebra of an arbitrary dimension. The algorithm is a variation of the Faddeev-LeVerrier-Souriau algorithm and is implemented in the open-source Computer Algebra System Maxima. Symbolic and numerical examples in different Clifford algebras are presented.

ABSTRACT. This paper is a brief note on the natural implementation of singular value decomposition (SVD) and polar decomposition of an arbitrary multivector in nondegenerate real (Clifford) geometric algebras of arbitrary dimension and signature. We naturally define these and other related structures (operation of Hermitian conjugation, Euclidean space, and Lie groups) in geometric algebras. The results can be used in various applications of geometric algebras in computer graphics, computer vision, data analysis, computer science, engineering, physics, big data, machine learning, etc.

ABSTRACT. Adversarial attacks on visual object tracking have attracted increasing attention in order to evaluate and improve the robustness and security of object tracking models. Most adversarial attack methods for object tracking are based on fully supervised attacks, i.e. labels are required. It is impractical to apply all labeled video object tracking to an attack. To this end, this paper proposes an unsupervised attack method against the visual object tracking model, which uses the cycle consistency principle of the object tracking model to make the forward tracking and backward tracking of the object tracking model as inconsistent as possible, resulting in effective countermeasures. sample. In addition, this paper proposes a contextual attack method, which utilizes the information of the attack object region and its surrounding context regions to simultaneously attack the object region and its surrounding context regions to reduce its response score to the attack. The proposed attack method is evaluated on different types of deep learning-based object trackers, and the experimental results on multiple benchmarks show that the proposed method has competitive attack results.

ABSTRACT. In this paper, we present a novel hybrid occlusion culling method for large scale scenes. The basic idea is to use an iterative hierarchical Z-buffer occlusion culling algorithm to execute a coarse-grained culling via compute shader, followed by a fine-grained culling via rasterization. We also propose a forward warping method to generate a low resolution approximated depth map to accelerate the culling process. Our solution requires only one indirect multidraw command and runs on GPU totally without any CPU read-back operations. Our experimental results indicate that our algorithm outperforms the existing solutions both in performance and culling rate.

ABSTRACT. Recovering 3D hand mesh from a monocular RGB image has a wide range of application scenarios such as VR/AR. The parametric hand model provides a good geometric prior to the shape of hand, and is commonly used to recover the 3D hand mesh. However, the rotation parameters of hand model are not easy to learn, which influences the accuracy of model-based methods. To address this problem, we take advantage of the inverse kinematic chains of hand to derive an analytical method, which can convert hand joint locations into rotation parameters. By integrating such analytical method into the neural network, we propose an end-to-end learnable model named IKHand to recover the 3D hand mesh. IKHand comprises detection module and mesh generation module. Detection module predicts the 3D hand keypoints while mesh generation module takes these keypoints to generate the 3D hand mesh. Experimental results show that our proposed method can generate impressive and robust 3D hand meshes under several challenging conditions, and can achieve superior accuracy in model-based methods.

ABSTRACT. We present Interactive Neural Video Editing (INVE), a real-time video editing solution, which can assist the video editing process by consistently propagating sparse frame edits to the entire video clip.

Our method is inspired by the recent work Layered Neural Atlas (LNA). LNA, however, suffers from two major drawbacks: (1) the method is too slow for interactive editing, and (2) it offers insufficient support for some editing use cases, including direct frame editing and rigid texture tracking.

To address these challenges we leverage and adopt highly efficient network architectures, powered by hash-grids encoding, to substantially improve processing speed. In addition, we learn bi-directional functions between image-atlas and introduce vectorized editing, which collectively enables a much greater variety of edits in both the atlas and the frames directly.

Compared to LNA, INVE reduces the learning and inferencing time by a factor of 5, and supports various video editing operations that LNA cannot. We showcase the superiority of INVE over LNA in interactive video editing through a comprehensive quantitative and qualitative analysis, highlighting its numerous advantages and improved performance.

ABSTRACT. Dynamic point clouds are widely used for 3D data representation in various applications such as immersive and mixed reality, robotics and autonomous driving. However, their irregularity and large scale make efficient compression and transmission a challenge. Existing methods require high bitrates to encode point clouds since temporal correlation is not well considered. This paper proposes an end-to-end dynamic point cloud compression network that operates in latent space, resulting in more accurate motion estimation and more effective motion compensation. Specifically, a multi-scale motion estimation network is introduced to obtain accurate motion vectors. Motion information computed at a coarser level is upsampled and warped to the finer level based on cost volume analysis for motion compensation. Additionally, a residual compression network is designed to mitigate the effects of noise and inaccurate predictions by encoding latent residuals, resulting in smaller conditional entropy and better results. The proposed method achieves an average 12.09\% BD-Rate gain over state-of-the-art Deep Dynamic Point Cloud Compression (D-DPCC) in experimental results. The novelty of our method lies in the use of latent space for all major operations and the introduction of the multi-scale motion estimation network to improve motion estimation accuracy.

ABSTRACT. Due to the high data dimensionality and the complexity of the problem, existing 3D mesh reconstruction models often require significant computational resources to achieve satisfactory results. While lightweight model based on knowledge distillation has been explored in many fields such as image classification, training a lightweight 3D mesh reconstruction model remains a challenging task. In this paper, we propose a method to learn a lightweight 3D mesh reconstruction network using knowledge distillation. Specifically, we introduce a novel approach called multi-stage and progressive knowledge distillation, which effectively enhances the guidance from the teacher network to the student network, thereby improving reconstruction performance. Additionally, we propose a projection-based spatial feature unpooling method to provide more accurate spatial features for the increased spatial points. Experimental results show that our lightweight 3D mesh reconstruction network has comparable performance to existing complex models while greatly reducing the number of parameters. Specifically, our method achieves its 98.97% accuracy while reducing the number of graph neural network parameters to 71.42% of the teacher network.

ABSTRACT. Learning-based fluid simulation has proliferated due to its ability to replicate the dynamics with substantial computational savings over traditional numerical solvers. To this end, Graph Neural Networks (GNNs) is a suitable tool to capture fluid dynamics through local particle interactions. Nonetheless, it remains challenging to model the long-range behaviors. To tackle this, this paper models the fluid flow via graphs at different scales in succinct consideration of scalability and physical constraints. We propose a novel multi-scale GNN for physics-informed fluid simulation (MSG) by introducing a non-parametric sampling and aggregation method to combine features from graphs with different resolutions. Our design reduces the size of the learnable model and accelerates the model inference time. In addition, zero velocity divergence is explicitly incorporated as a physical constraint through the training loss function. Finally, a fusion mechanism of consecutive predictions is incorporated to alleviate the inductive bias caused by the Markovian assumption. Extensive experiments corroborate the merits over leading particle-based neural network models in terms of both one-step accuracy (+6.7%) and long trajectory prediction (+16.9%). This comes with a run-time reduction by 2.8% over the best baseline method.

ABSTRACT. Deep learning has enabled image style transfer to make great strides forward. However, unlike many other styles, transferring the watercolor style to portraits is significantly challenging in image synthesis and style transfer. Pixel-correlation-based methods do not produce satisfactory watercolors. This is because portrait watercolors exhibit the sophisticated fusion of various painting techniques in local areas, which poses a problem for convolutional neural networks to accurately handle fine-grained features. Moreover, common but problematic multi-scale challenge greatly impedes the performance of existing style transfer methods with fixed receptive fields. Although it is possible to develop an image processing pipeline mimicking various watercolor effects, such algorithms are slow and fragile, especially for inputs of different scales. As a remedy, this paper proposes WCGAN, a generative adversarial network (GAN) architecture dedicated to watercolorization of portraits. Specifically, a novel localized style loss suitable for watercolorization is proposed to deal with local details. To handle portraits of different scales and improve robustness, a novel discriminator architecture with three parallel branches of varying receptive fields is introduced. In addition, the application of WCGAN is expanded to video style transfer where a novel kind of video training data based on random crops is developed to efficiently capture temporal consistency. Extensive experimental results from qualitative and quantitative analysis demonstrate that WCGAN generates state-of-the-art, high quality watercolors from portraits.

ABSTRACT. Graph neural networks have been used in the learning-based simulation of cloth and have received a lot of attention recently. Some learning-based graph networks lack information on cloth structure, the generated results are not plausible, and penetration is unavoidable. We present the PGN-Cloth model, which uses mesh to represent the state of the cloth system and computes the dynamics through graph neural networks. Our contributions include: (1) PGN-Cloth combine the ideas of physics-based deep learning and mass-spring models, adding EdgeLoss, CosLoss, and BendLoss to generate more plausible results; (2) an additional penetration loss is added to optimize the penetration problem that exists in the current state-of-the-art method; (3) our approach can significantly improve the training speed of the network with only a small increase in computational, and has better stability than traditional learning methods. The experimental results are better than state-of-the-art in several indicators, and there is no significant penetration in the generated results.

ABSTRACT. With the advancement and popularization of information technology, open-pit mines are also rapidly developing towards integration and digitization. Three-dimensional reconstruction technology has been successfully applied to surface scene modeling and geological structure reconstruction in open-pit mines. However, there is no open-pit mine integration model that can fuse above-ground scene information with underground geological information. In this paper, we propose an integrated modeling method for open-pit mines, which integrates the real scene on the ground with the underground geological model. Based on oblique photography technology, the real scene model above ground was established. Based on the surface suture method proposed in this paper, a three-dimensional underground geological model is constructed, and the above-ground and underground models are registered and fused to establish an integrated model of open-pit mine. The auxiliary planning system of open-pit mine is designed and implemented, and the functions of mining planning and output calculation are carried out based on the integrated model of open-pit mine, which assists users in mining planning and operation management, improving production efficiency and management level.

ABSTRACT. Cross-modal retrieval has attracted widespread attention in many cross-media similarity search applications, especially image-text retrieval in the fields of computer vision and natural language processing. Recently, visual and semantic embedding (VSE) learning has shown promising improvements on image-text retrieval tasks. Most existing VSE models employ two unrelated encoders to extract features, then use complex methods to contextualize and aggregate those features into holistic embeddings. Despite recent advances, existing approaches still suffer from two limitations: 1) without considering intermediate interaction and adequate alignment between different modalities, these models cannot guarantee the discriminative ability of representations; 2) existing feature aggregators are susceptible to certain noisy regions, which may lead to unreasonable pooling coefficients and affect the quality of the final aggregated features. To address these challenges, we propose a novel cross-modal retrieval model containing a well-designed alignment module and a novel multimodal fusion encoder, which aims to learn adequate alignment and interaction on aggregated features for effectively bridging the modality gap. Experiments on Microsoft COCO and Flickr30k datasets demonstrates the superiority of our model over the state-of-the-art methods.

ABSTRACT. Deep reinforcement learning (DRL) has demonstrated superior performance in playing video games and chess. However, applying DRL to complicated tasks like robot control is still challenging. Trial-and-error learning introduces additional uncertainty, while statistics such as rewards obscure detailed information, making retrospective analysis difficult. In this work, we propose RobotExplorer, a visual analytics system to help domain experts extract keyframes for diagnosis, understand the semantics of continuous actions, and summarize the impact of features on decision-making. Through case studies and user interviews conducted with deep learning experts, we demonstrate the effectiveness of RobotExplorer.

ABSTRACT. Intelligent garment, as an emerging intelligent wearable device, has been widely applied in fields such as sports training and medical rehabilitation. However, the current research in the field of smart wearables mainly focuses on sensor functionality and quantity, while neglecting user experience and interaction aspects. To address this issue, this study proposes a real-time 3D interactive system based on intelligent garment. The system utilizes lightweight sensor modules to collect human motion data and introduces a dual-stream fusion network based on pulse neural units for classifying and recognizing human movements, achieving real-time interaction between users and sensors. Additionally, the system incorporates 3D human visualization functionality, which visualizes the sensor data and recognized human actions as 3D models in real-time, providing accurate and comprehensive visual feedback to help users better understand and analyze the details and features of human motion. The system holds great potential for applications in motion detection, medical monitoring, virtual reality, and other fields. Accurate classification of human actions contributes to the development of personalized training plans and injury prevention strategies. In summary, this research has significant implications in the fields of intelligent garment, human motion monitoring, and digital twin visualization. The development of this system will drive advancements in wearable technology and enhance our understanding of human motion.

ABSTRACT. A great deal of research has been conducted in the area of audio-driven virtual character gestures and facial animation with some degree of success. However, there are few methods for generating full-body animations, and the portability of virtual character gestures and facial animations is not given enough attention. Therefore, we propose a deep learning-based Audio2AB method, which generates gesture animations and ARKit's 52 facial expression parameter blendshape weights based on audio, audio corresponding text, emotion labels and semantic relevance labels, to generate parametric data for full-body animations. This parameterization method can be used to drive full-body animations of virtual characters and improve the portability. In the experiment, we first downsample the gesture and facial datas to achieve the same temporal resolution for input datas and output gesture and face datas. So that the synthesized gestures and faces animation can be processed frame by frame through a sequence model. Secondly, our network is trained based on audio and other information as the dataset. Audio2AB network encodes audio, audio corresponding text, emotion labels, and semantic relevance labels, then fuses the text, emotion labels and semantic relevance labels into the audio to obtain better audio features. Then, we establish links between body, gesture, and facial decoders, and generate corresponding animation sequences through our proposed GAN-GF loss function. Finally, by using audio, audio corresponding text, emotional and semantic relevance labels as input, the trained Audio2AB network can generate gesture animation data and containing blendshape weights' facial data. Therefore, different 3D virtual character animations can be driven through parameterization. The experimental results show that our proposed method can generate significant gesture and facial animations.

ABSTRACT. Despite the recent progresses on 3D point cloud process with deep CNNs, the inability to extract local features remains a challenging problem. In addition, the current method only considers the spatial domain in the feature extraction process, so in this paper, we propose a graph convolutional network—S2ANet, which combines spectral and spatial features to point cloud processing. First, we calculate the local frequency of the point cloud in the spectral domain. Then we use local frequency to group points, and we provide a spectral aggregation convolution module to extract the features of the points grouped by the local frequency. At the same time, we also extract the local features in the spatial domain to supplement the final features. S2ANet was benchmarked on several point cloud analysis tasks where we achieved the state-of-the-art classification accuracy of 93.8\%, 88.0\%, and 83.1\% on ModelNet40, ShapeNetCore, and ScanObjectNN datasets, respectively. For indoor scene segmentation, training and testing are performed on the S3DIS dataset, and the mIoU is 62.4\%. Our code is released on https://github.com/sunxr99/S2ANet.

ABSTRACT. Recurrent recovery is a common method for video super-resolution (VSR) that models the correlation between frames via hidden states. However, the application of this structure in real-world scenarios can lead to unsatisfactory artifacts. We found that in real-world VSR training, the use of unknown and complex degradation can better simulate the degradation process in the real world. Based on this, we propose the RealFuVSR model, which simulates real-world degradation and mitigates artifacts caused by the VSR. Specifically, we propose a multiscale feature extraction module (MSF) module that extracts and fuses features from multiple scales, thereby facilitating the elimination of hidden state artifacts. To improve the accuracy of the hidden state alignment information, RealFuVSR uses an advanced optical flow-guided deformable convolution. Moreover, a cascaded residual upsampling module was used to eliminate noise caused by the upsampling process. The experiment demonstrates that RealFuVSR model can not only recover high-quality videos but also outperforms the state-of-the-art RealBasicVSR and RealESRGAN models.

ABSTRACT. For static scenes with multiple depth layers, the existing defocused image deblurring methods have the problems of edge ringing artifacts or insufficient deblurring degree due to inaccurate estimation of blur amount, In addition, the prior knowledge in non blind deconvolution is not strong, which leads to image detail recovery challenge. To this end, this paper proposes a blur map estimation method for defocused images based on the gradient difference of the boundary neighborhood, which uses the gradient difference of the boundary neighborhood to accurately obtain the amount of blurring, thus preventing boundary ringing artifacts. Then, the obtained blur map is used for blur detection to determine whether the image needs to be deblurred, thereby improving the efficiency of deblurring without manual intervention and judgment. Finally, a non blind deconvolution algorithm is designed to achieve image deblurring based on the blur amount selection strategy and sparse prior. Experimental results show that our method improves PSNR and SSIM by an average of 4.6% and 7.3%, respectively, compared to existing methods.

ABSTRACT. The hands and face in the sign language video are the most important parts for expressing sign language morphemes. However, we find that existing Continuous Sign Language Recognition (CSLR) works lack the mining of hand and face information in CNN backbones or apply expensive and time-consuming external extractors to explore this information. Besides, the signs have different length, while previous CSLR methods usually use a fixed-length window to segment the video to capture sequential features, which disturbs the perception of complete signs. In this paper, we propose a Multi-Scale Context-Aware network (MSCA-Net) to solve the problems above. Our MSCA-Net contains two main modules: 1) Multi-Scale Motion Attention (MSMA), using the differences among frames to perceive information of hands and face in multiple spatial scales, replacing the heavy feature extractors; 2) Multi-Scale Temporal Modeling (MSTM), exploring crucial temporal information in the sign language video from different temporal scales. We conduct extensive experiments on three widely-used sign language datasets, i.e., RWTH-PHOENIX-Weather-2014, RWTH-PHOENIX-Weather-2014T and CSL-Daily. The proposed MSCA-Net achieves state-of-the-art performance, which demonstrates the effectiveness of our approach.