ABSTRACT. Multipath-based simultaneous localization and mapping (SLAM) is a promising approach to obtain position information of transmitters and receivers as well as information regarding the propagation environments in future mobile communication systems. Usually, specular reflections of the radio signals occurring at flat surfaces are modeled by virtual anchors (VAs) that are mirror images of the physical anchors (PAs). In existing methods for multipath-based SLAM, each VA is assumed to generate only a single measurement. However, due to imperfections of the measurement equipment, such as non-calibrated antennas or model-mismatch due to roughness of the reflective surfaces, there are potentially multiple multipath components (MPCs) that are associated to one single VA. In this paper, we introduce a Bayesian particle-based sum-product algorithm (SPA) for multipath-based SLAM that can cope with multiple-measurements being associated to a single VA. Furthermore, we introduce a novel statistical measurement model that is strongly related to the radio signal. It introduces additional dispersion parameters into the likelihood function to capture additional MPC-related measurements. We demonstrate that the proposed SLAM method can robustly fuse multiple measurements per VA based on numerical simulations.

ABSTRACT. Tracking and surveillance of closely spaced targets requires advanced data association methods that are capable of handing unresolved measurements. Such phenomena usually occur due to limited sensor resolution and lead to track loss when not taken into account. In this paper, we propose a scalable method for tracking multiple targets when any two of them may be unresolved, in addition to the standard origin uncertainty. A loopy belief propagation algorithm is designed to efficiently approximate the marginal association probabilities and construct posterior target densities given a set of potentially unresolved measurements. The resulting algorithm scales quadratically in the number of targets and linearly with the number of measurements.

ABSTRACT. This paper explores evaluation of association marginals in multiple hypothesis tracking. The work builds upon recent results where loop belief propagation (LBP) has been used in single-hypothesis cases. There are two contributions in the paper. The first is a novel factor graph representation of the joint multihypothesis association posterior. The second contribution is two algorithms that both use LBP to evaluate association marginals. The first method uses total probability in conjunction with hypothesis-conditioned LBP, and is called PHD-LBP. The second method is an LBP algorithm running directly on the full multihypothesis association graph with novel, specialized message definitions that are derived in this paper and efficient to compute and store in memory, and is called MH-LBP. Results show that both algorithms perform well with high correlation with the exact marginals for the majority of the cases.

ABSTRACT. The majority of evidence fusion models presented in the literature is based on optimistic assumptions about reliability of the models producing beliefs and assumes that they are equally reliable. At the same time, the modelling of beliefs used in combination may have some limitations and may result in different reliability, which may decrease the performance of the combination. One way to confront this problem is to consider a discount rule utilizing reliability coefficients. One of the problems of using discounting is the way of modeling reliability coefficients. This paper proposes modeling reliability coefficients by considering a new effective measure of belief uncertainty. The new reliability coefficients are introduced in multimodel deep learning. A case study results demonstrate the feasibility of representing reliability by the belief uncertainty measure considered.

ABSTRACT. Developing machine capabilities to automatically ensure that a human teammate (or operator) maintains an adequate level of Situation Awareness (SA) and timely interventions (when necessary) to improve SA can not only improve mission performance, but it will also assist in efficiently reducing human error. In this paper, we extend our work on operator SA assessment presented in FUSION 2022 paper, where a new automated SA assessment method is devised by leveraging state-of-the-art Computer Vision (CV) techniques on object detection, segmentation and multi-target tracking (or multi-object tracking) for identifying relevant Situation Elements (SEs) in complex situations and extracting information necessary for calculating Probability of Attending of SEs. In particular, our new SA assessment method first leverages YOLOv7 and StrongSort for automated detection and tracking of SEs identified from video streams. Then, instance segmentation is employed to find the relevant pixel information of SEs (i.e., location, size, color) for computing the specific Visual Attention Allocation (VAA) values of probability of attending (P(SE)) and attention allocation proportion (f_i) via the use of the SEEV (i.e., Salience, Effort, Expectancy, and Value) SA model. Finally, operator SA is derived by analyzing the operator's visual attention via the use of eye-tracking to check if the operator is paying attention to relevant SEs. This paper concludes with a discussion of our current progress, technical issues and potential future extensions.

ABSTRACT. UAVs are a cutting-edge technology whose use is currently highly restricted due to their potentially dangerous characteristics. This is mainly due to the lack of legislation adopting them and allowing a safe control of these vehicles. Unmanned Air System Traffic Management (UTM) initiatives seek to regularise their use by means of validation and monitoring techniques for the trajectories of these aircraft, both before flight and in real time. For this purpose, in the UTM framework, drones will be collaborative, which implies the need to validate large quantities of trajectories that must be processed by a verification entity in order to avoid potential risks. This approach is similar to that present in other vehicles such as ships with AIS technology or aircraft with ADSB. Currently there are no UAV trajectory datasets that allow research in this field, so in this paper a dataset composed of the position and kinematics of the drones over time has been designed. By means of this dataset it is possible to perform machine learning problems that help the verification entity to fulfil its functionalities. In this work we propose a first approach to extract useful information by classifying the type of drone based on its movement dynamics. This information would be useful in the identification of the validity of the proposed trajectory for the airframe indicated by the user.

ABSTRACT. This paper deals with the resilient time provision based on an ensemble of clocks. In particular, the stress is laid on the combination of clock outputs and detecting possible faults. Two classes of fault detection methods, namely model-based and AI/ML-based, are discussed and analysed. In addition, a novel fault detection technique based on the solution separation principle is proposed and tailored for the area of the time provision. Selected fault detection methods are numerically evaluated using a model of an atomic clock ensemble.

ABSTRACT. The partial-update filter is a Kalman filter modification that can accommodate higher nonlinearities and uncertainties than a nominal and Schmidt Kalman filter. This robustness enhancement of the partial-update filter is attributed to its capability to limit the impact of incorrect updates by applying static percentages of the nominal Kalman update to user-selected states at any time step. To further extend the partial-update capabilities and applicability, this paper presents two methods for dynamically selecting the partial-update percentages based on nonlinearity metrics of the process and measurement model. By equipping the partial-update filter with a dynamic update percentage, the filter can effectively leverage situations where higher updates can be applied, and lower updates are deemed suitable, leading to filter statistical consistency and accuracy increase with respect to the nominal and static partial-update filters. Simulation results show that the proposed nonlinearity-aware partial-update methods achieve results near a manually tuned partial-update.

ABSTRACT. Active fault diagnosis (AFD) of stochastic large-scale systems in multiple model framework involves two stages: offline and online. In the offline stage, an excitation input generator is designed based on a Bellman function. In the online stage, the generator is utilized together with an estimator of the model indices. A similar estimator is used in the offline stage for the Bellman function calculation using the value iteration technique. However, due to the high dimensions of information states of the associated perfect state information problem, the estimator in the offline stage must involve approximations. The paper provides the relations for the estimate calculation using the Bayesian recursive relations and studies the effects of such approximations on the AFD decisions. In particular, the quality of the model index estimates is analyzed using a power network model.

ABSTRACT. The face morphing process will combine two or more facial images to generate a single morphed facial image demonstrating Face Recognition Systems (FRS) vulnerability. The attack potential of the morphing image directly depends on the perceptual image quality, and when generated with no visible artefacts, it can deceive both human observers and automatic FRS. The current softwares for face morphing generates a morphing image with ghosting artefacts, especially in the eye region, nose and mouth area, which may serve as a potential cue to detect morphing attacks. Hence in this work, we introduce a new dataset comprising 10710 facial images before and after manual post-processing to reduce the visual artefacts and to generate high-quality attacks. Further, we propose a novel single image-based Morph Attack Detection (S-MAD) technique based on the ensemble of features and classifiers using the scale-space domain. The novel concept in the proposed method is the multi-level fusion that combines the comparison scores from different features and classifiers. Extensive experiments are carried out on the newly generated high-quality face images with (i) Morphs before post-processing and (ii) Morphs after post-processing. Further, the experiments are also carried out on two different mediums such as (i) Digital and (ii) Print-scan (or re-digitized) with and without compression. Extensive experimental results are performed to benchmark the detection performance with the existing S-MAD techniques. Obtained results indicate the best performance of the proposed method over existing methods.

ABSTRACT. For the task of change detection (CD) in remote sensing images, deep convolution neural networks (CNNs)-based methods have recently aggregated transformer modules to improve the capability of global feature extraction. However, they suffer degraded CD performance on small changed areas due to the simple single-scale integration of deep CNNs and transformer modules. To address this issue, we propose a hybrid network based on multi-scale CNN-transformer structure, termed MCTNet, where the multi-scale global and local information is exploited to enhance the robustness of the CD performance on changed areas with different sizes. Especially, we design the ConvTrans block to adaptively aggregate global features from transformer modules and local features from CNN layers, which provides abundant global-local features with different scales. Experimental results demonstrate that our MCTNet achieves better detection performance than existing state-of-the-art CD methods.

ABSTRACT. The fusion of multiple images from different modalities is the process of generating a single output image that combines the useful information of all input images. Ideally, the information-rich content of each input image would be preserved, and the cognitive effort required by the user to extract this information should be smaller on the fused image than the one required to examine all images. We propose MobileFuse, an edge computing method targeted at processing large amount of imagery in a bandwidth limited environment using depthwise separable Deep Neural Networks (DNNs). The proposed approach is a hybrid between generative and blending based methods. Our approach can be applied in various fields which require low latency interaction with the user or with an autonomous system. The main challenge in training DNNs for image fusion is the sparsity of data with representative ground truth. Registering images from different sensors is a major challenge in itself, and generating a ground truth from them is another massive one. For this reason, we also propose a multi-focus and multi-lighting framework to generate training dataset using unregistered images. We show that our edge network can perform faster than its state-of-the-art baseline, while improving the fusion quality.

ABSTRACT. Automating sensing processes is of high interest to the control and automation community. Active sensing is focused on solving this task, usually with information based or task driven selection of optimal sensing actions. This paper presents an active sensing formulation that combines task based, in the form of standoff tracking, and information based active sensing by implementing the dual control for exploitation and exploration (DCEE) concept to control a mobile, limited FOV, sensor platform. The DCEE based cost function is integrated into the Monte Carlo tree search (MCTS) framework for non-Myopic decision making. Using the Bernoulli particle filter for single target tracking with bearing-only measurements, the DCEE observer control method is benchmarked against the popular Rényi divergence information metric with two different α values. Whilst the Rényi divergence performs marginally better when considering existence estimation, spatial results clearly demonstrate that our formulation is able to outperform the benchmark algorithm with improved target localisation performance resulting from outmanoeuvring of the target.

ABSTRACT. This paper addresses the problem of a moving target search with a discrete allocation of search effort on disjoint cones of observation. A cone covers several regions with different visibilities. The target moves at each time step from region to region. Target detection is optimized using two objective functions. We adapt Brown’s recursion and the FAB algorithm to this problem and apply them to compute search plans. We measure the quality of our solution with a continuous relaxation of the problem. In a set of numerical experiments we show that our algorithms perform well in terms of plan quality and computation time. Finally, we compare the plans optimizing each objective function.

ABSTRACT. This paper addresses the problem of localizing a stationary RF emitter with a mobile UAV, equipped with a single directional antenna. By rotating around its vertical axis, it determines a bearing towards the emitter. Our interest is in optimally selecting the measurement positions to achieve a fast localization. The majority of such systems described in the literature use greedy planning to select the next measurement position. This work experimentally tests an algorithm that performs a non-myopic planning until the final localization step. The algorithm is based on the policy rollout principle and showed good performance in previous simulative studies. It is adapted to match the needs of a real world setup and evaluated in flight trials. Adaptions include the avoidance of close range measurements to prevent inaccurate measurements at high elevation, and the filtering of poor measurements.

ABSTRACT. Factor graphs are a ubiquitous tool for multi-source inference in robotics and multi-sensor networks. They allow for heterogeneous measurements from many sources to be concurrently represented as factors in the state posterior distribution, so that inference can be conducted via sparse graphical methods. Adding measurements from many sources can supply robustness to state estimation, as seen in distributed pose graph optimization. However, adding excessive measurements to a factor graph can also quickly degrade their performance as more cycles are added to the graph. In both situations, the relevant quality is the redundancy of information. Drawing on recent work in information theory on partial information decomposition (PID), we articulate two potential definitions of redundancy in factor graphs, both within a common axiomatic framework for redundancy in factor graphs. This is the first application of PID to factor graphs, and only one of a few quantitative measures of redundancy.

ABSTRACT. Decentralized decision making methods often require significant communication between agents. A two-stage approach is presented to ensure a convergent decision making methodology with limited lines of communication. First, a scheme is outlined for randomized, directed communication between agents. Guarantees on connectivity and communication times are illustrated. Communication of optimal action trajectories for another agent often leads to changes in locally optimal actions. With the assumption that such breakpoints occur as a power law in search time, asymptotic convergence toward a globally optimal consensus is demonstrated.

ABSTRACT. This work introduces the PNT Chain, a fully distributed filtering solution to collaborative positioning with ranging radios in GNSS-degraded and denied environments. The PNT Chain uses an Undirected Acyclic Bayes Graph (UABG) to factor a Falman filter on the poses of an entire swarm into tractable cliques. The cliques are dsitributed across the physical nodes, which in combination with the use of equivalent propagation and equivalent measurement messages, keeps network traffic to an aboslute minimum. The distributed PNT Chain algorithm is compared with an idealized (and impractical) centralized filter, and a legacy relative navigation algorithm in two scenarios where only two UAVS in a swarm are receiving GNSS signals. In both cases, the distributed PNT Chain achieves a positional accuracy comparable with the completely centralized filter, and an order of magnitude better than the legacy approach, while only requiring 22% the computation cost of the centralized filter.

ABSTRACT. With social platforms being used by an increasing number of users, the Internet became a perfect place for extremist ideas and opinions to be created and propagated. Detection of extremist content can be an asset for security agencies but it comes with technical challenges and requires semi-automatic approaches, as reading the sheer amount of data released online is an impossible task for analysts. This paper investigates the use of learning models to detect extremist contents in French corpora and focuses on right-wing extremist detection. Several learning models have been developed including unsupervised approaches and neural ones. The models were applied to a data set gleamed online. Experiments show that data representation and parameters of models may affect the overall performance and extremist content can be accurately detected when parameters and thresholds are tuned correctly. These results are novel as they contribute to the analysis of social data conveying extremist ideas in French.

ABSTRACT. The efficiency and reliability of real-time incident detection models directly impact the affected corridors’ traffic safety and operational conditions. The recent emergence of cloud-based quantum computing infrastructure and innovations in noisy intermediate-scale quantum devices have revealed a new era of quantum-enhanced algorithms that can be leveraged to improve real-time incident detection accuracy. In this research, a hybrid machine learning model, which includes classical and quantum machine learning models, is developed to identify incidents using the connected vehicle (CV) data. The incident detection performance of the hybrid classical-quantum machine learning model is evaluated against baseline classical ML models. The framework is evaluated using data from a microsimulation tool for different incident scenarios. The results indicate that a hybrid neural network containing a 4-qubit quantum layer outperforms all other baseline models when training data is lacking. We have created three datasets: DS-1 with sufficient training data and DS-2 and DS-3 with insufficient training data. The hybrid model achieves a recall of 98.9%, 98.3%, and 96.6% for DS-1, DS-2, and DS-3, respectively. For DS-2 and DS-3, the average improvement in F2-score (measures the model’s performance to identify incidents correctly) achieved by the hybrid model is 1.9 and 7.8, respectively, compared to the classical models. It shows that with a realistic scenario with limited CVs at certain times on certain roadways, the hybrid classical-quantum ML model performs better than the classical models. With the continuing improvements in quantum computing infrastructure, the quantum ML models could be a promising alternative for CV-related applications when the available data is insufficient.

ABSTRACT. Fundamentally and practically, quantum networks and conventional networks are inextricably tied, since the basic quantum protocols such as teleportation require both networks and the conventional network fiber is also used for the quantum network. A Recursive System of Systems (RSOS) model is developed for quantum-conventional (QC) networks by modeling the correlations at various levels based on the failure and attack modes of quantum, conventional and hybrid components and the propagative effects across QC boundaries. A game-theoretic formulation is developed to capture the cost-benefit trade-offs of the provider in defending against component attacks, using sum-form utility functions. By applying the Nash Equilibrium results, the conditions and sensitivity functions of the survival probabilities of a QC network at different levels are derived using the strong dependencies between quantum and conventional infrastructures. The results provide insights into the dependencies between conventional and quantum networks, including cross QC boundary effects in terms of disruption impact of conventional networks on quantum networks, and vice versa.

ABSTRACT. Bayes' rule provides an undoubtedly powerful framework for statistical inference; however, the assumptions inherent in Bayesian filtering often cannot be realized in physical systems. Oftentimes, the true Bayesian posterior probability density function (pdf) is infinite-dimensional and lacks tractable implementations, in addition to errors induced by inaccurate realizations of the prior and likelihood pdfs. Though particle-based methods can provide versatile and computationally efficient approximations of Bayes' rule, they lack the theoretical ability to mitigate estimation errors incurred by erroneous measurement modeling. This work merges Stein Variational Gradient Descent, a nonlinear particle flow update scheme, with generalized variational inference, a method for formulating optimal non-Bayesian posteriors, to produce tractable variational posterior pdfs that remain robust to modeling errors. The new framework is demonstrated to outperform conventional filtering approaches in a simplified relative spacecraft navigation scenario.

ABSTRACT. The Particle filter can in theory estimate the state of any nonlinear system, but in practice it suffers from an exponential complexity in terms of the number of particles as the dimension of the state increases. The marginalized particle filter can potentially reduce this problem by improving the estimates, particularly for lower number of particles. However, it turns out that for certain systems, it does not provide any improvement in the accuracy of the estimate. The core cause of degeneracy is linked to when the uncertainty of the linear state conditioned on the nonlinear state is 0. Conditions for determining when this occurs are presented and applied to common constant velocity, constant acceleration and constant jerk models with various sampling methods. Interestingly, some combinations are useful while others should be avoided. These findings are supported using simulated systems.

ABSTRACT. In this work we present the variance-gamma driven state-space model (VGSSM) - a linear vector stochastic differential equation driven by the variance-gamma (VG) Lévy process, and propose a novel inference framework in such systems. There are closed form expressions for the first four moments of the marginals of the VG process, allowing for more flexible modelling than Brownian motion (BM), retaining BM as a limiting case. The conditionally Gaussian formulation of the variance-gamma process lends itself well to the use of a marginalised particle filter (MPF) which can include the estimation of model parameters as part of the sampling framework. As an example we present a state-space formulation of Langevin dynamics in the VGSSM for estimation of both the observed and the latent first-order dynamics of a system. We apply this specific Langevin formulation to synthetically generated data to validate the results of the MPF, followed by an application to foreign-exchange tick data to demonstrate the method for trend tracking in data sets that are irregularly sampled in time.

ABSTRACT. Recently, Convolutional Neural Network (CNN) has shown higher performance than other machine learning methods for land classification. In this paper, we propose a CNN fusion architecture for peatland site type classification by combining multisource and multiresolution data. The data is acquired by optical and radar satellite remote sensing, airborne laser scanning data and multi-source forest inventory GIS datasets. Based on our data, we are dealing with the high-dimensional class-imbalanced dataset for solving pixel-wise classification of peatlands. To reduce the data dimension and find an optimal subset of inputs, we first applied the sequential feature selection method. Then, we proposed a window-based pixel classification approach based on the selected inputs. This approach can extract the spatial information around each training sample in a defined window region and produce a pixel-wise classfication map. Experiments are carried out for ecological classification of peatlands in Finland.

ABSTRACT. Common visual tracking algorithms make use of bounding box (BB) motion models. These models are parameterized by quantities such as noise covariance parameters corresponding to the evolution of the position, velocity, aspect ratio, width, or height of the bounding box, or their respective velocities. The noise covariance parameters are often hand-selected, using no principled knowledge regarding various aspects such as camera pose or frame rate. This paper aims to analyze how these aspects influence parameter estimates obtained from annotated datasets that are well-known to the visual tracking community. To obtain the estimates, the recently developed measurement difference method (MDM) is modified and used.

ABSTRACT. Common visual tracking algorithms make use of measurement models whose parameters need to be specified. These are, namely, measurement noise covariance related to spatial error of detections provided by a visual detection algorithm, probability of detection, and expected number of clutter detections. The measurement model parameters are often hand selected, using no data-based knowledge. This paper proposes a technique to estimate the parameters by reliably associating detections to annotations in each video frame. The technique is verified on the publicly available MOT-17 dataset.

ABSTRACT. We exploit the model-free paradigm of reinforcement learning to optimize a set of decentralized guidance policies for a swarm of sensor platforms that cooperatively track a constant velocity target. Each platform is equipped with a noisy bearing sensor that measures the azimuth and elevation angle of the target. The measurements from all platforms are shared and an onboard Extended Kalman Filter is used to estimate the relative target state. This state estimation process is taken into account for the optimization of the guidance laws with the aim of proposing trajectories that increase the observability of the target. A multi-agent capable version of Proximal Policy Optimization is used and benchmarked with a Model-Predictive-Control reference approach.

ABSTRACT. This paper presents an exact method of efficiently evaluating radar task performance models required by quality of service radar resource management. The method assumes, that the utility for each task is monotonic in every radar control parameter dimension compared by the performance model. In comparison to the evaluation of all possible radar control parameter combinations, simulations show that the proposed method significantly decreases the required number of evaluations and runtime while still providing exact results. Though, the proposed method is slower than some other approximate methods that were evaluated, which however have the drawback of not guaranteeing exact results. Hence, the method is especially well suited for applications that require exact results with fast runtime. The method is evaluated using a single platform tracking performance model, but can easily be applied to other single platform or multi platform performance models.

ABSTRACT. Target state estimation with Doppler-only sensors has attracted a lot of attention due to its wide potential applications in target localization and tracking. While existing Doppler-only tracking methods rely on the assumption that Doppler sensors have been correctly registered, in many practical cases there can be significant registration errors which imply measurement biases and thus performance degradation in target state estimation. Motivated by this issue, the present paper addresses the problem of jointly estimating target state and sensor biases based on Doppler-only measurements. The proposed method consists of two phases, i.e., (1) raw estimation of the target state without considering sensor biases, followed by (2) a bias compensation step that relies on linearization of the measurement function and joint estimation of target state-sensor biases via a least square method. The Cramer-Rao lower bound (CRLB) in estimating sensor biases is evaluated and the performance of the proposed method is also assessed via simulations.

ABSTRACT. Passive acoustics can provide a variety of capabilities with applications in oceanographic research and maritime situational awareness. In this paper, we develop a method for the navigation of autonomous underwater vehicles (AUVs) in shallow water. Our approach relies on passively recorded signals from acoustic sources of opportunities (SOOs). By making use of the waveguide invariant, a measurement of the range to the SOO is extracted from the spectrogram of a single hydrophone. Range extraction requires knowledge of the range rate, i.e., the radial velocity between the SOO and AUV, computed from the pressure fields at different time intervals. A particle-based navigation filter fuses the range measurements with the AUV's internal velocity and heading measurements. As a result, the position error, which would otherwise increase over time, can be bounded. The ability to compute the range rate and range measurements from the pressure field measured using a single hydrophone is demonstrated on real data from the SWellEx-96 experiment. The capability of the developed navigation filter is shown based on synthetic data generated by the normal mode program KRAKEN.

ABSTRACT. The localization of sound sources in urban environments is a challenging task and the self-position and selforientation of the individual sensor nodes can usually not be assumed to be precisely known. Therefore in this paper, a method for simultaneous sensor and source localization under line-ofsight conditions in urban environment is proposed. The method is based on acoustic bearing and relative time difference of arrival measurements as well as self-position and self-orientation measurements of an asynchronous acoustic sensor network. Here, both direct wave and first-order reflections are taken into account. We derive the Cramér-Rao bound for the corresponding localization problem and compare it with results from a Monte Carlo simulation. We consider cases where the sensors provide all measurements as well as cases where parts of the measurements fail, resulting in sub-measurement data sets. Here, a focus is on scenarios with fully unknown sensor states. Depending on the measurement data set under consideration, the initial selfposition measurement is improved by taking into account the acoustic measurements, which in turn has a positive impact on the source localization accuracy in many considered cases.

ABSTRACT. This paper develops computationally efficient algorithms for the analysis of acoustic data to localize a target through improved angle of arrival estimation. The passive target localization problem has a wide range of applications in wireless communication, navigation, acoustic sensor networks, indoor localization, to name a few. We have focused on novel formulations and solution methods for target localization using time differences of arrival (TDoA) among distinct pairs of passive sensor nodes in an acoustic sensor network with known sensor positions.

ABSTRACT. By developing a method for automatically calibrating the extrinsic parameters of ship-mounted cameras, this paper tests combining Structure from Motion-algorithms with Hand-Eye calibration solvers in a novel algorithm which demonstrates an ability to discern the orientation of cameras with accuracy comparable to- or better than current manual methods, proven through tests with both synthetic and real-world data.

ABSTRACT. This paper presents a method of motion tracking of highly dynamic multi-link systems with Inertial Measurement Units (IMU) embedded with magnetometers in spaces with nonuniform magnetic fields. As IMUs may accumulate error due to drift, magnetometers are used to correct orientation estimation. With usual magnetic distortion compensated for assuming magnetic field uniformity of the space prior to motion measurement, this method introduces real-time calibration using two oppositely facing triaxial magnetometers, along with triaxial accelerometer and gyroscope. Using the described array of magnetometers, local hard iron distortion can be measured at any position. Along with inconsistency of distortion throughout a nonuniform space, the measured direction of magnetic north may also change. Rather than assuming a constant magnetic north direction like conventional methods, a sensor model for the magnetometer using the last measured magnetic north direction as the reference heading is used to compensate for the inconsistency. The proposed sensor suite and model reduces the error by half an order of magnitude less than conventional magnetometer usage. With the proposed method, drift is corrected for, even in nonuniform magnetic fields.

ABSTRACT. In this paper, we present a novel design for a multi-camera tracking system with occlusion-handling capabilities and its application to a highway traffic surveillance system. The fundamental concept follows the tracking-by-detection principle with monocular detectors and an LMB tracker for tracking the objects in the world frame. All data from the multi-view setup is combined into one consistent representation of the real-time traffic situation. In order to assess the inter-target occlusion scenarios in 3D, the vehicles are modeled as cuboids and their extents are estimated from the bounding boxes provided by the detectors. We re-transform the 3D occlusion estimation problem into the 2D camera space and present two methods for quantifying the occlusion state of the objects. Moreover, we propose a modification to the computation of the existence probability of undetected and occluded targets. Based on this, the tracking system is extended by an occlusion-aware detection model. We evaluate our occlusion-handling approach on a real-world traffic dataset from the Providentia++ project and show an improved tracking performance. We find that the number of misdetected targets is reduced and more track identities are preserved.

ABSTRACT. Subjective Bayesian estimation perceives probability density functions as expert opinions. Among various rules for combining the opinions, the product and the weighted geometric mean of densities are prominent. Nevertheless, closed-form representations are scarce and non-parametric approaches often suffer from the curse of dimensionality. This paper prospects the fusion of densities represented by non-parametric marginal densities and a parametric Gaussian copula. The explicit reconstruction of the joint densities followed by an optimisation step is avoided. A cheap approximate combination is proposed instead. The combination of marginal densities is tuned by a Gaussian term, while the proposed copula parameter uses moments of the marginal densities. The presented examples illustrate the approximative nature of the approach for non-Gaussian densities and highlight some numerical issues.

ABSTRACT. We present a new deterministic sampling method for arbitrary densities, unnormalized densities, and likelihoods. Our rejection-free and kernel-free method uses dense equal sphere packing of the volume under the density function (PoVuD). In order to obtain an ensemble that is better than independent random particles, we enforce some local homogeneity.

ABSTRACT. We propose a method for optimal Bayesian filtering with deterministic particles. In order to avoid particle degeneration, the filter step is not performed at once. Instead, the particles progressively flow from prior to posterior. This is achieved by splitting the filter step into a series of sub-steps. In each sub-step, optimal resampling is done by a map that replaces non-equally weighted particles with equally weighted ones. Inversions of the maps or monotonicity constraints are not required, greatly simplifying the procedure. The parameters of the mapping network are optimized w.r.t. to a particle set distance. This distance is differentiable, and compares non-equally and equally weighted particles. Composition of the map sequence provides a final mapping from prior to posterior particles. Radial basis function neural networks are used as maps. It is important that no intermediate continuous density representation is required. The entire flow works directly with particle representations. This avoids costly density estimation.

ABSTRACT. Fine-grained plant pathology classification is an important task for precision agriculture, but at the same time, it is challenging due to the subtle difference in plant categories. Variances in the lighting conditions, position, and stages of disease symptoms usually lead to degradation of classification accuracy. Knowledge distillation is a popular method to improve the model performance to deal with the indistinguishable image classification problem. It aims to have a well-optimised small student network guided by a large teacher network. Existing knowledge distillation methods mainly consider training a teacher network that needs a high storage space and considerable computing resources. Self-knowledge distillation methods have been proposed to distil knowledge from the same network. Although self-knowledge distillation saves time and space compared with knowledge distillation, it only learns label knowledge. In this paper, we propose a novel self-distillation method to recognize the fine-grained plant category, which considers holistic knowledge based on the Squeeze and Excitation Network. We label this new method as holistic self-distillation because it captures knowledge through spatial features and labels. The performance validation of the proposed approach is performed on two public fine-grained plant datasets: Plant Pathology 2021 and Plant Pathology 2020 with the accuracy of 98.22% and 90.72% respectively. We also present experiments on the state-of-the-art algorithm (ResNet-50). The classification results demonstrate the effectiveness of the proposed approach with respect to accuracy.

ABSTRACT. Deep metric learning techniques have been used for visual representation in various supervised and unsupervised learning tasks through learning embeddings of samples with deep networks. However, classic approaches, which employ a fixed distance metric as a similarity function between two embeddings, may lead to suboptimal performance for capturing the complex data distribution. The Bregman divergence generalizes measures of various distance metrics and arises throughout many fields of deep metric learning. In this paper, we first show how deep metric learning loss can arise from the Bregman divergence. We then introduce a novel method for learning empirical Bregman divergence directly from data based on parameterizing the convex function underlying the Bregman divergence with a deep learning setting. We further experimentally show that our approach performs effectively on five popular public datasets compared to other SOTA deep metric learning methods, particularly for pattern recognition problems.

ABSTRACT. In high-rate structural health monitoring, it is crucial to quickly and accurately assess the current state of a structure under dynamic loads. State information is needed to make informed decisions about timely interventions to prevent damage and extend the structure’s life. In previous studies, a dynamic reproduction of projectiles in ballistic environments (DROPBEAR) testbed was used to evaluate the accuracy of state estimation techniques through dynamic analysis. This paper extends previous research by incorporating the local eigenvalue modification procedure (LEMP) and data fusion techniques to create a more robust state estimate using optimal sampling methodologies. In previous studies, the process of estimating the state involved taking a measured frequency response of the structure, proposing frequency response profiles, and accepting the most similar profile as the new mean for the position estimate distribution. However, the LEMP allows for a faster approximation of the proposed model with linear time complexity, making it suitable for 2D or sequential damage cases. The study focuses on two proposed sampling methodology refinements: refining the selection of candidate test models from the position distribution and applying a Kalman filter after the distribution update to find the mean. Both refinements were effective in improving the position estimate and the accuracy of the state estimate, as shown by the time response assurance criterion and the signal-to-noise ratio with up to 17% improvement. These two metrics demonstrate the benefits of incorporating data fusion techniques into the high-rate state identification process.

ABSTRACT. Reinforcement learning is a promising candidate methodology for achieving situational awareness across an area of interest by controlling and processing data acquired from a multi-site, multi-modality sensing gird. We previously reported successful detection, tracking, classification, and identification of objects operating within the area of interest monitored by a sensing grid. We combined online kernel least squares policy iteration (an online reinforcement learning method combining dictionary learning with classical Q-learning) with a particle tracker to achieve these results. The work reported here extends these prior results to show that sensor fusion allows our online reinforcement learning methodology to successfully control a multi-modality sensor platform (consisting of a pan/tilt/zoom electro-optical camera, a radar, and passive radio frequency sensor) to maintain persistent surveillance of objects of interest. Our experimental results, based on simulations in a realistic test and experimentation location performed in Gazebo, show that our sensor fusion methodology allows the online reinforcement learning method to simultaneously learn both camera and other sensor control policies from processed simulated sensor data that perform as well as policies derived from known ground truth information. Moreover, our results show that the learned policies offer significant generalization ability, with the sensor platform being able to successfully track an observed object well past the observed training period. These results further build towards our ultimate goal of achieving automated situational awareness across a heterogeneous sensor grid.

ABSTRACT. This paper proposes a power allocation strategy for multi-target tracking in netted radar system under suppression jamming. The aim is to achieve better tracking accuracy with limited power. The information reduction factor is introduced in the posterior Cram\'{e}r–Rao lower bound (PCRLB) to indicate the uncertainty of the measurement caused by the jamming signal, which is a principal factor that makes the detection probability ($P_d$) being less than unity. This bound is used as the tracking performance metric. Then, a non-convex optimization problem regarding power allocation is established by minimizing the worst case tracking PCRLB. The gradient projection algorithm is used to solve the formulated problem. Finally, a closed-loop feedback framework is established by using the timely feedback results of target tracking to guide the subsequent power allocation. The simulation results verify the effectiveness of the performance metrics and the superiority of the proposed power allocation strategy.

ABSTRACT. The performance of data fusion and tracking algorithms often depends on parameters that not only describe the sensor system, but can also be task-specific. While for the sensor system tuning these variables is time-consuming and mostly requires expert knowledge, intrinsic parameters of targets under track can even be completely unobservable until the system is deployed. With state-of-the-art sensor systems growing more and more complex, the number of parameters naturally increases, necessitating the automatic optimization of the model variables. In this paper, the parameters of an interacting multiple model (IMM) filter are optimized solely using measurements, thus without necessity for any ground-truth data. The resulting method is evaluated through an ablation study on simulated data, where the trained model manages to match the performance of a filter parametrized with ground-truth values.

ABSTRACT. This paper introduces a statistical model and corresponding sequential Bayesian estimation method for terrain-based navigation using side-scan sonar (SSS) data. The presented approach relies on slant range measurements that have been extracted from a SSS ping. In particular, incorporating slant range measurements to landmarks for navigation constrains the location and altitude error of an autonomous platform in GPS-denied environments. The proposed navigation filter consists of a prediction step that is based on the unscented transform and an update step that relies on particle filtering. The SSS measurement model that is introduced aims to capture the highly nonlinear nature of SSS data while maintaining reasonable computational requirements in the particle-based update step. For our numerical results, we assume a scenario with a surface vehicle that performs SSS and compass measurements. The simulated scenario is consistent with our current hardware platform. We also discuss how the proposed method can be extended to autonomous underwater vehicles (AUVs) in a straightforward way and why the combination of SSS sensor and compass is particularly suitable for small autonomous platforms.

ABSTRACT. We propose a method for joint estimation of a host vehicle state and a map of the road based on global navigation satellite system (GNSS) and camera measurements. We model the road using a spline representation described by a parameter vector having a Gaussian prior representing the uncertainty of the prior map. Both GNSS and camera measurements, such as lane-mark measurements, have noise characteristics that vary in time. To adapt to the changing noise levels and hence improve positioning performance, we combine the sensor information in an interacting multiple-model (IMM) setting to choose the best combination of the estimators with the vehicle state and the parameter vector of the map as the state vector. In a simulation study, we compare vehicle models with varying complexity, and on a real road segment we show that the proposed method can accurately adjust to changing noise conditions and correct for errors in the prior map.

ABSTRACT. The superior performance of object detectors is often established under the condition that the test samples are in the same distribution as the training data. However, in many practical applications, out-of-distribution (OOD) instances are inevitable and usually lead to uncertainty in the results. In this paper, we propose a novel, intuitive, and scalable probabilistic object detection method for OOD detection. Unlike other uncertainty-modeling methods that either require huge computational costs to infer the weight distributions or rely on model training through synthetic outlier data, our method is able to distinguish between in-distribution (ID) data and OOD data via weight parameter sampling from proposed Gaussian distributions based on pre-trained networks. We demonstrate that our Bayesian object detector can achieve satisfactory OOD identification performance by reducing the FPR95 score by up to 8.19\% and increasing the AUROC score by up to 13.94\% when trained on BDD100k and VOC datasets as the ID datasets and evaluated on COCO2017 dataset as the OOD dataset.

ABSTRACT. Architectures that first convert point clouds to a grid representation and then apply convolutional neural networks achieve good performance for radar-based object detection. However, the transfer from irregular point cloud data to a dense grid structure is often associated with a loss of information, due to the discretization and aggregation of points. In this paper, we propose a novel architecture, multi-scale KPPillarsBEV, that aims to mitigate the negative effects of grid rendering. Specifically, we propose a novel grid rendering method, KPBEV, which leverages the descriptive power of kernel point convolutions to improve the encoding of local point cloud contexts during grid rendering. In addition, we propose a general multi-scale grid rendering formulation to incorporate multi-scale feature maps into convolutional backbones of detection networks with arbitrary grid rendering methods. We perform extensive experiments on the nuScenes dataset and evaluate the methods in terms of detection performance and computational complexity. The proposed multi-scale KPPillarsBEV architecture outperforms the baseline by 5.37% and the previous state of the art by 2.88% in Car AP4.0 (average precision for a matching threshold of 4 meters) on the nuScenes validation set. Moreover, the proposed single-scale KPBEV grid rendering improves the Car AP4.0 by 2.90% over the baseline while maintaining the same inference speed.

ABSTRACT. In this article, a new variational pansharpening method based on kernel estimation and regional extended low rank is proposed, which aims to generate a high resolution multispectral (HRMS) image by fusing the panchromatic (PAN) and multispectral (MS) image. First, an estimated blurring kernel is generated for the spectral constraint term, which can build the relationship between the MS and HRMS image more accurately and improve the spectral quality of the HRMS image. Second, a spatial constrain term is designed by adopting the proportional relationship of the PAN and HRMS image in gradient domain, which preserves the geometric information of the PAN image well. Third, according to sensor imaging principle, a prior constraint term is proposed based on regional extended low rank, which can improve the spatial clarity of HRMS image. The above three constraint terms are combined to form the proposed variational pansharpening method, and the ADMM method is applied for solving it. Finally, experiments show the effectiveness of the proposed method through comparing with other state-of-art pansharpening methods.

ABSTRACT. Consider a target being tracked by a cognitive radar network. If the target can intercept noisy radar emissions, how can it detect coordination in the radar network? By 'coordination' we mean that the radar emissions satisfy Pareto optimality with respect to multi-objective optimization over the objective functions of each radar and a constraint on total network power output. This paper provides a novel inverse multi-objective optimization approach for statistically detecting Pareto optimal ('coordinating') behavior, from a finite dataset of noisy radar emissions. Specifically, we develop necessary and sufficient conditions for radar network emissions to be consistent with multi-objective optimization (coordination), and we provide a statistical detector with theoretical guarantees for determining this consistency when radar emissions are observed in noise. We also provide numerical simulations which validate our approach. Note that while we make use of the specific framework of a radar network coordination problem, our results apply more generally to the field of inverse multi-objective optimization.

ABSTRACT. The training error of Machine Learning (ML) methods has been extensively used for performance assessment, and its low values have been used as a main justification for complex methods such as estimator fusion and ensembles, and hyper parameter tuning. We present two practical cases where independent tests indicate that the low training error is more of a reflection of over-fitting rather than their generalization ability. We derive a generic form of the generalization equations that separates the training error terms of ML methods from their epistemic terms that correspond to approximation and learnability properties. It provides a framework to separately account for both terms to ensure an overall high generalization performance. For regression estimation tasks, we derive conditions for performance enhancements achieved by hyper parameter tuning, and fusion and ensemble methods over their constituent methods. We present experimental measurements and ML estimates that illustrate the analytical results for the throughput profile estimation of a data transport infrastructure.

ABSTRACT. Affine formation control of multiagent systems has recently received increasing attention in various applications. The distributed control of these agents, under single integrator dynamics, relies on the observations of relative positions of the neighboring agents, which when unavailable is detrimental to the mission. In this paper, we propose an adaptive fusion estimator of the relative positions under intermittent and consecutive observation loss settings. A relative affine localization (RAL) solution is developed by exploiting the geometry of affine formation, which is then embedded into a distributed relative Kalman filtering (RKF) framework, leading to the geometry-aware relative Kalman filter (GA-RKF). We show through simulations that the GA-RKF exhibits enhanced robustness to both intermittent and consecutive observation losses, as compared to RAL and existing state-of-art methods.