ABSTRACT. To facilitate the incorporation of Unmanned Aerial Vehicles (UAV’s/drones) into maritime military operations, it is critical to be able to localize accurately the UAV with respect to the maritime vessel during the take-off and landing phases. This work addresses the study and implementation of a visual detection, tracking and position estimation method for a specific drone from a maritime vessel. First, a custom dataset of our research subject drone – a DJI Matrice M300 – was acquired. It contains thousand annotated images recorded during our field tests. A Python tool for automatic video annotation and processing based on video object tracking and manual evaluation is developed to generate the annotated images and ground truth bounding box data from the recorded videos. It also guarantees proper distribution of the bounding box sizes and positions in the dataset. This dataset was augmented then randomly split into training and validation sets and used to train and evaluate multiple algorithms for object detection and tracking. YOLOv5 detector [1] and OceanPlus tracker [2] were selected based on their standout performance in accuracy and processing time. The result of YOLOv5 detector with network size small and medium on our custom dataset is presented in the Table 1. Using the same low-cost hardware (ThinkPad P50 with an Intel i7-6700HQ CPU, 2GB VRAM Quadro M1000M GPU and CUDA 10.2), the OceanPlus tracker achieves 10,77 FPS with better accuracy and robustness compared to other tracking algorithms in our evaluation. The detection and tracking algorithms are integrated in one workflow, where the result of the detection will initiate the input for tracking. A redetection will be triggered as soon as low tracking confidence or failure occur. The position of the drone with respected to the maritime vessel is estimated using stereo triangulation. A realistic simulated environment in Unreal game engine was created to evaluate the position estimation result. It includes a maritime vessel model with a simulated stereo camera with a baseline of 2m, a Matrice M300 drone model, and other environment parameters: light, wave, etc. Using 4K stereo images, the proposed method allows to detect, track, and estimate the position of target drone at range of more than 100m with position estimation error below 10cm compared to the ground truth data from simulated environment.

ABSTRACT. Ship deck landing of Unmanned Aerial Vehicles (UAV’s/drones) in different kinds of environment conditions remains bottleneck for widespread deployment of UAVs for maritime operations. For safe operation, the relative motion between the UAV and the pitching and rolling deck of a moving ship must be estimated accurately and in real-time. This paper presents a visual Simultaneous Localization and Mapping (SLAM) method to real-time estimate the motion of the UAV with respect to its confined landing area on a maritime platform during landing phase.

Multiple state-of-the-art visual SLAM and Visual Odometry (VO) algorithms are analyzed, ORB-SLAM3 [1] and OV2SLAM [2] were selected to be evaluated with the EuRoC dataset [3]. Figure 1 shows the accuracy (ATE: Absolute Translational Error) and the computational speed (Tracking Time) performances of the two algorithms when using only stereo camera as input. ORBS-SLAM3 was selected for better accuracy and sufficient computational speed for our application.

A realistic simulated environment was created in Unreal Engine to perform the evaluation of the ORB-SLAM3 for the landing scenario of the drone at 16m height with downward camera. The accuracy and computational speed performance of different camera setups: stereo/monocular, 720p/376p resolution were shown in the Table 1. A merging map software feature is developed, allows to scan, and save the point cloud map of the ship deck as non-active map which then will be merged with the current active map as soon as enough matches between the two maps are found. This feature helps improve both speed and accuracy of the algorithm especially for monocular configuration. Other than motion estimation of the UAV, our visual SLAM technique also provides the possibility to determine a collision-free space which can be incorporated with motion planning of the autonomous UAV.

ABSTRACT. Generally the control of nonlinear systems yields a great challenge. However for a rather broad class of systems, which posses the differential flatness property, the trajectory tracking problem is rather easy to solve. Using the Lie-Backlund equivalence a system is differentially flat if there exists a Lie-Backlund transformation which that makes the system's Cartan field equivalent to the trivial vector field. By using the flat outputs and their adequate number of time derivatives all the system states and inputs can be expressed, the reference path is then designed in the flat outputs.

In this paper the trajectory tracking control of a simple planar overhead crane is synthesized, fusing the differential flatness property with a linear compensator designed with the active disturbance rejection control (ADRC) point-of-view. The main idea is that with a sufficiently smooth reference path the nominal plant input can be calculated beforehand, which (if there are no disturbances, the parameters are perfectly known and the initial conditions of the plant are exactly matched with the reference path at time zero) guides the system through the designed reference path in an open loop manner. In a real application however several uncertainties are present in the control loop. To mitigate these effects a linear compensator can be added, which ensures that the trajectory tracking error is exponentially decreasing, regardless of external effects and uncertainties.

The crane itself is an under-actuated mechanical system, although differentially flat, so the above mentioned control method results in an adequate trajectory tracking performance. The paper presents the theoretical foundation of the method, its design steps for the crane system and the simulation results, where the effects of uncertain parameters and external disturbances can be investigated.

ABSTRACT. Computed Tomography (CT)-guided intervention requires an expertized skill to physicians and a numerous of robot-assisted needle insertion system has been developed. As a challenging issue developing the CT-guided needle insertion robot system, the inside of a CT gantry is a special environment where there is no room for movement and no metal can be inserted because of causing an artifact on the acquired image. The gantry entry portion of the robot therefore requires miniaturization and non-metallization. The most difficult part of the non-metallization of a robot is the motor element. While many of the robot's structural materials can be made non-metallic thanks to advances in plastic materials, the actuator cannot be made non-metallic. In some cases, the actuator can be attached directly to the moving part, but this limits the angle of motion of the robot to where the actuator is outside the imaging range. In this report, we introduce a unique mechanism that enables the needle manipulation with a miniaturized configuration inside the CT gantry. The mechanism satisfies the requirements of power transmission between axes whose relative positions are indeterminate, miniaturization to prevent interference with robot motion, and non-metallization. This mechanism combines a single-node link and gears to transmit rotational power while passively responding to changes in the positional relationship between axes. The hinge fixation of the basic single-node link is aligned with the axis of rotation of the first gear, and the subsequent gears can be freely arranged to mesh with each other. The minimum configuration is 3 gears, one at the hinge, and two at each end. Theoretically, the number of gears can be increased, but it is desirable to use as few gears as possible because transmission accuracy will decrease due to backlash and other factors. The movement of the mechanism does not necessarily affect the state of the gears at all. If the absolute angle of the linkage changes, the positional relationship between adjacent gears changes while the distance between the axes remains constant. If the positional relationship of the meshing gears changes, the gear will rotate. If rotation is simply transmitted, this angular change does not affect the operation of the machine, but if rotation is transmitted for the purpose of angle determination, the angle input side must be adjusted. This depends on the changing tip position coordinates. Therefore, we developed an algorithm that outputs adjustment parameters upon input of information on the mechanism to be used. The algorithm derives an approximate equation from numerical analyses that limits the error from backlash. We applied this mechanism to our robot with a 25G needle and inserted the needle into a target from various angles. The accuracy was 2.5 mm and satisfied our requirements.

ABSTRACT. Joint flexibility is a property that exists in manipulators with transmission systems for torque amplification, including harmonics drives and transmission belts, or series-elastic actuators utilized for safe operation. When joint elasticity is significant, the application of controllers designed for rigid manipulators on a flexible system can cause a general degradation of performance over time, as well as loss of stability. Control strategies that mitigate these problems are necessary in any application where flexible manipulators require high positioning accuracy or if there is any close interaction with a human operator as a safety precaution.

In this paper, we propose an add-on dynamic gravity compensator for flexible joint manipulators. The designed compensator pumps sufficient power that matches the gravity power demand on the manipulator links, resulting in a reduced oscillatory response. The proposed power based gravity (PG) Compensator uses moving window batch least squares approach to implement an add-on control signal that facilitates the power match. The performance of the proposed PG Compensator was validated when added to a base-line Proportional Derivative (PD) controller and two PD-based benchmark controllers. MATLAB simulations revealed that the PG compensator decreases the transient response error by 49% and the settling time by 48% when added to the three benchmark controllers. In every case, the addition of the compensator improves the system response for regulation, allowing for faster convergence to steady-state. Experimental testing on a Quanser single-link flexible-joint beam showed a 23% transient error improvement over the baseline controller without the power compensator.

This work contributes to the development of a more effective easy-to-implement controllers for flexible joint manipulators and will improve in the safety and precision of deploying these systems in practical applications.

ABSTRACT. This paper presents a modification to the swarm intelligence algorithm to coordinate a fleet of (inexpensive) drones in a search and rescue mission. In this search and rescue scenario, it is common for the target to be moving and therefore renders the original swarm intelligence algorithm, designed for stationary target, to become inapplicable. Thus, the modification of the algorithm to use the triangulation method to approximate the current estimated position of the target as the globally known best solution. This modification allows the target to be moving while the drones, acting as search agents, will converge toward the target without loss of generality. Computer simulations are provided to demonstrate the workability and practicability of the modified swarm intelligence algorithm. Future work is discussed for a scenario where a target (a group of people) is split into multiple targets going in different directions.

ABSTRACT. The simultaneous localization and mapping problem nowadays has many implementations for single robot systems. The current challenge is to extend these algorithms to work efficiently with a multi-agent setup as well. One of the arising problems is the limited bandwidth for communication between the agents. The industry standard grid-map based representation is a very verbose one, thus making data exchange between the agents more challenging. In our paper we outline a new, polygon-based map representation and present our results so far. We recognize the importance of testing such a new architecture on a physical system. Therefore, we designed and implemented a robotic system, which is cost effective enough for our research group to be able to afford multiple instances of it, even on a tight budget.

The currently industry standard grid-map representation is relatively memory heavy, since we have to store information about each cell of the environment. Research is currently still in progress on how to reduce the size of it with merging neighboring cells, if they are similar. However, the size of this representation makes it challenging to use effectively in a multi-agent system, where agents should share their knowledge with others in real-time. Hence the vector-based map representations, where obstacles are represented as geometric shapes are gaining attention in recent years [1]. While this greatly reduced the memory footprint of the map, the fitting of these shapes on geometries which we haven’t fully observed yet involve many difficulties [2] [3]. We think that a good intermediate solution can be the estimation of the obstacles as polygons. This way we can approximate the boundaries with arbitrary precision, while keeping the storage requirements of the map low.

Environment exploration can be more efficient if we use frontiers to store, in which direction can we continue. We reached back to the article of González-Baños and Latombe [4] and used it as a base to develop the algorithm.

We find it important to test our research not only in simulations, but in physical environment, since after all that is where it has to be applicable. Therefore, we designed an omnidirectional robot, with respect to the cost effectiveness, so later we can deploy them in groups. Although the price of LIDAR sensors dropped significantly in the past years, it’s still expensive to buy multiple pieces of it, so we restricted ourselves to a single distance sensor, and that the robot is turning around to record the measurements. This however ruled out the opportunity to navigate precisely and safely in the environment, because of the low sampling rate. To resolve this issue, we introduced a camera-based localization system, using ArUco markers, which is also suitable for the multi-agent setup.

At this stage of the development, the algorithm is being refined for a single agent, but hopefully we will be able to soon extend it to multiple ones. We are also working towards being able to upgrade the distance sensor to LIDAR. On the long run we would like to develop a working, efficient multi-agent exploration system in 2D and design concepts which can be later used in 3D, with polygon mesh approximation.

References

[1] J.-C. L. Héctor H. González-Baños, "Navigation Strategies fro Exploring Indoor Environments," SAGE Publications, Pennsylvania, 2002. [2] L. J. L. D. W. Rolf Lakaemper, "Geometric Robot Mapping," ResearchGate, Poitiers, 2005.. [3] G. L. L. F. N. B. Khelifa Baizid, "Vector Maps: A Lightweight and Accurate Map Format for Multi-robot Systems," in Intelligent Robotics and Applications 9th International Conference, Tokyo, 2019. [4] L. F. G. L. a. N. B. Johann Dichtl, "PolyMap: A 2D Polygon-Based Map Format for Multi-robot Autonomous Indoor Localization and Maping," Springer Nature Switzerland, Douai, 2018.

ABSTRACT. Using mobile robots to collect data from wireless sensor network can reduce energy dissipation and this way improve network lifetime. Our problem is to plan paths for multiagent system to visit a set of sensor nodes in a sensing field with obstacles while minimizing the data collection time. Recontructing the path of an intruder in a guarded area is also a possible application of this technology. This paper applied a new approach for construct clusters of nodes that each robots must visiting. To design the clusters and visiting sequence of nodes a new algorithm is developed.

ABSTRACT. Deep Learning-based computer vision methods have achieved state-of-the-art performance in the last decade, prompting a renaissance in the field of Artificial Intelligence (AI). Still, despite recent successes, many challenges remain including the difficulty of generalization, the cost of large amounts of labelled training data, and the robustness of these solutions to disturbances of the input. These disturbances may include corruption of the input image either due to compression or natural effects, such as rain, fog or bright light sources, or difficulties presented by the structure of the 3D scene, such as a complex background or occlusions. Object occlusions are an integral part of the research of visual object detection because the performance of deep learning-based solutions is heavily affected by this phenomenon. While some solutions exist for neural network-based object detection under partial occlusions, these methods still implement the object detector as a passive observer in the scene. We argue, however, that an active object detector agent may be able to achieve better results by changing its own position to resolve occlusion, when possible. Visibility of occluded objects may be achieved as a result of a perception-action loop by a reinforcement learning-trained agent. In this paper, an OpenAI Gym-compatible virtual environment is presented that enables the creation of realistic occlusion datasets via differentiable rendering. An optimization method is also presented that calculates a differentiable loss function minimizing the occlusion of the rendered objects and determines the optimal movement of the agent to which the camera is fixed to progress toward a state where both objects are fully visible. A trained neural network is also introduced, which predicts the occlusion mask and the optimal camera movement via self-supervised learning in the environment. Our experiments presented in this paper show that the neural network model is able to outperform the gradient-based optimization, and efficiently avoid occlusion in simulated scenes containing multiple objects.