ABSTRACT. Mastectomy is one of the treatment options for breast cancer, which is often coupled with breast reconstruction surgery (BRS), to reconstruct the breast mound. BRS plays a vital role in ensuring the self-esteem and health-related quality of life (HRQoL) of patients who have undergone mastectomies. However, BRS is challenging, time-consuming, and subject to the surgeon's judgment in determining the amount of tissue harvested from the abdomen and the shape of the reconstructed breast. To date, there is no tool or guidance to maintain symmetry and appearance (including shape and volume) of the reconstructed breast. In this paper, we have developed an intuitive breast reconstruction system comprising a software application integrated with the simultaneous localization and mapping (SLAM) algorithm, and a low-cost RGB-D camera. Our SLAM-based breast reconstruction system is used to scan and reconstruct the patient's breast using our customized mesh-generating method prior to mastectomy. Using this reconstructed model, a patient-specific 3D printed mold is created to help shape the harvested tissue inserted into the mastectomy site during BRS. This mold serves as an objective method to ensure symmetry between the reconstructed and healthy breasts, in turn, improving the HRQoL of mastectomy patients. Validation experiments show that the mean repeatability and accuracy errors for the surface reconstruction are less than 1.5 mm. The software application was also tested on 15 patients (1 intraoperative) in a multi-center clinical trial under an IRB-approved protocol (Yale Medical School and Brigham and Women’s Hospital). The results show the consistent and accurate reconstruction of the breasts including the inframammary fold for different breast shapes and skin tones.

ABSTRACT. Surgical navigation systems are infrequently used for lateral skull base surgeries. we have developed a novel simultaneous localization and mapping (SLAM)-based navigation system that localizes the anatomy in real-time based on images without the use of a tracking system. Using this technology, a brief pan of the endoscope can create a 3D surface model of the operative field in real time from the surgeon’s perspective. This computerized surface model is then fused to preoperatively acquired CT images of the temporal bone, providing information about the relationship of the exposed tissue surface to underlying anatomic structures. We have tested our technology on both sides of four cadaver heads and five surgical steps of the translabyrinthine dissection. The reconstruction, registration and localization errors are less than 0.78 mm, 0.99mm, and 2.06mm respectively.

ABSTRACT. In this paper, we proposed a robotic system with a flexible end-effector and stereovision feedback for neurosurgery by integrating our previously-developed Concentric Tube Robot (CTR)\cite{He-CTR-ICRA2023}, the da Vinci Robot Research Kit (dVRK) \cite{kazanzides-dvrk-2014} and a customized dual endoscope camera subsystem. The CTR manipulator was teleoperated with the dVRK master tool manipulator (MTM). A virtual motion boundary was applied for the MTM by haptic feedback based on the CTR's workspace to guide the operator to control the CTR within its motion range. The manipulation performance of the proposed system was experimentally evaluated and the results of that showed under the stereovision feedback the manipulation accuracy of the CTR is 2.8 mm and the image transmission latency is 1.5 seconds. This preliminary study suggests that our proposed system has the potential of improving surgeons' manipulation performance in robot-assisted minimally invasive neurosurgery.

ABSTRACT. Active Constraints (AC), also named as Virtual Fixtures (VF), are a strategy to provide anisotropic haptic guidance for surgeons during use so that motions that comply with safety requirements are permitted, while those that breach safety requirements are negated. AC can be helpful also in a teleoperative surgical scenario, in which surgeons operating on the surgeon-side interface are separated from the patient-side, where surgical instruments are held and manipulatd by a robot. One challenging aspect of incorporating AC into a modern clinical setting is how to efficiently update pre-constructed AC geometries in real time to account for dynamic tissue movement. In this study, we designed a pipeline for implementing Forbidden Region AC (FRAC), where tissue movement is constantly captured by a depth-sensing camera. The effectiveness of the pipeline has been confirmed through in vitro trajectory tracking experiments along a deforming aorta phantom. Our experimental results demonstrate the capability of our method to provide timely corrective guidance when a violation of the safety region is detected.

ABSTRACT. This paper describes our recent in vivo robotic image guided surgery (IGS) experiences and how they are informing future research directions. We tested our system, which employs touch-based registration, using an intrinsically safe bystander study protocol. We observed significant soft tissue deformation. Toward addressing this, we took first steps toward integrating intraoperative ultrasound into our system. We calibrated the da Vinci’s “drop-in" ultrasound probe using the robot’s other hand to collect a set of image points with known spatial locations. In summary, our in vivo pilot studies have revealed significant soft tissue deformation, and we plan to pursue future research in combining ultrasound with tissue deformation models to augment our current surface-based registration processes.

ABSTRACT. Prostate cancer is one of the most common malignancies and the second leading cause of cancer death in men. Approximately 52,300 new cases of prostate cancer are detected in the UK every year, that’s more than 140 every day. Magnetic resonance imaging (MRI) has been widely used in the diagnosis of prostate cancer, as it can offer high-resolution tissue imaging at arbitrary orientations and monitor therapeutic agents, surgical tools, and tissue properties. Therefore, a robot - under the guidance of MRI - can target the tumor regions with high accuracy to obtain the biopsy samples for diagnosis, thus reducing unnecessary gland punctures and maximizing the utility of a minimally invasive system. However, as MR scanners require a strong magnetic field, ferromagnetic materials are precluded as they can cause a hazard to the device and patients, and paramagnetic materials can generate their own magnetic field which will distort the image quality. As a result, MR-safe actuators are required to power the robot. and metallic components are not allowed to be used inside the robot. Plus, due to the limited size of the MR bores, the robot operating inside should be as compact as possible. In this paper, a robotic system for MRI-guided prostate biopsy is proposed that has a compact size, with its workspace covering the whole prostate. It is powered by pneumatic actuators which can avoid the influence of electromagnetic interference. The working principle, mathematical model, and mechanism design are presented. The needle insertion experiment under an MR environment was conducted.

ABSTRACT. Craniosynostosis is a disorder in which the cranial sutures of an infant’s skull are prematurely fused. As a result, brain growth is restricted causing dysmorphism and functional sequelae. Treatment involves surgically removing the fused suture or remodeling the skull by osteotomies and can be performed either open or endoscopically. The endoscopic approach is minimally invasive. However, the extent of the osteotomies is limited using conventional surgical instruments due to the unique curvature of the skull. There is therefore a need to develop a novel steerable bone cutting tool that can perform more extensive osteotomies along the skull surface using minimal access incisions.

A novel steerable instrument was developed and prototyped, which allows performance of an extended range of osteotomies on the skull using key-hole incisions. The tool comprises a cable driven articulating bending section and end-effector with a cable driven bone punch that is manually operated using a driving handle. The end-effector also incorporates a dural protector, scalp retractor and flexible endoscope for visualization. The workspace of the tool was determined to have a deflection at the tool tip of 23.7 mm and 120° of yaw. A phantom craniosynostosis model was developed and used as a test bed. The tool was inserted through a single vertex incision and demonstrated reachability of the tool tip to distal areas of the skull along the frontal, parietal and occipital bones. The tool demonstrated the ability to successfully cut simulated bone. This tool represents a first stage in the development of a minimally invasive robotic cranial bone cutting instrument and is a paradigm shift in the treatment of patients with craniosynostosis.

ABSTRACT. Trauma to the ankle is an increasingly common injury and is a major source of long-term debility. Over two million ankle injuries occur each year in the United States alone. Over half a million ankle injuries require surgeries. Multiple studies have demonstrated accurate reduction of the distal tibiofibular joint (syndesmosis) as a critical predictor of good clinical outcome. Surgical manipulation of the tibia and fibula is necessary to properly align and reduce the syndesmosis space in ankle fractures involving sprains of the syndesmosis. However, current techniques of manual reduction utilizing open or percutaneous approaches have been shown to result in inaccurate reduction of the syndesmosis. We propose a novel system that combines intraoperative imaging based on low-dose cone-beam computed tomography (CBCT) and 3D-2D image registration with robotic manipulation of the fibula to precisely restore its anatomical relationship with the tibial incisura. Our long-term goal is to develop robotic assistance with intraoperative imaging for precise reduction of the syndesmosis, while minimizing radiation exposure to the patient and the surgical staff. The focus of this study is to investigate the feasibility of the robot design and the potential clinical workflow.

ABSTRACT. This paper presents an integrated design and test methodology of a custom made three degrees of freedom delta robot with linear actuators achieving micron-level accuracies. The application of the delta robot is to perform superficial tissue optical biopsy requiring a target end-effector motion resolution of 1 μm. The design methodology considers two parameters to determine the robot geometry and dimensions: the end-effector workspace (~ 5 mm3) and the end-effector target motion resolution (1 μm). The robot performance is evaluated using a non-contact metrology approach based on bright field microscopy to characterize its precision and kinematic performance. Our results demonstrate that the presented methodology can be used for designing high precision robots achieving accuracies <1 μm having the potential to be used for cellular level surgical procedures.

ABSTRACT. Colorectal cancer is one of the most common cancers, affecting both men and women. Each year nearly 50,000 patients in the US alone undergo partial colectomies for polyps which otherwise could have been removed endoscopically. The adoption of endoscopic procedures is greatly hindered by the limited dexterity of available, rigid endoscopic tools. We propose an endoscopically deployable, flexible robotic system which will allow for greater dexterity. In this paper, we computationally analyze design variables to determine the relationship between the robot design parameters and a target workspace. We find that higher maximum curvature and a longer proximal setup sheath with a shorter distal setup sheath allows for more voxels to be reached. Proceeding with the resulting design parameters of tubes, we can continue work on modeling, actuation, and control of the robotic manipulators. Building on this work, we ultimately hope to realize the potential benefits of ESD to reduce the number of unnecessary invasive colectomies as well as reduce the risk of infection, reoccurrence, and other adverse events.

ABSTRACT. Eye surgery requires high precision and steady manipulation inside of the human eyeball. There have been lots of studies about active needle-shaped robots for minimally invasive surgery. Recently, many studies on the stiffness chaning structure for these needle-shaped robot have been conducted. In this paper, we propose a mechanism that can change the stiffness of the needle at two different points. The proposed mechansim includes an outer tube, an inner tube, and a tendon. Each tube has two notched sessions (one is a distal and the other is a proximal). By changing the relative position (rotation/translation) between the tubes, the stiffness of the robotic needle system could be adjusted. Using the first prototype and manual experimental setup, the stiffness changing feasibility was confirmed.

ABSTRACT. Minimally invasive endovascular procedures involve the use of catheters that are guided through the blood vessels to reach the target area where the intervention is performed. During the procedure, there exists an inevitable frictional interaction between the catheter and the vessel walls. While this friction can enhance stability during the intervention, it can also pose a risk during navigation, potentially causing damage to the inner layer of the blood vessel wall and leading to thrombus formation. To mitigate the risk of adverse complications, we propose a novel concept of a variable friction catheter. This catheter utilizes the principle of ultrasonic lubrication to actively control the frictional forces experienced by the catheter during the procedure. The unique feature of this catheter is its ability to adjust its frictional properties in real-time, providing low friction for easier navigation and high friction for enhanced stability during the intervention. In this paper, we introduce a proof-of-concept prototype for a friction control module, a fundamental building block of the proposed catheter concept. Our preliminary experiments have shown that this prototype is capable of reducing friction by up to 50%, highlighting the viability of the design and its potential to enhance the safety and effectiveness of minimally invasive endovascular procedures.

ABSTRACT. The epidural injection is a medical intervention to inject therapeutics directly in the vicinity of the spinal cord and the nerves branching from it. Epidural needle insertion is a blind procedure that relies merely on the physician’s tactile feedback. Nevertheless, tactile feedback can be polluted with needle-tissue friction and vary from patient to patient. To achieve sub-millimetre accuracy, preempt neurological damage and reduce the radiation exposure time for patients and physicians, new technologies have been used. Inspired by the recent developments in spine surgery, we have studied the user experience who used our robot-assisted needle insertion system for epidural space localization and needle insertion in this study. In addition, the accuracy and repeatability of augmented reality-assisted epidural needle insertion were compared to that of non-assisted robotic needle insertion. For user experience assessment, NASA Task Load Index (TLX) was used and analyzed. The proposed system improved the accuracy, repeatability, and success rate of epidural needle insertion on an anatomical model. In addition, it reduced procedural time and was more effective from the users’ perspective.

ABSTRACT. Continuum robots have attracted considerable attention for applications in minimally invasive diagnostics and therapeutics over the past decade. The primary reason is their ability to navigate narrow and tortuous anatomical passageways, while guaranteeing safe interaction with the anatomy. The main difficulty, in designing continuum robots, is guaranteeing a large dexterous workspace i.e., a large set of points approachable from any direction. We show how this is achievable, when combining cable driven and magnetic ball chain robots. We present a kinematic model, used to describe the robot’s workspace, and present an experimental validation.

ABSTRACT. This paper explores combining concentric tube robots (CTR) and notched wrists for achieving greater workspace volumes and dexterity. A three-tube CTR is compared with two hybrid CTR and notched wrist systems. Each hybrid system comprised of a wrist with either 1-degree of freedom (DOF) or 2-DOF bending capabilities. For each system, ten million joint configurations were randomly sampled in simulation to obtain the total workspace volumes and dexterity maps. The hybrid system with a 1-DOF wrist generated the maximum dexterity values, and the hybrid system with a 2-DOF wrist generated the largest workspace volume. The improved workspace and dexterity from combining these two technologies demonstrate a tool that may be more suitable for intricate surgical operations.

ABSTRACT. The medical sector has emphasised increasing levels of autonomy to achieve safe and efficient robot-assisted surgeries. In this case, robust and intuitive manipulation of medical robots is crucial, and many tele-operated surgical robots have been developed, e.g., the da Vinci robotic platform. The tele-operation can offer high operation precision and intuitive manipulation. In addition, soft robots have led to the development of inherently safe and flexible interventional tools for medical applications, e.g, the minimally invasive surgery (MIS). Soft instruments are particular advantageous to navigate in tortuous anatomical environments with constrained space. Combining the tele-operation technology with soft robots might further result in a significant reduction in operation time and increase of surgeons' dexterity. The contribution of this work lies in the design and control of a tele-operated soft instrument for laparoscopic examination are proposed based on the miniaturised STIFF-FLOP manipulators (with a diameter of 11.5 mm). Specifically, the robot has two serially connected modules, which can seamlessly fit to commercially available 12 mm trocar ports used in MIS. The bending angle of the soft instrument can achieve 180 degrees. We also preliminarily validate the feasibility of the soft instrument.

ABSTRACT. Atrial Fibrillation (AFib) is the most common arrhythmia among the elderly population, where electrical activity becomes chaotic, leading to blood clots and strokes. During Radio Frequency Ablation (RFA), the arrhythmogenic sites within the cardiac tissue are burned off to reduce the undesired pulsation. Several studies showed excessive contact forces (more than 0.45 N) increase the incidence of tissue perforation, while inadequate force (less than 0.1 N) results in ineffective ablation. For the purpose of addressing the force estimation problem, finite element analysis can provide a useful tool to estimate the real-time tip contact force of the RFA catheters. In this work, a nonlinear planar finite element model of a steerable catheter was first developed with parametric material properties in ANSYS software. After that, a series of simulations based on each mechanical property was performed, and the deformed shape of the catheter was recorded. Next, validation was conducted by comparing the results of the simulation with experimental results between the range of 0-0.45 N to determine the material properties. The main contribution of this study was proposing a synthetic data generation, so as to train a light deep learning architecture for tip force estimation according to the finite element simulations. The proposed solution not only feeds the data-hungry methods based on deep learning with a sufficient amount of data, but also shows the feasibility of replacing the fast, accurate, and light-weight learning-based methods with slow finite element simulations.

ABSTRACT. Needle bending is a significant cause of error in biopsies, leading to lesion missampling and consequent cancer misdiagnosis. This paper presents the experimental testing of a new mechanism that detects the needle bending as soon as it occurs and immediately reduces the insertion force. This is achieved without employing sensors or actuators. Our in-vitro experiments on silicone-rubber phantoms indicate that the device can avoid deep insertions with deflected needles, thus potentially reducing the associated risks.

ABSTRACT. People over the age of sixty-five are most commonly affected by aortic stenosis (AS), a common heart valve disease. Transcatheter aortic valve implantation (TAVI) is a minimally invasive treatment for AS that replaces the function of the diseased native valve with a prosthetic device that relies on balloon catheters for device implantation. According to current clinical guidelines, the choice of the implantable device is based on preoperative sizing measurement by image-based technology. However, this assessment has inherent limitations that can lead to the selection of a sub-optimal prosthesis size, which in turn can lead to significant intra-operative complications such as aortic regurgitation or electrical signal disturbances in the heart.

Using balloon pressure and volume data, this paper proposes an intra-operative method for determining the size of the aortic annulus, taking into account its compliance and elliptical geometries. Intra-balloon pressure-volume curves were obtained using an inflation device operating a commercially available valvuloplasty balloon catheter. A sizing algorithm for the estimation of annulus dimensions was integrated via a characterised analytical model and a numerical model for balloon free-inflation.

Experiments were performed on circular and elliptical idealized aortic phantoms. Experimental results show that the pressure-volume data processed by the sizing algorithm can be used to determine the circular annulus diameter for all tissue stiffnesses. The measurement of stiffer elliptical annulus phantoms shows good precision and high repeatability. This work represents a significant step forward in improving the selection of TAVI devices by sizing the compliant aortic annulus with complex geometry using balloon catheters.

ABSTRACT. Endoscopic procedures have gained widespread use due to their ability to facilitate accurate diagnosis and treatment through minimally invasive surgery. While traditional bimanual platforms are commonly utilized, the use of secondary hand traction assisted devices offers additional benefits including precise and flexible tissue manipulation. In this work, a novel handheld endoscopic platform, namely EndoGRASP is proposed. EndoGRASP is a handheld endoscopic platform that includes a flexibility-retaining robotic overtube and an integrated actuation unit. This system features dedicated channels for instruments and endoscopes, including a biopsy channel and an active bending section. The biopsy channel enables free translation of instruments, while the active bending overtube compensates for the loss of flexibility in the endoscope and provides rigid support for the instruments and endoscope to bend independently. EndoGRASP improved the

ABSTRACT. Globally, the minimally invasive surgery (MIS) has been applied to more and more medical fields. The traditional endoscopes are adopted to help the surgeon to get the field of view (FOV) inside the body cavity. But it will also cause the problem of port-crowing and instrument-fencing which will interfere with the operation. This article proposed a magnetic anchored and cable driven endoscope for minimally invasive surgery to address these problems. This system mainly consists of two parts, the external unit and internal unit. The external unit contains one external permanent magnet (EPM) and its holding device. The internal unit is composed of one internal permanent magnet (IPM), one cable-driven mechanism, and a camera. The magnetic coupling between the EPM and IPM can actuate the internal endoscope to perform the translation motion and pan motion. And the flexible link can provide additional DOF to cover sufficient FOV for the surgeon in MIS. The force situation and kinematics are modeled based on the magnetic dipole theory and flexible mechanism. Preliminary experiments were conducted to validate the feasibility and clinical potential of the system. In the future, more experiments and tests need to be done to verify the accuracy of the modeling and the practicability in real clinical application. Also, the control algorithm will be developed to realize the visual servoing task which can help doctors perform the surgery more conveniently.

ABSTRACT. SciKit-Surgery provides open source libraries to support research and translation of applications for augmented reality in surgery. This paper discusses recent developments in SciKit-Surgery and case studies of SciKit-Surgery's use for applications such as SmartLiver, SciKit-SurgeryBARD and SciKit-Surgery's use to support research into visualisation and user interface design for augmented reality in surgery.

The availability of high quality software tools for research and translation is a key enabler for scientific progress. Research into surgical robotics, image guided surgery, and augmented reality for surgery brings together many disciplines and depends on a strong engineering base to provide the tools that researchers need (hardware interfaces, data management, data processing, visualisation, user interfaces). SciKit-Surgery was conceived as a more accessible replacement for existing toolkits written predominantly in C. Experience has taught us that whilst implementations in C could be robust and offer optimised performance, the need to learn the language and the difficulties of maintaining cross platform compilation presented too high a barrier to entry for the vast majority of researchers.

ABSTRACT. Wireless miniature soft robotic devices made of compliant and responsive materials have offered new interventional opportunities in hard-to-reach regions. Although various locomotion abilities have been achieved and studied, rare functions other than drug delivery have been properly incorporated into these soft machines. Our group has recently developed a medical system with a stent-shaped magnetic soft device (Stentbot) and the associated spatial magnetic actuation setup, which has accomplished the on-demand local drug delivery and the proof-of-concept flow diversion. The device offers a potential solution to clinical migrations or misplacements of stent-like structures. However, the most state-of-the-art single-material development compromises the performance in locomotion abilities and the efficacy of flow diversion. To overcome such a challenge, we propose a multi-material construction principle that provides versatility in design and fabrication. Based on the quantified modeling results, a prototype has been developed by leveraging different materials, which has demonstrated effective flow diversions among bifurcating structures. Further optimization of the design, material selections, and fabrication techniques will improve the working capacity of the device and enhance its real-world utilities to treat various diseases in the distal vasculatures.

ABSTRACT. The application of video see-through augmented reality (VST-AR) navigation in laparoscopic surgery helps visualize key anatomical structures that are hidden, which can enhance the surgeon's intraoperative perception and improve the safety of the surgery. This paper presents a VST-AR navigation system based on virtual markers and SLAM to solve the problems in practical applications, such as the lack of camera motion tracking, the reliance on manual registration of virtual 3D models, and invasive markers. The concept of virtual markers is introduced to achieve registration of the virtual 3D model to the real scene. SLAM-based technology enables laparoscope pose tracking and simultaneous updating of the virtual laparoscopic viewpoint through feature points of the patient's body surface. The method allows a multi-view perspective display of virtual 3D anatomical structures without invasive marker points. Quantitative assessment on performance of the proposed navigation system is performed by surgeons on a gallbladder model to determine its accuracy and potential for clinical application. Experimental results show that the system can locate key anatomical structures. The proposed navigation system enables effective guidance of key anatomical structures and improves the accuracy and success rate of laparoscopic procedures.

ABSTRACT. Direct thermal tissue measurements could prove invaluable in laparoscopy and laparo-assisted robotic surgery, where bipolar electrocoagulation or ultrasonic energy are often used to achieve haemostasis to maximise a clear view of the surgical field. We present a novel endoscope prototype for minimal invasive surgery: it integrates full stereoscopic vision and 3D-mapped, direct thermal measurements to evaluate the heat propagation over the surface target tissue during bipolar coagulation. The precise mapping of the multi-spectral images allows clinicians to quickly assess the risk of damage to sensitive tissues intra-operatively.

ABSTRACT. In anterior and posterior cruciate ligament (ACL and PCL) reconstructions, the intraoperative joint condition is different from preoperative CT due to the knee flexion applied during surgery. This study aims to provide non-invasive augmented reality (AR)-based surgical guidance for ACL and PCL reconstruction using the correlation model between knee surface and internal bones (femur and tibia). First, We defined individual finite element models for the knee surface, femur, and tibia. The correlation model is implemented by introducing mapping between finite element models. The internal bones move according to the shape deformation of the knee surface using the correlation model. Finally, registration between the depth data of the knee surface and the correlation model is performed to visualize AR. The magnitude of constraint forces is determined based on the registration result to deform the correlation model. The proposed method was evaluated with the flexion state knee CT and demonstrated successful AR compensating the knee movements.

ABSTRACT. Robotic navigation systems for radio frequency catheter ablation (RFCA) have been used effectively in clinical practice. Compared to manual navigation, such systems facilitate the operation and reduce radiation exposure for both the patient and the operator. Furthermore, they allow for higher contact forces resulting into more effective ablation lesions. However, if excessive contact force is applied it may lead to higher risk of cardiac perforation. To ensure high effectiveness and low complication risk in next-gen robotic navigation systems, tissue heat distribution should be taken into account. In this work, a meshless computational model for lesion prediction during robotic navigation assisted ablation is proposed. The model accounts for non-zero initial conditions and time dependent boundary conditions to simulate multi-site ablation. The meshless Fragile Points Method (FPM) is employed for the numerical solution of the model to ensure its suitability for clinical application, since FPM does not require the definition of a mesh. Simulations with two ablation sites and different catheter angles during ablation are performed for a 3D block of ventricular tissue. The proposed model has the capacity to predict lesion formation effectively by taking into account the heat accumulation of previously ablated sites of a multi-site ablation. Predicted lesions by the proposed multi-site ablation model are compared with lesions obtained from conventional single-site ablation simulation. It is demonstrated that if multi-site ablation is not considered, the lesion characteristics can be underestimated by up to 24.4%. Simulating the multi-site ablation conditions, transmural lesions may be obtained without excessive contact force. As a result, the cardiac perforation risk of the robotic navigation system may be reduced.

ABSTRACT. Obstetric ultrasound (US) is widely used in prenatal diagnosis to monitor the development and growth of the embryo or fetus and to detect congenital anomalies. The benefits offered by US in terms of timely diagnosis are extensive, but the quality of the examination is closely linked to the experience of the clinician [1]. Although proper training and assessment of acquired skills are considered of paramount importance in order to ensure a quality exam, there is no European standard establishing a training path with an objective assessment of operator’s capabilities. In fact, the experience is often evaluated merely on the basis of the number of clinical tests performed. However, an operator with daily US examination experience may not perform as well as a true expert due to inadequate training [2]. Many studies have studied gesture with the aim of establishing methods to discriminate between experts and beginners, which can also be used to study a specific training and objectively evaluate the acquired skills [4][5]. Inspired by these works and with the same aim, hand movement was also studied for fetal US [6]. This paper presents a study for the objective assessment of the operator’s experience in real obstetric US examinations based on hand gestures and forces applied with the US probe on the abdomen. A Data Recording System was designed to collect this information during real examinations performed by clinician with 3 different levels of experience (expert, intermediate and beginner) on pregnant women at the 2nd trimester. The results presented here focus on assessing a set of metrics with potential to provide an objective discrimination of the operator’s level of experience. This study was approved by the Regional Ethics Committee of Liguria (Italy) with the protocol number 379/2022 - DB id 12369.

ABSTRACT. In this work we propose an autonomous dual-arm system for needle insertion and central venous access. The system is composed of two Franka robotic arms that are precisely co-registered and collaborate to achieve accurate needle insertion by combining ultrasound and bioimpedance sensing to ensure robust deep vessels visualization and venipuncture detection. The proposed system performance is evaluated on a phantom trainer through experiments simulating the jugular vein access for cardiac catheterization purposes. Quantitative results show the system is able to localize the vein and perform autonomous needle insertion with high accuracy and placement error below 1.7mm, proving the potential of the technology for real clinical use.

ABSTRACT. During orthopaedic procedures, a steerable bone drill would facilitate the drilling of curved tunnel making it possible to reach difficult targets, prevent damage to surrounding tissue, or improve the fixation strength of bone anchors. This study presents the design of a flexible bone drill (Ø4 mm) that uses a hydraulic pressure wave that is transferred through a flexible, fluid-filled tube to drill through bone. The impulse is generated in the handle by a spring and is then transferred through the flexible fluid-filled tube to the hammer tip at the distal end of the drill. Seven hammer tip designs were evaluated on a bone phantom to determine the effect of tip shape on the penetration rate through bone. Although there was no clear effect of hammer tip shape on penetration rate, it was noted that blunt hammer tips accumulated dense bone material at their tip which could impede drilling performance when drilling longer tunnels. The flexible bone drill was able of effectively transmitting an impulse through the flexible tube in straight and curved orientation (45o and 90o), however, the efficiency of impulse transfer could be improved. The presented flexible bone drill design, which uses a hydraulic pressure wave to drill thorough bone, is a first step in the development of a steerable bone drill that would allow the surgeon to adjust the drilling trajectory during orthopaedic procedures.

ABSTRACT. Microsurgery is a particularly impactful yet challenging form of surgery. Robot assisted microsurgery has the potential to improve surgical dexterity and enable precise operation on such small scales in ways not previously possible. Intraocular microsurgery is a particularly challenging domain in part due to the lack of dexterity that is achievable with rigid instruments inserted through the eye. In this work, we present a new design for a millimeter-scale, dexterous wrist intended for microsurgery applications. The wrist is created via a state-of-the-art two-photon-polymerization (2PP) microfabrication technique, enabling the wrist to be constructed of flexible material with complex internal geometries and critical features at the micron-scale. The wrist features a square cross section with side length of 1.25 mm and total length of 3.75 mm. The wrist has three tendons routed down its length which, when actuated by small-scale linear actuators, enable bending in any plane. We present an integrated gripper actuated by a fourth tendon routed down the center of the wrist. We evaluate the wrist and gripper by characterizing its bend-angle. We achieve >90 degrees bending in both axes. We demonstrate out of plane bending as well as the device’s ability to grip while actuated.

Our integrated gripper/tendon-driven continuum design and meso-scale assembly techniques have the potential to enable small-scale wrists with more dexterity than has been previously demonstrated. Such a wrist could improve surgeon capabilities during teleoperation with the potential to improve patient outcomes in a variety of surgical applications, including intraocular surgery.

ABSTRACT. In robotic surgery, technical skill is a major challenge for surgeons and trainees. Although research in surgical skill assessment has made considerable progress, most of the methods are post-operative analysis, and only a few methods are intra-operative or real-time analysis. In this study, we introduce a method to extract the unusual movements which are rarely seen in Experts and identify the types of the unusual movements using IMU data and a semi-supervised learning approach. Our previously collected dataset includes 12 subjects (3 Experts: EX, 2 Fellows: FL, 3 Intermediates: IN, and 4 Novices: NO) performing surgical training tasks on a da Vinci Simulator. We used the data from Experts to train an Autoencoder to reconstruct the input data. An unusual movement will show a larger reconstruction error - a data pattern that Autoencoder has never seen from EX. NO showed significantly higher number of unusual movements and average error of the unusual movements than EX, FL, and IN, indicating NO had the worst performance, thus validating our method. Then we identified the types of unusual movements using clustering. We could see large wrist flexions, extensions, and deviations in these unusual movements. In this study, we introduced a method to extract the movements which were not observed in EX data, and we found that the use of wrist is a good indicator of surgical expertise level. We will seek for methods to help surgeons avoid these behaviors.

ABSTRACT. Deep learning classifiers have demonstrated their ability to provide robust accuracy for the treatment of combined signals including electroencephalography (EEG) and functional near infrared spectroscopy (fNIRS) [1, 2]. In this work, an evolutionary deep learning strategy is applied to classify different cognitive workload states that surgeons experience during laparoscopic surgery. The proposed learning strategy is applied to train an Evolutionary Multilayer Perceptron Neural Network (E-MLPNN), where multimodal raw data of EEG, fNIRS and Electrocardiogram (ECG) signals were collected and concatenated from a series of ten experiments using the back-end platform Multi-sensing AI Environment for Surgical Task & Role Optimisation (MAESTRO) as shown in Figure 1(a). Each experiment required surgical trainees to perform a simulated laparoscopic cholecystectomy (LCH), i.e. the removal of a gallbladder in a porcine model using a minimally invasive surgical technique as demonstrated in Figure 1(b). At each experiment, the level of Cognitive Workload (CWL) is assumed to increase as the mental activity increases during the surgical operation. As presented in Figure 1c, a number of tasks performed during the LCH were defined to measure the level of CWL.

ABSTRACT. Soft robots have exhibited excellent compatibility with functional and physical requirements of intraluminal procedures such as bronchoscopy and cardiovascular intervention. Despite their favourable mechanical compliance and scalable design, direct force and shape sensing have proved difficult to be embedded within the soft robot's structure. Also, the rate-dependency of soft sensors requires derivative-based calibration that amplifies data acquisition noise leading to large inaccuracy, especially at small forces. As an alternative method, in this study, we proposed a transfer learning-based calibration schema inherited from GoogLeNet. The proposed method was derivative-free and would capture temporal changes in electrical signals from the soft sensors by capturing image features in scalograms of wavelet transform. WaveLeNet, our derivative-free deep convolutional calibration model, had comparable accuracy over the full range of our soft flexural sensor (<5% error) compared to a previously validated rate-dependent calibration but substantially improved accuracy for small forces (<20mN).

ABSTRACT. Surgical tool detection and localisation during robotic assisted surgery is of significant importance. The estimated 3D pose can be subsequently employed in different processes, such as optimising the interaction of tools with registered patient tissue or allowing for motion planning of a robotic end effector, thus avoiding collisions with external agents, and ensuring patient safety. Traditional tracking systems, namely robotic systems, or optical trackers, usually suffer from high costs and, especially in the case of trackers, the need for tool redesign to incorporate trackable markers. Therefore, recent research has focused on image-based, markerless detection and localisation techniques. This paper presents a network capable of both surgical tool detection and 3D pose estimation using a monocular system. For the purposes of training and testing of the network, a novel dataset of 5370 images of “off-the-shelf” surgical tools in action was produced, namely a scalpel, a pair of scissors, a pair of forceps, and an electric burr. Upon testing, the network obtained a mean DICE coefficient of 85.0% for detection. Furthermore, position and orientation in 3D were predicted, with the network achieving an average position error of 5.5mm and an average orientation error of 3.3̊. The presented network exhibits a high level of versatility compared to the state of the art, as it does not requires any prior knowledge of the tool 3D structure, such as CAD information. Upon comparing the results with standard pose estimation techniques with the same dataset, most of the metrics exhibited lower errors when compared to their counterparts.

ABSTRACT. The increasing prevalence of prostate cancer has led to the widespread adoption of Robotic-Assisted Surgery (RAS) as a treatment option. Sentinel lymph node biopsy (SLNB) is a crucial component of prostate cancer surgery and requires accurate diagnostic evidence. This procedure can be improved by using a drop-in gamma probe, SENSEI system, to distinguish cancerous tissue from normal tissue. However, manual control of the probe using live gamma level display and audible feedback could be challenging for inexperienced surgeons, leading to the potential for missed detections. In this study, a deep imitation training workflow was proposed to automate the radioactive node detection procedure. The proposed training workflow uses simulation data to train an end-to-end vision-based gamma probe manipulation agent. The evaluation results showed that the proposed approach was capable to predict the next-step action and holds promise for further improvement and extension to a hardware setup.

ABSTRACT. Image-guided procedures have experienced a rapid increase in popularity in recent years. The advancements in medical imaging technology have led to a shift in medical images from being used primarily for diagnosis to being a critical tool in theragnostic and therapeutic procedures. This shift has resulted in the emergence of new fields such as interventional radiology (IR), therapeutic endoscopy (TE), and minimally invasive image-guided surgery (IGS), with an increasing number of professionals adopting these techniques in their clinical practices due to improved outcomes [1]. One of the most widely used imaging methods in these procedures is X-ray-based imaging, including computed tomography (CT), 2D C-arm fluoroscopy, and cone-beam CT scans. These procedures typically require the use of contrast agents (CA) to visualize soft tissues with high definition and contrast. However, the use of CA presents several challenges, including the limited volumes that can be used and the toxicity of the agents when they are injected intravascularly [2]. The CA also follows the patient’s hemodynamics, leading to transient visualization and asynchronous image guidance. In this paper, we aim to address the technical issues related to contrasted X-Ray images in image-guided therapy. We propose a deep learning approach that will allow for the visualization of vessels during image-guided procedures without the need for contrast agents, making these procedures safer, and more efficient, while providing real-time guidance.

ABSTRACT. This paper presents a comparison of denoising methods for low-field synthetic neurological MRI. The study included 44 subjects, and the results suggest that BM3D may be an effective tool for sufficiently denoising over a range of SNR values, which has implications for quantifying denoised synthetic results. The four-step process used to determine image quality consisted, first, of collecting High Field (HF) MRI datasets from open-source databases; second, of performing a linear transformation to Low-Field (LF) MRI datasets; third, of adding random noise to medial slices of both the original HF images and the converted LF images; finally, of comparing the differences between them by measuring image quality. Moreover, this work is useful to experiment with different configurations of medical devices from open source image databases without costly clinical trials.

ABSTRACT. Medical imaging is an essential tool for diagnosing and monitoring various health conditions. Robotic remoti- zation of such diagnostic medical procedures increases the safety of medical personnel and accessibility to the rural populace. However, ultrasound exams can be challenging and require skilled operators to obtain high- quality images. Automating such procedures requires programming robots to perform these dexterous medical skills. The programming constraint can be eliminated by leveraging human tutelage paradigms, enabling the robot to learn from observation and expert feedback. But, robots require massive libraries of demonstrations to learn effective policies using machine learning algo- rithms [?]. While such datasets are achievable for simple tasks, providing many demonstrations for contact-rich procedures such as ultrasound is not practical. This paper presents a novel method to learn complex contact-rich procedures by combining self-supervised practice with sparse expert feedback through coaching. The robotic ultrasound system (RUS) uses reinforcement learning (RL) to learn a policy for autonomous imaging of a urinary bladder phantom. Specifically, we use an off-policy soft actor-critic with a reward based on image quality assessed using a supervised neural network to learn the policy for ultrasound through practice. In addition to practice, experts provide online corrective feedback (coaching), which drives the robot to learn successful policies for ultrasound imaging. We show that leveraging expert feedback results in significantly increased performance than using a state-of-the-art RL policy.