ABSTRACT. Surgical skills assessment is a crucial step to help understanding surgical expertise and to provide technical knowledge to beginners. Scores, such as GOALS, have been designed to assess surgical skills. However, these scores are subjective and need experts to compute them. With the advent of robotic surgery, it is possible to compute Automated Performance Metrics (APMs) based on the motion of robotic arms to assess surgical skills. Several studies have demonstrated statistically significant differences between APMs from different levels of expertise. The majority of these studies performed a global analysis, i.e., studying the surgical procedure or training task as a whole. By using the Surgical Process Model (SPM) methodology, it is possible to describe the surgery at different levels of granularity and break it down into a sequence of elements. In this paper, we combine SPM and APMs to study global and local kinematic skills during robotic-assisted hysterectomies. Fifty-two robotic-assisted laparoscopic hysterectomies performed by expert and intermediate surgeons have been annotated at the phase level and the kinematic data have been synchronized and split according to these annotations to obtain kinematic data for the complete duration of the surgeries (global sequences) and for each surgical phase (local sequences). We computed 16 APMs for the two main robotic arms for each global and local kinematic sequence. Even if the global analysis allows surgical skill assessment with 4 statistically different APMs. The local analysis provided more information (15 APMs are significant for at least one phase) and some of them can be explained clinically.

ABSTRACT. Epidural analgesia involves injection of anesthetics into the epidural space, using a Touhy needle to proceed through the layers in the epidural region and a “loss of resistance” (LOR) syringe to sense the environment stiffness. One of the leading causes of the two most common complications of epidural analgesia - accidental dural puncture and failed epidural – is the anesthesiologist’s case experience. Robotic simulation is an appealing solution to help train novices in this task. Another benefit of it is the ability to record the kinematic information throughout the procedure. In this work, we used a haptic bimanual simulator to explore the effect LOR probing strategies had on procedure outcomes. Our results indicate that most participants probed more with the LOR syringe in successful trials, compared to unsuccessful trials. Furthermore, this result was more prominent in the three layers preceding the epidural space. Our findings can assist in creating better training protocols for epidural analgesia. Based on our findings, we expect that instructing anesthesia residents to use the LOR syringe more extensively and educating them to do so more when they are in proximity to the epidural space can help improve skill acquisition in this task. Future studies are needed to test this prediction and develop an optimal training approach.

ABSTRACT. In spine surgery, e.g., pedicle screw placement, patients are in the prone position, anesthetized, and breath using respirators. Respiration-induced spine movements may interfere with the operation. For evaluating spine surgery robotic systems without resorting to in-vivo experiments, it is necessary to provide a setup that realistically reproduces the spine movement in a lab. The literature is not very rich in documenting such a movement. In this study, we measured respiration-induced spine movements (T6-L5) in an open-spine surgery on a pig. A mechanical probe mounted on a force-controlled medical robotic arm was used for measurements. The range of motion (Mean ± 2SD) along the Z-axis (vertical, 3 mm ± 1.4 mm) is the most significant compared to the X- and Y-axis (0.4 ± 0.3 mm and 0.3 ± 0.2 mm, respectively). Then, we proposed ex-vivo experimental setups which could implement the measured motions to emulate the respiration-induced movement.

ABSTRACT. Up to half of the technical errors made by surgical trainees result from improper tool forces on tissue. This skill inadequacy is exacerbated in robotic minimally invasive surgery (RMIS) due to the perpetual technical barriers prohibiting robust haptic (touch) sensations in clinical RMIS systems. RMIS experts have developed visual-haptic acuity – the ability to visually estimate absent haptic sensations – through years of surgical practice on real patient tissue. For current RMIS trainees, limitations on working hours and caseloads severely constrain practice with real patient tissue. Given that skills gained in virtual training do not always transfer to the clinical setting, there is a critical need for improved visual-haptic acuity development. Previous research showed that haptic feedback provided during RMIS training helped trainees increase speed and accuracy, but these benefits have not been demonstrated beyond basic simulated training environments. Additionally, we lack objective methods for assessing trainees’ ability to visually estimate haptic sensations when operating on real patient tissue. For this reason, we are developing a modular data acquisition and multimodality haptic feedback system to catalyze visual-haptic acuity development for RMIS trainees. We expand on prior work, which demonstrated the efficacy of this system using an Intuitive da Vinci Si clinical robot, to better understand how it may be used for objective assessment of visual-haptic acuity during RMIS training, and real-time haptic feedback based on these assessments. We recorded surgical interactions between sensorized da Vinci instruments and an ex vivo porcine bowel as an intermediate and expert surgeon completed a simulated bowel running and knot tying task. Results highlight the difference in various kinetic metrics between intermediate and expert performance. When coupled to our haptic feedback devices, these measures can be used to both assess and augment tissue handling skill during training, which we envision will catalyze visual-haptic acuity development.

ABSTRACT. Medical robotics, particularly image-guided robotic systems, have revolutionized the surgical field by improving precision and accuracy. 3D Slicer\cite{pieper20043d}, an open-source platform, has become a crucial tool in this field as it allows for visualization, processing, and registration of 2D and 3D medical imaging data, making it an essential component in current research in robotic intervention\cite{he2013new}~\cite{mitchell2007development}. However, there is a missing component in 3D Slicer - a native physics engine for simulating the interaction of a robot with the anatomy. AMBF\cite{ambf}, an open-source software, was designed to address this issue by simulating the kinematics, dynamics, and interaction of complex surgical robots. By integrating 3D Slicer and AMBF using Robot Operating System (ROS), we can empower researchers to utilize both the extensive capabilities of 3D Slicer for visualization, processing, and registration of medical imaging data, and the physics-based constraint of AMBF for simulating the interaction of a robot with the anatomy. By combining these two platforms, researchers will have a comprehensive tool to study and develop projects in medical robotics, ultimately contributing to the advancement of the field.

ABSTRACT. Minimally invasive surgeries, such as laparoscopy, have improved patient safety and decreased recovery times. However, limited instrument dexterity and insufficient sensor feedback can adversely affect the outcomes of procedures. Soft optical sensing strategies are rapidly developing for soft robotic systems as a means to increase the controllability of soft compliant robots. Multiple optical sensors can be embedded in robotic systems to achieve both proprioceptive and exteroceptive capabilities. In this paper, we present a roughness tuning strategy for the fabrication of soft optical sensors to achieve the dual functionality of shape sensing combined with contact recognition within a single multi-modal sensor. The molds utilized to fabricate the soft sensors are roughened via laser micromachining to exhibit a distinctly bidirectional sensor response (optical gain and optical loss in opposite directions). We demonstrate the integration of these sensors into a fully soft robotic platform consisting of a multi-directional bending module with integrated 3D shape sensing and a gripper with tip position monitoring along with contact force prediction. The accuracy of our sensing strategy is shown in validation experiments and an in-vitro experiment is conducted in a mock laparoscopic environment to exhibit our robot’s functionality in a surgical scenario. Our robot can perform a peg transfer test in an in-vitro environment while monitoring the shape of the robot and the force exerted on objects of varying sizes and outputting the information to a surgeon via a graphical user interface.

ABSTRACT. Worldwide, millions of people are diagnosed with muscle-invasive urinary bladder cancer every year. The gold standard treatment consists in the surgical removal of the entire bladder and the urethra, followed by the re-direction of urine flow from the kidneys through the native ureters to an external collection bag, or the reconstruction of a neobladder with intestinal tissue. The risk of tumor recurrence, as well as the patient’s low quality of life are among the main limitations of such procedures. Although some artificial bladder systems have been proposed, the design of a fully implantable solution is still an open engineering challenge. Here we propose a novel sensorized hexagonal-shaped bladder allowing urine collection and fullness sensitivity restoration. The foldable structure features an origami structure, which allows to obtain a small encumbrance when empty, and a shape similar to the native bladder when the urine volume approaches the capacity of 250 ml. The integrated sensing system exploits the geometry of the structure and consists of eight thin textile sensors, whose resistance changes according to their stretching. By applying the sensors to the foldable walls of the bladder, it is possible to derive the opening degree of the folds during filling. The latter information is provided as input to a custom algorithm to enable the 3D reconstruction of the bladder and subsequent volume estimation. Despite the tendency to underestimate the volume, the system showed good results with an average volume estimation error of approximately 25 ml. Although prototypal, the authors presented a new robotic organ that paves the way towards fully implantable solutions for the lower urinary system.

ABSTRACT. Early lung tumors often appear in peripheral regions of the lung, making them difficult to access using traditional minimally invasive methods, such as bronchoscopy. Robotic solutions must be highly dexterous and miniaturized to reach peripheral lung regions. These properties can be found in soft robots making them ideal candidates for advancing bronchoscopic biopsy procedures. However, at the scale necessary for interventional bronchoscopy, the robot’s ability to transmit force is significantly limited. To address this limitation a soft robot with embedded steering, stabilization, and needle deployment capabilities is proposed to improve lung tissue biopsy procedures. Steering is accomplished via a fluidic bending actuator embedded into a continuum body. A radially-expansive actuator is also integrated into the continuum body to stabilize the robot within branches of the lung before taking a biopsy. An origami-inspired, bellows actuator deploys the needle from the tip of the robot once it has navigated to and stabilized at its target destination. The design and fabrication of these fluidic-actuated degrees of freedom enable the soft robot to maintain an overall diameter of 3.5 mm. At this scale, stabilization increases the robot’s effective stiffness by anchoring the robot to the surrounding anatomical tissue. Characterizations demonstrate that this effect increases the amount of force transmission through the robot’s tip. Needle deployment also demonstrates the ability to produce sufficient force when puncturing a tissue simulator. The soft robot is further evaluated with in-vitro experiments. Steering, stabilization, and needle deployment are used in sequence to propose a workflow in which the robot can reach the target region and successfully puncture the target tissue for biopsy.

ABSTRACT. The diaphragm is a critical muscle for the respiratory system, responsible for up to 70% of the inspiration effort. Phrenic nerve trauma or neuromuscular disease can generate severe diaphragm dysfunction that ultimately leads to respiratory failure. The current treatment for patients with severe diaphragm dysfunction is permanent airway tethering to mechanical ventilation, which greatly impacts patient’s quality of life and autonomy by hindering activities like speech, swallowing, and mobility. Soft robots are ideal to assist in complex biological functions like the contraction of the diaphragm. Diaphragmatic mechanical assistance using implantable soft robots has shown promising results in restoring respiratory function. However, the design of the soft robotic actuator can be optimized to effectively assist the diaphragm. Here, we present a soft robotic pneumatic actuator that inverts its curvature to efficiently displace the diaphragm and assist in the inspiratory effort, restoring physiological thoracic and abdominal pressurization levels. Moreover, we show how the respiratory simulator can replicate clinically relevant pleural pressure and abdominal pressure, demonstrating its potential as a platform to validate this technology.

ABSTRACT. This paper presents a computational model for predicting the shape of fully soft continuum magnetic robots (FSCMRs) under different actuation scenarios, including permanent magnets (PM) and electromagnetic fields (EM). The accuracy of the model is validated using experimental data, and the results show that the model has acceptable accuracy in predicting the shapes and tip positions of FSCMRs under hybrid actuation systems. The study demonstrates the potential of the model as a predictive tool for guiding soft magnetic robots in more versatile clinical applications. The experimental setup includes an external magnetic field generator, two robotic arms, a camera, and a computer for monitoring. The FSCMRs have a diameter of 600 µm and a length of 25 mm, and the test bed and FSCMR holder are 3D-printed. The simulation is performed using COMSOL Multiphysics software, and the mechanical properties are introduced from previous work. The simulated magnetic fields match three actuation scenarios: one PM, two PMs, and a hybrid system consisting of one PM and the EM system. The suitability and accuracy of the proposed numerical platform are assessed based on the shape classification of the FSCMR and the tip position. The results show that the platform has acceptable accuracy in predicting FSCMR shapes and tip positions under hybrid actuation scenarios, making it a useful predictive tool for guiding soft magnetic robots in clinical applications.

ABSTRACT. The field of laparoscopic surgery shows tremendous promise for improving patient outcomes and reducing the duration of hospital stays. However, the minimally invasive nature of these keyhole surgeries deprives surgeons of critical tactile feedback, which increases the chance of accidentally inflicting trauma upon tissues. We propose a soft robotic laparoscopic grasper designed with the primary objective of minimizing the forces needed to manipulate intestinal tissues, thus mitigating the likelihood of injuring the patient. The atraumatic nature of this device is achieved through a combination of inherently soft and compliant pneumatic actuators deployed via an expandable mechanism, an array of elastomeric force sensors, and an auxiliary suction functionality to establish an initial grip on targeted tissues without using a traditional "pinching" mechanism. This aggregation of features allows surgeons to circumferentially envelop intestinal segments in an atraumatic manner, thereby affording operators a reliable hold on abdominal tissues without the risk of causing damage. Testing has shown that the proposed grasper can hold an intestine phantom made from silicone with an attached 500g weight, as well as explanted porcine small intestine, and the soft sensors have demonstrated high accuracy when measuring contact forces in three dimensions.

ABSTRACT. This article presents an inflatable robot which is largely made of flexible plastic film. In the deflated state, it is easy to collapse, fold and roll into a small size, promising easier delivery by a carrier endoscope to the proximal colon. After reaching the target site, the robot is inflated and the surgery is performed.The robot is driven by a novel actuator, which uses a folded chamber to pull the cable to produce a large displacement without the need of displacement amplification mechanisms.

ABSTRACT. Capsule robots have the potential to provide untethered access to the gastrointestinal tract and perform simple tasks that could reduce invasiveness and provide a better alternative method of access than an endoscopy or colonoscopy. Current state-of-the-art for capsule robots already fulfil the need for inspection, but there is a gap that exists between the capabilities of current capsule robots when compared with those of endoscopic surgical robots. For example, strictures occurring throughout the gastrointestinal tract due to inflammation or Crohn's disease needs a source of pressure to break the strained organ and unblock the stenosis. Additionally, access to the distal parts of the small intestine remains difficult even for Endoscopic Balloon Dilation (EBD). A capsule robot with an actuator that can provide volumetric expansion could fulfil both of these needs, i.e., opening the lumen at a stricture site, anchoring for surgical procedures in difficult-to-access GI areas. This paper presents a capsule robot prototype with a soft internal actuator capable of providing wireless volumetric expansion. The inflation of the capsule is based on the chemical reaction between bicarbonate and citric acid, which releases carbon dioxide gas. The inflation of the internal actuator is wirelessly controlled through magnetic induction which generates thermal energy. Mechanisms that can provide wireless expansion have been previously presented. However, the method of actuation should not require operating at temperatures that can cause permanent tissue damage. In the current work the generated thermal energy is below limits of hyperthermia. The chemicals and the dissolution medium used are all safe for ingestion, making them suitable for gastrointestinal applications, and the capsule design provides a novel and promising alternative for ballooning operations without using electronics or a battery.

ABSTRACT. Colonoscopy is considered to be the gold standard for the detection of colorectal polyps and other colonic disease. In this study, a novel growing soft-continuum robot is presented as a proof of concept for a potential colonoscopy application. Compared to the state-of-the-art systems such as the ColonoSight system using an inflated balloon attached over the colonoscope shaft, or Neoguide actuated via electromechanical actuators, the proposed architecture utilizes the advantage of pneumatic propulsion as well as allowing control in three degrees of freedom. The growing robot concept pushes the end effector from the tip and prevents loss in propulsion force at the tip that results a lower mechanical forcing on the colon. Since the actuation is performed pneumatically and controlled via electromechanical interface, there is no need for users to apply high forces for pushing/pulling to progress and maneuver the robot.

ABSTRACT. In drug delivery, minimizing off-target accumulation in healthy areas is crucial to prevent toxicity and other complications. To achieve this, drug delivery platforms must be designed to either localize active compounds to the target site or selectively activate the portion that arrives in the targeted tissue. Living bacterial therapeutics can be equipped with sensors to accumulate in target regions, but low administrable doses and biological barriers limit their effectiveness. Magnetotactic bacteria (MTB) can respond to magnetic fields, making them potential externally controllable drug carriers. We previously showed that a rotating magnetic field enhances tumor colonization. Here a magnetostatic selection field is integrated into the control scheme which improves spatial selectivity and prevents off-target accumulation, leading to enhanced targeting of living microrobots in a 3D cancer model.

ABSTRACT. Surgery is a mentally demanding task that is focused on patient safety and requires the precise execution of motor control and decision making in a timely manner. Episodes of high Cognitive Workload (CWL) induced by stressors or distractions have been shown to lead to inferior performance potentially compromising patient safety. We have proposed a promising CWL assessment platform utilising a wide range of physiological sensors. However, there are some disadvantages associated with a complex multimodal sensing design, including high device cost, long set up time and the discomfort caused by wearing multiple wearable sensors for long periods during surgery. To address this problem, the proposed one-dimensional convolutional neural network (1D-CNN) model discussed here, offers an alternative solution to recognising CWL states, achieving satisfactory performance (91.2% accuracy) with the use of a wireless ECG sensor alone, showing great potential for widespread deployment in the operating room (OR).

ABSTRACT. Supervisory methods in shared control allow a dynamic adjustment of the level of autonomy in a surgical robot based on the current task demands and the capabilities of the human operator. Several benchmarks tasks are available to evaluate the performance of these controllers, however the operator can struggle with the task due to inexperience or limited environmental information. In this paper, we propose an admittance control strategy based on guidance priority adaptation to enable a human operator to assume a supervisory role during one-handed peg transfer task. We implement an epsilon-greedy maximum entropy inverse reinforcement learning (EG MaxEnt IRL) algorithm to enable an agent to control the surgical tool in a virtual environment while the human supervises the procedure. We successfully implement the proposed method and observe that the supervisory method can be further improved with a cooperative control, specifically a segmented control.

ABSTRACT. In neurosurgical procedures maximizing the resection of tumor tissue while avoiding healthy tissue is of paramount importance and a difficult task due to many factors, such as surrounding eloquent brain. Swiftly identifying tumor tissue for removal could increase surgical outcomes. The TumorID is a laser-induced fluorescence spectroscopy device that utilizes endogenous fluorophores such as NADH and FAD to detect tumor regions. With the goal of creating an endoscopic tool for intraoperative tumor detection in mind, a study of the TumorID was conducted to assess how the angle of incidence (AoI) affects the collected spectral response of the scanned tumor. For this study, flat and convex NADH/FAD gellan gum phantoms were scanned at various AoI (a range of 36 degrees) to observe the spectral behavior. Results showed that spectral signature did not change significantly across flat and convex phantoms, and the Area under Curve (AUC) values calculated for each spectrum had a standard deviation of 0.02 and 0.01 for flat and convex phantoms, respectively. Therefore, the study showed that AoI will affect the intensity of the spectral response, but the peaks representative of the endogenous fluorophores are still observable and similar. Future work includes conducting an AoI study with a longer working-distance lens, then incorporating said lens to design an endoscopic, intraoperative tumor detection device for minimally invasive surgery, with first applications in endonasal endoscopic approaches for pituitary tumors.

ABSTRACT. Augmented reality (AR) is becoming essential in several surgical specialities. Fusing the patient-specific preoperative planning information, typically 3D models extracted from CT scans or MRI, on the endoscopic images, it allows the surgeons to have thorough and detailed knowledge of the anatomical structure of the surgical target intra-operatively. An AR system’s first step consists of an initial rigid registration of the 3D models on the surgical image. Most of the approaches are manual or semi-automatic. While the latter exploits physical landmarks on the tissue or 3D surface features, the manual approach requires Human-Computer Interfaces (HCI) to manipulate the models. The choice of the HCI is fundamental in order to develop an easy-to-use application with a slight learning curve and supporting the surgeon. In fact, the way the manual registration is performed is strictly connected to the users’ physical and mental stress, influencing their final performances. This paper presents a human-centred usability study that aims to evaluate the most appropriate HCI for 3D model manipulation in AR surgical applications. A software interface was developed to control the 3D model manipulation using SpaceMouse ® and UltraLeap ® . Subjects were asked to perform the rigid manual registration of the 3D model of a kidney on the top of an image, showing the real kidney. Quantitative and qualitative evaluation was performed to select the HCI that minimizes the cognitive and physical load of the operators and maximize their performance.

ABSTRACT. Eversion-based mechanism has become increasingly popular in the field of robotics. This mechanism enable objects to turn inside out, similar to flipping a sock, allowing them to move through narrow spaces without making direct force on the environment. This type of movement can be useful for medical devices such as catheters and endoscopes, as it enables them to navigate through tight and curved spaces more safely. Feedback control can further enhance the accuracy and efficiency of the examination process.

In this study, an area-based method was introduced for calculating the total length of the eversion portion, which is typically controlled by a reel mechanism. The reel mechanism consists of a spool wrapped tightly with plastic tubing and connected to a motor. However, the diameter of the reel mechanism is varied by the number of plastic layers around the spool, making it challenging to calculate the total length using the standard roller model. Our method calculates the total length based on area and was validated using an optical tracking camera. The results show that our proposed method can predict the total length with a Root Mean Square Error of around 2 times, which is about 1.5% of the total length.

In conclusion, our study presents a promising solution for calculating the total length of eversion-based movements in robotics. The proposed method can reduce errors from the model and improve accuracy in medical examinations.

ABSTRACT. Due to its delicate subject matter and challenging operations, neurosurgery has always been in need of adapting new techniques and technologies. A neurosurgical procedure that could especially benefit from the use of such technologies is the Endoscopic Endonasal Transsphenoidal Surgery (EETS), a minimally invasive neurosurgical technique that is performed via an anterior sphenoidotomy and aims at the removal of sellar and parasellar lesions with the use of an endoscope and rigid instruments. Current standard instruments lack articulation limiting operative access and surgeon dexterity, and thus, could benefit from robotic articulation. In this study, a handheld robotic system with a series of detachable end-effectors for this approach is presented. This system is comprised of an articulated 3 degrees-of-freedom 3mm grasper with a modular design that allows for instrument expansion, its ergonomically designed handheld controller with a rotating joystick-body that can be placed at the position most comfortable for the user, and the accompanying control box. The robotic instruments were experimentally evaluated for their workspace, structural integrity, and force-delivery capabilities during a cadaver pilot study by a cohort of neurosurgeons with varied clinical experience. Results from this pre-clinical experimental procedure showcased enhanced dexterity and adequate robustness that could suggest feasibility in a clinical context, as well as improvement over current neurosurgical instruments.

ABSTRACT. Background and purpose: ProSpare is a self-insertable rectal obturator which can reduce motion error and anorectal dosimetry in prostate radiotherapy. It is being evaluated in the postoperative setting where 7/19 (37%) patients could not insert the device. Endorectal balloons are commonly used in prostate and post-prostatectomy radiotherapy and have good tolerance because they are inflatable. The aim was to create an inflatable device, like ERBs, which expands to become rigid and angled to act as a rectal stabiliser like ProSpare. Materials and methods: Devices were made using a custom laser welding system developed at the Hamlyn Centre, St Mary’s Hospital. This was followed by laser cutting and soldering to create the 3D shape. A three-sided and four-sided dual-truncated prism design and a three-sided reinforced actuator design were explored. Devices were inflated using a pneumatic compressor and their rigidity tested by tactile examination. Burst testing was followed by increasing the pressure 0.5 bar at a time on the pneumatic compressor. Minimum and maximum dimensions of the devices while folded/deflated and deployed/inflated were measured, respectively, to obtain expansion ratios by volume. Results: The three-sided dual-truncated prism design could not be folded but the four-sided device could. However, the four-sided device would push against the posterior anal wall during deployment. The reinforced actuator design was more successful: it could be folded and had an expansion ratio of 14.7 by volume and reached an angle of 20° on inflation. Its chambers started inflating, became rigid, and burst, when the pneumatic compressor reached 0.5, 1.0, and 1.5 bar, respectively. Conclusion: A device capable of inflating and deploying to become angled and rigid was created. The next steps include exploring ways to increase the angle, directly comparing rigidity to ERB with a force sensor, and adding steel discs so the device acts as an IGRT tool.

ABSTRACT. Surgical robots have revolutionized the minimally invasive surgery field by offering surgeons increased precision and dexterity. However, most existing training platforms require expensive and unwieldy control mechanisms, limiting their availability and convenience. The development of a low-cost and easily deployed control console can address these limitations, thereby potentially enhancing the effectiveness of robotic surgery training. This application can also be used to advance surgical robotics research in low-resource environments. This work aims to develop a novel mobile application that allows for the control of a surgical robot using a smartphone. 

The mobile application was designed and implemented to establish a wireless connection with the surgical robot system. A user-friendly interface was developed, offering intuitive controls and real-time feedback. The application's functionality includes controlling robotic arms and jaw angles. The app also includes a button that mimics the clutch of a da Vinci Research Kit (dVRK). A group of novices participated in the evaluation phase, performing movement tasks using the mobile application and the dVRK traditional control interface. Surgical performance metrics, including task completion times and accuracy, were measured and compared.

The evaluation revealed that surgeons using the mobile application exhibited comparable performance in terms of accuracy but not task completion time when compared to the traditional control interface. Nevertheless, we believe the application still has merit as a low-cost and readily available input device for surgical robotic training, as well as for facilitating medical robotics research in low-resource settings, as these are not time-sensitive use cases.

ABSTRACT. Breast-conserving surgery (BCS) is a surgical intervention for breast cancer where the surgeon resects the primary tumor and preserves the surrounding healthy tissue. These procedures have a high failure rate (over 30%) because it is often difficult to localize breast tumor boundaries intraoperatively. The NaviKnife system is a surgical navigation platform that was designed to address this challenge. Using electromagnetic navigation, the NaviKnife platform allows the surgeon to visualize the position of their resection tools relative to the tumor. One of the disadvantages of the NaviKnife platform, however, is that it is still heavily reliant on freehand guidance.

We hypothesize that haptic feedback, in the form of a virtual fixture, can further improve the NaviKnife system and consequently, the standard of care in BCS. In this paper, we describe the implementation of a virtual fixture system for BCS, using a recently published, open-source platform, SlicerROS2 and a haptic device called the Phantom Omni. We describe how to modify this programming platform for a new application, as well as the necessary hardware interfaces for deployment. More generally, we show a clinical application of SlicerROS2 to demonstrate how this platform can be adapted to image-guided robotics applications.

ABSTRACT. This paper proposes a position control strategy for a variable shared autonomy robotic platform for intra-procedural support in Structural Heart Disease treatment. The platform is created by robotizing the commercial MitraClipTM System (MCS), which treats mitral regurgitation by implanting a clip that grasps the valve leaflets. The paper introduces a Neural Network-based Inverse Kinematic Controller (IKC) that guarantees good trajectory tracking. The data set used for training the IKC was generated by exploring the whole workspace of the catheter. The control model proposed is tested for robustness at different motors' velocities, and its performance is compared to a state-of-the-art PID controller.

ABSTRACT. The success of percutaneous focal tumour ablation largely depends on precise needle placement and complete tumour coverage by the ablation zone. Many types of needle guidance devices exist to facilitate precise needle placement. Some examples of these devices include those rigidly attached to the Ultrasound (US) probe, enforcing the “in-plane” insertion while limiting the entry angles of the needle and US probe placement. Other types are the mechanical and robotic (i.e., with electronic characteristics) guiders potentially restricting the movement of the healthcare providers due to their position in the operating room and large size. We present a proof-of-concept of a 3D Slicer module developed as a Virtual Reality (VR) platform for surgical navigation based on streaming US images and a disposable mini-stereotactic frame. The patient-attached “mini” aiming device provides mechanical support for precise needle guidance without interfering with the movement of healthcare providers. The pose of this stereotactic needle guider is magnetically tracked and calibrated, allowing the surgeon to visualize the potential needle trajectory and ablation zone in a 3D VR environment prior to needle insertion. The accuracy of the system was assessed by performing two analyses: a rotational and a translational error, yielding preliminary results of (mean ± SD): rotational (2.19 ± 0.16°) and translational (2.76 ± 1.11 mm). The results, though preliminary, suggest that the combination of the path projection and ablation zone display can assist in the accurate placement of the active tip while ensuring complete tumour coverage.

ABSTRACT. An endovascular guidewire manipulation is essential for minimally-invasive clinical applications. This paper introduces a scalable learning pipeline to train AI-based agent models toward automated endovascular predictive device controls. Specifically, we create a scalable environment by pre-processing 3D CTA images, providing patient-specific 3D vessel geometry and the centerline of the coronary. Next, we apply a large quantity of randomly generated motion sequences from the proximal end to generate wire states associated with each environment using a physics-based device simulator. Then, we reformulate the control problem to a sequence-to-sequence learning problem, in which we use a Transformer-based model, trained to handle non-linear sequential forward/inverse transition functions. We then present the safety ratio and difference between the estimated force and the ground-truth in the test set. our AI-based agents provide an efficient approach to indicate when to turn on/off X-ray.

ABSTRACT. This paper presents a novel approach to produce controlled tissue heating via the robotic control of a surgical laser’s focus. In clinical practice, physicians defocus a laser beam whenever they wish to change its effect from cutting to heating – e.g., to thermally seal a blood vessel. The overarching goal of this research is to create technology to help physicians control the heating and prevent thermal injuries. In the manuscript, we describe the synthesis and experimental verification of a nonlinear controller capable of tracking prescribed temperature profiles. The proposed controller was able to achieve tracking error (RMSE) < 2.5 °C for a target temperature of 50 °C. Furthermore, the controller was able to achieve consistent tracking performance in four different types of tissue (agar-based phantoms, and ex-vivo bovine liver, bone, and chicken muscle) without the need for re-tuning of its parameters, thus indicating its robustness to variations in the physical/optical properties of the tissue being irradiated.

ABSTRACT. Bowel cancer and other bowel conditions are becoming increasingly prevalent conditions in the UK across all age groups. Every year, around 21,000 patients undergo stoma formation surgery, leading to either a temporary or permanent stoma being fitted. Stoma management presents many complications that lead to extensive physical and psychological burdens, including aesthetics, leakage, production of unpleasant odor, skin blistering, and infection, all contributing towards a more complex social life. The current state-of-the-art technology failed to produce a solution that overcomes the aforementioned complications. Furthermore, research in the field is only addressing some complications without providing a solution to address them all.

In partnership with gastroenterologists and stoma patients, we have designed and manufactured a proof-of-concept prototype of a novel controllable stoma valve that eliminates the use of an actual stoma bag and allows the stoma patients to have full control over the stool release hence improving their quality of life. We then evaluated the value for its function in two settings – when the valve is in a static body and when it is in a dynamic motion. This was done keeping in mind the real-world setting where the valve must remain leak-free when the user is either resting or moving. In this paper, we report the data from the experiments to detect leakage in static and dynamic conditions.

ABSTRACT. Research in surgical robotics and automation has made remarkable advancements in recent years thanks to new methods in computer vision, control, and deep learning. Autonomous end-effector manipulation is a challenging task in surgical robotics, and cutting with scissor tools is largely unexplored. A concurrent work explored path and trajectory generation for cutting deformable materials using the da Vinci Research Kit (dVRK). However, an efficient and realistic simulation is necessary for methods such as reinforcement learning (RL) or learned trajectory planning. Our previous work built a simulation for the dVRK in Unity for training RL algorithms on rigid body tasks [2]. To our knowledge, there is no dVRK simulation available that includes the cutting of deformable materials. This paper introduces a cutting simulation of a deformable mesh, which can represent a tissue layer, built onto our Unity dVRK simulation.

ABSTRACT. Intra-cardiac echocardiography (ICE) catheters are of crucial importance in cardiac ablation and structural heart procedures. However, achieving optimal anatomical views by precisely positioning these devices can be challenging for the cardiologist. To address this issue, a 6-degree-of-freedom robot has been designed and fabricated for actuating catheters with up to 4 tendons, as investigated in this research. The robot comprises four backdriveable tendon actuation units and a rotation and insertion unit. A pair of against and antagonist tendons were actuated on a sinusoidal motion while the position of the tip of the catheter was captured using an electromagnetic sensor. A static model based on the Cosserat slender rod theory was fitted to the recorded data through an optimization process, resulting in an estimation mean absolute error of 4.29 with a standard deviation of 2.76. The module of elasticity, Poisson's ratio, tendon compliance, the inner radius of the catheter, and tendon offset were determined to be 3.32 GPa, 0.41, 0.29N/mm, 1.01 mm, and 1.12 mm, respectively. The model demonstrated good compliance with the experimental recordings; however, the lack of hysteresis due to friction has resulted in the model's inability to differentiate between the loading and the unloading phases.

ABSTRACT. Large and staghorn renal calculi (kidney stones) are best treated through Percutaneous Nephrolitotomy (PCNL), a minimally invasive procedure that removes the stones via a small percutaneous, usually in the lumbar region or in the flank. As surgeons must avoid critical anatomical structures such as the colon, spleen, liver, and lung, the use of the conventional PCNL rigid and straight tools often results in incomplete stone removal and has a higher associated complication rate. In this vein, we have designed a handheld concentric tube robot dedicated to treating renal calculi minimally invasively.

ABSTRACT. Colonoscopy is considered the golden standard procedure for the screening of the lower gastro-intestinal (GI) and the prevention of the colorectal cancer (CRC), which nowadays represents the third most common cancer-related cause of death. Recent advancements in robotics, electronics and material science have produced the development of non-invasive devices to access the GI in clinical application, in the form medical indigestible capsule robots, as well as are paving the way for novel robotic system for colonoscopy. Robotic solutions are being proposed by the research community to improve accessibility and image quality for intervention and diagnosis in the GI tract, while reducing the discomfort on the patient. However, very few devices have the potential to successfully navigate the large intestine, due to the complicated structure of the mucosa in parallel with the significant challenge of designing miniaturized robots. Aiming at addressing the limitations of capsule endoscopy and robotic colonoscopy, the design of a novel tethered capsule system is presented in this research work. The robot aims at actively moving inside the colon to prevent the main discomfort factors in clinical settings, while providing a robust and reliable control of the robot position and imaging recording. The system comprises a track-based locomotion strategy that exploits a single motor and a worm gear mechanism to establish full-body track navigation, and the inflation of two toroidal soft chambers to enable the diameter-shifting capability for the system to adapt and match the local lumen of the intestine, thus enabling traction control and optimized field of view of the front cameras. This work presents the first miniaturised and modularised prototype of the SoftSCREEN system, designed to create a sterilisable reusable component and a cheap disposable component. We evaluated this first prototype in a 1:1 scale phantom.

ABSTRACT. Raman spectroscopy, a photonic modality based on the inelastic backscattering of coherent light, is a valuable asset to the intraoperative sensing space, offering non-ionizing potential and highly-specific molecular fingerprint-like spectroscopic signatures that can be used for diagnosis of pathological tissue in the dynamic surgical field. Though Raman suffers from weakness in intensity, Surface-Enhanced Raman Spectroscopy (SERS), which uses metal nanostructures to amplify Raman signals, can achieve detection sensitivities that rival traditional photonic modalities. Our lab has previously developed SERS-based Gold Nanostars that accumulate preferentially in certain brain tumors, allowing for discernment of pathological tissue from healthy tissue.

However, there still remains a need for a robotic Raman system and classification algorithm that can use Gold Nanostars to efficiently reconstruct a region of interest, to aid surgeons as they seek to improve extent-of-resection and overall patient survival. In this study, we outline a robotic Raman system that can reliably pinpoint the location and boundaries of a tumor embedded in healthy tissue, modeled here as a tissue-mimicking phantom with selectively infused Gold Nanostar regions. Further, due to the relative dearth of collected biological SERS or Raman data, we implement transfer learning and achieve 100% validation classification accuracy for Gold Nanostars compared to Control Agarose, thus providing a proof-of-concept for Raman-based deep learning training pipelines.

We reconstruct a surgical field of 30x60mm in 10.2 minutes, and achieve 98.2% accuracy, preserving relative measurements between features in the phantom. We also achieve an 84.3% Intersection-over-Union score, which is the extent of overlap between the ground truth and predicted reconstructions. Lastly, we also demonstrate that the Raman system and classification algorithm do not discern based on sample color, but instead on presence of SERS agents. This study provides a crucial step in the translation of intelligent Raman systems in intraoperative oncological spaces.

ABSTRACT. This study focuses on 3D breast surface reconstruction with 2D ultrasound images acquired by autonomous scanning. The presented single sensor surface reconstruction could increase the accuracy of magnetic resonance/ultrasound image reconstruction and subsequently of lesion detection by eliminating the need for multi-sensor calibration. In the presented approach, the robot scans the breast based on a confidence driven algorithm which eliminates the need of prior knowledge of the breast location, shape, and orientation. The confidence driven controller ensures that high quality acoustically coupled ultrasound, these images are then evaluated by the presented algorithm. The algorithm divides the Cartesian location of all individual scan-lines in three point-clouds; two low knowledge high density point clouds and a high knowledge low density point could. The two low knowledge high density point clouds are formed by the locations of acoustically coupled and non-coupled scan-lines. These points-clouds only provide knowledge about the if the breast is at that location in deformed state or not there. The third point cloud is filled with the transitional scan-lines (no contact to contact and vice versa), these points represent the breast surface in space in undeformed state. Our robot-assisted approach reconstructs the breast surface based on these three point-clouds with an accuracy of -3.87 pm 1.08 mm compared to the golden standard which is breast surface reconstruction based on markers.

ABSTRACT. Mitral valve regurgitation is the most common valvular disease, affecting 10% of the population over 75 years old. Left untreated, patients with mitral valve regurgitation can suffer declining cardiac health until cardiac failure and death. Mitral valve repair is generally preferred over valve replacement. However, there is a direct correlation between the volume of cases performed and surgical outcomes, therefore there is a demand for the ability of surgeons to practice repairs on patient specific models in advance of surgery, however, producing a mesh model of the valve geometry from TEE imaging remains a challenge. We have developed a 3D Slicer module incorporating DeepMitral to enable fully automated mitral valve leaflet segmentations. From leaflet segmentations, we can extract the atrial surface and generate a positive mold of the patient-specific leaflet geometry. These molds can then be 3D printed to be used in the production of dynamic silicone valve models for functional simulation in our beating heart phantom, enabling various repair techniques to be evaluated prior to surgery. Our 3D Slicer module simplifies the overall workflow for valve modelling by automating previously manual steps and incorporating them into a single user interface. We demonstrate an algorithm for extracting the atrial surface from an automatic leaflet segmentation, enabling creation of a positive mold mesh that can easily be 3D printed. By simplifying the modelling workflow, we aim to improve the clinical translatability of patient-specific heart valve modelling.

ABSTRACT. Over 570,000 new cases of bladder cancer are diagnosed worldwide every year. It is essential to detect new tumors as early as possible to reduce the mortality rate. In addition, the muscle invasiveness of lesions should be quantified to determine the optimal treatment plan.

Within the "Next-gen in-vivo cancer diagnostics" research project we propose a new cystoscopy instrument consisting of an optical coherence tomography (OCT) sensor, a camera and a light source, mounted on the tip of a concentric tube robot (CTR). The camera images could then be used to create 3-D reconstructions of the bladder wall and to quantifiy changes in its texture between successive cystoscopy sessions. In addition, the camera could guide the OCT sensor to investigate the bladder wall structure at the locations of possible tumors in order to investigate the malignancy and muscle invasiveness.

This research specifically reports on creating 3-D reconstructions of bladder phantoms and co-registration of successive sessions, in order to automatically detect and indicate changes in texture which might be related to the onset and growth of tumors.

The results show that cystoscopy images of the bladder could be reconstructed in 3-D and subsequently projected to a 2-D atlas. Registrations of successive sessions were effectively co-registered with help of the TPS algorithm and the system was able to automatically detect all six images of tumors which were added between the two sessions.

ABSTRACT. This paper presents a step towards autonomous robotic suturing. Specifically, we propose a method for suture needle state estimation during insertion into soft tissue based on electrical bioimpedance (EBI) sensing. EBI is an advantageous sensing modality in RMIS, given that it is non-invasive and requires only minor modifications to existing surgical instruments. In this study, we equip a surgical robot with EBI sensing capabilities, allowing the robot to measure the electrical impedance between a needle driver instrument and a common ground electrode. The proposed method requires a suture needle with insulation coating except for its tip, end, and notch in the middle. We conducted an experiment for concept validation based on ex vivo animal tissue where we obtained a 98.8% prediction accuracy on four different suture needle insertion states. Most interestingly, we could accurately determine when the needle tip exits after being pushed through soft tissue, which is challenging to do with, e.g., computer vision due to the needle’s small size and occlusions. The needle tip exiting is valuable information as often one wishes to grasp the needle tip with a second manipulator to complete the suture throw by pulling the needle through the tissue.