Suturing Tasks Automation Based on Skills Learned From Demonstrations: A Simulation Study

Abstract

In this work, we develop an open-source surgical simulation environment that includes a realistic model obtained by MRI-scanning a physical phantom, for the purpose of training and evaluating a Learning from Demonstration (LfD) algorithm for autonomous suturing. The LfD algorithm utilizes Dynamic Movement Primitives (DMP) and Locally Weighted Regression (LWR), but focuses on the needle trajectory, rather than the instruments, to obtain better generality with respect to needle grasps. We conduct a user study to collect multiple suturing demonstrations and perform a comprehensive analysis of the ability of the LfD algorithm to generalize from a demonstration at one location in one phantom to different locations in the same phantom and to a different phantom. Our results indicate good generalization, on the order of 91.5%, when learning from more experienced subjects, indicating the need to integrate skill assessment in the future.

I Introduction

Robotic surgery has revolutionized the field of medical science by providing surgeons with enhanced dexterity, visualization, and precision during minimally invasive procedures. In the recent two decades, the usage of surgical robots in hospitals has significantly increased [1]. As the technology becomes more widespread, there is an increasing need to improve task autonomy [2] for surgical robots to facilitate their use by clinicians.

For instance, the da Vinci® Surgical System (dVSS), one of the most widely adopted robotic platforms [3], has transformed surgical interventions, enabling complex procedures with improved patient outcomes. However, despite the remarkable capabilities of robotic systems, certain repetitive tasks, such as suturing, continue to demand significant manual intervention, which can impose cognitive load on surgeons and impact procedural efficiency.

Automating repetitive surgical tasks has emerged as a promising approach to alleviate the cognitive burden on surgeons, allowing them to focus on critical decision-making and enhancing patient care. In this regard, automating suturing has gained significant attention, aiming to reduce surgical time, enhance precision, and ensure consistent suture placement, thereby optimizing surgical outcomes.

In this paper, we present a method for automatic suturing path planning using the da Vinci Research Kit (dVRK) [4] in simulation with comprehensive analysis. Taking advantage of the Asynchronous Multi-Body Framework (AMBF) simulator [5, 6] that integrates seamlessly with dVRK and adding a phantom volume scanned by MRI, we construct a realistic simulation surgical scene to perform the suturing procedures for training the automation algorithms. Our approach leverages Learning from Demonstration (LfD) with Dynamic Movement Primitives (DMPs) [7, 8, 9] and Locally Weighted Regression (LWR) [10]. Also, we construct a user study to collect data from human subjects to train the robot on performing suturing tasks. Notably, this simulation scene allows us to have much higher flexibility for phantom and environment selections and overcome the restrictions on demonstration collection. Moreover, the simulation enables us to obtain the ground truth of objects for motion analysis and generality tests. Furthermore, the trained model is anticipated to be directly deployed to the physical dVRK, ensuring its practical applicability.

In summary, the contributions of this work are:

• A novel and realistic simulation environment using an MRI-scanned phantom

• A recording pipeline for suturing automation data collection in the simulation

• A method to improve the generality of the LfD algorithm by selecting the suture needle as the learning object

• A comprehensive assessment of generality by testing at different positions and with different phantoms

II Related Works

A series of researches have been done to push forward the effort on real-world bi-manual suturing tasks automation. Sen et al.[11] proposed the Suture Needle Angular Positioner (SNAP) to ensure a constant transformation between the robot manipulator and the suture needle so that it can facilitate the suturing automation procedure with sequential convex programming. The known transformation not only enables the researchers to regard the needle as an extended end-effector for calculating the kinematics, but also excludes the variance due to different needle grasps. On the other hand, the constant transformation restricts the generality of the algorithm. Varier et al. [12] first introduced Reinforcement Learning (RL) to accomplish the suturing automation. Nevertheless, Varier’s method requires to discretize the workspace of the robot which is a challenging task when implementing in real surgeries. Also, depending on the grid size, it might need tremendous computational power to enable the algorithm. Schwaner et al. [13, 14] presented a LfD algorithm prototype of suturing automation with DMPs using UR5 robot arms holding a da Vinci Large Needle Driver. Schwaner’s method integrates with computer-vision-based tracking and implements the algorithm on a real-world robot. Nevertheless, it collects demonstrations from the real world, which can be a challenging and time-consuming process. In addition, the complexity of collecting real-life demonstrations leads to greater difficulty when performing generality tests.

Due to the restrictions in the real world and ethical concerns, simulation environments have been developed to facilitate the process of obtaining demonstrations for LfD and medical training[5, 6, 15, 16, 17]. Collecting demonstrations from simulation can overcome some difficulties of obtaining ground truth when performing suturing tasks, such as suture needle tracking[18], and allows experimentation without risk. An ideal simulation can also optimize the experiment design by eliminating the random error due to the noise from the robot or the external environment and help to construct a better understanding of human motion patterns.

III Methodology

The following two sections present standard formulations of Dynamic Movement Primitives (DMP) and Locally Weighted Regression (LWR) as background information. These are followed by Section III-C, which describes the LfD pipeline that is based on the needle trajectory.

III-A Dynamic Movement Primitives

DMP[7, 8, 9, 19, 20, 21, 22] is a method for trajectory control and planning, which can represent complex motor actions without manual parameter tuning. In this work, we use discrete DMP to learn a point-to-point trajectory from a demonstration. Corresponding methodologies for position and orientation regeneration are shown in the following sections.

III-A1 Position

We utilize DMPs to encode Cartesian space robot position trajectories and generate the learning weights. DMPs can represent a movement trajectory with a group of second-order ordinary differential equations, as shown in Equation 1 [9]:

$$\begin{split}\tau^{2}\ddot{y}&=\alpha_{y}(\beta_{y}(g-y)+\tau\dot{y})+f(x)\\ \tau\dot{x}&=-\alpha_{x}x\end{split}$$ (1)

where $y$, which can also be represented as $y(t)\in\mathbb{R}^{3}$, is the Cartesian space position of the robot system’s end effector at time $t$; $x$ is a system variable which initiates at 1 and diminishes to 0 over time; $\alpha_{x}$, $\alpha_{y}$ and $\beta_{y}$ $\in\mathbb{R}^{+}$ are the gain coefficients; $g\in\mathbb{R}^{3}$ is the goal position of the end effector; $\tau\in\mathbb{R}^{+}$ is the time constant; and $f(x)$ is the nonlinear forcing term represented by Equation 2. Regardless of how the gain coefficients are chosen, the robot system will always converge to $g$ because the influence of $f(x)$ will be ignored when $x\rightarrow 0$. This feature can ensure the stability of the system.

The nonlinear forcing term $f(x)$ can be represented as[9]:

$$\begin{split}f(x)&=\frac{\sum_{i=1}^{N_{bfs}}\psi_{i}(x)w_{i}}{\sum_{i=1}^{N_{bfs}}\psi_{i}(x)}x(g-y_{0})\\ \psi_{i}(x)&=e^{-h_{i}(x-c_{i})^{2}}\\ h_{i}&=\frac{{N_{bfs}}^{1.5}}{\alpha_{x}c_{i}}\end{split}$$ (2)

where $\psi_{i}(x)$ is a basis function, which is a Gaussian function; $N_{bfs}$ is the number of basis functions; $y_{0}\in\mathbb{R}^{3}$ is the initial position of the end effector; $w_{i}\in\mathbb{R}^{3}$ represents the learning weights; $c_{i}$ is the center of the Gaussian function and $h_{i}$ is the variance of the Gaussian function.

For learning from demonstration, we can calculate the desired nonlinear term $f_{des}(x)$ from a given demonstration trajectory $[y_{demo},\dot{y}_{demo},\ddot{y}_{demo}]$, and then we can obtain the learning weights $w_{i}$ to regenerate the trajectories:

$$\begin{split}f_{des}(x)&=\tau^{2}\ddot{y}_{demo}-\alpha_{y}(\beta_{y}(g_{demo}-y_{demo})+\tau\dot{y}_{demo})\\ &=\frac{\sum_{i=1}^{N_{bfs}}\psi_{i}(x)w_{i}}{\sum_{i=1}^{N_{bfs}}\psi_{i}(x)}x(g_{demo}-{y_{demo}}(0))\end{split}$$ (3)

where $y_{demo}(0)$ is the initial state of the demonstration trajectory and $g_{demo}$ is the goal state of the demonstration trajectory.

For the time constant $\tau$, we can move it into the gain coefficients $\alpha_{x}$, $\beta_{y}$ and $\alpha_{y}$. Therefore, without losing any generality, we can set the time constant $\tau=1$ for simplification. After calculating the learning weights via LWR and substituting the learning weights into Equation 1, we can obtain the $\ddot{y}$ in Equation 1 and integrate $\ddot{y}$ to regenerate the trajectory $y$.

III-A2 Orientation

For orientation, we utilize quaternions [23, 24, 25, 26], $\mathbf{q}=v+\mathbf{u}\in S^{3}$, to represent the angular movements, where $v\in\mathbb{R}$, $\mathbf{u}\in\mathbb{R}^{3}$ and $S^{3}$ is a unit sphere in $\mathbb{R}^{4}$. For this kind of representation, $q$ and $-q$ are the same orientation. Similar to Equation 1, DMPs can also represent an angular movement trajectory with a group of second-order ordinary differential equations as follows[27]:

$$\begin{split}\tau^{2}\dot{\mathbf{\omega}}&=\alpha_{z}(\beta_{z}2log(g_{o}*\overline{q})-\tau\mathbf{\omega})+f_{o}(x)\\ \dot{q}&=\frac{1}{2}\mathbf{\omega}*q\\ \mathbf{\omega}&=2log(g_{o}*\overline{q})\end{split}$$ (4)

where $log(\cdot)$ refers to the natural logarithm[24]; $\overline{q}$ is the quaternion conjugation; $g_{o}\in S^{3}$ is the goal quaternion orientation; $\mathbf{\omega}\in\mathbb{R}^{3}$, which can also be denoted as $\mathbf{\omega}(t)$, is the angular velocity of the robot system at time $t$; $x$ is a system variable identical to Equation 1; $\alpha_{z}$ and $\beta_{z}$ are the gain coefficients; $\tau$ is the time constant; $f_{o}(x)$ is the nonlinear forcing term. As in the previous section, we can set the time constant $\tau=1$ for simplification. The rules of operations for quaternions are defined in the papers[25, 26].

The nonlinear forcing term $f_{o}(x)$ can be represented as:

$$\begin{split}f_{o}(x)&=D_{o}\frac{\sum_{i=1}^{N_{bfs}^{o}}\psi_{i}(x)w_{i}^{o}}{\sum_{i=1}^{N_{bfs}^{o}}\psi_{i}(x)}x\end{split}$$ (5)

where $D_{o}=diag(2log(g_{o}*q_{0}))\in\mathbb{R}^{3\times 3}$ is the scaling term; $q_{0}$ is the initial state of the orientation trajectory; $\psi_{i}$ is the basis function shown in Equation 2; $w_{i}^{o}\in\mathbb{R}^{3}$ is the learning weight for orientation trajectories.

Similar to Equation 3 , given the demonstration orientation trajectory $[q_{des},\mathbf{\omega}_{des},\dot{\mathbf{\omega}}_{des}]$ and Equation 4, we can calculate the learning weights $\mathbf{w}_{i}^{o}$ via solving the following equation:

$$\footnotesize\begin{split}\frac{\sum_{i=1}^{N_{bfs}^{o}}\psi_{i}(x)w_{i}^{o}}{\sum_{i=1}^{N_{bfs}^{o}}\psi_{i}(x)}x=D_{o}^{-1}(\dot{\mathbf{\omega}}_{des}-\alpha_{z}(\beta_{z}2log(g_{des}^{o}*\overline{q}_{des})-\mathbf{\omega}_{des}))\end{split}$$ (6)

where $g_{des}^{o}$ is the goal state of the demonstration orientation trajectory.

After obtaining the learning weights, we can integrate the quaternions in Equation 4 using the following formula:

$$\begin{split}\mathbf{q}(t+\Delta t)&=e^{\frac{\mathbf{\omega}(t)\Delta t}{2}}*\mathbf{q}(t)\\ e^{\mathbf{r}}&=\left\{\begin{array}[]{lr}cos(\lVert\mathbf{r}\rVert)+sin(\lVert\mathbf{r}\rVert)\frac{\mathbf{r}}{\lVert\mathbf{r}\rVert}&\mathbf{r}\neq\mathbf{0}\\ \mathbf{0}&\textrm{otherwise}\end{array}\right.\end{split}$$ (7)

If we limit the domain of the exponential map $e^{\mathbf{r}}$ : $\mathbb{R}^{3}\rightarrow S^{3}$ to $\lVert\mathbf{r}\rVert<\pi$ and the domain of the logarithmic map to $S^{3}/(-1+[0,0,0]^{T})$, then both mappings become one-to-one, continuously differentiable and inverse to each other. In addition, we also utilize phase stopping and goal switching[27] techniques for better performance.

III-B Locally Weighted Regression

Locally weighted regression[10] belongs to a class of nonparametric statistical techniques called locally weighted learning (LWL). LWR is a memory-based learning algorithm that can efficiently represent and train complex motor movements in autonomous adaptive control of robotic systems. The key advantage of LWR is its fast training speed, which only requires adding new training data to the memory.

We can use LWR to calculate the optimal learning weights $w_{i}$ for positions in Equation 3. Then, the cost function to be minimized is defined as:

$$\begin{split}J(w_{i})=\sum\psi_{i}(x)(f_{des}(x)-w_{i}(x(g_{demo}-y_{demo}(0))))^{2}\end{split}$$ (8)

Since $x$, which can be also denoted as $x(t)$, is a function of time $t$, therefore, denoting $y_{offset}=g_{demo}-y_{demo}(0)$, we can rewrite Equation 8 as follows:

$$\begin{split}J(w_{i})=\sum\psi_{i}(t)(f_{des}(t)-w_{i}(x(t)y_{offset})^{2}\end{split}$$ (9)

For solving Equation 9, construct a diagonal matrix $\Psi_{i}$ using $\psi_{i}$ along with time:

$$\small\begin{split}\Psi_{i}=\begin{bmatrix}\psi_{i}(t_{0})&\cdots&0&0\\ \vdots&\psi_{i}(t_{1})&\ddots&0\\ 0&\ddots&\ddots&\vdots\\ 0&0&\cdots&\psi_{i}(t_{n})\end{bmatrix}\end{split}$$ (10)

where $t_{0},t_{1},\cdots,t_{n}$ indicate the corresponding times of demonstration trajectory points. Then, we can convert the terms $x(t)y_{offset}$ and $f_{des}(t)$ into a matrix form as $s$ and $F_{d}$:

$$\begin{split}s=\begin{bmatrix}x(t_{0})y_{offset}\\ \vdots\\ x(t_{n})y_{offset}\end{bmatrix},\qquad F_{d}=\begin{bmatrix}f_{des}(t_{0})\\ \vdots\\ f_{des}(t_{n})\end{bmatrix}\end{split}$$ (11)

Substituting Equation 10 and Equation 11 into Equation 9, we can obtain:

$$\begin{split}J(w_{i})&=(F_{d}-s^{T}w_{i})^{T}\Psi_{i}(F_{d}-s^{T}w_{i})\end{split}$$ (12)

Using the least square method to optimize, we obtain the learning weight along with time $t$ [10]:

$$\begin{split}w_{i}=\frac{s^{T}\Psi_{i}F_{d}}{s^{T}\Psi_{i}s}\end{split}$$ (13)

III-C Learning From Demonstration Pipeline

In our work, we grasp the needle using a script and utilize a simple trajectory planning technique to move the PSMs to the desired poses for handing over the needle. Therefore, we implement learning from demonstration algorithms for motion 2 and 4 as shown in section IV-C.

The pipeline for implementing the LfD algorithm is shown in Figure 1.

Figure 1: Learning from Demonstration pipeline

The simulation object ground truth database contains the ground truth of all objects as described in section IV-A. We can obtain the ground-truth states of the needle, entry & exit points and PSM joints from both the demonstrations and the real-time simulation scene. Given those ground-truth states, we can find the desired start and goal states for the robot system’s end-effector.

IV Experiment Setup

IV-A Simulation Platform

As shown in Figure 2, we construct simulation scenes for suturing using AMBF[6]; the simulation platform contains:

• a suturing phantom choosing from two alternatives

• two simulated dVRK Patient Side Manipulators (PSMs) with Large Needle Drivers from Intuitive Surgical, Inc.

• one simulated dVRK Endoscope Camera Manipulator (ECM) with a stereo camera attached

• one suture needle with radius of 10.18 mm, 120-degree arc angle and thread attached

• red markers for entry and exit points

The dimensions of the dVRK arms are obtained from the measurements of the real-world first-generation dVSS.

(a) Phantom 1: Synthetic

(b) Phantom 2: Scanned

Figure 2: Simulation environments

To obtain phantom 2, we scanned the 3D-MED Soft Tissue Suture Pad using MRI and added the scanned volume to the simulation platform (Figure 3). Compared to CT scanning, MRI scanning can provide more details and has higher contrast for soft tissues or phantoms[28]. Therefore, we selected MRI to scan the phantom and this method can be further used to scan real tissues or complex phantoms in the future.

(a) Real-World Phantom

(b) MRI-Scanned Volume

Figure 3: 3D-MED soft tissue suture pad

IV-B Teleoperation Setup

For the teleoperation setup, we utilize the dVRK High Resolution Stereo Viewer (HRSV, also known as the viewer console) and Master Tool Manipulators (MTMs), as shown in Figure 4, to interact with the simulation platform. This setup is identical to the dVSS clinical setup and thus it can bring an immersive and realistic experience when teleoperating.

(a) Physical dVRK

(b) Simulation Scene in HRSV

Figure 4: Teleoperation setup

Integrating with the dVRK HRSV brings the stereo vision to the system users. The stereo viewer allows the users to generate pseudo 3D vision so that they can have a more realistic and accurate sense of the depth information.

IV-C User Study

For obtaining the training data for the LfD algorithm, we perform a user study under approved IRB protocols IRB-22-0593 at Worcester Polytechnic Institute and HIRB00000701 at Johns Hopkins University. We recruited 10 subjects, consisting of 8 males and 2 females. Among those 10 users, 4 users have previous experience with surgical training and using the dVRK. The human subjects are asked to perform simple continuous sutures without tying knots in the simulation, as shown in the supplemental video. A single-throw suture without tying a knot can be decomposed into the following five subtasks[14]:

1. Pick up the needle from the initial pose using the right arm (PSM2) and move PSM2 so that the needle tip is at the desired entry point

2. Insert the needle through the phantom using PSM2 to the exit point

3. Regrasp the needle with the left arm (PSM1).

4. Extract the needle with PSM1, completing the throw

5. Hand over the needle from PSM1 to PSM2 and move to the next initial pose

In the simulation scenes, we segmented the phantom and named different pairs of entry and exit markers as in Figure 5.

(a) Phantom 1: Synthetic

(b) Phantom 2: Scanned

Figure 5: Marker indices for phantoms

The users perform the suture task following the sequence of indices from 1 to 4. For each phantom, the users complete the suture task for all 4 pairs consecutively. The collected data is segmented manually and utilized as the training datasets.

IV-D Data Collection and Preprocessing

In this section, we describe our data collection techniques and strategies.

IV-D1 Data Collection Framework

The control and communication commands of the simulation follow the Collaborative Robotics Toolkit (CRTK) convention [29], which ensures compatibility with the physical dVRK. To collect demonstration data, we develop the data collection framework shown in Figure 6.

Figure 6: Motion data collection framework

The foot pedal presses are also included in the data collection. The raw collected data is in joint space and stored in ROS bags.

IV-D2 Data Preprocessing

The ground-truth poses of the needle, entry & exit points are directly subscribed from the simulation. To segment the raw collected data, we use a velocity-based filter to examine the points with given thresholds [30]. When stretching out the raw input data, for Cartesian space positions, a quintic interpolation is used. On the other hand, for the Cartesian space orientations, spherical linear interpolation[31] is used. For performance evaluation, we utilize the Dynamic Time Warping (DTW) technique [32, 33, 34] to synchronize trajectories with different lengths.

V Experimental Evaluation and Result

V-A Model Parameters

The hyperparameters we utilize in this paper are shown in Table I.

TABLE I: Model hyper-parameters

Name	Value	Meaning
$\alpha_{x}$	1	gain coefficient for the system variable $x$
$\alpha_{y}$ / $\alpha_{z}$	25	proportional gain for position and orientation LfD
$\beta_{y}$ / $\beta_{z}$	6.25	derivative gain for position and orientation LfD
$N_{pts}$	500 (100)	number of both LfD regenerated points for task 2 (for task 4)
$N_{bfs}$	100 (50)	number of both LfD basis functions for task 2 (for task 4)
$N_{bfs}^{o}$	40 (20)	number of both LfD basis functions for task 2 (for task 4)

V-B Trajectory Regeneration

First, we evaluate the performance of the LfD algorithm. For the evaluation, we select a random user’s data at the scan2 pair of entry and exit markers. Then, we train the LfD algorithm and test on the same pair of entry and exit markers. After that, we plot both the demonstrations and the regenerated trajectories in Figure 7.

(a) Position Trajectory

(b) Orientation Trajectory

Figure 7: Demonstration and regenerated trajectories of the suture needle at the pair scan2

From this figure, we can anticipate that the regenerated trajectories can perform the same suture task in the simulation scenes successfully.

Furthermore, we repeat the above evaluation test for all 8 pairs of entry & exit markers of all 10 users. We calculate not only the point-level errors like start and goal state errors but also the trajectory-level errors to assess the performance of the trajectory regeneration, as summarized in Table II. Due to some abrupt shifts in the trajectory or among different users, we may have a chance to observe high average errors and can evaluate the median of the errors instead to access the performance.

TABLE II: Errors of LfD regenerated trajectories

Item		Mean	STD
task 2   pos error (mm)	start	0	0
	goal	0.05	0.06
	trajectory	0.17	0.07
task 2   ori error (deg)	start	0	0
	goal	0.74	1.24
	trajectory	1.71	8.44
task 4   pos error (mm)	start	0.02	0.02
	goal	0.43	0.31
	trajectory	0.38	0.16
task 4   ori error (deg)	start	0	0
	goal	11.32 (median 2.71)	22.03
	trajectory	17.54 (median 1.42)	27.09

When substituting the regenerated trajectories into the simulation scenes, 76 trajectories successfully accomplish the suture task. The trajectory regeneration using the LfD algorithm achieves 95% success rate with reasonable start & goal state errors.

V-C Generality Test

For the next step, we assess the generality of the LfD algorithm by training the LfD algorithm on each pair of entry & exit markers from both phantoms shown in Figure 5 and testing it using all pairs of entry & exit markers. We observe the suture task completeness to assess the generality.

The generality can be quantized based on the classical model of probability and represented in the following four levels:

• 1.0 - Successfully complete the suture task

• 0.8 - Fail to complete the suture task, but only missing the exit marker when extracting, as shown in Figure8a

• 0.4 - Fail to complete the suture task due to missing the entry marker when inserting but still following a reasonable trajectory, as shown in Figure8b

• 0 - Fail to complete the suture task due to the other reasons

(a) Miss the exit marker

(b) Miss the entry marker

Figure 8: Partial completion cases

Going through the generality test for all users and all pairs of entry & exit markers, we obtain the heat map shown in Figure 9.

Figure 9: Generality test result

The average values of generality for all users are shown in Table IV. We can see that the LfD algorithm has reasonable generality for the selected suturing subtasks and is worth further investigation. In addition, we can anticipate that we may have a chance to achieve better performance by introducing more experienced or even professional human subjects.

TABLE III: Generality result

User	Generality
Experienced	0.915
Naive	0.742
Overall	0.811

TABLE IV: Success rate of experienced users

Task	Individual	Overall
1	256 / 256	256 / 256
2	202 / 256	202 / 256
3	197 / 202	197 / 256
4	181 / 197	181 / 256
5	181 / 181	181 / 256

V-D Task Execution Performance

According to the results shown in Table IV, we can see that the regenerated trajectories learned from the experienced users have much better performance in the generality test. Therefore, when assessing the overall suturing task success rate, we will only focus on the experienced users set to exclude the errors due to lack of acquaintance of the skills.

From Table IV, we can see that the regenerated trajectories learned from the experienced users have an overall success rate of 70.7% for suturing task completion. Nevertheless, we also find that most of the failures have a similar scenario as shown in Figure8a. For success rate evaluation, the result is binary instead of discrete values. Therefore, compared to the generality test result, the success rates would have smaller values due to the false negative cases.

Last but not least, we evaluate the task execution time. We record the time taken for completing a whole suturing procedure on each pair of entry & exit markers and obtain:

Figure 10: Task execution time

From Figure 10, we can see that for suturing tasks on a simple synthetic phantom, the LfD algorithm can slightly improve the time efficiency when completing the tasks. On the other hand, for the tasks on the scanned phantom with higher complexity, the LfD algorithm can reduce the total time for completing the suturing task by 20%.

VI Discussion and Future Development

In this work, we build a novel and realistic simulation scene using an MRI-scanned phantom, and construct a data collection pipeline for the simulation. Also, we present a LfD algorithm using DMP and LWR for suturing task automation with comprehensive analyses. As a result, we can see that the regenerated trajectories using the LfD algorithm can complete the suturing task with 95% success rate in the simulation. Also, the algorithm achieves high generality of 0.811 and time efficiency by a 20% reduction in task execution time. In addition, we can see that the regenerated trajectories learned from the experienced users usually have better performance. Therefore, for further development, we can introduce more experienced human subjects or professional human subjects such as surgeons. Moreover, we can implement skill assessment techniques [35, 36, 37, 38, 39] when preprocessing the data to exclude unskilled demonstrations.

Taking advantage of AMBF and CRTK, and integrating with advanced vision perception methods [40, 41, 42, 43, 44, 45, 46] to obtain the estimated poses of the suturing needle, we may further extend the proposed algorithm from the simulation to the real world.