Slip, or accidental loss, of grasped biological tissue can have negative consequences in all types of surgery (open, laparoscopic, robot-assisted). This work focuses on slip in robot-assisted surgery (RAS) with the goal of improving the quality of grasping and tool–tissue interactions. We report on a survey of 112 RAS surgeons, the results of which support the value of detecting and reducing slip in a variety of procedures. We conducted validation tests using a thermal slip sensor in a surgical grasper on tissue in vivo and ex vivo. The results of the survey and validation informed a user study to assess whether tissue slip feedback can improve performance and reduce effort in a phantom tissue manipulation task. With slip feedback, experienced subjects were significantly faster to complete the task, dropped tissue less (3% versus 38%), and experienced decreased mental demands and situational stress. These results provide motivation to further develop the sensor technology and incorporate it in robotic surgical equipment.
Robot-assisted surgery (RAS) using the da Vinci® Surgical System (Sunnyvale, CA) has become increasingly prevalent over the last decade  and holds promise for improving surgeons' accuracy and dexterity . However, the loss of direct manual contact with the surgical site results in the absence of tactile information. Surgeons instead learn to interact with tissue based mainly on visual cues. Unlike human hands, which have large sensorized surfaces, surgical graspers of all types (open handheld tools, laparoscopic, RAS) have small dimensions which can cause high pressures and result in crushing or damaging tissue when improperly used [3,4]. Surgeons aim to grasp tissue lightly enough to avoid crushing it but with sufficient grasp force to prevent accidental tissue loss. Achieving this grasp balance is challenging because the amount of force that avoids tissue damage and grasp loss simultaneously relies on many situational factors, including tissue type and health, grasper type, grasp angle, grasping history, and more. Some tissue slip is permissible provided enough remains in the jaws to prevent loss or damage. We believe that helping surgeons achieve atraumatic, reliable grasping is best served by sensing slip onset directly, rather than estimating it from visual and/or force cues.
An important skill in minimally invasive surgery is maintaining situational awareness of all relevant anatomy and tools. The da Vinci Xi surgical system allows surgeons to use three tools and an endoscope, but no more than two of these are under direct manual control at any time. RAS surgeons are taught to keep active tools in view but at times elect to use a nonactive tool to provide traction on tissue and maintain exposure. Given the size and layout of many anatomies, this nonactive tool may be outside of their immediate focus or even be offscreen. This can occur in large intestine work and in cholecystectomies, among other procedures (see Sec. 2). Consequences of any slip can include lengthened procedure time, tissue tearing, and loss of critical view.
Although tissue slip negatively impacts grasping and manipulation tasks in surgery, it is a relatively unexplored topic in the literature. These observations raised several questions regarding slip sensing onset in MIS, including its importance, in what scenarios it has value, and how well it works in practice. To answer these questions, we report on a survey of 112 RAS surgeons. The results of the survey, along with validation tests on both live and excised porcine tissue (Fig. 1), informed the design of a study with experienced MIS operators to assess the value of slip detection. The results of the study, presented in Sec. 4, confirm that slip detection increases speed and reduces mental demand.
Purpose and Methods.
To improve tissue–tool interactions in RAS, it is crucial to understand the needs of the surgeons who perform them. We prepared and distributed a survey to understand the importance of tissue slip. We sent electronic invitations and offered a $15 honorarium to 983 surgeons who opted in to receive surveys from Intuitive between May 23 and June 1, 2018.
The survey was estimated to take 15 min to complete and consisted of 19 questions (see Table 3 in the Appendix). The three main sections of the survey sought to understand (1) the clinical importance and occurrence of slip, (2) specific surgeon experiences and responses to tissue slip, and (3) preferences for receiving slip feedback. Identifying information was removed before analysis, which was conducted in matlab 2017b (MathWorks) using frequency distribution tests. Optional comments were free-text reflective format and designed to allow participants to elaborate on multiple choice answers.
112 (11%) of the 983 surgeons responded; see Table 2 in the Appendix for participant demographics. The surgeons were analyzed as a single population.
In question #1, surgeons ranked the mean clinical importance of tissue slip comparable to that of tissue crushing or tearing (6.0 ± 0.4 versus 5.6 ± 0.5 and 5.8 ± 0.6 out of 10). For reference, a score of 6 corresponds to a delay in procedure without requiring conversion or causing harm to the patient or user. For a full description, see Table 4 in the Appendix for Severity Evaluation Criteria from an Intuitive protocol based on Ref. . For the number of slip events in a typical RAS procedure, 4% reported slip never occurring, while 41% reported 1–2 times, 34% reported 3–5 times, and 21% reported >5 times.
The responses for questions #3–11 are summarized in Fig. 2; #12–17 are in Fig. 3. Surgeons reported tissue slip occurring more frequently during retraction (6.5 ± 0.6) than during critical view setup or while handling fragile tissue (4.1 ± 0.5 and 3.6 ± 0.6, respectively), but frustration and rated clinical risk (Table 4) across the three categories did not differ significantly. Importantly, clinical risk due to slip was deemed low with no harm occurring to the patient or user.
Surgeons described slip occurring more often on anatomies that consist of bulky stiff tissue (e.g., the vaginal cuff), during high-tension tasks (e.g., hernia repair, Nissen fundoplications), and during procedures where retraction is occurring outside the field of view (e.g., lower anterior resection, cholecystectomy). Surgeons are loath to grasp and regrasp fragile tissues with high consequences for tears due to slip (e.g., small bowel, blood vessels). Experience and care are required to apply just the right amount of tension to fragile tissue.
Surgeons tended to describe slip as annoying, frustrating, and time-wasting. However, when slip does occur and bleeding or tearing results, significant time and effort is required to fix it and is desirable to avoid. Because time in the operating room costs $20–$46 per minute , the cost of a single slip event can range from $1.60 to $230, and surgeons report slip occurring twice or more during a typical RAS case.
Most surgeons reported responding to slip by changing the angle of their grasp. However, when handling fragile tissue, surgeons were more likely to either change nothing about their grasp or to change the tool completely than during retraction or view setup. Most surgeons reported requiring 5–30 s to recover their prior operating pose as a result of tissue slip, although a non-negligible percentage reported spending a minute or more (Fig. 3). The longest recovery times were reported during handling fragile tissue; surgeons reported working more slowly to limit tissue tearing, damage, and bleeding.
Surgeons were asked to rank in order of (hypothetical) preference how they would like to receive feedback for slip of biological tissue during RAS. Surgeons' responses differed greatly when considering receiving feedback on a tool they were actively controlling as opposed to a nonactive instrument. However, whether the tool was actively controlled or not, surgeons overwhelmingly reported that they would prefer to receive some kind of feedback as opposed to nothing: >70% listed ignoring slip as the least preferred option for active and nonactive tools. Reported feedback preferences for slip in an actively controlled tool showed no clearly superior feedback modality. However, for providing slip in a nonactive tool, surgeons strongly favored receiving auditory cues as their top preference (48%), followed by visual displays either on the screen edge or on the tool (15% and 16%, respectively). Logically, for the nonactive tool, users did not prefer receiving haptic feedback (as either vibrations or resistive forces, <8% for each) because their hands would already be engaged with the other active tools (Fig. 4).
The overall results showed that slip is both a frustrating and common occurrence. Surgeons tend to work with three tools and switch among them, keeping two under active manual control and the focus of their visual attention, while the nonactive tool continues to passively hold tissue. Although slip at the active tools can result in bleeding, tearing, or loss of exposure, slip at the nonactive arm is unique because the same consequences apply but occur outside the focus of the surgeon's attention. This finding motivates the three-tool user study detailed in Sec. 4.
Purpose and Methods.
This characterization study sought to validate the tissue slip sensor  on living, perfused porcine tissues. We anticipated that the convective effects of blood flow (either in highly perfused tissue like mesentery or in a blood vessel with strong flow) may affect the performance of the device. In addition, we tested fat because its thermal conductivity is quite low when compared with other biological tissues (approximately 0.2 W/mK versus 0.5 W/mK ), although the model-based thermal prediction presented in Ref.  did not indicate that this would negatively impact the slip sensor's performance. Finally, we wished to compare the new data with our previous ex vivo results  to determine whether the device under in vivo conditions has comparable sensitivity. With these considerations in mind, we tested mesentery, artery, fat, and bowel.
where and are time derivatives of the differences in signals between thermistors on opposite sides of a miniature heating element. The software was modified to increase sensitivity to slips at low speed by reducing the low-pass filter frequency from 5 to 1 Hz.
Repeated measurements of slip were taken on in vivo tissues from a single porcine model (Fig. 4). Video recording using the da Vinci endoscope (30 fps, 736 × 486 pixels resolution) provided a ground truth measurement for timing and speed of slips. Digital image correlation (detailed in Ref. ) was used to calculate the amount of slip that occurred in each video. All motion tracking was done using the open source matlab tool DLTdv5 .
Figures 5 and 6 show results from the versus slip validation tests on in vivo tissue. Figure 5 shows how the tissue slip and resultant signal vary over time, and Fig. 6 compares to our ex vivo results from Ref. . While the past slip threshold was 5 standard deviations above the mean, the threshold here was set at 1.3 × 10−3 [°C/s]2, the value which participants chose most often during the user study in Sec. 4.
There was little difference in the results among the four tested in vivo tissues (fat, mesentery, artery, and bowel), as anticipated from the results reported in Ref. . There were no adverse effects due to tissue perfusion, even when grasping mesentery (which contains small, very superficial vessels) and a large, strongly pulsating artery (the mesenteric artery). Surgeons tend to avoid grasping such arteries because the risk of bleeding is too high. However, should they elect to grab near or on one anyway, the sensor has demonstrated robustness to the effects of bloodflow.
The signal produced in response to slip on in vivo tissue is not as large as that produced on ex vivo tissue. This is because the tested slip speeds were lower than those tested in Ref.  to ensure safety of the porcine model (Fig. 7). In previous testing, we observed that speed had a similar limiting effect on the peak slip signal obtained. Therefore, we feel confident in stating that the differences seen in Fig. 6 are primarily due to speed rather than perfusion or starting temperature. Far fewer trials (3–7 versus 30–50) were taken due to the live porcine model being a shared resource. However, with the same slip detection threshold used throughout, slip was detectable on all in vivo tissues well below our 3 mm specification as developed in Ref. , which allows provision of timely information to surgeons. Figure 7 relates onset slip velocity to detectable slip of the in vivo tissues; the ex vivo data are shown for comparison.
Motivated by results of the survey in Sec. 2, a study was conducted to evaluate whether experienced users would benefit from slip detection and feedback during RAS procedures. In particular, it was hypothesized that feedback from a nonactive tool (i.e., one that is not under direct manual control at a particular time) would be beneficial. For actively controlled tools, experienced users have become proficient at using visual cues to predict the onset of slip, so the benefit of additional feedback is less likely to be apparent. However, on a nonactive tool, a slip sensor has the opportunity to provide otherwise unobservable information.
Methods and Materials.
The hardware and software presented in Ref.  were used in this study with a minor modification to the algorithm to detect slip only when the tool jaws were closed and the tool was inactive. This helped reject false positives (due to uneven tissue contact) and unwanted slip alerts from an actively used tool.
With approval from the institutional review board of Stanford University (Protocol #46474), we recruited subjects from within Intuitive with experience in RAS surgical procedures, including switching between tools, moving the camera and clutching to recenter the workspace. No honoraria were provided. Twenty-four subjects participated (see Fig. 11 for demographics). Many participants are clinical development engineers (CDEs) at Intuitive who have extensive knowledge of anatomy, physiology, and surgery and help define and validate the clinical risks and requirements in developing products.
Several iterations on the task construct were tested in pilot studies before selecting the final construct. Important considerations included a requirement for user skill and dexterity, and the need to grasp and manipulate while dividing attention among multiple objectives. In the chosen scenario, users had to complete a dexterous manipulation task as quickly as possible while also maintaining situational awareness and attending to the periphery for unexpected events. If the task was too difficult, users became frustrated and did not take advantage of feedback in a seemingly impossible task. However, if the task was too easy, they would ignore the feedback because they believed they could finish before slip at the nonactive tool would matter.
Each subject received instruction on the study task, which was designed to mimic the situation in surgery where the surgeon is using the third arm to provide traction on tissue and thus exposure so they can perform some intervention with their other two (active) arms (Fig. 8). Synthetic chamois was chosen for its tissue-like properties—it retains moisture, is slightly stretchable and has similar friction to skin . As noted in Sec. 3.1, the ability to detect slip was not substantially different for live tissue; however, synthetic tissue is logistically easier to use in a series of tests.
The user was required to use the slip sensing tool (3 in Fig. 8) to retract the tissue (chamois, dotted with black ink markers, and wetted) behind a wall, then use two graspers (1,2 in Fig. 8) to loop the tissue over two target posts.
During half of the trials, subjects received auditory slip feedback as an audible chirp when slip onset occurred at the slip tool. Auditory feedback was selected based on the survey results (Sec. 2.2). During training, subjects explored the feedback's responsiveness and tuned the threshold to suit their preferences. During testing, they could request further changes to the slip sensitivity. The majority of users chose a slip threshold of approximately 1.3 × 10−3 [°C/s]2 which was also used as the threshold in the in vivo testing in Sec. 3. The other half of the trials had no auditory feedback. All trials were recorded for data collection purposes. As shown in Fig. 9, slip checking occurred only when the slip tool was not active and had been grasping tissue for ≥ 1 s.
Subjects were instructed to keep the endoscope 200–400 mm from their active workspace. This requirement combined with the setup dimensions forced them to have their third arm offscreen often (see Fig. 10). This forced subjects to rely on visual cues (the movement of the chamois dots, or exogenous tension in the material) in the absence of slip feedback.
Over each subject's 12 test trials, 7 external slips occurred. External slips involved the protocol director applying tension from out of view to the chamois via control strings, slowly enough that it would take 3–10 s for tissue loss to occur at the third arm. Tension was applied at 0.5–0.7 N/s up to a maximum pull force of 2.5 N. Subjects were instructed to respond to slip by moving the endoscope to check on the third arm. The external slip would stop as soon as they began responding. The external slips were designed to replicate observed or described RAS slips that may be outside the surgeon's control, due to breathing, organs shifting, poor coordination with an assistant, etc. Artificially increasing the frequency of slips while maintaining their apparent randomness and preventability was required for an efficient study because, as noted in Sec. 2.2, RAS normally occurs only 1–2 times over a case. The number and timing of occurrences were predetermined and randomized prior to testing, and external slips were present during training.
To elicit measurable responses from study participants, the demands were exaggerated over that in a typical RAS case. Subjects were informed that their task score was determined by their time to complete it (1 penalty point per second) and if loss of tissue ever occurred at the slip tool (100 penalty points, plus restart of the task). This penalty for catastrophic slip from the retracting arm combined with the time constraint was intended to recreate the division of attention in surgery. Although surgeons' main focus is on using their two active tools to complete an intervention, they must also maintain awareness of their surroundings. This penalty was deemed reasonable and appropriate to represent the possible consequences of slip, which can include restarting a procedure if exposure is lost, or spending minutes repairing torn or bleeding tissue.
After 4 training trials, 12 test trials were conducted: 6 with auditory slip feedback and 6 without in randomized order. Midway through testing, the protocol director asked participants questions about their experience completing the task with and without feedback: did they like or dislike slip feedback; did they feel more, less, or similarly confident with slip feedback; what feedback modality might they prefer during a real surgery? If the subject was a CDE or a surgeon, they were also asked to name any surgical processes during which they would anticipate wanting to receive slip feedback. After testing, subjects completed the surgery task load index  to self-assess the surgical workload for tasks with and without auditory slip feedback. Video data of test trials were used to evaluate users' responses to external slips.
Figure 11 shows the demographics and experience levels of the 24 study participants. Of the two surgeons who were tested, one was a general surgeon and one was thoracic. All users were highly familiar with using the da Vinci Xi but had differing experience levels, which were classified into “wet” and “dry” hours. “Wet hours” refer to RAS experience with biological materials, such as human, porcine, canine, or cadaveric models. “Dry hours” refer to RAS experience with nonbiological materials, like rubber bands, etc. CDEs tended to have more wet hours than nonmedical participants.
Each subject experienced seven slip events with auditory slip feedback and seven without. Their responses to each slip event fell into one of three categories: (1) they caught slip before it resulted in grasp loss, (2) they dropped the tissue because they did not respond in time, or (3) the slip went unnoticed or ignored, meaning the slip did not result in tissue loss but did not cause a response from the subject. In the case where subjects received auditory feedback, no slips were unnoticed because they received an audible cue each time. Figure 12 shows how participants responded to slips in the feedback and no feedback cases. Without auditory feedback, subjects dropped the tissue or did not respond to slip far more often than with feedback.
Table 1 shows some additional test metrics. The time to complete the task was significantly lower during trials with feedback, and subjects responded to slip significantly faster as well (2.8 s versus 3.3 s). This faster response could be further improved by tuning the sensor to respond faster to slips. Subjects' response to slip during feedback trials had two components: their own response to the auditory beep, and the slip sensor's response to the slip event which they themselves tuned. The takeaway is that given an auditory signal, subjects can respond in 1.3 s on average, whereas without it they require nearly three times that amount of time.
When subjects responded to slips with and without feedback, there was no significant difference in how long it took them to recover from the event. Typical responses involved moving the endoscope back to visualize the third arm, then using one or two other tools to help the third arm regrasp.
When auditory feedback was enabled, there were times when the sensor would alert subjects to a very minor slip (1.2 ± 1.3 times per trial), and subjects spent 1.9 ± 2.3 s responding to each noncritical event. Subjects disliked receiving these alerts and reported finding them disruptive. Classifying these as false positives is incorrect because slip in the jaws did occur, but these alerts frustrated subjects instead of helping them perform well on the task. Subjects sometimes requested elevated sensor thresholds after experiencing these noncritical alerts to reduce their occurrence. However, without feedback, subjects felt compelled to check on the third arm far more often, a total of 74 times as compared with only 10 total for the trials with feedback, and they reported disliking this as well. Subjects spent approximately the same amount of time glancing back with or without feedback (2.8 versus 2.7 s), but spent less time checking on the third arm when feedback was present. This may help to explain subjects' reported preference for feedback versus none.
After the manipulation portion of the study was done, subjects were asked to score their perceived workload during the trials with and without feedback using the surgery task load index (see Fig. 13 for description). The subjects' perceived mental demands, temporal demands, and situational stress all decreased significantly when they had auditory feedback; the other three workload ratings did not change significantly.
Halfway through testing, subjects were informally interviewed. All 24 subjects reported liking the slip feedback because they felt they could focus more on their active tools and accomplishing the task. They also felt they could ignore the periphery and enjoy a heightened ease of mind, relying on being told when they needed to respond to slip rather than glancing back to check. The vast majority of subjects reported feeling more confident in their ability to perform the task when feedback was present; only two subjects felt no change in their confidence, and none felt it diminish. Subjects generally agreed on how they wanted to have slip feedback implemented. They wanted only to receive feedback when slip would be catastrophic. However, their perception of what indicates impending catastrophe differed; subjects cited either high slip speed or small contact area as their desired metric.
When asked whether they liked the feedback modality, subjects had mixed responses. While most were content with auditory feedback, they felt that a more distinctive noise would be needed in an operating room because many other machines are constantly beeping as well. A third of subjects reported preference for a visual indication of slip over auditory, with several describing a display similar to the current da Vinci Xi offscreen indicator (a feature that provides an overlay on the user interface when an instrument exits the endoscope's field of view). No subjects reported preferring any form of haptic feedback, and most stated explicitly that receiving haptic feedback for a nonactive tool would be confusing and detrimental.
Finally, CDEs and surgeons were asked to reflect on the surgical procedures during which they would find slip feedback useful. Several procedures were recommended because important retraction can occur offscreen, e.g., cholecystectomies or any large intestine work, particularly in lower anterior resections. If additional traction is required, surgeons report applying it to the rectum without visualizing the (already offscreen) tool. Both surgeons reported that they may elect to retract with a wide field of view then zoom in to focus on a small region, which forces the third arm offscreen; loss of grasp is only known once they have already lost exposure. Other procedures were recommended because the retracted anatomy or object posed grasping challenges due to low surface friction (e.g., a lubricated Foley catheter) or requiring very high traction (e.g., Nissen fundoplication).
Slip of grasped biological tissue in surgery is an understudied problem that is worthy of our attention. This paper is based on the hypothesis that it would be desirable to detect slip—particularly as RAS systems may implement shared autonomy with one or more tools not under direct manual control. In supporting this work, Intuitive seeks to gather a deeper understanding of the challenges surrounding tissue grasping and manipulation to improve safety and the surgical experience.
Loss of grasped tissue can result in an unplanned reorientation of tissue, which can put vital structures at risk and cause tearing or bleeding. The consequences can be more serious if the surgeon is using electrocautery. At a minimum, having tissue slip from a grasper adds length to a procedure and adds to overall surgeon frustration and workflow disruption. We believe that the work presented here provides a convincing argument to pursue technology that improves grasping and manipulation. By itself, grasp force sensing provides only a piece of the information required to assess a reliable and atraumatic grasp and prevent slip.
The results from our survey motivated a user study to determine the benefits of slip feedback in a three-tool RAS procedure where offscreen tissue retraction was required. The task provides insight into future scenarios which may involve shared autonomy or automation of low-level surgical tasks. These scenarios will require more information regarding the system's interaction with biological materials and a heightened ability to communicate that information with the human user. To that end, testing the utility of supplying feedback to a tool that is not controlled by the human user can provide important insights.
The purpose of the user study was to explore whether participants would find feedback helpful if it provided information regarding tissue slip from the nonactive tool. The choice to focus on nonactive rather than active tools was motivated by the reasoning that there is likely a ceiling on performance of simple tasks, for which adding sensory feedback (or any other kind of assistance) will not result in an improvement in performance metrics with expert subjects. However, on the nonactive tool, a slip sensor can provide helpful and otherwise unobservable information.
We conducted a survey of 112 surgeons experienced in RAS. The study revealed that slip is frustrating and can prolong a procedure. Surgeons noted that slip in MIS can have serious consequences if the grasped tissue is fragile, electrocautery is on, or if slip is accompanied by tearing and/or bleeding.
We also presented new validation tests in a live porcine model on a sensor previously reported in Ref. . The tests demonstrated the validity of the sensor across a variety of tissue types and its robustness to perfusion in both mesentery and in strong arteries in a realistic surgical environment.
Having ascertained that the anemometric slip sensor works equally well for in vivo and ex vivo tissue, the user study task used a synthetic substitute for tissue. The results from the user study showed clearly that study participants perceived lower workloads during task performance and dropped tissue far less with slip feedback present. This information can enable prevention of grasp loss or at least termination of electrocautery to prevent additional damage. The presented results provide a solid incentive to continue this line of work toward producing a production-level tool with integrated slip sensing.
Next steps would seek to implement the sensor within the footprint of an unmodified tool and prepare it for use in a full RAS case. Exploring methods to ensure sensor functionality in the presence of electrocautery noise and creating a sensor package that can survive sterilization and autoclaving are remaining technical challenges. Whatever package is chosen, careful consideration of how signal wires are routed back out of the cannula from the tool's jaw is required. Implementation of an onboard microcontroller will help reduce the number of wires required (currently, seven are necessary).
An investigation into sensor fusion to provide more useful information to surgeons may prove fruitful as well. A frequent request from user study participants included a desire to know slip speed and/or grasped tissue surface area in addition to a notification regarding detection of slip onset. It is possible that combining this information to only provide notification of slip onset when the slip speed exceeds some threshold and/or when the remaining covered surface area is low may be most advantageous for users. Either of these conditions may provide a more effective indication of impending grasp loss.
Finally, an exploration of alternate feedback modalities for presenting slip information would be useful for the final implementation. Although the majority of user study participants reported preferring auditory feedback over any other modality, both tested surgeons anticipated preferring visual feedback. All medically trained participants agreed that the operating room is typically both visually and aurally saturated, so the slip feedback would need to be provided only when correct and relevant in order to draw the user's attention.
This work is a collaboration between the Biomimetics and Dexterous Manipulation Laboratory at Stanford University and Intuitive. All work was performed at Intuitive's Advanced Product Development Research laboratory in Sunnyvale, CA.
Intuitive Surgical (UBHKQ; Funder ID: 10.13039/100010477)