We are excited to introduce OpenCR-MuJoCo, an open-source project that brings tendon-driven continuum robots (TDCRs) into native MuJoCo through a mechanics-informed representation.

The motivation is simple. Continuum robots are compelling because they bend, conform, and make compliant contact along their whole body. That distributed compliance is one of the reasons they are promising for constrained manipulation, medical robotics, inspection, and delicate physical interaction.

It is also one of the reasons they are hard to simulate. For many existing learning and manipulation workflows, mature simulation infrastructure has become part of the research loop: fast rollouts, contact-rich scenes, robot arms, objects, actuators, teleoperation, and standard interfaces to learning code. In continuum robotics, we have a strong modeling foundation, but it is often harder to plug that foundation directly into the everyday simulation workflows used for contact-rich robot learning.

OpenCR-MuJoCo is our attempt to make that easier.

Building on the Existing Foundation

Continuum robotics has benefited enormously from physically grounded simulation methods. Cosserat rod models, finite-element methods, discrete elastic rod formulations, and tools such as SoRoSim, SOFA, Elastica, DisMech, and related simulators have shaped how the field understands continuum robot mechanics. These methods remain important, and our work is very much built on the foundation they provide. Here, we ask a more infrastructure-oriented question:

Can we put a TDCR directly inside MuJoCo in a way that is simple, fast, interpretable, and accurate enough to support real robot workflows?

The answer we explore in OpenCR-MuJoCo is deliberately simple in nature. We discretize the continuum backbone into a serial chain of rigid links connected by elastic joints. The key is that the joint stiffnesses are derived from the robot’s material and geometric properties, rather than treated only as arbitrary fit parameters. Tendons are routed through sites along the backbone and actuated using MuJoCo’s native tendon model.

This lets the TDCR live as a native MuJoCo model: it can contact objects, share a scene with a robot arm, use standard MuJoCo actuators and tendons, and connect naturally to teleoperation and learning code. At the same time, the model keeps an interpretable relationship to the underlying continuum mechanics.

What We Contribute

The main contribution is a practical way to represent and use TDCRs inside native MuJoCo, built around this mechanics-informed discretization.

On the modeling side, the TDCR backbone is represented by rigid links and elastic joints whose stiffness comes from beam theory. As the discretization is refined, the approximation converges toward the continuum rod model, while each parameter still has a physical interpretation. In practice, this gives a useful balance: the model is simple enough to run quickly inside MuJoCo, but structured enough that changing material, geometry, tendon routing, or discretization resolution remains meaningful.

On the systems side, the model is encoded in standard MJCF and uses MuJoCo’s existing physics pipeline. That means native tendon actuation, contact handling, rigid-object interaction, and robot-arm integration happen in one MuJoCo scene. For contact-rich manipulation, this is especially useful: the contact locations and forces do not need to be specified ahead of time, but can emerge from the interaction between the compliant robot body, the objects, and the environment.

This is the part we are most excited about for the community. If continuum robots can be easier to instantiate, control, validate, and benchmark in a common simulation environment, then more researchers can compare methods on shared tasks instead of rebuilding similar infrastructure from scratch.

Validation: From Mechanics to Hardware to Policies

A modeling workflow only becomes useful when researchers can trust it for the questions they want to ask. For OpenCR-MuJoCo, we evaluated that trust at three levels.

First, we compare against SoRoSim, a well-established Cosserat rod simulator. This is an important reference point because it tests whether the MuJoCo discretization behaves like a continuum-mechanics model, not merely whether it looks plausible. We evaluate static equilibrium shapes and dynamic tip-release trajectories across two materials spanning a large stiffness range, randomized loading conditions, and multiple discretization resolutions. In the practical operating regime used in our experiments, the static and dynamic errors are below 1% of robot length while maintaining real-time simulation.

Second, we compare against real hardware. We calibrate the MuJoCo model to a physical three-segment, nine-tendon TDCR. After system identification, the model reaches a mean tip tracking error of about 7.7 mm, or 4.1% of the 186.5 mm robot length, on held-out hardware data. This test is important because real TDCRs include effects that are difficult to model perfectly, such as friction, hysteresis, assembly tolerances, and tendon pretension.

Third, we test whether the native MuJoCo representation is useful for policy learning and sim-to-real transfer. We collect teleoperated demonstrations in simulation, train state-based imitation-learning policies, and deploy them zero-shot on the physical robot. The tasks are intentionally contact-rich: one policy wraps the continuum body around a cylinder to lift and drop it into a bin, while another flips a small wall-mounted switch from behind. In the reported experiments, real-world success rates match or slightly exceed simulation: 76% for the grasping task and 78% for the switch task.

We see this policy transfer as the ultimate practical test of the approach. Matching a reference model is necessary, and matching hardware trajectories is encouraging, but contact-rich manipulation asks whether the MuJoCo integration supports a full research workflow: model generation, calibration, teleoperation, data collection, policy training, and real robot deployment.

What Is Included

OpenCR-MuJoCo is meant to be research infrastructure, not just a static release for one paper. The repository includes:

• A config-driven TDCR generator for material-based stiffness, pretension, collision settings, and modular heterogeneous robots.
• Native MuJoCo scenes for TDCR-only setups and TDCRs mounted on a Franka Panda arm.
• Teleoperation and controller tools for joint- and task-space TDCR control.
• Evaluation scripts and bundled reference data for reproducing the validation.

For a new user, the intended workflow is lightweight: choose a JSON configuration, generate a MuJoCo scene, and run it with our teleop controller or a custom control loop.

Our Vision for Future Work

We are releasing OpenCR-MuJoCo because we want continuum robotics to have simulation infrastructure that is easier to use, easier to reproduce, and easier to benchmark with.

Many groups build different TDCRs with different tendon routings, different materials, and for different tasks. That diversity is valuable. At the same time, it can make methods difficult to compare. We hope OpenCR-MuJoCo can provide one useful common ground.

There is still a lot to do. OpenCR-MuJoCo currently focuses on slender tendon-driven continuum robots where the Kirchhoff rod assumptions are appropriate. Other soft robot morphologies, richer sensing, shape feedback, vision-based policies, broader benchmarks, and deeper comparisons between learning methods are all open directions.

If you work on continuum robot modeling, control, learning, hardware design, or benchmarking, we would love for you to try it and tell us what would make it more useful. Bug reports, feature requests, benchmark ideas, new robot configurations, and pull requests are all welcome.

Try the demo, clone the repository, reproduce the validation, or build a model of your own TDCR! We are looking forward to seeing what the community does with it.

Project Page GitHub Paper

In this post, we present a tutorial on how to use our open source sb-ftl-cr-planner codebase. With our implementation, you can define a 3D curve and execute our motion planner to perform FTL motion along the curve. The planner is implemented in Python and uses Matplotlib for visualization.

Problem Defintion

FTL motion is roughly characterized by having the CR body follows the path traced by its tip during insertion ¹. The desired path is given as an order set of 3D waypoints \(\mathcal{W}=\{\mathbf{w}_1, \mathbf{w}_2, ..., \mathbf{w}_n\}\). To formalize the requirments of FTL motion, we define two properties that should hold throughout the trajectory.

1. \(\textbf{Tip tracking}:\) The tip should be placed at waypoint \(\mathbf{w}_i\) when passing through it, and follow a smooth interpolation (e.g, linear) between consecutive waypoints.
2. \(\textbf{Shape following:}\) The inserted portion of the robot should minimize deviation from the corresponding segment of the path.

We consider a CR mounted to a 6-DOF base, representing practical systems where CRs are attached to serial manipulators. The CR configuration space is denoted by \(\mathcal{Q} \in \mathbb{R}^d\) and the combined system configuration space is \(\mathcal{K}=SE(3)\times\mathcal{Q}\). The goal of a FTL motion planner is to generate a sequence of \(N\) configurations \(\{(\mathbf{T}_{b,1}, \mathbf{q}_1),...,(\mathbf{T}_{b,N}, \mathbf{q}_N)\}\) that guides the robot through the path \(\mathcal{W}\) while obeying the two listed properties above.

Code Organization

The planner can be ran in headless mode or with a visualizer using Matplotlib. Installation instructions can be found in the README here. Three core classes are utilized in this repository.

Robot Class

The robot class ContinuumRobotModel(num_segments, segment_lengths, tendon_offset, *args, **kwargs) contains necessary helper functions used throughout the planner. Notable is the forward_kinematics(clark_coordinates: np.ndarray) which returns an array of points along the robot backbone. Forward kinematics is left as an abstract unimplemented method in the parent class. Our planner is model agnostic meaning any forward model (e.g, Constant Curvature, Cosserat Rod, Simulation) can be used. We provide a ConstantCurvature(*args, **kwargs) child class that implements the forward kinematics based on the the constant curvature model. To add in your own model, all you have to do is inherit the parent ContinuumRobotModel class and implement your own forward kinematics function.

Motion Planner Class

The FTL motion planner is implemented as inherited classes of the parent class GeneralMotionPlanner(*args, **kwargs). Similar to the robot class, it contains helper functions with the follow_path() function to be implemented in child classes. We provide 3 planners

1. No Cluster/Linear Sampling
2. Threshold Cluster
3. Direct Optimization

which all inherit the parent GeneralMotionPlanner class. Each have their own implementation of follow_path() and discussed further in the paper.

Desired Curve Class

To define the desired path for the CR to follow, the TaskGenerator(*args, **kwargs) class is used. Once initialized, this class has many methods to generate 3D curves. Some of the provided curves are,

1. Quadratic bezier curve: generate_curved_path()
2. Cubic bezier curve: generate_cubic_bezier_path()
3. Circular arc: generate_c_shape_path()
4. Straight zig-zag: generate_z_shape_path()
5. Spiral curve: generate_spiral_path()

This list is not exhaustive, but if nothing fits your use case, feel free to define your own custom path. As long as the curve is a numpy array of 3D points, our planner can follow it!

Code Walkthrough

We will now walkthrough the provided starter script run_example.py. You can do a quick run with

python run_example.py

or use any of the provided command line arguments (python run_example.py --help). A few useful variations are,

# Linear-search baseline on an S-shape, 20k library, no animation
python run_example.py --planner "No Cluster/Linear Sampling" --curve s \
    --num-samples 20000 --no-animate

# Use an externally produced shape library (skips library generation)
python run_example.py --shape-lib-path /path/to/library.json

# Save the dense motion plan (Clarke coords + base SE(3) + shape per step) to JSON
python run_example.py --save-history my_plan.json

First, the robot model is initialized with the desired system parameters.

robot = ConstantCurvature(
    num_segments=3,
    segment_lengths=[1, 1, 1],
    tendon_offset=[0.2, 0.2, 0.2],
    points_resolution=0.05,
)

After the robot is created, we define the desired path using the provided helper function and command line arguments,

generator = TaskGenerator(robot)
waypoints = _build_curve(generator, args.curve, args.num_waypoints, p1=args.p1, p2=args.p2)

by default, a quadratic bezier curve is created,

generator.generate_curved_path(
    np.array([0, 0, 0]),
    np.array([2, 0, 1.3]),
    np.array([0, 0, 1.6]),
    num_waypoints=num_waypoints,
)

Now it is time to create the shape library! The high dimensioal configuration space of \(\mathcal{K}\) and nonlinear CR dynamics make iterative optimiztion approaches computationally exspensive. Generating a shape library offline then performing global shape search is the technique that makes our method Model agnostic, Efficient and Theoretically Guaranteed. These benefits are discussed more in the paper.

general_follower = GeneralPathFollower(robot)
sampler = general_follower.get_sampling_methods_by_name([args.planner])[0]
...
follower = sampler(**sampler_kwargs)

Initalizing the sampler class generates the shape library using the provided robot object and saves the library as an attribute.

Generating a Shape Library: A core feature of our motion planner is that the shape library can be generated offline and reused. This enables the ability to generate very large or computationally exspensive libraries with complex models beyond just Constant Curvature once, and not have it effect the runtime of the planner. In run_example.py we generate a fresh library on each run as the cost for generation with the ConstantCurvature forward model is low and takes less than 30 seconds with 20,000 shapes.

Here is a visualization of 60 shapes from a shape library,

Finally, lets Follow-The-Leader!

history = follower.follow_path(waypoints)
interp_history, interp_waypoints = general_follower.interpolate_mp(
    history, steps_per_waypoint=args.interpolation_steps, enable_optimization=True
)

follow_path() generates the sparse motion plan, one configuration \((\textbf{T}_{b, i}, \textbf{q}_{i})\) for each waypoint \(\mathbf{w}_i\) while interpolate_mp() fills in the gaps between each waypoint by intelligently interpolating between each configuration from follow_path().

Once the motion plan is created, we include support to record deviation statistics and save the motion plan to disk.

Matplotlib is used to visualize the motion plan,

visualizer = PathVisualizer(robot)
visualizer.plot_history(history, waypoints, show_animation=True)
visualizer.plot_history(interp_history, waypoints, show_animation=True, animation_interval=0.01)

both the sparse motion plan and interpolated motion plan are animated. The animation for the sparse plan (left) and interpolated plan (right) are shown below.

Happy tendon-driven continuum robotic simulating!

Paper GitHub Website

References

In this blog we present a tutorial on how to use our open source OpenTDCRContactModel. With our code, you can build custom taskspaces with circular obstacles and apply our contact-aided search to tendon driven continuum robots to reach a target position in a 2D taskspace. The model is implemented in C++ and Matlab. We also provide an easy to use interface written in Python. For a quickstart, have a look to our Google collab demo notebook.

Model Defintion

The model is posed as a nonlinear optimisation problem. The robot is said to be defined by ‘m’ piecewise constant curvature arcs. The objective of this problem is to find a set of m curvatures that minimize the robots potential energy. The nonlinear equality constraint makes sure that the input tendon length is equal to the calculated tendon length. The nonlinear inequality constraint ensures that points on the robot lie outside all the obstacles.

Code Organization

Installation instructions along with the source can be found here. The two important classes to know for this implementation is the Robot(radius, disk_num) class and Node(Robot, length, actuation) class. Node is used to describe the robot at some point in joint space. Node also has a method run_forward_model(taskspace, bool_reduced, model_type) which calculates the forward kinematics of the robot given its current joint space configuration. Two options are avaliable for model_type, model_type="KINEMATIC_MATLAB" or model_type="KINEMATIC_CPP". You can access the Node’s cartesian coordinates with Node.ee after calling run_forward_model.

The Robot class stores general configuration parameters of the robot like tendons, length and radius. Since the model uses a piece-wise constant curvature arc representation, the number of disks is used to discretize the backbone into that many subsegments. The robot object is then fed into each node object as a parameter.

Note: The robot lies in the xz plane and the model currently supports circular obstacles and 2d contact interactions.

Code Walkthrough

Taskspace

Obstacles in the taskspace are defined by the Circle(radius, (x, y, z)) class. More complex shapes can be formed by superimposing multiple circles.

Defining a taskspace is easy, just initialize a taskspace object then set each obstacle you want in your taskspace. Or alternatively, a default taskspace in /workspaces can be used.

workspace = TaskspaceCircle()
radius = 0.01
obstacle_1 = Circle(radius, (0.02, 0.0, 0.03))
obstacle_2 = Circle(radius, (-0.04, 0.0, 0.03))
obstacle_3 = Circle(radius, (0.08, 0.0, 0.03))
workspace.set_obstacles(obstacle_1, 1, 1)
workspace.set_obstacles(obstacle_2, 1, 1)
workspace.set_obstacles(obstacle_3, 1, 1)

The parameters for set_obstacles are,

# workspace.set_obstacles(Circle, n, z_delta)
#     Circle: new obstacle to add to the taskspace
#     n: add n copies offset by z_delta
#     z_delta: distance to offset the n copies

Robot

Once the taskspace is created, a TDCR Robot object is also created by specifying the radius and number of discrete disks composing the robot. A Node object is also created representing the robot at some configuration in time. In the starting position, the Node will be set to have 0.001 length and 0.001 actuation.

      # mod_cr.Robot(radius, disks)
robot = mod_cr.Robot(6e-3, 30)

            # Node(Robot, length, actuation)
config_init = Node(robot, 0.001, 0.001)
config_init.set_init_guess(np.array([1]*robot1.nd))
config_init.T = np.eye(4)

# Run First forward kinematic iteration
config_init.run_forward_model(workspace, True, "KINEMATIC_CPP")

Motion Planning

Next a motion plan must be created. A target location can be specified and the following method will generate a sequence of joint space values (tendon actuation and tendon length). This sequence will then be applied to the robot with forward kinematics to traverse it through the taskspace. Paths can be created using the following taskspace method. This method will save a generated path to a Nx2 csv file with each row being formatted as segment_length, tendon_length

taskspace.generate_path(initial_Node, target=[x, y, z], filename='filename')

This method performs a greedy heuristic search through joint space sequences using Euclidean distance as a metric. More details can be found in Rao et al.² The heuristic can be replaced with a custom heuristic you wish to investigate. Feel free to play around with it. The heuristic is defined in /utils/taskspace.py. Alternatively, you may load one of the premade path located in /sample_paths.

Forward kinematics

After the path is created, we then iterate over each position in the joint space csv file previously created. The forward kinematics of each value is calculated and saved to a list traced_path.

prev_guess = config_init.var[0,::3]
sample_path = load_and_extract('sample_path.csv')
traced_path = [config_init]*len(sample_path)
for idx, iter in enumerate(sample_path):
    curr_node = Node(robot1, iter[0], iter[1])
    curr_node.set_init_guess(prev_guess)
    model_exitflag = curr_node.run_forward_model(workspace, True, "KINEMATIC_CPP")
    if model_exitflag:
        prev_guess = curr_node.var[0,::3]
        traced_path[idx] = curr_node
    else:
        print("Model did not converge")
        break

Selecting an Initial Guess: The step on setting an initial guess with config_init.set_init_guess is important! As explained in the Model Definition section, the model used in our simulation environment is a complex nonlinear optimization problem. Additionally, the mapping between joint space and task space is not a one-to-one mapping during implicit contact motion. Setting a correct initial guess is crucial for model convergence. A rule of thumb that has worked well for us is to run the forward kinematics solver sequentially. This is done by setting the initial segment length and tendon length of the robot to be (0.001, 0.001) or as 1mm in units for each. Then the model slowly progresses in 1mm increments to remain stable. In each time step, the curvature values of the previous time frame is used as an initial guess for the next.

Visualizing a Configuration

After the model has been run, and the nodes curvature values are determined, we can visualize the robot and taskspace by plotting it. The red dot is our target position.

config_init.plot_configuration(workspace)
helpers.saveFigure()
# plt.show()

Visualizing a Motion Plan

For every motion plan that is generated and simulated, it can be visualized as a mp4 video.

helpers.visualizing(traced_path, workspace, "filename")

Using the default taskspace and 3_sample_path.csv, the resulting video is:

Happy tendon driven continuum robotic simulating!

Paper GitHub Google Collab

References

While changing the robot design mechanically is viable, we asked ourselves whether we can harness more from the traditional single segment straight routing design to achieve more variation than just a constant-curvature. What if these additional forces/moments came from the environment instead of mechanical additions - which would be possible if the robot now leveraged the obstacles to change its curvature? This idea inspired our research on contact-aided navigation for these robots.

Reaching a target

Let’s consider a single segment TDCR in a planar workspace, cluttered with obstacles marked in blue. If the robot tip needs to reach the target position (red dot in the animation below), it is a fairly simple task. You basically need to select actutation values that minimize the distance between the tip and the target. We can set up a simple joint space search for this - where we consider three possible actions:

1. segment length increase (as its inserted into the workspace),
2. tendon length decrease, and
3. tendon length increase.

Starting from an initial joint value, these three actions can be sequentially applied to build a lattice-based representation of the configuration space.

A lattice is a discretization of the configuration space into a set of states, and connections between these states, where every connection represents a feasible path.³

For now, lets keep optimality aside and focus on finding a solution by performing a greedy search. The greedy search is guided by a heuristic - for our case, we can set this heuristic to simply be the Euclidean distance between the robot tip and the target. This simple heuristic works quite well! There some additional changes you can make to prevent it from getting stuck in minimas but we will leave that discussion for another day.

Reaching a target pose

Now that we have a solution to reach a target position, we can add an additional layer of challenge to our problem - How can we reach a target, with not just the desired position, but a desired orientation as well? We can retain all the machinery from the previous section, of building a lattice in the configuration space by sequentially applying joint space actions. But the more important question we need to answer is:

What would the heuristic look like to reach a target pose?

To reach a desired pose, we need to control x, y, and theta of the tip. However, we only have two controllable degrees-of-freedom where we can either change the length of the segment or bend it. The single segment is essentially underactuated for this task - leveraging contact-aided navigation allows the segment to use the obstacle as a kind-of passive degree of freedom to control the tip.

Our work, recently published in the IEEE Robotics & Automation Letters⁴, seeks to solve this problem. The heuristic basically assumes that the end-effector’s motion can be modeled as a series of constant-curvature arcs - arcs whose end-points can only lie on the surface of obstacles. This condition ensures that the use of obstacles as a degree of freeodm is captured by the heuristic.

You can watch some of the solutions found by our method in the video below :

Open-Source materials

For now, we are releasing a preliminary version of the TDCR that can account for contacts on GitHub (link below). It is based on the kinematic model proposed by Ashwin et al. ⁵.

Please watch this space for more details and tutorials!

Paper GitHub Video

References

In certain medical applications, physicians may want to inspect some region of interest for diagnostic purposes completing the procedure as fast as is safely possible to reduce costs and improve patient outcomes, especially if the patient is under anesthesia during the procedure. For example, the needle-diameter Continuum Reconfigurable Incisionless Surgical Parallel (CRISP) robot ^{1 2} was suggested to assist in the diagnosis of the cause of a pleural effusion (a serious medical condition that can cause the collapse of a patient’s lung) by visually inspecting the surface of the collapsed lung and chest wall inside the body³ (see Figure below).

Inspection Planning

This can modeled as an inspection-planning problem where we are given a robot equipped with a sensor and a set of points of interest (POI) in the environment to be inspected by the sensor. The problem calls for computing a minimal-length motion plan for the robot that maximizes the number of POI inspected.

Naively computed inspection plans may enable inspection of only a subset of the POI and may require motion plans significantly longer than an optimal plan, and hence may be undesirable or infeasible due to time constraints. Our goal is to compute kinematically feasible collision-free inspection plans that maximize the number of POI inspected, and of the plans that inspect those POI we compute a shortest one.

What makes it challenging

Inspection planning is computationally challenging because we need to simultaneously reason both about motion planning in a high-dimensional configuration space (the space of all parameters that determine the shape of the robot) as well as about maximizing the number of POI inspected.

There are multiple approaches to computing inspection plans, but here we focus on those that provide some formal guarantee on the quality of the solution. Generally speaking these methods (see, e.g., ^{4 5}) exhaustively search over the space of all motion plans thus guaranteeing asymptotic optimality. Roughly speaking, asymptotic optimality for inspection planning means these methods produce inspection plans whose length and the number of points inspected will asymptotically converge to those of an optimal inspection plan, given enough planning time.

Let’s dive deeper into the term we used earlier “exhaustively search over the space of all motion plans”. To do so, let’s go back to basics and consider shortest-path computation for a while (it may be worthwhile to visit previous posts on motion planning Part 1, Part 2). When we want to compute the shortest path to some point we can use the fact that any sub-path of a shortest path is also a shortest path. Namely, if we know that the shortest path between a and c passes through intermediate point b and we need to consider two paths connecting a and b, then the shorter one will always be better and we can discard the longer one. This is because if we take the shortest path connecting b and c it will always be preferable to append it to the shortest path connecting a and b. This is referred to as Bellman’s principle of optimality.

Going back to our problem of inspection planning this is not necessarily the case. To understand this, assume that we have two POIs and recall our two paths connecting a and b. Assume that the shorter one sees no POI while the longer one sees the first POI. Which one is better? We actually don’t know because it depends on the future: if we consider the first path, then we need to find a path between b and c that sees both POIs while if we consider the second path then we only need to find a path between b and c that sees the second POI.

To summarize, for each intermediate point, we need to consider the trade-off between short paths that see less POI and long paths that see more POI. This set has a name, “The Pareto-optimal frontier”. The set of Pareto-optimal inspection plans is the minimal set of inspection plans such that each plan is either shorter or has better coverage of the POI than any other inspection plan. Unfortunately, computing this set comes at the price of very long computation times as the size of the search space is exponential in the number of POI.

IRIS - Incremental Random Inspection-roadmap Search

To solve the slow running times of existing algorithms, we introduced a new algorithm which we called Incremental Random Inspection-roadmap Search, or IRIS, which is an asymptotically optimal inspection-planning algorithm ^{6 7}. Similar to sampling-based planners^{1 2}, IRIS incrementally constructs a sequence of increasingly dense roadmaps (graphs embedded in the configuration space) wherein each vertex represents a collision-free configuration and each edge a collision-free transition between configurations - and computes an inspection plan on the roadmaps as they are constructed.

Unfortunately, even the problem of computing an optimal inspection plan on a graph (and not in the configuration space) is computationally hard (see discussion above on computing the Pareto-optimal frontier). To circumvent this, we compute a near-optimal inspection plan on each roadmap. Basically, we use the fact that if two paths are “almost the same length” and see “almost the same set of POI”, essentially they are the same for all practical purposes and we can store only one (we have formal definitions for the “almost”s and the “essentially” but theses are out of the scope of this post).

This additional flexibility allows us to improve the quality of our inspection plan in an anytime manner, i.e., the algorithm can be stopped at any time and return the best inspection plan found up until that point. We achieve this by incrementally densifying the roadmap and then searching over the densified roadmap for a near-optimal inspection plan—a process that is repeated as time allows. By reducing the approximation factor (the mathematical tool we use to dictate which paths we consider paths as essentially the same) between iterations, we ensure that our method is asymptotically optimal.

This allowed us to demonstrate the efficacy of our approach in simulation for several complex robotic systems as demonstrated in the Figure above.

For additional details on algorithmic motion planning for continuum robots, see ⁸ and references within.

Paper GitHub

References

Steerable needles are a type of continuum robot, distinguished by their flexibility and capability to navigate through complex anatomical pathways. These robots are highly under-actuated, relying on interaction with the environment for movement and direction. The steering mechanism of steerable needles utilizes the asymmetric tip’s interaction with tissue forces during insertion. By externally rotating the needle shaft, the tip’s orientation is adjusted, allowing for controlled deflection towards desired directions upon further insertion. This method enables the needle to maneuver around obstacles such as bones, vessels, and nerves, accessing multiple targets without complete withdrawal. The reliance on environmental interaction for actuation not only enhances the safety and efficacy of medical interventions but also presents challenges and opportunities in motion planning and control within robotics, emphasizing navigation in environments with complex constraints and precision requirements.

Motion Planning for Steerable Needles

Steerable needles can be used to reach clinical targets for biopsy purposes while safely avoiding obstacles such as blood vessels. This clinical application can be modeled as a motion-planning problem which is the problem of determining a collision-free path or trajectory for a robot to move from its initial position to a desired goal position while avoiding obstacles in its environment (see also previous motion planning posts Part 1, Part 2).

Ideally, a motion-planning algorithm should first guarantee that it will compute a solution, if one exists, in finite time, or notify the user that no solution exists (we call such motion planners complete”). Moreover, the computed solution should strive to maximize some objective which in our setting is patient safety (we call such motion planners optimal). This can be quantified using metrics such as minimizing trajectory length, maximizing the minimal distance from obstacles and minimizing damage to sensitive tissue (see, e.g., ^{1 2 3 4}). Unfortunately, the lion’s share of planners developed for steerable needles offer no formal guarantee on completeness, let alone optimality.

Some prior motion planners for steerable needles (see, e.g, ^{5 6 7}) do aim to optimize motion-plan cost but they lack global optimality guarantees (this means that the cost of the solution they return may be a local optima). Some sampling-based planners (see, e.g., ^{8 9}) are known to be both complete and optimal but those properties are usually proven only when they number of samples they use approaches infinity (this is referred to as asymptotic optimality).

These challenges inspired us to consider variants of completeness or optimality relevant to medical applications: In a series of recent papers (see, ^{10 11}), we focus on specific types of guarantees relevant to real-world medical applications: resolution completeness and resolution optimality. Generally speaking, a resolution characterizes the discretization of some space such as the action space or configuration space of a robot. An algorithm is resolution complete if there exists a fine-enough resolution with which the algorithm finds a plan in finite time when a qualified solution exists, and otherwise correctly returns that no such plan exists. An algorithm is resolution optimal if it is resolution complete and if, when it does return a motion plan, the plan’s cost is guaranteed to be within a desired approximation factor of the cost of a globally optimal qualified motion plan. We illustrate this in the Figure below with an example showing searches with different resolutions for needle steering.

Resolution-Complete Search

We first presented Resolution-Complete Search (RCS), an efficient, resolution-complete motion planner for steerable needles based on a novel adaptation of multi-resolution planning ⁸. Under some mild assumptions on the system and the solution, the planner, in finite time, is guaranteed to find a plan as long as the problem admits a qualified solution. We then extended RCS to develop RCS*, that achieves resolution optimality ⁹.

We demonstrated the performance of our planners in two clinically realistic scenarios where the needle should reach a target while safely avoiding obstacles (e.g., blood vessels). In the setting of lung biopsy, where the needle is deployed through a bronchoscope and must steer through the lung parenchyma (the tissue of the lung outside the bronchial tubes) and in the setting of liver biopsy, where the needle is deployed into the liver through its anterior surface and must steer through the liver tissue. We compared in simulation our planner with several other steerable needle planners and demonstrated experimentally that RCS and RCS* outperform the state-of-the-art in terms of computation time, success rate, and plan quality.

For additional details on algorithmic motion planning for continuum robots, see ¹² and references within.

References

The idea is something I can relate to largely. Robotics Education is only just taking off in most parts of the world. With this project, I designed something that students at my home university, Manipal Institute of Technology in India, would be able to build. For the design, we considered various constraints such as component availability, 3D printer access, and skill level.

Components

The kit comprises a Tendon Driven Continuum Robot (TDCR), four actuators, two TI Launchpads, various complementary electronic components, and a custom casing to enclose the entire system. For the sake of brevity, I have omitted details thoroughly explained on our GitHub.

Manipulator

The manipulator of the TDCR consists of 10 Type-II spacer disks equally distributed along a Nitinol Backbone with a diameter of 1.4 mm. The robot is capable of 3 DOFs, namely bending across two planes and rotation about its axis. I used three independently actuated tendons made of Spectra Fiber and an actuator to rotate the backbone, enabling these degrees of freedom.

The body of the spacer disk is fixed to the backbone axially, while a bearing in the middle enables the rotational DOF. It is split into two parts to allow for the insertion of the bearing in the middle. I changed the regular ball bearing to a pulley to reduce the number of unique components. The profile of the two parts allows for easy assembly while ensuring that the actuation through the tendons is optimal. The grooves ensure the adhesive stays inside the segment and does not spill out. A small part, functioning as a spacer to make the pulley bore smaller, is designed with a unique profile that balances the required friction to grip the backbone and the need for a constant curvature.

Actuators

Our actuation module was modified to suit this kit’s requirements. I replaced the multi-part casing with a single unit, such that the new case is easier to manufacture while maintaining adequate performance. The assembly process is simplified as the singular casing eliminates input and output shaft misalignment. The motor and the gears can easily fit into designated spaces using dedicated slots. Since the actuator will only actuate tendons, I introduced an extra bearing at the output shaft to handle the increased radial tension and eliminate shaft eccentricity.

I used the Launchpad TI LAUNCHXL-F28069M, a microcontroller development board, to control the motors with two motor Drivers, BOOSTXL-DRV8305EVM. Since four motors and each motor driver can control one motor, two Launchpads and four motor drivers are required to actuate the continuum robot.

Housing

All components are placed inside a hexagonal casing with windows made of transparent acrylic sheets to view the system’s working. The three tendon actuators are fixed horizontally to reduce tendon routing. Six pulleys are placed inside to help reduce the tendon offset and increase the efficiency of tendon control. One actuation module is placed at the bottom and controls backbone rotation. To secure the electric components in the case, I designed custom 3D-printed mounts. All components are visible from the windows to allow the user to see the robot’s functioning, allowing better understanding and enabling intuition development.

I used several techniques to adapt the design for manufacturing with an FDM 3D Printer:

• teardrop method to ensure optimal vertical holes
• eliminated 90° overhangs using arcs
• used chamfers to reduce support material.
• counterbores for all bolt heads ensure a flush surface finish on all casing sides
• included partial slots for nuts to make assembly easier

Assembly Guide

Find a detailed assembly guide, printable STL Files and mechanical drawings on GitHub and watch the assembly video on YouTube: GitHub Video

I encourage you to try and fabricate this learning kit and contact us regarding any queries or further developments you make! We would love to hear about your experience and see your prototypes!

Epilogue

I took the design home and encouraged robotics enthusiasts at my home university to build and use OpenCR’s Learning Kit. The learning kit allows students to easily understand concepts, design, and implement various algorithms. I hope this kit or its successor will find its way into classrooms at different schools! Stay tuned for updates.

References

What makes motion planning particularly challenging for continuum robots?

First, let’s discuss one of the primary differences between continuum robots and more “traditional” robots (for lack of a better term)—modeling.

As you’ve seen in prior (and future, I promise) blogs on this site, modeling the mechanics of continuum robots is fundamentally different from more traditional serial-link manipulator arms.

As a subset of mechanics, consider just the forward kinematics (FK) problem for continuum robots, i.e., mapping the robot’s joint state to the robot’s resultant shape in space. For concentric tube robots this may be mapping the robot’s tubes’ rotation/translation values to the robot’s shape and for tendon-driven robots this may be mapping the robot’s tendon tensions/displacements to the robot’s shape.

What’s actually happening in the physical world is complicated here. There are elastic elements interacting with (potentially) other elastic elements in complex ways. There is difficult-to-model friction between components of the robot, hysteresis, potentially difficult-to-predict material properties, etc.

The continuum robotics community has taken a variety of approaches to modeling, but a fundamental trend exists: a trade-off between accuracy and speed-of-computation. That is, models which may be fast to compute are typically fast because they neglect to account for some of the physical phenomenon present in the actual robot. On the flip side, models that account for those phenomenon are slow to compute. In reality, there’s a spectrum here.

Constant curvature models, for instance, may take a purely geometric approach to FK, which is fast to compute, but tend to be less accurate. On the other end of the spectrum, you could consider leveraging something like finite element analysis to simulate the physics of a continuum robot to compute its shape. This would improve accuracy but would be much, much slower to compute. Somewhere in the middle of these you may find things like the Cosserat rod/string models, which are a popular way to model continuum robots and will be discussed in other blog posts here (this is the approach we take in my lab, usually, when we aren’t using machine learning-based methods).

Let’s contrast this with serial-link manipulator arms. For these, FK typically require only the multiplication of a handful of matrices, which is comparatively VERY fast, and comparatively VERY accurate, as these robots are rigid.

Okay, so why is this a problem when performing motion planning for continuum robots?

Well, one way to think about motion planning is that we are reasoning in advance of actually moving about the best way to move the robot to achieve our goals. This means that we need our ability to reason about motion to be accurate, otherwise how we predict the robot will move doesn’t mean much. For this reason, model inaccuracy plays havoc with naïve motion planning methods (such as we’ve discussed in last week’s post), and continuum robots have inaccurate models.

Further, the ways in which we are reasoning in advance about moving require us to consider a whole bunch of potential motions for the robot. Every vertex in the grid/lattice-based and sampling-based methods represents an FK computation. Every edge between them requires multiple (often many) FK computations. In the optimization-based methods, the computation of objective and constraint gradients frequently require many FK computations. So the time it takes to compute FK has HUGE implications in motion planning.

This modeling tension, fast but inaccurate or (somewhat) accurate but slow, is one of the fundamental challenges of applying motion planning to continuum robots, and the one I’ll focus on for the rest of this post.

How has the community overcome these challenges?

Modeling accuracy (or lack thereof)

The continuum robotics community (myself included) has largely taken an extremely unsatisfying approach to overcoming modeling inaccuracy. This approach, also used by much of the non-continuum robotics planning world, boils down to a simple idea: don’t let the robot get close to things. That way, if the model is inaccurate, hopefully the robot is far enough away from its environment that it won’t collide with it in unexpected ways.

As an example, in one of my papers¹, we build into the objective function a notion of distance from obstacles. Specifically, We define the cost of a configuration $\boldsymbol{q}$ as:

where $\texttt{clear}$ is, informally defined here, the signed distance from the robot’s body at that configuration to the nearest obstacle.

The cost of a path $\boldsymbol{\xi}$ then becomes \(\texttt{Cost}(\boldsymbol{\xi}) = \int_0^1 \texttt{cost}(\boldsymbol{\xi}(s))ds .\)

where $s$ is a normalized path arclength parameter. Note here that I’ve used little ‘c’ cost for cost over configuration and big ‘C’ Cost for cost over a path. Creative, I know.

You should really read the paper if you want formality here, but this should give you the gist. Also note that in the paper the notation is slightly different, but I’ve changed it here to more-closely match what I’ve used in the last post.

In that method, we leverage both sampling-based motion planning and optimization-based motion planning in parallel (building on [13]), where the method attempts to minimize the $\texttt{cost}$ of a path. This results in paths that travel as far from obstacles as possible, while still being constrained at the start and end configurations.

Computation speed (or lack thereof)

The community has taken a variety of approaches to overcome the slow computation speed of continuum robot mechanical models. Here are a few, with some examples.

The first, and most obvious, is the use of a fast-but-inaccurate mechanical model: For instance, in Lyons et al. ², the authors simplify the mechanical model of concentric tube robots in two key ways. First, they assume that the tubes are torsionally rigid. Second, they assume that the stiffness of each tube dominates all of the tubes nested inside of it. This means that the shape of a tube is assumed to be independent of the motions of the tubes internal to it and that each tube deploys in a constant-curvature circular shape. The mechanical model that leverages these assumptions is sufficiently fast for planning, but again as discussed above ad nauseam, comes at the significant cost of accuracy.

Another approach is to lean on advances in software engineering and programming languages to speed up the computation: Leibrandt et. al.³ leverage template metaprogramming in C++, which (I’m oversimplifying this explanation) moves a lot of expensive run-time computation to compile-time computation for concentric tube robots. This enabled very fast kinematics computation. The method leverages this in a PRM-style motion planner. In our paper¹ mentioned above, we use this template-based kinematic model, wrapping our parallel sampling and optimization method around it. Speaking of, our¹ use of parallelism is another example of leveraging software engineering concepts to speed up computation in planning for continuum robots.

We can also consider leveraging precomputation: In the paper ⁴ (its extension ⁵), the authors propose leveraging precomputation to build a roadmap for planning after the environment is known but in advance of when it is needed. This is a pretty specific (and restrictive) scenario, but makes sense in the application put forth by the papers, namely in surgery. These works present the case that pre-operative medical imaging provides the method with a 3D environment a day (or more) prior to when the surgery will happen. The method can then leverage that time to compute a dense, collision-free roadmap before it’s needed at surgery time. The proposed method utilizes the sampling-based method Rapidly-exploring Random Graph (RRG) (which is also presented in ⁶, in addition to PRM* and RRT*). The method also utilizes one of, if not the most precise (at the time, at least) mechanical models available for concentric tube robots. This dense, collision-free roadmap is built over the course of many hours using this accurate model and the anatomical environment segmented from the pre-operative medical imaging. Then, at the time of the surgery, the method proposes an interactive-rate supervisory control-style scheme in which a haptic device is used by the surgeon to input a desired tip position for the robot in the patient. Leveraging a nearest-neighbor data structure, the vertex in the roadmap that corresponds to the configuration with the robot’s tip closest to the desired position is then identified. Graph search is performed to identify a path on the roadmap from the current configuration to that configuration, and then the method “steps off” of the roadmap and leveraging damped least-squares iterative inverse kinematics ⁷⁸, which is conceptually very similar to resolved-rates control, drives the tip of the robot as close to the desired tip position as it can. This then repeats in a loop.

In our paper⁹, my group has built upon this concept in a few ways. Among other contributions, we adapt the concepts to work for tendon-driven continuum robots (⁴ and ⁵ are specific to concentric tube robots), introduce a modification to mechanical modeling and collision detection to improve speed, and remove the need to have the environment in advance of precomputing the dense roadmap. The method still heavily leverages precomputation, but only requires the environment immediately prior to when the supervisory control loop is started.

So what’s next?

What are the open problems or areas to be explored in motion planning for continuum robots? Here are a few that I’m personally excited about.

Leveraging learned models in planning

There has recently been a large interest in machine learning-based mechanical models of continuum robots, both for their potential to learn unmodeled (or unknowable/unmodelable) effects and their potential for fast computation (e.g., frequently the shape computation of a learned model is faster than the physics-based equivalent). I’ll leave the details of these methods to another post, written by myself or someone else. However germane to THIS post, how to leverage these types of models for motion planning of continuum robots is as-yet underexplored.

Principled planning under uncertainty

Motion planning in a way that handles uncertainty in a principled way has been leveraged to great effect in other areas of robotics. Given the uncertainty associated with continuum robots, the application/exploration of using these types of methods/formulations in planning for continuum robots has great potential.

Multi-fidelity planning

The fact that we have a variety of models for continuum robots that trade computation speed for accuracy implies exciting potential for the use of multi-fidelity planning. One can imagine, as maybe the most obvious example, using a fast but inaccurate model to prioritize edge evaluation in a lazy planning paradigm, look up Lazy PRM if that doesn’t make sense to you, where full edge evaluation is then done with a high-fidelity model.

Closing remarks

Hopefully the above alongside last week’s Part 1 were a relatively gentle introduction to motion planning and how it relates conceptually to continuum robots. This wasn’t intended to be a comprehensive survey of planning for continuum robots. Rather, my intention was to give you the vocabulary and an understanding of the rough concepts sufficient for you to investigate the rest of the literature yourself, and to innovate on the state-of-the-art as you need for your specific problems.

References

Motion planning stands as a central pillar in robotics. Planning enables robots and autonomous agents to move themselves and other parts of the world to a desired goal by choosing a sequence of appropriate actions. Like robotics in general, motion planning draws on a number of different fields in developing its tools. Approaches to motion planning have grown out of artificial intelligence, optimal control, and operations research.

That’s the first paragraph of my motion planning course syllabus. It’s a bit… grandiose, but I’ve found it to be effective in convincing the students that they are about to learn something interesting; hopefully it worked on you as well. It was also written by the professor who taught the course before I did, and whose syllabus I used as a starting point for my own (on the shoulders of giants etc etc etc). If that was sufficiently dramatic and you’re convinced to keep reading, let’s get to it.

More colloquially, motion planning can be broadly defined as the problem of computing motions, actions, or controls that move a robot through its environment in such a way that some task is performed while obeying some set of constraints.

In its most basic form, that task can be defined as moving from some start configuration to some goal configuration, with typical constraints being defined as obstacle avoidance over the motion, obeying joint limits, etc. Complexity in task definitions and constraint definitions builds from there.

I personally like to present motion planning as a generic optimization problem and then look specifically at how aspects of that problem are defined. Be forewarned, I’m going to grossly abuse notation/terminology and ignore nuance all over this post with the goal of imparting intuition rather than specifics.

Consider:

$\boldsymbol{\xi}^* = \underset{\boldsymbol{\xi}}{\texttt{argmin}}\ \texttt{cost}(\boldsymbol{\xi})$

$s.t. \ \ \ c(\boldsymbol{\xi}) \ge 0$ $\ \ \ \ \ \ \ \ d(\boldsymbol{\xi}) = 0$

where $\boldsymbol{\xi}$ represents some trajectory, path, or plan parameterization, $\texttt{cost}$ is some cost function defined over these, $\boldsymbol{\xi}^*$ is an optimal plan under the cost function, and $c$ and $d$ are generic, as-of-yet-unspecified inequality and equality constraints (or sets of such constraints).

This generic optimization formalization is sufficiently expressive to talk about most of the basics of motion planning, and by varying how aspects of this formalization are defined we can arrive at most of the ways people have historically thought about motion planning.

As an example, let’s consider the very simple (relative to other motion planning problems) task of moving a holonomic robot from some start state to some goal state while avoiding some set of obstacles in its environment.

If we conceptualize $\boldsymbol{\xi}$ as defining a space curve in the robot’s configuration space, we can parameterize $\boldsymbol{\xi}$ by the normalized path length such that $\boldsymbol{\xi}(0)$ is the robot’s start configuration and $\boldsymbol{\xi}(1)$ is the robot’s configuration after following the full plan.

Often people consider minimizing path length, so we can imagine defining $\texttt{cost}$ as the length of the path (under some appropriate definition of length).

We also want to constrain the trajectory such that the robot ends in a goal configuration (broadly as an element of some goal set), and such that the entire trajectory avoids obstacles.

We can then cast this simple problem into our optimization framework simply as:

$\boldsymbol{\xi}^* = \underset{\boldsymbol{\xi}}{\texttt{argmin}}\ \texttt{length}(\boldsymbol{\xi})$

$s.t. \ \ \ \boldsymbol{\xi}(0) = \textrm{start configuration}$

$\ \ \ \ \ \ \ \ \boldsymbol{\xi}(1) \in \textrm{goal set}$

$\ \ \ \ \ \ \ \ \boldsymbol{\xi}(s) \textrm{ is collision free } \forall s \in [0,1]$

As in most interesting things, key considerations lie in the details (which I’ve completely ignored above). How is your trajectory parameterized? How do you define your goal set? How do you represent your robot and environment geometry, enabling you to check for collision? Is length even the right objective to minimize? The answer to these questions depends on your specific problem/robot/etc. If I knew what this is for you, I’d just solve your problems myself and take all the credit.

Okay, so this is the first step—representing your problem. Next, let’s solve it.

How do methods typically work?

In my head I broadly group modern motion planning methods into three groups, discrete grid/lattice-based search methods, sampling-based methods, and optimization-based methods. There’s an implied fourth group, “other” or maybe “miscellany”, but I’m going to focus on the first three in this post.

Discrete grid/lattice-based search methods

At their simplest, these methods consider a graph or tree (which is just a special kind of graph) embedded in the robot’s configuration space. If you are unsure what I mean by “graph or tree” stop now and read this and this before moving on.

Notably for these methods, the graph is usually implied by a discretization over the robot’s possible actions. Imagine vertices of the graph as representing configurations or states of the robot, and edges as actions between states, which when applied from the source state result in the robot transitioning to the destination state. This action discretization is frequently defined in advance (and you can imagine applying this concept to robot’s with continuous action spaces simply by discretizing those action spaces). Due to the constant discretization, you can conceptualize these as creating something like a grid or lattice as the graph.

With this conceptualization, motion planning then reduces to a search over that graph, seeking the “shortest” path from your start vertex to any vertex that represents a goal configuration or state.

How does this map back to our optimization formalization above? The cost function is minimized by applying a graph-search method that minimizes the accumulation of edge cost (so you need to define a cost over edges and recognize that without substantial modification you’re limited to considering objectives that can be modeled by costs that accumulate). The constraints are enforced simply by not considering edges that would result in a violation of the constraints during the search.

Cannonical graph search methods (not specific to motion plannning) include Uniform Cost Search (UCS) (closely related to Djikstra’s algorithm) and A* search.

The key to applying these methods in motion planning is frequently in coming up with an effective and admissible/consistent heuristic (see A* details for what that means). More advanced methods applied to motion planning specifically include D*¹, D*-Lite², and LPA*³. It’s also very much worth noting that inadmissible heuristics can also be leveraged to potentially great effect (see Multi-Heuristic A*⁴).

Sampling-based methods

Perhaps the most popular class of motion planning methods, the sampling-based approaches are closely related to the graph-search methods. In fact, these methods frequently leverage graph search but on a graph constructed in a different manner.

Rather than a graph constructed via a discretization over the robot’s actions, the sampling-based methods leverage random sampling to construct the graph incrementally (this is why they are called sampling-based, as you may have guessed). The graph (or tree, which again, is just a special kind of graph) is embedded in the robot’s configuration space as above. It is constructed by iteratively randomly sampling states (or actions, sometimes) and attempting to connect the newly sampled state (or state resulting from the sampled action) to the graph/tree.

When adding states, (usually) the new state and the edge connecting it to the graph are checked for validity (e.g., joint-limit satisfaction and obstacle avoidance). If the state and/or edge are not valid, they aren’t added to the graph.

At some point, the hope is that the graph will contain a goal state, and then by running graph search (e.g., A*) on this randomly-generated graph the method will find a path from the start state to a goal. If the graph is a tree, this search is trivial as there’s only one path in the graph (ignoring some nuance, that’s what tree means).

The specifics of the graph, how the states/actions are sampled, how the connections are made, etc., are what distinguish the sampling-based methods from each other.

Okay, so let’s map these concepts back to our optimization formulation.

Cost: If there are multiple paths in the graph, then the graph search over the graph can provide the lowest cost path in the graph, however the lowest cost path possible may not be in the graph at any given iteration. Many of the earlier versions of these methods ignored cost and instead attempted to find any path that avoided obstacles, providing a property called probabilistic-completeness. Since then, other methods build upon this to provide a guarantee called asymptotic optimality, which intuitively means that their best path will approach an optimal path (in cost) as runtime progresses (only getting arbitrarily close to an optimal path in the limit though). This contrasts with the discrete grid/lattice-based search methods in that those methods typically only provide a property called resolution completeness/optimality which intuitively means the path is complete or optimal for the resolution of the discretization, but not necessarily in general.

Constraints: As with the discrete grid/lattice-based search methods, the sampling-based methods usually encode constraint satisfaction by only considering states/edges in the graph construction or search that satisfy the constraints, including obstacle avoidance.

Canonical probabilistically-complete methods include Probabilistic Roadmaps (PRM)⁵ and Rapidly-exploring Random Trees (RRT)⁶. Popular asymptotically-optimal methods include PRM*/RRT*⁷, Batch-Informed Trees (BIT)⁸, and Asymptotically-Optimal-RRT (AO-RRT)⁹.

Super-duper important point! Just because you’re using sampling-based methods does not mean you’re getting probabilistic completeness or asymptotic optimality. There are properties/assumptions you must satisfy in your specific application of these methods that are frequently nuanced, you’ll need to ensure that you satisfy these things. Read the papers detailing these methods for details!

Optimization-based methods

Given the formulation we started with, i.e., casting the motion planning problem as constrained optimization, you may be asking yourself: “wait, why don’t we just use optimization methods from, you know, mathematics, to solve this problem?” Great question! Enter the optimization-based motion planning methods.

This class of methods leverages nonlinear, constrained optimization methods from the field of optimization itself explicitly to solve the motion planning problem using variations of our above formulation—rather than relying on the graph-search to minimize cost.

As of now, the majority of these methods (with an exception mentioned below) leverage gradients (and frequently higher-order derivatives) of the objective function and/or of the constraint functions in iterative optimization methods. Think penalty methods, augmented Lagrangian methods, interior point methods, etc. If these concepts are new to you, there is an amazing textbook on the subject by Wright and Nocedal ¹⁰ that is open almost perpetually in my office.

At a high level there isn’t much more to the story. These methods take exactly the optimization-based formulation above and apply optimization techniques that work directly on such formulations.

Is it just that easy? Unfortunately, no. The prior paragraph is true at a high level, but the low level is where the details lie. The specific ways in which these methods define cost and their constraint functions such that they work well with the optimization techniques—e.g., ensuring they have gradients that are well-behaved—can make these methods a bit tricky to use in practice. However in many cases they end up working very, very well, which can make it worth the complexity.

Canonical methods include CHOMP¹¹, which leverages a variation of gradient descent, and TrajOpt¹² which leverages Sequential Quadratic Programming (SQP). A different method, that is not nearly as popular as those methods, combines sampling-based planning with interior point optimization¹³. I include it in this list because it was part of yours truly’s PhD work, and so here is a shameless plug.

There is also a method called Cross-Entropy Motion Planning (CEMP)¹⁴ that leverages a gradient-free optimization method (the cross-entropy method, as you may have guessed from the name). This is notable because many difficulties one may encounter in leveraging optimization-based methods may come from the gradients, or lack-thereof.

This concludes Part 1 of my introduction to motion planning for continuum robots. In next week’s Part 2 we will look at the challenges of motion planning for continuum robots. Stay tuned!

References

Keynotes

My keynote presentation at this year’s ICRA is all about the Open Continuum Robotics Project.

Accelerating Medical Robotics Innovation with the Open Continuum Robotics Project

Tuesday, 17:00 - 18:30 - Healthcare Robotics

Speaker: Jessica Burgner-Kahrs

Abstract: From robotically steered catheters and bronchoscopes to single port flexible instruments, continuum robotics has led to great advances in medical robotics over the past decade. Yet, research findings remain stubbornly difficult to reproduce because continuum robotics hardware and software are not readily available, and no common systems exist. Ultimately this slows the pace of innovation.

The Open Continuum Robotics Project combines hardware, software, and education with the goal of fostering more collaborative, cost-effective, and reproducible research. This enables rapid prototyping and iterative design, allowing researchers and clinicians to test and refine their ideas more quickly – including in resource-constrained settings. In this way, the Open Continuum Robotics Project accelerates the development of new technologies and treatments in medical robotics.

Workshop and Tutorials

ICRA 2023 features an incredible rich set of workshops and tutorials this year, which makes it very hard to choose which one to attend (or when to rush from one room to the next #FOMO). The following ones feature aspects of continuum robotics and are worth a look!

Monday

• MW02: Soft Growing Robots: From Search-and-Rescue to Intraluminal Interventions
• MW05: Shrinking the Cutting Edge: Making Small-Scale Medical Robots for Humans
• MW10: Origami-based structures for designing soft robots with new capabilities

Friday

• FW12: Geometric Representations: The Roles of Screw Theory, Lie algebra, & Geometric Algebra
• FW16: Soft Robotics: Fusing function with structure
• FW22: Force and shape perception for surgical instruments and robots
• FT29: Towards an accessible soft robotics toolbox and validation test rig

Talks

The majority of oral presentations of ICRA 2023 are adressing continuum robot modeling, with a focus on kinetostatics and dynamics using Cosserat theory and optimal control theory. It is fantastic to see a trend towards open toolboxes and knowledge mobilization! There are also some exciting soft continuum robot designs!

FEA-Based Soft Robotic Modeling: Simulating a Soft-Actuator in SOFA (I)

09:50-10:00, Paper TuAT2.9

Authors: Ferrentino, Pasquale; Roels, Ellen; Brancart, Joost; Terryn, Seppe ; Van Assche, Guy; Vanderborght, Bram

Abstract: Soft robotics modeling is a research topic that is evolving fast. Many techniques are present in literature but most of them require analytical models with a lot of equations that are time-consuming, hard to resolve, and not so easy to handle. For this reason, the help of a soft mechanics simulator is essential in this field. In fact, this paper presents a tutorial on how to build a soft-robot model using an open-source Finite Element Analysis (FEA) simulator, called SOFA. This software is able to generate a simulation scene from a code written in Python or XML, so it can be used by people that with different fields of competence like mechanical knowledge, knowledge of material properties and programming skills. As a case study, a Python simulation of a cable-driven soft actuator that makes contact with a rigid object is considered. The basic working principles of SOFA required to make a scene are explained step by step. In particular, it shows how to simulate the mechanics and animate the bending behavior of the actuator. Furthermore, it will be shown also how to retrieve and save data from simulation, demonstrating that SOFA can easily adapt to a multi-disciplinary subject as the research in soft-robotics, but also be useful for teaching simulation and programming language principles to engineering students.

SoRoSim: A MATLAB Toolbox for Hybrid Rigid-Soft Robots Based on the Geometric Variable-Strain Approach (I)

15:40-15:50, Paper TuBT2.5

Authors: Mathew, Anup Teejo; Ben Hmida, Ikhlas; Armanini, Costanza; Boyer, Frédéric; Renda, Federico

Abstract: Soft robotics has been a trending topic within the robotics community for almost two decades. However, available tools for the modeling and analysis of soft robots are still limited. This paper introduces a user-friendly MATLAB toolbox, SoRoSim, that integrates the Geometric Variable Strain model of Cosserat rods to facilitate the static and dynamic analysis of soft, rigid, or hybrid robotic systems. We present a brief overview of the design and structure of the toolbox and validate it by comparing its results with those published in the literature. To highlight the toolbox’s potential to efficiently model, simulate, optimize, and control various robotic systems, we demonstrate four sample applications. The demonstrated applications explore different actuator and external loading conditions of single-, branched-, open-, and closed-chain robotic systems. We think that the soft-robotics research community will significantly benefit from the SoRoSim toolbox for a wide variety of applications.

A Geometrically-Exact Assumed Strain Modes Approach for the Geometrico and Kinemato-Static Modellings of Continuum Parallel Robots (I)

15:50-16:00, Paper TuBT2.6

Authors: Briot, Sébastien; Boyer, Frédéric

Abstract: There is a growing interest on the study of continuum parallel robots (CPRs) due to their higher stiffness and better dynamics capacities than serial continuum robots (SCRs). Several works have focused on the computation of their geometrico- and kinemato-static models, that can be sorted into two main categories: (i) models based on the continuous Cosserat equations: They are very accurate but assessing elastic stability with them is tricky; (ii) discretized models: They allow easily checking the elastic stability but they require a large number of elastic variables to be accurate. In this paper, we extend an approach based on assumed strain modes developed for the dynamics of SCRs to the statics of CPRs. This method is able to predict the robot configuration with an excellent accuracy with a very limited number of elastic variables, contrary to other discretization methods. The method is also more than 100 times faster than finite differences for a better prediction accuracy. Finally, it is possible to assess the robot elastic stability by only checking the Hessian of the potential energy as for any discretization method, thus making the analysis of this property simpler.

Towards a Physics-Based Model for Steerable Eversion Growing Robots

16:00-16:10, Paper TuBT2.7

Authors: Wu, Zicong; De Iturrate Reyzabal, Mikel; Sadati, Seyedmohammadhadi; Liu, Hongbin; Ourselin, Sebastien; Leff, Daniel Richard; Katzschmann, Robert Kevin; Rhode, Kawal; Bergeles, Christos

Abstract: Soft robots that grow through eversion/apical extension can effectively navigate fragile environments such as ducts and vessels inside the human body. This letter presents the physics based model of a miniature steerable eversion growing robot. We demonstrate the robot’s growing, steering, stiffening and interaction capabilities. The interaction between two robot-internal components is explored, i.e., a steerable catheter for robot tip orientation, and a growing sheath for robot elongation/retraction. The behavior of the growing robot under different inner pressures and external tip forces is investigated. Simulations are carried out within the SOFA framework. Extensive experimentation with a physical robot setup demonstrates agreement with the simulations. The comparison demonstrates a mean absolute error of 10–20% between simulation and experimental results for curvature values, including catheter-only experiments, sheath-only experiments and full system experiments. To our knowledge, this is the first work to explore physics-based modelling of a tendon-driven steerable eversion growing robot. While our work is motivated by early breast cancer detection through mammary duct inspection and uses our MAMMOBOT robot prototype, our approach is general and relevant to similar growing robots.

Statics and Dynamics of Continuum Robots Based on Cosserat Rods and Optimal Control Theories (I)

16:20-16:30, Paper TuBT2.9

Authors: Boyer, Frédéric; Lebastard, Vincent; Candelier, Fabien; Renda, Federico; Alamir, Mazen

Abstract: This paper explores the relationship between optimal control and Cosserat beam theory from the perspective of solving the forward and inverse dynamics (and statics as a subcase) of continuous manipulators and snake-like bio-inspired locomotors. By invoking the principle of minimum potential energy, and the Gauss principle of least constraint, it is shown that the quasi-static and dynamic evolution of these robots, are solutions of optimal control problems (OCPs) in the space variable, which can be solved at each step (of loading or time) of a simulation with the shooting method. In addition to offering an alternative viewpoint on several simulation approaches proposed in the recent past, the optimal control viewpoint allows us to improve some of them while providing a better understanding of their numerical properties. The approach and its properties are illustrated through a set of numerical examples validated against a reference simulator.

COBRA: From Industrial to Medical Surgery with Slender Continuum Robots (I)

15:00-15:10, Paper WeBT2.1

Authors: Alatorre, David; Robles-Linares, Jose A.; Russo, Matteo; Elbanna, Mohamed A.; Wild, Samuel; Dong, Xin; Mohammad, Abdelkhalick; Kell, James; Norton, Andy; Axinte, Dragos

Abstract: The maintenance of critical industrial components is often hindered by limited access, tortuous passages, and complex geometries. In highly constrained environments, inspection tasks are currently performed with borescopes, but even skilled operators struggle with hard-to-reach targets and the limited mobility prevents in-situ repair when defects are identified. Thanks to an active shape control, snake-like and continuum robots can outperform borescopes for short range inspection as well as enable intervention. However, their actuation technology limits their scalability in length, as longer bodies pose control challenges due to their intrinsically low stiffness and space constraints. To overcome the limitations of both borescopes and continuum robots, we here propose a modular design at their intersection, with both active tendon-driven and passively flexible segments. The main elements of the novel design, including actuation and control interface, are described, and the system is demonstrated in scenarios for aerospace assets, nuclear installations, and robot-assisted surgery.

A Generalized Framework for Concentric Tube Robot Design Using Gradient-Based Optimization (I)

10:10-10:20, Paper ThAT2.8

Authors: Lin, Jui-Te;Girerd, Cedric; Yan, Jiayao; Hwang, John T.; Morimoto, Tania K.

Abstract: Concentric tube robots (CTRs) show particular promise for minimally invasive surgery due to their inherent compliance and ability to navigate in constrained environments. Due to variations in anatomy among patients and variations in task requirements among procedures, it is necessary to customize the design of these robots on a patient- or population-specific basis. However, the complex kinematics and large design space make the design problem challenging. Here we propose a computational framework that can efficiently optimize a robot design and a motion plan to enable safe navigation through the patient’s anatomy. The current framework is the first fully gradient-based method for CTR design optimization and motion planning, enabling an efficient and scalable solution for simultaneously optimizing continuous variables, even across multiple anatomies. The framework is demonstrated using two clinical examples, laryngoscopy and heart biopsy, where the optimization problems are solved for a single patient and across multiple patients, respectively.

Magnetic Soft Continuum Robots with Braided Reinforcement

10:20-10:30, Paper ThAT2.9

Authors: Lloyd, Peter Robert; Onaizah, Onaizah; Pittiglio, Giovanni; Chathuranga, Damith Suresh; Chandler, James Henry; Valdastri, Pietro

Abstract: Flexible catheters are used in a wide variety of surgical interventions including neurological, pancreatic and cardiovascular. In many cases a lack of dexterity and miniaturization along with excessive stiffness results in large regions of the anatomy being deemed inaccessible. Soft continuum robots have the potential to mitigate these issues. Due to its enormous potential for miniaturization, magnetic actuation is of particular interest in this field. Currently, flexible magnetic catheters often rely on forces of anatomical interaction to generate large deformations during navigation and for soft anatomical structures this could be considered potentially damaging. In this study we demonstrate the insertion of a high aspect ratio, 50 mm long by 2 mm diameter, soft magnetic catheter capable of navigating up to a 180 degree bend without the aid of interactive forces. This magnetic catheter is reinforced with a lengthwise braided structure and its magnetization allows it to shape form along tortuous paths. We demonstrate our innovation in a planar silicone pancreas phantom. We also compare our approach with a mechanically equivalent tip driven magnetic catheter and with an identically magnetized, unreinforced catheter.

Kinetostatic Modeling of Tendon-Driven Parallel Continuum Robots (I)

15:00-15:10, Paper ThBT5.1

Authors: Lilge, Sven; Burgner-Kahrs, Jessica

Abstract: Tendon-driven parallel continuum robots consist of multiple individual continuous kinematic chains, that are actuated in bending utilizing tendons routed along their backbones. This work derives and proposes a Cosserat rod based kinetostatic modeling framework for such parallel structures that allows for efficiently solving the forward, inverse and velocity kinetostatic problems. Using this model, the kinematic properties such as reachable workspace, singularities, manipulability and compliance of tendon-driven parallel continuum robots are studied in detail. Experiments are conducted using a real robotic prototype to validate the derived modeling approach. Overall, a median pose accuracy of 4.9 mm, corresponding to 3.4% of the continuum robots’ lengths, and 6.2◦ is achieved. The median of the model’s computation time results in 0.51 s on standard computing hardware. Fast computations of below 100 ms can be achieved, if an appropriate initial guess for solving the kinetostatic model is available, making the model suitable for a range of different applications including optimization or control.

Posters

The poster presentations at ICRA 2023 cover various aspects of (soft) continuum robots, including design, control, and sensing.

A Soft Hybrid-Actuated Continuum Robot Based on Dual Origami Structures

08:30-10:10, Paper TuPO1S-01.2

Authors: Tao, Jian; Hu, Qiqiang; Luo, Tianzhi; Dong, Erbao

Abstract: Soft continuum robots have shown tremendous potential for medical and industrial applications owing to their flexibility and continuous deformability. However, their telescopic and bending capabilities and variable stiffness are still limited. This study proposes a novel origami-inspired soft continuum robot to possess large telescopic and bending capabilities while improving stiffness based on the principle of antagonistic actuation. The soft robot consists of dual origami structures. The inner forms an air chamber actuated by pneumatics, and the outer is controlled by nine tendon-driven actuators. The proposed design uses the advantages of a hybrid actuation to achieve motion and stiffness control. The performance of the soft robot is studied experimentally based on single and three robot modules. Results show that the robot has an excellent stretch ratio and a maximum bending angle of 180°. The robot can also increase stiffness to resist the bending deformation induced by self-weight and loads.

Image-Based Pose Estimation and Shape Reconstruction for Robot Manipulators and Soft, Continuum Robots Via Differentiable Rendering

Tuesday 08:30-10:10, Paper TuPO1S-01.6, Room T8

Authors: Lu, Jingpei; Liu, Fei; Girerd, Cedric; Yip, Michael C.

Abstract: State estimation from measured data is crucial for robotic applications as autonomous systems rely on sensors to capture the motion and localize in the 3D world. Among sensors that are designed for measuring a robot’s pose, or for soft robots, their shape, vision sensors are favorable because they are information-rich, easy to set up, and cost-effective. With recent advancements in computer vision, deep learning-based methods no longer require markers for identifying feature points on the robot. However, learning-based methods are data-hungry and hence not suitable for soft and prototyping robots, as building such bench-marking datasets is usually infeasible. In this work, we achieve image-based robot pose estimation and shape reconstruction from camera images. Our method requires no precise robot meshes, but rather utilizes a differentiable renderer and primitive shapes. It hence can be applied to robots for which CAD models might not be available or are crude. Our parameter estimation pipeline is fully differentiable. The robot shape and pose are estimated iteratively by back-propagating the image loss to update the parameters. We demonstrate that our method of using geometrical shape primitives can achieve high accuracy in shape reconstruction for a soft continuum robot and pose estimation for a robot manipulator.

Discrete-Time Model Based Control of Soft Manipulator with FBG Sensing

08:30-10:10, Paper TuPO1S-01.8

Authors: Franco, Enrico; Aktas, Ayhan; Treratanakulchai, Shen; Garriga-Casanovas, Arnau; Donder, Abdulhamit; Rodriguez y Baena, Ferdinando

Abstract: In this article we investigate the discrete-time model based control of a planar soft continuum manipulator with proprioceptive sensing provided by fiber Bragg gratings. A control algorithm is designed with a discrete-time energy shaping approach which is extended to account for control-related lag of digital nature. A discrete-time nonlinear observer is employed to estimate the uncertain bending stiffness of the manipulator and to compensate constant matched disturbances. Simulations and experiments demonstrate the effectiveness of the controller compared to a continuous time implementation.

A Soft Robot with Three Dimensional Shape Sensing and Contact Recognition Multi-Modal Sensing Via Tunable Soft Optical Sensors

08:30-10:10, Paper TuPO1S-02.1

Authors: McCandless, Max; Juliá Wise, Frank; Russo, Sheila

Abstract: Soft optical sensing strategies are rapidly developing for soft robotic systems as a means to increase the controllability of soft compliant robots. In this paper, we present a roughness tuning strategy for the fabrication of soft optical sensors to achieve the dual functionality of shape sensing combined with contact recognition within a single multi-modal sensor. The molds used to fabricate the soft sensors are roughened via laser micromachining to achieve asymmetrical sensor responses when bent in opposite directions. We demonstrate the integration of these sensors into a fully soft robotic platform consisting of a multi-directional bending module with integrated 3D shape sensing and a gripper with tip position monitoring along with contact force recognition. We show the accuracy of our sensing strategy in validation experiments and a pick-andplace task is performed to demonstrate the robot’s functionality.

On Tendon Driven Continuum Robots with Compressible Backbones

08:30-10:10, Paper TuPO1S-03.7

Authors: Srivastava, Manu; Walker, Ian

Abstract: This paper discusses the effect of axial backbone compression on tendon-driven continuum robots. A new mechanics model for compensating for this effect that does not require tendon tension sensing or knowledge of manipulator material properties/stiffnesses is introduced and analyzed. In addition, we provide an analytical expression for the minimum preload on the tendons to achieve a given bend, a quantity determined empirically thus far. Our model is computationally efficient and achieves real time control on low cost hardware. The analysis is supported by experimental results demonstrating significant improvement over kinematics in open loop control of a tendon-driven continuum hose robot.

Data-Driven Estimation of Forces Along the Backbone of Concentric Tube Continuum Robots

Tuesday 15:00-16:40, Paper TuPO2S-02.6, Room T8

Authors: Donat, Heiko; Mohammadi, Pouya; Steil, Jochen J.

Abstract: Concentric tube continuum robots (CTCRs) belong to the family of continuum robots with applications in minimally invasive surgeries. Because of this application domain, measuring the external forces along the body of the robot is paramount. CTCRs are made up of thin elastic rods and are intended to be applied inside the human body, where conventional sensor-based measurements are not feasible. Consequently, research is resorting to estimate the forces through geometric, numeric, or optimization methods. However, these methods often suffer from slow convergence. In this paper, we introduce a novel data-driven approach for estimating contact forces along the body of a CTCR that offers an estimation precision comparable to the current state-of-the-art optimization-based approaches, but exhibits nearly two orders of magnitude faster convergence. The proposed method is scalable and exhibits a significant performance in response to a wide range of external forces. The approach was evaluated in simulations and on a real 2-tube CTCR.

Modeling of a Robotic Transcatheter Delivery System

09:00-10:40, Paper WePO1S-01.6

Authors: Nayar, Namrata Unnikrishnan; Qi, Ronghuai; Desai, Jaydev P.

Abstract: Intracardiac transcatheter systems guided by advanced imaging modalities are gaining popularity in treating mitral regurgitation in non-surgical candidates. Robotically steerable transcatheter systems must use model-based control strategies to ensure safer and more effective transcatheter procedures with less trauma while using smaller control gains. In this paper, a 4-DoF robotically steerable tendon-driven robot was fabricated, and the relationship between the tendon displacement and the joint angle was derived. This relation was derived in two parts to make this approach applicable to any other catheter system. A model was derived to determine the tendon tensions needed to achieve desired joint angles. Then, the tendon characteristics were studied, and a tendon elongation (TE) model was derived as a function of tendon length. Executing the modeling process in two steps makes it easy to introduce additional parameters like length, friction, and pose, to characterize complex systems like catheters. The TE model was used to actuate the joints of the robot and RMSE was computed to characterize its performance. Also, PID control was used along with the TE model to improve the system’s performance, and the contribution of the model and the controller in the system was recorded.

A Novel Concentric Tube Steerable Drilling Robot for Minimally Invasive Treatment of Spinal Tumors Using Cavity and U-Shape Drilling Techniques

09:00-10:40, Paper WePO1S-01.11

Authors: Sharma, Susheela; Park, Ji Hwan; Amadio, Jordan P.; Khadem, Mohsen; Alambeigi, Farshid

Abstract: In this paper, we present the design, fabrication, and evaluation of a novel flexible, yet structurally strong, Concentric Tube Steerable Drilling Robot (CT-SDR) to improve minimally invasive treatment of spinal tumors. Inspired by concentric tube robots, the proposed two degree-of-freedom (DoF) CT-SDR, for the first time, not only allows a surgeon to intuitively and quickly drill smooth planar and out-of-plane J- and U- shape curved trajectories, but it also, enables drilling cavities through a hard tissue in a minimally invasive fashion. We successfully evaluated the performance and efficacy of the proposed CT-SDR in drilling various planar and out-of-plane J-shape branch, U-shape, and cavity drilling scenarios on simulated bone materials.

Magnetic Ball Chain Robots for Endoluminal Interventions

09:00-10:40, Paper WePO1S-01.12

Authors: Pittiglio, Giovanni; Mencattelli, Margherita; Dupont, Pierre

Abstract: This paper introduces a novel class of hyperredundant robots comprised of chains of permanently magnetized spheres enclosed in a cylindrical polymer skin. With their shape controlled using an externally-applied magnetic field, the spherical joints of these robots enable them to bend to very small radii of curvature. These robots can be used as steerable tips for endoluminal instruments. A kinematic model is derived based on minimizing magnetic and elastic potential energy. Simulation is used to demonstrate the enhanced steerability of these robots in comparison to magnetic soft continuum robots designed using either distributed or lumped magnetic material. Experiments are included to validate the model and to demonstrate the steering capability of ball chain robots in bifurcating channels.

Model-Based Pose Estimation of Steerable Catheters under Bi-Plane Image Feedback

09:00-10:40, Paper WePO1S-03.2

Authors: Lawson, Jared; Chitale, Rohan; Simaan, Nabil

Abstract: Small catheters undergo significant torsional deflections during endovascular interventions. A key challenge in enabling robot control of these catheters is the estimation of their bending planes. This paper considers approaches for estimating these bending planes based on bi-plane image feedback. The proposed approaches attempt to minimize error between either the direct (position-based) or instantaneous (velocity-based) kinematics with the reconstructed kinematics from bi-plane image feedback. A comparison between these methods is carried out on a setup using two cameras in lieu of a bi-plane fluoroscopy setup. The results show that the position-based approach is less susceptible to segmentation noise and works best when the segment is in a non-straight configuration. These results suggest that estimation of the bending planes can be accompanied with errors under 30◦. Considering that the torsional buildup of these catheters can be more than 180◦, we believe that this method can be used for catheter control with improved safety due to the reduction of this uncertainty.

Image Segmentation for Continuum Robots from a Kinematic Prior

09:00-10:40, Paper WePO1S-03.4

Authors: Watson, Connor; Nguyen, Anna; Morimoto, Tania K.

Abstract: In this work, we address the problem of robust segmentation of a continuum robot from images without the need for training data or markers. We present a method that leverages information about the kinematics of these robots to produce an estimate of the robot shape, which is refined through optimization over global image statistics. Our approach can be straightforwardly applied to any continuum robot design and is able to handle partial occlusions of the robot body, as well as challenging background conditions. We validate our method experimentally for a concentric tube robot in a simulated surgical environment and show that our method significantly outperforms a naive projection of the robot shape and color thresholding, which is commonly used in current vision-based estimation algorithms for these robots. Overall, this work has the potential to improve the viability of vision-based state estimation for continuum robots in real-world settings.

An Equivalent Two Section Method for Calculating the Workspace of Multi-Segment Continuum Robots

09:00-10:40, Paper ThPO1S-12.4

Authors: Fan, Yeman; Liu, Dikai

Abstract: Obtaining the shape and size of a robot’s workspace is essential for both its design and control. However, determining the accurate workspace of a multi-segment continuum robot by graphic or analytical methods is a challenging task due to its inherent flexibility and complex structure. Existing numerical methods have limitations when applied to a continuum robot. This paper presents an Equivalent Two Section (ETS) method for calculating the workspace of multi-segment continuum robots. This method is based on the forward kinematics and a piecewise constant curvature (PCC) model to determine the boundaries of the workspace. In order to verify the proposed method, simulation experiments are conducted using six different maximum bending angles and seven different number of segments. Results of the ETS method are compared to the true workspaces of these configurations estimated by an exhaustive approach. The results show that the proposed ETS method is both efficient and accurate, and has small estimation errors. Discussions on the advantages and limitations of the proposed ETS method are also presented.

HaPPArray: Haptic Pneumatic Pouch Array for Feedback in Hand-Held Robots

15:00-16:40, Paper ThPO2S-23.1

Authors: Luo, Xiaolei; Lin, Jui-Te; Morimoto, Tania K.

Abstract: Haptic feedback can provide operators of hand-held robots with active guidance during challenging tasks and with critical information on environment interactions. Yet for such haptic feedback to be effective, it must be lightweight, capable of integration into a hand-held form factor, and capable of displaying easily discernible cues. We present the design and evaluation of HaPPArray — a haptic pneumatic pouch array — where the pneumatic pouches can be actuated alone or in sequence to provide information to the user. A 3x3 array of pouches was integrated into a handle, representative of an interface for a hand-held robot. When actuated individually, users were able to correctly identify the pouch being actuated with 86% accuracy, and when actuated in sequence, users were able to correctly identify the associated direction cue with 89% accuracy. These results, along with a demonstration of how the direction cues can be used for haptic guidance of a medical robot, suggest that HaPPArray can be an effective approach for providing haptic feedback for hand-held robots.