From CT Scan to Robotic Surgery Prototype: Kidney Stone Detection & PCNL Planning in ROS 2

ROS ROS General
, , , ,
1

Hi all,
I’d like to share a research prototype that connects medical CT imaging to ROS 2 visualization and robot simulation.
The pipeline starts from a DICOM / NIfTI CT scan, segments kidneys and nearby organs, finds stone candidates, builds 3D meshes, ranks PCNL (percutaneous nephrolithotomy) access corridors, then visualizes the anatomy in RViz and Gazebo. A Franka FR3 with a surgical needle can replay the selected path in simulation.
This is not a clinical tool and not medical advice. It is a learning / research MVP: geometric planning + kinematic replay, not a validated surgical system.

You can watch the project’s video LinkedIn Video or Youtube Video.

pcnl_pipeline_grid_1920x1080(1)
pcnl_pipeline_grid_1920x1080(1)1920×1080 260 KB

  • Load DICOM series or NIfTI with spacing / origin / affine preserved
  • Preprocess: HU clip, resample (~1 mm), normalize
  • TotalSegmentator for left/right kidney and context organs (colon, liver, spleen, lungs, vessels, …)
  • Temporary stone segmentation: high-HU threshold + connected components inside the kidney ROI
  • Per-stone report: volume (mm³), equivalent diameter, centroid, min/mean/max HU, laterality
  • Marching cubes → optional smooth/simplify → STL / OBJ / PLY (meshes are in millimetres)
  • PCNL planner: sample a posterolateral cone, ray-cast skin → kidney surface → stone, rank by length, parenchyma, ribs, hilum, upper pole, and organ hits
  • PySide6 GUI drives the flow; a small bridge launches ROS 2 without importing rclpy into the venv

What actually do in the app

There is one window. Buttons on top, the CT slice on the left, a log and a stone table on the right.

  1. Load the scan

    Click Open NIfTI… or Open DICOM folder…. The CT appears as a slice you can scroll through, like flipping pages of the body.

  2. Find the kidneys

    Segment kidneys paints the left and right kidney (and nearby organs) on the scan. Extract kidney ROIs zooms in so you are not searching the whole abdomen.

  3. Find the stones

    Detect stones (HU) looks for bright spots inside the kidney — stones are denser than soft tissue. A table lists each candidate: left or right, volume, diameter, and how bright it is.

  4. See it in 3D

    Export STL saves kidney and stone shapes as 3D files. Show 3D meshes opens them in the app. Open RViz shows the same models in ROS, with the planned needle line drawn on top.

  5. Plan the needle path

    Pick a stone in the table, then click Plan PCNL. The program tries many directions from the back, avoids organs where it can, and ranks safer / shorter routes. You can switch between the best path and the backups.

  6. Watch the robot

    Simüle et (FR3) opens a simulated operating room: the patient model on a table, and a Franka robot with a needle. The arm lines up with the path, goes in to the stone, and İğneyi çıkar pulls it back out.

ROS is used for the part it is good at: showing 3D models and moving a robot. The heavy image work stays in a normal desktop app, then hands over files. That split kept the demo understandable and let one window drive the whole story: scan → stone → path → robot.

3 Likes
ROS News for the week of September 28th, 2026