Tuning StereoSGBM on PS5 Camera (ROS 2 Galactic) for Dense Depth Maps

ROS ROS General
1

Hi everyone,

I’m working on setting up a stereo camera PS5 HD Camera in ROS 2 Galactic for stereo depth estimation using stereo_image_proc (DisparityNode), with the end goal of feeding the depth data into RTAB-Map for 3D reconstruction.

I transitioned from StereoBM to StereoSGBM (stereo_algorithm: 1) to handle the wide-angle camera setup better, but I’m having trouble finding the optimal combination of parameters which result in a dense point cloud with smooth surfaces. My depth output keeps swinging between two extremes:

  1. Extremely sparse/empty with huge black gaps on uniform surfaces (e.g., walls, desks, pillows).

  2. Overly noisy with heavy color “speckle” artifacts across the scene.

What I’ve configured/tried so far:

  • Matching P1 and P2 to window size: Calculated and tuned P1 and P2 based on correlation_window_size (testing window sizes 5, 7, 9, and 13 with corresponding P₁ = 8 × C × WS² and P₂ = 32 × C × WS²). High P2 values help smooth out surfaces, but rqt_reconfigure caps P2 at 4000.0, so I’ve been overriding parameters via CLI/launch files.

  • Filter adjustments:

    • Lowered uniqueness_ratio (from 15.0 down to 5.0–7.0) to force coverage on weakly textured regions.

    • Set texture_ratio to 0.

    • Tuned speckle_size (50–200) and speckle_range (2–4) to filter out isolated noise clusters.

  • Search range & Offsets: Set disparity_range to 128 (multiple of 16) and kept min_disparity at 0 (raising it above 0 completely wrecked mid/far range depth).

Despite these adjustments, large homogeneous surfaces still disintegrate or become heavily fragmented unless I push the correlation window size to absurdly high values (which causes blocky, stepped artifacts).

Here is the code of my DisparityNode in the launch file:

ComposableNode(
                    package='stereo_image_proc',
                    plugin='stereo_image_proc::DisparityNode',
                    name='disparity_node',
                    namespace='',
                    parameters=[{
                        'stereo_algorithm': 1,          # 0 = StereoBM (30 FPS), 1 = StereoSGBM
                        'approximate_sync': True,
                        'queue_size': 2,
                        'correlation_window_size': 13,
                        'min_disparity': 0,
                        'disparity_range': 128,
                        'uniqueness_ratio': 5.0,
                        'P1': 1350.0,                         # 8 * 1 * 9^2 (for smoothing boundaries)
                        'P2': 4000.0,                        # 32 * 1 * 9^2 (for smooth surfaces)
                        'speckle_size': 150,
                        'speckle_range': 4,
                        'full_dp': False,
                    }]
                ),

Here I attach the results I got so far.

1ks7j200s3th1
1ks7j200s3th11499×867 188 KB

2u6jvrjko3th1
2u6jvrjko3th11524×687 175 KB

Questions:

  1. Are there specific pre-filtering parameters (prefilter_cap, prefilter_size) or SGBM settings I’m overlooking for this specific camera lens/sensor?

  2. Could this be a rectification/calibration alignment issue rather than pure SGBM parameter tuning?

Any advice or working configuration examples for similar stereo setups in ROS 2 would be greatly appreciated!