Poor teleoperation experience and physics instability with UR robot in Isaacsim

Competitions AI for Industry Challenge
1

Hi,

I am experiencing significant difficulties teleoperating the UR robot using the keyboard controller provided in the aic repository. Compared to the Franka Emika (standard Isaac Lab asset), which is smooth and responsive, the UR robot feels “heavy,” unresponsive, and highly unstable.

Issues observed:

  1. Geometric Clipping / Collisions: The UR robot parts are clipping through each other.

  2. Keyboard Responsiveness: Moving the end-effector via keyboard is extremely difficult.

  3. Cable Interaction: The cable seems to be “stuck” or tangled with the gripper even when not intended

    Comparison with Franka:
    While the Franka Emika follows SE(3) commands precisely and handles collisions gracefully, the UR robot in this setup enters a “crazy” state.

    Visual Evidence:

    download
    download1145×808 93 KB

download
download1145×808 94.1 KB

download
download1225×734 75.6 KB

System Info:

  • OS: Ubuntu 24.04.2 LTS

  • GPU: NVIDIA GeForce RTX 5090

  • Driver Version: 580.126.09

  • Isaac Sim Version: 5.1.0

  • Isaac Lab Version: Latest

Code:

cd ~
git clone git@github.com:isaac-sim/IsaacLab.git
cd ~/IsaacLab
git clone git@github.com:intrinsic-dev/aic.git
cd ~/IsaacLab
./docker/container.py build base
cd ~/IsaacLab
./docker/container.py start base
./docker/container.py enter base
./isaaclab.sh --install
python -m pip install -e aic/aic_utils/aic_isaac/aic_isaaclab/source/aic_task

isaaclab -p aic/aic_utils/aic_isaac/aic_isaaclab/scripts/teleop.py \
    --task AIC-Task-v0 --num_envs 1 --teleop_device keyboard --enable_cameras

Could you provide guidance on how to:

  • prevent the clipping shown in the images
  • make the keyboard teleoperation as responsive as the Franka Emika example

Thank you very much!