Gamepad Teleoperation for Robotic Arm—A New Intuitive Control Experience

ROS ROS General
, , , ,
1

Hi everyone! :waving_hand:

I’d like to share a project that enables intuitive control of the PiPER robotic arm using a standard gamepad. This might be useful for those working on robotic arm teleoperation or looking for a ready-to-use control interface.

Overview

This project provides precise and intuitive control of the PiPER robotic arm through a conventional game controller in a visual environment. It supports multiple kinematics libraries and offers both joint-space and task-space control modes.

Key Features:

  • Gamepad-based teleoperation
  • Joint control and Pose control modes
  • Gripper control with intuitive triggers
  • Forward/Inverse kinematics support
  • Web-based visualization
  • Position memory and playback

Repository: GitHub - kehuanjack/Gamepad_PiPER: This project implements the functionality of teleoperating a PiPER robotic arm using a gamepad.
Organization: GitHub - agilexrobotics/Agilex-College: Agilex College

image
image1849×1010 246 KB

Environment Setup

Requirements:

  • Operating System: Ubuntu 20.04 or higher
  • Python: 3.9 or higher (Anaconda/Miniconda recommended)

Installation:

git clone https://github.com/kehuanjack/Gamepad_PiPER.git
cd Gamepad_PiPER

Choose your kinematics library (cuRobo recommended):

Option 1: pinocchio library (Python 3.9) ```bash conda create -n test_pinocchio python=3.9.* -y conda activate test_pinocchio pip3 install -r requirements_common.txt --upgrade conda install pinocchio=3.6.0 -c conda-forge pip install meshcat casadi ```

Note: Requires piper_ros installation and sourcing the ROS workspace.

Select in main.py and main_virtual.py:

from src.gamepad_pin import RoboticArmController
Option 2: PyRoKi library (Python ≥ 3.10) ```bash conda create -n test_pyroki python=3.10.* -y conda activate test_pyroki pip3 install -r requirements_common.txt --upgrade pip3 install pyroki@git+https://github.com/chungmin99/pyroki.git@f234516 ```

Select in main.py and main_virtual.py:

from src.gamepad_limit import RoboticArmController
# or
from src.gamepad_no_limit import RoboticArmController
Option 3: cuRobo library (Python ≥ 3.8, CUDA 11.8 recommended) ⭐ ```bash conda create -n test_curobo python=3.10.* -y conda activate test_curobo pip3 install -r requirements_common.txt --upgrade sudo apt install git-lfs && cd ../ git clone https://github.com/NVlabs/curobo.git && cd curobo pip3 install "numpy<2.0" "torch==2.0.0" pytest lark pip3 install -e . --no-build-isolation python3 -m pytest . cd ../Gamepad_PiPER ```

Select in main.py and main_virtual.py:

from src.gamepad_curobo import RoboticArmController

Quick Start

  1. Connect robotic arm and activate CAN:
   sudo ip link set can0 up type can bitrate 1000000

  1. Connect gamepad via USB or Bluetooth

  2. Launch control service:

   python3 main_virtual.py  # Recommended for first-time testing
   # or
   python3 main.py  # For real robot control
  1. Open web interface: Navigate to http://localhost:8080 for visualization

Gamepad Control Mapping

Button Functions

Button Short Press Long Press
HOME Connect/Disconnect arm -
START Switch upper control mode (Joint/Pose) Switch lower control mode
BACK Switch command mode (0x00/0xAD) -
Y Return to zero position -
A Save current position Clear saved position
B Restore last saved position -
X Toggle playback order Clear all positions
LB Increase speed factor Decrease speed factor
RB Increase movement speed Decrease movement speed

Joysticks & Triggers

Control Joint Mode Pose Mode
Left Stick J1 (Left/Right), J2 (Up/Down) End-effector X/Y movement
Right Stick J3 (Up/Down), J6 (Left/Right) End-effector Z & rotation around Z
D-pad J4 (Left/Right), J5 (Up/Down) End-effector rotation around X/Y
LT Close gripper Close gripper
RT Open gripper Open gripper

Additional Features

  • Gripper Control: 0-100% range with quick toggle at extremes
  • Speed Control: 0.25x to 5.0x multipliers, 10%-100% movement speed
  • Position Memory: Save multiple waypoints with sequential/reverse playback

:warning: Safety Warnings

  • Start with virtual testing: Run main_virtual.py first
  • Begin at low speeds until familiar with controls
  • Maintain safe distance during operation
  • Beware of singularities: Numerical IK solutions may cause sudden joint movements
  • Quick response mode (0xAD) is dangerous: Use with extreme caution

Contributing & Feedback

Feel free to try it out and share your feedback! Issues and pull requests are welcome on the GitHub repository.

Has anyone else worked on similar gamepad teleoperation projects? Would love to hear about your experiences!

Cheers! :robot:

2

Is this using MoveIt Servo? I.e. how does this compare to Realtime Arm Servoing — MoveIt Documentation: Humble documentation? This joystick/gamepad example node was removed in jazzy, it would be quite useful to have this support enabled again.