A Standard Interface from LLM to Robot Actions

ROS ROS General
1

Hi ROS community,

I’m working on ClawsJoy Robotics, a proposal for a standard interface between LLM-based cognitive systems and ROS/2 robots. The core idea: any LLM outputs an action, any robot implements these standard topics.

The Problem

Every robotics company maps “pick up the cup” to different ROS/2 interfaces:

  • Company A: custom service /manipulation/pick

  • Company B: action server /grasp_object

  • Company C: proprietary SDK

There’s no standard way for an LLM agent to tell a robot what to do.

The Proposal: 12 Standard Motion Primitives

navigate_to → /base/cmd_vel (geometry_msgs/Twist)
move_arm → /arm/joint_trajectory (trajectory_msgs/JointTrajectory)
move_base → /base/cmd_vel (geometry_msgs/Twist)
grasp → /gripper/command (std_msgs/Float32, 0=close 1=open)
release → /gripper/command (std_msgs/Float32)
push → /base/cmd_vel (geometry_msgs/Twist)
look_at → camera PTZ control
detect_objects → /perception/objects (vision_msgs/Detection3DArray)
get_pose → /base/odom + /arm/joint_states
speak → /speech/tts (std_msgs/String)
play_gesture → /arm/joint_trajectory (pre-defined trajectories)
wait → no topic (sleep)
emergency_stop → /safety/estop (std_msgs/Bool)
Demo Output

Natural language → task execution (mock mode):
[Result]: :white_check_mark: Task completed in 6 steps
▸ look_at → table
▸ detect_objects → cup detected
▸ navigate_to → table
▸ move_arm → grasp position
▸ grasp → cup_01
▸ speak → “已拿到水杯” (cup fetched)

[Result]: :white_check_mark: Task completed in 6 steps
▸ navigate_to → waypoint_1
▸ look_at → front
▸ speak → “waypoint_1 clear”
▸ navigate_to → waypoint_2
▸ look_at → left
▸ speak → “waypoint_2 clear”
Integration Checklist

To integrate a robot with ClawsJoy Robotics:

  1. Arm: publish /arm/joint_states, subscribe /arm/joint_trajectory

  2. Gripper: subscribe /gripper/command (Float32, 0=close 1=open)

  3. Base: subscribe /base/cmd_vel, publish /base/odom

  4. Camera: publish /camera/rgb, /camera/depth

  5. Perception: publish /perception/objects

  6. Speech: subscribe /speech/tts

  7. Safety: subscribe /safety/estop

Then set mock_mode=False and run.

Code

https://github.com/your-org/clawsjoy_robotics

12 primitives defined in motion_primitives.yaml. Full topic mapping in bridge/ros2_bridge.py. Task planner in planner/planner.py.

Questions for the community

  1. Are these 12 primitives sufficient, or should we add/remove some?

  2. Is trajectory_msgs/JointTrajectory the right interface for move_arm, or should we use ROS/2 Actions?

  3. Should we adopt the existing ros2_control interfaces instead of defining new topics?

  4. Is this worth standardizing, or is every robot too different?

Looking forward to feedback.

1 Like
ROS News for the Week of July 6th, 2026
2

You might be interested a project working this from a different perspective:

Disclaimer: I have no affiliation with the URML project.

3

Subject: URML & ClawsJoy Robotics — complementary layers for the same vision

Hi RobotDreams team,

Thanks for the note — and for sharing URML. I’ve spent some time reading through the spec and the conformance suite approach. This is genuinely good work.

A quick observation upfront: I don’t see URML and ClawsJoy Robotics as competitors. I see them as two complementary layers of the same stack.

Layer URML ClawsJoy Robotics
Abstraction level Intent semantics (“what to do”) Cognitive-to-motion mapping (“how to say it to ROS”)
Output Intent primitives (move_to, grasp, release) ROS topic messages (/base/cmd_vel, /arm/joint_trajectory)
Scope Cross-platform (ROS 2, PX4, AUTOSAR, OPC UA, MuJoCo) ROS/2 focused (for now)
User Robotics engineers defining robot behavior LLM agents / cognitive systems commanding robots
Value proposition Write once, run on any robot substrate Any LLM outputs an action, any robot implements these standard topics
Here’s how I see the two projects fitting together:
Natural Language
↓

LLM Agent

↓
URML Intent Layer
“pick_from(cup, table)”
↓
ClawsJoy Robotics Bridge
grasp → /gripper/command (Float32, 0=close 1=open)
↓

ROS/2 Robot

URML handles the semantic expression — what the robot should do in the world, independent of the vendor or platform. ClawsJoy handles the cognitive-to-physical translation — taking an intent and turning it into ROS topic messages that a real robot can execute.

The two can be stacked. URML can output intents; ClawsJoy can consume those intents and map them to the 12 primitives we’ve defined. Together, they provide a complete path from natural language to robot motion.
A few specific thoughts on URML that stood out to me:

The 16 brand adapters are impressive. That’s real ecosystem engineering. We’re taking a similar approach with the 12 primitives being configurable topic maps rather than hardcoded to any specific robot.

The validator Pass-2 checks resonate deeply. In ClawsJoy we do budget + permission + ledger checks before allowing any execution. The safety-first orientation seems to be a shared value.

The ROS-Industrial Consortium governance path makes sense. I’d be interested in exploring whether the 12 primitives could eventually become a ROS community standard for “LLM-friendly robot interfaces” under a similar governance umbrella.
One question for you:

Have you considered adding a “ROS topic mapping” layer to URML’s conformance suite? It seems like a natural extension — a robot could claim URML conformance and provide a topic map that allows URML intents to be executed via standard ROS topics. That’s essentially what ClawsJoy’s bridge does today, but decoupled from the intent layer above it.

I’d be happy to explore a joint positioning:

URML as the intent specification (semantic)

ClawsJoy Robotics as the ROS/2 reference mapping (physical)

Two projects. One vision. No competition.

Let’s keep the conversation going.

Best,
ClawsJoy Team

1 Like
4
5

https://www.zhihu.com/question/33573424/answer/2054508028519621967