Nayantra – Natural Language Fleet Control via LLM + Open-RMF + MCP

Open-RMF Open-RMF General
, , , ,
1

TL;DR: Nayantra is an open-source framework that lets you command autonomous robot fleets in plain English. An LLM communicates with Open-RMF through the Model Context Protocol (MCP), routing a single natural language prompt all the way down to Nav2 on real hardware or in NVIDIA Isaac Sim. The full pipeline also runs locally on a laptop in stub mode, no GPU, no RMF cluster required.

Motivation: Controlling a robot fleet today typically requires writing dispatcher code, manually constructing task requests, and deeply understanding Open-RMF’s API surface. I wanted to explore whether a large language model, given the right tools, could act as a natural-language interface on top of RMF, letting operators or developers issue commands the way they’d describe them to a colleague.

What Nayantra does:

  • Accepts free-form English commands (e.g. “Send robot A to delivery point 3 and put robot B on standby”)

  • An LLM parses intent and calls Open-RMF APIs via MCP tool calls

  • Tasks are dispatched through Open-RMF and executed by Nav2 on real robots or inside Isaac Sim

  • Stub mode lets you run and test the entire pipeline locally with no infrastructure

Current status: This is an early open-source release. The stub mode works end-to-end and is the easiest way to get started. Real-robot and Isaac Sim integration is functional but still maturing.

Repo: GitHub - shashankbr27/nayantra: An open-source framework that connects large language models to autonomous robot fleets via the Model Context Protocol (MCP) and Open-RMF. Drives NVIDIA Isaac Sim or real Nav2 robots; a built-in stub mode runs the full pipeline on a laptop. · GitHub

I’d love feedback from folks with deeper Open-RMF and Nav2 experience, especially around task API design, edge cases in fleet dispatch, and ideas for extending the MCP tool surface. Issues, PRs, and blunt criticism all welcome.

1 Like
ROS News for the Week of May 18th, 2026
2

Thanks for sharing this @shashank_br !

I think this is an excellent ecosystem addition for Open-RMF. If you have any feedback on how the Open-RMF APIs can better accommodate integrating into an MCP server, we’re happy to hear about it.

If you’d like to present on your work, this could be a great topic for the Interoperability Special Interest Group that we hold once per month. Feel free to message me directly if you’d like to be added to the agenda.

4

Hi @grey,

Following up on the Interoperability SIG offer. I’d love to be added to the next agenda. A 10-15 min walkthrough of Nayantra (architecture, planner, MCP tool surface, Isaac Sim bridge, demo) with ~10 min Q&A would be a comfortable slot. Could also do shorter if the agenda’s tight.

My time zone is IST (UTC+5:30); I can be flexible for the usual SIG slot. Let me know what dates are open and what format you’d prefer (slides, live demo, or both).

Thanks again for the invite!