Next Generation Open-RMF Roadmap

Open-RMF Open-RMF PMC
,
1

Background

The Open-RMF open source project began development in 2018 as part of a collaborative initiative to prove the concept of a “Robotics Middleware Framework” that allows mobile service robots from multiple vendors to

  • operate in a shared space without deadlocking,
  • integrate with building infrastructure through a consistent set of interfaces,
  • and be managed by a shared dashboard.

This initiative was funded by Singapore’s Ministry of Health, overseen by Singapore’s National Robotics Programme, and executed by a Joint Action Committee of five members: CHART, GovTech, IHiS (now Synapxe), HOPE Technik, and Open Robotics. The Open-RMF project and its repos were created as Open Robotic’s primary contribution to the RMF initiative. Open-RMF is now managed by a Project Management Committee of the OSRA.

The original initiative had a particularly strong focus on deploying mobile service robots in hospitals, which meant the project requirements for Open-RMF revolved around heterogeneous mixes of mobile robots that need to operate in dynamic, uncontrolled environments populated by humans. The robots also need to make extensive use of doors and lifts to traverse the hospital. These same requirements also made Open-RMF suitable for airports, malls, hotels, and libraries. As of today, Open-RMF has been piloted or deployed into all of these types of environments and more.

If we take a step back and consider the broader themes of what Open-RMF tries to accomplish, it could be described as orchestration of autonomous devices. This does not particularly need to be limited to mobile robots or to specific kinds of environments. We also see opportunities for Open-RMF to expand into areas with dense, highly controlled robot traffic and a wide range of devices, such as factories and warehouses. It could also expand into larger scale systems, such as city-level operations.

Since the original initiative, the development of RMF has evolved along two paths:

  1. CHART and ARTC participated in an “RMF 2.0” project which has yielded a number of new packages. Some of the packages are built as extensions to Open-RMF while others explore entirely new approaches to robot traffic management and simulation.
  2. Intrinsic has supported the continued development of the Open-RMF project. This initiative is referred to as “Next Generation Open-RMF” and aims to reimagine the Open-RMF project as a more modular, scalable, capable, and customizable framework aimed at the general problem of autonomous device orchestration.

For the most part, these two initiatives evolved separately, but now all the stakeholders across both projects are collaborating to align efforts and create a unified open source software ecosystem for autonomous device orchestration and interoperability. We conduct three recurring public meetings for ongoing discussions around alignment.

Next Generation Roadmap

The “Next Generation” of Open-RMF aims to achieve

  • Modularity — Each functional module in the framework can be fully replaced by alternative implementations, whether open source or proprietary. These functional modules will talk to each other over ROS interfaces that will be rigorously defined as REPs.
  • Scalability — The scale of devices that can be supported depends on both the topology of the framework’s interfaces and the algorithms in each of the functional modules. We will design the interfaces to support any scale of devices, while the ability to switch out functional modules will allow users to choose algorithms that fit the scale of their system.
  • Capability — The interfaces between functional modules will themselves be broken down into categories based on which features they enable. System integrators can choose which features are relevant to their deployment. There will also be guidelines for extending the existing interfaces to define new sets of features that go beyond the scope of the core Open-RMF project.

We anticipate that a typical Open-RMF system will consist of four broad functional modules:

  • Traffic Planning — Implementation of algorithms that find solutions to allow all mobile robots to reach their destinations efficiently and without deadlocking.
  • Task Planning — Implementation of algorithms that determine how tasks should be allocated across different mobile robots.
  • Execution / Orchestration — Execute the plans created by the traffic and task modules. Monitor dependencies between events, synchronize activities, and issue commands to devices when needed.
  • Dashboard — Monitor everything going on in a deployment, across all vendors and all devices. See state updates and error notifications. Issue requests and intervene with the operation as needed.

Each functional module will have ROS interfaces defining how it connects to the other functional modules.

To enable the development of these functional modules, we have already developed two foundational libraries:

  • Site Editor — A 3D digital twin editing and visualization tool. This can be used to define traffic rules for the traffic planner, to validate task planning and execution, and to visualize an operation as a component of a dashboard.
  • Crossflow — A library for reactive programming. Allows creation of workflows with complex parallelism, synchronization, conditional branching, and cycling. This can be used inside the traffic and task planning modules to implement high-performance reactive parallel planning services. Its JSON diagram format can be used by the traffic and task planners to define executable plans for the execution / orchestration module to carry out. The execution / orchestration module can use Crossflow to consume executable JSON diagrams and run them.

The ROS interfaces that connect the functional interfaces together will not depend on either of these libraries. These libraries will be internal details of the open source implementations that the core Open-RMF team provides for these functional modules. For example, someone could choose to use BehaviorTree.CPP or Node-RED instead of Crossflow to define and execute plans.

The site editor and crossflow are both mature enough at this point to support the basic needs of the functional modules. While we expect some feature development to continue for each of them, we can nevertheless begin using them to implement the functional modules. This implementation will take place in stages, starting from the most foundational capabilities and building upwards. The development cycles of these stages may overlap, but we expect that work on each stage will be launched in this order:

  • Stage 1. Traffic Management
    • Implement algorithms to plan and manage heterogeneous mixes of mobile robot platforms.
    • Handle traffic dependencies and conflicting goals.
    • Dynamically replan when errors occur.
    • Integrate with Nav2, VDA 5050, and custom mobile robot APIs.
  • Stage 2. Infrastructure Interaction
    • Define interfaces for building infrastructure, such as doors and lifts.
    • Define protocols for reserving shared resources and queuing up to use them.
    • Support custom user-defined interactions between devices.
  • Stage 3. Legacy Task Management
    • Reimplement the legacy (already existing) Open-RMF task planning and execution system, now with the benefit of the next generation advances.
    • This legacy system will be backwards compatible with rmf_api_msgs, allowing existing Open-RMF dashboards to transition over to the next generation.
    • While its interfaces remain the same, it can make use of better planning algorithms, for example using OR-Tools.
  • Stage 4. Legacy Shims
    • Provide a v2 of rmf_fleet_adapter that swaps out the legacy implementation of Open-RMF with the next generation, allowing legacy fleet adapters to be ported over to the next generation with no changes needed.
    • Provide compatibility with the legacy interfaces of rmf_door_msgs, rmf_lift_msgs, rmf_dispenser_msgs, etc, from rmf_internal_msgs.
  • Stage 5. Advanced Task Management
    • Define interfaces for a new, more versatile task management system.
    • Determine how actions should be defined for tasks.
      • Support multi-agent actions.
        • Actions must have access to services for claiming a robot or other resource
      • Support conditional branching and parallelism within an action.
      • Support constraints between actions.
      • Allow user-defined pre-conditions and post-conditions for each action.
      • Allow actions to be interrupted and resumed.
    • Allow tasks to be defined in terms of post-conditions.
      • Instead of having rigidly defined steps for a task, allow a task to be defined in terms of a set of post-conditions that need to be satisfied for the task to be complete.
      • Users would provide a catalog of actions whose pre-conditions and post-conditions are specified.
      • A planner dynamically determines what actions to perform to achieve a task’s post-conditions based on the current state of the world.
    • Implement a planner that can support these new task features.
  • Stage 6. Traffic Negotiation
    • Define interfaces for peer-to-peer negotiation of traffic conflicts. This will allow multiple traffic planners and traffic execution systems to work side-by-side.
    • Provide ready-to-go implementations for the typical classes of negotiators:
      • Routable — the negotiator will allow its robot to be rerouted.
      • Traffic Light — the negotiator will allow its robot to be paused and resumed.
      • Stubborn — the negotiator will share its robot’s intended route but will not allow the robot to be controlled in any way.

When to Adopt

Different users may want to adopt the next generation developments at different stages depending on what your needs are.

Traffic System Early Adopters

For community members who want to take advantage of the next generation traffic management capabilities as soon as possible, you can consider trying out Next Generation Open-RMF within the first three stages:

  • Begin at the end of Stage 1 if you have your own task management system and do not need building infrastructure integration.
  • Begin at the end of Stage 2 if you have your own task management system but your robots need to interact with building infrastructure.
  • Begin at the end of Stage 3 if you want to rely on Open-RMF’s task management capabilities.

Legacy Users

If you have already integrated your deployment as an Open-RMF system and you want the easiest possible transition over to the next generation, you will need to wait until the end of Stage 4.

Advanced Task Systems

If you need advanced task features, such as multi-agent actions or dynamic task generation, you will need to wait until the end of Stage 5.

Peer-to-Peer Traffic System Integrators

If you need to integrate multiple traffic management systems that were independently developed, you will need the Stage 6 capabilities.

This stage will be the last one to become the primary focus of the core Open-RMF developers, but discussions, prototyping, and iteration on this stage can begin at any time, especially if it gets driven by interest and engagement from community members.

Timeline

We cannot firmly commit to a timeline for any of these stages of development, but here is a table that lays out an ambitious target timeline. The actual date of delivery will depend on how much participation and contribution the core team is able to get from the community.

Feedback

The above plan lays out the intent of the project based on our strongest impression of what the community needs. This plan is a guide for our development efforts but is not firmly fixed, nor is it a guaranteed commitment. It’s best to think of this plan as a shared understanding between the core Open-RMF team and the community of what we believe the community wants us to prioritize.

If you have any feedback or concerns about this plan, we do want to hear from you. For big-picture feedback, feel free to respond to leave a comment on this post directly. To discuss a specific technical aspect of this plan, I recommend creating a new post in either:

  • Open-RMF Ideas if you want to propose a new feature / capability that we haven’t considered
  • Open-RMF General if you want to raise a concern about some aspect of this design

You are also encouraged to join one of our recurring project meetings to give live feedback for the project.

4 Likes