Letting Python Be Python: Should ROS 2 be less strict with the Python ecosystem? Notably pip, conda, venv

You say in the Github issue:

I do feel like ROS missed a lot of how/why people use python.

I don’t think this is true. ROS allows Python to be used. However, because of the constraints of the tooling, the age of the project, and the bandwidth of developers, it allows Python in a restricted way that adds complexity and long delays in compatibility. It’s not an awareness issue, it’s about hours of labor.

Modern packaging is one of the headaches. As @peci1 says in the issue you linked:

But I guess something has to move very soon, because it’s not manageable to still not support pyproject.toml in 2026 (I think :slight_smile: ).

It may need to move very soon to keep Python relevant as a first-class citizen inside ROS 2, but as far as I can tell this effort has been needed for years and is still outstanding:

I can speak for myself. I’m a heavy Python user, but in a ROS 2 context I will just kind of will ignore this until it gets fixed, whenever it gets fixed. From that issue, @clancette says:

Yes, that is probably the case. This needs to be investigated, but I suspect it will take changes to colcon, ament, and every single python package in the ROS 2 ecosystem. This is on the list to investigate, but since the Humble feature freeze is in 3 working days, this is almost certainly an item for the next release.

And later:

Still the same answer as #382 (comment) , unfortunately. We hope to have it fixed for Jazzy in May 2024.

Some more recent discussion here:

A couple days ago @peci1 asks:

Have the problems with symlink install been resolved? How do we use it? The PR says it will be enabled by a feature flag, but it seems there’s no flag added in the PR.

And what about the buildfarm support?

So, four years ago we start getting deprecation spam and it’s maybe getting close to being fixed on the packaging/setuptools deprecation side?

But it’s not mainstream-fixed yet. One reasonable conclusion is that fixing it is not an urgent priority for the community compared to all of the other community demands.

Most of the people I work with – students and companies – are Python-first rather than ROS-first.

I think this will be increasingly true. But it’s been true for a while. Anyone working in machine learning/AI is going to be Python-first especially.

But in my experience and understanding, there’s nothing about ROS 2 that’s really pointing to it being a Python-first toolkit. The broader community doesn’t demand it to be Python-first, we don’t put a shoulder into long-open issues and PR testing to help make it Python-first.

I certainly just suffer with setuptools.py deprecation spam in all of my projects. But there, a Python node tends to be an afterthought, throwaway, or simple tool or utility. It isn’t part of the core system I’m building.

Python is very useful but Python ROS 2 nodes still suffer from execution performance problems due to the rclpy executors. We have the experimental events executor for single-threaded Python nodes, which works great (and maybe isn’t still “experimental”? … I’m a Jazzy user). But if I want a simple multithreaded Python node, as far as I know it’s still going to hog a large amount of CPU compared to some other Python-ecosystem wrapper around a performant C, C++, Rust, or whatever backend.

I was really excited in the ROS 1 to ROS 2 transition to get Python launch files, but then years later I fully sit with the fact that Launch Python is an intermediate representation in a declarative system, and now that the frontends are mostly there, I’m taking my team back to XML.

Pixi+RoboStack+Conda solves many problems, but does not feel like fixing the core issue.

I think these are actually an ideal fix for Python-first roboticists and I think it’s okay that there are a couple of parallel tracks running now.

Lots of robotics projects don’t need ROS 2 at all. I think a good example of this is Speedfolding:

It’s Python-first, wraps C++ libraries for perception and motion, glues it all together with Python. All that type of tooling is getting better and for the ML/AI workflows it’s been the glue for years and years.

So what do your users need to use ROS for?

I think scoping the role of ROS in a larger project can sometimes make a nice clean separation between the part that runs in ROS 2 and the part that runs in a modern best-practices Python environment. Why force it into the ROS 2 Python world if you can create a decoupling break somewhere? You could have your Python env and your ROS 2 robot drivers running on your core system and just have them talk to each other.

Sometimes you want more up-to-date Python tooling, want to use ROS 2, and for deployment and reproducibility want pinned packages in a single environment. Pixi is perfect for this, IMO. I’ve been a happy Robostack user for many years, and it just gets better and better if you want to mix ROS 2 with modern Python.

The option to use Python and the work that people are doing on making Python better in ROS 2 is great, and I don’t want to diminish their efforts. I think there is an important push going on to resolve “the core issue.”

But it seems like it’s been a complex and lengthy process,maybe because of the way the ROS 2 tooling works and what seasoned ROS 2 users expect from that tooling. If I see “--symlink-install is not working” in a pure-ROS-2 workflow I run the other way, and if I have core functionality that I want to implement in Python that interoperates with ROS 2 I will probably set this up in Pixi.

Pixi is rapidly offering alternatives to Colcon and rosdep as far as I can tell. I think this is a great way for Python-first people to work, and I’m definitely interested in exploring the ROS build tooling even though I’m not “Python-first.” Creating and consuming my own Conda packages is a nice addition to workflows I care about personally.

But Colcon and ROS 2 buildfarm support remain critical things throughout the ROS 2 infrastructure and for most ROS 2 users. Whatever “Python-first” energy comes to core ROS 2 it’s constrained to be “Colcon-first” as well. And it needs to allow mainstream ROS 2 packaging to reach the widest ROS 2 audience.

So I think I’m thinking: sure, let Python be Python, but also let ROS be ROS. I’m sure eventually ROS 2 will converge on having first-class Python support, but it’s been a long and probably under-resourced project.

Pixi and conda-forge have been providing a good solution to this problem, and who knows how things end up in a few years.

If I wanted to use your package in my ROS 2 project, your lack of ROS 2 packaging would just cause me to use Pixi for my project.

For people who have less familiarity and who always consume ROS 2 via Tier 1 supported installs and binary buildfarm packages, it’ll add an element of friction if you don’t package your software as a ROS 2 Python package with easy dependency resolution with rosdep.

I don’t really think it’s that urgent of a problem in that sense, especially as we all talk more about these different options we’re trying out.

2 Likes