This has been bothering me for a while, especially as Python 3.12 further normalizes virtual environments. ROS appears to diverge or restrict parts of the standard Python workflow: why not instead embrace existing tools like venv, pip, conda, pytest, and asyncio?
[Me] Aside from the time investement on my side, I feel like the end user will not appreciate a ROS package much. Most people I know have a venv anyway, and much prefere to simply ‘pip install’ instead of fiddleling with rosdep.
[Peci] But this workflow is not very well aligned with ROS. Combining ROS and venvs has always been quite some pain. I agree that if you don’t need ros, venvs are great.
[Me] Yes this workflow does not align with ROS, but this is the workflow of most python tooling and the python community. It feels like – in many places – python is neutered, not allowed to use its own tools and style. All for minimal benefits to the user.
I understand many of the underlying design choices: […]. Still, I would trade these for integration with the Python ecosystem. In practice (not theory), many of these fundamentals are shaky for Python because projects often rely on environments and PyPI dependencies anyway. I can hardly tell students/colleagues “Do not use pypi/conda”.
[Peci] I personally never understood how can anyone build PyPI-based projects of any size larger than a toy example. I consider pip more a “random package downloader”.
What’s the chance pip will find compatible versions? I’d say zero, but reality says 90%.
I realize that many participants on this forum are primarily ROS developers, often with a strong C++ background, so I totally understand @peci1 perspective. My own background is different: I’m a Python developer building robots with ROS since Foxy in a research lab setting. Most of the people I work with – students and companies – are Python-first rather than ROS-first.
For these users, the friction is hard to justify: their applications work well with standard Python tooling, yet integrating with ROS requires significant workflow (and dependency) changes. I myself don’t see the point anymore when I could have different virtual environments communicating between each-other using a transport layer (which many user end-up doing using Docker… a bit overkill).
Anyway I am curious about how the growing divergence between ROS and mainstream Python tooling could be addressed before it widens further. Pixi+RoboStack+Conda solves many problems, but does not feel like fixing the core issue. The way forward might be embracing pip, conda and venv, for the – vast majority of – users that don’t care and are just prototyping. And for the users who care, having isolated environments per ros package/node (like pipx, uvtool and pixi global).
I do feel like ROS missed a lot of how/why people use python.
I don’t think this is true. ROS allows Python to be used. However, because of the constraints of the tooling, the age of the project, and the bandwidth of developers, it allows Python in a restricted way that adds complexity and long delays in compatibility. It’s not an awareness issue, it’s about hours of labor.
Modern packaging is one of the headaches. As @peci1 says in the issue you linked:
But I guess something has to move very soon, because it’s not manageable to still not support pyproject.toml in 2026 (I think ).
It may need to move very soon to keep Python relevant as a first-class citizen inside ROS 2, but as far as I can tell this effort has been needed for years and is still outstanding:
I can speak for myself. I’m a heavy Python user, but in a ROS 2 context I will just kind of will ignore this until it gets fixed, whenever it gets fixed. From that issue, @clancette says:
Yes, that is probably the case. This needs to be investigated, but I suspect it will take changes to colcon, ament, and every single python package in the ROS 2 ecosystem. This is on the list to investigate, but since the Humble feature freeze is in 3 working days, this is almost certainly an item for the next release.
And later:
Still the same answer as #382 (comment) , unfortunately. We hope to have it fixed for Jazzy in May 2024.
Have the problems with symlink install been resolved? How do we use it? The PR says it will be enabled by a feature flag, but it seems there’s no flag added in the PR.
And what about the buildfarm support?
So, four years ago we start getting deprecation spam and it’s maybe getting close to being fixed on the packaging/setuptools deprecation side?
But it’s not mainstream-fixed yet. One reasonable conclusion is that fixing it is not an urgent priority for the community compared to all of the other community demands.
Most of the people I work with – students and companies – are Python-first rather than ROS-first.
I think this will be increasingly true. But it’s been true for a while. Anyone working in machine learning/AI is going to be Python-first especially.
But in my experience and understanding, there’s nothing about ROS 2 that’s really pointing to it being a Python-first toolkit. The broader community doesn’t demand it to be Python-first, we don’t put a shoulder into long-open issues and PR testing to help make it Python-first.
I certainly just suffer with setuptools.py deprecation spam in all of my projects. But there, a Python node tends to be an afterthought, throwaway, or simple tool or utility. It isn’t part of the core system I’m building.
Python is very useful but Python ROS 2 nodes still suffer from execution performance problems due to the rclpy executors. We have the experimental events executor for single-threaded Python nodes, which works great (and maybe isn’t still “experimental”? … I’m a Jazzy user). But if I want a simple multithreaded Python node, as far as I know it’s still going to hog a large amount of CPU compared to some other Python-ecosystem wrapper around a performant C, C++, Rust, or whatever backend.
I was really excited in the ROS 1 to ROS 2 transition to get Python launch files, but then years later I fully sit with the fact that Launch Python is an intermediate representation in a declarative system, and now that the frontends are mostly there, I’m taking my team back to XML.
Pixi+RoboStack+Conda solves many problems, but does not feel like fixing the core issue.
I think these are actually an ideal fix for Python-first roboticists and I think it’s okay that there are a couple of parallel tracks running now.
Lots of robotics projects don’t need ROS 2 at all. I think a good example of this is Speedfolding:
It’s Python-first, wraps C++ libraries for perception and motion, glues it all together with Python. All that type of tooling is getting better and for the ML/AI workflows it’s been the glue for years and years.
So what do your users need to use ROS for?
I think scoping the role of ROS in a larger project can sometimes make a nice clean separation between the part that runs in ROS 2 and the part that runs in a modern best-practices Python environment. Why force it into the ROS 2 Python world if you can create a decoupling break somewhere? You could have your Python env and your ROS 2 robot drivers running on your core system and just have them talk to each other.
Sometimes you want more up-to-date Python tooling, want to use ROS 2, and for deployment and reproducibility want pinned packages in a single environment. Pixi is perfect for this, IMO. I’ve been a happy Robostack user for many years, and it just gets better and better if you want to mix ROS 2 with modern Python.
The option to use Python and the work that people are doing on making Python better in ROS 2 is great, and I don’t want to diminish their efforts. I think there is an important push going on to resolve “the core issue.”
But it seems like it’s been a complex and lengthy process,maybe because of the way the ROS 2 tooling works and what seasoned ROS 2 users expect from that tooling. If I see “--symlink-install is not working” in a pure-ROS-2 workflow I run the other way, and if I have core functionality that I want to implement in Python that interoperates with ROS 2 I will probably set this up in Pixi.
Pixi is rapidly offering alternatives to Colcon and rosdep as far as I can tell. I think this is a great way for Python-first people to work, and I’m definitely interested in exploring the ROS build tooling even though I’m not “Python-first.” Creating and consuming my own Conda packages is a nice addition to workflows I care about personally.
But Colcon and ROS 2 buildfarm support remain critical things throughout the ROS 2 infrastructure and for most ROS 2 users. Whatever “Python-first” energy comes to core ROS 2 it’s constrained to be “Colcon-first” as well. And it needs to allow mainstream ROS 2 packaging to reach the widest ROS 2 audience.
So I think I’m thinking: sure, let Python be Python, but also let ROS be ROS. I’m sure eventually ROS 2 will converge on having first-class Python support, but it’s been a long and probably under-resourced project.
Pixi and conda-forge have been providing a good solution to this problem, and who knows how things end up in a few years.
If I wanted to use your package in my ROS 2 project, your lack of ROS 2 packaging would just cause me to use Pixi for my project.
For people who have less familiarity and who always consume ROS 2 via Tier 1 supported installs and binary buildfarm packages, it’ll add an element of friction if you don’t package your software as a ROS 2 Python package with easy dependency resolution with rosdep.
I don’t really think it’s that urgent of a problem in that sense, especially as we all talk more about these different options we’re trying out.
@2lian maybe it would help the discussion if you could sketch up the ideal, Python-first experience. Like: the user starts with a fresh OS, what does he do to get developing with Python and ROS?
And , most importantly, how would the user consume (and get/install) the C++ packages (at least for usage in launch files).
For me, the Pixi workflow feels close to ideal in some ways. The tooling is very nice and I can mix and match OS and ROS distro easily. Binaries are available for a large number of packages. conda-forge is great for resolving dependencies for mixed Python/C++ projects.
One enhancement for Python-first users that I can imagine would be access to ROS 2 binaries built against newer Python versions.
As it stands, let’s say I want to try Python 3.14:
dan@computer:~$ pixi init doomed_ros2_package -c robostack-jazzy -c conda-forge
✔ Created /home/dan/doomed_ros2_package/pixi.toml
cd doomed_ros2_package
dan@computer:~/doomed_ros2_package$ pixi add python=3.14 ros-jazzy-ros-core
Error: × failed to solve requirements of environment 'default' for platform 'linux-
│ 64'
├─▶ × failed to solve the environment
│
╰─▶ Cannot solve the request because of: ros-jazzy-ros-core * cannot be
installed because there are no viable options:
├─ ros-jazzy-ros-core 0.11.0 | 0.11.0 | 0.11.0 | 0.11.0 | 0.11.0 |
0.11.0 | 0.11.0 would require
│ └─ numpy >=1.26.4,<2.0a0, which cannot be installed because there are
no viable options:
│ └─ numpy 1.26.4 | 1.26.4 | 1.26.4 | 1.26.4 | 1.26.4 would require
│ └─ python >=3.9,<3.10.0a0, for which no candidates were found.
└─ ros-jazzy-ros-core 0.11.0 would require
└─ python 3.12.* *_cpython, for which no candidates were found.
...
The robostack-<distro> channel packages are built against specfic versions of Python; I believe they’re now same as the Tier-1-supported Python for the ROS distro on the given platform.
In many cases, I think bleeding-edge Python is something that can have limited value to mainstream ROS 2 users. Key ecosystem packages don’t provide much support outside of Tier 1 Linux configurations even if core ROS 2 support is there. For example, this simple MoveIt issue I encountered years ago using Robostack.
On the other hand, I spent a lot of time running Robostack on Windows and it was nice to not be stuck on Python 3.8.3 forever. So unpinning ROS 2 distro from a specific Python version has advantages. “Everything works” is not necessarily something I needed in my use case there. Just some of the basics and RViz, so having difficult or non-existent package support outside of the basics was fine. When I was doing this, most of the packages I needed in the bigger system didn’t meaningfully support Windows anyway.
I’m not sure how easy or hard it is to build your own custom ros-<distro>-ros-core from source against a particular version of Python using Pixi. I’m playing around with it now but hitting some walls.
Thanks for you very detailed replies. It helps a lot understand the issue.
I agree that pixi is pretty much ideal for creating an environment. Building packages and working with multiple distros is not 100% there yet (I usually have to create one pixi.toml per distro I support in order to pixi-build them all). Anyway, not so important. My main worries with pixi is: “should ROS again change (part of) its build process? only for it to once again not be good enough in a few years after the Python community evolves?”. We already went through catkin, ament, colcon… Pixi+RoboStack could very well be a final solution, but spending time and energy on this gets me a little worried.
I do agree it’s a lot of work, we want a solution that will not increase the – already arduous – maintenance of ROS and ROS packages. On one side we have maintainers struggling to move forward, and the other side the users that are being held back. This is what I meant by “missed how/why people use python”, the current implementation is putting both sides in a bad position. I deeply appreciate the efforts from everyone, that’s why I want to not waste those efforts.
One thing I am reading a lot is ROS allows Python to be used, kinda “ROS has python inside” approach. Maybe that’s the problem and it should be the other way around? Because this means ROS has to maintain an entire python ecosystem, that’s a huge amount of work and will always lag behind the (gigantic) python community.
That’s what I meant by Letting Python be Python. If we could let it go, we wouldn’t need to spend energy chasing every Python changes, and ROS could focus on more important issues.
(I know that’s easy to say, hard to do.)
I don’t want to spam many dumb ideas without being aware of most of the design decisions. And I also don’t want to make “demands” to open source projects. So take the following as ideas, not as “please do this”.
Let’s take the example of gogo_keyboard that’s just sending Strings.
0. Pre-amble
One thing I noticed while writing. Maybe more environment isolation is the solution. Interoperability between env can be done with the transport layer and message types (which are pretty good). I can easily see each node/package having its own environment.
Okay I know that’s very far fetched and you are probably gauging your eyes out right now (sorry about that). However the python community clearly doesn’t see this as a cardinal sin. If all I need is to send a string on the network, this should be good enough. Furthermore if using pipxuvtool or pixi (or properly managed venv), then gogo_keyboard could live in a entirely isolated environment and not interfere with the code/dependencies of the other nodes/packages. The transport layer of ROS along with the default message types can handle interoperability between widely different environments. And a launcher could still launch it.
One thing is that, this imaginary py-ros2 does not need to be fully featured. Many projects – such as gogo_keyboard – are extremely simple. If gogo_keyboard is kept in an isolated environment, a full-fledged ros install could still launch and communicate with it.
This is equivalent to Pixi (however with less fine-grain selection of dependencies). However it would be compatible with the whole python ecosystem.
Having rosdep gather all python dependencies in ./src and compile them into a requirements.txt (kinda like a lock file) would already greatly help. If doing this, having python dependencies inside setup.py and pyproject.toml would do something. Then colcon should interface better with venvs.
That’s kinda likely to break things, so rosdep pyreqs ./src > requirements.txt would only consider “build from source” packages in ./src. This would already greatly help.
Right now I can do that, but it’s a long process, and not very reliable:
cd ros_ws
. /opt/ros/jazzy/setup.bash
rosdep install --from-paths src --ignore-src -r
python3 -m venv --system-site-packages ./.venv
. ./.venv/bin/activate
python3 -m pip install --upgrade pip wheel
cd ./src
python3 -m pip install ./my_pkg1 ./my_pkg2 ./my_pkg3 ... # for all python packages
python3 -m pip uninstall my_pkg1 my_pkg2 my_pkg3 # colcon will install them later
cd ..
python3 -m colcon build --symlink-install # calling `colcon` doesn't work, `python3 -m colcon` is necessary
source ./install/setup.bash
3. Isolated ROS packages
Let’s say I have gogo_keyboard on pypi. Then I should be able to make an isolated ros package that implements a simple pub/sub, or even simpler, can directly call entry points defined in the gogo_keyboard pypi package. During the colcon build process or rosdep, an environment is created exclusively for the gogo_keyboard package.
Let’s say
cd ros_ws/src
ros2 pkg create --build-type py-isolated my_package
This creates a properly formatted pyproject.toml, maybe with a few additions for ros. And I add gogo_keyboard to the pyproject.toml.
The only thing I really don’t like is “who is responsible for creating the venv?”. I would NOT want ROS or Colcon to do it, because that’s even more thing to support in ROS. But maybe that’s the only way.
Maybe something like this to work with any python tooling
cd ros_ws/src
ros2 pkg create --build-type py-isolated my_package
cd my_package
uv venv
uv add gogo_keyboard
cd ..
# following will create the venv, or pick the one the user created
rosdep install --from-paths src --ignore-src -r
But then how is ROS available inside this venv? because the venv will not be aware of ROS unless we use --system-site-packages. Maybe there’s a clean way by tweaking pyproject.toml? Idk.
4. Isolated Python environments
This groups point 0.1. and 3., maybe we shouldn’t build python packages? but propose environments instead. Colcon build will not touch them (maybe just keep a reference of the path to the env), and a launcher will activate then start the node in the env. Pixi can already more or less do that.
…
I am tired of writing so much, but here are what I’d see as the next steps. I am sure you’ll have better idea / improvements / critics.
It seems to me that what you’re maybe asking could be that the ros-core (only the bare bones) packages are available on pypi and can be used as python dependencies without explicitly installing ROS? I could imagine that. It could be said that everything in ros-core has to be compatible with all currently supported Python versions. It’s not so much packages and most of them don’t use Python directly, so it shouldn’t be a big problem.
Yes that would be a fantastic shortcut, but I’d assume that there is a good reason for it not existing yet. For something like this, using ROS only as a transport (and message serializer) would be enough. Nothing more, no executor because asyncio can do this job, no launcher because python entry-points already work…
I have been using Zenoh a lot recently instead of ROS 2 (because most of what I need is to send payloads). I can intercept ROS 2 messages with it, but not parse them, nor serialize them, nor declare a node. I’ve a feeling it should be possible to make a participant on the RMW that is fully pypi. Preferably using ros-core, but maybe without it, with very limited feature as a first proof of concept.
Now that I think of it, if distributing on pypi is too complex, maybe it might be possible to do pip install /opt/ros/jazzy/py_pgks to more cleanly expose ROS in a environment.
I think serialization and deserialization of arbitrary ROS 2 messages is exactly the thing that might make a limited-feature proof-of-concept relatively UNdesirable as an “officially supported” ROS 2 component. The message definitions are generated “inside” ROS 2 in some sense and users expect them to be there for an interop tool.
It’s not that you couldn’t “externalize” some necessary subset of messages. It’s that as a tool for general use cases, I don’t think ROS/OSRF should arbitrarily pick a subset to externalize and implement. It’ll just disappoint users.
I think it’d be excellent if there were a standalone PyPi or conda-forge implementation that can serialize and deserialize all core/standard ROS 2 messages with easy extension to custom messages, but I don’t know how hard that would be.
I’ve a feeling it should be possible to make a participant on the RMW that is fully pypi. Preferably using ros-core, but maybe without it, with very limited feature as a first proof of concept.
For those who are curious about the details, the ROS2 messages are encoded for DDS following the OMG DDSI-RTPS specification (see §10) in CDR format (see §9.3). But fortunately, you usually don’t need to implement a CDR encoder/decoder, since there are libraries for this in most languages (Python, Rust, C#, Javascript…)
At first I thought this was going to be really simple, but the failure, I think, illustrates something.
In their examples they use the pycdr library which provides a cdr decorator on a kind of dataclass-like declaration (probably are dataclasses under the hood):
...
from pycdr import cdr
from pycdr.types import int8, int32, uint32, float64
...
@cdr
class Vector3:
x: float64
y: float64
z: float64
@cdr
class Twist:
linear: Vector3
angular: Vector3
These classes then apparently automatically implement a .serialize() and .deserialize() option that works with CDR payloads.
OK, cool. This seems nice and clean. I have to re-write some decorated dataclasses to get message definitions, but for a well-scoped external PyPi participant, that seems like no big deal at all. If it gets cumbersome because I have a lot of custom messages, I could cook my own code-gen.
So now I want to consider this exact workflow in my project. Let’s take a look at pycdr’s PyPi page
The author of this package has not provided a project description
OK, so I guess to get an updated and maintained pycdr maybe I need to simply install Cyclone DDS Python bindings? That’s available on PyPi, right? cyclonedds · PyPI
What if I just add cyclonedds to the pixi.toml I’m trying to set up in my Conda/PyPi RMW participant? (or imagine I do this in my pure PyPi workflow, which I don’t have)
[pypi-dependencies]
cyclonedds = "*"
Then I get this:
(doomed_ros2_package) dan@computer:~/doomed_ros2_package$ pixi install
Error: × failed to solve the pypi requirements of environment 'default' for
│ platform 'linux-64'
├─▶ The build backend returned an error
╰─▶ Call to `setuptools.build_meta.build_wheel` failed (exit status: 1)
[stdout]
Could not locate cyclonedds. Try to set CYCLONEDDS_HOME or
CMAKE_PREFIX_PATH
Oh, of course. In order to use the Cyclone DDS Python bindings, I need to actually have Cyclone DDS installed. Where can I get it? Where should I install it? Why would I install it? I thought I was installing Zenoh to use a Zenoh participant on a ROS 2 Zenoh system.
OK, fine. I’ll go with the archived, five-year-old version of pycdr on PyPi. Alter my pixi.toml to add that in pypi-dependencies (this workflow is very nice!) and I get:
(doomed_ros2_package) dan@computer:~/doomed_ros2_package$ pixi install
Error: × failed to solve the pypi requirements of environment 'default' for
│ platform 'linux-64'
├─▶ failed to resolve pypi dependencies
╰─▶ Because all versions of pycdr were yanked and you require pycdr, we can
conclude that your requirements are unsatisfiable.
Yanked?
There’s probably a simple solution for this, but as I look for a next step so I don’t have to implement the CDR serialization/deserialization myself, I’m kind of going in circles.
All that said, I just don’t think that ROS to non-ROS interop is within the design scope of ROS 2.
I’ve been involved with a couple of projects that want to “inject a little ROS in a non-ROS application” and they tend to fall apart like this pycdr example, but much worse. It’s not even a ROS thing in this case. Seems to be on Eclipse at this point if we need to fully install Cyclone DDS to get a Python serializer/deserializer for CORBA CDR payloads.
If you scope it appropriately, are willing to write enough code, and intend to maintain the system, I think it should be easy enough already to make an external Python-first participant.
What it’s not as easy to do is find tooling for a casual user to build an arbitrary external participant that has convenient interop with arbitrary ROS 2 messages (even core ones).
It seems like one thing that would help is a truly stand-alone CORBA CDR implementation on PyPi and/or conda-forge. I’m still looking. Foxglove has one in Typescript.
So the pycdr issue above seems to be partially from aging documentation of ROS 2 - Native Zenoh bridging. I found pycdr2 here which I think can do the same job, except that it doesn’t seem at first glance to support the nice @cdr decorator: https://pypi.org/project/pycdr2/
I think this might be usable as the serialzation/deserialization part of your external RMW participant using Zenoh, with the overhead on you/your users of re-defining the required messages.
There are other projects that do try to provide serialization/deserialization of the common ROS 2 messages. Feels like there might be some promise in using rosbags.serde
The library supports two primary message formats:
Common Data Representation (CDR): This serialization format is widely utilized by most ROS2 middlewares. More information on CDR can be found here.
The project is aimed at decoding rosbags but maybe it’s not too hard to use live to decode the payloads you’re getting from your Zenoh participant?
I have been using Zenoh a lot recently instead of ROS 2 (because most of what I need is to send payloads).
For some context on my perspective why I’m not so interested in OSRF or other core ROS 2 maintainers spend much time on ROS-message interop outside of the ROS environment:
Message payloads and pub/sub are the least important part of why I use ROS. If that’s most of what I needed, I probably would not use ROS myself.
I use it for geometry and physics and nice abstractions of hardware.
I can keep track of transforms with tf, I can rely on ros2_control as a good abstraction of a motor/actuator system that allows me to run the same code to command real hardware using ros2_canopen motor controllers as I use to command Gazebo simulations. The simulations rely on bespoke physics models of articulated systems and having some scaffolding and structure to encode physics properties and kinematics into URDFs is useful.
A huge friction in the kind of work I do involves clashes among coordinate frame conventions, unit conventions, and three-dimensional transforms of wrenches, poses, and twists. Having everything be NAMED Wrench, Pose, and Twist and having everything be in SI units are just by themselves important constraints. Working “inside” ROS 2 brings these all in automatically. It’s not the only way, but this kind of thing is a well-thought-out part of a Robot Operating System.
There are other nice debugging tools that I find critical like visualizing the running node graph, but I think many other pub/sub middlewares can do that by themselves, so maybe I don’t even need rqt_graph in an alternative system.
There are a few other frameworks that do offer some of the critical stuff I need, but not quite yet as a one-stop-shop for physics and controls and simulation of geometrically complex robots that can take non-trivial 3D poses in their environment.
I think you’d probably find that a lot of ROS 2 users also have some subset of the tools outside of the middleware that are critical to their workflow. These things are not easily replaced.
I’m not saying interop isn’t important. I just feel like it’s pretty reasonable to have external interop, alternative installation, and other things like that be handled by non-ROS contributors to the broader ROS-compatible ecosystem, and there really is a lot of work being done there.
One thing I am reading a lot is ROS allows Python to be used, kinda “ROS has python inside” approach. Maybe that’s the problem and it should be the other way around?
Maybe in the long run this can be inverted in ROS 2, but I think a desire to “use ROS 2 without downloading and installing ROS 2” is a big part of the pain and friction I see in the ROS 2 ecosystem.
I don’t disagree that it represents a problem and a downside of ROS 2 as the broader Python ecosystem moves on, but I’m also not sure given the resources that it can be changed quickly without breaking existing users’ workflows.
Maybe it should be the other way around in some abstract counterfactual sense, but the real situation is hooked into a bunch of implementation decisions that need to be slowly and incrementally changed while satisfying a bunch of constraints.
So for me, I think if I want some Python package that isn’t Apt-installable and conflicts with Ubuntu 24.04 dependencies, but it is Pixi-installable with satisfiable deps and okay to run on Python 3.12 with Robostack-Jazzy, it’s a pretty good compromise that doesn’t mess with a typical rclpy user’s workflow.
I think maybe this is even more general than Zenoh snooping on a ROS 2 system using Zenoh RMW, and seems to allow bridging from ROS 2 that’s NOT running Zenoh to an external Zenoh system:
It doesn’t support ROS 2, but I was reminded of rospypi/simple after reading the posts here. I’ve used this countless times to get a simple node running on Windows fi.
Is this something like what you’d like to see @2lian?
Edit: seems like a related discussion:
(at least when pip installable packages to get just a minimal rclpy working is the intent)
note: I’ve not looked into the packages announced in that thread (also not an endorsement). It just seems like a similar approach to what rospypi/simple used.
Those eclipse-zenoh/zenoh-plugin-ros2dds are not really recommended anymore. They were used in the past before RMW Zenoh was a thing. The Zenoh team having made Cyclone DDS, they could very easily transfer payload from one to the other without (de)serializing it.
Anyway not so important, thanks a lot for looking this up. I totally see your point on simulation, motors and drivers (in the sense of software driver). ROS 2 unifying everything is it’s biggest strength. We all agree on the data format, and we are all inter-compatible. However, I don’t see why this couldn’t be compatible with python tooling. To me this unification happens on the transport+serialization layer, not when I (cannot) pip install.
@gavanderhoorn I found more promissing stuff, I’ll respond later.
For ROS2 a ROS package is actually a Ament package, and that’s what makes it hard to mix with pypi. The modern workflow with wheels doesn’t allow you to install outside of the site-packages directory which is required for the Ament index which the ROS tooling uses for finding the nodes.
I the case you don’t need nodes/actions/configuration in your package it also doesn’t have to be a “ROS” package. And you’ll have an okay experience with using venv and pip/uv.
Other ecosystems like aptrpmhomebrewnixconda/pixi etc don’t have the guardrails to deside where you package can install files into. This makes those ecosystems a much better choice for distributing ROS packages. As it can wrap any type of package.
Luckily all of those ecosystem work “kind of” well with PyPI as a second layer on their general base.
What I would like to see from the OSRF is a way to be less dependend on the structure that is currently required from the Ament Index. It seems that the index algorithm could be extended to also search in the site-packages and that would solve a large part of the problem to better allow for PyPI support.
Know that managing PyPI/apt/npm/cargo manually for 1000 packages (a normal ROS workspace) is much harder that what rosdep+colcon offers you. It’s just that when you want to go outside of the guardrails that you bump your head into the “Package Management Hell”.
We build Pixi to be able to manage these virtual environment workflows in combination with actual binary distribution, but while I would love it, also Pixi can’t be the “only” solution because it would get hate for the exact same reason. You don’t want to be locked into one ecosystem, different setups required different management tooling. This is what I believe the ROS ecosystem should move more towards. Opening up to allow for more distribution methodes. And becoming more of a pure CMake/Python/Rust package without their own wrapping format (Ament).
Thanks @ruben-arts for the details about pypi. The discussion you sent on the Pypi forum I think applies here. We should not install ROS using pip, it is kinda abusing the system and will lead pypi (and ROS) to become a mess. (although I am not gonna lie, it is cool that it is possible)
Just to not get off-track, I think no one needs to install all ROS using pip, but we should aim for a better way to use pip with ROS. If ROS is installed (and sourced), pip install gogo_keyboard should be fine, using a venv should also be fine (and maybe encouraged).
Just like how pip install pysdl2 will install the bindings to SDL2, but not install SDL2 itself. I don’t know much about pip, wheels, pybind11 and all, but something like this would be enough for me.
I clearly lack experience on the subject to dig further into what is possible and how much work it is, but I agree with your quote below.
Not maybe encouraged. Always encouraged I’m still eagerly awaiting the moment when workspaces start being venvs. I think that is the only reasonable way. With that, you would no longer need –break-system-packages with rosdep.
This can pub/sub, and ros2 node list+ros2 topic list display properly and ros2 topic echo+ros2 topic pub work just fine. Main issue is (indeed) the CDR, I used cyclonedds.idl that works very well (on python<=312). I (obviously) have to re-write all the messages myself: pyzeros2/myidl.py at main · 2lian/pyzeros2 · GitHub . That’s a lot of work, but the upside is that if you know the type of the message, you can create it by hand easily. So custom messages are possible.
Aside from this, the real issue with the CDR is the metadata. If you look at the topic name on Zenoh, I need:
the namespace + ros topic, easy
the ros message name, okay not that hard to deduce
the type hash. And this, I don’t know how to generate
Aside from this issue. Making a minimal pub/sub participant (no topic remapping, no lifecycle, no ros parameters, no launcher…) seems not that crazy. I did the current code in a day, with 150 lines (plus the CDR). Making it reliable and with different RMW is a whole different beast, but the fact that it wasn’t that hard is promising.
).