I don’t think it’s the style of the ROS docs but I feel like there should be more warning sidebars and stuff to catch the eye to help provide this historical context. I encountered this fact years ago and it’s really clarified why the Python launch is the way it is and why, despite ROS 1 using XML, ROS 2 packages all favor Python.
It doesn’t feel to me like there is any “official” or “beginner” way to encounter the fact that the Python is the backend and people just ran with it.
Looks like the XML and YAML frontends have been available for about six years and I think I’ve just recently encountered one significant package that’s actually using XML launch.
I am hoping to transition completely to XML soon. The declarative Python is just too verbose and hard to read. I wrote one of the most deranged lines of code I’ve ever written this year trying to use the PythonExpressionand I overuse OpaqueFunction for better clarity and readability even though I appreciate the intent behind a declarative configuration.
@gbiggs Sorry, but saying that declarative launch files are easier to statically analyze is simply not true. You can just as well traverse the AST and check for specific calls. In fact, I have already done this to build a graph of better-launch-launchfiles here. It’s not polished, but it gets the job done easily in less than 400 lines of code. And as many others here, I seriously question the need for this, especially when it’s the frontier implementation.
@jak the above link might be interesting to you as well! There is also now LaunchMap which is a visual editor for regular ROS2 launch files, but I’m not sure how complete it is.
Overall I don’t see any merit to having a declarative approach, especially when the implementation is as awkward as python launch. I also think it’s a weak excuse to now declare it as “only a backend”, as python launch files have been prominently featured as the default on every ROS2 documentation since its introduction. It wasn’t meant to just be a backend.
Lastly, as it was mentioned a few times by e.g. @christophebedard, the point of better_launch is not to be able to write nodes that require strict orchestration, and I don’t think anyone intends to do that. It’s a side effect of allowing to interact with nodes after starting them, and I think that’s a good thing.
Cool! So if we need to know for the safety-critical systems the launched system state without actually launching it, and better_launch can do it with AST, it looks to me that it could eventually replace the existing launch design!
I would really love to get better launch battle tested, and somehow integrate it with ROS 2 to improve the existing launch experience. Having it as a separate package is amazing for proving that it works, but I think for wider adoption it is necessary to integrate it directly to ROS 2. @ndahn Any plans towards this direction?
@danzimmerman, if we can have Python launch files that are easy to read, and more flexible than XML, is there any reason to transition back to using XML files?
if we can have Python launch files that are easy to read, and more flexible than XML, is there any reason to transition back to using XML files?
I think so.
I am strongly in favor of a declarative paradigm for system bringup in shared collaborative team settings. I think it’s useful to restrict the space of possibilities for what the developers are allowed to do outside their node code.
Arbitrary imperative Python code can be full of hidden state and introduce side-effects that aren’t even hinted at by the arguments and return values of functions. Easy for this to devolve into debugging chaos by invisibly tinkering with the same thing in different ways. This is rarely a problem in single-developer projects because one person’s mental model can handle it, but the bigger the team gets and the more you mix in less-experienced team members, the worse this gets.
A declarative approach ideally specifies the final state of the brought-up system, allowing the launch system to do the work of achieving that final state and to catch conflicts. Strong restrictions on the features of the launch system can help sub-teams reason about how their nodes should interact with the rest of the system to bring themselves up properly.
And in that vein, I want to focus specifically on this:
more flexible than XML
I kind of think flexibility should be an anti-goal for typical usage of launch or a replacement bringup system. I think heavy restrictions on allowable logic from a limited launch API are often good. I think side-effects especially are hell for teams reasoning about the bringup system.
And when you do need flexibility, I think the flexibility people are usually looking for is to launch 50 instances of the same node for a swarm or something like that.
This kind of thing is easily handled by launch.actions.OpaqueFunction
OpaqueFunction allows you to write arbitrary imperative Python code with whatever library imports you want. I’ve used it a lot for launch file flexibility. But I really do think that flexibility can be more of a detriment than a help for team projects.
That said, turning the focus to this:
launch files that are easy to read
I agree that ROS 2 Python launch files can easily become unreadable.
Honestly, I think declarative systems of all types suffer badly from verbose and sometimes convoluted syntax.
But I do think ROS 2 Python launch is especially bad for readability, especially compared to Pythonic Python. I think this could be improved a lot without breaking the declarative paradigm. But I also think maybe we’re just trying to shove Python in where it doesn’t belong. Even an ergonomic side-effect free pure-functional system implemented in Python could still be verbose and annoying to most Python programmers.
I don’t love XML for humans. It has excessive visual noise, and closing tags more than a few lines away gives poor developer experience when handwriting it without a good IDE, but I don’t think I’ve ever really struggled that much with this in ROS 1 XML launch files.
I find it more of a pain in URDF/.xacro files but not so bad in launch. Also, I think the ROS 1 to ROS 2 transition, especially in academic circles, is still lagging, and starting to see ROS 2 drivers that have XML launch files will probably help this.
In short, YAML would be better but XML is fine and familiar.
Overall, I think it’s a tough transition for a lot of programmers to deal with a lazy-evaluated declarative system. I know it has been for me.
I have absolutely felt the pain of thinking that ROS 2 Python launch files would be like writing ordinary Python and was disappointed when it wasn’t like that. It’s awful to replace a simple list or dict comprehension or f-string substitution with 50 lines of unreadable nonsense. It’s unfortunate that it’s so hard to discover how to mix in ordinary Python. Not at all good user experience.
However, I think a lot of the ROS 2 design intent is very good and well-thought-out in many areas, and rarely think it should be abandoned, even when the implementation is lacking. I think avoiding a full multi-paradigm programming language in launch descriptions except inside something that “looks weird” like launch.actions.OpaqueFunction is actually a pretty good design!
In addition to unreadable Python for launch, think debugging launch files is currently awful. Finding where an error happened is pretty bad. However, I also feel that something like 75%-80% of the time, that error is because I still have a really poor mental model for Python launch and the interrelationship of all the classes, what the arguments of those classes are allowed to be.
As I mentioned before, trying to use launch.substitutions.PythonExpression was especially deranged when attempted directly from Python.
If I accept the Python classes as an intermediate parser representation and my team leaves it behind for XML, I expect all these errors will go away provided the parser has been implemented correctly. Same with some the headaches @ndahn calls out re: types in launch files. This has indeed been a big issue, but if the Python expression parser is correctly implemented, I expect that it’s fine in XML.
I’ve kind of come to believe that Python launch files are simply setting people up to be disappointed and frustrated.
I also think XML will lead to a more uniform formatting and an easier implementation of a “house style” for launch files. And if there are missing features that are not yet implemented in the XML and YAML frontends, which I think can be an issue, I think the solution is to implement them.
I agree that it’s a big problem that ROS 2 Python is the default in the documentation. I think there should have been a documentation push six years ago when the XML frontend was introduced to flip the switch immediately and forcefully away from Python as default.
But the docs issue aside, it’s been at least three or four years since I first encountered the statement that the Python is a backend intermediate representation. I can find old comments of mine from 2022 pointing at others’ discussions of this. Python launch files got adopted, and I’m pretty sure I could find older. The XML badly lagged the ROS 2 release, but people wanted to use ROS 2 so they used Python. I don’t think anyone is “now declaring” this.
ROS 2 documentation lags are their own issue, IMO.
Re: launch possibly we need a Python frontend or alternative Python implementation? But as I mention above, I personally like the declarative paradigm and I don’t see much use for imperative Python in launch that can’t already be handled by launch.actions.OpaqueFunction, and a little bit of boilerplate. As I see it that’s basically a documentation issue too.
I think you did nice work on your package, I think a lot of people will find it useful. I think it’s nice to have auto-declaration of arguments, provided they have good, descriptive names, and I like that the –help can apparently be populated from the function arguments and docstrings. I like the idea of the TUI. I will probably try it out at some point.
I think considering a full replacement for launch is totally reasonable, even one that doesn’t try to be declarative.
That said I don’t love the README rhetoric that ROS 2 launch should be scrapped and abandoned, that launch is “unusable,” and so on, and these days these kinds of statements strongly turn me off to third-party projects.
I also think as the years have gone on and I’ve spent time in the GitHub issues and documentation and Discourse discussions about core ROS for more context, I feel more and more that we need more contributors working to make the core ROS 2 more ergonomic and performant and especially to learn enough about the existing system to improve not only the implementation but properly fill in the documentation.
Good job with better_launch, I went through a similar project a while ago and digging into the internals of the launch systems really is a pain.
Re: launch possibly we need a Python frontend or alternative Python implementation? But as I mention above, I personally like the declarative paradigm and I don’t see much use for imperative Python in launch that can’t already be handled by launch.actions.OpaqueFunction, and a little bit of boilerplate. As I see it that’s basically a documentation issue too.
This is part of what I tried to do with simple_launch, that is a wrapper around Python’s launch to make it (much) less verbose and possibly more declarative by hiding the boilerplate around OpaqueFunction (e.g. no .perform(context) ) everywhere:
from simple_launch import SimpleLauncher
sl = SimpleLauncher()
sl.declare_arg('use_robot', True)
def launch_setup():
if sl.arg('use_robot'):
do stuff
else:
do other stuff
return sl.launch_description()
generate_launch_description = sl.launch_description(opaque_function = launch_setup)
This is partly because back in 2020 XML or YAML were not suited for many cases, and teaching the backend Python syntax was not a good intro to launch files except if you wanted to discourage the audience.
As of today I focus the intro on XML syntax and just remind people that many tutorials or example will probably be in Python because of historical reasons.
First off: great work! A good abstraction from the launch python API was missing and the better_launch interfaces are much cleaner in my opinion, even if I remain a huge fan of declarative notation for the same reasons as previous posters.
From an interface perspective I would have preferred a non-comparative name like duck “launch_it” or the “launch_this” name you also use. “better_launch” for me seems to imply that it is not standard enough, which is important for example to decide whether to teach it in basic classes.
You can get very close with William’s ROSCon talk 2018 stating that xml might be the default thing people should use in the future (when it didn’t exist yet). There were several offline comments along the same lines during the remainder of the event.
Oh, I can’t tell you how awful it is to watch video of myself But it’s a good reference.
I can definitely tell all of you that we (myself and the rest of the ROS 2 team at the time) never intended the Python API that is launch and launch_ros to be the primary way to write launch files. It was meant to be the “backend” API.
However, it was also meant to be a public and well documented API too (this is in part because the equivalent Python API for ROS 1’s roslaunch was not very nice to use directly and we wanted to improve on that).
As others have already said, we ran out of time to finish the XML and YAML frontends and we needed something immediately, so we wrote user documentation for it as a stopgap. I personally regret that this is how it worked out, but we were being as pragmatic as we could given the constraints. We’ve since tried to claw that back and promote the XML and YAML frontends, but that’s obviously very difficult.
If you believe that the declarative approach doesn’t have value that’s ok. I encourage you to show us all how to do it better, which I think is what you’re trying to do with better_launch and I think that’s great! Of course, we always want people to feel like extending the existing projects is the best path to take, but sometimes that doesn’t make sense and you need to try something totally different. That’s fine and it’s how we get cool new things.
However, I believe it’s worth considering that the decision to have a declarative interface wasn’t something we came to arbitrarily. We made it that way based on our experience using and maintaining ROS 1’s roslaunch, as well as working with our customers (which at the time included a lot of companies wanting more out of their configuration management, which includes launch files). I can tell you, not being able to robustly describe what they wanted to do, without doing it, would be a red flag for them. So that definitely colored our decisions. And after so much time, I do not regret this design choice myself.
Similarly, I think there’s a reason that so many projects use markup languages (declarative) for configuration rather than scripts (possibly imperative), preferring a script (or template engine) to output the configuration (which itself is declarative) if dynamic behavior is needed. Just consider large examples like the relationship between HTML, CSS, and scripting like javascript, or something like Dockerfiles.
launch is kind of in a gray area here though, as launch files go beyond configuration in many cases, including behavior (e.g. reacting to lifecycle state or processes exiting) and also being extensible. Now days I think about the relationship between Dockerfile’s (which are mostly static-declarative) and more exotic equivalents enabled by LLB like envd, and I see that as a similar design space as to where launch files live. Part static configuration, part need to make that configuration parametric, part needing to be dynamic and reactive. It’s a complicated problem to solve generally.
I understand that folks want to just dive right in and do things like if arg.sim do X else do Y, but I really do think this kind of thing doesn’t scale very well. Or at least you have to be very careful in how you use it. Pushing folks towards a declarative style of launch file is absolutely an opinionated thing, and if anyone thinks that’s just not a good idea, then that’s absolutely ok. You don’t have to use it, but we had to have something as the default, and we are constantly trying to balance having an opinion that we think is good with allowing people the flexibility to do things the way they want.
A lot of the other things mentioned here and in the README for better_launch are what I would consider more “quality” and “style” complaints with launch (e.g. using asyncio, being awkward to use etc.), which is at times related to the high level concept of declarative vs imperative, but I won’t try to argue those individually (feel free to ask my opinion on specific items). I’ll just say that it might be helpful to keep in mind that launch is around 10 years old at this point, and I (along with others) was developing it at the same time as Python’s asyncio (it was actively changing as I was working on launch), so it may be that there are better ways to do things now. Also, keep in mind that we worked on launch after writing or working on catkin_make, catkin_make_isolated, catkin-tools, ament-tools, vcstools, and colcon, all of which were tools that have to run many subprocesses and handle their outputs and state, which makes launch very similar to them. So while you may not agree with the implementation’s choices (which again is totally fine), keep in mind that we did arrive there after much work and experience in that space.
I also want to highlight this. I 100% agree the Python API in launch is awful to use and we need something else to satisfy folks that want to use imperative-style Python launch files. Whether its making something like launch_frontend_py that appears to be imperative, but is just a front for the declarative backend, or something that’s actually (as far as I can tell) imperative like better_launch doesn’t matter much to me. But I do think integrating well with what’s already out there (for better or worse) is a must. And it seems you guys have already been looking to that.
Agreed. This lib seems even more readable than what I currently use. I have my own base node classes with ANSI styled output support and with custom log styles. I think PAL Robotics had something similar to this better_launch but I didn’t want to use a custom launch lib or OpaqueFunctions etc. everywhere.
ROS 1 (or ROS 2 xml) launch files were fairly horizontal, with ROS 2 vertical Python launch files, I was bored of doing scroll scroll scroll, change something, then scroll scroll scroll… it becomes a pain in a launch file with 3 includes, 5+ nodes. This is what I do now:
from launch import LaunchDescription
from launch.actions import (
DeclareLaunchArgument as ARG,
IncludeLaunchDescription as INCLUDE)
from launch.substitutions import (
LaunchConfiguration as VAR,
PathJoinSubstitution as PATHJ,
PythonExpression as EVAL)
from launch_ros.actions import Node
from launch_ros.substitutions import FindPackageShare as FPKG
DeclareLaunchArgument is already exposed as @expose_action('arg'). Even aliases like ARG = DeclareLaunchArgument in the header files would do. The maintainers have their reasons but I also don’t have the energy to structure this, create a PR, please the maintainers, and wait for a year for it to be backported to LTS versions. If they see it here and like it, they can implement it.
Putting the imports in a header in the your_pkg and shortly just importing import your_pkg.short_launch, makes it even shorter. short_launch.py could be a header in the official launch package. Unused imports might not be ideal but, it would be a good thing for newcomers. (Nobody is forcing nobody to use it but if it was in the tutorials, it would spread like a wildfire)
And then, my args are in the beginning of the launch file. Mostly horizontal lines with rather radical bracket styles, showing similarity to the XML launch files.
I also don’t like IBM punched card style 80 character limit, I use 120 or no limit for some lines, but even with 80, this shorter versions are easier to read/modify.
Not all launch files need IfElseSubstitution, LoadComposableNodes, etc. but I personally prefer, IF, LOADCN instead of explaining everything with the names and repeating all those names multiple times on multiple lines.
Also not using the short_launch.py header and using Python’s import aliases explains what those uppercase things are in-that-file already
IMO, a big part of the problem is hidden here, reacting to lifecycle state and processes exiting belong to different abstraction levels: the former requires insight into the process internal logic. Addressing both simultaneously is like implementing a networking stack without OSI layers – the result is overcomplicated and hardly satisfies anyone.
Totally agree, it might be convenient at a project scope, but it facilitates spaghetti system design which is harmful to the ecosystem.
Personally I am quite frustrated by XML and would like to see it phased out across the board in favor of YAML/JSON.
For now my focus is on fixing any issues users discover and getting a package into rosdistro that can be installed via apt and co. As you can see there is still an ongoing discussion on where the ROS2 launch system should go in the future. In this regard, better_launch is just my strongly worded opinion
Of course, if somehow the ROS2 team decides to integrate an imperative approach based on better launch I have no issues renaming it to something more neutral.
There is a lot to digest in this post, so I will focus on what I consider the key points. Sorry if I skip anything that was important to you!
I just as strongly disagree here. This is the C++ mindset where everything is const, private, non-virtual (and annoying). Personally I don’t think it’s my place (or responsibility) to tell others how to write launch files. A hammer doesn’t need to be covered in spikes so people don’t hold it upside down.
The same goes for hard-to-debug issues. I cannot prevent others from wearing thick gloves and holding the hammer upside down anyways, nor do I want to. That’s what QC and code reviews are for. I would also argue that even now launch files are too expressive to prevent such issues, even if you ignore OpaqueFunction/OpaqueCoroutine.
That being said, I agree that the ROS 1/2 launch system served its purpose well and absolutely has its place.
I don’t think people just flocked to the python “backend” just because it was presented as the default. You already pointed out that the XML and YAML frontends lagged behind, but in many cases they were also just not powerful enough. There’s also the clunkiness of if/unless, which the python backend unfortunately didn’t resolve… Other libraries like @OlivierKermorgant 's simple_launch are proof of this.
Fair enough, and while I still stand by these statements, I also admit that they came from a place of frustration. But you’ve also seen the state of ROS2`s launch implementation - it’s a fever dream. And I honestly think that it’s better to scrap it and start over than trying to salvage it.
Thank you for the insight on this! I fully accept that it’s relevant in some industries, and I don’t hate the idea. It’s just not the implementation I would have hoped for If time permits I might try to implement a declarative layer on top of better_launch to see how it feels. I’m also not sure if python/xml/yaml are the right choice for this, so I might explore some other options.
The gap better_launch is trying to fill is focused on usability, but it also comes with some benefits around predictability and maintainability.
I’d argue this is a matter of perspective. In better_launch you just tell the node the lifecycle state you want it to reach if it turns out to be a lifecycle node. There’s barely any overhead and the functionality is neatly contained. Again, the current launch files are not immune to bad coding habits, on top of being hard to write, hard to read, and hard to maintain.
Mostly avoided joining this conversation because it’s hard to keep up though I was mentioned and now that I’m trying to invite some more folks to join me in the launch improvement work, it seems worth at least chiming in a little.
It feels really unfortunate to me that the instinct is not “we can fix what we’ve got to be easier to use” - but instead “gotta toss it all out and start over”. But I don’t blame you, there’s a lot of momentum to fight against and a blank slate is a much easier starting point.
The reasoning guiding where I put my effort right now is: the community is on average going to default to the core tooling, and rather than fragmenting. I’d rather shape what we have into what we want it to look like, even if it’s kind of slow and painful. There’s a lot of good stuff and hard iterative learnings in there that never shake out in the first pass of a new tool. That isn’t to say there isn’t room for multiple tools for similar jobs - a large enough (and that’s some indicator of health) ecosystem has overlaps.
I want to emphasize that this entity doesn’t exist in the way that you imply with statements like this. You are the ROS 2 team, if you choose to be. Launch doesn’t have any particular active team except those who volunteer to improve it.
But, without a forum for those discussions to iterate more quickly, it can be hard to build context & consensus or identify the difference between hard design constraints vs achievable usability improvements. I invite you to become that team and help lead the usability discussion ROSGraph Working Group kickoff - if you feel so inclined.
But it’s a good reference.
though I was mentioned and now that I’m trying to invite some more folks to join me in the