# Announcing Ros2TraceAnalyzer: Fast tool to analyze and visualize timing of ROS 2 applications

**URL:** <https://discourse.openrobotics.org/t/announcing-ros2traceanalyzer-fast-tool-to-analyze-and-visualize-timing-of-ros-2-applications/41751>\
**Category:** Projects\
**Tags:** ros2, tracing, ros2\_tracing\
**Created:** [January 27, 2025, 2:27pm UTC](https://discourse.openrobotics.org/t/announcing-ros2traceanalyzer-fast-tool-to-analyze-and-visualize-timing-of-ros-2-applications/41751 "2025-01-27T14:27:44Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![wentasah](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/wentasah/32/12802_2.png) [@wentasah](https://discourse.openrobotics.org/u/wentasah)\
**Post date:** [January 27, 2025, 2:27pm UTC](https://discourse.openrobotics.org/t/announcing-ros2traceanalyzer-fast-tool-to-analyze-and-visualize-timing-of-ros-2-applications/41751/1 "2025-01-27T14:27:44Z")

</div>

Hello all,

we would like to announce the [Ros2TraceAnalyzer](https://github.com/skoudmar/Ros2TraceAnalyzer) tool. Its aim is to analyze traces obtained with `ros2_tracing` in a similar way as the `tracetools_analysis` package, but faster and with additional analyzes.

The tool is written in Rust and according to our benchmarks, it processes the traces 30× faster than `tracetools_analysis`. On our trace of a medium-sized project with 1.8 milion events and 78 MB, Ros2TraceAnalyzer needed 4.2 s to process and analyze the trace, while `tracetools_analysis` needed 125 s for the same trace. Ros2TraceAnalyzer is also able to analyze traces, where `tracetools_analysis` doesn’t work well, because it merges callbacks from different processes with the same address to the same graph.

Ros2traceAnalyzer also allows presenting the results in the form of interactive dependency graph with timing quantiles as shown below.

 ![alks-dep-graph](https://us1.discourse-cdn.com/flex022/uploads/ros/original/3X/a/a/aa32f6e3b1b877454e1def1a7de9b0f94ad0454a.png)

Ros2TraceAnalyzer is able to analyze C++ applications for ROS 2 Jazzy. For Python applications, only message latencies can be analyzed, because `rclpy` doesn’t include tracepoints for callbacks. We have also developed tracing support for Rust applications written with the [R2R](https://github.com/sequenceplanner/r2r/) library. This is not yet included in the upstream repository, but it is available in the [lttng-usr branch of our fork](https://github.com/skoudmar/r2r/tree/lttng-ust).

We consider the release to be of alpha quality and we’re interested in the feedback from the community.

Cc: @christophebedard @Ingo_Lutkebohle

---

<div class="post-metadata">

**Author:** ![JM\_ROS](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/jm_ros/32/12964_2.png) [@JM\_ROS](https://discourse.openrobotics.org/u/JM_ROS)\
**Post date:** [January 27, 2025, 3:46pm UTC](https://discourse.openrobotics.org/t/announcing-ros2traceanalyzer-fast-tool-to-analyze-and-visualize-timing-of-ros-2-applications/41751/2 "2025-01-27T15:46:32Z")

</div>

How does this package compare to the tracecompass plugin ?  
Does it provide any additional features compared to it ?

---

<div class="post-metadata">

**Author:** ![skoudmar](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/skoudmar/32/24148_2.png) [@skoudmar](https://discourse.openrobotics.org/u/skoudmar)\
**Post date:** [January 27, 2025, 5:00pm UTC](https://discourse.openrobotics.org/t/announcing-ros2traceanalyzer-fast-tool-to-analyze-and-visualize-timing-of-ros-2-applications/41751/3 "2025-01-27T17:00:15Z")

</div>

From what I understand about the trace compass plugin, it does not allow data aggregation. In trace compass, the edges represent individual messages (one edge = one message), but you cannot view a summary over the same repeating edge. We merge the edge to calculate a quantile/percentile value (one edge = multiple messages of the same type), as shown in the picture above.

---

<div class="post-metadata">

**Author:** ![JM\_ROS](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/jm_ros/32/12964_2.png) [@JM\_ROS](https://discourse.openrobotics.org/u/JM_ROS)\
**Post date:** [January 27, 2025, 5:12pm UTC](https://discourse.openrobotics.org/t/announcing-ros2traceanalyzer-fast-tool-to-analyze-and-visualize-timing-of-ros-2-applications/41751/4 "2025-01-27T17:12:55Z")

</div>

What would be the benefit of a summary ? In all my use cases of the tracing, I am interested on the exact circumstances of a certain sequence of events, that lead to my timing ‘outliers’. If you summarize, you can only say, there was a outlier, but you can’t reconstruct the events causing it…  
Or is the use case of this tool more in the area of finding out, that you have outliers in you timing in the first place ?

---

<div class="post-metadata">

**Author:** ![skoudmar](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/skoudmar/32/24148_2.png) [@skoudmar](https://discourse.openrobotics.org/u/skoudmar)\
**Post date:** [January 27, 2025, 5:38pm UTC](https://discourse.openrobotics.org/t/announcing-ros2traceanalyzer-fast-tool-to-analyze-and-visualize-timing-of-ros-2-applications/41751/5 "2025-01-27T17:38:42Z")

</div>

This tool is meant to be used to quickly verify that the system is behaving as expected, e.g., the communication actually happens, and the callbacks execute in the expected time.

1. Yes, this tool can be useful for finding IF there is an outlier in a large trace. You can then use trace compass to look it up and analyze the sequence.

2. It can also be used for measuring performance.

---

<div class="post-metadata">

**Author:** ![Guela\_kais](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/guela_kais/32/19525_2.png) [@Guela\_kais](https://discourse.openrobotics.org/u/Guela_kais)\
**Post date:** [January 28, 2025, 9:31am UTC](https://discourse.openrobotics.org/t/announcing-ros2traceanalyzer-fast-tool-to-analyze-and-visualize-timing-of-ros-2-applications/41751/6 "2025-01-28T09:31:35Z")

</div>

I’ve looked at the source code and I don’t understand how your tool connects to ros2 at all. It doesn’t seem to be based on [Ros2 Rust](https://github.com/ros2-rust/ros2_rust) or [r2r](https://github.com/sequenceplanner/r2r). Is it just a frontent for [Ros2trace](https://index.ros.org/p/ros2trace/)?

---

<div class="post-metadata">

**Author:** ![wentasah](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/wentasah/32/12802_2.png) [@wentasah](https://discourse.openrobotics.org/u/wentasah)\
**Post date:** [January 28, 2025, 12:15pm UTC](https://discourse.openrobotics.org/t/announcing-ros2traceanalyzer-fast-tool-to-analyze-and-visualize-timing-of-ros-2-applications/41751/7 "2025-01-28T12:15:16Z")

</div>

Yes, it could be considered a frontend for ros2trace. The tool processes trace events generated by tracepoints in `rclcpp`, `rcl` and `rmw`. This [r2r fork](https://github.com/skoudmar/r2r/tree/lttng-ust) (for now) extends r2r with rclcpp-compatible tracepoints so that you can trace and analyze r2r applications the same way as C++ ROS applications.

---

<div class="post-metadata">

**Author:** ![christophebedard](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/christophebedard/32/1514_2.png) [@christophebedard](https://discourse.openrobotics.org/u/christophebedard)\
**Post date:** [January 29, 2025, 12:40am UTC](https://discourse.openrobotics.org/t/announcing-ros2traceanalyzer-fast-tool-to-analyze-and-visualize-timing-of-ros-2-applications/41751/8 "2025-01-29T00:40:27Z")

</div>

If I understand the tool correctly, I think it would be more appropriate to describe it as a consumer (i.e., “analyzer”) of the trace data obtained using a combination of (1) the built-in ROS 2 tracing instrumentation and (2) custom tracing instrumentation in r2r that mimicks the `rclcpp` instrumentation.

`ros2trace` is just the package that provides the `ros2 trace` command, which allows you to enable tracing and collect trace data from a running application. As far as I understand, you just use `ros2 trace` as-is when using Ros2TraceAnalyzer, correct?

---

<div class="post-metadata">

**Author:** ![christophebedard](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/christophebedard/32/1514_2.png) [@christophebedard](https://discourse.openrobotics.org/u/christophebedard)\
**Post date:** [January 29, 2025, 12:40am UTC](https://discourse.openrobotics.org/t/announcing-ros2traceanalyzer-fast-tool-to-analyze-and-visualize-timing-of-ros-2-applications/41751/9 "2025-01-29T00:40:57Z")

</div>

@wentasah @skoudmar is the tracing instrumentation in r2r _exactly_ the same as the `rclcpp` instrumentation? Did you have to modify or adapt anything because of the Rust language? If so, it would be interesting to know. @wentasah mentioned at ROSCon that there was an issue with pointer addresses not matching up because of Rust.

---

<div class="post-metadata">

**Author:** ![skoudmar](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/skoudmar/32/24148_2.png) [@skoudmar](https://discourse.openrobotics.org/u/skoudmar)\
**Post date:** [January 29, 2025, 6:42pm UTC](https://discourse.openrobotics.org/t/announcing-ros2traceanalyzer-fast-tool-to-analyze-and-visualize-timing-of-ros-2-applications/41751/10 "2025-01-29T18:42:17Z")

</div>

> [@christophebedard](#):
>
> I think it would be more appropriate to describe it as a consumer (i.e., “analyzer”) of the trace data obtained using a combination of (1) the built-in ROS 2 tracing instrumentation and (2) custom tracing instrumentation in r2r that mimicks the `rclcpp` instrumentation.

Sorry for the confusion. Yes, this tool does not capture a trace but analyzes it.

Before capturing the trace with `ros2 trace` or with LTTng directly:

For C++, nothing additional is needed. The built-in ROS 2 tracing instrumentation is used. So, it should work out of the box.

For Rust, our fork of r2r is needed, and subscriber and timer callbacks must be passed to new functions `Node::subscribe_trace` or `Timer::on_tick` to register them.

> [@christophebedard](#):
>
> As far as I understand, you just use `ros2 trace` as-is when using Ros2TraceAnalyzer, correct?

Yes, you run `ros2 trace` to capture the trace and then `Ros2TraceAnalyzer` to analyze the trace.

> [@christophebedard](#):
>
> is the tracing instrumentation in r2r _exactly_ the same as the `rclcpp` instrumentation? Did you have to modify or adapt anything because of the Rust language? If so, it would be interesting to know.

No, but we tried to make it as close as possible so it could be opened in Trace Compass or `tracetools_analysis`.

There are some differences:

1. We could not use executor tracepoints because instead of an executor, r2r async functions spawned on user-chosen runtime such as tokio.rs.

2. The r2r’s spin function does not handle callback execution and only notifies the async runtime. The runtime then handles the callbacks. Therefore, we have added new tracepoints to the spin function to allow for the measurement of latency between the notification and execution.

3. We created the functions `Node::subscribe_trace` or `Timer::on_tick` to register callbacks and wrap them in instrumentation because callbacks are not normally passed to r2r.

4. Specifically for callbacks, we use sequential IDs instead of function pointers to allow function reusing.

> [@christophebedard](#):
>
> @wentasah mentioned at ROSCon that there was an issue with pointer addresses not matching up because of Rust.

We had problems with IDs because r2r initialized rcl objects on the stack, and after initialization, they were moved to stable heap allocation; therefore, the address did not match. In our fork of r2r we initialize them already in the heap.