I have been looking into rosidl::Buffer and the current buffer backend support, particularly for large variable-length payloads such as images and point clouds.
One idea I am interested in is a CPU shared-memory backend for rosidl::Buffer.
I realize that this overlaps to some extent with functionality that already exists at lower layers. For example, some middleware / RMW implementations already provide shared-memory transport or other zero-copy optimizations. Depending on the implementation, moving data through shared memory may therefore already be possible without introducing a dedicated Buffer backend.
What I am trying to understand is whether there is still a useful role for shared memory at the rosidl::Buffer layer.
My interest is specifically in large variable-length fields where avoiding copies of the payload itself is useful. Rather than treating shared memory only as a transport optimization for a serialized message, a Buffer backend could potentially make the payload storage itself shared and let a buffer-aware RMW transport a descriptor or reference when appropriate.
This seems potentially complementary to transport-level shared memory rather than necessarily a replacement for it. On the other hand, I can also imagine that the overlap with middleware-native SHM mechanisms may be a reason why such a backend has not been pursued.
In particular, I would be interested in hearing:
Has anyone already explored a CPU shared-memory backend for rosidl::Buffer?
Is sharing the backing storage of large variable-length payloads considered an intended use of the Buffer backend abstraction?
Before prototyping something in this direction, I wanted to check whether there is already related work or design discussion that I have missed. I previously explored a similar problem for CUDA IPC in this discussion, so I would especially like to avoid independently reimplementing something that is already being worked on elsewhere.
While working on this feature, I noticed that pub/sub message delivery time increases roughly in proportion to the buffer size, even though the payload size is expected to remain constant when using specialized buffer backends.
I looked into it a bit and found that std::vector<uint8_t>(buffer_size) zero-initializes the entire buffer, effectively calling memset for buffer_size bytes. Since the buffer does not need to be initialized in this context, I submitted a patch to avoid both the extra memory allocation and the zero-initialization.
Thanks for the patch, that sounds like a good detail to clean up.
This sounds like a reasonable use case to extend this capability. It is partially redundant with the embedded shared memory transports provided by some of the middlewares, but this likely gives more control and potentially consistency but has tradeoffs such as limited QoS support.
If this makes sense for your use cases it would be great to see an implementation. I don’t think that anyone else has been working on it.
I’m experimenting the shm backended buffer, combined with rmw_fastrtpspatches. To be honest, I’ve not read the whole code yet, and experimet results are a little bit messy, so it may take more time.
My first implementation is now complete, and I believe anyone using Linux with ROS 2 Lyrical or later should be able to try it. The backend plugin is memfdaccording to memfd_create used to create sharable memory with a file descriptor.
Benchmark results
I compared the latency of several backend and transport combinations using sensor_msgs/Image.
rosidl::Buffer backend
default CPU
this memfd backend
transport setting
inter-process
intra-process
This gives four configurations:
Inter CPU — normal DDS communication
Inter SHM — the path introduced and optimized by this implementation
Intra CPU — the ordinary optimized intra-process path
Intra SHM — the ordinary optimized intra-process path plus the buffer backend overhead
Latency was measured:
from immediately before publish()
to immediately after reading the first byte of the buffer
Here is the results of that combinations with various Payload from 64 B to 16 MiB.
Here are the results for payload sizes ranging from 64 B to 16 MiB. The values are median / p99 latency in microseconds.
When comparing the Inter CPU and Inter SHM results at small payload sizes, you may notice that Inter SHM has an additional overhead of about 300 microseconds. I suspect that this overhead mainly comes from creating the buffer descriptor from the buffer.
The Intra CPU and Intra SHM results support this interpretation: there is almost no difference between them, suggesting that using the SHM-backed buffer itself introduces very little overhead.
The latency of Inter SHM starts to increase above 256 KiB, even though the descriptor size remains almost constant. This was surprising, but I found that the increase came from the rmw_fastrtps_cpp implementation, and I submitted a patch to address it:
In my environment, the crossover point between Inter CPU and Inter SHM was around 256 KiB.
After applying the patch
Here are the results after applying the patch. In this case, the latency of Inter SHM remains almost constant.
This is expected behavior: with a memfd-backed buffer, the payload itself does not need to be serialized, and the descriptor size is almost constant regardless of the payload size.
You may also notice that Inter CPU, i.e. ordinary Fast DDS message passing, shows a significant latency increase around 1 MiB.
To investigate this, I compared the latency distributions of Inter CPU and Inter SHM. One notable difference is that the Inter CPU latency has a much wider, bimodal distribution.
I have not yet investigated the exact cause in detail, but it may be related to the DDS configuration. For example, fragmentation of the DDS payload may occur for larger messages. If so, the behavior might be improved or stabilized by tuning the Fast DDS communication parameters.
In contrast, Inter SHM is much more stable. I believe this is because only a small descriptor is sent through DDS, rather than the actual payload. Since the descriptor is small enough to avoid the behavior seen with large DDS payloads, the pub/sub latency remains stable regardless of the actual payload size.
My guess is that this is a benchmark artifact rather than a real 16 MiB optimization.
Since the drop appears only with the CPU backend, I suspect some size-dependent behavior in the normal buffer allocation/lifecycle, possibly in the allocator or below rclcpp/rmw. Repeating the same simple benchmark may also make allocation and memory reuse unusually favorable. I haven’t verified which effect is dominant yet.
I noticed that rmw_zenoh also added rosidl::Buffer support to lyrical branch, and the change was released in September.
The latency trend for memfd-based inter-process communication looks almost identical to rmw_fastrtps, so I think rmw_zenohis ready to try, not only with memfd but also with the CUDAbackend.
The Lyrical release notes still say “support in Zenoh is coming”, although support has now been released for Lyrical. It might be borth updating the documentation. I’m not sure which repository the doc is, but I can submit a PR if that would be helpful.
I’ve started experimenting with a CPU shared-memory backend for Windows, and one design question came up.
My understanding is that “BufferBackend” intentionally allows backend/vendor-specific descriptor types and metadata. However, I’m less sure how platform-specific details are expected to be handled.
For example, on Linux my experimental backend currently uses “memfd” and passes enough information in the descriptor for the receiving process to obtain/map the underlying allocation. If I add Windows support, the equivalent implementation would likely use Windows file mappings (“CreateFileMapping” / “OpenFileMapping”), and part of the descriptor would naturally become platform-specific.
That raises a design question: is it considered acceptable for a buffer backend descriptor to contain platform-specific variants, for example something conceptually like:
shared_memory_descriptor
common fields
backend/platform-specific handle information
Linux: memfd-related information
Windows: named file-mapping information
Or is the intention that a backend should expose a platform-independent descriptor format, with platform-specific details kept entirely behind the backend implementation?
I can imagine vendor-specific memory mechanisms having the same issue, so I’m mainly trying to understand where the intended abstraction boundary is between “rosidl::Buffer”, the backend descriptor, and the underlying OS/vendor IPC mechanism.
The current CUDA backend also seems fairly platform-specific in practice, so perhaps platform-specific descriptors are already considered acceptable, but I couldn’t tell whether that is intentional or just a property of the current implementation.
If a backend can support multiple platforms, it seems like one benefit would be that user code would no longer need to explicitly switch between platform-specific backends.
