SensorStream iOS App - Use your iPhone as a sensor platform

Want to stream your iPhone’s sensors to ROS 2? (Yes, you can even get point clouds!)

I made an app to do exactly this, available for download on the App Store!

SensorStream is an iPhone app + ROS2 node combo that streams live sensor data directly into your ROS2 environment over your local network or over USB.

It can stream the following sensors:

  • :camera: Rear/Front Camera - 1080p @ 30FPS
  • :straight_ruler: Depth Map - 320x180 @ 30FPS
  • ⠿⠿ Point Cloud - 57k points (rear) @ 30FPS , 200k+ points (front) @ 10FPS (over USB) with RGB colour values.
  • :compass: IMU - Accelerometer, Gyro, Orientation up to 100Hz
  • :satellite: GPS - Outdoor location tracking max @ 1 Hz

Currently tested / working with ROS 2 Humble and an iPhone 15 Pro.

Nobody has used transmission over USB for this specific use, making everything a lot quicker than alternatives.

Try it out and give some feedback! It’s a work in progress, let me know if there’s anything you want to see added or improved.

Check it out at the site: SensorStream site
Or read my blog (for the motivation behind the project): PaaS (Phone as a Sensor)

See the video below for a demonstration of the app and its features:

3 Likes

Hello, sorry for double posting, I saw this on linkedin, commented, but then thought maybe it’s better to keep discussion on discourse for searchability. Copy/paste of my comment.

Hi Martijn, this looks well put together and performant!

In case you’re interested, I was looking at this for Android in 2023. I found there were so many options for app development language and associated approaches to bridge to ROS.

I put together this overview sheet capturing the APIs and projects I came across

https://docs.google.com/spreadsheets/d/e/2PACX-1vQcdG5vW0oc3FjvMc3O5zqkuw4AiB6kwhM8xk5Lyz2KkTps0jod5-mzYV5kUldK3VMu9qOBqSSMOcly/pubhtml

lIn the end I tried a web approach and was able extract live pose from my Quest 2 headset to ROS2 via webxr, but didn’t take it very far

https://github.com/tim-fan/webxr_ros_pub

Some associated ROS discourse**https://discourse.openrobotics.org/t/project-proposal-mobile-app-for-publishing-sensor-data-ar-pose-and-depth/32497**

I do think there’s a big opportunity to better serve hobby+educational robotics with these phone->ros bridges, I’m glad to see your work in this area!