sloretz / sensors_for_ros

An Android app that publishes sensor and camera data on ROS 2 topics

Sensors for ROS

Sensors for ROS is an app that publishes sensor data from an Android device onto ROS 2 topics. Currently it supports ROS Humble.

Supported sensors

• Accelerometer
• Barometer
• Camera(s)
• Gyroscope
• Illuminance
• Magnetometer

This app is built using only CMake and C++. It does not use Java or Kotlin. ROS 2 packages up to rclcpp are cross-compiled. A successful build produces an .apk file called sensors_for_ros.apk in the build directory.

Inspiration

These projects were extremely helpful, and used as a reference for this one:

• https://github.com/cnlohr/rawdrawandroid
• https://github.com/ocornut/imgui/tree/master/examples/example_android_opengl3
• https://www.sisik.eu/blog/android/ndk/camera

How to install it

Currently the only way to get Sensors for ROS is to build it from source. It is not yet available on Google's app store.

How to build it from source

You do not need ROS installed on your machine to build the Sensors for ROS app. However, it's needed to use the sensor data being published by your Android device. Follow these instructions to install ROS Humble.

Computer setup

Download the Android SDK "Command-line tools only" version. Other versions may work, but this is the minimum needed.

Make a folder for the SDK and extract the archive.

mkdir ~/android-sdk
cd ~/android-sdk
unzip ~/Downloads/commandlinetools-linux-8512546_latest.zip

Install some Android SDK components (If it gives linkage error try installing sudo apt install openjdk-17-jre-headless)

./cmdline-tools/bin/sdkmanager --sdk_root=$HOME/android-sdk "build-tools;33.0.0" "platforms;android-30" "ndk;25.1.8937393"

Install adb

# If you're using Ubuntu
sudo apt install adb android-sdk-platform-tools-common
# If you're using Fedora
sudo dnf install android-tools

Install catkin-pkg, empy, and lark

# If you're using ubuntu
sudo apt install python3-catkin-pkg-modules python3-empy python3-lark-parser
# If you're using Fedora
sudo dnf install python3-catkin_pkg python3-empy python3-lark-parser

You may need to do additional setup to use adb. Follow the Set up a device for development instructions if you're using Ubuntu, or follow the instructions in this thread if you're using Fedora.

Create debug keys

You'll need to install openjdk to get access to keytool.

sudo apt install openjdk-11-jre-headless

Create a debug keystore

mkdir ~/.android
keytool -genkey -v -keystore ~/.android/debug.keystore -alias adb_debug_key -keyalg RSA -keysize 2048 -validity 10000 -storepass android -keypass android

Clone the repo

The official repo is sloretz/sensors_for_ros.

git clone https://github.com/sloretz/sensors_for_ros.git

Next initialize the git submodules.

git submodule init
git submodule update

Download ROS dependencies

Use vcstool to download the ROS packages we need to cross compile into the deps folder.

vcs import --input ros.repos deps/

Building the App

Build the software

mkdir build
cd build
cmake ../ -DANDROID_HOME=$HOME/android-sdk/
make -j`nproc`

Installing the App on your Android Device

Install the APK in the build directory onto a device.

adb install -r sensors_for_ros.apk

Development tips

Use logcat to view the logs from the app

adb logcat

Sometimes you may want to try out a permission without writing the code to request it. The app must be installed, but not running already for this command to work.

adb shell pm grant com.github.sloretz.sensors_for_ros android.permission.CAMERA

The main activity can be started directly from the CLI

adb shell am start -n com.github.sloretz.sensors_for_ros/android.app.NativeActivity

Getting stack traces

adb logcat | $HOME/android-sdk/ndk/*/ndk-stack -sym lib/arm64-v8a/

Random lessons

During development I documented problems I encountered and fixes for them in the Problems Encountered document.

sensors_for_ros's Issues

Build issue with ROS2 dependencies

I am on UBuntu 22.04 , maybe thats the reason

-- Found Python3: /usr/bin/python3.10 (found version "3.10.12") found components: Interpreter 
CMake Error at CMakeLists.txt:10 (find_package):
  By not providing "Findament_cmake_test.cmake" in CMAKE_MODULE_PATH this
  project has asked CMake to find a package configuration file provided by
  "ament_cmake_test", but CMake did not find one.

  Could not find a package configuration file provided by "ament_cmake_test"
  with any of the following names:

    ament_cmake_testConfig.cmake
    ament_cmake_test-config.cmake

  Add the installation prefix of "ament_cmake_test" to CMAKE_PREFIX_PATH or
  set "ament_cmake_test_DIR" to a directory containing one of the above
  files.  If "ament_cmake_test" provides a separate development package or
  SDK, be sure it has been installed.

Only one image is published

Thx for this great app.

When I build and put it on my phone (connect to it via its Hotspot), i can receive the sensor data perfectly.
However there is only a single image transmitted after I activate the image stream for a camera.
If I deactivate it and re-activate it, there is another image published.

Have you experienced similar things?

Licensing info

Hello, unfortunately I haven't found any licensing info. I wanted to use this in Kabot, however I can't until licensing issue is sorted out.

Slow publish rate

Hello, I've just got the app running, and I'm looking at the accelerometer publish rate - it is slower than I'd expected.

Running usb-tethered (instead of wifi) to my linux laptop running jammy/humble, and with no cameras enabled, I get about 4.5Hz accelerometer messages. At one point is appeared to be running at 30Hz, but quickly dropped back to 4Hz. Over wifi it drops to 2Hz.

Enabling camera, I get about 5Hz images, with greater than 1s latency (as observed in rqt_image_view).

Are these rates/latencies to be expected? Do I need to configure some DDS settings on my laptop to improve the rates? Or perhaps some android settings to look at?

No 'rosidl_typesupport_c' found

After successfully building the native and cross_compiler ROS2 dependencies, I created a new project that depends on these dependencies via the following CMakeLists.txt:

set(CMAKE_SYSTEM_NAME Android)
file(GLOB android_toolchain
  LIST_DIRECTORIES false
  "${ANDROID_HOME}/ndk/*/build/cmake/android.toolchain.cmake"
  )
set(CMAKE_TOOLCHAIN_FILE ${android_toolchain})
set(ANDROID_PLATFORM 30)
set(CMAKE_ANDROID_ARCH_ABI arm64-v8a)
set(CMAKE_FIND_ROOT_PATH ${CMAKE_CURRENT_SOURCE_DIR}/sensors_for_ros/build/deps)
set(AMENT_PREFIX_PATH ${CMAKE_CURRENT_SOURCE_DIR}/sensors_for_ros/build/deps)
set(ENV{PYTHONPATH} "${CMAKE_CURRENT_SOURCE_DIR}/sensors_for_ros/build/deps/_python_")

cmake_minimum_required(VERSION 3.24)

message("FIND ROOT PATH ${CMAKE_FIND_ROOT_PATH}")
message("AMENT ROOT PATH ${AMENT_PREFIX_PATH}")
message("PYTHON $ENV{PYTHONPATH}")

add_subdirectory(sensor_network_interfaces)

And the CMakeLists.txt for the dependent project is as follows:

cmake_minimum_required(VERSION 3.8)
project(sensor_network_interfaces)

if(CMAKE_COMPILER_IS_GNUCXX OR CMAKE_CXX_COMPILER_ID MATCHES "Clang")
  add_compile_options(-Wall -Wextra -Wpedantic)
endif()

# find dependencies
find_package(ament_cmake REQUIRED)
find_package(std_msgs REQUIRED)
find_package(common_interfaces REQUIRED)
find_package(builtin_interfaces REQUIRED)
find_package(rosidl_default_generators REQUIRED)

And I receive the following result:

-- Found ament_cmake: 1.3.3 (/root/workspace/multisensor_framework/sensors_for_ros/build/deps/share/ament_cmake/cmake)
-- Found Python3: /usr/bin/python3.10 (found version "3.10.6") found components: Interpreter 
-- Found std_msgs: 4.2.3 (/root/workspace/multisensor_framework/sensors_for_ros/build/deps/share/std_msgs/cmake)
CMake Error at sensors_for_ros/build/deps/share/rosidl_typesupport_c/cmake/get_used_typesupports.cmake:35 (message):
  No 'rosidl_typesupport_c' found
Call Stack (most recent call first):
  sensors_for_ros/build/deps/share/rosidl_typesupport_c/cmake/rosidl_typesupport_c-extras.cmake:8 (get_used_typesupports)
  sensors_for_ros/build/deps/share/rosidl_typesupport_c/cmake/rosidl_typesupport_cConfig.cmake:41 (include)
  sensors_for_ros/build/deps/share/builtin_interfaces/cmake/ament_cmake_export_dependencies-extras.cmake:21 (find_package)
  sensors_for_ros/build/deps/share/builtin_interfaces/cmake/builtin_interfacesConfig.cmake:41 (include)
  sensors_for_ros/build/deps/share/std_msgs/cmake/ament_cmake_export_dependencies-extras.cmake:21 (find_package)
  sensors_for_ros/build/deps/share/std_msgs/cmake/std_msgsConfig.cmake:41 (include)
  sensor_network_interfaces/CMakeLists.txt:10 (find_package)


-- Configuring incomplete, errors occurred!

Issue with build instructions for jammy

Hello, I just tried out building and running the package. Overall it went well!

Just noting that the instruction to
sudo apt install python3-lark-parser
did not work on jammy 22.04:
E: Unable to locate package python3-lark-parser
Instead I installed the package
sudo apt install python3-lark
and everything seemed to work.