openxc / openxc-android

Android library for accessing vehicle data from an OpenXC vehicle interface

OpenXC Android Library

This library is a part of the OpenXC project.

This Android library contains the tools required to read vehicle data from the vehicle's CAN bus through the OpenXC vehicle interface in any Android application.

Visit the OpenXC project page for installation instructions, usage details, and the source code documentation.

Building from Android Studio

Make sure you have Android SDK 22 installed (the default in newer versions of Android Studio might be later, e.g. 23 - you need to install SDK 22 to build OpenXC).

Open the library or enabler projects in Android Studio - done!

Tests

To run the unit tests suite:

$ ./gradlew test

To run the instrumentation tests (must have an attached Android device or emulator):

$ ./gradlew conectedCheck

Building from Command Line

The project requires Android Studio and is configured to build with Gradle.

Releasing the App and Library

• Update CHANGELOG.mkd.
• Update enabler/src/main/play/en-US/whatsnew
• Merge into master, run fab release which will tag and push to GitHub.
• Travis CI will take care of the rest.
• Check out next, bump the version in build.grade to the next release.

The release of the Enabler to the Play store and the library to JCenter and the Maven Central Repository is managed automatically with Travis CI. The Travis build environment must be configured with these encrypted environment variables:

• KEYSTORE_PASS - password to the release keystore in this repository.
• ALIAS_NAME - alias of the release key in the keystore.
• ALIAS_PASS - password for the release key.
• SERVICE_EMAIL - Google Play service account email, for API access to the Play store.
• BUGSNAG_TOKEN - Bugsnag API key, used to upload the Proguard mapping during the build and crash reporting in the release version of the app.

If you want to deploy from a local machine, you can either define those in your environment or in the local.properties file.

Crash Reporting

The pre-built version of the Enabler available through GitHub and the Google Play store uses Android Crash Reporting by Bugsnag. We're thankful to Bugsnag for offering their service to open source projects for free.

If you wish to link your own build of the app to a different Bugsnag account, you can set your own BUGSNAG_TOKEN in the build config (see above).

Contributing

Please see our Contribution Documents.

License

Copyright (c) 2011-2017 Ford Motor Company Licensed under the BSD license.

openxc-android's Issues

Add a "split" button to the enabler to force creating a new trace file

This would be useful if you are recording separate events and you want them in different trace files. Obviously this will require some modifications to the TraceFileDataSource.

Trace file can be lost if file isn't closed properly

I'm not sure exactly what happened, but I thought I was recording data on the way to work today but the file came up empty. The 8am-9am file had data, but 9am-10am was empty.

Trace file can overwrite itself

I was recording a trace file via USB and checked that after a few minutes, the size was still increasing and was over one megabyte. 30 minute later, that same file was only 64kb and had stopped updating 15 minutes later even though I was still driving. A new trace file started as the hot clicked over to 7pm and that one looked OK.

Are we hitting some file size limit?

Add time-dependent chart to Dashboard

Allow users to select which signal they would like to see plotted.

Request a larger USB bulk transfer from Android

Bluetooth disconnects if screen orientation changes or app is switched

Start VehicleService as a foreground service to notify user of activity

Over a recent long drive, I was running the enabler with trace file recording and trace uploading enabled, streaming music via Bluetooth and running Google Nav. I noticed a couple of times that the vehicle service and/or enabler were either crashing or getting killed by Android. On the way back without nav, it worked flawlessly. My suspicion is that my phone was running out of memory at the OpenXC service was getting killed - I could see in the logs that it was shutting down properly, there was just not explanation of why.

The recommended path is to start it as a foreground service and add a notification (required for foreground services).

The primary reason for this is to be able to notify the user via a notification that the vehicle service is active and connected (or if it's not), and a side benefit is that it may result in our process not being killed in low memory situations.

Require a custom Android permission to use exported VehicleManager

Consider try-with-resources for VehicleDataSource

As soon as Java 7 language features (see http://stackoverflow.com/questions/7153989/java-7-language-features-with-android) from "Diamond" and other JSRs should be available, it would be highly beneficial, if the VehicleDataSource could extend AutoCloseable, allowing it to work in a try-with-resources manner for automatic cleanup or resources in error cases.

FAILED BINDER TRANSACTION in Android at high data rates

I've been getting this with the CAN emulator lately, which pushes out way more data than any of our vehicles.

My suspicion is that we're pushing too much data over the AIDL, too fast. Perhaps we need to batch up a few messages and send them all at once.

BluetoothSocket.isConnected not available in Android < 3.1

This is blocking the library from working on 2.3 phones, most importantly.

Re-plugging USB OTG at micro-USB end causes quick disconnect

The weirdest issue I found when using the Nexus 7 is with USB, but it's a problem we can safely ignore for now. If the CAN translator stays powered on, i.e. you have a constant mini USB connection to a laptop, and then you unplug/replug the micro-USB side of the USB OTG adapter, it'll connect to the translator for a split second and then disconnect. This happens every time. If you unplug/replug from the USB-A side of the adapter, it works fine! The connection comes back stable. I need to learn more about USB OTG and what this adapter has on the inside. My assumption is that we're not doing some proper setup on the CAN translator when it stays powered on but has a USB host reconnect. I say this issue is safe to ignore because every scenario we're talking about in the car has the CAN translator powering down whenever the car is off.

Support reading OBD-II messages from commonly available hardware

There are many existing Bluetooth or wired OBD-II scanners (with a wide range of quality and performance), and it would be useful to have an option in the library to read as much OpenXC-compatible data from one of these devices.

For example, in the same way that the UsbVehicleDataSource reads OpenXC-formatted JSON messages and deserializes them into Java objects for Android apps to use, there could be a ScantoolDataSource that reads and deserializes OBD-II messages.

The Enabler app will also need to be updated to add an option to enable/disable the scantool data source.

This issue is also related to openxc/vi-firmware#24

Unable to profile NotificationThread

It doesn't seem to show up in Traceview, and oddly there is 52% of CPU time missing from the results.

Clean up Bluetooth interface classes

• Condition variable is no longer necessary without discovery, everything is synchronous
• Remove duplicate cancelDiscovery calls if they could have side effects
• Increase retry time from 5 seconds to 10 to give stack even more time to sober up
• Delay after canceling discovery to give stack more time to recover

Enabler can hang when changing Bluetooth settings

This problem started after f11b76f

Mention in debugging docs how to enable source lookup

Add openxc to source lookup paths, or try the "search for duplicate source files on the path" checkbox.

Investigate modifying the Andriod emulator GUI

Latest Android ADT ignores jar files in libs

Our workaround to get Eclipse to play nice no longer works - I found I had to add the jar files to the build path manually. ARGH.

Samsung Bluetooth stack extremely noisy, kills performance

http://stackoverflow.com/questions/10328852/bluetooth-spp-on-galaxy-note-with-2-3-6

By changing the way we read Bluetooth we can avoid Samsung's log messages.

Deploy v4.0 to the Maven central repository

Add a script to do this to the repo, too.

Throughput seems capped at ~45KB/s

With USB and Ethernet, where I know the potential throughput is much higher, I still get this:

I/BytestreamDataSourceMixin(27032): Transferred 4194.816 KB in 92 seconds at an average of 45.59582608695652 KB/s
I/BytestreamDataSourceMixin(27032): Transferred 5243.52 KB in 111 seconds at an average of 47.23891891891892 KB/s
I/BytestreamDataSourceMixin(27032): Transferred 6292.16 KB in 133 seconds at an average of 47.30947368421052 KB/s
I/BytestreamDataSourceMixin(27032): Transferred 7340.864 KB in 154 seconds at an average of 47.66794805194805 KB/s

The message count is jumping up at almost 1200 messages per second with Ethernet, even faster than USB, but the data throughput recorded here is still about 45-47KB/s. The CPU is not pegged - in this case the combined remote process and enabler app are only using ~35%. There is either something else that is blocking, or a bug in the transfer stats logger.

Enabler crashes on 2.3 phones, tries to instantiate USB stuff

Trace file recorder leaves blank files every our even if no data is being recorded

It should be easy to just not open the file for writing if we haven't received any data.

Detect and split trace files by trips

Instead of splitting trace files every hour detect when we haven't received data in awhile and split then, because that more reliably indicates a new trip.

Update android-maven-plugin to version 3.6.0 in parent pom file

Using Android ADT v22.0.1 and Maven 3.0.4, Maven fails to resolve android sdk path unless android-maven-plugin version 3.6.0 is used.

Enabler should report exceptions

This came up during one of the past OpenXC workshops. Someone tried to play back a tracefile that was still gzipped, so the Enabler threw a bunch of exceptions in the log, but there was no feedback to the user.

The main screen of the Enabler should show some kind of information about exceptions being thrown. The UI shouldn't badger the user or continuously display scrolling text that is too fast to read. It should also report any kind of problem it has parsing messages, whether from a tracefile or a live data stream.

Create new WritableMeasurement subclass

Document trace file header format in official docs

{"metadata": {
    "version": "v3.0",
    "vehicle_interface_id": "7ABF",
    "vehicle": {
        "make": "Ford",
        "model": "Mustang",
        "trim": "V6 Premium",
        "year": 2013
    },
    "description": "highway drive to work",
    "driver_name": "TJ Giuli",
    "vehicle_id": "17N1039247929"
}

{"name":"accelerator_pedal_position","value":0,"timestamp":1364314054.403000}
{"name":"engine_speed","value":0,"timestamp":1364314054.415000}
{"name":"torque_at_transmission…….

Allow checking if a vehicle measurement is supported by the underlying interface

The parent class provides a standard interface for all measurements, starting
with a way to determine if a specific data type is supported by the vehicle
you're running on:

if(vehicleService.supports(VehicleSpeed.class) {
    // it supports it
} else {
    // if we try to get it and it doesn't support it, we throw an
    // UnsupportedMeasurementException
    vehicleService.get(VehicleSpeed.class);
    /* boom */
}

Some measurements may be supported, but may not have been received from the CAN
translator yet - it could be a malfunction, or you're just checking too early
for an infrequently updated measurement:

if(vehicleService.supports(VehicleSpeed.class) {
    VehicleSpeed measurement = vehicleService.get(VehicleSpeed.class);
    if(measurement.isNone()) {
        // do stuff with it
    }
}

Aggregate trace data into a "vehicle state" object before sending over HTTP

The trace uploader really pummels a cellular connection, and there's lots of data we probably don't need. It might make more sense to mirror how we handle data in the trace analyzer and only send a "vehicle state" object which includes the values that have changed since the last state.

Bump app versions in AndroidManifest.xml along with version in Maven pom.xml

Add optional alerts to OpenXC enabler app

Through a settings activity it should be possible to enable audio alerts when power drops, the network drops and when the USB device drops.

Add optional frequency parameter to listener registration in VehicleService

The service can optionally notify you at a frequency less than the maximum (the
maximum depends on the measurement and can vary from 1Hz to 100Hz).

// Update me at most once per second
vehicleService.addListener(VehicleSpeed.class, listener, 1);

Add header metadata to trace files

Tracefiles should have metadata that includes the version number of the tracefile being generated, and any other metadata from the vehicle we're currently connected to, such as model, vehicle interface firmware version, etc.

Latest proposed format:

{"metadata": {
    "version": "v3.0",
    "vehicle_interface_id": "7ABF",
    "vehicle": {
        "make": "Ford",
        "model": "Mustang",
        "trim": "V6 Premium",
        "year": 2013
    },
    "description": "highway drive to work",
    "driver_name": "TJ Giuli",
    "vehicle_id": "17N1039247929"
}

{"name":"accelerator_pedal_position","value":0,"timestamp":1364314054.403000}
{"name":"engine_speed","value":0,"timestamp":1364314054.415000}
{"name":"torque_at_transmission…….

• All fields are optional, just leave out if not used
• Entire header JSON object is optional - if you don't plan to use any of the fields, don't include the header object at all. Trace file parsers should check if the first record has the metadata field.

Eclipse error: projects missing required source folder: 'gen'

When first importing openxc-android into eclipse, "Project 'openxc' is missing required source folder: gen' occurs. Creating an empty folder named 'gen' in each project, project -> clean -> clean all, then Build All fixes the errors allowing the projects to build.

Bluetooth can get stuck disabled and disconnected

A report from Joe:

I’ve been testing out the Bluesmirf device on the Sony box using BlueTerm on my Nexus S.

If the connection drops after a power cycle I can get it back to a state where it no longer disconnects.

1.   Disconnect RX pin 1/TX pin 0
   

2.   Enter command mode in BlueTerm using $$$
   

3.   Type in f,1 (Fast Data Mode)
   

4.   Reconnect RX/TX
   

The data runs smoothly again.

The problem is reconnecting to the OpenXC Enabler on the tablet. I’m thinking part of the problem reconnecting is the Bluetooth implemented on the tablet.

•      It doesn’t show BluetoothInterface under Data Sources. 
  

•      I’ve tried closing the Enabler, with and without selecting BT.
  

•      Repairing to Bluesmirf doesn’t help.
  

•      Only once was I able to reconnect to the Enabler. I’m not sure why that happened.
  

I can go back to BlueTerm and see the data flowing.

Running the Enabler on my phone when the BT disconnects after power cycling the openxc box, disable the BT in the Enabler Settings

1.   Go to BlueTerm and go through steps 1-3 above.
   

2.   Disconnect BlueTerm.
   

3.   Go to Enabler and enable BT again. This still doesn’t happen 100% of the time.
   

4.   Reconnect RX/TX.
   

The data flows again.

So the problem seems to be a combination of the BT on the android device, returning to Fast Data Mode, and a problem with Bluesmirf when power cycling the open xc box.

Sinks and sources controlled by Enabler disabled on context switch

We can let the Bluetooth and Network interfaces be controlled by the VehicleInterface (i.e. not shut them down when the Enabler goes away, see #22), but that doesn't work for the sources and sinks that run in the same process as the Enabler.

With the currently implementation, if we don't shut them down you get duplicates every time you reload the Enabler.

Include measurement unit conversion in the library

Determine unit test coverage of library

http://code.google.com/p/maven-android-plugin/wiki/EmmaMaven

http://blog.rabidgremlin.com/2010/11/19/android-tips-generating-a-coverage-report-for-your-unit-tests/

AgingData#getAge() returns milliseconds but reports to return seconds

https://github.com/openxc/openxc-android/blob/master/openxc/src/com/openxc/util/AgingData.java

Similarly, getTimestamp returns milliseconds but setTimestamp's parameter is expected to be in seconds (given the multiplication by 1000 on line 58).

Moreover, since most Java applications use milliseconds as a long (given System#currentTimeMillis()), why not simply report age as standard epoch time in milliseconds (as a long) instead of double?

Disable "Playback trace file" setting if "Record Trace" has been selected as the data source.

Due to my own user-errors I have accidentally recorded a tracefile of a trip while simultaneously playing back an old tracefile. The resultant new tracefile that is recorded is garbage as it has two different "trips" stored at the same time.

I think the enabler should do one or the other if both settings are checked. My vote would be to disable "Playback trace file" if the "Record Trace" setting is enabled.

Wait n microseconds before sending over AIDL to see if we can bundle a few messages

Consider not storing values in remote service side

Why do we construct the Measurement on the VehicleService side when reading data? We want to be able to get the last known value, so we parse and store it in a map. We will be getting a new value relatively soon anyway, and this is adding lots of complication. Instead, VehicleService.get() could be killed and the VehicleService can stop caring about storing the last value of everything. It's really just for the USB source anyway. This might also solve the problem of unrecognized signals.

Bluetooth discovery doesn't properly stop, interrupting data flow

One problem: Bluetooth discovery isn't stopping properly, so after transferring data for a few seconds it gets interrupted.

Another: Occasionally we won't be able to connect and bluez reports "Already Exists"

Intermittent bluetooth connection on Toshiba Thrive tablet

The issue seems to be in the bluetooth stack on the Toshiba Thrive. Bluetooth will only remain connected for a few seconds at a time.

Process:

1. Bluetooth connects to enabler
2. Data is sent for 5 - 20 seconds
3. Bluetooth drops
4. Less than a minute later, bluetooth is reconnected and loops back up to 2.

Add optional variance parameter to VehicleService listener registration

It can also wait for a certain amount of variance in the measurement before you
are notified.

// Update me as fast as possible (the 0 argument) but only if the speed has
// changed at least 10 m/s
vehicleService.addListener(VehicleSpeed.class, listener, 0, 10);

Losing reference to an RFCOMM socket

02-27 18:36:00.998: W/bt-sdp(18490): process_service_search_attr_rsp
02-27 18:36:01.008: E/bt-rfcomm(18490): RFCOMM_CreateConnection - already opened state:2, RFC state:4, MCB state:5
02-27 18:36:01.008: E/bt-btif(18490): bta_jv_rfcomm_connect, RFCOMM_CreateConnection failed
02-27 18:36:01.008: W/bt-btif(18490): invalid rfc slot id: 11
02-27 18:36:01.008: E/DeviceManager(18181): Could connect socket to SPP service on 00:06:66:46:C2:AF
02-27 18:36:01.028: W/BluetoothVehicleInterface(18181): Unable to connect to device at address 00:06:66:46:C2:AF
02-27 18:36:01.028: W/BluetoothVehicleInterface(18181): com.openxc.interfaces.bluetooth.BluetoothException: Could connect socket to SPP service on 00:06:66:46:C2:AF
02-27 18:36:01.028: W/BluetoothVehicleInterface(18181):     at com.openxc.interfaces.bluetooth.DeviceManager.setupSocket(DeviceManager.java:108)
02-27 18:36:01.028: W/BluetoothVehicleInterface(18181):     at com.openxc.interfaces.bluetooth.DeviceManager.connect(DeviceManager.java:67)
02-27 18:36:01.028: W/BluetoothVehicleInterface(18181):     at com.openxc.interfaces.bluetooth.BluetoothVehicleInterface.waitForConnection(BluetoothVehicleInterface.java:138)
02-27 18:36:01.028: W/BluetoothVehicleInterface(18181):     at com.openxc.sources.BytestreamDataSource.run(BytestreamDataSource.java:55)
02-27 18:36:01.028: W/BluetoothVehicleInterface(18181):     at java.lang.Thread.run(Thread.java:856)
02-27 18:36:01.028: W/BluetoothVehicleInterface(18181): Caused by: java.io.IOException: read failed, socket might closed or timeout, read ret: -1
02-27 18:36:01.028: W/BluetoothVehicleInterface(18181):     at android.bluetooth.BluetoothSocket.readAll(BluetoothSocket.java:492)
02-27 18:36:01.028: W/BluetoothVehicleInterface(18181):     at android.bluetooth.BluetoothSocket.waitSocketSignal(BluetoothSocket.java:469)
02-27 18:36:01.028: W/BluetoothVehicleInterface(18181):     at android.bluetooth.BluetoothSocket.connect(BluetoothSocket.java:322)
02-27 18:36:01.028: W/BluetoothVehicleInterface(18181):     at com.openxc.interfaces.bluetooth.DeviceManager.setupSocket(DeviceManager.java:101)
02-27 18:36:01.028: W/BluetoothVehicleInterface(18181):     ... 4 more
02-27 18:36:01.028: I/BluetoothVehicleInterface(18181): Unable to connect to target device -- sleeping for awhile before trying again
02-27 18:36:01.048: D/dalvikvm(18181): GC_CONCURRENT freed 355K, 8% free 7439K/8076K, paused 6ms+2ms, total 28ms
02-27 18:36:01.048: W/bt-rfcomm(18490): Port state disc_wait_ua Event ignored 12
02-27 18:36:01.048: W/bt-btif(18490): invalid rfc slot id: 7
02-27 18:36:01.048: A/PowerManager(18181): WakeLock finalized while still held: BluetoothVehicleInterface
02-27 18:36:05.188: E/bt-btm(18490): btm_sec_disconnected - Clearing Pending flag

Two interesting things to note:

Do we lose a reference to the RFCOMM socket without actually closing it?

How/where is the WakeLock leaked?

Switch tracefile timestamps to full datetime strings with time zone

Bluetooth status not read from stored settings on first launch of Enabler

I have to click over to settings, then click back (not changing anything) get the Bluetooth interface to start up.