curiousily / tensorflow-on-android-for-human-activity-recognition-with-lstms

iPython notebook and Android app that shows how to build LSTM model in TensorFlow and deploy it on Android

TensorFlow on Android for Human Activity Recognition with LSTMs

Full explanation can be found in this blog post. The source code is compatible with TensorFlow 1.1

Hands-On Machine Learning from Scratch

Interested in deeper understanding of Machine Learning algorithms? Implement them in Python from scratch:

Read the book here

tensorflow-on-android-for-human-activity-recognition-with-lstms's Issues

Imbalance dataset

How to set class weight for imbalance dataset?

Model is installed but doesnt load when opened as an app

The frozen_har.pb created following your tutorial doesn't work. also the frozen_har.pb on your github repo aslo doens't work. I am implementing this solution on a critical project. Please help me.

It gives out this error:
Caused by: java.lang.RuntimeException: Failed to load model from 'file:///android_asset/colab_original.pb'
at org.tensorflow.contrib.android.TensorFlowInferenceInterface.(TensorFlowInferenceInterface.java:100)
at io.github.introml.activityrecognition.TensorFlowClassifier.(TensorFlowClassifier.java:23)
at io.github.introml.activityrecognition.MainActivity.onCreate(MainActivity.java:55)
at android.app.Activity.performCreate(Activity.java:7955)
at android.app.Activity.performCreate(Activity.java:7944)
at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1307)
at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3463)
at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3635) 
at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:83) 
at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135) 
at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95) 
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2175) 
at android.os.Handler.dispatchMessage(Handler.java:107) 
at android.os.Looper.loop(Looper.java:237) 
at android.app.ActivityThread.main(ActivityThread.java:7860) 
at java.lang.reflect.Method.invoke(Native Method) 
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493) 
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1075) 
Caused by: java.io.IOException: Not a valid TensorFlow Graph serialization: NodeDef mentions attr 'index_type' not in Op<name=Fill; signature=dims:int32, value:T -> output:T; attr=T:type>; NodeDef: rnn/MultiRNNCellZeroState/BasicLSTMCellZeroState_1/zeros_1 = Fill[T=DT_FLOAT, index_type=DT_INT32](rnn/MultiRNNCellZeroState/BasicLSTMCellZeroState_1/concat_1, rnn/MultiRNNCellZeroState/BasicLSTMCellZeroState_1/zeros_1/Const). (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
at org.tensorflow.contrib.android.TensorFlowInferenceInterface.loadGraph(TensorFlowInferenceInterface.java:392)

Please help me. I am using this solution for a critical project of mine.

About Smartphone Placement

The accuracy of model is pretty good, but when applying it to the Android App, it becomes much more tricky.
When walking, the App told me I'm standing. When jogging, the App told me I'm walking upstairs... Only sitting & standing always be right.
It must be related to the placement of my Smartphone. So how would you test the model through your phone?
Landscape or Portrait ？ Put it in a fanny pack or just put it in a pocket, or just hold it by hand ?

Data pre-processing in Android app?

In the Android app, shouldn't there be similar data pre-processing steps similar to the training jupyter notebook?

how to save keras model as a .pb file like tensorflow?

I am getting Resource allocation error?

I tried to reduce batch size to 8 , even then got this error.
I am using NVIDIA 1050 Ti-4GB

What was the tensorflow version?

What was the tensorflow version?

Android app crash

Hi,

I'm trying to run the Android app in an Emulator on Android Studio 3.5.3 and it keeps crashing with the following error:

--------- beginning of crash
2020-02-18 15:58:39.090 10464-10464/io.github.introml.activityrecognition E/AndroidRuntime: FATAL EXCEPTION: main
Process: io.github.introml.activityrecognition, PID: 10464
java.lang.UnsatisfiedLinkError: dalvik.system.PathClassLoader[DexPathList[[zip file "/data/app/io.github.introml.activityrecognition-86bWxYjBSIyeobB-tUsbNw==/base.apk"],nativeLibraryDirectories=[/data/app/io.github.introml.activityrecognition-86bWxYjBSIyeobB-tUsbNw==/lib/x86, /system/lib, /system/product/lib]]] couldn't find "libtensorflow_inference.so"

However from what I see the file is included in the project....
Any ideas? Thanks.

Receiving Exception when running model generated

When i am running the python file in Google Colab. and generate frozen_har.pb file
I am receiving below exception.
If you can help me in resolving the issue.

    Process: io.github.introml.activityrecognition, PID: 23391
    java.lang.RuntimeException: Unable to start activity ComponentInfo{io.github.introml.activityrecognition/io.github.introml.activityrecognition.MainActivity}: org.tensorflow.TensorFlowException: Op type not registered 'AddV2'
        at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2659)
        at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2724)
        at android.app.ActivityThread.-wrap12(ActivityThread.java)
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1473)
        at android.os.Handler.dispatchMessage(Handler.java:102)
        at android.os.Looper.loop(Looper.java:154)
        at android.app.ActivityThread.main(ActivityThread.java:6123)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:867)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:757)
     Caused by: org.tensorflow.TensorFlowException: Op type not registered 'AddV2'
        at org.tensorflow.Graph.importGraphDef(Native Method)
        at org.tensorflow.Graph.importGraphDef(Graph.java:118)
        at org.tensorflow.Graph.importGraphDef(Graph.java:102)
        at org.tensorflow.contrib.android.TensorFlowInferenceInterface.loadGraph(TensorFlowInferenceInterface.java:390)
        at org.tensorflow.contrib.android.TensorFlowInferenceInterface.<init>(TensorFlowInferenceInterface.java:96)
        at io.github.introml.activityrecognition.TensorFlowClassifier.<init>(TensorFlowClassifier.java:23)
        at io.github.introml.activityrecognition.MainActivity.onCreate(MainActivity.java:54)
        at android.app.Activity.performCreate(Activity.java:6672)
        at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1140)
        at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2612)
        at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2724) 
        at android.app.ActivityThread.-wrap12(ActivityThread.java) 
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1473) 
        at android.os.Handler.dispatchMessage(Handler.java:102) 
        at android.os.Looper.loop(Looper.java:154) 
        at android.app.ActivityThread.main(ActivityThread.java:6123) 
        at java.lang.reflect.Method.invoke(Native Method) 
        at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:867) 
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:757) ```

The recognition rate is less accurate

Can you tell me the version of TensorFlow that was used for training at the time?