zahir2000 / hand-gesture-mobile-app Goto Github PK

View Code? Open in Web Editor NEW

License: MIT License

Dart 39.25% Makefile 46.15% Java 0.74% Kotlin 0.03% Shell 0.29% Swift 0.41% Objective-C 0.68% CMake 5.04% C++ 5.76% C 0.38% HTML 0.68% JavaScript 0.05% CSS 0.26% Batchfile 0.26%

hand-gesture-mobile-app's Introduction

Hand Gesture-based Food Kiosk Mobile Application

Prerequisites

Android Studio (for Android) (2021.2.1 Patch 1 & 2021.3.1 Patch 1)
Android SDK (API 28 or above) - use SDK Platforms in Android Studio
Android SDK Command-line Tools - use SDK Tools in Android Studio
Flutter SDK (3.3.8) - make sure no issues found on flutter doctor
~~Xcode (for iOS)~~ untested

Instruction for Android

Clone the project in Android Studio
Start an Emulator (SDK 30 and above preferred)
Navigate to pubspec.yaml and run Pub get command
Wait until the dependancies are installed
Navigate to External Libraries/Flutter Plugins/tflite_flutter_helper-0.3.1/android/src.main/kotlin.com.tfliteflutter.tflite_flutter_helper/TfliteFlutterHelperPlugin.kt or copy paste the modified file contents from here - skip to Step 7 if you copy-pasted
Go to line 143 and remove all the question marks ? from the onRequestPermissionsResult function parameters
Allow to modify the project files if it asks
Run the project and accept permissions it asks

Additional Information

If you want to enable your webcam for the emulator: Android: How to use webcam in emulator?
In some cases, the camera preview will show as rotated. Try to move your hand around if that's the case because I did not perform image augmentation to consider cases from different angles
To try out different models available, navigate to the project folder: assets/tflite/ and rename any model to model.tflite to replace the current model. Default is EfficientNet-Lite0
The gesture detection rate is set to every 3 seconds; to note: only the Lite[0-1] models infer in less than 3 seconds. You may change the timer from DETECTION_CHECK_TIMER inside main.dart