futhrevo / speechtrials

Simple experiments on android speech api

SpeechTrials

Simple experiments on android speech api

speechtrials's Issues

User should be able to login to app using either Google | Facebook authentication

Let us use Firebase for Login and Authentication, and User Profiles.

https://firebase.google.com/docs/auth/android/google-signin.

App should announce to the rider if the desired cooling is not met

The app should monitor the temperature profile (reading the temperature sensor data from BLE profiles) periodically and announce to the user if the desired cooling or desired temperature setting cannot be achieved based on the rate of temperature change. For example, if the desired temperature is 20 degrees and the minimum temperature measured in the helmet gets static at 27 degrees, then the App should announce that "we cannot cool below 27 degrees. We are trying our best to maintain your desired comfort level".

Home Screen should be designed like a Bike Dashboard

The Home Screen should be designed similar to a Bike Dashboard and display stats/indicators like:

1. Avg. Speed (should be updated every two seconds)
2. Distance traveled (should be updated every ten seconds)
3. Overspeed (Speed > 80 KM/HR) indicator
4. Time/Distance to Destination (if destination is set, in Google Navigation for example) (should be updated every 20 seconds)
5. Outside Temperature/Humidity Level (fetch via some weather API)
6. Inside Temperature/Humidity/Fan Level (High|Mid|Low) (fetched from FW over BLE)
7. Battery Level Indicator (fetched from FW over BLE)
8. Water Level Indicator (fetched from FW over BLE)
9. Bluetooth Connection Status

It should also have BUTTON's for quick and easy access to:

1. Tap to Speak (LARGE BUTTON positioned conveniently so user can easily tap with thumb for example)

Contextual Buttons/Actions:

1. When Bluetooth Connection is disconnected and couldn't be automatically restored within a given time interval (say 30 seconds) it should automatically display a Connect Button so the user can tap to connect.
2. When an incoming Call is detected a Large Button (similar to Drive Mode) to accept or reject the call.
3. When an incoming Message is detected a Large Button (similar to Drive Mode) to Listen to the message or Ignore it.

On App Startup ensure Bluetooth is On and provision to make Phone is Discoverable

When the App starts up do the following checks:

1. Turn Bluetooth ON (if it is OFF).
2. Make Device Discoverable (if it is NOT DISCOVERABLE).
3. Check if it is paired with a BluArmor (TBD)
4. If it is not paired with a BluArmor, and if ONLY ONE BluArmor is found, then pair with the BluArmor. (TBD)
5. If more than One BluArmor found, then show the list of BluArmor on screen, and prompt user for pairing with one of the devices listed. (TBD)

TBD: We need to discuss on the best strategy to implement 3, 4 and 5. For now listing them as a placeholder. Let me know your thoughts on how we should do this.

App should set the Fan Speed into a BLE profile based on certain conditions

1. App should monitor the external temperature and set the Fan speed to a setting that will result in the desired level of cooling.

2. App should monitor the rider motion state (still, moving slow, moving fast) to set the fan speed. For example Fan speed should be set to the Highest level when the rider is still for more than one minute or moving at slow speed (say < 5 km/hr) and set to Lowest level when the rider is moving fast (say > 40 km/hr) for more than two minutes.

The Speed Thresholds above should be configurable in app settings.

Auto Answer Call from Favorite Contacts in Drive Mode

When in Drive Mode the App should auto answer any incoming phone call from favorite contacts after announcing the caller name. If not in Drive mode this feature should be de-activated.

Navigate to Location sent in SMS in Drive Mode

While the App is in Drive Mode and "Read Message" Option is turned on, parse the message to check if it contains any location name / a google map link? If a location is found, ask user "Received Location from . Do you want to navigate there?"

If the user answers in the affirmative using an utterance like "Yes|Sure|Certainly|Please do|Of Course|Yes thanks|Yes Please|..." then say "Ok. Routing you to [destination name]".
Then automatically open google maps for that locations and start navigation with turn by turn directions.

If the user answers in the negative then ignore the message.

If the user doesn't say anything or the app doesn't get what the user said please ask the user again like "Say Go or Go Now to navigate to [destination name]"

If the user says "Navigate" then Open Google Maps and start navigation to the destination.

If the user says anything other than "GO or GO NOW" then quit the voice dialog.

Favorite Contacts and Preferences

User should be able to create a list of Favorite Contacts in their preferences.

User should be able to choose an option to auto answer incoming call from their Favorite Contacts.

User should be able to choose an option to auto-reject any other incoming calls.

User should be able to send a text message when auto-rejecting an incoming call. The text message should be configurable with the default set to "Sorry am unable to talk right now. Will call you later."

Incoming SMS Action set to No Action should Not Read SMS

When Incoming SMS Action is set to NO Action it should not Read SMS. Looks like this is not implemented currently. Please CHECK @futhrevo

App should track Water and Battery Level and announce when the levels are low

For Battery Level Indicator:

App should read the battery level from a BLE profile and alert the rider if the level is lower than a threshold value.

For Water Level Indicator:

App should derive a proxy for water level based on temperature gradient wrt time (dT/dt). The threshold for flagging empty tank can be configurable in app preferences with following settings (1C/min, 2C/min, 3C/min and 5C/min). The App should compute this gradient every minute and the moment it crosses the configured threshold value announce that "the water may be too low. Please refill water for continued cooling".

Let us use Firebase Remote Config https://firebase.google.com/docs/remote-config/use-config-android to store such config values. It will make it easy for us to publish changes to these values without a release cycle for the App.

Invoke the App and turn Drive Mode "ON" on various triggers

The app should open (come to Foreground) automatically based on the following triggers:

1. When the Device connects with a BT hands-free headset.

2. When the device detects motion via the built-in accelerometer.

3. When the user starts navigation on Google Map.

etc.

Conversational Dialog for Listening and Replying to Text Messages in Drive Mode

We need to enable a small conversational dialog for Listening and Replying to Text Messages.

1. When a new SMS is received the App should say "New Message from . To listen to this message say Listen." and prompt for a Voice Input

2. If the user says "Listen|Please|Read|Yes|Go Ahead|Listen Now" etc. then the App should Read the Message body, and on finishing ask "If you would like to Reply to this message say Reply"

3. If the user says Reply the App should prompt for a Voice Input and wait for the user to dictate the message.

4. On completion, the App should repeat the message that the user has input and ask "Do you want to Send it or Change it?"

5. If the user says "Send it" send the message

6. If the user says "Change it" go to step 3 above.

7. At each step when the user gets a voice prompt and doesn't respond within 15 - 20 seconds the App should prompt the user "Could you please repeat what you just said". If the user doesn't respond or the App doesn't speak any voice input or the app doesn't understand what the user said it should abort with a voice message like "Sorry I'm having trouble listening to what you are saying. Please try again in some time".

NOTE: This feature should be activated in Drive Mode only.

List of Incoming Call Actions under Settings Prefercence

One small change request to make the language more explicit, can we rename the options under "Incoming call action" to the following:

1. Reject All Calls
2. Answer Calls from Favorites only
3. Answer All Calls

Replace ". . ." with some Media sound notification like Google Speech prompt before and after speech

Currently ". . ." is being used to pad a message, but it reads as "dot dot dot".

It should be replaced with some media sound like a Notification Sound or something like that. The media can be played before and after the message.

Track Ride Stats

Collect the following stats for every ride:

0. Start and End Coordinates/Location Names
1. Rider Activity Detection using Google Play Services (Still, Walking, Driving, Tilting, ...)
2. Distance Travelled (in Kms)
3. Ride Time (in hours/mins)
4. Average Speed (in Km/hr)
5. No of times went over speed limit (80 km/hr)
6. Ride Miles Earned (For now set it to 10 times distance traveled in Km)
7. Mean External Temperature (pulled from some weather API)
8. Mean External Humidity (pulled from some weather API)
9. Mean altitude

Cache Last 10 Ride stats on the device. All ride stats should be archived to the cloud (Google Cloud Storage)

Using "Ok Google" to invoke our App should invoke the Voice Recognition Prompt and process any voice actions

User should be able to invoke our App using "Ok Google Open Rider". When the App opens it should present a voice prompt just like the Google Assistant.

Track My Ride

There should be a preference setting to set:

Track My Ride (On/Off)'

When Track My Ride is On The app should track the location coordinates (every 4 seconds). The app should buffer these location coordinates for a set time interval like one minute and sync to the cloud (use firebase or google cloud storage)

Galaxy S7 with Android N

On the Galaxy S7 , speaking a command worked most of the time. Sometimes the app failed to catch a name. The app also sometimes failed to respond when the microphone icon was clicked.

Auto Reject Calls from any non Favorite contacts in Drive Mode

When in Drive Mode the App should auto-reject any incoming phone calls from any contacts other than the Favorite contacts marked in App preferences.

Optionally, depending on the user's preferred settings, the app should send an automated text response like "Sorry am unable to receive your call right now. Will call you back later." This message should be customizable in App preference settings.

When not in Drive mode this feature should be deactivated.

Integrate Firebase Analytics to track various app Events

List of Events to track:

1. App Open/Close.
2. Bluetooth Connected / Disconnected.
3. Temperature Not Met.
4. Water Level Empty Indicator.
5. Low Battery Warning Indicator.
6. High-Speed Driving Indicator.
7. Other critical Exception / Error scenarios.

Have a notification or button to disable the App

When the App is activated a notification should pop up on the notifications panel. When the user clicks on the notification App should be stopped / de-activated.

Monitor and Re-establish BT/BLE Connection when in Drive Mode

While the App is in Drive Mode a background service should continuously monitor the state of the Bluetooth Classic and LE connections every 5 s / 10 s (should be configurable).

If any of these connections gets dropped the app should try to re-establish the connection promptly in the background.

When the connection is dropped the app should speak "Disconnected."
When the connection is back (re-established) the app should speak "Connected."