AI-powered voice assistant to assist the visually impaired in their daily tasks
AI Powered Vioice Assistant for Who are Blind or Have Low-Vision
be my eyes
power to the 253 million people who are blind or have low vision globally. Image to text technology
The tool can offer suggestions for recipes based on the contents of a user's refrigerator, for example, and provide a step-by-step guide on how to make them.
Quick Read
Scan documents
Currency scanner & counter
Barcode scanning
Features: Detecting 80 different categorical object types with their confidence level Priority-based detection Distance prediction & scaling algorithm Mobile phone activated camera for various items Speech-activated mode selection Multi-threading usage for asynchronous voice warnings and feedbacks
Modes: Aware mode: continuously speak out loud 3 items with the topmost priority Warn mode: only shout out warnings if any item is too close based on priority Search mode: focus on searching a particular item until it is found
Features
- Detecting 80 different categorical object types with their confidence level
- Priority-based detection
- Distance prediction & scaling algorithm
- Mobile phone activated camera for various items
- Speech-activated mode selection
- Multi-threading usage for asynchronous voice warnings and feedbacks
- SSD Mobilenet COCO API Model
Modes
- Aware mode: continuously speak out loud 3 items with the topmost priority
- Warn mode: only shout out warnings if any item is too close based on priority
- Search mode: focus on searching a particular item until it is found
Resources Links
Please click the link below to view our other resources regarding to this project submission.
How To Run
-
Clone this repository and make sure you have all the files downloaded
-
Set your environment and install the required python packages
It is recommended for you to use Anaconda, and specifically
python version 3.9.0
(newer version of python has some compatibility issues related to thecollections
package, other older python version is untested and might have some compatibility issues as well). Then, install all the required python packages by typing:pip install -r requirements.txt
-
Execute
main.py
ormain.ipynb
fileIf you want to use an external camera, you can connect via IP Webcam Pro and edit the
url
variable when running the model. By default, it is connected to your PC / Laptop default camera. You can also toggle theUSE_SPEECH
global variable toTrue
orFalse
. If it isTrue
, it will use speech recognition to choose and change the mode, or else it will manually prompt you for the mode input.