It is a completely Automated Eye Testing Machine. It is a game. It uses Voice Recognition AI! It is fun, interactive and addicting to play. All of it runs on the microcontroller Raspberry pi 4B!
This is my most innovative and fun work. A project that encompasses all of my work and showcases my coding skills!
Here is a link to the wonderfull ceremony where we secured Second Position and sadly my last day at university! https://www.linkedin.com/posts/muhammad-hammad-87787421a_four-years-and-they-went-like-that-mashallah-activity-7070119934269210624-aq94?utm_source=share&utm_medium=member_desktop
*Here is the link for the Voice Recognition model trained by us for this project! https://github.com/MuhammadHammad-git/Voice-Recognition-Model-for-FYP-Trained-in-CNN-
The Game includes a wooden board (Created frame by frame in Paint Every animation is made like this) placed in a forest environment that pops up and digs down in the ground! The letters appear on the board and their appearance, background and font are specific to the requirement for an eye test! The Letters displayed are Random and Are displayed randomly anywhere on the ground of the forest! There is grass animation as well to show multithreading and sequenced Animation.
Audio of the user is taken by either a headphone or microphone. The audio first passes through the sliding window algorithm that inputs the audio to the model in 1 sec samples. if the first window of audio does not produce the required result then the audio window is moved ahead 0.5 seconds and another audio sample is passed! Until all the requirement are met only then the model output is used! All this is to ensure that the patient has a seemless experience, i.e allowing him/her to say too early or too late or even repeat a letter without causing any issues.
At the end of the eye test, keeping in view the number of mistakes made by the patient in guessing letters, a result is generated that consists of a layman term category, visual acuity and approximate range of diopter of lens. The layman category is defined by taking inspiration from WHO, Visual acuity is guessed on the bases of a defined formula while Diopter lens value is guessed by a data set. The data set consists of 400 eye tests each taken by ourself by visiting General Hospital Lahore's Ophthalmology ward. Based on this data set we guess the approximate lens power the user can have! Because this is soo subjective it was unable to be guessed with a lower range!
Game is Optimized for Raspberry pi 4B. Can be run on any display with a resolution of 800x600 upto 4k. Any screen resolution below losses quality as game elements are compressed too much to fit into the screen.
Pygames library on Raspberry pi struggles with scaling issues and has bugs. I have handled the resolution in such a way as to reduce such bugs as much as possible. These bugs only happen randomly on some runs.
Some bugs include:
- unable to scale properly
- unable to sometimes remove default black background on transparent png's.
Game.Demonstration.mp4
https://drive.google.com/file/d/1WJ-i7jxUeieNWjcf0R-orHxnQ-HDCrY3/view?usp=drive_link Here Letters that blink green are recognized while those in red are missed by the model! Here, we can clearly observe that even in such a noisy environment our Model is working with an accuracy of more than 95%! This was a huge accompolishment for us as all of the data was collected by us through Audio recording! This audio data set is not available for free anywhere on the web! There are 5 Letters that appear on the screen T O Z F L. We collected 160 to 170 audio samples for each letter. The model we used here is trained using CNN while Google Teachable machines model was used as a prototype and benchmark to assess our model
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent