ivan-ngchakming / facial-recognition-database-management-system

Facial Recognition Database Management System (FRDMS) is a facial recognition system made for everyone.

Facial Recognition Database Management System

Note: This project is current being developed.

Introduction

Facial Recognition Database Management System (FRDMS) is a facial recognition system made for everyone.

Powered by python and react, and packaged into a single executable that can be used by anyone with zero dependencies required.

Usage

Download the latest release from the right side bar.

Windows

• Download FRDMS_WIN.exe.

• download Visual C++ Redistributable for Visual Studio 2019 for FRDMS to work properly.

• Double click on FRDMS_WIN.exe to start application.

MacOS

• Download FRDMS_MACOS

• Open terminal

• Browse to Download directory

  cd Download

• Change permission

  chmod 777 ./FRDMS_MACOS

• Start the application

  ./FRDMS_MACOS

Starter Database

Download FRDMS.db and place it in the same directory as FRDMS.exe to use a sample starter database. Or start FRDMS.exe directly to create a blank database.

Pages

Facial Recognition

Images

Profiles

Upload Image

Features

Facial Recognition (In-progress)

Find a face match by supplying an input image, and the system will search the database to locate possible matches

1. face identification (Done)

   Classifies a face to a specific identity selected from existing profile in the database

2. face verification (In-progress)

   Determine whether a pair of faces belongs to the same identity so un-identified faces can be grouped together

3. Batch processing (In-progress)

   To perform the 2 task above in batch.

Web Crawling (To-do)

Launch web crawling workers to craw through social platforms for images and profiles

Personnel Classification (To-do)

Face profiles can be classified into multiple groups using supervised or unsupervised machine learning models

Automatic Profile Creation (To-do)

Supply large amount of unlabelled images into the system to be classified by person. New unnamed profile can be created automatically when input photo fail to match any existing profiles Data crawled from social medias will be used to populate newly created profile as much as possible.

Live Video Recognition (To-do)

Identify known faces in the database from a live video

Contributing

For guidance on setting up a development environment and how to make a contribution to FRDMS, see the contributing guidelines.

Special Thanks

Special thanks to Kabiirk for making the project logo.

facial-recognition-database-management-system's Issues

Choose thumbnail for profile

Allow user to select which image to be used as the profile thumbnail.

• Create API endpoint to execute mutation on modifying or adding thumbnail to a portfolio
• Add frontend UI to allow user to select an image within a profile as the profile's thumbnail
• Send API call to the server upon user action

refactor: adopt Ariadne graphql apps folder structure

Refactor tasks to use graphql
Implement a scalable structure for flask back-end using Ariadne

Implement filters for portfolios

Implement the filtering function on the data table on /profiles.

• Modify back-end API to accept parameters for filtering (eg. by face count, name, and id)
• Modify front-end data table for user to select which filter to apply.

refactor: adopt components and page components structure

there are a lot of components in src/components that are very page specific, for example the CreatePortfolio components.

All components that are only use by a single page, or have low re-usability should be moved to their respective src/pages/[page name]/components/ directory.

MacOS Support

Add MacOS support by packaging FRDMS application into a single file using either pyinstaller or other libraries, so that FRDMS can run on MacOS standalone without the need to install node.js and python

Create profile editing page and form

Edit the ProfileDetails.jsx page to include a editing view so user can edit profile's name, and other details such as age, country, date of birth, etc...

Profiles Management

Allow user to edit and delete all profile data using the web interface.

• Delete single profile
• Delete multiple profiles
• Create profile editing API
• Create profile editing page and form #23
• Choose thumbnail for profile #21
• Merge profiles

refactor: migrate from javascript to typescript

• Setup project to use typescript #58

Migrate js and jsx files to ts and tsx files, open separate PRs to deal with one or a small number of files.

• ./src\App.jsx
• ./src\index.js
• ./src\style.js
• ./src\components\SelectToolbar.jsx
• ./src\components\context\MenuContext.jsx
• ./src\components\DataTable\DataTable.jsx
• ./src\components\DataTable\EnhancedTableHead.jsx
• ./src\components\images\CroppedImage.jsx
• ./src\components\images\Gallery.jsx
• ./src\components\images\Image.jsx
• ./src\components\nav\NavBar.jsx
• ./src\components\progress\LinearBarsProgress.jsx
• ./src\graphql\graphql.js
• ./src\graphql\index.js
• ./src\graphql\mutation.js
• ./src\graphql\query.js
• ./src\hooks\useImages.jsx
• ./src\hooks\useProfiles.jsx
• ./src\hooks\useTasks.jsx
• ./src\pages\Home.jsx
• ./src\pages\Images.jsx
• ./src\pages\ProfileDetails.jsx
• ./src\pages\FacialRecognition\index.jsx
• ./src\pages\FacialRecognition\components\faces
• ./src\pages\FacialRecognition\components\profiles
• ./src\pages\FacialRecognition\components\ImageAnalytics.jsx
• ./src\pages\FacialRecognition\components\ImageAnnotator.jsx
• ./src\pages\FacialRecognition\components\UploadImage.jsx
• ./src\pages\FacialRecognition\components\faces\FaceCard.jsx
• ./src\pages\FacialRecognition\components\faces\FaceCards.jsx
• ./src\pages\FacialRecognition\components\profiles\CreatePortfolio.jsx
• ./src\pages\FacialRecognition\components\profiles\DetailedProfileCard.jsx
• ./src\pages\FacialRecognition\components\profiles\ProfileCard.jsx
• ./src\pages\FacialRecognition\components\profiles\ProfileCards.jsx
• ./src\pages\profiles\Profiles.jsx
• ./src\pages\tasks\Create.jsx
• ./src\pages\tasks\Tasks.jsx
• ./src\utils\data.js
• ./src\utils\general.js
• ./src\utils\images.js
• ./src\utils\index.js

bug: batch recognition task not working when there is no profiles

Start the app with an empty database, and create a new recognition task at /tasks/create
No images or profiles will be created. (For input supply a directory with one or more images)

My guess is the scores array is empty as there is no profiles, therefore encounter some errors here
https://github.com/ivan0313/Facial-Recognition-Database-Management-System/blob/4585be9a53c6d6f1b0875d42d0f6a06e80abf1e5/server/tasks/tasks.py#L51

If verified that this is the case, simply skip to run the else block when there is no profiles found should fix the issue.
https://github.com/ivan0313/Facial-Recognition-Database-Management-System/blob/4585be9a53c6d6f1b0875d42d0f6a06e80abf1e5/server/tasks/tasks.py#L57-L64

refactor: adopt apollo useQuery and caching

Whenever Apollo Client fetches query results from your server, it automatically caches those results locally. This makes subsequent executions of the same query extremely fast.
See: https://www.apollographql.com/docs/react/data/queries/

• Setup the Apollo client and provider

Adopt useQuery to:

• useProfiles hook
• useImages hook
• useTasks hook

bug: Error: Maximum update depth exceeded. In SelectToolbar

Describe the bug
On /tasks page, Error: Maximum update depth exceeded. will be thrown. This is due to toolBarButtons being re-rendered in the select SelectToolbar.

Not sure how to fix yet.

build: migrate to MUI v5

Create app icon for FRDMS

Create an icon for the application

Mobile view is not responsive

There is only facial recognition i can not see facial recognition and the mobile website looks like this i know how to make it responsive should i work on that too

Originally posted by @demonicirfan in #14 (comment)

feat: consistent gallery and data table style

Update the gallery component to have a consistent styling as data table, especially since #39

This way the switch from gallery view to and from table view would be less jumpy. Both views would use the current style of the table view.

Another advantage is that both views can share the same tool bar, as well as the pagination component.

Configure prettier pre-commit hook

configure pre-commit hook to run prettier so that coding style can be consistence

ref: https://prettier.io/docs/en/precommit.html

Create workflow to automate publishing releases

Automate build react app, build pyinstaller app and release publishing on github action

How to use onnxruntime with flask multi-threads

Help!!! this isn't bug, it's a quesion.

Created a server that can run a session with multi-threads using Flask.

When run 3 threads that the GPU's memory less than 8G, the program can run. But when run 4 threads that the GPU's memory will be greater than 8G, the program have error: onnxruntime::CudaCall CUBLAS failure 3: CUBLAS_STATUS_ALLOC_FAILED.

I know that the problem is leaky of GPU's memory. But I hope that the program don't run crash. So I try to limit the number of threads, and set intra_op_num_threads = 2 or inter_op_num_threads = 2 or os.environ["OMP_NUM_THREADS"] = "2", but don't work.
Try to 'gpu_mem_limit', don't work either

import onnxruntime as rt
from flask import Flask, request
app = Flask(__name__)

sess = rt.InferenceSession(model_XXX, providers=['CUDAExecutionProvider'])

@app.route('/algorithm', methods=['POST'])
def parser():
    prediction = sess.run(...)

if __name__ == '__main__':
    app.run(host='127.0.0.1', port='12345', threaded=True)

My understanding is that the Flask HTTP server maybe use different sess for each call.
How can make each call use the same session of onnxruntime?

System information

• OS Platform and Distribution: Windows10
• ONNX Runtime version: 1.8
• Python version: python 3.7
• GPU model and memory: RTX3070 - 8G

Face Verification

Add new feature for when faces are being compared against know faces in the database, it will also be compared against unknown faces. Multiple unknown faces will be combined into a new profile in batch mode, where user can browse the list of unidentified profiles, edit or delete these profiles.

feat: adopt test frameworks and add tests

frontend

• use jest unit and integration testing
• use cypress for end-to-end testing

backend

• use unittest to create unit tests

Implement filters for Image gallery

Filter images by one or multiple profiles

refactor: consistent naming

Right now, the naming of Profile and Image are very inconsistent throughout the application, both front-end and back-end.
For example,
Profile vs Portfolio
Image vs Photo.

Let's renaming all occurrences, including UI text, variables, filenames, etc..., to use Profile and Image.

We'll split this issue to be completed in 2 separate PRs

• Front-end refactoring
• Back-end and Database refactoring

Front-end work should be relatively easy and suitable for new-comers, but back-end refactoring might be a bit tricky as it will involve modifying the database tables (or how not to modify the database tables)

Gallery Pagination

Add pagination feature to the Gallery component.

• create pagination UI #46
• modify API to allow rows per page parameter
• create front-end logic for pagination to function

refactor: data table as standalone reusable component

data table used in portfolio page is currently very hard to reuse, see example usage

• refactor all table related components from Profiles to src/components/tables
• moved columns definition to component props

Create Context Menu For Gallery

Create context menu for Gallery component to allow user to selected more actions such as delete, view image, view profile, edit profile, view image details, etc...

Gallery Image Change Size Dynamically

Allow user to view gallery images in difference sizes.

For example, a [lg, md, sm] switch, or a size slider.

Make sure the checkbox for image selecting is resized correctly after image size is changed and the checkbox color will detect its background average brightness and its color dynamically to ensure visibility of the checkbox.

Create Gallery view for Profiles

Create a Gallery view (the Gallery component used in the Image page) for profiles and allow user to toggle between table view and gallery view

Add Nested List to NavBar

Created nested list in NavBar to group similar pages, for example Facial Recognition and Batch Facial Recognition should be grouped together.

reference: https://material-ui.com/components/lists/#nested-list

feat: adopt data table to view all batch recognition tasks page

after refactoring data table #34, adopt the new data table component to batch recognition page

Create tasks object to perform face detection and recognition

Create task manager and worker to execute task in daemon thread

bug: gallery change page scroll to top

Currently, clicking next or previous page on image gallery will automatically scroll to top.
https://github.com/ivan0313/Facial-Recognition-Database-Management-System/blob/6cc0a7528cbe6ecfc6f6f9a4f34900a71dbcfb35/src/components/images/Gallery.jsx#L108

This is not an issue on the /images page, but it becomes an issue when on the profile detail page profile?id=1.

Modify the Gallery component so that when user click on next or prev page, the window will scroll to the top of the Gallery component, rather than the top of the page.

Images Mamangement

Allow user to edit and delete all photo data using the web interface.

• Delete single photo #29
• Delete multiple photos #10
• Rerun face detection and recognition on image

Batch Face Recognition

Add feature to allow user to upload multiple images and automatically start detection and recognition task for each uploaded image.

• Create Page to view all tasks #60
• Create page to add new task #33
• Create APIs to read, write and update tasks #36
• #19
• #24
• Store snapshot of task status so task information won't be lost on restarting the app