Digital retailers can easily capture data to quantify and analyze how users are interacting with their products online. How can traditional brick-and-mortar retailers capture similar analytics in order to remain competitive in the ever-changing retail field? With new technologies, traditional retailers can now gather and analyze information about how potential customers are interacting with their storefronts.
DisplaySense utilizes an ensemble of off-the-shelf hardware, open vision, facial expression sentiment analysis & deep learning techniques to quantify the effectiveness of a store front. Our solution is:
- Easily scalable to multiple stores, displays, aisles & security cameras
- Sensitive to customer’s privacy
- Is easily understandable via a centralized dashboard
- Can ensemble with other data sources
Leverage existing and new hardware at storefronts to record video to be analyzed
Extract demographic & sentiment features from video for analysis using AWS Rekognition
Display aggregated impressions & data of storefront in a dashboard
This can be a live stream from a camera or video recorded previously.
The video is pushed to an S3 bucket in order to easily interact with other AWS services in the next steps.
The stored video is then analyzed to detect persons, faces, gender, age, sentiments, facial features such as beard, mustache, eyeglasses etc.
Amazon Rekognition Video publishes the completion status of the video analysis to an Amazon SNS topic. If the video analysis is successful, you can get the results of the video analysis.
First-in, first-out Amazon SQS queue to get the completion status of the video analysis request. The response sent back by the SQS queue is a json object.
Displays aggregate results to show person and face traffic, gender and age composition, general mood from the analyzed video data(json object sent back by SQS queue).
Anaconda is the recommended solution for installing and managing the packages necessary for DisplaySense analytics. Please see the Anaconda installation instructions.
The requirements.txt
file lists all Python libraries necessary for utilizing DisplaySense, and they will be installed using the following commands:
pip install -r requirements.txt
conda install --file conda_requirements.txt
AWS Identity and Access Management (IAM) must be configured to manage access to AWS services and resources securely. Please see Amazon's getting set up documentation.
Once AWS access has been configured, you're ready to create a topic using Amazon SNS. A topic is a communication channel to send messages and subscribe to notifications. It provides an access point for publishers and subscribers to communicate with each other. Please see the SNS documentation for configuring an SNS Topic.
Finally, configure an Amazon SQS queue. Amazon SQS moves data between distributed application components and helps you decouple these components. Please see the SQS documentation for configuring a queue.
-
After forking and cloning this repository navigate to the repository and run the command
brew services start mongodb
. -
Run the command
python rekognition/video_analytics.py
-
Run
python app.py
-
Open your browser window and go to the url https://0.0.0.0:8080/
-
Enjoy the dashboard! When done, go back to the command line interface and press
ctrl + c
to quit the app.
- Investigate dlib C++ library for advanced, efficient, and scalable face detection
- Engage visual merchandiser & designers for a greater understanding of relevant storefront metrics and KPIs
- Replace Flask backend with Dash for more-responsive dynamic plots
- Integrate system with Raspberry Pi 3 or AWS DeepLens
- Incorporate AWS Kinesis for easy handling of streaming video
- Analyze customer sentiment towards specific projects by utilizing head pitch and yaw metrics for gaze tracking
- Incorporate predictive analytics
- Incorporate additional metrics on dashboard
https://docs.aws.amazon.com/rekognition/#lang/en_us
https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_tutorials.html
https://people.eecs.berkeley.edu/~jordan/sail/readings/rubin.pdf