Amazon Kinesis Video Streams serverless surveillance platform

You can find the related blogpost to this repository here: Design considerations for cost-effective video surveillance platforms with AWS IoT for Smart Homes

Deploying the infrastructure requires you to have sufficient privileges to do so. Please setup the AWS CLI beforehand. You can find a detailed explanation to every building block in the Solution Walkthrough.

This example is for experimental purposes only and is not production ready. The deployment of this sample can incur costs. Please ensure to remove infrastructure via the provided scripts when not needed anymore.

Supported Regions

Please note that this example only works in the Amazon Kinesis Video Streams supported Regions which can be found here https://docs.aws.amazon.com/general/latest/gr/akv.html

What the installation scripts do
Quick Start
How to clean up
Web Client Walkthrough
Solution Walkthrough

What the installation scripts do

Infrastructure

The script will deploy the required infrastructure, permissions and roles:

Creates a S3 Bucket for the AWS Lambda functions.
Builds the AWS Lambda functions, zips and uploads them to S3 Bucket.
Creates Thing Type, Thing Group, Fleet Provisioning Template and Claim Certificates in IoT Core.
Creates DynamoDB as "Manufacturing Database" that contains a device. (Credentials can be found in Web Application)
Deploys Amazon Cognito with a test user and password. (Credentials can be found in Web Application)
Amazon API Gateway with corresponding paths to forward requests to the AWS Lambda functions
Creates configuration for the web application
Creates configuration for the camera mock / raspberry

Camera Mock

The script will deploy the required infrastructure, permissions and roles:

Creates local files to automatically start the fleet provisioning process and the streaming to Amazon Kinesis Video Stream via WebRTC.
Zips the local created files
Creates a S3 Bucket and uploads the zip file
Deploys an EC2 instance via CloudFormation cfn-surveillance-camera-infrastructure.yaml that downloads the .zip file and executes the fleet provisioning and media streaming.

Folder Structure & Snippets

camera/
├─ scripts/
│  ├─ main.py
│  ├─ provisioning_handler.py
│  ├─ config.ini
│
├─ claim-certificates/
│
├─ service/
│  ├─ videostream.service
│  ├─ stream-video.sh
│
├─ cfn-surveillance-camera-mock-buckets.yaml
├─ cfn-surveillance-camera-mock.yaml
│
├─ install-mock.sh
├─ uninstall-mock.sh

The install-mock.sh is the entry point to create all related infrastructure and files of the EC2 camera mock.

The uninstall-mock.sh is the entry point to remove all related infrastructure and files of the EC2 camera mock.

The cfn-surveillance-camera-mock-buckets.yaml cloudformation template creates an S3 bucket that is used to store the scripts, claim-certificates and service folder in a zip file for the EC2 camera mock.

The cfn-surveillance-camera-mock.yaml cloudformation template creates infrastructure of an EC2 machine that will stream a sample video to Amazon Kinesis Video Streams.

The subfolder scripts contains the python scripts for provisioning new certificates based on the fleet provisioning by claim flow. The config.ini file is generated by the ./infrastructure/install-infrastructure.sh script and contains information of the mock device. (see code snippet)

The claim-certificates subfolder is generated by the ./infrastructure/install-infrastructure.sh script and contains the created claim certificates by the script. (see code snippet)

The service subfolder contains a Linux service videostream.service to run as a background progress and restart the video-stream.sh which is generated by the install-mock.sh. (see code snippet).

The video-stream.sh executes the amazon-kinesis-video-streams-webrtc-sdk-c sample on the mock camera, which is downloaded and installed during the startup of the EC2 instance. (see code snippet)

The folders scripts, claim-certificates, and service are zipped and uploaded to S3 to be available to the EC2 camera mock for download during startup. (see code snippet)

infrastructure/
├─ lambda/
│
├─ cfn-surveillance-camera-infrastructure.yaml
├─ cfn-surveillance-camera-buckets.yaml
│
├─ install-infrastructure.sh
├─ uninstall-infrastructure.sh

The install-infrastructure.sh is the entry point to create all related infrastructure and files of the backend services of the platform solution. It also generates the configuration files for the web-client .env and camera config.ini. (see code snippet)

The uninstall-infrastructure.sh is the entry point to remove all related infrastructure and files of the backend services of the platform solution.

The cfn-surveillance-camera-buckets.yaml cloudformation template creates an S3 bucket that is used to store the lambda functions a zip file for serverless api proxy pattern.

The cfn-surveillance-camera-infrastructure.yaml cloudformation template creates infrastructure of the backend services of the platform solution including Amazon Cognito, AWS Lambda, Amazon API Gateway, Amazon DynamoDB, AWS IoT Core and Amazon Kinesis Video Streams.

web-client/
├─ src/
│  ├─ sub/
│  │  ├─ Camera.tsx
│  │  ├─ Login.tsx
│  ├─ util/
│  │  ├─ Backend.tsx
│  │  ├─ Video.tsx
│  │  ├─ WebRTCViewer.tsx
│
├─ .env

The configuration files for the web-client .env are created by the ./infrastructure/install-infrastructure.sh script and contains the base url to the Amazon Api Gateway and Amazon Cognito. (see code snippet)

The WebRTCViewer.tsx component creates the connection to the Amazon Kinesis Video Streams service to consume via WebRTC the sample video stream from the camera mock. The Backend.tsx component handles the api calls to the Amazon Api Gateway. The Video.tsx is a wrapper component for the WebRTCViewer to display in the frontend.

The Camera.tsx component calls the backend functions to handle the camera binding. The Login.tsx component handles the redirecting to the Amazon Cognito hosted UI, the receiving of the tokens, and the storing of the tokens within a cookie.

Quick start

Please follow the instructions in the walktrough section of the corresponding blogpost on how to get started: Design considerations for cost-effective video surveillance platforms with AWS IoT for Smart Homes

How to clean up

Run the following script to remove the deployed infrastructure, permissions and roles.

cd infrastructure
sh ./uninstall-infrastructure.sh

Run the following script to remove the camera mock and its resources.

cd camera
sh ./uninstall-mock.sh

Open Cloud9 in the AWS Management Console
Select surveillance-camera-ide
Click Delete
Type Delete to agree.

Web Client Walkthrough

Copy the Username and Password and select Login

Enter the credentials, select a new password and setup a software mfa in the Cognito Hosted UI. see authenticator apps
Enter the provided Serial number and Secret and select Submit to bind the user to the provisioned camera. (see binding users to cameras)

Note: The secret and serial number are shown here in the clear for simplicity and experimental purpose only. You can add additional devices in the DynamoDB table. Serial numbers and secrets should always be protected when used in production.

Once the camera mock provision status is true, select BCM2835-00000000b211cf11 in the table.
1. Refresh the page to request a status update or if an error occurs

You will see the test stream from the camera mock as below.

Solution Walkthrough

Seamless onboarding of cameras to AWS
Registering and verifying access to backend services
Binding users to cameras
Create session policies for web client to access Kinesis Video Streams
Pushing and consuming video data from the camera

Overview

Seamless onboarding of cameras to AWS

You need a database with a device_table that holds information of the serial number of all the manufactured surveillance cameras. In this example, the serverless key-value database service Amazon DynamoDB is used to verify identities, to store user and device data.

This figure shows the onboarding flow of the surveillance cameras. Establish the first connection with the claim certificate (orange path). As soon as you verified the device and IoT Core provisioned a private certificate, establish the communication via this private certificate (green path).

The common bootstrap certificate (Claim Certificate) is received from IoT Core. (see ./camera/claim-certificate) Place this certificate along with your OS and your scripts on each surveillance camera.
Configure the camera to connect with the bootstrap certificate to the IoT Core endpoint requesting a unique private X.509 certificate. (see ./camera/scripts/main.py) Use the device’s serial number as client id. (see ./camera/config.ini)
You use the created pre-provisioning hook function (see ./infrastructure/lambda/CheckDeviceManufactured/index.js) to verify the device’s identity by looking up a device’s serial number or other secret in a pre-approved device_table in DynamoDB. (see device_table in DynamoDB)
If the camera is an approved device, the fleet provisioning template sets up the device configuration and attaches AWS IAM policies and AWS IoT Core policies to the private certificate. A new thing name identical to the client id is registered in AWS IoT. (see CameraProvisioningTemplate in AWS IoT)
The camera is now authorized with KVSCameraCertificateBasedIAMRole to connect to Kinesis Video Streams to push video data. (see Certificates and Policies in AWS IoT)

Registering and verifying access to backend services

The script builds the backend services based on this multi-tier serverless pattern where the logic layer comprises of AWS Lambda functions. This layer handles different aspects of your client, such as reading from databases, updating camera states and communicating with Kinesis Video Streams. Your users authenticate with Amazon Cognito, call Amazon API Gateway and proxy requests to AWS Lambda functions.

You set up an Amazon Cognito User Pool and use the OAuth 2.0 authentication protocol. (find your User Pool here)
You created a REST API with Amazon API Gateway and integrated Amazon Cognito with it.
The REST API will proxy the request to the respective Lambda function. (see ./infrastructure/lambda)
The web application (see ./web-client) implements Amazon Cognito by using the hosted UI of the service. You could also implement the logic by yourself using Amazon Cognito Identity SDK for JavaScript.
The REST API has GET and POST paths using the received tokens in the headers. (see Amazon API Gateway)

Binding users to cameras

You need to create a user convenient and secure mechanism to create the user-device-binding. The camera should only be bound to one user at a time and a user should not be able to register multiple devices by guessing the device identifier. Here, you provide the unique device identifier of the camera to the user (i.e. with a QR-Code). You also provide the user with a secret that is associated with the unique device identifier of the camera.

Users enter the device identifier of their camera along with a secret pin in the web application. The input is validated with a JSON validation schema in the API Gateway. The proxy pattern is known from above.
You use the id_token in the Lambda function to receive the unique identifier (see ./infrastructure/CreateUserDeviceBinding).
The same Lambda function reads from the device_table and writes to the user_device_table.
It looks up the device_id in the device_table and verify the secret to prevent incorrect user device bindings.
It looks up the device_id in the user_device_table. If there is no related device_id, create a new entry. Otherwise, reject the request as the device is already bound.

Create session policies for web client to access Kinesis Video Streams

Authenticated users send a GET request comprising the id_token and device_id.
Use the id_token in the Lambda function to receive the user unique identifier (i.e., email).
Verify the device_id and user_id from the user_device_table in DynamoDB.
On successful verification, call Identity & Access Management from the Lambda function to receive secret and access key with a session policy to access the stream with the device_id. Send the response to the client.
The client connects to the Signaling Server on Amazon Kinesis Video Streams.

Pushing and consuming video data from the camera

The Kinesis Video Streams Producer Libraries for C++ are build on the camera mock (see last block of ./camera/cfn-surveillance-camera-mock.yaml)

A shell script executes the sample from the amazon-kinesis-video-streams-webrtc-sdk-c. (create the camera mock and see ./service/stream-video.sh).

This script will run as background service (see ./service/videostream.service).

The web client uses the WebRTC SDK in JavaScript for Web Applications to consume the video stream of the camera (see ./web-client/src/util/WebRTCViewer.tsx)

The camera mock creates a Signaling Server in Amazon Kinesis Video Streams and connects to it. The web client will also connect to the Signaling Server.
The camera and the web client request the public ip addresses of each other via the STUN Server.
Interactive Connectivity Establishment (ICE) is a framework managing candidate discovery, connectivity checks, keepalives, and more. ICE uses STUN and/or TURN servers to determine ICE candidates comprised of public and private IP addresses and port numbers to use to connect.
The peers attempt to establish a peer-to-peer connection
The TURN server to relay the connection information is used as the fallback approach on peer-to-peer connection failure.

nauw / amazon-kinesis-video-streams-serverless-surveillance-platform Goto Github PK