aws-big-data-projects / heartrate-monitoring-using-aws-iot-and-aws-kinesis

you run a script to mimic multiple sensors publishing messages on an IoT MQTT topic, with one message published every second. The events get sent to AWS IoT, where an IoT rule is configured. The IoT rule captures all messages and sends them to Firehose. From there, Firehose writes the messages in batches to objects stored in S3. In S3, you set up a table in Athena and use QuickSight to analyze the IoT data.

HeartRate-Monitoring-using-AWS-IOT-and-AWS-KINESIS

Heart rate monitoring using aws-iot,aws-kinesis,amazon-athena and visualize the data using amazon quicksight

you run a script to mimic multiple sensors publishing messages on an IoT MQTT topic, with one message published every second. The events get sent to AWS IoT, where an IoT rule is configured. The IoT rule captures all messages and sends them to Firehose. From there, Firehose writes the messages in batches to objects stored in S3. In S3, you set up a table in Athena and use QuickSight to analyze the IoT data.

Configuring Firehose to write to S3

Configuring the AWS IoT rule

Generating sample data

Running the heartrate.py python script generates fictitious IoT messages from multiple userid values. The IoT rule sends the message to Firehose, which writes the data out to S3.

The script assumes that it has access to AWS CLI credentials and that boto3 is installed on the machine running the script.

Configuring Athena

create table using athenaquery.sql after the query completes u see new table being created.

Analyzing the data

Now, analyze this data using Athena via QuickSight.

Set up a data source

Log into QuickSight and choose Manage data, New data set. Choose Athena as a new data source.

Build an analysis

Reference Material:https://aws.amazon.com/blogs/big-data/derive-insights-from-iot-in-minutes-using-aws-iot-amazon-kinesis-firehose-amazon-athena-and-amazon-quicksight/

heartrate-monitoring-using-aws-iot-and-aws-kinesis's Issues

Analyzing the Data using Quicksight by using new features

the current system uses athena to provide the quicksight features . make changes so that the new data source is used for analysis using quicksight.

update the project to include recent changes

git the project locally , make changes as per new updates and run the project, on success update the github project

Generating new Sample Data as this doesn't work

make changes to Running the heartrate.py python script generates fictitious IoT messages from multiple userid values. The IoT rule sends the message to Firehose, which writes the data out to S3.

The script assumes that it has access to AWS CLI credentials and that boto3 is installed on the machine running the script.

The hearrate.py doesn't work as the sample data is invalid.