sshabb697 / azure-kusto-labs

Self-contained, hands-on-labs with detailed and step-by-step instructions, associated collateral (data, code etc) on trying out various features and integration points of Azure Data Explorer (Kusto)

About

This repository features self-contained, hands-on-labs with detailed and step-by-step instructions, associated collateral (data, code etc) on trying out various features and integration points of Azure Data Explorer (Kusto)

Labs

1. Kafka Ingestion series

Mutiple labs that cover stand-alone and distributed modes of Kafka ingestion across Azure HDInsight Kafka PaaS and Confluent Kafka cluster flavors (IaaS and PaaS).

2. Kubernetes Container Log Analytics

2.1. Kubernetes Container Log Analytics with Fluent-Bit

Featuring Fluent-Bit v1.3.11 for log collection and forwarding, Azure Event Hub as streaming source, and straight through ingestion into Azure Data Explorer with our Azure Event Hub integration.

3. Cosmos DB integration with Azure Data Explorer

3.1 Cosmos DB integration with Azure Data Explorer using change feed

This lab covers end to end integration of Cosmos DB with Azure Data Explorer using Cosmos DB change feed for building near real-time analytical solution with a flavor of Azure Data Explorer dashboards.

4. Databricks integration with Azure Data Explorer

4.1 Databricks integration with Azure Data Explorer using Azure Functions

This lab covers massive telemetry analytics architecture for ingestion, processing and analytics over 100s of TBs of data leveraging Databricks as an ETL tool and Azure Data Explorer(ADX) as an analytics store for building real time analytics scenarios. Also, it provides mechanisms for monitoring and errors retries. This is a proven architecture implemented by one of our very large enterprise customer.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.