In this github repository, I am experimenting building microservices with
- .NET Core
- Docker
- Kubernetes on Azure
My goal is to build a case study that I can use to evaluate the scalability possibilities of the technologies mentioned above and also build a real time web dashboard
- Other Azure resources used: table, queue, service bus, web site, load balancer.
- I am going to build a back end able to received and processed online donations with a monitoring web dashboard.
- The donations should be created by hundreds of thousands users entering donation amounts and credit card information on a web site and press send.
Here is the overview execution architecture:
I will not build the front end, but to simulate the donations sent I will create a .NET Core console application that can be instantiated up to 10 times in Docker containers in an Azure Kubernetes cluster.
Each instance will read a specific local JSON file donation[X].json containing 50 000 donations instances like this one
{
"Guid":"cd7af44d-db7b-4d4c-9157-052ce5f50836",
"FirstName":"Sonny","LastName":"Haking","Gender":"Male",
"Email":"[email protected]", "Phone":"310-632-6062","IpAddress":"138.27.230.192",
"Country":"Indonesia","Amount":"$91.37",
"CC_Number":"4026367644878790","CC_ExpMonth":12,"CC_ExpYear":2022,"CC_SecCode":233
}and execute an HTTP POST to a specific end point for each donation. The console has the ability to execute 10 HTTP POSTs in parallel. For every 500 donations sent, the application send to an Azure Service Bus channel (Publisher/Subscribers) a message containing performance information.
-
Deployment: The .NET Core console is deployed as a Kubernetes kind:Service and kind:StatefulSet, this allow each container instance machine name to be set to 'personsimulator-[VERSION]-sfs-[INDEX]', The machine name ends with a container index instance, for example 'personsimulator-1.0.50-sfs-0'. The index is used to load the local JSON file. See file donation-personsimulator-console--deployment.{Params}.yaml
A .NET Core REST API will implement the HTTP POST to received the donations. Multiple instances of the API process will be deployed and executed as a Docker container behind a load balancer provisioned using Azure Kubernetes Service. When a donation is received, it is
- Validated
- Pushed to an Azure Queue
- For every 500 donations received, the endpoint send to an Azure Service Bus channel (Publisher/Subscribers) a message containing performance information.
A .NET Core console application that can be instantiated multiple times as Docker containers instances in an Azure Kubernetes cluster will
- Pop messages from the Azure Queue
- Validate the data
- Store the data in an Azure Table
- Aggregate the amounts received per country for the last 500 donations and store the result into another Azure Table.
- For every 500 donations processed, the application send to an Azure Service Bus channel (Publisher/Subscribers) a message containing the aggregated country/amount information and other performance information.
An ASP.NET Core Web Application implementing
- An internal endpoint named SystemActivitiesController will
-
Receive the information sent by the the different processes via the Azure Service Bus channel
-
Store and aggregate the data in static dictionaries in memory
-
Communicate the information to the Dashboard browser side via HTTP Get
-
-
A Web Dashboard Single Page Application (SPA) written with React that display the performance information sent by the different processes and the donation amounts per country in charts and tables in
pseudo real time.
- The Web Dashboard is deployed in Azure in an AppService
The build and deployment processes consists of
- Compiling the .NET Core projects
- Creating the docker images locally
- Pushing the docker images to an Azure Container Registry
- Deploying the different container images to multiple container instances in an Azure Kubernetes cluster
All these steps are automated using PowerShell scripts and the Kubernetes command line tool KubeCtl.exe, running on an Azure VM.
- See powershell scripts in folder source
-
The file Create.Kubernetes.ps1 is used to create or delete the Azure Kubernete cluster.
-
The file DeploymentUtilityMaster.ps1 allow to build and deploy any or all the .NET core projects.
- Each of the .NET Core project implement a local file named DeploymentUtility.ps1 which get called by the DeploymentUtilityMaster.ps1
-
The file Deployment.Kubernetes.ps1 is used to deploy any of the .NET Core project as a container based on the last build and last container images created and pushed into Azure.
-
The file [KubernetesManager.psm1] is a re-usable library to execute Kubernetes YAML file and wait for the deployment.
-
The file Util.psm1 contains general helper functions.
A simple template engine written in PowerShell will execute the YAML template file located in folder
- See function processFile in file Util.psm1 contains general helper functions.
I recorded and narrated, a full demo of my case study.
- 01 Azure Kubernetes Cluster Creation
- 02 Building the .NET Core and Docker images
- 03 Kubernetes cluster is ready
- 05 Web dashboard deployment and review
- 04 Let's review the build result and the Azure Container Registry
- 06 Let's deploy the endpoint
- 07 Let's deploy the queue processor
- 08 Let's run the simulation (part I)
- 08 Let's run the simulation (part II)
- 09 Let's run the simulation (part III)
My Kubernetes cluster is running 3 Linux Azure virtual machines of type
- Standard_D2_v2 - 2 cpu - 7 Gb Ram
My .NET Core App / Container Instances are configured this way:
- Donation.PersonSimulator.Console 3 container instances
- Donation.RestApi.Entrance is 3 container instances
- Behind 1 Azure load balancer provisioned using Kubernetes Service.
- Donation.QueueProcessor.Console is 3 container instances
For a total of 60% of CPU for the all cluster and very little memory, the web dashboard reported performance information in row 0 in table below. (See videos 08 and 09)
| Kubernetes Cluster Configuration | Person Simulator | Rest Api | Queue Processor |
|---|---|---|---|
| 3 VM of type Standard_D2_v2 | 3 containers. 370 donations/S | 3 containers. 370 donations/S | 3 containers. 280 donations/S |
| 3 VM of type Standard_D2_v2 | 4 containers. 406 donations/S | 4 containers. 406 donations/S | 4 containers. 294 donations/S |
| 3 VM of type Standard_D2_v2 | 5 containers. 405 donations/S | 5 containers. 405 donations/S | 5 containers. 317 donations/S |
In this table I tried 4, 5 and 6 Linux Azure virtual machines of type
- Standard_D2_v2 - 2 cpu - 7 Gb Ram And also different number of container instances.
| Kubernetes Cluster Configuration | Person Simulator | Rest Api | Queue Processor |
|---|---|---|---|
| 4 VM of type Standard_D2_v2 (2 CPU, 7 Gb Ram) | 4 containers. 474 donations/S | 4 containers. 406 donations/S | 4 containers. 328 donations/S |
| 5 VM of type Standard_D2_v2 (2 CPU, 7 Gb Ram) | 5 containers. 523 donations/S | 5 containers. 523 donations/S | 5 containers. 427 donations/S |
| 5 VM of type Standard_D2_v2 (2 CPU, 7 Gb Ram) | 6 containers. 535 donations/S | 6 containers. 535 donations/S | 6 containers. 473 donations/S |
| 6 VM of type Standard_D2_v2 (2 CPU, 7 Gb Ram) | 5 containers. 506 donations/S | 5 containers. 506 donations/S | 5 containers. 417 donations/S |
| 6 VM of type Standard_D2_v2 (2 CPU, 7 Gb Ram) | 6 containers. 563 donations/S | 6 containers. 563 donations/S | 6 containers. 499 donations/S |
In this configuration my Kubernetes cluster run 3 Standard_D4_v3 (4 CPU, 17 Gb Ram) Linux Azure VMs.
| Kubernetes Cluster Configuration | Person Simulator | Rest Api | Queue Processor |
|---|---|---|---|
| 3 VM of type Standard_D4_v3 (4 CPU, 17 Gb Ram) 55% CPU used | 6 containers. 556 donations/S | 6 containers. xxx donations/S | 6 containers. 369 donations/S |
| 3 VM of type Standard_D4_v3 (4 CPU, 17 Gb Ram) 65% CPU used | 8 containers. 622 donations/S | 8 containers. xxx donations/S | 8 containers. 556 donations/S |
| 3 VM of type Standard_D4_v3 (4 CPU, 17 Gb Ram) 65% CPU used | 10 containers. 689 donations/S | 10 containers. xxx donations/S | 10 containers. 532 donations/S |
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent