plzm / always-on Goto Github PK

Id/demo critical failure scenarios throughout app stack.

• Chaos Engineering?

Build on 1.1 and 1.2 work already done

Implement per docs

https://docs.microsoft.com/en-us/azure/azure-app-configuration/concept-github-action
https://docs.microsoft.com/en-us/visualstudio/azure/vs-azure-tools-connected-services-app-configuration?view=vs-2019
https://docs.microsoft.com/en-us/azure/azure-app-configuration/enable-dynamic-configuration-aspnet-core?tabs=core5x
https://docs.microsoft.com/en-us/azure/azure-app-configuration/enable-dynamic-configuration-dotnet-core

Document steps to be able to deploy multiple stamps per region, regional dependencies, etc.

Determine how to fail AKS Node for load/availability test.

• Chaos Engineering?

Convert to central config instead of specifying env vars per workflow

Harden system security. Demo resilience to typical security risks, particularly DDoS.

• BreakingCloud

Elaborate the single end to end CD pipeline as follows:

• Global resources
• Per-region stamp

Deploy a regional infrastructure stamp for the AlwaysOn architecture.

• Script
• Pipeline

Design for n regions. Get 1 working, then extend.

Tune Cosmos DB for performance targets.

• Index policy

• Partition key

• Point read for FE API

• Multi-region write: determine appropriate consistency level. This may mean Eventual or Bounded Consistency as it's a game/high-throughput scenario so latency/blocking is key.

Base foundation deploy on ESLZ Wingtip
https://github.com/Azure/Enterprise-Scale/tree/main/docs/reference/wingtip

Enable AKS pods to access Key Vault.

Entire application "traffic light" system

• Build container images
• Deploy to ACR
• Pull and deploy updated containers on AKS

Automated tests to exercise API endpoints.

• Event Hub Message Consumer
• Cosmos DB Write (upsert)

• Cosmos DB Get
• Event Hub Msg Producer

NFRs

• Must be able to scale beyond 100,000 tps
• <=200 ms for P99 requests
• Availability > 99.99%

Includes retrieving App Insights Instrumentation key from config store (Key Vault) at API startup

NFRs

Must be able to scale beyond 100,000 tps
<=200 ms for P99 requests
Availability > 99.99%

• BreakingPoint

az aks --attach-acr acrId/Name

ARM template equivalent -

VNet with DDoS Standard
Subnet
NSG
AKV
ACR
Log Analytics
App Insights
Event Hub
(Add Geo-Pairing on added region)
Cosmos DB

App GW V2 with WAF
AKS
Container Insights

APIM
Front Door with WAF
Azure DNS