kislerdm / org-infrastructure-overview

The tool to facilitate understanding of complex systems using C4 model diagrams with dynamically-adjustable level of details.

org-infrastructure-overview: organisation infrastructure's architecture overview with dynamically-adjustable level of details

org-infrastructure-overview is the tool aiming to facilitate understanding dependencies between complex systems across (sub-)organisations using C4 model diagrams with dynamically-adjustable level of details. It illustrates inter-domains interactions spanning several teams and services based on their definition as the graph.

Contents

• Demo
• How to run
  • Prerequisites
  • Commands
• How to define graph
• Contribution
  • Further development

Demo

The screenshot illustrates the tool in action by visualising the infrastructure defined in the file.

Demo topology

The demo application is based on the demo org structure topology described below.

GIVEN

• The organisation Foo contains of two departments: DepartmentA and DepartmentB.
• The DepartmentA has two domains: DomainA and DomainB.
• The DomainA consists of three teams: Team0 (backend), Team1 (frontend).
• The DomainB has one team: Team2 (analytics and reconciliation).
• The DepartmentB has two teams: Team3 (streaming platform) and Team4 (CIAM).
• The Team0 has one outbound dependency: Team1, and two inbound dependencies: Team3 and Team4.
• The Team1 has two inbound dependencies: Team0 and Team4.
• The Team2 has one dependency: Team3.
• The Team3 has three dependencies: Team0, Team2 and Team4.
• The Team4 has three outbound dependencies: Team0, Team1 and Team3, and one inbound dependency: Team3.
• The Team0 owns two services: Service0 and Service1.
• The Team1 owns one service: Service2.
• The Team2 owns one service: Service3.
• The Team3 owns three services: Service4, Service5 and Service6.
• The Team4 owns three services: Service7, Service8 and Service9.
• Service0 consists of three containers:
  • App1: Kotlin application deployed as AWS EKS;
  • Database: AWS Aurora Postgres.
  • Cache: AWS Elasticache Redis.
• Service1 is a Kotlin application to run batch jobs on database.
• Service2 is a JavaScript application deployed to AWS EKS.
• Service3 consists of two containers:
  • App: Python batch processing app deployed to AWS EKS;
  • Database: analytics db AWS Redshift.
• Service4 consists of three containers:
  • App1: Kafka deployed as AWS MSK;
  • App2: Schema Registry deployed as AWS Glue Schema Registry;
• Service5 consists of three containers:
  • App1: Go application to sync domain events data to datalake deployed to AWS EKS;
  • Database: S3 bucket.
• Service6: secrets manager deployed as AWS Secretsmanager.
• Service7: is a Go application to mutate user's account deployed to AWS EKS;
• Service8 consists of four containers:
  • IAM: AWS Cognito;
  • l0: Go application deployed as AWS Lambda for AWS Cognito trigger 1;
  • l1: Go application deployed as AWS Lambda for AWS Cognito trigger 2;
  • l2: Go application deployed as AWS Lambda for AWS Cognito trigger 3;
• Service9 is AWS SES used as the email notification service.

Note that the above description is stored as the graph definition for the tool.

Test Scenario - Domain Level

WHEN

DomainA is selected

THEN

The following diagram is expected.

C4Container
    Enterprise_Boundary(Foo/DepartmentA, "Department A") {
        Component(Foo/DepartmentA/DomainA, "DomainA")
    }

Test Scenario - Container Level

WHEN

Service2 is selected

THEN

The following diagram is expected.

C4Container
    Enterprise_Boundary(Foo/DepartmentA/DomainA/Team1, "Team1") {
        Container(Foo/DepartmentA/DomainA/Team1/Service2, "Service2", "JavaScript/AWS EKS", "web application used by clients")
    }
    Enterprise_Boundary(Foo/DepartmentA/DomainA/Team0, "Team0") {
        System_Ext(Foo/DepartmentA/DomainA/Team0/Service0, "Service0")
    }
    Enterprise_Boundary(Foo/DepartmentB/Team4, "Team4") {
        System_Ext(Foo/DepartmentB/Team4/Service8, "Service8", "", "IAM")
    }
    Rel(Foo/DepartmentA/DomainA/Team1/Service2, Foo/DepartmentA/DomainA/Team0/Service0, "Uses to process user's requests", "sync, HTTP/JSON")
    Rel(Foo/DepartmentA/DomainA/Team1/Service2, Foo/DepartmentB/Team4/Service8, "Authenticates users", "sync, HTTP/JSON")

How to run

Prerequisites

• Node.js v18.16+

Commands

Run to setup:

npm install

Run to test:

npm run test

Run to launch a dev server:

npm run dev

Run to build:

npm run build

How to define graph

The graph definition rules:

• The graph must be defined according to the JSON schema.
• Each element in the graph is referred to as a node.
• Dependencies between nodes are defined by links.
• Every link represents a dependency, not data flow. The attribute from denotes the ID of the node dependent on the node denoted by the attribute to.
• The node ID is defined as a Unix path, which includes the IDs of the node's parent and the parent of its parent and so on recursively.
• The node ID is defined by using the names of the node and its parents. All special characters and spaces are removed, and the results are concatenated using the forward slash sign (/) as the separator.

The contextual topology

organisation 0
|-- department 0/0
|   |-- domain 0/0/0
|   |   |-- team 0/0/0/0
|   |   |   |-- service 0/0/0/0/0
|   |   |   |   |-- application 0/0/0/0/0/0
|   |   |   |   |-- application 0/0/0/0/0/1
|   |   |   |   |   ...
|   |   |   |   |-- queue 0/0/0/0/0/2
|   |   |   |   `-- database 0/0/0/0/0/P
|   |   |   |-- service 0/0/0/0/1
|   |   |   |   ...
|   |   |   `-- service 0/0/0/0/L
|   |   |-- team 0/0/0/1
|   |   |   `-- application 0/0/0/1/0
|   |   |   ...
|   |   `-- team 0/0/0/K
|   |-- domain 0/1
|   |   ...
|   |-- domain 0/X
|   |   `-- application 0/X/0
|   |   ...
|   `-- domain 0/M
|-- department 0/1
|   ...
|-- team 0/R
|   ...
|-- department 0/Y
|   `-- database 0/Y/0
|   ...
|-- department 0/Z
|   `-- team 0/Z/0
|   ...
`-- department 0/N

The context transitions from the organisation level down to the system level. The diagram of every level includes the nodes which belong to selected level's node and the linked nodes from other levels.

Graph definition example

GIVEN

• Two business domains DomainA and DomainB belong to the same business unit Unit0;
• DomainA updates the entity based on the domain events emitted by the DomainB;
• Both domains communicate via the Platform0 by publishing and consuming domain events.

WHEN

The given topology is intended to be illustrated using the org-infrastructure-overview tool.

THEN

The following graph shall be expected:

{
  "nodes": [
    {
      "name": "Unit0",
      "type": "department",
      "nodes": [
        {
          "name": "DomainA",
          "type": "domain"
        },
        {
          "name": "DomainB",
          "type": "domain"
        }
      ]
    },
    {
      "name": "Platform0",
      "type": "service"
    }
  ],
  "links": [
    {
      "from": "Unit0/DomainA",
      "to": "Unit0/DomainB",
      "description": "Uses to update the entity state"
    },
    {
      "from": "Unit0/DomainA",
      "to": "Platform0",
      "description": "Publishes and consumes domain events"
    },
    {
      "from": "Unit0/DomainB",
      "to": "Platform0",
      "description": "Publishes and consumes domain events"
    }
  ]
}

Contribution

The codebase is distributes under the MIT license. Please feel free to open PR, or issue with your request.

Further development

Given that the organisations which would benefit from the tool use spotify backstage, or opslevel, it would make sense to integrate the tool's logic as plugins to the mentioned systems.