stakpak / devx

A tool for generating, validating & sharing all your configurations, powered by CUE. Works with Kubernetes, Terraform, Compose, GitHub actions and much more...

Documentation

Introduction

DevX is a tool for building lightweight Internal Developer Platforms. Use DevX to build internal standards, prevent misconfigurations early, and enable infrastructure self-service.

Installation

Homebrew

brew tap stakpak/stakpak
brew install devx       

Download the binary

Releases page

Docker image

docker run --rm -v "$(pwd):/app" ghcr.io/stakpak/devx:latest -h

Quick start

➜ devx project init
➜ devx project update
➜ devx project gen
➜ devx build dev
🏭 Transforming stack for the "dev" environment...
[compose] applied resources to "build/dev/compose/docker-compose.yml"
[terraform] applied resources to "build/dev/terraform/generated.tf.json"

Usage

Configuration language

We use CUE to write strongly typed configurations. You can now shift YAML typos left, instead of detecting errors when applying configurations. You can easily transform CUE configuration files to and from YAML (CUE is a superset of YAML & JSON).

CUE is the result of years of experience writing configuration languages at Google, and seeks to improve the developer experience while avoiding some nasty pitfalls. CUE looks like JSON, while making declarative data definition, generation, and validation a breeze. You can find a primer on CUE here.

Create a stack (by Developers)

You create a stack to define the workload and its dependencies.

package main

import (
    "stakpak.dev/devx/v1"
    "stakpak.dev/devx/v1/traits"
)

stack: v1.#Stack & {
    components: {
        cowsay: {
            traits.#Workload
            containers: default: {
                image: "docker/whalesay"
                command: ["cowsay"]
                args: ["Hello DevX!"]
            }
        }
    }
}

Create your own stack builders or use community packages (by Platform Engineers)

You can customize how the stack is processed by writing declarative transformers.

package main

import (
    "stakpak.dev/devx/v2alpha1"
    "stakpak.dev/devx/v2alpha1/environments"
)

builders: v2alpha1.#Environments & {
    dev: environments.#Compose
}

Validation

Validate configurations while writing

➜ devx project validate
👌 Looks good

Platform capability discovery

➜ devx project discover --transformers
[🏷️  traits] "stakpak.dev/devx/v1/traits"
traits.#Workload	a component that runs a container 
traits.#Replicable	a component that can be horizontally scaled 
traits.#Exposable	a component that has endpoints that can be exposed 
traits.#Postgres	a postgres database 
traits.#Helm	a helm chart using helm repo 
traits.#HelmGit	a helm chart using git 
traits.#HelmOCI	a helm chart using oci 

[🏭 transformers] "stakpak.dev/devx/v1/transformers/argocd"
argocd.#AddHelmRelease	add a helm release  (requires trait:Helm)

[🏭 transformers] "stakpak.dev/devx/v1/transformers/compose"
compose.#AddComposeService	add a compose service  (requires trait:Workload)
compose.#ExposeComposeService	expose a compose service ports  (requires trait:Exposable)
compose.#AddComposePostgres	add a compose service for a postgres database  (requires trait:Postgres)

[🏭 transformers] "stakpak.dev/devx/v1/transformers/terraform"
terraform.#AddHelmRelease	add a helm release  (requires trait:Helm)

Package management

You can publish and share CUE packages directly through git repositories.

Create a new packages

Create a new repository to store your packages (you can host multiple packages per repository).

cue.mod
└── module.cue # module: "domain.com/platform"
subpackage
└── file.cue
file.cue

Add the package to module.cue

module: ""

packages: [
  "github.com/<org name>/<repo name>@<git revision>:",
]       	

For private packages (optional)

export GIT_USERNAME="username"
export GIT_PASSWORD="password"

Update packages (pulling updates will replace existing packages)

➜ devx project update

Contributors

George

Mohamed Hamza

Lam Tran

Ahmed Hesham

John Gosset

devx's Issues

Diff between environments

Diff is supported between different git revisions, It will be very useful to be able to detect the diff between different environments as well.

Some use-cases:

1. used when reviewing changes for promotion between environments
2. find discrepancies between environments for debugging
3. compare on-demand environments

devx run dev --list-all Error: field not found: builders

in the simple example i am trying to run taskfile

simple % devx run dev --list-all
Error: field not found: builders

Gitignore initialization

add cue.mod/pk to .gitignore on init.

ArgoCD plugin

An ArgoCD plugin to build stacks

Transformer arugments can clash each other

In kubernetes transformers, the following sample uses appName and serviceAccountName as arugments which will eventually be part of the component model and these names can not be used across any other transformer.

https://github.com/devopzilla/guku-devx/blob/5743b77354638b7fe4e592ddaf83df47e6691a19/pkg/guku.io/devx/v1/transformers/kubernetes/transformers.cue#L35-L36

As a suggestion, I would add some annotations field to $metadata and use it as a freemap then use it to define arguments, for example:

#SomeTransformer: v1.#Transformer & {
  v1.#Component
  …
  $metadata: annotations: kubernetes: deployment: appName: string | *$metadata.id 
}

Semantic versioning constraints for components

Use case

A service invoices depends on another service's stack users, and we to make sure that a supported version of users is deployed when building a local dev stack.

Proposal

Schema

#SemVer: {
	prefix: string | *""
	major:  uint | *0
	minor:  uint | *0
	patch:  uint | *0
	suffix: string | *""
	value:  "\(prefix)\(major).\(minor).\(patch)\(suffix)"
}

#Component: {
	$metadata: {
		id: string
		labels: [string]: string
		version: #SemVer
	}
	#Trait
}

Service invoices that depends on an imported stack of another service users and want to constrain the supported version

package main

import (
	"guku.io/devx/v1"
	"guku.io/devx/v1/traits"
)

stack: v1.#Stack & {
	components: {
		users: $metadata: version: {
			major: 10
			minor: >=20
		}
		invoices: {
			traits.#Workload
			containers: default: {
				image: "docker/whalesay"
				command: ["cowsay"]
				args: ["Hello DevX!"]
				env: USER_SERVICE_HOST: users.endpoints.default.host
			}
		}
	}
}

Build signatures

Mainly for supply chain security.

We have a PoC for cryptographically hashing builds, we should support client signatures for builds as well.

When creating a shared volume component it's added as a dependency in compose

Investigate integration with Backstage

Support rollbacks without checking out git manually

There is probably an SSH scheme package URL bug

Need to make sure we add the correct URL scheme when authenticating using an SSH key

Pin package dependencies

We should be able to pin package dependencies, and support upgrades following the main branch.

This command should install packages from pinned commits only (or pinning them if they don't exist)
devx project update

This command should upgrade the binned commit hashes to the latest commit or to a user-specified commit
devx project upgrade

Ultimately we should design the packaging & schema convention to be immutable like Nix or Grafana Thema, so it's very hard to introduce breaking changes to configurations.

Optional trait fields and maps are not printed out in discovery

We need a testing framework for transformers

Proposal

Test case schema

#TestCase: {
	name:        string
	description: string

	transformer: #Transformer
	input:       #Component
	result:      input & transformer

	expect: result
	assert: [string]: true
}

Example test case

package main

import (
	"guku.io/devx/v1"
	"guku.io/devx/v1/traits"
	"guku.io/devx/v1/transformers/compose"
)

_t1: v1.#TestCase & {
	name:        "exposable"
	description: ""

	transformer: compose.#ExposeComposeService
	input: {
		traits.#Exposable
		$metadata: id: "obi"
		endpoints: default: ports: [
			{
				port:   8080
				target: 80
			},
		]
	}
	result: _

	expect: {
		endpoints: default: host: "obi"
		$resources: compose: services: obi: ports: ["8080:80"]
	}
	assert: {
		"host is concrete": (result.endpoints.default.host & "123") == _|_
	}
}

Run test

cue eval

Pass environment variables during build

This could be done by adding an attribute like we do for @guku(generate) it could look like @guku(env=ENV_NAME)

Implement vscode integration for discovery

We need to implement a vscode plugin to show trait fields & do validations inside of vscode itself