nais / digdirator

Kubernetes Operator for automated registration and configuration of Digdir clients (ID-porten & Maskinporten)

digdirator

Digdirator is a Kubernetes cluster operator for automated registration and lifecycle management of ID-porten and Maskinporten clients (integrations) with feature Maskinporten Scopes (APIS).

CRD

The operator introduces two new Kinds:
IDPortenClient (shortname idportenclient) and MaskinportenClient (shortname maskinportenclient), and acts upon changes to these.

See the specs in liberator for details:

• config/crd/nais.io_idportenclients.yaml and
• config/crd/nais.io_maskinportenclients.yaml for details.

Sample resources:

• config/samples/idportenclient.yaml
• config/samples/maskinportenclient.yaml.

Lifecycle

Usage

Installation

make install

DigDir Setup

See the documentation over at DigDir for acquiring clients with the required scopes to access the self-service APIs:

• https://docs.digdir.no/docs/idporten/oidc/oidc_api_admin_maskinporten#hvordan-f%C3%A5-tilgang-
• https://docs.digdir.no/docs/idporten/oidc/oidc_api_admin#hvordan-f%C3%A5-tilgang-

Digdirator uses two separate clients (and thus also certificates) for administration of ID-porten and Maskinporten clients.

Use the pem2jwk utility to generate a public JWK with the correct x5c and kid parameters. This key should be registered/associated with your DigDir client out-of-band.

Digdirator will calculate the same kid using the configured certificate chain and use this when creating JWT grants for authenticating with the DigDir self-service APIs.

Google Cloud Platform Setup

Digdirator makes use of a few Google Cloud Platform products; Secret Manager and Cloud Key Management Service (KMS). We use these to store secrets and business certificates, respectively. These are needed for authenticating the DigDir client with Maskinporten using the JWT-bearer authorization grant.

You should set up Workload Identity for your GKE cluster.

Digdirator needs a Google IAM Service Account to access the GCP resources. With Workload Identity, this should work automagically as we use Google's libraries that should automatically pick up the Kubernetes Service Account tokens and perform the necessary exchanges.

Secret Manager

Digdirator needs two secrets for each client:

1. contains the client ID for the DigDir client
2. contains the certificate chain (in PEM format) for the business certificate used to sign the client assertions when acquiring Maskinporten tokens

The Google Service Account needs the following IAM role for both secrets:

• Secret Manager Secret Accessor (roles/secretmanager.secretAccessor)

Cloud KMS

The Google Service Account needs the following IAM role for each key in Cloud KMS:

• Cloud KMS CryptoKey Signer/Verifier (roles/cloudkms.signerVerifier)

Follow Google's documentation for importing keys.

The private key should be imported with the purpose set to ASYMMETRIC_SIGN, and the algorithm set to one of the RSASSA-PKCS1 v1_5 variants.

Configuration

Digdirator can be configured using either command-line flags or equivalent environment variables (i.e. -, . -> _ and uppercase), with DIGDIRATOR_ as prefix. E.g.:

digdir.admin.base-url -> DIGDIRATOR_ADMIN_BASE_URL

The following flags are available:

--cluster-name string                               The cluster in which this application should run.
--development-mode string                           Toggle for development mode. (default "false")
--digdir.admin.base-url string                      Base URL endpoint for interacting with Digdir Client Registration API
--digdir.common.access-token-lifetime int           Default lifetime (in seconds) for access tokens for all clients. (default 3600)
--digdir.common.client-name string                  Default name for all provisioned clients. Appears in the login prompt for ID-porten. (default "ARBEIDS- OG VELFERDSETATEN")
--digdir.common.client-uri string                   Default client URI for all provisioned clients. Appears in the back-button for the login prompt for ID-porten. (default "https://www.nav.no")
--digdir.common.session-lifetime int                Default lifetime (in seconds) for sessions (authorization and refresh token lifetime) for all clients. (default 7200)
--digdir.idporten.cert-chain string                 Secret path in Google Secret Manager to PEM file containing certificate chain for authenticating to DigDir.
--digdir.idporten.client-id string                  Client ID / issuer for JWT assertion when authenticating to DigDir.
--digdir.idporten.kms.key-path string               IDPorten KMS resource path used to sign JWT assertion when authenticating to DigDir.
--digdir.idporten.scopes string                     List of scopes for JWT assertion when authenticating to DigDir with IDporten.
--digdir.idporten.well-known-url string             URL to ID-porten well-known discovery metadata document.
--digdir.maskinporten.cert-chain string             Secret path in Google Secret Manager to PEM file containing certificate chain for authenticating to DigDir.
--digdir.maskinporten.client-id string              Client ID / issuer for JWT assertion when authenticating to DigDir.
--digdir.maskinporten.default.client-scope string   Default scope for provisioned Maskinporten clients, if none specified in spec. (default "nav:test/api")
--digdir.maskinporten.default.scope-prefix string   Default scope prefix for provisioned Maskinporten scopes. (default "nav")
--digdir.maskinporten.kms.key-path string           Maskinporten Google KmsConfig resource path used to sign JWT assertion when authenticating to DigDir.
--digdir.maskinporten.scopes string                 List of scopes for JWT assertion when authenticating to DigDir with Maskinporten.
--digdir.maskinporten.well-known-url string         URL to Maskinporten well-known discovery metadata document.
--features.maskinporten                             Feature toggle for maskinporten
--leader-election.enabled                           Toggle for enabling leader election. (default "false")
--leader-election.namespace string                  Namespace for the leader election resource. Needed if not running in-cluster (e.g. locally). If empty, will default to the same namespace as the running application. (default "")
--metrics-address string                            The address the metric endpoint binds to. (default ":8080")

At minimum, the following configuration must be provided:

• cluster-name
• digdir.admin.base-url
• digdir.idporten.cert-chain
• digdir.idporten.client-id
• digdir.idporten.kms.key-path
• digdir.idporten.scopes
• digdir.idporten.well-known-url
• digdir.maskinporten.cert-chain
• digdir.maskinporten.client-id
• digdir.maskinporten.kms.key-path
• digdir.maskinporten.scopes
• digdir.maskinporten.well-known-url

Equivalently, one can specify these properties using JSON, TOML, YAML, HCL, envfile and Java properties config files. Digdirator looks for a file named digdirator.<ext> in the directories [., /etc/].

Example configuration in YAML:

# ./digdirator.yaml

cluster-name: local
development-mode: true
features:
  maskinporten: true
digdir:
  admin:
    base-url: "https://api.test.samarbeid.digdir.no"
  idporten:
    client-id: "projects/<project>/secrets/<idporten-client-id>/versions/<version>"
    cert-chain: "projects/<project>/secrets/<idporten-cert-chain>/versions/<version>"
    kms:
      key-path: "projects/<project>/locations/<location>/keyRings/<keyring>/cryptoKeys/<key>/cryptoKeyVersions/<version>"
    scopes: "idporten:dcr.write idporten:dcr.read"
    well-known-url: "https://test.idporten.no/idporten-oidc-provider/.well-known/openid-configuration"
  maskinporten:
    client-id: "projects/<project>/secrets/<maskinporten-client-id>/versions/<version>"
    cert-chain: "projects/<project>/secrets/<maskinporten-cert-chain>/versions/<version>"
    kms:
      key-path: "projects/<project>/locations/<location>/keyRings/<keyring>/cryptoKeys/<key>/cryptoKeyVersions/<version>"
    scopes: "idporten:dcr.write idporten:dcr.read idporten:scopes.write"
    well-known-url: "https://test.maskinporten.no/.well-known/oauth-authorization-server"

Development

If you're running locally, make sure you have access to the GCP resources and that you're authenticated with Application Default Credentials:

gcloud auth login --update-adc

Then, assuming you have a Kubernetes cluster running locally (e.g. using minikube):

ulimit -n 4096  # for controller-gen
make run
make sample

Verifying the Digdirator image and its contents

The image is signed "keylessly" (is that a word?) using Sigstore cosign. To verify its authenticity run

cosign verify \
--certificate-identity "https://github.com/nais/digdirator/.github/workflows/build.yml@refs/heads/master" \
--certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
ghcr.io/nais/digdirator@sha256:<shasum>

The images are also attested with SBOMs in the CycloneDX format. You can verify these by running

cosign verify-attestation --type cyclonedx \
--certificate-identity "https://github.com/nais/digdirator/.github/workflows/build.yml@refs/heads/master" \
--certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
ghcr.io/nais/digdirator@sha256:<shasum>

digdirator's Issues

Improve key rotation

The key rotation procedures as of now aren't very fault tolerant. There are multiple problems:

1. The self-service API may be down or unavailable. If this happens, we should gracefully fall back to reusing existing keys - this obviously only applies for existing Clients with associated Secrets in the cluster.
2. The keys are eventually consistent within DigDir's systems, which means any new keys may be unrecognized for several minutes after registration. This should be remedied by having a "sliding window" of keys; any previous keys in use by pods, the newest (current) key, and the next key which should be generated and preregistered so that it is ready for the next rotation.
3. The Secret should be updated (without rotating the keys?) when the configuration (e.g. well-known URLs) have changed, or when the synchronizationHash is unset/cleared. Currently, we only update the Secret when spec.secretName is different than the synchronized secret name in the Status object.
4. The keys expire at DigDir exactly 1 year after registration, noted by the exp field in the response from DigDir's APIs. We should automatically trigger rotations when reconciling if the expiry is within a certain threshold, e.g. less than 1 month or so.

Remove dependency to Google Secret Manager

Digdirator loads some secrets from Google Secret Manager on startup, e.g. the client ID and public certificate chain for the x5c claim. We should rather provide these together with all the other configurations as part of the Helm feature/chart.

Edit:

This will be handled as part of a larger refactor of authenticating with the DigDir admin API.

Scope owner check

Scopes in Digdirator is owned by applications.

Today we assign a scope to the app id: cluster:team:app:<scope>

We should remove the validation of cluster: and only validate

team:app:<scope> this will make it easier for applikasjons to migrate to other clusters, and if we introduce multi-cluster an applikasjon dont need to worry about this.

Separate `Scope` resource for providers

Ref. b2652a5:

A client without any scopes is disallowed; ideally we shouldn't even
create a Maskinporten client if the spec only defines scopes to be
exposed and none consumed.

This commit adds a default consumer scope for the Maskinporten client if
none are defined. Ideally, the "exposed scope" directive should be split out
into its own resource and reconciler rather than being tied directly to a
MaskinportenClient resource.

Other pain points with tying exposure of scopes to a single MaskinportenClient:

• ownership becomes unclear:
  • multiple application manifests may define the same scope, but the source of truth is the last applied manifest
  • what happens if the same application in the same namespace, but in different clusters have the same exposed scope?
• tracking desired and actual state becomes difficult (e.g. how do we determine that a scope should be deleted?)

Delegering av scopes v/DigDir

Del 1 av delegering av tilganger for scopes. Se https://docs.digdir.no/docs/Maskinporten/maskinporten_func_delegering

Dette omhandler kun støtte for å skru på delegering av selve Maskinporten scopet. Oppsett hos Altinn er en separat oppgave.

Se også:

• https://nav-it.slack.com/archives/C5KUST8N6/p1708691964797019?thread_ts=1708329367.919129&channel=C5KUST8N6&message_ts=1708691964.797019
• https://nav-it.slack.com/archives/C013RTT99G9/p1712559933366389

Upgrade go-jose library

https://github.com/square/go-jose/ was deprecated in 2023

Swap to its successor: https://github.com/go-jose/go-jose

Digdirator trenger KMS tilgang for tenants

Digdirator trenger både KMS og Secret Manager tilgang

Disse 2 må lastes opp av en tenant i tillegg må vi få litt informasjonen fra tenant for å sette opp digdirator.

Nye URLer for API hos DigDIr

DigDir skal bytte ut selvbetjeningsapi-et sitt og bytter samtidig til nye URLer.

Vi må da bytte ut disse URLene hos oss.

dev:

• https://integrasjon-ver2.difi.no -> https://api.test.samarbeid.digdir.no/

prod:

• https://integrasjon.difi.no -> https://api.samarbeid.digdir.no/

https://docs.digdir.no/docs/idporten/oidc/oidc_api_admin.html#endepunkter

ACTION REQUIRED: Changes to pulling Chainguard Images

Hey there Chainguard here.

We noticed that you are using Chainguard Images, thank you! We wanted to make you aware of an upcoming change that will impact your project.

Starting August 16, 2023 public users will no longer be able to pull images from our registry (cgr.dev/chainguard) by tags other than latest or latest-dev. Please see the announcement for more information.

You are currently using the following.

In https://github.com/nais/digdirator/blob/0a50c69e871242657c19dde417cdb607adbd6692/.github/workflows/build.yml:

• cgr.dev/chainguard/go:1.20

In https://github.com/nais/digdirator/blob/0a50c69e871242657c19dde417cdb607adbd6692/Dockerfile:

• cgr.dev/chainguard/go:1.20

Our goal is to prevent your project from experiencing any disruptions. Please see the migration guide for options.

If there's more we can do to help please reply to this issue or email us at [email protected].

Thank you!

Exposed scope is never nil

The map returned by the method GetExposedScopes is never nil. The test should check len(exposedScopes) > 0

digdirator/controllers/common/reconciler.go

Lines 179 to 186 in 90ea819

	exposedScopes := instance.GetExposedScopes()
	scopes := r.scopes(tx)
	if exposedScopes != nil {
	if err := scopes.process(exposedScopes); err != nil {
	return fmt.Errorf("processing scopes: %w", err)
	}
	}

Forking for use at Domstolsadministrasjonen

Hi

We're looking into something similar to what you've created here and have some questions:

• Do you have any idea of how portable it is today? I see that I will have to rewrite from using Google to Vault.
• Have you implemented automatic recreation of expired keys? I don't see it in your interaction diagram

Best regards
Thor Åge Eldby