defenseunicorns / pepr Goto Github PK

An async self instantiating init capability in Pepr. (Similar to async IIFE without side effects)

Reasoning
Given you are using Pepr to create a module to replace the Zarf Agent, and a pertinent secret is attached to the pod as volume that must be read before anything else can happen,
When the webhook spins up,
Then it should have an init capability to read the secret contents and block any further processing until init capability finishes.

Running npx pepr build generates a zarf.yaml that has the Pepr image version hardcoded to v0.0.1

kind: ZarfPackageConfig
metadata:
  name: pepr-c8b24d7e-482b-5912-9177-0287a248ca83
  description: 'Pepr Module: uds pepr POC'
  url: https://github.com/defenseunicorns/pepr
  version: 0.0.1
components:
  - name: module
    required: true
    manifests:
      - name: module
        namespace: pepr-system
        files:
          - pepr-module-c8b24d7e-482b-5912-9177-0287a248ca83.yaml
    images:
      - ghcr.io/defenseunicorns/pepr/controller:v0.0.1

When using a.GenericKind Pepr assumes the plural for resources. We need logic to check if the kind.kind ends in y, and if so, add ies instead of s

Example of error:

in Capability

When(a.GenericKind, {
  group: "source.toolkit.fluxcd.io",
  version: "v1beta2",
  kind: "GitRepository",
})

Resulting MutatingWebhookConfiguration

  - apiGroups:
    - source.toolkit.fluxcd.io
    apiVersions:
    - v1beta2
    operations:
    - CREATE
    - UPDATE
    resources:
    - gitrepositorys

expectation

gitrepositories                   gitrepo            source.toolkit.fluxcd.io/v1beta2       true         GitRepository

No biggy, i can work around this and I will pick up this PR.

When using a tagged release of Pepr capability e.g v0.5.0 of Pepr-keycloak-authsvc (which references Pepr v0.9.0) and a newer different version of Pepr core e.g. 0.10.1, building the Pepr capability causes problems.

An example directory is here:
https://github.com/defenseunicorns/delivery-mvp-dubbd/tree/pepr-integration/mvp-pepr

Changing (by Renovate or Dependabot) the Pepr version and rebuilding the package causes the following problem to manifest:

$ npm ci
npm WARN deprecated [email protected]: this library is no longer supported
npm WARN deprecated [email protected]: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated [email protected]: request has been deprecated, see https://github.com/request/request/issues/3142

added 285 packages, and audited 286 packages in 2s

45 packages are looking for funding
  run `npm fund` for details

4 moderate severity vulnerabilities

Some issues need review, and may require choosing
a different dependency.

Run `npm audit` for details.
[kiran@kaibab pbme-pepr]$ npx pepr build
pepr.ts:11:22 - error TS2322: Type 'import("/home/kiran/work/git/zarf-big-bang-pbme/pbme-pepr/node_modules/@pepr/keycloak-authsvc/node_modules/pepr/dist/lib/capability").Capability' is not assignable to type 'import("/home/kiran/work/git/zarf-big-bang-pbme/pbme-pepr/node_modules/pepr/dist/lib/capability").Capability'.
  Types have separate declarations of a private property '_name'.

11 new PeprModule(cfg, [Keycloak, AuthService]);
                        ~~~~~~~~

pepr.ts:11:32 - error TS2322: Type 'import("/home/kiran/work/git/zarf-big-bang-pbme/pbme-pepr/node_modules/@pepr/keycloak-authsvc/node_modules/pepr/dist/lib/capability").Capability' is not assignable to type 'import("/home/kiran/work/git/zarf-big-bang-pbme/pbme-pepr/node_modules/pepr/dist/lib/capability").Capability'.

11 new PeprModule(cfg, [Keycloak, AuthService]);
                                  ~~~~~~~~~~~


Found 2 errors in the same file, starting at: pepr.ts:11

[error]		Command failed: ./node_modules/.bin/tsc

Currently the Secret data property is carried as Base64 encoded (unlike the GO SDK that does this for you), Pepr should have a wrapper for these functions, also should include an update to HelloPepr showing this:

Read from:

 if (data && data[p]) {
    return Buffer.from(data[p], "base64").toString("utf-8");
  }

Write back:
Buffer.from(secret).toString("base64");

Noticed a few small spelling errors when getting familiar with the product. I am creating a PR right now to address them if that is acceptable:

1. Change CongigMap to ConfigMap in actions.md
2. Change perp to pepr in module.md
3. Add space between Kubernetes mutating webhooks and are in README.md

implement as something that could be in a module (example). determine later if can go into the core.

Waiting for the next RC, but we need to upgrade to the 1.x version of the K8s client (currently under RC only) so that we can remove some findings for deprecated packages.

Related to kubernetes-client/javascript#754

1. pepr cli with any noun verb it doesn't know should display help. IE: pepr help
2. pepr dev should validate the version in the package.json is compatible. IE: an old version of the pepr cli should error if the package.json has a newer version
3. pepr should be able to import a CRD as a Kind. Best examples are the networking.istio.io objects, This should be part of the build/dev process. like pepr import crd virtualservices.networking.istio.io
4. pepr dev should make sure there aren't multiple mutatingwebhooks with the same endpoint name

• Load the mounted secret
• Handle decompression of the secret data
• Run with newly decompressed data

As a user of this repo I found it difficult to understand how I would implement more complex capabilities like the creation of resources. I believe if there were more examples of capabilities that included a wider scope of options it would be incredibly helpful as I work with Pepr.

With pepr 0.9 and a freshly npx pepr init hello-pepr.ts example plus the following When

When(a.Secret)
  .IsDeleted()
  .Then(request => {
    console.log('never gets here')
  });

Yields this error when a Secret is deleted:

[info]  c875659e-b14e-40c1-88ab-f25829c0b616 robinfo/configclient       Mutate request: /v1/Secret
TypeError: Cannot read properties of null (reading 'data')
    at convertFromBase64Map (/home/rob/du/pepr-bug/pepr-delete/node_modules/pepr/dist/lib.js:973:18)
    at processor (/home/rob/du/pepr-bug/pepr-delete/node_modules/pepr/dist/lib.js:1004:18)
    at mutate (/home/rob/du/pepr-bug/pepr-delete/node_modules/pepr/dist/lib.js:1160:30)
    at Layer.handle [as handle_request] (/home/rob/du/pepr-bug/pepr-delete/node_modules/express/lib/router/layer.js:95:5)
    at next (/home/rob/du/pepr-bug/pepr-delete/node_modules/express/lib/router/route.js:144:13)
    at Route.dispatch (/home/rob/du/pepr-bug/pepr-delete/node_modules/express/lib/router/route.js:114:3)
    at Layer.handle [as handle_request] (/home/rob/du/pepr-bug/pepr-delete/node_modules/express/lib/router/layer.js:95:5)
    at /home/rob/du/pepr-bug/pepr-delete/node_modules/express/lib/router/index.js:284:15
    at param (/home/rob/du/pepr-bug/pepr-delete/node_modules/express/lib/router/index.js:365:14)
    at param (/home/rob/du/pepr-bug/pepr-delete/node_modules/express/lib/router/index.js:376:14)
[2023-06-22T19:19:04.985Z] POST /mutate/ede9ce300a0d3e766b91d0d26075a7f2cbcca1b0f87f237074f4a51e82261e20?timeout=30s [500] 1 ms

Let's say I want to update the template for a replicaset:

When(a.ReplicaSet)
  .IsCreated()
  .Then(async req => {
    req.Raw.spec.template.metadata.annotations["spanny"] = "test";
  });

Kubernetes will keep creating new replicasets. The use case for this is to have the same flow for daemonsets, statefulsets, and deployments

Installed pepr globally with npm -g [email protected] and am seeing the following error:

[rob@lia pepr (main)]$ pepr init
node:internal/modules/cjs/loader:1056
  throw err;
  ^

Error: Cannot find module 'package.json'
Require stack:
- /opt/homebrew/lib/node_modules/pepr/dist/src/cli/build.js
- /opt/homebrew/lib/node_modules/pepr/dist/src/cli/index.js
- /opt/homebrew/lib/node_modules/pepr/dist/cli.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:1053:15)
    at Module._load (node:internal/modules/cjs/loader:898:27)
    at Module.require (node:internal/modules/cjs/loader:1120:19)
    at require (node:internal/modules/helpers:112:18)
    at Object.<anonymous> (/opt/homebrew/lib/node_modules/pepr/dist/src/cli/build.js:10:24)
    at Module._compile (node:internal/modules/cjs/loader:1239:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1293:10)
    at Module.load (node:internal/modules/cjs/loader:1096:32)
    at Module._load (node:internal/modules/cjs/loader:935:12)
    at Module.require (node:internal/modules/cjs/loader:1120:19) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/opt/homebrew/lib/node_modules/pepr/dist/src/cli/build.js',
    '/opt/homebrew/lib/node_modules/pepr/dist/src/cli/index.js',
    '/opt/homebrew/lib/node_modules/pepr/dist/cli.js'
  ]
}

Node.js v19.4.0

Need to choose a pattern for pepr setup/update that makes the most sense. Currently that's npm i -g pepr because updates for the CLI portion were cleaner, but once you have a Pepr Module, you'd have to separately run npm update in that project even if you updated the CLI globally. This might be confusing for some people, especially if less familiar with the oddities of NPM. On the other side, npx pepr should use the local version making that one less update, but the process to update pepr cli might be slightly more annoying.

Currently only mutating webhook configs are enabled, need to enable changing the MutateOrValidate flag and updating the k8s resources accordingly.

and allow both json and other standard formats.

Using Pepr 0.6.0 I noticed secrets were getting patched that were not part of any capability or module. Creating a junk file of a specific size and then storing it in a secret while a pepr module is running creates a secret that appears to have been mutated.

I suspect it is something to do with the encoding/decoding of the data and perhaps non UTF-8 encoded data?

$ dd if=/dev/urandom of=1000.k bs=1KB count=1
                                                                                                  
$ kubectl create secret generic test --from-file=foo=1000.k -n default
secret/test created

$ kubectl describe secret/test
Name:         test
Namespace:    default
Labels:       <none>
Annotations:  <none>

Type:  Opaque

Data
====
foo:  1822 bytes 

Notice the 1822 bytes, this will be 1000 bytes if a Pepr module is not running.

Now that the SDK is embedded/loaded by a Pepr Module (vs the other way around), we need to update the test flow to handle this--or possibly even removing it since pepr dev does a lot of this for you.

IE:

kubectl create ns pepr-demo
kubectl create configmap  example-1 -n pepr-demo

Currently, Pepr doesn't limit the ClusterRole or Webhook bindings to the current module. These should be limited if possible to not over-privilege the controller service account. We also need to evaluate how the webhook bindings work.

I'm not in a rush to get this working on windows native, but to how to:

1. get k3d running
2. figure out network routing
3. create commands for it.

Also add dev notes including Ava test details, #121 (review)

Given package.json has a version key set, I would like that version to be added to the zarf.yaml that pepr build generates.

Current behavior:
Version in package.json is ignored

Desired behavior:
Version is added as a metadata key in the generated zarf.yaml

As we've started to develop use cases for Pepr, it is becoming apparent that there are times when you need to share data between different Capabilities and CapabilityActions. After discussing several ways we might do this, @bburky @cmwylie19 @bdw617 @rjferguson21 and I landed on the general idea for persisting shared state within K8s and maintaining an automated watch system for changes. More thoughts and details are below:

Problem: we need to share data, and using K8s resource metadata exclusively is cumbersome and brittle. One option is to generate more K8s resources, which is more complex and slower and requires resource cleanup.

Idea: define a simple key-value store native to Pepr core and transparently available for users of the SDK. More details:

• Leverage a K8s CRD/CR (PeprStore) for pepr to maintain key-value state data per-capability
• Provide an API that generally matches the Web Storage API
• A watcher for all PeprStore resources (watchers are batch operations) that updates the internal state of the object for read
• A Store method available from Capabilities to bind data
• This can be exported for sharing data between Capabilities

// Use the 'When' function to create a new Capability Action, 'Store' to persist data
const { When, Store } = HelloPepr;

// Save some data to the store
Store.setItem("hello-pepr", "Hello Pepr!");

// Add an item to the store
Store.setItem("hello-pepr-2", "Hello Pepr! I'm a second item!");

// Read the data back from the store
console.log(Store.getItem("hello-pepr"));

// Update the data in the store
Store.setItem("hello-pepr", "Hello Pepr! I've been updated!");

// Get the length of the store
console.log(Store.length);

// Delete an item from the store
Store.removeItem("hello-pepr");

// Clear the store
Store.clear();

• Discuss write operations: batching, collisions, performance, failures, etc
• Discuss read/write contracts: i.e., don't return write until successful persistence, require force K8s resource get before returning a read

https://github.com/istio-ecosystem/authservice/blob/master/bookinfo-example/scripts/generate-self-signed-certs-for-ingress-gateway.sh

maybe not, look at cert-manager too?

IE: an old version of the pepr cli should error if the package.json has a newer version

Best examples are the networking.istio.io objects, This should be part of the build/dev process. like pepr import crd virtualservices.networking.istio.io

The image tag for pepr build and pepr deploy currently rely on a version entry that is generated via pepr init, but currently has no way to automatically update. This should be changed to reflect the actual version of Pepr used by the module so there is no version drift in prod.

if you create a kubernetes object in a When() you can't trigger on it in another When consistently since it might not be admitted yet. The idea of flowing between capabilities this way to create objects used further in is one of the biggest 🪄 things pepr should be allowed to do :)

example:
When(....)
.Then(async request => {
// create a new k8s secret with the kubernetes API
});
...
// This When will not find the previously created secret, because it's not persisted until it's fully admitted.
When(...)
.Then(async request => {
// read the above secret from the kubernetes API, we're not triggering on the same object from above.
});

Recommendations

• Add Pre-requisites to Read.me
• Add $ npm i -g pepr to Modules page
• Add information about validating webhooks - difference between mutating vs validating webhooks
• Add Troubleshooting section
  • Note the npm-ism (with versions) Add that npx can be used to easily keep npm installed link to external docs)
• Tutorial of how to use chatGPT to help you use paper (Internal blogin)
• Rename links from “Here” to a name describing the link to increase accessibility.
• Request for a common set of core Pepr modules, Delivery can consume common modules and modify not start from scratch
  • Survey top 5 common things that are complicated in delivery

• Add spell-check
• Add npm audit or other security-relevant scans
• Re-validated protected branch rules

Complete implementation of alwaysIgnore in the zarf section of generated package.json.

Reasoning
Given you are using Pepr to create a module to replace the Zarf Agent,
When a request comes to the admission controller with labels zarf.dev/agent: ignore or zarf-agent: patched,
Then the hook should ignore the request.

Links to Zarf code:

• Namespace ignore
• Pod Ignore
• Patched example

  "pepr": {
    "name": "zarf-agent",
    "uuid": "cb9dbea4-d6c1-5b60-b4f1-758ed8843298",
    "onError": "audit",
    "alwaysIgnore": {
      "namespaces": [],
      "labels": ["zarf.dev/agent: ignore"]
    }
  },

For the Pepr Zarf Agent in the Argo vertical there is an application secret that contains repo credentials that is a target for a mutation.

└─[0] <git:(argo 57060c1✱) > k get secret -n argocd argocd-repo-github-podinfo  -oyaml  
apiVersion: v1
data:
  name: cG9kaW5mbw==
  password: ""
  url: aHR0cHM6Ly9naXRodWIuY29tL3N0ZWZhbnByb2Rhbi9wb2RpbmZvLmdpdA==
  username: ""
kind: Secret
metadata:
  annotations:
    meta.helm.sh/release-name: argocd-baseline
    meta.helm.sh/release-namespace: argocd
  creationTimestamp: "2023-08-03T12:01:33Z"
  labels:
    app.kubernetes.io/instance: argocd-baseline
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/part-of: argocd
    app.kubernetes.io/version: v2.7.7
    argocd.argoproj.io/secret-type: repository
    helm.sh/chart: argo-cd-5.38.1
  name: argocd-repo-github-podinfo
  namespace: argocd
  resourceVersion: "2721"
  uid: 0bc24a4c-6b3b-49c3-98d6-85461942d03e
type: Opaque

When debugging I see this error which results in failing to watch this secret

[info]  9b60fe92-e214-4673-8afe-bdcd0145f827 argocd/argocd-repo-github-podinfo       Mutate request: /v1/Secret
TypeError [ERR_INVALID_ARG_TYPE]: The first argument must be of type string or an instance of Buffer, ArrayBuffer, or Array or an Array-like Object. Received null
    at new NodeError (node:internal/errors:405:5)
    at Function.from (node:buffer:333:9)
    at base64Decode (/Users/cmwylie19/pepr-zarf-agent/node_modules/pepr/dist/lib.js:986:17)
    at convertFromBase64Map (/Users/cmwylie19/pepr-zarf-agent/node_modules/pepr/dist/lib.js:975:21)
    at processor (/Users/cmwylie19/pepr-zarf-agent/node_modules/pepr/dist/lib.js:1004:18)
    at mutate (/Users/cmwylie19/pepr-zarf-agent/node_modules/pepr/dist/lib.js:1146:30)
    at Layer.handle [as handle_request] (/Users/cmwylie19/pepr-zarf-agent/node_modules/express/lib/router/layer.js:95:5)
    at next (/Users/cmwylie19/pepr-zarf-agent/node_modules/express/lib/router/route.js:144:13)
    at Route.dispatch (/Users/cmwylie19/pepr-zarf-agent/node_modules/express/lib/router/route.js:114:3)
    at Layer.handle [as handle_request] (/Users/cmwylie19/pepr-zarf-agent/node_modules/express/lib/router/layer.js:95:5) {
  code: 'ERR_INVALID_ARG_TYPE'
}
[2023-08-03T12:01:33.370Z] POST /mutate?timeout=30s [500] 3 

Despite having a code in place to do so:

When(a.Secret)
  .IsCreatedOrUpdated()
  .InNamespace("argocd")
  // .WithLabel("argocd.argoproj.io/secret-type","repository")
  .Then(secret => {
     Log.info("argocd-repo-github-podinfo ",secret.Raw?.metadata?.name)
    if (argoSecretLabels(secret)){
      try {

        secret.Raw = JSON.parse(
          _transformer.transformArgoSecret(
            secret.Raw,
            secret.Request,
            _initSecrets.zarfStateSecret.gitServer.address,
            _initSecrets.zarfStateSecret.gitServer.pushUsername,
            _initSecrets.zarfStateSecret.gitServer.pullPassword,
            _initSecrets.zarfStateSecret.gitServer.pullUsername

          )
        )
        console.log("secret", JSON.stringify(secret.Raw, undefined, 2));
      } catch (err) {
        Log.error("Error transforming argo secret", err)
      }
    }
  })

What is the recommended work around for this? Should I handle this secret outside of Pepr for now?

assume it's due to the format of the admissionreview. A workaround would be to get the object directly, not hard, but a bit inconsistent. Maybe there needs to be a PeprDeleteRequest<> that just doesn't have Raw?

Because pepr build -e is a special usecase not intended for use as a replacement for pepr.ts building, the output filename should not use the format pepr-<uuid>.js and instead should simply preserve the base filename of the entrypoint file. For Pepr Exported Capabilities this will be important and avoid the need to run mv dist/pepr*.js dist/index.js after a pepr build.

Issue to track submittal to CNCF. Process is outlined here and here.

Application Backlog: https://github.com/orgs/cncf/projects/14/views/1

• Basic form fields in the application
• Define roadmap @jeff-mccoy
• Contributing Guide @jeff-mccoy
• Why CNCF @jeff-mccoy
• Benefits to landscape @bdw617
• Cloud native questions
• Similar projects @bdw617
• Product / service seperation
• Other info

Target Q3 '24

As we work through creating/publishing Pepr modules it would be good to have some explanation of the process for packaging a module for use:

Some questions I don't know the answer to:

• Are we expecting modules to be published as Zarf packages, or NPM packages, or both?
• Do we have recommendations around when a Pepr module is deployed to a cluster? I assume it needs to be one of the first things but it would be good to provide example steps for the order of operations that aligns with the intended architecture of Pepr.

Relates to defenseunicorns/pepr-keycloak-authsvc#28