GithubHelp home page GithubHelp logo

crd2pulumi's Introduction

crd2pulumi

Generate typed CustomResources based on Kubernetes CustomResourceDefinitions.

Goals

crd2pulumi is a CLI tool that generates typed CustomResources based on Kubernetes CustomResourceDefinition (CRDs). CRDs allow you to extend the Kubernetes API by defining your own schemas for custom objects. While Pulumi lets you create CustomResources, there was previously no strong-typing for these objects since every schema was, well, custom. This can be a massive headache for popular CRDs such as cert-manager or istio, which contain thousands of lines of complex YAML schemas. By generating typed versions of CustomResources, crd2pulumi makes filling out their arguments more convenient by allowing you to leverage existing IDE type checking and autocomplete features.

Building and Installation

If you wish to use crd2pulumi without developing the tool itself, you can use one of the binary releases hosted on this repository.

Homebrew

crd2pulumi can be installed on Mac from the Pulumi Homebrew tap.

brew install pulumi/tap/crd2pulumi

crd2pulumi uses Go modules to manage dependencies. If you want to develop crd2pulumi itself, you'll need to have Go installed in order to build. Once you install this prerequisite, run the following to build the crd2pulumi binary and install it into $GOPATH/bin:

$ go build -ldflags="-X github.com/pulumi/crd2pulumi/gen.Version=dev" -o $GOPATH/bin/crd2pulumi main.go

The ldflags argument is necessary to dynamically set the crd2pulumi version at build time. However, the version itself can be anything, so you don't have to set it to dev.

Go should then automatically handle pulling the dependencies for you. If $GOPATH/bin is not on your path, you may want to move the crd2pulumi binary from $GOPATH/bin into a directory that is on your path.

Usage

crd2pulumi is a CLI tool that generates typed Kubernetes
CustomResources to use in Pulumi programs, based on a
CustomResourceDefinition YAML schema.

Usage:
  crd2pulumi [-dgnp] [--nodejsPath path] [--pythonPath path] [--dotnetPath path] [--goPath path] [--javaPath path] <crd1.yaml> [crd2.yaml ...] [flags]
  crd2pulumi [command]

Examples:
crd2pulumi --nodejs crontabs.yaml
crd2pulumi -dgnp crd-certificates.yaml crd-issuers.yaml crd-challenges.yaml
crd2pulumi --pythonPath=crds/python/istio --nodejsPath=crds/nodejs/istio crd-all.gen.yaml crd-mixer.yaml crd-operator.yaml
crd2pulumi --pythonPath=crds/python/gke https://raw.githubusercontent.com/GoogleCloudPlatform/gke-managed-certs/master/deploy/managedcertificates-crd.yaml

Notice that by just setting a language-specific output path (--pythonPath, --nodejsPath, etc) the code will
still get generated, so setting -p, -n, etc becomes unnecessary.


Available Commands:
  help        Help about any command
  version     Print the version number of crd2pulumi

Flags:
  -d, --dotnet              generate .NET
      --dotnetName string   name of .NET package (default "crds")
      --dotnetPath string   optional .NET output dir
  -f, --force               overwrite existing files
  -g, --go                  generate Go
      --goName string       name of Go package (default "crds")
      --goPath string       optional Go output dir
  -h, --help                help for crd2pulumi
  -j, --java                generate Java
      --javaName string     name of Java package (default "crds")
      --javaPath string     optional Java output dir
  -n, --nodejs              generate NodeJS
      --nodejsName string   name of NodeJS package (default "crds")
      --nodejsPath string   optional NodeJS output dir
  -p, --python              generate Python
      --pythonName string   name of Python package (default "crds")
      --pythonPath string   optional Python output dir

Use "crd2pulumi [command] --help" for more information about a command.

Setting only a language-specific flag will output the generated code in the default directory; so -d will output to crds/dotnet, -g will output to crds/go, -j will output to crds/java, -n will output to crds/nodejs, and -p will output to crds/python. You can also specify a language-specific path (--pythonPath, --nodejsPath, etc) to control where the code will be outputted, in which case setting -p, -n, etc becomes unnecessary.

Examples

Let's use the example CronTab CRD specified in resourcedefinition.yaml from the Kubernetes Documentation.

TypeScript

To generate a strongly-typed CronTab CustomResource in TypeScript, we can run this command:

$ crd2pulumi --nodejsPath ./crontabs resourcedefinition.yaml

Now let's import the generated code into a Pulumi program that provisions the CRD and creates an instance of it.

import * as crontabs from "./crontabs"
import * as pulumi from "@pulumi/pulumi"
import * as k8s from "@pulumi/kubernetes";

// Register the CronTab CRD.
const cronTabDefinition = new k8s.yaml.ConfigFile("my-crontab-definition", { file: "resourcedefinition.yaml" });

// Instantiate a CronTab resource.
const myCronTab = new crontabs.stable.v1.CronTab("my-new-cron-object",
{
    metadata: {
        name: "my-new-cron-object",
    },
    spec: {
        cronSpec: "* * * * */5",
        image: "my-awesome-cron-image",
    }
})

As you can see, the CronTab object is typed! For example, if you try to set cronSpec to a non-string or add an extra field, your IDE should immediately warn you.

Python

$ crd2pulumi --pythonPath ./crontabs resourcedefinition.yaml
import pulumi_kubernetes as k8s
import crontabs.pulumi_crds as crontabs


# Register the CronTab CRD.
crontab_definition = k8s.yaml.ConfigFile("my-crontab-definition", file="resourcedefinition.yaml")

# Instantiate a CronTab resource.
crontab_instance = crontabs.stable.v1.CronTab(
    "my-new-cron-object",
    metadata=k8s.meta.v1.ObjectMetaArgs(
        name="my-new-cron-object"
    ),
    spec=crontabs.stable.v1.CronTabSpecArgs(
        cron_spec="* * * */5",
        image="my-awesome-cron-image",
    )
)

Go

$ crd2pulumi --goPath ./crontabs resourcedefinition.yaml

Now we can access the NewCronTab() constructor. Create a main.go file with the following code. In this example, the Pulumi project's module is named crds-go-final, so the import path is crds-go-final/crontabs/stable/v1. Make sure to swap this out with your own module's name.

package main

import (
	crontabs_v1 "crds-go-final/crontabs/stable/v1"

	meta_v1 "github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/meta/v1"
	"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
	pulumi.Run(func(ctx *pulumi.Context) error {
    // Register the CronTab CRD.
    _, err := yaml.NewConfigFile(ctx, "my-crontab-definition",
      &yaml.ConfigFileArgs{
        File: "resourcedefinition.yaml",
      },
    )
    if err != nil {
      return err
    }

		// Instantiate a CronTab resource.
		_, err := crontabs_v1.NewCronTab(ctx, "cronTabInstance", &crontabs_v1.CronTabArgs{
			Metadata: &meta_v1.ObjectMetaArgs{
				Name: pulumi.String("my-new-cron-object"),
			},
			Spec: crontabs_v1.CronTabSpecArgs{
				CronSpec: pulumi.String("* * * * */5"),
				Image:    pulumi.String("my-awesome-cron-image"),
				Replicas: pulumi.IntPtr(3),
			},
		})
		if err != nil {
			return err
		}

		return nil
	})
}

C#

$ crd2pulumi --dotnetPath ./crontabs resourcedefinition.yaml
using Pulumi;
using Pulumi.Kubernetes.Yaml;
using Pulumi.Kubernetes.Types.Inputs.Meta.V1;

class MyStack : Stack
{
    public MyStack()
    {
    // Register a CronTab CRD.
    var cronTabDefinition = new Pulumi.Kubernetes.Yaml.ConfigFile("my-crontab-definition",
        new ConfigFileArgs{
            File = "resourcedefinition.yaml"
        }
    );

    // Instantiate a CronTab resource.
    var cronTabInstance = new Pulumi.Crds.Stable.V1.CronTab("cronTabInstance",
        new Pulumi.Kubernetes.Types.Inputs.Stable.V1.CronTabArgs{
            Metadata = new ObjectMetaArgs{
                Name = "my-new-cron-object"
            },
            Spec = new Pulumi.Kubernetes.Types.Inputs.Stable.V1.CronTabSpecArgs{
                CronSpec = "* * * * */5",
                Image = "my-awesome-cron-image"
            }
        });    
    }
}

If you get an Duplicate 'global::System.Runtime.Versioning.TargetFrameworkAttribute' attribute error when trying to run pulumi up, then try deleting the crontabs/bin and crontabs/obj folders.

Java

$ crd2pulumi --javaPath ./crontabs resourcedefinition.yaml
package com.example;

import com.pulumi.Pulumi;

public class MyStack {

    public static void main(String[] args) {
        Pulumi.run(ctx -> {
            // Register a CronTab CRD (Coming Soon - see https://www.pulumi.com/registry/packages/kubernetes/api-docs/yaml/configfile/)

            // Instantiate a CronTab resource.
            var cronTabInstance = new com.pulumi.crds.stable.v1.CronTab("cronTabInstance",
                    com.pulumi.crds.stable.v1.CronTabArgs.builder()
                            .metadata(com.pulumi.kubernetes.meta.v1.inputs.ObjectMetaArgs.builder()
                                    .name("my-new-cron-object")
                                    .build())
                            .spec(com.pulumi.kubernetes.stable.v1.inputs.CronTabSpecArgs.builder()
                                    .cronSpec("* * * * */5")
                                    .image("my-awesome-cron-image")
                                    .build())
                            .build());
        });
    }
}

Now let's run the program and perform the update.

$ pulumi up
Previewing update (dev):
  Type                                                      Name                Plan
  pulumi:pulumi:Stack                                       examples-dev
 +   ├─ kubernetes:stable.example.com:CronTab                   my-new-cron-object  create
 +   └─ kubernetes:apiextensions.k8s.io:CustomResourceDefinition  my-crontab-definition  create
Resources:
  + 2 to create
  1 unchanged
Do you want to perform this update? yes
Updating (dev):
  Type                                                      Name                Status
  pulumi:pulumi:Stack                                       examples-dev
 +   ├─ kubernetes:stable.example.com:CronTab                   my-new-cron-object  created
 +   └─ kubernetes:apiextensions.k8s.io:CustomResourceDefinition  my-crontab-definition  created
Outputs:
  urn: "urn:pulumi:dev::examples::kubernetes:stable.example.com/v1:CronTab::my-new-cron-object"
Resources:
  + 2 created
  1 unchanged
Duration: 17s
Permalink: https://app.pulumi.com/albert-zhong/examples/dev/updates/4

It looks like both the CronTab definition and instance were both created! Finally, let's verify that they were created by manually viewing the raw YAML data:

$ kubectl get ct -o yaml
- apiVersion: stable.example.com/v1
  kind: CronTab
  metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"stable.example.com/v1","kind":"CronTab","metadata":{"labels":{"app.kubernetes.io/managed-by":"pulumi"},"name":"my-new-cron-object"},"spec":{"cronSpec":"* * * * */5","image":"my-awesome-cron-image"}}
  creationTimestamp: "2020-08-10T09:50:38Z"
  generation: 1
  labels:
    app.kubernetes.io/managed-by: pulumi
  name: my-new-cron-object
  namespace: default
  resourceVersion: "1658962"
  selfLink: /apis/stable.example.com/v1/namespaces/default/crontabs/my-new-cron-object
  uid: 5e2c56a2-7332-49cf-b0fc-211a0892c3d5
  spec:
  cronSpec: '* * * * */5'
  image: my-awesome-cron-image
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

crd2pulumi's People

Contributors

akrantz01 avatar albert-zhong avatar blampe avatar dependabot[bot] avatar gregnr avatar hagaibarel avatar jlyshoel avatar joeduffy avatar lblackstone avatar lwj avatar madsbacha avatar mattolenik avatar mjeffryes avatar pgavlin avatar rquitales avatar shamil avatar stack72 avatar t0yv0 avatar viveklak avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

crd2pulumi's Issues

[Golang] AWS LB Controller TargetGroupBinding generates empty struct for TargetGroupBindingSpecServiceRefArgs

Essentially, it seems that when generating types from the AWS LB controller's TargetGroupBinding CRD, at least one struct is missing fields (TargetGroupBindingSpecServiceRefArgs).

Expected behavior

I'd expect it to have a Port field:

              serviceRef:
                description: serviceRef is a reference to a Kubernetes Service and
                  ServicePort.
                properties:
                  name:
                    description: Name is the name of the Service.
                    type: string
                  port:
                    anyOf:
                    - type: integer
                    - type: string
                    description: Port is the port of the ServicePort.
                    x-kubernetes-int-or-string: true
                required:
                - name
                - port
                type: object

Current behavior

no fields at all:

type TargetGroupBindingSpecServiceRefPortArgs struct {
}

Steps to reproduce

  1. have AWS LB Controller installed in your Kube cluster
  2. kubectl get crd targetgroupbindings.elbv2.k8s.aws -o yaml > tgb-crd.yml
  3. crd2pulumi --goPath tgb-crd.yml

Context (Environment)

Go 1.16
WSL2 Ubuntu 20.04

Affected feature

?

[nodejs] Type for generated metadata field incorrect

Expected behavior

Every Kubernetes object has a metadata field. When generating the NodeJS code for a CRD, I expect the type of the metadata field to be similar to the type of the core Kubernetes classes. E.g. I copied this from kubernetes.core.v1.Service:

    public readonly metadata!: pulumi.Output<outputs.meta.v1.ObjectMeta>;

Current behavior

The metadata field in my generated is this:

    public readonly metadata!: pulumi.Output<ObjectMeta | undefined>;

Since the metadata field always exists on a Kubernetes object, the | undefined part is not correct.

Steps to reproduce

  1. Take a GKE cluster.
  2. kubectl get crd backendconfigs.cloud.google.com -o yaml > backendconfigs.yaml
  3. crd2pulumi --nodejsName backendconfig --nodejsPath ./backendconfig backendconfigs.yaml
  4. Create an instance: const cfg = new backendconfig.cloud.v1.BackendConfig(...)
  5. Try to retrieve the name: cfg.metadata.name
  6. You get an error: Property 'name' does not exist on type 'Output<ObjectMeta | undefined>'.

Context (Environment)

I had to remove the | undefined part manually from the generated field. A lot more of the generated fields have the same | undefined section in the type definition, so there might be problems there as well.

When following the code path, I ended up here in the core Pulumi codegen packages:

https://github.com/pulumi/pulumi/blob/1f16423ede4c9b6266f2df5aa4ef2aa8c79ae54f/pkg/codegen/nodejs/gen.go#L263-L265

Affected feature

[TypeScript/Python] Missing definitions in generated code

Hello!

  • Vote on this issue by adding a 👍 reaction
  • To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already)

Issue details

AWS EKS has a CRD to apply security groups on pods. Using crd2pulumi (1.0.10) to create a TypeScript package, it seems some/all properties are missing from the generated package.

Steps to reproduce

  1. Launch an EKS cluster (1.21)
  2. Retrieve the CRD (or use the attached archive): kubectl get crd securitygrouppolicies.vpcresources.k8s.aws -oyaml > securitygrouppolicies.crd.yaml
  3. Generate the package: crd2pulumi --nodejs --nodejsName eks-securitygroup-policies --nodejsPath eks-crd/eks-securitygroup-policies securitygrouppolicies.crd.yaml
  4. Properties such as podSelector, securityGroups and serviceAccountSelector are all missing from the generated package

Expected: The generated package contains the same properties as declared in the YAML file
Actual: Properties mentioned above are missing

Attachments

securitygrouppolicies.crd.zip

Set up CI to run tests

Hello!

  • Vote on this issue by adding a 👍 reaction
  • If you want to implement this feature, comment to let us know (we'll work with you on design, scheduling, etc.)

Issue details

The make test target runs a few tests that should also be connected to the CI PR and release actions.

Affected area/feature

Outdated module dependencies

When generating a python component with this tool it generates a setup.py file that sets pulumi >2.0.0,<3.0.0 as requirement.

setup(name='pulumi_crds',
      version='${VERSION}',
      long_description=readme(),
      long_description_content_type='text/markdown',
      cmdclass={
          'install': InstallPluginCommand,
      },
      packages=find_packages(),
      package_data={
          'pulumi_crds': [
              'py.typed'
          ]
      },
      install_requires=[
          'parver>=0.2.1',
          'pulumi>=2.0.0,<3.0.0',
          'pyyaml>=5.1,<5.2',
          'requests>=2.21.0,<2.22.0',
          'semver>=2.8.1'
      ],
      zip_safe=False)

Expected behavior

Since version 3 is now the lastest version that should be set as requirement instead.

Steps to reproduce

  1. Take any crd
  2. Generate python code from it

e.g.

curl -sL https://github.com/jetstack/cert-manager/releases/latest/download/cert-manager.crds.yaml -o cert-manager-crds.yml
crd2pulumi --pythonPath crds/python/cert-manager cert-manager-crds.yml

Context

I'm trying to set up a repo that regularly builds commonly used crds like the cert-manager ones into pulumi components and makes them available for any language, since this would improve everyone's pulumi experience with kubernetes a lot imo.
But since version 3 is out now this would not be worth the effort to set up right now if the generated code is only for versions < 3.
Maybe it would also be an option to just generate go code for example in this repo and then use the new multi-language component feature to provide them for other languages. Not sure whether this improves the maintainability tho.

Can not gennerate usable AuthorizationPolicy for istio 1.9.0

Generating typed crds for istio 1.9.0 in .NET results in a unusable AuthorizationPolicy class

Expected behavior

crd2pulumi generates a correct AuthorizationPolicy and AuthorizationPolicySpecArgs class

Current behavior

AuthorizationPolicy.cs is created where the Spec property type is "object" and not a AuthorizationPolicySpecArgs. No class for Spec is created

Steps to reproduce

  1. Download the istio 1.9.0 crds and run crd2pulumi on them

Duplicate crd packages

Context

Hey, great work on crd2pulumi. One issue I just hit on trying to pulumi up after I imported my second crd2pulumi converted library was that both node packages had the default nodeName of "crd".

This resulted in an "incorrect semver" error because it tried to compare the version of the first library (which was an empty string by default) and the newly converted library (which also was an empty string) and node semver choked. Giving them both unique nodeName parameters cleared it up.

Possible solutions

  • Make at least 1 language target required. So users of crd2pulumi must specify nodejs and nodejsPath and nodeName.
  • attach a guid to the default "crd"

License

Can the license for crd2pulumi be clarified and a License file created please

Can't generate code for https://github.com/kubernetes-sigs/gateway-api/tree/v0.3.0/config/crd/bases

Hello!

  • Vote on this issue by adding a 👍 reaction
  • To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already)

Issue details

In https://github.com/kubernetes-sigs/gateway-api/tree/v0.3.0/config/crd/bases there are files the crd2pulumi can't convert:

$ crd2pulumi --pythonPath gateway_api networking.x-k8s.io_gatewayclasses.yaml --force
panic: fatal: error An assertion has failed: could not parse Pulumi package. source error: could not import spec: binding types: failed to bind type kubernetes:networking.x-k8s.io/v1alpha1:GatewayClassStatus: error binding default value for property "conditions": default values may only be provided for boolean, integer, number, and string properties

goroutine 1 [running]:
github.com/pulumi/pulumi/sdk/v3/go/common/util/contract.failfast(...)
	/home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/[email protected]/go/common/util/contract/failfast.go:23
github.com/pulumi/pulumi/sdk/v3/go/common/util/contract.AssertNoErrorf(0x13892c0, 0xc000202048, 0x120b7b4, 0x1e, 0x0, 0x0, 0x0)
	/home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/[email protected]/go/common/util/contract/assert.go:47 +0x1cd
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).SchemaPackageWithObjectMetaType(0xc00058fa90, 0xc0002fa768)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/generate.go:201 +0xb6
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).genPythonFiles(0xc00058fa90, 0x11eb909, 0x4, 0xc00051aaf0, 0x1, 0x1)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/python.go:40 +0x45
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).genPython(0xc00058fa90, 0x7ffe7024c7ba, 0xb, 0x11eb909, 0x4, 0x1, 0xc00051ab30)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/python.go:31 +0x45
github.com/pulumi/crd2pulumi/gen.Generate(0x0, 0xc000422230, 0x0, 0x0, 0x11eb909, 0x4, 0x11eb909, 0x4, 0x11eb909, 0x4, ...)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/generate.go:68 +0x285
github.com/pulumi/crd2pulumi/cmd.Execute.func2(0xc0003d22c0, 0xc0002d7100, 0x1, 0x4)
	/home/runner/work/crd2pulumi/crd2pulumi/cmd/root.go:157 +0x271
github.com/spf13/cobra.(*Command).execute(0xc0003d22c0, 0xc00003a0b0, 0x4, 0x4, 0xc0003d22c0, 0xc00003a0b0)
	/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:846 +0x2c2
github.com/spf13/cobra.(*Command).ExecuteC(0xc0003d22c0, 0xc00058ff00, 0x1, 0x1)
	/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:950 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
	/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:887
github.com/pulumi/crd2pulumi/cmd.Execute(0xc00005c748, 0x1a3d320)
	/home/runner/work/crd2pulumi/crd2pulumi/cmd/root.go:189 +0x666
main.main()
	/home/runner/work/crd2pulumi/crd2pulumi/main.go:25 +0x26
$ crd2pulumi --pythonPath gateway_api networking.x-k8s.io_gateways.yaml --force
panic: fatal: error An assertion has failed: could not parse Pulumi package. source error: could not import spec: binding types: failed to bind type kubernetes:networking.x-k8s.io/v1alpha1:GatewaySpecListenersTls: error binding default value for property "routeOverride": default values may only be provided for boolean, integer, number, and string properties

goroutine 1 [running]:
github.com/pulumi/pulumi/sdk/v3/go/common/util/contract.failfast(...)
	/home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/[email protected]/go/common/util/contract/failfast.go:23
github.com/pulumi/pulumi/sdk/v3/go/common/util/contract.AssertNoErrorf(0x13892c0, 0xc000590030, 0x120b7b4, 0x1e, 0x0, 0x0, 0x0)
	/home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/[email protected]/go/common/util/contract/assert.go:47 +0x1cd
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).SchemaPackageWithObjectMetaType(0xc00058fa90, 0xc00055e0c0)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/generate.go:201 +0xb6
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).genPythonFiles(0xc00058fa90, 0x11eb909, 0x4, 0xc0000d3d90, 0x1, 0x1)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/python.go:40 +0x45
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).genPython(0xc00058fa90, 0x7fffb73167c0, 0xb, 0x11eb909, 0x4, 0x1, 0xc0000d3dd0)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/python.go:31 +0x45
github.com/pulumi/crd2pulumi/gen.Generate(0x0, 0xc00051a920, 0x0, 0x0, 0x11eb909, 0x4, 0x11eb909, 0x4, 0x11eb909, 0x4, ...)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/generate.go:68 +0x285
github.com/pulumi/crd2pulumi/cmd.Execute.func2(0xc000570b00, 0xc00020c600, 0x1, 0x4)
	/home/runner/work/crd2pulumi/crd2pulumi/cmd/root.go:157 +0x271
github.com/spf13/cobra.(*Command).execute(0xc000570b00, 0xc00003a0b0, 0x4, 0x4, 0xc000570b00, 0xc00003a0b0)
	/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:846 +0x2c2
github.com/spf13/cobra.(*Command).ExecuteC(0xc000570b00, 0xc00058ff00, 0x1, 0x1)
	/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:950 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
	/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:887
github.com/pulumi/crd2pulumi/cmd.Execute(0xc00005c748, 0x1a3d320)
	/home/runner/work/crd2pulumi/crd2pulumi/cmd/root.go:189 +0x666
main.main()
	/home/runner/work/crd2pulumi/crd2pulumi/main.go:25 +0x26
crd2pulumi --pythonPath gateway_api networking.x-k8s.io_httproutes.yaml --force
panic: fatal: error An assertion has failed: could not parse Pulumi package. source error: could not import spec: binding types: failed to bind type kubernetes:networking.x-k8s.io/v1alpha1:HTTPRouteSpec: error binding default value for property "gateways": default values may only be provided for boolean, integer, number, and string properties

goroutine 1 [running]:
github.com/pulumi/pulumi/sdk/v3/go/common/util/contract.failfast(...)
	/home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/[email protected]/go/common/util/contract/failfast.go:23
github.com/pulumi/pulumi/sdk/v3/go/common/util/contract.AssertNoErrorf(0x13892c0, 0xc000538090, 0x120b7b4, 0x1e, 0x0, 0x0, 0x0)
	/home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/[email protected]/go/common/util/contract/assert.go:47 +0x1cd
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).SchemaPackageWithObjectMetaType(0xc00062da90, 0xc0005380c0)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/generate.go:201 +0xb6
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).genPythonFiles(0xc00062da90, 0x11eb909, 0x4, 0xc000329190, 0x1, 0x1)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/python.go:40 +0x45
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).genPython(0xc00062da90, 0x7ffe7328d7be, 0xb, 0x11eb909, 0x4, 0x1, 0xc0003291d0)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/python.go:31 +0x45
github.com/pulumi/crd2pulumi/gen.Generate(0x0, 0xc0005086b0, 0x0, 0x0, 0x11eb909, 0x4, 0x11eb909, 0x4, 0x11eb909, 0x4, ...)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/generate.go:68 +0x285
github.com/pulumi/crd2pulumi/cmd.Execute.func2(0xc000554840, 0xc00051a940, 0x1, 0x4)
	/home/runner/work/crd2pulumi/crd2pulumi/cmd/root.go:157 +0x271
github.com/spf13/cobra.(*Command).execute(0xc000554840, 0xc00003a0b0, 0x4, 0x4, 0xc000554840, 0xc00003a0b0)
	/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:846 +0x2c2
github.com/spf13/cobra.(*Command).ExecuteC(0xc000554840, 0xc00058ff00, 0x1, 0x1)
	/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:950 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
	/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:887
github.com/pulumi/crd2pulumi/cmd.Execute(0xc00005c748, 0x1a3d320)
	/home/runner/work/crd2pulumi/crd2pulumi/cmd/root.go:189 +0x666
main.main()
	/home/runner/work/crd2pulumi/crd2pulumi/main.go:25 +0x26
$ crd2pulumi --pythonPath gateway_api networking.x-k8s.io_tcproutes.yaml --force
panic: fatal: error An assertion has failed: could not parse Pulumi package. source error: could not import spec: binding types: failed to bind type kubernetes:networking.x-k8s.io/v1alpha1:TCPRouteSpec: error binding default value for property "gateways": default values may only be provided for boolean, integer, number, and string properties

goroutine 1 [running]:
github.com/pulumi/pulumi/sdk/v3/go/common/util/contract.failfast(...)
	/home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/[email protected]/go/common/util/contract/failfast.go:23
github.com/pulumi/pulumi/sdk/v3/go/common/util/contract.AssertNoErrorf(0x13892c0, 0xc0002fc258, 0x120b7b4, 0x1e, 0x0, 0x0, 0x0)
	/home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/[email protected]/go/common/util/contract/assert.go:47 +0x1cd
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).SchemaPackageWithObjectMetaType(0xc00058fa90, 0xc000116090)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/generate.go:201 +0xb6
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).genPythonFiles(0xc00058fa90, 0x11eb909, 0x4, 0xc0000735d0, 0x1, 0x1)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/python.go:40 +0x45
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).genPython(0xc00058fa90, 0x7fffd3be77bf, 0xb, 0x11eb909, 0x4, 0x1, 0xc000073670)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/python.go:31 +0x45
github.com/pulumi/crd2pulumi/gen.Generate(0x0, 0xc0005046b0, 0x0, 0x0, 0x11eb909, 0x4, 0x11eb909, 0x4, 0x11eb909, 0x4, ...)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/generate.go:68 +0x285
github.com/pulumi/crd2pulumi/cmd.Execute.func2(0xc000544b00, 0xc000510680, 0x1, 0x4)
	/home/runner/work/crd2pulumi/crd2pulumi/cmd/root.go:157 +0x271
github.com/spf13/cobra.(*Command).execute(0xc000544b00, 0xc00003a0b0, 0x4, 0x4, 0xc000544b00, 0xc00003a0b0)
	/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:846 +0x2c2
github.com/spf13/cobra.(*Command).ExecuteC(0xc000544b00, 0xc00058ff00, 0x1, 0x1)
	/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:950 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
	/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:887
github.com/pulumi/crd2pulumi/cmd.Execute(0xc00005c748, 0x1a3d320)
	/home/runner/work/crd2pulumi/crd2pulumi/cmd/root.go:189 +0x666
main.main()
	/home/runner/work/crd2pulumi/crd2pulumi/main.go:25 +0x26
$ crd2pulumi --pythonPath gateway_api networking.x-k8s.io_tlsroutes.yaml --force
panic: fatal: error An assertion has failed: could not parse Pulumi package. source error: could not import spec: binding types: failed to bind type kubernetes:networking.x-k8s.io/v1alpha1:TLSRouteSpec: error binding default value for property "gateways": default values may only be provided for boolean, integer, number, and string properties

goroutine 1 [running]:
github.com/pulumi/pulumi/sdk/v3/go/common/util/contract.failfast(...)
	/home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/[email protected]/go/common/util/contract/failfast.go:23
github.com/pulumi/pulumi/sdk/v3/go/common/util/contract.AssertNoErrorf(0x13892c0, 0xc000560078, 0x120b7b4, 0x1e, 0x0, 0x0, 0x0)
	/home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/[email protected]/go/common/util/contract/assert.go:47 +0x1cd
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).SchemaPackageWithObjectMetaType(0xc00058fa90, 0xc0001160a8)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/generate.go:201 +0xb6
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).genPythonFiles(0xc00058fa90, 0x11eb909, 0x4, 0xc0002a3da0, 0x1, 0x1)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/python.go:40 +0x45
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).genPython(0xc00058fa90, 0x7fffe04b57bf, 0xb, 0x11eb909, 0x4, 0x1, 0xc0002a3ee0)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/python.go:31 +0x45
github.com/pulumi/crd2pulumi/gen.Generate(0x0, 0xc00011e2e0, 0x0, 0x0, 0x11eb909, 0x4, 0x11eb909, 0x4, 0x11eb909, 0x4, ...)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/generate.go:68 +0x285
github.com/pulumi/crd2pulumi/cmd.Execute.func2(0xc000570b00, 0xc0002d7300, 0x1, 0x4)
	/home/runner/work/crd2pulumi/crd2pulumi/cmd/root.go:157 +0x271
github.com/spf13/cobra.(*Command).execute(0xc000570b00, 0xc00003a0b0, 0x4, 0x4, 0xc000570b00, 0xc00003a0b0)
	/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:846 +0x2c2
github.com/spf13/cobra.(*Command).ExecuteC(0xc000570b00, 0xc00058ff00, 0x1, 0x1)
	/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:950 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
	/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:887
github.com/pulumi/crd2pulumi/cmd.Execute(0xc00005c748, 0x1a3d320)
	/home/runner/work/crd2pulumi/crd2pulumi/cmd/root.go:189 +0x666
main.main()
	/home/runner/work/crd2pulumi/crd2pulumi/main.go:25 +0x26
$ crd2pulumi --pythonPath gateway_api networking.x-k8s.io_udproutes.yaml --force
panic: fatal: error An assertion has failed: could not parse Pulumi package. source error: could not import spec: binding types: failed to bind type kubernetes:networking.x-k8s.io/v1alpha1:UDPRouteSpec: error binding default value for property "gateways": default values may only be provided for boolean, integer, number, and string properties

goroutine 1 [running]:
github.com/pulumi/pulumi/sdk/v3/go/common/util/contract.failfast(...)
	/home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/[email protected]/go/common/util/contract/failfast.go:23
github.com/pulumi/pulumi/sdk/v3/go/common/util/contract.AssertNoErrorf(0x13892c0, 0xc000116c30, 0x120b7b4, 0x1e, 0x0, 0x0, 0x0)
	/home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/[email protected]/go/common/util/contract/assert.go:47 +0x1cd
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).SchemaPackageWithObjectMetaType(0xc0004b1a90, 0xc000302288)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/generate.go:201 +0xb6
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).genPythonFiles(0xc0004b1a90, 0x11eb909, 0x4, 0xc000073760, 0x1, 0x1)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/python.go:40 +0x45
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).genPython(0xc0004b1a90, 0x7ffcf15e27bf, 0xb, 0x11eb909, 0x4, 0x1, 0xc000073880)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/python.go:31 +0x45
github.com/pulumi/crd2pulumi/gen.Generate(0x0, 0xc000506340, 0x0, 0x0, 0x11eb909, 0x4, 0x11eb909, 0x4, 0x11eb909, 0x4, ...)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/generate.go:68 +0x285
github.com/pulumi/crd2pulumi/cmd.Execute.func2(0xc00015ab00, 0xc000510540, 0x1, 0x4)
	/home/runner/work/crd2pulumi/crd2pulumi/cmd/root.go:157 +0x271
github.com/spf13/cobra.(*Command).execute(0xc00015ab00, 0xc00003a0b0, 0x4, 0x4, 0xc00015ab00, 0xc00003a0b0)
	/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:846 +0x2c2
github.com/spf13/cobra.(*Command).ExecuteC(0xc00015ab00, 0xc0004b1f00, 0x1, 0x1)
	/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:950 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
	/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:887
github.com/pulumi/crd2pulumi/cmd.Execute(0xc00005c748, 0x1a3d320)
	/home/runner/work/crd2pulumi/crd2pulumi/cmd/root.go:189 +0x666
main.main()
	/home/runner/work/crd2pulumi/crd2pulumi/main.go:25 +0x26

Generate CRD via URL

Is it possible to specify a url for the CRD file instead of having to download the file first from Github / Source Repository for the CRD?

Cheers.

TypeScript codegen does not adhere to eslint

Expected behavior

For generated TypeScript to be eslint compliant.

Current behavior

Currently the TypeScript generated is not eslint compliant.

To make it compliant one must add the following rules:

  • top-level/* eslint-disable @typescript-eslint/no-explicit-any */ to allow undefined as any to be used in the get() method generated
  • // eslint-disable-next-line to allow (obj: any) in the isInstance() method generated
  • Ensure formatting is done on all generated code, as spacing is off.

Unnecessary ObjectMeta that can instead use the k8s built-in types

Expected behavior

Don't generate metadata and instead use built-in k8s input types.

Current behavior

Currently the metadata of a CustomResource is using a locally generated ObjectMeta for metadata instead of using the built-in types in pulumi-kubernetes.

import {ObjectMeta} from "../../meta/v1";
...
public readonly metadata!: pulumi.Output<ObjectMeta | undefined>;

...

export interface FooArgs {
    ...
    readonly metadata?: pulumi.Input<ObjectMeta>;
}

Desired:

import { input as k8sInputs, output as k8sOutputs } from "@pulumi/kubernetes/types";
...
public readonly metadata!: pulumi.Output<k8sOutputs.meta.v1.ObjectMeta>;

...

export interface FooArgs {
    ...
    readonly metadata?: pulumi.Input<k8sInputs.meta.v1.ObjectMeta>;
}

Related:

  • #34 per the unnecessary | undefined in the typedef

Also closes:

CRD for Percona Server for MongoDB Operator generates invalid typescript:

Hello!

  • Vote on this issue by adding a 👍 reaction
  • To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already)

Issue details

I would like to generate typescript CRDs for Percona Server for MongoDB Operator.

Source CRD: https://github.com/percona/percona-server-mongodb-operator/blob/main/deploy/crd.yaml

Unfortunately, it generates invalid code - see generated psmdb\index.ts:

// *** WARNING: this file was generated by crd2pulumi. ***
// *** Do not edit by hand unless you're certain you know what you are doing! ***

// Export sub-modules:
import * as v1 from "./v1";
import * as v1-1-0 from "./v1-1-0";
import * as v1-10-0 from "./v1-10-0";
import * as v1-11-0 from "./v1-11-0";
import * as v1-2-0 from "./v1-2-0";
import * as v1-3-0 from "./v1-3-0";
import * as v1-4-0 from "./v1-4-0";
import * as v1-5-0 from "./v1-5-0";
import * as v1-6-0 from "./v1-6-0";
import * as v1-7-0 from "./v1-7-0";
import * as v1-8-0 from "./v1-8-0";
import * as v1-9-0 from "./v1-9-0";
import * as v1alpha1 from "./v1alpha1";

export {
    v1,
    v1-1-0,
    v1-10-0,
    v1-11-0,
    v1-2-0,
    v1-3-0,
    v1-4-0,
    v1-5-0,
    v1-6-0,
    v1-7-0,
    v1-8-0,
    v1-9-0,
    v1alpha1,
};

This leads to typescript error ts(2300) with the message Duplicate identifier 'v1'

Steps to reproduce

  1. Download https://github.com/percona/percona-server-mongodb-operator/blob/main/deploy/crd.yaml
  2. execute crd2pulumi against it
  3. See typescript error

Expected:

I would expect crd2pulumi to ensure naming is valid for imports:

// This: 
import * as v1-1-0 from "./v1-1-0";
// Should be this: (replacing - with _)
import * as v1_1_0 from "./v1-1-0";

Actual:
Produces invalid code.

Types for apiVersion, kind, metadata, status should not also accept undefined

Expected behavior

Typedefs for k8s should not also accept ... | undefined

Current behavior

Currently the generated code produces the following incorrect typedefs:

public readonly apiVersion!: pulumi.Output<"foo/v1" | undefined>;
public readonly kind!: pulumi.Output<"Foo" | undefined>;
public readonly metadata!: pulumi.Output<ObjectMeta | undefined>;
public readonly status!: pulumi.Output<outputs.foo.bar | undefined>;

These need to be corrected to:

public readonly apiVersion!: pulumi.Output<"foo/v1">;
public readonly kind!: pulumi.Output<"Foo">;
public readonly metadata!: pulumi.Output<ObjectMeta>;
public readonly status!: pulumi.Output<outputs.foo.bar>;

See example of a k8s Deployment in pulumi-kubernetes that follows suit with the proposed corrections.


Related:

Python CRD's are broken because of hard coded utilities.py

Hello!

  • Vote on this issue by adding a 👍 reaction
  • To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already)

Issue details

Attempting to generate CRD's for use in Python, and when I try to use them I get an AttributeError saying that the _utilities.py class has no lazy_import function.

      File "/home/jeid/work/pulumicrds/./crd/secrets_store/pulumi_crds/__init__.py", line 17, in <module>
        meta = _utilities.lazy_import('pulumi_crds.meta')
    AttributeError: module 'crd.secrets_store.pulumi_crds._utilities' has no attribute 'lazy_import'

https://github.com/pulumi/crd2pulumi/blob/master/gen/python.go#L94 seems to hardcode utilities.py and is missing lazy_import, which is defined in the upstream pulumi codegen code here https://github.com/pulumi/pulumi/blob/b3c56f62c82875d8e907bf4b55f0017b19f2be18/pkg/codegen/python/gen.go#L2602

crd2pulumi is broken on all Python projects as of 671bb76 merging this commit which bumped Pulumi up to v3.21.0, which now requires a bunch of missing utilities.py functions.

Steps to reproduce

  1. wget https://raw.githubusercontent.com/kubernetes-sigs/secrets-store-csi-driver/main/deploy/secrets-store.csi.x-k8s.io_secretproviderclasses.yaml
  2. crd2pulumi --pythonPath ./crd/secrets_store resources/secrets_manager/secrets-store-crd.yaml
  3. import crd.secrets_store.pulumi_crds.secretsstore.v1alpha1 as secrets_store

Downgrading to v1.0.10 of crd2pulumi is broken because of #50 and v1.1.0 is broken because of this issue.

cannot have a constant value; only booleans, integers, numbers and strings may have constant values

I'm trying to automate SDK generation for major Kubernetes resources, such as Istio, Crossplane, and others; but unfortunately I'm running into some problems.

Here's two examples of such behaviour:

crd2pulumi --nodejsName istio --nodejsPath ../output https://doc.crds.dev/raw/github.com/istio/[email protected]
crd2pulumi --nodejsName istio --nodejsPath ../output https://doc.crds.dev/raw/github.com/crossplane/[email protected]

and here's the stacktrace for the Istio attempt:

panic: fatal: error An assertion has failed: could not parse Pulumi package. source error: could not import spec: <nil>: #/types/kubernetes:networking.x-k8s.io%2Fv1alpha1:TLSRouteSpec/properties/gateways/default: type kubernetes:networking.x-k8s.io/v1alpha1:TLSRouteSpecGateways cannot have a constant value; only booleans, integers, numbers and strings may have constant values; , and 29 other diagnostic(s)

goroutine 1 [running]:
github.com/pulumi/pulumi/sdk/v3/go/common/util/contract.failfast(...)
	/home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/[email protected]/go/common/util/contract/failfast.go:23
github.com/pulumi/pulumi/sdk/v3/go/common/util/contract.AssertNoErrorf(0x2041820, 0xc0003509c0, 0x1eac169, 0x1e, 0x0, 0x0, 0x0)
	/home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/[email protected]/go/common/util/contract/assert.go:47 +0x1cd
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).SchemaPackage(0xc000c6ba90, 0xc000000180)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/generate.go:237 +0xb6
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).genNodeJSFiles(0xc000c6ba90, 0x7ff7bfeff8e0, 0x5, 0xc000ca6cb0, 0x1, 0x1)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/nodejs.go:39 +0x45
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).genNodeJS(0xc000c6ba90, 0x7ff7bfeff8f3, 0x9, 0x7ff7bfeff8e0, 0x5, 0x16, 0xc0005281c0)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/nodejs.go:30 +0x45
github.com/pulumi/crd2pulumi/gen.Generate(0xc000138850, 0x0, 0x0, 0x0, 0x7ff7bfeff8e0, 0x5, 0x1e8ad7e, 0x4, 0x1e8ad7e, 0x4, ...)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/generate.go:70 +0x2f7
github.com/pulumi/crd2pulumi/cmd.Execute.func2(0xc000334dc0, 0xc0000a09b0, 0x1, 0x5)
	/home/runner/work/crd2pulumi/crd2pulumi/cmd/root.go:158 +0x271
github.com/spf13/cobra.(*Command).execute(0xc000334dc0, 0xc00003a1f0, 0x5, 0x5, 0xc000334dc0, 0xc00003a1f0)
	/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:846 +0x2c2
github.com/spf13/cobra.(*Command).ExecuteC(0xc000334dc0, 0xc00052ff00, 0x1, 0x1)
	/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:950 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
	/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:887
github.com/pulumi/crd2pulumi/cmd.Execute(0xc000084748, 0x27380e0)
	/home/runner/work/crd2pulumi/crd2pulumi/cmd/root.go:190 +0x666
main.main()
	/home/runner/work/crd2pulumi/crd2pulumi/main.go:25 +0x26

Generate List Resources like in kubernetes library

The main kubernetes sdk Resources usually have a ${Resource}List Resource that corresponds to a list of the base resources that can be used to get a list of existing resources(e.g. NamespaceList, ConfigMapList). It would be cool if crd2pulumi could be used to generate these for custom resources as some operators create CustomResources to be used when configuring its features(openebs BlockDevice, etc). The current work around is to modify the generated code to have a ${Resource}List resource that correctly fetches the list.

Affected feature

Code Generation

Error when trying to generate crd from rabbitmq-operator

Hello, I got an error when I try to convert the crd of rabbitmq-operator into pulumi for nodejs.

You could try it by downloading :

I use the following command

./crd2pulumi --nodejsPath crd/rabbitmq manifests/rabbitmq-operator.yml

And I got the following error :

panic: fatal: error An assertion has failed: could not parse Pulumi package. source error: could not import spec: binding types: failed to bind type kubernetes:rabbitmq.com/v1beta1:RabbitmqClusterSpec: error binding default value for property resources: default values may only be provided for boolean, integer, number, and string properties

Here is the part wich raised the error :

resources:
  default:
    limits:
      cpu: 2000m
      memory: 2Gi
    requests:
      cpu: 1000m
      memory: 2Gi
  description: ResourceRequirements describes the compute resource requirements.
  properties:
    limits:
      additionalProperties:
        anyOf:
        - type: integer
        - type: string
        pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
        x-kubernetes-int-or-string: true
      description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
      type: object
    requests:
      additionalProperties:
        anyOf:
        - type: integer
        - type: string
        pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
        x-kubernetes-int-or-string: true
      description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
      type: object

To solve the error I need to update the crd to use code like the one bellow

resources:
  description: ResourceRequirements describes the compute resource requirements.
  properties:
    limits:
      additionalProperties:
        anyOf:
        - type: integer
        - type: string
        pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
        x-kubernetes-int-or-string: true
      description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
      type: object
    requests:
      additionalProperties:
        anyOf:
        - type: integer
        - type: string
        pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
        x-kubernetes-int-or-string: true
      description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
      type: object

Maybe you are missinterpetring the following parts :

anyOf:
  - type: integer
  - type: string

and

x-kubernetes-int-or-string: true

Fails on SparkApplication

Downloading sparkoperator.k8s.io_sparkapplications.yaml and running via .\crd2pulumi.exe --nodejs .\SparkApplication.yaml, I get the below stack trace.

panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x1 addr=0x20 pc=0x50a16d]

goroutine 1 [running]:
bytes.(*Buffer).WriteString(0x0, 0xc000fd4150, 0x6e, 0x3, 0x3, 0xc000fd4150)
        /usr/local/go/src/bytes/buffer.go:181 +0x2d
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).genNodeJSFiles(0xc00061dcb0, 0x1, 0xc000accc10, 0x1)
        /Users/albert/go/src/github.com/pulumi/crd2pulumi/gen/nodejs.go:100 +0xb89
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).genNodeJS(0xc000525cb0, 0xc000398010, 0xb, 0xc0003b2780, 0x1)
        /Users/albert/go/src/github.com/pulumi/crd2pulumi/gen/nodejs.go:35 +0x36
github.com/pulumi/crd2pulumi/gen.Generate(0xc0003c0190, 0x0, 0x0, 0x0, 0xc000700100, 0x1, 0x2, 0xc000139d00, 0xeb5919, 0x1ac2680)
        /Users/albert/go/src/github.com/pulumi/crd2pulumi/gen/generate.go:95 +0x21b
github.com/pulumi/crd2pulumi/cmd.glob..func2(0x1ac2680, 0xc000700100, 0x1, 0x2)
        /Users/albert/go/src/github.com/pulumi/crd2pulumi/cmd/root.go:119 +0xe0
github.com/spf13/cobra.(*Command).execute(0x1ac2680, 0xc00007e050, 0x2, 0x3, 0x1ac2680, 0xc00007e050)
        /Users/albert/go/pkg/mod/github.com/spf13/[email protected]/command.go:846 +0x2a4
github.com/spf13/cobra.(*Command).ExecuteC(0x1ac2680, 0x0, 0xf45ec0, 0xc000032478)
        /Users/albert/go/pkg/mod/github.com/spf13/[email protected]/command.go:950 +0x350
github.com/spf13/cobra.(*Command).Execute(...)
        /Users/albert/go/pkg/mod/github.com/spf13/[email protected]/command.go:887
github.com/pulumi/crd2pulumi/cmd.Execute(...)
        /Users/albert/go/src/github.com/pulumi/crd2pulumi/cmd/root.go:131
main.main()
        /Users/albert/go/src/github.com/pulumi/crd2pulumi/main.go:22 +0x35

Environment
OS: Windows 10, x64
crd2pulumi: 1.0.3

Cannot generate Traefik CRD's in TypeScript

Here's the repro, lemme know if there's any additional info you need.

VERSION:

λ crd2pulumi -v
crd2pulumi version v1.0.5

REPRO:

λ git clone https://github.com/traefik/traefik-helm-chart
λ cd traefik-helm-chart/traefik/crds
λ crd2pulumi -n *
Successfully generated code.
λ tree crds
crds
└── nodejs
    ├── README.md
    ├── index.ts
    ├── meta
    │   └── v1.ts
    ├── package.json
    ├── provider.ts
    ├── tsconfig.json
    └── utilities.ts
λ cat crds/nodejs/index.ts
// *** WARNING: this file was generated by crd2pulumi. ***
// *** Do not edit by hand unless you're certain you know what you are doing! ***

// Export members:
export * from "./provider";

Autogenerated CRD unnecessarily(?) requires kubernetes plugin on `pulumi up`

After crd2pulumi'ing cert-manager crds yamls, I'm trying to use the generated code, but can't figure out where this dependency comes from on pulumi up.

Diagnostics:
  pulumi:providers:kubernetes (default_0_0_1):
    error: no resource plugin 'kubernetes-v0.0.1' found in the workspace or on your $PATH, install the plugin using `pulumi plugin install resource kubernetes v0.0.1`

package.json of my local autogenerated package:

{
  "name": "@pulumi/depo-cert-manager",
  "version": "0.0.1",
  "scripts": {
    "build": "tsc"
  },
  "peerDependencies": {
    "@pulumi/pulumi": "^2.22.0"
  },
  "pulumi": {
    "resource": true
  },
  "devDependencies": {
    "typescript": "^4.2.3"
  }
}

loading that package in the main pulumi program package.json with:

"dependencies": {
    "@pulumi/depo-cert-manager": "file:cert-manager-types",

What's causing pulumi to think of this package as a plugin?

I changed to "version": "0.0.2" , and I get the following on pulumi up:

Diagnostics:
  pulumi:providers:kubernetes (default_0_0_2):
    error: no resource plugin 'kubernetes-v0.0.2' found in the workspace or on your $PATH, install the plugin using `pulumi plugin install resource kubernetes v0.0.2`

Expected behavior

Strange workaround, updating the npm package version to match the installed kubernetes plugin version bypasses the blocking error

  "version": "2.8.2",
$ pulumi up
[...]
     Type                                                              Name                                       Plan       Info
     pulumi:pulumi:Stack                                               haze-omega
 +   ├─ kubernetes:cert-manager.io/v1:ClusterIssuer                    letsencrypt-staging                        create
[...]

Steps to reproduce

In a new pulumi typescript project

mkdir cert-manager
cd cert-manager/
mkdir crds
wget https://github.com/jetstack/cert-manager/releases/download/v1.2.0/cert-manager.crds.yaml --output-document crds/cert-manager.crds.yaml
crd2pulumi crds/cert-manager.crds.yaml --nodejsName depo-cert-manager --nodejsPath cert-manager-types --force
sed -i 's/"version": ""/"version": "0.0.1"/g' cert-manager-types/package.json
npm install typescript --save-dev
cd ../../../
npm install --save cert-manager/cert-manager-types/

Using local lib inside the pulumi project, something like this:

[...]
import * as cert from "@pulumi/depo-cert-manager"

const certName = 'vouch'
const certNamespaceName = 'cert-manager'
const certNamespace = new k8s.core.v1.Namespace(certNamespaceName, {
  metadata: { name: certNamespaceName, labels: { name: certName } }
})

const certManager = new k8s.helm.v3.Chart(certName, {
  repo: "jetstack",
  chart: "cert-manager",
  namespace: certNamespace.metadata.name, values: { installCRDs: true }
})

const clusterIssuerName = 'letsencrypt-staging'

const clusterIssuer = new cert.certmanager.v1.ClusterIssuer(clusterIssuerName, {
  metadata: { name: certNamespaceName, labels: { name: certName }, namespace: certNamespace.metadata.name },
  spec: {
    acme: {
      server: "https://acme-staging-v02.api.letsencrypt.org/directory",
      email: "[email protected]",
      privateKeySecretRef: {
        name: clusterIssuerName
      },
      solvers: [
        {
          http01: {
            ingress: {
              class: "nginx"
            }
          }
        }
      ]
    }
  }
})
[...]

Context (Environment)

Wanted TS type checking on writing a cert-manager ClusterIssuer definition.

Affected feature

Unknown(?)

Original Slack post

https://pulumi-community.slack.com/archives/CJ909TL6P/p1615095347031700

Missing import and attribute in .NET generated code

When generating CRDs code for .NET the generated code has no import for the correct KubernetesResource base class, as well as using the CrdsResourceType attribute, which is nowhere to be found.

Steps to reproduce

  1. run ./crd2pulumi -d cert-manager.crds.yaml
  2. open the Pulumi.Crds.csproj project
  3. the project has errors in all the generated CDR code files

Expected: the generated project compiles
Actual: the generated project doesn't compile

Example:

using System;
using System.Collections.Generic;
using System.Collections.Immutable;
using System.Threading.Tasks;
using Pulumi.Serialization;

namespace Pulumi.Crds.Certmanager.V1
{
    /// <summary>
    /// A Certificate resource should be created to ensure an up to date and signed x509 certificate is stored in the Kubernetes Secret resource named in `spec.secretName`.
    ///  The stored certificate will be renewed before it expires (as configured by `spec.renewBefore`).
    /// </summary>
    [CrdsResourceType("kubernetes:cert-manager.io/v1:Certificate")]
    public partial class Certificate : KubernetesResource
    {

In this file, the KubernetesResource is not resolved in any using namespace. Adding this:

using Pulumi.crds;

solves the issue, but it must be done in all the files.

Also, the CrdsResourceType attribute is not resolved, and I cannot find it anywhere.

There is no CronTabDefinition method if I follow your exemple

Hello,
if I use crd2pulumi to generate a nodejs code, I can't create the crd resource beacause there is no MyCrdDefinition method.

I have follow your exemple here : https://github.com/pulumi/crd2pulumi#examples

I run the command bellow :

crd2pulumi --nodejsPath ./crontabs resourcedefinition.yaml

But there is no crontabs.stable.CronTabDefinition() method in my code

import * as crontabs from "./crontabs"
import * as pulumi from "@pulumi/pulumi"

// Register the CronTab CRD.
const cronTabDefinition = new crontabs.stable.CronTabDefinition("my-crontab-definition")

I have the following error :

Property 'CronTabDefinition' does not exist on type 'typeof 

Unresolved Type 'Provider' in init.go

I've generated go types from this crd with crd2pulumi --goPath ./crds managedcertificates-crd.yaml and I get the following error in crds/init.go on line 26: Unresolved type 'Provider'.

image

Expected behavior

I expect that the output of crd2pulumi have no type errors

Current behavior

The output has type errors

Steps to reproduce

  1. git checkout https://github.com/GoogleCloudPlatform/gke-managed-certs/tree/b4bbbe4cf108ab28c224b340493a58d893bdf3c6
  2. cd gke-managed-certs/deploy
  3. crd2pulumi --goPath ./crds managedcertificates-crd.yaml
  4. open in GoLand and see the type error

Context (Environment)

I'm trying to generate a typed CRD resource. I'm using version 1.0.7 of crd2pulumi

Affected feature

go type generation

[Go] Missing definitions in generated code

Generated code is missing some type members causing compile errors

Steps to reproduce

Generate code from this crd definition:

$ crd2pulumi version
v1.0.6

$ crd2pulumi -g k8s_v1alpha1_encryptedsecret_crd.yaml

The resulting generated code is missing some types so it won't compile, tested on versions v1.0.6..8. However, generating the same crd using version v1.0.5 results in correct generated code (after changing the import sdk version to v3)

Python ObjectMeta output not propagating correctly

Python ObjectMeta output not propagating correctly.
It is impossible to use custom resource metadata name for other resources.
Case: I can't use cert-manager's ClusterIssuer metadata['name'] for the Certificate.

Expected behavior

issuer_resource.metadata['name'] is available

Current behavior

    error: Program failed with an unhandled exception:
    error: Traceback (most recent call last):
      File "/usr/local/bin/pulumi-language-python-exec", line 85, in <module>
        loop.run_until_complete(coro)
      File "/usr/local/Cellar/[email protected]/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
        return future.result()
      File "/path/to/project/.venv/lib/python3.9/site-packages/pulumi/runtime/stack.py", line 83, in run_in_stack
        await run_pulumi_func(lambda: Stack(func))
      File "/path/to/project/.venv/lib/python3.9/site-packages/pulumi/runtime/stack.py", line 51, in run_pulumi_func
        await RPC_MANAGER.rpcs.pop()
      File "/path/to/project/.venv/lib/python3.9/site-packages/pulumi/runtime/rpc_manager.py", line 67, in rpc_wrapper
        result = await rpc
      File "/path/to/project/.venv/lib/python3.9/site-packages/pulumi/output.py", line 91, in is_value_known
        return await is_known and not contains_unknowns(await future)
      File "/path/to/project/.venv/lib/python3.9/site-packages/pulumi/output.py", line 91, in is_value_known
        return await is_known and not contains_unknowns(await future)
      File "/path/to/project/.venv/lib/python3.9/site-packages/pulumi/output.py", line 91, in is_value_known
        return await is_known and not contains_unknowns(await future)
      [Previous line repeated 3 more times]
      File "/path/to/project/.venv/lib/python3.9/site-packages/pulumi/runtime/resource.py", line 358, in do_register
        resolver = await prepare_resource(res, ty, custom, props, opts)
      File "/path/to/project/.venv/lib/python3.9/site-packages/pulumi/runtime/resource.py", line 86, in prepare_resource
        serialized_props = await rpc.serialize_properties(props, property_dependencies_resources, res.translate_input_property)
      File "/path/to/project/.venv/lib/python3.9/site-packages/pulumi/runtime/rpc.py", line 78, in serialize_properties
        result = await serialize_property(v, deps, input_transformer)
      File "/path/to/project/.venv/lib/python3.9/site-packages/pulumi/runtime/rpc.py", line 227, in serialize_property
        obj[transformed_key] = await serialize_property(v, deps, input_transformer)
      File "/path/to/project/.venv/lib/python3.9/site-packages/pulumi/runtime/rpc.py", line 227, in serialize_property
        obj[transformed_key] = await serialize_property(v, deps, input_transformer)
      File "/path/to/project/.venv/lib/python3.9/site-packages/pulumi/runtime/rpc.py", line 199, in serialize_property
        value = await serialize_property(output.future(), deps, input_transformer)
      File "/path/to/project/.venv/lib/python3.9/site-packages/pulumi/runtime/rpc.py", line 185, in serialize_property
        future_return = await asyncio.ensure_future(awaitable)
      File "/path/to/project/.venv/lib/python3.9/site-packages/pulumi/output.py", line 116, in get_value
        val = await self._future
      File "/path/to/project/.venv/lib/python3.9/site-packages/pulumi/output.py", line 178, in run
        transformed: Input[U] = func(value)
      File "/path/to/project/.venv/lib/python3.9/site-packages/pulumi/output.py", line 236, in <lambda>
        return self.apply(lambda v: UNKNOWN if isinstance(v, Unknown) else cast(Any, v)[key], True)
    KeyError: 'name'
    error: an unhandled error occurred: Program exited with non-zero exit code: 1

Steps to reproduce

wget https://github.com/jetstack/cert-manager/releases/download/v0.15.0/cert-manager.crds.yaml
crd2pulumi -cert-manager.crds.yaml

pulumi_crds/meta_v1/outputs.py file:

# coding=utf-8
# *** WARNING: this file was generated by crd2pulumi. ***
# *** Do not edit by hand unless you're certain you know what you are doing! ***

import warnings
import pulumi
import pulumi.runtime
from typing import Any, Mapping, Optional, Sequence, Union
from .. import _utilities, _tables

__all__ = [
    'ObjectMeta',
]

@pulumi.output_type
class ObjectMeta(dict):
    def __init__(__self__):
        pass

    def _translate_property(self, prop):
        return _tables.CAMEL_TO_SNAKE_CASE_TABLE.get(prop) or prop

Steps:

    issuer_prod = ClusterIssuer(
        issuer_prod_name,
        metadata=ObjectMetaArgs(name=issuer_prod_name),
       [...]
    )

    
    cert = CustomResource(
        name, "cert-manager.io/v1alpha3", "Certificate",
        metadata={
       [...]
        },
        spec={
            "issuerRef": {
                "name": issuer_resource.metadata['name'],
                "kind": issuer_resource.kind,
            },
       [...]
     )

Context (Environment)

$ pulumi version
v2.21.2

$ crd2pulumi --version
crd2pulumi version v1.0.5
pulumi==2.21.2
pulumi-kubernetes==2.8.2

Affected feature

panic for complex default values

Eventually, the pulumi schema should support complex default values.

In the meantime, we could at least avoid the panic by either
a. erroring on complex default values, or
b. eliding complex default values, or
c. some combination of the above (e.g. error by default and elide if a flag is passed)

Originally posted by @pgavlin in #63 (comment)

Add typescript ignore comment

I have lot of errors like that:

error TS6133: 'pulumi' is declared but its value is never read.
error TS6133: 'ObjectMeta' is declared but its value is never read.

on generated files. For fast workaround, I propose to add a line // @ts-nocheck at the beginning of typescripts files generated by crd2pulumi.

crd properties with - in name

I am using a crd which has property names with - in it and the crd2pulumi failed with

panic: invalid Go source code:

crds/srlinux/v1alpha1/pulumiTypes.go
: 269:7: expected ';', found '-' (and 10 more errors)

goroutine 1 [running]:
github.com/pulumi/pulumi/pkg/v3/codegen/go.GeneratePackage.func1(0xc0001198f0, 0x24, 0xc000a88000, 0x497e4)
	/home/runner/go/pkg/mod/github.com/pulumi/pulumi/pkg/[email protected]/codegen/go/gen.go:2151 +0x46e
github.com/pulumi/pulumi/pkg/v3/codegen/go.GeneratePackage(0x1df1de5, 0xa, 0xc000532280, 0x2, 0xc000026508, 0x1)
	/home/runner/go/pkg/mod/github.com/pulumi/pulumi/pkg/[email protected]/codegen/go/gen.go:2272 +0x1b33
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).genGoFiles(0xc0002a5a90, 0x1de94c9, 0x4, 0xc0002163b0, 0x1, 0x1)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/golang.go:55 +0x345
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).genGo(0xc0002a5a90, 0xc00039b4a8, 0x7, 0x1de94c9, 0x4, 0x1, 0xc0002163f0)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/golang.go:32 +0x45
github.com/pulumi/crd2pulumi/gen.Generate(0x0, 0x0, 0x0, 0xc00011a7d0, 0x1de94c9, 0x4, 0x1de94c9, 0x4, 0x1de94c9, 0x4, ...)
	/home/runner/work/crd2pulumi/crd2pulumi/gen/generate.go:73 +0x20c
github.com/pulumi/crd2pulumi/cmd.Execute.func2(0xc00047d080, 0xc000716240, 0x1, 0x3)
	/home/runner/work/crd2pulumi/crd2pulumi/cmd/root.go:157 +0x271
github.com/spf13/cobra.(*Command).execute(0xc00047d080, 0xc00003a090, 0x3, 0x3, 0xc00047d080, 0xc00003a090)
	/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:846 +0x2c2
github.com/spf13/cobra.(*Command).ExecuteC(0xc00047d080, 0xc00013ff00, 0x1, 0x1)
	/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:950 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
	/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:887
github.com/pulumi/crd2pulumi/cmd.Execute(0xc00007a748, 0x263f000)
	/home/runner/work/crd2pulumi/crd2pulumi/cmd/root.go:189 +0x666
main.main()

input file:

          spec:
            description: SrlInterfaceSpec struct
            properties:
              interface:
                items:
                  description: Interface struct
                  properties:
                    admin-state:
                      default: enable
                      enum:
                      - disable
                      - enable
                      type: string

I changed the code as per below and this solves it for me.

// AddType converts the given OpenAPI `schema` to a ObjectTypeSpec and adds it
// to the `types` map under the given `name`. Recursively converts and adds all
// nested schemas as well.
func AddType(schema map[string]interface{}, name string, types map[string]pschema.ComplexTypeSpec) {
	properties, foundProperties, _ := unstruct.NestedMap(schema, "properties")
	description, _, _ := unstruct.NestedString(schema, "description")
	schemaType, _, _ := unstruct.NestedString(schema, "type")
	required, _, _ := unstruct.NestedStringSlice(schema, "required")

	propertySpecs := map[string]pschema.PropertySpec{}
	for propertyName := range properties {
		propertyName = strcase.UpperCamelCase(propertyName)
		propertySchema, _, _ := unstruct.NestedMap(properties, propertyName)
		propertyDescription, _, _ := unstruct.NestedString(propertySchema, "description")
		defaultValue, _, _ := unstruct.NestedFieldNoCopy(propertySchema, "default")
		propertySpecs[propertyName] = pschema.PropertySpec{
			TypeSpec:    GetTypeSpec(propertySchema, name+strings.Title(propertyName), types),
			Description: propertyDescription,
			Default:     defaultValue,
		}
	}

Python code imports do not work

Pulumi v3.6.1
Pulumi-kubernetes v3.5.0
crd2pulumi v1.0.8

I am trying to generate a Pulumi Python module for Cert-manager CRDs:

$ wget https://github.com/jetstack/cert-manager/releases/download/v1.4.0/cert-manager.crds.yaml
$ crd2pulumi --pythonPath certmanager cert-manager.crds.yaml
$ cat __main__.py
"""A Kubernetes Python Pulumi program"""

from certmanager.pulumi_crds import certmanager

$ pulumi preview

Previewing update (dev)

View Live: https://app.pulumi.com/drzero/pulumi-learn/dev/previews/da0b527b-a91f-46fc-b483-457e9e71877c

     Type                 Name              Plan       Info
 +   pulumi:pulumi:Stack  pulumi-learn-dev  create     3 errors
 
Diagnostics:
  pulumi:pulumi:Stack (pulumi-learn-dev):
    error: Program failed with an unhandled exception:
    error: Traceback (most recent call last):
      File "/home/abo/.pulumi/bin/pulumi-language-python-exec", line 92, in <module>
        loop.run_until_complete(coro)
      File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
        return future.result()
      File "/home/abo/sandbox/pulumi-learn/venv/lib/python3.8/site-packages/pulumi/runtime/stack.py", line 110, in run_in_stack
        await run_pulumi_func(lambda: Stack(func))
      File "/home/abo/sandbox/pulumi-learn/venv/lib/python3.8/site-packages/pulumi/runtime/stack.py", line 43, in run_pulumi_func
        func()
      File "/home/abo/sandbox/pulumi-learn/venv/lib/python3.8/site-packages/pulumi/runtime/stack.py", line 110, in <lambda>
        await run_pulumi_func(lambda: Stack(func))
      File "/home/abo/sandbox/pulumi-learn/venv/lib/python3.8/site-packages/pulumi/runtime/stack.py", line 133, in __init__
        func()
      File "/home/abo/.pulumi/bin/pulumi-language-python-exec", line 91, in <lambda>
        coro = pulumi.runtime.run_in_stack(lambda: runpy.run_path(args.PROGRAM, run_name='__main__'))
      File "/usr/lib/python3.8/runpy.py", line 282, in run_path
        return _run_code(code, mod_globals, init_globals,
      File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
        exec(code, run_globals)
      File "/home/abo/sandbox/pulumi-learn/./__main__.py", line 8, in <module>
        from certmanager.pulumi_crds import certmanager
      File "/home/abo/sandbox/pulumi-learn/./certmanager/pulumi_crds/__init__.py", line 9, in <module>
        from . import (
      File "/home/abo/sandbox/pulumi-learn/./certmanager/pulumi_crds/acme/__init__.py", line 7, in <module>
        from . import (
      File "/home/abo/sandbox/pulumi-learn/./certmanager/pulumi_crds/acme/v1/__init__.py", line 6, in <module>
        from .Challenge import *
      File "/home/abo/sandbox/pulumi-learn/./certmanager/pulumi_crds/acme/v1/Challenge.py", line 12, in <module>
        from pulumi_kubernetes import meta_v1 as _meta_v1
    ImportError: cannot import name 'meta_v1' from 'pulumi_kubernetes' (/home/abo/sandbox/pulumi-learn/venv/lib/python3.8/site-packages/pulumi_kubernetes/__init__.py)
    error: an unhandled error occurred: Program exited with non-zero exit code: 1

As far as I can see, there is no meta_v1 module in pulumi_kubernetes, it needs to do something like from pulumi_kubernetes.meta impart v1 as _meta_v1 instead.

Duplicate declarations in generated Golang code

Hello!

  • Vote on this issue by adding a 👍 reaction
  • To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already)

Issue details

When one attempts to use the crd2pulumi binary to generate a library from a CRD, the GoLang code errors due to duplicate declarations. Example image attached.

Screen Shot 2022-02-03 at 3 10 59 PM

Steps to reproduce

  1. Grab the ArgoRollout CRD declarations located here
  2. Initialize your pulumi stack
  3. Run crd2pulumi --goPath ./internal/argorollouts install.yaml

Expected: Pulumi will generate a working Golang implementation of the CRD
Actual: The GoLang generated by Pulumi is not valid.

Update pyyaml dependency on generated code to the same as pulumi-kubernetes

Hello!

  • Vote on this issue by adding a 👍 reaction
  • If you want to implement this feature, comment to let us know (we'll work with you on design, scheduling, etc.)

Issue details

Currently crd2pulumi generates code that has a dependency on pyyaml 5.2 (see here) whereas the kubernetes provider requires at least 5.3.1 (see here).

Can we bring them more in line?

Node code generation mixes Input types into Outputs

For example, with the crontab, it generates the following:

stable/v1/cronTab.ts:

import {ObjectMeta} from "../../meta/v1";
...
public readonly metadata!: pulumi.Output<ObjectMeta | undefined>;

../../meta/v1:

import * as k8s from "@pulumi/kubernetes";

export type ObjectMeta = k8s.types.input.meta.v1.ObjectMeta;

So now, the output field is actually reference the ObjectMeta input type.

Failure to parse nested default values

When attempting to parse the CRD referenced to nodejs in https://www.rabbitmq.com/kubernetes/operator/install-operator.html (i.e. https://github.com/rabbitmq/cluster-operator/releases/latest/download/cluster-operator.yml) crd2pulumi fails with an "error binding default value for property storage".

Expected behavior

Should compile CRD correctly

Current behavior

Fails with following error:

λ c:\dev\localApps\crd2pulumi\crd2pulumi.exe -n components\cluster-operator-trimmed.yml
panic: fatal: error An assertion has failed: could not parse Pulumi package. source error: could not import spec: binding types: failed to bind type kubernetes:rabbitmq.com/v1beta1:RabbitmqClusterSpecPersistence: error binding default value for property storage: default values may only be provided for boolean, integer, number, and string properties

goroutine 1 [running]:
github.com/pulumi/pulumi/sdk/v2/go/common/util/contract.failfast(...)
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/[email protected]/go/common/util/contract/failfast.go:23
github.com/pulumi/pulumi/sdk/v2/go/common/util/contract.AssertNoErrorf(0x12c1060, 0xc0012d60a0, 0x111c62a, 0x1e, 0x0, 0x0, 0x0)
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/[email protected]/go/common/util/contract/assert.go:47 +0x1ca
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).SchemaPackage(0xc000517ba8, 0x3)
        /home/runner/work/crd2pulumi/crd2pulumi/gen/generate.go:181 +0xbc
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).genNodeJSFiles(0xc000517ba8, 0x10fcb88, 0x4, 0xc00093d100, 0x1, 0x1)
        /home/runner/work/crd2pulumi/crd2pulumi/gen/nodejs.go:39 +0x4a
github.com/pulumi/crd2pulumi/gen.(*PackageGenerator).genNodeJS(0xc000517ba8, 0xc00008f600, 0xb, 0x10fcb88, 0x4, 0x1, 0xc00093d140)
        /home/runner/work/crd2pulumi/crd2pulumi/gen/nodejs.go:30 +0x4a
github.com/pulumi/crd2pulumi/gen.Generate(0xc00068b770, 0x0, 0x0, 0x0, 0x10fcb88, 0x4, 0x10fcb88, 0x4, 0x10fcb88, 0x4, ...)
        /home/runner/work/crd2pulumi/crd2pulumi/gen/generate.go:63 +0x287
github.com/pulumi/crd2pulumi/cmd.glob..func2(0x1ac8640, 0xc0002d88a0, 0x1, 0x2)
        /home/runner/work/crd2pulumi/crd2pulumi/cmd/root.go:153 +0x218
github.com/spf13/cobra.(*Command).execute(0x1ac8640, 0xc00003e090, 0x2, 0x3, 0x1ac8640, 0xc00003e090)
        /home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:846 +0x2a4
github.com/spf13/cobra.(*Command).ExecuteC(0x1ac8640, 0x4455a1, 0x1a51280, 0xc00005bf78)
        /home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:950 +0x350
github.com/spf13/cobra.(*Command).Execute(...)
        /home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:887
github.com/pulumi/crd2pulumi/cmd.Execute(...)
        /home/runner/work/crd2pulumi/crd2pulumi/cmd/root.go:165
main.main()
        /home/runner/work/crd2pulumi/crd2pulumi/main.go:25 +0x39

Steps to reproduce

  1. Retrieve operator from https://github.com/rabbitmq/cluster-operator/releases/latest/download/cluster-operator.yml (tested with 1.5.0)
  2. Trim away non-CRD elements from yaml
  3. Run crd2pulumi -n cluster-operator-trimmed.yml
  4. See error above.

Context (Environment)

It's likely the following section is responsible for the error:

              persistence:
                default:
                  storage: 10Gi
                description: The desired persistent storage configuration for each Pod in the cluster.
                properties:
                  storage:
                    anyOf:
                    - type: integer
                    - type: string
                    default: 10Gi
                    description: The requested size of the persistent volume attached to each Pod in the RabbitmqCluster. The format of this field matches that defined by kubernetes/apimachinery. See https://pkg.go.dev/k8s.io/apimachinery/pkg/api/resource#Quantity for more info on the format of this field.
                    pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                    x-kubernetes-int-or-string: true

Affected feature

Typescript generated types for inputs and outputs are not eslinted and have unused imports.

Expected behavior

index, input, and output types files should be eslint compliant and not include unused imports.

Current behavior

In the types directory generated, there are 3 files: index.ts, input.ts, output.ts

index.ts

  • Needs eslint formatting / spacing

input.ts

  • Needs eslint formatting / spacing
  • Needs eslint compliance with removing unused imports for outputs
  • Needs eslint compliance for interface params that use any e.g.
     // eslint-disable-next-line
      config?: pulumi.Input<{[key: string]: any}>;
    

output.ts

  • Needs eslint formatting / spacing
  • Needs eslint compliance with removing unused imports for inputs
  • Needs eslint compliance for interface params that use any e.g.
     // eslint-disable-next-line
      config?: pulumi.Input<{[key: string]: any}>;
    

Related:

Go Code not generating with v1.0.5

Hi,

regarding the latest release, crd2pulumi does not seem to generate the crd-definitions for go although it states success. Tried on MacOS as well as on linux amd x64.
Version 1.0.4 just works fine.

Can't update existing resource

  kubernetes:networking.istio.io/v1alpha3:VirtualService (kit):
    error: 1 error occurred:
        * the body of the request was in an unknown format - accepted media types include: application/json-patch+json, application/merge-patch+json, application/apply-patch+yaml

cert-manager CRDs fail to convert to dotnet

Steps to reproduce:

crd2pulumi --dotnetPath ./crds ./cert-manager.yaml

Expected result:
all CRDs created as .cs files in the ./crds folder

Actual result
errors on random files:

error: could not create file /path/to/crds/Certmanager/V1/Outputs/IssuerSpecVaultAuthAppRole.cs: open /path/to/crds/Certmanager/V1/Outputs/IssuerSpecVaultAuthAppRole.cs: too many open files

Re-Running crd2pulumi multiple times with -f flag eventually writes all of the required files.

Codegen only generates the static types for a CRD instance, but not the required CustomResourceDefinition resource to install the CRD

Expected behavior

Generate the CRD pulumi CustomResource, in addition to the statically typed classes for the CRD.

Current behavior

Currently, static types are the only generated output, but what is also required is the CRD ComponentResource itself such that the CRD can be installed into the cluster using Pulumi.

e.g.

const foobar = new k8s.apiextensions.v1.CustomResourceDefinition(name, {
       metadata: {...},
       spec: {...},
});
  • The generated CustomResourceDefinition should also be eslint-compliant and formatted

Support for Generating Code in Directory

I'm using the following command to generate Pulumi types for Cluster API components.

./crd2pulumi --nodejsName capp --nodejsPath ./src/capp ./capp-components.yaml --force

This creates a utilities.{ts,js} that has code for loading the Node version from package.json.

This code fails if the code is generated into a subdirectory of the node project.

It would be cool if we could provide another flag to provide a subpath to store the generated code

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.