HelmRelease: CRDs of disabled subcharts get installed anyway about helm-controller

[stefanprodan]

Ok turns out this is a bug upstream in Helm and never fixed helm/helm#8508

[stefanprodan]

@MBrouns if your main chart contains no CRDs, you could skip all CRDs with:

apiVersion: helm.toolkit.fluxcd.io/v2beta2
kind: HelmRelease
spec:
  install:
    crds: Skip
  upgrade:
    crds: Skip

Besides this there is no way we can fix this issue in Flux. In Flux we use the Helm SDK to get the CRDs and the SDK returns all CRDs from all sub-charts before values are compiled.

[MBrouns]

Thanks for that lightning-quick response! Unfortunately we do enable other subcharts in the mainchart that have CRDs, so I don't think that'll fix it for us.

The weird thing is that if I do a helm install directly of the same manifests, the CRD is not installed. Only when I install the chart through flux it is. Are you suggesting there are differences between the helm cli and helm sdk?

[stefanprodan]

What version of the Helm CLI are you using?

[MBrouns]

version.BuildInfo{Version:"v3.13.3", GitCommit:"c8b948945e52abba22ff885446a1486cb5fd3474", GitTreeState:"clean", GoVersion:"go1.20.11"}

[stefanprodan]

I don't see how the Helm CLI can do anything different, both the controller and the CLI call this code where all CRDs are merged from all sub-charts and applied:

https://github.com/helm/helm/blob/14d0c13e9eefff5b4a1b511cf50643529692ec94/pkg/action/install.go#L254-L263

[MBrouns]

I agree that it's weird, but it does seem to be what's happening. Running helm install test . --set 'subchart.enabled=false' results in the CRD not being installed for the chart defined in https://github.com/MBrouns/fluxcd-subchart-crd-test/tree/main/mainchart

Let me preface this by saying I'm not very familiar with helm's codebase, but I've done some digging to understand the issue better. I can confirm that when I put a breakpoint here https://github.com/helm/helm/blob/14d0c13e9eefff5b4a1b511cf50643529692ec94/pkg/action/install.go#L256 the length of chrt.CRDObjects() is zero if the subchart is disabled and one if it is enabled.

The crd is initially in the chrt.CRDObjects() at the start of the RunWithContext but gets removed in the call to

if err := chartutil.ProcessDependenciesWithMerge(chrt, vals); err != nil {
    return nil, err
}

As far as I can tell though, Flux calls the same RunWithContext here:

helm-controller/internal/action/install.go

Line 62 in 9059e7f

return install.RunWithContext(ctx, chrt, vals.AsMap())

, but the lines above it (

helm-controller/internal/action/install.go

Lines 58 to 60 in 9059e7f

	if err := applyCRDs(config, policy, chrt, setOriginVisitor(v2.GroupVersion.Group, obj.Namespace, obj.Name)); err != nil {
	return nil, fmt.Errorf("failed to apply CustomResourceDefinitions: %w", err)
	}

) do mention something about installing CRDs anyway. Isn't that possibly where the issue is?

[MBrouns]

I looked into it a bit further and took the very blunt approach of calling helmchartutil.ProcessDependenciesWithMerge before applyCRDs is called and that does result in the crd of the subchart not being installed.

Can you confirm whether this makes sense? If so, I'll make the linked PR proper and add stuff like test cases

[artem-nefedov]

Got bitten by this as well with splunk-otel-collector chart.
@stefanprodan Helm cli does not reproduce this problem (can be tested with helm template --include-crds command), so this is purely a flux helm-controller issue.

A workaround with spec.install.crds=Skip helped at least.