Provisionning fails with RKE 1.5.9 when `extra_env` is used for `kube-api` service about rke

[tmsdce]

Hi @jiaqiluo
Can you take a look at this issue as it seems related to 2e767c8

[tmsdce]

This bug prevents us from upgrading as kube-apiserver has environment variables set (like the RKE_AUDITLOG_CONFIG_CHECKSUM variable).

[jiaqiluo]

hi @tmsdce, thank you for reporting the issue. I can confirm that this is a bug.

Root Cause

This bug occurs in RKE v1.5.9.

If extra_env is set for kube-api as below in the cluster config file, the Env list in the container configuration for the kube-apiserver container will contain an empty string (""), which causes the creation to fail with the error invalid environment variable: returned from Docker.

services:
  kube-api:
    extra_env:
    - TEST_VAR=test

Workaround

If your cluster is stuck in the failed status, remove the extra_env for kube-api from the cluster config and run rke up again can bring the cluster back to active.

If you need to set the extra_env for kube-api, please use RKE v1.5.8 for now until the bug is fixed in the coming release.

[jiaqiluo]

the fix can be validated by using https://github.com/rancher/rke/releases/tag/v1.6.0-rc6

Root cause

Previously, we declared the Env slice with the size of the c.Services.KubeAPI.ExtraEnv field, which leads to empty strings at the beginning of the Env slice because we use Golang's append function to add new elements, and Docker is not happy with that.

What was fixed, or what changes have occurred

We fix this issue by declaring the Env variable as an empty slice.
We also enhance the getUniqStringList function to properly trim leading and trailing spaces in each element and to ignore empty strings.
We add unit tests for the getUniqStringList function and update the integration tests.

Areas or cases that should be tested

cluster provisioning and upgrading

What areas could experience regressions?

cluster provisioning and upgrading

Are the repro steps accurate/minimal?

yes

We should validate that setting extra env does not cause the reported error when either creating a new cluster or updating an existing cluster.

services:
  kube-api:
    extra_env:
    - TEST_VAR=test

[slickwarren]

tested the following on v1.6.0-rc9:

• provision an rke1 cluster with no extra_env -- pass
• provision an rke1 cluster with extra_env -- pass
• upgrade an rke1 cluster with no extra_env -> with extra_env -- pass
• upgrade an rke1 cluster with extra_env -> update extra_env -- pass
• upgrade an rke1 cluster with extra_env -> remove extra_env -- pass

tested with default versions of k8s (1.30.2)