Missing S3 variables on node Openstack cloud-init metadata about kops

[happycouak]

Thanks for your support.

Cheers

[hakman]

@happycouak This is by design, for security reasons. Giving access to the S3 bucket to nodes, would allow pods to read anything from the bucket, including secrets.

[happycouak]

@hakman thanks, make sens.

So is there a proper way to have nodeup bootrap nodes machines without manually updating etc/sysconfig/kops-configuration ?

[happycouak]

Also I note that except for S3 creds, metadata actually store all cloud credentials, so I wonder what would be the worst case if the node is compromised between cloud credentials and S3 access.

[hakman]

@zetaab do you remember the reason? Should --dns=none fix this?

[zetaab]

@hakman yes, when using --dns=none there are no credentials in normal nodes.

@happycouak I recommend using --dns=none always when using kOps with OpenStack.

Also it is weird for me that you are exporting AWS envs like:
export AWS_DEFAULT_REGION=xxx
export AWS_REGION=xxx
export AWS_ACCESS_KEY_ID=xxx
export AWS_SECRET_ACCESS_KEY=xxx

instead you should have openstack credentials (and S3_*) exported. Afaik kOps will not work without that

[happycouak]

The context is that the Openstack cloud I am using does not offer object storage service, so a S3 bucket is used to store kops states. Is it a supported target ?

Also you are right AWS* envs are useless, I unset them, and redeploy the whole cluster with --dns=none and everything works well.

Now all credentials are absent from cloud-init metadata and /etc/sysconfig/kops-configuration.

Last question: how are those creds are transfered to nodes ?

[zetaab]

nodes does not need any openstack credentials in node level. However, there are kubernetes secrets in kube-system namespace that are used by csi-cinder in normal nodes. Other components that uses credentials are located in control-planes

[happycouak]

Thanks, I get it regarding openstack credentials store in secret "openstack-project", but how S3 credentials are passed ton node during deployment ?

[hakman]

Thanks, I get it regarding openstack credentials store in secret "openstack-project", but how S3 credentials are passed ton node during deployment ?

S3 creds are not passed to the node at all. The node config is retrieved from kops-controller, which runs on the control-plane and can read from S3.

[zetaab]

nodes does not need S3 credentials for anything, so nodes does not have those.