Cannot `chmod` a read only filesystem about csi-driver

[pb6]

pkg/util/mount.go:102: options = append(options, "bind", "ro")

remove "ro" and it will work, as you cannot chmod/chown read only filesystem. But I'm not sure if that is what authors intended.

[aeugenio]

yeah the code from the examples just doesnt work

I0929 05:25:05.082499       1 certmanager.go:80] cert-manager: waiting for CertificateRequest to become ready csi-c677c0a826b2787d1e99e749ac58b0e62ffb1acdedca70dbe3201ec44a5b9b80
I0929 05:25:05.087437       1 certmanager.go:105] cert-manager: CA certificate written to file /csi-data-dir/csi-c677c0a826b2787d1e99e749ac58b0e62ffb1acdedca70dbe3201ec44a5b9b80/data/ca.pem
I0929 05:25:05.087542       1 certmanager.go:113] cert-manager: certificate written to file /csi-data-dir/csi-c677c0a826b2787d1e99e749ac58b0e62ffb1acdedca70dbe3201ec44a5b9b80/data/crt.pem
I0929 05:25:05.087617       1 certmanager.go:120] cert-manager: private key written to file: /csi-data-dir/csi-c677c0a826b2787d1e99e749ac58b0e62ffb1acdedca70dbe3201ec44a5b9b80/data/key.pem
E0929 05:25:05.087630       1 renew.go:181] volume already being watched, aborting second watcher: csi-c677c0a826b2787d1e99e749ac58b0e62ffb1acdedca70dbe3201ec44a5b9b80
E0929 05:25:05.088631       1 server.go:133] server: error: chmod /var/lib/kubelet/pods/35434cc0-caac-4513-b016-a284170381d8/volumes/kubernetes.io~csi/tls/mount: read-only file system

using the eks 1.17 and the latest of cert-manager and cert-manager-csi

helm list
NAME                        	REVISION	UPDATED                 	STATUS  	CHART                             	APP VERSION	NAMESPACE
cm                          	1       	Mon Sep 28 21:29:49 2020	DEPLOYED	cert-manager-v1.0.2               	v1.0.2     	cert-manager```

[robinbraemer]

Same problem here. I deployed the given example.

pod events

Warning  FailedMount  28s                  kubelet, XXXXXXX  Unable to attach or mount volumes: unmounted volumes=[tls], unattached volumes=[default-token-s5nk7 tls]: timed out waiting for the condition
  Warning  FailedMount  16s (x9 over 2m26s)  kubelet, XXXXXXX  MountVolume.SetUp failed for volume "tls" : rpc error: code = Unknown desc = chmod /var/lib/kubelet/pods/c7bb4ae4-9e00-4ccd-8769-e186f28f696f/volumes/kubernetes.io~csi/tls/mount: read-only file system

k version

Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.2", GitCommit:"52c56ce7a8272c798dbc29846288d7cd9fbae032", GitTreeState:"clean", BuildDate:"2020-04-16T11:56:40Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.8", GitCommit:"9f2892aab98fe339f3bd70e3c470144299398ace", GitTreeState:"clean", BuildDate:"2020-08-13T16:04:18Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}

[pb6]

Given that it does readonly bind mount (bind,ro), chown/chmod should fail afterwards. I changed code - removed 'ro' from mount options, and cert issuance succeeded,

total 12
-rw-------    1 root     root          1789 Oct 28 14:20 ca.pem
-rw-------    1 root     root          1464 Oct 28 14:20 crt.pem
-rw-------    1 root     root          1675 Oct 28 14:20 key.pem```

But I suspect that chmod comes from volume handling in k8s, and maybe it would be possible to specify owner and umask for created files.

[bmagic]

@pb6 What do you change to have it working ?

[xmclark]

I rebuilt the project from scratch, and republished the dockerfile, and this issue seems resolved. My image that I build from source:
https://github.com/users/xmclark/packages/container/package/cert-manager-csi