concourse_tls_ca_cert issue while upgrading from v6.7.5 to v7.1.0 about concourse-bosh-release

[bg-govau]

We experienced this as well when we went to v7.1.0.

I think whats happening is that if you are using the ops files from concourse-bosh-deployment to add tls to your web instance_group:

https://github.com/concourse/concourse-bosh-deployment/blob/5a9d729e9cdfd6a67fbfe91d1a9da322487d6ec0/cluster/operations/tls-vars.yml#L10-L18

https://github.com/concourse/concourse-bosh-deployment/blob/5a9d729e9cdfd6a67fbfe91d1a9da322487d6ec0/cluster/operations/tls.yml#L1-L3

Then the atc_tls cert has a ca part in credhub, which means CONCOURSE_TLS_CA_CERT is set to the ca, making concourse then expect a client cert signed by that ca.

In our case - we were using a reverse proxy in front of concourse, so for now we've just configured it to do what concourse now wants and send a valid client cert to the concourse web backend.

If you're not using a reverse proxy and are accessing concourse web directly, I'd assume the normal practice would be to use a real cert for atc_tls, in which case people might not have the ca part set when uploading it to credhub, and it might be all good.

It kind of feels like to me it'd be simpler for operators to allow them to explicitly enable requiring client cert in a separate property, instead of just toggling based on the presence of the ca part.

[divyamudundi]

Thank you @bg-govau. We dont have a reverse proxy in front of our concourse-web so we are using a custom-ops file to have the atc_tls ca removed from the web-nodes manifest for the 7.1.x deployment.

[pivotal-madeline-preston]

Experienced this as well. Our fix:

bosh -d credhub manifest > credhub.yml

Modified manifest from

tls:
        bind_port: 443
        cert: ((atc_tls))

To

tls:
        bind_port: 443
        cert:
          certificate: ((atc_tls.certificate))
          private_key: ((atc_tls.private_key))

bosh -d concourse deploy concourse.yml