Per-server TLS certificates about classicswarm

[vieux]

The use case we wanted to cover is the following:

docker client (client cert) <---------------------------------------------------> (server cert) docker daemon

So with Swarm it becomes:

docker client (client cert) <---> (server cert) docker swarm (client cert) <---> (server cert) docker daemon

With the same cert acting as client and server for Swarm

The case with different cert per daemon is more complex,
we want to keep setup easy :D
From the top of my head, I can see it working with the file "discovery backend but not with the others.

[vieux]

ping @aluzzardi

[aluzzardi]

@avaer: I'm not sure I completely understand your use case.

Isn't the whole point of a PKI with a CA to be able to manage different docker daemons that have different certificates?

I don't think it would be possible to override the CA on a per-server basis In a cluster system where nodes can pop up at any time through discovery, and I believe the advantage of using a CA is exactly to solve this particular issue.

Please correct me if I'm wrong.

[avaerkazmer]

For mostly historical reasons we're using one-off CA's on our hosts, each with its own access cert. It's not a very PKI-ish design, but it works perfectly fine with the docker daemon/clients.

We want to wrap this behind a swarm, but making that work requires either swarm support for specifying the cert set to use per-backend, or redoing all of the certs under one CA.

You're right on both counts though: it's hard to make this work in a discovery scenario (you'd need certificate exchange as part of discovery - nasty), and giving in to a full PKI architecture is probably the cleaner solution.

I guess it comes down to the goals of swarm: to flexibly wrap around any docker hosts, or to focus on implementing a particular (PKI-based) cluster/discovery architecture?

[gabrtv]

For mostly historical reasons we're using one-off CA's on our hosts, each with its own access cert. It's not a very PKI-ish design, but it works perfectly fine with the docker daemon/clients.

We want to wrap this behind a swarm, but making that work requires either swarm support for specifying the cert set to use per-backend, or redoing all of the certs under one CA.

Could you elaborate on this? I'm confused.

From what I can tell, the current cert validation requires pre-seeding certificates based on specific backends/hosts -- a process analogous to downloading/installing a cert in your browser before hitting each HTTPS website.

Standard PKI practice is to have a pool of CA certs that are used for validation. As long as:

1. the requested common-name matches the cert ("Host:" header in Docker API request?)
2. the cert chain is validated against the CA pool
3. the cert is not part of a revocation list

...validation should succeed. If I understand the OP correctly, the behavior I'm describing would solve the OP's problem (not to mention some of our own issues in docker/docker!).

As a newb to this part of the codebase, there's a 99% chance I'm misunderstanding the situation. Clarifications are much appreciated!

[avaerkazmer]

@gabrtv you're right, and docker handles PKI traditionally in the way you describe. You're also right that remodelling everything under "standard PKI practice" with a static CA pool would solve our problem.

However, in our case each host has its own CA and issues client certificates independently. On the client side we choose the key/cert/ca to use at connect time, depending on the host (there is no certificate "install" anywhere, it's completely dynamic). Everything is decentralized, and there is no single CA or client certificate that will grant access to all docker hosts.

This arrangement works with docker no problem (docker CLI itself has command line flags to do this). It doesn't work with swarm, which currently assumes that everything is under one CA and uses one single client cert to connect to all backends. The question is whether swarm should support this kind of configuration (which is nontrivial to do when you're automatically adding/removing hosts via the discovery protocol).

[SvenDowideit]

So swarm can't talk to a group of pre-existing Docker daemons - for example, a set of boot2docker base servers, where each will have its own CA (and other certs).

I was hoping to add a very simple docker-swarm join to my existing server's boot2docker boot scripts, and instantly have a swarm.

[SvenDowideit]

I'll do some work on the userguide docs to tell users what they need to know.

[SvenDowideit]

@vieux @aluzzardi looking at the docker-swarm manage commandline - does that mean you also can't have a TLS swarm socket and non-TLS Docker daemons, or non-TLS swarm daemon with a TLS Docker daemon? or a mix of TLS and non-TLS Docker daemons?

[SvenDowideit]

mm, also, can I use https://github.com/SvenDowideit/generate_cert ? that will generate the certs without requiring user input.

[cpuguy83]

@SvenDowideit I was messing with machine last night.
I think, or at least what I suggested, is to have machine generate a single CA (or allow one to be specified) and a single set of client keys, and then each new host gets a new set of server keys signed by that single CA.
This CA can then in turn be used to sign server keys for swarm.

[vieux]

@SvenDowideit at the moment, is all TLS or no TLS, no mix. We don't want to introduce 8 flags for TLS in swarm, 4 is already enough :D

You have to generate the certs with extendedKeyUsage = clientAuth,serverAuth

@ehazlett can you confirm for machine ?

[ehazlett]

Yeah after thinking about @cpuguy83 suggestion, I think this is the right way (single CA, single client, new hosts get new server certs).

I am working on this for Machine.

[vieux]

👍

[aluzzardi]

👍

If you have a mix of CAs, you can still use TLS with no --tls-verify.

However, as soon as you want to verify the certs, we simply assume that a proper PKI is in place.