Currently, the generation of the new CA is inter-mixed with step1 and step2. The initial steps should only focus on issuing a new certificate with the intermediate DNS name as an additional name and then reconfigure all of the agents to use this new DNS name.

Once the fleet is secure, we can run through the migration process to get everyone on a new CA.

Nick F. pointed out node default { } is not sufficient to get the cve20113872 class included in all node catalogs.

It's been suggested we simply include the class outside of any node scope in site.pp.

This is step2.

In the demo this afternoon Nigel asked for a way to help users see if masters are using a certificate that has not been fully migrated. This will be the case up until the end of step3.

The notes I grabbed are:

How do I tell if my Masters are using insecure cert names? Can we have a fact to report the issuer of the puppet master certificate?

The validate_re() functions need to be renamed and copied into the module proper. I'll take care of this.

This decision made by Jeff and Mike after talking about the implications of requiring both modules.

This should be a very unlikely edge case.

The tar command in the create-backup stage fails if any of the files are actually missing.

pe_step1_enable_intermediate_dns_name fails on all Debs and newer Centos systems when incrementing the "idx" counter from 0 to 1-=> returns 1. Oddly, when incrementing from 1 to 2, return code of 0 results, as expected. Spurious bash behavior to be sure -- this runs fine on Centos 5.5. Since the remediation scripts run with "set -e", any non zero status halts the script.

# Make sure certdnsnames are off.  This will prevent the master from issuing
# additional agent certificates that may be used to impersonate the master.
idx=0
echo -n "Making sure certdnsnames are turned off (${puppetconf}) ..." >&2
ruby -p -l -i.backup.${timestamp}.${idx} -e \
  'gsub(/^(\s*)(certdnsnames\b.*$)/) { "#{$1}# Disabled to mitigate CVE-2011-3872\n#{$1}# #{$2}" }' \
  "${puppetconf}"
((idx++))

root@ubnt-1004-64-1:/etc/puppetlabs/puppet/modules/cve20113872/bin# idx=0
root@ubnt-1004-64-1:/etc/puppetlabs/puppet/modules/cve20113872/bin# (( idx++ )); echo $?; echo $idx
1
1
root@ubnt-1004-64-1:/etc/puppetlabs/puppet/modules/cve20113872/bin# (( idx++ )); echo $?; echo $idx
0
2

warning: Unrecognised escape sequence '\.' in file /etc/puppetlabs/puppet/modules/cve20113872/manifests/init.pp at line 44
warning: Unrecognised escape sequence '\.' in file /etc/puppetlabs/puppet/modules/cve20113872/manifests/init.pp at line 46
warning: Unrecognised escape sequence '\.' in file /etc/puppetlabs/puppet/modules/cve20113872/manifests/init.pp at line 48

44 validate_re($agent_localcacert, "^/.*?\.pem$")

46 validate_re($agent_hostcrl, "^/.*?\.pem$")

48 validate_re($agent_pidfile, "^/.*?\.pid$")

I seem to remember geting these a lot whenever I tried passing regexes through to functions in the past, and that they're generally spurious.

https://github.com/puppetlabs/puppetlabs-cve20113872/blob/master/manifests/step4.pp#L80

if ($fact_is_puppetmaster != 'true') {

If someone has deleted the files that this fact relies upon, so the fact is empty, we'll end up applying this to the puppetmaster.

Do we have a tighter test we can apply?

I just ran through and tested this, and it works fine.

Assuming you have a standar puppet.conf resource like:

 file { "/etc/puppet/puppet.conf":
   content => template("puppetagent/puppet.conf.erb"),
   owner   => root,
   group   => root,
   mode    => 0644,
}

all you have to do is this to stop it interfering with the cve remediation process

 file { "/etc/puppet/puppet.conf":
   noop     => true,
   content => template("puppetagent/puppet.conf.erb"),
   owner   => root,
   group   => root,
   mode    => 0644,
}

The only setting we really need to leave disabled is certdnsnames.

At the end of step3, everything else should be "as it was" before starting this migration process.

Specifically, the Apache configuration:
At the end of Step 3, restore the puppetmaster.conf apache configuration to it's original state. This should result in the master ONLY trusting the new CA.

We need to give the user some way to see how much of their fleet has been secured. This is best reported by indicating which agents have switched to the intermediate DNS name in step 2 and which agents have yet to switch over.

The agents are reporting their --server setting using the agent_server fact. This will be evident in the $yamldir/node/ directory on the master.

Currently, the script will only place the include statement inside the default node; if there are no node statements, like with a default PE install, it will say it did something and have no effect on site.pp.

Also, due to the problem with DSL nodes we were discussing earlier (summary: the default node isn't merged with other DSL nodes; it's only applied to a node if that node doesn't have a more specific node statement that fits it), we probably just include the class at top scope in site.pp instead of in a default node statement.

On a RHEL 5.6 x86_64 Puppetmaster the webrick scan_certs script fails with the following error:

./scan_certs

./scan_certs:99:in display': undefined methodreduce' for #Hash:0x2b20c2fef340 (NoMethodError)
from ./scan_certs:68:in run' from ./scan_certs:130:inmain'
from ./scan_certs:145

puppet --version

2.6.9

ruby --version

ruby 1.8.5 (2006-08-25) [x86_64-linux]

Many of our users will have a single CA where every certificate that has ever been issued exists on the filesystem.

We should provide them with a simple tool to scan this whole store for certificates with Subject Alt Names, as well as some descriptive text that outlines what this output means.

e.g.

./scan_certificates.rb
./scan_certificates.rb --ssldir=/path/to/custom/ssl/dir

The following certificates have Subject Alternative Names set.

Certificate Name:     mycertificatename
Subject Alt Names:  [ 'foo', 'bar', 'baz' ]

https://github.com/puppetlabs/puppetlabs-cve20113872/blob/master/manifests/step2.pp#L30

We really just want to validate this isn't an empty string in my opinion.

We have people who run puppet directly as root.

Notes from the demo today. We noticed the certname is still configured to the "old" name even though the CN of the master is the intermediate name.

Puppet.conf needs to have [master] setting update the certname to the intermediate dns name in step1 or step2. This needs to be kept in sync with apache.

This isn't actually required, but it is desirable to keep things sane. If the configuration file says X but the reality is Y it's a problem.

The user needs to be able to reset a screwed up "step1"

From the code review: (18d52f4#commitcomment-654901)

nigelkersten repo collab about 16 hours ago
Edit
Delete
Am I right in thinking there's no problem if people successively run this script and end up with a whole series of CA/CRL certs in their pem file?

jeffmccune repo collab 20 minutes ago
The script is not meant to be run more than once. The results will be unexpected if it is. Suggestions?

nigelkersten repo collab 19 minutes ago
People will mess up this process.

We need to cope with that, either by providing an easy way to reset to the start, or by allowing this script to be run more than once.

My quick suggestion is:

Allow easy way to reset to the start.
Make script quit if it detects that it has already been run.
(assuming that's possible...)

jeffmccune repo collab just now
We mess around with a lot of files. I think it'll be easiest to provide a way to reset and try again.

I'm going to add a section to the beginning of Step1 that makes a backup copy of every file we modify. I then plan to implement a "reset" script that stops the master, restores the files to their places, then starts the master again.

This should provide us with the "oh crap, I messed up, now what?" answer.

Sound good?

If you were using any of the previous alt names in your agent nodes' server setting instead of using the master's exact CN (e.g. with an all-default install of puppet enterprise), the agent nodes will end up orphaned, as they're reaching the master at a DNS name not included in its certificate.

We may not be able to use this module with non-PE installs, but if so, we should at least be calling that out directly.

Using this issue as a record of the PE-dependencies the module itself has:

• PE-specific fact used to determine if the current agent is also a master:
  • https://github.com/puppetlabs/puppetlabs-cve20113872/blob/master/manifests/step4.pp#L80

Step3 - Make sure only the new CA is trusted by all of the agents. Currently, at the end of Step 3 the agents still trust the old CA.

At the end of step3 in the demo today the agent systems still trusted the previous CA. To be secure the agents need to only trust the new CA when they switch away from the intermediate DNS name.

@barn mentioned in the demo this afternoon that we should make a best effort to help the user select an intermediate DNS name that is not in the subject alternate name list.

Jeff will implement this check.

If a user is going to the trouble of "purifying" the master's original CN and alt names (by swapping out the CA and all certs), we should restore the original (and presumably preferred) server setting.

Is the step 4 class a good place to do this? It looks like it, but I might be missing something. For safety, we'd want to do it AFTER installing the new CA cert (and still before HUPping the daemon).

Nick points out the current migration script make an assumption about --certdnsnames working.

The scripts need to be updated to look for a dns_alt_names configuration setting and use that if it's present.

My plan to tackle this is simply to parse puppet master --configprint all | grep dns_alt_names and use the option if that test succeeds.

root@dev1-debian6:~# echo $PATH/var/lib/gems/1.8/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
root@dev1-debian6:~# export PATH="${CVE_PATH:-'/opt/puppet/bin'}:$PATH"root@dev1-debian6:~# echo $PATH
'/opt/puppet/bin':/var/lib/gems/1.8/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

instead:

root@dev1-debian6:~# echo $PATH/var/lib/gems/1.8/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
root@dev1-debian6:~# export PATH="${CVE_PATH:-/opt/puppet/bin}:$PATH"root@dev1-debian6:~# echo $PATH
/opt/puppet/bin:/var/lib/gems/1.8/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

note the quoting change.

I botched the DNS on my test run, and it got me a bunch of worrying errors (and eventually an inconsistent state when I tried to run the step1 script a second and third time). Although Jeff tells me I could have just run puppet agent --test and called it good, even with the errors.

But that's beside the point: if the user's DNS isn't prepared properly, we should just bail immediately.

Right now:

• The comment above the server setting in puppet master's [agent] block just says "name fix."
• The certificate_revocation = false change to agent node puppet.conf files has no comment at all.

Both of these should reference the CVE# if we're planning to leave them in place.

I've completed all the steps, and this is my result:

[root@pe-centos5 modules]# puppet cert list --all --ssldir /etc/puppetlabs/puppet/ssl
+ mco (A6:89:29:61:9E:21:5B:9E:13:4C:03:BD:24:D5:87:F6)
+ mcollective (8D:81:B0:4F:B2:20:09:CF:41:84:AF:C8:C2:E4:C3:42)
+ pe-centos4.localdomain (DA:B6:2A:D7:60:40:4F:BD:9F:D3:CD:1C:D0:0F:A0:21)
+ pe-centos5.localdomain (F0:F4:D9:FE:90:29:7E:81:A1:59:43:CC:AA:BB:4F:A8)
+ pe-centos6.localdomain (2E:E2:F3:81:D0:FC:A0:51:1F:EF:71:BA:47:A1:96:79)

[root@pe-centos5 modules]# puppet cert list --all --ssldir /etc/puppetlabs/puppet/ssl.previous
+ dashboard (16:34:8B:ED:EF:F2:5B:A3:85:D3:EE:C8:99:61:2C:1F)
+ mco (C3:F1:5A:04:D3:2F:AF:F5:C4:51:F7:24:6A:A2:D9:F0)
+ mcollective (89:E7:B9:EB:E5:C1:FF:1B:AC:1D:2B:6A:99:9E:C4:9B)
+ pe-centos4.localdomain (F5:3C:49:5C:32:F9:B6:17:BA:C5:D6:FC:3A:C4:AD:1E)
+ pe-centos5.localdomain (73:8F:3F:03:B4:1D:4D:1E:C5:4B:16:B0:27:A5:90:59)
+ pe-centos6.localdomain (F1:3F:AE:D2:7E:93:B4:44:4D:97:7F:FB:D7:C1:18:E1)

Note that 'dashboard' is now missing. This will break the inventory service and filebucket lookup.

[root@pe-centos5 cve20113872]# ./bin/pe_step3_generate_new_authority 
./bin/pe_step3_generate_new_authority: line 22: ruby: command not found

Looks like we lost the PATH setting in the refactor.

What do you think, do we mind if these classes stick around in site.pp after remediation?

Daniel mentioned in the demo today it would be nice if we provided an optional class to help the user setup name resolution in the situation that DNS changes aren't easy for them.

My notes are:
Help users run step0 - Setup /etc/hosts if the hostmaster is annoying.

The PE scripts have been fixed in #52 but the FOSS scripts need to be updated to not assume puppet cert --certdnsnames and instead detect if puppet cert --dns_alt_names should be used.

This is a placeholder ticket. More information and an ETA will be provided soon.

Daniel pointed out /bin/bash likely has edge cases on Solaris. For the purposes of checking if revocation is enabled or not we should use sh rather than bash.

This is in the cve20113872 class in the exec resource that turns off CRL checking at the bottom of the manifest.

Jeff will do this.