After upgrading a (simp-packer-built) EL6 puppetserver from SIMP 6.4.0 to SIMP 6.5.0, puppet agents failed with the error:
The puppetdb service as running correctly, and a separate puppet facts find to the same endpoint succeeded with a 200 HTTP response.
Configuring the F1 pupperserver.log to log at level=INFO confirmed that the the puppetserver's http client was failing its SSL handshake with PuppetDB:
noted that the recent pupmod-simp-pupmod PR #135 added ciphers to the puppetserver configuration.
Confirmed that the corresponding cipher-suites existed in /etc/puppetlabs/puppetdb/conf.d/jetty.ini.rpmnew, but was missing in "/etc/puppetlabs/puppetdb/conf.d/jetty.ini.
After adding the following line to puppetdb's jetty.ini (copied from jetty.ini.rpmnew and restarting the puppetdb and the puppetserver services, puppet agent -t runs started working again:
The first puppet agent run after upgrade adds cipher-suites to puppetserver.conf and webserver.conf and then restarts the puppetserver. It does not add cipher-suites to puppetdb configuration. Any puppet agent run after that fails with the error noted above. This is because there is a mismatch in configuration.
Another oddity that I saw in an EL6 upgrade was that after the yum update of the puppetserver and puppetdb RPMs, the previously running puppetdb and puppetserver services were both in dead states:
puppetdb is dead but pid file exists
puppetserver dead but subsys locked
I do not know what happens with a PE upgrade. Looks like there is a puppet_enterprise module used to configure puppetdb. That module is identified in simp-pupmod's module data. Then, confusingly, puppetdb is managed in simp-simp as well.
Can fix with a change to simp::puppetdb. Not especially DRY to have cipher suites configured in 2 places, but will solve the problem.
The best place to DRY the values is probably at the environment level, by setting one of the cipher suites and reusing it with the other using Hiera's alias() interpolation function.