During fresh upgrade, puppetserver fails to start with "Puppet::Error: Cannot determine basic system flavour"

Description

A problem has been found while testing an upgrade from a fresh install of SIMP 6.1.0-0 to SIMP 6.2.0-RC1, using the simp-doc procedure for General Upgrade Instructions > Incremental Updates by following the Verify SIMP server RPM upgrade pre-release checklist :


Failure

After running the unpack_dvd script and yum update, the puppetserver service fails with "Puppet::Error: Cannot determine basic system flavour" (See attachment for a more complete snippet).

  • The error stems from a known issue for puppetserver, especially with EL7. We've encountered it via several vectors during the development of later SIMP 5, early SIMP 6, and with PE.

    • The most recent manifestation of this issue was SIMP-5021—which had a fix merged in July, and is included with SIMP 6.2.0. However, the PR for that fix notes that the problem wasn't reproducible in the developer's test environment.

  • Using automated integration tests developed for SIMP-5223/SIMP-5257, this failure consistently occurs when upgrading fresh 6.1.0 box using an 6.2.0-RC1 ISO. (although a few inconsistencies observed early on suggest that catalog ordering + systemd's holdoff + restars_may_ sometimes mask the problem).


Analysis

As noted (and fixed) in SIMP-5021, older versions of the simp-pupmod module would incorrectly configure the puppetserver to uses the puppet agent's $vardir instead of the master's. This results in the puppetserver process running as user puppet but using -Djava.io.tmpdir=/opt/puppetlabs/puppet/cache/pserver_tmp.

The trouble during the 6.1.0-6.2.0-RC1 upgrade happens after the puppetserver *package is updated. Before the *puppetserver 2.8.1-1.el7 package is installed:

After the puppetserver 2.8.1-1.el7 package is installed:

The reason for the change in ownership of /opt/puppetlabs/puppet/cache is that the puppetserver-2.8.1-1.el7 package requires puppet-agent-1.10.6-1, which resets the permissions to 0750 root:root:


Workaround

For current testers, the following workaround can be applied after the puppetserver service fails to start:

  1. fix the permission of the puppsetserver's puppet resource file /opt/puppetlabs/puppet/cache owner=puppet group=puppet

  2. restart the puppetserver: systemctl restart puppetserver

  3. re-run puppet agent -t, which will now succeed.


Approaches for a permanent fix
  • Ideally, something like Package[$pupmod::master::service] -> pupmod::master::sysconfig ~> Service[$pupmod::master::service] (currently testing).

  • RPM triggers targeted at the %postinstall of specific versions of puppet-agent

  • A helper script, as we used with /usr/share/simp/upgrade_scripts/upgrade_simp_6.0.0_to_6.1.0

  • version-specific upgrade documentation for upgrading to 6.2.0.

Acceptance Criteria

None

Activity

Show:
Chris Tessmer
September 27, 2018, 6:14 PM

Fixed bug in `puppet_settings` fact

Epic Link

None

Story Points

8

Components

Assignee

Chris Tessmer

Sprint

None

Affects versions

Priority

High
Configure