How to restructure the RPM installation sequence to be more r10k/Code Manager friendly

Background

The current RPM method, using an rsync-based simp-adapter, for installing the SIMP puppet modules began as a reasonable solution. However, r10k and the PE Code Manager have solidified as the de-facto method for installing puppet modules onto systems.  As such, while we have attempted to be careful, we have found that our current installation method is lacking in sustainability with an r10k workflow and it does not encourage users to migrate to a more DevOps-friendly workflow in the future.

The goal of this decision is to determine a path forward for release in 6.4.0.

Please note that it is understood that the proposed solutions may be complex. The complexity is generally present to give users a seamless path towards an operational system.

Please ensure that feedback is constructive in nature and provides a proposed solution to the perceived issues.

Relevant data

The new system should meet the following requirements:

1. Does not assume that any infrastructure besides the initial SIMP server exists
   1. This includes the Internet
      
      
2. Supports validation of installed components for auditing purposes
   1. This is the original reason that RPMs were rolled
      
      
3. Is able to be understandably migrated to an r10k system as the user desires
   
   
4. Does not destroy existing environments
   1. This can be achieved using the deployment option for purge_levels in the configuration: https://github.com/puppetlabs/r10k/blob/master/doc/dynamic-environments/configuration.mkd#deployment
      
      
5. Cannot run "unnecessary services"
   1. Discussion may dictate that new services are necessary but each service that we add is an additional level of documentation, justification, and maintenance that is added to the stack.

Options considered

Option 1 - RPMs with Local r10k

Description

This model works around the concept that users should always be able to revert to any given module that has been installed by RPM at any time and that the entire workflow should be based around a standard Puppetfile. The impact to the current system is minimal in that only the simp-adapter will really need to change. All other components can use the same installation methods and hooks as they always have.

Module Installation

The idea is that a module installation will use the following procedure:

1. RPM installation to /usr/share/simp/modules/<author>/<modulename>
   1. This will always contain the latest version of the module by that author
      
      
2. Trigger of simp-adapter that will cause the following actions to be performed:
   1. Create /usr/share/simp/git if it does not exist
      1. This is our "local git repo" for lack of other options
   2. Create /usr/share/simp/git/puppet_modules if it does not exist
      1. This is the housing of all puppet modules that get installed
   3. Create a bare git repository at /usr/share/simp/git/puppet_modules/<name from metadata.json> if it does not exist
   4. Update the master branch of the repository to be the contents of the RPM
   5. Add a git tag to the repository that matches the version number in the module's metadata.json file
   6. Run an r10k deploy on the simp environment (or configured environment of your choice) if configured to do so
      1. This would use the simp-vendored-r10k packages since no other ones are guaranteed to actually exist on the system.
      2. PE installations would be configured to use puppet code deploy by default
         
         
3. Upon RPM removal, the git repository will remain so that environments do not break.
   
   

Example:

[root@host ~]# yum install pupmod-simp-issue
[root@host ~]# ls /usr/share/simp/modules/simp/issue/
CHANGELOG  Gemfile  LICENSE  README.md  Rakefile  files  manifests  metadata.json  spec
[root@host ~]# ls /usr/share/simp/git/puppet_modules/pupmod-simp-issue/
HEAD  branches  config  description  hooks  info  objects  refs
[root@host ~]# git clone /usr/share/simp/git/puppet_modules/pupmod-simp-issue
Cloning into 'pupmod-simp-issue'...
done.
[root@host ~]# ls pupmod-simp-issue/
CHANGELOG  Gemfile  LICENSE  README.md  Rakefile  files  manifests  metadata.json  spec
[root@host ~]# cd pupmod-simp-issue/
[root@host pupmod-simp-issue]# git log
commit e4d4e475cea8c2be590b6ac158fc998cc8dee293
Author: SIMP Adapter <simp_adapter@localhost>
Date:   Fri Jan 11 22:08:33 2019 +0000

    Import of 0.0.4
[root@host pupmod-simp-issue]#  git tag
0.0.4

Control Repository Installation

This is a bit more difficult since control repositories tend to be quite personal to the user and require more live malleability.

The proposal for a local control repository follows the general convention of having "hands off" materials in /usr and variable materials in /var. In this case, a directory at /var/simp/git would correlate with the /usr/share/simp/git space for items that are not to be modified (not saying that they can't be modified, just that they should not be).

The SIMP control repository would be housed at /usr/share/simp/git/control_repository and the simp RPM itself would take responsibility for updating this in a manner similar to that of the puppet modules and simp-adapter. The major difference here is that utmost care must be taken to not surprise the user with new environments or updates.

The exact workflow for how this will work will probably need to be based on usability experiments once we get past the general idea outlined in this section.

• Tags prefaces with simp- will be ours to manage authoritatively and will be documented as such
• The production and simp environments will be created if they do not exist and will be matched to each other out of the box
  • Unsure if we should attempt clean updates to the environments or if we should always make the user perform the update via git
• Like modules, tags will be made each time SIMP is updated and will contain a clean copy of the release content

Example:

[root@host ~]# yum install simp-6.4.0-0
...
Installed:
  simp.x86_64 0:6.4.0.el7
[root@host ~]# ls /var/simp/git/control_repository/
HEAD  branches  config  description  hooks  info  objects  refs
[root@host ~]# git clone /var/simp/git/control_repository -b production
Cloning into 'control_repository'...
done.
[root@host ~]# cd control_repository
[root@host control_repository]# ls
LICENSE  Puppetfile  README.md  data  environment.conf  hiera.yaml  manifests
[root@host control_repository]# cat Puppetfile
# Ideally this can simply be changed to the address of the git server
# (even the Internet servers) if desired
module_source = '/usr/share/simp/git'
...
mod 'simp-simp',
  :git => "#{module_source}/pupmod-simp-simp",
  :tag => '4.6.0'

mod 'simp-simplib',
  :git => "#{module_source}/pupmod-simp-simplib",
  :tag => '3.11.1'
...
[root@host control_repository]# git tag
simp-6.4.0-0
[root@host control_repository]# git branch -v --all
* production                0cfacbc SIMP 6.4.0
  remotes/origin/production 0cfacbc SIMP 6.4.0
  remotes/origin/simp       0cfacbc SIMP 6.4.0

Pros

Allows users to move everything to a git server, by simply bundling up the repos and importing them to their system of choice

 The transition to git and r10k is transparent to users who simply follow the latest SIMP releases

Follows current best practice in terms of control repository deployment

Gives users a 'soft' approach to getting used to advanced system management

Cons

Is relatively complex (though less mysterious than the current method since we are using community-understood mechanisms)

A lot of precise and tested documentation is going to need to be provided for working with this workflow. This is probably the case no matter what we do if we move to r10k since it now requires some sort of git-based workflow.

Action items

• Chris Tessmer08 Feb 2019 Flesh out and re-upload Option 2

Outcome

After a month waiting for input and various side discussions, we'll move forward with attempting Option 1. There was a partially completed Option 2 that we can assess once it is provided.