SIMP 6.4.0: What r10k-friendly RPM features should it include?

Background

• SIMP 6.4.0 will begin to treat the r10k/Code Manager workflow as a first-class citizen.
• Puppet modules are distributed as RPMs, signed by a known GPG key as part of SIMP's implementation of NIST 800.53, CM-7(5).
• Modern Puppet module management practices use git and r10k, a model that does not fit well with RPM-distribution.

(Distant background:)

• SIMP supports installations in offline/network-isolated enclaves.
• Installation by ISO is intended to be an easy way to get started and ensure that all files are present (including Puppet modules).

Relevant data

Evolution of SIMP-packaged Puppet module RPMs

• Early versions of SIMP: Puppet module RPMs installed directly into the master's $modulepath.
• SIMP 5.0(/4.2): To support Puppet environments, SIMP module RPMs installed with Prefix: /etc/puppet/environments/simp/modules.  On new installations, the simp/ environment directory was symlinked to production/.  
• SIMP 6.0: To accommodate environment deployment tools like r10k and Code Manager AND permit the original practice of installing directly into the environment directory where the Puppet Server could find them:
  • Puppet module RPMs were modified to install into Prefix: /usr/share/simp/modules
  • The simp_rpm_helper script was introduced to centralize the logic for modules' RPM scriptlets
    • Under default conditions, the contents of just-installed Puppet module RPM would be immediately rsynced from /usr/share/simp/modules/ to ${codedir}/environments/simp/modules/.
    • (This behavior depended on the configuration of copy_rpm_data in /etc/simp/adapter_config.yaml, and various safety checks.)
• SIMP 6.4: (This decision log)

The simp_rpm_helper script

Since the introduction of simp_rpm_helper in 6.0, the script's post-RPM management of ${codedir}/environments/simp/modules/ has been fraught with complications (not least being that most users don't know it's happening).  Some RPM edge cases can have dangerous implications in the modules directory, particularly when upgrading obsoleted modules (SIMP-3895, SIMP-5331).  Fixing or working around these problems (e.g., trigger scriptlets) can get intensely intricate, especially when combined with the tortuous RPM spec file logic inside simp-rake-helpers and its LUA and ERB assets. 

Most of these complications can be avoided if the simp_rpm_helper script's post-deploy is changed to maintain a version-collecting local git repository (Option 1, below).  Tagged versions accumulate in a module's local git repository and are never lost, and it is easy to add them to a Puppetfile and deploy with r10k, even from an offline monolithic ISO installation.  However, once we switch over to using local repositories with Puppetfile support, there are similar decisions to be made:

• Should we require r10k/Code Manager to deploy Puppet modules?
• Should we continue to automatically copy modules into ${codedir}/environments/simp/modules/ ?
  • If so, under what circumstances is this appropriate (or not)?
• How far is too far in terms of developing and maintaining assistive tooling to automatically support more advanced r10k assets?
  • Should we provide Puppetfiles?
  • Should we maintain a control repo?
  • Should we create an initial control repo on the file system when SIMP is initially installed from ISO or RPMs and an 'simp' environment does not already exist?
• How would that be implemented?

Options considered

./                # Puppet environment directory, or /usr/share/simp/puppetfiles/current/
│ 
├── Puppetfile    # top-level Puppetfile, includes other Puppetfiles (maintained by site admin)
└── Puppetfiles/  # contains Puppetfiles
    ├── site/     # the local site's Puppetfiles (maintained by site admin)
    └── simp/     # SIMP vendored Puppetfiles. <-- COPY /usr/… UPDATES INTO ENVIRONMENT'S simp/
        │         #   after installing new RPM modules, the updated reference Puppetfiles
        │         #   can be copied into this directory:
        ├── Puppetfile.core           # core SIMP modules
        ├── Puppetfile.extras.nfs     # SIMP modules for NFS
        ├── Puppetfile.extras.gitlab  # SIMP modules for Gitlab 
        └── ...

./                   # Puppet environment directory
│ 
├── Puppetfile       # Environment's Puppetfile
|                    #   (maintained by site admin, includes Puppetfile.simp)
└── Puppetfile.simp  # Puppetfile of tagged modules from local SIMP git repos
                     #   (generated by `simp gen_puppetfile`;
# tags match versions of currently-installed module RPMs)

Action items

Outcome

(Working assumption for Sprint 81 as of 08 Mar 2019): Option 1 + Option 2.5 as a stretch goal