Page Comparison

Info
Add your comments directly to the page. Include links to any relevant research, data, or feedback.

Page Properties

label

Status

colour	Yellow
title	In progress

Impact

Status

colour	Red
title	HIGH

Driver

Chris Tessmer

Approver

Contributors

Trevor Vaughan, Jeanne Greulich, Judith Johnson, Shark Bruhaha, Liz Nemsick, Michael Morrone,

Informed

Brandon Riden, Kendall Moore, Nick Markowski, Dylan Cochran, Steven Pritchard, Nick Miller, Adam Yohrling

Due date

05 April 2019

Outcome

(Sprint 81 is tentatively proceeding with Option 1 + Option 2.5 as a stretch goal)

Info
This log records the results of Sprint 81 planning discussions as of 28 Feb 2019 and 08 Mar 2019. It will remain open for comment until 05 April 2019.

...

Early versions of SIMP: Puppet module RPMs installed directly into the master's $modulepath.
SIMP 5.0(/4.2): To support Puppet environments, SIMP module RPMs installed with Prefix: /etc/puppet/environments/simp/modules. On new installations, the simp/ environment directory was symlinked to production/.
SIMP 6.0: To accommodate environment deployment tools like r10k and Code Manager AND permit the original practice of installing directly into the environment directory where the Puppet Server could find them:
- Puppet module RPMs were modified to install into Prefix: /usr/share/simp/modules
- The simp_rpm_helper script was introduced to centralize the logic for modules' RPM scriptlets
  - Under default conditions, the contents of just-installed Puppet module RPM would be immediately rsynced from /usr/share/simp/modules/ to ${codedir}/environments/simp/modules/.
  - (This behavior depended on the configuration of copy_rpm_data in /etc/simp/adapter_config.yaml, and various safety checks.)
SIMP 6.4: (This decision log)

The

...

`simp_rpm_helper` script

Since the introduction of simp_rpm_helper in 6.0, the script's post-RPM management of ${codedir}/environments/simp/modules/ has been fraught with complications (not least being that most users don't know it's happening). Some RPM edge cases can have dangerous implications in the modules directory, particularly when upgrading obsoleted modules (SIMP-3895, SIMP-5331). Fixing or working around these problems (e.g., trigger scriptlets) can get intensely intricate, especially when combined with the tortuous RPM spec file logic inside simp-rake-helpers and its LUA and ERB assets.

...

Option 1: Puppet module RPMs updates a local git repository Option 2: (Includes Option 1) Maintain a reference Puppetfile(s) to use local git repositories

Description

A local bare git repository is created for each Puppet module:

This is the most basic strategy, and implements the inital steps of an earlier r10k decision:

When a SIMP-packaged Puppet module RPM is installed, the simp_rpm_helper script will:
- ensure a local bare git repository exists at /usr/share/simp/git/puppet_modules/<author>-<module>/
- push and tag a new commit each time a new RPM version of the module is installed
Site admins who wish to manage their Puppet environments using r10k or Code Manager can reference the Puppet modules using the local repositories and tags.
Tagged versions accumulate in the module's local git repository over time

As module RPMs are installed, a reference Puppetfile(s) is kept up-to-date under /usr/share/simp/puppetfiles/:

The reference Puppetfile is modular, with separate SIMP-specific file(s) that can be easily replaced after new module RPMs are delivered. Here is a potential directory structure:

./                # Puppet environment directory, or /usr/share/simp/puppetfiles/current/
│ 
├── Puppetfile    # top-level Puppetfile, includes other Puppetfiles (maintained by site admin)
└── Puppetfiles/  # contains Puppetfiles
    ├── site/     # the local site's Puppetfiles (maintained by site admin)
    └── simp/     # SIMP vendored Puppetfiles. <-- COPY /usr/… UPDATES INTO ENVIRONMENT'S simp/
        │         #   after installing new RPM modules, the updated reference Puppetfiles
        │         #   can be copied into this directory:
        ├── Puppetfile.core           # core SIMP modules
        ├── Puppetfile.extras.nfs     # SIMP modules for NFS
        ├── Puppetfile.extras.gitlab  # SIMP modules for Gitlab 
        └── ...

The contents are automatically updated whenever SIMP-packaged module RPMs update their local git repos

Usage

Installing a SIMP-packaged pupmod-xxx-yyy RPM with the updated simp_rpm_helper script automatically triggers this behavior.
Git tags accumulate in each module's repository.

Initial setup by site admins:
- Site admins copy the contents of /usr/share/simp/puppetfiles/current/ into their own Puppet environment directory (e.g., production).
- Site admins create one or more Puppetfiles in Puppetfiles/site. The set of Puppetfiles contain entries for all of their site-specific modules
- Site admins update the top-level Puppetfile to include their newly created Puppetfiles and the SIMP vendored Puppetfiles appropriate for their site.
After the initial copy, site admins will:
- Continue editing their local copy of Puppetfile and Puppetfiles/site/*
- Update environments to the current (RPM-delivered) versions of all SIMP modules by simply replacing the local simp/* with the contents of /usr/share/simp/puppetfiles/current/Puppetfiles/simp/* .

Related Issues

Jira Legacy

server	System JIRA
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	45e63be1-2551-37a6-b050-86a38172f71d
key	SIMP-6192

Jira Legacy

server	System JIRA
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	45e63be1-2551-37a6-b050-86a38172f71d
key	SIMP-3948

Jira Legacy

server	System JIRA
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	45e63be1-2551-37a6-b050-86a38172f71d
key	SIMP-2942

Jira Legacy

server	System JIRA
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	45e63be1-2551-37a6-b050-86a38172f71d
key	SIMP-6197

Pros and cons

Immediately provides Puppetfile users with tagged git repos for each module

Does not require any external git hosting service

Works with network-isolated server installations

Allows switching modules that have the same name, but different author. This is not possible with SIMP 6.0.x - SIMP 6.3.x, because of RPM spec file bugs.

This option provides a separation between RPM files and Puppet environments

Puppet environments (and their modules) are no longer directly updated after SIMP RPM module upgrades

Non-r10k admins must manually copy the files for each module into the appropriate environment
r10k admins must manually update Puppetfile for each RPM module to reflect the most current RPM installation.
This could be mitigated by adding a legacy option to adapter_config.yaml that rsyncs the RPM-delivered files like SIMP 6.3—however: that has the double disadvantage of maintaining multiple behaviors for each scriptlet message in simp_rpm_helper and not addressing the systemic bugs in the legacy simp_rake_helper's logic (e.g., SIMP-3895, SIMP-5331)

Immediately provides a working Puppetfile that can deploy tagged modules using r10k puppetfile install

Simplifies the move to using control repositories and r10k environment deploy

Site admins decide when/if/how to update their Puppetfiles, and in which environments

Reference Puppetfiles always deploys the current RPM-delivered versions of each module

The Puppetfiles/simp/ directory provides a simple place to drop updated SIMP Puppetfiles.

This option encourages, but does not require (or provide) a git Control Repository.

This option expects site admins to use r10k/Code Manager to deploy modules (if not environments)

Puppet environments (and their modules) are no longer directly updated after SIMP RPM module upgrades

Existing SIMP users that are not fluent on r10k will need documentation/support to allow them to take advantage of the Puppetfiles

Estimated cost

Status


colour	Green
title	SMALL

Status

colour	YELLOW
title	MEDIUM

	Option 2.5: (refines Option 2) Provide a `simp` command to generate a Puppetfile that uses local git repositories	Option 3: (Includes Option 1 + Option 2) Automatically create and maintain a local Control Repository
Description	Provide a `simp` command for users to generate a Puppetfile

o load the current

on-demand. The Puppetfile will reference all RPM-installed Puppet modules from their tagged local git repositories:

Image Modified

The reference Puppetfile loads a separate SIMP-specific Puppetfile.simp that can be easily replaced with the output of later simp gen_puppetfile runs:

./                   # Puppet environment directory
│ 
├── Puppetfile       # Environment's Puppetfile
|                    #   (maintained by site admin, includes Puppetfile.simp)
└── Puppetfile.simp  # Puppetfile of tagged modules from local SIMP git repos
                     #   (generated by `simp gen_puppetfile`;
# tags match versions of currently-installed module RPMs)

Info

This is currently no longer considered a viable option for SIMP 6.4.0.

It is included here to documentation the fact that we considered it.

The decision log "How to restructure the RPM installation sequence to be more r10k/Code Manager friendly" assumed that we would automatically provide a local bare git control repository. The intent was to provide a simple and safe experience for inexperienced Puppet administrators by automatically providing a local repository on their behalf.

However, after exploring different implementation options, it's not clear that this strategy would be simpler or safer.

What has become clear is that most implementations are likely to add more trouble and complexity for site admins and SIMP developers than the idea is worth.

Usage

Initial setup by site admins:

Create/edit the target environment's Puppetfile to include Puppetfile.simp
- (This could automated as an option of simp gen_puppetfile)
Generate Puppetfile.simp with simp gen_puppetfile > ${

environment

env_dir}/Puppetfile.simp

After the initial copy, site admins

will:Install new SIMP Puppet module RPMs

may:

Generate

new

a current Puppetfile.simp with simp gen_puppetfile > ${

environment

env_dir}/Puppetfile.simp

N/A

Related Issues

Jira Legacy

server	System JIRA
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	45e63be1-2551-37a6-b050-86a38172f71d
key	SIMP-6197

N/A

Pros and cons

Less magical and elaborate

and magical

than Option 2

Immediately provides a working Puppetfile that can deploy tagged modules using r10k puppetfile install

Site admins decide when/if/how to update their Puppetfiles, and in which environments

Reference

Generated Puppetfiles always

deploys

deploy the current RPM-delivered versions of each module

The Puppetfiles/simp/ directory provides a simple place to drop updated SIMP Puppetfiles.

Keeps simp_rpm_helper focused on its task of managing post-RPM local git repos

This option encourages, but does not require (or provide) a git Control Repository.

This option expects site admins to use r10k/Code Manager to deploy modules (if not environments)

Puppet environments (and their modules) are no longer directly updated after SIMP RPM module upgrades

Existing SIMP users that are not fluent on r10k will need documentation/support to allow them to take advantage of the Puppetfiles

Sounds convenient when described at a high level

Likely to make too many assumptions

Requires the creation and maintenance of original tooling and logic.

Nullifies the advantages of switching to community-supported deployment tools by adding a layer on top of them.

Makes control repo management complicated and brittle for both admins and SIMP development.

(Re-)confuses the separation between RPM-delivered files, SCM, control repositories, and local SIMP configurations.

Proposals so far have tended to unwittingly re-engineer similar problems to the temporarily-obsoleted module upgrade dilemma that afflicts the original design of SIMP's simp-rpm-helper + module RPMs.

No matter what we build, it will be wrong for someone.

Estimated cost

Status

colour	YELLOW
title	MEDIUM

Status

colour	Red
title	INFINITE

Action items

Outcome

(Working assumption for Sprint 81 as of 08 Mar 2019): Option 1 + Option 2.5 as a stretch goal

...

Versions Compared

Old Version 31

New Version Current

Key

The

`simp_rpm_helper` script

Action items

Outcome

Page Comparison

Versions Compared

Old Version 31

New Version Current

Key

The

simp_rpm_helper script

Action items

Outcome

`simp_rpm_helper` script