Description

This article is an artifact of:

•  
• .
• 

It tracks:

• Capabilities/files/actions provided by the old (SIMP ≤ 6.3) versions of:
  • simp-environment (6.3.0)
  • simp-adapter (0.1.1)
  • simp-rsync (6.2.1)
  • simp
  • the ISO auto.cfg 
• What / how / if these capabilities will be handled in the new (SIMP ≥ 6.4) ecosystem.

Goals:

• Understand what responsibilities the components in the new SIMP ≥ 6.4 ecosystem will have
  • See article: SIMP 6.4+: Design `simp environment` command
• Ensure no capabilities are missed during the transition
• Document/create issues for anything we don't have a solution for

Brief Summary of SIMP 6.4 

Starting with 6.4, SIMP's philosophy of handling SIMP Environments is changing:

• Don't modify an actual environment, unless explicitly requested by the user
  • This prohibits unexpected automatic environment updates
  • "Explicitly requested by the user" includes the initial simp config after an ISO installation.

Summary of SIMP ≥ 6.4 changes, by component

This section summarizes the changes mapped in the sections above, to collect requirements for each component:

Legend:

WHITE	Mapped, but relevant Jira issues have not been researched
: reason	Not Jira us
YELLOW	Jira issues are still being researched for this mapping
BLUE	Jira issues exist to address this mapping
	PRs have been submitted to address this mapping's Jira Issues
GREEN	Mapping + Jira Issues are complete

simp-cli commands 

`simp environment [COMMAND]` 

	Mapping	Jira Issues	Notes
	A1.2: simp env new|update ENVIRONMENT will copy files from %{prefix} to /var/simp/environments/${ENVIRONMENT}/
	A2.2: simp env new|update ENVIRONMENT runs a post-action step that applies SELinux fixfiles restore to the /var/simp/environments/${ENVIRONMENT}/ + subdirectories	OmniEnvController calls #fix as part of #new:  DirEnv#fix:  SecondaryDirEnv#fix:
	A2.3: simp env fix ENVIRONMENT applies the correct SELinux contexts on demand	simp env fix  DirEnv#fix + SecondaryDirEnv#fix:
	A3.2: (Related: A2.2) simp env new|update ENVIRONMENT runs a post-action step that applies Puppet user settings & groups  to:	OmniEnvController delegates #fix to Env objects:
	A3.2.1: $codedir/environments/$ENVIRONMENT/ (group only)
	A3.2.2: /var/simp/environments/$ENVIRONMENT/site_files/ (group only)
	A3.2.3: /opt/puppetlabs/server/data/puppetserver/simp/environments/$ENVIRONMENT/ (user + group)
	A5.2: The simp env command will ensure the cacertkey during fix ENVIRONMENT and after new|update ENVIRONMENT.	SecondaryDirEnv#new:  SecondaryDirEnv#fix:  : should this be in #fix?
	A6.3: create new Extra/Omni environments on demand with simp environment new.	PuppetDirEnv#new:  SecondaryDirEnv#new:  WritableDirEnv#new:
	A7.2: Environments can be removed with simp env rm ENVIRONMENT
	B3.2: The simp env command will ensure the correct puppet permissions are set during fix ENVIRONMENT and after new|update ENVIRONMENT.  B3.2.1: Note the changes above include several users, paths, and the digest algorithm. B3.2.2: Note that these specific UID/GIDs may no longer be necessary.  Now that the simp command is setting them, it can just check what user is configured to own what files etc.	See A3.2 + children Puppet:  Secondary:  Writable:
	C1.2: The command simp env new|update ENVIRONMENT will copy files from %{prefix}/???/rsync/ to /var/simp/environments/${ENVIRONMENT}/rsync/ See A1.		Related to A1.2, but focuses on rsync/, which will have its own skeleton directory
	C2.1 The command simp env new|update ENVIRONMENT will copy files from %{prefix} to /var/simp/environments/${ENVIRONMENT}/rsync/ This should include any logic needed to ensure a basic DNS environment However, the logic in the (simp-rsync) RPM seems strange	: Look into basic DNS env : Review RPM strangeness
	C3.2: The simp env command will ensure the correct FACLS are set during fix ENVIRONMENT and after new|update ENVIRONMENT.
	C5.2:  The simp env command will ensure the correct symlinks (if any) are managed during  new|update|remove ENVIRONMENT
	D1.2: simp environment command would then install into the appropriate secondary environment directory, when these boot files exist.

`simp config` 

	Mapping	Jira Issues	Notes
	A6.2: move the initial ISO install environment deploy logic into an early stage of simp config	Uncomment OmniEnvContoller#create and integration test (follows )

simp-environment (RPM)

	Mapping	Jira Issues	Notes
	A1.1: simp-environment.spec installs all files to %{prefix}, and does not install files into %{_var}
	A2.1: simp-environment.spec still provides SELinux policies for %{_var} A2.1.1: %build still builds SELinux module A2.1.2: %post (initial install only) applies the initial installs' fixfiles restore to the top-level /var/simp  A2.1.2: %post no longer applies fixfiles for separate subdirectories under /var/simp/environments/  See A2.2	remove policies from simp-environment:  create new RPM:
	A3.1:  %post (initial install only) applies the Puppet user settings & groups at the top-level /var/simp
	A4.1:  (Related: E1) Consolidate %post's yum repo creation logic into the existing script simp-utils:scripts/sbin/updaterepos	simp-environment RPM:   simp-utils RPM:	See:  E1.1
	A4.1.1: The two have slight differences between them that should be carried over into updaterepos	simp-utils RPM:	See: simp-utils RPM/ E1.2
	A4.1.2:  Remove logic from RPM %post		createrepo removed in PR#23 The above pull request removed the yum repo stuff from simp-env but another ticket was opened to do the work in simp-utils.
	A5.1: remove cacertkey ensure logic from %post		patched in PR#22
	A6.1: remove `/usr/local/sbin/simp_rpm_helper --rpm_dir=%{prefix} --rpm_section='post' --rpm_status=$1 --preserve --target_dir='.'` from %post		patched in PR#22
	A7.1: remove `/usr/local/sbin/simp_rpm_helper --rpm_dir=%{prefix} --rpm_section='post' --rpm_status=$1 --preserve --target_dir='.'` from %postun		patched in PR#22

simp-adapter (RPM)

	Mapping	Jira Issues	Notes
	B1: keep this for now? (see discussion)		PR#15, undone by PR#17
	B2.1: remove this behavior from %post
	B3.1: Remove this behavior from %post	Incorporated into


simp-rsync (RPM)

	Mapping	Jira Issues	Notes
	C1.1.1: Change Prefix: + %{prefix}to /usr/share/simp/environment_templates/rsync/
	C1.1.2: Change any references to %{rsync_dir} from all RPM scriptlets
	C2.2: Remove OBE %pre logic
	C3.1:  Move rsync.facl into %{prefix} ( /usr/share/simp/... ), like the rest of the %files
	C3.3: Remove from %post
	C4.1: Remove *.rpmnew deletion logic from %post
	C5.1: Remove OS symlink logic from %post (code) and %preun (code)		still in 7.0.0-0: %post %preun

simp-core (ISO)

	Mapping	Jira Issues	Notes
	D1.1: Remove this logic.  No extra copy of files already located in /var/www/yum needed.
	D1.2: simp environment command would then install into the appropriate secondary environment directory, when these boot files exist.	: As of 5/14, this is debatable—it may install into the secondary environment skeleton

simp-core (RPM)

	Mapping	Jira Issues	Notes
	D2.1: Remove obsolete hiera_upgrade script

simp-utils (RPM)

	Mapping	Jira Issues	Notes
	E1.1:  Consolidate %post's yum repo creation logic into the existing script simp-utils:scripts/sbin/updaterepos. (See A4.1)
	E1.2: The two have slight differences between them that should be carried over into updaterepos

Mapped capabilities: SIMP ≤ 6.3 to SIMP ≥ 6.4

For each component:

• Identify relevant SIMP ≤ 6.3 capabilities
• Assign each capability an Alpha+Number id (A1, A2) 
  • incomplete ids (or ids with questions) are orange, completely-mapped ids are green.
• Map SIMP ≤ 6.3 capabilities to SIMP ≥ 6.4
  • Add mappings as id+Number (A1.1, A1.2, ...)

simp-environment (6.3.0)  [Analyzed, Mapped: 7/7, Questions: 0]

• A1.) simp-environment.spec essentially installs to two filesystem roots:
  • SIMP ≤ 6.3 simp-environment 6.3.0:
    • %{prefix} ( /usr/share/simp/environments/simp/ )
    • %{_var} ( used to manage %{_var}/simp/environments/simp/ )
      
  • SIMP ≥ 6.4 Mappings:
    • A1.1: simp-environment.spec installs all files to %{prefix}, and does not install files into %{_var}                                                                                                     
      
    • A1.2: simp env new|update ENVIRONMENT will copy files from %{prefix} to /var/simp/environments/${ENVIRONMENT}/ 


• A2.)  Provides SELinux policies for different /var/simp/ subdirectories so the various services can access files
  • SIMP ≤ 6.3 simp-environment 6.3.0:
    • On %build, builds SELinux module
      • Defines paths and contexts in simp-environment.fc
      • Defines module in  simp-environment.te
    • On %post, builds + loads an SELinux policy and runs fixfiles so puppet can read from /var/simp directories
    • On %uninstall, removes SELinux policy and runs fixfiles
  • SIMP ≥ 6.4 Mappings:
    • A2.1: simp-environment.spec still provides SELinux policies for %{_var}
      • A2.1.1: %build still builds SELinux module
      • A2.1.2: %post (initial install only) applies the initial installs' fixfiles restore to the top-level /var/simp 
      • A2.1.2: %post no longer applies fixfiles for separate subdirectories under /var/simp/environments/ 
        • See A2.2
      • A2.1.3: What should %uninstall do? Nothing special is required.
    • A2.2: simp env new|update ENVIRONMENT runs a post-action step that applies SELinux fixfiles restore to the /var/simp/environments/${ENVIRONMENT}/ + subdirectories 
    • A2.3: simp env fix ENVIRONMENT reapplies the correct SELinux context on demand
      
      
• A3.)  Sets Puppet user & group permissions based on the settings in puppet config print
  • SIMP ≤ 6.3 simp-environment 6.3.0:
    • Affects %{prefix} and %{_var} directory trees
      
• SIMP ≥ 6.4 Mappings:
  • A3.1:  %post (initial install only) applies the Puppet user settings & groups at the top-level /var/simp 
  • A3.2: (Related: A2.2) simp env new|update ENVIRONMENT runs a post-action step that applies Puppet user settings & groups  to
    • A3.2.1: $codedir/environments/$ENVIRONMENT/ (group only)
    • A3.2.2: /var/simp/environments/$ENVIRONMENT/site_files/ (group only)
    • A3.2.3: /opt/puppetlabs/server/data/puppetserver/simp/environments/$ENVIRONMENT/ (user + group)

• A4.)  %post creates a yum repo directory structure and runs createrepo under /var/www/yum/${os}/...: 
  •  This does not affect the secondary SIMP environment directories; it's under /var/www/yum
  •  It's not clear why this logic is in simp-environment.spec, which doesn't seem to deliver these files.
  • SIMP ≥ 6.4 Mappings (None):
    • A4.1:  (Related: E1) Consolidate %post's yum repo creation logic into the existing script simp-utils:scripts/sbin/updaterepos.
      • A4.1.1: The two have slight differences between them that should be carried over into updaterepos
      • A4.1.2:  Remove logic from RPM %post
• A5.) On %post, ensures that the cacertkey (at %{_var}/simp/environments/simp/FakeCA/cacertkey) has some random gibberish in it if it doesn't exist. 
  • SIMP ≥ 6.4 Mappings:
    • A5.1: remove this behavior from %post
    • A5.2: The simp env command will ensure the cacertkey during fix ENVIRONMENT and after new|update ENVIRONMENT.
      
      
• A6.) On %post, runs simp_rpm_helper 
  • SIMP ≤ 6.3 simp-environment 6.3.0:
    • Comment: # Needed for migrating the environment data into the codedir for an initial install
    • /usr/local/sbin/simp_rpm_helper --rpm_dir=%{prefix} --rpm_section='post' --rpm_status=$1 --preserve --target_dir='.'
  • SIMP ≥ 6.4 Mappings:
    • A6.1: remove this behavior from %post

    • A6.2: move the initial ISO install environment deploy logic into an early stage of simp config .
    • A6.3: create new Extra/Omni environments on demand with simp environment new.
• A7.)  On %postun, runs simp_rpm_helper 
  • SIMP ≤ 6.3 simp-environment 6.3.0:
    • Comment: # Needed for cleaning up the data from codedir as appropriate for an erase
    • /usr/local/sbin/simp_rpm_helper --rpm_dir=%{prefix} --rpm_section='postun' --rpm_status=$1 --preserve --target_dir='.'
  • SIMP ≥ 6.4 Mappings:
    • A7.1: Remove from %postun

    • A7.2: Environments can be removed with simp env rm ENVIRONMENT

simp-adapter (0.1.1)  [Analyzed, Mapped: 3/3, Questions: 0]

• B1.) simp-adapter 0.1.1 %pre and %posttrans have operations to prevent the global Hiera 3 hiera.yaml.simp file delivered with simp-adapter <= 0.0.6 and its hiera.yaml link created in that RPM's %post from being removed during upgrade if it may be in use.
  • See the simp-adapter 0.1.1 acceptance tests for details on when hiera.yaml files (global and environment) are modified during install/upgrade:
    • Upgrading simp-adapter from version <= 0.0.6
      • When global hiera.yaml is linked to hiera.yaml.simp
        • it should retain hiera.yaml and hiera.yaml.simp
      • When global hiera.yaml is not linked to hiera.yaml.simp
        • it should remove hiera.yaml.simp but not remove hiera.yaml
    • Uninstalling simp-adapter and legacy global Hiera 3 config exists
      • When global hiera.yaml is linked to hiera.yaml.simp
        • it should remove hiera.yaml and hiera.yaml.simp, but keep hiera.yaml.simpbak
      • When global hiera.yaml is not linked to hiera.yaml.simp
        • it should remove hiera.yaml.simp, but keep hiera.yaml and hiera.yaml.simpbak
  • SIMP ≥ 6.4 Mappings:
    • This logic should be retained.

• B2.)  simp-adapter 0.1.1 %post (install-only) logic (removed after 0.1.1) creates /etc/simp/adapter_config.yaml:
  • SIMP ≤ 6.3 simp-environment 6.3.0

    • Under the following conditions:

      • Only during a new RPM install ($1 -eq 1)

      • Only when the Linux kernel contained simp install (e.g., a SIMP ISO installation)

    • %post will create %{prefix}/adapter_config.yaml with the following content:

      # This file was modified by simp-adapter during a SIMP install
# on ${date}:
target_directory: 'auto'
copy_rpm_data: true

  • SIMP ≥ 6.4 Mappings:
    • B2.1: remove this behavior from %post
      
      
• B3.) simp-adapter 0.1.1 %post logic fixes problems with Puppet RPMs:
  • SIMP ≤ 6.3 simp-environment 6.3.0
    • For FOSS puppetserver (install or upgrade)
      • Nails up puppet user UID and GID to 52, changing the UID/GID if the installed ids are incorrect.
      • Fixes permissions of /opt/puppetlabs/, /etc/puppetlabs, /var/log/puppetlabs/, and /var/run/puppetlabs/ to match the nailed UID/GID
      • Restarts puppetserver to pick up changes.
      • Creates puppdetdb user and group
    • Sets the digest algorithm used by puppet to sha256 (install or upgrade)
    • Install-only, fixes the permissions of puppet-agent, puppetserver, and puppetdb directories
      • References https://tickets.puppetlabs.com/browse/PA-726, which is still in the ACCEPTED state but has no Fix version assigned
  • SIMP ≥ 6.4 Mappings:
    • B3.1: Remove this behavior from %post
    • B3.2: The simp env command will ensure the correct puppet permissions are set during fix ENVIRONMENT and after new|update ENVIRONMENT. 
      • B3.2.1: Note the changes above include several users, paths, and the digest algorithm.
      • B3.2.2: Note that these specific UID/GIDs may no longer be necessary.  Now that the simp command is setting them, it can just check what user is configured to own what files etc.

simp-rsync (6.2.1) [Analyzed, Mapped:6/6, Questions: 1]

• C1.) RPM delivers a directory tree directly under /var/simp/environments/simp/rsync/
  • SIMP ≤ 6.3 simp-rsync 6.2.1:
    • %global rsync_dir /var/simp/environments/simp/rsync
    • Prefix: %{rsync_dir}
    • %files delivers rsync facls file to  %{rsync_dir}/.rsync.facl as a %config file
    • Note: many RPM scriptlets only refer to %{rsync_dir}, which breaks when rpm --prefix=NEWPATH
  • SIMP ≥ 6.4 Mappings:
    • C1.1:  The simp-rsync RPM only delivers files to %{prefix}, which is now /usr/share/simp/environment_templates/rsync/
      • C1.1.1: Change Prefix: + %{prefix} to /usr/share/simp/environment_templates/rsync/
      • C1.1.2: Change any references to %{rsync_dir} from all RPM scriptlets 
    • C1.2: The command simp env new|update ENVIRONMENT will copy files from %{prefix} to /var/simp/environments/${ENVIRONMENT}/rsync/
      • See A1.2
        
        
• C2.) %pre has strange logic
  • SIMP ≤ 6.3 simp-rsync 6.2.1:
    • Pre-removes directories under {%rsync_dir}/ ("# Remove the directories that we're going to replace with symlinks")

    •   Includes a "# Make sure upgrades work properly!" section that looks like it might try to symlink the LICENSE file as a directory if the default directory is missing under bind_dns/.

      • This looks like a bug in the logic
  • SIMP ≥ 6.4 Mappings:
    •  (nice-to-have, for later): Should we expose arguments to template the skeleton DNS environment when it is copied/updates?
    • C2.1: The command simp env new|update ENVIRONMENT will copy files from %{prefix} to /var/simp/environments/${ENVIRONMENT}/rsync/
      • This should include any logic needed to ensure a basic DNS environment
    • C2.2: Remove %pre logic

• C3.) %post runs setfacl --restore on {%rsync_dir}, using the delivered .rsync.facl file.
  • SIMP ≤ 6.3 simp-rsync 6.2.1:
    
    • %{rsync_dir} = /var/simp/environments/simp/rsync
    • The rsync.facl definitions use relative paths
  • SIMP ≥ 6.4 Mappings:
    •  The simp-rsync RPM only delivers files to %{prefix} (/usr/share/simp/environment_templates/rsync/); see C1.1
    • C3.1:  Move rsync.facl into %{prefix} (/usr/...), like the rest of the %files
    • C3.2: The simp env command will ensure the correct FACLS are set during fix ENVIRONMENT and after new|update ENVIRONMENT.
    • C3.3: Remove from %post
• C4.) %post deletes all *.rpmnew files under {%rsync_dir}/
  • SIMP ≤ 6.3 simp-rsync 6.2.1:
    • This was a workaround to problems associated with C1 (delivering files to /var/simp/environments/simp/rsync/ as %config)
      • Example problem: rsyncing meaningless *.rpmnew files to hosts that have no use for them
  • SIMP ≥ 6.4 Mappings:
    •  The SIMP ≤ 6.3 workaround should not be necessary now because:
      • the RPM delivers a skeleton to somewhere under /usr/share/simp/, which is not an rsync source.
      • the files no longer need to be %config, so .rpmnew files won't get generated, anyway
    • C4.1: Remove logic from %post
      • Prerequisites: C1.1 and C1.2
        
        
• C5.) RPM scriptlets auto-ensure that all directories called RedHat have a CentOS symlink under /var/simp/environments/simp/rsync
  • SIMP ≤ 6.3 simp-rsync 6.2.1:
    • %post symlinks RedHat directories to CentOS under {%rsync_dir}/
      •  The scriptlets will add a symlink to any directory named RedHat, regardless of its location (buggy logic)
    • %preun deletes any symlinks named CentOS under {%rsync_dir}/
      •  The scriptlets will delete a CentOS symlink in any directory named RedHat, regardless of its subdirectory (buggy logic)
  • SIMP ≥ 6.4 Mappings:
    • C5.1: Remove logic from %post and %preun 
      • Prerequisites: C1.1 and C1.2
        
    • C5.2: The simp env command will ensure the correct symlinks (if any) are managed during  new|update|remove ENVIRONMENT.
      
      
• C6.) RPM scriptlets auto-ensure that all directories called RedHat have a CentOS symlink under /var/simp/environments/simp/rsync 
  • SIMP ≥ 6.4 Mappings:
    • See C5.1

simp-core (6.3.3) [Analyzed, Mapped: 2/2]

• D1.) ISO auto.cfg
  • SIMP ≤ 6.3 simp-core 6.3.3:
    • delivers boot files required for TFTPboot into /var/simp/environments/simp/rsync/
  • SIMP ≥ 6.4 Mappings:
    • D1.1: Remove this logic.  No extra copy of files already located in /var/www/yum needed.
    • D1.2: simp environment command would then install into the appropriate secondary environment directory, when these boot files exist.
• D2.)  %post in simp.spec
  • SIMP ≤ 6.3 simp-core 6.3.3:

    • Includes obsolete logic to run the (defunct) `hiera_upgrade` script  under certain conditions (-d ${puppet_environmentpath}/simp)

      • This logic will never run; hiera_upgrade script was been gone for two years
        • added to simp-core in 2015 to support SIMP 5.1.X upgrades
        • removed from simp-utils in 2017 as part of SIMP 6.1.0
  • SIMP ≥ 6.4 Mappings:
    • D2.1: Remove obsolete hiera_upgrade script (SIMP-6507)

simp-utils

• E1.) The script simp-utils:scripts/sbin/updaterepos 

  • (mostly) duplicates the %post yum repo createrepo logic in simp-environment.

  • SIMP ≥ 6.4 Mappings:
    • E1.1:  Consolidate %post's yum repo creation logic into the existing script simp-utils:scripts/sbin/updaterepos.
      • See A4.1 
    • E1.2: The two have slight differences between them that should be carried over into updaterepos

Further steps

What happens during an upgrade?

Reference

• SIMP Environments — overview & terminology of "Environments" within SIMP
• Puppet deployment scenarios — use cases SIMP supports for deploying Puppet modules and environments