SIMP 6.4: Map old/new files and capabilities


Description

This article is an artifact of:

It tracks:

  • Capabilities/files/actions provided by the old (SIMP ≤ 6.3) versions of:
  • What / how / if these capabilities will be handled in the new (SIMP ≥ 6.4) ecosystem.

Goals:

  • Understand what responsibilities the components in the new SIMP ≥ 6.4 ecosystem will have
  • Ensure no capabilities are missed during the transition
  • Document/create issues for anything we don't have a solution for


Brief Summary of SIMP 6.4 

Starting with 6.4, SIMP's philosophy of handling SIMP Environments is changing:

Summary of SIMP ≥ 6.4 changes, by component

This section summarizes the changes mapped in the sections above, to collect requirements for each component:

Legend:

WHITEMapped, but relevant Jira issues have not been researched 


ISSUE TBD: reason

Not Jira us
YELLOWJira issues are still being researched for this mapping
BLUEJira issues exist to address this mapping
(tick)PRs have been submitted to address this mapping's Jira Issues
(tick) GREENMapping + Jira Issues are complete


simp-cli commands 

`simp environment [COMMAND]

Related article



MappingJira IssuesNotes

A1.2: simp env new|update ENVIRONMENT will copy files from %{prefix} to /var/simp/environments/${ENVIRONMENT}/ 

SIMP-6419 - Getting issue details... STATUS

SECONDARY ENV
(tick)

A2.2: simp env new|update ENVIRONMENT runs a post-action step that applies SELinux fixfiles restore to the /var/simp/environments/${ENVIRONMENT}/ + subdirectories 

  • OmniEnvController calls #fix as part of #new SIMP-6345 - Getting issue details... STATUS
    • DirEnv#fix SIMP-6408 - Getting issue details... STATUS
      • SecondaryDirEnv#fix: SIMP-6417 - Getting issue details... STATUS

OMNI CONTROLLER

SECONDARY ENV

(tick)

A2.3simp env fix ENVIRONMENT applies the correct SELinux contexts on demand  

  • simp env fix  SIMP-6408 - Getting issue details... STATUS
  • DirEnv#fix + SecondaryDirEnv#fix:  SIMP-6417 - Getting issue details... STATUS
SECONDARY ENV
(tick)

A3.2: (Related: A2.2simp env new|update ENVIRONMENT runs a post-action step that applies Puppet user settings & groups  to:

  • OmniEnvController delegates #fix to Env objects:  SIMP-6345 - Getting issue details... STATUS
OMNI CONTROLLER
(tick)
  • A3.2.1$codedir/environments/$ENVIRONMENT/ (group only)
SIMP-6416 - Getting issue details... STATUS
PUPPET ENV
(tick)
  • A3.2.2/var/simp/environments/$ENVIRONMENT/site_files/ (group only)
SIMP-6417 - Getting issue details... STATUS
SECONDARY ENV
(tick)
  • A3.2.3: /opt/puppetlabs/server/data/puppetserver/simp/environments/$ENVIRONMENT(user + group)
SIMP-6422 - Getting issue details... STATUS
WRITABLE ENV (error)
(tick)

A5.2: The simp env command will ensure the cacertkey during fix ENVIRONMENT and after new|update ENVIRONMENT.

  • SecondaryDirEnv#new SIMP-6419 - Getting issue details... STATUS
  • SecondaryDirEnv#fix: 
    • ISSUE TBD: should this be in #fix?
SECONDARY ENV
(tick)

A6.3: create new Extra/Omni environments on demand with simp environment new.

  • PuppetDirEnv#new:  SIMP-6418 - Getting issue details... STATUS
  • SecondaryDirEnv#new:  SIMP-6419 - Getting issue details... STATUS
  • WritableDirEnv#new:  SIMP-6421 - Getting issue details... STATUS

PUPPET ENV

SECONDARY ENV

WRITABLE ENV 


A7.2: Environments can be removed with simp env rm ENVIRONMENT

SIMP-6612 - Getting issue details... STATUS

PUPPET ENV

SECONDARY ENV


(tick)

B3.2: The simp env command will ensure the correct puppet permissions are set during fix ENVIRONMENT and after new|update ENVIRONMENT

  • B3.2.1: Note the changes above include several users, paths, and the digest algorithm.

  • B3.2.2: Note that these specific UID/GIDs may no longer be necessary.  Now that the simp command is setting them, it can just check what user is configured to own what files etc.

DUPLICATE See A3.2 + children

  • Puppet:  SIMP-6416 - Getting issue details... STATUS
  • Secondary:  SIMP-6417 - Getting issue details... STATUS
  • Writable:  SIMP-6422 - Getting issue details... STATUS


PUPPET ENV

SECONDARY ENV

WRITABLE ENV (error)

(tick)

C1.2: The command simp env new|update ENVIRONMENT will copy files from %{prefix}/???/rsync/ to /var/simp/environments/${ENVIRONMENT}/rsync/

SIMP-6419 - Getting issue details... STATUS

SECONDARY ENV

Related to A1.2, but focuses on rsync/, which will have its own skeleton directory

(tick)

C2.1 The command simp env new|update ENVIRONMENT will copy files from %{prefix} to /var/simp/environments/${ENVIRONMENT}/rsync/

  • SIMP-6419 - Getting issue details... STATUS
  • ISSUE TBD: Look into basic DNS env
  • ISSUE TBD: Review RPM strangeness
SECONDARY ENV
(tick)C3.2: The simp env command will ensure the correct FACLS are set during fix ENVIRONMENT and after new|update ENVIRONMENT.
SIMP-6417 - Getting issue details... STATUS


SECONDARY ENV


C5.2:  The simp env command will ensure the correct symlinks (if any) are managed during  new|update|remove ENVIRONMENT SIMP-6620 - Getting issue details... STATUS SECONDARY ENV
(tick)D1.2: simp environment command would then install into the appropriate secondary environment directory, when these boot files exist. SIMP-6621 - Getting issue details... STATUS SECONDARY ENV


`simp config


MappingJira IssuesNotes
(tick)

A6.2move the initial ISO install environment deploy logic into an early stage of simp config 

Uncomment OmniEnvContoller#create and integration test SIMP-6533 - Getting issue details... STATUS

  • (follows  SIMP-6418 - Getting issue details... STATUS )


simp-environment (RPM)


MappingJira IssuesNotes
(tick)

A1.1: simp-environment.spec installs all files to %{prefix}, and does not install files into %{_var}

SIMP-6311 - Getting issue details... STATUS

(tick)

A2.1simp-environment.spec still provides SELinux policies for %{_var}

  • A2.1.1%build still builds SELinux module

  • A2.1.2: %post (initial install only) applies the initial installs' fixfiles restore to the top-level /var/simp 

  • A2.1.2: %post no longer applies fixfiles for separate subdirectories under /var/simp/environments/ 

    • See A2.2

  1. remove policies from simp-environment:  SIMP-4379 - Getting issue details... STATUS
  2. create new RPM: SIMP-6559 - Getting issue details... STATUS


A3.1 %post (initial install only) applies the Puppet user settings & groups at the top-level /var/simp 

SIMP-6614 - Getting issue details... STATUS




A4.1:  (Related: E1Consolidate %post's yum repo creation logic into the existing script simp-utils:scripts/sbin/updaterepos

See:  E1.1


A4.1.1: The two have slight differences between them that should be carried over into updaterepos

See: simp-utils RPM/ E1.2
(tick)

A4.1.2:  Remove logic from RPM %post

SIMP-6402 - Getting issue details... STATUS
  • createrepo removed in PR#23
    • The above pull request removed the yum repo stuff from simp-env but another ticket was opened to do the work in simp-utils.
(tick)

A5.1: remove cacertkey ensure logic from %post

SIMP-6311 - Getting issue details... STATUS patched in PR#22
(tick)

A6.1: remove `/usr/local/sbin/simp_rpm_helper --rpm_dir=%{prefix} --rpm_section='post' --rpm_status=$1 --preserve --target_dir='.'` from %post

SIMP-6311 - Getting issue details... STATUS


patched in PR#22

(tick)

A7.1: remove `/usr/local/sbin/simp_rpm_helper --rpm_dir=%{prefix} --rpm_section='post' --rpm_status=$1 --preserve --target_dir='.'` from %postun

SIMP-6311 - Getting issue details... STATUS
patched in PR#22


simp-adapter (RPM)


MappingJira IssuesNotes
(tick)

B1: keep this for now? (see discussion)

SIMP-6584 - Getting issue details... STATUS PR#15, undone by PR#17
(tick)

B2.1: remove this behavior from %post

SIMP-6125 - Getting issue details... STATUS
(tick)

B3.1: Remove this behavior from %post

Incorporated into   SIMP-6607 - Getting issue details... STATUS


simp-
rsync (RPM)


MappingJira IssuesNotes
(tick)

C1.1.1: Change Prefix: %{prefix}to /usr/share/simp/environment_templates/rsync/

SIMP-6508 - Getting issue details... STATUS
(tick)

C1.1.2: Change any references to %{rsync_dir} from all RPM scriptlets 

SIMP-6508 - Getting issue details... STATUS

(tick)C2.2: Remove OBE %pre logic SIMP-6616 - Getting issue details... STATUS
(tick)

C3.1:  Move rsync.facl into %{prefix} ( /usr/share/simp/... ), like the rest of the %files

SIMP-6508 - Getting issue details... STATUS
(tick)

C3.3: Remove from %post

SIMP-6613 - Getting issue details... STATUS
(tick)

C4.1: Remove *.rpmnew deletion logic from %post

SIMP-6613 - Getting issue details... STATUS


(tick)

C5.1Remove OS symlink logic from %post (code) and %preun (code)

SIMP-6613 - Getting issue details... STATUS

still in 7.0.0-0:

%post

%preun


simp-core (ISO)


MappingJira IssuesNotes
(tick)

D1.1Remove this logic.  No extra copy of files already located in /var/www/yum needed.

SIMP-6578 - Getting issue details... STATUS

D1.2: simp environment command would then install into the appropriate secondary environment directory, when these boot files exist.

ISSUE TBD: As of 5/14, this is debatable—it may install into the secondary environment skeleton



simp-core (RPM)


MappingJira IssuesNotes
(tick)

D2.1: Remove obsolete hiera_upgrade script

SIMP-6507 - Getting issue details... STATUS


simp-utils (RPM)


MappingJira IssuesNotes

SIMP-6514 - Getting issue details... STATUS




  • E1.2: The two have slight differences between them that should be carried over into updaterepos
SIMP-6522 - Getting issue details... STATUS


Mapped capabilities: SIMP ≤ 6.3 to SIMP ≥ 6.4


For each component:

  • Identify relevant SIMP ≤ 6.3 capabilities
  • Assign each capability an Alpha+Number id (A1, A2) 
    • incomplete ids (or ids with questions) are orange, completely-mapped ids are green.
  • Map SIMP ≤ 6.3 capabilities to SIMP ≥ 6.4
    • Add mappings as id+Number (A1.1, A1.2, ...)

simp-environment (6.3.0)  [Analyzed, Mapped: 7/7, Questions: 0]


  • A1.) simp-environment.spec essentially installs to two filesystem roots:
    • SIMP ≤ 6.3 simp-environment 6.3.0:
      • %{prefix}/usr/share/simp/environments/simp/ )
      • %{_var} ( used to manage %{_var}/simp/environments/simp/ )
    • SIMP ≥ 6.4 Mappings:
      • A1.1: simp-environment.spec installs all files to %{prefix}, and does not install files into %{_var}                                                                                                      
      • A1.2: simp env new|update ENVIRONMENT will copy files from %{prefix} to /var/simp/environments/${ENVIRONMENT}/ 

  • A2.)  Provides SELinux policies for different /var/simp/ subdirectories so the various services can access files
    • SIMP ≤ 6.3 simp-environment 6.3.0:
    • SIMP ≥ 6.4 Mappings:
      • A2.1simp-environment.spec still provides SELinux policies for %{_var}
        • A2.1.1%build still builds SELinux module
        • A2.1.2: %post (initial install only) applies the initial installs' fixfiles restore to the top-level /var/simp 
        • A2.1.2: %post no longer applies fixfiles for separate subdirectories under /var/simp/environments/ 
          • See A2.2
        • A2.1.3: What should %uninstall do? Nothing special is required.
      • A2.2: simp env new|update ENVIRONMENT runs a post-action step that applies SELinux fixfiles restore to the /var/simp/environments/${ENVIRONMENT}/ + subdirectories 
      • A2.3simp env fix ENVIRONMENT reapplies the correct SELinux context on demand

  • A3.)  Sets Puppet user & group permissions based on the settings in puppet config print
    • SIMP ≤ 6.3 simp-environment 6.3.0:
      • Affects %{prefix} and %{_var} directory trees
    • SIMP ≥ 6.4 Mappings:
      • A3.1:  %post (initial install only) applies the Puppet user settings & groups at the top-level /var/simp 
      • A3.2: (Related: A2.2simp env new|update ENVIRONMENT runs a post-action step that applies Puppet user settings & groups  to
        • A3.2.1$codedir/environments/$ENVIRONMENT/ (group only)
        • A3.2.2/var/simp/environments/$ENVIRONMENT/site_files/ (group only)
        • A3.2.3/opt/puppetlabs/server/data/puppetserver/simp/environments/$ENVIRONMENT/ (user + group)
  • A4.)  %post creates a yum repo directory structure and runs createrepo under /var/www/yum/${os}/...
    • (info) This does not affect the secondary SIMP environment directories; it's under /var/www/yum
    • (warning) It's not clear why this logic is in simp-environment.spec, which doesn't seem to deliver these files.
    • SIMP ≥ 6.4 Mappings (None):
      • A4.1:  (Related: E1Consolidate %post's yum repo creation logic into the existing script simp-utils:scripts/sbin/updaterepos.
        • A4.1.1: The two have slight differences between them that should be carried over into updaterepos
        • A4.1.2:  Remove logic from RPM %post
  • A5.) On %post, ensures that the cacertkey (at %{_var}/simp/environments/simp/FakeCA/cacertkey) has some random gibberish in it if it doesn't exist. 
    • SIMP ≥ 6.4 Mappings:
      • A5.1: remove this behavior from %post
      • A5.2: The simp env command will ensure the cacertkey during fix ENVIRONMENT and after new|update ENVIRONMENT.

  • A6.) On %post, runs simp_rpm_helper 
    • SIMP ≤ 6.3 simp-environment 6.3.0:
      • Comment: # Needed for migrating the environment data into the codedir for an initial install
      • /usr/local/sbin/simp_rpm_helper --rpm_dir=%{prefix} --rpm_section='post' --rpm_status=$1 --preserve --target_dir='.'
    • SIMP ≥ 6.4 Mappings:
      • A6.1: remove this behavior from %post
      • A6.2: move the initial ISO install environment deploy logic into an early stage of simp config .
      • A6.3: create new Extra/Omni environments on demand with simp environment new.
  • A7.)  On %postun, runs simp_rpm_helper 
    • SIMP ≤ 6.3 simp-environment 6.3.0:
      • Comment: # Needed for cleaning up the data from codedir as appropriate for an erase
      • /usr/local/sbin/simp_rpm_helper --rpm_dir=%{prefix} --rpm_section='postun' --rpm_status=$1 --preserve --target_dir='.'
    • SIMP ≥ 6.4 Mappings:
      • A7.1: Remove from %postun
      • A7.2: Environments can be removed with simp env rm ENVIRONMENT

simp-adapter (0.1.1)  [Analyzed, Mapped: 3/3, Questions: 0]

  • B1.) simp-adapter 0.1.1 %pre and %posttrans have operations to prevent the global Hiera 3 hiera.yaml.simp file delivered with simp-adapter <= 0.0.6 and its hiera.yaml link created in that RPM's %post from being removed during upgrade if it may be in use.
    • See the simp-adapter 0.1.1 acceptance tests for details on when hiera.yaml files (global and environment) are modified during install/upgrade:
      • Upgrading simp-adapter from version <= 0.0.6
        • When global hiera.yaml is linked to hiera.yaml.simp
          • it should retain hiera.yaml and hiera.yaml.simp
        • When global hiera.yaml is not linked to hiera.yaml.simp
          • it should remove hiera.yaml.simp but not remove hiera.yaml
      • Uninstalling simp-adapter and legacy global Hiera 3 config exists
        • When global hiera.yaml is linked to hiera.yaml.simp
          • it should remove hiera.yaml and hiera.yaml.simp, but keep hiera.yaml.simpbak
        • When global hiera.yaml is not linked to hiera.yaml.simp
          • it should remove hiera.yaml.simp, but keep hiera.yaml and hiera.yaml.simpbak
    • SIMP ≥ 6.4 Mappings:
      • This logic should be retained.
  • B2.)  simp-adapter 0.1.1 %post (install-only) logic (removed after 0.1.1) creates /etc/simp/adapter_config.yaml:
    • SIMP ≤ 6.3 simp-environment 6.3.0
      • Under the following conditions:

        • Only during a new RPM install ($1 -eq 1)

        • Only when the Linux kernel contained simp install (e.g., a SIMP ISO installation)

      • %post will create %{prefix}/adapter_config.yaml with the following content:

        # This file was modified by simp-adapter during a SIMP install
        # on ${date}:
        target_directory: 'auto'
        copy_rpm_data: true

    • SIMP ≥ 6.4 Mappings:
      • B2.1: remove this behavior from %post

  • B3.) simp-adapter 0.1.1 %post logic fixes problems with Puppet RPMs:
    • SIMP ≤ 6.3 simp-environment 6.3.0
      • For FOSS puppetserver (install or upgrade)
        • Nails up puppet user UID and GID to 52, changing the UID/GID if the installed ids are incorrect.
        • Fixes permissions of /opt/puppetlabs//etc/puppetlabs/var/log/puppetlabs/, and /var/run/puppetlabs/ to match the nailed UID/GID
        • Restarts puppetserver to pick up changes.
        • Creates puppdetdb user and group
      • Sets the digest algorithm used by puppet to sha256 (install or upgrade)
      • Install-only, fixes the permissions of puppet-agentpuppetserver, and puppetdb directories
    • SIMP ≥ 6.4 Mappings:
      • B3.1: Remove this behavior from %post
      • B3.2: The simp env command will ensure the correct puppet permissions are set during fix ENVIRONMENT and after new|update ENVIRONMENT
        • B3.2.1: Note the changes above include several users, paths, and the digest algorithm.
        • B3.2.2: Note that these specific UID/GIDs may no longer be necessary.  Now that the simp command is setting them, it can just check what user is configured to own what files etc.

simp-rsync (6.2.1) [Analyzed, Mapped:6/6, Questions: 1]


  • C1.) RPM delivers a directory tree directly under /var/simp/environments/simp/rsync/
    • SIMP ≤ 6.3 simp-rsync 6.2.1:
    • SIMP ≥ 6.4 Mappings:
      • C1.1:  The simp-rsync RPM only delivers files to %{prefix}, which is now /usr/share/simp/environment_templates/rsync/
        • C1.1.1: Change Prefix:%{prefix} to /usr/share/simp/environment_templates/rsync/
        • C1.1.2: Change any references to %{rsync_dir} from all RPM scriptlets 
      • C1.2: The command simp env new|update ENVIRONMENT will copy files from %{prefix} to /var/simp/environments/${ENVIRONMENT}/rsync/
  • C2.) %pre has strange logic
    • SIMP ≤ 6.3 simp-rsync 6.2.1:
      • Pre-removes directories under {%rsync_dir}/ ("# Remove the directories that we're going to replace with symlinks")
      • (warning)  Includes a "# Make sure upgrades work properly!" section that looks like it might try to symlink the LICENSE file as a directory if the default directory is missing under bind_dns/.

        • This looks like a bug in the logic
    • SIMP ≥ 6.4 Mappings:
      • (question) (nice-to-have, for later): Should we expose arguments to template the skeleton DNS environment when it is copied/updates?
      • C2.1: The command simp env new|update ENVIRONMENT will copy files from %{prefix} to /var/simp/environments/${ENVIRONMENT}/rsync/
        • This should include any logic needed to ensure a basic DNS environment
      • C2.2: Remove %pre logic
  • C3.) %post runs setfacl --restore on {%rsync_dir}, using the delivered .rsync.facl file.
    • SIMP ≤ 6.3 simp-rsync 6.2.1:
      • %{rsync_dir}/var/simp/environments/simp/rsync
      • The rsync.facl definitions use relative paths
    • SIMP ≥ 6.4 Mappings:
      • (info) The simp-rsync RPM only delivers files to %{prefix} (/usr/share/simp/environment_templates/rsync/); see C1.1
      • C3.1:  Move rsync.facl into %{prefix} (/usr/...), like the rest of the %files
      • C3.2: The simp env command will ensure the correct FACLS are set during fix ENVIRONMENT and after new|update ENVIRONMENT.
      • C3.3: Remove from %post
  • C4.) %post deletes all *.rpmnew files under {%rsync_dir}/
    • SIMP ≤ 6.3 simp-rsync 6.2.1:
      • This was a workaround to problems associated with C1 (delivering files to /var/simp/environments/simp/rsync/ as %config)
        • Example problem: rsyncing meaningless *.rpmnew files to hosts that have no use for them
    • SIMP ≥ 6.4 Mappings:
      • (info) The SIMP ≤ 6.3 workaround should not be necessary now because:
        • the RPM delivers a skeleton to somewhere under /usr/share/simp/, which is not an rsync source.
        • the files no longer need to be %config, so .rpmnew files won't get generated, anyway
      • C4.1: Remove logic from %post
        • Prerequisites: C1.1 and C1.2

  • C5.) RPM scriptlets auto-ensure that all directories called RedHat have a CentOS symlink under /var/simp/environments/simp/rsync
    • SIMP ≤ 6.3 simp-rsync 6.2.1:
      • %post symlinks RedHat directories to CentOS under {%rsync_dir}/
        • (warning) The scriptlets will add a symlink to any directory named RedHat, regardless of its location (buggy logic)
      • %preun deletes any symlinks named CentOS under {%rsync_dir}/
        • (warning) The scriptlets will delete a CentOS symlink in any directory named RedHat, regardless of its subdirectory (buggy logic)
    • SIMP ≥ 6.4 Mappings:
      • C5.1: Remove logic from %post and %preun 
        • Prerequisites: C1.1 and C1.2
      • C5.2: The simp env command will ensure the correct symlinks (if any) are managed during  new|update|remove ENVIRONMENT.

  • C6.) RPM scriptlets auto-ensure that all directories called RedHat have a CentOS symlink under /var/simp/environments/simp/rsync 
    • SIMP ≥ 6.4 Mappings:
      • See C5.1

simp-core (6.3.3) [Analyzed, Mapped: 2/2]

  • D1.) ISO auto.cfg
  • D2.)  %post in simp.spec
    • SIMP ≥ 6.4 Mappings:
      • D2.1: Remove obsolete hiera_upgrade script (SIMP-6507)

simp-utils


Further steps

What happens during an upgrade?

Reference