SIMP 6.4: Map old/new files and capabilities


Description

This article is an artifact of:

  •  
  • .

It tracks:

  • Capabilities/files/actions provided by the old (SIMP ≤ 6.3) versions of:
  • What / how / if these capabilities will be handled in the new (SIMP ≥ 6.4) ecosystem.

Goals:

  • Understand what responsibilities the components in the new SIMP ≥ 6.4 ecosystem will have
  • Ensure no capabilities are missed during the transition
  • Document/create issues for anything we don't have a solution for


Brief Summary of SIMP 6.4 

Starting with 6.4, SIMP's philosophy of handling SIMP Environments is changing:

Summary of SIMP ≥ 6.4 changes, by component

This section summarizes the changes mapped in the sections above, to collect requirements for each component:

Legend:

WHITEMapped, but relevant Jira issues have not been researched 


: reason

Not Jira us
YELLOWJira issues are still being researched for this mapping
BLUEJira issues exist to address this mapping
(tick)PRs have been submitted to address this mapping's Jira Issues
(tick) GREENMapping + Jira Issues are complete


simp-cli commands 

`simp environment [COMMAND]



MappingJira IssuesNotes

A1.2: simp env new|update ENVIRONMENT will copy files from %{prefix} to /var/simp/environments/${ENVIRONMENT}/ 

(tick)

A2.2: simp env new|update ENVIRONMENT runs a post-action step that applies SELinux fixfiles restore to the /var/simp/environments/${ENVIRONMENT}/ + subdirectories 

  • OmniEnvController calls #fix as part of #new
    • DirEnv#fix
      • SecondaryDirEnv#fix:

(tick)

A2.3simp env fix ENVIRONMENT applies the correct SELinux contexts on demand  

  • simp env fix 
  • DirEnv#fix + SecondaryDirEnv#fix: 
(tick)

A3.2: (Related: A2.2simp env new|update ENVIRONMENT runs a post-action step that applies Puppet user settings & groups  to:

  • OmniEnvController delegates #fix to Env objects: 
(tick)
  • A3.2.1$codedir/environments/$ENVIRONMENT/ (group only)
(tick)
  • A3.2.2/var/simp/environments/$ENVIRONMENT/site_files/ (group only)
(tick)
  • A3.2.3: /opt/puppetlabs/server/data/puppetserver/simp/environments/$ENVIRONMENT(user + group)
 (error)
(tick)

A5.2: The simp env command will ensure the cacertkey during fix ENVIRONMENT and after new|update ENVIRONMENT.

  • SecondaryDirEnv#new
  • SecondaryDirEnv#fix: 
    • : should this be in #fix?
(tick)

A6.3: create new Extra/Omni environments on demand with simp environment new.

  • PuppetDirEnv#new: 
  • SecondaryDirEnv#new: 
  • WritableDirEnv#new: 

 


A7.2: Environments can be removed with simp env rm ENVIRONMENT


(tick)

B3.2: The simp env command will ensure the correct puppet permissions are set during fix ENVIRONMENT and after new|update ENVIRONMENT

  • B3.2.1: Note the changes above include several users, paths, and the digest algorithm.

  • B3.2.2: Note that these specific UID/GIDs may no longer be necessary.  Now that the simp command is setting them, it can just check what user is configured to own what files etc.

 See A3.2 + children

  • Puppet: 
  • Secondary: 
  • Writable: 


 (error)

(tick)

C1.2: The command simp env new|update ENVIRONMENT will copy files from %{prefix}/???/rsync/ to /var/simp/environments/${ENVIRONMENT}/rsync/

Related to A1.2, but focuses on rsync/, which will have its own skeleton directory

(tick)

C2.1 The command simp env new|update ENVIRONMENT will copy files from %{prefix} to /var/simp/environments/${ENVIRONMENT}/rsync/

  • : Look into basic DNS env
  • : Review RPM strangeness
(tick)C3.2: The simp env command will ensure the correct FACLS are set during fix ENVIRONMENT and after new|update ENVIRONMENT.



C5.2:  The simp env command will ensure the correct symlinks (if any) are managed during  new|update|remove ENVIRONMENT
(tick)D1.2: simp environment command would then install into the appropriate secondary environment directory, when these boot files exist.


`simp config


MappingJira IssuesNotes
(tick)

A6.2move the initial ISO install environment deploy logic into an early stage of simp config 

Uncomment OmniEnvContoller#create and integration test

  • (follows  )


simp-environment (RPM)


MappingJira IssuesNotes
(tick)

A1.1: simp-environment.spec installs all files to %{prefix}, and does not install files into %{_var}


(tick)

A2.1simp-environment.spec still provides SELinux policies for %{_var}

  • A2.1.1%build still builds SELinux module

  • A2.1.2: %post (initial install only) applies the initial installs' fixfiles restore to the top-level /var/simp 

  • A2.1.2: %post no longer applies fixfiles for separate subdirectories under /var/simp/environments/ 

    • See A2.2

  1. remove policies from simp-environment: 
  2. create new RPM:


A3.1 %post (initial install only) applies the Puppet user settings & groups at the top-level /var/simp 




A4.1:  (Related: E1Consolidate %post's yum repo creation logic into the existing script simp-utils:scripts/sbin/updaterepos

See:  E1.1


A4.1.1: The two have slight differences between them that should be carried over into updaterepos

See: simp-utils RPM/ E1.2
(tick)

A4.1.2:  Remove logic from RPM %post

  • createrepo removed in PR#23
    • The above pull request removed the yum repo stuff from simp-env but another ticket was opened to do the work in simp-utils.
(tick)

A5.1: remove cacertkey ensure logic from %post

patched in PR#22
(tick)

A6.1: remove `/usr/local/sbin/simp_rpm_helper --rpm_dir=%{prefix} --rpm_section='post' --rpm_status=$1 --preserve --target_dir='.'` from %post


patched in PR#22

(tick)

A7.1: remove `/usr/local/sbin/simp_rpm_helper --rpm_dir=%{prefix} --rpm_section='post' --rpm_status=$1 --preserve --target_dir='.'` from %postun

patched in PR#22


simp-adapter (RPM)


MappingJira IssuesNotes
(tick)

B1: keep this for now? (see discussion)

PR#15, undone by PR#17
(tick)

B2.1: remove this behavior from %post


(tick)

B3.1: Remove this behavior from %post

Incorporated into  


simp-
rsync (RPM)


MappingJira IssuesNotes
(tick)

C1.1.1: Change Prefix: %{prefix}to /usr/share/simp/environment_templates/rsync/


(tick)

C1.1.2: Change any references to %{rsync_dir} from all RPM scriptlets 


(tick)C2.2: Remove OBE %pre logic
(tick)

C3.1:  Move rsync.facl into %{prefix} ( /usr/share/simp/... ), like the rest of the %files


(tick)

C3.3: Remove from %post


(tick)

C4.1: Remove *.rpmnew deletion logic from %post


(tick)

C5.1Remove OS symlink logic from %post (code) and %preun (code)

still in 7.0.0-0:

%post

%preun


simp-core (ISO)


MappingJira IssuesNotes
(tick)

D1.1Remove this logic.  No extra copy of files already located in /var/www/yum needed.



D1.2: simp environment command would then install into the appropriate secondary environment directory, when these boot files exist.

: As of 5/14, this is debatable—it may install into the secondary environment skeleton



simp-core (RPM)


MappingJira IssuesNotes
(tick)

D2.1: Remove obsolete hiera_upgrade script



simp-utils (RPM)


MappingJira IssuesNotes




  • E1.2: The two have slight differences between them that should be carried over into updaterepos


Mapped capabilities: SIMP ≤ 6.3 to SIMP ≥ 6.4


For each component:

  • Identify relevant SIMP ≤ 6.3 capabilities
  • Assign each capability an Alpha+Number id (A1, A2) 
    • incomplete ids (or ids with questions) are orange, completely-mapped ids are green.
  • Map SIMP ≤ 6.3 capabilities to SIMP ≥ 6.4
    • Add mappings as id+Number (A1.1, A1.2, ...)

simp-environment (6.3.0)  [Analyzed, Mapped: 7/7, Questions: 0]


  • A1.) simp-environment.spec essentially installs to two filesystem roots:
    • SIMP ≤ 6.3 simp-environment 6.3.0:
      • %{prefix}/usr/share/simp/environments/simp/ )
      • %{_var} ( used to manage %{_var}/simp/environments/simp/ )
    • SIMP ≥ 6.4 Mappings:
      • A1.1: simp-environment.spec installs all files to %{prefix}, and does not install files into %{_var}                                                                                                      
      • A1.2: simp env new|update ENVIRONMENT will copy files from %{prefix} to /var/simp/environments/${ENVIRONMENT}/ 

  • A2.)  Provides SELinux policies for different /var/simp/ subdirectories so the various services can access files
    • SIMP ≤ 6.3 simp-environment 6.3.0:
    • SIMP ≥ 6.4 Mappings:
      • A2.1simp-environment.spec still provides SELinux policies for %{_var}
        • A2.1.1%build still builds SELinux module
        • A2.1.2: %post (initial install only) applies the initial installs' fixfiles restore to the top-level /var/simp 
        • A2.1.2: %post no longer applies fixfiles for separate subdirectories under /var/simp/environments/ 
          • See A2.2
        • A2.1.3: What should %uninstall do? Nothing special is required.
      • A2.2: simp env new|update ENVIRONMENT runs a post-action step that applies SELinux fixfiles restore to the /var/simp/environments/${ENVIRONMENT}/ + subdirectories 
      • A2.3simp env fix ENVIRONMENT reapplies the correct SELinux context on demand

  • A3.)  Sets Puppet user & group permissions based on the settings in puppet config print
    • SIMP ≤ 6.3 simp-environment 6.3.0:
      • Affects %{prefix} and %{_var} directory trees
    • SIMP ≥ 6.4 Mappings:
      • A3.1:  %post (initial install only) applies the Puppet user settings & groups at the top-level /var/simp 
      • A3.2: (Related: A2.2simp env new|update ENVIRONMENT runs a post-action step that applies Puppet user settings & groups  to
        • A3.2.1$codedir/environments/$ENVIRONMENT/ (group only)
        • A3.2.2/var/simp/environments/$ENVIRONMENT/site_files/ (group only)
        • A3.2.3/opt/puppetlabs/server/data/puppetserver/simp/environments/$ENVIRONMENT/ (user + group)
  • A4.)  %post creates a yum repo directory structure and runs createrepo under /var/www/yum/${os}/...
    • (info) This does not affect the secondary SIMP environment directories; it's under /var/www/yum
    • (warning) It's not clear why this logic is in simp-environment.spec, which doesn't seem to deliver these files.
    • SIMP ≥ 6.4 Mappings (None):
      • A4.1:  (Related: E1Consolidate %post's yum repo creation logic into the existing script simp-utils:scripts/sbin/updaterepos.
        • A4.1.1: The two have slight differences between them that should be carried over into updaterepos
        • A4.1.2:  Remove logic from RPM %post
  • A5.) On %post, ensures that the cacertkey (at %{_var}/simp/environments/simp/FakeCA/cacertkey) has some random gibberish in it if it doesn't exist. 
    • SIMP ≥ 6.4 Mappings:
      • A5.1: remove this behavior from %post
      • A5.2: The simp env command will ensure the cacertkey during fix ENVIRONMENT and after new|update ENVIRONMENT.

  • A6.) On %post, runs simp_rpm_helper 
    • SIMP ≤ 6.3 simp-environment 6.3.0:
      • Comment: # Needed for migrating the environment data into the codedir for an initial install
      • /usr/local/sbin/simp_rpm_helper --rpm_dir=%{prefix} --rpm_section='post' --rpm_status=$1 --preserve --target_dir='.'
    • SIMP ≥ 6.4 Mappings:
      • A6.1: remove this behavior from %post
      • A6.2: move the initial ISO install environment deploy logic into an early stage of simp config .
      • A6.3: create new Extra/Omni environments on demand with simp environment new.
  • A7.)  On %postun, runs simp_rpm_helper 
    • SIMP ≤ 6.3 simp-environment 6.3.0:
      • Comment: # Needed for cleaning up the data from codedir as appropriate for an erase
      • /usr/local/sbin/simp_rpm_helper --rpm_dir=%{prefix} --rpm_section='postun' --rpm_status=$1 --preserve --target_dir='.'
    • SIMP ≥ 6.4 Mappings:
      • A7.1: Remove from %postun
      • A7.2: Environments can be removed with simp env rm ENVIRONMENT

simp-adapter (0.1.1)  [Analyzed, Mapped: 3/3, Questions: 0]

  • B1.) simp-adapter 0.1.1 %pre and %posttrans have operations to prevent the global Hiera 3 hiera.yaml.simp file delivered with simp-adapter <= 0.0.6 and its hiera.yaml link created in that RPM's %post from being removed during upgrade if it may be in use.
    • See the simp-adapter 0.1.1 acceptance tests for details on when hiera.yaml files (global and environment) are modified during install/upgrade:
      • Upgrading simp-adapter from version <= 0.0.6
        • When global hiera.yaml is linked to hiera.yaml.simp
          • it should retain hiera.yaml and hiera.yaml.simp
        • When global hiera.yaml is not linked to hiera.yaml.simp
          • it should remove hiera.yaml.simp but not remove hiera.yaml
      • Uninstalling simp-adapter and legacy global Hiera 3 config exists
        • When global hiera.yaml is linked to hiera.yaml.simp
          • it should remove hiera.yaml and hiera.yaml.simp, but keep hiera.yaml.simpbak
        • When global hiera.yaml is not linked to hiera.yaml.simp
          • it should remove hiera.yaml.simp, but keep hiera.yaml and hiera.yaml.simpbak
    • SIMP ≥ 6.4 Mappings:
      • This logic should be retained.
  • B2.)  simp-adapter 0.1.1 %post (install-only) logic (removed after 0.1.1) creates /etc/simp/adapter_config.yaml:
    • SIMP ≤ 6.3 simp-environment 6.3.0
      • Under the following conditions:

        • Only during a new RPM install ($1 -eq 1)

        • Only when the Linux kernel contained simp install (e.g., a SIMP ISO installation)

      • %post will create %{prefix}/adapter_config.yaml with the following content:

        # This file was modified by simp-adapter during a SIMP install
        # on ${date}:
        target_directory: 'auto'
        copy_rpm_data: true

    • SIMP ≥ 6.4 Mappings:
      • B2.1: remove this behavior from %post

  • B3.) simp-adapter 0.1.1 %post logic fixes problems with Puppet RPMs:
    • SIMP ≤ 6.3 simp-environment 6.3.0
      • For FOSS puppetserver (install or upgrade)
        • Nails up puppet user UID and GID to 52, changing the UID/GID if the installed ids are incorrect.
        • Fixes permissions of /opt/puppetlabs//etc/puppetlabs/var/log/puppetlabs/, and /var/run/puppetlabs/ to match the nailed UID/GID
        • Restarts puppetserver to pick up changes.
        • Creates puppdetdb user and group
      • Sets the digest algorithm used by puppet to sha256 (install or upgrade)
      • Install-only, fixes the permissions of puppet-agentpuppetserver, and puppetdb directories
    • SIMP ≥ 6.4 Mappings:
      • B3.1: Remove this behavior from %post
      • B3.2: The simp env command will ensure the correct puppet permissions are set during fix ENVIRONMENT and after new|update ENVIRONMENT
        • B3.2.1: Note the changes above include several users, paths, and the digest algorithm.
        • B3.2.2: Note that these specific UID/GIDs may no longer be necessary.  Now that the simp command is setting them, it can just check what user is configured to own what files etc.

simp-rsync (6.2.1) [Analyzed, Mapped:6/6, Questions: 1]


  • C1.) RPM delivers a directory tree directly under /var/simp/environments/simp/rsync/
    • SIMP ≤ 6.3 simp-rsync 6.2.1:
    • SIMP ≥ 6.4 Mappings:
      • C1.1:  The simp-rsync RPM only delivers files to %{prefix}, which is now /usr/share/simp/environment_templates/rsync/
        • C1.1.1: Change Prefix:%{prefix} to /usr/share/simp/environment_templates/rsync/
        • C1.1.2: Change any references to %{rsync_dir} from all RPM scriptlets 
      • C1.2: The command simp env new|update ENVIRONMENT will copy files from %{prefix} to /var/simp/environments/${ENVIRONMENT}/rsync/
  • C2.) %pre has strange logic
    • SIMP ≤ 6.3 simp-rsync 6.2.1:
      • Pre-removes directories under {%rsync_dir}/ ("# Remove the directories that we're going to replace with symlinks")
      • (warning)  Includes a "# Make sure upgrades work properly!" section that looks like it might try to symlink the LICENSE file as a directory if the default directory is missing under bind_dns/.

        • This looks like a bug in the logic
    • SIMP ≥ 6.4 Mappings:
      • (question) (nice-to-have, for later): Should we expose arguments to template the skeleton DNS environment when it is copied/updates?
      • C2.1: The command simp env new|update ENVIRONMENT will copy files from %{prefix} to /var/simp/environments/${ENVIRONMENT}/rsync/
        • This should include any logic needed to ensure a basic DNS environment
      • C2.2: Remove %pre logic
  • C3.) %post runs setfacl --restore on {%rsync_dir}, using the delivered .rsync.facl file.
    • SIMP ≤ 6.3 simp-rsync 6.2.1:
      • %{rsync_dir}/var/simp/environments/simp/rsync
      • The rsync.facl definitions use relative paths
    • SIMP ≥ 6.4 Mappings:
      • (info) The simp-rsync RPM only delivers files to %{prefix} (/usr/share/simp/environment_templates/rsync/); see C1.1
      • C3.1:  Move rsync.facl into %{prefix} (/usr/...), like the rest of the %files
      • C3.2: The simp env command will ensure the correct FACLS are set during fix ENVIRONMENT and after new|update ENVIRONMENT.
      • C3.3: Remove from %post
  • C4.) %post deletes all *.rpmnew files under {%rsync_dir}/
    • SIMP ≤ 6.3 simp-rsync 6.2.1:
      • This was a workaround to problems associated with C1 (delivering files to /var/simp/environments/simp/rsync/ as %config)
        • Example problem: rsyncing meaningless *.rpmnew files to hosts that have no use for them
    • SIMP ≥ 6.4 Mappings:
      • (info) The SIMP ≤ 6.3 workaround should not be necessary now because:
        • the RPM delivers a skeleton to somewhere under /usr/share/simp/, which is not an rsync source.
        • the files no longer need to be %config, so .rpmnew files won't get generated, anyway
      • C4.1: Remove logic from %post
        • Prerequisites: C1.1 and C1.2

  • C5.) RPM scriptlets auto-ensure that all directories called RedHat have a CentOS symlink under /var/simp/environments/simp/rsync
    • SIMP ≤ 6.3 simp-rsync 6.2.1:
      • %post symlinks RedHat directories to CentOS under {%rsync_dir}/
        • (warning) The scriptlets will add a symlink to any directory named RedHat, regardless of its location (buggy logic)
      • %preun deletes any symlinks named CentOS under {%rsync_dir}/
        • (warning) The scriptlets will delete a CentOS symlink in any directory named RedHat, regardless of its subdirectory (buggy logic)
    • SIMP ≥ 6.4 Mappings:
      • C5.1: Remove logic from %post and %preun 
        • Prerequisites: C1.1 and C1.2
      • C5.2: The simp env command will ensure the correct symlinks (if any) are managed during  new|update|remove ENVIRONMENT.

  • C6.) RPM scriptlets auto-ensure that all directories called RedHat have a CentOS symlink under /var/simp/environments/simp/rsync 
    • SIMP ≥ 6.4 Mappings:
      • See C5.1

simp-core (6.3.3) [Analyzed, Mapped: 2/2]

  • D1.) ISO auto.cfg
  • D2.)  %post in simp.spec
    • SIMP ≥ 6.4 Mappings:
      • D2.1: Remove obsolete hiera_upgrade script (SIMP-6507)

simp-utils


Further steps

What happens during an upgrade?

Reference