SIMP 6.4: Map old/new files and capabilities
Description
This article is an artifact of:
- - SIMP-6387Getting issue details... STATUS
- - SIMP-6391Getting issue details... STATUS .
- - SIMP-6562Getting issue details... STATUS
It tracks:
- Capabilities/files/actions provided by the old (SIMP ≤ 6.3) versions of:
simp-environment (6.3.0)
simp-adapter
(0.1.1)simp-rsync
(6.2.1)simp
- the ISO
auto.cfg
- What / how / if these capabilities will be handled in the new (SIMP ≥ 6.4) ecosystem.
Goals:
- Understand what responsibilities the components in the new SIMP ≥ 6.4 ecosystem will have
- See article: SIMP 6.4+: Design `simp environment` command
- Ensure no capabilities are missed during the transition
- Document/create issues for anything we don't have a solution for
Brief Summary of SIMP 6.4
Starting with 6.4, SIMP's philosophy of handling SIMP Environments is changing:
- Don't modify an actual environment, unless explicitly requested by the user
- This prohibits unexpected automatic environment updates
- "Explicitly requested by the user" includes the initial
simp config
after an ISO installation.
Summary of SIMP ≥ 6.4 changes, by component
This section summarizes the changes mapped in the sections above, to collect requirements for each component:
Legend:
WHITE | Mapped, but relevant Jira issues have not been researched |
ISSUE TBD: reason | Not Jira us |
YELLOW | Jira issues are still being researched for this mapping |
BLUE | Jira issues exist to address this mapping |
PRs have been submitted to address this mapping's Jira Issues | |
GREEN | Mapping + Jira Issues are complete |
simp-cli
commands
`simp environment
[COMMAND]
`
Related article
Mapping | Jira Issues | Notes | |
---|---|---|---|
A1.2: | SECONDARY ENV | ||
A2.2: | OMNI CONTROLLER SECONDARY ENV | ||
A2.3: | SECONDARY ENV | ||
A3.2: (Related: A2.2) | OMNI CONTROLLER | ||
| PUPPET ENV | ||
| SECONDARY ENV | ||
| WRITABLE ENV | ||
A5.2: The | SECONDARY ENV | ||
A6.3: create new Extra/Omni environments on demand with | PUPPET ENV SECONDARY ENV WRITABLE ENV | ||
A7.2: Environments can be removed with | PUPPET ENV SECONDARY ENV | ||
B3.2: The
| DUPLICATE See A3.2 + children
| PUPPET ENV SECONDARY ENV WRITABLE ENV | |
C1.2: The command
| SECONDARY ENV Related to A1.2, but focuses on | ||
C2.1 The command
| SECONDARY ENV | ||
C3.2: The simp env command will ensure the correct FACLS are set during fix ENVIRONMENT and after new|update ENVIRONMENT . | SECONDARY ENV | ||
C5.2: The simp env command will ensure the correct symlinks (if any) are managed during new|update|remove ENVIRONMENT | - SIMP-6620Getting issue details... STATUS | SECONDARY ENV | |
D1.2: simp environment command would then install into the appropriate secondary environment directory, when these boot files exist. | - SIMP-6621Getting issue details... STATUS | SECONDARY ENV |
`simp config
`
Mapping | Jira Issues | Notes | |
---|---|---|---|
A6.2: move the initial ISO install environment deploy logic into an early stage of | Uncomment |
simp-environment
(
RPM)
Mapping | Jira Issues | Notes | |
---|---|---|---|
A1.1: | |||
A2.1:
| |||
A3.1: |
-
SIMP-6614Getting issue details...
STATUS
| ||
A4.1: (Related: E1) Consolidate |
| See: E1.1 | |
A4.1.1: The two have slight differences between them that should be carried over into |
| See: simp-utils RPM/ E1.2 | |
A4.1.2: Remove logic from RPM | - SIMP-6402Getting issue details... STATUS |
| |
A5.1: remove cacertkey ensure logic from | - SIMP-6311Getting issue details... STATUS | patched in PR#22 | |
A6.1: remove `/usr/local/sbin/simp_rpm_helper --rpm_dir=%{prefix} --rpm_section='post' --rpm_status=$1 --preserve --target_dir='.'` from | |||
A7.1: remove `/usr/local/sbin/simp_rpm_helper --rpm_dir=%{prefix} --rpm_section='post' --rpm_status=$1 --preserve --target_dir='.'` from | patched in PR#22 |
simp-adapter
(RPM)
Mapping | Jira Issues | Notes | |
---|---|---|---|
B1: keep this for now? (see discussion) | - SIMP-6584Getting issue details... STATUS | PR#15, undone by PR#17 | |
B2.1: remove this behavior from | - SIMP-6125Getting issue details... STATUS | ||
B3.1: Remove this behavior from | Incorporated into - SIMP-6607Getting issue details... STATUS |
simp-
rsync
(RPM)
simp-
Mapping | Jira Issues | Notes | |
---|---|---|---|
C1.1.1: Change | - SIMP-6508Getting issue details... STATUS | ||
C1.1.2: Change any references to | |||
C2.2: Remove OBE %pre logic | - SIMP-6616Getting issue details... STATUS | ||
C3.1: Move | - SIMP-6508Getting issue details... STATUS | ||
C3.3: Remove from %post | - SIMP-6613Getting issue details... STATUS | ||
C4.1: Remove | |||
C5.1: Remove OS symlink logic from | - SIMP-6613Getting issue details... STATUS | still in 7.0.0-0: |
simp-
core
(ISO)
Mapping | Jira Issues | Notes | |
---|---|---|---|
D1.1: Remove this logic. No extra copy of files already located in | - SIMP-6578Getting issue details... STATUS | ||
D1.2: | ISSUE TBD: As of 5/14, this is debatable—it may install into the secondary environment skeleton |
simp-
core
(RPM)
Mapping | Jira Issues | Notes | |
---|---|---|---|
D2.1: Remove obsolete | - SIMP-6507Getting issue details... STATUS |
simp-
utils
(RPM)
Mapping | Jira Issues | Notes | |
---|---|---|---|
| - SIMP-6514Getting issue details... STATUS | ||
| - SIMP-6522Getting issue details... STATUS |
Mapped capabilities: SIMP ≤ 6.3 to SIMP ≥ 6.4
For each component:
- Identify relevant SIMP ≤ 6.3 capabilities
- Assign each capability an Alpha+Number id (A1, A2)
- incomplete ids (or ids with questions) are orange, completely-mapped ids are green.
- Map SIMP ≤ 6.3 capabilities to SIMP ≥ 6.4
- Add mappings as id+Number (A1.1, A1.2, ...)
simp-environment
(6.3.0) [Analyzed, Mapped: 7/7, Questions: 0]
- A1.)
simp-environment.spec
essentially installs to two filesystem roots:- SIMP ≤ 6.3
simp-environment
6.3.0:%{prefix}
(/usr/share/simp/environments/simp/
)%{_var}
( used to manage%{_var
}/simp/environments/simp/
)
- SIMP ≥ 6.4 Mappings:
- A1.1:
simp-environment.spec
installs all files to%{prefix}
, and does not install files into%{_var}
- A1.2:
simp env new|update ENVIRONMENT
will copy files from%{prefix}
to/var/simp/environments/${ENVIRONMENT}/
- A1.1:
- SIMP ≤ 6.3
- A2.) Provides SELinux policies for different
/var/simp/
subdirectories so the various services can access files- SIMP ≤ 6.3
simp-environment
6.3.0:- On
%build
, builds SELinux module- Defines paths and contexts in simp-environment.fc
- Defines module in simp-environment.te
- On
%post
, builds + loads an SELinux policy and runsfixfiles
so puppet can read from/var/simp
directories - On
%uninstall
, removes SELinux policy and runsfixfiles
- On
- SIMP ≥ 6.4 Mappings:
- A2.1:
simp-environment.spec
still provides SELinux policies for%{_var}
- A2.1.1:
%build
still builds SELinux module - A2.1.2:
%post
(initial install only) applies the initial installs'fixfiles restore
to the top-level/var/simp
- A2.1.2:
%post
no longer appliesfixfiles
for separate subdirectories under/var/simp/environments/
- See A2.2
A2.1.3: What should %uninstall do?Nothing special is required.
- A2.1.1:
- A2.2:
simp env new|update ENVIRONMENT
runs a post-action step that applies SELinuxfixfiles restore
to the/var/simp/environments/${ENVIRONMENT}/ + subdirectories
- A2.3:
simp env fix ENVIRONMENT
reapplies the correct SELinux context on demand
- A2.1:
- SIMP ≤ 6.3
- A3.) Sets Puppet user & group permissions based on the settings in
puppet config print
- SIMP ≤ 6.3
simp-environment
6.3.0:- Affects
%{prefix}
and%{_var}
directory trees
- Affects
- SIMP ≤ 6.3
- SIMP ≥ 6.4 Mappings:
- A3.1:
%post
(initial install only) applies the Puppet user settings & groups at the top-level/var/simp
- A3.2: (Related: A2.2)
simp env new|update ENVIRONMENT
runs a post-action step that applies Puppet user settings & groups to- A3.2.1:
$codedir/environments/$ENVIRONMENT/
(group only) - A3.2.2:
/var/simp/environments/$ENVIRONMENT/site_files/
(group only) A3.2.3:/opt/puppetlabs/server/data/puppetserver/simp/environments/$ENVIRONMENT/
(user + group)
- A3.2.1:
- A3.1:
- A4.)
%post
creates a yum repo directory structure and runscreaterepo
under/var/www/yum/${os}/...
:- This does not affect the secondary SIMP environment directories; it's under
/var/www/yum
- It's not clear why this logic is in
simp-environment.spec
, which doesn't seem to deliver these files. - SIMP ≥ 6.4 Mappings (None):
- A4.1: (Related: E1) Consolidate
%post
's yum repo creation logic into the existing scriptsimp-utils:scripts/sbin/updaterepos
.- A4.1.1: The two have slight differences between them that should be carried over into
updaterepos
- A4.1.2: Remove logic from RPM
%post
- A4.1.1: The two have slight differences between them that should be carried over into
- A4.1: (Related: E1) Consolidate
- This does not affect the secondary SIMP environment directories; it's under
- A5.) On
%post
, ensures that the cacertkey (at%{_var}/simp/environments/simp/FakeCA/cacertkey
) has some random gibberish in it if it doesn't exist.- SIMP ≥ 6.4 Mappings:
- A5.1: remove this behavior from
%post
- A5.2: The
simp env
command will ensure thecacertkey
duringfix ENVIRONMENT
and afternew|update ENVIRONMENT
.
- A5.1: remove this behavior from
- SIMP ≥ 6.4 Mappings:
- A6.) On
%post
, runs simp_rpm_helper- SIMP ≤ 6.3
simp-environment
6.3.0:- Comment:
# Needed for migrating the environment data into the codedir for an initial install
/usr/local/sbin/simp_rpm_helper --rpm_dir=%{prefix} --rpm_section='post' --rpm_status=$1 --preserve --target_dir='.'
- Comment:
- SIMP ≥ 6.4 Mappings:
- A6.1: remove this behavior from
%post
- A6.2: move the initial ISO install environment deploy logic into an early stage of
simp config
. - A6.3: create new Extra/Omni environments on demand with
simp environment new
.
- A6.1: remove this behavior from
- SIMP ≤ 6.3
- A7.) On
%postun
, runs simp_rpm_helper- SIMP ≤ 6.3
simp-environment
6.3.0:- Comment:
# Needed for cleaning up the data from codedir as appropriate for an erase
/usr/local/sbin/simp_rpm_helper --rpm_dir=%{prefix} --rpm_section='postun' --rpm_status=$1 --preserve --target_dir='.'
- Comment:
- SIMP ≥ 6.4 Mappings:
- A7.1: Remove from
%postun
- A7.2: Environments can be removed with
simp env rm ENVIRONMENT
- A7.1: Remove from
- SIMP ≤ 6.3
simp-adapter
(0.1.1) [Analyzed, Mapped: 3/3, Questions: 0]
- B1.)
simp-adapter 0.1.1 %pre
and %posttrans have operations to prevent the global Hiera 3hiera.yaml.simp
file delivered with simp-adapter <= 0.0.6 and itshiera.yaml
link created in that RPM's%post
from being removed during upgrade if it may be in use.- See the simp-adapter 0.1.1 acceptance tests for details on when
hiera.yaml
files (global and environment) are modified during install/upgrade:- Upgrading simp-adapter from version <= 0.0.6
- When global
hiera.yaml
is linked tohiera.yaml.simp
- it should retain
hiera.yaml
andhiera.yaml.simp
- it should retain
- When global
hiera.yaml
is not linked tohiera.yaml.simp
- it should remove
hiera.yaml.simp
but not removehiera.yaml
- it should remove
- When global
- Uninstalling simp-adapter and legacy global Hiera 3 config exists
- When global
hiera.yaml
is linked tohiera.yaml.simp
- it should remove
hiera.yaml
andhiera.yaml.simp
, but keephiera.yaml.simpbak
- it should remove
- When global
hiera.yaml
is not linked tohiera.yaml.simp
- it should remove
hiera.yaml.simp
, but keephiera.yaml
andhiera.yaml.simpbak
- it should remove
- When global
- Upgrading simp-adapter from version <= 0.0.6
- SIMP ≥ 6.4 Mappings:
- This logic should be retained.
- See the simp-adapter 0.1.1 acceptance tests for details on when
- B2.) simp-adapter 0.1.1 %post (install-only) logic (removed after 0.1.1) creates
/etc/simp/adapter_config.yaml
:- SIMP ≤ 6.3
simp-environment
6.3.0Under the following conditions:
Only during a new RPM install (
$1 -eq 1
)Only when the Linux kernel contained
simp install
(e.g., a SIMP ISO installation)
%post
will create%{prefix}/adapter_config.yaml
with the following content:# This file was modified by simp-adapter during a SIMP install
# on ${date}:
target_directory: 'auto'
copy_rpm_data: true
- SIMP ≥ 6.4 Mappings:
- B2.1: remove this behavior from
%post
- B2.1: remove this behavior from
- SIMP ≤ 6.3
- B3.) simp-adapter 0.1.1 %post logic fixes problems with Puppet RPMs:
- SIMP ≤ 6.3
simp-environment
6.3.0- For FOSS
puppetserver
(install or upgrade)- Nails up
puppet
user UID and GID to 52, changing the UID/GID if the installed ids are incorrect. - Fixes permissions of
/opt/puppetlabs/
,/etc/puppetlabs
,/var/log/puppetlabs/
, and/var/run/puppetlabs/
to match the nailed UID/GID - Restarts
puppetserver
to pick up changes. - Creates
puppdetdb
user and group
- Nails up
- Sets the digest algorithm used by
puppet
tosha256
(install or upgrade) - Install-only, fixes the permissions of
puppet-agent
,puppetserver
, andpuppetdb
directories- References https://tickets.puppetlabs.com/browse/PA-726, which is still in the
ACCEPTED
state but has noFix version
assigned
- References https://tickets.puppetlabs.com/browse/PA-726, which is still in the
- For FOSS
- SIMP ≥ 6.4 Mappings:
- B3.1: Remove this behavior from
%post
- B3.2: The
simp env
command will ensure the correctpuppet
permissions are set duringfix ENVIRONMENT
and afternew|update ENVIRONMENT
.- B3.2.1: Note the changes above include several users, paths, and the digest algorithm.
- B3.2.2: Note that these specific UID/GIDs may no longer be necessary. Now that the
simp
command is setting them, it can just check what user is configured to own what files etc.
- B3.1: Remove this behavior from
- SIMP ≤ 6.3
simp-rsync (6.2.1) [Analyzed, Mapped:6/6, Questions: 1]
- C1.) RPM delivers a directory tree directly under
/var/simp/environments/simp/rsync/
- SIMP ≤ 6.3
simp-rsync
6.2.1:%global rsync_dir /var/simp/environments/simp/rsync
Prefix: %{rsync_dir}
%files
delivers rsync facls file to%{rsync_dir}/.rsync.facl
as a%config
file- Note: many RPM scriptlets only refer to
%{rsync_dir}
, which breaks whenrpm --prefix
=NEWPATH
- SIMP ≥ 6.4 Mappings:
- C1.1: The simp-rsync RPM only delivers files to
, which is now%{prefix}
/usr/share/simp/environment_templates/rsync/
- C1.1.1: Change
Prefix:
+%{prefix}
to/usr/share/simp/environment_templates/rsync/
- C1.1.2: Change any references to
%{rsync_dir}
from all RPM scriptlets
- C1.1.1: Change
- C1.2: The command
simp env new|update ENVIRONMENT
will copy files from%{prefix}
to/var/simp/environments/${ENVIRONMENT}/rsync/
- See A1.2
- See A1.2
- C1.1: The simp-rsync RPM only delivers files to
- SIMP ≤ 6.3
- C2.)
%pre has strange logic
- SIMP ≤ 6.3
simp-rsync
6.2.1:Pre-removes directories under
{%rsync_dir}/ ("# Remove the directories that we're going to replace with symlinks")
Includes a "
# Make sure upgrades work properly!
" section that looks like it might try to symlink theLICENSE
file as a directory if thedefault
directory is missing underbind_dns
/.- This looks like a bug in the logic
- SIMP ≥ 6.4 Mappings:
- (nice-to-have, for later): Should we expose arguments to template the skeleton DNS environment when it is copied/updates?
- C2.1: The command
simp env new|update ENVIRONMENT
will copy files from%{prefix}
to/var/simp/environments/${ENVIRONMENT}/rsync/
- This should include any logic needed to ensure a basic DNS environment
- C2.2: Remove %pre logic
- SIMP ≤ 6.3
- C3.)
%post
runssetfacl --restore
on{%rsync_dir}
, using the delivered.rsync.facl
file.- SIMP ≤ 6.3
simp-rsync
6.2.1:%{rsync_dir}
=/var/simp/environments/simp/rsync
The rsync.facl
definitions use relative paths
- SIMP ≥ 6.4 Mappings:
- The simp-rsync RPM only delivers files to
%{prefix}
(/usr/share/simp/environment_templates/rsync/
); see C1.1 - C3.1: Move
rsync.facl
into%{prefix}
(/usr/...), like the rest of the%files
- C3.2: The
simp env
command will ensure the correct FACLS are set duringfix ENVIRONMENT
and afternew|update ENVIRONMENT
. - C3.3: Remove from %post
- The simp-rsync RPM only delivers files to
- SIMP ≤ 6.3
- C4.)
%post
deletes all*.rpmnew
files under{%rsync_dir}/
- SIMP ≤ 6.3 simp-rsync 6.2.1:
- This was a workaround to problems associated with C1 (delivering files to
/var/simp/environments/simp/rsync/
as%config)
- Example problem: rsyncing meaningless
*.rpmnew
files to hosts that have no use for them
- Example problem: rsyncing meaningless
- This was a workaround to problems associated with C1 (delivering files to
- SIMP ≥ 6.4 Mappings:
- The SIMP ≤ 6.3 workaround should not be necessary now because:
- the RPM delivers a skeleton to somewhere under
/usr/share/simp
/, which is not an rsync source. - the files no longer need to be
%config
, so.rpmnew
files won't get generated, anyway
- the RPM delivers a skeleton to somewhere under
- C4.1: Remove logic from
%post
- Prerequisites: C1.1 and C1.2
- Prerequisites: C1.1 and C1.2
- The SIMP ≤ 6.3 workaround should not be necessary now because:
- SIMP ≤ 6.3 simp-rsync 6.2.1:
- C5.) RPM scriptlets auto-ensure that all directories called
RedHat
have aCentOS
symlink under/var/simp/environments/simp/rsync
- SIMP ≤ 6.3
simp-rsync
6.2.1:
symlinks%post
RedHat
directories toCentOS
under{%rsync_dir}/
- The scriptlets will add a symlink to any directory named
RedHat
, regardless of its location (buggy logic)
- The scriptlets will add a symlink to any directory named
%preun
deletes any symlinks namedCentOS
under{%rsync_dir}/
- The scriptlets will delete a
CentOS
symlink in any directory namedRedHat
, regardless of its subdirectory (buggy logic)
- The scriptlets will delete a
- SIMP ≥ 6.4 Mappings:
- C5.1: Remove logic from
%post and %preun
- Prerequisites: C1.1 and C1.2
- Prerequisites: C1.1 and C1.2
- C5.2: The
simp env
command will ensure the correct symlinks (if any) are managed duringnew|update|remove ENVIRONMENT
.
- C5.1: Remove logic from
- SIMP ≤ 6.3
- C6.) RPM scriptlets auto-ensure that all directories called
RedHat
have aCentOS
symlink under/var/simp/environments/simp/rsync
- SIMP ≥ 6.4 Mappings:
- See C5.1
- SIMP ≥ 6.4 Mappings:
simp-core (6.3.3)
[Analyzed, Mapped: 2/2]
- D1.) ISO auto.cfg
- SIMP ≤ 6.3
simp-core
6.3.3: - SIMP ≥ 6.4 Mappings:
- D1.1: Remove this logic. No extra copy of files already located in
/var/www/yum
needed. - D1.2:
simp environment
command would then install into the appropriate secondary environment directory, when these boot files exist.
- D1.1: Remove this logic. No extra copy of files already located in
- SIMP ≤ 6.3
- D2.)
%post
insimp.spec
- SIMP ≤ 6.3
simp-core
6.3.3:Includes obsolete logic to run the (defunct) `
hiera_upgrade`
script under certain conditions (-d
${puppet_environmentpath}/simp
)- This logic will never run;
hiera_upgrade
script was been gone for two years- added to
simp-core
in 2015 to support SIMP 5.1.X upgrades - removed from
simp-utils
in 2017 as part of SIMP 6.1.0
- added to
- This logic will never run;
- SIMP ≥ 6.4 Mappings:
- D2.1: Remove obsolete
hiera_upgrade
script (SIMP-6507)
- D2.1: Remove obsolete
- SIMP ≤ 6.3
simp-utils
E1.) The script
simp-utils:scripts/sbin/updaterepos
(mostly) duplicates the
%post
yum repocreaterepo
logic insimp-environment
.
- SIMP ≥ 6.4 Mappings:
- E1.1: Consolidate
%post
's yum repo creation logic into the existing scriptsimp-utils:scripts/sbin/updaterepos
.- See A4.1
- E1.2: The two have slight differences between them that should be carried over into
updaterepos
- E1.1: Consolidate
Further steps
What happens during an upgrade?
Reference
- SIMP Environments — overview & terminology of "Environments" within SIMP
- Puppet deployment scenarios — use cases SIMP supports for deploying Puppet modules and environments