...
blank - This test is not OS specific
| Component | Required Tests | Test | EL7 | EL6 | OL7 | OL6 | RH7 | RH6 | Status | Notes | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | simp-core | Verify Puppet File matches release versions | This must be complete before any other tests are done | ||||||||
| 2 | Puppet 5/hiera 5 | 1 | For all core modules: Verify that gitlab-ci.yml is running correct testing matrix (see above) and that puppet 5 tests are no longer "allow failure" . Make sure BEAKER_PUPPET_COLLECTION: 'puppet5' is set for puppet 5 tests. | sample gitlab-ci.yml This addresses Puppet 5 part of 6.3 | |||||||
| 3 | Puppet 5/Hiera 5 | 2 | For all core modules: verify full matrix acceptance tests pass. |
|  | | |  | | The test by default will run only default. This addresses puppet 5/hiera 5 update part of 6.3. Need to run beaker:suites[ALL,ALL] for pupmod-simp-* modules. Perhaps we could create and entry in gitlab-cm.yml called "Release" That will run this. | |
| 4 | simp-core | verify puppetfile and dependencies.yaml by building ISO | | | | | | | Should result in ISO for download for Centos. Just verify it can be built for Redhat Genral release testing. | ||
| 5 | simp-packer | 4 | Verify installtion from ISO passes using simp-packer on ISO, both FIPS and no-FIPS, encrypted | | | | | | | Should result in vagrant 6 vagrant boxes for testsing. General Release | |
| 6 | installation | verify installation from RPM | | | | | | | General release. This test is accomplished by simp-core acceptance test install_from_rpm for CentOS 6&7. |
If run with SIMP_BEAKER_OS='oel' it will run the oel tests. | |||||||||||||||||||||||||||||||
| 7 | installation | verify installation from RPM in FIPS | | | | | | | General release. This test is accomplished by simp-core acceptance test install_from_rpm for CentOS 6&7. Can we expand the node set to OEL 6&7? | ||||||||||||||||||||||
| 8 | installation | verify installation from RPM for PE | | | | |  | | Redhat built on AWS? simp cli was updated make sure you test out simp cli | ||||||||||||||||||||||
| 9 | tlog, mate, x2go, ima | 5 | Verify dependencies.yaml has all dependencies for new modules by installing these from a system built by an iso with no internet connectivity. | | | | | | | Redhat built on AWS? | |||||||||||||||||||||
| 10 | tpm2 | 5 | Verify dependencies.yaml has all dependencies by installing these from a system built by an iso with no internet connectivity. | | | | | | | Redhat on AWS? These modules don't work in 6 or on OracleLinux. I think testing in CentOS 7 is good enough. | |||||||||||||||||||||
| 11 | compliance_markup | Verify turning on compliance enforcement works. review compliance report. | | | This addresses stig updates and Compliance Markup modules changes. | ||||||||||||||||||||||||||
| 12 | simp-core | Verify Changelog.rst | |||||||||||||||||||||||||||||
| 13 | simp-doc, | 'Upgrading SIMP' has an entry for the latest version and is correct | See 6.3 Planning page in JIRA | ||||||||||||||||||||||||||||
| 14 | simp-doc, tpm2, ima | Review TPM documentation to make sure it is up to date | TPM and IMA were seperated out for this release. | ||||||||||||||||||||||||||||
| 15 | simp-doc, tlog | Review Session Auditing and links in read the docs to make sure TLOG documentation is correct. | TLOG was added for this release | ||||||||||||||||||||||||||||
| 16 | simp-doc, x2go, mate | Review How to manage Workstation entry for updates for mate and x2go | X2go, Mate were added for this release | ||||||||||||||||||||||||||||
| 17 | simp-doc, hiera 5 and puppet 5 upgrade | Review docs to make sure the hiera data directory has been updated to reflect changes. And that the version of puppet currently supported has been updated to puppet 5. | hiera and puppet updated to version 5. | ||||||||||||||||||||||||||||
| 18 | ISO | SIMP server upgrade from RPM using the upgrade instructions. |
| |
| |
| |
| See simp-integration_test for automated upgrade test. this should be done for servers in both FIPS and NOFIPS mode. A straight `yum update` following the General Upgrade instructions for RPM-based SIMP installations will kill puppetserver and prevent it from restarting. (This is
The fix is procedural: upgrade steps (and fixes) in "Upgrading from SIMP-6.1.0 to SIMP-6.2.0" procedure Pay attention to Auditd upgrade because it is a major version upgrade. | |||||||||||||||||||||
| 19 | PE | Upgrade PE SIMP server from 6.2 puppet 4 to puppet 5 (PE 2018.1) and simp 6.3 |
| | | | Not sure what OS it is most important to test this on. | ||||||||||||||||||||||||
| 20 | ISO | Dogfooding | Or "drink your own home brew", which ever you prefer. | ||||||||||||||||||||||||||||
| 21 | Final RPM testing | RPMs have been uploaded to packagecloud | Run the simp-core install_from_rpm test and versify that it passes. This will pull the packages from package.io and do an install. | | | | | in simp-core run
| |||||||||||||||||||||||
| 22 | Puppet Forge Testing | After RPMs have been tag with an annotated tag | run the simp-core test install_from_core_module test to test the modules pushed to Puppet Forge | | | |
| in simp-core run
|
Regression/Integration Testing (most of these test should eventually be tested by simp-packer or simp-core tests. Use the packer boxes created to do testing were applicable.)
...