BLUE | New version since last SIMP release |
YELLOW | New component since last SIMP release |
GRAY | Omitted component |
WHITE | Same version as last SIMP release |
RED | TBD: Outstanding problem |
BOLD | Ready to release |
BOLD ITALICS | Tag published but not verified as released |
GREEN | New version tagged, Forge released, RPM released |
Component | Proposed Version | Version in 6.3.2 | Latest Version | Unit Test Status | Acceptance Test Status | GitHub Released | Forge Released | RPM Released | Changelog |
---|---|---|---|---|---|---|---|---|---|
simp-core | 6.3.3 | 6.3.2 | 6.3.2 | <Link to Travis CI> | <Link to GitLab CI> | Y | Y | Y | |
simp-extras | 6.3.3 | 6.3.2 | 6.3.2 | <Link to Travis CI> | <Link to GitLab CI> | N/A | N/A | Y | |
simp-doc | 6.3.3 | 6.3.2 | 6.3.2 | <Link to Travis CI> | <Link to GitLab CI> | Y | N/A | Y | |
pupmod-simp-sssd | 6.1.5 | 6.1.4 | 6.1.4 | <Link to Travis CI> | <Link to GitLab CI> | Y | Y | Y |
The full release testing procedure is documented in the Official SIMP Docs.
Puppet modules will be tested in GitLab and TravisCI with the following versions:
Release | Puppet | Ruby | Notes |
---|---|---|---|
SIMP 6.3.3-0 | Version to be included in SIMP 6.3.3-0 ISOs |
Release | Puppet | Ruby | Notes |
---|---|---|---|
PE 2017.2 | 4.10.* | 2.1.9 | SIMP 6.1.0, SIMP 6.2.0 also tested on this version
|
PE 2018.1.4 | 5.5.6 | 2.4.4 | |
Version to be included in SIMP 6.3.3-0 ISOs |
The tests in this section focus on new features or major, SIMP-wide changes.
Legend:
- Means this version needs to be tested
- This version does not need to be tested.
- Testing probably sufficient from other versions.
- Testing has failed.
- Testing completed and Passed.
blank - This test is not OS specific
green | these test are automated |
---|
Component | Required Tasks | Test | EL7 | EL6 | OL7 | OL6 | RH7 | RH6 | Status | Notes | |
---|---|---|---|---|---|---|---|---|---|---|---|
1 | simp-core | Verify Puppet File matches release versions | ![]() | ![]() | ![]() | ![]() | This must be completed before any other tests are done | ||||
2 | installation | Verify installation from ISO | ![]() | ![]() | ![]() | ![]() | |||||
3 | installation | Verify installation from RPM | |||||||||
4 | simp-core | Create and verify Changelog.rst | ![]() | ![]() | ![]() | ![]() | This addresses STIG updates and Compliance Markup modules changes. | ||||
5 | simp-doc, | 'Upgrading SIMP' has an entry for the latest version and is correct | ![]() | ![]() | |||||||
6 | simp-integration_test | SIMP server upgrade from RPM using the upgrade instructions. Document any steps that are needed outside the ordinary upgrade. Then, if there are additional steps, have someone else use the docs to do the upgrade to verify. | Tiny change and no ISO updates | See simp-integration_test for automated upgrade test. this should be done for servers in both FIPS and NOFIPS mode. | |||||||
7 | RPMs and Released Modules | Dogfooding | ![]() | ![]() | Or "drink your own home brew", which ever you prefer.
| ||||||
8 | Final RPM testing | RPMs have been uploaded to packagecloud | Run the simp-core install_from_rpm test and versify that it passes. This will pull the packages from packagecloud.io and do an install. | in simp-core run
and then do it all again with BEAKER_fips=yes | |||||||
9 | Puppet Forge Testing | After modules have been tag with an annotated tag | run the simp-core test install_from_core_module test to test the modules pushed to Puppet Forge | in simp-core run
and then do it all again with BEAKER_fips=yes |
The tests in this section are regression tests to ensure unmodified capabilities still function. These tests should use the packer boxes created in the New/Changed Feature tests, were applicable.
Note that most of these test should eventually be automated by simp-packer, simp-integration_test, or simp-core tests so check there and remove them from these list as they are automated elsewhere. |
Test | EL6 | EL7 | Status | Notes | |
---|---|---|---|---|---|
1 | Verify non-standard BIOS boot options from the ISO:
| Skipped for 6.3.3 due to no relevant changes | |||
2 | Verify all boot Options in UEFI mode from ISO:
| Skipped for 6.3.3 due to no relevant changes | |||
3 | PXE Boot Testing BIOS | Skipped for 6.3.3 due to no relevant changes | |||
4 | PXE Boot Testing UEFI | Skipped for 6.3.3 due to no relevant changes | |||
5 | Bootstrap simp-lite scenario | Skipped for 6.3.3 due to no relevant changes | |||
6 | Verify simp-lite operations:
| ||||
7 | Bootstrap poss scenario | Skipped for 6.3.3 due to no relevant changes | |||
8 | Bootstrap run-once scenario | Skipped for 6.3.3 due to no relevant changes | |||
9 | Verify ability to switch FIPS from on to off | Skipped for 6.3.3 due to no relevant changes | |||
10 | Verify ability to turn on and off auditing and selinux | Skipped for 6.3.3 due to no relevant changes | |||
11 | General Review of documentation:
| Skipped for 6.3.3 due to no relevant changes | |||
12 | simp-utils: executables that are not tested otherwise work as advertised
| Skipped for 6.3.3 due to no relevant changes |
Change the query in the table below to set fixVersion to