6.3.0 Testing Matrix

Versions for gitlab and travis that release 6.3 will be tested with:

Release	Puppet	Ruby	Notes
PE 2017.2	4.10.*	2.1.9	SIMP 6.1.0, SIMP 6.2.0 also tested on this version
PE 2017.3.6	5.3.6	2.4.4
PE 2018.1	5.5.6	2.4.4

- Means this version needs to be tested

- This version does not need to be tested.

- Testing probably sufficient from other versions.

- Testing has failed.

- Testing completed and Passed.

blank - This test is not OS specific

	Component	Required Tests	Test	Status	Notes
1	simp-core		Verify Puppet File matches release versions		This must be complete before any other tests are done
2	Puppet 5/hiera 5	1	For all core modules: Verify that gitlab-ci.yml is running correct testing matrix (see above) and that puppet 5 tests are no longer "allow failure" . Make sure BEAKER_PUPPET_COLLECTION: 'puppet5' is set for puppet 5 tests.		gitlab-ci.yml sample gitlab-ci.yml This addresses Puppet 5 part of 6.3
3	Puppet 5/Hiera 5	2	For all core modules: verify full matrix acceptance tests pass.		The test by default will run only default. This addresses puppet 5/hiera 5 update part of 6.3
4	simp-core		verify puppetfile and dependencies.yaml by building ISO		Should result in ISO for download for Centos. Just verify it can be built for Redhat Genral release testing.
5	simp-packer	4	Verify installtion from ISO passes using simp-packer on ISO, both FIPS and no-FIPS, encrypted		Should result in vagrant 6 vagrant boxes for testsing. General Release
6	installation	4	Verify installation of non standard modes works (min and chose your disks partitions)		General Release
7	installation		verify installation from RPM		General release
8	installation		verify installation from RPM in FIPS		Don't know if OL has fips mode but it is not tested anywhere else, but RedHat family is tested via packer.
9	installation		verify installation from RPM for PE		Redhat built on AWS? simp cli was updated make sure you test out simp cli
10	tlog, mate, x2go, ima	5	Verify dependencies.yaml has all dependencies for new modules by installing these from a system built by an iso with no internet connectivity.		Redhat built on AWS?
11	tpm2	5	Verify dependencies.yaml has all dependencies by installing these from a system built by an iso with no internet connectivity.		Redhat on AWS? These modules don't work in 6 or on OracleLinux. I think testing in CentOS 7 is good enough.
12	compliance_markup		Verify turning on compliance enforcement works. review compliance report.		This addresses stig updates and Compliance Markup modules changes.
13	simp-core		Verify Changelog.rst
14	simp-doc,		'Upgrading SIMP' has an entry for the latest version and is correct		See 6.3 Planning page in JIRA
15	simp-doc, tpm2, ima		Review TPM documentation to make sure it is up to date		TPM and IMA were seperated out for this release.
16	simp-doc, tlog		Review Session Auditing and links in read the docs to make sure TLOG documentation is correct.		TLOG was added for this release
17	simp-doc, x2go, mate		Review How to manage Workstation entry for updates for mate and x2go		X2go, Mate were added for this release
18	simp-doc		Genral Review of Documents: Getting Started User Guide HOW TOs FAQS		General Review of docs.
19	ISO		Functional UEFI Boot		Not configured in Simp-packer yet. Still have to do this by hand.
20	ISO		SIMP server upgrade from RPM	SIMP-5383 - Getting issue details... STATUS	A straight `yum update` following the General Upgrade instructions for RPM-based SIMP installations will kill puppetserver and prevent it from restarting. (This is SIMP-5021 - Getting issue details... STATUS + 6.1.0) The fix is procedural: upgrade steps (and fixes) in "Upgrading from SIMP-6.1.0 to SIMP-6.2.0" procedure Pay attention to Auditd upgrade because it is a major version upgrade.
21	PE		Upgrade PE SIMP server from 6.2 puppet 4 to puppet 5 (PE 2018.1) and simp 6.3		Not sure what OS it is most important to test this on.
22	ISO	5	SIMP Client PXE Boot Testing - BIOS
23	ISO	5	SIMP Client PXE Boot Testing - UEFI
24	ISO		Dogfooding

mp-util