Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Version History

« Previous Version 38 Next »

Table of Contents

Puppet Versions Testing

Puppet version for GitLab and TravisCI

Puppet modules will be tested in GitLab and TravisCI with the following versions:

ReleasePuppetRubyNotes
PE 2017.24.10.*2.1.9

SIMP 6.1.0, SIMP 6.2.0 also tested on this version

  • Modules may have progressed past this and that is OK. There is no longer a guarantee that 4.10.* will be supported
PE 2018.1.45.5.62.4.4

5.5.72.4.4Version to be included in SIMP 6.3.0 ISOs

Puppet version for simp-core (integration) and SIMP ISO testing

ReleasePuppetRubyNotes
SIMP 6.3.05.5.72.4.4Version to be included in SIMP 6.3.0 ISOs

OS Versions For Testing

  • SIMP acceptance tests will be executed with the following OS versions:
    • CentOS 7.5
    • CentOS 6.10
    • RedHat 7.5
    • RedHat 6.10
    • OracleLinux 7.5
    • OracleLinux 6.10
  • SIMP-6.3.0 ISOs for CentOS 7.5 and CentOS 6.10 will be built and tested.
  • If licenses are available, SIMP-6.3.0 ISOs for RedHat 7.5 and RedHat 6.10 will be built and tested.


Tests - New/Changed Features

The tests in this section focus on new features or major, SIMP-wide changes.

Legend:

(lightbulb) - Means this version needs to be tested

(grey lightbulb) - This version does not need to be tested.

(question) - Testing probably sufficient from other versions.

(error) - Testing has failed.

(tick) - Testing completed and Passed.

blank - This test is not OS specific

greenthese test are automated



Component

Required

Tests

TestEL7EL6OL7OL6RH7RH6StatusNotes
1simp-core
Verify Puppet File matches release versions






This must be completed before any other tests are done

2Puppet 5/hiera 51

For all core components, including simp-core:

  • Verify a .gitlab-ci.yml exists and the project is configured in gitlab
  • Verify that .gitlab-ci.yml is running correct testing matrix (see above) and that puppet 5 tests are no longer "allow failure" .
  • For puppet modules and simp-core tests, make sure BEAKER_PUPPET_COLLECTION: 'puppet5' is set for puppet 5 tests.






SIMP-5529
  • This addresses Puppet 5 part of 6.3
  • Sample .gitlab-ci.yml
  • Some components have acceptance tests that are not being executed in GitLab because no corresponding project exists in GitLab and/or the project is missing a .gitlab-ci.yml file.  simp-utils is one such project.

SIMP-5585 - Getting issue details... STATUS


3Puppet 5/Hiera 52For all core components: verify full matrix acceptance tests pass.

(lightbulb)


(lightbulb)(lightbulb)(lightbulb)(grey lightbulb)(grey lightbulb)
  • This addresses puppet 5/hiera 5 update part of 6.3. 
  • 'beaker:suites', by default will run only the 'default' suite and any suites that have added themselves to the 'default' suite via their metadata.yml files.   So, we Nneed to run beaker:suites[ALL,ALL] for pupmod-simp-* modules with BEAKER_PUPPET_COLLECTION=puppet5 and with and without BEAKER_FIPS=yes.

SIMP-5537 - Getting issue details... STATUS

4simp-core
Verify puppetfile and dependencies.yaml by building ISO(lightbulb)(lightbulb)(grey lightbulb)(grey lightbulb)(lightbulb)(lightbulb)
  • This is a normal release testing activity.
  • The dependencies.yaml has to be manually reviewed each release to make sure it has the appropriate list of dependencies for non-SIMP-owned modules, appropriate other dependendencies for  any modules that require it, and appropriate obsoletes and RPM release qualifiers.
  • Should result in ISO for CentOS. 
  • Just verify it can be built for RedHat (AWS?)

SIMP-5540 - Getting issue details... STATUS

5simp-packer4Verify installation from ISO passes using simp-packer on ISO, both FIPS and no-FIPS, encrypted+FIPS(lightbulb)(lightbulb)(grey lightbulb)(grey lightbulb)(lightbulb)(lightbulb)
  • This is a normal release testing activity.
  • Should result in 6 vagrant boxes for testing, 3 for CentOS 7 and 3 for CentOS 6

SIMP-5548 - Getting issue details... STATUS

6

simp-core


Verify installation from RPM.

This simp-core test also verifies

  • rsyslog forwarding
  • ldap logins
  • local logins



(lightbulb)(lightbulb)(lightbulb)(lightbulb)(lightbulb)(lightbulb)

This is a normal release testing activity.

In simp-core run using the tar file generated during ISO build

set BEAKER_release_tarball=<location of the tarball  for el7>

  • beaker:suites[install_from_tar]
  • SIMP_BEAKER_OS=oel beaker:suites[install_from_tar]

set BEAKER_release_tarball=<location of the tarball  for el6>

  • beaker:suites[install_from_tar,el6_server]
  • SIMP_BEAKER_OS=oel beaker:suites[install_from_tar,el6_server]

SIMP-5598 - Getting issue details... STATUS

 
7simp-core

Verify installation from RPM in FIPS

This test also verifies

  • rsyslog forwarding
  • ldap logins
  • local logins
(lightbulb)(lightbulb)(lightbulb)(lightbulb)(grey lightbulb)(grey lightbulb)

This is a normal release testing activity.

For all tests set BEAKER_fips=yes

set BEAKER_release_tarball=<location of the tarball  for el7>

  • beaker:suites[install_from_tar]
  • SIMP_BEAKER_OS=oel beaker:suites[install_from_tar]

set BEAKER_release_tarball=<location of the tarball  for el6>

  • beaker:suites[install_from_tar,el6_server]
  • SIMP_BEAKER_OS=oel beaker:suites[install_from_tar,el6_server]


SIMP-5598 - Getting issue details... STATUS

8installation
Verify installation from RPM for PE(lightbulb)(lightbulb)(lightbulb)(lightbulb)(question)(question)

RedHat built on AWS?

simp cli was updated make sure you test out simp cli

9tlog, mate, x2go, ima5Verify packages.yaml files have all dependencies for new modules by installing these from a system built by an ISO with no internet connectivity.(lightbulb)(lightbulb)(grey lightbulb)(grey lightbulb)(lightbulb)(lightbulb)

RedHat built on AWS?


SIMP-5549 - Getting issue details... STATUS


10tpm25Verify packages.yaml files have all dependencies by installing these from a system built by an ISO with no internet connectivity.(lightbulb)(grey lightbulb)(grey lightbulb)(grey lightbulb)(question)(grey lightbulb)

RedHat on AWS? These modules don't work in 6 or on OracleLinux.  I think testing in CentOS 7 is good enough.


SIMP-5550 - Getting issue details... STATUS

11compliance_markup
Verify turning on compliance enforcement works.  Review compliance report.(lightbulb)(lightbulb)




This addresses STIG updates and Compliance Markup modules changes.

Should we run an oscap scan as well?

SIMP-5601 - Getting issue details... STATUS

12simp-core
Verify Changelog.rst







13simp-doc,
'Upgrading SIMP' has an entry for the latest version and is correct






See 6.3 Planning page in Jira

SIMP-5543 - Getting issue details... STATUS

14simp-doc, tpm2, ima
Review TPM documentation to make sure it is up to date






TPM and IMA were separated out for this release.

SIMP-5224 - Getting issue details... STATUS

15simp-doc, tlog
Review Session Auditing and links  in read the docs to make sure TLOG documentation is correct.






TLOG was added for this release

SIMP-5224 - Getting issue details... STATUS

16simp-doc, x2go, mate
Review How to manage Workstation entry for updates for mate and x2go






X2go, Mate  were added for this release

SIMP-5224 - Getting issue details... STATUS

17simp-doc, hiera 5 and puppet 5 upgrade
Review docs to make sure the hiera data directory has been updated to reflect changes.  And that the version of puppet currently supported has been updated to puppet 5. 






hiera and puppet updated to version 5.

SIMP-5224 - Getting issue details... STATUS

18simp-integration_test
SIMP server upgrade from RPM using the upgrade instructions.  Document any steps that are needed outside the ordinary upgrade.  Then, if there are additional steps, have someone else use the docs to do the upgrade to verify.

(lightbulb)

(lightbulb)

(lightbulb)


(lightbulb)

(question)


(question) SIMP-5383 - Getting issue details... STATUS

See simp-integration_test for automated upgrade test.

this should be done for servers in both FIPS and NOFIPS mode.

A straight `yum update` following the General Upgrade instructions for RPM-based SIMP installations will kill puppetserver and prevent it from restarting.   (This is  SIMP-5021 - Getting issue details... STATUS  + 6.1.0)

The fix is procedural: upgrade steps (and fixes)  in "Upgrading from SIMP-6.1.0 to SIMP-6.2.0" procedure 

Pay attention to Auditd upgrade because it is a major version upgrade.


19PE

Upgrade PE SIMP server from 6.2

puppet 4 to puppet 5  (PE 2018.1) and simp 6.3

(lightbulb)


(lightbulb)(lightbulb)(lightbulb)


Not sure what OS it is most important to test this on.
20RPMs and Released Modules

Dogfooding








Or "drink your own home brew", which ever you prefer.

  • CentOS 7: SIMP server and SIMP client
  • CentOS 6: SIMP client only

SIMP-5577 - Getting issue details... STATUS

21Final RPM testingRPMs have been uploaded to packagecloudRun the simp-core install_from_rpm test and versify that it passes.   This will pull the packages from package.io and do an install.(lightbulb)(lightbulb)(lightbulb)(lightbulb)



in simp-core run

  • SIMP_BEAKER_OS=oel beaker:suites[install_from_rpm]
  • beaker:suites[install_from_rpm]
  • beaker:suites[install_from_rpm,el6_server]
  • SIMP_BEAKER_OS=oel beaker:suites[install_from_rpm,el6_server]

and then do it all again with BEAKER_fips=yes

22Puppet Forge TestingAfter modules have been tag with an annotated tagrun the simp-core test install_from_core_module test to test the modules pushed to Puppet Forge(lightbulb)(lightbulb)(lightbulb)

(lightbulb)





in simp-core run

  • beaker:suites[install_from_core_module]
  • SIMP_BEAKER_OS=oel beaker:suites[install_from_core_module]
  • beaker:suites[install_from_core_module,el6_server]
  • SIMP_BEAKER_OS=oel beaker:suites[install_from_core_module,el6_server]

and then do it all again with BEAKER_fips=yes


Tests - Regression/Integration Testing

The tests in this section are regression tests to ensure unmodified capabilities still function.  These tests should use the packer boxes created in the New/Changed Feature tests, were applicable.  Note that most of these test should eventually be automated by simp-packer, simp-integration_test, or simp-core tests.


TestEL6EL7Notes
1

Verify non-standard BIOS boot options from the ISO:

  •   choose own partitions
  •   minimum installation


(SIMP, FIPS and Encryption are all tested by Packer.)

SIMP-5551 - Getting issue details... STATUS

2

Verify all boot Options in UEFI mode from ISO:

  • FIPS
  • no-FIPS
  • FIPS+Encrypted
  • FIPS+choose own partitions
  • minimum


 SIMP-5587 - Getting issue details... STATUS
3PXE Boot Testing BIOS

 SIMP-5579 - Getting issue details... STATUS
4PXE Boot Testing UEFI

 SIMP-5580 - Getting issue details... STATUS
5Bootstrap simp-lite scenario

 SIMP-5588 - Getting issue details... STATUS
6

Verify simp-lite operations:

  • login operations (PAM, LDAP, local user)
  • NFS operations (home directory)
  • logging operations (rsyslog)
  • auditing operations



7Bootstrap poss scenario


8Bootstrap run-once scenario


9Verify ability to switch FIPS from on to off

Switching from off to on is known to have problems because of ciphers.

SIMP-5605 - Getting issue details... STATUS

10Verify ability to turn on and off auditing and selinux

 SIMP-5605 - Getting issue details... STATUS
11

General Review of documentation:

  • Getting Started
  • User Guide
  • HOW TOs
  • FAQS



12simp-utils:

executables that are not tested otherwise work as advertised

  • unpack_dvd


The following have acceptance tests that simply need to be executed:

  • gen_ldap_update
  • updaterepos




  • No labels