Liz Test - Release Ticket Tables

1

Release Components

Identify components to be released, verify tests pass, push annotated tags, update SIMP release confluence page. https://simp.readthedocs.io/en/latest/contributors_guide/maintenance/Tagging_and_Releasing_Components.html

3

Y

Y

Y

-

-

-

-

-

-

-

-

-

1-1

Identify components to release

1

-

-

-

-

-

-

-

-

-

-

-

-

1-2

Release components

2

-

-

-

-

-

-

-

-

-

-

-

-

2

Update simp-core pre-release tests with release-specific changes

Update simp-core default, ipa, and install_from_tar acceptance test suites for release-specific changes. This includes updating the nodesets for the supported OSs.

simp-core

5

Y

N

N

-

-

-

-

-

-

-

-

-

Tests will be executed for the appropriate OSs permutations in the nodeset, but separate tickets per OS should not be created.

2-1

Update tests and nodesets

4

-

-

-

-

-

-

-

-

-

-

-

-

2-2

Review updates

1

-

-

-

-

-

-

-

-

-

-

-

-

3

Update simp-packer with release-specific changes

Update simp-packer for release-specific changes

simp-packer

3

Y

N

N

-

-

-

-

-

-

-

-

-

3-1

Update code and documentation

2

-

-

-

-

-

-

-

-

-

-

-

-

3-2

Review changes

1

-

-

-

-

-

-

-

-

-

-

-

-

4

Update simp-core with released components

Update simp-core files (Puppetfile.pinned, metadata.json, src/assets/simp/build/simp.spec) and verify ISOs can be built and default acceptance test suite passes. Pushing to GitLab will execute the

simp-core

1

3

Y

Y

Y

-

-

-

-

-

-

-

-

-

4-1

Update files, build ISOs and test

2

-

-

-

-

-

-

-

-

-

-

-

-

4-2

Review updates

1

-

-

-

-

-

-

-

-

-

-

-

-

5

Create ISOs, validate and publish the ISOs and tar files

simp-core

4

Y

Y

Y

-

-

-

-

-

-

-

-

-

Only the child should be replicated for OSs for which we build an ISO

5-1

Build ISO

-

-

-

N

Y

Y

N

N

N

N

N

N

5-2

Publish artifacts to simp-project.com

-

-

-

-

-

-

-

-

-

-

-

-

6

Validate ISO by building packer boxes

Use simp-packer to build SIMP server packer boxes. Basic bootstrap validation is done as part of the packer build.

5

Y

Y

Y

-

-

-

-

-

-

-

-

-

6-1

Build packer boxes BIOS boot

Boxes to build: FIPS enabled with encrypted disk, FIPS enabled with unencrypted disk, FIPS disabled with encrypted disk, not FIPS disabled with unencrypted disk

-

-

-

N

Y

Y

N

N

N

N

N

N

6-2

Build packer boxes UEFI boot

Boxes to build: FIPS enabled with encrypted disk, FIPS enabled with unencrypted disk, FIPS disabled with encrypted disk, not FIPS disabled with unencrypted disk

-

-

-

N

Y

Y

N

N

N

N

N

N

7

Verify installation from RPM

Run simp-core’s install_from_tar test

simp-core

5

Y

Y

Y

-

-

-

-

-

-

-

-

-

Only the child should be replicated for OSs for which there is a nodeset for the install_from_tar test

Depending upon changes made from Alpha to RC1 and from RC1 to Final, we may not have to execute these tests with each interim release.

7-1

Execute test

See https://github.com/simp/simp-core/blob/master/spec/acceptance/suites/README.md for description of environment variables that can be set to point to the tar file

-

-

-

N

Y

Y

N

N

N

N

N

N

8

Create upgrade instructions

Document any steps that are needed outside of the generic upgrade instructions. 

simp-doc

4

10

N

Y

Y

N

N

N

N

N

N

First time we deliver an EL8 ISO, we will only have upgrade instructions for EL7

Manual tests for now. Will migrate to simp-integration_test for automated upgrade test.

8-1

Manually execute upgrade with FIPS enabled

4

8-2

Manually execute upgrade with FIPS disabled

1

8-2

Write upgrade instructions

4

8-3

Review instructions

Review the instructions for clarity, grammar, spelling, formatting, etc. Verification will be done in a separate ticket

1

9

Verify upgrade instructions

Verify upgrade instructions and make any necessary adjustments to them.

simp-doc

8

9-1

Execute instructions

2

9-2

Update instructions

1

10

Create initial SIMP changelog

simp-doc

8

Y

N

N

-

-

-

-

-

-

-

-

-

10-1

Create changelog

6

10-2

Review changes

2

11

Identify release-specific tests

Based on the initial Changelog for the release, identify integration tests that must be done with a fully configured SIMP server and (if necessary) clients. These should be tests that are not adequately tested in component acceptance tests.

10

5

Y

N

N

-

-

-

-

-

-

-

-

-

11-1

Determine release-specific manual tests

4

11-2

Create tickets for each

1

12

Update Changelog for bug fixes

simp-doc

3

N

Y

Y

-

-

-

-

-

-

-

-

-

12-1

Identify bug fixes and update changelog

2

12-2

Review changes

1

13

PXE boot UEFI

14

PXE boot BIOS

15

Verify non-standard ISO UEFI boot options

Manually verify the choose your own partitions and minimum installation ISO boot options

15-1

Verify the choose your own partitions option

15-2

Verify the minimum installation option

16

Verify non-standard ISO BIOS boot options

Manually verify the ‘choose your own partitions' and ‘minimum installation’ ISO boot options

16-1

Verify the choose your own partitions option

16-2

Verify the minimum installation option

7

RPMs have been uploaded to packagecloud

in simp-core run

• SIMP_BEAKER_OS=oel beaker:suites[install_from_rpm]

• beaker:suites[install_from_rpm]

• beaker:suites[install_from_rpm,el6_server]

• SIMP_BEAKER_OS=oel beaker:suites[install_from_rpm,el6_server]

and then do it all again with BEAKER_fips=yes

Note that most of these test should eventually be automated by simp-packer, simp-integration_test, or simp-core tests so check there and remove them from these list as they are automated elsewhere.

1

Verify non-standard BIOS boot options from the ISO:

•   choose own partitions

•   minimum installation

(SIMP, FIPS and Encryption are all tested by Packer.)

2

Verify all boot Options in UEFI mode from ISO:

• FIPS

• no-FIPS

• FIPS+Encrypted

• FIPS+choose own partitions

• minimum

3

PXE Boot Testing BIOS