Skip to end of metadata
Go to start of metadata

You are viewing an old version of this content. View the current version.

Compare with Current View Version History

« Previous Version 51 Next »

Introduction

This page lists three tables with typical tasks that are required for a SIMP release:

  • Initial Release Candidate Tickets - Typical tasks that are required for the initial release candidate (e.g., ‘Alpha’).

  • Interim Release Candidate Tickets - Typical tasks that are required for interim release candidates (e.g., ‘Beta’, ‘RC1’).

  • Final Release Tickets - Typical tasks that are required for the final release.

Copies of these tables should be customized for individual SIMP releases and then used to generate the described release tickets.

  • Some test tasks from the initial release candidate table may need to be added to an interim release candidate or final release table, when the bugs fixed could impact the test results.

  • Some test and documentation tasks may be omitted because the scope of the release is very targeted (e.g., a patch SIMP release).

In a release cycle, it probably makes the most sense to generate the tickets for just a single release candidate at a time, not for all the release candidates to get to the final release. In other words, only generate tickets from one table at a time. This will allow us to fine tune how many release candidates are required and the tests that need to be executed, based on the bugs found with each release candidate.

Also, be sure to update these tables when

  • the list of supported OSs changes

  • manual tests have been automated

  • new capabilities must be added to the integration regression test list.

Table Conventions

Unless otherwise noted, the following conventions apply to each table:

  • Stories are indicated with a <number> ID, whereas subtasks to a story are indicated with <parent number>-<child number> ID.

  • The Blocker field of a story is used to indicate that this story/subtask requires the listed stories/subtasks to be completed before it can be done.

  • The Story Points field of a subtask may contain an initial guess at the appropriate story points for that subtask. Adjust as needed!

  • When Y appears in one of the OS fields of a story (e.g. EL8) , that story will be replicated for the permutation noted.

  • When Y appears in one of the OS fields of a subtask, that subtask will be replicated for the permutation noted within the story.

  • N, -, and a blank entry are all treated as ‘do not replicate’ indicators.

  • The Notes field will not be included in the ticket and is intended for maintainers of these tables.

Initial Release Candidate Tickets

The table below lists the typical tasks that are required for the initial release candidate (e.g., ‘Alpha’). The largest number of bugs are found during the initial release candidate. So, some of the tasks may have to be repeated after bugs are fixed and some of the tasks may be blocked until bugs are fixed. The sprint planning for this release should be flexible, accordingly.

ID

Ticket Summary (< 100 chars)

Description

Component

Blockers

Story Points

EL7

EL8

OEL7

OEL8

RH7

RH8

Notes

1

Release Components

Identify components to be released, verify tests pass, push annotated tags, update SIMP release confluence page. https://simp.readthedocs.io/en/latest/contributors_guide/maintenance/Tagging_and_Releasing_Components.html

-

3

-

-

-

-

-

-

1-1

Identify components to release

-

1

-

-

-

-

-

-

1-2

Release components

1-1

2

-

-

-

-

-

-

10

Create initial SIMP changelog


simp-doc

8

-

-

-

-

-

-

10-1

Create changelog

6

-

-

-

-

-

-

10-2

Review changes

2

-

-

-

-

-

-

2

Update simp-core pre-release tests with release-specific changes

Update simp-core default, ipa, install_from_tar, and simp_lite acceptance test suites for release-specific changes. This includes updating the nodesets and .gitlab-ci.yml for any changes to the supported OSs.

simp-core

-

5

-

-

-

-

-

-

Ideally, this should be done after the initial changelog has been generated, as the changelog informs the work to be done.

Tests will be executed for the appropriate OSs permutations in the nodeset, but separate tickets per OS should not be created.

2-1

Update tests and nodesets

-

4

-

-

-

-

-

-

2-2

Review updates

2-1

1

-

-

-

-

-

-

3

Update simp-packer with release-specific changes

Update simp-packer for release-specific changes. Be sure to tag the previous simp-packer version, if the updates will break testing functionality built for the prior SIMP release.

simp-packer

-

3

-

-

-

-

-

-

Ideally, this should be done after the initial changelog has been generated, as the changelog informs the work to be done.

3-1

Update code and documentation

-

2

-

-

-

-

-

-

3-2

Review changes

3-1

1

-

-

-

-

-

-

11

Identify release-specific tests and document updates

Based on the initial Changelog for the release, identify (1) integration tests that must be done with a fully configured SIMP server and (if necessary) clients and (2) simp-doc documentation that may be affected by the changes. The tests should be tests that are not adequately tested in component acceptance tests.

10

7

-

-

-

-

-

-

11-1

Determine release-specific manual tests

4

11-2

Determine simp-doc pages that need to be reviewed for accuracy

Identify pages that require an indepth examination.

2

-

-

-

-

-

-

11-3

Create tickets for each test and simp-doc page review

1

-

-

-

-

-

-

4

Update simp-core with released components

Update simp-core files (Puppetfile.pinned, metadata.json, src/assets/simp/build/simp.spec) and verify ISOs can be built and the default and ipa acceptance test suites pass.

simp-core

1,2

3

-

-

-

-

-

-

4-1

Update files, build ISOs and execute tests

-

2

-

-

-

-

-

-

4-2

Review updates

4-1

1

-

-

-

-

-

-

5

Create test ISO, and publish the ISO and its tar file

simp-core

4

2

Y

Y

-

-

-

-

This step may actually have to be done several times before we get to an initial release candidate build that can be published. If additional tickets are warranted, they can be created on the fly.

5-1

Build ISO

Use official RPM signing keys when the artifacts are to be published anywhere at simp-project.com.

-

1

Y

Y

-

-

-

-

5-2

Publish artifacts

Interim artifacts may be published to unstable folders at simp-project.com or other unofficial shared locations.

5-1

1

-

-

-

-

-

-

6

Validate ISO by building packer boxes

Use simp-packer to build SIMP server packer boxes. Basic bootstrap validation is done as part of the packer build.

5

Y

Y

-

-

-

-

6-1

Build BIOS boot box with FIPS enabled and encrypted disk

-

-

-

-

-

-

-

6-2

Build BIOS boot box with FIPS enabled and unencrypted disk

-

-

-

-

-

-

-

6-3

Build BIOS boot box with FIPS disabled and encrypted disk

-

-

-

-

-

-

-

6-4

Build BIOS boot box with FIPS disabled and unencrypted disk

-

-

-

-

-

-

-

6-5

Build UEFI boot box with FIPS enabled and encrypted disk

-

-

-

-

-

-

-

6-6

Build UEFI boot box with FIPS enabled and unencrypted disk

-

-

-

-

-

-

-

6-7

Build UEFI boot box with FIPS disabled and encrypted disk

-

-

-

-

-

-

-

6-8

Build UEFI boot box with FIPS disabled and unencrypted disk

-

-

-

-

-

-

-

7

Verify installation from RPMs in tar file

Run simp-core’s install_from_tar test

simp-core

5

-

-

-

-

-

-

-

7-1

Execute test

See https://github.com/simp/simp-core/blob/master/spec/acceptance/suites/README.md for description of environment variables that can be set to point to the tar file

1

Y

Y

-

-

-

-

8

Create upgrade instructions

Document any steps that are needed outside of the generic upgrade instructions. 

simp-doc

4

10

Y

Y

-

-

-

-

8-1

Manually execute upgrade with FIPS enabled

4

-

-

-

-

-

-

8-2

Manually execute upgrade with FIPS disabled

1

-

-

-

-

-

-

8-3

Write upgrade instructions

4

-

-

-

-

-

-

8-4

Review instructions

Review the instructions for clarity, grammar, spelling, formatting, etc. Verification will be done in a separate ticket

1

-

-

-

-

-

-

9

Verify upgrade instructions

Verify upgrade instructions and make any necessary adjustments to them.

simp-doc

8

Y

Y

-

-

-

-

9-1

Execute instructions

2

-

-

-

-

-

-

9-2

Update instructions

1

-

-

-

-

-

-

13

PXE boot UEFI

Manually verify clients can PXE boot (UEFI) from a SIMP-managed tftpboot server

5

Y

Y

-

-

-

-

13-1

PXE boot (UEFI) FIPs enabled, disk encrypted, same OS as tftpboot server

-

-

-

-

-

-

13-2

PXE boot (UEFI) FIPs enabled, disk unencrypted, same OS as tftpboot server

-

-

-

-

-

-

13-3

PXE boot (UEFI) FIPs disabled, disk encrypted, same OS as tftpboot server

-

-

-

-

-

-

13-4

PXE boot (UEFI) FIPs disabled, disk unencrypted, same OS as tftpboot server

-

-

-

-

-

-

13-5

PXE boot (UEFI) FIPs enabled, disk encrypted, clients from tftpboot server of different OS

-

-

-

-

-

-

14

PXE boot BIOS

Manually verify clients can PXE boot (BIOS) from a SIMP-managed tftpboot server

Y

Y

-

-

-

-

14-1

PXE boot (BIOS) FIPs enabled, disk encrypted, same OS as tftpboot server

-

-

-

-

-

-

14-2

PXE boot (BIOS) FIPs enabled, disk unencrypted, same OS as tftpboot server

-

-

-

-

-

-

14-3

PXE boot (BIOS) FIPs disabled, disk encrypted, same OS as tftpboot server

-

-

-

-

-

-

14-4

PXE boot (BIOS) FIPs disabled, disk unencrypted, same OS as tftpboot server

-

-

-

-

-

-

14-5

PXE boot (BIOS) FIPs enabled, disk encrypted, clients from tftpboot server of different OS

-

-

-

-

-

-

15

Verify non-standard ISO UEFI boot options

Manually verify the choose your own partitions and minimum installation ISO boot options

5

Y

Y

-

-

-

-

15-1

Verify the choose your own partitions option

-

-

-

-

-

-

15-2

Verify the minimum installation option

-

-

-

-

-

-

16

Verify non-standard ISO BIOS boot options

Manually verify the ‘choose your own partitions' and ‘minimum installation’ ISO boot options

5

Y

Y

-

-

-

-

16-1

Verify the choose your own partitions option

-

-

-

-

-

-

16-2

Verify the minimum installation option

-

-

-

-

-

-

17

Dogfood released modules and RPMs

Use released modules in development environments that exercise as many of the modules as possible. Use RPMs of released assets on SIMP servers.

5

-

-

-

-

-

-

17-1

Deploy modules and RPMs to development environments

Deploy modules to as many OS variants as possible. Install RPMs to SIMP servers on as many OS variants as possible.

Y

Y

-

-

-

-

17-2

Examine logs for issues

Y

Y

-

-

-

-

Interim Release Candidate Tickets

The table below lists the typical tasks that are required for the interim release candidates (e.g., ‘Beta’, ‘RC1’). Fewer and fewer bugs are found with each interim release candidate. Accordingly, any manual integration testing should be targeted to verification of the bug fixes, whereas as many automated integration tests as resources allow should be executed! Be sure to examine the manual tests in this table and the initial release candidate tickets table and customize before generating tickets.

ID

Ticket Summary (< 100 chars)

Description

Component

Blockers

Story Points

EL7

EL8

OEL7

OEL8

RH7

RH8

Notes

1

Release Components

Identify components to be released, verify tests pass, push annotated tags, update SIMP release confluence page. https://simp.readthedocs.io/en/latest/contributors_guide/maintenance/Tagging_and_Releasing_Components.html

-

3

-

-

-

-

-

-

1-1

Identify components to release

-

1

-

-

-

-

-

-

1-2

Release components

1-1

2

-

-

-

-

-

-

4

Update simp-core with released components

Update simp-core files (Puppetfile.pinned, metadata.json, src/assets/simp/build/simp.spec) and verify ISOs can be built and the default and ipa acceptance test suites pass.

simp-core

1

3

-

-

-

-

-

-

4-1

Update files, build ISOs and execute tests

-

2

-

-

-

-

-

-

4-2

Review updates

4-1

1

-

-

-

-

-

-

5

Create test ISO, and publish the ISO and its tar file

simp-core

4

2

Y

Y

-

-

-

-

5-1

Build ISO

Use official RPM signing keys when the artifacts are to be published anywhere at simp-project.com.

-

1

-

-

-

-

-

-

5-2

Publish artifacts

Interim artifacts may be published to unstable folders at simp-project.com or other unofficial shared locations.

5-1

1

-

-

-

-

-

-

6

Validate ISO by building packer boxes

Use simp-packer to build SIMP server packer boxes. Basic bootstrap validation is done as part of the packer build.

5

Y

Y

-

-

-

-

6-1

Build BIOS boot box with FIPS enabled and encrypted disk

-

-

-

-

-

-

-

6-2

Build BIOS boot box with FIPS enabled and unencrypted disk

-

-

-

-

-

-

-

6-3

Build BIOS boot box with FIPS disabled and encrypted disk

-

-

-

-

-

-

-

6-4

Build BIOS boot box with FIPS disabled and unencrypted disk

-

-

-

-

-

-

-

6-5

Build UEFI boot box with FIPS enabled and encrypted disk

-

-

-

-

-

-

-

6-6

Build UEFI boot box with FIPS enabled and unencrypted disk

-

-

-

-

-

-

-

6-7

Build UEFI boot box with FIPS disabled and encrypted disk

-

-

-

-

-

-

-

6-8

Build UEFI boot box with FIPS disabled and unencrypted disk

-

-

-

-

-

-

-

7

Verify installation from RPMs in tar file

Run simp-core’s install_from_tar test

simp-core

5

-

-

-

-

-

-

-

This test can be omitted if there are no changes to component packaging since the previous candidate. Component behavior is adequately tested by the simp-core default suite with each simp-core check-in.

7-1

Execute test

See https://github.com/simp/simp-core/blob/master/spec/acceptance/suites/README.md for description of environment variables that can be set to point to the tar file

1

Y

Y

-

-

-

-

12

Update Changelog for bug fixes

simp-doc

1

3

-

-

-

-

-

-

12-1

Identify bug fixes and update changelog

-

2

-

-

-

-

-

-

12-2

Review changes

12-1

1

-

-

-

-

-

-

17

Dogfood released modules and RPMs

Use released modules in development environments that exercise as many of the modules as possible. Use RPMs of released assets on SIMP servers.

5

-

-

-

-

-

-

17-1

Deploy modules and RPMs to development environments

Deploy modules to as many OS variants as possible. Install RPMs to SIMP servers on as many OS variants as possible.

Y

Y

-

-

-

-

17-2

Examine logs for issues

Y

Y

-

-

-

-

Final Release Tickets

The table below lists the typical tasks that are required for the final release. By the time we get to this phase of the release process, there are few, if any bugs found, and very little manual testing is required.

ID

Ticket Summary (< 100 chars)

Description

Component

Blockers

Story Points

EL7

EL8

OEL7

OEL8

RH7

RH8

Notes

12

Update Changelog for bug fixes

simp-doc

1

3

-

-

-

-

-

-

12-1

Identify bug fixes and update changelog

-

2

-

-

-

-

-

-

12-2

Review changes

12-1

1

-

-

-

-

-

-

21

Review user documentation

General review of user documentation. Intent is to make sure the information is current and intelligible. Does not include ‘Changelog’, ‘Contributing to SIMP’, ‘Security Concept of Operations’, ‘Security Control Mapping’, ‘Vulnerability Supplement’ and ‘License’ sections.

simp-doc

-

-

-

-

-

-

This is required for each major release, nice-to-have for each minor release, and not necessary for each patch release.

Subtasks should correspond to major sections of the documentation, an will need to be updated accordingly when the docs structure changes.

21-1

Review ‘Quick Start’

Review and update as necessary

-

-

-

-

-

-

21-3

Review ‘Getting Started’

Review and update as necessary

-

-

-

-

-

-

21-4

Review ‘User Guide’

Review and update as necessary

-

-

-

-

-

-

21-5

Review ‘HOWTO’

Review and update as necessary

-

-

-

-

-

-

21-6

Review ‘FAQ’

Review and update as necessary

-

-

-

-

-

-

21-7

Review ‘Help’ and ‘Contact’ sections

Review and update as necessary

-

-

-

-

-

-

21-8

Review ‘Glossary of Terms’

Review and update as necessary

-

-

-

-

-

-

22

Review security documentation

General review of security-related documentation. Intent is to make sure the information is current and intelligible.

simp-doc

-

-

-

-

-

-

This is required for each major release, nice-to-have for each minor release, and not necessary for each patch release.

Subtasks should correspond to major sections of the documentation, an will need to be updated accordingly when the docs structure changes.

22-1

Review ‘Security Concept of Operations’

Review and update as necessary

-

-

-

-

-

-

22-2

Review ‘Security Control Mapping’

Review and update as necessary

-

-

-

-

-

-

22-3

Review ‘Vulnerability Supplement’

Review and update as necessary

-

-

-

-

-

-

23

Finalize simp-doc

23-1

Remove any reference to release candidates

23-2

Release

1

Release Components

Identify components to be released, including simp-doc, verify tests pass, push annotated tags, update SIMP release confluence page. https://simp.readthedocs.io/en/latest/contributors_guide/maintenance/Tagging_and_Releasing_Components.html

-

3

-

-

-

-

-

-

1-1

Identify components to release

-

1

-

-

-

-

-

-

1-2

Release components

1-1

2

-

-

-

-

-

-

4

Update simp-core with released components

Update simp-core files (Puppetfile.pinned, metadata.json, src/assets/simp/build/simp.spec) and verify ISOs can be built and the default and ipa acceptance test suites pass.

All components in the Puppetfile.pinned should be referencing a GitHub tag.

simp-core

1

3

-

-

-

-

-

-

4-1

Update files, build ISOs and execute tests

-

2

-

-

-

-

-

-

4-2

Review updates

Make sure there are no version discrepancies in the Puppetfile.pinned, metadata.json and simp.spec files.

4-1

1

-

-

-

-

-

-

18

Verify simp/simp_core meta module

Verify that the simp/simp_core meta module has appropriate dependencies and its dependencies have all been published to PuppetForge by running the install_from_core_module test suite.

4

18-1

Update the install_from_core_module test with any release-specific changes

18-2

Execute check for PuppetForge publication

Execute ‘bundle exec rake puppetfile:check’ and examine output to identify any modules that have not been published to PuppetForge

18-3

Publish any missing modules to PuppetForge

18-4

Execute the install_from_core_module test

17

Dogfood released modules and RPMs

Use released modules in development environments that exercise as many of the modules as possible. Use signed asset RPMs on SIMP servers.

5-1

17-1

Deploy modules and RPMs to development environments

Deploy modules to as many OS variants as possible. Install RPMs to SIMP servers on as many OS variants as possible.

17-2

Examine logs for issues

5

Create final ISO, verify with simp-packer, and publish the ISOs and their tar files

This is the final build and spot check via simp-packer.

simp-core

4

Y

Y

-

-

-

-

5-1

Build ISO

Use official RPM signing keys when the artifacts are to be published anywhere at simp-project.com.

-

1

-

-

-

-

-

-

5-2

Build BIOS boot box with FIPS enabled and encrypted disk

5-1

-

-

-

-

-

-

5-3

Build BIOS boot box with FIPS enabled and unencrypted disk

5-1

-

-

-

-

-

-

5-4

Build BIOS boot box with FIPS disabled and encrypted disk

5-1

-

-

-

-

-

-

5-5

Build BIOS boot box with FIPS disabled and unencrypted disk

5-1

-

-

-

-

-

-

5-6

Build UEFI boot box with FIPS enabled and encrypted disk

5-1

-

-

-

-

-

-

5-7

Build UEFI boot box with FIPS enabled and unencrypted disk

5-1

-

-

-

-

-

-

5-8

Build UEFI boot box with FIPS disabled and encrypted disk

5-1

-

-

-

-

-

-

5-9

Build UEFI boot box with FIPS disabled and unencrypted disk

5-1

-

-

-

-

-

-

5-10

Publish artifacts

Final artifacts that have passed simp-packer validation are published to official release folders at simp-project.com.

5-2, 5-3, 5-4, 5-5, 5-6, 5-7, 5-8, 5-9

1

-

-

-

-

-

-

-

-

-

-

-

-

Y

Y

-

-

-

-

Y

Y

-

-

-

-

Combined Table

The table below lists the standard tasks that must be done with each release.

  • The table assumes 3 candidate releases of SIMP (Alpha, RC1 and Final ), but should be adjusted based on the complexity of the release and bugs found during testing.

    • Only 2 candidate releases may be required for a SIMP patch release.

    • More than 3 candidate releases may be required for a SIMP major release.

    • Not all the tests may need to be repeated if the changes made don’t impact the test. For example, bug fixes to simp-extras modules won’t necessarily require full regression tests to be executed, when these modules are not exercised in the regression tests.

  • Stories or subtasks will be replicated based their release candidate and OS fields.

    • When Y appears in one of the release candidate or OS fields of a story, that story will be replicated for the permutation noted.

      • When just release candidate permutations or just OS permutations are noted, that story will be replicated for individual release candidates/OSs selected.

      • When both release candidate and OS permutations are listed, the story will be replicated for each OS for each release candidate.

    • When Y appears in one of the release candidate or OS fields of a subtask, that subtask will be replicated for the permutation noted within the story.

    • N, -, and a blank entry are all treated as ‘do not replicate’ indicators.


ID

Ticket Summary (< 100 chars)

Description

Component

Blockers

Story Points

Alpha (Y/N)

RC1 (Y/N)

Final (Y/N)

EL7 (Y/N)

EL8 (Y/N)

OEL7 (Y/N)

OEL8 (Y/N)

RH7 (Y/N)

RH8 (Y/N)

Notes

10

Create initial SIMP changelog


simp-doc

8

Y

N

N

-

-

-

-

-

-

10-1

Create changelog

6

-

-

-

-

-

-

-

-

-

10-2

Review changes

2

-

-

-

-

-

-

-

-

-

11

Identify release-specific tests

Based on the initial Changelog for the release, identify integration tests that must be done with a fully configured SIMP server and (if necessary) clients. These should be tests that are not adequately tested in component acceptance tests.

10

5

Y

N

N

-

-

-

-

-

-

11-1

Determine release-specific manual tests

4

Y

N

N

-

-

-

-

-

-

11-2

Create tickets for each

1

-

-

-

-

-

-

-

-

-

12

Update Changelog for bug fixes

simp-doc

3

N

Y

Y

-

-

-

-

-

-

12-1

Identify bug fixes and update changelog

2

-

-

-

-

-

-

-

-

-

12-2

Review changes

1

-

-

-

-

-

-

-

-

-

13

PXE boot UEFI

Manually verify clients can PXE boot (UEFI) from a SIMP-managed tftpboot server

Y

Y

Y

Y

Y

N

N

N

N

PXE boot UEFI testing is complicated because it cannot be done with VirtualBox. It has to be done with libvirt or real hardware. This needs to be documented in simp-doc (there is a ticket for this!)

These are split into numerous subtasks because they are not yet automated.

13-1

PXE boot (UEFI) FIPs enabled, disk encrypted, same OS as tftpboot server

-

-

-

-

-

-

-

-

-

13-2

PXE boot (UEFI) FIPs enabled, disk unencrypted, same OS as tftpboot server

-

-

-

-

-

-

-

-

-

13-3

PXE boot (UEFI) FIPs disabled, disk encrypted, same OS as tftpboot server

-

-

-

-

-

-

-

-

-

13-4

PXE boot (UEFI) FIPs disabled, disk unencrypted, same OS as tftpboot server

-

-

-

-

-

-

-

-

-

13-5

PXE boot (UEFI) FIPs enabled, disk encrypted, clients from tftpboot server of different OS

-

-

-

-

-

-

-

-

-

14

PXE boot BIOS

Manually verify clients can PXE boot (BIOS) from a SIMP-managed tftpboot server

Y

Y

Y

Y

Y

N

N

N

N

These are split into numerous subtasks because they are not yet automated.

14-1

PXE boot (BIOS) FIPs enabled, disk encrypted, same OS as tftpboot server

-

-

-

-

-

-

-

-

-

14-2

PXE boot (BIOS) FIPs enabled, disk unencrypted, same OS as tftpboot server

-

-

-

-

-

-

-

-

-

14-3

PXE boot (BIOS) FIPs disabled, disk encrypted, same OS as tftpboot server

-

-

-

-

-

-

-

-

-

14-4

PXE boot (BIOS) FIPs disabled, disk unencrypted, same OS as tftpboot server

-

-

-

-

-

-

-

-

-

14-5

PXE boot (BIOS) FIPs enabled, disk encrypted, clients from tftpboot server of different OS

-

-

-

-

-

-

-

-

-

15

Verify non-standard ISO UEFI boot options

Manually verify the choose your own partitions and minimum installation ISO boot options

Y

Y

Y

Y

Y

N

N

N

N

15-1

Verify the choose your own partitions option

-

-

-

-

-

-

-

-

-

15-2

Verify the minimum installation option

-

-

-

-

-

-

-

-

-

16

Verify non-standard ISO BIOS boot options

Manually verify the ‘choose your own partitions' and ‘minimum installation’ ISO boot options

Y

Y

Y

Y

Y

N

N

N

N

16-1

Verify the choose your own partitions option

-

-

-

-

-

-

-

-

-

16-2

Verify the minimum installation option

-

-

-

-

-

-

-

-

-

17

Dogfood released modules and RPMs

Use released modules in development environments that exercise as many of the modules as possible

Y

Y

Y

-

-

-

-

-

-

17-1

Deploy modules and RPMs to development environments

Deploy modules to as many OS variants as possible

-

-

-

-

-

-

-

-

-

17-2

Examine logs for issues

-

-

-

-

-

-

-

-

-

18

Verify simp/simp_core meta module

Verify that the simp/simp_core meta module has appropriate dependencies and its dependencies have all been published to PuppetForge by running the install_from_core_module test suite.

N

N

Y

-

-

-

-

-

-

18-1

Update the install_from_core_module test with any release-specific changes

-

-

-

-

-

-

-

-

-

18-2

Execute check for PuppetForge publication

Execute ‘bundle exec rake puppetfile:check’ and examine output to identify any modules that have not been published to PuppetForge

18-3

Publish any missing modules to PuppetForge

18-2

18-4

Execute the install_from_core_module test

18-3

19

Announce release

N

N

Y

20

Post-release: Verify RPM publication

Verify RPMs have been uploaded to simp-project.com. This includes Puppet module RPMs, SIMP application RPMs, and any other auxiliary, application RPMs.

N

N

Y

-

-

-

-

-

-

20-1

Verify Puppet module RPMs have been published

Execute ‘bundle exec rake puppetfile:check’ and examine output to identify any modules that have not been published to simp-project.com.

-

-

-

-

-

-

-

-

-

20-2

Verify other RPMs have been published

Run the install_from_rpm test. See https://github.com/simp/simp-core/blob/master/spec/acceptance/suites/README.md . If a nodeset for the OS under test does not exist, create it.


-

-

-

Y

Y

Y

Y

N

N

The simp-project.com repos have different structures for EL and OEL, so replicating this test for each OS is helpful.

20-3

Publish any missing RPMs

21

Review user documentation

General review of user documentation. Intent is to make sure the information is current and intelligible. Does not include ‘Changelog’, ‘Contributing to SIMP’, ‘Security Concept of Operations’, ‘Security Control Mapping’, ‘Vulnerability Supplement’ and ‘License’ sections.


This is required for each major release, nice-to-have for each minor release, and not necessary for each patch release.

Subtasks should correspond to major sections of the documentation, an will need to be updated accordingly when the docs structure changes.

21-1

Review ‘Quick Start’

Review and update as necessary

21-3

Review ‘Getting Started’

Review and update as necessary

21-4

Review ‘User Guide’

Review and update as necessary

21-5

Review ‘HOWTO’

Review and update as necessary

21-6

Review ‘FAQ’

Review and update as necessary

21-7

Review ‘Help’ and ‘Contact’ sections

Review and update as necessary

21-8

Review ‘Glossary of Terms’

Review and update as necessary

22

Review security documentation

General review of security-related documentation. Intent is to make sure the information is current and intelligible.

This is required for each major release, nice-to-have for each minor release, and not necessary for each patch release.

Subtasks should correspond to major sections of the documentation, an will need to be updated accordingly when the docs structure changes.

22-1

Review ‘Security Concept of Operations’

Review and update as necessary

22-2

Review ‘Security Control Mapping’

Review and update as necessary

22-3

Review ‘Vulnerability Supplement’

Review and update as necessary

23

Execute manual tests

Miscellaneous tests that are not addressed (fully) with automation.

23-1

Verify rsyslog local and forwarded logging in simp-core default suite

simp-core's default suite has an extensive rsyslog integration test for local logging and log forwarding. In lieu of testing with mock log events via logger (as is done in pupmod-simp-rsyslog and pupmod-simp-simp_rsyslog), this test stimulates events that generate log messages from actual applications (e.g., puppetserver error logs). Unfortunately, due to rsyslog itself, the rsyslog forwarding verifications have proven to be unreliable. As a stopgap measure, the tests were modified to skip any rsyslog test that fails in the simp-core default suite, in lieu of failing. Unfortunately, this has the potential to hide actual problems. So this ticket is to verify that all the checks executed in this test actually work.

23-2

Verify compliance report in simp-core default suite

Examine the compliance report generated by the simp-core default suite and verify there are no incorrect mappings or unexpected non-compliance. (There will be some non-compliance for overrides that allow the test to run.)

24

Verify poss scenario

Manually verify SIMP server and a client operate under the expected security measures when the SIMP server is bootstrapped with the ‘poss’ scenario.

Y

Y

-

-

-

-

Verify using a SIMP server and kickstart client with the same OS.

24-1

Bootstrap a SIMP server and verify all security measures are enforced.

24-2

Kick a client and verify no security measures are enforced

  • If the auditd service is running, it has no rules no rules.
    ** 'auditctl -l' returned 'No rules'

  • If the firewalld service is running, the default zone is not the 99_simp zone
    ** 'firewall-cmd --get-default-zone' returns 'public'

  • haveged service does not exist
    ** 'systemctl status haveged' returned 'Unit haveged.service could not be found.'

  • logrotate configuration, /etc/logrotate.conf, does not have 'include /etc/logrotate.simp.d'

  • pam configuration, /etc/pam.d/system-auth does not have "This file managed by Puppet"

  • SIMP-specific PKI directories, /etc/pki/simp/ and /etc/pki/simp_apps/, do not exist.

  • sssd service should not be running and should not be configured
    ** 'systemctl status sssd' returned 'Active: inactive (dead)
    ** No /etc/sssd/sssd.conf

  • stunnel service does not exist
    ** 'systemctl status stunnel' returned 'Unit stunnel.service could not be found.'

  • rsyslog service is running but is not configured for SIMP, i.e.
    /etc/rsyslog.conf does not have '$IncludeConfig /etc/rsyslog.simp.d/*.conf'

  • tcpwrappers is not configured on OSs that support tcpwrappers
    ** If /etc/hosts.allow exists, it is just comments
    ** If /etc/hosts.deny exists, it is just comments

  • No labels