Make Gitlab CI a required PR check for all GitHub SIMP repos

Description

As a replacement for our Travis CI checks, we shold make the GitLab CI checks mandatory.

This will be a bit of a culture shift:

  • An authorized user on the gitlab.com/simp will need to push a PR over to the gitlab mirror in order to kick off the job (for security reasons, the CI pipeline the runs acceptance tests should not be triggered at-will by PRs from external contributors).

  • Not all github.com/simp repos are presently mirrored on gitlab.com/simp

  • Not all github.com/simp repos have GitLab CI pipelines yet

Acceptance Criteria

None

Activity

Show:
Liz Nemsick
January 4, 2021, 1:40 PM

Some component acceptance tests fail on GitLab, even though they run successfully on faster servers. For example, the rubygem-simp-cli tests will randomly fail with ssh connection failures. This transition is going to be a problem for those components.

Chris Tessmer
January 4, 2021, 10:44 PM

🤔 Components with intermittently-failing acceptance tests seem like they already have problems. What do we do with them now?

Chris Tessmer
January 4, 2021, 10:51 PM

Depending on the answer, we can formalize our expectations/response in the components' pipeline config:

  • If we're comfortable that a GitLab CI test might fail, we can add `allow_failure: true`, gate it behind a conditional (like we do with `<<: *with_SIMP_ACCEPTANCE_MATRIX_LEVEL_3`), or just remove it.

  • If we want to know when a test fails but have the chance to okay it anyway, we could add a (blocking) manual step that only appears after the failure.

  • If it's absolutely required that the test always runs and always passes: we can add a `retry` condition (or just accept the failure and hit "retry").

Chris Tessmer
January 4, 2021, 10:54 PM

(Although considering how time- and resource-intensive acceptance tests can be, I'd be wary of the `retry` condition until something like gitlab-ce#27988 is implemented.)

Epic Link

Story Points

8

Components

Assignee

Chris Tessmer

Sprint

Sprint_128

Priority

High