As a replacement for our Travis CI checks, we shold make the GitLab CI checks mandatory.
This will be a bit of a culture shift:
An authorized user on the gitlab.com/simp will need to push a PR over to the gitlab mirror in order to kick off the job (for security reasons, the CI pipeline the runs acceptance tests should not be triggered at-will by PRs from external contributors).
Not all github.com/simp repos are presently mirrored on gitlab.com/simp
Not all github.com/simp repos have GitLab CI pipelines yet
Some component acceptance tests fail on GitLab, even though they run successfully on faster servers. For example, the rubygem-simp-cli tests will randomly fail with ssh connection failures. This transition is going to be a problem for those components.
🤔 Components with intermittently-failing acceptance tests seem like they already have problems. What do we do with them now?
Depending on the answer, we can formalize our expectations/response in the components' pipeline config:
If we're comfortable that a GitLab CI test might fail, we can add `allow_failure: true`, gate it behind a conditional (like we do with `<<: *with_SIMP_ACCEPTANCE_MATRIX_LEVEL_3`), or just remove it.
If we want to know when a test fails but have the chance to okay it anyway, we could add a (blocking) manual step that only appears after the failure.
Following today's discussion with , here are all the current simp GitLab CI pipelines that contain jobs with allow_failure: true, linked to their present commit refs for historical preservation: