Overview

This document provides an overview of the current SIMP RELENG ecosystem as of 10/28/2021 as it supports the upcoming 6.6.0 release.

RELENG Ecosystem

SIMP CE code is managed by repositories under the GitHub SIMP organization. Contributors fork repositories and contribute changes using Pull Requests (PRs). Each contribution’s PR encapsulates its own reviews, CI/CD workflows, and any discussion or patches until it is approved and merged into the upstream repository.

GitHub Actions

GitHub Actions (GHA) automates the CI/CD for most repositories that are included in/support the SIMP super-release. This generally includes:

• On PRs and merges:

  • Running automated checks, linting, and tests

• On PRs:

  • Triggering GLCI pipelines

  • (merging will auto-trigger GLCI as the new commits are mirrored)

• On annotated Tags:

  • Creating a GitHub release

  • Building/publishing packages (e.g., Puppet modules, Ruby Gems) to upstream repositories

  • Building/signing/attaching RPMs, either a.) as a follow-on to a tag-driven GitHub Release, OR b.) from a manual GHA trigger.

    • RPMs are attached to the relevant GitHub release as downloadable assets

    • See puppetsync/modules/profile/files/_github/workflows/release_rpms.yml at main · simp/puppetsync

Each repository’s GHA workflow configurations are kept under .github/workflows/.

• Common workflows are managed by Puppetsync as standardized assets and should not be edited directly. Puppetsync-managed workflows identify themselves in their header comments.

• Some repositories have their own unique workflows. These files are maintained directly and are not managed by Puppetsync. Puppetsync-managed workflows identify themselves in their header comments.

SIMP Puppet modules

GHA workflows are identical across all SIMP Puppet modules, managed by Puppetsync (as type pupmod).

SIMP RubyGems

Most GHA workflows for RubyGems are managed by Puppetsync (as type rubygem)

• RubyGem workflows may have several variations, particularly the tag-to-release workflows: depending on the gems, it may need to publish to rubygems.org, build an RPM, or just release to GitHub.

Unique workflows are managed per-repository.

Other SIMP assets

Other SIMP assets (e.g., simp-utils, pkg-r10k) may share common GHA workflows managed by Puppetsync (as type simp-unknown) and/or unique workflows that are managed per-repository.

Notable edge cases

Using GHA workflows to build RPMs for non-SIMP forked repo mirrors

Forked repo mirrors don’t use SIMP GHA workflows, so new tags to don’t trigger relases and RPM builds.

However, the release_rpms.yml workflow (a standardized asset in every repo that builds its own RPMs) is designed to work as a manual action that can release/build RPMs for any other SIMP repository (when provided a token with sufficient scope).

simp-core

TODO GitLab CI

TODO SIMP RELENG Tools

TODO Rake tasks

TODO Puppetsync

TODO bolt-pulp3

TODO releng-misc Bolt project

releng::download_assets_from_puppetfile

TODO Lifecycles

el8 modularity

TODO RPM lifecycle

GitHub (UNSTABLE-2) → UNSTABLE → STABLE

TODO RPMs, GPG keys, and Yum repositories

TODO Building and Signing RPMs

Updating the Download Server Repositories

Updating experimental repositories with a SIMP release’s GitHub RPMs

Running the following Bolt plan (from releng-misc) will download RPMs from GitHub releases and stage them local yum repositories:

The plan:

1. Determines what to download by referencing Puppetfile.pinned in simp-core’s master branch:

   • A :tag 's corresponding GitHub release

   • The HEAD-most tagged GitHub release along a repo’s tracking :branch

     • NOTE: The plan looks a maximum of 30 commits back to find the HEAD-most release

2. Downloads all assets from each repo’s GitHub release into a _release_assets/ directory

3. Creates four subdirectories containing yum repositories and hard-linked RPMs:

   • _release_assets/el8/

   • _release_assets/el7/

   • _release_assets/el8.src/

   • _release_assets/el7.src/

These directories can be transferred to the SIMP download server and used during development.

Uploading SIMP RPMs to the download server

Publishing to the download server is currently manual process.

Use rsync -n to check what will change before uploading RPMs:

After confirming the changes, run the command again without the -n

Updating a super-release’s dep repositories with bolt-pulp3

Currently only the repo_packages.yaml file for EL8 is maintained. It has recently been moved to simp-core. Read the project’s README and`bolt plan show` for instructions.

Manually triggering GHA RPM builds

The release_rpms.yml workflow will release/build/sign/attach RPMs for a component’s GitHub release. It is automatically triggered by its repo's tag_deploy*.yml workflow, following its GitHub release. The workflow file is a standardized asset, and is present in every SIMP component repo that builds its own RPMs.

RPMs are built by running the rake task pkg:single inside the SIMP build container

• This is done to ensure that RPMs are built using the overrides in simp-core’s build/rpm/dependencies.yaml file.

RPMs are signed within a centos8 SIMP build container, using a GPG key that is only available to workflows running under the simp/ GitHub org.

Manually triggering GHA RPM builds

The release_rpms.yml workflow can be triggered manually from the GitHub web interface and API. This can be necessary when the target is a forked repo mirror without its own tag_deploy_*.yml workflows, or an error prevented GHA from building RPMs during a component’s tagged release.

The workflow has many configurable options when triggered manually. Notably, it can be configured to release, build, and attach signed RPMs to tagged commites in any other repository under the simp/ org.

The manual trigger has the following requirements:

• A GitHub token with sufficient scope

• A target GitHub release, or at the very least, a tag commit (the workflow can optionally create a release from a pre-existing taga)

• The repo must be able to build RPMs from simp-core, using rake pkg:single

To manually trigger an RPM build:

1. From a SIMP component repo’s GitHub actions UI, select the RELENG: Build + attach RPMs to GitHub Release action

2. Click the “Run workflow” drop-down in the upper right

3. Fill out the relevant inputs, including an API token (and optionally, a different target repo)

4. Click “Run workflow”

Open

Open

Open

TODO Notable exceptions