simp-rake-helpers build flow + planning diagrams (6.6.0)

These high-level flow diagrams map out the current (pre-6.6.0) logic of simp-rake-helpers ISO builds, with annotations relevant to the two major build updates for 6.6.0: “slim” mirrors of modular DNF repos and JMG’s restructure to support multiple OSes.

The diagrams.net source for the diagrams is embedded in each PNG file.

1 build:auto
2 iso:build
3 build:yum:sync
4 pkg:checksig
5 pkg:repoclosure
- 5.1 Alternatives to --lookaside
- 5.2 (none yet) Alternatives to --tempcache

build:auto

The build:auto task is the starting point for development and release builds.

iso:build

The iso:build task is usually executed near the end of build:auto. It can also be run on its own.

build:yum:sync

The build:yum:sync task is executed by build:auto, prior to iso:build. It can also be run on its own.

pkg:checksig

The pkg:checksig task is executed by iso:build, just prior to pkg:repoclosure. It is not available as a public rake task.

Note (because it’s not obvious): I thought pkg:checksig's args might need an update to take multiple paths in rpm_dir (in order to accommodate JMG’s multiple OS/external repos work). However, the task internally glob-expands rpm_dir (into rpm_dirs), so it’s already possible to pass in multiple directories—although the string might end up looking extra weird.

Two drawbacks are worth mentioning:

If the paths are very different and don’t wildcard well, it will be necessary to use literal brace expansion (ex: {p,q}) instead.
But since it’s an argument for a Rake task, the commas in a brace expansion won’t work from the command line without some additional quote/escape ugliness.

From the command line (needs the quotes and escaped comma):

pkg:checksig['/path/to/{dir1/sub1\,dir2/sub2}']

rake-helpers style Ruby code just needs quotes:

Rake::Task["pkg:checksig"].invoke('/path/to/{dir1/sub1,dir2/sub2}')

If we split out the repos into the different directories JMG’s been working on, the brace expansion might look something like this:
Rake::Task["pkg:checksig"].invoke('/path/to/rpm_dirs/{SIMP,puppet,postgresql96,epel8}','/path/to/key_dir')

pkg:repoclosure

The pkg:repoclosure task is executed by iso:build, just after pkg:checksig.

EL8 repoclosure CLI is missing args from EL7 repoclosure!

EL7	EL8	Notes

EL7	EL8	Notes
-c C, --config=CONFIG	-c C, --config=CONFIG	Undocumented in EL8
-n --newest	--newest
-t (--tempcache)
-r R, --repoid=REPOID	--repo <repoid>	(Stalled) WIP repoclosure enhancements by m-blaha · Pull Request #255 · rpm-software-management/dnf-plugins-core
-l L, --lookaside=LOOKASIDE

Alternatives to --lookaside

CentOS repocloses their own builds without -- lookaside= by combining --repofrompath=REPO_ID,REPO_PATH and --repo=REPO_ID :

dnf repoclosure \
  --arch=x86_64 --arch=i686 --arch=noarch \
  --repofrompath=target-repo,file:///repos/compose/os \
  --repo=target-repo \
  --check=target-repo \
  --repofrompath=lookaside-repo-0,file:///repos/lookaside/BaseOS \
  --repo=lookaside-repo-0 \
  --repofrompath=lookaside-repo-1,file://repos/lookaside/AppStream \
  --repo=lookaside-repo-1

This is much better: Both options are native to core DNF. They set up the lookaside repos just for the run of this particular repoclosure—there’s no longer any need to stage a separate <repo>.conf or yum.conf > [main]\ninclude=<repo>.conf ahead of time.

EL7's version of repoclosure should be able to do this, too—it actually provides its own --repofrompath= The only difference is that EL7 uses --repoid= and EL8 is documented with --repo=—but EL8 actually still honors --repoid=, too.