simp-rake-helpers build flow + planning diagrams (6.6.0)

Owned by Chris Tessmer
Last updated: Apr 02, 2021
3 min read

These high-level flow diagrams map out the current (pre-6.6.0) logic of simp-rake-helpers ISO builds, with annotations relevant to the two major build updates for 6.6.0: “slim” mirrors of modular DNF repos and JMG’s restructure to support multiple OSes.

The diagrams.net source for the diagrams is embedded in each PNG file.

 

build:auto

The build:auto task is the starting point for development and release builds.

iso:build

The iso:build task is usually executed near the end of build:auto. It can also be run on its own.

 

build:yum:sync

The build:yum:sync task is executed by build:auto, prior to iso:build. It can also be run on its own.

 

pkg:checksig

The pkg:checksig task is executed by iso:build, just prior to pkg:repoclosure. It is not available as a public rake task.

Note (because it’s not obvious): I thought pkg:checksig's args might need an update to take multiple paths in rpm_dir (in order to accommodate JMG’s multiple OS/external repos work). However, the task internally glob-expands rpm_dir (into rpm_dirs), so it’s already possible to pass in multiple directories—although the string might end up looking extra weird.

Two drawbacks are worth mentioning:

  1. If the paths are very different and don’t wildcard well, it will be necessary to use literal brace expansion (ex: {p,q}) instead.

  2. But since it’s an argument for a Rake task, the commas in a brace expansion won’t work from the command line without some additional quote/escape ugliness.

From the command line (needs the quotes and escaped comma):

pkg:checksig['/path/to/{dir1/sub1\,dir2/sub2}']

rake-helpers style Ruby code just needs quotes:

Rake::Task["pkg:checksig"].invoke('/path/to/{dir1/sub1,dir2/sub2}')

If we split out the repos into the different directories JMG’s been working on, the brace expansion might look something like this:
Rake::Task["pkg:checksig"].invoke('/path/to/rpm_dirs/{SIMP,puppet,postgresql96,epel8}','/path/to/key_dir')

 

pkg:repoclosure

The pkg:repoclosure task is executed by iso:build, just after pkg:checksig.

EL8 repoclosure CLI is missing args from EL7 repoclosure!

EL8 repoclosure CLI is missing args from EL7 repoclosure!

EL7

EL8

Notes

EL7

EL8

Notes

-c C, --config=CONFIG

-c C, --config=CONFIG

Undocumented in EL8

-n --newest

--newest

 

-t (--tempcache)

 

-r R, --repoid=REPOID

--repo <repoid>

(Stalled) WIP repoclosure enhancements by m-blaha · Pull Request #255 · rpm-software-management/dnf-plugins-core

-l L, --lookaside=LOOKASIDE

 

 

Alternatives to --lookaside

CentOS repocloses their own builds without -- lookaside= by combining --repofrompath=REPO_ID,REPO_PATH and --repo=REPO_ID :

dnf repoclosure \ --arch=x86_64 --arch=i686 --arch=noarch \ --repofrompath=target-repo,file:///repos/compose/os \ --repo=target-repo \ --check=target-repo \ --repofrompath=lookaside-repo-0,file:///repos/lookaside/BaseOS \ --repo=lookaside-repo-0 \ --repofrompath=lookaside-repo-1,file://repos/lookaside/AppStream \ --repo=lookaside-repo-1

This is much better: Both options are native to core DNF. They set up the lookaside repos just for the run of this particular repoclosure—there’s no longer any need to stage a separate <repo>.conf or yum.conf > [main]\ninclude=<repo>.conf ahead of time.

EL7's version of repoclosure should be able to do this, too—it actually provides its own --repofrompath= The only difference is that EL7 uses --repoid= and EL8 is documented with --repo=—but EL8 actually still honors --repoid=, too.

EL7 repoclosure still won’t work with this syntax

EL7 repoclosure understands --repofrompath=, but not --check , so this alternative won’t work in EL7 unless I can find something else.

Or we just need to build a different check string between EL7 or EL8

 

(none yet) Alternatives to --tempcache