SIMP 6.6.0 planning for EL8 modularity + ISO distribution

EL8 AppStream/modularity

Starting with EL8, many RPMs are distributed in Application Streams (based on DNF modularity). This is not optional—a basic EL8 installation may require packages from both BaseOS and AppStream, and EPEL distributes some software in the new epel-modular repository.

New Repository metadata

Repositories with modules must provide an additional layer of modular (modulemd) metadata.

• The modulemd manifest describes every module stream and their RPMs across the entire repository.

• Modular repositories can only be built with a modularity-aware repository tool, like createrepo_c.

• The old createrepo command cannot build modular repositories—it is unaware of (and will destroy) modular metadata. DNF will be unable to install modular RPMs in a repository built by createrepo.

• Modular repositories may include a mix of standard RPMs and modular RPMs.

New RPM & DNF behaviours

• DNF must be used to install modular RPMs; yum does not support modularity

• Each modular RPM is built with a special ModularityLabel tag.

• DNF will not install an RPM with a ModularityTag unless it can be associated with modulemd metadata from an available repository.

SIMP ISO Build

Requirements

1. The SIMP ISO MUST be self-contained and capable of installing and boot-strapping a fully-functional OS and SIMP server on an isolated (offline) network.

2. Packages MUST install correctly from the SIMP ISO and repositories mirrored from the SIMP ISO

3. On systems with packages installed from the SIMP ISO/mirrors that are later configured to use those packages' original source repositories/mirrors:

   1. All ISO-distributed packages MUST upgrade correctly (including modular packages)

   2. All ISO-distributed package metadata MUST integrate seamlessly with the upstream (including modules and streams)

4. The ISO build process SHOULD continue to only include the packages it needs

Caveats

1. If necessary, the SIMP 6.6.0 ISO MAY mirror the entire base OS’s DVD AppStream/ repository if it would avoid delays to the SIMP 6.6.0 build process.

2. However, the ISO MUST NOT mirror entire third-party modular repositories (like epel-modular) just to provide a few packages it requires from single streams (e.g., 389-directory-server:stable)

3. “SIMP RPMs” (packaged by the SIMP project, included in the tarball) will NOT be distributed in module streams for the SIMP 6.6.0 release (and probably never).

Constraints

1. SIMP 6.6.0 requires RPMs from at least one module stream from epel-modular: 389-directory-server:stable, in order to support 389ds.

Conclusions

These requirements (derived from the In order to build SIMP ISOs that distribute modular RPMs:

The ISO build process MUST distribute modular RPMs in “slim” subsets of their original module streams.

The alternatives are to include the entire modular repository (too big) or not ship dependencies (ISO is no longer self-sufficient)

1. In order to maintain seamless interoperability, each “slim” modular stream subset MUST use the same name + stream + version + context + architecture (N:S:V:C:A) as their upstream sources

2. If feasible, the “slimming” process SHOULD be generalized enough to apply to packages from external sources like epel-modular AND the base OS’s DVD AppStream/ repository.

The old createrepo command MUST NEVER be run on modular a repository

This applies to both the ISO’s build process and post-installation local tooling on SIMP systems.

1. TODO The ISO’s modular repositories MUST be built with modularity-aware tools, like createrepo_c or creatrerepo_mod from modulemd-tools.

2. TODO Local SIMP tooling MUST be changed to NEVER use createrepo to manage modular repositories:

Implementation hints

• Places we use createrepo:

  • TODO At least two sections of the ISO’s ks auto.cfg in simp-core (EL8 only; not yet created)

  • TODO UpdateOsYumRepositoriesAction#apply() in simp config

  • TODO The update_yum_repo() method in unpack_dvd

“Slim” subset repos' modulemd metadata MUST remain intact when mirrored or rebuilt.

1. The modular repos MUST NOT be mirrored or rebuilt without it

2. The modulemd metadata will always be necessary in order to install these repos' modular packages—and once lost it cannot be recreated from their RPMs (unlike standard RPMs).

3. TODO Local SIMP tooling that mirrors modular repositories MUST preserve modulemd metadata (e.g., dnf reposync --download-metadata)