[WIP] SIMP 6.6.0 planning for EL8 modularity + ISO distribution

Problem

EL8 AppStream modularity

Starting with EL8, some RPMs are distributed within Application Streams, a new distribution system that uses DNF modularity.

• Application Streams & modularity are not optional—a basic EL8 installation requires packages from both BaseOS and AppStream.

To use Application Streams' modules, DNF requires repositories to provide an additional layer of metadata (modulemd). This metadata provides a repository-wide manifest of all module streams and the RPMs that belong to them.

i The SIMP ISO relies on

SIMP ISO Requirements

1. The SIMP ISO MUST be self-contained and capable of installing and boot-strapping a fully-functional OS and SIMP server on an isolated (offline) network.

2. Packages MUST install correctly from the SIMP ISO or the local mirror created from the SIMP ISO

3. On systems with packages installed from the SIMP ISO that are later configured to use those packages' original source repositories/mirrors:

   1. All ISO-installed packages MUST upgrade correctly

   2. All ISO-installed package metadata MUST integrate seamlessly (including modules and streams)

4. The ISO build process SHOULD continue to only include the packages it needs

Simp ISO 6.6.0 Caveats

1. If necessary, the SIMP 6.6.0 ISO MAY mirror the entire base OS’s DVD AppStream/ repository if it would avoid delays to the SIMP 6.6.0 build process.

2. However, the ISO MUST NOT mirror entire third-party modular repositories (like epel-modular) just to provide a few necessary packages from single streams (e.g., 389-directory-server:stable)

3. The SIMP project’s RPMs/tarball MUST NOT require modularity for the SIMP 6.6.0 release (and probably never).

6.6.0 Constraints

• SIMP 6.6.0 requires RPMs from at least one module stream from epel-modular: 389-directory-server:stable, in order to support 389ds.

• DNF refuses to install any RPM with a ModularityLabel header unless it can be associated with modular metadata from an available repository.

• Building/rebuilding repos with the old createrepo command will destroy a repo’s modular metadata and make any modular RPMs unable to install

Conclusions

These requirements (derived from the In order to build SIMP ISOs that distribute modular RPMs:

The ISO build process MUST distribute modular RPMs in “slim” subsets of their original module streams.

The alternatives are to include the entire modular repository (too big) or not ship dependencies (ISO is no longer self-sufficient)

1. In order to maintain seamless interoperability, each “slim” modular stream subset MUST use the same name + stream + version + context + architecture (N:S:V:C:A) as their upstream sources

2. If feasible, the “slimming” process SHOULD be generalized enough to apply to packages from external sources like epel-modular AND the base OS’s DVD AppStream/ repository.

The old createrepo command MUST NEVER be run on modular a repository

This applies to both the ISO’s build process and post-installation local tooling on SIMP systems.

1. TODO The ISO’s modular repositories MUST be built with modularity-aware tools, like createrepo_c or creatrerepo_mod from modulemd-tools.

2. TODO Local SIMP tooling MUST be changed to NEVER use createrepo to manage modular repositories:
   

In implementation terms,

• TODO At least two sections of the ISO’s ks auto.cfg in simp-core (EL8 only; not yet created)

• TODO UpdateOsYumRepositoriesAction#apply() in simp config

• TODO The update_yum_repo() method in unpack_dvd

“Slim” subset repos' modulemd metadata MUST remain intact when mirrored or rebuilt.

The repos MUST NOT be mirrored or rebuilt without it

The modulemd metadata will always be necessary in order to install these repos' modular packages, and it once lost it cannot be recreated from the RPMs.

1. TODO Local SIMP tooling that mirrors modular repositories MUST preserve modulemd metadata (e.g., dnf reposync --download-metadata )