Page Comparison

Problem

EL8 AppStream/modularity

Starting with EL8, some many RPMs are distributed within in Application Streams, a new distribution system that uses (based on DNF modularity.

...

). This is not optional—a basic EL8 installation

...

may require packages from both BaseOS and AppStream, and EPEL distributes some software in the new epel-modular repository.

...

To use Application Streams' modules, DNF requires repositories to

New Repository metadata

Repositories with modules must provide an additional layer of metadata modular (modulemd) . This metadata provides a repository-wide manifest of all module streams and the RPMs that belong to them.

i The SIMP ISO relies on

...

metadata.

• The modulemd manifest describes every module stream and their RPMs across the entire repository.

• Modular repositories can onlybe built with a modularity-aware repository tool, like createrepo_c.

• The old createrepo command cannotbuild modular repositories—it is unaware of (and will destroy) modular metadata. DNF will be unable to install modular RPMs in a repository built by createrepo.

• Modular repositories may include a mix of standard RPMs and modular RPMs.

New RPM & DNF behaviours

• DNF must be used to install modular RPMs; yum does not support modularity

• Each modular RPM is built with a special ModularityLabel tag.

• DNF will not install an RPM with a ModularityTag unless it can be associated with modulemd metadata from an available repository.

AppStream Lifecycle

• Each RHEL AppStream (module stream) has its own lifecycle, which is usually not the full 10-year lifecycle of the Base OS: https://access.redhat.com/support/policy/updates/rhel8-app-streams-life-cycle.

  Image Added

• This includes most default streams.

• Some streams retire within two years (the first RHEL stream retires in April 2021).

SIMP ISO Build

Requirements

1. The SIMP ISO MUST be self-contained and capable of installing and boot-strapping a fully-functional OS and SIMP server on an isolated (offline) network.

2. Packages MUST install correctly from the SIMP ISO or the local mirror created and repositories mirrored from the SIMP ISO

3. On systems with packages installed from the SIMP ISO/mirrors that are later configured to use those packages' original source repositories/mirrors:

   1. All ISO-installed distributed packages MUST upgrade correctly (including modular packages)

   2. All ISO-installed distributed package metadata MUST integrate seamlessly with the upstream (including modules and streams)

4. The ISO build process SHOULD continue to only include the packages it needs

...

Caveats

1. If necessary, the SIMP 6.6.0 ISO MAY mirror the entire base OS’s DVD AppStream/ repository if it would avoid delays to the SIMP 6.6.0 build process.

2. However, the ISO MUST NOT mirror entire third-party modular repositories (like epel-modular) just to provide a few necessary packages it requires from single streams (e.g., 389-directory-server:stable)

3. The SIMP project’s RPMs/tarball MUST NOT require modularity “SIMP RPMs” (packaged by the SIMP project, included in the tarball) will NOT be distributed in module streams for the SIMP 6.6.0 release (and probably never).

...

Constraints

1. SIMP 6.6.0 requires RPMs from at least one module stream from epel-modular: 389-directory-server:stable, in order to support 389ds.

...

DNF refuses to install any RPM with a ModularityLabel header unless it can be associated with modular metadata from an available repository.

...

Building/rebuilding repos with the old createrepo command will destroy a repo’s modular metadata and make any modular RPMs unable to install

Conclusions

...

Requirements for SIMP ISO build + EL8 modularity

The following (derived from the information above) should enable us to build SIMP ISOs that distribute modular RPMs:

1. The ISO build process MUST distribute modular RPMs in “slim” subsets of their original module streams.

The alternatives are to include the entire modular repository (too big) or not ship dependencies (ISO is no longer self-sufficient)

1. In order to maintain seamless interoperability, each “slim” modular stream subset MUST use the same name + stream + version + context + architecture (N:S:V:C:A) as their upstream sources

2. If feasible, the “slimming” process SHOULD be generalized enough to apply to packages from external sources like epel-modular AND the base OS’s DVD AppStream/ repository.

Alternatives considered were to include the entire modular repository (too big) or not ship modular dependencies (ISO + download server would no be longer self-contained).

2. The old createrepo command MUST NEVER be run on a modular

...

repository

This applies to both the ISO’s build process and post-installation local tooling on SIMP systems.

1. Status
   colour Purple title TODO

   The ISO’s modular repositories MUST be built with modularity-aware tools, like createrepo_c or creatrerepocreaterepo_mod from modulemd-tools.status

   colour	Purple
   title	TODO

2. Local SIMP tooling MUST be changed to NEVER use createrepo to manage on modular repositories:
   

In implementation terms,

• Status
  colour Purple title TODO

  At least two sections of the ISO’s ks auto.cfg in simp-core (EL8 only; not yet created)

• Status
  colour Purple title TODO

  UpdateOsYumRepositoriesAction#apply() in simp config

• Status
  colour Purple title TODO

  The update_yum_repo() method in unpack_dvd

...

1. .

3. Modular repos' modulemd metadata MUST remain intact when mirrored or rebuilt.

...

1. Modular repos MUST NOT be mirrored or rebuilt without it

2. The modulemd metadata will always be necessary in order to install these repos' modular

...

1. packages—and once lost it cannot be recreated from

...

1. their RPMs (unlike standard RPMs).

2. Status
   colour Purple title TODO

   Local SIMP tooling that mirrors modular repositories MUST preserve modulemd metadata (e.g., dnf reposync --download-metadata)

The “Slim” subset repos are of particular concern, since SIMP tooling is guaranteed to encounter them.