DNF modularity notes

Questions

Assuming at least some of packages we need to ship with SIMP already belong to repo modules, how can/should we handle:

pkglist.txt + packages.yaml => building ISO

Is it possible for the pkg/tar/ISO build process to still create a usable ISO?

Filtering an OS ISO down to its pkglist + adding external packages in packages.yaml

70% Confidence it’s Possible — but will be complicated

We can use dir2module to re-package specific modular packages into slimmer versions of their own modules and then use mergerepo_c to collect them into a slimmed-down AppStream/ repo (based on pkglist.txt) or SIMP/ (based on packages.yml). However, there are several issues that make this complicated:

RPM packages from repo modules contain a ModularityLabel header that is unique to the module’s name + stream + version + context + architecture (NVSCA). DNF must install these RPMs from modules (with modular metadata)—and will refuse to install them as ursine packages.
dir2module requires a complete NVSCA string generating module metadata for a directory of RPMs.
At a minimum, a repackaged (slim) module’s name and stream must match its source repo’s module metadata.
- This is required so the packages maintain continuity with the complete upstream repo (e.g., receiving update from the complete AppStream repo, epel-modular, etc)
- The module’s version number must evaluate to more than the versions of earlier modules and less than the version of later modules in the full source repository.
The ModularityLabel header is string unique to a module builds' NVSC for a particular platform. The header data in RPMs packaged by RHEL/CentOS build system is currently in NVSC format, however this data is arbitrary and cannot be relied upon to provide accurate NVSC data for the module.
The only canonical source for a module’s correct NVSCA/P data is the source repo’s metadata (generally under repodata/{XXXXX}-modules.yaml.gz, and defined by repodata/repomd.xml under <data type="modules">`)

Remaining unknowns:

It is not yet tested (but possible) that context and/or profile (/P) must match as well
It is not yet tested (but possible) that a repacked “slim” repo may be still problematic for DNF when resolving its metadata/packages alongside a full repo with identical NVSCA details.

Testing ISO RPMs' completeness with a repoclosure

Recommendation:

Use dnf repoclosure (see below for arguments)

To run a repoclosure against ISO-bound/based repos that are staged on the local filesystem:

dnf repoclosure 
  --disablerepo=\* \
  --repofrompath=base,/mnt/BaseOS \
  --repofrompath test1,/opt/dnf_reposync_from_iso \
  --arch x86_64 \
  --check test1 --check base

Notes:

Make sure there is a --repofrompath= for each local repository directory that should be considered during resolution.
The --arch x86_64
Using these arguments, the repoclosure does not require purpose-built *.repo files and does not need to run within mock or a chroot.

Impacts to unpack_dvd & kickstart repositories

How should unpack_dvd handle repos that do/may contain modules?

Recommendation:
Assuming we can limit any module editing to the tarball/ISO build process (above), unpack_dvd should just mount the ISO as a loopback device and sync everything with dnf reposync --download-metadata … .

How does SELinux support this?

Where do the policies for the packages in various module streams come from?

Notes:

So far, the selinux-*policy RPMs have handled whatever I’ve tried
Ancient policy wiki at https://fedoraproject.org/wiki/SELinuxModularityDesign (probably outdated)

Supporting questions

Can we repackage a “thinned-out” module with only select packages?
- probably:
  - EL8 RPM prereqs: createrepo_c (appstream) + modulemd-tools (epel)
  - Process:
    1. Create initial repo with ursine modules
      mkdir -p $NEW_REPO_DIR/Packages/ursine cp "${URSINE_PACKAGE_FILES[@]}" "$NEW_REPO_DIR/Packages/ursine/" cd "$NEW_REPO_DIR" createrepo_c .
    2. Create a modular repo from packages that already have a common module header
      WARNING The approach of taking NSVCA from the RPM headers is incomplete and cannot be relied upon to be accurate or available—the ModularityLabel header can contain any String
      DO NOT PRODUCTIZE Use another means of obtaining this data; preferably from the repo itself (the data is sourced from the source repository’s {XXXXX}-modules.yaml.gz file)
      # Get the ModularityLabel from the RPMs ### WARNING: All RPMs in the module must have a SINGLE and IDENTICAL ModularityLabel find "$DIR_WITH_RPMS" -name \*.rpm \ -exec rpm -qp {} --qf '%{ModularityLabel}\n' \; \ | sort -u ### WARNING: EL and EPEL module headers are aurrent in NSVC format by #### CONVENTION; in reality the ModularityLabel cannot be relied #### upon for correct NSVC data! MODULE_HEADER=nodejs:10:8020020200707141642:6a468ee4

Modularity CLI examples

Simple dnf module commands

View all modules

dnf module list --all

View a module’s available streams

# dnf module list --available nodejs
Waiting for process with pid 79896 to finish.
Last metadata expiration check: 0:00:01 ago on Fri 12 Mar 2021 02:03:02 AM UTC.
CentOS Linux 8 - AppStream
Name                   Stream                 Profiles                                             Summary
nodejs                 10 [d]                 common [d], development, minimal, s2i                Javascript runtime
nodejs                 12                     common [d], development, minimal, s2i                Javascript runtime
nodejs                 14                     common [d], development, minimal, s2i                Javascript runtime

Extra Packages for Enterprise Linux Modular 8 - x86_64
Name                   Stream                 Profiles                                             Summary
nodejs                 13                     default, development, minimal                        Javascript runtime

Hint: [d]efault, [e]nabled, [x]disabled, [i]nstalled

View packages in a module

dnf module repoquery nodejs

It looks dnf module repoquery only reports on the default stream, regardless of the module-spec specified (tested on EL8.3)

View modules/profiles that provide a package

dnf module provides nodejs

This is one of the few (two?) module commands that reports which DNF Repo hosts the module

View the packages in each profile (for a specific stream)

]# dnf module info --profile  nodejs:12
Last metadata expiration check: 2:59:10 ago on Thu 11 Mar 2021 11:02:32 PM UTC.
Name        : nodejs:12:8030020210304194546:30b713e6:x86_64
common      : nodejs
            : npm
development : nodejs
            : nodejs-devel
            : npm
minimal     : nodejs
s2i         : nodejs
            : nodejs-nodemon
            : npm

Find all modular RPM files that under a directory and print their module headers

This could identify dnf modules that would need to be re-created when shipping modular RPMs independently from their base repos:

find "$DIR_WITH_RPMS" -name \*.rpm \
  -exec rpm -qp {} --qf '%{NVRA}   %{ModularityLabel}\n' \; \
  | grep -v '(none)' \
  | tee modular_rpms.txt

The %{ModularityLabel} macro is returned in (module)name:stream:version:context:arch (NSVCA) format , so the result looks like this:

389-ds-base-1.4.3.8-5.module_el8.3.0+473+53682548.x86_64   389-ds:1.4:8030020200831174107:618f7055
389-ds-base-devel-1.4.3.8-5.module_el8.3.0+473+53682548.x86_64   389-ds:1.4:8030020200831174107:618f7055
389-ds-base-legacy-tools-1.4.3.8-5.module_el8.3.0+473+53682548.x86_64   389-ds:1.4:8030020200831174107:618f7055
389-ds-base-libs-1.4.3.8-5.module_el8.3.0+473+53682548.x86_64   389-ds:1.4:8030020200831174107:618f7055
389-ds-base-snmp-1.4.3.8-5.module_el8.3.0+473+53682548.x86_64   389-ds:1.4:8030020200831174107:618f7055
HdrHistogram-2.1.11-2.module_el8.2.0+460+6583c1d0.noarch   jmc:rhel8:8020020200731165725:21dc74c6
HdrHistogram-javadoc-2.1.11-2.module_el8.2.0+460+6583c1d0.noarch   jmc:rhel8:8020020200731165725:21dc74c6
Judy-1.0.5-18.module_el8.1.0+217+4d875839.x86_64   mariadb:10.3:8010020191115015915:cdc1202b
ant-1.10.5-1.module_el8.0.0+47+197dca37.noarch   ant:1.10:8000020190624202340:f7e686af
ant-lib-1.10.5-1.module_el8.0.0+47+197dca37.noarch   ant:1.10:8000020190624202340:f7e686af
aopalliance-1.0-17.module_el8.0.0+39+6a9b6e22.noarch   maven:3.5:8000020190624140656:f7e686af
aopalliance-1.0-20.module_el8.3.0+568+0c23fd64.noarch   maven:3.6:8030020201104064112:a623df05

It may be possible to re-create specific module/streams based on the RPM’s unique headers. However, there will need to be a second data source, because the RPM ModularityLabel has significant limitations:

By convention, RPMs' ModularityLabel string is in NVSC format, which doesn’t include profile information. Any module streams constructed solely on the RPM header would lose all the original streams' profiles.

Note that the repo2module tool just creates a default profile called everything.

We can’t rely on RPMs' ModularityLabel string to be in NVSC format, either!

“The ModularityLabel can be any string at all. In Fedora, we have a convention to use name:stream:version:context to indicate from which build the RPM originally came from, but this is not to be relied upon. It may change at any time and it also may not be accurately reflective of the module in which it currently resides, due to component-reuse in the Module Build System.”

— https://sgallagh.wordpress.com/2019/08/14/sausage-factory-modules-fake-it-till-you-make-it/

Identify all unique modules/streams from a collection of RPM files

find "$DIR_WITH_RPMS" -name \*.rpm \
  -exec rpm -qp {} --qf '%{ModularityLabel}\n' \; \
  | grep -v '^(none)' | sort | uniq -c \
  | sort -nk1,1 \
  | tee unique_rpm_module_streams.txt

Mirror the contents of a DNF repository, preserving all modules and package groups.

The current state of CentOS 8 createrepo_c + modulemd-tools (EPEL8) allow us to:

(Works from CentOS 8.3 and CentOS 7.8, requires packages dnf and dnf-plugin-core )

An example of this mirroring a mounted CentOS 8.3 ISO’s AppStream repository:

PATH_TO_LOCAL_MIRROR=/path/to/Appstream
PATH_TO_SOURCE_REPO=/mnt/AppStream

dnf reposync \
  --download-metadata --downloadcomps \
  --download-path "$PATH_TO_LOCAL_MIRROR" \
  --repofrompath iso,"$PATH_TO_SOURCE_REPO" \
  --repoid iso

# Useful EL8-only options: --remote-time --norepopath

dnf reposync should probably be the only kind of modular-capable mirroring that on-site tools like unpack_dvd should use.

DNF repoclosure

The ISO build process (really the tar build process) runs repoclosure to make sure the packages on the ISO will be self-contained.

dnf repoclosure --repofrompath iso,"$PWD" --repo appstream --repo baseos

Judging by the bug report at https://bugzilla.redhat.com/show_bug.cgi?id=1547041, dnf repoclosure is not module-aware, in the sense that its resolver does not consider packages in all available modules/streams.

The current behavior only resolves module packages using default or enabled streams & profiles.

Clean DNF install --downloadonly with all deps

Make sure you enable/disable the repos to match what you intend for resolution; --config is an option, too.

DNF_DLONLY_TARGET_PACKAGE=httpd
DNF_DLONLY_INSTALL_ROOT=/root/fake-install-dir
DNF_DLONLY_PACKAGE_DIR=/root/downloadonly

dnf install --downloadonly \
  --setopt=install_weak_deps=False \
  --installroot="$DNF_DLONLY_INSTALL_ROOT" \
  --downloaddir="$DNF_DLONLY_PACKAGE_DIR" \
  --disablerepo=\* \
  --enablerepo=baseos \
  --enablerepo=appstream \
  --releasever=8 \
  "$DNF_DLONLY_TARGET_PACKAGE"

Notes:

An an --enablerepo= for each repo to consider during resolution.
Pointing to an empty --installroot= will cause dnf install --downloadonly to download EVERY dependency, including the packages for the baseos.
In this case, --setopt=install_weak_deps=False may be useful to ignore weak RPM dependencies and cut down on the download size.

Documentation

https://docs.fedoraproject.org/en-US/modularity/
https://dnf.readthedocs.io/en/latest/
https://dnf-plugins-core.readthedocs.io/en/latest/
https://sgallagh.wordpress.com/2019/08/14/sausage-factory-modules-fake-it-till-you-make-it/
- Notes on building modules independently of Fedora’s hosted build system
https://pagure.io/modularity/issue/141
- > * AGREED: Profiles cannot be removed during the lifetime of the stream in order to avoid breaking scripts. Options may exist for exceptional cases on an individual basis. (contyk, 15:41:06)