Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Table of Contents
maxLevel3

Questions

Assuming at least some of the packages involved belong to repo modules, how can/should we handle:

pkglist + packages.yaml => building tarball/ISO

Can the pkg/tar/ISO build process still create a usable ISO as it currently does, or will something have to change?

  • filtering an OS ISO down to its pkglist

e

Note

Possible (but involved)

We can probably use dir2module to re-package just the modules we need and mergerepo_c to build them into a slimmed-down AppStream based on pkglist.txt (or a SIMP/ repo for packages.yml)

Details at the end of the RPM module header examples

Warning

Currently failing:

Currently, an AppStream repo mirrored with dnf reposync fails dnf repoclosure rather badly, while the real AppStream repo only has two unresolved deps (both ursine).

If it can be done: How?

...

Table of Contents
maxLevel3

Questions

Assuming at least some of packages we need to ship with SIMP already belong to repo modules, how can/should we handle:

pkglist.txt + packages.yaml => building ISO

Is it possible for the pkg/tar/ISO build process to still create a usable ISO?

Filtering an OS ISO down to its pkglist + adding external packages in packages.yaml

Note

70% Confidence it’s Possible — but will be complicated

We can use dir2module to re-package specific modular packages into slimmer versions of their own modules and then use mergerepo_c to collect them into a slimmed-down AppStream/ repo (based on pkglist.txt) or SIMP/ (based on packages.yml). However, there are several issues that make this complicated:

  • RPM packages from repo modules contain a ModularityLabel header that is unique to the module’s name + stream + version + context + architecture (NVSCA). DNF must install these RPMs from modules (with modular metadata)—and will refuse to install them as ursine packages.

  • dir2module requires a complete NVSCA string generating module metadata for a directory of RPMs.

  • At a minimum, a repackaged (slim) module’s name and stream must match its source repo’s module metadata.

    • This is required so the packages maintain continuity with the complete upstream repo (e.g., receiving update from the complete AppStream repo, epel-modular, etc)

    • The module’s versionnumber must evaluate to more than the versions of earlier modules and less than the version of later modules in the full source repository.

  • The ModularityLabel header is string unique to a module builds' NVSCfor a particular platform. The header data in RPMs packaged by RHEL/CentOS build system is currently in NVSC format, however this data is arbitrary and cannot be relied uponto provide accurate NVSC data for the module.

  • The only canonical source for a module’s correct NVSCA/P data is the source repo’s metadata (generally under repodata/{XXXXX}-modules.yaml.gz, and defined by repodata/repomd.xml under <data type="modules">`)

Remaining unknowns:

  • (question) (warning) It is not yet tested (but possible) that context and/or profile (/P) must match as well

  • (question) (warning) It is not yet tested (but possible) that a repacked “slim” repo may be still problematic for DNF when resolving its metadata/packages alongside a full repo with identical NVSCA details.

Testing ISO RPMs' completeness with a repoclosure

Tip

Recommendation:

Use dnf repoclosure (see below for arguments)

To run a repoclosure against ISO-bound/based repos that are staged on the local filesystem:

Code Block
dnf repoclosure 
  --disablerepo=\* \
  --repofrompath=base,/mnt/BaseOS \
  --repofrompath test1,/opt/dnf_reposync_from_iso \
  --arch x86_64 \
  --check test1 --check base

Notes:

  • Make sure there is a --repofrompath= for each local repository directory that should be considered during resolution.

  • The --arch x86_64

  • Using these arguments, the repoclosure does not require purpose-built *.repo files and does not need to run within mock or a chroot.

Impacts to unpack_dvd & kickstart repositories

...

Notes:

Supporting questions

  • Can we repackage a “thinned-out” module with only select packages?

    • probably:

      • EL8 RPM prereqs: createrepo_c (appstream) + modulemd-tools (epel)

      • Process:

        1. Create initial repo with ursine modules

          Code Block
          mkdir -p $NEW_REPO_DIR/Packages/ursine
          cp "${URSINE_PACKAGE_FILES[@]}" "$NEW_REPO_DIR/Packages/ursine/"
          cd "$NEW_REPO_DIR"
          createrepo_c .
        2. Create a modular repo from packages that already have a common module header

          Status
          colourYellow
          titleWARNING
          The approach of taking NSVCA from the RPM headers is incomplete and cannot be relied upon to be accurate or available—the ModularityLabel header can contain any String
          Status
          colourRed
          titleDO NOT PRODUCTIZE
          Use another means of obtaining this data; preferably from the repo itself (the data is sourced from the source repository’s {XXXXX}-modules.yaml.gz file)

          Code Block
          # Get the ModularityLabel from the RPMs
          ### WARNING: All RPMs in the module must have a SINGLE and IDENTICAL ModularityLabel
          find "$DIR_WITH_RPMS" -name \*.rpm  \
            -exec rpm -qp {} --qf '%{ModularityLabel}\n' \;  \
            | sort -u
          
          ### WARNING: EL and EPEL module headers are aurrent in NSVC format by 
          ####         CONVENTION; in reality the ModularityLabel cannot be relied 
          ####         upon for correct NSVC data!
          MODULE_HEADER=nodejs:10:8020020200707141642:6a468ee4
          
          

...

Info

Judging by the bug report at https://bugzilla.redhat.com/show_bug.cgi?id=1547041, dnf repoclosure is not module-aware, in the sense that its resolver does not consider packages in all available modules/streams.

The current behavior only resolves module packages using default or enabled streams & profiles.

Clean DNF install --downloadonly with all deps

...

Make sure you enable/disable the repos to match what you intend for resolution; --config is an option, too.

info
Code Block
languagebash
DNF_DLONLY_TARGET_PACKAGE=httpd
DNF_DLONLY_INSTALL_ROOT=/root/fake-install-dir
DNF_DLONLY_PACKAGE_DIR=/root/downloadonly

dnf install --downloadonly \
  --setopt=install_weak_deps=False \
  --installroot="$DNF_DLONLY_INSTALL_ROOT" \
  --downloaddir="$DNF_DLONLY_PACKAGE_DIR" \
  --disablerepo=\* \
  --enablerepo=appstreambaseos \
  --disablerepoenablerepo=testappstream \
  --releasever=8 \
  "$DNF_DLONLY_TARGET_PACKAGE"

Notes:

  • An an --enablerepo= for each repo to consider during resolution.

  • Pointing to an empty --installroot= will cause dnf install --downloadonly to download EVERY dependency, including the packages for the baseos.

  • In this case, --setopt=install_weak_deps=False may be useful to ignore weak RPM dependencies and cut down on the download size.

Documentation