Versions Compared

Old Version 19

changes.mady.by.user Chris Tessmer

Saved on

compared with

New Version 20

changes.mady.by.user Judith Johnson

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents
maxLevel3

...

Note

80% Confidence it’s Possible — but it will be complicated and hacky

  • The current methods of downloading external packages AND pruning OS ISOs down to a minimal set of packages will both result in broken modular repositories.

    • This could be solved by recreating the RPM’s module streams with exactly the same NVSCANSVCA values as the source repository—but practically every step of that will be hacky and require unsupported tooling or several workarounds.

  • The repo manipulation methods (createrepo, repoclosure) aren’t module-aware

...

  • RPM packages from repo modules contain a ModularityLabel header that is unique to the module’s name + stream + version + context + architecture (NVSCANSVCA). DNF must install these RPMs from modules (with modular metadata)—and will refuse to install them as ursine packages.

    • dir2module and repo2modules require a complete NVSCA NSVCA string generating module metadata for a directory of RPMs.

    • 5️⃣ The dir2module script provided by the EPEL8 RPM modulemd-tools-0.7-1.el8.noarch does not create a default (or any) profile for the module it creates; it is effectively useless; use repo2module instead (which is still missing the Arch in NVCSNSVCA).

  • A repackaged (slim) module’s name and stream and context and architecture must match the values in the source repo’s metadata for the original module.

    • This is required so the packages maintain continuity with the complete upstream repo (e.g., receiving update from the complete AppStream repo, epel-modular, etc)

    • The versionnumber must evaluate to more than the earlier (module versions) and less than the version of later modules in the full source repository. This number (like stream and context) is an arbitrary string set by the build platform, so we have to get it

    • For practical purposes, you need to mirror the repo’s metadata YAML at the same time as you retrieve the packages—it might be updated later, even if the packages you see hosted there are the same

  • The ModularityLabel header is string unique to a module builds' NVSC NSVCfor a particular platform. The header data in RPMs packaged by RHEL/CentOS build system looks useful because a string it’s in NVSC NSVC format, however this data is actually arbitrary and cannot be relied uponto provide accurate NVSC NSVC data for the module.

  • The only canonical source for a module’s correct NVSCANSVCA/P data is the source repo’s metadata (generally under repodata/{XXXXX}-modules.yaml.gz, and defined by repodata/repomd.xml under <data type="modules">`)

  • 🎉 I’ve tested a repacked “slim” repo alongside a repo with the full module w/identical NVSCA NSVCA details, and it successfully resolved its metadata/packages with the full module.

    • This should also behave correctly with updated modules, but I haven’t been able to stage that yet.

...

  • On a stock EL8 server, dnf reposync requires needs the dnf-plugins-core package

  • On a stock EL7 server, dnf reposync requires needs the dnf and dnf-plugins-core packages

...

Info

Notes:

...

View packages in a module

Code Block
dnf module repoquery nodejs
Info

It looks dnf module

repoquery nodejs
Info

It looks dnf module repoquery only reports on the default stream, regardless of the module-spec specified (tested on EL8.3)

View modules/profiles that provide a package

Code Block
dnf module provides nodejs
Info

This is one of the few (two?) module commands that reports which DNF Repo hosts the module

repoquery only reports on the default stream, regardless of the module-spec specified (tested on EL8.3)

View modules/profiles that provide a package

Code Block
dnf module provides 389-ds-base
Info

This is one of the few (two?) module commands that reports which DNF Repo hosts the module

Info

Note that it also looks for packages across multiple repos and module:streams

Example:

Code Block
# dnf module provides 389-ds-base
Last metadata expiration check: 1:20:30 ago on Wed Mar 24 18:09:28 2021.
389-ds-base-1.4.3.17-1.module_el8+10764+2b5f8656.x86_64
Module   : 389-directory-server:stable:820201201092549:9edba152:x86_64
Profiles : default legacy minimal
Repo     : epel-modular
Summary  : 389 Directory Server

389-ds-base-1.4.3.8-6.module_el8.3.0+604+ab7bf9cc.x86_64
Module   : 389-ds:1.4:8030020201222185615:618f7055:x86_64
Profiles :
Repo     : appstream
Summary  : 389 Directory Server (base)

389-ds-base-1.4.4.9-1.module_el8+10763+39cf6b48.x86_64
Module   : 389-directory-server:testing:820201201092622:9edba152:x86_64
Profiles : default legacy minimal
Repo     : epel-modular
Summary  : 389 Directory Server

389-ds-base-2.0.1-1.module_el8+10522+e95198da.x86_64
Module   : 389-directory-server:next:820201104083723:9edba152:x86_64
Profiles : default minimal
Repo     : epel-modular
Summary  : 389 Directory Server

View the packages in each profile (for a specific stream)

...

Note

By convention, RPMs' ModularityLabel string is in NVSC NSVC format, which doesn’t include profile information. Any module streams constructed solely on the RPM header would lose all the original streams' profiles.

...

Note

We can’t rely on RPMs' ModularityLabel string to be in NVSC NSVC format, either!

“The ModularityLabel can be any string at all. In Fedora, we have a convention to use name:stream:version:context to indicate from which build the RPM originally came from, but this is not to be relied upon. It may change at any time and it also may not be accurately reflective of the module in which it currently resides, due to component-reuse in the Module Build System.”

— https://sgallagh.wordpress.com/2019/08/14/sausage-factory-modules-fake-it-till-you-make-it/

...

  • Create a modular repo from packages that already have a common module header
    (warning)

    Status
    colourYellow
    titleWARNING
    The approach of taking NSVCA from the RPM headers below is a.) incomplete and b.) cannot be relied upon to be accurate or available—the ModularityLabel header can contain any String

    (warning)

    Status
    colourRed
    titleDO NOT PRODUCTIZE
    Use another means of obtaining this data; preferably from the repo itself (the data is sourced from the source repository’s {XXXXX}-modules.yaml.gz file)

    Code Block
    # Get the ModularityLabel from the RPMs
##########################################################
#### UPDATE: DO NOT USE OR PRODUCTIZE THIS TECHNIQUE #####
##########################################################

# Notes: 
#  - All RPMs in the module must have a SINGLE and IDENTICAL ModularityLabel
#  - The only thing required of this String is that it is unique to RPMs 
#    from other modules (and different versions/contexts of this module)
find "$DIR_WITH_RPMS" -name \*.rpm  \
  -exec rpm -qp {} --qf '%{ModularityLabel}\n' \;  \
  | sort -u

### WARNING: the ModularityLabel headers in RPMs build by EL and EPEL 
####         are (currently) in N:S:V:C format by convention, but in 
#####        reality this string is arbitrary and cannot be relied upon
#####        to reflect the actual source module's metadata.
#####        The actual NVSCA/P data can *ONLY* be obtained from the 
#####        original repo's metadata
MODULE_HEADER=nodejs:10:8020020200707141642:6a468ee4

...