Making SIMP Repo OS specific and breaking it to sub-repos like simp-project internet repos.

The following is an overview of how the directories in the DVD and yum repos would be laid out to allow for different OS versions. It will also split the simp repository into several repositories to match the layout in the simp-project site.

Update simp-gpgkeys

Previously: all the keys were under one directory
- GPGKEY/RPM*
New Configuration: Split them according to repo:
- GPGKEY/simp/All the simp keys
- GPGKEY/puppet/All the puppet keys
- GPGKEY/postgresql/All the puppet key
- GPGKEY/epel/All the CentOS and RHEL Keys

This will allow us to link them to specific repo when creating repos in puppet module and

allow us, when downloading the from packages.yaml, to determine which repo the downloaded package should belong to.

Update rubygem-simp-rake-helpers:

Restructure the DVD Overlay file:

(Note: There are other files in these directories, if not shown, they will not be moved or reconfigured)

Previously: the DVD_overlay had:
- ks
- packagelist
- SIMP
  - GPGKEYS
  - noarch
    - All the noarch simp RPMS
  - arch (usually absent because no arch spec RPMS)
    - All the arch specific RPMS
- Development Key (if development)
New configuration:
- ks
- packagelist
- SIMP
  - simp (indicating the simp repo)
    - All the simp RPMs generated during tar build, noarch and arch
    - GPGKEYS
      - Dev key or nothing if not a dev build

Restructure the directory on the OS DVD:

Previously: it downloaded the RPMs in packages.yaml and put them in yum_data/packages and added any one off RPMs from the aux_packages directory.

Then it creates the staging directory under the build directory and one directory for each ISO it is building. I am calling this the OS staging directory.

The OS staging directory is the top level of what will be the ISO when it is finished. So here it extracts all the OS files from the OS DVD and prunes them. It extracts the DVD_overlay file to this directory (see above what is in there). Then it copies the files in yum_data/packages to the SIMP directory in the staging directory according to arch/noarch and links the noarch ones under the arch directory and creates the repo in SIMP/arch.

builddir
- yum_data
  - packages.yaml
  - packages
    - RPMs downloaded from the internet
  - aux_packages
    - any one off RPMs the user puts here.
- SIMP_ISO_STAGING
  - <OS version name> (This is the top level of the brand new ISO to be created)
    - (There is a lot more data under here but we are not concerned with this, it will all remain as it.)
    - OS data
    - SIMP
      - noarch
        simp noarch RPMs (either built earlier or extracted from tar file)
        dependency noarch RPMs (both one off and packages.yaml)
      - arch
        simp arch RPMs (either built earlier or extracted from tar file)
        dependency arch RPMs (both one off and packages.yaml)
        links to any RPM in the noarch directory
        repodata

New configuration: The RPMs from packages.yaml will be downloaded in the same manner. After they are downloaded the signature of the RPM will be checked against in each of the directories in simp-gpgkeys. The RPM will be placed in appropriate repo. Any RPMs not signed by a key from the known repos will be placed in an UNKNOWN repository. If any RPMs are in the UNKNOWN repository, the build will fail.

One off aux_packages will be checked in the same manner and copied to the appropriate repo under yum_data/packages directory.

Then it will start creating the ISO in the same manner, creating the staging directory and pruning the OS ISO into it. Then extracting the DVD_Overlay file (which it in the new format)

Then it will just merge what is under yum_data/packages into staging directory SIMP and run repo create for each directory under SIMP.

Problem: If using tarball how do I get GPGKEYS? Maybe extract from the simp-gpgkey RPM in the tarball? Or is just getting it from simp-core/src/assets ok?

The New directory structure will look like:

builddir
- yum_data
  - packages.yaml
  - packages
    - simp
      - any RPM downloaded from packages.yaml or copied from aux_packages and signed by keys in GPGKEYS/simp
    - puppet
      - any RPM downloaded from packages.yaml or copied from aux_packages and signed by keys in GPGKEYS/puppet
    - postgresql
      - any RPM downloaded from packages.yaml or copied from aux_packages and signed by keys in GPGKEYS/postgresql
    - epel
      - any RPM downloaded from packages.yaml or copied from aux_packages and signed by keys in GPGKEYS/epel
  - aux_packages
    - any one off RPMs the user puts here.
- SIMP_ISO_STAGING
  - <OS Version DIr> (this is the top level of the brand new SIMP ISO to be created)
    - All the OS data
    - SIMP
      - simp
        GPGKEYS (dir with gpg keys from simp-gpgkeys/simp and the dev key if there was one.)
        simp RPMs from the DVD_overlay file (arch and no arch)
        any RPM merged from yum_data/packages/simp
        repodata
      - puppet
        GPGKEYS
        any RPM merged from yum_data/packages/puppet
        repodata
      - epel
        GPGKEYS
        any RPM merged from yum_data/packages/epel
        repodata
      - postgresql
        GPGKEYS
        any RPM merged from yum_data/packages/postgresql
        repodata

PROBLEM: aux_packages: what if they have a different key? Should add a feature to allow users to a key?

Other things that need to be updated:

need to createrepo on all repositories
update repoclosure to include all repos

Update unpack dvd:

Update unpack DVD to copy the contents of the SIMP to /var/www/yum/SIMP/<os major version>/<arch>/ instead of just plopping it in /var/www/yum/SIMP.

The OS and the arch could be determined from the file on the DVD.

(Question… should we put el or CentOS or Redhat in there.? Probably)

Update pupmod-simp-simp

Update the creation of the simp repos to create the 4 repos and enable them where necessary.

Update simp-core DVD files

The kickstart files in simp-core will have to be updated to point to repos in the correct locations.

The DVD/auto.cfg files need to be updated.