CI: Puppet 5 spec tests have started failing for pupmod-simp-compliance_markup

Description

At some point between 08 July 2020 and 15 July 2020, Puppet 5 spec tests started failing for previously working commits:

Example pipeline at https://travis-ci.org/github/simp/pupmod-simp-compliance_markup/builds/710437767:

Example failure from https://travis-ci.org/github/simp/pupmod-simp-compliance_markup/jobs/710437770#L544:

Acceptance Criteria

None

Activity

Show:
Steven Pritchard
September 2, 2020, 8:51 PM

6.17.0 (also released on 2020-07-14) and the recently-released 6.18.0 are also broken. Tests pass on 6.16.0.

Steven Pritchard
September 9, 2020, 3:56 PM

Attaching a working (6.16.0) and a broken (6.18.0) compliance_report.yaml for reference.

 

Steven Pritchard
September 10, 2020, 3:28 PM

I'm narrowing down where the problem is.

At https://github.com/simp/pupmod-simp-compliance_markup/blob/7761d7d9425bad606b0cd4b880cbc8ac18f2727d/lib/puppetx/simp/compliance_mapper.rb#L431 on a working version of puppet, @profile_list looks like

but on a broken version of puppet, it looks like



Steven Pritchard
September 10, 2020, 4:22 PM

It appears that setting the compliance profiles via the global $compliance_profile is what broke. Example here: https://github.com/simp/pupmod-simp-compliance_markup/blob/7761d7d9425bad606b0cd4b880cbc8ac18f2727d/spec/classes/init_spec.rb#L139

Steven Pritchard
September 11, 2020, 6:01 PM
Edited

For reasons that I have yet to determine, the HIERA_compliance_data_dir environment variable is getting lost, which is breaking classes/init_spec.rb. I'm currently testing a refactor of the tests to avoid using the environment variable.

Epic Link

None

Story Points

None

Components

Assignee

Steven Pritchard

Affects versions

Priority

Medium
Configure