SIMP Compliance Engine slows puppet agent

Description

A test system running CentOS 7 and SIMP 6.4 has very high latency during puppet agent runs when the SIMP Compliance Engine is turned on. I used the documentation found at Enable STIG Mode to configure the test system.

Specs:
4G RAM
1 or 2 CPU (tried both, no difference)

I noticed before running the puppet agent with the compliance engine enabled the following figure in /opt/puppetserver/puppet/cache/state/last_run_summary.yaml:
config_retrieval: 14.3011...

Both disa_stig and nist compliance profiles had a similar effect:
config_retrieval: 609.8544...

Notice a 43x increase in time

Overall time for a puppet run jumps from about 37s to about 10m30s when comparing with and without the compliance engine enabled.

I tried disabling selinux and antivirus; none are effective. No logs that I've inspected indicate a problem either:

  • /var/log/:
    audit/audit.log
    secure
    messages
    puppet-agent.log
    puppet-agent-err.log
    puppetserver.log

  • journalctl

'iotop' shows no disk i/o bottlenecks. 'top' shows that the java process related to puppetserver runs at 100% cpu load, but only one core. RAM shows no limitations on processes.

Acceptance Criteria

None

Activity

Show:
Mark Fitch
July 24, 2020, 6:07 PM

This issue was caused by a configuration error stemming from possibly out-dated documentation on STIG Enforcing Mode and can be closed. Please reference for further information.

Mark Fitch
July 24, 2020, 6:12 PM

Issue closed but documentation needs to be corrected as indicated in

Steven Pritchard
July 31, 2020, 2:09 PM

This sounds a problem with your Puppet server. What are the specs on it? Is it memory constrained?

Steven Pritchard
July 31, 2020, 5:09 PM

More important question: was this before or after you changed compliance_markup::enforcement to an array?

Mark Fitch
August 3, 2020, 10:56 AM

No, this issue was not caused by lack of resources on the puppet server. I noticed no strain on CPU or RAM. The problem was fixed when I changed the value type from a string to an array.

Labels

None

Epic Link

None

Story Points

None

Affects versions

Priority

Medium

Assignee

Steven Pritchard
Configure