simp config should be able to create data files outside of an environment


Much of the SIMP installation process has been updated to be more compatible with tools like r10k and Code Manager. Unfortunately `simp config` still poses a major burden to those workflows as it does not allow for creating the data files that SIMP needs in order to function correctly outside of a puppet environment. A user should be able to run `simp config` on a system that already has a populated `production` environment and generate any data files into a local directory instead of a Puppet environment.

Acceptance Criteria



Liz Nemsick
August 7, 2019, 2:51 PM

Liz Nemsick:glitch_crab: 10:29 AM
simp config generates a SIMP Omni-Environment. In the configuration suggested in SIMP-6938, what do you want it to do about the secondary environment?
offline files don't make sense without the SIMP Omni-Environment. Are you assuming the user will create that on their own?
Kendall Moore 10:30 AM
I want it to do nothing except for creating data files
Trevor Vaughan:simp: 10:30 AM
@Liz Nemsick Can we capture the Q&A in the ticket please so that we don't lose anything?
Liz Nemsick:glitch_crab: 10:30 AM
Are you assuming it will copy over the puppet skeleton files to modify?
@Trevor Vaughan yes, after the discussion has concluded

It modifies the SIMP server's host YAML file
in addition to creating the simp_config_settings.yaml
Also, do you want it to apply it's other actions (turning off CentOS repos, etc) or do you simply want a dry-run for hiera files?
Other actions it takes include setting up Puppet configuration for bootstrap
There is a list of actions taken in simp-doc
new messages
Kendall Moore 10:33 AM
I attempted to find this list in read the docs but haven't been successful
Liz Nemsick:glitch_crab: 10:33 AM
Kendall Moore 10:34 AM
I just want simp config to generate simp_config_settings.yaml and the SIMP server host file outside of an environment
Liz Nemsick:glitch_crab: 10:35 AM
OK, but do you want it to grab the SIMP-provided example SIMP server yaml file and modify that template?
Do you want the whole Puppet skeleton, including the environment.conf that includes the secondary path in the module path?
and the site.pp?
Kendall Moore 10:36 AM
Not worried about site.pp
Liz Nemsick:glitch_crab: 10:36 AM
and the hieradata hierarchy
Kendall Moore 10:36 AM
I don't see any mention of environment.conf in the docs so I don't know what simp config does to it
I'm not worried about the hiera.yaml
Liz Nemsick:glitch_crab: 10:37 AM
So, the SIMP server hiera.yaml makes sense within the SIMP environment skeleton. That is what I am getting at:
Kendall Moore 10:37 AM
I suppose having flags that allow you to generate any/all of those is a good thing
But primarily it's the global data and the host-specific data
Liz Nemsick:glitch_crab: 10:38 AM
No more flags leads to confusion!!! Have you seen the simp config is more than a page already!
Kendall Moore 10:38 AM
Hey we love docs, right?
Liz Nemsick:glitch_crab: 10:39 AM
There are so many twiddles already in simp config that it is a nightmare to understand and test
Kendall Moore 10:39 AM
I'm sympathetic to that but it's still not really usable to people that use r10k or code manager as is without them
Liz Nemsick:glitch_crab: 10:40 AM
Why isn't the option of having simp config create a SIMP Omni-environment named other than production a solution to the problem? It does what you want, you can check in the files and then delete the directories if you don't need them (edited)
Kendall Moore 10:41 AM
Because if it creates an environment in the environment path then it breaks the deployment strategy of those tools
Liz Nemsick:glitch_crab: 10:41 AM
Creating the files without the entire SIMP Omni environment is more problematic IMHO
Kendall Moore 10:41 AM
I don't want a single thing ever other than the tool that I'm using (let's say r10k for now) to touch my environment
Liz Nemsick:glitch_crab: 10:41 AM
The user will forget to create the secondary environment and it won't work
You are running simp config, so you are touching the environment
Kendall Moore 10:43 AM
The entire point of that ticket was basically to suggest that I should be able to do that without touching the environment
Liz Nemsick:glitch_crab: 10:43 AM
You also know what to do with the results. The puppet environment will be blown away if not checked into a control repo (edited)
Trevor Vaughan:simp: 10:43 AM
I guess we could add something like a -C option and just spew everything into an "alternate root"
Liz Nemsick:glitch_crab: 10:43 AM
How will you create the secondary environment in your workflow?
Trevor Vaughan:simp: 10:43 AM
Kind of like tar
That way, you can have a big reference bundle of insanity and figure it out at your leisure
Liz Nemsick:glitch_crab: 10:45 AM
A pair of roots: a root for /etc/puppetlabs/code/environments and a root for /var/simp/environments?
the generated environment.conf would be incorrect and have to be fixed
(fixed when you actually use the environments)
Kendall Moore 10:46 AM
Is there somewhere that lists out what actually gets put into /var/simp/environments
Liz Nemsick:glitch_crab: 10:46 AM
run simp environment new myenv and look at the log of all the actions done
Yes there is a list, but it may still be in a simp-doc PR

Liz Nemsick
August 7, 2019, 2:57 PM

new messages
Kendall Moore 10:51 AM
I'm going to look at what's in there but I doubt anything other than simp-specific things are ever going to live in there (hence the name) so I'm thinking I'm probably ok with simp config modifying what's in there
what I don't want is for simp config to modify anything in /etc/puppetlabs/code/environments unless I have opted-in
I'm not going to argue one way or the other what the default behavior should be so long as I have a way to generate the data that would go into that directory and just put it somewhere else instead
Liz Nemsick:glitch_crab: 10:53 AM
I still would like to understand your workflow so that I understand how you are going to create the required SIMP secondary environment
Without significant work on your part, SIMP will not work without it
I am very uncomfortable giving you a mechanism to create a non-functioning environment (Puppet only)
In other words, if we give the user a simp config option that they don't understand and it doesn't create a functioning environment, we will be fielding bug reports/issues left and right
Kendall Moore 10:56 AM
to me that sounds like a good case for making the default behavior of simp config do what it does right now
and then give power users the ability to do things that may not work unless you understand the impacts of it

Liz Nemsick
August 7, 2019, 3:26 PM

Kendall Moore 10:56 AM
to me that sounds like a good case for making the default behavior of simp config do what it does right now
and then give power users the ability to do things that may not work unless you understand the impacts of it
it looks to me like the only thing that gets created in the secondary environment that I'm going to care about by default is the FakeCA stuff
if I'm using krb5 then that will be relevant too but I think most people I've ever encountered aren't
Trevor Vaughan:simp: 11:01 AM
krb5 usage is VERY rare
Which is too bad :stuck_out_tongue:
Kendall Moore 11:01 AM
yeah, I agree
@Liz Nemsick if simp config creates the secondary environment but doesn't create the primary (is this what we're calling it?) it's not going to break anything
the existence of the secondary environment doesn't actually negatively impact anything right?
Liz Nemsick:glitch_crab: 11:03 AM
(puppet environment, secondary environment, writable environment See
Kendall Moore 11:03 AM
I see, so the puppet environment then
you can create secondary environments and that will never conflict with an r10k deployment
so the workflow I'm proposing is that I would like simp config to generate the global Hiera settings (simp_config_settings.yaml) and the SIMP host specific data file but do so in a directory that I specify
Liz Nemsick:glitch_crab: 11:04 AM
secondary environment also contains rsync data
Kendall Moore 11:05 AM
I see that as well but again is not something that I'm worried about typically
for the purpose of this let's just say it's safe to include the rsync data as well
if I don't use it then so be it
when you run simp config if it populates the secondary environment then that's fine, though I'll say I'd like it if there was something written to stdout or a log that told me it had done so
Liz Nemsick:glitch_crab: 11:06 AM
(already logged)
Kendall Moore 11:06 AM
that way when I deploy the rest of the puppet environment, it will already be there
ok great, that's covered then
Trevor Vaughan:simp: 11:07 AM
We did confirm that, with the current method, if you do everything as documented and then use r10k how you normally would, it happily hulk-smashes through everything that we did.
Kendall Moore 11:07 AM
yeah that sounds correct
I would expect it to do that
Trevor Vaughan:simp: 11:08 AM
Also, if you already used r10k to deploy to production it doesn't do anything.
Liz Nemsick:glitch_crab: 11:10 AM
What do you want to do about creating the (FakeCA) application PKI needed for the SIMP-server in the secondary environment?
simp config does that for you
Kendall Moore 11:10 AM
if it's self contained within the secondary environment then go ahead and do it
and per your statement Trevor that functionality (which I think is correct) is exactly what spawned this conversation
if there's a production environment that's already been deployed I don't want simp config to do anything to it at all
I do however want it to be able to create the data files that it would otherwise put in there
Liz Nemsick:glitch_crab: 11:12 AM
simp bootstrap only operates on the production environment. Is it safe to say you do not need any actions for bootstrap (e.g. configuring Puppet digest algorithm)? You will have to handle that yourself
Kendall Moore 11:12 AM
I think that's completely fair
I wouldn't expect simp bootstrap to have to adhere to any of this
if you run that command you're assuming that you're bootstrapping an initial environment
Liz Nemsick:glitch_crab: 11:13 AM
Did you read through all the other simp config actions? I would like to make sure there is nothing else there essential to how you want to use the new environment
Kendall Moore 11:13 AM
I read through all of the release notes
I don't know if that's comprehensive though
Liz Nemsick:glitch_crab: 11:14 AM
I'm talking about the table I referenced above
Kendall Moore 11:16 AM
So are the things in /etc/puppetlabs/puppet only modified if the --force-config option is set to true (assuming a production environment exists)?
Also based on reading this I'm going to have to go back on something I said before.... I would actually like if the site.pp was created in addition to the two YAML files
I didn't realize that's where the scenario was set
It's likely that I would just modify that by hand in my own site.pp or in my ENC but I could see forgetting to do so if I don't see the file populated
Liz Nemsick:glitch_crab: 11:19 AM
--force-config Allow 'simp config' to apply config changes
when the Puppet environment to be created
already exists. Allows *ALL* config
actions to be applied (system, Puppet
global, Puppet environment). Does not
recreate the SIMP omni-environment.
Are you assuming simp config will create the offline puppet environment directory completely from the skeleton or do you expect it to modify something you have pre-populated
new messages
Kendall Moore 11:21 AM
I'm expecting it to use the skeleton

Liz Nemsick:glitch_crab: 11:21 AM
OK. This is what I think you are saying:
• create a new SIMP Omni-environment for a named environment from SIMP skeleton dirs and allow specification of the puppet environment directory ( which may be outside of /etc/puppetlabs/code/environments)
• restrict the subset of actions to those that modify the Puppet environment files and secondary files
• fail if the SIMP Omni-environment (with alt Puppet dir) already exists
Kendall Moore 11:25 AM


Epic Link


Story Points



Affects versions