Puppet design

[Ewoud Kohl van Wijngaarden]

On Wed, May 22, 2013 at 05:40:46PM +0200, Ewoud Kohl van Wijngaarden wrote:
> I added the EL6 hosts to our foreman instance, but Fedora has a newer
> puppet version (2.7) than EPEL (2.6). Since a 2.7 client can't talk to a
> 2.6 master, I'd like to propose we start using the puppetlabs repo and
> upgrade to 3.1.

This should have been a bit longer. Remember that anything we now decide
can be changed later. I'm just giving many options.

## Repository location

Currently there's a repository at ssh://puppet@foreman.ovirt.org/puppet
with a post-receive hook installed that converts each branch into an
environment in /etc/puppet/environments, but I think want to be on
gerrit.ovirt.org so we need request a repository and figure out how to
do deployments on merges. This could be a jenkins job, but maybe setting
up gerrit to mirror is sufficient.

## General repository layout

I think we all agree we want to somehow manage it in a git repository.
But when it comes to the modules, we have some options

- Have modules as git submodules
This is something we do at $employer and works reasonable well, you have
good control over your modules. On the other hand, manual dependency
checking. I heard that you can use puppet module to locally install and
then check into git as well, but no experience here. Also, git
submodules are a bit cumbersome to handle.

- Use librarian-puppet
The tagline on http://librarian-puppet.com/ is actually 'You can all
stop using git submodules now'. I have no experience with it and would
require modifications on the git hook if we decide to continue using
that. One thing I am unsure of here is dependencies. When does
librarian-puppet decide to update and will that break other modules?

librarian-puppet seems like the cool choice, but I have no experience
with it so can't judge it.

## Actual modules

When we get to the actual modules, I generally like to use all the
foreman installer modules to keep the systems in line, but one downside
is already that theforeman/apache is quite different from
puppetlabs/apache and both claim the name apache. While foreman is
moving towards more common modules, that's something I expect for 1.3
(though we can just run development modules when that becomes an
option).

Then also comes the question of which modules we want to use and what we
actually want to automate with puppet. I'd suggest to cheat here by
making it up as we go along. Just set a goal (manage user logins for
example) and just grow from there.

## Data

Assuming we have a puppet repository, some modules with logic, we will
need to add data. Again, multiple options.

- Have a data module
Just statically write all the data in a separate data module. Not that
desirable I think, but certainly an option

- Set parameters through foreman
Foreman can override class parameters through the ENC, but you can't add
defines here. So you may end up with a mix of data module and data in
another system.

- Hiera
Since puppet 3.0 fully integrated, allows you to override class
parameters in a hierarchical manner using multiple backends. Also allows
you to add defines. One big benefit can be that you can manage it in a
separate git repositories due to the hierarchical nature. One public and
one private. The former can be for public data and the latter can also
hold SSL private keys, maybe passwords. One challenge may be to keep
multiple repositories in sync in terms of deployment.

## Concrete proposal

A long time ago I wrote https://github.com/ekohl/ovirt-infra-puppet.
It's set up using the submodules way, only has a single module (for
jenkins) and hardcodes data, but it is a start. It has a decent start to
add users and I'd like to expand on that. That way we can easily create
accounts for us admins on all servers.

The next step could be to add puppetdb to our setup so we can use
exported resources to generate the icinga hosts list, but an easier
option is to start setting up a resources.ovirt.org configuration so we
can start to migrate linode01. resources.ovirt.org is just serving
static content so should be very easy.