Re: Puppet design

On Wed, May 22, 2013 at 05:40:46PM +0200, Ewoud Kohl van Wijngaarden wrote: This should have been a bit longer. Remember that anything we now decide can be changed later. I'm just giving many options. ## Repository location Currently there's a repository at ssh://puppet@foreman.ovirt.org/puppet with a post-receive hook installed that converts each branch into an environment in /etc/puppet/environments, but I think want to be on gerrit.ovirt.org so we need request a repository and figure out how to do deployments on merges. This could be a jenkins job, but maybe setting up gerrit to mirror is sufficient. ## General repository layout I think we all agree we want to somehow manage it in a git repository. But when it comes to the modules, we have some options - Have modules as git submodules This is something we do at $employer and works reasonable well, you have good control over your modules. On the other hand, manual dependency checking. I heard that you can use puppet module to locally install and then check into git as well, but no experience here. Also, git submodules are a bit cumbersome to handle. - Use librarian-puppet The tagline on http://librarian-puppet.com/ is actually 'You can all stop using git submodules now'. I have no experience with it and would require modifications on the git hook if we decide to continue using that. One thing I am unsure of here is dependencies. When does librarian-puppet decide to update and will that break other modules? librarian-puppet seems like the cool choice, but I have no experience with it so can't judge it. ## Actual modules When we get to the actual modules, I generally like to use all the foreman installer modules to keep the systems in line, but one downside is already that theforeman/apache is quite different from puppetlabs/apache and both claim the name apache. While foreman is moving towards more common modules, that's something I expect for 1.3 (though we can just run development modules when that becomes an option). Then also comes the question of which modules we want to use and what we actually want to automate with puppet. I'd suggest to cheat here by making it up as we go along. Just set a goal (manage user logins for example) and just grow from there. ## Data Assuming we have a puppet repository, some modules with logic, we will need to add data. Again, multiple options. - Have a data module Just statically write all the data in a separate data module. Not that desirable I think, but certainly an option - Set parameters through foreman Foreman can override class parameters through the ENC, but you can't add defines here. So you may end up with a mix of data module and data in another system. - Hiera Since puppet 3.0 fully integrated, allows you to override class parameters in a hierarchical manner using multiple backends. Also allows you to add defines. One big benefit can be that you can manage it in a separate git repositories due to the hierarchical nature. One public and one private. The former can be for public data and the latter can also hold SSL private keys, maybe passwords. One challenge may be to keep multiple repositories in sync in terms of deployment. ## Concrete proposal A long time ago I wrote https://github.com/ekohl/ovirt-infra-puppet. It's set up using the submodules way, only has a single module (for jenkins) and hardcodes data, but it is a start. It has a decent start to add users and I'd like to expand on that. That way we can easily create accounts for us admins on all servers. The next step could be to add puppetdb to our setup so we can use exported resources to generate the icinga hosts list, but an easier option is to start setting up a resources.ovirt.org configuration so we can start to migrate linode01. resources.ovirt.org is just serving static content so should be very easy.