Puppet modules organization

This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --1dB4TkCn7j6Q3CsmV52CpB6RJVHb7kXe6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi everyone! I'm starting a thread to discuss the puppet modules organization. There are two proposed ways of organizing them: 1.- Using a unique module named ovirt_infra 2.- Using multiple modules, named ovirt_* Feel free to propose other alternatives, the main points for each one are= : 1.- Everything inside one module, easy to find 1.- Easy to add a new class, just create the file 1.- Easy to create hard to maintain code 1.- Easy to create very interdependent code 2.- Enforces modularization of the different code (one module, one task),= that brings 2.- Easier to test 2.- Safe to reuse 2.- More organized (not everything in the same place) 2.- It's the most common way of organizing puppet manifests, so the main guidelines, patterns and most of the documentation expects this way of wo= rking. Please send your comments and if too many I'll open a pad with the them f= or easy review. I vote for #2, modularized organization. --=20 David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Email: dcaro@redhat.com Web: www.redhat.com RHT Global #: 82-62605 --1dB4TkCn7j6Q3CsmV52CpB6RJVHb7kXe6 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJToAwjAAoJEEBxx+HSYmnDFHMH/2kVB6rDRgMMu7xmwbPdbBtH 64+RtXM+8uqkUfcdQVd7RTk/mzVnj8jPupiRBE/KLspkLDVLIuRObQYkqUD/Iwcr yv4DGhacdV6bMRbt4gLf4u2FcFLbS95EySqK0qnN7Y+qAJNiFQz1mjaCsSO++7sR I3jYLMX1k18JlBZv6OI8hDR8U/0SQKT4aFFzUZP06cRNBrBDCJQ/GjNB6VL+fTUu 1g0GzazfjstFYELtlICnzgG24SIWB4OD3Hv7qw+TWuH45T5PczCyuer6/1ENhPJb Afr21ZwnKdOGAI6cgeDHaSJ7bexAUu+YaOpzIWRs2YobS3E8JoqGckHhyg+4wJI= =lhwM -----END PGP SIGNATURE----- --1dB4TkCn7j6Q3CsmV52CpB6RJVHb7kXe6--

On Tue, Jun 17, 2014 at 11:36:35AM +0200, David Caro wrote:
Hi everyone!
I'm starting a thread to discuss the puppet modules organization.
There are two proposed ways of organizing them:
1.- Using a unique module named ovirt_infra 2.- Using multiple modules, named ovirt_*
Feel free to propose other alternatives, the main points for each one are:
1.- Everything inside one module, easy to find 1.- Easy to add a new class, just create the file 1.- Easy to create hard to maintain code 1.- Easy to create very interdependent code
2.- Enforces modularization of the different code (one module, one task), that brings 2.- Easier to test 2.- Safe to reuse 2.- More organized (not everything in the same place) 2.- It's the most common way of organizing puppet manifests, so the main guidelines, patterns and most of the documentation expects this way of working.
Please send your comments and if too many I'll open a pad with the them for easy review.
I vote for #2, modularized organization.
I'd be in favor of a roles/profiles pattern[1]. Given we use foreman, I think the roles pattern is filled by foreman hosts (including host groups and since foreman 1.5 configuration groups). That leaves a profiles module. I think ovirt_infra suits that, so option 1. That said, I'm not against renaming ovirt_infra to profiles and adding a roles module. Then in foreman you only select a single role. Also, I disagree it's the main guideline to build one module per role. If your task is to define all profiles, then it's still one task. Another disadvantage of many modules is that many of the standard puppet testing tools work per module. If you want to write tests for every module, you end up with a lot of files. We can have a look at writing custom scripts that perform all tests, but there's something to say about using standard tools puppet module authors are used to. (See garethr/puppet-module-skeleton[1] for some ideas of a standard module.) [1]: http://www.craigdunn.org/2012/05/239/ [2]: https://github.com/garethr/puppet-module-skeleton

This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --XQefpraS2ml66TDEdfxCbQkDxHiGFq4FF Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Tue 17 Jun 2014 04:54:45 PM CEST, Ewoud Kohl van Wijngaarden wrote:
On Tue, Jun 17, 2014 at 11:36:35AM +0200, David Caro wrote:
Hi everyone!
I'm starting a thread to discuss the puppet modules organization.
There are two proposed ways of organizing them:
1.- Using a unique module named ovirt_infra 2.- Using multiple modules, named ovirt_*
Feel free to propose other alternatives, the main points for each one = are:
1.- Everything inside one module, easy to find 1.- Easy to add a new class, just create the file 1.- Easy to create hard to maintain code 1.- Easy to create very interdependent code
2.- Enforces modularization of the different code (one module, one tas= k), that brings 2.- Easier to test 2.- Safe to reuse 2.- More organized (not everything in the same place) 2.- It's the most common way of organizing puppet manifests, so the ma= in guidelines, patterns and most of the documentation expects this way of= working.
Please send your comments and if too many I'll open a pad with the the= m for easy review.
I vote for #2, modularized organization.
I'd be in favor of a roles/profiles pattern[1]. Given we use foreman, I=
think the roles pattern is filled by foreman hosts (including host groups and since foreman 1.5 configuration groups). That leaves a profiles module. I think ovirt_infra suits that, so option 1.
That's not against the modules view I expressed before, and it's against = what we have now, let me explain: A basic example, right now we have the class ovirt_infra::user, that is n= ot a profile, and it does not belong inside a profile module. If you see the link that you sent [1] it has this structure: Node -> Role -> Profiles -> Modules -> Resources Where a profile is: "Unlike roles, which are named in a more human representation of the serv= er function, a profile incorporates individual components to represent a log= ical technology stack" The users class does not fit in any case in that definition. Instead it's= in the modules level. But right now, our modules level is empty (we just have external modules = there) and what I propose is to extract all the individual classes that do not f= it the profiles definition to their own module (one per class, as they should be= logically independent at that level). Just as it explains on the same paragraph: "the webserver profile is made up of the httpd, memcache and php componen= ts. In Puppet, these lower level components are represented by your modules." So we are not using the profiles/roles pattern right now. And I proposed = using it (maybe not that explicitly, but that was the idea), and that require t= he modularization of a few classes that we already have. That post also agrees with modularization: "I=E2=80=99ve talked about profiles and roles like they are some special = case and modules being something else. In reality, all of these classes can be, an= d should be modularised. I make a logical distinction between the profile a= nd role modules, and everything else (e.g.: modules that provide resources)."
That said, I'm not against renaming ovirt_infra to profiles and adding =
a
roles module. Then in foreman you only select a single role.
I vote for this also, so we will end up with a layout similar to this: site | - role | | - manifests | | | - puppetmaster.pp | | | - jenkins_master.pp | | | - jenkins_slave.pp | | ... | - profile | | - manifests | | - base.pp | | - jnlp_slave.pp | | - www.pp | | - nagios.pp | ...=09 | - ovirt_mod1 (custom module that is not a profile, like ovirt_user) | - ovirt_mod2 (custom module 2, for example, ovirt_repo) =2E.. That will help us easily control changes on the configuration of the host= groups, and review changes on that configuration too. But we will need another re= po to hold the sensitive parameters (passwords and all), maybe using hiera (I'v= e been playing with non-public gerrit repos, that can be useful).
Also, I disagree it's the main guideline to build one module per role. If your task is to define all profiles, then it's still one task.
Another disadvantage of many modules is that many of the standard puppe=
t
testing tools work per module. If you want to write tests for every module, you end up with a lot of files. We can have a look at writing custom scripts that perform all tests, but there's something to say about using standard tools puppet module authors are used to. (See garethr/puppet-module-skeleton[1] for some ideas of a standard module.)=
I don't see this as a disadvantage, but as an advantage, having tests for= each module means that the tests are specific to that module, are clearer and = easier to write and troubleshoot. From that same author you can see that he's qu= ite used to create modules, instead of having one big module per installation= =2E It's fairly easy to write a script to run the tests on each module, that = should be no problem.
[1]: http://www.craigdunn.org/2012/05/239/ [2]: https://github.com/garethr/puppet-module-skeleton _______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
--=20 David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Email: dcaro@redhat.com Web: www.redhat.com RHT Global #: 82-62605 --XQefpraS2ml66TDEdfxCbQkDxHiGFq4FF Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJToGK0AAoJEEBxx+HSYmnDSbkH/39gS/wIBoHrPFGmYrMtbC2H 1WAldW+etYa548C03c3aSYc9OtEAD2wEwrDZ1gaIh6UFKYblFdTI0ALpMdc60Ort YSBuXOS4y374rlTRdhonakbZwS8ghL7rQhEb+Z/E0FQc0CxQUgiWqz7rnEUzFxeq jBfRcxH6ZM1o5xTuDHAw56pZUzwxoR0wTikYMz/OmQ6I2HEbDlkmh0vciOsnNaKO JCfzUxIc3PlFjN6qIDqxMZ2gX2gKYmeRn7o8u7FCznNJItqqP1k0TcfZoPx+MaXO EQc2sWhumoAH9jgswOfklfrNzoai4saxrso0r4QkT493lHuGMbdXj59GarUx3Qw= =mAiu -----END PGP SIGNATURE----- --XQefpraS2ml66TDEdfxCbQkDxHiGFq4FF--

#2 makes much more sense to me I vote for it ----- Original Message -----
From: "David Caro" <dcaroest@redhat.com> To: "infra" <infra@ovirt.org> Cc: "Eyal Edri" <eedri@redhat.com>, "Ohad Basan" <obasan@redhat.com>, "knesenko" <knesenko@redhat.com>, "Ewoud Kohl van Wijngaarden" <ewoud+ovirt@kohlvanwijngaarden.nl>, "Michael Scherer" <mscherer@redhat.com> Sent: Tuesday, June 17, 2014 12:36:35 PM Subject: Puppet modules organization
Hi everyone!
I'm starting a thread to discuss the puppet modules organization.
There are two proposed ways of organizing them:
1.- Using a unique module named ovirt_infra 2.- Using multiple modules, named ovirt_*
Feel free to propose other alternatives, the main points for each one are:
1.- Everything inside one module, easy to find 1.- Easy to add a new class, just create the file 1.- Easy to create hard to maintain code 1.- Easy to create very interdependent code
2.- Enforces modularization of the different code (one module, one task), that brings 2.- Easier to test 2.- Safe to reuse 2.- More organized (not everything in the same place) 2.- It's the most common way of organizing puppet manifests, so the main guidelines, patterns and most of the documentation expects this way of working.
Please send your comments and if too many I'll open a pad with the them for easy review.
I vote for #2, modularized organization.
-- David Caro
Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D
Email: dcaro@redhat.com Web: www.redhat.com RHT Global #: 82-62605

Hi everyone! =20 I'm starting a thread to discuss the puppet modules organization. =20 There are two proposed ways of organizing them: =20 1.- Using a unique module named ovirt_infra 2.- Using multiple modules, named ovirt_* =20 Feel free to propose other alternatives, the main points for each one are= : =20 1.- Everything inside one module, easy to find 1.- Easy to add a new class, just create the file 1.- Easy to create hard to maintain code 1.- Easy to create very interdependent code =20 =20 2.- Enforces modularization of the different code (one module, one task),=
--=-g+a6HLRbbxbAzjZm6yyu Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Le mardi 17 juin 2014 =C3=A0 11:36 +0200, David Caro a =C3=A9crit : that
brings 2.- Easier to test 2.- Safe to reuse 2.- More organized (not everything in the same place) 2.- It's the most common way of organizing puppet manifests, so the main guidelines, patterns and most of the documentation expects this way of wo= rking. =20 =20 Please send your comments and if too many I'll open a pad with the them f= or easy review. =20 =20 I vote for #2, modularized organization.
Everything would still be in the same git, just to be clear ? I would vote for #2, and in fact, I would even as far as separating the ovirt specific stuff from the "functionnal" module. But this can be done later. --=20 Michael Scherer Open Source and Standards, Sysadmin --=-g+a6HLRbbxbAzjZm6yyu Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAABAgAGBQJTqCvCAAoJEE89Wa+PrSK9XjAQAIz+PhpeRo8wnEonTk5QRwsT vY17YR7GVa4jeBSAIWxtRnJLIgiNdUDz6l1Hh1NMdKpfoUf6dYiDPGI4w902d0f1 WlwpwIrfpywlfxlnHgPM2b4N/Lu8EMHSR8lmoBhiDy9S7ejI6OQf0luZl7yQwMgp 6JXgN2pvZrtclQ9zepbJC9Ay0Qrn9I6HBpx/gEpMK1dxkX0YmwgUOdbR7etX0eYU 4oQmyrwes50pr+/CX/Ytj4aGl/BMw9ofYP9OLtKtPYnXlE56CBjejAKkdOPtB9pG LQjprB9wPMceEy87tqAGZjGPYt2+JuxhgNTEl0TpFXGlmHdKJu5m7gW/BykgwclM DWfgNsa2xjfJPjm5NAnaZx6lTk1YvijfQMlKEvj7xo1XrBpsUguVhY8F7bDPnPRy mAtMsoVeXnv2YwRnODrNkYeVdqFIRA5HWnEIBDl3gHuCNkXyk42/LTZGGW/OthiO 2HKYWW7bQmzgpvq/rXUSAFkK9ZKKCCnupG6lSMwesrDX6tyL4oQuRVXvPmGLYQFn fJkhZzZappyv0YK0UjuXc9b8UFFTvk05uu16sUFK0qgBU+wI8qCQLy9h4Ug32Hro cn6CyZN7qN0TC8OOZ+cMQX5mQ58oJtt++DaQE2vycQCCpRRa2N33wbLq9ld2Qfrh aOQjTXmOBlTXHYraWXRJ =EJDM -----END PGP SIGNATURE----- --=-g+a6HLRbbxbAzjZm6yyu--
participants (4)
-
David Caro
-
Ewoud Kohl van Wijngaarden
-
Michael Scherer
-
Ohad Basan