Proposal for hosting our own set of modules
--=-xFOcEmQD0W2VdLZ69dG3 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi, so following the discussion on puppet module, I would propose that we create a new infra-puppet-modules module on gerrit.=20 The division between this module and the current one would be like this : infra-puppet would hold the manifests, the site, the Puppetfile , etc infra-puppet-modules would hold directories, 1 per module we develop (not the external one we use, they would still be on github or anywhere, pulled by librarian-puppet) . Since we are using librarian-puppet ( https://github.com/rodjek/librarian-puppet ) and r10k ( https://github.com/adrienthebo/r10k ) , it requires use to have the modules/ directory to be managed by librarian-puppet. In turn we need to have the modules in git, we can address them by path:=20 mod "puppetlabs/apt", :git =3D> "git://github.com/fake/puppet-modules.git", :path =3D> "modules/apt" This wouldn't requires to change much, besides adding the module to Puppetf= ile and creating a git repository. If no one disagree, I will request the git repository. ( in the mean time, i did create a sample awstats repository for stats.ovir= t.org, so we can have something to push ) --=20 Michael Scherer Open Source and Standards, Sysadmin --=-xFOcEmQD0W2VdLZ69dG3 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAABAgAGBQJTqDofAAoJEE89Wa+PrSK9rZkP/3ADQuMcsNLlKTFk+9szPgAT awtULTDos/IEs7/EPD16U8eYPRcTeNiWdtiAOgW33aKkewkIMZspiUVsZtcWDyPq kUzO/RTSgsMgVT1iXnCAJOJI048glfJhZgSTVzrQYovupjjC6dYtSWCESdvWlk68 UwK7MQZ1xRMUL9Iq65IfPft+N4cr1xhXjbxCnNcZHXYknQ3vZMQCSxcMMlW7pd/9 CNin8wKqOHkyZa6bFEbxqHtltB7qYBr4QJKfOyJI7kMBd6UsnX1zJwRVsMD8fe4L 4q5n7FsLNT9mWnNUuadHosx5ijttiSIa+2aR83Q9PqQWfr36Qf+VeMBV2m6FFB5D rbe0pLluuIuAVsxdDU7T/kO81Dmb9aCaoMiTY+l6lEuDjDUsv54XUhCKppVndJjG nBK+z8jMyoOIoU/hn/YELo/n969qFAKVFBg7oINlcJW53VkSDdjW7xdf2hnYeh5v cP3znlt4DUKaYi1MENyVW0vDyBlIt2X/U49bqD069Mqihy0xp3ym4dQ3i8mQoXTm j9ao5rPM6A916SmhFJxqkZpStgb7Mcwhl9FhVde/QvtZyVyJlNq2cuJsat1UxKfV YhdSUuFWpFG/p827adtwYXANYP0OSMLOp6fP0COZSVpQm6hS/HQcCc9CJXvn6JVv LcG7+oxmlnCt2Y8fAWzy =cMcE -----END PGP SIGNATURE----- --=-xFOcEmQD0W2VdLZ69dG3--
On Mon, Jun 23, 2014 at 04:30:55PM +0200, Michael Scherer wrote:
Hi,
so following the discussion on puppet module, I would propose that we create a new infra-puppet-modules module on gerrit.
The division between this module and the current one would be like this :
infra-puppet would hold the manifests, the site, the Puppetfile , etc
infra-puppet-modules would hold directories, 1 per module we develop (not the external one we use, they would still be on github or anywhere, pulled by librarian-puppet) .
Since we are using librarian-puppet ( https://github.com/rodjek/librarian-puppet ) and r10k
We're only using r10k, not librarian-puppet.
( https://github.com/adrienthebo/r10k ) , it requires use to have the modules/ directory to be managed by librarian-puppet. In turn we need to have the modules in git, we can address them by path:
mod "puppetlabs/apt", :git => "git://github.com/fake/puppet-modules.git", :path => "modules/apt"
This wouldn't requires to change much, besides adding the module to Puppetfile and creating a git repository.
If no one disagree, I will request the git repository.
( in the mean time, i did create a sample awstats repository for stats.ovirt.org, so we can have something to push )
Unless we plan to make them reusable for other projects, I don't see the benefit. If we do plan to make them reusable, we should IMHO also publish them on the forge. Another potential issue is how we decide when to deploy. We could have a specific commit ID and update our Puppetfile every time, but again, little benefit over having them in one tree. In case you're unaware, we already load modules/* (which is managed by r10k) and site/* (tracked in git). That means we can host our modules in site and spit off when they are reusable. And if we do plan on creating modules repos, I'd be in favor of having one git repo per puppet module since that is what most people would expect.
--=-GoS11saLxRw+sj5i/l+X Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Le lundi 23 juin 2014 =C3=A0 16:42 +0200, Ewoud Kohl van Wijngaarden a =C3=A9crit :
On Mon, Jun 23, 2014 at 04:30:55PM +0200, Michael Scherer wrote:
Hi, =20 so following the discussion on puppet module, I would propose that we create a new infra-puppet-modules module on gerrit.=20 =20 The division between this module and the current one would be like this : =20 infra-puppet would hold the manifests, the site, the Puppetfile , etc =20 infra-puppet-modules would hold directories, 1 per module we develop (not the external one we use, they would still be on github or anywhere= , pulled by librarian-puppet) . =20 Since we are using librarian-puppet ( https://github.com/rodjek/librarian-puppet ) and r10k =20 We're only using r10k, not librarian-puppet.
( https://github.com/adrienthebo/r10k ) , it requires use to have the modules/ directory to be managed by librarian-puppet. In turn we need t= o have the modules in git, we can address them by path:=20 =20 mod "puppetlabs/apt", :git =3D> "git://github.com/fake/puppet-modules.git", :path =3D> "modules/apt" =20 =20 This wouldn't requires to change much, besides adding the module to Pup=
Indeed.=20 But in the end, we still use Puppetfile nonetheless, no ? petfile and creating a git repository.
=20 If no one disagree, I will request the git repository. =20 ( in the mean time, i did create a sample awstats repository for stats.= ovirt.org, so we can have something to push ) =20 Unless we plan to make them reusable for other projects, I don't see the benefit. If we do plan to make them reusable, we should IMHO also publish them on the forge. =20 Another potential issue is how we decide when to deploy. We could have a specific commit ID and update our Puppetfile every time, but again, little benefit over having them in one tree.
I was under the impression you could just give a tag and so use master ? ( and using branch for development )
In case you're unaware, we already load modules/* (which is managed by r10k) and site/* (tracked in git). That means we can host our modules in site and spit off when they are reusable.
It was not very obvious that site/* was for modules, indeed :) But my fault, I should also have read the doc in the git repository who clearly say that. I must also say that using r10k for different environments seems a bit overkill, as we seem to only have 1 single environment anyway ( but I am not using r10k usually, as I do not have enough systems for that and write all stuff myself ).=20
And if we do plan on creating modules repos, I'd be in favor of having one git repo per puppet module since that is what most people would expect.
One git repo per module is a bit annoying when we are planning to write a lot of modules. But I guess it all depend on the time it take to create one. I would definitely prefer a approach of 1 big git as long as we are "growing" fast and maybe split later, since it permit faster growth ? --=20 Michael Scherer Open Source and Standards, Sysadmin --=-GoS11saLxRw+sj5i/l+X Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAABAgAGBQJTqE+rAAoJEE89Wa+PrSK9eSQP/2IJxedKswUoVphKXKsp+Lox FfabCRRosySEQ31tD2F8VZvzB0Xy/SPrdyK/4H9u+G4H1t6FazZfPkh4q2abo4F6 JgZSZP++bNiIozrlXRNqYRJa0jcQli3oPnpjWzXDunhXI+8SXkejeN7jtUmydOxj Y6dzjaEHLSsggke9Uk42PmRPdXVLLdNyP7e8fRdMIHfP8+OvJm9X52Cz86JPPv2f hUBLkam5ZNwWgQEaPSSvPy+LGo4FBnvJ9yeGa5ODNiU2NvFtve0wsx9hRdVkOZzx lO7rx22a6AYcfYrpJhroFZA8bzkegGiDp6kzVkogvkFtlfDazKZycXsu2idJqYQR uKXjjf+H+ERhfpZ6FfEXDoypgaqlRK4eceinSgBfMv/wqGtSx9TZe7NqvHBTmgLA R/0aw/3SlGdVsWKFgaxaZW6ZIeh/PvCc83XJtuy8EjkEBIQeCR18Sz2zggjes/FB jpZHCJwUOdbCGh9L1Lpo6phuJNTuVOC5r9xjk2NIfFMWtFyDCakecH1ncdtvvK/B 3czgGauq3Opz04cAx1f1nPAj8Q1Mrv53N6MvQfT+3IJ8vjdXsINyPrJ4aJvSCYBM T6mr9vAYHPZHsPHwSj9FLwm27lIRdfDnPlOe1Y1KvPEZbEj++h3y7n8aoShmEoe4 waKCRscrjNh1oMb4ff+l =zOY/ -----END PGP SIGNATURE----- --=-GoS11saLxRw+sj5i/l+X--
On Mon, Jun 23, 2014 at 06:02:51PM +0200, Michael Scherer wrote:
Le lundi 23 juin 2014 à 16:42 +0200, Ewoud Kohl van Wijngaarden a écrit :
On Mon, Jun 23, 2014 at 04:30:55PM +0200, Michael Scherer wrote:
Hi,
so following the discussion on puppet module, I would propose that we create a new infra-puppet-modules module on gerrit.
The division between this module and the current one would be like this :
infra-puppet would hold the manifests, the site, the Puppetfile , etc
infra-puppet-modules would hold directories, 1 per module we develop (not the external one we use, they would still be on github or anywhere, pulled by librarian-puppet) .
Since we are using librarian-puppet ( https://github.com/rodjek/librarian-puppet ) and r10k
We're only using r10k, not librarian-puppet.
Indeed.
But in the end, we still use Puppetfile nonetheless, no ?
Correct, but librarian handles dependencies where r10k ignores them. That means you must manually specificy any dependency.
( https://github.com/adrienthebo/r10k ) , it requires use to have the modules/ directory to be managed by librarian-puppet. In turn we need to have the modules in git, we can address them by path:
mod "puppetlabs/apt", :git => "git://github.com/fake/puppet-modules.git", :path => "modules/apt"
This wouldn't requires to change much, besides adding the module to Puppetfile and creating a git repository.
If no one disagree, I will request the git repository.
( in the mean time, i did create a sample awstats repository for stats.ovirt.org, so we can have something to push )
Unless we plan to make them reusable for other projects, I don't see the benefit. If we do plan to make them reusable, we should IMHO also publish them on the forge.
Another potential issue is how we decide when to deploy. We could have a specific commit ID and update our Puppetfile every time, but again, little benefit over having them in one tree.
I was under the impression you could just give a tag and so use master ? ( and using branch for development )
You can, but how do we know we want to update and to which version? branches can easily break compatibility and it will bite you at some point.
In case you're unaware, we already load modules/* (which is managed by r10k) and site/* (tracked in git). That means we can host our modules in site and spit off when they are reusable.
It was not very obvious that site/* was for modules, indeed :)
But my fault, I should also have read the doc in the git repository who clearly say that.
I must also say that using r10k for different environments seems a bit overkill, as we seem to only have 1 single environment anyway ( but I am not using r10k usually, as I do not have enough systems for that and write all stuff myself ).
The alternative is either include them or git submodules. I'm not a fan of the former and I've seen people have issues with the latter where people accidentally commit an older version. It's also hard to spot in reviews because you only see two git commit IDs with no idea if it's an upgrade or downgrade. That's why a Puppetfile-based solution was chosen. Minor nitpick: technically we have 2 environments and we should make an effort to remove master. This is done by changing the default branch to production and then remove master. The only way I know is logging into gerrit through SSH and changing infra-puppet.git/HEAD to point to production instead of master. Then you can use the gerrit UI to delete the master branch.
And if we do plan on creating modules repos, I'd be in favor of having one git repo per puppet module since that is what most people would expect.
One git repo per module is a bit annoying when we are planning to write a lot of modules. But I guess it all depend on the time it take to create one. I would definitely prefer a approach of 1 big git as long as we are "growing" fast and maybe split later, since it permit faster growth ?
That's exactly what I was thinking.
--=-0uQkNJQzolsX4XkvG6de Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Le mardi 24 juin 2014 =C3=A0 09:59 +0200, Ewoud Kohl van Wijngaarden a =C3=A9crit :
On Mon, Jun 23, 2014 at 06:02:51PM +0200, Michael Scherer wrote:
Le lundi 23 juin 2014 =C3=A0 16:42 +0200, Ewoud Kohl van Wijngaarden a =C3=A9crit :
On Mon, Jun 23, 2014 at 04:30:55PM +0200, Michael Scherer wrote:
( https://github.com/adrienthebo/r10k ) , it requires use to have t= he modules/ directory to be managed by librarian-puppet. In turn we ne= ed to have the modules in git, we can address them by path:=20 =20 mod "puppetlabs/apt", :git =3D> "git://github.com/fake/puppet-modules.git", :path =3D> "modules/apt" =20 =20 This wouldn't requires to change much, besides adding the module to= Puppetfile and creating a git repository. =20 If no one disagree, I will request the git repository. =20 ( in the mean time, i did create a sample awstats repository for st= ats.ovirt.org, so we can have something to push ) =20 Unless we plan to make them reusable for other projects, I don't see = the benefit. If we do plan to make them reusable, we should IMHO also publish them on the forge. =20 Another potential issue is how we decide when to deploy. We could hav= e a specific commit ID and update our Puppetfile every time, but again, little benefit over having them in one tree. =20 I was under the impression you could just give a tag and so use master = ? ( and using branch for development ) =20 You can, but how do we know we want to update and to which version? branches can easily break compatibility and it will bite you at some point.
It depend on the release model of our own modules. We can have a policy of "master should be deployable and forever retro compatible" but once expressed like this, it doesn't sound sustainable (even if that's what we would do with a single git repository ).=20
In case you're unaware, we already load modules/* (which is managed b= y r10k) and site/* (tracked in git). That means we can host our modules= in site and spit off when they are reusable. =20 It was not very obvious that site/* was for modules, indeed :) =20 But my fault, I should also have read the doc in the git repository who clearly say that. =20 I must also say that using r10k for different environments seems a bit overkill, as we seem to only have 1 single environment anyway ( but I a= m not using r10k usually, as I do not have enough systems for that and write all stuff myself ).=20 =20 The alternative is either include them or git submodules. I'm not a fan of the former and I've seen people have issues with the latter where people accidentally commit an older version. It's also hard to spot in reviews because you only see two git commit IDs with no idea if it's an upgrade or downgrade. That's why a Puppetfile-based solution was chosen.
I am not fan of git submodule either. But the Puppetfile file part is good, it just the multiple env that I found weird. Now, if that's unavoidable, no problem.
Minor nitpick: technically we have 2 environments and we should make an effort to remove master. This is done by changing the default branch to production and then remove master. The only way I know is logging into gerrit through SSH and changing infra-puppet.git/HEAD to point to production instead of master. Then you can use the gerrit UI to delete the master branch.
As we discussed on IRC later, I would be +1 for this, provided that this doesn't break too much assumption in docs ( especially gerrit doc around ). But as we discussed too, maybe we should just push to use git-review.
And if we do plan on creating modules repos, I'd be in favor of havin= g one git repo per puppet module since that is what most people would expect. =20 One git repo per module is a bit annoying when we are planning to write a lot of modules. But I guess it all depend on the time it take to create one. I would definitely prefer a approach of 1 big git as long a= s we are "growing" fast and maybe split later, since it permit faster growth ? =20 That's exactly what I was thinking.
This also bring the question of using external module vs using our own. I am more the kind of guy that write his own (while I am not the kind of guy who write his own code), mostly because I started to use puppet before the forge was up, and because I did have very precise ideas on what I wanted to achieve, but I am not against using stuff if that's the best practices :) I would however make sure we have proper guidelines on when we take a module, as I am not sure they are all equally good :/ --=20 Michael Scherer Open Source and Standards, Sysadmin --=-0uQkNJQzolsX4XkvG6de Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAABAgAGBQJTqfJBAAoJEE89Wa+PrSK98pUQAJfpr4bpjL+fV7f2F/PE9eXd ZXpeO4pih2AkfKmppIRo7Icc+UZkz4rD92fm7G1WjW/0eLQQrCtFDKWPhiDBq26z 7qjvIAqOP5nH/cc/0alS7b6B6HRGOzvlmoTeOJG7MhhoV3fKrtBDAZkefWL2QDAI Fvss7v1scSMSyVUpD//Ats6EKcJXIebGcYEzIqxFwzVe66D6q6ruF6MOVwBUGH/X xclKyqMdNsZy58kRhM30/Ths/G9lio+q6VFfKxDS29T9NhO39SL//FekaLyTeAoi whtNeEJHTq5Dg1RAdPW8hR0n7JOyGfpPUulgq/XQg0qngSnj7xehCu0nnpjw3uvj aDAVD36ocUbx9J6PpCMWcEwtPgIMSN+DsdVT8yTuadfzzPW+eexTb4hNz1mNR2xb sNvxkuS5sz6M/oVs6oVS3BaAXOGJs7ZUJMIlUYNRRj+PM6rpo81OKOqzhlCQF3oO HrC7a7Wsslq7TZWcgJ5kc62choT9WVizIU9kWQOLWJxD4kHxuidryRpN6O4dtjk9 GZQi8aIr/VHGG+xu84gNef4OXCWtG4N2+R6r1pCKiHuWrCcrRg8nQlktW3F2EItu dsg/v9lSkdNDe7pBZHuzM7JGG+EY4IwpR+i0PEeQ8Q/WM0Wq5IQBARD7YgNCOrWf K6RwXhpb+JWZX4U58C2s =MU7c -----END PGP SIGNATURE----- --=-0uQkNJQzolsX4XkvG6de--
participants (2)
-
Ewoud Kohl van Wijngaarden -
Michael Scherer