Proposal for hosting our own set of modules

[Ewoud Kohl van Wijngaarden]

On Mon, Jun 23, 2014 at 06:02:51PM +0200, Michael Scherer wrote:
> Le lundi 23 juin 2014 à 16:42 +0200, Ewoud Kohl van Wijngaarden a
> écrit :
> > On Mon, Jun 23, 2014 at 04:30:55PM +0200, Michael Scherer wrote:
> > > Hi,
> > > 
> > > so following the discussion on puppet module, I would propose that we
> > > create a new infra-puppet-modules module on gerrit. 
> > > 
> > > The division between this module and the current one would be like
> > > this :
> > > 
> > > infra-puppet would hold the manifests, the site, the Puppetfile , etc
> > > 
> > > infra-puppet-modules would hold directories, 1 per module we develop
> > > (not the external one we use, they would still be on github or anywhere,
> > > pulled by librarian-puppet) .
> > > 
> > > Since we are using librarian-puppet
> > > ( https://github.com/rodjek/librarian-puppet ) and r10k
> > 
> > We're only using r10k, not librarian-puppet.
> 
> Indeed. 
> 
> But in the end, we still use Puppetfile nonetheless, no ?

Correct, but librarian handles dependencies where r10k ignores them.
That means you must manually specificy any dependency.

> > > ( https://github.com/adrienthebo/r10k ) , it requires use to have the
> > > modules/ directory to be managed by librarian-puppet. In turn we need to
> > > have the modules in git, we can address them by path: 
> > > 
> > >   mod "puppetlabs/apt",
> > >     :git => "git://github.com/fake/puppet-modules.git",
> > >     :path => "modules/apt"
> > > 
> > > 
> > > This wouldn't requires to change much, besides adding the module to Puppetfile and creating a git repository.
> > > 
> > > If no one disagree, I will request the git repository.
> > > 
> > > ( in the mean time, i did create a sample awstats repository for stats.ovirt.org, so we can have something to push )
> > 
> > Unless we plan to make them reusable for other projects, I don't see the
> > benefit. If we do plan to make them reusable, we should IMHO also
> > publish them on the forge.
> > 
> > Another potential issue is how we decide when to deploy. We could have a
> > specific commit ID and update our Puppetfile every time, but again,
> > little benefit over having them in one tree.
> 
> I was under the impression you could just give a tag and so use master ?
> ( and using branch for development )

You can, but how do we know we want to update and to which version?
branches can easily break compatibility and it will bite you at some
point.

> > In case you're unaware, we already load modules/* (which is managed by
> > r10k) and site/* (tracked in git). That means we can host our modules in
> > site and spit off when they are reusable.
> 
> It was not very obvious that site/* was for modules, indeed :)
> 
> But my fault, I should also have read the doc in the git repository who
> clearly say that.
> 
> I must also say that using r10k for different environments seems a bit
> overkill, as we seem to only have 1 single environment anyway ( but I am
> not using r10k usually, as I do not have enough systems for that and
> write all stuff myself ). 

The alternative is either include them or git submodules. I'm not a fan
of the former and I've seen people have issues with the latter where
people accidentally commit an older version. It's also hard to spot in
reviews because you only see two git commit IDs with no idea if it's an
upgrade or downgrade. That's why a Puppetfile-based solution was chosen.

Minor nitpick: technically we have 2 environments and we should make an
effort to remove master. This is done by changing the default branch to
production and then remove master. The only way I know is logging into
gerrit through SSH and changing infra-puppet.git/HEAD to point to
production instead of master. Then you can use the gerrit UI to delete
the master branch.

> > And if we do plan on creating modules repos, I'd be in favor of having
> > one git repo per puppet module since that is what most people would
> > expect.
> 
> One git repo per module is a bit annoying when we are planning to write
> a lot of modules. But I guess it all depend on the time it take to
> create one. I would definitely prefer a approach of 1 big git as long as
> we are "growing" fast and maybe split later, since it permit faster
> growth ?

That's exactly what I was thinking.