Infrastructure for Jenkins

Fri Jul 13 00:50:39 UTC 2012

On 07/12/2012 12:06 PM, Karsten 'quaid' Wade wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 07/12/2012 08:51 AM, Mike Burns wrote:
>> On Thu, 2012-07-12 at 17:57 +0300, Itamar Heim wrote:
>>> On 07/12/2012 05:55 PM, Eyal Edri wrote:
>>>>
>>>> ----- Original Message -----
>>>>> From: "Karsten 'quaid' Wade" <kwade at redhat.com> To:
>>>>> infra at ovirt.org Sent: Thursday, July 12, 2012 5:31:41 PM
>>>>> Subject: Re: Infrastructure for Jenkins
>>>>>
>> On 07/12/2012 05:57 AM, Jabs, Joachim wrote:
>>>>>>> Hi Guys,
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> as previously offered:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Im willing to provide you guys infrastructure for
>>>>>>> Jenkins. Im currently trying out getting oVirt running on
>>>>>>> a cluster (we have 192 Cores and 3TB RAM aviable) for
>>>>>>> providing help with problems around ovirt that might
>>>>>>> arise.
>>>>>>>
>>>>>>> The Cluster is not commercially used (Not right now) and
>>>>>>> for me its also a good opportunity to learn about ovirt
>>>>>>> and also about hosting VM related infrastructure. This
>>>>>>> will also be a test for stability and maintainability
>>>>>>> around the hardware used.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> If you guys sum up what you need, I think I might be able
>>>>>>> to do something for you there. If you have any questions
>>>>>>> regarding the offer, just ask.
>> Hey, thanks for following. This is a great offer, +1 from me.
>>
>> Ewoud, Eyal (& others?) - what does Joachim need to do? Is this a
>> good place to start? --
>>>>> This is an excellent offer and opportunity. Since we're
>>>>> talking about VMs, we can start with basic hardware spec and
>>>>> increase it after we'll see the bottle necks in real time.
>>>>>
>>>>> I would start with 3 VMs each running 16GB RAM, 200GB DISK,
>>>>> 32 GB SWAP, each with 4/8 cores each.
>>>>>
>>>>> Itamar, mburns -> any recommendation on how many VMs per
>>>>> hardware node should we use?
>>>>>
>>>> considering the workload, i'd start with not overcomitting
>>>> resources, and later analyzing behavior
>>> Agree with Itamar.  I'd also consider having a vm or 2 that are
>>> running various target distros where we can build packages for
>>> different distros.
> +1
>
> Robert - are you interested in working on the package autobuild for
> RHEL & rebuilds?
Yes.  I see getting easy access to the auto-builds in repo's as a top 
priority.  Followed by getting the changes needed to build EL6 into 
Jenkins and get those auto-builds.  I also would like to get auto-builds 
setup for the 3.1 engine so any cheery picked updates will get built and 
made available for people without require infra admin time.  Create the 
same thing for VDSM if / when a branch gets created.

We have 2 issues that need resolved to get auto builds into repo's
1) We need additional storage on linode01 or we need to move the repo's 
to there own system.
2) We need someone with access to both system to get the scripts to move 
the data around.  I am sure I could get it done but I would need my 
access raised on linode01 and access to the Jenkins Master but this 
should be pretty easy once the storage issues is resolved.

EL6 builds the steps that we need.
1) We need to incorporate the EL6 patches so we can build the package.  
Do we do it using a EL6 specific .spec files or do we use if statements 
inside the exist spec?
2) We need to get builds running on some EL6 based hosts.  I have a few 
small hosts I could kick in but both nodes are just 8 cores / 8G of ram 
each.
3) Move the nightly over to the repo for EL6 builds.

> I figure we have the Fedora expertise already. Anyone know enough to
> get the Debian/Ubuntu packages building? (Maybe even automate
> populating an Ubuntu PPA?) Same questions for OpenSUSE.
Before we worry about getting them in a PPA we need to worry about 
building the packages.  This requires more then Just building oVirt 
packages we need to build jboss 7.11 packages and depending on the 
Debian version we might requires an newer version of PostgreSQL. There 
is a lot of packaging that needs to be done to get oVirt running on 
Debian based systems.
>
> We have a classic chicken-and-egg problem with being of interest to
> the OpenSUSE. Debian, Ubuntu, etc. communities - we need builds to
> generate the interest of people who would want to maintain the
> packages in their downstream distro repo.
Right now the packing needs is so great it is hard to get anyone 
interested into build oVirt into there distro but once you have some 
working Debian based packages then other Debian maintainers will pick 
them up and incorporate them into there respective Distro's
>>> Also, 200GB disk seems a bit excessive to me.  Since this is
>>> running on oVirt, maybe a thin provisioned disk would be a good
>>> choice.  The slaves generally don't need as much disk.
> Let's keep iterating on the content on http://ovirt.org/wiki/Jenkins,
> not just the service minimums, but cleaning the setup steps, etc.
>
> - - Karsten
> - -- 
> Karsten 'quaid' Wade, Sr. Analyst - Community Growth
> http://TheOpenSourceWay.org  .^\  http://community.redhat.com
> @quaid (identi.ca/twitter/IRC)  \v'  gpg: AD0E0C41
>
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.12 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iD8DBQFP/vYN2ZIOBq0ODEERAlwxAJ9DQ63HE7tOvVbDZdOM/b3HJ44a6QCdFsbQ
> IK74b3u5rsJkxK4kuJKm9zQ=
> =I0Jm
> -----END PGP SIGNATURE-----
> _______________________________________________
> Infra mailing list
> Infra at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/infra
Thanks
Robert