how to get files from jenkins.ovirt.org to www.ovirt.org

[Eyal Edri]

----- Original Message -----
> From: "Mike Burns" <mburns at redhat.com>
> To: "Eyal Edri" <eedri at redhat.com>
> Cc: "infra" <infra at ovirt.org>
> Sent: Tuesday, July 31, 2012 3:33:01 PM
> Subject: Re: how to get files from jenkins.ovirt.org to www.ovirt.org
> 
> On Tue, 2012-07-31 at 02:46 -0400, Eyal Edri wrote:
> > Hi,
> > 
> > after looking into the process and testing it via
> > jenkins.ovirt.info,
> > here is my proposal:
> > 
> > 1. Nightly job [1] will collect rpms from all ovirt projects (other
> > jenkins jobs that build rpms per commit).
> >    1.1 all files will be kept in a flat dir under $WORKSPCE/rpms
> >    1.2 using 'publish over ssh' [2] plugin the job will then copy
> > files to their destination according to their names. [3]
> >        for e.g 	'ovirt-engine-3.1.0-3.fc17.noarch.rpm' ==>
> > ovirt.org/var/www/html/releases/nightly/rpm/Fedora/17/noarch
> >                 'ovirt-engine-3.1.0-3.fc17.src.rpm' ==>
> > ovirt.org/var/www/html/releases/nightly/rpm/Fedora/17/src
> >                 'ovirt-engine-3.1.0-3.bz2' ==>
> > ovirt.org/var/www/html/releases/nightly/src'
> >    1.3 publish ssh plugin can execute a command after its done
> >    copying
> > the file, so all is left is to run 'createrepo' on the relevant
> > dirs.
> > [4]
> >  
> > 
> > 2. a separate cron cleaning script needs to run on ovirt.org to
> > delete
> > old rpms and run createrepo.
> > 
> 
> Maybe upload to a non-published directory first,
> like /home/jenkins/nightly, and have the cron job handle cleanup of
> old
> rpms, adding of new rpms, and running createrepo?

sure, i've got not problem with that.. 

there is one thing i don't understand, and sorry if not an expert in yum:

we say we want to keep history for X builds, will all versions be kept under one repository? 
so 'createrepo' will run on all versions in that directory? 

so running yum install file.verX will work even if the repo has a x+1 version?

> 
> If not, how is the cron job going to cleanup old rpms?
> 
> Uploading to a static location that isn't public (and is
> under /home/jenkins) would make [3] and [4] unnecessary.  I think I'd
> rather avoid having a non-user with sudo and r/w on a publicly served
> folder.
> 

sounds right.

> The publish ssh plugin can touch a file
> /home/jenkins/nightly/FINISHED
> when it's done, the cron job locally on ovirt.org can watch for that
> file and then process the directory structure.

good idea, we can have the cronjob monitor the dir for the FINISHED file,
and delete it when it's done.

> 
> Mike
> 
> > 
> > +1 for pushing this forward or comments?
> > 
> > Eyal.
> > 
> > 
> > [1]
> > http://jenkins.ovirt.org/view/rpms/job/publish_ovirt_rpms_nightly/,
> > this job currently works only for fedora17 rpms (that's what we
> > have
> > now).
> >     we'll need to improve it to be more generic (maybe as a matrix
> > job) or duplicate it for each opertating system.
> > [2] https://wiki.jenkins-ci.org/display/JENKINS/Publish+Over+SSH
> > +Plugin, using pub/private key with user jenkins to connect
> > [3] jenkins user will need r/q permissions to the nightly folder
> > [4] will need sudo access for it.
> > _______________________________________________
> > Infra mailing list
> > Infra at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/infra
> 
> 
>