[lago-devel] [ovirt-devel] ovirt tests failing on missing libxml2-python

[Yaniv Kaul]

On Wed, Jun 29, 2016 at 11:15 PM, Barak Korren <bkorren at redhat.com> wrote:

> On 29 June 2016 at 21:45, Yaniv Kaul <ykaul at redhat.com> wrote:
> > On Mon, Jun 27, 2016 at 9:45 AM, Barak Korren <bkorren at redhat.com>
> wrote:
> >>
> >> >
> >> >
> >> > It means that packages will be fetched EVERY time from outside, which
> >> > may be
> >> > slow(er).
> >> > Y.
> >> >
> >>
> >> We can (and mostly already have) setup simple caches to prevent that.
> >
> >
> > How do you set up cache on a developer's laptop?
> >
> We may have been unclear in our intentions, we want to make the
> pre-syncing optional not remove it completely. It does make sense on
> the laptop (sometimes), but not so much in the CI env.
>
> > The repo directory does not need to be cleaned every time.
>
> This is an assumption that may break if we end up having any corrupt
> or failing packages in the cache. It also make it hard to "go back in
> time" if we want to test without some update.
> (Cleaning corrupt caches an re-running is easy in a local setting, in
> CI you end up dealing with angry devs getting false '-1's)
>

True, and we don't want that. Developers have to trust the CI system.
This is an important point.


>
> > It can also be
> > resync'ed from a central repo - which still going to be faster than any
> > other fetching.
> > (hopefully sync'ed into the slave /dev/shm btw).
>
> It could be faster, but could also be slower if you end up fetching
> more then you have to. (if engine setup fails on missing dependency,
> you just spent needless time fetching VDSM dpes)
> Also fetching by itself may not be the bottleneck in all cases, it is
> surely slow when fetching from PHX to TLV, but when fetching from the
> Squid proxy's RAM inside PHX it can actually end up being faster then
> copying from the local disk.
>

I always fetch and store on /dev/shm/repostore
It's faster than anything else.

I did copy its content once to the disk, so when the host reboots, it
rsync's this to /dev/shm/repostore , then tests begin.
That perhaps is indeed not very needed in CI.


> >> The existing solution looks more like premature optimization gone badly
> >> IMO.
> >
> > Try to run ovirt-system-tests, clean the repo and re-run - it's 20-30
> > minutes at least longer - which is far more than what it takes to run the
> > whole test suite.
>
> I wonder how many of those minutes are spend on fetching things we
> actually need, and how much is spent on overhead. I suspect that
> without a local cache, the test run will be longer, but not as long as
> the pre-fetching+tests takes currently. More importantly, this may
> allow the CI to fail faster. I think we should at least test that.
>
> > I completely agree the manual maintenance is an annoyance, wish we had
> > something in between.
>
> Maybe we can take a middle ground, pre-fetch, but also enable external
> repos in CI (perhaps with some way to log and find out what was not
> pre-fetched).
>

This is what the code is supposed to do, I suspect. reposync syncs between
what you already have and what you fetch, no?
Y.


> --
> Barak Korren
> bkorren at redhat.com
> RHEV-CI Team
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/lago-devel/attachments/20160702/350ced6c/attachment-0001.html>