<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Wed, Jun 29, 2016 at 11:15 PM, Barak Korren <span dir="ltr"><<a href="mailto:bkorren@redhat.com" target="_blank">bkorren@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On 29 June 2016 at 21:45, Yaniv Kaul <<a href="mailto:ykaul@redhat.com">ykaul@redhat.com</a>> wrote:<br>
> On Mon, Jun 27, 2016 at 9:45 AM, Barak Korren <<a href="mailto:bkorren@redhat.com">bkorren@redhat.com</a>> wrote:<br>
>><br>
>> ><br>
>> ><br>
>> > It means that packages will be fetched EVERY time from outside, which<br>
>> > may be<br>
>> > slow(er).<br>
>> > Y.<br>
>> ><br>
>><br>
>> We can (and mostly already have) setup simple caches to prevent that.<br>
><br>
><br>
> How do you set up cache on a developer's laptop?<br>
><br>
</span>We may have been unclear in our intentions, we want to make the<br>
pre-syncing optional not remove it completely. It does make sense on<br>
the laptop (sometimes), but not so much in the CI env.<br>
<span class=""><br>
> The repo directory does not need to be cleaned every time.<br>
<br>
</span>This is an assumption that may break if we end up having any corrupt<br>
or failing packages in the cache. It also make it hard to "go back in<br>
time" if we want to test without some update.<br>
(Cleaning corrupt caches an re-running is easy in a local setting, in<br>
CI you end up dealing with angry devs getting false '-1's)<br></blockquote><div><br></div><div>True, and we don't want that. Developers have to trust the CI system.</div><div>This is an important point.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<span class=""><br>
> It can also be<br>
> resync'ed from a central repo - which still going to be faster than any<br>
> other fetching.<br>
> (hopefully sync'ed into the slave /dev/shm btw).<br>
<br>
</span>It could be faster, but could also be slower if you end up fetching<br>
more then you have to. (if engine setup fails on missing dependency,<br>
you just spent needless time fetching VDSM dpes)<br>
Also fetching by itself may not be the bottleneck in all cases, it is<br>
surely slow when fetching from PHX to TLV, but when fetching from the<br>
Squid proxy's RAM inside PHX it can actually end up being faster then<br>
copying from the local disk.<br></blockquote><div><br></div><div>I always fetch and store on /dev/shm/repostore</div><div>It's faster than anything else.</div><div><br></div><div>I did copy its content once to the disk, so when the host reboots, it rsync's this to /dev/shm/repostore , then tests begin.</div><div>That perhaps is indeed not very needed in CI.</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<span class=""><br>
>> The existing solution looks more like premature optimization gone badly<br>
>> IMO.<br>
><br>
> Try to run ovirt-system-tests, clean the repo and re-run - it's 20-30<br>
> minutes at least longer - which is far more than what it takes to run the<br>
> whole test suite.<br>
<br>
</span>I wonder how many of those minutes are spend on fetching things we<br>
actually need, and how much is spent on overhead. I suspect that<br>
without a local cache, the test run will be longer, but not as long as<br>
the pre-fetching+tests takes currently. More importantly, this may<br>
allow the CI to fail faster. I think we should at least test that.<br>
<span class=""><br>
> I completely agree the manual maintenance is an annoyance, wish we had<br>
> something in between.<br>
<br>
</span>Maybe we can take a middle ground, pre-fetch, but also enable external<br>
repos in CI (perhaps with some way to log and find out what was not<br>
pre-fetched).<br></blockquote><div><br></div><div>This is what the code is supposed to do, I suspect. reposync syncs between what you already have and what you fetch, no?</div><div>Y.</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="HOEnZb"><div class="h5"><br>
--<br>
Barak Korren<br>
<a href="mailto:bkorren@redhat.com">bkorren@redhat.com</a><br>
RHEV-CI Team<br>
</div></div></blockquote></div><br></div></div>