[ovirt-devel] Vdsm tests are 4X times faster on travis

[Barak Korren]

On 7 December 2016 at 14:15, Nir Soffer <nsoffer at redhat.com> wrote:
> On Mon, Dec 5, 2016 at 10:11 AM, Barak Korren <bkorren at redhat.com> wrote:
>> On 5 December 2016 at 10:07, Nir Soffer <nsoffer at redhat.com> wrote:
>>>
>>> 20 seconds setup sounds great.
>>>
>>> Can we try the patches with vdsm builds?
>>
>> We'll probably merge today, if not, I'll manually cherry-pick this for
>> the vdsm jobs.
>
> With all patches merged, builds take now:
>
> http://jenkins.ovirt.org/job/vdsm_master_check-patch-el7-x86_64/4373/console
> 00:07:13.065 Finished: SUCCESS
>
> http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc24-x86_64/5991/console
> 00:08:19.035 Finished: SUCCESS
>
> Last week it was 19-20 mintues, great improvement!



>
> What is missing now small improvement in the logs - add log for each part of the
> build with the time it took.
>
> - setup
> - run script
> - cleanup
>
> The run script part is something that only project maintainers can optimize, the
> rest can be optimized only by ci maintainers.

WRT to times - mock_runner.sh does print those out to the output page
(surrounded by many asterisks and other symbols...)

WRT separation of logs - we already mostly have that, you can see
individual step logs in the job artifacts. For example for one of the
jobs above:

http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc24-x86_64/5991/artifact/exported-artifacts/logs/mocker-fedora-24-x86_64.fc24.check-patch.sh/check-patch.sh.log

But yeah, we could probably make the output in the main Jenkins output
page better. That would require potentially breaking changes to
"mock_runner.sh", so I'd rather focus on ultimately replacing it...

We do already have https://ovirt-jira.atlassian.net/browse/OVIRT-682
for discussing this issue.

> I think we should have metrics collection system and keep these times
> so we can detect regressions and improvement easily. But the first step
> is measuring the time.

Jenkins (partially) gives us that, see here:
http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc24-x86_64/buildTimeTrend

I completely agree that the UX here is not as good as it can and
should be, and we do have plans to make it A LOT better, please bare
with us in the meantime...

-- 
Barak Korren
bkorren at redhat.com
RHCE, RHCi, RHV-DevOps Team
https://ifireball.wordpress.com/