[ovirt-devel] [ OST Failure Report ] [ oVirt master ] [ 03-07-2017 ] [ 006_migrations.migrate_vm ]

Wed Jul 5 07:30:04 UTC 2017

On Wed, Jul 5, 2017 at 10:13 AM Eyal Edri <eedri at redhat.com> wrote:

> On Wed, Jul 5, 2017 at 10:02 AM, Yaniv Kaul <ykaul at redhat.com> wrote:
>
>>
>>
>> On Wed, Jul 5, 2017 at 9:39 AM, Irit Goihman <igoihman at redhat.com> wrote:
>>
>>> https://gerrit.ovirt.org/#/c/78536 was indeed the offending patch, the
>>> change was reverted and OST should pass now.
>>>
>>
>> - Do we know why?
>> - O-S-T seems to be a great tool for finding JSON-RPC/STOMP issues. I
>> suggest running it on every change related to these.
>>
>
> In addition, if we have collectd installed now, can't we add a test that
> will check if CPU/Memory consumption spike above the normal and fail before
> it reach actions like vm run/migration?
>

on my list.

>
>
>>
>> Y.
>>
>>
>>>
>>> On Tue, Jul 4, 2017 at 5:19 PM, Eyal Edri <eedri at redhat.com> wrote:
>>>
>>>> Guys,
>>>>
>>>> I think we proved which vdsm works ( git sha1: 28558d7 ) and what was
>>>> the changelog since until the point it fails, so you have the list of
>>>> changes and steps to reproduce locally.
>>>> Which again, is reproducible on CI and locally, so please go over the
>>>> changes done or reproduce the problem locally and see the issue on a live
>>>> system.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Jul 4, 2017 at 5:07 PM, Piotr Kliczewski <
>>>> piotr.kliczewski at gmail.com> wrote:
>>>>
>>>>> Looking at the last experimental job the reason of the failure is:
>>>>>
>>>>> 2017-07-04 09:39:10,491-04 ERROR
>>>>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default
>>>>> task-18) [] Operation Failed: [Cannot run VM. There is no host that
>>>>> satisfies current scheduling constraints. See below for details:, The host
>>>>> lago-basic-suite-master-host0 did not satisfy internal filter CPUOverloaded
>>>>> because its CPU is too loaded.]
>>>>>
>>>>> Do we think that vdsm increased its cpu consumption recently?
>>>>>
>>>>> On Tue, Jul 4, 2017 at 3:54 PM, Irit Goihman <igoihman at redhat.com>
>>>>> wrote:
>>>>>
>>>>>> I've checked vdsm logs and couldn't find anything related to my
>>>>>> change.
>>>>>> I'll run OST without my changes and see if it runs successfully.
>>>>>>
>>>>>> On Tue, Jul 4, 2017 at 4:49 PM, Eyal Edri <eedri at redhat.com> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Jul 4, 2017 at 4:29 PM, Dafna Ron <dron at redhat.com> wrote:
>>>>>>>
>>>>>>>> This issue is reproduced locally as well.
>>>>>>>>
>>>>>>>> you can run the following to reproduce locally
>>>>>>>>
>>>>>>>> ./run_suite.sh -s
>>>>>>>> http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-el7-x86_64/2694/
>>>>>>>> basic-suite-master
>>>>>>>>
>>>>>>>> you will have the environment still running which would allow to
>>>>>>>> view the live environment.
>>>>>>>> if you have any issues please ping me and I will help any way I
>>>>>>>> can.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Dafna
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> Here is the list of changes done from the vdsm that is verified ( in
>>>>>>> tested now ) to HEAD:
>>>>>>>
>>>>>>> * 74b2276 - (HEAD -> master, origin/master, origin/HEAD) stomp: add
>>>>>>> integration tests for client reconnect (6 hours ago) Irit Goihman <
>>>>>>> igoihman at redhat.com>
>>>>>>> * 2a2f6cd - stomp: set default heartbeat values and add grace period
>>>>>>> (6 hours ago) Irit Goihman <igoihman at redhat.com>
>>>>>>> * 56c306a - tests: Make random uuid test repeatable (17 hours ago)
>>>>>>> Nir Soffer <nsoffer at redhat.com>
>>>>>>> * 864d4e3 - python3: Fix UUID packing/unpacking on python 3 (17
>>>>>>> hours ago) Nir Soffer <nsoffer at redhat.com>
>>>>>>> * 4ac4221 - python3: Improve uuid packing tests (17 hours ago) Nir
>>>>>>> Soffer <nsoffer at redhat.com>
>>>>>>> * d264c8d - python3: Run misc_test in python 3 (17 hours ago) Nir
>>>>>>> Soffer <nsoffer at redhat.com>
>>>>>>> * f923b0b - storage: Added disk type change logging (18 hours ago)
>>>>>>> Denis Chaplygin <dchaplyg at redhat.com>
>>>>>>> * f1d54a1 - net: Unneeded newline is added when updating only the
>>>>>>> mtu (25 hours ago) Edward Haas <edwardh at redhat.com>
>>>>>>> * 9056d61 - virt: metadata: remove dead code (26 hours ago)
>>>>>>> Francesco Romani <fromani at redhat.com>
>>>>>>> * 08982b4 - virt: network: use core.find_device_guest_address (31
>>>>>>> hours ago) Francesco Romani <fromani at redhat.com>
>>>>>>> * 62e2bc5 - python3: Run qcow2_test on python 3 (2 days ago) Nir
>>>>>>> Soffer <nsoffer at redhat.com>
>>>>>>> * 42f5efb - stomp: implement client reconnect (2 days ago) Irit
>>>>>>> Goihman <igoihman at redhat.com>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 07/04/2017 01:35 PM, Barak Korren wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 4 July 2017 at 14:32, Irit Goihman <igoihman at redhat.com> wrote:
>>>>>>>>
>>>>>>>>> https://gerrit.ovirt.org/#/c/78536 broke network functional tests
>>>>>>>>> but a fix was merged today: https://gerrit.ovirt.org/#/c/78925/
>>>>>>>>>
>>>>>>>>> I tried to run OST with my fix yesterday and still encountered the
>>>>>>>>> same failures.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Here is a reproducer of the failure with the fix patch:
>>>>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1061/
>>>>>>>>
>>>>>>>> So that was not it probably...
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Barak Korren
>>>>>>>> RHV DevOps team , RHCE, RHCi
>>>>>>>> Red Hat EMEA
>>>>>>>> redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Devel mailing listDevel at ovirt.orghttp://lists.ovirt.org/mailman/listinfo/devel
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Devel mailing list
>>>>>>>> Devel at ovirt.org
>>>>>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Eyal edri
>>>>>>>
>>>>>>>
>>>>>>> ASSOCIATE MANAGER
>>>>>>>
>>>>>>> RHV DevOps
>>>>>>>
>>>>>>> EMEA VIRTUALIZATION R&D
>>>>>>>
>>>>>>>
>>>>>>> Red Hat EMEA <https://www.redhat.com/>
>>>>>>> <https://red.ht/sig> TRIED. TESTED. TRUSTED.
>>>>>>> <https://redhat.com/trusted>
>>>>>>> phone: +972-9-7692018 <+972%209-769-2018>
>>>>>>> irc: eedri (on #tlv #rhev-dev #rhev-integ)
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> IRIT GOIHMAN
>>>>>>
>>>>>> SOFTWARE ENGINEER
>>>>>>
>>>>>> EMEA VIRTUALIZATION R&D
>>>>>>
>>>>>> Red Hat EMEA <https://www.redhat.com/>
>>>>>>
>>>>>> <https://red.ht/sig>
>>>>>> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
>>>>>> @redhatnews <https://twitter.com/redhatnews>   Red Hat
>>>>>> <https://www.linkedin.com/company/red-hat>   Red Hat
>>>>>> <https://www.facebook.com/RedHatInc>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Devel mailing list
>>>>>> Devel at ovirt.org
>>>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Eyal edri
>>>>
>>>>
>>>> ASSOCIATE MANAGER
>>>>
>>>> RHV DevOps
>>>>
>>>> EMEA VIRTUALIZATION R&D
>>>>
>>>>
>>>> Red Hat EMEA <https://www.redhat.com/>
>>>> <https://red.ht/sig> TRIED. TESTED. TRUSTED.
>>>> <https://redhat.com/trusted>
>>>> phone: +972-9-7692018 <+972%209-769-2018>
>>>> irc: eedri (on #tlv #rhev-dev #rhev-integ)
>>>>
>>>
>>>
>>>
>>> --
>>>
>>> IRIT GOIHMAN
>>>
>>> SOFTWARE ENGINEER
>>>
>>> EMEA VIRTUALIZATION R&D
>>>
>>> Red Hat EMEA <https://www.redhat.com/>
>>>
>>> <https://red.ht/sig>
>>> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
>>> @redhatnews <https://twitter.com/redhatnews>   Red Hat
>>> <https://www.linkedin.com/company/red-hat>   Red Hat
>>> <https://www.facebook.com/RedHatInc>
>>>
>>> _______________________________________________
>>> Devel mailing list
>>> Devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>
>>
>>
>
>
> --
>
> Eyal edri
>
>
> ASSOCIATE MANAGER
>
> RHV DevOps
>
> EMEA VIRTUALIZATION R&D
>
>
> Red Hat EMEA <https://www.redhat.com/>
> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
> phone: +972-9-7692018 <+972%209-769-2018>
> irc: eedri (on #tlv #rhev-dev #rhev-integ)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/devel/attachments/20170705/b5a053fa/attachment-0001.html>