<p dir="ltr"></p>
<p dir="ltr">On Nov 20, 2016 6:30 PM, "Eyal Edri" <<a href="mailto:eedri@redhat.com">eedri@redhat.com</a>> wrote:<br>
><br>
> Renaming title and adding devel.<br>
><br>
> On Sun, Nov 20, 2016 at 2:36 PM, Piotr Kliczewski <<a href="mailto:pkliczew@redhat.com">pkliczew@redhat.com</a>> wrote:<br>
>><br>
>> The last failure seems to be storage related.<br>
>><br>
>> @Nir please take a look.<br>
>><br>
>> Here is engine side error:<br>
>><br>
>> 2016-11-20 05:54:59,605 DEBUG [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (default task-5) [59fc0074] Exception: org.ovirt.engine.core.vdsbroker.irsbroker.IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: u'spUUID=1ca141f1-b64d-4a52-8861-05c7de2a72b2, msdUUID=7d4bf750-4fb8-463f-bbb0-92156c47306e'<br>
>><br>
>> and here is vdsm:<br>
>><br>
>> jsonrpc.Executor/5::ERROR::2016-11-20 05:54:56,331::multipath::95::Storage.Multipath::(resize_devices) Could not resize device 360014052749733c7b8248628637b990f<br>
>> Traceback (most recent call last):<br>
>> File "/usr/share/vdsm/storage/multipath.py", line 93, in resize_devices<br>
>> _resize_if_needed(guid)<br>
>> File "/usr/share/vdsm/storage/multipath.py", line 101, in _resize_if_needed<br>
>> for slave in devicemapper.getSlaves(name)]<br>
>> File "/usr/share/vdsm/storage/multipath.py", line 158, in getDeviceSize<br>
>> bs, phyBs = getDeviceBlockSizes(devName)<br>
>> File "/usr/share/vdsm/storage/multipath.py", line 150, in getDeviceBlockSizes<br>
>> "queue", "logical_block_size")).read())<br>
>> IOError: [Errno 2] No such file or directory: '/sys/block/sdb/queue/logical_block_size'<br>
><br>
><br>
><br>
> We now see a different error in master [1], which also indicates the hosts are in a problematic state: ( failing 'assign_hosts_network_label' test )<br>
><br>
> status: 409<br>
> reason: Conflict<br>
> detail: Cannot add Label. Operation can be performed only when Host status is Maintenance, Up, NonOperational.</p>
<p dir="ltr">I believe you are mixing unrelated issues. <br>
I've seen this once and I have an unproven theory :<br>
The previous suite restarts Engine after LDAP configuration then performs its test, which is quite short (24 seconds on my poor laptop + few additional secs between suites). <br>
I'm not convinced it is enough time for hosts status to be updated in Engine back to UP state. </p>
<p dir="ltr">Y. </p>
<p dir="ltr">> -------------------- >> begin captured logging << --------------------<br>
><br>
><br>
> [1] <a href="http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/3506/testReport/junit/(root)/006_network_by_label/assign_hosts_network_label/">http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/3506/testReport/junit/(root)/006_network_by_label/assign_hosts_network_label/</a><br>
><br>
> <br>
>><br>
>><br>
>><br>
>> On Sun, Nov 20, 2016 at 12:50 PM, Eyal Edri <<a href="mailto:eedri@redhat.com">eedri@redhat.com</a>> wrote:<br>
>>><br>
>>><br>
>>><br>
>>> On Sun, Nov 20, 2016 at 1:42 PM, Yaniv Kaul <<a href="mailto:ykaul@redhat.com">ykaul@redhat.com</a>> wrote:<br>
>>>><br>
>>>><br>
>>>><br>
>>>> On Sun, Nov 20, 2016 at 1:30 PM, Yaniv Kaul <<a href="mailto:ykaul@redhat.com">ykaul@redhat.com</a>> wrote:<br>
>>>>><br>
>>>>><br>
>>>>><br>
>>>>> On Sun, Nov 20, 2016 at 1:18 PM, Eyal Edri <<a href="mailto:eedri@redhat.com">eedri@redhat.com</a>> wrote:<br>
>>>>>><br>
>>>>>> the test fails to run VM because no hosts are in UP state(?) [1], not sure it is related to the triggering patch[2]<br>
>>>>>><br>
>>>>>> status: 400<br>
>>>>>> reason: Bad Request<br>
>>>>>> detail: There are no hosts to use. Check that the cluster contains at least one host in Up state.<br>
>>>>>><br>
>>>>>> Thoughts? Shouldn't we fail the test earlier we hosts are not UP? <br>
>>>>><br>
>>>>><br>
>>>>> Yes. It's more likely that we are picking the wrong host or so, but who knows - where are the engine and VDSM logs?<br>
>>>><br>
>>>><br>
>>>> A simple grep on the engine.log[1] finds serveral unrelated issues I'm not sure are reported, it's despairing to even begin...<br>
>>>> That being said, I don't see the issue there. We may need better logging on the API level, to see what is being sent. Is it consistent?<br>
>>><br>
>>><br>
>>> Just failed now the first time, I didn't see it before.<br>
>>> <br>
>>>><br>
>>>> Y.<br>
>>>><br>
>>>><br>
>>>> [1] <a href="http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_4.0/3015/artifact/exported-artifacts/basic_suite_4.0.sh-el7/exported-artifacts/test_logs/basic-suite-4.0/post-004_basic_sanity.py/lago-basic-suite-4-0-engine/_var_log_ovirt-engine/engine.log">http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_4.0/3015/artifact/exported-artifacts/basic_suite_4.0.sh-el7/exported-artifacts/test_logs/basic-suite-4.0/post-004_basic_sanity.py/lago-basic-suite-4-0-engine/_var_log_ovirt-engine/engine.log</a> <br>
>>>>><br>
>>>>> Y.<br>
>>>>> <br>
>>>>>><br>
>>>>>><br>
>>>>>><br>
>>>>>> [1] <a href="http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_4.0/3015/testReport/junit/(root)/004_basic_sanity/vm_run/">http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_4.0/3015/testReport/junit/(root)/004_basic_sanity/vm_run/</a><br>
>>>>>> [2] <a href="http://jenkins.ovirt.org/job/ovirt-engine_4.0_build-artifacts-el7-x86_64/1535/changes#detail">http://jenkins.ovirt.org/job/ovirt-engine_4.0_build-artifacts-el7-x86_64/1535/changes#detail</a><br>
>>>>>><br>
>>>>>><br>
>>>>>><br>
>>>>>> On Sun, Nov 20, 2016 at 1:00 PM, <<a href="mailto:jenkins@jenkins.phx.ovirt.org">jenkins@jenkins.phx.ovirt.org</a>> wrote:<br>
>>>>>>><br>
>>>>>>> Build: <a href="http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_4.0/3015/">http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_4.0/3015/</a>,<br>
>>>>>>> Build Number: 3015,<br>
>>>>>>> Build Status: FAILURE<br>
>>>>>>> _______________________________________________<br>
>>>>>>> Infra mailing list<br>
>>>>>>> <a href="mailto:Infra@ovirt.org">Infra@ovirt.org</a><br>
>>>>>>> <a href="http://lists.ovirt.org/mailman/listinfo/infra">http://lists.ovirt.org/mailman/listinfo/infra</a><br>
>>>>>>><br>
>>>>>><br>
>>>>>><br>
>>>>>><br>
>>>>>> -- <br>
>>>>>> Eyal Edri<br>
>>>>>> Associate Manager<br>
>>>>>> RHV DevOps<br>
>>>>>> EMEA ENG Virtualization R&D<br>
>>>>>> Red Hat Israel<br>
>>>>>><br>
>>>>>> phone: +972-9-7692018<br>
>>>>>> irc: eedri (on #tlv #rhev-dev #rhev-integ)<br>
>>>>><br>
>>>>><br>
>>>><br>
>>><br>
>>><br>
>>><br>
>>> -- <br>
>>> Eyal Edri<br>
>>> Associate Manager<br>
>>> RHV DevOps<br>
>>> EMEA ENG Virtualization R&D<br>
>>> Red Hat Israel<br>
>>><br>
>>> phone: +972-9-7692018<br>
>>> irc: eedri (on #tlv #rhev-dev #rhev-integ)<br>
>><br>
>><br>
><br>
><br>
><br>
> -- <br>
> Eyal Edri<br>
> Associate Manager<br>
> RHV DevOps<br>
> EMEA ENG Virtualization R&D<br>
> Red Hat Israel<br>
><br>
> phone: +972-9-7692018<br>
> irc: eedri (on #tlv #rhev-dev #rhev-integ)<br></p>