Re: [ovirt-devel] [CQ ovirt master] [ovirt-engine] - not passing for 10 days

On Wed, Nov 14, 2018 at 12:42 PM Dominik Holler <dholler(a)redhat.com&gt; wrote: Dominik, thanks for the research and for Martin's and your reverts/fixes. Finally Engine passes OST https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11153/ and QE can expect a build tomorrow, after 2 weeks of droughts. > > > > > > > > > > > > >>> 2) the patch is wrong too, as in HandleVdsCpuFlagsOrClusterChangedCommand > > >>>> it just goes ahead and sets the cluster cpu to whatever the host reported > > >>>> regardless if it is valid or not. Steven, please fix that (line 96 in > > >>>> backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/HandleVdsCpuFlagsOrClusterChangedCommand.java). > > >>>> It needs to pass the validation or we need some other solution. > > >>>> 3) regardless, we should make 4.3 work too , I tried to play with it a bit > > >>>> in https://gerrit.ovirt.org/#/c/95407/, let’s see… > > >>>> > > >>>> Thanks, > > >>>> michal > > >>>> > > >>>> > > >>>> > > >>>> On Mon, Nov 12, 2018 at 3:58 PM Dominik Holler <dholler(a)redhat.com&gt; wrote: > > >>>> > > >>>> On Mon, 12 Nov 2018 13:45:54 +0100 > > >>>> Martin Perina <mperina(a)redhat.com&gt; wrote: > > >>>> > > >>>> On Mon, Nov 12, 2018 at 12:58 PM Dominik Holler <dholler(a)redhat.com&gt; > > >>>> > > >>>> wrote: > > >>>> > > >>>> > > >>>> On Mon, 12 Nov 2018 12:29:17 +0100 > > >>>> Martin Perina <mperina(a)redhat.com&gt; wrote: > > >>>> > > >>>> On Mon, Nov 12, 2018 at 12:20 PM Dafna Ron <dron(a)redhat.com&gt; wrote: > > >>>> > > >>>> There are currently two issues failing ovirt-engine on CQ ovirt > > >>>> > > >>>> master: > > >>>> > > >>>> > > >>>> 1. edit vm pool is causing failure in different tests. it has a > > >>>> > > >>>> patch > > >>>> > > >>>> *waiting > > >>>> > > >>>> to be merged*: https://gerrit.ovirt.org/#/c/95354/ > > >>>> > > >>>> > > >>>> Merged > > >>>> > > >>>> > > >>>> 2. we have a failure in upgrade suite as well to run vm but this > > >>>> > > >>>> seems > > >>>> > > >>>> to > > >>>> > > >>>> be related to the tests as well: > > >>>> 2018-11-12 05:41:07,831-05 WARN > > >>>> [org.ovirt.engine.core.bll.validator.VirtIoRngValidator] > > >>>> > > >>>> (default > > >>>> > > >>>> task-1) > > >>>> > > >>>> [] Random number source URANDOM is not supported in cluster > > >>>> > > >>>> 'test-cluster' > > >>>> > > >>>> compatibility version 4.0. > > >>>> > > >>>> here is the full error from the upgrade suite failure in run vm: > > >>>> https://pastebin.com/XLHtWGGx > > >>>> > > >>>> Here is the latest failure: > > >>>> > > >>>> > > >>>> > > >>>> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_cha... > > >>>> > > >>>> > > >>>> > > >>>> I will try to take a look later today > > >>>> > > >>>> > > >>>> I have the idea that this might be related to > > >>>> https://gerrit.ovirt.org/#/c/95377/ , and I check in > > >>>> > > >>>> > > >>>> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-te... > > >>>> > > >>>> > > >>>> , but I have to stop now, if not solved I can go on later today. > > >>>> > > >>>> > > >>>> OK, both CI and above manual OST job went fine, so I've just merged the > > >>>> revert patch. I will take a look at it later in detail, we should > > >>>> > > >>>> really be > > >>>> > > >>>> testing 4.3 on master and not 4.2 > > >>>> > > >>>> > > >>>> Ack. > > >>>> > > >>>> Now > > >>>> > > >>>> > > >>>> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_cha... > > >>>> is failing on > > >>>> File > > >>>> > > >>>> "/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", > > >>>> line 698, in run_vms > > >>>> api.vms.get(VM0_NAME).start(start_params) > > >>>> status: 400 > > >>>> reason: Bad Request > > >>>> > > >>>> 2018-11-12 10:06:30,722-05 INFO > > >>>> [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) > > >>>> [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Candidate host > > >>>> 'lago-basic-suite-master-host-1' ('dbfe1b0c-f940-4dba-8fb1-0cfe5ca7ddfc') > > >>>> was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' > > >>>> (correlation id: b8d11cb0-5be9-4b7e-b45a-c95fa1f18681) > > >>>> 2018-11-12 10:06:30,722-05 INFO > > >>>> [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) > > >>>> [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Candidate host > > >>>> 'lago-basic-suite-master-host-0' ('e83a63ca-381e-40db-acb2-65a3e7953e11') > > >>>> was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' > > >>>> (correlation id: b8d11cb0-5be9-4b7e-b45a-c95fa1f18681) > > >>>> 2018-11-12 10:06:30,723-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] > > >>>> (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Can't find VDS to > > >>>> run the VM '57a66eff-8cbf-4643-b045-43d4dda80c66' on, so this VM will not > > >>>> be run. > > >>>> > > >>>> Is this related to > > >>>> https://gerrit.ovirt.org/#/c/95310/ > > >>>> ? > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> Thanks, > > >>>> Dafna > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> On Mon, Nov 12, 2018 at 9:23 AM Dominik Holler < > > >>>> > > >>>> dholler(a)redhat.com&gt; > > >>>> > > >>>> wrote: > > >>>> > > >>>> > > >>>> On Sun, 11 Nov 2018 19:04:40 +0200 > > >>>> Dan Kenigsberg <danken(a)redhat.com&gt; wrote: > > >>>> > > >>>> On Sun, Nov 11, 2018 at 5:27 PM Eyal Edri <eedri(a)redhat.com&gt; > > >>>> > > >>>> wrote: > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> On Sun, Nov 11, 2018 at 5:24 PM Eyal Edri <eedri(a)redhat.com&gt; > > >>>> > > >>>> > > >>>> wrote: > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> On Sun, Nov 11, 2018 at 5:20 PM Dan Kenigsberg < > > >>>> > > >>>> danken(a)redhat.com&gt; > > >>>> > > >>>> wrote: > > >>>> > > >>>> > > >>>> On Sun, Nov 11, 2018 at 4:36 PM Ehud Yonasi < > > >>>> > > >>>> eyonasi(a)redhat.com&gt; > > >>>> > > >>>> > > >>>> wrote: > > >>>> > > >>>> > > >>>> Hey, > > >>>> I've seen that CQ Master is not passing ovirt-engine for > > >>>> > > >>>> 10 > > >>>> > > >>>> days > > >>>> > > >>>> and fails on test suite called restore_vm0_networking > > >>>> > > >>>> here's a snap error regarding it: > > >>>> > > >>>> https://pastebin.com/7msEYqKT > > >>>> > > >>>> Link to a sample job with the error: > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artif... > > >>>> > > >>>> > > >>>> > > >>>> I cannot follow this link because I'm 4 minutes too late > > >>>> > > >>>> jenkins.ovirt.org uses an invalid security certificate. > > >>>> > > >>>> The > > >>>> > > >>>> certificate expired on November 11, 2018, 5:13:25 PM > > >>>> > > >>>> GMT+2. The > > >>>> > > >>>> current time is November 11, 2018, 5:17 PM. > > >>>> > > >>>> > > >>>> > > >>>> Yes, we're looking into that issue now. > > >>>> > > >>>> > > >>>> > > >>>> Fixed, you should be able to access it now. > > >>>> > > >>>> > > >>>> OST fails during restore_vm0_networking in line 101 of > > >>>> 004_basic_sanity.py while comparing > > >>>> vm_service.get().status == state > > >>>> > > >>>> It seems that instead of reporting back the VM status, Engine > > >>>> > > >>>> set > > >>>> > > >>>> garbage > > >>>> > > >>>> "The response content type 'text/html; charset=iso-8859-1' > > >>>> > > >>>> isn't the > > >>>> > > >>>> expected XML" > > >>>> > > >>>> > > >>>> The relevant line in > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/arti... > > >>>> > > >>>> seems to be > > >>>> 192.168.201.1 - - [11/Nov/2018:04:27:43 -0500] "GET > > >>>> /ovirt-engine/api/vms/26088164-d1a0-4254-a377-5d3c242c8105 > > >>>> > > >>>> HTTP/1.1" > > >>>> > > >>>> 503 299 > > >>>> > > >>>> and I guess the 503 error message is sent in HTML instead of XML. > > >>>> > > >>>> If I run manually > > >>>> https://gerrit.ovirt.org/#/c/95354/ > > >>>> with latest build of engine-master > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_... > > >>>> > > >>>> basic suite seems to be happy: > > >>>> https://jenkins.ovirt.org/view/oVirt system > > >>>> tests/job/ovirt-system-tests_manual/3484/ > > >>>> > > >>>> > > >>>> I do not know what could cause that, and engine.log does not > > >>>> > > >>>> mention > > >>>> > > >>>> it. But it seems like a problem in engine API hence +Martin > > >>>> > > >>>> Perina > > >>>> > > >>>> and > > >>>> > > >>>> +Ondra Machacek . > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> Can some1 have a look at it and help to resolve the > > >>>> > > >>>> issue? > > >>>> > > >>>> > > >>>> > > >>>> _______________________________________________ > > >>>> Infra mailing list -- infra(a)ovirt.org > > >>>> To unsubscribe send an email to infra-leave(a)ovirt.org > > >>>> Privacy Statement: > > >>>> > > >>>> https://www.ovirt.org/site/privacy-policy/ > > >>>> > > >>>> oVirt Code of Conduct: > > >>>> > > >>>> https://www.ovirt.org/community/about/community-guidelines/ > > >>>> > > >>>> List Archives: > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> https://lists.ovirt.org/archives/list/infra@ovirt.org/message/ZQAYWTLZJKG... > > >>>> > > >>>> > > >>>> _______________________________________________ > > >>>> Devel mailing list -- devel(a)ovirt.org > > >>>> To unsubscribe send an email to devel-leave(a)ovirt.org > > >>>> Privacy Statement: > > >>>> > > >>>> https://www.ovirt.org/site/privacy-policy/ > > >>>> > > >>>> oVirt Code of Conduct: > > >>>> > > >>>> https://www.ovirt.org/community/about/community-guidelines/ > > >>>> > > >>>> List Archives: > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/R5LOJH73XCL... > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> -- > > >>>> > > >>>> Eyal edri > > >>>> > > >>>> > > >>>> MANAGER > > >>>> > > >>>> RHV/CNV DevOps > > >>>> > > >>>> EMEA VIRTUALIZATION R&D > > >>>> > > >>>> > > >>>> Red Hat EMEA > > >>>> > > >>>> TRIED. TESTED. TRUSTED. > > >>>> phone: +972-9-7692018 > > >>>> irc: eedri (on #tlv #rhev-dev #rhev-integ) > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> -- > > >>>> > > >>>> Eyal edri > > >>>> > > >>>> > > >>>> MANAGER > > >>>> > > >>>> RHV/CNV DevOps > > >>>> > > >>>> EMEA VIRTUALIZATION R&D > > >>>> > > >>>> > > >>>> Red Hat EMEA > > >>>> > > >>>> TRIED. TESTED. TRUSTED. > > >>>> phone: +972-9-7692018 > > >>>> irc: eedri (on #tlv #rhev-dev #rhev-integ) > > >>>> > > >>>> _______________________________________________ > > >>>> Devel mailing list -- devel(a)ovirt.org > > >>>> To unsubscribe send an email to devel-leave(a)ovirt.org > > >>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > > >>>> oVirt Code of Conduct: > > >>>> > > >>>> https://www.ovirt.org/community/about/community-guidelines/ > > >>>> > > >>>> List Archives: > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/DA6Q5RE5JO3... > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>> > > >> > > > > > > _______________________________________________ > Infra mailing list -- infra(a)ovirt.org > To unsubscribe send an email to infra-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/6PC2MYTQXFL...