[CQ ovirt master] [ovirt-engine] - not passing for 10 days
Hey, I've seen that CQ Master is not passing ovirt-engine for 10 days and fails on test suite called restore_vm0_networking here's a snap error regarding it: https://pastebin.com/7msEYqKT Link to a sample job with the error: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifact... Can some1 have a look at it and help to resolve the issue?
On Sun, Nov 11, 2018 at 4:36 PM Ehud Yonasi <eyonasi@redhat.com> wrote:
Hey, I've seen that CQ Master is not passing ovirt-engine for 10 days and fails on test suite called restore_vm0_networking here's a snap error regarding it:
Link to a sample job with the error:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifact...
I cannot follow this link because I'm 4 minutes too late jenkins.ovirt.org uses an invalid security certificate. The certificate expired on November 11, 2018, 5:13:25 PM GMT+2. The current time is November 11, 2018, 5:17 PM.
Can some1 have a look at it and help to resolve the issue?
_______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/ZQAYWTLZJKGPJ2...
On Sun, Nov 11, 2018 at 5:20 PM Dan Kenigsberg <danken@redhat.com> wrote:
On Sun, Nov 11, 2018 at 4:36 PM Ehud Yonasi <eyonasi@redhat.com> wrote:
Hey, I've seen that CQ Master is not passing ovirt-engine for 10 days and
fails on test suite called restore_vm0_networking
here's a snap error regarding it:
Link to a sample job with the error:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifact...
I cannot follow this link because I'm 4 minutes too late
jenkins.ovirt.org uses an invalid security certificate. The certificate expired on November 11, 2018, 5:13:25 PM GMT+2. The current time is November 11, 2018, 5:17 PM.
Yes, we're looking into that issue now.
Can some1 have a look at it and help to resolve the issue?
_______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/ZQAYWTLZJKGPJ2...
Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/R5LOJH73XCLLFO...
-- Eyal edri MANAGER RHV/CNV DevOps EMEA VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
On Sun, Nov 11, 2018 at 5:24 PM Eyal Edri <eedri@redhat.com> wrote:
On Sun, Nov 11, 2018 at 5:20 PM Dan Kenigsberg <danken@redhat.com> wrote:
On Sun, Nov 11, 2018 at 4:36 PM Ehud Yonasi <eyonasi@redhat.com> wrote:
Hey, I've seen that CQ Master is not passing ovirt-engine for 10 days and
fails on test suite called restore_vm0_networking
here's a snap error regarding it:
Link to a sample job with the error:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifact...
I cannot follow this link because I'm 4 minutes too late
jenkins.ovirt.org uses an invalid security certificate. The certificate expired on November 11, 2018, 5:13:25 PM GMT+2. The current time is November 11, 2018, 5:17 PM.
Yes, we're looking into that issue now.
Fixed, you should be able to access it now.
Can some1 have a look at it and help to resolve the issue?
_______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/ZQAYWTLZJKGPJ2...
Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/R5LOJH73XCLLFO...
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
-- Eyal edri MANAGER RHV/CNV DevOps EMEA VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
On Sun, Nov 11, 2018 at 5:27 PM Eyal Edri <eedri@redhat.com> wrote:
On Sun, Nov 11, 2018 at 5:24 PM Eyal Edri <eedri@redhat.com> wrote:
On Sun, Nov 11, 2018 at 5:20 PM Dan Kenigsberg <danken@redhat.com> wrote:
On Sun, Nov 11, 2018 at 4:36 PM Ehud Yonasi <eyonasi@redhat.com> wrote:
Hey, I've seen that CQ Master is not passing ovirt-engine for 10 days and fails on test suite called restore_vm0_networking here's a snap error regarding it:
Link to a sample job with the error:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifact...
I cannot follow this link because I'm 4 minutes too late
jenkins.ovirt.org uses an invalid security certificate. The certificate expired on November 11, 2018, 5:13:25 PM GMT+2. The current time is November 11, 2018, 5:17 PM.
Yes, we're looking into that issue now.
Fixed, you should be able to access it now.
OST fails during restore_vm0_networking in line 101 of 004_basic_sanity.py while comparing vm_service.get().status == state It seems that instead of reporting back the VM status, Engine set garbage "The response content type 'text/html; charset=iso-8859-1' isn't the expected XML" I do not know what could cause that, and engine.log does not mention it. But it seems like a problem in engine API hence +Martin Perina and +Ondra Machacek .
Can some1 have a look at it and help to resolve the issue?
_______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/ZQAYWTLZJKGPJ2...
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/R5LOJH73XCLLFO...
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA
TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA
TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
On Sun, 11 Nov 2018 19:04:40 +0200 Dan Kenigsberg <danken@redhat.com> wrote:
On Sun, Nov 11, 2018 at 5:27 PM Eyal Edri <eedri@redhat.com> wrote:
On Sun, Nov 11, 2018 at 5:24 PM Eyal Edri <eedri@redhat.com> wrote:
On Sun, Nov 11, 2018 at 5:20 PM Dan Kenigsberg <danken@redhat.com> wrote:
On Sun, Nov 11, 2018 at 4:36 PM Ehud Yonasi <eyonasi@redhat.com> wrote:
Hey, I've seen that CQ Master is not passing ovirt-engine for 10 days and fails on test suite called restore_vm0_networking here's a snap error regarding it:
Link to a sample job with the error:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifact...
I cannot follow this link because I'm 4 minutes too late
jenkins.ovirt.org uses an invalid security certificate. The certificate expired on November 11, 2018, 5:13:25 PM GMT+2. The current time is November 11, 2018, 5:17 PM.
Yes, we're looking into that issue now.
Fixed, you should be able to access it now.
OST fails during restore_vm0_networking in line 101 of 004_basic_sanity.py while comparing vm_service.get().status == state
It seems that instead of reporting back the VM status, Engine set garbage "The response content type 'text/html; charset=iso-8859-1' isn't the expected XML"
The relevant line in https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifac... seems to be 192.168.201.1 - - [11/Nov/2018:04:27:43 -0500] "GET /ovirt-engine/api/vms/26088164-d1a0-4254-a377-5d3c242c8105 HTTP/1.1" 503 299 and I guess the 503 error message is sent in HTML instead of XML. If I run manually https://gerrit.ovirt.org/#/c/95354/ with latest build of engine-master http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... basic suite seems to be happy: https://jenkins.ovirt.org/view/oVirt system tests/job/ovirt-system-tests_manual/3484/
I do not know what could cause that, and engine.log does not mention it. But it seems like a problem in engine API hence +Martin Perina and +Ondra Machacek .
Can some1 have a look at it and help to resolve the issue?
_______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/ZQAYWTLZJKGPJ2...
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/R5LOJH73XCLLFO...
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA
TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA
TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/DA6Q5RE5JO3FYI...
There are currently two issues failing ovirt-engine on CQ ovirt master: 1. edit vm pool is causing failure in different tests. it has a patch *waiting to be merged*: https://gerrit.ovirt.org/#/c/95354/ 2. we have a failure in upgrade suite as well to run vm but this seems to be related to the tests as well: 2018-11-12 05:41:07,831-05 WARN [org.ovirt.engine.core.bll.validator.VirtIoRngValidator] (default task-1) [] Random number source URANDOM is not supported in cluster 'test-cluster' compatibility version 4.0. here is the full error from the upgrade suite failure in run vm: https://pastebin.com/XLHtWGGx Here is the latest failure: https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... Thanks, Dafna On Mon, Nov 12, 2018 at 9:23 AM Dominik Holler <dholler@redhat.com> wrote:
On Sun, 11 Nov 2018 19:04:40 +0200 Dan Kenigsberg <danken@redhat.com> wrote:
On Sun, Nov 11, 2018 at 5:27 PM Eyal Edri <eedri@redhat.com> wrote:
On Sun, Nov 11, 2018 at 5:24 PM Eyal Edri <eedri@redhat.com> wrote:
On Sun, Nov 11, 2018 at 5:20 PM Dan Kenigsberg <danken@redhat.com>
wrote:
On Sun, Nov 11, 2018 at 4:36 PM Ehud Yonasi <eyonasi@redhat.com>
wrote:
Hey, I've seen that CQ Master is not passing ovirt-engine for 10 days
and fails on test suite called restore_vm0_networking
here's a snap error regarding it:
Link to a sample job with the error:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifact...
I cannot follow this link because I'm 4 minutes too late
jenkins.ovirt.org uses an invalid security certificate. The certificate expired on November 11, 2018, 5:13:25 PM GMT+2. The current time is November 11, 2018, 5:17 PM.
Yes, we're looking into that issue now.
Fixed, you should be able to access it now.
OST fails during restore_vm0_networking in line 101 of 004_basic_sanity.py while comparing vm_service.get().status == state
It seems that instead of reporting back the VM status, Engine set garbage "The response content type 'text/html; charset=iso-8859-1' isn't the expected XML"
The relevant line in
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifac... seems to be 192.168.201.1 - - [11/Nov/2018:04:27:43 -0500] "GET /ovirt-engine/api/vms/26088164-d1a0-4254-a377-5d3c242c8105 HTTP/1.1" 503 299 and I guess the 503 error message is sent in HTML instead of XML.
If I run manually https://gerrit.ovirt.org/#/c/95354/ with latest build of engine-master
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... basic suite seems to be happy: https://jenkins.ovirt.org/view/oVirt system tests/job/ovirt-system-tests_manual/3484/
I do not know what could cause that, and engine.log does not mention it. But it seems like a problem in engine API hence +Martin Perina and +Ondra Machacek .
Can some1 have a look at it and help to resolve the issue?
_______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/ZQAYWTLZJKGPJ2...
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/R5LOJH73XCLLFO...
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA
TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA
TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/DA6Q5RE5JO3FYI...
On Mon, Nov 12, 2018 at 12:20 PM Dafna Ron <dron@redhat.com> wrote:
There are currently two issues failing ovirt-engine on CQ ovirt master:
1. edit vm pool is causing failure in different tests. it has a patch *waiting to be merged*: https://gerrit.ovirt.org/#/c/95354/
Merged
2. we have a failure in upgrade suite as well to run vm but this seems to be related to the tests as well: 2018-11-12 05:41:07,831-05 WARN [org.ovirt.engine.core.bll.validator.VirtIoRngValidator] (default task-1) [] Random number source URANDOM is not supported in cluster 'test-cluster' compatibility version 4.0.
here is the full error from the upgrade suite failure in run vm: https://pastebin.com/XLHtWGGx
Here is the latest failure: https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change...
I will try to take a look later today
Thanks, Dafna
On Mon, Nov 12, 2018 at 9:23 AM Dominik Holler <dholler@redhat.com> wrote:
On Sun, 11 Nov 2018 19:04:40 +0200 Dan Kenigsberg <danken@redhat.com> wrote:
On Sun, Nov 11, 2018 at 5:27 PM Eyal Edri <eedri@redhat.com> wrote:
On Sun, Nov 11, 2018 at 5:24 PM Eyal Edri <eedri@redhat.com> wrote:
On Sun, Nov 11, 2018 at 5:20 PM Dan Kenigsberg <danken@redhat.com>
wrote:
On Sun, Nov 11, 2018 at 4:36 PM Ehud Yonasi <eyonasi@redhat.com>
wrote:
> > Hey, > I've seen that CQ Master is not passing ovirt-engine for 10 days and fails on test suite called restore_vm0_networking > here's a snap error regarding it: > > https://pastebin.com/7msEYqKT > > Link to a sample job with the error: > > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifact...
I cannot follow this link because I'm 4 minutes too late
jenkins.ovirt.org uses an invalid security certificate. The certificate expired on November 11, 2018, 5:13:25 PM GMT+2. The current time is November 11, 2018, 5:17 PM.
Yes, we're looking into that issue now.
Fixed, you should be able to access it now.
OST fails during restore_vm0_networking in line 101 of 004_basic_sanity.py while comparing vm_service.get().status == state
It seems that instead of reporting back the VM status, Engine set garbage "The response content type 'text/html; charset=iso-8859-1' isn't the expected XML"
The relevant line in
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifac... seems to be 192.168.201.1 - - [11/Nov/2018:04:27:43 -0500] "GET /ovirt-engine/api/vms/26088164-d1a0-4254-a377-5d3c242c8105 HTTP/1.1" 503 299 and I guess the 503 error message is sent in HTML instead of XML.
If I run manually https://gerrit.ovirt.org/#/c/95354/ with latest build of engine-master
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... basic suite seems to be happy: https://jenkins.ovirt.org/view/oVirt system tests/job/ovirt-system-tests_manual/3484/
I do not know what could cause that, and engine.log does not mention it. But it seems like a problem in engine API hence +Martin Perina and +Ondra Machacek .
> > Can some1 have a look at it and help to resolve the issue? > > > _______________________________________________ > Infra mailing list -- infra@ovirt.org > To unsubscribe send an email to infra-leave@ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
> List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/ZQAYWTLZJKGPJ2...
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/R5LOJH73XCLLFO...
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA
TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA
TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/DA6Q5RE5JO3FYI...
-- Martin Perina Associate Manager, Software Engineering Red Hat Czech s.r.o.
On Mon, 12 Nov 2018 12:29:17 +0100 Martin Perina <mperina@redhat.com> wrote:
On Mon, Nov 12, 2018 at 12:20 PM Dafna Ron <dron@redhat.com> wrote:
There are currently two issues failing ovirt-engine on CQ ovirt master:
1. edit vm pool is causing failure in different tests. it has a patch *waiting to be merged*: https://gerrit.ovirt.org/#/c/95354/
Merged
2. we have a failure in upgrade suite as well to run vm but this seems to be related to the tests as well: 2018-11-12 05:41:07,831-05 WARN [org.ovirt.engine.core.bll.validator.VirtIoRngValidator] (default task-1) [] Random number source URANDOM is not supported in cluster 'test-cluster' compatibility version 4.0.
here is the full error from the upgrade suite failure in run vm: https://pastebin.com/XLHtWGGx
Here is the latest failure: https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change...
I will try to take a look later today
I have the idea that this might be related to https://gerrit.ovirt.org/#/c/95377/ , and I check in https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... , but I have to stop now, if not solved I can go on later today.
Thanks, Dafna
On Mon, Nov 12, 2018 at 9:23 AM Dominik Holler <dholler@redhat.com> wrote:
On Sun, 11 Nov 2018 19:04:40 +0200 Dan Kenigsberg <danken@redhat.com> wrote:
On Sun, Nov 11, 2018 at 5:27 PM Eyal Edri <eedri@redhat.com> wrote:
On Sun, Nov 11, 2018 at 5:24 PM Eyal Edri <eedri@redhat.com> wrote:
On Sun, Nov 11, 2018 at 5:20 PM Dan Kenigsberg <danken@redhat.com>
wrote:
> > On Sun, Nov 11, 2018 at 4:36 PM Ehud Yonasi <eyonasi@redhat.com> wrote: > > > > Hey, > > I've seen that CQ Master is not passing ovirt-engine for 10 days and fails on test suite called restore_vm0_networking > > here's a snap error regarding it: > > > > https://pastebin.com/7msEYqKT > > > > Link to a sample job with the error: > > > > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifact...
> > I cannot follow this link because I'm 4 minutes too late > > jenkins.ovirt.org uses an invalid security certificate. The > certificate expired on November 11, 2018, 5:13:25 PM GMT+2. The > current time is November 11, 2018, 5:17 PM.
Yes, we're looking into that issue now.
Fixed, you should be able to access it now.
OST fails during restore_vm0_networking in line 101 of 004_basic_sanity.py while comparing vm_service.get().status == state
It seems that instead of reporting back the VM status, Engine set garbage "The response content type 'text/html; charset=iso-8859-1' isn't the expected XML"
The relevant line in
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifac... seems to be 192.168.201.1 - - [11/Nov/2018:04:27:43 -0500] "GET /ovirt-engine/api/vms/26088164-d1a0-4254-a377-5d3c242c8105 HTTP/1.1" 503 299 and I guess the 503 error message is sent in HTML instead of XML.
If I run manually https://gerrit.ovirt.org/#/c/95354/ with latest build of engine-master
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... basic suite seems to be happy: https://jenkins.ovirt.org/view/oVirt system tests/job/ovirt-system-tests_manual/3484/
I do not know what could cause that, and engine.log does not mention it. But it seems like a problem in engine API hence +Martin Perina and +Ondra Machacek .
> > > > > > Can some1 have a look at it and help to resolve the issue? > > > > > > _______________________________________________ > > Infra mailing list -- infra@ovirt.org > > To unsubscribe send an email to infra-leave@ovirt.org > > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > > oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
> > List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/ZQAYWTLZJKGPJ2...
> _______________________________________________ > Devel mailing list -- devel@ovirt.org > To unsubscribe send an email to devel-leave@ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/R5LOJH73XCLLFO...
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA
TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA
TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/DA6Q5RE5JO3FYI...
On Mon, Nov 12, 2018 at 12:58 PM Dominik Holler <dholler@redhat.com> wrote:
On Mon, 12 Nov 2018 12:29:17 +0100 Martin Perina <mperina@redhat.com> wrote:
On Mon, Nov 12, 2018 at 12:20 PM Dafna Ron <dron@redhat.com> wrote:
There are currently two issues failing ovirt-engine on CQ ovirt master:
1. edit vm pool is causing failure in different tests. it has a patch *waiting to be merged*: https://gerrit.ovirt.org/#/c/95354/
Merged
2. we have a failure in upgrade suite as well to run vm but this seems
to
be related to the tests as well: 2018-11-12 05:41:07,831-05 WARN [org.ovirt.engine.core.bll.validator.VirtIoRngValidator] (default task-1) [] Random number source URANDOM is not supported in cluster 'test-cluster' compatibility version 4.0.
here is the full error from the upgrade suite failure in run vm: https://pastebin.com/XLHtWGGx
Here is the latest failure:
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change...
I will try to take a look later today
I have the idea that this might be related to https://gerrit.ovirt.org/#/c/95377/ , and I check in https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... , but I have to stop now, if not solved I can go on later today.
OK, both CI and above manual OST job went fine, so I've just merged the revert patch. I will take a look at it later in detail, we should really be testing 4.3 on master and not 4.2
Thanks, Dafna
On Mon, Nov 12, 2018 at 9:23 AM Dominik Holler <dholler@redhat.com>
wrote:
On Sun, 11 Nov 2018 19:04:40 +0200 Dan Kenigsberg <danken@redhat.com> wrote:
On Sun, Nov 11, 2018 at 5:27 PM Eyal Edri <eedri@redhat.com>
wrote:
On Sun, Nov 11, 2018 at 5:24 PM Eyal Edri <eedri@redhat.com>
wrote:
> > > > On Sun, Nov 11, 2018 at 5:20 PM Dan Kenigsberg < danken@redhat.com> wrote: >> >> On Sun, Nov 11, 2018 at 4:36 PM Ehud Yonasi <eyonasi@redhat.com>
wrote:
>> > >> > Hey, >> > I've seen that CQ Master is not passing ovirt-engine for 10 days and fails on test suite called restore_vm0_networking >> > here's a snap error regarding it: >> > >> > https://pastebin.com/7msEYqKT >> > >> > Link to a sample job with the error: >> > >> >
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifact...
>> >> I cannot follow this link because I'm 4 minutes too late >> >> jenkins.ovirt.org uses an invalid security certificate. The >> certificate expired on November 11, 2018, 5:13:25 PM GMT+2. The >> current time is November 11, 2018, 5:17 PM. > > > Yes, we're looking into that issue now.
Fixed, you should be able to access it now.
OST fails during restore_vm0_networking in line 101 of 004_basic_sanity.py while comparing vm_service.get().status == state
It seems that instead of reporting back the VM status, Engine set garbage "The response content type 'text/html; charset=iso-8859-1' isn't the expected XML"
The relevant line in
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifac...
seems to be 192.168.201.1 - - [11/Nov/2018:04:27:43 -0500] "GET /ovirt-engine/api/vms/26088164-d1a0-4254-a377-5d3c242c8105 HTTP/1.1" 503 299 and I guess the 503 error message is sent in HTML instead of XML.
If I run manually https://gerrit.ovirt.org/#/c/95354/ with latest build of engine-master
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/...
basic suite seems to be happy: https://jenkins.ovirt.org/view/oVirt system tests/job/ovirt-system-tests_manual/3484/
I do not know what could cause that, and engine.log does not mention it. But it seems like a problem in engine API hence +Martin Perina and +Ondra Machacek .
> > > >> >> >> > >> > Can some1 have a look at it and help to resolve the issue? >> > >> > >> > _______________________________________________ >> > Infra mailing list -- infra@ovirt.org >> > To unsubscribe send an email to infra-leave@ovirt.org >> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> > oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
>> > List Archives:
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/ZQAYWTLZJKGPJ2...
>> _______________________________________________ >> Devel mailing list -- devel@ovirt.org >> To unsubscribe send an email to devel-leave@ovirt.org >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >> List Archives:
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/R5LOJH73XCLLFO...
> > > > -- > > Eyal edri > > > MANAGER > > RHV/CNV DevOps > > EMEA VIRTUALIZATION R&D > > > Red Hat EMEA > > TRIED. TESTED. TRUSTED. > phone: +972-9-7692018 > irc: eedri (on #tlv #rhev-dev #rhev-integ)
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA
TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives:
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/DA6Q5RE5JO3FYI...
-- Martin Perina Associate Manager, Software Engineering Red Hat Czech s.r.o.
On Mon, 12 Nov 2018 13:45:54 +0100 Martin Perina <mperina@redhat.com> wrote:
On Mon, Nov 12, 2018 at 12:58 PM Dominik Holler <dholler@redhat.com> wrote:
On Mon, 12 Nov 2018 12:29:17 +0100 Martin Perina <mperina@redhat.com> wrote:
On Mon, Nov 12, 2018 at 12:20 PM Dafna Ron <dron@redhat.com> wrote:
There are currently two issues failing ovirt-engine on CQ ovirt master:
1. edit vm pool is causing failure in different tests. it has a patch *waiting to be merged*: https://gerrit.ovirt.org/#/c/95354/
Merged
2. we have a failure in upgrade suite as well to run vm but this seems
to
be related to the tests as well: 2018-11-12 05:41:07,831-05 WARN [org.ovirt.engine.core.bll.validator.VirtIoRngValidator] (default task-1) [] Random number source URANDOM is not supported in cluster 'test-cluster' compatibility version 4.0.
here is the full error from the upgrade suite failure in run vm: https://pastebin.com/XLHtWGGx
Here is the latest failure:
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change...
I will try to take a look later today
I have the idea that this might be related to https://gerrit.ovirt.org/#/c/95377/ , and I check in https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... , but I have to stop now, if not solved I can go on later today.
OK, both CI and above manual OST job went fine, so I've just merged the revert patch. I will take a look at it later in detail, we should really be testing 4.3 on master and not 4.2
Ack. Now https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... is failing on File "/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 698, in run_vms api.vms.get(VM0_NAME).start(start_params) status: 400 reason: Bad Request 2018-11-12 10:06:30,722-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Candidate host 'lago-basic-suite-master-host-1' ('dbfe1b0c-f940-4dba-8fb1-0cfe5ca7ddfc') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: b8d11cb0-5be9-4b7e-b45a-c95fa1f18681) 2018-11-12 10:06:30,722-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Candidate host 'lago-basic-suite-master-host-0' ('e83a63ca-381e-40db-acb2-65a3e7953e11') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: b8d11cb0-5be9-4b7e-b45a-c95fa1f18681) 2018-11-12 10:06:30,723-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Can't find VDS to run the VM '57a66eff-8cbf-4643-b045-43d4dda80c66' on, so this VM will not be run. Is this related to https://gerrit.ovirt.org/#/c/95310/ ?
Thanks, Dafna
On Mon, Nov 12, 2018 at 9:23 AM Dominik Holler <dholler@redhat.com>
wrote:
On Sun, 11 Nov 2018 19:04:40 +0200 Dan Kenigsberg <danken@redhat.com> wrote:
On Sun, Nov 11, 2018 at 5:27 PM Eyal Edri <eedri@redhat.com>
wrote:
> > > > On Sun, Nov 11, 2018 at 5:24 PM Eyal Edri <eedri@redhat.com> wrote: >> >> >> >> On Sun, Nov 11, 2018 at 5:20 PM Dan Kenigsberg < danken@redhat.com> wrote: >>> >>> On Sun, Nov 11, 2018 at 4:36 PM Ehud Yonasi <eyonasi@redhat.com>
wrote:
>>> > >>> > Hey, >>> > I've seen that CQ Master is not passing ovirt-engine for 10 days and fails on test suite called restore_vm0_networking >>> > here's a snap error regarding it: >>> > >>> > https://pastebin.com/7msEYqKT >>> > >>> > Link to a sample job with the error: >>> > >>> >
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifact...
>>> >>> I cannot follow this link because I'm 4 minutes too late >>> >>> jenkins.ovirt.org uses an invalid security certificate. The >>> certificate expired on November 11, 2018, 5:13:25 PM GMT+2. The >>> current time is November 11, 2018, 5:17 PM. >> >> >> Yes, we're looking into that issue now. > > > Fixed, you should be able to access it now.
OST fails during restore_vm0_networking in line 101 of 004_basic_sanity.py while comparing vm_service.get().status == state
It seems that instead of reporting back the VM status, Engine set garbage "The response content type 'text/html; charset=iso-8859-1' isn't the expected XML"
The relevant line in
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifac...
seems to be 192.168.201.1 - - [11/Nov/2018:04:27:43 -0500] "GET /ovirt-engine/api/vms/26088164-d1a0-4254-a377-5d3c242c8105 HTTP/1.1" 503 299 and I guess the 503 error message is sent in HTML instead of XML.
If I run manually https://gerrit.ovirt.org/#/c/95354/ with latest build of engine-master
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/...
basic suite seems to be happy: https://jenkins.ovirt.org/view/oVirt system tests/job/ovirt-system-tests_manual/3484/
I do not know what could cause that, and engine.log does not mention it. But it seems like a problem in engine API hence +Martin Perina and +Ondra Machacek .
> >> >> >> >>> >>> >>> > >>> > Can some1 have a look at it and help to resolve the issue? >>> > >>> > >>> > _______________________________________________ >>> > Infra mailing list -- infra@ovirt.org >>> > To unsubscribe send an email to infra-leave@ovirt.org >>> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>> > List Archives:
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/ZQAYWTLZJKGPJ2...
>>> _______________________________________________ >>> Devel mailing list -- devel@ovirt.org >>> To unsubscribe send an email to devel-leave@ovirt.org >>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>> List Archives:
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/R5LOJH73XCLLFO...
>> >> >> >> -- >> >> Eyal edri >> >> >> MANAGER >> >> RHV/CNV DevOps >> >> EMEA VIRTUALIZATION R&D >> >> >> Red Hat EMEA >> >> TRIED. TESTED. TRUSTED. >> phone: +972-9-7692018 >> irc: eedri (on #tlv #rhev-dev #rhev-integ) > > > > -- > > Eyal edri > > > MANAGER > > RHV/CNV DevOps > > EMEA VIRTUALIZATION R&D > > > Red Hat EMEA > > TRIED. TESTED. TRUSTED. > phone: +972-9-7692018 > irc: eedri (on #tlv #rhev-dev #rhev-integ) _______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives:
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/DA6Q5RE5JO3FYI...
Martin? can you please look at the patch that Dominik sent? We need to resolve this as we have not had an engine build for the last 11 days On Mon, Nov 12, 2018 at 3:58 PM Dominik Holler <dholler@redhat.com> wrote:
On Mon, 12 Nov 2018 13:45:54 +0100 Martin Perina <mperina@redhat.com> wrote:
On Mon, Nov 12, 2018 at 12:58 PM Dominik Holler <dholler@redhat.com> wrote:
On Mon, 12 Nov 2018 12:29:17 +0100 Martin Perina <mperina@redhat.com> wrote:
On Mon, Nov 12, 2018 at 12:20 PM Dafna Ron <dron@redhat.com> wrote:
There are currently two issues failing ovirt-engine on CQ ovirt master:
1. edit vm pool is causing failure in different tests. it has a patch *waiting to be merged*: https://gerrit.ovirt.org/#/c/95354/
Merged
2. we have a failure in upgrade suite as well to run vm but this
seems to
be related to the tests as well: 2018-11-12 05:41:07,831-05 WARN [org.ovirt.engine.core.bll.validator.VirtIoRngValidator] (default task-1) [] Random number source URANDOM is not supported in cluster 'test-cluster' compatibility version 4.0.
here is the full error from the upgrade suite failure in run vm: https://pastebin.com/XLHtWGGx
Here is the latest failure:
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change...
I will try to take a look later today
I have the idea that this might be related to https://gerrit.ovirt.org/#/c/95377/ , and I check in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests...
, but I have to stop now, if not solved I can go on later today.
OK, both CI and above manual OST job went fine, so I've just merged the revert patch. I will take a look at it later in detail, we should really be testing 4.3 on master and not 4.2
Ack.
Now
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... is failing on File "/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 698, in run_vms api.vms.get(VM0_NAME).start(start_params) status: 400 reason: Bad Request
2018-11-12 10:06:30,722-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Candidate host 'lago-basic-suite-master-host-1' ('dbfe1b0c-f940-4dba-8fb1-0cfe5ca7ddfc') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: b8d11cb0-5be9-4b7e-b45a-c95fa1f18681) 2018-11-12 10:06:30,722-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Candidate host 'lago-basic-suite-master-host-0' ('e83a63ca-381e-40db-acb2-65a3e7953e11') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: b8d11cb0-5be9-4b7e-b45a-c95fa1f18681) 2018-11-12 10:06:30,723-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Can't find VDS to run the VM '57a66eff-8cbf-4643-b045-43d4dda80c66' on, so this VM will not be run.
Is this related to https://gerrit.ovirt.org/#/c/95310/ ?
Thanks, Dafna
On Mon, Nov 12, 2018 at 9:23 AM Dominik Holler <dholler@redhat.com>
wrote:
On Sun, 11 Nov 2018 19:04:40 +0200 Dan Kenigsberg <danken@redhat.com> wrote:
> On Sun, Nov 11, 2018 at 5:27 PM Eyal Edri <eedri@redhat.com>
wrote:
> > > > > > > > On Sun, Nov 11, 2018 at 5:24 PM Eyal Edri <eedri@redhat.com>
wrote:
> >> > >> > >> > >> On Sun, Nov 11, 2018 at 5:20 PM Dan Kenigsberg < danken@redhat.com> wrote: > >>> > >>> On Sun, Nov 11, 2018 at 4:36 PM Ehud Yonasi < eyonasi@redhat.com>
wrote: > >>> > > >>> > Hey, > >>> > I've seen that CQ Master is not passing ovirt-engine for 10 days and fails on test suite called restore_vm0_networking > >>> > here's a snap error regarding it: > >>> > > >>> > https://pastebin.com/7msEYqKT > >>> > > >>> > Link to a sample job with the error: > >>> > > >>> >
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifact...
> >>> > >>> I cannot follow this link because I'm 4 minutes too late > >>> > >>> jenkins.ovirt.org uses an invalid security certificate. The > >>> certificate expired on November 11, 2018, 5:13:25 PM GMT+2.
The
> >>> current time is November 11, 2018, 5:17 PM. > >> > >> > >> Yes, we're looking into that issue now. > > > > > > Fixed, you should be able to access it now. > > OST fails during restore_vm0_networking in line 101 of > 004_basic_sanity.py while comparing > vm_service.get().status == state > > It seems that instead of reporting back the VM status, Engine set garbage > "The response content type 'text/html; charset=iso-8859-1' isn't the > expected XML" >
The relevant line in
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifac...
seems to be 192.168.201.1 - - [11/Nov/2018:04:27:43 -0500] "GET /ovirt-engine/api/vms/26088164-d1a0-4254-a377-5d3c242c8105 HTTP/1.1" 503 299 and I guess the 503 error message is sent in HTML instead of XML.
If I run manually https://gerrit.ovirt.org/#/c/95354/ with latest build of engine-master
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/...
basic suite seems to be happy: https://jenkins.ovirt.org/view/oVirt system tests/job/ovirt-system-tests_manual/3484/
> I do not know what could cause that, and engine.log does not mention > it. But it seems like a problem in engine API hence +Martin Perina and > +Ondra Machacek . > > > > > > >> > >> > >> > >>> > >>> > >>> > > >>> > Can some1 have a look at it and help to resolve the issue? > >>> > > >>> > > >>> > _______________________________________________ > >>> > Infra mailing list -- infra@ovirt.org > >>> > To unsubscribe send an email to infra-leave@ovirt.org > >>> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > >>> > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > >>> > List Archives:
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/ZQAYWTLZJKGPJ2...
> >>> _______________________________________________ > >>> Devel mailing list -- devel@ovirt.org > >>> To unsubscribe send an email to devel-leave@ovirt.org > >>> Privacy Statement:
https://www.ovirt.org/site/privacy-policy/
> >>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > >>> List Archives:
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/R5LOJH73XCLLFO...
> >> > >> > >> > >> -- > >> > >> Eyal edri > >> > >> > >> MANAGER > >> > >> RHV/CNV DevOps > >> > >> EMEA VIRTUALIZATION R&D > >> > >> > >> Red Hat EMEA > >> > >> TRIED. TESTED. TRUSTED. > >> phone: +972-9-7692018 > >> irc: eedri (on #tlv #rhev-dev #rhev-integ) > > > > > > > > -- > > > > Eyal edri > > > > > > MANAGER > > > > RHV/CNV DevOps > > > > EMEA VIRTUALIZATION R&D > > > > > > Red Hat EMEA > > > > TRIED. TESTED. TRUSTED. > > phone: +972-9-7692018 > > irc: eedri (on #tlv #rhev-dev #rhev-integ) > _______________________________________________ > Devel mailing list -- devel@ovirt.org > To unsubscribe send an email to devel-leave@ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives:
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/DA6Q5RE5JO3FYI...
On Tue, Nov 13, 2018 at 11:02 AM Dafna Ron <dron@redhat.com> wrote:
Martin? can you please look at the patch that Dominik sent? We need to resolve this as we have not had an engine build for the last 11 days
Yesterday I've merged Dominik's revert patch https://gerrit.ovirt.org/95377 which should switch cluster level back to 4.2. Below mentioned change https://gerrit.ovirt.org/95310 is relevant only to cluster level 4.3, am I right Michal? The build mentioned https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... is from yesterday. Are we sure that it was executed only after #95377 was merged? I'd like to see the results from latest https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... but unfortunately it already waits more than an hour for available hosts ...
On Mon, Nov 12, 2018 at 3:58 PM Dominik Holler <dholler@redhat.com> wrote:
On Mon, 12 Nov 2018 13:45:54 +0100 Martin Perina <mperina@redhat.com> wrote:
On Mon, Nov 12, 2018 at 12:58 PM Dominik Holler <dholler@redhat.com> wrote:
On Mon, 12 Nov 2018 12:29:17 +0100 Martin Perina <mperina@redhat.com> wrote:
On Mon, Nov 12, 2018 at 12:20 PM Dafna Ron <dron@redhat.com> wrote:
There are currently two issues failing ovirt-engine on CQ ovirt master:
1. edit vm pool is causing failure in different tests. it has a patch *waiting to be merged*: https://gerrit.ovirt.org/#/c/95354/
Merged
2. we have a failure in upgrade suite as well to run vm but this
seems to
be related to the tests as well: 2018-11-12 05:41:07,831-05 WARN [org.ovirt.engine.core.bll.validator.VirtIoRngValidator] (default task-1) [] Random number source URANDOM is not supported in cluster 'test-cluster' compatibility version 4.0.
here is the full error from the upgrade suite failure in run vm: https://pastebin.com/XLHtWGGx
Here is the latest failure:
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change...
I will try to take a look later today
I have the idea that this might be related to https://gerrit.ovirt.org/#/c/95377/ , and I check in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests...
, but I have to stop now, if not solved I can go on later today.
OK, both CI and above manual OST job went fine, so I've just merged the revert patch. I will take a look at it later in detail, we should really be testing 4.3 on master and not 4.2
Ack.
Now
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... is failing on File "/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 698, in run_vms api.vms.get(VM0_NAME).start(start_params) status: 400 reason: Bad Request
2018-11-12 10:06:30,722-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Candidate host 'lago-basic-suite-master-host-1' ('dbfe1b0c-f940-4dba-8fb1-0cfe5ca7ddfc') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: b8d11cb0-5be9-4b7e-b45a-c95fa1f18681) 2018-11-12 10:06:30,722-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Candidate host 'lago-basic-suite-master-host-0' ('e83a63ca-381e-40db-acb2-65a3e7953e11') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: b8d11cb0-5be9-4b7e-b45a-c95fa1f18681) 2018-11-12 10:06:30,723-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Can't find VDS to run the VM '57a66eff-8cbf-4643-b045-43d4dda80c66' on, so this VM will not be run.
Is this related to https://gerrit.ovirt.org/#/c/95310/ ?
Thanks, Dafna
On Mon, Nov 12, 2018 at 9:23 AM Dominik Holler <
dholler@redhat.com>
wrote:
> On Sun, 11 Nov 2018 19:04:40 +0200 > Dan Kenigsberg <danken@redhat.com> wrote: > > > On Sun, Nov 11, 2018 at 5:27 PM Eyal Edri <eedri@redhat.com>
wrote:
> > > > > > > > > > > > On Sun, Nov 11, 2018 at 5:24 PM Eyal Edri <eedri@redhat.com>
wrote:
> > >> > > >> > > >> > > >> On Sun, Nov 11, 2018 at 5:20 PM Dan Kenigsberg < danken@redhat.com> > wrote: > > >>> > > >>> On Sun, Nov 11, 2018 at 4:36 PM Ehud Yonasi < eyonasi@redhat.com>
> wrote: > > >>> > > > >>> > Hey, > > >>> > I've seen that CQ Master is not passing ovirt-engine for 10 days > and fails on test suite called restore_vm0_networking > > >>> > here's a snap error regarding it: > > >>> > > > >>> > https://pastebin.com/7msEYqKT > > >>> > > > >>> > Link to a sample job with the error: > > >>> > > > >>> > >
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifact...
> > > >>> > > >>> I cannot follow this link because I'm 4 minutes too late > > >>> > > >>> jenkins.ovirt.org uses an invalid security certificate. The > > >>> certificate expired on November 11, 2018, 5:13:25 PM GMT+2. The > > >>> current time is November 11, 2018, 5:17 PM. > > >> > > >> > > >> Yes, we're looking into that issue now. > > > > > > > > > Fixed, you should be able to access it now. > > > > OST fails during restore_vm0_networking in line 101 of > > 004_basic_sanity.py while comparing > > vm_service.get().status == state > > > > It seems that instead of reporting back the VM status, Engine set > garbage > > "The response content type 'text/html; charset=iso-8859-1' isn't the > > expected XML" > > > > The relevant line in > >
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifac...
> seems to be > 192.168.201.1 - - [11/Nov/2018:04:27:43 -0500] "GET > /ovirt-engine/api/vms/26088164-d1a0-4254-a377-5d3c242c8105 HTTP/1.1" 503 299 > and I guess the 503 error message is sent in HTML instead of XML. > > If I run manually > https://gerrit.ovirt.org/#/c/95354/ > with latest build of engine-master > >
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/...
> basic suite seems to be happy: > https://jenkins.ovirt.org/view/oVirt system > tests/job/ovirt-system-tests_manual/3484/ > > > > I do not know what could cause that, and engine.log does not mention > > it. But it seems like a problem in engine API hence +Martin Perina and > > +Ondra Machacek . > > > > > > > > > > > >> > > >> > > >> > > >>> > > >>> > > >>> > > > >>> > Can some1 have a look at it and help to resolve the issue? > > >>> > > > >>> > > > >>> > _______________________________________________ > > >>> > Infra mailing list -- infra@ovirt.org > > >>> > To unsubscribe send an email to infra-leave@ovirt.org > > >>> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > > >>> > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > > >>> > List Archives: >
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/ZQAYWTLZJKGPJ2...
> > > >>> _______________________________________________ > > >>> Devel mailing list -- devel@ovirt.org > > >>> To unsubscribe send an email to devel-leave@ovirt.org > > >>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > > >>> oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > > >>> List Archives: >
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/R5LOJH73XCLLFO...
> > > >> > > >> > > >> > > >> -- > > >> > > >> Eyal edri > > >> > > >> > > >> MANAGER > > >> > > >> RHV/CNV DevOps > > >> > > >> EMEA VIRTUALIZATION R&D > > >> > > >> > > >> Red Hat EMEA > > >> > > >> TRIED. TESTED. TRUSTED. > > >> phone: +972-9-7692018 > > >> irc: eedri (on #tlv #rhev-dev #rhev-integ) > > > > > > > > > > > > -- > > > > > > Eyal edri > > > > > > > > > MANAGER > > > > > > RHV/CNV DevOps > > > > > > EMEA VIRTUALIZATION R&D > > > > > > > > > Red Hat EMEA > > > > > > TRIED. TESTED. TRUSTED. > > > phone: +972-9-7692018 > > > irc: eedri (on #tlv #rhev-dev #rhev-integ) > > _______________________________________________ > > Devel mailing list -- devel@ovirt.org > > To unsubscribe send an email to devel-leave@ovirt.org > > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > > List Archives: >
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/DA6Q5RE5JO3FYI...
> >
-- Martin Perina Associate Manager, Software Engineering Red Hat Czech s.r.o.
On Tue, 13 Nov 2018 11:56:37 +0100 Martin Perina <mperina@redhat.com> wrote:
On Tue, Nov 13, 2018 at 11:02 AM Dafna Ron <dron@redhat.com> wrote:
Martin? can you please look at the patch that Dominik sent? We need to resolve this as we have not had an engine build for the last 11 days
Yesterday I've merged Dominik's revert patch https://gerrit.ovirt.org/95377 which should switch cluster level back to 4.2. Below mentioned change https://gerrit.ovirt.org/95310 is relevant only to cluster level 4.3, am I right Michal?
The build mentioned https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... is from yesterday. Are we sure that it was executed only after #95377 was merged? I'd like to see the results from latest https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... but unfortunately it already waits more than an hour for available hosts ...
https://gerrit.ovirt.org/#/c/95283/ results in http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms succeeding. The next merged change https://gerrit.ovirt.org/#/c/95310/ results in http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms failing with 2018-11-12 17:35:10,109-05 INFO [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Running command: RunVmOnceCommand internal: false. Entities affected : ID: d10aa133-b9b6-455d-8137-ab822d1c1971 Type: VMAction group RUN_VM with role type USER 2018-11-12 17:35:10,113-05 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] method: getVmManager, params: [d10aa133-b9b6-455d-8137-ab822d1c1971], timeElapsed: 4ms 2018-11-12 17:35:10,128-05 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] method: getAllForClusterWithStatus, params: [2ca9ccd8-61f0-470c-ba3f-07766202f260, Up], timeElapsed: 7ms 2018-11-12 17:35:10,129-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Candidate host 'lago-basic-suite-master-host-1' ('282860ab-8873-4702-a2be-100a6da111af') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 6930b632-5593-4481-bf2a-a1d8b14a583a) 2018-11-12 17:35:10,129-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Candidate host 'lago-basic-suite-master-host-0' ('c48eca36-ea98-46b2-8473-f184833e68a8') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 6930b632-5593-4481-bf2a-a1d8b14a583a) 2018-11-12 17:35:10,130-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Can't find VDS to run the VM 'd10aa133-b9b6-455d-8137-ab822d1c1971' on, so this VM will not be run. in https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... Is this helpful for you?
On Mon, Nov 12, 2018 at 3:58 PM Dominik Holler <dholler@redhat.com> wrote:
On Mon, 12 Nov 2018 13:45:54 +0100 Martin Perina <mperina@redhat.com> wrote:
On Mon, Nov 12, 2018 at 12:58 PM Dominik Holler <dholler@redhat.com> wrote:
On Mon, 12 Nov 2018 12:29:17 +0100 Martin Perina <mperina@redhat.com> wrote:
On Mon, Nov 12, 2018 at 12:20 PM Dafna Ron <dron@redhat.com> wrote:
> There are currently two issues failing ovirt-engine on CQ ovirt master: > > 1. edit vm pool is causing failure in different tests. it has a patch *waiting > to be merged*: https://gerrit.ovirt.org/#/c/95354/ >
Merged
> > 2. we have a failure in upgrade suite as well to run vm but this seems to > be related to the tests as well: > 2018-11-12 05:41:07,831-05 WARN > [org.ovirt.engine.core.bll.validator.VirtIoRngValidator] (default task-1) > [] Random number source URANDOM is not supported in cluster 'test-cluster' > compatibility version 4.0. > > here is the full error from the upgrade suite failure in run vm: > https://pastebin.com/XLHtWGGx > > Here is the latest failure: >
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change...
>
I will try to take a look later today
I have the idea that this might be related to https://gerrit.ovirt.org/#/c/95377/ , and I check in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests...
, but I have to stop now, if not solved I can go on later today.
OK, both CI and above manual OST job went fine, so I've just merged the revert patch. I will take a look at it later in detail, we should really be testing 4.3 on master and not 4.2
Ack.
Now
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... is failing on File "/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 698, in run_vms api.vms.get(VM0_NAME).start(start_params) status: 400 reason: Bad Request
2018-11-12 10:06:30,722-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Candidate host 'lago-basic-suite-master-host-1' ('dbfe1b0c-f940-4dba-8fb1-0cfe5ca7ddfc') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: b8d11cb0-5be9-4b7e-b45a-c95fa1f18681) 2018-11-12 10:06:30,722-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Candidate host 'lago-basic-suite-master-host-0' ('e83a63ca-381e-40db-acb2-65a3e7953e11') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: b8d11cb0-5be9-4b7e-b45a-c95fa1f18681) 2018-11-12 10:06:30,723-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Can't find VDS to run the VM '57a66eff-8cbf-4643-b045-43d4dda80c66' on, so this VM will not be run.
Is this related to https://gerrit.ovirt.org/#/c/95310/ ?
> > > Thanks, > Dafna > > > > > On Mon, Nov 12, 2018 at 9:23 AM Dominik Holler <
dholler@redhat.com>
wrote:
> >> On Sun, 11 Nov 2018 19:04:40 +0200 >> Dan Kenigsberg <danken@redhat.com> wrote: >> >> > On Sun, Nov 11, 2018 at 5:27 PM Eyal Edri <eedri@redhat.com> wrote: >> > > >> > > >> > > >> > > On Sun, Nov 11, 2018 at 5:24 PM Eyal Edri <eedri@redhat.com>
wrote:
>> > >> >> > >> >> > >> >> > >> On Sun, Nov 11, 2018 at 5:20 PM Dan Kenigsberg < danken@redhat.com> >> wrote: >> > >>> >> > >>> On Sun, Nov 11, 2018 at 4:36 PM Ehud Yonasi < eyonasi@redhat.com>
>> wrote: >> > >>> > >> > >>> > Hey, >> > >>> > I've seen that CQ Master is not passing ovirt-engine for 10 days >> and fails on test suite called restore_vm0_networking >> > >>> > here's a snap error regarding it: >> > >>> > >> > >>> > https://pastebin.com/7msEYqKT >> > >>> > >> > >>> > Link to a sample job with the error: >> > >>> > >> > >>> > >>
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifact...
>> >> > >>> >> > >>> I cannot follow this link because I'm 4 minutes too late >> > >>> >> > >>> jenkins.ovirt.org uses an invalid security certificate. The >> > >>> certificate expired on November 11, 2018, 5:13:25 PM GMT+2. The >> > >>> current time is November 11, 2018, 5:17 PM. >> > >> >> > >> >> > >> Yes, we're looking into that issue now. >> > > >> > > >> > > Fixed, you should be able to access it now. >> > >> > OST fails during restore_vm0_networking in line 101 of >> > 004_basic_sanity.py while comparing >> > vm_service.get().status == state >> > >> > It seems that instead of reporting back the VM status, Engine set >> garbage >> > "The response content type 'text/html; charset=iso-8859-1' isn't the >> > expected XML" >> > >> >> The relevant line in >> >>
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifac...
>> seems to be >> 192.168.201.1 - - [11/Nov/2018:04:27:43 -0500] "GET >> /ovirt-engine/api/vms/26088164-d1a0-4254-a377-5d3c242c8105 HTTP/1.1" 503 299 >> and I guess the 503 error message is sent in HTML instead of XML. >> >> If I run manually >> https://gerrit.ovirt.org/#/c/95354/ >> with latest build of engine-master >> >>
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/...
>> basic suite seems to be happy: >> https://jenkins.ovirt.org/view/oVirt system >> tests/job/ovirt-system-tests_manual/3484/ >> >> >> > I do not know what could cause that, and engine.log does not mention >> > it. But it seems like a problem in engine API hence +Martin Perina and >> > +Ondra Machacek . >> > >> > >> > >> > > >> > >> >> > >> >> > >> >> > >>> >> > >>> >> > >>> > >> > >>> > Can some1 have a look at it and help to resolve the issue? >> > >>> > >> > >>> > >> > >>> > _______________________________________________ >> > >>> > Infra mailing list -- infra@ovirt.org >> > >>> > To unsubscribe send an email to infra-leave@ovirt.org >> > >>> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> > >>> > oVirt Code of Conduct: >> https://www.ovirt.org/community/about/community-guidelines/ >> > >>> > List Archives: >>
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/ZQAYWTLZJKGPJ2...
>> >> > >>> _______________________________________________ >> > >>> Devel mailing list -- devel@ovirt.org >> > >>> To unsubscribe send an email to devel-leave@ovirt.org >> > >>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> > >>> oVirt Code of Conduct: >> https://www.ovirt.org/community/about/community-guidelines/ >> > >>> List Archives: >>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/R5LOJH73XCLLFO...
>> >> > >> >> > >> >> > >> >> > >> -- >> > >> >> > >> Eyal edri >> > >> >> > >> >> > >> MANAGER >> > >> >> > >> RHV/CNV DevOps >> > >> >> > >> EMEA VIRTUALIZATION R&D >> > >> >> > >> >> > >> Red Hat EMEA >> > >> >> > >> TRIED. TESTED. TRUSTED. >> > >> phone: +972-9-7692018 >> > >> irc: eedri (on #tlv #rhev-dev #rhev-integ) >> > > >> > > >> > > >> > > -- >> > > >> > > Eyal edri >> > > >> > > >> > > MANAGER >> > > >> > > RHV/CNV DevOps >> > > >> > > EMEA VIRTUALIZATION R&D >> > > >> > > >> > > Red Hat EMEA >> > > >> > > TRIED. TESTED. TRUSTED. >> > > phone: +972-9-7692018 >> > > irc: eedri (on #tlv #rhev-dev #rhev-integ) >> > _______________________________________________ >> > Devel mailing list -- devel@ovirt.org >> > To unsubscribe send an email to devel-leave@ovirt.org >> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> > oVirt Code of Conduct: >> https://www.ovirt.org/community/about/community-guidelines/ >> > List Archives: >>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/DA6Q5RE5JO3FYI...
>> >>
On 13 Nov 2018, at 12:20, Dominik Holler <dholler@redhat.com> wrote:
On Tue, 13 Nov 2018 11:56:37 +0100 Martin Perina <mperina@redhat.com> wrote:
On Tue, Nov 13, 2018 at 11:02 AM Dafna Ron <dron@redhat.com> wrote:
Martin? can you please look at the patch that Dominik sent? We need to resolve this as we have not had an engine build for the last 11 days
Yesterday I've merged Dominik's revert patch https://gerrit.ovirt.org/95377 which should switch cluster level back to 4.2. Below mentioned change https://gerrit.ovirt.org/95310 is relevant only to cluster level 4.3, am I right Michal?
The build mentioned https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... is from yesterday. Are we sure that it was executed only after #95377 was merged? I'd like to see the results from latest https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... but unfortunately it already waits more than an hour for available hosts ...
https://gerrit.ovirt.org/#/c/95283/ results in http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms succeeding.
The next merged change https://gerrit.ovirt.org/#/c/95310/ results in http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms failing with 2018-11-12 17:35:10,109-05 INFO [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Running command: RunVmOnceCommand internal: false. Entities affected : ID: d10aa133-b9b6-455d-8137-ab822d1c1971 Type: VMAction group RUN_VM with role type USER 2018-11-12 17:35:10,113-05 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] method: getVmManager, params: [d10aa133-b9b6-455d-8137-ab822d1c1971], timeElapsed: 4ms 2018-11-12 17:35:10,128-05 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] method: getAllForClusterWithStatus, params: [2ca9ccd8-61f0-470c-ba3f-07766202f260, Up], timeElapsed: 7ms 2018-11-12 17:35:10,129-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Candidate host 'lago-basic-suite-master-host-1' ('282860ab-8873-4702-a2be-100a6da111af') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 6930b632-5593-4481-bf2a-a1d8b14a583a) 2018-11-12 17:35:10,129-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Candidate host 'lago-basic-suite-master-host-0' ('c48eca36-ea98-46b2-8473-f184833e68a8') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 6930b632-5593-4481-bf2a-a1d8b14a583a) 2018-11-12 17:35:10,130-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Can't find VDS to run the VM 'd10aa133-b9b6-455d-8137-ab822d1c1971' on, so this VM will not be run. in https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests...
Is this helpful for you?
actually, there ire two issues 1) cluster is still 4.3 even after Martin’s revert. 2) the patch is wrong too, as in HandleVdsCpuFlagsOrClusterChangedCommand it just goes ahead and sets the cluster cpu to whatever the host reported regardless if it is valid or not. Steven, please fix that (line 96 in backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/HandleVdsCpuFlagsOrClusterChangedCommand.java). It needs to pass the validation or we need some other solution. 3) regardless, we should make 4.3 work too , I tried to play with it a bit in https://gerrit.ovirt.org/#/c/95407/ <https://gerrit.ovirt.org/#/c/95407/>, let’s see… Thanks, michal
On Mon, Nov 12, 2018 at 3:58 PM Dominik Holler <dholler@redhat.com> wrote:
On Mon, 12 Nov 2018 13:45:54 +0100 Martin Perina <mperina@redhat.com> wrote:
On Mon, Nov 12, 2018 at 12:58 PM Dominik Holler <dholler@redhat.com> wrote:
On Mon, 12 Nov 2018 12:29:17 +0100 Martin Perina <mperina@redhat.com> wrote:
> On Mon, Nov 12, 2018 at 12:20 PM Dafna Ron <dron@redhat.com> wrote: > >> There are currently two issues failing ovirt-engine on CQ ovirt master: >> >> 1. edit vm pool is causing failure in different tests. it has a patch *waiting >> to be merged*: https://gerrit.ovirt.org/#/c/95354/ >> > > Merged > >> >> 2. we have a failure in upgrade suite as well to run vm but this seems to >> be related to the tests as well: >> 2018-11-12 05:41:07,831-05 WARN >> [org.ovirt.engine.core.bll.validator.VirtIoRngValidator] (default task-1) >> [] Random number source URANDOM is not supported in cluster 'test-cluster' >> compatibility version 4.0. >> >> here is the full error from the upgrade suite failure in run vm: >> https://pastebin.com/XLHtWGGx >> >> Here is the latest failure: >>
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change...
>> > > I will try to take a look later today >
I have the idea that this might be related to https://gerrit.ovirt.org/#/c/95377/ , and I check in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests...
, but I have to stop now, if not solved I can go on later today.
OK, both CI and above manual OST job went fine, so I've just merged the revert patch. I will take a look at it later in detail, we should really be testing 4.3 on master and not 4.2
Ack.
Now
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... is failing on File "/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 698, in run_vms api.vms.get(VM0_NAME).start(start_params) status: 400 reason: Bad Request
2018-11-12 10:06:30,722-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Candidate host 'lago-basic-suite-master-host-1' ('dbfe1b0c-f940-4dba-8fb1-0cfe5ca7ddfc') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: b8d11cb0-5be9-4b7e-b45a-c95fa1f18681) 2018-11-12 10:06:30,722-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Candidate host 'lago-basic-suite-master-host-0' ('e83a63ca-381e-40db-acb2-65a3e7953e11') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: b8d11cb0-5be9-4b7e-b45a-c95fa1f18681) 2018-11-12 10:06:30,723-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Can't find VDS to run the VM '57a66eff-8cbf-4643-b045-43d4dda80c66' on, so this VM will not be run.
Is this related to https://gerrit.ovirt.org/#/c/95310/ ?
>> >> >> Thanks, >> Dafna >> >> >> >> >> On Mon, Nov 12, 2018 at 9:23 AM Dominik Holler <
dholler@redhat.com>
wrote: >> >>> On Sun, 11 Nov 2018 19:04:40 +0200 >>> Dan Kenigsberg <danken@redhat.com> wrote: >>> >>>> On Sun, Nov 11, 2018 at 5:27 PM Eyal Edri <eedri@redhat.com> wrote: >>>>> >>>>> >>>>> >>>>> On Sun, Nov 11, 2018 at 5:24 PM Eyal Edri <eedri@redhat.com>
wrote: >>>>>> >>>>>> >>>>>> >>>>>> On Sun, Nov 11, 2018 at 5:20 PM Dan Kenigsberg < danken@redhat.com> >>> wrote: >>>>>>> >>>>>>> On Sun, Nov 11, 2018 at 4:36 PM Ehud Yonasi < eyonasi@redhat.com>
>>> wrote: >>>>>>>> >>>>>>>> Hey, >>>>>>>> I've seen that CQ Master is not passing ovirt-engine for 10 days >>> and fails on test suite called restore_vm0_networking >>>>>>>> here's a snap error regarding it: >>>>>>>> >>>>>>>> https://pastebin.com/7msEYqKT >>>>>>>> >>>>>>>> Link to a sample job with the error: >>>>>>>> >>>>>>>> >>>
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifact...
>>> >>>>>>> >>>>>>> I cannot follow this link because I'm 4 minutes too late >>>>>>> >>>>>>> jenkins.ovirt.org uses an invalid security certificate. The >>>>>>> certificate expired on November 11, 2018, 5:13:25 PM GMT+2. The >>>>>>> current time is November 11, 2018, 5:17 PM. >>>>>> >>>>>> >>>>>> Yes, we're looking into that issue now. >>>>> >>>>> >>>>> Fixed, you should be able to access it now. >>>> >>>> OST fails during restore_vm0_networking in line 101 of >>>> 004_basic_sanity.py while comparing >>>> vm_service.get().status == state >>>> >>>> It seems that instead of reporting back the VM status, Engine set >>> garbage >>>> "The response content type 'text/html; charset=iso-8859-1' isn't the >>>> expected XML" >>>> >>> >>> The relevant line in >>> >>>
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifac...
>>> seems to be >>> 192.168.201.1 - - [11/Nov/2018:04:27:43 -0500] "GET >>> /ovirt-engine/api/vms/26088164-d1a0-4254-a377-5d3c242c8105 HTTP/1.1" 503 299 >>> and I guess the 503 error message is sent in HTML instead of XML. >>> >>> If I run manually >>> https://gerrit.ovirt.org/#/c/95354/ >>> with latest build of engine-master >>> >>>
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/...
>>> basic suite seems to be happy: >>> https://jenkins.ovirt.org/view/oVirt system >>> tests/job/ovirt-system-tests_manual/3484/ >>> >>> >>>> I do not know what could cause that, and engine.log does not mention >>>> it. But it seems like a problem in engine API hence +Martin Perina and >>>> +Ondra Machacek . >>>> >>>> >>>> >>>>> >>>>>> >>>>>> >>>>>> >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> Can some1 have a look at it and help to resolve the issue? >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Infra mailing list -- infra@ovirt.org >>>>>>>> To unsubscribe send an email to infra-leave@ovirt.org >>>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>>> oVirt Code of Conduct: >>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>>>> List Archives: >>>
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/ZQAYWTLZJKGPJ2...
>>> >>>>>>> _______________________________________________ >>>>>>> Devel mailing list -- devel@ovirt.org >>>>>>> To unsubscribe send an email to devel-leave@ovirt.org >>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>> oVirt Code of Conduct: >>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>>> List Archives: >>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/R5LOJH73XCLLFO...
>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Eyal edri >>>>>> >>>>>> >>>>>> MANAGER >>>>>> >>>>>> RHV/CNV DevOps >>>>>> >>>>>> EMEA VIRTUALIZATION R&D >>>>>> >>>>>> >>>>>> Red Hat EMEA >>>>>> >>>>>> TRIED. TESTED. TRUSTED. >>>>>> phone: +972-9-7692018 >>>>>> irc: eedri (on #tlv #rhev-dev #rhev-integ) >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Eyal edri >>>>> >>>>> >>>>> MANAGER >>>>> >>>>> RHV/CNV DevOps >>>>> >>>>> EMEA VIRTUALIZATION R&D >>>>> >>>>> >>>>> Red Hat EMEA >>>>> >>>>> TRIED. TESTED. TRUSTED. >>>>> phone: +972-9-7692018 >>>>> irc: eedri (on #tlv #rhev-dev #rhev-integ) >>>> _______________________________________________ >>>> Devel mailing list -- devel@ovirt.org >>>> To unsubscribe send an email to devel-leave@ovirt.org >>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>> oVirt Code of Conduct: >>> https://www.ovirt.org/community/about/community-guidelines/ >>>> List Archives: >>>
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/DA6Q5RE5JO3FYI...
>>> >>> >
On Tue, Nov 13, 2018 at 12:49 PM Michal Skrivanek <mskrivan@redhat.com> wrote:
On 13 Nov 2018, at 12:20, Dominik Holler <dholler@redhat.com> wrote:
On Tue, 13 Nov 2018 11:56:37 +0100 Martin Perina <mperina@redhat.com> wrote:
On Tue, Nov 13, 2018 at 11:02 AM Dafna Ron <dron@redhat.com> wrote:
Martin? can you please look at the patch that Dominik sent? We need to resolve this as we have not had an engine build for the last 11 days
Yesterday I've merged Dominik's revert patch https://gerrit.ovirt.org/95377 which should switch cluster level back to 4.2. Below mentioned change https://gerrit.ovirt.org/95310 is relevant only to cluster level 4.3, am I right Michal?
The build mentioned
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... is from yesterday. Are we sure that it was executed only after #95377 was merged? I'd like to see the results from latest
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... but unfortunately it already waits more than an hour for available hosts ...
https://gerrit.ovirt.org/#/c/95283/ results in
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms succeeding.
The next merged change https://gerrit.ovirt.org/#/c/95310/ results in
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms failing with 2018-11-12 17:35:10,109-05 INFO [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Running command: RunVmOnceCommand internal: false. Entities affected : ID: d10aa133-b9b6-455d-8137-ab822d1c1971 Type: VMAction group RUN_VM with role type USER 2018-11-12 17:35:10,113-05 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] method: getVmManager, params: [d10aa133-b9b6-455d-8137-ab822d1c1971], timeElapsed: 4ms 2018-11-12 17:35:10,128-05 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] method: getAllForClusterWithStatus, params: [2ca9ccd8-61f0-470c-ba3f-07766202f260, Up], timeElapsed: 7ms 2018-11-12 17:35:10,129-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Candidate host 'lago-basic-suite-master-host-1' ('282860ab-8873-4702-a2be-100a6da111af') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 6930b632-5593-4481-bf2a-a1d8b14a583a) 2018-11-12 17:35:10,129-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Candidate host 'lago-basic-suite-master-host-0' ('c48eca36-ea98-46b2-8473-f184833e68a8') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 6930b632-5593-4481-bf2a-a1d8b14a583a) 2018-11-12 17:35:10,130-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Can't find VDS to run the VM 'd10aa133-b9b6-455d-8137-ab822d1c1971' on, so this VM will not be run. in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests...
Is this helpful for you?
actually, there ire two issues 1) cluster is still 4.3 even after Martin’s revert.
https://gerrit.ovirt.org/#/c/95409/ should align cluster level with dc level 2) the patch is wrong too, as in HandleVdsCpuFlagsOrClusterChangedCommand
it just goes ahead and sets the cluster cpu to whatever the host reported regardless if it is valid or not. Steven, please fix that (line 96 in backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/HandleVdsCpuFlagsOrClusterChangedCommand.java). It needs to pass the validation or we need some other solution. 3) regardless, we should make 4.3 work too , I tried to play with it a bit in https://gerrit.ovirt.org/#/c/95407/, let’s see…
Thanks, michal
On Mon, Nov 12, 2018 at 3:58 PM Dominik Holler <dholler@redhat.com> wrote:
On Mon, 12 Nov 2018 13:45:54 +0100 Martin Perina <mperina@redhat.com> wrote:
On Mon, Nov 12, 2018 at 12:58 PM Dominik Holler <dholler@redhat.com>
wrote:
On Mon, 12 Nov 2018 12:29:17 +0100 Martin Perina <mperina@redhat.com> wrote:
On Mon, Nov 12, 2018 at 12:20 PM Dafna Ron <dron@redhat.com> wrote:
There are currently two issues failing ovirt-engine on CQ ovirt
master:
1. edit vm pool is causing failure in different tests. it has a
patch
*waiting
to be merged*: https://gerrit.ovirt.org/#/c/95354/
Merged
2. we have a failure in upgrade suite as well to run vm but this
seems
to
be related to the tests as well: 2018-11-12 05:41:07,831-05 WARN [org.ovirt.engine.core.bll.validator.VirtIoRngValidator]
(default
task-1)
[] Random number source URANDOM is not supported in cluster
'test-cluster'
compatibility version 4.0.
here is the full error from the upgrade suite failure in run vm: https://pastebin.com/XLHtWGGx
Here is the latest failure:
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change...
I will try to take a look later today
I have the idea that this might be related to https://gerrit.ovirt.org/#/c/95377/ , and I check in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests...
, but I have to stop now, if not solved I can go on later today.
OK, both CI and above manual OST job went fine, so I've just merged the revert patch. I will take a look at it later in detail, we should
really be
testing 4.3 on master and not 4.2
Ack.
Now
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... is failing on File
"/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 698, in run_vms api.vms.get(VM0_NAME).start(start_params) status: 400 reason: Bad Request
2018-11-12 10:06:30,722-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Candidate host 'lago-basic-suite-master-host-1' ('dbfe1b0c-f940-4dba-8fb1-0cfe5ca7ddfc') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: b8d11cb0-5be9-4b7e-b45a-c95fa1f18681) 2018-11-12 10:06:30,722-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Candidate host 'lago-basic-suite-master-host-0' ('e83a63ca-381e-40db-acb2-65a3e7953e11') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: b8d11cb0-5be9-4b7e-b45a-c95fa1f18681) 2018-11-12 10:06:30,723-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Can't find VDS to run the VM '57a66eff-8cbf-4643-b045-43d4dda80c66' on, so this VM will not be run.
Is this related to https://gerrit.ovirt.org/#/c/95310/ ?
Thanks, Dafna
On Mon, Nov 12, 2018 at 9:23 AM Dominik Holler <
dholler@redhat.com>
wrote:
On Sun, 11 Nov 2018 19:04:40 +0200 Dan Kenigsberg <danken@redhat.com> wrote:
On Sun, Nov 11, 2018 at 5:27 PM Eyal Edri <eedri@redhat.com>
wrote:
On Sun, Nov 11, 2018 at 5:24 PM Eyal Edri <eedri@redhat.com>
wrote:
On Sun, Nov 11, 2018 at 5:20 PM Dan Kenigsberg <
danken@redhat.com>
wrote:
On Sun, Nov 11, 2018 at 4:36 PM Ehud Yonasi <
eyonasi@redhat.com>
wrote:
Hey, I've seen that CQ Master is not passing ovirt-engine for
10
days
and fails on test suite called restore_vm0_networking
here's a snap error regarding it:
Link to a sample job with the error:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifact...
I cannot follow this link because I'm 4 minutes too late
jenkins.ovirt.org uses an invalid security certificate.
The
certificate expired on November 11, 2018, 5:13:25 PM
GMT+2. The
current time is November 11, 2018, 5:17 PM.
Yes, we're looking into that issue now.
Fixed, you should be able to access it now.
OST fails during restore_vm0_networking in line 101 of 004_basic_sanity.py while comparing vm_service.get().status == state
It seems that instead of reporting back the VM status, Engine
set
garbage
"The response content type 'text/html; charset=iso-8859-1'
isn't the
expected XML"
The relevant line in
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifac...
seems to be 192.168.201.1 - - [11/Nov/2018:04:27:43 -0500] "GET /ovirt-engine/api/vms/26088164-d1a0-4254-a377-5d3c242c8105
HTTP/1.1"
503 299
and I guess the 503 error message is sent in HTML instead of XML.
If I run manually https://gerrit.ovirt.org/#/c/95354/ with latest build of engine-master
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/...
basic suite seems to be happy: https://jenkins.ovirt.org/view/oVirt system tests/job/ovirt-system-tests_manual/3484/
I do not know what could cause that, and engine.log does not
mention
it. But it seems like a problem in engine API hence +Martin
Perina
and
+Ondra Machacek .
Can some1 have a look at it and help to resolve the
issue?
_______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement:
https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/ZQAYWTLZJKGPJ2...
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement:
https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/R5LOJH73XCLLFO...
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA
TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA
TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/DA6Q5RE5JO3FYI...
-- Martin Perina Associate Manager, Software Engineering Red Hat Czech s.r.o.
On Tue, 13 Nov 2018 13:01:09 +0100 Martin Perina <mperina@redhat.com> wrote:
On Tue, Nov 13, 2018 at 12:49 PM Michal Skrivanek <mskrivan@redhat.com> wrote:
On 13 Nov 2018, at 12:20, Dominik Holler <dholler@redhat.com> wrote:
On Tue, 13 Nov 2018 11:56:37 +0100 Martin Perina <mperina@redhat.com> wrote:
On Tue, Nov 13, 2018 at 11:02 AM Dafna Ron <dron@redhat.com> wrote:
Martin? can you please look at the patch that Dominik sent? We need to resolve this as we have not had an engine build for the last 11 days
Yesterday I've merged Dominik's revert patch https://gerrit.ovirt.org/95377 which should switch cluster level back to 4.2. Below mentioned change https://gerrit.ovirt.org/95310 is relevant only to cluster level 4.3, am I right Michal?
The build mentioned
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... is from yesterday. Are we sure that it was executed only after #95377 was merged? I'd like to see the results from latest
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... but unfortunately it already waits more than an hour for available hosts ...
https://gerrit.ovirt.org/#/c/95283/ results in
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms succeeding.
The next merged change https://gerrit.ovirt.org/#/c/95310/ results in
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms failing with 2018-11-12 17:35:10,109-05 INFO [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Running command: RunVmOnceCommand internal: false. Entities affected : ID: d10aa133-b9b6-455d-8137-ab822d1c1971 Type: VMAction group RUN_VM with role type USER 2018-11-12 17:35:10,113-05 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] method: getVmManager, params: [d10aa133-b9b6-455d-8137-ab822d1c1971], timeElapsed: 4ms 2018-11-12 17:35:10,128-05 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] method: getAllForClusterWithStatus, params: [2ca9ccd8-61f0-470c-ba3f-07766202f260, Up], timeElapsed: 7ms 2018-11-12 17:35:10,129-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Candidate host 'lago-basic-suite-master-host-1' ('282860ab-8873-4702-a2be-100a6da111af') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 6930b632-5593-4481-bf2a-a1d8b14a583a) 2018-11-12 17:35:10,129-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Candidate host 'lago-basic-suite-master-host-0' ('c48eca36-ea98-46b2-8473-f184833e68a8') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 6930b632-5593-4481-bf2a-a1d8b14a583a) 2018-11-12 17:35:10,130-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Can't find VDS to run the VM 'd10aa133-b9b6-455d-8137-ab822d1c1971' on, so this VM will not be run. in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests...
Is this helpful for you?
actually, there ire two issues 1) cluster is still 4.3 even after Martin’s revert.
https://gerrit.ovirt.org/#/c/95409/ should align cluster level with dc level
This change aligns the cluster level, but https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... consuming build result from https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... looks like that this does not solve the issue: File "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 698, in run_vms api.vms.get(VM0_NAME).start(start_params) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line 31193, in start headers={"Correlation-Id":correlation_id} File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122, in request persistent_auth=self.__persistent_auth File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 79, in do_request persistent_auth) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 162, in __do_request raise errors.RequestError(response_code, response_reason, response_body) RequestError: status: 400 reason: Bad Request engine.log: 2018-11-14 03:10:36,802-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Candidate host 'lago-basic-suite-master-host-0' ('4dbfb937-ac4b-4cef-8ae3-124944829add') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 99e282ea-577a-4dab-857b-285b1df5e6f6) 2018-11-14 03:10:36,802-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Candidate host 'lago-basic-suite-master-host-1' ('731e5055-706e-4310-a062-045e32ffbfeb') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 99e282ea-577a-4dab-857b-285b1df5e6f6) 2018-11-14 03:10:36,802-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Can't find VDS to run the VM 'dc1e1e92-1e5c-415e-8ac2-b919017adf40' on, so this VM will not be run.
2) the patch is wrong too, as in HandleVdsCpuFlagsOrClusterChangedCommand
it just goes ahead and sets the cluster cpu to whatever the host reported regardless if it is valid or not. Steven, please fix that (line 96 in backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/HandleVdsCpuFlagsOrClusterChangedCommand.java). It needs to pass the validation or we need some other solution. 3) regardless, we should make 4.3 work too , I tried to play with it a bit in https://gerrit.ovirt.org/#/c/95407/, let’s see…
Thanks, michal
On Mon, Nov 12, 2018 at 3:58 PM Dominik Holler <dholler@redhat.com> wrote:
On Mon, 12 Nov 2018 13:45:54 +0100 Martin Perina <mperina@redhat.com> wrote:
On Mon, Nov 12, 2018 at 12:58 PM Dominik Holler <dholler@redhat.com>
wrote:
On Mon, 12 Nov 2018 12:29:17 +0100 Martin Perina <mperina@redhat.com> wrote:
On Mon, Nov 12, 2018 at 12:20 PM Dafna Ron <dron@redhat.com> wrote:
There are currently two issues failing ovirt-engine on CQ ovirt
master:
1. edit vm pool is causing failure in different tests. it has a
patch
*waiting
to be merged*: https://gerrit.ovirt.org/#/c/95354/
Merged
2. we have a failure in upgrade suite as well to run vm but this
seems
to
be related to the tests as well: 2018-11-12 05:41:07,831-05 WARN [org.ovirt.engine.core.bll.validator.VirtIoRngValidator]
(default
task-1)
[] Random number source URANDOM is not supported in cluster
'test-cluster'
compatibility version 4.0.
here is the full error from the upgrade suite failure in run vm: https://pastebin.com/XLHtWGGx
Here is the latest failure:
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change...
I will try to take a look later today
I have the idea that this might be related to https://gerrit.ovirt.org/#/c/95377/ , and I check in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests...
, but I have to stop now, if not solved I can go on later today.
OK, both CI and above manual OST job went fine, so I've just merged the revert patch. I will take a look at it later in detail, we should
really be
testing 4.3 on master and not 4.2
Ack.
Now
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... is failing on File
"/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 698, in run_vms api.vms.get(VM0_NAME).start(start_params) status: 400 reason: Bad Request
2018-11-12 10:06:30,722-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Candidate host 'lago-basic-suite-master-host-1' ('dbfe1b0c-f940-4dba-8fb1-0cfe5ca7ddfc') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: b8d11cb0-5be9-4b7e-b45a-c95fa1f18681) 2018-11-12 10:06:30,722-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Candidate host 'lago-basic-suite-master-host-0' ('e83a63ca-381e-40db-acb2-65a3e7953e11') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: b8d11cb0-5be9-4b7e-b45a-c95fa1f18681) 2018-11-12 10:06:30,723-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Can't find VDS to run the VM '57a66eff-8cbf-4643-b045-43d4dda80c66' on, so this VM will not be run.
Is this related to https://gerrit.ovirt.org/#/c/95310/ ?
Thanks, Dafna
On Mon, Nov 12, 2018 at 9:23 AM Dominik Holler <
dholler@redhat.com>
wrote:
On Sun, 11 Nov 2018 19:04:40 +0200 Dan Kenigsberg <danken@redhat.com> wrote:
On Sun, Nov 11, 2018 at 5:27 PM Eyal Edri <eedri@redhat.com>
wrote:
On Sun, Nov 11, 2018 at 5:24 PM Eyal Edri <eedri@redhat.com>
wrote:
On Sun, Nov 11, 2018 at 5:20 PM Dan Kenigsberg <
danken@redhat.com>
wrote:
On Sun, Nov 11, 2018 at 4:36 PM Ehud Yonasi <
eyonasi@redhat.com>
wrote:
Hey, I've seen that CQ Master is not passing ovirt-engine for
10
days
and fails on test suite called restore_vm0_networking
here's a snap error regarding it:
Link to a sample job with the error:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifact...
I cannot follow this link because I'm 4 minutes too late
jenkins.ovirt.org uses an invalid security certificate.
The
certificate expired on November 11, 2018, 5:13:25 PM
GMT+2. The
current time is November 11, 2018, 5:17 PM.
Yes, we're looking into that issue now.
Fixed, you should be able to access it now.
OST fails during restore_vm0_networking in line 101 of 004_basic_sanity.py while comparing vm_service.get().status == state
It seems that instead of reporting back the VM status, Engine
set
garbage
"The response content type 'text/html; charset=iso-8859-1'
isn't the
expected XML"
The relevant line in
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifac...
seems to be 192.168.201.1 - - [11/Nov/2018:04:27:43 -0500] "GET /ovirt-engine/api/vms/26088164-d1a0-4254-a377-5d3c242c8105
HTTP/1.1"
503 299
and I guess the 503 error message is sent in HTML instead of XML.
If I run manually https://gerrit.ovirt.org/#/c/95354/ with latest build of engine-master
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/...
basic suite seems to be happy: https://jenkins.ovirt.org/view/oVirt system tests/job/ovirt-system-tests_manual/3484/
I do not know what could cause that, and engine.log does not
mention
it. But it seems like a problem in engine API hence +Martin
Perina
and
+Ondra Machacek .
Can some1 have a look at it and help to resolve the
issue?
_______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement:
https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/ZQAYWTLZJKGPJ2...
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement:
https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/R5LOJH73XCLLFO...
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA
TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA
TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/DA6Q5RE5JO3FYI...
On Wed, 14 Nov 2018 09:27:39 +0100 Dominik Holler <dholler@redhat.com> wrote:
On Tue, 13 Nov 2018 13:01:09 +0100 Martin Perina <mperina@redhat.com> wrote:
On Tue, Nov 13, 2018 at 12:49 PM Michal Skrivanek <mskrivan@redhat.com> wrote:
On 13 Nov 2018, at 12:20, Dominik Holler <dholler@redhat.com> wrote:
On Tue, 13 Nov 2018 11:56:37 +0100 Martin Perina <mperina@redhat.com> wrote:
On Tue, Nov 13, 2018 at 11:02 AM Dafna Ron <dron@redhat.com> wrote:
Martin? can you please look at the patch that Dominik sent? We need to resolve this as we have not had an engine build for the last 11 days
Yesterday I've merged Dominik's revert patch https://gerrit.ovirt.org/95377 which should switch cluster level back to 4.2. Below mentioned change https://gerrit.ovirt.org/95310 is relevant only to cluster level 4.3, am I right Michal?
The build mentioned
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... is from yesterday. Are we sure that it was executed only after #95377 was merged? I'd like to see the results from latest
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... but unfortunately it already waits more than an hour for available hosts ...
https://gerrit.ovirt.org/#/c/95283/ results in
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms succeeding.
The next merged change https://gerrit.ovirt.org/#/c/95310/ results in
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms failing with 2018-11-12 17:35:10,109-05 INFO [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Running command: RunVmOnceCommand internal: false. Entities affected : ID: d10aa133-b9b6-455d-8137-ab822d1c1971 Type: VMAction group RUN_VM with role type USER 2018-11-12 17:35:10,113-05 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] method: getVmManager, params: [d10aa133-b9b6-455d-8137-ab822d1c1971], timeElapsed: 4ms 2018-11-12 17:35:10,128-05 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] method: getAllForClusterWithStatus, params: [2ca9ccd8-61f0-470c-ba3f-07766202f260, Up], timeElapsed: 7ms 2018-11-12 17:35:10,129-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Candidate host 'lago-basic-suite-master-host-1' ('282860ab-8873-4702-a2be-100a6da111af') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 6930b632-5593-4481-bf2a-a1d8b14a583a) 2018-11-12 17:35:10,129-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Candidate host 'lago-basic-suite-master-host-0' ('c48eca36-ea98-46b2-8473-f184833e68a8') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 6930b632-5593-4481-bf2a-a1d8b14a583a) 2018-11-12 17:35:10,130-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Can't find VDS to run the VM 'd10aa133-b9b6-455d-8137-ab822d1c1971' on, so this VM will not be run. in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests...
Is this helpful for you?
actually, there ire two issues 1) cluster is still 4.3 even after Martin’s revert.
https://gerrit.ovirt.org/#/c/95409/ should align cluster level with dc level
This change aligns the cluster level, but https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... consuming build result from https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... looks like that this does not solve the issue: File "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 698, in run_vms api.vms.get(VM0_NAME).start(start_params) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line 31193, in start headers={"Correlation-Id":correlation_id} File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122, in request persistent_auth=self.__persistent_auth File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 79, in do_request persistent_auth) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 162, in __do_request raise errors.RequestError(response_code, response_reason, response_body) RequestError: status: 400 reason: Bad Request
engine.log: 2018-11-14 03:10:36,802-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Candidate host 'lago-basic-suite-master-host-0' ('4dbfb937-ac4b-4cef-8ae3-124944829add') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 99e282ea-577a-4dab-857b-285b1df5e6f6) 2018-11-14 03:10:36,802-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Candidate host 'lago-basic-suite-master-host-1' ('731e5055-706e-4310-a062-045e32ffbfeb') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 99e282ea-577a-4dab-857b-285b1df5e6f6) 2018-11-14 03:10:36,802-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Can't find VDS to run the VM 'dc1e1e92-1e5c-415e-8ac2-b919017adf40' on, so this VM will not be run.
https://gerrit.ovirt.org/#/c/95283/ results in http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms succeeding. The next merged change https://gerrit.ovirt.org/#/c/95310/ results in http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms failing with File "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 698, in run_vms api.vms.get(VM0_NAME).start(start_params) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line 31193, in start headers={"Correlation-Id":correlation_id} File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122, in request persistent_auth=self.__persistent_auth File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 79, in do_request persistent_auth) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 162, in __do_request raise errors.RequestError(response_code, response_reason, response_body) RequestError: status: 400 reason: Bad Request So even if the Cluster Level should be 4.2 now, still https://gerrit.ovirt.org/#/c/95310/ seems influence the behavior.
2) the patch is wrong too, as in HandleVdsCpuFlagsOrClusterChangedCommand
it just goes ahead and sets the cluster cpu to whatever the host reported regardless if it is valid or not. Steven, please fix that (line 96 in backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/HandleVdsCpuFlagsOrClusterChangedCommand.java). It needs to pass the validation or we need some other solution. 3) regardless, we should make 4.3 work too , I tried to play with it a bit in https://gerrit.ovirt.org/#/c/95407/, let’s see…
Thanks, michal
On Mon, Nov 12, 2018 at 3:58 PM Dominik Holler <dholler@redhat.com> wrote:
On Mon, 12 Nov 2018 13:45:54 +0100 Martin Perina <mperina@redhat.com> wrote:
On Mon, Nov 12, 2018 at 12:58 PM Dominik Holler <dholler@redhat.com>
wrote:
On Mon, 12 Nov 2018 12:29:17 +0100 Martin Perina <mperina@redhat.com> wrote:
On Mon, Nov 12, 2018 at 12:20 PM Dafna Ron <dron@redhat.com> wrote:
There are currently two issues failing ovirt-engine on CQ ovirt
master:
1. edit vm pool is causing failure in different tests. it has a
patch
*waiting
to be merged*: https://gerrit.ovirt.org/#/c/95354/
Merged
2. we have a failure in upgrade suite as well to run vm but this
seems
to
be related to the tests as well: 2018-11-12 05:41:07,831-05 WARN [org.ovirt.engine.core.bll.validator.VirtIoRngValidator]
(default
task-1)
[] Random number source URANDOM is not supported in cluster
'test-cluster'
compatibility version 4.0.
here is the full error from the upgrade suite failure in run vm: https://pastebin.com/XLHtWGGx
Here is the latest failure:
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change...
I will try to take a look later today
I have the idea that this might be related to https://gerrit.ovirt.org/#/c/95377/ , and I check in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests...
, but I have to stop now, if not solved I can go on later today.
OK, both CI and above manual OST job went fine, so I've just merged the revert patch. I will take a look at it later in detail, we should
really be
testing 4.3 on master and not 4.2
Ack.
Now
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... is failing on File
"/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 698, in run_vms api.vms.get(VM0_NAME).start(start_params) status: 400 reason: Bad Request
2018-11-12 10:06:30,722-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Candidate host 'lago-basic-suite-master-host-1' ('dbfe1b0c-f940-4dba-8fb1-0cfe5ca7ddfc') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: b8d11cb0-5be9-4b7e-b45a-c95fa1f18681) 2018-11-12 10:06:30,722-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Candidate host 'lago-basic-suite-master-host-0' ('e83a63ca-381e-40db-acb2-65a3e7953e11') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: b8d11cb0-5be9-4b7e-b45a-c95fa1f18681) 2018-11-12 10:06:30,723-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Can't find VDS to run the VM '57a66eff-8cbf-4643-b045-43d4dda80c66' on, so this VM will not be run.
Is this related to https://gerrit.ovirt.org/#/c/95310/ ?
Thanks, Dafna
On Mon, Nov 12, 2018 at 9:23 AM Dominik Holler <
dholler@redhat.com>
wrote:
On Sun, 11 Nov 2018 19:04:40 +0200 Dan Kenigsberg <danken@redhat.com> wrote:
On Sun, Nov 11, 2018 at 5:27 PM Eyal Edri <eedri@redhat.com>
wrote:
On Sun, Nov 11, 2018 at 5:24 PM Eyal Edri <eedri@redhat.com>
wrote:
On Sun, Nov 11, 2018 at 5:20 PM Dan Kenigsberg <
danken@redhat.com>
wrote:
On Sun, Nov 11, 2018 at 4:36 PM Ehud Yonasi <
eyonasi@redhat.com>
wrote:
Hey, I've seen that CQ Master is not passing ovirt-engine for
10
days
and fails on test suite called restore_vm0_networking
here's a snap error regarding it:
Link to a sample job with the error:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifact...
I cannot follow this link because I'm 4 minutes too late
jenkins.ovirt.org uses an invalid security certificate.
The
certificate expired on November 11, 2018, 5:13:25 PM
GMT+2. The
current time is November 11, 2018, 5:17 PM.
Yes, we're looking into that issue now.
Fixed, you should be able to access it now.
OST fails during restore_vm0_networking in line 101 of 004_basic_sanity.py while comparing vm_service.get().status == state
It seems that instead of reporting back the VM status, Engine
set
garbage
"The response content type 'text/html; charset=iso-8859-1'
isn't the
expected XML"
The relevant line in
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifac...
seems to be 192.168.201.1 - - [11/Nov/2018:04:27:43 -0500] "GET /ovirt-engine/api/vms/26088164-d1a0-4254-a377-5d3c242c8105
HTTP/1.1"
503 299
and I guess the 503 error message is sent in HTML instead of XML.
If I run manually https://gerrit.ovirt.org/#/c/95354/ with latest build of engine-master
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/...
basic suite seems to be happy: https://jenkins.ovirt.org/view/oVirt system tests/job/ovirt-system-tests_manual/3484/
I do not know what could cause that, and engine.log does not
mention
it. But it seems like a problem in engine API hence +Martin
Perina
and
+Ondra Machacek .
Can some1 have a look at it and help to resolve the
issue?
_______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement:
https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/ZQAYWTLZJKGPJ2...
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement:
https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/R5LOJH73XCLLFO...
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA
TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA
TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/DA6Q5RE5JO3FYI...
On 14 Nov 2018, at 10:50, Dominik Holler <dholler@redhat.com> wrote:
On Wed, 14 Nov 2018 09:27:39 +0100 Dominik Holler <dholler@redhat.com> wrote:
On Tue, 13 Nov 2018 13:01:09 +0100 Martin Perina <mperina@redhat.com> wrote:
On Tue, Nov 13, 2018 at 12:49 PM Michal Skrivanek <mskrivan@redhat.com> wrote:
On 13 Nov 2018, at 12:20, Dominik Holler <dholler@redhat.com> wrote:
On Tue, 13 Nov 2018 11:56:37 +0100 Martin Perina <mperina@redhat.com> wrote:
On Tue, Nov 13, 2018 at 11:02 AM Dafna Ron <dron@redhat.com> wrote:
Martin? can you please look at the patch that Dominik sent? We need to resolve this as we have not had an engine build for the last 11 days
Yesterday I've merged Dominik's revert patch https://gerrit.ovirt.org/95377 which should switch cluster level back to 4.2. Below mentioned change https://gerrit.ovirt.org/95310 is relevant only to cluster level 4.3, am I right Michal?
The build mentioned
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... is from yesterday. Are we sure that it was executed only after #95377 was merged? I'd like to see the results from latest
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... but unfortunately it already waits more than an hour for available hosts ...
https://gerrit.ovirt.org/#/c/95283/ results in
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms succeeding.
The next merged change https://gerrit.ovirt.org/#/c/95310/ results in
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms failing with 2018-11-12 17:35:10,109-05 INFO [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Running command: RunVmOnceCommand internal: false. Entities affected : ID: d10aa133-b9b6-455d-8137-ab822d1c1971 Type: VMAction group RUN_VM with role type USER 2018-11-12 17:35:10,113-05 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] method: getVmManager, params: [d10aa133-b9b6-455d-8137-ab822d1c1971], timeElapsed: 4ms 2018-11-12 17:35:10,128-05 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] method: getAllForClusterWithStatus, params: [2ca9ccd8-61f0-470c-ba3f-07766202f260, Up], timeElapsed: 7ms 2018-11-12 17:35:10,129-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Candidate host 'lago-basic-suite-master-host-1' ('282860ab-8873-4702-a2be-100a6da111af') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 6930b632-5593-4481-bf2a-a1d8b14a583a) 2018-11-12 17:35:10,129-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Candidate host 'lago-basic-suite-master-host-0' ('c48eca36-ea98-46b2-8473-f184833e68a8') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 6930b632-5593-4481-bf2a-a1d8b14a583a) 2018-11-12 17:35:10,130-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Can't find VDS to run the VM 'd10aa133-b9b6-455d-8137-ab822d1c1971' on, so this VM will not be run. in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests...
Is this helpful for you?
actually, there ire two issues 1) cluster is still 4.3 even after Martin’s revert.
https://gerrit.ovirt.org/#/c/95409/ should align cluster level with dc level
This change aligns the cluster level, but https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... consuming build result from https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... looks like that this does not solve the issue: File "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 698, in run_vms api.vms.get(VM0_NAME).start(start_params) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line 31193, in start headers={"Correlation-Id":correlation_id} File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122, in request persistent_auth=self.__persistent_auth File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 79, in do_request persistent_auth) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 162, in __do_request raise errors.RequestError(response_code, response_reason, response_body) RequestError: status: 400 reason: Bad Request
engine.log: 2018-11-14 03:10:36,802-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Candidate host 'lago-basic-suite-master-host-0' ('4dbfb937-ac4b-4cef-8ae3-124944829add') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 99e282ea-577a-4dab-857b-285b1df5e6f6) 2018-11-14 03:10:36,802-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Candidate host 'lago-basic-suite-master-host-1' ('731e5055-706e-4310-a062-045e32ffbfeb') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 99e282ea-577a-4dab-857b-285b1df5e6f6) 2018-11-14 03:10:36,802-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Can't find VDS to run the VM 'dc1e1e92-1e5c-415e-8ac2-b919017adf40' on, so this VM will not be run.
https://gerrit.ovirt.org/#/c/95283/ results in http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms succeeding.
The next merged change https://gerrit.ovirt.org/#/c/95310/ results in http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms failing with File "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 698, in run_vms api.vms.get(VM0_NAME).start(start_params) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line 31193, in start headers={"Correlation-Id":correlation_id} File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122, in request persistent_auth=self.__persistent_auth File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 79, in do_request persistent_auth) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 162, in __do_request raise errors.RequestError(response_code, response_reason, response_body) RequestError: status: 400 reason: Bad Request
So even if the Cluster Level should be 4.2 now, still https://gerrit.ovirt.org/#/c/95310/ seems influence the behavior.
I really do not see how it can affect 4.2. Are you sure the cluster is really 4.2? Sadly it’s not being logged at all But if it really seem to matter (and since it needs a fix anyway for 4.3) feel free to revert it of course
2) the patch is wrong too, as in HandleVdsCpuFlagsOrClusterChangedCommand
it just goes ahead and sets the cluster cpu to whatever the host reported regardless if it is valid or not. Steven, please fix that (line 96 in backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/HandleVdsCpuFlagsOrClusterChangedCommand.java). It needs to pass the validation or we need some other solution. 3) regardless, we should make 4.3 work too , I tried to play with it a bit in https://gerrit.ovirt.org/#/c/95407/, let’s see…
Thanks, michal
On Mon, Nov 12, 2018 at 3:58 PM Dominik Holler <dholler@redhat.com> wrote:
On Mon, 12 Nov 2018 13:45:54 +0100 Martin Perina <mperina@redhat.com> wrote:
On Mon, Nov 12, 2018 at 12:58 PM Dominik Holler <dholler@redhat.com>
wrote:
On Mon, 12 Nov 2018 12:29:17 +0100 Martin Perina <mperina@redhat.com> wrote:
On Mon, Nov 12, 2018 at 12:20 PM Dafna Ron <dron@redhat.com> wrote:
There are currently two issues failing ovirt-engine on CQ ovirt
master:
1. edit vm pool is causing failure in different tests. it has a
patch
*waiting
to be merged*: https://gerrit.ovirt.org/#/c/95354/
Merged
2. we have a failure in upgrade suite as well to run vm but this
seems
to
be related to the tests as well: 2018-11-12 05:41:07,831-05 WARN [org.ovirt.engine.core.bll.validator.VirtIoRngValidator]
(default
task-1)
[] Random number source URANDOM is not supported in cluster
'test-cluster'
compatibility version 4.0.
here is the full error from the upgrade suite failure in run vm: https://pastebin.com/XLHtWGGx
Here is the latest failure:
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change...
I will try to take a look later today
I have the idea that this might be related to https://gerrit.ovirt.org/#/c/95377/ , and I check in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests...
, but I have to stop now, if not solved I can go on later today.
OK, both CI and above manual OST job went fine, so I've just merged the revert patch. I will take a look at it later in detail, we should
really be
testing 4.3 on master and not 4.2
Ack.
Now
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... is failing on File
"/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 698, in run_vms api.vms.get(VM0_NAME).start(start_params) status: 400 reason: Bad Request
2018-11-12 10:06:30,722-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Candidate host 'lago-basic-suite-master-host-1' ('dbfe1b0c-f940-4dba-8fb1-0cfe5ca7ddfc') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: b8d11cb0-5be9-4b7e-b45a-c95fa1f18681) 2018-11-12 10:06:30,722-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Candidate host 'lago-basic-suite-master-host-0' ('e83a63ca-381e-40db-acb2-65a3e7953e11') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: b8d11cb0-5be9-4b7e-b45a-c95fa1f18681) 2018-11-12 10:06:30,723-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Can't find VDS to run the VM '57a66eff-8cbf-4643-b045-43d4dda80c66' on, so this VM will not be run.
Is this related to https://gerrit.ovirt.org/#/c/95310/ ?
Thanks, Dafna
On Mon, Nov 12, 2018 at 9:23 AM Dominik Holler <
dholler@redhat.com>
wrote:
On Sun, 11 Nov 2018 19:04:40 +0200 Dan Kenigsberg <danken@redhat.com> wrote:
On Sun, Nov 11, 2018 at 5:27 PM Eyal Edri <eedri@redhat.com>
wrote:
On Sun, Nov 11, 2018 at 5:24 PM Eyal Edri <eedri@redhat.com>
wrote:
On Sun, Nov 11, 2018 at 5:20 PM Dan Kenigsberg <
danken@redhat.com>
wrote:
On Sun, Nov 11, 2018 at 4:36 PM Ehud Yonasi <
eyonasi@redhat.com>
wrote:
Hey, I've seen that CQ Master is not passing ovirt-engine for
10
days
and fails on test suite called restore_vm0_networking
here's a snap error regarding it:
Link to a sample job with the error:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifact...
I cannot follow this link because I'm 4 minutes too late
jenkins.ovirt.org uses an invalid security certificate.
The
certificate expired on November 11, 2018, 5:13:25 PM
GMT+2. The
current time is November 11, 2018, 5:17 PM.
Yes, we're looking into that issue now.
Fixed, you should be able to access it now.
OST fails during restore_vm0_networking in line 101 of 004_basic_sanity.py while comparing vm_service.get().status == state
It seems that instead of reporting back the VM status, Engine
set
garbage
"The response content type 'text/html; charset=iso-8859-1'
isn't the
expected XML"
The relevant line in
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifac...
seems to be 192.168.201.1 - - [11/Nov/2018:04:27:43 -0500] "GET /ovirt-engine/api/vms/26088164-d1a0-4254-a377-5d3c242c8105
HTTP/1.1"
503 299
and I guess the 503 error message is sent in HTML instead of XML.
If I run manually https://gerrit.ovirt.org/#/c/95354/ with latest build of engine-master
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/...
basic suite seems to be happy: https://jenkins.ovirt.org/view/oVirt system tests/job/ovirt-system-tests_manual/3484/
I do not know what could cause that, and engine.log does not
mention
it. But it seems like a problem in engine API hence +Martin
Perina
and
+Ondra Machacek .
Can some1 have a look at it and help to resolve the
issue?
_______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement:
https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/ZQAYWTLZJKGPJ2...
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement:
https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/R5LOJH73XCLLFO...
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA
TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA
TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/DA6Q5RE5JO3FYI...
On Wed, 14 Nov 2018 11:24:10 +0100 Michal Skrivanek <mskrivan@redhat.com> wrote:
On 14 Nov 2018, at 10:50, Dominik Holler <dholler@redhat.com> wrote:
On Wed, 14 Nov 2018 09:27:39 +0100 Dominik Holler <dholler@redhat.com> wrote:
On Tue, 13 Nov 2018 13:01:09 +0100 Martin Perina <mperina@redhat.com> wrote:
On Tue, Nov 13, 2018 at 12:49 PM Michal Skrivanek <mskrivan@redhat.com> wrote:
On 13 Nov 2018, at 12:20, Dominik Holler <dholler@redhat.com> wrote:
On Tue, 13 Nov 2018 11:56:37 +0100 Martin Perina <mperina@redhat.com> wrote:
On Tue, Nov 13, 2018 at 11:02 AM Dafna Ron <dron@redhat.com> wrote:
Martin? can you please look at the patch that Dominik sent? We need to resolve this as we have not had an engine build for the last 11 days
Yesterday I've merged Dominik's revert patch https://gerrit.ovirt.org/95377 which should switch cluster level back to 4.2. Below mentioned change https://gerrit.ovirt.org/95310 is relevant only to cluster level 4.3, am I right Michal?
The build mentioned
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... is from yesterday. Are we sure that it was executed only after #95377 was merged? I'd like to see the results from latest
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... but unfortunately it already waits more than an hour for available hosts ...
https://gerrit.ovirt.org/#/c/95283/ results in
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms succeeding.
The next merged change https://gerrit.ovirt.org/#/c/95310/ results in
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms failing with 2018-11-12 17:35:10,109-05 INFO [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Running command: RunVmOnceCommand internal: false. Entities affected : ID: d10aa133-b9b6-455d-8137-ab822d1c1971 Type: VMAction group RUN_VM with role type USER 2018-11-12 17:35:10,113-05 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] method: getVmManager, params: [d10aa133-b9b6-455d-8137-ab822d1c1971], timeElapsed: 4ms 2018-11-12 17:35:10,128-05 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] method: getAllForClusterWithStatus, params: [2ca9ccd8-61f0-470c-ba3f-07766202f260, Up], timeElapsed: 7ms 2018-11-12 17:35:10,129-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Candidate host 'lago-basic-suite-master-host-1' ('282860ab-8873-4702-a2be-100a6da111af') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 6930b632-5593-4481-bf2a-a1d8b14a583a) 2018-11-12 17:35:10,129-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Candidate host 'lago-basic-suite-master-host-0' ('c48eca36-ea98-46b2-8473-f184833e68a8') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 6930b632-5593-4481-bf2a-a1d8b14a583a) 2018-11-12 17:35:10,130-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Can't find VDS to run the VM 'd10aa133-b9b6-455d-8137-ab822d1c1971' on, so this VM will not be run. in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests...
Is this helpful for you?
actually, there ire two issues 1) cluster is still 4.3 even after Martin’s revert.
https://gerrit.ovirt.org/#/c/95409/ should align cluster level with dc level
This change aligns the cluster level, but https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... consuming build result from https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... looks like that this does not solve the issue: File "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 698, in run_vms api.vms.get(VM0_NAME).start(start_params) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line 31193, in start headers={"Correlation-Id":correlation_id} File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122, in request persistent_auth=self.__persistent_auth File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 79, in do_request persistent_auth) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 162, in __do_request raise errors.RequestError(response_code, response_reason, response_body) RequestError: status: 400 reason: Bad Request
engine.log: 2018-11-14 03:10:36,802-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Candidate host 'lago-basic-suite-master-host-0' ('4dbfb937-ac4b-4cef-8ae3-124944829add') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 99e282ea-577a-4dab-857b-285b1df5e6f6) 2018-11-14 03:10:36,802-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Candidate host 'lago-basic-suite-master-host-1' ('731e5055-706e-4310-a062-045e32ffbfeb') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 99e282ea-577a-4dab-857b-285b1df5e6f6) 2018-11-14 03:10:36,802-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Can't find VDS to run the VM 'dc1e1e92-1e5c-415e-8ac2-b919017adf40' on, so this VM will not be run.
https://gerrit.ovirt.org/#/c/95283/ results in http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms succeeding.
The next merged change https://gerrit.ovirt.org/#/c/95310/ results in http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms failing with File "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 698, in run_vms api.vms.get(VM0_NAME).start(start_params) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line 31193, in start headers={"Correlation-Id":correlation_id} File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122, in request persistent_auth=self.__persistent_auth File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 79, in do_request persistent_auth) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 162, in __do_request raise errors.RequestError(response_code, response_reason, response_body) RequestError: status: 400 reason: Bad Request
So even if the Cluster Level should be 4.2 now, still https://gerrit.ovirt.org/#/c/95310/ seems influence the behavior.
I really do not see how it can affect 4.2.
Me neither.
Are you sure the cluster is really 4.2? Sadly it’s not being logged at all
screenshot from local execution https://imgur.com/a/yiWBw3c
But if it really seem to matter (and since it needs a fix anyway for 4.3) feel free to revert it of course
I will post a revert change and check if this changes the behavior.
2) the patch is wrong too, as in HandleVdsCpuFlagsOrClusterChangedCommand
it just goes ahead and sets the cluster cpu to whatever the host reported regardless if it is valid or not. Steven, please fix that (line 96 in backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/HandleVdsCpuFlagsOrClusterChangedCommand.java). It needs to pass the validation or we need some other solution. 3) regardless, we should make 4.3 work too , I tried to play with it a bit in https://gerrit.ovirt.org/#/c/95407/, let’s see…
Thanks, michal
On Mon, Nov 12, 2018 at 3:58 PM Dominik Holler <dholler@redhat.com> wrote:
On Mon, 12 Nov 2018 13:45:54 +0100 Martin Perina <mperina@redhat.com> wrote:
On Mon, Nov 12, 2018 at 12:58 PM Dominik Holler <dholler@redhat.com>
wrote:
On Mon, 12 Nov 2018 12:29:17 +0100 Martin Perina <mperina@redhat.com> wrote:
On Mon, Nov 12, 2018 at 12:20 PM Dafna Ron <dron@redhat.com> wrote:
There are currently two issues failing ovirt-engine on CQ ovirt
master:
1. edit vm pool is causing failure in different tests. it has a
patch
*waiting
to be merged*: https://gerrit.ovirt.org/#/c/95354/
Merged
2. we have a failure in upgrade suite as well to run vm but this
seems
to
be related to the tests as well: 2018-11-12 05:41:07,831-05 WARN [org.ovirt.engine.core.bll.validator.VirtIoRngValidator]
(default
task-1)
[] Random number source URANDOM is not supported in cluster
'test-cluster'
compatibility version 4.0.
here is the full error from the upgrade suite failure in run vm: https://pastebin.com/XLHtWGGx
Here is the latest failure:
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change...
I will try to take a look later today
I have the idea that this might be related to https://gerrit.ovirt.org/#/c/95377/ , and I check in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests...
, but I have to stop now, if not solved I can go on later today.
OK, both CI and above manual OST job went fine, so I've just merged the revert patch. I will take a look at it later in detail, we should
really be
testing 4.3 on master and not 4.2
Ack.
Now
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... is failing on File
"/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 698, in run_vms api.vms.get(VM0_NAME).start(start_params) status: 400 reason: Bad Request
2018-11-12 10:06:30,722-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Candidate host 'lago-basic-suite-master-host-1' ('dbfe1b0c-f940-4dba-8fb1-0cfe5ca7ddfc') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: b8d11cb0-5be9-4b7e-b45a-c95fa1f18681) 2018-11-12 10:06:30,722-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Candidate host 'lago-basic-suite-master-host-0' ('e83a63ca-381e-40db-acb2-65a3e7953e11') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: b8d11cb0-5be9-4b7e-b45a-c95fa1f18681) 2018-11-12 10:06:30,723-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Can't find VDS to run the VM '57a66eff-8cbf-4643-b045-43d4dda80c66' on, so this VM will not be run.
Is this related to https://gerrit.ovirt.org/#/c/95310/ ?
Thanks, Dafna
On Mon, Nov 12, 2018 at 9:23 AM Dominik Holler <
dholler@redhat.com>
wrote:
On Sun, 11 Nov 2018 19:04:40 +0200 Dan Kenigsberg <danken@redhat.com> wrote:
On Sun, Nov 11, 2018 at 5:27 PM Eyal Edri <eedri@redhat.com>
wrote:
On Sun, Nov 11, 2018 at 5:24 PM Eyal Edri <eedri@redhat.com>
wrote:
On Sun, Nov 11, 2018 at 5:20 PM Dan Kenigsberg <
danken@redhat.com>
wrote:
On Sun, Nov 11, 2018 at 4:36 PM Ehud Yonasi <
eyonasi@redhat.com>
wrote:
Hey, I've seen that CQ Master is not passing ovirt-engine for
10
days
and fails on test suite called restore_vm0_networking
here's a snap error regarding it:
Link to a sample job with the error:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifact...
I cannot follow this link because I'm 4 minutes too late
jenkins.ovirt.org uses an invalid security certificate.
The
certificate expired on November 11, 2018, 5:13:25 PM
GMT+2. The
current time is November 11, 2018, 5:17 PM.
Yes, we're looking into that issue now.
Fixed, you should be able to access it now.
OST fails during restore_vm0_networking in line 101 of 004_basic_sanity.py while comparing vm_service.get().status == state
It seems that instead of reporting back the VM status, Engine
set
garbage
"The response content type 'text/html; charset=iso-8859-1'
isn't the
expected XML"
The relevant line in
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifac...
seems to be 192.168.201.1 - - [11/Nov/2018:04:27:43 -0500] "GET /ovirt-engine/api/vms/26088164-d1a0-4254-a377-5d3c242c8105
HTTP/1.1"
503 299
and I guess the 503 error message is sent in HTML instead of XML.
If I run manually https://gerrit.ovirt.org/#/c/95354/ with latest build of engine-master
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/...
basic suite seems to be happy: https://jenkins.ovirt.org/view/oVirt system tests/job/ovirt-system-tests_manual/3484/
I do not know what could cause that, and engine.log does not
mention
it. But it seems like a problem in engine API hence +Martin
Perina
and
+Ondra Machacek .
Can some1 have a look at it and help to resolve the
issue?
_______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement:
https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/ZQAYWTLZJKGPJ2...
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement:
https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/R5LOJH73XCLLFO...
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA
TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA
TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/DA6Q5RE5JO3FYI...
On Wed, Nov 14, 2018 at 12:42 PM Dominik Holler <dholler@redhat.com> wrote:
On Wed, 14 Nov 2018 11:24:10 +0100 Michal Skrivanek <mskrivan@redhat.com> wrote:
On 14 Nov 2018, at 10:50, Dominik Holler <dholler@redhat.com> wrote:
On Wed, 14 Nov 2018 09:27:39 +0100 Dominik Holler <dholler@redhat.com> wrote:
On Tue, 13 Nov 2018 13:01:09 +0100 Martin Perina <mperina@redhat.com> wrote:
On Tue, Nov 13, 2018 at 12:49 PM Michal Skrivanek <mskrivan@redhat.com> wrote:
On 13 Nov 2018, at 12:20, Dominik Holler <dholler@redhat.com> wrote:
On Tue, 13 Nov 2018 11:56:37 +0100 Martin Perina <mperina@redhat.com> wrote:
On Tue, Nov 13, 2018 at 11:02 AM Dafna Ron <dron@redhat.com> wrote:
Martin? can you please look at the patch that Dominik sent? We need to resolve this as we have not had an engine build for the last 11 days
Yesterday I've merged Dominik's revert patch https://gerrit.ovirt.org/95377 which should switch cluster level back to 4.2. Below mentioned change https://gerrit.ovirt.org/95310 is relevant only to cluster level 4.3, am I right Michal?
The build mentioned
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... is from yesterday. Are we sure that it was executed only after #95377 was merged? I'd like to see the results from latest
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... but unfortunately it already waits more than an hour for available hosts ...
https://gerrit.ovirt.org/#/c/95283/ results in
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms succeeding.
The next merged change https://gerrit.ovirt.org/#/c/95310/ results in
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms failing with 2018-11-12 17:35:10,109-05 INFO [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Running command: RunVmOnceCommand internal: false. Entities affected : ID: d10aa133-b9b6-455d-8137-ab822d1c1971 Type: VMAction group RUN_VM with role type USER 2018-11-12 17:35:10,113-05 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] method: getVmManager, params: [d10aa133-b9b6-455d-8137-ab822d1c1971], timeElapsed: 4ms 2018-11-12 17:35:10,128-05 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] method: getAllForClusterWithStatus, params: [2ca9ccd8-61f0-470c-ba3f-07766202f260, Up], timeElapsed: 7ms 2018-11-12 17:35:10,129-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Candidate host 'lago-basic-suite-master-host-1' ('282860ab-8873-4702-a2be-100a6da111af') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 6930b632-5593-4481-bf2a-a1d8b14a583a) 2018-11-12 17:35:10,129-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Candidate host 'lago-basic-suite-master-host-0' ('c48eca36-ea98-46b2-8473-f184833e68a8') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 6930b632-5593-4481-bf2a-a1d8b14a583a) 2018-11-12 17:35:10,130-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Can't find VDS to run the VM 'd10aa133-b9b6-455d-8137-ab822d1c1971' on, so this VM will not be run. in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests...
Is this helpful for you?
actually, there ire two issues 1) cluster is still 4.3 even after Martin’s revert.
https://gerrit.ovirt.org/#/c/95409/ should align cluster level with dc level
This change aligns the cluster level, but https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... consuming build result from https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... looks like that this does not solve the issue: File "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 698, in run_vms api.vms.get(VM0_NAME).start(start_params) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line 31193, in start headers={"Correlation-Id":correlation_id} File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122, in request persistent_auth=self.__persistent_auth File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 79, in do_request persistent_auth) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 162, in __do_request raise errors.RequestError(response_code, response_reason, response_body) RequestError: status: 400 reason: Bad Request
engine.log: 2018-11-14 03:10:36,802-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Candidate host 'lago-basic-suite-master-host-0' ('4dbfb937-ac4b-4cef-8ae3-124944829add') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 99e282ea-577a-4dab-857b-285b1df5e6f6) 2018-11-14 03:10:36,802-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Candidate host 'lago-basic-suite-master-host-1' ('731e5055-706e-4310-a062-045e32ffbfeb') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 99e282ea-577a-4dab-857b-285b1df5e6f6) 2018-11-14 03:10:36,802-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Can't find VDS to run the VM 'dc1e1e92-1e5c-415e-8ac2-b919017adf40' on, so this VM will not be run.
https://gerrit.ovirt.org/#/c/95283/ results in http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms succeeding.
The next merged change https://gerrit.ovirt.org/#/c/95310/ results in http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms failing with File "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 698, in run_vms api.vms.get(VM0_NAME).start(start_params) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line 31193, in start headers={"Correlation-Id":correlation_id} File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122, in request persistent_auth=self.__persistent_auth File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 79, in do_request persistent_auth) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 162, in __do_request raise errors.RequestError(response_code, response_reason, response_body) RequestError: status: 400 reason: Bad Request
So even if the Cluster Level should be 4.2 now, still https://gerrit.ovirt.org/#/c/95310/ seems influence the behavior.
I really do not see how it can affect 4.2.
Me neither.
Are you sure the cluster is really 4.2? Sadly it’s not being logged at all
screenshot from local execution https://imgur.com/a/yiWBw3c
But if it really seem to matter (and since it needs a fix anyway for 4.3) feel free to revert it of course
I will post a revert change and check if this changes the behavior.
Dominik, thanks for the research and for Martin's and your reverts/fixes. Finally Engine passes OST https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11153/ and QE can expect a build tomorrow, after 2 weeks of droughts.
2) the patch is wrong too, as in HandleVdsCpuFlagsOrClusterChangedCommand
it just goes ahead and sets the cluster cpu to whatever the host reported regardless if it is valid or not. Steven, please fix that (line 96 in backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/HandleVdsCpuFlagsOrClusterChangedCommand.java). It needs to pass the validation or we need some other solution. 3) regardless, we should make 4.3 work too , I tried to play with it a bit in https://gerrit.ovirt.org/#/c/95407/, let’s see…
Thanks, michal
On Mon, Nov 12, 2018 at 3:58 PM Dominik Holler <dholler@redhat.com> wrote:
On Mon, 12 Nov 2018 13:45:54 +0100 Martin Perina <mperina@redhat.com> wrote:
On Mon, Nov 12, 2018 at 12:58 PM Dominik Holler <dholler@redhat.com>
wrote:
On Mon, 12 Nov 2018 12:29:17 +0100 Martin Perina <mperina@redhat.com> wrote:
On Mon, Nov 12, 2018 at 12:20 PM Dafna Ron <dron@redhat.com> wrote:
There are currently two issues failing ovirt-engine on CQ ovirt
master:
1. edit vm pool is causing failure in different tests. it has a
patch
*waiting
to be merged*: https://gerrit.ovirt.org/#/c/95354/
Merged
2. we have a failure in upgrade suite as well to run vm but this
seems
to
be related to the tests as well: 2018-11-12 05:41:07,831-05 WARN [org.ovirt.engine.core.bll.validator.VirtIoRngValidator]
(default
task-1)
[] Random number source URANDOM is not supported in cluster
'test-cluster'
compatibility version 4.0.
here is the full error from the upgrade suite failure in run vm: https://pastebin.com/XLHtWGGx
Here is the latest failure:
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change...
I will try to take a look later today
I have the idea that this might be related to https://gerrit.ovirt.org/#/c/95377/ , and I check in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests...
, but I have to stop now, if not solved I can go on later today.
OK, both CI and above manual OST job went fine, so I've just merged the revert patch. I will take a look at it later in detail, we should
really be
testing 4.3 on master and not 4.2
Ack.
Now
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... is failing on File
"/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 698, in run_vms api.vms.get(VM0_NAME).start(start_params) status: 400 reason: Bad Request
2018-11-12 10:06:30,722-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Candidate host 'lago-basic-suite-master-host-1' ('dbfe1b0c-f940-4dba-8fb1-0cfe5ca7ddfc') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: b8d11cb0-5be9-4b7e-b45a-c95fa1f18681) 2018-11-12 10:06:30,722-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Candidate host 'lago-basic-suite-master-host-0' ('e83a63ca-381e-40db-acb2-65a3e7953e11') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: b8d11cb0-5be9-4b7e-b45a-c95fa1f18681) 2018-11-12 10:06:30,723-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-3) [b8d11cb0-5be9-4b7e-b45a-c95fa1f18681] Can't find VDS to run the VM '57a66eff-8cbf-4643-b045-43d4dda80c66' on, so this VM will not be run.
Is this related to https://gerrit.ovirt.org/#/c/95310/ ?
Thanks, Dafna
On Mon, Nov 12, 2018 at 9:23 AM Dominik Holler <
dholler@redhat.com>
wrote:
On Sun, 11 Nov 2018 19:04:40 +0200 Dan Kenigsberg <danken@redhat.com> wrote:
On Sun, Nov 11, 2018 at 5:27 PM Eyal Edri <eedri@redhat.com>
wrote:
On Sun, Nov 11, 2018 at 5:24 PM Eyal Edri <eedri@redhat.com>
wrote:
On Sun, Nov 11, 2018 at 5:20 PM Dan Kenigsberg <
danken@redhat.com>
wrote:
On Sun, Nov 11, 2018 at 4:36 PM Ehud Yonasi <
eyonasi@redhat.com>
wrote:
Hey, I've seen that CQ Master is not passing ovirt-engine for
10
days
and fails on test suite called restore_vm0_networking
here's a snap error regarding it:
Link to a sample job with the error:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifact...
I cannot follow this link because I'm 4 minutes too late
jenkins.ovirt.org uses an invalid security certificate.
The
certificate expired on November 11, 2018, 5:13:25 PM
GMT+2. The
current time is November 11, 2018, 5:17 PM.
Yes, we're looking into that issue now.
Fixed, you should be able to access it now.
OST fails during restore_vm0_networking in line 101 of 004_basic_sanity.py while comparing vm_service.get().status == state
It seems that instead of reporting back the VM status, Engine
set
garbage
"The response content type 'text/html; charset=iso-8859-1'
isn't the
expected XML"
The relevant line in
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11113/artifac...
seems to be 192.168.201.1 - - [11/Nov/2018:04:27:43 -0500] "GET /ovirt-engine/api/vms/26088164-d1a0-4254-a377-5d3c242c8105
HTTP/1.1"
503 299
and I guess the 503 error message is sent in HTML instead of XML.
If I run manually https://gerrit.ovirt.org/#/c/95354/ with latest build of engine-master
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/...
basic suite seems to be happy: https://jenkins.ovirt.org/view/oVirt system tests/job/ovirt-system-tests_manual/3484/
I do not know what could cause that, and engine.log does not
mention
it. But it seems like a problem in engine API hence +Martin
Perina
and
+Ondra Machacek .
Can some1 have a look at it and help to resolve the
issue?
_______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement:
https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/ZQAYWTLZJKGPJ2...
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement:
https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/R5LOJH73XCLLFO...
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA
TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA
TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/devel@ovirt.org/message/DA6Q5RE5JO3FYI...
_______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/6PC2MYTQXFL4PQ...
On Wed, Nov 14, 2018 at 5:07 PM Dan Kenigsberg <danken@redhat.com> wrote:
On Wed, Nov 14, 2018 at 12:42 PM Dominik Holler <dholler@redhat.com> wrote:
On Wed, 14 Nov 2018 11:24:10 +0100 Michal Skrivanek <mskrivan@redhat.com> wrote:
On 14 Nov 2018, at 10:50, Dominik Holler <dholler@redhat.com> wrote:
On Wed, 14 Nov 2018 09:27:39 +0100 Dominik Holler <dholler@redhat.com> wrote:
On Tue, 13 Nov 2018 13:01:09 +0100 Martin Perina <mperina@redhat.com> wrote:
On Tue, Nov 13, 2018 at 12:49 PM Michal Skrivanek <mskrivan@redhat.com> wrote:
> > > On 13 Nov 2018, at 12:20, Dominik Holler <dholler@redhat.com> wrote: > > On Tue, 13 Nov 2018 11:56:37 +0100 > Martin Perina <mperina@redhat.com> wrote: > > On Tue, Nov 13, 2018 at 11:02 AM Dafna Ron <dron@redhat.com> wrote: > > Martin? can you please look at the patch that Dominik sent? > We need to resolve this as we have not had an engine build for the last 11 > days > > > Yesterday I've merged Dominik's revert patch > https://gerrit.ovirt.org/95377 > which should switch cluster level back to 4.2. Below mentioned change > https://gerrit.ovirt.org/95310 is relevant only to cluster level 4.3, am I > right Michal? > > The build mentioned > > https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... > is from yesterday. Are we sure that it was executed only after #95377 was > merged? I'd like to see the results from latest > > https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... > but unfortunately it already waits more than an hour for available hosts > ... > > > > > > https://gerrit.ovirt.org/#/c/95283/ results in > > http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... > which is used in > > https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... > results in run_vms succeeding. > > The next merged change > https://gerrit.ovirt.org/#/c/95310/ results in > > http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... > which is used in > > https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... > results in run_vms failing with > 2018-11-12 17:35:10,109-05 INFO > [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-1) > [6930b632-5593-4481-bf2a-a1d8b14a583a] Running command: RunVmOnceCommand > internal: false. Entities affected : ID: > d10aa133-b9b6-455d-8137-ab822d1c1971 Type: VMAction group RUN_VM with role > type USER > 2018-11-12 17:35:10,113-05 DEBUG > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] method: > getVmManager, params: [d10aa133-b9b6-455d-8137-ab822d1c1971], timeElapsed: > 4ms > 2018-11-12 17:35:10,128-05 DEBUG > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] method: > getAllForClusterWithStatus, params: [2ca9ccd8-61f0-470c-ba3f-07766202f260, > Up], timeElapsed: 7ms > 2018-11-12 17:35:10,129-05 INFO > [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-1) > [6930b632-5593-4481-bf2a-a1d8b14a583a] Candidate host > 'lago-basic-suite-master-host-1' ('282860ab-8873-4702-a2be-100a6da111af') > was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' > (correlation id: 6930b632-5593-4481-bf2a-a1d8b14a583a) > 2018-11-12 17:35:10,129-05 INFO > [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-1) > [6930b632-5593-4481-bf2a-a1d8b14a583a] Candidate host > 'lago-basic-suite-master-host-0' ('c48eca36-ea98-46b2-8473-f184833e68a8') > was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' > (correlation id: 6930b632-5593-4481-bf2a-a1d8b14a583a) > 2018-11-12 17:35:10,130-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] > (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Can't find VDS to > run the VM 'd10aa133-b9b6-455d-8137-ab822d1c1971' on, so this VM will not > be run. > in > > https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... > > Is this helpful for you? > > > > actually, there ire two issues > 1) cluster is still 4.3 even after Martin’s revert. >
https://gerrit.ovirt.org/#/c/95409/ should align cluster level with dc level
This change aligns the cluster level, but https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... consuming build result from https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... looks like that this does not solve the issue: File "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 698, in run_vms api.vms.get(VM0_NAME).start(start_params) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line 31193, in start headers={"Correlation-Id":correlation_id} File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122, in request persistent_auth=self.__persistent_auth File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 79, in do_request persistent_auth) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 162, in __do_request raise errors.RequestError(response_code, response_reason, response_body) RequestError: status: 400 reason: Bad Request
engine.log: 2018-11-14 03:10:36,802-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Candidate host 'lago-basic-suite-master-host-0' ('4dbfb937-ac4b-4cef-8ae3-124944829add') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 99e282ea-577a-4dab-857b-285b1df5e6f6) 2018-11-14 03:10:36,802-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Candidate host 'lago-basic-suite-master-host-1' ('731e5055-706e-4310-a062-045e32ffbfeb') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 99e282ea-577a-4dab-857b-285b1df5e6f6) 2018-11-14 03:10:36,802-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Can't find VDS to run the VM 'dc1e1e92-1e5c-415e-8ac2-b919017adf40' on, so this VM will not be run.
https://gerrit.ovirt.org/#/c/95283/ results in http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms succeeding.
The next merged change https://gerrit.ovirt.org/#/c/95310/ results in http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms failing with File "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 698, in run_vms api.vms.get(VM0_NAME).start(start_params) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line 31193, in start headers={"Correlation-Id":correlation_id} File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122, in request persistent_auth=self.__persistent_auth File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 79, in do_request persistent_auth) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 162, in __do_request raise errors.RequestError(response_code, response_reason, response_body) RequestError: status: 400 reason: Bad Request
So even if the Cluster Level should be 4.2 now, still https://gerrit.ovirt.org/#/c/95310/ seems influence the behavior.
I really do not see how it can affect 4.2.
Me neither.
Are you sure the cluster is really 4.2? Sadly it’s not being logged at all
screenshot from local execution https://imgur.com/a/yiWBw3c
But if it really seem to matter (and since it needs a fix anyway for 4.3) feel free to revert it of course
I will post a revert change and check if this changes the behavior.
Dominik, thanks for the research and for Martin's and your reverts/fixes. Finally Engine passes OST https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11153/ and QE can expect a build tomorrow, after 2 weeks of droughts.
unfortunately, the drought continues. Barrak tells me that something is broken in the nightly cron job copying the the tested repo onto the master-snapshot one. +Edri: please make it a priority to have it fixed.
On Thu, Nov 15, 2018 at 12:43 PM Dan Kenigsberg <danken@redhat.com> wrote:
On Wed, Nov 14, 2018 at 5:07 PM Dan Kenigsberg <danken@redhat.com> wrote:
On Wed, Nov 14, 2018 at 12:42 PM Dominik Holler <dholler@redhat.com>
On Wed, 14 Nov 2018 11:24:10 +0100 Michal Skrivanek <mskrivan@redhat.com> wrote:
On 14 Nov 2018, at 10:50, Dominik Holler <dholler@redhat.com>
wrote:
On Wed, 14 Nov 2018 09:27:39 +0100 Dominik Holler <dholler@redhat.com> wrote:
On Tue, 13 Nov 2018 13:01:09 +0100 Martin Perina <mperina@redhat.com> wrote:
> On Tue, Nov 13, 2018 at 12:49 PM Michal Skrivanek <
mskrivan@redhat.com>
> wrote: > >> >> >> On 13 Nov 2018, at 12:20, Dominik Holler <dholler@redhat.com> wrote: >> >> On Tue, 13 Nov 2018 11:56:37 +0100 >> Martin Perina <mperina@redhat.com> wrote: >> >> On Tue, Nov 13, 2018 at 11:02 AM Dafna Ron <dron@redhat.com> wrote: >> >> Martin? can you please look at the patch that Dominik sent? >> We need to resolve this as we have not had an engine build for
>> days >> >> >> Yesterday I've merged Dominik's revert patch >> https://gerrit.ovirt.org/95377 >> which should switch cluster level back to 4.2. Below mentioned change >> https://gerrit.ovirt.org/95310 is relevant only to cluster level 4.3, am I >> right Michal? >> >> The build mentioned >> >> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... >> is from yesterday. Are we sure that it was executed only after #95377 was >> merged? I'd like to see the results from latest >> >> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... >> but unfortunately it already waits more than an hour for available hosts >> ... >> >> >> >> >> >> https://gerrit.ovirt.org/#/c/95283/ results in >> >> http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... >> which is used in >> >> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... >> results in run_vms succeeding. >> >> The next merged change >> https://gerrit.ovirt.org/#/c/95310/ results in >> >> http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... >> which is used in >> >> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... >> results in run_vms failing with >> 2018-11-12 17:35:10,109-05 INFO >> [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-1) >> [6930b632-5593-4481-bf2a-a1d8b14a583a] Running command: RunVmOnceCommand >> internal: false. Entities affected : ID: >> d10aa133-b9b6-455d-8137-ab822d1c1971 Type: VMAction group RUN_VM with role >> type USER >> 2018-11-12 17:35:10,113-05 DEBUG >> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >> (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] method: >> getVmManager, params: [d10aa133-b9b6-455d-8137-ab822d1c1971], timeElapsed: >> 4ms >> 2018-11-12 17:35:10,128-05 DEBUG >> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >> (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] method: >> getAllForClusterWithStatus, params: [2ca9ccd8-61f0-470c-ba3f-07766202f260, >> Up], timeElapsed: 7ms >> 2018-11-12 17:35:10,129-05 INFO >> [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-1) >> [6930b632-5593-4481-bf2a-a1d8b14a583a] Candidate host >> 'lago-basic-suite-master-host-1' ('282860ab-8873-4702-a2be-100a6da111af') >> was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' >> (correlation id: 6930b632-5593-4481-bf2a-a1d8b14a583a) >> 2018-11-12 17:35:10,129-05 INFO >> [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-1) >> [6930b632-5593-4481-bf2a-a1d8b14a583a] Candidate host >> 'lago-basic-suite-master-host-0' ('c48eca36-ea98-46b2-8473-f184833e68a8') >> was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' >> (correlation id: 6930b632-5593-4481-bf2a-a1d8b14a583a) >> 2018-11-12 17:35:10,130-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] >> (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Can't find VDS to >> run the VM 'd10aa133-b9b6-455d-8137-ab822d1c1971' on, so this VM will not >> be run. >> in >> >> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... >> >> Is this helpful for you? >> >> >> >> actually, there ire two issues >> 1) cluster is still 4.3 even after Martin’s revert. >> > > https://gerrit.ovirt.org/#/c/95409/ should align cluster level with dc level >
This change aligns the cluster level, but
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests...
consuming build result from
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change...
looks like that this does not solve the issue: File "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py",
api.vms.get(VM0_NAME).start(start_params) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line 31193, in start headers={"Correlation-Id":correlation_id} File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122, in request persistent_auth=self.__persistent_auth File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py",
persistent_auth) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py",
raise errors.RequestError(response_code, response_reason, response_body) RequestError: status: 400 reason: Bad Request
engine.log: 2018-11-14 03:10:36,802-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Candidate host 'lago-basic-suite-master-host-0' ('4dbfb937-ac4b-4cef-8ae3-124944829add') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 99e282ea-577a-4dab-857b-285b1df5e6f6) 2018-11-14 03:10:36,802-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Candidate host 'lago-basic-suite-master-host-1' ('731e5055-706e-4310-a062-045e32ffbfeb') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 99e282ea-577a-4dab-857b-285b1df5e6f6) 2018-11-14 03:10:36,802-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Can't find VDS to run the VM 'dc1e1e92-1e5c-415e-8ac2-b919017adf40' on, so this VM will not be run.
https://gerrit.ovirt.org/#/c/95283/ results in
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/...
which is used in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests...
results in run_vms succeeding.
The next merged change https://gerrit.ovirt.org/#/c/95310/ results in
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/...
which is used in
https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests...
results in run_vms failing with File "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py",
api.vms.get(VM0_NAME).start(start_params) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line 31193, in start headers={"Correlation-Id":correlation_id} File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122, in request persistent_auth=self.__persistent_auth File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py",
persistent_auth) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py",
wrote: the last 11 line 698, in run_vms line 79, in do_request line 162, in __do_request line 698, in run_vms line 79, in do_request line 162, in __do_request
raise errors.RequestError(response_code, response_reason, response_body) RequestError: status: 400 reason: Bad Request
So even if the Cluster Level should be 4.2 now, still https://gerrit.ovirt.org/#/c/95310/ seems influence the behavior.
I really do not see how it can affect 4.2.
Me neither.
Are you sure the cluster is really 4.2? Sadly it’s not being logged at all
screenshot from local execution https://imgur.com/a/yiWBw3c
But if it really seem to matter (and since it needs a fix anyway for 4.3) feel free to revert it of course
I will post a revert change and check if this changes the behavior.
Dominik, thanks for the research and for Martin's and your reverts/fixes. Finally Engine passes OST https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11153/ and QE can expect a build tomorrow, after 2 weeks of droughts.
unfortunately, the drought continues.
Sorry, missing the content or meaning, what does drought means?
Barrak tells me that something is broken in the nightly cron job copying the the tested repo onto the master-snapshot one.
Dafna, can you check this?
+Edri: please make it a priority to have it fixed.
-- Eyal edri MANAGER RHV/CNV DevOps EMEA VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
On Thu, Nov 15, 2018 at 12:45 PM Eyal Edri <eedri@redhat.com> wrote:
On Thu, Nov 15, 2018 at 12:43 PM Dan Kenigsberg <danken@redhat.com> wrote:
On Wed, Nov 14, 2018 at 5:07 PM Dan Kenigsberg <danken@redhat.com> wrote:
On Wed, Nov 14, 2018 at 12:42 PM Dominik Holler <dholler@redhat.com> wrote:
On Wed, 14 Nov 2018 11:24:10 +0100 Michal Skrivanek <mskrivan@redhat.com> wrote:
On 14 Nov 2018, at 10:50, Dominik Holler <dholler@redhat.com> wrote:
On Wed, 14 Nov 2018 09:27:39 +0100 Dominik Holler <dholler@redhat.com> wrote:
> On Tue, 13 Nov 2018 13:01:09 +0100 > Martin Perina <mperina@redhat.com> wrote: > >> On Tue, Nov 13, 2018 at 12:49 PM Michal Skrivanek <mskrivan@redhat.com> >> wrote: >> >>> >>> >>> On 13 Nov 2018, at 12:20, Dominik Holler <dholler@redhat.com> wrote: >>> >>> On Tue, 13 Nov 2018 11:56:37 +0100 >>> Martin Perina <mperina@redhat.com> wrote: >>> >>> On Tue, Nov 13, 2018 at 11:02 AM Dafna Ron <dron@redhat.com> wrote: >>> >>> Martin? can you please look at the patch that Dominik sent? >>> We need to resolve this as we have not had an engine build for the last 11 >>> days >>> >>> >>> Yesterday I've merged Dominik's revert patch >>> https://gerrit.ovirt.org/95377 >>> which should switch cluster level back to 4.2. Below mentioned change >>> https://gerrit.ovirt.org/95310 is relevant only to cluster level 4.3, am I >>> right Michal? >>> >>> The build mentioned >>> >>> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... >>> is from yesterday. Are we sure that it was executed only after #95377 was >>> merged? I'd like to see the results from latest >>> >>> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... >>> but unfortunately it already waits more than an hour for available hosts >>> ... >>> >>> >>> >>> >>> >>> https://gerrit.ovirt.org/#/c/95283/ results in >>> >>> http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... >>> which is used in >>> >>> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... >>> results in run_vms succeeding. >>> >>> The next merged change >>> https://gerrit.ovirt.org/#/c/95310/ results in >>> >>> http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... >>> which is used in >>> >>> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... >>> results in run_vms failing with >>> 2018-11-12 17:35:10,109-05 INFO >>> [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-1) >>> [6930b632-5593-4481-bf2a-a1d8b14a583a] Running command: RunVmOnceCommand >>> internal: false. Entities affected : ID: >>> d10aa133-b9b6-455d-8137-ab822d1c1971 Type: VMAction group RUN_VM with role >>> type USER >>> 2018-11-12 17:35:10,113-05 DEBUG >>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >>> (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] method: >>> getVmManager, params: [d10aa133-b9b6-455d-8137-ab822d1c1971], timeElapsed: >>> 4ms >>> 2018-11-12 17:35:10,128-05 DEBUG >>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >>> (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] method: >>> getAllForClusterWithStatus, params: [2ca9ccd8-61f0-470c-ba3f-07766202f260, >>> Up], timeElapsed: 7ms >>> 2018-11-12 17:35:10,129-05 INFO >>> [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-1) >>> [6930b632-5593-4481-bf2a-a1d8b14a583a] Candidate host >>> 'lago-basic-suite-master-host-1' ('282860ab-8873-4702-a2be-100a6da111af') >>> was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' >>> (correlation id: 6930b632-5593-4481-bf2a-a1d8b14a583a) >>> 2018-11-12 17:35:10,129-05 INFO >>> [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-1) >>> [6930b632-5593-4481-bf2a-a1d8b14a583a] Candidate host >>> 'lago-basic-suite-master-host-0' ('c48eca36-ea98-46b2-8473-f184833e68a8') >>> was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' >>> (correlation id: 6930b632-5593-4481-bf2a-a1d8b14a583a) >>> 2018-11-12 17:35:10,130-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] >>> (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Can't find VDS to >>> run the VM 'd10aa133-b9b6-455d-8137-ab822d1c1971' on, so this VM will not >>> be run. >>> in >>> >>> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... >>> >>> Is this helpful for you? >>> >>> >>> >>> actually, there ire two issues >>> 1) cluster is still 4.3 even after Martin’s revert. >>> >> >> https://gerrit.ovirt.org/#/c/95409/ should align cluster level with dc level >> > > This change aligns the cluster level, but > https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... > consuming build result from > https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... > looks like that this does not solve the issue: > File "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 698, in run_vms > api.vms.get(VM0_NAME).start(start_params) > File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line 31193, in start > headers={"Correlation-Id":correlation_id} > File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122, in request > persistent_auth=self.__persistent_auth > File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 79, in do_request > persistent_auth) > File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 162, in __do_request > raise errors.RequestError(response_code, response_reason, response_body) > RequestError: > status: 400 > reason: Bad Request > > engine.log: > 2018-11-14 03:10:36,802-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Candidate host 'lago-basic-suite-master-host-0' ('4dbfb937-ac4b-4cef-8ae3-124944829add') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 99e282ea-577a-4dab-857b-285b1df5e6f6) > 2018-11-14 03:10:36,802-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Candidate host 'lago-basic-suite-master-host-1' ('731e5055-706e-4310-a062-045e32ffbfeb') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 99e282ea-577a-4dab-857b-285b1df5e6f6) > 2018-11-14 03:10:36,802-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Can't find VDS to run the VM 'dc1e1e92-1e5c-415e-8ac2-b919017adf40' on, so this VM will not be run. > >
https://gerrit.ovirt.org/#/c/95283/ results in http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms succeeding.
The next merged change https://gerrit.ovirt.org/#/c/95310/ results in http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... which is used in https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... results in run_vms failing with File "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py", line 698, in run_vms api.vms.get(VM0_NAME).start(start_params) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line 31193, in start headers={"Correlation-Id":correlation_id} File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122, in request persistent_auth=self.__persistent_auth File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 79, in do_request persistent_auth) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 162, in __do_request raise errors.RequestError(response_code, response_reason, response_body) RequestError: status: 400 reason: Bad Request
So even if the Cluster Level should be 4.2 now, still https://gerrit.ovirt.org/#/c/95310/ seems influence the behavior.
I really do not see how it can affect 4.2.
Me neither.
Are you sure the cluster is really 4.2? Sadly it’s not being logged at all
screenshot from local execution https://imgur.com/a/yiWBw3c
But if it really seem to matter (and since it needs a fix anyway for 4.3) feel free to revert it of course
I will post a revert change and check if this changes the behavior.
Dominik, thanks for the research and for Martin's and your reverts/fixes. Finally Engine passes OST https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11153/ and QE can expect a build tomorrow, after 2 weeks of droughts.
unfortunately, the drought continues.
Sorry, missing the content or meaning, what does drought means?
Pardon my flowery language. I mean 2 weeks of no ovirt-engine builds.
Barrak tells me that something is broken in the nightly cron job copying the the tested repo onto the master-snapshot one.
Dafna, can you check this?
+Edri: please make it a priority to have it fixed.
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA
TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
I am checking the failed jobs However, Please note that I think you are confusing issues. Currently, we (CI) have a problem in the job that syncs the package to the snapshot repo. this jobs run nightly and we had no way of knowing it would fail until today. Before today, we had several regressions which lasted for two weeks which means no package was build at all. So different issues On Thu, Nov 15, 2018 at 10:54 AM Dan Kenigsberg <danken@redhat.com> wrote:
On Thu, Nov 15, 2018 at 12:45 PM Eyal Edri <eedri@redhat.com> wrote:
On Thu, Nov 15, 2018 at 12:43 PM Dan Kenigsberg <danken@redhat.com>
On Wed, Nov 14, 2018 at 5:07 PM Dan Kenigsberg <danken@redhat.com>
wrote:
On Wed, Nov 14, 2018 at 12:42 PM Dominik Holler <dholler@redhat.com>
wrote:
On Wed, 14 Nov 2018 11:24:10 +0100 Michal Skrivanek <mskrivan@redhat.com> wrote:
> On 14 Nov 2018, at 10:50, Dominik Holler <dholler@redhat.com>
wrote:
> > On Wed, 14 Nov 2018 09:27:39 +0100 > Dominik Holler <dholler@redhat.com> wrote: > >> On Tue, 13 Nov 2018 13:01:09 +0100 >> Martin Perina <mperina@redhat.com> wrote: >> >>> On Tue, Nov 13, 2018 at 12:49 PM Michal Skrivanek < mskrivan@redhat.com> >>> wrote: >>> >>>> >>>> >>>> On 13 Nov 2018, at 12:20, Dominik Holler <dholler@redhat.com> wrote: >>>> >>>> On Tue, 13 Nov 2018 11:56:37 +0100 >>>> Martin Perina <mperina@redhat.com> wrote: >>>> >>>> On Tue, Nov 13, 2018 at 11:02 AM Dafna Ron <dron@redhat.com> wrote: >>>> >>>> Martin? can you please look at the patch that Dominik sent? >>>> We need to resolve this as we have not had an engine build for the last 11 >>>> days >>>> >>>> >>>> Yesterday I've merged Dominik's revert patch >>>> https://gerrit.ovirt.org/95377 >>>> which should switch cluster level back to 4.2. Below mentioned change >>>> https://gerrit.ovirt.org/95310 is relevant only to cluster level 4.3, am I >>>> right Michal? >>>> >>>> The build mentioned >>>> >>>> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... >>>> is from yesterday. Are we sure that it was executed only after #95377 was >>>> merged? I'd like to see the results from latest >>>> >>>> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... >>>> but unfortunately it already waits more than an hour for available hosts >>>> ... >>>> >>>> >>>> >>>> >>>> >>>> https://gerrit.ovirt.org/#/c/95283/ results in >>>> >>>> http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... >>>> which is used in >>>> >>>> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... >>>> results in run_vms succeeding. >>>> >>>> The next merged change >>>> https://gerrit.ovirt.org/#/c/95310/ results in >>>> >>>> http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... >>>> which is used in >>>> >>>> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... >>>> results in run_vms failing with >>>> 2018-11-12 17:35:10,109-05 INFO >>>> [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-1) >>>> [6930b632-5593-4481-bf2a-a1d8b14a583a] Running command: RunVmOnceCommand >>>> internal: false. Entities affected : ID: >>>> d10aa133-b9b6-455d-8137-ab822d1c1971 Type: VMAction group RUN_VM with role >>>> type USER >>>> 2018-11-12 17:35:10,113-05 DEBUG >>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >>>> (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] method: >>>> getVmManager, params: [d10aa133-b9b6-455d-8137-ab822d1c1971], timeElapsed: >>>> 4ms >>>> 2018-11-12 17:35:10,128-05 DEBUG >>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >>>> (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] method: >>>> getAllForClusterWithStatus, params: [2ca9ccd8-61f0-470c-ba3f-07766202f260, >>>> Up], timeElapsed: 7ms >>>> 2018-11-12 17:35:10,129-05 INFO >>>> [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-1) >>>> [6930b632-5593-4481-bf2a-a1d8b14a583a] Candidate host >>>> 'lago-basic-suite-master-host-1' ('282860ab-8873-4702-a2be-100a6da111af') >>>> was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' >>>> (correlation id: 6930b632-5593-4481-bf2a-a1d8b14a583a) >>>> 2018-11-12 17:35:10,129-05 INFO >>>> [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-1) >>>> [6930b632-5593-4481-bf2a-a1d8b14a583a] Candidate host >>>> 'lago-basic-suite-master-host-0' ('c48eca36-ea98-46b2-8473-f184833e68a8') >>>> was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' >>>> (correlation id: 6930b632-5593-4481-bf2a-a1d8b14a583a) >>>> 2018-11-12 17:35:10,130-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] >>>> (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Can't find VDS to >>>> run the VM 'd10aa133-b9b6-455d-8137-ab822d1c1971' on, so
>>>> be run. >>>> in >>>> >>>> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... >>>> >>>> Is this helpful for you? >>>> >>>> >>>> >>>> actually, there ire two issues >>>> 1) cluster is still 4.3 even after Martin’s revert. >>>> >>> >>> https://gerrit.ovirt.org/#/c/95409/ should align cluster level with dc level >>> >> >> This change aligns the cluster level, but >> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... >> consuming build result from >> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... >> looks like that this does not solve the issue: >> File "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py",
>> api.vms.get(VM0_NAME).start(start_params) >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line 31193, in start >> headers={"Correlation-Id":correlation_id} >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122, in request >> persistent_auth=self.__persistent_auth >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py",
>> persistent_auth) >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py",
>> raise errors.RequestError(response_code, response_reason, response_body) >> RequestError: >> status: 400 >> reason: Bad Request >> >> engine.log: >> 2018-11-14 03:10:36,802-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Candidate host 'lago-basic-suite-master-host-0' ('4dbfb937-ac4b-4cef-8ae3-124944829add') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 99e282ea-577a-4dab-857b-285b1df5e6f6) >> 2018-11-14 03:10:36,802-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Candidate host 'lago-basic-suite-master-host-1' ('731e5055-706e-4310-a062-045e32ffbfeb') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 99e282ea-577a-4dab-857b-285b1df5e6f6) >> 2018-11-14 03:10:36,802-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Can't find VDS to run the VM 'dc1e1e92-1e5c-415e-8ac2-b919017adf40' on, so this VM will not be run. >> >> > > > https://gerrit.ovirt.org/#/c/95283/ results in > http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... > which is used in > https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... > results in run_vms succeeding. > > The next merged change > https://gerrit.ovirt.org/#/c/95310/ results in > http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... > which is used in > https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... > results in run_vms failing with > File "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py",
> api.vms.get(VM0_NAME).start(start_params) > File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line 31193, in start > headers={"Correlation-Id":correlation_id} > File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122, in request > persistent_auth=self.__persistent_auth > File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py",
> persistent_auth) > File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py",
wrote: this VM will not line 698, in run_vms line 79, in do_request line 162, in __do_request line 698, in run_vms line 79, in do_request line 162, in __do_request
> raise errors.RequestError(response_code, response_reason, response_body) > RequestError: > status: 400 > reason: Bad Request > > > So even if the Cluster Level should be 4.2 now, > still https://gerrit.ovirt.org/#/c/95310/ seems influence the behavior.
I really do not see how it can affect 4.2.
Me neither.
Are you sure the cluster is really 4.2? Sadly it’s not being logged at all
screenshot from local execution https://imgur.com/a/yiWBw3c
But if it really seem to matter (and since it needs a fix anyway for 4.3) feel free to revert it of course
I will post a revert change and check if this changes the behavior.
Dominik, thanks for the research and for Martin's and your reverts/fixes. Finally Engine passes OST https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11153/ and QE can expect a build tomorrow, after 2 weeks of droughts.
unfortunately, the drought continues.
Sorry, missing the content or meaning, what does drought means?
Pardon my flowery language. I mean 2 weeks of no ovirt-engine builds.
Barrak tells me that something is broken in the nightly cron job copying the the tested repo onto the master-snapshot one.
Dafna, can you check this?
+Edri: please make it a priority to have it fixed.
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA
TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
On Thu, Nov 15, 2018 at 1:11 PM Dafna Ron <dron@redhat.com> wrote:
I am checking the failed jobs However, Please note that I think you are confusing issues. Currently, we (CI) have a problem in the job that syncs the package to the snapshot repo. this jobs run nightly and we had no way of knowing it would fail until today. Before today, we had several regressions which lasted for two weeks which means no package was build at all. So different issues
No confusion here. There have been multiple production bugs that blocked CQ for 2 weeks. Only now we are blocked on an automation bug. I hope you fix it soon, since quite justly, QE do not want to consume a repo that may change midday.
On Thu, 15 Nov 2018 at 13:11, Dafna Ron <dron@redhat.com> wrote:
I am checking the failed jobs However, Please note that I think you are confusing issues. Currently, we (CI) have a problem in the job that syncs the package to the snapshot repo. this jobs run nightly and we had no way of knowing it would fail until today. Before today, we had several regressions which lasted for two weeks which means no package was build at all. So different issues
It should be fixed now
On Thu, Nov 15, 2018 at 10:54 AM Dan Kenigsberg <danken@redhat.com> wrote:
On Thu, Nov 15, 2018 at 12:45 PM Eyal Edri <eedri@redhat.com> wrote:
On Thu, Nov 15, 2018 at 12:43 PM Dan Kenigsberg <danken@redhat.com>
On Wed, Nov 14, 2018 at 5:07 PM Dan Kenigsberg <danken@redhat.com>
wrote:
On Wed, Nov 14, 2018 at 12:42 PM Dominik Holler <dholler@redhat.com>
wrote:
On Wed, 14 Nov 2018 11:24:10 +0100 Michal Skrivanek <mskrivan@redhat.com> wrote:
> > On 14 Nov 2018, at 10:50, Dominik Holler <dholler@redhat.com>
wrote:
> > > > On Wed, 14 Nov 2018 09:27:39 +0100 > > Dominik Holler <dholler@redhat.com> wrote: > > > >> On Tue, 13 Nov 2018 13:01:09 +0100 > >> Martin Perina <mperina@redhat.com> wrote: > >> > >>> On Tue, Nov 13, 2018 at 12:49 PM Michal Skrivanek < mskrivan@redhat.com> > >>> wrote: > >>> > >>>> > >>>> > >>>> On 13 Nov 2018, at 12:20, Dominik Holler < dholler@redhat.com> wrote: > >>>> > >>>> On Tue, 13 Nov 2018 11:56:37 +0100 > >>>> Martin Perina <mperina@redhat.com> wrote: > >>>> > >>>> On Tue, Nov 13, 2018 at 11:02 AM Dafna Ron <dron@redhat.com> wrote: > >>>> > >>>> Martin? can you please look at the patch that Dominik sent? > >>>> We need to resolve this as we have not had an engine build for the last 11 > >>>> days > >>>> > >>>> > >>>> Yesterday I've merged Dominik's revert patch > >>>> https://gerrit.ovirt.org/95377 > >>>> which should switch cluster level back to 4.2. Below mentioned change > >>>> https://gerrit.ovirt.org/95310 is relevant only to cluster level 4.3, am I > >>>> right Michal? > >>>> > >>>> The build mentioned > >>>> > >>>> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... > >>>> is from yesterday. Are we sure that it was executed only after #95377 was > >>>> merged? I'd like to see the results from latest > >>>> > >>>> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... > >>>> but unfortunately it already waits more than an hour for available hosts > >>>> ... > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> https://gerrit.ovirt.org/#/c/95283/ results in > >>>> > >>>> http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... > >>>> which is used in > >>>> > >>>> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... > >>>> results in run_vms succeeding. > >>>> > >>>> The next merged change > >>>> https://gerrit.ovirt.org/#/c/95310/ results in > >>>> > >>>> http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... > >>>> which is used in > >>>> > >>>> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... > >>>> results in run_vms failing with > >>>> 2018-11-12 17:35:10,109-05 INFO > >>>> [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-1) > >>>> [6930b632-5593-4481-bf2a-a1d8b14a583a] Running command: RunVmOnceCommand > >>>> internal: false. Entities affected : ID: > >>>> d10aa133-b9b6-455d-8137-ab822d1c1971 Type: VMAction group RUN_VM with role > >>>> type USER > >>>> 2018-11-12 17:35:10,113-05 DEBUG > >>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > >>>> (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] method: > >>>> getVmManager, params: [d10aa133-b9b6-455d-8137-ab822d1c1971], timeElapsed: > >>>> 4ms > >>>> 2018-11-12 17:35:10,128-05 DEBUG > >>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > >>>> (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] method: > >>>> getAllForClusterWithStatus, params: [2ca9ccd8-61f0-470c-ba3f-07766202f260, > >>>> Up], timeElapsed: 7ms > >>>> 2018-11-12 17:35:10,129-05 INFO > >>>> [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-1) > >>>> [6930b632-5593-4481-bf2a-a1d8b14a583a] Candidate host > >>>> 'lago-basic-suite-master-host-1' ('282860ab-8873-4702-a2be-100a6da111af') > >>>> was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' > >>>> (correlation id: 6930b632-5593-4481-bf2a-a1d8b14a583a) > >>>> 2018-11-12 17:35:10,129-05 INFO > >>>> [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-1) > >>>> [6930b632-5593-4481-bf2a-a1d8b14a583a] Candidate host > >>>> 'lago-basic-suite-master-host-0' ('c48eca36-ea98-46b2-8473-f184833e68a8') > >>>> was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' > >>>> (correlation id: 6930b632-5593-4481-bf2a-a1d8b14a583a) > >>>> 2018-11-12 17:35:10,130-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] > >>>> (default task-1) [6930b632-5593-4481-bf2a-a1d8b14a583a] Can't find VDS to > >>>> run the VM 'd10aa133-b9b6-455d-8137-ab822d1c1971' on, so
> >>>> be run. > >>>> in > >>>> > >>>> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... > >>>> > >>>> Is this helpful for you? > >>>> > >>>> > >>>> > >>>> actually, there ire two issues > >>>> 1) cluster is still 4.3 even after Martin’s revert. > >>>> > >>> > >>> https://gerrit.ovirt.org/#/c/95409/ should align cluster level with dc level > >>> > >> > >> This change aligns the cluster level, but > >> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... > >> consuming build result from > >> https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change... > >> looks like that this does not solve the issue: > >> File "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py",
> >> api.vms.get(VM0_NAME).start(start_params) > >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line 31193, in start > >> headers={"Correlation-Id":correlation_id} > >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122, in request > >> persistent_auth=self.__persistent_auth > >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py",
> >> persistent_auth) > >> File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py",
> >> raise errors.RequestError(response_code, response_reason, response_body) > >> RequestError: > >> status: 400 > >> reason: Bad Request > >> > >> engine.log: > >> 2018-11-14 03:10:36,802-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Candidate host 'lago-basic-suite-master-host-0' ('4dbfb937-ac4b-4cef-8ae3-124944829add') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 99e282ea-577a-4dab-857b-285b1df5e6f6) > >> 2018-11-14 03:10:36,802-05 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Candidate host 'lago-basic-suite-master-host-1' ('731e5055-706e-4310-a062-045e32ffbfeb') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'CPU-Level' (correlation id: 99e282ea-577a-4dab-857b-285b1df5e6f6) > >> 2018-11-14 03:10:36,802-05 ERROR [org.ovirt.engine.core.bll.RunVmCommand] (default task-3) [99e282ea-577a-4dab-857b-285b1df5e6f6] Can't find VDS to run the VM 'dc1e1e92-1e5c-415e-8ac2-b919017adf40' on, so this VM will not be run. > >> > >> > > > > > > https://gerrit.ovirt.org/#/c/95283/ results in > > http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... > > which is used in > > https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... > > results in run_vms succeeding. > > > > The next merged change > > https://gerrit.ovirt.org/#/c/95310/ results in > > http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64/... > > which is used in > > https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests... > > results in run_vms failing with > > File "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/basic-suite-master/test-scenarios/004_basic_sanity.py",
> > api.vms.get(VM0_NAME).start(start_params) > > File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line 31193, in start > > headers={"Correlation-Id":correlation_id} > > File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122, in request > > persistent_auth=self.__persistent_auth > > File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py",
> > persistent_auth) > > File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py",
wrote: this VM will not line 698, in run_vms line 79, in do_request line 162, in __do_request line 698, in run_vms line 79, in do_request line 162, in __do_request
> > raise errors.RequestError(response_code, response_reason, response_body) > > RequestError: > > status: 400 > > reason: Bad Request > > > > > > So even if the Cluster Level should be 4.2 now, > > still https://gerrit.ovirt.org/#/c/95310/ seems influence the behavior. > > I really do not see how it can affect 4.2.
Me neither.
> Are you sure the cluster is really 4.2? Sadly it’s not being logged at all
screenshot from local execution https://imgur.com/a/yiWBw3c
> But if it really seem to matter (and since it needs a fix anyway for 4.3) feel free to revert it of course >
I will post a revert change and check if this changes the behavior.
Dominik, thanks for the research and for Martin's and your reverts/fixes. Finally Engine passes OST
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11153/
and QE can expect a build tomorrow, after 2 weeks of droughts.
unfortunately, the drought continues.
Sorry, missing the content or meaning, what does drought means?
Pardon my flowery language. I mean 2 weeks of no ovirt-engine builds.
Barrak tells me that something is broken in the nightly cron job copying the the tested repo onto the master-snapshot one.
Dafna, can you check this?
+Edri: please make it a priority to have it fixed.
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA
TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
On Thu, Nov 15, 2018 at 1:16 PM Barak Korren <bkorren@redhat.com> wrote:
On Thu, 15 Nov 2018 at 13:11, Dafna Ron <dron@redhat.com> wrote:
I am checking the failed jobs However, Please note that I think you are confusing issues. Currently, we (CI) have a problem in the job that syncs the package to the snapshot repo. this jobs run nightly and we had no way of knowing it would fail until today. Before today, we had several regressions which lasted for two weeks which means no package was build at all. So different issues
It should be fixed now
Would you trigger it now (mid-day!)? master-snapshot still carries the ancient ovirt-engine-0:4.3.0-0.0.master.20181101091940.git61310aa
On Thu, Nov 15, 2018 at 11:20 AM Dan Kenigsberg <danken@redhat.com> wrote:
On Thu, Nov 15, 2018 at 1:16 PM Barak Korren <bkorren@redhat.com> wrote:
On Thu, 15 Nov 2018 at 13:11, Dafna Ron <dron@redhat.com> wrote:
I am checking the failed jobs However, Please note that I think you are confusing issues. Currently, we (CI) have a problem in the job that syncs the package to
the snapshot repo. this jobs run nightly and we had no way of knowing it would fail until today.
Before today, we had several regressions which lasted for two weeks which means no package was build at all. So different issues
It should be fixed now
Would you trigger it now (mid-day!)? master-snapshot still carries the ancient ovirt-engine-0:4.3.0-0.0.master.20181101091940.git61310aa
As this is the way to fix the issue, It ran. They have the new package: ovirt-engine-0:4.3.0-0.0.master.20181114214053.gitee7737e.el7.noarch
On Thu, Nov 15, 2018 at 1:29 PM Dafna Ron <dron@redhat.com> wrote:
On Thu, Nov 15, 2018 at 11:20 AM Dan Kenigsberg <danken@redhat.com> wrote:
On Thu, Nov 15, 2018 at 1:16 PM Barak Korren <bkorren@redhat.com> wrote:
On Thu, 15 Nov 2018 at 13:11, Dafna Ron <dron@redhat.com> wrote:
I am checking the failed jobs However, Please note that I think you are confusing issues. Currently, we (CI) have a problem in the job that syncs the package to the snapshot repo. this jobs run nightly and we had no way of knowing it would fail until today. Before today, we had several regressions which lasted for two weeks which means no package was build at all. So different issues
It should be fixed now
Would you trigger it now (mid-day!)? master-snapshot still carries the ancient ovirt-engine-0:4.3.0-0.0.master.20181101091940.git61310aa
As this is the way to fix the issue, It ran. They have the new package: ovirt-engine-0:4.3.0-0.0.master.20181114214053.gitee7737e.el7.noarch
yes you are right; `dnf clean all` fixed it on my side too :-/
participants (8)
-
Barak Korren -
Dafna Ron -
Dan Kenigsberg -
Dominik Holler -
Ehud Yonasi -
Eyal Edri -
Martin Perina -
Michal Skrivanek