September 2018 - Infra - oVirt List Archives

[JIRA] (OVIRT-2497) ovirt-master in project vdsm failing on migrate vm test
by Dafna Ron (oVirt JIRA) 17 Sep '18

17 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2497?page=com.atlassian.jira.… ] Dafna Ron updated OVIRT-2497: ----------------------------- Resolution: Fixed Status: Done (was: To Do) > ovirt-master in project vdsm failing on migrate vm test > ------------------------------------------------------- > > Key: OVIRT-2497 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2497 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Dafna Ron > Assignee: infra > Priority: Highest > Labels: ost_code_regression, ost_failures > > we have a failure in migrate vm for project vdsm > full logs can be found here: > https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10309/artifa… > reported patch and root cause patch do not seem related to the failure. > I suspect this is the change that caused the issue: > https://gerrit.ovirt.org/#/c/94248/ > error: https://pastebin.com/XNGFgfvL -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100092)

1 0

[JIRA] (OVIRT-2497) ovirt-master in project vdsm failing on migrate vm test
by Dafna Ron (oVirt JIRA) 17 Sep '18

17 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2497?page=com.atlassian.jira.… ] Dafna Ron commented on OVIRT-2497: ---------------------------------- https://gerrit.ovirt.org/#/c/94347/ this was fixed. > ovirt-master in project vdsm failing on migrate vm test > ------------------------------------------------------- > > Key: OVIRT-2497 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2497 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Dafna Ron > Assignee: infra > Priority: Highest > Labels: ost_code_regression, ost_failures > > we have a failure in migrate vm for project vdsm > full logs can be found here: > https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10309/artifa… > reported patch and root cause patch do not seem related to the failure. > I suspect this is the change that caused the issue: > https://gerrit.ovirt.org/#/c/94248/ > error: https://pastebin.com/XNGFgfvL -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100092)

1 0

[ OST Failure Report ] [ oVirt Master (vdsm) ] [ 14-08-2018 ] [ 006_migrations.migrate_vm ]
by Dafna Ron 17 Sep '18

17 Sep '18

Hi, We have a failure in vm migration on project vdsm master branch. I think the issue is related to this change: https://gerrit.ovirt.org/#/c/94248/ - https://gerrit.ovirt.org/#/c/94248/ Can you please have a look? full logs: https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10309/artifa… error: 2018-09-14 07:15:43,102-0400 ERROR (migsrc/ade7210f) [virt.vm] (vmId='ade7210f-992e-4256-a33f-1340552892c9') __init__() takes exactly 3 arguments (2 given) (migration:281) 2018-09-14 07:15:43,102-0400 DEBUG (migsrc/ade7210f) [jsonrpc.Notification] Sending event {"params": {"notify_time": 4296575510, "ade7210f-992e-4256-a33f-1340552892c9": {"status": "Up"}}, "jsonrpc": "2.0", "method": "|virt|VM_status|ade72 10f-992e-4256-a33f-1340552892c9"} (__init__:181) 2018-09-14 07:15:43,103-0400 DEBUG (JsonRpc (StompReactor)) [yajsonrpc.protocols.stomp.AsyncClient] Stomp connection established (stompclient:138) 2018-09-14 07:15:43,104-0400 DEBUG (migsrc/ade7210f) [vds] Sending notification |virt|VM_status|ade7210f-992e-4256-a33f-1340552892c9 with params {'notify_time': 4296575510, 'ade7210f-992e-4256-a33f-1340552892c9': {'status': 'Up'}} (clien tIF:225) 2018-09-14 07:15:43,104-0400 ERROR (migsrc/ade7210f) [virt.vm] (vmId='ade7210f-992e-4256-a33f-1340552892c9') Failed to migrate (migration:449) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/virt/migration.py", line 401, in _regular_run self._setupVdsConnection() File "/usr/lib/python2.7/site-packages/vdsm/virt/migration.py", line 241, in _setupVdsConnection self._destServer = jsonrpcvdscli.connect(requestQueue, client) File "/usr/lib/python2.7/site-packages/vdsm/jsonrpcvdscli.py", line 259, in connect return _Server(client, xml_compat) File "/usr/lib/python2.7/site-packages/vdsm/jsonrpcvdscli.py", line 135, in __init__ self._schema = vdsmapi.Schema.vdsm_api(api_strict_mode) File "/usr/lib/python2.7/site-packages/vdsm/api/vdsmapi.py", line 155, in vdsm_api return Schema(schema_types, *args, **kwargs) TypeError: __init__() takes exactly 3 arguments (2 given)

2 3

[oVirt Jenkins] ovirt-appliance_4.2_build-artifacts-el7-x86_64 - Build # 197 - Failure!
by jenkins＠jenkins.phx.ovirt.org 17 Sep '18

17 Sep '18

Project: http://jenkins.ovirt.org/job/ovirt-appliance_4.2_build-artifacts-el7-x86_64/ Build: http://jenkins.ovirt.org/job/ovirt-appliance_4.2_build-artifacts-el7-x86_64… Build Number: 197 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #197 [Gal Ben Haim] Add jobs for ovs-cni [Yuval Turgeman] build: add ovirt-engine-extension-aaa-ldap-setup ----------------- Failed Tests: ----------------- No tests ran.

1 4

jira issues for didi
by Yedidyah Bar David 17 Sep '18

17 Sep '18

Hi all, Yesterday I sent an email to infra-support (with subject "Add gerrit comment about actual CQ results"). I didn't receive any automatic notification or anything. I now logged into jira with my account there (ybardavi) and noticed there is a pending request I made some time ago to change my email there to didi at redhat. I tried finishing the process but whatever I did it failed. I then searched all tickets reported by didi at redhat and see that last one I can find is from Apr, and the next ones (total 4, including the one from yesterday) were not found. So somehow the address didi at redhat is marked as spammer, or problematic in some other way... I see these options now: 1. Just ignore this, do not use jira, do not email infra-support, use the web interface when I must. 2. Have someone (who?) fix this - either assign the address didi at redhat as a valid address for my existing account, or change my email address there, or something like that. 3. Configure my mail client (gmail web interface) to use ybardavi at redhat when sending email to infra-support. I guess/hope this is possible, didn't try yet. I don't like this option, because I prefer to use didi@, and it has the downside of having data there with both addresses unrelated to each other, but it's obviously the simplest. Fixes/opinions/ideas/etc are welcome :-) Thanks, -- Didi

2 3

[oVirt Jenkins] ovirt-node-ng-image_master_build-artifacts-el7-x86_64 - Build # 55 - Failure!
by jenkins＠jenkins.phx.ovirt.org 17 Sep '18

17 Sep '18

Project: http://jenkins.ovirt.org/job/ovirt-node-ng-image_master_build-artifacts-el7… Build: http://jenkins.ovirt.org/job/ovirt-node-ng-image_master_build-artifacts-el7… Build Number: 55 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #55 [Yuval Turgeman] stdci: move build-artifacts to a v1 job ----------------- Failed Tests: ----------------- No tests ran.

1 1

[JIRA] (OVIRT-2463) http://192.168.201.1:8585/el7/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found
by Ehud Yonasi (oVirt JIRA) 17 Sep '18

17 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2463?page=com.atlassian.jira.… ] Ehud Yonasi commented on OVIRT-2463: ------------------------------------ We've discussed on IRC, and you've been able to install engine > http://192.168.201.1:8585/el7/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found > ---------------------------------------------------------------------------------------- > > Key: OVIRT-2463 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2463 > Project: oVirt - virtualization made easy > Issue Type: Bug > Components: Change Queue > Reporter: Anton Marchukov > Assignee: infra > Labels: ost_failures > > Project name: > ovirt-ansible-engine-setup > Branch: > master > Failed Suite: > upgrade-from-release-suite.el7.x86_64 > Failed test/failed phase: > 001_upgrade_engine.test_initialize_engine > Link to first reported failed build: > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/9255/ > Link to first reported failed patch: > ??? -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100092)

1 0

[JIRA] (OVIRT-2498) Failing KubeVirt CI
by Barak Korren (oVirt JIRA) 17 Sep '18

17 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2498?page=com.atlassian.jira.… ] Barak Korren commented on OVIRT-2498: ------------------------------------- [~pkotas] please be more specific about what you mean by monitoring access. you should already be able to monitor the job as it runs in multiple ways via Blue Ocean, the Jenkins old UI or the STDCI UI. WRT Docker connection issues - where is the Docker command being lunched? If its launched from inside the containers that Kubevirt-CI creates, then its really not something I can control (AFAIK its sets up its own networking inside a special "dnsmasq" container). If the thing that fails tries to connect directly from the STDCI environemnt to some service, it may have to do with the fact that we have a proxy configured in the environment. you ned to make sure the connections you're trying to setup and not being routed via the proxy by either setting the 'no_proxy' env var or unsetting the 'http_proxy' env var. I added code to do this in Kubevirt's test.sh a while ago, but maybe someone removed it. > Failing KubeVirt CI > ------------------- > > Key: OVIRT-2498 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2498 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Petr Kotas > Assignee: infra > > Hi, > I am working on fixing the issues on the KubeVirt e2e test suites. This > task is directly related to unstable CI, due to unknown errors. > The progress is reported in the CNV trello: > https://trello.com/c/HNXcMEQu/161-epic-improve-ci > I am creating this issue since the KubeVirt experience random timeouts on > random tests most of the times when test suites run. > The issue from outside is showing as timeouts on difference part of tests. > Sometimes the tests fails in set up phase, again due to random timeout. > The example in the link bellow timed out for network connection on > localhost. > [check-patch.k8s-1.11.0-dev.el7.x86_64] > requests.exceptions.ReadTimeout: > UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. > (read timeout=60) > Example of failing test suites is here > https://jenkins.ovirt.org/job/kubevirt_kubevirt_standard-check-pr/1916/cons… > The list of errors related to the failing CI can be found in my notes > https://docs.google.com/document/d/1_ll1DOMHgCRHn_Df9i4uvtRFyMK-bDCHEeGfJFT… > I am not sure whether KubeVirt already shared the resource requirements, so > I provide short summary: > *Resources for KubeVirt e2e tests:* > - at least 12GB of RAM - we start 3 nodes (3 docker images) each require > 4GB of RAM > - exposed /dev/kvm to enable native virtualization > - cached images, since these are used to build the test cluster: > - kubevirtci/os-3.10.0-crio:latest > - kubevirtci/os-3.10.0-multus:latest > - kubevirtci/os-3.10.0:latest > - kubevirtci/k8s-1.10.4:latest > - kubevirtci/k8s-multus-1.11.1:latest > - kubevirtci/k8s-1.11.0:latest > How can we overcome this? Can we work together to build a suitable > requirements for running the tests so it passes each time? > Kind regards, > Petr Kotas -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100092)

1 0

[JIRA] (OVIRT-2498) Failing KubeVirt CI
by Petr Kotas (oVirt JIRA) 17 Sep '18

17 Sep '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2498?page=com.atlassian.jira.… ] Petr Kotas commented on OVIRT-2498: ----------------------------------- Hi Barak, I understand and agree that mailing list is a great place to share the knowledge. I will write a summary there once we come up with a solution for this issue. And we need to figure the solution swiftly as TLV has holidays approaching. The issues I have described are recurring for almost a 3 months and are blocking us to progress with our work. We already work on fixing the issue from our site and are working on additional fixes to provide even more stable tests. The other part is be sure, we are not crashing the CI. Would you be able to give me a monitoring access so I can see whether there are any race conditions, or we deplete some resources? Regarding the timeouts. We are not relying on them. The timeout you have seen in the logs, is from your infrastructure. It signals, there is networking issue and the docker cannot connect to localhost, which is weird. So please, can I have monitoring access? And can you please check whether the network does not have any issues? Thank you for your help! I appreciate it. Best, Petr On Sun, Sep 16, 2018 at 8:08 AM Barak Korren (oVirt JIRA) < > Failing KubeVirt CI > ------------------- > > Key: OVIRT-2498 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2498 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Petr Kotas > Assignee: infra > > Hi, > I am working on fixing the issues on the KubeVirt e2e test suites. This > task is directly related to unstable CI, due to unknown errors. > The progress is reported in the CNV trello: > https://trello.com/c/HNXcMEQu/161-epic-improve-ci > I am creating this issue since the KubeVirt experience random timeouts on > random tests most of the times when test suites run. > The issue from outside is showing as timeouts on difference part of tests. > Sometimes the tests fails in set up phase, again due to random timeout. > The example in the link bellow timed out for network connection on > localhost. > [check-patch.k8s-1.11.0-dev.el7.x86_64] > requests.exceptions.ReadTimeout: > UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. > (read timeout=60) > Example of failing test suites is here > https://jenkins.ovirt.org/job/kubevirt_kubevirt_standard-check-pr/1916/cons… > The list of errors related to the failing CI can be found in my notes > https://docs.google.com/document/d/1_ll1DOMHgCRHn_Df9i4uvtRFyMK-bDCHEeGfJFT… > I am not sure whether KubeVirt already shared the resource requirements, so > I provide short summary: > *Resources for KubeVirt e2e tests:* > - at least 12GB of RAM - we start 3 nodes (3 docker images) each require > 4GB of RAM > - exposed /dev/kvm to enable native virtualization > - cached images, since these are used to build the test cluster: > - kubevirtci/os-3.10.0-crio:latest > - kubevirtci/os-3.10.0-multus:latest > - kubevirtci/os-3.10.0:latest > - kubevirtci/k8s-1.10.4:latest > - kubevirtci/k8s-multus-1.11.1:latest > - kubevirtci/k8s-1.11.0:latest > How can we overcome this? Can we work together to build a suitable > requirements for running the tests so it passes each time? > Kind regards, > Petr Kotas -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100092)

1 0

[oVirt Jenkins] ovirt-appliance_master_build-artifacts-fc28-x86_64 - Build # 62 - Failure!
by jenkins＠jenkins.phx.ovirt.org 17 Sep '18

17 Sep '18

Project: http://jenkins.ovirt.org/job/ovirt-appliance_master_build-artifacts-fc28-x8… Build: http://jenkins.ovirt.org/job/ovirt-appliance_master_build-artifacts-fc28-x8… Build Number: 62 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #62 [Yuval Turgeman] automation: remove build-artifacts* from stdci-v2 ----------------- Failed Tests: ----------------- No tests ran.

1 1