From dron at redhat.com Thu Mar 1 04:18:52 2018 From: dron at redhat.com (Dafna Ron) Date: Thu, 1 Mar 2018 04:18:52 +0000 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2/Master (ovirt-engine-metrics) ] [ 28-02-2018 ] [ 002_bootstrap.verify_add_hosts] In-Reply-To: References: Message-ID: Hi, Shirly looked at the issue and she thinks that its related to bug: https://bugzilla.redhat.com/show_bug.cgi?id=1540260 On Wed, Feb 28, 2018 at 12:36 PM, Dafna Ron wrote: > Hi, > > We have a failure in test 002_bootstrap.verify_add_hosts. > > the issue seems to be a miss-configured role in ansible > > > *Link and headline of suspected patches: * > > > > > > > *Remove use of end_play and fail module - > https://gerrit.ovirt.org/#/c/88250/ > Link to > Job:http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/950/ > Link to > all > logs:http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/950/artifacts > (Relevant) > error snippet from the log: * > > 2018-02-28 05:01:29,984 p=17139 u=ovirt | TASK [oVirt.ovirt-include-vars-files : Include oVirt metrics config.yml.d vars directory] *** > 2018-02-28 05:01:30,007 p=17139 u=ovirt | ok: [lago-basic-suite-4-2-host-0 -> localhost] => { > "ansible_facts": {}, > "ansible_included_var_files": [], > "changed": false > } > 2018-02-28 05:01:30,016 p=17139 u=ovirt | TASK [oVirt.ovirt-initial-validations/validate-config-yml : If output plugin is elasticsearch, validate host address is set] *** > 2018-02-28 05:01:30,047 p=17139 u=ovirt | changed: [lago-basic-suite-4-2-host-0 -> localhost] => {} > > MSG: > > ERROR oVirt Metrics store is not configured. This host will not be configured to send metrics > > 2018-02-28 05:01:30,056 p=17139 u=ovirt | TASK [oVirt.ovirt-initial-validations/validate-config-yml : If output plugin is fluentd, validate host address is set] *** > 2018-02-28 05:01:30,073 p=17139 u=ovirt | skipping: [lago-basic-suite-4-2-host-0] => { > "changed": false, > "skip_reason": "Conditional result was False" > } > 2018-02-28 05:01:30,082 p=17139 u=ovirt | TASK [oVirt.ovirt-initial-validations/validate-config-yml : Validate viaq_metrics_store parameter is set] *** > 2018-02-28 05:01:30,111 p=17139 u=ovirt | changed: [lago-basic-suite-4-2-host-0 -> localhost] => {} > > MSG: > > ERROR viaq_metrics_store parameter is mandatory. Please set the parameter to 'true' or 'false'. > > 2018-02-28 05:01:30,119 p=17139 u=ovirt | TASK [oVirt.ovirt-initial-validations/validate-config-yml : Validate viaq_metrics_store parameter] *** > 2018-02-28 05:01:30,137 p=17139 u=ovirt | skipping: [lago-basic-suite-4-2-host-0] => { > "changed": false, > "skip_reason": "Conditional result was False" > } > 2018-02-28 05:01:30,146 p=17139 u=ovirt | TASK [oVirt.ovirt-initial-validations/validate-config-yml : Validate openshift_deployment_type parameter is set] *** > 2018-02-28 05:01:30,168 p=17139 u=ovirt | fatal: [lago-basic-suite-4-2-host-0]: FAILED! => {} > > MSG: > > The conditional check 'viaq_metrics_store == true' failed. The error was: error while evaluating conditional (viaq_metrics_store == true): 'viaq_metrics_store' is undefined > > The error appears to have been in '/usr/share/ansible/roles/oVirt.metrics/roles/oVirt.ovirt-initial-validations/validate-config-yml/tasks/main.yml': line 48, column 3, but may > be elsewhere in the file depending on the exact syntax problem. > > The offending line appears to be: > > > - name: "Validate openshift_deployment_type parameter is set" > ^ here > > > ** > > > > > > *Thanks, Dafna* > -------------- next part -------------- An HTML attachment was scrubbed... URL: From didi at redhat.com Thu Mar 1 07:09:17 2018 From: didi at redhat.com (Yedidyah Bar David) Date: Thu, 1 Mar 2018 09:09:17 +0200 Subject: [ovirt-devel] [rhev-devel] Fwd: [ACTION REQUIRED] upgrade to Fedora 27 for otopi on master In-Reply-To: References: <1719121.5Det8mNKmA@awels> Message-ID: (Re-adding devel) On Thu, Mar 1, 2018 at 4:52 AM, Greg Sheremeta wrote: > Setup worked for me immediately after upgrade. I'll forward you my repos in > a separate email. > > This was an interesting sequence: > > greg at dauntless:/etc/yum.repos.d % sudo dnf update otopi -y > Failed to synchronize cache for repo 'region51-chrome-gnome-shell', > disabling. > Failed to synchronize cache for repo 'rcm-tools-fedora-rpms', disabling. > Last metadata expiration check: 0:29:21 ago on Wed 28 Feb 2018 09:16:13 PM > EST. > Package otopi available, but not installed. > No match for argument: otopi > Error: No packages marked for upgrade. > greg at dauntless:/etc/yum.repos.d % sudo dnf install otopi -y > Failed to synchronize cache for repo 'region51-chrome-gnome-shell', > disabling. > Failed to synchronize cache for repo 'rcm-tools-fedora-rpms', disabling. > Last metadata expiration check: 0:29:36 ago on Wed 28 Feb 2018 09:16:13 PM > EST. > Package python2-otopi-1.8.0-0.0.master.20180228112959.git6eccc31.fc27.noarch > is already installed, skipping. > Dependencies resolved. > Nothing to do. > Complete! Adding Yuval for review. Yuval recently re-introduced python3 support to otopi, and following the fedora python packaging guidelines. Yuval, reading the guidelines again, it seems like it's valid to keep the package name 'otopi' and make it Provide: python2-otopi, instead of the opposite. WDYT? > > > > On Wed, Feb 28, 2018 at 5:20 PM, Alexander Wels wrote: >> >> On Wednesday, February 28, 2018 12:02:11 PM EST Greg Sheremeta wrote: >> >> > ---------- Forwarded message ---------- >> >> > From: Greg Sheremeta >> >> > Date: Wed, Feb 28, 2018 at 12:01 PM >> >> > Subject: [ACTION REQUIRED] upgrade to Fedora 27 for otopi on master >> >> > To: Yedidyah Bar David >> >> > Cc: Eli Mesika , devel >> >> > >> >> > >> >> > [changing subject] >> >> > >> >> > Looks like we all have to upgrade Fedora devel machines to Fedora 27. >> >> > >> >> > Greg >> >> > >> >> >> >> I upgraded to F27 and now I am getting something about: >> >> >> >> [awels at awels ovirt-engine]$ ./bin/engine-setup >> ***L:ERROR Internal error: No module named ovirt_setup_lib >> >> I tried removing ovirt-engine, and re-installing it, and dnf is telling >> me: >> >> >> >> Error: >> Problem: conflicting requests >> - nothing provides rubygem-fluent-plugin-rewrite-tag-filter needed by >> ovirt-engine-4.3.0-0.0.master.20180225182524.git8b8f3a52a6.fc27.noarch >> - nothing provides rubygem-fluent-plugin-rewrite-tag-filter needed by >> ovirt-engine-4.3.0-0.0.master.20180226201906.git5831910bef.fc27.noarch >> - nothing provides rubygem-fluent-plugin-rewrite-tag-filter needed by >> ovirt-engine-4.3.0-0.0.master.20180227170348.git8b617da448.fc27.noarch >> >> What am I missing repo wise? >> >> >> >> > On Wed, Feb 28, 2018 at 11:05 AM, Yedidyah Bar David >> >> > >> >> > wrote: >> >> > > (Adding devel, was asked twice already) >> >> > > >> >> > > On Wed, Feb 28, 2018 at 4:52 PM, Eli Mesika >> > > wrote: >> >> > > > Hi Didi >> >> > > > >> >> > > > I am getting : >> >> > > > >> >> > > > >> >> > > > ***L:ERROR Internal error: type object 'Stages' has no attribute >> >> > > > 'ANSWER_FILE_GENERATED' >> >> > > > >> >> > > > Traceback (most recent call last): >> >> > > > File "/usr/lib/python2.7/site-packages/otopi/__main__.py", line 88, >> > > > in >> >> > > > >> >> > > > main >> >> > > > >> >> > > > installer.execute() >> >> > > > >> >> > > > File "/usr/lib/python2.7/site-packages/otopi/main.py", line 153, in >> >> > > > >> >> > > > execute >> >> > > > >> >> > > > sys.exc_info()[2], >> >> > > > >> >> > > > File "/usr/lib/python2.7/site-packages/otopi/util.py", line 81, in >> >> > > > >> >> > > > raiseExceptionInformation >> >> > > > >> >> > > > exec('raise info[1], None, info[2]') >> >> > > > >> >> > > > File "/usr/lib/python2.7/site-packages/otopi/main.py", line 147, in >> >> > > > >> >> > > > execute >> >> > > > >> >> > > > self.context.loadPlugins() >> >> > > > >> >> > > > File "/usr/lib/python2.7/site-packages/otopi/context.py", line 859, >> > > > in >> >> > > > >> >> > > > loadPlugins >> >> > > > >> >> > > > self._loadPluginGroups(plugindir, needgroups, loadedgroups) >> >> > > > >> >> > > > File "/usr/lib/python2.7/site-packages/otopi/context.py", line 113, >> > > > in >> >> > > > >> >> > > > _loadPluginGroups >> >> > > > >> >> > > > self._loadPlugins(path, path, groupname) >> >> > > > >> >> > > > File "/usr/lib/python2.7/site-packages/otopi/context.py", line 70, >> > > > in >> >> > > > >> >> > > > _loadPlugins >> >> > > > >> >> > > > self._loadPlugins(base, d, groupname) >> >> > > > >> >> > > > File "/usr/lib/python2.7/site-packages/otopi/context.py", line 70, >> > > > in >> >> > > > >> >> > > > _loadPlugins >> >> > > > >> >> > > > self._loadPlugins(base, d, groupname) >> >> > > > >> >> > > > File "/usr/lib/python2.7/site-packages/otopi/context.py", line 101, >> > > > in >> >> > > > >> >> > > > _loadPlugins >> >> > > > >> >> > > > os.path.basename(path), >> >> > > > >> >> > > > File "/usr/lib/python2.7/site-packages/otopi/util.py", line 105, in >> >> > > > >> >> > > > loadModule >> >> > > > >> >> > > > mod_desc >> >> > > > >> >> > > > File >> >> > > > >> >> > > > "/home/emesika/ovirt-engine-1481197/share/ovirt-engine/setup >> >> > > >> >> > > /bin/../plugins/ovirt-engine-common/base/core/__init__.py", >> >> > > >> >> > > > line 24, in >> >> > > > >> >> > > > from . import answerfile >> >> > > > >> >> > > > File >> >> > > > >> >> > > > "/home/emesika/ovirt-engine-1481197/share/ovirt-engine/setup >> >> > > >> >> > > /bin/../plugins/ovirt-engine-common/base/core/answerfile.py", >> >> > > >> >> > > > line 38, in >> >> > > > >> >> > > > class Plugin(plugin.PluginBase): >> >> > > > File >> >> > > > >> >> > > > "/home/emesika/ovirt-engine-1481197/share/ovirt-engine/setup >> >> > > >> >> > > /bin/../plugins/ovirt-engine-common/base/core/answerfile.py", >> >> > > >> >> > > > line 60, in Plugin >> >> > > > >> >> > > > otopicons.Stages.ANSWER_FILE_GENERATED, >> >> > > > >> >> > > > PluginLoadException: type object 'Stages' has no attribute >> >> > > > 'ANSWER_FILE_GENERATED' >> >> > > >> >> > > Please update otopi. >> >> > > >> >> > > If you are on fedora, you'll need to upgrade to fc27 first. >> >> > > >> >> > > Good luck, >> >> > > -- >> >> > > Didi >> >> > > _______________________________________________ >> >> > > Devel mailing list >> >> > > Devel at ovirt.org >> >> > > http://lists.ovirt.org/mailman/listinfo/devel >> >> >> >> > > -- Didi From danken at redhat.com Thu Mar 1 10:55:39 2018 From: danken at redhat.com (Dan Kenigsberg) Date: Thu, 1 Mar 2018 12:55:39 +0200 Subject: [ovirt-devel] Inventory Analyzer In-Reply-To: <4e3162a0-dc3c-3eb5-eaad-fba1854b9dfc@redhat.com> References: <4e3162a0-dc3c-3eb5-eaad-fba1854b9dfc@redhat.com> Message-ID: On Mon, Feb 12, 2018 at 5:50 PM, Jason Bryant wrote: > Hello, > > I have attempted a few times to upload a log-collector which has a confirmed > DB in it but I continually get the following error: > > > An internal error occurred: > > ('analyzer failed, rc=%s, err=%r', 255, 'Unable to detect the database dump > from sosreport, aborting..\n') > > See logs for full stacktrace. Have you forgotten to attach them? Which version is the analyzer? Which version was used to collect the logs? We had a pug due to postgres95 using different filename for pgdump, it might be what you're seeing. From eraviv at redhat.com Thu Mar 1 12:25:16 2018 From: eraviv at redhat.com (Eitan Raviv) Date: Thu, 1 Mar 2018 14:25:16 +0200 Subject: [ovirt-devel] ERROR Internal error: type object 'Stages' has no attribute 'ANSWER_FILE_GENERATED' Message-ID: Hi, I am getting the above error on the latest master tip when running engine set-up. Anybody can advise? Thanks -- Eitan Raviv IRC: erav (#ovirt #vdsm #devel #rhev-dev) -------------- next part -------------- An HTML attachment was scrubbed... URL: From didi at redhat.com Thu Mar 1 12:53:25 2018 From: didi at redhat.com (Yedidyah Bar David) Date: Thu, 1 Mar 2018 14:53:25 +0200 Subject: [ovirt-devel] ERROR Internal error: type object 'Stages' has no attribute 'ANSWER_FILE_GENERATED' In-Reply-To: References: Message-ID: On Thu, Mar 1, 2018 at 2:25 PM, Eitan Raviv wrote: > Hi, > > I am getting the above error on the latest master tip when running engine > set-up. > > Anybody can advise? Already discussed here, search for '[ACTION REQUIRED] upgrade to Fedora 27 for otopi on master'. Best regards, > > Thanks > > -- > Eitan Raviv > IRC: erav (#ovirt #vdsm #devel #rhev-dev) -- Didi From eraviv at redhat.com Thu Mar 1 13:23:00 2018 From: eraviv at redhat.com (Eitan Raviv) Date: Thu, 1 Mar 2018 15:23:00 +0200 Subject: [ovirt-devel] ERROR Internal error: type object 'Stages' has no attribute 'ANSWER_FILE_GENERATED' In-Reply-To: References: Message-ID: sorry, did not mention i am on rhel 7.3 On Thu, Mar 1, 2018 at 2:53 PM, Yedidyah Bar David wrote: > On Thu, Mar 1, 2018 at 2:25 PM, Eitan Raviv wrote: > > Hi, > > > > I am getting the above error on the latest master tip when running engine > > set-up. > > > > Anybody can advise? > > Already discussed here, search for '[ACTION REQUIRED] upgrade to > Fedora 27 for otopi on master'. > > Best regards, > > > > > Thanks > > > > -- > > Eitan Raviv > > IRC: erav (#ovirt #vdsm #devel #rhev-dev) > > > > -- > Didi > -- Eitan Raviv IRC: erav (#ovirt #vdsm #devel #rhev-dev) -------------- next part -------------- An HTML attachment was scrubbed... URL: From didi at redhat.com Thu Mar 1 13:29:22 2018 From: didi at redhat.com (Yedidyah Bar David) Date: Thu, 1 Mar 2018 15:29:22 +0200 Subject: [ovirt-devel] ERROR Internal error: type object 'Stages' has no attribute 'ANSWER_FILE_GENERATED' In-Reply-To: References: Message-ID: On Thu, Mar 1, 2018 at 3:23 PM, Eitan Raviv wrote: > sorry, did not mention i am on rhel 7.3 Fine - so as mentioned there, 'yum update otopi' please... Good luck, > > On Thu, Mar 1, 2018 at 2:53 PM, Yedidyah Bar David wrote: >> >> On Thu, Mar 1, 2018 at 2:25 PM, Eitan Raviv wrote: >> > Hi, >> > >> > I am getting the above error on the latest master tip when running >> > engine >> > set-up. >> > >> > Anybody can advise? >> >> Already discussed here, search for '[ACTION REQUIRED] upgrade to >> Fedora 27 for otopi on master'. >> >> Best regards, >> >> > >> > Thanks >> > >> > -- >> > Eitan Raviv >> > IRC: erav (#ovirt #vdsm #devel #rhev-dev) >> >> >> >> -- >> Didi > > > > > -- > Eitan Raviv > IRC: erav (#ovirt #vdsm #devel #rhev-dev) -- Didi From jbryant at redhat.com Thu Mar 1 13:39:33 2018 From: jbryant at redhat.com (Jason Bryant) Date: Thu, 1 Mar 2018 08:39:33 -0500 Subject: [ovirt-devel] Inventory Analyzer In-Reply-To: References: <4e3162a0-dc3c-3eb5-eaad-fba1854b9dfc@redhat.com> Message-ID: Hi Dan, I was using the Web UI and I uploaded the entire logcollector un-edited so nothing was forgotten. As for the version of the analyzer whatever is public facing. As for postgres95 I am not sure that is correct as other logcollectors uploaded before and after worked no problem from the same browser/session. Thank you, Jason On 03/01/2018 05:55 AM, Dan Kenigsberg wrote: > On Mon, Feb 12, 2018 at 5:50 PM, Jason Bryant wrote: >> Hello, >> >> I have attempted a few times to upload a log-collector which has a confirmed >> DB in it but I continually get the following error: >> >> >> An internal error occurred: >> >> ('analyzer failed, rc=%s, err=%r', 255, 'Unable to detect the database dump >> from sosreport, aborting..\n') >> >> See logs for full stacktrace. > Have you forgotten to attach them? > Which version is the analyzer? Which version was used to collect the > logs? We had a pug due to postgres95 using different filename for > pgdump, it might be what you're seeing. From awels at redhat.com Thu Mar 1 13:43:54 2018 From: awels at redhat.com (Alexander Wels) Date: Thu, 01 Mar 2018 08:43:54 -0500 Subject: [ovirt-devel] ERROR Internal error: type object 'Stages' has no attribute 'ANSWER_FILE_GENERATED' In-Reply-To: References: Message-ID: <29244320.FUGIzBzJOG@awels> On Thursday, March 1, 2018 7:25:16 AM EST Eitan Raviv wrote: > Hi, > > I am getting the above error on the latest master tip when running engine > set-up. > > Anybody can advise? > > Thanks You need to update otopi, if you are running Fedora and its not 27 you need to update. From sbonazzo at redhat.com Thu Mar 1 13:59:52 2018 From: sbonazzo at redhat.com (Sandro Bonazzola) Date: Thu, 1 Mar 2018 14:59:52 +0100 Subject: [ovirt-devel] ERROR Internal error: type object 'Stages' has no attribute 'ANSWER_FILE_GENERATED' In-Reply-To: References: Message-ID: 2018-03-01 14:23 GMT+01:00 Eitan Raviv : > sorry, did not mention i am on rhel 7.3 > I would suggest to update to rhel 7.4 then as well. > > On Thu, Mar 1, 2018 at 2:53 PM, Yedidyah Bar David > wrote: > >> On Thu, Mar 1, 2018 at 2:25 PM, Eitan Raviv wrote: >> > Hi, >> > >> > I am getting the above error on the latest master tip when running >> engine >> > set-up. >> > >> > Anybody can advise? >> >> Already discussed here, search for '[ACTION REQUIRED] upgrade to >> Fedora 27 for otopi on master'. >> >> Best regards, >> >> > >> > Thanks >> > >> > -- >> > Eitan Raviv >> > IRC: erav (#ovirt #vdsm #devel #rhev-dev) >> >> >> >> -- >> Didi >> > > > > -- > Eitan Raviv > IRC: erav (#ovirt #vdsm #devel #rhev-dev) > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA TRIED. TESTED. TRUSTED. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mihajlov at linux.vnet.ibm.com Thu Mar 1 14:49:04 2018 From: mihajlov at linux.vnet.ibm.com (Viktor Mihajlovski) Date: Thu, 1 Mar 2018 15:49:04 +0100 Subject: [ovirt-devel] FC27 updates broken for ovirt-4.2 In-Reply-To: References: <4d150e0c-b1ac-a338-94d9-c7b20b490420@linux.vnet.ibm.com> <20180227122533.da6b13b0df05a37df52b3f9a@danny.cz> <7992c62b-135c-0917-59b5-0d2df9d4bf4c@linux.vnet.ibm.com> <20180227140351.81ab62c68e8b6206d129eb09@danny.cz> <21dae6fc-1468-b390-a547-a4510d0c5ab2@redhat.com> <20180228093932.3dcf8a4dae60053ba0acf793@danny.cz> Message-ID: On 28.02.2018 17:27, Cole Robinson wrote: > On 02/28/2018 03:39 AM, Dan Hor?k wrote: >> On Tue, 27 Feb 2018 16:26:28 -0500 >> Cole Robinson wrote: >> >>> On 02/27/2018 08:03 AM, Dan Hor?k wrote: >>>> On Tue, 27 Feb 2018 13:54:06 +0100 >>>> Viktor Mihajlovski wrote: >>>> >>>>> On 27.02.2018 13:35, Nir Soffer wrote: >>>>>> ?????? ??? ??, 27 ????? 2018, 13:25, ??? Dan Hor?k >>>>>> ?: >>>>>> >>>>>>> On Tue, 27 Feb 2018 10:13:15 +0100 >>>>>>> Viktor Mihajlovski wrote: >>>>>>> >>>>>>>> On 27.02.2018 01:26, Nir Soffer wrote: >>>>>>>>> ?????? ??? ??, 26 ????? 2018, 22:10, ??? Yaniv Kaul >>>>>>>>> ?: >>>>>>>>> >>>>>>>>>> On Mon, Feb 26, 2018 at 7:17 PM, Viktor Mihajlovski < >>>>>>>>>> mihajlov at linux.vnet.ibm.com> wrote: >>>>>>>>>> >>>>>>>>>>> I just tried to update the ovirt packages on my FC27 host, >>>>>>>>>>> but failed due to https://gerrit.ovirt.org/#/c/87628/ >>>>>>>>>>> >>>>>>>>>>> vdsm now requires libvirt >= 3.10.0-132 but Fedora 27 has >>>>>>>>>>> only 3.7.0-4 the moment. >>>>>>>>>>> >>>>>>>>>>> It's generic Fedora 27, but since I run on s390, >>>>>>>>>>> cross-posting to s390 list. >>>>>>>>>>> >>>>>>>>>>> I guess there's good reason to require libvirt 3.10. Is there >>>>>>>>>>> any chance that we can get libvirt updated for Fedora 27? >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Perhaps use the virt-preview[1] repo for now? >>>>>>>>>> >>>>>>>>> >>>>>>>>> Yes, we require virt-preview for Fedora. This is why that patch >>>>>>>>> did not fail in the CI. >>>>>>>> Makes sense, unfortunately virt-preview doesn't contains s390 >>>>>>>> binaries at this point in time. Would be great if at least >>>>>>>> libvirt and qemu could be built for s390. >>>>>>> >>>>>>> looks like it's even x86_64 only, /me wonders what it would >>>>>>> require to offer other arches (aarch64, ppc64le, s390x) as well >>>>>>> >>>>>> >>>>>> If we need to support platform not supported in virt-preview, we >>>>>> need to chage the requirement so it is used only on x86_64. >>>>>> >>>>>> Victor, would you like to send a patch? >>>>> I believe there was a good reason to bump the libvirt requirement >>>>> in the vdsm package (some bugfix). Ideally, virt-preview should be >>>>> build for s390 as well. >>>>> If I'm not mistaking, the script >>>>> https://github.com/crobinso/build-fedora-virt-preview is used to >>>>> build the RPMs and populate the repository. >>>>> >>>>> Dan, Cole: what would it take to run this on the fedora-390 build >>>>> machine? >>>> >>>> after a brief look the script needs to be made multi-arch-aware (it >>>> hard-codes x86_64 in some places), when it calls mock, and then it >>>> needs some HW (we have ppc64le and s390x even now, aarch64 might >>>> take a while), overall it looks doable to me. Cole, what do you >>>> think? >>>> >>> >>> I'm open to the idea in theory but in practice right now the script >>> uses mock locally so it's basically tied to the one build machine I >>> use which is x86. I have arm32 and aarch64 hardware locally but TBH I >>> have very little interest in running a build farm and dealing with >>> all the issues of connecting to remote machines, pulling down build >>> output, etc. In fact I've been meaning to move virt-preview to copr >>> for a long while which is going to tie it even deeper to x86, this >>> would make virt-preview easier to enable and let me scrap much of my >>> custom code for building/uploading repo contents. >> >> copr would give us ppc64le and probably aarch64 too in addition to x86, >> but can't help with s390. We already have build infra internally >> covering all arches driven by Jenkins, with plans to move the workloads >> to CentOS infra. I'll look whether or how it could be used for >> multi-arch virt-preview repos. >> >>> We could re-implement it using koji scratch builds which have multiple >>> arch support nowadays but I did that in the past for x86 and I recall >>> feeling it was quite brittle, though I don't remember the details, >>> maybe it was just my implementation. >> >> I suspect the problem with koji scratch builds in this case is that >> they can't be used in buildroots, which the virt-preview stack requires. >> > > Good point, but generally virt-preview packages are very loosely coupled > and don't have strong version dependencies on one another. Occasionally > a small dependency needs to be updated like libiscsi or libcacard but in > fact it's been a long time since that's been required. > > - Cole > Hi Cole, for the time being I've patched vdsm to require libvirt 3.7.0-4 (which contains the hotplug patch required by vdsm). Unfortunately, this release is still only in updates-testing, which lead to a build failure on ovirt Jenkins. Do you have an idea by when 3.7.0-4 will be propagated to fedora updates? Thanks! -- Regards, Viktor Mihajlovski From crobinso at redhat.com Thu Mar 1 19:12:53 2018 From: crobinso at redhat.com (Cole Robinson) Date: Thu, 1 Mar 2018 14:12:53 -0500 Subject: [ovirt-devel] FC27 updates broken for ovirt-4.2 In-Reply-To: References: <4d150e0c-b1ac-a338-94d9-c7b20b490420@linux.vnet.ibm.com> <20180227122533.da6b13b0df05a37df52b3f9a@danny.cz> <7992c62b-135c-0917-59b5-0d2df9d4bf4c@linux.vnet.ibm.com> <20180227140351.81ab62c68e8b6206d129eb09@danny.cz> <21dae6fc-1468-b390-a547-a4510d0c5ab2@redhat.com> <20180228093932.3dcf8a4dae60053ba0acf793@danny.cz> Message-ID: <71ddc165-ba78-f601-5000-355de68b1d41@redhat.com> On 03/01/2018 09:49 AM, Viktor Mihajlovski wrote: > On 28.02.2018 17:27, Cole Robinson wrote: >> On 02/28/2018 03:39 AM, Dan Hor?k wrote: >>> On Tue, 27 Feb 2018 16:26:28 -0500 >>> Cole Robinson wrote: >>> >>>> On 02/27/2018 08:03 AM, Dan Hor?k wrote: >>>>> On Tue, 27 Feb 2018 13:54:06 +0100 >>>>> Viktor Mihajlovski wrote: >>>>> >>>>>> On 27.02.2018 13:35, Nir Soffer wrote: >>>>>>> ?????? ??? ??, 27 ????? 2018, 13:25, ??? Dan Hor?k >>>>>>> ?: >>>>>>> >>>>>>>> On Tue, 27 Feb 2018 10:13:15 +0100 >>>>>>>> Viktor Mihajlovski wrote: >>>>>>>> >>>>>>>>> On 27.02.2018 01:26, Nir Soffer wrote: >>>>>>>>>> ?????? ??? ??, 26 ????? 2018, 22:10, ??? Yaniv Kaul >>>>>>>>>> ?: >>>>>>>>>> >>>>>>>>>>> On Mon, Feb 26, 2018 at 7:17 PM, Viktor Mihajlovski < >>>>>>>>>>> mihajlov at linux.vnet.ibm.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> I just tried to update the ovirt packages on my FC27 host, >>>>>>>>>>>> but failed due to https://gerrit.ovirt.org/#/c/87628/ >>>>>>>>>>>> >>>>>>>>>>>> vdsm now requires libvirt >= 3.10.0-132 but Fedora 27 has >>>>>>>>>>>> only 3.7.0-4 the moment. >>>>>>>>>>>> >>>>>>>>>>>> It's generic Fedora 27, but since I run on s390, >>>>>>>>>>>> cross-posting to s390 list. >>>>>>>>>>>> >>>>>>>>>>>> I guess there's good reason to require libvirt 3.10. Is there >>>>>>>>>>>> any chance that we can get libvirt updated for Fedora 27? >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Perhaps use the virt-preview[1] repo for now? >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Yes, we require virt-preview for Fedora. This is why that patch >>>>>>>>>> did not fail in the CI. >>>>>>>>> Makes sense, unfortunately virt-preview doesn't contains s390 >>>>>>>>> binaries at this point in time. Would be great if at least >>>>>>>>> libvirt and qemu could be built for s390. >>>>>>>> >>>>>>>> looks like it's even x86_64 only, /me wonders what it would >>>>>>>> require to offer other arches (aarch64, ppc64le, s390x) as well >>>>>>>> >>>>>>> >>>>>>> If we need to support platform not supported in virt-preview, we >>>>>>> need to chage the requirement so it is used only on x86_64. >>>>>>> >>>>>>> Victor, would you like to send a patch? >>>>>> I believe there was a good reason to bump the libvirt requirement >>>>>> in the vdsm package (some bugfix). Ideally, virt-preview should be >>>>>> build for s390 as well. >>>>>> If I'm not mistaking, the script >>>>>> https://github.com/crobinso/build-fedora-virt-preview is used to >>>>>> build the RPMs and populate the repository. >>>>>> >>>>>> Dan, Cole: what would it take to run this on the fedora-390 build >>>>>> machine? >>>>> >>>>> after a brief look the script needs to be made multi-arch-aware (it >>>>> hard-codes x86_64 in some places), when it calls mock, and then it >>>>> needs some HW (we have ppc64le and s390x even now, aarch64 might >>>>> take a while), overall it looks doable to me. Cole, what do you >>>>> think? >>>>> >>>> >>>> I'm open to the idea in theory but in practice right now the script >>>> uses mock locally so it's basically tied to the one build machine I >>>> use which is x86. I have arm32 and aarch64 hardware locally but TBH I >>>> have very little interest in running a build farm and dealing with >>>> all the issues of connecting to remote machines, pulling down build >>>> output, etc. In fact I've been meaning to move virt-preview to copr >>>> for a long while which is going to tie it even deeper to x86, this >>>> would make virt-preview easier to enable and let me scrap much of my >>>> custom code for building/uploading repo contents. >>> >>> copr would give us ppc64le and probably aarch64 too in addition to x86, >>> but can't help with s390. We already have build infra internally >>> covering all arches driven by Jenkins, with plans to move the workloads >>> to CentOS infra. I'll look whether or how it could be used for >>> multi-arch virt-preview repos. >>> >>>> We could re-implement it using koji scratch builds which have multiple >>>> arch support nowadays but I did that in the past for x86 and I recall >>>> feeling it was quite brittle, though I don't remember the details, >>>> maybe it was just my implementation. >>> >>> I suspect the problem with koji scratch builds in this case is that >>> they can't be used in buildroots, which the virt-preview stack requires. >>> >> >> Good point, but generally virt-preview packages are very loosely coupled >> and don't have strong version dependencies on one another. Occasionally >> a small dependency needs to be updated like libiscsi or libcacard but in >> fact it's been a long time since that's been required. >> >> - Cole >> > Hi Cole, for the time being I've patched vdsm to require libvirt 3.7.0-4 > (which contains the hotplug patch required by vdsm). Unfortunately, this > release is still only in updates-testing, which lead to a build failure > on ovirt Jenkins. > Do you have an idea by when 3.7.0-4 will be propagated to fedora updates? > Thanks! > I submitted it for 'updates' yesterday, so whenever the next compose is. Is this about vdsm packages in fedora, or an external repo that builds packages for upstream? - Cole From emesika at redhat.com Thu Mar 1 23:17:26 2018 From: emesika at redhat.com (Eli Mesika) Date: Fri, 2 Mar 2018 01:17:26 +0200 Subject: [ovirt-devel] [ACTION REQUIRED] upgrade to Fedora 27 for otopi on master In-Reply-To: References: Message-ID: I have upgraded to fc27 I am not able to add a host to a PPC cluster : "no otopi module ..." otopi is installed rpm -qa |grep otopi otopi-java-1.8.0-0.0.master.20180228112959.git6eccc31.fc27.noarch otopi-common-1.8.0-0.0.master.20180228112959.git6eccc31.fc27.noarch python2-otopi-devtools-1.8.0-0.0.master.20180228112959.git6eccc31.fc27.noarch python2-otopi-1.8.0-0.0.master.20180228112959.git6eccc31.fc27.noarch Any ideas ? On Wed, Feb 28, 2018 at 7:01 PM, Greg Sheremeta wrote: > [changing subject] > > Looks like we all have to upgrade Fedora devel machines to Fedora 27. > > Greg > > On Wed, Feb 28, 2018 at 11:05 AM, Yedidyah Bar David > wrote: > >> (Adding devel, was asked twice already) >> >> On Wed, Feb 28, 2018 at 4:52 PM, Eli Mesika wrote: >> > Hi Didi >> > >> > I am getting : >> > >> > >> > ***L:ERROR Internal error: type object 'Stages' has no attribute >> > 'ANSWER_FILE_GENERATED' >> > Traceback (most recent call last): >> > File "/usr/lib/python2.7/site-packages/otopi/__main__.py", line 88, >> in >> > main >> > installer.execute() >> > File "/usr/lib/python2.7/site-packages/otopi/main.py", line 153, in >> > execute >> > sys.exc_info()[2], >> > File "/usr/lib/python2.7/site-packages/otopi/util.py", line 81, in >> > raiseExceptionInformation >> > exec('raise info[1], None, info[2]') >> > File "/usr/lib/python2.7/site-packages/otopi/main.py", line 147, in >> > execute >> > self.context.loadPlugins() >> > File "/usr/lib/python2.7/site-packages/otopi/context.py", line 859, >> in >> > loadPlugins >> > self._loadPluginGroups(plugindir, needgroups, loadedgroups) >> > File "/usr/lib/python2.7/site-packages/otopi/context.py", line 113, >> in >> > _loadPluginGroups >> > self._loadPlugins(path, path, groupname) >> > File "/usr/lib/python2.7/site-packages/otopi/context.py", line 70, in >> > _loadPlugins >> > self._loadPlugins(base, d, groupname) >> > File "/usr/lib/python2.7/site-packages/otopi/context.py", line 70, in >> > _loadPlugins >> > self._loadPlugins(base, d, groupname) >> > File "/usr/lib/python2.7/site-packages/otopi/context.py", line 101, >> in >> > _loadPlugins >> > os.path.basename(path), >> > File "/usr/lib/python2.7/site-packages/otopi/util.py", line 105, in >> > loadModule >> > mod_desc >> > File >> > "/home/emesika/ovirt-engine-1481197/share/ovirt-engine/setup >> /bin/../plugins/ovirt-engine-common/base/core/__init__.py", >> > line 24, in >> > from . import answerfile >> > File >> > "/home/emesika/ovirt-engine-1481197/share/ovirt-engine/setup >> /bin/../plugins/ovirt-engine-common/base/core/answerfile.py", >> > line 38, in >> > class Plugin(plugin.PluginBase): >> > File >> > "/home/emesika/ovirt-engine-1481197/share/ovirt-engine/setup >> /bin/../plugins/ovirt-engine-common/base/core/answerfile.py", >> > line 60, in Plugin >> > otopicons.Stages.ANSWER_FILE_GENERATED, >> > PluginLoadException: type object 'Stages' has no attribute >> > 'ANSWER_FILE_GENERATED' >> > >> >> Please update otopi. >> >> If you are on fedora, you'll need to upgrade to fc27 first. >> >> Good luck, >> -- >> Didi >> _______________________________________________ >> Devel mailing list >> Devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mihajlov at linux.vnet.ibm.com Fri Mar 2 09:12:08 2018 From: mihajlov at linux.vnet.ibm.com (Viktor Mihajlovski) Date: Fri, 2 Mar 2018 10:12:08 +0100 Subject: [ovirt-devel] FC27 updates broken for ovirt-4.2 In-Reply-To: <71ddc165-ba78-f601-5000-355de68b1d41@redhat.com> References: <4d150e0c-b1ac-a338-94d9-c7b20b490420@linux.vnet.ibm.com> <20180227122533.da6b13b0df05a37df52b3f9a@danny.cz> <7992c62b-135c-0917-59b5-0d2df9d4bf4c@linux.vnet.ibm.com> <20180227140351.81ab62c68e8b6206d129eb09@danny.cz> <21dae6fc-1468-b390-a547-a4510d0c5ab2@redhat.com> <20180228093932.3dcf8a4dae60053ba0acf793@danny.cz> <71ddc165-ba78-f601-5000-355de68b1d41@redhat.com> Message-ID: <562c3e84-3e66-2ae8-b546-9e0b0a56b792@linux.vnet.ibm.com> On 01.03.2018 20:12, Cole Robinson wrote: [...] >>> - Cole >>> >> Hi Cole, for the time being I've patched vdsm to require libvirt 3.7.0-4 >> (which contains the hotplug patch required by vdsm). Unfortunately, this >> release is still only in updates-testing, which lead to a build failure >> on ovirt Jenkins. >> Do you have an idea by when 3.7.0-4 will be propagated to fedora updates? >> Thanks! >> > > I submitted it for 'updates' yesterday, so whenever the next compose is. > > Is this about vdsm packages in fedora, or an external repo that builds > packages for upstream? > > - Cole > the package has made it to 'updates' yesterday, thanks! I was referring to vdsm in the upstream ovirt project. With the current F27 'updates' vdsm is building again on s390x. -- Regards, Viktor Mihajlovski From dron at redhat.com Fri Mar 2 20:42:22 2018 From: dron at redhat.com (Dafna Ron) Date: Fri, 2 Mar 2018 20:42:22 +0000 Subject: [ovirt-devel] OST Failure - Weekly update [24/02/2018-02/03/2018] Message-ID: Hello, I would like to update on this week's failures and OST current status. We had a failed change that caused other changes to fail: 1. Remove use of end_play and fail module - https://gerrit.ovirt.org/#/c/88250/ date: 27-02-2018 version: master/4.2 test failed: 002_bootstrap.verify_add_hosts fix: reverted until it can be fixed: https://gerrit.ovirt.org/#/c/88327/ bug: https://bugzilla.redhat.com/show_bug.cgi?id=1540260 I re-triggered some of the failed changes that were effected by the above change. However, I am still seeing ovirt-engine project is failing in cq so I am expecting some new issues to appear which would be reported next week. *Below you can see the chart for this week's resolved issues but cause of failure: * Code= regression of working components/functionalities Configurations - package related issues Other = failed build artifacts Infra = infrastructure/OST/Lago related issues *Below is a chart of resolved failures based on ovirt version* *Below is a chart showing failures by suite type: * Thank you, Dafna -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 32296 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 28589 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 9193 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 23312 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 6221 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 7696 bytes Desc: not available URL: From didi at redhat.com Sun Mar 4 06:46:06 2018 From: didi at redhat.com (Yedidyah Bar David) Date: Sun, 4 Mar 2018 08:46:06 +0200 Subject: [ovirt-devel] otopi on PPC (Was: [ACTION REQUIRED] upgrade to Fedora 27 for otopi on master) Message-ID: (Changed subject) On Fri, Mar 2, 2018 at 1:17 AM, Eli Mesika wrote: > I have upgraded to fc27 > I am not able to add a host to a PPC cluster : "no otopi module ..." Where do you get this error? > > otopi is installed On the engine? Host? Both? Generally it's enough to have it on the engine. > > rpm -qa |grep otopi > otopi-java-1.8.0-0.0.master.20180228112959.git6eccc31.fc27.noarch > otopi-common-1.8.0-0.0.master.20180228112959.git6eccc31.fc27.noarch > python2-otopi-devtools-1.8.0-0.0.master.20180228112959.git6eccc31.fc27.noarch > python2-otopi-1.8.0-0.0.master.20180228112959.git6eccc31.fc27.noarch > > Any ideas ? I have no experience with PPC, sorry. But in theory it should work the same, so let's try to debug "as usual". Thanks and best regards, > > > On Wed, Feb 28, 2018 at 7:01 PM, Greg Sheremeta wrote: >> >> [changing subject] >> >> Looks like we all have to upgrade Fedora devel machines to Fedora 27. >> >> Greg >> >> On Wed, Feb 28, 2018 at 11:05 AM, Yedidyah Bar David >> wrote: >>> >>> (Adding devel, was asked twice already) >>> >>> On Wed, Feb 28, 2018 at 4:52 PM, Eli Mesika wrote: >>> > Hi Didi >>> > >>> > I am getting : >>> > >>> > >>> > ***L:ERROR Internal error: type object 'Stages' has no attribute >>> > 'ANSWER_FILE_GENERATED' >>> > Traceback (most recent call last): >>> > File "/usr/lib/python2.7/site-packages/otopi/__main__.py", line 88, >>> > in >>> > main >>> > installer.execute() >>> > File "/usr/lib/python2.7/site-packages/otopi/main.py", line 153, in >>> > execute >>> > sys.exc_info()[2], >>> > File "/usr/lib/python2.7/site-packages/otopi/util.py", line 81, in >>> > raiseExceptionInformation >>> > exec('raise info[1], None, info[2]') >>> > File "/usr/lib/python2.7/site-packages/otopi/main.py", line 147, in >>> > execute >>> > self.context.loadPlugins() >>> > File "/usr/lib/python2.7/site-packages/otopi/context.py", line 859, >>> > in >>> > loadPlugins >>> > self._loadPluginGroups(plugindir, needgroups, loadedgroups) >>> > File "/usr/lib/python2.7/site-packages/otopi/context.py", line 113, >>> > in >>> > _loadPluginGroups >>> > self._loadPlugins(path, path, groupname) >>> > File "/usr/lib/python2.7/site-packages/otopi/context.py", line 70, in >>> > _loadPlugins >>> > self._loadPlugins(base, d, groupname) >>> > File "/usr/lib/python2.7/site-packages/otopi/context.py", line 70, in >>> > _loadPlugins >>> > self._loadPlugins(base, d, groupname) >>> > File "/usr/lib/python2.7/site-packages/otopi/context.py", line 101, >>> > in >>> > _loadPlugins >>> > os.path.basename(path), >>> > File "/usr/lib/python2.7/site-packages/otopi/util.py", line 105, in >>> > loadModule >>> > mod_desc >>> > File >>> > >>> > "/home/emesika/ovirt-engine-1481197/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-common/base/core/__init__.py", >>> > line 24, in >>> > from . import answerfile >>> > File >>> > >>> > "/home/emesika/ovirt-engine-1481197/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-common/base/core/answerfile.py", >>> > line 38, in >>> > class Plugin(plugin.PluginBase): >>> > File >>> > >>> > "/home/emesika/ovirt-engine-1481197/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-common/base/core/answerfile.py", >>> > line 60, in Plugin >>> > otopicons.Stages.ANSWER_FILE_GENERATED, >>> > PluginLoadException: type object 'Stages' has no attribute >>> > 'ANSWER_FILE_GENERATED' >>> > >>> >>> Please update otopi. >>> >>> If you are on fedora, you'll need to upgrade to fc27 first. >>> >>> Good luck, >>> -- >>> Didi >>> _______________________________________________ >>> Devel mailing list >>> Devel at ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/devel >> >> > -- Didi From dbelenky at redhat.com Sun Mar 4 15:18:04 2018 From: dbelenky at redhat.com (Daniel Belenky) Date: Sun, 4 Mar 2018 17:18:04 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 (ovirt-engine) ] [ 04-03-2018 ] [ 004_basic_sanity.disk_operations ] Message-ID: Hi, The following test failed OST: 004_basic_sanity.disk_operations. Link to suspected patch: https://gerrit.ovirt.org/c/88404/ Link to the failed job: http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1019/ Link to all test logs: - engine - host 0 - host 1 Error snippet from engine: 2018-03-04 09:50:14,823-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ForkJoinPool-1-worker-12) [] EVENT_ID: VM_DOWN_ERROR(119), VM vm2 is down with error. Exit message: Lost connection with qemu process. Error snippet from host: Mar 4 09:56:27 lago-basic-suite-4-2-host-1 libvirtd: 2018-03-04 14:56:27.831+0000: 1189: error : qemuDomainAgentAvailable:6010 : Guest agent is not responding: QEMU guest agent is not connected Thanks, -- DANIEL BELENKY RHV DEVOPS -------------- next part -------------- An HTML attachment was scrubbed... URL: From ykaul at redhat.com Sun Mar 4 15:30:28 2018 From: ykaul at redhat.com (Yaniv Kaul) Date: Sun, 4 Mar 2018 17:30:28 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 (ovirt-engine) ] [ 04-03-2018 ] [ 004_basic_sanity.disk_operations ] In-Reply-To: References: Message-ID: On Sun, Mar 4, 2018 at 5:18 PM, Daniel Belenky wrote: > Hi, > > The following test failed OST: 004_basic_sanity.disk_operations. > > Link to suspected patch: https://gerrit.ovirt.org/c/88404/ > Link to the failed job: http://jenkins.ovirt.org/ > job/ovirt-4.2_change-queue-tester/1019/ > Link to all test logs: > > - engine > > - host 0 > > - host 1 > > > Error snippet from engine: > > 2018-03-04 09:50:14,823-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ForkJoinPool-1-worker-12) [] EVENT_ID: VM_DOWN_ERROR(119), VM vm2 is down with error. Exit message: Lost connection with qemu process. > > > Error snippet from host: > > Mar 4 09:56:27 lago-basic-suite-4-2-host-1 libvirtd: 2018-03-04 14:56:27.831+0000: 1189: error : qemuDomainAgentAvailable:6010 : Guest agent is not responding: QEMU guest agent is not connected > > That's not surprising - there's no guest agent there. However, Benny and I were just discussing this issue as I've seen this failure on my host as well: ovirtlago.testlib: ERROR: Unhandled exception in Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt-4.2_change-queue-tester/ovirt-system-tests/basic-suite-4.2/test-scenarios/004_basic_sanity.py", line 519, in all_jobs_finished jobs = engine.jobs_service().list(search='correlation_id=%s' % correlation_id) TypeError: list() got an unexpected keyword argument 'search' lago.utils: ERROR: Error while running thread Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue queue.put({'return': func()}) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 78, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/home/jenkins/workspace/ovirt-4.2_change-queue-tester/ovirt-system-tests/basic-suite-4.2/test-scenarios/004_basic_sanity.py", line 522, in live_storage_migration testlib.assert_true_within_long(all_jobs_finished) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 271, in assert_true_within_long assert_equals_within_long(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 258, in assert_equals_within_long func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt-4.2_change-queue-tester/ovirt-system-tests/basic-suite-4.2/test-scenarios/004_basic_sanity.py", line 519, in all_jobs_finished jobs = engine.jobs_service().list(search='correlation_id=%s' % correlation_id) TypeError: list() got an unexpected keyword argument 'search' Y. > Thanks, > > -- > > DANIEL BELENKY > > RHV DEVOPS > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nsoffer at redhat.com Sun Mar 4 15:48:10 2018 From: nsoffer at redhat.com (Nir Soffer) Date: Sun, 04 Mar 2018 15:48:10 +0000 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 (ovirt-engine) ] [ 04-03-2018 ] [ 004_basic_sanity.disk_operations ] In-Reply-To: References: Message-ID: On Sun, Mar 4, 2018 at 5:31 PM Yaniv Kaul wrote: > On Sun, Mar 4, 2018 at 5:18 PM, Daniel Belenky > wrote: > >> Hi, >> >> The following test failed OST: 004_basic_sanity.disk_operations. >> >> Link to suspected patch: https://gerrit.ovirt.org/c/88404/ >> Link to the failed job: >> http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1019/ >> Link to all test logs: >> >> - engine >> >> - host 0 >> >> - host 1 >> >> >> Error snippet from engine: >> >> 2018-03-04 09:50:14,823-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ForkJoinPool-1-worker-12) [] EVENT_ID: VM_DOWN_ERROR(119), VM vm2 is down with error. Exit message: Lost connection with qemu process. >> >> >> Error snippet from host: >> >> Mar 4 09:56:27 lago-basic-suite-4-2-host-1 libvirtd: 2018-03-04 14:56:27.831+0000: 1189: error : qemuDomainAgentAvailable:6010 : Guest agent is not responding: QEMU guest agent is not connected >> >> > That's not surprising - there's no guest agent there. > There are 2 issues here: - we are testing without guest agent when this is the recommended configuration (snapshots may not be consistent without guest agent) - vdsm should not report errors about guest agent since it does not know if guest agent is installed or not. This message should be an INFO message like "could not stop the vm using guest agent, falling back to ..." Generally we should not see ERROR or WARN message in OST. Any repeating error or warning should be reported as a bug. Nir -------------- next part -------------- An HTML attachment was scrubbed... URL: From bzlotnik at redhat.com Sun Mar 4 16:01:42 2018 From: bzlotnik at redhat.com (Benny Zlotnik) Date: Sun, 4 Mar 2018 18:01:42 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 (ovirt-engine) ] [ 04-03-2018 ] [ 004_basic_sanity.disk_operations ] In-Reply-To: References: Message-ID: https://gerrit.ovirt.org/#/c/84338/ this patch was merged today and it seems to be causing the problem since it uses an SDK feature which is currently only available in the 4.3 SDK versions It looks like it should be available in the SDK when it's built against the 4.2.29 ovirt-engine-api-model On Sun, Mar 4, 2018 at 5:18 PM, Daniel Belenky wrote: > Hi, > > The following test failed OST: 004_basic_sanity.disk_operations. > > Link to suspected patch: https://gerrit.ovirt.org/c/88404/ > Link to the failed job: http://jenkins.ovirt.org/ > job/ovirt-4.2_change-queue-tester/1019/ > Link to all test logs: > > - engine > > - host 0 > > - host 1 > > > Error snippet from engine: > > 2018-03-04 09:50:14,823-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ForkJoinPool-1-worker-12) [] EVENT_ID: VM_DOWN_ERROR(119), VM vm2 is down with error. Exit message: Lost connection with qemu process. > > > Error snippet from host: > > Mar 4 09:56:27 lago-basic-suite-4-2-host-1 libvirtd: 2018-03-04 14:56:27.831+0000: 1189: error : qemuDomainAgentAvailable:6010 : Guest agent is not responding: QEMU guest agent is not connected > > > Thanks, > > -- > > DANIEL BELENKY > > RHV DEVOPS > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eedri at redhat.com Sun Mar 4 16:19:35 2018 From: eedri at redhat.com (Eyal Edri) Date: Sun, 4 Mar 2018 18:19:35 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 (ovirt-engine) ] [ 04-03-2018 ] [ 004_basic_sanity.disk_operations ] In-Reply-To: References: Message-ID: I've prepared a revert patch [1] in the meantime, but its strange that it passed check-patch, can you verify which SDK version was used and which one you need? [1] https://gerrit.ovirt.org/#/c/88441/ On Sun, Mar 4, 2018 at 6:01 PM, Benny Zlotnik wrote: > https://gerrit.ovirt.org/#/c/84338/ this patch was merged today and it > seems to be causing the problem since it uses an SDK feature which is > currently only available in the 4.3 SDK versions > It looks like it should be available in the SDK when it's built against > the 4.2.29 ovirt-engine-api-model > > On Sun, Mar 4, 2018 at 5:18 PM, Daniel Belenky > wrote: > >> Hi, >> >> The following test failed OST: 004_basic_sanity.disk_operations. >> >> Link to suspected patch: https://gerrit.ovirt.org/c/88404/ >> Link to the failed job: http://jenkins.ovirt.org/ >> job/ovirt-4.2_change-queue-tester/1019/ >> Link to all test logs: >> >> - engine >> >> - host 0 >> >> - host 1 >> >> >> Error snippet from engine: >> >> 2018-03-04 09:50:14,823-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ForkJoinPool-1-worker-12) [] EVENT_ID: VM_DOWN_ERROR(119), VM vm2 is down with error. Exit message: Lost connection with qemu process. >> >> >> Error snippet from host: >> >> Mar 4 09:56:27 lago-basic-suite-4-2-host-1 libvirtd: 2018-03-04 14:56:27.831+0000: 1189: error : qemuDomainAgentAvailable:6010 : Guest agent is not responding: QEMU guest agent is not connected >> >> >> Thanks, >> >> -- >> >> DANIEL BELENKY >> >> RHV DEVOPS >> >> _______________________________________________ >> Devel mailing list >> Devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel >> > > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -- Eyal edri MANAGER RHV DevOps EMEA VIRTUALIZATION R&D Red Hat EMEA TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ykaul at redhat.com Sun Mar 4 16:42:36 2018 From: ykaul at redhat.com (Yaniv Kaul) Date: Sun, 4 Mar 2018 18:42:36 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 (ovirt-engine) ] [ 04-03-2018 ] [ 004_basic_sanity.disk_operations ] In-Reply-To: References: Message-ID: On Sun, Mar 4, 2018 at 5:48 PM, Nir Soffer wrote: > On Sun, Mar 4, 2018 at 5:31 PM Yaniv Kaul wrote: > >> On Sun, Mar 4, 2018 at 5:18 PM, Daniel Belenky >> wrote: >> >>> Hi, >>> >>> The following test failed OST: 004_basic_sanity.disk_operations. >>> >>> Link to suspected patch: https://gerrit.ovirt.org/c/88404/ >>> Link to the failed job: http://jenkins.ovirt.org/ >>> job/ovirt-4.2_change-queue-tester/1019/ >>> Link to all test logs: >>> >>> - engine >>> >>> - host 0 >>> >>> - host 1 >>> >>> >>> Error snippet from engine: >>> >>> 2018-03-04 09:50:14,823-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ForkJoinPool-1-worker-12) [] EVENT_ID: VM_DOWN_ERROR(119), VM vm2 is down with error. Exit message: Lost connection with qemu process. >>> >>> >>> Error snippet from host: >>> >>> Mar 4 09:56:27 lago-basic-suite-4-2-host-1 libvirtd: 2018-03-04 14:56:27.831+0000: 1189: error : qemuDomainAgentAvailable:6010 : Guest agent is not responding: QEMU guest agent is not connected >>> >>> >> That's not surprising - there's no guest agent there. >> > > There are 2 issues here: > - we are testing without guest agent when this is the recommended > configuration > (snapshots may not be consistent without guest agent) > We are still using Cirros. I need to get a CentOS with cloud-init uploaded (WIP...) > - vdsm should not report errors about guest agent since it does not know if > guest agent is installed or not. This message should be an INFO message > like > "could not stop the vm using guest agent, falling back to ..." > > Generally we should not see ERROR or WARN message in OST. Any repeating > error or warning should be reported as a bug. > There are several on storage... Should we file BZs on them? Y. > > Nir > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nsoffer at redhat.com Sun Mar 4 22:51:19 2018 From: nsoffer at redhat.com (Nir Soffer) Date: Sun, 04 Mar 2018 22:51:19 +0000 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 (ovirt-engine) ] [ 04-03-2018 ] [ 004_basic_sanity.disk_operations ] In-Reply-To: References: Message-ID: On Sun, Mar 4, 2018 at 6:43 PM Yaniv Kaul wrote: > On Sun, Mar 4, 2018 at 5:48 PM, Nir Soffer wrote: > >> On Sun, Mar 4, 2018 at 5:31 PM Yaniv Kaul wrote: >> >>> On Sun, Mar 4, 2018 at 5:18 PM, Daniel Belenky >>> wrote: >>> >>>> Hi, >>>> >>>> The following test failed OST: 004_basic_sanity.disk_operations. >>>> >>>> Link to suspected patch: https://gerrit.ovirt.org/c/88404/ >>>> Link to the failed job: >>>> http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1019/ >>>> Link to all test logs: >>>> >>>> - engine >>>> >>>> - host 0 >>>> >>>> - host 1 >>>> >>>> >>>> Error snippet from engine: >>>> >>>> 2018-03-04 09:50:14,823-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ForkJoinPool-1-worker-12) [] EVENT_ID: VM_DOWN_ERROR(119), VM vm2 is down with error. Exit message: Lost connection with qemu process. >>>> >>>> >>>> Error snippet from host: >>>> >>>> Mar 4 09:56:27 lago-basic-suite-4-2-host-1 libvirtd: 2018-03-04 14:56:27.831+0000: 1189: error : qemuDomainAgentAvailable:6010 : Guest agent is not responding: QEMU guest agent is not connected >>>> >>>> >>> That's not surprising - there's no guest agent there. >>> >> >> There are 2 issues here: >> - we are testing without guest agent when this is the recommended >> configuration >> (snapshots may not be consistent without guest agent) >> > > We are still using Cirros. I need to get a CentOS with cloud-init uploaded > (WIP...) > > >> - vdsm should not report errors about guest agent since it does not know >> if >> guest agent is installed or not. This message should be an INFO message >> like >> "could not stop the vm using guest agent, falling back to ..." >> >> Generally we should not see ERROR or WARN message in OST. Any repeating >> error or warning should be reported as a bug. >> > > There are several on storage... Should we file BZs on them? > I think we have only warnings now, but they should be fixed, so please file a bug. > Y. > > >> >> Nir >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From didi at redhat.com Mon Mar 5 13:01:04 2018 From: didi at redhat.com (Yedidyah Bar David) Date: Mon, 5 Mar 2018 15:01:04 +0200 Subject: [ovirt-devel] hosted-engine ovirt-system-tests Message-ID: Hi all, Recently I have been pushing various patches to OST in order to verify specific bugs/fixes I was working on, using them with the "manual" jenkins job but with no immediate intention to get them merged. Main reason is that it's not clear what's the best approach to get such tests merged. Do we want a new suite? How often does it run? How long does it take? Do we have enough resources? etc. Do we have plans re this? Bugs/tickets/etc.? Do we want to do something? Specifically, do we want (eventually) many more suites? This seems unmanageable. A few suites, perhaps (also) using lago snapshots? Didn't try that myself, might be useful and relevant. Some examples: https://gerrit.ovirt.org/79203 https://gerrit.ovirt.org/79215 https://gerrit.ovirt.org/84813 https://gerrit.ovirt.org/88201 https://gerrit.ovirt.org/88234 And current one - this and the next ones in the stack: https://gerrit.ovirt.org/88331 Best regards, -- Didi From dron at redhat.com Mon Mar 5 13:07:42 2018 From: dron at redhat.com (Dafna Ron) Date: Mon, 5 Mar 2018 13:07:42 +0000 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 (ovirt-engine) ] [ 05-03-2018 ] [ 006_migrations.migrate_vm ] Message-ID: Hi, We have a failure in OST on test 006_migrations.migrate_vm. >From what I can see, the migration succeeded but because there was a KeyError: 'cpuUsage' engine assume the v is down and tries to retry migration. the re-try fails with vm already exists. *Link and headline of suspected patches: *Failed change: *https://gerrit.ovirt.org/#/c/88432/2 - * *engine : Events coming too soon on refresh caps* *CQ reported this as root ca*use but I don't think its related as well: *https://gerrit.ovirt.org/#/c/88404/ - * *core: Auto SD selection on template version update* *Link to Job:http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1021/ Link to all logs:http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1021/artifact/ (Relevant) error snippet from the log: * *migration starts and finishs *but fails on vm state on eyError: 'cpuUsage' *2018-03-05 05:21:33,269-0500 INFO (jsonrpc/3) [api.virt] START migrate(params={u'incomingLimit': 2, u'tunneled': u'false', u'dstqemu': u'192.0.3.2', u'autoConverge': u'false', u'src': u'lago-basic-suite-4-2-host-0', u'enableGuestEvents': False, u'dst': u'lago-basic-suite-4-2-host-1:54321', u'vmId': u'a80596a2-f57b-4878-adc3-772363e42783', u'abortOnError': u'true', u'outgoingLimit': 2, u'compressed': u'false', u'method': u'online'}) from=::ffff:192.168.200.2,42452, flow_id=123f5fd3-81a4-4ba9-9034-525674696629, vmId=a80596a2-f57b-4878-adc3-772363e42783 (api:46)2018-03-05 05:21:33,271-0500 INFO (jsonrpc/3) [api.virt] FINISH migrate return={'status': {'message': 'Migration in progress', 'code': 0}, 'progress': 0} from=::ffff:192.168.200.2,42452, flow_id=123f5fd3-81a4-4ba9-9034-525674696629, vmId=a80596a2-f57b-4878-adc3-772363e42783 (api:52)2018-03-05 05:21:33,271-0500 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call VM.migrate succeeded in 0.00 seconds (__init__:573)2018-03-05 05:21:33,473-0500 INFO (monitor/bd2f874) [IOProcessClient] Closing client ioprocess-0 (__init__:583)2018-03-05 05:21:34,936-0500 INFO (migsrc/a80596a2) [virt.vm] (vmId='a80596a2-f57b-4878-adc3-772363e42783') Creation of destination VM took: 1 seconds (migration:473)2018-03-05 05:21:34,936-0500 INFO (migsrc/a80596a2) [virt.vm] (vmId='a80596a2-f57b-4878-adc3-772363e42783') starting migration to qemu+tls://lago-basic-suite-4-2-host-1/system with miguri tcp://192.0.3.2 (migration:502)2018-03-05 05:21:36,356-0500 INFO (jsonrpc/7) [api.host] START getAllVmStats() from=::ffff:192.168.200.2,42452 (api:46)2018-03-05 05:21:36,359-0500 INFO (jsonrpc/7) [api.host] FINISH getAllVmStats return={'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} from=::ffff:192.168.200.2,42452 (api:52)2018-03-05 05:21:36,361-0500 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:573)2018-03-05 05:21:38,861-0500 INFO (libvirt/events) [virt.vm] (vmId='a80596a2-f57b-4878-adc3-772363e42783') CPU stopped: onSuspend (vm:6063)2018-03-05 05:21:40,109-0500 WARN (periodic/0) [virt.periodic.VmDispatcher] could not run on ['a80596a2-f57b-4878-adc3-772363e42783'] (periodic:323)2018-03-05 05:21:40,109-0500 INFO (migsrc/a80596a2) [virt.vm] (vmId='a80596a2-f57b-4878-adc3-772363e42783') migration took 6 seconds to complete (migration:514)2018-03-05 05:21:40,110-0500 INFO (migsrc/a80596a2) [virt.vm] (vmId='a80596a2-f57b-4878-adc3-772363e42783') Changed state to Down: Migration succeeded (code=4) (vm:1677)2018-03-05 05:21:40,116-0500 INFO (migsrc/a80596a2) [virt.vm] (vmId='a80596a2-f57b-4878-adc3-772363e42783') Stopping connection (guestagent:438)2018-03-05 05:21:41,118-0500 WARN (periodic/3) [virt.periodic.VmDispatcher] could not run on ['a80596a2-f57b-4878-adc3-772363e42783'] (periodic:323)2018-03-05 05:21:42,063-0500 WARN (periodic/2) [virt.vmstats] Missing stat: 'balloon.current' for vm a80596a2-f57b-4878-adc3-772363e42783 (vmstats:552)2018-03-05 05:21:42,064-0500 ERROR (periodic/2) [virt.vmstats] VM metrics collection failed (vmstats:264)Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/virt/vmstats.py", line 197, in send_metrics data[prefix + '.cpu.usage'] = stat['cpuUsage']KeyError: 'cpuUsage'* ** -------------- next part -------------- An HTML attachment was scrubbed... URL: From sbonazzo at redhat.com Mon Mar 5 13:10:48 2018 From: sbonazzo at redhat.com (Sandro Bonazzola) Date: Mon, 5 Mar 2018 14:10:48 +0100 Subject: [ovirt-devel] hosted-engine ovirt-system-tests In-Reply-To: References: Message-ID: Il 5 mar 2018 2:02 PM, "Yedidyah Bar David" ha scritto: Hi all, Recently I have been pushing various patches to OST in order to verify specific bugs/fixes I was working on, using them with the "manual" jenkins job but with no immediate intention to get them merged. Main reason is that it's not clear what's the best approach to get such tests merged. Do we want a new suite? How often does it run? How long does it take? Do we have enough resources? etc. Do we have plans re this? Bugs/tickets/etc.? Do we want to do something? Specifically, do we want (eventually) many more suites? This seems unmanageable. A few suites, perhaps (also) using lago snapshots? Didn't try that myself, might be useful and relevant. Didn't check since I am on mobile phone but if all the tests are regression tests I think itv would be useful run the test daily. AFAIK we don't have tests running per patch on hosted engine commits so it should be ok adding them to existing test suite Some examples: https://gerrit.ovirt.org/79203 https://gerrit.ovirt.org/79215 https://gerrit.ovirt.org/84813 https://gerrit.ovirt.org/88201 https://gerrit.ovirt.org/88234 And current one - this and the next ones in the stack: https://gerrit.ovirt.org/88331 Best regards, -- Didi _______________________________________________ Infra mailing list Infra at ovirt.org http://lists.ovirt.org/mailman/listinfo/infra -------------- next part -------------- An HTML attachment was scrubbed... URL: From didi at redhat.com Mon Mar 5 13:22:57 2018 From: didi at redhat.com (Yedidyah Bar David) Date: Mon, 5 Mar 2018 15:22:57 +0200 Subject: [ovirt-devel] hosted-engine ovirt-system-tests In-Reply-To: References: Message-ID: On Mon, Mar 5, 2018 at 3:10 PM, Sandro Bonazzola wrote: > > > Il 5 mar 2018 2:02 PM, "Yedidyah Bar David" ha scritto: > > Hi all, > > Recently I have been pushing various patches to OST in order to verify > specific bugs/fixes I was working on, using them with the "manual" > jenkins job but with no immediate intention to get them merged. Main > reason is that it's not clear what's the best approach to get such > tests merged. Do we want a new suite? How often does it run? How long > does it take? Do we have enough resources? etc. > > Do we have plans re this? Bugs/tickets/etc.? > > Do we want to do something? > > Specifically, do we want (eventually) many more suites? This seems > unmanageable. A few suites, perhaps (also) using lago snapshots? > Didn't try that myself, might be useful and relevant. > > > Didn't check since I am on mobile phone but if all the tests are > regression tests I think itv would be useful run the test daily. AFAIK we > don't have tests running per patch on hosted engine commits so it should be > ok adding them to existing test suite > > OK, cleaned up a bit the last (current) one to make it ready for review, can be merged once the fix it verifies [1] is merged. But some of the other patches are for the engine, which does run per-push, thus does risk a considerable slowdown if simply merged. [1] https://gerrit.ovirt.org/#/q/Ib3f2294acb200e43cb61c44d9c1960338368527b,n,z > > > Some examples: > https://gerrit.ovirt.org/79203 > https://gerrit.ovirt.org/79215 > https://gerrit.ovirt.org/84813 > https://gerrit.ovirt.org/88201 > https://gerrit.ovirt.org/88234 > > And current one - this and the next ones in the stack: > https://gerrit.ovirt.org/88331 > > Best regards, > -- > Didi > _______________________________________________ > Infra mailing list > Infra at ovirt.org > http://lists.ovirt.org/mailman/listinfo/infra > > > -- Didi -------------- next part -------------- An HTML attachment was scrubbed... URL: From ykaul at redhat.com Mon Mar 5 15:26:28 2018 From: ykaul at redhat.com (Yaniv Kaul) Date: Mon, 5 Mar 2018 17:26:28 +0200 Subject: [ovirt-devel] hosted-engine ovirt-system-tests In-Reply-To: References: Message-ID: On Mon, Mar 5, 2018 at 3:01 PM, Yedidyah Bar David wrote: > Hi all, > > Recently I have been pushing various patches to OST in order to verify > specific bugs/fixes I was working on, using them with the "manual" > jenkins job but with no immediate intention to get them merged. Main > reason is that it's not clear what's the best approach to get such > tests merged. Do we want a new suite? How often does it run? How long > does it take? Do we have enough resources? etc. > I'll answer generally, since specifically for each test we'll need to see. The guidelines that I have (in my head, not written down anywhere) are really: 1. Be efficient in resource - HW mainly. My guideline is quite simple - can it run on my laptop (8GB of RAM). Not everything can fit of course on my laptop (the performance suite, the upcoming OpenShift on oVirt suite, etc.), but I try. 2. Be quick - whatever we can test in parallel to other tests, is much preferred. We have quite a bit of 'dead time' between tests, we need to use them. 3. Bend the rules, but don't cheat - I move the yum cache repo to /dev/shm, etc. - to make things quicker, but I don't cheat and install deps. ahead of time, etc. 4. Most suites do not run too often (several times a day) - I think it's OK to add to those that run once a day or so. 5. Strive for others (QE!) to contribute to the suite. The more we can collaborate, the better. 6. Generally, we have enough gaps in our positive tests, that I rather not introduce negative tests. 7. Whatever we can do in order to ensure QE does not get a dead-on-arrival or broken functionality build - the better. > > Do we have plans re this? Bugs/tickets/etc.? > Bugzilla, but I did not see anything there for quite some time. > > Do we want to do something? > > Specifically, do we want (eventually) many more suites? This seems > unmanageable. A few suites, perhaps (also) using lago snapshots? > Didn't try that myself, might be useful and relevant. > > Some examples: > https://gerrit.ovirt.org/79203 > https://gerrit.ovirt.org/79215 > https://gerrit.ovirt.org/84813 > https://gerrit.ovirt.org/88201 > https://gerrit.ovirt.org/88234 > > And current one - this and the next ones in the stack: > https://gerrit.ovirt.org/88331 Many of those change the default installation and test the 'interesting' combinations - which I think is great. I'd be happy for a setup with custom '3rd party' certificate, Kerberos auth., ovirtmgmt on a VLAN interface, etc. Y. > > Best regards, > -- > Didi > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sbonazzo at redhat.com Tue Mar 6 07:11:44 2018 From: sbonazzo at redhat.com (Sandro Bonazzola) Date: Tue, 6 Mar 2018 08:11:44 +0100 Subject: [ovirt-devel] Fwd: Still Failing: oVirt/ovirt-engine#4888 (master - 1cc377f) In-Reply-To: <5a9e3adce891d_43fb8dce18a501883848@3936add0-41b2-48e4-89b1-c2a08e956057.mail> References: <5a9e3adce891d_43fb8dce18a501883848@3936add0-41b2-48e4-89b1-c2a08e956057.mail> Message-ID: Hi, not sure who's monitoring Travis failures and maintaining Travis jobs, but looks like we have an error in the test suite It's currently failing on: ------------------------------------------------------- T E S T S ------------------------------------------------------- Running org.ovirt.engine.core.dal.job.ExecutionMessageDirectorTest Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.234 sec Running org.ovirt.engine.core.dal.dbbroker.DbConnectionUtilTest Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.936 sec <<< FAILURE! org.ovirt.engine.core.dal.dbbroker.DbConnectionUtilTest Time elapsed: 936 sec <<< FAILURE! java.lang.AssertionError: Unable to init tests Looks pretty much similar to what happens in jenkins for fcraw builds, tracked here: https://bugzilla.redhat.com/show_bug.cgi?id=1550033 ---------- Forwarded message ---------- From: Travis CI Date: 2018-03-06 7:53 GMT+01:00 Subject: Still Failing: oVirt/ovirt-engine#4888 (master - 1cc377f) To: sbonazzo at redhat.com *oVirt / ovirt-engine * (master ) Build #4888 is still failing. 4 minutes and 38 seconds *Yedidyah Bar David* 1cc377f Changeset ? packaging: setup: postgres95: Refuse to upgrade if new data dir is not empty Change-Id: I9f69ceb2b8ea773d1bb883c3e81734a3c4c8af3c Bug-Url: https://bugzilla.redhat.com/1546771 Signed-off-by: Yedidyah Bar David *Want to know about upcoming build environment updates?* Would you like to stay up-to-date with the upcoming Travis CI build environment updates? We set up a mailing list for you! Sign up here . Documentation about Travis CI Need help? Mail support ! Choose who receives these build notification emails in your configuration file . *Would you like to test your private code?* Travis CI for Private Projects could be your new best friend! -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA TRIED. TESTED. TRUSTED. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rmohr at redhat.com Tue Mar 6 08:34:47 2018 From: rmohr at redhat.com (Roman Mohr) Date: Tue, 6 Mar 2018 09:34:47 +0100 Subject: [ovirt-devel] Fwd: Still Failing: oVirt/ovirt-engine#4888 (master - 1cc377f) In-Reply-To: References: <5a9e3adce891d_43fb8dce18a501883848@3936add0-41b2-48e4-89b1-c2a08e956057.mail> Message-ID: On Tue, Mar 6, 2018 at 8:11 AM, Sandro Bonazzola wrote: > Hi, > not sure who's monitoring Travis failures and maintaining Travis jobs, > I fear that was me until I moved to KubeVirt and that currently no one monitors it. It is mostly useful to generate the sonarqube code-analysis. > but looks like we have an error in the test suite > I guess this is related to the postgresql database version provided to travis. See https://travis-ci.org/oVirt/ovirt-engine/builds/349675802#L497. It complains about: psql:./packaging/dbscripts/common_sp.sql:1305: ERROR: type jsonb does not exist Is postgres 9.2 still correct, or is there something needed to enable jsonb? See https://github.com/oVirt/ovirt-engine/blob/master/.travis.yml#L17 for the current version used. To be honest I am not sure if that free sonarqube offer still exists. You can probably just remove the .travis.yml file. Best Regards, Roman > It's currently failing on: > > ------------------------------------------------------- > T E S T S > ------------------------------------------------------- > Running org.ovirt.engine.core.dal.job.ExecutionMessageDirectorTest > Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.234 sec > Running org.ovirt.engine.core.dal.dbbroker.DbConnectionUtilTest > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.936 sec <<< FAILURE! > org.ovirt.engine.core.dal.dbbroker.DbConnectionUtilTest Time elapsed: 936 sec <<< FAILURE! > java.lang.AssertionError: Unable to init tests > > Looks pretty much similar to what happens in jenkins for fcraw builds, > tracked here: https://bugzilla.redhat.com/show_bug.cgi?id=1550033 > > > > ---------- Forwarded message ---------- > From: Travis CI > Date: 2018-03-06 7:53 GMT+01:00 > Subject: Still Failing: oVirt/ovirt-engine#4888 (master - 1cc377f) > To: sbonazzo at redhat.com > > > *oVirt / ovirt-engine > * > (master ) > Build #4888 is still failing. > > 4 minutes and 38 seconds > *Yedidyah Bar David* 1cc377f > Changeset > ? > > packaging: setup: postgres95: Refuse to upgrade if new data dir is not > empty > > Change-Id: I9f69ceb2b8ea773d1bb883c3e81734a3c4c8af3c > Bug-Url: https://bugzilla.redhat.com/1546771 > Signed-off-by: Yedidyah Bar David > > *Want to know about upcoming build environment updates?* > > Would you like to stay up-to-date with the upcoming Travis CI build > environment updates? We set up a mailing list for you! Sign up here > . > Documentation about Travis CI > Need help? Mail support ! > Choose who receives these build notification emails in your configuration > file . > > *Would you like to test your private code?* > > Travis CI for Private Projects > > could be your new best friend! > > > > -- > > SANDRO BONAZZOLA > > ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D > > Red Hat EMEA > > TRIED. TESTED. TRUSTED. > > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rmohr at redhat.com Tue Mar 6 08:39:08 2018 From: rmohr at redhat.com (Roman Mohr) Date: Tue, 6 Mar 2018 09:39:08 +0100 Subject: [ovirt-devel] Fwd: Still Failing: oVirt/ovirt-engine#4888 (master - 1cc377f) In-Reply-To: References: <5a9e3adce891d_43fb8dce18a501883848@3936add0-41b2-48e4-89b1-c2a08e956057.mail> Message-ID: On Tue, Mar 6, 2018 at 9:34 AM, Roman Mohr wrote: > > > On Tue, Mar 6, 2018 at 8:11 AM, Sandro Bonazzola > wrote: > >> Hi, >> not sure who's monitoring Travis failures and maintaining Travis jobs, >> > > I fear that was me until I moved to KubeVirt and that currently no one > monitors it. It is mostly useful to generate the sonarqube code-analysis. > > >> but looks like we have an error in the test suite >> > > > I guess this is related to the postgresql database version provided to > travis. See https://travis-ci.org/oVirt/ovirt-engine/builds/349675802#L497. > It complains about: > > psql:./packaging/dbscripts/common_sp.sql:1305: ERROR: type jsonb does not > exist > > Is postgres 9.2 still correct, or is there something needed to enable > jsonb? See https://github.com/oVirt/ovirt-engine/blob/master/. > travis.yml#L17 for the current version used. > > To be honest I am not sure if that free sonarqube offer still exists. You > can probably just remove the .travis.yml file. > Oh nemo.sonarqube.org is still free for opensource project, they just changed the mein url: https://sonarcloud.io/dashboard?id=org.ovirt.engine%3Aroot So it might still be useful. There is this other code analysis too, so up to you guys if you find it useful. > > Best Regards, > > Roman > > >> It's currently failing on: >> >> ------------------------------------------------------- >> T E S T S >> ------------------------------------------------------- >> Running org.ovirt.engine.core.dal.job.ExecutionMessageDirectorTest >> Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.234 sec >> Running org.ovirt.engine.core.dal.dbbroker.DbConnectionUtilTest >> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.936 sec <<< FAILURE! >> org.ovirt.engine.core.dal.dbbroker.DbConnectionUtilTest Time elapsed: 936 sec <<< FAILURE! >> java.lang.AssertionError: Unable to init tests >> >> Looks pretty much similar to what happens in jenkins for fcraw builds, >> tracked here: https://bugzilla.redhat.com/show_bug.cgi?id=1550033 >> >> >> >> ---------- Forwarded message ---------- >> From: Travis CI >> Date: 2018-03-06 7:53 GMT+01:00 >> Subject: Still Failing: oVirt/ovirt-engine#4888 (master - 1cc377f) >> To: sbonazzo at redhat.com >> >> >> *oVirt / ovirt-engine >> * >> (master ) >> Build #4888 is still failing. >> >> 4 minutes and 38 seconds >> *Yedidyah Bar David* 1cc377f >> Changeset >> ? >> >> packaging: setup: postgres95: Refuse to upgrade if new data dir is not >> empty >> >> Change-Id: I9f69ceb2b8ea773d1bb883c3e81734a3c4c8af3c >> Bug-Url: https://bugzilla.redhat.com/1546771 >> Signed-off-by: Yedidyah Bar David >> >> *Want to know about upcoming build environment updates?* >> >> Would you like to stay up-to-date with the upcoming Travis CI build >> environment updates? We set up a mailing list for you! Sign up here >> . >> Documentation about Travis CI >> Need help? Mail support ! >> Choose who receives these build notification emails in your configuration >> file . >> >> *Would you like to test your private code?* >> >> Travis CI for Private Projects >> >> could be your new best friend! >> >> >> >> -- >> >> SANDRO BONAZZOLA >> >> ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D >> >> Red Hat EMEA >> >> TRIED. TESTED. TRUSTED. >> >> >> _______________________________________________ >> Devel mailing list >> Devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sbonazzo at redhat.com Tue Mar 6 08:40:15 2018 From: sbonazzo at redhat.com (Sandro Bonazzola) Date: Tue, 6 Mar 2018 09:40:15 +0100 Subject: [ovirt-devel] Fwd: Still Failing: oVirt/ovirt-engine#4888 (master - 1cc377f) In-Reply-To: References: <5a9e3adce891d_43fb8dce18a501883848@3936add0-41b2-48e4-89b1-c2a08e956057.mail> Message-ID: 2018-03-06 9:39 GMT+01:00 Roman Mohr : > > > On Tue, Mar 6, 2018 at 9:34 AM, Roman Mohr wrote: > >> >> >> On Tue, Mar 6, 2018 at 8:11 AM, Sandro Bonazzola >> wrote: >> >>> Hi, >>> not sure who's monitoring Travis failures and maintaining Travis jobs, >>> >> >> I fear that was me until I moved to KubeVirt and that currently no one >> monitors it. It is mostly useful to generate the sonarqube code-analysis. >> >> >>> but looks like we have an error in the test suite >>> >> >> >> I guess this is related to the postgresql database version provided to >> travis. See https://travis-ci.org/oVirt/ovirt-engine/builds/349675802#L4 >> 97. It complains about: >> >> psql:./packaging/dbscripts/common_sp.sql:1305: ERROR: type jsonb does >> not exist >> >> Is postgres 9.2 still correct, or is there something needed to enable >> jsonb? See https://github.com/oVirt/ovirt-engine/blob/master/.travi >> s.yml#L17 for the current version used. >> > No, postgres 9.5 is needed now > >> To be honest I am not sure if that free sonarqube offer still exists. You >> can probably just remove the .travis.yml file. >> > > > Oh nemo.sonarqube.org is still free for opensource project, they just > changed the mein url: https://sonarcloud.io/dashboard?id=org.ovirt.engine% > 3Aroot > > So it might still be useful. There is this other code analysis too, so up > to you guys if you find it useful. > great, let's keep it > > >> >> Best Regards, >> >> Roman >> >> >>> It's currently failing on: >>> >>> ------------------------------------------------------- >>> T E S T S >>> ------------------------------------------------------- >>> Running org.ovirt.engine.core.dal.job.ExecutionMessageDirectorTest >>> Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.234 sec >>> Running org.ovirt.engine.core.dal.dbbroker.DbConnectionUtilTest >>> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.936 sec <<< FAILURE! >>> org.ovirt.engine.core.dal.dbbroker.DbConnectionUtilTest Time elapsed: 936 sec <<< FAILURE! >>> java.lang.AssertionError: Unable to init tests >>> >>> Looks pretty much similar to what happens in jenkins for fcraw builds, >>> tracked here: https://bugzilla.redhat.com/show_bug.cgi?id=1550033 >>> >>> >>> >>> ---------- Forwarded message ---------- >>> From: Travis CI >>> Date: 2018-03-06 7:53 GMT+01:00 >>> Subject: Still Failing: oVirt/ovirt-engine#4888 (master - 1cc377f) >>> To: sbonazzo at redhat.com >>> >>> >>> *oVirt / ovirt-engine >>> * >>> (master ) >>> Build #4888 is still failing. >>> >>> 4 minutes and 38 seconds >>> *Yedidyah Bar David* 1cc377f >>> Changeset >>> ? >>> >>> packaging: setup: postgres95: Refuse to upgrade if new data dir is >>> not empty >>> >>> Change-Id: I9f69ceb2b8ea773d1bb883c3e81734a3c4c8af3c >>> Bug-Url: https://bugzilla.redhat.com/1546771 >>> Signed-off-by: Yedidyah Bar David >>> >>> *Want to know about upcoming build environment updates?* >>> >>> Would you like to stay up-to-date with the upcoming Travis CI build >>> environment updates? We set up a mailing list for you! Sign up here >>> . >>> Documentation about Travis CI >>> Need help? Mail support ! >>> Choose who receives these build notification emails in your configuration >>> file . >>> >>> *Would you like to test your private code?* >>> >>> Travis CI for Private Projects >>> >>> could be your new best friend! >>> >>> >>> >>> -- >>> >>> SANDRO BONAZZOLA >>> >>> ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D >>> >>> Red Hat EMEA >>> >>> TRIED. TESTED. TRUSTED. >>> >>> >>> _______________________________________________ >>> Devel mailing list >>> Devel at ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/devel >>> >> >> > -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA TRIED. TESTED. TRUSTED. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sbonazzo at redhat.com Tue Mar 6 10:03:09 2018 From: sbonazzo at redhat.com (Sandro Bonazzola) Date: Tue, 6 Mar 2018 11:03:09 +0100 Subject: [ovirt-devel] Packages not landed on tested repo Message-ID: Hi, looks like we have some packages which never passed OST, never landed in ovirt-4.2-snapshot repo and are currently causing failure building the appliance from 4.2 snapshot repo: 03:17:20,893 INFO program:Error: Package: ovirt-engine-4.2.2.3-0.0.master.20180305171049.git0e81394.el7.centos.noarch (ovirt-4.2-snapshot) 03:17:20,893 INFO program:Requires: ovirt-ansible-roles >= 1.1.2 03:17:20,894 INFO program:Error: Package: ovirt-engine-4.2.2.3-0.0.master.20180305171049.git0e81394.el7.centos.noarch (ovirt-4.2-snapshot) 03:17:20,896 INFO program:Requires: ovirt-js-dependencies 03:17:20,897 INFO program:Error: Package: ovirt-engine-4.2.2.3-0.0.master.20180305171049.git0e81394.el7.centos.noarch (ovirt-4.2-snapshot) 03:17:20,897 INFO program:Requires: ovirt-engine-wildfly-overlay >= 11.0.1 03:17:20,899 INFO program:Error: Package: ovirt-engine-api-explorer-0.0.3-0.alpha.1.20180215git9b54c9c.el7.centos.noarch (ovirt-4.2-snapshot) 03:17:20,900 INFO program:Requires: ovirt-js-dependencies >= 1.2 I'm now trying to get the missing packages pass thorough the whole testing pipeline but be aware those packages have already been released 3 months ago so they are stable and manually tested and were passing master OST. I've no clue on why they are missing in the 4.2 testing pipeline. -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA TRIED. TESTED. TRUSTED. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bkorren at redhat.com Tue Mar 6 13:00:34 2018 From: bkorren at redhat.com (Barak Korren) Date: Tue, 6 Mar 2018 15:00:34 +0200 Subject: [ovirt-devel] Packages not landed on tested repo In-Reply-To: References: Message-ID: On 6 March 2018 at 12:03, Sandro Bonazzola wrote: > > Hi, > > looks like we have some packages which never passed OST, never landed in ovirt-4.2-snapshot repo > > and are currently causing failure building the appliance from 4.2 snapshot repo: > > > 03:17:20,893 INFO program:Error: Package: ovirt-engine-4.2.2.3-0.0.master.20180305171049.git0e81394.el7.centos.noarch (ovirt-4.2-snapshot) > 03:17:20,893 INFO program:Requires: ovirt-ansible-roles >= 1.1.2 > 03:17:20,894 INFO program:Error: Package: ovirt-engine-4.2.2.3-0.0.master.20180305171049.git0e81394.el7.centos.noarch (ovirt-4.2-snapshot) > 03:17:20,896 INFO program:Requires: ovirt-js-dependencies > 03:17:20,897 INFO program:Error: Package: ovirt-engine-4.2.2.3-0.0.master.20180305171049.git0e81394.el7.centos.noarch (ovirt-4.2-snapshot) > 03:17:20,897 INFO program:Requires: ovirt-engine-wildfly-overlay >= 11.0.1 > 03:17:20,899 INFO program:Error: Package: ovirt-engine-api-explorer-0.0.3-0.alpha.1.20180215git9b54c9c.el7.centos.noarch (ovirt-4.2-snapshot) > 03:17:20,900 INFO program:Requires: ovirt-js-dependencies >= 1.2 > > I'm now trying to get the missing packages pass thorough the whole testing > pipeline but be aware those packages have already been released 3 months > ago so they are stable and manually tested and were passing master OST. > I've no clue on why they are missing in the 4.2 testing pipeline. > 4.2 CQ was completely broken for a long time (engine was completely missing), so nothing sent to it passed. We should have probably re-issued merges once we stabilized it. -- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted -------------- next part -------------- An HTML attachment was scrubbed... URL: From mperina at redhat.com Tue Mar 6 15:09:33 2018 From: mperina at redhat.com (Martin Perina) Date: Tue, 6 Mar 2018 16:09:33 +0100 Subject: [ovirt-devel] Fwd: Still Failing: oVirt/ovirt-engine#4888 (master - 1cc377f) In-Reply-To: References: <5a9e3adce891d_43fb8dce18a501883848@3936add0-41b2-48e4-89b1-c2a08e956057.mail> Message-ID: On Tue, Mar 6, 2018 at 9:34 AM, Roman Mohr wrote: > > > On Tue, Mar 6, 2018 at 8:11 AM, Sandro Bonazzola > wrote: > >> Hi, >> not sure who's monitoring Travis failures and maintaining Travis jobs, >> > > I fear that was me until I moved to KubeVirt and that currently no one > monitors it. It is mostly useful to generate the sonarqube code-analysis. > > >> but looks like we have an error in the test suite >> > > > I guess this is related to the postgresql database version provided to > travis. See https://travis-ci.org/oVirt/ovirt-engine/builds/349675802#L497. > It complains about: > > psql:./packaging/dbscripts/common_sp.sql:1305: ERROR: type jsonb does not > exist > > Is postgres 9.2 still correct, or is there something needed to enable > jsonb? See https://github.com/oVirt/ovirt-engine/blob/master/. > travis.yml#L17 for the current version used. > ?No, PostgreSQL 9.5 is required for oVirt 4.2+? > To be honest I am not sure if that free sonarqube offer still exists. You > can probably just remove the .travis.yml file. > > Best Regards, > > Roman > > >> It's currently failing on: >> >> ------------------------------------------------------- >> T E S T S >> ------------------------------------------------------- >> Running org.ovirt.engine.core.dal.job.ExecutionMessageDirectorTest >> Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.234 sec >> Running org.ovirt.engine.core.dal.dbbroker.DbConnectionUtilTest >> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.936 sec <<< FAILURE! >> org.ovirt.engine.core.dal.dbbroker.DbConnectionUtilTest Time elapsed: 936 sec <<< FAILURE! >> java.lang.AssertionError: Unable to init tests >> >> Looks pretty much similar to what happens in jenkins for fcraw builds, >> tracked here: https://bugzilla.redhat.com/show_bug.cgi?id=1550033 >> >> >> >> ---------- Forwarded message ---------- >> From: Travis CI >> Date: 2018-03-06 7:53 GMT+01:00 >> Subject: Still Failing: oVirt/ovirt-engine#4888 (master - 1cc377f) >> To: sbonazzo at redhat.com >> >> >> *oVirt / ovirt-engine >> * >> (master ) >> Build #4888 is still failing. >> >> 4 minutes and 38 seconds >> *Yedidyah Bar David* 1cc377f >> Changeset >> ? >> >> packaging: setup: postgres95: Refuse to upgrade if new data dir is not >> empty >> >> Change-Id: I9f69ceb2b8ea773d1bb883c3e81734a3c4c8af3c >> Bug-Url: https://bugzilla.redhat.com/1546771 >> Signed-off-by: Yedidyah Bar David >> >> *Want to know about upcoming build environment updates?* >> >> Would you like to stay up-to-date with the upcoming Travis CI build >> environment updates? We set up a mailing list for you! Sign up here >> . >> Documentation about Travis CI >> Need help? Mail support ! >> Choose who receives these build notification emails in your configuration >> file . >> >> *Would you like to test your private code?* >> >> Travis CI for Private Projects >> >> could be your new best friend! >> >> >> >> -- >> >> SANDRO BONAZZOLA >> >> ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D >> >> Red Hat EMEA >> >> TRIED. TESTED. TRUSTED. >> >> >> _______________________________________________ >> Devel mailing list >> Devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel >> > > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -- Martin Perina Associate Manager, Software Engineering Red Hat Czech s.r.o. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ederevea at redhat.com Tue Mar 6 23:51:06 2018 From: ederevea at redhat.com (Evgheni Dereveanchin) Date: Wed, 7 Mar 2018 00:51:06 +0100 Subject: [ovirt-devel] planned Jenkins restart Message-ID: Hi everyone, I'll be performing a planned Jenkins restart within the next hour. No new jobs will be scheduled during this maintenance period. I will inform you once it is over. Regards, Evgheni Dereveanchin -------------- next part -------------- An HTML attachment was scrubbed... URL: From ederevea at redhat.com Wed Mar 7 01:07:37 2018 From: ederevea at redhat.com (Evgheni Dereveanchin) Date: Wed, 7 Mar 2018 02:07:37 +0100 Subject: [ovirt-devel] planned Jenkins restart In-Reply-To: References: Message-ID: Maintenance completed, Jenkins back up and running. Plugin security updates were installed this time: *https://ovirt-jira.atlassian.net/browse/OVIRT-1912 * As always - if you see any issues please report them to Jira. Regards, Evgheni Dereveanchin On Wed, Mar 7, 2018 at 12:51 AM, Evgheni Dereveanchin wrote: > Hi everyone, > > I'll be performing a planned Jenkins restart within the next hour. > No new jobs will be scheduled during this maintenance period. > I will inform you once it is over. > > Regards, > Evgheni Dereveanchin > > -- Regards, Evgheni Dereveanchin -------------- next part -------------- An HTML attachment was scrubbed... URL: From ahadas at redhat.com Wed Mar 7 08:42:31 2018 From: ahadas at redhat.com (Arik Hadas) Date: Wed, 7 Mar 2018 10:42:31 +0200 Subject: [ovirt-devel] [ovirt-users] Unremovable disks created through the API In-Reply-To: <20180306211941.GI2787@redhat.com> References: <20180306191824.GA27515@redhat.com> <20180306211941.GI2787@redhat.com> Message-ID: On Tue, Mar 6, 2018 at 11:19 PM, Richard W.M. Jones wrote: > On Tue, Mar 06, 2018 at 11:14:40PM +0200, Arik Hadas wrote: > > On Tue, Mar 6, 2018 at 9:18 PM, Richard W.M. Jones > > wrote: > > > > > > > > I've been playing with disk uploads through the API. As a result > > > I now have lots of disks in the states "Paused by System" and > > > "Paused by User". They are not attached to any VM, and I'm logged > > > in as admin at internal, but there seems to be no way to use them. > > > Even worse I've now run out of space so can't do anything else. > > > > > > How can I remove them? > > > > > > > Screenshot: http://oirase.annexia.org/tmp/ovirt.png > > > > > > Hi Richard, > > > > Selecting Upload->Cancel at that tab will remove such a disk. > > Note that it may take a minute or two. > > Yes, that works, thanks. > (Moving to devel-list) BTW, I think that the import process should include a preliminary phase where ovirt-engine is informed that the import process starts. Currently, IIUC, the new process is designed to be: 1. virt-v2v uploads the disks 2. virt-v2v calls an API with the OVF configuration so ovirt-engine will add the VM and attach the uploaded disks to that VM IMHO, the process should be comprised of: 1. virt-v2v calls an API with the (probably partial since the OS and other things are unknown at that point) OVF configuration 2. virt-v2v uploads the disks 3. virt-v2v provides the up-to-date configuration Step #1 will enable ovirt-engine: 1. Most importantly, to cleanup uploaded disks in case of an error during the import process. Otherwise, we require the client to clean them up, which can be challenging (e.g., if the virt-v2v process crashes). 2. To inform the user that the process has started - so he won't get surprised by seeing disks being uploaded suddenly. That will give a context to these upload operations. 3. To inform the user about the progress of the import process, much like we do today when importing VMs from vSphere to RHV. 4. To perform validations on the (partial) VM configuration, e.g., verifying that no VM with the same name exists/verifying there is enough space (optionally mapping different disks to different storage devices) and so on, before uploading the disks. The gaps I see: 1. We don't have a command for step #1 yet but that's something we can provide relatively quickly. We need it also to support uploading an OVA via oVirt's webadmin. 2. We have a command for step #3, but it is not exposed via the API. > > Rich. > > -- > Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~ > rjones > Read my programming and virtualization blog: http://rwmj.wordpress.com > Fedora Windows cross-compiler. Compile Windows programs, test, and > build Windows installers. Over 100 libraries supported. > http://fedoraproject.org/wiki/MinGW > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rjones at redhat.com Wed Mar 7 10:05:53 2018 From: rjones at redhat.com (Richard W.M. Jones) Date: Wed, 7 Mar 2018 10:05:53 +0000 Subject: [ovirt-devel] [ovirt-users] Unremovable disks created through the API In-Reply-To: References: <20180306191824.GA27515@redhat.com> <20180306211941.GI2787@redhat.com> Message-ID: <20180307100553.GJ2787@redhat.com> On Wed, Mar 07, 2018 at 10:42:31AM +0200, Arik Hadas wrote: > (Moving to devel-list) > BTW, I think that the import process should include a preliminary phase > where ovirt-engine is informed that the import process starts. > > Currently, IIUC, the new process is designed to be: > 1. virt-v2v uploads the disks > 2. virt-v2v calls an API with the OVF configuration so ovirt-engine will > add the VM and attach the uploaded disks to that VM Yes, this is how it happens now, see: https://www.redhat.com/archives/libguestfs/2018-March/msg00021.html > IMHO, the process should be comprised of: > 1. virt-v2v calls an API with the (probably partial since the OS and other > things are unknown at that point) OVF configuration Almost nothing is known at this point, I'm not sure what we could provide. Perhaps just number and virtual size of disks. It doesn't sound like it would be OVF, but something else. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/ From tgolembi at redhat.com Wed Mar 7 11:01:12 2018 From: tgolembi at redhat.com (=?UTF-8?B?VG9tw6HFoSBHb2xlbWJpb3Zza8O9?=) Date: Wed, 7 Mar 2018 12:01:12 +0100 Subject: [ovirt-devel] [ovirt-users] Unremovable disks created through the API In-Reply-To: References: <20180306191824.GA27515@redhat.com> <20180306211941.GI2787@redhat.com> Message-ID: <20180307120112.7347f2d3@fiorina> Hi, this sounds like a good idea in general. Few interconnected questions though... On Wed, 7 Mar 2018 10:42:31 +0200 Arik Hadas wrote: > IMHO, the process should be comprised of: > 1. virt-v2v calls an API with the (probably partial since the OS and other > things are unknown at that point) OVF configuration > 2. virt-v2v uploads the disks > 3. virt-v2v provides the up-to-date configuration > > Step #1 will enable ovirt-engine: > 1. Most importantly, to cleanup uploaded disks in case of an error during > the import process. Otherwise, we require the client to clean them up, > which can be challenging (e.g., if the virt-v2v process crashes). Who will handle the removal in case of problems? Engine after timeout? Or is the only benefit that administrator can remove all disks in one step by removing the VM? Note that the uploads do not timeout at the moment. However strange that might be. So I assume removing the disks/VM will be impossible anyway because of locking. > 2. To inform the user that the process has started - so he won't get > surprised by seeing disks being uploaded suddenly. That will give a context > to these upload operations. The uploaded disks will still remain unattached though. Or do you plan for Engine to create and attach the disks? > 3. To inform the user about the progress of the import process, much like > we do today when importing VMs from vSphere to RHV. How will this be handled? Will Engine report the progress in the Virtual Machines view and compute something based on the upload progress? Tomas > 4. To perform validations on the (partial) VM configuration, e.g., > verifying that no VM with the same name exists/verifying there is enough > space (optionally mapping different disks to different storage devices) and > so on, before uploading the disks. > > > The gaps I see: > 1. We don't have a command for step #1 yet but that's something we can > provide relatively quickly. We need it also to support uploading an OVA via > oVirt's webadmin. > 2. We have a command for step #3, but it is not exposed via the API. -- Tom?? Golembiovsk? From ahadas at redhat.com Wed Mar 7 11:26:39 2018 From: ahadas at redhat.com (Arik Hadas) Date: Wed, 7 Mar 2018 13:26:39 +0200 Subject: [ovirt-devel] [ovirt-users] Unremovable disks created through the API In-Reply-To: <20180307100553.GJ2787@redhat.com> References: <20180306191824.GA27515@redhat.com> <20180306211941.GI2787@redhat.com> <20180307100553.GJ2787@redhat.com> Message-ID: On Wed, Mar 7, 2018 at 12:05 PM, Richard W.M. Jones wrote: > On Wed, Mar 07, 2018 at 10:42:31AM +0200, Arik Hadas wrote: > > (Moving to devel-list) > > BTW, I think that the import process should include a preliminary phase > > where ovirt-engine is informed that the import process starts. > > > > Currently, IIUC, the new process is designed to be: > > 1. virt-v2v uploads the disks > > 2. virt-v2v calls an API with the OVF configuration so ovirt-engine will > > add the VM and attach the uploaded disks to that VM > > Yes, this is how it happens now, see: > > https://www.redhat.com/archives/libguestfs/2018-March/msg00021.html > > > IMHO, the process should be comprised of: > > 1. virt-v2v calls an API with the (probably partial since the OS and > other > > things are unknown at that point) OVF configuration > > Almost nothing is known at this point, I'm not sure what we could > provide. Perhaps just number and virtual size of disks. It doesn't > sound like it would be OVF, but something else. > Interesting, that contradicts my intuition - I would imagine that most of the things are actually known (the things that appear in the top-level part of the domain xml: memory size, memory size, num of CPUs, name,.. ) and only things that depend on the content of the disks or things that depend on installations during the conversion are unknown. But anyway, it is enough IMO to send the name, memory, CPU and size of the disks to present something useful to the user and make the necessary validations at that point. > > Rich. > > -- > Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~ > rjones > Read my programming and virtualization blog: http://rwmj.wordpress.com > virt-df lists disk usage of guests without needing to install any > software inside the virtual machine. Supports Linux and Windows. > http://people.redhat.com/~rjones/virt-df/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rjones at redhat.com Wed Mar 7 11:41:18 2018 From: rjones at redhat.com (Richard W.M. Jones) Date: Wed, 7 Mar 2018 11:41:18 +0000 Subject: [ovirt-devel] [ovirt-users] Unremovable disks created through the API In-Reply-To: References: <20180306191824.GA27515@redhat.com> <20180306211941.GI2787@redhat.com> <20180307100553.GJ2787@redhat.com> Message-ID: <20180307114118.GL2787@redhat.com> On Wed, Mar 07, 2018 at 01:26:39PM +0200, Arik Hadas wrote: > Interesting, that contradicts my intuition - I would imagine that most of > the things are actually known (the things that appear in the top-level part > of the domain xml: memory size, memory size, num of CPUs, name,.. ) and > only things that depend on the content of the disks or things that depend > on installations during the conversion are unknown. > But anyway, it is enough IMO to send the name, memory, CPU and size of the > disks to present something useful to the user and make the necessary > validations at that point. Some of those things are known, but they didn't seem to me to be that interesting for oVirt to know in advance. In any case what's precisely known before conversion is: (1) The 'source' struct and sub-structs: https://github.com/libguestfs/libguestfs/blob/ba53251ab912b8bac9e00c1022adc6ba9bdf70a3/v2v/types.mli#L59 (2) The 'overlay' struct (one per disk): https://github.com/libguestfs/libguestfs/blob/ba53251ab912b8bac9e00c1022adc6ba9bdf70a3/v2v/types.mli#L175 Note only virtual disk size is known, which is near to useless for provisioning storage. (3) The 'target' struct (one per disk): https://github.com/libguestfs/libguestfs/blob/ba53251ab912b8bac9e00c1022adc6ba9bdf70a3/v2v/types.mli#L191 What's unknown are guest capabilities (hence nothing about what devices should be presented to the guest), inspection data, target bus mapping, real size of disks, etc. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top From ahadas at redhat.com Wed Mar 7 13:12:58 2018 From: ahadas at redhat.com (Arik Hadas) Date: Wed, 7 Mar 2018 15:12:58 +0200 Subject: [ovirt-devel] [ovirt-users] Unremovable disks created through the API In-Reply-To: <20180307120112.7347f2d3@fiorina> References: <20180306191824.GA27515@redhat.com> <20180306211941.GI2787@redhat.com> <20180307120112.7347f2d3@fiorina> Message-ID: On Wed, Mar 7, 2018 at 1:01 PM, Tom?? Golembiovsk? wrote: > Hi, > > this sounds like a good idea in general. Few interconnected questions > though... > > On Wed, 7 Mar 2018 10:42:31 +0200 > Arik Hadas wrote: > > > IMHO, the process should be comprised of: > > 1. virt-v2v calls an API with the (probably partial since the OS and > other > > things are unknown at that point) OVF configuration > > 2. virt-v2v uploads the disks > > 3. virt-v2v provides the up-to-date configuration > > > > Step #1 will enable ovirt-engine: > > 1. Most importantly, to cleanup uploaded disks in case of an error during > > the import process. Otherwise, we require the client to clean them up, > > which can be challenging (e.g., if the virt-v2v process crashes). > > Who will handle the removal in case of problems? Engine after timeout? > Or is the only benefit that administrator can remove all disks in one > step by removing the VM? > So I was thinking that the wrapper command will hold the amount of disks that should be uploaded for the VM. If for some predefined period of time no new disk is added or an upload doesn't make any progress (assuming the uploads are done sequentially), to fail the import operation and that would roll back the resources (disks, VMs) that were created as part of the import process. > > Note that the uploads do not timeout at the moment. However strange that > might be. So I assume removing the disks/VM will be impossible anyway > because of locking. > Yeah, I think no timeout was defined because uploading from the browser is relatively fragile and we didn't want the upload to fail, and the partial disks to be removed, due to browser issues but rather to be able to resume their upload. But different logic can be implemented in the wrapping command. As for locking, we don't have to call RemoveDisk but instead, to terminate the upload which will eventually remove the disks. > > > > 2. To inform the user that the process has started - so he won't get > > surprised by seeing disks being uploaded suddenly. That will give a > context > > to these upload operations. > > The uploaded disks will still remain unattached though. Or do you plan > for Engine to create and attach the disks? > Right, so when we have that "context" and a VM entity in the databse, we can attach the disks to the VM when they are created. > > > > 3. To inform the user about the progress of the import process, much like > > we do today when importing VMs from vSphere to RHV. > > How will this be handled? Will Engine report the progress in the Virtual > Machines view and compute something based on the upload progress? > Yes, I don't see why not showing that in the status field (at the VMs tab) as we do today for VMs that are being imported. The engine would need to know the estimated actual sizes of the disks in order to compute it though. > > Tomas > > > 4. To perform validations on the (partial) VM configuration, e.g., > > verifying that no VM with the same name exists/verifying there is enough > > space (optionally mapping different disks to different storage devices) > and > > so on, before uploading the disks. > > > > > > The gaps I see: > > 1. We don't have a command for step #1 yet but that's something we can > > provide relatively quickly. We need it also to support uploading an OVA > via > > oVirt's webadmin. > > 2. We have a command for step #3, but it is not exposed via the API. > > > -- > Tom?? Golembiovsk? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ahadas at redhat.com Wed Mar 7 13:20:54 2018 From: ahadas at redhat.com (Arik Hadas) Date: Wed, 7 Mar 2018 15:20:54 +0200 Subject: [ovirt-devel] [ovirt-users] Unremovable disks created through the API In-Reply-To: <20180307114118.GL2787@redhat.com> References: <20180306191824.GA27515@redhat.com> <20180306211941.GI2787@redhat.com> <20180307100553.GJ2787@redhat.com> <20180307114118.GL2787@redhat.com> Message-ID: On Wed, Mar 7, 2018 at 1:41 PM, Richard W.M. Jones wrote: > On Wed, Mar 07, 2018 at 01:26:39PM +0200, Arik Hadas wrote: > > Interesting, that contradicts my intuition - I would imagine that most of > > the things are actually known (the things that appear in the top-level > part > > of the domain xml: memory size, memory size, num of CPUs, name,.. ) and > > only things that depend on the content of the disks or things that depend > > on installations during the conversion are unknown. > > But anyway, it is enough IMO to send the name, memory, CPU and size of > the > > disks to present something useful to the user and make the necessary > > validations at that point. > > Some of those things are known, but they didn't seem to me to be that > interesting for oVirt to know in advance. In any case what's > precisely known before conversion is: > > (1) The 'source' struct and sub-structs: > > https://github.com/libguestfs/libguestfs/blob/ > ba53251ab912b8bac9e00c1022adc6ba9bdf70a3/v2v/types.mli#L59 > > (2) The 'overlay' struct (one per disk): > > https://github.com/libguestfs/libguestfs/blob/ > ba53251ab912b8bac9e00c1022adc6ba9bdf70a3/v2v/types.mli#L175 > > Note only virtual disk size is known, which is near to useless for > provisioning storage. > > (3) The 'target' struct (one per disk): > > https://github.com/libguestfs/libguestfs/blob/ > ba53251ab912b8bac9e00c1022adc6ba9bdf70a3/v2v/types.mli#L191 > > What's unknown are guest capabilities (hence nothing about what > devices should be presented to the guest), inspection data, target bus > mapping, real size of disks, etc. > > I see. I think it is sufficient - the information from the 'source' struct seems enough just to produce a representative VM entity in the database that would be reflected in the UI with status 'importing' and for general validations, and the estimated size on the 'target' struct would be enough for storage validations and optionally for choosing the right target storage domain. The other things are relatively hidden in oVirt's UI and can be added at the last phase. BTW, that's how import from VMware/Xen currently works - we add a VM entity based on the domain XML we get from vCenter and at the last phase add the missing parts when getting the generated OVF from virt-v2v. So for instance, the VM would have no graphics device until that last phase. > Rich. > > -- > Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~ > rjones > Read my programming and virtualization blog: http://rwmj.wordpress.com > virt-top is 'top' for virtual machines. Tiny program with many > powerful monitoring features, net stats, disk stats, logging, etc. > http://people.redhat.com/~rjones/virt-top > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rjones at redhat.com Wed Mar 7 13:37:26 2018 From: rjones at redhat.com (Richard W.M. Jones) Date: Wed, 7 Mar 2018 13:37:26 +0000 Subject: [ovirt-devel] [ovirt-users] Unremovable disks created through the API In-Reply-To: References: <20180306191824.GA27515@redhat.com> <20180306211941.GI2787@redhat.com> <20180307120112.7347f2d3@fiorina> Message-ID: <20180307133726.GN2787@redhat.com> On Wed, Mar 07, 2018 at 03:12:58PM +0200, Arik Hadas wrote: > If for some predefined period of time no new disk is added or an upload > doesn't make any progress (assuming the uploads are done sequentially), to > fail the import operation and that would roll back the resources (disks, > VMs) that were created as part of the import process. At the moment we're actually trying to remove the disk on failure. However the disk_service.remove() API does nothing (doesn't even fail). Perhaps because the transfer isn't finalized on the error path? Anyway the code - which needs review - is: https://www.redhat.com/archives/libguestfs/2018-March/msg00024.html Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW From tgolembi at redhat.com Wed Mar 7 13:53:01 2018 From: tgolembi at redhat.com (=?UTF-8?B?VG9tw6HFoSBHb2xlbWJpb3Zza8O9?=) Date: Wed, 7 Mar 2018 14:53:01 +0100 Subject: [ovirt-devel] [ovirt-users] Unremovable disks created through the API In-Reply-To: References: <20180306191824.GA27515@redhat.com> <20180306211941.GI2787@redhat.com> <20180307120112.7347f2d3@fiorina> Message-ID: <20180307145301.3924bb59@fiorina> On Wed, 7 Mar 2018 15:12:58 +0200 Arik Hadas wrote: > On Wed, Mar 7, 2018 at 1:01 PM, Tom?? Golembiovsk? > wrote: > > > Hi, > > > > this sounds like a good idea in general. Few interconnected questions > > though... > > > > On Wed, 7 Mar 2018 10:42:31 +0200 > > Arik Hadas wrote: > > > > > IMHO, the process should be comprised of: > > > 1. virt-v2v calls an API with the (probably partial since the OS and > > other > > > things are unknown at that point) OVF configuration > > > 2. virt-v2v uploads the disks > > > 3. virt-v2v provides the up-to-date configuration > > > > > > Step #1 will enable ovirt-engine: > > > 1. Most importantly, to cleanup uploaded disks in case of an error during > > > the import process. Otherwise, we require the client to clean them up, > > > which can be challenging (e.g., if the virt-v2v process crashes). > > > > Who will handle the removal in case of problems? Engine after timeout? > > Or is the only benefit that administrator can remove all disks in one > > step by removing the VM? > > > > So I was thinking that the wrapper command will hold the amount of disks > that should be uploaded for the VM. > If for some predefined period of time no new disk is added or an upload > doesn't make any progress (assuming the uploads are done sequentially), to > fail the import operation and that would roll back the resources (disks, > VMs) that were created as part of the import process. > > > > > > Note that the uploads do not timeout at the moment. However strange that > > might be. So I assume removing the disks/VM will be impossible anyway > > because of locking. > > > > Yeah, I think no timeout was defined because uploading from the browser is > relatively fragile and we didn't want the upload to fail, and the partial > disks to be removed, due to browser issues but rather to be able to resume > their upload. But different logic can be implemented in the wrapping > command. Would it make sense to add the timeout parameter to API? That way the client can choose whether it wants one and how big. No timeout could still be the default. Tomas > As for locking, we don't have to call RemoveDisk but instead, to terminate > the upload which will eventually remove the disks. > > > > > > > > > 2. To inform the user that the process has started - so he won't get > > > surprised by seeing disks being uploaded suddenly. That will give a > > context > > > to these upload operations. > > > > The uploaded disks will still remain unattached though. Or do you plan > > for Engine to create and attach the disks? > > > > Right, so when we have that "context" and a VM entity in the databse, we > can attach the disks to the VM when they are created. > > > > > > > > > 3. To inform the user about the progress of the import process, much like > > > we do today when importing VMs from vSphere to RHV. > > > > How will this be handled? Will Engine report the progress in the Virtual > > Machines view and compute something based on the upload progress? > > > > Yes, I don't see why not showing that in the status field (at the VMs tab) > as we do today for VMs that are being imported. > The engine would need to know the estimated actual sizes of the disks in > order to compute it though. > > > > > > Tomas > > > > > 4. To perform validations on the (partial) VM configuration, e.g., > > > verifying that no VM with the same name exists/verifying there is enough > > > space (optionally mapping different disks to different storage devices) > > and > > > so on, before uploading the disks. > > > > > > > > > The gaps I see: > > > 1. We don't have a command for step #1 yet but that's something we can > > > provide relatively quickly. We need it also to support uploading an OVA > > via > > > oVirt's webadmin. > > > 2. We have a command for step #3, but it is not exposed via the API. > > > > > > -- > > Tom?? Golembiovsk? > > -- Tom?? Golembiovsk? From ahadas at redhat.com Wed Mar 7 14:07:49 2018 From: ahadas at redhat.com (Arik Hadas) Date: Wed, 7 Mar 2018 16:07:49 +0200 Subject: [ovirt-devel] [ovirt-users] Unremovable disks created through the API In-Reply-To: <20180307133726.GN2787@redhat.com> References: <20180306191824.GA27515@redhat.com> <20180306211941.GI2787@redhat.com> <20180307120112.7347f2d3@fiorina> <20180307133726.GN2787@redhat.com> Message-ID: On Wed, Mar 7, 2018 at 3:37 PM, Richard W.M. Jones wrote: > On Wed, Mar 07, 2018 at 03:12:58PM +0200, Arik Hadas wrote: > > If for some predefined period of time no new disk is added or an upload > > doesn't make any progress (assuming the uploads are done sequentially), > to > > fail the import operation and that would roll back the resources (disks, > > VMs) that were created as part of the import process. > > At the moment we're actually trying to remove the disk on failure. > However the disk_service.remove() API does nothing (doesn't even > fail). Perhaps because the transfer isn't finalized on the error > path? > That's weird, it should fail to acquire the disk's lock. In any case, I think that removing the disk this way would be wrong as we also store an entity in the image_transfer table that should be removed (otherwise, another attempt to upload the same disk would probably fail). Daniel/Nir, I can't find a way to cancel an ongoing upload-image operation through the API (as we have in the webadmin), am I missing it or is it missing? > > Anyway the code - which needs review - is: > > https://www.redhat.com/archives/libguestfs/2018-March/msg00024.html > > Rich. > > -- > Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~ > rjones > Read my programming and virtualization blog: http://rwmj.wordpress.com > Fedora Windows cross-compiler. Compile Windows programs, test, and > build Windows installers. Over 100 libraries supported. > http://fedoraproject.org/wiki/MinGW > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michal.skrivanek at redhat.com Wed Mar 7 21:48:33 2018 From: michal.skrivanek at redhat.com (Michal Skrivanek) Date: Wed, 7 Mar 2018 13:48:33 -0800 Subject: [ovirt-devel] [ovirt-users] Unremovable disks created through the API In-Reply-To: References: <20180306191824.GA27515@redhat.com> <20180306211941.GI2787@redhat.com> <20180307100553.GJ2787@redhat.com> <20180307114118.GL2787@redhat.com> Message-ID: On 07 Mar 2018, at 14:20, Arik Hadas wrote: On Wed, Mar 7, 2018 at 1:41 PM, Richard W.M. Jones wrote: > On Wed, Mar 07, 2018 at 01:26:39PM +0200, Arik Hadas wrote: > > Interesting, that contradicts my intuition - I would imagine that most of > > the things are actually known (the things that appear in the top-level > part > > of the domain xml: memory size, memory size, num of CPUs, name,.. ) and > > only things that depend on the content of the disks or things that depend > > on installations during the conversion are unknown. > > But anyway, it is enough IMO to send the name, memory, CPU and size of > the > > disks to present something useful to the user and make the necessary > > validations at that point. > > Some of those things are known, but they didn't seem to me to be that > interesting for oVirt to know in advance. In any case what's > precisely known before conversion is: > > (1) The 'source' struct and sub-structs: > > https://github.com/libguestfs/libguestfs/blob/ > ba53251ab912b8bac9e00c1022adc6ba9bdf70a3/v2v/types.mli#L59 > > (2) The 'overlay' struct (one per disk): > > https://github.com/libguestfs/libguestfs/blob/ > ba53251ab912b8bac9e00c1022adc6ba9bdf70a3/v2v/types.mli#L175 > > Note only virtual disk size is known, which is near to useless for > provisioning storage. > > (3) The 'target' struct (one per disk): > > https://github.com/libguestfs/libguestfs/blob/ > ba53251ab912b8bac9e00c1022adc6ba9bdf70a3/v2v/types.mli#L191 > > What's unknown are guest capabilities (hence nothing about what > devices should be presented to the guest), inspection data, target bus > mapping, real size of disks, etc. > > I see. I think it is sufficient - the information from the 'source' struct seems enough just to produce a representative VM entity in the database that would be reflected in the UI with status 'importing' and for general validations, For having an entity, yes. For meaningful validations, not really. It's not so interesting to see much in oVirt, at least not for the virt-v2v command line usage and the estimated size on the 'target' struct would be enough for storage validations and optionally for choosing the right target storage domain. The other things are relatively hidden in oVirt's UI and can be added at the last phase. What I do not like that much about is that you change the VM at the end substantially, and for the import duration it's locked anyway The name has a value, and progress, but again, only for a oVirt GUI user - and when you are driving the process from cmdline it's just not that important It's not so difficult to do it like that "externally" - just blindly create a completely stub VM in your caller app(wrapper; another integration), and then replace it. We do not need to push that into v2v directly BTW, that's how import from VMware/Xen currently works - we add a VM entity based on the domain XML we get from vCenter and at the last phase add the missing parts when getting the generated OVF from virt-v2v. So for instance, the VM would have no graphics device until that last phase. > Rich. > > -- > Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~ > rjones > Read my programming and virtualization blog: http://rwmj.wordpress.com > virt-top is 'top' for virtual machines. Tiny program with many > powerful monitoring features, net stats, disk stats, logging, etc. > http://people.redhat.com/~rjones/virt-top > _______________________________________________ Devel mailing list Devel at ovirt.org http://lists.ovirt.org/mailman/listinfo/devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From ahadas at redhat.com Wed Mar 7 22:14:57 2018 From: ahadas at redhat.com (Arik Hadas) Date: Thu, 8 Mar 2018 00:14:57 +0200 Subject: [ovirt-devel] [ovirt-users] Unremovable disks created through the API In-Reply-To: References: <20180306191824.GA27515@redhat.com> <20180306211941.GI2787@redhat.com> <20180307100553.GJ2787@redhat.com> <20180307114118.GL2787@redhat.com> Message-ID: On Wed, Mar 7, 2018 at 11:48 PM, Michal Skrivanek < michal.skrivanek at redhat.com> wrote: > > > On 07 Mar 2018, at 14:20, Arik Hadas wrote: > > > > On Wed, Mar 7, 2018 at 1:41 PM, Richard W.M. Jones > wrote: > >> On Wed, Mar 07, 2018 at 01:26:39PM +0200, Arik Hadas wrote: >> > Interesting, that contradicts my intuition - I would imagine that most >> of >> > the things are actually known (the things that appear in the top-level >> part >> > of the domain xml: memory size, memory size, num of CPUs, name,.. ) and >> > only things that depend on the content of the disks or things that >> depend >> > on installations during the conversion are unknown. >> > But anyway, it is enough IMO to send the name, memory, CPU and size of >> the >> > disks to present something useful to the user and make the necessary >> > validations at that point. >> >> Some of those things are known, but they didn't seem to me to be that >> interesting for oVirt to know in advance. In any case what's >> precisely known before conversion is: >> >> (1) The 'source' struct and sub-structs: >> >> https://github.com/libguestfs/libguestfs/blob/ba53251ab912b8 >> bac9e00c1022adc6ba9bdf70a3/v2v/types.mli#L59 >> >> (2) The 'overlay' struct (one per disk): >> >> https://github.com/libguestfs/libguestfs/blob/ba53251ab912b8 >> bac9e00c1022adc6ba9bdf70a3/v2v/types.mli#L175 >> >> Note only virtual disk size is known, which is near to useless for >> provisioning storage. >> >> (3) The 'target' struct (one per disk): >> >> https://github.com/libguestfs/libguestfs/blob/ba53251ab912b8 >> bac9e00c1022adc6ba9bdf70a3/v2v/types.mli#L191 >> >> What's unknown are guest capabilities (hence nothing about what >> devices should be presented to the guest), inspection data, target bus >> mapping, real size of disks, etc. >> >> > I see. I think it is sufficient - the information from the 'source' struct > seems enough just to produce a representative VM entity in the database > that would be reflected in the UI with status 'importing' and for general > validations, > > > For having an entity, yes. For meaningful validations, not really. > I was thinking mostly about validating the name and the amount of free space in the target storage domain. But in the future, when/if the imported VMs could be from different architecture (PPC?) or different CPU type (q35?) we can also validate that the target cluster is valid. > It's not so interesting to see much in oVirt, at least not for the > virt-v2v command line usage > > Functionality-wise, I agree. But in a serious management application you don't expect to see memory=0 or memory=N/A when you open the general tab of the VM that is being imported. If we can easily get the information from the 'source' struct, I think we should. > and the estimated size on the 'target' struct would be enough for storage > validations and optionally for choosing the right target storage domain. > The other things are relatively hidden in oVirt's UI and can be added at > the last phase. > > > What I do not like that much about is that you change the VM at the end > substantially, and for the import duration it's locked anyway > The name has a value, and progress, but again, only for a oVirt GUI user - > and when you are driving the process from cmdline it's just not that > important > Right, but I tend to think that it's better to generalize the process in a way that the cmdline flow will make an extra step(s) rather than having separate and different flows. > > It's not so difficult to do it like that "externally" - just blindly > create a completely stub VM in your caller app(wrapper; another > integration), and then replace it. We do not need to push that into v2v > directly > > It depends on the amount and quality of validations you want to make at the beginning of the process and whether or not ovirt-engine should instruct v2v where to upload the disks to (maybe there should be an option to upload a disk without specifying the target storage domain so it will be selected automatically). v2v will anyway need to trigger a call to ovirt-engine in order to create that context, that wrapper command. If the data we need at the beginning is already available to v2v at that point, providing it to that call shouldn't be that complex, right? > > BTW, that's how import from VMware/Xen currently works - we add a VM > entity based on the domain XML we get from vCenter and at the last phase > add the missing parts when getting the generated OVF from virt-v2v. So for > instance, the VM would have no graphics device until that last phase. > > > >> Rich. >> >> -- >> Richard Jones, Virtualization Group, Red Hat >> http://people.redhat.com/~rjones >> Read my programming and virtualization blog: http://rwmj.wordpress.com >> virt-top is 'top' for virtual machines. Tiny program with many >> powerful monitoring features, net stats, disk stats, logging, etc. >> http://people.redhat.com/~rjones/virt-top >> > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sabose at redhat.com Thu Mar 8 04:55:04 2018 From: sabose at redhat.com (Sahina Bose) Date: Thu, 8 Mar 2018 10:25:04 +0530 Subject: [ovirt-devel] [OST][HC] Failed to deploy HE Message-ID: Error is - The host has been set in non_operational status, please check engine logs, fix accordingly and re-deploy. However there are no engine logs available. Is this error seen in HE suite as well? On Thu, Mar 8, 2018 at 8:58 AM, wrote: > Project: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic- > suite-master/ > Build: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic- > suite-master/213/ > Build Number: 213 > Build Status: Failure > Triggered By: Started by timer > > ------------------------------------- > Changes Since Last Success: > ------------------------------------- > Changes for Build #213 > [Sandro Bonazzola] image-ng-suite-4.2: sync 001_initialize_engine > > [Barak Korren] Enable mock to use bootstrap chroot for FC27/RAW > > [Sandro Bonazzola] gerrit-admin: move to el7 > > [Daniel Belenky] Add symlinks resolving capabilities to usrc > > [Barak Korren] Fix auto-merge in downstream instances > > [Dafna Ron] sync_mirror: removing fc24 > > > > > ----------------- > Failed Tests: > ----------------- > No tests ran. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ehaas at redhat.com Thu Mar 8 07:51:09 2018 From: ehaas at redhat.com (Edward Haas) Date: Thu, 8 Mar 2018 09:51:09 +0200 Subject: [ovirt-devel] [VDSM] Remove fcraw build as gating Message-ID: Hi All, Fcraw build on VDSM check-patch has been failing recently (again). fcraw cannot act as gating to the VDSM CI, as it is unstable (by definition). If there is a strong interest for some to track VDSM against this unstable distribution, it better be performed in parallel to the VDSM developing flow. With the current state, we are unable to trust the CI and it causes us to either overwrite its output (which I consider very bad) or just get stuck frequently. Could we please move this job to be triggered once a day as a nightly job and whoever is interested in its output to get notifications for it? Please. lets not make it a gating to VDSM development. Thanks, Edy. -------------- next part -------------- An HTML attachment was scrubbed... URL: From eedri at redhat.com Thu Mar 8 08:01:30 2018 From: eedri at redhat.com (Eyal Edri) Date: Thu, 8 Mar 2018 10:01:30 +0200 Subject: [ovirt-devel] [VDSM] Remove fcraw build as gating In-Reply-To: References: Message-ID: On Thu, Mar 8, 2018 at 9:51 AM, Edward Haas wrote: > Hi All, > > Fcraw build on VDSM check-patch has been failing recently (again). > > fcraw cannot act as gating to the VDSM CI, as it is unstable (by > definition). > If there is a strong interest for some to track VDSM against this unstable > distribution, it better be performed in parallel to the VDSM developing > flow. > > With the current state, we are unable to trust the CI and it causes us to > either overwrite its output (which I consider very bad) or just get stuck > frequently. > > Could we please move this job to be triggered once a day as a nightly job > and whoever is interested in its output to get notifications for it? > Please. lets not make it a gating to VDSM development. > > +1 from me, I didn't see any real value from fcraw so far, especially if it doesn't save us adding new fedora versions when they are out. We should try to stabalize fc28 when its out. Also, CI team won't be adding nightly jobs for projects on demand on production system, you're welcome to test it on your local env if needed. > Thanks, > Edy. > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -- Eyal edri MANAGER RHV DevOps EMEA VIRTUALIZATION R&D Red Hat EMEA TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ) -------------- next part -------------- An HTML attachment was scrubbed... URL: From bkorren at redhat.com Thu Mar 8 08:05:09 2018 From: bkorren at redhat.com (Barak Korren) Date: Thu, 8 Mar 2018 10:05:09 +0200 Subject: [ovirt-devel] [ACTION REQUIRED] fixing CI on FC27 and RAWHIDE Message-ID: Hi to all maintainers, TL;DR: If you use yum-utils on fedora (For e.g using yum-builddep) you need to switch to dnf-utils. As you may know we had encountered many issues when trying to get CI to work well for FC27 and Fedora RAWHIDE. One of the main reasons for the issues we've seen was that we were using 'yum' instead of 'dnf' to setup Fedora build and test environments. In FC27 and RAWHIDe, some packages simply stopped being compatible with yum. It took us a while to resolve this since we needed to get some patches merged upstream to the tools we use, but since yesterday we switched to using DNF for setting up FC environments. While we were waiting for the tools to get patched, our work-around was to freeze the mirrors our CI system uses so that newer and incompatible packages were invisible in CI. Since we now have a fix in place we started re-syncing our mirrors to get newer packages. A side effect of this, is that dnf-incompatible code in build and test scripts was exposed. One such common case is scripts using 'yum-builddep' with 'yum-utils' installed instead of 'dnf-utils'. In this case yum-incompatible packages will fail to be downloaded. Bottom line - if you use yum-based tools in your build/test scripts, they need to be changed to dnf-based tools when those scripts run in Fedora. Sometime this can be as simple as changing the packages listed in your *.packages files for Fedora. See for example this patch which switches from yum-utils to dnf-utils: https://gerrit.ovirt.org/c/88630/ -- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted From msivak at redhat.com Thu Mar 8 10:11:04 2018 From: msivak at redhat.com (Martin Sivak) Date: Thu, 8 Mar 2018 11:11:04 +0100 Subject: [ovirt-devel] [OST][HC] Failed to deploy HE In-Reply-To: References: Message-ID: Hi Sahina, you need to synchronize couple of fixes to make everything work properly, most importantly - hosted engine setup https://gerrit.ovirt.org/#/c/88311/ - vdsm https://gerrit.ovirt.org/#/c/88370/ Those were all merged already (yesterday in fact), but it takes time to propagate them to the OST repos I guess. Best regards Martin Sivak On Thu, Mar 8, 2018 at 5:55 AM, Sahina Bose wrote: > Error is - The host has been set in non_operational status, please check > engine logs, fix accordingly and re-deploy. > > However there are no engine logs available. Is this error seen in HE suite > as well? > > On Thu, Mar 8, 2018 at 8:58 AM, wrote: >> >> Project: >> http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-master/ >> Build: >> http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-master/213/ >> Build Number: 213 >> Build Status: Failure >> Triggered By: Started by timer >> >> ------------------------------------- >> Changes Since Last Success: >> ------------------------------------- >> Changes for Build #213 >> [Sandro Bonazzola] image-ng-suite-4.2: sync 001_initialize_engine >> >> [Barak Korren] Enable mock to use bootstrap chroot for FC27/RAW >> >> [Sandro Bonazzola] gerrit-admin: move to el7 >> >> [Daniel Belenky] Add symlinks resolving capabilities to usrc >> >> [Barak Korren] Fix auto-merge in downstream instances >> >> [Dafna Ron] sync_mirror: removing fc24 >> >> >> >> >> ----------------- >> Failed Tests: >> ----------------- >> No tests ran. > > > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel From dron at redhat.com Thu Mar 8 11:53:48 2018 From: dron at redhat.com (Dafna Ron) Date: Thu, 8 Mar 2018 11:53:48 +0000 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 (ovirt-hosted-engine-setup) ] [ 08-03-2018 ] [ 004_basic_sanity.run_vms ] Message-ID: Hi, We have a failed test on ovirt-hosted-engine-setup basic suite. We failed to run a vm with internal engine error. *Link and headline of suspected patches: Add engine fqdn to inventory and allow dynamic inventory scripts - https://gerrit.ovirt.org/#/c/88622/ Link to Job:http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1064/ Link to all logs:http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1064/artifact/ (Relevant) error snippet from the log: 2018-03-07 17:03:38,374-05 INFO [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-21) [a514da44-d44f-46b9-8bdf-08ba2d513929] Running command: RunVmOnceCommand internal: false. Entities affected : ID: 634c6a46-d057-4509-be3b-710716cbd56d Type: VMAction group RUN_VM with role type USER, ID: 634c6a46-d057-4509-be3b-710716cbd56d Type: VMAction group EDIT_ADMIN_VM_PROPERTIES with role type ADMIN2018-03-07 17:03:38,379-05 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-21) [a514da44-d44f-46b9-8bdf-08ba2d513929] method: getVmManager, params: [634c6a46-d057-4509-be3b-710716cbd56d], timeElapsed: 5ms2018-03-07 17:03:38,391-05 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-21) [a514da44-d44f-46b9-8bdf-08ba2d513929] method: getAllForClusterWithStatus, params: [14cad400-49a0-44e0-ab15-9da778f08082, Up], timeElapsed: 7ms2018-03-07 17:03:38,408-05 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-21) [a514da44-d44f-46b9-8bdf-08ba2d513929] method: getVdsManager, params: [8ef2f490-1b76-46e0-b9fe-f3412a36e03b], timeElapsed: 0ms2018-03-07 17:03:38,419-05 DEBUG [org.ovirt.engine.core.dal.dbbroker.CustomSQLErrorCodeSQLExceptionTranslator] (default task-21) [a514da44-d44f-46b9-8bdf-08ba2d513929] Translating SQLException with SQL state '23505', error code '0', message [ERROR: duplicate key value violates unique constraint "name_server_pkey" Detail: Key (dns_resolver_configuration_id, address)=(8a941b6a-83aa-44de-9800-2a1ea6e8e029, 192.168.200.1) already exists. Where: SQL statement "INSERT INTO name_server( address, position, dns_resolver_configuration_id) VALUES ( v_address, v_position, v_dns_resolver_configuration_id)"PL/pgSQL function insertnameserver(uuid,character varying,smallint) line 3 at SQL statement]; SQL was [{call insertnameserver(?, ?, ?)}] for task [CallableStatementCallback]2018-03-07 17:03:38,420-05 ERROR [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-21) [a514da44-d44f-46b9-8bdf-08ba2d513929] Command 'org.ovirt.engine.core.bll.RunVmOnceCommand' failed: CallableStatementCallback; SQL [{call insertnameserver(?, ?, ?)}]; ERROR: duplicate key value violates unique constraint "name_server_pkey" Detail: Key (dns_resolver_configuration_id, address)=(8a941b6a-83aa-44de-9800-2a1ea6e8e029, 192.168.200.1) already exists. Where: SQL statement "INSERT INTO name_server( address, position, dns_resolver_configuration_id) VALUES ( v_address, v_position, v_dns_resolver_configuration_id)"PL/pgSQL function insertnameserver(uuid,character varying,smallint) line 3 at SQL statement; nested exception is org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "name_server_pkey" Detail: Key (dns_resolver_configuration_id, address)=(8a941b6a-83aa-44de-9800-2a1ea6e8e029, 192.168.200.1) already exists. Where: SQL statement "INSERT INTO name_server( address, position, dns_resolver_configuration_id) VALUES ( v_address, v_position, v_dns_resolver_configuration_id)"PL/pgSQL function insertnameserver(uuid,character varying,smallint) line 3 at SQL statement2018-03-07 17:03:38,420-05 ERROR [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-21) [a514da44-d44f-46b9-8bdf-08ba2d513929] Exception: org.springframework.dao.DuplicateKeyException: CallableStatementCallback; SQL [{call insertnameserver(?, ?, ?)}]; ERROR: duplicate key value violates unique constraint "name_server_pkey" Detail: Key (dns_resolver_configuration_id, address)=(8a941b6a-83aa-44de-9800-2a1ea6e8e029, 192.168.200.1) already exists. Where: SQL statement "INSERT INTO name_server( address, position, dns_resolver_configuration_id) VALUES (: v_address, v_position, v_dns_resolver_configuration_id)"PL/pgSQL function insertnameserver(uuid,character varying,smallint) line 3 at SQL statement; nested exception is org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "name_server_pkey" Detail: Key (dns_resolver_configuration_id, address)=(8a941b6a-83aa-44de-9800-2a1ea6e8e029, 192.168.200.1) already exists. Where: SQL statement "INSERT INTO name_server( address, position, dns_resolver_configuration_id) VALUES ( v_address, v_position, v_dns_resolver_configuration_id)"PL/pgSQL function insertnameserver(uuid,character varying,smallint) line 3 at SQL statement at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:239) [spring-jdbc.jar:4.3.9.RELEASE] at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:73) [spring-jdbc.jar:4.3.9.RELEASE] at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1099) [spring-jdbc.jar:4.3.9.RELEASE] at org.springframework.jdbc.core.JdbcTemplate.call(JdbcTemplate.java:1135) [spring-jdbc.jar:4.3.9.RELEASE] at org.springframework.jdbc.core.simple.AbstractJdbcCall.executeCallInternal(AbstractJdbcCall.java:405) [spring-jdbc.jar:4.3.9.RELEASE] at org.springframework.jdbc.core.simple.AbstractJdbcCall.doExecute(AbstractJdbcCall.java:365) [spring-jdbc.jar:4.3.9.RELEASE] at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:198) [spring-jdbc.jar:4.3.9.RELEASE] at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:135) [dal.jar:] at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:130) [dal.jar:] at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeModification(SimpleJdbcCallsHandler.java:76) [dal.jar:] at org.ovirt.engine.core.dao.network.DnsResolverConfigurationDaoImpl.saveNameServersByDnsResolverConfigurationId(DnsResolverConfigurationDaoImpl.java:66) [dal.jar:] at org.ovirt.engine.core.dao.network.DnsResolverConfigurationDaoImpl.update(DnsResolverConfigurationDaoImpl.java:110) [dal.jar:] at org.ovirt.engine.core.dao.network.DnsResolverConfigurationDaoImpl.update(DnsResolverConfigurationDaoImpl.java:15) [dal.jar:] at org.ovirt.engine.core.dao.VdsDynamicDaoImpl.updateDnsResolverConfiguration(VdsDynamicDaoImpl.java:156) [dal.jar:] at org.ovirt.engine.core.dao.VdsDynamicDaoImpl.update(VdsDynamicDaoImpl.java:145) [dal.jar:] at org.ovirt.engine.core.dao.VdsDynamicDaoImpl.updateIfNeeded(VdsDynamicDaoImpl.java:343) [dal.jar:] at org.ovirt.engine.core.dao.VdsDynamicDaoImpl.updateIfNeeded(VdsDynamicDaoImpl.java:35) [dal.jar:] at org.ovirt.engine.core.vdsbroker.VdsManager.updateDynamicData(VdsManager.java:473) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.VdsManager.updatePendingData(VdsManager.java:511) [vdsbroker.jar:] at org.ovirt.engine.core.bll.scheduling.pending.PendingResourceManager.notifyHostManagers(PendingResourceManager.java:227) [bll.jar:] at org.ovirt.engine.core.bll.scheduling.SchedulingManager.schedule(SchedulingManager.java:361) [bll.jar:] at org.ovirt.engine.core.bll.RunVmCommand.getVdsToRunOn(RunVmCommand.java:853) [bll.jar:] at org.ovirt.engine.core.bll.RunVmCommand.runVm(RunVmCommand.java:256) [bll.jar:] at org.ovirt.engine.core.bll.RunVmCommand.perform(RunVmCommand.java:430) [bll.jar:] at org.ovirt.engine.core.bll.RunVmCommand.executeVmCommand(RunVmCommand.java:355) [bll.jar:] at org.ovirt.engine.core.bll.VmCommand.executeCommand(VmCommand.java:161) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1133) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1285) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1934) [bll.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:164) [utils.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:103) [utils.jar:] at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1345) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:400) [bll.jar:] at org.ovirt.engine.core.bll.executor.DefaultBackendActionExecutor.execute(DefaultBackendActionExecutor.java:13) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:468) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:450) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:403) [bll.jar:] at sun.reflect.GeneratedMethodAccessor134.invoke(Unknown Source) [:1.8.0_161] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_161] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_161] at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:92) [wildfly-weld-ejb-11.0.0.Final.jar:11.0.0.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContext.interceptorChainCompleted(WeldInvocationContext.java:98) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContext.proceed(WeldInvocationContext.java:117) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12) [common.jar:] at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source) [:1.8.0_161] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_161] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_161] at org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContext.invokeNext(WeldInvocationContext.java:83) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContext.proceed(WeldInvocationContext.java:115) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.weld.bean.InterceptorImpl.intercept(InterceptorImpl.java:108) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:82) [wildfly-weld-ejb-11.0.0.Final.jar:11.0.0.Final] at org.jboss.as.weld.interceptors.EjbComponentInterceptorSupport.delegateInterception(EjbComponentInterceptorSupport.java:60) at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:76) at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:88) at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:101) at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at org.ovirt.engine.core.bll.interceptors.CorrelationIdTrackerInterceptor.aroundInvoke(CorrelationIdTrackerInterceptor.java:13) [bll.jar:] at sun.reflect.GeneratedMethodAccessor66.invoke(Unknown Source) [:1.8.0_161] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_161] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_161] at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:89) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45) [wildfly-ee-11.0.0.Final.jar:11.0.0.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:40) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53) at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:264) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:379) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:244) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:609) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:81) at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view2.runAction(Unknown Source) [common.jar:] at org.ovirt.engine.api.restapi.resource.BackendResource.doAction(BackendResource.java:250) at org.ovirt.engine.api.restapi.resource.AbstractBackendActionableResource.doAction(AbstractBackendActionableResource.java:84) at org.ovirt.engine.api.restapi.resource.AbstractBackendActionableResource.doAction(AbstractBackendActionableResource.java:125) at org.ovirt.engine.api.restapi.resource.BackendVmResource.start(BackendVmResource.java:450) at org.ovirt.engine.api.v3.V3Server.adaptAction(V3Server.java:217) at org.ovirt.engine.api.v3.servers.V3VmServer.start(V3VmServer.java:216) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_161] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_161] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_161] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_161] at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:140) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] at org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:138) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] at org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:107) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] at org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:133) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] at org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:101) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:406) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:213) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:228) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final] at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:81) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:274) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchToPath(ServletInitialHandler.java:209) at io.undertow.servlet.spec.RequestDispatcherImpl.forwardImpl(RequestDispatcherImpl.java:221) at io.undertow.servlet.spec.RequestDispatcherImpl.forwardImplSetup(RequestDispatcherImpl.java:147) at io.undertow.servlet.spec.RequestDispatcherImpl.forward(RequestDispatcherImpl.java:111) at org.ovirt.engine.api.restapi.invocation.VersionFilter.doFilter(VersionFilter.java:180) at org.ovirt.engine.api.restapi.invocation.VersionFilter.doFilter(VersionFilter.java:98) at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.api.restapi.invocation.CurrentFilter.doFilter(CurrentFilter.java:117) at org.ovirt.engine.api.restapi.invocation.CurrentFilter.doFilter(CurrentFilter.java:72) at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.aaa.filters.RestApiSessionMgmtFilter.doFilter(RestApiSessionMgmtFilter.java:78) [aaa.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.aaa.filters.EnforceAuthFilter.doFilter(EnforceAuthFilter.java:42) [aaa.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter.doFilter(SsoRestApiNegotiationFilter.java:84) [aaa.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter.doFilter(SsoRestApiAuthFilter.java:47) [aaa.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:59) [aaa.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.aaa.filters.RestApiSessionValidationFilter.doFilter(RestApiSessionValidationFilter.java:35) [aaa.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.api.restapi.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:111) at org.ovirt.engine.api.restapi.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:102) at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.utils.servlet.CORSSupportFilter.doFilter(CORSSupportFilter.java:284) [utils.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:326) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:812) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_161] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_161] at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_161]Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "name_server_pkey" Detail: Key (dns_resolver_configuration_id, address)=(8a941b6a-83aa-44de-9800-2a1ea6e8e029, 192.168.200.1) already exists. Where: SQL statement "INSERT INTO name_server( address, position, dns_resolver_configuration_id) VALUES ( v_address, v_position, v_dns_resolver_configuration_id)"PL/pgSQL function insertnameserver(uuid,character varying,smallint) line 3 at SQL statement at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2157) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1886) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:555) at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:417) at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:410) at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.execute(CachedPreparedStatement.java:303) at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.execute(WrappedPreparedStatement.java:442) at org.springframework.jdbc.core.JdbcTemplate$6.doInCallableStatement(JdbcTemplate.java:1138) [spring-jdbc.jar:4.3.9.RELEASE] at org.springframework.jdbc.core.JdbcTemplate$6.doInCallableStatement(JdbcTemplate.java:1135) [spring-jdbc.jar:4.3.9.RELEASE] at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1083) [spring-jdbc.jar:4.3.9.RELEASE] ... 215 more2018-03-07 17:03:38,453-05 DEBUG [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Executing batch for procedure Updatevds_interface_statistics2018-03-07 17:03:38,453-05 DEBUG [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Mapped params: [v_iface_status, v_sample_time, v_rx_offset, v_rx_rate, v_tx_rate, v_tx_drop, v_tx_offset, v_id, v_rx_drop, v_rx_total, v_tx_total, v_vds_id]2018-03-07 17:03:38,453-05 DEBUG [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Mapped params: [v_iface_status, v_sample_time, v_rx_offset, v_rx_rate, v_tx_rate, v_tx_drop, v_tx_offset, v_id, v_rx_drop, v_rx_total, v_tx_total, v_vds_id]2018-03-07 17:03:38,453-05 DEBUG [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Mapped params: [v_iface_status, v_sample_time, v_rx_offset, v_rx_rate, v_tx_rate, v_tx_drop, v_tx_offset, v_id, v_rx_drop, v_rx_total, v_tx_total, v_vds_id]2018-03-07 17:03:38,453-05 DEBUG [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Mapped params: [v_iface_status, v_sample_time, v_rx_offset, v_rx_rate, v_tx_rate, v_tx_drop, v_tx_offset, v_id, v_rx_drop, v_rx_total, v_tx_total, v_vds_id]2018-03-07 17:03:38,453-05 DEBUG [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Mapped params: [v_iface_status, v_sample_time, v_rx_offset, v_rx_rate, v_tx_rate, v_tx_drop, v_tx_offset, v_id, v_rx_drop, v_rx_total, v_tx_total, v_vds_id]2018-03-07 17:03:38,456-05 DEBUG [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Executed batch2018-03-07 17:03:38,457-05 DEBUG [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Executing batch for procedure UpdateNumaNodeStatistics2018-03-07 17:03:38,457-05 DEBUG [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Mapped params: [v_numa_node_id, v_mem_free, v_usage_mem_percent, v_cpu_sys, v_cpu_user, v_cpu_idle, v_usage_cpu_percent]2018-03-07 17:03:38,459-05 DEBUG [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Executed batch2018-03-07 17:03:38,462-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-21) [a514da44-d44f-46b9-8bdf-08ba2d513929] EVENT_ID: USER_FAILED_RUN_VM(54), Failed to run VM backup_vm (User: admin at in:2018-03-07 17:03:38,466-05 DEBUG [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (EE-ManagedThreadFactory-engine-Thread-182) [] Executing task: EE-ManagedThreadFactory-engine-Thread-1822018-03-07 17:03:38,467-05 INFO [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-21) [a514da44-d44f-46b9-8bdf-08ba2d513929] Lock freed to object 'EngineLock:{exclusiveLocks='[634c6a46-d057-4509-be3b-710716cbd56d=VM]', sharedLocks=''}'2018-03-07 17:03:38,467-05 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-21) [a514da44-d44f-46b9-8bdf-08ba2d513929] method: runAction, params: [RunVmOnce, RunVmOnceParams:{commandId='d5358409-e005-4c8b-a702-d4fa394ee9d2', user='null', commandType='Unknown', vmId='634c6a46-d057-4509-be3b-710716cbd56d'}], timeElapsed: 201ms2018-03-07 17:03:38,473-05 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-21) [] Operation Failed: [Internal Engine Error]* -------------- next part -------------- An HTML attachment was scrubbed... URL: From msivak at redhat.com Thu Mar 8 12:05:20 2018 From: msivak at redhat.com (Martin Sivak) Date: Thu, 8 Mar 2018 13:05:20 +0100 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 (ovirt-hosted-engine-setup) ] [ 08-03-2018 ] [ 004_basic_sanity.run_vms ] In-Reply-To: References: Message-ID: Does not seem to be hosted engine related.. there is no way we could cause internal engine error by using the API alone. Martin On Thu, Mar 8, 2018 at 12:53 PM, Dafna Ron wrote: > Hi, > > We have a failed test on ovirt-hosted-engine-setup basic suite. > We failed to run a vm with internal engine error. > > Link and headline of suspected patches: > > Add engine fqdn to inventory and allow dynamic inventory scripts - > https://gerrit.ovirt.org/#/c/88622/ > > Link to Job: > > http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1064/ > > Link to all logs: > > http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1064/artifact/ > > (Relevant) error snippet from the log: > > > > > 2018-03-07 17:03:38,374-05 INFO > [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-21) > [a514da44-d44f-46b9-8bdf-08ba2d513929] Running command: RunVmOnceCommand > internal: false. Entities affected : ID: 634c6a46-d057-4509-be3b-710 > 716cbd56d Type: VMAction group RUN_VM with role type USER, ID: > 634c6a46-d057-4509-be3b-710716cbd56d Type: VMAction group > EDIT_ADMIN_VM_PROPERTIES with role type ADMIN > 2018-03-07 17:03:38,379-05 DEBUG > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > (default task-21) [a514da44-d44f-46b9-8bdf-08ba2d513929] method: > getVmManager, params: [634c6a46-d057-4509-be3b-710716cbd56d], timeElap > sed: 5ms > 2018-03-07 17:03:38,391-05 DEBUG > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > (default task-21) [a514da44-d44f-46b9-8bdf-08ba2d513929] method: > getAllForClusterWithStatus, params: [14cad400-49a0-44e0-ab15-9da778f08 > 082, Up], timeElapsed: 7ms > 2018-03-07 17:03:38,408-05 DEBUG > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > (default task-21) [a514da44-d44f-46b9-8bdf-08ba2d513929] method: > getVdsManager, params: [8ef2f490-1b76-46e0-b9fe-f3412a36e03b], timeEla > psed: 0ms > 2018-03-07 17:03:38,419-05 DEBUG > [org.ovirt.engine.core.dal.dbbroker.CustomSQLErrorCodeSQLExceptionTranslator] > (default task-21) [a514da44-d44f-46b9-8bdf-08ba2d513929] Translating > SQLException with SQL state '23505', error code '0', messa > ge [ERROR: duplicate key value violates unique constraint "name_server_pkey" > Detail: Key (dns_resolver_configuration_id, > address)=(8a941b6a-83aa-44de-9800-2a1ea6e8e029, 192.168.200.1) already > exists. > Where: SQL statement "INSERT INTO > name_server( > address, > position, > dns_resolver_configuration_id) > VALUES ( > v_address, > v_position, > v_dns_resolver_configuration_id)" > PL/pgSQL function insertnameserver(uuid,character varying,smallint) line 3 > at SQL statement]; SQL was [{call insertnameserver(?, ?, ?)}] for task > [CallableStatementCallback] > 2018-03-07 17:03:38,420-05 ERROR > [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-21) > [a514da44-d44f-46b9-8bdf-08ba2d513929] Command > 'org.ovirt.engine.core.bll.RunVmOnceCommand' failed: > CallableStatementCallback; SQL [{call inse > rtnameserver(?, ?, ?)}]; ERROR: duplicate key value violates unique > constraint "name_server_pkey" > Detail: Key (dns_resolver_configuration_id, > address)=(8a941b6a-83aa-44de-9800-2a1ea6e8e029, 192.168.200.1) already > exists. > Where: SQL statement "INSERT INTO > name_server( > address, > position, > dns_resolver_configuration_id) > VALUES ( > v_address, > v_position, > v_dns_resolver_configuration_id)" > PL/pgSQL function insertnameserver(uuid,character varying,smallint) line 3 > at SQL statement; nested exception is org.postgresql.util.PSQLException: > ERROR: duplicate key value violates unique constraint "name_server_pkey" > Detail: Key (dns_resolver_configuration_id, > address)=(8a941b6a-83aa-44de-9800-2a1ea6e8e029, 192.168.200.1) already > exists. > Where: SQL statement "INSERT INTO > name_server( > address, > position, > dns_resolver_configuration_id) > VALUES ( > v_address, > v_position, > v_dns_resolver_configuration_id)" > PL/pgSQL function insertnameserver(uuid,character varying,smallint) line 3 > at SQL statement > 2018-03-07 17:03:38,420-05 ERROR > [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-21) > [a514da44-d44f-46b9-8bdf-08ba2d513929] Exception: > org.springframework.dao.DuplicateKeyException: CallableStatementCallback; > SQL [{call insertn > ameserver(?, ?, ?)}]; ERROR: duplicate key value violates unique constraint > "name_server_pkey" > Detail: Key (dns_resolver_configuration_id, > address)=(8a941b6a-83aa-44de-9800-2a1ea6e8e029, 192.168.200.1) already > exists. > Where: SQL statement "INSERT INTO > name_server( > address, > position, > dns_resolver_configuration_id) > VALUES ( > : > v_address, > v_position, > v_dns_resolver_configuration_id)" > PL/pgSQL function insertnameserver(uuid,character varying,smallint) line 3 > at SQL statement; nested exception is org.postgresql.util.PSQLException: > ERROR: duplicate key value violates unique constraint "name_server_pkey" > Detail: Key (dns_resolver_configuration_id, > address)=(8a941b6a-83aa-44de-9800-2a1ea6e8e029, 192.168.200.1) already > exists. > Where: SQL statement "INSERT INTO > name_server( > address, > position, > dns_resolver_configuration_id) > VALUES ( > v_address, > v_position, > v_dns_resolver_configuration_id)" > PL/pgSQL function insertnameserver(uuid,character varying,smallint) line 3 > at SQL statement > at > org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:239) > [spring-jdbc.jar:4.3.9.RELEASE] > at > org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:73) > [spring-jdbc.jar:4.3.9.RELEASE] > at > org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1099) > [spring-jdbc.jar:4.3.9.RELEASE] > at > org.springframework.jdbc.core.JdbcTemplate.call(JdbcTemplate.java:1135) > [spring-jdbc.jar:4.3.9.RELEASE] > at > org.springframework.jdbc.core.simple.AbstractJdbcCall.executeCallInternal(AbstractJdbcCall.java:405) > [spring-jdbc.jar:4.3.9.RELEASE] > at > org.springframework.jdbc.core.simple.AbstractJdbcCall.doExecute(AbstractJdbcCall.java:365) > [spring-jdbc.jar:4.3.9.RELEASE] > at > org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:198) > [spring-jdbc.jar:4.3.9.RELEASE] > at > org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:135) > [dal.jar:] > at > org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:130) > [dal.jar:] > at > org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeModification(SimpleJdbcCallsHandler.java:76) > [dal.jar:] > at > org.ovirt.engine.core.dao.network.DnsResolverConfigurationDaoImpl.saveNameServersByDnsResolverConfigurationId(DnsResolverConfigurationDaoImpl.java:66) > [dal.jar:] > at > org.ovirt.engine.core.dao.network.DnsResolverConfigurationDaoImpl.update(DnsResolverConfigurationDaoImpl.java:110) > [dal.jar:] > at > org.ovirt.engine.core.dao.network.DnsResolverConfigurationDaoImpl.update(DnsResolverConfigurationDaoImpl.java:15) > [dal.jar:] > at > org.ovirt.engine.core.dao.VdsDynamicDaoImpl.updateDnsResolverConfiguration(VdsDynamicDaoImpl.java:156) > [dal.jar:] > at > org.ovirt.engine.core.dao.VdsDynamicDaoImpl.update(VdsDynamicDaoImpl.java:145) > [dal.jar:] > at > org.ovirt.engine.core.dao.VdsDynamicDaoImpl.updateIfNeeded(VdsDynamicDaoImpl.java:343) > [dal.jar:] > at > org.ovirt.engine.core.dao.VdsDynamicDaoImpl.updateIfNeeded(VdsDynamicDaoImpl.java:35) > [dal.jar:] > at > org.ovirt.engine.core.vdsbroker.VdsManager.updateDynamicData(VdsManager.java:473) > [vdsbroker.jar:] > at > org.ovirt.engine.core.vdsbroker.VdsManager.updatePendingData(VdsManager.java:511) > [vdsbroker.jar:] > at > org.ovirt.engine.core.bll.scheduling.pending.PendingResourceManager.notifyHostManagers(PendingResourceManager.java:227) > [bll.jar:] > at > org.ovirt.engine.core.bll.scheduling.SchedulingManager.schedule(SchedulingManager.java:361) > [bll.jar:] > at > org.ovirt.engine.core.bll.RunVmCommand.getVdsToRunOn(RunVmCommand.java:853) > [bll.jar:] > at > org.ovirt.engine.core.bll.RunVmCommand.runVm(RunVmCommand.java:256) > [bll.jar:] > at > org.ovirt.engine.core.bll.RunVmCommand.perform(RunVmCommand.java:430) > [bll.jar:] > at > org.ovirt.engine.core.bll.RunVmCommand.executeVmCommand(RunVmCommand.java:355) > [bll.jar:] > at > org.ovirt.engine.core.bll.VmCommand.executeCommand(VmCommand.java:161) > [bll.jar:] > at > org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1133) > [bll.jar:] > at > org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1285) > [bll.jar:] > at > org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1934) > [bll.jar:] > at > org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:164) > [utils.jar:] > at > org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:103) > [utils.jar:] > at > org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1345) > [bll.jar:] > at > org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:400) > [bll.jar:] > at > org.ovirt.engine.core.bll.executor.DefaultBackendActionExecutor.execute(DefaultBackendActionExecutor.java:13) > [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:468) > [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:450) > [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:403) > [bll.jar:] > at sun.reflect.GeneratedMethodAccessor134.invoke(Unknown Source) > [:1.8.0_161] > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > [rt.jar:1.8.0_161] > at java.lang.reflect.Method.invoke(Method.java:498) > [rt.jar:1.8.0_161] > at > org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) > at > org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:92) > [wildfly-weld-ejb-11.0.0.Final.jar:11.0.0.Final] > at > org.jboss.weld.interceptor.proxy.WeldInvocationContext.interceptorChainCompleted(WeldInvocationContext.java:98) > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > at > org.jboss.weld.interceptor.proxy.WeldInvocationContext.proceed(WeldInvocationContext.java:117) > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > at > org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12) > [common.jar:] > at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source) > [:1.8.0_161] > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > [rt.jar:1.8.0_161] > at java.lang.reflect.Method.invoke(Method.java:498) > [rt.jar:1.8.0_161] > at > org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73) > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > at > org.jboss.weld.interceptor.proxy.WeldInvocationContext.invokeNext(WeldInvocationContext.java:83) > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > at > org.jboss.weld.interceptor.proxy.WeldInvocationContext.proceed(WeldInvocationContext.java:115) > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > at > org.jboss.weld.bean.InterceptorImpl.intercept(InterceptorImpl.java:108) > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > at > org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:82) > [wildfly-weld-ejb-11.0.0.Final.jar:11.0.0.Final] > at > org.jboss.as.weld.interceptors.EjbComponentInterceptorSupport.delegateInterception(EjbComponentInterceptorSupport.java:60) > at > org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:76) > at > org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:88) > at > org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:101) > at > org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) > at > org.ovirt.engine.core.bll.interceptors.CorrelationIdTrackerInterceptor.aroundInvoke(CorrelationIdTrackerInterceptor.java:13) > [bll.jar:] > at sun.reflect.GeneratedMethodAccessor66.invoke(Unknown Source) > [:1.8.0_161] > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > [rt.jar:1.8.0_161] > at java.lang.reflect.Method.invoke(Method.java:498) > [rt.jar:1.8.0_161] > at > org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:89) > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) > [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45) > [wildfly-ee-11.0.0.Final.jar:11.0.0.Final] > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:40) > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53) > at > org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52) > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) > [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:264) > [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] > at > org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:379) > [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] > at > org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:244) > [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) > at > org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73) > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > at > org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89) > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) > [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) > [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) > [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) > [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) > [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) > [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60) > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438) > at > org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:609) > at > org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57) > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53) > at > org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) > at > org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) > at > org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:81) > at > org.ovirt.engine.core.common.interfaces.BackendLocal$$$view2.runAction(Unknown > Source) [common.jar:] > at > org.ovirt.engine.api.restapi.resource.BackendResource.doAction(BackendResource.java:250) > at > org.ovirt.engine.api.restapi.resource.AbstractBackendActionableResource.doAction(AbstractBackendActionableResource.java:84) > at > org.ovirt.engine.api.restapi.resource.AbstractBackendActionableResource.doAction(AbstractBackendActionableResource.java:125) > at > org.ovirt.engine.api.restapi.resource.BackendVmResource.start(BackendVmResource.java:450) > at org.ovirt.engine.api.v3.V3Server.adaptAction(V3Server.java:217) > at > org.ovirt.engine.api.v3.servers.V3VmServer.start(V3VmServer.java:216) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > [rt.jar:1.8.0_161] > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > [rt.jar:1.8.0_161] > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > [rt.jar:1.8.0_161] > at java.lang.reflect.Method.invoke(Method.java:498) > [rt.jar:1.8.0_161] > at > org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:140) > [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] > at > org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) > [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] > at > org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) > [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] > at > org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:138) > [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] > at > org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:107) > [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] > at > org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:133) > [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] > at > org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:101) > [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] > at > org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:406) > [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] > at > org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:213) > [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] > at > org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:228) > [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] > at > org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) > [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] > at > org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) > [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) > [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final] > at > io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) > at > io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:81) > at > io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) > at > io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) > at > io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) > at > io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) > at > io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) > at > io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:274) > at > io.undertow.servlet.handlers.ServletInitialHandler.dispatchToPath(ServletInitialHandler.java:209) > at > io.undertow.servlet.spec.RequestDispatcherImpl.forwardImpl(RequestDispatcherImpl.java:221) > at > io.undertow.servlet.spec.RequestDispatcherImpl.forwardImplSetup(RequestDispatcherImpl.java:147) > at > io.undertow.servlet.spec.RequestDispatcherImpl.forward(RequestDispatcherImpl.java:111) > at > org.ovirt.engine.api.restapi.invocation.VersionFilter.doFilter(VersionFilter.java:180) > at > org.ovirt.engine.api.restapi.invocation.VersionFilter.doFilter(VersionFilter.java:98) > at > io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) > at > io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) > at > org.ovirt.engine.api.restapi.invocation.CurrentFilter.doFilter(CurrentFilter.java:117) > at > org.ovirt.engine.api.restapi.invocation.CurrentFilter.doFilter(CurrentFilter.java:72) > at > io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) > at > io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) > at > org.ovirt.engine.core.aaa.filters.RestApiSessionMgmtFilter.doFilter(RestApiSessionMgmtFilter.java:78) > [aaa.jar:] > at > io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) > at > io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) > at > org.ovirt.engine.core.aaa.filters.EnforceAuthFilter.doFilter(EnforceAuthFilter.java:42) > [aaa.jar:] > at > io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) > at > io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) > at > org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter.doFilter(SsoRestApiNegotiationFilter.java:84) > [aaa.jar:] > at > io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) > at > io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) > at > org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter.doFilter(SsoRestApiAuthFilter.java:47) > [aaa.jar:] > at > io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) > at > io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) > at > org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:59) > [aaa.jar:] > at > io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) > at > io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) > at > org.ovirt.engine.core.aaa.filters.RestApiSessionValidationFilter.doFilter(RestApiSessionValidationFilter.java:35) > [aaa.jar:] > at > io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) > at > io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) > at > org.ovirt.engine.api.restapi.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:111) > at > org.ovirt.engine.api.restapi.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:102) > at > io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) > at > io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) > at > org.ovirt.engine.core.utils.servlet.CORSSupportFilter.doFilter(CORSSupportFilter.java:284) > [utils.jar:] > at > io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) > at > io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) > at > io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84) > at > io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) > at > io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) > at > org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) > at > io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) > at > io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) > at > io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) > at > io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) > at > io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) > at > io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) > at > io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) > at > io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) > at > io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) > at > io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) > at > io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) > at > io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) > at > io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) > at > org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) > at > io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) > at > io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) > at > io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) > at > io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) > at > io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) > at > io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) > at > io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) > at > io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) > at > org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) > at > org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) > at > org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) > at > org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) > at > io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) > at > io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) > at > io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) > at > io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) > at > io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) > at > io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) > at > org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) > at > org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) > at > org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) > at > org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) > at > io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) > at > io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) > at > io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) > at > io.undertow.server.Connectors.executeRootHandler(Connectors.java:326) > at > io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:812) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > [rt.jar:1.8.0_161] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [rt.jar:1.8.0_161] > at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_161] > Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value > violates unique constraint "name_server_pkey" > Detail: Key (dns_resolver_configuration_id, > address)=(8a941b6a-83aa-44de-9800-2a1ea6e8e029, 192.168.200.1) already > exists. > Where: SQL statement "INSERT INTO > name_server( > address, > position, > dns_resolver_configuration_id) > VALUES ( > v_address, > v_position, > v_dns_resolver_configuration_id)" > PL/pgSQL function insertnameserver(uuid,character varying,smallint) line 3 > at SQL statement > at > org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2157) > at > org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1886) > at > org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at > org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:555) > at > org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:417) > at > org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:410) > at > org.jboss.jca.adapters.jdbc.CachedPreparedStatement.execute(CachedPreparedStatement.java:303) > at > org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.execute(WrappedPreparedStatement.java:442) > at > org.springframework.jdbc.core.JdbcTemplate$6.doInCallableStatement(JdbcTemplate.java:1138) > [spring-jdbc.jar:4.3.9.RELEASE] > at > org.springframework.jdbc.core.JdbcTemplate$6.doInCallableStatement(JdbcTemplate.java:1135) > [spring-jdbc.jar:4.3.9.RELEASE] > at > org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1083) > [spring-jdbc.jar:4.3.9.RELEASE] > ... 215 more > > 2018-03-07 17:03:38,453-05 DEBUG > [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] > (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Executing batch for > procedure Updatevds_interface_statistics > 2018-03-07 17:03:38,453-05 DEBUG > [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] > (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Mapped params: > [v_iface_status, v_sample_time, v_rx_offset, v_rx_rate, v_tx_rate, > v_tx_drop, v_tx_offset, v_id, v_rx_drop, v_rx_total, v_tx_total, v_vds_id] > 2018-03-07 17:03:38,453-05 DEBUG > [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] > (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Mapped params: > [v_iface_status, v_sample_time, v_rx_offset, v_rx_rate, v_tx_rate, > v_tx_drop, v_tx_offset, v_id, v_rx_drop, v_rx_total, v_tx_total, v_vds_id] > 2018-03-07 17:03:38,453-05 DEBUG > [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] > (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Mapped params: > [v_iface_status, v_sample_time, v_rx_offset, v_rx_rate, v_tx_rate, > v_tx_drop, v_tx_offset, v_id, v_rx_drop, v_rx_total, v_tx_total, v_vds_id] > 2018-03-07 17:03:38,453-05 DEBUG > [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] > (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Mapped params: > [v_iface_status, v_sample_time, v_rx_offset, v_rx_rate, v_tx_rate, > v_tx_drop, v_tx_offset, v_id, v_rx_drop, v_rx_total, v_tx_total, v_vds_id] > 2018-03-07 17:03:38,453-05 DEBUG > [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] > (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Mapped params: > [v_iface_status, v_sample_time, v_rx_offset, v_rx_rate, v_tx_rate, > v_tx_drop, v_tx_offset, v_id, v_rx_drop, v_rx_total, v_tx_total, v_vds_id] > 2018-03-07 17:03:38,456-05 DEBUG > [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] > (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Executed batch > 2018-03-07 17:03:38,457-05 DEBUG > [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] > (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Executing batch for > procedure UpdateNumaNodeStatistics > 2018-03-07 17:03:38,457-05 DEBUG > [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] > (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Mapped params: > [v_numa_node_id, v_mem_free, v_usage_mem_percent, v_cpu_sys, v_cpu_user, > v_cpu_idle, v_usage_cpu_percent] > 2018-03-07 17:03:38,459-05 DEBUG > [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] > (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Executed batch > 2018-03-07 17:03:38,462-05 ERROR > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > (default task-21) [a514da44-d44f-46b9-8bdf-08ba2d513929] EVENT_ID: > USER_FAILED_RUN_VM(54), Failed to run VM backup_vm (User: admin at in: > 2018-03-07 17:03:38,466-05 DEBUG > [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] > (EE-ManagedThreadFactory-engine-Thread-182) [] Executing task: > EE-ManagedThreadFactory-engine-Thread-182 > 2018-03-07 17:03:38,467-05 INFO > [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-21) > [a514da44-d44f-46b9-8bdf-08ba2d513929] Lock freed to object > 'EngineLock:{exclusiveLocks='[634c6a46-d057-4509-be3b-710716cbd56d=VM]', > sharedLocks=''}' > 2018-03-07 17:03:38,467-05 DEBUG > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > (default task-21) [a514da44-d44f-46b9-8bdf-08ba2d513929] method: runAction, > params: [RunVmOnce, > RunVmOnceParams:{commandId='d5358409-e005-4c8b-a702-d4fa394ee9d2', > user='null', commandType='Unknown', > vmId='634c6a46-d057-4509-be3b-710716cbd56d'}], timeElapsed: 201ms > 2018-03-07 17:03:38,473-05 ERROR > [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default > task-21) [] Operation Failed: [Internal Engine Error] > > > > > > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel From sbonazzo at redhat.com Thu Mar 8 12:46:30 2018 From: sbonazzo at redhat.com (Sandro Bonazzola) Date: Thu, 8 Mar 2018 13:46:30 +0100 Subject: [ovirt-devel] he-basic-ansible-suite-master and he-basic-suite-master Message-ID: Hi, I'm looking at diffs between he-basic-ansible-suite-master and he-basic-suite-master in oVirt Sytem Test. Since we are now defaulting to the ansible deployment, aren't these 2 suites substantially testing the same flow? Can we drop one of them? Or should we change the non ansible suite to enforce using legacy otopi flow? Thanks, -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA TRIED. TESTED. TRUSTED. -------------- next part -------------- An HTML attachment was scrubbed... URL: From msivak at redhat.com Thu Mar 8 12:49:36 2018 From: msivak at redhat.com (Martin Sivak) Date: Thu, 8 Mar 2018 13:49:36 +0100 Subject: [ovirt-devel] he-basic-ansible-suite-master and he-basic-suite-master In-Reply-To: References: Message-ID: Isn't it enforcing the legacy flow already? I think it was last time I tried. Martin On Thu, Mar 8, 2018 at 1:46 PM, Sandro Bonazzola wrote: > Hi, > I'm looking at diffs between he-basic-ansible-suite-master and > he-basic-suite-master in oVirt Sytem Test. Since we are now defaulting to > the ansible deployment, aren't these 2 suites substantially testing the > same flow? > > Can we drop one of them? Or should we change the non ansible suite to > enforce using legacy otopi flow? > Thanks, > -- > > SANDRO BONAZZOLA > > ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D > > Red Hat EMEA > > TRIED. TESTED. TRUSTED. > > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ykaul at redhat.com Thu Mar 8 12:49:23 2018 From: ykaul at redhat.com (Yaniv Kaul) Date: Thu, 8 Mar 2018 14:49:23 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 (ovirt-hosted-engine-setup) ] [ 08-03-2018 ] [ 004_basic_sanity.run_vms ] In-Reply-To: References: Message-ID: On Thu, Mar 8, 2018 at 1:53 PM, Dafna Ron wrote: > Hi, > > We have a failed test on ovirt-hosted-engine-setup basic suite. > We failed to run a vm with internal engine error. > > > > > > > > > > > > > > > > > > > *Link and headline of suspected patches: Add engine fqdn to inventory and > allow dynamic inventory scripts - https://gerrit.ovirt.org/#/c/88622/ > Link to > Job:http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1064/ > Link to > all > logs:http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1064/artifact/ > (Relevant) > error snippet from the log: 2018-03-07 17:03:38,374-05 INFO > [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-21) > [a514da44-d44f-46b9-8bdf-08ba2d513929] Running command: RunVmOnceCommand > internal: false. Entities affected : ID: > 634c6a46-d057-4509-be3b-710716cbd56d Type: VMAction group RUN_VM with role > type USER, ID: 634c6a46-d057-4509-be3b-710716cbd56d Type: VMAction group > EDIT_ADMIN_VM_PROPERTIES with role type ADMIN2018-03-07 17:03:38,379-05 > DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > (default task-21) [a514da44-d44f-46b9-8bdf-08ba2d513929] method: > getVmManager, params: [634c6a46-d057-4509-be3b-710716cbd56d], timeElapsed: > 5ms2018-03-07 17:03:38,391-05 DEBUG > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > (default task-21) [a514da44-d44f-46b9-8bdf-08ba2d513929] method: > getAllForClusterWithStatus, params: [14cad400-49a0-44e0-ab15-9da778f08082, > Up], timeElapsed: 7ms2018-03-07 17:03:38,408-05 DEBUG > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > (default task-21) [a514da44-d44f-46b9-8bdf-08ba2d513929] method: > getVdsManager, params: [8ef2f490-1b76-46e0-b9fe-f3412a36e03b], timeElapsed: > 0ms2018-03-07 17:03:38,419-05 DEBUG > [org.ovirt.engine.core.dal.dbbroker.CustomSQLErrorCodeSQLExceptionTranslator] > (default task-21) [a514da44-d44f-46b9-8bdf-08ba2d513929] Translating > SQLException with SQL state '23505', error code '0', message [ERROR: > duplicate key value violates unique constraint "name_server_pkey"* > A quick Google search on the above seems to point to https://bugzilla.redhat.com/show_bug.cgi?id=1547070 , which says: Steps to Reproduce: 1. Refresh caps during start VM operation So might be relevant, I think. Y. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > * Detail: Key (dns_resolver_configuration_id, > address)=(8a941b6a-83aa-44de-9800-2a1ea6e8e029, 192.168.200.1) already > exists. Where: SQL statement "INSERT INTO name_server( > address, position, dns_resolver_configuration_id) VALUES > ( v_address, v_position, > v_dns_resolver_configuration_id)"PL/pgSQL function > insertnameserver(uuid,character varying,smallint) line 3 at SQL statement]; > SQL was [{call insertnameserver(?, ?, ?)}] for task > [CallableStatementCallback]2018-03-07 17:03:38,420-05 ERROR > [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-21) > [a514da44-d44f-46b9-8bdf-08ba2d513929] Command > 'org.ovirt.engine.core.bll.RunVmOnceCommand' failed: > CallableStatementCallback; SQL [{call insertnameserver(?, ?, ?)}]; ERROR: > duplicate key value violates unique constraint "name_server_pkey" Detail: > Key (dns_resolver_configuration_id, > address)=(8a941b6a-83aa-44de-9800-2a1ea6e8e029, 192.168.200.1) already > exists. Where: SQL statement "INSERT INTO name_server( > address, position, dns_resolver_configuration_id) VALUES > ( v_address, v_position, > v_dns_resolver_configuration_id)"PL/pgSQL function > insertnameserver(uuid,character varying,smallint) line 3 at SQL statement; > nested exception is org.postgresql.util.PSQLException: ERROR: duplicate key > value violates unique constraint "name_server_pkey" Detail: Key > (dns_resolver_configuration_id, > address)=(8a941b6a-83aa-44de-9800-2a1ea6e8e029, 192.168.200.1) already > exists. Where: SQL statement "INSERT INTO name_server( > address, position, dns_resolver_configuration_id) VALUES > ( v_address, v_position, > v_dns_resolver_configuration_id)"PL/pgSQL function > insertnameserver(uuid,character varying,smallint) line 3 at SQL > statement2018-03-07 17:03:38,420-05 ERROR > [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-21) > [a514da44-d44f-46b9-8bdf-08ba2d513929] Exception: > org.springframework.dao.DuplicateKeyException: CallableStatementCallback; > SQL [{call insertnameserver(?, ?, ?)}]; ERROR: duplicate key value violates > unique constraint "name_server_pkey" Detail: Key > (dns_resolver_configuration_id, > address)=(8a941b6a-83aa-44de-9800-2a1ea6e8e029, 192.168.200.1) already > exists. Where: SQL statement "INSERT INTO name_server( > address, position, dns_resolver_configuration_id) VALUES (: > v_address, v_position, v_dns_resolver_configuration_id)"PL/pgSQL > function insertnameserver(uuid,character varying,smallint) line 3 at SQL > statement; nested exception is org.postgresql.util.PSQLException: ERROR: > duplicate key value violates unique constraint "name_server_pkey" Detail: > Key (dns_resolver_configuration_id, > address)=(8a941b6a-83aa-44de-9800-2a1ea6e8e029, 192.168.200.1) already > exists. Where: SQL statement "INSERT INTO name_server( > address, position, dns_resolver_configuration_id) VALUES > ( v_address, v_position, > v_dns_resolver_configuration_id)"PL/pgSQL function > insertnameserver(uuid,character varying,smallint) line 3 at SQL > statement at > org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:239) > [spring-jdbc.jar:4.3.9.RELEASE] at > org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:73) > [spring-jdbc.jar:4.3.9.RELEASE] at > org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1099) > [spring-jdbc.jar:4.3.9.RELEASE] at > org.springframework.jdbc.core.JdbcTemplate.call(JdbcTemplate.java:1135) > [spring-jdbc.jar:4.3.9.RELEASE] at > org.springframework.jdbc.core.simple.AbstractJdbcCall.executeCallInternal(AbstractJdbcCall.java:405) > [spring-jdbc.jar:4.3.9.RELEASE] at > org.springframework.jdbc.core.simple.AbstractJdbcCall.doExecute(AbstractJdbcCall.java:365) > [spring-jdbc.jar:4.3.9.RELEASE] at > org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:198) > [spring-jdbc.jar:4.3.9.RELEASE] at > org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:135) > [dal.jar:] at > org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:130) > [dal.jar:] at > org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeModification(SimpleJdbcCallsHandler.java:76) > [dal.jar:] at > org.ovirt.engine.core.dao.network.DnsResolverConfigurationDaoImpl.saveNameServersByDnsResolverConfigurationId(DnsResolverConfigurationDaoImpl.java:66) > [dal.jar:] at > org.ovirt.engine.core.dao.network.DnsResolverConfigurationDaoImpl.update(DnsResolverConfigurationDaoImpl.java:110) > [dal.jar:] at > org.ovirt.engine.core.dao.network.DnsResolverConfigurationDaoImpl.update(DnsResolverConfigurationDaoImpl.java:15) > [dal.jar:] at > org.ovirt.engine.core.dao.VdsDynamicDaoImpl.updateDnsResolverConfiguration(VdsDynamicDaoImpl.java:156) > [dal.jar:] at > org.ovirt.engine.core.dao.VdsDynamicDaoImpl.update(VdsDynamicDaoImpl.java:145) > [dal.jar:] at > org.ovirt.engine.core.dao.VdsDynamicDaoImpl.updateIfNeeded(VdsDynamicDaoImpl.java:343) > [dal.jar:] at > org.ovirt.engine.core.dao.VdsDynamicDaoImpl.updateIfNeeded(VdsDynamicDaoImpl.java:35) > [dal.jar:] at > org.ovirt.engine.core.vdsbroker.VdsManager.updateDynamicData(VdsManager.java:473) > [vdsbroker.jar:] at > org.ovirt.engine.core.vdsbroker.VdsManager.updatePendingData(VdsManager.java:511) > [vdsbroker.jar:] at > org.ovirt.engine.core.bll.scheduling.pending.PendingResourceManager.notifyHostManagers(PendingResourceManager.java:227) > [bll.jar:] at > org.ovirt.engine.core.bll.scheduling.SchedulingManager.schedule(SchedulingManager.java:361) > [bll.jar:] at > org.ovirt.engine.core.bll.RunVmCommand.getVdsToRunOn(RunVmCommand.java:853) > [bll.jar:] at > org.ovirt.engine.core.bll.RunVmCommand.runVm(RunVmCommand.java:256) > [bll.jar:] at > org.ovirt.engine.core.bll.RunVmCommand.perform(RunVmCommand.java:430) > [bll.jar:] at > org.ovirt.engine.core.bll.RunVmCommand.executeVmCommand(RunVmCommand.java:355) > [bll.jar:] at > org.ovirt.engine.core.bll.VmCommand.executeCommand(VmCommand.java:161) > [bll.jar:] at > org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1133) > [bll.jar:] at > org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1285) > [bll.jar:] at > org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1934) > [bll.jar:] at > org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:164) > [utils.jar:] at > org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:103) > [utils.jar:] at > org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1345) > [bll.jar:] at > org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:400) > [bll.jar:] at > org.ovirt.engine.core.bll.executor.DefaultBackendActionExecutor.execute(DefaultBackendActionExecutor.java:13) > [bll.jar:] at > org.ovirt.engine.core.bll.Backend.runAction(Backend.java:468) > [bll.jar:] at > org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:450) > [bll.jar:] at > org.ovirt.engine.core.bll.Backend.runAction(Backend.java:403) > [bll.jar:] at sun.reflect.GeneratedMethodAccessor134.invoke(Unknown > Source) [:1.8.0_161] at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > [rt.jar:1.8.0_161] at > java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_161] > at > org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)at > org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) > at > org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:92) > [wildfly-weld-ejb-11.0.0.Final.jar:11.0.0.Final] at > org.jboss.weld.interceptor.proxy.WeldInvocationContext.interceptorChainCompleted(WeldInvocationContext.java:98) > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at > org.jboss.weld.interceptor.proxy.WeldInvocationContext.proceed(WeldInvocationContext.java:117) > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at > org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12) > [common.jar:] at > sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source) > [:1.8.0_161] at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > [rt.jar:1.8.0_161] at > java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_161] > at > org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73) > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at > org.jboss.weld.interceptor.proxy.WeldInvocationContext.invokeNext(WeldInvocationContext.java:83) > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at > org.jboss.weld.interceptor.proxy.WeldInvocationContext.proceed(WeldInvocationContext.java:115) > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at > org.jboss.weld.bean.InterceptorImpl.intercept(InterceptorImpl.java:108) > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at > org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:82) > [wildfly-weld-ejb-11.0.0.Final.jar:11.0.0.Final] at > org.jboss.as.weld.interceptors.EjbComponentInterceptorSupport.delegateInterception(EjbComponentInterceptorSupport.java:60) > at > org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:76) > at > org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:88) > at > org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:101) > at > org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) > at > org.ovirt.engine.core.bll.interceptors.CorrelationIdTrackerInterceptor.aroundInvoke(CorrelationIdTrackerInterceptor.java:13) > [bll.jar:] at sun.reflect.GeneratedMethodAccessor66.invoke(Unknown > Source) [:1.8.0_161] at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > [rt.jar:1.8.0_161] at > java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_161] > at > org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:89) > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) > [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45) > [wildfly-ee-11.0.0.Final.jar:11.0.0.Final] at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:40) > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53) > at > org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52) > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) > [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:264) > [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] at > org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:379) > [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] at > org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:244) > [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) > at > org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73) > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at > org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89) > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) > [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) > [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) > [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) > [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) > [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) > [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60) > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438) > at > org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:609) > at > org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57) > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53) > at > org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) > at > org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) > at > org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:81) > at > org.ovirt.engine.core.common.interfaces.BackendLocal$$$view2.runAction(Unknown > Source) [common.jar:] at > org.ovirt.engine.api.restapi.resource.BackendResource.doAction(BackendResource.java:250) > at > org.ovirt.engine.api.restapi.resource.AbstractBackendActionableResource.doAction(AbstractBackendActionableResource.java:84) > at > org.ovirt.engine.api.restapi.resource.AbstractBackendActionableResource.doAction(AbstractBackendActionableResource.java:125) > at > org.ovirt.engine.api.restapi.resource.BackendVmResource.start(BackendVmResource.java:450) > at org.ovirt.engine.api.v3.V3Server.adaptAction(V3Server.java:217) > at > org.ovirt.engine.api.v3.servers.V3VmServer.start(V3VmServer.java:216) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > [rt.jar:1.8.0_161] at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > [rt.jar:1.8.0_161] at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > [rt.jar:1.8.0_161] at > java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_161] > at > org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:140) > [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] at > org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) > [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] at > org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) > [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] at > org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:138) > [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] at > org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:107) > [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] at > org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:133) > [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] at > org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:101) > [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] at > org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:406) > [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] at > org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:213) > [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] at > org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:228) > [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] at > org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) > [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] at > org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) > [resteasy-jaxrs-3.0.24.Final.jar:3.0.24.Final] at > javax.servlet.http.HttpServlet.service(HttpServlet.java:790) > [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final] at > io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) > at > io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:81) > at > io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) > at > io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) > at > io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) > at > io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) > at > io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) > at > io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:274) > at > io.undertow.servlet.handlers.ServletInitialHandler.dispatchToPath(ServletInitialHandler.java:209) > at > io.undertow.servlet.spec.RequestDispatcherImpl.forwardImpl(RequestDispatcherImpl.java:221) > at > io.undertow.servlet.spec.RequestDispatcherImpl.forwardImplSetup(RequestDispatcherImpl.java:147) > at > io.undertow.servlet.spec.RequestDispatcherImpl.forward(RequestDispatcherImpl.java:111) > at > org.ovirt.engine.api.restapi.invocation.VersionFilter.doFilter(VersionFilter.java:180) > at > org.ovirt.engine.api.restapi.invocation.VersionFilter.doFilter(VersionFilter.java:98) > at > io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) > at > io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) > at > org.ovirt.engine.api.restapi.invocation.CurrentFilter.doFilter(CurrentFilter.java:117) > at > org.ovirt.engine.api.restapi.invocation.CurrentFilter.doFilter(CurrentFilter.java:72) > at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) > at > io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) > at > org.ovirt.engine.core.aaa.filters.RestApiSessionMgmtFilter.doFilter(RestApiSessionMgmtFilter.java:78) > [aaa.jar:] at > io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) > at > io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) > at > org.ovirt.engine.core.aaa.filters.EnforceAuthFilter.doFilter(EnforceAuthFilter.java:42) > [aaa.jar:] at > io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) > at > io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) > at > org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter.doFilter(SsoRestApiNegotiationFilter.java:84) > [aaa.jar:] at > io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) > at > io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) > at > org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter.doFilter(SsoRestApiAuthFilter.java:47) > [aaa.jar:] at > io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) > at > io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) > at > org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:59) > [aaa.jar:] at > io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) > at > io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) > at > org.ovirt.engine.core.aaa.filters.RestApiSessionValidationFilter.doFilter(RestApiSessionValidationFilter.java:35) > [aaa.jar:] at > io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) > at > io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) > at > org.ovirt.engine.api.restapi.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:111) > at > org.ovirt.engine.api.restapi.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:102) > at > io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) > at > io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) > at > org.ovirt.engine.core.utils.servlet.CORSSupportFilter.doFilter(CORSSupportFilter.java:284) > [utils.jar:] at > io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) > at > io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) > at > io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84) > at > io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) > at > io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) > at > org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) > at > io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) > at > io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) > at > io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) > at > io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) > at > io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) > at > io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) > at > io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) > at > io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) > at > io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) > at > io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) > at > io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) > at > io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) > at > io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) > at > org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) > at > io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) > at > io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) > at > io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) > at > io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) > at > io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) > at > io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) > at > io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) > at > io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) > at > org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) > at > org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) > at > org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) > at > org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) > at > io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) > at > io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) > at > io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) > at > io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) > at > io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) > at > io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) > at > org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) > at > org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) > at > org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) > at > org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) > at > io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) > at > io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) > at > io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) > at > io.undertow.server.Connectors.executeRootHandler(Connectors.java:326) > at > io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:812) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > [rt.jar:1.8.0_161] at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [rt.jar:1.8.0_161] at java.lang.Thread.run(Thread.java:748) > [rt.jar:1.8.0_161]Caused by: org.postgresql.util.PSQLException: ERROR: > duplicate key value violates unique constraint "name_server_pkey" Detail: > Key (dns_resolver_configuration_id, > address)=(8a941b6a-83aa-44de-9800-2a1ea6e8e029, 192.168.200.1) already > exists. Where: SQL statement "INSERT INTO name_server( > address, position, dns_resolver_configuration_id) VALUES > ( v_address, v_position, > v_dns_resolver_configuration_id)"PL/pgSQL function > insertnameserver(uuid,character varying,smallint) line 3 at SQL > statement at > org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2157) > at > org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1886) > at > org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) > at > org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:555) > at > org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:417) > at > org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:410) > at > org.jboss.jca.adapters.jdbc.CachedPreparedStatement.execute(CachedPreparedStatement.java:303) > at > org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.execute(WrappedPreparedStatement.java:442) > at > org.springframework.jdbc.core.JdbcTemplate$6.doInCallableStatement(JdbcTemplate.java:1138) > [spring-jdbc.jar:4.3.9.RELEASE] at > org.springframework.jdbc.core.JdbcTemplate$6.doInCallableStatement(JdbcTemplate.java:1135) > [spring-jdbc.jar:4.3.9.RELEASE] at > org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1083) > [spring-jdbc.jar:4.3.9.RELEASE] ... 215 more2018-03-07 > 17:03:38,453-05 DEBUG > [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] > (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Executing batch for > procedure Updatevds_interface_statistics2018-03-07 17:03:38,453-05 DEBUG > [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] > (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Mapped params: > [v_iface_status, v_sample_time, v_rx_offset, v_rx_rate, v_tx_rate, > v_tx_drop, v_tx_offset, v_id, v_rx_drop, v_rx_total, v_tx_total, > v_vds_id]2018-03-07 17:03:38,453-05 DEBUG > [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] > (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Mapped params: > [v_iface_status, v_sample_time, v_rx_offset, v_rx_rate, v_tx_rate, > v_tx_drop, v_tx_offset, v_id, v_rx_drop, v_rx_total, v_tx_total, > v_vds_id]2018-03-07 17:03:38,453-05 DEBUG > [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] > (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Mapped params: > [v_iface_status, v_sample_time, v_rx_offset, v_rx_rate, v_tx_rate, > v_tx_drop, v_tx_offset, v_id, v_rx_drop, v_rx_total, v_tx_total, > v_vds_id]2018-03-07 17:03:38,453-05 DEBUG > [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] > (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Mapped params: > [v_iface_status, v_sample_time, v_rx_offset, v_rx_rate, v_tx_rate, > v_tx_drop, v_tx_offset, v_id, v_rx_drop, v_rx_total, v_tx_total, > v_vds_id]2018-03-07 17:03:38,453-05 DEBUG > [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] > (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Mapped params: > [v_iface_status, v_sample_time, v_rx_offset, v_rx_rate, v_tx_rate, > v_tx_drop, v_tx_offset, v_id, v_rx_drop, v_rx_total, v_tx_total, > v_vds_id]2018-03-07 17:03:38,456-05 DEBUG > [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] > (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Executed > batch2018-03-07 17:03:38,457-05 DEBUG > [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] > (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Executing batch for > procedure UpdateNumaNodeStatistics2018-03-07 17:03:38,457-05 DEBUG > [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] > (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Mapped params: > [v_numa_node_id, v_mem_free, v_usage_mem_percent, v_cpu_sys, v_cpu_user, > v_cpu_idle, v_usage_cpu_percent]2018-03-07 17:03:38,459-05 DEBUG > [org.ovirt.engine.core.dal.dbbroker.BatchProcedureExecutionConnectionCallback] > (EE-ManagedThreadFactory-engineScheduled-Thread-82) [] Executed > batch2018-03-07 17:03:38,462-05 ERROR > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > (default task-21) [a514da44-d44f-46b9-8bdf-08ba2d513929] EVENT_ID: > USER_FAILED_RUN_VM(54), Failed to run VM backup_vm (User: > admin at in:2018-03-07 17:03:38,466-05 DEBUG > [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] > (EE-ManagedThreadFactory-engine-Thread-182) [] Executing task: > EE-ManagedThreadFactory-engine-Thread-1822018-03-07 17:03:38,467-05 INFO > [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-21) > [a514da44-d44f-46b9-8bdf-08ba2d513929] Lock freed to object > 'EngineLock:{exclusiveLocks='[634c6a46-d057-4509-be3b-710716cbd56d=VM]', > sharedLocks=''}'2018-03-07 17:03:38,467-05 DEBUG > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > (default task-21) [a514da44-d44f-46b9-8bdf-08ba2d513929] method: runAction, > params: [RunVmOnce, > RunVmOnceParams:{commandId='d5358409-e005-4c8b-a702-d4fa394ee9d2', > user='null', commandType='Unknown', > vmId='634c6a46-d057-4509-be3b-710716cbd56d'}], timeElapsed: 201ms2018-03-07 > 17:03:38,473-05 ERROR > [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default > task-21) [] Operation Failed: [Internal Engine Error]* > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From didi at redhat.com Thu Mar 8 12:55:24 2018 From: didi at redhat.com (Yedidyah Bar David) Date: Thu, 8 Mar 2018 14:55:24 +0200 Subject: [ovirt-devel] he-basic-ansible-suite-master and he-basic-suite-master In-Reply-To: References: Message-ID: On Thu, Mar 8, 2018 at 2:49 PM, Martin Sivak wrote: > Isn't it enforcing the legacy flow already? I think it was last time I > tried. > It should: https://gerrit.ovirt.org/86398 > > Martin > > On Thu, Mar 8, 2018 at 1:46 PM, Sandro Bonazzola > wrote: > >> Hi, >> I'm looking at diffs between he-basic-ansible-suite-master and >> he-basic-suite-master in oVirt Sytem Test. Since we are now defaulting to >> the ansible deployment, aren't these 2 suites substantially testing the >> same flow? >> >> Can we drop one of them? Or should we change the non ansible suite to >> enforce using legacy otopi flow? >> Thanks, >> -- >> >> SANDRO BONAZZOLA >> >> ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D >> >> Red Hat EMEA >> >> TRIED. TESTED. TRUSTED. >> >> >> _______________________________________________ >> Devel mailing list >> Devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel >> > > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -- Didi -------------- next part -------------- An HTML attachment was scrubbed... URL: From sbonazzo at redhat.com Thu Mar 8 13:08:23 2018 From: sbonazzo at redhat.com (sbonazzo at redhat.com) Date: Thu, 08 Mar 2018 13:08:23 +0000 Subject: [ovirt-devel] Invitation: oVirt System Test Hackathon @ Tue Mar 13, 2018 (devel@ovirt.org) Message-ID: <94eb2c1a6834de97380566e65e48@google.com> You have been invited to the following event. Title: oVirt System Test Hackathon Please join us in an ovirt-system-tests hackathon pushing new tests and improving existing ones for testing Hosted Engine. Git repo is available: https://gerrit.ovirt.org/gitweb?p=ovirt-system-tests.git;a=summary Integration, Node and CI team will be available for helping in the effort and reviewing patches. When: Tue Mar 13, 2018 Where: #ovirt IRC channel Calendar: devel at ovirt.org Who: * sbonazzo at redhat.com - organizer * devel at ovirt.org * users at ovirt.org Event details: https://www.google.com/calendar/event?action=VIEW&eid=MDhuZTY5bXRxbGo0cnNmMzIzMzEyMjYwZnEgZGV2ZWxAb3ZpcnQub3Jn&tok=MTkjc2JvbmF6em9AcmVkaGF0LmNvbTQxMmNiNWJhMzE2NzhlMDA0NWM2MzYyODI2ZDNlMjI2NzE2N2U0OTA&ctz=Europe/Rome&hl=en Invitation from Google Calendar: https://www.google.com/calendar/ You are receiving this courtesy email at the account devel at ovirt.org because you are an attendee of this event. To stop receiving future updates for this event, decline this event. Alternatively you can sign up for a Google account at https://www.google.com/calendar/ and control your notification settings for your entire calendar. Forwarding this invitation could allow any recipient to modify your RSVP response. Learn more at https://support.google.com/calendar/answer/37135#forwarding -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 1746 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: invite.ics Type: application/ics Size: 1782 bytes Desc: not available URL: From sbonazzo at redhat.com Thu Mar 8 13:57:38 2018 From: sbonazzo at redhat.com (Sandro Bonazzola) Date: Thu, 8 Mar 2018 14:57:38 +0100 Subject: [ovirt-devel] Heads up: dropping fcraw jobs from jenkins Message-ID: Following today call, I'm going to push a patch for dropping all rawhide jobs from jenkins. This should help avoiding distractions and reducing patches blocked by rawhide issues. -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA TRIED. TESTED. TRUSTED. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sbonazzo at redhat.com Thu Mar 8 14:21:16 2018 From: sbonazzo at redhat.com (sbonazzo at redhat.com) Date: Thu, 08 Mar 2018 14:21:16 +0000 Subject: [ovirt-devel] Updated invitation: oVirt System Test Hackathon @ Tue Mar 13, 2018 (devel@ovirt.org) Message-ID: <001a113c533888f4210566e7637a@google.com> This event has been changed. Title: oVirt System Test Hackathon Please join us in an ovirt-system-tests hackathon pushing new tests and improving existing ones for testing Hosted Engine.Git repo is available: https://gerrit.ovirt.org/gitweb?p=ovirt-system-tests.git;a=summaryIntegration, Node and CI team will be available for helping in the effort and reviewing patches.Here's a public trello board tracking the efforts: https://trello.com/b/Pp76YoRL (changed) When: Tue Mar 13, 2018 Where: #ovirt IRC channel Calendar: devel at ovirt.org Who: * sbonazzo at redhat.com - organizer * devel at ovirt.org * users at ovirt.org Event details: https://www.google.com/calendar/event?action=VIEW&eid=MDhuZTY5bXRxbGo0cnNmMzIzMzEyMjYwZnEgZGV2ZWxAb3ZpcnQub3Jn&tok=MTkjc2JvbmF6em9AcmVkaGF0LmNvbTQxMmNiNWJhMzE2NzhlMDA0NWM2MzYyODI2ZDNlMjI2NzE2N2U0OTA&ctz=Europe/Rome&hl=en Invitation from Google Calendar: https://www.google.com/calendar/ You are receiving this courtesy email at the account devel at ovirt.org because you are an attendee of this event. To stop receiving future updates for this event, decline this event. Alternatively you can sign up for a Google account at https://www.google.com/calendar/ and control your notification settings for your entire calendar. Forwarding this invitation could allow any recipient to modify your RSVP response. Learn more at https://support.google.com/calendar/answer/37135#forwarding -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 2222 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: invite.ics Type: application/ics Size: 2268 bytes Desc: not available URL: From sbonazzo at redhat.com Fri Mar 9 09:02:13 2018 From: sbonazzo at redhat.com (Sandro Bonazzola) Date: Fri, 9 Mar 2018 10:02:13 +0100 Subject: [ovirt-devel] qemu-kvm-ev-2.9.0-16.el7_4.14.1 has been released Message-ID: Hi, qemu-kvm-ev-2.9.0-16.el7_4.14.1 has been tagged for release and should land on mirrors.centos.org on Monday, March 12th 2018. Here's the ChangeLog: * Thu Mar 08 2018 Sandro Bonazzola - ev-2.9.0-16.el7_4.14.1 - Removing RH branding from package name * Thu Jan 18 2018 Miroslav Rezanina - rhev-2.9.0-16.el7_4.14 - kvm-fw_cfg-fix-memory-corruption-when-all-fw_cfg-slots-a.patch [bz#1534649] - kvm-mirror-Fix-inconsistent-backing-AioContext-for-after.patch [bz#1535125] - Resolves: bz#1534649 (Qemu crashes when all fw_cfg slots are used [rhel-7.4.z]) - Resolves: bz#1535125 (Mirror jobs for drives with iothreads make QEMU to abort with "block.c:1895: bdrv_attach_child: Assertion `bdrv_get_aio_context(parent_bs) == bdrv_get_aio_context(child_bs)' failed." [rhel-7.4.z]) Regards, -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA TRIED. TESTED. TRUSTED. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dron at redhat.com Fri Mar 9 19:36:53 2018 From: dron at redhat.com (Dafna Ron) Date: Fri, 9 Mar 2018 19:36:53 +0000 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 (ovirt-engine) ] [ 04-03-2018 ] [ 004_basic_sanity.disk_operations ] In-Reply-To: References: Message-ID: https://bugzilla.redhat.com/show_bug.cgi?id=1553893 On Sun, Mar 4, 2018 at 10:51 PM, Nir Soffer wrote: > On Sun, Mar 4, 2018 at 6:43 PM Yaniv Kaul wrote: > >> On Sun, Mar 4, 2018 at 5:48 PM, Nir Soffer wrote: >> >>> On Sun, Mar 4, 2018 at 5:31 PM Yaniv Kaul wrote: >>> >>>> On Sun, Mar 4, 2018 at 5:18 PM, Daniel Belenky >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> The following test failed OST: 004_basic_sanity.disk_operations. >>>>> >>>>> Link to suspected patch: https://gerrit.ovirt.org/c/88404/ >>>>> Link to the failed job: http://jenkins.ovirt.org/ >>>>> job/ovirt-4.2_change-queue-tester/1019/ >>>>> Link to all test logs: >>>>> >>>>> - engine >>>>> >>>>> - host 0 >>>>> >>>>> - host 1 >>>>> >>>>> >>>>> Error snippet from engine: >>>>> >>>>> 2018-03-04 09:50:14,823-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ForkJoinPool-1-worker-12) [] EVENT_ID: VM_DOWN_ERROR(119), VM vm2 is down with error. Exit message: Lost connection with qemu process. >>>>> >>>>> >>>>> Error snippet from host: >>>>> >>>>> Mar 4 09:56:27 lago-basic-suite-4-2-host-1 libvirtd: 2018-03-04 14:56:27.831+0000: 1189: error : qemuDomainAgentAvailable:6010 : Guest agent is not responding: QEMU guest agent is not connected >>>>> >>>>> >>>> That's not surprising - there's no guest agent there. >>>> >>> >>> There are 2 issues here: >>> - we are testing without guest agent when this is the recommended >>> configuration >>> (snapshots may not be consistent without guest agent) >>> >> >> We are still using Cirros. I need to get a CentOS with cloud-init >> uploaded (WIP...) >> >> >>> - vdsm should not report errors about guest agent since it does not know >>> if >>> guest agent is installed or not. This message should be an INFO message >>> like >>> "could not stop the vm using guest agent, falling back to ..." >>> >>> Generally we should not see ERROR or WARN message in OST. Any repeating >>> error or warning should be reported as a bug. >>> >> >> There are several on storage... Should we file BZs on them? >> > > I think we have only warnings now, but they should be fixed, so please > file a bug. > > >> Y. >> >> >>> >>> Nir >>> >> >> > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dron at redhat.com Fri Mar 9 20:02:23 2018 From: dron at redhat.com (Dafna Ron) Date: Fri, 9 Mar 2018 20:02:23 +0000 Subject: [ovirt-devel] OST Failure - Weekly update [03/03/2018-09/03/2018] Message-ID: Hello, I would like to update on this week's failures and OST current status. We had a few failure this week: 1. A change in OST caused a failure in one of the tests. this was due to using a 4.3 SDK feature which is not available in the current SDK version. The change was https://gerrit.ovirt.org/#/c/84338 - https://gerrit.ovirt.org/#/c/84338/ We reverted the change: https://gerrit.ovirt.org/#/c/88441/ However, this merge actually exposed an issue in the vdsm logs which report guest agent errors. bug was opened: https://bugzilla.redhat.com/show_bug.cgi?id=1553893 2. We discovered that there was a syntax error in 'release_branches' in ovirt-ansible. we re-merged after fixing the issue - Thanks you Barak/Sandro and Ondra for investigating and fixing the issue. This solved issues that were caused by running tests from stable older ansible packages. 3. fcraw build-artifacts jobs were removed since they were causing changes to fail running on cq. it was decided that its better to disable those jobs for now. 4. we have a second failure in build-artifacts for fc27 which caused several changes that were merged in cockpit-ovirt to fail. Thank you Douglas for acting quickly to resolve the issue. *Below you can see the chart for this week's resolved issues but cause of failure: * Code = regression of working components/functionalities Configurations = package related issues Other = failed build artifacts Infra = infrastructure/OST/Lago related issues *Below is a chart of resolved failures based on ovirt version* *Below is a chart showing failures by suite type: * Thank you, Dafna -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 5764 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 8053 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 23779 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 30988 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 7018 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 27912 bytes Desc: not available URL: From danken at redhat.com Fri Mar 9 20:26:01 2018 From: danken at redhat.com (Dan Kenigsberg) Date: Fri, 9 Mar 2018 22:26:01 +0200 Subject: [ovirt-devel] OST Failure - Weekly update [03/03/2018-09/03/2018] In-Reply-To: References: Message-ID: On Fri, Mar 9, 2018 at 10:02 PM, Dafna Ron wrote: > Hello, > > I would like to update on this week's failures and OST current status. > > We had a few failure this week: > > 1. A change in OST caused a failure in one of the tests. this was due to > using a 4.3 SDK feature which is not available in the current SDK version. > The change was https://gerrit.ovirt.org/#/c/84338 - > https://gerrit.ovirt.org/#/c/84338/ > We reverted the change: https://gerrit.ovirt.org/#/c/88441/ > I believe that we where able to take this patch due to a bug in OST, which I believe Daniel already works on: the patch modified a file that is linked from the suite-4.2, but the CI job for the patch triggered only suite-master. We should be more prudent, as long as we use cross-suite symlinks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dbelenky at redhat.com Sun Mar 11 10:29:39 2018 From: dbelenky at redhat.com (Daniel Belenky) Date: Sun, 11 Mar 2018 12:29:39 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 ] [ 002_bootstrap.update_default_cluster ] Message-ID: Hi, The following patch failed to pass OST: https://gerrit.ovirt.org/#/c/88738/2 Link to failed build: here Link to all logs: here Error snippet from engine.log : 2018-03-11 06:10:58,521-04 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-14) [] Operation Failed: [Cannot edit Cluster. The chosen CPU is not supported.] Thanks, -- DANIEL BELENKY RHV DEVOPS -------------- next part -------------- An HTML attachment was scrubbed... URL: From danken at redhat.com Sun Mar 11 10:40:02 2018 From: danken at redhat.com (Dan Kenigsberg) Date: Sun, 11 Mar 2018 12:40:02 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 ] [ 002_bootstrap.update_default_cluster ] In-Reply-To: References: Message-ID: On Sun, Mar 11, 2018 at 12:29 PM, Daniel Belenky wrote: > Hi, > > The following patch failed to pass OST: https://gerrit.ovirt.org/#/c/88738/2 it is not efficient to link to the patch without giving its subject and author db: add func to turn table columns to empty string by Martin Perina. it wastes the time of unrelated people, or make them ignore the (highly appreciated!) OST failure reports. > Link to failed build: here > Link to all logs: here > > Error snippet from engine.log: > > 2018-03-11 06:10:58,521-04 ERROR > [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default > task-14) [] Operation Failed: [Cannot edit Cluster. The chosen CPU is not > supported.] > > > Thanks, > -- > > DANIEL BELENKY > > RHV DEVOPS > > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel From nsoffer at redhat.com Mon Mar 12 10:50:53 2018 From: nsoffer at redhat.com (Nir Soffer) Date: Mon, 12 Mar 2018 10:50:53 +0000 Subject: [ovirt-devel] Heads up: dropping fcraw jobs from jenkins In-Reply-To: References: Message-ID: On Thu, Mar 8, 2018 at 4:00 PM Sandro Bonazzola wrote: > Following today call, I'm going to push a patch for dropping all rawhide > jobs from jenkins. > This should help avoiding distractions and reducing patches blocked by > rawhide issues. > Hi Sandro, I worked hard to get ioprocess and ovirt-imageio working on fcraw, so this change will create extra work for me to fix the regressions later when we enable fcraw again. Please revert the change for the ovirt-imageio and ioprocess projects. Thanks, Nir -------------- next part -------------- An HTML attachment was scrubbed... URL: From amureini at redhat.com Mon Mar 12 10:59:53 2018 From: amureini at redhat.com (Allon Mureinik) Date: Mon, 12 Mar 2018 12:59:53 +0200 Subject: [ovirt-devel] [ovirt-users] qemu-kvm-ev-2.9.0-16.el7_4.14.1 has been released In-Reply-To: References: Message-ID: >From oVirt's perspective - this build includes a fix that allows for live storage migration of a disk that uses iothreads. I've already posted a vdsm patch to require it, reviews are welcome: https://gerrit.ovirt.org/#/c/88770/ And thanks for the quick turnaround here, Sandro! On Fri, Mar 9, 2018 at 11:02 AM, Sandro Bonazzola wrote: > Hi, qemu-kvm-ev-2.9.0-16.el7_4.14.1 > has been tagged for > release and should land on mirrors.centos.org on Monday, March 12th 2018. > > Here's the ChangeLog: > > * Thu Mar 08 2018 Sandro Bonazzola - > ev-2.9.0-16.el7_4.14.1 - Removing RH branding from package name * Thu Jan > 18 2018 Miroslav Rezanina - rhev-2.9.0-16.el7_4.14 > - kvm-fw_cfg-fix-memory-corruption-when-all-fw_cfg-slots-a.patch > [bz#1534649] - kvm-mirror-Fix-inconsistent-backing-AioContext-for-after.patch > [bz#1535125] - Resolves: bz#1534649 (Qemu crashes when all fw_cfg slots are > used [rhel-7.4.z]) - Resolves: bz#1535125 (Mirror jobs for drives with > iothreads make QEMU to abort with "block.c:1895: bdrv_attach_child: > Assertion `bdrv_get_aio_context(parent_bs) == > bdrv_get_aio_context(child_bs)' failed." [rhel-7.4.z]) > Regards, > > -- > > SANDRO BONAZZOLA > > ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D > > Red Hat EMEA > > TRIED. TESTED. TRUSTED. > > > _______________________________________________ > Users mailing list > Users at ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From didi at redhat.com Mon Mar 12 13:11:30 2018 From: didi at redhat.com (Yedidyah Bar David) Date: Mon, 12 Mar 2018 15:11:30 +0200 Subject: [ovirt-devel] Heads up: dropping fcraw jobs from jenkins In-Reply-To: References: Message-ID: On Mon, Mar 12, 2018 at 12:50 PM, Nir Soffer wrote: > On Thu, Mar 8, 2018 at 4:00 PM Sandro Bonazzola wrote: >> >> Following today call, I'm going to push a patch for dropping all rawhide >> jobs from jenkins. >> This should help avoiding distractions and reducing patches blocked by >> rawhide issues. > > > Hi Sandro, > > I worked hard to get ioprocess and ovirt-imageio working on fcraw, so this > change will create extra work for me to fix the regressions later when we > enable fcraw again. > > Please revert the change for the ovirt-imageio and ioprocess projects. I'd like too to keep fcraw, for otopi, now pushed https://gerrit.ovirt.org/88816 . Perhaps for the time being it's best if other projects' maintainers do the same, if they want to, and commit to fixing issues in fcraw quickly (I hope I do...). Best regards, -- Didi From eedri at redhat.com Mon Mar 12 14:36:28 2018 From: eedri at redhat.com (Eyal Edri) Date: Mon, 12 Mar 2018 16:36:28 +0200 Subject: [ovirt-devel] Heads up: dropping fcraw jobs from jenkins In-Reply-To: References: Message-ID: On Mon, Mar 12, 2018 at 3:11 PM, Yedidyah Bar David wrote: > On Mon, Mar 12, 2018 at 12:50 PM, Nir Soffer wrote: > > On Thu, Mar 8, 2018 at 4:00 PM Sandro Bonazzola > wrote: > >> > >> Following today call, I'm going to push a patch for dropping all rawhide > >> jobs from jenkins. > >> This should help avoiding distractions and reducing patches blocked by > >> rawhide issues. > > > > > > Hi Sandro, > > > > I worked hard to get ioprocess and ovirt-imageio working on fcraw, so > this > > change will create extra work for me to fix the regressions later when we > > enable fcraw again. > > > > Please revert the change for the ovirt-imageio and ioprocess projects. > > I'd like too to keep fcraw, for otopi, now pushed > https://gerrit.ovirt.org/88816 . > > Perhaps for the time being it's best if other projects' maintainers do > the same, if they want to, and commit to fixing issues in fcraw quickly > (I hope I do...). > Maybe its a good oppertunity to move the projects to V2, so the settings will be done inside the project itself and not in Jenkisn repo. Also, maintainers needs to understand that if fcraw build is failing no other distro will be deployed to tested ( as communicated numerous times already ), so if you're not sure fcraw will work keep it on check-patch only and don't add build-artifacts jobs. > > Best regards, > -- > Didi > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -- Eyal edri MANAGER RHV DevOps EMEA VIRTUALIZATION R&D Red Hat EMEA TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ) -------------- next part -------------- An HTML attachment was scrubbed... URL: From sbonazzo at redhat.com Mon Mar 12 14:44:51 2018 From: sbonazzo at redhat.com (Sandro Bonazzola) Date: Mon, 12 Mar 2018 15:44:51 +0100 Subject: [ovirt-devel] Heads up: dropping fcraw jobs from jenkins In-Reply-To: References: Message-ID: 2018-03-12 15:36 GMT+01:00 Eyal Edri : > > > On Mon, Mar 12, 2018 at 3:11 PM, Yedidyah Bar David > wrote: > >> On Mon, Mar 12, 2018 at 12:50 PM, Nir Soffer wrote: >> > On Thu, Mar 8, 2018 at 4:00 PM Sandro Bonazzola >> wrote: >> >> >> >> Following today call, I'm going to push a patch for dropping all >> rawhide >> >> jobs from jenkins. >> >> This should help avoiding distractions and reducing patches blocked by >> >> rawhide issues. >> > >> > >> > Hi Sandro, >> > >> > I worked hard to get ioprocess and ovirt-imageio working on fcraw, so >> this >> > change will create extra work for me to fix the regressions later when >> we >> > enable fcraw again. >> > >> > Please revert the change for the ovirt-imageio and ioprocess projects. >> >> I'd like too to keep fcraw, for otopi, now pushed >> https://gerrit.ovirt.org/88816 . >> >> Perhaps for the time being it's best if other projects' maintainers do >> the same, if they want to, and commit to fixing issues in fcraw quickly >> (I hope I do...). >> > > Maybe its a good oppertunity to move the projects to V2, so the settings > will be done inside the project itself and not in Jenkisn repo. > Also, maintainers needs to understand that if fcraw build is failing no > other distro will be deployed to tested ( as communicated numerous times > already ), so if you're not sure fcraw will work > keep it on check-patch only and don't add build-artifacts jobs. > Can you point to the documentation for V2? > > >> >> Best regards, >> -- >> Didi >> _______________________________________________ >> Devel mailing list >> Devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel >> > > > > -- > > Eyal edri > > > MANAGER > > RHV DevOps > > EMEA VIRTUALIZATION R&D > > > Red Hat EMEA > TRIED. TESTED. TRUSTED. > phone: +972-9-7692018 <+972%209-769-2018> > irc: eedri (on #tlv #rhev-dev #rhev-integ) > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA TRIED. TESTED. TRUSTED. -------------- next part -------------- An HTML attachment was scrubbed... URL: From eedri at redhat.com Mon Mar 12 14:47:36 2018 From: eedri at redhat.com (Eyal Edri) Date: Mon, 12 Mar 2018 16:47:36 +0200 Subject: [ovirt-devel] Heads up: dropping fcraw jobs from jenkins In-Reply-To: References: Message-ID: Adding Barak and Daniel who can help with it, I think there is initial documentation under the GitHub section on RTD, but more is on the way. On Mon, Mar 12, 2018 at 4:44 PM, Sandro Bonazzola wrote: > > > 2018-03-12 15:36 GMT+01:00 Eyal Edri : > >> >> >> On Mon, Mar 12, 2018 at 3:11 PM, Yedidyah Bar David >> wrote: >> >>> On Mon, Mar 12, 2018 at 12:50 PM, Nir Soffer wrote: >>> > On Thu, Mar 8, 2018 at 4:00 PM Sandro Bonazzola >>> wrote: >>> >> >>> >> Following today call, I'm going to push a patch for dropping all >>> rawhide >>> >> jobs from jenkins. >>> >> This should help avoiding distractions and reducing patches blocked by >>> >> rawhide issues. >>> > >>> > >>> > Hi Sandro, >>> > >>> > I worked hard to get ioprocess and ovirt-imageio working on fcraw, so >>> this >>> > change will create extra work for me to fix the regressions later when >>> we >>> > enable fcraw again. >>> > >>> > Please revert the change for the ovirt-imageio and ioprocess projects. >>> >>> I'd like too to keep fcraw, for otopi, now pushed >>> https://gerrit.ovirt.org/88816 . >>> >>> Perhaps for the time being it's best if other projects' maintainers do >>> the same, if they want to, and commit to fixing issues in fcraw quickly >>> (I hope I do...). >>> >> >> Maybe its a good oppertunity to move the projects to V2, so the settings >> will be done inside the project itself and not in Jenkisn repo. >> Also, maintainers needs to understand that if fcraw build is failing no >> other distro will be deployed to tested ( as communicated numerous times >> already ), so if you're not sure fcraw will work >> keep it on check-patch only and don't add build-artifacts jobs. >> > > Can you point to the documentation for V2? > > >> >> >>> >>> Best regards, >>> -- >>> Didi >>> _______________________________________________ >>> Devel mailing list >>> Devel at ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/devel >>> >> >> >> >> -- >> >> Eyal edri >> >> >> MANAGER >> >> RHV DevOps >> >> EMEA VIRTUALIZATION R&D >> >> >> Red Hat EMEA >> TRIED. TESTED. TRUSTED. >> phone: +972-9-7692018 <+972%209-769-2018> >> irc: eedri (on #tlv #rhev-dev #rhev-integ) >> >> _______________________________________________ >> Devel mailing list >> Devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel >> > > > > -- > > SANDRO BONAZZOLA > > ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D > > Red Hat EMEA > > TRIED. TESTED. TRUSTED. > > -- Eyal edri MANAGER RHV DevOps EMEA VIRTUALIZATION R&D Red Hat EMEA TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ) -------------- next part -------------- An HTML attachment was scrubbed... URL: From michal.skrivanek at redhat.com Mon Mar 12 16:30:01 2018 From: michal.skrivanek at redhat.com (Michal Skrivanek) Date: Mon, 12 Mar 2018 17:30:01 +0100 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 ] [ 002_bootstrap.update_default_cluster ] In-Reply-To: References: Message-ID: <59AA8F90-3BB1-4D62-8237-1856514282C8@redhat.com> > On 11 Mar 2018, at 11:40, Dan Kenigsberg wrote: > > On Sun, Mar 11, 2018 at 12:29 PM, Daniel Belenky wrote: >> Hi, >> >> The following patch failed to pass OST: https://gerrit.ovirt.org/#/c/88738/2 > > it is not efficient to link to the patch without giving its subject and author > db: add func to turn table columns to empty string > by Martin Perina. > it wastes the time of unrelated people, or make them ignore the > (highly appreciated!) OST failure reports. it seems unrelated. Did anyonre try to rerun with or without that patch? > >> Link to failed build: here >> Link to all logs: here >> >> Error snippet from engine.log: >> >> 2018-03-11 06:10:58,521-04 ERROR >> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default >> task-14) [] Operation Failed: [Cannot edit Cluster. The chosen CPU is not >> supported.] no idea what that test is doing. How do I find the corresponding test in OST sources? Thanks, michal >> >> >> Thanks, >> -- >> >> DANIEL BELENKY >> >> RHV DEVOPS >> >> >> _______________________________________________ >> Devel mailing list >> Devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > > From dron at redhat.com Mon Mar 12 16:34:03 2018 From: dron at redhat.com (Dafna Ron) Date: Mon, 12 Mar 2018 16:34:03 +0000 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 ] [ 002_bootstrap.update_default_cluster ] In-Reply-To: <59AA8F90-3BB1-4D62-8237-1856514282C8@redhat.com> References: <59AA8F90-3BB1-4D62-8237-1856514282C8@redhat.com> Message-ID: for now we decided to skip the test: https://gerrit.ovirt.org/#/c/88835/ There is a bz opened on the issue: https://bugzilla.redhat.com/show_bug.cgi?id=1554377 On Mon, Mar 12, 2018 at 4:30 PM, Michal Skrivanek < michal.skrivanek at redhat.com> wrote: > > > > On 11 Mar 2018, at 11:40, Dan Kenigsberg wrote: > > > > On Sun, Mar 11, 2018 at 12:29 PM, Daniel Belenky > wrote: > >> Hi, > >> > >> The following patch failed to pass OST: https://gerrit.ovirt.org/#/c/ > 88738/2 > > > > it is not efficient to link to the patch without giving its subject and > author > > db: add func to turn table columns to empty string > > by Martin Perina. > > it wastes the time of unrelated people, or make them ignore the > > (highly appreciated!) OST failure reports. > > it seems unrelated. Did anyonre try to rerun with or without that patch? > > > > >> Link to failed build: here > >> Link to all logs: here > >> > >> Error snippet from engine.log: > >> > >> 2018-03-11 06:10:58,521-04 ERROR > >> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] > (default > >> task-14) [] Operation Failed: [Cannot edit Cluster. The chosen CPU is > not > >> supported.] > > no idea what that test is doing. How do I find the corresponding test in > OST sources? > > Thanks, > michal > > >> > >> > >> Thanks, > >> -- > >> > >> DANIEL BELENKY > >> > >> RHV DEVOPS > >> > >> > >> _______________________________________________ > >> Devel mailing list > >> Devel at ovirt.org > >> http://lists.ovirt.org/mailman/listinfo/devel > > _______________________________________________ > > Devel mailing list > > Devel at ovirt.org > > http://lists.ovirt.org/mailman/listinfo/devel > > > > > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dron at redhat.com Mon Mar 12 18:24:58 2018 From: dron at redhat.com (Dafna Ron) Date: Mon, 12 Mar 2018 18:24:58 +0000 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 ] [ 002_bootstrap.update_default_cluster ] In-Reply-To: References: <59AA8F90-3BB1-4D62-8237-1856514282C8@redhat.com> Message-ID: We just had a failure in master 002_bootstrap.add_mac_pool with the same error on edit cluster. http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6259/testReport/(root)/002_bootstrap/add_mac_pool/ either I skipped the wrong test or we have a bigger issue. Thanks, Dafna On Mon, Mar 12, 2018 at 4:34 PM, Dafna Ron wrote: > for now we decided to skip the test: https://gerrit.ovirt.org/#/c/88835/ > There is a bz opened on the issue: https://bugzilla.redhat.com/ > show_bug.cgi?id=1554377 > > On Mon, Mar 12, 2018 at 4:30 PM, Michal Skrivanek < > michal.skrivanek at redhat.com> wrote: > >> >> >> > On 11 Mar 2018, at 11:40, Dan Kenigsberg wrote: >> > >> > On Sun, Mar 11, 2018 at 12:29 PM, Daniel Belenky >> wrote: >> >> Hi, >> >> >> >> The following patch failed to pass OST: https://gerrit.ovirt.org/#/c/8 >> 8738/2 >> > >> > it is not efficient to link to the patch without giving its subject and >> author >> > db: add func to turn table columns to empty string >> > by Martin Perina. >> > it wastes the time of unrelated people, or make them ignore the >> > (highly appreciated!) OST failure reports. >> >> it seems unrelated. Did anyonre try to rerun with or without that patch? >> >> > >> >> Link to failed build: here >> >> Link to all logs: here >> >> >> >> Error snippet from engine.log: >> >> >> >> 2018-03-11 06:10:58,521-04 ERROR >> >> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] >> (default >> >> task-14) [] Operation Failed: [Cannot edit Cluster. The chosen CPU is >> not >> >> supported.] >> >> no idea what that test is doing. How do I find the corresponding test in >> OST sources? >> >> Thanks, >> michal >> >> >> >> >> >> >> Thanks, >> >> -- >> >> >> >> DANIEL BELENKY >> >> >> >> RHV DEVOPS >> >> >> >> >> >> _______________________________________________ >> >> Devel mailing list >> >> Devel at ovirt.org >> >> http://lists.ovirt.org/mailman/listinfo/devel >> > _______________________________________________ >> > Devel mailing list >> > Devel at ovirt.org >> > http://lists.ovirt.org/mailman/listinfo/devel >> > >> > >> >> _______________________________________________ >> Devel mailing list >> Devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bkorren at redhat.com Tue Mar 13 06:43:30 2018 From: bkorren at redhat.com (Barak Korren) Date: Tue, 13 Mar 2018 08:43:30 +0200 Subject: [ovirt-devel] Heads up: dropping fcraw jobs from jenkins In-Reply-To: References: Message-ID: On 12 March 2018 at 16:44, Sandro Bonazzola wrote: > > > > > 2018-03-12 15:36 GMT+01:00 Eyal Edri : > >> >> >> On Mon, Mar 12, 2018 at 3:11 PM, Yedidyah Bar David >> wrote: >> >>> On Mon, Mar 12, 2018 at 12:50 PM, Nir Soffer wrote: >>> > On Thu, Mar 8, 2018 at 4:00 PM Sandro Bonazzola >>> wrote: >>> >> >>> >> Following today call, I'm going to push a patch for dropping all >>> rawhide >>> >> jobs from jenkins. >>> >> This should help avoiding distractions and reducing patches blocked by >>> >> rawhide issues. >>> > >>> > >>> > Hi Sandro, >>> > >>> > I worked hard to get ioprocess and ovirt-imageio working on fcraw, so >>> this >>> > change will create extra work for me to fix the regressions later when >>> we >>> > enable fcraw again. >>> > >>> > Please revert the change for the ovirt-imageio and ioprocess projects. >>> >>> I'd like too to keep fcraw, for otopi, now pushed >>> https://gerrit.ovirt.org/88816 . >>> >>> Perhaps for the time being it's best if other projects' maintainers do >>> the same, if they want to, and commit to fixing issues in fcraw quickly >>> (I hope I do...). >>> >> >> Maybe its a good oppertunity to move the projects to V2, so the settings >> will be done inside the project itself and not in Jenkisn repo. >> Also, maintainers needs to understand that if fcraw build is failing no >> other distro will be deployed to tested ( as communicated numerous times >> already ), so if you're not sure fcraw will work >> keep it on check-patch only and don't add build-artifacts jobs. >> > > Can you point to the documentation for V2? > The 1st one is on-the-house ;) Want us to send patches to move otopi to v2? Docs are being worked on but in essence V2 makes the UX for gerrit projects similar to the way things have worked for GitHub projects for a while now [1]. V2 does provide way more then what is documented there (you need more to be able to cover complex scenarios like supporting specific sets of architectures in specific distributions), but what we have there is enough to get the basic ideas and do initial work. [1]: http://ovirt-infra-docs.readthedocs.io/en/latest/CI/Using_STDCI_with_GitHub/#the-automationyaml-file -- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted -------------- next part -------------- An HTML attachment was scrubbed... URL: From didi at redhat.com Tue Mar 13 07:02:34 2018 From: didi at redhat.com (Yedidyah Bar David) Date: Tue, 13 Mar 2018 09:02:34 +0200 Subject: [ovirt-devel] Heads up: dropping fcraw jobs from jenkins In-Reply-To: References: Message-ID: On Tue, Mar 13, 2018 at 8:43 AM, Barak Korren wrote: > > On 12 March 2018 at 16:44, Sandro Bonazzola wrote: >> >> >> >> >> >> 2018-03-12 15:36 GMT+01:00 Eyal Edri : >>> >>> >>> >>> On Mon, Mar 12, 2018 at 3:11 PM, Yedidyah Bar David >>> wrote: >>>> >>>> On Mon, Mar 12, 2018 at 12:50 PM, Nir Soffer wrote: >>>> > On Thu, Mar 8, 2018 at 4:00 PM Sandro Bonazzola >>>> > wrote: >>>> >> >>>> >> Following today call, I'm going to push a patch for dropping all >>>> >> rawhide >>>> >> jobs from jenkins. >>>> >> This should help avoiding distractions and reducing patches blocked >>>> >> by >>>> >> rawhide issues. >>>> > >>>> > >>>> > Hi Sandro, >>>> > >>>> > I worked hard to get ioprocess and ovirt-imageio working on fcraw, so >>>> > this >>>> > change will create extra work for me to fix the regressions later when >>>> > we >>>> > enable fcraw again. >>>> > >>>> > Please revert the change for the ovirt-imageio and ioprocess projects. >>>> >>>> I'd like too to keep fcraw, for otopi, now pushed >>>> https://gerrit.ovirt.org/88816 . >>>> >>>> Perhaps for the time being it's best if other projects' maintainers do >>>> the same, if they want to, and commit to fixing issues in fcraw quickly >>>> (I hope I do...). >>> >>> >>> Maybe its a good oppertunity to move the projects to V2, so the settings >>> will be done inside the project itself and not in Jenkisn repo. >>> Also, maintainers needs to understand that if fcraw build is failing no >>> other distro will be deployed to tested ( as communicated numerous times >>> already ), so if you're not sure fcraw will work >>> keep it on check-patch only and don't add build-artifacts jobs. >> >> >> Can you point to the documentation for V2? > > > The 1st one is on-the-house ;) Want us to send patches to move otopi to v2? You know, patches are always welcome :-)) But I can't promise I'll have too much time to work with you on this. Before continuing, and even reading the docs: Is it possible to remain in V1 for some branches (say, 4.2) and move to V2 for others (master)? Also, if you ask me, now might be a good time to change our naming. While we call our "Build and test standards", _Standards_, I am not aware of any other project or CI system that uses them. 'automation/' was not a good name, as it had high chances to collide with other stuff. Same is for 'automation.yaml'. Perhaps rename at least latter to 'oVirt-CI-conf.yaml' or something like this? Over time, projects will move to V2 and eventually get rid of 'automation/'. Alternatively, find if there are any accepted real standards for these things, and consider adopting them. I am not aware of any, personally, didn't search. > > Docs are being worked on but in essence V2 makes the UX for gerrit projects > similar to the way things have worked for GitHub projects for a while now > [1]. > > V2 does provide way more then what is documented there (you need more to be > able to cover complex scenarios like supporting specific sets of > architectures in specific distributions), but what we have there is enough > to get the basic ideas and do initial work. > > [1]: > http://ovirt-infra-docs.readthedocs.io/en/latest/CI/Using_STDCI_with_GitHub/#the-automationyaml-file > > -- > Barak Korren > RHV DevOps team , RHCE, RHCi > Red Hat EMEA > redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel -- Didi From bkorren at redhat.com Tue Mar 13 07:26:37 2018 From: bkorren at redhat.com (Barak Korren) Date: Tue, 13 Mar 2018 09:26:37 +0200 Subject: [ovirt-devel] Heads up: dropping fcraw jobs from jenkins In-Reply-To: References: Message-ID: On 13 March 2018 at 09:02, Yedidyah Bar David wrote: > On Tue, Mar 13, 2018 at 8:43 AM, Barak Korren wrote: >> >> On 12 March 2018 at 16:44, Sandro Bonazzola wrote: >>> >>> >>> >>> >>> >>> 2018-03-12 15:36 GMT+01:00 Eyal Edri : >>>> >>>> >>>> >>>> On Mon, Mar 12, 2018 at 3:11 PM, Yedidyah Bar David >>>> wrote: >>>>> >>>>> On Mon, Mar 12, 2018 at 12:50 PM, Nir Soffer wrote: >>>>> > On Thu, Mar 8, 2018 at 4:00 PM Sandro Bonazzola >>>>> > wrote: >>>>> >> >>>>> >> Following today call, I'm going to push a patch for dropping all >>>>> >> rawhide >>>>> >> jobs from jenkins. >>>>> >> This should help avoiding distractions and reducing patches blocked >>>>> >> by >>>>> >> rawhide issues. >>>>> > >>>>> > >>>>> > Hi Sandro, >>>>> > >>>>> > I worked hard to get ioprocess and ovirt-imageio working on fcraw, so >>>>> > this >>>>> > change will create extra work for me to fix the regressions later when >>>>> > we >>>>> > enable fcraw again. >>>>> > >>>>> > Please revert the change for the ovirt-imageio and ioprocess projects. >>>>> >>>>> I'd like too to keep fcraw, for otopi, now pushed >>>>> https://gerrit.ovirt.org/88816 . >>>>> >>>>> Perhaps for the time being it's best if other projects' maintainers do >>>>> the same, if they want to, and commit to fixing issues in fcraw quickly >>>>> (I hope I do...). >>>> >>>> >>>> Maybe its a good oppertunity to move the projects to V2, so the settings >>>> will be done inside the project itself and not in Jenkisn repo. >>>> Also, maintainers needs to understand that if fcraw build is failing no >>>> other distro will be deployed to tested ( as communicated numerous times >>>> already ), so if you're not sure fcraw will work >>>> keep it on check-patch only and don't add build-artifacts jobs. >>> >>> >>> Can you point to the documentation for V2? >> >> >> The 1st one is on-the-house ;) Want us to send patches to move otopi to v2? > > You know, patches are always welcome :-)) > > But I can't promise I'll have too much time to work with you on this. > Before continuing, and even reading the docs: Is it possible to remain in > V1 for some branches (say, 4.2) and move to V2 for others (master)? Its possible, as well as having both work side-by-side (and run the same tests/builds twice on every patch). But this will make for a confusing UX... > > Also, if you ask me, now might be a good time to change our naming. > While we call our "Build and test standards", _Standards_, I am not aware > of any other project or CI system that uses them. We're trying to change that. > 'automation/' was not > a good name, as it had high chances to collide with other stuff. Same is > for 'automation.yaml'. Perhaps rename at least latter to 'oVirt-CI-conf.yaml' > or something like this? Over time, projects will move to V2 and eventually > get rid of 'automation/'. V2 supports several other names for the yaml file, the directory name can be customized as well. > > Alternatively, find if there are any accepted real standards for these things, > and consider adopting them. I am not aware of any, personally, didn't search. There are none, everything we`ve found there is a de-facto standard that is hard-wired to a very specific set of implementation concepts and technologies (e.g. .travis.yaml and Jenkinsfile) to my knowledge we`re the only ones that tried to define things in such a way that enables different implementations. > >> >> Docs are being worked on but in essence V2 makes the UX for gerrit projects >> similar to the way things have worked for GitHub projects for a while now >> [1]. >> >> V2 does provide way more then what is documented there (you need more to be >> able to cover complex scenarios like supporting specific sets of >> architectures in specific distributions), but what we have there is enough >> to get the basic ideas and do initial work. >> >> [1]: >> http://ovirt-infra-docs.readthedocs.io/en/latest/CI/Using_STDCI_with_GitHub/#the-automationyaml-file >> >> -- >> Barak Korren >> RHV DevOps team , RHCE, RHCi >> Red Hat EMEA >> redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted >> >> _______________________________________________ >> Devel mailing list >> Devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel > > > > -- > Didi -- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted From danken at redhat.com Tue Mar 13 07:29:35 2018 From: danken at redhat.com (Dan Kenigsberg) Date: Tue, 13 Mar 2018 09:29:35 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 ] [ 002_bootstrap.update_default_cluster ] In-Reply-To: References: <59AA8F90-3BB1-4D62-8237-1856514282C8@redhat.com> Message-ID: On Mon, Mar 12, 2018 at 8:24 PM, Dafna Ron wrote: > We just had a failure in master 002_bootstrap.add_mac_pool with the same > error on edit cluster. > > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6259/testReport/(root)/002_bootstrap/add_mac_pool/ > > either I skipped the wrong test or we have a bigger issue. We certainly do. As before, the error pops up on an attempt to update the cluster (this time it is changing only the mac pool of the cluster). CPU is not specified by the command, so it should not have changed at all. Still, something fills a CPU, and chooses a wrong value. cluster_service.update( cluster=sdk4.types.Cluster( mac_pool=sdk4.types.MacPool( id=pool.id, ) ) ) 2018-03-12 13:58:56,263-04 WARN [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Validation of action 'UpdateCluster' failed for user admin at internal-authz. Reasons: VAR__TYPE__CLUSTER,VAR__ACTION__UPDATE,ACTION_TYPE_FAILED_CPU_NOT_FOUND,VAR__TYPE__CLUSTER 2018-03-12 13:58:56,264-04 INFO [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Lock freed to object 'EngineLock:{exclusiveLocks='[]', sharedLocks='[]'}' 2018-03-12 13:58:56,264-04 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-19) [79d12090-a8e8-438c-bbce-1bf09a42c5fb] method: runAction, params: [UpdateCluster, ManagementNetworkOnClusterOperationParameters:{commandId='bebe80f7-f8ca-4d01-aed8-28e463d0f435', user='null', commandType='Unknown'}], timeElapsed: 50ms 2018-03-12 13:58:56,269-04 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-19) [] Operation Failed: [Cannot edit Cluster. The chosen CPU is not supported.] From eedri at redhat.com Tue Mar 13 08:27:25 2018 From: eedri at redhat.com (Eyal Edri) Date: Tue, 13 Mar 2018 10:27:25 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 ] [ 002_bootstrap.update_default_cluster ] In-Reply-To: References: <59AA8F90-3BB1-4D62-8237-1856514282C8@redhat.com> Message-ID: On Tue, Mar 13, 2018 at 9:29 AM, Dan Kenigsberg wrote: > On Mon, Mar 12, 2018 at 8:24 PM, Dafna Ron wrote: > > We just had a failure in master 002_bootstrap.add_mac_pool with the same > > error on edit cluster. > > > > http://jenkins.ovirt.org/job/ovirt-master_change-queue- > tester/6259/testReport/(root)/002_bootstrap/add_mac_pool/ > > > > either I skipped the wrong test or we have a bigger issue. > > We certainly do. As before, the error pops up on an attempt to update > the cluster (this time it is changing only the mac pool of the > cluster). CPU is not specified by the command, so it should not have > changed at all. Still, something fills a CPU, and chooses a wrong > value. > > cluster_service.update( > cluster=sdk4.types.Cluster( > mac_pool=sdk4.types.MacPool( > id=pool.id, > ) > ) > ) > > 2018-03-12 13:58:56,263-04 WARN > [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) > [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Validation of action > 'UpdateCluster' failed for user admin at internal-authz. Reasons: > VAR__TYPE__CLUSTER,VAR__ACTION__UPDATE,ACTION_TYPE_ > FAILED_CPU_NOT_FOUND,VAR__TYPE__CLUSTER > 2018-03-12 13:58:56,264-04 INFO > [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) > [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Lock freed to object > 'EngineLock:{exclusiveLocks='[]', sharedLocks='[]'}' > 2018-03-12 13:58:56,264-04 DEBUG > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > (default task-19) [79d12090-a8e8-438c-bbce-1bf09a42c5fb] method: > runAction, params: [UpdateCluster, > ManagementNetworkOnClusterOperationParameters:{commandId=' > bebe80f7-f8ca-4d01-aed8-28e463d0f435', > user='null', commandType='Unknown'}], timeElapsed: 50ms > 2018-03-12 13:58:56,269-04 ERROR > [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] > (default task-19) [] Operation Failed: [Cannot edit Cluster. The > chosen CPU is not supported.] > So I guess we can't skip this test as well, and this issue has to be fixed right? > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -- Eyal edri MANAGER RHV DevOps EMEA VIRTUALIZATION R&D Red Hat EMEA TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ) -------------- next part -------------- An HTML attachment was scrubbed... URL: From michal.skrivanek at redhat.com Tue Mar 13 10:57:50 2018 From: michal.skrivanek at redhat.com (Michal Skrivanek) Date: Tue, 13 Mar 2018 11:57:50 +0100 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 ] [ 002_bootstrap.update_default_cluster ] In-Reply-To: References: <59AA8F90-3BB1-4D62-8237-1856514282C8@redhat.com> Message-ID: <3B429D57-6C6C-4559-8D38-87A2658F3905@redhat.com> > On 13 Mar 2018, at 09:27, Eyal Edri wrote: > > > > On Tue, Mar 13, 2018 at 9:29 AM, Dan Kenigsberg > wrote: > On Mon, Mar 12, 2018 at 8:24 PM, Dafna Ron > wrote: > > We just had a failure in master 002_bootstrap.add_mac_pool with the same > > error on edit cluster. Does it fail consistently? Did you narrow down the commit(s) where it started to happen? Was there any other update done at that time? Thanks, michal > > > > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6259/testReport/(root)/002_bootstrap/add_mac_pool/ > > > > either I skipped the wrong test or we have a bigger issue. > > We certainly do. As before, the error pops up on an attempt to update > the cluster (this time it is changing only the mac pool of the > cluster). CPU is not specified by the command, so it should not have > changed at all. Still, something fills a CPU, and chooses a wrong > value. > > cluster_service.update( > cluster=sdk4.types.Cluster( > mac_pool=sdk4.types.MacPool( > id=pool.id , > ) > ) > ) > > 2018-03-12 13:58:56,263-04 WARN > [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) > [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Validation of action > 'UpdateCluster' failed for user admin at internal-authz. Reasons: > VAR__TYPE__CLUSTER,VAR__ACTION__UPDATE,ACTION_TYPE_FAILED_CPU_NOT_FOUND,VAR__TYPE__CLUSTER > 2018-03-12 13:58:56,264-04 INFO > [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) > [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Lock freed to object > 'EngineLock:{exclusiveLocks='[]', sharedLocks='[]'}' > 2018-03-12 13:58:56,264-04 DEBUG > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > (default task-19) [79d12090-a8e8-438c-bbce-1bf09a42c5fb] method: > runAction, params: [UpdateCluster, > ManagementNetworkOnClusterOperationParameters:{commandId='bebe80f7-f8ca-4d01-aed8-28e463d0f435', > user='null', commandType='Unknown'}], timeElapsed: 50ms > 2018-03-12 13:58:56,269-04 ERROR > [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] > (default task-19) [] Operation Failed: [Cannot edit Cluster. The > chosen CPU is not supported.] > > So I guess we can't skip this test as well, and this issue has to be fixed right? > > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > > > > -- > EYAL EDRI > > MANAGER > RHV DEVOPS > EMEA VIRTUALIZATION R&D > > Red Hat?EMEA > TRIED. TESTED. TRUSTED. > phone: +972-9-7692018 > irc: eedri (on #tlv #rhev-dev #rhev-integ) > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From dron at redhat.com Tue Mar 13 16:11:05 2018 From: dron at redhat.com (Dafna Ron) Date: Tue, 13 Mar 2018 16:11:05 +0000 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (otopi) ] [ 13-03-2018 ] [ 002_bootstrap.verify_add_hosts + 002_bootstrap.add_hosts ] Message-ID: Hi, CQ reported failure on both basic and upgrade suites in otpi project. *Link and headline of suspected patches: * *core: Use python3 when possible- https://gerrit.ovirt.org/#/c/87276/ Link to Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6285/ Link to all logs:* http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6285/artifacts/ *(Relevant) error snippet from the log: at org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand.lambda$executeCommand$2(AddVdsCommand.java:217) [bll.jar:] at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:96) [utils.jar:] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_161] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_161] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_161] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_161] at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_161] at org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250) [javax.enterprise.concurrent-1.0.jar:] at org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManagedThreadFactory.java:78)2018-03-13 09:22:40,287-04 ERROR [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [7e847b89] Error during deploy dialog2018-03-13 09:22:40,288-04 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-25) [] Entered SsoRestApiAuthFilter2018-03-13 09:22:40,288-04 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-25) [] SsoRestApiAuthFilter authenticating with sso2018-03-13 09:22:40,288-04 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-25) [] SsoRestApiAuthFilter authenticating using BEARER header2018-03-13 09:22:40,290-04 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-25) [] SsoRestApiAuthFilter successfully authenticated using BEARER header2018-03-13 09:22:40,290-04 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter] (default task-25) [] Entered SsoRestApiNegotiationFilter2018-03-13 09:22:40,292-04 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter] (default task-25) [] SsoRestApiNegotiationFilter Not performing Negotiate Auth2018-03-13 09:22:40,297-04 ERROR [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (EE-ManagedThreadFactory-engine-Thread-2) [7e847b89] Error during host lago-basic-suite-master-host-1 install2018-03-13 09:22:40,311-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-2) [7e847b89] EVENT_ID: VDS_INSTALL_IN_PROGRESS_ERROR(511), An error has occurred during installation of Host lago-basic-suite-master-host-1: Unexpected error during execution: /tmp/ovirt-8LoLZqxUw8/otopi: line 29: /tmp/ovirt-8LoLZqxUw8/otopi-functions: No such file or directory.2018-03-13 09:22:40,311-04 ERROR [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (EE-ManagedThreadFactory-engine-Thread-2) [7e847b89] Error during host lago-basic-suite-master-host-1 install, preferring first exception: Unexpected connection termination2018-03-13 09:22:40,311-04 ERROR [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand] (EE-ManagedThreadFactory-engine-Thread-2) [7e847b89] Host installation failed for host '9cca064c-4034-4428-9cc5-d4e902083dfe', 'lago-basic-suite-master-host-1': Unexpected connection termination* -------------- next part -------------- An HTML attachment was scrubbed... URL: From sbonazzo at redhat.com Tue Mar 13 16:26:04 2018 From: sbonazzo at redhat.com (Sandro Bonazzola) Date: Tue, 13 Mar 2018 17:26:04 +0100 Subject: [ovirt-devel] ovirt-system-tests hackathon report Message-ID: 4 people accepted calendar invite: - Devin A. Bougie - Francesco Romani - Jiri Belka - suporte, logicworks 4 people tentatively accepted calendar invite: - Amnon Maimon - Andreas Bleischwitz - Arnaud Lauriou - Stephen Pesini 2 mailing lists accepted calendar invite: users at ovirt.org, devel at ovirt.org (don't ask me how) so I may have missed someone in above list 4 patches got merged: Add check for host update to the 1st host. Merged Yaniv Kaul ovirt-system-tests master (add_upgrade_check) 4:10 PM basic-suite-master: add vnic_profile_mappings to register vm Merged Eitan Raviv ovirt-system-tests master (register-template-vnic-mapping) 2:50 PM Revert "ovirt-4.2: Skipping 002_bootstrap.update_default_cluster" Merged Eyal Edri ovirt-system-tests master 11:36 AM seperate 4.2 tests and utils from master Merged Eyal Edri ovirt-system-tests master 11:35 AM 13 patches has been pushed / reviewed / rebased Add gdeploy to ovirt-4.2.repo Daniel Belenky ovirt-system-tests master 4:53 PM Cleanup of test code - next() replaced with any() Martin Siv?k ovirt-system-tests master 4:51 PM Add network queues custom property and use it in the vnic profile for VM0 Yaniv Kaul ovirt-system-tests master (multi_queue_config) 4:49 PM new suite: he-basic-iscsi-suite-master Yuval Turgeman ovirt-system-tests master (he-basic-iscsi-suite-master) 4:47 PM Collect host-deploy bundle from the engine Yedidyah Bar David ovirt-system-tests master 4:41 PM network-suite-master: Make openstack_client_config fixture available to all ... Merge Conflict Marcin Mirecki ovirt-system-tests master 3:39 PM new suite: he-basic-ng-ansible-suite-master Sandro Bonazzola ovirt-system-tests master (he-basic-ng-ansible-suite-master) 3:37 PM Enable and move additional tests to 002 Yaniv Kaul ovirt-system-tests master (move_more_to_002) 3:08 PM common: ovirt-4.2.repo Sandro Bonazzola ovirt-system-tests master 2:34 PM networking: Introducing test_stateless_vm_duplicate_mac_addr_vnic_does_not_be... Leon Goldberg ovirt-system-tests master 2:08 PM hc: Updating gdeploy conf to create vdo volumes Sahina Bose ovirt-system-tests master 12:55 PM master: add USB to the sanity VM Michal Skrivanek ovirt-system-tests master 12:54 PM Test hosted-engine cleanup Yedidyah Bar David ovirt-system-tests master 9:39 AM Feedback from the event: - "if we want to add many more tests to OST, and I think we do, we need to do some change there to allow that. Current framework is simply not scalable enough" - not joining the hackathon because "I'd be like an elephant in a porcelain shop" - "I'm not sure I'm OK with the flood of suites that we have - the more we have, the harder it is to sync and maintain but more importantly - to run." - "We can't keep adding new suite for each parameter we want to test, it adds overhead to monitoring, resources and maintenance." - invite wasn't clear enough. I found people on #ovirt on Freenode and on Red Hat IRC servers and redirected them to OFTC IRC server (my fault, hopefully managed to workaround it by talking to people) Lessons learned: - Calendar invites to mailing lists doesn't work well, need a different way to track mailing list members joining the events. - Invites needs to be pedantic on how to join the event, not leaving space for interpretation and misunderstanding. - We need a contribution guide to ovirt-system-test: we need to make people comfortable in trying to add a new test and we need to ensure that we won't reject a day of work because the patch doesn't match core contributors plannings on number of suites, resources and so on - The ovirt-system-tests check patch script is not good enough. It triggers too many sequential suites on every single patch pushed, and fails due to timeout taking more than 6 hours to complete. - The way ovirt-system-test collects rpms from defined repos is not smart enough: it doesn't take the latest version of a given package, just the first found in sequential order of the repos, Thanks everyone who participated to the event! if you have time please continue improving ovirt-system-test even if today event is almost completed! -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA TRIED. TESTED. TRUSTED. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ykaul at redhat.com Tue Mar 13 18:00:05 2018 From: ykaul at redhat.com (Yaniv Kaul) Date: Tue, 13 Mar 2018 20:00:05 +0200 Subject: [ovirt-devel] [ovirt-users] ovirt-system-tests hackathon report In-Reply-To: References: Message-ID: On Mar 13, 2018 6:27 PM, "Sandro Bonazzola" wrote: 4 people accepted calendar invite: - Devin A. Bougie - Francesco Romani - Jiri Belka - suporte, logicworks 4 people tentatively accepted calendar invite: - Amnon Maimon - Andreas Bleischwitz - Arnaud Lauriou - Stephen Pesini 2 mailing lists accepted calendar invite: users at ovirt.org, devel at ovirt.org (don't ask me how) so I may have missed someone in above list 4 patches got merged: Add check for host update to the 1st host. Merged Yaniv Kaul ovirt-system-tests master (add_upgrade_check) 4:10 PM basic-suite-master: add vnic_profile_mappings to register vm Merged Eitan Raviv ovirt-system-tests master (register-template-vnic-mapping) 2:50 PM Revert "ovirt-4.2: Skipping 002_bootstrap.update_default_cluster" Merged Eyal Edri ovirt-system-tests master 11:36 AM seperate 4.2 tests and utils from master Merged Eyal Edri ovirt-system-tests master 11:35 AM 13 patches has been pushed / reviewed / rebased Add gdeploy to ovirt-4.2.repo Daniel Belenky ovirt-system-tests master 4:53 PM Cleanup of test code - next() replaced with any() Martin Siv?k ovirt-system-tests master 4:51 PM Add network queues custom property and use it in the vnic profile for VM0 Yaniv Kaul ovirt-system-tests master (multi_queue_config) 4:49 PM new suite: he-basic-iscsi-suite-master Yuval Turgeman ovirt-system-tests master (he-basic-iscsi-suite-master) 4:47 PM Collect host-deploy bundle from the engine Yedidyah Bar David ovirt-system-tests master 4:41 PM network-suite-master: Make openstack_client_config fixture available to all ... Merge Conflict Marcin Mirecki ovirt-system-tests master 3:39 PM new suite: he-basic-ng-ansible-suite-master Sandro Bonazzola ovirt-system-tests master (he-basic-ng-ansible-suite-master) 3:37 PM Enable and move additional tests to 002 Yaniv Kaul ovirt-system-tests master (move_more_to_002) 3:08 PM common: ovirt-4.2.repo Sandro Bonazzola ovirt-system-tests master 2:34 PM networking: Introducing test_stateless_vm_duplicate_ mac_addr_vnic_does_not_be... Leon Goldberg ovirt-system-tests master 2:08 PM hc: Updating gdeploy conf to create vdo volumes Sahina Bose ovirt-system-tests master 12:55 PM master: add USB to the sanity VM Michal Skrivanek ovirt-system-tests master 12:54 PM Test hosted-engine cleanup Yedidyah Bar David ovirt-system-tests master 9:39 AM Nice list of patches! Feedback from the event: - "if we want to add many more tests to OST, and I think we do, we need to do some change there to allow that. Current framework is simply not scalable enough" More specific and constructive ideas are welcome. We know we want to move to pytest, for example. - not joining the hackathon because "I'd be like an elephant in a porcelain shop" - "I'm not sure I'm OK with the flood of suites that we have - the more we have, the harder it is to sync and maintain but more importantly - to run." - "We can't keep adding new suite for each parameter we want to test, it adds overhead to monitoring, resources and maintenance." I tend to agree, but not sure how to elegantly solve it. - invite wasn't clear enough. I found people on #ovirt on Freenode and on Red Hat IRC servers and redirected them to OFTC IRC server (my fault, hopefully managed to workaround it by talking to people) Lessons learned: - Calendar invites to mailing lists doesn't work well, need a different way to track mailing list members joining the events. - Invites needs to be pedantic on how to join the event, not leaving space for interpretation and misunderstanding. - We need a contribution guide to ovirt-system-test: we need to make people comfortable in trying to add a new test and we need to ensure that we won't reject a day of work because the patch doesn't match core contributors plannings on number of suites, resources and so on Agree. Y. - The ovirt-system-tests check patch script is not good enough. It triggers too many sequential suites on every single patch pushed, and fails due to timeout taking more than 6 hours to complete. - The way ovirt-system-test collects rpms from defined repos is not smart enough: it doesn't take the latest version of a given package, just the first found in sequential order of the repos, Thanks everyone who participated to the event! if you have time please continue improving ovirt-system-test even if today event is almost completed! -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA TRIED. TESTED. TRUSTED. _______________________________________________ Users mailing list Users at ovirt.org http://lists.ovirt.org/mailman/listinfo/users -------------- next part -------------- An HTML attachment was scrubbed... URL: From ykaul at redhat.com Tue Mar 13 18:01:44 2018 From: ykaul at redhat.com (Yaniv Kaul) Date: Tue, 13 Mar 2018 20:01:44 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (otopi) ] [ 13-03-2018 ] [ 002_bootstrap.verify_add_hosts + 002_bootstrap.add_hosts ] In-Reply-To: References: Message-ID: On Mar 13, 2018 6:11 PM, "Dafna Ron" wrote: Hi, CQ reported failure on both basic and upgrade suites in otpi project. *Link and headline of suspected patches: **core: Use python3 when possible- https://gerrit.ovirt.org/#/c/87276/ * Please revert. Y. *Link to Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6285/ Link to all logs:*http://jenkins.ovirt.org/job/ovirt-master_change-queue- tester/6285/artifacts/ *(Relevant) error snippet from the log: at org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand.lambda$executeCommand$2(AddVdsCommand.java:217) [bll.jar:] at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:96) [utils.jar:] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_161] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_161] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_161] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_161] at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_161] at org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250) [javax.enterprise.concurrent-1.0.jar:] at org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManagedThreadFactory.java:78)2018-03-13 09:22:40,287-04 ERROR [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [7e847b89] Error during deploy dialog2018-03-13 09:22:40,288-04 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-25) [] Entered SsoRestApiAuthFilter2018-03-13 09:22:40,288-04 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-25) [] SsoRestApiAuthFilter authenticating with sso2018-03-13 09:22:40,288-04 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-25) [] SsoRestApiAuthFilter authenticating using BEARER header2018-03-13 09:22:40,290-04 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-25) [] SsoRestApiAuthFilter successfully authenticated using BEARER header2018-03-13 09:22:40,290-04 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter] (default task-25) [] Entered SsoRestApiNegotiationFilter2018-03-13 09:22:40,292-04 DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter] (default task-25) [] SsoRestApiNegotiationFilter Not performing Negotiate Auth2018-03-13 09:22:40,297-04 ERROR [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (EE-ManagedThreadFactory-engine-Thread-2) [7e847b89] Error during host lago-basic-suite-master-host-1 install2018-03-13 09:22:40,311-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-2) [7e847b89] EVENT_ID: VDS_INSTALL_IN_PROGRESS_ERROR(511), An error has occurred during installation of Host lago-basic-suite-master-host-1: Unexpected error during execution: /tmp/ovirt-8LoLZqxUw8/otopi: line 29: /tmp/ovirt-8LoLZqxUw8/otopi-functions: No such file or directory.2018-03-13 09:22:40,311-04 ERROR [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (EE-ManagedThreadFactory-engine-Thread-2) [7e847b89] Error during host lago-basic-suite-master-host-1 install, preferring first exception: Unexpected connection termination2018-03-13 09:22:40,311-04 ERROR [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand] (EE-ManagedThreadFactory-engine-Thread-2) [7e847b89] Host installation failed for host '9cca064c-4034-4428-9cc5-d4e902083dfe', 'lago-basic-suite-master-host-1': Unexpected connection termination* _______________________________________________ Devel mailing list Devel at ovirt.org http://lists.ovirt.org/mailman/listinfo/devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From dbelenky at redhat.com Tue Mar 13 18:14:06 2018 From: dbelenky at redhat.com (Daniel Belenky) Date: Tue, 13 Mar 2018 20:14:06 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (otopi) ] [ 13-03-2018 ] [ 002_bootstrap.verify_add_hosts + 002_bootstrap.add_hosts ] In-Reply-To: References: Message-ID: Hi, I've discussed this issue with Didi and Yuval, and I understood that rebuilding ovirt-host-deploy with new otopi should fix this. The last commit in ovirt-host-deploy has been rebuilt and now is being tested in CQ. Since CQ has prevented the change in otopi from making it's way to the tested repo, this failure applies only to changes in otopi project and will not affect others. So maybe we can wait for a proper fix (in case rebuilding ovirt-host-deploy won't solve the issue) instead of reverting the change. On Tue, Mar 13, 2018 at 8:01 PM, Yaniv Kaul wrote: > > > On Mar 13, 2018 6:11 PM, "Dafna Ron" wrote: > > Hi, > > CQ reported failure on both basic and upgrade suites in otpi project. > > *Link and headline of suspected patches: **core: Use python3 when > possible- https://gerrit.ovirt.org/#/c/87276/ > * > > > Please revert. > Y. > > > > > > *Link to > Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6285/ > Link > to all logs:*http://jenkins.ovirt.org/job/ovirt-master_change-queue-teste > r/6285/artifacts/ > > > > > > > > > > > > > > > > > > > > > > > > > > > *(Relevant) error snippet from the log: at > org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand.lambda$executeCommand$2(AddVdsCommand.java:217) > [bll.jar:] at org.ovirt.engine.core.utils.th > readpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:96) > [utils.jar:] at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [rt.jar:1.8.0_161] at > java.util.concurrent.FutureTask.run(FutureTask.java:266) > [rt.jar:1.8.0_161] at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > [rt.jar:1.8.0_161] at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [rt.jar:1.8.0_161] at java.lang.Thread.run(Thread.java:748) > [rt.jar:1.8.0_161] at > org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250) > [javax.enterprise.concurrent-1.0.jar:] at > org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManagedThreadFactory.java:78)2018-03-13 > 09:22:40,287-04 ERROR [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] > (VdsDeploy) [7e847b89] Error during deploy dialog2018-03-13 09:22:40,288-04 > DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default > task-25) [] Entered SsoRestApiAuthFilter2018-03-13 09:22:40,288-04 DEBUG > [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-25) > [] SsoRestApiAuthFilter authenticating with sso2018-03-13 09:22:40,288-04 > DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default > task-25) [] SsoRestApiAuthFilter authenticating using BEARER > header2018-03-13 09:22:40,290-04 DEBUG > [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-25) > [] SsoRestApiAuthFilter successfully authenticated using BEARER > header2018-03-13 09:22:40,290-04 DEBUG > [org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter] (default > task-25) [] Entered SsoRestApiNegotiationFilter2018-03-13 09:22:40,292-04 > DEBUG [org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter] > (default task-25) [] SsoRestApiNegotiationFilter Not performing Negotiate > Auth2018-03-13 09:22:40,297-04 ERROR > [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] > (EE-ManagedThreadFactory-engine-Thread-2) [7e847b89] Error during host > lago-basic-suite-master-host-1 install2018-03-13 09:22:40,311-04 ERROR > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > (EE-ManagedThreadFactory-engine-Thread-2) [7e847b89] EVENT_ID: > VDS_INSTALL_IN_PROGRESS_ERROR(511), An error has occurred during > installation of Host lago-basic-suite-master-host-1: Unexpected error > during execution: /tmp/ovirt-8LoLZqxUw8/otopi: line 29: > /tmp/ovirt-8LoLZqxUw8/otopi-functions: No such file or directory.2018-03-13 > 09:22:40,311-04 ERROR [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] > (EE-ManagedThreadFactory-engine-Thread-2) [7e847b89] Error during host > lago-basic-suite-master-host-1 install, preferring first exception: > Unexpected connection termination2018-03-13 09:22:40,311-04 ERROR > [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand] > (EE-ManagedThreadFactory-engine-Thread-2) [7e847b89] Host installation > failed for host '9cca064c-4034-4428-9cc5-d4e902083dfe', > 'lago-basic-suite-master-host-1': Unexpected connection termination* > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > > > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -- DANIEL BELENKY RHV DEVOPS -------------- next part -------------- An HTML attachment was scrubbed... URL: From eedri at redhat.com Tue Mar 13 19:55:59 2018 From: eedri at redhat.com (Eyal Edri) Date: Tue, 13 Mar 2018 21:55:59 +0200 Subject: [ovirt-devel] [ovirt-users] ovirt-system-tests hackathon report In-Reply-To: References: Message-ID: On Tue, Mar 13, 2018 at 8:00 PM, Yaniv Kaul wrote: > > > On Mar 13, 2018 6:27 PM, "Sandro Bonazzola" wrote: > > 4 people accepted calendar invite: > - Devin A. Bougie > - Francesco Romani > - Jiri Belka > - suporte, logicworks > > 4 people tentatively accepted calendar invite: > - Amnon Maimon > - Andreas Bleischwitz > - Arnaud Lauriou > - Stephen Pesini > > 2 mailing lists accepted calendar invite: users at ovirt.org, devel at ovirt.org > (don't ask me how) so I may have missed someone in above list > > > 4 patches got merged: > Add check for host update to the 1st host. > Merged Yaniv Kaul > > ovirt-system-tests > master > (add_upgrade_check) > 4:10 > PM > basic-suite-master: add vnic_profile_mappings to register vm > Merged Eitan Raviv > > ovirt-system-tests > master > (register-template-vnic-mapping) > 2:50 > PM > Revert "ovirt-4.2: Skipping 002_bootstrap.update_default_cluster" > Merged Eyal Edri > > ovirt-system-tests > > master > 11:36 > AM > seperate 4.2 tests and utils from master > Merged Eyal Edri > > ovirt-system-tests > > master > 11:35 > AM > > 13 patches has been pushed / reviewed / rebased > > Add gdeploy to ovirt-4.2.repo > Daniel Belenky > > ovirt-system-tests > > master > 4:53 > PM > Cleanup of test code - next() replaced with any() > > Martin Siv?k > > ovirt-system-tests > > master > 4:51 > PM > Add network queues custom property and use it in the vnic profile for VM0 > > Yaniv Kaul > > ovirt-system-tests > master > (multi_queue_config) > 4:49 > PM > new suite: he-basic-iscsi-suite-master > Yuval Turgeman > > ovirt-system-tests > master > (he-basic-iscsi-suite-master) > 4:47 > PM > Collect host-deploy bundle from the engine > > Yedidyah Bar David > > ovirt-system-tests > > master > 4:41 > PM > network-suite-master: Make openstack_client_config fixture available to > all ... Merge Conflict Marcin Mirecki > > ovirt-system-tests > > master > 3:39 > PM > new suite: he-basic-ng-ansible-suite-master > > Sandro Bonazzola > > ovirt-system-tests > master > (he-basic-ng-ansible-suite-master) > 3:37 > PM > Enable and move additional tests to 002 > Yaniv Kaul > > ovirt-system-tests > master > (move_more_to_002) > 3:08 > PM > common: ovirt-4.2.repo > Sandro Bonazzola > > ovirt-system-tests > > master > 2:34 > PM > networking: Introducing test_stateless_vm_duplicate_ma > c_addr_vnic_does_not_be... > Leon Goldberg > > ovirt-system-tests > > master > 2:08 > PM > hc: Updating gdeploy conf to create vdo volumes > > Sahina Bose > > ovirt-system-tests > > master > 12:55 > PM > master: add USB to the sanity VM > Michal Skrivanek > > ovirt-system-tests > > master > 12:54 > PM > Test hosted-engine cleanup > Yedidyah Bar David > > ovirt-system-tests > > master > 9:39 > AM > > > Nice list of patches! > > > > Feedback from the event: > - "if we want to add many more tests to OST, and I think we do, we need to > do some change there to allow that. Current framework is simply not > scalable enough" > > > More specific and constructive ideas are welcome. We know we want to move > to pytest, for example. > +1, there is a lot of things we need to improve in OST, the network suite is a good example of going forward with PyTest, we have other idea for improvements like splitting the suites into multiple projects using a new feature in std-ci, dropping the reposync file or making it optional ( actually ovirt-demo-tool is already doing that, using release rpms ). If there are more ideas, we can consider doing an infra hackathon where we can focus in improving infrastructure and moving to PyTest most of the tests ( at least in master ) . > > - not joining the hackathon because "I'd be like an elephant in a > porcelain shop" > - "I'm not sure I'm OK with the flood of suites that we have - the more > we have, the harder it is to sync and maintain but more importantly - to > run." > - "We can't keep adding new suite for each parameter we want to test, it > adds overhead to monitoring, resources and maintenance." > > > I tend to agree, but not sure how to elegantly solve it. > We need to think on a new design for it which includes scalability and multiple maintainers, the requirements and usage has changed significantly over the years, I also don't have an easy solution for it, yet at least. > > - invite wasn't clear enough. I found people on #ovirt on Freenode and on > Red Hat IRC servers and redirected them to OFTC IRC server (my fault, > hopefully managed to workaround it by talking to people) > > > Lessons learned: > - Calendar invites to mailing lists doesn't work well, need a different > way to track mailing list members joining the events. > - Invites needs to be pedantic on how to join the event, not leaving space > for interpretation and misunderstanding. > - We need a contribution guide to ovirt-system-test: we need to make > people comfortable in trying to add a new test and we need to ensure that > we won't reject a day of work because the patch doesn't match core > contributors plannings on number of suites, resources and so on > > This could have been better if the single OST maintainer could be part of the planning and also participate and give feedback and assistance, unfourtunately it was decided to do the hackhaton without him present. I agree we need to improve our contribution guide, and work has started towards it, but any specific feedback or tickets on what can be improved will surely help. > > Agree. > Y. > > - The ovirt-system-tests check patch script is not good enough. It > triggers too many sequential suites on every single patch pushed, and fails > due to timeout taking more than 6 hours to complete. > > This is very close to be much better, we're 1-2 weeks away from implemeting a new feature in STD CI V2 which will allow us to replace the existing 'change resolver' and eventually run all suites in check-patch in parallel, dramatically reducing runtime. > - The way ovirt-system-test collects rpms from defined repos is not smart > enough: it doesn't take the latest version of a given package, just the > first found in sequential order of the repos, > > Its not that its not smart, its by design, repoman uses "only-latest" option to get the first RPM he finds from the repos, I'm pretty sure there was a good reason behind it when it was written a few years ago, I'll try to remember and update. In any case, the relevant patch where this was needed had a different problem with dynamic replacement of repos which we need to think on a solution for. > > Thanks everyone who participated to the event! if you have time please > continue improving ovirt-system-test even if today event is almost > completed! > > > -- > > SANDRO BONAZZOLA > > ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D > > Red Hat EMEA > > TRIED. TESTED. TRUSTED. > > > _______________________________________________ > Users mailing list > Users at ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > > > > _______________________________________________ > Infra mailing list > Infra at ovirt.org > http://lists.ovirt.org/mailman/listinfo/infra > > -- Eyal edri MANAGER RHV DevOps EMEA VIRTUALIZATION R&D Red Hat EMEA TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ) -------------- next part -------------- An HTML attachment was scrubbed... URL: From rgolan at redhat.com Tue Mar 13 20:48:17 2018 From: rgolan at redhat.com (Roy Golan) Date: Tue, 13 Mar 2018 20:48:17 +0000 Subject: [ovirt-devel] [ovirt-users] ovirt-system-tests hackathon report In-Reply-To: References: Message-ID: I missed it :( indeed calendar invite to the list didn't work. On Tue, 13 Mar 2018 at 21:58 Eyal Edri wrote: > On Tue, Mar 13, 2018 at 8:00 PM, Yaniv Kaul wrote: > >> >> >> On Mar 13, 2018 6:27 PM, "Sandro Bonazzola" wrote: >> >> 4 people accepted calendar invite: >> - Devin A. Bougie >> - Francesco Romani >> - Jiri Belka >> - suporte, logicworks >> >> 4 people tentatively accepted calendar invite: >> - Amnon Maimon >> - Andreas Bleischwitz >> - Arnaud Lauriou >> - Stephen Pesini >> >> 2 mailing lists accepted calendar invite: users at ovirt.org, >> devel at ovirt.org (don't ask me how) so I may have missed someone in above >> list >> >> >> 4 patches got merged: >> Add check for host update to the 1st host. >> Merged Yaniv Kaul >> >> ovirt-system-tests >> master >> (add_upgrade_check) >> 4:10 >> PM >> basic-suite-master: add vnic_profile_mappings to register vm >> Merged Eitan Raviv >> >> ovirt-system-tests >> master >> (register-template-vnic-mapping) >> 2:50 >> PM >> Revert "ovirt-4.2: Skipping 002_bootstrap.update_default_cluster" >> Merged Eyal Edri >> >> ovirt-system-tests >> >> master >> 11:36 >> AM >> seperate 4.2 tests and utils from master >> Merged Eyal Edri >> >> ovirt-system-tests >> >> master >> 11:35 >> AM >> >> 13 patches has been pushed / reviewed / rebased >> >> Add gdeploy to ovirt-4.2.repo >> Daniel Belenky >> >> ovirt-system-tests >> >> master >> 4:53 >> PM >> Cleanup of test code - next() replaced with any() >> >> Martin Siv?k >> >> ovirt-system-tests >> >> master >> 4:51 >> PM >> Add network queues custom property and use it in the vnic profile for VM0 >> >> Yaniv Kaul >> >> ovirt-system-tests >> master >> (multi_queue_config) >> 4:49 >> PM >> new suite: he-basic-iscsi-suite-master >> Yuval Turgeman >> >> ovirt-system-tests >> master >> (he-basic-iscsi-suite-master) >> 4:47 >> PM >> Collect host-deploy bundle from the engine >> >> Yedidyah Bar David >> >> ovirt-system-tests >> >> master >> 4:41 >> PM >> network-suite-master: Make openstack_client_config fixture available to >> all ... Merge Conflict Marcin Mirecki >> >> ovirt-system-tests >> >> master >> 3:39 >> PM >> new suite: he-basic-ng-ansible-suite-master >> >> Sandro Bonazzola >> >> ovirt-system-tests >> master >> (he-basic-ng-ansible-suite-master) >> 3:37 >> PM >> Enable and move additional tests to 002 >> Yaniv Kaul >> >> ovirt-system-tests >> master >> (move_more_to_002) >> 3:08 >> PM >> common: ovirt-4.2.repo >> Sandro Bonazzola >> >> ovirt-system-tests >> >> master >> 2:34 >> PM >> networking: Introducing >> test_stateless_vm_duplicate_mac_addr_vnic_does_not_be... >> >> Leon Goldberg >> >> ovirt-system-tests >> >> master >> 2:08 >> PM >> hc: Updating gdeploy conf to create vdo volumes >> >> Sahina Bose >> >> ovirt-system-tests >> >> master >> 12:55 >> PM >> master: add USB to the sanity VM >> Michal Skrivanek >> >> ovirt-system-tests >> >> master >> 12:54 >> PM >> Test hosted-engine cleanup >> Yedidyah Bar David >> >> ovirt-system-tests >> >> master >> 9:39 >> AM >> >> >> Nice list of patches! >> >> >> >> Feedback from the event: >> - "if we want to add many more tests to OST, and I think we do, we need >> to do some change there to allow that. Current framework is simply not >> scalable enough" >> >> >> More specific and constructive ideas are welcome. We know we want to move >> to pytest, for example. >> > > +1, there is a lot of things we need to improve in OST, the network suite > is a good example of going forward with PyTest, we have other idea for > improvements like splitting the suites into multiple projects using a new > feature in std-ci, > dropping the reposync file or making it optional ( actually > ovirt-demo-tool is already doing that, using release rpms ). > > If there are more ideas, we can consider doing an infra hackathon where we > can focus in improving infrastructure and moving to PyTest most of the > tests ( at least in master ) . > > >> >> - not joining the hackathon because "I'd be like an elephant in a >> porcelain shop" >> - "I'm not sure I'm OK with the flood of suites that we have - the more >> we have, the harder it is to sync and maintain but more importantly - to >> run." >> - "We can't keep adding new suite for each parameter we want to test, it >> adds overhead to monitoring, resources and maintenance." >> >> >> I tend to agree, but not sure how to elegantly solve it. >> > > We need to think on a new design for it which includes scalability and > multiple maintainers, the requirements and usage has changed significantly > over the years, > I also don't have an easy solution for it, yet at least. > > >> >> - invite wasn't clear enough. I found people on #ovirt on Freenode and on >> Red Hat IRC servers and redirected them to OFTC IRC server (my fault, >> hopefully managed to workaround it by talking to people) >> >> >> Lessons learned: >> - Calendar invites to mailing lists doesn't work well, need a different >> way to track mailing list members joining the events. >> - Invites needs to be pedantic on how to join the event, not leaving >> space for interpretation and misunderstanding. >> - We need a contribution guide to ovirt-system-test: we need to make >> people comfortable in trying to add a new test and we need to ensure that >> we won't reject a day of work because the patch doesn't match core >> contributors plannings on number of suites, resources and so on >> >> > This could have been better if the single OST maintainer could be part of > the planning and also participate and give feedback and assistance, > unfourtunately it was decided to do the hackhaton without him present. > I agree we need to improve our contribution guide, and work has started > towards it, but any specific feedback or tickets on what can be improved > will surely help. > > >> >> Agree. >> Y. >> >> - The ovirt-system-tests check patch script is not good enough. It >> triggers too many sequential suites on every single patch pushed, and fails >> due to timeout taking more than 6 hours to complete. >> >> > This is very close to be much better, we're 1-2 weeks away from > implemeting a new feature in STD CI V2 which will allow us to replace the > existing 'change resolver' and eventually run all suites in check-patch in > parallel, dramatically reducing runtime. > > >> - The way ovirt-system-test collects rpms from defined repos is not smart >> enough: it doesn't take the latest version of a given package, just the >> first found in sequential order of the repos, >> >> > Its not that its not smart, its by design, repoman uses "only-latest" > option to get the first RPM he finds from the repos, I'm pretty sure there > was a good reason behind it when it was written a few years ago, I'll try > to remember and update. > In any case, the relevant patch where this was needed had a different > problem with dynamic replacement of repos which we need to think on a > solution for. > > >> >> Thanks everyone who participated to the event! if you have time please >> continue improving ovirt-system-test even if today event is almost >> completed! >> >> >> -- >> >> SANDRO BONAZZOLA >> >> ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D >> >> Red Hat EMEA >> >> TRIED. TESTED. TRUSTED. >> >> >> _______________________________________________ >> Users mailing list >> Users at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/users >> >> >> >> _______________________________________________ >> Infra mailing list >> Infra at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/infra >> >> > > > -- > > Eyal edri > > > MANAGER > > RHV DevOps > > EMEA VIRTUALIZATION R&D > > > Red Hat EMEA > TRIED. TESTED. TRUSTED. > phone: +972-9-7692018 <+972%209-769-2018> > irc: eedri (on #tlv #rhev-dev #rhev-integ) > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From dron at redhat.com Tue Mar 13 21:24:59 2018 From: dron at redhat.com (Dafna Ron) Date: Tue, 13 Mar 2018 21:24:59 +0000 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 ] [ 002_bootstrap.update_default_cluster ] In-Reply-To: <3B429D57-6C6C-4559-8D38-87A2658F3905@redhat.com> References: <59AA8F90-3BB1-4D62-8237-1856514282C8@redhat.com> <3B429D57-6C6C-4559-8D38-87A2658F3905@redhat.com> Message-ID: On Tue, Mar 13, 2018 at 10:57 AM, Michal Skrivanek < michal.skrivanek at redhat.com> wrote: > > > On 13 Mar 2018, at 09:27, Eyal Edri wrote: > > > > On Tue, Mar 13, 2018 at 9:29 AM, Dan Kenigsberg wrote: > >> On Mon, Mar 12, 2018 at 8:24 PM, Dafna Ron wrote: >> > We just had a failure in master 002_bootstrap.add_mac_pool with the same >> > error on edit cluster. >> > > Does it fail consistently? > yes. > Did you narrow down the commit(s) where it started to happen? > First change reported failed by CQ on this issue is this one: https://gerrit.ovirt.org/#/c/88738/2 - db: add func to turn table columns to empty string (this was reported by Daniel at the beginning of this thread) Was there any other update done at that time? > > there are always other changes submitted. but CQ tries to isolate the change that it believes is causing the failure by reducing the change it tests until it gets to one single change. We cannot have OST keep failing for a long time, especially on a big project like ovirt-engine. if we cannot have a fix on this quickly I think we should start skipping failed tests to allow changes to pass successfully until the bug is fixed. Thanks, Dafna > Thanks, > michal > > > >> > http://jenkins.ovirt.org/job/ovirt-master_change-queue-teste >> r/6259/testReport/(root)/002_bootstrap/add_mac_pool/ >> > >> > either I skipped the wrong test or we have a bigger issue. >> >> We certainly do. As before, the error pops up on an attempt to update >> the cluster (this time it is changing only the mac pool of the >> cluster). CPU is not specified by the command, so it should not have >> changed at all. Still, something fills a CPU, and chooses a wrong >> value. >> >> cluster_service.update( >> cluster=sdk4.types.Cluster( >> mac_pool=sdk4.types.MacPool( >> id=pool.id, >> ) >> ) >> ) >> >> 2018-03-12 13:58:56,263-04 WARN >> [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) >> [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Validation of action >> 'UpdateCluster' failed for user admin at internal-authz. Reasons: >> VAR__TYPE__CLUSTER,VAR__ACTION__UPDATE,ACTION_TYPE_FAILED_ >> CPU_NOT_FOUND,VAR__TYPE__CLUSTER >> 2018-03-12 13:58:56,264-04 INFO >> [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) >> [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Lock freed to object >> 'EngineLock:{exclusiveLocks='[]', sharedLocks='[]'}' >> 2018-03-12 13:58:56,264-04 DEBUG >> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >> (default task-19) [79d12090-a8e8-438c-bbce-1bf09a42c5fb] method: >> runAction, params: [UpdateCluster, >> ManagementNetworkOnClusterOperationParameters:{commandId='be >> be80f7-f8ca-4d01-aed8-28e463d0f435', >> user='null', commandType='Unknown'}], timeElapsed: 50ms >> 2018-03-12 13:58:56,269-04 ERROR >> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] >> (default task-19) [] Operation Failed: [Cannot edit Cluster. The >> chosen CPU is not supported.] >> > > So I guess we can't skip this test as well, and this issue has to be fixed > right? > > > >> _______________________________________________ >> Devel mailing list >> Devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel >> > > > > -- > Eyal edri > > MANAGER > > RHV DevOps > > EMEA VIRTUALIZATION R&D > > > Red Hat EMEA > TRIED. TESTED. TRUSTED. > phone: +972-9-7692018 <+972%209-769-2018> > irc: eedri (on #tlv #rhev-dev #rhev-integ) > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michal.skrivanek at redhat.com Tue Mar 13 21:32:09 2018 From: michal.skrivanek at redhat.com (Michal Skrivanek) Date: Tue, 13 Mar 2018 22:32:09 +0100 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 ] [ 002_bootstrap.update_default_cluster ] In-Reply-To: References: <59AA8F90-3BB1-4D62-8237-1856514282C8@redhat.com> <3B429D57-6C6C-4559-8D38-87A2658F3905@redhat.com> Message-ID: <7CFE1871-70D5-4A5A-98A7-F4D0CD576D67@redhat.com> > On 13 Mar 2018, at 22:24, Dafna Ron wrote: > > > > On Tue, Mar 13, 2018 at 10:57 AM, Michal Skrivanek > wrote: > > >> On 13 Mar 2018, at 09:27, Eyal Edri > wrote: >> >> >> >> On Tue, Mar 13, 2018 at 9:29 AM, Dan Kenigsberg > wrote: >> On Mon, Mar 12, 2018 at 8:24 PM, Dafna Ron > wrote: >> > We just had a failure in master 002_bootstrap.add_mac_pool with the same >> > error on edit cluster. > > Does it fail consistently? > > yes. that?s good > > > Did you narrow down the commit(s) where it started to happen? > > First change reported failed by CQ on this issue is this one: https://gerrit.ovirt.org/#/c/88738/2 > - db: add func to turn table columns to empty string (this was reported by Daniel at the beginning of this thread) > > Was there any other update done at that time? > > there are always other changes submitted. but CQ tries to isolate the change that it believes is causing the failure by reducing the change it tests until it gets to one single change. I meant changes like major update of packages or any other configuration change > > > We cannot have OST keep failing for a long time, especially on a big project like ovirt-engine. if we cannot have a fix on this quickly I think we should start skipping failed tests to allow changes to pass successfully until the bug is fixed. sure. But in this case you?re just going to hit the same problem in the next test. Please enable back the one you commented out, and try to revert that patch instead. There is a chance it changed the behavior because somehow the tests using Default cluster somehow rely on undefined values (not sure if that?s even intentional, but that?s the way it is written), and that patch may have changed it perhaps. Eli? Thanks, michal > > Thanks, > Dafna > > > Thanks, > michal > >> > >> > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6259/testReport/(root)/002_bootstrap/add_mac_pool/ >> > >> > either I skipped the wrong test or we have a bigger issue. >> >> We certainly do. As before, the error pops up on an attempt to update >> the cluster (this time it is changing only the mac pool of the >> cluster). CPU is not specified by the command, so it should not have >> changed at all. Still, something fills a CPU, and chooses a wrong >> value. >> >> cluster_service.update( >> cluster=sdk4.types.Cluster( >> mac_pool=sdk4.types.MacPool( >> id=pool.id , >> ) >> ) >> ) >> >> 2018-03-12 13:58:56,263-04 WARN >> [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) >> [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Validation of action >> 'UpdateCluster' failed for user admin at internal-authz. Reasons: >> VAR__TYPE__CLUSTER,VAR__ACTION__UPDATE,ACTION_TYPE_FAILED_CPU_NOT_FOUND,VAR__TYPE__CLUSTER >> 2018-03-12 13:58:56,264-04 INFO >> [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) >> [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Lock freed to object >> 'EngineLock:{exclusiveLocks='[]', sharedLocks='[]'}' >> 2018-03-12 13:58:56,264-04 DEBUG >> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >> (default task-19) [79d12090-a8e8-438c-bbce-1bf09a42c5fb] method: >> runAction, params: [UpdateCluster, >> ManagementNetworkOnClusterOperationParameters:{commandId='bebe80f7-f8ca-4d01-aed8-28e463d0f435', >> user='null', commandType='Unknown'}], timeElapsed: 50ms >> 2018-03-12 13:58:56,269-04 ERROR >> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] >> (default task-19) [] Operation Failed: [Cannot edit Cluster. The >> chosen CPU is not supported.] >> >> So I guess we can't skip this test as well, and this issue has to be fixed right? >> >> >> _______________________________________________ >> Devel mailing list >> Devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel >> >> >> >> -- >> EYAL EDRI >> >> MANAGER >> RHV DEVOPS >> EMEA VIRTUALIZATION R&D >> >> Red Hat?EMEA >> TRIED. TESTED. TRUSTED. >> phone: +972-9-7692018 >> irc: eedri (on #tlv #rhev-dev #rhev-integ) >> _______________________________________________ >> Devel mailing list >> Devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dron at redhat.com Tue Mar 13 21:49:10 2018 From: dron at redhat.com (Dafna Ron) Date: Tue, 13 Mar 2018 21:49:10 +0000 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 ] [ 002_bootstrap.update_default_cluster ] In-Reply-To: <7CFE1871-70D5-4A5A-98A7-F4D0CD576D67@redhat.com> References: <59AA8F90-3BB1-4D62-8237-1856514282C8@redhat.com> <3B429D57-6C6C-4559-8D38-87A2658F3905@redhat.com> <7CFE1871-70D5-4A5A-98A7-F4D0CD576D67@redhat.com> Message-ID: On Tue, Mar 13, 2018 at 9:32 PM, Michal Skrivanek < michal.skrivanek at redhat.com> wrote: > > > On 13 Mar 2018, at 22:24, Dafna Ron wrote: > > > > On Tue, Mar 13, 2018 at 10:57 AM, Michal Skrivanek < > michal.skrivanek at redhat.com> wrote: > >> >> >> On 13 Mar 2018, at 09:27, Eyal Edri wrote: >> >> >> >> On Tue, Mar 13, 2018 at 9:29 AM, Dan Kenigsberg >> wrote: >> >>> On Mon, Mar 12, 2018 at 8:24 PM, Dafna Ron wrote: >>> > We just had a failure in master 002_bootstrap.add_mac_pool with the >>> same >>> > error on edit cluster. >>> >> >> Does it fail consistently? >> > > yes. > > > that?s good > > > > >> Did you narrow down the commit(s) where it started to happen? >> > > First change reported failed by CQ on this issue is this one: > https://gerrit.ovirt.org/#/c/88738/2 > - db: add func to turn table columns to empty string (this was reported > by Daniel at the beginning of this thread) > > Was there any other update done at that time? >> >> there are always other changes submitted. but CQ tries to isolate the > change that it believes is causing the failure by reducing the change it > tests until it gets to one single change. > > > I meant changes like major update of packages or any other configuration > change > :) the last one I saw was from Eli on Friday but I don't think its related. since this is a big project and there a lot of changes submitted daily, maybe someone more qualified them me can have a look and see if anything catches their eyes? > > > We cannot have OST keep failing for a long time, especially on a big > project like ovirt-engine. if we cannot have a fix on this quickly I think > we should start skipping failed tests to allow changes to pass successfully > until the bug is fixed. > > > sure. But in this case you?re just going to hit the same problem in the > next test. Please enable back the one you commented out, and try to revert > that patch instead. There is a chance it changed the behavior because > somehow the tests using Default cluster somehow rely on undefined values > (not sure if that?s even intentional, but that?s the way it is written), > and that patch may have changed it perhaps. Eli? > > cool. I think that Eyal has reverted my skip test so we can try to revert the change reported. > Thanks, > michal > > > Thanks, > Dafna > > > >> Thanks, >> michal >> >> > >>> > http://jenkins.ovirt.org/job/ovirt-master_change-queue-teste >>> r/6259/testReport/(root)/002_bootstrap/add_mac_pool/ >>> > >>> > either I skipped the wrong test or we have a bigger issue. >>> >>> We certainly do. As before, the error pops up on an attempt to update >>> the cluster (this time it is changing only the mac pool of the >>> cluster). CPU is not specified by the command, so it should not have >>> changed at all. Still, something fills a CPU, and chooses a wrong >>> value. >>> >>> cluster_service.update( >>> cluster=sdk4.types.Cluster( >>> mac_pool=sdk4.types.MacPool( >>> id=pool.id, >>> ) >>> ) >>> ) >>> >>> 2018-03-12 13:58:56,263-04 WARN >>> [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) >>> [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Validation of action >>> 'UpdateCluster' failed for user admin at internal-authz. Reasons: >>> VAR__TYPE__CLUSTER,VAR__ACTION__UPDATE,ACTION_TYPE_FAILED_CP >>> U_NOT_FOUND,VAR__TYPE__CLUSTER >>> 2018-03-12 13:58:56,264-04 INFO >>> [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) >>> [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Lock freed to object >>> 'EngineLock:{exclusiveLocks='[]', sharedLocks='[]'}' >>> 2018-03-12 13:58:56,264-04 DEBUG >>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >>> (default task-19) [79d12090-a8e8-438c-bbce-1bf09a42c5fb] method: >>> runAction, params: [UpdateCluster, >>> ManagementNetworkOnClusterOperationParameters:{commandId='be >>> be80f7-f8ca-4d01-aed8-28e463d0f435', >>> user='null', commandType='Unknown'}], timeElapsed: 50ms >>> 2018-03-12 13:58:56,269-04 ERROR >>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] >>> (default task-19) [] Operation Failed: [Cannot edit Cluster. The >>> chosen CPU is not supported.] >>> >> >> So I guess we can't skip this test as well, and this issue has to be >> fixed right? >> >> >> >>> _______________________________________________ >>> Devel mailing list >>> Devel at ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/devel >>> >> >> >> >> -- >> Eyal edri >> >> MANAGER >> >> RHV DevOps >> >> EMEA VIRTUALIZATION R&D >> >> >> Red Hat EMEA >> TRIED. TESTED. TRUSTED. >> phone: +972-9-7692018 <+972%209-769-2018> >> irc: eedri (on #tlv #rhev-dev #rhev-integ) >> _______________________________________________ >> Devel mailing list >> Devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel >> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From didi at redhat.com Wed Mar 14 06:30:40 2018 From: didi at redhat.com (Yedidyah Bar David) Date: Wed, 14 Mar 2018 08:30:40 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (otopi) ] [ 13-03-2018 ] [ 002_bootstrap.verify_add_hosts + 002_bootstrap.add_hosts ] In-Reply-To: References: Message-ID: On Tue, Mar 13, 2018 at 8:14 PM, Daniel Belenky wrote: > Hi, > > I've discussed this issue with Didi and Yuval, and I understood that > rebuilding ovirt-host-deploy with new otopi should fix this. > The last commit in ovirt-host-deploy has been rebuilt and now is being > tested in CQ. > Since CQ has prevented the change in otopi from making it's way to the > tested repo, this failure applies only to changes in otopi project and will > not affect others. > So maybe we can wait for a proper fix (in case rebuilding ovirt-host-deploy > won't solve the issue) instead of reverting the change. > > On Tue, Mar 13, 2018 at 8:01 PM, Yaniv Kaul wrote: >> >> >> >> On Mar 13, 2018 6:11 PM, "Dafna Ron" wrote: >> >> Hi, >> >> CQ reported failure on both basic and upgrade suites in otpi project. >> >> Link and headline of suspected patches: >> >> core: Use python3 when possible- https://gerrit.ovirt.org/#/c/87276/ >> >> >> Please revert. >> Y. >> >> >> >> Link to Job: >> >> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6285/ >> >> Link to all logs: >> >> >> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6285/artifacts/ >> >> (Relevant) error snippet from the log: >> >> >> >> >> at >> org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand.lambda$executeCommand$2(AddVdsCommand.java:217) >> [bll.jar:] >> at >> org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:96) >> [utils.jar:] >> at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >> [rt.jar:1.8.0_161] >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> [rt.jar:1.8.0_161] >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >> [rt.jar:1.8.0_161] >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >> [rt.jar:1.8.0_161] >> at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_161] >> at >> org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250) >> [javax.enterprise.concurrent-1.0.jar:] >> at >> org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManagedThreadFactory.java:78) >> >> 2018-03-13 09:22:40,287-04 ERROR >> [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [7e847b89] >> Error during deploy dialog >> 2018-03-13 09:22:40,288-04 DEBUG >> [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-25) >> [] Entered SsoRestApiAuthFilter >> 2018-03-13 09:22:40,288-04 DEBUG >> [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-25) >> [] SsoRestApiAuthFilter authenticating with sso >> 2018-03-13 09:22:40,288-04 DEBUG >> [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-25) >> [] SsoRestApiAuthFilter authenticating using BEARER header >> 2018-03-13 09:22:40,290-04 DEBUG >> [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-25) >> [] SsoRestApiAuthFilter successfully authenticated using BEARER header >> 2018-03-13 09:22:40,290-04 DEBUG >> [org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter] (default >> task-25) [] Entered SsoRestApiNegotiationFilter >> 2018-03-13 09:22:40,292-04 DEBUG >> [org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter] (default >> task-25) [] SsoRestApiNegotiationFilter Not performing Negotiate Auth >> 2018-03-13 09:22:40,297-04 ERROR >> [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] >> (EE-ManagedThreadFactory-engine-Thread-2) [7e847b89] Error during host >> lago-basic-suite-master-host-1 install >> 2018-03-13 09:22:40,311-04 ERROR >> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >> (EE-ManagedThreadFactory-engine-Thread-2) [7e847b89] EVENT_ID: >> VDS_INSTALL_IN_PROGRESS_ERROR(511), An error has occurred during >> installation of Host lago-basic-suite-master-host-1: Unexpected error during >> execution: /tmp/ovirt-8LoLZqxUw8/otopi: line 29: >> /tmp/ovirt-8LoLZqxUw8/otopi-functions: No such file or directory >> . >> 2018-03-13 09:22:40,311-04 ERROR >> [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] >> (EE-ManagedThreadFactory-engine-Thread-2) [7e847b89] Error during host >> lago-basic-suite-master-host-1 install, preferring first exception: >> Unexpected connection termination >> 2018-03-13 09:22:40,311-04 ERROR >> [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand] >> (EE-ManagedThreadFactory-engine-Thread-2) [7e847b89] Host installation >> failed for host '9cca064c-4034-4428-9cc5-d4e902083dfe', >> 'lago-basic-suite-master-host-1': Unexpected connection termination >> Should be fixed (for now) by: https://gerrit.ovirt.org/88931 We still might revert eventually, need to analyze upgrades from 4.2 to 4.3. >> >> >> >> >> >> _______________________________________________ >> Devel mailing list >> Devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel >> >> >> >> _______________________________________________ >> Devel mailing list >> Devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel > > > > > -- > > DANIEL BELENKY > > RHV DEVOPS > > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel -- Didi From dbelenky at redhat.com Wed Mar 14 07:22:52 2018 From: dbelenky at redhat.com (Daniel Belenky) Date: Wed, 14 Mar 2018 09:22:52 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (otopi) ] [ 13-03-2018 ] [ 002_bootstrap.verify_add_hosts + 002_bootstrap.add_hosts ] In-Reply-To: References: Message-ID: The last fix mentioned by Didi indeed fixed the problem. I've re-enqueued the original patch to otopi. It was successfully tested and deployed by the CI. On Wed, Mar 14, 2018 at 8:30 AM, Yedidyah Bar David wrote: > On Tue, Mar 13, 2018 at 8:14 PM, Daniel Belenky > wrote: > > Hi, > > > > I've discussed this issue with Didi and Yuval, and I understood that > > rebuilding ovirt-host-deploy with new otopi should fix this. > > The last commit in ovirt-host-deploy has been rebuilt and now is being > > tested in CQ. > > Since CQ has prevented the change in otopi from making it's way to the > > tested repo, this failure applies only to changes in otopi project and > will > > not affect others. > > So maybe we can wait for a proper fix (in case rebuilding > ovirt-host-deploy > > won't solve the issue) instead of reverting the change. > > > > On Tue, Mar 13, 2018 at 8:01 PM, Yaniv Kaul wrote: > >> > >> > >> > >> On Mar 13, 2018 6:11 PM, "Dafna Ron" wrote: > >> > >> Hi, > >> > >> CQ reported failure on both basic and upgrade suites in otpi project. > >> > >> Link and headline of suspected patches: > >> > >> core: Use python3 when possible- https://gerrit.ovirt.org/#/c/87276/ > >> > >> > >> Please revert. > >> Y. > >> > >> > >> > >> Link to Job: > >> > >> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6285/ > >> > >> Link to all logs: > >> > >> > >> http://jenkins.ovirt.org/job/ovirt-master_change-queue- > tester/6285/artifacts/ > >> > >> (Relevant) error snippet from the log: > >> > >> > >> > >> > >> at > >> org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand. > lambda$executeCommand$2(AddVdsCommand.java:217) > >> [bll.jar:] > >> at > >> org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$ > InternalWrapperRunnable.run(ThreadPoolUtil.java:96) > >> [utils.jar:] > >> at > >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > >> [rt.jar:1.8.0_161] > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) > >> [rt.jar:1.8.0_161] > >> at > >> java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1149) > >> [rt.jar:1.8.0_161] > >> at > >> java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:624) > >> [rt.jar:1.8.0_161] > >> at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_161] > >> at > >> org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ > ManagedThread.run(ManagedThreadFactoryImpl.java:250) > >> [javax.enterprise.concurrent-1.0.jar:] > >> at > >> org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ > ElytronManagedThread.run(ElytronManagedThreadFactory.java:78) > >> > >> 2018-03-13 09:22:40,287-04 ERROR > >> [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) > [7e847b89] > >> Error during deploy dialog > >> 2018-03-13 09:22:40,288-04 DEBUG > >> [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default > task-25) > >> [] Entered SsoRestApiAuthFilter > >> 2018-03-13 09:22:40,288-04 DEBUG > >> [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default > task-25) > >> [] SsoRestApiAuthFilter authenticating with sso > >> 2018-03-13 09:22:40,288-04 DEBUG > >> [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default > task-25) > >> [] SsoRestApiAuthFilter authenticating using BEARER header > >> 2018-03-13 09:22:40,290-04 DEBUG > >> [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default > task-25) > >> [] SsoRestApiAuthFilter successfully authenticated using BEARER header > >> 2018-03-13 09:22:40,290-04 DEBUG > >> [org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter] > (default > >> task-25) [] Entered SsoRestApiNegotiationFilter > >> 2018-03-13 09:22:40,292-04 DEBUG > >> [org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter] > (default > >> task-25) [] SsoRestApiNegotiationFilter Not performing Negotiate Auth > >> 2018-03-13 09:22:40,297-04 ERROR > >> [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] > >> (EE-ManagedThreadFactory-engine-Thread-2) [7e847b89] Error during host > >> lago-basic-suite-master-host-1 install > >> 2018-03-13 09:22:40,311-04 ERROR > >> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > >> (EE-ManagedThreadFactory-engine-Thread-2) [7e847b89] EVENT_ID: > >> VDS_INSTALL_IN_PROGRESS_ERROR(511), An error has occurred during > >> installation of Host lago-basic-suite-master-host-1: Unexpected error > during > >> execution: /tmp/ovirt-8LoLZqxUw8/otopi: line 29: > >> /tmp/ovirt-8LoLZqxUw8/otopi-functions: No such file or directory > >> . > >> 2018-03-13 09:22:40,311-04 ERROR > >> [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] > >> (EE-ManagedThreadFactory-engine-Thread-2) [7e847b89] Error during host > >> lago-basic-suite-master-host-1 install, preferring first exception: > >> Unexpected connection termination > >> 2018-03-13 09:22:40,311-04 ERROR > >> [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand] > >> (EE-ManagedThreadFactory-engine-Thread-2) [7e847b89] Host installation > >> failed for host '9cca064c-4034-4428-9cc5-d4e902083dfe', > >> 'lago-basic-suite-master-host-1': Unexpected connection termination > >> > > Should be fixed (for now) by: > > https://gerrit.ovirt.org/88931 > > We still might revert eventually, need to analyze upgrades from 4.2 to 4.3. > > >> > >> > >> > >> > >> > >> _______________________________________________ > >> Devel mailing list > >> Devel at ovirt.org > >> http://lists.ovirt.org/mailman/listinfo/devel > >> > >> > >> > >> _______________________________________________ > >> Devel mailing list > >> Devel at ovirt.org > >> http://lists.ovirt.org/mailman/listinfo/devel > > > > > > > > > > -- > > > > DANIEL BELENKY > > > > RHV DEVOPS > > > > > > _______________________________________________ > > Devel mailing list > > Devel at ovirt.org > > http://lists.ovirt.org/mailman/listinfo/devel > > > > -- > Didi > -- DANIEL BELENKY RHV DEVOPS -------------- next part -------------- An HTML attachment was scrubbed... URL: From dron at redhat.com Wed Mar 14 16:46:00 2018 From: dron at redhat.com (Dafna Ron) Date: Wed, 14 Mar 2018 16:46:00 +0000 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 ] [ 002_bootstrap.update_default_cluster ] In-Reply-To: References: <59AA8F90-3BB1-4D62-8237-1856514282C8@redhat.com> <3B429D57-6C6C-4559-8D38-87A2658F3905@redhat.com> <7CFE1871-70D5-4A5A-98A7-F4D0CD576D67@redhat.com> Message-ID: Eli updated the bug with a fix that reverts parts of the reported patch: https://gerrit.ovirt.org/#/c/89005/ waiting for verification and merge. Thanks! Dafna On Tue, Mar 13, 2018 at 9:49 PM, Dafna Ron wrote: > > > On Tue, Mar 13, 2018 at 9:32 PM, Michal Skrivanek < > michal.skrivanek at redhat.com> wrote: > >> >> >> On 13 Mar 2018, at 22:24, Dafna Ron wrote: >> >> >> >> On Tue, Mar 13, 2018 at 10:57 AM, Michal Skrivanek < >> michal.skrivanek at redhat.com> wrote: >> >>> >>> >>> On 13 Mar 2018, at 09:27, Eyal Edri wrote: >>> >>> >>> >>> On Tue, Mar 13, 2018 at 9:29 AM, Dan Kenigsberg >>> wrote: >>> >>>> On Mon, Mar 12, 2018 at 8:24 PM, Dafna Ron wrote: >>>> > We just had a failure in master 002_bootstrap.add_mac_pool with the >>>> same >>>> > error on edit cluster. >>>> >>> >>> Does it fail consistently? >>> >> >> yes. >> >> >> that?s good >> >> >> >> >>> Did you narrow down the commit(s) where it started to happen? >>> >> >> First change reported failed by CQ on this issue is this one: >> https://gerrit.ovirt.org/#/c/88738/2 >> - db: add func to turn table columns to empty string (this was reported >> by Daniel at the beginning of this thread) >> >> Was there any other update done at that time? >>> >>> there are always other changes submitted. but CQ tries to isolate the >> change that it believes is causing the failure by reducing the change it >> tests until it gets to one single change. >> >> >> I meant changes like major update of packages or any other configuration >> change >> > > :) the last one I saw was from Eli on Friday but I don't think its > related. since this is a big project and there a lot of changes submitted > daily, maybe someone more qualified them me can have a look and see if > anything catches their eyes? > >> >> >> We cannot have OST keep failing for a long time, especially on a big >> project like ovirt-engine. if we cannot have a fix on this quickly I think >> we should start skipping failed tests to allow changes to pass successfully >> until the bug is fixed. >> >> >> sure. But in this case you?re just going to hit the same problem in the >> next test. Please enable back the one you commented out, and try to revert >> that patch instead. There is a chance it changed the behavior because >> somehow the tests using Default cluster somehow rely on undefined values >> (not sure if that?s even intentional, but that?s the way it is written), >> and that patch may have changed it perhaps. Eli? >> >> cool. I think that Eyal has reverted my skip test so we can try to revert > the change reported. > > >> Thanks, >> michal >> >> >> Thanks, >> Dafna >> >> >> >>> Thanks, >>> michal >>> >>> > >>>> > http://jenkins.ovirt.org/job/ovirt-master_change-queue-teste >>>> r/6259/testReport/(root)/002_bootstrap/add_mac_pool/ >>>> > >>>> > either I skipped the wrong test or we have a bigger issue. >>>> >>>> We certainly do. As before, the error pops up on an attempt to update >>>> the cluster (this time it is changing only the mac pool of the >>>> cluster). CPU is not specified by the command, so it should not have >>>> changed at all. Still, something fills a CPU, and chooses a wrong >>>> value. >>>> >>>> cluster_service.update( >>>> cluster=sdk4.types.Cluster( >>>> mac_pool=sdk4.types.MacPool( >>>> id=pool.id, >>>> ) >>>> ) >>>> ) >>>> >>>> 2018-03-12 13:58:56,263-04 WARN >>>> [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) >>>> [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Validation of action >>>> 'UpdateCluster' failed for user admin at internal-authz. Reasons: >>>> VAR__TYPE__CLUSTER,VAR__ACTION__UPDATE,ACTION_TYPE_FAILED_CP >>>> U_NOT_FOUND,VAR__TYPE__CLUSTER >>>> 2018-03-12 13:58:56,264-04 INFO >>>> [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) >>>> [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Lock freed to object >>>> 'EngineLock:{exclusiveLocks='[]', sharedLocks='[]'}' >>>> 2018-03-12 13:58:56,264-04 DEBUG >>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >>>> (default task-19) [79d12090-a8e8-438c-bbce-1bf09a42c5fb] method: >>>> runAction, params: [UpdateCluster, >>>> ManagementNetworkOnClusterOperationParameters:{commandId='be >>>> be80f7-f8ca-4d01-aed8-28e463d0f435', >>>> user='null', commandType='Unknown'}], timeElapsed: 50ms >>>> 2018-03-12 13:58:56,269-04 ERROR >>>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] >>>> (default task-19) [] Operation Failed: [Cannot edit Cluster. The >>>> chosen CPU is not supported.] >>>> >>> >>> So I guess we can't skip this test as well, and this issue has to be >>> fixed right? >>> >>> >>> >>>> _______________________________________________ >>>> Devel mailing list >>>> Devel at ovirt.org >>>> http://lists.ovirt.org/mailman/listinfo/devel >>>> >>> >>> >>> >>> -- >>> Eyal edri >>> >>> MANAGER >>> >>> RHV DevOps >>> >>> EMEA VIRTUALIZATION R&D >>> >>> >>> Red Hat EMEA >>> TRIED. TESTED. TRUSTED. >>> >>> phone: +972-9-7692018 <+972%209-769-2018> >>> irc: eedri (on #tlv #rhev-dev #rhev-integ) >>> _______________________________________________ >>> Devel mailing list >>> Devel at ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/devel >>> >>> >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tnisan at redhat.com Thu Mar 15 08:11:15 2018 From: tnisan at redhat.com (Tal Nisan) Date: Thu, 15 Mar 2018 10:11:15 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 ] [ 002_bootstrap.update_default_cluster ] In-Reply-To: References: <59AA8F90-3BB1-4D62-8237-1856514282C8@redhat.com> <3B429D57-6C6C-4559-8D38-87A2658F3905@redhat.com> <7CFE1871-70D5-4A5A-98A7-F4D0CD576D67@redhat.com> Message-ID: I've reviewed and marked +1, I'll need another reviewer though for this matter. I've also based one of my patches on top of it and it passed OST: http://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_manual/2387/ Dafna, prior to Eli's patch all OST jobs failed? On Wed, Mar 14, 2018 at 6:46 PM, Dafna Ron wrote: > Eli updated the bug with a fix that reverts parts of the reported patch: > https://gerrit.ovirt.org/#/c/89005/ > > waiting for verification and merge. > > Thanks! > Dafna > > > > > On Tue, Mar 13, 2018 at 9:49 PM, Dafna Ron wrote: > >> >> >> On Tue, Mar 13, 2018 at 9:32 PM, Michal Skrivanek < >> michal.skrivanek at redhat.com> wrote: >> >>> >>> >>> On 13 Mar 2018, at 22:24, Dafna Ron wrote: >>> >>> >>> >>> On Tue, Mar 13, 2018 at 10:57 AM, Michal Skrivanek < >>> michal.skrivanek at redhat.com> wrote: >>> >>>> >>>> >>>> On 13 Mar 2018, at 09:27, Eyal Edri wrote: >>>> >>>> >>>> >>>> On Tue, Mar 13, 2018 at 9:29 AM, Dan Kenigsberg >>>> wrote: >>>> >>>>> On Mon, Mar 12, 2018 at 8:24 PM, Dafna Ron wrote: >>>>> > We just had a failure in master 002_bootstrap.add_mac_pool with the >>>>> same >>>>> > error on edit cluster. >>>>> >>>> >>>> Does it fail consistently? >>>> >>> >>> yes. >>> >>> >>> that?s good >>> >>> >>> >>> >>>> Did you narrow down the commit(s) where it started to happen? >>>> >>> >>> First change reported failed by CQ on this issue is this one: >>> https://gerrit.ovirt.org/#/c/88738/2 >>> - db: add func to turn table columns to empty string (this was >>> reported by Daniel at the beginning of this thread) >>> >>> Was there any other update done at that time? >>>> >>>> there are always other changes submitted. but CQ tries to isolate the >>> change that it believes is causing the failure by reducing the change it >>> tests until it gets to one single change. >>> >>> >>> I meant changes like major update of packages or any other configuration >>> change >>> >> >> :) the last one I saw was from Eli on Friday but I don't think its >> related. since this is a big project and there a lot of changes submitted >> daily, maybe someone more qualified them me can have a look and see if >> anything catches their eyes? >> >>> >>> >>> We cannot have OST keep failing for a long time, especially on a big >>> project like ovirt-engine. if we cannot have a fix on this quickly I think >>> we should start skipping failed tests to allow changes to pass successfully >>> until the bug is fixed. >>> >>> >>> sure. But in this case you?re just going to hit the same problem in the >>> next test. Please enable back the one you commented out, and try to revert >>> that patch instead. There is a chance it changed the behavior because >>> somehow the tests using Default cluster somehow rely on undefined values >>> (not sure if that?s even intentional, but that?s the way it is written), >>> and that patch may have changed it perhaps. Eli? >>> >>> cool. I think that Eyal has reverted my skip test so we can try to >> revert the change reported. >> >> >>> Thanks, >>> michal >>> >>> >>> Thanks, >>> Dafna >>> >>> >>> >>>> Thanks, >>>> michal >>>> >>>> > >>>>> > http://jenkins.ovirt.org/job/ovirt-master_change-queue-teste >>>>> r/6259/testReport/(root)/002_bootstrap/add_mac_pool/ >>>>> > >>>>> > either I skipped the wrong test or we have a bigger issue. >>>>> >>>>> We certainly do. As before, the error pops up on an attempt to update >>>>> the cluster (this time it is changing only the mac pool of the >>>>> cluster). CPU is not specified by the command, so it should not have >>>>> changed at all. Still, something fills a CPU, and chooses a wrong >>>>> value. >>>>> >>>>> cluster_service.update( >>>>> cluster=sdk4.types.Cluster( >>>>> mac_pool=sdk4.types.MacPool( >>>>> id=pool.id, >>>>> ) >>>>> ) >>>>> ) >>>>> >>>>> 2018-03-12 13:58:56,263-04 WARN >>>>> [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) >>>>> [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Validation of action >>>>> 'UpdateCluster' failed for user admin at internal-authz. Reasons: >>>>> VAR__TYPE__CLUSTER,VAR__ACTION__UPDATE,ACTION_TYPE_FAILED_CP >>>>> U_NOT_FOUND,VAR__TYPE__CLUSTER >>>>> 2018-03-12 13:58:56,264-04 INFO >>>>> [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) >>>>> [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Lock freed to object >>>>> 'EngineLock:{exclusiveLocks='[]', sharedLocks='[]'}' >>>>> 2018-03-12 13:58:56,264-04 DEBUG >>>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >>>>> (default task-19) [79d12090-a8e8-438c-bbce-1bf09a42c5fb] method: >>>>> runAction, params: [UpdateCluster, >>>>> ManagementNetworkOnClusterOperationParameters:{commandId='be >>>>> be80f7-f8ca-4d01-aed8-28e463d0f435', >>>>> user='null', commandType='Unknown'}], timeElapsed: 50ms >>>>> 2018-03-12 13:58:56,269-04 ERROR >>>>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] >>>>> (default task-19) [] Operation Failed: [Cannot edit Cluster. The >>>>> chosen CPU is not supported.] >>>>> >>>> >>>> So I guess we can't skip this test as well, and this issue has to be >>>> fixed right? >>>> >>>> >>>> >>>>> _______________________________________________ >>>>> Devel mailing list >>>>> Devel at ovirt.org >>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>> >>>> >>>> >>>> >>>> -- >>>> Eyal edri >>>> >>>> MANAGER >>>> >>>> RHV DevOps >>>> >>>> EMEA VIRTUALIZATION R&D >>>> >>>> >>>> Red Hat EMEA >>>> TRIED. TESTED. TRUSTED. >>>> >>>> phone: +972-9-7692018 <+972%209-769-2018> >>>> irc: eedri (on #tlv #rhev-dev #rhev-integ) >>>> _______________________________________________ >>>> Devel mailing list >>>> Devel at ovirt.org >>>> http://lists.ovirt.org/mailman/listinfo/devel >>>> >>>> >>>> >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tnisan at redhat.com Thu Mar 15 09:18:33 2018 From: tnisan at redhat.com (Tal Nisan) Date: Thu, 15 Mar 2018 11:18:33 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 ] [ 002_bootstrap.update_default_cluster ] In-Reply-To: References: <59AA8F90-3BB1-4D62-8237-1856514282C8@redhat.com> <3B429D57-6C6C-4559-8D38-87A2658F3905@redhat.com> <7CFE1871-70D5-4A5A-98A7-F4D0CD576D67@redhat.com> Message-ID: Thanks to the reviewers, merged on master now. Working with Dafna on getting it fixed on 4.2 and understanding whether 4.1.10 is affected (probably the most important question as we've already built it and it should be shipped to customers). On Thu, Mar 15, 2018 at 10:11 AM, Tal Nisan wrote: > I've reviewed and marked +1, I'll need another reviewer though for this > matter. > I've also based one of my patches on top of it and it passed OST: > http://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ > ovirt-system-tests_manual/2387/ > > Dafna, prior to Eli's patch all OST jobs failed? > > > On Wed, Mar 14, 2018 at 6:46 PM, Dafna Ron wrote: > >> Eli updated the bug with a fix that reverts parts of the reported patch: >> https://gerrit.ovirt.org/#/c/89005/ >> >> waiting for verification and merge. >> >> Thanks! >> Dafna >> >> >> >> >> On Tue, Mar 13, 2018 at 9:49 PM, Dafna Ron wrote: >> >>> >>> >>> On Tue, Mar 13, 2018 at 9:32 PM, Michal Skrivanek < >>> michal.skrivanek at redhat.com> wrote: >>> >>>> >>>> >>>> On 13 Mar 2018, at 22:24, Dafna Ron wrote: >>>> >>>> >>>> >>>> On Tue, Mar 13, 2018 at 10:57 AM, Michal Skrivanek < >>>> michal.skrivanek at redhat.com> wrote: >>>> >>>>> >>>>> >>>>> On 13 Mar 2018, at 09:27, Eyal Edri wrote: >>>>> >>>>> >>>>> >>>>> On Tue, Mar 13, 2018 at 9:29 AM, Dan Kenigsberg >>>>> wrote: >>>>> >>>>>> On Mon, Mar 12, 2018 at 8:24 PM, Dafna Ron wrote: >>>>>> > We just had a failure in master 002_bootstrap.add_mac_pool with the >>>>>> same >>>>>> > error on edit cluster. >>>>>> >>>>> >>>>> Does it fail consistently? >>>>> >>>> >>>> yes. >>>> >>>> >>>> that?s good >>>> >>>> >>>> >>>> >>>>> Did you narrow down the commit(s) where it started to happen? >>>>> >>>> >>>> First change reported failed by CQ on this issue is this one: >>>> https://gerrit.ovirt.org/#/c/88738/2 >>>> - db: add func to turn table columns to empty string (this was >>>> reported by Daniel at the beginning of this thread) >>>> >>>> Was there any other update done at that time? >>>>> >>>>> there are always other changes submitted. but CQ tries to isolate the >>>> change that it believes is causing the failure by reducing the change it >>>> tests until it gets to one single change. >>>> >>>> >>>> I meant changes like major update of packages or any other >>>> configuration change >>>> >>> >>> :) the last one I saw was from Eli on Friday but I don't think its >>> related. since this is a big project and there a lot of changes submitted >>> daily, maybe someone more qualified them me can have a look and see if >>> anything catches their eyes? >>> >>>> >>>> >>>> We cannot have OST keep failing for a long time, especially on a big >>>> project like ovirt-engine. if we cannot have a fix on this quickly I think >>>> we should start skipping failed tests to allow changes to pass successfully >>>> until the bug is fixed. >>>> >>>> >>>> sure. But in this case you?re just going to hit the same problem in the >>>> next test. Please enable back the one you commented out, and try to revert >>>> that patch instead. There is a chance it changed the behavior because >>>> somehow the tests using Default cluster somehow rely on undefined values >>>> (not sure if that?s even intentional, but that?s the way it is written), >>>> and that patch may have changed it perhaps. Eli? >>>> >>>> cool. I think that Eyal has reverted my skip test so we can try to >>> revert the change reported. >>> >>> >>>> Thanks, >>>> michal >>>> >>>> >>>> Thanks, >>>> Dafna >>>> >>>> >>>> >>>>> Thanks, >>>>> michal >>>>> >>>>> > >>>>>> > http://jenkins.ovirt.org/job/ovirt-master_change-queue-teste >>>>>> r/6259/testReport/(root)/002_bootstrap/add_mac_pool/ >>>>>> > >>>>>> > either I skipped the wrong test or we have a bigger issue. >>>>>> >>>>>> We certainly do. As before, the error pops up on an attempt to update >>>>>> the cluster (this time it is changing only the mac pool of the >>>>>> cluster). CPU is not specified by the command, so it should not have >>>>>> changed at all. Still, something fills a CPU, and chooses a wrong >>>>>> value. >>>>>> >>>>>> cluster_service.update( >>>>>> cluster=sdk4.types.Cluster( >>>>>> mac_pool=sdk4.types.MacPool( >>>>>> id=pool.id, >>>>>> ) >>>>>> ) >>>>>> ) >>>>>> >>>>>> 2018-03-12 13:58:56,263-04 WARN >>>>>> [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) >>>>>> [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Validation of action >>>>>> 'UpdateCluster' failed for user admin at internal-authz. Reasons: >>>>>> VAR__TYPE__CLUSTER,VAR__ACTION__UPDATE,ACTION_TYPE_FAILED_CP >>>>>> U_NOT_FOUND,VAR__TYPE__CLUSTER >>>>>> 2018-03-12 13:58:56,264-04 INFO >>>>>> [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) >>>>>> [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Lock freed to object >>>>>> 'EngineLock:{exclusiveLocks='[]', sharedLocks='[]'}' >>>>>> 2018-03-12 13:58:56,264-04 DEBUG >>>>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >>>>>> (default task-19) [79d12090-a8e8-438c-bbce-1bf09a42c5fb] method: >>>>>> runAction, params: [UpdateCluster, >>>>>> ManagementNetworkOnClusterOperationParameters:{commandId='be >>>>>> be80f7-f8ca-4d01-aed8-28e463d0f435', >>>>>> user='null', commandType='Unknown'}], timeElapsed: 50ms >>>>>> 2018-03-12 13:58:56,269-04 ERROR >>>>>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] >>>>>> (default task-19) [] Operation Failed: [Cannot edit Cluster. The >>>>>> chosen CPU is not supported.] >>>>>> >>>>> >>>>> So I guess we can't skip this test as well, and this issue has to be >>>>> fixed right? >>>>> >>>>> >>>>> >>>>>> _______________________________________________ >>>>>> Devel mailing list >>>>>> Devel at ovirt.org >>>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Eyal edri >>>>> >>>>> MANAGER >>>>> >>>>> RHV DevOps >>>>> >>>>> EMEA VIRTUALIZATION R&D >>>>> >>>>> >>>>> Red Hat EMEA >>>>> TRIED. TESTED. TRUSTED. >>>>> >>>>> phone: +972-9-7692018 <+972%209-769-2018> >>>>> irc: eedri (on #tlv #rhev-dev #rhev-integ) >>>>> _______________________________________________ >>>>> Devel mailing list >>>>> Devel at ovirt.org >>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>> >>>>> >>>>> >>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tnisan at redhat.com Thu Mar 15 09:26:51 2018 From: tnisan at redhat.com (Tal Nisan) Date: Thu, 15 Mar 2018 11:26:51 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 ] [ 002_bootstrap.update_default_cluster ] In-Reply-To: References: <59AA8F90-3BB1-4D62-8237-1856514282C8@redhat.com> <3B429D57-6C6C-4559-8D38-87A2658F3905@redhat.com> <7CFE1871-70D5-4A5A-98A7-F4D0CD576D67@redhat.com> Message-ID: I've backported to 4.2, kind reviewers from master please review as well - https://gerrit.ovirt.org/#/c/89035/ On Thu, Mar 15, 2018 at 11:18 AM, Tal Nisan wrote: > Thanks to the reviewers, merged on master now. > Working with Dafna on getting it fixed on 4.2 and understanding whether > 4.1.10 is affected (probably the most important question as we've already > built it and it should be shipped to customers). > > > On Thu, Mar 15, 2018 at 10:11 AM, Tal Nisan wrote: > >> I've reviewed and marked +1, I'll need another reviewer though for this >> matter. >> I've also based one of my patches on top of it and it passed OST: >> http://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovi >> rt-system-tests_manual/2387/ >> >> Dafna, prior to Eli's patch all OST jobs failed? >> >> >> On Wed, Mar 14, 2018 at 6:46 PM, Dafna Ron wrote: >> >>> Eli updated the bug with a fix that reverts parts of the reported patch: >>> https://gerrit.ovirt.org/#/c/89005/ >>> >>> waiting for verification and merge. >>> >>> Thanks! >>> Dafna >>> >>> >>> >>> >>> On Tue, Mar 13, 2018 at 9:49 PM, Dafna Ron wrote: >>> >>>> >>>> >>>> On Tue, Mar 13, 2018 at 9:32 PM, Michal Skrivanek < >>>> michal.skrivanek at redhat.com> wrote: >>>> >>>>> >>>>> >>>>> On 13 Mar 2018, at 22:24, Dafna Ron wrote: >>>>> >>>>> >>>>> >>>>> On Tue, Mar 13, 2018 at 10:57 AM, Michal Skrivanek < >>>>> michal.skrivanek at redhat.com> wrote: >>>>> >>>>>> >>>>>> >>>>>> On 13 Mar 2018, at 09:27, Eyal Edri wrote: >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Mar 13, 2018 at 9:29 AM, Dan Kenigsberg >>>>>> wrote: >>>>>> >>>>>>> On Mon, Mar 12, 2018 at 8:24 PM, Dafna Ron wrote: >>>>>>> > We just had a failure in master 002_bootstrap.add_mac_pool with >>>>>>> the same >>>>>>> > error on edit cluster. >>>>>>> >>>>>> >>>>>> Does it fail consistently? >>>>>> >>>>> >>>>> yes. >>>>> >>>>> >>>>> that?s good >>>>> >>>>> >>>>> >>>>> >>>>>> Did you narrow down the commit(s) where it started to happen? >>>>>> >>>>> >>>>> First change reported failed by CQ on this issue is this one: >>>>> https://gerrit.ovirt.org/#/c/88738/2 >>>>> - db: add func to turn table columns to empty string (this was >>>>> reported by Daniel at the beginning of this thread) >>>>> >>>>> Was there any other update done at that time? >>>>>> >>>>>> there are always other changes submitted. but CQ tries to isolate the >>>>> change that it believes is causing the failure by reducing the change it >>>>> tests until it gets to one single change. >>>>> >>>>> >>>>> I meant changes like major update of packages or any other >>>>> configuration change >>>>> >>>> >>>> :) the last one I saw was from Eli on Friday but I don't think its >>>> related. since this is a big project and there a lot of changes submitted >>>> daily, maybe someone more qualified them me can have a look and see if >>>> anything catches their eyes? >>>> >>>>> >>>>> >>>>> We cannot have OST keep failing for a long time, especially on a big >>>>> project like ovirt-engine. if we cannot have a fix on this quickly I think >>>>> we should start skipping failed tests to allow changes to pass successfully >>>>> until the bug is fixed. >>>>> >>>>> >>>>> sure. But in this case you?re just going to hit the same problem in >>>>> the next test. Please enable back the one you commented out, and try to >>>>> revert that patch instead. There is a chance it changed the behavior >>>>> because somehow the tests using Default cluster somehow rely on undefined >>>>> values (not sure if that?s even intentional, but that?s the way it is >>>>> written), and that patch may have changed it perhaps. Eli? >>>>> >>>>> cool. I think that Eyal has reverted my skip test so we can try to >>>> revert the change reported. >>>> >>>> >>>>> Thanks, >>>>> michal >>>>> >>>>> >>>>> Thanks, >>>>> Dafna >>>>> >>>>> >>>>> >>>>>> Thanks, >>>>>> michal >>>>>> >>>>>> > >>>>>>> > http://jenkins.ovirt.org/job/ovirt-master_change-queue-teste >>>>>>> r/6259/testReport/(root)/002_bootstrap/add_mac_pool/ >>>>>>> > >>>>>>> > either I skipped the wrong test or we have a bigger issue. >>>>>>> >>>>>>> We certainly do. As before, the error pops up on an attempt to update >>>>>>> the cluster (this time it is changing only the mac pool of the >>>>>>> cluster). CPU is not specified by the command, so it should not have >>>>>>> changed at all. Still, something fills a CPU, and chooses a wrong >>>>>>> value. >>>>>>> >>>>>>> cluster_service.update( >>>>>>> cluster=sdk4.types.Cluster( >>>>>>> mac_pool=sdk4.types.MacPool( >>>>>>> id=pool.id, >>>>>>> ) >>>>>>> ) >>>>>>> ) >>>>>>> >>>>>>> 2018-03-12 13:58:56,263-04 WARN >>>>>>> [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) >>>>>>> [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Validation of action >>>>>>> 'UpdateCluster' failed for user admin at internal-authz. Reasons: >>>>>>> VAR__TYPE__CLUSTER,VAR__ACTION__UPDATE,ACTION_TYPE_FAILED_CP >>>>>>> U_NOT_FOUND,VAR__TYPE__CLUSTER >>>>>>> 2018-03-12 13:58:56,264-04 INFO >>>>>>> [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) >>>>>>> [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Lock freed to object >>>>>>> 'EngineLock:{exclusiveLocks='[]', sharedLocks='[]'}' >>>>>>> 2018-03-12 13:58:56,264-04 DEBUG >>>>>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInt >>>>>>> erceptor] >>>>>>> (default task-19) [79d12090-a8e8-438c-bbce-1bf09a42c5fb] method: >>>>>>> runAction, params: [UpdateCluster, >>>>>>> ManagementNetworkOnClusterOperationParameters:{commandId='be >>>>>>> be80f7-f8ca-4d01-aed8-28e463d0f435', >>>>>>> user='null', commandType='Unknown'}], timeElapsed: 50ms >>>>>>> 2018-03-12 13:58:56,269-04 ERROR >>>>>>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] >>>>>>> (default task-19) [] Operation Failed: [Cannot edit Cluster. The >>>>>>> chosen CPU is not supported.] >>>>>>> >>>>>> >>>>>> So I guess we can't skip this test as well, and this issue has to be >>>>>> fixed right? >>>>>> >>>>>> >>>>>> >>>>>>> _______________________________________________ >>>>>>> Devel mailing list >>>>>>> Devel at ovirt.org >>>>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Eyal edri >>>>>> >>>>>> MANAGER >>>>>> >>>>>> RHV DevOps >>>>>> >>>>>> EMEA VIRTUALIZATION R&D >>>>>> >>>>>> >>>>>> Red Hat EMEA >>>>>> TRIED. TESTED. TRUSTED. >>>>>> >>>>>> phone: +972-9-7692018 <+972%209-769-2018> >>>>>> irc: eedri (on #tlv #rhev-dev #rhev-integ) >>>>>> _______________________________________________ >>>>>> Devel mailing list >>>>>> Devel at ovirt.org >>>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From danken at redhat.com Thu Mar 15 10:49:21 2018 From: danken at redhat.com (Dan Kenigsberg) Date: Thu, 15 Mar 2018 12:49:21 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 ] [ 002_bootstrap.update_default_cluster ] In-Reply-To: References: <59AA8F90-3BB1-4D62-8237-1856514282C8@redhat.com> <3B429D57-6C6C-4559-8D38-87A2658F3905@redhat.com> <7CFE1871-70D5-4A5A-98A7-F4D0CD576D67@redhat.com> Message-ID: And I've started http://jenkins.ovirt.org/job/ovirt-system-tests_manual/2392/console to see if it OST passes there. On Thu, Mar 15, 2018 at 11:26 AM, Tal Nisan wrote: > I've backported to 4.2, kind reviewers from master please review as well - > https://gerrit.ovirt.org/#/c/89035/ > > > > On Thu, Mar 15, 2018 at 11:18 AM, Tal Nisan wrote: > >> Thanks to the reviewers, merged on master now. >> Working with Dafna on getting it fixed on 4.2 and understanding whether >> 4.1.10 is affected (probably the most important question as we've already >> built it and it should be shipped to customers). >> >> >> On Thu, Mar 15, 2018 at 10:11 AM, Tal Nisan wrote: >> >>> I've reviewed and marked +1, I'll need another reviewer though for this >>> matter. >>> I've also based one of my patches on top of it and it passed OST: >>> http://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovi >>> rt-system-tests_manual/2387/ >>> >>> Dafna, prior to Eli's patch all OST jobs failed? >>> >>> >>> On Wed, Mar 14, 2018 at 6:46 PM, Dafna Ron wrote: >>> >>>> Eli updated the bug with a fix that reverts parts of the reported >>>> patch: https://gerrit.ovirt.org/#/c/89005/ >>>> >>>> waiting for verification and merge. >>>> >>>> Thanks! >>>> Dafna >>>> >>>> >>>> >>>> >>>> On Tue, Mar 13, 2018 at 9:49 PM, Dafna Ron wrote: >>>> >>>>> >>>>> >>>>> On Tue, Mar 13, 2018 at 9:32 PM, Michal Skrivanek < >>>>> michal.skrivanek at redhat.com> wrote: >>>>> >>>>>> >>>>>> >>>>>> On 13 Mar 2018, at 22:24, Dafna Ron wrote: >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Mar 13, 2018 at 10:57 AM, Michal Skrivanek < >>>>>> michal.skrivanek at redhat.com> wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> On 13 Mar 2018, at 09:27, Eyal Edri wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, Mar 13, 2018 at 9:29 AM, Dan Kenigsberg >>>>>>> wrote: >>>>>>> >>>>>>>> On Mon, Mar 12, 2018 at 8:24 PM, Dafna Ron wrote: >>>>>>>> > We just had a failure in master 002_bootstrap.add_mac_pool with >>>>>>>> the same >>>>>>>> > error on edit cluster. >>>>>>>> >>>>>>> >>>>>>> Does it fail consistently? >>>>>>> >>>>>> >>>>>> yes. >>>>>> >>>>>> >>>>>> that?s good >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> Did you narrow down the commit(s) where it started to happen? >>>>>>> >>>>>> >>>>>> First change reported failed by CQ on this issue is this one: >>>>>> https://gerrit.ovirt.org/#/c/88738/2 >>>>>> - db: add func to turn table columns to empty string (this was >>>>>> reported by Daniel at the beginning of this thread) >>>>>> >>>>>> Was there any other update done at that time? >>>>>>> >>>>>>> there are always other changes submitted. but CQ tries to isolate >>>>>> the change that it believes is causing the failure by reducing the change >>>>>> it tests until it gets to one single change. >>>>>> >>>>>> >>>>>> I meant changes like major update of packages or any other >>>>>> configuration change >>>>>> >>>>> >>>>> :) the last one I saw was from Eli on Friday but I don't think its >>>>> related. since this is a big project and there a lot of changes submitted >>>>> daily, maybe someone more qualified them me can have a look and see if >>>>> anything catches their eyes? >>>>> >>>>>> >>>>>> >>>>>> We cannot have OST keep failing for a long time, especially on a big >>>>>> project like ovirt-engine. if we cannot have a fix on this quickly I think >>>>>> we should start skipping failed tests to allow changes to pass successfully >>>>>> until the bug is fixed. >>>>>> >>>>>> >>>>>> sure. But in this case you?re just going to hit the same problem in >>>>>> the next test. Please enable back the one you commented out, and try to >>>>>> revert that patch instead. There is a chance it changed the behavior >>>>>> because somehow the tests using Default cluster somehow rely on undefined >>>>>> values (not sure if that?s even intentional, but that?s the way it is >>>>>> written), and that patch may have changed it perhaps. Eli? >>>>>> >>>>>> cool. I think that Eyal has reverted my skip test so we can try to >>>>> revert the change reported. >>>>> >>>>> >>>>>> Thanks, >>>>>> michal >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Dafna >>>>>> >>>>>> >>>>>> >>>>>>> Thanks, >>>>>>> michal >>>>>>> >>>>>>> > >>>>>>>> > http://jenkins.ovirt.org/job/ovirt-master_change-queue-teste >>>>>>>> r/6259/testReport/(root)/002_bootstrap/add_mac_pool/ >>>>>>>> > >>>>>>>> > either I skipped the wrong test or we have a bigger issue. >>>>>>>> >>>>>>>> We certainly do. As before, the error pops up on an attempt to >>>>>>>> update >>>>>>>> the cluster (this time it is changing only the mac pool of the >>>>>>>> cluster). CPU is not specified by the command, so it should not have >>>>>>>> changed at all. Still, something fills a CPU, and chooses a wrong >>>>>>>> value. >>>>>>>> >>>>>>>> cluster_service.update( >>>>>>>> cluster=sdk4.types.Cluster( >>>>>>>> mac_pool=sdk4.types.MacPool( >>>>>>>> id=pool.id, >>>>>>>> ) >>>>>>>> ) >>>>>>>> ) >>>>>>>> >>>>>>>> 2018-03-12 13:58:56,263-04 WARN >>>>>>>> [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) >>>>>>>> [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Validation of action >>>>>>>> 'UpdateCluster' failed for user admin at internal-authz. Reasons: >>>>>>>> VAR__TYPE__CLUSTER,VAR__ACTION__UPDATE,ACTION_TYPE_FAILED_CP >>>>>>>> U_NOT_FOUND,VAR__TYPE__CLUSTER >>>>>>>> 2018-03-12 13:58:56,264-04 INFO >>>>>>>> [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) >>>>>>>> [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Lock freed to object >>>>>>>> 'EngineLock:{exclusiveLocks='[]', sharedLocks='[]'}' >>>>>>>> 2018-03-12 13:58:56,264-04 DEBUG >>>>>>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInt >>>>>>>> erceptor] >>>>>>>> (default task-19) [79d12090-a8e8-438c-bbce-1bf09a42c5fb] method: >>>>>>>> runAction, params: [UpdateCluster, >>>>>>>> ManagementNetworkOnClusterOperationParameters:{commandId='be >>>>>>>> be80f7-f8ca-4d01-aed8-28e463d0f435', >>>>>>>> user='null', commandType='Unknown'}], timeElapsed: 50ms >>>>>>>> 2018-03-12 13:58:56,269-04 ERROR >>>>>>>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] >>>>>>>> (default task-19) [] Operation Failed: [Cannot edit Cluster. The >>>>>>>> chosen CPU is not supported.] >>>>>>>> >>>>>>> >>>>>>> So I guess we can't skip this test as well, and this issue has to be >>>>>>> fixed right? >>>>>>> >>>>>>> >>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Devel mailing list >>>>>>>> Devel at ovirt.org >>>>>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Eyal edri >>>>>>> >>>>>>> MANAGER >>>>>>> >>>>>>> RHV DevOps >>>>>>> >>>>>>> EMEA VIRTUALIZATION R&D >>>>>>> >>>>>>> >>>>>>> Red Hat EMEA >>>>>>> TRIED. TESTED. TRUSTED. >>>>>>> >>>>>>> phone: +972-9-7692018 <+972%209-769-2018> >>>>>>> irc: eedri (on #tlv #rhev-dev #rhev-integ) >>>>>>> _______________________________________________ >>>>>>> Devel mailing list >>>>>>> Devel at ovirt.org >>>>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tnisan at redhat.com Thu Mar 15 10:53:10 2018 From: tnisan at redhat.com (Tal Nisan) Date: Thu, 15 Mar 2018 12:53:10 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 ] [ 002_bootstrap.update_default_cluster ] In-Reply-To: References: <59AA8F90-3BB1-4D62-8237-1856514282C8@redhat.com> <3B429D57-6C6C-4559-8D38-87A2658F3905@redhat.com> <7CFE1871-70D5-4A5A-98A7-F4D0CD576D67@redhat.com> Message-ID: Oh, but I've started the same, that's why I built the artifacts to begin with.. Anyway, seems like the 002_bootstrap has passed, I'll merge now On Thu, Mar 15, 2018 at 12:49 PM, Dan Kenigsberg wrote: > And I've started http://jenkins.ovirt.org/job/ovirt-system-tests_manual/ > 2392/console to see if it OST passes there. > > On Thu, Mar 15, 2018 at 11:26 AM, Tal Nisan wrote: > >> I've backported to 4.2, kind reviewers from master please review as well >> - https://gerrit.ovirt.org/#/c/89035/ >> >> >> >> On Thu, Mar 15, 2018 at 11:18 AM, Tal Nisan wrote: >> >>> Thanks to the reviewers, merged on master now. >>> Working with Dafna on getting it fixed on 4.2 and understanding whether >>> 4.1.10 is affected (probably the most important question as we've already >>> built it and it should be shipped to customers). >>> >>> >>> On Thu, Mar 15, 2018 at 10:11 AM, Tal Nisan wrote: >>> >>>> I've reviewed and marked +1, I'll need another reviewer though for this >>>> matter. >>>> I've also based one of my patches on top of it and it passed OST: >>>> http://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovi >>>> rt-system-tests_manual/2387/ >>>> >>>> Dafna, prior to Eli's patch all OST jobs failed? >>>> >>>> >>>> On Wed, Mar 14, 2018 at 6:46 PM, Dafna Ron wrote: >>>> >>>>> Eli updated the bug with a fix that reverts parts of the reported >>>>> patch: https://gerrit.ovirt.org/#/c/89005/ >>>>> >>>>> waiting for verification and merge. >>>>> >>>>> Thanks! >>>>> Dafna >>>>> >>>>> >>>>> >>>>> >>>>> On Tue, Mar 13, 2018 at 9:49 PM, Dafna Ron wrote: >>>>> >>>>>> >>>>>> >>>>>> On Tue, Mar 13, 2018 at 9:32 PM, Michal Skrivanek < >>>>>> michal.skrivanek at redhat.com> wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> On 13 Mar 2018, at 22:24, Dafna Ron wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, Mar 13, 2018 at 10:57 AM, Michal Skrivanek < >>>>>>> michal.skrivanek at redhat.com> wrote: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 13 Mar 2018, at 09:27, Eyal Edri wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Mar 13, 2018 at 9:29 AM, Dan Kenigsberg >>>>>>>> wrote: >>>>>>>> >>>>>>>>> On Mon, Mar 12, 2018 at 8:24 PM, Dafna Ron >>>>>>>>> wrote: >>>>>>>>> > We just had a failure in master 002_bootstrap.add_mac_pool with >>>>>>>>> the same >>>>>>>>> > error on edit cluster. >>>>>>>>> >>>>>>>> >>>>>>>> Does it fail consistently? >>>>>>>> >>>>>>> >>>>>>> yes. >>>>>>> >>>>>>> >>>>>>> that?s good >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>> Did you narrow down the commit(s) where it started to happen? >>>>>>>> >>>>>>> >>>>>>> First change reported failed by CQ on this issue is this one: >>>>>>> https://gerrit.ovirt.org/#/c/88738/2 >>>>>>> - db: add func to turn table columns to empty string (this was >>>>>>> reported by Daniel at the beginning of this thread) >>>>>>> >>>>>>> Was there any other update done at that time? >>>>>>>> >>>>>>>> there are always other changes submitted. but CQ tries to isolate >>>>>>> the change that it believes is causing the failure by reducing the change >>>>>>> it tests until it gets to one single change. >>>>>>> >>>>>>> >>>>>>> I meant changes like major update of packages or any other >>>>>>> configuration change >>>>>>> >>>>>> >>>>>> :) the last one I saw was from Eli on Friday but I don't think its >>>>>> related. since this is a big project and there a lot of changes submitted >>>>>> daily, maybe someone more qualified them me can have a look and see if >>>>>> anything catches their eyes? >>>>>> >>>>>>> >>>>>>> >>>>>>> We cannot have OST keep failing for a long time, especially on a big >>>>>>> project like ovirt-engine. if we cannot have a fix on this quickly I think >>>>>>> we should start skipping failed tests to allow changes to pass successfully >>>>>>> until the bug is fixed. >>>>>>> >>>>>>> >>>>>>> sure. But in this case you?re just going to hit the same problem in >>>>>>> the next test. Please enable back the one you commented out, and try to >>>>>>> revert that patch instead. There is a chance it changed the behavior >>>>>>> because somehow the tests using Default cluster somehow rely on undefined >>>>>>> values (not sure if that?s even intentional, but that?s the way it is >>>>>>> written), and that patch may have changed it perhaps. Eli? >>>>>>> >>>>>>> cool. I think that Eyal has reverted my skip test so we can try to >>>>>> revert the change reported. >>>>>> >>>>>> >>>>>>> Thanks, >>>>>>> michal >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Dafna >>>>>>> >>>>>>> >>>>>>> >>>>>>>> Thanks, >>>>>>>> michal >>>>>>>> >>>>>>>> > >>>>>>>>> > http://jenkins.ovirt.org/job/ovirt-master_change-queue-teste >>>>>>>>> r/6259/testReport/(root)/002_bootstrap/add_mac_pool/ >>>>>>>>> > >>>>>>>>> > either I skipped the wrong test or we have a bigger issue. >>>>>>>>> >>>>>>>>> We certainly do. As before, the error pops up on an attempt to >>>>>>>>> update >>>>>>>>> the cluster (this time it is changing only the mac pool of the >>>>>>>>> cluster). CPU is not specified by the command, so it should not >>>>>>>>> have >>>>>>>>> changed at all. Still, something fills a CPU, and chooses a wrong >>>>>>>>> value. >>>>>>>>> >>>>>>>>> cluster_service.update( >>>>>>>>> cluster=sdk4.types.Cluster( >>>>>>>>> mac_pool=sdk4.types.MacPool( >>>>>>>>> id=pool.id, >>>>>>>>> ) >>>>>>>>> ) >>>>>>>>> ) >>>>>>>>> >>>>>>>>> 2018-03-12 13:58:56,263-04 WARN >>>>>>>>> [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) >>>>>>>>> [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Validation of action >>>>>>>>> 'UpdateCluster' failed for user admin at internal-authz. Reasons: >>>>>>>>> VAR__TYPE__CLUSTER,VAR__ACTION__UPDATE,ACTION_TYPE_FAILED_CP >>>>>>>>> U_NOT_FOUND,VAR__TYPE__CLUSTER >>>>>>>>> 2018-03-12 13:58:56,264-04 INFO >>>>>>>>> [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-19) >>>>>>>>> [79d12090-a8e8-438c-bbce-1bf09a42c5fb] Lock freed to object >>>>>>>>> 'EngineLock:{exclusiveLocks='[]', sharedLocks='[]'}' >>>>>>>>> 2018-03-12 13:58:56,264-04 DEBUG >>>>>>>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInt >>>>>>>>> erceptor] >>>>>>>>> (default task-19) [79d12090-a8e8-438c-bbce-1bf09a42c5fb] method: >>>>>>>>> runAction, params: [UpdateCluster, >>>>>>>>> ManagementNetworkOnClusterOperationParameters:{commandId='be >>>>>>>>> be80f7-f8ca-4d01-aed8-28e463d0f435', >>>>>>>>> user='null', commandType='Unknown'}], timeElapsed: 50ms >>>>>>>>> 2018-03-12 13:58:56,269-04 ERROR >>>>>>>>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] >>>>>>>>> (default task-19) [] Operation Failed: [Cannot edit Cluster. The >>>>>>>>> chosen CPU is not supported.] >>>>>>>>> >>>>>>>> >>>>>>>> So I guess we can't skip this test as well, and this issue has to >>>>>>>> be fixed right? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Devel mailing list >>>>>>>>> Devel at ovirt.org >>>>>>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Eyal edri >>>>>>>> >>>>>>>> MANAGER >>>>>>>> >>>>>>>> RHV DevOps >>>>>>>> >>>>>>>> EMEA VIRTUALIZATION R&D >>>>>>>> >>>>>>>> >>>>>>>> Red Hat EMEA >>>>>>>> TRIED. TESTED. TRUSTED. >>>>>>>> >>>>>>>> phone: +972-9-7692018 <+972%209-769-2018> >>>>>>>> irc: eedri (on #tlv #rhev-dev #rhev-integ) >>>>>>>> _______________________________________________ >>>>>>>> Devel mailing list >>>>>>>> Devel at ovirt.org >>>>>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >> _______________________________________________ >> Devel mailing list >> Devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vszocs at redhat.com Thu Mar 15 15:42:24 2018 From: vszocs at redhat.com (Vojtech Szocs) Date: Thu, 15 Mar 2018 16:42:24 +0100 Subject: [ovirt-devel] UI plugin API updates Message-ID: Dear community, the UI plugin API will be updated to reflect recent oVirt web administration UI design changes. The relevant patch is already merged in master branch [1] and the associated BZ [2] is targeted for 4.3 release. *What's new* Two new API functions, addPrimaryMenuContainer and addSecondaryMenuPlace, allow you to add custom secondary menu items to the vertical navigation menu. You can target both existing (core) and custom primary menu items when adding secondary ones. *What's changed* Some API functions were renamed to stay consistent with current UI design, i.e. reflecting the absence of "main" and "sub" tabs. - addMainTab => addPrimaryMenuPlace - addSubTab => addDetailPlace - setTabContentUrl => setPlaceContentUrl - setTabAccessible => setPlaceAccessible - addMainTabActionButton => addMenuPlaceActionButton - addSubTabActionButton => addDetailPlaceActionButton You can still use the original functions mentioned above, but doing so will yield a warning in the browser console, for example: *addMainTab is deprecated, please use addPrimaryMenuPlace instead.* In addition, for functions that used to deal with "main" or "sub" tabs, the options object no longer supports alignRight (boolean) parameter. That's because PatternFly tabs widget [3] expects all tabs to be aligned next to each other, flowing from left to right. We'll be updating the UI plugins feature page shortly to reflect all the changes. [1] https://gerrit.ovirt.org/#/c/88690/ [2] https://bugzilla.redhat.com/1553902 [3] http://www.patternfly.org/pattern-library/widgets/#tabs Regards, Vojtech -------------- next part -------------- An HTML attachment was scrubbed... URL: From dron at redhat.com Thu Mar 15 16:24:10 2018 From: dron at redhat.com (Dafna Ron) Date: Thu, 15 Mar 2018 16:24:10 +0000 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (cockpit-ovirt) ] [ 15-03-2018 ] [ 098_ovirt_provider_ovn.use_ovn_provider ] Message-ID: Hi, We have a failure on master for test 098_ovirt_provider_ovn.use_ovn_provider in project cockpit-ovirt. This seems to be a race because object is locked. also, the actual failure is logged as WARN and not ERROR. I don't think the patch is actually related to the failure but I think the test should be fixed. can you please review to make sure we do not have an actual regression and let me know if we need to open a bz to fix the test? *Link and headline of suspected patches: * *https://gerrit.ovirt.org/#/c/89020/2 - * *wizard: Enable scroll on start page for low-res screensLink to Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6374 Link to all logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6374/artifacts (Relevant) error snippet from the log: 2018-03-15 10:05:00,160-04 DEBUG [org.ovirt.engine.core.sso.servlets.OAuthTokenInfoServlet] (default task-10) [] Sending json response2018-03-15 10:05:00,160-04 DEBUG [org.ovirt.engine.core.sso.utils.TokenCleanupUtility] (default task-10) [] Not cleaning up expired tokens2018-03-15 10:05:00,169-04 INFO [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [789edb23] Lock Acquired to object 'EngineLock:{exclusiveLocks='[c38a67ec-0b48-4e6f-be85-70c700df5483=PROVIDER]', sharedLocks=''}'2018-03-15 10:05:00,184-04 INFO [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [789edb23] Running command: SyncNetworkProviderCommand internal: true.2018-03-15 10:05:00,228-04 DEBUG [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall] (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] Compiled stored procedure. Call string is [{call getdcidbyexternalnetworkid(?)}]2018-03-15 10:05:00,228-04 DEBUG [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall] (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] SqlCall for procedure [GetDcIdByExternalNetworkId] compiled2018-03-15 10:05:00,229-04 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] method: runQuery, params: [GetAllExternalNetworksOnProvider, IdQueryParameters:{refresh='false', filtered='false'}], timeElapsed: 353ms2018-03-15 10:05:00,239-04 INFO [org.ovirt.engine.core.bll.network.dc.AddNetworkCommand] (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] Failed to Acquire Lock to object 'EngineLock:{exclusiveLocks='[network_1=NETWORK, c38a67ec-0b48-4e6f-be85-70c700df5483=PROVIDER]', sharedLocks=''}'2018-03-15 10:05:00,239-04 WARN [org.ovirt.engine.core.bll.network.dc.AddNetworkCommand] (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] Validation of action 'AddNetwork' failed for user admin at internal-authz. Reasons: VAR__TYPE__NETWORK,VAR__ACTION__ADD,ACTION_TYPE_FAILED_PROVIDER_LOCKED,$providerId c38a67ec-0b48-4e6f-be85-70c700df54832018-03-15 10:05:00,240-04 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] method: runAction, params: [AddNetwork, AddNetworkStoragePoolParameters:{commandId='61b365ec-27c1-49af-ad72-f907df8befcd', user='null', commandType='Unknown'}], timeElapsed: 10ms2018-03-15 10:05:00,250-04 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-13) [] Operation Failed: [Cannot add Network. Related operation on provider with the id c38a67ec-0b48-4e6f-be85-70c700df5483 is currently in progress. Please try again later.]2018-03-15 10:05:00,254-04 DEBUG [org.ovirt.engine.core.utils.servlet.LocaleFilter] (default task-14) [] Incoming locale 'en-US'. Filter determined locale to be 'en-US'2018-03-15 10:05:00,254-04 DEBUG [org.ovirt.engine.core.sso.servlets.OAuthTokenServlet] (default task-14) [] Entered OAuthTokenServlet Query String: null, Parameters : password = ***, grant_type = password, scope = ovirt-app-api ovirt-ext=token-info:validate, username = admin at internal, * -------------- next part -------------- An HTML attachment was scrubbed... URL: From dholler at redhat.com Thu Mar 15 20:21:47 2018 From: dholler at redhat.com (Dominik Holler) Date: Thu, 15 Mar 2018 21:21:47 +0100 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (cockpit-ovirt) ] [ 15-03-2018 ] [ 098_ovirt_provider_ovn.use_ovn_provider ] In-Reply-To: References: Message-ID: <20180315212147.48669c08@t460p> On Thu, 15 Mar 2018 16:24:10 +0000 Dafna Ron wrote: > Hi, > > We have a failure on master for test > 098_ovirt_provider_ovn.use_ovn_provider in project cockpit-ovirt. > This seems to be a race because object is locked. also, the actual > failure is logged as WARN and not ERROR. > > I don't think the patch is actually related to the failure but I > think the test should be fixed. > can you please review to make sure we do not have an actual > regression and let me know if we need to open a bz to fix the test? > > > *Link and headline of suspected patches: * > *https://gerrit.ovirt.org/#/c/89020/2 > - * > > > > > > > > > > > > > > > > > > > > > > > > > > > *wizard: Enable scroll on start page for low-res screensLink to > Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6374 > Link > to all > logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6374/artifacts > (Relevant) > error snippet from the log: 2018-03-15 10:05:00,160-04 DEBUG > [org.ovirt.engine.core.sso.servlets.OAuthTokenInfoServlet] (default > task-10) [] Sending json response2018-03-15 10:05:00,160-04 DEBUG > [org.ovirt.engine.core.sso.utils.TokenCleanupUtility] (default > task-10) [] Not cleaning up expired tokens2018-03-15 10:05:00,169-04 > INFO > [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] > (EE-ManagedThreadFactory-engineScheduled-Thread-90) [789edb23] Lock > Acquired to object > 'EngineLock:{exclusiveLocks='[c38a67ec-0b48-4e6f-be85-70c700df5483=PROVIDER]', > sharedLocks=''}'2018-03-15 10:05:00,184-04 INFO > [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] > (EE-ManagedThreadFactory-engineScheduled-Thread-90) [789edb23] > Running command: SyncNetworkProviderCommand internal: true.2018-03-15 > 10:05:00,228-04 DEBUG > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall] > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] Compiled > stored procedure. Call string is [{call > getdcidbyexternalnetworkid(?)}]2018-03-15 10:05:00,228-04 DEBUG > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall] > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] SqlCall for > procedure [GetDcIdByExternalNetworkId] compiled2018-03-15 > 10:05:00,229-04 DEBUG > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] method: > runQuery, params: [GetAllExternalNetworksOnProvider, > IdQueryParameters:{refresh='false', filtered='false'}], timeElapsed: > 353ms2018-03-15 10:05:00,239-04 INFO > [org.ovirt.engine.core.bll.network.dc.AddNetworkCommand] (default > task-13) [e1328379-17b7-49f8-beb2-cf8331784828] Failed to Acquire > Lock to object 'EngineLock:{exclusiveLocks='[network_1=NETWORK, > c38a67ec-0b48-4e6f-be85-70c700df5483=PROVIDER]', > sharedLocks=''}'2018-03-15 10:05:00,239-04 WARN > [org.ovirt.engine.core.bll.network.dc.AddNetworkCommand] (default > task-13) [e1328379-17b7-49f8-beb2-cf8331784828] Validation of action > 'AddNetwork' failed for user admin at internal-authz. Reasons: > VAR__TYPE__NETWORK,VAR__ACTION__ADD,ACTION_TYPE_FAILED_PROVIDER_LOCKED,$providerId > c38a67ec-0b48-4e6f-be85-70c700df54832018-03-15 10:05:00,240-04 DEBUG > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] method: > runAction, params: [AddNetwork, > AddNetworkStoragePoolParameters:{commandId='61b365ec-27c1-49af-ad72-f907df8befcd', > user='null', commandType='Unknown'}], timeElapsed: 10ms2018-03-15 > 10:05:00,250-04 ERROR > [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] > (default task-13) [] Operation Failed: [Cannot add Network. Related > operation on provider with the id > c38a67ec-0b48-4e6f-be85-70c700df5483 is currently in progress. Please > try again later.]2018-03-15 10:05:00,254-04 DEBUG > [org.ovirt.engine.core.utils.servlet.LocaleFilter] (default task-14) > [] Incoming locale 'en-US'. Filter determined locale to be > 'en-US'2018-03-15 10:05:00,254-04 DEBUG > [org.ovirt.engine.core.sso.servlets.OAuthTokenServlet] (default > task-14) [] Entered OAuthTokenServlet Query String: null, > Parameters : password = ***, grant_type = password, scope = > ovirt-app-api ovirt-ext=token-info:validate, username = > admin at internal, * I will care about this. The problem is that SyncNetworkProviderCommand is running in the background and locking the provider, which blocks the lock for the tested AddNetworkCommand. The related changes are core: Add locking for Add and RemoveNetworkCommand https://gerrit.ovirt.org/#/c/85480/ and core: Add SyncNetworkProviderCommand https://gerrit.ovirt.org/#/c/85134/ From dron at redhat.com Fri Mar 16 09:32:47 2018 From: dron at redhat.com (Dafna Ron) Date: Fri, 16 Mar 2018 09:32:47 +0000 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (cockpit-ovirt) ] [ 15-03-2018 ] [ 098_ovirt_provider_ovn.use_ovn_provider ] In-Reply-To: <20180315212147.48669c08@t460p> References: <20180315212147.48669c08@t460p> Message-ID: Thank you for the fast reply and help. On Thu, Mar 15, 2018 at 8:21 PM, Dominik Holler wrote: > On Thu, 15 Mar 2018 16:24:10 +0000 > Dafna Ron wrote: > > > Hi, > > > > We have a failure on master for test > > 098_ovirt_provider_ovn.use_ovn_provider in project cockpit-ovirt. > > This seems to be a race because object is locked. also, the actual > > failure is logged as WARN and not ERROR. > > > > I don't think the patch is actually related to the failure but I > > think the test should be fixed. > > can you please review to make sure we do not have an actual > > regression and let me know if we need to open a bz to fix the test? > > > > > > *Link and headline of suspected patches: * > > *https://gerrit.ovirt.org/#/c/89020/2 > > - * > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *wizard: Enable scroll on start page for low-res screensLink to > > Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6374 > > Link > > to all > > logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue- > tester/6374/artifacts > > tester/6374/artifacts>(Relevant) > > error snippet from the log: 2018-03-15 10:05:00,160-04 DEBUG > > [org.ovirt.engine.core.sso.servlets.OAuthTokenInfoServlet] (default > > task-10) [] Sending json response2018-03-15 10:05:00,160-04 DEBUG > > [org.ovirt.engine.core.sso.utils.TokenCleanupUtility] (default > > task-10) [] Not cleaning up expired tokens2018-03-15 10:05:00,169-04 > > INFO > > [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] > > (EE-ManagedThreadFactory-engineScheduled-Thread-90) [789edb23] Lock > > Acquired to object > > 'EngineLock:{exclusiveLocks='[c38a67ec-0b48-4e6f-be85- > 70c700df5483=PROVIDER]', > > sharedLocks=''}'2018-03-15 10:05:00,184-04 INFO > > [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] > > (EE-ManagedThreadFactory-engineScheduled-Thread-90) [789edb23] > > Running command: SyncNetworkProviderCommand internal: true.2018-03-15 > > 10:05:00,228-04 DEBUG > > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$ > PostgresSimpleJdbcCall] > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] Compiled > > stored procedure. Call string is [{call > > getdcidbyexternalnetworkid(?)}]2018-03-15 10:05:00,228-04 DEBUG > > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$ > PostgresSimpleJdbcCall] > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] SqlCall for > > procedure [GetDcIdByExternalNetworkId] compiled2018-03-15 > > 10:05:00,229-04 DEBUG > > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] method: > > runQuery, params: [GetAllExternalNetworksOnProvider, > > IdQueryParameters:{refresh='false', filtered='false'}], timeElapsed: > > 353ms2018-03-15 10:05:00,239-04 INFO > > [org.ovirt.engine.core.bll.network.dc.AddNetworkCommand] (default > > task-13) [e1328379-17b7-49f8-beb2-cf8331784828] Failed to Acquire > > Lock to object 'EngineLock:{exclusiveLocks='[network_1=NETWORK, > > c38a67ec-0b48-4e6f-be85-70c700df5483=PROVIDER]', > > sharedLocks=''}'2018-03-15 10:05:00,239-04 WARN > > [org.ovirt.engine.core.bll.network.dc.AddNetworkCommand] (default > > task-13) [e1328379-17b7-49f8-beb2-cf8331784828] Validation of action > > 'AddNetwork' failed for user admin at internal-authz. Reasons: > > VAR__TYPE__NETWORK,VAR__ACTION__ADD,ACTION_TYPE_FAILED_PROVIDER_LOCKED,$ > providerId > > c38a67ec-0b48-4e6f-be85-70c700df54832018-03-15 10:05:00,240-04 DEBUG > > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] method: > > runAction, params: [AddNetwork, > > AddNetworkStoragePoolParameters:{commandId='61b365ec-27c1- > 49af-ad72-f907df8befcd', > > user='null', commandType='Unknown'}], timeElapsed: 10ms2018-03-15 > > 10:05:00,250-04 ERROR > > [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] > > (default task-13) [] Operation Failed: [Cannot add Network. Related > > operation on provider with the id > > c38a67ec-0b48-4e6f-be85-70c700df5483 is currently in progress. Please > > try again later.]2018-03-15 10:05:00,254-04 DEBUG > > [org.ovirt.engine.core.utils.servlet.LocaleFilter] (default task-14) > > [] Incoming locale 'en-US'. Filter determined locale to be > > 'en-US'2018-03-15 10:05:00,254-04 DEBUG > > [org.ovirt.engine.core.sso.servlets.OAuthTokenServlet] (default > > task-14) [] Entered OAuthTokenServlet Query String: null, > > Parameters : password = ***, grant_type = password, scope = > > ovirt-app-api ovirt-ext=token-info:validate, username = > > admin at internal, * > > I will care about this. > The problem is that SyncNetworkProviderCommand is running in the > background and locking the provider, which blocks the lock for the > tested AddNetworkCommand. > The related changes are > core: Add locking for Add and RemoveNetworkCommand > https://gerrit.ovirt.org/#/c/85480/ > and > core: Add SyncNetworkProviderCommand > https://gerrit.ovirt.org/#/c/85134/ > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dholler at redhat.com Fri Mar 16 14:52:52 2018 From: dholler at redhat.com (Dominik Holler) Date: Fri, 16 Mar 2018 15:52:52 +0100 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (cockpit-ovirt) ] [ 15-03-2018 ] [ 098_ovirt_provider_ovn.use_ovn_provider ] In-Reply-To: References: <20180315212147.48669c08@t460p> Message-ID: <20180316155252.1fd28b5b@t460p> I have created https://bugzilla.redhat.com/show_bug.cgi?id=1557419 and https://bugzilla.redhat.com/show_bug.cgi?id=1557424 to discuss how failing to acquire locks in engine should be handled in engine's REST-API, in ovirt-sdk or in application (OST in this case). On Fri, 16 Mar 2018 09:32:47 +0000 Dafna Ron wrote: > Thank you for the fast reply and help. > > On Thu, Mar 15, 2018 at 8:21 PM, Dominik Holler > wrote: > > > On Thu, 15 Mar 2018 16:24:10 +0000 > > Dafna Ron wrote: > > > > > Hi, > > > > > > We have a failure on master for test > > > 098_ovirt_provider_ovn.use_ovn_provider in project cockpit-ovirt. > > > This seems to be a race because object is locked. also, the actual > > > failure is logged as WARN and not ERROR. > > > > > > I don't think the patch is actually related to the failure but I > > > think the test should be fixed. > > > can you please review to make sure we do not have an actual > > > regression and let me know if we need to open a bz to fix the > > > test? > > > > > > > > > *Link and headline of suspected patches: * > > > *https://gerrit.ovirt.org/#/c/89020/2 > > > - * > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *wizard: Enable scroll on start page for low-res screensLink to > > > Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6374 > > > Link > > > to all > > > logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue- > > tester/6374/artifacts > > > > tester/6374/artifacts>(Relevant) > > > error snippet from the log: 2018-03-15 10:05:00,160-04 > > > DEBUG [org.ovirt.engine.core.sso.servlets.OAuthTokenInfoServlet] > > > (default task-10) [] Sending json response2018-03-15 > > > 10:05:00,160-04 DEBUG > > > [org.ovirt.engine.core.sso.utils.TokenCleanupUtility] (default > > > task-10) [] Not cleaning up expired tokens2018-03-15 > > > 10:05:00,169-04 INFO > > > [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] > > > (EE-ManagedThreadFactory-engineScheduled-Thread-90) [789edb23] > > > Lock Acquired to object > > > 'EngineLock:{exclusiveLocks='[c38a67ec-0b48-4e6f-be85- > > 70c700df5483=PROVIDER]', > > > sharedLocks=''}'2018-03-15 10:05:00,184-04 INFO > > > [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] > > > (EE-ManagedThreadFactory-engineScheduled-Thread-90) [789edb23] > > > Running command: SyncNetworkProviderCommand internal: > > > true.2018-03-15 10:05:00,228-04 DEBUG > > > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$ > > PostgresSimpleJdbcCall] > > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] Compiled > > > stored procedure. Call string is [{call > > > getdcidbyexternalnetworkid(?)}]2018-03-15 10:05:00,228-04 DEBUG > > > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$ > > PostgresSimpleJdbcCall] > > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] SqlCall > > > for procedure [GetDcIdByExternalNetworkId] compiled2018-03-15 > > > 10:05:00,229-04 DEBUG > > > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] method: > > > runQuery, params: [GetAllExternalNetworksOnProvider, > > > IdQueryParameters:{refresh='false', filtered='false'}], > > > timeElapsed: 353ms2018-03-15 10:05:00,239-04 INFO > > > [org.ovirt.engine.core.bll.network.dc.AddNetworkCommand] (default > > > task-13) [e1328379-17b7-49f8-beb2-cf8331784828] Failed to Acquire > > > Lock to object 'EngineLock:{exclusiveLocks='[network_1=NETWORK, > > > c38a67ec-0b48-4e6f-be85-70c700df5483=PROVIDER]', > > > sharedLocks=''}'2018-03-15 10:05:00,239-04 WARN > > > [org.ovirt.engine.core.bll.network.dc.AddNetworkCommand] (default > > > task-13) [e1328379-17b7-49f8-beb2-cf8331784828] Validation of > > > action 'AddNetwork' failed for user admin at internal-authz. Reasons: > > > VAR__TYPE__NETWORK,VAR__ACTION__ADD,ACTION_TYPE_FAILED_PROVIDER_LOCKED,$ > > providerId > > > c38a67ec-0b48-4e6f-be85-70c700df54832018-03-15 10:05:00,240-04 > > > DEBUG > > > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] method: > > > runAction, params: [AddNetwork, > > > AddNetworkStoragePoolParameters:{commandId='61b365ec-27c1- > > 49af-ad72-f907df8befcd', > > > user='null', commandType='Unknown'}], timeElapsed: 10ms2018-03-15 > > > 10:05:00,250-04 ERROR > > > [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] > > > (default task-13) [] Operation Failed: [Cannot add Network. > > > Related operation on provider with the id > > > c38a67ec-0b48-4e6f-be85-70c700df5483 is currently in progress. > > > Please try again later.]2018-03-15 10:05:00,254-04 DEBUG > > > [org.ovirt.engine.core.utils.servlet.LocaleFilter] (default > > > task-14) [] Incoming locale 'en-US'. Filter determined locale to > > > be 'en-US'2018-03-15 10:05:00,254-04 DEBUG > > > [org.ovirt.engine.core.sso.servlets.OAuthTokenServlet] (default > > > task-14) [] Entered OAuthTokenServlet Query String: null, > > > Parameters : password = ***, grant_type = password, scope = > > > ovirt-app-api ovirt-ext=token-info:validate, username = > > > admin at internal, * > > > > I will care about this. > > The problem is that SyncNetworkProviderCommand is running in the > > background and locking the provider, which blocks the lock for the > > tested AddNetworkCommand. > > The related changes are > > core: Add locking for Add and RemoveNetworkCommand > > https://gerrit.ovirt.org/#/c/85480/ > > and > > core: Add SyncNetworkProviderCommand > > https://gerrit.ovirt.org/#/c/85134/ > > > > > > From dron at redhat.com Fri Mar 16 20:55:50 2018 From: dron at redhat.com (Dafna Ron) Date: Fri, 16 Mar 2018 20:55:50 +0000 Subject: [ovirt-devel] OST Failure - Weekly update [09/03/2018-16/03/2018] Message-ID: Hello, I would like to update on this week's failures and OST current status. We had a very eventful week with 2 regressions identified by OST 1. On 11-03-2018 the CI team reported a regression in project ovirt-engine. The change reported was https://gerrit.ovirt.org/#/c/88738/ - db: add func to turn table columns to empty string. This caused a failure in both master and 4.2 in OST but was also identified to be an issue in 4.1. The fix for all 3 versions can be tracked here: https://bugzilla.redhat.com/show_bug.cgi?id=1554377 and was submitted on 15-03-2018. I hope that in future we would resolve such regression much faster as it took us 5 days to resolve this issue but I would also like to thank all that helped to resolve it once the issue was investigated more deeply. 2. on 13-03-2018 we found a regression in otopi: *core: Use python3 when possible- https://gerrit.ovirt.org/#/c/87276/ * *The fix was* submitted *on the same day **https://gerrit.ovirt.org/88931 - **spec: Package otopi-functions* We also had 1 configuration related issue which was fixed by the CI team and we had failed build-artifacts, mostly in fc27. this was due to the change in fedora to not allow the use of yum and had to be solved by the projects making minor patch changes. *Below you can see the chart for this week's resolved issues but cause of failure:*Code = regression of working components/functionalities Configurations = package related issues Other = failed build artifacts Infra = infrastructure/OST/Lago related issues *Below is a chart of resolved failures based on ovirt version* *Below is a chart showing failures by suite type: * Thank you, Dafna -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 24305 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 6200 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 27670 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 7715 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 6766 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 26960 bytes Desc: not available URL: From danken at redhat.com Sat Mar 17 14:16:49 2018 From: danken at redhat.com (Dan Kenigsberg) Date: Sat, 17 Mar 2018 16:16:49 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (cockpit-ovirt) ] [ 15-03-2018 ] [ 098_ovirt_provider_ovn.use_ovn_provider ] In-Reply-To: <20180316155252.1fd28b5b@t460p> References: <20180315212147.48669c08@t460p> <20180316155252.1fd28b5b@t460p> Message-ID: Thanks for filing those, but let us not keep them secret: Bug 1557419 - Recommend that a call should be tried again sounds most reasonable. Providing more information to the client makes sense Bug 1557424 - Automatically retry call failed because he failed to acquire a lock is more tricky. when would REST perform the retry? how often? We must make sure we never cause a livelock. Can we alternatively take a normal blocking lock when we create an external network (instead of a trylock)? On Fri, Mar 16, 2018 at 4:52 PM, Dominik Holler wrote: > I have created > https://bugzilla.redhat.com/show_bug.cgi?id=1557419 and > https://bugzilla.redhat.com/show_bug.cgi?id=1557424 > to discuss how failing to acquire locks in engine should be handled in > engine's REST-API, in ovirt-sdk or in application (OST in this case). > > > On Fri, 16 Mar 2018 09:32:47 +0000 > Dafna Ron wrote: > >> Thank you for the fast reply and help. >> >> On Thu, Mar 15, 2018 at 8:21 PM, Dominik Holler >> wrote: >> >> > On Thu, 15 Mar 2018 16:24:10 +0000 >> > Dafna Ron wrote: >> > >> > > Hi, >> > > >> > > We have a failure on master for test >> > > 098_ovirt_provider_ovn.use_ovn_provider in project cockpit-ovirt. >> > > This seems to be a race because object is locked. also, the actual >> > > failure is logged as WARN and not ERROR. >> > > >> > > I don't think the patch is actually related to the failure but I >> > > think the test should be fixed. >> > > can you please review to make sure we do not have an actual >> > > regression and let me know if we need to open a bz to fix the >> > > test? >> > > >> > > >> > > *Link and headline of suspected patches: * >> > > *https://gerrit.ovirt.org/#/c/89020/2 >> > > - * >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > *wizard: Enable scroll on start page for low-res screensLink to >> > > Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6374 >> > > Link >> > > to all >> > > logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue- >> > tester/6374/artifacts >> > > > > tester/6374/artifacts>(Relevant) >> > > error snippet from the log: 2018-03-15 10:05:00,160-04 >> > > DEBUG [org.ovirt.engine.core.sso.servlets.OAuthTokenInfoServlet] >> > > (default task-10) [] Sending json response2018-03-15 >> > > 10:05:00,160-04 DEBUG >> > > [org.ovirt.engine.core.sso.utils.TokenCleanupUtility] (default >> > > task-10) [] Not cleaning up expired tokens2018-03-15 >> > > 10:05:00,169-04 INFO >> > > [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> > > (EE-ManagedThreadFactory-engineScheduled-Thread-90) [789edb23] >> > > Lock Acquired to object >> > > 'EngineLock:{exclusiveLocks='[c38a67ec-0b48-4e6f-be85- >> > 70c700df5483=PROVIDER]', >> > > sharedLocks=''}'2018-03-15 10:05:00,184-04 INFO >> > > [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> > > (EE-ManagedThreadFactory-engineScheduled-Thread-90) [789edb23] >> > > Running command: SyncNetworkProviderCommand internal: >> > > true.2018-03-15 10:05:00,228-04 DEBUG >> > > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$ >> > PostgresSimpleJdbcCall] >> > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] Compiled >> > > stored procedure. Call string is [{call >> > > getdcidbyexternalnetworkid(?)}]2018-03-15 10:05:00,228-04 DEBUG >> > > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$ >> > PostgresSimpleJdbcCall] >> > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] SqlCall >> > > for procedure [GetDcIdByExternalNetworkId] compiled2018-03-15 >> > > 10:05:00,229-04 DEBUG >> > > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >> > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] method: >> > > runQuery, params: [GetAllExternalNetworksOnProvider, >> > > IdQueryParameters:{refresh='false', filtered='false'}], >> > > timeElapsed: 353ms2018-03-15 10:05:00,239-04 INFO >> > > [org.ovirt.engine.core.bll.network.dc.AddNetworkCommand] (default >> > > task-13) [e1328379-17b7-49f8-beb2-cf8331784828] Failed to Acquire >> > > Lock to object 'EngineLock:{exclusiveLocks='[network_1=NETWORK, >> > > c38a67ec-0b48-4e6f-be85-70c700df5483=PROVIDER]', >> > > sharedLocks=''}'2018-03-15 10:05:00,239-04 WARN >> > > [org.ovirt.engine.core.bll.network.dc.AddNetworkCommand] (default >> > > task-13) [e1328379-17b7-49f8-beb2-cf8331784828] Validation of >> > > action 'AddNetwork' failed for user admin at internal-authz. Reasons: >> > > VAR__TYPE__NETWORK,VAR__ACTION__ADD,ACTION_TYPE_FAILED_PROVIDER_LOCKED,$ >> > providerId >> > > c38a67ec-0b48-4e6f-be85-70c700df54832018-03-15 10:05:00,240-04 >> > > DEBUG >> > > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >> > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] method: >> > > runAction, params: [AddNetwork, >> > > AddNetworkStoragePoolParameters:{commandId='61b365ec-27c1- >> > 49af-ad72-f907df8befcd', >> > > user='null', commandType='Unknown'}], timeElapsed: 10ms2018-03-15 >> > > 10:05:00,250-04 ERROR >> > > [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] >> > > (default task-13) [] Operation Failed: [Cannot add Network. >> > > Related operation on provider with the id >> > > c38a67ec-0b48-4e6f-be85-70c700df5483 is currently in progress. >> > > Please try again later.]2018-03-15 10:05:00,254-04 DEBUG >> > > [org.ovirt.engine.core.utils.servlet.LocaleFilter] (default >> > > task-14) [] Incoming locale 'en-US'. Filter determined locale to >> > > be 'en-US'2018-03-15 10:05:00,254-04 DEBUG >> > > [org.ovirt.engine.core.sso.servlets.OAuthTokenServlet] (default >> > > task-14) [] Entered OAuthTokenServlet Query String: null, >> > > Parameters : password = ***, grant_type = password, scope = >> > > ovirt-app-api ovirt-ext=token-info:validate, username = >> > > admin at internal, * >> > >> > I will care about this. >> > The problem is that SyncNetworkProviderCommand is running in the >> > background and locking the provider, which blocks the lock for the >> > tested AddNetworkCommand. >> > The related changes are >> > core: Add locking for Add and RemoveNetworkCommand >> > https://gerrit.ovirt.org/#/c/85480/ >> > and >> > core: Add SyncNetworkProviderCommand >> > https://gerrit.ovirt.org/#/c/85134/ >> > >> > >> > > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > > From dholler at redhat.com Mon Mar 19 08:40:32 2018 From: dholler at redhat.com (Dominik Holler) Date: Mon, 19 Mar 2018 09:40:32 +0100 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (cockpit-ovirt) ] [ 15-03-2018 ] [ 098_ovirt_provider_ovn.use_ovn_provider ] In-Reply-To: References: <20180315212147.48669c08@t460p> <20180316155252.1fd28b5b@t460p> Message-ID: <20180319094032.16183bde@t460p> On Sat, 17 Mar 2018 16:16:49 +0200 Dan Kenigsberg wrote: > Thanks for filing those, but let us not keep them secret: > > Bug 1557419 - Recommend that a call should be tried again > sounds most reasonable. Providing more information to the client > makes sense > > Bug 1557424 - Automatically retry call failed because he failed to > acquire a lock > is more tricky. when would REST perform the retry? If the command returns a machine-readable hint that the conflict is temporary. > how often? We must make sure we never cause a livelock. > I agree that it is a good idea to have an upper limit for the count of retries to prevent a livelock. > Can we alternatively take a normal blocking lock when we create an > external network (instead of a trylock)? > This might work, but currently no command is doing this. (The lock is acquired in CommandBase.acquireLockInternal() which could be overridden for all commands using an external network provider.) But I would prefer a solution which solve the issue for other commands, too. > On Fri, Mar 16, 2018 at 4:52 PM, Dominik Holler > wrote: > > I have created > > https://bugzilla.redhat.com/show_bug.cgi?id=1557419 and > > https://bugzilla.redhat.com/show_bug.cgi?id=1557424 > > to discuss how failing to acquire locks in engine should be handled > > in engine's REST-API, in ovirt-sdk or in application (OST in this > > case). > > > > > > On Fri, 16 Mar 2018 09:32:47 +0000 > > Dafna Ron wrote: > > > >> Thank you for the fast reply and help. > >> > >> On Thu, Mar 15, 2018 at 8:21 PM, Dominik Holler > >> wrote: > >> > >> > On Thu, 15 Mar 2018 16:24:10 +0000 > >> > Dafna Ron wrote: > >> > > >> > > Hi, > >> > > > >> > > We have a failure on master for test > >> > > 098_ovirt_provider_ovn.use_ovn_provider in project > >> > > cockpit-ovirt. This seems to be a race because object is > >> > > locked. also, the actual failure is logged as WARN and not > >> > > ERROR. > >> > > > >> > > I don't think the patch is actually related to the failure but > >> > > I think the test should be fixed. > >> > > can you please review to make sure we do not have an actual > >> > > regression and let me know if we need to open a bz to fix the > >> > > test? > >> > > > >> > > > >> > > *Link and headline of suspected patches: * > >> > > *https://gerrit.ovirt.org/#/c/89020/2 > >> > > - * > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > *wizard: Enable scroll on start page for low-res screensLink to > >> > > Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6374 > >> > > Link > >> > > to all > >> > > logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue- > >> > tester/6374/artifacts > >> > > >> > tester/6374/artifacts>(Relevant) > >> > > error snippet from the log: 2018-03-15 10:05:00,160-04 > >> > > DEBUG > >> > > [org.ovirt.engine.core.sso.servlets.OAuthTokenInfoServlet] > >> > > (default task-10) [] Sending json response2018-03-15 > >> > > 10:05:00,160-04 DEBUG > >> > > [org.ovirt.engine.core.sso.utils.TokenCleanupUtility] (default > >> > > task-10) [] Not cleaning up expired tokens2018-03-15 > >> > > 10:05:00,169-04 INFO > >> > > [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] > >> > > (EE-ManagedThreadFactory-engineScheduled-Thread-90) [789edb23] > >> > > Lock Acquired to object > >> > > 'EngineLock:{exclusiveLocks='[c38a67ec-0b48-4e6f-be85- > >> > 70c700df5483=PROVIDER]', > >> > > sharedLocks=''}'2018-03-15 10:05:00,184-04 INFO > >> > > [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] > >> > > (EE-ManagedThreadFactory-engineScheduled-Thread-90) [789edb23] > >> > > Running command: SyncNetworkProviderCommand internal: > >> > > true.2018-03-15 10:05:00,228-04 DEBUG > >> > > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$ > >> > PostgresSimpleJdbcCall] > >> > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] > >> > > Compiled stored procedure. Call string is [{call > >> > > getdcidbyexternalnetworkid(?)}]2018-03-15 10:05:00,228-04 DEBUG > >> > > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$ > >> > PostgresSimpleJdbcCall] > >> > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] > >> > > SqlCall for procedure [GetDcIdByExternalNetworkId] > >> > > compiled2018-03-15 10:05:00,229-04 DEBUG > >> > > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > >> > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] > >> > > method: runQuery, params: [GetAllExternalNetworksOnProvider, > >> > > IdQueryParameters:{refresh='false', filtered='false'}], > >> > > timeElapsed: 353ms2018-03-15 10:05:00,239-04 INFO > >> > > [org.ovirt.engine.core.bll.network.dc.AddNetworkCommand] > >> > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] > >> > > Failed to Acquire Lock to object > >> > > 'EngineLock:{exclusiveLocks='[network_1=NETWORK, > >> > > c38a67ec-0b48-4e6f-be85-70c700df5483=PROVIDER]', > >> > > sharedLocks=''}'2018-03-15 10:05:00,239-04 WARN > >> > > [org.ovirt.engine.core.bll.network.dc.AddNetworkCommand] > >> > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] > >> > > Validation of action 'AddNetwork' failed for user > >> > > admin at internal-authz. Reasons: > >> > > VAR__TYPE__NETWORK,VAR__ACTION__ADD,ACTION_TYPE_FAILED_PROVIDER_LOCKED,$ > >> > providerId > >> > > c38a67ec-0b48-4e6f-be85-70c700df54832018-03-15 10:05:00,240-04 > >> > > DEBUG > >> > > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > >> > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] > >> > > method: runAction, params: [AddNetwork, > >> > > AddNetworkStoragePoolParameters:{commandId='61b365ec-27c1- > >> > 49af-ad72-f907df8befcd', > >> > > user='null', commandType='Unknown'}], timeElapsed: > >> > > 10ms2018-03-15 10:05:00,250-04 ERROR > >> > > [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] > >> > > (default task-13) [] Operation Failed: [Cannot add Network. > >> > > Related operation on provider with the id > >> > > c38a67ec-0b48-4e6f-be85-70c700df5483 is currently in progress. > >> > > Please try again later.]2018-03-15 10:05:00,254-04 DEBUG > >> > > [org.ovirt.engine.core.utils.servlet.LocaleFilter] (default > >> > > task-14) [] Incoming locale 'en-US'. Filter determined locale > >> > > to be 'en-US'2018-03-15 10:05:00,254-04 DEBUG > >> > > [org.ovirt.engine.core.sso.servlets.OAuthTokenServlet] (default > >> > > task-14) [] Entered OAuthTokenServlet Query String: null, > >> > > Parameters : password = ***, grant_type = password, scope = > >> > > ovirt-app-api ovirt-ext=token-info:validate, username = > >> > > admin at internal, * > >> > > >> > I will care about this. > >> > The problem is that SyncNetworkProviderCommand is running in the > >> > background and locking the provider, which blocks the lock for > >> > the tested AddNetworkCommand. > >> > The related changes are > >> > core: Add locking for Add and RemoveNetworkCommand > >> > https://gerrit.ovirt.org/#/c/85480/ > >> > and > >> > core: Add SyncNetworkProviderCommand > >> > https://gerrit.ovirt.org/#/c/85134/ > >> > > >> > > >> > > > > > _______________________________________________ > > Devel mailing list > > Devel at ovirt.org > > http://lists.ovirt.org/mailman/listinfo/devel > > > > From dron at redhat.com Mon Mar 19 12:06:59 2018 From: dron at redhat.com (Dafna Ron) Date: Mon, 19 Mar 2018 12:06:59 +0000 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (Ovirt-ngine) ] [ 19-03-2018 ] [ 002_bootstrap.verify_notifier ] Message-ID: Hi, We had a failure in test 002_bootstrap.verify_notifier. I can't see anything wrong with the notifier and I don't think it should be related to the change that was reported. the test itself is looking for vdc_stop in messages which I do not indeed see but I am not sure what is the cause and how the reported change related to the failure. Can you please take a look? *Link and headline of suspected patches: * *core: USB in osinfo configuration depends on chipset - https://gerrit.ovirt.org/#/c/88777/ Link to Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6429/ Link to all logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6429/artifacts (Relevant) error snippet from the log: * *this is the error from *the api: Error Message Failed grep for VDC_STOP with code 1. Output: -------------------- >> begin captured logging << -------------------- lago.ssh: DEBUG: start task:f1231b27-f796-406c-8618-17b0868725bc:Get ssh client for lago-basic-suite-master-engine: lago.ssh: DEBUG: end task:f1231b27-f796-406c-8618-17b0868725bc:Get ssh client for lago-basic-suite-master-engine: lago.ssh: DEBUG: Running 1cce7c0c on lago-basic-suite-master-engine: grep VDC_STOP /var/log/messages lago.ssh: DEBUG: Command 1cce7c0c on lago-basic-suite-master-engine returned with 1 --------------------- >> end captured logging << --------------------- Stacktrace File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 1456, in verify_notifier 'Failed grep for VDC_STOP with code {0}. Output: {1}'.format(result.code, result.out) File "/usr/lib/python2.7/site-packages/nose/tools/trivial.py", line 29, in eq_ raise AssertionError(msg or "%r != %r" % (a, b)) 'Failed grep for VDC_STOP with code 1. Output: \n-------------------- >> begin captured logging << --------------------\nlago.ssh: DEBUG: start task:f1231b27-f796-406c-8618-17b0868725bc:Get ssh client for lago-basic-suite-master-engine:\nlago.ssh: DEBUG: end task:f1231b27-f796-406c-8618-17b0868725bc:Get ssh client for lago-basic-suite-master-engine:\nlago.ssh: DEBUG: Running 1cce7c0c on lago-basic-suite-master-engine: grep VDC_STOP /var/log/messages\nlago.ssh: DEBUG: Command 1cce7c0c on lago-basic-suite-master-engine returned with 1\n--------------------- >> end captured logging << ---------------------' ** -------------- next part -------------- An HTML attachment was scrubbed... URL: From dron at redhat.com Mon Mar 19 12:25:05 2018 From: dron at redhat.com (Dafna Ron) Date: Mon, 19 Mar 2018 12:25:05 +0000 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 19-03-2018 ] [002_bootstrap.add_instance_type ] Message-ID: Hi, We have a failure in test 002_bootstrap.add_instance_type. There seem to be a NullPointerException on template type which is causing this failure. The same change that was reported at the last failure is reported as the root cause for this failure as well, but I am not sure how it would cause this failure. Can you please check? *Link and headline of suspected patches: reported as failed: * *core: fix removal of vm-host device - https://gerrit.ovirt.org/#/c/89145/ * *reported as root cause: * *core: USB in osinfo configuration depends on chipset - https://gerrit.ovirt.org/#/c/88777/ * *Link to Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6431 Link to all logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6431/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-002_bootstrap.py/ (Relevant) error snippet from the log: 2018-03-19 04:59:01,040-04 INFO [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] Lock Acquired to object 'EngineLock:{exclusiveLocks='[99a9dfb3-9a13-4595-a795-693493e722be=TEMPLATE, myinstancetype=TEMPLATE_NAME]', sharedLocks='[]'}'2018-03-19 04:59:01,087-04 INFO [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] Running command: AddVmTemplateCommand internal: false. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_TEMPLATE with role type USER2018-03-19 04:59:01,139-04 INFO [org.ovirt.engine.core.bll.storage.disk.CreateAllTemplateDisksCommand] (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] Running command: CreateAllTemplateDisksCommand internal: true.2018-03-19 04:59:01,205-04 INFO [org.ovirt.engine.core.utils.transaction.TransactionSupport] (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] transaction rolled back2018-03-19 04:59:01,205-04 ERROR [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] Command 'org.ovirt.engine.core.bll.AddVmTemplateCommand' failed: null2018-03-19 04:59:01,205-04 ERROR [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] Exception: java.lang.NullPointerException at org.ovirt.engine.core.bll.utils.EmulatedMachineUtils.getEffective(EmulatedMachineUtils.java:30) [bll.jar:] at org.ovirt.engine.core.bll.utils.EmulatedMachineUtils.getEffectiveChipset(EmulatedMachineUtils.java:21) [bll.jar:] at org.ovirt.engine.core.bll.utils.VmDeviceUtils.updateUsbSlots(VmDeviceUtils.java:744) [bll.jar:] at org.ovirt.engine.core.bll.utils.VmDeviceUtils.copyVmDevices(VmDeviceUtils.java:1519) [bll.jar:] at org.ovirt.engine.core.bll.utils.VmDeviceUtils.copyVmDevices(VmDeviceUtils.java:1565) [bll.jar:] at org.ovirt.engine.core.bll.AddVmTemplateCommand.lambda$executeCommand$4(AddVmTemplateCommand.java:362) [bll.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:202) [utils.jar:] at org.ovirt.engine.core.bll.AddVmTemplateCommand.executeCommand(AddVmTemplateCommand.java:345) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1133) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1286) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1935) [bll.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:164) [utils.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:103) [utils.jar:] at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1346) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:400) [bll.jar:] at org.ovirt.engine.core.bll.executor.DefaultBackendActionExecutor.execute(DefaultBackendActionExecutor.java:13) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:468) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:450) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:403) [bll.jar:] at sun.reflect.GeneratedMethodAccessor149.invoke(Unknown Source) [:1.8.0_161] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_161] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_161] at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:92) [wildfly-weld-ejb-11.0.0.Final.jar:11.0.0.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContext.interceptorChainCompleted(WeldInvocationContext.java:98) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContext.proceed(WeldInvocationContext.java:117) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12) [common.jar:] at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source) [:1.8.0_161] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_161] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_161] at org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContext.invokeNext(WeldInvocationContext.java:83) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContext.proceed(WeldInvocationContext.java:115) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.weld.bean.InterceptorImpl.intercept(InterceptorImpl.java:108) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] at org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:82) [wildfly-weld-ejb-11.0.0.Final.jar:11.0.0.Final] at org.jboss.as.weld.interceptors.EjbComponentInterceptorSupport.delegateInterception(EjbComponentInterceptorSupport.java:60) at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:76) at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:88) at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:101) at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at org.ovirt.engine.core.bll.interceptors.CorrelationIdTrackerInterceptor.aroundInvoke(CorrelationIdTrackerInterceptor.java:13) [bll.jar:] at sun.reflect.GeneratedMethodAccessor66.invoke(Unknown Source) [:1.8.0_161] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_161] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_161] at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:89) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422):* *End of error: * at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:326) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:812) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_161] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_161] at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_161] 2018-03-19 04:59:01,234-04 DEBUG [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall] (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] Compiled stored procedure. Call string is [{call get_entity_snapshot_by_command_id(?)}] 2018-03-19 04:59:01,234-04 DEBUG [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall] (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] SqlCall for procedure [get_entity_snapshot_by_command_id] compiled 2018-03-19 04:59:01,239-04 INFO [org.ovirt.engine.core.bll.CommandCompensator] (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] Command [id=a7a51354-ae84-4109-a55e-b27558afbf2a]: Compensating NEW_ENTITY_ID of org.ovirt.engine.core.common.businessentities.VmTemplate; snapshot: 99a9dfb3-9a13-4595-a795-693493e722be. 2018-03-19 04:59:01,278-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] EVENT_ID: USER_ADD_VM_TEMPLATE_FAILURE(36), Failed creating Template myinstancetype. 2018-03-19 04:59:01,295-04 INFO [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] Lock freed to object 'EngineLock:{exclusiveLocks='[99a9dfb3-9a13-4595-a795-693493e722be=TEMPLATE, myinstancetype=TEMPLATE_NAME]', sharedLocks='[]'}' 2018-03-19 04:59:01,295-04 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] method: runAction, params: [AddVmTemplate, AddVmTemplateParameters:{commandId='a7a51354-ae84-4109-a55e-b27558afbf2a', user='admin', commandType='AddVmTemplate'}], timeElapsed: 301ms 2018-03-19 04:59:01,301-04 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-5) [] Operation Failed: [Internal Engine Error] 2018-03-19 04:59:01,547-04 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [5a890b17-51ec-4398-8d64-82cc71939e6e] Command 'AddVmTemplate' id: 'a7a51354-ae84-4109-a55e-b27558afbf2a' execution didn't complete, not proceeding to perform the next operation 2018-03-19 04:59:01,547-04 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [5a890b17-51ec-4398-8d64-82cc71939e6e] Command 'AddVmTemplate' id: 'a7a51354-ae84-4109-a55e-b27558afbf2a' child commands '[8484f4d2-9ddf-48bd-9877-7c99ef555255]' executions were completed, status 'FAILED' 2018-03-19 04:59:01,581-04 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [db5dd3dd-8b48-4647-a02d-723fcd6a7dda] Command 'ImportRepoImage' (id: 'a6475141-53ef-4ecd-bc17-ee8eae168dc7') waiting on child command id: '528fde29-9b5b-4a90-b62a-606fac3cd6a6' type:'DownloadImage' to complete 2018-03-19 04:59:02,363-04 DEBUG [org.ovirt.otopi.dialog.MachineDialogParser] (VdsDeploy) [665e1c35] Got: ***L:INFO Yum install: 349/515: gdisk-0.8.6-5.el7.x86_64 2018-03-19 04:59:02,364-04 DEBUG [org.ovirt.otopi.dialog.MachineDialogParser] (VdsDeploy) [665e1c35] nextEvent: Log INFO Yum install: 349/515: gdisk-0.8.6-5.el7.x86_64 2018-03-19 04:59:02,570-04 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) [665e1c35] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), Installing Host lago-basic-suite-master-host-1. Yum install: 349/515: gdisk-0.8.6-5.el7.x86_64. 2018-03-19 04:59:02,666-04 ERROR [org.ovirt.engine.core.bll.AddVmTemplateCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-86) [5a890b17-51ec-4398-8d64-82cc71939e6e] Ending command 'org.ovirt.engine.core.bll.AddVmTemplateCommand' with failure. ** -------------- next part -------------- An HTML attachment was scrubbed... URL: From smelamud at redhat.com Mon Mar 19 13:25:50 2018 From: smelamud at redhat.com (Shmuel Melamud) Date: Mon, 19 Mar 2018 15:25:50 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 19-03-2018 ] [002_bootstrap.add_instance_type ] In-Reply-To: References: Message-ID: Hi! Forgot about instance type that don't have a cluster. Fixing it now. Shmuel On Mon, Mar 19, 2018 at 2:25 PM, Dafna Ron wrote: > Hi, > > We have a failure in test 002_bootstrap.add_instance_type. > There seem to be a NullPointerException on template type which is causing > this failure. > The same change that was reported at the last failure is reported as the > root cause for this failure as well, but I am not sure how it would cause > this failure. > Can you please check? > > > Link and headline of suspected patches: > > reported as failed: > core: fix removal of vm-host device - https://gerrit.ovirt.org/#/c/89145/ > > reported as root cause: > > core: USB in osinfo configuration depends on chipset - > https://gerrit.ovirt.org/#/c/88777/ > > Link to Job: > > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6431 > > Link to all logs: > > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6431/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-002_bootstrap.py/ > > (Relevant) error snippet from the log: > > > > > 2018-03-19 04:59:01,040-04 INFO > [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-5) > [5a890b17-51ec-4398-8d64-82cc71939e6e] Lock Acquired to object > 'EngineLock:{exclusiveLocks='[99a9dfb3-9a13-4595-a795-693493e722be=TEMPLATE, > myinstancetype=TEMPLATE_NAME]', sharedLocks='[]'}' > 2018-03-19 04:59:01,087-04 INFO > [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-5) > [5a890b17-51ec-4398-8d64-82cc71939e6e] Running command: AddVmTemplateCommand > internal: false. Entities affected : ID: aaa00000-0000-0000-0 > 000-123456789aaa Type: SystemAction group CREATE_TEMPLATE with role type > USER > 2018-03-19 04:59:01,139-04 INFO > [org.ovirt.engine.core.bll.storage.disk.CreateAllTemplateDisksCommand] > (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] Running command: > CreateAllTemplateDisksCommand internal: true. > 2018-03-19 04:59:01,205-04 INFO > [org.ovirt.engine.core.utils.transaction.TransactionSupport] (default > task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] transaction rolled back > 2018-03-19 04:59:01,205-04 ERROR > [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-5) > [5a890b17-51ec-4398-8d64-82cc71939e6e] Command > 'org.ovirt.engine.core.bll.AddVmTemplateCommand' failed: null > 2018-03-19 04:59:01,205-04 ERROR > [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-5) > [5a890b17-51ec-4398-8d64-82cc71939e6e] Exception: > java.lang.NullPointerException > at > org.ovirt.engine.core.bll.utils.EmulatedMachineUtils.getEffective(EmulatedMachineUtils.java:30) > [bll.jar:] > at > org.ovirt.engine.core.bll.utils.EmulatedMachineUtils.getEffectiveChipset(EmulatedMachineUtils.java:21) > [bll.jar:] > at > org.ovirt.engine.core.bll.utils.VmDeviceUtils.updateUsbSlots(VmDeviceUtils.java:744) > [bll.jar:] > at > org.ovirt.engine.core.bll.utils.VmDeviceUtils.copyVmDevices(VmDeviceUtils.java:1519) > [bll.jar:] > at > org.ovirt.engine.core.bll.utils.VmDeviceUtils.copyVmDevices(VmDeviceUtils.java:1565) > [bll.jar:] > at > org.ovirt.engine.core.bll.AddVmTemplateCommand.lambda$executeCommand$4(AddVmTemplateCommand.java:362) > [bll.jar:] > at > org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:202) > [utils.jar:] > at > org.ovirt.engine.core.bll.AddVmTemplateCommand.executeCommand(AddVmTemplateCommand.java:345) > [bll.jar:] > at > org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1133) > [bll.jar:] > at > org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1286) > [bll.jar:] > at > org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1935) > [bll.jar:] > at > org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:164) > [utils.jar:] > at > org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:103) > [utils.jar:] > at > org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1346) > [bll.jar:] > at > org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:400) > [bll.jar:] > at > org.ovirt.engine.core.bll.executor.DefaultBackendActionExecutor.execute(DefaultBackendActionExecutor.java:13) > [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:468) > [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:450) > [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:403) > [bll.jar:] > at sun.reflect.GeneratedMethodAccessor149.invoke(Unknown Source) > [:1.8.0_161] > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > [rt.jar:1.8.0_161] > at java.lang.reflect.Method.invoke(Method.java:498) > [rt.jar:1.8.0_161] > at > org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) > at > org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:92) > [wildfly-weld-ejb-11.0.0.Final.jar:11.0.0.Final] > at > org.jboss.weld.interceptor.proxy.WeldInvocationContext.interceptorChainCompleted(WeldInvocationContext.java:98) > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > at > org.jboss.weld.interceptor.proxy.WeldInvocationContext.proceed(WeldInvocationContext.java:117) > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > at > org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12) > [common.jar:] > at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source) > [:1.8.0_161] > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > [rt.jar:1.8.0_161] > at java.lang.reflect.Method.invoke(Method.java:498) > [rt.jar:1.8.0_161] > at > org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73) > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > at > org.jboss.weld.interceptor.proxy.WeldInvocationContext.invokeNext(WeldInvocationContext.java:83) > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > at > org.jboss.weld.interceptor.proxy.WeldInvocationContext.proceed(WeldInvocationContext.java:115) > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > at > org.jboss.weld.bean.InterceptorImpl.intercept(InterceptorImpl.java:108) > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > at > org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:82) > [wildfly-weld-ejb-11.0.0.Final.jar:11.0.0.Final] > at > org.jboss.as.weld.interceptors.EjbComponentInterceptorSupport.delegateInterception(EjbComponentInterceptorSupport.java:60) > at > org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:76) > at > org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:88) > at > org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:101) > at > org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at > org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) > at > org.ovirt.engine.core.bll.interceptors.CorrelationIdTrackerInterceptor.aroundInvoke(CorrelationIdTrackerInterceptor.java:13) > [bll.jar:] > at sun.reflect.GeneratedMethodAccessor66.invoke(Unknown Source) > [:1.8.0_161] > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > [rt.jar:1.8.0_161] > at java.lang.reflect.Method.invoke(Method.java:498) > [rt.jar:1.8.0_161] > at > org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:89) > at > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > : > > > End of error: > > at > io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) > at > org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) > at > org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) > at > org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) > at > org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) > at > io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) > at > io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) > at > io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) > at > io.undertow.server.Connectors.executeRootHandler(Connectors.java:326) > at > io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:812) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > [rt.jar:1.8.0_161] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [rt.jar:1.8.0_161] > at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_161] > > 2018-03-19 04:59:01,234-04 DEBUG > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall] > (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] Compiled stored > procedure. Call string is [{call get_entity_snapshot_by_command_id(?)}] > 2018-03-19 04:59:01,234-04 DEBUG > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall] > (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] SqlCall for > procedure [get_entity_snapshot_by_command_id] compiled > 2018-03-19 04:59:01,239-04 INFO > [org.ovirt.engine.core.bll.CommandCompensator] (default task-5) > [5a890b17-51ec-4398-8d64-82cc71939e6e] Command > [id=a7a51354-ae84-4109-a55e-b27558afbf2a]: Compensating NEW_ENTITY_ID of > org.ovirt.engine.core.common.businessentities.VmTemplate; snapshot: > 99a9dfb3-9a13-4595-a795-693493e722be. > 2018-03-19 04:59:01,278-04 ERROR > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] EVENT_ID: > USER_ADD_VM_TEMPLATE_FAILURE(36), Failed creating Template myinstancetype. > 2018-03-19 04:59:01,295-04 INFO > [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-5) > [5a890b17-51ec-4398-8d64-82cc71939e6e] Lock freed to object > 'EngineLock:{exclusiveLocks='[99a9dfb3-9a13-4595-a795-693493e722be=TEMPLATE, > myinstancetype=TEMPLATE_NAME]', sharedLocks='[]'}' > 2018-03-19 04:59:01,295-04 DEBUG > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] method: runAction, > params: [AddVmTemplate, > AddVmTemplateParameters:{commandId='a7a51354-ae84-4109-a55e-b27558afbf2a', > user='admin', commandType='AddVmTemplate'}], timeElapsed: 301ms > 2018-03-19 04:59:01,301-04 ERROR > [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default > task-5) [] Operation Failed: [Internal Engine Error] > 2018-03-19 04:59:01,547-04 INFO > [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] > (EE-ManagedThreadFactory-engineScheduled-Thread-83) > [5a890b17-51ec-4398-8d64-82cc71939e6e] Command 'AddVmTemplate' id: > 'a7a51354-ae84-4109-a55e-b27558afbf2a' execution didn't complete, not > proceeding to perform the next operation > 2018-03-19 04:59:01,547-04 INFO > [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] > (EE-ManagedThreadFactory-engineScheduled-Thread-83) > [5a890b17-51ec-4398-8d64-82cc71939e6e] Command 'AddVmTemplate' id: > 'a7a51354-ae84-4109-a55e-b27558afbf2a' child commands > '[8484f4d2-9ddf-48bd-9877-7c99ef555255]' executions were completed, status > 'FAILED' > 2018-03-19 04:59:01,581-04 INFO > [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] > (EE-ManagedThreadFactory-engineScheduled-Thread-83) > [db5dd3dd-8b48-4647-a02d-723fcd6a7dda] Command 'ImportRepoImage' (id: > 'a6475141-53ef-4ecd-bc17-ee8eae168dc7') waiting on child command id: > '528fde29-9b5b-4a90-b62a-606fac3cd6a6' type:'DownloadImage' to complete > 2018-03-19 04:59:02,363-04 DEBUG > [org.ovirt.otopi.dialog.MachineDialogParser] (VdsDeploy) [665e1c35] Got: > ***L:INFO Yum install: 349/515: gdisk-0.8.6-5.el7.x86_64 > 2018-03-19 04:59:02,364-04 DEBUG > [org.ovirt.otopi.dialog.MachineDialogParser] (VdsDeploy) [665e1c35] > nextEvent: Log INFO Yum install: 349/515: gdisk-0.8.6-5.el7.x86_64 > 2018-03-19 04:59:02,570-04 INFO > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > (VdsDeploy) [665e1c35] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), Installing > Host lago-basic-suite-master-host-1. Yum install: 349/515: > gdisk-0.8.6-5.el7.x86_64. > 2018-03-19 04:59:02,666-04 ERROR > [org.ovirt.engine.core.bll.AddVmTemplateCommand] > (EE-ManagedThreadFactory-engineScheduled-Thread-86) > [5a890b17-51ec-4398-8d64-82cc71939e6e] Ending command > 'org.ovirt.engine.core.bll.AddVmTemplateCommand' with failure. > > > > > > From dron at redhat.com Mon Mar 19 14:12:54 2018 From: dron at redhat.com (Dafna Ron) Date: Mon, 19 Mar 2018 14:12:54 +0000 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 19-03-2018 ] [002_bootstrap.add_instance_type ] In-Reply-To: References: Message-ID: thank you for the fast response. once you have the fix, can you please sent it to us? On Mon, Mar 19, 2018 at 1:25 PM, Shmuel Melamud wrote: > Hi! > > Forgot about instance type that don't have a cluster. Fixing it now. > > Shmuel > > On Mon, Mar 19, 2018 at 2:25 PM, Dafna Ron wrote: > > Hi, > > > > We have a failure in test 002_bootstrap.add_instance_type. > > There seem to be a NullPointerException on template type which is causing > > this failure. > > The same change that was reported at the last failure is reported as the > > root cause for this failure as well, but I am not sure how it would cause > > this failure. > > Can you please check? > > > > > > Link and headline of suspected patches: > > > > reported as failed: > > core: fix removal of vm-host device - https://gerrit.ovirt.org/#/c/ > 89145/ > > > > reported as root cause: > > > > core: USB in osinfo configuration depends on chipset - > > https://gerrit.ovirt.org/#/c/88777/ > > > > Link to Job: > > > > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6431 > > > > Link to all logs: > > > > http://jenkins.ovirt.org/job/ovirt-master_change-queue- > tester/6431/artifact/exported-artifacts/basic-suit-master- > el7/test_logs/basic-suite-master/post-002_bootstrap.py/ > > > > (Relevant) error snippet from the log: > > > > > > > > > > 2018-03-19 04:59:01,040-04 INFO > > [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-5) > > [5a890b17-51ec-4398-8d64-82cc71939e6e] Lock Acquired to object > > 'EngineLock:{exclusiveLocks='[99a9dfb3-9a13-4595-a795- > 693493e722be=TEMPLATE, > > myinstancetype=TEMPLATE_NAME]', sharedLocks='[]'}' > > 2018-03-19 04:59:01,087-04 INFO > > [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-5) > > [5a890b17-51ec-4398-8d64-82cc71939e6e] Running command: > AddVmTemplateCommand > > internal: false. Entities affected : ID: aaa00000-0000-0000-0 > > 000-123456789aaa Type: SystemAction group CREATE_TEMPLATE with role type > > USER > > 2018-03-19 04:59:01,139-04 INFO > > [org.ovirt.engine.core.bll.storage.disk.CreateAllTemplateDisksCommand] > > (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] Running command: > > CreateAllTemplateDisksCommand internal: true. > > 2018-03-19 04:59:01,205-04 INFO > > [org.ovirt.engine.core.utils.transaction.TransactionSupport] (default > > task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] transaction rolled back > > 2018-03-19 04:59:01,205-04 ERROR > > [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-5) > > [5a890b17-51ec-4398-8d64-82cc71939e6e] Command > > 'org.ovirt.engine.core.bll.AddVmTemplateCommand' failed: null > > 2018-03-19 04:59:01,205-04 ERROR > > [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-5) > > [5a890b17-51ec-4398-8d64-82cc71939e6e] Exception: > > java.lang.NullPointerException > > at > > org.ovirt.engine.core.bll.utils.EmulatedMachineUtils.getEffective( > EmulatedMachineUtils.java:30) > > [bll.jar:] > > at > > org.ovirt.engine.core.bll.utils.EmulatedMachineUtils. > getEffectiveChipset(EmulatedMachineUtils.java:21) > > [bll.jar:] > > at > > org.ovirt.engine.core.bll.utils.VmDeviceUtils. > updateUsbSlots(VmDeviceUtils.java:744) > > [bll.jar:] > > at > > org.ovirt.engine.core.bll.utils.VmDeviceUtils. > copyVmDevices(VmDeviceUtils.java:1519) > > [bll.jar:] > > at > > org.ovirt.engine.core.bll.utils.VmDeviceUtils. > copyVmDevices(VmDeviceUtils.java:1565) > > [bll.jar:] > > at > > org.ovirt.engine.core.bll.AddVmTemplateCommand.lambda$executeCommand$4( > AddVmTemplateCommand.java:362) > > [bll.jar:] > > at > > org.ovirt.engine.core.utils.transaction.TransactionSupport. > executeInNewTransaction(TransactionSupport.java:202) > > [utils.jar:] > > at > > org.ovirt.engine.core.bll.AddVmTemplateCommand.executeCommand( > AddVmTemplateCommand.java:345) > > [bll.jar:] > > at > > org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction( > CommandBase.java:1133) > > [bll.jar:] > > at > > org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScop > e(CommandBase.java:1286) > > [bll.jar:] > > at > > org.ovirt.engine.core.bll.CommandBase.runInTransaction( > CommandBase.java:1935) > > [bll.jar:] > > at > > org.ovirt.engine.core.utils.transaction.TransactionSupport. > executeInSuppressed(TransactionSupport.java:164) > > [utils.jar:] > > at > > org.ovirt.engine.core.utils.transaction.TransactionSupport. > executeInScope(TransactionSupport.java:103) > > [utils.jar:] > > at > > org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1346) > > [bll.jar:] > > at > > org.ovirt.engine.core.bll.CommandBase.executeAction( > CommandBase.java:400) > > [bll.jar:] > > at > > org.ovirt.engine.core.bll.executor.DefaultBackendActionExecutor.execute( > DefaultBackendActionExecutor.java:13) > > [bll.jar:] > > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:468) > > [bll.jar:] > > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend. > java:450) > > [bll.jar:] > > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:403) > > [bll.jar:] > > at sun.reflect.GeneratedMethodAccessor149.invoke(Unknown Source) > > [:1.8.0_161] > > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:43) > > [rt.jar:1.8.0_161] > > at java.lang.reflect.Method.invoke(Method.java:498) > > [rt.jar:1.8.0_161] > > at > > org.jboss.as.ee.component.ManagedReferenceMethodIntercep > tor.processInvocation(ManagedReferenceMethodInterceptor.java:52) > > at > > org.jboss.invocation.InterceptorContext.proceed( > InterceptorContext.java:422) > > at > > org.jboss.invocation.InterceptorContext$Invocation. > proceed(InterceptorContext.java:509) > > at > > org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed( > DelegatingInterceptorInvocationContext.java:92) > > [wildfly-weld-ejb-11.0.0.Final.jar:11.0.0.Final] > > at > > org.jboss.weld.interceptor.proxy.WeldInvocationContext. > interceptorChainCompleted(WeldInvocationContext.java:98) > > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > > at > > org.jboss.weld.interceptor.proxy.WeldInvocationContext. > proceed(WeldInvocationContext.java:117) > > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > > at > > org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply( > LoggingInterceptor.java:12) > > [common.jar:] > > at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source) > > [:1.8.0_161] > > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:43) > > [rt.jar:1.8.0_161] > > at java.lang.reflect.Method.invoke(Method.java:498) > > [rt.jar:1.8.0_161] > > at > > org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$ > SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73) > > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > > at > > org.jboss.weld.interceptor.proxy.WeldInvocationContext.invokeNext( > WeldInvocationContext.java:83) > > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > > at > > org.jboss.weld.interceptor.proxy.WeldInvocationContext. > proceed(WeldInvocationContext.java:115) > > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > > at > > org.jboss.weld.bean.InterceptorImpl.intercept(InterceptorImpl.java:108) > > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > > at > > org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed( > DelegatingInterceptorInvocationContext.java:82) > > [wildfly-weld-ejb-11.0.0.Final.jar:11.0.0.Final] > > at > > org.jboss.as.weld.interceptors.EjbComponentInterceptorSupport > .delegateInterception(EjbComponentInterceptorSupport.java:60) > > at > > org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor. > delegateInterception(Jsr299BindingsInterceptor.java:76) > > at > > org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor. > doMethodInterception(Jsr299BindingsInterceptor.java:88) > > at > > org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor. > processInvocation(Jsr299BindingsInterceptor.java:101) > > at > > org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1. > processInvocation(UserInterceptorFactory.java:63) > > at > > org.jboss.invocation.InterceptorContext.proceed( > InterceptorContext.java:422) > > at > > org.jboss.invocation.InterceptorContext$Invocation. > proceed(InterceptorContext.java:509) > > at > > org.ovirt.engine.core.bll.interceptors.CorrelationIdTrackerIntercepto > r.aroundInvoke(CorrelationIdTrackerInterceptor.java:13) > > [bll.jar:] > > at sun.reflect.GeneratedMethodAccessor66.invoke(Unknown Source) > > [:1.8.0_161] > > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:43) > > [rt.jar:1.8.0_161] > > at java.lang.reflect.Method.invoke(Method.java:498) > > [rt.jar:1.8.0_161] > > at > > org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor. > processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:89) > > at > > org.jboss.invocation.InterceptorContext.proceed( > InterceptorContext.java:422) > > : > > > > > > End of error: > > > > at > > io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call( > ContextClassLoaderSetupAction.java:43) > > at > > org.wildfly.extension.undertow.security.SecurityContextThreadSetupActi > on.lambda$create$0(SecurityContextThreadSetupAction.java:105) > > at > > org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$ > UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService. > java:1508) > > at > > org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$ > UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService. > java:1508) > > at > > org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$ > UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService. > java:1508) > > at > > io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest( > ServletInitialHandler.java:272) > > at > > io.undertow.servlet.handlers.ServletInitialHandler.access$ > 000(ServletInitialHandler.java:81) > > at > > io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest( > ServletInitialHandler.java:104) > > at > > io.undertow.server.Connectors.executeRootHandler(Connectors.java:326) > > at > > io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:812) > > at > > java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1149) > > [rt.jar:1.8.0_161] > > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:624) > > [rt.jar:1.8.0_161] > > at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_161] > > > > 2018-03-19 04:59:01,234-04 DEBUG > > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$ > PostgresSimpleJdbcCall] > > (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] Compiled stored > > procedure. Call string is [{call get_entity_snapshot_by_command_id(?)}] > > 2018-03-19 04:59:01,234-04 DEBUG > > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$ > PostgresSimpleJdbcCall] > > (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] SqlCall for > > procedure [get_entity_snapshot_by_command_id] compiled > > 2018-03-19 04:59:01,239-04 INFO > > [org.ovirt.engine.core.bll.CommandCompensator] (default task-5) > > [5a890b17-51ec-4398-8d64-82cc71939e6e] Command > > [id=a7a51354-ae84-4109-a55e-b27558afbf2a]: Compensating NEW_ENTITY_ID of > > org.ovirt.engine.core.common.businessentities.VmTemplate; snapshot: > > 99a9dfb3-9a13-4595-a795-693493e722be. > > 2018-03-19 04:59:01,278-04 ERROR > > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > > (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] EVENT_ID: > > USER_ADD_VM_TEMPLATE_FAILURE(36), Failed creating Template > myinstancetype. > > 2018-03-19 04:59:01,295-04 INFO > > [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-5) > > [5a890b17-51ec-4398-8d64-82cc71939e6e] Lock freed to object > > 'EngineLock:{exclusiveLocks='[99a9dfb3-9a13-4595-a795- > 693493e722be=TEMPLATE, > > myinstancetype=TEMPLATE_NAME]', sharedLocks='[]'}' > > 2018-03-19 04:59:01,295-04 DEBUG > > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > > (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] method: > runAction, > > params: [AddVmTemplate, > > AddVmTemplateParameters:{commandId='a7a51354-ae84-4109- > a55e-b27558afbf2a', > > user='admin', commandType='AddVmTemplate'}], timeElapsed: 301ms > > 2018-03-19 04:59:01,301-04 ERROR > > [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default > > task-5) [] Operation Failed: [Internal Engine Error] > > 2018-03-19 04:59:01,547-04 INFO > > [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] > > (EE-ManagedThreadFactory-engineScheduled-Thread-83) > > [5a890b17-51ec-4398-8d64-82cc71939e6e] Command 'AddVmTemplate' id: > > 'a7a51354-ae84-4109-a55e-b27558afbf2a' execution didn't complete, not > > proceeding to perform the next operation > > 2018-03-19 04:59:01,547-04 INFO > > [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] > > (EE-ManagedThreadFactory-engineScheduled-Thread-83) > > [5a890b17-51ec-4398-8d64-82cc71939e6e] Command 'AddVmTemplate' id: > > 'a7a51354-ae84-4109-a55e-b27558afbf2a' child commands > > '[8484f4d2-9ddf-48bd-9877-7c99ef555255]' executions were completed, > status > > 'FAILED' > > 2018-03-19 04:59:01,581-04 INFO > > [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] > > (EE-ManagedThreadFactory-engineScheduled-Thread-83) > > [db5dd3dd-8b48-4647-a02d-723fcd6a7dda] Command 'ImportRepoImage' (id: > > 'a6475141-53ef-4ecd-bc17-ee8eae168dc7') waiting on child command id: > > '528fde29-9b5b-4a90-b62a-606fac3cd6a6' type:'DownloadImage' to complete > > 2018-03-19 04:59:02,363-04 DEBUG > > [org.ovirt.otopi.dialog.MachineDialogParser] (VdsDeploy) [665e1c35] Got: > > ***L:INFO Yum install: 349/515: gdisk-0.8.6-5.el7.x86_64 > > 2018-03-19 04:59:02,364-04 DEBUG > > [org.ovirt.otopi.dialog.MachineDialogParser] (VdsDeploy) [665e1c35] > > nextEvent: Log INFO Yum install: 349/515: gdisk-0.8.6-5.el7.x86_64 > > 2018-03-19 04:59:02,570-04 INFO > > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > > (VdsDeploy) [665e1c35] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), Installing > > Host lago-basic-suite-master-host-1. Yum install: 349/515: > > gdisk-0.8.6-5.el7.x86_64. > > 2018-03-19 04:59:02,666-04 ERROR > > [org.ovirt.engine.core.bll.AddVmTemplateCommand] > > (EE-ManagedThreadFactory-engineScheduled-Thread-86) > > [5a890b17-51ec-4398-8d64-82cc71939e6e] Ending command > > 'org.ovirt.engine.core.bll.AddVmTemplateCommand' with failure. > > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dholler at redhat.com Mon Mar 19 14:38:44 2018 From: dholler at redhat.com (Dominik Holler) Date: Mon, 19 Mar 2018 15:38:44 +0100 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (cockpit-ovirt) ] [ 15-03-2018 ] [ 098_ovirt_provider_ovn.use_ovn_provider ] In-Reply-To: <20180319094032.16183bde@t460p> References: <20180315212147.48669c08@t460p> <20180316155252.1fd28b5b@t460p> <20180319094032.16183bde@t460p> Message-ID: <20180319153844.0f657d1c@t460p> I have overseen that many other commands handle this problem by using a waiting lock, so the related commands in this issue should do the same. I created Adding a new external network fails during auto-sync is running https://bugzilla.redhat.com/show_bug.cgi?id=1558054 to track this. From my point of view the other related two bugs I created are not required anymore, because other commands seems to use a waiting lock, too. Should we close 1557419 and 1557424? On Mon, 19 Mar 2018 09:40:32 +0100 Dominik Holler wrote: > On Sat, 17 Mar 2018 16:16:49 +0200 > Dan Kenigsberg wrote: > > > Thanks for filing those, but let us not keep them secret: > > > > Bug 1557419 - Recommend that a call should be tried again > > sounds most reasonable. Providing more information to the client > > makes sense > > > > Bug 1557424 - Automatically retry call failed because he failed to > > acquire a lock > > is more tricky. when would REST perform the retry? > > If the command returns a machine-readable hint that the conflict is > temporary. > > > how often? We must make sure we never cause a livelock. > > > > I agree that it is a good idea to have an upper limit for the count of > retries to prevent a livelock. > > > Can we alternatively take a normal blocking lock when we create an > > external network (instead of a trylock)? > > > > This might work, but currently no command is doing this. > (The lock is acquired in CommandBase.acquireLockInternal() which could > be overridden for all commands using an external network provider.) > But I would prefer a solution which solve the issue for other > commands, too. > > > On Fri, Mar 16, 2018 at 4:52 PM, Dominik Holler > > wrote: > > > I have created > > > https://bugzilla.redhat.com/show_bug.cgi?id=1557419 and > > > https://bugzilla.redhat.com/show_bug.cgi?id=1557424 > > > to discuss how failing to acquire locks in engine should be > > > handled in engine's REST-API, in ovirt-sdk or in application (OST > > > in this case). > > > > > > > > > On Fri, 16 Mar 2018 09:32:47 +0000 > > > Dafna Ron wrote: > > > > > >> Thank you for the fast reply and help. > > >> > > >> On Thu, Mar 15, 2018 at 8:21 PM, Dominik Holler > > >> wrote: > > >> > > >> > On Thu, 15 Mar 2018 16:24:10 +0000 > > >> > Dafna Ron wrote: > > >> > > > >> > > Hi, > > >> > > > > >> > > We have a failure on master for test > > >> > > 098_ovirt_provider_ovn.use_ovn_provider in project > > >> > > cockpit-ovirt. This seems to be a race because object is > > >> > > locked. also, the actual failure is logged as WARN and not > > >> > > ERROR. > > >> > > > > >> > > I don't think the patch is actually related to the failure > > >> > > but I think the test should be fixed. > > >> > > can you please review to make sure we do not have an actual > > >> > > regression and let me know if we need to open a bz to fix the > > >> > > test? > > >> > > > > >> > > > > >> > > *Link and headline of suspected patches: * > > >> > > *https://gerrit.ovirt.org/#/c/89020/2 > > >> > > - * > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > *wizard: Enable scroll on start page for low-res screensLink > > >> > > to > > >> > > Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6374 > > >> > > Link > > >> > > to all > > >> > > logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue- > > >> > tester/6374/artifacts > > >> > > > >> > tester/6374/artifacts>(Relevant) > > >> > > error snippet from the log: 2018-03-15 10:05:00,160-04 > > >> > > DEBUG > > >> > > [org.ovirt.engine.core.sso.servlets.OAuthTokenInfoServlet] > > >> > > (default task-10) [] Sending json response2018-03-15 > > >> > > 10:05:00,160-04 DEBUG > > >> > > [org.ovirt.engine.core.sso.utils.TokenCleanupUtility] > > >> > > (default task-10) [] Not cleaning up expired tokens2018-03-15 > > >> > > 10:05:00,169-04 INFO > > >> > > [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] > > >> > > (EE-ManagedThreadFactory-engineScheduled-Thread-90) > > >> > > [789edb23] Lock Acquired to object > > >> > > 'EngineLock:{exclusiveLocks='[c38a67ec-0b48-4e6f-be85- > > >> > 70c700df5483=PROVIDER]', > > >> > > sharedLocks=''}'2018-03-15 10:05:00,184-04 INFO > > >> > > [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] > > >> > > (EE-ManagedThreadFactory-engineScheduled-Thread-90) > > >> > > [789edb23] Running command: SyncNetworkProviderCommand > > >> > > internal: true.2018-03-15 10:05:00,228-04 DEBUG > > >> > > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$ > > >> > PostgresSimpleJdbcCall] > > >> > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] > > >> > > Compiled stored procedure. Call string is [{call > > >> > > getdcidbyexternalnetworkid(?)}]2018-03-15 10:05:00,228-04 > > >> > > DEBUG > > >> > > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$ > > >> > PostgresSimpleJdbcCall] > > >> > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] > > >> > > SqlCall for procedure [GetDcIdByExternalNetworkId] > > >> > > compiled2018-03-15 10:05:00,229-04 DEBUG > > >> > > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > > >> > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] > > >> > > method: runQuery, params: [GetAllExternalNetworksOnProvider, > > >> > > IdQueryParameters:{refresh='false', filtered='false'}], > > >> > > timeElapsed: 353ms2018-03-15 10:05:00,239-04 INFO > > >> > > [org.ovirt.engine.core.bll.network.dc.AddNetworkCommand] > > >> > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] > > >> > > Failed to Acquire Lock to object > > >> > > 'EngineLock:{exclusiveLocks='[network_1=NETWORK, > > >> > > c38a67ec-0b48-4e6f-be85-70c700df5483=PROVIDER]', > > >> > > sharedLocks=''}'2018-03-15 10:05:00,239-04 WARN > > >> > > [org.ovirt.engine.core.bll.network.dc.AddNetworkCommand] > > >> > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] > > >> > > Validation of action 'AddNetwork' failed for user > > >> > > admin at internal-authz. Reasons: > > >> > > VAR__TYPE__NETWORK,VAR__ACTION__ADD,ACTION_TYPE_FAILED_PROVIDER_LOCKED,$ > > >> > providerId > > >> > > c38a67ec-0b48-4e6f-be85-70c700df54832018-03-15 > > >> > > 10:05:00,240-04 DEBUG > > >> > > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > > >> > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] > > >> > > method: runAction, params: [AddNetwork, > > >> > > AddNetworkStoragePoolParameters:{commandId='61b365ec-27c1- > > >> > 49af-ad72-f907df8befcd', > > >> > > user='null', commandType='Unknown'}], timeElapsed: > > >> > > 10ms2018-03-15 10:05:00,250-04 ERROR > > >> > > [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] > > >> > > (default task-13) [] Operation Failed: [Cannot add Network. > > >> > > Related operation on provider with the id > > >> > > c38a67ec-0b48-4e6f-be85-70c700df5483 is currently in > > >> > > progress. Please try again later.]2018-03-15 10:05:00,254-04 > > >> > > DEBUG [org.ovirt.engine.core.utils.servlet.LocaleFilter] > > >> > > (default task-14) [] Incoming locale 'en-US'. Filter > > >> > > determined locale to be 'en-US'2018-03-15 10:05:00,254-04 > > >> > > DEBUG [org.ovirt.engine.core.sso.servlets.OAuthTokenServlet] > > >> > > (default task-14) [] Entered OAuthTokenServlet Query String: > > >> > > null, Parameters : password = ***, grant_type = password, > > >> > > scope = ovirt-app-api ovirt-ext=token-info:validate, > > >> > > username = admin at internal, * > > >> > > > >> > I will care about this. > > >> > The problem is that SyncNetworkProviderCommand is running in > > >> > the background and locking the provider, which blocks the lock > > >> > for the tested AddNetworkCommand. > > >> > The related changes are > > >> > core: Add locking for Add and RemoveNetworkCommand > > >> > https://gerrit.ovirt.org/#/c/85480/ > > >> > and > > >> > core: Add SyncNetworkProviderCommand > > >> > https://gerrit.ovirt.org/#/c/85134/ > > >> > > > >> > > > >> > > > > > > > _______________________________________________ > > > Devel mailing list > > > Devel at ovirt.org > > > http://lists.ovirt.org/mailman/listinfo/devel > > > > > > > From danken at redhat.com Mon Mar 19 15:14:41 2018 From: danken at redhat.com (Dan Kenigsberg) Date: Mon, 19 Mar 2018 17:14:41 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (cockpit-ovirt) ] [ 15-03-2018 ] [ 098_ovirt_provider_ovn.use_ovn_provider ] In-Reply-To: <20180319153844.0f657d1c@t460p> References: <20180315212147.48669c08@t460p> <20180316155252.1fd28b5b@t460p> <20180319094032.16183bde@t460p> <20180319153844.0f657d1c@t460p> Message-ID: I think you should keep Bug 1557419 - Recommend that a call should be tried again but it's for Ondra to decide. On Mon, Mar 19, 2018 at 4:38 PM, Dominik Holler wrote: > I have overseen that many other commands handle this problem by using a > waiting lock, so the related commands in this issue should do the same. > I created > Adding a new external network fails during auto-sync is running > https://bugzilla.redhat.com/show_bug.cgi?id=1558054 > to track this. > > From my point of view the other related two bugs I created are not > required anymore, because other commands seems to use a waiting lock, > too. > Should we close 1557419 and 1557424? > > > On Mon, 19 Mar 2018 09:40:32 +0100 > Dominik Holler wrote: > >> On Sat, 17 Mar 2018 16:16:49 +0200 >> Dan Kenigsberg wrote: >> >> > Thanks for filing those, but let us not keep them secret: >> > >> > Bug 1557419 - Recommend that a call should be tried again >> > sounds most reasonable. Providing more information to the client >> > makes sense >> > >> > Bug 1557424 - Automatically retry call failed because he failed to >> > acquire a lock >> > is more tricky. when would REST perform the retry? >> >> If the command returns a machine-readable hint that the conflict is >> temporary. >> >> > how often? We must make sure we never cause a livelock. >> > >> >> I agree that it is a good idea to have an upper limit for the count of >> retries to prevent a livelock. >> >> > Can we alternatively take a normal blocking lock when we create an >> > external network (instead of a trylock)? >> > >> >> This might work, but currently no command is doing this. >> (The lock is acquired in CommandBase.acquireLockInternal() which could >> be overridden for all commands using an external network provider.) >> But I would prefer a solution which solve the issue for other >> commands, too. >> >> > On Fri, Mar 16, 2018 at 4:52 PM, Dominik Holler >> > wrote: >> > > I have created >> > > https://bugzilla.redhat.com/show_bug.cgi?id=1557419 and >> > > https://bugzilla.redhat.com/show_bug.cgi?id=1557424 >> > > to discuss how failing to acquire locks in engine should be >> > > handled in engine's REST-API, in ovirt-sdk or in application (OST >> > > in this case). >> > > >> > > >> > > On Fri, 16 Mar 2018 09:32:47 +0000 >> > > Dafna Ron wrote: >> > > >> > >> Thank you for the fast reply and help. >> > >> >> > >> On Thu, Mar 15, 2018 at 8:21 PM, Dominik Holler >> > >> wrote: >> > >> >> > >> > On Thu, 15 Mar 2018 16:24:10 +0000 >> > >> > Dafna Ron wrote: >> > >> > >> > >> > > Hi, >> > >> > > >> > >> > > We have a failure on master for test >> > >> > > 098_ovirt_provider_ovn.use_ovn_provider in project >> > >> > > cockpit-ovirt. This seems to be a race because object is >> > >> > > locked. also, the actual failure is logged as WARN and not >> > >> > > ERROR. >> > >> > > >> > >> > > I don't think the patch is actually related to the failure >> > >> > > but I think the test should be fixed. >> > >> > > can you please review to make sure we do not have an actual >> > >> > > regression and let me know if we need to open a bz to fix the >> > >> > > test? >> > >> > > >> > >> > > >> > >> > > *Link and headline of suspected patches: * >> > >> > > *https://gerrit.ovirt.org/#/c/89020/2 >> > >> > > - * >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > *wizard: Enable scroll on start page for low-res screensLink >> > >> > > to >> > >> > > Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6374 >> > >> > > Link >> > >> > > to all >> > >> > > logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue- >> > >> > tester/6374/artifacts >> > >> > > > > >> > tester/6374/artifacts>(Relevant) >> > >> > > error snippet from the log: 2018-03-15 10:05:00,160-04 >> > >> > > DEBUG >> > >> > > [org.ovirt.engine.core.sso.servlets.OAuthTokenInfoServlet] >> > >> > > (default task-10) [] Sending json response2018-03-15 >> > >> > > 10:05:00,160-04 DEBUG >> > >> > > [org.ovirt.engine.core.sso.utils.TokenCleanupUtility] >> > >> > > (default task-10) [] Not cleaning up expired tokens2018-03-15 >> > >> > > 10:05:00,169-04 INFO >> > >> > > [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> > >> > > (EE-ManagedThreadFactory-engineScheduled-Thread-90) >> > >> > > [789edb23] Lock Acquired to object >> > >> > > 'EngineLock:{exclusiveLocks='[c38a67ec-0b48-4e6f-be85- >> > >> > 70c700df5483=PROVIDER]', >> > >> > > sharedLocks=''}'2018-03-15 10:05:00,184-04 INFO >> > >> > > [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> > >> > > (EE-ManagedThreadFactory-engineScheduled-Thread-90) >> > >> > > [789edb23] Running command: SyncNetworkProviderCommand >> > >> > > internal: true.2018-03-15 10:05:00,228-04 DEBUG >> > >> > > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$ >> > >> > PostgresSimpleJdbcCall] >> > >> > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] >> > >> > > Compiled stored procedure. Call string is [{call >> > >> > > getdcidbyexternalnetworkid(?)}]2018-03-15 10:05:00,228-04 >> > >> > > DEBUG >> > >> > > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$ >> > >> > PostgresSimpleJdbcCall] >> > >> > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] >> > >> > > SqlCall for procedure [GetDcIdByExternalNetworkId] >> > >> > > compiled2018-03-15 10:05:00,229-04 DEBUG >> > >> > > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >> > >> > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] >> > >> > > method: runQuery, params: [GetAllExternalNetworksOnProvider, >> > >> > > IdQueryParameters:{refresh='false', filtered='false'}], >> > >> > > timeElapsed: 353ms2018-03-15 10:05:00,239-04 INFO >> > >> > > [org.ovirt.engine.core.bll.network.dc.AddNetworkCommand] >> > >> > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] >> > >> > > Failed to Acquire Lock to object >> > >> > > 'EngineLock:{exclusiveLocks='[network_1=NETWORK, >> > >> > > c38a67ec-0b48-4e6f-be85-70c700df5483=PROVIDER]', >> > >> > > sharedLocks=''}'2018-03-15 10:05:00,239-04 WARN >> > >> > > [org.ovirt.engine.core.bll.network.dc.AddNetworkCommand] >> > >> > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] >> > >> > > Validation of action 'AddNetwork' failed for user >> > >> > > admin at internal-authz. Reasons: >> > >> > > VAR__TYPE__NETWORK,VAR__ACTION__ADD,ACTION_TYPE_FAILED_PROVIDER_LOCKED,$ >> > >> > providerId >> > >> > > c38a67ec-0b48-4e6f-be85-70c700df54832018-03-15 >> > >> > > 10:05:00,240-04 DEBUG >> > >> > > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >> > >> > > (default task-13) [e1328379-17b7-49f8-beb2-cf8331784828] >> > >> > > method: runAction, params: [AddNetwork, >> > >> > > AddNetworkStoragePoolParameters:{commandId='61b365ec-27c1- >> > >> > 49af-ad72-f907df8befcd', >> > >> > > user='null', commandType='Unknown'}], timeElapsed: >> > >> > > 10ms2018-03-15 10:05:00,250-04 ERROR >> > >> > > [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] >> > >> > > (default task-13) [] Operation Failed: [Cannot add Network. >> > >> > > Related operation on provider with the id >> > >> > > c38a67ec-0b48-4e6f-be85-70c700df5483 is currently in >> > >> > > progress. Please try again later.]2018-03-15 10:05:00,254-04 >> > >> > > DEBUG [org.ovirt.engine.core.utils.servlet.LocaleFilter] >> > >> > > (default task-14) [] Incoming locale 'en-US'. Filter >> > >> > > determined locale to be 'en-US'2018-03-15 10:05:00,254-04 >> > >> > > DEBUG [org.ovirt.engine.core.sso.servlets.OAuthTokenServlet] >> > >> > > (default task-14) [] Entered OAuthTokenServlet Query String: >> > >> > > null, Parameters : password = ***, grant_type = password, >> > >> > > scope = ovirt-app-api ovirt-ext=token-info:validate, >> > >> > > username = admin at internal, * >> > >> > >> > >> > I will care about this. >> > >> > The problem is that SyncNetworkProviderCommand is running in >> > >> > the background and locking the provider, which blocks the lock >> > >> > for the tested AddNetworkCommand. >> > >> > The related changes are >> > >> > core: Add locking for Add and RemoveNetworkCommand >> > >> > https://gerrit.ovirt.org/#/c/85480/ >> > >> > and >> > >> > core: Add SyncNetworkProviderCommand >> > >> > https://gerrit.ovirt.org/#/c/85134/ >> > >> > >> > >> > >> > >> > >> > > >> > > _______________________________________________ >> > > Devel mailing list >> > > Devel at ovirt.org >> > > http://lists.ovirt.org/mailman/listinfo/devel >> > > >> > > >> > From sbonazzo at redhat.com Mon Mar 19 15:19:38 2018 From: sbonazzo at redhat.com (Sandro Bonazzola) Date: Mon, 19 Mar 2018 16:19:38 +0100 Subject: [ovirt-devel] Bug 1557532 - CVE-2018-1000134 unboundid-ldapsdk: Incorrect Access Control vulnerability in process function in SimpleBindRequest class [fedora-all] Message-ID: Hi, before I start building the same package for CentOS Virt SIG, please help testing the new 4.0.5 build on Fedora and ensure aaa-ldap package still work with the new build. If it works for you. please give karma to the build to allow it to be released to stable. unboundid-ldapsdk-4.0.5-1.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2018-c188d3f09a unboundid-ldapsdk-4.0.5-1.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2018-0a473d6e7b unboundid-ldapsdk-4.0.5-1.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-e8635ed222 Thanks, -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA sbonazzo at redhat.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From dron at redhat.com Mon Mar 19 16:02:37 2018 From: dron at redhat.com (Dafna Ron) Date: Mon, 19 Mar 2018 16:02:37 +0000 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 (ovirt-engine) ] [ 19-03-2018 ] [ 004_basic_sanity.verify_vm_import ] Message-ID: Hi, We have a failed change on test 004_basic_sanity.verify_vm_import. I can't see anything in the logs that suggests there was a problem with the import and the error in the junit also suggest this can possibly be in the api version. can you please check the issue? *Link and headline of suspected patches: * *core: Call endAction() of all child commands in ImportVmCommand - https://gerrit.ovirt.org/#/c/89165/ Link to Job:http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1362 Link to all logs:http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1362/artifact/exported-artifacts/basic-suit-4.2-el7/test_logs/basic-suite-4.2/post-004_basic_sanity.py/ (Relevant) error snippet from the log: 2018-03-19 11:01:59,679-04 INFO [org.ovirt.engine.core.bll.exportimport.ImportVmCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [] Lock freed to object 'EngineLock:{exclusiveLocks='[imported_vm=VM_NAME, 3494a94b-c253-440b-a6cb-c4c818b19ae1=VM]', sharedLocks='[cc4255e8-8e1d-4608-a3fe-a2096ed4e9f9=REMOTE_VM]'}'* Error Message False != True after 180 seconds Stacktrace Traceback (most recent call last): File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 78, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/home/jenkins/workspace/ovirt-4.2_change-queue-tester/ovirt-system-tests/basic-suite-4.2/test-scenarios/004_basic_sanity.py", line 596, in verify_vm_import lambda: vm_service.get().status == types.VmStatus.DOWN File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 267, in assert_true_within_short assert_equals_within_short(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 251, in assert_equals_within_short func, value, SHORT_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 237, in assert_equals_within '%s != %s after %s seconds' % (res, value, timeout) AssertionError: False != True after 180 seconds ** -------------- next part -------------- An HTML attachment was scrubbed... URL: From smelamud at redhat.com Mon Mar 19 18:46:06 2018 From: smelamud at redhat.com (Shmuel Melamud) Date: Mon, 19 Mar 2018 20:46:06 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 19-03-2018 ] [002_bootstrap.add_instance_type ] In-Reply-To: References: Message-ID: Here is the fix: https://gerrit.ovirt.org/#/c/89187/ On Mon, Mar 19, 2018 at 4:12 PM, Dafna Ron wrote: > thank you for the fast response. once you have the fix, can you please sent > it to us? > > > > On Mon, Mar 19, 2018 at 1:25 PM, Shmuel Melamud wrote: >> >> Hi! >> >> Forgot about instance type that don't have a cluster. Fixing it now. >> >> Shmuel >> >> On Mon, Mar 19, 2018 at 2:25 PM, Dafna Ron wrote: >> > Hi, >> > >> > We have a failure in test 002_bootstrap.add_instance_type. >> > There seem to be a NullPointerException on template type which is >> > causing >> > this failure. >> > The same change that was reported at the last failure is reported as the >> > root cause for this failure as well, but I am not sure how it would >> > cause >> > this failure. >> > Can you please check? >> > >> > >> > Link and headline of suspected patches: >> > >> > reported as failed: >> > core: fix removal of vm-host device - >> > https://gerrit.ovirt.org/#/c/89145/ >> > >> > reported as root cause: >> > >> > core: USB in osinfo configuration depends on chipset - >> > https://gerrit.ovirt.org/#/c/88777/ >> > >> > Link to Job: >> > >> > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6431 >> > >> > Link to all logs: >> > >> > >> > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6431/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-002_bootstrap.py/ >> > >> > (Relevant) error snippet from the log: >> > >> > >> > >> > >> > 2018-03-19 04:59:01,040-04 INFO >> > [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-5) >> > [5a890b17-51ec-4398-8d64-82cc71939e6e] Lock Acquired to object >> > >> > 'EngineLock:{exclusiveLocks='[99a9dfb3-9a13-4595-a795-693493e722be=TEMPLATE, >> > myinstancetype=TEMPLATE_NAME]', sharedLocks='[]'}' >> > 2018-03-19 04:59:01,087-04 INFO >> > [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-5) >> > [5a890b17-51ec-4398-8d64-82cc71939e6e] Running command: >> > AddVmTemplateCommand >> > internal: false. Entities affected : ID: aaa00000-0000-0000-0 >> > 000-123456789aaa Type: SystemAction group CREATE_TEMPLATE with role type >> > USER >> > 2018-03-19 04:59:01,139-04 INFO >> > [org.ovirt.engine.core.bll.storage.disk.CreateAllTemplateDisksCommand] >> > (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] Running command: >> > CreateAllTemplateDisksCommand internal: true. >> > 2018-03-19 04:59:01,205-04 INFO >> > [org.ovirt.engine.core.utils.transaction.TransactionSupport] (default >> > task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] transaction rolled back >> > 2018-03-19 04:59:01,205-04 ERROR >> > [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-5) >> > [5a890b17-51ec-4398-8d64-82cc71939e6e] Command >> > 'org.ovirt.engine.core.bll.AddVmTemplateCommand' failed: null >> > 2018-03-19 04:59:01,205-04 ERROR >> > [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-5) >> > [5a890b17-51ec-4398-8d64-82cc71939e6e] Exception: >> > java.lang.NullPointerException >> > at >> > >> > org.ovirt.engine.core.bll.utils.EmulatedMachineUtils.getEffective(EmulatedMachineUtils.java:30) >> > [bll.jar:] >> > at >> > >> > org.ovirt.engine.core.bll.utils.EmulatedMachineUtils.getEffectiveChipset(EmulatedMachineUtils.java:21) >> > [bll.jar:] >> > at >> > >> > org.ovirt.engine.core.bll.utils.VmDeviceUtils.updateUsbSlots(VmDeviceUtils.java:744) >> > [bll.jar:] >> > at >> > >> > org.ovirt.engine.core.bll.utils.VmDeviceUtils.copyVmDevices(VmDeviceUtils.java:1519) >> > [bll.jar:] >> > at >> > >> > org.ovirt.engine.core.bll.utils.VmDeviceUtils.copyVmDevices(VmDeviceUtils.java:1565) >> > [bll.jar:] >> > at >> > >> > org.ovirt.engine.core.bll.AddVmTemplateCommand.lambda$executeCommand$4(AddVmTemplateCommand.java:362) >> > [bll.jar:] >> > at >> > >> > org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:202) >> > [utils.jar:] >> > at >> > >> > org.ovirt.engine.core.bll.AddVmTemplateCommand.executeCommand(AddVmTemplateCommand.java:345) >> > [bll.jar:] >> > at >> > >> > org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1133) >> > [bll.jar:] >> > at >> > >> > org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1286) >> > [bll.jar:] >> > at >> > >> > org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1935) >> > [bll.jar:] >> > at >> > >> > org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:164) >> > [utils.jar:] >> > at >> > >> > org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:103) >> > [utils.jar:] >> > at >> > org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1346) >> > [bll.jar:] >> > at >> > >> > org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:400) >> > [bll.jar:] >> > at >> > >> > org.ovirt.engine.core.bll.executor.DefaultBackendActionExecutor.execute(DefaultBackendActionExecutor.java:13) >> > [bll.jar:] >> > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:468) >> > [bll.jar:] >> > at >> > org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:450) >> > [bll.jar:] >> > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:403) >> > [bll.jar:] >> > at sun.reflect.GeneratedMethodAccessor149.invoke(Unknown Source) >> > [:1.8.0_161] >> > at >> > >> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> > [rt.jar:1.8.0_161] >> > at java.lang.reflect.Method.invoke(Method.java:498) >> > [rt.jar:1.8.0_161] >> > at >> > >> > org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) >> > at >> > >> > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) >> > at >> > >> > org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) >> > at >> > >> > org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:92) >> > [wildfly-weld-ejb-11.0.0.Final.jar:11.0.0.Final] >> > at >> > >> > org.jboss.weld.interceptor.proxy.WeldInvocationContext.interceptorChainCompleted(WeldInvocationContext.java:98) >> > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] >> > at >> > >> > org.jboss.weld.interceptor.proxy.WeldInvocationContext.proceed(WeldInvocationContext.java:117) >> > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] >> > at >> > >> > org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12) >> > [common.jar:] >> > at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source) >> > [:1.8.0_161] >> > at >> > >> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> > [rt.jar:1.8.0_161] >> > at java.lang.reflect.Method.invoke(Method.java:498) >> > [rt.jar:1.8.0_161] >> > at >> > >> > org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73) >> > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] >> > at >> > >> > org.jboss.weld.interceptor.proxy.WeldInvocationContext.invokeNext(WeldInvocationContext.java:83) >> > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] >> > at >> > >> > org.jboss.weld.interceptor.proxy.WeldInvocationContext.proceed(WeldInvocationContext.java:115) >> > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] >> > at >> > org.jboss.weld.bean.InterceptorImpl.intercept(InterceptorImpl.java:108) >> > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] >> > at >> > >> > org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:82) >> > [wildfly-weld-ejb-11.0.0.Final.jar:11.0.0.Final] >> > at >> > >> > org.jboss.as.weld.interceptors.EjbComponentInterceptorSupport.delegateInterception(EjbComponentInterceptorSupport.java:60) >> > at >> > >> > org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:76) >> > at >> > >> > org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:88) >> > at >> > >> > org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:101) >> > at >> > >> > org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) >> > at >> > >> > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) >> > at >> > >> > org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) >> > at >> > >> > org.ovirt.engine.core.bll.interceptors.CorrelationIdTrackerInterceptor.aroundInvoke(CorrelationIdTrackerInterceptor.java:13) >> > [bll.jar:] >> > at sun.reflect.GeneratedMethodAccessor66.invoke(Unknown Source) >> > [:1.8.0_161] >> > at >> > >> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> > [rt.jar:1.8.0_161] >> > at java.lang.reflect.Method.invoke(Method.java:498) >> > [rt.jar:1.8.0_161] >> > at >> > >> > org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:89) >> > at >> > >> > org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) >> > : >> > >> > >> > End of error: >> > >> > at >> > >> > io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) >> > at >> > >> > org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) >> > at >> > >> > org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) >> > at >> > >> > org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) >> > at >> > >> > org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) >> > at >> > >> > io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) >> > at >> > >> > io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) >> > at >> > >> > io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) >> > at >> > io.undertow.server.Connectors.executeRootHandler(Connectors.java:326) >> > at >> > io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:812) >> > at >> > >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >> > [rt.jar:1.8.0_161] >> > at >> > >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >> > [rt.jar:1.8.0_161] >> > at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_161] >> > >> > 2018-03-19 04:59:01,234-04 DEBUG >> > >> > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall] >> > (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] Compiled stored >> > procedure. Call string is [{call get_entity_snapshot_by_command_id(?)}] >> > 2018-03-19 04:59:01,234-04 DEBUG >> > >> > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall] >> > (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] SqlCall for >> > procedure [get_entity_snapshot_by_command_id] compiled >> > 2018-03-19 04:59:01,239-04 INFO >> > [org.ovirt.engine.core.bll.CommandCompensator] (default task-5) >> > [5a890b17-51ec-4398-8d64-82cc71939e6e] Command >> > [id=a7a51354-ae84-4109-a55e-b27558afbf2a]: Compensating NEW_ENTITY_ID of >> > org.ovirt.engine.core.common.businessentities.VmTemplate; snapshot: >> > 99a9dfb3-9a13-4595-a795-693493e722be. >> > 2018-03-19 04:59:01,278-04 ERROR >> > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >> > (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] EVENT_ID: >> > USER_ADD_VM_TEMPLATE_FAILURE(36), Failed creating Template >> > myinstancetype. >> > 2018-03-19 04:59:01,295-04 INFO >> > [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-5) >> > [5a890b17-51ec-4398-8d64-82cc71939e6e] Lock freed to object >> > >> > 'EngineLock:{exclusiveLocks='[99a9dfb3-9a13-4595-a795-693493e722be=TEMPLATE, >> > myinstancetype=TEMPLATE_NAME]', sharedLocks='[]'}' >> > 2018-03-19 04:59:01,295-04 DEBUG >> > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >> > (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] method: >> > runAction, >> > params: [AddVmTemplate, >> > >> > AddVmTemplateParameters:{commandId='a7a51354-ae84-4109-a55e-b27558afbf2a', >> > user='admin', commandType='AddVmTemplate'}], timeElapsed: 301ms >> > 2018-03-19 04:59:01,301-04 ERROR >> > [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default >> > task-5) [] Operation Failed: [Internal Engine Error] >> > 2018-03-19 04:59:01,547-04 INFO >> > [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] >> > (EE-ManagedThreadFactory-engineScheduled-Thread-83) >> > [5a890b17-51ec-4398-8d64-82cc71939e6e] Command 'AddVmTemplate' id: >> > 'a7a51354-ae84-4109-a55e-b27558afbf2a' execution didn't complete, not >> > proceeding to perform the next operation >> > 2018-03-19 04:59:01,547-04 INFO >> > [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] >> > (EE-ManagedThreadFactory-engineScheduled-Thread-83) >> > [5a890b17-51ec-4398-8d64-82cc71939e6e] Command 'AddVmTemplate' id: >> > 'a7a51354-ae84-4109-a55e-b27558afbf2a' child commands >> > '[8484f4d2-9ddf-48bd-9877-7c99ef555255]' executions were completed, >> > status >> > 'FAILED' >> > 2018-03-19 04:59:01,581-04 INFO >> > [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] >> > (EE-ManagedThreadFactory-engineScheduled-Thread-83) >> > [db5dd3dd-8b48-4647-a02d-723fcd6a7dda] Command 'ImportRepoImage' (id: >> > 'a6475141-53ef-4ecd-bc17-ee8eae168dc7') waiting on child command id: >> > '528fde29-9b5b-4a90-b62a-606fac3cd6a6' type:'DownloadImage' to complete >> > 2018-03-19 04:59:02,363-04 DEBUG >> > [org.ovirt.otopi.dialog.MachineDialogParser] (VdsDeploy) [665e1c35] Got: >> > ***L:INFO Yum install: 349/515: gdisk-0.8.6-5.el7.x86_64 >> > 2018-03-19 04:59:02,364-04 DEBUG >> > [org.ovirt.otopi.dialog.MachineDialogParser] (VdsDeploy) [665e1c35] >> > nextEvent: Log INFO Yum install: 349/515: gdisk-0.8.6-5.el7.x86_64 >> > 2018-03-19 04:59:02,570-04 INFO >> > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >> > (VdsDeploy) [665e1c35] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), >> > Installing >> > Host lago-basic-suite-master-host-1. Yum install: 349/515: >> > gdisk-0.8.6-5.el7.x86_64. >> > 2018-03-19 04:59:02,666-04 ERROR >> > [org.ovirt.engine.core.bll.AddVmTemplateCommand] >> > (EE-ManagedThreadFactory-engineScheduled-Thread-86) >> > [5a890b17-51ec-4398-8d64-82cc71939e6e] Ending command >> > 'org.ovirt.engine.core.bll.AddVmTemplateCommand' with failure. >> > >> > >> > >> > >> > >> > > > From michal.skrivanek at redhat.com Tue Mar 20 07:49:35 2018 From: michal.skrivanek at redhat.com (Michal Skrivanek) Date: Tue, 20 Mar 2018 08:49:35 +0100 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (Ovirt-ngine) ] [ 19-03-2018 ] [ 002_bootstrap.verify_notifier ] In-Reply-To: References: Message-ID: <46392D70-A8F0-4D32-BAD1-5F1AA0680AC7@redhat.com> I?m not sure what is that test actually testing, if it depends on the previous host action which fails but is not verified it still may be relevant to Shmuel?s patch adding author of the test and the notifier owner > On 19 Mar 2018, at 13:06, Dafna Ron wrote: > > Hi, > > We had a failure in test 002_bootstrap.verify_notifier. > I can't see anything wrong with the notifier and I don't think it should be related to the change that was reported. > > the test itself is looking for vdc_stop in messages which I do not indeed see but I am not sure what is the cause and how the reported change related to the failure. > > Can you please take a look? > > > > Link and headline of suspected patches: > core: USB in osinfo configuration depends on chipset - https://gerrit.ovirt.org/#/c/88777/ > > Link to Job: > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6429/ > > Link to all logs: > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6429/artifacts > > (Relevant) error snippet from the log: > > > this is the error from the api: > > > Error Message > > Failed grep for VDC_STOP with code 1. Output: > -------------------- >> begin captured logging << -------------------- > lago.ssh: DEBUG: start task:f1231b27-f796-406c-8618-17b0868725bc:Get ssh client for lago-basic-suite-master-engine: > lago.ssh: DEBUG: end task:f1231b27-f796-406c-8618-17b0868725bc:Get ssh client for lago-basic-suite-master-engine: > lago.ssh: DEBUG: Running 1cce7c0c on lago-basic-suite-master-engine: grep VDC_STOP /var/log/messages > lago.ssh: DEBUG: Command 1cce7c0c on lago-basic-suite-master-engine returned with 1 > --------------------- >> end captured logging << --------------------- > Stacktrace > > File "/usr/lib64/python2.7/unittest/case.py", line 369, in run > testMethod() > File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest > self.test(*self.arg) > File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test > test() > File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper > return func(get_test_prefix(), *args, **kwargs) > File "/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 1456, in verify_notifier > 'Failed grep for VDC_STOP with code {0}. Output: {1}'.format(result.code, result.out) > File "/usr/lib/python2.7/site-packages/nose/tools/trivial.py", line 29, in eq_ > raise AssertionError(msg or "%r != %r" % (a, b)) > 'Failed grep for VDC_STOP with code 1. Output: \n-------------------- >> begin captured logging << --------------------\nlago.ssh: DEBUG: start task:f1231b27-f796-406c-8618-17b0868725bc:Get ssh client for lago-basic-suite-master-engine:\nlago.ssh: DEBUG: end task:f1231b27-f796-406c-8618-17b0868725bc:Get ssh client for lago-basic-suite-master-engine:\nlago.ssh: DEBUG: Running 1cce7c0c on lago-basic-suite-master-engine: grep VDC_STOP /var/log/messages\nlago.ssh: DEBUG: Command 1cce7c0c on lago-basic-suite-master-engine returned with 1\n--------------------- >> end captured logging << ---------------------' > > > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From fromani at redhat.com Tue Mar 20 09:07:31 2018 From: fromani at redhat.com (Francesco Romani) Date: Tue, 20 Mar 2018 10:07:31 +0100 Subject: [ovirt-devel] [vdsm] network test failure Message-ID: <78651164-7ebd-cb24-556e-909be31a8d68@redhat.com> Hi all, we had a bogus failure on CI again, some network test failed and it seems totally unrelated to the patch being tested: http://jenkins.ovirt.org/job/vdsm_master_check-patch-el7-x86_64/22410/consoleFull could someone please have a look? Bests, -- Francesco Romani Senior SW Eng., Virtualization R&D Red Hat IRC: fromani github: @fromanirh From ykaul at redhat.com Tue Mar 20 09:33:49 2018 From: ykaul at redhat.com (Yaniv Kaul) Date: Tue, 20 Mar 2018 11:33:49 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (Ovirt-ngine) ] [ 19-03-2018 ] [ 002_bootstrap.verify_notifier ] In-Reply-To: <46392D70-A8F0-4D32-BAD1-5F1AA0680AC7@redhat.com> References: <46392D70-A8F0-4D32-BAD1-5F1AA0680AC7@redhat.com> Message-ID: On Tue, Mar 20, 2018 at 9:49 AM, Michal Skrivanek < michal.skrivanek at redhat.com> wrote: > I?m not sure what is that test actually testing, if it depends on the > previous host action which fails but is not verified it still may be > relevant to Shmuel?s patch > adding author of the test and the notifier owner > It is checking that the notifier works - which sends SNMP notifications on our events. I happen to picked an event which is VDC_STOP - which happens when the engine is restarted - which happens earlier, when we configure it. Y. > > On 19 Mar 2018, at 13:06, Dafna Ron wrote: > > Hi, > > We had a failure in test 002_bootstrap.verify_notifier. > I can't see anything wrong with the notifier and I don't think it should > be related to the change that was reported. > > the test itself is looking for vdc_stop in messages which I do not indeed > see but I am not sure what is the cause and how the reported change related > to the failure. > > Can you please take a look? > > > > *Link and headline of suspected patches: * > > > > > > > *core: USB in osinfo configuration depends on chipset - > https://gerrit.ovirt.org/#/c/88777/ > Link to > Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6429/ > Link > to all > logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6429/artifacts > (Relevant) > error snippet from the log: * > *this is the error from *the api: > > > Error Message > > Failed grep for VDC_STOP with code 1. Output: > -------------------- >> begin captured logging << -------------------- > lago.ssh: DEBUG: start task:f1231b27-f796-406c-8618-17b0868725bc:Get ssh client for lago-basic-suite-master-engine: > lago.ssh: DEBUG: end task:f1231b27-f796-406c-8618-17b0868725bc:Get ssh client for lago-basic-suite-master-engine: > lago.ssh: DEBUG: Running 1cce7c0c on lago-basic-suite-master-engine: grep VDC_STOP /var/log/messages > lago.ssh: DEBUG: Command 1cce7c0c on lago-basic-suite-master-engine returned with 1 > --------------------- >> end captured logging << --------------------- > > Stacktrace > > File "/usr/lib64/python2.7/unittest/case.py", line 369, in run > testMethod() > File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest > self.test(*self.arg) > File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test > test() > File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper > return func(get_test_prefix(), *args, **kwargs) > File "/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 1456, in verify_notifier > 'Failed grep for VDC_STOP with code {0}. Output: {1}'.format(result.code, result.out) > File "/usr/lib/python2.7/site-packages/nose/tools/trivial.py", line 29, in eq_ > raise AssertionError(msg or "%r != %r" % (a, b)) > 'Failed grep for VDC_STOP with code 1. Output: \n-------------------- >> begin captured logging << --------------------\nlago.ssh: DEBUG: start task:f1231b27-f796-406c-8618-17b0868725bc:Get ssh client for lago-basic-suite-master-engine:\nlago.ssh: DEBUG: end task:f1231b27-f796-406c-8618-17b0868725bc:Get ssh client for lago-basic-suite-master-engine:\nlago.ssh: DEBUG: Running 1cce7c0c on lago-basic-suite-master-engine: grep VDC_STOP /var/log/messages\nlago.ssh: DEBUG: Command 1cce7c0c on lago-basic-suite-master-engine returned with 1\n--------------------- >> end captured logging << ---------------------' > > > ** > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From danken at redhat.com Tue Mar 20 10:14:58 2018 From: danken at redhat.com (Dan Kenigsberg) Date: Tue, 20 Mar 2018 12:14:58 +0200 Subject: [ovirt-devel] [vdsm] network test failure In-Reply-To: <78651164-7ebd-cb24-556e-909be31a8d68@redhat.com> References: <78651164-7ebd-cb24-556e-909be31a8d68@redhat.com> Message-ID: +Petr On Tue, Mar 20, 2018 at 11:07 AM, Francesco Romani wrote: > Hi all, > > > we had a bogus failure on CI again, some network test failed and it > seems totally unrelated to the patch being tested: > > > http://jenkins.ovirt.org/job/vdsm_master_check-patch-el7-x86_64/22410/consoleFull > > > could someone please have a look? > > > Bests, > > -- > Francesco Romani > Senior SW Eng., Virtualization R&D > Red Hat > IRC: fromani github: @fromanirh > From ahadas at redhat.com Tue Mar 20 15:03:11 2018 From: ahadas at redhat.com (Arik Hadas) Date: Tue, 20 Mar 2018 17:03:11 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 (ovirt-engine) ] [ 19-03-2018 ] [ 004_basic_sanity.verify_vm_import ] In-Reply-To: References: Message-ID: On Mon, Mar 19, 2018 at 6:02 PM, Dafna Ron wrote: > Hi, > > We have a failed change on test 004_basic_sanity.verify_vm_import. > I can't see anything in the logs that suggests there was a problem with > the import and the error in the junit also suggest this can possibly be in > the api version. > > can you please check the issue? > We found the issue - it happens only when importing VM as new entity (clone), SetVmStatus is not called in that case and so the VM remains in status ImageLockd. Shmuel will post a fix shortly. > > *Link and headline of suspected patches: * > > > > > > > > > > *core: Call endAction() of all child commands in ImportVmCommand - > https://gerrit.ovirt.org/#/c/89165/ > Link to > Job:http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1362 > Link to > all > logs:http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1362/artifact/exported-artifacts/basic-suit-4.2-el7/test_logs/basic-suite-4.2/post-004_basic_sanity.py/ > (Relevant) > error snippet from the log: 2018-03-19 11:01:59,679-04 INFO > [org.ovirt.engine.core.bll.exportimport.ImportVmCommand] > (EE-ManagedThreadFactory-engineScheduled-Thread-22) [] Lock freed to object > 'EngineLock:{exclusiveLocks='[imported_vm=VM_NAME, > 3494a94b-c253-440b-a6cb-c4c818b19ae1=VM]', > sharedLocks='[cc4255e8-8e1d-4608-a3fe-a2096ed4e9f9=REMOTE_VM]'}'* > Error Message > > False != True after 180 seconds > > Stacktrace > > Traceback (most recent call last): > File "/usr/lib64/python2.7/unittest/case.py", line 369, in run > testMethod() > File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest > self.test(*self.arg) > File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test > test() > File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper > return func(get_test_prefix(), *args, **kwargs) > File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 78, in wrapper > prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs > File "/home/jenkins/workspace/ovirt-4.2_change-queue-tester/ovirt-system-tests/basic-suite-4.2/test-scenarios/004_basic_sanity.py", line 596, in verify_vm_import > lambda: vm_service.get().status == types.VmStatus.DOWN > File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 267, in assert_true_within_short > assert_equals_within_short(func, True, allowed_exceptions) > File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 251, in assert_equals_within_short > func, value, SHORT_TIMEOUT, allowed_exceptions=allowed_exceptions > File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 237, in assert_equals_within > '%s != %s after %s seconds' % (res, value, timeout) > AssertionError: False != True after 180 seconds > > ** > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dron at redhat.com Tue Mar 20 15:13:39 2018 From: dron at redhat.com (Dafna Ron) Date: Tue, 20 Mar 2018 15:13:39 +0000 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 (ovirt-engine) ] [ 19-03-2018 ] [ 004_basic_sanity.verify_vm_import ] In-Reply-To: References: Message-ID: Thank you. can you please also add the fix here? On Tue, Mar 20, 2018 at 3:03 PM, Arik Hadas wrote: > > > On Mon, Mar 19, 2018 at 6:02 PM, Dafna Ron wrote: > >> Hi, >> >> We have a failed change on test 004_basic_sanity.verify_vm_import. >> I can't see anything in the logs that suggests there was a problem with >> the import and the error in the junit also suggest this can possibly be in >> the api version. >> >> can you please check the issue? >> > > We found the issue - it happens only when importing VM as new entity > (clone), SetVmStatus is not called in that case and so the VM remains in > status ImageLockd. Shmuel will post a fix shortly. > > >> >> *Link and headline of suspected patches: * >> >> >> >> >> >> >> >> >> >> *core: Call endAction() of all child commands in ImportVmCommand - >> https://gerrit.ovirt.org/#/c/89165/ >> Link to >> Job:http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1362 >> Link to >> all >> logs:http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1362/artifact/exported-artifacts/basic-suit-4.2-el7/test_logs/basic-suite-4.2/post-004_basic_sanity.py/ >> (Relevant) >> error snippet from the log: 2018-03-19 11:01:59,679-04 INFO >> [org.ovirt.engine.core.bll.exportimport.ImportVmCommand] >> (EE-ManagedThreadFactory-engineScheduled-Thread-22) [] Lock freed to object >> 'EngineLock:{exclusiveLocks='[imported_vm=VM_NAME, >> 3494a94b-c253-440b-a6cb-c4c818b19ae1=VM]', >> sharedLocks='[cc4255e8-8e1d-4608-a3fe-a2096ed4e9f9=REMOTE_VM]'}'* >> Error Message >> >> False != True after 180 seconds >> >> Stacktrace >> >> Traceback (most recent call last): >> File "/usr/lib64/python2.7/unittest/case.py", line 369, in run >> testMethod() >> File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest >> self.test(*self.arg) >> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test >> test() >> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper >> return func(get_test_prefix(), *args, **kwargs) >> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 78, in wrapper >> prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs >> File "/home/jenkins/workspace/ovirt-4.2_change-queue-tester/ovirt-system-tests/basic-suite-4.2/test-scenarios/004_basic_sanity.py", line 596, in verify_vm_import >> lambda: vm_service.get().status == types.VmStatus.DOWN >> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 267, in assert_true_within_short >> assert_equals_within_short(func, True, allowed_exceptions) >> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 251, in assert_equals_within_short >> func, value, SHORT_TIMEOUT, allowed_exceptions=allowed_exceptions >> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 237, in assert_equals_within >> '%s != %s after %s seconds' % (res, value, timeout) >> AssertionError: False != True after 180 seconds >> >> ** >> >> _______________________________________________ >> Devel mailing list >> Devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ahadas at redhat.com Tue Mar 20 15:19:20 2018 From: ahadas at redhat.com (Arik Hadas) Date: Tue, 20 Mar 2018 17:19:20 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt 4.2 (ovirt-engine) ] [ 19-03-2018 ] [ 004_basic_sanity.verify_vm_import ] In-Reply-To: References: Message-ID: On Tue, Mar 20, 2018 at 5:13 PM, Dafna Ron wrote: > Thank you. > can you please also add the fix here? > https://gerrit.ovirt.org/#/c/89250/ > > > On Tue, Mar 20, 2018 at 3:03 PM, Arik Hadas wrote: > >> >> >> On Mon, Mar 19, 2018 at 6:02 PM, Dafna Ron wrote: >> >>> Hi, >>> >>> We have a failed change on test 004_basic_sanity.verify_vm_import. >>> I can't see anything in the logs that suggests there was a problem with >>> the import and the error in the junit also suggest this can possibly be in >>> the api version. >>> >>> can you please check the issue? >>> >> >> We found the issue - it happens only when importing VM as new entity >> (clone), SetVmStatus is not called in that case and so the VM remains in >> status ImageLockd. Shmuel will post a fix shortly. >> >> >>> >>> *Link and headline of suspected patches: * >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> *core: Call endAction() of all child commands in ImportVmCommand - >>> https://gerrit.ovirt.org/#/c/89165/ >>> Link to >>> Job:http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1362 >>> Link to >>> all >>> logs:http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1362/artifact/exported-artifacts/basic-suit-4.2-el7/test_logs/basic-suite-4.2/post-004_basic_sanity.py/ >>> (Relevant) >>> error snippet from the log: 2018-03-19 11:01:59,679-04 INFO >>> [org.ovirt.engine.core.bll.exportimport.ImportVmCommand] >>> (EE-ManagedThreadFactory-engineScheduled-Thread-22) [] Lock freed to object >>> 'EngineLock:{exclusiveLocks='[imported_vm=VM_NAME, >>> 3494a94b-c253-440b-a6cb-c4c818b19ae1=VM]', >>> sharedLocks='[cc4255e8-8e1d-4608-a3fe-a2096ed4e9f9=REMOTE_VM]'}'* >>> Error Message >>> >>> False != True after 180 seconds >>> >>> Stacktrace >>> >>> Traceback (most recent call last): >>> File "/usr/lib64/python2.7/unittest/case.py", line 369, in run >>> testMethod() >>> File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest >>> self.test(*self.arg) >>> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test >>> test() >>> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper >>> return func(get_test_prefix(), *args, **kwargs) >>> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 78, in wrapper >>> prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs >>> File "/home/jenkins/workspace/ovirt-4.2_change-queue-tester/ovirt-system-tests/basic-suite-4.2/test-scenarios/004_basic_sanity.py", line 596, in verify_vm_import >>> lambda: vm_service.get().status == types.VmStatus.DOWN >>> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 267, in assert_true_within_short >>> assert_equals_within_short(func, True, allowed_exceptions) >>> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 251, in assert_equals_within_short >>> func, value, SHORT_TIMEOUT, allowed_exceptions=allowed_exceptions >>> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 237, in assert_equals_within >>> '%s != %s after %s seconds' % (res, value, timeout) >>> AssertionError: False != True after 180 seconds >>> >>> ** >>> >>> _______________________________________________ >>> Devel mailing list >>> Devel at ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/devel >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ehaas at redhat.com Tue Mar 20 17:12:34 2018 From: ehaas at redhat.com (Edward Haas) Date: Tue, 20 Mar 2018 19:12:34 +0200 Subject: [ovirt-devel] [vdsm] network test failure In-Reply-To: References: <78651164-7ebd-cb24-556e-909be31a8d68@redhat.com> Message-ID: The tests ran on a fc26 slave and our bond option default map is in sync with the el7 kernel. It looks like we MUST generate a bond default map on every run. I'm a bit surprised it never happened until now, perhaps I'm not interpreting correctly the tests helper code? Petr? Assuming I'm correct here, I'll try to post a fix. Thanks, Edy. On Tue, Mar 20, 2018 at 12:14 PM, Dan Kenigsberg wrote: > +Petr > > On Tue, Mar 20, 2018 at 11:07 AM, Francesco Romani > wrote: > > Hi all, > > > > > > we had a bogus failure on CI again, some network test failed and it > > seems totally unrelated to the patch being tested: > > > > > > http://jenkins.ovirt.org/job/vdsm_master_check-patch-el7- > x86_64/22410/consoleFull > > > > > > could someone please have a look? > > > > > > Bests, > > > > -- > > Francesco Romani > > Senior SW Eng., Virtualization R&D > > Red Hat > > IRC: fromani github: @fromanirh > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fromani at redhat.com Wed Mar 21 09:57:48 2018 From: fromani at redhat.com (Francesco Romani) Date: Wed, 21 Mar 2018 10:57:48 +0100 Subject: [ovirt-devel] [vdsm][maintainership] proposal for a new stable branch policy Message-ID: <789d5be7-ffe8-272a-9d67-69dcf8c15662@redhat.com> Hi all, Recently Milan, Petr and me discussed the state of ovirt.4.2, considering that release 4.2.2 is still pending and this prevents merging of patches in the sub-branch ovirt-4.2. We agreed we could improve the handling of the stable branch(es), in order to make the process smoother and more predictable. Currently, we avoid doing Z- branches (e.g. ovirt-4.2.2, ovirt-4.2.3...) as much as we can, to avoid the hassle of double-backporting patches to stable branch. However, if a release hits unexpected delay, this policy causes different hassles: the Y-branch (ovirt-4.2, ovirt-4.3) is effectively locked, so patches already queued and ready for next releases can't be merged and need to wait. The new proposed policy is the following: - we will keep working exactly as now until we hit a certain RC version. We choosed RC3, a rule of thumb made out of experience. - if RC3 is final, everyone's happy and things resume as usual - if RC3 is NOT final, we will branch out at RC3 -- from that moment on, patches for next version could be accepted on the Y-branch -- stabilization of the late Z version will continue on the Z-branch -- patches will be backported twice Example using made up numbers - We just released ovirt 4.3.1 - We are working on the ovirt-4.3 branch - The last tag is v4.30.10, from ovirt-4.3 branch - We accept patches for ovirt 4.3.2 on the ovirt-4.3 branch - We keep collecting patches, until we tag v4.30.11 (oVirt 4.3.2 RC 1). Tag is made from ovirt-4.3 branch. - Same for tags 4.30.12 - oVirt 4.3.2 RC 2 and 4.30.13 - oVirt 4.3.2 RC 3. Both tags are made from ovirt-4.3 branch. - Damn, RC3 is not final. We branch out ovirt-4.3.2 form branch ovirt-4.3 from the same commit pointed by tag 4.30.13 - Next tags (4.30.13.1) for ovirt 4.3.2 will be taken from the ovirt-4.3.2 branch I believe this approach will make predictable for everyone if and when the branch will be made, so when the patches could be merged and where. The only drawback I can see - and that I realized while writing the example - is the version number which can be awkward: v4.30.11 -> 4.3.2 RC1 v4.30.12 -> 4.3.2 RC2 v4.30.13 -> 4.3.2 RC3 v4.30.13.1 -> 4.3.2 RC4 ?!?! v4.30.13.5 -> 4.3.2 RC5 ?!?! Perhaps we should move to four versions digit? So we could have v4.30.11.0 -> 4.3.2 RC1 v4.30.11.1 -> 4.3.2 RC2 v4.30.11.2 -> 4.3.2 RC3 v4.30.11.3 -> 4.3.2 RC4 v4.30.11.4 -> 4.3.2 RC5 I don't see any real drawback in using 4-digit versions by default, besides a minor increase in complexity, which is balanced by more predictable and consistent versions. Plus, we already had 4-digit versions in Vdsm, so packaging should work just fine. Please share your thoughts, -- Francesco Romani Senior SW Eng., Virtualization R&D Red Hat IRC: fromani github: @fromanirh From dron at redhat.com Wed Mar 21 10:13:47 2018 From: dron at redhat.com (Dafna Ron) Date: Wed, 21 Mar 2018 10:13:47 +0000 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 19-03-2018 ] [002_bootstrap.add_instance_type ] In-Reply-To: References: Message-ID: This issue was resolved on 4.2 and master. Thank you for resolving it quickly. On Mon, Mar 19, 2018 at 6:46 PM, Shmuel Melamud wrote: > Here is the fix: https://gerrit.ovirt.org/#/c/89187/ > > On Mon, Mar 19, 2018 at 4:12 PM, Dafna Ron wrote: > > thank you for the fast response. once you have the fix, can you please > sent > > it to us? > > > > > > > > On Mon, Mar 19, 2018 at 1:25 PM, Shmuel Melamud > wrote: > >> > >> Hi! > >> > >> Forgot about instance type that don't have a cluster. Fixing it now. > >> > >> Shmuel > >> > >> On Mon, Mar 19, 2018 at 2:25 PM, Dafna Ron wrote: > >> > Hi, > >> > > >> > We have a failure in test 002_bootstrap.add_instance_type. > >> > There seem to be a NullPointerException on template type which is > >> > causing > >> > this failure. > >> > The same change that was reported at the last failure is reported as > the > >> > root cause for this failure as well, but I am not sure how it would > >> > cause > >> > this failure. > >> > Can you please check? > >> > > >> > > >> > Link and headline of suspected patches: > >> > > >> > reported as failed: > >> > core: fix removal of vm-host device - > >> > https://gerrit.ovirt.org/#/c/89145/ > >> > > >> > reported as root cause: > >> > > >> > core: USB in osinfo configuration depends on chipset - > >> > https://gerrit.ovirt.org/#/c/88777/ > >> > > >> > Link to Job: > >> > > >> > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6431 > >> > > >> > Link to all logs: > >> > > >> > > >> > http://jenkins.ovirt.org/job/ovirt-master_change-queue- > tester/6431/artifact/exported-artifacts/basic-suit-master- > el7/test_logs/basic-suite-master/post-002_bootstrap.py/ > >> > > >> > (Relevant) error snippet from the log: > >> > > >> > > >> > > >> > > >> > 2018-03-19 04:59:01,040-04 INFO > >> > [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-5) > >> > [5a890b17-51ec-4398-8d64-82cc71939e6e] Lock Acquired to object > >> > > >> > 'EngineLock:{exclusiveLocks='[99a9dfb3-9a13-4595-a795- > 693493e722be=TEMPLATE, > >> > myinstancetype=TEMPLATE_NAME]', sharedLocks='[]'}' > >> > 2018-03-19 04:59:01,087-04 INFO > >> > [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-5) > >> > [5a890b17-51ec-4398-8d64-82cc71939e6e] Running command: > >> > AddVmTemplateCommand > >> > internal: false. Entities affected : ID: aaa00000-0000-0000-0 > >> > 000-123456789aaa Type: SystemAction group CREATE_TEMPLATE with role > type > >> > USER > >> > 2018-03-19 04:59:01,139-04 INFO > >> > [org.ovirt.engine.core.bll.storage.disk. > CreateAllTemplateDisksCommand] > >> > (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] Running > command: > >> > CreateAllTemplateDisksCommand internal: true. > >> > 2018-03-19 04:59:01,205-04 INFO > >> > [org.ovirt.engine.core.utils.transaction.TransactionSupport] (default > >> > task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] transaction rolled > back > >> > 2018-03-19 04:59:01,205-04 ERROR > >> > [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-5) > >> > [5a890b17-51ec-4398-8d64-82cc71939e6e] Command > >> > 'org.ovirt.engine.core.bll.AddVmTemplateCommand' failed: null > >> > 2018-03-19 04:59:01,205-04 ERROR > >> > [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-5) > >> > [5a890b17-51ec-4398-8d64-82cc71939e6e] Exception: > >> > java.lang.NullPointerException > >> > at > >> > > >> > org.ovirt.engine.core.bll.utils.EmulatedMachineUtils.getEffective( > EmulatedMachineUtils.java:30) > >> > [bll.jar:] > >> > at > >> > > >> > org.ovirt.engine.core.bll.utils.EmulatedMachineUtils. > getEffectiveChipset(EmulatedMachineUtils.java:21) > >> > [bll.jar:] > >> > at > >> > > >> > org.ovirt.engine.core.bll.utils.VmDeviceUtils. > updateUsbSlots(VmDeviceUtils.java:744) > >> > [bll.jar:] > >> > at > >> > > >> > org.ovirt.engine.core.bll.utils.VmDeviceUtils. > copyVmDevices(VmDeviceUtils.java:1519) > >> > [bll.jar:] > >> > at > >> > > >> > org.ovirt.engine.core.bll.utils.VmDeviceUtils. > copyVmDevices(VmDeviceUtils.java:1565) > >> > [bll.jar:] > >> > at > >> > > >> > org.ovirt.engine.core.bll.AddVmTemplateCommand.lambda$ > executeCommand$4(AddVmTemplateCommand.java:362) > >> > [bll.jar:] > >> > at > >> > > >> > org.ovirt.engine.core.utils.transaction.TransactionSupport. > executeInNewTransaction(TransactionSupport.java:202) > >> > [utils.jar:] > >> > at > >> > > >> > org.ovirt.engine.core.bll.AddVmTemplateCommand.executeCommand( > AddVmTemplateCommand.java:345) > >> > [bll.jar:] > >> > at > >> > > >> > org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction( > CommandBase.java:1133) > >> > [bll.jar:] > >> > at > >> > > >> > org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScop > e(CommandBase.java:1286) > >> > [bll.jar:] > >> > at > >> > > >> > org.ovirt.engine.core.bll.CommandBase.runInTransaction( > CommandBase.java:1935) > >> > [bll.jar:] > >> > at > >> > > >> > org.ovirt.engine.core.utils.transaction.TransactionSupport. > executeInSuppressed(TransactionSupport.java:164) > >> > [utils.jar:] > >> > at > >> > > >> > org.ovirt.engine.core.utils.transaction.TransactionSupport. > executeInScope(TransactionSupport.java:103) > >> > [utils.jar:] > >> > at > >> > org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1346) > >> > [bll.jar:] > >> > at > >> > > >> > org.ovirt.engine.core.bll.CommandBase.executeAction( > CommandBase.java:400) > >> > [bll.jar:] > >> > at > >> > > >> > org.ovirt.engine.core.bll.executor.DefaultBackendActionExecutor. > execute(DefaultBackendActionExecutor.java:13) > >> > [bll.jar:] > >> > at org.ovirt.engine.core.bll.Backend.runAction(Backend. > java:468) > >> > [bll.jar:] > >> > at > >> > org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:450) > >> > [bll.jar:] > >> > at org.ovirt.engine.core.bll.Backend.runAction(Backend. > java:403) > >> > [bll.jar:] > >> > at sun.reflect.GeneratedMethodAccessor149.invoke(Unknown > Source) > >> > [:1.8.0_161] > >> > at > >> > > >> > sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:43) > >> > [rt.jar:1.8.0_161] > >> > at java.lang.reflect.Method.invoke(Method.java:498) > >> > [rt.jar:1.8.0_161] > >> > at > >> > > >> > org.jboss.as.ee.component.ManagedReferenceMethodIntercep > tor.processInvocation(ManagedReferenceMethodInterceptor.java:52) > >> > at > >> > > >> > org.jboss.invocation.InterceptorContext.proceed( > InterceptorContext.java:422) > >> > at > >> > > >> > org.jboss.invocation.InterceptorContext$Invocation. > proceed(InterceptorContext.java:509) > >> > at > >> > > >> > org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed( > DelegatingInterceptorInvocationContext.java:92) > >> > [wildfly-weld-ejb-11.0.0.Final.jar:11.0.0.Final] > >> > at > >> > > >> > org.jboss.weld.interceptor.proxy.WeldInvocationContext. > interceptorChainCompleted(WeldInvocationContext.java:98) > >> > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > >> > at > >> > > >> > org.jboss.weld.interceptor.proxy.WeldInvocationContext. > proceed(WeldInvocationContext.java:117) > >> > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > >> > at > >> > > >> > org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply( > LoggingInterceptor.java:12) > >> > [common.jar:] > >> > at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown > Source) > >> > [:1.8.0_161] > >> > at > >> > > >> > sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:43) > >> > [rt.jar:1.8.0_161] > >> > at java.lang.reflect.Method.invoke(Method.java:498) > >> > [rt.jar:1.8.0_161] > >> > at > >> > > >> > org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$ > SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73) > >> > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > >> > at > >> > > >> > org.jboss.weld.interceptor.proxy.WeldInvocationContext.invokeNext( > WeldInvocationContext.java:83) > >> > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > >> > at > >> > > >> > org.jboss.weld.interceptor.proxy.WeldInvocationContext. > proceed(WeldInvocationContext.java:115) > >> > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > >> > at > >> > org.jboss.weld.bean.InterceptorImpl.intercept( > InterceptorImpl.java:108) > >> > [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > >> > at > >> > > >> > org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed( > DelegatingInterceptorInvocationContext.java:82) > >> > [wildfly-weld-ejb-11.0.0.Final.jar:11.0.0.Final] > >> > at > >> > > >> > org.jboss.as.weld.interceptors.EjbComponentInterceptorSupport > .delegateInterception(EjbComponentInterceptorSupport.java:60) > >> > at > >> > > >> > org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor. > delegateInterception(Jsr299BindingsInterceptor.java:76) > >> > at > >> > > >> > org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor. > doMethodInterception(Jsr299BindingsInterceptor.java:88) > >> > at > >> > > >> > org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor. > processInvocation(Jsr299BindingsInterceptor.java:101) > >> > at > >> > > >> > org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1. > processInvocation(UserInterceptorFactory.java:63) > >> > at > >> > > >> > org.jboss.invocation.InterceptorContext.proceed( > InterceptorContext.java:422) > >> > at > >> > > >> > org.jboss.invocation.InterceptorContext$Invocation. > proceed(InterceptorContext.java:509) > >> > at > >> > > >> > org.ovirt.engine.core.bll.interceptors.CorrelationIdTrackerIntercepto > r.aroundInvoke(CorrelationIdTrackerInterceptor.java:13) > >> > [bll.jar:] > >> > at sun.reflect.GeneratedMethodAccessor66.invoke(Unknown > Source) > >> > [:1.8.0_161] > >> > at > >> > > >> > sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:43) > >> > [rt.jar:1.8.0_161] > >> > at java.lang.reflect.Method.invoke(Method.java:498) > >> > [rt.jar:1.8.0_161] > >> > at > >> > > >> > org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor. > processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:89) > >> > at > >> > > >> > org.jboss.invocation.InterceptorContext.proceed( > InterceptorContext.java:422) > >> > : > >> > > >> > > >> > End of error: > >> > > >> > at > >> > > >> > io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call( > ContextClassLoaderSetupAction.java:43) > >> > at > >> > > >> > org.wildfly.extension.undertow.security. > SecurityContextThreadSetupAction.lambda$create$0( > SecurityContextThreadSetupAction.java:105) > >> > at > >> > > >> > org.wildfly.extension.undertow.deployment. > UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0( > UndertowDeploymentInfoService.java:1508) > >> > at > >> > > >> > org.wildfly.extension.undertow.deployment. > UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0( > UndertowDeploymentInfoService.java:1508) > >> > at > >> > > >> > org.wildfly.extension.undertow.deployment. > UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0( > UndertowDeploymentInfoService.java:1508) > >> > at > >> > > >> > io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest( > ServletInitialHandler.java:272) > >> > at > >> > > >> > io.undertow.servlet.handlers.ServletInitialHandler.access$ > 000(ServletInitialHandler.java:81) > >> > at > >> > > >> > io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest( > ServletInitialHandler.java:104) > >> > at > >> > io.undertow.server.Connectors.executeRootHandler(Connectors.java:326) > >> > at > >> > io.undertow.server.HttpServerExchange$1.run( > HttpServerExchange.java:812) > >> > at > >> > > >> > java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1149) > >> > [rt.jar:1.8.0_161] > >> > at > >> > > >> > java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:624) > >> > [rt.jar:1.8.0_161] > >> > at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_161] > >> > > >> > 2018-03-19 04:59:01,234-04 DEBUG > >> > > >> > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$ > PostgresSimpleJdbcCall] > >> > (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] Compiled > stored > >> > procedure. Call string is [{call get_entity_snapshot_by_ > command_id(?)}] > >> > 2018-03-19 04:59:01,234-04 DEBUG > >> > > >> > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$ > PostgresSimpleJdbcCall] > >> > (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] SqlCall for > >> > procedure [get_entity_snapshot_by_command_id] compiled > >> > 2018-03-19 04:59:01,239-04 INFO > >> > [org.ovirt.engine.core.bll.CommandCompensator] (default task-5) > >> > [5a890b17-51ec-4398-8d64-82cc71939e6e] Command > >> > [id=a7a51354-ae84-4109-a55e-b27558afbf2a]: Compensating > NEW_ENTITY_ID of > >> > org.ovirt.engine.core.common.businessentities.VmTemplate; snapshot: > >> > 99a9dfb3-9a13-4595-a795-693493e722be. > >> > 2018-03-19 04:59:01,278-04 ERROR > >> > [org.ovirt.engine.core.dal.dbbroker.auditloghandling. > AuditLogDirector] > >> > (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] EVENT_ID: > >> > USER_ADD_VM_TEMPLATE_FAILURE(36), Failed creating Template > >> > myinstancetype. > >> > 2018-03-19 04:59:01,295-04 INFO > >> > [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-5) > >> > [5a890b17-51ec-4398-8d64-82cc71939e6e] Lock freed to object > >> > > >> > 'EngineLock:{exclusiveLocks='[99a9dfb3-9a13-4595-a795- > 693493e722be=TEMPLATE, > >> > myinstancetype=TEMPLATE_NAME]', sharedLocks='[]'}' > >> > 2018-03-19 04:59:01,295-04 DEBUG > >> > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > >> > (default task-5) [5a890b17-51ec-4398-8d64-82cc71939e6e] method: > >> > runAction, > >> > params: [AddVmTemplate, > >> > > >> > AddVmTemplateParameters:{commandId='a7a51354-ae84-4109- > a55e-b27558afbf2a', > >> > user='admin', commandType='AddVmTemplate'}], timeElapsed: 301ms > >> > 2018-03-19 04:59:01,301-04 ERROR > >> > [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] > (default > >> > task-5) [] Operation Failed: [Internal Engine Error] > >> > 2018-03-19 04:59:01,547-04 INFO > >> > [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] > >> > (EE-ManagedThreadFactory-engineScheduled-Thread-83) > >> > [5a890b17-51ec-4398-8d64-82cc71939e6e] Command 'AddVmTemplate' id: > >> > 'a7a51354-ae84-4109-a55e-b27558afbf2a' execution didn't complete, not > >> > proceeding to perform the next operation > >> > 2018-03-19 04:59:01,547-04 INFO > >> > [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] > >> > (EE-ManagedThreadFactory-engineScheduled-Thread-83) > >> > [5a890b17-51ec-4398-8d64-82cc71939e6e] Command 'AddVmTemplate' id: > >> > 'a7a51354-ae84-4109-a55e-b27558afbf2a' child commands > >> > '[8484f4d2-9ddf-48bd-9877-7c99ef555255]' executions were completed, > >> > status > >> > 'FAILED' > >> > 2018-03-19 04:59:01,581-04 INFO > >> > [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] > >> > (EE-ManagedThreadFactory-engineScheduled-Thread-83) > >> > [db5dd3dd-8b48-4647-a02d-723fcd6a7dda] Command 'ImportRepoImage' (id: > >> > 'a6475141-53ef-4ecd-bc17-ee8eae168dc7') waiting on child command id: > >> > '528fde29-9b5b-4a90-b62a-606fac3cd6a6' type:'DownloadImage' to > complete > >> > 2018-03-19 04:59:02,363-04 DEBUG > >> > [org.ovirt.otopi.dialog.MachineDialogParser] (VdsDeploy) [665e1c35] > Got: > >> > ***L:INFO Yum install: 349/515: gdisk-0.8.6-5.el7.x86_64 > >> > 2018-03-19 04:59:02,364-04 DEBUG > >> > [org.ovirt.otopi.dialog.MachineDialogParser] (VdsDeploy) [665e1c35] > >> > nextEvent: Log INFO Yum install: 349/515: gdisk-0.8.6-5.el7.x86_64 > >> > 2018-03-19 04:59:02,570-04 INFO > >> > [org.ovirt.engine.core.dal.dbbroker.auditloghandling. > AuditLogDirector] > >> > (VdsDeploy) [665e1c35] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), > >> > Installing > >> > Host lago-basic-suite-master-host-1. Yum install: 349/515: > >> > gdisk-0.8.6-5.el7.x86_64. > >> > 2018-03-19 04:59:02,666-04 ERROR > >> > [org.ovirt.engine.core.bll.AddVmTemplateCommand] > >> > (EE-ManagedThreadFactory-engineScheduled-Thread-86) > >> > [5a890b17-51ec-4398-8d64-82cc71939e6e] Ending command > >> > 'org.ovirt.engine.core.bll.AddVmTemplateCommand' with failure. > >> > > >> > > >> > > >> > > >> > > >> > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From phoracek at redhat.com Wed Mar 21 10:15:35 2018 From: phoracek at redhat.com (Petr Horacek) Date: Wed, 21 Mar 2018 11:15:35 +0100 Subject: [ovirt-devel] [vdsm] network test failure In-Reply-To: References: <78651164-7ebd-cb24-556e-909be31a8d68@redhat.com> Message-ID: I tried to retrigger the build several times, it was always executed on el7 machine, maybe it picks fc26 only when all other machines are taken? Shouldn't be "Permission denied" problem detected in link_bond_test.py:setup_module()? It runs "check_sysfs_bond_permission". 2018-03-20 18:12 GMT+01:00 Edward Haas : > The tests ran on a fc26 slave and our bond option default map is in sync > with the el7 kernel. > It looks like we MUST generate a bond default map on every run. > > I'm a bit surprised it never happened until now, perhaps I'm not > interpreting correctly the tests helper code? Petr? > Assuming I'm correct here, I'll try to post a fix. > > Thanks, > Edy. > > On Tue, Mar 20, 2018 at 12:14 PM, Dan Kenigsberg > wrote: > >> +Petr >> >> On Tue, Mar 20, 2018 at 11:07 AM, Francesco Romani >> wrote: >> > Hi all, >> > >> > >> > we had a bogus failure on CI again, some network test failed and it >> > seems totally unrelated to the patch being tested: >> > >> > >> > http://jenkins.ovirt.org/job/vdsm_master_check-patch-el7-x86 >> _64/22410/consoleFull >> > >> > >> > could someone please have a look? >> > >> > >> > Bests, >> > >> > -- >> > Francesco Romani >> > Senior SW Eng., Virtualization R&D >> > Red Hat >> > IRC: fromani github: @fromanirh >> > >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From didi at redhat.com Wed Mar 21 10:19:16 2018 From: didi at redhat.com (Yedidyah Bar David) Date: Wed, 21 Mar 2018 12:19:16 +0200 Subject: [ovirt-devel] [vdsm][maintainership] proposal for a new stable branch policy In-Reply-To: <789d5be7-ffe8-272a-9d67-69dcf8c15662@redhat.com> References: <789d5be7-ffe8-272a-9d67-69dcf8c15662@redhat.com> Message-ID: On Wed, Mar 21, 2018 at 11:57 AM, Francesco Romani wrote: > Hi all, > > > Recently Milan, Petr and me discussed the state of ovirt.4.2, > considering that release 4.2.2 is still pending and this prevents > merging of patches in the sub-branch ovirt-4.2. > > We agreed we could improve the handling of the stable branch(es), in > order to make the process smoother and more predictable. > > > Currently, we avoid doing Z- branches (e.g. ovirt-4.2.2, ovirt-4.2.3...) > as much as we can, to avoid the hassle of double-backporting patches to > stable branch. > > However, if a release hits unexpected delay, this policy causes > different hassles: the Y-branch (ovirt-4.2, ovirt-4.3) is effectively > locked, so patches already queued > > and ready for next releases can't be merged and need to wait. > > > The new proposed policy is the following: > > - we will keep working exactly as now until we hit a certain RC version. > We choosed RC3, a rule of thumb made out of experience. > > - if RC3 is final, everyone's happy and things resume as usual > > - if RC3 is NOT final, we will branch out at RC3 > > -- from that moment on, patches for next version could be accepted on > the Y-branch > > -- stabilization of the late Z version will continue on the Z-branch > > -- patches will be backported twice > > > Example using made up numbers > > - We just released ovirt 4.3.1 > > - We are working on the ovirt-4.3 branch > > - The last tag is v4.30.10, from ovirt-4.3 branch > > - We accept patches for ovirt 4.3.2 on the ovirt-4.3 branch > > - We keep collecting patches, until we tag v4.30.11 (oVirt 4.3.2 RC 1). > Tag is made from ovirt-4.3 branch. > > - Same for tags 4.30.12 - oVirt 4.3.2 RC 2 and 4.30.13 - oVirt 4.3.2 RC > 3. Both tags are made from ovirt-4.3 branch. > > - Damn, RC3 is not final. We branch out ovirt-4.3.2 form branch > ovirt-4.3 from the same commit pointed by tag 4.30.13 > > - Next tags (4.30.13.1) for ovirt 4.3.2 will be taken from the > ovirt-4.3.2 branch > > > I believe this approach will make predictable for everyone if and when > the branch will be made, so when the patches could be merged and where. > > > The only drawback I can see - and that I realized while writing the > example - is the version number which can be awkward: > > > v4.30.11 -> 4.3.2 RC1 > > v4.30.12 -> 4.3.2 RC2 > > v4.30.13 -> 4.3.2 RC3 > > v4.30.13.1 -> 4.3.2 RC4 ?!?! > > v4.30.13.5 -> 4.3.2 RC5 ?!?! In principle, nothing forces you to align vdsm versions exactly with oVirt (rc) releases. In otopi, as a much simpler example (and there are many here), I do mostly what I want, as needed. I usually bump versions when I have enough bugs fixed for a next project release, but _also_ if I want to introduce some change that some other project needs - so that I can bump the 'Requires: otopi >= X.Y.Z' line of the other project. So the main question to ask, if you do want to keep above scheme, is: Do you expect, in above example, to have to tag/build vdsm for 4.3.3 before 4.3.2 is released? If not, then you can have: v4.30.11 -> 4.3.2 RC1 v4.30.12 -> 4.3.2 RC2 v4.30.13 -> 4.3.2 RC3 v4.30.14 -> 4.3.2 RC4 v4.30.15 -> 4.3.2 RC5 And the only problem will be what to do in the -4.3 branch when you branch -4.3.2 at RC3. Can't think about a good answer for this... It's probably not a good idea to keep it at 4.30.13 (or .12), because you want builds from it to have higher versions than from the -4.3.2 branch. > > > Perhaps we should move to four versions digit? So we could have > > > v4.30.11.0 -> 4.3.2 RC1 > > v4.30.11.1 -> 4.3.2 RC2 > > v4.30.11.2 -> 4.3.2 RC3 > > v4.30.11.3 -> 4.3.2 RC4 > > v4.30.11.4 -> 4.3.2 RC5 > > > I don't see any real drawback in using 4-digit versions by default, > besides a minor increase in complexity, which is balanced by more > predictable and consistent > versions. Plus, we already had 4-digit versions in Vdsm, so packaging > should work just fine. Makes sense to me. Not sure if this was discussed, but in practice the engine already does that, more-or-less (the tags, not branches). One of the "prices" of adding branches is having to maintain CI for them. Perhaps this policy should also consider what we do if/when migrating to STDCI v2, see e.g.: http://lists.ovirt.org/pipermail/infra/2018-March/033224.html And also affect, if needed, this v2 project. Obviously it would be great if you could have automatically had CI handled in the same single commit that bumps the version. You will still have the price of having to cherry-pick, but I can't see how you can escape that. > > Please share your thoughts, > > > -- > Francesco Romani > Senior SW Eng., Virtualization R&D > Red Hat > IRC: fromani github: @fromanirh > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel -- Didi From michal.skrivanek at redhat.com Wed Mar 21 10:28:20 2018 From: michal.skrivanek at redhat.com (Michal Skrivanek) Date: Wed, 21 Mar 2018 11:28:20 +0100 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (Ovirt-ngine) ] [ 19-03-2018 ] [ 002_bootstrap.verify_notifier ] In-Reply-To: References: <46392D70-A8F0-4D32-BAD1-5F1AA0680AC7@redhat.com> Message-ID: > On 20 Mar 2018, at 10:33, Yaniv Kaul wrote: > > > > On Tue, Mar 20, 2018 at 9:49 AM, Michal Skrivanek > wrote: > I?m not sure what is that test actually testing, if it depends on the previous host action which fails but is not verified it still may be relevant to Shmuel?s patch > adding author of the test and the notifier owner > > It is checking that the notifier works - which sends SNMP notifications on our events. I happen to picked an event which is VDC_STOP - which happens when the engine is restarted - which happens earlier, when we configure it. not sure if that?s reliable then. Doesn?t look related to that patch at all. Dafna, does the same error reproduce all the time after that exact patch? > Y. > > >> On 19 Mar 2018, at 13:06, Dafna Ron > wrote: >> >> Hi, >> >> We had a failure in test 002_bootstrap.verify_notifier. >> I can't see anything wrong with the notifier and I don't think it should be related to the change that was reported. >> >> the test itself is looking for vdc_stop in messages which I do not indeed see but I am not sure what is the cause and how the reported change related to the failure. >> >> Can you please take a look? >> >> >> >> Link and headline of suspected patches: >> core: USB in osinfo configuration depends on chipset - https://gerrit.ovirt.org/#/c/88777/ >> >> Link to Job: >> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6429/ >> >> Link to all logs: >> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6429/artifacts >> >> (Relevant) error snippet from the log: >> >> >> this is the error from the api: >> >> >> Error Message >> >> Failed grep for VDC_STOP with code 1. Output: >> -------------------- >> begin captured logging << -------------------- >> lago.ssh: DEBUG: start task:f1231b27-f796-406c-8618-17b0868725bc:Get ssh client for lago-basic-suite-master-engine: >> lago.ssh: DEBUG: end task:f1231b27-f796-406c-8618-17b0868725bc:Get ssh client for lago-basic-suite-master-engine: >> lago.ssh: DEBUG: Running 1cce7c0c on lago-basic-suite-master-engine: grep VDC_STOP /var/log/messages >> lago.ssh: DEBUG: Command 1cce7c0c on lago-basic-suite-master-engine returned with 1 >> --------------------- >> end captured logging << --------------------- >> Stacktrace >> >> File "/usr/lib64/python2.7/unittest/case.py", line 369, in run >> testMethod() >> File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest >> self.test(*self.arg) >> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test >> test() >> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper >> return func(get_test_prefix(), *args, **kwargs) >> File "/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 1456, in verify_notifier >> 'Failed grep for VDC_STOP with code {0}. Output: {1}'.format(result.code, result.out) >> File "/usr/lib/python2.7/site-packages/nose/tools/trivial.py", line 29, in eq_ >> raise AssertionError(msg or "%r != %r" % (a, b)) >> 'Failed grep for VDC_STOP with code 1. Output: \n-------------------- >> begin captured logging << --------------------\nlago.ssh: DEBUG: start task:f1231b27-f796-406c-8618-17b0868725bc:Get ssh client for lago-basic-suite-master-engine:\nlago.ssh: DEBUG: end task:f1231b27-f796-406c-8618-17b0868725bc:Get ssh client for lago-basic-suite-master-engine:\nlago.ssh: DEBUG: Running 1cce7c0c on lago-basic-suite-master-engine: grep VDC_STOP /var/log/messages\nlago.ssh: DEBUG: Command 1cce7c0c on lago-basic-suite-master-engine returned with 1\n--------------------- >> end captured logging << ---------------------' >> >> >> >> _______________________________________________ >> Devel mailing list >> Devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dron at redhat.com Wed Mar 21 11:18:07 2018 From: dron at redhat.com (Dafna Ron) Date: Wed, 21 Mar 2018 11:18:07 +0000 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (Ovirt-ngine) ] [ 19-03-2018 ] [ 002_bootstrap.verify_notifier ] In-Reply-To: References: <46392D70-A8F0-4D32-BAD1-5F1AA0680AC7@redhat.com> Message-ID: The patch was reported to cause 3 different issues but I cannot say for sure that the notifier error was actually one of them. I can say that all issues from master and 4.2 were cleared once the fix from Shmuel was merged and tested in cq. On Wed, Mar 21, 2018 at 10:28 AM, Michal Skrivanek < michal.skrivanek at redhat.com> wrote: > > > On 20 Mar 2018, at 10:33, Yaniv Kaul wrote: > > > > On Tue, Mar 20, 2018 at 9:49 AM, Michal Skrivanek < > michal.skrivanek at redhat.com> wrote: > >> I?m not sure what is that test actually testing, if it depends on the >> previous host action which fails but is not verified it still may be >> relevant to Shmuel?s patch >> adding author of the test and the notifier owner >> > > It is checking that the notifier works - which sends SNMP notifications on > our events. I happen to picked an event which is VDC_STOP - which happens > when the engine is restarted - which happens earlier, when we configure it. > > > not sure if that?s reliable then. Doesn?t look related to that patch at > all. Dafna, does the same error reproduce all the time after that exact > patch? > > Y. > > >> >> On 19 Mar 2018, at 13:06, Dafna Ron wrote: >> >> Hi, >> >> We had a failure in test 002_bootstrap.verify_notifier. >> I can't see anything wrong with the notifier and I don't think it should >> be related to the change that was reported. >> >> the test itself is looking for vdc_stop in messages which I do not indeed >> see but I am not sure what is the cause and how the reported change related >> to the failure. >> >> Can you please take a look? >> >> >> >> *Link and headline of suspected patches: * >> >> >> >> >> >> >> *core: USB in osinfo configuration depends on chipset - >> https://gerrit.ovirt.org/#/c/88777/ >> Link to >> Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6429/ >> Link >> to all >> logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6429/artifacts >> (Relevant) >> error snippet from the log: * >> *this is the error from *the api: >> >> >> Error Message >> >> Failed grep for VDC_STOP with code 1. Output: >> -------------------- >> begin captured logging << -------------------- >> lago.ssh: DEBUG: start task:f1231b27-f796-406c-8618-17b0868725bc:Get ssh client for lago-basic-suite-master-engine: >> lago.ssh: DEBUG: end task:f1231b27-f796-406c-8618-17b0868725bc:Get ssh client for lago-basic-suite-master-engine: >> lago.ssh: DEBUG: Running 1cce7c0c on lago-basic-suite-master-engine: grep VDC_STOP /var/log/messages >> lago.ssh: DEBUG: Command 1cce7c0c on lago-basic-suite-master-engine returned with 1 >> --------------------- >> end captured logging << --------------------- >> >> Stacktrace >> >> File "/usr/lib64/python2.7/unittest/case.py", line 369, in run >> testMethod() >> File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest >> self.test(*self.arg) >> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test >> test() >> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper >> return func(get_test_prefix(), *args, **kwargs) >> File "/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 1456, in verify_notifier >> 'Failed grep for VDC_STOP with code {0}. Output: {1}'.format(result.code, result.out) >> File "/usr/lib/python2.7/site-packages/nose/tools/trivial.py", line 29, in eq_ >> raise AssertionError(msg or "%r != %r" % (a, b)) >> 'Failed grep for VDC_STOP with code 1. Output: \n-------------------- >> begin captured logging << --------------------\nlago.ssh: DEBUG: start task:f1231b27-f796-406c-8618-17b0868725bc:Get ssh client for lago-basic-suite-master-engine:\nlago.ssh: DEBUG: end task:f1231b27-f796-406c-8618-17b0868725bc:Get ssh client for lago-basic-suite-master-engine:\nlago.ssh: DEBUG: Running 1cce7c0c on lago-basic-suite-master-engine: grep VDC_STOP /var/log/messages\nlago.ssh: DEBUG: Command 1cce7c0c on lago-basic-suite-master-engine returned with 1\n--------------------- >> end captured logging << ---------------------' >> >> >> ** >> _______________________________________________ >> Devel mailing list >> Devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel >> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ehaas at redhat.com Wed Mar 21 11:59:26 2018 From: ehaas at redhat.com (Edward Haas) Date: Wed, 21 Mar 2018 13:59:26 +0200 Subject: [ovirt-devel] [vdsm] network test failure In-Reply-To: References: <78651164-7ebd-cb24-556e-909be31a8d68@redhat.com> Message-ID: On Wed, Mar 21, 2018 at 12:15 PM, Petr Horacek wrote: > I tried to retrigger the build several times, it was always executed on > el7 machine, maybe it picks fc26 only when all other machines are taken? > > Shouldn't be "Permission denied" problem detected in > link_bond_test.py:setup_module()? It runs "check_sysfs_bond_permission". > I think that the error is from attempting to set a values that is not acceptable by the attribute, not because there is no permission to access sysfs. > 2018-03-20 18:12 GMT+01:00 Edward Haas : > >> The tests ran on a fc26 slave and our bond option default map is in sync >> with the el7 kernel. >> It looks like we MUST generate a bond default map on every run. >> >> I'm a bit surprised it never happened until now, perhaps I'm not >> interpreting correctly the tests helper code? Petr? >> Assuming I'm correct here, I'll try to post a fix. >> >> Thanks, >> Edy. >> >> On Tue, Mar 20, 2018 at 12:14 PM, Dan Kenigsberg >> wrote: >> >>> +Petr >>> >>> On Tue, Mar 20, 2018 at 11:07 AM, Francesco Romani >>> wrote: >>> > Hi all, >>> > >>> > >>> > we had a bogus failure on CI again, some network test failed and it >>> > seems totally unrelated to the patch being tested: >>> > >>> > >>> > http://jenkins.ovirt.org/job/vdsm_master_check-patch-el7-x86 >>> _64/22410/consoleFull >>> > >>> > >>> > could someone please have a look? >>> > >>> > >>> > Bests, >>> > >>> > -- >>> > Francesco Romani >>> > Senior SW Eng., Virtualization R&D >>> > Red Hat >>> > IRC: fromani github: @fromanirh >>> > >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From godas at redhat.com Wed Mar 21 12:55:28 2018 From: godas at redhat.com (Gobinda Das) Date: Wed, 21 Mar 2018 18:25:28 +0530 Subject: [ovirt-devel] UI is not coming up after pulled latest code from master Message-ID: Hi, UI keeps on loading and I can see below error in browser. webadmin-0.js:422 Wed Mar 21 14:59:47 GMT+530 2018 com.google.gwt.logging.client.LogConfiguration SEVERE: Possible problem with your *.gwt.xml module file. The compile time user.agent value (gecko1_8) does not match the runtime user.agent value (safari). Expect more errors. com.google.gwt.useragent.client.UserAgentAsserter$UserAgentAssertionError: Possible problem with your *.gwt.xml module file. The compile time user.agent value (gecko1_8) does not match the runtime user.agent value (safari). Expect more errors. at Unknown.L(webadmin-0.js) at Unknown.oj(webadmin-0.js) at Unknown.new pj(webadmin-0.js) at Unknown.nj(webadmin-0.js) at Unknown.nb(webadmin-0.js) at Unknown.qb(webadmin-0.js) at Unknown.eval(webadmin-0.js) -- Thanks, Gobinda -------------- next part -------------- An HTML attachment was scrubbed... URL: From gshereme at redhat.com Wed Mar 21 13:01:01 2018 From: gshereme at redhat.com (Greg Sheremeta) Date: Wed, 21 Mar 2018 09:01:01 -0400 Subject: [ovirt-devel] UI is not coming up after pulled latest code from master In-Reply-To: References: Message-ID: That message is probably not related, and is generally harmless. Compile with DEV_EXTRA_BUILD_FLAGS_GWT_DEFAULTS="-Dgwt.userAgent=safari,gecko1_8" to stop that message. Does it work in firefox? Please share your ui.log from the server. Greg On Wed, Mar 21, 2018 at 8:55 AM, Gobinda Das wrote: > Hi, > UI keeps on loading and I can see below error in browser. > > webadmin-0.js:422 Wed Mar 21 14:59:47 GMT+530 2018 > com.google.gwt.logging.client.LogConfiguration > SEVERE: Possible problem with your *.gwt.xml module file. > The compile time user.agent value (gecko1_8) does not match the runtime > user.agent value (safari). > Expect more errors. > com.google.gwt.useragent.client.UserAgentAsserter$UserAgentAssertionError: > Possible problem with your *.gwt.xml module file. > The compile time user.agent value (gecko1_8) does not match the runtime > user.agent value (safari). > Expect more errors. > at Unknown.L(webadmin-0.js) > at Unknown.oj(webadmin-0.js) > at Unknown.new pj(webadmin-0.js) > at Unknown.nj(webadmin-0.js) > at Unknown.nb(webadmin-0.js) > at Unknown.qb(webadmin-0.js) > at Unknown.eval(webadmin-0.js) > > -- > Thanks, > Gobinda > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -- GREG SHEREMETA SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX Red Hat NA gshereme at redhat.com IRC: gshereme -------------- next part -------------- An HTML attachment was scrubbed... URL: From bzlotnik at redhat.com Wed Mar 21 13:07:01 2018 From: bzlotnik at redhat.com (Benny Zlotnik) Date: Wed, 21 Mar 2018 15:07:01 +0200 Subject: [ovirt-devel] UI is not coming up after pulled latest code from master In-Reply-To: References: Message-ID: Had this issue too (FF & Chrome), I've recompiled, ran engine-setup again and it resolved On Wed, Mar 21, 2018 at 3:01 PM, Greg Sheremeta wrote: > That message is probably not related, and is generally harmless. Compile > with DEV_EXTRA_BUILD_FLAGS_GWT_DEFAULTS="-Dgwt.userAgent=safari,gecko1_8" > to stop that message. > > Does it work in firefox? Please share your ui.log from the server. > > Greg > > On Wed, Mar 21, 2018 at 8:55 AM, Gobinda Das wrote: > >> Hi, >> UI keeps on loading and I can see below error in browser. >> >> webadmin-0.js:422 Wed Mar 21 14:59:47 GMT+530 2018 >> com.google.gwt.logging.client.LogConfiguration >> SEVERE: Possible problem with your *.gwt.xml module file. >> The compile time user.agent value (gecko1_8) does not match the runtime >> user.agent value (safari). >> Expect more errors. >> com.google.gwt.useragent.client.UserAgentAsserter$UserAgentAssertionError: >> Possible problem with your *.gwt.xml module file. >> The compile time user.agent value (gecko1_8) does not match the runtime >> user.agent value (safari). >> Expect more errors. >> at Unknown.L(webadmin-0.js) >> at Unknown.oj(webadmin-0.js) >> at Unknown.new pj(webadmin-0.js) >> at Unknown.nj(webadmin-0.js) >> at Unknown.nb(webadmin-0.js) >> at Unknown.qb(webadmin-0.js) >> at Unknown.eval(webadmin-0.js) >> >> -- >> Thanks, >> Gobinda >> >> _______________________________________________ >> Devel mailing list >> Devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel >> > > > > -- > > GREG SHEREMETA > > SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX > > Red Hat NA > > > > gshereme at redhat.com IRC: gshereme > > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mzamazal at redhat.com Wed Mar 21 13:36:38 2018 From: mzamazal at redhat.com (Milan Zamazal) Date: Wed, 21 Mar 2018 14:36:38 +0100 Subject: [ovirt-devel] [vdsm][maintainership] proposal for a new stable branch policy In-Reply-To: <789d5be7-ffe8-272a-9d67-69dcf8c15662@redhat.com> (Francesco Romani's message of "Wed, 21 Mar 2018 10:57:48 +0100") References: <789d5be7-ffe8-272a-9d67-69dcf8c15662@redhat.com> Message-ID: <87in9pczyx.fsf@redhat.com> Francesco Romani writes: > Recently Milan, Petr and me discussed the state of ovirt.4.2, > considering that release 4.2.2 is still pending and this prevents > merging of patches in the sub-branch ovirt-4.2. > > We agreed we could improve the handling of the stable branch(es), in > order to make the process smoother and more predictable. > > > Currently, we avoid doing Z- branches (e.g. ovirt-4.2.2, ovirt-4.2.3...) > as much as we can, to avoid the hassle of double-backporting patches to > stable branch. > > However, if a release hits unexpected delay, this policy causes > different hassles: the Y-branch (ovirt-4.2, ovirt-4.3) is effectively > locked, so patches already queued > > and ready for next releases can't be merged and need to wait. > > > The new proposed policy is the following: > > - we will keep working exactly as now until we hit a certain RC version. > We choosed RC3, a rule of thumb made out of experience. I think the idea actually was to choose the RC after which patch acceptation is restricted (e.g. blockers only). That would be consistent with the problem being solved, i.e. not to block patch merging and to clearly distinguish between patches to be merged to X.Y only and patches to be merged to X.Y.Z. If may typically be RC3, but I wouldn't hard-bind the branching to a particular version. > - if RC3 is final, everyone's happy and things resume as usual > > - if RC3 is NOT final, we will branch out at RC3 > > -- from that moment on, patches for next version could be accepted on > the Y-branch > > -- stabilization of the late Z version will continue on the Z-branch > > -- patches will be backported twice > > > Example using made up numbers > > - We just released ovirt 4.3.1 > > - We are working on the ovirt-4.3 branch > > - The last tag is v4.30.10, from ovirt-4.3 branch > > - We accept patches for ovirt 4.3.2 on the ovirt-4.3 branch > > - We keep collecting patches, until we tag v4.30.11 (oVirt 4.3.2 RC 1). > Tag is made from ovirt-4.3 branch. > > - Same for tags 4.30.12 - oVirt 4.3.2 RC 2 and 4.30.13 - oVirt 4.3.2 RC > 3. Both tags are made from ovirt-4.3 branch. > > - Damn, RC3 is not final. We branch out ovirt-4.3.2 form branch > ovirt-4.3 from the same commit pointed by tag 4.30.13 > > - Next tags (4.30.13.1) for ovirt 4.3.2 will be taken from the > ovirt-4.3.2 branch > > > I believe this approach will make predictable for everyone if and when > the branch will be made, so when the patches could be merged and where. > > > The only drawback I can see - and that I realized while writing the > example - is the version number which can be awkward: > > > v4.30.11 -> 4.3.2 RC1 > > v4.30.12 -> 4.3.2 RC2 > > v4.30.13 -> 4.3.2 RC3 > > v4.30.13.1 -> 4.3.2 RC4 ?!?! > > v4.30.13.5 -> 4.3.2 RC5 ?!?! Our current versions are not tied to RC versions (e.g. there can be more than one tag per one RC), so I can see no special problem with using v4.30.13.1 for RC4 and v4.30.13.2 for RC5. > Perhaps we should move to four versions digit? So we could have > > > v4.30.11.0 -> 4.3.2 RC1 > > v4.30.11.1 -> 4.3.2 RC2 > > v4.30.11.2 -> 4.3.2 RC3 > > v4.30.11.3 -> 4.3.2 RC4 > > v4.30.11.4 -> 4.3.2 RC5 > > > I don't see any real drawback in using 4-digit versions by default, > besides a minor increase in complexity, which is balanced by more > predictable and consistent > versions. Plus, we already had 4-digit versions in Vdsm, so packaging > should work just fine. > > Please share your thoughts, Yedidyah Bar David writes: > So the main question to ask, if you do want to keep above scheme, > is: Do you expect, in above example, to have to tag/build vdsm for 4.3.3 > before 4.3.2 is released? We probably want to make a new 4.3 tag as soon as one new patch is merged to the 4.3 branch after 4.3.Z is branched out, to distinguish any builds (private, snapshots, ?) made from that. From godas at redhat.com Thu Mar 22 08:27:37 2018 From: godas at redhat.com (Gobinda Das) Date: Thu, 22 Mar 2018 13:57:37 +0530 Subject: [ovirt-devel] UI is not coming up after pulled latest code from master In-Reply-To: References: Message-ID: Still no luck.I recompiled as did engine-setup but still same issue. I tried in both (FF and Chrome) I also tried to Compile with DEV_EXTRA_BUILD_FLAGS_GWT_DEFAULTS="-Dgwt.userAgent=safari,gecko1_8" but webadmin giving compilation error like: [INFO] [INFO] --- gwt-maven-plugin:2.8.0:compile (gwtcompile) @ webadmin --- [INFO] info AspectJ Weaver Version 1.8.10 built on Monday Dec 12, 2016 at 19:07:48 GMT [INFO] info register classloader sun.misc.Launcher$AppClassLoader at 18b4aac2 [INFO] info using configuration file:/home/godas/workspace/ovirt-engine/frontend/webadmin/modules/gwt-aop/target/gwt-aop-4.3.0-SNAPSHOT.jar!/META-INF/aop.xml [INFO] info register aspect org.ovirt.engine.ui.gwtaop.DontPrune [INFO] Mar 22, 2018 12:58:37 PM java.util.prefs.FileSystemPreferences$1 run [INFO] INFO: Created user preferences directory. [ERROR] The attempt to retrieve files in /home/godas/workspace/ovirt-engine/frontend/webadmin/modules/webadmin/target/webadmin-4.3.0-SNAPSHOT/WEB-INF/classes failed. [ERROR] java.io.IOException: User limit of inotify watches reached [ERROR] at sun.nio.fs.LinuxWatchService$Poller.implRegister(LinuxWatchService.java:264) [ERROR] at sun.nio.fs.AbstractPoller.processRequests(AbstractPoller.java:260) [ERROR] at sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService.java:364) [ERROR] at java.lang.Thread.run(Thread.java:748) [ERROR] The attempt to retrieve files in /home/godas/workspace/ovirt-engine/frontend/webadmin/modules/webadmin/src/main/java failed. [ERROR] java.io.IOException: User limit of inotify watches reached [ERROR] at sun.nio.fs.LinuxWatchService$Poller.implRegister(LinuxWatchService.java:264) [ERROR] at sun.nio.fs.AbstractPoller.processRequests(AbstractPoller.java:260) [ERROR] at sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService.java:364) [ERROR] at java.lang.Thread.run(Thread.java:748) [ERROR] The attempt to retrieve files in /home/godas/workspace/ovirt-engine/frontend/webadmin/modules/webadmin/target/generated-sources/annotations failed. [ERROR] java.io.IOException: User limit of inotify watches reached [ERROR] at sun.nio.fs.LinuxWatchService$Poller.implRegister(LinuxWatchService.java:264) [ERROR] at sun.nio.fs.AbstractPoller.processRequests(AbstractPoller.java:260) [ERROR] at sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService.java:364) [ERROR] at java.lang.Thread.run(Thread.java:748) [ERROR] The attempt to retrieve files in /home/godas/workspace/ovirt-engine/frontend/webadmin/modules/webadmin/target/generated-sources/gwt failed. [ERROR] java.io.IOException: User limit of inotify watches reached [ERROR] at sun.nio.fs.LinuxWatchService$Poller.implRegister(LinuxWatchService.java:264) [ERROR] at sun.nio.fs.AbstractPoller.processRequests(AbstractPoller.java:260) [ERROR] at sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService.java:364) [ERROR] at java.lang.Thread.run(Thread.java:748) [ERROR] The attempt to retrieve files in /home/godas/workspace/ovirt-engine/frontend/webadmin/modules/webadmin/target/webadmin-4.3.0-SNAPSHOT/WEB-INF/classes failed. [ERROR] java.io.IOException: User limit of inotify watches reached [ERROR] at sun.nio.fs.LinuxWatchService$Poller.implRegister(LinuxWatchService.java:264) [ERROR] at sun.nio.fs.AbstractPoller.processRequests(AbstractPoller.java:260) [ERROR] at sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService.java:364) [ERROR] at java.lang.Thread.run(Thread.java:748) [ERROR] The attempt to retrieve files in /home/godas/workspace/ovirt-engine/frontend/webadmin/modules/webadmin/src/main/java failed. [ERROR] java.io.IOException: User limit of inotify watches reached [ERROR] at sun.nio.fs.LinuxWatchService$Poller.implRegister(LinuxWatchService.java:264) [ERROR] at sun.nio.fs.AbstractPoller.processRequests(AbstractPoller.java:260) [ERROR] at sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService.java:364) [ERROR] at java.lang.Thread.run(Thread.java:748) [ERROR] The attempt to retrieve files in /home/godas/workspace/ovirt-engine/frontend/webadmin/modules/webadmin/target/generated-sources/annotations failed. [ERROR] java.io.IOException: User limit of inotify watches reached [ERROR] at sun.nio.fs.LinuxWatchService$Poller.implRegister(LinuxWatchService.java:264) [ERROR] at sun.nio.fs.AbstractPoller.processRequests(AbstractPoller.java:260) [ERROR] at sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService.java:364) [ERROR] at java.lang.Thread.run(Thread.java:748) [ERROR] The attempt to retrieve files in /home/godas/workspace/ovirt-engine/frontend/webadmin/modules/webadmin/target/generated-sources/gwt failed. [ERROR] java.io.IOException: User limit of inotify watches reached [ERROR] at sun.nio.fs.LinuxWatchService$Poller.implRegister(LinuxWatchService.java:264) [ERROR] at sun.nio.fs.AbstractPoller.processRequests(AbstractPoller.java:260) [ERROR] at sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService.java:364) [ERROR] at java.lang.Thread.run(Thread.java:748) [ERROR] The attempt to retrieve files in /home/godas/workspace/ovirt-engine/frontend/webadmin/modules/webadmin/target/webadmin-4.3.0-SNAPSHOT/WEB-INF/classes failed. [ERROR] java.io.IOException: User limit of inotify watches reached [ERROR] at sun.nio.fs.LinuxWatchService$Poller.implRegister(LinuxWatchService.java:264) [ERROR] at sun.nio.fs.AbstractPoller.processRequests(AbstractPoller.java:260) [ERROR] at sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService.java:364) [ERROR] at java.lang.Thread.run(Thread.java:748) [ERROR] The attempt to retrieve files in /home/godas/workspace/ovirt-engine/frontend/webadmin/modules/webadmin/src/main/java failed. [ERROR] java.io.IOException: User limit of inotify watches reached [ERROR] at sun.nio.fs.LinuxWatchService$Poller.implRegister(LinuxWatchService.java:264) [ERROR] at sun.nio.fs.AbstractPoller.processRequests(AbstractPoller.java:260) [ERROR] at sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService.java:364) [ERROR] at java.lang.Thread.run(Thread.java:748) [ERROR] The attempt to retrieve files in /home/godas/workspace/ovirt-engine/frontend/webadmin/modules/webadmin/target/generated-sources/annotations failed. [ERROR] java.io.IOException: User limit of inotify watches reached [ERROR] at sun.nio.fs.LinuxWatchService$Poller.implRegister(LinuxWatchService.java:264) [ERROR] at sun.nio.fs.AbstractPoller.processRequests(AbstractPoller.java:260) [ERROR] at sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService.java:364) [ERROR] at java.lang.Thread.run(Thread.java:748) [ERROR] The attempt to retrieve files in /home/godas/workspace/ovirt-engine/frontend/webadmin/modules/webadmin/target/generated-sources/gwt failed. [ERROR] java.io.IOException: User limit of inotify watches reached [ERROR] at sun.nio.fs.LinuxWatchService$Poller.implRegister(LinuxWatchService.java:264) [ERROR] at sun.nio.fs.AbstractPoller.processRequests(AbstractPoller.java:260) [ERROR] at sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService.java:364) [ERROR] at java.lang.Thread.run(Thread.java:748) [INFO] Compiling module org.ovirt.engine.ui.webadmin.WebAdmin [INFO] warning javax.* types are not being woven because the weaver option '-Xset:weaveJavaxPackages=true' has not been specified [INFO] Computing all possible rebind results for 'com.gwtplatform.mvp.client.ApplicationController' [INFO] Rebinding com.gwtplatform.mvp.client.ApplicationController [INFO] Invoking generator com.gwtplatform.mvp.rebind.ApplicationControllerGenerator [INFO] [ERROR] The type 'org.ovirt.engine.ui.webadmin.system.ApplicationInit' was not found, either the class name is wrong or there are compile errors in your code. [INFO] [ERROR] The type 'org.ovirt.engine.ui.webadmin.system.ApplicationInit' was not found, either the class name is wrong or there are compile errors in your code. [INFO] [ERROR] There was a problem generating the ApplicationController, this can be caused by bad GWT module configuration or compile errors in your source code. [INFO] [WARN] For the following type(s), generated source was never committed (did you forget to call commit()?) [INFO] [WARN] com.gwtplatform.mvp.client.ApplicationControllerImpl [INFO] [ERROR] Could not find com.gwtplatform.mvp.client.ApplicationControllerImpl in types compiled from source. Is the source glob too strict? [INFO] [ERROR] Errors in 'gen/com/google/gwt/lang/org_00046ovirt_00046engine_00046ui_00046webadmin_00046WebAdmin__EntryMethodHolder.java' [INFO] [ERROR] Line 3: Rebind result 'com.gwtplatform.mvp.client.ApplicationControllerImpl' could not be found [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] oVirt Findbugs Filters ............................. SUCCESS [ 0.914 s] [INFO] ovirt-root ......................................... SUCCESS [ 1.138 s] [INFO] oVirt Build Tools root ............................. SUCCESS [ 0.480 s] [INFO] oVirt checkstyle ................................... SUCCESS [ 1.161 s] [INFO] oVirt Checkstyle Checks ............................ SUCCESS [ 1.298 s] [INFO] oVirt Modules - backend ............................ SUCCESS [ 0.477 s] [INFO] oVirt Manager ...................................... SUCCESS [ 0.440 s] [INFO] oVirt Engine dependencies .......................... SUCCESS [ 0.406 s] [INFO] oVirt Engine common dependencies ................... SUCCESS [ 1.485 s] [INFO] oVirt Engine tools dependencies .................... SUCCESS [ 0.490 s] [INFO] Utilities to extend java.logging framework ......... SUCCESS [ 0.781 s] [INFO] Extensions API root ................................ SUCCESS [ 0.175 s] [INFO] ovirt-engine-extensions-api ........................ SUCCESS [ 4.015 s] [INFO] oVirt Modules - manager ............................ SUCCESS [ 1.279 s] [INFO] Universal utilities ................................ SUCCESS [ 13.829 s] [INFO] Extensions manager ................................. SUCCESS [ 2.554 s] [INFO] GWT UI Compatibility ............................... SUCCESS [ 2.837 s] [INFO] Common Code ........................................ SUCCESS [ 16.813 s] [INFO] Common utilities ................................... SUCCESS [ 21.710 s] [INFO] Branding package ................................... SUCCESS [ 3.830 s] [INFO] ovirt-engine sso ................................... SUCCESS [ 2.750 s] [INFO] Data Access Layer .................................. SUCCESS [ 13.241 s] [INFO] engine scheduler bean .............................. SUCCESS [ 2.494 s] [INFO] Vds broker ......................................... SUCCESS [ 19.855 s] [INFO] builtin-extensions ................................. SUCCESS [ 1.471 s] [INFO] Search Backend ..................................... SUCCESS [ 5.694 s] [INFO] Backend Authentication, Authorization and Accounting SUCCESS [ 3.395 s] [INFO] Custom Application Server Authentication Plugin .... SUCCESS [ 1.529 s] [INFO] Backend Logic @Service bean ........................ SUCCESS [01:19 min] [INFO] oVirt RESTful API Backend Integration .............. SUCCESS [ 0.623 s] [INFO] oVirt RESTful API interface ........................ SUCCESS [ 0.563 s] [INFO] oVirt Engine API Definition ........................ SUCCESS [ 26.998 s] [INFO] oVirt Engine API Commom Parent POM ................. SUCCESS [ 0.632 s] [INFO] oVirt Engine API Common JAX-RS ..................... SUCCESS [ 4.128 s] [INFO] oVirt RESTful API Backend Integration Type Mappers . SUCCESS [ 11.256 s] [INFO] oVirt RESTful API Backend Integration JAX-RS Resources SUCCESS [ 58.203 s] [INFO] oVirt RESTful API Backend Integration Webapp ....... SUCCESS [ 0.719 s] [INFO] oVirt RESTful API Documentation .................... SUCCESS [ 0.960 s] [INFO] Custom Logger Using Extensions ..................... SUCCESS [ 1.394 s] [INFO] oVirt Engine Web Root .............................. SUCCESS [ 0.890 s] [INFO] ovirt-engine services .............................. SUCCESS [ 2.293 s] [INFO] ovirt-engine docs .................................. SUCCESS [ 1.543 s] [INFO] ovirt-engine welcome ............................... SUCCESS [ 3.594 s] [INFO] oVirt Engine Microbenchmarks ....................... SUCCESS [ 6.231 s] [INFO] oVirt Engine Tools ................................. SUCCESS [ 5.764 s] [INFO] oVirt Engine extensions tool ....................... SUCCESS [ 2.274 s] [INFO] oVirt Engine sso client registration tool .......... SUCCESS [ 1.398 s] [INFO] oVirt Modules :: Frontend .......................... SUCCESS [ 0.504 s] [INFO] oVirt Modules :: Brands ............................ SUCCESS [ 0.622 s] [INFO] oVirt Engine brand ................................. SUCCESS [ 1.265 s] [INFO] oVirt Modules :: Webadmin .......................... SUCCESS [ 0.539 s] [INFO] oVirt UI Modules ................................... SUCCESS [ 0.657 s] [INFO] AOP tweaks for GWT compiler ........................ SUCCESS [ 2.388 s] [INFO] Extensions for GWT ................................. SUCCESS [ 2.587 s] [INFO] UI Utils Compatibility (for UICommon) .............. SUCCESS [ 3.766 s] [INFO] Frontend for GWT UI Projects ....................... SUCCESS [ 14.865 s] [INFO] UICommonWeb ........................................ SUCCESS [ 30.380 s] [INFO] oVirt GWT UI common infrastructure ................. SUCCESS [ 21.623 s] [INFO] Frontend Assembly Descriptors ...................... SUCCESS [ 0.455 s] [INFO] WebAdmin ........................................... FAILURE [ 51.844 s] [INFO] oVirt Server EAR ................................... SKIPPED [INFO] ovirt-engine maven make ............................ SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 07:48 min [INFO] Finished at: 2018-03-22T12:59:06+05:30 [INFO] Final Memory: 640M/2353M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.codehaus.mojo:gwt-maven-plugin:2.8.0:compile (gwtcompile) on project webadmin: Command [[ [ERROR] /bin/sh -c '/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-5.b14.fc27.x86_64/jre/bin/java' '-javaagent:/home/godas/.m2/repository/org/aspectj/aspectjweaver/1.8.10/aspectjweaver-1.8.10.jar' '-Dgwt.jjs.permutationWorkerFactory=com.google.gwt.dev.ThreadedPermutationWorkerFactory' '-Dgwt.jjs.maxThreads=4' '-Djava.io.tmpdir=/home/godas/workspace/ovirt-engine/frontend/webadmin/modules/webadmin/target/tmp' '-Djava.util.prefs.systemRoot=/home/godas/workspace/ovirt-engine/frontend/webadmin/modules/webadmin/target/tmp' '-Djava.util.prefs.userRoot=/home/godas/workspace/ovirt-engine/frontend/webadmin/modules/webadmin/target/tmp' '-Djava.util.logging.config.class=org.ovirt.engine.ui.gwtaop.JavaLoggingConfig' '-Xms1024M' '-Xmx8192M' '-Dgwt.dontPrune=org\.ovirt\.engine\.core\.(common|compat)\..*' 'com.google.gwt.dev.Compiler' '-logLevel' 'INFO' '-war' '/home/godas/workspace/ovirt-engine/frontend/webadmin/modules/webadmin/target/generated-gwt' '-localWorkers' '4' '-failOnError' '-XfragmentCount' '-1' '-sourceLevel' 'auto' '-style' 'OBF' '-gen' '/home/godas/workspace/ovirt-engine/frontend/webadmin/modules/webadmin/gen' 'org.ovirt.engine.ui.webadmin.WebAdmin' [ERROR] ]] failed with status 1 [ERROR] -> [Help 1] On Wed, Mar 21, 2018 at 6:37 PM, Benny Zlotnik wrote: > Had this issue too (FF & Chrome), I've recompiled, ran engine-setup again > and it resolved > > On Wed, Mar 21, 2018 at 3:01 PM, Greg Sheremeta > wrote: > >> That message is probably not related, and is generally harmless. Compile >> with DEV_EXTRA_BUILD_FLAGS_GWT_DEFAULTS="-Dgwt.userAgent=safari,gecko1_8" >> to stop that message. >> >> Does it work in firefox? Please share your ui.log from the server. >> >> Greg >> >> On Wed, Mar 21, 2018 at 8:55 AM, Gobinda Das wrote: >> >>> Hi, >>> UI keeps on loading and I can see below error in browser. >>> >>> webadmin-0.js:422 Wed Mar 21 14:59:47 GMT+530 2018 >>> com.google.gwt.logging.client.LogConfiguration >>> SEVERE: Possible problem with your *.gwt.xml module file. >>> The compile time user.agent value (gecko1_8) does not match the runtime >>> user.agent value (safari). >>> Expect more errors. >>> com.google.gwt.useragent.client.UserAgentAsserter$UserAgentAssertionError: >>> Possible problem with your *.gwt.xml module file. >>> The compile time user.agent value (gecko1_8) does not match the runtime >>> user.agent value (safari). >>> Expect more errors. >>> at Unknown.L(webadmin-0.js) >>> at Unknown.oj(webadmin-0.js) >>> at Unknown.new pj(webadmin-0.js) >>> at Unknown.nj(webadmin-0.js) >>> at Unknown.nb(webadmin-0.js) >>> at Unknown.qb(webadmin-0.js) >>> at Unknown.eval(webadmin-0.js) >>> >>> -- >>> Thanks, >>> Gobinda >>> >>> _______________________________________________ >>> Devel mailing list >>> Devel at ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/devel >>> >> >> >> >> -- >> >> GREG SHEREMETA >> >> SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX >> >> Red Hat NA >> >> >> >> gshereme at redhat.com IRC: gshereme >> >> >> _______________________________________________ >> Devel mailing list >> Devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel >> > > -- Thanks, Gobinda -------------- next part -------------- An HTML attachment was scrubbed... URL: From sbonazzo at redhat.com Thu Mar 22 09:00:00 2018 From: sbonazzo at redhat.com (Sandro Bonazzola) Date: Thu, 22 Mar 2018 10:00:00 +0100 Subject: [ovirt-devel] [VDSM] Remove fcraw build as gating In-Reply-To: References: Message-ID: 2018-03-08 9:01 GMT+01:00 Eyal Edri : > > > On Thu, Mar 8, 2018 at 9:51 AM, Edward Haas wrote: > >> Hi All, >> >> Fcraw build on VDSM check-patch has been failing recently (again). >> >> fcraw cannot act as gating to the VDSM CI, as it is unstable (by >> definition). >> If there is a strong interest for some to track VDSM against this >> unstable distribution, it better be performed in parallel to the VDSM >> developing flow. >> >> With the current state, we are unable to trust the CI and it causes us to >> either overwrite its output (which I consider very bad) or just get stuck >> frequently. >> >> Could we please move this job to be triggered once a day as a nightly job >> and whoever is interested in its output to get notifications for it? >> Please. lets not make it a gating to VDSM development. >> >> > +1 from me, I didn't see any real value from fcraw so far, especially if > it doesn't save us adding new fedora versions when they are out. > We should try to stabalize fc28 when its out. > I want just to note that not having fcraw working in the past means we won't be ready for fc28 when it will be out. Not having fcraw now means we won't be ready for fc29 and so on for all the future fedora versions. This means we won't be able to support fedora in a reliable way. > > Also, CI team won't be adding nightly jobs for projects on demand on > production system, you're welcome to test it on your local env if needed. > > >> Thanks, >> Edy. >> >> _______________________________________________ >> Devel mailing list >> Devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel >> > > > > -- > > Eyal edri > > > MANAGER > > RHV DevOps > > EMEA VIRTUALIZATION R&D > > > Red Hat EMEA > TRIED. TESTED. TRUSTED. > phone: +972-9-7692018 <+972%209-769-2018> > irc: eedri (on #tlv #rhev-dev #rhev-integ) > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA sbonazzo at redhat.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From eedri at redhat.com Thu Mar 22 09:13:37 2018 From: eedri at redhat.com (Eyal Edri) Date: Thu, 22 Mar 2018 11:13:37 +0200 Subject: [ovirt-devel] [VDSM] Remove fcraw build as gating In-Reply-To: References: Message-ID: On Thu, Mar 22, 2018 at 11:00 AM, Sandro Bonazzola wrote: > > > 2018-03-08 9:01 GMT+01:00 Eyal Edri : > >> >> >> On Thu, Mar 8, 2018 at 9:51 AM, Edward Haas wrote: >> >>> Hi All, >>> >>> Fcraw build on VDSM check-patch has been failing recently (again). >>> >>> fcraw cannot act as gating to the VDSM CI, as it is unstable (by >>> definition). >>> If there is a strong interest for some to track VDSM against this >>> unstable distribution, it better be performed in parallel to the VDSM >>> developing flow. >>> >>> With the current state, we are unable to trust the CI and it causes us >>> to either overwrite its output (which I consider very bad) or just get >>> stuck frequently. >>> >>> Could we please move this job to be triggered once a day as a nightly >>> job and whoever is interested in its output to get notifications for it? >>> Please. lets not make it a gating to VDSM development. >>> >>> >> +1 from me, I didn't see any real value from fcraw so far, especially if >> it doesn't save us adding new fedora versions when they are out. >> We should try to stabalize fc28 when its out. >> > > I want just to note that not having fcraw working in the past means we > won't be ready for fc28 when it will be out. > Not having fcraw now means we won't be ready for fc29 and so on for all > the future fedora versions. > This means we won't be able to support fedora in a reliable way. > > Might be, but it doesn't seem to be a priority for most developers and only causing noise and blocking projects from being merged and published. We've supported fedora in the past in CI without running fcraw, and while it took more time to stablize, at least the builds worked in the end. We're planning on adding support for running OST on Fedora during oVirt 4.3 cycle, I'm sure it will help stabilize Fedora based released of oVirt. > > >> >> Also, CI team won't be adding nightly jobs for projects on demand on >> production system, you're welcome to test it on your local env if needed. >> >> >>> Thanks, >>> Edy. >>> >>> _______________________________________________ >>> Devel mailing list >>> Devel at ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/devel >>> >> >> >> >> -- >> >> Eyal edri >> >> >> MANAGER >> >> RHV DevOps >> >> EMEA VIRTUALIZATION R&D >> >> >> Red Hat EMEA >> TRIED. TESTED. TRUSTED. >> phone: +972-9-7692018 <+972%209-769-2018> >> irc: eedri (on #tlv #rhev-dev #rhev-integ) >> >> _______________________________________________ >> Devel mailing list >> Devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel >> > > > > -- > > SANDRO BONAZZOLA > > ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D > > Red Hat EMEA > > sbonazzo at redhat.com > > > -- Eyal edri MANAGER RHV DevOps EMEA VIRTUALIZATION R&D Red Hat EMEA TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ) -------------- next part -------------- An HTML attachment was scrubbed... URL: From gshereme at redhat.com Thu Mar 22 10:12:17 2018 From: gshereme at redhat.com (Greg Sheremeta) Date: Thu, 22 Mar 2018 06:12:17 -0400 Subject: [ovirt-devel] UI is not coming up after pulled latest code from master In-Reply-To: References: Message-ID: Gobinda, try this: https://github.com/guard/listen/wiki/Increasing-the-amount-of-inotify-watchers Mine: $ cat /etc/sysctl.conf # fs.inotify.max_user_watches=524288 I just verified that master is working fine for me. Greg On Thu, Mar 22, 2018 at 4:27 AM, Gobinda Das wrote: > Still no luck.I recompiled as did engine-setup but still same issue. I > tried in both (FF and Chrome) > I also tried to Compile with DEV_EXTRA_BUILD_FLAGS_GWT_ > DEFAULTS="-Dgwt.userAgent=safari,gecko1_8" but webadmin giving > compilation error like: > > [INFO] > [INFO] --- gwt-maven-plugin:2.8.0:compile (gwtcompile) @ webadmin --- > [INFO] info AspectJ Weaver Version 1.8.10 built on Monday Dec 12, 2016 at > 19:07:48 GMT > [INFO] info register classloader sun.misc.Launcher$AppClassLoader at 18b4aac2 > [INFO] info using configuration file:/home/godas/workspace/ > ovirt-engine/frontend/webadmin/modules/gwt-aop/ > target/gwt-aop-4.3.0-SNAPSHOT.jar!/META-INF/aop.xml > [INFO] info register aspect org.ovirt.engine.ui.gwtaop.DontPrune > [INFO] Mar 22, 2018 12:58:37 PM java.util.prefs.FileSystemPreferences$1 > run > [INFO] INFO: Created user preferences directory. > [ERROR] The attempt to retrieve files in /home/godas/workspace/ovirt- > engine/frontend/webadmin/modules/webadmin/target/ > webadmin-4.3.0-SNAPSHOT/WEB-INF/classes failed. > [ERROR] java.io.IOException: User limit of inotify watches reached > [ERROR] at sun.nio.fs.LinuxWatchService$Poller.implRegister( > LinuxWatchService.java:264) > [ERROR] at sun.nio.fs.AbstractPoller.processRequests( > AbstractPoller.java:260) > [ERROR] at sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService. > java:364) > [ERROR] at java.lang.Thread.run(Thread.java:748) > [ERROR] The attempt to retrieve files in /home/godas/workspace/ovirt- > engine/frontend/webadmin/modules/webadmin/src/main/java failed. > [ERROR] java.io.IOException: User limit of inotify watches reached > [ERROR] at sun.nio.fs.LinuxWatchService$Poller.implRegister( > LinuxWatchService.java:264) > [ERROR] at sun.nio.fs.AbstractPoller.processRequests( > AbstractPoller.java:260) > [ERROR] at sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService. > java:364) > [ERROR] at java.lang.Thread.run(Thread.java:748) > [ERROR] The attempt to retrieve files in /home/godas/workspace/ovirt- > engine/frontend/webadmin/modules/webadmin/target/generated-sources/annotations > failed. > [ERROR] java.io.IOException: User limit of inotify watches reached > [ERROR] at sun.nio.fs.LinuxWatchService$Poller.implRegister( > LinuxWatchService.java:264) > [ERROR] at sun.nio.fs.AbstractPoller.processRequests( > AbstractPoller.java:260) > [ERROR] at sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService. > java:364) > [ERROR] at java.lang.Thread.run(Thread.java:748) > [ERROR] The attempt to retrieve files in /home/godas/workspace/ovirt- > engine/frontend/webadmin/modules/webadmin/target/generated-sources/gwt > failed. > [ERROR] java.io.IOException: User limit of inotify watches reached > [ERROR] at sun.nio.fs.LinuxWatchService$Poller.implRegister( > LinuxWatchService.java:264) > [ERROR] at sun.nio.fs.AbstractPoller.processRequests( > AbstractPoller.java:260) > [ERROR] at sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService. > java:364) > [ERROR] at java.lang.Thread.run(Thread.java:748) > [ERROR] The attempt to retrieve files in /home/godas/workspace/ovirt- > engine/frontend/webadmin/modules/webadmin/target/ > webadmin-4.3.0-SNAPSHOT/WEB-INF/classes failed. > [ERROR] java.io.IOException: User limit of inotify watches reached > [ERROR] at sun.nio.fs.LinuxWatchService$Poller.implRegister( > LinuxWatchService.java:264) > [ERROR] at sun.nio.fs.AbstractPoller.processRequests( > AbstractPoller.java:260) > [ERROR] at sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService. > java:364) > [ERROR] at java.lang.Thread.run(Thread.java:748) > [ERROR] The attempt to retrieve files in /home/godas/workspace/ovirt- > engine/frontend/webadmin/modules/webadmin/src/main/java failed. > [ERROR] java.io.IOException: User limit of inotify watches reached > [ERROR] at sun.nio.fs.LinuxWatchService$Poller.implRegister( > LinuxWatchService.java:264) > [ERROR] at sun.nio.fs.AbstractPoller.processRequests( > AbstractPoller.java:260) > [ERROR] at sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService. > java:364) > [ERROR] at java.lang.Thread.run(Thread.java:748) > [ERROR] The attempt to retrieve files in /home/godas/workspace/ovirt- > engine/frontend/webadmin/modules/webadmin/target/generated-sources/annotations > failed. > [ERROR] java.io.IOException: User limit of inotify watches reached > [ERROR] at sun.nio.fs.LinuxWatchService$Poller.implRegister( > LinuxWatchService.java:264) > [ERROR] at sun.nio.fs.AbstractPoller.processRequests( > AbstractPoller.java:260) > [ERROR] at sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService. > java:364) > [ERROR] at java.lang.Thread.run(Thread.java:748) > [ERROR] The attempt to retrieve files in /home/godas/workspace/ovirt- > engine/frontend/webadmin/modules/webadmin/target/generated-sources/gwt > failed. > [ERROR] java.io.IOException: User limit of inotify watches reached > [ERROR] at sun.nio.fs.LinuxWatchService$Poller.implRegister( > LinuxWatchService.java:264) > [ERROR] at sun.nio.fs.AbstractPoller.processRequests( > AbstractPoller.java:260) > [ERROR] at sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService. > java:364) > [ERROR] at java.lang.Thread.run(Thread.java:748) > [ERROR] The attempt to retrieve files in /home/godas/workspace/ovirt- > engine/frontend/webadmin/modules/webadmin/target/ > webadmin-4.3.0-SNAPSHOT/WEB-INF/classes failed. > [ERROR] java.io.IOException: User limit of inotify watches reached > [ERROR] at sun.nio.fs.LinuxWatchService$Poller.implRegister( > LinuxWatchService.java:264) > [ERROR] at sun.nio.fs.AbstractPoller.processRequests( > AbstractPoller.java:260) > [ERROR] at sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService. > java:364) > [ERROR] at java.lang.Thread.run(Thread.java:748) > [ERROR] The attempt to retrieve files in /home/godas/workspace/ovirt- > engine/frontend/webadmin/modules/webadmin/src/main/java failed. > [ERROR] java.io.IOException: User limit of inotify watches reached > [ERROR] at sun.nio.fs.LinuxWatchService$Poller.implRegister( > LinuxWatchService.java:264) > [ERROR] at sun.nio.fs.AbstractPoller.processRequests( > AbstractPoller.java:260) > [ERROR] at sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService. > java:364) > [ERROR] at java.lang.Thread.run(Thread.java:748) > [ERROR] The attempt to retrieve files in /home/godas/workspace/ovirt- > engine/frontend/webadmin/modules/webadmin/target/generated-sources/annotations > failed. > [ERROR] java.io.IOException: User limit of inotify watches reached > [ERROR] at sun.nio.fs.LinuxWatchService$Poller.implRegister( > LinuxWatchService.java:264) > [ERROR] at sun.nio.fs.AbstractPoller.processRequests( > AbstractPoller.java:260) > [ERROR] at sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService. > java:364) > [ERROR] at java.lang.Thread.run(Thread.java:748) > [ERROR] The attempt to retrieve files in /home/godas/workspace/ovirt- > engine/frontend/webadmin/modules/webadmin/target/generated-sources/gwt > failed. > [ERROR] java.io.IOException: User limit of inotify watches reached > [ERROR] at sun.nio.fs.LinuxWatchService$Poller.implRegister( > LinuxWatchService.java:264) > [ERROR] at sun.nio.fs.AbstractPoller.processRequests( > AbstractPoller.java:260) > [ERROR] at sun.nio.fs.LinuxWatchService$Poller.run(LinuxWatchService. > java:364) > [ERROR] at java.lang.Thread.run(Thread.java:748) > [INFO] Compiling module org.ovirt.engine.ui.webadmin.WebAdmin > [INFO] warning javax.* types are not being woven because the weaver option > '-Xset:weaveJavaxPackages=true' has not been specified > [INFO] Computing all possible rebind results for > 'com.gwtplatform.mvp.client.ApplicationController' > [INFO] Rebinding com.gwtplatform.mvp.client.ApplicationController > [INFO] Invoking generator com.gwtplatform.mvp.rebind. > ApplicationControllerGenerator > [INFO] [ERROR] The type 'org.ovirt.engine.ui.webadmin.system.ApplicationInit' > was not found, either the class name is wrong or there are compile errors > in your code. > [INFO] [ERROR] The type 'org.ovirt.engine.ui.webadmin.system.ApplicationInit' > was not found, either the class name is wrong or there are compile errors > in your code. > [INFO] [ERROR] There was a problem generating the > ApplicationController, this can be caused by bad GWT module configuration > or compile errors in your source code. > [INFO] [WARN] For the following type(s), generated source was never > committed (did you forget to call commit()?) > [INFO] [WARN] com.gwtplatform.mvp.client.ApplicationControllerImpl > [INFO] [ERROR] Could not find com.gwtplatform.mvp.client.ApplicationControllerImpl > in types compiled from source. Is the source glob too strict? > [INFO] [ERROR] Errors in 'gen/com/google/gwt/lang/org_ > 00046ovirt_00046engine_00046ui_00046webadmin_00046WebAdmin__ > EntryMethodHolder.java' > [INFO] [ERROR] Line 3: Rebind result 'com.gwtplatform.mvp.client.ApplicationControllerImpl' > could not be found > [INFO] ------------------------------------------------------------ > ------------ > [INFO] Reactor Summary: > [INFO] > [INFO] oVirt Findbugs Filters ............................. SUCCESS [ > 0.914 s] > [INFO] ovirt-root ......................................... SUCCESS [ > 1.138 s] > [INFO] oVirt Build Tools root ............................. SUCCESS [ > 0.480 s] > [INFO] oVirt checkstyle ................................... SUCCESS [ > 1.161 s] > [INFO] oVirt Checkstyle Checks ............................ SUCCESS [ > 1.298 s] > [INFO] oVirt Modules - backend ............................ SUCCESS [ > 0.477 s] > [INFO] oVirt Manager ...................................... SUCCESS [ > 0.440 s] > [INFO] oVirt Engine dependencies .......................... SUCCESS [ > 0.406 s] > [INFO] oVirt Engine common dependencies ................... SUCCESS [ > 1.485 s] > [INFO] oVirt Engine tools dependencies .................... SUCCESS [ > 0.490 s] > [INFO] Utilities to extend java.logging framework ......... SUCCESS [ > 0.781 s] > [INFO] Extensions API root ................................ SUCCESS [ > 0.175 s] > [INFO] ovirt-engine-extensions-api ........................ SUCCESS [ > 4.015 s] > [INFO] oVirt Modules - manager ............................ SUCCESS [ > 1.279 s] > [INFO] Universal utilities ................................ SUCCESS [ > 13.829 s] > [INFO] Extensions manager ................................. SUCCESS [ > 2.554 s] > [INFO] GWT UI Compatibility ............................... SUCCESS [ > 2.837 s] > [INFO] Common Code ........................................ SUCCESS [ > 16.813 s] > [INFO] Common utilities ................................... SUCCESS [ > 21.710 s] > [INFO] Branding package ................................... SUCCESS [ > 3.830 s] > [INFO] ovirt-engine sso ................................... SUCCESS [ > 2.750 s] > [INFO] Data Access Layer .................................. SUCCESS [ > 13.241 s] > [INFO] engine scheduler bean .............................. SUCCESS [ > 2.494 s] > [INFO] Vds broker ......................................... SUCCESS [ > 19.855 s] > [INFO] builtin-extensions ................................. SUCCESS [ > 1.471 s] > [INFO] Search Backend ..................................... SUCCESS [ > 5.694 s] > [INFO] Backend Authentication, Authorization and Accounting SUCCESS [ > 3.395 s] > [INFO] Custom Application Server Authentication Plugin .... SUCCESS [ > 1.529 s] > [INFO] Backend Logic @Service bean ........................ SUCCESS [01:19 > min] > [INFO] oVirt RESTful API Backend Integration .............. SUCCESS [ > 0.623 s] > [INFO] oVirt RESTful API interface ........................ SUCCESS [ > 0.563 s] > [INFO] oVirt Engine API Definition ........................ SUCCESS [ > 26.998 s] > [INFO] oVirt Engine API Commom Parent POM ................. SUCCESS [ > 0.632 s] > [INFO] oVirt Engine API Common JAX-RS ..................... SUCCESS [ > 4.128 s] > [INFO] oVirt RESTful API Backend Integration Type Mappers . SUCCESS [ > 11.256 s] > [INFO] oVirt RESTful API Backend Integration JAX-RS Resources SUCCESS [ > 58.203 s] > [INFO] oVirt RESTful API Backend Integration Webapp ....... SUCCESS [ > 0.719 s] > [INFO] oVirt RESTful API Documentation .................... SUCCESS [ > 0.960 s] > [INFO] Custom Logger Using Extensions ..................... SUCCESS [ > 1.394 s] > [INFO] oVirt Engine Web Root .............................. SUCCESS [ > 0.890 s] > [INFO] ovirt-engine services .............................. SUCCESS [ > 2.293 s] > [INFO] ovirt-engine docs .................................. SUCCESS [ > 1.543 s] > [INFO] ovirt-engine welcome ............................... SUCCESS [ > 3.594 s] > [INFO] oVirt Engine Microbenchmarks ....................... SUCCESS [ > 6.231 s] > [INFO] oVirt Engine Tools ................................. SUCCESS [ > 5.764 s] > [INFO] oVirt Engine extensions tool ....................... SUCCESS [ > 2.274 s] > [INFO] oVirt Engine sso client registration tool .......... SUCCESS [ > 1.398 s] > [INFO] oVirt Modules :: Frontend .......................... SUCCESS [ > 0.504 s] > [INFO] oVirt Modules :: Brands ............................ SUCCESS [ > 0.622 s] > [INFO] oVirt Engine brand ................................. SUCCESS [ > 1.265 s] > [INFO] oVirt Modules :: Webadmin .......................... SUCCESS [ > 0.539 s] > [INFO] oVirt UI Modules ................................... SUCCESS [ > 0.657 s] > [INFO] AOP tweaks for GWT compiler ........................ SUCCESS [ > 2.388 s] > [INFO] Extensions for GWT ................................. SUCCESS [ > 2.587 s] > [INFO] UI Utils Compatibility (for UICommon) .............. SUCCESS [ > 3.766 s] > [INFO] Frontend for GWT UI Projects ....................... SUCCESS [ > 14.865 s] > [INFO] UICommonWeb ........................................ SUCCESS [ > 30.380 s] > [INFO] oVirt GWT UI common infrastructure ................. SUCCESS [ > 21.623 s] > [INFO] Frontend Assembly Descriptors ...................... SUCCESS [ > 0.455 s] > [INFO] WebAdmin ........................................... FAILURE [ > 51.844 s] > [INFO] oVirt Server EAR ................................... SKIPPED > [INFO] ovirt-engine maven make ............................ SKIPPED > [INFO] ------------------------------------------------------------ > ------------ > [INFO] BUILD FAILURE > [INFO] ------------------------------------------------------------ > ------------ > [INFO] Total time: 07:48 min > [INFO] Finished at: 2018-03-22T12:59:06+05:30 > [INFO] Final Memory: 640M/2353M > [INFO] ------------------------------------------------------------ > ------------ > [ERROR] Failed to execute goal org.codehaus.mojo:gwt-maven-plugin:2.8.0:compile > (gwtcompile) on project webadmin: Command [[ > [ERROR] /bin/sh -c '/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-5.b14.fc27.x86_64/jre/bin/java' > '-javaagent:/home/godas/.m2/repository/org/aspectj/aspectjweaver/1.8.10/aspectjweaver-1.8.10.jar' > '-Dgwt.jjs.permutationWorkerFactory=com.google.gwt.dev. > ThreadedPermutationWorkerFactory' '-Dgwt.jjs.maxThreads=4' > '-Djava.io.tmpdir=/home/godas/workspace/ovirt-engine/ > frontend/webadmin/modules/webadmin/target/tmp' > '-Djava.util.prefs.systemRoot=/home/godas/workspace/ovirt- > engine/frontend/webadmin/modules/webadmin/target/tmp' > '-Djava.util.prefs.userRoot=/home/godas/workspace/ovirt- > engine/frontend/webadmin/modules/webadmin/target/tmp' > '-Djava.util.logging.config.class=org.ovirt.engine.ui.gwtaop.JavaLoggingConfig' > '-Xms1024M' '-Xmx8192M' '-Dgwt.dontPrune=org\.ovirt\. > engine\.core\.(common|compat)\..*' 'com.google.gwt.dev.Compiler' > '-logLevel' 'INFO' '-war' '/home/godas/workspace/ovirt- > engine/frontend/webadmin/modules/webadmin/target/generated-gwt' > '-localWorkers' '4' '-failOnError' '-XfragmentCount' '-1' '-sourceLevel' > 'auto' '-style' 'OBF' '-gen' '/home/godas/workspace/ovirt- > engine/frontend/webadmin/modules/webadmin/gen' > 'org.ovirt.engine.ui.webadmin.WebAdmin' > [ERROR] ]] failed with status 1 > [ERROR] -> [Help 1] > > > > On Wed, Mar 21, 2018 at 6:37 PM, Benny Zlotnik > wrote: > >> Had this issue too (FF & Chrome), I've recompiled, ran engine-setup again >> and it resolved >> >> On Wed, Mar 21, 2018 at 3:01 PM, Greg Sheremeta >> wrote: >> >>> That message is probably not related, and is generally harmless. Compile >>> with DEV_EXTRA_BUILD_FLAGS_GWT_DEFAULTS="-Dgwt.userAgent=safari,gecko1_8" >>> to stop that message. >>> >>> Does it work in firefox? Please share your ui.log from the server. >>> >>> Greg >>> >>> On Wed, Mar 21, 2018 at 8:55 AM, Gobinda Das wrote: >>> >>>> Hi, >>>> UI keeps on loading and I can see below error in browser. >>>> >>>> webadmin-0.js:422 Wed Mar 21 14:59:47 GMT+530 2018 >>>> com.google.gwt.logging.client.LogConfiguration >>>> SEVERE: Possible problem with your *.gwt.xml module file. >>>> The compile time user.agent value (gecko1_8) does not match the runtime >>>> user.agent value (safari). >>>> Expect more errors. >>>> com.google.gwt.useragent.client.UserAgentAsserter$UserAgentAssertionError: >>>> Possible problem with your *.gwt.xml module file. >>>> The compile time user.agent value (gecko1_8) does not match the runtime >>>> user.agent value (safari). >>>> Expect more errors. >>>> at Unknown.L(webadmin-0.js) >>>> at Unknown.oj(webadmin-0.js) >>>> at Unknown.new pj(webadmin-0.js) >>>> at Unknown.nj(webadmin-0.js) >>>> at Unknown.nb(webadmin-0.js) >>>> at Unknown.qb(webadmin-0.js) >>>> at Unknown.eval(webadmin-0.js) >>>> >>>> -- >>>> Thanks, >>>> Gobinda >>>> >>>> _______________________________________________ >>>> Devel mailing list >>>> Devel at ovirt.org >>>> http://lists.ovirt.org/mailman/listinfo/devel >>>> >>> >>> >>> >>> -- >>> >>> GREG SHEREMETA >>> >>> SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX >>> >>> Red Hat NA >>> >>> >>> >>> gshereme at redhat.com IRC: gshereme >>> >>> >>> _______________________________________________ >>> Devel mailing list >>> Devel at ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/devel >>> >> >> > > > -- > Thanks, > Gobinda > -- GREG SHEREMETA SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX Red Hat NA gshereme at redhat.com IRC: gshereme -------------- next part -------------- An HTML attachment was scrubbed... URL: From michal.skrivanek at redhat.com Thu Mar 22 13:45:11 2018 From: michal.skrivanek at redhat.com (Michal Skrivanek) Date: Thu, 22 Mar 2018 14:45:11 +0100 Subject: [ovirt-devel] Fwd: Project for profiles and defaults for libvirt domains References: <20180320142031.GB23007@wheatley> Message-ID: <467B69E4-8A29-4EFF-B355-D4C95EB34D41@redhat.com> cross-posting to the right list, by mistake a wrong ovirt list was used originally > Begin forwarded message: > > From: Martin Kletzander > Subject: [kubevirt-dev] Project for profiles and defaults for libvirt domains > Date: 20 March 2018 at 15:20:31 CET > To: libvir-list at redhat.com, openstack-dev at lists.openstack.org, ovirt-devel at redhat.com, kubevirt-dev at googlegroups.com, virt-tools-list at redhat.com, cockpit-devel at lists.fedorahosted.org > > Hi everyone! > > First of all sorry for such wide distribution, but apparently that's the > best way to make sure we cooperate nicely. So please be considerate as > this is a cross-post between huge amount of mailing lists. > > After some discussions with developers from different projects that work > with libvirt one cannot but notice some common patterns and workarounds. > So I set off to see how can we make all our lives better and our coding > more effective (and maybe more fun as well). If all goes well we will > create a project that will accommodate most of the defaulting, policies, > workarounds and other common algorithms around libvirt domain > definitions. And since early design gets you half way, I would like to > know your feedback on several key points as well as on the general idea. > Also correct me brutally in case I'm wrong. > > In order to not get confused in the following descriptions, I will refer > to this project idea using the name `virtuned`, but there is really no > name for it yet (although an abbreviation for "Virtualization > Abstraction Definition and Hypervisor Delegation" would suit well, > IMHO). > > Here are some common problems and use cases that virtuned could solve > (or help with). Don't take it as something that's impossible to solve > on your own, but rather something that could be de-duplicated from > multiple projects or "done right" instead of various hack-ish solutions. > > 1) Default devices/values > > Libvirt itself must default to whatever values there were before any > particular element was introduced due to the fact that it strives to > keep the guest ABI stable. That means, for example, that it can't just > add -vmcoreinfo option (for KASLR support) or magically add the pvpanic > device to all QEMU machines, even though it would be useful, as that > would change the guest ABI. > > For default values this is even more obvious. Let's say someone figures > out some "pretty good" default values for various HyperV enlightenment > feature tunables. Libvirt can't magically change them, but each one of > the projects building on top of it doesn't want to keep that list > updated and take care of setting them in every new XML. Some projects > don't even expose those to the end user as a knob, while others might. > > One more thing could be automatically figuring out best values based on > libosinfo-provided data. > > 2) Policies > > Lot of the time there are parts of the domain definition that need to be > added, but nobody really cares about them. Sometimes it's enough to > have few templates, another time you might want to have a policy > per-scenario and want to combine them in various ways. For example with > the data provided by point 1). > > For example if you want PCI-Express, you need the q35 machine type, but > you don't really want to care about the machine type. Or you want to > use SPICE, but you don't want to care about adding QXL. > > What if some of these policies could be specified once (using some DSL > for example), and used by virtuned to merge them in a unified and > predictable way? > > 3) Abstracting the XML > > This is probably just usable for stateless apps, but it might happen > that some apps don't really want to care about the XML at all. They > just want an abstract view of the domain, possibly add/remove a device > and that's it. We could do that as well. I can't really tell how much > of a demand there is for it, though. > > 4) Identifying devices properly > > In contrast to the previous point, stateful apps might have a problem > identifying devices after hotplug. For example, let's say you don't > care about the addresses and leave that up to libvirt. You hotplug a > device into the domain and dump the new XML of it. Depending on what > type of device it was, you might need to identify it based on different > values. It could be for disks, for > interfaces etc. For some devices it might not even be possible and you > need to remember the addresses of all the previous devices and then > parse them just to identify that one device and then throw them away. > > With new enough libvirt you could use the user aliases for that, but > turns out it's not that easy to use them properly anyway. Also the > aliases won't help users identify that device inside the guest. > > > We really should've gone with new attribute for the user alias instead > of using an existing one, given how many problems that is causing. > > > 5) Generating the right XML snippet for device hot-(un)plug > > This is kind of related to some previous points. > > When hot-plugging a device and creating an XML snippet for it, you want > to keep the defaults from point 1) and policies from 2) in mind. Or > something related to the already existing domain which you can describe > systematically. And adding something for identification (see previous > point). > > Doing the hot-unplug is easy depending on how much information about > that device is saved by your application. The less you save about the > device (or show to the user in a GUI, if applicable) the harder it might > be to generate an XML that libvirt will accept. Again, some problems > with this should be fixed in libvirt, some of them are easy to > workaround. But having a common ground that takes care of this should > help some projects. > > Hot-unplug could be implemented just based on the alias. This is > something that would fit into libvirt as well. > > ======================================================================== > > To mention some pre-existing solutions: > > - I understand OpenStack has some really sensible and wisely chosen > and/or tested default values. > > - I know KubeVirt has VirtualMachinePresets. That is something closely > related to points 1) and 2). Also their abstraction of the XML might > be usable for point 3). > > - There was an effort on creating policy based configuration of libvirt > objects called libvirt-designer. This is closely related to points 2) > and 3). Unfortunately there was no much going on lately and part of > virt-manager repository has currently more features implemented with > the same ideas in mind, just not exported for public use. > > We could utilize some of the above to various extents. > > Let me know what you think and have a nice day. > Martin > > -- > You received this message because you are subscribed to the Google Groups "kubevirt-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an email to kubevirt-dev+unsubscribe at googlegroups.com. > To post to this group, send email to kubevirt-dev at googlegroups.com. > To view this discussion on the web visit https://groups.google.com/d/msgid/kubevirt-dev/20180320142031.GB23007%40wheatley. > For more options, visit https://groups.google.com/d/optout. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mkletzan at redhat.com Thu Mar 22 14:54:01 2018 From: mkletzan at redhat.com (Martin Kletzander) Date: Thu, 22 Mar 2018 15:54:01 +0100 Subject: [ovirt-devel] [virt-tools-list] Project for profiles and defaults for libvirt domains In-Reply-To: <20180320151012.GU4530@redhat.com> References: <20180320142031.GB23007@wheatley> <20180320151012.GU4530@redhat.com> Message-ID: <20180322145401.GD19999@wheatley> [ I fixed up ovirt-devel at redhat.com to be devel at ovirt.org since the former is deprecated. I'm also not trimming down much of the reply so that they can get the whole picture. Sorry for the confusion ] On Tue, Mar 20, 2018 at 03:10:12PM +0000, Daniel P. Berrang? wrote: >On Tue, Mar 20, 2018 at 03:20:31PM +0100, Martin Kletzander wrote: >> 1) Default devices/values >> >> Libvirt itself must default to whatever values there were before any >> particular element was introduced due to the fact that it strives to >> keep the guest ABI stable. That means, for example, that it can't just >> add -vmcoreinfo option (for KASLR support) or magically add the pvpanic >> device to all QEMU machines, even though it would be useful, as that >> would change the guest ABI. >> >> For default values this is even more obvious. Let's say someone figures >> out some "pretty good" default values for various HyperV enlightenment >> feature tunables. Libvirt can't magically change them, but each one of >> the projects building on top of it doesn't want to keep that list >> updated and take care of setting them in every new XML. Some projects >> don't even expose those to the end user as a knob, while others might. > >This gets very tricky, very fast. > >Lets say that you have an initial good set of hyperv config >tunables. Now sometime passes and it is decided that there is a >different, better set of config tunables. If the module that is >providing this policy to apps like OpenStack just updates itself >to provide this new policy, this can cause problems with the >existing deployed applications in a number of ways. > >First the new config probably depends on specific versions of >libvirt and QEMU, and you can't mandate to consuming apps which >versions they must be using. So you need a matrix of libvirt + >QEMU + config option settings. > >Even if you have the matching libvirt & QEMU versions, it is not >safe to assume the application will want to use the new policy. >An application may need live migration compatibility with older >versions. Or it may need to retain guaranteed ABI compatibility >with the way the VM was previously launched and be using transient >guests, generating the XML fresh each time. > >The application will have knowledge about when it wants to use new >vs old hyperv tunable policy, but exposing that to your policy module >is very tricky because it is inherantly application specific logic >largely determined by the way the application code is written. > The idea was for updating XML based on policy, which is something you want for new machines. You should then keep the XML per domain and only do changes to if requested by the user or when libvirt fills in new values in a guest ABI compatible fashion. > >> One more thing could be automatically figuring out best values based on >> libosinfo-provided data. >> >> 2) Policies >> >> Lot of the time there are parts of the domain definition that need to be >> added, but nobody really cares about them. Sometimes it's enough to >> have few templates, another time you might want to have a policy >> per-scenario and want to combine them in various ways. For example with >> the data provided by point 1). >> >> For example if you want PCI-Express, you need the q35 machine type, but >> you don't really want to care about the machine type. Or you want to >> use SPICE, but you don't want to care about adding QXL. >> >> What if some of these policies could be specified once (using some DSL >> for example), and used by virtuned to merge them in a unified and >> predictable way? >> >> 3) Abstracting the XML >> >> This is probably just usable for stateless apps, but it might happen >> that some apps don't really want to care about the XML at all. They >> just want an abstract view of the domain, possibly add/remove a device >> and that's it. We could do that as well. I can't really tell how much >> of a demand there is for it, though. > >It is safe to say that applications do not want to touch XML at all. >Any non-trivial application has created an abstraction around XML, >so that they have an API to express what they want, rather than >manipulating of strings to format/parse XML. > Sure, this was just meant to be a question as to whether it's worth pursuing or not. You make a good point on why it is not (at least for existing apps). However, since this was optional, the way this would look without the XML abstraction is that both input and output would be valid domain definitions, ultimately resulting in something similar to virt-xml with the added benefit of applying a policy from a file/string either supplied by the application itself. Whether that policy was taken from a common repository of such knowledge is orthogonal to this idea. Since you would work with the same data, the upgrade could be incremental as you'd only let virtuned fill in values for new options and could slowly move on to using it for some pre-existing ones. None of the previous approaches did this, if I'm not mistaken. Of course it gets more difficult when you need to expose all the bits libvirt does and keep them in sync (as you write below). [...] >If there was something higher level that gets more interesting, >but the hard bit is that you still need a way to get at all the >low level bits becuase a higher level abstracted API will never >cover every niche use case. > Oh, definitely not every, but I see two groups of projects that have a lot in common between themselves and between the groups as well. On the other hand just templating and defaults is something that's easy enough to do that it's not worth outsourcing that into another one's codebase. >> 4) Identifying devices properly >> >> In contrast to the previous point, stateful apps might have a problem >> identifying devices after hotplug. For example, let's say you don't >> care about the addresses and leave that up to libvirt. You hotplug a >> device into the domain and dump the new XML of it. Depending on what >> type of device it was, you might need to identify it based on different >> values. It could be for disks, for >> interfaces etc. For some devices it might not even be possible and you >> need to remember the addresses of all the previous devices and then >> parse them just to identify that one device and then throw them away. >> >> With new enough libvirt you could use the user aliases for that, but >> turns out it's not that easy to use them properly anyway. Also the >> aliases won't help users identify that device inside the guest. > >NB, relating between host device config and guest visible device >config is a massive problem space in its own right, and not very >easy to address. In OpenStack we ended up defining a concept of >"device tagging" via cloud-init metadata, where openstack allows >users to set opaque string tags against devices their VM has. >OpenStack that generates a metadata file that records various >pieces of identifying hardware attributes (PCI address, MAC >addr, disk serial, etc) alongside the user tag. This metadata >file is exposed to the guest with the hope that there's enough >info to allow the user to decide which device is to be used for >which purpose > This is good point, but I was mostly thinking about identifying devices from the host POV between two different XMLs (pre- and post- some XML-modifying action, like hotplug). >https://specs.openstack.org/openstack/nova-specs/specs/mitaka/approved/virt-device-role-tagging.html >https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/10/html/networking_guide/use-tagging > >> >> We really should've gone with new attribute for the user alias instead >> of using an existing one, given how many problems that is causing. >> >> >> 5) Generating the right XML snippet for device hot-(un)plug >> >> This is kind of related to some previous points. >> >> When hot-plugging a device and creating an XML snippet for it, you want >> to keep the defaults from point 1) and policies from 2) in mind. Or >> something related to the already existing domain which you can describe >> systematically. And adding something for identification (see previous >> point). >> >> Doing the hot-unplug is easy depending on how much information about >> that device is saved by your application. The less you save about the >> device (or show to the user in a GUI, if applicable) the harder it might >> be to generate an XML that libvirt will accept. Again, some problems >> with this should be fixed in libvirt, some of them are easy to >> workaround. But having a common ground that takes care of this should >> help some projects. >> >> Hot-unplug could be implemented just based on the alias. This is >> something that would fit into libvirt as well. >> >> ======================================================================== >> >> To mention some pre-existing solutions: >> >> - I understand OpenStack has some really sensible and wisely chosen >> and/or tested default values. > >In terms of default devices and OS specific choices, OpenStack's >decisions have been largely inspired by previous work in oVirt >and / or virt-manager. So there's obviously overlap in the >conceptual area, but there's also plenty that is very specific >to OpenStack - untangling the two extract the common bits from >the app specific bits is hard. > It definitely is, but do you think it's so difficult it's worthless to pursuit? I did a tiny PoC based on the code from virt-manager, which was trivial mainly thanks to the XMLBuilder for the domain objects. Maybe exposing an easy way to work with the XML would be enough for some projects. Little birdie from oVirt told me that they would like some of sort of thing that does what you can achieve with virt-xml if we, for example, made it work on pure XML definitions without connecting to libvirt. >> - I know KubeVirt has VirtualMachinePresets. That is something closely >> related to points 1) and 2). Also their abstraction of the XML might >> be usable for point 3). >> >> - There was an effort on creating policy based configuration of libvirt >> objects called libvirt-designer. This is closely related to points 2) >> and 3). Unfortunately there was no much going on lately and part of >> virt-manager repository has currently more features implemented with >> the same ideas in mind, just not exported for public use. > >This is the same kind of problem we faced wrt libvirt-gconfig and >libvirt-gobject usage from virt-manager - it has an extensive code >base that already works, and rewriting it to use something new >is alot of work for no short-term benefit. libvirt-gconfig/gobject >were supposed to be the "easy" bits for virt-manager to adopt, as >they don't really include much logic that would step on virt-manager's >toes. libvirt-designer was going to be a very opinionated library >and in retrospective that makes it even harder to consider adopting >it for usage in virt-manager, as it'll have signficant liklihood >of making functionally significant changes in behaviour. > The initial idea (which I forgot to mention) was that all the decisions libvirt currently does (so that it keeps the guest ABI stable) would be moved into data (let's say some DSL) and it could then be switched or adjusted if that's not what the mgmt app wants (on a per-definition basis, of course). I didn't feel very optimistic about the upstream acceptance for that idea, so I figured that there could be something that lives beside libvirt, helps with some policies if requested and then the resulting XML could be fed into libvirt for determining the rest. >There's also the problem with use of native libraries that would >impact many apps. We only got OpenStack to grudgingly allow the By native you mean actual binary libraries or native to the OpenStack code as in python module? Because what I had in mind for this project was a python module with optional wrapper for REST API. >use of libosinfo native library via GObject Introspection, by >promising to do work to turn the osinfo database into an approved >stable format which OpenStack could then consume directly, dropping >the native API usage :-( Incidentally, the former was done (formal >spec for the DB format), but the latter was not yet (direct DB usage >by OpenStack) > > >BTW, I don't like that I'm being so negative to your proposal :-( >I used to hope that we would be able to build higher level APIs on >top of libvirt to reduce the overlap between different applications >reinventing the wheel. Even the simplest bits we tried like the >gconfig/gobject API are barely used. libvirt-designer is basically >a failure. Though admittedly it didn't have enough development resource >applied to make it compelling, in retrospect adoption was always going >to be a hard sell except in greenfield developments. > I'm glad for the knowledge you provided. So maybe instead of focusing on de-duplication of existing codebases we could _at least_ aim at future mgmt apps. OTOH improving documentation on how to properly build higher level concepts on top of libvirt would benefit them as well. >Libosinfo is probably the bit we've had most success with, and has >most promise for the future, particularly now that we formally allow >apps to read the osinfo database directly and bypass the API. It is >quite easy to fit into existing application codebases which helps alot. >Even there I'm still disappointed that we only have GNOME Boxes using >the kickstart generator part of osinfo - oVirt and Oz both still have >their own kickstart generator code for automating OS installs. > >In general though, I fear anything API based is going to be a really >hard sell to get wide adoption for based on what we've seen before. > >I think the biggest bang-for-buck is identifying more areas where we >can turn code into data. There's definitely scope for recording more >types of information in the osinfo database. There might also be >scope for defining entirely new databases to complement the osinfo >data, if something looks out of scope for libosinfo. > >Regards, >Daniel >-- >|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| >|: https://libvirt.org -o- https://fstop138.berrange.com :| >|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Digital signature URL: From berrange at redhat.com Thu Mar 22 17:18:22 2018 From: berrange at redhat.com (Daniel =?utf-8?B?UC4gQmVycmFuZ8Op?=) Date: Thu, 22 Mar 2018 17:18:22 +0000 Subject: [ovirt-devel] [kubevirt-dev] Re: [virt-tools-list] Project for profiles and defaults for libvirt domains In-Reply-To: <20180322145401.GD19999@wheatley> References: <20180320142031.GB23007@wheatley> <20180320151012.GU4530@redhat.com> <20180322145401.GD19999@wheatley> Message-ID: <20180322171753.GU3583@redhat.com> On Thu, Mar 22, 2018 at 03:54:01PM +0100, Martin Kletzander wrote: > > > > > One more thing could be automatically figuring out best values based on > > > libosinfo-provided data. > > > > > > 2) Policies > > > > > > Lot of the time there are parts of the domain definition that need to be > > > added, but nobody really cares about them. Sometimes it's enough to > > > have few templates, another time you might want to have a policy > > > per-scenario and want to combine them in various ways. For example with > > > the data provided by point 1). > > > > > > For example if you want PCI-Express, you need the q35 machine type, but > > > you don't really want to care about the machine type. Or you want to > > > use SPICE, but you don't want to care about adding QXL. > > > > > > What if some of these policies could be specified once (using some DSL > > > for example), and used by virtuned to merge them in a unified and > > > predictable way? > > > > > > 3) Abstracting the XML > > > > > > This is probably just usable for stateless apps, but it might happen > > > that some apps don't really want to care about the XML at all. They > > > just want an abstract view of the domain, possibly add/remove a device > > > and that's it. We could do that as well. I can't really tell how much > > > of a demand there is for it, though. > > > > It is safe to say that applications do not want to touch XML at all. > > Any non-trivial application has created an abstraction around XML, > > so that they have an API to express what they want, rather than > > manipulating of strings to format/parse XML. > > > > Sure, this was just meant to be a question as to whether it's worth > pursuing or not. You make a good point on why it is not (at least for > existing apps). > > However, since this was optional, the way this would look without the > XML abstraction is that both input and output would be valid domain > definitions, ultimately resulting in something similar to virt-xml with > the added benefit of applying a policy from a file/string either > supplied by the application itself. Whether that policy was taken from > a common repository of such knowledge is orthogonal to this idea. Since > you would work with the same data, the upgrade could be incremental as > you'd only let virtuned fill in values for new options and could slowly > move on to using it for some pre-existing ones. None of the previous > approaches did this, if I'm not mistaken. Of course it gets more > difficult when you need to expose all the bits libvirt does and keep > them in sync (as you write below). That has implications for how mgmt app deals with XML. Nova has object models for representing XML in memory, but it doesn't aim to have loss-less roundtrip from parse -> object -> format. So if Nova gets basic XML from virttuned, parses it into its object to let it set more fields and then formats it again, chances are it will have lost a bunch of stuff from virttuned. Of course if you know about this need upfront you can design the application such that it can safely round-trip, but this is just example of problem with integrating to existing apps. The other thing that concerns is that there are dependancies between different bits of XML for a given device. ie if feature X is set to a certain value, that prevents use of feature Y. So if virttuned sets feature X, but the downstream application uses feature Y, the final result can be incompatible. The application won't know this because it doesn't control what stuff virttuned would be setting. This can in turn cause ordering constraints. eg the application needs to say that virtio-net is being used, then virttuned can set some defaults like enabling vhost-net, and then the application can fill in more bits that it cares about. Or if we let virttuned go first, setting virtio-net model + vhost-net, then application wants to change model to e1000e, it has to be aware that it must now delete the vhost-net bit that virtuned added. This ends up being more complicated that just ignoring virttuned and coding up use of vhost-net in application code. > > This is the same kind of problem we faced wrt libvirt-gconfig and > > libvirt-gobject usage from virt-manager - it has an extensive code > > base that already works, and rewriting it to use something new > > is alot of work for no short-term benefit. libvirt-gconfig/gobject > > were supposed to be the "easy" bits for virt-manager to adopt, as > > they don't really include much logic that would step on virt-manager's > > toes. libvirt-designer was going to be a very opinionated library > > and in retrospective that makes it even harder to consider adopting > > it for usage in virt-manager, as it'll have signficant liklihood > > of making functionally significant changes in behaviour. > > > > The initial idea (which I forgot to mention) was that all the decisions > libvirt currently does (so that it keeps the guest ABI stable) would be > moved into data (let's say some DSL) and it could then be switched or > adjusted if that's not what the mgmt app wants (on a per-definition > basis, of course). I didn't feel very optimistic about the upstream > acceptance for that idea, so I figured that there could be something > that lives beside libvirt, helps with some policies if requested and > then the resulting XML could be fed into libvirt for determining the > rest. I can't even imagine how we would go about encoding the stable guest ABI logic libvirt does today in data ! > > > There's also the problem with use of native libraries that would > > impact many apps. We only got OpenStack to grudgingly allow the > > By native you mean actual binary libraries or native to the OpenStack > code as in python module? Because what I had in mind for this project > was a python module with optional wrapper for REST API. I meant native binary libraries. ie openstack is not happy in general with adding dependancies on new OS services, because there's a big time lag for getting them into all distros. By comparison a pure python library, they can just handle automatically in their deployment tools, just pip installing on any OS distro straight from pypi. This is what made use of libosinfo a hard sell in Nova. The same thing is seen with Go / Rust where some applications have decided they're better of actually re-implementing the libvirt RPC protocol in Go / Rust rather than use the libvirt.so client. I think this is a bad tradeoff in general, but I can see why they like it Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| From dron at redhat.com Fri Mar 23 12:02:03 2018 From: dron at redhat.com (Dafna Ron) Date: Fri, 23 Mar 2018 12:02:03 +0000 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (ovirt-engine-nodejs-modules) ] [ 23-03-2018] [ 002_bootstrap.check_update_host ] Message-ID: Hi, we had a failure reported in CQ for change: https://gerrit.ovirt.org/#/c/89338/ - Bump 1.5.2-1. I don't think the failure is related to the change. it seems to be an issue with mom failing to start. *Link and headline of suspected patches: https://gerrit.ovirt.org/#/c/89338/ - Bump 1.5.2-1Link to Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6512/ Link to all logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6512/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-002_bootstrap.py/ (Relevant) error snippet from the log: * *engine: *018-03-23 07:12:33,435-04 ERROR [org.ovirt.engine.core.bll.host.HostUpgradeManager] (EE-ManagedThreadFactory-commandCoordinator-Thread-1) [7a4f81e6-6ae6-4444-ace6-de757e78f41e] Failed to run check-update of host 'lago-basic-suite-master- host-0'. 2018-03-23 07:12:33,436-04 ERROR [org.ovirt.engine.core.bll.hostdeploy.HostUpdatesChecker] (EE-ManagedThreadFactory-commandCoordinator-Thread-1) [7a4f81e6-6ae6-4444-ace6-de757e78f41e] Failed to check if updates are available for host 'lag o-basic-suite-master-host-0' with error message 'Failed to run check-update of host 'lago-basic-suite-master-host-0'.' 2018-03-23 07:12:33,441-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-commandCoordinator-Thread-1) [7a4f81e6-6ae6-4444-ace6-de757e78f41e] EVENT_ID: HOST_AVAILABLE_UPDATES_FAILED(8 39), Failed to check for available updates on host lago-basic-suite-master-host-0 with message 'Failed to run check-update of host 'lago-basic-suite-master-host-0'.'. 2018-03-23 07:12:33,566-04 DEBUG [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-11) [] START, GetAllVmStatsVDSCommand(HostName = lago-basic-suite-master-host-0, VdsIdVDS CommandParametersBase:{hostId='dc339bcf-b769-41f6-8252-c61fffd56b17'}), log id: 4eb92a5a 2018-03-23 07:12:33,567-04 DEBUG [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.impl.Message] (EE-ManagedThreadFactory-engineScheduled-Thread-11) [] SEND destination:jms.topic.vdsm_requests reply-to:jms.topic.vdsm_responses content-length:103 *host-0:* vdsm log show: momStatus': 'inactive' *Mom log: * 2018-03-23 07:09:48,849 - mom - INFO - MOM starting 2018-03-23 07:09:48,864 - mom.HostMonitor - INFO - Host Monitor starting 2018-03-23 07:09:48,867 - mom - INFO - hypervisor interface vdsmjsonrpcclient 2018-03-23 07:09:48,983 - mom - ERROR - Failed to initialize MOM threads Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/mom/__init__.py", line 29, in run hypervisor_iface = self.get_hypervisor_interface() File "/usr/lib/python2.7/site-packages/mom/__init__.py", line 217, in get_hypervisor_interface return module.instance(self.config) File "/usr/lib/python2.7/site-packages/mom/HypervisorInterfaces/vdsmjsonrpcclientInterface.py", line 96, in instance return JsonRpcVdsmClientInterface() File "/usr/lib/python2.7/site-packages/mom/HypervisorInterfaces/vdsmjsonrpcclientInterface.py", line 31, in __init__ self._vdsm_api = client.connect(host="localhost") File "/usr/lib/python2.7/site-packages/vdsm/client.py", line 139, in connect raise ConnectionError(host, port, use_tls, timeout, e) ConnectionError: Connection to localhost:54321 with use_tls=True, timeout=60 failed: [Errno 111] Connection refused 2018-03-23 07:09:54,475 - mom - INFO - MOM starting 2018-03-23 07:09:54,499 - mom.HostMonitor - INFO - Host Monitor starting 2018-03-23 07:09:54,500 - mom - INFO - hypervisor interface vdsmjsonrpcclient 2018-03-23 07:09:54,725 - mom.HostMonitor - INFO - HostMonitor is ready 2018-03-23 07:09:54,813 - mom.GuestManager - INFO - Guest Manager starting: multi-thread 2018-03-23 07:09:54,819 - mom.Policy - INFO - Loaded policy '00-defines' 2018-03-23 07:09:54,820 - mom.Policy - INFO - Loaded policy '01-parameters' 2018-03-23 07:09:54,841 - mom.Policy - INFO - Loaded policy '02-balloon' 2018-03-23 07:09:54,871 - mom.Policy - INFO - Loaded policy '03-ksm' 2018-03-23 07:09:54,909 - mom.Policy - INFO - Loaded policy '04-cputune' 2018-03-23 07:09:54,951 - mom.Policy - INFO - Loaded policy '05-iotune' 2018-03-23 07:09:54,956 - mom.PolicyEngine - INFO - Policy Engine starting 2018-03-23 07:09:54,957 - mom.RPCServer - INFO - Using unix socket /var/run/vdsm/mom-vdsm.sock 2018-03-23 07:09:54,957 - mom.RPCServer - INFO - RPC Server starting 2018-03-23 07:10:10,020 - mom.Controllers.KSM - INFO - Updating KSM configuration: pages_to_scan:0 merge_across_nodes:1 run:0 sleep_millisecs:0 lago-basic-suite-master-host-0/_var_log/vdsm/mom.log (END) ** -------------- next part -------------- An HTML attachment was scrubbed... URL: From nsoffer at redhat.com Fri Mar 23 15:04:32 2018 From: nsoffer at redhat.com (Nir Soffer) Date: Fri, 23 Mar 2018 15:04:32 +0000 Subject: [ovirt-devel] oVirt on the main page of redhatofficial.github.io Message-ID: https://redhatofficial.github.io/#!/main [image: Screenshot from 2018-03-23 18-01-07.png] Thanks for the great work Brian! Nir -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Screenshot from 2018-03-23 18-01-07.png Type: image/png Size: 125521 bytes Desc: not available URL: From dron at redhat.com Fri Mar 23 16:14:31 2018 From: dron at redhat.com (Dafna Ron) Date: Fri, 23 Mar 2018 16:14:31 +0000 Subject: [ovirt-devel] OST Failure - Weekly update [17/03/2018-23/03/2018] Message-ID: Hello, I would like to update on this week's failures and OST current status. On 19-03-2018 - the CI team reported 3 different failures. On Master branch the failed changes reported were: *core: fix removal of vm-host device - https://gerrit.ovirt.org/#/c/89145/ * *core: USB in osinfo configuration depends on chipset - https://gerrit.ovirt.org/#/c/88777/ * *On 4.2 *branch, the reported change was: *core: Call endAction() of all child commands in ImportVmCommand - https://gerrit.ovirt.org/#/c/89165/ * The fix's for the regressions were merged the following day (20-03-2018) https://gerrit.ovirt.org/#/c/89250/- core: Replace generic unlockVm() logic in ImportVmCommand https://gerrit.ovirt.org/#/c/89187/ - core: Fix NPE when creating an instance type On 20-03-2018 - the CI team discovered an issue on the job's cleanup which caused random failures on changes testing due to failure in docker cleanup. There is an open Jira on the issue: https://ovirt-jira.atlassian.net/browse/OVIRT-1939 *Below you can see the chart for this week's resolved issues but cause of failure:*Code = regression of working components/functionalities Configurations = package related issues Other = failed build artifacts Infra = infrastructure/OST/Lago related issues *Below is a chart of resolved failures based on ovirt version* *Below is a chart showing failures by suite type: * Thank you, Dafna -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 23605 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 7163 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 26288 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 6237 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 5835 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 27001 bytes Desc: not available URL: From gshereme at redhat.com Sat Mar 24 17:10:03 2018 From: gshereme at redhat.com (Greg Sheremeta) Date: Sat, 24 Mar 2018 13:10:03 -0400 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (ovirt-engine-nodejs-modules) ] [ 23-03-2018] [ 002_bootstrap.check_update_host ] In-Reply-To: References: Message-ID: On Fri, Mar 23, 2018 at 8:02 AM, Dafna Ron wrote: > Hi, > > we had a failure reported in CQ for change: https://gerrit.ovirt.org/#/c/8 > 9338/ - Bump 1.5.2-1. > > I don't think the failure is related to the change. > Definitely not related > it seems to be an issue with mom failing to start. > +Martin > > > > > > > > > > *Link and headline of suspected patches: > https://gerrit.ovirt.org/#/c/89338/ - > Bump 1.5.2-1Link to > Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6512/ > Link > to all > logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6512/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-002_bootstrap.py/ > (Relevant) > error snippet from the log: * > > > *engine: *018-03-23 07:12:33,435-04 ERROR [org.ovirt.engine.core.bll.host.HostUpgradeManager] > (EE-ManagedThreadFactory-commandCoordinator-Thread-1) > [7a4f81e6-6ae6-4444-ace6-de757e78f41e] Failed to run check-update of host > 'lago-basic-suite-master- > host-0'. > 2018-03-23 07:12:33,436-04 ERROR [org.ovirt.engine.core.bll.hostdeploy.HostUpdatesChecker] > (EE-ManagedThreadFactory-commandCoordinator-Thread-1) > [7a4f81e6-6ae6-4444-ace6-de757e78f41e] Failed to check if updates are > available for host 'lag > o-basic-suite-master-host-0' with error message 'Failed to run > check-update of host 'lago-basic-suite-master-host-0'.' > 2018-03-23 07:12:33,441-04 ERROR [org.ovirt.engine.core.dal.dbb > roker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-commandCoordinator-Thread-1) > [7a4f81e6-6ae6-4444-ace6-de757e78f41e] EVENT_ID: > HOST_AVAILABLE_UPDATES_FAILED(8 > 39), Failed to check for available updates on host > lago-basic-suite-master-host-0 with message 'Failed to run check-update of > host 'lago-basic-suite-master-host-0'.'. > 2018-03-23 07:12:33,566-04 DEBUG [org.ovirt.engine.core.vdsbrok > er.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-11) > [] START, GetAllVmStatsVDSCommand(HostName = > lago-basic-suite-master-host-0, VdsIdVDS > CommandParametersBase:{hostId='dc339bcf-b769-41f6-8252-c61fffd56b17'}), > log id: 4eb92a5a > 2018-03-23 07:12:33,567-04 DEBUG [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.impl.Message] > (EE-ManagedThreadFactory-engineScheduled-Thread-11) [] SEND > destination:jms.topic.vdsm_requests > reply-to:jms.topic.vdsm_responses > content-length:103 > > > > *host-0:* > vdsm log show: momStatus': 'inactive' > > > *Mom log: * > 2018-03-23 07:09:48,849 - mom - INFO - MOM starting > 2018-03-23 07:09:48,864 - mom.HostMonitor - INFO - Host Monitor starting > 2018-03-23 07:09:48,867 - mom - INFO - hypervisor interface > vdsmjsonrpcclient > 2018-03-23 07:09:48,983 - mom - ERROR - Failed to initialize MOM threads > Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/mom/__init__.py", line 29, in run > hypervisor_iface = self.get_hypervisor_interface() > File "/usr/lib/python2.7/site-packages/mom/__init__.py", line 217, in > get_hypervisor_interface > return module.instance(self.config) > File "/usr/lib/python2.7/site-packages/mom/HypervisorInterfaces/vdsmjsonrpcclientInterface.py", > line 96, in instance > return JsonRpcVdsmClientInterface() > File "/usr/lib/python2.7/site-packages/mom/HypervisorInterfaces/vdsmjsonrpcclientInterface.py", > line 31, in __init__ > self._vdsm_api = client.connect(host="localhost") > File "/usr/lib/python2.7/site-packages/vdsm/client.py", line 139, in > connect > raise ConnectionError(host, port, use_tls, timeout, e) > ConnectionError: Connection to localhost:54321 with use_tls=True, > timeout=60 failed: [Errno 111] Connection refused > 2018-03-23 07:09:54,475 - mom - INFO - MOM starting > 2018-03-23 07:09:54,499 - mom.HostMonitor - INFO - Host Monitor starting > 2018-03-23 07:09:54,500 - mom - INFO - hypervisor interface > vdsmjsonrpcclient > 2018-03-23 07:09:54,725 - mom.HostMonitor - INFO - HostMonitor is ready > 2018-03-23 07:09:54,813 - mom.GuestManager - INFO - Guest Manager > starting: multi-thread > 2018-03-23 07:09:54,819 - mom.Policy - INFO - Loaded policy '00-defines' > 2018-03-23 07:09:54,820 - mom.Policy - INFO - Loaded policy '01-parameters' > 2018-03-23 07:09:54,841 - mom.Policy - INFO - Loaded policy '02-balloon' > 2018-03-23 07:09:54,871 - mom.Policy - INFO - Loaded policy '03-ksm' > 2018-03-23 07:09:54,909 - mom.Policy - INFO - Loaded policy '04-cputune' > 2018-03-23 07:09:54,951 - mom.Policy - INFO - Loaded policy '05-iotune' > 2018-03-23 07:09:54,956 - mom.PolicyEngine - INFO - Policy Engine starting > 2018-03-23 07:09:54,957 - mom.RPCServer - INFO - Using unix socket > /var/run/vdsm/mom-vdsm.sock > 2018-03-23 07:09:54,957 - mom.RPCServer - INFO - RPC Server starting > 2018-03-23 07:10:10,020 - mom.Controllers.KSM - INFO - Updating KSM > configuration: pages_to_scan:0 merge_across_nodes:1 run:0 sleep_millisecs:0 > lago-basic-suite-master-host-0/_var_log/vdsm/mom.log (END) > > ** > > _______________________________________________ > Infra mailing list > Infra at ovirt.org > http://lists.ovirt.org/mailman/listinfo/infra > > -- GREG SHEREMETA SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX Red Hat NA gshereme at redhat.com IRC: gshereme -------------- next part -------------- An HTML attachment was scrubbed... URL: From gshereme at redhat.com Sat Mar 24 17:19:51 2018 From: gshereme at redhat.com (Greg Sheremeta) Date: Sat, 24 Mar 2018 13:19:51 -0400 Subject: [ovirt-devel] OST Failure - Weekly update [17/03/2018-23/03/2018] In-Reply-To: References: Message-ID: Hi, Is there an ongoing engine master OST failure blocking? [ INFO ] Stage: Misc configuration [ INFO ] Stage: Package installation [ INFO ] Stage: Misc configuration [ ERROR ] Failed to execute stage \'Misc configuration\': Failed to start service \'openvswitch\' [ INFO ] Yum Performing yum transaction rollback These are unrelated code changes: http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x86_64/4644/ https://gerrit.ovirt.org/#/c/89347/ and http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x86_64/4647/ https://gerrit.ovirt.org/67166 But they both die in 001, with exactly 1.24MB in the log and 'Failed to start service openvswitch' 001_initialize_engine.py.junit.xml 1.24 MB Full file: http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x86_64/4644/artifact/exported-artifacts/basic-suite-master__logs/001_initialize_engine.py.junit.xml On Fri, Mar 23, 2018 at 12:14 PM, Dafna Ron wrote: > Hello, > > I would like to update on this week's failures and OST current status. > > On 19-03-2018 - the CI team reported 3 different failures. > > On Master branch the failed changes reported were: > > > *core: fix removal of vm-host device - https://gerrit.ovirt.org/#/c/89145/ > * > > *core: USB in osinfo configuration depends on chipset - > https://gerrit.ovirt.org/#/c/88777/ * > *On 4.2 *branch, the reported change was: > > > > *core: Call endAction() of all child commands in ImportVmCommand - > https://gerrit.ovirt.org/#/c/89165/ * > The fix's for the regressions were merged the following day (20-03-2018) > > https://gerrit.ovirt.org/#/c/89250/- core: Replace generic unlockVm() > logic in ImportVmCommand > https://gerrit.ovirt.org/#/c/89187/ - core: Fix NPE when creating an > instance type > > On 20-03-2018 - the CI team discovered an issue on the job's cleanup which > caused random failures on changes testing due to failure in docker cleanup. > There is an open Jira on the issue: https://ovirt-jira.atlassian. > net/browse/OVIRT-1939 > > > > *Below you can see the chart for this week's resolved issues but cause of > failure:*Code = regression of working components/functionalities > Configurations = package related issues > Other = failed build artifacts > Infra = infrastructure/OST/Lago related issues > > > > > > > > > > > *Below is a chart of resolved failures based on ovirt version* > > > > > > > > > > > > > *Below is a chart showing failures by suite type: * > Thank you, > Dafna > > > _______________________________________________ > Infra mailing list > Infra at ovirt.org > http://lists.ovirt.org/mailman/listinfo/infra > > -- GREG SHEREMETA SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX Red Hat NA gshereme at redhat.com IRC: gshereme -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 7163 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 27001 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 5835 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 26288 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 23605 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 6237 bytes Desc: not available URL: From gshereme at redhat.com Sat Mar 24 18:22:38 2018 From: gshereme at redhat.com (Greg Sheremeta) Date: Sat, 24 Mar 2018 14:22:38 -0400 Subject: [ovirt-devel] OST Failure - Weekly update [17/03/2018-23/03/2018] In-Reply-To: References: Message-ID: performance suite fails with the same openvswitch error. http://jenkins.ovirt.org/job/ovirt-system-tests_ performance-suite-master/152/ ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 001_initialize_engine.initialize_engine On Sat, Mar 24, 2018 at 1:19 PM, Greg Sheremeta wrote: > Hi, > > Is there an ongoing engine master OST failure blocking? > > [ INFO ] Stage: Misc configuration > [ INFO ] Stage: Package installation > [ INFO ] Stage: Misc configuration > [ ERROR ] Failed to execute stage \'Misc configuration\': Failed to start > service \'openvswitch\' > [ INFO ] Yum Performing yum transaction rollback > > > These are unrelated code changes: > > http://jenkins.ovirt.org/job/ovirt-system-tests_master_ > check-patch-el7-x86_64/4644/ > https://gerrit.ovirt.org/#/c/89347/ > > and > http://jenkins.ovirt.org/job/ovirt-system-tests_master_ > check-patch-el7-x86_64/4647/ > https://gerrit.ovirt.org/67166 > > But they both die in 001, with exactly 1.24MB in the log and 'Failed to > start service openvswitch' > 001_initialize_engine.py.junit.xml 1.24 MB > > Full file: http://jenkins.ovirt.org/job/ovirt-system-tests_ > master_check-patch-el7-x86_64/4644/artifact/exported- > artifacts/basic-suite-master__logs/001_initialize_engine.py.junit.xml > > > On Fri, Mar 23, 2018 at 12:14 PM, Dafna Ron wrote: > >> Hello, >> >> I would like to update on this week's failures and OST current status. >> >> On 19-03-2018 - the CI team reported 3 different failures. >> >> On Master branch the failed changes reported were: >> >> >> *core: fix removal of vm-host device - >> https://gerrit.ovirt.org/#/c/89145/ * >> >> *core: USB in osinfo configuration depends on chipset - >> https://gerrit.ovirt.org/#/c/88777/ * >> *On 4.2 *branch, the reported change was: >> >> >> >> *core: Call endAction() of all child commands in ImportVmCommand - >> https://gerrit.ovirt.org/#/c/89165/ * >> The fix's for the regressions were merged the following day (20-03-2018) >> >> https://gerrit.ovirt.org/#/c/89250/- core: Replace generic unlockVm() >> logic in ImportVmCommand >> https://gerrit.ovirt.org/#/c/89187/ - core: Fix NPE when creating an >> instance type >> >> On 20-03-2018 - the CI team discovered an issue on the job's cleanup >> which caused random failures on changes testing due to failure in docker >> cleanup. There is an open Jira on the issue: >> https://ovirt-jira.atlassian.net/browse/OVIRT-1939 >> >> >> >> *Below you can see the chart for this week's resolved issues but cause of >> failure:*Code = regression of working components/functionalities >> Configurations = package related issues >> Other = failed build artifacts >> Infra = infrastructure/OST/Lago related issues >> >> >> >> >> >> >> >> >> >> >> *Below is a chart of resolved failures based on ovirt version* >> >> >> >> >> >> >> >> >> >> >> >> >> *Below is a chart showing failures by suite type: * >> Thank you, >> Dafna >> >> >> _______________________________________________ >> Infra mailing list >> Infra at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/infra >> >> > > > -- > > GREG SHEREMETA > > SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX > > Red Hat NA > > > > gshereme at redhat.com IRC: gshereme > > -- GREG SHEREMETA SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX Red Hat NA gshereme at redhat.com IRC: gshereme -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 27001 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 5835 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 23605 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 26288 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 6237 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 7163 bytes Desc: not available URL: From ykaul at redhat.com Sun Mar 25 07:07:17 2018 From: ykaul at redhat.com (Yaniv Kaul) Date: Sun, 25 Mar 2018 10:07:17 +0300 Subject: [ovirt-devel] OST Failure - Weekly update [17/03/2018-23/03/2018] In-Reply-To: References: Message-ID: + Network team. I'm not sure if we've moved to ovs 2.9 already? Y. On Sat, Mar 24, 2018 at 8:19 PM, Greg Sheremeta wrote: > Hi, > > Is there an ongoing engine master OST failure blocking? > > [ INFO ] Stage: Misc configuration > [ INFO ] Stage: Package installation > [ INFO ] Stage: Misc configuration > [ ERROR ] Failed to execute stage \'Misc configuration\': Failed to start > service \'openvswitch\' > [ INFO ] Yum Performing yum transaction rollback > > > These are unrelated code changes: > > http://jenkins.ovirt.org/job/ovirt-system-tests_master_ > check-patch-el7-x86_64/4644/ > https://gerrit.ovirt.org/#/c/89347/ > > and > http://jenkins.ovirt.org/job/ovirt-system-tests_master_ > check-patch-el7-x86_64/4647/ > https://gerrit.ovirt.org/67166 > > But they both die in 001, with exactly 1.24MB in the log and 'Failed to > start service openvswitch' > 001_initialize_engine.py.junit.xml 1.24 MB > > Full file: http://jenkins.ovirt.org/job/ovirt-system-tests_ > master_check-patch-el7-x86_64/4644/artifact/exported- > artifacts/basic-suite-master__logs/001_initialize_engine.py.junit.xml > > > On Fri, Mar 23, 2018 at 12:14 PM, Dafna Ron wrote: > >> Hello, >> >> I would like to update on this week's failures and OST current status. >> >> On 19-03-2018 - the CI team reported 3 different failures. >> >> On Master branch the failed changes reported were: >> >> >> *core: fix removal of vm-host device - >> https://gerrit.ovirt.org/#/c/89145/ * >> >> *core: USB in osinfo configuration depends on chipset - >> https://gerrit.ovirt.org/#/c/88777/ * >> *On 4.2 *branch, the reported change was: >> >> >> >> *core: Call endAction() of all child commands in ImportVmCommand - >> https://gerrit.ovirt.org/#/c/89165/ * >> The fix's for the regressions were merged the following day (20-03-2018) >> >> https://gerrit.ovirt.org/#/c/89250/- core: Replace generic unlockVm() >> logic in ImportVmCommand >> https://gerrit.ovirt.org/#/c/89187/ - core: Fix NPE when creating an >> instance type >> >> On 20-03-2018 - the CI team discovered an issue on the job's cleanup >> which caused random failures on changes testing due to failure in docker >> cleanup. There is an open Jira on the issue: >> https://ovirt-jira.atlassian.net/browse/OVIRT-1939 >> >> >> >> *Below you can see the chart for this week's resolved issues but cause of >> failure:*Code = regression of working components/functionalities >> Configurations = package related issues >> Other = failed build artifacts >> Infra = infrastructure/OST/Lago related issues >> >> >> >> >> >> >> >> >> >> >> *Below is a chart of resolved failures based on ovirt version* >> >> >> >> >> >> >> >> >> >> >> >> >> *Below is a chart showing failures by suite type: * >> Thank you, >> Dafna >> >> >> _______________________________________________ >> Infra mailing list >> Infra at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/infra >> >> > > > -- > > GREG SHEREMETA > > SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX > > Red Hat NA > > > > gshereme at redhat.com IRC: gshereme > > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 7163 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 27001 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 26288 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 6237 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 23605 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 5835 bytes Desc: not available URL: From didi at redhat.com Sun Mar 25 09:04:18 2018 From: didi at redhat.com (Yedidyah Bar David) Date: Sun, 25 Mar 2018 12:04:18 +0300 Subject: [ovirt-devel] OST Failure - Weekly update [17/03/2018-23/03/2018] In-Reply-To: References: Message-ID: basic suite failed for me too. /var/log/messages has[1]: Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open vSwitch Database Unit... Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System error Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: /etc/openvswitch/conf.db does not exist ... (warning). Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty database /etc/openvswitch/conf.db runuser: System error Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service: control process exited, code=exited status=1 Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to start Open vSwitch Database Unit. Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Dependency failed for Open vSwitch. Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Job openvswitch.service/start failed with result 'dependency'. Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Dependency failed for Open vSwitch Forwarding Unit. Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Job ovs-vswitchd.service/start failed with result 'dependency'. Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit ovsdb-server.service entered failed state. Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service failed. Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion failed for Open vSwitch Delete Transient Ports. Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service holdoff time over, scheduling restart. Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open vSwitch Database Unit... Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: Removed session 14. Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Removed slice User Slice of root. Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Stopping User Slice of root. Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System error Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: /etc/openvswitch/conf.db does not exist ... (warning). Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty database /etc/openvswitch/conf.db runuser: System error Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service: control process exited, code=exited status=1 Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to start Open vSwitch Database Unit. Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit ovsdb-server.service entered failed state. Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service failed. Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion failed for Open vSwitch Delete Transient Ports. Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service holdoff time over, scheduling restart. Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open vSwitch Database Unit... Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System error Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: /etc/openvswitch/conf.db does not exist ... (warning). Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty database /etc/openvswitch/conf.db runuser: System error Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service: control process exited, code=exited status=1 Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to start Open vSwitch Database Unit. Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit ovsdb-server.service entered failed state. Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service failed. Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion failed for Open vSwitch Delete Transient Ports. Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Created slice User Slice of root. Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: New session 17 of user root. Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting User Slice of root. Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Started Session 17 of user root. Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Session 17 of user root. Mar 25 04:42:05 lago-basic-suite-master-engine kernel: DCCP: Activated CCID 2 (TCP-like) Mar 25 04:42:05 lago-basic-suite-master-engine kernel: DCCP: Activated CCID 3 (TCP-Friendly Rate Control) Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service holdoff time over, scheduling restart. Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open vSwitch Database Unit... Mar 25 04:42:05 lago-basic-suite-master-engine kernel: sctp: Hash tables configured (bind 256/256) Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: Removed session 17. Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Removed slice User Slice of root. Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Stopping User Slice of root. Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System error Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: /etc/openvswitch/conf.db does not exist ... (warning). Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty database /etc/openvswitch/conf.db runuser: System error Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] Mar 25 04:42:05 lago-basic-suite-master-engine systemd: ovsdb-server.service: control process exited, code=exited status=1 [1] http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x86_64/4651/artifact/exported-artifacts/basic-suite-master__logs/test_logs/basic-suite-master/post-001_initialize_engine.py/lago-basic-suite-master-engine/_var_log/messages/*view*/ On Sun, Mar 25, 2018 at 10:07 AM, Yaniv Kaul wrote: > + Network team. > I'm not sure if we've moved to ovs 2.9 already? > Y. > > On Sat, Mar 24, 2018 at 8:19 PM, Greg Sheremeta > wrote: > >> Hi, >> >> Is there an ongoing engine master OST failure blocking? >> >> [ INFO ] Stage: Misc configuration >> [ INFO ] Stage: Package installation >> [ INFO ] Stage: Misc configuration >> [ ERROR ] Failed to execute stage \'Misc configuration\': Failed to start >> service \'openvswitch\' >> [ INFO ] Yum Performing yum transaction rollback >> >> >> These are unrelated code changes: >> >> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >> -patch-el7-x86_64/4644/ >> https://gerrit.ovirt.org/#/c/89347/ >> >> and >> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >> -patch-el7-x86_64/4647/ >> https://gerrit.ovirt.org/67166 >> >> But they both die in 001, with exactly 1.24MB in the log and 'Failed to >> start service openvswitch' >> 001_initialize_engine.py.junit.xml 1.24 MB >> >> Full file: http://jenkins.ovirt.org/job/ovirt-system-tests_master >> _check-patch-el7-x86_64/4644/artifact/exported-artifacts/bas >> ic-suite-master__logs/001_initialize_engine.py.junit.xml >> >> >> On Fri, Mar 23, 2018 at 12:14 PM, Dafna Ron wrote: >> >>> Hello, >>> >>> I would like to update on this week's failures and OST current status. >>> >>> On 19-03-2018 - the CI team reported 3 different failures. >>> >>> On Master branch the failed changes reported were: >>> >>> >>> *core: fix removal of vm-host device - >>> https://gerrit.ovirt.org/#/c/89145/ * >>> >>> *core: USB in osinfo configuration depends on chipset - >>> https://gerrit.ovirt.org/#/c/88777/ * >>> *On 4.2 *branch, the reported change was: >>> >>> >>> >>> *core: Call endAction() of all child commands in ImportVmCommand - >>> https://gerrit.ovirt.org/#/c/89165/ * >>> The fix's for the regressions were merged the following day (20-03-2018) >>> >>> https://gerrit.ovirt.org/#/c/89250/- core: Replace generic unlockVm() >>> logic in ImportVmCommand >>> https://gerrit.ovirt.org/#/c/89187/ - core: Fix NPE when creating an >>> instance type >>> >>> On 20-03-2018 - the CI team discovered an issue on the job's cleanup >>> which caused random failures on changes testing due to failure in docker >>> cleanup. There is an open Jira on the issue: >>> https://ovirt-jira.atlassian.net/browse/OVIRT-1939 >>> >>> >>> >>> *Below you can see the chart for this week's resolved issues but cause >>> of failure:*Code = regression of working components/functionalities >>> Configurations = package related issues >>> Other = failed build artifacts >>> Infra = infrastructure/OST/Lago related issues >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> *Below is a chart of resolved failures based on ovirt version* >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> *Below is a chart showing failures by suite type: * >>> Thank you, >>> Dafna >>> >>> >>> _______________________________________________ >>> Infra mailing list >>> Infra at ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/infra >>> >>> >> >> >> -- >> >> GREG SHEREMETA >> >> SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX >> >> Red Hat NA >> >> >> >> gshereme at redhat.com IRC: gshereme >> >> >> _______________________________________________ >> Devel mailing list >> Devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel >> > > > _______________________________________________ > Infra mailing list > Infra at ovirt.org > http://lists.ovirt.org/mailman/listinfo/infra > > -- Didi -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 23605 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 7163 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 27001 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 26288 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 6237 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 5835 bytes Desc: not available URL: From didi at redhat.com Sun Mar 25 09:32:41 2018 From: didi at redhat.com (Yedidyah Bar David) Date: Sun, 25 Mar 2018 12:32:41 +0300 Subject: [ovirt-devel] OST Failure - Weekly update [17/03/2018-23/03/2018] In-Reply-To: References: Message-ID: On Sun, Mar 25, 2018 at 12:04 PM, Yedidyah Bar David wrote: > basic suite failed for me too. > > /var/log/messages has[1]: > > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open > vSwitch Database Unit... > Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System > error > Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: > /etc/openvswitch/conf.db does not exist ... (warning). > Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty > database /etc/openvswitch/conf.db runuser: System error > Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: > ovsdb-server.service: control process exited, code=exited status=1 > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to start > Open vSwitch Database Unit. > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Dependency failed > for Open vSwitch. > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Job > openvswitch.service/start failed with result 'dependency'. > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Dependency failed > for Open vSwitch Forwarding Unit. > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Job > ovs-vswitchd.service/start failed with result 'dependency'. > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit > ovsdb-server.service entered failed state. > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: > ovsdb-server.service failed. > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion failed > for Open vSwitch Delete Transient Ports. > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: > ovsdb-server.service holdoff time over, scheduling restart. > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open > vSwitch Database Unit... > Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: Removed > session 14. > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Removed slice User > Slice of root. > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Stopping User > Slice of root. > Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System > error > Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: > /etc/openvswitch/conf.db does not exist ... (warning). > Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty > database /etc/openvswitch/conf.db runuser: System error > Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: > ovsdb-server.service: control process exited, code=exited status=1 > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to start > Open vSwitch Database Unit. > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit > ovsdb-server.service entered failed state. > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: > ovsdb-server.service failed. > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion failed > for Open vSwitch Delete Transient Ports. > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: > ovsdb-server.service holdoff time over, scheduling restart. > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open > vSwitch Database Unit... > Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System > error > Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: > /etc/openvswitch/conf.db does not exist ... (warning). > Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty > database /etc/openvswitch/conf.db runuser: System error > Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: > ovsdb-server.service: control process exited, code=exited status=1 > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to start > Open vSwitch Database Unit. > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit > ovsdb-server.service entered failed state. > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: > ovsdb-server.service failed. > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion failed > for Open vSwitch Delete Transient Ports. > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Created slice User > Slice of root. > Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: New session > 17 of user root. > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting User > Slice of root. > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Started Session 17 > of user root. > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Session > 17 of user root. > Mar 25 04:42:05 lago-basic-suite-master-engine kernel: DCCP: Activated > CCID 2 (TCP-like) > Mar 25 04:42:05 lago-basic-suite-master-engine kernel: DCCP: Activated > CCID 3 (TCP-Friendly Rate Control) > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: > ovsdb-server.service holdoff time over, scheduling restart. > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open > vSwitch Database Unit... > Mar 25 04:42:05 lago-basic-suite-master-engine kernel: sctp: Hash tables > configured (bind 256/256) > Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: Removed > session 17. > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Removed slice User > Slice of root. > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Stopping User > Slice of root. > Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System > error > Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: > /etc/openvswitch/conf.db does not exist ... (warning). > Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty > database /etc/openvswitch/conf.db runuser: System error > Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] > Mar 25 04:42:05 lago-basic-suite-master-engine systemd: > ovsdb-server.service: control process exited, code=exited status=1 > > [1] http://jenkins.ovirt.org/job/ovirt-system-tests_master_ > check-patch-el7-x86_64/4651/artifact/exported-artifacts/ > basic-suite-master__logs/test_logs/basic-suite-master/post- > 001_initialize_engine.py/lago-basic-suite-master-engine/_ > var_log/messages/*view*/ > > Talked with danken, he asked to check if it's an selinux issue. It is. audit lot has: type=AVC msg=audit(1521967325.146:675): avc: denied { create } for pid=3787 comm="runuser" scontext=system_u:system_r:openvswitch_t:s0 tcontext=system_u:system_r:openvswitch_t:s0 tclass=netlink_audit_socket type=SYSCALL msg=audit(1521967325.146:675): arch=c000003e syscall=41 success=no exit=-13 a0=10 a1=3 a2=9 a3=7ffc4e12b930 items=0 ppid=3786 pid=3787 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="runuser" exe="/usr/sbin/runuser" subj=system_u:system_r:openvswitch_t:s0 key=(null) type=PROCTITLE msg=audit(1521967325.146:675): proctitle=72756E75736572002D2D75736572006F70656E76737769746368002D2D006F767364622D746F6F6C002D76636F6E736F6C653A6F666600736368656D612D76657273696F6E002F7573722F73686172652F6F70656E767377697463682F767377697463682E6F7673736368656D61 type=AVC msg=audit(1521967325.150:676): avc: denied { create } for pid=3789 comm="runuser" scontext=system_u:system_r:openvswitch_t:s0 tcontext=system_u:system_r:openvswitch_t:s0 tclass=netlink_audit_socket type=SYSCALL msg=audit(1521967325.150:676): arch=c000003e syscall=41 success=no exit=-13 a0=10 a1=3 a2=9 a3=7ffe03060130 items=0 ppid=3755 pid=3789 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="runuser" exe="/usr/sbin/runuser" subj=system_u:system_r:openvswitch_t:s0 key=(null) type=PROCTITLE msg=audit(1521967325.150:676): http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x86_64/4651/artifact/exported-artifacts/basic-suite-master__logs/test_logs/basic-suite-master/post-001_initialize_engine.py/lago-basic-suite-master-engine/_var_log/audit/audit.log And it's 2.9: Mar 25 04:38:39 lago-basic-suite-master-engine yum[1183]: Installed: 1:python2-openvswitch-2.9.0-3.el7.noarch > > On Sun, Mar 25, 2018 at 10:07 AM, Yaniv Kaul wrote: > >> + Network team. >> I'm not sure if we've moved to ovs 2.9 already? >> Y. >> >> On Sat, Mar 24, 2018 at 8:19 PM, Greg Sheremeta >> wrote: >> >>> Hi, >>> >>> Is there an ongoing engine master OST failure blocking? >>> >>> [ INFO ] Stage: Misc configuration >>> [ INFO ] Stage: Package installation >>> [ INFO ] Stage: Misc configuration >>> [ ERROR ] Failed to execute stage \'Misc configuration\': Failed to >>> start service \'openvswitch\' >>> [ INFO ] Yum Performing yum transaction rollback >>> >>> >>> These are unrelated code changes: >>> >>> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >>> -patch-el7-x86_64/4644/ >>> https://gerrit.ovirt.org/#/c/89347/ >>> >>> and >>> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >>> -patch-el7-x86_64/4647/ >>> https://gerrit.ovirt.org/67166 >>> >>> But they both die in 001, with exactly 1.24MB in the log and 'Failed to >>> start service openvswitch' >>> 001_initialize_engine.py.junit.xml 1.24 MB >>> >>> Full file: http://jenkins.ovirt.org/job/ovirt-system-tests_master >>> _check-patch-el7-x86_64/4644/artifact/exported-artifacts/bas >>> ic-suite-master__logs/001_initialize_engine.py.junit.xml >>> >>> >>> On Fri, Mar 23, 2018 at 12:14 PM, Dafna Ron wrote: >>> >>>> Hello, >>>> >>>> I would like to update on this week's failures and OST current status. >>>> >>>> On 19-03-2018 - the CI team reported 3 different failures. >>>> >>>> On Master branch the failed changes reported were: >>>> >>>> >>>> *core: fix removal of vm-host device - >>>> https://gerrit.ovirt.org/#/c/89145/ * >>>> >>>> *core: USB in osinfo configuration depends on chipset - >>>> https://gerrit.ovirt.org/#/c/88777/ * >>>> *On 4.2 *branch, the reported change was: >>>> >>>> >>>> >>>> *core: Call endAction() of all child commands in ImportVmCommand - >>>> https://gerrit.ovirt.org/#/c/89165/ * >>>> The fix's for the regressions were merged the following day >>>> (20-03-2018) >>>> >>>> https://gerrit.ovirt.org/#/c/89250/- core: Replace generic unlockVm() >>>> logic in ImportVmCommand >>>> https://gerrit.ovirt.org/#/c/89187/ - core: Fix NPE when creating an >>>> instance type >>>> >>>> On 20-03-2018 - the CI team discovered an issue on the job's cleanup >>>> which caused random failures on changes testing due to failure in docker >>>> cleanup. There is an open Jira on the issue: >>>> https://ovirt-jira.atlassian.net/browse/OVIRT-1939 >>>> >>>> >>>> >>>> *Below you can see the chart for this week's resolved issues but cause >>>> of failure:*Code = regression of working components/functionalities >>>> Configurations = package related issues >>>> Other = failed build artifacts >>>> Infra = infrastructure/OST/Lago related issues >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> *Below is a chart of resolved failures based on ovirt version* >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> *Below is a chart showing failures by suite type: * >>>> Thank you, >>>> Dafna >>>> >>>> >>>> _______________________________________________ >>>> Infra mailing list >>>> Infra at ovirt.org >>>> http://lists.ovirt.org/mailman/listinfo/infra >>>> >>>> >>> >>> >>> -- >>> >>> GREG SHEREMETA >>> >>> SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX >>> >>> Red Hat NA >>> >>> >>> >>> gshereme at redhat.com IRC: gshereme >>> >>> >>> _______________________________________________ >>> Devel mailing list >>> Devel at ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/devel >>> >> >> >> _______________________________________________ >> Infra mailing list >> Infra at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/infra >> >> > > > -- > Didi > -- Didi -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 6237 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 27001 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 23605 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 5835 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 7163 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 26288 bytes Desc: not available URL: From danken at redhat.com Sun Mar 25 11:22:05 2018 From: danken at redhat.com (Dan Kenigsberg) Date: Sun, 25 Mar 2018 14:22:05 +0300 Subject: [ovirt-devel] OST Failure - Weekly update [17/03/2018-23/03/2018] In-Reply-To: References: Message-ID: Which version of selinux-policy do we have on the Engine image? *Bug 1538936* - Open vSwitch selinux policy needs updating [rhel-7.4.z] was fixed in selinux-policy-3.13.1-166.el7_4.9 which is available in http://mirror.centos.org/centos-7/7/updates/x86_64/Packages/selinux-policy-targeted-3.13.1-166.el7_4.9.noarch.rpm On Sun, Mar 25, 2018 at 12:32 PM, Yedidyah Bar David wrote: > On Sun, Mar 25, 2018 at 12:04 PM, Yedidyah Bar David > wrote: > >> basic suite failed for me too. >> >> /var/log/messages has[1]: >> >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open >> vSwitch Database Unit... >> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System >> error >> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >> /etc/openvswitch/conf.db does not exist ... (warning). >> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty >> database /etc/openvswitch/conf.db runuser: System error >> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >> ovsdb-server.service: control process exited, code=exited status=1 >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to start >> Open vSwitch Database Unit. >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Dependency failed >> for Open vSwitch. >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Job >> openvswitch.service/start failed with result 'dependency'. >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Dependency failed >> for Open vSwitch Forwarding Unit. >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Job >> ovs-vswitchd.service/start failed with result 'dependency'. >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit >> ovsdb-server.service entered failed state. >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >> ovsdb-server.service failed. >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion failed >> for Open vSwitch Delete Transient Ports. >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >> ovsdb-server.service holdoff time over, scheduling restart. >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open >> vSwitch Database Unit... >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: Removed >> session 14. >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Removed slice >> User Slice of root. >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Stopping User >> Slice of root. >> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System >> error >> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >> /etc/openvswitch/conf.db does not exist ... (warning). >> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty >> database /etc/openvswitch/conf.db runuser: System error >> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >> ovsdb-server.service: control process exited, code=exited status=1 >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to start >> Open vSwitch Database Unit. >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit >> ovsdb-server.service entered failed state. >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >> ovsdb-server.service failed. >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion failed >> for Open vSwitch Delete Transient Ports. >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >> ovsdb-server.service holdoff time over, scheduling restart. >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open >> vSwitch Database Unit... >> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System >> error >> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >> /etc/openvswitch/conf.db does not exist ... (warning). >> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty >> database /etc/openvswitch/conf.db runuser: System error >> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >> ovsdb-server.service: control process exited, code=exited status=1 >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to start >> Open vSwitch Database Unit. >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit >> ovsdb-server.service entered failed state. >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >> ovsdb-server.service failed. >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion failed >> for Open vSwitch Delete Transient Ports. >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Created slice >> User Slice of root. >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: New >> session 17 of user root. >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting User >> Slice of root. >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Started Session >> 17 of user root. >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Session >> 17 of user root. >> Mar 25 04:42:05 lago-basic-suite-master-engine kernel: DCCP: Activated >> CCID 2 (TCP-like) >> Mar 25 04:42:05 lago-basic-suite-master-engine kernel: DCCP: Activated >> CCID 3 (TCP-Friendly Rate Control) >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >> ovsdb-server.service holdoff time over, scheduling restart. >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open >> vSwitch Database Unit... >> Mar 25 04:42:05 lago-basic-suite-master-engine kernel: sctp: Hash tables >> configured (bind 256/256) >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: Removed >> session 17. >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Removed slice >> User Slice of root. >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Stopping User >> Slice of root. >> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System >> error >> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >> /etc/openvswitch/conf.db does not exist ... (warning). >> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty >> database /etc/openvswitch/conf.db runuser: System error >> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >> ovsdb-server.service: control process exited, code=exited status=1 >> >> [1] http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >> -patch-el7-x86_64/4651/artifact/exported-artifacts/basic- >> suite-master__logs/test_logs/basic-suite-master/post-001_ >> initialize_engine.py/lago-basic-suite-master-engine/_var_ >> log/messages/*view*/ >> >> > Talked with danken, he asked to check if it's an selinux issue. It is. > audit lot has: > > type=AVC msg=audit(1521967325.146:675): avc: denied { create } for pid=3787 comm="runuser" scontext=system_u:system_r:openvswitch_t:s0 tcontext=system_u:system_r:openvswitch_t:s0 tclass=netlink_audit_socket > type=SYSCALL msg=audit(1521967325.146:675): arch=c000003e syscall=41 success=no exit=-13 a0=10 a1=3 a2=9 a3=7ffc4e12b930 items=0 ppid=3786 pid=3787 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="runuser" exe="/usr/sbin/runuser" subj=system_u:system_r:openvswitch_t:s0 key=(null) > type=PROCTITLE msg=audit(1521967325.146:675): proctitle=72756E75736572002D2D75736572006F70656E76737769746368002D2D006F767364622D746F6F6C002D76636F6E736F6C653A6F666600736368656D612D76657273696F6E002F7573722F73686172652F6F70656E767377697463682F767377697463682E6F7673736368656D61 > type=AVC msg=audit(1521967325.150:676): avc: denied { create } for pid=3789 comm="runuser" scontext=system_u:system_r:openvswitch_t:s0 tcontext=system_u:system_r:openvswitch_t:s0 tclass=netlink_audit_socket > type=SYSCALL msg=audit(1521967325.150:676): arch=c000003e syscall=41 success=no exit=-13 a0=10 a1=3 a2=9 a3=7ffe03060130 items=0 ppid=3755 pid=3789 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="runuser" exe="/usr/sbin/runuser" subj=system_u:system_r:openvswitch_t:s0 key=(null) > type=PROCTITLE msg=audit(1521967325.150:676): > > http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x86_64/4651/artifact/exported-artifacts/basic-suite-master__logs/test_logs/basic-suite-master/post-001_initialize_engine.py/lago-basic-suite-master-engine/_var_log/audit/audit.log > > And it's 2.9: > > Mar 25 04:38:39 lago-basic-suite-master-engine yum[1183]: Installed: 1:python2-openvswitch-2.9.0-3.el7.noarch > > > >> >> On Sun, Mar 25, 2018 at 10:07 AM, Yaniv Kaul wrote: >> >>> + Network team. >>> I'm not sure if we've moved to ovs 2.9 already? >>> Y. >>> >>> On Sat, Mar 24, 2018 at 8:19 PM, Greg Sheremeta >>> wrote: >>> >>>> Hi, >>>> >>>> Is there an ongoing engine master OST failure blocking? >>>> >>>> [ INFO ] Stage: Misc configuration >>>> [ INFO ] Stage: Package installation >>>> [ INFO ] Stage: Misc configuration >>>> [ ERROR ] Failed to execute stage \'Misc configuration\': Failed to >>>> start service \'openvswitch\' >>>> [ INFO ] Yum Performing yum transaction rollback >>>> >>>> >>>> These are unrelated code changes: >>>> >>>> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >>>> -patch-el7-x86_64/4644/ >>>> https://gerrit.ovirt.org/#/c/89347/ >>>> >>>> and >>>> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >>>> -patch-el7-x86_64/4647/ >>>> https://gerrit.ovirt.org/67166 >>>> >>>> But they both die in 001, with exactly 1.24MB in the log and 'Failed >>>> to start service openvswitch' >>>> 001_initialize_engine.py.junit.xml 1.24 MB >>>> >>>> Full file: http://jenkins.ovirt.org/job/ovirt-system-tests_master >>>> _check-patch-el7-x86_64/4644/artifact/exported-artifacts/bas >>>> ic-suite-master__logs/001_initialize_engine.py.junit.xml >>>> >>>> >>>> On Fri, Mar 23, 2018 at 12:14 PM, Dafna Ron wrote: >>>> >>>>> Hello, >>>>> >>>>> I would like to update on this week's failures and OST current status. >>>>> >>>>> On 19-03-2018 - the CI team reported 3 different failures. >>>>> >>>>> On Master branch the failed changes reported were: >>>>> >>>>> >>>>> *core: fix removal of vm-host device - >>>>> https://gerrit.ovirt.org/#/c/89145/ * >>>>> >>>>> *core: USB in osinfo configuration depends on chipset - >>>>> https://gerrit.ovirt.org/#/c/88777/ * >>>>> *On 4.2 *branch, the reported change was: >>>>> >>>>> >>>>> >>>>> *core: Call endAction() of all child commands in ImportVmCommand - >>>>> https://gerrit.ovirt.org/#/c/89165/ * >>>>> The fix's for the regressions were merged the following day >>>>> (20-03-2018) >>>>> >>>>> https://gerrit.ovirt.org/#/c/89250/- core: Replace generic unlockVm() >>>>> logic in ImportVmCommand >>>>> https://gerrit.ovirt.org/#/c/89187/ - core: Fix NPE when creating an >>>>> instance type >>>>> >>>>> On 20-03-2018 - the CI team discovered an issue on the job's cleanup >>>>> which caused random failures on changes testing due to failure in docker >>>>> cleanup. There is an open Jira on the issue: >>>>> https://ovirt-jira.atlassian.net/browse/OVIRT-1939 >>>>> >>>>> >>>>> >>>>> *Below you can see the chart for this week's resolved issues but cause >>>>> of failure:*Code = regression of working components/functionalities >>>>> Configurations = package related issues >>>>> Other = failed build artifacts >>>>> Infra = infrastructure/OST/Lago related issues >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> *Below is a chart of resolved failures based on ovirt version* >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> *Below is a chart showing failures by suite type: * >>>>> Thank you, >>>>> Dafna >>>>> >>>>> >>>>> _______________________________________________ >>>>> Infra mailing list >>>>> Infra at ovirt.org >>>>> http://lists.ovirt.org/mailman/listinfo/infra >>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> GREG SHEREMETA >>>> >>>> SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX >>>> >>>> Red Hat NA >>>> >>>> >>>> >>>> gshereme at redhat.com IRC: gshereme >>>> >>>> >>>> _______________________________________________ >>>> Devel mailing list >>>> Devel at ovirt.org >>>> http://lists.ovirt.org/mailman/listinfo/devel >>>> >>> >>> >>> _______________________________________________ >>> Infra mailing list >>> Infra at ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/infra >>> >>> >> >> >> -- >> Didi >> > > > > -- > Didi > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 26288 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 6237 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 27001 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 5835 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 7163 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 23605 bytes Desc: not available URL: From gshereme at redhat.com Sun Mar 25 15:20:34 2018 From: gshereme at redhat.com (Greg Sheremeta) Date: Sun, 25 Mar 2018 11:20:34 -0400 Subject: [ovirt-devel] OST Failure - Weekly update [17/03/2018-23/03/2018] In-Reply-To: References: Message-ID: currently selinux-policy-3.13.1-166.el7_4.4.noarch updating selinux-policy on engine gets me past 001, and then 002 dies: http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x86_64/4663/console 15:05:45 # initialize_engine: rpm -qa: 15:05:46 CommandStatus(code=0, out='selinux-policy-3.13.1-166.el7_4.4.noarch\n', err='') 15:06:13 Package selinux-policy.noarch 0:3.13.1-166.el7_4.9 will be an update Updated: selinux-policy.noarch 0:3.13.1-166.el7_4.9 rpm -qa: 15:06:13 CommandStatus(code=0, out='selinux-policy-3.13.1-166.el7_4.9.noarch\n', err='') But later in 002 15:08:47 RuntimeError: 1 hosts failed installation: 15:08:47 lago-basic-suite-master-host-0: install_failed Perhaps selinux-policy needs to be updated on the hosts too? Not my area of expertise :) Greg On Sun, Mar 25, 2018 at 7:22 AM, Dan Kenigsberg wrote: > Which version of selinux-policy do we have on the Engine image? > > *Bug 1538936* - Open > vSwitch selinux policy needs updating [rhel-7.4.z] > > was fixed in selinux-policy-3.13.1-166.el7_4.9 which is available in > http://mirror.centos.org/centos-7/7/updates/x86_64/ > Packages/selinux-policy-targeted-3.13.1-166.el7_4.9.noarch.rpm > > > On Sun, Mar 25, 2018 at 12:32 PM, Yedidyah Bar David > wrote: > >> On Sun, Mar 25, 2018 at 12:04 PM, Yedidyah Bar David >> wrote: >> >>> basic suite failed for me too. >>> >>> /var/log/messages has[1]: >>> >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open >>> vSwitch Database Unit... >>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System >>> error >>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >>> /etc/openvswitch/conf.db does not exist ... (warning). >>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty >>> database /etc/openvswitch/conf.db runuser: System error >>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>> ovsdb-server.service: control process exited, code=exited status=1 >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to start >>> Open vSwitch Database Unit. >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Dependency >>> failed for Open vSwitch. >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Job >>> openvswitch.service/start failed with result 'dependency'. >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Dependency >>> failed for Open vSwitch Forwarding Unit. >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Job >>> ovs-vswitchd.service/start failed with result 'dependency'. >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit >>> ovsdb-server.service entered failed state. >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>> ovsdb-server.service failed. >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion failed >>> for Open vSwitch Delete Transient Ports. >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>> ovsdb-server.service holdoff time over, scheduling restart. >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open >>> vSwitch Database Unit... >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: Removed >>> session 14. >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Removed slice >>> User Slice of root. >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Stopping User >>> Slice of root. >>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System >>> error >>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >>> /etc/openvswitch/conf.db does not exist ... (warning). >>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty >>> database /etc/openvswitch/conf.db runuser: System error >>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>> ovsdb-server.service: control process exited, code=exited status=1 >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to start >>> Open vSwitch Database Unit. >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit >>> ovsdb-server.service entered failed state. >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>> ovsdb-server.service failed. >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion failed >>> for Open vSwitch Delete Transient Ports. >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>> ovsdb-server.service holdoff time over, scheduling restart. >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open >>> vSwitch Database Unit... >>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System >>> error >>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >>> /etc/openvswitch/conf.db does not exist ... (warning). >>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty >>> database /etc/openvswitch/conf.db runuser: System error >>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>> ovsdb-server.service: control process exited, code=exited status=1 >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to start >>> Open vSwitch Database Unit. >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit >>> ovsdb-server.service entered failed state. >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>> ovsdb-server.service failed. >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion failed >>> for Open vSwitch Delete Transient Ports. >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Created slice >>> User Slice of root. >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: New >>> session 17 of user root. >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting User >>> Slice of root. >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Started Session >>> 17 of user root. >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Session >>> 17 of user root. >>> Mar 25 04:42:05 lago-basic-suite-master-engine kernel: DCCP: Activated >>> CCID 2 (TCP-like) >>> Mar 25 04:42:05 lago-basic-suite-master-engine kernel: DCCP: Activated >>> CCID 3 (TCP-Friendly Rate Control) >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>> ovsdb-server.service holdoff time over, scheduling restart. >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open >>> vSwitch Database Unit... >>> Mar 25 04:42:05 lago-basic-suite-master-engine kernel: sctp: Hash tables >>> configured (bind 256/256) >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: Removed >>> session 17. >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Removed slice >>> User Slice of root. >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Stopping User >>> Slice of root. >>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System >>> error >>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >>> /etc/openvswitch/conf.db does not exist ... (warning). >>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty >>> database /etc/openvswitch/conf.db runuser: System error >>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>> ovsdb-server.service: control process exited, code=exited status=1 >>> >>> [1] http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >>> -patch-el7-x86_64/4651/artifact/exported-artifacts/basic-sui >>> te-master__logs/test_logs/basic-suite-master/post-001_initia >>> lize_engine.py/lago-basic-suite-master-engine/_var_log/messages/*view*/ >>> >>> >> Talked with danken, he asked to check if it's an selinux issue. It is. >> audit lot has: >> >> type=AVC msg=audit(1521967325.146:675): avc: denied { create } for pid=3787 comm="runuser" scontext=system_u:system_r:openvswitch_t:s0 tcontext=system_u:system_r:openvswitch_t:s0 tclass=netlink_audit_socket >> type=SYSCALL msg=audit(1521967325.146:675): arch=c000003e syscall=41 success=no exit=-13 a0=10 a1=3 a2=9 a3=7ffc4e12b930 items=0 ppid=3786 pid=3787 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="runuser" exe="/usr/sbin/runuser" subj=system_u:system_r:openvswitch_t:s0 key=(null) >> type=PROCTITLE msg=audit(1521967325.146:675): proctitle=72756E75736572002D2D75736572006F70656E76737769746368002D2D006F767364622D746F6F6C002D76636F6E736F6C653A6F666600736368656D612D76657273696F6E002F7573722F73686172652F6F70656E767377697463682F767377697463682E6F7673736368656D61 >> type=AVC msg=audit(1521967325.150:676): avc: denied { create } for pid=3789 comm="runuser" scontext=system_u:system_r:openvswitch_t:s0 tcontext=system_u:system_r:openvswitch_t:s0 tclass=netlink_audit_socket >> type=SYSCALL msg=audit(1521967325.150:676): arch=c000003e syscall=41 success=no exit=-13 a0=10 a1=3 a2=9 a3=7ffe03060130 items=0 ppid=3755 pid=3789 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="runuser" exe="/usr/sbin/runuser" subj=system_u:system_r:openvswitch_t:s0 key=(null) >> type=PROCTITLE msg=audit(1521967325.150:676): >> >> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x86_64/4651/artifact/exported-artifacts/basic-suite-master__logs/test_logs/basic-suite-master/post-001_initialize_engine.py/lago-basic-suite-master-engine/_var_log/audit/audit.log >> >> And it's 2.9: >> >> Mar 25 04:38:39 lago-basic-suite-master-engine yum[1183]: Installed: 1:python2-openvswitch-2.9.0-3.el7.noarch >> >> >> >>> >>> On Sun, Mar 25, 2018 at 10:07 AM, Yaniv Kaul wrote: >>> >>>> + Network team. >>>> I'm not sure if we've moved to ovs 2.9 already? >>>> Y. >>>> >>>> On Sat, Mar 24, 2018 at 8:19 PM, Greg Sheremeta >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> Is there an ongoing engine master OST failure blocking? >>>>> >>>>> [ INFO ] Stage: Misc configuration >>>>> [ INFO ] Stage: Package installation >>>>> [ INFO ] Stage: Misc configuration >>>>> [ ERROR ] Failed to execute stage \'Misc configuration\': Failed to >>>>> start service \'openvswitch\' >>>>> [ INFO ] Yum Performing yum transaction rollback >>>>> >>>>> >>>>> These are unrelated code changes: >>>>> >>>>> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >>>>> -patch-el7-x86_64/4644/ >>>>> https://gerrit.ovirt.org/#/c/89347/ >>>>> >>>>> and >>>>> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >>>>> -patch-el7-x86_64/4647/ >>>>> https://gerrit.ovirt.org/67166 >>>>> >>>>> But they both die in 001, with exactly 1.24MB in the log and 'Failed >>>>> to start service openvswitch' >>>>> 001_initialize_engine.py.junit.xml 1.24 MB >>>>> >>>>> Full file: http://jenkins.ovirt.org/job/ovirt-system-tests_master >>>>> _check-patch-el7-x86_64/4644/artifact/exported-artifacts/bas >>>>> ic-suite-master__logs/001_initialize_engine.py.junit.xml >>>>> >>>>> >>>>> On Fri, Mar 23, 2018 at 12:14 PM, Dafna Ron wrote: >>>>> >>>>>> Hello, >>>>>> >>>>>> I would like to update on this week's failures and OST current >>>>>> status. >>>>>> >>>>>> On 19-03-2018 - the CI team reported 3 different failures. >>>>>> >>>>>> On Master branch the failed changes reported were: >>>>>> >>>>>> >>>>>> *core: fix removal of vm-host device - >>>>>> https://gerrit.ovirt.org/#/c/89145/ * >>>>>> >>>>>> *core: USB in osinfo configuration depends on chipset - >>>>>> https://gerrit.ovirt.org/#/c/88777/ * >>>>>> *On 4.2 *branch, the reported change was: >>>>>> >>>>>> >>>>>> >>>>>> *core: Call endAction() of all child commands in ImportVmCommand - >>>>>> https://gerrit.ovirt.org/#/c/89165/ * >>>>>> The fix's for the regressions were merged the following day >>>>>> (20-03-2018) >>>>>> >>>>>> https://gerrit.ovirt.org/#/c/89250/- core: Replace generic >>>>>> unlockVm() logic in ImportVmCommand >>>>>> https://gerrit.ovirt.org/#/c/89187/ - core: Fix NPE when creating an >>>>>> instance type >>>>>> >>>>>> On 20-03-2018 - the CI team discovered an issue on the job's cleanup >>>>>> which caused random failures on changes testing due to failure in docker >>>>>> cleanup. There is an open Jira on the issue: >>>>>> https://ovirt-jira.atlassian.net/browse/OVIRT-1939 >>>>>> >>>>>> >>>>>> >>>>>> *Below you can see the chart for this week's resolved issues but >>>>>> cause of failure:*Code = regression of working >>>>>> components/functionalities >>>>>> Configurations = package related issues >>>>>> Other = failed build artifacts >>>>>> Infra = infrastructure/OST/Lago related issues >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> *Below is a chart of resolved failures based on ovirt version* >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> *Below is a chart showing failures by suite type: * >>>>>> Thank you, >>>>>> Dafna >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Infra mailing list >>>>>> Infra at ovirt.org >>>>>> http://lists.ovirt.org/mailman/listinfo/infra >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> GREG SHEREMETA >>>>> >>>>> SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX >>>>> >>>>> Red Hat NA >>>>> >>>>> >>>>> >>>>> gshereme at redhat.com IRC: gshereme >>>>> >>>>> >>>>> _______________________________________________ >>>>> Devel mailing list >>>>> Devel at ovirt.org >>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>> >>>> >>>> >>>> _______________________________________________ >>>> Infra mailing list >>>> Infra at ovirt.org >>>> http://lists.ovirt.org/mailman/listinfo/infra >>>> >>>> >>> >>> >>> -- >>> Didi >>> >> >> >> >> -- >> Didi >> > > -- GREG SHEREMETA SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX Red Hat NA gshereme at redhat.com IRC: gshereme -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 5835 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 6237 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 7163 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 26288 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 27001 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 23605 bytes Desc: not available URL: From gshereme at redhat.com Sun Mar 25 17:11:07 2018 From: gshereme at redhat.com (Greg Sheremeta) Date: Sun, 25 Mar 2018 13:11:07 -0400 Subject: [ovirt-devel] OST Failure - Weekly update [17/03/2018-23/03/2018] In-Reply-To: References: Message-ID: Indeed, updating selinux-policy on both engine and hosts passes. Change If671d938: [test do not merge] test selinux-policy update on engine and hosts | gerrit.ovirt Code Review https://gerrit.ovirt.org/#/c/89427/ http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x86_64/4676/consoleFull So I guess the images need updates. On Sun, Mar 25, 2018 at 11:20 AM, Greg Sheremeta wrote: > currently selinux-policy-3.13.1-166.el7_4.4.noarch > > updating selinux-policy on engine gets me past 001, and then 002 dies: > http://jenkins.ovirt.org/job/ovirt-system-tests_master_ > check-patch-el7-x86_64/4663/console > > 15:05:45 # initialize_engine: > rpm -qa: > 15:05:46 CommandStatus(code=0, out='selinux-policy-3.13.1-166.el7_4.4.noarch\n', > err='') > 15:06:13 Package selinux-policy.noarch 0:3.13.1-166.el7_4.9 will be > an update > Updated: selinux-policy.noarch 0:3.13.1-166.el7_4.9 > rpm -qa: > 15:06:13 CommandStatus(code=0, out='selinux-policy-3.13.1-166.el7_4.9.noarch\n', > err='') > > But later in 002 > 15:08:47 RuntimeError: 1 hosts failed installation: > 15:08:47 lago-basic-suite-master-host-0: install_failed > > Perhaps selinux-policy needs to be updated on the hosts too? Not my area > of expertise :) > > Greg > > On Sun, Mar 25, 2018 at 7:22 AM, Dan Kenigsberg wrote: > >> Which version of selinux-policy do we have on the Engine image? >> >> *Bug 1538936* - Open >> vSwitch selinux policy needs updating [rhel-7.4.z] >> >> was fixed in selinux-policy-3.13.1-166.el7_4.9 which is available in >> http://mirror.centos.org/centos-7/7/updates/x86_64/Packages/ >> selinux-policy-targeted-3.13.1-166.el7_4.9.noarch.rpm >> >> >> On Sun, Mar 25, 2018 at 12:32 PM, Yedidyah Bar David >> wrote: >> >>> On Sun, Mar 25, 2018 at 12:04 PM, Yedidyah Bar David >>> wrote: >>> >>>> basic suite failed for me too. >>>> >>>> /var/log/messages has[1]: >>>> >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open >>>> vSwitch Database Unit... >>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System >>>> error >>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >>>> /etc/openvswitch/conf.db does not exist ... (warning). >>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty >>>> database /etc/openvswitch/conf.db runuser: System error >>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>> ovsdb-server.service: control process exited, code=exited status=1 >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to start >>>> Open vSwitch Database Unit. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Dependency >>>> failed for Open vSwitch. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Job >>>> openvswitch.service/start failed with result 'dependency'. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Dependency >>>> failed for Open vSwitch Forwarding Unit. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Job >>>> ovs-vswitchd.service/start failed with result 'dependency'. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit >>>> ovsdb-server.service entered failed state. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>> ovsdb-server.service failed. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion >>>> failed for Open vSwitch Delete Transient Ports. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>> ovsdb-server.service holdoff time over, scheduling restart. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open >>>> vSwitch Database Unit... >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: Removed >>>> session 14. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Removed slice >>>> User Slice of root. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Stopping User >>>> Slice of root. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System >>>> error >>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >>>> /etc/openvswitch/conf.db does not exist ... (warning). >>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty >>>> database /etc/openvswitch/conf.db runuser: System error >>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>> ovsdb-server.service: control process exited, code=exited status=1 >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to start >>>> Open vSwitch Database Unit. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit >>>> ovsdb-server.service entered failed state. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>> ovsdb-server.service failed. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion >>>> failed for Open vSwitch Delete Transient Ports. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>> ovsdb-server.service holdoff time over, scheduling restart. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open >>>> vSwitch Database Unit... >>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System >>>> error >>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >>>> /etc/openvswitch/conf.db does not exist ... (warning). >>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty >>>> database /etc/openvswitch/conf.db runuser: System error >>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>> ovsdb-server.service: control process exited, code=exited status=1 >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to start >>>> Open vSwitch Database Unit. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit >>>> ovsdb-server.service entered failed state. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>> ovsdb-server.service failed. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion >>>> failed for Open vSwitch Delete Transient Ports. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Created slice >>>> User Slice of root. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: New >>>> session 17 of user root. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting User >>>> Slice of root. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Started Session >>>> 17 of user root. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting >>>> Session 17 of user root. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine kernel: DCCP: Activated >>>> CCID 2 (TCP-like) >>>> Mar 25 04:42:05 lago-basic-suite-master-engine kernel: DCCP: Activated >>>> CCID 3 (TCP-Friendly Rate Control) >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>> ovsdb-server.service holdoff time over, scheduling restart. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open >>>> vSwitch Database Unit... >>>> Mar 25 04:42:05 lago-basic-suite-master-engine kernel: sctp: Hash >>>> tables configured (bind 256/256) >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: Removed >>>> session 17. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Removed slice >>>> User Slice of root. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Stopping User >>>> Slice of root. >>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: System >>>> error >>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >>>> /etc/openvswitch/conf.db does not exist ... (warning). >>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty >>>> database /etc/openvswitch/conf.db runuser: System error >>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>> ovsdb-server.service: control process exited, code=exited status=1 >>>> >>>> [1] http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >>>> -patch-el7-x86_64/4651/artifact/exported-artifacts/basic-sui >>>> te-master__logs/test_logs/basic-suite-master/post-001_initia >>>> lize_engine.py/lago-basic-suite-master-engine/_var_log/messages/*view*/ >>>> >>>> >>> Talked with danken, he asked to check if it's an selinux issue. It is. >>> audit lot has: >>> >>> type=AVC msg=audit(1521967325.146:675): avc: denied { create } for pid=3787 comm="runuser" scontext=system_u:system_r:openvswitch_t:s0 tcontext=system_u:system_r:openvswitch_t:s0 tclass=netlink_audit_socket >>> type=SYSCALL msg=audit(1521967325.146:675): arch=c000003e syscall=41 success=no exit=-13 a0=10 a1=3 a2=9 a3=7ffc4e12b930 items=0 ppid=3786 pid=3787 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="runuser" exe="/usr/sbin/runuser" subj=system_u:system_r:openvswitch_t:s0 key=(null) >>> type=PROCTITLE msg=audit(1521967325.146:675): proctitle=72756E75736572002D2D75736572006F70656E76737769746368002D2D006F767364622D746F6F6C002D76636F6E736F6C653A6F666600736368656D612D76657273696F6E002F7573722F73686172652F6F70656E767377697463682F767377697463682E6F7673736368656D61 >>> type=AVC msg=audit(1521967325.150:676): avc: denied { create } for pid=3789 comm="runuser" scontext=system_u:system_r:openvswitch_t:s0 tcontext=system_u:system_r:openvswitch_t:s0 tclass=netlink_audit_socket >>> type=SYSCALL msg=audit(1521967325.150:676): arch=c000003e syscall=41 success=no exit=-13 a0=10 a1=3 a2=9 a3=7ffe03060130 items=0 ppid=3755 pid=3789 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="runuser" exe="/usr/sbin/runuser" subj=system_u:system_r:openvswitch_t:s0 key=(null) >>> type=PROCTITLE msg=audit(1521967325.150:676): >>> >>> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x86_64/4651/artifact/exported-artifacts/basic-suite-master__logs/test_logs/basic-suite-master/post-001_initialize_engine.py/lago-basic-suite-master-engine/_var_log/audit/audit.log >>> >>> And it's 2.9: >>> >>> Mar 25 04:38:39 lago-basic-suite-master-engine yum[1183]: Installed: 1:python2-openvswitch-2.9.0-3.el7.noarch >>> >>> >>> >>>> >>>> On Sun, Mar 25, 2018 at 10:07 AM, Yaniv Kaul wrote: >>>> >>>>> + Network team. >>>>> I'm not sure if we've moved to ovs 2.9 already? >>>>> Y. >>>>> >>>>> On Sat, Mar 24, 2018 at 8:19 PM, Greg Sheremeta >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> Is there an ongoing engine master OST failure blocking? >>>>>> >>>>>> [ INFO ] Stage: Misc configuration >>>>>> [ INFO ] Stage: Package installation >>>>>> [ INFO ] Stage: Misc configuration >>>>>> [ ERROR ] Failed to execute stage \'Misc configuration\': Failed to >>>>>> start service \'openvswitch\' >>>>>> [ INFO ] Yum Performing yum transaction rollback >>>>>> >>>>>> >>>>>> These are unrelated code changes: >>>>>> >>>>>> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >>>>>> -patch-el7-x86_64/4644/ >>>>>> https://gerrit.ovirt.org/#/c/89347/ >>>>>> >>>>>> and >>>>>> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >>>>>> -patch-el7-x86_64/4647/ >>>>>> https://gerrit.ovirt.org/67166 >>>>>> >>>>>> But they both die in 001, with exactly 1.24MB in the log and 'Failed >>>>>> to start service openvswitch' >>>>>> 001_initialize_engine.py.junit.xml 1.24 MB >>>>>> >>>>>> Full file: http://jenkins.ovirt.org/job/ovirt-system-tests_master >>>>>> _check-patch-el7-x86_64/4644/artifact/exported-artifacts/bas >>>>>> ic-suite-master__logs/001_initialize_engine.py.junit.xml >>>>>> >>>>>> >>>>>> On Fri, Mar 23, 2018 at 12:14 PM, Dafna Ron wrote: >>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> I would like to update on this week's failures and OST current >>>>>>> status. >>>>>>> >>>>>>> On 19-03-2018 - the CI team reported 3 different failures. >>>>>>> >>>>>>> On Master branch the failed changes reported were: >>>>>>> >>>>>>> >>>>>>> *core: fix removal of vm-host device - >>>>>>> https://gerrit.ovirt.org/#/c/89145/ * >>>>>>> >>>>>>> *core: USB in osinfo configuration depends on chipset - >>>>>>> https://gerrit.ovirt.org/#/c/88777/ * >>>>>>> *On 4.2 *branch, the reported change was: >>>>>>> >>>>>>> >>>>>>> >>>>>>> *core: Call endAction() of all child commands in ImportVmCommand - >>>>>>> https://gerrit.ovirt.org/#/c/89165/ * >>>>>>> The fix's for the regressions were merged the following day >>>>>>> (20-03-2018) >>>>>>> >>>>>>> https://gerrit.ovirt.org/#/c/89250/- core: Replace generic >>>>>>> unlockVm() logic in ImportVmCommand >>>>>>> https://gerrit.ovirt.org/#/c/89187/ - core: Fix NPE when creating >>>>>>> an instance type >>>>>>> >>>>>>> On 20-03-2018 - the CI team discovered an issue on the job's cleanup >>>>>>> which caused random failures on changes testing due to failure in docker >>>>>>> cleanup. There is an open Jira on the issue: >>>>>>> https://ovirt-jira.atlassian.net/browse/OVIRT-1939 >>>>>>> >>>>>>> >>>>>>> >>>>>>> *Below you can see the chart for this week's resolved issues but >>>>>>> cause of failure:*Code = regression of working >>>>>>> components/functionalities >>>>>>> Configurations = package related issues >>>>>>> Other = failed build artifacts >>>>>>> Infra = infrastructure/OST/Lago related issues >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> *Below is a chart of resolved failures based on ovirt version* >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> *Below is a chart showing failures by suite type: * >>>>>>> Thank you, >>>>>>> Dafna >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Infra mailing list >>>>>>> Infra at ovirt.org >>>>>>> http://lists.ovirt.org/mailman/listinfo/infra >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> GREG SHEREMETA >>>>>> >>>>>> SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX >>>>>> >>>>>> Red Hat NA >>>>>> >>>>>> >>>>>> >>>>>> gshereme at redhat.com IRC: gshereme >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Devel mailing list >>>>>> Devel at ovirt.org >>>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Infra mailing list >>>>> Infra at ovirt.org >>>>> http://lists.ovirt.org/mailman/listinfo/infra >>>>> >>>>> >>>> >>>> >>>> -- >>>> Didi >>>> >>> >>> >>> >>> -- >>> Didi >>> >> >> > > > -- > > GREG SHEREMETA > > SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX > > Red Hat NA > > > > gshereme at redhat.com IRC: gshereme > > -- GREG SHEREMETA SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX Red Hat NA gshereme at redhat.com IRC: gshereme -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 7163 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 6237 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 5835 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 23605 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 26288 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 27001 bytes Desc: not available URL: From eedri at redhat.com Sun Mar 25 17:15:08 2018 From: eedri at redhat.com (Eyal Edri) Date: Sun, 25 Mar 2018 20:15:08 +0300 Subject: [ovirt-devel] OST Failure - Weekly update [17/03/2018-23/03/2018] In-Reply-To: References: Message-ID: Just wondering, if the bug requires updated selinux, shouldn't the ovs PKG enforce it I'm the spec file? On Mar 25, 2018 20:12, "Greg Sheremeta" wrote: > Indeed, updating selinux-policy on both engine and hosts passes. > > Change If671d938: [test do not merge] test selinux-policy update on engine > and hosts | gerrit.ovirt Code Review > https://gerrit.ovirt.org/#/c/89427/ > http://jenkins.ovirt.org/job/ovirt-system-tests_master_ > check-patch-el7-x86_64/4676/consoleFull > > So I guess the images need updates. > > On Sun, Mar 25, 2018 at 11:20 AM, Greg Sheremeta > wrote: > >> currently selinux-policy-3.13.1-166.el7_4.4.noarch >> >> updating selinux-policy on engine gets me past 001, and then 002 dies: >> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >> -patch-el7-x86_64/4663/console >> >> 15:05:45 # initialize_engine: >> rpm -qa: >> 15:05:46 CommandStatus(code=0, out='selinux-policy-3.13.1-166.el7_4.4.noarch\n', >> err='') >> 15:06:13 Package selinux-policy.noarch 0:3.13.1-166.el7_4.9 will >> be an update >> Updated: selinux-policy.noarch 0:3.13.1-166.el7_4.9 >> rpm -qa: >> 15:06:13 CommandStatus(code=0, out='selinux-policy-3.13.1-166.el7_4.9.noarch\n', >> err='') >> >> But later in 002 >> 15:08:47 RuntimeError: 1 hosts failed installation: >> 15:08:47 lago-basic-suite-master-host-0: install_failed >> >> Perhaps selinux-policy needs to be updated on the hosts too? Not my area >> of expertise :) >> >> Greg >> >> On Sun, Mar 25, 2018 at 7:22 AM, Dan Kenigsberg >> wrote: >> >>> Which version of selinux-policy do we have on the Engine image? >>> >>> *Bug 1538936* - Open >>> vSwitch selinux policy needs updating [rhel-7.4.z] >>> >>> was fixed in selinux-policy-3.13.1-166.el7_4.9 which is available in >>> http://mirror.centos.org/centos-7/7/updates/x86_64/Packages/ >>> selinux-policy-targeted-3.13.1-166.el7_4.9.noarch.rpm >>> >>> >>> On Sun, Mar 25, 2018 at 12:32 PM, Yedidyah Bar David >>> wrote: >>> >>>> On Sun, Mar 25, 2018 at 12:04 PM, Yedidyah Bar David >>>> wrote: >>>> >>>>> basic suite failed for me too. >>>>> >>>>> /var/log/messages has[1]: >>>>> >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open >>>>> vSwitch Database Unit... >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: >>>>> System error >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >>>>> /etc/openvswitch/conf.db does not exist ... (warning). >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty >>>>> database /etc/openvswitch/conf.db runuser: System error >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>> ovsdb-server.service: control process exited, code=exited status=1 >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to >>>>> start Open vSwitch Database Unit. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Dependency >>>>> failed for Open vSwitch. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Job >>>>> openvswitch.service/start failed with result 'dependency'. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Dependency >>>>> failed for Open vSwitch Forwarding Unit. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Job >>>>> ovs-vswitchd.service/start failed with result 'dependency'. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit >>>>> ovsdb-server.service entered failed state. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>> ovsdb-server.service failed. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion >>>>> failed for Open vSwitch Delete Transient Ports. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>> ovsdb-server.service holdoff time over, scheduling restart. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open >>>>> vSwitch Database Unit... >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: Removed >>>>> session 14. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Removed slice >>>>> User Slice of root. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Stopping User >>>>> Slice of root. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: >>>>> System error >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >>>>> /etc/openvswitch/conf.db does not exist ... (warning). >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty >>>>> database /etc/openvswitch/conf.db runuser: System error >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>> ovsdb-server.service: control process exited, code=exited status=1 >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to >>>>> start Open vSwitch Database Unit. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit >>>>> ovsdb-server.service entered failed state. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>> ovsdb-server.service failed. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion >>>>> failed for Open vSwitch Delete Transient Ports. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>> ovsdb-server.service holdoff time over, scheduling restart. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open >>>>> vSwitch Database Unit... >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: >>>>> System error >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >>>>> /etc/openvswitch/conf.db does not exist ... (warning). >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty >>>>> database /etc/openvswitch/conf.db runuser: System error >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>> ovsdb-server.service: control process exited, code=exited status=1 >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to >>>>> start Open vSwitch Database Unit. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit >>>>> ovsdb-server.service entered failed state. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>> ovsdb-server.service failed. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion >>>>> failed for Open vSwitch Delete Transient Ports. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Created slice >>>>> User Slice of root. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: New >>>>> session 17 of user root. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting User >>>>> Slice of root. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Started >>>>> Session 17 of user root. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting >>>>> Session 17 of user root. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine kernel: DCCP: Activated >>>>> CCID 2 (TCP-like) >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine kernel: DCCP: Activated >>>>> CCID 3 (TCP-Friendly Rate Control) >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>> ovsdb-server.service holdoff time over, scheduling restart. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open >>>>> vSwitch Database Unit... >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine kernel: sctp: Hash >>>>> tables configured (bind 256/256) >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: Removed >>>>> session 17. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Removed slice >>>>> User Slice of root. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Stopping User >>>>> Slice of root. >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: >>>>> System error >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >>>>> /etc/openvswitch/conf.db does not exist ... (warning). >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating empty >>>>> database /etc/openvswitch/conf.db runuser: System error >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>> ovsdb-server.service: control process exited, code=exited status=1 >>>>> >>>>> [1] http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >>>>> -patch-el7-x86_64/4651/artifact/exported-artifacts/basic-sui >>>>> te-master__logs/test_logs/basic-suite-master/post-001_initia >>>>> lize_engine.py/lago-basic-suite-master-engine/_var_log/messa >>>>> ges/*view*/ >>>>> >>>>> >>>> Talked with danken, he asked to check if it's an selinux issue. It is. >>>> audit lot has: >>>> >>>> type=AVC msg=audit(1521967325.146:675): avc: denied { create } for pid=3787 comm="runuser" scontext=system_u:system_r:openvswitch_t:s0 tcontext=system_u:system_r:openvswitch_t:s0 tclass=netlink_audit_socket >>>> type=SYSCALL msg=audit(1521967325.146:675): arch=c000003e syscall=41 success=no exit=-13 a0=10 a1=3 a2=9 a3=7ffc4e12b930 items=0 ppid=3786 pid=3787 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="runuser" exe="/usr/sbin/runuser" subj=system_u:system_r:openvswitch_t:s0 key=(null) >>>> type=PROCTITLE msg=audit(1521967325.146:675): proctitle=72756E75736572002D2D75736572006F70656E76737769746368002D2D006F767364622D746F6F6C002D76636F6E736F6C653A6F666600736368656D612D76657273696F6E002F7573722F73686172652F6F70656E767377697463682F767377697463682E6F7673736368656D61 >>>> type=AVC msg=audit(1521967325.150:676): avc: denied { create } for pid=3789 comm="runuser" scontext=system_u:system_r:openvswitch_t:s0 tcontext=system_u:system_r:openvswitch_t:s0 tclass=netlink_audit_socket >>>> type=SYSCALL msg=audit(1521967325.150:676): arch=c000003e syscall=41 success=no exit=-13 a0=10 a1=3 a2=9 a3=7ffe03060130 items=0 ppid=3755 pid=3789 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="runuser" exe="/usr/sbin/runuser" subj=system_u:system_r:openvswitch_t:s0 key=(null) >>>> type=PROCTITLE msg=audit(1521967325.150:676): >>>> >>>> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x86_64/4651/artifact/exported-artifacts/basic-suite-master__logs/test_logs/basic-suite-master/post-001_initialize_engine.py/lago-basic-suite-master-engine/_var_log/audit/audit.log >>>> >>>> And it's 2.9: >>>> >>>> Mar 25 04:38:39 lago-basic-suite-master-engine yum[1183]: Installed: 1:python2-openvswitch-2.9.0-3.el7.noarch >>>> >>>> >>>> >>>>> >>>>> On Sun, Mar 25, 2018 at 10:07 AM, Yaniv Kaul wrote: >>>>> >>>>>> + Network team. >>>>>> I'm not sure if we've moved to ovs 2.9 already? >>>>>> Y. >>>>>> >>>>>> On Sat, Mar 24, 2018 at 8:19 PM, Greg Sheremeta >>>>>> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> Is there an ongoing engine master OST failure blocking? >>>>>>> >>>>>>> [ INFO ] Stage: Misc configuration >>>>>>> [ INFO ] Stage: Package installation >>>>>>> [ INFO ] Stage: Misc configuration >>>>>>> [ ERROR ] Failed to execute stage \'Misc configuration\': Failed to >>>>>>> start service \'openvswitch\' >>>>>>> [ INFO ] Yum Performing yum transaction rollback >>>>>>> >>>>>>> >>>>>>> These are unrelated code changes: >>>>>>> >>>>>>> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >>>>>>> -patch-el7-x86_64/4644/ >>>>>>> https://gerrit.ovirt.org/#/c/89347/ >>>>>>> >>>>>>> and >>>>>>> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >>>>>>> -patch-el7-x86_64/4647/ >>>>>>> https://gerrit.ovirt.org/67166 >>>>>>> >>>>>>> But they both die in 001, with exactly 1.24MB in the log and 'Failed >>>>>>> to start service openvswitch' >>>>>>> 001_initialize_engine.py.junit.xml 1.24 MB >>>>>>> >>>>>>> Full file: http://jenkins.ovirt.org/job/ovirt-system-tests_master >>>>>>> _check-patch-el7-x86_64/4644/artifact/exported-artifacts/bas >>>>>>> ic-suite-master__logs/001_initialize_engine.py.junit.xml >>>>>>> >>>>>>> >>>>>>> On Fri, Mar 23, 2018 at 12:14 PM, Dafna Ron wrote: >>>>>>> >>>>>>>> Hello, >>>>>>>> >>>>>>>> I would like to update on this week's failures and OST current >>>>>>>> status. >>>>>>>> >>>>>>>> On 19-03-2018 - the CI team reported 3 different failures. >>>>>>>> >>>>>>>> On Master branch the failed changes reported were: >>>>>>>> >>>>>>>> >>>>>>>> *core: fix removal of vm-host device - >>>>>>>> https://gerrit.ovirt.org/#/c/89145/ * >>>>>>>> >>>>>>>> *core: USB in osinfo configuration depends on chipset - >>>>>>>> https://gerrit.ovirt.org/#/c/88777/ * >>>>>>>> *On 4.2 *branch, the reported change was: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> *core: Call endAction() of all child commands in ImportVmCommand - >>>>>>>> https://gerrit.ovirt.org/#/c/89165/ * >>>>>>>> The fix's for the regressions were merged the following day >>>>>>>> (20-03-2018) >>>>>>>> >>>>>>>> https://gerrit.ovirt.org/#/c/89250/- core: Replace generic >>>>>>>> unlockVm() logic in ImportVmCommand >>>>>>>> https://gerrit.ovirt.org/#/c/89187/ - core: Fix NPE when creating >>>>>>>> an instance type >>>>>>>> >>>>>>>> On 20-03-2018 - the CI team discovered an issue on the job's >>>>>>>> cleanup which caused random failures on changes testing due to failure in >>>>>>>> docker cleanup. There is an open Jira on the issue: >>>>>>>> https://ovirt-jira.atlassian.net/browse/OVIRT-1939 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> *Below you can see the chart for this week's resolved issues but >>>>>>>> cause of failure:*Code = regression of working >>>>>>>> components/functionalities >>>>>>>> Configurations = package related issues >>>>>>>> Other = failed build artifacts >>>>>>>> Infra = infrastructure/OST/Lago related issues >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> *Below is a chart of resolved failures based on ovirt version* >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> *Below is a chart showing failures by suite type: * >>>>>>>> Thank you, >>>>>>>> Dafna >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Infra mailing list >>>>>>>> Infra at ovirt.org >>>>>>>> http://lists.ovirt.org/mailman/listinfo/infra >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> GREG SHEREMETA >>>>>>> >>>>>>> SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX >>>>>>> >>>>>>> Red Hat NA >>>>>>> >>>>>>> >>>>>>> >>>>>>> gshereme at redhat.com IRC: gshereme >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Devel mailing list >>>>>>> Devel at ovirt.org >>>>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Infra mailing list >>>>>> Infra at ovirt.org >>>>>> http://lists.ovirt.org/mailman/listinfo/infra >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Didi >>>>> >>>> >>>> >>>> >>>> -- >>>> Didi >>>> >>> >>> >> >> >> -- >> >> GREG SHEREMETA >> >> SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX >> >> Red Hat NA >> >> >> >> gshereme at redhat.com IRC: gshereme >> >> > > > > -- > > GREG SHEREMETA > > SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX > > Red Hat NA > > > > gshereme at redhat.com IRC: gshereme > > > _______________________________________________ > Infra mailing list > Infra at ovirt.org > http://lists.ovirt.org/mailman/listinfo/infra > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 7163 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 5835 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 27001 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 23605 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 26288 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 6237 bytes Desc: not available URL: From gshereme at redhat.com Mon Mar 26 12:29:49 2018 From: gshereme at redhat.com (Greg Sheremeta) Date: Mon, 26 Mar 2018 08:29:49 -0400 Subject: [ovirt-devel] OST Failure - Weekly update [17/03/2018-23/03/2018] In-Reply-To: References: Message-ID: Eyal, danken told me that ovn cannot require selinux, so the images must be updated. Looks like Gal is updating the images here: https://gerrit.ovirt.org/#/c/89430/ (ng: Update the CentOS image, just merged) Gal, will this fix it? On Sun, Mar 25, 2018 at 1:15 PM, Eyal Edri wrote: > Just wondering, if the bug requires updated selinux, shouldn't the ovs PKG > enforce it I'm the spec file? > > > > On Mar 25, 2018 20:12, "Greg Sheremeta" wrote: > >> Indeed, updating selinux-policy on both engine and hosts passes. >> >> Change If671d938: [test do not merge] test selinux-policy update on >> engine and hosts | gerrit.ovirt Code Review >> https://gerrit.ovirt.org/#/c/89427/ >> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >> -patch-el7-x86_64/4676/consoleFull >> >> So I guess the images need updates. >> >> On Sun, Mar 25, 2018 at 11:20 AM, Greg Sheremeta >> wrote: >> >>> currently selinux-policy-3.13.1-166.el7_4.4.noarch >>> >>> updating selinux-policy on engine gets me past 001, and then 002 dies: >>> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >>> -patch-el7-x86_64/4663/console >>> >>> 15:05:45 # initialize_engine: >>> rpm -qa: >>> 15:05:46 CommandStatus(code=0, out='selinux-policy-3.13.1-166.el7_4.4.noarch\n', >>> err='') >>> 15:06:13 Package selinux-policy.noarch 0:3.13.1-166.el7_4.9 will >>> be an update >>> Updated: selinux-policy.noarch 0:3.13.1-166.el7_4.9 >>> rpm -qa: >>> 15:06:13 CommandStatus(code=0, out='selinux-policy-3.13.1-166.el7_4.9.noarch\n', >>> err='') >>> >>> But later in 002 >>> 15:08:47 RuntimeError: 1 hosts failed installation: >>> 15:08:47 lago-basic-suite-master-host-0: install_failed >>> >>> Perhaps selinux-policy needs to be updated on the hosts too? Not my area >>> of expertise :) >>> >>> Greg >>> >>> On Sun, Mar 25, 2018 at 7:22 AM, Dan Kenigsberg >>> wrote: >>> >>>> Which version of selinux-policy do we have on the Engine image? >>>> >>>> *Bug 1538936* - Open >>>> vSwitch selinux policy needs updating [rhel-7.4.z] >>>> >>>> was fixed in selinux-policy-3.13.1-166.el7_4.9 which is available in >>>> http://mirror.centos.org/centos-7/7/updates/x86_64/Packages/ >>>> selinux-policy-targeted-3.13.1-166.el7_4.9.noarch.rpm >>>> >>>> >>>> On Sun, Mar 25, 2018 at 12:32 PM, Yedidyah Bar David >>>> wrote: >>>> >>>>> On Sun, Mar 25, 2018 at 12:04 PM, Yedidyah Bar David >>>>> wrote: >>>>> >>>>>> basic suite failed for me too. >>>>>> >>>>>> /var/log/messages has[1]: >>>>>> >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open >>>>>> vSwitch Database Unit... >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: >>>>>> System error >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >>>>>> /etc/openvswitch/conf.db does not exist ... (warning). >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating >>>>>> empty database /etc/openvswitch/conf.db runuser: System error >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>> ovsdb-server.service: control process exited, code=exited status=1 >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to >>>>>> start Open vSwitch Database Unit. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Dependency >>>>>> failed for Open vSwitch. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Job >>>>>> openvswitch.service/start failed with result 'dependency'. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Dependency >>>>>> failed for Open vSwitch Forwarding Unit. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Job >>>>>> ovs-vswitchd.service/start failed with result 'dependency'. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit >>>>>> ovsdb-server.service entered failed state. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>> ovsdb-server.service failed. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion >>>>>> failed for Open vSwitch Delete Transient Ports. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>> ovsdb-server.service holdoff time over, scheduling restart. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open >>>>>> vSwitch Database Unit... >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: >>>>>> Removed session 14. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Removed slice >>>>>> User Slice of root. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Stopping User >>>>>> Slice of root. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: >>>>>> System error >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >>>>>> /etc/openvswitch/conf.db does not exist ... (warning). >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating >>>>>> empty database /etc/openvswitch/conf.db runuser: System error >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>> ovsdb-server.service: control process exited, code=exited status=1 >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to >>>>>> start Open vSwitch Database Unit. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit >>>>>> ovsdb-server.service entered failed state. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>> ovsdb-server.service failed. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion >>>>>> failed for Open vSwitch Delete Transient Ports. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>> ovsdb-server.service holdoff time over, scheduling restart. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open >>>>>> vSwitch Database Unit... >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: >>>>>> System error >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >>>>>> /etc/openvswitch/conf.db does not exist ... (warning). >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating >>>>>> empty database /etc/openvswitch/conf.db runuser: System error >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>> ovsdb-server.service: control process exited, code=exited status=1 >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to >>>>>> start Open vSwitch Database Unit. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit >>>>>> ovsdb-server.service entered failed state. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>> ovsdb-server.service failed. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion >>>>>> failed for Open vSwitch Delete Transient Ports. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Created slice >>>>>> User Slice of root. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: New >>>>>> session 17 of user root. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting User >>>>>> Slice of root. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Started >>>>>> Session 17 of user root. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting >>>>>> Session 17 of user root. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine kernel: DCCP: >>>>>> Activated CCID 2 (TCP-like) >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine kernel: DCCP: >>>>>> Activated CCID 3 (TCP-Friendly Rate Control) >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>> ovsdb-server.service holdoff time over, scheduling restart. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting Open >>>>>> vSwitch Database Unit... >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine kernel: sctp: Hash >>>>>> tables configured (bind 256/256) >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: >>>>>> Removed session 17. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Removed slice >>>>>> User Slice of root. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Stopping User >>>>>> Slice of root. >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: >>>>>> System error >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >>>>>> /etc/openvswitch/conf.db does not exist ... (warning). >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating >>>>>> empty database /etc/openvswitch/conf.db runuser: System error >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>> ovsdb-server.service: control process exited, code=exited status=1 >>>>>> >>>>>> [1] http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >>>>>> -patch-el7-x86_64/4651/artifact/exported-artifacts/basic-sui >>>>>> te-master__logs/test_logs/basic-suite-master/post-001_initia >>>>>> lize_engine.py/lago-basic-suite-master-engine/_var_log/messa >>>>>> ges/*view*/ >>>>>> >>>>>> >>>>> Talked with danken, he asked to check if it's an selinux issue. It is. >>>>> audit lot has: >>>>> >>>>> type=AVC msg=audit(1521967325.146:675): avc: denied { create } for pid=3787 comm="runuser" scontext=system_u:system_r:openvswitch_t:s0 tcontext=system_u:system_r:openvswitch_t:s0 tclass=netlink_audit_socket >>>>> type=SYSCALL msg=audit(1521967325.146:675): arch=c000003e syscall=41 success=no exit=-13 a0=10 a1=3 a2=9 a3=7ffc4e12b930 items=0 ppid=3786 pid=3787 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="runuser" exe="/usr/sbin/runuser" subj=system_u:system_r:openvswitch_t:s0 key=(null) >>>>> type=PROCTITLE msg=audit(1521967325.146:675): proctitle=72756E75736572002D2D75736572006F70656E76737769746368002D2D006F767364622D746F6F6C002D76636F6E736F6C653A6F666600736368656D612D76657273696F6E002F7573722F73686172652F6F70656E767377697463682F767377697463682E6F7673736368656D61 >>>>> type=AVC msg=audit(1521967325.150:676): avc: denied { create } for pid=3789 comm="runuser" scontext=system_u:system_r:openvswitch_t:s0 tcontext=system_u:system_r:openvswitch_t:s0 tclass=netlink_audit_socket >>>>> type=SYSCALL msg=audit(1521967325.150:676): arch=c000003e syscall=41 success=no exit=-13 a0=10 a1=3 a2=9 a3=7ffe03060130 items=0 ppid=3755 pid=3789 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="runuser" exe="/usr/sbin/runuser" subj=system_u:system_r:openvswitch_t:s0 key=(null) >>>>> type=PROCTITLE msg=audit(1521967325.150:676): >>>>> >>>>> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x86_64/4651/artifact/exported-artifacts/basic-suite-master__logs/test_logs/basic-suite-master/post-001_initialize_engine.py/lago-basic-suite-master-engine/_var_log/audit/audit.log >>>>> >>>>> And it's 2.9: >>>>> >>>>> Mar 25 04:38:39 lago-basic-suite-master-engine yum[1183]: Installed: 1:python2-openvswitch-2.9.0-3.el7.noarch >>>>> >>>>> >>>>> >>>>>> >>>>>> On Sun, Mar 25, 2018 at 10:07 AM, Yaniv Kaul >>>>>> wrote: >>>>>> >>>>>>> + Network team. >>>>>>> I'm not sure if we've moved to ovs 2.9 already? >>>>>>> Y. >>>>>>> >>>>>>> On Sat, Mar 24, 2018 at 8:19 PM, Greg Sheremeta >>>>>> > wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> Is there an ongoing engine master OST failure blocking? >>>>>>>> >>>>>>>> [ INFO ] Stage: Misc configuration >>>>>>>> [ INFO ] Stage: Package installation >>>>>>>> [ INFO ] Stage: Misc configuration >>>>>>>> [ ERROR ] Failed to execute stage \'Misc configuration\': Failed to >>>>>>>> start service \'openvswitch\' >>>>>>>> [ INFO ] Yum Performing yum transaction rollback >>>>>>>> >>>>>>>> >>>>>>>> These are unrelated code changes: >>>>>>>> >>>>>>>> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >>>>>>>> -patch-el7-x86_64/4644/ >>>>>>>> https://gerrit.ovirt.org/#/c/89347/ >>>>>>>> >>>>>>>> and >>>>>>>> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >>>>>>>> -patch-el7-x86_64/4647/ >>>>>>>> https://gerrit.ovirt.org/67166 >>>>>>>> >>>>>>>> But they both die in 001, with exactly 1.24MB in the log and 'Failed >>>>>>>> to start service openvswitch' >>>>>>>> 001_initialize_engine.py.junit.xml 1.24 MB >>>>>>>> >>>>>>>> Full file: http://jenkins.ovirt.org/job/ovirt-system-tests_master >>>>>>>> _check-patch-el7-x86_64/4644/artifact/exported-artifacts/bas >>>>>>>> ic-suite-master__logs/001_initialize_engine.py.junit.xml >>>>>>>> >>>>>>>> >>>>>>>> On Fri, Mar 23, 2018 at 12:14 PM, Dafna Ron >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> I would like to update on this week's failures and OST current >>>>>>>>> status. >>>>>>>>> >>>>>>>>> On 19-03-2018 - the CI team reported 3 different failures. >>>>>>>>> >>>>>>>>> On Master branch the failed changes reported were: >>>>>>>>> >>>>>>>>> >>>>>>>>> *core: fix removal of vm-host device - >>>>>>>>> https://gerrit.ovirt.org/#/c/89145/ * >>>>>>>>> >>>>>>>>> *core: USB in osinfo configuration depends on chipset - >>>>>>>>> https://gerrit.ovirt.org/#/c/88777/ * >>>>>>>>> *On 4.2 *branch, the reported change was: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> *core: Call endAction() of all child commands in ImportVmCommand - >>>>>>>>> https://gerrit.ovirt.org/#/c/89165/ * >>>>>>>>> The fix's for the regressions were merged the following day >>>>>>>>> (20-03-2018) >>>>>>>>> >>>>>>>>> https://gerrit.ovirt.org/#/c/89250/- core: Replace generic >>>>>>>>> unlockVm() logic in ImportVmCommand >>>>>>>>> https://gerrit.ovirt.org/#/c/89187/ - core: Fix NPE when creating >>>>>>>>> an instance type >>>>>>>>> >>>>>>>>> On 20-03-2018 - the CI team discovered an issue on the job's >>>>>>>>> cleanup which caused random failures on changes testing due to failure in >>>>>>>>> docker cleanup. There is an open Jira on the issue: >>>>>>>>> https://ovirt-jira.atlassian.net/browse/OVIRT-1939 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> *Below you can see the chart for this week's resolved issues but >>>>>>>>> cause of failure:*Code = regression of working >>>>>>>>> components/functionalities >>>>>>>>> Configurations = package related issues >>>>>>>>> Other = failed build artifacts >>>>>>>>> Infra = infrastructure/OST/Lago related issues >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> *Below is a chart of resolved failures based on ovirt version* >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> *Below is a chart showing failures by suite type: * >>>>>>>>> Thank you, >>>>>>>>> Dafna >>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Infra mailing list >>>>>>>>> Infra at ovirt.org >>>>>>>>> http://lists.ovirt.org/mailman/listinfo/infra >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> GREG SHEREMETA >>>>>>>> >>>>>>>> SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX >>>>>>>> >>>>>>>> Red Hat NA >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> gshereme at redhat.com IRC: gshereme >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Devel mailing list >>>>>>>> Devel at ovirt.org >>>>>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Infra mailing list >>>>>>> Infra at ovirt.org >>>>>>> http://lists.ovirt.org/mailman/listinfo/infra >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Didi >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Didi >>>>> >>>> >>>> >>> >>> >>> -- >>> >>> GREG SHEREMETA >>> >>> SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX >>> >>> Red Hat NA >>> >>> >>> >>> gshereme at redhat.com IRC: gshereme >>> >>> >> >> >> >> -- >> >> GREG SHEREMETA >> >> SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX >> >> Red Hat NA >> >> >> >> gshereme at redhat.com IRC: gshereme >> >> >> _______________________________________________ >> Infra mailing list >> Infra at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/infra >> >> -- GREG SHEREMETA SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX Red Hat NA gshereme at redhat.com IRC: gshereme -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 23605 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 6237 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 5835 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 7163 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 26288 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 27001 bytes Desc: not available URL: From eedri at redhat.com Mon Mar 26 12:32:57 2018 From: eedri at redhat.com (Eyal Edri) Date: Mon, 26 Mar 2018 15:32:57 +0300 Subject: [ovirt-devel] OST Failure - Weekly update [17/03/2018-23/03/2018] In-Reply-To: References: Message-ID: On Mon, Mar 26, 2018 at 3:29 PM, Greg Sheremeta wrote: > Eyal, danken told me that ovn cannot require selinux, so the images must > be updated. > Yea, I got that, and Gal indeed went a head and updated the image, just wondered why it wasn't fixed also on OVN side. > > Looks like Gal is updating the images here: https://gerrit.ovirt. > org/#/c/89430/ (ng: Update the CentOS image, just merged) > > Gal, will this fix it? > I think Gal verified it, so should be working now, try rebasing your patch. > > > On Sun, Mar 25, 2018 at 1:15 PM, Eyal Edri wrote: > >> Just wondering, if the bug requires updated selinux, shouldn't the ovs >> PKG enforce it I'm the spec file? >> >> >> >> On Mar 25, 2018 20:12, "Greg Sheremeta" wrote: >> >>> Indeed, updating selinux-policy on both engine and hosts passes. >>> >>> Change If671d938: [test do not merge] test selinux-policy update on >>> engine and hosts | gerrit.ovirt Code Review >>> https://gerrit.ovirt.org/#/c/89427/ >>> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >>> -patch-el7-x86_64/4676/consoleFull >>> >>> So I guess the images need updates. >>> >>> On Sun, Mar 25, 2018 at 11:20 AM, Greg Sheremeta >>> wrote: >>> >>>> currently selinux-policy-3.13.1-166.el7_4.4.noarch >>>> >>>> updating selinux-policy on engine gets me past 001, and then 002 dies: >>>> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >>>> -patch-el7-x86_64/4663/console >>>> >>>> 15:05:45 # initialize_engine: >>>> rpm -qa: >>>> 15:05:46 CommandStatus(code=0, out='selinux-policy-3.13.1-166.el7_4.4.noarch\n', >>>> err='') >>>> 15:06:13 Package selinux-policy.noarch 0:3.13.1-166.el7_4.9 will >>>> be an update >>>> Updated: selinux-policy.noarch 0:3.13.1-166.el7_4.9 >>>> rpm -qa: >>>> 15:06:13 CommandStatus(code=0, out='selinux-policy-3.13.1-166.el7_4.9.noarch\n', >>>> err='') >>>> >>>> But later in 002 >>>> 15:08:47 RuntimeError: 1 hosts failed installation: >>>> 15:08:47 lago-basic-suite-master-host-0: install_failed >>>> >>>> Perhaps selinux-policy needs to be updated on the hosts too? Not my >>>> area of expertise :) >>>> >>>> Greg >>>> >>>> On Sun, Mar 25, 2018 at 7:22 AM, Dan Kenigsberg >>>> wrote: >>>> >>>>> Which version of selinux-policy do we have on the Engine image? >>>>> >>>>> *Bug 1538936* - Open >>>>> vSwitch selinux policy needs updating [rhel-7.4.z] >>>>> >>>>> was fixed in selinux-policy-3.13.1-166.el7_4.9 which is available in >>>>> http://mirror.centos.org/centos-7/7/updates/x86_64/Packages/ >>>>> selinux-policy-targeted-3.13.1-166.el7_4.9.noarch.rpm >>>>> >>>>> >>>>> On Sun, Mar 25, 2018 at 12:32 PM, Yedidyah Bar David >>>>> wrote: >>>>> >>>>>> On Sun, Mar 25, 2018 at 12:04 PM, Yedidyah Bar David >>>>> > wrote: >>>>>> >>>>>>> basic suite failed for me too. >>>>>>> >>>>>>> /var/log/messages has[1]: >>>>>>> >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting >>>>>>> Open vSwitch Database Unit... >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: >>>>>>> System error >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >>>>>>> /etc/openvswitch/conf.db does not exist ... (warning). >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating >>>>>>> empty database /etc/openvswitch/conf.db runuser: System error >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>>> ovsdb-server.service: control process exited, code=exited status=1 >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to >>>>>>> start Open vSwitch Database Unit. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Dependency >>>>>>> failed for Open vSwitch. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Job >>>>>>> openvswitch.service/start failed with result 'dependency'. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Dependency >>>>>>> failed for Open vSwitch Forwarding Unit. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Job >>>>>>> ovs-vswitchd.service/start failed with result 'dependency'. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit >>>>>>> ovsdb-server.service entered failed state. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>>> ovsdb-server.service failed. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion >>>>>>> failed for Open vSwitch Delete Transient Ports. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>>> ovsdb-server.service holdoff time over, scheduling restart. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting >>>>>>> Open vSwitch Database Unit... >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: >>>>>>> Removed session 14. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Removed >>>>>>> slice User Slice of root. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Stopping >>>>>>> User Slice of root. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: >>>>>>> System error >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >>>>>>> /etc/openvswitch/conf.db does not exist ... (warning). >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating >>>>>>> empty database /etc/openvswitch/conf.db runuser: System error >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>>> ovsdb-server.service: control process exited, code=exited status=1 >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to >>>>>>> start Open vSwitch Database Unit. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit >>>>>>> ovsdb-server.service entered failed state. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>>> ovsdb-server.service failed. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion >>>>>>> failed for Open vSwitch Delete Transient Ports. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>>> ovsdb-server.service holdoff time over, scheduling restart. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting >>>>>>> Open vSwitch Database Unit... >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: >>>>>>> System error >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >>>>>>> /etc/openvswitch/conf.db does not exist ... (warning). >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating >>>>>>> empty database /etc/openvswitch/conf.db runuser: System error >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>>> ovsdb-server.service: control process exited, code=exited status=1 >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to >>>>>>> start Open vSwitch Database Unit. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit >>>>>>> ovsdb-server.service entered failed state. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>>> ovsdb-server.service failed. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion >>>>>>> failed for Open vSwitch Delete Transient Ports. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Created >>>>>>> slice User Slice of root. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: New >>>>>>> session 17 of user root. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting >>>>>>> User Slice of root. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Started >>>>>>> Session 17 of user root. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting >>>>>>> Session 17 of user root. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine kernel: DCCP: >>>>>>> Activated CCID 2 (TCP-like) >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine kernel: DCCP: >>>>>>> Activated CCID 3 (TCP-Friendly Rate Control) >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>>> ovsdb-server.service holdoff time over, scheduling restart. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting >>>>>>> Open vSwitch Database Unit... >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine kernel: sctp: Hash >>>>>>> tables configured (bind 256/256) >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: >>>>>>> Removed session 17. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Removed >>>>>>> slice User Slice of root. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Stopping >>>>>>> User Slice of root. >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: >>>>>>> System error >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >>>>>>> /etc/openvswitch/conf.db does not exist ... (warning). >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating >>>>>>> empty database /etc/openvswitch/conf.db runuser: System error >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>>> ovsdb-server.service: control process exited, code=exited status=1 >>>>>>> >>>>>>> [1] http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >>>>>>> -patch-el7-x86_64/4651/artifact/exported-artifacts/basic-sui >>>>>>> te-master__logs/test_logs/basic-suite-master/post-001_initia >>>>>>> lize_engine.py/lago-basic-suite-master-engine/_var_log/messa >>>>>>> ges/*view*/ >>>>>>> >>>>>>> >>>>>> Talked with danken, he asked to check if it's an selinux issue. It >>>>>> is. audit lot has: >>>>>> >>>>>> type=AVC msg=audit(1521967325.146:675): avc: denied { create } for pid=3787 comm="runuser" scontext=system_u:system_r:openvswitch_t:s0 tcontext=system_u:system_r:openvswitch_t:s0 tclass=netlink_audit_socket >>>>>> type=SYSCALL msg=audit(1521967325.146:675): arch=c000003e syscall=41 success=no exit=-13 a0=10 a1=3 a2=9 a3=7ffc4e12b930 items=0 ppid=3786 pid=3787 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="runuser" exe="/usr/sbin/runuser" subj=system_u:system_r:openvswitch_t:s0 key=(null) >>>>>> type=PROCTITLE msg=audit(1521967325.146:675): proctitle=72756E75736572002D2D75736572006F70656E76737769746368002D2D006F767364622D746F6F6C002D76636F6E736F6C653A6F666600736368656D612D76657273696F6E002F7573722F73686172652F6F70656E767377697463682F767377697463682E6F7673736368656D61 >>>>>> type=AVC msg=audit(1521967325.150:676): avc: denied { create } for pid=3789 comm="runuser" scontext=system_u:system_r:openvswitch_t:s0 tcontext=system_u:system_r:openvswitch_t:s0 tclass=netlink_audit_socket >>>>>> type=SYSCALL msg=audit(1521967325.150:676): arch=c000003e syscall=41 success=no exit=-13 a0=10 a1=3 a2=9 a3=7ffe03060130 items=0 ppid=3755 pid=3789 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="runuser" exe="/usr/sbin/runuser" subj=system_u:system_r:openvswitch_t:s0 key=(null) >>>>>> type=PROCTITLE msg=audit(1521967325.150:676): >>>>>> >>>>>> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x86_64/4651/artifact/exported-artifacts/basic-suite-master__logs/test_logs/basic-suite-master/post-001_initialize_engine.py/lago-basic-suite-master-engine/_var_log/audit/audit.log >>>>>> >>>>>> And it's 2.9: >>>>>> >>>>>> Mar 25 04:38:39 lago-basic-suite-master-engine yum[1183]: Installed: 1:python2-openvswitch-2.9.0-3.el7.noarch >>>>>> >>>>>> >>>>>> >>>>>>> >>>>>>> On Sun, Mar 25, 2018 at 10:07 AM, Yaniv Kaul >>>>>>> wrote: >>>>>>> >>>>>>>> + Network team. >>>>>>>> I'm not sure if we've moved to ovs 2.9 already? >>>>>>>> Y. >>>>>>>> >>>>>>>> On Sat, Mar 24, 2018 at 8:19 PM, Greg Sheremeta < >>>>>>>> gshereme at redhat.com> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Is there an ongoing engine master OST failure blocking? >>>>>>>>> >>>>>>>>> [ INFO ] Stage: Misc configuration >>>>>>>>> [ INFO ] Stage: Package installation >>>>>>>>> [ INFO ] Stage: Misc configuration >>>>>>>>> [ ERROR ] Failed to execute stage \'Misc configuration\': Failed >>>>>>>>> to start service \'openvswitch\' >>>>>>>>> [ INFO ] Yum Performing yum transaction rollback >>>>>>>>> >>>>>>>>> >>>>>>>>> These are unrelated code changes: >>>>>>>>> >>>>>>>>> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >>>>>>>>> -patch-el7-x86_64/4644/ >>>>>>>>> https://gerrit.ovirt.org/#/c/89347/ >>>>>>>>> >>>>>>>>> and >>>>>>>>> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >>>>>>>>> -patch-el7-x86_64/4647/ >>>>>>>>> https://gerrit.ovirt.org/67166 >>>>>>>>> >>>>>>>>> But they both die in 001, with exactly 1.24MB in the log and 'Failed >>>>>>>>> to start service openvswitch' >>>>>>>>> 001_initialize_engine.py.junit.xml 1.24 MB >>>>>>>>> >>>>>>>>> Full file: http://jenkins.ovirt.org/job/ovirt-system-tests_master >>>>>>>>> _check-patch-el7-x86_64/4644/artifact/exported-artifacts/bas >>>>>>>>> ic-suite-master__logs/001_initialize_engine.py.junit.xml >>>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, Mar 23, 2018 at 12:14 PM, Dafna Ron >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hello, >>>>>>>>>> >>>>>>>>>> I would like to update on this week's failures and OST current >>>>>>>>>> status. >>>>>>>>>> >>>>>>>>>> On 19-03-2018 - the CI team reported 3 different failures. >>>>>>>>>> >>>>>>>>>> On Master branch the failed changes reported were: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> *core: fix removal of vm-host device - >>>>>>>>>> https://gerrit.ovirt.org/#/c/89145/ * >>>>>>>>>> >>>>>>>>>> *core: USB in osinfo configuration depends on chipset - >>>>>>>>>> https://gerrit.ovirt.org/#/c/88777/ * >>>>>>>>>> *On 4.2 *branch, the reported change was: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> *core: Call endAction() of all child commands in ImportVmCommand >>>>>>>>>> - https://gerrit.ovirt.org/#/c/89165/ * >>>>>>>>>> The fix's for the regressions were merged the following day >>>>>>>>>> (20-03-2018) >>>>>>>>>> >>>>>>>>>> https://gerrit.ovirt.org/#/c/89250/- core: Replace generic >>>>>>>>>> unlockVm() logic in ImportVmCommand >>>>>>>>>> https://gerrit.ovirt.org/#/c/89187/ - core: Fix NPE when >>>>>>>>>> creating an instance type >>>>>>>>>> >>>>>>>>>> On 20-03-2018 - the CI team discovered an issue on the job's >>>>>>>>>> cleanup which caused random failures on changes testing due to failure in >>>>>>>>>> docker cleanup. There is an open Jira on the issue: >>>>>>>>>> https://ovirt-jira.atlassian.net/browse/OVIRT-1939 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> *Below you can see the chart for this week's resolved issues but >>>>>>>>>> cause of failure:*Code = regression of working >>>>>>>>>> components/functionalities >>>>>>>>>> Configurations = package related issues >>>>>>>>>> Other = failed build artifacts >>>>>>>>>> Infra = infrastructure/OST/Lago related issues >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> *Below is a chart of resolved failures based on ovirt version* >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> *Below is a chart showing failures by suite type: * >>>>>>>>>> Thank you, >>>>>>>>>> Dafna >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Infra mailing list >>>>>>>>>> Infra at ovirt.org >>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/infra >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> >>>>>>>>> GREG SHEREMETA >>>>>>>>> >>>>>>>>> SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX >>>>>>>>> >>>>>>>>> Red Hat NA >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> gshereme at redhat.com IRC: gshereme >>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Devel mailing list >>>>>>>>> Devel at ovirt.org >>>>>>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Infra mailing list >>>>>>>> Infra at ovirt.org >>>>>>>> http://lists.ovirt.org/mailman/listinfo/infra >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Didi >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Didi >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> GREG SHEREMETA >>>> >>>> SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX >>>> >>>> Red Hat NA >>>> >>>> >>>> >>>> gshereme at redhat.com IRC: gshereme >>>> >>>> >>> >>> >>> >>> -- >>> >>> GREG SHEREMETA >>> >>> SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX >>> >>> Red Hat NA >>> >>> >>> >>> gshereme at redhat.com IRC: gshereme >>> >>> >>> _______________________________________________ >>> Infra mailing list >>> Infra at ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/infra >>> >>> > > > -- > > GREG SHEREMETA > > SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX > > Red Hat NA > > > > gshereme at redhat.com IRC: gshereme > > -- Eyal edri MANAGER RHV DevOps EMEA VIRTUALIZATION R&D Red Hat EMEA TRIED. TESTED. TRUSTED. phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ) -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 7163 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 26288 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 5835 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 6237 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 27001 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 23605 bytes Desc: not available URL: From danken at redhat.com Mon Mar 26 12:52:47 2018 From: danken at redhat.com (Dan Kenigsberg) Date: Mon, 26 Mar 2018 15:52:47 +0300 Subject: [ovirt-devel] OST Failure - Weekly update [17/03/2018-23/03/2018] In-Reply-To: References: Message-ID: On Mon, Mar 26, 2018 at 3:32 PM, Eyal Edri wrote: > > > On Mon, Mar 26, 2018 at 3:29 PM, Greg Sheremeta > wrote: > >> Eyal, danken told me that ovn cannot require selinux, so the images must >> be updated. >> > > Yea, I got that, and Gal indeed went a head and updated the image, just > wondered why it wasn't fixed also on OVN side. > They are. https://bugzilla.redhat.com/show_bug.cgi?id=1549673 fixes this in ovs-2.9.0-6.el7fdn but that is not released yet. > >> >> Looks like Gal is updating the images here: https://gerrit.ovirt.org >> /#/c/89430/ (ng: Update the CentOS image, just merged) >> >> Gal, will this fix it? >> > > I think Gal verified it, so should be working now, try rebasing your patch. > > >> >> >> On Sun, Mar 25, 2018 at 1:15 PM, Eyal Edri wrote: >> >>> Just wondering, if the bug requires updated selinux, shouldn't the ovs >>> PKG enforce it I'm the spec file? >>> >>> >>> >>> On Mar 25, 2018 20:12, "Greg Sheremeta" wrote: >>> >>>> Indeed, updating selinux-policy on both engine and hosts passes. >>>> >>>> Change If671d938: [test do not merge] test selinux-policy update on >>>> engine and hosts | gerrit.ovirt Code Review >>>> https://gerrit.ovirt.org/#/c/89427/ >>>> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >>>> -patch-el7-x86_64/4676/consoleFull >>>> >>>> So I guess the images need updates. >>>> >>>> On Sun, Mar 25, 2018 at 11:20 AM, Greg Sheremeta >>>> wrote: >>>> >>>>> currently selinux-policy-3.13.1-166.el7_4.4.noarch >>>>> >>>>> updating selinux-policy on engine gets me past 001, and then 002 dies: >>>>> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >>>>> -patch-el7-x86_64/4663/console >>>>> >>>>> 15:05:45 # initialize_engine: >>>>> rpm -qa: >>>>> 15:05:46 CommandStatus(code=0, out='selinux-policy-3.13.1-166.el7_4.4.noarch\n', >>>>> err='') >>>>> 15:06:13 Package selinux-policy.noarch 0:3.13.1-166.el7_4.9 >>>>> will be an update >>>>> Updated: selinux-policy.noarch 0:3.13.1-166.el7_4.9 >>>>> rpm -qa: >>>>> 15:06:13 CommandStatus(code=0, out='selinux-policy-3.13.1-166.el7_4.9.noarch\n', >>>>> err='') >>>>> >>>>> But later in 002 >>>>> 15:08:47 RuntimeError: 1 hosts failed installation: >>>>> 15:08:47 lago-basic-suite-master-host-0: install_failed >>>>> >>>>> Perhaps selinux-policy needs to be updated on the hosts too? Not my >>>>> area of expertise :) >>>>> >>>>> Greg >>>>> >>>>> On Sun, Mar 25, 2018 at 7:22 AM, Dan Kenigsberg >>>>> wrote: >>>>> >>>>>> Which version of selinux-policy do we have on the Engine image? >>>>>> >>>>>> *Bug 1538936* - Open >>>>>> vSwitch selinux policy needs updating [rhel-7.4.z] >>>>>> >>>>>> was fixed in selinux-policy-3.13.1-166.el7_4.9 which is available in >>>>>> http://mirror.centos.org/centos-7/7/updates/x86_64/Packages/ >>>>>> selinux-policy-targeted-3.13.1-166.el7_4.9.noarch.rpm >>>>>> >>>>>> >>>>>> On Sun, Mar 25, 2018 at 12:32 PM, Yedidyah Bar David >>>>> > wrote: >>>>>> >>>>>>> On Sun, Mar 25, 2018 at 12:04 PM, Yedidyah Bar David < >>>>>>> didi at redhat.com> wrote: >>>>>>> >>>>>>>> basic suite failed for me too. >>>>>>>> >>>>>>>> /var/log/messages has[1]: >>>>>>>> >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting >>>>>>>> Open vSwitch Database Unit... >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: >>>>>>>> System error >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >>>>>>>> /etc/openvswitch/conf.db does not exist ... (warning). >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating >>>>>>>> empty database /etc/openvswitch/conf.db runuser: System error >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>>>> ovsdb-server.service: control process exited, code=exited status=1 >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to >>>>>>>> start Open vSwitch Database Unit. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Dependency >>>>>>>> failed for Open vSwitch. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Job >>>>>>>> openvswitch.service/start failed with result 'dependency'. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Dependency >>>>>>>> failed for Open vSwitch Forwarding Unit. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Job >>>>>>>> ovs-vswitchd.service/start failed with result 'dependency'. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit >>>>>>>> ovsdb-server.service entered failed state. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>>>> ovsdb-server.service failed. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion >>>>>>>> failed for Open vSwitch Delete Transient Ports. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>>>> ovsdb-server.service holdoff time over, scheduling restart. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting >>>>>>>> Open vSwitch Database Unit... >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: >>>>>>>> Removed session 14. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Removed >>>>>>>> slice User Slice of root. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Stopping >>>>>>>> User Slice of root. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: >>>>>>>> System error >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >>>>>>>> /etc/openvswitch/conf.db does not exist ... (warning). >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating >>>>>>>> empty database /etc/openvswitch/conf.db runuser: System error >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>>>> ovsdb-server.service: control process exited, code=exited status=1 >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to >>>>>>>> start Open vSwitch Database Unit. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit >>>>>>>> ovsdb-server.service entered failed state. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>>>> ovsdb-server.service failed. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion >>>>>>>> failed for Open vSwitch Delete Transient Ports. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>>>> ovsdb-server.service holdoff time over, scheduling restart. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting >>>>>>>> Open vSwitch Database Unit... >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: >>>>>>>> System error >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >>>>>>>> /etc/openvswitch/conf.db does not exist ... (warning). >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating >>>>>>>> empty database /etc/openvswitch/conf.db runuser: System error >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>>>> ovsdb-server.service: control process exited, code=exited status=1 >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Failed to >>>>>>>> start Open vSwitch Database Unit. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Unit >>>>>>>> ovsdb-server.service entered failed state. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>>>> ovsdb-server.service failed. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Assertion >>>>>>>> failed for Open vSwitch Delete Transient Ports. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Created >>>>>>>> slice User Slice of root. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: New >>>>>>>> session 17 of user root. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting >>>>>>>> User Slice of root. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Started >>>>>>>> Session 17 of user root. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting >>>>>>>> Session 17 of user root. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine kernel: DCCP: >>>>>>>> Activated CCID 2 (TCP-like) >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine kernel: DCCP: >>>>>>>> Activated CCID 3 (TCP-Friendly Rate Control) >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>>>> ovsdb-server.service holdoff time over, scheduling restart. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Starting >>>>>>>> Open vSwitch Database Unit... >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine kernel: sctp: Hash >>>>>>>> tables configured (bind 256/256) >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd-logind: >>>>>>>> Removed session 17. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Removed >>>>>>>> slice User Slice of root. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: Stopping >>>>>>>> User Slice of root. >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: runuser: >>>>>>>> System error >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: >>>>>>>> /etc/openvswitch/conf.db does not exist ... (warning). >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: Creating >>>>>>>> empty database /etc/openvswitch/conf.db runuser: System error >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine ovs-ctl: [FAILED] >>>>>>>> Mar 25 04:42:05 lago-basic-suite-master-engine systemd: >>>>>>>> ovsdb-server.service: control process exited, code=exited status=1 >>>>>>>> >>>>>>>> [1] http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >>>>>>>> -patch-el7-x86_64/4651/artifact/exported-artifacts/basic-sui >>>>>>>> te-master__logs/test_logs/basic-suite-master/post-001_initia >>>>>>>> lize_engine.py/lago-basic-suite-master-engine/_var_log/messa >>>>>>>> ges/*view*/ >>>>>>>> >>>>>>>> >>>>>>> Talked with danken, he asked to check if it's an selinux issue. It >>>>>>> is. audit lot has: >>>>>>> >>>>>>> type=AVC msg=audit(1521967325.146:675): avc: denied { create } for pid=3787 comm="runuser" scontext=system_u:system_r:openvswitch_t:s0 tcontext=system_u:system_r:openvswitch_t:s0 tclass=netlink_audit_socket >>>>>>> type=SYSCALL msg=audit(1521967325.146:675): arch=c000003e syscall=41 success=no exit=-13 a0=10 a1=3 a2=9 a3=7ffc4e12b930 items=0 ppid=3786 pid=3787 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="runuser" exe="/usr/sbin/runuser" subj=system_u:system_r:openvswitch_t:s0 key=(null) >>>>>>> type=PROCTITLE msg=audit(1521967325.146:675): proctitle=72756E75736572002D2D75736572006F70656E76737769746368002D2D006F767364622D746F6F6C002D76636F6E736F6C653A6F666600736368656D612D76657273696F6E002F7573722F73686172652F6F70656E767377697463682F767377697463682E6F7673736368656D61 >>>>>>> type=AVC msg=audit(1521967325.150:676): avc: denied { create } for pid=3789 comm="runuser" scontext=system_u:system_r:openvswitch_t:s0 tcontext=system_u:system_r:openvswitch_t:s0 tclass=netlink_audit_socket >>>>>>> type=SYSCALL msg=audit(1521967325.150:676): arch=c000003e syscall=41 success=no exit=-13 a0=10 a1=3 a2=9 a3=7ffe03060130 items=0 ppid=3755 pid=3789 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="runuser" exe="/usr/sbin/runuser" subj=system_u:system_r:openvswitch_t:s0 key=(null) >>>>>>> type=PROCTITLE msg=audit(1521967325.150:676): >>>>>>> >>>>>>> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x86_64/4651/artifact/exported-artifacts/basic-suite-master__logs/test_logs/basic-suite-master/post-001_initialize_engine.py/lago-basic-suite-master-engine/_var_log/audit/audit.log >>>>>>> >>>>>>> And it's 2.9: >>>>>>> >>>>>>> Mar 25 04:38:39 lago-basic-suite-master-engine yum[1183]: Installed: 1:python2-openvswitch-2.9.0-3.el7.noarch >>>>>>> >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> On Sun, Mar 25, 2018 at 10:07 AM, Yaniv Kaul >>>>>>>> wrote: >>>>>>>> >>>>>>>>> + Network team. >>>>>>>>> I'm not sure if we've moved to ovs 2.9 already? >>>>>>>>> Y. >>>>>>>>> >>>>>>>>> On Sat, Mar 24, 2018 at 8:19 PM, Greg Sheremeta < >>>>>>>>> gshereme at redhat.com> wrote: >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> Is there an ongoing engine master OST failure blocking? >>>>>>>>>> >>>>>>>>>> [ INFO ] Stage: Misc configuration >>>>>>>>>> [ INFO ] Stage: Package installation >>>>>>>>>> [ INFO ] Stage: Misc configuration >>>>>>>>>> [ ERROR ] Failed to execute stage \'Misc configuration\': Failed >>>>>>>>>> to start service \'openvswitch\' >>>>>>>>>> [ INFO ] Yum Performing yum transaction rollback >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> These are unrelated code changes: >>>>>>>>>> >>>>>>>>>> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >>>>>>>>>> -patch-el7-x86_64/4644/ >>>>>>>>>> https://gerrit.ovirt.org/#/c/89347/ >>>>>>>>>> >>>>>>>>>> and >>>>>>>>>> http://jenkins.ovirt.org/job/ovirt-system-tests_master_check >>>>>>>>>> -patch-el7-x86_64/4647/ >>>>>>>>>> https://gerrit.ovirt.org/67166 >>>>>>>>>> >>>>>>>>>> But they both die in 001, with exactly 1.24MB in the log and 'Failed >>>>>>>>>> to start service openvswitch' >>>>>>>>>> 001_initialize_engine.py.junit.xml 1.24 MB >>>>>>>>>> >>>>>>>>>> Full file: http://jenkins.ovirt.org/job/ovirt-system-tests_master >>>>>>>>>> _check-patch-el7-x86_64/4644/artifact/exported-artifacts/bas >>>>>>>>>> ic-suite-master__logs/001_initialize_engine.py.junit.xml >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Fri, Mar 23, 2018 at 12:14 PM, Dafna Ron >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hello, >>>>>>>>>>> >>>>>>>>>>> I would like to update on this week's failures and OST current >>>>>>>>>>> status. >>>>>>>>>>> >>>>>>>>>>> On 19-03-2018 - the CI team reported 3 different failures. >>>>>>>>>>> >>>>>>>>>>> On Master branch the failed changes reported were: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> *core: fix removal of vm-host device - >>>>>>>>>>> https://gerrit.ovirt.org/#/c/89145/ * >>>>>>>>>>> >>>>>>>>>>> *core: USB in osinfo configuration depends on chipset - >>>>>>>>>>> https://gerrit.ovirt.org/#/c/88777/ * >>>>>>>>>>> *On 4.2 *branch, the reported change was: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> *core: Call endAction() of all child commands in ImportVmCommand >>>>>>>>>>> - https://gerrit.ovirt.org/#/c/89165/ * >>>>>>>>>>> The fix's for the regressions were merged the following day >>>>>>>>>>> (20-03-2018) >>>>>>>>>>> >>>>>>>>>>> https://gerrit.ovirt.org/#/c/89250/- core: Replace generic >>>>>>>>>>> unlockVm() logic in ImportVmCommand >>>>>>>>>>> https://gerrit.ovirt.org/#/c/89187/ - core: Fix NPE when >>>>>>>>>>> creating an instance type >>>>>>>>>>> >>>>>>>>>>> On 20-03-2018 - the CI team discovered an issue on the job's >>>>>>>>>>> cleanup which caused random failures on changes testing due to failure in >>>>>>>>>>> docker cleanup. There is an open Jira on the issue: >>>>>>>>>>> https://ovirt-jira.atlassian.net/browse/OVIRT-1939 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> *Below you can see the chart for this week's resolved issues but >>>>>>>>>>> cause of failure:*Code = regression of working >>>>>>>>>>> components/functionalities >>>>>>>>>>> Configurations = package related issues >>>>>>>>>>> Other = failed build artifacts >>>>>>>>>>> Infra = infrastructure/OST/Lago related issues >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> *Below is a chart of resolved failures based on ovirt version* >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> *Below is a chart showing failures by suite type: * >>>>>>>>>>> Thank you, >>>>>>>>>>> Dafna >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Infra mailing list >>>>>>>>>>> Infra at ovirt.org >>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/infra >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> >>>>>>>>>> GREG SHEREMETA >>>>>>>>>> >>>>>>>>>> SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX >>>>>>>>>> >>>>>>>>>> Red Hat NA >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> gshereme at redhat.com IRC: gshereme >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Devel mailing list >>>>>>>>>> Devel at ovirt.org >>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/devel >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Infra mailing list >>>>>>>>> Infra at ovirt.org >>>>>>>>> http://lists.ovirt.org/mailman/listinfo/infra >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Didi >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Didi >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> GREG SHEREMETA >>>>> >>>>> SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX >>>>> >>>>> Red Hat NA >>>>> >>>>> >>>>> >>>>> gshereme at redhat.com IRC: gshereme >>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> >>>> GREG SHEREMETA >>>> >>>> SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX >>>> >>>> Red Hat NA >>>> >>>> >>>> >>>> gshereme at redhat.com IRC: gshereme >>>> >>>> >>>> _______________________________________________ >>>> Infra mailing list >>>> Infra at ovirt.org >>>> http://lists.ovirt.org/mailman/listinfo/infra >>>> >>>> >> >> >> -- >> >> GREG SHEREMETA >> >> SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX >> >> Red Hat NA >> >> >> >> gshereme at redhat.com IRC: gshereme >> >> > > > > -- > > Eyal edri > > > MANAGER > > RHV DevOps > > EMEA VIRTUALIZATION R&D > > > Red Hat EMEA > TRIED. TESTED. TRUSTED. > phone: +972-9-7692018 <+972%209-769-2018> > irc: eedri (on #tlv #rhev-dev #rhev-integ) > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 5835 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 6237 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 7163 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 23605 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 27001 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 26288 bytes Desc: not available URL: From dron at redhat.com Mon Mar 26 16:02:58 2018 From: dron at redhat.com (Dafna Ron) Date: Mon, 26 Mar 2018 17:02:58 +0100 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (log-collector/engine/vdsm) ] [ 23/25/26-03-2018 ] [004_basic_sanity.hotplug_cpu ] Message-ID: Hi, We have a failure that seems to be random and happening in several projects. from what I can see, we are failing due to a timing issue in the test itself because we are querying the vm after its been destroyed in engine. looking at engine, I can see that the vm was actually shut down, I would like to disable this test until we can fix the issue since it already failed about 7 different patches from different projects. *Link to Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6521/ Link to all logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6521/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-004_basic_sanity.py/ (Relevant) error snippet from the log: * *engine: 018-03-23 14:44:14,156-04 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (ForkJoinPool-1-worker-5) [] Failed to destroy VM '7020944f-cd26-4c31-b381-0ce6be733fd7' because VM does not exist, ignoring2018-03-23 14:44:14,156-04 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (ForkJoinPool-1-worker-5) [] FINISH, DestroyVDSCommand, log id: 646766a2018-03-23 14:44:14,156-04 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (ForkJoinPool-1-worker-5) [] method: runVdsCommand, params: [Destroy, DestroyVmVDSCommandParameters:{hostId='c40b11df-7053-437e-8acc-c5e91c1694bc', vmId='7020944f-cd26-4c31-b381-0ce6be733fd7', secondsToWait='0', gracefully='false', reason='', ignoreNoVm='true'}], timeElapsed: 14ms2018-03-23 14:44:14,156-04 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (ForkJoinPool-1-worker-5) [] method: getVmManager, params: [7020944f-cd26-4c31-b381-0ce6be733fd7], timeElapsed: 0ms2018-03-23 14:44:14,156-04 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-5) [] VM '7020944f-cd26-4c31-b381-0ce6be733fd7'(vm2) moved from 'Up' --> 'Down'2018-03-23 14:44:14,156-04 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (ForkJoinPool-1-worker-5) [] method: getVmManager, params: [7020944f-cd26-4c31-b381-0ce6be733fd7], timeElapsed: 0ms2018-03-23 14:44:14,177-04 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-4) [a463501f-f01a-4048-93ca-02647308394b] EVENT_ID: USER_STOP_VM(33), VM vm2 powered off by admin at internal-authz (Host: lago-basic-suite-master-host-0).* *vdsm: 2018-03-23 14:44:14,024-0400 INFO (libvirt/events) [virt.vm] (vmId='7020944f-cd26-4c31-b381-0ce6be733fd7') Changed state to Down: Admin shut down from the engine (code=6) (vm:1684)2018-03-23 14:44:14,028-0400 INFO (jsonrpc/1) [api.virt] FINISH destroy return={'status': {'message': 'Machine destroyed', 'code': 0}} from=::ffff:192.168.201.4,53454, flow_id=a463501f-f01a-4048-93ca-02647308394b, vmId=7020944f-cd26-4c31-b381-0ce6be733fd7 (api:52)2018-03-23 14:44:14,028-0400 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call VM.destroy succeeded in 1.72 seconds (__init__:311)2018-03-23 14:44:14,031-0400 INFO (periodic/2) [Executor] Worker was discarded (executor:305)2018-03-23 14:44:14,033-0400 INFO (libvirt/events) [virt.vm] (vmId='7020944f-cd26-4c31-b381-0ce6be733fd7') Stopping connection (guestagent:440)2018-03-23 14:44:14,033-0400 ERROR (libvirt/events) [vds] Error running VM callback (clientIF:666)Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/clientIF.py", line 629, in dispatchLibvirtEvents v.onLibvirtLifecycleEvent(event, detail, None)AttributeError: 'NoneType' object has no attribute 'onLibvirtLifecycleEvent'2018-03-23 14:44:14,078-0400 INFO (jsonrpc/4) [api.virt] START destroy(gracefulAttempts=1) from=::ffff:192.168.201.4,53454, vmId=7020944f-cd26-4c31-b381-0ce6be733fd7 (api:46)2018-03-23 14:44:14,078-0400 INFO (jsonrpc/4) [api] FINISH destroy error=Virtual machine does not exist: {'vmId': u'7020944f-cd26-4c31-b381-0ce6be733fd7'} (api:127)2018-03-23 14:44:14,079-0400 INFO (jsonrpc/4) [api.virt] FINISH destroy return={'status': {'message': "Virtual machine does not exist: {'vmId': u'7020944f-cd26-4c31-b381-0ce6be733fd7'}", 'code': 1}} from=::ffff:192.168.201.4,53454, vmId=7020944f-cd26-4c31-b381-0ce6be733fd7 (api:52)* ** -------------- next part -------------- An HTML attachment was scrubbed... URL: From dron at redhat.com Tue Mar 27 15:45:16 2018 From: dron at redhat.com (Dafna Ron) Date: Tue, 27 Mar 2018 16:45:16 +0100 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 27-03-2018 ] [ 004_basic_sanity.disk_operations ] Message-ID: Hi, We have a failure on 004_basic_sanity.disk_operations. the reported change seems to me to be connected to the failure. *Link and headline of suspected patches: core: introduce CreateSnapshotForVm - https://gerrit.ovirt.org/#/c/87671/ Link to Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6564/ Link to all logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6564/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-004_basic_sanity.py/ (Relevant) error snippet from the log: 2018-03-27 11:30:16,167-04 WARN [org.ovirt.engine.core.bll.snapshots.CreateSnapshotForVmCommand] (default task-11) [9f713a3d-82a9-4db8-8207-bb69a0a2c550] Validation of action 'CreateSnapshotForVm' failed for user admin at internal-authz. Reasons: VAR__ACTION__CREATE,VAR__TYPE__SNAPSHOT,ACTION_TYPE_FAILED_SNAPSHOT_IS_BEING_TAKEN_FOR_VM,$VmName vm12018-03-27 11:30:16,176-04 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-11) [9f713a3d-82a9-4db8-8207-bb69a0a2c550] method: runAction, params: [CreateSnapshotForVm, CreateSnapshotForVmParameters:{commandId='d8bfdb0e-5ae4-4fed-8852-a0b29e377708', user='null', commandType='Unknown', vmId='65e7fea5-8b7d-4281-bd0d-6a84de207a00'}], timeElapsed: 57ms2018-03-27 11:30:16,186-04 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-11) [] Operation Failed: [Cannot create Snapshot. Snapshot is currently being created for VM vm1.]2018-03-27 11:30:16,820-04 INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-100) [16c08ce6-9a73-489f-a1f2-56e21699f14a] Command 'CopyImageGroupVolumesData' (id: '4c4023b4-21e8-4a98-8ac3-aba51967d160') waiting on child command id: '3d96b933-6e3b-4838-94bc-db3ff0cadcdc' type:'CopyData' to complete2018-03-27 11:30:16,833-04 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (EE-ManagedThreadFactory-engineScheduled-Thread-100) [16c08ce6-9a73-489f-a1f2-56e21699f14a] method: get, params: [a85992e9-40b8-4c80-9c93-3bf767f6efa4], timeElapsed: 8ms* -------------- next part -------------- An HTML attachment was scrubbed... URL: From bzlotnik at redhat.com Tue Mar 27 19:03:55 2018 From: bzlotnik at redhat.com (Benny Zlotnik) Date: Tue, 27 Mar 2018 22:03:55 +0300 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 27-03-2018 ] [ 004_basic_sanity.disk_operations ] In-Reply-To: References: Message-ID: Looking into this On Tue, Mar 27, 2018 at 6:45 PM, Dafna Ron wrote: > Hi, > > We have a failure on 004_basic_sanity.disk_operations. the reported > change seems to me to be connected to the failure. > > > > > > > > > > > > > > > > > > > > *Link and headline of suspected patches: core: introduce > CreateSnapshotForVm - https://gerrit.ovirt.org/#/c/87671/ > Link to > Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6564/ > Link > to all > logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6564/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-004_basic_sanity.py/ > (Relevant) > error snippet from the log: 2018-03-27 11:30:16,167-04 WARN > [org.ovirt.engine.core.bll.snapshots.CreateSnapshotForVmCommand] (default > task-11) [9f713a3d-82a9-4db8-8207-bb69a0a2c550] Validation of action > 'CreateSnapshotForVm' failed for user admin at internal-authz. Reasons: > VAR__ACTION__CREATE,VAR__TYPE__SNAPSHOT,ACTION_TYPE_FAILED_SNAPSHOT_IS_BEING_TAKEN_FOR_VM,$VmName > vm12018-03-27 11:30:16,176-04 DEBUG > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > (default task-11) [9f713a3d-82a9-4db8-8207-bb69a0a2c550] method: runAction, > params: [CreateSnapshotForVm, > CreateSnapshotForVmParameters:{commandId='d8bfdb0e-5ae4-4fed-8852-a0b29e377708', > user='null', commandType='Unknown', > vmId='65e7fea5-8b7d-4281-bd0d-6a84de207a00'}], timeElapsed: 57ms2018-03-27 > 11:30:16,186-04 ERROR > [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default > task-11) [] Operation Failed: [Cannot create Snapshot. Snapshot is > currently being created for VM vm1.]2018-03-27 11:30:16,820-04 INFO > [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] > (EE-ManagedThreadFactory-engineScheduled-Thread-100) > [16c08ce6-9a73-489f-a1f2-56e21699f14a] Command 'CopyImageGroupVolumesData' > (id: '4c4023b4-21e8-4a98-8ac3-aba51967d160') waiting on child command id: > '3d96b933-6e3b-4838-94bc-db3ff0cadcdc' type:'CopyData' to > complete2018-03-27 11:30:16,833-04 DEBUG > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > (EE-ManagedThreadFactory-engineScheduled-Thread-100) > [16c08ce6-9a73-489f-a1f2-56e21699f14a] method: get, params: > [a85992e9-40b8-4c80-9c93-3bf767f6efa4], timeElapsed: 8ms* > -------------- next part -------------- An HTML attachment was scrubbed... URL: From danken at redhat.com Wed Mar 28 07:15:55 2018 From: danken at redhat.com (Dan Kenigsberg) Date: Wed, 28 Mar 2018 10:15:55 +0300 Subject: [ovirt-devel] new time-based test randomly failing Message-ID: I suppose this test should be reworked, or at least, have its constants tweaked. FAIL: test_slow_handler_sync (common.logutils_test.TestThreadedHandler) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/jenkins/workspace/vdsm_master_check-patch-fc27-x86_64/vdsm/tests/common/logutils_test.py", line 319, in test_slow_handler_sync self.assertGreater(max(workers_time), 0.9) AssertionError: 0.8714249134063721 not greater than 0.9 From amureini at redhat.com Wed Mar 28 07:47:10 2018 From: amureini at redhat.com (Allon Mureinik) Date: Wed, 28 Mar 2018 10:47:10 +0300 Subject: [ovirt-devel] [ENGINE] JUnit version upgrade Message-ID: Hi oVirt Engine developers, I've just merged a patch to bump oVirt Engine's JUnit requirement to the latest available 5.1 GA. As a first step, I've integrated the vintage engine, which is a drop-in replacement that can run old JUnit 4 (and even JUnit 3!) tests, just with a newer runtime engine. This isn't the final goal, but a first step towards moving to the newer Jupiter engine, which won't be a clean dependency replacement. Please let me know if you encounter any problem. Regards, Allon -------------- next part -------------- An HTML attachment was scrubbed... URL: From bzlotnik at redhat.com Wed Mar 28 08:13:48 2018 From: bzlotnik at redhat.com (Benny Zlotnik) Date: Wed, 28 Mar 2018 11:13:48 +0300 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 27-03-2018 ] [ 004_basic_sanity.disk_operations ] In-Reply-To: References: Message-ID: This issue looks like it's caused by the DB lock being released before the engine memory lock, there was a similar race with the LSM test a few months back. I'll apply the same fix for the snapshot_cold_merge. . On Tue, Mar 27, 2018 at 10:03 PM, Benny Zlotnik wrote: > Looking into this > > On Tue, Mar 27, 2018 at 6:45 PM, Dafna Ron wrote: > >> Hi, >> >> We have a failure on 004_basic_sanity.disk_operations. the reported >> change seems to me to be connected to the failure. >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> *Link and headline of suspected patches: core: introduce >> CreateSnapshotForVm - https://gerrit.ovirt.org/#/c/87671/ >> Link to >> Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6564/ >> Link >> to all >> logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6564/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-004_basic_sanity.py/ >> (Relevant) >> error snippet from the log: 2018-03-27 11:30:16,167-04 WARN >> [org.ovirt.engine.core.bll.snapshots.CreateSnapshotForVmCommand] (default >> task-11) [9f713a3d-82a9-4db8-8207-bb69a0a2c550] Validation of action >> 'CreateSnapshotForVm' failed for user admin at internal-authz. Reasons: >> VAR__ACTION__CREATE,VAR__TYPE__SNAPSHOT,ACTION_TYPE_FAILED_SNAPSHOT_IS_BEING_TAKEN_FOR_VM,$VmName >> vm12018-03-27 11:30:16,176-04 DEBUG >> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >> (default task-11) [9f713a3d-82a9-4db8-8207-bb69a0a2c550] method: runAction, >> params: [CreateSnapshotForVm, >> CreateSnapshotForVmParameters:{commandId='d8bfdb0e-5ae4-4fed-8852-a0b29e377708', >> user='null', commandType='Unknown', >> vmId='65e7fea5-8b7d-4281-bd0d-6a84de207a00'}], timeElapsed: 57ms2018-03-27 >> 11:30:16,186-04 ERROR >> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default >> task-11) [] Operation Failed: [Cannot create Snapshot. Snapshot is >> currently being created for VM vm1.]2018-03-27 11:30:16,820-04 INFO >> [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] >> (EE-ManagedThreadFactory-engineScheduled-Thread-100) >> [16c08ce6-9a73-489f-a1f2-56e21699f14a] Command 'CopyImageGroupVolumesData' >> (id: '4c4023b4-21e8-4a98-8ac3-aba51967d160') waiting on child command id: >> '3d96b933-6e3b-4838-94bc-db3ff0cadcdc' type:'CopyData' to >> complete2018-03-27 11:30:16,833-04 DEBUG >> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >> (EE-ManagedThreadFactory-engineScheduled-Thread-100) >> [16c08ce6-9a73-489f-a1f2-56e21699f14a] method: get, params: >> [a85992e9-40b8-4c80-9c93-3bf767f6efa4], timeElapsed: 8ms* >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mzamazal at redhat.com Wed Mar 28 08:18:51 2018 From: mzamazal at redhat.com (Milan Zamazal) Date: Wed, 28 Mar 2018 10:18:51 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (log-collector/engine/vdsm) ] [ 23/25/26-03-2018 ] [004_basic_sanity.hotplug_cpu ] In-Reply-To: (Dafna Ron's message of "Mon, 26 Mar 2018 17:02:58 +0100") References: Message-ID: <874ll0ipec.fsf@redhat.com> Dafna Ron writes: > We have a failure that seems to be random and happening in several > projects. Does this failure occur only recently or has it been present for ages? > from what I can see, we are failing due to a timing issue in the test > itself because we are querying the vm after its been destroyed in engine. > looking at engine, I can see that the vm was actually shut down, No, the VM was shut down in another test. It's already running again in hotplug_cpu. > I would like to disable this test until we can fix the issue since it > already failed about 7 different patches from different projects. I can see that Gal has already increased the timeout. I think the test could be split to reduce the waiting delay, I'll post a patch for that. > *Link to > Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6521/ > Link > to all > logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6521/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-004_basic_sanity.py/ > (Relevant) > error snippet from the log: * > > > > > > > > > > > > > *engine: 018-03-23 14:44:14,156-04 INFO > [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] > (ForkJoinPool-1-worker-5) [] Failed to destroy VM > '7020944f-cd26-4c31-b381-0ce6be733fd7' because VM does not exist, > ignoring2018-03-23 14:44:14,156-04 INFO > [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] > (ForkJoinPool-1-worker-5) [] FINISH, DestroyVDSCommand, log id: > 646766a2018-03-23 14:44:14,156-04 DEBUG > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > (ForkJoinPool-1-worker-5) [] method: runVdsCommand, params: [Destroy, > DestroyVmVDSCommandParameters:{hostId='c40b11df-7053-437e-8acc-c5e91c1694bc', > vmId='7020944f-cd26-4c31-b381-0ce6be733fd7', secondsToWait='0', > gracefully='false', reason='', ignoreNoVm='true'}], timeElapsed: > 14ms2018-03-23 14:44:14,156-04 DEBUG > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > (ForkJoinPool-1-worker-5) [] method: getVmManager, params: > [7020944f-cd26-4c31-b381-0ce6be733fd7], timeElapsed: 0ms2018-03-23 > 14:44:14,156-04 INFO > [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] > (ForkJoinPool-1-worker-5) [] VM '7020944f-cd26-4c31-b381-0ce6be733fd7'(vm2) > moved from 'Up' --> 'Down'2018-03-23 14:44:14,156-04 DEBUG > [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] > (ForkJoinPool-1-worker-5) [] method: getVmManager, params: > [7020944f-cd26-4c31-b381-0ce6be733fd7], timeElapsed: 0ms2018-03-23 > 14:44:14,177-04 INFO > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > (default task-4) [a463501f-f01a-4048-93ca-02647308394b] EVENT_ID: > USER_STOP_VM(33), VM vm2 powered off by admin at internal-authz (Host: > lago-basic-suite-master-host-0).* > > > > > > > > > > > > > > > > > > > *vdsm: 2018-03-23 14:44:14,024-0400 INFO (libvirt/events) [virt.vm] > (vmId='7020944f-cd26-4c31-b381-0ce6be733fd7') Changed state to Down: Admin > shut down from the engine (code=6) (vm:1684)2018-03-23 14:44:14,028-0400 > INFO (jsonrpc/1) [api.virt] FINISH destroy return={'status': {'message': > 'Machine destroyed', 'code': 0}} from=::ffff:192.168.201.4,53454, > flow_id=a463501f-f01a-4048-93ca-02647308394b, > vmId=7020944f-cd26-4c31-b381-0ce6be733fd7 (api:52)2018-03-23 > 14:44:14,028-0400 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call > VM.destroy succeeded in 1.72 seconds (__init__:311)2018-03-23 > 14:44:14,031-0400 INFO (periodic/2) [Executor] Worker was discarded > (executor:305)2018-03-23 14:44:14,033-0400 INFO (libvirt/events) [virt.vm] > (vmId='7020944f-cd26-4c31-b381-0ce6be733fd7') Stopping connection > (guestagent:440)2018-03-23 14:44:14,033-0400 ERROR (libvirt/events) [vds] > Error running VM callback (clientIF:666)Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/vdsm/clientIF.py", line 629, in > dispatchLibvirtEvents v.onLibvirtLifecycleEvent(event, detail, > None)AttributeError: 'NoneType' object has no attribute > 'onLibvirtLifecycleEvent'2018-03-23 14:44:14,078-0400 INFO (jsonrpc/4) > [api.virt] START destroy(gracefulAttempts=1) > from=::ffff:192.168.201.4,53454, vmId=7020944f-cd26-4c31-b381-0ce6be733fd7 > (api:46)2018-03-23 14:44:14,078-0400 INFO (jsonrpc/4) [api] FINISH destroy > error=Virtual machine does not exist: {'vmId': > u'7020944f-cd26-4c31-b381-0ce6be733fd7'} (api:127)2018-03-23 > 14:44:14,079-0400 INFO (jsonrpc/4) [api.virt] FINISH destroy > return={'status': {'message': "Virtual machine does not exist: {'vmId': > u'7020944f-cd26-4c31-b381-0ce6be733fd7'}", 'code': 1}} > from=::ffff:192.168.201.4,53454, vmId=7020944f-cd26-4c31-b381-0ce6be733fd7 > (api:52)* > > ** > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel From mzamazal at redhat.com Wed Mar 28 08:23:32 2018 From: mzamazal at redhat.com (Milan Zamazal) Date: Wed, 28 Mar 2018 10:23:32 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (log-collector/engine/vdsm) ] [ 23/25/26-03-2018 ] [004_basic_sanity.hotplug_cpu ] In-Reply-To: <874ll0ipec.fsf@redhat.com> (Milan Zamazal's message of "Wed, 28 Mar 2018 10:18:51 +0200") References: <874ll0ipec.fsf@redhat.com> Message-ID: <87zi2sham3.fsf@redhat.com> Milan Zamazal writes: > Dafna Ron writes: > >> We have a failure that seems to be random and happening in several >> projects. > > Does this failure occur only recently or has it been present for ages? > >> from what I can see, we are failing due to a timing issue in the test >> itself because we are querying the vm after its been destroyed in engine. >> looking at engine, I can see that the vm was actually shut down, > > No, the VM was shut down in another test. It's already running again in > hotplug_cpu. > >> I would like to disable this test until we can fix the issue since it >> already failed about 7 different patches from different projects. > > I can see that Gal has already increased the timeout. I think the test > could be split to reduce the waiting delay, I'll post a patch for that. BTW I think the primary cause of the trouble are the infamous Cirros networking recovery problems. From dron at redhat.com Wed Mar 28 09:01:12 2018 From: dron at redhat.com (Dafna Ron) Date: Wed, 28 Mar 2018 10:01:12 +0100 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (log-collector/engine/vdsm) ] [ 23/25/26-03-2018 ] [004_basic_sanity.hotplug_cpu ] In-Reply-To: <87zi2sham3.fsf@redhat.com> References: <874ll0ipec.fsf@redhat.com> <87zi2sham3.fsf@redhat.com> Message-ID: Thanks Milan, Can you post the fix that you added on the mail? Thanks, Dafna On Wed, Mar 28, 2018 at 9:23 AM, Milan Zamazal wrote: > Milan Zamazal writes: > > > Dafna Ron writes: > > > >> We have a failure that seems to be random and happening in several > >> projects. > > > > Does this failure occur only recently or has it been present for ages? > > > >> from what I can see, we are failing due to a timing issue in the test > >> itself because we are querying the vm after its been destroyed in > engine. > >> looking at engine, I can see that the vm was actually shut down, > > > > No, the VM was shut down in another test. It's already running again in > > hotplug_cpu. > > > >> I would like to disable this test until we can fix the issue since it > >> already failed about 7 different patches from different projects. > > > > I can see that Gal has already increased the timeout. I think the test > > could be split to reduce the waiting delay, I'll post a patch for that. > > BTW I think the primary cause of the trouble are the infamous Cirros > networking recovery problems. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dron at redhat.com Wed Mar 28 09:01:52 2018 From: dron at redhat.com (Dafna Ron) Date: Wed, 28 Mar 2018 10:01:52 +0100 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 27-03-2018 ] [ 004_basic_sanity.disk_operations ] In-Reply-To: References: Message-ID: Thanks Beni. can you please paste the fix that you submit to the mail? Thanks, Dafna On Wed, Mar 28, 2018 at 9:13 AM, Benny Zlotnik wrote: > This issue looks like it's caused by the DB lock being released before the > engine memory lock, there was a similar race with the LSM test a few months > back. I'll apply the same fix for the snapshot_cold_merge. > . > > On Tue, Mar 27, 2018 at 10:03 PM, Benny Zlotnik > wrote: > >> Looking into this >> >> On Tue, Mar 27, 2018 at 6:45 PM, Dafna Ron wrote: >> >>> Hi, >>> >>> We have a failure on 004_basic_sanity.disk_operations. the reported >>> change seems to me to be connected to the failure. >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> *Link and headline of suspected patches: core: introduce >>> CreateSnapshotForVm - https://gerrit.ovirt.org/#/c/87671/ >>> Link to >>> Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6564/ >>> Link >>> to all >>> logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6564/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-004_basic_sanity.py/ >>> (Relevant) >>> error snippet from the log: 2018-03-27 11:30:16,167-04 WARN >>> [org.ovirt.engine.core.bll.snapshots.CreateSnapshotForVmCommand] (default >>> task-11) [9f713a3d-82a9-4db8-8207-bb69a0a2c550] Validation of action >>> 'CreateSnapshotForVm' failed for user admin at internal-authz. Reasons: >>> VAR__ACTION__CREATE,VAR__TYPE__SNAPSHOT,ACTION_TYPE_FAILED_SNAPSHOT_IS_BEING_TAKEN_FOR_VM,$VmName >>> vm12018-03-27 11:30:16,176-04 DEBUG >>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >>> (default task-11) [9f713a3d-82a9-4db8-8207-bb69a0a2c550] method: runAction, >>> params: [CreateSnapshotForVm, >>> CreateSnapshotForVmParameters:{commandId='d8bfdb0e-5ae4-4fed-8852-a0b29e377708', >>> user='null', commandType='Unknown', >>> vmId='65e7fea5-8b7d-4281-bd0d-6a84de207a00'}], timeElapsed: 57ms2018-03-27 >>> 11:30:16,186-04 ERROR >>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default >>> task-11) [] Operation Failed: [Cannot create Snapshot. Snapshot is >>> currently being created for VM vm1.]2018-03-27 11:30:16,820-04 INFO >>> [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] >>> (EE-ManagedThreadFactory-engineScheduled-Thread-100) >>> [16c08ce6-9a73-489f-a1f2-56e21699f14a] Command 'CopyImageGroupVolumesData' >>> (id: '4c4023b4-21e8-4a98-8ac3-aba51967d160') waiting on child command id: >>> '3d96b933-6e3b-4838-94bc-db3ff0cadcdc' type:'CopyData' to >>> complete2018-03-27 11:30:16,833-04 DEBUG >>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >>> (EE-ManagedThreadFactory-engineScheduled-Thread-100) >>> [16c08ce6-9a73-489f-a1f2-56e21699f14a] method: get, params: >>> [a85992e9-40b8-4c80-9c93-3bf767f6efa4], timeElapsed: 8ms* >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bzlotnik at redhat.com Wed Mar 28 11:48:59 2018 From: bzlotnik at redhat.com (Benny Zlotnik) Date: Wed, 28 Mar 2018 14:48:59 +0300 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 27-03-2018 ] [ 004_basic_sanity.disk_operations ] In-Reply-To: References: Message-ID: https://gerrit.ovirt.org/#/c/89534/ https://gerrit.ovirt.org/#/c/89537/ On Wed, Mar 28, 2018 at 12:01 PM, Dafna Ron wrote: > Thanks Beni. > > can you please paste the fix that you submit to the mail? > > Thanks, > Dafna > > > On Wed, Mar 28, 2018 at 9:13 AM, Benny Zlotnik > wrote: > >> This issue looks like it's caused by the DB lock being released before >> the engine memory lock, there was a similar race with the LSM test a few >> months back. I'll apply the same fix for the snapshot_cold_merge. >> . >> >> On Tue, Mar 27, 2018 at 10:03 PM, Benny Zlotnik >> wrote: >> >>> Looking into this >>> >>> On Tue, Mar 27, 2018 at 6:45 PM, Dafna Ron wrote: >>> >>>> Hi, >>>> >>>> We have a failure on 004_basic_sanity.disk_operations. the reported >>>> change seems to me to be connected to the failure. >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> *Link and headline of suspected patches: core: introduce >>>> CreateSnapshotForVm - https://gerrit.ovirt.org/#/c/87671/ >>>> Link to >>>> Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6564/ >>>> Link >>>> to all >>>> logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6564/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-004_basic_sanity.py/ >>>> (Relevant) >>>> error snippet from the log: 2018-03-27 11:30:16,167-04 WARN >>>> [org.ovirt.engine.core.bll.snapshots.CreateSnapshotForVmCommand] (default >>>> task-11) [9f713a3d-82a9-4db8-8207-bb69a0a2c550] Validation of action >>>> 'CreateSnapshotForVm' failed for user admin at internal-authz. Reasons: >>>> VAR__ACTION__CREATE,VAR__TYPE__SNAPSHOT,ACTION_TYPE_FAILED_SNAPSHOT_IS_BEING_TAKEN_FOR_VM,$VmName >>>> vm12018-03-27 11:30:16,176-04 DEBUG >>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >>>> (default task-11) [9f713a3d-82a9-4db8-8207-bb69a0a2c550] method: runAction, >>>> params: [CreateSnapshotForVm, >>>> CreateSnapshotForVmParameters:{commandId='d8bfdb0e-5ae4-4fed-8852-a0b29e377708', >>>> user='null', commandType='Unknown', >>>> vmId='65e7fea5-8b7d-4281-bd0d-6a84de207a00'}], timeElapsed: 57ms2018-03-27 >>>> 11:30:16,186-04 ERROR >>>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default >>>> task-11) [] Operation Failed: [Cannot create Snapshot. Snapshot is >>>> currently being created for VM vm1.]2018-03-27 11:30:16,820-04 INFO >>>> [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] >>>> (EE-ManagedThreadFactory-engineScheduled-Thread-100) >>>> [16c08ce6-9a73-489f-a1f2-56e21699f14a] Command 'CopyImageGroupVolumesData' >>>> (id: '4c4023b4-21e8-4a98-8ac3-aba51967d160') waiting on child command id: >>>> '3d96b933-6e3b-4838-94bc-db3ff0cadcdc' type:'CopyData' to >>>> complete2018-03-27 11:30:16,833-04 DEBUG >>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >>>> (EE-ManagedThreadFactory-engineScheduled-Thread-100) >>>> [16c08ce6-9a73-489f-a1f2-56e21699f14a] method: get, params: >>>> [a85992e9-40b8-4c80-9c93-3bf767f6efa4], timeElapsed: 8ms* >>>> >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mzamazal at redhat.com Wed Mar 28 11:52:28 2018 From: mzamazal at redhat.com (Milan Zamazal) Date: Wed, 28 Mar 2018 13:52:28 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (log-collector/engine/vdsm) ] [ 23/25/26-03-2018 ] [004_basic_sanity.hotplug_cpu ] In-Reply-To: (Dafna Ron's message of "Wed, 28 Mar 2018 10:01:12 +0100") References: <874ll0ipec.fsf@redhat.com> <87zi2sham3.fsf@redhat.com> Message-ID: <87efk4h0xv.fsf@redhat.com> Dafna Ron writes: > Can you post the fix that you added on the mail? I split hotplug_cpu test in https://gerrit.ovirt.org/89551 We'll see whether it is actually useful once the disk_operations failure is fixed. If it still causes problem then hotplug_cpu_guest_check introduced by the patch should be disabled until we can log into the guest OS reliably. > On Wed, Mar 28, 2018 at 9:23 AM, Milan Zamazal wrote: > >> Milan Zamazal writes: >> >> > Dafna Ron writes: >> > >> >> We have a failure that seems to be random and happening in several >> >> projects. >> > >> > Does this failure occur only recently or has it been present for ages? >> > >> >> from what I can see, we are failing due to a timing issue in the test >> >> itself because we are querying the vm after its been destroyed in >> engine. >> >> looking at engine, I can see that the vm was actually shut down, >> > >> > No, the VM was shut down in another test. It's already running again in >> > hotplug_cpu. >> > >> >> I would like to disable this test until we can fix the issue since it >> >> already failed about 7 different patches from different projects. >> > >> > I can see that Gal has already increased the timeout. I think the test >> > could be split to reduce the waiting delay, I'll post a patch for that. >> >> BTW I think the primary cause of the trouble are the infamous Cirros >> networking recovery problems. >> From gbenhaim at redhat.com Wed Mar 28 13:19:40 2018 From: gbenhaim at redhat.com (Gal Ben Haim) Date: Wed, 28 Mar 2018 16:19:40 +0300 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 27-03-2018 ] [ 004_basic_sanity.disk_operations ] In-Reply-To: References: Message-ID: On Wed, Mar 28, 2018 at 2:48 PM, Benny Zlotnik wrote: > https://gerrit.ovirt.org/#/c/89534/ > https://gerrit.ovirt.org/#/c/89537/ > Merged. > > > On Wed, Mar 28, 2018 at 12:01 PM, Dafna Ron wrote: > >> Thanks Beni. >> >> can you please paste the fix that you submit to the mail? >> >> Thanks, >> Dafna >> >> >> On Wed, Mar 28, 2018 at 9:13 AM, Benny Zlotnik >> wrote: >> >>> This issue looks like it's caused by the DB lock being released before >>> the engine memory lock, there was a similar race with the LSM test a few >>> months back. I'll apply the same fix for the snapshot_cold_merge. >>> . >>> >>> On Tue, Mar 27, 2018 at 10:03 PM, Benny Zlotnik >>> wrote: >>> >>>> Looking into this >>>> >>>> On Tue, Mar 27, 2018 at 6:45 PM, Dafna Ron wrote: >>>> >>>>> Hi, >>>>> >>>>> We have a failure on 004_basic_sanity.disk_operations. the reported >>>>> change seems to me to be connected to the failure. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> *Link and headline of suspected patches: core: introduce >>>>> CreateSnapshotForVm - https://gerrit.ovirt.org/#/c/87671/ >>>>> Link to >>>>> Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6564/ >>>>> Link >>>>> to all >>>>> logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6564/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-004_basic_sanity.py/ >>>>> (Relevant) >>>>> error snippet from the log: 2018-03-27 11:30:16,167-04 WARN >>>>> [org.ovirt.engine.core.bll.snapshots.CreateSnapshotForVmCommand] (default >>>>> task-11) [9f713a3d-82a9-4db8-8207-bb69a0a2c550] Validation of action >>>>> 'CreateSnapshotForVm' failed for user admin at internal-authz. Reasons: >>>>> VAR__ACTION__CREATE,VAR__TYPE__SNAPSHOT,ACTION_TYPE_FAILED_SNAPSHOT_IS_BEING_TAKEN_FOR_VM,$VmName >>>>> vm12018-03-27 11:30:16,176-04 DEBUG >>>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >>>>> (default task-11) [9f713a3d-82a9-4db8-8207-bb69a0a2c550] method: runAction, >>>>> params: [CreateSnapshotForVm, >>>>> CreateSnapshotForVmParameters:{commandId='d8bfdb0e-5ae4-4fed-8852-a0b29e377708', >>>>> user='null', commandType='Unknown', >>>>> vmId='65e7fea5-8b7d-4281-bd0d-6a84de207a00'}], timeElapsed: 57ms2018-03-27 >>>>> 11:30:16,186-04 ERROR >>>>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default >>>>> task-11) [] Operation Failed: [Cannot create Snapshot. Snapshot is >>>>> currently being created for VM vm1.]2018-03-27 11:30:16,820-04 INFO >>>>> [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] >>>>> (EE-ManagedThreadFactory-engineScheduled-Thread-100) >>>>> [16c08ce6-9a73-489f-a1f2-56e21699f14a] Command 'CopyImageGroupVolumesData' >>>>> (id: '4c4023b4-21e8-4a98-8ac3-aba51967d160') waiting on child command id: >>>>> '3d96b933-6e3b-4838-94bc-db3ff0cadcdc' type:'CopyData' to >>>>> complete2018-03-27 11:30:16,833-04 DEBUG >>>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >>>>> (EE-ManagedThreadFactory-engineScheduled-Thread-100) >>>>> [16c08ce6-9a73-489f-a1f2-56e21699f14a] method: get, params: >>>>> [a85992e9-40b8-4c80-9c93-3bf767f6efa4], timeElapsed: 8ms* >>>>> >>>> >>>> >>> >> > > _______________________________________________ > Infra mailing list > Infra at ovirt.org > http://lists.ovirt.org/mailman/listinfo/infra > > -- *GAL bEN HAIM* RHV DEVOPS -------------- next part -------------- An HTML attachment was scrubbed... URL: From nsoffer at redhat.com Wed Mar 28 17:08:35 2018 From: nsoffer at redhat.com (Nir Soffer) Date: Wed, 28 Mar 2018 17:08:35 +0000 Subject: [ovirt-devel] new time-based test randomly failing In-Reply-To: References: Message-ID: On Wed, Mar 28, 2018 at 10:15 AM Dan Kenigsberg wrote: > I suppose this test should be reworked, or at least, have its constants > tweaked. > > FAIL: test_slow_handler_sync (common.logutils_test.TestThreadedHandler) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File > "/home/jenkins/workspace/vdsm_master_check-patch-fc27-x86_64/vdsm/tests/common/logutils_test.py", > line 319, in test_slow_handler_sync > self.assertGreater(max(workers_time), 0.9) > AssertionError: 0.8714249134063721 not greater than 0.9 > I tried to fix this last week, but for some reason the fix does not help on the CI. I think we can remove this test, it is reproducing the issues with the standard logging code, and does not test the new code. Nir -------------- next part -------------- An HTML attachment was scrubbed... URL: From didi at redhat.com Thu Mar 29 10:46:55 2018 From: didi at redhat.com (Yedidyah Bar David) Date: Thu, 29 Mar 2018 13:46:55 +0300 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 27-03-2018 ] [ 004_basic_sanity.disk_operations ] In-Reply-To: References: Message-ID: On Wed, Mar 28, 2018 at 4:19 PM, Gal Ben Haim wrote: > > > On Wed, Mar 28, 2018 at 2:48 PM, Benny Zlotnik wrote: >> >> https://gerrit.ovirt.org/#/c/89534/ >> https://gerrit.ovirt.org/#/c/89537/ > > > Merged. Seems like it failed again similarly: 08:05:07 vm1_snapshots_service.snapshot_service(snapshot.id).remove() 08:05:07 File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 20230, in remove 08:05:07 self._internal_remove(headers, query, wait) 08:05:07 File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 271, in _internal_remove 08:05:07 return future.wait() if wait else future 08:05:07 File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait 08:05:07 return self._code(response) 08:05:07 File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 268, in callback 08:05:07 self._check_fault(response) 08:05:07 File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 132, in _check_fault 08:05:07 self._raise_error(response, body) 08:05:07 File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 118, in _raise_error 08:05:07 raise error 08:05:07 Error: Fault reason is "Operation Failed". Fault detail is "[Cannot remove Snapshot. Snapshot is currently being created for VM vm1.]". HTTP response code is 409. http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x86_64/4770/consoleFull Can you please check? Thanks. >> >> >> >> On Wed, Mar 28, 2018 at 12:01 PM, Dafna Ron wrote: >>> >>> Thanks Beni. >>> >>> can you please paste the fix that you submit to the mail? >>> >>> Thanks, >>> Dafna >>> >>> >>> On Wed, Mar 28, 2018 at 9:13 AM, Benny Zlotnik >>> wrote: >>>> >>>> This issue looks like it's caused by the DB lock being released before >>>> the engine memory lock, there was a similar race with the LSM test a few >>>> months back. I'll apply the same fix for the snapshot_cold_merge. >>>> . >>>> >>>> On Tue, Mar 27, 2018 at 10:03 PM, Benny Zlotnik >>>> wrote: >>>>> >>>>> Looking into this >>>>> >>>>> On Tue, Mar 27, 2018 at 6:45 PM, Dafna Ron wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> We have a failure on 004_basic_sanity.disk_operations. the reported >>>>>> change seems to me to be connected to the failure. >>>>>> >>>>>> Link and headline of suspected patches: >>>>>> >>>>>> core: introduce CreateSnapshotForVm - >>>>>> https://gerrit.ovirt.org/#/c/87671/ >>>>>> >>>>>> >>>>>> Link to Job: >>>>>> >>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6564/ >>>>>> >>>>>> Link to all logs: >>>>>> >>>>>> >>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6564/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-004_basic_sanity.py/ >>>>>> >>>>>> (Relevant) error snippet from the log: >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> 2018-03-27 11:30:16,167-04 WARN >>>>>> [org.ovirt.engine.core.bll.snapshots.CreateSnapshotForVmCommand] (default >>>>>> task-11) [9f713a3d-82a9-4db8-8207-bb69a0a2c550] Validation of action >>>>>> 'CreateSnapshotForVm' failed for user admin at internal-authz. Re >>>>>> asons: >>>>>> VAR__ACTION__CREATE,VAR__TYPE__SNAPSHOT,ACTION_TYPE_FAILED_SNAPSHOT_IS_BEING_TAKEN_FOR_VM,$VmName >>>>>> vm1 >>>>>> 2018-03-27 11:30:16,176-04 DEBUG >>>>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >>>>>> (default task-11) [9f713a3d-82a9-4db8-8207-bb69a0a2c550] method: runAction, >>>>>> params: [CreateSnapshotForVm, CreateSnapshotForVmParameters >>>>>> :{commandId='d8bfdb0e-5ae4-4fed-8852-a0b29e377708', user='null', >>>>>> commandType='Unknown', vmId='65e7fea5-8b7d-4281-bd0d-6a84de207a00'}], >>>>>> timeElapsed: 57ms >>>>>> 2018-03-27 11:30:16,186-04 ERROR >>>>>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default >>>>>> task-11) [] Operation Failed: [Cannot create Snapshot. Snapshot is currently >>>>>> being created for VM vm1.] >>>>>> 2018-03-27 11:30:16,820-04 INFO >>>>>> [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] >>>>>> (EE-ManagedThreadFactory-engineScheduled-Thread-100) >>>>>> [16c08ce6-9a73-489f-a1f2-56e21699f14a] Command 'CopyImageGroupVolumesData' >>>>>> (id: '4c4023 >>>>>> b4-21e8-4a98-8ac3-aba51967d160') waiting on child command id: >>>>>> '3d96b933-6e3b-4838-94bc-db3ff0cadcdc' type:'CopyData' to complete >>>>>> 2018-03-27 11:30:16,833-04 DEBUG >>>>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] >>>>>> (EE-ManagedThreadFactory-engineScheduled-Thread-100) >>>>>> [16c08ce6-9a73-489f-a1f2-56e21699f14a] method: get, params: >>>>>> [a85992e9-40b8-4c80-9c >>>>>> 93-3bf767f6efa4], timeElapsed: 8ms >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> >> >> _______________________________________________ >> Infra mailing list >> Infra at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/infra >> > > > > -- > GAL bEN HAIM > RHV DEVOPS > > _______________________________________________ > Infra mailing list > Infra at ovirt.org > http://lists.ovirt.org/mailman/listinfo/infra > -- Didi From bzlotnik at redhat.com Thu Mar 29 10:51:12 2018 From: bzlotnik at redhat.com (Benny Zlotnik) Date: Thu, 29 Mar 2018 13:51:12 +0300 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 27-03-2018 ] [ 004_basic_sanity.disk_operations ] In-Reply-To: References: Message-ID: I forgot to add the polling after the second snapshot as well, it was added in this patch: https://gerrit.ovirt.org/#/c/89577/ Sorry for the inconvenience On Thu, Mar 29, 2018 at 1:46 PM, Yedidyah Bar David wrote: > On Wed, Mar 28, 2018 at 4:19 PM, Gal Ben Haim wrote: > > > > > > On Wed, Mar 28, 2018 at 2:48 PM, Benny Zlotnik > wrote: > >> > >> https://gerrit.ovirt.org/#/c/89534/ > >> https://gerrit.ovirt.org/#/c/89537/ > > > > > > Merged. > > Seems like it failed again similarly: > > 08:05:07 vm1_snapshots_service.snapshot_service(snapshot.id).remove() > 08:05:07 File > "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line > 20230, in remove > 08:05:07 self._internal_remove(headers, query, wait) > 08:05:07 File > "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 271, > in _internal_remove > 08:05:07 return future.wait() if wait else future > 08:05:07 File > "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in > wait > 08:05:07 return self._code(response) > 08:05:07 File > "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 268, > in callback > 08:05:07 self._check_fault(response) > 08:05:07 File > "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 132, > in _check_fault > 08:05:07 self._raise_error(response, body) > 08:05:07 File > "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 118, > in _raise_error > 08:05:07 raise error > 08:05:07 Error: Fault reason is "Operation Failed". Fault detail is > "[Cannot remove Snapshot. Snapshot is currently being created for VM > vm1.]". HTTP response code is 409. > > http://jenkins.ovirt.org/job/ovirt-system-tests_master_ > check-patch-el7-x86_64/4770/consoleFull > > Can you please check? Thanks. > > >> > >> > >> > >> On Wed, Mar 28, 2018 at 12:01 PM, Dafna Ron wrote: > >>> > >>> Thanks Beni. > >>> > >>> can you please paste the fix that you submit to the mail? > >>> > >>> Thanks, > >>> Dafna > >>> > >>> > >>> On Wed, Mar 28, 2018 at 9:13 AM, Benny Zlotnik > >>> wrote: > >>>> > >>>> This issue looks like it's caused by the DB lock being released before > >>>> the engine memory lock, there was a similar race with the LSM test a > few > >>>> months back. I'll apply the same fix for the snapshot_cold_merge. > >>>> . > >>>> > >>>> On Tue, Mar 27, 2018 at 10:03 PM, Benny Zlotnik > >>>> wrote: > >>>>> > >>>>> Looking into this > >>>>> > >>>>> On Tue, Mar 27, 2018 at 6:45 PM, Dafna Ron wrote: > >>>>>> > >>>>>> Hi, > >>>>>> > >>>>>> We have a failure on 004_basic_sanity.disk_operations. the reported > >>>>>> change seems to me to be connected to the failure. > >>>>>> > >>>>>> Link and headline of suspected patches: > >>>>>> > >>>>>> core: introduce CreateSnapshotForVm - > >>>>>> https://gerrit.ovirt.org/#/c/87671/ > >>>>>> > >>>>>> > >>>>>> Link to Job: > >>>>>> > >>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6564/ > >>>>>> > >>>>>> Link to all logs: > >>>>>> > >>>>>> > >>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue- > tester/6564/artifact/exported-artifacts/basic-suit-master- > el7/test_logs/basic-suite-master/post-004_basic_sanity.py/ > >>>>>> > >>>>>> (Relevant) error snippet from the log: > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> 2018-03-27 11:30:16,167-04 WARN > >>>>>> [org.ovirt.engine.core.bll.snapshots.CreateSnapshotForVmCommand] > (default > >>>>>> task-11) [9f713a3d-82a9-4db8-8207-bb69a0a2c550] Validation of > action > >>>>>> 'CreateSnapshotForVm' failed for user admin at internal-authz. Re > >>>>>> asons: > >>>>>> VAR__ACTION__CREATE,VAR__TYPE__SNAPSHOT,ACTION_TYPE_FAILED_ > SNAPSHOT_IS_BEING_TAKEN_FOR_VM,$VmName > >>>>>> vm1 > >>>>>> 2018-03-27 11:30:16,176-04 DEBUG > >>>>>> [org.ovirt.engine.core.common.di.interceptor. > DebugLoggingInterceptor] > >>>>>> (default task-11) [9f713a3d-82a9-4db8-8207-bb69a0a2c550] method: > runAction, > >>>>>> params: [CreateSnapshotForVm, CreateSnapshotForVmParameters > >>>>>> :{commandId='d8bfdb0e-5ae4-4fed-8852-a0b29e377708', user='null', > >>>>>> commandType='Unknown', vmId='65e7fea5-8b7d-4281-bd0d- > 6a84de207a00'}], > >>>>>> timeElapsed: 57ms > >>>>>> 2018-03-27 11:30:16,186-04 ERROR > >>>>>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] > (default > >>>>>> task-11) [] Operation Failed: [Cannot create Snapshot. Snapshot is > currently > >>>>>> being created for VM vm1.] > >>>>>> 2018-03-27 11:30:16,820-04 INFO > >>>>>> [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] > >>>>>> (EE-ManagedThreadFactory-engineScheduled-Thread-100) > >>>>>> [16c08ce6-9a73-489f-a1f2-56e21699f14a] Command > 'CopyImageGroupVolumesData' > >>>>>> (id: '4c4023 > >>>>>> b4-21e8-4a98-8ac3-aba51967d160') waiting on child command id: > >>>>>> '3d96b933-6e3b-4838-94bc-db3ff0cadcdc' type:'CopyData' to complete > >>>>>> 2018-03-27 11:30:16,833-04 DEBUG > >>>>>> [org.ovirt.engine.core.common.di.interceptor. > DebugLoggingInterceptor] > >>>>>> (EE-ManagedThreadFactory-engineScheduled-Thread-100) > >>>>>> [16c08ce6-9a73-489f-a1f2-56e21699f14a] method: get, params: > >>>>>> [a85992e9-40b8-4c80-9c > >>>>>> 93-3bf767f6efa4], timeElapsed: 8ms > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>> > >>>> > >>> > >> > >> > >> _______________________________________________ > >> Infra mailing list > >> Infra at ovirt.org > >> http://lists.ovirt.org/mailman/listinfo/infra > >> > > > > > > > > -- > > GAL bEN HAIM > > RHV DEVOPS > > > > _______________________________________________ > > Infra mailing list > > Infra at ovirt.org > > http://lists.ovirt.org/mailman/listinfo/infra > > > > > > -- > Didi > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mzamazal at redhat.com Thu Mar 29 12:30:20 2018 From: mzamazal at redhat.com (Milan Zamazal) Date: Thu, 29 Mar 2018 14:30:20 +0200 Subject: [ovirt-devel] oVirt log analyzer Message-ID: <87bmf7rrmr.fsf@redhat.com> Hi, during last year Outreachy internship a tool for analyzing oVirt logs was created. When it is provided with oVirt logs (such as SOS reports, logs gathered by Lago, single or multiple log files) it tries to identify and classify important lines from the logs and present them in a structured form. Its primary purpose is to get a quick and easy overview of actions and errors. The tool analyses given logs and produces text files with the extracted information. There is an Emacs user interface that presents the output in a nice way with added functionality such as filtering. Emacs haters can use the plain text files or write another user interface. :-) You can get ovirt-log-analyzer from https://github.com/mz-pdm/ovirt-log-analyzer README.md explains how to use it. Note that ovirt-log-analyzer has been created within the limited resources of an Outreachy internship with some additional work and not everything is perfect. Feel free to make improvements. Regards, Milan From mzamazal at redhat.com Thu Mar 29 12:34:36 2018 From: mzamazal at redhat.com (Milan Zamazal) Date: Thu, 29 Mar 2018 14:34:36 +0200 Subject: [ovirt-devel] [ OST Failure Report ] [ oVirt Master (log-collector/engine/vdsm) ] [ 23/25/26-03-2018 ] [004_basic_sanity.hotplug_cpu ] In-Reply-To: <87efk4h0xv.fsf@redhat.com> (Milan Zamazal's message of "Wed, 28 Mar 2018 13:52:28 +0200") References: <874ll0ipec.fsf@redhat.com> <87zi2sham3.fsf@redhat.com> <87efk4h0xv.fsf@redhat.com> Message-ID: <877epvrrfn.fsf@redhat.com> Milan Zamazal writes: > Dafna Ron writes: > >> Can you post the fix that you added on the mail? > > I split hotplug_cpu test in https://gerrit.ovirt.org/89551 After some thought, I have a different fix: https://gerrit.ovirt.org/89589 This one checks whether ssh (and not only network) is available after resume and if not then it reboots the VM as if network wasn't available. > We'll see whether it is actually useful once the disk_operations failure > is fixed. If it still causes problem then hotplug_cpu_guest_check > introduced by the patch should be disabled until we can log into the > guest OS reliably. > >> On Wed, Mar 28, 2018 at 9:23 AM, Milan Zamazal wrote: >> >>> Milan Zamazal writes: >>> >>> > Dafna Ron writes: >>> > >>> >> We have a failure that seems to be random and happening in several >>> >> projects. >>> > >>> > Does this failure occur only recently or has it been present for ages? >>> > >>> >> from what I can see, we are failing due to a timing issue in the test >>> >> itself because we are querying the vm after its been destroyed in >>> engine. >>> >> looking at engine, I can see that the vm was actually shut down, >>> > >>> > No, the VM was shut down in another test. It's already running again in >>> > hotplug_cpu. >>> > >>> >> I would like to disable this test until we can fix the issue since it >>> >> already failed about 7 different patches from different projects. >>> > >>> > I can see that Gal has already increased the timeout. I think the test >>> > could be split to reduce the waiting delay, I'll post a patch for that. >>> >>> BTW I think the primary cause of the trouble are the infamous Cirros >>> networking recovery problems. >>> From fromani at redhat.com Thu Mar 29 13:37:32 2018 From: fromani at redhat.com (Francesco Romani) Date: Thu, 29 Mar 2018 15:37:32 +0200 Subject: [ovirt-devel] [vdsm] network/integration test failures Message-ID: <64515659-9609-4a9a-45c2-526a908ece05@redhat.com> Hi developers, we had another network CI failure while testing unrelated changes: http://jenkins.ovirt.org/job/vdsm_master_check-patch-el7-x86_64/22708/consoleFull *12:20:19* self = *12:20:19* *12:20:19* def test_bond_clear_arp_ip_target(self): *12:20:19* OPTIONS_A = { *12:20:19* 'mode': '1', *12:20:19* 'arp_interval': '1000', *12:20:19* 'arp_ip_target': '192.168.122.1'} *12:20:19* OPTIONS_B = { *12:20:19* 'mode': '1', *12:20:19* 'arp_interval': '1000'} *12:20:19* *12:20:19* with bond_device() as bond: *12:20:19* > bond.set_options(OPTIONS_A) *12:20:19* *12:20:19* network/integration/link_bond_test.py:171: Could please some network developer have a look? Bests, -- Francesco Romani Senior SW Eng., Virtualization R&D Red Hat IRC: fromani github: @fromanirh -------------- next part -------------- An HTML attachment was scrubbed... URL: From danken at redhat.com Thu Mar 29 14:01:59 2018 From: danken at redhat.com (Dan Kenigsberg) Date: Thu, 29 Mar 2018 17:01:59 +0300 Subject: [ovirt-devel] [vdsm] network/integration test failures In-Reply-To: <64515659-9609-4a9a-45c2-526a908ece05@redhat.com> References: <64515659-9609-4a9a-45c2-526a908ece05@redhat.com> Message-ID: On Thu, Mar 29, 2018 at 4:37 PM, Francesco Romani wrote: > Hi developers, > > > we had another network CI failure while testing unrelated changes: > > > http://jenkins.ovirt.org/job/vdsm_master_check-patch-el7-x86_64/22708/consoleFull > > > 12:20:19 self = testMethod=test_bond_clear_arp_ip_target> > 12:20:19 > 12:20:19 def test_bond_clear_arp_ip_target(self): > 12:20:19 OPTIONS_A = { > 12:20:19 'mode': '1', > 12:20:19 'arp_interval': '1000', > 12:20:19 'arp_ip_target': '192.168.122.1'} > 12:20:19 OPTIONS_B = { > 12:20:19 'mode': '1', > 12:20:19 'arp_interval': '1000'} > 12:20:19 > 12:20:19 with bond_device() as bond: > 12:20:19 > bond.set_options(OPTIONS_A) > 12:20:19 > 12:20:19 network/integration/link_bond_test.py:171: > > Could please some network developer have a look? > > Bests, > > -- > Francesco Romani > Senior SW Eng., Virtualization R&D > Red Hat > IRC: fromani github: @fromanirh > > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel So sorry for this noise. I've prematurely merged a patch that moved tests around, and mistakenly approved a CI failure there, which blocked the real problem. https://gerrit.ovirt.org/#/c/89562/ (merged recently) should solve your reported problem. Note, however, that we still have a problem with ovs-vsctl randomly hanging our integration tests since we've upgraded to openvswitch-2.9 last weekend. From danken at redhat.com Thu Mar 29 16:17:51 2018 From: danken at redhat.com (Dan Kenigsberg) Date: Thu, 29 Mar 2018 19:17:51 +0300 Subject: [ovirt-devel] new time-based test randomly failing In-Reply-To: References: Message-ID: On Wed, Mar 28, 2018 at 8:08 PM, Nir Soffer wrote: > On Wed, Mar 28, 2018 at 10:15 AM Dan Kenigsberg wrote: >> >> I suppose this test should be reworked, or at least, have its constants >> tweaked. >> >> FAIL: test_slow_handler_sync (common.logutils_test.TestThreadedHandler) >> ---------------------------------------------------------------------- >> Traceback (most recent call last): >> File >> "/home/jenkins/workspace/vdsm_master_check-patch-fc27-x86_64/vdsm/tests/common/logutils_test.py", >> line 319, in test_slow_handler_sync >> self.assertGreater(max(workers_time), 0.9) >> AssertionError: 0.8714249134063721 not greater than 0.9 > > > I tried to fix this last week, but for some reason the fix does not help on > the CI. > > I think we can remove this test, it is reproducing the issues with the > standard > logging code, and does not test the new code. Ok: https://gerrit.ovirt.org/89618 From nsoffer at redhat.com Thu Mar 29 16:27:11 2018 From: nsoffer at redhat.com (Nir Soffer) Date: Thu, 29 Mar 2018 16:27:11 +0000 Subject: [ovirt-devel] new time-based test randomly failing In-Reply-To: References: Message-ID: On Thu, Mar 29, 2018 at 7:17 PM Dan Kenigsberg wrote: > On Wed, Mar 28, 2018 at 8:08 PM, Nir Soffer wrote: > > On Wed, Mar 28, 2018 at 10:15 AM Dan Kenigsberg > wrote: > >> > >> I suppose this test should be reworked, or at least, have its constants > >> tweaked. > >> > >> FAIL: test_slow_handler_sync (common.logutils_test.TestThreadedHandler) > >> ---------------------------------------------------------------------- > >> Traceback (most recent call last): > >> File > >> > "/home/jenkins/workspace/vdsm_master_check-patch-fc27-x86_64/vdsm/tests/common/logutils_test.py", > >> line 319, in test_slow_handler_sync > >> self.assertGreater(max(workers_time), 0.9) > >> AssertionError: 0.8714249134063721 not greater than 0.9 > > > > > > I tried to fix this last week, but for some reason the fix does not help > on > > the CI. > > > > I think we can remove this test, it is reproducing the issues with the > > standard > > logging code, and does not test the new code. > > Ok: https://gerrit.ovirt.org/89618 Thanks, akced. -------------- next part -------------- An HTML attachment was scrubbed... URL: From gshereme at redhat.com Thu Mar 29 17:55:14 2018 From: gshereme at redhat.com (Greg Sheremeta) Date: Thu, 29 Mar 2018 13:55:14 -0400 Subject: [ovirt-devel] oVirt log analyzer In-Reply-To: <87bmf7rrmr.fsf@redhat.com> References: <87bmf7rrmr.fsf@redhat.com> Message-ID: Nice! I think a nice RFE would be to surface this info in the UI. On Thu, Mar 29, 2018 at 8:30 AM, Milan Zamazal wrote: > Hi, during last year Outreachy internship a tool for analyzing oVirt > logs was created. When it is provided with oVirt logs (such as SOS > reports, logs gathered by Lago, single or multiple log files) it tries > to identify and classify important lines from the logs and present them > in a structured form. Its primary purpose is to get a quick and easy > overview of actions and errors. > > The tool analyses given logs and produces text files with the extracted > information. There is an Emacs user interface that presents the output > in a nice way with added functionality such as filtering. Emacs haters > can use the plain text files or write another user interface. :-) > > You can get ovirt-log-analyzer from > https://github.com/mz-pdm/ovirt-log-analyzer > README.md explains how to use it. > > Note that ovirt-log-analyzer has been created within the limited > resources of an Outreachy internship with some additional work and not > everything is perfect. Feel free to make improvements. > > Regards, > Milan > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -- GREG SHEREMETA SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX Red Hat NA gshereme at redhat.com IRC: gshereme -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjelinek at redhat.com Thu Mar 29 18:11:01 2018 From: tjelinek at redhat.com (Tomas Jelinek) Date: Thu, 29 Mar 2018 20:11:01 +0200 Subject: [ovirt-devel] oVirt log analyzer In-Reply-To: References: <87bmf7rrmr.fsf@redhat.com> Message-ID: On Thu, Mar 29, 2018 at 7:55 PM, Greg Sheremeta wrote: > Nice! I think a nice RFE would be to surface this info in the UI. > > On Thu, Mar 29, 2018 at 8:30 AM, Milan Zamazal > wrote: > >> Hi, during last year Outreachy internship a tool for analyzing oVirt >> logs was created. When it is provided with oVirt logs (such as SOS >> reports, logs gathered by Lago, single or multiple log files) it tries >> to identify and classify important lines from the logs and present them >> in a structured form. Its primary purpose is to get a quick and easy >> overview of actions and errors. >> > I would add that it can correlate more log files (from engine/vdsm/libvirt/quemu) and show a unified view of them. It can follow the life of one entity (such as a VM) and show what was going on with it across the system. I have used it a lot to look for races and it was pretty useful for that. > >> The tool analyses given logs and produces text files with the extracted >> information. There is an Emacs user interface that presents the output >> in a nice way with added functionality such as filtering. Emacs haters >> can use the plain text files or write another user interface. :-) >> >> You can get ovirt-log-analyzer from >> https://github.com/mz-pdm/ovirt-log-analyzer >> README.md explains how to use it. >> >> Note that ovirt-log-analyzer has been created within the limited >> resources of an Outreachy internship with some additional work and not >> everything is perfect. Feel free to make improvements. >> >> Regards, >> Milan >> _______________________________________________ >> Devel mailing list >> Devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel >> > > > > -- > > GREG SHEREMETA > > SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX > > Red Hat NA > > > > gshereme at redhat.com IRC: gshereme > > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bkorren at redhat.com Thu Mar 29 18:49:08 2018 From: bkorren at redhat.com (Barak Korren) Date: Thu, 29 Mar 2018 21:49:08 +0300 Subject: [ovirt-devel] oVirt log analyzer In-Reply-To: References: <87bmf7rrmr.fsf@redhat.com> Message-ID: On 29 March 2018 at 21:11, Tomas Jelinek wrote: > > > On Thu, Mar 29, 2018 at 7:55 PM, Greg Sheremeta > wrote: > >> Nice! I think a nice RFE would be to surface this info in the UI. >> >> On Thu, Mar 29, 2018 at 8:30 AM, Milan Zamazal >> wrote: >> >>> Hi, during last year Outreachy internship a tool for analyzing oVirt >>> logs was created. When it is provided with oVirt logs (such as SOS >>> reports, logs gathered by Lago, single or multiple log files) it tries >>> to identify and classify important lines from the logs and present them >>> in a structured form. Its primary purpose is to get a quick and easy >>> overview of actions and errors. >>> >> > I would add that it can correlate more log files (from > engine/vdsm/libvirt/quemu) and show a unified view of them. > It can follow the life of one entity (such as a VM) and show what was > going on with it across the system. I have used it a lot to look for races > and it was pretty useful for that. > I wonder if we can automate running it on OST failures to get automated failure analysis. > > >> >>> The tool analyses given logs and produces text files with the extracted >>> information. There is an Emacs user interface that presents the output >>> in a nice way with added functionality such as filtering. Emacs haters >>> can use the plain text files or write another user interface. :-) >>> >>> You can get ovirt-log-analyzer from >>> https://github.com/mz-pdm/ovirt-log-analyzer >>> README.md explains how to use it. >>> >>> Note that ovirt-log-analyzer has been created within the limited >>> resources of an Outreachy internship with some additional work and not >>> everything is perfect. Feel free to make improvements. >>> >>> Regards, >>> Milan >>> _______________________________________________ >>> Devel mailing list >>> Devel at ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/devel >>> >> >> >> >> -- >> >> GREG SHEREMETA >> >> SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX >> >> Red Hat NA >> >> >> >> gshereme at redhat.com IRC: gshereme >> >> >> _______________________________________________ >> Devel mailing list >> Devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel >> > > > _______________________________________________ > Devel mailing list > Devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel > -- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted -------------- next part -------------- An HTML attachment was scrubbed... URL: From dron at redhat.com Thu Mar 29 20:20:53 2018 From: dron at redhat.com (Dafna Ron) Date: Thu, 29 Mar 2018 21:20:53 +0100 Subject: [ovirt-devel] OST Failure - Weekly update [24/03/2018-29/03/2018] Message-ID: Hello, I would like to update on this week's failures and OST current status. This week I am sending the report a bit earlier due to the holidays. We had a few failures this week but I am glad to report that the issues were fixed quickly. on 23-03-2018 we started getting a random failure which seems to be an issue with the ost test hotplug_cpu This fix was submitted by Milan to fix the issue: https://gerrit.ovirt.org/89589 On the 27-03-2018 we had a failure on basic_sanity.disk_operations This seems to be related to locking race and was fixed by Benny on 3 different fixes: https://gerrit.ovirt.org/#/c/89534/ https://gerrit.ovirt.org/#/c/89537/ https://gerrit.ovirt.org/#/c/89577/ There was an issue with build-artifacts on the vdsm project which was an infra failure as well as a code issue (2 different builds). this was also resolved quickly. Infra issues: The ci team discovered and fixed a failure in container cleanup on the the hosts. Daniel submitted this patch: https://gerrit.ovirt.org/#/c/89402/ and it seemed to have resolved the issue. Gal has been investigating an issue which causes failure due to lack of memory on the host. for now, Gal submitted a fix: https://gerrit.ovirt.org/#/c/89588/ Lastly we had a packaging issue on 4.2 which was resolved as well. *Below you can see the chart for this week's resolved issues but cause of failure:Code* = regression of working components/functionalities *Infra* = infrastructure/OST Infrastructure/Lago related issues/Power outages *OST* Tests - package related issues, failed build artifacts *Below is a chart of resolved failures based on ovirt version* *Below is a chart showing failures by suite type: * *Thanks, * *Dafna* -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 5277 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 25364 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 7006 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 29900 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 4965 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 29748 bytes Desc: not available URL: From danken at redhat.com Thu Mar 29 20:36:16 2018 From: danken at redhat.com (Dan Kenigsberg) Date: Thu, 29 Mar 2018 23:36:16 +0300 Subject: [ovirt-devel] new time-based test randomly failing In-Reply-To: References: Message-ID: On Thu, Mar 29, 2018 at 7:27 PM, Nir Soffer wrote: > > On Thu, Mar 29, 2018 at 7:17 PM Dan Kenigsberg wrote: >> >> On Wed, Mar 28, 2018 at 8:08 PM, Nir Soffer wrote: >> > On Wed, Mar 28, 2018 at 10:15 AM Dan Kenigsberg >> > wrote: >> >> >> >> I suppose this test should be reworked, or at least, have its constants >> >> tweaked. >> >> >> >> FAIL: test_slow_handler_sync (common.logutils_test.TestThreadedHandler) >> >> ---------------------------------------------------------------------- >> >> Traceback (most recent call last): >> >> File >> >> >> >> "/home/jenkins/workspace/vdsm_master_check-patch-fc27-x86_64/vdsm/tests/common/logutils_test.py", >> >> line 319, in test_slow_handler_sync >> >> self.assertGreater(max(workers_time), 0.9) >> >> AssertionError: 0.8714249134063721 not greater than 0.9 >> > >> > >> > I tried to fix this last week, but for some reason the fix does not help >> > on >> > the CI. >> > >> > I think we can remove this test, it is reproducing the issues with the >> > standard >> > logging code, and does not test the new code. >> >> Ok: https://gerrit.ovirt.org/89618 > > > Thanks, akced. That was fast. Time to mention https://gerrit.ovirt.org/#/q/status:open+project:vdsm+branch:master+topic:storage-mailbox ? From nsoffer at redhat.com Thu Mar 29 20:49:02 2018 From: nsoffer at redhat.com (Nir Soffer) Date: Thu, 29 Mar 2018 20:49:02 +0000 Subject: [ovirt-devel] new time-based test randomly failing In-Reply-To: References: Message-ID: On Thu, Mar 29, 2018 at 11:36 PM Dan Kenigsberg wrote: > On Thu, Mar 29, 2018 at 7:27 PM, Nir Soffer wrote: > > > > On Thu, Mar 29, 2018 at 7:17 PM Dan Kenigsberg > wrote: > >> > >> On Wed, Mar 28, 2018 at 8:08 PM, Nir Soffer wrote: > >> > On Wed, Mar 28, 2018 at 10:15 AM Dan Kenigsberg > >> > wrote: > >> >> > >> >> I suppose this test should be reworked, or at least, have its > constants > >> >> tweaked. > >> >> > >> >> FAIL: test_slow_handler_sync > (common.logutils_test.TestThreadedHandler) > >> >> > ---------------------------------------------------------------------- > >> >> Traceback (most recent call last): > >> >> File > >> >> > >> >> > "/home/jenkins/workspace/vdsm_master_check-patch-fc27-x86_64/vdsm/tests/common/logutils_test.py", > >> >> line 319, in test_slow_handler_sync > >> >> self.assertGreater(max(workers_time), 0.9) > >> >> AssertionError: 0.8714249134063721 not greater than 0.9 > >> > > >> > > >> > I tried to fix this last week, but for some reason the fix does not > help > >> > on > >> > the CI. > >> > > >> > I think we can remove this test, it is reproducing the issues with the > >> > standard > >> > logging code, and does not test the new code. > >> > >> Ok: https://gerrit.ovirt.org/89618 > > > > > > Thanks, akced. > > That was fast. Time to mention > > https://gerrit.ovirt.org/#/q/status:open+project:vdsm+branch:master+topic:storage-mailbox I'll try to finish with this next week. Thanks for this work! -------------- next part -------------- An HTML attachment was scrubbed... URL: From nsoffer at redhat.com Fri Mar 30 11:13:29 2018 From: nsoffer at redhat.com (Nir Soffer) Date: Fri, 30 Mar 2018 11:13:29 +0000 Subject: [ovirt-devel] [VDSM] Network test failing randomly - needs un-ordered compare? Message-ID: We have this failure in unrelated patch: http://jenkins.ovirt.org/job/vdsm_master_check-patch-el7-x86_64/22723/consoleFull The issue seems to comparing strings instead of sets. The next run was successful. I did not check if it run on the same slave. http://jenkins.ovirt.org/job/vdsm_master_check-patch-el7-x86_64/22724/ *00:08:51.242* =================================== FAILURES =================================== *00:08:51.243* ________________ LinkBondTests.test_bond_set_two_arp_ip_targets ________________*00:08:51.243* *00:08:51.243* self = *00:08:51.244* *00:08:51.244* def test_bond_set_two_arp_ip_targets(self):*00:08:51.244* OPTIONS = {*00:08:51.244* 'mode': '1',*00:08:51.245* 'arp_interval': '1000',*00:08:51.245* 'arp_ip_target': '10.1.3.1,10.1.2.1'}*00:08:51.245* *00:08:51.245* with bond_device() as bond:*00:08:51.245* bond.set_options(OPTIONS)*00:08:51.246* bond.refresh()*00:08:51.246* > self.assertEqual(bond.options, OPTIONS)*00:08:51.246* E AssertionError: {'mode': '1', 'arp_interval': '1000', 'arp_ip_target': '10.1.2.1,10.1.3.1'} != {'mode': '1', 'arp_interval': '1000', 'arp_ip_target': '10.1.3.1,10.1.2.1'}*00:08:51.248* E - {'arp_interval': '1000', 'arp_ip_target': '10.1.2.1,10.1.3.1', 'mode': '1'}*00:08:51.248* E ? ---------*00:08:51.249* E *00:08:51.249* E + {'arp_interval': '1000', 'arp_ip_target': '10.1.3.1,10.1.2.1', 'mode': '1'}*00:08:51.249* E ? +++++++++*00:08:51.250* *00:08:51.250* network/integration/link_bond_test.py:159: AssertionError*00:08:51.250* ------------------------------ Captured log call -------------------------------*00:08:51.251* sysfs_driver.py 57 INFO Bond bond_ge9DfJ has been created.*00:08:51.251* cmdutils.py 150 DEBUG /sbin/ip link set dev bond_ge9DfJ down (cwd None)*00:08:51.252* cmdutils.py 158 DEBUG SUCCESS: = ''; = 0*00:08:51.252* sysfs_driver.py 102 INFO Bond bond_ge9DfJ options set: {'mode': '1', 'arp_interval': '1000', 'arp_ip_target': '10.1.3.1,10.1.2.1'}.*00:08:51.253* sysfs_driver.py 67 INFO Bond bond_ge9DfJ has been destroyed.*00:08:51.254* =============== 1 failed, 36 passed, 12 skipped in 10.57 seconds =============== -------------- next part -------------- An HTML attachment was scrubbed... URL: From kyland\sundw at coretek.com.cn Thu Mar 29 08:49:58 2018 From: kyland\sundw at coretek.com.cn (sundw) Date: Thu, 29 Mar 2018 16:49:58 +0800 Subject: [ovirt-devel] Failed To Build ovirt-node following the guide Message-ID: <201803291649568693113@coretek.com.cn> Hello,guys? I built ovirt-node following this guide(https://www.ovirt.org/develop/projects/node/building/). But I failed. I got the following Error message after running "make iso publish": Error creating Live CD : Failed to find package 'ovirt-node-plugin-vdsm' : No package(s) available to install Could you please give some advice? BTW: Is the content of this url "https://www.ovirt.org/develop/projects/node/building/" OUT OF DATE? ??? ????????????/?????? 13378105625 -------------- next part -------------- An HTML attachment was scrubbed... URL: From kyland\sundw at coretek.com.cn Fri Mar 30 03:58:21 2018 From: kyland\sundw at coretek.com.cn (sundw) Date: Fri, 30 Mar 2018 11:58:21 +0800 Subject: [ovirt-devel] Build virt-node on fedora26 failed Message-ID: <201803301158105426622@coretek.com.cn> Hi! I build virt-node base on this guide(https://www.ovirt.org/develop/projects/node/building/). I got the following error messages after I run "make publish": Installing tmp.repos/SRPMS/ovirt-node-3.7.0-0.0.master.fc27.src.rpm error: Failed build dependencies: /usr/share/selinux/devel/policyhelp is needed by ovirt-node-3.7.0-0.0.master.fc27.noarch make[1]: *** [Makefile:813: rpm] Error 1 make[1]: Leaving directory '/home/coretek/git/ovirt-node' make: *** [Makefile:827: publish] Error 2 Can anyone give me some suggestions? Thanks! ??? ????????????/?????? 13378105625 -------------- next part -------------- An HTML attachment was scrubbed... URL: