[ovirt-devel] Re: OST upgrade-from-release-suite-master update_cluster_versions failing

Adding Andrej Krejcir and Lucia = Patch https://gerrit.ovirt.org/#/c/102839/ - engine: UpdateClusterCommand cleanup is causing the regression in engine Lucia/Andrej, can you please issue a fix? Thanks for the quick report Dominik! Dafna On Tue, Aug 27, 2019 at 5:20 PM Dafna Ron <dron(a)redhat.com&gt; wrote: > I see there is an ovirt-engine package in the master queue: > https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_cha... > > we should expect it to fail > > > On Tue, Aug 27, 2019 at 5:15 PM Dafna Ron <dron(a)redhat.com&gt; wrote: > >> its failing so I guess we do have some issue in the pre-merged packages >> on cluster version >> >> On Tue, Aug 27, 2019 at 5:14 PM Dafna Ron <dron(a)redhat.com&gt; wrote: >> >>> odd.... I thought it passed... >>> >>> >>> On Tue, Aug 27, 2019 at 5:11 PM Dominik Holler <dholler(a)redhat.com&gt; >>> wrote: >>> >>>> On Tue, 27 Aug 2019 16:36:45 +0300 >>>> Dafna Ron <dron(a)redhat.com&gt; wrote: >>>> >>>> > passed with latest package: >>>> > >>>> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-te... >>>> > >>>> >>>> I don't get it. Isn't 5454 failing? >>>> >>>> >>>> > >>>> > On Tue, Aug 27, 2019 at 3:57 PM Dafna Ron <dron(a)redhat.com&gt; wrote: >>>> > >>>> > > Dominik, >>>> > > seems it passes with no changes: >>>> > > >>>> https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-te... >>>> > > the issue if exists is in engine before merge (unless we will see >>>> a failed >>>> > > CQ soon) >>>> > > I ran another manual test on the latest built package of engine so >>>> lets >>>> > > see if we have an issue that has not been identified yet. >>>> > > >>>> > > adding Dusan in case CQ starts failing >>>> > > >>>> > > >>>> > > On Tue, Aug 27, 2019 at 2:55 PM Dafna Ron <dron(a)redhat.com&gt; wrote: >>>> > > >>>> > >> Running it without a new package to see if this is reproduces on >>>> packages >>>> > >> from tested with no changes. >>>> > >> >>>> > >> >>>> > >> On Tue, Aug 27, 2019 at 1:49 PM Dominik Holler < >>>> dholler(a)redhat.com&gt; >>>> > >> wrote: >>>> > >> >>>> > >>> Hi, >>>> > >>> for me OST upgrade-from-release-suite-master >>>> update_cluster_versions >>>> > >>> is failing with >>>> > >>> >>>> > >>> 004_basic_sanity.update_cluster_versions (from nosetests) >>>> > >>> Failing for the past 1 build (Since Failed#5442 ) >>>> > >>> Took 1 min 16 sec. >>>> > >>> add description >>>> > >>> Error Message >>>> > >>> >>>> > >>> Fault reason is "Operation Failed". Fault detail is "[Update of >>>> cluster >>>> > >>> compatibility version failed because there are VMs/Templates >>>> > >>> [vm-with-iface, vm-with-iface-template, vm0, vm1] with incorrect >>>> > >>> configuration. To fix the issue, please go to each of them, >>>> edit, change >>>> > >>> the Custom Compatibility Version of the VM/Template to the >>>> cluster level >>>> > >>> you want to update the cluster to and press OK. If the save does >>>> not pass, >>>> > >>> fix the dialog validation. After successful cluster update, you >>>> can revert >>>> > >>> your Custom Compatibility Version change.]". HTTP response code >>>> is 500. >>>> > >>> >>>> > >>> Stacktrace >>>> > >>> >>>> > >>> Traceback (most recent call last): >>>> > >>> File "/usr/lib64/python2.7/unittest/case.py", line 369, in run >>>> > >>> testMethod() >>>> > >>> File "/usr/lib/python2.7/site-packages/nose/case.py", line >>>> 197, in >>>> > >>> runTest >>>> > >>> self.test(*self.arg) >>>> > >>> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", >>>> line >>>> > >>> 142, in wrapped_test >>>> > >>> test() >>>> > >>> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", >>>> line 60, >>>> > >>> in wrapper >>>> > >>> return func(get_test_prefix(), *args, **kwargs) >>>> > >>> File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", >>>> line 79, >>>> > >>> in wrapper >>>> > >>> prefix.virt_env.engine_vm().get_api(api_ver=4), *args, >>>> **kwargs >>>> > >>> File >>>> > >>> >>>> "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/upgrade-from-release-suite-master/test-scenarios-after-upgrade/004_basic_sanity.py", >>>> > >>> line 184, in update_cluster_versions >>>> > >>> minor=minor >>>> > >>> File >>>> > >>> >>>> "/home/jenkins/workspace/ovirt-system-tests_manual/ovirt-system-tests/upgrade-from-release-suite-master/test-scenarios-after-upgrade/004_basic_sanity.py", >>>> > >>> line 130, in _update_cluster_version >>>> > >>> version=new_version >>>> > >>> File >>>> "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line >>>> > >>> 3943, in update >>>> > >>> return self._internal_update(cluster, headers, query, wait) >>>> > >>> File >>>> "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line >>>> > >>> 253, in _internal_update >>>> > >>> return future.wait() if wait else future >>>> > >>> File >>>> "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line >>>> > >>> 55, in wait >>>> > >>> return self._code(response) >>>> > >>> File >>>> "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line >>>> > >>> 250, in callback >>>> > >>> self._check_fault(response) >>>> > >>> File >>>> "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line >>>> > >>> 132, in _check_fault >>>> > >>> self._raise_error(response, body) >>>> > >>> File >>>> "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line >>>> > >>> 118, in _raise_error >>>> > >>> raise error >>>> > >>> Error: Fault reason is "Operation Failed". Fault detail is >>>> "[Update of >>>> > >>> cluster compatibility version failed because there are >>>> VMs/Templates >>>> > >>> [vm-with-iface, vm-with-iface-template, vm0, vm1] with incorrect >>>> > >>> configuration. To fix the issue, please go to each of them, >>>> edit, change >>>> > >>> the Custom Compatibility Version of the VM/Template to the >>>> cluster level >>>> > >>> you want to update the cluster to and press OK. If the save does >>>> not pass, >>>> > >>> fix the dialog validation. After successful cluster update, you >>>> can revert >>>> > >>> your Custom Compatibility Version change.]". HTTP response code >>>> is 500. >>>> > >>> >>>> > >>> Please find an example run in >>>> > >>> https://jenkins.ovirt.org/view/oVirt system >>>> > >>> tests/job/ovirt-system-tests_manual/5442 >>>> > >>> >>>> > >>> Is this a known error, or is someone already addressing this >>>> issue? >>>> > >>> Thanks >>>> > >>> Dominik >>>> > >>> >>>> > >> >>>> >>>>