Re: URGENT - ovirt-engine master is failing for the past 9 days

Thanks Andrej, Tal, can you merge? Thanks, Dafna On Mon, Sep 2, 2019 at 2:42 PM Andrej Krejcir <akrejcir(a)redhat.com&gt; wrote: > This patch fixes the issue: https://gerrit.ovirt.org/#/c/103019/ > > On Mon, 2 Sep 2019 at 10:16, Andrej Krejcir <akrejcir(a)redhat.com&gt; wrote: > >> I will post a fix later today. Sorry for the delay, It took me a while >> to figure out what is the problem. >> >> On Sun, 1 Sep 2019 at 14:58, Tal Nisan <tnisan(a)redhat.com&gt; wrote: >> >>> OST failing for 9 days is totally unacceptable, the only reason I'm not >>> sending a revert patch is that it'll make a bloody mess with the upgrade >>> scripts as we have two on top of it, this needs to be fixed ASAP >>> >>> On Sun, Sep 1, 2019 at 3:27 PM Dafna Ron <dron(a)redhat.com&gt; wrote: >>> >>>> Hi, >>>> >>>> We have been failing CQ master for project ovirt-engine for the past 9 >>>> days. >>>> this was reported last week and we have not yet seen a fix for this >>>> issue. >>>> >>>> the patch reported by CQ is >>>> https://gerrit.ovirt.org/#/c/101913/10 - core: Change CPU config to >>>> secure/insecure concept >>>> >>>> logs for latest failure can be found here: >>>> >>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/15638/artif... >>>> >>>> Error: >>>> 2019-09-01 06:31:17,248-04 INFO >>>> [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-1) >>>> [174bc463-9a98-441f-ad14-4d97fefde4fa] Running command: >>>> UpdateClusterCommand internal: false. Entities affected : ID: >>>> 0407bc06-4bef-4c47-99e1-0e42f9ced996 Type: ClusterAction group >>>> EDIT_CLUSTER_CONFIGURATION with role type ADMIN >>>> 2019-09-01 06:31:17,263-04 INFO >>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >>>> (default task-1) [174bc463-9a98-441f-ad14-4d97fefde4fa] EVENT_ID: >>>> CLUSTER_UPDATE_CPU_WHEN_DEPRECATED(9,029), Modified the CPU Type to Intel >>>> Haswell-noTSX Family when upgrading the Compatibility Version of Cluster >>>> test-cluster because the previous CPU Type was deprecated. >>>> 2019-09-01 06:31:17,319-04 WARN >>>> [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-1) [18fd7817] >>>> Validation of action 'UpdateVm' failed for user admin@internal-authz. >>>> Reasons: >>>> VAR__ACTION__UPDATE,VAR__TYPE__VM,ACTION_TYPE_FAILED_ILLEGAL_OS_TYPE_IS_NOT_SUPPORTED_BY_ARCHITECTURE_TYPE >>>> 2019-09-01 06:31:17,329-04 WARN >>>> [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-1) [62694a6a] >>>> Validation of action 'UpdateVm' failed for user admin@internal-authz. >>>> Reasons: >>>> VAR__ACTION__UPDATE,VAR__TYPE__VM,ACTION_TYPE_FAILED_ILLEGAL_OS_TYPE_IS_NOT_SUPPORTED_BY_ARCHITECTURE_TYPE >>>> 2019-09-01 06:31:17,334-04 WARN >>>> [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-1) [10e41cd3] >>>> Validation of action 'UpdateVm' failed for user admin@internal-authz. >>>> Reasons: >>>> VAR__ACTION__UPDATE,VAR__TYPE__VM,ACTION_TYPE_FAILED_ILLEGAL_OS_TYPE_IS_NOT_SUPPORTED_BY_ARCHITECTURE_TYPE >>>> 2019-09-01 06:31:17,345-04 WARN >>>> [org.ovirt.engine.core.bll.UpdateVmTemplateCommand] (default task-1) >>>> [26a471dd] Validation of action 'UpdateVmTemplate' failed for user >>>> admin@internal-authz. Reasons: >>>> VAR__ACTION__UPDATE,VAR__TYPE__VM_TEMPLATE,VMT_CANNOT_UPDATE_ILLEGAL_FIELD >>>> 2019-09-01 06:31:17,352-04 WARN >>>> [org.ovirt.engine.core.bll.UpdateVmTemplateCommand] (default task-1) >>>> [32fd89b5] Validation of action 'UpdateVmTemplate' failed for user >>>> admin@internal-authz. Reasons: >>>> VAR__ACTION__UPDATE,VAR__TYPE__VM_TEMPLATE,VMT_CANNOT_UPDATE_ILLEGAL_FIELD >>>> 2019-09-01 06:31:17,363-04 ERROR >>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >>>> (default task-1) [32fd89b5] EVENT_ID: >>>> CLUSTER_CANNOT_UPDATE_VM_COMPATIBILITY_VERSION(12,005), Cannot update >>>> compatibility version of Vm/Template: [vm-with-iface], Message: [No Message] >>>> 2019-09-01 06:31:17,371-04 ERROR >>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >>>> (default task-1) [32fd89b5] EVENT_ID: >>>> CLUSTER_CANNOT_UPDATE_VM_COMPATIBILITY_VERSION(12,005), Cannot update >>>> compatibility version of Vm/Template: [vm-with-iface-template], Message: >>>> [No Message] >>>> 2019-09-01 06:31:17,380-04 ERROR >>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >>>> (default task-1) [32fd89b5] EVENT_ID: >>>> CLUSTER_CANNOT_UPDATE_VM_COMPATIBILITY_VERSION(12,005), Cannot update >>>> compatibility version of Vm/Template: [vm0], Message: [No Message] >>>> 2019-09-01 06:31:17,388-04 ERROR >>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >>>> (default task-1) [32fd89b5] EVENT_ID: >>>> CLUSTER_CANNOT_UPDATE_VM_COMPATIBILITY_VERSION(12,005), Cannot update >>>> compatibility version of Vm/Template: [vm1], Message: [No Message] >>>> 2019-09-01 06:31:17,405-04 INFO >>>> [org.ovirt.engine.core.bll.CommandCompensator] (default task-1) [32fd89b5] >>>> Command [id=50898f00-4956-484a-94ff-f273adc2e93e]: Compensating >>>> UPDATED_ONLY_ENTITY of >>>> org.ovirt.engine.core.common.businessentities.Cluster; snapshot: Cluster >>>> [test-cluster]. >>>> 2019-09-01 06:31:17,422-04 ERROR >>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >>>> (default task-1) [32fd89b5] EVENT_ID: USER_UPDATE_CLUSTER_FAILED(812), >>>> Failed to update Host cluster (User: admin@internal-authz) >>>> 2019-09-01 06:31:17,422-04 INFO >>>> [org.ovirt.engine.core.bll.UpdateClusterCommand] (default task-1) >>>> [32fd89b5] Lock freed to object >>>> 'EngineLock:{exclusiveLocks='[5d3b2f0f-f05f-41c6-b61a-c517704f79fd=TEMPLATE, >>>> 728f5530-4e34-4c92-beef-ca494ec104b9=TEMPLATE]', >>>> sharedLocks='[a107316a-a961-403f-adb2-d01f22f0b8f1=VM, >>>> dc3a2ad8-6019-4e00-85e5-d9fba7390d4f=VM, >>>> 6348722f-61ae-40be-a2b8-bd9d086b06dc=VM]'}' >>>> 2019-09-01 06:31:17,424-04 ERROR >>>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default >>>> task-1) [] Operation Failed: [Update of cluster compatibility version >>>> failed because there are VMs/Templates [vm-with-iface, >>>> vm-with-iface-template, vm0, vm1] with incorrect configuration. To fix the >>>> issue, please go to each of them, edit, change the Custom Compatibility >>>> Version of the VM/Template to the cluster level you want to update the >>>> cluster to and press OK. If the save does not pass, fix the dialog >>>> validation. After successful cluster update, you can revert your Custom >>>> Compatibility Version change.] >>>> (END) >>>> >>>>