Re: [ovirt-devel] [ OST Failure Report ] [ oVirt master ] [ 2017-09-07 ] [import_template_from_glance]

That was more a less the my guess from the brief look I took at the stacktrace. I don't want to +2 a patch in the network area, but I'd suggest merging this patch (which seems right regardless of the recent failure), and see if it solves the OST issue. OST will run with the network patch reverted, so I need to bring it into

> > > On Mon, 11 Sep 2017 at 21:30 Roy Golan <rgolan(a)redhat.com&gt; wrote: > >> I think I solved it with https://gerrit.ovirt.org/#/c/81636/ so the >> revert can be reverted. +Martin Mucha <mmucha(a)redhat.com&gt; want to test >> it? It is working on my end. >> >> sorry its this https://gerrit.ovirt.org/c/81637/2 > >> >> On Mon, 11 Sep 2017 at 18:41 Dan Kenigsberg <danken(a)redhat.com&gt; wrote: >> >>> On Mon, Sep 11, 2017 at 4:46 PM, Dusan Fodor <dfodor(a)redhat.com&gt; wrote: >>> > >>> > Hello all, >>> > this same error still causes OST to fail. >>> > >>> >>> We are aware of it, but do not understand it at all. >>> >>> Anyway, a revert is proposed and https://gerrit.ovirt.org/#/c/81618/ >>> would be merged once approved by CI. >>> >>> > >>> > Jobs: >>> > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2466 >>> > >>> > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2471 >>> > >>> > Logs: >>> > >>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2466/artifa... >>> > >>> > >>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2471/artifa... >>> > >>> > >>> > On Mon, Sep 11, 2017 at 1:22 PM, Eyal Edri <eedri(a)redhat.com&gt; wrote: >>> >> >>> >> >>> >> >>> >> On Mon, Sep 11, 2017 at 1:21 PM, Dan Kenigsberg <danken(a)redhat.com&gt; >>> wrote: >>> >>> >>> >>> On Mon, Sep 11, 2017 at 8:59 AM, Dan Kenigsberg <danken(a)redhat.com&gt; >>> wrote: >>> >>> > On Thu, Sep 7, 2017 at 6:12 PM, Evgheni Dereveanchin >>> >>> > <ederevea(a)redhat.com&gt; wrote: >>> >>> >> Test failed: [ import_template_from_glance ] >>> >>> >> >>> >>> >> Link to suspected patches: >>> >>> >> https://gerrit.ovirt.org/#/c/80450/ >>> >>> > >>> >>> > Martin, this is "core: when initializing MacPool also register in >>> it >>> >>> > nics in snapshots" >>> >>> > and the error seems somewhat related to it >>> >>> >>> >>> Gil, can you tell us which is the last Engine patch that passed >>> OST? I >>> >>> tried to build https://gerrit.ovirt.org/#/c/76309/ but I'm guessing >>> >>> that "ci please build" does not work on merged patches. >>> >> >>> >> >>> >> Not sure I follow, the latest working engine should be deployed to >>> tested repo, >>> >> So if you'll run the manual job w/o any custom repos it should work >>> ( if it doesn't, then it means something slipped in the tested repo and we >>> should investigate ). >>> >> >>> >> But anyhow, your 'ci please build' also worked, and produced RPMs on >>> the job, so you can still try it: >>> >> >>> >> Patch Set 26: >>> >> Build Successful >>> >> >>> http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-on-deman... >>> : SUCCESS >>> >> >>> http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-on-deman... >>> : SUCCESS >>> >> >>> >> >>> >>> >>> >>> >>> >>> > >>> >>> >> >>> >>> >> Link to Job: >>> >>> >> >>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2397/ >>> >>> >> >>> >>> >> Link to logs: >>> >>> >> >>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2397/artifa... >>> >>> >> >>> >>> >> Error snippet from log: >>> >>> >> >>> >>> >> <error> >>> >>> >> >>> >>> >> 2017-09-07 08:21:48,657-04 INFO >>> >>> >> [org.ovirt.engine.core.bll.AddVmTemplateCommand] >>> >>> >> (EE-ManagedThreadFactory-engineScheduled-Thread-21) [74b41edb] >>> Lock Acquired >>> >>> >> to object >>> >>> >> >>> 'EngineLock:{exclusiveLocks='[2c2b56b5-cac6-469d-b0e0-9744309ca8c1=TEMPLATE, >>> >>> >> CirrOS_0.3.5_for_x86_64_glance_template=TEMPLATE_NAME]', >>> sharedLocks='[]'}' >>> >>> >> 2017-09-07 08:21:48,675-04 INFO >>> >>> >> [org.ovirt.engine.core.bll.AddVmTemplateCommand] >>> >>> >> (EE-ManagedThreadFactory-engineScheduled-Thread-21) [74b41edb] >>> Running >>> >>> >> command: AddVmTemplateCommand internal: true. Entities affected >>> : ID: >>> >>> >> d279c4e9-09e7-4dd9-9eff-10b31ee2adfc Type: StoragePoolAction >>> group >>> >>> >> CREATE_TEMPLATE with role type USER >>> >>> >> 2017-09-07 08:21:48,695-04 INFO >>> >>> >> >>> [org.ovirt.engine.core.bll.storage.disk.CreateAllTemplateDisksCommand] >>> >>> >> (EE-ManagedThreadFactory-engineScheduled-Thread-21) [74b41edb] >>> Running >>> >>> >> command: CreateAllTemplateDisksCommand internal: true. >>> >>> >> 2017-09-07 08:21:48,722-04 INFO >>> >>> >> [org.ovirt.engine.core.utils.transaction.TransactionSupport] >>> >>> >> (EE-ManagedThreadFactory-engineScheduled-Thread-21) [74b41edb] >>> transaction >>> >>> >> rolled back >>> >>> >> 2017-09-07 08:21:48,722-04 ERROR >>> >>> >> [org.ovirt.engine.core.bll.AddVmTemplateCommand] >>> >>> >> (EE-ManagedThreadFactory-engineScheduled-Thread-21) [74b41edb] >>> Command >>> >>> >> 'org.ovirt.engine.core.bll.AddVmTemplateCommand' failed: null >>> >>> >> 2017-09-07 08:21:48,722-04 ERROR >>> >>> >> [org.ovirt.engine.core.bll.AddVmTemplateCommand] >>> >>> >> (EE-ManagedThreadFactory-engineScheduled-Thread-21) [74b41edb] >>> Exception: >>> >>> >> java.lang.NullPointerException >>> >>> >> at >>> >>> >> >>> org.ovirt.engine.core.bll.utils.VgamemVideoSettings.getVramMultiplier(VgamemVideoSettings.java:63) >>> >>> >> [bll.jar:] >>> >>> >> at >>> >>> >> >>> org.ovirt.engine.core.bll.utils.VgamemVideoSettings.getQxlVideoDeviceSettings(VgamemVideoSettings.java:40) >>> >>> >> [bll.jar:] >>> >>> >> at >>> >>> >> >>> org.ovirt.engine.core.bll.utils.VideoDeviceSettings.getVideoDeviceSettings(VideoDeviceSettings.java:34) >>> >>> >> [bll.jar:] >>> >>> >> at >>> >>> >> >>> org.ovirt.engine.core.bll.utils.VideoDeviceSettings.getVideoDeviceSpecParams(VideoDeviceSettings.java:49) >>> >>> >> [bll.jar:] >>> >>> >> at >>> >>> >> >>> org.ovirt.engine.core.bll.utils.VmDeviceUtils.getVideoDeviceSpecParams(VmDeviceUtils.java:586) >>> >>> >> [bll.jar:] >>> >>> >> at >>> >>> >> >>> org.ovirt.engine.core.bll.utils.VmDeviceUtils.copyVmDevices(VmDeviceUtils.java:1435) >>> >>> >> [bll.jar:] >>> >>> >> at >>> >>> >> >>> org.ovirt.engine.core.bll.utils.VmDeviceUtils.copyVmDevices(VmDeviceUtils.java:1559) >>> >>> >> [bll.jar:] >>> >>> >> >>> >>> >> ... >>> >>> >> >>> >>> >> 2017-09-07 08:21:48,739-04 INFO >>> >>> >> [org.ovirt.engine.core.bll.AddVmTemplateCommand] >>> >>> >> (EE-ManagedThreadFactory-engineScheduled-Thread-21) [74b41edb] >>> Command >>> >>> >> [id=3d633182-982a-46f2-b4f1-27c5d04397a7]: Compensating >>> NEW_ENTITY_ID of >>> >>> >> org.ovirt.engine.core.common.businessentities.VmTemplate; >>> snapshot: >>> >>> >> 2c2b56b5-cac6-469d-b0e0-9744309ca8c1. >>> >>> >> 2017-09-07 08:21:48,813-04 ERROR >>> >>> >> >>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >>> >>> >> (EE-ManagedThreadFactory-engineScheduled-Thread-21) [74b41edb] >>> EVENT_ID: >>> >>> >> USER_ADD_VM_TEMPLATE_FAILURE(36), Failed creating Template >>> >>> >> CirrOS_0.3.5_for_x86_64_glance_template. >>> >>> >> 2017-09-07 08:21:48,834-04 INFO >>> >>> >> [org.ovirt.engine.core.bll.AddVmTemplateCommand] >>> >>> >> (EE-ManagedThreadFactory-engineScheduled-Thread-21) [74b41edb] >>> Lock freed to >>> >>> >> object >>> >>> >> >>> 'EngineLock:{exclusiveLocks='[2c2b56b5-cac6-469d-b0e0-9744309ca8c1=TEMPLATE, >>> >>> >> CirrOS_0.3.5_for_x86_64_glance_template=TEMPLATE_NAME]', >>> sharedLocks='[]'}' >>> >>> >> 2017-09-07 08:21:48,848-04 DEBUG >>> >>> >> >>> [org.ovirt.engine.core.dal.dbbroker.CustomSQLErrorCodeSQLExceptionTranslator] >>> >>> >> (EE-ManagedThreadFactory-engineScheduled-Thread-21) [74b41edb] >>> Translating >>> >>> >> SQLException with SQL state '23503', error code '0', message >>> [ERROR: insert >>> >>> >> or update on table "disk_vm_element" violates foreign key >>> constraint >>> >>> >> "fk_disk_vm_element_vm_static" >>> >>> >> Detail: Key (vm_id)=(2c2b56b5-cac6-469d-b0e0-9744309ca8c1) is >>> not present >>> >>> >> in table "vm_static". >>> >>> >> Where: SQL statement "INSERT INTO disk_vm_element ( >>> >>> >> disk_id, >>> >>> >> vm_id, >>> >>> >> is_boot, >>> >>> >> pass_discard, >>> >>> >> disk_interface, >>> >>> >> is_using_scsi_reservation) >>> >>> >> VALUES ( >>> >>> >> v_disk_id, >>> >>> >> v_vm_id, >>> >>> >> v_is_boot, >>> >>> >> v_pass_discard, >>> >>> >> v_disk_interface, >>> >>> >> v_is_using_scsi_reservation)" >>> >>> >> PL/pgSQL function >>> insertdiskvmelement(uuid,uuid,boolean,boolean,character >>> >>> >> varying,boolean) line 3 at SQL statement]; SQL was [{call >>> >>> >> insertdiskvmelement(?, ?, ?, ?, ?, ?)}] for task >>> [CallableStatementCallback] >>> >>> >> 2017-09-07 08:21:48,850-04 INFO >>> >>> >> [org.ovirt.engine.core.utils.transaction.TransactionSupport] >>> >>> >> (EE-ManagedThreadFactory-engineScheduled-Thread-21) [74b41edb] >>> transaction >>> >>> >> rolled back >>> >>> >> 2017-09-07 08:21:48,877-04 INFO >>> >>> >> >>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >>> >>> >> (EE-ManagedThreadFactory-engineScheduled-Thread-21) [74b41edb] >>> EVENT_ID: >>> >>> >> USER_IMPORT_IMAGE_AS_TEMPLATE_FINISHED_SUCCESS(3,018), User >>> >>> >> admin@internal-authz successfully imported image >>> >>> >> CirrOS_0.3.5_for_x86_64_glance_template as template >>> >>> >> CirrOS_0.3.5_for_x86_64_glance_template to domain iscsi. >>> >>> >> 2017-09-07 08:21:48,878-04 ERROR >>> >>> >> [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] >>> >>> >> (EE-ManagedThreadFactory-engineScheduled-Thread-21) [74b41edb] >>> Failed >>> >>> >> invoking callback end method 'onSucceeded' for command >>> >>> >> '9c1ba73a-f6bb-4291-b1f2-d5806e38ba18' with exception >>> >>> >> 'CallableStatementCallback; SQL [{call insertdiskvmelement(?, ?, >>> ?, ?, ?, >>> >>> >> ?)}]; ERROR: insert or update on table "disk_vm_element" >>> violates foreign >>> >>> >> key constraint "fk_disk_vm_element_vm_static" >>> >>> >> Detail: Key (vm_id)=(2c2b56b5-cac6-469d-b0e0-9744309ca8c1) is >>> not present >>> >>> >> in table "vm_static". >>> >>> >> Where: SQL statement "INSERT INTO disk_vm_element ( >>> >>> >> disk_id, >>> >>> >> vm_id, >>> >>> >> is_boot, >>> >>> >> pass_discard, >>> >>> >> disk_interface, >>> >>> >> is_using_scsi_reservation) >>> >>> >> VALUES ( >>> >>> >> v_disk_id, >>> >>> >> v_vm_id, >>> >>> >> v_is_boot, >>> >>> >> v_pass_discard, >>> >>> >> v_disk_interface, >>> >>> >> v_is_using_scsi_reservation)" >>> >>> >> PL/pgSQL function >>> insertdiskvmelement(uuid,uuid,boolean,boolean,character >>> >>> >> varying,boolean) line 3 at SQL statement; nested exception is >>> >>> >> org.postgresql.util.PSQLException: ERROR: insert or update on >>> table >>> >>> >> "disk_vm_element" violates foreign key constraint >>> >>> >> "fk_disk_vm_element_vm_static" >>> >>> >> Detail: Key (vm_id)=(2c2b56b5-cac6-469d-b0e0-9744309ca8c1) is >>> not present >>> >>> >> in table "vm_static". >>> >>> >> Where: SQL statement "INSERT INTO disk_vm_element ( >>> >>> >> disk_id, >>> >>> >> vm_id, >>> >>> >> is_boot, >>> >>> >> pass_discard, >>> >>> >> disk_interface, >>> >>> >> is_using_scsi_reservation) >>> >>> >> VALUES ( >>> >>> >> v_disk_id, >>> >>> >> v_vm_id, >>> >>> >> v_is_boot, >>> >>> >> v_pass_discard, >>> >>> >> v_disk_interface, >>> >>> >> v_is_using_scsi_reservation)" >>> >>> >> PL/pgSQL function >>> insertdiskvmelement(uuid,uuid,boolean,boolean,character >>> >>> >> varying,boolean) line 3 at SQL statement', the callback is >>> marked for end >>> >>> >> method retries >>> >>> >> 2017-09-07 08:21:49,887-04 INFO >>> >>> >> [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] >>> >>> >> (EE-ManagedThreadFactory-engineScheduled-Thread-49) [74b41edb] >>> Command >>> >>> >> 'AddVmTemplate' id: '3d633182-982a-46f2-b4f1-27c5d04397a7' >>> execution didn't >>> >>> >> complete, not proceeding to perform the next operation >>> >>> >> 2017-09-07 08:21:49,887-04 INFO >>> >>> >> [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] >>> >>> >> (EE-ManagedThreadFactory-engineScheduled-Thread-49) [74b41edb] >>> Command >>> >>> >> 'AddVmTemplate' id: '3d633182-982a-46f2-b4f1-27c5d04397a7' child >>> commands >>> >>> >> '[d168e950-e9c5-4c9b-9502-bff613d7f14d]' executions were >>> completed, status >>> >>> >> 'FAILED' >>> >>> >> >>> >>> >> >>> >>> >> </error> >>> >>> >> >>> >>> >> -- >>> >>> >> Regards, >>> >>> >> Evgheni Dereveanchin >>> >>> >> >>> >>> >> _______________________________________________ >>> >>> >> Devel mailing list >>> >>> >> Devel(a)ovirt.org >>> >>> >> http://lists.ovirt.org/mailman/listinfo/devel >>> >>> _______________________________________________ >>> >>> Devel mailing list >>> >>> Devel(a)ovirt.org >>> >>> http://lists.ovirt.org/mailman/listinfo/devel >>> >>> >>> >>> >>> >> >>> >> >>> >> >>> >> -- >>> >> >>> >> Eyal edri >>> >> >>> >> >>> >> ASSOCIATE MANAGER >>> >> >>> >> RHV DevOps >>> >> >>> >> EMEA VIRTUALIZATION R&D >>> >> >>> >> >>> >> Red Hat EMEA >>> >> >>> >> TRIED. TESTED. TRUSTED. >>> >> phone: +972-9-7692018 <+972%209-769-2018> >>> >> irc: eedri (on #tlv #rhev-dev #rhev-integ) >>> >> >>> >> _______________________________________________ >>> >> Devel mailing list >>> >> Devel(a)ovirt.org >>> >> http://lists.ovirt.org/mailman/listinfo/devel >>> > >>> > >>> _______________________________________________ >>> Devel mailing list >>> Devel(a)ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/devel >>> >> > _______________________________________________ > Devel mailing list > Devel(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel >