[ovirt-devel] Re: OST gating failed on - test_import_vm1

On 15. 6. 2021, at 12:00, Eyal Shenitzky <eshenitz(a)redhat.com&gt; wrote: Hi All, As part of OST gating verification, the verification failed with the following message - gating2 (43) : OST build 6687 failed with: test_import_vm1 failed: engine = <ovirtsdk4.services.SystemService object at 0x7f9129f03668> event_id = [1165], timeout = 600 @contextlib.contextmanager def wait_for_event(engine, event_id, timeout=assertions.LONG_TIMEOUT): ''' event_id could either be an int - a single event ID or a list - multiple event IDs that all will be checked ''' events = engine.events_service() last_event = int(events.list(max=2)[0].id) try: > yield ost_utils/ost_utils/engine_utils.py:36: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ engine = <ovirtsdk4.services.SystemService object at 0x7f9129f03668> correlation_id = 'test_validate_ova_import_vm', vm_name = 'imported_vm' imported_url = 'ova:///var/tmp/ova_vm.ova', storage_domain = 'iscsi' cluster_name = 'test-cluster' def _import_ova(engine, correlation_id, vm_name, imported_url, storage_domain, cluster_name): sd = engine.storage_domains_service().list(search='name={}'.format(storage_domain))[0] cluster = engine.clusters_service().list(search='name={}'.format(cluster_name))[0] imports_service = engine.external_vm_imports_service() host = test_utils.get_first_active_host_by_name(engine) with engine_utils.wait_for_event(engine, 1165): # IMPORTEXPORT_STARTING_IMPORT_VM imports_service.add( types.ExternalVmImport( name=vm_name, provider=types.ExternalVmProviderType.KVM, url=imported_url, cluster=types.Cluster( id=cluster.id ), storage_domain=types.StorageDomain( id=sd.id ), host=types.Host( id=host.id ), sparse=True > ), async_=True, query={'correlation_id': correlation_id} ) basic-suite-master/test-scenarios/test_004_basic_sanity.py:935: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <ovirtsdk4.services.ExternalVmImportsService object at 0x7f9129d24860> import_ = <ovirtsdk4.types.ExternalVmImport object at 0x7f9129d58978> headers = None, query = {'correlation_id': 'test_validate_ova_import_vm'} wait = True, kwargs = {'async_': True} def add( self, import_, headers=None, query=None, wait=True, **kwargs ): """ This operation is used to import a virtual machine from external hypervisor, such as KVM, XEN or VMware. For example import of a virtual machine from VMware can be facilitated using the following request: [source] ---- POST /externalvmimports ---- With request body of type <<types/external_vm_import,ExternalVmImport>>, for example: [source,xml] ---- <external_vm_import> <vm> <name>my_vm</name> </vm> <cluster id="360014051136c20574f743bdbd28177fd" /> <storage_domain id="8bb5ade5-e988-4000-8b93-dbfc6717fe50" /> <name>vm_name_as_is_in_vmware</name> <sparse>true</sparse> <username>vmware_user</username> <password>123456</password> <provider>VMWARE</provider> <url> vpx://wmware_user@vcenter-host/DataCenter/Cluster/esxi-host?no_verify=1</url> <drivers_iso id="virtio-win-1.6.7.iso" /> </external_vm_import> ---- """ # Check the types of the parameters: Service._check_types([ ('import_', import_, types.ExternalVmImport), ]) # Build the URL: Patch set 4:Verified -1 The OST run as part of verification for patch - https://gerrit.ovirt.org/#/c/ovirt-engine/+/115192/ Can someone from Virt/OST team have a look? you should be able to review logs in generic way you can ee 2021-06-15 11:08:37,515+02 ERROR [org.ovirt.engine.core.bll.exportimport.ImportVmFromExternalUrlCommand] (default task-2) [test_validate_ova_import_vm] Exception: java.lang.NullPointerException at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.exportimport.ImportVmFromExternalUrlCommand$ExternalVmImporter.performImport(ImportVmFromExternalUrlCommand.java:116) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.exportimport.ImportVmFromExternalUrlCommand.executeCommand(ImportVmFromExternalUrlCommand.java:65) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1174) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1332) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:2008) likely caused by 2021-06-15 11:08:37,513+02 ERROR [org.ovirt.engine.core.bll.GetVmFromOvaQuery] (default task-2) [test_validate_ova_import_vm] Exception: org.ovirt.engine.core.common.utils.ansible.AnsibleRunnerCallException: Task Run query script failed to execute. Please check logs for more details: /var/log/ovirt-engine/ova/ovirt-query-ova-ansible-20210615110831-lago-basic-suite-master-host-0-test_validate_ova_import_vm.log seeing then the following error in ansible log: 2021-06-15 11:08:37 CEST - fatal: [lago-basic-suite-master-host-0]: FAILED! => {"changed": true, "msg": "non-zero return code", "rc": 1, "stderr": "Shared connection to lago-basic-suite-master-host-0 closed.\r\n", "stderr_lines": ["Shared connection to lago-basic-suite-master-host-0 closed."], "stdout": "Traceback (most recent call last):\r\n File \"/root/.ansible/tmp/ansible-tmp-1623748114.0795734-68052-92827534955240/query_ova.py\", line 81, in <module>\r\n ovf = get_ovf_from_dir(ova_path, sys.argv[3], templates)\r\n File \"/root/.ansible/tmp/ansible-tmp-1623748114.0795734-68052-92827534955240/query_ova.py\", line 47, in get_ovf_from_dir\r\n files = os.listdir(ova_path)\r\nFileNotFoundError: [Errno 2] No such file or directory: '/var/tmp/ova_vm.ova'\r\n", "stdout_lines": ["Traceback (most recent call last):", " File \"/root/.ansible/tmp/ansible-tmp-1623748114.0795734-68052-92827534955240/query_ova.py\", line 81, in <module>", " ovf = get_ovf_from_dir(ova_path, sys.argv[3], templates)", " File \"/root/.ansible/tmp/ansible-tmp-1623748114.0795734-68052-92827534955240/query_ova.py\", line 47, in get_ovf_from_dir", " files = os.listdir(ova_path)", "FileNotFoundError: [Errno 2] No such file or directory: '/var/tmp/ova_vm.ova'"]} so it probably wasn’t even exported….and sure, there’s 2021-06-15 11:06:27,270+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-89) [] EVENT_ID: IMPORTEXPORT_EXPORT_VM_TO_OVA_FAILED(1,225), Failed to export Vm vm2 as a Virtual Appliance to path /var/tmp/ova_vm.ova on Host lago-basic-suite-master-host-0 and looking at the export ansible log it looks like it aborted in the middle since the last line in ansible log of exort is 2021-06-15 11:06:13 CEST - TASK [ovirt-ova-export-pre-pack : Retrieving the temporary path for the OVA file] *** and then I’m a bit lost, but in the end it’s not so far fetched to suspect connection with your patch touching snapshots since that’s what’s being used here during export. So generally the best would be to rerun OST locally with this patch (or manual jenkins job, but running it locally would allow you to examine it further) and certainly virt or infra should take a look as well, since raising Interl Engine Error is never a good idea, and abruptly cur ansible log neither. Thanks, michal -- Regards, Eyal Shenitzky