November 2020 - Devel - oVirt List Archives

[VDSM] Started getting virNetTLSContextCheckCertTimes:154 : The server certificate /etc/pki/vdsm/certs/vdsmcert.pem has expired
by Edward Haas 11 Jan '24

11 Jan '24

Hi, I have a VM which acts as an oVirt host with VDSM installed. I use it for testing without connecting it to Engine. Recently VDSM fails to come up and I see this error: Sep 26 17:58:18 localhost libvirtd: 2018-09-26 14:58:18.978+0000: 12176: error : virNetTLSContextCheckCertTimes:154 : The server certificate /etc/pki/vdsm/certs/vdsmcert.pem has expired Any hint on how to generate or fetch a new certificate without the help of Engine? Thanks, Edy.

3 5

Error: Adding new Host to ovirt-engine
by Ahmad Khiet 14 Aug '23

14 Aug '23

Hi, Can't add new host to ovirt engine, because the following error: 2019-06-12 12:23:09,664 p=4134 u=engine | TASK [ovirt-host-deploy-facts : Set facts] ************************************* 2019-06-12 12:23:09,684 p=4134 u=engine | ok: [10.35.1.17] => { "ansible_facts": { "ansible_python_interpreter": "/usr/bin/python2", "host_deploy_vdsm_version": "4.40.0" }, "changed": false } 2019-06-12 12:23:09,697 p=4134 u=engine | TASK [ovirt-provider-ovn-driver : Install ovs] ********************************* 2019-06-12 12:23:09,726 p=4134 u=engine | fatal: [10.35.1.17]: FAILED! => {} MSG: The conditional check 'cluster_switch == "ovs" or (ovn_central is defined and ovn_central | ipaddr and ovn_engine_cluster_version is version_compare('4.2', '>='))' failed. The error was: The ipaddr filter requires python's netaddr be installed on the ansible controller The error appears to be in '/home/engine/apps/engine/share/ovirt-engine/playbooks/roles/ovirt-provider-ovn-driver/tasks/configure.yml': line 3, column 5, but may be elsewhere in the file depending on the exact syntax problem. The offending line appears to be: - block: - name: Install ovs ^ here 2019-06-12 12:23:09,728 p=4134 u=engine | PLAY RECAP ********************************************************************* 2019-06-12 12:23:09,728 p=4134 u=engine | 10.35.1.17 : ok=3 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 whats missing!? Thanks -- Ahmad Khiet Red Hat <https://www.redhat.com/> akhiet(a)redhat.com M: +972-54-6225629 <https://red.ht/sig>

7 10

Purging inactive maintainers from vdsm-master-maintainers
by Nir Soffer 21 Jun '21

21 Jun '21

I want to remove inactive contributors from vdsm-master-maintainers. I suggest the simple rule of 2 years of inactivity for removing from this group, based on git log. See the list below for current status: https://gerrit.ovirt.org/#/admin/groups/106,members Thoughts? Nir

6 11

Merge rights changes in the oVirt Engine project
by Tal Nisan 09 Apr '21

09 Apr '21

Hi everyone, As you probably know we are now in a mode in which we develop our next zstream version on the master branch as opposed to how we worked before where the master version was dedicated for the next major version. This makes the rapid changes in master to be delivered to customers in a much higher cadence thus affecting stability. Due to that we think it's best that from now on merges in the master branch will be done only by stable branch maintainers after inspecting those closely. What you need to do in order to get your patch merged: - Have it pass Jenkins - Have it get code review +2 - Have it mark verified +1 - It's always encourage to have it tested by OST, for bigger changes it's a must Once you have all those covered, please add me as a reviewer and I'll examine it and merge if everything seems right, if I haven't done it in a timely manner feel free to ping me.

3 4

Error Java SDK Issue??
by Geschwentner, Patrick 02 Apr '21

02 Apr '21

Dear Ladies and Gentlemen! I am currently working with the java-sdk and I encountered a problem. If I would like to retrieve the disk details, I get the following error: Disk currDisk = ovirtConnection.followLink(diskAttachment.disk()); The Error is occurring in this line: [cid:image001.png@01D44537.AF127FD0] The getResponst looks quiet ok. (I inspected: [cid:image002.png@01D44537.AF127FD0] and it looks ok). Error: wrong number of arguments The code is quiet similar to what you published on github (https://github.com/oVirt/ovirt-engine-sdk-java/blob/master/sdk/src/test/jav… ). Can you confirm the defect? Best regards Patrick

6 11

CentOS Stream support
by Michal Skrivanek 01 Feb '21

01 Feb '21

Hi all, we would like to ask about interest in community about oVirt moving to CentOS Stream. There were some requests before but it’s hard to see how many people would really like to see that. With CentOS releases lagging behind RHEL for months it’s interesting to consider moving to CentOS Stream as it is much more up to date and allows us to fix bugs faster, with less workarounds and overhead for maintaining old code. E.g. our current integration tests do not really pass on CentOS 8.1 and we can’t really do much about that other than wait for more up to date packages. It would also bring us closer to make oVirt run smoothly on RHEL as that is also much closer to Stream than it is to outdated CentOS. So..would you like us to support CentOS Stream? We don’t really have capacity to run 3 different platforms, would you still want oVirt to support CentOS Stream if it means “less support” for regular CentOS? There are some concerns about Stream being a bit less stable, do you share those concerns? Thank you for your comments, michal

6 12

Dropping all el7 OST runs
by Marcin Sobczyk 15 Dec '20

15 Dec '20

Hi, since all the important suites are working on el8 already, we're planning to drop all el7 OST runs with [1] very soon. This means we can finally say goodbye to py2 and other legacy stuff! We still need to move manual OST runs not to use containers for that to happen. This effort is tracked here [2]. Regards, Marcin [1] https://gerrit.ovirt.org/112378 [2] https://ovirt-jira.atlassian.net/browse/OST-145

5 13

planned Jenkins restart
by Evgheni Dereveanchin 02 Dec '20

02 Dec '20

Hi everyone, I'll be performing a planned Jenkins restart within the next hour. No new CI jobs will be scheduled during this maintenance period. I will inform you once it is back online. -- Regards, Evgheni Dereveanchin

2 2

[OST] Error updating ost images - packages with conflicts/broken deps
by Nir Soffer 30 Nov '20

30 Nov '20

Trying dnf update on ost system fails now with: $ sudo dnf update ... Error: Problem 1: package ost-images-el8-host-deps-installed-1-202011270303.x86_64 requires ost-images-el8-upgrade = 1-202011270303, but none of the providers can be installed - cannot install both ost-images-el8-upgrade-1-202011290315.x86_64 and ost-images-el8-upgrade-1-202011270303.x86_64 - cannot install the best update candidate for package ost-images-el8-upgrade-1-202011270303.x86_64 - cannot install the best update candidate for package ost-images-el8-host-deps-installed-1-202011270303.x86_64 Problem 2: package ost-images-el8-engine-deps-installed-1-202011270303.x86_64 requires ost-images-el8-upgrade = 1-202011270303, but none of the providers can be installed - cannot install both ost-images-el8-upgrade-1-202011290315.x86_64 and ost-images-el8-upgrade-1-202011270303.x86_64 - package ost-images-el8-engine-installed-1-202011290315.x86_64 requires ost-images-el8-upgrade = 1-202011290315, but none of the providers can be installed - cannot install the best update candidate for package ost-images-el8-engine-installed-1-202011270303.x86_64 - cannot install the best update candidate for package ost-images-el8-engine-deps-installed-1-202011270303.x86_64 Problem 3: problem with installed package ost-images-el8-host-deps-installed-1-202011270303.x86_64 - package ost-images-el8-host-deps-installed-1-202011270303.x86_64 requires ost-images-el8-upgrade = 1-202011270303, but none of the providers can be installed - cannot install both ost-images-el8-upgrade-1-202011290315.x86_64 and ost-images-el8-upgrade-1-202011270303.x86_64 - package ost-images-el8-host-installed-1-202011290315.x86_64 requires ost-images-el8-upgrade = 1-202011290315, but none of the providers can be installed - cannot install the best update candidate for package ost-images-el8-host-installed-1-202011270303.x86_64 (try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages) $ sudo dnf update --nobest works, but complains about: Skipping packages with conflicts: (add '--best --allowerasing' to command line to force their upgrade): ost-images-el8-upgrade x86_64 1-202011290315 ost-images 15 M Skipping packages with broken dependencies: ost-images-el8-engine-installed x86_64 1-202011290315 ost-images 1.0 G ost-images-el8-host-installed x86_64 1-202011290315 ost-images 518 M I have not tried to run ost with the updated packages yet. Nir

2 3

Planned gerrit.ovirt.org outage 24.11.2020 from 21:00 UTC
by Evgheni Dereveanchin 30 Nov '20

30 Nov '20

Hi everyone, I will be performing planned upgrades on gerrit.ovirt.org tonight. The following services will be unreachable for up to several hours: * gerrit.ovirt.org - code review service * jenkins.ovirt.org - continuous integration service Further announcements will be sent out once the outage window opens and closes. -- Regards, Evgheni Dereveanchin

3 7

Breaking change introduced in vdsm-jsonrpc-java-1.6.0
by Artur Socha 30 Nov '20

30 Nov '20

Hi, We have recently introduced the breaking change in vdsm-jsonrpc-java [1]. Reactive streams package has been replaced with java.util.concurrent.Flow (shipped with JDK). This change is released with vdsm-jsonrpc-java-1.6.0. This change had to be reflected in ovirt-engine as well (otherwise it would not compile). I was handled by this patch [2]. Unfortunately these two changes must be in sync. Please update vdsm-jsonrpc-java to 1.6.0 if you are already working on ovirt engine revision that includes [2]. [1] https://gerrit.ovirt.org/#/c/vdsm-jsonrpc-java/+/109916/ [2] https://gerrit.ovirt.org/#/c/ovirt-engine/+/112347/ Please accept my apologies for not letting you know earlier, ideally before these patches got merged. -- Artur Socha Senior Software Engineer, RHV Red Hat

1 0

Health check endpoint in the engine hanging forever on CheckDBConnection
by Marcin Sobczyk 27 Nov '20

27 Nov '20

Hi, I'm trying out (I think) the latest build of ovirt-engine in OST [1] and the basic suite fails when we do engine reconfiguration and then restart the service [2]. After restarting we wait on the health check endpoint status here [3]. This however ends with a timeout. I tried running manually: curl -D - http://engine/ovirt-engine/services/health and that command also hangs forever. In the engine log the last related entry seem to be: 2020-11-26 16:20:15,761+01 DEBUG [org.ovirt.engine.core.utils.servlet.LocaleFilter] (default task-7) [] Incoming locale 'en-US'. Filter determined locale to be 'en-US' 2020-11-26 16:20:15,761+01 DEBUG [org.ovirt.engine.core.services.HealthStatus] (default task-7) [] Health Status servlet: entry 2020-11-26 16:20:15,761+01 DEBUG [org.ovirt.engine.core.services.HealthStatus] (default task-7) [] Calling CheckDBConnection query Has anyone else also encountered that? Regards, Marcin [1] ovirt-engine-4.4.4.3-0.0.master.20201126133903.gitc2c805a2662.el8.noarch [2] https://github.com/oVirt/ovirt-system-tests/blob/3e2fc267b376a12eda131fa0e9… [3] https://github.com/oVirt/ovirt-system-tests/blob/3e2fc267b376a12eda131fa0e9…

4 5

ovirt-engine has been tagged (ovirt-engine-4.4.4.2)
by Tal Nisan 25 Nov '20

25 Nov '20

1 0

Failure on checkInterfaces: uncategorized SQLException
by Yedidyah Bar David 25 Nov '20

25 Nov '20

Hi all, I ran OST manually with suite he-basic-master, and it failed at a rather late stage (010_local_maintenance_cli. My changes were for 008_restart_he_vm, which passed). It failed because the hosted-engine VM was already on host-1, which wasn't expected. I spent quite some time trying to understand this, and I *think* the root cause was quite before that. engine.log: 2020-11-25 09:48:50,864+01 INFO [org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand] (EE-ManagedThreadFactory-engine-Thread-273) [2e1072c7] Host setup networks finished. Lock released. Monitoring can run now for host 'lago-he-basic-suite-master-host-0.lago.local' from data-center 'Default' 2020-11-25 09:48:50,866+01 INFO [org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand] (EE-ManagedThreadFactory-engine-Thread-273) [2e1072c7] Lock freed to object 'EngineLock:{exclusiveLocks='[HOST_NETWORKfba14f41-7ece-46ca-af65-7476e9fd2018=HOST_NETWORK]', sharedLocks=''}' 2020-11-25 09:48:50,923+01 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-273) [2e1072c7] EVENT_ID: PERSIST_NETWORK_ON_HOST_FINISHED(1,146), (1/2): Successfully applied changes for network(s) Labeled_Network on host lago-he-basic-suite-master-host-0.lago.local. (User: admin@internal-authz) 2020-11-25 09:48:53,883+01 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-47) [2e1072c7] Command 'PropagateNetworksToClusterHosts' id: '6e61f53b-c2b7-4a35-a840-67076e6d66cb' child commands '[fe7f022e-c13b-4924-a6ea-cc56407e6969, ca990d6e-199b-473d-928d-0a7e6d8cf073]' executions were completed, status 'SUCCEEDED' 2020-11-25 09:48:54,891+01 INFO [org.ovirt.engine.core.bll.network.cluster.PropagateNetworksToClusterHostsCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-85) [2e1072c7] Ending command 'org.ovirt.engine.core.bll.network.cluster.PropagateNetworksToClusterHostsCommand' successfully. 2020-11-25 09:49:24,749+01 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-92) [] Failure on checkInterfaces on update runtime info for host 'lago-he-basic-suite-master-host-0.lago.local': PreparedStatementCallback; uncategorized SQLException for SQL [select * from getallnetworkbyclusterid(?, ?, ?)]; SQL state [null]; error code [0]; IJ031013: Interrupted attempting lock: org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@7d88842; nested exception is java.sql.SQLException: IJ031013: Interrupted attempting lock: org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@7d88842 2020-11-25 09:49:24,752+01 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-92) [] ResourceManager::refreshVdsRunTimeInfo: host=lago-he-basic-suite-master-host-0.lago.local(fba14f41-7ece-46ca-af65-7476e9fd2018): PreparedStatementCallback; uncategorized SQLException for SQL [select * from getvdsdynamicbyvdsid(?)]; SQL state [null]; error code [0]; IJ031013: Interrupted attempting lock: org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@7d88842; nested exception is java.sql.SQLException: IJ031013: Interrupted attempting lock: org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@7d88842 2020-11-25 09:49:24,761+01 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-92) [] Timer update runtime info failed. Exception: SQLException: IJ031013: Interrupted attempting lock: org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@7d88842 Any idea? -- Didi

1 1

VDSM CI patches stuck in queue for several hours
by Ales Musil 25 Nov '20

25 Nov '20

Hi, most of my patches lately are getting stuck in CI. For example [0] is hanging there for 17 hours waiting for a container that would be able to run nmstate functional tests. Can someone please take a look at that? Thanks, Ales [0] https://jenkins.ovirt.org/job/vdsm_standard-check-patch/25066/ -- Ales Musil Software Engineer - RHV Network Red Hat EMEA <https://www.redhat.com> amusil(a)redhat.com IM: amusil <https://red.ht/sig>

3 2

"No module named pathlib" errors in OST
by Marcin Sobczyk 24 Nov '20

24 Nov '20

Hi, there was a new version of 'importlib_metadata' library pushed to pypi. Even though it's compatible only with py3, it's picked up by py2 pip. I posted a patch to fix the version to the previous one: https://gerrit.ovirt.org/#/c/112325/ If you see this failure in other suites than basic please notify me and I will align the patch. Regards, Marcin

2 1

Branching out 4.3 in ovirt-system-tests
by Marcin Sobczyk 23 Nov '20

23 Nov '20

Hi all, after minimizing the usage of lago in basic suite, and some minor adjustments in the network suite, we are finally able to remove lago OST plugin as a dependency [1]. This however comes with a price of keeping lots of ugly ifology, i.e. [2][3]. There's big disparity between OST runs we have on el7 and el8. There's also tons of symlink-based code sharing between suites - be it 4.3 suites and master suites or simply different types of suites. The basic suite has its own 'test_utils', which is copied/symlinked in multiple places. There's also 'ost_utils', which is really messy ATM. It's very hard to keep track and maintain all of this... At this moment, we are able to run basic suite and network suite on el8, with prebuilt ost-images and without lago plugin. HE suites should be the next step. We have patches that make them py3-compatible that probably still need some attention [4][5]. We don't have any prebuilt HE ost-images, but this will be handled in the nearest future. I think it's good time to detach ourselves from the legacy stuff and start with a clean slate. My proposition would be to branch out 4.3 in ovirt-system-tests and not use py2/el7 in the master branch at all. This would allow us to focus on py3, el8 and ost-images efforts while keeping the legacy stuff intact. WDYT? Regards, Marcin [1] https://gerrit.ovirt.org/#/c/111643/ [2] https://gerrit.ovirt.org/#/c/111643/6/basic-suite-master/control.sh [3] https://gerrit.ovirt.org/#/c/111643/6/basic-suite-master/test-scenarios/con… [4] https://gerrit.ovirt.org/108809 [5] https://gerrit.ovirt.org/110097

3 8

Can't find storage server connection
by francesco＠shellrent.com 23 Nov '20

23 Nov '20

Hi all, I'm using oVirt SDK python for retrieving info about storage domain, in several hosts (centos7/ovirt4.3 and centos8/ovirt4.4), but the script exits with the following error in some of them: Traceback (most recent call last): File "get_uuid.py", line 70, in <module> storage_domain = sds_service.list(search='name=data-foo')[0] File "/root/.local/lib/python2.7/site-packages/ovirtsdk4/services.py", line 26296, in list return self._internal_get(headers, query, wait) File "/root/.local/lib/python2.7/site-packages/ovirtsdk4/service.py", line 211, in _internal_get return future.wait() if wait else future File "/root/.local/lib/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait return self._code(response) File "/root/.local/lib/python2.7/site-packages/ovirtsdk4/service.py", line 208, in callback self._check_fault(response) File "/root/.local/lib/python2.7/site-packages/ovirtsdk4/service.py", line 132, in _check_fault self._raise_error(response, body) File "/root/.local/lib/python2.7/site-packages/ovirtsdk4/service.py", line 118, in _raise_error raise error ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is "Can't find storage server connection for id '92444a95-0be7-4589-ac46-1ed6dfe7ed4c'.". HTTP response code is 500. The portion of the script that search for the storage domain is the following: sds_service = connection.system_service().storage_domains_service() storage_domain = sds_service.list(search='name={}'.format(storage_domain_name))[0] Now: I have no real clue on which ID "92444a95-0be7-4589-ac46-1ed6dfe7ed4c'" but digging in the engine logs it refers to StorageServerConnections ID: [root@ovirt-engine ovirt-engine]# zgrep 92444a95-0be7-4589-ac46-1ed6dfe7ed4c *.gz engine.log-20201108.gz:2020-11-07 06:05:54,352+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-79) [3ffb810c] START, ConnectStorageServerVDSCommand(HostName = another-server.foo.com, StorageServerConnectionManagementVDSParameters:{hostId='7d202bc7-002b-4426-8446-99b6b346874e', storagePoolId='82d0b3de-0334-451c-8321-c3533de9a894', storageType='LOCALFS', connectionList='[StorageServerConnections:{id='92444a95-0be7-4589-ac46-1ed6dfe7ed4c', connection='/data', iqn='null', vfsType='null', mountOptions='null', nfsVersion='null', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}]', sendNetworkEventOnFailure='true'}), log id: 1509fe17 As I said, I tried to execute the script from several hosts: some of them with oVirt 4.3 and other with oVirt 4.4 but it can run, or not, on both of versions. I also did the following test: I tried execute the script on different host passing different data storage: - let's say that on host foo-host1, executing the script querying for its own data storage named foo-data1 works correctly; - and let's say that on host foo-host2, the same script querying for its own data storage foo-data2 return the mentioned error; If I execute the script on foo-host2 querying the foo-data1 domain data, I got the error. If I execute on foo-host1 querying foo-data2, I still got the error. When I try to manage the storage domain via ovirt-engine GUI on the hosts that the script exit with the mentioned error, I recieve the following error: Uncaught exception occurred. Please try reloading the page. Details: (TypeError) : Cannot read property 'a' of null Please have your administrator check the UI logs The logs: 2020-11-23 09:59:31,934+01 ERROR [org.ovirt.engine.ui.frontend.server.gwt.OvirtRemoteLoggingService] (default task-239) [] Permutation name: 68A62BEB12822F65FE66B14A9E16480A 2020-11-23 09:59:31,934+01 ERROR [org.ovirt.engine.ui.frontend.server.gwt.OvirtRemoteLoggingService] (default task-239) [] Uncaught exception: com.google.gwt.core.client.JavaScriptException: (TypeError) : Cannot read property 'a' of null at org.ovirt.engine.ui.uicommonweb.models.storage.FileStorageModel.$lambda$0(FileStorageModel.java:34) at org.ovirt.engine.ui.uicommonweb.models.storage.FileStorageModel$lambda$0$Type.onSuccess(FileStorageModel.java:34) at org.ovirt.engine.ui.frontend.Frontend$1.$onSuccess(Frontend.java:227) at org.ovirt.engine.ui.frontend.Frontend$1.onSuccess(Frontend.java:227) at org.ovirt.engine.ui.frontend.communication.OperationProcessor$1.$onSuccess(OperationProcessor.java:133) at org.ovirt.engine.ui.frontend.communication.OperationProcessor$1.onSuccess(OperationProcessor.java:133) at org.ovirt.engine.ui.frontend.communication.GWTRPCCommunicationProvider$5$1.$onSuccess(GWTRPCCommunicationProvider.java:270) at org.ovirt.engine.ui.frontend.communication.GWTRPCCommunicationProvider$5$1.onSuccess(GWTRPCCommunicationProvider.java:270) at com.google.gwt.user.client.rpc.impl.RequestCallbackAdapter.onResponseReceived(RequestCallbackAdapter.java:198) at com.google.gwt.http.client.Request.$fireOnResponseReceived(Request.java:233) at com.google.gwt.http.client.RequestBuilder$1.onReadyStateChange(RequestBuilder.java:409) at Unknown.eval(webadmin-0.js) at com.google.gwt.core.client.impl.Impl.apply(Impl.java:306) at com.google.gwt.core.client.impl.Impl.entry0(Impl.java:345) at Unknown.eval(webadmin-0.js) Any slightest idea on what is going on? Thank you for your time and help, Francesco

1 0

[OST] Testing master suite on using python 2.7?!
by Nir Soffer 19 Nov '20

19 Nov '20

I'm trying to add a test module for image transfer: https://gerrit.ovirt.org/c/112274/ The test use concurrent.futures module form the standard library. This module is not available in python 2.7 but we don't support 2.7 in master and it was EOL since Jan 2020. The test fail when starting the suite: [2020-11-18T16:17:10.526Z] ============================= test session starts ============================== [2020-11-18T16:17:10.526Z] platform linux2 -- Python 2.7.5, pytest-4.6.9, py-1.9.0, pluggy-0.13.1 -- /usr/bin/python2 ... [2020-11-18T16:17:11.105Z] ==================================== ERRORS ==================================== [2020-11-18T16:17:11.105Z] ___ ERROR collecting basic-suite-master/test-scenarios/008_image_transfer.py ___ [2020-11-18T16:17:11.105Z] ImportError while importing test module '/home/jenkins/agent/workspace/ovirt-system-tests_standard-check-patch/ovirt-system-tests/basic-suite-master/test-scenarios/008_image_transfer.py' [2020-11-18T16:17:11.105Z] Hint: make sure your test modules/packages have valid Python names. [2020-11-18T16:17:11.105Z] Traceback: [2020-11-18T16:17:11.105Z] ../basic-suite-master/test-scenarios/008_image_transfer.py:22: in <module> [2020-11-18T16:17:11.105Z] import concurrent.futures [2020-11-18T16:17:11.105Z] E ImportError: No module named concurrent.futures Should we use pytest.skip() to skip this test when running on python 2? Or just remove the python 2 build, I don't have any idea why we run master code python 2. Nir

2 3

vdsm CI is failing
by Marcin Sobczyk 18 Nov '20

18 Nov '20

Hi, vdsm CI is failing with: [2020-11-16T10:51:40.132Z] No match for argument: ovirt-imageio-client-2.1.1 [2020-11-16T10:51:40.132Z] Error: Unable to find a match: ovirt-imageio-client-2.1.1 example run: https://jenkins.ovirt.org/blue/organizations/jenkins/vdsm_standard-check-pa… could you please take a look at it? Regards, Marcin

3 3

ovirt-engine has been tagged (ovirt-engine-4.4.4.1)
by Tal Nisan 18 Nov '20

18 Nov '20

1 0

About cluster level 4.5 and oVirt 4.4.3 release notes
by Sandro Bonazzola 18 Nov '20

18 Nov '20

Hi, it has been raised today on #ovirt channel that there's no mention of the 4.5 cluster level new features and about its requirements on RHEL 8.3 in release notes at https://ovirt.org/release/4.4.3/ Can you please send an update to the oVirt website adding the missing information? Thanks, -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*

2 5

[OST] Network suites fail CI builds
by Nir Soffer 17 Nov '20

17 Nov '20

I had many failures in recent OST patches, so I posted this change: https://gerrit.ovirt.org/c/112174/ This patch does not change anything, but it modifies the lago vm configuration so it triggers 8 jobs. 2 network test suites failed: https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_sta… Can someone look at the network suite failures? If these suites are not stable, we should not included them in the CI for OST patches, or mark them as expected failures so they do not fail the build. I triggered another build since I see lot of random failures in other suites. Nir

4 6

expect OST failures on el8 for the next couple of hours
by Marcin Sobczyk 16 Nov '20

16 Nov '20

Hi all, while working on cleanup procedures for ost-images I accidentally removed the RPMs from the templates server. Not my day... I'm rebuilding them right now and will republish them ASAP, but you if you try running el8-based basic/networking suites you will probably end up with a failure. Hopefully when the cleanup patch is ready I will never have to do this manually again and will avoid mistakes like this in the future. Very sorry for the inconvenience. Regards, Marcin

1 0

bz 1875386 openssl conf files point at qemu-ca-certificate: status
by Yedidyah Bar David 15 Nov '20

15 Nov '20

Hi all, I now finished verifying my patches, currently pushed to github [1]. Verified clean engine-setup and engine-setup upgrade from current master. Didn't verify adding hosts nor encrypted migration between them, but I think it should work. With the top patch there, also verified pki-enroll-pkcs12.sh, which verifies also pki-enroll-request.sh, with both default and --ca-file=qemu-ca. Now: Currently, that's 16 patches. The entire stack diff ('git diff' from master) is 580 lines, whereas with the current split ('git log '--format=%h %s' -u' from master) it's 1045 lines (~ twice), but perhaps easier to follow, and shows my thought/refinement process, although probably many comments you might have on earlier patches I already addressed in later ones. I also do not think we must have all 16 patches add all the extra load on CI, so didn't yet push to gerrit. So I wonder if just to squash them to a single one, or something in between, both for review and for eventual merging. We definitely need all of them merged at once, or if we want to prevent breakage and still merge splitted, quite a lot more work (which I don't have time for). Comments are welcome. [1] https://github.com/didib/ovirt-engine/commits/qemu-ca-fixes Best regards, -- Didi

2 2

Re: [oVirt Jenkins] ovirt-system-tests_he-basic-suite-master - Build # 1818 - Failure!
by Yedidyah Bar David 15 Nov '20

15 Nov '20

On Sat, Nov 14, 2020 at 5:29 AM <jenkins(a)jenkins.phx.ovirt.org> wrote: > > Project: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/ > Build: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/1818/ https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/1818… Error Message 'module' object has no attribute 'guest_os_image_name' Stacktrace Traceback (most recent call last): File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/loader.py", line 418, in loadTestsFromName addr.filename, addr.module) File "/usr/lib/python2.7/site-packages/nose/importer.py", line 47, in importFromPath return self.importFromDir(dir_path, fqname) File "/usr/lib/python2.7/site-packages/nose/importer.py", line 94, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "/home/jenkins/agent/workspace/ovirt-system-tests_he-basic-suite-master/ovirt-system-tests/he-basic-suite-master/test-scenarios/002_bootstrap.py", line 88, in <module> GUEST_IMAGE_NAME = versioning.guest_os_image_name() AttributeError: 'module' object has no attribute 'guest_os_image_name' Seems to be caused by: https://gerrit.ovirt.org/112196 Can you please have a look? Thanks. Best regards, -- Didi

1 0

ovirt-engine has been tagged (ovirt-engine-4.4.3.12)
by Tal Nisan 13 Nov '20

13 Nov '20

2 1

Travis builds
by Vojtech Juranek 11 Nov '20

11 Nov '20

Hi, recently, I noticed Travis builds waits in the queue for a very long time. Looking around I found out Travis probably reduced resources for OSS projects. Also I found out all projects should migrate from travis-ci.org to travis- ci.com by end of December 2020 [1] (see Q. When will the migration from travis-ci.org to travis-ci.com be completed?). Recently they announce new pricing model [2] when OSS projects will get some initial credit and after that either has to pay or ask the support for another credit (see section "Building on a public repositories only" in [2]). Maybe time to migrate away from travis-ci to something else, e.g. GH Actions? Thoughts? Vojta [1] https://docs.travis-ci.com/user/migrate/open-source-repository-migration [2] https://blog.travis-ci.com/2020-11-02-travis-ci-new-billing

3 4

ovirt-engine has been tagged (ovirt-engine-4.4.4)
by Tal Nisan 11 Nov '20

11 Nov '20

1 0

[ANN] oVirt 4.4.3 is now generally available
by Sandro Bonazzola 10 Nov '20

10 Nov '20

oVirt 4.4.3 is now generally available The oVirt project is excited to announce the general availability of oVirt 4.4.3 , as of November 10th, 2020. This release unleashes an altogether more powerful and flexible open source virtualization solution that encompasses hundreds of individual changes and a wide range of enhancements across the engine, storage, network, user interface, and analytics, as compared to oVirt 4.3. Important notes before you install / upgrade Please note that oVirt 4.4 only supports clusters and data centers with compatibility version 4.2 and above. If clusters or data centers are running with an older compatibility version, you need to upgrade them to at least 4.2 (4.3 is recommended). Please note that in RHEL 8 / CentOS 8 several devices that worked on EL7 are no longer supported. For example, the megaraid_sas driver is removed. If you use Enterprise Linux 8 hosts you can try to provide the necessary drivers for the deprecated hardware using the DUD method (See the users’ mailing list thread on this at https://lists.ovirt.org/archives/list/users@ovirt.org/thread/NDSVUZSESOXEFJ… ) Documentation - If you want to try oVirt as quickly as possible, follow the instructions on the Download <https://ovirt.org/download/> page. - For complete installation, administration, and usage instructions, see the oVirt Documentation <https://ovirt.org/documentation/>. - For upgrading from a previous version, see the oVirt Upgrade Guide <https://ovirt.org/documentation/upgrade_guide/>. - For a general overview of oVirt, see About oVirt <https://ovirt.org/community/about.html>. What’s new in oVirt 4.4.3 Release? This update is the third in a series of stabilization updates to the 4.4 series. This release is available now on x86_64 architecture for: - Red Hat Enterprise Linux 8.2 / 8.3 - CentOS Linux (or similar) 8.2 - CentOS Stream (tech preview) This release supports Hypervisor Hosts on x86_64 and ppc64le architectures for: - Red Hat Enterprise Linux 8.2 / 8.3 - CentOS Linux (or similar) 8.2 - CentOS Stream (tech preview) Please note that this release introduces cluster compatibility level 4.5 which is supported only with Red Hat Enterprise Linux 8.3 or equivalent. oVirt Node and Appliance have been updated, including: - oVirt 4.4.3: https://www.ovirt.org/release/4.4.3/ - Ansible 2.9.15: https://github.com/ansible/ansible/blob/stable-2.9/changelogs/CHANGELOG-v2.… - Glusterfs 7.8: https://docs.gluster.org/en/latest/release-notes/7.8/ See the release notes [1] for installation instructions and a list of new features and bugs fixed. Notes: - oVirt Appliance is already available for CentOS Linux 8 - oVirt Node NG will be available soon for CentOS Linux 8 Additional resources: - Read more about the oVirt 4.4.3 release highlights: https://www.ovirt.org/release/4.4.3/ - Get more oVirt project updates on Twitter: https://twitter.com/ovirt - Check out the latest project news on the oVirt blog: http://www.ovirt.org/blog/ [1] https://www.ovirt.org/release/4.4.3/ <https://www.ovirt.org/release/4.4.3/> [2] https://resources.ovirt.org/pub/ovirt-4.4/iso/ -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

1 0

Change compatibility level for data center/cluster during upgrade - what exactly are we changing?
by Patrick Chiang 10 Nov '20

10 Nov '20

Hi all, From the upgrade procedure, when we completed most upgrade tasks, we have to change the compatibility level in cluster and data center, and then restart all guest VM afterwards.It seems no down time expected with these 2 changes from the procedure. I wonder what exactly we are changing (a setting in the database/file??) and most important of all, whether there will be a few seconds down time when we made those changes to "data center"? Because I worried if a data center change could lead to storage domain's down time and eventually force hypervisors, guest VM restart before expected time. (My hosts/guest VM directly use HP 3par's storage as boot disks/data disks...) Hope anyone can share any experience on this. Thank you, Patrick Chiang

2 1

[OST] Network suites fails, 6+ days runtime?
by Nir Soffer 09 Nov '20

09 Nov '20

I have a trivial patch: https://gerrit.ovirt.org/c/112068 This should not have any effect on the network tests, but I ran the tests twice, and the network suite fails in both runs: - https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_sta… - https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_sta… The first run waited in the queue, 1 day 12 hours then ran for 11 hours. The second run waited in the queue for 2 days 21 hours and then ran for 19 hours. This is a total of 135 hours waiting for test results! Looks like we don't have the resources to test OST patches. The current way testing 8 suites for every patch is not workable. Can someone look at the network suite failure? Should we remove it from the test matrix? Nir

3 4

How to set up a (rh)el8 machine for running OST
by Marcin Sobczyk 09 Nov '20

09 Nov '20

Hi All, there are multiple pieces of information floating around on how to set up a machine for running OST. Some of them outdated (like dealing with el7), some of them more recent, but still a bit messy. Not long ago, in some email conversation, Milan presented an ansible playbook that provided the steps necessary to do that. We've picked up the playbook, tweaked it a bit, made a convenience shell script wrapper that runs it, and pushed that into OST project [1]. This script, along with the playbook, should be our single-source-of-truth, one-stop solution for the job. It's been tested by a couple of persons and proved to be able to set up everything on a bare (rh)el8 machine. If you encounter any problems with the script please either report it on the devel mailing list, directly to me, or simply file a patch. Let's keep it maintained. Regards, Marcin [1] https://gerrit.ovirt.org/#/c/111749/

6 14

virt-sparsify failed (was: [oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly - Build # 479 - Failure!)
by Yedidyah Bar David 09 Nov '20

09 Nov '20

Hi all, On Mon, Oct 12, 2020 at 5:17 AM <jenkins(a)jenkins.phx.ovirt.org> wrote: > > Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/ > Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… Above failed with: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… vdsm.log has: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly… 2020-10-11 22:05:14,695-0400 INFO (jsonrpc/1) [api.host] FINISH getJobs return={'jobs': {'05eaea44-7e4c-4442-9926-2bcb696520f1': {'id': '05eaea44-7e4c-4442-9926-2bcb696520f1', 'status': 'failed', 'description': 'sparsify_volume', 'job_type': 'storage', 'error': {'code': 100, 'message': 'General Exception: (\'Command [\\\'/usr/bin/virt-sparsify\\\', \\\'--machine-readable\\\', \\\'--in-place\\\', \\\'/rhev/data-center/mnt/192.168.200.4:_exports_nfs_share1/8b292c13-fd8a-4a7c-903c-5724ec742c10/images/a367c179-2ac9-4930-abeb-848229f81c97/515fcf06-8743-45d1-9af8-61a0c48e8c67\\\'] failed with rc=1 out=b\\\'3/12\\\\n{ "message": "libguestfs error: guestfs_launch failed.\\\\\\\\nThis usually means the libguestfs appliance failed to start or crashed.\\\\\\\\nDo:\\\\\\\\n export LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1\\\\\\\\nand run the command again. For further information, read:\\\\\\\\n http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs\\\\\\\\nYou can also run \\\\\\\'libguestfs-test-tool\\\\\\\' and post the *complete* output\\\\\\\\ninto a bug report or message to the libguestfs mailing list.", "timestamp": "2020-10-11T22:05:08.397538670-04:00", "type": "error" }\\\\n\\\' err=b"virt-sparsify: error: libguestfs error: guestfs_launch failed.\\\\nThis usually means the libguestfs appliance failed to start or crashed.\\\\nDo:\\\\n export LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1\\\\nand run the command again. For further information, read:\\\\n http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs\\\\nYou can also run \\\'libguestfs-test-tool\\\' and post the *complete* output\\\\ninto a bug report or message to the libguestfs mailing list.\\\\n\\\\nIf reporting bugs, run virt-sparsify with debugging enabled and include the \\\\ncomplete output:\\\\n\\\\n virt-sparsify -v -x [...]\\\\n"\',)'}}}, 'status': {'code': 0, 'message': 'Done'}} from=::ffff:192.168.201.4,43318, flow_id=365642f4-2fe2-45df-937a-f4ca435eea38 (api:54) 2020-10-11 22:05:14,695-0400 DEBUG (jsonrpc/1) [jsonrpc.JsonRpcServer] Return 'Host.getJobs' in bridge with {'05eaea44-7e4c-4442-9926-2bcb696520f1': {'id': '05eaea44-7e4c-4442-9926-2bcb696520f1', 'status': 'failed', 'description': 'sparsify_volume', 'job_type': 'storage', 'error': {'code': 100, 'message': 'General Exception: (\'Command [\\\'/usr/bin/virt-sparsify\\\', \\\'--machine-readable\\\', \\\'--in-place\\\', \\\'/rhev/data-center/mnt/192.168.200.4:_exports_nfs_share1/8b292c13-fd8a-4a7c-903c-5724ec742c10/images/a367c179-2ac9-4930-abeb-848229f81c97/515fcf06-8743-45d1-9af8-61a0c48e8c67\\\'] failed with rc=1 out=b\\\'3/12\\\\n{ "message": "libguestfs error: guestfs_launch failed.\\\\\\\\nThis usually means the libguestfs appliance failed to start or crashed.\\\\\\\\nDo:\\\\\\\\n export LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1\\\\\\\\nand run the command again. For further information, read:\\\\\\\\n http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs\\\\\\\\nYou can also run \\\\\\\'libguestfs-test-tool\\\\\\\' and post the *complete* output\\\\\\\\ninto a bug report or message to the libguestfs mailing list.", "timestamp": "2020-10-11T22:05:08.397538670-04:00", "type": "error" }\\\\n\\\' err=b"virt-sparsify: error: libguestfs error: guestfs_launch failed.\\\\nThis usually means the libguestfs appliance failed to start or crashed.\\\\nDo:\\\\n export LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1\\\\nand run the command again. For further information, read:\\\\n http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs\\\\nYou can also run \\\'libguestfs-test-tool\\\' and post the *complete* output\\\\ninto a bug report or message to the libguestfs mailing list.\\\\n\\\\nIf reporting bugs, run virt-sparsify with debugging enabled and include the \\\\ncomplete output:\\\\n\\\\n virt-sparsify -v -x [...]\\\\n"\',)'}}} (__init__:356) /var/log/messages has: Oct 11 22:04:51 lago-basic-suite-master-host-0 kvm[80601]: 1 guest now active Oct 11 22:05:06 lago-basic-suite-master-host-0 journal[80557]: Domain id=1 name='guestfs-hl0ntvn92rtkk2u0' uuid=05ea5a53-562f-49f8-a8ca-76b45c5325b4 is tainted: custom-argv Oct 11 22:05:06 lago-basic-suite-master-host-0 journal[80557]: Domain id=1 name='guestfs-hl0ntvn92rtkk2u0' uuid=05ea5a53-562f-49f8-a8ca-76b45c5325b4 is tainted: host-cpu Oct 11 22:05:06 lago-basic-suite-master-host-0 kvm[80801]: 2 guests now active Oct 11 22:05:08 lago-basic-suite-master-host-0 journal[80557]: internal error: End of file from qemu monitor Oct 11 22:05:08 lago-basic-suite-master-host-0 kvm[80807]: 1 guest now active Oct 11 22:05:08 lago-basic-suite-master-host-0 journal[80557]: cannot resolve symlink /tmp/libguestfseTG8xF/console.sock: No such file or directoryKN<F3>L^? Oct 11 22:05:08 lago-basic-suite-master-host-0 journal[80557]: cannot resolve symlink /tmp/libguestfseTG8xF/guestfsd.sock: No such file or directoryKN<F3>L^? Oct 11 22:05:08 lago-basic-suite-master-host-0 journal[80557]: cannot lookup default selinux label for /tmp/libguestfs1WkcF7/overlay1.qcow2 Oct 11 22:05:08 lago-basic-suite-master-host-0 journal[80557]: cannot lookup default selinux label for /var/tmp/.guestfs-36/appliance.d/kernel Oct 11 22:05:08 lago-basic-suite-master-host-0 journal[80557]: cannot lookup default selinux label for /var/tmp/.guestfs-36/appliance.d/initrd Oct 11 22:05:08 lago-basic-suite-master-host-0 vdsm[74096]: ERROR Job '05eaea44-7e4c-4442-9926-2bcb696520f1' failed#012Traceback (most recent call last):#012 File "/usr/lib/python3.6/site-packages/vdsm/jobs.py", line 159, in run#012 self._run()#012 File "/usr/lib/python3.6/site-packages/vdsm/storage/sdm/api/sparsify_volume.py", line 57, in _run#012 virtsparsify.sparsify_inplace(self._vol_info.path)#012 File "/usr/lib/python3.6/site-packages/vdsm/virtsparsify.py", line 40, in sparsify_inplace#012 commands.run(cmd)#012 File "/usr/lib/python3.6/site-packages/vdsm/common/commands.py", line 101, in run#012 raise cmdutils.Error(args, p.returncode, out, err)#012vdsm.common.cmdutils.Error: Command ['/usr/bin/virt-sparsify', '--machine-readable', '--in-place', '/rhev/data-center/mnt/192.168.200.4:_exports_nfs_share1/8b292c13-fd8a-4a7c-903c-5724ec742c10/images/a367c179-2ac9-4930-abeb-848229f81c97/515fcf06-8743-45d1-9af8-61a0c48e8c67'] failed with rc=1 out=b'3/12\n{ "message": "libguestfs error: guestfs_launch failed.\\nThis usually means the libguestfs appliance failed to start or crashed.\\nDo:\\n export LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1\\nand run the command again. For further information, read:\\n http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs\\nYou can also run \'libguestfs-test-tool\' and post the *complete* output\\ninto a bug report or message to the libguestfs mailing list.", "timestamp": "2020-10-11T22:05:08.397538670-04:00", "type": "error" }\n' err=b"virt-sparsify: error: libguestfs error: guestfs_launch failed.\nThis usually means the libguestfs appliance failed to start or crashed.\nDo:\n export LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1\nand run the command again. For further information, read:\n http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs\nYou can also run 'libguestfs-test-tool' and post the *complete* output\ninto a bug report or message to the libguestfs mailing list.\n\nIf reporting bugs, run virt-sparsify with debugging enabled and include the \ncomplete output:\n\n virt-sparsify -v -x [...]\n" The next run of the job (480) did finish successfully. No idea if it was already fixed by a patch, or is simply a random/env issue. Is it possible to pass LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1 without patching vdsm? Not sure. In any case, if this does not cause too much excess debug logging, perhaps better always pass it, to help retroactively analyze such failures. Or, patch virt-sparsify/libguestfs/whatever to always log at least enough information on failure even without passing these. Best regards, > Build Number: 479 > Build Status: Failure > Triggered By: Started by timer > > ------------------------------------- > Changes Since Last Success: > ------------------------------------- > Changes for Build #479 > [hbraha] network: bond active slave test > > > > > ----------------- > Failed Tests: > ----------------- > 1 tests failed. > FAILED: basic-suite-master.test-scenarios.004_basic_sanity.test_sparsify_disk1 > > Error Message: > AssertionError: False != True after 600 seconds > > Stack Trace: > api_v4 = <ovirtsdk4.Connection object at 0x7fe717c60e50> > > @order_by(_TEST_LIST) > def test_sparsify_disk1(api_v4): > engine = api_v4.system_service() > disk_service = test_utils.get_disk_service(engine, DISK1_NAME) > with test_utils.TestEvent(engine, 1325): # USER_SPARSIFY_IMAGE_START event > disk_service.sparsify() > > with test_utils.TestEvent(engine, 1326): # USER_SPARSIFY_IMAGE_FINISH_SUCCESS > > pass > > ../basic-suite-master/test-scenarios/004_basic_sanity.py:295: > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > /usr/lib64/python2.7/contextlib.py:24: in __exit__ > self.gen.next() > ../ost_utils/ost_utils/engine_utils.py:44: in wait_for_event > lambda: > ../ost_utils/ost_utils/assertions.py:97: in assert_true_within_long > assert_equals_within_long(func, True, allowed_exceptions) > ../ost_utils/ost_utils/assertions.py:82: in assert_equals_within_long > func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > > func = <function <lambda> at 0x7fe7176dfb18>, value = True, timeout = 600 > allowed_exceptions = [], initial_wait = 0 > error_message = 'False != True after 600 seconds' > > def assert_equals_within( > func, value, timeout, allowed_exceptions=None, initial_wait=10, > error_message=None > ): > allowed_exceptions = allowed_exceptions or [] > with _EggTimer(timeout) as timer: > while not timer.elapsed(): > try: > res = func() > if res == value: > return > except Exception as exc: > if _instance_of_any(exc, allowed_exceptions): > time.sleep(3) > continue > > LOGGER.exception("Unhandled exception in %s", func) > raise > > if initial_wait == 0: > time.sleep(3) > else: > time.sleep(initial_wait) > initial_wait = 0 > try: > if error_message is None: > error_message = '%s != %s after %s seconds' % (res, value, timeout) > > raise AssertionError(error_message) > E AssertionError: False != True after 600 seconds > > ../ost_utils/ost_utils/assertions.py:60: AssertionError_______________________________________________ > Infra mailing list -- infra(a)ovirt.org > To unsubscribe send an email to infra-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/65FECSE7EBWNS… -- Didi

5 14

OST check-patch jobs are stuck on archive artifacts stage
by Amit Bawer 08 Nov '20

08 Nov '20

Hi, Exemplified here, started last night and still hasn't finished archiving: https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/13195… Thanks.

2 2

Testing image transfer and backup with OST environment
by Nir Soffer 08 Nov '20

08 Nov '20

I want to share useful info from the OST hackathon we had this week. Image transfer must work with real hostnames to allow server certificate verification. Inside the OST environment, engine and hosts names are resolvable, but on the host (or vm) running OST, the names are not available. This can be fixed by adding the engine and hosts to /etc/hosts like this: $ cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.200.2 engine 192.168.200.3 lago-basic-suite-master-host-0 192.168.200.4 lago-basic-suite-master-host-1 It would be if this was automated by OST. You can get the details using: $ cd src/ovirt-system-tests/deployment-xxx $ lago status OST keeps the deployment directory in the source directory. Be careful if you like to "git clean -dxf' since it will delete all the deployment and you will have to kill the vms manually later. The next thing we need is the engine ca cert. It can be fetched like this: $ curl -k 'https://engine/ovirt-engine/services/pki-resource?resource=ca-certificate&f…' > ca.pem I would expect OST to do this and put the file in the deployment directory. To upload or download images, backup vms or use other modern examples from the sdk, you need to have a configuration file like this: $ cat ~/.config/ovirt.conf [engine] engine_url = https://engine username = admin@internal password = 123 cafile = ca.pem With this uploading from the same directory where ca.pem is located will work. If you want it to work from any directory, use absolute path to the file. I created a test image using qemu-img and qemu-io: $ qemu-img create -f qcow2 test.qcow2 1g To write some data to the test image we can use qemu-io. This writes 64k of data (b"\xf0" * 64 * 1024) to offset 1 MiB. $ qemu-io -f qcow2 -c "write -P 240 1m 64k" test.qcow2 Since this image contains only 64k of data, uploading it should be instant. The last part we need is the imageio client package: $ dnf install ovirt-imageio-client To upload the image, we need at least one host up and storage domains created. I did not find a way to prepare OST, so simply run this after run_tests completed. It took about an hour. To upload the image to raw sparse disk we can use: $ python3 /usr/share/doc/python3-ovirt-engine-sdk4/examples/upload_disk.py -c engine --sd-name nfs --disk-sparse --disk-format raw test.qcow2 [ 0.0 ] Checking image... [ 0.0 ] Image format: qcow2 [ 0.0 ] Disk format: raw [ 0.0 ] Disk content type: data [ 0.0 ] Disk provisioned size: 1073741824 [ 0.0 ] Disk initial size: 1073741824 [ 0.0 ] Disk name: test.raw [ 0.0 ] Disk backup: False [ 0.0 ] Connecting... [ 0.0 ] Creating disk... [ 36.3 ] Disk ID: 26df08cf-3dec-47b9-b776-0e2bc564b6d5 [ 36.3 ] Creating image transfer... [ 38.2 ] Transfer ID: de8cfac9-ead2-4304-b18b-a1779d647716 [ 38.2 ] Transfer host name: lago-basic-suite-master-host-1 [ 38.2 ] Uploading image... [ 100.00% ] 1.00 GiB, 1.79 seconds, 571.50 MiB/s [ 40.0 ] Finalizing image transfer... [ 44.1 ] Upload completed successfully I uploaded this before I added the hosts to /etc/hosts, so the upload was done via the proxy. Yes, it took 36 seconds to create the disk. To download the disk use: $ python3 /usr/share/doc/python3-ovirt-engine-sdk4/examples/download_disk.py -c engine 5ac63c72-6296-46b1-a068-b1039c8ecbd1 downlaod.qcow2 [ 0.0 ] Connecting... [ 0.2 ] Creating image transfer... [ 1.6 ] Transfer ID: a99e2a43-8360-4661-81dc-02828a88d586 [ 1.6 ] Transfer host name: lago-basic-suite-master-host-1 [ 1.6 ] Downloading image... [ 100.00% ] 1.00 GiB, 0.32 seconds, 3.10 GiB/s [ 1.9 ] Finalizing image transfer... We can verify the transfers using checksums. Here we create a checksum of the remote disk: $ python3 /usr/share/doc/python3-ovirt-engine-sdk4/examples/checksum_disk.py -c engine 26df08cf-3dec-47b9-b776-0e2bc564b6d5 { "algorithm": "blake2b", "block_size": 4194304, "checksum": "a79a1efae73484e0218403e6eb715cdf109c8e99c2200265b779369339cf347b" } And checksum of the downloaded image - they should match: $ python3 /usr/share/doc/python3-ovirt-engine-sdk4/examples/checksum_image.py downlaod.qcow2 { "algorithm": "blake2b", "block_size": 4194304, "checksum": "a79a1efae73484e0218403e6eb715cdf109c8e99c2200265b779369339cf347b" } Same upload to iscsi domain, using qcow2 format: $ python3 /usr/share/doc/python3-ovirt-engine-sdk4/examples/upload_disk.py -c engine --sd-name iscsi --disk-sparse --disk-format qcow2 test.qcow2 [ 0.0 ] Checking image... [ 0.0 ] Image format: qcow2 [ 0.0 ] Disk format: cow [ 0.0 ] Disk content type: data [ 0.0 ] Disk provisioned size: 1073741824 [ 0.0 ] Disk initial size: 458752 [ 0.0 ] Disk name: test.qcow2 [ 0.0 ] Disk backup: False [ 0.0 ] Connecting... [ 0.0 ] Creating disk... [ 27.8 ] Disk ID: e7ef253e-7baa-4d4a-a9b2-1a6b7db13f41 [ 27.8 ] Creating image transfer... [ 30.0 ] Transfer ID: 88328857-ac99-4ee1-9618-6b3cd14a7db8 [ 30.0 ] Transfer host name: lago-basic-suite-master-host-0 [ 30.0 ] Uploading image... [ 100.00% ] 1.00 GiB, 0.31 seconds, 3.28 GiB/s [ 30.3 ] Finalizing image transfer... [ 35.4 ] Upload completed successfully Again, creating the disk is very slow, not sure why. Probably having a storage server on a nested vm is not a good idea. We can compare the checksum with the source image since checksum are computed from the guest content: [nsoffer@ost ~]$ python3 /usr/share/doc/python3-ovirt-engine-sdk4/examples/checksum_disk.py -c engine e7ef253e-7baa-4d4a-a9b2-1a6b7db13f41 { "algorithm": "blake2b", "block_size": 4194304, "checksum": "a79a1efae73484e0218403e6eb715cdf109c8e99c2200265b779369339cf347b" } Finally, we can try real images using virt-builder: $ virt-builder fedora-32 Will create a new Fedora 32 server image in the current directory. See --help for many useful options to create different format, set root password, or install packages. Uploading this image is much slower: $ python3 /usr/share/doc/python3-ovirt-engine-sdk4/examples/upload_disk.py -c engine --sd-name nfs --disk-sparse --disk-format raw fedora-32.img [ 0.0 ] Checking image... [ 0.0 ] Image format: raw [ 0.0 ] Disk format: raw [ 0.0 ] Disk content type: data [ 0.0 ] Disk provisioned size: 6442450944 [ 0.0 ] Disk initial size: 6442450944 [ 0.0 ] Disk name: fedora-32.raw [ 0.0 ] Disk backup: False [ 0.0 ] Connecting... [ 0.0 ] Creating disk... [ 36.8 ] Disk ID: b17126f3-fa03-4c22-8f59-ef599b64a42e [ 36.8 ] Creating image transfer... [ 38.5 ] Transfer ID: fe82fb86-b87a-4e49-b9cd-f1f4334e7852 [ 38.5 ] Transfer host name: lago-basic-suite-master-host-0 [ 38.5 ] Uploading image... [ 100.00% ] 6.00 GiB, 99.71 seconds, 61.62 MiB/s [ 138.2 ] Finalizing image transfer... [ 147.8 ] Upload completed successfully At the current state of OST, we should avoid such long tests. Using backup_vm.py and other examples should work in the same way. I posted this patch to improve nfs performance, please review: https://gerrit.ovirt.org/c/112067/ Nir

4 9

回复:Re: Host's status how to change to UP when I execute Management--> Active command
by 李伏琼 05 Nov '20

05 Nov '20

thanks very much --------------原始邮件-------------- 发件人："Liran Rotenberg "<lrotenbe(a)redhat.com>; 发送时间：2020年11月5日(星期四) 晚上11:52 收件人："lifuqiong(a)sunyainfo.com" <lifuqiong(a)sunyainfo.com>; 抄送："users "<users(a)ovirt.org>;"devel "<devel(a)ovirt.org>; 主题：[ovirt-devel] Re: Host's status how to change to UP when I execute Management--> Active command ----------------------------------- Hi,The engine will monitor that host. Once it's reachable and we can get information from it the status will change accordingly. Please check out the HostMonitoring class. On Thu, Nov 5, 2020 at 11:57 AM lifuqiong(a)sunyainfo.com <lifuqiong(a)sunyainfo.com> wrote: Hi,      I checkout ovirt-engine's code with branch 4.2;  When execute Host's Management --> Active  command,  ovirt engine just set vds's status to "Unassigned" but how the vds's status changed to "UP" in ovirt-engine?  The code in ActiveVdsCommand.executeCommand() as follows: protected void executeCommand() {         final VDS vds = getVds();         try (EngineLock monitoringLock = acquireMonitorLock("Activate host")) {             executionHandler.updateSpecificActionJobCompleted(vds.getId(), ActionType.MaintenanceVds, false);             setSucceeded(setVdsStatus(VDSStatus.Unassigned).getSucceeded());             if (getSucceeded()) {                 TransactionSupport.executeInNewTransaction(() -> {                     // set network to operational / non-operational                     List<Network> networks = networkDao.getAllForCluster(vds.getClusterId());                     networkClusterHelper.setStatus(vds.getClusterId(), networks);                     return null;                 });                 // Start glusterd service on the node, which would haven been stopped due to maintenance                 if (vds.getClusterSupportsGlusterService()) {                     runVdsCommand(VDSCommandType.ManageGlusterService,                             new GlusterServiceVDSParameters(vds.getId(), Arrays.asList("glusterd"), "restart"));                     // starting vdo service                     GlusterStatus isRunning = glusterUtil.isVDORunning(vds.getId());                     switch (isRunning) {                     case DOWN:                         log.info("VDO service is down in host : '{}' with id '{}', starting VDO service",                                 vds.getHostName(),                                 vds.getId());                         startVDOService(vds);                         break;                     case UP:                         log.info("VDO service is up in host : '{}' with id '{}', skipping starting of VDO service",                                 vds.getHostName(),                                 vds.getId());                            break;                     case UNKNOWN:                         log.info("VDO service is not installed host : '{}' with id '{}', ignoring to start VDO service",                                 vds.getHostName(),                                 vds.getId());                         break;                     }                 }             }         }     } Your Sincerely Mark Lee _______________________________________________ Devel mailing list -- devel(a)ovirt.org To unsubscribe send an email to devel-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/36745S77NLH3K…

1 0

Re: Host's status how to change to UP when I execute Management--> Active command
by Liran Rotenberg 05 Nov '20

05 Nov '20

Hi, The engine will monitor that host. Once it's reachable and we can get information from it the status will change accordingly. Please check out the HostMonitoring class. On Thu, Nov 5, 2020 at 11:57 AM lifuqiong(a)sunyainfo.com < lifuqiong(a)sunyainfo.com> wrote: > > Hi, > I checkout ovirt-engine's code with branch 4.2; When execute Host's > Management --> Active command, ovirt engine just set vds's status to > "Unassigned" but how the vds's status changed to "UP" in ovirt-engine? The > code in ActiveVdsCommand.executeCommand() as follows: > protected void executeCommand() { > > final VDS vds = getVds(); > try (EngineLock monitoringLock = acquireMonitorLock("Activate > host")) { > executionHandler.updateSpecificActionJobCompleted(vds.getId(), > ActionType.MaintenanceVds, false); > > setSucceeded(setVdsStatus(VDSStatus.Unassigned).getSucceeded()); > > if (getSucceeded()) { > TransactionSupport.executeInNewTransaction(() -> { > // set network to operational / non-operational > List<Network> networks = > networkDao.getAllForCluster(vds.getClusterId()); > networkClusterHelper.setStatus(vds.getClusterId(), > networks); > return null; > }); > > // Start glusterd service on the node, which would haven > been stopped due to maintenance > if (vds.getClusterSupportsGlusterService()) { > runVdsCommand(VDSCommandType.ManageGlusterService, > new GlusterServiceVDSParameters(vds.getId(), > Arrays.asList("glusterd"), "restart")); > // starting vdo service > GlusterStatus isRunning = > glusterUtil.isVDORunning(vds.getId()); > switch (isRunning) { > case DOWN: > log.info("VDO service is down in host : '{}' with > id '{}', starting VDO service", > vds.getHostName(), > vds.getId()); > startVDOService(vds); > break; > case UP: > log.info("VDO service is up in host : '{}' with > id '{}', skipping starting of VDO service", > vds.getHostName(), > vds.getId()); > break; > case UNKNOWN: > log.info("VDO service is not installed host : > '{}' with id '{}', ignoring to start VDO service", > vds.getHostName(), > vds.getId()); > break; > } > > } > } > } > } > > > Your Sincerely > Mark Lee > _______________________________________________ > Devel mailing list -- devel(a)ovirt.org > To unsubscribe send an email to devel-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/devel@ovirt.org/message/36745S77NLH3K… >

1 0

Building lago from source
by Nir Soffer 05 Nov '20

05 Nov '20

I'm trying to test with ost: https://github.com/lago-project/lago/pull/815 So clone the project on the ost vm and built rpms: make make rpm The result is: lago-1.0.2-1.el8.noarch.rpm python3-lago-1.0.2-1.el8.noarch.rpm But the lago version installed by setup_for_ost.sh is: $ rpm -q lago lago-1.0.11-1.el8.noarch I tried to install lago from master, and then lago_init fail: $ lago_init /usr/share/ost-images/el8-engine-installed.qcow2 -k /usr/share/ost-images/el8_id_rsa Using images ost-images-el8-host-installed-1-202011021248.x86_64, ost-images-el8-engine-installed-1-202011021248.x86_64 containing ovirt-engine-4.4.4-0.0.master.20201031195930.git8f858d6c01d.el8.noarch vdsm-4.40.35.1-1.el8.x86_64 usage: lago [-h] [-l {info,debug,error,warning}] [--logdepth LOGDEPTH] [--version] [--out-format {default,flat,json,yaml}] [--prefix-path PREFIX_PATH] [--workdir-path WORKDIR_PATH] [--prefix-name PREFIX_NAME] [--ssh-user SSH_USER] [--ssh-password SSH_PASSWORD] [--ssh-tries SSH_TRIES] [--ssh-timeout SSH_TIMEOUT] [--libvirt_url LIBVIRT_URL] [--libvirt-user LIBVIRT_USER] [--libvirt-password LIBVIRT_PASSWORD] [--default_vm_type DEFAULT_VM_TYPE] [--default_vm_provider DEFAULT_VM_PROVIDER] [--default_root_password DEFAULT_ROOT_PASSWORD] [--lease_dir LEASE_DIR] [--reposync-dir REPOSYNC_DIR] [--ignore-warnings] VERB ... lago: error: unrecognized arguments: --ssh-key /home/nsoffer/src/ovirt-system-tests/deployment-basic-suite-master /home/nsoffer/src/ovirt-system-tests/basic-suite-master/LagoInitFile Do we use a customized lago version for ost? Where is the source? Nir

2 1

Host's status how to change to UP when I execute Management--> Active command
by lifuqiong＠sunyainfo.com 05 Nov '20

05 Nov '20

Hi, I checkout ovirt-engine's code with branch 4.2; When execute Host's Management --> Active command, ovirt engine just set vds's status to "Unassigned" but how the vds's status changed to "UP" in ovirt-engine? The code in ActiveVdsCommand.executeCommand() as follows: protected void executeCommand() { final VDS vds = getVds(); try (EngineLock monitoringLock = acquireMonitorLock("Activate host")) { executionHandler.updateSpecificActionJobCompleted(vds.getId(), ActionType.MaintenanceVds, false); setSucceeded(setVdsStatus(VDSStatus.Unassigned).getSucceeded()); if (getSucceeded()) { TransactionSupport.executeInNewTransaction(() -> { // set network to operational / non-operational List<Network> networks = networkDao.getAllForCluster(vds.getClusterId()); networkClusterHelper.setStatus(vds.getClusterId(), networks); return null; }); // Start glusterd service on the node, which would haven been stopped due to maintenance if (vds.getClusterSupportsGlusterService()) { runVdsCommand(VDSCommandType.ManageGlusterService, new GlusterServiceVDSParameters(vds.getId(), Arrays.asList("glusterd"), "restart")); // starting vdo service GlusterStatus isRunning = glusterUtil.isVDORunning(vds.getId()); switch (isRunning) { case DOWN: log.info("VDO service is down in host : '{}' with id '{}', starting VDO service", vds.getHostName(), vds.getId()); startVDOService(vds); break; case UP: log.info("VDO service is up in host : '{}' with id '{}', skipping starting of VDO service", vds.getHostName(), vds.getId()); break; case UNKNOWN: log.info("VDO service is not installed host : '{}' with id '{}', ignoring to start VDO service", vds.getHostName(), vds.getId()); break; } } } } } Your Sincerely Mark Lee

1 0

Re: [ovirt-users] Re: Removal of dpdk
by Dominik Holler 04 Nov '20

04 Nov '20

Hi Florian, thanks for your thoughts! On Tue, Nov 3, 2020 at 3:21 PM Florian Schmid via Users <users(a)ovirt.org> wrote: > Hi Ales, > > what do you mean with "not maintained for a long time"? > The oVirt integration of dpdk was not maintained. > DPDK is heavily developed and make the linux network extremely fast. > > I don't think, that SR-IOV can replace it, > The removal of dpdk is about removing the dpdk support from oVirt hosts only. We wonder if there is someone using dpdk to attach oVirt VMs to physical NICs. We are aware that many users use SR-IOV, especially for scenarios of enabling a high count of Ethernet frames for VMs or requiring a low latency, but we are not aware of users using dpdk to connect the oVirt VMs to the physical NICs of the host. > because packets must be still processed by the kernel, which is really > slow and CPU demanding. > In SR-IOV the packets might be processed by the guest kernel, not but the host kernel. oVirt is focused on the host kernel, while the guest OS is managed by the user of oVirt. Did this explanation address your concerns? BR Florian > > ------------------------------ > *Von: *"Ales Musil" <amusil(a)redhat.com> > *An: *"Nir Soffer" <nsoffer(a)redhat.com> > *CC: *"users" <users(a)ovirt.org>, "devel" <devel(a)ovirt.org> > *Gesendet: *Dienstag, 3. November 2020 13:56:12 > *Betreff: *[ovirt-users] Re: Removal of dpdk > > > > On Tue, Nov 3, 2020 at 1:52 PM Nir Soffer <nsoffer(a)redhat.com> wrote: > >> On Tue, Nov 3, 2020 at 1:07 PM Ales Musil <amusil(a)redhat.com> wrote: >> >>> Hello, >>> we have decided to remove dpdk in the upcoming version of oVirt namely >>> 4.4.4. Let us know if there are any concerns about this. >>> >> >> Can you give more info why we want to remove this feature, and what is >> the replacement for existing users? >> >> Nir >> > > Sure, > the feature was only experimental and not maintained for a long time. The > replacement is to use SR-IOV > which is supported by oVirt. > > Thanks, > Ales > > > -- > > Ales Musil > > Software Engineer - RHV Network > > Red Hat EMEA <https://www.redhat.com> > > amusil(a)redhat.com IM: amusil > <https://red.ht/sig> > > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/3FHIRQKEEKLGW… > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/ZCAYTHVZPOZF3… > >

2 1

Libvirt driver iothread property for virtio-scsi disks
by Nir Soffer 04 Nov '20

04 Nov '20

The docs[1] say: - The optional iothread attribute assigns the disk to an IOThread as defined by the range for the domain iothreads value. Multiple disks may be assigned to the same IOThread and are numbered from 1 to the domain iothreads value. Available for a disk device target configured to use "virtio" bus and "pci" or "ccw" address types. Since 1.2.8 (QEMU 2.1) Does it mean that virtio-scsi disks do not use iothreads? I'm experiencing a horrible performance using nested vms (up to 2 levels of nesting) when accessing NFS storage running on one of the VMs. The NFS server is using scsi disk. My theory is: - Writing to NFS server is very slow (too much nesting, slow disk) - Not using iothreads (because we don't use virtio?) - Guest CPU is blocked by slow I/O Does this make sense? [1] https://libvirt.org/formatdomain.html#hard-drives-floppy-disks-cdroms Nir

3 4

Re: [ovirt-users] Re: Removal of dpdk
by Ales Musil 03 Nov '20

03 Nov '20

On Tue, Nov 3, 2020 at 3:17 PM Florian Schmid <fschmid(a)ubimet.com> wrote: > Hi Ales, > > what do you mean with "not maintained for a long time"? > DPDK is heavily developed and make the linux network extremely fast. > > I don't think, that SR-IOV can replace it, because packets must be still > processed by the kernel, which is really slow and CPU demanding. > > BR Florian > > ------------------------------ > *Von: *"Ales Musil" <amusil(a)redhat.com> > *An: *"Nir Soffer" <nsoffer(a)redhat.com> > *CC: *"users" <users(a)ovirt.org>, "devel" <devel(a)ovirt.org> > *Gesendet: *Dienstag, 3. November 2020 13:56:12 > *Betreff: *[ovirt-users] Re: Removal of dpdk > > > The dpdk inside oVirt is not maintained, of course that dpdk as technology/project is still developed. > > On Tue, Nov 3, 2020 at 1:52 PM Nir Soffer <nsoffer(a)redhat.com> wrote: > >> On Tue, Nov 3, 2020 at 1:07 PM Ales Musil <amusil(a)redhat.com> wrote: >> >>> Hello, >>> we have decided to remove dpdk in the upcoming version of oVirt namely >>> 4.4.4. Let us know if there are any concerns about this. >>> >> >> Can you give more info why we want to remove this feature, and what is >> the replacement for existing users? >> >> Nir >> > > Sure, > the feature was only experimental and not maintained for a long time. The > replacement is to use SR-IOV > which is supported by oVirt. > > Thanks, > Ales > > > -- > > Ales Musil > > Software Engineer - RHV Network > > Red Hat EMEA <https://www.redhat.com> > > amusil(a)redhat.com IM: amusil > <https://red.ht/sig> > > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/3FHIRQKEEKLGW… > -- Ales Musil Software Engineer - RHV Network Red Hat EMEA <https://www.redhat.com> amusil(a)redhat.com IM: amusil <https://red.ht/sig>

1 0

Removal of dpdk
by Ales Musil 03 Nov '20

03 Nov '20

Hello, we have decided to remove dpdk in the upcoming version of oVirt namely 4.4.4. Let us know if there are any concerns about this. Thank you. Regards, Ales Musil -- Ales Musil Software Engineer - RHV Network Red Hat EMEA <https://www.redhat.com> amusil(a)redhat.com IM: amusil <https://red.ht/sig>

2 2

ovirt-engine has been tagged (ovirt-engine-4.4.3.10)
by Tal Nisan 02 Nov '20

02 Nov '20

1 0

Re: [virt-devel] Timeout failures in the OST
by Nir Soffer 02 Nov '20

02 Nov '20

On Mon, Nov 2, 2020 at 2:27 PM Benny Zlotnik <bzlotnik(a)redhat.com> wrote: Issues like this belong to oVirt devel mailing list. > looks like live merge failed[1]: > 2020-11-01 10:31:49,903+0100 ERROR (periodic/0) [virt.vm] (vmId='fcabfd2e-2937-4419-9b25-78fdd2b9c7c2') Unable to get watermarks for drive vdb: invalid argument: invalid path /rhev/data-center/mnt/blockSD/97b6175e-b6a9-419b-bd54-7c1e38c1bf71/images/fbb11a06-b8ef-4078-9530-978e7ca8ea0b/a911ad89-e461-4db4-88bf-a5d6590608b5 not assigned to domain (vm:1213) This may be a bug in vdsm live merge flow, tring to monitor a volume after the volume was already removed, or it may be libvirt/qemu bug. > ... > 2020-11-01 10:31:53,138+0100 ERROR (jsonrpc/1) [virt.vm] (vmId='fcabfd2e-2937-4419-9b25-78fdd2b9c7c2') merge: libvirt does not support volume chain monitoring. Unable to perform live merge. drive: vdb, alias: ua-fbb11a06-b8ef-4078-9530-978e7ca8ea0b, chains: {} (vm:5411) > > libvirt logs report: > ... > 2020-11-01 09:30:50.021+0000: 40137: error : virProcessRunInFork:1161 : internal error: child reported (status=125): > 2020-11-01 09:30:50.025+0000: 40137: error : virProcessRunInFork:1161 : internal error: child reported (status=125): internal error: child reported (status=125): > 2020-11-01 09:30:50.025+0000: 40137: warning : qemuDomainSnapshotDiskUpdateSource:15582 : Unable to move disk metadata on vm vm0 > 2020-11-01 09:31:45.539+0000: 40134: error : qemuMonitorJSONCheckError:412 : internal error: unable to execute QEMU command 'blockdev-del': Node libvirt-6-format is in use > 2020-11-01 09:31:45.539+0000: 40134: error : qemuMonitorJSONCheckError:412 : internal error: unable to execute QEMU command 'blockdev-del': Block device libvirt-6-storage is in use > 2020-11-01 09:31:45.540+0000: 40134: error : qemuMonitorJSONCheckError:412 : internal error: unable to execute QEMU command 'blockdev-del': Node 'libvirt-7-format' is busy: node is used as backing hd of 'libvirt-6-format' > 2020-11-01 09:31:45.541+0000: 40134: error : qemuMonitorJSONCheckError:412 : internal error: unable to execute QEMU command 'blockdev-del': Block device libvirt-7-storage is in use > 2020-11-01 09:31:45.900+0000: 40133: error : qemuDomainGetBlockInfo:12272 : invalid argument: invalid path /rhev/data-center/mnt/blockSD/97b6175e-b6a9-419b-bd54-7c1e38c1bf71/images/fbb11a06-b8ef-4078-9530-978e7ca8ea0b/a911ad89-e461-4db4-88bf-a5d6590608b5 not assigned to domain These smell like libvirt/qemu bug. Is this reproducible with RHEL 8.3? > Looks like the issue previously discussed in "[rhev-devel] Live storage migration instability in OST" two months ago has resurfaced > > > > Another issue seems to be the removal of the source disk: > 2020-11-01 10:31:48,056+0100 ERROR (tasks/3) [storage.StorageDomainManifest] removed image dir: /rhev/data-center/mnt/192.168.202.2:_exports_nfs_share1/3ca0e492-45f2-4383-b149-439043408bce/images/_remove_me_fbb11a06-b8ef-4078-9530-978e7ca8ea0b can't be removed (fileSD:258) > 2020-11-01 10:31:48,056+0100 ERROR (tasks/3) [storage.TaskManager.Task] (Task='70db80c2-076a-4ba1-a65d-821e6b5fe52c') Unexpected error (task:880) > Traceback (most recent call last): > File "/usr/lib/python3.6/site-packages/vdsm/storage/fileSD.py", line 251, in purgeImage > self.oop.os.rmdir(toDelDir) > File "/usr/lib/python3.6/site-packages/vdsm/storage/outOfProcess.py", line 238, in rmdir > self._iop.rmdir(path) > File "/usr/lib/python3.6/site-packages/ioprocess/__init__.py", line 550, in rmdir > return self._sendCommand("rmdir", {"path": path}, self.timeout) > File "/usr/lib/python3.6/site-packages/ioprocess/__init__.py", line 479, in _sendCommand > raise OSError(errcode, errstr) > OSError: [Errno 39] Directory not empty > > During handling of the above exception, another exception occurred: > > Traceback (most recent call last): > File "/usr/lib/python3.6/site-packages/vdsm/storage/task.py", line 887, in _run > return fn(*args, **kargs) > File "/usr/lib/python3.6/site-packages/vdsm/storage/task.py", line 350, in run > return self.cmd(*self.argslist, **self.argsdict) > File "/usr/lib/python3.6/site-packages/vdsm/storage/securable.py", line 79, in wrapper > return method(self, *args, **kwargs) > File "/usr/lib/python3.6/site-packages/vdsm/storage/sp.py", line 1947, in purgeImage > domain.purgeImage(sdUUID, imgUUID, volsByImg, discard) > File "/usr/lib/python3.6/site-packages/vdsm/storage/sd.py", line 855, in purgeImage > self._manifest.purgeImage(sdUUID, imgUUID, volsImgs, discard) > File "/usr/lib/python3.6/site-packages/vdsm/storage/fileSD.py", line 259, in purgeImage > raise se.ImageDeleteError("%s %s" % (imgUUID, str(e))) > vdsm.storage.exception.ImageDeleteError: Could not remove all image's volumes: ('fbb11a06-b8ef-4078-9530-978e7ca8ea0b [Errno 39] Directory not empty',) > > But it's unclear what the leftover is Maybe be leftover from previous failed LSM. I think we need a better error message here, it should list the files in the non-empty directory. > [1] https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/12782… > > > On Mon, Nov 2, 2020 at 1:44 PM Steven Rosenberg <srosenbe(a)redhat.com> wrote: >> >> Dear Benny, >> >> Thank you for your response. >> >> Here is the timeout engine log from one of the ps 45 failures [1]. >> >> It seems like this timeout is related to the engine failing and the ost scripts are not designed to detect the fail, thus timing out: >> >> 2020-11-01 10:31:55,175+01 ERROR [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-67) [live_storage_migration] Command id: '60b3f7fc-93db-48b2-82a1-8a93c47e18e1 failed child command status for step 'MERGE_STATUS' >> >> >> >> >> [1] https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/12796… >> >> With Best Regards. >> >> Steven. >> >> >> >> On Mon, Nov 2, 2020 at 12:51 PM Benny Zlotnik <bzlotnik(a)redhat.com> wrote: >>> >>> Can you link to the relevant engine/vdsm logs? >>> The timeout in the tests indicates that the desired state wasn't >>> reached so test logs don't provide the information about what exactly >>> happened >>> >>> On Sun, Nov 1, 2020 at 5:10 PM Steven Rosenberg <srosenbe(a)redhat.com> wrote: >>> > >>> > Dear virt-devel, >>> > >>> > We are currently experiencing many timeout failures in various patch sets for a gerrit issue 11395 [1]. >>> > >>> > The timeouts occur intermittently and seem to be unrelated to the changes which are only in the 004 module [2] and should have only affected VM1 / Disk1. >>> > >>> > We could use some advice on addressing these issues as well as a review of the patch to ensure we can move this patch forward. The patch sets and relevant timeouts are as follows: >>> > >>> > PS 40: >>> > >>> > test_live_storage_migration – test 004 [3] >>> > >>> > PS 41: >>> > >>> > on test_verify_engine_backup – test 002 [4] >>> > >>> > PS 43: >>> > >>> > on test_virtual_machines - test 100 [5] >>> > >>> > PS 45: >>> > >>> > on test_live_storage_migration – 004 [6] >>> > >>> > >>> > >>> > >>> > [1] https://gerrit.ovirt.org/#/c/111395/ >>> > [2] basic-suite-master/test-scenarios/004_basic_sanity.py >>> > [3] https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/12628… >>> > [4] https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/12659… >>> > [5] https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/12744… >>> > [6] https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/12782… >>> > >>> > With Best Regards. >>> > >>> > Steven >>> > >>> > >>>

1 0

Fwd: Timeout failures in the OST
by Steven Rosenberg 02 Nov '20

02 Nov '20

---------- Forwarded message --------- From: Steven Rosenberg <srosenbe(a)redhat.com> Date: Sun, Nov 1, 2020 at 5:09 PM Subject: Timeout failures in the OST To: virt-devel <virt-devel(a)redhat.com> Dear ovirt Personnel, We are currently experiencing many timeout failures in various patch sets for a gerrit issue 11395 [1]. The timeouts occur intermittently and seem to be unrelated to the changes which are only in the 004 module [2] and should have only affected VM1 / Disk1. We could use some advice on addressing these issues as well as a review of the patch to ensure we can move this patch forward. The patch sets and relevant timeouts are as follows: PS 40: test_live_storage_migration – test 004 [3] PS 41: on test_verify_engine_backup – test 002 [4] PS 43: on test_virtual_machines - test 100 [5] PS 45: on test_live_storage_migration – 004 [6] [1] https://gerrit.ovirt.org/#/c/111395/ [2] basic-suite-master/test-scenarios/004_basic_sanity.py [3] https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/12628… [4] https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/12659… [5] https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/12744… [6] https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/12782… With Best Regards. Steven

1 0