July 2019 - Users - oVirt List Archives

SPM and Task error ...
by Enrico 22 Nov '24

22 Nov '24

Hi all, my ovirt cluster has got 3 Hypervisors runnig Centos 7.5.1804 vdsm is 4.20.39.1-1.el7, ovirt engine is 4.2.4.5-1.el7, the storage systems are HP MSA P2000 and 2050 (fibre channel). I need to stop one of the hypervisors for maintenance but this system is the storage pool manager. For this reason I decided to manually activate SPM in one of the other nodes but this operation is not successful. In the ovirt engine (engine.log) the error is this: 2019-07-25 12:39:16,744+02 INFO [org.ovirt.engine.core.bll.storage.pool.ForceSelectSPMCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] Running command: ForceSelectSPMCommand internal: false. Entities affected : ID: 81c9bd3c-ae0a-467f-bf7f-63ab30cd8d9e Type: VDSAction group MANIPULATE_HOST with role type ADMIN 2019-07-25 12:39:16,745+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SpmStopOnIrsVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] START, SpmStopOnIrsVDSCommand( SpmStopOnIrsVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7', ignoreFailoverLimit='false'}), log id: 37bf4639 2019-07-25 12:39:16,747+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] START, ResetIrsVDSCommand( ResetIrsVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7', ignoreFailoverLimit='false', vdsId='751f3e99-b95e-4c31-bc38-77f5661a0bdc', ignoreStopFailed='false'}), log id: 2522686f 2019-07-25 12:39:16,749+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] START, SpmStopVDSCommand(HostName = infn-vm05.management, SpmStopVDSCommandParameters:{hostId='751f3e99-b95e-4c31-bc38-77f5661a0bdc', storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7'}), log id: 1810fd8b 2019-07-25 12:39:16,758+02 *ERROR* [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] SpmStopVDSCommand::Not stopping SPM on vds 'infn-vm05.management', pool id '18d57688-6ed4-43b8-bd7c-0665b55950b7' as there are uncleared tasks 'Task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e', status 'running'' 2019-07-25 12:39:16,758+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] FINISH, SpmStopVDSCommand, log id: 1810fd8b 2019-07-25 12:39:16,758+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] FINISH, ResetIrsVDSCommand, log id: 2522686f 2019-07-25 12:39:16,758+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SpmStopOnIrsVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] FINISH, SpmStopOnIrsVDSCommand, log id: 37bf4639 2019-07-25 12:39:16,760+02 *ERROR* [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] EVENT_ID: USER_FORCE_SELECTED_SPM_STOP_FAILED(4,096), Failed to force select infn-vm07.management as the SPM due to a failure to stop the current SPM. while in the hypervisor (SPM) vdsm.log: 2019-07-25 12:39:16,744+02 INFO [org.ovirt.engine.core.bll.storage.pool.ForceSelectSPMCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] Running command: ForceSelectSPMCommand internal: false. Entities affected : ID: 81c9bd3c-ae0a-467f-bf7f-63ab30cd8d9e Type: VDSAction group MANIPULATE_HOST with role type ADMIN 2019-07-25 12:39:16,745+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SpmStopOnIrsVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] START, SpmStopOnIrsVDSCommand( SpmStopOnIrsVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7', ignoreFailoverLimit='false'}), log id: 37bf4639 2019-07-25 12:39:16,747+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] START, ResetIrsVDSCommand( ResetIrsVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7', ignoreFailoverLimit='false', vdsId='751f3e99-b95e-4c31-bc38-77f5661a0bdc', ignoreStopFailed='false'}), log id: 2522686f 2019-07-25 12:39:16,749+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] START, SpmStopVDSCommand(HostName = infn-vm05.management, SpmStopVDSCommandParameters:{hostId='751f3e99-b95e-4c31-bc38-77f5661a0bdc', storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7'}), log id: 1810fd8b 2019-07-25 12:39:16,758+02 *ERROR* [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] SpmStopVDSCommand::Not stopping SPM on vds 'infn-vm05.management', pool id '18d57688-6ed4-43b8-bd7c-0665b55950b7' as there are uncleared tasks 'Task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e', status 'running'' 2019-07-25 12:39:16,758+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] FINISH, SpmStopVDSCommand, log id: 1810fd8b 2019-07-25 12:39:16,758+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] FINISH, ResetIrsVDSCommand, log id: 2522686f 2019-07-25 12:39:16,758+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SpmStopOnIrsVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] FINISH, SpmStopOnIrsVDSCommand, log id: 37bf4639 2019-07-25 12:39:16,760+02 *ERROR* [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] EVENT_ID: USER_FORCE_SELECTED_SPM_STOP_FAILED(4,096), Failed to force select infn-vm07.management as the SPM due to a failure to stop the current SPM. 2019-07-25 12:39:18,660+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Task id 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' has passed pre-polling period time and should be polled. Pre-polling period is 60000 millis. 2019-07-25 12:39:18,660+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Task id 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' has passed pre-polling period time and should be polled. Pre-polling period is 60000 millis. 2019-07-25 12:39:18,750+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Task id 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' has passed pre-polling period time and should be polled. Pre-polling period is 60000 millis. 2019-07-25 12:39:18,750+02 *ERROR* [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] BaseAsyncTask::logEndTaskFailure: Task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' (Parent Command 'Unknown', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure: 2019-07-25 12:39:18,750+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' 2019-07-25 12:39:18,751+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7', ignoreFailoverLimit='false', taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: 34ae2b2f 2019-07-25 12:39:18,752+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] START, HSMClearTaskVDSCommand(HostName = infn-vm05.management, HSMTaskGuidBaseVDSCommandParameters:{hostId='751f3e99-b95e-4c31-bc38-77f5661a0bdc', taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: d3a78ad 2019-07-25 12:39:18,757+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] FINISH, HSMClearTaskVDSCommand, log id: d3a78ad 2019-07-25 12:39:18,757+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] FINISH, SPMClearTaskVDSCommand, log id: 34ae2b2f 2019-07-25 12:39:18,757+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] SPMAsyncTask::ClearAsyncTask: At time of attempt to clear task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' the response code was 'TaskStateError' and message was 'Operation is not allowed in this task state: ("can't clean in state running",)'. Task will not be cleaned 2019-07-25 12:39:18,757+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] BaseAsyncTask::onTaskEndSuccess: Task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' (Parent Command 'Unknown', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2019-07-25 12:39:18,757+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' 2019-07-25 12:39:18,758+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7', ignoreFailoverLimit='false', taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: 42de0c2b 2019-07-25 12:39:18,759+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] START, HSMClearTaskVDSCommand(HostName = infn-vm05.management, HSMTaskGuidBaseVDSCommandParameters:{hostId='751f3e99-b95e-4c31-bc38-77f5661a0bdc', taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: 4895c79c 2019-07-25 12:39:18,764+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] FINISH, HSMClearTaskVDSCommand, log id: 4895c79c 2019-07-25 12:39:18,764+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] FINISH, SPMClearTaskVDSCommand, log id: 42de0c2b 2019-07-25 12:39:18,764+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] SPMAsyncTask::ClearAsyncTask: At time of attempt to clear task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' the response code was 'TaskStateError' and message was 'Operation is not allowed in this task state: ("can't clean in state running",)'. Task will not be cleaned 2019-07-25 12:39:18,764+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Task id 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' has passed pre-polling period time and should be polled. Pre-polling period is 60000 millis. 2019-07-25 12:39:18,764+02 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Cleaning zombie tasks: Clearing async task 'Unknown' that started at 'Fri May 03 14:48:50 CEST 2019' 2019-07-25 12:39:18,764+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' 2019-07-25 12:39:18,765+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7', ignoreFailoverLimit='false', taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: da77af2 2019-07-25 12:39:18,766+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] START, HSMClearTaskVDSCommand(HostName = infn-vm05.management, HSMTaskGuidBaseVDSCommandParameters:{hostId='751f3e99-b95e-4c31-bc38-77f5661a0bdc', taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: 530694fb 2019-07-25 12:39:18,771+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] FINISH, HSMClearTaskVDSCommand, log id: 530694fb 2019-07-25 12:39:18,771+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] FINISH, SPMClearTaskVDSCommand, log id: da77af2 2019-07-25 12:39:18,771+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] SPMAsyncTask::ClearAsyncTask: At time of attempt to clear task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' the response code was 'TaskStateError' and message was 'Operation is not allowed in this task state: ("can't clean in state running",)'. Task will not be cleaned there is some relation between this error and a task that has remained hanging, from SPM server: # vdsm-client Task getInfo taskID=fdcf4d1b-82fe-49a6-b233-323ebe568f8e { "verb": "prepareMerge", "id": "fdcf4d1b-82fe-49a6-b233-323ebe568f8e" } # vdsm-client Task getStatus taskID=fdcf4d1b-82fe-49a6-b233-323ebe568f8e { "message": "running job 1 of 1", "code": 0, "taskID": "fdcf4d1b-82fe-49a6-b233-323ebe568f8e", "taskResult": "", "taskState": "running" } How can I solve this problem ? Thanks a lot for your help !! Best Regards Enrico -- _______________________________________________________________________ Enrico Becchetti Servizio di Calcolo e Reti Istituto Nazionale di Fisica Nucleare - Sezione di Perugia Via Pascoli,c/o Dipartimento di Fisica 06123 Perugia (ITALY) Phone:+39 075 5852777 Mail: Enrico.Becchetti<at>pg.infn.it _______________________________________________________________________

5 15

Changing disk QoS causes segfault with IO-Threads enabled (oVirt 4.3.0.4-1.el7)
by jloh＠squiz.net 23 Apr '24

23 Apr '24

We recently upgraded to 4.3.0 and have found that when changing disk QoS settings on VMs whilst IO-Threads is enabled causes them to segfault and the VM to reboot. We've been able to replicate this across several VMs. VMs with IO-Threads disabled/turned off do not segfault when changing the QoS. Mar 1 11:49:06 srvXX kernel: IO iothread1[30468]: segfault at fffffffffffffff8 ip 0000557649f2bd24 sp 00007f80de832f60 error 5 in qemu-kvm[5576498dd000+a03000] Mar 1 11:49:06 srvXX abrt-hook-ccpp: invalid number 'iothread1' Mar 1 11:49:11 srvXX libvirtd: 2019-03-01 00:49:11.116+0000: 13365: error : qemuMonitorIORead:609 : Unable to read from monitor: Connection reset by peer Happy to supply some more logs to someone if they'll help but just wondering whether anyone else has experienced this or knows of a current fix other than turning io-threads off. Cheers.

4 3

Unable to install oVirt on RHEL7.5
by SS00514758＠techmahindra.com 05 Jan '24

05 Jan '24

Hi All, I am unable to install oVirt on RHEL7.5, to install it I am taking reference of below link, https://www.ovirt.org/documentation/install-guide/chap-Installing_oVirt.html But though it is not working for me, couple of dependencies is not getting installed, and because of this I am not able to run the ovirt-engine, below are the depencies packages that unable to install, Error: Package: collectd-write_http-5.8.0-6.1.el7.x86_64 (@ovirt-4.2-centos-opstools) Requires: collectd(x86-64) = 5.8.0-6.1.el7 Removing: collectd-5.8.0-6.1.el7.x86_64 (@ovirt-4.2-centos-opstools) collectd(x86-64) = 5.8.0-6.1.el7 Updated By: collectd-5.8.1-1.el7.x86_64 (epel) collectd(x86-64) = 5.8.1-1.el7 Available: collectd-5.7.2-1.el7.x86_64 (ovirt-4.2-centos-opstools) collectd(x86-64) = 5.7.2-1.el7 Available: collectd-5.7.2-3.el7.x86_64 (ovirt-4.2-centos-opstools) collectd(x86-64) = 5.7.2-3.el7 Available: collectd-5.8.0-2.el7.x86_64 (ovirt-4.2-centos-opstools) collectd(x86-64) = 5.8.0-2.el7 Available: collectd-5.8.0-3.el7.x86_64 (ovirt-4.2-centos-opstools) collectd(x86-64) = 5.8.0-3.el7 Available: collectd-5.8.0-5.el7.x86_64 (ovirt-4.2-centos-opstools) collectd(x86-64) = 5.8.0-5.el7 Help me to install this. Looking forward to resolve this issue. Regards Sumit Sahay

6 8

Re: Failed to synchronize networks of Provider ovirt-provider-ovn
by Mail SET Inc. Group 27 Oct '23

27 Oct '23

Yes, i use same manual to change WebUI SSL. ovirt-ca-file= is a same SSL file which use WebUI. Yes, i restart ovirt-provider-ovn, i restart engine, i restart all what i can restart. Nothing... > 12 сент. 2018 г., в 16:11, Dominik Holler <dholler(a)redhat.com> написал(а): > > On Wed, 12 Sep 2018 14:23:54 +0300 > "Mail SET Inc. Group" <mail(a)set-pro.net> wrote: > >> Ok! > > Not exactly, please use users(a)ovirt.org for such questions. > Other should benefit from this questions, too. > Please write the next mail to users(a)ovirt.org and keep me in CC. > >> What i did: >> >> 1) install oVirt «from box» (4.2.5.2-1.el7); >> 2) generate own ssl for my engine using my FreeIPA CA, Install it and > > What means "Install it"? You can use the doc from the following link > https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.2/ht… > > Ensure that ovirt-ca-file= in > /etc/ovirt-provider-ovn/conf.d/10-setup-ovirt-provider-ovn.conf > points to the correct file and ovirt-provider-ovn is restarted. > >> get tis issue; >> >> >> [root@engine ~]# tail -n 50 /var/log/ovirt-provider-ovn.log >> 2018-09-12 14:10:23,828 root [SSL: CERTIFICATE_VERIFY_FAILED] >> certificate verify failed (_ssl.c:579) Traceback (most recent call >> last): File "/usr/share/ovirt-provider-ovn/handlers/base_handler.py", >> line 133, in _handle_request method, path_parts, content >> File "/usr/share/ovirt-provider-ovn/handlers/selecting_handler.py", >> line 175, in handle_request return >> self.call_response_handler(handler, content, parameters) File >> "/usr/share/ovirt-provider-ovn/handlers/keystone.py", line 33, in >> call_response_handler return response_handler(content, parameters) >> File "/usr/share/ovirt-provider-ovn/handlers/keystone_responses.py", >> line 62, in post_tokens user_password=user_password) File >> "/usr/share/ovirt-provider-ovn/auth/plugin_facade.py", line 26, in >> create_token return auth.core.plugin.create_token(user_at_domain, >> user_password) File >> "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/plugin.py", line >> 48, in create_token timeout=self._timeout()) File >> "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/sso.py", line 75, >> in create_token username, password, engine_url, ca_file, timeout) >> File "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/sso.py", line >> 91, in _get_sso_token timeout=timeout File >> "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/sso.py", line 54, >> in wrapper response = func(*args, **kwargs) File >> "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/sso.py", line 47, >> in wrapper raise BadGateway(e) BadGateway: [SSL: >> CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579) >> >> >> [root@engine ~]# tail -n 20 /var/log/ovirt-engine/engine.log >> 2018-09-12 14:10:23,773+03 INFO >> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> (EE-ManagedThreadFactory-engineScheduled-Thread-47) [316db685] Lock >> Acquired to object >> 'EngineLock:{exclusiveLocks='[14e4fb72-9764-4757-b37d-4d487995571a=PROVIDER]', >> sharedLocks=''}' 2018-09-12 14:10:23,778+03 INFO >> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> (EE-ManagedThreadFactory-engineScheduled-Thread-47) [316db685] >> Running command: SyncNetworkProviderCommand internal: true. >> 2018-09-12 14:10:23,836+03 ERROR >> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> (EE-ManagedThreadFactory-engineScheduled-Thread-47) [316db685] >> Command >> 'org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand' >> failed: EngineException: (Failed with error Bad Gateway and code >> 5050) 2018-09-12 14:10:23,837+03 INFO >> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> (EE-ManagedThreadFactory-engineScheduled-Thread-47) [316db685] Lock >> freed to object >> 'EngineLock:{exclusiveLocks='[14e4fb72-9764-4757-b37d-4d487995571a=PROVIDER]', >> sharedLocks=''}' 2018-09-12 14:14:12,477+03 INFO >> [org.ovirt.engine.core.sso.utils.AuthenticationUtils] (default >> task-6) [] User admin@internal successfully logged in with scopes: >> ovirt-app-admin ovirt-app-api ovirt-app-portal >> ovirt-ext=auth:sequence-priority=~ ovirt-ext=revoke:revoke-all >> ovirt-ext=token-info:authz-search >> ovirt-ext=token-info:public-authz-search >> ovirt-ext=token-info:validate ovirt-ext=token:password-access >> 2018-09-12 14:14:12,587+03 INFO >> [org.ovirt.engine.core.bll.aaa.CreateUserSessionCommand] (default >> task-6) [1bf1b763] Running command: CreateUserSessionCommand >> internal: false. 2018-09-12 14:14:12,628+03 INFO >> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >> (default task-6) [1bf1b763] EVENT_ID: USER_VDC_LOGIN(30), User >> admin@internal-authz connecting from '10.0.3.61' using session >> 's8jAm7BUJGlicthm6yZBA3CUM8QpRdtwFaK3M/IppfhB3fHFB9gmNf0cAlbl1xIhcJ2WX+ww7e71Ri+MxJSsIg==' >> logged in. 2018-09-12 14:14:30,972+03 INFO >> [org.ovirt.engine.core.bll.provider.ImportProviderCertificateCommand] >> (default task-6) [ee3cc8a7-4485-4fdf-a0c2-e9d67b5cfcd3] Running >> command: ImportProviderCertificateCommand internal: false. Entities >> affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: >> SystemAction group CREATE_STORAGE_POOL with role type ADMIN >> 2018-09-12 14:14:30,982+03 INFO >> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >> (default task-6) [ee3cc8a7-4485-4fdf-a0c2-e9d67b5cfcd3] EVENT_ID: >> PROVIDER_CERTIFICATE_IMPORTED(213), Certificate for provider >> ovirt-provider-ovn was imported. (User: admin@internal-authz) >> 2018-09-12 14:14:31,006+03 INFO >> [org.ovirt.engine.core.bll.provider.TestProviderConnectivityCommand] >> (default task-6) [a48d94ab-b0b2-42a2-a667-0525b4c652ea] Running >> command: TestProviderConnectivityCommand internal: false. Entities >> affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: >> SystemAction group CREATE_STORAGE_POOL with role type ADMIN >> 2018-09-12 14:14:31,058+03 ERROR >> [org.ovirt.engine.core.bll.provider.TestProviderConnectivityCommand] >> (default task-6) [a48d94ab-b0b2-42a2-a667-0525b4c652ea] Command >> 'org.ovirt.engine.core.bll.provider.TestProviderConnectivityCommand' >> failed: EngineException: (Failed with error Bad Gateway and code >> 5050) 2018-09-12 14:15:10,954+03 INFO >> [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] >> (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread >> pool 'default' is using 0 threads out of 1, 5 threads waiting for >> tasks. 2018-09-12 14:15:10,954+03 INFO >> [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] >> (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread >> pool 'engine' is using 0 threads out of 500, 16 threads waiting for >> tasks and 0 tasks in queue. 2018-09-12 14:15:10,954+03 INFO >> [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] >> (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread >> pool 'engineScheduled' is using 0 threads out of 100, 100 threads >> waiting for tasks. 2018-09-12 14:15:10,954+03 INFO >> [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] >> (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread >> pool 'engineThreadMonitoring' is using 1 threads out of 1, 0 threads >> waiting for tasks. 2018-09-12 14:15:10,954+03 INFO >> [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] >> (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread >> pool 'hostUpdatesChecker' is using 0 threads out of 5, 2 threads >> waiting for tasks. 2018-09-12 14:15:23,843+03 INFO >> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> (EE-ManagedThreadFactory-engineScheduled-Thread-61) [2455041f] Lock >> Acquired to object >> 'EngineLock:{exclusiveLocks='[14e4fb72-9764-4757-b37d-4d487995571a=PROVIDER]', >> sharedLocks=''}' 2018-09-12 14:15:23,849+03 INFO >> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> (EE-ManagedThreadFactory-engineScheduled-Thread-61) [2455041f] >> Running command: SyncNetworkProviderCommand internal: true. >> 2018-09-12 14:15:23,900+03 ERROR >> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> (EE-ManagedThreadFactory-engineScheduled-Thread-61) [2455041f] >> Command >> 'org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand' >> failed: EngineException: (Failed with error Bad Gateway and code >> 5050) 2018-09-12 14:15:23,901+03 INFO >> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> (EE-ManagedThreadFactory-engineScheduled-Thread-61) [2455041f] Lock >> freed to object >> 'EngineLock:{exclusiveLocks='[14e4fb72-9764-4757-b37d-4d487995571a=PROVIDER]', >> sharedLocks=''}' >> >> >> [root@engine ~]# >> cat /etc/ovirt-provider-ovn/conf.d/10-setup-ovirt-provider-ovn.conf # >> This file is automatically generated by engine-setup. Please do not >> edit manually [OVN REMOTE] ovn-remote=ssl:127.0.0.1:6641 >> [SSL] >> https-enabled=true >> ssl-cacert-file=/etc/pki/ovirt-engine/ca.pem >> ssl-cert-file=/etc/pki/ovirt-engine/certs/ovirt-provider-ovn.cer >> ssl-key-file=/etc/pki/ovirt-engine/keys/ovirt-provider-ovn.key.nopass >> [OVIRT] >> ovirt-sso-client-secret=Ms7Gw9qNT6IkXu7oA54tDmxaZDIukABV >> ovirt-host=https://engine.set.local:443 >> ovirt-sso-client-id=ovirt-provider-ovn >> ovirt-ca-file=/etc/pki/ovirt-engine/apache-ca.pem >> [PROVIDER] >> provider-host=engine.set.local >> >> >>> 12 сент. 2018 г., в 13:59, Dominik Holler <dholler(a)redhat.com> >>> написал(а): >>> >>> On Wed, 12 Sep 2018 13:04:53 +0300 >>> "Mail SET Inc. Group" <mail(a)set-pro.net> wrote: >>> >>>> Hello Dominik! >>>> I have a same issue with OVN provider and SSL >>>> https://www.mail-archive.com/users@ovirt.org/msg47020.html >>>> <https://www.mail-archive.com/users@ovirt.org/msg47020.html> But >>>> certificate changes not helps to resolve it. Maybe you can help me >>>> with this? >>> >>> Sure. Can you please share the relevant lines of >>> ovirt-provider-ovn.log and engine.log, and the information if you >>> are using the certificates generated by engine-setup with >>> users(a)ovirt.org ? Thanks, >>> Dominik >>> >> > >

5 13

engine-setup failing on 4.3.2 -> 4.3.3 fails during Engine schema refresh fail
by Edward Berger 18 Jul '23

18 Jul '23

I was trying to upgrade a hyperconverged oVirt hosted engine and failed in the engine-setup command with these error and warnings. ... [ INFO ] Creating/refreshing Engine database schema [ ERROR ] schema.sh: FATAL: Cannot execute sql command: --file=/usr/share/ovirt-engine/dbscripts/upgrade/04_03_0830_add_foreign_key_to_image_transfers.sql [ ERROR ] Failed to execute stage 'Misc configuration': Engine schema refresh failed ... [ INFO ] Yum Verify: 16/16: ovirt-engine-tools.noarch 0:4.3.3.5-1.el7 - e [WARNING] Rollback of DWH database postponed to Stage "Clean up" [ INFO ] Rolling back database schema ... [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination [ ERROR ] Execution of setup failed Attaching engine-setup logfile.

4 6

oVirt host deploy: disable multipath
by femi adegoke 27 Mar '23

27 Mar '23

https://bugzilla.redhat.com/show_bug.cgi?id=1472356 There are 2 drives in my install that oVirt is claiming as multipath. I have tried blacklisting at /etc/multipath/ but that hasn't worked. Any better solutions? I need these 2 drives to be non-multipathed.

4 19

Out-of-sync networks can only be detached
by Sakhi Hadebe 27 Feb '23

27 Feb '23

Hi, I have a 3-node oVirt cluster. I have configured 2 logical networks: ovirtmgmt and public. Public logical network is attached in only 2 nodes and failing to attach on the 3rd node with the below error Invalid operation, out-of-sync network 'public' can only be detached. Please have been stuck on this for almost the whole day now. How do I fix this error? -- Regards, Sakhi Hadebe

3 5

Network Address Change
by Paul.LKW 28 Dec '22

28 Dec '22

Hi All: I just has a case, I need to change the oVirt host and engine IP address due to data center decommission I checked in the hosted-engine host there are some files I could change ; in ovirt-hosted-engine/hosted-engine.conf ca_subject="O=simple.com, CN=1.2.3.4" gateway=1.2.3.254 and of course I need to change the ovirtmgmt interface IP too, I think just change the above line could do the tick, but where could I change the other host IP in the cluster ? I think I have to be lost all the host as once changed the hosted-engine host IP as it is in diff. sub net. Does there any command line tools could do that or someone has such experience could share? Best Regards, Paul.LKW

4 6

looking for some ISV backup software which integrates the backup API
by Nathanaël Blanchet 08 Jun '22

08 Jun '22

Hello, We are about to change our backup provider, and I find it is a great chance to choose a full supported ovirt backup solution. I currently use this python script vm-backup-scheduler (https://github.com/wefixit-AT/oVirtBackup) but it is not the workflow officially suggested by the community (https://www.ovirt.org/develop/release-management/features/storage/backup-re…) I've been looking for a long time an ISV who supports such an API, but the only one I found is this one : Acronis Backup Advanced suggested here https://access.redhat.com/ecosystem/search/#/ecosystem/Red%20Hat%20Enterpri… I ran the trial version, but it doesn't seem to do better than the vm-backup-scheduler script, and it doesn't seem to use the backup API (attach a clone as a disk to an existing vm). Can you suggest me some other ISV solutions, if they ever exist... or share me your backup experience?

5 4

OVS switch type for hosted-engine
by Devin A. Bougie 06 May '22

06 May '22

Is it possible to setup a hosted engine using the OVS switch type instead of Legacy? If it's not possible to start out as OVS, instructions for switching from Legacy to OVS after the fact would be greatly appreciated. Many thanks, Devin

4 5

USB3 redirection
by Rik Theys 19 Apr '22

19 Apr '22

Hi, I'm trying to assign a USB3 controller to a CentOS 7.4 VM in oVirt 4.1 with USB redirection enabled. I've created the following file in /etc/ovirt-engine/osinfo.conf.d: 01-usb.properties with content os.other.devices.usb.controller.value = nec-xhci and have restarted ovirt-engine. If I disable USB-support in the web interface for the VM, the xhci controller is added to the VM (I can see it in the qemu-kvm commandline), but usb redirection is not available. If I enable USB-support in the UI, no xhci controller is added (only 4 uhci controllers). Is there a way to make the controllers for usb redirection xhci controllers? Regards, Rik -- Rik Theys System Engineer KU Leuven - Dept. Elektrotechniek (ESAT) Kasteelpark Arenberg 10 bus 2440 - B-3001 Leuven-Heverlee +32(0)16/32.11.07 ---------------------------------------------------------------- <<Any errors in spelling, tact or fact are transmission errors>>

3 5

OVN routing and firewalling in oVirt
by Gianluca Cecchi 19 Apr '22

19 Apr '22

Hello, how do we manage routing between different OVN networks in oVirt? And between OVN networks and physical ones? Based on architecture read here: http://openvswitch.org/support/dist-docs/ovn-architecture.7.html I see terms for logical routers and gateway routers respectively but how to apply to oVirt configuration? Do I have to choose between setting up a specialized VM or a physical one: is it applicable/advisable to put on oVirt host itself the gateway functionality? Is there any security policy (like security groups in Openstack) to implement? Thanks, Gianluca

4 6

Install hosted-engine - Task Get local VM IP failed
by florentl 10 Mar '22

10 Mar '22

Hi all, I try to install hosted-engine on node : ovirt-node-ng-4.2.3-0.20180518. Every times I get stuck on : [ ERROR ] fatal: [localhost]: FAILED! => {"attempts": 50, "changed": true, "cmd": "virsh -r net-dhcp-leases default | grep -i 00:16:3e:6c:5a:91 | awk '{ print $5 }' | cut -f1 -d'/'", "delta": "0:00:00.108872", "end": "2018-06-01 11:17:34.421769", "rc": 0, "start": "2018-06-01 11:17:34.312897", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} I tried with static IP Address and with DHCP but both failed. To be more specific, I installed three nodes, deployed glusterfs with the wizard. I'm in a nested virtualization environment for this lab (Vmware Esxi Hypervisor). My node IP is : 192.168.176.40 / and I want the hosted-engine vm has 192.168.176.43. Thanks, Florent

7 17

Host needs to be reinstalled after configuring power management
by Andrew DeMaria 02 Feb '22

02 Feb '22

Hi, I am running ovirt 4.3 and have found the following action item immediately after configuring power management for a host: Host needs to be reinstalled as important configuration changes were applied on it. The thing is - I've just freshly installed this host and it seems strange that I need to reinstall it. Is there a better way to install a host and configure power management without having to reinstall it after? Thanks, Andrew

4 3

Lots of storage.MailBox.SpmMailMonitor
by Fabrice Bacchella 06 Jan '22

06 Jan '22

My vdsm log files are huge: -rw-r--r-- 1 vdsm kvm 1.8G Nov 22 11:32 vdsm.log And this is juste half an hour of logs: $ head -1 vdsm.log 2018-11-22 11:01:12,132+0100 ERROR (mailbox-spm) [storage.MailBox.SpmMailMonitor] mailbox 2 checksum failed, not clearing mailbox, clearing new mail (data='...lots of data', expected='\xa4\x06\x08\x00') (mailbox:612) I just upgraded vdsm: $ rpm -qi vdsm Name : vdsm Version : 4.20.43

3 5

poweroff and reboot with ovirt_vm ansible module
by Nathanaël Blanchet 18 May '21

18 May '21

Hello, is there a way to poweroff or reboot (without stopped and running state) a vm with the ovirt_vm ansible module? -- Nathanaël Blanchet Supervision réseau Pôle Infrastrutures Informatiques 227 avenue Professeur-Jean-Louis-Viala 34193 MONTPELLIER CEDEX 5 Tél. 33 (0)4 67 54 84 55 Fax 33 (0)4 67 54 84 14 blanchet(a)abes.fr

7 11

OVN and change of mgmt network
by Gianluca Cecchi 26 Jan '21

26 Jan '21

Hello, I previously had OVN running on engine (as OVN provider with northd and northbound and southbound DBs) and hosts (with OVN controller). After changing mgmt ip of hosts (engine has retained instead the same ip), I executed again on them the command: vdsm-tool ovn-config <ip_of_engine> <nel_local_ip_of_host> Now I think I have to clean up some things, eg: 1) On engine where I get these lines below systemctl status ovn-northd.service -l . . . Sep 29 14:41:42 ovmgr1 ovsdb-server[940]: ovs|00005|reconnect|ERR|tcp: 10.4.167.40:37272: no response to inactivity probe after 5 seconds, disconnecting Oct 03 11:52:00 ovmgr1 ovsdb-server[940]: ovs|00006|reconnect|ERR|tcp: 10.4.167.41:52078: no response to inactivity probe after 5 seconds, disconnecting The two IPs are the old ones of two hosts It seems that a restart of the services has fixed... Can anyone confirm if I have to do anything else? 2) On hosts (there are 3 hosts with OVN on ip 10.4.192.32/33/34) where I currently have this output [root@ov301 ~]# ovs-vsctl show 3a38c5bb-0abf-493d-a2e6-345af8aedfe3 Bridge br-int fail_mode: secure Port "ovn-1dce5b-0" Interface "ovn-1dce5b-0" type: geneve options: {csum="true", key=flow, remote_ip="10.4.192.32"} Port "ovn-ddecf0-0" Interface "ovn-ddecf0-0" type: geneve options: {csum="true", key=flow, remote_ip="10.4.192.33"} Port "ovn-fd413b-0" Interface "ovn-fd413b-0" type: geneve options: {csum="true", key=flow, remote_ip="10.4.168.74"} Port br-int Interface br-int type: internal ovs_version: "2.7.2" [root@ov301 ~]# The IPs of kind 10.4.192.x are ok. But there is a left-over of an old host I initially used for tests, corresponding to 10.4.168.74, that now doesn't exist anymore How can I clean records for 1) and 2)? Thanks, Gianluca

3 4

Can't import some VMs after storage domain detach and reattach to new datacenter.
by m black 10 Dec '20

10 Dec '20

7 14

"gluster-ansible-roles is not installed on Host" error on Cockpit
by Hesham Ahmed 26 Nov '20

26 Nov '20

On a new 4.3.1 oVirt Node installation, when trying to deploy HCI (also when trying adding a new gluster volume to existing clusters) using Cockpit, an error is displayed "gluster-ansible-roles is not installed on Host. To continue deployment, please install gluster-ansible-roles on Host and try again". There is no package named gluster-ansible-roles in the repositories: [root@localhost ~]# yum install gluster-ansible-roles Loaded plugins: enabled_repos_upload, fastestmirror, imgbased-persist, package_upload, product-id, search-disabled-repos, subscription-manager, vdsmupgrade This system is not registered with an entitlement server. You can use subscription-manager to register. Loading mirror speeds from cached hostfile * ovirt-4.3-epel: mirror.horizon.vn No package gluster-ansible-roles available. Error: Nothing to do Uploading Enabled Repositories Report Cannot upload enabled repos report, is this client registered? This is due to check introduced here: https://gerrit.ovirt.org/#/c/98023/1/dashboard/src/helpers/AnsibleUtil.js Changing the line from: [ "rpm", "-qa", "gluster-ansible-roles" ], { "superuser":"require" } to [ "rpm", "-qa", "gluster-ansible" ], { "superuser":"require" } resolves the issue. The above code snippet is installed at /usr/share/cockpit/ovirt-dashboard/app.js on oVirt node and can be patched by running "sed -i 's/gluster-ansible-roles/gluster-ansible/g' /usr/share/cockpit/ovirt-dashboard/app.js && systemctl restart cockpit"

2 1

Error exporting into ova
by Gianluca Cecchi 31 Aug '20

31 Aug '20

Hello, I'm playing with export_vm_as_ova.py downloaded from the examples github: https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/export_v… My environment is oVirt 4.3.3.7 with iSCSI storage domain. It fails leaving an ova.tmp file In webadmin gui: Starting to export Vm enginecopy1 as a Virtual Appliance 7/19/1911:55:12 AM VDSM ov301 command TeardownImageVDS failed: Cannot deactivate Logical Volume: ('General Storage Exception: ("5 [] [\' Logical volume fa33df49-b09d-4f86-9719-ede649542c21/0420ef47-0ad0-4cf9-babd-d89383f7536b in use.\']\\nfa33df49-b09d-4f86-9719-ede649542c21/[\'a7480dc5-b5ca-4cb3-986d-77bc12165be4\', \'0420ef47-0ad0-4cf9-babd-d89383f7536b\']",)',) 7/19/1912:25:36 PM Failed to export Vm enginecopy1 as a Virtual Appliance to path /save_ova/base/dump/myvm2.ova on Host ov301 7/19/1912:25:37 PM During export I have this qemu-img process creating the disk over the loop device: root 30878 30871 0 11:55 pts/2 00:00:00 su -p -c qemu-img convert -T none -O qcow2 '/rhev/data-center/mnt/blockSD/fa33df49-b09d-4f86-9719-ede649542c21/images/59a4a324-4c99-4ff5-abb1-e9bbac83292a/0420ef47-0ad0-4cf9-babd-d89383f7536b' '/dev/loop1' vdsm vdsm 30882 30878 10 11:55 ? 00:00:00 qemu-img convert -T none -O qcow2 /rhev/data-center/mnt/blockSD/fa33df49-b09d-4f86-9719-ede649542c21/images/59a4a324-4c99-4ff5-abb1-e9bbac83292a/0420ef47-0ad0-4cf9-babd-d89383f7536b /dev/loop1 The ova.tmp file is getting filled while command runs eg: [root@ov301 ]# du -sh /save_ova/base/dump/myvm2.ova.tmp 416M /save_ova/base/dump/myvm2.ova.tmp [root@ov301 sysctl.d]# [root@ov301 sysctl.d]# du -sh /save_ova/base/dump/myvm2.ova.tmp 911M /save_ova/base/dump/myvm2.ova.tmp [root@ov301 ]# and the final generated / not completed file is in this state: [root@ov301 ]# qemu-img info /save_ova/base/dump/myvm2.ova.tmp image: /save_ova/base/dump/myvm2.ova.tmp file format: raw virtual size: 30G (32217446400 bytes) disk size: 30G [root@ov301 sysctl.d]# But I notice that the timestamp of the file is about 67 minutes after start of job and well after the notice of its failure.... [root@ov301 sysctl.d]# ll /save_ova/base/dump/ total 30963632 -rw-------. 1 root root 32217446400 Jul 19 13:02 myvm2.ova.tmp [root@ov301 sysctl.d]# [root@ov301 sysctl.d]# du -sh /save_ova/base/dump/myvm2.ova.tmp 30G /save_ova/base/dump/myvm2.ova.tmp [root@ov301 sysctl.d]# In engine.log the first error I see is 30 minutes after start 2019-07-19 12:25:31,563+02 ERROR [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] Ansible playbook execution failed: Timeout occurred while executing Ansible playbook. 2019-07-19 12:25:31,563+02 INFO [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] Ansible playbook command has exited with value: 1 2019-07-19 12:25:31,564+02 ERROR [org.ovirt.engine.core.bll.CreateOvaCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] Failed to create OVA. Please check logs for more details: /var/log/ovirt-engine/ova/ovirt-export-ova-ansible-20190719115531-ov301-2001ddf4.log 2019-07-19 12:25:31,565+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.TeardownImageVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] START, TeardownImageVDSCommand(HostName = ov301, ImageActionsVDSCommandParameters:{hostId='8ef1ce6f-4e38-486c-b3a4-58235f1f1d06'}), log id: 3d2246f7 2019-07-19 12:25:36,569+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM ov301 command TeardownImageVDS failed: Cannot deactivate Logical Volume: ('General Storage Exception: ("5 [] [\' Logical volume fa33df49-b09d-4f86-9719-ede649542c21/0420ef47-0ad0-4cf9-babd-d89383f7536b in use.\']\\nfa33df49-b09d-4f86-9719-ede649542c21/[\'a7480dc5-b5ca-4cb3-986d-77bc12165be4\', \'0420ef47-0ad0-4cf9-babd-d89383f7536b\']",)',) In ansible playbook suggested log file I don't see anything useful. It ends with timestamps when the script has been launched. Last lines are: 2019-07-19 11:55:33,877 p=5699 u=ovirt | TASK [ovirt-ova-export-pre-pack : Retrieving the temporary path for the OVA file] *** 2019-07-19 11:55:34,198 p=5699 u=ovirt | changed: [ov301] => { "changed": true, "dest": "/save_ova/base/dump/myvm2.ova.tmp", "gid": 0, "group": "root", "mode": "0600", "owner": "root", "secontext": "system_u:object_r:nfs_t:s0", "size": 32217446912, "state": "file", "uid": 0 } 2019-07-19 11:55:34,204 p=5699 u=ovirt | TASK [ovirt-ova-pack : Run packing script] ************************************* It seems 30 minutes... for timeout? About what, ansible job? Or possibly implicit user session created when running the python script? The snapshot has been correctly deleted (as I see also in engine.log), I don't see it in webadmin gui. Any known problem? Just for test I executed again at 14:24 and I see same Ansible error at 14:54 The snapshot gets deleted, while the qemu-img command still continues.... [root@ov301 sysctl.d]# ps -ef | grep qemu-img root 13504 13501 0 14:24 pts/1 00:00:00 su -p -c qemu-img convert -T none -O qcow2 '/rhev/data-center/mnt/blockSD/fa33df49-b09d-4f86-9719-ede649542c21/images/59a4a324-4c99-4ff5-abb1-e9bbac83292a/0420ef47-0ad0-4cf9-babd-d89383f7536b' '/dev/loop0' vdsm vdsm 13505 13504 3 14:24 ? 00:01:26 qemu-img convert -T none -O qcow2 /rhev/data-center/mnt/blockSD/fa33df49-b09d-4f86-9719-ede649542c21/images/59a4a324-4c99-4ff5-abb1-e9bbac83292a/0420ef47-0ad0-4cf9-babd-d89383f7536b /dev/loop0 root 17587 24530 0 15:05 pts/0 00:00:00 grep --color=auto qemu-img [root@ov301 sysctl.d]# [root@ov301 sysctl.d]# du -sh /save_ova/base/dump/myvm2.ova.tmp 24G /save_ova/base/dump/myvm2.ova.tmp [root@ov301 sysctl.d]# ll /save_ova/base/dump/myvm2.ova.tmp -rw-------. 1 root root 32217446400 Jul 19 15:14 /save_ova/base/dump/myvm2.ova.tmp [root@ov301 sysctl.d]# and then continues until image copy completes, but at this time the job has already aborted and so the completion of the ova composition doesn't go ahead... and I remain with the ova.tmp file... How to extend timeout? Thanks in advance, Gianluca

4 11

deprecating export domain?
by Charles Kozler 30 Aug '20

30 Aug '20

Hello, I recently read on this list from a redhat member that export domain is either being deprecated or looking at being deprecated To that end, can you share details? Can you share any notes/postings/bz's that document this? I would imagine something like this would be discussed in larger audience This seems like a somewhat significant change to make and I am curious where this is scheduled? Currently, a lot of my backups rely explicitly on an export domain for online snapshots, so I'd like to plan accordingly Thanks!

11 21

Re: Single instance scaleup.
by Strahil 05 Jun '20

05 Jun '20

Hi Leo, As you do not have a distributed volume , you can easily switch to replica 2 arbiter 1 or replica 3 volumes. You can use the following for adding the bricks: https://access.redhat.com/documentation/en-US/Red_Hat_Storage/2.1/html/Admi… Best Regards, Strahil NikolivOn May 26, 2019 10:54, Leo David <leoalex(a)gmail.com> wrote: > > Hi Stahil, > Thank you so much for yout input ! > > gluster volume info > > > Volume Name: engine > Type: Distribute > Volume ID: d7449fc2-cc35-4f80-a776-68e4a3dbd7e1 > Status: Started > Snapshot Count: 0 > Number of Bricks: 1 > Transport-type: tcp > Bricks: > Brick1: 192.168.80.191:/gluster_bricks/engine/engine > Options Reconfigured: > nfs.disable: on > transport.address-family: inet > storage.owner-uid: 36 > storage.owner-gid: 36 > features.shard: on > performance.low-prio-threads: 32 > performance.strict-o-direct: off > network.remote-dio: off > network.ping-timeout: 30 > user.cifs: off > performance.quick-read: off > performance.read-ahead: off > performance.io-cache: off > cluster.eager-lock: enable > Volume Name: ssd-samsung > Type: Distribute > Volume ID: 76576cc6-220b-4651-952d-99846178a19e > Status: Started > Snapshot Count: 0 > Number of Bricks: 1 > Transport-type: tcp > Bricks: > Brick1: 192.168.80.191:/gluster_bricks/sdc/data > Options Reconfigured: > cluster.eager-lock: enable > performance.io-cache: off > performance.read-ahead: off > performance.quick-read: off > user.cifs: off > network.ping-timeout: 30 > network.remote-dio: off > performance.strict-o-direct: on > performance.low-prio-threads: 32 > features.shard: on > storage.owner-gid: 36 > storage.owner-uid: 36 > transport.address-family: inet > nfs.disable: on > > The other two hosts will be 192.168.80.192/193 - this is gluster dedicated network over 10GB sfp+ switch. > - host 2 wil have identical harware configuration with host 1 ( each disk is actually a raid0 array ) > - host 3 has: > - 1 ssd for OS > - 1 ssd - for adding to engine volume in a full replica 3 > - 2 ssd's in a raid 1 array to be added as arbiter for the data volume ( ssd-samsung ) > So the plan is to have "engine" scaled in a full replica 3, and "ssd-samsung" scalled in a replica 3 arbitrated. > > > > > On Sun, May 26, 2019 at 10:34 AM Strahil <hunter86_bg(a)yahoo.com> wrote: >> >> Hi Leo, >> >> Gluster is quite smart, but in order to provide any hints , can you provide output of 'gluster volume info <glustervol>'. >> If you have 2 more systems , keep in mind that it is best to mirror the storage on the second replica (2 disks on 1 machine -> 2 disks on the new machine), while for the arbiter this is not neccessary. >> >> What is your network and NICs ? Based on my experience , I can recommend at least 10 gbit/s interfase(s). >> >> Best Regards, >> Strahil Nikolov >> >> On May 26, 2019 07:52, Leo David <leoalex(a)gmail.com> wrote: >>> >>> Hello Everyone, >>> Can someone help me to clarify this ? >>> I have a single-node 4.2.8 installation ( only two gluster storage domains - distributed single drive volumes ). Now I just got two identintical servers and I would like to go for a 3 nodes bundle. >>> Is it possible ( after joining the new nodes to the cluster ) to expand the existing volumes across the new nodes and change them to replica 3 arbitrated ? >>> If so, could you share with me what would it be the procedure ? >>> Thank you very much ! >>> >>> Leo > > > > -- > Best regards, Leo David

4 5

Failed to add storage domain
by thunderlight1＠gmail.com 31 May '20

31 May '20

Hi! I have installed oVirt using the iso ovirt-node-ng-installer-4.3.2-2019031908.el7. I the did run the Host-engine deployment through Cockpit. I got an error when it tries to create the domain storage. It sucessfully mounted the NFS-share on the host. Bellow is the error I got: 2019-04-14 10:40:38,967+0200 INFO ansible skipped {'status': 'SKIPPED', 'ansible_task': u'Check storage domain free space', 'ansible_host': u'localhost', 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml', 'ansible_type': 'task'} 2019-04-14 10:40:38,967+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fb6918ad9d0> kwargs 2019-04-14 10:40:39,516+0200 INFO ansible task start {'status': 'OK', 'ansible_task': u'ovirt.hosted_engine_setup : Activate storage domain', 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml', 'ansible_type': 'task'} 2019-04-14 10:40:39,516+0200 DEBUG ansible on_any args TASK: ovirt.hosted_engine_setup : Activate storage domain kwargs is_conditional:False 2019-04-14 10:40:41,923+0200 DEBUG var changed: host "localhost" var "otopi_storage_domain_details" type "<type 'dict'>" value: "{ "changed": false, "exception": "Traceback (most recent call last):\n File \"/tmp/ansible_ovirt_storage_domain_payload_xSFxOp/__main__.py\", line 664, in main\n storage_domains_module.post_create_check(sd_id)\n File \"/tmp/ansible_ovirt_storage_domain_payload_xSFxOp/__main__.py\", line 526, in post_create_check\n id=storage_domain.id,\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py\", line 3053, in add\n return self._internal_add(storage_domain, headers, query, wait)\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py\", line 232, in _internal_add\n return future.wait() if wait else future\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py\", line 55, in wait\n return self._code(response)\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py\", line 229, in callback\n self._check_fault(response)\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py\", line 132, in _check_fault\n self._raise_error(response , body)\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py\", line 118, in _raise_error\n raise error\nError: Fault reason is \"Operation Failed\". Fault detail is \"[]\". HTTP response code is 400.\n", "failed": true, "msg": "Fault reason is \"Operation Failed\". Fault detail is \"[]\". HTTP response code is 400." }" 2019-04-14 10:40:41,924+0200 DEBUG var changed: host "localhost" var "ansible_play_hosts" type "<type 'list'>" value: "[]" 2019-04-14 10:40:41,924+0200 DEBUG var changed: host "localhost" var "play_hosts" type "<type 'list'>" value: "[]" 2019-04-14 10:40:41,924+0200 DEBUG var changed: host "localhost" var "ansible_play_batch" type "<type 'list'>" value: "[]" 2019-04-14 10:40:41,924+0200 ERROR ansible failed {'status': 'FAILED', 'ansible_type': 'task', 'ansible_task': u'Activate storage domain', 'ansible_result': u'type: <type \'dict\'>\nstr: {\'_ansible_parsed\': True, u\'exception\': u\'Traceback (most recent call last):\\n File "/tmp/ansible_ovirt_storage_domain_payload_xSFxOp/__main__.py", line 664, in main\\n storage_domains_module.post_create_check(sd_id)\\n File "/tmp/ansible_ovirt_storage_domain_payload_xSFxOp/__main__.py", line 526', 'task_duration': 2, 'ansible_host': u'localhost', 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml'} 2019-04-14 10:40:41,924+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fb691843190> kwargs ignore_errors:None 2019-04-14 10:40:41,928+0200 INFO ansible stats { "ansible_playbook": "/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml", "ansible_playbook_duration": "00:37 Minutes", "ansible_result": "type: <type 'dict'>\nstr: {u'localhost': {'unreachable': 0, 'skipped': 6, 'ok': 23, 'changed': 1, 'failures': 1}}", "ansible_type": "finish", "status": "FAILED" } 2019-04-14 10:40:41,928+0200 INFO SUMMARY: Duration Task Name -------- -------- [ < 1 sec ] Execute just a specific set of steps [ 00:01 ] Force facts gathering [ 00:01 ] Check local VM dir stat [ 00:01 ] Obtain SSO token using username/password credentials [ 00:01 ] Fetch host facts [ < 1 sec ] Fetch cluster ID [ 00:01 ] Fetch cluster facts [ 00:01 ] Fetch Datacenter facts [ < 1 sec ] Fetch Datacenter ID [ < 1 sec ] Fetch Datacenter name [ 00:02 ] Add NFS storage domain [ 00:01 ] Get storage domain details [ 00:01 ] Find the appliance OVF [ 00:01 ] Parse OVF [ < 1 sec ] Get required size [ FAILED ] Activate storage domain 2019-04-14 10:40:41,928+0200 DEBUG ansible on_any args <ansible.executor.stats.AggregateStats object at 0x7fb69404eb90> kwargs Any suggestions on how fix this?

2 2

How to connect to a guest with vGPU ?
by Josep Manel Andrés Moscardó 29 May '20

29 May '20

Hi, I got vGPU through mdev working but I am wondering how I would connect to the client and make use of the GPU. So far I try to access the console through SPICE and at some point in the boot process it switches to GPU and I cannot see anything else. Thanks. -- Josep Manel Andrés Moscardó Systems Engineer, IT Operations EMBL Heidelberg T +49 6221 387-8394

3 4

Vm suddenly paused with error "vm has paused due to unknown storage error"
by Jasper Siero 18 Feb '20

18 Feb '20

Hi all, Since we upgraded our Ovirt nodes to CentOS 7 a vm (not a specific one but never more then one) will sometimes pause suddenly with the error "VM ... has paused due to unknown storage error". It happens now two times in a month. The Ovirt node uses san storage for the vm's running on it. When a specific vm is pausing with an error the other vm's keeps running without problems. The vm runs without problems after unpausing it. Versions: CentOS Linux release 7.1.1503 vdsm-4.14.17-0 libvirt-daemon-1.2.8-16 vdsm.log: VM Channels Listener::DEBUG::2015-10-25 07:43:54,382::vmChannels::95::vds::(_handle_timeouts) Timeout on fileno 78. libvirtEventLoop::INFO::2015-10-25 07:43:56,177::vm::4602::vm.Vm::(_onIOError) vmId=`77f07ae0-cc3e-4ae2-90ec-7fba7b11deeb`::abnormal vm stop device virtio-disk0 error eother libvirtEventLoop::DEBUG::2015-10-25 07:43:56,178::vm::5204::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`77f07ae0-cc3e-4ae2-90ec-7fba7b11deeb`::event Suspended detail 2 opaque None libvirtEventLoop::INFO::2015-10-25 07:43:56,178::vm::4602::vm.Vm::(_onIOError) vmId=`77f07ae0-cc3e-4ae2-90ec-7fba7b11deeb`::abnormal vm stop device virtio-disk0 error eother ........... libvirtEventLoop::INFO::2015-10-25 07:43:56,180::vm::4602::vm.Vm::(_onIOError) vmId=`77f07ae0-cc3e-4ae2-90ec-7fba7b11deeb`::abnormal vm stop device virtio-disk0 error eother specific error part in libvirt vm log: block I/O error in device 'drive-virtio-disk0': Unknown error 32758 (32758) ........... block I/O error in device 'drive-virtio-disk0': Unknown error 32758 (32758) engine.log: 2015-10-25 07:44:48,945 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-40) [a43dcc8] VM diataal-prod-cas1 77f07ae0-cc3e-4ae2-90ec-7fba7b11deeb moved from Up --> Paused 2015-10-25 07:44:49,003 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-40) [a43dcc8] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM diataal-prod-cas1 has paused due to unknown storage error. Has anyone experienced the same problem or knows a way to solve this? Kind regards, Jasper

3 3