December 2019 - Users - oVirt List Archives

Unable to install oVirt on RHEL7.5
by SS00514758＠techmahindra.com 15 Jan '26

15 Jan '26

Hi All, I am unable to install oVirt on RHEL7.5, to install it I am taking reference of below link, https://www.ovirt.org/documentation/install-guide/chap-Installing_oVirt.html But though it is not working for me, couple of dependencies is not getting installed, and because of this I am not able to run the ovirt-engine, below are the depencies packages that unable to install, Error: Package: collectd-write_http-5.8.0-6.1.el7.x86_64 (@ovirt-4.2-centos-opstools) Requires: collectd(x86-64) = 5.8.0-6.1.el7 Removing: collectd-5.8.0-6.1.el7.x86_64 (@ovirt-4.2-centos-opstools) collectd(x86-64) = 5.8.0-6.1.el7 Updated By: collectd-5.8.1-1.el7.x86_64 (epel) collectd(x86-64) = 5.8.1-1.el7 Available: collectd-5.7.2-1.el7.x86_64 (ovirt-4.2-centos-opstools) collectd(x86-64) = 5.7.2-1.el7 Available: collectd-5.7.2-3.el7.x86_64 (ovirt-4.2-centos-opstools) collectd(x86-64) = 5.7.2-3.el7 Available: collectd-5.8.0-2.el7.x86_64 (ovirt-4.2-centos-opstools) collectd(x86-64) = 5.8.0-2.el7 Available: collectd-5.8.0-3.el7.x86_64 (ovirt-4.2-centos-opstools) collectd(x86-64) = 5.8.0-3.el7 Available: collectd-5.8.0-5.el7.x86_64 (ovirt-4.2-centos-opstools) collectd(x86-64) = 5.8.0-5.el7 Help me to install this. Looking forward to resolve this issue. Regards Sumit Sahay

8 10

SPM and Task error ...
by Enrico 22 Nov '24

22 Nov '24

Hi all, my ovirt cluster has got 3 Hypervisors runnig Centos 7.5.1804 vdsm is 4.20.39.1-1.el7, ovirt engine is 4.2.4.5-1.el7, the storage systems are HP MSA P2000 and 2050 (fibre channel). I need to stop one of the hypervisors for maintenance but this system is the storage pool manager. For this reason I decided to manually activate SPM in one of the other nodes but this operation is not successful. In the ovirt engine (engine.log) the error is this: 2019-07-25 12:39:16,744+02 INFO [org.ovirt.engine.core.bll.storage.pool.ForceSelectSPMCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] Running command: ForceSelectSPMCommand internal: false. Entities affected : ID: 81c9bd3c-ae0a-467f-bf7f-63ab30cd8d9e Type: VDSAction group MANIPULATE_HOST with role type ADMIN 2019-07-25 12:39:16,745+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SpmStopOnIrsVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] START, SpmStopOnIrsVDSCommand( SpmStopOnIrsVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7', ignoreFailoverLimit='false'}), log id: 37bf4639 2019-07-25 12:39:16,747+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] START, ResetIrsVDSCommand( ResetIrsVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7', ignoreFailoverLimit='false', vdsId='751f3e99-b95e-4c31-bc38-77f5661a0bdc', ignoreStopFailed='false'}), log id: 2522686f 2019-07-25 12:39:16,749+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] START, SpmStopVDSCommand(HostName = infn-vm05.management, SpmStopVDSCommandParameters:{hostId='751f3e99-b95e-4c31-bc38-77f5661a0bdc', storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7'}), log id: 1810fd8b 2019-07-25 12:39:16,758+02 *ERROR* [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] SpmStopVDSCommand::Not stopping SPM on vds 'infn-vm05.management', pool id '18d57688-6ed4-43b8-bd7c-0665b55950b7' as there are uncleared tasks 'Task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e', status 'running'' 2019-07-25 12:39:16,758+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] FINISH, SpmStopVDSCommand, log id: 1810fd8b 2019-07-25 12:39:16,758+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] FINISH, ResetIrsVDSCommand, log id: 2522686f 2019-07-25 12:39:16,758+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SpmStopOnIrsVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] FINISH, SpmStopOnIrsVDSCommand, log id: 37bf4639 2019-07-25 12:39:16,760+02 *ERROR* [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] EVENT_ID: USER_FORCE_SELECTED_SPM_STOP_FAILED(4,096), Failed to force select infn-vm07.management as the SPM due to a failure to stop the current SPM. while in the hypervisor (SPM) vdsm.log: 2019-07-25 12:39:16,744+02 INFO [org.ovirt.engine.core.bll.storage.pool.ForceSelectSPMCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] Running command: ForceSelectSPMCommand internal: false. Entities affected : ID: 81c9bd3c-ae0a-467f-bf7f-63ab30cd8d9e Type: VDSAction group MANIPULATE_HOST with role type ADMIN 2019-07-25 12:39:16,745+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SpmStopOnIrsVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] START, SpmStopOnIrsVDSCommand( SpmStopOnIrsVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7', ignoreFailoverLimit='false'}), log id: 37bf4639 2019-07-25 12:39:16,747+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] START, ResetIrsVDSCommand( ResetIrsVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7', ignoreFailoverLimit='false', vdsId='751f3e99-b95e-4c31-bc38-77f5661a0bdc', ignoreStopFailed='false'}), log id: 2522686f 2019-07-25 12:39:16,749+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] START, SpmStopVDSCommand(HostName = infn-vm05.management, SpmStopVDSCommandParameters:{hostId='751f3e99-b95e-4c31-bc38-77f5661a0bdc', storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7'}), log id: 1810fd8b 2019-07-25 12:39:16,758+02 *ERROR* [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] SpmStopVDSCommand::Not stopping SPM on vds 'infn-vm05.management', pool id '18d57688-6ed4-43b8-bd7c-0665b55950b7' as there are uncleared tasks 'Task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e', status 'running'' 2019-07-25 12:39:16,758+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] FINISH, SpmStopVDSCommand, log id: 1810fd8b 2019-07-25 12:39:16,758+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] FINISH, ResetIrsVDSCommand, log id: 2522686f 2019-07-25 12:39:16,758+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SpmStopOnIrsVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] FINISH, SpmStopOnIrsVDSCommand, log id: 37bf4639 2019-07-25 12:39:16,760+02 *ERROR* [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] EVENT_ID: USER_FORCE_SELECTED_SPM_STOP_FAILED(4,096), Failed to force select infn-vm07.management as the SPM due to a failure to stop the current SPM. 2019-07-25 12:39:18,660+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Task id 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' has passed pre-polling period time and should be polled. Pre-polling period is 60000 millis. 2019-07-25 12:39:18,660+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Task id 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' has passed pre-polling period time and should be polled. Pre-polling period is 60000 millis. 2019-07-25 12:39:18,750+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Task id 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' has passed pre-polling period time and should be polled. Pre-polling period is 60000 millis. 2019-07-25 12:39:18,750+02 *ERROR* [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] BaseAsyncTask::logEndTaskFailure: Task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' (Parent Command 'Unknown', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure: 2019-07-25 12:39:18,750+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' 2019-07-25 12:39:18,751+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7', ignoreFailoverLimit='false', taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: 34ae2b2f 2019-07-25 12:39:18,752+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] START, HSMClearTaskVDSCommand(HostName = infn-vm05.management, HSMTaskGuidBaseVDSCommandParameters:{hostId='751f3e99-b95e-4c31-bc38-77f5661a0bdc', taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: d3a78ad 2019-07-25 12:39:18,757+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] FINISH, HSMClearTaskVDSCommand, log id: d3a78ad 2019-07-25 12:39:18,757+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] FINISH, SPMClearTaskVDSCommand, log id: 34ae2b2f 2019-07-25 12:39:18,757+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] SPMAsyncTask::ClearAsyncTask: At time of attempt to clear task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' the response code was 'TaskStateError' and message was 'Operation is not allowed in this task state: ("can't clean in state running",)'. Task will not be cleaned 2019-07-25 12:39:18,757+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] BaseAsyncTask::onTaskEndSuccess: Task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' (Parent Command 'Unknown', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2019-07-25 12:39:18,757+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' 2019-07-25 12:39:18,758+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7', ignoreFailoverLimit='false', taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: 42de0c2b 2019-07-25 12:39:18,759+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] START, HSMClearTaskVDSCommand(HostName = infn-vm05.management, HSMTaskGuidBaseVDSCommandParameters:{hostId='751f3e99-b95e-4c31-bc38-77f5661a0bdc', taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: 4895c79c 2019-07-25 12:39:18,764+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] FINISH, HSMClearTaskVDSCommand, log id: 4895c79c 2019-07-25 12:39:18,764+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] FINISH, SPMClearTaskVDSCommand, log id: 42de0c2b 2019-07-25 12:39:18,764+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] SPMAsyncTask::ClearAsyncTask: At time of attempt to clear task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' the response code was 'TaskStateError' and message was 'Operation is not allowed in this task state: ("can't clean in state running",)'. Task will not be cleaned 2019-07-25 12:39:18,764+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Task id 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' has passed pre-polling period time and should be polled. Pre-polling period is 60000 millis. 2019-07-25 12:39:18,764+02 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Cleaning zombie tasks: Clearing async task 'Unknown' that started at 'Fri May 03 14:48:50 CEST 2019' 2019-07-25 12:39:18,764+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' 2019-07-25 12:39:18,765+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7', ignoreFailoverLimit='false', taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: da77af2 2019-07-25 12:39:18,766+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] START, HSMClearTaskVDSCommand(HostName = infn-vm05.management, HSMTaskGuidBaseVDSCommandParameters:{hostId='751f3e99-b95e-4c31-bc38-77f5661a0bdc', taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: 530694fb 2019-07-25 12:39:18,771+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] FINISH, HSMClearTaskVDSCommand, log id: 530694fb 2019-07-25 12:39:18,771+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] FINISH, SPMClearTaskVDSCommand, log id: da77af2 2019-07-25 12:39:18,771+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] SPMAsyncTask::ClearAsyncTask: At time of attempt to clear task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' the response code was 'TaskStateError' and message was 'Operation is not allowed in this task state: ("can't clean in state running",)'. Task will not be cleaned there is some relation between this error and a task that has remained hanging, from SPM server: # vdsm-client Task getInfo taskID=fdcf4d1b-82fe-49a6-b233-323ebe568f8e { "verb": "prepareMerge", "id": "fdcf4d1b-82fe-49a6-b233-323ebe568f8e" } # vdsm-client Task getStatus taskID=fdcf4d1b-82fe-49a6-b233-323ebe568f8e { "message": "running job 1 of 1", "code": 0, "taskID": "fdcf4d1b-82fe-49a6-b233-323ebe568f8e", "taskResult": "", "taskState": "running" } How can I solve this problem ? Thanks a lot for your help !! Best Regards Enrico -- _______________________________________________________________________ Enrico Becchetti Servizio di Calcolo e Reti Istituto Nazionale di Fisica Nucleare - Sezione di Perugia Via Pascoli,c/o Dipartimento di Fisica 06123 Perugia (ITALY) Phone:+39 075 5852777 Mail: Enrico.Becchetti<at>pg.infn.it _______________________________________________________________________

5 15

Changing disk QoS causes segfault with IO-Threads enabled (oVirt 4.3.0.4-1.el7)
by jloh＠squiz.net 23 Apr '24

23 Apr '24

We recently upgraded to 4.3.0 and have found that when changing disk QoS settings on VMs whilst IO-Threads is enabled causes them to segfault and the VM to reboot. We've been able to replicate this across several VMs. VMs with IO-Threads disabled/turned off do not segfault when changing the QoS. Mar 1 11:49:06 srvXX kernel: IO iothread1[30468]: segfault at fffffffffffffff8 ip 0000557649f2bd24 sp 00007f80de832f60 error 5 in qemu-kvm[5576498dd000+a03000] Mar 1 11:49:06 srvXX abrt-hook-ccpp: invalid number 'iothread1' Mar 1 11:49:11 srvXX libvirtd: 2019-03-01 00:49:11.116+0000: 13365: error : qemuMonitorIORead:609 : Unable to read from monitor: Connection reset by peer Happy to supply some more logs to someone if they'll help but just wondering whether anyone else has experienced this or knows of a current fix other than turning io-threads off. Cheers.

4 3

Re: Failed to synchronize networks of Provider ovirt-provider-ovn
by Mail SET Inc. Group 27 Oct '23

27 Oct '23

Yes, i use same manual to change WebUI SSL. ovirt-ca-file= is a same SSL file which use WebUI. Yes, i restart ovirt-provider-ovn, i restart engine, i restart all what i can restart. Nothing... > 12 сент. 2018 г., в 16:11, Dominik Holler <dholler(a)redhat.com> написал(а): > > On Wed, 12 Sep 2018 14:23:54 +0300 > "Mail SET Inc. Group" <mail(a)set-pro.net> wrote: > >> Ok! > > Not exactly, please use users(a)ovirt.org for such questions. > Other should benefit from this questions, too. > Please write the next mail to users(a)ovirt.org and keep me in CC. > >> What i did: >> >> 1) install oVirt «from box» (4.2.5.2-1.el7); >> 2) generate own ssl for my engine using my FreeIPA CA, Install it and > > What means "Install it"? You can use the doc from the following link > https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.2/ht… > > Ensure that ovirt-ca-file= in > /etc/ovirt-provider-ovn/conf.d/10-setup-ovirt-provider-ovn.conf > points to the correct file and ovirt-provider-ovn is restarted. > >> get tis issue; >> >> >> [root@engine ~]# tail -n 50 /var/log/ovirt-provider-ovn.log >> 2018-09-12 14:10:23,828 root [SSL: CERTIFICATE_VERIFY_FAILED] >> certificate verify failed (_ssl.c:579) Traceback (most recent call >> last): File "/usr/share/ovirt-provider-ovn/handlers/base_handler.py", >> line 133, in _handle_request method, path_parts, content >> File "/usr/share/ovirt-provider-ovn/handlers/selecting_handler.py", >> line 175, in handle_request return >> self.call_response_handler(handler, content, parameters) File >> "/usr/share/ovirt-provider-ovn/handlers/keystone.py", line 33, in >> call_response_handler return response_handler(content, parameters) >> File "/usr/share/ovirt-provider-ovn/handlers/keystone_responses.py", >> line 62, in post_tokens user_password=user_password) File >> "/usr/share/ovirt-provider-ovn/auth/plugin_facade.py", line 26, in >> create_token return auth.core.plugin.create_token(user_at_domain, >> user_password) File >> "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/plugin.py", line >> 48, in create_token timeout=self._timeout()) File >> "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/sso.py", line 75, >> in create_token username, password, engine_url, ca_file, timeout) >> File "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/sso.py", line >> 91, in _get_sso_token timeout=timeout File >> "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/sso.py", line 54, >> in wrapper response = func(*args, **kwargs) File >> "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/sso.py", line 47, >> in wrapper raise BadGateway(e) BadGateway: [SSL: >> CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579) >> >> >> [root@engine ~]# tail -n 20 /var/log/ovirt-engine/engine.log >> 2018-09-12 14:10:23,773+03 INFO >> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> (EE-ManagedThreadFactory-engineScheduled-Thread-47) [316db685] Lock >> Acquired to object >> 'EngineLock:{exclusiveLocks='[14e4fb72-9764-4757-b37d-4d487995571a=PROVIDER]', >> sharedLocks=''}' 2018-09-12 14:10:23,778+03 INFO >> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> (EE-ManagedThreadFactory-engineScheduled-Thread-47) [316db685] >> Running command: SyncNetworkProviderCommand internal: true. >> 2018-09-12 14:10:23,836+03 ERROR >> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> (EE-ManagedThreadFactory-engineScheduled-Thread-47) [316db685] >> Command >> 'org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand' >> failed: EngineException: (Failed with error Bad Gateway and code >> 5050) 2018-09-12 14:10:23,837+03 INFO >> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> (EE-ManagedThreadFactory-engineScheduled-Thread-47) [316db685] Lock >> freed to object >> 'EngineLock:{exclusiveLocks='[14e4fb72-9764-4757-b37d-4d487995571a=PROVIDER]', >> sharedLocks=''}' 2018-09-12 14:14:12,477+03 INFO >> [org.ovirt.engine.core.sso.utils.AuthenticationUtils] (default >> task-6) [] User admin@internal successfully logged in with scopes: >> ovirt-app-admin ovirt-app-api ovirt-app-portal >> ovirt-ext=auth:sequence-priority=~ ovirt-ext=revoke:revoke-all >> ovirt-ext=token-info:authz-search >> ovirt-ext=token-info:public-authz-search >> ovirt-ext=token-info:validate ovirt-ext=token:password-access >> 2018-09-12 14:14:12,587+03 INFO >> [org.ovirt.engine.core.bll.aaa.CreateUserSessionCommand] (default >> task-6) [1bf1b763] Running command: CreateUserSessionCommand >> internal: false. 2018-09-12 14:14:12,628+03 INFO >> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >> (default task-6) [1bf1b763] EVENT_ID: USER_VDC_LOGIN(30), User >> admin@internal-authz connecting from '10.0.3.61' using session >> 's8jAm7BUJGlicthm6yZBA3CUM8QpRdtwFaK3M/IppfhB3fHFB9gmNf0cAlbl1xIhcJ2WX+ww7e71Ri+MxJSsIg==' >> logged in. 2018-09-12 14:14:30,972+03 INFO >> [org.ovirt.engine.core.bll.provider.ImportProviderCertificateCommand] >> (default task-6) [ee3cc8a7-4485-4fdf-a0c2-e9d67b5cfcd3] Running >> command: ImportProviderCertificateCommand internal: false. Entities >> affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: >> SystemAction group CREATE_STORAGE_POOL with role type ADMIN >> 2018-09-12 14:14:30,982+03 INFO >> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >> (default task-6) [ee3cc8a7-4485-4fdf-a0c2-e9d67b5cfcd3] EVENT_ID: >> PROVIDER_CERTIFICATE_IMPORTED(213), Certificate for provider >> ovirt-provider-ovn was imported. (User: admin@internal-authz) >> 2018-09-12 14:14:31,006+03 INFO >> [org.ovirt.engine.core.bll.provider.TestProviderConnectivityCommand] >> (default task-6) [a48d94ab-b0b2-42a2-a667-0525b4c652ea] Running >> command: TestProviderConnectivityCommand internal: false. Entities >> affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: >> SystemAction group CREATE_STORAGE_POOL with role type ADMIN >> 2018-09-12 14:14:31,058+03 ERROR >> [org.ovirt.engine.core.bll.provider.TestProviderConnectivityCommand] >> (default task-6) [a48d94ab-b0b2-42a2-a667-0525b4c652ea] Command >> 'org.ovirt.engine.core.bll.provider.TestProviderConnectivityCommand' >> failed: EngineException: (Failed with error Bad Gateway and code >> 5050) 2018-09-12 14:15:10,954+03 INFO >> [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] >> (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread >> pool 'default' is using 0 threads out of 1, 5 threads waiting for >> tasks. 2018-09-12 14:15:10,954+03 INFO >> [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] >> (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread >> pool 'engine' is using 0 threads out of 500, 16 threads waiting for >> tasks and 0 tasks in queue. 2018-09-12 14:15:10,954+03 INFO >> [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] >> (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread >> pool 'engineScheduled' is using 0 threads out of 100, 100 threads >> waiting for tasks. 2018-09-12 14:15:10,954+03 INFO >> [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] >> (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread >> pool 'engineThreadMonitoring' is using 1 threads out of 1, 0 threads >> waiting for tasks. 2018-09-12 14:15:10,954+03 INFO >> [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] >> (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread >> pool 'hostUpdatesChecker' is using 0 threads out of 5, 2 threads >> waiting for tasks. 2018-09-12 14:15:23,843+03 INFO >> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> (EE-ManagedThreadFactory-engineScheduled-Thread-61) [2455041f] Lock >> Acquired to object >> 'EngineLock:{exclusiveLocks='[14e4fb72-9764-4757-b37d-4d487995571a=PROVIDER]', >> sharedLocks=''}' 2018-09-12 14:15:23,849+03 INFO >> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> (EE-ManagedThreadFactory-engineScheduled-Thread-61) [2455041f] >> Running command: SyncNetworkProviderCommand internal: true. >> 2018-09-12 14:15:23,900+03 ERROR >> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> (EE-ManagedThreadFactory-engineScheduled-Thread-61) [2455041f] >> Command >> 'org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand' >> failed: EngineException: (Failed with error Bad Gateway and code >> 5050) 2018-09-12 14:15:23,901+03 INFO >> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> (EE-ManagedThreadFactory-engineScheduled-Thread-61) [2455041f] Lock >> freed to object >> 'EngineLock:{exclusiveLocks='[14e4fb72-9764-4757-b37d-4d487995571a=PROVIDER]', >> sharedLocks=''}' >> >> >> [root@engine ~]# >> cat /etc/ovirt-provider-ovn/conf.d/10-setup-ovirt-provider-ovn.conf # >> This file is automatically generated by engine-setup. Please do not >> edit manually [OVN REMOTE] ovn-remote=ssl:127.0.0.1:6641 >> [SSL] >> https-enabled=true >> ssl-cacert-file=/etc/pki/ovirt-engine/ca.pem >> ssl-cert-file=/etc/pki/ovirt-engine/certs/ovirt-provider-ovn.cer >> ssl-key-file=/etc/pki/ovirt-engine/keys/ovirt-provider-ovn.key.nopass >> [OVIRT] >> ovirt-sso-client-secret=Ms7Gw9qNT6IkXu7oA54tDmxaZDIukABV >> ovirt-host=https://engine.set.local:443 >> ovirt-sso-client-id=ovirt-provider-ovn >> ovirt-ca-file=/etc/pki/ovirt-engine/apache-ca.pem >> [PROVIDER] >> provider-host=engine.set.local >> >> >>> 12 сент. 2018 г., в 13:59, Dominik Holler <dholler(a)redhat.com> >>> написал(а): >>> >>> On Wed, 12 Sep 2018 13:04:53 +0300 >>> "Mail SET Inc. Group" <mail(a)set-pro.net> wrote: >>> >>>> Hello Dominik! >>>> I have a same issue with OVN provider and SSL >>>> https://www.mail-archive.com/users@ovirt.org/msg47020.html >>>> <https://www.mail-archive.com/users@ovirt.org/msg47020.html> But >>>> certificate changes not helps to resolve it. Maybe you can help me >>>> with this? >>> >>> Sure. Can you please share the relevant lines of >>> ovirt-provider-ovn.log and engine.log, and the information if you >>> are using the certificates generated by engine-setup with >>> users(a)ovirt.org ? Thanks, >>> Dominik >>> >> > >

5 13

engine-setup failing on 4.3.2 -> 4.3.3 fails during Engine schema refresh fail
by Edward Berger 18 Jul '23

18 Jul '23

I was trying to upgrade a hyperconverged oVirt hosted engine and failed in the engine-setup command with these error and warnings. ... [ INFO ] Creating/refreshing Engine database schema [ ERROR ] schema.sh: FATAL: Cannot execute sql command: --file=/usr/share/ovirt-engine/dbscripts/upgrade/04_03_0830_add_foreign_key_to_image_transfers.sql [ ERROR ] Failed to execute stage 'Misc configuration': Engine schema refresh failed ... [ INFO ] Yum Verify: 16/16: ovirt-engine-tools.noarch 0:4.3.3.5-1.el7 - e [WARNING] Rollback of DWH database postponed to Stage "Clean up" [ INFO ] Rolling back database schema ... [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination [ ERROR ] Execution of setup failed Attaching engine-setup logfile.

4 6

oVirt host deploy: disable multipath
by femi adegoke 27 Mar '23

27 Mar '23

https://bugzilla.redhat.com/show_bug.cgi?id=1472356 There are 2 drives in my install that oVirt is claiming as multipath. I have tried blacklisting at /etc/multipath/ but that hasn't worked. Any better solutions? I need these 2 drives to be non-multipathed.

4 19

Out-of-sync networks can only be detached
by Sakhi Hadebe 27 Feb '23

27 Feb '23

Hi, I have a 3-node oVirt cluster. I have configured 2 logical networks: ovirtmgmt and public. Public logical network is attached in only 2 nodes and failing to attach on the 3rd node with the below error Invalid operation, out-of-sync network 'public' can only be detached. Please have been stuck on this for almost the whole day now. How do I fix this error? -- Regards, Sakhi Hadebe

3 5

Network Address Change
by Paul.LKW 28 Dec '22

28 Dec '22

Hi All: I just has a case, I need to change the oVirt host and engine IP address due to data center decommission I checked in the hosted-engine host there are some files I could change ; in ovirt-hosted-engine/hosted-engine.conf ca_subject="O=simple.com, CN=1.2.3.4" gateway=1.2.3.254 and of course I need to change the ovirtmgmt interface IP too, I think just change the above line could do the tick, but where could I change the other host IP in the cluster ? I think I have to be lost all the host as once changed the hosted-engine host IP as it is in diff. sub net. Does there any command line tools could do that or someone has such experience could share? Best Regards, Paul.LKW

4 6

looking for some ISV backup software which integrates the backup API
by Nathanaël Blanchet 08 Jun '22

08 Jun '22

Hello, We are about to change our backup provider, and I find it is a great chance to choose a full supported ovirt backup solution. I currently use this python script vm-backup-scheduler (https://github.com/wefixit-AT/oVirtBackup) but it is not the workflow officially suggested by the community (https://www.ovirt.org/develop/release-management/features/storage/backup-re…) I've been looking for a long time an ISV who supports such an API, but the only one I found is this one : Acronis Backup Advanced suggested here https://access.redhat.com/ecosystem/search/#/ecosystem/Red%20Hat%20Enterpri… I ran the trial version, but it doesn't seem to do better than the vm-backup-scheduler script, and it doesn't seem to use the backup API (attach a clone as a disk to an existing vm). Can you suggest me some other ISV solutions, if they ever exist... or share me your backup experience?

5 4

OVS switch type for hosted-engine
by Devin A. Bougie 06 May '22

06 May '22

Is it possible to setup a hosted engine using the OVS switch type instead of Legacy? If it's not possible to start out as OVS, instructions for switching from Legacy to OVS after the fact would be greatly appreciated. Many thanks, Devin

4 5

USB3 redirection
by Rik Theys 19 Apr '22

19 Apr '22

Hi, I'm trying to assign a USB3 controller to a CentOS 7.4 VM in oVirt 4.1 with USB redirection enabled. I've created the following file in /etc/ovirt-engine/osinfo.conf.d: 01-usb.properties with content os.other.devices.usb.controller.value = nec-xhci and have restarted ovirt-engine. If I disable USB-support in the web interface for the VM, the xhci controller is added to the VM (I can see it in the qemu-kvm commandline), but usb redirection is not available. If I enable USB-support in the UI, no xhci controller is added (only 4 uhci controllers). Is there a way to make the controllers for usb redirection xhci controllers? Regards, Rik -- Rik Theys System Engineer KU Leuven - Dept. Elektrotechniek (ESAT) Kasteelpark Arenberg 10 bus 2440 - B-3001 Leuven-Heverlee +32(0)16/32.11.07 ---------------------------------------------------------------- <<Any errors in spelling, tact or fact are transmission errors>>

3 5

OVN routing and firewalling in oVirt
by Gianluca Cecchi 19 Apr '22

19 Apr '22

Hello, how do we manage routing between different OVN networks in oVirt? And between OVN networks and physical ones? Based on architecture read here: http://openvswitch.org/support/dist-docs/ovn-architecture.7.html I see terms for logical routers and gateway routers respectively but how to apply to oVirt configuration? Do I have to choose between setting up a specialized VM or a physical one: is it applicable/advisable to put on oVirt host itself the gateway functionality? Is there any security policy (like security groups in Openstack) to implement? Thanks, Gianluca

4 6

Install hosted-engine - Task Get local VM IP failed
by florentl 10 Mar '22

10 Mar '22

Hi all, I try to install hosted-engine on node : ovirt-node-ng-4.2.3-0.20180518. Every times I get stuck on : [ ERROR ] fatal: [localhost]: FAILED! => {"attempts": 50, "changed": true, "cmd": "virsh -r net-dhcp-leases default | grep -i 00:16:3e:6c:5a:91 | awk '{ print $5 }' | cut -f1 -d'/'", "delta": "0:00:00.108872", "end": "2018-06-01 11:17:34.421769", "rc": 0, "start": "2018-06-01 11:17:34.312897", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} I tried with static IP Address and with DHCP but both failed. To be more specific, I installed three nodes, deployed glusterfs with the wizard. I'm in a nested virtualization environment for this lab (Vmware Esxi Hypervisor). My node IP is : 192.168.176.40 / and I want the hosted-engine vm has 192.168.176.43. Thanks, Florent

7 17

Host needs to be reinstalled after configuring power management
by Andrew DeMaria 02 Feb '22

02 Feb '22

Hi, I am running ovirt 4.3 and have found the following action item immediately after configuring power management for a host: Host needs to be reinstalled as important configuration changes were applied on it. The thing is - I've just freshly installed this host and it seems strange that I need to reinstall it. Is there a better way to install a host and configure power management without having to reinstall it after? Thanks, Andrew

4 3

Lots of storage.MailBox.SpmMailMonitor
by Fabrice Bacchella 06 Jan '22

06 Jan '22

My vdsm log files are huge: -rw-r--r-- 1 vdsm kvm 1.8G Nov 22 11:32 vdsm.log And this is juste half an hour of logs: $ head -1 vdsm.log 2018-11-22 11:01:12,132+0100 ERROR (mailbox-spm) [storage.MailBox.SpmMailMonitor] mailbox 2 checksum failed, not clearing mailbox, clearing new mail (data='...lots of data', expected='\xa4\x06\x08\x00') (mailbox:612) I just upgraded vdsm: $ rpm -qi vdsm Name : vdsm Version : 4.20.43

3 5

Libgfapi considerations
by Jayme 01 Jul '21

01 Jul '21

Are there currently any known issues with using libgfapi in the latest stable version of ovirt in hci deployments? I have recently enabled it and have noticed a significant (over 4x) increase in io performance on my vms. I’m concerned however since it does not seem to be an ovirt default setting. Is libgfapi considered safe and stable to use in ovirt 4.3 hci?

9 16

poweroff and reboot with ovirt_vm ansible module
by Nathanaël Blanchet 18 May '21

18 May '21

Hello, is there a way to poweroff or reboot (without stopped and running state) a vm with the ovirt_vm ansible module? -- Nathanaël Blanchet Supervision réseau Pôle Infrastrutures Informatiques 227 avenue Professeur-Jean-Louis-Viala 34193 MONTPELLIER CEDEX 5 Tél. 33 (0)4 67 54 84 55 Fax 33 (0)4 67 54 84 14 blanchet(a)abes.fr

7 11

OVN and change of mgmt network
by Gianluca Cecchi 26 Jan '21

26 Jan '21

Hello, I previously had OVN running on engine (as OVN provider with northd and northbound and southbound DBs) and hosts (with OVN controller). After changing mgmt ip of hosts (engine has retained instead the same ip), I executed again on them the command: vdsm-tool ovn-config <ip_of_engine> <nel_local_ip_of_host> Now I think I have to clean up some things, eg: 1) On engine where I get these lines below systemctl status ovn-northd.service -l . . . Sep 29 14:41:42 ovmgr1 ovsdb-server[940]: ovs|00005|reconnect|ERR|tcp: 10.4.167.40:37272: no response to inactivity probe after 5 seconds, disconnecting Oct 03 11:52:00 ovmgr1 ovsdb-server[940]: ovs|00006|reconnect|ERR|tcp: 10.4.167.41:52078: no response to inactivity probe after 5 seconds, disconnecting The two IPs are the old ones of two hosts It seems that a restart of the services has fixed... Can anyone confirm if I have to do anything else? 2) On hosts (there are 3 hosts with OVN on ip 10.4.192.32/33/34) where I currently have this output [root@ov301 ~]# ovs-vsctl show 3a38c5bb-0abf-493d-a2e6-345af8aedfe3 Bridge br-int fail_mode: secure Port "ovn-1dce5b-0" Interface "ovn-1dce5b-0" type: geneve options: {csum="true", key=flow, remote_ip="10.4.192.32"} Port "ovn-ddecf0-0" Interface "ovn-ddecf0-0" type: geneve options: {csum="true", key=flow, remote_ip="10.4.192.33"} Port "ovn-fd413b-0" Interface "ovn-fd413b-0" type: geneve options: {csum="true", key=flow, remote_ip="10.4.168.74"} Port br-int Interface br-int type: internal ovs_version: "2.7.2" [root@ov301 ~]# The IPs of kind 10.4.192.x are ok. But there is a left-over of an old host I initially used for tests, corresponding to 10.4.168.74, that now doesn't exist anymore How can I clean records for 1) and 2)? Thanks, Gianluca

3 4

encrypted GENEVE traffic
by Pavel Nakonechnyi 18 Jan '21

18 Jan '21

Dear oVirt Community, From my understanding oVirt does not support Open vSwitch IPSEC tunneling for GENEVE traffic (which is described on pages http://docs.openvswitch.org/en/latest/howto/ipsec/ and http://docs.openvswitch.org/en/latest/tutorials/ipsec/) Are there plans to introduce such support? (or explicitly not to..) Is it possible to somehow manually configure such tunneling for existing virtual networks? (even in a limited way) Alternatively, is it possible to deploy oVirt on top of the tunneled (i.e. via VXLAN, IPSec) interfaces? This will allow to encrypt all management traffic. Such requirement arises when using oVirt deployment on third-party premises with untrusted network. Thank in advance for any clarifications. :) -- WBR, Pavel +32478910884

2 7

Can't import some VMs after storage domain detach and reattach to new datacenter.
by m black 10 Dec '20

10 Dec '20

7 14

"gluster-ansible-roles is not installed on Host" error on Cockpit
by Hesham Ahmed 26 Nov '20

26 Nov '20

On a new 4.3.1 oVirt Node installation, when trying to deploy HCI (also when trying adding a new gluster volume to existing clusters) using Cockpit, an error is displayed "gluster-ansible-roles is not installed on Host. To continue deployment, please install gluster-ansible-roles on Host and try again". There is no package named gluster-ansible-roles in the repositories: [root@localhost ~]# yum install gluster-ansible-roles Loaded plugins: enabled_repos_upload, fastestmirror, imgbased-persist, package_upload, product-id, search-disabled-repos, subscription-manager, vdsmupgrade This system is not registered with an entitlement server. You can use subscription-manager to register. Loading mirror speeds from cached hostfile * ovirt-4.3-epel: mirror.horizon.vn No package gluster-ansible-roles available. Error: Nothing to do Uploading Enabled Repositories Report Cannot upload enabled repos report, is this client registered? This is due to check introduced here: https://gerrit.ovirt.org/#/c/98023/1/dashboard/src/helpers/AnsibleUtil.js Changing the line from: [ "rpm", "-qa", "gluster-ansible-roles" ], { "superuser":"require" } to [ "rpm", "-qa", "gluster-ansible" ], { "superuser":"require" } resolves the issue. The above code snippet is installed at /usr/share/cockpit/ovirt-dashboard/app.js on oVirt node and can be patched by running "sed -i 's/gluster-ansible-roles/gluster-ansible/g' /usr/share/cockpit/ovirt-dashboard/app.js && systemctl restart cockpit"

2 1

Error exporting into ova
by Gianluca Cecchi 31 Aug '20

31 Aug '20

Hello, I'm playing with export_vm_as_ova.py downloaded from the examples github: https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/export_v… My environment is oVirt 4.3.3.7 with iSCSI storage domain. It fails leaving an ova.tmp file In webadmin gui: Starting to export Vm enginecopy1 as a Virtual Appliance 7/19/1911:55:12 AM VDSM ov301 command TeardownImageVDS failed: Cannot deactivate Logical Volume: ('General Storage Exception: ("5 [] [\' Logical volume fa33df49-b09d-4f86-9719-ede649542c21/0420ef47-0ad0-4cf9-babd-d89383f7536b in use.\']\\nfa33df49-b09d-4f86-9719-ede649542c21/[\'a7480dc5-b5ca-4cb3-986d-77bc12165be4\', \'0420ef47-0ad0-4cf9-babd-d89383f7536b\']",)',) 7/19/1912:25:36 PM Failed to export Vm enginecopy1 as a Virtual Appliance to path /save_ova/base/dump/myvm2.ova on Host ov301 7/19/1912:25:37 PM During export I have this qemu-img process creating the disk over the loop device: root 30878 30871 0 11:55 pts/2 00:00:00 su -p -c qemu-img convert -T none -O qcow2 '/rhev/data-center/mnt/blockSD/fa33df49-b09d-4f86-9719-ede649542c21/images/59a4a324-4c99-4ff5-abb1-e9bbac83292a/0420ef47-0ad0-4cf9-babd-d89383f7536b' '/dev/loop1' vdsm vdsm 30882 30878 10 11:55 ? 00:00:00 qemu-img convert -T none -O qcow2 /rhev/data-center/mnt/blockSD/fa33df49-b09d-4f86-9719-ede649542c21/images/59a4a324-4c99-4ff5-abb1-e9bbac83292a/0420ef47-0ad0-4cf9-babd-d89383f7536b /dev/loop1 The ova.tmp file is getting filled while command runs eg: [root@ov301 ]# du -sh /save_ova/base/dump/myvm2.ova.tmp 416M /save_ova/base/dump/myvm2.ova.tmp [root@ov301 sysctl.d]# [root@ov301 sysctl.d]# du -sh /save_ova/base/dump/myvm2.ova.tmp 911M /save_ova/base/dump/myvm2.ova.tmp [root@ov301 ]# and the final generated / not completed file is in this state: [root@ov301 ]# qemu-img info /save_ova/base/dump/myvm2.ova.tmp image: /save_ova/base/dump/myvm2.ova.tmp file format: raw virtual size: 30G (32217446400 bytes) disk size: 30G [root@ov301 sysctl.d]# But I notice that the timestamp of the file is about 67 minutes after start of job and well after the notice of its failure.... [root@ov301 sysctl.d]# ll /save_ova/base/dump/ total 30963632 -rw-------. 1 root root 32217446400 Jul 19 13:02 myvm2.ova.tmp [root@ov301 sysctl.d]# [root@ov301 sysctl.d]# du -sh /save_ova/base/dump/myvm2.ova.tmp 30G /save_ova/base/dump/myvm2.ova.tmp [root@ov301 sysctl.d]# In engine.log the first error I see is 30 minutes after start 2019-07-19 12:25:31,563+02 ERROR [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] Ansible playbook execution failed: Timeout occurred while executing Ansible playbook. 2019-07-19 12:25:31,563+02 INFO [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] Ansible playbook command has exited with value: 1 2019-07-19 12:25:31,564+02 ERROR [org.ovirt.engine.core.bll.CreateOvaCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] Failed to create OVA. Please check logs for more details: /var/log/ovirt-engine/ova/ovirt-export-ova-ansible-20190719115531-ov301-2001ddf4.log 2019-07-19 12:25:31,565+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.TeardownImageVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] START, TeardownImageVDSCommand(HostName = ov301, ImageActionsVDSCommandParameters:{hostId='8ef1ce6f-4e38-486c-b3a4-58235f1f1d06'}), log id: 3d2246f7 2019-07-19 12:25:36,569+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM ov301 command TeardownImageVDS failed: Cannot deactivate Logical Volume: ('General Storage Exception: ("5 [] [\' Logical volume fa33df49-b09d-4f86-9719-ede649542c21/0420ef47-0ad0-4cf9-babd-d89383f7536b in use.\']\\nfa33df49-b09d-4f86-9719-ede649542c21/[\'a7480dc5-b5ca-4cb3-986d-77bc12165be4\', \'0420ef47-0ad0-4cf9-babd-d89383f7536b\']",)',) In ansible playbook suggested log file I don't see anything useful. It ends with timestamps when the script has been launched. Last lines are: 2019-07-19 11:55:33,877 p=5699 u=ovirt | TASK [ovirt-ova-export-pre-pack : Retrieving the temporary path for the OVA file] *** 2019-07-19 11:55:34,198 p=5699 u=ovirt | changed: [ov301] => { "changed": true, "dest": "/save_ova/base/dump/myvm2.ova.tmp", "gid": 0, "group": "root", "mode": "0600", "owner": "root", "secontext": "system_u:object_r:nfs_t:s0", "size": 32217446912, "state": "file", "uid": 0 } 2019-07-19 11:55:34,204 p=5699 u=ovirt | TASK [ovirt-ova-pack : Run packing script] ************************************* It seems 30 minutes... for timeout? About what, ansible job? Or possibly implicit user session created when running the python script? The snapshot has been correctly deleted (as I see also in engine.log), I don't see it in webadmin gui. Any known problem? Just for test I executed again at 14:24 and I see same Ansible error at 14:54 The snapshot gets deleted, while the qemu-img command still continues.... [root@ov301 sysctl.d]# ps -ef | grep qemu-img root 13504 13501 0 14:24 pts/1 00:00:00 su -p -c qemu-img convert -T none -O qcow2 '/rhev/data-center/mnt/blockSD/fa33df49-b09d-4f86-9719-ede649542c21/images/59a4a324-4c99-4ff5-abb1-e9bbac83292a/0420ef47-0ad0-4cf9-babd-d89383f7536b' '/dev/loop0' vdsm vdsm 13505 13504 3 14:24 ? 00:01:26 qemu-img convert -T none -O qcow2 /rhev/data-center/mnt/blockSD/fa33df49-b09d-4f86-9719-ede649542c21/images/59a4a324-4c99-4ff5-abb1-e9bbac83292a/0420ef47-0ad0-4cf9-babd-d89383f7536b /dev/loop0 root 17587 24530 0 15:05 pts/0 00:00:00 grep --color=auto qemu-img [root@ov301 sysctl.d]# [root@ov301 sysctl.d]# du -sh /save_ova/base/dump/myvm2.ova.tmp 24G /save_ova/base/dump/myvm2.ova.tmp [root@ov301 sysctl.d]# ll /save_ova/base/dump/myvm2.ova.tmp -rw-------. 1 root root 32217446400 Jul 19 15:14 /save_ova/base/dump/myvm2.ova.tmp [root@ov301 sysctl.d]# and then continues until image copy completes, but at this time the job has already aborted and so the completion of the ova composition doesn't go ahead... and I remain with the ova.tmp file... How to extend timeout? Thanks in advance, Gianluca

4 11

deprecating export domain?
by Charles Kozler 30 Aug '20

30 Aug '20

Hello, I recently read on this list from a redhat member that export domain is either being deprecated or looking at being deprecated To that end, can you share details? Can you share any notes/postings/bz's that document this? I would imagine something like this would be discussed in larger audience This seems like a somewhat significant change to make and I am curious where this is scheduled? Currently, a lot of my backups rely explicitly on an export domain for online snapshots, so I'd like to plan accordingly Thanks!

11 21

Upgrade ovirt from 3.4 to 4.3
by lu.alfonsi＠almaviva.it 19 Jun '20

19 Jun '20

Good morning, i have a difficult enviroment with 20 Hypervisors based on ovirt 3.4.3-1 and i would like to reach the 4.3 version. Which are the best steps to achieve these objective? Thanks in advance Luigi

3 10

Re: Single instance scaleup.
by Strahil 05 Jun '20

05 Jun '20

Hi Leo, As you do not have a distributed volume , you can easily switch to replica 2 arbiter 1 or replica 3 volumes. You can use the following for adding the bricks: https://access.redhat.com/documentation/en-US/Red_Hat_Storage/2.1/html/Admi… Best Regards, Strahil NikolivOn May 26, 2019 10:54, Leo David <leoalex(a)gmail.com> wrote: > > Hi Stahil, > Thank you so much for yout input ! > > gluster volume info > > > Volume Name: engine > Type: Distribute > Volume ID: d7449fc2-cc35-4f80-a776-68e4a3dbd7e1 > Status: Started > Snapshot Count: 0 > Number of Bricks: 1 > Transport-type: tcp > Bricks: > Brick1: 192.168.80.191:/gluster_bricks/engine/engine > Options Reconfigured: > nfs.disable: on > transport.address-family: inet > storage.owner-uid: 36 > storage.owner-gid: 36 > features.shard: on > performance.low-prio-threads: 32 > performance.strict-o-direct: off > network.remote-dio: off > network.ping-timeout: 30 > user.cifs: off > performance.quick-read: off > performance.read-ahead: off > performance.io-cache: off > cluster.eager-lock: enable > Volume Name: ssd-samsung > Type: Distribute > Volume ID: 76576cc6-220b-4651-952d-99846178a19e > Status: Started > Snapshot Count: 0 > Number of Bricks: 1 > Transport-type: tcp > Bricks: > Brick1: 192.168.80.191:/gluster_bricks/sdc/data > Options Reconfigured: > cluster.eager-lock: enable > performance.io-cache: off > performance.read-ahead: off > performance.quick-read: off > user.cifs: off > network.ping-timeout: 30 > network.remote-dio: off > performance.strict-o-direct: on > performance.low-prio-threads: 32 > features.shard: on > storage.owner-gid: 36 > storage.owner-uid: 36 > transport.address-family: inet > nfs.disable: on > > The other two hosts will be 192.168.80.192/193 - this is gluster dedicated network over 10GB sfp+ switch. > - host 2 wil have identical harware configuration with host 1 ( each disk is actually a raid0 array ) > - host 3 has: > - 1 ssd for OS > - 1 ssd - for adding to engine volume in a full replica 3 > - 2 ssd's in a raid 1 array to be added as arbiter for the data volume ( ssd-samsung ) > So the plan is to have "engine" scaled in a full replica 3, and "ssd-samsung" scalled in a replica 3 arbitrated. > > > > > On Sun, May 26, 2019 at 10:34 AM Strahil <hunter86_bg(a)yahoo.com> wrote: >> >> Hi Leo, >> >> Gluster is quite smart, but in order to provide any hints , can you provide output of 'gluster volume info <glustervol>'. >> If you have 2 more systems , keep in mind that it is best to mirror the storage on the second replica (2 disks on 1 machine -> 2 disks on the new machine), while for the arbiter this is not neccessary. >> >> What is your network and NICs ? Based on my experience , I can recommend at least 10 gbit/s interfase(s). >> >> Best Regards, >> Strahil Nikolov >> >> On May 26, 2019 07:52, Leo David <leoalex(a)gmail.com> wrote: >>> >>> Hello Everyone, >>> Can someone help me to clarify this ? >>> I have a single-node 4.2.8 installation ( only two gluster storage domains - distributed single drive volumes ). Now I just got two identintical servers and I would like to go for a 3 nodes bundle. >>> Is it possible ( after joining the new nodes to the cluster ) to expand the existing volumes across the new nodes and change them to replica 3 arbitrated ? >>> If so, could you share with me what would it be the procedure ? >>> Thank you very much ! >>> >>> Leo > > > > -- > Best regards, Leo David

4 5

Failed to add storage domain
by thunderlight1＠gmail.com 31 May '20

31 May '20

Hi! I have installed oVirt using the iso ovirt-node-ng-installer-4.3.2-2019031908.el7. I the did run the Host-engine deployment through Cockpit. I got an error when it tries to create the domain storage. It sucessfully mounted the NFS-share on the host. Bellow is the error I got: 2019-04-14 10:40:38,967+0200 INFO ansible skipped {'status': 'SKIPPED', 'ansible_task': u'Check storage domain free space', 'ansible_host': u'localhost', 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml', 'ansible_type': 'task'} 2019-04-14 10:40:38,967+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fb6918ad9d0> kwargs 2019-04-14 10:40:39,516+0200 INFO ansible task start {'status': 'OK', 'ansible_task': u'ovirt.hosted_engine_setup : Activate storage domain', 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml', 'ansible_type': 'task'} 2019-04-14 10:40:39,516+0200 DEBUG ansible on_any args TASK: ovirt.hosted_engine_setup : Activate storage domain kwargs is_conditional:False 2019-04-14 10:40:41,923+0200 DEBUG var changed: host "localhost" var "otopi_storage_domain_details" type "<type 'dict'>" value: "{ "changed": false, "exception": "Traceback (most recent call last):\n File \"/tmp/ansible_ovirt_storage_domain_payload_xSFxOp/__main__.py\", line 664, in main\n storage_domains_module.post_create_check(sd_id)\n File \"/tmp/ansible_ovirt_storage_domain_payload_xSFxOp/__main__.py\", line 526, in post_create_check\n id=storage_domain.id,\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py\", line 3053, in add\n return self._internal_add(storage_domain, headers, query, wait)\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py\", line 232, in _internal_add\n return future.wait() if wait else future\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py\", line 55, in wait\n return self._code(response)\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py\", line 229, in callback\n self._check_fault(response)\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py\", line 132, in _check_fault\n self._raise_error(response , body)\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py\", line 118, in _raise_error\n raise error\nError: Fault reason is \"Operation Failed\". Fault detail is \"[]\". HTTP response code is 400.\n", "failed": true, "msg": "Fault reason is \"Operation Failed\". Fault detail is \"[]\". HTTP response code is 400." }" 2019-04-14 10:40:41,924+0200 DEBUG var changed: host "localhost" var "ansible_play_hosts" type "<type 'list'>" value: "[]" 2019-04-14 10:40:41,924+0200 DEBUG var changed: host "localhost" var "play_hosts" type "<type 'list'>" value: "[]" 2019-04-14 10:40:41,924+0200 DEBUG var changed: host "localhost" var "ansible_play_batch" type "<type 'list'>" value: "[]" 2019-04-14 10:40:41,924+0200 ERROR ansible failed {'status': 'FAILED', 'ansible_type': 'task', 'ansible_task': u'Activate storage domain', 'ansible_result': u'type: <type \'dict\'>\nstr: {\'_ansible_parsed\': True, u\'exception\': u\'Traceback (most recent call last):\\n File "/tmp/ansible_ovirt_storage_domain_payload_xSFxOp/__main__.py", line 664, in main\\n storage_domains_module.post_create_check(sd_id)\\n File "/tmp/ansible_ovirt_storage_domain_payload_xSFxOp/__main__.py", line 526', 'task_duration': 2, 'ansible_host': u'localhost', 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml'} 2019-04-14 10:40:41,924+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fb691843190> kwargs ignore_errors:None 2019-04-14 10:40:41,928+0200 INFO ansible stats { "ansible_playbook": "/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml", "ansible_playbook_duration": "00:37 Minutes", "ansible_result": "type: <type 'dict'>\nstr: {u'localhost': {'unreachable': 0, 'skipped': 6, 'ok': 23, 'changed': 1, 'failures': 1}}", "ansible_type": "finish", "status": "FAILED" } 2019-04-14 10:40:41,928+0200 INFO SUMMARY: Duration Task Name -------- -------- [ < 1 sec ] Execute just a specific set of steps [ 00:01 ] Force facts gathering [ 00:01 ] Check local VM dir stat [ 00:01 ] Obtain SSO token using username/password credentials [ 00:01 ] Fetch host facts [ < 1 sec ] Fetch cluster ID [ 00:01 ] Fetch cluster facts [ 00:01 ] Fetch Datacenter facts [ < 1 sec ] Fetch Datacenter ID [ < 1 sec ] Fetch Datacenter name [ 00:02 ] Add NFS storage domain [ 00:01 ] Get storage domain details [ 00:01 ] Find the appliance OVF [ 00:01 ] Parse OVF [ < 1 sec ] Get required size [ FAILED ] Activate storage domain 2019-04-14 10:40:41,928+0200 DEBUG ansible on_any args <ansible.executor.stats.AggregateStats object at 0x7fb69404eb90> kwargs Any suggestions on how fix this?

2 2

How to connect to a guest with vGPU ?
by Josep Manel Andrés Moscardó 29 May '20

29 May '20

Hi, I got vGPU through mdev working but I am wondering how I would connect to the client and make use of the GPU. So far I try to access the console through SPICE and at some point in the boot process it switches to GPU and I cannot see anything else. Thanks. -- Josep Manel Andrés Moscardó Systems Engineer, IT Operations EMBL Heidelberg T +49 6221 387-8394

3 4

Ubuntu 18.04 and 16.04 cloud images hang at boot up
by suaro＠live.com 07 May '20

07 May '20

I'm using oVirt 4.3 (latest ) and able to successfully provision Centos VMs without any problems. When I attempt to provision Ubuntu VMs, they hang at startup. The console shows : ... ... [ 4.010016] Btrfs loaded [ 101.268594] random: nonblocking pool is initialized It stays like this indefinitely. Again, I have no problems with Centos images, but need Ubuntu Any tips greatly appreciated.

3 2

ovirt-guest-agent for CentOS 8
by Eduardo Mayoral 30 Mar '20

30 Mar '20

Hi, Just like many of you I am testing my first CentOS 8 VMs on top of ovirt. I am not finding the package ovirt-guest-agent.noarch . Closest I can find is qemu-guest-agent.x86_64 After installing and starting it, I do see information reported on the "Guest info" tab. Can anybody confirm if this is indeed the agent we should be using? Is there - or will there be - a more specific package for ovirt guests? Thanks! -- Eduardo Mayoral Jimeno Systems engineer, platform department. Arsys Internet. emayoral(a)arsys.es - +34 941 620 105 - ext 2153

6 7

Re: Hyperconverged setup questions
by Strahil 29 Feb '20

29 Feb '20

Hi Marko, I guess you can use distributed-replicated volumes and oVirt cluster with host triplets. Best Regards, Strahil NikolovOn Oct 10, 2019 15:30, "Vrgotic, Marko" <M.Vrgotic(a)activevideo.com> wrote: > > Dear oVirt, > > > > Is it possible to add oVirt 3Hosts/Gluster hyperconverged cluster to existing oVirt setup? I need this to achieve Local storage performance, but still have pool of Hypevisors available. > > Is it possible to have more than 3Hosts in Hyperconverged setup? > > > > I have currently 1Shared Cluster (NFS based storage, where also SHE is hosted) and 2Local Storage clusters. > > > > oVirt current version running is 4.3.4. > > > > Kindly awaiting your reply. > > > > > > — — — > Met vriendelijke groet / Kind regards, > > Marko Vrgotic > > ActiveVideo > > > >

6 12

Re: Damaged hard disk in volume replica gluster
by Strahil 25 Feb '20

25 Feb '20

As you are replacing an old brick, you have to recreate the old LV and mount it on the same location. Then you can use the gluster's "reset-brick" (i thing ovirt UI has that option too) and all data will be replicated there. You got also the "replace-brick" option if you decided to change the mount location. P.S.: with replica volumes your volume should be still working, in case it has stopped - you have to investigate before proceeding. Best Regards, Strahil NikolovOn Oct 12, 2019 13:12, matteo fedeli <matmilan97(a)gmail.com> wrote: > > Hi, I have in my ovirt HCI a volume that not work properly because one of three hdd fail. I bought a new hdd, I recreate old lvm partitioning and mount point. Now, how can I attach this empty new brick? > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/U5XQ4M2K5RBKL…

5 20

terraform integration
by Nathanaël Blanchet 25 Feb '20

25 Feb '20

Hello, I'm interested by installing okd on ovirt with the official openshift installer (https://github.com/openshift/installer) but ovirt is not yet supported. Regarding https://bugzilla.redhat.com/show_bug.cgi?id=1578255 and https://lists.ovirt.org/archives/list/users@ovirt.org/thread/EF7OQUVTY53GV3… , how ovirt 4.3.7 should integrate openshift installer integration with terraform? -- Nathanaël Blanchet Supervision réseau Pôle Infrastrutures Informatiques 227 avenue Professeur-Jean-Louis-Viala 34193 MONTPELLIER CEDEX 5 Tél. 33 (0)4 67 54 84 55 Fax 33 (0)4 67 54 84 14 blanchet(a)abes.fr

4 36

Vm suddenly paused with error "vm has paused due to unknown storage error"
by Jasper Siero 18 Feb '20

18 Feb '20

Hi all, Since we upgraded our Ovirt nodes to CentOS 7 a vm (not a specific one but never more then one) will sometimes pause suddenly with the error "VM ... has paused due to unknown storage error". It happens now two times in a month. The Ovirt node uses san storage for the vm's running on it. When a specific vm is pausing with an error the other vm's keeps running without problems. The vm runs without problems after unpausing it. Versions: CentOS Linux release 7.1.1503 vdsm-4.14.17-0 libvirt-daemon-1.2.8-16 vdsm.log: VM Channels Listener::DEBUG::2015-10-25 07:43:54,382::vmChannels::95::vds::(_handle_timeouts) Timeout on fileno 78. libvirtEventLoop::INFO::2015-10-25 07:43:56,177::vm::4602::vm.Vm::(_onIOError) vmId=`77f07ae0-cc3e-4ae2-90ec-7fba7b11deeb`::abnormal vm stop device virtio-disk0 error eother libvirtEventLoop::DEBUG::2015-10-25 07:43:56,178::vm::5204::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`77f07ae0-cc3e-4ae2-90ec-7fba7b11deeb`::event Suspended detail 2 opaque None libvirtEventLoop::INFO::2015-10-25 07:43:56,178::vm::4602::vm.Vm::(_onIOError) vmId=`77f07ae0-cc3e-4ae2-90ec-7fba7b11deeb`::abnormal vm stop device virtio-disk0 error eother ........... libvirtEventLoop::INFO::2015-10-25 07:43:56,180::vm::4602::vm.Vm::(_onIOError) vmId=`77f07ae0-cc3e-4ae2-90ec-7fba7b11deeb`::abnormal vm stop device virtio-disk0 error eother specific error part in libvirt vm log: block I/O error in device 'drive-virtio-disk0': Unknown error 32758 (32758) ........... block I/O error in device 'drive-virtio-disk0': Unknown error 32758 (32758) engine.log: 2015-10-25 07:44:48,945 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-40) [a43dcc8] VM diataal-prod-cas1 77f07ae0-cc3e-4ae2-90ec-7fba7b11deeb moved from Up --> Paused 2015-10-25 07:44:49,003 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-40) [a43dcc8] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM diataal-prod-cas1 has paused due to unknown storage error. Has anyone experienced the same problem or knows a way to solve this? Kind regards, Jasper

3 3

Re: Cannot Increase Hosted Engine VM Memory
by Serhiy Morhun 13 Feb '20

13 Feb '20

Hello, did anyone find a resolution for this issue? I'm having exactly the same problem: Hosted Engine VM is running with 5344MB of RAM, when trying to increase to 8192 it would not accept the change because the difference is not divisible by 256. When trying to increase to 8160 the change is accepted but log shows "Hotset memory: changed the amount of memory on VM HostedEngine from 5344 to 5344" at the same time amount of guaranteed memory does increase to 8160 which, in turn, starts generating error messages that VM does not have all the guaranteed RAM. Serhiy Morhun -- *-----------------------------------------------------------------------------------*** *THE INFORMATION CONTAINED IN THIS MESSAGE (E-MAIL AND ANY ATTACHMENTS) IS INTENDED ONLY FOR THE INDIVIDUAL AND CONFIDENTIAL USE OF THE DESIGNATED RECIPIENT(S).* If any reader of this message is not an intended recipient or any agent responsible for delivering it to an intended recipient, you are hereby notified that you have received this document in error, and that any review, dissemination, distribution, copying or other use of this message is prohibited. If you have received this message in error, please notify us immediately by reply e-mail message or by telephone and delete the original message from your e-mail system and/or computer database. Thank you. *-----------------------------------------------------------------------------------* **NOTICE**: *You are advised that e-mail correspondence and attachments between the public and the Ridgewood Board of Education are obtainable by any person who files a request under the NJ Open Public Records Act (OPRA) unless it is subject to a specific OPRA exception. You should have no expectation that the content of e-mails sent to or from school district e-mail addresses, or between the public and school district officials and employees, will remain private.* *-----------------------------------------------------------------------------------*

3 3

Ovirt-engine-ha cannot to see live status of Hosted Engine
by asm＠pioner.kz 01 Feb '20

01 Feb '20

Good day for all. I have some issues with Ovirt 4.2.6. But now the main this of it: I have two Centos 7 Nodes with same config and last Ovirt 4.2.6 with Hostedengine with disk on NFS storage. Also some of virtual machines working good. But, when HostedEngine running on one node (srv02.local) everything is fine. After migrating to another node (srv00.local), i see that agent cannot to check livelinness of HostedEngine. After few minutes HostedEngine going to reboot and after some time i see some situation. After migration to another node (srv00.local) all looks OK. hosted-engine --vm-status commang when HosterEngine on srv00 node: --== Host 1 status ==-- conf_on_shared_storage : True Status up-to-date : True Hostname : srv02.local Host ID : 1 Engine status : {"reason": "vm not running on this host", "health": "bad", "vm": "down_unexpected", "detail": "unknown"} Score : 0 stopped : False Local maintenance : False crc32 : ecc7ad2d local_conf_timestamp : 78328 Host timestamp : 78328 Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=78328 (Tue Sep 18 12:44:18 2018) host-id=1 score=0 vm_conf_refresh_time=78328 (Tue Sep 18 12:44:18 2018) conf_on_shared_storage=True maintenance=False state=EngineUnexpectedlyDown stopped=False timeout=Fri Jan 2 03:49:58 1970 --== Host 2 status ==-- conf_on_shared_storage : True Status up-to-date : True Hostname : srv00.local Host ID : 2 Engine status : {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"} Score : 3400 stopped : False Local maintenance : False crc32 : 1d62b106 local_conf_timestamp : 326288 Host timestamp : 326288 Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=326288 (Tue Sep 18 12:44:21 2018) host-id=2 score=3400 vm_conf_refresh_time=326288 (Tue Sep 18 12:44:21 2018) conf_on_shared_storage=True maintenance=False state=EngineStarting stopped=False Log agent.log from srv00.local: MainThread::INFO::2018-09-18 12:40:51,749::states::779::ovirt_hosted_engine_ha.agent.hosted_engine.HostedE ngine::(consume) VM is powering up.. MainThread::INFO::2018-09-18 12:40:52,052::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine. HostedEngine::(_monitoring_loop) Current state EngineStarting (score: 3400) MainThread::INFO::2018-09-18 12:41:01,066::states::779::ovirt_hosted_engine_ha.agent.hosted_engine.HostedE ngine::(consume) VM is powering up.. MainThread::INFO::2018-09-18 12:41:01,374::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine. HostedEngine::(_monitoring_loop) Current state EngineStarting (score: 3400) MainThread::INFO::2018-09-18 12:41:11,393::state_machine::169::ovirt_hosted_engine_ha.agent.hosted_engine. HostedEngine::(refresh) Global metadata: {'maintenance': False} MainThread::INFO::2018-09-18 12:41:11,393::state_machine::174::ovirt_hosted_engine_ha.agent.hosted_engine. HostedEngine::(refresh) Host srv02.local.pioner.kz (id 1): {'conf_on_shared_storage': True, 'extra': 'meta data_parse_version=1\nmetadata_feature_version=1\ntimestamp=78128 (Tue Sep 18 12:40:58 2018)\nhost-id=1\ns core=0\nvm_conf_refresh_time=78128 (Tue Sep 18 12:40:58 2018)\nconf_on_shared_storage=True\nmaintenance=Fa lse\nstate=EngineUnexpectedlyDown\nstopped=False\ntimeout=Fri Jan 2 03:49:58 1970\n', 'hostname': 'srv02. local.pioner.kz', 'alive': True, 'host-id': 1, 'engine-status': {'reason': 'vm not running on this host', 'health': 'bad', 'vm': 'down_unexpected', 'detail': 'unknown'}, 'score': 0, 'stopped': False, 'maintenance ': False, 'crc32': 'e18e3f22', 'local_conf_timestamp': 78128, 'host-ts': 78128} MainThread::INFO::2018-09-18 12:41:11,393::state_machine::177::ovirt_hosted_engine_ha.agent.hosted_engine. HostedEngine::(refresh) Local (id 2): {'engine-health': {'reason': 'failed liveliness check', 'health': 'b ad', 'vm': 'up', 'detail': 'Up'}, 'bridge': True, 'mem-free': 12763.0, 'maintenance': False, 'cpu-load': 0 .0364, 'gateway': 1.0, 'storage-domain': True} MainThread::INFO::2018-09-18 12:41:11,393::states::779::ovirt_hosted_engine_ha.agent.hosted_engine.HostedE ngine::(consume) VM is powering up.. MainThread::INFO::2018-09-18 12:41:11,703::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine. HostedEngine::(_monitoring_loop) Current state EngineStarting (score: 3400) MainThread::INFO::2018-09-18 12:41:21,716::states::779::ovirt_hosted_engine_ha.agent.hosted_engine.HostedE ngine::(consume) VM is powering up.. MainThread::INFO::2018-09-18 12:41:22,020::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine. HostedEngine::(_monitoring_loop) Current state EngineStarting (score: 3400) MainThread::INFO::2018-09-18 12:41:31,033::states::779::ovirt_hosted_engine_ha.agent.hosted_engine.HostedE ngine::(consume) VM is powering up.. MainThread::INFO::2018-09-18 12:41:31,344::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine. HostedEngine::(_monitoring_loop) Current state EngineStarting (score: 3400) As we can see, agent thinking that HostedEngine just in powering up mode. I cannot to do anythink with it. I allready reinstalled many times srv00 node without success. One time i even has to uninstall ovirt* and vdsm* software. Also here one interesting point, after installing just "yum install http://resources.ovirt.org/pub/yum-repo/ovirt-release42.rpm" on this node i try to install this node from engine web interface with "Deploy" action. But, installation was unsuccesfull, before i didnt install ovirt-hosted-engine-ha on this node. I dont see in documentation that its need bofore installation of new hosts. But this is for information and checking. After installing ovirt-hosted-engine-ha node was installed with HostedEngine support. But the main issue not changed. Thanks in advance for help. BR, Alexandr

3 5

Re: [ANN] oVirt 4.3.7 Third Release Candidate is now available for testing
by Strahil 31 Jan '20

31 Jan '20

I got upgraded to RC3 and now cannot power any VM . Constantly getting I/O error, but checking at gluster level - I can dd from each disk or even create a new one. Removing the HighAvailability doesn't help. I guess I should restore the engine from the gluster snapshot and rollback via 'yum history undo last'. Does anyone else have my issues ? Best Regards, Strahil NikolovOn Nov 13, 2019 15:31, Sandro Bonazzola <sbonazzo(a)redhat.com> wrote: > > > > Il giorno mer 13 nov 2019 alle ore 14:25 Sandro Bonazzola <sbonazzo(a)redhat.com> ha scritto: >> >> >> >> Il giorno mer 13 nov 2019 alle ore 13:56 Florian Schmid <fschmid(a)ubimet.com> ha scritto: >>> >>> Hello, >>> >>> I have a question about bugs, which are flagged as [downstream clone - 4.3.7], but are not yet released. >>> >>> I'm talking about this bug: >>> https://bugzilla.redhat.com/show_bug.cgi?id=1749202 >>> >>> I can't see it in 4.3.7 release notes. Will it be included in a further release candidate? This fix is very important I think and I can't upgrade yet because of this bug. >> >> >> >> Looking at the bug, the fix was done with $ git tag --contains 12bd5cb1fe7c95e29b4065fca968913722fe9eaa >> ovirt-engine-4.3.6.6 >> ovirt-engine-4.3.6.7 >> ovirt-engine-4.3.7.0 >> ovirt-engine-4.3.7.1 >> >> So the fix is already included in release oVirt 4.3.6. > > > Sent a fix to 4.3.6 release notes: https://github.com/oVirt/ovirt-site/pull/2143. @Ryan Barry can you please review? > > >> >> >> >> >> >> >>> >>> >>> BR Florian Schmid >>> >>> ________________________________ >>> Von: "Sandro Bonazzola" <sbonazzo(a)redhat.com> >>> An: "users" <users(a)ovirt.org> >>> Gesendet: Mittwoch, 13. November 2019 13:34:59 >>> Betreff: [ovirt-users] [ANN] oVirt 4.3.7 Third Release Candidate is now available for testing >>> >>> The oVirt Project is pleased to announce the availability of the oVirt 4.3.7 Third Release Candidate for testing, as of November 13th, 2019. >>> >>> This update is a release candidate of the seventh in a series of stabilization updates to the 4.3 series. >>> This is pre-release software. This pre-release should not to be used in production. >>> >>> This release is available now on x86_64 architecture for: >>> * Red Hat Enterprise Linux 7.7 or later (but <8) >>> * CentOS Linux (or similar) 7.7 or later (but <8) >>> >>> This release supports Hypervisor Hosts on x86_64 and ppc64le architectures for: >>> * Red Hat Enterprise Linux 7.7 or later (but <8) >>> * CentOS Linux (or similar) 7.7 or later (but <8) >>> * oVirt Node 4.3 (available for x86_64 only) has been built consuming CentOS 7.7 Release >>> >>> See the release notes [1] for known issues, new features and bugs fixed. >>> >>> While testing this release candidate please note that oVirt node now includes: >>> - ansible 2.9.0 >>> - GlusterFS 6.6 >>> >>> Notes: >>> - oVirt Appliance is already available >>> - oVirt Node is already available >>> >>> Additional Resources: >>> * Read more about the oVirt 4.3.7 release highlights: http://www.ovirt.org/release/4.3.7/ >>> * Get more oVirt Project updates on Twitter: https://twitter.com/ovirt >>> * Check out the latest project news on the oVirt blog:http://www.ovirt.org/blog/ >>> >>> [1] http://www.ovirt.org/release/4.3.7/ >>> [2] http://resources.ovirt.org/pub/ovirt-4.3-pre/iso/ >>> >>> -- >>> >>> Sandro Bonazzola >>> >>> MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV >>> >>> Red Hat EMEA >>> >>> sbonazzo(a)redhat.com >>> >>> Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. >>> >>> _______________________________________________ >>> Users mailing list -- users(a)ovirt.org >>> To unsubscribe send an email to users-leave(a)ovirt.org >>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/24QUREJPZHTSM… >> >> >> >> -- >> >> Sandro Bonazzola >> >> MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV >> >> Red Hat EMEA >> >> sbonazzo(a)redhat.com >> >> Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. > > > > -- > > Sandro Bonazzola > > MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV > > Red Hat EMEA > > sbonazzo(a)redhat.com > > Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.

7 20

ovirt upgrade
by David David 24 Jan '20

24 Jan '20

hi all how do upgrade from 4.2.5 to 4.3.7 ? what steps are necessary, are the same as when upgrading from 4.1 to 4.2? # engine-upgrade-check # yum update ovirt*setup* # engine-setup thanks

3 3

Low disk space on Storage
by suporte＠logicworks.pt 12 Jan '20

12 Jan '20

Hi, I'm running ovirt Version:4.3.4.3-1.el7 My filesystem disk has 30 GB free space. Cannot start a VM due to an I/O error storage. When tryng to move the disk to another storage domain get this error: Error while executing action: Cannot move Virtual Disk. Low disk space on Storage Domain DATA4. The sum of pre-allocated disk is the total of the storage domain disk. Any idea what can I do to move a disk to other storage domain? Many thanks -- Jose Ferradeira http://www.logicworks.pt

5 5

Hyperconverged setup - storage architecture - scaling
by Leo David 10 Jan '20

10 Jan '20

Hello Everyone, Reading through the document: "Red Hat Hyperconverged Infrastructure for Virtualization 1.5 Automating RHHI for Virtualization deployment" Regarding storage scaling, i see the following statements: *2.7. SCALINGRed Hat Hyperconverged Infrastructure for Virtualization is supported for one node, and for clusters of 3, 6, 9, and 12 nodes.The initial deployment is either 1 or 3 nodes.There are two supported methods of horizontally scaling Red Hat Hyperconverged Infrastructure for Virtualization:* *1 Add new hyperconverged nodes to the cluster, in sets of three, up to the maximum of 12 hyperconverged nodes.* *2 Create new Gluster volumes using new disks on existing hyperconverged nodes.You cannot create a volume that spans more than 3 nodes, or expand an existing volume so that it spans across more than 3 nodes at a time* *2.9.1. Prerequisites for geo-replicationBe aware of the following requirements and limitations when configuring geo-replication:One geo-replicated volume onlyRed Hat Hyperconverged Infrastructure for Virtualization (RHHI for Virtualization) supports only one geo-replicated volume. Red Hat recommends backing up the volume that stores the data of your virtual machines, as this is usually contains the most valuable data.* ------ Also in oVirtEngine UI, when I add a brick to an existing volume i get the following warning: *"Expanding gluster volume in a hyper-converged setup is not recommended as it could lead to degraded performance. To expand storage for cluster, it is advised to add additional gluster volumes." * Those things are raising a couple of questions that maybe for some for you guys are easy to answer, but for me it creates a bit of confusion... I am also referring to RedHat product documentation, because I treat oVirt as production-ready as RHHI is. *1*. Is there any reason for not going to distributed-replicated volumes ( ie: spread one volume across 6,9, or 12 nodes ) ? - ie: is recomanded that in a 9 nodes scenario I should have 3 separated volumes, but how should I deal with the folowing question *2.* If only one geo-replicated volume can be configured, how should I deal with 2nd and 3rd volume replication for disaster recovery *3.* If the limit of hosts per datacenter is 250, then (in theory ) the recomended way in reaching this treshold would be to create 20 separated oVirt logical clusters with 12 nodes per each ( and datacenter managed from one ha-engine ) ? *4.* In present, I have the folowing one 9 nodes cluster , all hosts contributing with 2 disks each to a single replica 3 distributed replicated volume. They where added to the volume in the following order: node1 - disk1 node2 - disk1 ...... node9 - disk1 node1 - disk2 node2 - disk2 ...... node9 - disk2 At the moment, the volume is arbitrated, but I intend to go for full distributed replica 3. Is this a bad setup ? Why ? It oviously brakes the redhat recommended rules... Is there anyone so kind to discuss on these things ? Thank you very much ! Leo -- Best regards, Leo David -- Best regards, Leo David

3 5

Re: Ovirt OVN help needed
by Strahil 10 Jan '20

10 Jan '20

Hi Dominik, Thanks for your reply. On ovirt1 I got the following: [root@ovirt1 openvswitch]# less ovn-controller.log-20191216.gz 2019-12-15T01:49:02.988Z|00032|vlog|INFO|opened log file /var/log/openvswitch/ovn-controller.log 2019-12-16T01:18:02.114Z|00033|vlog|INFO|closing log file ovn-controller.log-20191216.gz (END) Same is on the other node: [root@ovirt2 openvswitch]# less ovn-controller.log-20191216.gz 2019-12-15T01:26:03.477Z|00028|vlog|INFO|opened log file /var/log/openvswitch/ovn-controller.log 2019-12-16T01:30:01.718Z|00029|vlog|INFO|closing log file ovn-controller.log-20191216.gz (END) The strange thing is that the geneve tunnels are there: [root@ovirt1 ~]# ovs-vsctl show c0e938f1-b5b5-4d5a-9cda-29dae2986f29 Bridge br-int fail_mode: secure Port "ovn-25cc77-0" Interface "ovn-25cc77-0" type: geneve options: {csum="true", key=flow, remote_ip="192.168.1.64"} Port "ovn-566849-0" Interface "ovn-566849-0" type: geneve options: {csum="true", key=flow, remote_ip="192.168.1.41"} Port br-int Interface br-int type: internal Port "vnet2" Interface "vnet2" ovs_version: "2.11.0" [root@ovirt1 ~]# ovs-vsctl list ports ovs-vsctl: unknown table "ports" [root@ovirt1 ~]# ovs-vsctl list port _uuid : fbf40569-925e-4430-a7c5-c78d58979bbc bond_active_slave : [] bond_downdelay : 0 bond_fake_iface : false bond_mode : [] bond_updelay : 0 cvlans : [] external_ids : {} fake_bridge : false interfaces : [3207c0cb-3000-40f2-a850-83548f76f090]lacp : [] mac : [] name : "vnet2" other_config : {} protected : false qos : [] rstp_statistics : {} rstp_status : {} statistics : {} status : {} tag : [] trunks : [] vlan_mode : [] _uuid : 8947f82d-a089-429b-8843-71371314cb52 bond_active_slave : [] bond_downdelay : 0 bond_fake_iface : false bond_mode : [] bond_updelay : 0 cvlans : [] external_ids : {} fake_bridge : false interfaces : [ec6a6688-e5d6-4346-ac47-ece1b8379440]lacp : [] mac : [] name : br-int other_config : {} protected : false qos : [] rstp_statistics : {} rstp_status : {} statistics : {} status : {} tag : [] trunks : [] vlan_mode : [] _uuid : 72d612be-853e-43e9-8f5c-ce66cef0bebe bond_active_slave : [] bond_downdelay : 0 bond_fake_iface : false bond_mode : [] bond_updelay : 0 cvlans : [] external_ids : {ovn-chassis-id="5668499c-7dd0-41ee-bc5d-2e6ee9cd61c3(a)192.168.1.41"} fake_bridge : false interfaces : [a31574fe-515b-420b-859d-7f2ac729638f]lacp : [] mac : [] name : "ovn-566849-0" other_config : {} protected : false qos : [] rstp_statistics : {} rstp_status : {} statistics : {} status : {} tag : [] trunks : [] vlan_mode : [] _uuid : 2043a15f-ec39-4cc3-b875-7be00423dd7a bond_active_slave : [] bond_downdelay : 0 bond_fake_iface : false bond_mode : [] bond_updelay : 0 cvlans : [] external_ids : {ovn-chassis-id="25cc77b3-046f-45c5-af0c-ffb2f77d73f1(a)192.168.1.64"} fake_bridge : false interfaces : [f9a9e3ff-070e-4044-b601-7f7394dc295f]lacp : [] mac : [] name : "ovn-25cc77-0" other_config : {} protected : false qos : [] rstp_statistics : {} rstp_status : {} statistics : {} status : {} tag : [] trunks : [] vlan_mode : [] [root@ovirt1 ~]# [root@ovirt2 ~]# ovs-vsctl show 3dbab138-6b90-44c5-af05-b8a944c9bf20 Bridge br-int fail_mode: secure Port "ovn-baa019-0" Interface "ovn-baa019-0" type: geneve options: {csum="true", key=flow, remote_ip="192.168.1.90"} Port br-int Interface br-int type: internal Port "vnet5" Interface "vnet5" Port "ovn-566849-0" Interface "ovn-566849-0" type: geneve options: {csum="true", key=flow, remote_ip="192.168.1.41"} ovs_version: "2.11.0" [root@ovirt2 ~]# ovs-vsctl list port _uuid : 151e1188-f07a-4750-a620-392a08e7e7fe bond_active_slave : [] bond_downdelay : 0 bond_fake_iface : false bond_mode : [] bond_updelay : 0 cvlans : [] external_ids : {ovn-chassis-id="baa0199e-d1a4-484c-af13-a41bcad19dbc(a)192.168.1.90"} fake_bridge : false interfaces : [4d4bc12a-609a-4917-b839-d4f652acdc33]lacp : [] mac : [] name : "ovn-baa019-0" other_config : {} protected : false qos : [] rstp_statistics : {} rstp_status : {} statistics : {} status : {} tag : [] trunks : [] vlan_mode : [] _uuid : 3a862f96-b3ec-46a9-bcf6-f385e5def410 bond_active_slave : [] bond_downdelay : 0 bond_fake_iface : false bond_mode : [] bond_updelay : 0 cvlans : [] external_ids : {} fake_bridge : false interfaces : [777f2819-ca27-4890-8d2f-11349ca0d398]lacp : [] mac : [] name : br-int other_config : {} protected : false qos : [] rstp_statistics : {} rstp_status : {} statistics : {} status : {} tag : [] trunks : [] vlan_mode : [] _uuid : a65109fa-f8b4-4670-8ae8-a2bd0bf6aba3 bond_active_slave : [] bond_downdelay : 0 bond_fake_iface : false bond_mode : [] bond_updelay : 0 cvlans : [] external_ids : {ovn-chassis-id="5668499c-7dd0-41ee-bc5d-2e6ee9cd61c3(a)192.168.1.41"} fake_bridge : false interfaces : [ed442077-f897-4e0b-97a1-a8051e9c3d56]lacp : [] mac : [] name : "ovn-566849-0" other_config : {} protected : false qos : [] rstp_statistics : {} rstp_status : {} statistics : {} status : {} tag : [] trunks : [] vlan_mode : [] _uuid : a1622e6f-fcd0-4a8a-b259-ca4d0ccf1cd2 bond_active_slave : [] bond_downdelay : 0 bond_fake_iface : false bond_mode : [] bond_updelay : 0 cvlans : [] external_ids : {} fake_bridge : false interfaces : [ca368654-54f3-49d0-a71c-8894426df6bf]lacp : [] mac : [] name : "vnet5" other_config : {} protected : false qos : [] rstp_statistics : {} rstp_status : {} statistics : {} status : {} tag : [] trunks : [] vlan_mode : [] [root@ovirt2 ~]# Best Regards, Strahil Nikolov On Dec 16, 2019 23:28, Dominik Holler <dholler(a)redhat.com> wrote: > > > > On Sat, Dec 14, 2019 at 11:36 AM Strahil Nikolov <hunter86_bg(a)yahoo.com> wrote: >> >> Hi Dominik, >> >> yes I was looking for those settings. >> >> I have added again the external provider , but I guess the mess is even bigger as I made some stupid decisions (like removing 2 port groups :) without knowing what I'm doing) . >> Sadly I can't remove all packages on the engine and hosts and reinstall them from scratch. >> >> Pip fails to install the openstacksdk (centOS7 is not great for such tasks) on the engine and my lack of knowledge in OVN makes it even more difficult. >> >> So the symptoms are that 2 machines can communicate with each other only if they are on the same host ,while on separate - no communications is happening. >> > > This indicates that the tunnels between the hosts are not created. > Can you please check the /var/log/openvswitch/ovn-controller.log on both hosts for errors and warnings, or share parts of the files here? > If this does not point us to a problem, ovn has to be reconfigured. If possible, most easy way to do this would be to ensure that > ovirt-provider-ovn is the default network provider of the cluster of the hosts, put one host after another in maintance mode and reinstall. > > >> >> How I created the network via UI: >> >> 1. Networks - new >> 2. Fill in the name >> 3. Create on external provider >> 4. Network Port security -> disabled (even undefined does not work) >> 5.Connect to physical network -> ovirtmgmt >> >> >> I would be happy to learn more about OVN and thus I would like to make it work. >> >> Here is some info from the engine: >> >> [root@engine ~]# ovn-nbctl show >> switch 1288ed26-471c-4bc2-8a7d-4531f306f44c (ovirt-pxelan-2a88b2e0-d04b-4196-ad50-074501e4ed08) >> port c1eba112-5eed-4c04-b25c-d3dcfb934546 >> addresses: ["56:6f:5a:65:00:06"] >> port 8b52ab60-f474-4d51-b258-cb2e0a53c34a >> type: localnet >> addresses: ["unknown"] >> port b2753040-881b-487a-92a1-9721da749be4 >> addresses: ["56:6f:5a:65:00:09"] >> [root@engine ~]# ovn-sbctl show >> Chassis "5668499c-7dd0-41ee-bc5d-2e6ee9cd61c3" >> hostname: "ovirt3.localdomain" >> Encap geneve >> ip: "192.168.1.41" >> options: {csum="true"} >> Chassis "baa0199e-d1a4-484c-af13-a41bcad19dbc" >> hostname: "ovirt1.localdomain" >> Encap geneve >> ip: "192.168.1.90" >> options: {csum="true"} >> Chassis "25cc77b3-046f-45c5-af0c-ffb2f77d73f1" >> hostname: "ovirt2.localdomain" >> Encap geneve >> ip: "192.168.1.64" >> options: {csum="true"} >> Port_Binding "b2753040-881b-487a-92a1-9721da749be4" >> Port_Binding &quo

4 8

AWX and error using ovirt as an inventory source
by Gianluca Cecchi 09 Jan '20

09 Jan '20

Hello, I have awx 9.0.1 and ansible 2.8.5 in container of a CentOS 7.7 server. I'm trying to use oVirt 4.3.6.7-1.el7 as a source of an inventory in awx but I get error when syncing Find at bottom below the error messages. I see that in recent past (around June this year) there were some problems, but they should be solved now, correct? There was also a problem in syncing when some powered off VMs were present in oVirt env, but I think this solved too, correct? Any way to replicate / test from command line of awx container? I try some things but in command line I always get error regarding oVirt inventory script requires ovirt-engine-sdk-python >= 4.0.0 that I think depends on not using correct command line and/or not setting needed env. Thanks in advance, Gianluca 2.536 INFO Updating inventory 4: MYDC_OVIRT 3.011 INFO Reading Ansible inventory source: /var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/plugins/inventory/ovirt4.py 3.013 INFO Using VIRTUAL_ENV: /var/lib/awx/venv/ansible 3.013 INFO Using PATH: /var/lib/awx/venv/ansible/bin:/var/lib/awx/venv/awx/bin:/var/lib/awx/venv/awx/bin:/var/lib/awx/venv/awx/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 3.013 INFO Using PYTHONPATH: /var/lib/awx/venv/ansible/lib/python3.6/site-packages: Traceback (most recent call last): File "/var/lib/awx/venv/awx/bin/awx-manage", line 11, in <module> load_entry_point('awx==9.0.1.0', 'console_scripts', 'awx-manage')() File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/__init__.py", line 158, in manage execute_from_command_line(sys.argv) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line utility.execute() File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/management/base.py", line 323, in run_from_argv self.execute(*args, **cmd_options) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/management/base.py", line 364, in execute output = self.handle(*args, **options) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/management/commands/inventory_import.py", line 1153, in handle raise exc File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/management/commands/inventory_import.py", line 1043, in handle venv_path=venv_path, verbosity=self.verbosity).load() File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/management/commands/inventory_import.py", line 214, in load return self.command_to_json(base_args + ['--list']) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/management/commands/inventory_import.py", line 197, in command_to_json self.method, proc.returncode, stdout, stderr)) RuntimeError: ansible-inventory failed (rc=1) with stdout: stderr: ansible-inventory 2.8.5 config file = /etc/ansible/ansible.cfg configured module search path = ['/var/lib/awx/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules'] ansible python module location = /usr/lib/python3.6/site-packages/ansible executable location = /usr/bin/ansible-inventory python version = 3.6.8 (default, Oct 7 2019, 17:58:22) [GCC 8.2.1 20180905 (Red Hat 8.2.1-3)] Using /etc/ansible/ansible.cfg as config file [WARNING]: * Failed to parse /var/lib/awx/venv/awx/lib64/python3.6/site- packages/awx/plugins/inventory/ovirt4.py with script plugin: Inventory script (/var/lib/awx/venv/awx/lib64/python3.6/site- packages/awx/plugins/inventory/ovirt4.py) had an execution error: File "/usr/lib/python3.6/site-packages/ansible/inventory/manager.py", line 268, in parse_source plugin.parse(self._inventory, self._loader, source, cache=cache) File "/usr/lib/python3.6/site-packages/ansible/plugins/inventory/script.py", line 161, in parse raise AnsibleParserError(to_native(e)) [WARNING]: Unable to parse /var/lib/awx/venv/awx/lib64/python3.6/site- packages/awx/plugins/inventory/ovirt4.py as an inventory source ERROR! No inventory was parsed, please check your configuration and options.

3 23

HCL: 4.3.7: Hosted engine fails
by Christian Reiss 08 Jan '20

08 Jan '20

Hey all, Using a homogeneous ovirt-node-ng-4.3.7-0.20191121.0 freshly created cluster using node installer I am unable to deploy the hosted engine. Everything else worked. In vdsm.log is a line, just after attempting to start the engine: libvirtError: the CPU is incompatible with host CPU: Host CPU does not provide required features: virt-ssbd I am using AMD EPYC 7282 16-Core Processors. I have attached - vdsm.log (during and failing the start) - messages (for bootup / libvirt messages) - dmesg (grub / boot config) - deploy.log (browser output during deployment) - virt-capabilites (virsh -r capabilities) I can't think -or don't know- off any other log files of interest here, but I am more than happy to oblige. notectl check tells me Status: OK Bootloader ... OK Layer boot entries ... OK Valid boot entries ... OK Mount points ... OK Separate /var ... OK Discard is used ... OK Basic storage ... OK Initialized VG ... OK Initialized Thin Pool ... OK Initialized LVs ... OK Thin storage ... OK Checking available space in thinpool ... OK Checking thinpool auto-extend ... OK vdsmd ... OK layers: ovirt-node-ng-4.3.7-0.20191121.0: ovirt-node-ng-4.3.7-0.20191121.0+1 bootloader: default: ovirt-node-ng-4.3.7-0.20191121.0 (3.10.0-1062.4.3.el7.x86_64) entries: ovirt-node-ng-4.3.7-0.20191121.0 (3.10.0-1062.4.3.el7.x86_64): index: 0 title: ovirt-node-ng-4.3.7-0.20191121.0 (3.10.0-1062.4.3.el7.x86_64) kernel: /boot/ovirt-node-ng-4.3.7-0.20191121.0+1/vmlinuz-3.10.0-1062.4.3.el7.x86_64 args: "ro crashkernel=auto rd.lvm.lv=onn_node01/swap rd.lvm.lv=onn_node01/ovirt-node-ng-4.3.7-0.20191121.0+1 rhgb quiet LANG=en_GB.UTF-8 img.bootid=ovirt-node-ng-4.3.7-0.20191121.0+1" initrd: /boot/ovirt-node-ng-4.3.7-0.20191121.0+1/initramfs-3.10.0-1062.4.3.el7.x86_64.img root: /dev/onn_node01/ovirt-node-ng-4.3.7-0.20191121.0+1 current_layer: ovirt-node-ng-4.3.7-0.20191121.0+1 The odd thing is the hosted engine vm does get started during initial configuration and works. Just when the ansible stuff is done an its moved over to ha storage the CPU quirks start. So far I learned that ssbd is a mitigation protection but the flag is not in my cpu. Well, ssbd is virt-ssbd is not. I am *starting* with ovirt. I would really, really welcome it if recommendations would include clues on how to make it happen. I do rtfm, but I was unable to find anything (or any solution) anywhere. Not after 80 hours of working on this. Thank you all. -Chris. -- Christian Reiss - email(a)christian-reiss.de /"\ ASCII Ribbon support(a)alpha-labs.net \ / Campaign X against HTML WEB alpha-labs.net / \ in eMails GPG Retrieval https://gpg.christian-reiss.de GPG ID ABCD43C5, 0x44E29126ABCD43C5 GPG fingerprint = 9549 F537 2596 86BA 733C A4ED 44E2 9126 ABCD 43C5 "It's better to reign in hell than to serve in heaven.", John Milton, Paradise lost.

2 3

OVirt Engine Server Died - Steps for Rebuilding the Ovirt Engine System
by bob.franzke＠mdaemon.com 07 Jan '20

07 Jan '20

Full disclosure here.....I am not an Ovirt Expert. I am a network Engineer that has been forced to take over sysadmin duties for a departed co-worker. I have little experience with Ovirt so apologies up front for anything I say that comes across as stupid or "RTM" questions. Normally I would do just that but I am in a bind and am trying to figure this out quickly. We have an OVirt installation setup that consists of 4 nodes and a server that hosts the ovirt-engine all running CentOS 7. The server that hosts the engine has a pair of failing hard drives and I need to replace the hardware ASAP. Need to outline the steps needed to build a new server to serve as and replace the ovirt engine server. I have backed up the entire /etc directory and the backups being done nightly by the engine itself. I also backed up the iscsi info and took a printout of all the disk arrangement . The disk has gotten so bad at this point that the DB won't back up any longer. Get fatal:backup failed error when trying to run the ovirt backup tool. Also the Ovirt management site is not rendering and I am not sure why. Is there anything else I need to make sure I backup in order to migrate the engine from one server to another? Also, until I can get the engine running again, is there any tool available to manage the VMs on the hosts themselves. The VMs on the hosts are running but need a way to manage them if needed in case something happens while the engine is being repaired. Any info on this as well as what to backup and the steps to move the engine from one server to another would be much much appreciated. Sorry I know this a real RTM type post but I am in a bind and need a solution rather quickly. Thanks in advance.

4 11

Question about HCI gluster_inventory.yml
by John Call 07 Jan '20

07 Jan '20

Hi ovirt-users, I'm trying to automate my HCI deployment, but can't figure out how to specify multiple network interfaces in gluster_inventory.yml. My servers have two NICs, one for ovirtmgmt (and everything else), and the other is just for Gluster. How should I populate the inventory/vars file? Is this correct? [root@rhhi1 hc-ansible-deployment]# pwd /etc/ansible/roles/gluster.ansible/playbooks/hc-ansible-deployment [root@rhhi1 hc-ansible-deployment]# cat gluster_inventory.yml --lots of stuff omitted-- hc_nodes: hosts: host1-STORAGE-fqdn: host2-STORAGE-fqdn: host3-STORAGE-fqdn: vars: cluster_nodes: - host1-ovirtmgmt-fqdn - host2-ovirtmgmt-fqdn - host3-ovirtmgmt-fqdn gluster_features_hci_cluster: "{{ cluster_nodes }}" gluster: host2-STORAGE-fqdn: host3-STORAGE-fqdn: storage_domains: [{"name":"data","host":"host1-STORAGE-fqdn","address":"host1-STORAGE-fqdn","path":"/data","mount_options":"backup-volfile-servers=host2-STORAGE-fqdn:host3-STORAGE-fqdn"},{"name":"vmstore","host":"host1-STORAGE-fqdn","address":"host1-STORAGE-fqdn","path":"/vmstore","mount_options":"backup-volfile-servers=host2-STORAGE-fqdn:host3-STORAGE-fqdn"}]

2 3

Re: null
by Strahil 06 Jan '20

06 Jan '20

I do use automatic migration policy. The main question you have to solve is: 1. Why the nodes became 'Non-operational' .Usually this happens when the management interface (in your case HoatedEngine VM) could not reach the nodes over the management network. By default, management is going over the ovirtmgmt network. I guess you have created the new network. Marked that new network as management network and then the switch was off , causing 'Non-Operational state'. 2. Migrating VMs is usually a safe approach, but this behavior is quite strange. If a node ia Non-operational -> there could be no successful migration. 3. Some of the VMs got paused due to storage issue. Are you using GlusterFS, NFS or iSCSI ? If yes, you need to clarify why you lost your storage. I guess for now you can mark each VM to be migrated only manually (VM -> Edit) and if they are critical VMs , set a high Availability from each VM's Edit options. In such case, if a node fails, the VMs will be restarted on another node. 4. Have you setup node fencing ? For example APC, iLo, iDRAC and other fencing mechanisms can allow the HostedEngine use another Host as fencing proxy and to reset the problematic Hypervisor. P.S.: You can define the following alias in '~/.bashrc' : alias virsh='virsh -c qemu:///system?authfile=/etc/ovirt-hosted-engine/virsh_auth.conf' Then you can verify your VMs even when a HostedEngine is down: 'virsh list --all' Best Regards, Strahil NikolovOn Dec 27, 2019 08:40, zhouhao(a)vip.friendtimes.net wrote: > > I had a crash yesterday in my ovirt cluster, which is made up of 3 nodes. > > I just tried to add a new network, but the whole cluster crashed > > I added a new network to my cluster, but while I was debugging the newswitch, when the switch was poweroff, the node detected the network card status down and then moved to Non-Operational state. > > > > At this time all of 3 nodes moved to Non-Operational state. > > All virtual machines have started automatic migration,When I received the alert email, all virtual machines were suspended > > > > > > In 15 minutes my newswitch were power up again.The 3 ovirt-nodes become active again, but many virtual machines become unresponsive or suspended due to forced migration, and only a few virtual machines are pulled up again due to cancelled migration > > After I tried to terminate the migration tasks and restart ovirt-engine service, I was still unable to restore most of the virtual machines, so I had to restart 3 ovirt-nodes to restore my virtual machine > > I didn't recover all the virtual machines until an hour later > > > Then I modify my migration policy to " Do Not Migrate Virtual Machines" > > Which migration Policy do you recommend? > > I'm afraid to use cluster... > > ________________________________ > zhouhao(a)vip.friendtimes.net

2 1

ovirt 4.3.7 + Gluster in hyperconverged (production design)
by adrianquintero＠gmail.com 02 Jan '20

02 Jan '20

Hi, After playing a bit with oVirt and Gluster in our pre-production environment for the last year, we have decided to move forward with a our production design using ovirt 4.3.7 + Gluster in a hyperconverged setup. For this we are looking get answers to a few questions that will help out with our design and eventually lead to our production deployment phase: Current HW specs (total servers = 18): 1.- Server type: DL380 GEN 9 2.- Memory: 256GB 3.-Disk QTY per hypervisor: - 2x600GB SAS (RAID 0) for the OS - 9x1.2TB SSD (RAID[0, 6..]..? ) for GLUSTER. 4.-Network: - Bond1: 2x10Gbps - Bond2: 2x10Gbps (for migration and gluster) Our plan is to build 2x9 node clusters, however the following questions come up: 1.-Should we build 2 separate environments each with its own engine? or should we do 1 engine that will manage both clusters? 2.-What would be the best gluster volume layout for #1 above with regards to RAID configuration: - JBOD or RAID6 or…?. - what is the benefit or downside of using JBOD vs RAID 6 for this particular scenario? 3.-Would you recommend Ansible-based deployment (if supported)? If yes where would I find the documentation for it? or should we just deploy using the UI?. - I have reviewed the following and in Chapter 5 it only mentions Web UI https://access.redhat.com/documentation/en-us/red_hat_hyperconverged_infras… - Also looked at https://github.com/gluster/gluster-ansible/tree/master/playbooks/hc-ansible… but could not get it to work properly. 4.-what is the recommended max server qty in a hyperconverged setup with gluster, 12, 15, 18...? Thanks, Adrian

3 2

Re: Issue deploying self hosted engine on new install
by Sang Un Ahn 02 Jan '20

02 Jan '20

Hi, I have figured it out that the root cause of the deployment failure is timing out while the hosted engine was trying to connect to host vis SSH as shown in engine.log (located in /var/log/ovirt-hosted-engine-setup/engine-logs-2019-12-31T06:34:38Z/ovirt-engine): 2019-12-31 15:43:06,082+09 ERROR [org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand] (default task-1) [f48796e7-a4c5-4c09-a70d-956f0c4249b4] Failed to establish session with host 'alice-ovirt-01.sdfarm.kr': SSH connection timed out connecting to 'root(a)alice-ovirt-01.sdfarm.kr' 2019-12-31 15:43:06,085+09 WARN [org.ovirt.engine.core.bll.hostdeploy.AddVdsCommand] (default task-1) [f48796e7-a4c5-4c09-a70d-956f0c4249b4] Validation of action 'AddVds' failed for user admin@internal-authz. Reasons: VAR__ACTION__ADD,VAR__TYPE__HOST,$server alice-ovirt-01.sdfarm.kr,VDS_CANNOT_CONNECT_TO_SERVER 2019-12-31 15:43:06,129+09 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-1) [] Operation Failed: [Cannot add Host. Connecting to host via SSH has failed, verify that the host is reachable (IP address, routable address etc.) You may refer to the engine.log file for further details.] The FQDN of hosted engine (alice-ovirt-engine.sdfarm.kr <http://alice-ovirt-engine.sdfarm.kr/>) is resolved as well as the host (alice-ovirt-01.sdfarm.kr <http://alice-ovirt-01.sdfarm.kr/>) and SSH is the one of services that are allowed by firewalld. I believe the rules of firewalld is automatically configured during the deployment to work with hosted engine and the host. Also root access is configured to be allowed at the first stage of deployment. I was just wondering how I can verify the hosted engine can access to the host at this stage? Once it fails to deploy, the deployment script make all things rolled back (I believe it cleans all up) and the vm-status of hosted-engine is un-deployed. Thank you in advance, Best regards, Sang-Un

2 4

Re: 4.2.8 to 4.3.7 > Management slow
by Strahil 02 Jan '20

02 Jan '20

You can manually change the I/O scheduler of the disks and if that works better for you, put a rule in udev. Here is mine: [root@engine rules.d]# cat /etc/udev/rules.d/90-default-io-scheduler.rules ACTION=="add|change", KERNEL=="sd*[!0-9]", ATTR{queue/scheduler}="none" ACTION=="add|change", KERNEL=="vd*[!0-9]", ATTR{queue/scheduler}="none" [root@engine rules.d]# cat /sys/block/vda/queue/scheduler [none] mq-deadline kyber Best Regards, Strahil NikolovOn Dec 31, 2019 12:30, Demeter Tibor <tdemeter(a)itsmart.hu> wrote: > > Dear Users, > > I've successfully upgraded my 4 node hyperconverged system from 4.2.8 to 4.3.7. > After upgrade everything seems to working fine, but the whole management system seems very slow. > Spends many seconds when I clicking on "virtual machines" or I want to edit a virtual machines. > The speed of vms and the IO is fine. > > It is running on a glusterfs (distributed replicate, on 3 node, 9 bricks). There are no errors, everything fine. but terrible slow:( > The engine vm has 0.2-0.3 load. > > What can I do? > > Thanks in advance and I wish Happy New Year! > > Regards, > Tibor > >

2 1

Re: Issue deploying self hosted engine on new install
by thomas＠hoberg.net 02 Jan '20

02 Jan '20

Yes, I have had the same and posted about it here somewhere: I believe it's an incompatible Ansible change. Here is the critical part if the message below: "The 'ovirt_host_facts' module has been renamed to 'ovirt_host_info', and the renamed one no longer returns ansible_facts" and that change was made in the transition of Ansible 2.8* to 2.9, from what I gathered. I guess I should just make it a bug report if you find the same message in your logs. [ ERROR ] fatal: [localhost]: FAILED! => {"ansible_facts": {"ovirt_hosts": [{"address": "xdrd1022s.priv.atos.fr", "affinity_labels": [], "auto_numa_status": "unknown", "certificate": {"organization": "priv.atos.fr", "subject": "O=priv.atos.fr,CN=xdrd1022s.priv.atos.fr"}, "cluster": {"href": "/ovirt-engine/api/clusters/c407e776-1c3c-11ea-aeed-00163e56112a", "id": "c407e776-1c3c-11ea-aeed-00163e56112a"}, "comment": "", "cpu": {"speed": 0.0, "topology": {}}, "device_passthrough": {"enabled": false}, "devices": [], "external_network_provider_configurations": [], "external_status": "ok", "hardware_information": {"supported_rng_sources": []}, "hooks": [], "href": "/ovirt-engine/api/hosts/a5bb73a1-f923-4568-8dda-434e07f7e243", "id": "a5bb73a1-f923-4568-8dda-434e07f7e243", "katello_errata": [], "kdump_status": "unknown", "ksm": {"enabled": false}, "max_scheduling_memory": 0, "memory": 0, "name": "xdrd1022s.priv.atos.fr", "network_attachments": [], "nics": [], "numa_nodes": [], "numa_supporte d": false, "os": {"custom_kernel_cmdline": ""}, "permissions": [], "port": 54321, "power_management": {"automatic_pm_enabled": true, "enabled": false, "kdump_detection": true, "pm_proxies": []}, "protocol": "stomp", "se_linux": {}, "spm": {"priority": 5, "status": "none"}, "ssh": {"fingerprint": "SHA256:wqpBWq9Kb9+Nb3Jwtw61QJzo+R4gGOP2dLubssU5EPs", "port": 22}, "statistics": [], "status": "install_failed", "storage_connection_extensions": [], "summary": {"total": 0}, "tags": [], "transparent_huge_pages": {"enabled": false}, "type": "rhel", "unmanaged_networks": [], "update_available": false, "vgpu_placement": "consolidated"}]}, "attempts": 120, "changed": false, "deprecations": [{"msg": "The 'ovirt_host_facts' module has been renamed to 'ovirt_host_info', and the renamed one no longer returns ansible_facts", "version": "2.13"}]}

4 5

Re: Node not starting | blk_cloned_rq_check_limits: over max size limit
by Strahil 31 Dec '19

31 Dec '19

By the way, why do you use multipath for local storage like this EVO nvme ? Happy New Year ! Best Regards, Strahil NikolovOn Dec 31, 2019 21:51, Strahil <hunter86_bg(a)yahoo.com> wrote: > > You can check https://access.redhat.com/solutions/2437991 & https://access.redhat.com/solutions/3014361 > > You have 2 options: > 1. Set a udev rule like this one (replace NETAPP with your storage) > ACTION!="add|change", GOTO="rule_end" ENV{ID_VENDOR}=="NETAPP*", RUN+="/bin/sh -c 'echo 4096 > /sys%p/queue/max_sectors_kb'" LABEL="rule_end" > > 2. Set max_sectors_kb in devices section of multipath.conf > You will need to stop LVM and then flush the device map , so the new option to take effect (faster is to reboot) > > Good Luck & Happy New Year. > > Best Regards, > Strahil Nikolov > On Dec 31, 2019 17:53, Stefan Wolf <shb256(a)gmail.com> wrote: > > > > hi all, > > > > i ve 4 nodes running with current ovirt. > > I ve only a problem on one host even after a fresh installation . > > I ve installed the latest image. > > Than I add the node to the cluster > > Everything is working good. > > After this I configure the network. > > BUT, after a restart the host does not come up again. > > I got this error: blk_cloned_rq_check_limits: over max size limit > > every 5 seconds > > > > I can continue with control-D > > or I can login with root password to fix the problem. but i dont know what is the problem and where does it came from > > > > I ve also changed the sas disk to nvme storage, but I ve changed this on every host. And this problem exists only on one host > > > > i found this https://lists.centos.org/pipermail/centos/2017-December/167727.html > > the output is > > [root@kvm380 ~]# ./test.sh > > Sys Block Node : Device max_sectors_kb max_hw_sectors_kb > > /sys/block/dm-0 : onn_kvm380-pool00_tmeta 256 4096 > > /sys/block/dm-1 : onn_kvm380-pool00_tdata 256 4096 > > /sys/block/dm-10 : onn_kvm380-var 256 4096 > > /sys/block/dm-11 : onn_kvm380-tmp 256 4096 > > /sys/block/dm-12 : onn_kvm380-home 256 4096 > > /sys/block/dm-13 : onn_kvm380-var_crash 256 4096 > > /sys/block/dm-2 : onn_kvm380-pool00-tpool 256 4096 > > /sys/block/dm-3 : onn_kvm380-ovirt--node--ng--4.3.7--0.20191121.0+1 256 4096 > > /sys/block/dm-4 : onn_kvm380-swap 256 4096 > > /sys/block/dm-5 : eui.0025385991b1e27a 512 2048 > > /sys/block/dm-6 : eui.0025385991b1e27a1 512 2048 > > /sys/block/dm-7 : onn_kvm380-pool00 256 4096 > > /sys/block/dm-8 : onn_kvm380-var_log_audit 256 4096 > > /sys/block/dm-9 : onn_kvm380-var_log 256 4096 > > cat: /sys/block/nvme0n1/device/vendor: Datei oder Verzeichnis nicht gefunden > > /sys/block/nvme0n1: Samsung SSD 970 EVO 1TB 512 2048 > > /sys/block/sda : HP LOGICAL VOLUME 256 4096 > > > > is the nvme not starting correct > > [root@kvm380 ~]# systemctl status multipathd > > ● multipathd.service - Device-Mapper Multipath Device Controller > > Loaded: loaded (/usr/lib/systemd/system/multipathd.service; enabled; vendor preset: enabled) > > Active: active (running) since Di 2019-12-31 16:16:32 CET; 31min ago > > Process: 1919 ExecStart=/sbin/multipathd (code=exited, status=0/SUCCESS) > > Process: 1916 ExecStartPre=/sbin/multipath -A (code=exited, status=0/SUCCESS) > > Process: 1911 ExecStartPre=/sbin/modprobe dm-multipath (code=exited, status=0/SUCCESS) > > Main PID: 1921 (multipathd) > > Tasks: 7 > > CGroup: /system.slice/multipathd.service > > └─1921 /sbin/multipathd > > > > Dez 31 16:47:58 kvm380.durchhalten.intern multipathd[1921]: nvme0n1: mark as failed > > Dez 31 16:47:58 kvm380.durchhalten.intern multipathd[1921]: eui.0025385991b1e27a: Entering recovery mode: max_retries=4 > > Dez 31 16:47:58 kvm380.durchhalten.intern multipathd[1921]: eui.0025385991b1e27a: remaining active paths: 0 > > Dez 31 16:48:02 kvm380.durchhalten.intern multipathd[1921]: 259:0: reinstated > > Dez 31 16:48:02 kvm380.durchhalten.intern multipathd[1921]: eui.0025385991b1e27a: queue_if_no_path enabled > > Dez 31 16:48:02 kvm380.durchhalten.intern multipathd[1921]: eui.0025385991b1e27a: Recovered to normal mode > > Dez 31 16:48:02 kvm380.durchhalten.intern multipathd[1921]: eui.0025385991b1e27a: remaining active paths: 1 > > Dez 31 16:48:03 kvm380.durchhalten.intern multipathd[1921]: nvme0n1: mark as failed > > Dez 31 16:48:03 kvm380.durchhalten.intern multipathd[1921]: eui.0025385991b1e27a: Entering recovery mode: max_retries=4 > > Dez 31 16:48:03 kvm380.durchhalten.intern multipathd[1921]: eui.0025385991b1e27a: remaining active paths: 0 > > > > why is it marked as failed? > > > > if i create a new volume with cockpit and use it for bricks for gluster, every thing is fine. until reboot > > > > > > maybe some one can point me the direction > > _______________________________________________ > > Users mailing list -- users(a)ovirt.org > > To unsubscribe send an email to users-leave(a)ovirt.org > > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/MHHFFWAY5T5BV…

1 0

Re: Node not starting | blk_cloned_rq_check_limits: over max size limit
by Strahil 31 Dec '19

31 Dec '19

You can check https://access.redhat.com/solutions/2437991 & https://access.redhat.com/solutions/3014361 You have 2 options: 1. Set a udev rule like this one (replace NETAPP with your storage) ACTION!="add|change", GOTO="rule_end" ENV{ID_VENDOR}=="NETAPP*", RUN+="/bin/sh -c 'echo 4096 > /sys%p/queue/max_sectors_kb'" LABEL="rule_end" 2. Set max_sectors_kb in devices section of multipath.conf You will need to stop LVM and then flush the device map , so the new option to take effect (faster is to reboot) Good Luck & Happy New Year. Best Regards, Strahil Nikolov On Dec 31, 2019 17:53, Stefan Wolf <shb256(a)gmail.com> wrote: > > hi all, > > i ve 4 nodes running with current ovirt. > I ve only a problem on one host even after a fresh installation . > I ve installed the latest image. > Than I add the node to the cluster > Everything is working good. > After this I configure the network. > BUT, after a restart the host does not come up again. > I got this error: blk_cloned_rq_check_limits: over max size limit > every 5 seconds > > I can continue with control-D > or I can login with root password to fix the problem. but i dont know what is the problem and where does it came from > > I ve also changed the sas disk to nvme storage, but I ve changed this on every host. And this problem exists only on one host > > i found this https://lists.centos.org/pipermail/centos/2017-December/167727.html > the output is > [root@kvm380 ~]# ./test.sh > Sys Block Node : Device max_sectors_kb max_hw_sectors_kb > /sys/block/dm-0 : onn_kvm380-pool00_tmeta 256 4096 > /sys/block/dm-1 : onn_kvm380-pool00_tdata 256 4096 > /sys/block/dm-10 : onn_kvm380-var 256 4096 > /sys/block/dm-11 : onn_kvm380-tmp 256 4096 > /sys/block/dm-12 : onn_kvm380-home 256 4096 > /sys/block/dm-13 : onn_kvm380-var_crash 256 4096 > /sys/block/dm-2 : onn_kvm380-pool00-tpool 256 4096 > /sys/block/dm-3 : onn_kvm380-ovirt--node--ng--4.3.7--0.20191121.0+1 256 4096 > /sys/block/dm-4 : onn_kvm380-swap 256 4096 > /sys/block/dm-5 : eui.0025385991b1e27a 512 2048 > /sys/block/dm-6 : eui.0025385991b1e27a1 512 2048 > /sys/block/dm-7 : onn_kvm380-pool00 256 4096 > /sys/block/dm-8 : onn_kvm380-var_log_audit 256 4096 > /sys/block/dm-9 : onn_kvm380-var_log 256 4096 > cat: /sys/block/nvme0n1/device/vendor: Datei oder Verzeichnis nicht gefunden > /sys/block/nvme0n1: Samsung SSD 970 EVO 1TB 512 2048 > /sys/block/sda : HP LOGICAL VOLUME 256 4096 > > is the nvme not starting correct > [root@kvm380 ~]# systemctl status multipathd > ● multipathd.service - Device-Mapper Multipath Device Controller > Loaded: loaded (/usr/lib/systemd/system/multipathd.service; enabled; vendor preset: enabled) > Active: active (running) since Di 2019-12-31 16:16:32 CET; 31min ago > Process: 1919 ExecStart=/sbin/multipathd (code=exited, status=0/SUCCESS) > Process: 1916 ExecStartPre=/sbin/multipath -A (code=exited, status=0/SUCCESS) > Process: 1911 ExecStartPre=/sbin/modprobe dm-multipath (code=exited, status=0/SUCCESS) > Main PID: 1921 (multipathd) > Tasks: 7 > CGroup: /system.slice/multipathd.service > └─1921 /sbin/multipathd > > Dez 31 16:47:58 kvm380.durchhalten.intern multipathd[1921]: nvme0n1: mark as failed > Dez 31 16:47:58 kvm380.durchhalten.intern multipathd[1921]: eui.0025385991b1e27a: Entering recovery mode: max_retries=4 > Dez 31 16:47:58 kvm380.durchhalten.intern multipathd[1921]: eui.0025385991b1e27a: remaining active paths: 0 > Dez 31 16:48:02 kvm380.durchhalten.intern multipathd[1921]: 259:0: reinstated > Dez 31 16:48:02 kvm380.durchhalten.intern multipathd[1921]: eui.0025385991b1e27a: queue_if_no_path enabled > Dez 31 16:48:02 kvm380.durchhalten.intern multipathd[1921]: eui.0025385991b1e27a: Recovered to normal mode > Dez 31 16:48:02 kvm380.durchhalten.intern multipathd[1921]: eui.0025385991b1e27a: remaining active paths: 1 > Dez 31 16:48:03 kvm380.durchhalten.intern multipathd[1921]: nvme0n1: mark as failed > Dez 31 16:48:03 kvm380.durchhalten.intern multipathd[1921]: eui.0025385991b1e27a: Entering recovery mode: max_retries=4 > Dez 31 16:48:03 kvm380.durchhalten.intern multipathd[1921]: eui.0025385991b1e27a: remaining active paths: 0 > > why is it marked as failed? > > if i create a new volume with cockpit and use it for bricks for gluster, every thing is fine. until reboot > > > maybe some one can point me the direction > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/MHHFFWAY5T5BV…

1 0

Node not starting | blk_cloned_rq_check_limits: over max size limit
by Stefan Wolf 31 Dec '19

31 Dec '19

hi all, i ve 4 nodes running with current ovirt. I ve only a problem on one host even after a fresh installation . I ve installed the latest image. Than I add the node to the cluster Everything is working good. After this I configure the network. BUT, after a restart the host does not come up again. I got this error: blk_cloned_rq_check_limits: over max size limit every 5 seconds I can continue with control-D or I can login with root password to fix the problem. but i dont know what is the problem and where does it came from I ve also changed the sas disk to nvme storage, but I ve changed this on every host. And this problem exists only on one host i found this https://lists.centos.org/pipermail/centos/2017-December/167727.html the output is [root@kvm380 ~]# ./test.sh Sys Block Node : Device max_sectors_kb max_hw_sectors_kb /sys/block/dm-0 : onn_kvm380-pool00_tmeta 256 4096 /sys/block/dm-1 : onn_kvm380-pool00_tdata 256 4096 /sys/block/dm-10 : onn_kvm380-var 256 4096 /sys/block/dm-11 : onn_kvm380-tmp 256 4096 /sys/block/dm-12 : onn_kvm380-home 256 4096 /sys/block/dm-13 : onn_kvm380-var_crash 256 4096 /sys/block/dm-2 : onn_kvm380-pool00-tpool 256 4096 /sys/block/dm-3 : onn_kvm380-ovirt--node--ng--4.3.7--0.20191121.0+1 256 4096 /sys/block/dm-4 : onn_kvm380-swap 256 4096 /sys/block/dm-5 : eui.0025385991b1e27a 512 2048 /sys/block/dm-6 : eui.0025385991b1e27a1 512 2048 /sys/block/dm-7 : onn_kvm380-pool00 256 4096 /sys/block/dm-8 : onn_kvm380-var_log_audit 256 4096 /sys/block/dm-9 : onn_kvm380-var_log 256 4096 cat: /sys/block/nvme0n1/device/vendor: Datei oder Verzeichnis nicht gefunden /sys/block/nvme0n1: Samsung SSD 970 EVO 1TB 512 2048 /sys/block/sda : HP LOGICAL VOLUME 256 4096 is the nvme not starting correct [root@kvm380 ~]# systemctl status multipathd ● multipathd.service - Device-Mapper Multipath Device Controller Loaded: loaded (/usr/lib/systemd/system/multipathd.service; enabled; vendor preset: enabled) Active: active (running) since Di 2019-12-31 16:16:32 CET; 31min ago Process: 1919 ExecStart=/sbin/multipathd (code=exited, status=0/SUCCESS) Process: 1916 ExecStartPre=/sbin/multipath -A (code=exited, status=0/SUCCESS) Process: 1911 ExecStartPre=/sbin/modprobe dm-multipath (code=exited, status=0/SUCCESS) Main PID: 1921 (multipathd) Tasks: 7 CGroup: /system.slice/multipathd.service └─1921 /sbin/multipathd Dez 31 16:47:58 kvm380.durchhalten.intern multipathd[1921]: nvme0n1: mark as failed Dez 31 16:47:58 kvm380.durchhalten.intern multipathd[1921]: eui.0025385991b1e27a: Entering recovery mode: max_retries=4 Dez 31 16:47:58 kvm380.durchhalten.intern multipathd[1921]: eui.0025385991b1e27a: remaining active paths: 0 Dez 31 16:48:02 kvm380.durchhalten.intern multipathd[1921]: 259:0: reinstated Dez 31 16:48:02 kvm380.durchhalten.intern multipathd[1921]: eui.0025385991b1e27a: queue_if_no_path enabled Dez 31 16:48:02 kvm380.durchhalten.intern multipathd[1921]: eui.0025385991b1e27a: Recovered to normal mode Dez 31 16:48:02 kvm380.durchhalten.intern multipathd[1921]: eui.0025385991b1e27a: remaining active paths: 1 Dez 31 16:48:03 kvm380.durchhalten.intern multipathd[1921]: nvme0n1: mark as failed Dez 31 16:48:03 kvm380.durchhalten.intern multipathd[1921]: eui.0025385991b1e27a: Entering recovery mode: max_retries=4 Dez 31 16:48:03 kvm380.durchhalten.intern multipathd[1921]: eui.0025385991b1e27a: remaining active paths: 0 why is it marked as failed? if i create a new volume with cockpit and use it for bricks for gluster, every thing is fine. until reboot maybe some one can point me the direction

1 0

4.2.8 to 4.3.7 > Management slow
by Demeter Tibor 31 Dec '19

31 Dec '19

Dear Users, I've successfully upgraded my 4 node hyperconverged system from 4.2.8 to 4.3.7. After upgrade everything seems to working fine, but the whole management system seems very slow. Spends many seconds when I clicking on "virtual machines" or I want to edit a virtual machines. The speed of vms and the IO is fine. It is running on a glusterfs (distributed replicate, on 3 node, 9 bricks). There are no errors, everything fine. but terrible slow:( The engine vm has 0.2-0.3 load. What can I do? Thanks in advance and I wish Happy New Year! Regards, Tibor

1 0

Re: Issue deploying self hosted engine on new install
by Strahil 31 Dec '19

31 Dec '19

Hi, Most probably it is a bug. Can you add 'debugger: on_failed' to the task ? Maybe it will provide a hint where is the issue. Best Regards, Strahil NikolovOn Dec 31, 2019 06:36, realapyo(a)gmail.com wrote: > > Hi, > > I have encountered the same error that results in the failure of hosted engine deployment. I have tried to replace "ovirt_host_facts" by "ovirt_host_info" in /usr/share/ansible/roles/ovirt.hosted_engine_setup/tasks/bootstrap_local_vm/05_add_host.yml however it did not help with the following error message: > > 2019-12-31 11:23:27,595+0900 INFO otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:109 TASK [ovirt.hosted_engine_setup : Wait for the host to be up] > 2019-12-31 11:23:28,997+0900 DEBUG otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:103 {u'msg': u"The conditional check 'host_result_up_check is succeeded and host_result_up_check.ansible_facts.ovirt_hosts|length >= 1 and ( host_result_up_check.ansible_facts.ovirt_hosts[0].status == 'up' or host_result_up_check.ansible_facts.ovirt_hosts[0].status == 'non_operational' )' failed. The error was: error while evaluating conditional (host_result_up_check is succeeded and host_result_up_check.ansible_facts.ovirt_hosts|length >= 1 and ( host_result_up_check.ansible_facts.ovirt_hosts[0].status == 'up' or host_result_up_check.ansible_facts.ovirt_hosts[0].status == 'non_operational' )): 'dict object' has no attribute 'ansible_facts'", u'_ansible_no_log': False} > 2019-12-31 11:23:29,098+0900 ERROR otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:107 fatal: [localhost]: FAILED! => {"msg": "The conditional check 'host_result_up_check is succeeded and host_result_up_check.ansible_facts.ovirt_hosts|length >= 1 and ( host_result_up_check.ansible_facts.ovirt_hosts[0].status == 'up' or host_result_up_check.ansible_facts.ovirt_hosts[0].status == 'non_operational' )' failed. The error was: error while evaluating conditional (host_result_up_check is succeeded and host_result_up_check.ansible_facts.ovirt_hosts|length >= 1 and ( host_result_up_check.ansible_facts.ovirt_hosts[0].status == 'up' or host_result_up_check.ansible_facts.ovirt_hosts[0].status == 'non_operational' )): 'dict object' has no attribute 'ansible_facts'"} > > According to the latest doc of Ansible (2.9) regarding "ovirt_host_facts", it is deprecated and one should use "ovirt_host_info" instead as you can refer here: https://docs.ansible.com/ansible/latest/modules/ovirt_host_facts_module.html > > And one of the requirements to use "ovirt_host_info" is "ovirt-engine-sdk-python >= 4.3.0" however in my installation, the installed package is "python-ovirt-engine-sdk4 (4.3.2-2.el7)" instead of "ovirt-engine-sdk-python". The "ovirt-engine-sdk-python" package is available however the version is 3.6.9.1-1.el7. > > I have been trying to install the latest oVirt (based in 4.3 repository) and hyperconverged mode using 3 nodes. Tried already with cockpit installation and now debugging it by using CLI installation. > > Is this a bug or is there already a fix? > > Best regards, > Sang-Un > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/V2OYCPZAHUVLF…

1 0

Re: Cannot add Foreman as a provider
by Strahil 30 Dec '19

30 Dec '19

Maybe 6.1 means Foreman with Red Hat Satellite 6.1 ? Best Regards, Strahil NikolovOn Dec 30, 2019 16:41, Robert Webb <rwebb(a)ropeguru.com> wrote: > > oVirt – 4.3.7 > > Foreman - foreman-1.24.0-15.el7.noarch > > > > I have not tried any downgrades at this point. > > > > Looking at this page, https://www.ovirt.org/documentation/admin-guide/chap-External_Providers.html , it would seem that 4.3.x of oVirt is not compatible with any version of Foreman, as the page shows oVirt 4.2 is supported with Foreman 6.1. Which also confuses me as I cannot find any Foreman version 6.1. > > > > > > From: Strahil Nikolov <hunter86_bg(a)yahoo.com> > Sent: Sunday, December 29, 2019 5:00 PM > To: users <users(a)ovirt.org>; Robert Webb <rwebb(a)ropeguru.com> > Subject: Re: [ovirt-users] Cannot add Foreman as a provider > >

2 1

Re: Cannot add Foreman as a provider
by Strahil Nikolov 30 Dec '19

30 Dec '19

Hi Robert, I'm just speculating, but maybe the package on the oVirt is too new or too old ? What version is your Foreman ? Have you tried to downgrade/upgrade the packages for the integration between the 2 systems ? Best Regards,Strahil Nikolov В неделя, 29 декември 2019 г., 18:53:53 ч. Гринуич+2, Robert Webb <rwebb(a)ropeguru.com> написа: So I just setup a brand new Foreman instance and have been trying to get it setup with oVirt. I was able to install the oVirt plugins under Foreman and have the oVirt engine setup as a computer resource. Where I am running into an issue is adding Foreman as an additional provider under oVirt. In the log I get: Command 'org.ovirt.engine.core.bll.provider.TestProviderConnectivityCommand' failed: EngineException: https://foreman.saroza-webb.internal:443/api/v2/discovered_hosts?format=jso… (Failed with error PROVIDER_FAILURE and code 5050) I have done some searching, but everything I have found seems to be from years ago and deals with oVirt 3.5.x Any thoughts? _______________________________________________ Users mailing list -- users(a)ovirt.org To unsubscribe send an email to users-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/CO64DW24PDJJC…

2 1

Re: what is "use host" field in storage domain creation??
by Strahil 30 Dec '19

30 Dec '19

I think SPM is not doing initial setup - just other tasks - like snapshots creation/deletion. Best Regards, Strahil NikolovOn Dec 30, 2019 11:57, yam yam <hyunooudy(a)gmail.com> wrote: > > isn't it SPM's job? > and is the "use host" required only for setup? > > thanks for quick replying. > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/IRQCB37EXLOHQ…

1 0

what is "use host" field in storage domain creation??
by hyunooudy＠gmail.com 30 Dec '19

30 Dec '19

hello and happy new year~ I am wondering the role of "use host" field in storage domain creation. https://www.ovirt.org/documentation/install-guide/chap-Configuring_Storage.… above link says all communication to the storage domain is through "use host". but I can't understand inefficiently passing through that "use host" even though every host can directly access to all domains. it seems like i am misunderstanding something.

3 2

what is "use host" field in storage domain creation??
by hyunooudy＠gmail.com 30 Dec '19

30 Dec '19

hello and happy new year~ I am wondering role of "use host" field in storage domain creation. https://www.ovirt.org/documentation/install-guide/chap-Configuring_Storage.… above link says all communication to the storage domain is through "use host". but I can't understand inefficiently passing through that "use host" even though every host can directly access to all domains. it seems like i am misunderstanding something.

1 0

Cannot add Foreman as a provider
by Robert Webb 29 Dec '19

29 Dec '19

So I just setup a brand new Foreman instance and have been trying to get it setup with oVirt. I was able to install the oVirt plugins under Foreman and have the oVirt engine setup as a computer resource. Where I am running into an issue is adding Foreman as an additional provider under oVirt. In the log I get: Command 'org.ovirt.engine.core.bll.provider.TestProviderConnectivityCommand' failed: EngineException: https://foreman.saroza-webb.internal:443/api/v2/discovered_hosts?format=jso… (Failed with error PROVIDER_FAILURE and code 5050) I have done some searching, but everything I have found seems to be from years ago and deals with oVirt 3.5.x Any thoughts?

1 0

Metrics
by gregory.day＠target.com 29 Dec '19

29 Dec '19

Hello. I'm trying to deploy the metrics store and the ansible run fails at "TASK [/usr/share/ansible/roles/oVirt.metrics/roles/oVirt.origin-on-ovirt : Collect oVirt VM facts]" I've tried adding the -vvv and ANSIBLE_DEBUG=1, but that isn't giving me any additional information on why it is failing. fatal: [localhost]: FAILED! => { "attempts": 5, "censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false } When I edit the playbook and set no_log: true, the output is shown, but the resulting log shows that the "ansible_facts.ovirt_vms[0].fqdn" dictionary key does not exist. What is the best way to work around this issue? Thanks, Greg

2 1

No virtio-win vfd oVirt 4.3.7
by Vrgotic, Marko 27 Dec '19

27 Dec '19

Dear oVIrt, I am trying to install Windows 10 on oVirt, following procedure on: https://www.ovirt.org/documentation/vmm-guide/chap-Installing_Windows_Virtu… Unfortunately, there is no option as explained in the “Installing Windows on VirtIO-Optimized Hardware”. There is just no “virtio-win.vfd” option when attaching Floppy. Is this valid. I have ISO Domain, which is not hosted by Hosted Engine, but mounte additionally. Is this required in order to have the virtio-win.vfd option? If not, please let me know what am I missing. In the meanwhile, I am working on building custom Wind 10 ISO which contains Virtio/Virtio-SCSI drivers. Kindly awaiting your reply. ----- kind regards/met vrindelijke groet Marko Vrgotic Sr. System Engineer @ System Administration ActiveVideo o: +31 (35) 6774131 e: m.vrgotic(a)activevideo.com<mailto:m.vrgotic@activevideo.com> w: www.activevideo.com<http://www.activevideo.com> ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The Netherlands. The information contained in this message may be legally privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or destroy any copy of this message.

2 2

ovirt-host 4.4.0 Task Update system failed to execute - Running QEMU processes found, cannot upgrade Vdsm
by alexandermurashkin＠msn.com 27 Dec '19

27 Dec '19

On a regular basis, there is "Task Update system failed to execute" error in the engine.log. At the same time, the ovirt-host-mgmt-ansible-check log contains "Running QEMU processes found, cannot upgrade Vdsm" error. Is there some configuration setting that controls it? Why oVirt is trying to update ovirt-host package on the host at all if there are virtual machines running? Note that 2019-12-27 11:57:21,880-06 INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (EE-ManagedScheduledExecutorService-engineThreadMonitoringThreadPool-Thread-1) [] Thread pool 'hostUpdatesChecker' is using 0 threads out of 5, 3 threads waiting for tasks. 2019-12-27 12:02:30,878-06 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedExecutorService-hostUpdatesChecker-Thread-4) [] EVENT_ID: ANSIBLE_RUNNER_EVENT_NOTIFICATION(559), Check for update of host harrier. Gathering Facts. 2019-12-27 12:02:45,990-06 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedExecutorService-hostUpdatesChecker-Thread-4) [] EVENT_ID: ANSIBLE_RUNNER_EVENT_NOTIFICATION(559), Check for update of host harrier. Install ovirt-host package if it isn't installed. 2019-12-27 12:02:55,024-06 ERROR [org.ovirt.engine.core.bll.host.HostUpgradeManager] (EE-ManagedExecutorService-hostUpdatesChecker-Thread-4) [] Failed to read host packages: Task Update system failed to execute: 2019-12-27 12:02:55,025-06 ERROR [org.ovirt.engine.core.bll.hostdeploy.HostUpdatesChecker] (EE-ManagedExecutorService-hostUpdatesChecker-Thread-4) [] Failed to check if updates are available for host 'harrier' with error message 'Task Update system failed to execute: ' 2019-12-27 12:02:55,106-06 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedExecutorService-hostUpdatesChecker-Thread-4) [] EVENT_ID: HOST_AVAILABLE_UPDATES_FAILED(839), Failed to check for available updates on host harrier with message 'Task Update system failed to execute: '. 2 # rpm -q ovirt-host ovirt-host-4.4.0-0.2.alpha.el8.x86_64 # dnf update Updating Subscription Management repositories. Last metadata expiration check: 0:11:31 ago on Fri Dec 27 13:16:55 2019. Error: Running QEMU processes found, cannot upgrade Vdsm. # dnf update ovirt-host Updating Subscription Management repositories. Extra Packages for Enterprise Linux 8 - x86_64 7.3 kB/s | 5.3 kB 00:00 GlusterFS 6 packages for x86_64 2.8 kB/s | 3.0 kB 00:01 GlusterFS 6 noarch packages 18 kB/s | 3.0 kB 00:00 virtio-win builds roughly matching what will be shipped in upcoming RHEL 8.2 kB/s | 3.0 kB 00:00 Copr repo for EL8_collection owned by sbonazzo 10 kB/s | 3.6 kB 00:00 Copr repo for gluster-ansible owned by sac 13 kB/s | 3.3 kB 00:00 Copr repo for ovsdbapp owned by mdbarroso 14 kB/s | 3.3 kB 00:00 Copr repo for nmstate-stable owned by nmstate 14 kB/s | 3.3 kB 00:00 Copr repo for NetworkManager-1.20 owned by networkmanager 14 kB/s | 3.3 kB 00:00 Latest oVirt master nightly snapshot 4.0 kB/s | 3.0 kB 00:00 Latest oVirt master additional nightly snapshot 5.0 kB/s | 3.0 kB 00:00 Red Hat CodeReady Linux Builder for RHEL 8 x86_64 (RPMs) 14 kB/s | 4.5 kB 00:00 Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs) 27 kB/s | 4.1 kB 00:00 Red Hat Enterprise Linux 8 for x86_64 - AppStream (RPMs) 36 kB/s | 4.5 kB 00:00 Dependencies resolved. Nothing to do. Complete! ------------------------- ovirt-host-mgmt-ansible-check log -------------------------------------------- 2019-12-27 12:02:45 CST - TASK [ovirt-host-upgrade : Update system] ************************************** 2019-12-27 12:02:55 CST - fatal: [harrier....]: FAILED! => {"changed": false, "failures": [], "msg": "Unknown Error occured: Running QEMU processes found, cannot upgrade Vdsm.", "rc": 1, "results": []} 2019-12-27 12:02:55 CST - { "status" : "OK", "msg" : "", "data" : { "event" : "runner_on_failed", "uuid" : "74b495fa-2534-4850-b905-7255fed1fc56", "stdout" : "fatal: [harrier...]: FAILED! => {\"changed\": false, \"failures\": [], \"msg\": \"Unknown Error occured: Running QEMU processes found, cannot upgrade Vdsm.\", \"rc\": 1, \"results\": []}", "counter" : 26, "pid" : 27445, "created" : "2019-12-27T18:02:53.245099", "end_line" : 25, "runner_ident" : "094f2f72-28d3-11ea-bc88-525400060930", "start_line" : 24, "event_data" : { "play_pattern" : "all", "play" : "all", "event_loop" : null, "task_args" : "", "remote_addr" : "harrier....", "res" : { "_ansible_no_log" : false, "changed" : false, "results" : [ ], "msg" : "Unknown Error occured: Running QEMU processes found, cannot upgrade Vdsm.", "rc" : 1, "invocation" : { "module_args" : { "lock_timeout" : 300, "update_cache" : true, "conf_file" : null, "exclude" : [ ], "allow_downgrade" : false, "disable_gpg_check" : false, "disable_excludes" : null, "validate_certs" : true, "state" : "latest", "disablerepo" : [ ], "releasever" : null, "skip_broken" : false, "autoremove" : false, "download_dir" : null, "installroot" : "/", "install_weak_deps" : true, "name" : [ "*" ], "download_only" : false, "bugfix" : false, "list" : null, "install_repoquery" : true, "update_only" : false, "disable_plugin" : [ ], "enablerepo" : [ ], "security" : false, "enable_plugin" : [ ] } }, "failures" : [ ] }, "pid" : 27445, "play_uuid" : "52540006-0930-56fb-1424-000000000006", "task_uuid" : "52540006-0930-56fb-1424-000000000012", "task" : "Update system", "playbook_uuid" : "758d0326-a52a-4fe9-87b3-6be0d3492784", "playbook" : "ovirt-host-upgrade.yml", "task_action" : "yum", "host" : "harrier....", "ignore_errors" : null, "role" : "ovirt-host-upgrade", "task_path" : "/usr/share/ovirt-engine/ansible-runner-service-project/project/roles/ovirt-host-upgrade/tasks/main.yml:18" }, "parent_uuid" : "52540006-0930-56fb-1424-000000000012" } }

1 1

I had a crash yesterday in my ovirt cluster,I just tried to add a new network，Which migration Policy do you recommend?
by zhouhao＠vip.friendtimes.net 27 Dec '19

27 Dec '19

I had a crash yesterday in my ovirt cluster, which is made up of 3 nodes. I just tried to add a new network, but the whole cluster crashed I added a new network to my cluster, but while I was debugging the newswitch, when the switch was poweroff, the node detected the network card status down and then moved to Non-Operational state. At this time all of 3 nodes moved to Non-Operational state. All virtual machines have started automatic migration,When I received the alert email, all virtual machines were suspended In 15 minutes my newswitch were power up again.The 3 ovirt-nodes become active again, but many virtual machines become unresponsive or suspended due to forced migration, and only a few virtual machines are pulled up again due to cancelled migration After I tried to terminate the migration tasks and restart ovirt-engine service, I was still unable to restore most of the virtual machines, so I had to restart 3 ovirt-nodes to restore my virtual machine I didn't recover all the virtual machines until an hour later Then I modify my migration policy to " Do Not Migrate Virtual Machines" Which migration Policy do you recommend? I'm afraid to use cluster... zhouhao(a)vip.friendtimes.net

1 0

Re: Overt Node | Non-Responsive
by Strahil 26 Dec '19

26 Dec '19

Hi Vijay, Can you share the error messages in thelogs ,as the screenshot shows only 'INFO' messages. Best Regards, Strahil NikolovOn Dec 26, 2019 10:17, Vijay Sachdeva <vijay.sachdeva(a)indiqus.com> wrote: > > Dear Team, > > > > I have a host on which I am using local storage for my VMs. Now I rebooted my host because it was shifted to another location, right now in vdsm logs I see errors like: > > > > > > Where in host I can see my mounted LVM volume is perfectly fine. I started VSDM services and libvirtd, but host is not leaving Non-Responsive state. > > > > Any help ..!! > > > > > > Vijay Sachdeva > > > >

1 0

Re: oVirt and Containers
by Jan Zmeskal 26 Dec '19

26 Dec '19

Okay, so this topic is quite vast, but I believe I can at the very least give you a few pointers and maybe others might chime in as well. Firstly, there's the Kubevirt project. It enables you to manage both application containers and virtual machines workloads (that cannot be easily containerized) in a shared environment. Another benefit is getting advantages of the powerful Kubernetes scheduler. I myself am not too familiar with Kubevirt, so I can only offer this high-level overview. More info here: https://kubevirt.io/ Then there is another approach which I am more familiar with. You might want to use oVirt as an infrastructure layer on top of which you run containerized workflow. This is achieved by deploying either OpenShift <https://www.openshift.com/> or the upstream project OKD <https://www.okd.io/> in the oVirt virtual machines. In that scenario, oVirt VMs are considered by OpenShift as compute resources and are used for scheduling containers. There are some advantages to this setup and two come into mind. Firstly, you can scale such OpenShift cluster up or down by adding/removing oVirt VMs according to your needs. Secondly, you don't need to set up all of this yourself. For OpenShift 3, Red Hat provides detailed guide on how to go about this. Part of that guide are Ansible playbooks that automate the deployment for you as long as you provide required variables. More info here: https://docs.openshift.com/container-platform/3.11/install_config/configuri… When it comes to OpenShift 4, there are two types of deployment. There's UPI - user provisioned infrastructure. In that scenario, you prepare all the resources for OpenShift 4 beforehand and deploy it in that existing environment. And there's also IPI - installer provisioned infrastructure. This means that you just give the installer access to your environment (e.g. AWS public cloud) and the installer provisions resources for itself based on recommendations and best practices. At this point, neither UPI nor IPI is supported for oVirt. However there is a GitHub repository <https://github.com/sa-ne/openshift4-rhv-upi> that can guide you through UPI installation on oVirt and also provides automation playbooks for that. I have personally followed the steps from the repository and deployed OpenShift 4.2 on top of oVirt without any major issues. As far as I remember, I might have needed occasional variable here and there but the process worked. Hope this helps! Jan On Tue, Dec 24, 2019 at 8:21 PM Robert Webb <rwebb(a)ropeguru.com> wrote: > Hi Jan, > > Honestly, I didn't have anything specific in mind, just what is being used > out there today and what may be more prevalent. > > Just getting my oVIrt set up and want to know what might be recommended. > Would probably be mostly deploying images like Homeassistent, piHole, etc.. > for now. > > I guess if there is good oVirt direct integration, it would be nice to > keep it all in a single interface. > > Thanks.. > > ________________________________________ > From: Jan Zmeskal <jzmeskal(a)redhat.com> > Sent: Tuesday, December 24, 2019 1:54 PM > To: Robert Webb > Cc: users > Subject: Re: [ovirt-users] oVirt and Containers > > Hi Robert, > > there are different answers based on what you mean by integrating oVirt > and containers. Do you mean: > > - Installing container management (Kubernetes or OpenShift) on top of > oVirt and using oVirt as infrastructure? > - Managing containers from oVirt interface? > - Running VM workloads inside containers? > - Something different? > > I can elaborate more based on your specific needs > > Best regards > Jan > > On Tue, Dec 24, 2019 at 3:52 PM Robert Webb <rwebb(a)ropeguru.com<mailto: > rwebb(a)ropeguru.com>> wrote: > I was searching around to try and figure out the best way to integrate > oVirt and containers. > > I have found some sites that discuss it but all of them are like 2017 and > older. > > Any recommendations? > > Just build VM’s to host containers or is there some direct integration? > > Here are a couple of the old sites > > https://fromanirh.github.io/containers-in-ovirt.html > > > https://kubernetes.io/docs/setup/production-environment/on-premises-vm/ovir… > > > https://www.ovirt.org/develop/release-management/features/integration/conta… > _______________________________________________ > Users mailing list -- users(a)ovirt.org<mailto:users@ovirt.org> > To unsubscribe send an email to users-leave(a)ovirt.org<mailto: > users-leave(a)ovirt.org> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/7BIW2Z3ILBSAG… > > > -- > > Jan Zmeskal > > Quality Engineer, RHV Core System > > Red Hat <https://www.redhat.com> > > [ > https://marketing-outfit-prod-images.s3-us-west-2.amazonaws.com/f5445ae0c9d… > ]<https://www.redhat.com> > > -- Jan Zmeskal Quality Engineer, RHV Core System Red Hat <https://www.redhat.com> <https://www.redhat.com>

2 1

Overt-Engine Down | Host Unresponsive
by Vijay Sachdeva 26 Dec '19

26 Dec '19

Dear Team, I have one question, if Ovirt-engine goes down for some reason. Will it impact my host to go un-responsive and all VMs down? I just faced a situation where my host was in un-responsive status and all VMs were down. I restart the ovirt-engine service and host got activated. Can anyone let me know this situation? Thanks Vijay Sachdeva

2 1

oVirt on a dedicated server with a single NIC and IP
by alexcustos＠gmail.com 26 Dec '19

26 Dec '19

Hi, My goal is to set up the latest oVirt on a dedicated server (Hetzner) with a single NIC and IP. Then using a Reverse Proxy and/or VPN (on the same host) to manage connections to VMs. I expected I could add a subnet as an alias to the main interface, or set it up on VLAN, or using vSwitch at the last resort, but all my attempts failed. Some configurations work until the first reboot, then VDSM screw up the main interface and I lost connection to the server, or Hosted Engine doesn't start from global maintenance. I've run out of good ideas already. I'm extremely sorry if this question already been discussed somewhere, but I found nothing helpful. I would be very appreciated for any hint on how to achieve this or at least a confirmation that the desired configuration is viable.

2 2

Overt Node | Non-Responsive
by Vijay Sachdeva 26 Dec '19

26 Dec '19

Dear Team, I have a host on which I am using local storage for my VMs. Now I rebooted my host because it was shifted to another location, right now in vdsm logs I see errors like: Where in host I can see my mounted LVM volume is perfectly fine. I started VSDM services and libvirtd, but host is not leaving Non-Responsive state. Any help ..!! Vijay Sachdeva

1 0

Re: [ovirt 4.2]vdsm host was shutdown unexpectly, configure of some vms was changed or lost when host was re-powered on and vm was started
by Strahil 25 Dec '19

25 Dec '19

Hi, If the VMs are starting properly and they don't use cloudinit - then the issue is not oVirt specific, but guest specific (Linux/Windows depending on guest OS). So your should check: 1. Does your host have any Netwprks out of sync (Host's network tab) If yes - put the server into maintenance and fix the issue (host's network tab) 2. Check each VM's configuration if it is defined to use CloudInit -> if yes, verify that cloudinit's service is running on the guest 3. Verify each problematic guest's network settings. If needed, set a static IP and try to ping another IP from the same subnet/Vlan . Best Regards, Strahil NikolovOn Dec 25, 2019 11:41, lifuqiong(a)sunyainfo.com wrote: >> >> >>> Dear All: >>> My ovirt engine managed two vdsms with nfs storage on another nfs server, it worked fine about three months. >>> One of host (host_1.3; ip:172.18.1.3) was created about 16 vms, but the host_1.3 was shutdown unexpectly about 2019-12-23 16:11, when re-started host and vms; half of the vms was lost some configure or changed , such as lost theirs ip etc.(the vm name is 'zzh_Chain49_ACG_M' in the vdsm.log) >>> >>> the vm zzh_Chain49_ACG_M was create by rest API through vm's templated . the template is 20.1.1.161; the vm zzh_Chain49_ACG_M was created by ovirt rest api and changed the ip to 20.1.1.219 >>> by ovirt rest api. But the ip was changed to templated ip when the accident happened. >>> >>> the vm's os is centos. >>> >>> Hope get help from you soon, Thank you. >>> >>> Mark >>> Sincerely.

1 0

[ovirt 4.2]vdsm host was shutdown unexpectly, configure of some vms was changed or lost when host was re-powered on and vm was started
by lifuqiong＠sunyainfo.com 25 Dec '19

25 Dec '19

Dear All: My ovirt engine managed two vdsms with nfs storage on another nfs server, it worked fine about three months. One of host (host_1.3; ip:172.18.1.3) was created about 16 vms, but the host_1.3 was shutdown unexpectly about 2019-12-23 16:11, when re-started host and vms; half of the vms was lost some configure or changed , such as lost theirs ip etc.(the vm name is 'zzh_Chain49_ACG_M' in the vdsm.log) the vm zzh_Chain49_ACG_M was create by rest API through vm's templated . the template is 20.1.1.161; the vm zzh_Chain49_ACG_M was created by ovirt rest api and changed the ip to 20.1.1.219 by ovirt rest api. But the ip was changed to templated ip when the accident happened. the vm's os is centos. Hope get help from you soon, Thank you. Mark Sincerely.

1 0

Re: Subscribe to mailing list and install oVirt with freeIPA
by Yedidyah Bar David 25 Dec '19

25 Dec '19

On Wed, Dec 25, 2019 at 10:45 AM Kien Pham <ptkcc(a)outlook.com> wrote: > > Dear oVirt team, > > I have post a thread on forum but I got problem with mailing list. How can I subscribe the mailing list. > > This is the question. > > Dear all, > > I am installing oVirt but I got an error when running engine-setup and get the following notification: > > [ INFO ] Restarting httpd > [ ERROR ] Failed to execute stage 'Closing up': Failed to start service 'httpd' > [ INFO ] Stage: Clean up > Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20191224111455-ppsefr.log > [ INFO ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20191224111745-setup.conf' > [ INFO ] Stage: Pre-termination > [ INFO ] Stage: Termination > [ ERROR ] Execution of setup failed > > I think that the problem is freeIPA. I installed freeIPA to manage user authetication. But IPA is currently using port 80 so httpd service cannot start. Right. This isn't supported, for exactly this reason. The ovirt-engine rpm conflicts with ipa-server. How did you install it? I now see that there is also a package python2-ipaserver, and in fedora and el8 also python3-ipaserver. Now pushed a patch to conflict with them as well: https://gerrit.ovirt.org/105935 Anyway, you'll have to set them up on two different machines. Thanks for the report, and sorry for the failure... Good luck and best regards, -- Didi

1 0

Re: oVirt and Containers
by Jan Zmeskal 24 Dec '19

24 Dec '19

Hi Robert, there are different answers based on what you mean by integrating oVirt and containers. Do you mean: - Installing container management (Kubernetes or OpenShift) on top of oVirt and using oVirt as infrastructure? - Managing containers from oVirt interface? - Running VM workloads inside containers? - Something different? I can elaborate more based on your specific needs Best regards Jan On Tue, Dec 24, 2019 at 3:52 PM Robert Webb <rwebb(a)ropeguru.com> wrote: > I was searching around to try and figure out the best way to integrate > oVirt and containers. > > > > I have found some sites that discuss it but all of them are like 2017 and > older. > > > > Any recommendations? > > > > Just build VM’s to host containers or is there some direct integration? > > > > Here are a couple of the old sites > > > > https://fromanirh.github.io/containers-in-ovirt.html > > > > > https://kubernetes.io/docs/setup/production-environment/on-premises-vm/ovir… > > > > > https://www.ovirt.org/develop/release-management/features/integration/conta… > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/7BIW2Z3ILBSAG… > -- Jan Zmeskal Quality Engineer, RHV Core System Red Hat <https://www.redhat.com> <https://www.redhat.com>

2 1

Subscribe to mailing list and install oVirt with freeIPA
by Kien Pham 24 Dec '19

24 Dec '19

Dear oVirt team, I have post a thread on forum but I got problem with mailing list. How can I subscribe the mailing list. This is the question. Dear all, I am installing oVirt but I got an error when running engine-setup and get the following notification: [ INFO ] Restarting httpd [ ERROR ] Failed to execute stage 'Closing up': Failed to start service 'httpd' [ INFO ] Stage: Clean up Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20191224111455-ppsefr.log [ INFO ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20191224111745-setup.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination [ ERROR ] Execution of setup failed I think that the problem is freeIPA. I installed freeIPA to manage user authetication. But IPA is currently using port 80 so httpd service cannot start. Is there any suggest. Thank for all reply. Best regards. Kimloai.

1 0

Re: ovirt 4.3.7 + Gluster in hyperconverged (production design)
by Strahil 24 Dec '19

24 Dec '19

On Dec 24, 2019 02:18, Jayme <jaymef(a)gmail.com> wrote: > > If you can afford it I would definitely do raid. Being able to monitor and replace disks at the raid level is much easier than brick. With raid I’d do a gluster arbiter setup so your aren’t losing too much space. > > Keep an eye on libgfapi. It’s not default setting due to a few bugs but I’ve been testing it in my ssd hci cluster and have been seeing up to 5x io improvement. Also jumbo frames on those 10Gbps switches. > > Someone else will probably chime in re the other questions. I believe the GUI can only deploy a three sever cluster then you have to add the remaining hosts afterward. > > On Mon, Dec 23, 2019 at 5:56 PM <adrianquintero(a)gmail.com> wrote: >> >> Hi, >> After playing a bit with oVirt and Gluster in our pre-production environment for the last year, we have decided to move forward with a our production design using ovirt 4.3.7 + Gluster in a hyperconverged setup. >> >> For this we are looking get answers to a few questions that will help out with our design and eventually lead to our production deployment phase: >> >> Current HW specs (total servers = 18): >> 1.- Server type: DL380 GEN 9 >> 2.- Memory: 256GB >> 3.-Disk QTY per hypervisor: >> - 2x600GB SAS (RAID 0) for the OS I would pick Raid1 and split that between OS and brick for engine 's gluster volume >> - 9x1.2TB SSD (RAID[0, 6..]..? ) for GLUSTER. If you had bigger NICs I would preffer RAID0. Have you considered JBOD approach (each disk is presented as a standalone LUN from the raid array) ? Raid 6 is waste for your SSDs (when you already have replica 3 volumes). Ask HPE to provide SSDs from different manufacturing batches. I think that with SSD thee best compromise will be RAID 5 with replica3 for faster reads. As the SSDs will be overwritten with the same ammount of data, there could be a situation where all 3 from a replica set will be in predictive failiure (which is not nice) - that's why a raid 5 is a good option. >> 4.-Network: >> - Bond1: 2x10Gbps >> - Bond2: 2x10Gbps (for migration and gluster) Do not use Active-Backup bond as you will loose bandwidth. LACP (with hashing on layer 3+ layer4) is a good option, but it depends if your machines will be in the same LAN segment (communicating over layer 2 and no gateway inbetween). >> Our plan is to build 2x9 node clusters, however the following questions come up: >> >> 1.-Should we build 2 separate environments each with its own engine? or should we do 1 engine that will manage both clusters? If you decide to split - you have less fault tollerance - for example loosing 3 ot of 18 is better than loosing 3 out of 9 nodes :) >> 2.-What would be the best gluster volume layout for #1 above with regards to RAID configuration: >> - JBOD or RAID6 or…?. RAID6 wastes alot of space >> - what is the benefit or downside of using JBOD vs RAID 6 for this particular scenario? >> 3.-Would you recommend Ansible-based deployment (if supported)? If yes where would I find the documentation for it? or should we just deploy using the UI?. There is a topic in the mailing list about issues with ansible.It's up to you. >> - I have reviewed the following and in Chapter 5 it only mentions Web UI https://access.redhat.com/documentation/en-us/red_hat_hyperconverged_infras… >> - Also looked at https://github.com/gluster/gluster-ansible/tree/master/playbooks/hc-ansible… but could not get it to work properly. >> >> 4.-what is the recommended max server qty in a hyperconverged setup with gluster, 12, 15, 18...? I don't remmember but I think it was 9 nodes for Hyperconverged setup in RHV. >> Thanks, >> >> Adrian Best Regards, Strahil Nikolov

1 0

Re: Problems after 4.3.8 update
by Mahdi Adnan 24 Dec '19

24 Dec '19

We had an issue after upgrading Gluster nodes to 6.6, we had memory leak in gluster self heal daemon which cause it to do infinite heal and ate all of the nodes RAM "128GB", we had to downgrade to 6.5 to solve the problem. I think it is related to the changes made to the SHD code in version 6.6 ( 1760706 <https://bugzilla.redhat.com/1760706>) I might find the time and try to bug report the issue. Anyway, glad you solved your issue.

1 0

oVirt and Containers
by Robert Webb 24 Dec '19

24 Dec '19

I was searching around to try and figure out the best way to integrate oVirt and containers. I have found some sites that discuss it but all of them are like 2017 and older. Any recommendations? Just build VM's to host containers or is there some direct integration? Here are a couple of the old sites https://fromanirh.github.io/containers-in-ovirt.html https://kubernetes.io/docs/setup/production-environment/on-premises-vm/ovir… https://www.ovirt.org/develop/release-management/features/integration/conta…

1 0

Problems with ovirt-web-ui-1.6.0-1.el7.noarch
by Matthias Leopold 24 Dec '19

24 Dec '19

Hi, after upgrading oVirt from 4.3.5 to 4.3.7 some things in the VM Portal stopped working for users with "User" role: * running guest agent is not recognized ("It looks like no guest agent is configured on the VM.") * CD/ISO list is "[Empty]" for this I found "- limit CD/ISO list by data center" in rpm changelog, but this doesn't help me in any way Assigning other roles (UserVmRunTimeManager, PowerUserRole) doesn't change anything. /var/log/ovirt-engine/ui.log is quiet. Downgrading to ovirt-web-ui-1.5.3-1.el7.noarch resolves both issues. Am I the only one who is experiencing this? thx for advice Matthias

2 3

Re: Strategy to Mange Ovirt VMs Created from Templates
by Strahil 23 Dec '19

23 Dec '19

Sealing the VM is not something specific to oVirt, and is also valid for KVM. Actually sealing the VM is to run virt-sysprep against the VM/disk. You can check http://libguestfs.org/virt-sysprep.1.html and especially the --list-operations which can help you understand what ia being cleaned up. So you can: 1. Update the VM 2. Install and enable cloudinit service 3. Maybe try to create your ansible user and allow ssh keys (Using ansible over root is not good and is a bad practice) 4. Power off and seal the machine as a template (details at https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Virtualiza… ) 5. Create VM from the template and power it up manually. 6. Verify that the new VM matches your needs and remove. About cloud init - you should know that is used extensively in OpenStack and VmWare has it's own solution similar to CloudInit. Some examples can be found at: https://cloudinit.readthedocs.io/en/latest/topics/examples.html So you can change passwords, configure network, create your ansible user with necessary permissions and way more. Best Regards, Strahil NikolovOn Dec 23, 2019 18:46, jeremy_tourville(a)hotmail.com wrote: > > Thank you for your reply Luca, > In general your work flow is helpful and makes sense to me. > > I meant to say above- "As part of the template creation process ***the Ovirt docs*** say to seal the VM". > > So I think I understand that you need to use both processes (seal template + cloudinit) to get everything to work as desired. I'd still appreciate any more specifics about what sealing a VM does. > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/BRLNWOQB6FM4B…

1 0

Strategy to Mange Ovirt VMs Created from Templates
by jeremy_tourville＠hotmail.com 23 Dec '19

23 Dec '19

I want to be able to manage VMs using Ansible. As part of the template creation process it says to seal the VM. Can someone tell me what sealing does to a Linux VM? I understand it removes some of things that make the VM unique but no real specifics. So, if I want to manage a VM created from a template would this general process work?- Seal the VM Install CloudInit and keys, accounts, etc Shut off VM and create template from it. Create new VM using Ansbile & CloudInit CloudInit would have just enough info so that you could manage the VM with Ansible. Would that work? I am just starting to explore what CloudInit can do and what it is. I am brand new to it. I didn't find enough info on template sealing to help me devise a full cycle management strategy. Perhaps there are other/easier methods? Thanks for your advice and input.

3 4

[ovirt 4.2]vdsm host was shutdown unexpectly, configure of some vms was changed or lost when host was re-powered on and vm was started
by lifuqiong＠sunyainfo.com 23 Dec '19

23 Dec '19

Dear All: My ovirt engine managed two vdsms with nfs storage on another nfs server, it worked fine about three months. One of host (host_1.3; ip:172.18.1.3) was created about 16 vms, but the host_1.3 was shutdown unexpectly about 2019-12-23 16:11, when re-started host and vms; half of the vms was lost some configure or changed , such as lost theirs ip etc.(the vm name is 'zzh_Chain49_ACG_M' in the vdsm.log) the vm zzh_Chain49_ACG_M was create by rest API through vm's templated . the template is 20.1.1.161; the vm zzh_Chain49_ACG_M was created by ovirt rest api and changed the ip to 20.1.1.219 by ovirt rest api. But the ip was changed to templated ip when the accident happened. the vm's os is centos. Hope get help from you soon, Thank you. Mark Sincerely.

1 0

[ovirt 4.2]vdsm host was shutdown unexpectly, configure of some vms was changed or lost when host was re-powered on and vm was started
by lifuqiong＠sunyainfo.com 23 Dec '19

23 Dec '19

Dear All: My ovirt engine managed two vdsms with nfs storage on another nfs server, it worked fine about three months. One of host (host_1.3; ip:172.18.1.3) was created about 16 vms, but the host_1.3 was shutdown unexpectly about 2019-12-23 16:11, when re-started host and vms; half of the vms was lost some configure or changed , such as lost theirs ip etc.(the vm name is 'zzh_Chain49_ACG_M' in the vdsm.log) the vm zzh_Chain49_ACG_M was create by rest API through vm's templated . the template is 20.1.1.161; the vm zzh_Chain49_ACG_M was created by ovirt rest api and changed the ip to 20.1.1.219 by ovirt rest api. But the ip was changed to templated ip when the accident happened. the vm's os is centos. Hope get help from you soon, Thank you. Mark Sincerely.

1 0

VM Import Fails
by Vijay Sachdeva 23 Dec '19

23 Dec '19

Hi All, I am trying to import a VM from export domain, but import fails. Setup: Source DC has a NFS shared storage with two Hosts Destination DC has a local storage configured using LVM. Note: Used common export domain to export the VM. Anyone, please help me on this case to understand why it’s failing. Thanks Vijay Sachdeva

2 1

iSCSI connections and storage put into maintenance
by Gianluca Cecchi 22 Dec '19

22 Dec '19

Hello, I have a cluster composed by two hosts connected to some iSCSI storage domains. I have put one host into maintenance and I see that all the iSCSI sessions are closed. So far so good. Then I put one storage domain into maintenance and I would expect to see on the active host the (2) iSCSI sessions versus this storage domain to get closed. Instead I continue to see them up and "multipath -l" command gives me the 2 paths for the LUN part of the storage domain connection. Is this expected and only when I execute detach of the SD I will get the closure of the iSCSI connections? Because the detach phase will unregister the VMs and I would like not to do it, but at the same time I will have a planned maintenance for this storage domain and I would like to clean the connections on host side, to let multipath happy when they will go down for some hours... Thanks in advance for any insight. Gianluca

2 2

Ovirt 4.4 node installation failed for yum-utils
by usurse＠redhat.com 20 Dec '19

20 Dec '19

Has anyone got this issue? Host deploy failed with the requirement of yum-utils ~~~ 2019-12-19 20:16:54 IST - TASK [ovirt-host-deploy-facts : Install yum-utils] ***************************** 2019-12-19 20:17:27 IST - fatal: [Hostname: FAILED! => {"changed": false, "failures": ["No package yum-utils available."], "msg": "Failed to install some of the specified packages", "rc": 1, "results": []} 2019-12-19 20:17:27 IST - { "status" : "OK", "msg" : "", "data" : { "event" : "runner_on_failed", "uuid" : "9793bb18-562b-407e-9149-782b3e7f2d65", "stdout" : "fatal: [Hostname]: FAILED! => {\"changed\": false, \"failures\": [\"No package yum-utils available.\"], \"msg\": \"Failed to install some of the specified packages\", \"rc\": 1, \"results\": []}", ~~~

2 2

List VMS with ansible version 2.9 ovirt_vms_info module
by csyeow＠gmail.com 19 Dec '19

19 Dec '19

Hi All is someone use ansible to list guest vm and ovirt node on ansible 2.9 with ovirt_vms_info ? i read this https://lists.ovirt.org/pipermail/users/2017-May/081956.html but this is for ansible 2.8 and below with ovirt_vms_facts module. i new to ansible, hope someone manage to provide some sample playbook to start list vm on ovirt

3 5

[NEED YOUR INPUT] | Server Virtualization Trends 2019 - Statistical Survey | oVirt users
by Paweł Mączka 19 Dec '19

19 Dec '19

Hello everybody, Have you even wonder about worldwide virtualization market, who are undisputed leader and who is chasing the market from a back seat? I’m personally decided with my team (Storware) to work on data research on Server Virtualization Trends 2019. I'm contacting you because I believe, that you are using VM environment (for sure oVirt 😊) and you could give a value to our community. Your vote is crucial. My goal is to reach 1000 specialist (now we have 250 votes) on the market and convince them to vote 😊. I will be pleased if you agree to take part in this short survey (link below). The answer to the following questions will take you no more than 1 minute. Thank you in advance for your time, respondents' data will not be made public. I will inform you when the report is ready. http://bit.ly/VMTrends2019 Thanks In advance! Pozdrawiam/Best Regards Paweł Mączka Chief Technology Officer, VP +48 730 602 659 e-mail: p.maczka(a)storware.eu<mailto:p.maczka@storware.eu> Storware Sp. z o.o. ul. Leszno 8/44 01-192 Warszawa www.storware.eu<http://www.storware.eu/> [backupmoster (1)] Want to talk? Book a one-to-one session with me on https://calendly.com/p-maczka/30min/

1 0

Re: Strange disk behaviour within VMs
by Alessio Lustri 19 Dec '19

19 Dec '19

Hello, did you ever solved your issue? I’m experiencing the exact same thing!

1 0

vdsmd 4.4.0 throws an exception in asyncore.py while updating OVF data
by alexandermurashkin＠msn.com 19 Dec '19

19 Dec '19

vdsmd 4.4.0 throws an exception in asyncore.py while updating OVF data. First exception occured on December 14th and happens every hour since then. An hour before the first exception, we updated engine's and host's rpms packages. We did not run engine-backup so we cannot restore oVirt database if it got corrupted. Currently, both engine (RHEL 7.7) and host (RHEL 8.1) have the most recent packages (and were rebooted). Neither the updates, nor reboots resolve the issue. In general, oVirt is working. We can run virtual machines with disks in the storage domain. We can create new virtual machines. But some functionality is not available, for example, we cannot move disks between domains. The storage domain in question is on NFS. After the issue appeared, we successfully created Glusterfs domain but its OVF update also failed. Interesting that it was possible to move disks to it before OVF failure. Here is the vdsmd traceback for your convinience 2019-12-17 16:36:58,393-0600 ERROR (Reactor thread) [vds.dispatcher] uncaptured python exception, closing channel <yajsonrpc.betterAsyncore.Dispatcher connected ('::ffff:172.20.1.142', 38002, 0, 0) at 0x7fbda865ed30> ( <class 'TypeError'>:object of type 'NoneType' has no len() [/usr/lib64/python3.6/asyncore.py|readwrite|108] [/usr/lib64/python3.6/asyncore.py|handle_read_event|423] [/usr/lib/python3.6/site-packages/yajsonrpc/betterAsyncore.py|handle_read|71] [/usr/lib/python3.6/site-packages/yajsonrpc/betterAsyncore.py|_delegate_call|168] [/usr/lib/python3.6/site-packages/vdsm/protocoldetector.py|handle_read|115] ) (betterAsyncore:179) I appreciate your help, Alexander Murashkin ------ Current Versions (that have the exception) ------ Engine ovirt-engine-4.4.0-0.0.master.20191204120550.git04d5d05.el7.noarch Host vdsm-4.40.0-1363.gitf6a1ba0a0.el8.x86_64 vdsm-python-4.40.0-1363.gitf6a1ba0a0.el8.noarch vdsm-yajsonrpc-4.40.0-1363.gitf6a1ba0a0.el8.noarch python3-libs-3.6.8-15.1.el8.x86_64 ------ December 13th Versions (that did not have the exception) ------ Engine ovirt-engine-4.4.0-0.0.master.20191204120550.git04d5d05.el7.noarch --- not sure, but probably the same as now Host vdsm-4.40.0-1360.git821afbbc2.el8.x86_64 vdsm-python-4.40.0-1360.git821afbbc2.el8.noarch vdsm-yajsonrpc-4.40.0-1360.git821afbbc2.el8.noarch python3-libs-3.6.8-15.1.el8.x86_64 ------ vdsm.log ------ 2019-12-17 16:36:58,393-0600 ERROR (Reactor thread) [vds.dispatcher] uncaptured python exception, closing channel <yajsonrpc.betterAsyncore.Dispatcher connected ('::ffff:172.20.1.142', 38002, 0, 0) at 0x7fbda865ed30> (<class 'TypeError'>:object of type 'NoneType' has no len() [/usr/lib64/python3.6/asyncore.py|readwrite|108] [/usr/lib64/python3.6/asyncore.py|handle_read_event|423] [/usr/lib/python3.6/site-packages/yajsonrpc/betterAsyncore.py|handle_read|71] [/usr/lib/python3.6/site-packages/yajsonrpc/betterAsyncore.py|_delegate_call|168] [/usr/lib/python3.6/site-packages/vdsm/protocoldetector.py|handle_read|115]) (betterAsyncore:179) ----- engine.log ------ 2019-12-17 16:36:58,395-06 ERROR [org.ovirt.engine.core.bll.storage.ovfstore.UploadStreamCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-95) [6c787bf3] Command 'org.ovirt.engine.core.bll.storage.ovfstore.UploadStreamCommand' failed: EngineException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketException: Connection reset (Failed with error VDS_NETWORK_ERROR and code 5022) ------ Web Interface Events ------ Storage Domain Events Dec 17, 2019, 4:36:58 PM Failed to update VMs/Templates OVF data for Storage Domain storedom3 in Data Center Default. 36b41d9c oVirt Dec 17, 2019, 4:36:58 PM Failed to update OVF disks fcc661df-b2e3-4625-be40-52b65033c6d7, OVF data isn't updated on those OVF stores (Data Center Default, Storage Domain storedom3). 36b41d9c oVirt Host Events Dec 17, 2019, 4:38:20 PM Status of host poplar was set to Up. 399a2181 oVirt Dec 17, 2019, 4:36:58 PM Host poplar is not responding. Host cannot be fenced automatically because power management for the host is disabled. oVirt

2 2

Re: Self hosted engine to hci
by Strahil 18 Dec '19

18 Dec '19

I have never done this , so you can simulate it on VMs before doint it : 1. Add the new host and put it in maintenance 2. You can backup your Hosted Engine VM and any critical VMs(just incase) 3. Create a new Gluster volume (either replica 3 or replica 2 arbiter 1) 4. Add the new volume via the UI as new storage 5. Do a storage migration from the distributed to the replica 3 volume. 6. Get rid of the old distributed volume and use it for something useful. 7. Remove the new hosts' maintenance and test live migration 8. Set global maintenance (via hosted-engine command) and poweroff the engine VM. 9. Manually power it up on a new host. 10. Remove global maintenance and shutdown a VM 11. Change the same VM to power up only on specific host and power it up. 12. Remove the limitation of that test VM (from step 10 & 11) And you are ready to go. Best Regards, Strahil Nikolov On Dec 18, 2019 16:39, Ernest Clyde Chua <ernestclydeachua(a)gmail.com> wrote: > > Hello currently i have a server running glusterfs on distributed 1 and a self hosted engine. > And planning to add two servers for hci. > > Do i add new hosts and change gluster volume type to replicated or backup all the vms then start from scratch? > > Is there any recommendation on this? > >

1 0

Self hosted engine to hci
by Ernest Clyde Chua 18 Dec '19

18 Dec '19

Hello currently i have a server running glusterfs on distributed 1 and a self hosted engine. And planning to add two servers for hci. Do i add new hosts and change gluster volume type to replicated or backup all the vms then start from scratch? Is there any recommendation on this?

1 0

NMA nodes
by suporte＠logicworks.pt 18 Dec '19

18 Dec '19

Hi, My host only shows one NUMA node. It means that I cannot setup a high performence VM? Thanks -- Jose Ferradeira http://www.logicworks.pt

2 3

ovirt hosted-engine on iSCSI offering one target
by wodel youchi 18 Dec '19

18 Dec '19

Hi, We have an oVirt Platforme using the 4.1 version. when the platforme was installed, it was made of : - Two HP Proliant DL380 G9 as hypervisors - One HP MSA1040 for iSCSI - One Synology for NFS - Two switches, one for network/vm traffic, the second for storage traffic. The problem : the hosted-engine domain was created using iSCSI on the HP MSA. The problem is that this disk array does not give the possibility to create different targets, it presents just one target. At that time we create both the hosted-engine and the first data domain using the same target, and we didn't pay attention to the information saying "i*f you are using iSCSI storage, do not use the same iSCSI target for the shared storage domain and data storage domain*". Question : - what problems can be generated by this (mis-)configuration? - Is this a must to do (correct) configuration. Regards.

3 2

Updating oVirt with gluster-ansible and Gluster 7?
by Victor Hooi 18 Dec '19

18 Dec '19

Hi, I'm planning to bring up a 3-node Ovirt/Gluster cluster, to do some comparison against an existing 3-node Proxmox/Ceph cluster we have. However, after reading the documentation Firstly, the oVirt documentation <https://www.ovirt.org/documentation/gluster-hyperconverged/chap-Single_node…> mentions that it uses gDeploy to setup Gluster. However, I saw from this commit <https://github.com/gluster/gdeploy/commit/57c19a4c233cccef0ccda468f697f1564…> that gDeploy has been deprecated since August 2018, in favour of gluster-ansible <https://github.com/gluster/gluster-ansible>. Is there a plan to move to gluster-ansible? Secondly, I saw there was a recently a new major Gluster release - Gluster 7 <https://www.gluster.org/announcing-gluster-7-0/>. Are there any plans to add this to oVirt? (I saw there was some performance improvements, hoping this will help with VM hosting). Thanks, Victor

1 0

Cannot start VM 2 hosts
by Stefan Wolf 17 Dec '19

17 Dec '19

I ve got 4 Hosts after changeing the harddrive on every Host and normal updates I am not able anymore to start VM's on 2 of these 4 hosts. I am also not able to migrate a running vm to these hosts. This is the error log for start up 2019-12-17 16:01:28,326+01 INFO [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-5) [3beff980-0f1a-461b-a6da-89c6f4dea6b3] Lock Acquired to object 'EngineLock:{exclusiveLocks='[40cf6c27-6464-4fa7-bc01-9158cb03328b=VM]', sharedLocks=''}' 2019-12-17 16:01:28,609+01 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (default task-5) [3beff980-0f1a-461b-a6da-89c6f4dea6b3] START, IsVmDuringInitiatingVDSCommand( IsVmDuringInitiatingVDSCommandParameters:{vmId='40cf6c27-6464-4fa7-bc01-9158cb03328b'}), log id: 5706d380 2019-12-17 16:01:28,611+01 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (default task-5) [3beff980-0f1a-461b-a6da-89c6f4dea6b3] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 5706d380 2019-12-17 16:01:28,854+01 INFO [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-5) [3beff980-0f1a-461b-a6da-89c6f4dea6b3] Running command: RunVmOnceCommand internal: false. Entities affected : ID: 40cf6c27-6464-4fa7-bc01-9158cb03328b Type: VMAction group RUN_VM with role type USER, ID: 40cf6c27-6464-4fa7-bc01-9158cb03328b Type: VMAction group EDIT_ADMIN_VM_PROPERTIES with role type ADMIN 2019-12-17 16:01:28,999+01 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (default task-5) [3beff980-0f1a-461b-a6da-89c6f4dea6b3] START, UpdateVmDynamicDataVDSCommand( UpdateVmDynamicDataVDSCommandParameters:{hostId='null', vmId='40cf6c27-6464-4fa7-bc01-9158cb03328b', vmDynamic='org.ovirt.engine.core.common.businessentities.VmDynamic@147620b'}), log id: 612dabdf 2019-12-17 16:01:29,097+01 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (default task-5) [3beff980-0f1a-461b-a6da-89c6f4dea6b3] FINISH, UpdateVmDynamicDataVDSCommand, return: , log id: 612dabdf 2019-12-17 16:01:29,106+01 INFO [org.ovirt.engine.core.vdsbroker.CreateVDSCommand] (default task-5) [3beff980-0f1a-461b-a6da-89c6f4dea6b3] START, CreateVDSCommand( CreateVDSCommandParameters:{hostId='d38bae4c-8494-4861-ae5a-38992db338e5', vmId='40cf6c27-6464-4fa7-bc01-9158cb03328b', vm='VM [z-push]'}), log id: 56abde11 2019-12-17 16:01:29,110+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand] (default task-5) [3beff980-0f1a-461b-a6da-89c6f4dea6b3] START, CreateBrokerVDSCommand(HostName = kvm360.durchhalten.intern, CreateVDSCommandParameters:{hostId='d38bae4c-8494-4861-ae5a-38992db338e5', vmId='40cf6c27-6464-4fa7-bc01-9158cb03328b', vm='VM [z-push]'}), log id: 36d088ab 2019-12-17 16:01:29,163+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand] (default task-5) [3beff980-0f1a-461b-a6da-89c6f4dea6b3] VM <?xml version="1.0" encoding="UTF-8"?><domain type="kvm" xmlns:ovirt-tune="http://ovirt.org/vm/tune/1.0" xmlns:ovirt-vm="http://ovirt.org/vm/1.0"> <name>z-push</name> <uuid>40cf6c27-6464-4fa7-bc01-9158cb03328b</uuid> <memory>4194304</memory> <currentMemory>4194304</currentMemory> <iothreads>1</iothreads> <maxMemory slots="16">16777216</maxMemory> <vcpu current="4">16</vcpu> <sysinfo type="smbios"> <system> <entry name="manufacturer">oVirt</entry> <entry name="product">OS-NAME:</entry> <entry name="version">OS-VERSION:</entry> <entry name="serial">HOST-SERIAL:</entry> <entry name="uuid">40cf6c27-6464-4fa7-bc01-9158cb03328b</entry> </system> </sysinfo> <clock offset="variable" adjustment="0"> <timer name="rtc" tickpolicy="catchup"/> <timer name="pit" tickpolicy="delay"/> <timer name="hpet" present="no"/> </clock> <features> <acpi/> </features> <cpu match="exact"> <model>SandyBridge</model> <feature name="pcid" policy="require"/> <feature name="spec-ctrl" policy="require"/> <feature name="ssbd" policy="require"/> <topology cores="1" threads="1" sockets="16"/> <numa> <cell id="0" cpus="0,1,2,3" memory="4194304"/> </numa> </cpu> <cputune/> <devices> <input type="mouse" bus="ps2"/> <channel type="unix"> <target type="virtio" name="ovirt-guest-agent.0"/> <source mode="bind" path="/var/lib/libvirt/qemu/channels/40cf6c27-6464-4fa7-bc01-9158cb03328b.ovirt-guest-agent.0"/> </channel> <channel type="unix"> <target type="virtio" name="org.qemu.guest_agent.0"/> <source mode="bind" path="/var/lib/libvirt/qemu/channels/40cf6c27-6464-4fa7-bc01-9158cb03328b.org.qemu.guest_agent.0"/> </channel> <rng model="virtio"> <backend model="random">/dev/urandom</backend> <alias name="ua-375334ff-0392-4314-879d-d89d81af5b69"/> </rng> <watchdog model="i6300esb" action="none"> <alias name="ua-5a4b6796-47bd-40e4-99ac-81b9546e6824"/> <address bus="0x00" domain="0x0000" function="0x0" slot="0x08" type="pci"/> </watchdog> <memballoon model="virtio"> <stats period="5"/> <alias name="ua-6d47e6f5-fd4f-452d-bcca-895b08d896f8"/> <address bus="0x00" domain="0x0000" function="0x0" slot="0x06" type="pci"/> </memballoon> <video> <model type="qxl" vram="32768" heads="1" ram="65536" vgamem="16384"/> <alias name="ua-79117281-c935-4cfc-aa21-9f028d817d79"/> <address bus="0x00" domain="0x0000" function="0x0" slot="0x02" type="pci"/> </video> <controller type="usb" model="piix3-uhci" index="0"> <address bus="0x00" domain="0x0000" function="0x2" slot="0x01" type="pci"/> </controller> <controller type="virtio-serial" index="0" ports="16"> <alias name="ua-b82337cf-7f59-4390-ad5b-4c940e205fae"/> <address bus="0x00" domain="0x0000" function="0x0" slot="0x05" type="pci"/> </controller> <graphics type="spice" port="-1" autoport="yes" passwd="*****" passwdValidTo="1970-01-01T00:00:01" tlsPort="-1"> <channel name="main" mode="secure"/> <channel name="inputs" mode="secure"/> <channel name="cursor" mode="secure"/> <channel name="playback" mode="secure"/> <channel name="record" mode="secure"/> <channel name="display" mode="secure"/> <channel name="smartcard" mode="secure"/> <channel name="usbredir" mode="secure"/> <listen type="network" network="vdsm-ovirtmgmt"/> </graphics> <channel type="spicevmc"> <target type="virtio" name="com.redhat.spice.0"/> </channel> <interface type="bridge"> <model type="virtio"/> <link state="up"/> <source bridge="ovirtmgmt"/> <driver queues="4" name="vhost"/> <alias name="ua-db1017dd-9b76-4245-85fd-7490185434ac"/> <address bus="0x00" domain="0x0000" function="0x0" slot="0x03" type="pci"/> <boot order="2"/> <mac address="56:6f:c1:d7:00:02"/> <mtu size="1500"/> <filterref filter="vdsm-no-mac-spoofing"/> <bandwidth/> </interface> <disk type="file" device="cdrom" snapshot="no"> <driver name="qemu" type="raw" error_policy="report"/> <source file="" startupPolicy="optional"> <seclabel model="dac" type="none" relabel="no"/> </source> <target dev="hdc" bus="ide"/> <readonly/> <alias name="ua-c2fa60b6-be89-487c-b01f-ed14c2d4624f"/> <address bus="1" controller="0" unit="0" type="drive" target="0"/> </disk> <disk snapshot="no" type="file" device="disk"> <target dev="sda" bus="scsi"/> <source file="/rhev/data-center/0e279f02-0c73-11e9-bc47-00163e150480/80364cc8-8afd-4482-884d-d63d5a7988df/images/38f2e3bb-687f-4f01-b769-3a5403136181/1a9627d8-181b-4790-aa47-569dd1e6e9fb"> <seclabel model="dac" type="none" relabel="no"/> </source> <driver name="qemu" io="threads" type="qcow2" error_policy="stop" cache="none"/> <alias name="ua-38f2e3bb-687f-4f01-b769-3a5403136181"/> <address bus="0" controller="0" unit="0" type="drive" target="0"/> <boot order="1"/> <serial>38f2e3bb-687f-4f01-b769-3a5403136181</serial> </disk> </devices> <pm> <suspend-to-disk enabled="no"/> <suspend-to-mem enabled="no"/> </pm> <os> <type arch="x86_64" machine="pc-i440fx-rhel7.6.0">hvm</type> <smbios mode="sysinfo"/> <bootmenu enable="yes" timeout="30000"/> </os> <metadata> <ovirt-tune:qos/> <ovirt-vm:vm> <ovirt-vm:minGuaranteedMemoryMb type="int">4096</ovirt-vm:minGuaranteedMemoryMb> <ovirt-vm:clusterVersion>4.3</ovirt-vm:clusterVersion> <ovirt-vm:custom/> <ovirt-vm:device mac_address="56:6f:c1:d7:00:02"> <ovirt-vm:custom/> </ovirt-vm:device> <ovirt-vm:device devtype="disk" name="sda"> <ovirt-vm:poolID>0e279f02-0c73-11e9-bc47-00163e150480</ovirt-vm:poolID> <ovirt-vm:volumeID>1a9627d8-181b-4790-aa47-569dd1e6e9fb</ovirt-vm:volumeID> <ovirt-vm:imageID>38f2e3bb-687f-4f01-b769-3a5403136181</ovirt-vm:imageID> <ovirt-vm:domainID>80364cc8-8afd-4482-884d-d63d5a7988df</ovirt-vm:domainID> </ovirt-vm:device> <ovirt-vm:launchPaused>false</ovirt-vm:launchPaused> <ovirt-vm:resumeBehavior>auto_resume</ovirt-vm:resumeBehavior> </ovirt-vm:vm> </metadata> </domain> 2019-12-17 16:01:29,198+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand] (default task-5) [3beff980-0f1a-461b-a6da-89c6f4dea6b3] FINISH, CreateBrokerVDSCommand, return: , log id: 36d088ab 2019-12-17 16:01:29,252+01 INFO [org.ovirt.engine.core.vdsbroker.CreateVDSCommand] (default task-5) [3beff980-0f1a-461b-a6da-89c6f4dea6b3] FINISH, CreateVDSCommand, return: WaitForLaunch, log id: 56abde11 2019-12-17 16:01:29,252+01 INFO [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-5) [3beff980-0f1a-461b-a6da-89c6f4dea6b3] Lock freed to object 'EngineLock:{exclusiveLocks='[40cf6c27-6464-4fa7-bc01-9158cb03328b=VM]', sharedLocks=''}' 2019-12-17 16:01:29,319+01 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-5) [3beff980-0f1a-461b-a6da-89c6f4dea6b3] EVENT_ID: USER_STARTED_VM(153), VM z-push was started by admin@internal-authz (Host: kvm360.durchhalten.intern). 2019-12-17 16:01:30,412+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmsStatisticsFetcher] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [] Fetched 1 VMs from VDS 'd38bae4c-8494-4861-ae5a-38992db338e5' 2019-12-17 16:01:30,620+01 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [] EVENT_ID: VM_CONSOLE_DISCONNECTED(168), User <UNKNOWN> got disconnected from VM z-push. 2019-12-17 16:01:30,766+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DumpXmlsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [] START, DumpXmlsVDSCommand(HostName = kvm360.durchhalten.intern, Params:{hostId='d38bae4c-8494-4861-ae5a-38992db338e5', vmIds='[40cf6c27-6464-4fa7-bc01-9158cb03328b]'}), log id: 4be2467a 2019-12-17 16:01:30,774+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DumpXmlsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [] FINISH, DumpXmlsVDSCommand, return: {40cf6c27-6464-4fa7-bc01-9158cb03328b=<?xml version="1.0" encoding="UTF-8"?><domain type="kvm" xmlns:ovirt-tune="http://ovirt.org/vm/tune/1.0" xmlns:ovirt-vm="http://ovirt.org/vm/1.0"><name>z-push</name><uuid>40cf6c27-6464-4fa7-bc01-9158cb03328b</uuid><memory>4194304</memory><currentMemory>4194304</currentMemory><iothreads>1</iothreads><maxMemory slots="16">16777216</maxMemory><vcpu current="4">16</vcpu><sysinfo type="smbios"><system><entry name="manufacturer">oVirt</entry><entry name="product">OS-NAME:</entry><entry name="version">OS-VERSION:</entry><entry name="serial">HOST-SERIAL:</entry><entry name="uuid">40cf6c27-6464-4fa7-bc01-9158cb03328b</entry></system></sysinfo><clock offset="variable" adjustment="0"><timer name="rtc" tickpolicy="catchup"></timer><timer name="pit" tickpolicy="delay"></timer ><timer name="hpet" present="no"></timer></clock><features><acpi></acpi></features><cpu match="exact"><model>SandyBridge</model><feature name="pcid" policy="require"></feature><feature name="spec-ctrl" policy="require"></feature><feature name="ssbd" policy="require"></feature><topology cores="1" threads="1" sockets="16"></topology><numa><cell id="0" cpus="0,1,2,3" memory="4194304"></cell></numa></cpu><cputune></cputune><devices><input type="mouse" bus="ps2"></input><channel type="unix"><target type="virtio" name="ovirt-guest-agent.0"></target><source mode="bind" path="/var/lib/libvirt/qemu/channels/40cf6c27-6464-4fa7-bc01-9158cb03328b.ovirt-guest-agent.0"></source></channel><channel type="unix"><target type="virtio" name="org.qemu.guest_agent.0"></target><source mode="bind" path="/var/lib/libvirt/qemu/channels/40cf6c27-6464-4fa7-bc01-9158cb03328b.org.qemu.guest_agent.0"></source></channel><rng model="virtio"><backend model="random">/dev/urandom</backend><alias name="ua-375334ff-0392 -4314-879d-d89d81af5b69"></alias></rng><watchdog model="i6300esb" action="none"><alias name="ua-5a4b6796-47bd-40e4-99ac-81b9546e6824"></alias><address bus="0x00" domain="0x0000" function="0x0" slot="0x08" type="pci"></address></watchdog><memballoon model="virtio"><stats period="5"></stats><alias name="ua-6d47e6f5-fd4f-452d-bcca-895b08d896f8"></alias><address bus="0x00" domain="0x0000" function="0x0" slot="0x06" type="pci"></address></memballoon><video><model type="qxl" vram="32768" heads="1" ram="65536" vgamem="16384"></model><alias name="ua-79117281-c935-4cfc-aa21-9f028d817d79"></alias><address bus="0x00" domain="0x0000" function="0x0" slot="0x02" type="pci"></address></video><controller type="usb" model="piix3-uhci" index="0"><address bus="0x00" domain="0x0000" function="0x2" slot="0x01" type="pci"></address></controller><controller type="virtio-serial" index="0" ports="16"><alias name="ua-b82337cf-7f59-4390-ad5b-4c940e205fae"></alias><address bus="0x00" domain="0x0000" function=" 0x0" slot="0x05" type="pci"></address></controller><graphics type="spice" port="-1" autoport="yes" passwd="*****" passwdValidTo="1970-01-01T00:00:01" tlsPort="-1"><channel name="main" mode="secure"></channel><channel name="inputs" mode="secure"></channel><channel name="cursor" mode="secure"></channel><channel name="playback" mode="secure"></channel><channel name="record" mode="secure"></channel><channel name="display" mode="secure"></channel><channel name="smartcard" mode="secure"></channel><channel name="usbredir" mode="secure"></channel><listen type="network" network="vdsm-ovirtmgmt"></listen></graphics><channel type="spicevmc"><target type="virtio" name="com.redhat.spice.0"></target></channel><interface type="bridge"><model type="virtio"></model><link state="up"></link><source bridge="ovirtmgmt"></source><driver queues="4" name="vhost"></driver><alias name="ua-db1017dd-9b76-4245-85fd-7490185434ac"></alias><address bus="0x00" domain="0x0000" function="0x0" slot="0x03" type="pci">< /address><boot order="2"></boot><mac address="56:6f:c1:d7:00:02"></mac><mtu size="1500"></mtu><filterref filter="vdsm-no-mac-spoofing"></filterref><bandwidth></bandwidth></interface><disk type="file" device="cdrom" snapshot="no"><driver name="qemu" type="raw" error_policy="report"></driver><source file="" startupPolicy="optional"><seclabel model="dac" type="none" relabel="no"></seclabel></source><target dev="hdc" bus="ide"></target><readonly></readonly><alias name="ua-c2fa60b6-be89-487c-b01f-ed14c2d4624f"></alias><address bus="1" controller="0" unit="0" type="drive" target="0"></address></disk><disk snapshot="no" type="file" device="disk"><target dev="sda" bus="scsi"></target><source file="/rhev/data-center/0e279f02-0c73-11e9-bc47-00163e150480/80364cc8-8afd-4482-884d-d63d5a7988df/images/38f2e3bb-687f-4f01-b769-3a5403136181/1a9627d8-181b-4790-aa47-569dd1e6e9fb"><seclabel model="dac" type="none" relabel="no"></seclabel></source><driver name="qemu" io="threads" type="qcow2" error_polic y="stop" cache="none"></driver><alias name="ua-38f2e3bb-687f-4f01-b769-3a5403136181"></alias><address bus="0" controller="0" unit="0" type="drive" target="0"></address><boot order="1"></boot><serial>38f2e3bb-687f-4f01-b769-3a5403136181</serial></disk></devices><pm><suspend-to-disk enabled="no"></suspend-to-disk><suspend-to-mem enabled="no"></suspend-to-mem></pm><os><type arch="x86_64" machine="pc-i440fx-rhel7.6.0">hvm</type><smbios mode="sysinfo"></smbios><bootmenu enable="yes" timeout="30000"></bootmenu></os><metadata><ovirt-tune:qos></ovirt-tune:qos><ovirt-vm:vm><ovirt-vm:minGuaranteedMemoryMb type="int">4096</ovirt-vm:minGuaranteedMemoryMb><ovirt-vm:clusterVersion>4.3</ovirt-vm:clusterVersion><ovirt-vm:custom></ovirt-vm:custom><ovirt-vm:device mac_address="56:6f:c1:d7:00:02"><ovirt-vm:custom></ovirt-vm:custom></ovirt-vm:device><ovirt-vm:device devtype="disk" name="sda"><ovirt-vm:poolID>0e279f02-0c73-11e9-bc47-00163e150480</ovirt-vm:poolID><ovirt-vm:volumeID>1a9627d8-181b-4790-aa4 7-569dd1e6e9fb</ovirt-vm:volumeID><ovirt-vm:imageID>38f2e3bb-687f-4f01-b769-3a5403136181</ovirt-vm:imageID><ovirt-vm:domainID>80364cc8-8afd-4482-884d-d63d5a7988df</ovirt-vm:domainID></ovirt-vm:device><ovirt-vm:launchPaused>false</ovirt-vm:launchPaused><ovirt-vm:resumeBehavior>auto_resume</ovirt-vm:resumeBehavior></ovirt-vm:vm></metadata></domain>}, log id: 4be2467a 2019-12-17 16:01:30,918+01 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.VmDevicesMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [] VM '40cf6c27-6464-4fa7-bc01-9158cb03328b' managed non pluggable device was removed unexpectedly from libvirt: 'VmDevice:{id='VmDeviceId:{deviceId='5f17a444-36e8-4854-a51b-f355aab2d8a8', vmId='40cf6c27-6464-4fa7-bc01-9158cb03328b'}', device='virtio-scsi', type='CONTROLLER', specParams='[ioThreadId=1]', address='', managed='true', plugged='false', readOnly='false', deviceAlias='', customProperties='[]', snapshotId='null', logicalName='null', hostDevice='null'}' 2019-12-17 16:01:32,762+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-12) [] VM '40cf6c27-6464-4fa7-bc01-9158cb03328b' was reported as Down on VDS 'd38bae4c-8494-4861-ae5a-38992db338e5'(kvm360.durchhalten.intern) 2019-12-17 16:01:32,764+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (ForkJoinPool-1-worker-12) [] START, DestroyVDSCommand(HostName = kvm360.durchhalten.intern, DestroyVmVDSCommandParameters:{hostId='d38bae4c-8494-4861-ae5a-38992db338e5', vmId='40cf6c27-6464-4fa7-bc01-9158cb03328b', secondsToWait='0', gracefully='false', reason='', ignoreNoVm='true'}), log id: 3478fcba 2019-12-17 16:01:33,283+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (ForkJoinPool-1-worker-12) [] FINISH, DestroyVDSCommand, return: , log id: 3478fcba 2019-12-17 16:01:33,283+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-12) [] VM '40cf6c27-6464-4fa7-bc01-9158cb03328b'(z-push) moved from 'WaitForLaunch' --> 'Down' 2019-12-17 16:01:33,510+01 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ForkJoinPool-1-worker-12) [] EVENT_ID: VM_DOWN_ERROR(119), VM z-push is down with error. Exit message: The name org.fedoraproject.FirewallD1 was not provided by any .service files. 2019-12-17 16:01:33,512+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-12) [] add VM '40cf6c27-6464-4fa7-bc01-9158cb03328b'(z-push) to rerun treatment 2019-12-17 16:01:33,743+01 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.VmsMonitoring] (ForkJoinPool-1-worker-12) [] Rerun VM '40cf6c27-6464-4fa7-bc01-9158cb03328b'. Called from VDS 'kvm360.durchhalten.intern' 2019-12-17 16:01:33,877+01 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-753) [] EVENT_ID: USER_INITIATED_RUN_VM_FAILED(151), Failed to run VM z-push on Host kvm360.durchhalten.intern. 2019-12-17 16:01:34,057+01 INFO [org.ovirt.engine.core.bll.RunVmOnceCommand] (EE-ManagedThreadFactory-engine-Thread-753) [] Lock Acquired to object 'EngineLock:{exclusiveLocks='[40cf6c27-6464-4fa7-bc01-9158cb03328b=VM]', sharedLocks=''}' 2019-12-17 16:01:34,085+01 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (EE-ManagedThreadFactory-engine-Thread-753) [] START, IsVmDuringInitiatingVDSCommand( IsVmDuringInitiatingVDSCommandParameters:{vmId='40cf6c27-6464-4fa7-bc01-9158cb03328b'}), log id: 4074b804 2019-12-17 16:01:34,086+01 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (EE-ManagedThreadFactory-engine-Thread-753) [] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 4074b804 2019-12-17 16:01:34,134+01 WARN [org.ovirt.engine.core.bll.RunVmOnceCommand] (EE-ManagedThreadFactory-engine-Thread-753) [] Validation of action 'RunVmOnce' failed for user admin@internal-authz. Reasons: VAR__ACTION__RUN,VAR__TYPE__VM,VAR__ACTION__RUN,VAR__TYPE__VM,SCHEDULING_NO_HOSTS 2019-12-17 16:01:34,135+01 INFO [org.ovirt.engine.core.bll.RunVmOnceCommand] (EE-ManagedThreadFactory-engine-Thread-753) [] Lock freed to object 'EngineLock:{exclusiveLocks='[40cf6c27-6464-4fa7-bc01-9158cb03328b=VM]', sharedLocks=''}' 2019-12-17 16:01:34,300+01 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-753) [] EVENT_ID: USER_FAILED_RUN_VM(54), Failed to run VM z-push (User: admin@internal-authz). 2019-12-17 16:01:34,314+01 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (EE-ManagedThreadFactory-engine-Thread-754) [2ecacb63] Running command: ProcessDownVmCommand internal: true. 2019-12-17 16:01:46,128+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmsStatisticsFetcher] (EE-ManagedThreadFactory-engineScheduled-Thread-9) [] Fetched 0 VMs from VDS 'd38bae4c-8494-4861-ae5a-38992db338e5' and this for migration 2019-12-17 16:05:42,411+01 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (default task-12) [ec2a5fc9-20e5-45f8-a38b-2a5dc96dadfe] Lock Acquired to object 'EngineLock:{exclusiveLocks='[46755b18-ac33-4e62-b1f0-a746bfeed59b=VM]', sharedLocks=''}' 2019-12-17 16:05:42,685+01 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (default task-12) [ec2a5fc9-20e5-45f8-a38b-2a5dc96dadfe] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: 46755b18-ac33-4e62-b1f0-a746bfeed59b Type: VMAction group MIGRATE_VM with role type USER 2019-12-17 16:05:43,043+01 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (default task-12) [ec2a5fc9-20e5-45f8-a38b-2a5dc96dadfe] START, MigrateVDSCommand( MigrateVDSCommandParameters:{hostId='7895de21-d174-471f-a0e5-340d59be1bb0', vmId='46755b18-ac33-4e62-b1f0-a746bfeed59b', srcHost='kvm380.durchhalten.intern', dstVdsId='d104ef2f-98a4-4a99-b28d-2725294f8626', dstHost='kvm320.durchhalten.intern:54321', migrationMethod='ONLINE', tunnelMigration='false', migrationDowntime='0', autoConverge='true', migrateCompressed='false', consoleAddress='null', maxBandwidth='62', enableGuestEvents='true', maxIncomingMigrations='2', maxOutgoingMigrations='2', convergenceSchedule='[init=[{name=setDowntime, params=[100]}], stalling=[{limit=1, action={name=setDowntime, params=[150]}}, {limit=2, action={name=setDowntime, params=[200]}}, {limit=3, action={name=setDowntime, params=[300]}}, {limit=4, action={name=setDowntime, params=[400]}}, {limit=6, action={name=setDowntime, params=[500]}}, {li mit=-1, action={name=abort, params=[]}}]]', dstQemu='192.168.200.231'}), log id: 57fb24a3 2019-12-17 16:05:43,048+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (default task-12) [ec2a5fc9-20e5-45f8-a38b-2a5dc96dadfe] START, MigrateBrokerVDSCommand(HostName = kvm380.durchhalten.intern, MigrateVDSCommandParameters:{hostId='7895de21-d174-471f-a0e5-340d59be1bb0', vmId='46755b18-ac33-4e62-b1f0-a746bfeed59b', srcHost='kvm380.durchhalten.intern', dstVdsId='d104ef2f-98a4-4a99-b28d-2725294f8626', dstHost='kvm320.durchhalten.intern:54321', migrationMethod='ONLINE', tunnelMigration='false', migrationDowntime='0', autoConverge='true', migrateCompressed='false', consoleAddress='null', maxBandwidth='62', enableGuestEvents='true', maxIncomingMigrations='2', maxOutgoingMigrations='2', convergenceSchedule='[init=[{name=setDowntime, params=[100]}], stalling=[{limit=1, action={name=setDowntime, params=[150]}}, {limit=2, action={name=setDowntime, params=[200]}}, {limit=3, action={name=setDowntime, params=[300]}}, {limit=4, action={name=setDowntime, params=[400] }}, {limit=6, action={name=setDowntime, params=[500]}}, {limit=-1, action={name=abort, params=[]}}]]', dstQemu='192.168.200.231'}), log id: 6765eb33 2019-12-17 16:05:44,068+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (default task-12) [ec2a5fc9-20e5-45f8-a38b-2a5dc96dadfe] FINISH, MigrateBrokerVDSCommand, return: , log id: 6765eb33 2019-12-17 16:05:44,126+01 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (default task-12) [ec2a5fc9-20e5-45f8-a38b-2a5dc96dadfe] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 57fb24a3 2019-12-17 16:05:44,205+01 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-12) [ec2a5fc9-20e5-45f8-a38b-2a5dc96dadfe] EVENT_ID: VM_MIGRATION_START(62), Migration started (VM: TestVM, Source: kvm380.durchhalten.intern, Destination: kvm320.durchhalten.intern, User: admin@internal-authz). 2019-12-17 16:05:44,588+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-2) [] VM '46755b18-ac33-4e62-b1f0-a746bfeed59b' was reported as Down on VDS 'd104ef2f-98a4-4a99-b28d-2725294f8626'(kvm320.durchhalten.intern) 2019-12-17 16:05:44,591+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (ForkJoinPool-1-worker-2) [] START, DestroyVDSCommand(HostName = kvm320.durchhalten.intern, DestroyVmVDSCommandParameters:{hostId='d104ef2f-98a4-4a99-b28d-2725294f8626', vmId='46755b18-ac33-4e62-b1f0-a746bfeed59b', secondsToWait='0', gracefully='false', reason='', ignoreNoVm='true'}), log id: 1afbc395 2019-12-17 16:05:44,853+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (ForkJoinPool-1-worker-2) [] Failed to destroy VM '46755b18-ac33-4e62-b1f0-a746bfeed59b' because VM does not exist, ignoring 2019-12-17 16:05:44,854+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (ForkJoinPool-1-worker-2) [] FINISH, DestroyVDSCommand, return: , log id: 1afbc395 2019-12-17 16:05:44,854+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-2) [] VM '46755b18-ac33-4e62-b1f0-a746bfeed59b'(TestVM) was unexpectedly detected as 'Down' on VDS 'd104ef2f-98a4-4a99-b28d-2725294f8626'(kvm320.durchhalten.intern) (expected on '7895de21-d174-471f-a0e5-340d59be1bb0') 2019-12-17 16:05:44,854+01 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-2) [] Migration of VM 'TestVM' to host 'kvm320.durchhalten.intern' failed: VM destroyed during the startup. 2019-12-17 16:05:44,861+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-9) [] VM '46755b18-ac33-4e62-b1f0-a746bfeed59b'(TestVM) moved from 'MigratingFrom' --> 'Up' 2019-12-17 16:05:44,862+01 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-9) [] Adding VM '46755b18-ac33-4e62-b1f0-a746bfeed59b'(TestVM) to re-run list 2019-12-17 16:05:44,927+01 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.VmsMonitoring] (ForkJoinPool-1-worker-9) [] Rerun VM '46755b18-ac33-4e62-b1f0-a746bfeed59b'. Called from VDS 'kvm380.durchhalten.intern' 2019-12-17 16:05:44,944+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-879) [] START, MigrateStatusVDSCommand(HostName = kvm380.durchhalten.intern, MigrateStatusVDSCommandParameters:{hostId='7895de21-d174-471f-a0e5-340d59be1bb0', vmId='46755b18-ac33-4e62-b1f0-a746bfeed59b'}), log id: 127dcc70 2019-12-17 16:05:44,952+01 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-879) [] FINISH, MigrateStatusVDSCommand, return: , log id: 127dcc70 2019-12-17 16:05:45,228+01 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-879) [] EVENT_ID: VM_MIGRATION_TO_SERVER_FAILED(120), Migration failed (VM: TestVM, Source: kvm380.durchhalten.intern, Destination: kvm320.durchhalten.intern). 2019-12-17 16:05:45,329+01 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (EE-ManagedThreadFactory-engine-Thread-879) [] Lock freed to object 'EngineLock:{exclusiveLocks='[46755b18-ac33-4e62-b1f0-a746bfeed59b=VM]', sharedLocks=''}' maybe someone can give me a hint thx shb

1 1

Re: Ovirt OVN help needed
by Strahil 16 Dec '19

16 Dec '19

Hi Dominik, All, I've checked 'https://lists.ovirt.org/archives/list/users@ovirt.org/thread/W6U4XJHNMYMD3W…' and the user managed to clear up and start over. I have removed the ovn-external-provider from UI, but I forgot to copy the data from the fields. Do you know any refference guide (or any tips & tricks) for adding OVN ? Thanks in advance. Best Regards, Strahil NikolovOn Dec 12, 2019 20:49, Strahil <hunter86_bg(a)yahoo.com> wrote: > > Hi Dominik, > > Thanks for the reply. > > Sadly the openstack module is missing on the engine and I have to figure it out. > > Can't I just undeploy the ovn and then redeploy it back ? > > Best Regards, > Strahil Nikolov > > On Dec 12, 2019 09:32, Dominik Holler <dholler(a)redhat.com> wrote: >> >> The cleanest way to clean up is to remove all entities on the OpenStack Network API on ovirt-provider-ovn, e.g. by something like >> https://gist.github.com/dominikholler/19bcdc5f14f42ab5f069086fd2ff5e37#file… >> This should work, if not, please report a bug. >> >> To bypass the ovirt-provider-ovn, which is not recommended and might end in an inconsistent state, you could use ovn-nbctl . >> >> >> >> On Thu, Dec 12, 2019 at 3:33 AM Strahil Nikolov <hunter86_bg(a)yahoo.com> wrote: >>> >>> Hi Community, >>> >>> can someone hint me how to get rid of some ports? I just want to 'reset' my ovn setup. >>> >>> Here is what I have so far: >>> >>> [root@ovirt1 openvswitch]# ovs-vsctl list interface >>> _uuid : be89c214-10e4-4a97-a9eb-1b82bc433a24 >>> admin_state : up >>> bfd : {} >>> bfd_status : {} >>> cfm_fault : [] >>> cfm_fault_status : [] >>> cfm_flap_count : [] >>> cfm_health : [] >>> cfm_mpid : [] >>> cfm_remote_mpids : [] >>> cfm_remote_opstate : [] >>> duplex : [] >>> error : [] >>> external_ids : {} >>> ifindex : 35 >>> ingress_policing_burst: 0 >>> ingress_policing_rate: 0 >>> lacp_current : [] >>> link_resets : 0 >>> link_speed : [] >>> link_state : up >>> lldp : {} >>> mac : [] >>> mac_in_use : "7a:7d:1d:a7:43:1d" >>> mtu : [] >>> mtu_request : [] >>> name : "ovn-25cc77-0" >>> ofport : 6 >>> ofport_request : [] >>> options : {csum="true", key=flow, remote_ip="192.168.1.64"} >>> other_config : {} >>> statistics : {rx_bytes=0, rx_packets=0, tx_bytes=0, tx_packets=0} >>> status : {tunnel_egress_iface=ovirtmgmt, tunnel_egress_iface_carrier=up} >>> type : geneve >>> >>> _uuid : ec6a6688-e5d6-4346-ac47-ece1b8379440 >>> admin_state : down >>> bfd : {} >>> bfd_status : {} >>> cfm_fault : [] >>> cfm_fault_status : [] >>> cfm_flap_count : [] >>> cfm_health : [] >>> cfm_mpid : [] >>> cfm_remote_mpids : [] >>> cfm_remote_opstate : [] >>> duplex : [] >>> error : [] >>> external_ids : {} >>> ifindex : 13 >>> ingress_policing_burst: 0 >>> ingress_policing_rate: 0 >>> lacp_current : [] >>> link_resets : 0 >>> link_speed : [] >>> link_state : down >>> lldp : {} >>> mac : [] >>> mac_in_use : "66:36:dd:63:dc:48" >>> mtu : 1500 >>> mtu_request : [] >>> name : br-int >>> ofport : 65534 >>> ofport_request : [] >>> options : {} >>> other_config : {} >>> statistics : {collisions=0, rx_bytes=0, rx_crc_err=0, rx_dropped=0, rx_errors=0, rx_frame_err=0, rx_over_err=0, rx_packets=0, tx_bytes=0, tx_dropped=0, tx_errors=0, tx_packets=0} >>> status : {driver_name=openvswitch} >>> type : internal >>> >>> _uuid : 1e511b4d-f7c2-499f-bd8c-07236e7bb7af >>> admin_state : up >>> bfd : {} >>> bfd_status : {} >>> cfm_fault : [] >>> cfm_fault_status : [] >>> cfm_flap_count : [] >>> cfm_health : [] >>> cfm_mpid : [] >>> cfm_remote_mpids : [] >>> cfm_remote_opstate : [] >>> duplex : [] >>> error : [] >>> external_ids : {} >>> ifindex : 35 >>> ingress_policing_burst: 0 >>> ingress_policing_rate: 0 >>> lacp_current : [] >>> link_resets : 0 >>> link_speed : [] >>> link_state : up >>> lldp : {} >>> mac : [] >>> mac_in_use : "1a:85:d1:d9:e2:a5" >>> mtu : [] >>> mtu_request : [] >>> name : "ovn-566849-0" >>> ofport : 5 >>> ofport_request : [] >>> options : {csum="true", key=flow, remote_ip="192.168.1.41"} >>> other_config : {} >>> statistics : {rx_bytes=0, rx_packets=0, tx_bytes=0, tx_packets=0} >>> status : {tunnel_egress_iface=ovirtmgmt, tunnel_egress_iface_carrier=up} >>> type : geneve >>> >>> >>> When I try to remove a port - it never ends (just hanging): >>> >>> [root@ovirt1 openvswitch]# ovs-vsctl --dry-run del-port br-int ovn-25cc77-0 >>> In journal I see only this: >>> дек 12 04:13:57 ovirt1.localdomain ovs-vsctl[22030]: ovs|00001|vsctl|INFO|Called as ovs-vsctl --dry-run del-port br-int ovn-25cc77-0 >>> >>> The stranger part to me is the log output: >>> >>> [root@ovirt1 openvswitch]# grep ovn-25cc77-0 /var/log/openvswitch/*.log >>> /var/log/openvswitch/ovs-vswitchd.log:2019-12-12T01:26:28.642Z|00032|bridge|INFO|bridge br-int: added interface ovn-25cc77-0 on port 14 >>> /var/log/openvswitch/ovs-vswitchd.log:2019-12-12T01:45:15.646Z|00113|bridge|INFO|bridge br-int: deleted interface ovn-25cc77-0 on port 14 >>> /var/log/openvswitch/ovs-vswitchd.log:2019-12-12T01:45:15.861Z|00116|bridge|INFO|bridge br-int: added interface ovn-25cc77-0 on port 2 >>> /var/log/openvswitch/ovs-vswitchd.log:2019-12-12T01:50:36.678Z|00118|bridge|INFO|bridge br-int: deleted interface ovn-25cc77-0 on port 2 >>> /var/log/openvswitch/ovs-vswitchd.log:2019-12-12T01:52:31.180Z|00121|bridge|INFO|bridge br-int: added interface ovn-25cc77-0 on port 3 >>> /var/log/openvswitch/ovs-vswitchd.log:2019-12-12T01:55:09.734Z|00125|bridge|INFO|bridge br-int: deleted interface ovn-25cc77-0 on port 3 >>> /var/log/openvswitch/ovs-vswitchd.log:2019-12-12T01:58:15.138Z|00127|bridge|INFO|bridge br-int: added interface ovn-25cc77-0 on port 6 >>> >>> I'm also attaching the verbose output of the dryrun. >>> >>> Thanks in advance. >>> >>> Best Regards, >>> Strahil Nikolov >>> _______________________________________________ >>> Users mailing list -- users(a)ovirt.org >>> To unsubscribe send an email to users-leave(a)ovirt.org >>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/IG7YJPINVAF2B…

3 3

Cannot forward traffic through VXLAN
by k.betsis＠gmail.com 16 Dec '19

16 Dec '19

Hi all I have a VM network created with some hosts and I have included a vyos router acting as a Layer 2 extension to another hypervisor through VXLAN. I can see traffic reaching VMs from the other hypervisor to the ovirt hosted VMs. I can see traffic leaving the VMs hosted on the ovirt hypervisor. However, i do not see return traffic reaching the vyos VXLAN hosted on ovirt. I believe the VM network drops return traffic based on the destination MAC address. However, i have created the VM Network with security disabled. Can you please assist on how to troubleshoot?

2 9

Postgres stuck on 100% after Engine upgrade from 4.2.8 to 4.3.7
by Vrgotic, Marko 16 Dec '19

16 Dec '19

Hi oVirt, Since oVirt upgrade from 4.2.8 to 4.3.7, SHE postgres is running 100% CPU with 4 processes, related to ovirt_engine_history DB: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2617 postgres 20 0 372668 30876 26352 R 100.0 0.2 63:40.19 postgres: ovirt_engine_history ovirt_engine_history 127.0.0.1(50226) INSERT 3554 postgres 20 0 368680 15120 11716 R 100.0 0.1 56:45.72 postgres: ovirt_engine_history ovirt_engine_history 127.0.0.1(50448) BIND 2623 postgres 20 0 368068 17640 14896 R 99.3 0.1 42:35.36 postgres: ovirt_engine_history ovirt_engine_history 127.0.0.1(50228) BIND 2880 postgres 20 0 368680 20404 17004 R 99.0 0.1 65:13.96 postgres: ovirt_engine_history ovirt_engine_history 127.0.0.1(50252) BIND Output of select * from pg_stat_activity; is giving me following: 16401 | ovirt_engine_history | 2617 | 16386 | ovirt_engine_history | | 127.0.0.1 | | 50226 | 2019-12-09 13:37:07.779261+01 | 2019-12-09 13:37:09.555704+01 | 2019-12-09 13:39:04.915431+01 | 2019-12-09 13:39:04.915432+01 | | | active | 260332629 | 260332629 | INSERT INTO vm_disks_usage_samples_history (history_datetime,vm_id,disks_usage) VALUES ($1,$2,$3) | client backend 16401 | ovirt_engine_history | 2623 | 16386 | ovirt_engine_history | | 127.0.0.1 | | 50228 | 2019-12-09 13:37:07.820982+01 | 2019-12-09 14:00:00.042448+01 | 2019-12-09 14:00:00.997228+01 | 2019-12-09 14:00:00.997229+01 | | | active | 260333458 | 260332629 | SELECT +| client backend | | | | | | | | | | | | | | | | | | history_datetime, +| | | | | | | | | | | | | | | | | | | vm_id, +| | | | | | | | | | | | | | | | | | | disks_usage +| | | | | | | | | | | | | | | | | | | FROM vm_disks_usage_samples_history +| | | | | | | | | | | | | | | | | | | WHERE history_datetime >= '2019-12-09 07:00:00.000000+0100' +| | | | | | | | | | | | | | | | | | | AND history_datetime < '2019-12-09 08:00:00.000000+0100' +| | | | | | | | | | | | | | | | | | | ORDER BY history_id DESC, +| | | | | | | | | | | | | | | | | | | vm_id | 16401 | ovirt_engine_history | 2624 | 16386 | ovirt_engine_history | | 127.0.0.1 | | 50230 | 2019-12-09 13:37:07.826391+01 | | | 2019-12-09 13:37:07.835886+01 | Client | ClientRead | idle | | | | client backend 16401 | ovirt_engine_history | 2625 | 16386 | ovirt_engine_history | | 127.0.0.1 | | 50232 | 2019-12-09 13:37:07.847262+01 | | | 2019-12-09 13:37:07.850961+01 | Client | ClientRead | idle | | | | client backend 13214 | postgres | 6275 | 10 | postgres | psql | | | -1 | 2019-12-09 14:37:13.890996+01 | 2019-12-09 14:46:36.439887+01 | 2019-12-09 14:46:36.439887+01 | 2019-12-09 14:46:36.439897+01 | | | active | | 260332629 | select * from pg_stat_activity; | client backend [A screenshot of a computer Description automatically generated] Any ideas why this could be happening? Kindly awaiting your reply. ----- kind regards/met vrindelijke groet Marko Vrgotic ActiveVideo

2 3

Re: Problems after 4.3.8 update
by Strahil 15 Dec '19

15 Dec '19

I didn't report it as nobody has mentioned it and I thought it's a onetime issue. I am,now, quite confident that is a bug. Are you using the gluster fuse mounts ( the ones in /rhev...) or libgfapi ? Can you open a case in the bugzilla.redhat.com ? Best Regards, Strahil NikolovOn Dec 15, 2019 13:16, Jayme <jaymef(a)gmail.com> wrote: > > I compared each file across my nodes and synced them. It seems to have resolved my issue. > > I wonder if there is a problem with 6.5 to 6.6 upgrade that is causing the problem? It’s strange that it seems to have happened to more than one person. I was also following proper upgrade procedure. > > > > On Sun, Dec 15, 2019 at 3:09 AM <hunter86_bg(a)yahoo.com> wrote: >> >> I don't know. I had the same issues when I migrated my gluster from v6.5 to 6.6 (currently running v7.0). >> Just get the newest file and rsync it to the rest of the bricks. It will solve the '?????? ??????' problem. >> >> Best Regards, >> Strahil Nikolov >> В неделя, 15 декември 2019 г., 3:49:27 ч. Гринуич+2, Jayme <jaymef(a)gmail.com> написа: >> >> >> on that page it says to check open bugs and the migration bug you mention does not appear to be on the list. Has it been resolved or is it just missing from this page? >> >> On Sat, Dec 14, 2019 at 7:53 PM Strahil Nikolov <hunter86_bg(a)yahoo.com> wrote: >>> >>> Nah... this is not gonna fix your issue and is unnecessary. >>> Just compare the data from all bricks ... most probably the 'Last Updated' is different and the gfid of the file is different. >>> Find the brick that has the most fresh data, and replace (move away as a backup and rsync) the file from last good copy to the other bricks. >>> You can also run a 'full heal'. >>> >>> Best Regards, >>> Strahil Nikolov >>> >>> В събота, 14 декември 2019 г., 21:18:44 ч. Гринуич+2, Jayme <jaymef(a)gmail.com> написа: >>> >>> >>> *Update* >>> >>> Situation has improved. All VMs and engine are running. I'm left right now with about 2 heal entries in each glusterfs storage volume that will not heal. >>> >>> In all cases each heal entry is related to an OVF_STORE image and the problem appears to be an issue with the gluster metadata for those ovf_store images. When I look at the files shown in gluster volume heal info output I'm seeing question marks on the meta files which indicates an attribute/gluster problem (even though there is no split-brain). And I get input/output error when attempting to do anything with the files. >>> >>> If I look at the files on each host in /gluster_bricks they all look fine. I only see question marks on the meta files when look at the file in /rhev mounts >>> >>> Does anyone know how I can correct the attributes on these OVF_STORE files? I've tried putting each host in maintenance and re-activating to re-mount gluster volumes. I've also stopped and started all gluster volumes. >>> >>> I'm thinking I might be able to solve this by shutting down all VMs and placing all hosts in maintenance and safely restarting the entire cluster.. but that may not be necessary? >>> >>> On Fri, Dec 13, 2019 at 12:59 AM Jayme <jaymef(a)gmail.com> wrote: >>>> >>>> I believe I was able to get past this by stopping the engine volume then unmounting the glusterfs engine mount on all hosts and re-starting the volume. I was able to start hostedengine on host0. >>>> >>>> I'm still facing a few problems: >>>> >>>> 1. I'm still seeing this issue in each host's logs: >>>> >>>> Dec 13 00:57:54 orchard0 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed scanning for OVF_STORE due to Command Volume.getInfo with args {'storagepoolID': '00000000-0000-0000-0000-000000000000', 'storagedomainID': 'd70b171e-7488-4d52-8cad-bbc581dbf16e', 'volumeID': u'2632f423-ed89-43d9-93a9-36738420b866', 'imageID': u'd909dc74-5bbd-4e39-b9b5-755c167a6ee8'} failed:#012(code=201, message=Volume does not exist: (u'2632f423-ed89-43d9-93a9-36738420b866',)) >>>> Dec 13 00:57:54 orchard0 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Unable to identify the OVF_STORE volume, falling back to initial vm.conf. Please ensure you already added your first data domain for regular VMs >>>> >>>> >>>> 2. Most of my gluster volumes still have un-healed entires which I can't seem to heal. I'm not sure what the answer is here. >>>> >>>> On Fri, Dec 13, 2019 at 12:33 AM Jayme <jaymef(a)gmail.com> wrote: >>>>> >>>>> I was able to get the hosted engine started manually via Virsh after re-creating a missing symlink in /var/run/vdsm/storage -- I later shut it down and am still having the same problem with ha broker starting. It appears that the problem *might* be with a corrupt ha metadata file, although gluster is not stating there is split brain on the engine volume >>>>> >>>>> I'm seeing this: >>>>> >>>>> ls -al /rhev/data-center/mnt/glusterSD/orchard0\:_engine/d70b171e-7488-4d52-8cad-bbc581dbf16e/ha_agent/ >>>>> ls: cannot access /rhev/data-center/mnt/glusterSD/orchard0:_engine/d70b171e-7488-4d52-8cad-bbc581dbf16e/ha_agent/hosted-engine.metadata: Input/output error >>>>> total 0 >>>>> drwxr-xr-x. 2 vdsm kvm 67 Dec 13 00:30 . >>>>> drwxr-xr-x. 6 vdsm kvm 64 Aug 6 2018 .. >>>>> lrwxrwxrwx. 1 vdsm kvm 132 Dec 13 00:30 hosted-engine.lockspace -> /var/run/vdsm/storage/d70b171e-7488-4d52-8cad-bbc581dbf16e/03a8ee8e-91f5-4e06-904b-9ed92a9706eb/db2699ce-6349-4020-b52d-8ab11d01e26d >>>>> l?????????? ? ? ? ? ? hosted-engine.metadata >>>>> >>>>> Clearly showing some sort of issue with hosted-engine.metadata on the client mount. >>>>> >>>>> on each node in /gluster_bricks I see this: >>>>> >>>>> # ls -al /gluster_bricks/engine/engine/d70b171e-7488-4d52-8cad-bbc581dbf16e/ha_agent/ >>>>> total 0 >>>>> drwxr-xr-x. 2 vdsm kvm 67 Dec 13 00:31 . >>>>> drwxr-xr-x. 6 vdsm kvm 64 Aug 6 2018 .. >>>>> lrwxrwxrwx. 2 vdsm kvm 132 Dec 13 00:31 hosted-engine.lockspace -> /var/run/vdsm/storage/d70b171e-7488-4d52-8cad-bbc581dbf16e/03a8ee8e-91f5-4e06-904b-9ed92a9706eb/db2699ce-6349-4020-b52d-8ab11d01e26d >>>>> lrwxrwxrwx. 2 vdsm kvm 132 Dec 12 16:30 hosted-engine.metadata -> /var/run/vdsm/storage/d70b171e-7488-4d52-8cad-bbc581dbf16e/66bf05fa-bf50-45ec-98d8-d00002040317/a2250415-5ff0-42ab-8071-cd9d67c3048c >>>>> >>>>> ls -al /var/run/vdsm/storage/d70b171e-7488-4d52-8cad-bbc581dbf16e/66bf05fa-bf50-45ec-98d8-d00002040317/a2250415-5ff0-42ab-8071-cd9d67c3048c >>>>> -rw-rw----. 1 vdsm kvm 1073741824 Dec 12 16:48 /var/run/vdsm/storage/d70b171e-7488-4d52-8cad-bbc581dbf16e/66bf05fa-bf50-45ec-98d8-d00002040317/a2250415-5ff0-42ab-8071-cd9d67c3048c >>>>> >>>>> >>>>> I'm not sure how to proceed at this point. Do I have data corruption, a gluster split-brain issue or something else? Maybe I just need to re-generate metadata for the hosted engine? >>>>> >>>>> On Thu, Dec 12, 2019 at 6:36 PM Jayme <jaymef(a)gmail.com> wrote: >>>>>> >>>>>> I'm running a three server HCI. Up and running on 4.3.7 with no problems. Today I updated to 4.3.8. Engine upgraded fine, rebooted. First host updated fine, rebooted and let all gluster volumes heal. Put second host in maintenance, upgraded successfully, rebooted. Waited for gluster volumes to heal for over an hour but the heal process was not completing. I tried restarting gluster services as well as the host with no success. >>>>>> >>>>>> I'm in a state right now where there are pending heals on almost all of my volumes. Nothing is reporting split-brain, but the heals are not completing. >>>>>> >>>>>> All vms are still currently running except hosted engine. Hosted engine was running but on the 2nd host I upgraded I was seeing errors such as: >>>>>> >>>>>> Dec 12 16:34:39 orchard2 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed scanning for OVF_STORE due to Command Volume.getInfo with args {'storagepoolID': '00000000-0000-0000-0000-000000000000', 'storagedomainID': 'd70b171e-7488-4d52-8cad-bbc581dbf16e', 'volumeID': u'2632f423-ed89-43d9-93a9-36738420b866', 'imageID': u'd909dc74-5bbd-4e39-b9b5-755c167a6ee8'} failed:#012(code=201, message=Volume does not exist: (u'2632f423-ed89-43d9-93a9-36738420b866',)) >>>>>> >>>>>> I shut down the engine VM and attempted a manual heal on the engine volume. I cannot start the engine on any host now. I get: >>>>>> >>>>>> The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable. >>>>>> >>>>>> I'm seeing ovirt-ha-agent crashing on all three nodes: >>>>>> >>>>>> Dec 12 18:30:48 orchard0 python: detected unhandled Python exception in '/usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker' >>>>>> Dec 12 18:30:48 orchard0 abrt-server: Duplicate: core backtrace >>>>>> Dec 12 18:30:48 orchard0 abrt-server: DUP_OF_DIR: /var/tmp/abrt/Python-2019-03-14-21:02:52-44318 >>>>>> Dec 12 18:30:48 orchard0 abrt-server: Deleting problem directory Python-2019-12-12-18:30:48-23193 (dup of Python-2019-03-14-21:02:52-44318) >>>>>> Dec 12 18:30:49 orchard0 vdsm[6087]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? >>>>>> Dec 12 18:30:49 orchard0 systemd: ovirt-ha-broker.service: main process exited, code=exited, status=1/FAILURE >>>>>> Dec 12 18:30:49 orchard0 systemd: Unit ovirt-ha-broker.service entered failed state. >>>>>> Dec 12 18:30:49 orchard0 systemd: ovirt-ha-broker.service failed. >>>>>> Dec 12 18:30:49 orchard0 systemd: ovirt-ha-broker.service holdoff time over, scheduling restart. >>>>>> Dec 12 18:30:49 orchard0 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. >>>>>> Dec 12 18:30:49 orchard0 systemd: Stopped oVirt Hosted Engine High Availability Communications Broker. >>>>>> >>>>>> >>>>>> Here is what gluster volume heal info on engine looks like, it's similar on other volumes as well (although more heals pending on some of those): >>>>>> >>>>>> gluster volume heal engine info >>>>>> Brick gluster0:/gluster_bricks/engine/engine >>>>>> /d70b171e-7488-4d52-8cad-bbc581dbf16e/images/d909dc74-5bbd-4e39-b9b5-755c167a6ee8/2632f423-ed89-43d9-93a9-36738420b866.meta >>>>>> /d70b171e-7488-4d52-8cad-bbc581dbf16e/images/053171e4-f782-42d7-9115-c602beb3c826/627b8f93-5373-48bb-bd20-a308a455e082.meta >>>>>> /d70b171e-7488-4d52-8cad-bbc581dbf16e/master/tasks/a9b11e33-9b93-46a0-a36e-85063fd53ebe.backup >>>>>> /d70b171e-7488-4d52-8cad-bbc581dbf16e/dom_md/ids >>>>>> Status: Connected >>>>>> Number of entries: 4 >>>>>> >>>>>> Brick gluster1:/gluster_bricks/engine/engine >>>>>> /d70b171e-7488-4d52-8cad-bbc581dbf16e/images/d909dc74-5bbd-4e39-b9b5-755c167a6ee8/2632f423-ed89-43d9-93a9-36738420b866.meta >>>>>> /d70b171e-7488-4d52-8cad-bbc581dbf16e/master/tasks/a9b11e33-9b93-46a0-a36e-85063fd53ebe.backup >>>>>> /d70b171e-7488-4d52-8cad-bbc581dbf16e/images/053171e4-f782-42d7-9115-c602beb3c826/627b8f93-5373-48bb-bd20-a308a455e082.meta >>>>>> /d70b171e-7488-4d52-8cad-bbc581dbf16e/dom_md/ids >>>>>> Status: Connected >>>>>> Number of entries: 4 >>>>>> >>>>>> Brick gluster2:/gluster_bricks/engine/engine >>>>>> Status: Connected >>>>>> Number of entries: 0 >>>>>> >>>>>> I don't see much in vdsm.log and gluster logs look fairly normal to me, I'm not seeing any obvious errors in the gluster logs. >>>>>> >>>>>> As far as I can tell the underlying storage is fine. Why are my gluster volumes not healing and why is self-hosted engine failing to start? >>>>>> >>>>>> More agent and broker logs: >>>>>> >>>>>> ==> agent.log <== >>>>>> MainThread::ERROR::2019-12-12 18:36:09,056::hosted_engine::559::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_broker) Failed to start necessary monitors >>>>>> MainThread::ERROR::2019-12-12 18:36:09,058::agent::144::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Traceback (most recent call last): >>>>>> File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent >>>>>> return action(he) >>>>>> File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper >>>>>> return he.start_monitoring() >>>>>> File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring >>>>>> self._initialize_broker() >>>>>> File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker >>>>>> m.get('options', {})) >>>>>> File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 89, in start_monitor >>>>>> ).format(t=type, o=options, e=e) >>>>>> RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': None, 'network_test': None, 'tcp_t_port': None, 'addr': '10.11.0.254'}] >>>>>> >>>>>> MainThread::ERROR::2019-12-12 18:36:09,058::agent::145::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Trying to restart agent >>>>>> MainThread::ERROR::2019-12-12 18:36:19,619::hosted_engine::559::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_broker) Failed to start necessary monitors >>>>>> MainThread::ERROR::2019-12-12 18:36:19,619::agent::144::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Traceback (most recent call last): >>>>>> File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent >>>>>> return action(he) >>>>>> File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper >>>>>> return he.start_monitoring() >>>>>> File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring >>>>>> self._initialize_broker() >>>>>> File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker >>>>>> m.get('options', {})) >>>>>> File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 89, in start_monitor >>>>>> ).format(t=type, o=options, e=e) >>>>>> RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': None, 'network_test': None, 'tcp_t_port': None, 'addr': '10.11.0.254'}] >>>>>> >>>>>> MainThread::ERROR::2019-12-12 18:36:19,619::agent::145::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Trying to restart agent >>>>>> MainThread::ERROR::2019-12-12 18:36:30,568::hosted_engine::559::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_broker) Failed to start necessary monitors >>>>>> MainThread::ERROR::2019-12-12 18:36:30,570::agent::144::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Traceback (most recent call last): >>>>>> File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent >>>>>> return action(he) >>>>>> File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper >>>>>> return he.start_monitoring() >>>>>> File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring >>>>>> self._initialize_broker() >>>>>> File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker >>>>>> m.get('options', {})) >>>>>> File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 89, in start_monitor >>>>>> ).format(t=type, o=options, e=e) >>>>>> RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': None, 'network_test': None, 'tcp_t_port': None, 'addr': '10.11.0.254'}] >>>>>> >>>>>> MainThread::ERROR::2019-12-12 18:36:30,570::agent::145::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Trying to restart agent >>>>>> MainThread::ERROR::2019-12-12 18:36:41,581::hosted_engine::559::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_broker) Failed to start necessary monitors >>>>>> MainThread::ERROR::2019-12-12 18:36:41,583::agent::144::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Traceback (most recent call last): >>>>>> File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent >>>>>> return action(he) >>>>>> File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper >>>>>> return he.start_monitoring() >>>>>> File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring >>>>>> self._initialize_broker() >>>>>> File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker >>>>>> m.get('options', {})) >>>>>> File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 89, in start_monitor >>>>>> ).format(t=type, o=options, e=e) >>>>>> RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': None, 'network_test': None, 'tcp_t_port': None, 'addr': '10.11.0.254'}] >>>>>> >>>>>> MainThread::ERROR::2019-12-12 18:36:41,583::agent::145::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Trying to restart agent >>>>>> >>>>>> >>> _______________________________________________ >>> Users mailing list -- users(a)ovirt.org >>> To unsubscribe send an email to users-leave(a)ovirt.org >>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/U5YFDWCQJYNAL…

2 1

Problems after 4.3.8 update
by Jayme 15 Dec '19

15 Dec '19

I'm running a three server HCI. Up and running on 4.3.7 with no problems. Today I updated to 4.3.8. Engine upgraded fine, rebooted. First host updated fine, rebooted and let all gluster volumes heal. Put second host in maintenance, upgraded successfully, rebooted. Waited for gluster volumes to heal for over an hour but the heal process was not completing. I tried restarting gluster services as well as the host with no success. I'm in a state right now where there are pending heals on almost all of my volumes. Nothing is reporting split-brain, but the heals are not completing. All vms are still currently running except hosted engine. Hosted engine was running but on the 2nd host I upgraded I was seeing errors such as: Dec 12 16:34:39 orchard2 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed scanning for OVF_STORE due to Command Volume.getInfo with args {'storagepoolID': '00000000-0000-0000-0000-000000000000', 'storagedomainID': 'd70b171e-7488-4d52-8cad-bbc581dbf16e', 'volumeID': u'2632f423-ed89-43d9-93a9-36738420b866', 'imageID': u'd909dc74-5bbd-4e39-b9b5-755c167a6ee8'} failed:#012(code=201, message=Volume does not exist: (u'2632f423-ed89-43d9-93a9-36738420b866',)) I shut down the engine VM and attempted a manual heal on the engine volume. I cannot start the engine on any host now. I get: The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable. I'm seeing ovirt-ha-agent crashing on all three nodes: Dec 12 18:30:48 orchard0 python: detected unhandled Python exception in '/usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker' Dec 12 18:30:48 orchard0 abrt-server: Duplicate: core backtrace Dec 12 18:30:48 orchard0 abrt-server: DUP_OF_DIR: /var/tmp/abrt/Python-2019-03-14-21:02:52-44318 Dec 12 18:30:48 orchard0 abrt-server: Deleting problem directory Python-2019-12-12-18:30:48-23193 (dup of Python-2019-03-14-21:02:52-44318) Dec 12 18:30:49 orchard0 vdsm[6087]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Dec 12 18:30:49 orchard0 systemd: ovirt-ha-broker.service: main process exited, code=exited, status=1/FAILURE Dec 12 18:30:49 orchard0 systemd: Unit ovirt-ha-broker.service entered failed state. Dec 12 18:30:49 orchard0 systemd: ovirt-ha-broker.service failed. Dec 12 18:30:49 orchard0 systemd: ovirt-ha-broker.service holdoff time over, scheduling restart. Dec 12 18:30:49 orchard0 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Dec 12 18:30:49 orchard0 systemd: Stopped oVirt Hosted Engine High Availability Communications Broker. Here is what gluster volume heal info on engine looks like, it's similar on other volumes as well (although more heals pending on some of those): gluster volume heal engine info Brick gluster0:/gluster_bricks/engine/engine /d70b171e-7488-4d52-8cad-bbc581dbf16e/images/d909dc74-5bbd-4e39-b9b5-755c167a6ee8/2632f423-ed89-43d9-93a9-36738420b866.meta /d70b171e-7488-4d52-8cad-bbc581dbf16e/images/053171e4-f782-42d7-9115-c602beb3c826/627b8f93-5373-48bb-bd20-a308a455e082.meta /d70b171e-7488-4d52-8cad-bbc581dbf16e/master/tasks/a9b11e33-9b93-46a0-a36e-85063fd53ebe.backup /d70b171e-7488-4d52-8cad-bbc581dbf16e/dom_md/ids Status: Connected Number of entries: 4 Brick gluster1:/gluster_bricks/engine/engine /d70b171e-7488-4d52-8cad-bbc581dbf16e/images/d909dc74-5bbd-4e39-b9b5-755c167a6ee8/2632f423-ed89-43d9-93a9-36738420b866.meta /d70b171e-7488-4d52-8cad-bbc581dbf16e/master/tasks/a9b11e33-9b93-46a0-a36e-85063fd53ebe.backup /d70b171e-7488-4d52-8cad-bbc581dbf16e/images/053171e4-f782-42d7-9115-c602beb3c826/627b8f93-5373-48bb-bd20-a308a455e082.meta /d70b171e-7488-4d52-8cad-bbc581dbf16e/dom_md/ids Status: Connected Number of entries: 4 Brick gluster2:/gluster_bricks/engine/engine Status: Connected Number of entries: 0 I don't see much in vdsm.log and gluster logs look fairly normal to me, I'm not seeing any obvious errors in the gluster logs. As far as I can tell the underlying storage is fine. Why are my gluster volumes not healing and why is self-hosted engine failing to start? More agent and broker logs: ==> agent.log <== MainThread::ERROR::2019-12-12 18:36:09,056::hosted_engine::559::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_broker) Failed to start necessary monitors MainThread::ERROR::2019-12-12 18:36:09,058::agent::144::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent return action(he) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper return he.start_monitoring() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring self._initialize_broker() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker m.get('options', {})) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 89, in start_monitor ).format(t=type, o=options, e=e) RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': None, 'network_test': None, 'tcp_t_port': None, 'addr': '10.11.0.254'}] MainThread::ERROR::2019-12-12 18:36:09,058::agent::145::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Trying to restart agent MainThread::ERROR::2019-12-12 18:36:19,619::hosted_engine::559::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_broker) Failed to start necessary monitors MainThread::ERROR::2019-12-12 18:36:19,619::agent::144::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent return action(he) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper return he.start_monitoring() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring self._initialize_broker() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker m.get('options', {})) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 89, in start_monitor ).format(t=type, o=options, e=e) RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': None, 'network_test': None, 'tcp_t_port': None, 'addr': '10.11.0.254'}] MainThread::ERROR::2019-12-12 18:36:19,619::agent::145::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Trying to restart agent MainThread::ERROR::2019-12-12 18:36:30,568::hosted_engine::559::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_broker) Failed to start necessary monitors MainThread::ERROR::2019-12-12 18:36:30,570::agent::144::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent return action(he) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper return he.start_monitoring() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring self._initialize_broker() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker m.get('options', {})) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 89, in start_monitor ).format(t=type, o=options, e=e) RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': None, 'network_test': None, 'tcp_t_port': None, 'addr': '10.11.0.254'}] MainThread::ERROR::2019-12-12 18:36:30,570::agent::145::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Trying to restart agent MainThread::ERROR::2019-12-12 18:36:41,581::hosted_engine::559::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_broker) Failed to start necessary monitors MainThread::ERROR::2019-12-12 18:36:41,583::agent::144::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent return action(he) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper return he.start_monitoring() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring self._initialize_broker() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker m.get('options', {})) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 89, in start_monitor ).format(t=type, o=options, e=e) RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': None, 'network_test': None, 'tcp_t_port': None, 'addr': '10.11.0.254'}] MainThread::ERROR::2019-12-12 18:36:41,583::agent::145::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Trying to restart agent

3 7