September 2019 - Users - oVirt List Archives

Unable to install oVirt on RHEL7.5
by SS00514758＠techmahindra.com 15 Jan '26

15 Jan '26

Hi All, I am unable to install oVirt on RHEL7.5, to install it I am taking reference of below link, https://www.ovirt.org/documentation/install-guide/chap-Installing_oVirt.html But though it is not working for me, couple of dependencies is not getting installed, and because of this I am not able to run the ovirt-engine, below are the depencies packages that unable to install, Error: Package: collectd-write_http-5.8.0-6.1.el7.x86_64 (@ovirt-4.2-centos-opstools) Requires: collectd(x86-64) = 5.8.0-6.1.el7 Removing: collectd-5.8.0-6.1.el7.x86_64 (@ovirt-4.2-centos-opstools) collectd(x86-64) = 5.8.0-6.1.el7 Updated By: collectd-5.8.1-1.el7.x86_64 (epel) collectd(x86-64) = 5.8.1-1.el7 Available: collectd-5.7.2-1.el7.x86_64 (ovirt-4.2-centos-opstools) collectd(x86-64) = 5.7.2-1.el7 Available: collectd-5.7.2-3.el7.x86_64 (ovirt-4.2-centos-opstools) collectd(x86-64) = 5.7.2-3.el7 Available: collectd-5.8.0-2.el7.x86_64 (ovirt-4.2-centos-opstools) collectd(x86-64) = 5.8.0-2.el7 Available: collectd-5.8.0-3.el7.x86_64 (ovirt-4.2-centos-opstools) collectd(x86-64) = 5.8.0-3.el7 Available: collectd-5.8.0-5.el7.x86_64 (ovirt-4.2-centos-opstools) collectd(x86-64) = 5.8.0-5.el7 Help me to install this. Looking forward to resolve this issue. Regards Sumit Sahay

8 10

SPM and Task error ...
by Enrico 22 Nov '24

22 Nov '24

Hi all, my ovirt cluster has got 3 Hypervisors runnig Centos 7.5.1804 vdsm is 4.20.39.1-1.el7, ovirt engine is 4.2.4.5-1.el7, the storage systems are HP MSA P2000 and 2050 (fibre channel). I need to stop one of the hypervisors for maintenance but this system is the storage pool manager. For this reason I decided to manually activate SPM in one of the other nodes but this operation is not successful. In the ovirt engine (engine.log) the error is this: 2019-07-25 12:39:16,744+02 INFO [org.ovirt.engine.core.bll.storage.pool.ForceSelectSPMCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] Running command: ForceSelectSPMCommand internal: false. Entities affected : ID: 81c9bd3c-ae0a-467f-bf7f-63ab30cd8d9e Type: VDSAction group MANIPULATE_HOST with role type ADMIN 2019-07-25 12:39:16,745+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SpmStopOnIrsVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] START, SpmStopOnIrsVDSCommand( SpmStopOnIrsVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7', ignoreFailoverLimit='false'}), log id: 37bf4639 2019-07-25 12:39:16,747+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] START, ResetIrsVDSCommand( ResetIrsVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7', ignoreFailoverLimit='false', vdsId='751f3e99-b95e-4c31-bc38-77f5661a0bdc', ignoreStopFailed='false'}), log id: 2522686f 2019-07-25 12:39:16,749+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] START, SpmStopVDSCommand(HostName = infn-vm05.management, SpmStopVDSCommandParameters:{hostId='751f3e99-b95e-4c31-bc38-77f5661a0bdc', storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7'}), log id: 1810fd8b 2019-07-25 12:39:16,758+02 *ERROR* [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] SpmStopVDSCommand::Not stopping SPM on vds 'infn-vm05.management', pool id '18d57688-6ed4-43b8-bd7c-0665b55950b7' as there are uncleared tasks 'Task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e', status 'running'' 2019-07-25 12:39:16,758+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] FINISH, SpmStopVDSCommand, log id: 1810fd8b 2019-07-25 12:39:16,758+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] FINISH, ResetIrsVDSCommand, log id: 2522686f 2019-07-25 12:39:16,758+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SpmStopOnIrsVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] FINISH, SpmStopOnIrsVDSCommand, log id: 37bf4639 2019-07-25 12:39:16,760+02 *ERROR* [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] EVENT_ID: USER_FORCE_SELECTED_SPM_STOP_FAILED(4,096), Failed to force select infn-vm07.management as the SPM due to a failure to stop the current SPM. while in the hypervisor (SPM) vdsm.log: 2019-07-25 12:39:16,744+02 INFO [org.ovirt.engine.core.bll.storage.pool.ForceSelectSPMCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] Running command: ForceSelectSPMCommand internal: false. Entities affected : ID: 81c9bd3c-ae0a-467f-bf7f-63ab30cd8d9e Type: VDSAction group MANIPULATE_HOST with role type ADMIN 2019-07-25 12:39:16,745+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SpmStopOnIrsVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] START, SpmStopOnIrsVDSCommand( SpmStopOnIrsVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7', ignoreFailoverLimit='false'}), log id: 37bf4639 2019-07-25 12:39:16,747+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] START, ResetIrsVDSCommand( ResetIrsVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7', ignoreFailoverLimit='false', vdsId='751f3e99-b95e-4c31-bc38-77f5661a0bdc', ignoreStopFailed='false'}), log id: 2522686f 2019-07-25 12:39:16,749+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] START, SpmStopVDSCommand(HostName = infn-vm05.management, SpmStopVDSCommandParameters:{hostId='751f3e99-b95e-4c31-bc38-77f5661a0bdc', storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7'}), log id: 1810fd8b 2019-07-25 12:39:16,758+02 *ERROR* [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] SpmStopVDSCommand::Not stopping SPM on vds 'infn-vm05.management', pool id '18d57688-6ed4-43b8-bd7c-0665b55950b7' as there are uncleared tasks 'Task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e', status 'running'' 2019-07-25 12:39:16,758+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] FINISH, SpmStopVDSCommand, log id: 1810fd8b 2019-07-25 12:39:16,758+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] FINISH, ResetIrsVDSCommand, log id: 2522686f 2019-07-25 12:39:16,758+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SpmStopOnIrsVDSCommand] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] FINISH, SpmStopOnIrsVDSCommand, log id: 37bf4639 2019-07-25 12:39:16,760+02 *ERROR* [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] EVENT_ID: USER_FORCE_SELECTED_SPM_STOP_FAILED(4,096), Failed to force select infn-vm07.management as the SPM due to a failure to stop the current SPM. 2019-07-25 12:39:18,660+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Task id 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' has passed pre-polling period time and should be polled. Pre-polling period is 60000 millis. 2019-07-25 12:39:18,660+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Task id 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' has passed pre-polling period time and should be polled. Pre-polling period is 60000 millis. 2019-07-25 12:39:18,750+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Task id 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' has passed pre-polling period time and should be polled. Pre-polling period is 60000 millis. 2019-07-25 12:39:18,750+02 *ERROR* [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] BaseAsyncTask::logEndTaskFailure: Task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' (Parent Command 'Unknown', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure: 2019-07-25 12:39:18,750+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' 2019-07-25 12:39:18,751+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7', ignoreFailoverLimit='false', taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: 34ae2b2f 2019-07-25 12:39:18,752+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] START, HSMClearTaskVDSCommand(HostName = infn-vm05.management, HSMTaskGuidBaseVDSCommandParameters:{hostId='751f3e99-b95e-4c31-bc38-77f5661a0bdc', taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: d3a78ad 2019-07-25 12:39:18,757+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] FINISH, HSMClearTaskVDSCommand, log id: d3a78ad 2019-07-25 12:39:18,757+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] FINISH, SPMClearTaskVDSCommand, log id: 34ae2b2f 2019-07-25 12:39:18,757+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] SPMAsyncTask::ClearAsyncTask: At time of attempt to clear task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' the response code was 'TaskStateError' and message was 'Operation is not allowed in this task state: ("can't clean in state running",)'. Task will not be cleaned 2019-07-25 12:39:18,757+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] BaseAsyncTask::onTaskEndSuccess: Task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' (Parent Command 'Unknown', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2019-07-25 12:39:18,757+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' 2019-07-25 12:39:18,758+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7', ignoreFailoverLimit='false', taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: 42de0c2b 2019-07-25 12:39:18,759+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] START, HSMClearTaskVDSCommand(HostName = infn-vm05.management, HSMTaskGuidBaseVDSCommandParameters:{hostId='751f3e99-b95e-4c31-bc38-77f5661a0bdc', taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: 4895c79c 2019-07-25 12:39:18,764+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] FINISH, HSMClearTaskVDSCommand, log id: 4895c79c 2019-07-25 12:39:18,764+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] FINISH, SPMClearTaskVDSCommand, log id: 42de0c2b 2019-07-25 12:39:18,764+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] SPMAsyncTask::ClearAsyncTask: At time of attempt to clear task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' the response code was 'TaskStateError' and message was 'Operation is not allowed in this task state: ("can't clean in state running",)'. Task will not be cleaned 2019-07-25 12:39:18,764+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Task id 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' has passed pre-polling period time and should be polled. Pre-polling period is 60000 millis. 2019-07-25 12:39:18,764+02 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Cleaning zombie tasks: Clearing async task 'Unknown' that started at 'Fri May 03 14:48:50 CEST 2019' 2019-07-25 12:39:18,764+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' 2019-07-25 12:39:18,765+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7', ignoreFailoverLimit='false', taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: da77af2 2019-07-25 12:39:18,766+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] START, HSMClearTaskVDSCommand(HostName = infn-vm05.management, HSMTaskGuidBaseVDSCommandParameters:{hostId='751f3e99-b95e-4c31-bc38-77f5661a0bdc', taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: 530694fb 2019-07-25 12:39:18,771+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] FINISH, HSMClearTaskVDSCommand, log id: 530694fb 2019-07-25 12:39:18,771+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] FINISH, SPMClearTaskVDSCommand, log id: da77af2 2019-07-25 12:39:18,771+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] SPMAsyncTask::ClearAsyncTask: At time of attempt to clear task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' the response code was 'TaskStateError' and message was 'Operation is not allowed in this task state: ("can't clean in state running",)'. Task will not be cleaned there is some relation between this error and a task that has remained hanging, from SPM server: # vdsm-client Task getInfo taskID=fdcf4d1b-82fe-49a6-b233-323ebe568f8e { "verb": "prepareMerge", "id": "fdcf4d1b-82fe-49a6-b233-323ebe568f8e" } # vdsm-client Task getStatus taskID=fdcf4d1b-82fe-49a6-b233-323ebe568f8e { "message": "running job 1 of 1", "code": 0, "taskID": "fdcf4d1b-82fe-49a6-b233-323ebe568f8e", "taskResult": "", "taskState": "running" } How can I solve this problem ? Thanks a lot for your help !! Best Regards Enrico -- _______________________________________________________________________ Enrico Becchetti Servizio di Calcolo e Reti Istituto Nazionale di Fisica Nucleare - Sezione di Perugia Via Pascoli,c/o Dipartimento di Fisica 06123 Perugia (ITALY) Phone:+39 075 5852777 Mail: Enrico.Becchetti<at>pg.infn.it _______________________________________________________________________

5 15

Changing disk QoS causes segfault with IO-Threads enabled (oVirt 4.3.0.4-1.el7)
by jloh＠squiz.net 23 Apr '24

23 Apr '24

We recently upgraded to 4.3.0 and have found that when changing disk QoS settings on VMs whilst IO-Threads is enabled causes them to segfault and the VM to reboot. We've been able to replicate this across several VMs. VMs with IO-Threads disabled/turned off do not segfault when changing the QoS. Mar 1 11:49:06 srvXX kernel: IO iothread1[30468]: segfault at fffffffffffffff8 ip 0000557649f2bd24 sp 00007f80de832f60 error 5 in qemu-kvm[5576498dd000+a03000] Mar 1 11:49:06 srvXX abrt-hook-ccpp: invalid number 'iothread1' Mar 1 11:49:11 srvXX libvirtd: 2019-03-01 00:49:11.116+0000: 13365: error : qemuMonitorIORead:609 : Unable to read from monitor: Connection reset by peer Happy to supply some more logs to someone if they'll help but just wondering whether anyone else has experienced this or knows of a current fix other than turning io-threads off. Cheers.

4 3

Re: Failed to synchronize networks of Provider ovirt-provider-ovn
by Mail SET Inc. Group 27 Oct '23

27 Oct '23

Yes, i use same manual to change WebUI SSL. ovirt-ca-file= is a same SSL file which use WebUI. Yes, i restart ovirt-provider-ovn, i restart engine, i restart all what i can restart. Nothing... > 12 сент. 2018 г., в 16:11, Dominik Holler <dholler(a)redhat.com> написал(а): > > On Wed, 12 Sep 2018 14:23:54 +0300 > "Mail SET Inc. Group" <mail(a)set-pro.net> wrote: > >> Ok! > > Not exactly, please use users(a)ovirt.org for such questions. > Other should benefit from this questions, too. > Please write the next mail to users(a)ovirt.org and keep me in CC. > >> What i did: >> >> 1) install oVirt «from box» (4.2.5.2-1.el7); >> 2) generate own ssl for my engine using my FreeIPA CA, Install it and > > What means "Install it"? You can use the doc from the following link > https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.2/ht… > > Ensure that ovirt-ca-file= in > /etc/ovirt-provider-ovn/conf.d/10-setup-ovirt-provider-ovn.conf > points to the correct file and ovirt-provider-ovn is restarted. > >> get tis issue; >> >> >> [root@engine ~]# tail -n 50 /var/log/ovirt-provider-ovn.log >> 2018-09-12 14:10:23,828 root [SSL: CERTIFICATE_VERIFY_FAILED] >> certificate verify failed (_ssl.c:579) Traceback (most recent call >> last): File "/usr/share/ovirt-provider-ovn/handlers/base_handler.py", >> line 133, in _handle_request method, path_parts, content >> File "/usr/share/ovirt-provider-ovn/handlers/selecting_handler.py", >> line 175, in handle_request return >> self.call_response_handler(handler, content, parameters) File >> "/usr/share/ovirt-provider-ovn/handlers/keystone.py", line 33, in >> call_response_handler return response_handler(content, parameters) >> File "/usr/share/ovirt-provider-ovn/handlers/keystone_responses.py", >> line 62, in post_tokens user_password=user_password) File >> "/usr/share/ovirt-provider-ovn/auth/plugin_facade.py", line 26, in >> create_token return auth.core.plugin.create_token(user_at_domain, >> user_password) File >> "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/plugin.py", line >> 48, in create_token timeout=self._timeout()) File >> "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/sso.py", line 75, >> in create_token username, password, engine_url, ca_file, timeout) >> File "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/sso.py", line >> 91, in _get_sso_token timeout=timeout File >> "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/sso.py", line 54, >> in wrapper response = func(*args, **kwargs) File >> "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/sso.py", line 47, >> in wrapper raise BadGateway(e) BadGateway: [SSL: >> CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579) >> >> >> [root@engine ~]# tail -n 20 /var/log/ovirt-engine/engine.log >> 2018-09-12 14:10:23,773+03 INFO >> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> (EE-ManagedThreadFactory-engineScheduled-Thread-47) [316db685] Lock >> Acquired to object >> 'EngineLock:{exclusiveLocks='[14e4fb72-9764-4757-b37d-4d487995571a=PROVIDER]', >> sharedLocks=''}' 2018-09-12 14:10:23,778+03 INFO >> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> (EE-ManagedThreadFactory-engineScheduled-Thread-47) [316db685] >> Running command: SyncNetworkProviderCommand internal: true. >> 2018-09-12 14:10:23,836+03 ERROR >> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> (EE-ManagedThreadFactory-engineScheduled-Thread-47) [316db685] >> Command >> 'org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand' >> failed: EngineException: (Failed with error Bad Gateway and code >> 5050) 2018-09-12 14:10:23,837+03 INFO >> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> (EE-ManagedThreadFactory-engineScheduled-Thread-47) [316db685] Lock >> freed to object >> 'EngineLock:{exclusiveLocks='[14e4fb72-9764-4757-b37d-4d487995571a=PROVIDER]', >> sharedLocks=''}' 2018-09-12 14:14:12,477+03 INFO >> [org.ovirt.engine.core.sso.utils.AuthenticationUtils] (default >> task-6) [] User admin@internal successfully logged in with scopes: >> ovirt-app-admin ovirt-app-api ovirt-app-portal >> ovirt-ext=auth:sequence-priority=~ ovirt-ext=revoke:revoke-all >> ovirt-ext=token-info:authz-search >> ovirt-ext=token-info:public-authz-search >> ovirt-ext=token-info:validate ovirt-ext=token:password-access >> 2018-09-12 14:14:12,587+03 INFO >> [org.ovirt.engine.core.bll.aaa.CreateUserSessionCommand] (default >> task-6) [1bf1b763] Running command: CreateUserSessionCommand >> internal: false. 2018-09-12 14:14:12,628+03 INFO >> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >> (default task-6) [1bf1b763] EVENT_ID: USER_VDC_LOGIN(30), User >> admin@internal-authz connecting from '10.0.3.61' using session >> 's8jAm7BUJGlicthm6yZBA3CUM8QpRdtwFaK3M/IppfhB3fHFB9gmNf0cAlbl1xIhcJ2WX+ww7e71Ri+MxJSsIg==' >> logged in. 2018-09-12 14:14:30,972+03 INFO >> [org.ovirt.engine.core.bll.provider.ImportProviderCertificateCommand] >> (default task-6) [ee3cc8a7-4485-4fdf-a0c2-e9d67b5cfcd3] Running >> command: ImportProviderCertificateCommand internal: false. Entities >> affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: >> SystemAction group CREATE_STORAGE_POOL with role type ADMIN >> 2018-09-12 14:14:30,982+03 INFO >> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >> (default task-6) [ee3cc8a7-4485-4fdf-a0c2-e9d67b5cfcd3] EVENT_ID: >> PROVIDER_CERTIFICATE_IMPORTED(213), Certificate for provider >> ovirt-provider-ovn was imported. (User: admin@internal-authz) >> 2018-09-12 14:14:31,006+03 INFO >> [org.ovirt.engine.core.bll.provider.TestProviderConnectivityCommand] >> (default task-6) [a48d94ab-b0b2-42a2-a667-0525b4c652ea] Running >> command: TestProviderConnectivityCommand internal: false. Entities >> affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: >> SystemAction group CREATE_STORAGE_POOL with role type ADMIN >> 2018-09-12 14:14:31,058+03 ERROR >> [org.ovirt.engine.core.bll.provider.TestProviderConnectivityCommand] >> (default task-6) [a48d94ab-b0b2-42a2-a667-0525b4c652ea] Command >> 'org.ovirt.engine.core.bll.provider.TestProviderConnectivityCommand' >> failed: EngineException: (Failed with error Bad Gateway and code >> 5050) 2018-09-12 14:15:10,954+03 INFO >> [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] >> (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread >> pool 'default' is using 0 threads out of 1, 5 threads waiting for >> tasks. 2018-09-12 14:15:10,954+03 INFO >> [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] >> (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread >> pool 'engine' is using 0 threads out of 500, 16 threads waiting for >> tasks and 0 tasks in queue. 2018-09-12 14:15:10,954+03 INFO >> [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] >> (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread >> pool 'engineScheduled' is using 0 threads out of 100, 100 threads >> waiting for tasks. 2018-09-12 14:15:10,954+03 INFO >> [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] >> (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread >> pool 'engineThreadMonitoring' is using 1 threads out of 1, 0 threads >> waiting for tasks. 2018-09-12 14:15:10,954+03 INFO >> [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] >> (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread >> pool 'hostUpdatesChecker' is using 0 threads out of 5, 2 threads >> waiting for tasks. 2018-09-12 14:15:23,843+03 INFO >> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> (EE-ManagedThreadFactory-engineScheduled-Thread-61) [2455041f] Lock >> Acquired to object >> 'EngineLock:{exclusiveLocks='[14e4fb72-9764-4757-b37d-4d487995571a=PROVIDER]', >> sharedLocks=''}' 2018-09-12 14:15:23,849+03 INFO >> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> (EE-ManagedThreadFactory-engineScheduled-Thread-61) [2455041f] >> Running command: SyncNetworkProviderCommand internal: true. >> 2018-09-12 14:15:23,900+03 ERROR >> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> (EE-ManagedThreadFactory-engineScheduled-Thread-61) [2455041f] >> Command >> 'org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand' >> failed: EngineException: (Failed with error Bad Gateway and code >> 5050) 2018-09-12 14:15:23,901+03 INFO >> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] >> (EE-ManagedThreadFactory-engineScheduled-Thread-61) [2455041f] Lock >> freed to object >> 'EngineLock:{exclusiveLocks='[14e4fb72-9764-4757-b37d-4d487995571a=PROVIDER]', >> sharedLocks=''}' >> >> >> [root@engine ~]# >> cat /etc/ovirt-provider-ovn/conf.d/10-setup-ovirt-provider-ovn.conf # >> This file is automatically generated by engine-setup. Please do not >> edit manually [OVN REMOTE] ovn-remote=ssl:127.0.0.1:6641 >> [SSL] >> https-enabled=true >> ssl-cacert-file=/etc/pki/ovirt-engine/ca.pem >> ssl-cert-file=/etc/pki/ovirt-engine/certs/ovirt-provider-ovn.cer >> ssl-key-file=/etc/pki/ovirt-engine/keys/ovirt-provider-ovn.key.nopass >> [OVIRT] >> ovirt-sso-client-secret=Ms7Gw9qNT6IkXu7oA54tDmxaZDIukABV >> ovirt-host=https://engine.set.local:443 >> ovirt-sso-client-id=ovirt-provider-ovn >> ovirt-ca-file=/etc/pki/ovirt-engine/apache-ca.pem >> [PROVIDER] >> provider-host=engine.set.local >> >> >>> 12 сент. 2018 г., в 13:59, Dominik Holler <dholler(a)redhat.com> >>> написал(а): >>> >>> On Wed, 12 Sep 2018 13:04:53 +0300 >>> "Mail SET Inc. Group" <mail(a)set-pro.net> wrote: >>> >>>> Hello Dominik! >>>> I have a same issue with OVN provider and SSL >>>> https://www.mail-archive.com/users@ovirt.org/msg47020.html >>>> <https://www.mail-archive.com/users@ovirt.org/msg47020.html> But >>>> certificate changes not helps to resolve it. Maybe you can help me >>>> with this? >>> >>> Sure. Can you please share the relevant lines of >>> ovirt-provider-ovn.log and engine.log, and the information if you >>> are using the certificates generated by engine-setup with >>> users(a)ovirt.org ? Thanks, >>> Dominik >>> >> > >

5 13

engine-setup failing on 4.3.2 -> 4.3.3 fails during Engine schema refresh fail
by Edward Berger 17 Jul '23

17 Jul '23

I was trying to upgrade a hyperconverged oVirt hosted engine and failed in the engine-setup command with these error and warnings. ... [ INFO ] Creating/refreshing Engine database schema [ ERROR ] schema.sh: FATAL: Cannot execute sql command: --file=/usr/share/ovirt-engine/dbscripts/upgrade/04_03_0830_add_foreign_key_to_image_transfers.sql [ ERROR ] Failed to execute stage 'Misc configuration': Engine schema refresh failed ... [ INFO ] Yum Verify: 16/16: ovirt-engine-tools.noarch 0:4.3.3.5-1.el7 - e [WARNING] Rollback of DWH database postponed to Stage "Clean up" [ INFO ] Rolling back database schema ... [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination [ ERROR ] Execution of setup failed Attaching engine-setup logfile.

4 6

oVirt host deploy: disable multipath
by femi adegoke 27 Mar '23

27 Mar '23

https://bugzilla.redhat.com/show_bug.cgi?id=1472356 There are 2 drives in my install that oVirt is claiming as multipath. I have tried blacklisting at /etc/multipath/ but that hasn't worked. Any better solutions? I need these 2 drives to be non-multipathed.

4 19

Out-of-sync networks can only be detached
by Sakhi Hadebe 26 Feb '23

26 Feb '23

Hi, I have a 3-node oVirt cluster. I have configured 2 logical networks: ovirtmgmt and public. Public logical network is attached in only 2 nodes and failing to attach on the 3rd node with the below error Invalid operation, out-of-sync network 'public' can only be detached. Please have been stuck on this for almost the whole day now. How do I fix this error? -- Regards, Sakhi Hadebe

3 5

Network Address Change
by Paul.LKW 28 Dec '22

28 Dec '22

Hi All: I just has a case, I need to change the oVirt host and engine IP address due to data center decommission I checked in the hosted-engine host there are some files I could change ; in ovirt-hosted-engine/hosted-engine.conf ca_subject="O=simple.com, CN=1.2.3.4" gateway=1.2.3.254 and of course I need to change the ovirtmgmt interface IP too, I think just change the above line could do the tick, but where could I change the other host IP in the cluster ? I think I have to be lost all the host as once changed the hosted-engine host IP as it is in diff. sub net. Does there any command line tools could do that or someone has such experience could share? Best Regards, Paul.LKW

4 6

looking for some ISV backup software which integrates the backup API
by Nathanaël Blanchet 08 Jun '22

08 Jun '22

Hello, We are about to change our backup provider, and I find it is a great chance to choose a full supported ovirt backup solution. I currently use this python script vm-backup-scheduler (https://github.com/wefixit-AT/oVirtBackup) but it is not the workflow officially suggested by the community (https://www.ovirt.org/develop/release-management/features/storage/backup-re…) I've been looking for a long time an ISV who supports such an API, but the only one I found is this one : Acronis Backup Advanced suggested here https://access.redhat.com/ecosystem/search/#/ecosystem/Red%20Hat%20Enterpri… I ran the trial version, but it doesn't seem to do better than the vm-backup-scheduler script, and it doesn't seem to use the backup API (attach a clone as a disk to an existing vm). Can you suggest me some other ISV solutions, if they ever exist... or share me your backup experience?

5 4

OVS switch type for hosted-engine
by Devin A. Bougie 06 May '22

06 May '22

Is it possible to setup a hosted engine using the OVS switch type instead of Legacy? If it's not possible to start out as OVS, instructions for switching from Legacy to OVS after the fact would be greatly appreciated. Many thanks, Devin

4 5

USB3 redirection
by Rik Theys 19 Apr '22

19 Apr '22

Hi, I'm trying to assign a USB3 controller to a CentOS 7.4 VM in oVirt 4.1 with USB redirection enabled. I've created the following file in /etc/ovirt-engine/osinfo.conf.d: 01-usb.properties with content os.other.devices.usb.controller.value = nec-xhci and have restarted ovirt-engine. If I disable USB-support in the web interface for the VM, the xhci controller is added to the VM (I can see it in the qemu-kvm commandline), but usb redirection is not available. If I enable USB-support in the UI, no xhci controller is added (only 4 uhci controllers). Is there a way to make the controllers for usb redirection xhci controllers? Regards, Rik -- Rik Theys System Engineer KU Leuven - Dept. Elektrotechniek (ESAT) Kasteelpark Arenberg 10 bus 2440 - B-3001 Leuven-Heverlee +32(0)16/32.11.07 ---------------------------------------------------------------- <<Any errors in spelling, tact or fact are transmission errors>>

3 5

OVN routing and firewalling in oVirt
by Gianluca Cecchi 19 Apr '22

19 Apr '22

Hello, how do we manage routing between different OVN networks in oVirt? And between OVN networks and physical ones? Based on architecture read here: http://openvswitch.org/support/dist-docs/ovn-architecture.7.html I see terms for logical routers and gateway routers respectively but how to apply to oVirt configuration? Do I have to choose between setting up a specialized VM or a physical one: is it applicable/advisable to put on oVirt host itself the gateway functionality? Is there any security policy (like security groups in Openstack) to implement? Thanks, Gianluca

4 6

Install hosted-engine - Task Get local VM IP failed
by florentl 10 Mar '22

10 Mar '22

Hi all, I try to install hosted-engine on node : ovirt-node-ng-4.2.3-0.20180518. Every times I get stuck on : [ ERROR ] fatal: [localhost]: FAILED! => {"attempts": 50, "changed": true, "cmd": "virsh -r net-dhcp-leases default | grep -i 00:16:3e:6c:5a:91 | awk '{ print $5 }' | cut -f1 -d'/'", "delta": "0:00:00.108872", "end": "2018-06-01 11:17:34.421769", "rc": 0, "start": "2018-06-01 11:17:34.312897", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} I tried with static IP Address and with DHCP but both failed. To be more specific, I installed three nodes, deployed glusterfs with the wizard. I'm in a nested virtualization environment for this lab (Vmware Esxi Hypervisor). My node IP is : 192.168.176.40 / and I want the hosted-engine vm has 192.168.176.43. Thanks, Florent

7 17

Host needs to be reinstalled after configuring power management
by Andrew DeMaria 02 Feb '22

02 Feb '22

Hi, I am running ovirt 4.3 and have found the following action item immediately after configuring power management for a host: Host needs to be reinstalled as important configuration changes were applied on it. The thing is - I've just freshly installed this host and it seems strange that I need to reinstall it. Is there a better way to install a host and configure power management without having to reinstall it after? Thanks, Andrew

4 3

Lots of storage.MailBox.SpmMailMonitor
by Fabrice Bacchella 06 Jan '22

06 Jan '22

My vdsm log files are huge: -rw-r--r-- 1 vdsm kvm 1.8G Nov 22 11:32 vdsm.log And this is juste half an hour of logs: $ head -1 vdsm.log 2018-11-22 11:01:12,132+0100 ERROR (mailbox-spm) [storage.MailBox.SpmMailMonitor] mailbox 2 checksum failed, not clearing mailbox, clearing new mail (data='...lots of data', expected='\xa4\x06\x08\x00') (mailbox:612) I just upgraded vdsm: $ rpm -qi vdsm Name : vdsm Version : 4.20.43

3 5

poweroff and reboot with ovirt_vm ansible module
by Nathanaël Blanchet 18 May '21

18 May '21

Hello, is there a way to poweroff or reboot (without stopped and running state) a vm with the ovirt_vm ansible module? -- Nathanaël Blanchet Supervision réseau Pôle Infrastrutures Informatiques 227 avenue Professeur-Jean-Louis-Viala 34193 MONTPELLIER CEDEX 5 Tél. 33 (0)4 67 54 84 55 Fax 33 (0)4 67 54 84 14 blanchet(a)abes.fr

7 11

OVN and change of mgmt network
by Gianluca Cecchi 26 Jan '21

26 Jan '21

Hello, I previously had OVN running on engine (as OVN provider with northd and northbound and southbound DBs) and hosts (with OVN controller). After changing mgmt ip of hosts (engine has retained instead the same ip), I executed again on them the command: vdsm-tool ovn-config <ip_of_engine> <nel_local_ip_of_host> Now I think I have to clean up some things, eg: 1) On engine where I get these lines below systemctl status ovn-northd.service -l . . . Sep 29 14:41:42 ovmgr1 ovsdb-server[940]: ovs|00005|reconnect|ERR|tcp: 10.4.167.40:37272: no response to inactivity probe after 5 seconds, disconnecting Oct 03 11:52:00 ovmgr1 ovsdb-server[940]: ovs|00006|reconnect|ERR|tcp: 10.4.167.41:52078: no response to inactivity probe after 5 seconds, disconnecting The two IPs are the old ones of two hosts It seems that a restart of the services has fixed... Can anyone confirm if I have to do anything else? 2) On hosts (there are 3 hosts with OVN on ip 10.4.192.32/33/34) where I currently have this output [root@ov301 ~]# ovs-vsctl show 3a38c5bb-0abf-493d-a2e6-345af8aedfe3 Bridge br-int fail_mode: secure Port "ovn-1dce5b-0" Interface "ovn-1dce5b-0" type: geneve options: {csum="true", key=flow, remote_ip="10.4.192.32"} Port "ovn-ddecf0-0" Interface "ovn-ddecf0-0" type: geneve options: {csum="true", key=flow, remote_ip="10.4.192.33"} Port "ovn-fd413b-0" Interface "ovn-fd413b-0" type: geneve options: {csum="true", key=flow, remote_ip="10.4.168.74"} Port br-int Interface br-int type: internal ovs_version: "2.7.2" [root@ov301 ~]# The IPs of kind 10.4.192.x are ok. But there is a left-over of an old host I initially used for tests, corresponding to 10.4.168.74, that now doesn't exist anymore How can I clean records for 1) and 2)? Thanks, Gianluca

3 4

Can't import some VMs after storage domain detach and reattach to new datacenter.
by m black 09 Dec '20

09 Dec '20

7 14

"gluster-ansible-roles is not installed on Host" error on Cockpit
by Hesham Ahmed 26 Nov '20

26 Nov '20

On a new 4.3.1 oVirt Node installation, when trying to deploy HCI (also when trying adding a new gluster volume to existing clusters) using Cockpit, an error is displayed "gluster-ansible-roles is not installed on Host. To continue deployment, please install gluster-ansible-roles on Host and try again". There is no package named gluster-ansible-roles in the repositories: [root@localhost ~]# yum install gluster-ansible-roles Loaded plugins: enabled_repos_upload, fastestmirror, imgbased-persist, package_upload, product-id, search-disabled-repos, subscription-manager, vdsmupgrade This system is not registered with an entitlement server. You can use subscription-manager to register. Loading mirror speeds from cached hostfile * ovirt-4.3-epel: mirror.horizon.vn No package gluster-ansible-roles available. Error: Nothing to do Uploading Enabled Repositories Report Cannot upload enabled repos report, is this client registered? This is due to check introduced here: https://gerrit.ovirt.org/#/c/98023/1/dashboard/src/helpers/AnsibleUtil.js Changing the line from: [ "rpm", "-qa", "gluster-ansible-roles" ], { "superuser":"require" } to [ "rpm", "-qa", "gluster-ansible" ], { "superuser":"require" } resolves the issue. The above code snippet is installed at /usr/share/cockpit/ovirt-dashboard/app.js on oVirt node and can be patched by running "sed -i 's/gluster-ansible-roles/gluster-ansible/g' /usr/share/cockpit/ovirt-dashboard/app.js && systemctl restart cockpit"

2 1

Error exporting into ova
by Gianluca Cecchi 31 Aug '20

31 Aug '20

Hello, I'm playing with export_vm_as_ova.py downloaded from the examples github: https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/export_v… My environment is oVirt 4.3.3.7 with iSCSI storage domain. It fails leaving an ova.tmp file In webadmin gui: Starting to export Vm enginecopy1 as a Virtual Appliance 7/19/1911:55:12 AM VDSM ov301 command TeardownImageVDS failed: Cannot deactivate Logical Volume: ('General Storage Exception: ("5 [] [\' Logical volume fa33df49-b09d-4f86-9719-ede649542c21/0420ef47-0ad0-4cf9-babd-d89383f7536b in use.\']\\nfa33df49-b09d-4f86-9719-ede649542c21/[\'a7480dc5-b5ca-4cb3-986d-77bc12165be4\', \'0420ef47-0ad0-4cf9-babd-d89383f7536b\']",)',) 7/19/1912:25:36 PM Failed to export Vm enginecopy1 as a Virtual Appliance to path /save_ova/base/dump/myvm2.ova on Host ov301 7/19/1912:25:37 PM During export I have this qemu-img process creating the disk over the loop device: root 30878 30871 0 11:55 pts/2 00:00:00 su -p -c qemu-img convert -T none -O qcow2 '/rhev/data-center/mnt/blockSD/fa33df49-b09d-4f86-9719-ede649542c21/images/59a4a324-4c99-4ff5-abb1-e9bbac83292a/0420ef47-0ad0-4cf9-babd-d89383f7536b' '/dev/loop1' vdsm vdsm 30882 30878 10 11:55 ? 00:00:00 qemu-img convert -T none -O qcow2 /rhev/data-center/mnt/blockSD/fa33df49-b09d-4f86-9719-ede649542c21/images/59a4a324-4c99-4ff5-abb1-e9bbac83292a/0420ef47-0ad0-4cf9-babd-d89383f7536b /dev/loop1 The ova.tmp file is getting filled while command runs eg: [root@ov301 ]# du -sh /save_ova/base/dump/myvm2.ova.tmp 416M /save_ova/base/dump/myvm2.ova.tmp [root@ov301 sysctl.d]# [root@ov301 sysctl.d]# du -sh /save_ova/base/dump/myvm2.ova.tmp 911M /save_ova/base/dump/myvm2.ova.tmp [root@ov301 ]# and the final generated / not completed file is in this state: [root@ov301 ]# qemu-img info /save_ova/base/dump/myvm2.ova.tmp image: /save_ova/base/dump/myvm2.ova.tmp file format: raw virtual size: 30G (32217446400 bytes) disk size: 30G [root@ov301 sysctl.d]# But I notice that the timestamp of the file is about 67 minutes after start of job and well after the notice of its failure.... [root@ov301 sysctl.d]# ll /save_ova/base/dump/ total 30963632 -rw-------. 1 root root 32217446400 Jul 19 13:02 myvm2.ova.tmp [root@ov301 sysctl.d]# [root@ov301 sysctl.d]# du -sh /save_ova/base/dump/myvm2.ova.tmp 30G /save_ova/base/dump/myvm2.ova.tmp [root@ov301 sysctl.d]# In engine.log the first error I see is 30 minutes after start 2019-07-19 12:25:31,563+02 ERROR [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] Ansible playbook execution failed: Timeout occurred while executing Ansible playbook. 2019-07-19 12:25:31,563+02 INFO [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] Ansible playbook command has exited with value: 1 2019-07-19 12:25:31,564+02 ERROR [org.ovirt.engine.core.bll.CreateOvaCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] Failed to create OVA. Please check logs for more details: /var/log/ovirt-engine/ova/ovirt-export-ova-ansible-20190719115531-ov301-2001ddf4.log 2019-07-19 12:25:31,565+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.TeardownImageVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] START, TeardownImageVDSCommand(HostName = ov301, ImageActionsVDSCommandParameters:{hostId='8ef1ce6f-4e38-486c-b3a4-58235f1f1d06'}), log id: 3d2246f7 2019-07-19 12:25:36,569+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM ov301 command TeardownImageVDS failed: Cannot deactivate Logical Volume: ('General Storage Exception: ("5 [] [\' Logical volume fa33df49-b09d-4f86-9719-ede649542c21/0420ef47-0ad0-4cf9-babd-d89383f7536b in use.\']\\nfa33df49-b09d-4f86-9719-ede649542c21/[\'a7480dc5-b5ca-4cb3-986d-77bc12165be4\', \'0420ef47-0ad0-4cf9-babd-d89383f7536b\']",)',) In ansible playbook suggested log file I don't see anything useful. It ends with timestamps when the script has been launched. Last lines are: 2019-07-19 11:55:33,877 p=5699 u=ovirt | TASK [ovirt-ova-export-pre-pack : Retrieving the temporary path for the OVA file] *** 2019-07-19 11:55:34,198 p=5699 u=ovirt | changed: [ov301] => { "changed": true, "dest": "/save_ova/base/dump/myvm2.ova.tmp", "gid": 0, "group": "root", "mode": "0600", "owner": "root", "secontext": "system_u:object_r:nfs_t:s0", "size": 32217446912, "state": "file", "uid": 0 } 2019-07-19 11:55:34,204 p=5699 u=ovirt | TASK [ovirt-ova-pack : Run packing script] ************************************* It seems 30 minutes... for timeout? About what, ansible job? Or possibly implicit user session created when running the python script? The snapshot has been correctly deleted (as I see also in engine.log), I don't see it in webadmin gui. Any known problem? Just for test I executed again at 14:24 and I see same Ansible error at 14:54 The snapshot gets deleted, while the qemu-img command still continues.... [root@ov301 sysctl.d]# ps -ef | grep qemu-img root 13504 13501 0 14:24 pts/1 00:00:00 su -p -c qemu-img convert -T none -O qcow2 '/rhev/data-center/mnt/blockSD/fa33df49-b09d-4f86-9719-ede649542c21/images/59a4a324-4c99-4ff5-abb1-e9bbac83292a/0420ef47-0ad0-4cf9-babd-d89383f7536b' '/dev/loop0' vdsm vdsm 13505 13504 3 14:24 ? 00:01:26 qemu-img convert -T none -O qcow2 /rhev/data-center/mnt/blockSD/fa33df49-b09d-4f86-9719-ede649542c21/images/59a4a324-4c99-4ff5-abb1-e9bbac83292a/0420ef47-0ad0-4cf9-babd-d89383f7536b /dev/loop0 root 17587 24530 0 15:05 pts/0 00:00:00 grep --color=auto qemu-img [root@ov301 sysctl.d]# [root@ov301 sysctl.d]# du -sh /save_ova/base/dump/myvm2.ova.tmp 24G /save_ova/base/dump/myvm2.ova.tmp [root@ov301 sysctl.d]# ll /save_ova/base/dump/myvm2.ova.tmp -rw-------. 1 root root 32217446400 Jul 19 15:14 /save_ova/base/dump/myvm2.ova.tmp [root@ov301 sysctl.d]# and then continues until image copy completes, but at this time the job has already aborted and so the completion of the ova composition doesn't go ahead... and I remain with the ova.tmp file... How to extend timeout? Thanks in advance, Gianluca

4 11

deprecating export domain?
by Charles Kozler 30 Aug '20

30 Aug '20

Hello, I recently read on this list from a redhat member that export domain is either being deprecated or looking at being deprecated To that end, can you share details? Can you share any notes/postings/bz's that document this? I would imagine something like this would be discussed in larger audience This seems like a somewhat significant change to make and I am curious where this is scheduled? Currently, a lot of my backups rely explicitly on an export domain for online snapshots, so I'd like to plan accordingly Thanks!

11 21

Re: Single instance scaleup.
by Strahil 05 Jun '20

05 Jun '20

Hi Leo, As you do not have a distributed volume , you can easily switch to replica 2 arbiter 1 or replica 3 volumes. You can use the following for adding the bricks: https://access.redhat.com/documentation/en-US/Red_Hat_Storage/2.1/html/Admi… Best Regards, Strahil NikolivOn May 26, 2019 10:54, Leo David <leoalex(a)gmail.com> wrote: > > Hi Stahil, > Thank you so much for yout input ! > > gluster volume info > > > Volume Name: engine > Type: Distribute > Volume ID: d7449fc2-cc35-4f80-a776-68e4a3dbd7e1 > Status: Started > Snapshot Count: 0 > Number of Bricks: 1 > Transport-type: tcp > Bricks: > Brick1: 192.168.80.191:/gluster_bricks/engine/engine > Options Reconfigured: > nfs.disable: on > transport.address-family: inet > storage.owner-uid: 36 > storage.owner-gid: 36 > features.shard: on > performance.low-prio-threads: 32 > performance.strict-o-direct: off > network.remote-dio: off > network.ping-timeout: 30 > user.cifs: off > performance.quick-read: off > performance.read-ahead: off > performance.io-cache: off > cluster.eager-lock: enable > Volume Name: ssd-samsung > Type: Distribute > Volume ID: 76576cc6-220b-4651-952d-99846178a19e > Status: Started > Snapshot Count: 0 > Number of Bricks: 1 > Transport-type: tcp > Bricks: > Brick1: 192.168.80.191:/gluster_bricks/sdc/data > Options Reconfigured: > cluster.eager-lock: enable > performance.io-cache: off > performance.read-ahead: off > performance.quick-read: off > user.cifs: off > network.ping-timeout: 30 > network.remote-dio: off > performance.strict-o-direct: on > performance.low-prio-threads: 32 > features.shard: on > storage.owner-gid: 36 > storage.owner-uid: 36 > transport.address-family: inet > nfs.disable: on > > The other two hosts will be 192.168.80.192/193 - this is gluster dedicated network over 10GB sfp+ switch. > - host 2 wil have identical harware configuration with host 1 ( each disk is actually a raid0 array ) > - host 3 has: > - 1 ssd for OS > - 1 ssd - for adding to engine volume in a full replica 3 > - 2 ssd's in a raid 1 array to be added as arbiter for the data volume ( ssd-samsung ) > So the plan is to have "engine" scaled in a full replica 3, and "ssd-samsung" scalled in a replica 3 arbitrated. > > > > > On Sun, May 26, 2019 at 10:34 AM Strahil <hunter86_bg(a)yahoo.com> wrote: >> >> Hi Leo, >> >> Gluster is quite smart, but in order to provide any hints , can you provide output of 'gluster volume info <glustervol>'. >> If you have 2 more systems , keep in mind that it is best to mirror the storage on the second replica (2 disks on 1 machine -> 2 disks on the new machine), while for the arbiter this is not neccessary. >> >> What is your network and NICs ? Based on my experience , I can recommend at least 10 gbit/s interfase(s). >> >> Best Regards, >> Strahil Nikolov >> >> On May 26, 2019 07:52, Leo David <leoalex(a)gmail.com> wrote: >>> >>> Hello Everyone, >>> Can someone help me to clarify this ? >>> I have a single-node 4.2.8 installation ( only two gluster storage domains - distributed single drive volumes ). Now I just got two identintical servers and I would like to go for a 3 nodes bundle. >>> Is it possible ( after joining the new nodes to the cluster ) to expand the existing volumes across the new nodes and change them to replica 3 arbitrated ? >>> If so, could you share with me what would it be the procedure ? >>> Thank you very much ! >>> >>> Leo > > > > -- > Best regards, Leo David

4 5

Failed to add storage domain
by thunderlight1＠gmail.com 31 May '20

31 May '20

Hi! I have installed oVirt using the iso ovirt-node-ng-installer-4.3.2-2019031908.el7. I the did run the Host-engine deployment through Cockpit. I got an error when it tries to create the domain storage. It sucessfully mounted the NFS-share on the host. Bellow is the error I got: 2019-04-14 10:40:38,967+0200 INFO ansible skipped {'status': 'SKIPPED', 'ansible_task': u'Check storage domain free space', 'ansible_host': u'localhost', 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml', 'ansible_type': 'task'} 2019-04-14 10:40:38,967+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fb6918ad9d0> kwargs 2019-04-14 10:40:39,516+0200 INFO ansible task start {'status': 'OK', 'ansible_task': u'ovirt.hosted_engine_setup : Activate storage domain', 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml', 'ansible_type': 'task'} 2019-04-14 10:40:39,516+0200 DEBUG ansible on_any args TASK: ovirt.hosted_engine_setup : Activate storage domain kwargs is_conditional:False 2019-04-14 10:40:41,923+0200 DEBUG var changed: host "localhost" var "otopi_storage_domain_details" type "<type 'dict'>" value: "{ "changed": false, "exception": "Traceback (most recent call last):\n File \"/tmp/ansible_ovirt_storage_domain_payload_xSFxOp/__main__.py\", line 664, in main\n storage_domains_module.post_create_check(sd_id)\n File \"/tmp/ansible_ovirt_storage_domain_payload_xSFxOp/__main__.py\", line 526, in post_create_check\n id=storage_domain.id,\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py\", line 3053, in add\n return self._internal_add(storage_domain, headers, query, wait)\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py\", line 232, in _internal_add\n return future.wait() if wait else future\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py\", line 55, in wait\n return self._code(response)\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py\", line 229, in callback\n self._check_fault(response)\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py\", line 132, in _check_fault\n self._raise_error(response , body)\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py\", line 118, in _raise_error\n raise error\nError: Fault reason is \"Operation Failed\". Fault detail is \"[]\". HTTP response code is 400.\n", "failed": true, "msg": "Fault reason is \"Operation Failed\". Fault detail is \"[]\". HTTP response code is 400." }" 2019-04-14 10:40:41,924+0200 DEBUG var changed: host "localhost" var "ansible_play_hosts" type "<type 'list'>" value: "[]" 2019-04-14 10:40:41,924+0200 DEBUG var changed: host "localhost" var "play_hosts" type "<type 'list'>" value: "[]" 2019-04-14 10:40:41,924+0200 DEBUG var changed: host "localhost" var "ansible_play_batch" type "<type 'list'>" value: "[]" 2019-04-14 10:40:41,924+0200 ERROR ansible failed {'status': 'FAILED', 'ansible_type': 'task', 'ansible_task': u'Activate storage domain', 'ansible_result': u'type: <type \'dict\'>\nstr: {\'_ansible_parsed\': True, u\'exception\': u\'Traceback (most recent call last):\\n File "/tmp/ansible_ovirt_storage_domain_payload_xSFxOp/__main__.py", line 664, in main\\n storage_domains_module.post_create_check(sd_id)\\n File "/tmp/ansible_ovirt_storage_domain_payload_xSFxOp/__main__.py", line 526', 'task_duration': 2, 'ansible_host': u'localhost', 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml'} 2019-04-14 10:40:41,924+0200 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fb691843190> kwargs ignore_errors:None 2019-04-14 10:40:41,928+0200 INFO ansible stats { "ansible_playbook": "/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml", "ansible_playbook_duration": "00:37 Minutes", "ansible_result": "type: <type 'dict'>\nstr: {u'localhost': {'unreachable': 0, 'skipped': 6, 'ok': 23, 'changed': 1, 'failures': 1}}", "ansible_type": "finish", "status": "FAILED" } 2019-04-14 10:40:41,928+0200 INFO SUMMARY: Duration Task Name -------- -------- [ < 1 sec ] Execute just a specific set of steps [ 00:01 ] Force facts gathering [ 00:01 ] Check local VM dir stat [ 00:01 ] Obtain SSO token using username/password credentials [ 00:01 ] Fetch host facts [ < 1 sec ] Fetch cluster ID [ 00:01 ] Fetch cluster facts [ 00:01 ] Fetch Datacenter facts [ < 1 sec ] Fetch Datacenter ID [ < 1 sec ] Fetch Datacenter name [ 00:02 ] Add NFS storage domain [ 00:01 ] Get storage domain details [ 00:01 ] Find the appliance OVF [ 00:01 ] Parse OVF [ < 1 sec ] Get required size [ FAILED ] Activate storage domain 2019-04-14 10:40:41,928+0200 DEBUG ansible on_any args <ansible.executor.stats.AggregateStats object at 0x7fb69404eb90> kwargs Any suggestions on how fix this?

2 2

How to connect to a guest with vGPU ?
by Josep Manel Andrés Moscardó 29 May '20

29 May '20

Hi, I got vGPU through mdev working but I am wondering how I would connect to the client and make use of the GPU. So far I try to access the console through SPICE and at some point in the boot process it switches to GPU and I cannot see anything else. Thanks. -- Josep Manel Andrés Moscardó Systems Engineer, IT Operations EMBL Heidelberg T +49 6221 387-8394

3 4

ovirt-guest-agent for CentOS 8
by Eduardo Mayoral 30 Mar '20

30 Mar '20

Hi, Just like many of you I am testing my first CentOS 8 VMs on top of ovirt. I am not finding the package ovirt-guest-agent.noarch . Closest I can find is qemu-guest-agent.x86_64 After installing and starting it, I do see information reported on the "Guest info" tab. Can anybody confirm if this is indeed the agent we should be using? Is there - or will there be - a more specific package for ovirt guests? Thanks! -- Eduardo Mayoral Jimeno Systems engineer, platform department. Arsys Internet. emayoral(a)arsys.es - +34 941 620 105 - ext 2153

6 7

Vm suddenly paused with error "vm has paused due to unknown storage error"
by Jasper Siero 17 Feb '20

17 Feb '20

Hi all, Since we upgraded our Ovirt nodes to CentOS 7 a vm (not a specific one but never more then one) will sometimes pause suddenly with the error "VM ... has paused due to unknown storage error". It happens now two times in a month. The Ovirt node uses san storage for the vm's running on it. When a specific vm is pausing with an error the other vm's keeps running without problems. The vm runs without problems after unpausing it. Versions: CentOS Linux release 7.1.1503 vdsm-4.14.17-0 libvirt-daemon-1.2.8-16 vdsm.log: VM Channels Listener::DEBUG::2015-10-25 07:43:54,382::vmChannels::95::vds::(_handle_timeouts) Timeout on fileno 78. libvirtEventLoop::INFO::2015-10-25 07:43:56,177::vm::4602::vm.Vm::(_onIOError) vmId=`77f07ae0-cc3e-4ae2-90ec-7fba7b11deeb`::abnormal vm stop device virtio-disk0 error eother libvirtEventLoop::DEBUG::2015-10-25 07:43:56,178::vm::5204::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`77f07ae0-cc3e-4ae2-90ec-7fba7b11deeb`::event Suspended detail 2 opaque None libvirtEventLoop::INFO::2015-10-25 07:43:56,178::vm::4602::vm.Vm::(_onIOError) vmId=`77f07ae0-cc3e-4ae2-90ec-7fba7b11deeb`::abnormal vm stop device virtio-disk0 error eother ........... libvirtEventLoop::INFO::2015-10-25 07:43:56,180::vm::4602::vm.Vm::(_onIOError) vmId=`77f07ae0-cc3e-4ae2-90ec-7fba7b11deeb`::abnormal vm stop device virtio-disk0 error eother specific error part in libvirt vm log: block I/O error in device 'drive-virtio-disk0': Unknown error 32758 (32758) ........... block I/O error in device 'drive-virtio-disk0': Unknown error 32758 (32758) engine.log: 2015-10-25 07:44:48,945 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-40) [a43dcc8] VM diataal-prod-cas1 77f07ae0-cc3e-4ae2-90ec-7fba7b11deeb moved from Up --> Paused 2015-10-25 07:44:49,003 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-40) [a43dcc8] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM diataal-prod-cas1 has paused due to unknown storage error. Has anyone experienced the same problem or knows a way to solve this? Kind regards, Jasper

3 3

Ovirt-engine-ha cannot to see live status of Hosted Engine
by asm＠pioner.kz 01 Feb '20

01 Feb '20

Good day for all. I have some issues with Ovirt 4.2.6. But now the main this of it: I have two Centos 7 Nodes with same config and last Ovirt 4.2.6 with Hostedengine with disk on NFS storage. Also some of virtual machines working good. But, when HostedEngine running on one node (srv02.local) everything is fine. After migrating to another node (srv00.local), i see that agent cannot to check livelinness of HostedEngine. After few minutes HostedEngine going to reboot and after some time i see some situation. After migration to another node (srv00.local) all looks OK. hosted-engine --vm-status commang when HosterEngine on srv00 node: --== Host 1 status ==-- conf_on_shared_storage : True Status up-to-date : True Hostname : srv02.local Host ID : 1 Engine status : {"reason": "vm not running on this host", "health": "bad", "vm": "down_unexpected", "detail": "unknown"} Score : 0 stopped : False Local maintenance : False crc32 : ecc7ad2d local_conf_timestamp : 78328 Host timestamp : 78328 Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=78328 (Tue Sep 18 12:44:18 2018) host-id=1 score=0 vm_conf_refresh_time=78328 (Tue Sep 18 12:44:18 2018) conf_on_shared_storage=True maintenance=False state=EngineUnexpectedlyDown stopped=False timeout=Fri Jan 2 03:49:58 1970 --== Host 2 status ==-- conf_on_shared_storage : True Status up-to-date : True Hostname : srv00.local Host ID : 2 Engine status : {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"} Score : 3400 stopped : False Local maintenance : False crc32 : 1d62b106 local_conf_timestamp : 326288 Host timestamp : 326288 Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=326288 (Tue Sep 18 12:44:21 2018) host-id=2 score=3400 vm_conf_refresh_time=326288 (Tue Sep 18 12:44:21 2018) conf_on_shared_storage=True maintenance=False state=EngineStarting stopped=False Log agent.log from srv00.local: MainThread::INFO::2018-09-18 12:40:51,749::states::779::ovirt_hosted_engine_ha.agent.hosted_engine.HostedE ngine::(consume) VM is powering up.. MainThread::INFO::2018-09-18 12:40:52,052::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine. HostedEngine::(_monitoring_loop) Current state EngineStarting (score: 3400) MainThread::INFO::2018-09-18 12:41:01,066::states::779::ovirt_hosted_engine_ha.agent.hosted_engine.HostedE ngine::(consume) VM is powering up.. MainThread::INFO::2018-09-18 12:41:01,374::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine. HostedEngine::(_monitoring_loop) Current state EngineStarting (score: 3400) MainThread::INFO::2018-09-18 12:41:11,393::state_machine::169::ovirt_hosted_engine_ha.agent.hosted_engine. HostedEngine::(refresh) Global metadata: {'maintenance': False} MainThread::INFO::2018-09-18 12:41:11,393::state_machine::174::ovirt_hosted_engine_ha.agent.hosted_engine. HostedEngine::(refresh) Host srv02.local.pioner.kz (id 1): {'conf_on_shared_storage': True, 'extra': 'meta data_parse_version=1\nmetadata_feature_version=1\ntimestamp=78128 (Tue Sep 18 12:40:58 2018)\nhost-id=1\ns core=0\nvm_conf_refresh_time=78128 (Tue Sep 18 12:40:58 2018)\nconf_on_shared_storage=True\nmaintenance=Fa lse\nstate=EngineUnexpectedlyDown\nstopped=False\ntimeout=Fri Jan 2 03:49:58 1970\n', 'hostname': 'srv02. local.pioner.kz', 'alive': True, 'host-id': 1, 'engine-status': {'reason': 'vm not running on this host', 'health': 'bad', 'vm': 'down_unexpected', 'detail': 'unknown'}, 'score': 0, 'stopped': False, 'maintenance ': False, 'crc32': 'e18e3f22', 'local_conf_timestamp': 78128, 'host-ts': 78128} MainThread::INFO::2018-09-18 12:41:11,393::state_machine::177::ovirt_hosted_engine_ha.agent.hosted_engine. HostedEngine::(refresh) Local (id 2): {'engine-health': {'reason': 'failed liveliness check', 'health': 'b ad', 'vm': 'up', 'detail': 'Up'}, 'bridge': True, 'mem-free': 12763.0, 'maintenance': False, 'cpu-load': 0 .0364, 'gateway': 1.0, 'storage-domain': True} MainThread::INFO::2018-09-18 12:41:11,393::states::779::ovirt_hosted_engine_ha.agent.hosted_engine.HostedE ngine::(consume) VM is powering up.. MainThread::INFO::2018-09-18 12:41:11,703::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine. HostedEngine::(_monitoring_loop) Current state EngineStarting (score: 3400) MainThread::INFO::2018-09-18 12:41:21,716::states::779::ovirt_hosted_engine_ha.agent.hosted_engine.HostedE ngine::(consume) VM is powering up.. MainThread::INFO::2018-09-18 12:41:22,020::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine. HostedEngine::(_monitoring_loop) Current state EngineStarting (score: 3400) MainThread::INFO::2018-09-18 12:41:31,033::states::779::ovirt_hosted_engine_ha.agent.hosted_engine.HostedE ngine::(consume) VM is powering up.. MainThread::INFO::2018-09-18 12:41:31,344::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine. HostedEngine::(_monitoring_loop) Current state EngineStarting (score: 3400) As we can see, agent thinking that HostedEngine just in powering up mode. I cannot to do anythink with it. I allready reinstalled many times srv00 node without success. One time i even has to uninstall ovirt* and vdsm* software. Also here one interesting point, after installing just "yum install http://resources.ovirt.org/pub/yum-repo/ovirt-release42.rpm" on this node i try to install this node from engine web interface with "Deploy" action. But, installation was unsuccesfull, before i didnt install ovirt-hosted-engine-ha on this node. I dont see in documentation that its need bofore installation of new hosts. But this is for information and checking. After installing ovirt-hosted-engine-ha node was installed with HostedEngine support. But the main issue not changed. Thanks in advance for help. BR, Alexandr

3 5

Hyperconverged setup - storage architecture - scaling
by Leo David 10 Jan '20

10 Jan '20

Hello Everyone, Reading through the document: "Red Hat Hyperconverged Infrastructure for Virtualization 1.5 Automating RHHI for Virtualization deployment" Regarding storage scaling, i see the following statements: *2.7. SCALINGRed Hat Hyperconverged Infrastructure for Virtualization is supported for one node, and for clusters of 3, 6, 9, and 12 nodes.The initial deployment is either 1 or 3 nodes.There are two supported methods of horizontally scaling Red Hat Hyperconverged Infrastructure for Virtualization:* *1 Add new hyperconverged nodes to the cluster, in sets of three, up to the maximum of 12 hyperconverged nodes.* *2 Create new Gluster volumes using new disks on existing hyperconverged nodes.You cannot create a volume that spans more than 3 nodes, or expand an existing volume so that it spans across more than 3 nodes at a time* *2.9.1. Prerequisites for geo-replicationBe aware of the following requirements and limitations when configuring geo-replication:One geo-replicated volume onlyRed Hat Hyperconverged Infrastructure for Virtualization (RHHI for Virtualization) supports only one geo-replicated volume. Red Hat recommends backing up the volume that stores the data of your virtual machines, as this is usually contains the most valuable data.* ------ Also in oVirtEngine UI, when I add a brick to an existing volume i get the following warning: *"Expanding gluster volume in a hyper-converged setup is not recommended as it could lead to degraded performance. To expand storage for cluster, it is advised to add additional gluster volumes." * Those things are raising a couple of questions that maybe for some for you guys are easy to answer, but for me it creates a bit of confusion... I am also referring to RedHat product documentation, because I treat oVirt as production-ready as RHHI is. *1*. Is there any reason for not going to distributed-replicated volumes ( ie: spread one volume across 6,9, or 12 nodes ) ? - ie: is recomanded that in a 9 nodes scenario I should have 3 separated volumes, but how should I deal with the folowing question *2.* If only one geo-replicated volume can be configured, how should I deal with 2nd and 3rd volume replication for disaster recovery *3.* If the limit of hosts per datacenter is 250, then (in theory ) the recomended way in reaching this treshold would be to create 20 separated oVirt logical clusters with 12 nodes per each ( and datacenter managed from one ha-engine ) ? *4.* In present, I have the folowing one 9 nodes cluster , all hosts contributing with 2 disks each to a single replica 3 distributed replicated volume. They where added to the volume in the following order: node1 - disk1 node2 - disk1 ...... node9 - disk1 node1 - disk2 node2 - disk2 ...... node9 - disk2 At the moment, the volume is arbitrated, but I intend to go for full distributed replica 3. Is this a bad setup ? Why ? It oviously brakes the redhat recommended rules... Is there anyone so kind to discuss on these things ? Thank you very much ! Leo -- Best regards, Leo David -- Best regards, Leo David

3 5

hyperconverged single node with SSD cache fails gluster creation
by thomas＠hoberg.net 01 Dec '19

01 Dec '19

I am seeing more success than failures at creating single and triple node hyperconverged setups after some weeks of experimentation so I am branching out to additional features: In this case the ability to use SSDs as cache media for hard disks. I tried first with a single node that combined caching and compression and that fails during the creation of LVMs. I tried again without the VDO compression, but actually the results where identical whilst VDO compression but without the LV cache worked ok. I tried various combinations, using less space etc., but the results are always the same and unfortunately rather cryptic (substituted the physical disk label with {disklabel}): TASK [gluster.infra/roles/backend_setup : Extend volume group] ***************** failed: [{hostname}] (item={u'vgname': u'gluster_vg_{disklabel}p1', u'cachethinpoolname': u'gluster_thinpool_gluster_vg_{disklabel}p1', u'cachelvname': u'cachelv_gluster_thinpool_gluster_vg_{disklabel}p1', u'cachedisk': u'/dev/sda4', u'cachemetalvname': u'cache_gluster_thinpool_gluster_vg_{disklabel}p1', u'cachemode': u'writeback', u'cachemetalvsize': u'70G', u'cachelvsize': u'630G'}) => {"ansible_loop_var": "item", "changed": false, "err": " Physical volume \"/dev/mapper/vdo_{disklabel}p1\" still in use\n", "item": {"cachedisk": "/dev/sda4", "cachelvname": "cachelv_gluster_thinpool_gluster_vg_{disklabel}p1", "cachelvsize": "630G", "cachemetalvname": "cache_gluster_thinpool_gluster_vg_{disklabel}p1", "cachemetalvsize": "70G", "cachemode": "writeback", "cachethinpoolname": "gluster_thinpool_gluster_vg_{disklabel}p1", "vgname": "gluster_vg_{disklabel}p1"}, "msg": "Unable to reduce gluster_vg_{disklabel}p1 by /dev/dm-15.", "rc": 5} somewhere within that I see something that points to a race condition ("still in use"). Unfortunately I have not been able to pinpoint the raw logs which are used at that stage and I wasn't able to obtain more info. At this point quite a bit of storage setup is already done, so rolling back for a clean new attempt, can be a bit complicated, with reboots to reconcile the kernel with data on disk. I don't actually believe it's related to single node and I'd be quite happy to move the creation of the SSD cache to a later stage, but in a VDO setup, this looks slightly complex to someone without intimate knowledge of LVS-with-cache-and-perhaps-thin/VDO/Gluster all thrown into one. Needless the feature set (SSD caching & compressed-dedup) sounds terribly attractive but when things don't just work, it's more terrifying.

5 7

How to pass parameters between VDSM Hooks domxml in single run
by Vrgotic, Marko 06 Nov '19

06 Nov '19

Dear oVIrt, A while ago we discussed on ways to change/update content of parameters of domxml in certain action. As I mentioned before, we have added the VDSMHook 60_nsupdate which removes the DNS record entries when a VM is destroyed: … domxml = hooking.read_domxml() name = domxml.getElementsByTagName('name')[0] name = " ".join(name.nodeValue for name in name.childNodes if name.nodeType == name.TEXT_NODE) nsupdate_commands = """server {server_ip} update delete {vm_name}.example.com a update delete {vm_name}. example.com aaaa update delete {vm_name}. example.com txt send """.format(server_ip="172.16.1.10", vm_name=name) … The goal: However, we did not want to execute remove dns records when VM is only migrated. Since its considered a “destroy” action we took following approach. * In state “before_vm_migrate_source add hook which will write flag “is_migration” to domxml * Once VM is scheduled for migration, this hook should add the flag “is_migration” to domxml * Once 60_nsupdate is triggered, it will check for the flag and if there, skip executing dns record action, but only remove the flag “is_migration” from domxml of the VM … domxml = hooking.read_domxml() migration = domxml.createElement("is_migration") domxml.getElementsByTagName("domain")[0].appendChild(migration) logging.info("domxml_updated {}".format(domxml.toprettyxml())) hooking.write_domxml(domxml) … When executing first time, we observed that flag “ <name>hookiesvm</name> <uuid>fcfa66cb-b251-43a3-8e2b-f33b3024a749</uuid> <metadata xmlns:ns0="http://ovirt.org/vm/tune/1.0" xmlns:ns1="http://ovirt.org/vm/1.0"> <ns0:qos/> <ovirt-vm:vm xmlns:ovirt-vm="http://ovirt.org/vm/1.0"> <ovirt-vm:clusterVersion>4.3</ovirt-vm:clusterVersion> <ovirt-vm:destroy_on_reboot type="bool">False</ovirt-vm:destroy_on_reboot> <ovirt-vm:launchPaused>false</ovirt-vm:launchPaused> <ovirt-vm:memGuaranteedSize type="int">1024</ovirt-vm:memGuaranteedSize> <ovirt-vm:minGuaranteedMemoryMb type="int">1024</ovirt-vm:minGuaranteedMemoryMb> ...skipping... <address bus="0x00" domain="0x0000" function="0x0" slot="0x09" type="pci"/> </rng> </devices> <seclabel model="selinux" relabel="yes" type="dynamic"> <label>system_u:system_r:svirt_t:s0:c169,c575</label> <imagelabel>system_u:object_r:svirt_image_t:s0:c169,c575</imagelabel> </seclabel> <seclabel model="dac" relabel="yes" type="dynamic"> <label>+107:+107</label> <imagelabel>+107:+107</imagelabel> </seclabel> <is_migration/> </domain> is added to domxml, but was present once 60_nsupdate hook was executed. The question: How do we make sure that, when domxml is updated, that the update is visible/usable by following hook, in single run? How to pass these changes between hooks? Kindly awaiting your reply. — — — Met vriendelijke groet / Kind regards, Marko Vrgotic

2 6

There are too many tasks running on the server. Does it affect my system?
by zhouhao＠vip.friendtimes.net 18 Oct '19

18 Oct '19

There are too many tasks running on the server. Does it affect my system？

2 3

disk locked after export as OVA
by adam_xu＠adagene.com.cn 15 Oct '19

15 Oct '19

Hello, everyone. I tried to export a vm as OVA. I got a error in webui: VDSM ovirt1.ntbaobei.com command HSMGetAllTasksStatusesVDS failed: Volume Group not big enough: (u'Not enough free extents for extending LV d9f2378f-92f0-4bc1-96a8-20a0f2c575cb/c515a3f9-0590-4ebe-81c5-9d0993f1fec9 (free=848, needed=872)',) seems no enough space in the storage domain. I deleted the vm which i wanted to export as OVA before, but I saw a disk with id 8140d2e7-6908-438e-a646-23e58a33913e left in the storage and the status is locked. here's some log in the engine.log: 2018-09-17 14:27:03,467+08 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (EE-ManagedThreadFactory-engine-Thread-107849) [a407f6a9-445d-4ccc-b998-f9dfc6dc67ec] START, CopyImageVDSCommand( CopyImageVDSCommandParameters:{storagePoolId='79432500-ad45-11e8-98f3-00163e188641', ignoreFailoverLimit='false', storageDomainId='d9f2378f-92f0-4bc1-96a8-20a0f2c575cb', imageGroupId='ee55bbe7-8001-4c82-b1a6-0cac7710704b', imageId='701a7472-8c00-4b9f-aa87-4837eb128695', dstImageGroupId='8140d2e7-6908-438e-a646-23e58a33913e', vmId='b3aa623e-3072-430e-87d6-b81a3b07f466', dstImageId='c515a3f9-0590-4ebe-81c5-9d0993f1fec9', imageDescription='', dstStorageDomainId='d9f2378f-92f0-4bc1-96a8-20a0f2c575cb', copyVolumeType='LeafVol', volumeFormat='COW', preallocate='Sparse', postZero='false', discard='false', force='true'}), log id: 164e3046 2018-09-17 14:27:03,467+08 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (EE-ManagedThreadFactory-engine-Thread-107849) [a407f6a9-445d-4ccc-b998-f9dfc6dc67ec] ++ dstImageGUID=8140d2e7-6908-438e-a646-23e58a33913e I think maybe some command did not been excuted after I export the vm as OVA. I can find the similar case here: https://access.redhat.com/solutions/1298893 I have used the tool unlock_entity.sh but cannot find any disk that is locked. So what should I do to delete the disk that is been locked? yours Adam

2 1

Re: change connection string in db
by Strahil 14 Oct '19

14 Oct '19

I think that there is a special file on the shared storage (something like OVA) that stores the necessary data. Have you tried with: hosted-engine --set-shared-config storage <server1>:/engine hosted-engine --set-shared-config mnt_options backup-volfile-servers=<server2>:<server3> Best Regards, Strahil NikolovOn Sep 30, 2019 19:14, olaf.buitelaar(a)gmail.com wrote: > > Dear oVirt users, > > one thing i still cannot find out, is where the engine gathers the storage= value from in the /etc/ovirt-hosted-engine/hosted-engine.conf > I suppose it's somewhere in a answers file, but i cannot find it. > Any points are appreciated. hopfully this is the last place where the old connection string lives. > > Thanks Olaf > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/GPPKRN5YLUMEJ…

2 1

Re: VDI
by Alex McWhirter 09 Oct '19

09 Oct '19

To achieve that all you need to do is create a template of the desktop base vm, make sure the vm type is set to desktop. Afterwards just create new vms from that template. As long as the VM type is set to desktop each new VM will use a qcow overlay on top of the base image. Taking this a step further you can then create VM pools from said template, allowing users to dynamically be assigned a new VM on login. Granted pools are usually stateless, so you need to have network file storage. We use pools for windows 10 VDI instances, where we use sysprep to autojoin the new pool vm to the domain where redirected folders are already setup. For VDI only use spice protocol. By default we found spice to be semi lackluster, so we do apply custom settings and we have recompiled spice on both servers and clients with h264 support. This is not 100% necessary, but makes things like youtube much more usable. We have also backported some audio patches to KVM. CentOS 8 should resolve a lot of these customizations that we've had to do. As far as updating, pretty much. We create a VM from the template, update it, then push it back as a new version of the template. The pools are set to always use the latest template version. Users have to log out, then back in to the VDI system in order to get the new image as logging out will destroy the users current instance and create a new one on log in. On 2019-09-23 15:16, Fabio Marzocca wrote: > Hi Alex, thanks for answering. > > I am approaching and studying oVirt in order to propose the solution to a customer as a replacement for a commercial solution they have now. > They only need Desktop virtualization. > Sorry for the silly question, but I can't find a way to deploy a VM (template) to users as a "linked-clone", meaning that the users' image still refers to the original image but modification are written (and afterwards read) from a new location. This technique is called Copy-on-write. > Can this be achieved with oVirt? > > Then, what is the Best Practice to update WIndows OS for the all the users? Currently they simply "check-out" the Gold Image, update it and check-in, while all users are running... > > Fabio > > On Mon, Sep 23, 2019 at 8:04 PM Alex McWhirter <Alex(a)triadic.us> wrote: > > yes, we do. All spice, with some customizations done at source level for spice / kvm packages. > > On 2019-09-23 13:44, Fabio Marzocca wrote: > Is there anyone who uses oVirt as a full VDI environment? I would have a bunch of questions... > > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/D44YB5VOKNBNC…

5 4

oVirt Software after upgrade
by Rick A 07 Oct '19

07 Oct '19

I'm wondering why after upgrading our hosts, when viewing the host details, it's still showing the OS Description as 4.3.3.1. We are using the oVirt Node. Shouldn't the OS Description show the version we just upgraded to? I do the upgrade using the web UI. Thanks OS Version: RHEL - 7 - 6.1810.2.el7.centos OS Description: oVirt Node 4.3.3.1 <-------------------------------------- Kernel Version: 3.10.0 - 957.10.1.el7.x86_64 KVM Version: 2.12.0 - 18.el7_6.3.1 LIBVIRT Version: libvirt-4.5.0-10.el7_6.6 VDSM Version: vdsm-4.30.13-1.el7 SPICE Version: 0.14.0 - 6.el7_6.1 GlusterFS Version: glusterfs-5.5-1.el7 CEPH Version: librbd1-10.2.5-4.el7 Open vSwitch Version: openvswitch-2.10.1-3.el7 Kernel Features: PTI: 1, IBRS: 1, RETP: 0, SSBD: 3 VNC Encryption: Disabled

2 1

Managed Block Storage/Ceph: Experiences from Catastrophic Hardware failure
by Dan Poltawski 06 Oct '19

06 Oct '19

Yesterday we had a catastrophic hardware failure with one of our nodes using ceph and the experimental cinderlib integration. Unfortunately the ovirt cluster recover the situation well and took some manual intervention to resolve. I thought I'd share what happened and how we resolved it in case there is any best practice to share/bugs which are worth creating to help others in similar situaiton. We are early in our use of ovirt, so its quite possible we have things incorreclty configured. Our setup: We have two nodes, hosted engine on iSCSI, about 40vms all using managed block storage mounting the rbd volumes directly. I hadn't configured power management (perhaps this is the fundamental problem). Yesterday a hardware fault caused one of the nodes to crash and stay down awaiting user input in POST screens, taking 20 vms with it. The hosted engine was fortunately on the 'good' node and detected that the node had become unresponsive, but noted 'Host cannot be fenced automatically because power management for the host is disabled.'. At this point, knowing that one node was dead, I wanted to bring up the failed vms on the good node. However, the vms were appearing in an unknown state and I couldn't do any operations on them. It wasn't clear to me what the best course of action to do there would be. I am not sure if there is a way to mark the node as failed? In my urgency to try and resolve the situation I managed to get the failed node startred back up, shortly after it came up the engine detected that all the vms were down, I put the failed host into maintaince mode and tried to start the failed vms. Unfortunately the failed vms did not start up cleanly - it turned out that they still had rbd locks preventing writing from the failed node. To finally gets the vms to start I then manually went through every vm's managed block, found the id and found the lock and removed it: rbd lock list rbd/volume-{id} rbd lock remove rbd/voleume-{id} 'auto {lockid}' {lockername} Some overall thoughts I had: * I'm not sure what the best course of action is to notify the engine about a catastrophic hardware failure? If power management was configured, I suppose it would've removed the power and marked them all down? * Would ovirt have been able to deal with clearing the rbd locks, or did I miss a trick somewhere to resolve this situation with manually going through each device and clering the lock? * Might it be possible for ovirt to detect when the rbd images are locked for writing and prevent launching? regards, Dan ________________________________ The Networking People (TNP) Limited. Registered office: Network House, Caton Rd, Lancaster, LA1 3PE. Registered in England & Wales with company number: 07667393 This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.

3 2

[ANN] oVirt 4.3.6 is now generally available
by Sandro Bonazzola 06 Oct '19

06 Oct '19

The oVirt Project is pleased to announce the general availability of oVirt 4.3.6 as of September 26th, 2019. This update is the sixth in a series of stabilization updates to the 4.3 series. This release is available now on x86_64 architecture for: * Red Hat Enterprise Linux 7.7 or later (but < 8) * CentOS Linux (or similar) 7.7 or later (but < 8) This release supports Hypervisor Hosts on x86_64 and ppc64le architectures for: * Red Hat Enterprise Linux 7.7 or later (but < 8) * CentOS Linux (or similar) 7.7 or later (but < 8) * oVirt Node 4.3 (available for x86_64 only) Due to Fedora 28 being now at end of life this release is missing experimental tech preview for x86_64 and s390x architectures for Fedora 28. We are working on Fedora 29 and 30 support and we may re-introduce experimental support for Fedora in next release. See the release notes [1] for installation / upgrade instructions and a list of new features and bugs fixed. Notes: - oVirt Appliance is already available - oVirt Node is already available[2] oVirt Node and Appliance have been updated including: - oVirt 4.3.6: http://www.ovirt.org/release/4.3.6/ - Wildfly 17.0.1: https://wildfly.org/news/2019/07/07/WildFly-1701-Released/ - Latest CentOS 7.7 updates including: - Release for CentOS Linux 7 (1908) on the x86_64 Architecture <https://lists.centos.org/pipermail/centos-announce/2019-September/023405.ht…> - CEBA-2019:2601 CentOS 7 NetworkManager BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023423.ht…> - CEBA-2019:2023 CentOS 7 efivar BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023445.ht…> - CEBA-2019:2614 CentOS 7 firewalld BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023412.ht…> - CEBA-2019:2227 CentOS 7 grubby BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023441.ht…> - CESA-2019:2258 Moderate CentOS 7 http-parser Security Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023439.ht…> - CESA-2019:2600 Important CentOS 7 kernel Security Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023444.ht…> - CEBA-2019:2599 CentOS 7 krb5 BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023420.ht…> - CEBA-2019:2358 CentOS 7 libguestfs BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023421.ht…> - CEBA-2019:2679 CentOS 7 libvirt BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023422.ht…> - CEBA-2019:2501 CentOS 7 rsyslog BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023431.ht…> - CEBA-2019:2355 CentOS 7 selinux-policy BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023432.ht…> - CEBA-2019:2612 CentOS 7 sg3_utils BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023433.ht…> - CEBA-2019:2602 CentOS 7 sos BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023434.ht…> - CEBA-2019:2564 CentOS 7 subscription-manager BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023435.ht…> - CEBA-2019:2356 CentOS 7 systemd BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023436.ht…> - CEBA-2019:2605 CentOS 7 tuned BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023437.ht…> - CEBA-2019:2871 CentOS 7 tzdata BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023450.ht…> - latest CentOS Virt and Storage SIG updates: - Ansible 2.8.5: https://github.com/ansible/ansible/blob/stable-2.8/changelogs/CHANGELOG-v2.… - Glusterfs 6.5: https://docs.gluster.org/en/latest/release-notes/6.5/ - QEMU KVM EV 2.12.0-33.1 : https://cbs.centos.org/koji/buildinfo?buildID=26484 Given the amount of security fixes provided by this release, upgrade is recommended as soon as practical. Additional Resources: * Read more about the oVirt 4.3.6 release highlights: http://www.ovirt.org/release/4.3.6/ * Get more oVirt Project updates on Twitter: https://twitter.com/ovirt * Check out the latest project news on the oVirt blog: http://www.ovirt.org/blog/ [1] http://www.ovirt.org/release/4.3.6/ [2] http://resources.ovirt.org/pub/ovirt-4.3/iso/ -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/>*Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*

13 33

oVirt 4.3.5 glusterfs 6.3 performance tunning
by adrianquintero＠gmail.com 04 Oct '19

04 Oct '19

Hello, I have a hyperconverged setup using ovirt 4.3.5 and the "optimize for ovirt store" seems to fail on gluster volumes. I am seeing poor performance and trying to see how should I tune gluster to give better performance. Can you provide any suggestions on the following volume settings(parameters)? option Value ------ ----- cluster.lookup-unhashed on cluster.lookup-optimize on cluster.min-free-disk 10% cluster.min-free-inodes 5% cluster.rebalance-stats off cluster.subvols-per-directory (null) cluster.readdir-optimize off cluster.rsync-hash-regex (null) cluster.extra-hash-regex (null) cluster.dht-xattr-name trusted.glusterfs.dht cluster.randomize-hash-range-by-gfid off cluster.rebal-throttle normal cluster.lock-migration off cluster.force-migration off cluster.local-volume-name (null) cluster.weighted-rebalance on cluster.switch-pattern (null) cluster.entry-change-log on cluster.read-subvolume (null) cluster.read-subvolume-index -1 cluster.read-hash-mode 1 cluster.background-self-heal-count 8 cluster.metadata-self-heal off cluster.data-self-heal off cluster.entry-self-heal off cluster.self-heal-daemon on cluster.heal-timeout 600 cluster.self-heal-window-size 1 cluster.data-change-log on cluster.metadata-change-log on cluster.data-self-heal-algorithm full cluster.eager-lock enable disperse.eager-lock on disperse.other-eager-lock on disperse.eager-lock-timeout 1 disperse.other-eager-lock-timeout 1 cluster.quorum-type auto cluster.quorum-count (null) cluster.choose-local off cluster.self-heal-readdir-size 1KB cluster.post-op-delay-secs 1 cluster.ensure-durability on cluster.consistent-metadata no cluster.heal-wait-queue-length 128 cluster.favorite-child-policy none cluster.full-lock yes diagnostics.latency-measurement off diagnostics.dump-fd-stats off diagnostics.count-fop-hits off diagnostics.brick-log-level INFO diagnostics.client-log-level INFO diagnostics.brick-sys-log-level CRITICAL diagnostics.client-sys-log-level CRITICAL diagnostics.brick-logger (null) diagnostics.client-logger (null) diagnostics.brick-log-format (null) diagnostics.client-log-format (null) diagnostics.brick-log-buf-size 5 diagnostics.client-log-buf-size 5 diagnostics.brick-log-flush-timeout 120 diagnostics.client-log-flush-timeout 120 diagnostics.stats-dump-interval 0 diagnostics.fop-sample-interval 0 diagnostics.stats-dump-format json diagnostics.fop-sample-buf-size 65535 diagnostics.stats-dnscache-ttl-sec 86400 performance.cache-max-file-size 0 performance.cache-min-file-size 0 performance.cache-refresh-timeout 1 performance.cache-priority performance.cache-size 32MB performance.io-thread-count 16 performance.high-prio-threads 16 performance.normal-prio-threads 16 performance.low-prio-threads 32 performance.least-prio-threads 1 performance.enable-least-priority on performance.iot-watchdog-secs (null) performance.iot-cleanup-disconnected- reqsoff performance.iot-pass-through false performance.io-cache-pass-through false performance.cache-size 128MB performance.qr-cache-timeout 1 performance.cache-invalidation false performance.ctime-invalidation false performance.flush-behind on performance.nfs.flush-behind on performance.write-behind-window-size 1MB performance.resync-failed-syncs-after -fsyncoff performance.nfs.write-behind-window-s ize1MB performance.strict-o-direct on performance.nfs.strict-o-direct off performance.strict-write-ordering off performance.nfs.strict-write-ordering off performance.write-behind-trickling-wr iteson performance.aggregate-size 128KB performance.nfs.write-behind-tricklin g-writeson performance.lazy-open yes performance.read-after-open yes performance.open-behind-pass-through false performance.read-ahead-page-count 4 performance.read-ahead-pass-through false performance.readdir-ahead-pass-throug h false performance.md-cache-pass-through false performance.md-cache-timeout 1 performance.cache-swift-metadata true performance.cache-samba-metadata false performance.cache-capability-xattrs true performance.cache-ima-xattrs true performance.md-cache-statfs off performance.xattr-cache-list performance.nl-cache-pass-through false features.encryption off network.frame-timeout 1800 network.ping-timeout 30 network.tcp-window-size (null) client.ssl off network.remote-dio off client.event-threads 4 client.tcp-user-timeout 0 client.keepalive-time 20 client.keepalive-interval 2 client.keepalive-count 9 network.tcp-window-size (null) network.inode-lru-limit 16384 auth.allow * auth.reject (null) transport.keepalive 1 server.allow-insecure on server.root-squash off server.all-squash off server.anonuid 65534 server.anongid 65534 server.statedump-path /var/run/gluster server.outstanding-rpc-limit 64 server.ssl off auth.ssl-allow * server.manage-gids off server.dynamic-auth on client.send-gids on server.gid-timeout 300 server.own-thread (null) server.event-threads 4 server.tcp-user-timeout 42 server.keepalive-time 20 server.keepalive-interval 2 server.keepalive-count 9 transport.listen-backlog 1024 transport.address-family inet performance.write-behind on performance.read-ahead off performance.readdir-ahead on performance.io-cache off performance.open-behind on performance.quick-read off performance.nl-cache off performance.stat-prefetch on performance.client-io-threads on performance.nfs.write-behind on performance.nfs.read-ahead off performance.nfs.io-cache off performance.nfs.quick-read off performance.nfs.stat-prefetch off performance.nfs.io-threads off performance.force-readdirp true performance.cache-invalidation false performance.global-cache-invalidation true features.uss off features.snapshot-directory .snaps features.show-snapshot-directory off features.tag-namespaces off network.compression off network.compression.window-size -15 network.compression.mem-level 8 network.compression.min-size 0 network.compression.compression-level -1 network.compression.debug false features.default-soft-limit 80% features.soft-timeout 60 features.hard-timeout 5 features.alert-time 86400 features.quota-deem-statfs off geo-replication.indexing off geo-replication.indexing off geo-replication.ignore-pid-check off geo-replication.ignore-pid-check off features.quota off features.inode-quota off features.bitrot disable debug.trace off debug.log-history no debug.log-file no debug.exclude-ops (null) debug.include-ops (null) debug.error-gen off debug.error-failure (null) debug.error-number (null) debug.random-failure off debug.error-fops (null) nfs.disable on features.read-only off features.worm off features.worm-file-level off features.worm-files-deletable on features.default-retention-period 120 features.retention-mode relax features.auto-commit-period 180 storage.linux-aio off storage.batch-fsync-mode reverse-fsync storage.batch-fsync-delay-usec 0 storage.owner-uid 36 storage.owner-gid 36 storage.node-uuid-pathinfo off storage.health-check-interval 30 storage.build-pgfid off storage.gfid2path on storage.gfid2path-separator : storage.reserve 1 storage.health-check-timeout 10 storage.fips-mode-rchecksum off storage.force-create-mode 0 storage.force-directory-mode 0 storage.create-mask 777 storage.create-directory-mask 777 storage.max-hardlinks 100 features.ctime on config.gfproxyd off cluster.server-quorum-type server cluster.server-quorum-ratio 0 changelog.changelog off changelog.changelog-dir {{ brick.path }}/.glusterfs/changelogs changelog.encoding ascii changelog.rollover-time 15 changelog.fsync-interval 5 changelog.changelog-barrier-timeout 120 changelog.capture-del-path off features.barrier disable features.barrier-timeout 120 features.trash off features.trash-dir .trashcan features.trash-eliminate-path (null) features.trash-max-filesize 5MB features.trash-internal-op off cluster.enable-shared-storage disable locks.trace off locks.mandatory-locking off cluster.disperse-self-heal-daemon enable cluster.quorum-reads no client.bind-insecure (null) features.shard on features.shard-block-size 64MB features.shard-lru-limit 16384 features.shard-deletion-rate 100 features.scrub-throttle lazy features.scrub-freq biweekly features.scrub false features.expiry-time 120 features.cache-invalidation off features.cache-invalidation-timeout 60 features.leases off features.lease-lock-recall-timeout 60 disperse.background-heals 8 disperse.heal-wait-qlength 128 cluster.heal-timeout 600 dht.force-readdirp on disperse.read-policy gfid-hash cluster.shd-max-threads 8 cluster.shd-wait-qlength 10000 cluster.locking-scheme granular cluster.granular-entry-heal enable features.locks-revocation-secs 0 features.locks-revocation-clear-all false features.locks-revocation-max-blocked 0 features.locks-monkey-unlocking false features.locks-notify-contention no features.locks-notify-contention-dela y 5 disperse.shd-max-threads 1 disperse.shd-wait-qlength 1024 disperse.cpu-extensions auto disperse.self-heal-window-size 1 cluster.use-compound-fops off performance.parallel-readdir off performance.rda-request-size 131072 performance.rda-low-wmark 4096 performance.rda-high-wmark 128KB performance.rda-cache-limit 10MB performance.nl-cache-positive-entry false performance.nl-cache-limit 10MB performance.nl-cache-timeout 60 cluster.brick-multiplex off cluster.max-bricks-per-process 250 disperse.optimistic-change-log on disperse.stripe-cache 4 cluster.halo-enabled False cluster.halo-shd-max-latency 99999 cluster.halo-nfsd-max-latency 5 cluster.halo-max-latency 5 cluster.halo-max-replicas 99999 cluster.halo-min-replicas 2 features.selinux on cluster.daemon-log-level INFO debug.delay-gen off delay-gen.delay-percentage 10% delay-gen.delay-duration 100000 delay-gen.enable disperse.parallel-writes on features.sdfs off features.cloudsync off features.ctime on ctime.noatime on feature.cloudsync-storetype (null) features.enforce-mandatory-lock off thank you!

2 2

oVirt 4.3.5 WARN no gluster network found in cluster
by adrianquintero＠gmail.com 03 Oct '19

03 Oct '19

Hi, I have a 4.3.5 hyperconverged setup with 3 hosts, each host has 2x10G NIC ports Host1: NIC1: 192.168.1.11 NIC2: 192.168.0.67 (Gluster) Host2: NIC1: 10.10.1.12 NIC2: 192.168.0.68 (Gluster) Host3: NIC1: 10.10.1.13 NIC2: 192.168.0.69 (Gluster) I am able to ping all the gluster IPs from within the hosts. i.e from host1 i can ping 192.168.0.68 and 192.168.0.69 However from the HostedEngine VM I cant ping any of those IPs [root@ovirt-engine ~]# ping 192.168.0.9 PING 192.168.0.60 (192.168.0.60) 56(84) bytes of data. From 10.10.255.5 icmp_seq=1 Time to live exceeded and on the HostedEngine I see the following WARNINGS (only for host1) which make me think that I am not using a separate network exclusively for gluster. 2019-08-21 21:04:34,215-04 WARN [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturn] (DefaultQuartzScheduler10) [397b6038] Could not associate brick 'host1.example.com:/gluster_bricks/engine/engine' of volume 'ac1f73ce-cdf0-4bb9-817d-xxxxxxcxxx' with correct network as no gluster network found in cluster 'xxxxxxxx11e9-b8d3-00163e5d860d' Any ideas? thank you!! 2019-08-21 21:04:34,220-04 WARN [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturn] (DefaultQuartzScheduler10) [397b6038] Could not associate brick 'vmm01.virt.ord1d:/gluster_bricks/data/data' of volume 'bc26633a-9a0b-49de-b714-97e76f222a02' with correct network as no gluster network found in cluster 'e98e2c16-c31e-11e9-b8d3-00163e5d860d' 2019-08-21 21:04:34,224-04 WARN [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturn] (DefaultQuartzScheduler10) [397b6038] Could not associate brick 'host1:/gluster_bricks/vmstore/vmstore' of volume 'xxxxxxxxx-ca96-45cc-9e0f-649055e0e07b' with correct network as no gluster network found in cluster 'e98e2c16-c31e-11e9-b8d3xxxxxxxxxxxxxxx'

3 4

web admin: snapshots UI
by Yedidyah Bar David 03 Oct '19

03 Oct '19

Hi all, I'd like to ask/suggest something: In the previous UI (4.2?), when you entered the VM snapshots page, you saw per each snapshot some details that you do not see in the current UI. Specifically, to see the creation date, you have to press General. So I got used to writing longer descriptions, with the creation date in them, but that also does not work well, because in the main view only the start of the description is shown. At least for me (combination of browser, OS, etc.), there is a lot of white unused space between the end of the presented-part of the description, and the "> General" after it. Can we somehow use this space to either add the creation date, or more/all of the description, or even make this customizable? And perhaps make this a sortable table (so that you can sort by description, or if you added Data, you can sort by that, or memory, or whatever)? I don't mind opening a bug/RFE for this, if more people agree that it's useful, and unless there are some (unknown to me) reasons against that (other than the time needed for implementing this, which I expect is quite minimal for the simplest option of just showing more of the description). Thanks and best regards, -- Didi

2 2

UX team is looking for feedback and your thoughts around user settings!
by Sandro Bonazzola 02 Oct '19

02 Oct '19

oVirt UX team at Red Hat is currently looking for feedback and thoughts on the area of user settings within oVirt, both the VM and Administrator portal. If you have any ideas around user settings, we would love to hear from you! Please follow the link below[1] which will take you to a short survey that will ask some questions around proposed settings and give you the opportunity to suggest settings you would want to use. Thank you! [1] https://redhatdg.co1.qualtrics.com/jfe/form/SV_a9uTOLlQ41lV7KJ -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/>*Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

2 1

Incremental backup using ovirt api
by smidhunraj＠gmail.com 02 Oct '19

02 Oct '19

Hi, I tried to take incremental backup of a vm using this script. public function downloadDiskIncremental(){ $data=array(); $xmlStr = "<disk_attachment> <disk id='1371afdd-91d4-4bc7-9792-0efbf6bbd1c9'> <backup>incremental</backup> </disk> </disk_attachment>"; $curlParam=array( "url" => "vms/4044e014-7e20-4dbc-abe5-64690ec45f63/diskattachments", "method" => "POST", "data" =>$xmlStr, ); } But it is throwing me error as Array ( [status] => error [message] => For correct usage, see: https://ovirt.bobcares.com/ovirt-engine/api/v4/model#services/disk-attachme… Please help me with this issue....

2 1

change connection string in db
by olaf.buitelaar＠gmail.com 01 Oct '19

01 Oct '19

Dear oVirt users, I'm currently migrating our gluster setup, so i've done a gluster replace brick to the new machines. Now i'm trying to update the connection strings of the related storage domains including the one hosting the ovirt-engine (which i believe cannot be brought down for maintenance). At the same time i'm trying to disable the "Use managed gluster volume" feature. i've had tested this in a lab setup, but somehow i'm running into issues on the actual setup. On the lab setup it was enough to run a query like this; UPDATE public.storage_server_connections SET "connection"='10.201.0.6:/ovirt-kube',gluster_volume_id=NULL,mount_options='backup-volfile-servers=10.201.0.1:10.201.0.2:10.201.0.3:10.201.0.5:10.201.0.4:10.201.0.7:10.201.0.8:10.201.0.9' WHERE id='29aae3ce-61e4-4fcd-a8f2-ab0a0c07fa48'; on the live setup i also seem to run a query like this; UPDATE public.gluster_volumes SET task_id=NULL WHERE id='9a552d7a-8a0d-4bae-b5a2-1cb8a7edf5c9'; i couldn't really find where this task_id relates to, but it does make the checkbox for "Use managed gluster volume" being unchecked in the web interface. in the lab setup it was enough to run within the hosted engine; - service ovirt-engine restart and then bring an ovirt-host machine to maintenance, and active it again. and the changed connection string was being mounted in the /rhev/data-center/mnt/glusterSD/ directory. Also the vm's after being shutdown and brought up again, started using the new connection string. But now on the production instance, when i restart the engine the connection string is restored to the original values in the storage_server_connections table. I don't really understand where the engine gathers this information from. Any advice on how to actually change the connection strings would by highly appreciated. Thanks Olaf

1 3

Creating vm from snapshot via API
by Aleksei Ivanovich 01 Oct '19

01 Oct '19

Hi. I'm trying to create vm from a snapshot using API. If I just provide the new vm name, source snapshot ID and cluster name the new vm is been created successfully on the same storage domain. But if I additionally provide diskattachments to create it on another storage I get error: "Cannot add VM. One or more provided storage domains are either not in active status or of an illegal type for the requested operation". I can't find the reason why I can't create vm on another storage via API while I can do the same via web-portal. My oVirt Engine version is 4.1.1.8-1.el7.centos. Here is my code: curl.exe -k -u "user@domain.suffix:*************" -H "Content-Type: application/xml" -H "Accept: application/xml" -X POST https://192.168.0.1/ovirt-engine/api/vms -d "<vm><name>vm-Test1</name><snapshots><snapshot id=\"4b73c79b-efed-4362-8843-91f5cc2d9775\"/></snapshots><cluster><name>MyCluster</name></cluster><disk_attachments><disk_attachment><disk><image_id>6525c157-4e73-440b-adc8-596f1d03ef3c</image_id><storage_domains><storage_domain id=\"52a003d7-cec1-473e-b521-1a551f55abd1\"/></storage_domains></disk></disk_attachment></disk_attachments></vm>" The answer is: <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <fault> <detail>[Cannot add VM. One or more provided storage domains are either not in active status or of an illegal type for the requested operation.]</detail> <reason>Operation Failed</reason> </fault>

1 2

Re: Ovirt 4.2.7 won't start and drops to emergency console
by Strahil 30 Sep '19

30 Sep '19

What happens when it complain that there is no VGs ? When you run 'vgs' what is the output? Also, take a look into https://www.redhat.com/archives/linux-lvm/2016-February/msg00012.html I have the feeling that you need to disable all lvs - not only the thin pool, but also the thin LVs (first). Best Regards, Strahil NikolovOn Sep 29, 2019 23:00, jeremy_tourville(a)hotmail.com wrote: > > Yes, I can take the downtime. Actually, I don't have any choice at the moment because it is a single node setup. :) I think this is a distributed volume from the research I have performed. I posted the lvchange command in my last post, this was the result- I ran the command lvchange -an /dev/gluster_vg1/lv_datadisks When I do this I get the message "Volume group "gluster_vg1" not found. Cannot process volume group gluster_vg1". I also tried the command the way you specified with just the LV and get the same results. > > I had placed the system in Global maintenance mode prior to the reboot. Upon reboot I got the messages about the various gluster volumes not being able to be mounted because of timeout issues. That is what started my OP. I think we are both thinking along the same lines regarding the issue. I think the question is how do you fix a volume that the system won't mount? It does seem likely that the thinpool needs to be repaired but what do you do if you can't even perform the first step in the procedure? > > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/2X73CAASB4UCO…

2 1

Re: How should ovirt-engine be repaired if it is hung up
by Strahil 30 Sep '19

30 Sep '19

1. Open VNC console and check the situation. Just like any other VM. 2. You can get the config via checking the vdsm log on the host where the system has started properly last time (as current situation is bad, check previous power up event). You can save the xml and use that to define the VM in virsh, but you also need to define ovirtmgmt network -> check in the web , as I don't have access to my systems right now. If virsh complaints about the disks - check that the storage domains' link is properly set to your storage. Best Regards, Strahil NikolovOn Sep 30, 2019 05:49, wangyu13476969128(a)126.com wrote: > > 1. How should ovirt-engine be repaired if it is hung up? > 2. How do I export the ovirt-engine configuration file? > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/XNY4GUISXGCDY…

1 0

How should ovirt-engine be repaired if it is hung up
by wangyu13476969128＠126.com 30 Sep '19

30 Sep '19

1. How should ovirt-engine be repaired if it is hung up? 2. How do I export the ovirt-engine configuration file?

2 2

Re: Ovirt 4.2.7 won't start and drops to emergency console
by Strahil 29 Sep '19

29 Sep '19

Can you suffer downtime ? You can try something like this (I'm improvising): Set to global maintenance (either via UI or hosted-engine --set-maintenance --mode=global) Stop the engine. Stop ovirt-ha-agent ovirt-ha-broker vdsmd supervdsmd sanlock glusterd. Stop all gluster processes via thr script in /usr/share/gluster*..../something-named-kill Disable all LVs in the vg via: lvchange -an LV If it doesn't work try disabling the VG: vgchange -an VG and then enabling it again: vgchange -ay VG Last try to repair the thinpool via: lvchange --repair VG/thinpool Last disable the VG and give the system a reboot to verify everything works. Best Regards, Strahil NikolovOn Sep 29, 2019 14:39, jeremy_tourville(a)hotmail.com wrote: > > Thank you for the reply. Please pardon my ignorance, I'm not very good with GlusterFS. I don't think this is a replicated volume (though I could be wrong) I built a single node hyperconverged hypervisor. I was reviewing my gdeploy file from when I originally built the system. I have the following values: > PV = /dev/sda > VG1= gluster_vg1 > LV1= engine_lv (thick) > LV2 = gluster_vg1 thinpool > LV3 = lv_vmdisks (thinpooll) > LV4 = lv_datadisks (thinpool) > > So according to the article I read in my OP, it says to deactivate the volumes under the thin pool as the first step. I ran the command lvchange -an /dev/gluster_vg1/lv_datadisks When I do this I am told Volume group "gluster_vg1" not found. Cannot process volume group gluster_vg1. > > That seems consistent with the timeout error message. How do you fix this if you can't access the volumes? Thoughts? > > > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/HZXE3NCSFWCUI…

2 1

Cannot enable maintenance mode
by Bruno Martins 29 Sep '19

29 Sep '19

Hello guys, I am being unable to put a host from a two nodes cluster into maintenance mode in order to remove it from the cluster afterwards. This is what I see in engine.log: 2019-09-27 16:20:58,364 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-45) [4cc251c9] Correlation ID: 4cc251c9, Job ID: 65731fbb-db34-49a9-ab56-9fba59bc0ee0, Call Stack: null, Custom Event ID: -1, Message: Host CentOS-H1 cannot change into maintenance mode - not all Vms have been migrated successfully. Consider manual intervention: stopping/migrating Vms: Non interactive user (User: admin). Host has been rebooted multiple times. vdsClient shows no VM's running. What else can I do? Kind regards, Bruno Martins

1 0

Re: Ovirt 4.2.7 won't start and drops to emergency console
by Strahil 29 Sep '19

29 Sep '19

If it's a replicated volume - then you can safely rebuild your bricks and don't even tryhto repair. There is no guarantee that the issue will not reoccur. Best Regards, Strahil NikolovOn Sep 29, 2019 00:22, jeremy_tourville(a)hotmail.com wrote: > > I see evidence that appears to be a problem with gluster. /var/log/messages has multiple occurrences of: WARNING: /dev/gluster_vg1/lv_<name>: Thin's tihin-pool needs inspection. Also, if I run vgs I am returned info for my other volume groups but not gluster_vg1. Lastly, when reviewing journalctl -xb | grep -i timed I see messages from systemd > Job dev-gluster_vg1-lv_vmdisks.device/start timed.out. > Timed out waiting for device dev-glustet_vg1-lv_vmdisks.device > > Thses messages are happening for both > /dev/gluster_vg1/lv_datadisks > /dev/gluster_vg1/lv_vmdisks > > I did review the article here but I am unable to change to VG1. > https://mellowhost.com/billing/index.php?rp=/knowledgebase/65/How-to-Repair… > > Can anyone assist with a procedure on how to repair? > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/VCHU2Q4RD5QWN…

2 1

Ovirt 4.2.7 won't start and drops to emergency console
by jeremy_tourville＠hotmail.com 28 Sep '19

28 Sep '19

I see evidence that appears to be a problem with gluster. /var/log/messages has multiple occurrences of: WARNING: /dev/gluster_vg1/lv_<name>: Thin's tihin-pool needs inspection. Also, if I run vgs I am returned info for my other volume groups but not gluster_vg1. Lastly, when reviewing journalctl -xb | grep -i timed I see messages from systemd Job dev-gluster_vg1-lv_vmdisks.device/start timed.out. Timed out waiting for device dev-glustet_vg1-lv_vmdisks.device Thses messages are happening for both /dev/gluster_vg1/lv_datadisks /dev/gluster_vg1/lv_vmdisks I did review the article here but I am unable to change to VG1. https://mellowhost.com/billing/index.php?rp=/knowledgebase/65/How-to-Repair… Can anyone assist with a procedure on how to repair?

1 0

Re: Super Low VM disk IO via Shared Storage
by Strahil 28 Sep '19

28 Sep '19

Why don't you try with 4096 ? Most block devices have a blcok size of 4096 and anything bellow is slowing them down. Best Regards, Strahil NikolovOn Sep 24, 2019 17:40, Amit Bawer <abawer(a)redhat.com> wrote: > > have you reproduced performance issue when checking this directly with the shared storage mount, outside the VMs? > > On Tue, Sep 24, 2019 at 4:53 PM Vrgotic, Marko <M.Vrgotic(a)activevideo.com> wrote: >> >> Dear oVirt, >> >> >> >> I have executed some tests regarding IO disk speed on the VMs, running on shared storage and local storage in oVirt. >> >> >> >> Results of the tests on local storage domains: >> >> avlocal2: >> >> [root@mpollocalcheck22 ~]# dd if=/dev/zero of=/tmp/test2.img bs=512 count=100000 oflag=dsync >> >> 100000+0 records in >> >> 100000+0 records out >> >> 51200000 bytes (51 MB) copied, 45.9756 s, 1.1 MB/s >> >> >> >> avlocal3: >> >> [root@mpollocalcheck3 ~]# dd if=/dev/zero of=/tmp/test2.img bs=512 count=100000 oflag=dsync >> >> 100000+0 records in >> >> 100000+0 records out >> >> 51200000 bytes (51 MB) copied, 43.6179 s, 1.2 MB/s >> >> >> >> Results of the test on shared storage domain: >> >> avshared: >> >> [root@mpoludctest4udc-1 ~]# dd if=/dev/zero of=/tmp/test2.img bs=512 count=100000 oflag=dsync >> >> 100000+0 records in >> >> 100000+0 records out >> >> 51200000 bytes (51 MB) copied, 283.499 s, 181 kB/s >> >> >> >> Why is it so low? Is there anything I can do to tune, configure VDSM or other service to speed this up? >> >> Any advice is appreciated. >> >> >> >> Shared storage is based on Netapp with 20Gbps LACP path from Hypervisor to Netapp volume, and set to MTU 9000. Used protocol is NFS4.0. >> >> oVirt is 4.3.4.3 SHE. >> >> >> >> >> >> _______________________________________________ >> Users mailing list -- users(a)ovirt.org >> To unsubscribe send an email to users-leave(a)ovirt.org >> Privacy Statement: <

4 6

Changing certificates for oVirt 4.3.5
by TomK 28 Sep '19

28 Sep '19

Hey All, Would anyone have a more recent wiki on changing all certificates, including VDSM ones? Have this page but it's for version 3. https://access.redhat.com/solutions/2409751 Thinking the process didn't change much but wanted to ask if there's anything more recent floating around. -- Thx, TK.

2 3

Incremental backup using ovirt api
by smidhunraj＠gmail.com 28 Sep '19

28 Sep '19

Hi, I tried to take incremental backup of a vm using this script. public function downloadDiskIncremental(){ $data=array(); $xmlStr = "<disk_attachment> <disk id='1371afdd-91d4-4bc7-9792-0efbf6bbd1c9'> <backup>incremental</backup> </disk> </disk_attachment>"; $curlParam=array( "url" => "vms/4044e014-7e20-4dbc-abe5-64690ec45f63/diskattachments", "method" => "POST", "data" =>$xmlStr, ); } But it is throwing me error as Array ( [status] => error [message] => For correct usage, see: https://ovirt.bobcares.com/ovirt-engine/api/v4/model#services/disk-attachme… Please help me with this issue....

1 0

Re: [ANN] oVirt 4.3.6 is now generally available
by Strahil 28 Sep '19

28 Sep '19

I have upgraded from engine and after reboot the engine knows that I have no more updates pending. This is my flow: 0.Engine upgrade 1. Host is moved in maintenance (from UI) - no ticks for stopping gluster. Automatically evacuates all VMs. 2. Use UI to check for updates 3. Update host (keep tick for autoreboot) 4. Wait for hostto come up and general system & gluster healthcheck Best Regards, Strahil Nikolov

1 0

Re: Migrate running hosts to new Engine
by Strahil 28 Sep '19

28 Sep '19

I would go with fixing your current engine that trying to migrate without downtime. Have you tries to create a backup and then shutdown current engine and restore from backup ? P.S.: Before doing anything, try to find the xml configuration of each VM in the past vdsm configs.As that data is only in the engine, you won't be able to power up any machine without that configuration. Best Regards, Strahil NikolovOn Sep 27, 2019 16:11, Magnus Isaksson <magnus(a)vmar.se> wrote: > > Hello all, > > I need to migrate 4 hosts running CentOS with oVirt 4.2 to a new Engine that runs 4.3 and not disrupt the VMs on these hosts? The VMs need to be running, i have a slim option of downtime for them, but rather not. > > The reason is that my current engine 4.2 will not upgrade to 4.3, tried so many times, but all i get is a white page after the upgrade, according to engine-setup, was a success. > > So, how can i do this? > > Reagrds > Magnus Isaksson > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/7WLV5VF6BPDIO…

1 0

Re: NFS based - incremental backups
by Strahil 28 Sep '19

28 Sep '19

Geo replication requires master and slave volumes , but I'm not sure if the slave volumeshould also be replica 3. Maybe in your case it coould be replica 1 or something lile that. In KVM I would create a snapshot, back it up and then merge all disks back to one (a.k.a. block commit). So a simple script with user for the API, can allow you to do that without paying anything extra to a 3rd party. Best Regards, Strahil Nikolov On Sep 27, 2019 08:25, Leo David <leoalex(a)gmail.com> wrote: > > Hello Everyone, > Ive been struggling since a while to find out a propper solution to have a full backup scenario for a production environment. > In the past, we have used Proxmox, and the scheduled incremenral nfs based full vm backups is a thing that we really miss. > As far as i know, at this point the only way to have a backup in oVirt / rhev is by using gluster geo-replication feature. > This is nice, but as far as i know it lacks some important features: > - ability to have incremental backups to restore vms from > - ability to backup vms placed on different storage domains ( only one storage domain can be geo-replicated !!! some vms have disks on ssd volume, some on hdd, some on both) > - need to setup an external 3 nodes gluster cluster ( although a workaround would be to have single bricks based volumes for a single instance ) > I know we can create snaps, but they will die with the platform in a fail scenario, and neither they can be scheduled. > We have found bacchus project that looked promising, although it had a pretty hassled way to achieve backups ( create snap, create vm from the snap, export vm to export domain, delete vm, delete snap - all in a scheduled fashion ) > As a mention, Proxmox is incrementaly creating a tar archive of the vm disk content, and places it to an external network storage like nfs. This allowed us to backup/reatore both linux and windows vms very easily. > Now, I know this have been discussed before, but i would like to know if there are at least any future plans to implement this feature in the next releases. > Personally, i consider this a major, and quite decent feature to have with the platform, without the need to pay for 3rd party solutions that may or may not achieve the goal while adding extra pieces to the stack. Geo-replication is a good and nice feature, but in my oppinion it is not what a "backup domain" would be. > Have a nice day, > > Leo > >

1 0

engine-iso-uploader error
by Fabrice SOLER 27 Sep '19

27 Sep '19

Hello, I use engine-iso-uploader with ssh (and not NFS). It was working but now we have an error. I have tried two commands : engine-iso-uploader -v --iso-domain=lp-ducharmoy-iso upload eole-2.6.2.1-alternate-amd64.iso and engine-iso-uploader --iso-domain=lp-ducharmoy-iso list *and the error is :* ERROR: Fault reason is "Operation Failed". Fault detail is "Can't find storage server connection for id '7779c519-9f79-4339-81e0-f89f5e9f08b2'.". HTTP response code is 500. INFO: Use the -h option to see usage. I have read the /var/log/httpd/ssl_access_log , so we can read that API is not accessible : 192.168.83.83 - - [27/Sep/2019:12:01:06 -0400] "POST /ovirt-engine/sso/oauth/token HTTP/1.1" 200 310 192.168.83.83 - - [27/Sep/2019:12:01:07 -0400] "POST /ovirt-engine/sso/oauth/token-info HTTP/1.1" 200 323 192.168.83.83 - - [27/Sep/2019:12:01:07 -0400] "GET /ovirt-engine/api HTTP/1.1" 200 975 192.168.83.83 - - [27/Sep/2019:12:01:07 -0400] "GET /ovirt-engine/api/storagedomains HTTP/1.1" *500* 188 I have changed the admin password, do you think it could be come from this ? With verbose : [root@infra-eple iso]# engine-iso-uploader -v --iso-domain=lp-ducharmoy-iso list Please provide the REST API password for the admin@internal oVirt Engine user (CTRL+D to abort): DEBUG: API Vendor(None) API Version(4.2.0) ERROR: Fault reason is "Operation Failed". Fault detail is "Can't find storage server connection for id '7779c519-9f79-4339-81e0-f89f5e9f08b2'.". HTTP response code is 500. INFO: Use the -h option to see usage. DEBUG: Configuration: DEBUG: command: list DEBUG: Traceback (most recent call last): DEBUG: File "/usr/bin/engine-iso-uploader", line 1521, in <module> DEBUG: isoup = ISOUploader(conf) DEBUG: File "/usr/bin/engine-iso-uploader", line 448, in __init__ DEBUG: self.list_all_ISO_storage_domains() DEBUG: File "/usr/bin/engine-iso-uploader", line 559, in list_all_ISO_storage_domains DEBUG: domainAry = svc.storage_domains_service().list() DEBUG: File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 21936, in list DEBUG: return self._internal_get(headers, query, wait) DEBUG: File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211, in _internal_get DEBUG: return future.wait() if wait else future DEBUG: File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait DEBUG: return self._code(response) DEBUG: File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 208, in callback DEBUG: self._check_fault(response) DEBUG: File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 132, in _check_fault DEBUG: self._raise_error(response, body) DEBUG: File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 118, in _raise_error DEBUG: raise error DEBUG: Error: Fault reason is "Operation Failed". Fault detail is "Can't find storage server connection for id '7779c519-9f79-4339-81e0-f89f5e9f08b2'.". HTTP response code is 500. Sincerely, Fabrice SOLER --

1 0

engine-iso-uploader error
by Fabrice SOLER 27 Sep '19

27 Sep '19

Hello, We use engine-iso-uploader with ssh (and not NFS). It was working but now we have an error. We tried two commands : engine-iso-uploader -v --iso-domain=lp-ducharmoy-iso upload eole-2.6.2.1-alternate-amd64.iso and engine-iso-uploader --iso-domain=lp-ducharmoy-iso list *and the error is :* ERROR: Fault reason is "Operation Failed". Fault detail is "Can't find storage server connection for id '7779c519-9f79-4339-81e0-f89f5e9f08b2'.". HTTP response code is 500. INFO: Use the -h option to see usage. Do you know where it could come from ? With verbose : [root@infra-eple iso]# engine-iso-uploader -v --iso-domain=lp-ducharmoy-iso list Please provide the REST API password for the admin@internal oVirt Engine user (CTRL+D to abort): DEBUG: API Vendor(None) API Version(4.2.0) ERROR: Fault reason is "Operation Failed". Fault detail is "Can't find storage server connection for id '7779c519-9f79-4339-81e0-f89f5e9f08b2'.". HTTP response code is 500. INFO: Use the -h option to see usage. DEBUG: Configuration: DEBUG: command: list DEBUG: Traceback (most recent call last): DEBUG: File "/usr/bin/engine-iso-uploader", line 1521, in <module> DEBUG: isoup = ISOUploader(conf) DEBUG: File "/usr/bin/engine-iso-uploader", line 448, in __init__ DEBUG: self.list_all_ISO_storage_domains() DEBUG: File "/usr/bin/engine-iso-uploader", line 559, in list_all_ISO_storage_domains DEBUG: domainAry = svc.storage_domains_service().list() DEBUG: File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 21936, in list DEBUG: return self._internal_get(headers, query, wait) DEBUG: File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 211, in _internal_get DEBUG: return future.wait() if wait else future DEBUG: File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait DEBUG: return self._code(response) DEBUG: File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 208, in callback DEBUG: self._check_fault(response) DEBUG: File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 132, in _check_fault DEBUG: self._raise_error(response, body) DEBUG: File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 118, in _raise_error DEBUG: raise error DEBUG: Error: Fault reason is "Operation Failed". Fault detail is "Can't find storage server connection for id '7779c519-9f79-4339-81e0-f89f5e9f08b2'.". HTTP response code is 500. Sincerely, Fabrice SOLER

1 0

Migrate running hosts to new Engine
by Magnus Isaksson 27 Sep '19

27 Sep '19

Hello all, I need to migrate 4 hosts running CentOS with oVirt 4.2 to a new Engine that runs 4.3 and not disrupt the VMs on these hosts? The VMs need to be running, i have a slim option of downtime for them, but rather not. The reason is that my current engine 4.2 will not upgrade to 4.3, tried so many times, but all i get is a white page after the upgrade, according to engine-setup, was a success. So, how can i do this? Reagrds Magnus Isaksson

1 0

Reg: Ovirt 4.3.5 - getting error while intergrating openstack glance(image) provider
by lakshmanan J 27 Sep '19

27 Sep '19

Hi All, I'm getting brlow error when I tried to integrate openstack glance,(image) into ovirt 4.3.5 manager. Plz help to troubleshoot. Error: Failed with error PROVIDER_FAILURE and code 5050 Regards Lakshmanan J

1 0

Re: oVirt and CentOS Stream
by Strahil 27 Sep '19

27 Sep '19

Should I understand that the most tested platform will be CentOS Stream 8 ? Will Fedora & CentOS 8 still viable option ? Best Regards, Strahil NikolovOn Sep 26, 2019 16:13, Sandro Bonazzola <sbonazzo(a)redhat.com> wrote: > > Progress cannot be made without change. As technologists, we recognize this every day. Most of the time, these changes are iterative: progressive additions of features to projects like oVirt. Sometimes those changes are small, and sometimes not. And that’s, of course, just talking about our project. But one of the biggest strengths of our community’s software is that we are not alone, and because of that, changes to other projects have ripple effects that can affect our own, even in positive ways. > > > This week, our collaborators in the CentOS Project have announced a change in the way their software is released moving forward. > > Beyond this week release of CentOS Linux 8, the CentOS team has announced CentOS Stream, a rolling release distribution that will be the "midstream" between Fedora and Red Hat Enterprise Linux. > > > To be clear, Fedora will remain as the first upstream of RHEL. But when RHEL branches off, CentOS will be upstream for ongoing work on those RHEL versions. This change gives public visibility into ongoing work on released RHEL, and a place for developers and community projects like ours to collaborate at that level. > > > CentOS has been heading in the direction of Stream for quite some time. CentOS SIGs — the special interest groups for virtualization, storage, config management, and so on — have given our project a way to provide input into the CentOS platform. Now, though, we can actually provide direct improvements or fixes to CentOS Stream platform that will be beneficial to our project. Long-term, those project-specific changes can find their way into the next release of RHEL, providing smoother transitions for those users of our downstream projects. > > > We expect the positive effects to extend beyond our own project’s ecosystem. Through CentOS Stream, developers will have early access to new features and content that are being built into the upcoming RHEL version. This will help to allow next-generation applications to have compatibility with future versions of RHEL. > > > The benefits of these changes are clear for ecosystem developers working on projects such as ours, writing hardware drivers, or extending protocols for RHEL. As we push the innovations that start in Fedora through the new CentOS Stream, the community will have a clear vision of the future of Red Hat Enterprise Linux. > > > For users of oVirt, we expect CentOS Stream to be the preferred upstream platform on which oVirt should be run, especially with the capability for our users to now contribute changes to our software and the community-built platform on which oVirt runs. > > > > -- > > Sandro Bonazzola > > MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV > > Red Hat EMEA > > sbonazzo(a)redhat.com > > Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.

3 3

How to take incremental backup of a vm
by smidhunraj＠gmail.com 27 Sep '19

27 Sep '19

Can you tell me the method on how to take incremental backup of a VM in ovirt .Is there any opensource python script available to get this done.

3 2

NFS based - incremental backups
by Leo David 27 Sep '19

27 Sep '19

Hello Everyone, Ive been struggling since a while to find out a propper solution to have a full backup scenario for a production environment. In the past, we have used Proxmox, and the scheduled incremenral nfs based full vm backups is a thing that we really miss. As far as i know, at this point the only way to have a backup in oVirt / rhev is by using gluster geo-replication feature. This is nice, but as far as i know it lacks some important features: - ability to have incremental backups to restore vms from - ability to backup vms placed on different storage domains ( only one storage domain can be geo-replicated !!! some vms have disks on ssd volume, some on hdd, some on both) - need to setup an external 3 nodes gluster cluster ( although a workaround would be to have single bricks based volumes for a single instance ) I know we can create snaps, but they will die with the platform in a fail scenario, and neither they can be scheduled. We have found bacchus project that looked promising, although it had a pretty hassled way to achieve backups ( create snap, create vm from the snap, export vm to export domain, delete vm, delete snap - all in a scheduled fashion ) As a mention, Proxmox is incrementaly creating a tar archive of the vm disk content, and places it to an external network storage like nfs. This allowed us to backup/reatore both linux and windows vms very easily. Now, I know this have been discussed before, but i would like to know if there are at least any future plans to implement this feature in the next releases. Personally, i consider this a major, and quite decent feature to have with the platform, without the need to pay for 3rd party solutions that may or may not achieve the goal while adding extra pieces to the stack. Geo-replication is a good and nice feature, but in my oppinion it is not what a "backup domain" would be. Have a nice day, Leo

1 0

not able to import ova file
by Crazy Ayansh 27 Sep '19

27 Sep '19

Hi Team, I am getting below error message when i am trying to import a vm from ova method. [image: image.png] could anyone please help me out here. Hosted engine version:-4.2 Please let me know if any other info required from me. Thanks

1 1

4.2.7 to 4.3.5
by Demeter Tibor 26 Sep '19

26 Sep '19

Dear list members, I would like to upgrade my Centos 7.6 based ovirt 4.2 to 4.3.5. How is safe the upgrade ? How is stable the actual branch ? It seems to me, in 4.3.x there are a lot of cases caused by live migration, snapshot deletion, etc. Are these really existing problems? So, I'm a little bit afraid of the upgrade. It is a productive system, I don't want to make an outage. Thanks in advance, Regards, Tibor

2 1

oVirt and CentOS Stream
by Sandro Bonazzola 26 Sep '19

26 Sep '19

Progress cannot be made without change. As technologists, we recognize this every day. Most of the time, these changes are iterative: progressive additions of features to projects like oVirt. Sometimes those changes are small, and sometimes not. And that’s, of course, just talking about our project. But one of the biggest strengths of our community’s software is that we are not alone, and because of that, changes to other projects have ripple effects that can affect our own, even in positive ways. This week, our collaborators in the CentOS Project have announced a change <https://lists.centos.org/pipermail/centos-announce/2019-September/023449.ht…> in the way their software is released moving forward. Beyond this week release of CentOS Linux 8, the CentOS team has announced CentOS Stream, a rolling release distribution that will be the "midstream" between Fedora and Red Hat Enterprise Linux. To be clear, Fedora will remain as the first upstream of RHEL. But when RHEL branches off, CentOS will be upstream for ongoing work on those RHEL versions. This change gives public visibility into ongoing work on released RHEL, and a place for developers and community projects like ours to collaborate at that level. CentOS has been heading in the direction of Stream for quite some time. CentOS SIGs — the special interest groups for virtualization, storage, config management, and so on — have given our project a way to provide input into the CentOS platform. Now, though, we can actually provide direct improvements or fixes to CentOS Stream platform that will be beneficial to our project. Long-term, those project-specific changes can find their way into the next release of RHEL, providing smoother transitions for those users of our downstream projects. We expect the positive effects to extend beyond our own project’s ecosystem. Through CentOS Stream, developers will have early access to new features and content that are being built into the upcoming RHEL version. This will help to allow next-generation applications to have compatibility with future versions of RHEL. The benefits of these changes are clear for ecosystem developers working on projects such as ours, writing hardware drivers, or extending protocols for RHEL. As we push the innovations that start in Fedora through the new CentOS Stream, the community will have a clear vision of the future of Red Hat Enterprise Linux. For users of oVirt, we expect CentOS Stream to be the preferred upstream platform on which oVirt should be run, especially with the capability for our users to now contribute changes to our software and the community-built platform on which oVirt runs. -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/>*Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

1 0

Re: Change hostname of physical hosts under an oVirt and Gluster combination
by Strahil 26 Sep '19

26 Sep '19

Unassigned in 99% of the cases means engine cannot communicate with vdsm.service on the host. Check that vdsm.service & glusterd.service are running. Check gluster volume status for each volume. If they are OK - from UI just select activate. Best Regards, Strahil NikolovOn Sep 25, 2019 22:24, TomK <tomkcpr(a)mdevsys.com> wrote: > > On 9/25/2019 8:49 AM, TomK wrote: > > On 9/24/2019 10:55 AM, Sahina Bose wrote: > >> > >> > >> On Tue, Sep 24, 2019 at 6:38 PM TomK <tomkcpr(a)mdevsys.com > >> <mailto:tomkcpr@mdevsys.com>> wrote: > >> > >> Hey Sahina, > >> > >> Thanks very much. > >> > >> I've taken a quick glance and it does mention I need a third host > >> which > >> I don't have. > >> > >> However, since I wrote, I removed everything. Had to force remove > >> hosts > >> however the gluster setup and data in it stayed (Or so I'm thinking). > >> > >> I'll read the article more thoroughly however is there a specific > >> procedure for adding existing oVirt gluster volumes back into a > >> cluster? > >> > >> One additional question. Is the sequence below correct? > >> > >> 1) Create Data Center > >> 2) Create Cluster within the DC > >> > >> > >> When you create the cluster - there's an Import cluster option, if I > >> remember correctly. This should discover the peers and volumes and add > >> it to the engine > > > > Trying that now. On a side note, the autodiscovery box is hard to edit. > > > > So in the meantime, trying to up the version to take advantage of some > > new features and ensure Gluster 6 is fully supported. Not having luck > > with the oVirt UI upgrade at the moment ( Not a self hosted engine. It's > > on a separate VM. ): > > > > --> Processing Conflict: > > ovirt-engine-setup-plugin-ovirt-engine-4.3.5.5-1.el7.noarch conflicts > > ovirt-engine < 4.2.6 > > --> Finished Dependency Resolution > > Error: Package: ovirt-engine-ui-extensions-1.0.6-1.el7.noarch (ovirt-4.3) > > Requires: ovirt-engine-webadmin-portal >= 4.3 > > Installed: > > ovirt-engine-webadmin-portal-4.2.1.7-1.el7.centos.noarch (@ovirt-4.2) > > ovirt-engine-webadmin-portal = 4.2.1.7-1.el7.centos > > Error: ovirt-engine-setup-plugin-ovirt-engine conflicts with > > ovirt-engine-4.2.1.7-1.el7.centos.noarch > > You could try using --skip-broken to work around the problem > > You could try running: rpm -Va --nofiles --nodigest > > > > > > > > Cheers, > > TK > > > > Just upgraded to 4.3.5. > > What do I do if I can't remove a host from a cluster? Maintenance > window options Active and Maintenance are grayed out. Host is currently > showing as Unassigned with an exclamation mark next to it. Items listed > include: > > "Power Management is not configured for this Host. Enable Power Management" > "Gluster status is disconnected for this host. Restart Glusterd service" > "Host has no default route." > > > I try to modify some of this thinking maybe it will nudge the server out > of it's slumber but get this when trying to fix Power Management: > > > "Error while executing action: > mdskvm-p01.nix.mds.xyz: > Cannot edit Host. Host parameters cannot be modified while Host is > operational. > Please switch Host to Maintenance mode first." > > > Of course I can't put it into maintenance since it's grayed out. > > Anyway to remove this host and start fresh? > > Cheers, > TK > > > >> > >> 3) Create a Domain within the Cluster > >> 4) Add hosts to DC > >> 5) Import Gluster Volume > >> > >> > >> 4 & 5 not required if you change step 2 to import. > >> You are running into errors below because the engine sees the addition > >> of new hosts as gluster peer probing to a new cluster > >> > >> > >> > >> Some issues. I create a DC, add a Cluster then add the Host to it. > >> Then > >> I try to add another the second node but it says It's already in a > >> cluster. There is no other cluster and the second host is not listed > >> anywhere. > >> > >> So I try to remove the first host to res

2 1

Outage Notification: Jenkins and Resources Are Not Accessible
by Anton Marchukov 26 Sep '19

26 Sep '19

Hello All. Please note that oVirt services hosted at PHX datacenter are not accessible. Main services running there are: jenkins.ovirt.org resources.ovirt.org This looks to be network related and we are working with the relevant teams to resolve it. Anton. -- Anton Marchukov Associate Manager - RHV DevOps - Red Hat

1 1

Re: ovirtmgmt STP disabled - any reason behind that
by Strahil 26 Sep '19

26 Sep '19

Did I miss someone's reply on this one ? Best Regards, Strahil NikolovOn Sep 24, 2019 00:46, Strahil Nikolov <hunter86_bg(a)yahoo.com> wrote: > > Hi All, > > I recently got an issue and due to lack of debug time , I'm still not sure if it was a loop on the network. > Can someone clarify why the ovirtmgmt bridge has STP disabled ? Any reason behind that ? > > Also, what is the proper way to enable STP on those bridges ? > > For now , I have set highest priority of the STP on my router and I don't expect the issue to reoccur (if it really was a loop), but I want to avoid future issues like that. > > > Best Regards, > Strahil Nikolov

1 0

Re: Urgent help needed / Snapshot deletion failure
by Strahil 26 Sep '19

26 Sep '19

Is it really down ? Have you tried to migrate it to another host. I found out yesterday that one of my VMs was constantly reporting errors (relayed to snapshot delete) and the errors stopped when the system got migrated to another host. Best Regards, Strahil NikolovOn Sep 25, 2019 15:27, smirta(a)gmx.net wrote: > > Dear all we are desperate > > We have tried to delete a Snapshot and it hangs at merging the snapshots. We've found out that this is a known bug because the merge process is being called with wrong parameters according to the bug report https://bugzilla.redhat.com/show_bug.cgi?id=1601212. The Snapshot's Disks are in illegal state and the VM is locked. Since it is a very important VM for our non-profit organization, we need to have this machine back online as soon as possible. Is there a way to fix this without updating to qemu-kvm-ev-2.12.0? At least back to the state before the deletion would be fantastic. > > Our vdsm.log: > 2019-09-25 14:01:41,283+0200 WARN (vdsm.Scheduler) [Executor] Worker blocked: <Worker name=jsonrpc/0 running <Task <JsonRpcTask {'params': {u'topVolUUID': u'8a9f190f-2725-4535-a69d-c74e4e57d372', u'vmID': u'1422899a-2151-4d5d-9d66-e74f19084542', u'drive': {u'imageID': u'eb2bce92-e758-4bea-93fa-02a56574b932', u'volumeID': u'8a9f190f-2725-4535-a69d-c74e4e57d372', u'domainID': u'022f39ee-eeb8-4b51-9549-9d7e3c88d4a8', u'poolID': u'00000001-0001-0001-0001-000000000307'}, u'bandwidth': u'0', u'jobUUID': u'c3fdc4a9-9d6d-424a-b9df-b96be5622e0a', u'baseVolUUID': u'3e15121b-0795-4056-bafe-448068c9ec71'}, 'jsonrpc': '2.0', 'method': u'VM.merge', 'id': u'9cd540b7-a32f-4f95-9fe2-9ce70d5b6478'} at 0x7f674fec5710> timeout=60, duration=6420 at 0x7f674fec58d0> task#=1896299 at 0x7f674c06ae90>, traceback: > File: "/usr/lib64/python2.7/threading.py", line 785, in __bootstrap > self.__bootstrap_inner() > File: "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner > self.run() > File: "/usr/lib64/python2.7/threading.py", line 765, in run > self.__target(*self.__args, **self.__kwargs) > File: "/usr/lib/python2.7/site-packages/vdsm/common/concurrent.py", line 194, in run > ret = func(*args, **kwargs) > File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 301, in _run > self._execute_task() > File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 315, in _execute_task > task() > File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 391, in __call__ > self._callable() > File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 523, in __call__ > self._handler(self._ctx, self._req) > File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 566, in _serveRequest > response = self._handle_request(req, ctx) > File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 606, in _handle_request > res = method(**params) > File: "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 197, in _dynamicMethod > result = fn(*methodArgs) > File: "<string>", line 2, in merge > File: "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in method > ret = func(*args, **kwargs) > File: "<string>", line 2, in merge > File: "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 122, in method > ret = func(*args, **kwargs) > File: "/usr/lib/python2.7/site-packages/vdsm/API.py", line 739, in merge > drive, baseVolUUID, topVolUUID, bandwidth, jobUUID) > File: "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 6041, in merge > self.updateVmJobs() > File: "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 5818, in updateVmJobs > self._vmJobs = self.queryBlockJobs() > File: "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 5832, in queryBlockJobs > with self._jobsLock: > File: "/usr/lib/python2.7/site-packages/pthreading.py", line 60, in __enter__ > self.acquire() > File: "/usr/lib/python2.7/site-packages/pthreading.py", line 68, in acquire > rc = self.lock() if blocking else self.trylock() > File: "/usr/lib/python2.7/site-packages/pthread.py", line 96, in lock > return _libpthread.pthread_mutex_lock(self._mutex) (executor:363) > > > Kind regards > Simon > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/LW7BNZBO5AAGN…

1 0

Change hostname of physical hosts under an oVirt and Gluster combination
by TomK 25 Sep '19

25 Sep '19

Hey All, Would like to change the hostname of hosts within a gluster volume that's been added to oVirt. I also need to readd the bricks to the gluster volume under the new hostname. What's the recommended approach to do so when all this is added to oVirt? -- Thx, TK.

2 7

Managed Block Storage: ceph detach_volume failing after migration
by Dan Poltawski 25 Sep '19

25 Sep '19

On ovirt 4.3.5 we are seeing various problems related to the rbd device staying mapped after a guest has been live migrated. This causes problems migrating the guest back, as well as rebooting the guest when it starts back up on the original host. The error returned is ‘nrbd: unmap failed: (16) Device or resource busy’. I’ve pasted the full vdsm log below. As far as I can tell this isn’t happening 100% of the time, and seems to be more prevalent on busy guests. (Not sure if I should create a bug for this, so thought I’d start here first) Thanks, Dan Sep 24 19:26:18 mario vdsm[5485]: ERROR FINISH detach_volume error=Managed Volume Helper failed.: ('Error executing helper: Command [\'/usr/libexec/vdsm/managedvolume-helper\', \'detach\'] failed with rc=1 out=\'\' err=\'oslo.privsep.daemon: Running privsep helper: [\\\'sudo\\\', \\\'privsep-helper\\\', \\\'--privsep_context\\\', \\\'os_brick.privileged.default\\\', \\\'--privsep_sock_path\\\', \\\'/tmp/tmptQzb10/privsep.sock\\\']\\noslo.privsep.daemon: Spawned new privsep daemon via rootwrap\\noslo.privsep.daemon: privsep daemon starting\\noslo.privsep.daemon: privsep process running with uid/gid: 0/0\\noslo.privsep.daemon: privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none\\noslo.privsep.daemon: privsep daemon running as pid 76076\\nTraceback (most recent call last):\\n File "/usr/libexec/vdsm/managedvolume-helper", line 154, in <module>\\n sys.exit(main(sys.argv[1:]))\\n File "/usr/libexec/vdsm/managedvolume-helper", line 77, in main\\n args.command(args)\\n File "/usr/libexec/vdsm/managedvolume-helper", line 149, in detach\\n ignore_errors=False)\\n File "/usr/lib/python2.7/site-packages/vdsm/storage/nos_brick.py", line 121, in disconnect_volume\\n run_as_root=True)\\n File "/usr/lib/python2.7/site-packages/os_brick/executor.py", line 52, in _execute\\n result = self.__execute(*args, **kwargs)\\n File "/usr/lib/python2.7/site-packages/os_brick/privileged/rootwrap.py", line 169, in execute\\n return execute_root(*cmd, **kwargs)\\n File "/usr/lib/python2.7/site-packages/oslo_privsep/priv_context.py", line 241, in _wrap\\n return self.channel.remote_call(name, args, kwargs)\\n File "/usr/lib/python2.7/site-packages/oslo_privsep/daemon.py", line 203, in remote_call\\n raise exc_type(*result[2])\\noslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.\\nCommand: rbd unmap /dev/rbd/rbd/volume-0e8c1056-45d6-4740-934d-eb07a9f73160 --conf /tmp/brickrbd_LCKezP --id ovirt --mon_host 172.16.10.13:3300 --mon_host 172.16.10.14:3300 --mon_host 172.16.10.12:6789\\nExit code: 16\\nStdout: u\\\'\\\'\\nStderr: u\\\'rbd: sysfs write failed\\\\nrbd: unmap failed: (16) Device or resource busy\\\\n\\\'\\n\'',)#012Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 124, in method#012 ret = func(*args, **kwargs)#012 File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1766, in detach_volume#012 return managedvolume.detach_volume(vol_id)#012 File "/usr/lib/python2.7/site-packages/vdsm/storage/managedvolume.py", line 67, in wrapper#012 return func(*args, **kwargs)#012 File "/usr/lib/python2.7/site-packages/vdsm/storage/managedvolume.py", line 135, in detach_volume#012 run_helper("detach", vol_info)#012 File "/usr/lib/python2.7/site-packages/vdsm/storage/managedvolume.py", line 179, in run_helper#012 sub_cmd, cmd_input=cmd_input)#012 File "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 56, in __call__#012 return callMethod()#012 File "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 54, in <lambda>#012 **kwargs)#012 File "<string>", line 2, in managedvolume_run_helper#012 File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in _callmethod#012 raise convert_to_error(kind, result)#012ManagedVolumeHelperFailed: Managed Volume Helper failed.: ('Error executing helper: Command [\'/usr/libexec/vdsm/managedvolume-helper\', \'detach\'] failed with rc=1 out=\'\' err=\'oslo.privsep.daemon: Running privsep helper: [\\\'sudo\\\', \\\'privsep-helper\\\', \\\'--privsep_context\\\', \\\'os_brick.privileged.default\\\', \\\'--privsep_sock_path\\\', \\\'/tmp/tmptQzb10/privsep.sock\\\']\\noslo.privsep.daemon: Spawned new privsep daemon via rootwrap\\noslo.privsep.daemon: privsep daemon starting\\noslo.privsep.daemon: privsep process running with uid/gid: 0/0\\noslo.privsep.daemon: privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none\\noslo.privsep.daemon: privsep daemon running as pid 76076\\nTraceback (most recent call last):\\n File "/usr/libexec/vdsm/managedvolume-helper", line 154, in <module>\\n sys.exit(main(sys.argv[1:]))\\n File "/usr/libexec/vdsm/managedvolume-helper", line 77, in main\\n args.command(args)\\n File "/usr/libexec/vdsm/managedvolume-helper", line 149, in detach\\n ignore_errors=False)\\n File "/usr/lib/python2.7/site-packages/vdsm/storage/nos_brick.py", line 121, in disconnect_volume\\n run_as_root=True)\\n File "/usr/lib/python2.7/site-packages/os_brick/executor.py", line 52, in _execute\\n result = self.__execute(*args, **kwargs)\\n File "/usr/lib/python2.7/site-packages/os_brick/privileged/rootwrap.py", line 169, in execute\\n return execute_root(*cmd, **kwargs)\\n File "/usr/lib/python2.7/site-packages/oslo_privsep/priv_context.py", line 241, in _wrap\\n return self.channel.remote_call(name, args, kwargs)\\n File "/usr/lib/python2.7/site-packages/oslo_privsep/daemon.py", line 203, in remote_call\\n raise exc_type(*result[2])\\noslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.\\nCommand: rbd unmap /dev/rbd/rbd/volume-0e8c1056-45d6-4740-934d-eb07a9f73160 --conf /tmp/brickrbd_LCKezP --id ovirt --mon_host 172.16.10.13:3300 --mon_host 172.16.10.14:3300 --mon_host 172.16.10.12:6789\\nExit code: 16\\nStdout: u\\\'\\\'\\nStderr: u\\\'rbd: sysfs write failed\\\\nrbd: unmap failed: (16) Device or resource busy\\\\n\\\'\\n\'',) ________________________________ The Networking People (TNP) Limited. Registered office: Network House, Caton Rd, Lancaster, LA1 3PE. Registered in England & Wales with company number: 07667393 This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.

4 4

Urgent help needed / Snapshot deletion failure
by smirta＠gmx.net 25 Sep '19

25 Sep '19

Dear all we are desperate We have tried to delete a Snapshot and it hangs at merging the snapshots. We've found out that this is a known bug because the merge process is being called with wrong parameters according to the bug report https://bugzilla.redhat.com/show_bug.cgi?id=1601212. The Snapshot's Disks are in illegal state and the VM is locked. Since it is a very important VM for our non-profit organization, we need to have this machine back online as soon as possible. Is there a way to fix this without updating to qemu-kvm-ev-2.12.0? At least back to the state before the deletion would be fantastic. Our vdsm.log: 2019-09-25 14:01:41,283+0200 WARN (vdsm.Scheduler) [Executor] Worker blocked: <Worker name=jsonrpc/0 running <Task <JsonRpcTask {'params': {u'topVolUUID': u'8a9f190f-2725-4535-a69d-c74e4e57d372', u'vmID': u'1422899a-2151-4d5d-9d66-e74f19084542', u'drive': {u'imageID': u'eb2bce92-e758-4bea-93fa-02a56574b932', u'volumeID': u'8a9f190f-2725-4535-a69d-c74e4e57d372', u'domainID': u'022f39ee-eeb8-4b51-9549-9d7e3c88d4a8', u'poolID': u'00000001-0001-0001-0001-000000000307'}, u'bandwidth': u'0', u'jobUUID': u'c3fdc4a9-9d6d-424a-b9df-b96be5622e0a', u'baseVolUUID': u'3e15121b-0795-4056-bafe-448068c9ec71'}, 'jsonrpc': '2.0', 'method': u'VM.merge', 'id': u'9cd540b7-a32f-4f95-9fe2-9ce70d5b6478'} at 0x7f674fec5710> timeout=60, duration=6420 at 0x7f674fec58d0> task#=1896299 at 0x7f674c06ae90>, traceback: File: "/usr/lib64/python2.7/threading.py", line 785, in __bootstrap self.__bootstrap_inner() File: "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner self.run() File: "/usr/lib64/python2.7/threading.py", line 765, in run self.__target(*self.__args, **self.__kwargs) File: "/usr/lib/python2.7/site-packages/vdsm/common/concurrent.py", line 194, in run ret = func(*args, **kwargs) File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 301, in _run self._execute_task() File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 315, in _execute_task task() File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 391, in __call__ self._callable() File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 523, in __call__ self._handler(self._ctx, self._req) File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 566, in _serveRequest response = self._handle_request(req, ctx) File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 606, in _handle_request res = method(**params) File: "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 197, in _dynamicMethod result = fn(*methodArgs) File: "<string>", line 2, in merge File: "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in method ret = func(*args, **kwargs) File: "<string>", line 2, in merge File: "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 122, in method ret = func(*args, **kwargs) File: "/usr/lib/python2.7/site-packages/vdsm/API.py", line 739, in merge drive, baseVolUUID, topVolUUID, bandwidth, jobUUID) File: "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 6041, in merge self.updateVmJobs() File: "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 5818, in updateVmJobs self._vmJobs = self.queryBlockJobs() File: "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 5832, in queryBlockJobs with self._jobsLock: File: "/usr/lib/python2.7/site-packages/pthreading.py", line 60, in __enter__ self.acquire() File: "/usr/lib/python2.7/site-packages/pthreading.py", line 68, in acquire rc = self.lock() if blocking else self.trylock() File: "/usr/lib/python2.7/site-packages/pthread.py", line 96, in lock return _libpthread.pthread_mutex_lock(self._mutex) (executor:363) Kind regards Simon

1 1

How to delete obsolete Data Centers with no hosts, but with domains inside
by Claudio Soprano 25 Sep '19

25 Sep '19

Hi to all, We are using ovirt to manage 6 Data Centers, 3 of them are old Data Centers with no hosts inside, but with domains, storage and VMs not running. We left them because we wanted to have some backups in case of failure of the new Data Centers created. Time pass and now we would like to remove these Data Centers, but we got no way for now to remove them. If we try to remove the Storage Domains (using remove o destroy) we get "Error while executing action: Cannot destroy the master Storage Domain from the Data Center without another active Storage Domain to take its place. -Either activate another Storage Domain in the Data Center, or remove the Data Center. -If you have problems with the master Data Domain, consider following the recovery process described in the documentation, or contact your system administrator." if we try to remove the Data Center directly we get "Error while executing action: Cannot remove Data Center. There is no active Host in the Data Center." How can we solve the problem ? It can be done via ovirt-shell or using some script or via ovirt management interface ? Thanks in advance Claudio

2 6

Add External Provider - OpenStack Glance - Test Failed - "Failed to communicate with the External Provider"
by Pravin Mohandass 25 Sep '19

25 Sep '19

We have installed Ovirt Manager 4.3 with couple of KVM compute host added into cluster. When we add OpenStack Glance as an External Provider, failing with following error message - "Failed to communicate with the External Provider" We are able to add the external provider for Openstack Neutron and Openstack Cinder and import the storage and network in to ovirt manager and use it for vm. We are getting following error message on engine.log file "Failed with error PROVIDER_FAILURE and code 5050". But we are able to fetch the images using postman tool. Requesting you to provide some insights on adding the Glance provider.

1 0

[ANN] oVirt 4.3.6 Sixth Release Candidate is now available for testing
by Sandro Bonazzola 25 Sep '19

25 Sep '19

The oVirt Project is pleased to announce the availability of the oVirt 4.3.6 Sixth Release Candidate for testing, as of September 25th, 2019. This update is a release candidate of the sixth in a series of stabilization updates to the 4.3 series. This is pre-release software. This pre-release should not to be used in production. This release is available now on x86_64 architecture for: * Red Hat Enterprise Linux 7.7 or later (but <8) * CentOS Linux (or similar) 7.7 or later (but <8) This release supports Hypervisor Hosts on x86_64 and ppc64le architectures for: * Red Hat Enterprise Linux 7.7 or later (but <8) * CentOS Linux (or similar) 7.7 or later (but <8) * oVirt Node 4.3 (available for x86_64 only) has been built consuming CentOS 7.7 Release See the release notes [1] for known issues, new features and bugs fixed. Notes: - oVirt Appliance is already available - oVirt Node is already available Additional Resources: * Read more about the oVirt 4.3.6 release highlights: http://www.ovirt.org/release/4.3.6/ * Get more oVirt Project updates on Twitter: https://twitter.com/ovirt * Check out the latest project news on the oVirt blog: http://www.ovirt.org/blog/ [1] http://www.ovirt.org/release/4.3.6/ [2] http://resources.ovirt.org/pub/ovirt-4.3-pre/iso/ -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/>*Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

1 0

Messed up 4.2.3.1 installation - SSL handshake ERROR
by souvaliotimaria＠mail.com 25 Sep '19

25 Sep '19

Hello, everyone! So, I have an experimental installation of ovirt 4.2.3.1 with 3 nodes and glustered. Recently I deployed a new installation with ovirt 4.3.5.2, 3nodes and glustered storage here as well. The thing is, in my enthousiasm I thought "hey! what if I can import the experimental nodes as hosts in the new installation in a new cluster and see what happens? Will the 4.3.5.2 engine see them? Probably yeah. But will it see the VMs I have there?" And so I imported the experimental nodes. Without detaching them from their hosted engine. I could see the only VM that was active at the moment and not one of the suspended ones and of course I could not see the 4.2.3.1 HE VM. I have removed the hosts from the new installation and I have tried reconnecting the old engine and its nodes. Passwordless ssh works just fine, but the problem persists. hosted-engine --vm-status reports stale-data on node 2 and node 3 The thing is I know I messed the experimental installation (and I blame only my curiosity), SSL handshake is no longer feasable and I can't remove the hosts from the initial Cluster to Import them again. Basically everything is either in the process of activating without ever being able to do so or down or Non responsive. I would like to find a way around this, as I have seen in other posts in the ovirt forum that the SSL handshake error appears in some other cases and I would like to have a know-how if an occasion like this occurs in the future in production. Is it possible to re-deploy the engine on the nodes and not lose the glustered space or the existing VMs? Can the HE be destroyed and then deployed from scratch? What about the glustered space and the VMs' space? Will the VMs just take up space without being able to neither bring them up nor destroy them? I know I'm asking a lot and it was my fault to begin with but I am really curious if we can see this through. Thanks in advance

1 0

oVirt node - post-upgrade tasks
by dan.munteanu＠mdc-berlin.de 25 Sep '19

25 Sep '19

Dear oVirt community, recently I've started to use oVirt as a replacement for KVM + VirtManager and I've decided to use the oVirt Node installed on Dell EMC servers with self-hosted engine. So far, everything works fine, except that, each time I'm updating the nodes, I must reinstall Dell OMSA needed by the monitoring system (via SNMP). Is there any way to automatize the OMSA installation as a post-upgrade task/hook? Thank you Dan

1 0

Got a RequestError: status: 409 reason: Conflict
by smidhunraj＠gmail.com 24 Sep '19

24 Sep '19

I'm trying to clone the snapshot into a new vm. The tool I am using is ovirtBAckup from the github wefixit-AT The link is here https://github.com/wefixit-AT/oVirtBackup this piece of code snippet throws me error. if not config.get_dry_run(): api.vms.add(params.VM(name=vm_clone_name, memory=vm.get_memory(), cluster=api.clusters.get(config.get_cluster_name()), snapshots=snapshots_param)) VMTools.wait_for_vm_operation(api, config, "Cloning", vm_from_list) print 'hellooooo' logger.info("Cloning finished") The above lines are from the 325 line number of backup.py of the github. I am getting error as !!! Got a RequestError: status: 409 reason: Conflict detail: Cannot add VM. The VM is performing an operation on a Snapshot. Please wait for the operation to finish, and try again. How can i further debug the code to know what is happening wrong in my program.I am new to python please help me.

2 1

Super Low VM disk IO via Shared Storage
by Vrgotic, Marko 24 Sep '19

24 Sep '19

Dear oVirt, I have executed some tests regarding IO disk speed on the VMs, running on shared storage and local storage in oVirt. Results of the tests on local storage domains: avlocal2: [root@mpollocalcheck22 ~]# dd if=/dev/zero of=/tmp/test2.img bs=512 count=100000 oflag=dsync 100000+0 records in 100000+0 records out 51200000 bytes (51 MB) copied, 45.9756 s, 1.1 MB/s avlocal3: [root@mpollocalcheck3 ~]# dd if=/dev/zero of=/tmp/test2.img bs=512 count=100000 oflag=dsync 100000+0 records in 100000+0 records out 51200000 bytes (51 MB) copied, 43.6179 s, 1.2 MB/s Results of the test on shared storage domain: avshared: [root@mpoludctest4udc-1 ~]# dd if=/dev/zero of=/tmp/test2.img bs=512 count=100000 oflag=dsync 100000+0 records in 100000+0 records out 51200000 bytes (51 MB) copied, 283.499 s, 181 kB/s Why is it so low? Is there anything I can do to tune, configure VDSM or other service to speed this up? Any advice is appreciated. Shared storage is based on Netapp with 20Gbps LACP path from Hypervisor to Netapp volume, and set to MTU 9000. Used protocol is NFS4.0. oVirt is 4.3.4.3 SHE.

2 1

neofetch and spice/vnc console
by Nathanaël Blanchet 24 Sep '19

24 Sep '19

Hello When running neofetch in spice/vnc console, terminal become unreadable, does it come from spice/vnc or graphic emulation or neofetch binary? Screenfetch alternative is ok. -- Nathanaël Blanchet Supervision réseau Pôle Infrastrutures Informatiques 227 avenue Professeur-Jean-Louis-Viala 34193 MONTPELLIER CEDEX 5 Tél. 33 (0)4 67 54 84 55 Fax 33 (0)4 67 54 84 14 blanchet(a)abes.fr

1 0

Internal Memory filter not satisfied question
by Vrgotic, Marko 24 Sep '19

24 Sep '19

Dear oVirt, I need help clarifying how internal memory filter is used, at which values it looks at. The log message below is clear, but I am failing to see from where are the memory values calculated, is it based on Guarenteed, Allocated or MaxAvailable memory a VM can get/have? Or something else. We have a shared cluster, with hosts having different memory amount: Tried to create 17VMs, and following errors were observed: 2019-09-20 11:46:10,496Z WARN [org.ovirt.engine.core.bll.RunVmCommand] (EE-ManagedThreadFactory-engine-Thread-967) [2d316914-84dc-4410-856f-f8576e411574] Validation of action 'RunVm' failed for user mvrgotic@ictv.com@ictv.com-authz. Reasons: VAR__ACTION__RUN,VAR__TYPE__VM,SCHEDULING_ALL_HOSTS_FILTERED_OUT,VAR__FILTERTYPE__INTERNAL,$hostName ovirt-hv-08.avinity.tv,$filterName Memory,$availableMem 3513,VAR__DETAIL__NOT_ENOUGH_MEMORY,SCHEDULING_HOST_FILTERED_REASON_WITH_DETAIL,VAR__FILTERTYPE__INTERNAL,$hostName ovirt-hv-02.avinity.tv,$filterName Memory,$availableMem 1605,VAR__DETAIL__NOT_ENOUGH_MEMORY,SCHEDULING_HOST_FILTERED_REASON_WITH_DETAIL,VAR__FILTERTYPE__INTERNAL,$hostName ovirt-hv-05.avinity.tv,$filterName Memory,$availableMem 1265,VAR__DETAIL__NOT_ENOUGH_MEMORY,SCHEDULING_HOST_FILTERED_REASON_WITH_DETAIL,VAR__FILTERTYPE__INTERNAL,$hostName ovirt-hv-20.avinity.tv,$filterName Memory,$availableMem 195,VAR__DETAIL__NOT_ENOUGH_MEMORY,SCHEDULING_HOST_FILTERED_REASON_WITH_DETAIL,VAR__FILTERTYPE__INTERNAL,$hostName ovirt-hv-01.avinity.tv,$filterName Memory,$availableMem 177,VAR__DETAIL__NOT_ENOUGH_MEMORY,SCHEDULING_HOST_FILTERED_REASON_WITH_DETAIL,VAR__FILTERTYPE__INTERNAL,$hostName ovirt-hv-06.avinity.tv,$filterName Memory,$availableMem 2845,VAR__DETAIL__NOT_ENOUGH_MEMORY,SCHEDULING_HOST_FILTERED_REASON_WITH_DETAIL,VAR__FILTERTYPE__INTERNAL,$hostName ovirt-hv-09.avinity.tv,$filterName Memory,$availableMem 480,VAR__DETAIL__NOT_ENOUGH_MEMORY,SCHEDULING_HOST_FILTERED_REASON_WITH_DETAIL,VAR__FILTERTYPE__INTERNAL,$hostName ovirt-hv-03.avinity.tv,$filterName Memory,$availableMem 3116,VAR__DETAIL__NOT_ENOUGH_MEMORY,SCHEDULING_HOST_FILTERED_REASON_WITH_DETAIL,VAR__FILTERTYPE__INTERNAL,$hostName ovirt-hv-21.avinity.tv,$filterName Memory,$availableMem 1342,VAR__DETAIL__NOT_ENOUGH_MEMORY,SCHEDULING_HOST_FILTERED_REASON_WITH_DETAIL 2019-09-20 11:46:10,577Z INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (EE-ManagedThreadFactory-engine-Thread-851) [2d316914-84dc-4410-856f-f8576e411574] Candidate host 'ovirt-hv-08.avinity.tv' ('adebaa4f-5402-47c7-8634-5779a4b3f10f') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'Memory' (correlation id: null) To provide you with bit more detail on our Cluster setup: oVirt SHE 4.3.4.3-1 Cluster Settings and Hosts actual memory usage [A screenshot of a cell phone Description automatically generated][A screenshot of a cell phone Description automatically generated][A screenshot of a cell phone Description automatically generated] And here is the VM information” Name: testvmmvr-th11stitcher Defined Memory: 4096 MB Origin: oVirt Description: Physical Memory Guaranteed: 1024 MB Run On: Any Host in Cluster Template: av-centos-75-baseimage (Thin/Dependent) Guest OS Memory Free/Cached/Buffered: 3453 / 2 / 156 MB Custom Properties: Not Configured Operating System: Red Hat Enterprise Linux 7.x x64 Number of CPU Cores: 4 (1:4:1) Cluster Compatibility Version: 4.3 BIOS Type: Default Guest CPU Count: 4 VM ID: testvmmvr Graphics protocol: VNC Guest CPU Type: SandyBridge,+pcid,+spec-ctrl,+ssbd Video Type: QXL Highly Available: No Priority: Low Number of Monitors: 1 FQDN: testvmmvr-th11stitcher Optimized for: Server USB Policy: Disabled Hardware Clock Time Offset: Etc/GMT Created By: Deployment User Kindly awaiting your reply. If any data is missing, please let me know. — — — Met vriendelijke groet / Kind regards, Marko Vrgotic

1 0

Re: One of the brick in the GFS is down ,How can I slove it?
by Amit Bawer 24 Sep '19

24 Sep '19

On Tuesday, September 24, 2019, zhouhao(a)vip.friendtimes.net < zhouhao(a)vip.friendtimes.net> wrote: > sorry ，execute “gluster peer status” lost a brick，I worry about losing > data > > ------------------------------ > zhouhao(a)vip.friendtimes.net > > > *From:* Amit Bawer <abawer(a)redhat.com> > *Date:* 2019-09-24 15:42 > *To:* zhouhao(a)vip.friendtimes.net > *CC:* users <users(a)ovirt.org> > *Subject:* Re: [ovirt-users] Re: One of the brick in the GFS is down ,How > can I slove it? > it is preferable to have all GFS nodes online and running first, you can > follow troubleshooting here: > > https://www.ovirt.org/documentation/gluster-hyperconverged/chap- > Troubleshooting.html > > > On Tue, Sep 24, 2019 at 7:18 AM zhouhao(a)vip.friendtimes.net < > zhouhao(a)vip.friendtimes.net> wrote: > >> >> >> ------------------------------ >> zhouhao(a)vip.friendtimes.net >> >> >> *From:* zhouhao(a)vip.friendtimes.net >> *Date:* 2019-09-24 11:41 >> *To:* zhouhao <zhouhao(a)vip.friendtimes.net>; users <users(a)ovirt.org> >> *Subject:* Re: [ovirt-users] One of the brick in the GFS is down ,How >> can I slove it? >> Can I click The 'start' button?? >> >> >> ------------------------------ >> zhouhao(a)vip.friendtimes.net >> >> >> *From:* zhouhao(a)vip.friendtimes.net >> *Date:* 2019-09-24 11:36 >> *To:* users <users(a)ovirt.org> >> *Subject:* [ovirt-users] One of the brick in the GFS is down ,How can I >> slove it? >> >> *How can I slove it safely? There are 50 vms running in this GFS* >> >> >> *the mail messages below:* >> >> *Time:* 2019-09-24 04:47:20.511 >> *Message:* Detected change in status of brick 192.168.3.16:/vmdata/gfs >> of volume bojoy-GFS of cluster bojoy-cluster from UP to DOWN via cli. >> *Severity:* WARNING >> >> status below: >> >> >> GFS service status >> >> _______________________________________________ >> Users mailing list -- users(a)ovirt.org >> To unsubscribe send an email to users-leave(a)ovirt.org >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> oVirt Code of Conduct: https://www.ovirt.org/community/about/community- >> guidelines/ >> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/ >> message/TBCAJIHSV2AU24IFAMR43GIHRUECWW2V/ >> >

4 3

Unexpected SSO error with first task
by Vrgotic, Marko 24 Sep '19

24 Sep '19

Dear oVirt, In our software tests, we deploy oVirt VMs using ansible and teardown also using ansible. We noticed that each time we run nightly tests, using Jenkins user in ovirt, in one of the Jobs, first task, creating VM, fails with following error, TASK [Login to oVirt] ********************************************************** An exception occurred during task execution. To see the full traceback, use -vvv. The error was: AuthError: Error during SSO authentication invalid_grant : The provided authorization grant for the username and password has expired. fatal: [PVyrZMy0-csm-2.avinity.tv<http://pvyrzmy0-csm-2.avinity.tv/> -> localhost]: FAILED! => {"changed": false, "msg": "Error during SSO authentication invalid_grant : The provided authorization grant for the username and password has expired."} Immediately after, Token seem to be renewed and all other VMs are created without issue. The play looks like following: --- # file: provision-ovirt.yml - name: Create VM in OVirt hosts: ovirt-vms gather_facts: false vars: - state: running pre_tasks: - name: Login to oVirt delegate_to: localhost ovirt_auth: url: "{{ ovirt_engine_url }}" username: "{{ ovirt_engine_user }}" password: "{{ ovirt_engine_password }}" ca_file: "{{ ovirt_engine_cafile | default(omit) }}" insecure: "{{ ovirt_engine_insecure | default(true) }}" run_once: true tasks: - name: "Get agent keys" delegate_to: localhost command: ssh-add -L register: ssh_agent_pubkeys run_once: true changed_when: False - name: Create new VMs from template delegate_to: localhost ovirt_vm: auth: "{{ ovirt_auth }}" name: "{{ inventory_hostname_short }}" nics: - name: nic1 profile_name: tenant1 interface: virtio cloud_init_nics: - nic_name: eth0 nic_boot_protocol: dhcp nic_on_boot: true cloud_init: host_name: "{{ inventory_hostname_short }}" authorized_ssh_keys: "{{ ssh_agent_pubkeys.stdout }}" state: "{{ state }}" cluster: "{{ovirt_cluster}}" template: "{{ images_to_template[os_image] }}" instance_type: "{{ os_flavor_to_ovirt_instance_type[os_flavor] }}" - name: "Wait until the ansible user can log into the host (cloud-init needs to have finished)" command: ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no {{ansible_user}}@{{inventory_hostname}} exit register: ssh_output delegate_to: localhost until: ssh_output.rc == 0 retries: 30 delay: 5 changed_when: False post_tasks: - name: Logout from oVirt delegate_to: localhost run_once: true ovirt_auth: state: absent ovirt_auth: "{{ ovirt_auth }}" tags: - always How does token relates to user session in oVirt? We would expect that once we logout, token should be expired and upon new login just get renewed/new token to proceed. Is it a bug? Is it in the play that we are doing something incorrectly? Kindly awaiting your eply. — — — Met vriendelijke groet / Kind regards, Marko Vrgotic

1 0

the virtual machine crushed and I cant shutdown the vm successfully
by zhouhao＠vip.friendtimes.net 24 Sep '19

24 Sep '19

The trouble bothered me for a long time;Some times my vm will crush and I cant shutdown it,I had to reboot the ovirt-node to slove it; the vm's error below The ovirt-node's error below The vm's threading on the ovirt-node, the IO ratio is 100% and the vm's process change to defunct I can not kill it,every time I had to shutdown the ovirt-node on the engine website ,the vm's status always in the way to shutdown ,even if I wait for it for hours; It either failed or is shutting down and the "power off" cant shutdown the vm too. ---------------------------------------------------------------- The other infomation about my ovirt-node node-version： node hardware zhouhao(a)vip.friendtimes.net

4 7

Reboot a guest via ansible in oVirt
by Gianluca Cecchi 24 Sep '19

24 Sep '19

Hello, I see that there is this Ansible module for vSphere that contains also the state "restarted": https://docs.ansible.com/ansible/latest/modules/vmware_guest_powerstate_mod… For oVirt I see the ovirt_vm module where state can be one of: [ absent, next_run, present, registered, running, stopped, suspended, exported ] Can we think to add a restart / restarted one to it too? BTW: I see discrepancy between this page: https://www.ovirt.org/develop/release-management/features/infra/ansible_mod… where many module names and links to github source are plural (eg ovirt_disks) and actual ansible docs and github sources that are singular (eg. ovirt_disk) Thanks, Gianluca

2 1

Creating Bricks via the REST API
by Julian Schill 24 Sep '19

24 Sep '19

Under "Compute -> Hosts -> Host Name -> Storage Devices" one can create new Gluster Bricks on free partitions. How can i do this via the REST API?

4 5

ovirtmgmt STP disabled - any reason behind that
by Strahil Nikolov 23 Sep '19

23 Sep '19

Hi All, I recently got an issue and due to lack of debug time , I'm still not sure if it was a loop on the network.Can someone clarify why the ovirtmgmt bridge has STP disabled ? Any reason behind that ? Also, what is the proper way to enable STP on those bridges ? For now , I have set highest priority of the STP on my router and I don't expect the issue to reoccur (if it really was a loop), but I want to avoid future issues like that. Best Regards,Strahil Nikolov

1 0

VDI
by Fabio Marzocca 23 Sep '19

23 Sep '19

Is there anyone who uses oVirt as a full VDI environment? I would have a bunch of questions...

2 1

Re: Creating Bricks via the REST API
by Strahil 23 Sep '19

23 Sep '19

What about gluster API ? I think that's a doable solution . Best Regards, Strahil NikolovOn Sep 23, 2019 18:38, Amit Bawer <abawer(a)redhat.com> wrote: > > +Sahina Bose > Thanks for your clarification Julian, creating new Gluster brick from ovirt's REST API is not currently supported, only from UI. > > On Mon, Sep 23, 2019 at 6:25 PM Julian Schill <schill-julian(a)gmx.de> wrote: >> >> > This is documented in section 6.92.2. of [1] >> > >> > [1] >> > https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.3/... >> >> Section 6.92.2 documents a POST request that lets one add *existing* bricks to an existing volume. >> This doesn't answer my question on how one can create *new* bricks via the REST API. >> _______________________________________________ >> Users mailing list -- users(a)ovirt.org >> To unsubscribe send an email to users-leave(a)ovirt.org >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/QMPFTXA2WX3JM…

1 0

Re: oVirt + Gluster
by Strahil 23 Sep '19

23 Sep '19

Recently both oVirt node and Hypervisor have moved to gluster v6 (although I have moved a little bit earlier). For me v6 of Gluster is quite stable and if you dig up really nice , you will be able to even setup 'gtop' which gives great info. Ovirt is quite well integrated with gluster - doing health checks ,provide monitoring, allow creating additional gluster volumes and many more. Still, if you have FC available - it should be way easier to setup and tune. Docs are one of the weak stuff , but you can always check RHV (4.0+) and Oracle docs - as oVirt is upstream of RedHat's and Oracle's Virtualization. Best Regards, Strahil NikolovOn Sep 23, 2019 17:03, Edward Berger <edwberger(a)gmail.com> wrote: > > Current oVirt node-ng uses 6 > > ]# yum list installed |grep gluster > gluster-ansible-cluster.noarch 1.0.0-1.el7 installed > gluster-ansible-features.noarch 1.0.5-3.el7 installed > gluster-ansible-infra.noarch 1.0.4-3.el7 installed > gluster-ansible-maintenance.noarch 1.0.1-1.el7 installed > gluster-ansible-repositories.noarch 1.0.1-1.el7 installed > gluster-ansible-roles.noarch 1.0.5-4.el7 installed > glusterfs.x86_64 6.4-1.el7 installed > glusterfs-api.x86_64 6.4-1.el7 installed > glusterfs-cli.x86_64 6.4-1.el7 installed > glusterfs-client-xlators.x86_64 6.4-1.el7 installed > glusterfs-events.x86_64 6.4-1.el7 installed > glusterfs-fuse.x86_64 6.4-1.el7 installed > glusterfs-geo-replication.x86_64 6.4-1.el7 installed > glusterfs-libs.x86_64 6.4-1.el7 installed > glusterfs-rdma.x86_64 6.4-1.el7 installed > glusterfs-server.x86_64 6.4-1.el7 installed > libvirt-daemon-driver-storage-gluster.x86_64 4.5.0-10.el7_6.12 installed > python2-gluster.x86_64 6.4-1.el7 installed > vdsm-gluster.x86_64 4.30.24-1.el7 installed > > On Mon, Sep 23, 2019 at 12:22 AM TomK <tomkcpr(a)mdevsys.com> wrote: >> >> Hey All, >> >> Seeing GlusterFS up to version 7, do any of the oVirt versions support >> anything higher then 3.X? >> >> Or is Gluster not the preferred distributed file system choice for oVirt? >> >> -- >> Thx, >> TK. >> _______________________________________________ >> Users mailing list -- users(a)ovirt.org >> To unsubscribe send an email to users-leave(a)ovirt.org >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/NNKXLMIGLV3GJ…

1 0

oVirt + Gluster
by TomK 23 Sep '19

23 Sep '19

Hey All, Seeing GlusterFS up to version 7, do any of the oVirt versions support anything higher then 3.X? Or is Gluster not the preferred distributed file system choice for oVirt? -- Thx, TK.

2 1

Re: oVirt + Gluster
by Strahil 23 Sep '19

23 Sep '19

On the opposite - ovirt supports up to v6 of gluster and in cockpit, you can setup your hyperconverged setup (on gluster) abd your hosted engine ontop. Best Regards, Strahil NikolovOn Sep 23, 2019 07:20, TomK <tomkcpr(a)mdevsys.com> wrote: > > Hey All, > > Seeing GlusterFS up to version 7, do any of the oVirt versions support > anything higher then 3.X? > > Or is Gluster not the preferred distributed file system choice for oVirt? > > -- > Thx, > TK. > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/NNKXLMIGLV3GJ…

2 1

ifcfg-bond0 getting overwritten.
by TomK 22 Sep '19

22 Sep '19

Hey All, My ifcfg-bond0 is getting overwritten on reboot. Need to add a search string to the configuration. What's overwriting it and should I be trying to add a search string to another location? -- Thx, TK.

3 4

Re: ifcfg-bond0 getting overwritten.
by Strahil 22 Sep '19

22 Sep '19

This depends if you use NetworkManager.service for other connections. If you don't use NetworkManager , then that's a working approach. Best Regards, Strahil NikolovOn Sep 22, 2019 18:08, Bell Levin <blevin(a)redhat.com> wrote: > > Hey, > I would like to add a small point to Edward's explanation. > > If the config does really get overwritten, it probably means that more was changed to the bond but the search string. > If indeed what you are looking for is to add a dns search line, you should add it to /etc/resolv.conf. > > On Sun, Sep 22, 2019 at 4:44 PM Edward Berger <edwberger(a)gmail.com> wrote: >> >> vdsm creates persistant network configs that overwrite manual changes at reboot >> in /var/lib/vdsm/persistence/netconf >> >> You can check your other hosts for any differences there. >> >> It is recommended that networks are set up and managed through ovirt engine. >> >> On Sun, Sep 22, 2019 at 6:01 AM TomK <tomkcpr(a)mdevsys.com> wrote: >>> >>> This occurs on only one of the oVirt hosts in my cluster. >>> -- >>> Thx, >>> TK. >>> >>> On 9/22/2019 12:08 AM, TomK wrote: >>> > Hey All, >>> > >>> > My ifcfg-bond0 is getting overwritten on reboot. Need to add a search >>> > string to the configuration. >>> > >>> > What's overwriting it and should I be trying to add a search string to >>> > another location? >>> > >>> _______________________________________________ >>> Users mailing list -- users(a)ovirt.org >>> To unsubscribe send an email to users-leave(a)ovirt.org >>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/K5QTMQX4OGCKQ… >> >> _______________________________________________ >> Users mailing list -- users(a)ovirt.org >> To unsubscribe send an email to users-leave(a)ovirt.org >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/DXIDOP24DKP7A…

1 0

Linked Clones?
by marzoccafabio＠gmail.com 22 Sep '19

22 Sep '19

Hi, I am approaching and studying oVirt in order to propose the solution to a customer as a replacement for a commercial solution they have now. They only need Desktop virtualization. Sorry for the silly question, but I can't find a way to deploy a VM (template) to users as a "linked-clone", meaning that the users' image still refers to the original image but modification are written (and afterwards read) from a new location. This technique is called Copy-on-write. Can this be achieved with oVirt? Fabio

3 2

CentOS 7.7 has been released: note it's not supported for oVirt 4.2
by Sandro Bonazzola 19 Sep '19

19 Sep '19

CentOS 7.7 has been released this week. Please note that CentOS 7.7 is not supported for oVirt 4.2 and older releases. If you want to upgrade to CentOS 7.7 please upgrade to oVirt 4.3 too. If you don't plan to upgrade to oVirt 4.3 please be sure your yum repo files points to http://mirror.centos.org/centos/7.6.1810/ instead of http://mirror.centos.org/centos/7/. The http://mirror.centos.org/centos/7/ URL is now pointing to http://mirror.centos.org/centos/7.7.1908/ so yum upgrade will bring you 7.7 content which may break your 4.2 installation. Thanks, -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/>*Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

2 2

Re: Upgrade from 4.3.5 rc5 to 4.3.6 rc5 strange errors
by Strahil 19 Sep '19

19 Sep '19

Yeah, I've noticed I was searching for the wrong library (patching from phone seems to be not such a smart idea :) ) . Installing libibverbs fixed the issue, and a bug is opened so devs can find where the deps were missing. Did you notice this behaviour on your setup ? Best Regards, Strahil NikolovOn Sep 19, 2019 10:49, Guillaume Pavese <guillaume.pavese(a)interactiv-group.com> wrote: > > "yum provides */libibverbs.so" will tell you that this file is provided by rdma-core-devel > So to fix this messages when you login, you can do : "yum install rdma-core-devel" > > > Guillaume Pavese > Ingénieur Système et Réseau > Interactiv-Group > > > On Thu, Sep 19, 2019 at 1:56 AM Strahil <hunter86_bg(a)yahoo.com> wrote: >> >> Hello Community, >> >> I was waiting for 7.7 to come up before I upgrade my lab. >> >> The engine got upgraded without issues, same for OS - but after the system powered up I receive the following: >> >> [root@ovirt1 ~]# ssh engine >> root@engine's password: >> Last login: Wed Sep 18 19:48:32 2019 from ovirt1.localdomainnet_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory >> net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5) >> PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory >> PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4) >> net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory >> net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5) >> PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory >> PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4) >> [root@engine ~]# >> >> Anyone else experiecing this ? >> I can't find libmlx4 or libmlx5 in the repos. >> >> P.S.: I have a gluster snapshot, so I can rollback if needed. >> For now I will postpone my hosts' upgrade. >> >> Best Regards, >> Strahil Nikolov >> >> _______________________________________________ >> Users mailing list -- users(a)ovirt.org >> To unsubscribe send an email to users-leave(a)ovirt.org >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/PYBXMOZ7EVW4O… > > > Ce message et toutes les pièces jointes (ci-après le “message”) sont établis à l’intention exclusive de ses destinataires et sont confidentiels. Si vous recevez ce message par erreur, merci de le détruire et d’en avertir immédiatement l’expéditeur. Toute utilisation de ce message non conforme a sa destination, toute diffusion ou toute publication, totale ou partielle, est interdite, sauf autorisation expresse. L’internet ne permettant pas d’assurer l’intégrité de ce message . Interactiv-group (et ses filiales) décline(nt) toute responsabilité au titre de ce message, dans l’hypothèse ou il aurait été modifié. IT, ES, UK.

2 1

disable selinux ?
by Nardus Geldenhuys 19 Sep '19

19 Sep '19

Hi Hope you are well. Quick question. Can I disable SELINUX on the ovirt nodes ? Will there be any issues. I know that you cant migrate from SELINUX to DISABLED SELINUX nodes. Regards Nar

2 1

Best Practices to update a Windows VM
by marzoccafabio＠gmail.com 19 Sep '19

19 Sep '19

Once created a "main" VM we keep it alone to embed Windows updates. Then we clone this into a "service" VM which we convert into a Template to deploy to users. Now, what is the Best Practice to update WIndows OS for the all the users?

1 0

Disk locked after backup
by Fabio Cesar Hansen 19 Sep '19

19 Sep '19

Hi. I am using the https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/download… script to backup my vms. The problem is that always after the backup finishes, the snapshot is locked. Has anyone ever experienced this? Node version: OS Version: RHEL - 7 - 6.1810.2.el7.centos OS Description: oVirt Node 4.3.5.2 Kernel Version: 3.10.0 - 957.27.2.el7.x86_64 KVM Version: 2.12.0 - 18.el7_6.7.1 LIBVIRT Version: libvirt-4.5.0-10.el7_6.12 VDSM Version: vdsm-4.30.24-1.el7 Engine version: 4.3.5.5-1.el7 Thanks

2 1

Re: disable selinux ?
by Strahil 19 Sep '19

19 Sep '19

I think I used to have a vdsm issue a long time ago. Why would you disable SELINUX, it provides various options to adapt to non-SELINUX aware apps. Best Regards, Strahil NikolovOn Sep 19, 2019 10:54, Nardus Geldenhuys <nardusg(a)gmail.com> wrote: > > Hi > > Hope you are well. Quick question. > > Can I disable SELINUX on the ovirt nodes ? Will there be any issues. I know that you cant migrate from SELINUX to DISABLED SELINUX nodes. > > Regards > > Nar

1 0

Re: Linked Clones?
by Strahil 19 Sep '19

19 Sep '19

At least KVM can do that. If I understood correctly, you want a base disk for multiple VMs and the delta should be stored in separate disk, right? I am using VDO (with gluster) under my storage, so any duplicate data is deduplicated (and if yiu enable compression - compressed) by VDO. Have you thought about VDO - it works with almost any block storage ?Of course both technology will reduce the I/O performance compared to raw storage. Also, let's see if actually oVirt has such option... Best Regards, Strahil Nikolov On Sep 19, 2019 10:23, marzoccafabio(a)gmail.com wrote: > > Hi, > > I am approaching and studying oVirt in order to propose the solution to a customer as a replacement for a commercial solution they have now. > They only need Desktop virtualization. > Sorry for the silly question, but I can't find a way to deploy a VM (template) to users as a "linked-clone", meaning that the users' image still refers to the original image but modification are written (and afterwards read) from a new location. This technique is called Copy-on-write. > Can this be achieved with oVirt? > > Fabio > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/3XITFLEJDICOF…

1 0

Upgrade from 4.3.5 rc5 to 4.3.6 rc5 strange errors
by Strahil 19 Sep '19

19 Sep '19

Hello Community, I was waiting for 7.7 to come up before I upgrade my lab. The engine got upgraded without issues, same for OS - but after the system powered up I receive the following: [root@ovirt1 ~]# ssh engine root@engine's password: Last login: Wed Sep 18 19:48:32 2019 from ovirt1.localdomainnet_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5) PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4) net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5) PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4) [root@engine ~]# Anyone else experiecing this ? I can't find libmlx4 or libmlx5 in the repos. P.S.: I have a gluster snapshot, so I can rollback if needed. For now I will postpone my hosts' upgrade. Best Regards, Strahil Nikolov

2 1

how to add devices to a qemu config.xml of a virtual machine
by pascal＠butterflyit.com 18 Sep '19

18 Sep '19

Hi, I would like to be able to share a local folder with my remote desktop hosted by ovirt. Looking everywhere it seems that what is needed is to have spice-webdavd installed on the guest and have qemu set with this channel org.spice-space.webdav.0 I don't see in the web admin interface any ways to add those channels, but found on the web an addon which could maybe work vdsm-hook-qemucmdline but can't find a package compiled for the latest version and how to install it Also I noticed in the latest version of ovirt 4.3.5 there are 2 flags: Enable SPICE file transfer Enable SPICE clipboard copy and paste But that doesn't seem to change anything Any help appreciated

2 2

Disk locked after backup
by fchansen＠outlook.com 18 Sep '19

18 Sep '19

Hi. I am using the https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/download… script to backup my vms. The problem is that always after the backup finishes, the snapshot is locked. Has anyone ever experienced this? Node version: OS Version: RHEL - 7 - 6.1810.2.el7.centos OS Description: oVirt Node 4.3.5.2 Kernel Version: 3.10.0 - 957.27.2.el7.x86_64 KVM Version: 2.12.0 - 18.el7_6.7.1 LIBVIRT Version: libvirt-4.5.0-10.el7_6.12 VDSM Version: vdsm-4.30.24-1.el7 Engine version: 4.3.5.5-1.el7 Thanks

1 0

Disk locked after backup
by fchansen＠outlook.com 18 Sep '19

18 Sep '19

Hi. I am using the https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/download… script to backup my vms. The problem is that always after the backup finishes, the snapshot is locked. Has anyone ever experienced this? Node version: OS Version: RHEL - 7 - 6.1810.2.el7.centos OS Description: oVirt Node 4.3.5.2 Kernel Version: 3.10.0 - 957.27.2.el7.x86_64 KVM Version: 2.12.0 - 18.el7_6.7.1 LIBVIRT Version: libvirt-4.5.0-10.el7_6.12 VDSM Version: vdsm-4.30.24-1.el7 Engine version: 4.3.5.5-1.el7 Thanks

1 0

Disk locked after backup
by Fabio Cesar Hansen 18 Sep '19

18 Sep '19

Hi. I am using the https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/download… script to backup my vms. The problem is that always after the backup finishes, the snapshot is locked. Has anyone ever experienced this? Node version: OS Version: RHEL - 7 - 6.1810.2.el7.centos OS Description: oVirt Node 4.3.5.2 Kernel Version: 3.10.0 - 957.27.2.el7.x86_64 KVM Version: 2.12.0 - 18.el7_6.7.1 LIBVIRT Version: libvirt-4.5.0-10.el7_6.12 VDSM Version: vdsm-4.30.24-1.el7 Engine version: 4.3.5.5-1.el7 Thanks

1 0

Re: Changing ISO domains
by Strahil 18 Sep '19

18 Sep '19

Actually ISO domains are deprecated as stated by the devs in https://bugzilla.redhat.com/show_bug.cgi?id=1673059 There is no warning yet, which makes the confusion. Best Regards, Strahil NikolovOn Sep 18, 2019 15:22, Gianluca Cecchi <gianluca.cecchi(a)gmail.com> wrote: > > On Wed, Sep 18, 2019 at 12:41 PM Mark Steele <msteele(a)telvue.com> wrote: >> >> Well - not in 3.5 :-) >> >> *** >> Mark Steele >> CIO / VP Technical Operations | TelVue Corporation >> TelVue - We Share Your Vision >> 16000 Horizon Way, Suite 100 | Mt. Laurel, NJ 08054 >> 800.885.8886 x128 | msteele(a)telvue.com | http://www.telvue.com >> twitter: http://twitter.com/telvue | facebook: https://www.facebook.com/telvue >> >> >> On Wed, Sep 18, 2019 at 6:24 AM Strahil <hunter86_bg(a)yahoo.com> wrote: >>> >>> ISO domains are deprecated. >>> I guess you can upload to regular data domains. >>> >>> Best Regards, >>> Strahil Nikolov > > But afaik also on 4.3 only export domains are deprecated, not ISO ones. > See here for RHV too: > https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.3/ht… > > Still it remains the problem of using ISOs on block based data domains when trying to swap CDs > https://bugzilla.redhat.com/show_bug.cgi?id=1589763 > > So in these scenarios ISO domains are still needed > HIH, > Gianluca

1 0

Re: Upgrade from 4.3.5 rc5 to 4.3.6 rc5 strange errors
by Strahil 18 Sep '19

18 Sep '19

Opened a bug in bugzilla (https://bugzilla.redhat.com/show_bug.cgi?id=1753363) , if it is a duplicate - please send me an e-mail to relate to my bug post. Best Regards, Strahil NikolovOn Sep 18, 2019 20:18, Strahil <hunter86_bg(a)yahoo.com> wrote: > > Hm... It seems that I missed the complain about libibverbs.so.1 which is part of package libibverbs-22.1-3.el7.x86_64 . > > Once I have deployed that, no more issues are observed. > > I guess there is a missing dependency. > > Best Regards, > Strahil Nikolov > > On Sep 18, 2019 19:57, Strahil <hunter86_bg(a)yahoo.com> wrote: >> >> Hello Community, >> >> I was waiting for 7.7 to come up before I upgrade my lab. >> >> The engine got upgraded without issues, same for OS - but after the system powered up I receive the following: >> >> [root@ovirt1 ~]# ssh engine >> root@engine's password: >> Last login: Wed Sep 18 19:48:32 2019 from ovirt1.localdomainnet_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory >> net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5) >> PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory >> PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4) >> net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory >> net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5) >> PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory >> PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4) >> [root@engine ~]# >> >> Anyone else experiecing this ? >> I can't find libmlx4 or libmlx5 in the repos. >> >> P.S.: I have a gluster snapshot, so I can rollback if needed. >> For now I will postpone my hosts' upgrade. >> >> Best Regards, >> Strahil Nikolov

1 0

Re: Upgrade from 4.3.5 rc5 to 4.3.6 rc5 strange errors
by Strahil 18 Sep '19

18 Sep '19

Hm... It seems that I missed the complain about libibverbs.so.1 which is part of package libibverbs-22.1-3.el7.x86_64 . Once I have deployed that, no more issues are observed. I guess there is a missing dependency. Best Regards, Strahil NikolovOn Sep 18, 2019 19:57, Strahil <hunter86_bg(a)yahoo.com> wrote: > > Hello Community, > > I was waiting for 7.7 to come up before I upgrade my lab. > > The engine got upgraded without issues, same for OS - but after the system powered up I receive the following: > > [root@ovirt1 ~]# ssh engine > root@engine's password: > Last login: Wed Sep 18 19:48:32 2019 from ovirt1.localdomainnet_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory > net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5) > PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory > PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4) > net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory > net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5) > PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory > PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4) > [root@engine ~]# > > Anyone else experiecing this ? > I can't find libmlx4 or libmlx5 in the repos. > > P.S.: I have a gluster snapshot, so I can rollback if needed. > For now I will postpone my hosts' upgrade. > > Best Regards, > Strahil Nikolov

1 0

no hosted engine deploy while adding new host (oVirt 4.3.5)
by lars.stolpe＠bvg.de 18 Sep '19

18 Sep '19

I tryed to add a new host to my oVirt environment. I remember a checkbox whether the engine HA wil depoyed or not in oVirt 4.1. Now the environmentist oVirt 4.3.5. In the add host dialog windows I cant find that option anymore. In edit window after host installation ther ist also no hosted engine tab. hosted-engine --deploy don't ask, if it is an additional host (like it used to in 4.1), so it will install a new engine instead of depoying the existing engine. Is there a new way of depoying the engine HA to additional hosts?

2 2

Find MAC for VM
by Colin Coe 18 Sep '19

18 Sep '19

Hi all I've got the following Python snippet which works fine under 4.3 but I need it to work under 4.1 as well. --- nics_service = vms_service.vm_service(vm.id).nics_service() nic = nics_service.list(search='name=nic1')[0] --- Under 4.1, it barfs on "nics_service.list(search='name=nic1')[0]". What do I need to do to make it work under 4.1? Thanks

2 5

Re: Changing ISO domains
by Strahil 18 Sep '19

18 Sep '19

ISO domains are deprecated. I guess you can upload to regular data domains. Best Regards, Strahil NikolovOn Sep 17, 2019 16:50, Mark Steele <msteele(a)telvue.com> wrote: > > Hello, > > I recently removed our ISO domain from out cluster (oVirt Engine Version: 3.5.0.1-1.el6) because it was on a storage location we no longer want to use. > > I am now not clear on how to add the new ISO domain in. I created a new ISO domain type via storage and pointed to an NFS share on the new storage, however it does not behave as I anticipated (I cannot see ISO's when creating new VM's) > > I suspect I missed a step - how are new ISO domains added to a cluster? > > Mark > > *** > Mark Steele > CIO / VP Technical Operations | TelVue Corporation > TelVue - We Share Your Vision > 16000 Horizon Way, Suite 100 | Mt. Laurel, NJ 08054 > 800.885.8886 x128 | msteele(a)telvue.com | http://www.telvue.com > twitter: http://twitter.com/telvue | facebook: https://www.facebook.com/telvue

3 2

ovirt-imagio-proxy upload speed slow
by Dev Ops 18 Sep '19

18 Sep '19

I am working on integrating a backup solution for our ovirt environment and having issues with the time it takes to backup the VM's. This backup solution is simply taking a snapshot and making a clone and backing the clone up to a backup server. A VM that is 100 gig takes 52 minutes to back up. The same VM doing a file backup using the same product, and bypassing their rhv plugin, takes 14 minutes. So the throughput is there but the ovirt imageio-proxy process seems to be what manages how images are uploaded and is officially my bottle neck. Load is not high on the engine or kvm hosts. I had bumped up the Upload image size from 100MB to 10gig weeks ago and that didn't seem to help. [root@blah-lab-engine ~]# engine-config -a |grep Upload UploadImageChunkSizeKB: 10240000 version: general [root@bgl-vms-engine ~]# rpm -qa |grep ovirt-image ovirt-imageio-proxy-1.4.6-1.el7.noarch ovirt-imageio-common-1.4.6-1.el7.x86_64 ovirt-imageio-proxy-setup-1.4.6-1.el7.noarch I have seen bugs reported to redhat about this but I am running above the affected releases. engine software is 4.2.8.2-1.el7 Any idea what we can tweak to open up this bottleneck?

5 8

Re: Changing ISO domains
by Strahil 18 Sep '19

18 Sep '19

Yes. But check SELINUX and DAC permissions. Best Regards, Strahil NikolovOn Sep 17, 2019 18:04, Mark Steele <msteele(a)telvue.com> wrote: > > OK - so if I put them in the 1111-11111... directory with scp that should work? > > > *** > Mark Steele > CIO / VP Technical Operations | TelVue Corporation > TelVue - We Share Your Vision > 16000 Horizon Way, Suite 100 | Mt. Laurel, NJ 08054 > 800.885.8886 x128 | msteele(a)telvue.com | http://www.telvue.com > twitter: http://twitter.com/telvue | facebook: https://www.facebook.com/telvue > > > On Tue, Sep 17, 2019 at 10:57 AM Derek Atkins <derek(a)ihtfp.com> wrote: >> >> If you push down a level or two you'll see your .iso files. >> I just scp them directly in. >> -derek >> >> >> On Tue, September 17, 2019 10:52 am, Mark Steele wrote: >> > Additional information: >> > >> > The directory for the domain appear to have been created properly so I am >> > not clear on why the upload from the ovirt engine is failing: >> > drwxr-xr-x. 3 36 36 88 Sep 17 10:40 . >> > drwxr-xr-x. 5 36 36 126 Oct 17 2018 .. >> > drwxr-xr-x. 4 36 36 46 Sep 17 10:40 bdd2a547-dde9-4248-b2de-ae67063da8e4 >> > -rwxr-xr-x. 1 36 36 0 Sep 17 10:50 __DIRECT_IO_TEST__ >> > [root@phl-tevestore-01 iso-store]# >> > >> > *** >> > *Mark Steele* >> > CIO / VP Technical Operations | TelVue Corporation >> > TelVue - We Share Your Vision >> > 16000 Horizon Way, Suite 100 | Mt. Laurel, NJ 08054 >> > 800.885.8886 x128 | msteele(a)telvue.com | http://www.telvue.com >> > twitter: http://twitter.com/telvue | facebook: >> > https://www.facebook.com/telvue >> > >> > >> > On Tue, Sep 17, 2019 at 10:46 AM Mark Steele <msteele(a)telvue.com> wrote: >> > >> >> I was not using the uploader - I have the new domain active and attached >> >> now. >> >> >> >> I am attempting to upload the iso using the following command: >> >> >> >> engine-iso-uploader --iso-domain=phl-iso-03 upload >> >> ./windows-server-2012.iso >> >> >> >> Unfortunately I keep getting this error: >> >> >> >> Uploading, please wait... >> >> ERROR: mount.nfs: Connection timed out >> >> >> >> >> >> >> >> *** >> >> *Mark Steele* >> >> CIO / VP Technical Operations | TelVue Corporation >> >> TelVue - We Share Your Vision >> >> 16000 Horizon Way, Suite 100 | Mt. Laurel, NJ 08054 >> >> 800.885.8886 x128 | msteele(a)telvue.com | http://www.telvue.com >> >> twitter: http://twitter.com/telvue

1 0

Re: Disaster recovery active-active question
by Strahil 18 Sep '19

18 Sep '19

We use Active-Active Hp XP replication between 2 DCs for our VmWare Clusters. I guess you can use the same replication for oVirt or even Hyper-V. Best Regards, Strahil NikolovOn Sep 17, 2019 14:30, wodel youchi <wodel.youchi(a)gmail.com> wrote: > > Hi, > > On the active-active DR, in the documentation it is said : > "You require replicated storage that is writeable on both sites to allow virtual machines to migrate between sites and continue running on the site’s storage." > > I know that it's not a direct oVirt question, but if someone has implemented this ... > > Which type of storage that can offer both real time synchronization and read/write on both ends on the replicated volumes? > > > Regards.

1 0

GFS/GFS2
by marzoccafabio＠gmail.com 18 Sep '19

18 Sep '19

Is GFS/GFS2 supported by oVirt as shared storage?

3 3

FW: user role
by Staniforth, Paul 18 Sep '19

18 Sep '19

Hi Kim, If you add them to a group in AD then add the users and group to oVirt you can add the permissions to the group. Then add the VmCreator and DiskProfileUser to the Datacenter. Add the UserTemplateBasedVM permission to any templates you want them to be able to use Add the VnicProfileUser permission to any vNIC profiles they should have access ( make sure public isn't enabled on any vNIC profiles you don't want them to have access to) When they create a VM they will then get UserVmManager permission and the DiskOperator permissions on any disks. This means they should be able to do all 1 to 8 for their own VMs and also delete their VMs and Disks. Paul Staniforth To view the terms under which this email is distributed, please go to:- http://leedsbeckett.ac.uk/disclaimer/email/

1 0

Changing ISO domains
by Mark Steele 17 Sep '19

17 Sep '19

Hello, I recently removed our ISO domain from out cluster (oVirt Engine Version: 3.5.0.1-1.el6) because it was on a storage location we no longer want to use. I am now not clear on how to add the new ISO domain in. I created a new ISO domain type via storage and pointed to an NFS share on the new storage, however it does not behave as I anticipated (I cannot see ISO's when creating new VM's) I suspect I missed a step - how are new ISO domains added to a cluster? Mark *** *Mark Steele* CIO / VP Technical Operations | TelVue Corporation TelVue - We Share Your Vision 16000 Horizon Way, Suite 100 | Mt. Laurel, NJ 08054 800.885.8886 x128 | msteele(a)telvue.com | http://www.telvue.com twitter: http://twitter.com/telvue | facebook: https://www.facebook.com/telvue

3 9

Disaster recovery active-active question
by wodel youchi 17 Sep '19

17 Sep '19

Hi, On the active-active DR, in the documentation it is said : "You require replicated storage that is writeable on both sites to allow virtual machines to migrate between sites and continue running on the site’s storage." I know that it's not a direct oVirt question, but if someone has implemented this ... Which type of storage that can offer both real time synchronization and read/write on both ends on the replicated volumes? Regards.

1 0

Re: the virtual machine crushed and I cant shutdown the vm successfully
by Strahil 17 Sep '19

17 Sep '19

Have you checked the Hypervisor's logs. I would start with logs from libvirt and vdsm (host) and then the logs on your engine. What OS is your VM and the version of the VM itself (you can check in the UI). Best Regards, Starhil NikolovOn Sep 17, 2019 10:56, zhouhao(a)vip.friendtimes.net wrote: > > There is no useful information on Google. I can't solve this problem. I can only restart the ovirt-node > > ________________________________ > zhouhao(a)vip.friendtimes.net >> >> >> From: tedhima(a)yahoo.co.uk >> Date: 2019-09-17 15:30 >> To: users; devel; zhouhao(a)vip.friendtimes.net >> Subject: Re: [ovirt-users] the virtual machine crushed and I cant shutdown the vm successfully >> Probably related to XFS defragmentation, have you tried to google the error message? >> >> https://bugzilla.kernel.org/show_bug.cgi?id=73831 >> https://centos.org/forums/viewtopic.php?t=52412 >> https://blog.codecentric.de/en/2017/04/xfs-possible-memory-allocation-deadl… >> On Tuesday, 17 September 2019, 13:55:45 GMT+7, zhouhao(a)vip.friendtimes.net <zhouhao(a)vip.friendtimes.net> wrote: >> >> >>> >>> The trouble bothered me for a long time;Some times my vm will crush and I cant shutdown it,I had to reboot the ovirt-node to slove it; >>> the vm's error below >>> >>> The ovirt-node's error below >>> >>> The vm's threading on the ovirt-node, the IO ratio is 100% >>> >>> >>> and the vm's process change to defunct >>> I can not kill it,every time I had to shutdown the ovirt-node >>> >>> on the engine website ,the vm's status always in the way to shutdown ,even if I wait for it for hours; >>> <

1 0

Gluster: Bricks remove failed
by toslavik＠yandex.ru 17 Sep '19

17 Sep '19

Hi. There is an ovirt-hosted-engine on gluster volume engine Replicate replica count 3 Migrate to other drives. I do: gluster volume add-brick engine clemens:/gluster-bricks/engine tiberius:/gluster-bricks/engine octavius:/gluster-bricks/engine force volume add-brick: success gluster volume remove-brick engine tiberius:/engine/datastore clemens:/engine/datastore octavius:/engine/datastore start volume remove-brick start: success ID: dd9453d3-b688-4ed8-ad37-ba901615046c gluster volume remove-brick engine octavius:/engine/datastore status Node Rebalanced-files size scanned failures skipped status run time in h:m:s --------- ----------- ----------- ----------- ----------- ----------- ------------ -------------- localhost 7 50.0GB 34 1 0 completed 0:00:02 clemens 11 21.0MB 31 0 0 completed 0:00:02 tiberius 12 25.0MB 36 0 0 completed 0:00:02 gluster volume status engine Status of volume: engine Gluster process TCP Port RDMA Port Online Pid ------------------------------------------------------------------------------ Brick octavius:/engine/datastore 49156 0 Y 15669 Brick tiberius:/engine/datastore 49156 0 Y 15930 Brick clemens:/engine/datastore 49156 0 Y 16193 Brick clemens:/gluster-bricks/engine 49159 0 Y 6168 Brick tiberius:/gluster-bricks/engine 49163 0 Y 29524 Brick octavius:/gluster-bricks/engine 49159 0 Y 50056 Self-heal Daemon on localhost N/A N/A Y 50087 Self-heal Daemon on clemens N/A N/A Y 6263 Self-heal Daemon on tiberius N/A N/A Y 29583 Task Status of Volume engine ------------------------------------------------------------------------------ Task : Remove brick ID : dd9453d3-b688-4ed8-ad37-ba901615046c Removed bricks: tiberius:/engine/datastore clemens:/engine/datastore octavius:/engine/datastore Status : completed But the data did not migrate du -hs /gluster-bricks/engine/ /engine/datastore/ 49M /gluster-bricks/engine/ 20G /engine/datastore/ Can you give some advice?

2 3

oVirt 4.3 gluster hooks
by Staniforth, Paul 17 Sep '19

17 Sep '19

Hello, In an oVirt 4.3 cluster there are a number of gluster hooks out of sync. When I try to resolve any of the conflicts from the engine it comes back with operation cancelled with the error message "Error while executing action Update Gluster Hook: Internal Engine Error" In the engine log there are a number of errors including 2019-09-16 10:38:28,584+01 ERROR [org.ovirt.engine.core.vdsbroker.gluster.UpdateGlusterHookVDSCommand] (EE-ManagedThreadFactory-engine-Thread-2210463) [62ee29dd-5f80-47ad-8e5a-c97e7c3b7acb] Command 'UpdateGlusterHookVDSCommand(HostName = hostname@domainname, GlusterHookVDSParameters:{hostId='86571cea-a20c-4087-8bbf-103d26b3c4eb'})' execution failed: VDSGenericException: VDSErrorException: Failed to UpdateGlusterHookVDS, error = Internal JSON-RPC error: {'reason': "Attempt to call function: <bound method GlusterHook.update of <vdsm.gluster.apiwrapper.GlusterHook object at 0x7fb34421f510>> with arguments: (u'add-brick', u'PRE', u'28Quota-enable-root-xattr-heal.sh', u'IyEvYmluL3NoCgojIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjCiMjIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KIyMgVGhlIHNjcmlwdHMKIyMgSS4gICBhZGQtYnJpY2svcHJlL1MyOFF1b3RhLXJvb3QteGF0dHItaGVhbC5zaCAoaXRzZWxmKQojIyBJSS4gIGFkZC1icmljay9wb3N0L2Rpc2FibGVkLXJvb3QteGF0dHItaGVhbC5zaCBBTkQKIyMgY29sbGVjdGl2ZWx5IGFyY2hpZXZlcyB0aGUgam9iIG9mIGhlYWxpbmcgdGhlICdsaW1pdC1zZXQnIHhhdHRyIHVwb24KIyMgYWRkLWJyaWNrIHRvIHRoZSBnbHVzdGVyIHZvbHVtZS4KIyMKIyMgVGhpcyBzY3JpcHQgaXMgdGhlICdjb250cm9sbGluZycgc2NyaXB0LiBVcG9uIGFkZC1icmljayB0aGlzIHNjcmlwdCBlbmFibGVzCiMjIHRoZSBjb3JyZXNwb25kaW5nIHNjcmlwdCBiYXNlZCBvbiB0aGUgc3RhdHVzIG9mIHRoZSB2b2x1bWUuCiMjIElmIHZvbHVtZSBpcyBzdGFydGVkIC0gZW5hYmxlIGFkZC1icmljay9wb3N0IHNjcmlwdAojIyBlbHNlICAgICAgICAgICAgICAgICAtIGVuYWJsZSBzdGFydC9wb3N0IHNjcmlwdC4KIyMKIyMgVGhlIGVuYWJsaW5nIGFuZCBkaXNhYmxpbmcgb2YgYSBzY3JpcHQgaXMgYmFzZWQgb24gdGhlIGdsdXN0ZXJkJ3MgbG9naWMsCiMjIHRoYXQgaXQgb25seSBydW5zIHRoZSBzY3JpcHRzIHdoaWNoIHN0YXJ0cyBpdHMgbmFtZSB3aXRoICdTJy4gU28sCiMjIEVuYWJsZSAtIHN5bWxpbmsgdGhlIGZpbGUgdG8gJ1MnKi4KIyMgRGlzYWJsZS0gdW5saW5rIHN5bWxpbmsKIyMgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQojIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjCgpPUFRTUEVDPSJ2b2xuYW1lOix2ZXJzaW9uOixnZC13b3JrZGlyOix2b2x1bWUtb3A6IgpQUk9HTkFNRT0iUXVvdGEteGF0dHItaGVhbC1hZGQtYnJpY2stcHJlIgpWT0xfTkFNRT0KR0xVU1RFUkRfV09SS0RJUj0KVk9MVU1FX09QPQpWRVJTSU9OPQpFTkFCTEVEX05BTUU9IlMyOFF1b3RhLXJvb3QteGF0dHItaGVhbC5zaCIKRElTQUJMRURfTkFNRT0iZGlzYWJsZWQtcXVvdGEtcm9vdC14YXR0ci1oZWFsLnNoIgoKZW5hYmxlICgpCnsKICAgICAgICBsbiAtc2YgJERJU0FCTEVEX1NUQVRFICQxOwp9CgojIy0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQojIyBQYXJzZSB0aGUgYXJndW1lbnRzCiMjLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCkFSR1M9JChnZXRvcHQgLWwgJE9QVFNQRUMgIC1uYW1lICRQUk9HTkFNRSAkQCkKZXZhbCBzZXQgLS0gIiRBUkdTIgoKd2hpbGUgdHJ1ZTsKZG8KICAgIGNhc2UgJDEgaW4KICAgICAgICAtLXZvbG5hbWUpCiAgICAgICAgICAgIHNoaWZ0CiAgICAgICAgICAgIFZPTF9OQU1FPSQxCiAgICAgICAgICAgIDs7CiAgICAgICAgLS1nZC13b3JrZGlyKQogICAgICAgICAgICBzaGlmdAogICAgICAgICAgICBHTFVTVEVSRF9XT1JLRElSPSQxCiAgICAgICAgICAgIDs7CiAgICAgICAgLS12b2x1bWUtb3ApCiAgICAgICAgICAgIHNoaWZ0CiAgICAgICAgICAgIFZPTFVNRV9PUD0kMQogICAgICAgICAgICA7OwogICAgICAgIC0tdmVyc2lvbikKICAgICAgICAgICAgc2hpZnQKICAgICAgICAgICAgVkVSU0lPTj0kMQogICAgICAgICAgICA7OwogICAgICAgICopCiAgICAgICAgICAgIHNoaWZ0CiAgICAgICAgICAgIGJyZWFrCiAgICAgICAgICAgIDs7CiAgICBlc2FjCiAgICBzaGlmdApkb25lCiMjLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQoKRElTQUJMRURfU1RBVEU9IiRHTFVTVEVSRF9XT1JLRElSL2hvb2tzLyRWRVJTSU9OL2FkZC1icmljay9wb3N0LyRESVNBQkxFRF9OQU1FIgpFTkFCTEVEX1NUQVRFX1NUQVJUPSIkR0xVU1RFUkRfV09SS0RJUi9ob29rcy8kVkVSU0lPTi9zdGFydC9wb3N0LyRFTkFCTEVEX05BTUUiCkVOQUJMRURfU1RBVEVfQUREX0JSSUNLPSIkR0xVU1RFUkRfV09SS0RJUi9ob29rcy8kVkVSU0lPTi9hZGQtYnJpY2svcG9zdC8kRU5BQkxFRF9OQU1FIjsKCiMjIFdoeSB0byBwcm9jZWVkIGlmIHRoZSByZXF1aXJlZCBzY3JpcHQgaXRzZWxmIGlzIG5vdCBwcmVzZW50PwpscyAkRElTQUJMRURfU1RBVEU7CmlmIFsgMCAtbmUgJD8gXQp0aGVuCiAgICAgICAgZXhpdCAkPzsKZmkKCiMjIElzIHF1b3RhIGVuYWJsZWQ/CkZMQUc9YGNhdCAkR0xVU1RFUkRfV09SS0RJUi92b2xzLyRWT0xfTkFNRS9pbmZvIHwgZ3JlcCAiXmZlYXR1cmVzLnF1b3RhPSIgXAogICAgICB8IGF3ayAtRic9JyAne3ByaW50ICRORn0nYDsKaWYgWyAiJEZMQUciICE9ICJvbiIgXQp0aGVuCiAgICAgICAgZXhpdCAkRVhJVF9TVUNDRVNTOwpmaQoKIyMgSXMgdm9sdW1lIHN0YXJ0ZWQ/CkZMQUc9YGNhdCAkR0xVU1RFUkRfV09SS0RJUi92b2xzLyRWT0xfTkFNRS9pbmZvIHwgZ3JlcCAiXnN0YXR1cz0iIFwKICAgICAgfCBhd2sgLUYnPScgJ3twcmludCAkTkZ9J2A7CmlmIFsgIiRGTEFHIiAhPSAiMSIgXQp0aGVuCiAgICAgICAgZW5hYmxlICRFTkFCTEVEX1NUQVRFX1NUQVJUOwogICAgICAgIGV4aXQgJD8KZmkKCmVuYWJsZSAkRU5BQkxFRF9TVEFURV9BRERfQlJJQ0s7CmV4aXQgJD8K') error: update() takes exactly 6 arguments (5 given)"}, code = -32603 To view the terms under which this email is distributed, please go to:- http://leedsbeckett.ac.uk/disclaimer/email/ To view the terms under which this email is distributed, please go to:- http://leedsbeckett.ac.uk/disclaimer/email/

1 0

user role
by kim.kargaard＠noroff.no 17 Sep '19

17 Sep '19

Hi, We are an Higher Education Institution and have set up oVirt 4.3 as a Virtual Lab Environment, for both on-campus and distance learning students. We have integrated AD and can log in with AD credentials for users. However, I am struggling to find the right user role for normal student users. Basically, they need to be able to do the following: 1. Create a new VM and select from either a template or select an uploaded ISO 2. Start the VM 3. Restart the VM 4. Shutdown the VM 5. Snapshot the VM 6. Connect to the VM using either spice or vnc/novnc 7. Create a disk 8. Connect to the student network (only would be preferable here) as we have management networks that we dont want the students be be able to see or select (if possible) I have tested using the PowerUserRole, but that does not allow me to select VNC/noVNC or spice and does not allow me to install from an ISO, only a template. Any suggestions on role that I can give these users? Thanks. Kind regards Kim

3 2

Re: Does cluster upgrade wait for heal before proceeding to next host?
by Strahil 17 Sep '19

17 Sep '19

As far as I know, The UI allows the administrator to override the gluster status, although I always use that protection. In my opinion gluster health (especially in my setup - replica2 arbiter1) is very important. Best Regards, Strahil NikolovOn Sep 17, 2019 00:47, Jayme <jaymef(a)gmail.com> wrote: > > Strahil, yes this is similar to the approach I have used to upgrade my hci cluster. This thread is in regards to the ui cluster upgrade procedure. It fails to update after the first host is rebooted because the gluster volume is still in a healing state and the attempt to set the second host to maintenance fails because quorum is not met. > > On Mon, Sep 16, 2019 at 1:41 PM Strahil <hunter86_bg(a)yahoo.com> wrote: >> >> I keep reading this chain and I still don't get what/who should wait for the cluster to heal... >> Is there some kind of built-in autopatching feature? >> >> Here is my approach: >> 1. Set global maintenance >> 2. Power off the engine >> 3. Create a gluster snapshot of the engine's volume >> 4. Power on engine manually >> 5. Check engine status >> 6. Upgrade engine >> 7. Upgrade engine's OS >> 8. Reboot engine and check health >> 9. Remove global maintenance >> 10. Set a host into local maintenance (evacuate all VMs) >> 11. Use UI to patch the host (enable autoreboot) >> 12. When host is up - login and check gluster volumes' heal status >> 13. Remove maintenance for the host and repeate for the rest of the cluster. >> >> I realize that for large clusters this approach is tedious and an automatic approach can be scripted. >> >> Best Regards, >> Strahil Nikolov >> >> On Sep 16, 2019 11:02, Kaustav Majumder <kmajumde(a)redhat.com> wrote: >>> >>> Hi Jayme, >>> It would be great if you could raise a bug regarding the same. >>> >>> On Wed, Sep 11, 2019 at 5:05 PM Jayme <jaymef(a)gmail.com> wrote: >>>> >>>> This sounds similar to the issue I hit with the cluster upgrade process in my environment. I have large 2tb ssds and most of my vms are several hundred Gbs in size. The heal process after host reboot can take 5-10 minutes to complete. I may be able to address this with better gluster tuning. >>>> >>>> Either way the upgrade process should be aware of the heal status and wait for it to complete before attempting to move on to the next host. >>>> >>>> >>>> On Wed, Sep 11, 2019 at 3:53 AM Sahina Bose <sabose(a)redhat.com> wrote: >>>>> >>>>> >>>>> >>>>> On Fri, Aug 9, 2019 at 3:41 PM Martin Perina <mperina(a)redhat.com> wrote: >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Aug 8, 2019 at 10:25 AM Sandro Bonazzola <sbonazzo(a)redhat.com> wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> Il giorno mar 6 ago 2019 alle ore 23:17 Jayme <jaymef(a)gmail.com> ha scritto: >>>>>>>> >>>>>>>> I’m aware of the heal process but it’s unclear to me if the update continues to run while the volumes are healing and resumes when they are done. There doesn’t seem to be any indication in the ui (unless I’m mistaken) >>>>>>> >>>>>>> >>>>>>> Adding @Martin Perina , @Sahina Bose and

1 0

Re: Does cluster upgrade wait for heal before proceeding to next host?
by Strahil 16 Sep '19

16 Sep '19

I keep reading this chain and I still don't get what/who should wait for the cluster to heal... Is there some kind of built-in autopatching feature? Here is my approach: 1. Set global maintenance 2. Power off the engine 3. Create a gluster snapshot of the engine's volume 4. Power on engine manually 5. Check engine status 6. Upgrade engine 7. Upgrade engine's OS 8. Reboot engine and check health 9. Remove global maintenance 10. Set a host into local maintenance (evacuate all VMs) 11. Use UI to patch the host (enable autoreboot) 12. When host is up - login and check gluster volumes' heal status 13. Remove maintenance for the host and repeate for the rest of the cluster. I realize that for large clusters this approach is tedious and an automatic approach can be scripted. Best Regards, Strahil NikolovOn Sep 16, 2019 11:02, Kaustav Majumder <kmajumde(a)redhat.com> wrote: > > Hi Jayme, > It would be great if you could raise a bug regarding the same. > > On Wed, Sep 11, 2019 at 5:05 PM Jayme <jaymef(a)gmail.com> wrote: >> >> This sounds similar to the issue I hit with the cluster upgrade process in my environment. I have large 2tb ssds and most of my vms are several hundred Gbs in size. The heal process after host reboot can take 5-10 minutes to complete. I may be able to address this with better gluster tuning. >> >> Either way the upgrade process should be aware of the heal status and wait for it to complete before attempting to move on to the next host. >> >> >> On Wed, Sep 11, 2019 at 3:53 AM Sahina Bose <sabose(a)redhat.com> wrote: >>> >>> >>> >>> On Fri, Aug 9, 2019 at 3:41 PM Martin Perina <mperina(a)redhat.com> wrote: >>>> >>>> >>>> >>>> On Thu, Aug 8, 2019 at 10:25 AM Sandro Bonazzola <sbonazzo(a)redhat.com> wrote: >>>>> >>>>> >>>>> >>>>> Il giorno mar 6 ago 2019 alle ore 23:17 Jayme <jaymef(a)gmail.com> ha scritto: >>>>>> >>>>>> I’m aware of the heal process but it’s unclear to me if the update continues to run while the volumes are healing and resumes when they are done. There doesn’t seem to be any indication in the ui (unless I’m mistaken) >>>>> >>>>> >>>>> Adding @Martin Perina , @Sahina Bose and @Laura Wright on this, hyperconverged deployments using cluster upgrade command would probably need some improvement. >>>> >>>> >>>> The cluster upgrade process continues to the 2nd host after the 1st host becomes Up. If 2nd host then fails to switch to maintenance, we stop the upgrade process to prevent breakage. >>>> Sahina, is gluster healing process status exposed in RESTAPI? If so, does it makes sense to wait for healing to be finished before trying to move next host to maintenance? Or any other ideas how to improve? >>> >>> >>> I need to cross-check this, if we expose the heal count in the gluster bricks. Moving a host to maintenance does check if there are pending heal entries or possibility of quorum loss. And this would prevent the additional hosts to upgrade. >>> +Gobinda Das +Sachidananda URS >>> >>>>> >>>>> >>>>>> >>>>>> >>>>>> On Tue, Aug 6, 2019 at 6:06 PM Robert O'Kane <okane(a)khm.de> wrote: >>>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> Often(?), updates to a hypervisor that also has (provides) a Gluster >>>>>>> brick takes the hypervisor offline (updates often require a reboot). >>>>>>> >>>>>>> This reboot then makes the brick "out of sync" and it has to be resync'd. >>>>>>> >>>>>>> I find it a "feature" than another host that is also part of a gluster >>>>>>> domain can not be updated (rebooted) before all the bricks are updated >>>>>>> in order to guarantee there is not data loss. It is called Quorum, or? >>>>>>> >>>>>>> Always let the heal process end. Then the next update can start. >>>>>>> For me there is ALWAYS a healing time before Gluster is happy again. >>>>>>> >>>>>>> Cheers, >>>>>>> >>>>>>> Robert O'Kane >>>>>>> >>>>>>> >>>>>>> Am 06.08.2019 um 16:38 schrieb Shani Leviim: >>>>>>> > Hi Jayme, >>>>>>> > I can't recall such a healing time. >>>>>>> > Can you please retry and attach the engine & vdsm logs so we'll be smarter? >>>>>>> > >>>>>>> > *Regards, >>>>>>> > * >>>>>>> > *Shani Leviim >>>>>>> > * >>>>>>> > >>>>>>> > >>>>>>> > On Tue, Aug 6, 2019 at 5:24 PM Jayme <jaymef(a)gmail.com >>>>>>> > <mailto:jaymef@gmail.com>> wrote: >>>>>>> > >>>>>>> > I've yet to have cluster upgrade finish updating my three host HCI >>>>>>> > cluster. The most recent try was today moving from oVirt 4.3.3 to >>>>>>> > 4.3.5.5. The first host updates normally, but when it moves on to >>>>>>> > the second host it fails to put it in maintenance and the cluster >>>>>>> > upgrade stops. >>>>>>> > >>>>>>> > I suspect this is due to that fact that after my hosts are updated >>>>>>> > it takes 10 minutes or more for all volumes to sync/heal. I have >>>>>>> > 2Tb SSDs. >>>>>>> > >>>>>>> > Does the cluster upgrade process take heal time in to account before >>>>>>> > attempting to place the next host in maintenance to upgrade it? Or >>>>>>> > is there something else that may be at fault here, or perhaps a >>>>>>> > reason why the heal process takes 10 minutes after reboot to complete? >>>>>>> > _______________________________________________ >>>>>>> > Users mailing list -- users(a)ovirt.org <mailto:users@ovirt.org> >>>>>>> > To unsubscribe send an email to users-leave(a)ovirt.org >>>>>>> > <mailto:users-leave@ovirt.org> >>>>>>> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>> > oVirt Code of Conduct: >>>>>>> > https://www.ovirt.org/community/about/community-guidelines/ >>>>>>> > List Archives: >>>>>>> > https://lists.ovirt.org/archives/list/users@ovirt.org/message/5XM3QB3364ZYI… >>>>>>> > >>>>>>> > >>>>>>> > _______________________________________________ >>>>>>> > Users mailing list -- users(a)ovirt.org >>>>>>> > To unsubscribe send an email to users-leave(a)ovirt.org >>>>>>> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>> > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>>>>>> > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/GBX3L23MWGMTF… >>>>>>> > >>>>>>> >>>>>>> -- >>>>>>> Systems Administrator >>>>>>> Kunsthochschule für Medien Köln >>>>>>> Peter-Welter-Platz 2 >>>>>>> 50676 Köln >>>>>>> _______________________________________________ >>>>>>> Users mailing list -- users(a)ovirt.org >>>>>>> To unsubscribe send an email to users-leave(a)ovirt.org >>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>>>>>> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/OBAHFFFTDOI7L… >>>>>> >>>>>> _______________________________________________ >>>>>> Users mailing list -- users(a)ovirt.org >>>>>> To unsubscribe send an email to users-leave(a)ovirt.org >>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>>>>> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/T27ROHWZPJL47… >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Sandro Bonazzola >>>>> >>>>> MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV >>>>> >>>>> Red Hat EMEA >>>>> >>>>> sbonazzo(a)redhat.com >>>>> >>>>> Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. >>>> >>>> >>>> >>>> -- >>>> Martin Perina >>>> Manager, Software Engineering >>>> Red Hat Czech s.r.o. >> >> _______________________________________________ >> Users mailing list -- users(a)ovirt.org >> To unsubscribe send an email to users-leave(a)ovirt.org >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/I4KLDBPYBCQEM… > > > > -- > > Thanks, > > Kaustav Majumder

2 1

Cannot Activate iSCSI Storage Domain After Errors
by Clint Boggio 16 Sep '19

16 Sep '19

Good Day To all; oVirt 4.3.4.3-1.el7 6 Dell R710 oVirt Nodes 10 gig Infiniband iSCSI Standalone Engine After an error during scheduled automated VM backkup, the iSCSi datastore that the VMs beinng backed up lived on suddenly went inactive and refuse to activate. The backup job did not finish as the source datastore went inactive but the task did clear from the console after i restarted the ovirt-engine service. I have tried: 1. Restart the ovirt-engine service 2. Restarted the engine physical server. 3. Tried to migrate the VM disks to a different datastore (failure) The virtual machines that are currently living on the inactive datastore are still running and accessible. Trying to reboot one of those VM as a test fails as the VM will not come up due to the system believing that the datastore is inactive. From engin.log: 2019-09-15 17:09:08,787-05 INFO [org.ovirt.engine.core.bll.storage.disk.RemoveDiskCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-8) [fbdd9b7] Removing 'Wildix4-PBX_snapshot_metadata' disk 2019-09-15 17:09:13,399-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-commandCoordinator-Thread-8) [fbdd9b7] EVENT_ID: IRS_BROKER_COMMAND_FAILURE(10,803), VDSM command DeleteImageGroupVDS failed: General Storage Exception: ('5 [\' Logical volume d1c5f4da-9024-4599-9135-4dabafe86cf8/c5f64501-142b-4a38-9b9c-faf283a64b06 changed.\'] [\' WARNING: This metadata update is NOT backed up.\', \' WARNING: Combining activation change with other commands is not advised.\', \' /dev/mapper/36001405f448f60ecea7479f9d920ac51: Checksum error at offset 19198423553536\', " Couldn\'t read volume group metadata from /dev/mapper/36001405f448f60ecea7479f9d920ac51.", \' Metadata location on /dev/mapper/36001405f448f60ecea7479f9d920ac51 at 19198423553536 has invalid summary for VG.\', \' Failed to read metadata summary from /dev/mapper/36001405f448f60ecea7479f9d920ac51\', \' Failed to scan VG from /dev/mapper/36001405f448f60ecea7479f9d920ac51\', \' Volume group "d1c5f4da-9024-4599-9135-4dabafe86cf8" not found\', \' Cannot process volume group d1c5f4da-9024-4599-9135-4dabafe86cf8\']\nd1c5f4da-9024-4599-9135-4dabafe86cf8/{\'c5f64501-142b-4a38-9b9c-faf283a64b06\': ImgsPar(imgs=[\'aadbfed3-af54-4400-af79-65d09b74c687\'], parent=\'00000000-0000-0000-0000-000000000000\')}',) 2019-09-15 17:09:13,399-05 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-8) [fbdd9b7] IrsBroker::Failed::DeleteImageGroupVDS: IRSGenericException: IRSErrorException: Failed to DeleteImageGroupVDS, error = General Storage Exception: ('5 [\' Logical volume d1c5f4da-9024-4599-9135-4dabafe86cf8/c5f64501-142b-4a38-9b9c-faf283a64b06 changed.\'] [\' WARNING: This metadata update is NOT backed up.\', \' WARNING: Combining activation change with other commands is not advised.\', \' /dev/mapper/36001405f448f60ecea7479f9d920ac51: Checksum error at offset 19198423553536\', " Couldn\'t read volume group metadata from /dev/mapper/36001405f448f60ecea7479f9d920ac51.", \' Metadata location on /dev/mapper/36001405f448f60ecea7479f9d920ac51 at 19198423553536 has invalid summary for VG.\', \' Failed to read metadata summary from /dev/mapper/36001405f448f60ecea7479f9d920ac51\', \' Failed to scan VG from /dev/mapper/36001405f448f60ecea7479f9d920ac51\', \' Volume group "d1c5f4da-9024-4599-9135-4dabafe86cf8" not found\', \' Cannot process volume group d1c5f4da-9024-4599-9135-4dabafe86cf8\']\nd1c5f4da-9024-4599-9135-4dabafe86cf8/{\'c5f64501-142b-4a38-9b9c-faf283a64b06\': ImgsPar(imgs=[\'aadbfed3-af54-4400-af79-65d09b74c687\'], parent=\'00000000-0000-0000-0000-000000000000\')}',), code = 200 2019-09-15 17:09:13,463-05 ERROR [org.ovirt.engine.core.bll.storage.disk.image.RemoveImageCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-8) [fbdd9b7] Command 'org.ovirt.engine.core.bll.storage.disk.image.RemoveImageCommand' failed: EngineException: org.ovirt.engine.core.vdsbroker.irsbroker.IRSErrorException: IRSGenericException: IRSErrorException: Failed to DeleteImageGroupVDS, error = General Storage Exception: ('5 [\' Logical volume d1c5f4da-9024-4599-9135-4dabafe86cf8/c5f64501-142b-4a38-9b9c-faf283a64b06 changed.\'] [\' WARNING: This metadata update is NOT backed up.\', \' WARNING: Combining activation change with other commands is not advised.\', \' /dev/mapper/36001405f448f60ecea7479f9d920ac51: Checksum error at offset 19198423553536\', " Couldn\'t read volume group metadata from /dev/mapper/36001405f448f60ecea7479f9d920ac51.", \' Metadata location on /dev/mapper/36001405f448f60ecea7479f9d920ac51 at 19198423553536 has invalid summary for VG.\', \' Failed to read metadata summary from /dev/mapper/36001405f448f60ecea7479f9d920ac51\', \' Failed to scan VG from /dev/mapper/36001405f448f60ecea7479f9d920ac51\', \' Volume group "d1c5f4da-9024-4599-9135-4dabafe86cf8" not found\', \' Cannot process volume group d1c5f4da-9024-4599-9135-4dabafe86cf8\']\nd1c5f4da-9024-4599-9135-4dabafe86cf8/{\'c5f64501-142b-4a38-9b9c-faf283a64b06\': ImgsPar(imgs=[\'aadbfed3-af54-4400-af79-65d09b74c687\'], parent=\'00000000-0000-0000-0000-000000000000\')}',), code = 200 (Failed with error StorageException and code 200) <metadata xmlns:ns0="http://ovirt.org/vm/tune/1.0" xmlns:ovirt-vm="http://ovirt.org/vm/1.0"> Any help or input would be greaatly appreciated.

1 0

Re: Virt-Viewer issue
by Staniforth, Paul 16 Sep '19

16 Sep '19

Hello, you should use remote-viewer, it's part of the virt-viewer package. Paul Staniforth School of Built Environment Engineering and Computing. Leeds Beckett University Networked Systems Analyst, Research and Engineering Support. tel: +44 (0)113 28123754 email: p.staniforth(a)leedsbeckett.ac.uk ________________________________ From: Lozada, Agustin T <Agustin.Lozada(a)centerpointenergy.com> Sent: 13 September 2019 15:38 To: users(a)ovirt.org Subject: [ovirt-users] Virt-Viewer issue Im building my first VM in ovirt with gluster and Im getting this issue when I'm opening a console session using virt-viewer. Does anyone has encountered this issue? [cid:image001.png@01D56A15.FECFDAF0] To view the terms under which this email is distributed, please go to:- http://leedsbeckett.ac.uk/disclaimer/email/

1 0

Does cluster upgrade wait for heal before proceeding to next host?
by Jayme 16 Sep '19

16 Sep '19

I've yet to have cluster upgrade finish updating my three host HCI cluster. The most recent try was today moving from oVirt 4.3.3 to 4.3.5.5. The first host updates normally, but when it moves on to the second host it fails to put it in maintenance and the cluster upgrade stops. I suspect this is due to that fact that after my hosts are updated it takes 10 minutes or more for all volumes to sync/heal. I have 2Tb SSDs. Does the cluster upgrade process take heal time in to account before attempting to place the next host in maintenance to upgrade it? Or is there something else that may be at fault here, or perhaps a reason why the heal process takes 10 minutes after reboot to complete?

8 9

CentOS 6 in oVirt
by Dominik Holler 16 Sep '19

16 Sep '19

Hello, are the official CentOS 6 cloud images known to work in oVirt? I tried, but they seem not to work in oVirt [1], but on plain libvirt. Are there any pitfalls known during using them in oVirt? Thanks Dominik [1] https://gerrit.ovirt.org/#/c/101117/

2 2

VMs stuck at Migrating status in Dashboard- Trying to cancel migration will yield "cannot cancel migration for non-migrating vm"
by paul.christian.suba＠cevalogistics.com 16 Sep '19

16 Sep '19

Saw this problem initially on one host. Host was stuck in entering maintenance mode since it could not migrate a VM from itself. Manually shutdown that VM and also restarted the host. After restart, some VMs are stuck in a migrating status while trying to "migrate" to the rebooted host. oVirt Node 4.3.1

2 2

How do I make delta clones ? OVM (aka Xen) vs Ovirt
by Tim Tuck 15 Sep '19

15 Sep '19

Hi all, I'm new to Ovirt coming from a OVM ( aka Xen ) environment and I'm trying to work out how to do similar things. In my OVM environment I have VMs that are 100GB in size. These VMs are templates that are used for running customer workshops so we create and destroy them quite often. In OVM I can clone 10 x 100GB VM's in about 1 minute, the cloning process just creates a "delta" VM rather than copying the entire 100GB. Those VM's can then be run and the on-disk size of the delta VM grows as it runs and deviates from the master. The same goes for templates, creating a 100GB VM from a template takes only a few seconds. Creating a template from a VM only takes a few seconds. My problem is that I cant seem to do any of this using Ovirt. it appears that there is no way to tell the cloning process " I want 10" like I can in OVM - am I missing something here? In my tests I find that even if a "snapshot" is made, the entire VM is copied rather than creating a delta. Creating a VM from a template is no different in that it copies the entire VM. So... Is it possible for Ovirt to create multiple clones on request ? Can Ovirt do "delta" copies rather than copy the entire VM ? If Ovirt can do these things, how do I set it up to to do that ? Any help appreciated. thanks Tim --- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

2 1

Virt-Viewer issue
by Lozada, Agustin T 13 Sep '19

13 Sep '19

Im building my first VM in ovirt with gluster and Im getting this issue when I'm opening a console session using virt-viewer. Does anyone has encountered this issue? [cid:image001.png@01D56A15.FECFDAF0]

1 0

Re: VMs stuck at Migrating status in Dashboard- Trying to cancel migration will yield "cannot cancel migration for non-migrating vm"
by Strahil 12 Sep '19

12 Sep '19

Have you marked your host as 'recently rebooted' (or whatever was the name in the UI)? Best Regards, Strahil NikolovOn Sep 12, 2019 18:15, paul.christian.suba(a)cevalogistics.com wrote: > > Saw this problem initially on one host. Host was stuck in entering maintenance mode since it could not migrate a VM from itself. > Manually shutdown that VM and also restarted the host. > After restart, some VMs are stuck in a migrating status while trying to "migrate" to the rebooted host. > > oVirt Node 4.3.1 > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/QFMNWH63FQIVZ…

1 0

oVirt conference 2019 - Rome - October 4th
by Sandro Bonazzola 12 Sep '19

12 Sep '19

The oVirt Italian community <https://www.facebook.com/groups/ovirt.italia/> and Extraordy <https://www.extraordy.com/> in collaboration with Red Hat <https://www.redhat.com>, Oracle <https://www.oracle.com>, and Storware <https://storware.eu/> are pleased to invite you at oVirt Conference 2019 in Rome on October 4th 2019! Tickets are available at https://www.eventbrite.it/e/biglietti-ovirt-conference-2019-innovate-the-da… Thanks -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/>*Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

2 1

Re: VMs inaccessible from Ubuntu/Debian-based OS
by Strahil 12 Sep '19

12 Sep '19

Hi, I have just tested ovirt VNC console (.vv file) with remote-viewer and it works. Ensure that you have acees to hosts' ports 5900-5999 , as this is the VNC port. Otherwise you can configure noVNC, but it requires port 6100 (based on memory). Best Regards, Strahil NikolovOn Aug 30, 2019 20:22, Souvalioti Maria <souvaliotimaria(a)mail.com> wrote: > > Hello Strahil and thank you for your reply. > > Virt-viewer is installed but mozilla is trying to open the console.vv by remote-viewer. In my understanding virt-viewer and remote-viewer are somewhat the same? (Remote-viewer usin virt-viewer) Should I access the VM's console from the ternimal? I have tried changing the VM's graphics type from spice to spice+vnc but still nothing. All I get is a blank window and a message stating that it cannot access the graphic server. > > Thanks again for your help. > > Maria Souvalioti > > > Sent from BlueMail > On 30 Aug 2019, at 7:48 pm, Strahil <hunter86_bg(a)yahoo.com> wrote: >> >> You need virt-viewer installed on your ubuntu or change console type to spice. >> >> >> Best Regards, >> Strahil NikolovOn Aug 30, 2019 15:35, souvaliotimaria(a)mail.com wrote: >>> >>> >>> Hello all, >>> >>> I'm having an issue the past couple of days. I have tried anything I could find to solve this but with no success. >>> >>> I have a three node ovirt installation, glustered, hyperconverged and there i have a few VMs. The installation is for experimental reasons before we merge it in our DC. Anyway, though I can connect to the VMs' console from Fedora and Windows, I can't from Ubuntu. >>> >>> I have tried installing the browser-spice-plugin that's the corresponding package to spice-xpi and I got no results. >>> >>> I purged the browser-spice-plugin and then I installed the spice-client package, downloaded the spice-xpi from Fedora (FC19) (as instructed in https://www.ovirt.org/develop/infra/testing/spice.html) copied the libnsISpicec.so to /usr/lib/mozilla/plugins and made sure that xserver-xorg-video-qxl and spice-vdagent were installed and in their latest version, and still nothing. >>> >>> No matter what I have tried, I can't gain access to the console. No matter the browser I use, the message I get is "Unable to connect to the graphic server /tmp/mozilla_msouval0/console-1.vv". >>> >>> I have checked the logs and couldn't find anything useful, maybe I'm not checking the right logs? >>> I run tcpdump on both the node the VM is being hosted and the Ubuntu machine I'm using and though on the UBuntu I capture a few packets (6) on both sides but on the Ubuntu side there were 15 packets received by the filter. >>> >>> Could you please guide towards the right way to solve this? Could this be an ntp problem? >>> >>> ovirt node version: 4.2.3.1 >>> Workstation: Jessie >>> >>> (I thought maybe Jessie was too old, and I run the same steps from Ubuntu 14.04 and Ubuntu 16.04, but the problem still remains) >>> >>> Thank you in advance for any help. >>> Maria >>> ________________________________ >>> >>> Users mailing list -- users(a)ovirt.org >>> To unsubscribe send an email to users-leave(a)ovirt.org >>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/4VMMYEJNSHGUH…

3 2

Health Warning: Cinnamon toxic when adding EL7 nodes as oVirt hosts
by thomas＠hoberg.net 12 Sep '19

12 Sep '19

Of course a desktop on a virtualization host isn't really recommended, but actually, managing highly or even high-available supports services as VMs on a set of fat workstations under the control of oVirt can be rather useful... And I prefer Cinnamon so much over the other desktops, it gets installed almost first after setup... Which turns out to be a mistake with oVirt. Anyhow, to make a long journey of hair-pulling root-cause searching short: Installing Cinnamon from EPEL has Otopi not find rpmUtils for one reason or another, even if I can see it on both sides, the engine and the target host. Remove Cinnamon, it works again and you can re-install Cinnamon after. I didn't test re-deploys, but I'd expect them to fail. Don't know if this should be a bug report, since Cinnamon very unfortunately doesn't seem official just yet. This is what you may expect to see in the engine logs: 2019-08-22 09:49:24,152+0200 DEBUG otopi.context context._executeMethod:145 method exception Traceback (most recent call last): File "/tmp/ovirt-BAJG2FYLMu/pythonlib/otopi/context.py", line 132, in _executeMethod method['method']() File "/tmp/ovirt-BAJG2FYLMu/otopi-plugins/ovirt-host-deploy/kdump/packages.py", line 216, in _customization self._kexec_tools_version_supported() File "/tmp/ovirt-BAJG2FYLMu/otopi-plugins/ovirt-host-deploy/kdump/packages.py", line 143, in _kexec_tools_version_supported from rpmUtils.miscutils import compareEVR ModuleNotFoundError: No module named 'rpmUtils' 2019-08-22 09:49:24,152+0200 ERROR otopi.context context._executeMethod:154 Failed to execute stage 'Environment customization': No module named 'rpmUtils'

2 10

email
by Mustafa Taha 12 Sep '19

12 Sep '19

mustafa.taha(a)exalt.ps

1 0

Deploying the Self-Hosted Engine Loop
by mustafa.taha.mu95＠gmail.com 11 Sep '19

11 Sep '19

Hi i am trying to deploy self-hot with latest ovirt 4.3 on Centos 7.6.1810 but deploying -first stage of deploy- stick on following logs: [ INFO ] TASK [ovirt.hosted_engine_setup : Set new IPv4 subnet prefix] [ INFO ] skipping: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Search again with another prefix] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Define 3rd chunk] [ INFO ] skipping: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Set 3rd chunk] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Get ip route] [ INFO ] changed: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fail if can't find an available subnet] [ INFO ] skipping: [localhost] can anybody help ?

1 0

Locked Snapshot
by Peter Smith 11 Sep '19

11 Sep '19

Hi, I have been trying to get a backup solution working for my VM's, and have somehow managed to break the snapshot on one of my VM's. I'm still fairly new to oVirt, we're migrating from VMware. I have had this cluster running since May. The cluster: We are running oVirt on CentOS 7, with Hosted Engine. oVirt 4.3.5.4-1.el7 (according to the Help->About in the Admin Console) Hardware is 5 x Dell M620 Blades (300Gb internal storage for host OS) with ISCSI storage for the VM's (Dell M3660i), 10GB Network throughout. The VM in question is allocated 50Gb of storage and there is 461GB free in the storage domain. So I've been playing with bareos 18.2.5 with the ovirt plugin found here https://github.com/eurotux/bareos-contrib/tree/master/fd-plugins/ovirt-plug… I got bareos hooked up to ovirt. It created a snapshot, downloaded it, then at the end it failed to remove the snapshot. I though it may have been because I was looking round bareos while it was doing the backup. Went into oVirt deleted the snapshot no issues. Re ran the backup job in bareos, leaving bareos alone this time. The backup job failed again being unable to remove the snapshot. So I figured I'd go in and delete it, but I get an error saying "Cannot remove Snapshot: The following disks are locked: VM-NAME_vmdisk1. Please try again in a few minutes.". In the Admin console if I expand the disks on the snapshot it shows that the image is locked. And before you ask the bareos user is setup as a SuperUser. If I go to the Events from the menu on the left of the admin console I can see that (Sorry Names/IP's changed to protect the guilty): Sep 10, 2019, 10:42:39 AM Image Download with disk VM-NAME_vmdisk1 succeeded. Sep 10, 2019, 10:42:38 AM VDSM hyper2 command TeardownImageVDS failed: Cannot deactivate Logical Volume: ('General Storage Exception: ("5 [] [\' Logical volume 46a9dfd4-4871-46b2-a740-8fe430c70b4b/bbe315b9-e797-4cf8-b18b-6afeb33589cc in use.\', \' Logical volume 46a9dfd4-4871-46b2-a740-8fe430c70b4b/d3dfa2c2-45fa-4081-98c5-890dd7a5990a in use.\']\\n46a9dfd4-4871-46b2-a740-8fe430c70b4b/[\'bbe315b9-e797-4cf8-b18b-6afeb33589cc\', \'d3dfa2c2-45fa-4081-98c5-890dd7a5990a\']",)',) Sep 10, 2019, 10:22:09 AM Image Download with disk VM-NAME_vmdisk1 was initiated by bareos@internal-authz. Sep 10, 2019, 10:21:55 AM Snapshot 'VM-NAME-backup-1f952494-8245-479e-8d1d-9503d6fe8f9f' creation for VM 'VM-NAME' has been completed. Sep 10, 2019, 10:21:36 AM Snapshot 'VM-NAME-backup-1f952494-8245-479e-8d1d-9503d6fe8f9f' creation for VM 'VM-NAME' was initiated by bareos@internal-authz. Sep 10, 2019, 10:21:36 AM Backup of virtual machine 'VM-NAME using snapshot 'VM-NAME-backup-1f952494-8245-479e-8d1d-9503d6fe8f9f' is starting. Sep 10, 2019, 10:21:36 AM User bareos@internal-authz connecting from '10.*******' using session '****************' logged in. To me it looks like bareos tried to delete the snapshot whilst it was still downloading it, which failed as expected. Does anyone know how to fix/recover from this? If you need log files, please could you tell me which ones? Regards, Peter Smith

3 3

How to change Master Data Storage Domain
by Mark Steele 11 Sep '19

11 Sep '19

Good morning, I have a Storage Domain that I would like to retire and move that role to a new storage domain. Both Domains exist on my current Data Center and I have moved all disks from the existing Data (Master) domain to the new Data Domain. The only thing that is still associated with the Data (Master) domain are two OVF_STORE items: Alias Virtual Size Actual Size Allocation Policy Storage Domain Storage Type Creation Date Attached To Alignment Status Description OVF_STORE < 1 GB < 1 GB Preallocated phl-datastore NFS 2014-Nov-14, 20:00 Unknown OK OVF_STORE OVF_STORE < 1 GB < 1 GB Preallocated phl-datastore NFS 2014-Nov-14, 20:00 Unknown OK OVF_STORE What is the procedure for moving / removing these items and 'promoting' the other Data Domain to Master? Our current version is: oVirt Engine Version: 3.5.0.1-1.el6 (it's old but reliable) Best regards, *** *Mark Steele* CIO / VP Technical Operations | TelVue Corporation TelVue - We Share Your Vision 16000 Horizon Way, Suite 100 | Mt. Laurel, NJ 08054 800.885.8886 x128 | msteele(a)telvue.com | http://www.telvue.com twitter: http://twitter.com/telvue | facebook: https://www.facebook.com/telvue

3 4

ovirt with glusterfs data domain - very slow writing speed on Windows server virtual machine
by lyubomir.grancharov＠bottleshipvfx.com 11 Sep '19

11 Sep '19

Hi folks, I have been experimenting with oVirt cluster based on glusterfs for the past few days. (first-timer). The cluster is up and running and it consists of 4 nodes and has 4 replicas. When I try to deploy Windows Server VM I encounter the following issue: The disk of the VM has ok reading speed ( close to bare metal) but the writing speed is very slow. ( about 10 times slower than it is supposed to be). Can anyone give me any suggestion, please? Thanks in advance! Here are the settings of the glusterfs volume: Volume Name: bottle-volume Type: Replicate Volume ID: 869b8d1e-1266-4820-8dcd-4fea92346b90 Status: Started Snapshot Count: 0 Number of Bricks: 1 x 4 = 4 Transport-type: tcp Bricks: Brick1: cnode01.bottleship.local:/gluster_bricks/brick-cnode01/brick-cnode01 Brick2: cnode02.bottleship.local:/gluster_bricks/brick-cnode02/brick-cnode02 Brick3: cnode03.bottleship.local:/gluster_bricks/brick-cnode03/brick-cnode03 Brick4: cnode04.bottleship.local:/gluster_bricks/brick-cnode04/brick-cnode04 Options Reconfigured: network.ping-timeout: 30 cluster.granular-entry-heal: enable performance.strict-o-direct: on storage.owner-gid: 36 storage.owner-uid: 36 server.event-threads: 4 client.event-threads: 4 cluster.choose-local: off features.shard: on cluster.shd-wait-qlength: 10000 cluster.shd-max-threads: 8 cluster.locking-scheme: granular cluster.data-self-heal-algorithm: full cluster.server-quorum-type: server cluster.quorum-type: auto cluster.eager-lock: enable network.remote-dio: off performance.low-prio-threads: 32 performance.io-cache: off performance.read-ahead: off performance.quick-read: off auth.allow: * user.cifs: off transport.address-family: inet nfs.disable: on performance.client-io-threads: off server.allow-insecure: on Please let me know if you need any more configuration info or hardware specs. best, Lyubo

2 1

Procedure to replace out out of three hyperconverged nodes
by thomas＠hoberg.net 11 Sep '19

11 Sep '19

On my silent Atom based three node Hyperconverged journey I hit upon a snag: Evidently they are too slow for Ansible. The Gluster storage part went all great and perfect on fresh oVirt node images that I had configured to leave an empty partition instead of the standard /dev/sdb, but the HostedEngine setup part would then fail without any log-visible error while the transient VM HostedEngineLocal was supposed to be launched and the Wizard would just show "deployment failed" and go ahead and delete the VM. I then moved the SSD to a more powerful Xeon-D 1541 CPU and after some fiddling with the network (I miss good old eth0!), this also failed the deployment, but also failed to delete the temporary VM image, because that actually turned out to be running: I could even connect to its console and investigate the logs for any clues as to what might have gone wrong (nothing visible): Evidently Ansible was running out of patience just a tiny bit too early. And then I kicked it into high-gear with an i7-7700K again using the same SSD with a working three node Gluster all in sync, which still took what felt like an hour to creep through every step, but got it done, primary node on i7, secondary nodes on Atoms, with full migration capabilities etc. I then had to do some fiddling, because the HostedEngine had configued the Cluster CPU architecture to Skylake-Spectre, but after that I migrated it to an Atom node and was now ready to move the primary to the intended Atom hardware target. But at that point the overlay network has already been configured and evidently it's tied to the device name of the 10Gbit NIC on the i7 workstation and I haven't been able to make it work with the Atom. The Gluster runs fine, but the CPU node is reported "non-operational" and re-installation fails, because the ovirtmgmt network isn't properly configured. That specific issue may be seem way out of what oVirt should support, yet a HA-embedded edge platform may very well see nodes having to be replaced or renewed with as little interruption or downtime as possible, which is why I am asking the larger question: How can you replace a) a failed "burned" node or b) upgrade nodes while maintaining fault tolerance? The distinction in b) would be that it's a planned maneuver during normal operations without downtime. I'd want to do it pretty much like I have been playing with compute nodes, creating new ones, pushing VMs on them, pushing them out to other hosts, removing and replacing them seamlessly... Except that the Gluster nodes are special and much harder to replace, than a pure Gluster storage brick... from what I see I welcome any help - for fixing the network config in my limiping Atom 1:3 cluster - eliminating the need to fiddle with an i7 because of Ansible timing - ensuring long-term operability of a software defined datacenter with changing hardware

2 2

gluster
by mailing-ovirt＠qic.ca 11 Sep '19

11 Sep '19

Hi, I see options to deploy ovirt with gluster during the initial rollout, however I can't seem to find information as to how I can add it following a non gluster initial setup: GlusterFS Version: [N/A] Thanks Simon

4 5

VM crushed and cant shutdown successfully,I shoud to reboot the ovirt-node machine
by zhouhao＠vip.friendtimes.net 10 Sep '19

10 Sep '19

VM crushed and cant shutdown successfully,It always on the way to shutdown，even if I wait it for 10 hours；Everytimes I shoud to reboot the ovirt-node machine This comes up 4 times, It appes both on the ovirt-noe 4.3 and 4.2; This bug appers when my VM is running at high load（seems memory and disk's w/r） My storage domain is local on host; When the bug appers,it could cost my node's io,looks like the picture below；But sometimes it will not The vm's threading on the ovirt-node, the IO ratio is 100% Everytime the vm's process will change to defunct I can not kill it,I had to shutdown the ovirt-node on the engine website ,the vm's status always in the way to shutdown ,even if I wait for it for hours; It either failed or is shutting down and the "power off" cant shutdown the vm too. the vm's error below The ovirt-node's error below ---------------------------------------------------------------- The other infomation about my ovirt-node node-version： node hardware zhouhao(a)vip.friendtimes.net

1 0

Python to retrive the VM-Config from the snapshot
by Sven Achtelik 10 Sep '19

10 Sep '19

Hi All, I'm trying to build a custom solution for Backups using the python. Everything works and I can get my disk copies. The last thing is getting the VM config and I can't find any hint on how to get this done with python. Is there a way to actually get this information out easily ? Somehting like * Grab the wanted vm configuration from the needed snapshot - it'll be under initialization/configuration/data URL = SERVER:PORT/api/vms/VM_ID/snapshots/ID Method = GET (with All-Content:true header) I also have to mention that I'm not a profession programmer and work with what I can find as documentation or code examples. http://ovirt.github.io/ovirt-engine-sdk/4.1/ https://github.com/oVirt/ovirt-engine-sdk/tree/master/sdk/examples If some experienced python programmer could give a hint where to look at that would be great. Thank you, Sven

2 1

Re: Python to retrive the VM-Config from the snapshot
by Strahil 10 Sep '19

10 Sep '19

If the system is running you can use virsh dumpxml. Best Regards, Strahil NikolovOn Sep 10, 2019 16:12, Sven Achtelik <Sven.Achtelik(a)eps.aero> wrote: > > Hi All, > > > > I’m trying to build a custom solution for Backups using the python. Everything works and I can get my disk copies. The last thing is getting the VM config and I can’t find any hint on how to get this done with python. Is there a way to actually get this information out easily ? Somehting like > > Grab the wanted vm configuration from the needed snapshot - it’ll be under initialization/configuration/data > > URL = SERVER:PORT/api/vms/VM_ID/snapshots/ID Method = GET (with All-Content:true header) > > I also have to mention that I’m not a profession programmer and work with what I can find as documentation or code examples. > > http://ovirt.github.io/ovirt-engine-sdk/4.1/ > > https://github.com/oVirt/ovirt-engine-sdk/tree/master/sdk/examples > > If some experienced python programmer could give a hint where to look at that would be great. > > Thank you, > > Sven

1 0

nfs
by mailing-ovirt＠qic.ca 10 Sep '19

10 Sep '19

I`m trying to mount a nfs share. if I manually mount it from ssh, I can access it without issues. However when I do it from the web config, it keeps failing: Not sure how to solve that. Thanks Simon 2019-09-09 09:08:47,601-0400 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:47,610-0400 INFO (jsonrpc/3) [vdsm.api] START repoStats(domains=[u'4f4828c2-8f78-4566-a69d-a37cd73446d4']) from=::1,42394, task_id=6c1eeef7-ff8c-4d0d-be85-ced03aae4bc2 (api:48) 2019-09-09 09:08:47,610-0400 INFO (jsonrpc/3) [vdsm.api] FINISH repoStats return={u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.00101971', 'lastCheck': '0.4', 'valid': True}} from=::1,42394, task_id=6c1eeef7-ff8c-4d0d-be85-ced03aae4bc2 (api:54) 2019-09-09 09:08:47,611-0400 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:47,839-0400 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:47,846-0400 INFO (jsonrpc/2) [api.host] START getCapabilities() from=::1,42394 (api:48) 2019-09-09 09:08:48,149-0400 INFO (jsonrpc/2) [root] managedvolume not supported: Managed Volume Not Supported. Missing package os-brick.: ('Cannot import os_brick',) (caps:152) 2019-09-09 09:08:49,212-0400 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.01 seconds (__init__:312) 2019-09-09 09:08:49,263-0400 INFO (jsonrpc/2) [root] /usr/libexec/vdsm/hooks/after_get_caps/50_openstacknet: rc=0 err= (hooks:114) 2019-09-09 09:08:49,497-0400 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:49,657-0400 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:49,665-0400 INFO (jsonrpc/7) [vdsm.api] START repoStats(domains=[u'4f4828c2-8f78-4566-a69d-a37cd73446d4']) from=::1,42394, task_id=14b596bf-bc1c-412b-b1fc-8d53f17d88f7 (api:48) 2019-09-09 09:08:49,666-0400 INFO (jsonrpc/7) [vdsm.api] FINISH repoStats return={u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.00101971', 'lastCheck': '2.4', 'valid': True}} from=::1,42394, task_id=14b596bf-bc1c-412b-b1fc-8d53f17d88f7 (api:54) 2019-09-09 09:08:49,667-0400 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:49,800-0400 INFO (jsonrpc/2) [root] /usr/libexec/vdsm/hooks/after_get_caps/openstacknet_utils.py: rc=0 err= (hooks:114) 2019-09-09 09:08:50,464-0400 INFO (jsonrpc/2) [root] /usr/libexec/vdsm/hooks/after_get_caps/ovirt_provider_ovn_hook: rc=0 err= (hooks:114) 2019-09-09 09:08:50,467-0400 INFO (jsonrpc/2) [api.host] FINISH getCapabilities return={'status': {'message': 'Done', 'code': 0}, 'info': {u'HBAInventory': {u'iSCSI': [{u'InitiatorName': u'iqn.1994-05.com.redhat:d8c85fc0ab85'}], u'FC': []}, u'packages2': {u'kernel': {u'release': u'957.27.2.el7.x86_64', u'version': u'3.10.0'}, u'spice-server': {u'release': u'6.el7_6.1', u'version': u'0.14.0'}, u'librbd1': {u'release': u'4.el7', u'version': u'10.2.5'}, u'vdsm': {u'release': u'1.el7', u'version': u'4.30.24'}, u'qemu-kvm': {u'release': u'18.el7_6.7.1', u'version': u'2.12.0'}, u'openvswitch': {u'release': u'4.el7', u'version': u'2.11.0'}, u'libvirt': {u'release': u'10.el7_6.12', u'version': u'4.5.0'}, u'ovirt-hosted-engine-ha': {u'release': u'1.el7', u'version': u'2.3.3'}, u'qemu-img': {u'release': u'18.el7_6.7.1', u'version': u'2.12.0'}, u'mom': {u'release': u'1.el7.centos', u'version': u'0.5.12'}, u'glusterfs-cli': {u'release': u'1.el7', u'version': u'6.5'}}, u'numaNodeDistance': {u'1': [20, 10], u'0': [10, 20]}, u'cpuModel': u'Intel(R) Xeon(R) CPU E5-2640 v2 @ 2.00GHz', u'nestedVirtualization': False, u'liveMerge': u'true', u'hooks': {u'before_vm_start': {u'50_hostedengine': {u'md5': u'95c810cdcfe4195302a59574a5148289'}, u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'after_network_setup': {u'30_ethtool_options': {u'md5': u'ce1fbad7aa0389e3b06231219140bf0d'}}, u'after_vm_destroy': {u'delete_vhostuserclient_hook': {u'md5': u'c2f279cc9483a3f842f6c29df13994c1'}, u'50_vhostmd': {u'md5': u'bdf4802c0521cf1bae08f2b90a9559cf'}}, u'after_vm_start': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'ea0a5a715da8c1badbcda28e8b8fa00e'}}, u'after_device_migrate_destination': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'6226fbc4d1602994828a3904fc1b875d'}}, u'before_device_migrate_destination': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}}, u'after_device_create': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'6226fbc4d1602994828a3904fc1b875d'}}, u'after_nic_hotplug': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'6226fbc4d1602994828a3904fc1b875d'}}, u'before_vm_migrate_destination': {u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'before_vm_dehibernate': {u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'before_nic_hotplug': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}, u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'ec8d5d7ba063a109f749cd63c7a2b58d'}, u'20_ovirt_provider_ovn_vhostuser_hook': {u'md5': u'a8af653b7386c138b2e6e9738bd6b62c'}, u'10_ovirt_provider_ovn_hook': {u'md5': u'73822988042847bab1ea832a6b9fa837'}}, u'after_get_caps': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'5c3a9ab6e06e039bdd220c0216e45809'}, u'ovirt_provider_ovn_hook': {u'md5': u'1a6ce22288fc2466fcc339da252ef624'}}, u'before_network_setup': {u'50_fcoe': {u'md5': u'28c352339c8beef1e1b05c67d106d062'}}, u'before_device_create': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}, u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'ec8d5d7ba063a109f749cd63c7a2b58d'}, u'20_ovirt_provider_ovn_vhostuser_hook': {u'md5': u'a8af653b7386c138b2e6e9738bd6b62c'}, u'10_ovirt_provider_ovn_hook': {u'md5': u'73822988042847bab1ea832a6b9fa837'}}}, u'supportsIPv6': True, u'realtimeKernel': False, u'vmTypes': [u'kvm'], u'liveSnapshot': u'true', u'cpuThreads': u'32', u'kdumpStatus': 0, u'deferred_preallocation': True, u'networks': {u'ovirtmgmt': {u'iface': u'ovirtmgmt', u'ipv6autoconf': True, u'addr': u'192.168.20.101', u'dhcpv6': False, u'ipv6addrs': [], u'switch': u'legacy', u'bridged': True, u'southbound': u'em1', u'dhcpv4': True, u'netmask': u'255.255.255.0', u'ipv4defaultroute': True, u'stp': u'off', u'ipv4addrs': [u'192.168.20.101/24'], u'mtu': u'1500', u'ipv6gateway': u'::', u'gateway': u'192.168.20.1', u'ports': [u'em1']}}, u'kernelArgs': u'BOOT_IMAGE=/vmlinuz-3.10.0-957.27.2.el7.x86_64 root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rhgb quiet LANG=en_US.UTF-8', u'domain_versions': [0, 2, 3, 4, 5], u'bridges': {u'ovirtmgmt': {u'ipv6autoconf': True, u'addr': u'192.168.20.101', u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': True, u'netmask': u'255.255.255.0', u'ipv4defaultroute': True, u'stp': u'off', u'ipv4addrs': [u'192.168.20.101/24'], u'ipv6gateway': u'::', u'gateway': u'192.168.20.1', u'opts': {u'multicast_last_member_count': u'2', u'vlan_protocol': u'0x8100', u'hash_elasticity': u'4', u'multicast_query_response_interval': u'1000', u'group_fwd_mask': u'0x0', u'multicast_snooping': u'1', u'multicast_startup_query_interval': u'3125', u'hello_timer': u'0', u'multicast_querier_interval': u'25500', u'max_age': u'2000', u'hash_max': u'512', u'stp_state': u'0', u'topology_change_detected': u'0', u'priority': u'32768', u'multicast_igmp_version': u'2', u'multicast_membership_interval': u'26000', u'root_path_cost': u'0', u'root_port': u'0', u'multicast_stats_enabled': u'0', u'multicast_startup_query_count': u'2', u'nf_call_iptables': u'0', u'vlan_stats_enabled': u'0', u'hello_time': u'200', u'topology_change': u'0', u'bridge_id': u'8000.f8bc1234c8a8', u'topology_change_timer': u'0', u'ageing_time': u'30000', u'nf_call_ip6tables': u'0', u'multicast_mld_version': u'1', u'gc_timer': u'6672', u'root_id': u'8000.f8bc1234c8a8', u'nf_call_arptables': u'0', u'group_addr': u'1:80:c2:0:0:0', u'multicast_last_member_interval': u'100', u'default_pvid': u'1', u'multicast_query_interval': u'12500', u'multicast_query_use_ifaddr': u'0', u'tcn_timer': u'0', u'multicast_router': u'1', u'vlan_filtering': u'0', u'multicast_querier': u'0', u'forward_delay': u'0'}, u'ports': [u'em1']}}, u'uuid': u'4c4c4544-0046-3310-8034-c8c04f4a3032', u'onlineCpus': u'0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,1,3,5,7,9,11,13,15,17,19,21,23, 25,27,29,31', u'nameservers': [u'192.168.168.1'], u'nics': {u'em4': {u'ipv6autoconf': True, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:ab', u'ipv6gateway': u'::', u'gateway': u''}, u'idrac': {u'ipv6autoconf': True, u'addr': u'', u'speed': 0, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:37:d8:b1', u'ipv6gateway': u'::', u'gateway': u''}, u'em1': {u'ipv6autoconf': False, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:a8', u'ipv6gateway': u'::', u'gateway': u''}, u'em3': {u'ipv6autoconf': True, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:aa', u'ipv6gateway': u'::', u'gateway': u''}, u'em2': {u'ipv6autoconf': True, u'addr': u'', u'speed': 100, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:a9', u'ipv6gateway': u'::', u'gateway': u''}}, u'software_revision': u'1', u'hostdevPassthrough': u'false', u'clusterLevels': [u'4.1', u'4.2', u'4.3'], u'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,cl flush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,c onstant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfm perf,eagerfpu,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16 ,xtpr,pdcm,pcid,dca,sse4_1,sse4_2,x2apic,popcnt,tsc_deadline_timer,aes,xsave ,avx,f16c,rdrand,lahf_lm,ssbd,ibrs,ibpb,stibp,tpr_shadow,vnmi,flexpriority,e pt,vpid,fsgsbase,smep,erms,xsaveopt,dtherm,ida,arat,pln,pts,md_clear,spec_ct rl,intel_stibp,flush_l1d,model_Opteron_G2,model_kvm32,model_SandyBridge-IBRS ,model_Opteron_G1,model_core2duo,model_pentium,model_Westmere-IBRS,model_qem u64,model_kvm64,model_coreduo,model_Nehalem,model_Nehalem-IBRS,model_IvyBrid ge,model_Westmere,model_SandyBridge,model_IvyBridge-IBRS,model_qemu32,model_ Penryn,model_pentium2,model_pentium3,model_Conroe,model_486', u'kernelFeatures': {u'IBRS': 0, u'RETP': 1, u'SSBD': 3, u'PTI': 1}, u'ISCSIInitiatorName': u'iqn.1994-05.com.redhat:d8c85fc0ab85', u'netConfigDirty': u'False', u'supported_block_sizes': [512], u'selinux': {u'mode': u'1'}, u'autoNumaBalancing': 1, u'reservedMem': u'321', u'bondings': {}, u'software_version': u'4.30.24', u'supportedENGINEs': [u'4.1', u'4.2', u'4.3'], u'vncEncrypted': False, u'backupEnabled': False, u'cpuSpeed': u'1199.951', u'numaNodes': {u'1': {u'totalMemory': u'32768', u'cpus': [1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31]}, u'0': {u'totalMemory': u'32704', u'cpus': [0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30]}}, u'cpuSockets': u'2', u'vlans': {}, u'version_name': u'Snow Man', 'lastClientIface': 'lo', u'cpuCores': u'16', u'hostedEngineDeployed': True, u'hugepages': [1048576, 2048], u'guestOverhead': u'65', u'additionalFeatures': [u'libgfapi_supported'], u'openstack_binding_host_ids': {u'OPENSTACK_OVN': u'baremetal1.ledinfo.com', u'OPEN_VSWITCH': u'baremetal1.ledinfo.com', u'OVIRT_PROVIDER_OVN': u'c83bc734-603c-4a99-a5b9-5833ef1d90c7'}, u'kvmEnabled': u'true', u'memSize': u'64195', u'emulatedMachines': [u'pc-i440fx-rhel7.1.0', u'pc-q35-rhel7.3.0', u'rhel6.3.0', u'pc-i440fx-rhel7.5.0', u'pc-i440fx-rhel7.0.0', u'rhel6.1.0', u'pc-q35-rhel7.6.0', u'pc-i440fx-rhel7.4.0', u'rhel6.6.0', u'pc-q35-rhel7.5.0', u'rhel6.2.0', u'pc', u'pc-i440fx-rhel7.3.0', u'q35', u'pc-i440fx-rhel7.2.0', u'rhel6.4.0', u'pc-q35-rhel7.4.0', u'pc-i440fx-rhel7.6.0', u'rhel6.0.0', u'rhel6.5.0'], u'rngSources': [u'hwrng', u'random'], u'operatingSystem': {u'release': u'6.1810.2.el7.centos', u'pretty_name': u'CentOS Linux 7 (Core)', u'version': u'7', u'name': u'RHEL'}}} from=::1,42394 (api:54) 2019-09-09 09:08:50,493-0400 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.getCapabilities succeeded in 2.65 seconds (__init__:312) 2019-09-09 09:08:51,518-0400 INFO (jsonrpc/5) [api.host] START getAllVmStats() from=::1,42344 (api:48) 2019-09-09 09:08:51,518-0400 INFO (jsonrpc/5) [api.host] FINISH getAllVmStats return={'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} from=::1,42344 (api:54) 2019-09-09 09:08:51,519-0400 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:51,551-0400 INFO (jsonrpc/3) [vdsm.api] START connectStorageServer(domType=1, spUUID=u'00000000-0000-0000-0000-000000000000', conList=[{u'tpgt': u'1', u'id': u'00000000-0000-0000-0000-000000000000', u'connection': u'192.168.20.125:/ovirt', u'iqn': u'', u'user': u'', u'ipv6_enabled': u'false', u'protocol_version': u'auto', u'password': '********', u'port': u''}], options=None) from=::ffff:192.168.20.116,41974, flow_id=1b259177-b83c-4066-a045-b3a9db670a1e, task_id=3d0e32c4-288b-4be4-98ef-90fb676b57f2 (api:48) 2019-09-09 09:08:51,556-0400 INFO (jsonrpc/3) [storage.StorageServer.MountConnection] Creating directory u'/rhev/data-center/mnt/192.168.20.125:_ovirt' (storageServer:168) 2019-09-09 09:08:51,556-0400 INFO (jsonrpc/3) [storage.fileUtils] Creating directory: /rhev/data-center/mnt/192.168.20.125:_ovirt mode: None (fileUtils:199) 2019-09-09 09:08:51,556-0400 INFO (jsonrpc/3) [storage.Mount] mounting 192.168.20.125:/ovirt at /rhev/data-center/mnt/192.168.20.125:_ovirt (mount:204) 2019-09-09 09:08:51,684-0400 INFO (jsonrpc/3) [IOProcessClient] (Global) Starting client (__init__:308) 2019-09-09 09:08:51,705-0400 INFO (ioprocess/9120) [IOProcess] (Global) Starting ioprocess (__init__:434) 2019-09-09 09:08:51,739-0400 INFO (jsonrpc/3) [vdsm.api] FINISH connectStorageServer return={'statuslist': [{'status': 0, 'id': u'00000000-0000-0000-0000-000000000000'}]} from=::ffff:192.168.20.116,41974, flow_id=1b259177-b83c-4066-a045-b3a9db670a1e, task_id=3d0e32c4-288b-4be4-98ef-90fb676b57f2 (api:54) 2019-09-09 09:08:51,741-0400 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call StoragePool.connectStorageServer succeeded in 0.19 seconds (__init__:312) 2019-09-09 09:08:51,992-0400 INFO (jsonrpc/4) [vdsm.api] START connectStorageServer(domType=1, spUUID=u'00000000-0000-0000-0000-000000000000', conList=[{u'tpgt': u'1', u'id': u'1280de25-c248-41d3-8552-50ef964d0ee0', u'connection': u'192.168.20.125:/ovirt', u'iqn': u'', u'user': u'', u'ipv6_enabled': u'false', u'protocol_version': u'auto', u'password': '********', u'port': u''}], options=None) from=::ffff:192.168.20.116,41974, flow_id=360af944, task_id=44612f73-58c9-4597-9a29-c1e65e149c0a (api:48) 2019-09-09 09:08:51,999-0400 INFO (jsonrpc/4) [vdsm.api] FINISH connectStorageServer return={'statuslist': [{'status': 0, 'id': u'1280de25-c248-41d3-8552-50ef964d0ee0'}]} from=::ffff:192.168.20.116,41974, flow_id=360af944, task_id=44612f73-58c9-4597-9a29-c1e65e149c0a (api:54) 2019-09-09 09:08:52,000-0400 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call StoragePool.connectStorageServer succeeded in 0.01 seconds (__init__:312) 2019-09-09 09:08:52,010-0400 INFO (jsonrpc/6) [vdsm.api] START createStorageDomain(storageType=1, sdUUID=u'785a9a50-96c0-4cac-9c80-d0e2dd5bd162', domainName=u'ovirt-omv', typeSpecificArg=u'192.168.20.125:/ovirt', domClass=1, domVersion=u'5', block_size=512, max_hosts=250, options=None) from=::ffff:192.168.20.116,41974, flow_id=360af944, task_id=62008086-97f8-4986-b329-cc193b84fd95 (api:48) 2019-09-09 09:08:52,727-0400 WARN (jsonrpc/6) [storage.LVM] Reloading VGs failed (vgs=[u'785a9a50-96c0-4cac-9c80-d0e2dd5bd162'] rc=5 out=[] err=[' Volume group "785a9a50-96c0-4cac-9c80-d0e2dd5bd162" not found', ' Cannot process volume group 785a9a50-96c0-4cac-9c80-d0e2dd5bd162']) (lvm:470) 2019-09-09 09:08:52,731-0400 INFO (itmap/0) [IOProcessClient] (/baremetal3.ledinfo.com:_export) Starting client (__init__:308) 2019-09-09 09:08:52,751-0400 INFO (ioprocess/9141) [IOProcess] (/baremetal3.ledinfo.com:_export) Starting ioprocess (__init__:434) 2019-09-09 09:08:52,753-0400 INFO (itmap/1) [IOProcessClient] (/baremetal3.ledinfo.com:_import) Starting client (__init__:308) 2019-09-09 09:08:52,777-0400 INFO (itmap/2) [IOProcessClient] (/baremetal3.ledinfo.com:_ISO) Starting client (__init__:308) 2019-09-09 09:08:52,783-0400 INFO (ioprocess/9150) [IOProcess] (/baremetal3.ledinfo.com:_import) Starting ioprocess (__init__:434) 2019-09-09 09:08:52,802-0400 INFO (itmap/3) [IOProcessClient] (/192.168.20.125:_ovirt) Starting client (__init__:308) 2019-09-09 09:08:52,822-0400 INFO (ioprocess/9157) [IOProcess] (/baremetal3.ledinfo.com:_ISO) Starting ioprocess (__init__:434) 2019-09-09 09:08:52,828-0400 INFO (ioprocess/9160) [IOProcess] (/192.168.20.125:_ovirt) Starting ioprocess (__init__:434) 2019-09-09 09:08:52,831-0400 INFO (jsonrpc/6) [storage.StorageDomain] sdUUID=785a9a50-96c0-4cac-9c80-d0e2dd5bd162 domainName=ovirt-omv remotePath=192.168.20.125:/ovirt domClass=1, block_size=512, alignment=1048576 (nfsSD:86) 2019-09-09 09:08:52,866-0400 INFO (jsonrpc/6) [IOProcessClient] (785a9a50-96c0-4cac-9c80-d0e2dd5bd162) Starting client (__init__:308) 2019-09-09 09:08:52,889-0400 INFO (ioprocess/9174) [IOProcess] (785a9a50-96c0-4cac-9c80-d0e2dd5bd162) Starting ioprocess (__init__:434) 2019-09-09 09:08:52,895-0400 INFO (jsonrpc/6) [storage.StorageDomain] Creating domain metadata directory u'/rhev/data-center/mnt/192.168.20.125:_ovirt/785a9a50-96c0-4cac-9c80-d0e2dd 5bd162/dom_md' (fileSD:453) 2019-09-09 09:08:52,926-0400 INFO (jsonrpc/6) [storage.xlease] Formatting index for lockspace u'785a9a50-96c0-4cac-9c80-d0e2dd5bd162' (version=1) (xlease:691) 2019-09-09 09:08:53,047-0400 INFO (jsonrpc/6) [storage.StorageDomain] Creating domain images directory u'/rhev/data-center/mnt/192.168.20.125:_ovirt/785a9a50-96c0-4cac-9c80-d0e2dd 5bd162/images' (nfsSD:110) 2019-09-09 09:08:53,067-0400 INFO (jsonrpc/6) [storage.StorageDomain] Removing remnants of deleted images [] (fileSD:747) 2019-09-09 09:08:53,072-0400 ERROR (jsonrpc/6) [storage.initSANLock] Cannot initialize SANLock for domain 785a9a50-96c0-4cac-9c80-d0e2dd5bd162 (clusterlock:258) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/clusterlock.py", line 249, in initSANLock sdUUID, idsPath, align=alignment, sector=block_size) File "/usr/lib/python2.7/site-packages/vdsm/storage/compat.py", line 52, in decorator return func(*args, **kwargs) SanlockException: (19, 'Sanlock lockspace write failure', 'No such device') 2019-09-09 09:08:53,073-0400 INFO (jsonrpc/6) [vdsm.api] FINISH createStorageDomain error=Could not initialize cluster lock: () from=::ffff:192.168.20.116,41974, flow_id=360af944, task_id=62008086-97f8-4986-b329-cc193b84fd95 (api:52) 2019-09-09 09:08:53,074-0400 ERROR (jsonrpc/6) [storage.TaskManager.Task] (Task='62008086-97f8-4986-b329-cc193b84fd95') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "<string>", line 2, in createStorageDomain File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 2619, in createStorageDomain storageType, domVersion, block_size, alignment) File "/usr/lib/python2.7/site-packages/vdsm/storage/nfsSD.py", line 120, in create fsd.initSPMlease() File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 918, in initSPMlease return self._manifest.initDomainLock() File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 609, in initDomainLock self._domainLock.initLock(self.getDomainLease()) File "/usr/lib/python2.7/site-packages/vdsm/storage/clusterlock.py", line 303, in initLock block_size=self._block_size) File "/usr/lib/python2.7/site-packages/vdsm/storage/clusterlock.py", line 259, in initSANLock raise se.ClusterLockInitError() ClusterLockInitError: Could not initialize cluster lock: () 2019-09-09 09:08:53,075-0400 INFO (jsonrpc/6) [storage.TaskManager.Task] (Task='62008086-97f8-4986-b329-cc193b84fd95') aborting: Task is aborted: 'Could not initialize cluster lock: ()' - code 701 (task:1181) 2019-09-09 09:08:53,076-0400 ERROR (jsonrpc/6) [storage.Dispatcher] FINISH createStorageDomain error=Could not initialize cluster lock: () (dispatcher:83) 2019-09-09 09:08:53,077-0400 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call StorageDomain.create failed (error 701) in 1.07 seconds (__init__:312) 2019-09-09 09:08:53,425-0400 INFO (jsonrpc/0) [api.host] START getStats() from=::ffff:192.168.20.116,41974 (api:48) 2019-09-09 09:08:53,439-0400 INFO (jsonrpc/1) [vdsm.api] START disconnectStorageServer(domType=1, spUUID=u'00000000-0000-0000-0000-000000000000', conList=[{u'tpgt': u'1', u'id': u'1280de25-c248-41d3-8552-50ef964d0ee0', u'connection': u'192.168.20.125:/ovirt', u'iqn': u'', u'user': u'', u'ipv6_enabled': u'false', u'protocol_version': u'auto', u'password': '********', u'port': u''}], options=None) from=::ffff:192.168.20.116,41974, flow_id=e003809f-c7ef-4105-ac27-48be39b5c7b1, task_id=da5b627a-de84-4083-9454-aaecd6bb8aed (api:48) 2019-09-09 09:08:53,440-0400 INFO (jsonrpc/1) [storage.Mount] unmounting /rhev/data-center/mnt/192.168.20.125:_ovirt (mount:212) 2019-09-09 09:08:53,478-0400 INFO (jsonrpc/0) [vdsm.api] START repoStats(domains=()) from=::ffff:192.168.20.116,41974, task_id=42f53132-6ef6-4aff-9ab3-91d19c24bd93 (api:48) 2019-09-09 09:08:53,479-0400 INFO (jsonrpc/0) [vdsm.api] FINISH repoStats return={u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.00104514', 'lastCheck': '3.4', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000798507', 'lastCheck': '3.5', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000768283', 'lastCheck': '3.3', 'valid': True}} from=::ffff:192.168.20.116,41974, task_id=42f53132-6ef6-4aff-9ab3-91d19c24bd93 (api:54) 2019-09-09 09:08:53,481-0400 INFO (jsonrpc/0) [vdsm.api] START multipath_health() from=::ffff:192.168.20.116,41974, task_id=34ab20bd-b1ce-4bbe-ad1c-af0e78d38b1c (api:48) 2019-09-09 09:08:53,481-0400 INFO (jsonrpc/0) [vdsm.api] FINISH multipath_health return={} from=::ffff:192.168.20.116,41974, task_id=34ab20bd-b1ce-4bbe-ad1c-af0e78d38b1c (api:54) 2019-09-09 09:08:53,523-0400 INFO (jsonrpc/0) [api.host] FINISH getStats return={'status': {'message': 'Done', 'code': 0}, 'info': {'cpuStatistics': {'24': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.53'}, '25': {'cpuUser': '0.46', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.47'}, '26': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '27': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '20': {'cpuUser': '2.06', 'nodeIndex': 0, 'cpuSys': '0.40', 'cpuIdle': '97.54'}, '21': {'cpuUser': '0.99', 'nodeIndex': 1, 'cpuSys': '0.33', 'cpuIdle': '98.68'}, '22': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '23': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.74'}, '28': {'cpuUser': '0.99', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '98.88'}, '29': {'cpuUser': '0.46', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.47'}, '1': {'cpuUser': '2.85', 'nodeIndex': 1, 'cpuSys': '1.66', 'cpuIdle': '95.49'}, '0': {'cpuUser': '0.46', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.41'}, '3': {'cpuUser': '0.73', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.20'}, '2': {'cpuUser': '0.33', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.47'}, '5': {'cpuUser': '0.20', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.73'}, '4': {'cpuUser': '0.99', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '98.68'}, '7': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.74'}, '6': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '99.33'}, '9': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '8': {'cpuUser': '0.93', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '98.80'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.53'}, '13': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '12': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '15': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '14': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '17': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '16': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.87'}, '19': {'cpuUser': '1.79', 'nodeIndex': 1, 'cpuSys': '0.46', 'cpuIdle': '97.75'}, '18': {'cpuUser': '8.69', 'nodeIndex': 0, 'cpuSys': '1.72', 'cpuIdle': '89.59'}, '31': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '30': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.80'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31474'}, '0': {'memPercent': 7, 'memFree': '30445'}}, 'memShared': 0, 'haScore': 3400, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'vmCount': 0, 'memUsed': '3', 'storageDomains': {u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.00104514', 'lastCheck': '3.4', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000798507', 'lastCheck': '3.5', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000768283', 'lastCheck': '3.3', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'idrac': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'idrac', 'tx': '180', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '0', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '0', 'state': 'down'}, 'ovirtmgmt': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '1075026007', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '43585639790', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '618193132', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '618193132', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '0', 'state': 'down'}, 'em4': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em4', 'tx': '192', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034533.476725, 'rx': '38644112', 'state': 'up'}, 'br-int': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'br-int', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '0', 'state': 'down'}, 'em1': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em1', 'tx': '1115341150', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034533.476725, 'rx': '45376217511', 'state': 'up'}, 'genev_sys_6081': {'rxErrors': '0', 'txErrors': '8', 'speed': '1000', 'rxDropped': '0', 'name': 'genev_sys_6081', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '0', 'state': 'up'}, 'em3': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em3', 'tx': '192', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034533.476725, 'rx': '38644112', 'state': 'up'}, 'em2': {'rxErrors': '0', 'txErrors': '0', 'speed': '100', 'rxDropped': '0', 'name': 'em2', 'tx': '0', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034533.476725, 'rx': '16049108', 'state': 'up'}}, 'txDropped': '0', 'anonHugePages': '306', 'ksmPages': 100, 'elapsedTime': '223906.36', 'cpuLoad': '0.14', 'cpuSys': '0.24', 'diskStats': {'/var/log': {'free': '136508'}, '/var/run/vdsm/': {'free': '31856'}, '/tmp': {'free': '136508'}}, 'cpuUserVdsmd': '2.78', 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 63031, 'bootTime': '1567810572', 'haStats': {'active': True, 'configured': True, 'score': 3400, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'multipathHealth': {}, 'rxDropped': '0', 'outgoingVmMigrations': 0, 'swapTotal': 0, 'swapFree': 0, 'hugepages': defaultdict(<type 'dict'>, {1048576: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}, 2048: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}), 'dateTime': '2019-09-09T13:08:53 GMT', 'cpuUser': '0.75', 'memFree': 62775, 'cpuIdle': '99.01', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd': '1.06'}} from=::ffff:192.168.20.116,41974 (api:54) 2019-09-09 09:08:53,534-0400 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.11 seconds (__init__:312) 2019-09-09 09:08:54,001-0400 INFO (jsonrpc/1) [vdsm.api] FINISH disconnectStorageServer return={'statuslist': [{'status': 0, 'id': u'1280de25-c248-41d3-8552-50ef964d0ee0'}]} from=::ffff:192.168.20.116,41974, flow_id=e003809f-c7ef-4105-ac27-48be39b5c7b1, task_id=da5b627a-de84-4083-9454-aaecd6bb8aed (api:54) 2019-09-09 09:08:54,003-0400 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call StoragePool.disconnectStorageServer succeeded in 0.57 seconds (__init__:312) 2019-09-09 09:08:54,888-0400 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:54,895-0400 INFO (jsonrpc/2) [api.host] START getStats() from=::1,42394 (api:48) 2019-09-09 09:08:54,946-0400 INFO (jsonrpc/2) [vdsm.api] START repoStats(domains=()) from=::1,42394, task_id=f1acae4b-2563-4335-96c7-8b5097aa4127 (api:48) 2019-09-09 09:08:54,947-0400 INFO (jsonrpc/2) [vdsm.api] FINISH repoStats return={u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.00104514', 'lastCheck': '4.8', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000798507', 'lastCheck': '5.0', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000768283', 'lastCheck': '4.8', 'valid': True}} from=::1,42394, task_id=f1acae4b-2563-4335-96c7-8b5097aa4127 (api:54) 2019-09-09 09:08:54,948-0400 INFO (jsonrpc/2) [vdsm.api] START multipath_health() from=::1,42394, task_id=58483dc0-2a50-409a-b991-16dbb57360a7 (api:48) 2019-09-09 09:08:54,949-0400 INFO (jsonrpc/2) [vdsm.api] FINISH multipath_health return={} from=::1,42394, task_id=58483dc0-2a50-409a-b991-16dbb57360a7 (api:54) 2019-09-09 09:08:54,977-0400 INFO (jsonrpc/2) [api.host] FINISH getStats return={'status': {'message': 'Done', 'code': 0}, 'info': {'cpuStatistics': {'24': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.53'}, '25': {'cpuUser': '0.46', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.47'}, '26': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '27': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '20': {'cpuUser': '2.06', 'nodeIndex': 0, 'cpuSys': '0.40', 'cpuIdle': '97.54'}, '21': {'cpuUser': '0.99', 'nodeIndex': 1, 'cpuSys': '0.33', 'cpuIdle': '98.68'}, '22': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '23': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.74'}, '28': {'cpuUser': '0.99', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '98.88'}, '29': {'cpuUser': '0.46', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.47'}, '1': {'cpuUser': '2.85', 'nodeIndex': 1, 'cpuSys': '1.66', 'cpuIdle': '95.49'}, '0': {'cpuUser': '0.46', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.41'}, '3': {'cpuUser': '0.73', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.20'}, '2': {'cpuUser': '0.33', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.47'}, '5': {'cpuUser': '0.20', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.73'}, '4': {'cpuUser': '0.99', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '98.68'}, '7': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.74'}, '6': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '99.33'}, '9': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '8': {'cpuUser': '0.93', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '98.80'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.53'}, '13': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '12': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '15': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '14': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '17': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '16': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.87'}, '19': {'cpuUser': '1.79', 'nodeIndex': 1, 'cpuSys': '0.46', 'cpuIdle': '97.75'}, '18': {'cpuUser': '8.69', 'nodeIndex': 0, 'cpuSys': '1.72', 'cpuIdle': '89.59'}, '31': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '30': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.80'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31474'}, '0': {'memPercent': 7, 'memFree': '30445'}}, 'memShared': 0, 'haScore': 3400, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'vmCount': 0, 'memUsed': '3', 'storageDomains': {u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.00104514', 'lastCheck': '4.8', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000798507', 'lastCheck': '5.0', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000768283', 'lastCheck': '4.8', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'idrac': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'idrac', 'tx': '180', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '0', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '0', 'state': 'down'}, 'ovirtmgmt': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '1075046072', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '43586697397', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '618194231', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '618194231', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '0', 'state': 'down'}, 'em4': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em4', 'tx': '192', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034534.944025, 'rx': '38644112', 'state': 'up'}, 'br-int': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'br-int', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '0', 'state': 'down'}, 'em1': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em1', 'tx': '1115653922', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034534.944025, 'rx': '45377347900', 'state': 'up'}, 'genev_sys_6081': {'rxErrors': '0', 'txErrors': '8', 'speed': '1000', 'rxDropped': '0', 'name': 'genev_sys_6081', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '0', 'state': 'up'}, 'em3': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em3', 'tx': '192', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034534.944025, 'rx': '38644176', 'state': 'up'}, 'em2': {'rxErrors': '0', 'txErrors': '0', 'speed': '100', 'rxDropped': '0', 'name': 'em2', 'tx': '0', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034534.944025, 'rx': '16049108', 'state': 'up'}}, 'txDropped': '0', 'anonHugePages': '306', 'ksmPages': 100, 'elapsedTime': '223907.83', 'cpuLoad': '0.14', 'cpuSys': '0.24', 'diskStats': {'/var/log': {'free': '136508'}, '/var/run/vdsm/': {'free': '31856'}, '/tmp': {'free': '136508'}}, 'cpuUserVdsmd': '2.78', 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 63031, 'bootTime': '1567810572', 'haStats': {'active': True, 'configured': True, 'score': 3400, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'multipathHealth': {}, 'rxDropped': '0', 'outgoingVmMigrations': 0, 'swapTotal': 0, 'swapFree': 0, 'hugepages': defaultdict(<type 'dict'>, {1048576: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}, 2048: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}), 'dateTime': '2019-09-09T13:08:54 GMT', 'cpuUser': '0.75', 'memFree': 62775, 'cpuIdle': '99.01', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd': '1.06'}} from=::1,42394 (api:54) 2019-09-09 09:08:54,986-0400 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.09 seconds (__init__:312) 2019-09-09 09:08:55,842-0400 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:55,850-0400 INFO (jsonrpc/3) [api.virt] START getStats() from=::1,42394, vmId=aeed8192-5313-45d6-b9f5-fb2db09fb646 (api:48) 2019-09-09 09:08:55,850-0400 INFO (jsonrpc/3) [api] FINISH getStats error=Virtual machine does not exist: {'vmId': u'aeed8192-5313-45d6-b9f5-fb2db09fb646'} (api:129) 2019-09-09 09:08:55,850-0400 INFO (jsonrpc/3) [api.virt] FINISH getStats return={'status': {'message': "Virtual machine does not exist: {'vmId': u'aeed8192-5313-45d6-b9f5-fb2db09fb646'}", 'code': 1}} from=::1,42394, vmId=aeed8192-5313-45d6-b9f5-fb2db09fb646 (api:54) 2019-09-09 09:08:55,851-0400 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call VM.getStats failed (error 1) in 0.00 seconds (__init__:312) 2019-09-09 09:08:57,526-0400 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:57,534-0400 INFO (jsonrpc/6) [api.host] START getCapabilities() from=::1,42394 (api:48) 2019-09-09 09:08:57,627-0400 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:57,636-0400 INFO (jsonrpc/1) [vdsm.api] START repoStats(domains=[u'4f4828c2-8f78-4566-a69d-a37cd73446d4']) from=::1,42394, task_id=6da33ef3-de09-48a3-8ba9-6c31d5e2942b (api:48) 2019-09-09 09:08:57,636-0400 INFO (jsonrpc/1) [vdsm.api] FINISH repoStats return={u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000798507', 'lastCheck': '0.4', 'valid': True}} from=::1,42394, task_id=6da33ef3-de09-48a3-8ba9-6c31d5e2942b (api:54) 2019-09-09 09:08:57,637-0400 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:57,870-0400 INFO (jsonrpc/6) [root] managedvolume not supported: Managed Volume Not Supported. Missing package os-brick.: ('Cannot import os_brick',) (caps:152) 2019-09-09 09:08:58,884-0400 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:58,990-0400 INFO (jsonrpc/6) [root] /usr/libexec/vdsm/hooks/after_get_caps/50_openstacknet: rc=0 err= (hooks:114) 2019-09-09 09:08:59,048-0400 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:59,055-0400 INFO (jsonrpc/5) [vdsm.api] START repoStats(domains=[u'4f4828c2-8f78-4566-a69d-a37cd73446d4']) from=::1,42394, task_id=933c8ee8-2330-46ed-a609-38a519447156 (api:48) 2019-09-09 09:08:59,056-0400 INFO (jsonrpc/5) [vdsm.api] FINISH repoStats return={u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000798507', 'lastCheck': '1.8', 'valid': True}} from=::1,42394, task_id=933c8ee8-2330-46ed-a609-38a519447156 (api:54) 2019-09-09 09:08:59,057-0400 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:59,526-0400 INFO (jsonrpc/6) [root] /usr/libexec/vdsm/hooks/after_get_caps/openstacknet_utils.py: rc=0 err= (hooks:114) 2019-09-09 09:08:59,896-0400 INFO (jsonrpc/3) [api.host] START getAllVmStats() from=::ffff:192.168.20.116,41974 (api:48) 2019-09-09 09:08:59,897-0400 INFO (jsonrpc/3) [api.host] FINISH getAllVmStats return={'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} from=::ffff:192.168.20.116,41974 (api:54) 2019-09-09 09:08:59,898-0400 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:09:00,188-0400 INFO (periodic/0) [vdsm.api] START repoStats(domains=()) from=internal, task_id=aec0525d-393b-49fa-a0d4-96bc19f829e3 (api:48) 2019-09-09 09:09:00,190-0400 INFO (periodic/0) [vdsm.api] FINISH repoStats return={u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000991915', 'lastCheck': '0.1', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000985319', 'lastCheck': '0.2', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000768283', 'lastCheck': '3.1', 'valid': True}} from=internal, task_id=aec0525d-393b-49fa-a0d4-96bc19f829e3 (api:54) 2019-09-09 09:09:00,192-0400 INFO (jsonrpc/6) [root] /usr/libexec/vdsm/hooks/after_get_caps/ovirt_provider_ovn_hook: rc=0 err= (hooks:114) 2019-09-09 09:09:00,194-0400 INFO (jsonrpc/6) [api.host] FINISH getCapabilities return={'status': {'message': 'Done', 'code': 0}, 'info': {u'HBAInventory': {u'iSCSI': [{u'InitiatorName': u'iqn.1994-05.com.redhat:d8c85fc0ab85'}], u'FC': []}, u'packages2': {u'kernel': {u'release': u'957.27.2.el7.x86_64', u'version': u'3.10.0'}, u'spice-server': {u'release': u'6.el7_6.1', u'version': u'0.14.0'}, u'librbd1': {u'release': u'4.el7', u'version': u'10.2.5'}, u'vdsm': {u'release': u'1.el7', u'version': u'4.30.24'}, u'qemu-kvm': {u'release': u'18.el7_6.7.1', u'version': u'2.12.0'}, u'openvswitch': {u'release': u'4.el7', u'version': u'2.11.0'}, u'libvirt': {u'release': u'10.el7_6.12', u'version': u'4.5.0'}, u'ovirt-hosted-engine-ha': {u'release': u'1.el7', u'version': u'2.3.3'}, u'qemu-img': {u'release': u'18.el7_6.7.1', u'version': u'2.12.0'}, u'mom': {u'release': u'1.el7.centos', u'version': u'0.5.12'}, u'glusterfs-cli': {u'release': u'1.el7', u'version': u'6.5'}}, u'numaNodeDistance': {u'1': [20, 10], u'0': [10, 20]}, u'cpuModel': u'Intel(R) Xeon(R) CPU E5-2640 v2 @ 2.00GHz', u'nestedVirtualization': False, u'liveMerge': u'true', u'hooks': {u'before_vm_start': {u'50_hostedengine': {u'md5': u'95c810cdcfe4195302a59574a5148289'}, u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'after_network_setup': {u'30_ethtool_options': {u'md5': u'ce1fbad7aa0389e3b06231219140bf0d'}}, u'after_vm_destroy': {u'delete_vhostuserclient_hook': {u'md5': u'c2f279cc9483a3f842f6c29df13994c1'}, u'50_vhostmd': {u'md5': u'bdf4802c0521cf1bae08f2b90a9559cf'}}, u'after_vm_start': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'ea0a5a715da8c1badbcda28e8b8fa00e'}}, u'after_device_migrate_destination': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'6226fbc4d1602994828a3904fc1b875d'}}, u'before_device_migrate_destination': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}}, u'after_device_create': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'6226fbc4d1602994828a3904fc1b875d'}}, u'after_nic_hotplug': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'6226fbc4d1602994828a3904fc1b875d'}}, u'before_vm_migrate_destination': {u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'before_vm_dehibernate': {u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'before_nic_hotplug': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}, u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'ec8d5d7ba063a109f749cd63c7a2b58d'}, u'20_ovirt_provider_ovn_vhostuser_hook': {u'md5': u'a8af653b7386c138b2e6e9738bd6b62c'}, u'10_ovirt_provider_ovn_hook': {u'md5': u'73822988042847bab1ea832a6b9fa837'}}, u'after_get_caps': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'5c3a9ab6e06e039bdd220c0216e45809'}, u'ovirt_provider_ovn_hook': {u'md5': u'1a6ce22288fc2466fcc339da252ef624'}}, u'before_network_setup': {u'50_fcoe': {u'md5': u'28c352339c8beef1e1b05c67d106d062'}}, u'before_device_create': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}, u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'ec8d5d7ba063a109f749cd63c7a2b58d'}, u'20_ovirt_provider_ovn_vhostuser_hook': {u'md5': u'a8af653b7386c138b2e6e9738bd6b62c'}, u'10_ovirt_provider_ovn_hook': {u'md5': u'73822988042847bab1ea832a6b9fa837'}}}, u'supportsIPv6': True, u'realtimeKernel': False, u'vmTypes': [u'kvm'], u'liveSnapshot': u'true', u'cpuThreads': u'32', u'kdumpStatus': 0, u'deferred_preallocation': True, u'networks': {u'ovirtmgmt': {u'iface': u'ovirtmgmt', u'ipv6autoconf': True, u'addr': u'192.168.20.101', u'dhcpv6': False, u'ipv6addrs': [], u'switch': u'legacy', u'bridged': True, u'southbound': u'em1', u'dhcpv4': True, u'netmask': u'255.255.255.0', u'ipv4defaultroute': True, u'stp': u'off', u'ipv4addrs': [u'192.168.20.101/24'], u'mtu': u'1500', u'ipv6gateway': u'::', u'gateway': u'192.168.20.1', u'ports': [u'em1']}}, u'kernelArgs': u'BOOT_IMAGE=/vmlinuz-3.10.0-957.27.2.el7.x86_64 root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rhgb quiet LANG=en_US.UTF-8', u'domain_versions': [0, 2, 3, 4, 5], u'bridges': {u'ovirtmgmt': {u'ipv6autoconf': True, u'addr': u'192.168.20.101', u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': True, u'netmask': u'255.255.255.0', u'ipv4defaultroute': True, u'stp': u'off', u'ipv4addrs': [u'192.168.20.101/24'], u'ipv6gateway': u'::', u'gateway': u'192.168.20.1', u'opts': {u'multicast_last_member_count': u'2', u'vlan_protocol': u'0x8100', u'hash_elasticity': u'4', u'multicast_query_response_interval': u'1000', u'group_fwd_mask': u'0x0', u'multicast_snooping': u'1', u'multicast_startup_query_interval': u'3125', u'hello_timer': u'0', u'multicast_querier_interval': u'25500', u'max_age': u'2000', u'hash_max': u'512', u'stp_state': u'0', u'topology_change_detected': u'0', u'priority': u'32768', u'multicast_igmp_version': u'2', u'multicast_membership_interval': u'26000', u'root_path_cost': u'0', u'root_port': u'0', u'multicast_stats_enabled': u'0', u'multicast_startup_query_count': u'2', u'nf_call_iptables': u'0', u'vlan_stats_enabled': u'0', u'hello_time': u'200', u'topology_change': u'0', u'bridge_id': u'8000.f8bc1234c8a8', u'topology_change_timer': u'0', u'ageing_time': u'30000', u'nf_call_ip6tables': u'0', u'multicast_mld_version': u'1', u'gc_timer': u'5700', u'root_id': u'8000.f8bc1234c8a8', u'nf_call_arptables': u'0', u'group_addr': u'1:80:c2:0:0:0', u'multicast_last_member_interval': u'100', u'default_pvid': u'1', u'multicast_query_interval': u'12500', u'multicast_query_use_ifaddr': u'0', u'tcn_timer': u'0', u'multicast_router': u'1', u'vlan_filtering': u'0', u'multicast_querier': u'0', u'forward_delay': u'0'}, u'ports': [u'em1']}}, u'uuid': u'4c4c4544-0046-3310-8034-c8c04f4a3032', u'onlineCpus': u'0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,1,3,5,7,9,11,13,15,17,19,21,23, 25,27,29,31', u'nameservers': [u'192.168.168.1'], u'nics': {u'em4': {u'ipv6autoconf': True, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:ab', u'ipv6gateway': u'::', u'gateway': u''}, u'idrac': {u'ipv6autoconf': True, u'addr': u'', u'speed': 0, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:37:d8:b1', u'ipv6gateway': u'::', u'gateway': u''}, u'em1': {u'ipv6autoconf': False, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:a8', u'ipv6gateway': u'::', u'gateway': u''}, u'em3': {u'ipv6autoconf': True, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:aa', u'ipv6gateway': u'::', u'gateway': u''}, u'em2': {u'ipv6autoconf': True, u'addr': u'', u'speed': 100, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:a9', u'ipv6gateway': u'::', u'gateway': u''}}, u'software_revision': u'1', u'hostdevPassthrough': u'false', u'clusterLevels': [u'4.1', u'4.2', u'4.3'], u'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,cl flush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,c onstant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfm perf,eagerfpu,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16 ,xtpr,pdcm,pcid,dca,sse4_1,sse4_2,x2apic,popcnt,tsc_deadline_timer,aes,xsave ,avx,f16c,rdrand,lahf_lm,ssbd,ibrs,ibpb,stibp,tpr_shadow,vnmi,flexpriority,e pt,vpid,fsgsbase,smep,erms,xsaveopt,dtherm,ida,arat,pln,pts,md_clear,spec_ct rl,intel_stibp,flush_l1d,model_Opteron_G2,model_kvm32,model_SandyBridge-IBRS ,model_Opteron_G1,model_core2duo,model_pentium,model_Westmere-IBRS,model_qem u64,model_kvm64,model_coreduo,model_Nehalem,model_Nehalem-IBRS,model_IvyBrid ge,model_Westmere,model_SandyBridge,model_IvyBridge-IBRS,model_qemu32,model_ Penryn,model_pentium2,model_pentium3,model_Conroe,model_486', u'kernelFeatures': {u'IBRS': 0, u'RETP': 1, u'SSBD': 3, u'PTI': 1}, u'ISCSIInitiatorName': u'iqn.1994-05.com.redhat:d8c85fc0ab85', u'netConfigDirty': u'False', u'supported_block_sizes': [512], u'selinux': {u'mode': u'1'}, u'autoNumaBalancing': 1, u'reservedMem': u'321', u'bondings': {}, u'software_version': u'4.30.24', u'supportedENGINEs': [u'4.1', u'4.2', u'4.3'], u'vncEncrypted': False, u'backupEnabled': False, u'cpuSpeed': u'1199.951', u'numaNodes': {u'1': {u'totalMemory': u'32768', u'cpus': [1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31]}, u'0': {u'totalMemory': u'32704', u'cpus': [0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30]}}, u'cpuSockets': u'2', u'vlans': {}, u'version_name': u'Snow Man', 'lastClientIface': 'lo', u'cpuCores': u'16', u'hostedEngineDeployed': True, u'hugepages': [1048576, 2048], u'guestOverhead': u'65', u'additionalFeatures': [u'libgfapi_supported'], u'openstack_binding_host_ids': {u'OPENSTACK_OVN': u'baremetal1.ledinfo.com', u'OPEN_VSWITCH': u'baremetal1.ledinfo.com', u'OVIRT_PROVIDER_OVN': u'c83bc734-603c-4a99-a5b9-5833ef1d90c7'}, u'kvmEnabled': u'true', u'memSize': u'64195', u'emulatedMachines': [u'pc-i440fx-rhel7.1.0', u'pc-q35-rhel7.3.0', u'rhel6.3.0', u'pc-i440fx-rhel7.5.0', u'pc-i440fx-rhel7.0.0', u'rhel6.1.0', u'pc-q35-rhel7.6.0', u'pc-i440fx-rhel7.4.0', u'rhel6.6.0', u'pc-q35-rhel7.5.0', u'rhel6.2.0', u'pc', u'pc-i440fx-rhel7.3.0', u'q35', u'pc-i440fx-rhel7.2.0', u'rhel6.4.0', u'pc-q35-rhel7.4.0', u'pc-i440fx-rhel7.6.0', u'rhel6.0.0', u'rhel6.5.0'], u'rngSources': [u'hwrng', u'random'], u'operatingSystem': {u'release': u'6.1810.2.el7.centos', u'pretty_name': u'CentOS Linux 7 (Core)', u'version': u'7', u'name': u'RHEL'}}} from=::1,42394 (api:54) 2019-09-09 09:09:00,231-0400 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.getCapabilities succeeded in 2.69 seconds (__init__:312) 2019-09-09 09:09:05,021-0400 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:09:05,028-0400 INFO (jsonrpc/0) [api.host] START getStats() from=::1,42394 (api:48) 2019-09-09 09:09:05,077-0400 INFO (jsonrpc/0) [vdsm.api] START repoStats(domains=()) from=::1,42394, task_id=12e8837e-c9cd-45b9-aecb-477f86e12ec5 (api:48) 2019-09-09 09:09:05,078-0400 INFO (jsonrpc/0) [vdsm.api] FINISH repoStats return={u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000991915', 'lastCheck': '5.0', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000985319', 'lastCheck': '5.1', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.00093201', 'lastCheck': '4.8', 'valid': True}} from=::1,42394, task_id=12e8837e-c9cd-45b9-aecb-477f86e12ec5 (api:54) 2019-09-09 09:09:05,079-0400 INFO (jsonrpc/0) [vdsm.api] START multipath_health() from=::1,42394, task_id=42c12b61-4d82-4a06-b692-e11dfbb0bca6 (api:48) 2019-09-09 09:09:05,080-0400 INFO (jsonrpc/0) [vdsm.api] FINISH multipath_health return={} from=::1,42394, task_id=42c12b61-4d82-4a06-b692-e11dfbb0bca6 (api:54) 2019-09-09 09:09:05,106-0400 INFO (jsonrpc/0) [api.host] FINISH getStats return={'status': {'message': 'Done', 'code': 0}, 'info': {'cpuStatistics': {'24': {'cpuUser': '0.46', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.41'}, '25': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '26': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '27': {'cpuUser': '1.66', 'nodeIndex': 1, 'cpuSys': '0.40', 'cpuIdle': '97.94'}, '20': {'cpuUser': '1.66', 'nodeIndex': 0, 'cpuSys': '0.86', 'cpuIdle': '97.48'}, '21': {'cpuUser': '0.73', 'nodeIndex': 1, 'cpuSys': '0.40', 'cpuIdle': '98.87'}, '22': {'cpuUser': '0.93', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '98.80'}, '23': {'cpuUser': '0.93', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '98.94'}, '28': {'cpuUser': '1.00', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '98.73'}, '29': {'cpuUser': '0.60', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.27'}, '1': {'cpuUser': '4.31', 'nodeIndex': 1, 'cpuSys': '4.05', 'cpuIdle': '91.64'}, '0': {'cpuUser': '0.60', 'nodeIndex': 0, 'cpuSys': '0.40', 'cpuIdle': '99.00'}, '3': {'cpuUser': '0.27', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.66'}, '2': {'cpuUser': '7.23', 'nodeIndex': 0, 'cpuSys': '1.06', 'cpuIdle': '91.71'}, '5': {'cpuUser': '0.20', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.67'}, '4': {'cpuUser': '1.99', 'nodeIndex': 0, 'cpuSys': '0.86', 'cpuIdle': '97.15'}, '7': {'cpuUser': '0.27', 'nodeIndex': 1, 'cpuSys': '0.27', 'cpuIdle': '99.46'}, '6': {'cpuUser': '1.13', 'nodeIndex': 0, 'cpuSys': '0.60', 'cpuIdle': '98.27'}, '9': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.80'}, '8': {'cpuUser': '0.80', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '98.87'}, '11': {'cpuUser': '0.66', 'nodeIndex': 1, 'cpuSys': '0.40', 'cpuIdle': '98.94'}, '10': {'cpuUser': '0.33', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '99.40'}, '13': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '12': {'cpuUser': '1.33', 'nodeIndex': 0, 'cpuSys': '0.46', 'cpuIdle': '98.21'}, '15': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '14': {'cpuUser': '0.46', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '99.27'}, '17': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '16': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '19': {'cpuUser': '6.77', 'nodeIndex': 1, 'cpuSys': '1.59', 'cpuIdle': '91.64'}, '18': {'cpuUser': '14.40', 'nodeIndex': 0, 'cpuSys': '3.65', 'cpuIdle': '81.95'}, '31': {'cpuUser': '0.46', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.41'}, '30': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31468'}, '0': {'memPercent': 7, 'memFree': '30418'}}, 'memShared': 0, 'haScore': 3400, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'vmCount': 0, 'memUsed': '3', 'storageDomains': {u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000991915', 'lastCheck': '5.0', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000985319', 'lastCheck': '5.1', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.00093201', 'lastCheck': '4.8', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'idrac': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'idrac', 'tx': '180', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '0', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '0', 'state': 'down'}, 'ovirtmgmt': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '1075063579', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '43586979921', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '618221192', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '618221192', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '0', 'state': 'down'}, 'em4': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em4', 'tx': '192', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034545.075349, 'rx': '38652218', 'state': 'up'}, 'br-int': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'br-int', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '0', 'state': 'down'}, 'em1': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em1', 'tx': '1115671653', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034545.075349, 'rx': '45377642078', 'state': 'up'}, 'genev_sys_6081': {'rxErrors': '0', 'txErrors': '8', 'speed': '1000', 'rxDropped': '0', 'name': 'genev_sys_6081', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '0', 'state': 'up'}, 'em3': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em3', 'tx': '192', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034545.075349, 'rx': '38652218', 'state': 'up'}, 'em2': {'rxErrors': '0', 'txErrors': '0', 'speed': '100', 'rxDropped': '0', 'name': 'em2', 'tx': '0', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034545.075349, 'rx': '16049684', 'state': 'up'}}, 'txDropped': '0', 'anonHugePages': '320', 'ksmPages': 100, 'elapsedTime': '223917.96', 'cpuLoad': '0.13', 'cpuSys': '0.55', 'diskStats': {'/var/log': {'free': '136515'}, '/var/run/vdsm/': {'free': '31856'}, '/tmp': {'free': '136515'}}, 'cpuUserVdsmd': '4.25', 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 63032, 'bootTime': '1567810572', 'haStats': {'active': True, 'configured': True, 'score': 3400, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'multipathHealth': {}, 'rxDropped': '0', 'outgoingVmMigrations': 0, 'swapTotal': 0, 'swapFree': 0, 'hugepages': defaultdict(<type 'dict'>, {1048576: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}, 2048: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}), 'dateTime': '2019-09-09T13:09:05 GMT', 'cpuUser': '1.56', 'memFree': 62776, 'cpuIdle': '97.89', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd': '1.99'}} from=::1,42394 (api:54) 2019-09-09 09:09:05,115-0400 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.09 seconds (__init__:312)

2 2

ovirt-engine-extension-aaa-ldap-setup
by Rick A 09 Sep '19

09 Sep '19

Trying to get oVirt to use LDAP for user authentication. It seems to bind, but when I test it (Login Flow and Search) before applying the settings, I get the message below (I can confirm the credentials ARE valid). I also tried to set it up manually and still get the same result. I'm using the same information on Bugzilla and it works fine. I've seen similar threads here but without much answers. Anyone have any good links or docs I can view to get this set up? oVirt Node Version: 4.3.3.6-1.el7 API: <--Authn.InvokeCommands.AUTHENTICATE_CREDENTIALS profile='mydomain.com' result=CREDENTIALS_INVALID

3 5

network config change
by mailing-ovirt＠qic.ca 09 Sep '19

09 Sep '19

Greetings, So we have a single hosted engine ovirt box deployed. When this was deployed, the network requirements were poorly determined and there was no port teaming or any vlan configuration. now we need to change that. I have searched all over for any form of procedure or guide as to how we do this. in summary: I need to go from a setup that uses em1 as it's default device and gateway to a setup where em1 and 3 are bonded into bond0 and now use that instead. As for the vlan we could live with keeping the native vlan configured on the cisco side for management and only add extra ones for VMs. any assistance as to what I need to go through from a bash shell would be appreciated. thanks Simon

3 3

VM cant shutdown successfully,It's status always in the way to shutdown
by zhouhao＠vip.friendtimes.net 09 Sep '19

09 Sep '19

Sometimes on the engine website ,the vm's status always in the way to shutdown ,even if I wait for it for hours; It either failed or is shutting down and the "power off" cant shutdown the vm too. zhouhao(a)vip.friendtimes.net

2 2

Failure while import VM from export domain
by Timmi 08 Sep '19

08 Sep '19

Hi oVirt list, I'm planing to migrate my oVirt installation on a new HW. My current platform is running a 4.2.8. The new one a 4.3.5. I'm currently trying to import the VMs through the export domain. But some of my old VMs are failing. Which log file would be interesting to check? Also maybe there is a better way to copy more less all VMs from my current oVirt to the new one. Best regards Christoph

2 2

HP NetXen Incorporated NX3031 driver
by Leo David 07 Sep '19

07 Sep '19

Hi everyone, I have this bloody card installed on a node, and it seems that no driver can be loaded for it, "ip link sh" does not show it. Any recommendation about installing this network card on oVirt 4.2.8 ? lspci -v .... 05:00.0 Ethernet controller: NetXen Incorporated NX3031 Multifunction 1/10-Gigabit Server Adapter (rev 42) Subsystem: Hewlett-Packard Company NC522SFP Dual Port 10GbE Server Adapter Flags: fast devsel, IRQ 31 Memory at d9c00000 (64-bit, non-prefetchable) [size=2M] Memory at da000000 (64-bit, non-prefetchable) [size=32M] Expansion ROM at d8000000 [disabled] [size=64K] Capabilities: [40] MSI-X: Enable- Count=64 Masked- Capabilities: [80] Power Management version 3 Capabilities: [a0] MSI: Enable- Count=1/32 Maskable- 64bit+ Capabilities: [c0] Express Endpoint, MSI 00 Capabilities: [100] Advanced Error Reporting Capabilities: [140] Device Serial Number 59-69-46-61-6e-48-73-75 Kernel modules: netxen_nic 05:00.1 Ethernet controller: NetXen Incorporated NX3031 Multifunction 1/10-Gigabit Server Adapter (rev 42) Subsystem: Hewlett-Packard Company NC522SFP Dual Port 10GbE Server Adapter Flags: fast devsel, IRQ 33 Memory at d9e00000 (64-bit, non-prefetchable) [size=2M] Memory at dc000000 (64-bit, non-prefetchable) [size=32M] Capabilities: [40] MSI-X: Enable- Count=64 Masked- Capabilities: [80] Power Management version 3 Capabilities: [a0] MSI: Enable- Count=1/32 Maskable- 64bit+ Capabilities: [c0] Express Endpoint, MSI 00 Capabilities: [100] Advanced Error Reporting Capabilities: [140] Device Serial Number 59-69-46-61-6e-48-73-75 Kernel modules: netxen_nic dmesg | grep netxen [ 2.188078] netxen_nic 0000:05:00.0: 2MB memory map [ 2.188371] netxen_nic 0000:05:00.0: Timeout reached waiting for rom done [ 2.188442] netxen_nic 0000:05:00.0: Error getting board config info. [ 2.189488] netxen_nic: probe of 0000:05:00.0 failed with error -5 [ 2.190438] netxen_nic 0000:05:00.1: 2MB memory map [ 2.190741] netxen_nic 0000:05:00.1: Timeout reached waiting for rom done [ 2.190808] netxen_nic 0000:05:00.1: Error getting board config info. [ 2.190964] netxen_nic: probe of 0000:05:00.1 failed with error -5 ... Thank you very much ! Leo -- Best regards, Leo David

2 1

Windows server std 2019 64 bit support
by Alex K 06 Sep '19

06 Sep '19

Hi all, I am trying to find out if Windows server std 2019 64 bit is supported at ovirt 3.4. I have checked the following links: https://www.ovirt.org/documentation/vmm-guide/chap-Introduction.html https://access.redhat.com/articles/973133 https://www.windowsservercatalog.com/results.aspx?&bCatID=1521&cpID=24121&a… From the above I deduce that RHEV 4.3.5 supports it and ovirt does not include it in the supported list of guest OS. Also KVM on CentOS 7.7 seems to be supporting it as RHEL 7.7 is listed as supported platform. Have you tried to load Windows server std 2019 64 bit successfullyon ovirt 3.4? Thank you.

2 2

ovirt-guest: Network device hanging shortly after startup (after Upgrade to 4.3.5)
by julius.kempa＠astelon.ch 06 Sep '19

06 Sep '19

We have a some old guests (SLES 10.4) which has been worked corectly till 4.2.7 (Manager + Host) After Upgrade to 4.3.5, we have got a network issue, that the gues network device, not depending on used device type, stopts to work after some seconds.... after about 15-20 pings device is hanging. Sometimes, in the first time after the device hungs up, the message "no buffer space available" comes from ping. Does someone knows a solution for it? greetings Julius

3 3

[ANN] oVirt 4.3.6 Fifth Release Candidate is now available for testing
by Sandro Bonazzola 05 Sep '19

05 Sep '19

The oVirt Project is pleased to announce the availability of the oVirt 4.3.6 Fifth Release Candidate for testing, as of September 5th, 2019. This update is a release candidate of the sixth in a series of stabilization updates to the 4.3 series. This is pre-release software. This pre-release should not to be used in production. This release is available now on x86_64 architecture for: * Red Hat Enterprise Linux 7.7 or later (but <8) * CentOS Linux (or similar) 7.7 or later (but <8) This release supports Hypervisor Hosts on x86_64 and ppc64le architectures for: * Red Hat Enterprise Linux 7.7 or later (but <8) * CentOS Linux (or similar) 7.7 or later (but <8) * oVirt Node 4.3 (available for x86_64 only) has been built consuming CentOS 7.7 Candidate Release See the release notes [1] for known issues, new features and bugs fixed. Notes: - oVirt Appliance is already available - oVirt Node is already available based on CentOS 7.7 Candidate Release Additional Resources: * Read more about the oVirt 4.3.6 release highlights: http://www.ovirt.org/release/4.3.6/ * Get more oVirt Project updates on Twitter: https://twitter.com/ovirt * Check out the latest project news on the oVirt blog: http://www.ovirt.org/blog/ [1] http://www.ovirt.org/release/4.3.6/ [2] http://resources.ovirt.org/pub/ovirt-4.3-pre/iso/ -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/>*Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*

1 0

VM crushed and cant shutdown successfully,I shoud to reboot the ovirt-node machine
by zhouhao＠vip.friendtimes.net 05 Sep '19

05 Sep '19

VM crushed and cant shutdown successfully,It always on the way to shutdown，even if I wait it for 10 hours；Everytimes I shoud to reboot the ovirt-node machine This comes up 4 times, It appes both on the ovirt-noe 4.3 and 4.2; This bug appers when my VM is running at high load（seems memory and disk's w/r） My storage domain is local on host; When the bug appers,it could cost my node's io,looks like the picture below；But sometimes it will not The vm's threading on the ovirt-node, the IO ratio is 100% Everytime the vm's process will change to defunct I can not kill it,I had to shutdown the ovirt-node on the engine website ,the vm's status always in the way to shutdown ,even if I wait for it for hours; It either failed or is shutting down and the "power off" cant shutdown the vm too. the vm's error below The ovirt-node's error below ---------------------------------------------------------------- The other infomation about my ovirt-node node-version： node hardware zhouhao(a)vip.friendtimes.net

1 0

Re: [ANN] oVirt 4.3.6 Fourth Release Candidate is now available for testing
by Strahil 05 Sep '19

05 Sep '19

Hi Sandro, Can the update work with 7.6 ? I'm asking because CentOS 7.7 is still building and unavailable. Best Regards, Strahil NikolovOn Aug 29, 2019 10:47, Sandro Bonazzola <sbonazzo(a)redhat.com> wrote: > > The oVirt Project is pleased to announce the availability of the oVirt 4.3.6 Fourth Release Candidate for testing, as of August 29th, 2019. > > This update is a release candidate of the sixth in a series of stabilization updates to the 4.3 series. > This is pre-release software. This pre-release should not to be used in production. > > This release is available now on x86_64 architecture for: > * Red Hat Enterprise Linux 7.7 or later (but <8) > * CentOS Linux (or similar) 7.7 or later (but <8) > > This release supports Hypervisor Hosts on x86_64 and ppc64le architectures for: > * Red Hat Enterprise Linux 7.7 or later (but <8) > * CentOS Linux (or similar) 7.7 or later (but <8) > * oVirt Node 4.3 (available for x86_64 only) will be made available when CentOS 7.7 will be released. > > See the release notes [1] for known issues, new features and bugs fixed. > > Notes: > - oVirt Appliance is already available > - oVirt Node is not yet available, pending CentOS 7.7 release to be available > > Additional Resources: > * Read more about the oVirt 4.3.6 release highlights: http://www.ovirt.org/release/4.3.6/ > * Get more oVirt Project updates on Twitter: https://twitter.com/ovirt > * Check out the latest project news on the oVirt blog:http://www.ovirt.org/blog/ > > [1] http://www.ovirt.org/release/4.3.6/ > [2] http://resources.ovirt.org/pub/ovirt-4.3-pre/iso/ > > -- > > Sandro Bonazzola > > MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV > > Red Hat EMEA > > sbonazzo(a)redhat.com > > Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.

2 1

Re: Any real experiences with nested virtualisation?
by Strahil 05 Sep '19

05 Sep '19

I guess you want to do something like this one : http://www.chrisj.cloud/?q=node/8 As Ovirt is upstream of RHV , I believe that RDO and Ovirt have almost the same integration as in the post. Still,such setup requires a lot of knowledge about KVM, oVirt and OpenStack. Best Regards, Strahil NikolovOn Sep 4, 2019 21:36, thomas(a)hoberg.net wrote: > > First, it's a scenario the oVirt team seem to actually embrace themselves. > > I believe I have seen that type of deployment mentioned either in one of the forward looking blog posts or Red Hat summit videos. > > And then it simply makes a lot of sense to have those crucial management VMs that control the swarm of resources in a cloud run under a fault-tolerant VM management environment such as oVirt. > > As for nested virtualization, I can only provide some negative experience with oVirt. > > I've done nested virtualization with VMware, running ESX on VMWare Workstation and then some VMs underneath that ESX successfully. > > When I tried to do the same with oVirt, running an oVirt hyperconverged cluster as VMs on VMware workstation, I got reasonably far, but then every VM I tried to launch nested would just stop right at boot. > > It first happened with the hosted-engine, as that tries to start as a VM, but it happened more obviously when I tried to migrate VMs (just a plain Fedora 30 in this case) from a physical compute node running oVirt nodeOS to a virtualized compute node, running the same current oVirt nodeOS image on VMware Workstation (Windows 2016 host, if that matters). > > The machine would start the live-migration properly and then just freeze and stay frozen until I migrated it back or indeed to any other non-virtualized node. > > I haven't as yet tried to do that with nested KVM, which I believe might actually work: But it's on my things to test, so I'll report back, once I get around it. > > Actually I am wondering, a) how deep this nesting would be allowed to go and b) if the 'leaf' nodes at the last nesting level should actually have nesting support activated in their kernel parameters... > > Any insight here would be well appreciated. > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/6YDBQGIXUWTDB…

1 0

Any real experiences with nested virtualisation?
by tonyppe＠gmail.com 04 Sep '19

04 Sep '19

I have ovirt 4.3.5 and Openstack hosts. There are 2 x openstack compute and 2 x ovirt hosts with hosted manager. I think i can simplify the management of this if I were to virtualise the openstack compute nodes as VMs on ovirt. I wanted to reach out here to gauge any other users experiences with it if at all. From the research I have done it looks like from the VM perspective, migration must not be allowed. Any info / comments / feedback appreciated. Regards,

2 1

Re: Cannot Increase Hosted Engine VM Memory
by Dionysis K 04 Sep '19

04 Sep '19

Now i noticed another strange i deactivated the memory balloon option from the VM hosted engine and tried to update the values again with the same memory it still keeps the value of memory size at 5120 while the value of physical memory guaranteed it changed to what we desire at the 8192 value and then the error appears again in the vm engine i ckecked the engine log file it shows the same errors! is there are other logfile locations that can give us any clues ?

1 0

Re: Cannot Increase Hosted Engine VM Memory
by Douglas Duckworth 04 Sep '19

04 Sep '19

Yes, I do. Gold crown indeed. It's the "HostedEngine" as seen attached! Thanks, Douglas Duckworth, MSc, LFCS HPC System Administrator Scientific Computing Unit<https://scu.med.cornell.edu> Weill Cornell Medicine 1300 York Avenue New York, NY 10065 E: doug(a)med.cornell.edu<mailto:doug@med.cornell.edu> O: 212-746-6305 F: 212-746-8690 On Wed, Jan 23, 2019 at 12:02 PM Simone Tiraboschi <stirabos(a)redhat.com<mailto:stirabos@redhat.com>> wrote: On Wed, Jan 23, 2019 at 5:51 PM Douglas Duckworth <dod2014(a)med.cornell.edu<mailto:dod2014@med.cornell.edu>> wrote: Hi Simone Can I get help with this issue? Still cannot increase memory for Hosted Engine. From the logs it seams that the engine is trying to hotplug memory to the engine VM which is something it should not happen. The engine should simply update engine VM configuration in the OVF_STORE and require a reboot of the engine VM. Quick question, in the VM panel do you see a gold crown symbol on the Engine VM? Thanks, Douglas Duckworth, MSc, LFCS HPC System Administrator Scientific Computing Unit<https://scu.med.cornell.edu> Weill Cornell Medicine 1300 York Avenue New York, NY 10065 E: doug(a)med.cornell.edu<mailto:doug@med.cornell.edu> O: 212-746-6305 F: 212-746-8690 On Thu, Jan 17, 2019 at 8:08 AM Douglas Duckworth <dod2014(a)med.cornell.edu<mailto:dod2014@med.cornell.edu>> wrote: Sure, they're attached. In "first attempt" the error seems to be: 2019-01-17 07:49:24,795-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-29) [680f82b3-7612-4d91-afdc-43937aa298a2] EVENT_ID: FAILED_HOT_SET_MEMORY_NOT_DIVIDABLE(2,048), Failed to hot plug memory to VM HostedEngine. Amount of added memory (4000MiB) is not dividable by 256MiB. Followed by: 2019-01-17 07:49:24,814-05 WARN [org.ovirt.engine.core.bll.UpdateRngDeviceCommand] (default task-29) [26f5f3ed] Validation of action 'UpdateRngDevice' failed for user admin@internal-authz. Reasons: ACTION_TYPE_FAILED_VM_IS_RUNNING 2019-01-17 07:49:24,815-05 ERROR [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-29) [26f5f3ed] Updating RNG device of VM HostedEngine (adf14389-1563-4b1a-9af6-4b40370a825b) failed. Old RNG device = VmRngDevice:{id='VmDeviceId:{deviceId='6435b2b5-163c-4f0c-934e-7994da60dc89', vmId='adf14389-1563-4b1a-9af6-4b40370a825b'}', device='virtio', type='RNG', specParams='[source=urandom]', address='', managed='true', plugged='true', readOnly='false', deviceAlias='', customProperties='null', snapshotId='null', logicalName='null', hostDevice='null'}. New RNG device = VmRngDevice:{id='VmDeviceId:{deviceId='6435b2b5-163c-4f0c-934e-7994da60dc89', vmId='adf14389-1563-4b1a-9af6-4b40370a825b'}', device='virtio', type='RNG', specParams='[source=urandom]', address='', managed='true', plugged='true', readOnly='false', deviceAlias='', customProperties='null', snapshotId='null', logicalName='null', hostDevice='null'}. In "second attempt" I used values that are dividable by 256 MiB so that's no longer present. Though same error: 2019-01-17 07:56:59,795-05 INFO [org.ovirt.engine.core.vdsbroker.SetAmountOfMemoryVDSCommand] (default task-22) [7059a48f] START, SetAmountOfMemoryVDSCommand(HostName = ovirt-hv1.med.cornell.edu<http://ovirt-hv1.med.cornell.edu>, Params:{hostId='cdd5ffda-95c7-4ffa-ae40-be66f1d15c30', vmId='adf14389-1563-4b1a-9af6-4b40370a825b', memoryDevice='VmDevice:{id='VmDeviceId:{deviceId='7f7d97cc-c273-4033-af53-bc9033ea3abe', vmId='adf14389-1563-4b1a-9af6-4b40370a825b'}', device='memory', type='MEMORY', specParams='[node=0, size=2048]', address='', managed='true', plugged='true', readOnly='false', deviceAlias='', customProperties='null', snapshotId='null', logicalName='null', hostDevice='null'}', minAllocatedMem='6144'}), log id: 50873daa 2019-01-17 07:56:59,855-05 INFO [org.ovirt.engine.core.vdsbroker.SetAmountOfMemoryVDSCommand] (default task-22) [7059a48f] FINISH, SetAmountOfMemoryVDSCommand, log id: 50873daa 2019-01-17 07:56:59,862-05 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-22) [7059a48f] EVENT_ID: HOT_SET_MEMORY(2,039), Hotset memory: changed the amount of memory on VM HostedEngine from 4096 to 4096 2019-01-17 07:56:59,881-05 WARN [org.ovirt.engine.core.bll.UpdateRngDeviceCommand] (default task-22) [28fd4c82] Validation of action 'UpdateRngDevice' failed for user admin@internal-authz. Reasons: ACTION_TYPE_FAILED_VM_IS_RUNNING 2019-01-17 07:56:59,882-05 ERROR [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-22) [28fd4c82] Updating RNG device of VM HostedEngine (adf14389-1563-4b1a-9af6-4b40370a825b) failed. Old RNG device = VmRngDevice:{id='VmDeviceId:{deviceId='6435b2b5-163c-4f0c-934e-7994da60dc89', vmId='adf14389-1563-4b1a-9af6-4b40370a825b'}', device='virtio', type='RNG', specParams='[source=urandom]', address='', managed='true', plugged='true', readOnly='false', deviceAlias='', customProperties='null', snapshotId='null', logicalName='null', hostDevice='null'}. New RNG device = VmRngDevice:{id='VmDeviceId:{deviceId='6435b2b5-163c-4f0c-934e-7994da60dc89', vmId='adf14389-1563-4b1a-9af6-4b40370a825b'}', device='virtio', type='RNG', specParams='[source=urandom]', address='', managed='true', plugged='true', readOnly='false', deviceAlias='', customProperties='null', snapshotId='null', logicalName='null', hostDevice='null'}. This message repeats throughout engine.log: 2019-01-17 07:55:43,270-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-89) [] EVENT_ID: VM_MEMORY_UNDER_GUARANTEED_VALUE(148), VM HostedEngine on host ovirt-hv1.med.cornell.edu<http://ovirt-hv1.med.cornell.edu> was guaranteed 8192 MB but currently has 4224 MB As you can see attached the host has plenty of memory. Thank you Simone! Thanks, Douglas Duckworth, MSc, LFCS HPC System Administrator Scientific Computing Unit<https://scu.med.cornell.edu> Weill Cornell Medicine 1300 York Avenue New York, NY 10065 E: doug(a)med.cornell.edu<mailto:doug@med.cornell.edu> O: 212-746-6305 F: 212-746-8690 On Thu, Jan 17, 2019 at 5:09 AM Simone Tiraboschi <stirabos(a)redhat.com<mailto:stirabos@redhat.com>> wrote: On Wed, Jan 16, 2019 at 8:22 PM Douglas Duckworth <dod2014(a)med.cornell.edu<mailto:dod2014@med.cornell.edu>> wrote: Sorry for accidental send. Anyway I try to increase physical memory however it won't go above 4096MB. The hypervisor has 64GB. Do I need to modify this value with Hosted Engine offline? No, it's not required. Can you please attach your engine.log for the relevant time frame? Thanks, Douglas Duckworth, MSc, LFCS HPC System Administrator Scientific Computing Unit<https://scu.med.cornell.edu> Weill Cornell Medicine 1300 York Avenue New York, NY 10065 E: doug(a)med.cornell.edu<mailto:doug@med.cornell.edu> O: 212-746-6305 F: 212-746-8690 On Wed, Jan 16, 2019 at 1:58 PM Douglas Duckworth <dod2014(a)med.cornell.edu<mailto:dod2014@med.cornell.edu>> wrote: Hello I am trying to increase Hosted Engine physical memory above 4GB Thanks, Douglas Duckworth, MSc, LFCS HPC System Administrator Scientific Computing Unit<https://scu.med.cornell.edu> Weill Cornell Medicine 1300 York Avenue New York, NY 10065 E: doug(a)med.cornell.edu<mailto:doug@med.cornell.edu> O: 212-746-6305 F: 212-746-8690 _______________________________________________ Users mailing list -- users(a)ovirt.org<mailto:users@ovirt.org> To unsubscribe send an email to users-leave(a)ovirt.org<mailto:users-leave@ovirt.org> Privacy Statement: https://www.ovirt.org/site/privacy-policy/<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ovirt.org_site_pri…> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ovirt.org_communit…> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/WGSXQVVPJJ2CR…<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.ovirt.org_archiv…>

3 8

iSCSI Multipath/multiple gateways
by dan.poltawski＠tnp.net.uk 04 Sep '19

04 Sep '19

My iSCSI target (ceph-based) has multiple gateways, I’d like to ensure my hosted storage is aware of these such that each gateway can be rebooted for maintance without impacting service. What is the appropriate way to configure this so that each host knows of the multipath configuration? I do not have multiple network paths, just multiple gateways on the same network, so I think the multipath UI tool isn’t the way to do this. Any help/docs pointers appreciated. Regards, Dan

3 2

the virtual machine crushed and I cant shutdown the vm successfully
by zhouhao＠vip.friendtimes.net 04 Sep '19

04 Sep '19

Dear ovirt developers: The trouble bothered me for a long time;Some times my vm will crush and I cant shutdown it,I had to reboot the ovirt-node to slove it; the vm's error below The ovirt-node's error below The vm's threading on the ovirt-node, the IO ratio is 100% and the vm's process change to defunct I can not kill it,every time I had to shutdown the ovirt-node on the engine website ,the vm's status always in the way to shutdown ,even if I wait for it for hours; It either failed or is shutting down and the "power off" cant shutdown the vm too. ---------------------------------------------------------------- The other infomation about my ovirt-node node-version： node hardware zhouhao(a)vip.friendtimes.net

2 2

the virtual machine crushed and I cant shutdown the vm successfully
by zhouhao＠vip.friendtimes.net 03 Sep '19

03 Sep '19

Dear ovirt developers: The trouble bothered me for a long time;Some times my vm will crush and I cant shutdown it,I had to reboot the ovirt-node to slove it; the vm's error below The ovirt-node's error below The vm's threading on the ovirt-node, the IO ratio is 100% and the vm's process change to defunct I can not kill it,every time I had to shutdown the ovirt-node on the engine website ,the vm's status always in the way to shutdown ,even if I wait for it for hours; It either failed or is shutting down and the "power off" cant shutdown the vm too. ---------------------------------------------------------------- The other infomation about my ovirt-node node-version： node hardware zhouhao(a)vip.friendtimes.net

1 0

the virtual machine crushed and I cant shutdown the vm successfully
by zhouhao＠vip.friendtimes.net 03 Sep '19

03 Sep '19

Dear ovirt developers: The trouble bothered me for a long time;Some times my vm will crush and I cant shutdown it,I had to reboot the ovirt-node to slove it; the vm's error below The ovirt-node's error below The vm's threading on the ovirt-node, the IO ratio is 100% and the vm's process change to defunct I can not kill it,every time I had to shutdown the ovirt-node on the engine website ,the vm's status always in the way to shutdown ,even if I wait for it for hours; It either failed or is shutting down and the "power off" cant shutdown the vm too. ---------------------------------------------------------------- The other infomation about my ovirt-node node-version： node hardware zhouhao(a)vip.friendtimes.net

1 0

iSCSI Multipath/multiple gateways
by Dan Poltawski 03 Sep '19

03 Sep '19

My iSCSI target (ceph-based) has multiple gateways, I’d like to ensure my hosted storage is aware of these such that each gateway can be rebooted for maintance without impacting service. What is the appropriate way to configure this so that each host knows of the multipath configuration? I do not have multiple network paths, just multiple gateways on the same network, so I think the multipath UI tool isn’t the way to do this. Any help/docs pointers appreciated. Regards, Dan ________________________________ The Networking People (TNP) Limited. Registered office: Network House, Caton Rd, Lancaster, LA1 3PE. Registered in England & Wales with company number: 07667393 This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.

1 0

engine.log flooded with "Field 'foo' can not be updated when status is 'Up'"
by Matthias Leopold 02 Sep '19

02 Sep '19

Hi, I updated my production oVirt environment from 4.3.3 to 4.3.5 today. Everything went fine so far, but there's one annoying phenomenon: When I log into the "Administration Portal" and request the VM list ("/ovirt-engine/webadmin/?locale=en_US#vms") engine.log is flooded with lines like WARN [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-10618) [54d8c375-aa72-42f8-876e-8777d9d1a08a] Field 'balloonEnabled' can not be updated when status is 'Up' "Field", task and UUID vary and the flood stops after a while. Also listing or trying to edit other entities seems to trigger this "storm" or loop over and over again to a point that log file size is becoming an issue and interface is becoming sluggish. I can also see that CPU usage of engine java process goes up. When I log out everything is quiet and "VM Portal" is not affected at all. I have seen lines like that before and know that they are usually OK (when changing VM properties), but these logs used to be linked to singular events. I suspect that the present behaviour might be linked to VMs that have "Pending Virtual Machine changes", which are in most cases "Custom Compatibility Version" changes that still stem from the upgrade to Cluster Version 4.3. I can't be sure and I can't resolve all these pending changes now, but these should not be causing such annoying behaviour in the first place. I resorted to setting engine log level to "ERROR" right now to at least stop the log file from growing, but this not a solution. I can still see CPU load going up when using the interface. I very much hope that someone can explain whats happening and tell me how to resolve this. thanks a lot Matthias

4 8

Intel Xeon X5460
by Marcello Gentile 01 Sep '19

01 Sep '19

Is this procesor supported? https://ark.intel.com/content/www/us/en/ark/products/33087/intel-xeon-proce… Seems to meet all the requirements but when I install the self hosted engine it fails with the error message below: [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "The host has been set in non_operational status, deployment errors: code 156: Host Ovirt-Node1.adsdata.ca<http://Ovirt-Node1.adsdata.ca> moved to Non-Operational state as host CPU type is not supported in this cluster compatibility version or is not supported at all, code 9000: Failed to verify Power Management configuration for Host Ovirt-Node1.adsdata.ca<http://Ovirt-Node1.adsdata.ca>., fix accordingly and re-deploy."} Thanks for any help

3 3

How to modify DC's default route setting?
by gotoh＠eis.co.jp 01 Sep '19

01 Sep '19

We have to change default route on a production Ovirt system ( ovirt4.3 : engine + 9 hosts + glusterFS subsystem, approx 90 virtual machines.) , due to network enhancement We changed engine's default gateway and could permanently change host's default route by modifying the persistent setting. However, DC's logical network's default route remains unchanged. Consequently, the logical network became out of sync. We would be glad if anyone could instruct us how to change logical network's default route of DC. Thanks in advance, Hideo Goto

1 0

Error during self-hosted install using cockpit
by Marcello Gentile 01 Sep '19

01 Sep '19

Install fails with ERROR ansible failed {'status': 'FAILED', 'ansible_type': 'task', 'ansible_task': u'Fail with error description', 'ansible_result': u"type: <type 'dict'>\nstr: {'msg': u'The host has been set in non_operational status, deployment errors: code 156: Host Ovirt-Node1.adsdata.ca<http://Ovirt-Node1.adsdata.ca> moved to Non-Operational state as host CPU type is not supported in this cluster compatibility version or is not supported at all, code 9000: Failed to verify Power Management con", 'task_duration': 1, 'ansible_host': u'localhost', 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml'}

2 1

Engine died while creating VM
by Erick Perez - Quadrian Enterprises 31 Aug '19

31 Aug '19

Hi, I was just creating a VM on a freshly installed 2-node selfhosted ovirt setup with a NFS backend. NFS connections are v4.2 for VM data domain What other log is needed? Thanks, in advance. Here is the output of the last 300 lines of /var/log/messages [root@hvm002 ~]# [root@hvm002 ~]# tail /var/log/messages -n 300 Aug 26 04:34:56 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:34:56 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:34:56 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:35:06 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:35:06 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:35:06 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:35:06 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:35:07 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:35:07 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:35:07 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:35:07 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:35:07 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:35:07 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:35:08 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:35:15 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:35:15 hvm002 systemd: Starting Cockpit Web Service... Aug 26 04:35:15 hvm002 systemd: Started Cockpit Web Service. Aug 26 04:35:15 hvm002 cockpit-ws: Using certificate: /etc/cockpit/ws-certs.d/0-self-signed.cert Aug 26 04:35:15 hvm002 cockpit-ws: couldn't read from connection: Peer sent fatal TLS alert: Certificate is bad Aug 26 04:35:17 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:35:17 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:35:17 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:35:17 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:35:17 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:35:17 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:35:17 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:35:17 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:35:17 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:35:17 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:35:23 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:35:27 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:35:27 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:35:27 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:35:27 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:35:28 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:35:28 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:35:28 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:35:28 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:35:28 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:35:28 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:35:36 hvm002 vdsm[7369]: WARN Worker blocked: <Worker name=jsonrpc/5 running <Task <JsonRpcTask {'params': {u'connectionParams': [{u'protocol_version': u'auto', u'connection': u'10.10.10.2:/mnt/lv_vdo01', u'user': u'kvm', u'id': u'e29cf818-5ee5-46e1-85c1-8aeefa33e95d'}], u'storagepoolID': u'00000000-0000-0000-0000-000000000000', u'domainType': 1}, 'jsonrpc': '2.0', 'method': u'StoragePool.connectStorageServer', 'id': u'767bed1a-dc31-44f6-98a8-94faebc82345'} at 0x7f08f83cadd0> timeout=60, duration=60.00 at 0x7f091c0aadd0> task#=4 at 0x7f091c0c6e10>, traceback:#012File: "/usr/lib64/python2.7/threading.py", line 785, in __bootstrap#012 self.__bootstrap_inner()#012File: "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner#012 self.run()#012File: "/usr/lib64/python2.7/threading.py", line 765, in run#012 self.__target(*self.__args, **self.__kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/common/concurrent.py", line 195, in run#012 ret = func(*args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 301, in _run#012 self._execute_task()#012File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 315, in _execute_task#012 task()#012File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 391, in __call__#012 self._callable()#012File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 262, in __call__#012 self._handler(self._ctx, self._req)#012File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 305, in _serveRequest#012 response = self._handle_request(req, ctx)#012File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 345, in _handle_request#012 res = method(**params)#012File: "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 198, in _dynamicMethod#012 result = fn(*methodArgs)#012File: "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1102, in connectStorageServer#012 connectionParams)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/dispatcher.py", line 74, in wrapper#012 result = ctask.prepare(func, *args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 108, in wrapper#012 return m(self, *a, **kw)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 1179, in prepare#012 result = self._run(func, *args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run#012 return fn(*args, **kargs)#012File: "<string>", line 2, in connectStorageServer#012File: "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in method#012 ret = func(*args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 2416, in connectStorageServer#012 conObj.connect()#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/storageServer.py", line 431, in connect#012 return self._mountCon.connect()#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/storageServer.py", line 172, in connect#012 self._mount.mount(self.options, self._vfsType, cgroup=self.CGROUP)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/mount.py", line 207, in mount#012 cgroup=cgroup)#012File: "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 56, in __call__#012 return callMethod()#012File: "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 54, in <lambda>#012 **kwargs)#012File: "<string>", line 2, in mount#012File: "/usr/lib64/python2.7/multiprocessing/managers.py", line 759, in _callmethod#012 kind, result = conn.recv() Aug 26 04:35:38 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:35:38 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:35:38 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:35:38 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:35:38 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:35:38 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:35:38 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:35:38 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:35:38 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:35:38 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:35:38 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:35:48 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:35:48 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:35:48 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:35:48 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:35:49 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:35:49 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:35:49 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:35:49 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:35:49 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:35:49 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:35:52 hvm002 systemd: Created slice User Slice of root. Aug 26 04:35:52 hvm002 systemd-logind: New session 2 of user root. Aug 26 04:35:52 hvm002 systemd: Started Session 2 of user root. Aug 26 04:35:53 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:35:59 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:35:59 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:35:59 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:35:59 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:35:59 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:35:59 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:35:59 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:35:59 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:35:59 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:35:59 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:36:08 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:36:09 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:36:09 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:36:09 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:36:09 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:36:10 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:36:10 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:36:10 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:36:10 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:36:10 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:36:10 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:36:20 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:36:20 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:36:20 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:36:20 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:36:20 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:36:20 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:36:20 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:36:20 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:36:20 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:36:20 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:36:23 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:36:30 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:36:30 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:36:30 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:36:30 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:36:31 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:36:31 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:36:31 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:36:31 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:36:31 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:36:31 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:36:36 hvm002 vdsm[7369]: WARN Worker blocked: <Worker name=jsonrpc/5 running <Task <JsonRpcTask {'params': {u'connectionParams': [{u'protocol_version': u'auto', u'connection': u'10.10.10.2:/mnt/lv_vdo01', u'user': u'kvm', u'id': u'e29cf818-5ee5-46e1-85c1-8aeefa33e95d'}], u'storagepoolID': u'00000000-0000-0000-0000-000000000000', u'domainType': 1}, 'jsonrpc': '2.0', 'method': u'StoragePool.connectStorageServer', 'id': u'767bed1a-dc31-44f6-98a8-94faebc82345'} at 0x7f08f83cadd0> timeout=60, duration=120.00 at 0x7f091c0aadd0> task#=4 at 0x7f091c0c6e10>, traceback:#012File: "/usr/lib64/python2.7/threading.py", line 785, in __bootstrap#012 self.__bootstrap_inner()#012File: "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner#012 self.run()#012File: "/usr/lib64/python2.7/threading.py", line 765, in run#012 self.__target(*self.__args, **self.__kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/common/concurrent.py", line 195, in run#012 ret = func(*args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 301, in _run#012 self._execute_task()#012File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 315, in _execute_task#012 task()#012File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 391, in __call__#012 self._callable()#012File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 262, in __call__#012 self._handler(self._ctx, self._req)#012File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 305, in _serveRequest#012 response = self._handle_request(req, ctx)#012File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 345, in _handle_request#012 res = method(**params)#012File: "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 198, in _dynamicMethod#012 result = fn(*methodArgs)#012File: "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1102, in connectStorageServer#012 connectionParams)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/dispatcher.py", line 74, in wrapper#012 result = ctask.prepare(func, *args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 108, in wrapper#012 return m(self, *a, **kw)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 1179, in prepare#012 result = self._run(func, *args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run#012 return fn(*args, **kargs)#012File: "<string>", line 2, in connectStorageServer#012File: "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in method#012 ret = func(*args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 2416, in connectStorageServer#012 conObj.connect()#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/storageServer.py", line 431, in connect#012 return self._mountCon.connect()#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/storageServer.py", line 172, in connect#012 self._mount.mount(self.options, self._vfsType, cgroup=self.CGROUP)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/mount.py", line 207, in mount#012 cgroup=cgroup)#012File: "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 56, in __call__#012 return callMethod()#012File: "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 54, in <lambda>#012 **kwargs)#012File: "<string>", line 2, in mount#012File: "/usr/lib64/python2.7/multiprocessing/managers.py", line 759, in _callmethod#012 kind, result = conn.recv() Aug 26 04:36:38 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:36:41 hvm002 python: detected unhandled Python exception in '/usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker' Aug 26 04:36:41 hvm002 abrt-server: Duplicate: core backtrace Aug 26 04:36:41 hvm002 abrt-server: DUP_OF_DIR: /var/tmp/abrt/Python-2019-08-26-04:32:28-3432 Aug 26 04:36:41 hvm002 abrt-server: Deleting problem directory Python-2019-08-26-04:36:41-16985 (dup of Python-2019-08-26-04:32:28-3432) Aug 26 04:36:41 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:36:41 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:36:41 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:36:41 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:36:41 hvm002 dbus[3444]: [system] Activating service name='org.freedesktop.problems' (using servicehelper) Aug 26 04:36:41 hvm002 dbus[3444]: [system] Successfully activated service 'org.freedesktop.problems' Aug 26 04:36:41 hvm002 systemd: ovirt-ha-broker.service: main process exited, code=exited, status=1/FAILURE Aug 26 04:36:41 hvm002 systemd: Unit ovirt-ha-broker.service entered failed state. Aug 26 04:36:41 hvm002 systemd: ovirt-ha-broker.service failed. Aug 26 04:36:41 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:36:41 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:36:41 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:36:41 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:36:41 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:36:41 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:36:41 hvm002 systemd: ovirt-ha-broker.service holdoff time over, scheduling restart. Aug 26 04:36:41 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:36:41 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Communications Broker. Aug 26 04:36:41 hvm002 systemd: Started oVirt Hosted Engine High Availability Communications Broker. Aug 26 04:36:42 hvm002 systemd: Started Session c15 of user root. Aug 26 04:36:42 hvm002 systemd: Started Session c16 of user root. Aug 26 04:36:51 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:36:51 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:36:51 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:36:51 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:36:52 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:36:52 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:36:52 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:36:52 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:36:52 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:36:52 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:36:53 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:37:02 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:37:02 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:37:02 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:37:02 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:37:02 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:37:02 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:37:02 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:37:02 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:37:02 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:37:02 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:37:08 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:37:12 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:37:12 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:37:12 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:37:12 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:37:13 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:37:13 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:37:13 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:37:13 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:37:13 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:37:13 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:37:23 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:37:23 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:37:23 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:37:23 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:37:23 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:37:23 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:37:23 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:37:23 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:37:23 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:37:23 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:37:23 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:37:33 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:37:33 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:37:33 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:37:33 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:37:34 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:37:34 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:37:34 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:37:34 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:37:34 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:37:34 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:37:38 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:37:42 hvm002 vdsm[7369]: WARN Worker blocked: <Worker name=jsonrpc/3 running <Task <JsonRpcTask {'params': {u'connectionParams': [{u'protocol_version': u'auto', u'connection': u'10.10.10.2:/mnt/lv_vdo01', u'user': u'kvm', u'id': u'e29cf818-5ee5-46e1-85c1-8aeefa33e95d'}], u'storagepoolID': u'00000000-0000-0000-0000-000000000000', u'domainType': 1}, 'jsonrpc': '2.0', 'method': u'StoragePool.connectStorageServer', 'id': u'c8887c86-fb5e-4be1-9552-45620144b74e'} at 0x7f08f8259650> timeout=60, duration=60.00 at 0x7f08f8259750> task#=8 at 0x7f091c110c50>, traceback:#012File: "/usr/lib64/python2.7/threading.py", line 785, in __bootstrap#012 self.__bootstrap_inner()#012File: "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner#012 self.run()#012File: "/usr/lib64/python2.7/threading.py", line 765, in run#012 self.__target(*self.__args, **self.__kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/common/concurrent.py", line 195, in run#012 ret = func(*args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 301, in _run#012 self._execute_task()#012File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 315, in _execute_task#012 task()#012File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 391, in __call__#012 self._callable()#012File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 262, in __call__#012 self._handler(self._ctx, self._req)#012File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 305, in _serveRequest#012 response = self._handle_request(req, ctx)#012File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 345, in _handle_request#012 res = method(**params)#012File: "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 198, in _dynamicMethod#012 result = fn(*methodArgs)#012File: "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1102, in connectStorageServer#012 connectionParams)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/dispatcher.py", line 74, in wrapper#012 result = ctask.prepare(func, *args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 108, in wrapper#012 return m(self, *a, **kw)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 1179, in prepare#012 result = self._run(func, *args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run#012 return fn(*args, **kargs)#012File: "<string>", line 2, in connectStorageServer#012File: "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in method#012 ret = func(*args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 2416, in connectStorageServer#012 conObj.connect()#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/storageServer.py", line 431, in connect#012 return self._mountCon.connect()#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/storageServer.py", line 172, in connect#012 self._mount.mount(self.options, self._vfsType, cgroup=self.CGROUP)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/mount.py", line 207, in mount#012 cgroup=cgroup)#012File: "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 56, in __call__#012 return callMethod()#012File: "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 54, in <lambda>#012 **kwargs)#012File: "<string>", line 2, in mount#012File: "/usr/lib64/python2.7/multiprocessing/managers.py", line 759, in _callmethod#012 kind, result = conn.recv() Aug 26 04:37:44 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:37:44 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:37:44 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:37:44 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:37:44 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:37:44 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:37:44 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:37:44 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:37:44 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:37:44 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:37:53 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:37:54 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:37:54 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:37:54 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:37:54 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:37:55 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:37:55 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:37:55 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:37:55 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:37:55 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:37:55 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:38:05 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:38:05 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:38:05 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:38:05 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:38:05 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:38:05 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:38:05 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:38:05 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:38:05 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:38:05 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:38:08 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:38:15 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:38:15 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:38:15 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:38:15 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:38:16 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:38:16 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:38:16 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:38:16 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:38:16 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:38:16 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:38:23 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:38:26 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:38:26 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:38:26 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:38:26 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:38:26 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:38:26 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:38:26 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:38:26 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:38:26 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:38:26 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:38:36 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:38:36 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:38:36 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:38:36 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:38:37 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:38:37 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:38:37 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:38:37 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:38:37 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:38:37 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:38:38 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:38:42 hvm002 vdsm[7369]: WARN Worker blocked: <Worker name=jsonrpc/3 running <Task <JsonRpcTask {'params': {u'connectionParams': [{u'protocol_version': u'auto', u'connection': u'10.10.10.2:/mnt/lv_vdo01', u'user': u'kvm', u'id': u'e29cf818-5ee5-46e1-85c1-8aeefa33e95d'}], u'storagepoolID': u'00000000-0000-0000-0000-000000000000', u'domainType': 1}, 'jsonrpc': '2.0', 'method': u'StoragePool.connectStorageServer', 'id': u'c8887c86-fb5e-4be1-9552-45620144b74e'} at 0x7f08f8259650> timeout=60, duration=120.00 at 0x7f08f8259750> task#=8 at 0x7f091c110c50>, traceback:#012File: "/usr/lib64/python2.7/threading.py", line 785, in __bootstrap#012 self.__bootstrap_inner()#012File: "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner#012 self.run()#012File: "/usr/lib64/python2.7/threading.py", line 765, in run#012 self.__target(*self.__args, **self.__kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/common/concurrent.py", line 195, in run#012 ret = func(*args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 301, in _run#012 self._execute_task()#012File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 315, in _execute_task#012 task()#012File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 391, in __call__#012 self._callable()#012File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 262, in __call__#012 self._handler(self._ctx, self._req)#012File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 305, in _serveRequest#012 response = self._handle_request(req, ctx)#012File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 345, in _handle_request#012 res = method(**params)#012File: "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 198, in _dynamicMethod#012 result = fn(*methodArgs)#012File: "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1102, in connectStorageServer#012 connectionParams)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/dispatcher.py", line 74, in wrapper#012 result = ctask.prepare(func, *args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 108, in wrapper#012 return m(self, *a, **kw)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 1179, in prepare#012 result = self._run(func, *args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run#012 return fn(*args, **kargs)#012File: "<string>", line 2, in connectStorageServer#012File: "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in method#012 ret = func(*args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 2416, in connectStorageServer#012 conObj.connect()#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/storageServer.py", line 431, in connect#012 return self._mountCon.connect()#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/storageServer.py", line 172, in connect#012 self._mount.mount(self.options, self._vfsType, cgroup=self.CGROUP)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/mount.py", line 207, in mount#012 cgroup=cgroup)#012File: "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 56, in __call__#012 return callMethod()#012File: "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 54, in <lambda>#012 **kwargs)#012File: "<string>", line 2, in mount#012File: "/usr/lib64/python2.7/multiprocessing/managers.py", line 759, in _callmethod#012 kind, result = conn.recv() Aug 26 04:38:47 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:38:47 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:38:47 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:38:47 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:38:47 hvm002 python: detected unhandled Python exception in '/usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker' Aug 26 04:38:47 hvm002 abrt-server: Duplicate: core backtrace Aug 26 04:38:47 hvm002 abrt-server: DUP_OF_DIR: /var/tmp/abrt/Python-2019-08-26-04:32:28-3432 Aug 26 04:38:47 hvm002 abrt-server: Deleting problem directory Python-2019-08-26-04:38:47-17322 (dup of Python-2019-08-26-04:32:28-3432) Aug 26 04:38:47 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:38:47 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:38:47 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:38:47 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:38:47 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:38:47 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:38:47 hvm002 systemd: ovirt-ha-broker.service: main process exited, code=exited, status=1/FAILURE Aug 26 04:38:47 hvm002 systemd: Unit ovirt-ha-broker.service entered failed state. Aug 26 04:38:47 hvm002 systemd: ovirt-ha-broker.service failed. Aug 26 04:38:48 hvm002 systemd: ovirt-ha-broker.service holdoff time over, scheduling restart. Aug 26 04:38:48 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:38:48 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Communications Broker. Aug 26 04:38:48 hvm002 systemd: Started oVirt Hosted Engine High Availability Communications Broker. Aug 26 04:38:48 hvm002 systemd: Started Session c17 of user root. Aug 26 04:38:48 hvm002 systemd: Started Session c18 of user root. Aug 26 04:38:53 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:38:57 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:38:57 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:38:57 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:38:57 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:38:58 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:38:58 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:38:58 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:38:58 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:38:58 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:38:58 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:39:08 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:39:08 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:39:08 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:39:08 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:39:08 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:39:08 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:39:08 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:39:08 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:39:08 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:39:08 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:39:08 hvm002 systemd: ovirt-ha-agent.service failed. [root@hvm002 ~]# --------------------- Erick Perez ---------------------

1 2

Error while trying to import Xenserver OVA to Ovirt 4.3.5
by Erick Perez - Quadrian Enterprises 31 Aug '19

31 Aug '19

Hi there, I have a Xen Server 6.2 that we want to migrate its VMs to ovrt 4.3.5 on Centos.hosts. I exported the VMs from Xen to an OVA file and placed the OVA file in one of the Ovirt nodes. While using the import OVA web interface I get this error: VDSM hvm001.unachi.ac.pa command GetOvaInfoVDS failed: Internal JSON-RPC error: {'reason': "Attempt to call function: <bound method Global.getExternalVmFromOva of <vdsm.API.Global object at 0x7f558c050d10>> with arguments: (u'/opt/UNACHI VMs2.ova',) error: not all arguments converted during string formatting"} What will the correct procedure be? thanks, --------------------- Erick Perez ---------------------

1 0