- Users - oVirt List Archives

Any real experiences with nested virtualisation?
by tonyppe＠gmail.com 04 Sep '19

04 Sep '19

I have ovirt 4.3.5 and Openstack hosts. There are 2 x openstack compute and 2 x ovirt hosts with hosted manager. I think i can simplify the management of this if I were to virtualise the openstack compute nodes as VMs on ovirt. I wanted to reach out here to gauge any other users experiences with it if at all. From the research I have done it looks like from the VM perspective, migration must not be allowed. Any info / comments / feedback appreciated. Regards,

2 1

Re: Cannot Increase Hosted Engine VM Memory
by Dionysis K 04 Sep '19

04 Sep '19

Now i noticed another strange i deactivated the memory balloon option from the VM hosted engine and tried to update the values again with the same memory it still keeps the value of memory size at 5120 while the value of physical memory guaranteed it changed to what we desire at the 8192 value and then the error appears again in the vm engine i ckecked the engine log file it shows the same errors! is there are other logfile locations that can give us any clues ?

1 0

Re: Cannot Increase Hosted Engine VM Memory
by Douglas Duckworth 04 Sep '19

04 Sep '19

Yes, I do. Gold crown indeed. It's the "HostedEngine" as seen attached! Thanks, Douglas Duckworth, MSc, LFCS HPC System Administrator Scientific Computing Unit<https://scu.med.cornell.edu> Weill Cornell Medicine 1300 York Avenue New York, NY 10065 E: doug(a)med.cornell.edu<mailto:doug@med.cornell.edu> O: 212-746-6305 F: 212-746-8690 On Wed, Jan 23, 2019 at 12:02 PM Simone Tiraboschi <stirabos(a)redhat.com<mailto:stirabos@redhat.com>> wrote: On Wed, Jan 23, 2019 at 5:51 PM Douglas Duckworth <dod2014(a)med.cornell.edu<mailto:dod2014@med.cornell.edu>> wrote: Hi Simone Can I get help with this issue? Still cannot increase memory for Hosted Engine. From the logs it seams that the engine is trying to hotplug memory to the engine VM which is something it should not happen. The engine should simply update engine VM configuration in the OVF_STORE and require a reboot of the engine VM. Quick question, in the VM panel do you see a gold crown symbol on the Engine VM? Thanks, Douglas Duckworth, MSc, LFCS HPC System Administrator Scientific Computing Unit<https://scu.med.cornell.edu> Weill Cornell Medicine 1300 York Avenue New York, NY 10065 E: doug(a)med.cornell.edu<mailto:doug@med.cornell.edu> O: 212-746-6305 F: 212-746-8690 On Thu, Jan 17, 2019 at 8:08 AM Douglas Duckworth <dod2014(a)med.cornell.edu<mailto:dod2014@med.cornell.edu>> wrote: Sure, they're attached. In "first attempt" the error seems to be: 2019-01-17 07:49:24,795-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-29) [680f82b3-7612-4d91-afdc-43937aa298a2] EVENT_ID: FAILED_HOT_SET_MEMORY_NOT_DIVIDABLE(2,048), Failed to hot plug memory to VM HostedEngine. Amount of added memory (4000MiB) is not dividable by 256MiB. Followed by: 2019-01-17 07:49:24,814-05 WARN [org.ovirt.engine.core.bll.UpdateRngDeviceCommand] (default task-29) [26f5f3ed] Validation of action 'UpdateRngDevice' failed for user admin@internal-authz. Reasons: ACTION_TYPE_FAILED_VM_IS_RUNNING 2019-01-17 07:49:24,815-05 ERROR [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-29) [26f5f3ed] Updating RNG device of VM HostedEngine (adf14389-1563-4b1a-9af6-4b40370a825b) failed. Old RNG device = VmRngDevice:{id='VmDeviceId:{deviceId='6435b2b5-163c-4f0c-934e-7994da60dc89', vmId='adf14389-1563-4b1a-9af6-4b40370a825b'}', device='virtio', type='RNG', specParams='[source=urandom]', address='', managed='true', plugged='true', readOnly='false', deviceAlias='', customProperties='null', snapshotId='null', logicalName='null', hostDevice='null'}. New RNG device = VmRngDevice:{id='VmDeviceId:{deviceId='6435b2b5-163c-4f0c-934e-7994da60dc89', vmId='adf14389-1563-4b1a-9af6-4b40370a825b'}', device='virtio', type='RNG', specParams='[source=urandom]', address='', managed='true', plugged='true', readOnly='false', deviceAlias='', customProperties='null', snapshotId='null', logicalName='null', hostDevice='null'}. In "second attempt" I used values that are dividable by 256 MiB so that's no longer present. Though same error: 2019-01-17 07:56:59,795-05 INFO [org.ovirt.engine.core.vdsbroker.SetAmountOfMemoryVDSCommand] (default task-22) [7059a48f] START, SetAmountOfMemoryVDSCommand(HostName = ovirt-hv1.med.cornell.edu<http://ovirt-hv1.med.cornell.edu>, Params:{hostId='cdd5ffda-95c7-4ffa-ae40-be66f1d15c30', vmId='adf14389-1563-4b1a-9af6-4b40370a825b', memoryDevice='VmDevice:{id='VmDeviceId:{deviceId='7f7d97cc-c273-4033-af53-bc9033ea3abe', vmId='adf14389-1563-4b1a-9af6-4b40370a825b'}', device='memory', type='MEMORY', specParams='[node=0, size=2048]', address='', managed='true', plugged='true', readOnly='false', deviceAlias='', customProperties='null', snapshotId='null', logicalName='null', hostDevice='null'}', minAllocatedMem='6144'}), log id: 50873daa 2019-01-17 07:56:59,855-05 INFO [org.ovirt.engine.core.vdsbroker.SetAmountOfMemoryVDSCommand] (default task-22) [7059a48f] FINISH, SetAmountOfMemoryVDSCommand, log id: 50873daa 2019-01-17 07:56:59,862-05 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-22) [7059a48f] EVENT_ID: HOT_SET_MEMORY(2,039), Hotset memory: changed the amount of memory on VM HostedEngine from 4096 to 4096 2019-01-17 07:56:59,881-05 WARN [org.ovirt.engine.core.bll.UpdateRngDeviceCommand] (default task-22) [28fd4c82] Validation of action 'UpdateRngDevice' failed for user admin@internal-authz. Reasons: ACTION_TYPE_FAILED_VM_IS_RUNNING 2019-01-17 07:56:59,882-05 ERROR [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-22) [28fd4c82] Updating RNG device of VM HostedEngine (adf14389-1563-4b1a-9af6-4b40370a825b) failed. Old RNG device = VmRngDevice:{id='VmDeviceId:{deviceId='6435b2b5-163c-4f0c-934e-7994da60dc89', vmId='adf14389-1563-4b1a-9af6-4b40370a825b'}', device='virtio', type='RNG', specParams='[source=urandom]', address='', managed='true', plugged='true', readOnly='false', deviceAlias='', customProperties='null', snapshotId='null', logicalName='null', hostDevice='null'}. New RNG device = VmRngDevice:{id='VmDeviceId:{deviceId='6435b2b5-163c-4f0c-934e-7994da60dc89', vmId='adf14389-1563-4b1a-9af6-4b40370a825b'}', device='virtio', type='RNG', specParams='[source=urandom]', address='', managed='true', plugged='true', readOnly='false', deviceAlias='', customProperties='null', snapshotId='null', logicalName='null', hostDevice='null'}. This message repeats throughout engine.log: 2019-01-17 07:55:43,270-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-89) [] EVENT_ID: VM_MEMORY_UNDER_GUARANTEED_VALUE(148), VM HostedEngine on host ovirt-hv1.med.cornell.edu<http://ovirt-hv1.med.cornell.edu> was guaranteed 8192 MB but currently has 4224 MB As you can see attached the host has plenty of memory. Thank you Simone! Thanks, Douglas Duckworth, MSc, LFCS HPC System Administrator Scientific Computing Unit<https://scu.med.cornell.edu> Weill Cornell Medicine 1300 York Avenue New York, NY 10065 E: doug(a)med.cornell.edu<mailto:doug@med.cornell.edu> O: 212-746-6305 F: 212-746-8690 On Thu, Jan 17, 2019 at 5:09 AM Simone Tiraboschi <stirabos(a)redhat.com<mailto:stirabos@redhat.com>> wrote: On Wed, Jan 16, 2019 at 8:22 PM Douglas Duckworth <dod2014(a)med.cornell.edu<mailto:dod2014@med.cornell.edu>> wrote: Sorry for accidental send. Anyway I try to increase physical memory however it won't go above 4096MB. The hypervisor has 64GB. Do I need to modify this value with Hosted Engine offline? No, it's not required. Can you please attach your engine.log for the relevant time frame? Thanks, Douglas Duckworth, MSc, LFCS HPC System Administrator Scientific Computing Unit<https://scu.med.cornell.edu> Weill Cornell Medicine 1300 York Avenue New York, NY 10065 E: doug(a)med.cornell.edu<mailto:doug@med.cornell.edu> O: 212-746-6305 F: 212-746-8690 On Wed, Jan 16, 2019 at 1:58 PM Douglas Duckworth <dod2014(a)med.cornell.edu<mailto:dod2014@med.cornell.edu>> wrote: Hello I am trying to increase Hosted Engine physical memory above 4GB Thanks, Douglas Duckworth, MSc, LFCS HPC System Administrator Scientific Computing Unit<https://scu.med.cornell.edu> Weill Cornell Medicine 1300 York Avenue New York, NY 10065 E: doug(a)med.cornell.edu<mailto:doug@med.cornell.edu> O: 212-746-6305 F: 212-746-8690 _______________________________________________ Users mailing list -- users(a)ovirt.org<mailto:users@ovirt.org> To unsubscribe send an email to users-leave(a)ovirt.org<mailto:users-leave@ovirt.org> Privacy Statement: https://www.ovirt.org/site/privacy-policy/<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ovirt.org_site_pri…> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ovirt.org_communit…> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/WGSXQVVPJJ2CR…<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.ovirt.org_archiv…>

3 8

iSCSI Multipath/multiple gateways
by dan.poltawski＠tnp.net.uk 04 Sep '19

04 Sep '19

My iSCSI target (ceph-based) has multiple gateways, I’d like to ensure my hosted storage is aware of these such that each gateway can be rebooted for maintance without impacting service. What is the appropriate way to configure this so that each host knows of the multipath configuration? I do not have multiple network paths, just multiple gateways on the same network, so I think the multipath UI tool isn’t the way to do this. Any help/docs pointers appreciated. Regards, Dan

3 2

the virtual machine crushed and I cant shutdown the vm successfully
by zhouhao＠vip.friendtimes.net 04 Sep '19

04 Sep '19

Dear ovirt developers: The trouble bothered me for a long time;Some times my vm will crush and I cant shutdown it,I had to reboot the ovirt-node to slove it; the vm's error below The ovirt-node's error below The vm's threading on the ovirt-node, the IO ratio is 100% and the vm's process change to defunct I can not kill it,every time I had to shutdown the ovirt-node on the engine website ,the vm's status always in the way to shutdown ,even if I wait for it for hours; It either failed or is shutting down and the "power off" cant shutdown the vm too. ---------------------------------------------------------------- The other infomation about my ovirt-node node-version： node hardware zhouhao(a)vip.friendtimes.net

2 2

the virtual machine crushed and I cant shutdown the vm successfully
by zhouhao＠vip.friendtimes.net 04 Sep '19

04 Sep '19

Dear ovirt developers: The trouble bothered me for a long time;Some times my vm will crush and I cant shutdown it,I had to reboot the ovirt-node to slove it; the vm's error below The ovirt-node's error below The vm's threading on the ovirt-node, the IO ratio is 100% and the vm's process change to defunct I can not kill it,every time I had to shutdown the ovirt-node on the engine website ,the vm's status always in the way to shutdown ,even if I wait for it for hours; It either failed or is shutting down and the "power off" cant shutdown the vm too. ---------------------------------------------------------------- The other infomation about my ovirt-node node-version： node hardware zhouhao(a)vip.friendtimes.net

1 0

the virtual machine crushed and I cant shutdown the vm successfully
by zhouhao＠vip.friendtimes.net 04 Sep '19

04 Sep '19

Dear ovirt developers: The trouble bothered me for a long time;Some times my vm will crush and I cant shutdown it,I had to reboot the ovirt-node to slove it; the vm's error below The ovirt-node's error below The vm's threading on the ovirt-node, the IO ratio is 100% and the vm's process change to defunct I can not kill it,every time I had to shutdown the ovirt-node on the engine website ,the vm's status always in the way to shutdown ,even if I wait for it for hours; It either failed or is shutting down and the "power off" cant shutdown the vm too. ---------------------------------------------------------------- The other infomation about my ovirt-node node-version： node hardware zhouhao(a)vip.friendtimes.net

1 0

iSCSI Multipath/multiple gateways
by Dan Poltawski 03 Sep '19

03 Sep '19

My iSCSI target (ceph-based) has multiple gateways, I’d like to ensure my hosted storage is aware of these such that each gateway can be rebooted for maintance without impacting service. What is the appropriate way to configure this so that each host knows of the multipath configuration? I do not have multiple network paths, just multiple gateways on the same network, so I think the multipath UI tool isn’t the way to do this. Any help/docs pointers appreciated. Regards, Dan ________________________________ The Networking People (TNP) Limited. Registered office: Network House, Caton Rd, Lancaster, LA1 3PE. Registered in England & Wales with company number: 07667393 This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.

1 0

engine.log flooded with "Field 'foo' can not be updated when status is 'Up'"
by Matthias Leopold 02 Sep '19

02 Sep '19

Hi, I updated my production oVirt environment from 4.3.3 to 4.3.5 today. Everything went fine so far, but there's one annoying phenomenon: When I log into the "Administration Portal" and request the VM list ("/ovirt-engine/webadmin/?locale=en_US#vms") engine.log is flooded with lines like WARN [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-10618) [54d8c375-aa72-42f8-876e-8777d9d1a08a] Field 'balloonEnabled' can not be updated when status is 'Up' "Field", task and UUID vary and the flood stops after a while. Also listing or trying to edit other entities seems to trigger this "storm" or loop over and over again to a point that log file size is becoming an issue and interface is becoming sluggish. I can also see that CPU usage of engine java process goes up. When I log out everything is quiet and "VM Portal" is not affected at all. I have seen lines like that before and know that they are usually OK (when changing VM properties), but these logs used to be linked to singular events. I suspect that the present behaviour might be linked to VMs that have "Pending Virtual Machine changes", which are in most cases "Custom Compatibility Version" changes that still stem from the upgrade to Cluster Version 4.3. I can't be sure and I can't resolve all these pending changes now, but these should not be causing such annoying behaviour in the first place. I resorted to setting engine log level to "ERROR" right now to at least stop the log file from growing, but this not a solution. I can still see CPU load going up when using the interface. I very much hope that someone can explain whats happening and tell me how to resolve this. thanks a lot Matthias

4 8

Intel Xeon X5460
by Marcello Gentile 02 Sep '19

02 Sep '19

Is this procesor supported? https://ark.intel.com/content/www/us/en/ark/products/33087/intel-xeon-proce… Seems to meet all the requirements but when I install the self hosted engine it fails with the error message below: [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "The host has been set in non_operational status, deployment errors: code 156: Host Ovirt-Node1.adsdata.ca<http://Ovirt-Node1.adsdata.ca> moved to Non-Operational state as host CPU type is not supported in this cluster compatibility version or is not supported at all, code 9000: Failed to verify Power Management configuration for Host Ovirt-Node1.adsdata.ca<http://Ovirt-Node1.adsdata.ca>., fix accordingly and re-deploy."} Thanks for any help

3 3

How to modify DC's default route setting?
by gotoh＠eis.co.jp 01 Sep '19

01 Sep '19

We have to change default route on a production Ovirt system ( ovirt4.3 : engine + 9 hosts + glusterFS subsystem, approx 90 virtual machines.) , due to network enhancement We changed engine's default gateway and could permanently change host's default route by modifying the persistent setting. However, DC's logical network's default route remains unchanged. Consequently, the logical network became out of sync. We would be glad if anyone could instruct us how to change logical network's default route of DC. Thanks in advance, Hideo Goto

1 0

Error during self-hosted install using cockpit
by Marcello Gentile 01 Sep '19

01 Sep '19

Install fails with ERROR ansible failed {'status': 'FAILED', 'ansible_type': 'task', 'ansible_task': u'Fail with error description', 'ansible_result': u"type: <type 'dict'>\nstr: {'msg': u'The host has been set in non_operational status, deployment errors: code 156: Host Ovirt-Node1.adsdata.ca<http://Ovirt-Node1.adsdata.ca> moved to Non-Operational state as host CPU type is not supported in this cluster compatibility version or is not supported at all, code 9000: Failed to verify Power Management con", 'task_duration': 1, 'ansible_host': u'localhost', 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml'}

2 1

Engine died while creating VM
by Erick Perez - Quadrian Enterprises 01 Sep '19

01 Sep '19

Hi, I was just creating a VM on a freshly installed 2-node selfhosted ovirt setup with a NFS backend. NFS connections are v4.2 for VM data domain What other log is needed? Thanks, in advance. Here is the output of the last 300 lines of /var/log/messages [root@hvm002 ~]# [root@hvm002 ~]# tail /var/log/messages -n 300 Aug 26 04:34:56 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:34:56 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:34:56 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:35:06 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:35:06 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:35:06 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:35:06 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:35:07 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:35:07 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:35:07 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:35:07 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:35:07 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:35:07 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:35:08 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:35:15 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:35:15 hvm002 systemd: Starting Cockpit Web Service... Aug 26 04:35:15 hvm002 systemd: Started Cockpit Web Service. Aug 26 04:35:15 hvm002 cockpit-ws: Using certificate: /etc/cockpit/ws-certs.d/0-self-signed.cert Aug 26 04:35:15 hvm002 cockpit-ws: couldn't read from connection: Peer sent fatal TLS alert: Certificate is bad Aug 26 04:35:17 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:35:17 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:35:17 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:35:17 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:35:17 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:35:17 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:35:17 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:35:17 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:35:17 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:35:17 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:35:23 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:35:27 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:35:27 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:35:27 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:35:27 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:35:28 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:35:28 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:35:28 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:35:28 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:35:28 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:35:28 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:35:36 hvm002 vdsm[7369]: WARN Worker blocked: <Worker name=jsonrpc/5 running <Task <JsonRpcTask {'params': {u'connectionParams': [{u'protocol_version': u'auto', u'connection': u'10.10.10.2:/mnt/lv_vdo01', u'user': u'kvm', u'id': u'e29cf818-5ee5-46e1-85c1-8aeefa33e95d'}], u'storagepoolID': u'00000000-0000-0000-0000-000000000000', u'domainType': 1}, 'jsonrpc': '2.0', 'method': u'StoragePool.connectStorageServer', 'id': u'767bed1a-dc31-44f6-98a8-94faebc82345'} at 0x7f08f83cadd0> timeout=60, duration=60.00 at 0x7f091c0aadd0> task#=4 at 0x7f091c0c6e10>, traceback:#012File: "/usr/lib64/python2.7/threading.py", line 785, in __bootstrap#012 self.__bootstrap_inner()#012File: "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner#012 self.run()#012File: "/usr/lib64/python2.7/threading.py", line 765, in run#012 self.__target(*self.__args, **self.__kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/common/concurrent.py", line 195, in run#012 ret = func(*args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 301, in _run#012 self._execute_task()#012File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 315, in _execute_task#012 task()#012File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 391, in __call__#012 self._callable()#012File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 262, in __call__#012 self._handler(self._ctx, self._req)#012File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 305, in _serveRequest#012 response = self._handle_request(req, ctx)#012File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 345, in _handle_request#012 res = method(**params)#012File: "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 198, in _dynamicMethod#012 result = fn(*methodArgs)#012File: "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1102, in connectStorageServer#012 connectionParams)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/dispatcher.py", line 74, in wrapper#012 result = ctask.prepare(func, *args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 108, in wrapper#012 return m(self, *a, **kw)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 1179, in prepare#012 result = self._run(func, *args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run#012 return fn(*args, **kargs)#012File: "<string>", line 2, in connectStorageServer#012File: "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in method#012 ret = func(*args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 2416, in connectStorageServer#012 conObj.connect()#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/storageServer.py", line 431, in connect#012 return self._mountCon.connect()#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/storageServer.py", line 172, in connect#012 self._mount.mount(self.options, self._vfsType, cgroup=self.CGROUP)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/mount.py", line 207, in mount#012 cgroup=cgroup)#012File: "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 56, in __call__#012 return callMethod()#012File: "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 54, in <lambda>#012 **kwargs)#012File: "<string>", line 2, in mount#012File: "/usr/lib64/python2.7/multiprocessing/managers.py", line 759, in _callmethod#012 kind, result = conn.recv() Aug 26 04:35:38 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:35:38 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:35:38 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:35:38 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:35:38 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:35:38 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:35:38 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:35:38 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:35:38 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:35:38 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:35:38 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:35:48 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:35:48 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:35:48 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:35:48 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:35:49 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:35:49 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:35:49 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:35:49 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:35:49 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:35:49 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:35:52 hvm002 systemd: Created slice User Slice of root. Aug 26 04:35:52 hvm002 systemd-logind: New session 2 of user root. Aug 26 04:35:52 hvm002 systemd: Started Session 2 of user root. Aug 26 04:35:53 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:35:59 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:35:59 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:35:59 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:35:59 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:35:59 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:35:59 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:35:59 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:35:59 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:35:59 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:35:59 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:36:08 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:36:09 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:36:09 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:36:09 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:36:09 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:36:10 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:36:10 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:36:10 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:36:10 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:36:10 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:36:10 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:36:20 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:36:20 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:36:20 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:36:20 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:36:20 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:36:20 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:36:20 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:36:20 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:36:20 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:36:20 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:36:23 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:36:30 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:36:30 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:36:30 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:36:30 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:36:31 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:36:31 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:36:31 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:36:31 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:36:31 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:36:31 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:36:36 hvm002 vdsm[7369]: WARN Worker blocked: <Worker name=jsonrpc/5 running <Task <JsonRpcTask {'params': {u'connectionParams': [{u'protocol_version': u'auto', u'connection': u'10.10.10.2:/mnt/lv_vdo01', u'user': u'kvm', u'id': u'e29cf818-5ee5-46e1-85c1-8aeefa33e95d'}], u'storagepoolID': u'00000000-0000-0000-0000-000000000000', u'domainType': 1}, 'jsonrpc': '2.0', 'method': u'StoragePool.connectStorageServer', 'id': u'767bed1a-dc31-44f6-98a8-94faebc82345'} at 0x7f08f83cadd0> timeout=60, duration=120.00 at 0x7f091c0aadd0> task#=4 at 0x7f091c0c6e10>, traceback:#012File: "/usr/lib64/python2.7/threading.py", line 785, in __bootstrap#012 self.__bootstrap_inner()#012File: "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner#012 self.run()#012File: "/usr/lib64/python2.7/threading.py", line 765, in run#012 self.__target(*self.__args, **self.__kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/common/concurrent.py", line 195, in run#012 ret = func(*args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 301, in _run#012 self._execute_task()#012File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 315, in _execute_task#012 task()#012File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 391, in __call__#012 self._callable()#012File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 262, in __call__#012 self._handler(self._ctx, self._req)#012File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 305, in _serveRequest#012 response = self._handle_request(req, ctx)#012File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 345, in _handle_request#012 res = method(**params)#012File: "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 198, in _dynamicMethod#012 result = fn(*methodArgs)#012File: "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1102, in connectStorageServer#012 connectionParams)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/dispatcher.py", line 74, in wrapper#012 result = ctask.prepare(func, *args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 108, in wrapper#012 return m(self, *a, **kw)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 1179, in prepare#012 result = self._run(func, *args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run#012 return fn(*args, **kargs)#012File: "<string>", line 2, in connectStorageServer#012File: "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in method#012 ret = func(*args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 2416, in connectStorageServer#012 conObj.connect()#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/storageServer.py", line 431, in connect#012 return self._mountCon.connect()#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/storageServer.py", line 172, in connect#012 self._mount.mount(self.options, self._vfsType, cgroup=self.CGROUP)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/mount.py", line 207, in mount#012 cgroup=cgroup)#012File: "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 56, in __call__#012 return callMethod()#012File: "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 54, in <lambda>#012 **kwargs)#012File: "<string>", line 2, in mount#012File: "/usr/lib64/python2.7/multiprocessing/managers.py", line 759, in _callmethod#012 kind, result = conn.recv() Aug 26 04:36:38 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:36:41 hvm002 python: detected unhandled Python exception in '/usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker' Aug 26 04:36:41 hvm002 abrt-server: Duplicate: core backtrace Aug 26 04:36:41 hvm002 abrt-server: DUP_OF_DIR: /var/tmp/abrt/Python-2019-08-26-04:32:28-3432 Aug 26 04:36:41 hvm002 abrt-server: Deleting problem directory Python-2019-08-26-04:36:41-16985 (dup of Python-2019-08-26-04:32:28-3432) Aug 26 04:36:41 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:36:41 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:36:41 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:36:41 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:36:41 hvm002 dbus[3444]: [system] Activating service name='org.freedesktop.problems' (using servicehelper) Aug 26 04:36:41 hvm002 dbus[3444]: [system] Successfully activated service 'org.freedesktop.problems' Aug 26 04:36:41 hvm002 systemd: ovirt-ha-broker.service: main process exited, code=exited, status=1/FAILURE Aug 26 04:36:41 hvm002 systemd: Unit ovirt-ha-broker.service entered failed state. Aug 26 04:36:41 hvm002 systemd: ovirt-ha-broker.service failed. Aug 26 04:36:41 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:36:41 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:36:41 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:36:41 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:36:41 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:36:41 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:36:41 hvm002 systemd: ovirt-ha-broker.service holdoff time over, scheduling restart. Aug 26 04:36:41 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:36:41 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Communications Broker. Aug 26 04:36:41 hvm002 systemd: Started oVirt Hosted Engine High Availability Communications Broker. Aug 26 04:36:42 hvm002 systemd: Started Session c15 of user root. Aug 26 04:36:42 hvm002 systemd: Started Session c16 of user root. Aug 26 04:36:51 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:36:51 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:36:51 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:36:51 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:36:52 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:36:52 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:36:52 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:36:52 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:36:52 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:36:52 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:36:53 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:37:02 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:37:02 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:37:02 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:37:02 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:37:02 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:37:02 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:37:02 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:37:02 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:37:02 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:37:02 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:37:08 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:37:12 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:37:12 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:37:12 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:37:12 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:37:13 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:37:13 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:37:13 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:37:13 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:37:13 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:37:13 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:37:23 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:37:23 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:37:23 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:37:23 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:37:23 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:37:23 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:37:23 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:37:23 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:37:23 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:37:23 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:37:23 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:37:33 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:37:33 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:37:33 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:37:33 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:37:34 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:37:34 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:37:34 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:37:34 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:37:34 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:37:34 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:37:38 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:37:42 hvm002 vdsm[7369]: WARN Worker blocked: <Worker name=jsonrpc/3 running <Task <JsonRpcTask {'params': {u'connectionParams': [{u'protocol_version': u'auto', u'connection': u'10.10.10.2:/mnt/lv_vdo01', u'user': u'kvm', u'id': u'e29cf818-5ee5-46e1-85c1-8aeefa33e95d'}], u'storagepoolID': u'00000000-0000-0000-0000-000000000000', u'domainType': 1}, 'jsonrpc': '2.0', 'method': u'StoragePool.connectStorageServer', 'id': u'c8887c86-fb5e-4be1-9552-45620144b74e'} at 0x7f08f8259650> timeout=60, duration=60.00 at 0x7f08f8259750> task#=8 at 0x7f091c110c50>, traceback:#012File: "/usr/lib64/python2.7/threading.py", line 785, in __bootstrap#012 self.__bootstrap_inner()#012File: "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner#012 self.run()#012File: "/usr/lib64/python2.7/threading.py", line 765, in run#012 self.__target(*self.__args, **self.__kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/common/concurrent.py", line 195, in run#012 ret = func(*args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 301, in _run#012 self._execute_task()#012File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 315, in _execute_task#012 task()#012File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 391, in __call__#012 self._callable()#012File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 262, in __call__#012 self._handler(self._ctx, self._req)#012File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 305, in _serveRequest#012 response = self._handle_request(req, ctx)#012File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 345, in _handle_request#012 res = method(**params)#012File: "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 198, in _dynamicMethod#012 result = fn(*methodArgs)#012File: "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1102, in connectStorageServer#012 connectionParams)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/dispatcher.py", line 74, in wrapper#012 result = ctask.prepare(func, *args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 108, in wrapper#012 return m(self, *a, **kw)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 1179, in prepare#012 result = self._run(func, *args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run#012 return fn(*args, **kargs)#012File: "<string>", line 2, in connectStorageServer#012File: "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in method#012 ret = func(*args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 2416, in connectStorageServer#012 conObj.connect()#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/storageServer.py", line 431, in connect#012 return self._mountCon.connect()#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/storageServer.py", line 172, in connect#012 self._mount.mount(self.options, self._vfsType, cgroup=self.CGROUP)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/mount.py", line 207, in mount#012 cgroup=cgroup)#012File: "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 56, in __call__#012 return callMethod()#012File: "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 54, in <lambda>#012 **kwargs)#012File: "<string>", line 2, in mount#012File: "/usr/lib64/python2.7/multiprocessing/managers.py", line 759, in _callmethod#012 kind, result = conn.recv() Aug 26 04:37:44 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:37:44 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:37:44 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:37:44 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:37:44 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:37:44 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:37:44 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:37:44 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:37:44 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:37:44 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:37:53 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:37:54 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:37:54 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:37:54 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:37:54 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:37:55 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:37:55 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:37:55 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:37:55 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:37:55 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:37:55 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:38:05 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:38:05 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:38:05 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:38:05 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:38:05 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:38:05 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:38:05 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:38:05 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:38:05 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:38:05 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:38:08 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:38:15 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:38:15 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:38:15 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:38:15 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:38:16 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:38:16 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:38:16 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:38:16 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:38:16 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:38:16 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:38:23 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:38:26 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:38:26 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:38:26 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:38:26 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:38:26 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:38:26 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:38:26 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:38:26 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:38:26 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:38:26 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:38:36 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:38:36 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:38:36 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:38:36 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:38:37 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:38:37 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:38:37 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:38:37 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:38:37 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:38:37 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:38:38 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:38:42 hvm002 vdsm[7369]: WARN Worker blocked: <Worker name=jsonrpc/3 running <Task <JsonRpcTask {'params': {u'connectionParams': [{u'protocol_version': u'auto', u'connection': u'10.10.10.2:/mnt/lv_vdo01', u'user': u'kvm', u'id': u'e29cf818-5ee5-46e1-85c1-8aeefa33e95d'}], u'storagepoolID': u'00000000-0000-0000-0000-000000000000', u'domainType': 1}, 'jsonrpc': '2.0', 'method': u'StoragePool.connectStorageServer', 'id': u'c8887c86-fb5e-4be1-9552-45620144b74e'} at 0x7f08f8259650> timeout=60, duration=120.00 at 0x7f08f8259750> task#=8 at 0x7f091c110c50>, traceback:#012File: "/usr/lib64/python2.7/threading.py", line 785, in __bootstrap#012 self.__bootstrap_inner()#012File: "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner#012 self.run()#012File: "/usr/lib64/python2.7/threading.py", line 765, in run#012 self.__target(*self.__args, **self.__kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/common/concurrent.py", line 195, in run#012 ret = func(*args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 301, in _run#012 self._execute_task()#012File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 315, in _execute_task#012 task()#012File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 391, in __call__#012 self._callable()#012File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 262, in __call__#012 self._handler(self._ctx, self._req)#012File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 305, in _serveRequest#012 response = self._handle_request(req, ctx)#012File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 345, in _handle_request#012 res = method(**params)#012File: "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 198, in _dynamicMethod#012 result = fn(*methodArgs)#012File: "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1102, in connectStorageServer#012 connectionParams)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/dispatcher.py", line 74, in wrapper#012 result = ctask.prepare(func, *args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 108, in wrapper#012 return m(self, *a, **kw)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 1179, in prepare#012 result = self._run(func, *args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run#012 return fn(*args, **kargs)#012File: "<string>", line 2, in connectStorageServer#012File: "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in method#012 ret = func(*args, **kwargs)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 2416, in connectStorageServer#012 conObj.connect()#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/storageServer.py", line 431, in connect#012 return self._mountCon.connect()#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/storageServer.py", line 172, in connect#012 self._mount.mount(self.options, self._vfsType, cgroup=self.CGROUP)#012File: "/usr/lib/python2.7/site-packages/vdsm/storage/mount.py", line 207, in mount#012 cgroup=cgroup)#012File: "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 56, in __call__#012 return callMethod()#012File: "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 54, in <lambda>#012 **kwargs)#012File: "<string>", line 2, in mount#012File: "/usr/lib64/python2.7/multiprocessing/managers.py", line 759, in _callmethod#012 kind, result = conn.recv() Aug 26 04:38:47 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:38:47 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:38:47 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:38:47 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:38:47 hvm002 python: detected unhandled Python exception in '/usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker' Aug 26 04:38:47 hvm002 abrt-server: Duplicate: core backtrace Aug 26 04:38:47 hvm002 abrt-server: DUP_OF_DIR: /var/tmp/abrt/Python-2019-08-26-04:32:28-3432 Aug 26 04:38:47 hvm002 abrt-server: Deleting problem directory Python-2019-08-26-04:38:47-17322 (dup of Python-2019-08-26-04:32:28-3432) Aug 26 04:38:47 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:38:47 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:38:47 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:38:47 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:38:47 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:38:47 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:38:47 hvm002 systemd: ovirt-ha-broker.service: main process exited, code=exited, status=1/FAILURE Aug 26 04:38:47 hvm002 systemd: Unit ovirt-ha-broker.service entered failed state. Aug 26 04:38:47 hvm002 systemd: ovirt-ha-broker.service failed. Aug 26 04:38:48 hvm002 systemd: ovirt-ha-broker.service holdoff time over, scheduling restart. Aug 26 04:38:48 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:38:48 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Communications Broker. Aug 26 04:38:48 hvm002 systemd: Started oVirt Hosted Engine High Availability Communications Broker. Aug 26 04:38:48 hvm002 systemd: Started Session c17 of user root. Aug 26 04:38:48 hvm002 systemd: Started Session c18 of user root. Aug 26 04:38:53 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:38:57 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:38:57 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:38:57 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:38:57 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:38:58 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:38:58 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:38:58 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:38:58 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:38:58 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:38:58 hvm002 systemd: ovirt-ha-agent.service failed. Aug 26 04:39:08 hvm002 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Aug 26 04:39:08 hvm002 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Aug 26 04:39:08 hvm002 systemd: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:39:08 hvm002 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Aug 26 04:39:08 hvm002 vdsm[7369]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? Aug 26 04:39:08 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Aug 26 04:39:08 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 432, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 556, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor#012 ).format(t=type, o=options, e=e)#012RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'tcp_t_address': '', 'network_test': 'dns', 'tcp_t_port': '', 'addr': '172.21.48.1'}] Aug 26 04:39:08 hvm002 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Aug 26 04:39:08 hvm002 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Aug 26 04:39:08 hvm002 systemd: Unit ovirt-ha-agent.service entered failed state. Aug 26 04:39:08 hvm002 systemd: ovirt-ha-agent.service failed. [root@hvm002 ~]# --------------------- Erick Perez ---------------------

1 2

Error while trying to import Xenserver OVA to Ovirt 4.3.5
by Erick Perez - Quadrian Enterprises 01 Sep '19

01 Sep '19

Hi there, I have a Xen Server 6.2 that we want to migrate its VMs to ovrt 4.3.5 on Centos.hosts. I exported the VMs from Xen to an OVA file and placed the OVA file in one of the Ovirt nodes. While using the import OVA web interface I get this error: VDSM hvm001.unachi.ac.pa command GetOvaInfoVDS failed: Internal JSON-RPC error: {'reason': "Attempt to call function: <bound method Global.getExternalVmFromOva of <vdsm.API.Global object at 0x7f558c050d10>> with arguments: (u'/opt/UNACHI VMs2.ova',) error: not all arguments converted during string formatting"} What will the correct procedure be? thanks, --------------------- Erick Perez ---------------------

1 0

4.3.5 VM Guest filesystem corruption while deploying from template and root owned files problem.
by levin＠mydream.com.hk 31 Aug '19

31 Aug '19

Just upgraded from 4.2.7 to 4.3.5 with preserving 4.2 compatibility in cluster level. Everything is smooth and keep up and running over two weeks. However when we just want to deploy from existing Centos Linux VM Template, the VM shown "not a correct XFS inode." "file /grub2/i386-pc/normal.mod not found" error. Also tried another RHEL and OEL template, they all have same problem. So I tried to install a new Linux and converting to template to see whether it is existing disk problem, but it appear the same xfs error after template deployment. Have been tried on qcow2 and raw format. no different. The interesting finding is Windows Server Template doesn't suffer this problem. So is it from image tools bug? Secondly, I noticed there are intermittent issue where ovirt node create image objects owned by root user, and that cause VM fail to start. It happens likely when creating VM image snapshot and after restore snapshot. Anyone can point me out where should I looks into. Thank you!

1 0

Re: VMs inaccessible from Ubuntu/Debian-based OS
by Strahil 30 Aug '19

30 Aug '19

You need virt-viewer installed on your ubuntu or change console type to spice. Best Regards, Strahil NikolovOn Aug 30, 2019 15:35, souvaliotimaria(a)mail.com wrote: > > Hello all, > > I'm having an issue the past couple of days. I have tried anything I could find to solve this but with no success. > > I have a three node ovirt installation, glustered, hyperconverged and there i have a few VMs. The installation is for experimental reasons before we merge it in our DC. Anyway, though I can connect to the VMs' console from Fedora and Windows, I can't from Ubuntu. > > I have tried installing the browser-spice-plugin that's the corresponding package to spice-xpi and I got no results. > > I purged the browser-spice-plugin and then I installed the spice-client package, downloaded the spice-xpi from Fedora (FC19) (as instructed in https://www.ovirt.org/develop/infra/testing/spice.html) copied the libnsISpicec.so to /usr/lib/mozilla/plugins and made sure that xserver-xorg-video-qxl and spice-vdagent were installed and in their latest version, and still nothing. > > No matter what I have tried, I can't gain access to the console. No matter the browser I use, the message I get is "Unable to connect to the graphic server /tmp/mozilla_msouval0/console-1.vv". > > I have checked the logs and couldn't find anything useful, maybe I'm not checking the right logs? > I run tcpdump on both the node the VM is being hosted and the Ubuntu machine I'm using and though on the UBuntu I capture a few packets (6) on both sides but on the Ubuntu side there were 15 packets received by the filter. > > Could you please guide towards the right way to solve this? Could this be an ntp problem? > > ovirt node version: 4.2.3.1 > Workstation: Jessie > > (I thought maybe Jessie was too old, and I run the same steps from Ubuntu 14.04 and Ubuntu 16.04, but the problem still remains) > > Thank you in advance for any help. > Maria > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/4VMMYEJNSHGUH…

2 1

VMs inaccessible from Ubuntu/Debian-based OS
by souvaliotimaria＠mail.com 30 Aug '19

30 Aug '19

Hello all, I'm having an issue the past couple of days. I have tried anything I could find to solve this but with no success. I have a three node ovirt installation, glustered, hyperconverged and there i have a few VMs. The installation is for experimental reasons before we merge it in our DC. Anyway, though I can connect to the VMs' console from Fedora and Windows, I can't from Ubuntu. I have tried installing the browser-spice-plugin that's the corresponding package to spice-xpi and I got no results. I purged the browser-spice-plugin and then I installed the spice-client package, downloaded the spice-xpi from Fedora (FC19) (as instructed in https://www.ovirt.org/develop/infra/testing/spice.html) copied the libnsISpicec.so to /usr/lib/mozilla/plugins and made sure that xserver-xorg-video-qxl and spice-vdagent were installed and in their latest version, and still nothing. No matter what I have tried, I can't gain access to the console. No matter the browser I use, the message I get is "Unable to connect to the graphic server /tmp/mozilla_msouval0/console-1.vv". I have checked the logs and couldn't find anything useful, maybe I'm not checking the right logs? I run tcpdump on both the node the VM is being hosted and the Ubuntu machine I'm using and though on the UBuntu I capture a few packets (6) on both sides but on the Ubuntu side there were 15 packets received by the filter. Could you please guide towards the right way to solve this? Could this be an ntp problem? ovirt node version: 4.2.3.1 Workstation: Jessie (I thought maybe Jessie was too old, and I run the same steps from Ubuntu 14.04 and Ubuntu 16.04, but the problem still remains) Thank you in advance for any help. Maria

1 0

Hosted-engine inaccessible
by Tau Makgaile 29 Aug '19

29 Aug '19

Hi, I am currently experiencing a problem with my Hosted-engine. Hosted-engine disconnected after increasing / partition. The increase went well but after some time the hosted-enigine VM disconnected and has since been giving alerts such as* re-initializingFSM*. Though VMs underneth are running, Hosted-engine --vm-status: *"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail"* There is no backup to restore at the moment. I am looking for a way to bring it up without redeploying the hosted engine. Thanks in advance for your help. Kind regards, Tau

2 1

python sdk create a windows machine with initial run sysprep
by Harry Conings 29 Aug '19

29 Aug '19

Hi i am trying to create virtual machines from a template but i cannot seem to find how to ad the inital run sysprep option how should i give that option ? my code <code> NEW_VM_NAME = 'W10MYMACINE' TEMPLATE_NAME = 'W10TEMP' CLUSTER_NAME= 'MYCLUSTER' try: api.vms.add(params.VM(name=NEW_VM_NAME, cluster=api.clusters.get(CLUSTER_NAME), template=api.templates.get(TEMPLATE_NAME))) print 'VM was created from Template successfully' print 'Waiting for VM to reach Down status' while api.vms.get(NEW_VM_NAME).status.state != 'down': sleep(1) except Exception as e: print 'Failed to create VM from Template:\n%s' % str(e) </code>

2 3

Error preparing image
by Paul Wright 29 Aug '19

29 Aug '19

Hi, I have just taken over an Ovirt cluster running 4.1.2.2, it comprises of three hosts of which, two can run the HE. This message floods /var/log/messages on the hosts that can run the HE. Aug 28 17:39:56 ovirt-03 journal: ovirt-ha-agent ovirt_hosted_engine_ha.lib.image.Image ERROR Error preparing image - storagepoolID: 00000000-0000-0000-0000-000000000000 - storagedomainID: 0ea4f86e-3d20-422f-baf0-bb0f6ca3d0b9 - imageID: 3019b1a4-4776-42d9-a3d3-43930acfe681 - volumeID: 28ebf041-7222-4954-af2d-0889cb5f0cd0: Cannot prepare illegal volume: (u'28ebf041-7222-4954-af2d-0889cb5f0cd0',) What does it mean? Many thanks, Paul -- <http://www.freefolk.com/> See our latest work <http://freefolk.com/advertising> Freefolk 140 Wardour Street, London W1F 8ZT freefolk.com <http://www.freefolk.com/> / +44 (0)20 7734 9110 @freefolkstudios <https://www.twitter.com/freefolkstudios> / facebook/freefolkstudios <https://www.facebook.com/freefolkstudios> / instagram/freefolkstudios <https://www.instagram.com/freefolkstudios> / linkedin <https://www.linkedin.com/company/finish> <http://freefolk.com/film-and-tv/>

1 0

[ANN] oVirt 4.3.6 Fourth Release Candidate is now available for testing
by Sandro Bonazzola 29 Aug '19

29 Aug '19

The oVirt Project is pleased to announce the availability of the oVirt 4.3.6 Fourth Release Candidate for testing, as of August 29th, 2019. This update is a release candidate of the sixth in a series of stabilization updates to the 4.3 series. This is pre-release software. This pre-release should not to be used in production. This release is available now on x86_64 architecture for: * Red Hat Enterprise Linux 7.7 or later (but <8) * CentOS Linux (or similar) 7.7 or later (but <8) This release supports Hypervisor Hosts on x86_64 and ppc64le architectures for: * Red Hat Enterprise Linux 7.7 or later (but <8) * CentOS Linux (or similar) 7.7 or later (but <8) * oVirt Node 4.3 (available for x86_64 only) will be made available when CentOS 7.7 will be released. See the release notes [1] for known issues, new features and bugs fixed. Notes: - oVirt Appliance is already available - oVirt Node is not yet available, pending CentOS 7.7 release to be available Additional Resources: * Read more about the oVirt 4.3.6 release highlights: http://www.ovirt.org/release/4.3.6/ * Get more oVirt Project updates on Twitter: https://twitter.com/ovirt * Check out the latest project news on the oVirt blog: http://www.ovirt.org/blog/ [1] http://www.ovirt.org/release/4.3.6/ [2] http://resources.ovirt.org/pub/ovirt-4.3-pre/iso/ -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/>*Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

1 0

Re: Installing new oVirt node fails
by Lucie Leistnerova 29 Aug '19

29 Aug '19

Hi, I'm sorry for my late response. On 8/13/19 12:21 PM, Winfried de Heiden wrote: > I guess installation will fail, most likeley beacause of some > "strange" characters in the ssh pub-keys > Installation is incomplete and therfor vdsm will start complaining. > > In the past, I was able to reproduce oVirt installation(issues) using > nested kvm. For some reason the old method > (https://ovirt.org/develop/developer-guide/vdsm/developers.html) does > notwork anymore. No more nested kvm anymore? I'm prety shure my > installtion issues are ssh-pubkey related, but I need a virtual > hypervisor to reproduce... > I'm QE and I never installed vdsm other than from provided rpms. So I can't help with that guide. I can just check the deploy logs from the engine, whether I can get more information what was wrong. > Winfried > > > Op 12-08-19 om 13:19 schreef Lucie Leistnerova: >> On 8/12/19 10:34 AM, Winfried de Heiden wrote: >>> Hi all, >>> >>> No, the certificate was OK, no UTF characters found. I think I found >>> the problem however; the public keys as found in >>> /root/.ssh/authorized_keys. >>> >>> One or two keys did contain a "@" in the description. After removing >>> these keys, installation went well. I'll try to reproduce using a >>> virtual hypervisor, smells like a bug. >>> >> That sounds strange, @ is 0x40. Otopi complains about 0x3c, that >> characters start with À. >> Please send the host deploy log. >> >> >>> Winfried >>> >>> Op 12-08-19 om 07:59 schreef Lucie Leistnerova: >>>> >>>> Hi Winfried, >>>> >>>> On 8/11/19 7:20 PM, Winfried de Heiden wrote: >>>>> Hi all, >>>>> >>>>> Installing/regestering a host to ovirt (ovirt: 4.3.5.5-1.el7 + >>>>> oVirt Node 4.3.5.2)) will fail with, seems lang something using >>>>> yum goes wrong. Installing will fail... >>>>> >>>> You may hit this issue >>>> >>>> https://bugzilla.redhat.com/show_bug.cgi?id=1733438 >>>> >>>> Check your certificates for utf characters. >>>> >>>>> 019-08-11 18:40:44,408+0200 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:204 DIALOG:SEND **%EventStart STAGE misc METHOD otopi.plugins.otopi.network.ssh.Plugin._append_key (None) >>>>> 2019-08-11 18:40:44,409+0200 DEBUG otopi.context context._executeMethod:145 method exception >>>>> Traceback (most recent call last): >>>>> File "/tmp/ovirt-sMdZ7tvESy/pythonlib/otopi/context.py", line 132, in _executeMethod >>>>> method['method']() >>>>> File "/tmp/ovirt-sMdZ7tvESy/otopi-plugins/otopi/network/ssh.py", line 152, in _append_key >>>>> constants.CoreEnv.MODIFIED_FILES >>>>> File "/tmp/ovirt-sMdZ7tvESy/pythonlib/otopi/filetransaction.py", line 151, in __init__ >>>>> self._content = self._content.encode("utf-8") >>>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 750: ordinal not in range(128) >>>>> 2019-08-11 18:40:44,409+0200ERROR otopi.context context._executeMethod:154 Failed to execute stage 'Misc configuration': 'ascii' codec can't decode byte 0xc3 in position 750: ordinal not in range(128) >>>>> 2019-08-11 18:40:44,410+0200 DEBUG otopi.transaction transaction.abort:119 aborting 'Yum Transaction' >>>>> 2019-08-11 18:40:44,410+0200 INFO otopi.plugins.otopi.packagers.yumpackager yumpackager.info:80 Yum Performing yum transaction rollback >>>>> Loaded plugins: fastestmirror, product-id, subscription-manager, versionlock >>>>> >>>>> Afterwards, oVirt Node will complain: >>>>> >>>>> aug 11 19:13:32 bigvirt.blabla.bla vdsm[2342]: ERROR ssl handshake: SSLError, address: ::ffff:10.19.69.3 >>>>> aug 11 19:13:55 bigvirt.blabla.bla vdsm[2342]: ERROR ssl handshake: SSLError, address: ::ffff:10.19.69.3 >>>>> aug 11 19:14:18 bigvirt.blabla.bla vdsm[2342]: ERROR ssl handshake: SSLError, address: ::ffff:10.19.69.3 >>>>> aug 11 19:14:41 bigvirt.blabla.bla vdsm[2342]: ERROR ssl handshake: SSLError, address: ::ffff:10.19.69.3 >>>>> >>>>> Anyone? >>>>> >>>>> Winfried >>>>> >>>>> _______________________________________________ >>>>> Users mailing list --users(a)ovirt.org >>>>> To unsubscribe send an email tousers-leave(a)ovirt.org >>>>> Privacy Statement:https://www.ovirt.org/site/privacy-policy/ >>>>> oVirt Code of Conduct:https://www.ovirt.org/community/about/community-guidelines/ >>>>> List Archives:https://lists.ovirt.org/archives/list/users@ovirt.org/message/2A4Q… >>>> Best regards, >>>> >>>> Lucie >>>> >>>> -- >>>> Lucie Leistnerova >>>> Senior Quality Engineer, QE Cloud, RHVM >>>> Red Hat EMEA >>>> >>>> IRC: lleistne @ #rhev-qe >>> >> -- >> Lucie Leistnerova >> Senior Quality Engineer, QE Cloud, RHVM >> Red Hat EMEA >> >> IRC: lleistne @ #rhev-qe > -- Lucie Leistnerova Senior Quality Engineer, QE Cloud, RHVM Red Hat EMEA IRC: lleistne @ #rhev-qe

2 1

ovirt api snapshot date format?
by Bill James 29 Aug '19

29 Aug '19

I'm trying to use the ovirt REST api to read snapshot date but the number I'm getting back doesn't appear to be in epoch time. What format is it? Example: "date":1565299725928,"snapshot_status":"ok" Should be "Aug 8, 2019", epoch conversion put it at Tue May 16 16:25:28 51572. (accessed via https://$HOSTNAME/ovirt-engine/api/vms/$value/snapshots)

2 1

Removing Unknown VM's on Dead Cluster
by Ross, Travis 28 Aug '19

28 Aug '19

I've clearly seen this is a pretty common issue with many different methods for cleaning up dead clusters. I'm having issues with unknown VM's not allowing my host into Maintenance Mode. Version: ovirt-release36-3.6.7-1.noarch Situation: Another site repurposed a host entirely without properly cleaning it's status in my Engine. No option to delete host. I've attempted the "Confirm Host is Rebooted" -> Maintenance Mode. However the error is: Cannot Switch the following Hosts to Maintenance mode: ... One or more running VMs are indicated as non-migratable. The status of all the listed VM's is "UNKNOWN" w/ "?" next to them. If I look at the settings for these VM's they already have "Do not allow migration" checked. Also I'm unable to modify the VM since the Host doesn't exist. Can't just delete the Domain because the Host exists. Can't delete Storage because host is not in Maintenance Mode. Can't set Maintenance Mode because VM's can't migrate. At this point all I want to do is upgrade to latest ovirt, however I need to get this orphaned Domain out of my single node cluster, which still runs a ~10 VM's hosted on Gluster Storage. Thoughts on whether I could just backup these VM's, re-install ovirt fresh then import the VM's? Thanks, Travis Ross W 512.231.4235 M 512.547.9437 [cid:image003.png@01D55D96.2A607620] KLA-Tencor is now KLA! Learn more at kla.com <https://www.kla.com/>

1 0

Update single node environment from 4.3.3 to 4.3.5 problem
by Gianluca Cecchi 28 Aug '19

28 Aug '19

Hello, after updating hosted engine from 4.3.3 to 4.3.5 and then the only host composing the environment (plain CentOS 7.6) it seems it is not able to start vdsm daemons kernel installed with update is kernel-3.10.0-957.27.2.el7.x86_64 Same problem also if using previous running kernel 3.10.0-957.12.2.el7.x86_64 [root@ovirt01 vdsm]# uptime 00:50:08 up 25 min, 3 users, load average: 0.60, 0.67, 0.60 [root@ovirt01 vdsm]# [root@ovirt01 vdsm]# systemctl status vdsmd -l ● vdsmd.service - Virtual Desktop Server Manager Loaded: loaded (/etc/systemd/system/vdsmd.service; enabled; vendor preset: enabled) Active: failed (Result: start-limit) since Fri 2019-08-23 00:37:27 CEST; 7s ago Process: 25810 ExecStartPre=/usr/libexec/vdsm/vdsmd_init_common.sh --pre-start (code=exited, status=1/FAILURE) Aug 23 00:37:27 ovirt01.mydomain systemd[1]: Failed to start Virtual Desktop Server Manager. Aug 23 00:37:27 ovirt01.mydomain systemd[1]: Unit vdsmd.service entered failed state. Aug 23 00:37:27 ovirt01.mydomain systemd[1]: vdsmd.service failed. Aug 23 00:37:27 ovirt01.mydomain systemd[1]: vdsmd.service holdoff time over, scheduling restart. Aug 23 00:37:27 ovirt01.mydomain systemd[1]: Stopped Virtual Desktop Server Manager. Aug 23 00:37:27 ovirt01.mydomain systemd[1]: start request repeated too quickly for vdsmd.service Aug 23 00:37:27 ovirt01.mydomain systemd[1]: Failed to start Virtual Desktop Server Manager. Aug 23 00:37:27 ovirt01.mydomain systemd[1]: Unit vdsmd.service entered failed state. Aug 23 00:37:27 ovirt01.mydomain systemd[1]: vdsmd.service failed. [root@ovirt01 vdsm]# [root@ovirt01 vdsm]# pwd /var/log/vdsm [root@ovirt01 vdsm]# ll -t | head total 118972 -rw-r--r--. 1 root root 3406465 Aug 23 00:25 supervdsm.log -rw-r--r--. 1 root root 73621 Aug 23 00:25 upgrade.log -rw-r--r--. 1 vdsm kvm 0 Aug 23 00:01 vdsm.log -rw-r--r--. 1 vdsm kvm 538480 Aug 22 23:46 vdsm.log.1.xz -rw-r--r--. 1 vdsm kvm 187486 Aug 22 23:46 mom.log -rw-r--r--. 1 vdsm kvm 621320 Aug 22 22:01 vdsm.log.2.xz -rw-r--r--. 1 root root 374464 Aug 22 22:00 supervdsm.log.1.xz -rw-r--r--. 1 vdsm kvm 2097122 Aug 22 21:53 mom.log.1 -rw-r--r--. 1 vdsm kvm 636212 Aug 22 20:01 vdsm.log.3.xz [root@ovirt01 vdsm]# link to upgrade.log contents here: https://drive.google.com/file/d/17jtX36oH1hlbNUAiVhdBkVDbd28QegXG/view?usp=… link to supervdsm.log (in gzip format) here: https://drive.google.com/file/d/1l61ePU-eFS_xVHEAHnJthzTTnTyzu0MP/view?usp=… It seems that since update I get these kind of lines inside it... restore-net::DEBUG::2019-08-22 23:56:38,591::cmdutils::133::root::(exec_cmd) /sbin/tc filter del dev eth0 pref 5000 (cwd None) restore-net::DEBUG::2019-08-22 23:56:38,595::cmdutils::141::root::(exec_cmd) FAILED: <err> = 'RTNETLINK answers: Invalid argument\nWe have an error talking to the kernel\n'; <rc> = 2 [root@ovirt01 vdsm]# systemctl status supervdsmd -l ● supervdsmd.service - Auxiliary vdsm service for running helper functions as root Loaded: loaded (/usr/lib/systemd/system/supervdsmd.service; static; vendor preset: enabled) Active: active (running) since Fri 2019-08-23 00:25:17 CEST; 23min ago Main PID: 4540 (supervdsmd) Tasks: 3 CGroup: /system.slice/supervdsmd.service └─4540 /usr/bin/python2 /usr/share/vdsm/supervdsmd --sockfile /var/run/vdsm/svdsm.sock Aug 23 00:25:17 ovirt01.mydomain systemd[1]: Started Auxiliary vdsm service for running helper functions as root. [root@ovirt01 vdsm]# [root@ovirt01 vdsm]# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master ovirtmgmt state UP group default qlen 1000 link/ether b8:ae:ed:7f:17:11 brd ff:ff:ff:ff:ff:ff 3: wlan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 00:c2:c6:a4:18:c5 brd ff:ff:ff:ff:ff:ff 4: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 36:21:c1:5e:70:aa brd ff:ff:ff:ff:ff:ff 5: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 46:d8:db:81:41:4e brd ff:ff:ff:ff:ff:ff 22: ovirtmgmt: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether b8:ae:ed:7f:17:11 brd ff:ff:ff:ff:ff:ff inet 192.168.1.211/24 brd 192.168.1.255 scope global ovirtmgmt valid_lft forever preferred_lft forever [root@ovirt01 vdsm]# [root@ovirt01 vdsm]# ip route show default via 192.168.1.1 dev ovirtmgmt 192.168.1.0/24 dev ovirtmgmt proto kernel scope link src 192.168.1.211 [root@ovirt01 vdsm]# [root@ovirt01 vdsm]# brctl show bridge name bridge id STP enabled interfaces ovirtmgmt 8000.b8aeed7f1711 no eth0 [root@ovirt01 vdsm]# [root@ovirt01 vdsm]# systemctl status openvswitch ● openvswitch.service - Open vSwitch Loaded: loaded (/usr/lib/systemd/system/openvswitch.service; enabled; vendor preset: disabled) Active: active (exited) since Fri 2019-08-23 00:25:09 CEST; 26min ago Process: 3894 ExecStart=/bin/true (code=exited, status=0/SUCCESS) Main PID: 3894 (code=exited, status=0/SUCCESS) Tasks: 0 CGroup: /system.slice/openvswitch.service Aug 23 00:25:09 ovirt01.mydomain systemd[1]: Starting Open vSwitch... Aug 23 00:25:09 ovirt01.mydomain systemd[1]: Started Open vSwitch. [root@ovirt01 vdsm]# ovs-vsctl show 02539902-1788-4796-9cdf-cf11ce8436bb Bridge br-int fail_mode: secure Port br-int Interface br-int type: internal ovs_version: "2.11.0" [root@ovirt01 vdsm]# any hints? Thanks Gianluca

4 16