- Users - oVirt List Archives

Re: Can't connect "https://FQDN/ovirt-engine" after reinstall ovirt-engine
by Александр Егоров 20 Mar '19

20 Mar '19

How do I repair the ovirt-engine? engine.log in attachment

2 1

Re: vender_id syntax UserDefinedVMProperties
by Strahil 20 Mar '19

20 Mar '19

Can't you make the script check if it windows or Linux and skip if it's Linux? Best Regards, Strahil NikolovOn Mar 19, 2019 23:02, Darin Schmidt <darinschmidt(a)gmail.com> wrote: > > You also need to have this code hooked in: > cd /usr/libexec/vdsm/hooks/before_vm_start/ > vi 99_mask_kvm > > #!/usr/bin/python2 > > import hooking > domxml = hooking.read_domxml() > > hyperv = domxml.getElementsByTagName('hyperv')[0] > smm = domxml.createElement('vendor_id') > smm.setAttribute('state', 'on') > smm.setAttribute('value', '1234567890ab') > hyperv.appendChild(smm) > > features = domxml.getElementsByTagName('features')[0] > kvm = domxml.createElement('kvm') > hidden = domxml.createElement('hidden') > hidden.setAttribute('state', 'on') > kvm.appendChild(hidden) > features.appendChild(kvm) > > hooking.write_domxml(domxml) > > > only problem now is that I cant boot a linux VM with the vendor_is portion there...... > > On Mon, Mar 18, 2019 at 3:30 PM Darin Schmidt <darinschmidt(a)gmail.com> wrote: >> >> Seems that the system has to be running with bios Q35 UEFI. Standard bios does not work. System is operational now. >> >> On Mon, Mar 18, 2019, 6:30 AM Darin Schmidt <darinschmidt(a)gmail.com> wrote: >>> >>> Still no luck getting the gtx 1080 to enable inside the VM. I see the code is being generated in the xml with the hook. But I still get error code 43. Someone mentioned doing it with eufi bios and that worked for them. So when I get back from work today, perhaps ill give that a try. >>> >>> On Mon, Mar 18, 2019, 6:10 AM Darin Schmidt <darinschmidt(a)gmail.com> wrote: >>>> >>>> I have gotten the system to see the card, its in device manager. The problem seems to be that I cannot use it in the VM because from what I have been finding out is that it gets and error code 43. Nvidia drivers disable the card if it detects that its being used in a VM. I have found some code to use to hook it into the xml before_vm_starts. >>>> >>>> 99_mask_kvm >>>> #!/usr/bin/python2 >>>> >>>> import hooking >>>> domxml = hooking.read_domxml() >>>> >>>> hyperv = domxml.getElementsByTagName('hyperv')[0] >>>> smm = domxml.createElement('vendor_id') >>>> smm.setAttribute('state', 'on') >>>> smm.setAttribute('value', '1234567890ab') >>>> hyperv.appendChild(smm) >>>> >>>> features = domxml.getElementsByTagName('features')[0] >>>> kvm = domxml.createElement('kvm') >>>> hidden = domxml.createElement('hidden') >>>> hidden.setAttribute('state', 'on') >>>> kvm.appendChild(hidden) >>>> features.appendChild(kvm) >>>> >>>> hooking.write_domxml(domxml) >>>> >>>> >>>> I am currently reinstalling the drivers to see if this helps. >>>> >>>> kvm off and vender_id is now in the xml code that get generated when the VM is started. Im going off of examples Im finding online. Perhaps I just need to add the 10de to it instead of some generic # others are using. >>>> >>>> On Mon, Mar 18, 2019 at 6:02 AM Nisim Simsolo <nsimsolo(a)redhat.com> wrote: >>>>> >>>>> Hi >>>>> >>>>> Vendor ID of Nvidia is usually 10de. >>>>> You can locate 'vendor ID:

2 1

Daily reboots of Hosted Engine?
by Juhani Rautiainen 20 Mar '19

20 Mar '19

Hi! Hosted engine reboots itself almost daily. Is this by design? If not, where should I be searching for the clues why it shuts down? Someone is giving reboot order to HE because /var/log/messages in contains this: Mar 19 12:05:00 ovirtmgr qemu-ga: info: guest-shutdown called, mode: powerdown Mar 19 12:05:00 ovirtmgr systemd: Started Delayed Shutdown Service. And I'm still running v4.3.0 because upgrade to that was bit painful and haven't dared to new round. Thanks, -Juhani

3 9

vender_id syntax UserDefinedVMProperties
by Darin Schmidt 19 Mar '19

19 Mar '19

Hello all, im trying to figure out how to configure the custom properties to enable my NVIDIA card to work in the VM. Its my understanding that the drives dont work because it detects its in a VM...... Im trying to do something like this: engine-config -s UserDefinedVMProperties="kvmhidden=^(true|false)$;{type=vendor_id;state={^(on|off)$;value=^([0-9])$}}" But thats clearly not working. If I do this: engine-config -s UserDefinedVMProperties="kvmhidden=^(true|false)$;vendor_id={state=^(on|off)$;value=^([0-9])$}" It works but, the options are messed up. Im not sure how to find out the correct syntax to get this to work. Would appreciate any advice.

2 5

oVirt 4.3 - create windows2012 failed due to ovirt-imageio-proxy
by jingjie.jiang＠oracle.com 19 Mar '19

19 Mar '19

Hi, I tried to create windows2012 vm on nfs data domain, but the disk was locked. Found the error message as following: Connection to ovirt-imageio-proxy service has failed. Make sure the service is installed, configured, and ovirt-engine certificate is registered as a valid CA in the browser. Is this known issue? Thanks, Jingjie

3 2

Change cluster cpu type with hosted engine
by Fabrice SOLER 19 Mar '19

19 Mar '19

Hello, I need to create a windows 10 virtual machine but I have an error : I have a fresh ovirt installation (version 4.2.8) with an hosted engine. At the hosted engine installation there was no question about the cluster cpu type, it should be great if in the future version it could be. To change an host to another cluster this host need to be in maintenance mode, and the hosted engine will be power off. I have created another Cluster with an SandyBridge Family CPU type, but to move the hosted engine to this new cluster the hosted should be power off. Is there someone who can help ? Sincerely, --

3 3

Re: Ovirt 4.3.1 problem with HA agent
by Strahil 19 Mar '19

19 Mar '19

>> >> 1.2 All bricks healed (gluster volume heal data info summary) and no split-brain >> > >> > >> > >> > gluster volume heal data info >> > >> > Brick node-msk-gluster203:/opt/gluster/data >> > Status: Connected >> > Number of entries: 0 >> > >> > Brick node-msk-gluster205:/opt/gluster/data >> > <gfid:18c78043-0943-48f8-a4fe-9b23e2ba3404> >> > <gfid:b6f7d8e7-1746-471b-a49d-8d824db9fd72> >> > <gfid:6db6a49e-2be2-4c4e-93cb-d76c32f8e422> >> > <gfid:e39cb2a8-5698-4fd2-b49c-102e5ea0a008> >> > <gfid:5fad58f8-4370-46ce-b976-ac22d2f680ee> >> > <gfid:7d0b4104-6ad6-433f-9142-7843fd260c70> >> > <gfid:706cd1d9-f4c9-4c89-aa4c-42ca91ab827e> >> > Status: Connected >> > Number of entries: 7 >> > >> > Brick node-msk-gluster201:/opt/gluster/data >> > <gfid:18c78043-0943-48f8-a4fe-9b23e2ba3404> >> > <gfid:b6f7d8e7-1746-471b-a49d-8d824db9fd72> >> > <gfid:6db6a49e-2be2-4c4e-93cb-d76c32f8e422> >> > <gfid:e39cb2a8-5698-4fd2-b49c-102e5ea0a008> >> > <gfid:5fad58f8-4370-46ce-b976-ac22d2f680ee> >> > <gfid:7d0b4104-6ad6-433f-9142-7843fd260c70> >> > <gfid:706cd1d9-f4c9-4c89-aa4c-42ca91ab827e> >> > Status: Connected >> > Number of entries: 7 >> > >> >> Data needs healing. >> Run: cluster volume heal data full > > This does not work. Yeah, That's because my phone corrects the 'gluster' to 'cluster' Usually gluster daemons detect need of heal, but with 'gluster volume heal data full && sleep 5 && gluster volume heal data info summary && sleep 5 && gluster volume heal data info summary', you can force syncing and get the result. Let's see what happens with DNS. Best Regards, Strahil Nikolov

1 0

Hosted -engine is down and cannot be restarted
by ada per 19 Mar '19

19 Mar '19

Hello everyone, For a strange reason the hosted engine went down and I cannot restart it. I tried manually restarting it without any success can you please advice? For all the nodes the engine status is the same as the one below. --== Host nodex. (id: 6) status ==-- conf_on_shared_storage : True Status up-to-date : True Hostname : nodex Host ID : 6 Engine status : {"reason": "bad vm status", "health": "bad", "vm": "down_unexpected", "detail": "Down"} Score : 3400 stopped : False Local maintenance : False crc32 : 323a9f45 local_conf_timestamp : 2648874 Host timestamp : 2648874 Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=2648874 (Tue Mar 19 12:25:44 2019) host-id=6 score=3400 vm_conf_refresh_time=2648874 (Tue Mar 19 12:25:44 2019) conf_on_shared_storage=True maintenance=False state=GlobalMaintenance stopped=False When I try the commands root@node5# hosted-engine --vm-shutdown I ge the response: root@node5# Command VM.shutdown with args {'delay': '120', 'message': 'VM is shutting down!', 'vmID': 'a492d2eb-1dfd-470d-a141-3e55d2189275'} failed:(code=1, message=Virtual machine does not exist) But when I run : hosted-engine --vm-start I get the response: VM exists and is down, cleaning up and restarting Below you can see the # journalctl -u ovirt-ha-agent logs Mar 14 12:04:42 node7. ovirt-ha-agent[4134]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Unhandled monitoring loop exception Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 430, in start_monitoring self._monitoring_loop() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 449, in _monitoring_loop for old_state, state, delay in self.fsm: File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/fsm/machine.py", line 127, in next new_data = self.refresh(self._state.data) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/state_machine.py", line 81, in refresh stats.update(self.hosted_engine.collect_stats()) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 737, in collect_stats all_stats = self._broker.get_stats_from_storage() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 143, in get_stats_from_storage result = self._proxy.get_stats() File "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__ return self.__send(self.__name, args) File "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in __request verbose=self.__verbose File "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request return self.single_request(host, handler, request_body, verbose) File "/usr/lib64/python2.7/xmlrpclib.py", line 1301, in single_request self.send_content(h, request_body) File "/usr/lib64/python2.7/xmlrpclib.py", line 1448, in send_content connection.endheaders(request_body) File "/usr/lib64/python2.7/httplib.py", line 1037, in endheaders self._send_output(message_body) File "/usr/lib64/python2.7/httplib.py", line 881, in _send_output self.send(msg) File "/usr/lib64/python2.7/httplib.py", line 843, in send self.connect() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py", line 52, in connect self.sock.connect(base64.b16decode(self.host)) File "/usr/lib64/python2.7/socket.py", line 224, in meth return getattr(self._sock,name)(*args) error: [Errno 2] No such file or directory Mar 14 12:04:42 node7. ovirt-ha-agent[4134]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent return action(he) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper return he.start_monitoring() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 437, in start_monitoring self.publish(stopped) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 337, in publish self._push_to_storage(blocks) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 708, in _push_to_storage self._broker.put_stats_on_storage(self.host_id, blocks) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 113, in put_stats_on_storage self._proxy.put_stats(host_id, xmlrpclib.Binary(data)) File "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__ return self.__send(self.__name, args) File "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in __request verbose=self.__verbose File "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request return self.single_request(host, handler, request_body, verbose) File "/usr/lib64/python2.7/xmlrpclib.py", line 1301, in single_request self.send_content(h, request_body) File "/usr/lib64/python2.7/xmlrpclib.py", line 1448, in send_content connection.endheaders(request_body) File "/usr/lib64/python2.7/httplib.py", line 1037, in endheaders self._send_output(message_body) File "/usr/lib64/python2.7/httplib.py", line 881, in _send_output self.send(msg) File "/usr/lib64/python2.7/httplib.py", line 843, in send self.connect() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py", line 52, in connect self.sock.connect(base64.b16decode(self.host)) File "/usr/lib64/python2.7/socket.py", line 224, in meth return getattr(self._sock,name)(*args) error: [Errno 2] No such file or directory Mar 14 12:04:42 node7. ovirt-ha-agent[4134]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Mar 14 12:04:42 node7. systemd[1]: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Mar 14 12:04:42 node7. systemd[1]: Unit ovirt-ha-agent.service entered failed state. Mar 14 12:04:42 node7. systemd[1]: ovirt-ha-agent.service failed. Mar 14 12:04:52 node7. systemd[1]: ovirt-ha-agent.service holdoff time over, scheduling restart. Mar 14 12:04:52 node7. systemd[1]: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Mar 14 12:04:52 node7. systemd[1]: Started oVirt Hosted Engine High Availability Monitoring Agent. Mar 14 12:04:52 node7. ovirt-ha-agent[31765]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Mar 14 12:04:52 node7. ovirt-ha-agent[31765]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent return action(he) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper return he.start_monitoring() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 413, in start_monitoring self._initialize_broker() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 537, in _initialize_broker m.get('options', {})) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor ).format(t=type, o=options, e=e) RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'ping', options: {'addr': '19 Mar 14 12:04:52 node7. ovirt-ha-agent[31765]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Mar 14 12:04:52 node7. systemd[1]: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Mar 14 12:04:52 node7. systemd[1]: Unit ovirt-ha-agent.service entered failed state. Mar 14 12:04:52 node7. systemd[1]: ovirt-ha-agent.service failed. Mar 14 12:05:02 node7. systemd[1]: ovirt-ha-agent.service holdoff time over, scheduling restart. Mar 14 12:05:02 node7. systemd[1]: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Mar 14 12:05:02 node7. systemd[1]: Started oVirt Hosted Engine High Availability Monitoring Agent. Mar 14 12:06:55 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to stop engine vm with /usr/sbin/hosted-engine --vm-poweroff: Co (code=1, message=Virtual machine does not exist: {'vmId': u'a492d2eb-1dfd-470d-a141-3e55d2189275'}) Mar 14 12:06:55 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to stop engine VM: Command VM.destroy with args {'vmID': 'a492d2 (code=1, message=Virtual machine does not exist: {'vmId': u'a492d2eb-1dfd-470d-a141-3e55d2189275'}) Mar 15 14:28:16 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost Mar 15 14:28:36 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost Mar 15 14:29:00 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost Mar 15 14:29:22 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost Mar 15 14:29:44 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost Mar 15 14:30:06 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost Mar 15 14:30:28 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost Mar 15 14:30:50 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost Mar 15 14:31:12 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost Mar 15 14:31:33 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost Mar 15 14:31:56 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost Mar 15 14:32:18 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost Mar 15 14:32:40 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost Mar 15 14:33:02 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost

2 8

Host unresponsive after upgrade 4.2.8 -> 4.3.2 failed
by Artem Tambovskiy 19 Mar '19

19 Mar '19

Hello, Just started upgrading my small cluster to from 4.2.8 to 4.3.2 and endup in the situation that one of the hosts is not working after upgrade. For some reason vdsmd is not starting up, I have tried to restart it manually with no luck: Any ideas on what could be the reason? [root@ovirt2 log]# systemctl restart vdsmd A dependency job for vdsmd.service failed. See 'journalctl -xe' for details. [root@ovirt2 log]# journalctl -xe -- Unit ovirt-ha-agent.service has finished shutting down. Mar 19 15:47:47 ovirt2.domain.org systemd[1]: Starting Virtual Desktop Server Manager... -- Subject: Unit vdsmd.service has begun start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit vdsmd.service has begun starting up. Mar 19 15:47:47 ovirt2.domain.org vdsmd_init_common.sh[56717]: vdsm: Running mkdirs Mar 19 15:47:47 ovirt2.domain.org vdsmd_init_common.sh[56717]: vdsm: Running configure_coredump Mar 19 15:47:47 ovirt2.domain.org vdsmd_init_common.sh[56717]: vdsm: Running configure_vdsm_logs Mar 19 15:47:47 ovirt2.domain.org vdsmd_init_common.sh[56717]: vdsm: Running wait_for_network Mar 19 15:47:47 ovirt2.domain.org supervdsmd[56716]: Supervdsm failed to start: 'module' object has no attribute 'Accounting' Mar 19 15:47:47 ovirt2.domain.org python2[56716]: detected unhandled Python exception in '/usr/share/vdsm/supervdsmd' Mar 19 15:47:48 ovirt2.domain.org abrt-server[56745]: Duplicate: core backtrace Mar 19 15:47:48 ovirt2.domain.org abrt-server[56745]: DUP_OF_DIR: /var/tmp/abrt/Python-2019-03-19-14:23:04-17292 Mar 19 15:47:48 ovirt2.domain.org abrt-server[56745]: Deleting problem directory Python-2019-03-19-15:47:47-56716 (dup of Python-2019-03-19-14:23:04-17292 Mar 19 15:47:49 ovirt2.domain.org daemonAdapter[56716]: Traceback (most recent call last): Mar 19 15:47:49 ovirt2.domain.org daemonAdapter[56716]: File "/usr/share/vdsm/supervdsmd", line 26, in <module> Mar 19 15:47:49 ovirt2.domain.org daemonAdapter[56716]: supervdsm_server.main(sys.argv[1:]) Mar 19 15:47:49 ovirt2.domain.org daemonAdapter[56716]: File "/usr/lib/python2.7/site-packages/vdsm/supervdsm_server.py", line 294, in main Mar 19 15:47:49 ovirt2.domain.org daemonAdapter[56716]: module_name)) Mar 19 15:47:49 ovirt2.domain.org daemonAdapter[56716]: File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module Mar 19 15:47:49 ovirt2.domain.org daemonAdapter[56716]: __import__(name) Mar 19 15:47:49 ovirt2.domain.org daemonAdapter[56716]: File "/usr/lib/python2.7/site-packages/vdsm/supervdsm_api/systemd.py", line 34, in <module> Mar 19 15:47:49 ovirt2.domain.org daemonAdapter[56716]: cmdutils.Accounting.CPU, Mar 19 15:47:49 ovirt2.domain.org daemonAdapter[56716]: AttributeError: 'module' object has no attribute 'Accounting' Mar 19 15:47:49 ovirt2.domain.org systemd[1]: supervdsmd.service: main process exited, code=exited, status=1/FAILURE Mar 19 15:47:49 ovirt2.domain.org systemd[1]: Unit supervdsmd.service entered failed state. Mar 19 15:47:49 ovirt2.domain.org systemd[1]: supervdsmd.service failed. Mar 19 15:47:49 ovirt2.domain.org systemd[1]: supervdsmd.service holdoff time over, scheduling restart. Mar 19 15:47:49 ovirt2.domain.org systemd[1]: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Mar 19 15:47:49 ovirt2.domain.org systemd[1]: Stopped Auxiliary vdsm service for running helper functions as root. -- Subject: Unit supervdsmd.service has finished shutting down -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit supervdsmd.service has finished shutting down. Mar 19 15:47:49 ovirt2.domain.org systemd[1]: Started Auxiliary vdsm service for running helper functions as root. -- Subject: Unit supervdsmd.service has finished start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit supervdsmd.service has finished starting up. -- -- The start-up result is done. Mar 19 15:47:50 ovirt2.domain.org supervdsmd[56757]: Supervdsm failed to start: 'module' object has no attribute 'Accounting' Mar 19 15:47:50 ovirt2.domain.org python2[56757]: detected unhandled Python exception in '/usr/share/vdsm/supervdsmd' -- Regards, Artem

1 0

Re: Ovirt 4.3.1 problem with HA agent
by Strahil 19 Mar '19

19 Mar '19

Hi Alexei, >> 1.2 All bricks healed (gluster volume heal data info summary) and no split-brain > > > > gluster volume heal data info > > Brick node-msk-gluster203:/opt/gluster/data > Status: Connected > Number of entries: 0 > > Brick node-msk-gluster205:/opt/gluster/data > <gfid:18c78043-0943-48f8-a4fe-9b23e2ba3404> > <gfid:b6f7d8e7-1746-471b-a49d-8d824db9fd72> > <gfid:6db6a49e-2be2-4c4e-93cb-d76c32f8e422> > <gfid:e39cb2a8-5698-4fd2-b49c-102e5ea0a008> > <gfid:5fad58f8-4370-46ce-b976-ac22d2f680ee> > <gfid:7d0b4104-6ad6-433f-9142-7843fd260c70> > <gfid:706cd1d9-f4c9-4c89-aa4c-42ca91ab827e> > Status: Connected > Number of entries: 7 > > Brick node-msk-gluster201:/opt/gluster/data > <gfid:18c78043-0943-48f8-a4fe-9b23e2ba3404> > <gfid:b6f7d8e7-1746-471b-a49d-8d824db9fd72> > <gfid:6db6a49e-2be2-4c4e-93cb-d76c32f8e422> > <gfid:e39cb2a8-5698-4fd2-b49c-102e5ea0a008> > <gfid:5fad58f8-4370-46ce-b976-ac22d2f680ee> > <gfid:7d0b4104-6ad6-433f-9142-7843fd260c70> > <gfid:706cd1d9-f4c9-4c89-aa4c-42ca91ab827e> > Status: Connected > Number of entries: 7 > Data needs healing. Run: cluster volume heal data full If it still doesn't heal (check in 5 min),go to /rhev/data-center/mnt/glusterSD/msk-gluster-facility.xxxx_data And run 'find . -exec stat {}\;' without the quotes. As I have understood you, ovirt Hosted Engine is running and can be started on all nodes except 1. >> >> 2. Go to the problematic host and check the mount point is there > > > > No mount point on problematic node /rhev/data-center/mnt/glusterSD/msk-gluster-facility.xxxx:_data > If I create a mount point manually, it is deleted after the node is activated. > > Other nodes can mount this volume without problems. Only this node have connection problems after update. > > Here is a part of the log at the time of activation of the node: > > vdsm log > > 2019-03-18 16:46:00,548+0300 INFO (jsonrpc/5) [vds] Setting Hosted Engine HA local maintenance to False (API:1630) > 2019-03-18 16:46:00,549+0300 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.setHaMaintenanceMode succeeded in 0.00 seconds (__init__:573) > 2019-03-18 16:46:00,581+0300 INFO (jsonrpc/7) [vdsm.api] START connectStorageServer(domType=7, spUUID=u'5a5cca91-01f8-01af-0297-00000000025f', conList=[{u'id': u'5799806e-7969-45da-b17d-b47a63e6a8e4', u'connection': u'msk-gluster-facility.xxxx:/data', u'iqn': u'', u'user': u'', u'tpgt': u'1', u'vfs_type': u'glusterfs', u'password': '********', u'port': u''}], options=None) from=::ffff:10.77.253.210,56630, flow_id=81524ed, task_id=5f353993-95de-480d-afea-d32dc94fd146 (api:46) > 2019-03-18 16:46:00,621+0300 INFO (jsonrpc/7) [storage.StorageServer.MountConnection] Creating directory u'/rhev/data-center/mnt/glusterSD/msk-gluster-facility.xxxx:_data' (storageServer:167) > 2019-03-18 16:46:00,622+0300 INFO (jsonrpc/7) [storage.fileUtils] Creating directory: /rhev/data-center/mnt/glusterSD/msk-gluster-facility.xxxx:_data mode: None (fileUtils:197) > 2019-03-18 16:46:00,622+0300 WARN (jsonrpc/7) [storage.StorageServer.MountConnection] gluster server u'msk-gluster-facility.xxxx' is not in bricks ['node-msk-gluster203', 'node-msk-gluster205', 'node-msk-gluster201'], possibly mounting duplicate servers (storageServer:317) This seems very strange. As you have hidden the hostname, I'm not use which on is this. Check that DNS can be resolved from all hosts and the hostname of this Host is resolvable. Also check if it in the peer list. Try to manually mount the cluster volume: mount -t glusterfs msk-gluster-facility.xxxx:/data /mnt Is this a second FQDN/IP of this server? If so, gluster accepts that via gluster peer probe IP2 >> 2.1. Check permissions (should be vdsm:kvm) and fix with chown -R if needed >> 2.2. Check the OVF_STORE from the logs that it exists > > > How can i do this? Go to /rhev/data-center/mnt/glusterSD/host_engine and use find inside the domain UUID for files that are not owned by vdsm:KVM. I usually run 'chown -R vdsm:KVM 823xx-xxxx-yyyy-zzz' and it will fix any misconfiguration. Best Regards, Strahil Nikolov

2 1

Live migration failed
by Bong Shau Fui 19 Mar '19

19 Mar '19

Hi: I deployed 2 ovirt hosts and an ovirt engine in a nested KVM server. I've a windows vm setup and tried to perform live migration but failed. I checked on the hosts and found them meeting the live migration requirements, or at least that's what I thought. I took the requirement from the below document. https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Virtualiza… The hosts, both source and destination are quite empty, with only the hosted engine, 1 centos vm and the windows VM in the cluster. I can do a live migration for the centos vm successfully. But when I tried live migration on the hosted-engine vm it failed immediately with a message "No available host to migrate VMs to". When I tried to migrate the windows VM the message box that let me choose the destination host popped up but failed after a while. I'd like to ask where can I get more information with regards to live-migration apart from /var/log/ovirt-engine/engine.log ? I also checked on the ovirt hosts' /var/log/vdsm/vdsm.log but found nothing pointing to the reason why it failed. Below is the extract from /var/log/ovirt-engine/engine.log when the live-migration took place 2019-03-12 14:37:58,159+08 INFO [org.ovirt.engine.core.sso.utils.AuthenticationUtils] (default task-131) [] User admin@internal successfully logged in with scopes: ovirt-app-api ovirt-ext=token-info:authz-search ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate ovirt-ext=token:password-access 2019-03-12 14:37:58,450+08 INFO [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-59) [77888830] Lock freed to object 'EngineLock:{exclusiveLocks='[d113be83-2740-4246-a1f2-b9344889c3cf=PROVIDER]', sharedLocks=''}' 2019-03-12 14:38:02,544+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-50) [] BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2019-03-12 14:38:12,677+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-16) [] BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2019-03-12 14:38:21,650+08 INFO [org.ovirt.engine.core.bll.aaa.SessionDataContainer] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [] Not removing session 'xDiHqqa6l+g8cngM26TTCfW7NeLN3WgWChsx28wUM391vAngSxwtyCkLbQxZR1AbJ5I+2bkPZNQijMUk0jLZcA==', session has running commands for user 'admin@internal-authz'. 2019-03-12 14:38:22,782+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-49) [] BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2019-03-12 14:38:33,018+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [] BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2019-03-12 14:38:43,261+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-59) [77888830] BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2019-03-12 14:38:53,528+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-13) [] BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2019-03-12 14:39:03,759+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [] BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2019-03-12 14:39:14,011+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-60) [] BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2019-03-12 14:39:21,660+08 INFO [org.ovirt.engine.core.bll.aaa.SessionDataContainer] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [] Not removing session 'xDiHqqa6l+g8cngM26TTCfW7NeLN3WgWChsx28wUM391vAngSxwtyCkLbQxZR1AbJ5I+2bkPZNQijMUk0jLZcA==', session has running commands for user 'admin@internal-authz'. 2019-03-12 14:39:24,122+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [] BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2019-03-12 14:39:29,773+08 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (default task-131) [7f0cf113-55e8-4def-9c68-de3b91d6d641] Lock Acquired to object 'EngineLock:{exclusiveLocks='[5cad5c5f-5aab-46ec-a28e-d484abc0401d=VM]', sharedLocks=''}' 2019-03-12 14:39:29,887+08 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (default task-131) [7f0cf113-55e8-4def-9c68-de3b91d6d641] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: 5cad5c5f-5aab-46ec-a28e-d484abc0401d Type: VMAction group MIGRATE_VM with role type USER 2019-03-12 14:39:30,019+08 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (default task-131) [7f0cf113-55e8-4def-9c68-de3b91d6d641] START, MigrateVDSCommand( MigrateVDSCommandParameters:{hostId='f9014bc4-485c-4eb0-a9bc-42d13ed68f41', vmId='5cad5c5f-5aab-46ec-a28e-d484abc0401d', srcHost='host2.xxxx.com', dstVdsId='1bc9b9e9-1e90-4570-9930-08416d1927cc', dstHost='host3.xxxx.com:54321', migrationMethod='ONLINE', tunnelMigration='false', migrationDowntime='0', autoConverge='true', migrateCompressed='false', consoleAddress='null', maxBandwidth='null', enableGuestEvents='true', maxIncomingMigrations='2', maxOutgoingMigrations='2', convergenceSchedule='[init=[{name=setDowntime, params=[100]}], stalling=[{limit=1, action={name=setDowntime, params=[150]}}, {limit=2, action={name=setDowntime, params=[200]}}, {limit=3, action={name=setDowntime, params=[300]}}, {limit=4, action={name=setDowntime, params=[400]}}, {limit=6, action={name=setDowntime, params=[500]}}, {limit=-1, action={nam e=abort, params=[]}}]]', dstQemu='192.168.138.135'}), log id: 7eeb678c 2019-03-12 14:39:30,022+08 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (default task-131) [7f0cf113-55e8-4def-9c68-de3b91d6d641] START, MigrateBrokerVDSCommand(HostName = host2.xxxx.com, MigrateVDSCommandParameters:{hostId='f9014bc4-485c-4eb0-a9bc-42d13ed68f41', vmId='5cad5c5f-5aab-46ec-a28e-d484abc0401d', srcHost='host2.xxxx.com', dstVdsId='1bc9b9e9-1e90-4570-9930-08416d1927cc', dstHost='host3.xxxx.com:54321', migrationMethod='ONLINE', tunnelMigration='false', migrationDowntime='0', autoConverge='true', migrateCompressed='false', consoleAddress='null', maxBandwidth='null', enableGuestEvents='true', maxIncomingMigrations='2', maxOutgoingMigrations='2', convergenceSchedule='[init=[{name=setDowntime, params=[100]}], stalling=[{limit=1, action={name=setDowntime, params=[150]}}, {limit=2, action={name=setDowntime, params=[200]}}, {limit=3, action={name=setDowntime, params=[300]}}, {limit=4, action={name=setDowntime, params=[400]}}, {limit=6, action={name=set Downtime, params=[500]}}, {limit=-1, action={name=abort, params=[]}}]]', dstQemu='192.168.138.135'}), log id: 5cef4981 2019-03-12 14:39:30,039+08 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (default task-131) [7f0cf113-55e8-4def-9c68-de3b91d6d641] FINISH, MigrateBrokerVDSCommand, return: , log id: 5cef4981 2019-03-12 14:39:30,048+08 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (default task-131) [7f0cf113-55e8-4def-9c68-de3b91d6d641] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 7eeb678c 2019-03-12 14:39:30,067+08 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-131) [7f0cf113-55e8-4def-9c68-de3b91d6d641] EVENT_ID: VM_MIGRATION_START(62), Migration started (VM: Win_2016_1, Source: host2.xxxx.com, Destination: host3, User: admin@internal-authz). 2019-03-12 14:39:33,901+08 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-3) [] VM '5cad5c5f-5aab-46ec-a28e-d484abc0401d' was reported as Down on VDS '1bc9b9e9-1e90-4570-9930-08416d1927cc'(host3) 2019-03-12 14:39:33,903+08 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (ForkJoinPool-1-worker-3) [] START, DestroyVDSCommand(HostName = host3, DestroyVmVDSCommandParameters:{hostId='1bc9b9e9-1e90-4570-9930-08416d1927cc', vmId='5cad5c5f-5aab-46ec-a28e-d484abc0401d', secondsToWait='0', gracefully='false', reason='', ignoreNoVm='true'}), log id: c853ba5 2019-03-12 14:39:34,211+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-73) [] BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2019-03-12 14:39:34,604+08 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (ForkJoinPool-1-worker-3) [] Failed to destroy VM '5cad5c5f-5aab-46ec-a28e-d484abc0401d' because VM does not exist, ignoring 2019-03-12 14:39:34,605+08 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (ForkJoinPool-1-worker-3) [] FINISH, DestroyVDSCommand, return: , log id: c853ba5 2019-03-12 14:39:34,605+08 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-3) [] VM '5cad5c5f-5aab-46ec-a28e-d484abc0401d'(Win_2016_1) was unexpectedly detected as 'Down' on VDS '1bc9b9e9-1e90-4570-9930-08416d1927cc'(ohost3) (expected on 'f9014bc4-485c-4eb0-a9bc-42d13ed68f41') 2019-03-12 14:39:34,605+08 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-3) [] Migration of VM 'Win_2016_1' to host 'host3' failed: VM destroyed during the startup. 2019-03-12 14:39:34,615+08 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-10) [] VM '5cad5c5f-5aab-46ec-a28e-d484abc0401d'(Win_2016_1) moved from 'MigratingFrom' --> 'Up' 2019-03-12 14:39:34,615+08 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-10) [] Adding VM '5cad5c5f-5aab-46ec-a28e-d484abc0401d'(Win_2016_1) to re-run list 2019-03-12 14:39:34,621+08 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.VmsMonitoring] (ForkJoinPool-1-worker-10) [] Rerun VM '5cad5c5f-5aab-46ec-a28e-d484abc0401d'. Called from VDS 'host2.xxxx.com' 2019-03-12 14:39:34,752+08 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-53959) [] START, MigrateStatusVDSCommand(HostName = host2.xxxx.com, MigrateStatusVDSCommandParameters:{hostId='f9014bc4-485c-4eb0-a9bc-42d13ed68f41', vmId='5cad5c5f-5aab-46ec-a28e-d484abc0401d'}), log id: 7ded4ad7 2019-03-12 14:39:34,760+08 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-53959) [] FINISH, MigrateStatusVDSCommand, return: , log id: 7ded4ad7 2019-03-12 14:39:34,786+08 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-53959) [] EVENT_ID: VM_MIGRATION_TO_SERVER_FAILED(120), Migration failed (VM: Win_2016_1, Source: host2.xxxx.com, Destination: host3). Any help is greatly appreciated. regards, Bong SF

2 1

Re: VM has been paused due to a storage I/O error
by Strahil 19 Mar '19

19 Mar '19

This should not happen with replica 3 volumes. Are you sure you don't have a fluster brick out of sync/disconnected ? Best Regards, Strahil Nikolov

3 2

Where to find the hooks' print
by zodaoko＠gmail.com 19 Mar '19

19 Mar '19

Hi there, I created a before_set_num_of_cpus hook: # more /usr/libexec/vdsm/hooks/before_set_num_of_cpus/before.py #!/usr/bin/python import os import sys if os.environ.has_key('never_existed'): sys.stderr.write('cantsetcpu: before_cpu_set: cannot set cpu.\n') sys.exit(2) else: sys.stdout.write('hook ok.\n') sys.exit(0) But I cannot find the message "hook ok" in engine.log or vdsm.log, where to find it? Thank you very much. Thank you, -Zhen

1 0

Host affinity hard rule doesn't work
by zodaoko＠gmail.com 19 Mar '19

19 Mar '19

Hi there, Here is my setup: oVirt engine: 4.2.8 1. Create an affinity group as below: VM affinity rule: positive + enforcing Host affinity rule: disabled. VMs: 2 VMs added Hosts: No host selected. 2. Run the 2 VMs, they are running on the same host, say host1. 3. Change the affinity group's host affinity: Host affinity rule: positive + enforcing Hosts: host2 added. I expect that the 2 VMs can migrate to host2, but that never happen, is this expected? snippet of engine.log: 2019-03-13 07:47:05,747Z INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] Candidate host 'dub-svrfarm24' ('76b13e75-d01b-4dec-9298-1fad72b46525') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'VmAffinityGroups' (correlation id: null) 2019-03-13 07:47:05,747Z DEBUG [org.ovirt.engine.core.bll.scheduling.arem.AffinityRulesEnforcer] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] VM 822b37b7-5da3-453c-b775-d4192c2fdcae is NOT a viable candidate for solving the affinity group violation situation. 2019-03-13 07:47:05,747Z DEBUG [org.ovirt.engine.core.bll.scheduling.arem.AffinityRulesEnforcer] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] No vm to hosts soft-affinity group violation detected 2019-03-13 07:47:05,749Z DEBUG [org.ovirt.engine.core.bll.scheduling.arem.AffinityRulesEnforcer] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] No affinity group collision detected for cluster 8fe88b8c-966c-4c21-839d-e2437cc6b73d. Standing by. 2019-03-13 07:47:05,749Z DEBUG [org.ovirt.engine.core.bll.scheduling.arem.AffinityRulesEnforcer] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] No affinity group collision detected for cluster 3beac2ea-ed04-4f40-9ce3-5a9a67cebd8c. Standing by. 2019-03-13 07:47:05,750Z DEBUG [org.ovirt.engine.core.bll.scheduling.arem.AffinityRulesEnforcer] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] No affinity group collision detected for cluster da32d154-4303-11e9-9607-00163eaab080. Standing by. Thank you, -Zhen

3 4

Forum available
by Josep Manel Andrés Moscardó 18 Mar '19

18 Mar '19

Hi all, I am just wondering if anyone like me would like to have everything that is bump here in a forum, with all the benefits it brings (and people will still be able to subscribe and reply through email). Something like Discourse would be nice in my opinion. Best.

10 22

Cancel storage migration task?
by Alan G 18 Mar '19

18 Mar '19

Hi, I accidentally triggered a storage migration for a large vdisk that will take some hours to complete. Is there a way to cleanly cancel the task, such that the vdisk will remain on the original domain? Thanks, Alan

2 4

Hosted Engine I/O scheduler
by Strahil 18 Mar '19

18 Mar '19

Dear All, I have just noticed that my Hosted Engine has a strange I/O scheduler: Last login: Sun Mar 17 18:14:26 2019 from 192.168.1.43 [root@engine ~]# cat /sys/block/vda/queue/scheduler [mq-deadline] kyber none [root@engine ~]# Based on my experience anything than noop/none is useless and performance degrading for a VM. Is there any reason that we have this scheduler ? It is quite pointless to process (and delay) the I/O in the VM and then process (and again delay) on Host Level . If there is no reason to keep the deadline, I will open a bug about it. Best Regards, Strahil Nikolov

3 2

oVirt 4.3.2 RC - ISCSI LUN
by a.e.pool＠iway.na 18 Mar '19

18 Mar '19

I have deployed the latest oVirt 4.3.2 host and the hosted engine without problems, until you have to select the storage. I configured a brand new FReenAS 11 with ISCSI on three 1TB HDDs, the oVirt engine sees the iscsi target but when trying to select the LUN it shows that the LUN is already in use. Anyone come across this issue?

1 0

Ovirt 4.3.1 problem with HA agent
by Strahil 18 Mar '19

18 Mar '19

Hi guys, After updating to 4.3.1 I had an issue where the ovirt-ha-broker was complaining that it couldn't ping the gateway. As I have seen that before - I stopped ovirt-ha-agent, ovirt-ha-broker, vdsmd, supervdsmd and sanlock on the nodes and reinitialized the lockspace. I gues s I didn't do it properly as now I receive: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf Any hints how to fix this ? Of course a redeploy is possible, but I prefer to recover from that. Best Regards, Strahil Nikolov

4 21

Bandwidth problem
by Leo David 18 Mar '19

18 Mar '19

Hello Everyone, I have 10Gb connections setup for all the hosts in the cluster, for both management/vm and gluster traffic ( separate network cards ) The problem is that i just cannot pass 1Gb/s traffic between vms ( even between vms running on the same hosts ! - which makes the things more weird... ). Traffic measured by using iperf tool. Is there a way I can check waht could be the problem ? Network card type, vm drivers, any suggestion ? I just do not know where to look for a possible cause. Thank you very much ! Leo -- Best regards, Leo David

2 1

Re: self-hosted ovirt-engine down
by Strahil 18 Mar '19

18 Mar '19

In order to be easier to read, you can rotate the log before starting the engine.Should be something like: hosted-engine --set-maintenance --mode=global && logrotate -f /etc/vdsm/logrotate/vdsm && hosted-engine -- vm-start Best Regards, Strahil NikolovOn Mar 16, 2019 10:52, Simone Tiraboschi <stirabos(a)redhat.com> wrote: > > > > On Sat, Mar 16, 2019 at 7:44 AM <siovelrm(a)gmail.com> wrote: >> >> Hi, I have a big problem with ovirt. I use version 4.2.7 with self-hosted. The problem is that when I try to raise the vm of the ovirt-engine with the command: hosted-engine --vm-start, it appears in the output >> "VM exists and is down, cleaning up and restarting" >> when running: hosted-engine --vm-status appears >> --== Host 1 status ==-- >> >> conf_on_shared_storage : True >> Status up-to-date : True >> Hostname : node1.softel.cu >> Host ID : 1 >> Engine status : {"reason": "bad vm status", "health": "bad", "vm": "down_unexpected", "detail": "Down"} >> Score : 0 >> stopped : False >> Local maintenance : False >> crc32 : 02c3b5a4 >> local_conf_timestamp : 49529 >> Host timestamp : 49529 >> Extra metadata (valid at timestamp): >> metadata_parse_version=1 >> metadata_feature_version=1 >> timestamp=49529 (Sat Mar 16 02:39:10 2019) >> host-id=1 >> score=0 >> vm_conf_refresh_time=49529 (Sat Mar 16 02:39:11 2019) >> conf_on_shared_storage=True >> maintenance=False >> state=EngineUnexpectedlyDown >> stopped=False >> timeout=Thu Jan 1 08:49:39 1970 >> >> in /var/log/messages >> Mar 16 02:35:34 node1 vdsm[26151]: WARN Attempting to remove a non existing network: ovirtmgmt/0c3e1c08-3928-47f1-96a8-c6a8d0dc3241 >> Mar 16 02:35:34 node1 vdsm[26151]: WARN Attempting to remove a non existing net user: ovirtmgmt/0c3e1c08-3928-47f1-96a8-c6a8d0dc3241 >> Mar 16 02:35:34 node1 vdsm[26151]: WARN Attempting to remove a non existing network: ovirtmgmt/0c3e1c08-3928-47f1-96a8-c6a8d0dc3241 >> Mar 16 02:35:34 node1 vdsm[26151]: WARN Attempting to remove a non existing net user: ovirtmgmt/0c3e1c08-3928-47f1-96a8-c6a8d0dc3241 >> Mar 16 02:35:34 node1 vdsm[26151]: WARN File: /var/lib/libvirt/qemu/channels/0c3e1c08-3928-47f1-96a8-c6a8d0dc3241.org.qemu.guest_agent.0 already removed >> >> Please help!! > > > I'd suggest to check in /var/log/vdsm/vdsm.log why it failed to start. > > >> >> _______________________________________________ >> Users mailing list -- users(a)ovirt.org >> To unsubscribe send an email to users-leave(a)ovirt.org >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/OTXJFCVCAHMYZ…

2 1

Error creating a storage domain (On Cisco UCS Only)
by nico.kruger＠darkmatter.ae 18 Mar '19

18 Mar '19

Hi Guys, I am trying to install a new cluster... I currently have a 9 node and two 6 node oVirt Clusters... (these were installed on 4.1 and upgraded to 4.2) So i want to build a new cluster, which is working fine on this HP notebook i use for testing (using single node gluster deployment) But when i try install this on my production servers which are Cisco UCS servers i keep getting this error: [ ERROR ] Error: Fault reason is "Operation Failed". Fault detail is "[Error creating a storage domain]". HTTP response code is 400. [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "Fault reason is \"Operation Failed\". Fault detail is \"[Error creating a storage domain]\". HTTP response code is 400."} This happens during storage creation after the hosted-engine is built and after gluster has been deployed (error happens for both single and 3 replica deployments) I just cant see how an install on one type of server is successful but not on the UCS servers (which i am running my other ovirt clusters on) BTW i dont think the issue is related to Gluster Storage Create as i tried using NFS and Local storage and get the same error (on UCS server only) I am using the ovirt-node-ng-installer-4.2.0-2019011406.el7.iso install ISO Below is a tail from ovirt-hosted-engine-setup-ansible-create_storage_domain Log file 2019-01-27 11:09:49,754+0400 INFO ansible ok {'status': 'OK', 'ansible_task': u'Fetch Datacenter name', 'ansible_host': u'localhost', 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/create_storage_domain.yml', 'ansible_type': 'task'} 2019-01-27 11:09:49,754+0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f9f9b7e2d50> kwargs 2019-01-27 11:09:50,478+0400 INFO ansible task start {'status': 'OK', 'ansible_task': u'Add NFS storage domain', 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/create_storage_domain.yml', 'ansible_type': 'task'} 2019-01-27 11:09:50,479+0400 DEBUG ansible on_any args TASK: Add NFS storage domain kwargs is_conditional:False 2019-01-27 11:09:51,151+0400 DEBUG var changed: host "localhost" var "otopi_storage_domain_details_nfs" type "<type 'dict'>" value: "{ "changed": false, "skip_reason": "Conditional result was False", "skipped": true }" 2019-01-27 11:09:51,151+0400 INFO ansible skipped {'status': 'SKIPPED', 'ansible_task': u'Add NFS storage domain', 'ansible_host': u'localhost', 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/create_storage_domain.yml', 'ansible_type': 'task'} 2019-01-27 11:09:51,151+0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f9f9b7e2610> kwargs 2019-01-27 11:09:51,820+0400 INFO ansible task start {'status': 'OK', 'ansible_task': u'Add glusterfs storage domain', 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/create_storage_domain.yml', 'ansible_type': 'task'} 2019-01-27 11:09:51,821+0400 DEBUG ansible on_any args TASK: Add glusterfs storage domain kwargs is_conditional:False 2019-01-27 11:10:02,045+0400 DEBUG var changed: host "localhost" var "otopi_storage_domain_details_gluster" type "<type 'dict'>" value: "{ "changed": false, "exception": "Traceback (most recent call last):\n File \"/tmp/ansible_ovirt_storage_domain_payload_Xous24/__main__.py\", line 682, in main\n ret = storage_domains_module.create()\n File \"/tmp/ansible_ovirt_storage_domain_payload_Xous24/ansible_ovirt_storage_domain_payload.zip/ansible/module_utils/ovirt.py\", line 587, in create\n **kwargs\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py\", line 24225, in add\n return self._internal_add(storage_domain, headers, query, wait)\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py\", line 232, in _internal_add\n return future.wait() if wait else future\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py\", line 55, in wait\n return self._code(response)\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py\", line 229, in callback\n self._check_fault(response)\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py\", line 132, in _check_fault\n self._raise_error(response, body)\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py\", line 118, in _raise_error\n raise error\nError: Fault reason is \"Operation Failed\". Fault detail is \"[Error creating a storage domain]\". HTTP response code is 400.\n", "failed": true, "msg": "Fault reason is \"Operation Failed\". Fault detail is \"[Error creating a storage domain]\". HTTP response code is 400." }" 2019-01-27 11:10:02,045+0400 DEBUG var changed: host "localhost" var "ansible_play_hosts" type "<type 'list'>" value: "[]" 2019-01-27 11:10:02,045+0400 DEBUG var changed: host "localhost" var "play_hosts" type "<type 'list'>" value: "[]" 2019-01-27 11:10:02,045+0400 DEBUG var changed: host "localhost" var "ansible_play_batch" type "<type 'list'>" value: "[]" 2019-01-27 11:10:02,046+0400 ERROR ansible failed {'status': 'FAILED', 'ansible_type': 'task', 'ansible_task': u'Add glusterfs storage domain', 'ansible_result': u'type: <type \'dict\'>\nstr: {\'_ansible_parsed\': True, u\'exception\': u\'Traceback (most recent call last):\\n File "/tmp/ansible_ovirt_storage_domain_payload_Xous24/__main__.py", line 682, in main\\n ret = storage_domains_module.create()\\n File "/tmp/ansible_ovirt_storage_domain_payload_Xous24/ansible_ovirt_storage_domain_pay', 'ansible_host': u'localhost', 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/create_storage_domain.yml'} 2019-01-27 11:10:02,046+0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f9f9b859f50> kwargs ignore_errors:None 2019-01-27 11:10:02,048+0400 INFO ansible stats {'status': 'FAILED', 'ansible_playbook_duration': 39.840701, 'ansible_result': u"type: <type 'dict'>\nstr: {u'localhost': {'unreachable': 0, 'skipped': 2, 'ok': 13, 'changed': 0, 'failures': 1}}", 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/create_storage_domain.yml', 'ansible_type': 'finish'} 2019-01-27 11:10:02,048+0400 DEBUG ansible on_any args <ansible.executor.stats.AggregateStats object at 0x7f9f9da59a50> kwargs This issue is driving me crazy... any assistance on what i can do would be greatly appreciated Nico

5 8

vdsm should decouple with managed glusterfs services
by levin＠mydream.com.hk 18 Mar '19

18 Mar '19

Hi, I had experience two time of 3-node hyper-converged 4.2.8 ovirt cluster total outage due to vdsm reactivate the unresponsive node, and cause the multiple glusterfs daemon restart. As a result, all VM was paused and some of disk image was corrupted. At the very beginning, one of the ovirt node was overloaded due to high memory and CPU, the hosted-engine have trouble to collect status from vdsm and mark it as unresponsive and it start migrate the workload to healthy node. However, when it start migrate, second ovirt node being unresponsive where vdsm try reactivate the 1st unresponsive node and restart it's glusterd. So the gluster domain was acquiring the quorum and waiting for timeout. If 1st node reactivation is success and every other node can survive the timeout, it will be an idea case. Unfortunately, the second node cannot pick up the VM being migrated due to gluster I/O timeout, so second node at that moment was marked as unresponsive, and so on... vdsm is restarting the glusterd on the second node which cause disaster. All node are racing on gluster volume self-healing, and i can't mark the cluster as maintenance mode as well. What I can do is try to resume the paused VM via virsh and issue shutdown for each domain, also hard shutdown for un-resumable VM. After number of VM shutdown and wait the gluster healing completed, the cluster state back to normal, and I try to start the VM being manually stopped, most of them can be started normally, but number of VM was crashed or un-startable, instantly I found the image files of un-startable VM was owned by root(can't explain why), and can be restarted after chmod. Two of them still cannot start with "bad volume specification" error. One of them can start to boot loader, but the LVM metadata were lost. The impact was huge when vdsm restart the glusterd without human invention.

3 3

Enabled Base and Update repos?
by nico.kruger＠darkmatter.ae 18 Mar '19

18 Mar '19

Hi Guys, Should Centos Base and Update repos be enabled on ovirt hosts or only the dependancies and ovirt repos (the ones from rpm release)

1 0

Libgfapisupport messes disk image ownership
by Hesham Ahmed 18 Mar '19

18 Mar '19

I had reported this here: https://bugzilla.redhat.com/show_bug.cgi?id=1687126 Has anyone else faced this with 4.3.1?

3 3