- Users - oVirt List Archives

Re: Can't connect "https://FQDN/ovirt-engine" after reinstall ovirt-engine
by Александр Егоров 21 Mar '19

21 Mar '19

On the new system, I restored from the backup ovirt engine 4.1. Restored without errors engine-backup --mode=restore --file=file_name --log=log_file_name --provision-db --provision-dwh-db --restore-permissions Also restored /etc/pki from backup. I did not know what should be the rights to the folders, now they are: drwxr-xr-x. 7 root root 213 мар 21 06:17 CA drwxr-xr-x. 4 root root 73 мар 20 23:53 ca-trust drwxr-xr-x. 2 root root 123 мар 21 06:17 fwupd drwxr-xr-x. 2 root root 92 мар 21 06:17 fwupd-metadata drwxr-xr-x. 2 root root 21 мар 20 23:53 java drwxr-xr-x. 2 root root 103 мар 20 23:53 nssdb drwxr-xr-x. 2 root root 30 мар 20 23:53 nss-legacy drwxr-xr-x. 6 ovirt ovirt 4,0K мар 21 05:49 ovirt-engine drwxr-xr-x. 2 ovirt-vmconsole ovirt-vmconsole 142 окт 14 2016 ovirt-vmconsole drwxr-xr-x. 2 root root 243 мар 21 06:17 rpm-gpg drwxr-xr-x. 2 root root 6 окт 31 08:08 rsyslog drwxr-xr-x. 5 root root 104 мар 21 06:17 tls in the ovirt-engine directory for all files, the owner is ovirt, for the ovirt-vmconsole directory for all files is the owner for ovirt-vmconsole I can connect to the web interface at "https://FQDN/ovirt-engine", I can log in using my login/password, but the Data Center is not active. Here is what it writes in the engine.log: 2019-03-21 16:03:02,029+09 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to ovirt-n1.vs.jak/192.168.0.6 2019-03-21 16:03:04,386+09 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (DefaultQuartzScheduler9) [5ecc17d5] Failure to refresh host 'ovirt-n1.vs.jak' runtime info: org/apache/commons/lang/StringUtils 2019-03-21 16:03:04,387+09 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler9) [5ecc17d5] Failed to invoke scheduled method onTimer: null 2019-03-21 16:03:04,419+09 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to ovirt-n2.vs.jak/192.168.0.7 2019-03-21 16:03:05,487+09 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (DefaultQuartzScheduler10) [8bfc0e5] Failure to refresh host 'ovirt-n2.vs.jak' runtime info: org/apache/commons/lang/StringUtils 2019-03-21 16:03:05,488+09 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler10) [8bfc0e5] Failed to invoke scheduled method onTimer: null 2019-03-21 16:03:07,436+09 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (DefaultQuartzScheduler9) [5e8dcb07] Failure to refresh host 'ovirt-n1.vs.jak' runtime info: org/apache/commons/lang/StringUtils 2019-03-21 16:03:07,437+09 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler9) [5e8dcb07] Failed to invoke scheduled method onTimer: null 2019-03-21 16:03:08,521+09 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (DefaultQuartzScheduler10) [486f9b4] Failure to refresh host 'ovirt-n2.vs.jak' runtime info: org/apache/commons/lang/StringUtils 2019-03-21 16:03:08,521+09 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler10) [486f9b4] Failed to invoke scheduled method onTimer: null

1 0

Re: Hosted Engine I/O scheduler
by Strahil 21 Mar '19

21 Mar '19

Unfortunately, the ideal scheduler really depends on storage configuration. Gluster, ZFS, iSCSI, FC, and NFS don't align on a single "best" configuration (to say nothing of direct LUNs on guests), then there's workload considerations. > > The scale team is aiming for a balanced "default" policy rather than one which is best for a specific environment. > > That said, I'm optimistic that the results will let us give better recommendations if your workload/storage benefits from a different scheduler I completely disagree ! If you use anything other than noop/none (depending if multiqueue is on), your scheduler inside the VM will reorder and delay your I/O. Then the I/O will be received by the Host and this repeats again. I can point to SuSe and Red Hat knowledge base where both vendors highly recommend noop/none as schedulers for VM. It has nothing in common with the backend - that's in control of the hosts I/O scheduler. Can some one tell me under which section should I open a bug ? Bugzilla is not newbie-friendly and I should admit that opening bugs for RHEL/CentOS is far easier. The best bug section might be ovirt appliance - related , as this is only valid for VMs and not bare-metal Engine. Best Regards, Strahil Nikolov

2 1

Re: Hosted Engine I/O scheduler
by Strahil 21 Mar '19

21 Mar '19

Hi Darrel, Still, based on my experience we shouldn't queue our I/O in the VM, just to do the same in the Host. I'm still considering if I should keep deadline in my hosts or to switch to 'cfq'. After all, I'm using Hyper-converged oVirt and this needs testing. What I/O scheduler are you using on the host? Best Regards, Strahil NikolovOn Mar 18, 2019 19:15, Darrell Budic <budic(a)onholyground.com> wrote: > > Checked this on mine, see the same thing. Switching the engine to noop definitely feels more responsive. > > I checked on some VMs as well, it looks like virtio drives (vda, vdb….) get mq-deadline by default, but virtscsi gets noop. I used to think the tuned profile for virtual-guest would set noop, but apparently not… > > -Darrell > >> On Mar 18, 2019, at 1:58 AM, Strahil Nikolov <hunter86_bg(a)yahoo.com> wrote: >> >> Hi All, >> >> I have changed my I/O scheduler to none and here are the results so far: >> >> Before (mq-deadline): >> Adding a disk to VM (initial creation) START: 2019-03-17 16:34:46.709 >> Adding a disk to VM (initial creation) COMPLETED: 2019-03-17 16:45:17.996 >> >> After (none): >> Adding a disk to VM (initial creation) START: 2019-03-18 08:52:02.xxx >> Adding a disk to VM (initial creation) COMPLETED: 2019-03-18 08:52:20.xxx >> >> Of course the results are inconclusive, as I have tested only once - but I feel the engine more responsive. >> >> Best Regards, >> Strahil Nikolov >> >> В неделя, 17 март 2019 г., 18:30:23 ч. Гринуич+2, Strahil <hunter86_bg(a)yahoo.com> написа: >> >> >> Dear All, >> >> I have just noticed that my Hosted Engine has a strange I/O scheduler: >> >> Last login: Sun Mar 17 18:14:26 2019 from 192.168.1.43 >> [root@engine ~]# cat /sys/block/vda/queue/scheduler >> [mq-deadline] kyber none >> [root@engine ~]# >> >> Based on my experience anything than noop/none is useless and performance degrading for a VM. >> >> Is there any reason that we have this scheduler ? >> It is quite pointless to process (and delay) the I/O in the VM and then process (and again delay) on Host Level . >> >> If there is no reason to keep the deadline, I will open a bug about it. >> >> Best Regards, >> Strahil Nikolov >> >> Dear All, >> >> I have just noticed that my Hosted Engine has a strange I/O scheduler: >> >> Last login: Sun Mar 17 18:14:26 2019 from 192.168.1.43 >> [root@engine ~]# cat /sys/block/vda/queue/scheduler >> [mq-deadline] kyber none >> [root@engine ~]# >> >> Based on my experience anything than noop/none is useless and performance degrading for a VM. >> >>

4 5

Backup Ovirt VMs with Python not working
by daniel94.oeller＠gmail.com 20 Mar '19

20 Mar '19

Hello, I have set up Ovirt Backup how it is explained in this link: http://blog.infratic.com/create-ovirtrhevs-vm-backup/ I also have changed the settings on the default.conf. When I started the backup it works until it begins to create the qcow2 file It writes: qemu-img: Could not open '/home/DC01/2019-03-20-20/DC01_Disk1.qcow2': Failed to get shared "write" lock Is another process using the image? Have anyone ideas and can help me? Thanks

2 1

Re: Save Virtual Machine in NFS partition
by Strahil 20 Mar '19

20 Mar '19

Hi , Without the engine - this will be hard.The engine knows the configuration and the disks of each VM so the Hypervisor can start it. In your case I would focus on the engine as this could be (or not) the easier approach. Otherwise you need to run qemu-img against each image (just an uuid, no meta or lease ) , find your image and copy it to a standalone KVM. Once you have the disk, you should try to recall what was the host CPU type (should be the same on the whole cluster) and create a new VM using a COPY of your image . If KVM manages to start and the machine boots - you are done. As you can see , fixing the engine might be easier - especially with the help of the community. Best Regards, Strahil NikolovOn Mar 20, 2019 18:01, siovelrm(a)gmail.com wrote: > > Hi, I had problems with my ovirt-engine and it does not raise. I have some virtual machines in an NFS domain that is located in one of the nodes and I want to get one of the virtual machines. The problem is that what appears is a series of identifiers (ID) and I don't know what the virtual machine is, because by name I can not find it. There are some .meta files where I can see the Description of some Virtual Machines. Please I need help > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/NXNAZBOSGTIRC…

1 0

Save Virtual Machine in NFS partition
by siovelrm＠gmail.com 20 Mar '19

20 Mar '19

Hi, I had problems with my ovirt-engine and it does not raise. I have some virtual machines in an NFS domain that is located in one of the nodes and I want to get one of the virtual machines. The problem is that what appears is a series of identifiers (ID) and I don't know what the virtual machine is, because by name I can not find it. There are some .meta files where I can see the Description of some Virtual Machines. Please I need help

1 0

4.2.7 Importing qcow2 linux as disk
by Arnaud DEBEC 20 Mar '19

20 Mar '19

Hi, I want to import a Centos 7 qcow2 VM on oVirt 4.2.7. 0. the Centos 7 VM was previously running under libvirt / kvm hypervisor. 1. on Ovirt 4.2.7, I imported the VM qcow2 via Storage > Disks > Upload > Start > etc. 2. I created a VM and attached the disk and specifying the disk is bootable 3. When the machine boots, I get "Warning: /dev/disk/by-uuid/XXXXXXXXXX does not exist" Before going in deeper detail in the investigation, I would like to make sure if the previous steps make sens for importing a VM under qcow2 format. Most of documentation I found to import libvirt /KVM are done by using the Compute > VM > import > KVM (via Libvirt) which is not what I am looking for. Thank you.

1 0

Fwd: Re: Change cluster cpu type with hosted engine
by Fabrice SOLER 20 Mar '19

20 Mar '19

Hi, As you can read on the screenshot below, my servers have got Nehalem CPU type, so the hosted engine detect the correct CPU type. You can read on this link https://www.intel.com/content/www/us/en/support/articles/000006105/processo… that Windows 10 x64 does not support Nehalem CPU type. Bill is cutting the fonctionnality for the old CPU, I will be able to comment but i will not. But I need to use Windows 10 x64 with this servers for my students !!! I am in a dead-end. Sincerely, Fabrice SOLER -------- Message transféré -------- Sujet : Re: [ovirt-users] Change cluster cpu type with hosted engine Date : Tue, 12 Mar 2019 15:49:14 -0400 De : Fabrice SOLER <fabrice.soler(a)ac-guadeloupe.fr> Pour : Simone Tiraboschi <stirabos(a)redhat.com> Le 12/03/2019 à 11:21, Simone Tiraboschi a écrit : > > > On Tue, Mar 12, 2019 at 1:03 PM Fabrice SOLER > <fabrice.soler(a)ac-guadeloupe.fr > <mailto:fabrice.soler@ac-guadeloupe.fr>> wrote: > > Hello, > > I need to create a windows 10 virtual machine but I have an error : > > I have a fresh ovirt installation (version 4.2.8) with an hosted > engine. At the hosted engine installation there was no question > about the cluster cpu type, it should be great if in the future > version it could be. > > > It's by design: we simply let the engine choose the best CPU according > to the characteristics of the first host. > Can you please detail which CPU are you using on the first host and > how the engine detected it adding the first host to the first cluster? Hello, The first host is a DELL R610 (an old server). At the installation, the cluster was created with an CPU Architecture x86_64 and CPU type Intel Nehalem Family. > > To change an host to another cluster this host need to be in > maintenance mode, and the hosted engine will be power off. > > I have created another Cluster with an SandyBridge Family CPU > type, but to move the hosted engine to this new cluster the hosted > should be power off. > > Is there someone who can help ? > > Sincerely, > > -- > _______________________________________________ > Users mailing list -- users(a)ovirt.org <mailto:users@ovirt.org> > To unsubscribe send an email to users-leave(a)ovirt.org > <mailto:users-leave@ovirt.org> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/KFH3ZLPA7KZSS… > --

1 0

[ANN] oVirt 4.3.2 is now generally available
by Sandro Bonazzola 20 Mar '19

20 Mar '19

The oVirt Project is pleased to announce the general availability of oVirt 4.3.2, as of March 19th, 2019. This update is the second in a series of stabilization updates to the 4.3 series. This release is available now on x86_64 architecture for: * Red Hat Enterprise Linux 7.6 or later * CentOS Linux (or similar) 7.6 or later This release supports Hypervisor Hosts on x86_64 and ppc64le architectures for: * Red Hat Enterprise Linux 7.6 or later * CentOS Linux (or similar) 7.6 or later * oVirt Node 4.3 (available for x86_64 only) Experimental tech preview for x86_64 and s390x architectures for Fedora 28 is also included. See the release notes [1] for installation / upgrade instructions and a list of new features and bugs fixed. Notes: - oVirt Appliance is already available - oVirt Node is already available[2] oVirt Node has been updated including: - oVirt 4.3.2: http://www.ovirt.org/release/4.3.2/ - Latest CentOS updates (no relevant errata available up to now on https://lists.centos.org/pipermail/centos-announce ) Additional Resources: * Read more about the oVirt 4.3.2 release highlights: http://www.ovirt.org/release/4.3.2/ * Get more oVirt Project updates on Twitter: https://twitter.com/ovirt * Check out the latest project news on the oVirt blog: http://www.ovirt.org/blog/ [1] http://www.ovirt.org/release/4.3.2/ [2] http://resources.ovirt.org/pub/ovirt-4.3/iso/ -- SANDRO BONAZZOLA MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://red.ht/sig>

2 3

Re: Can't connect "https://FQDN/ovirt-engine" after reinstall ovirt-engine
by Александр Егоров 20 Mar '19

20 Mar '19

How do I repair the ovirt-engine? engine.log in attachment

2 1

Re: vender_id syntax UserDefinedVMProperties
by Strahil 20 Mar '19

20 Mar '19

Can't you make the script check if it windows or Linux and skip if it's Linux? Best Regards, Strahil NikolovOn Mar 19, 2019 23:02, Darin Schmidt <darinschmidt(a)gmail.com> wrote: > > You also need to have this code hooked in: > cd /usr/libexec/vdsm/hooks/before_vm_start/ > vi 99_mask_kvm > > #!/usr/bin/python2 > > import hooking > domxml = hooking.read_domxml() > > hyperv = domxml.getElementsByTagName('hyperv')[0] > smm = domxml.createElement('vendor_id') > smm.setAttribute('state', 'on') > smm.setAttribute('value', '1234567890ab') > hyperv.appendChild(smm) > > features = domxml.getElementsByTagName('features')[0] > kvm = domxml.createElement('kvm') > hidden = domxml.createElement('hidden') > hidden.setAttribute('state', 'on') > kvm.appendChild(hidden) > features.appendChild(kvm) > > hooking.write_domxml(domxml) > > > only problem now is that I cant boot a linux VM with the vendor_is portion there...... > > On Mon, Mar 18, 2019 at 3:30 PM Darin Schmidt <darinschmidt(a)gmail.com> wrote: >> >> Seems that the system has to be running with bios Q35 UEFI. Standard bios does not work. System is operational now. >> >> On Mon, Mar 18, 2019, 6:30 AM Darin Schmidt <darinschmidt(a)gmail.com> wrote: >>> >>> Still no luck getting the gtx 1080 to enable inside the VM. I see the code is being generated in the xml with the hook. But I still get error code 43. Someone mentioned doing it with eufi bios and that worked for them. So when I get back from work today, perhaps ill give that a try. >>> >>> On Mon, Mar 18, 2019, 6:10 AM Darin Schmidt <darinschmidt(a)gmail.com> wrote: >>>> >>>> I have gotten the system to see the card, its in device manager. The problem seems to be that I cannot use it in the VM because from what I have been finding out is that it gets and error code 43. Nvidia drivers disable the card if it detects that its being used in a VM. I have found some code to use to hook it into the xml before_vm_starts. >>>> >>>> 99_mask_kvm >>>> #!/usr/bin/python2 >>>> >>>> import hooking >>>> domxml = hooking.read_domxml() >>>> >>>> hyperv = domxml.getElementsByTagName('hyperv')[0] >>>> smm = domxml.createElement('vendor_id') >>>> smm.setAttribute('state', 'on') >>>> smm.setAttribute('value', '1234567890ab') >>>> hyperv.appendChild(smm) >>>> >>>> features = domxml.getElementsByTagName('features')[0] >>>> kvm = domxml.createElement('kvm') >>>> hidden = domxml.createElement('hidden') >>>> hidden.setAttribute('state', 'on') >>>> kvm.appendChild(hidden) >>>> features.appendChild(kvm) >>>> >>>> hooking.write_domxml(domxml) >>>> >>>> >>>> I am currently reinstalling the drivers to see if this helps. >>>> >>>> kvm off and vender_id is now in the xml code that get generated when the VM is started. Im going off of examples Im finding online. Perhaps I just need to add the 10de to it instead of some generic # others are using. >>>> >>>> On Mon, Mar 18, 2019 at 6:02 AM Nisim Simsolo <nsimsolo(a)redhat.com> wrote: >>>>> >>>>> Hi >>>>> >>>>> Vendor ID of Nvidia is usually 10de. >>>>> You can locate 'vendor ID:

2 1

Daily reboots of Hosted Engine?
by Juhani Rautiainen 20 Mar '19

20 Mar '19

Hi! Hosted engine reboots itself almost daily. Is this by design? If not, where should I be searching for the clues why it shuts down? Someone is giving reboot order to HE because /var/log/messages in contains this: Mar 19 12:05:00 ovirtmgr qemu-ga: info: guest-shutdown called, mode: powerdown Mar 19 12:05:00 ovirtmgr systemd: Started Delayed Shutdown Service. And I'm still running v4.3.0 because upgrade to that was bit painful and haven't dared to new round. Thanks, -Juhani

3 9

vender_id syntax UserDefinedVMProperties
by Darin Schmidt 19 Mar '19

19 Mar '19

Hello all, im trying to figure out how to configure the custom properties to enable my NVIDIA card to work in the VM. Its my understanding that the drives dont work because it detects its in a VM...... Im trying to do something like this: engine-config -s UserDefinedVMProperties="kvmhidden=^(true|false)$;{type=vendor_id;state={^(on|off)$;value=^([0-9])$}}" But thats clearly not working. If I do this: engine-config -s UserDefinedVMProperties="kvmhidden=^(true|false)$;vendor_id={state=^(on|off)$;value=^([0-9])$}" It works but, the options are messed up. Im not sure how to find out the correct syntax to get this to work. Would appreciate any advice.

2 5

oVirt 4.3 - create windows2012 failed due to ovirt-imageio-proxy
by jingjie.jiang＠oracle.com 19 Mar '19

19 Mar '19

Hi, I tried to create windows2012 vm on nfs data domain, but the disk was locked. Found the error message as following: Connection to ovirt-imageio-proxy service has failed. Make sure the service is installed, configured, and ovirt-engine certificate is registered as a valid CA in the browser. Is this known issue? Thanks, Jingjie

3 2

Change cluster cpu type with hosted engine
by Fabrice SOLER 19 Mar '19

19 Mar '19

Hello, I need to create a windows 10 virtual machine but I have an error : I have a fresh ovirt installation (version 4.2.8) with an hosted engine. At the hosted engine installation there was no question about the cluster cpu type, it should be great if in the future version it could be. To change an host to another cluster this host need to be in maintenance mode, and the hosted engine will be power off. I have created another Cluster with an SandyBridge Family CPU type, but to move the hosted engine to this new cluster the hosted should be power off. Is there someone who can help ? Sincerely, --

3 3

Re: Ovirt 4.3.1 problem with HA agent
by Strahil 19 Mar '19

19 Mar '19

>> >> 1.2 All bricks healed (gluster volume heal data info summary) and no split-brain >> > >> > >> > >> > gluster volume heal data info >> > >> > Brick node-msk-gluster203:/opt/gluster/data >> > Status: Connected >> > Number of entries: 0 >> > >> > Brick node-msk-gluster205:/opt/gluster/data >> > <gfid:18c78043-0943-48f8-a4fe-9b23e2ba3404> >> > <gfid:b6f7d8e7-1746-471b-a49d-8d824db9fd72> >> > <gfid:6db6a49e-2be2-4c4e-93cb-d76c32f8e422> >> > <gfid:e39cb2a8-5698-4fd2-b49c-102e5ea0a008> >> > <gfid:5fad58f8-4370-46ce-b976-ac22d2f680ee> >> > <gfid:7d0b4104-6ad6-433f-9142-7843fd260c70> >> > <gfid:706cd1d9-f4c9-4c89-aa4c-42ca91ab827e> >> > Status: Connected >> > Number of entries: 7 >> > >> > Brick node-msk-gluster201:/opt/gluster/data >> > <gfid:18c78043-0943-48f8-a4fe-9b23e2ba3404> >> > <gfid:b6f7d8e7-1746-471b-a49d-8d824db9fd72> >> > <gfid:6db6a49e-2be2-4c4e-93cb-d76c32f8e422> >> > <gfid:e39cb2a8-5698-4fd2-b49c-102e5ea0a008> >> > <gfid:5fad58f8-4370-46ce-b976-ac22d2f680ee> >> > <gfid:7d0b4104-6ad6-433f-9142-7843fd260c70> >> > <gfid:706cd1d9-f4c9-4c89-aa4c-42ca91ab827e> >> > Status: Connected >> > Number of entries: 7 >> > >> >> Data needs healing. >> Run: cluster volume heal data full > > This does not work. Yeah, That's because my phone corrects the 'gluster' to 'cluster' Usually gluster daemons detect need of heal, but with 'gluster volume heal data full && sleep 5 && gluster volume heal data info summary && sleep 5 && gluster volume heal data info summary', you can force syncing and get the result. Let's see what happens with DNS. Best Regards, Strahil Nikolov

1 0

Hosted -engine is down and cannot be restarted
by ada per 19 Mar '19

19 Mar '19

Hello everyone, For a strange reason the hosted engine went down and I cannot restart it. I tried manually restarting it without any success can you please advice? For all the nodes the engine status is the same as the one below. --== Host nodex. (id: 6) status ==-- conf_on_shared_storage : True Status up-to-date : True Hostname : nodex Host ID : 6 Engine status : {"reason": "bad vm status", "health": "bad", "vm": "down_unexpected", "detail": "Down"} Score : 3400 stopped : False Local maintenance : False crc32 : 323a9f45 local_conf_timestamp : 2648874 Host timestamp : 2648874 Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=2648874 (Tue Mar 19 12:25:44 2019) host-id=6 score=3400 vm_conf_refresh_time=2648874 (Tue Mar 19 12:25:44 2019) conf_on_shared_storage=True maintenance=False state=GlobalMaintenance stopped=False When I try the commands root@node5# hosted-engine --vm-shutdown I ge the response: root@node5# Command VM.shutdown with args {'delay': '120', 'message': 'VM is shutting down!', 'vmID': 'a492d2eb-1dfd-470d-a141-3e55d2189275'} failed:(code=1, message=Virtual machine does not exist) But when I run : hosted-engine --vm-start I get the response: VM exists and is down, cleaning up and restarting Below you can see the # journalctl -u ovirt-ha-agent logs Mar 14 12:04:42 node7. ovirt-ha-agent[4134]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Unhandled monitoring loop exception Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 430, in start_monitoring self._monitoring_loop() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 449, in _monitoring_loop for old_state, state, delay in self.fsm: File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/fsm/machine.py", line 127, in next new_data = self.refresh(self._state.data) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/state_machine.py", line 81, in refresh stats.update(self.hosted_engine.collect_stats()) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 737, in collect_stats all_stats = self._broker.get_stats_from_storage() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 143, in get_stats_from_storage result = self._proxy.get_stats() File "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__ return self.__send(self.__name, args) File "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in __request verbose=self.__verbose File "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request return self.single_request(host, handler, request_body, verbose) File "/usr/lib64/python2.7/xmlrpclib.py", line 1301, in single_request self.send_content(h, request_body) File "/usr/lib64/python2.7/xmlrpclib.py", line 1448, in send_content connection.endheaders(request_body) File "/usr/lib64/python2.7/httplib.py", line 1037, in endheaders self._send_output(message_body) File "/usr/lib64/python2.7/httplib.py", line 881, in _send_output self.send(msg) File "/usr/lib64/python2.7/httplib.py", line 843, in send self.connect() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py", line 52, in connect self.sock.connect(base64.b16decode(self.host)) File "/usr/lib64/python2.7/socket.py", line 224, in meth return getattr(self._sock,name)(*args) error: [Errno 2] No such file or directory Mar 14 12:04:42 node7. ovirt-ha-agent[4134]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent return action(he) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper return he.start_monitoring() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 437, in start_monitoring self.publish(stopped) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 337, in publish self._push_to_storage(blocks) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 708, in _push_to_storage self._broker.put_stats_on_storage(self.host_id, blocks) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 113, in put_stats_on_storage self._proxy.put_stats(host_id, xmlrpclib.Binary(data)) File "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__ return self.__send(self.__name, args) File "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in __request verbose=self.__verbose File "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request return self.single_request(host, handler, request_body, verbose) File "/usr/lib64/python2.7/xmlrpclib.py", line 1301, in single_request self.send_content(h, request_body) File "/usr/lib64/python2.7/xmlrpclib.py", line 1448, in send_content connection.endheaders(request_body) File "/usr/lib64/python2.7/httplib.py", line 1037, in endheaders self._send_output(message_body) File "/usr/lib64/python2.7/httplib.py", line 881, in _send_output self.send(msg) File "/usr/lib64/python2.7/httplib.py", line 843, in send self.connect() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py", line 52, in connect self.sock.connect(base64.b16decode(self.host)) File "/usr/lib64/python2.7/socket.py", line 224, in meth return getattr(self._sock,name)(*args) error: [Errno 2] No such file or directory Mar 14 12:04:42 node7. ovirt-ha-agent[4134]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Mar 14 12:04:42 node7. systemd[1]: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Mar 14 12:04:42 node7. systemd[1]: Unit ovirt-ha-agent.service entered failed state. Mar 14 12:04:42 node7. systemd[1]: ovirt-ha-agent.service failed. Mar 14 12:04:52 node7. systemd[1]: ovirt-ha-agent.service holdoff time over, scheduling restart. Mar 14 12:04:52 node7. systemd[1]: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Mar 14 12:04:52 node7. systemd[1]: Started oVirt Hosted Engine High Availability Monitoring Agent. Mar 14 12:04:52 node7. ovirt-ha-agent[31765]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Mar 14 12:04:52 node7. ovirt-ha-agent[31765]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent return action(he) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper return he.start_monitoring() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 413, in start_monitoring self._initialize_broker() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 537, in _initialize_broker m.get('options', {})) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 86, in start_monitor ).format(t=type, o=options, e=e) RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'ping', options: {'addr': '19 Mar 14 12:04:52 node7. ovirt-ha-agent[31765]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Mar 14 12:04:52 node7. systemd[1]: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Mar 14 12:04:52 node7. systemd[1]: Unit ovirt-ha-agent.service entered failed state. Mar 14 12:04:52 node7. systemd[1]: ovirt-ha-agent.service failed. Mar 14 12:05:02 node7. systemd[1]: ovirt-ha-agent.service holdoff time over, scheduling restart. Mar 14 12:05:02 node7. systemd[1]: Stopped oVirt Hosted Engine High Availability Monitoring Agent. Mar 14 12:05:02 node7. systemd[1]: Started oVirt Hosted Engine High Availability Monitoring Agent. Mar 14 12:06:55 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to stop engine vm with /usr/sbin/hosted-engine --vm-poweroff: Co (code=1, message=Virtual machine does not exist: {'vmId': u'a492d2eb-1dfd-470d-a141-3e55d2189275'}) Mar 14 12:06:55 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to stop engine VM: Command VM.destroy with args {'vmID': 'a492d2 (code=1, message=Virtual machine does not exist: {'vmId': u'a492d2eb-1dfd-470d-a141-3e55d2189275'}) Mar 15 14:28:16 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost Mar 15 14:28:36 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost Mar 15 14:29:00 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost Mar 15 14:29:22 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost Mar 15 14:29:44 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost Mar 15 14:30:06 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost Mar 15 14:30:28 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost Mar 15 14:30:50 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost Mar 15 14:31:12 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost Mar 15 14:31:33 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost Mar 15 14:31:56 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost Mar 15 14:32:18 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost Mar 15 14:32:40 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost Mar 15 14:33:02 node7. ovirt-ha-agent[31822]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Engine VM stopped on localhost

2 8

Host unresponsive after upgrade 4.2.8 -> 4.3.2 failed
by Artem Tambovskiy 19 Mar '19

19 Mar '19

Hello, Just started upgrading my small cluster to from 4.2.8 to 4.3.2 and endup in the situation that one of the hosts is not working after upgrade. For some reason vdsmd is not starting up, I have tried to restart it manually with no luck: Any ideas on what could be the reason? [root@ovirt2 log]# systemctl restart vdsmd A dependency job for vdsmd.service failed. See 'journalctl -xe' for details. [root@ovirt2 log]# journalctl -xe -- Unit ovirt-ha-agent.service has finished shutting down. Mar 19 15:47:47 ovirt2.domain.org systemd[1]: Starting Virtual Desktop Server Manager... -- Subject: Unit vdsmd.service has begun start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit vdsmd.service has begun starting up. Mar 19 15:47:47 ovirt2.domain.org vdsmd_init_common.sh[56717]: vdsm: Running mkdirs Mar 19 15:47:47 ovirt2.domain.org vdsmd_init_common.sh[56717]: vdsm: Running configure_coredump Mar 19 15:47:47 ovirt2.domain.org vdsmd_init_common.sh[56717]: vdsm: Running configure_vdsm_logs Mar 19 15:47:47 ovirt2.domain.org vdsmd_init_common.sh[56717]: vdsm: Running wait_for_network Mar 19 15:47:47 ovirt2.domain.org supervdsmd[56716]: Supervdsm failed to start: 'module' object has no attribute 'Accounting' Mar 19 15:47:47 ovirt2.domain.org python2[56716]: detected unhandled Python exception in '/usr/share/vdsm/supervdsmd' Mar 19 15:47:48 ovirt2.domain.org abrt-server[56745]: Duplicate: core backtrace Mar 19 15:47:48 ovirt2.domain.org abrt-server[56745]: DUP_OF_DIR: /var/tmp/abrt/Python-2019-03-19-14:23:04-17292 Mar 19 15:47:48 ovirt2.domain.org abrt-server[56745]: Deleting problem directory Python-2019-03-19-15:47:47-56716 (dup of Python-2019-03-19-14:23:04-17292 Mar 19 15:47:49 ovirt2.domain.org daemonAdapter[56716]: Traceback (most recent call last): Mar 19 15:47:49 ovirt2.domain.org daemonAdapter[56716]: File "/usr/share/vdsm/supervdsmd", line 26, in <module> Mar 19 15:47:49 ovirt2.domain.org daemonAdapter[56716]: supervdsm_server.main(sys.argv[1:]) Mar 19 15:47:49 ovirt2.domain.org daemonAdapter[56716]: File "/usr/lib/python2.7/site-packages/vdsm/supervdsm_server.py", line 294, in main Mar 19 15:47:49 ovirt2.domain.org daemonAdapter[56716]: module_name)) Mar 19 15:47:49 ovirt2.domain.org daemonAdapter[56716]: File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module Mar 19 15:47:49 ovirt2.domain.org daemonAdapter[56716]: __import__(name) Mar 19 15:47:49 ovirt2.domain.org daemonAdapter[56716]: File "/usr/lib/python2.7/site-packages/vdsm/supervdsm_api/systemd.py", line 34, in <module> Mar 19 15:47:49 ovirt2.domain.org daemonAdapter[56716]: cmdutils.Accounting.CPU, Mar 19 15:47:49 ovirt2.domain.org daemonAdapter[56716]: AttributeError: 'module' object has no attribute 'Accounting' Mar 19 15:47:49 ovirt2.domain.org systemd[1]: supervdsmd.service: main process exited, code=exited, status=1/FAILURE Mar 19 15:47:49 ovirt2.domain.org systemd[1]: Unit supervdsmd.service entered failed state. Mar 19 15:47:49 ovirt2.domain.org systemd[1]: supervdsmd.service failed. Mar 19 15:47:49 ovirt2.domain.org systemd[1]: supervdsmd.service holdoff time over, scheduling restart. Mar 19 15:47:49 ovirt2.domain.org systemd[1]: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Mar 19 15:47:49 ovirt2.domain.org systemd[1]: Stopped Auxiliary vdsm service for running helper functions as root. -- Subject: Unit supervdsmd.service has finished shutting down -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit supervdsmd.service has finished shutting down. Mar 19 15:47:49 ovirt2.domain.org systemd[1]: Started Auxiliary vdsm service for running helper functions as root. -- Subject: Unit supervdsmd.service has finished start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit supervdsmd.service has finished starting up. -- -- The start-up result is done. Mar 19 15:47:50 ovirt2.domain.org supervdsmd[56757]: Supervdsm failed to start: 'module' object has no attribute 'Accounting' Mar 19 15:47:50 ovirt2.domain.org python2[56757]: detected unhandled Python exception in '/usr/share/vdsm/supervdsmd' -- Regards, Artem

1 0

Re: Ovirt 4.3.1 problem with HA agent
by Strahil 19 Mar '19

19 Mar '19

Hi Alexei, >> 1.2 All bricks healed (gluster volume heal data info summary) and no split-brain > > > > gluster volume heal data info > > Brick node-msk-gluster203:/opt/gluster/data > Status: Connected > Number of entries: 0 > > Brick node-msk-gluster205:/opt/gluster/data > <gfid:18c78043-0943-48f8-a4fe-9b23e2ba3404> > <gfid:b6f7d8e7-1746-471b-a49d-8d824db9fd72> > <gfid:6db6a49e-2be2-4c4e-93cb-d76c32f8e422> > <gfid:e39cb2a8-5698-4fd2-b49c-102e5ea0a008> > <gfid:5fad58f8-4370-46ce-b976-ac22d2f680ee> > <gfid:7d0b4104-6ad6-433f-9142-7843fd260c70> > <gfid:706cd1d9-f4c9-4c89-aa4c-42ca91ab827e> > Status: Connected > Number of entries: 7 > > Brick node-msk-gluster201:/opt/gluster/data > <gfid:18c78043-0943-48f8-a4fe-9b23e2ba3404> > <gfid:b6f7d8e7-1746-471b-a49d-8d824db9fd72> > <gfid:6db6a49e-2be2-4c4e-93cb-d76c32f8e422> > <gfid:e39cb2a8-5698-4fd2-b49c-102e5ea0a008> > <gfid:5fad58f8-4370-46ce-b976-ac22d2f680ee> > <gfid:7d0b4104-6ad6-433f-9142-7843fd260c70> > <gfid:706cd1d9-f4c9-4c89-aa4c-42ca91ab827e> > Status: Connected > Number of entries: 7 > Data needs healing. Run: cluster volume heal data full If it still doesn't heal (check in 5 min),go to /rhev/data-center/mnt/glusterSD/msk-gluster-facility.xxxx_data And run 'find . -exec stat {}\;' without the quotes. As I have understood you, ovirt Hosted Engine is running and can be started on all nodes except 1. >> >> 2. Go to the problematic host and check the mount point is there > > > > No mount point on problematic node /rhev/data-center/mnt/glusterSD/msk-gluster-facility.xxxx:_data > If I create a mount point manually, it is deleted after the node is activated. > > Other nodes can mount this volume without problems. Only this node have connection problems after update. > > Here is a part of the log at the time of activation of the node: > > vdsm log > > 2019-03-18 16:46:00,548+0300 INFO (jsonrpc/5) [vds] Setting Hosted Engine HA local maintenance to False (API:1630) > 2019-03-18 16:46:00,549+0300 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.setHaMaintenanceMode succeeded in 0.00 seconds (__init__:573) > 2019-03-18 16:46:00,581+0300 INFO (jsonrpc/7) [vdsm.api] START connectStorageServer(domType=7, spUUID=u'5a5cca91-01f8-01af-0297-00000000025f', conList=[{u'id': u'5799806e-7969-45da-b17d-b47a63e6a8e4', u'connection': u'msk-gluster-facility.xxxx:/data', u'iqn': u'', u'user': u'', u'tpgt': u'1', u'vfs_type': u'glusterfs', u'password': '********', u'port': u''}], options=None) from=::ffff:10.77.253.210,56630, flow_id=81524ed, task_id=5f353993-95de-480d-afea-d32dc94fd146 (api:46) > 2019-03-18 16:46:00,621+0300 INFO (jsonrpc/7) [storage.StorageServer.MountConnection] Creating directory u'/rhev/data-center/mnt/glusterSD/msk-gluster-facility.xxxx:_data' (storageServer:167) > 2019-03-18 16:46:00,622+0300 INFO (jsonrpc/7) [storage.fileUtils] Creating directory: /rhev/data-center/mnt/glusterSD/msk-gluster-facility.xxxx:_data mode: None (fileUtils:197) > 2019-03-18 16:46:00,622+0300 WARN (jsonrpc/7) [storage.StorageServer.MountConnection] gluster server u'msk-gluster-facility.xxxx' is not in bricks ['node-msk-gluster203', 'node-msk-gluster205', 'node-msk-gluster201'], possibly mounting duplicate servers (storageServer:317) This seems very strange. As you have hidden the hostname, I'm not use which on is this. Check that DNS can be resolved from all hosts and the hostname of this Host is resolvable. Also check if it in the peer list. Try to manually mount the cluster volume: mount -t glusterfs msk-gluster-facility.xxxx:/data /mnt Is this a second FQDN/IP of this server? If so, gluster accepts that via gluster peer probe IP2 >> 2.1. Check permissions (should be vdsm:kvm) and fix with chown -R if needed >> 2.2. Check the OVF_STORE from the logs that it exists > > > How can i do this? Go to /rhev/data-center/mnt/glusterSD/host_engine and use find inside the domain UUID for files that are not owned by vdsm:KVM. I usually run 'chown -R vdsm:KVM 823xx-xxxx-yyyy-zzz' and it will fix any misconfiguration. Best Regards, Strahil Nikolov

2 1

Live migration failed
by Bong Shau Fui 19 Mar '19

19 Mar '19

Hi: I deployed 2 ovirt hosts and an ovirt engine in a nested KVM server. I've a windows vm setup and tried to perform live migration but failed. I checked on the hosts and found them meeting the live migration requirements, or at least that's what I thought. I took the requirement from the below document. https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Virtualiza… The hosts, both source and destination are quite empty, with only the hosted engine, 1 centos vm and the windows VM in the cluster. I can do a live migration for the centos vm successfully. But when I tried live migration on the hosted-engine vm it failed immediately with a message "No available host to migrate VMs to". When I tried to migrate the windows VM the message box that let me choose the destination host popped up but failed after a while. I'd like to ask where can I get more information with regards to live-migration apart from /var/log/ovirt-engine/engine.log ? I also checked on the ovirt hosts' /var/log/vdsm/vdsm.log but found nothing pointing to the reason why it failed. Below is the extract from /var/log/ovirt-engine/engine.log when the live-migration took place 2019-03-12 14:37:58,159+08 INFO [org.ovirt.engine.core.sso.utils.AuthenticationUtils] (default task-131) [] User admin@internal successfully logged in with scopes: ovirt-app-api ovirt-ext=token-info:authz-search ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate ovirt-ext=token:password-access 2019-03-12 14:37:58,450+08 INFO [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-59) [77888830] Lock freed to object 'EngineLock:{exclusiveLocks='[d113be83-2740-4246-a1f2-b9344889c3cf=PROVIDER]', sharedLocks=''}' 2019-03-12 14:38:02,544+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-50) [] BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2019-03-12 14:38:12,677+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-16) [] BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2019-03-12 14:38:21,650+08 INFO [org.ovirt.engine.core.bll.aaa.SessionDataContainer] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [] Not removing session 'xDiHqqa6l+g8cngM26TTCfW7NeLN3WgWChsx28wUM391vAngSxwtyCkLbQxZR1AbJ5I+2bkPZNQijMUk0jLZcA==', session has running commands for user 'admin@internal-authz'. 2019-03-12 14:38:22,782+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-49) [] BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2019-03-12 14:38:33,018+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [] BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2019-03-12 14:38:43,261+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-59) [77888830] BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2019-03-12 14:38:53,528+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-13) [] BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2019-03-12 14:39:03,759+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [] BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2019-03-12 14:39:14,011+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-60) [] BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2019-03-12 14:39:21,660+08 INFO [org.ovirt.engine.core.bll.aaa.SessionDataContainer] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [] Not removing session 'xDiHqqa6l+g8cngM26TTCfW7NeLN3WgWChsx28wUM391vAngSxwtyCkLbQxZR1AbJ5I+2bkPZNQijMUk0jLZcA==', session has running commands for user 'admin@internal-authz'. 2019-03-12 14:39:24,122+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [] BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2019-03-12 14:39:29,773+08 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (default task-131) [7f0cf113-55e8-4def-9c68-de3b91d6d641] Lock Acquired to object 'EngineLock:{exclusiveLocks='[5cad5c5f-5aab-46ec-a28e-d484abc0401d=VM]', sharedLocks=''}' 2019-03-12 14:39:29,887+08 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (default task-131) [7f0cf113-55e8-4def-9c68-de3b91d6d641] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: 5cad5c5f-5aab-46ec-a28e-d484abc0401d Type: VMAction group MIGRATE_VM with role type USER 2019-03-12 14:39:30,019+08 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (default task-131) [7f0cf113-55e8-4def-9c68-de3b91d6d641] START, MigrateVDSCommand( MigrateVDSCommandParameters:{hostId='f9014bc4-485c-4eb0-a9bc-42d13ed68f41', vmId='5cad5c5f-5aab-46ec-a28e-d484abc0401d', srcHost='host2.xxxx.com', dstVdsId='1bc9b9e9-1e90-4570-9930-08416d1927cc', dstHost='host3.xxxx.com:54321', migrationMethod='ONLINE', tunnelMigration='false', migrationDowntime='0', autoConverge='true', migrateCompressed='false', consoleAddress='null', maxBandwidth='null', enableGuestEvents='true', maxIncomingMigrations='2', maxOutgoingMigrations='2', convergenceSchedule='[init=[{name=setDowntime, params=[100]}], stalling=[{limit=1, action={name=setDowntime, params=[150]}}, {limit=2, action={name=setDowntime, params=[200]}}, {limit=3, action={name=setDowntime, params=[300]}}, {limit=4, action={name=setDowntime, params=[400]}}, {limit=6, action={name=setDowntime, params=[500]}}, {limit=-1, action={nam e=abort, params=[]}}]]', dstQemu='192.168.138.135'}), log id: 7eeb678c 2019-03-12 14:39:30,022+08 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (default task-131) [7f0cf113-55e8-4def-9c68-de3b91d6d641] START, MigrateBrokerVDSCommand(HostName = host2.xxxx.com, MigrateVDSCommandParameters:{hostId='f9014bc4-485c-4eb0-a9bc-42d13ed68f41', vmId='5cad5c5f-5aab-46ec-a28e-d484abc0401d', srcHost='host2.xxxx.com', dstVdsId='1bc9b9e9-1e90-4570-9930-08416d1927cc', dstHost='host3.xxxx.com:54321', migrationMethod='ONLINE', tunnelMigration='false', migrationDowntime='0', autoConverge='true', migrateCompressed='false', consoleAddress='null', maxBandwidth='null', enableGuestEvents='true', maxIncomingMigrations='2', maxOutgoingMigrations='2', convergenceSchedule='[init=[{name=setDowntime, params=[100]}], stalling=[{limit=1, action={name=setDowntime, params=[150]}}, {limit=2, action={name=setDowntime, params=[200]}}, {limit=3, action={name=setDowntime, params=[300]}}, {limit=4, action={name=setDowntime, params=[400]}}, {limit=6, action={name=set Downtime, params=[500]}}, {limit=-1, action={name=abort, params=[]}}]]', dstQemu='192.168.138.135'}), log id: 5cef4981 2019-03-12 14:39:30,039+08 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (default task-131) [7f0cf113-55e8-4def-9c68-de3b91d6d641] FINISH, MigrateBrokerVDSCommand, return: , log id: 5cef4981 2019-03-12 14:39:30,048+08 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (default task-131) [7f0cf113-55e8-4def-9c68-de3b91d6d641] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 7eeb678c 2019-03-12 14:39:30,067+08 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-131) [7f0cf113-55e8-4def-9c68-de3b91d6d641] EVENT_ID: VM_MIGRATION_START(62), Migration started (VM: Win_2016_1, Source: host2.xxxx.com, Destination: host3, User: admin@internal-authz). 2019-03-12 14:39:33,901+08 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-3) [] VM '5cad5c5f-5aab-46ec-a28e-d484abc0401d' was reported as Down on VDS '1bc9b9e9-1e90-4570-9930-08416d1927cc'(host3) 2019-03-12 14:39:33,903+08 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (ForkJoinPool-1-worker-3) [] START, DestroyVDSCommand(HostName = host3, DestroyVmVDSCommandParameters:{hostId='1bc9b9e9-1e90-4570-9930-08416d1927cc', vmId='5cad5c5f-5aab-46ec-a28e-d484abc0401d', secondsToWait='0', gracefully='false', reason='', ignoreNoVm='true'}), log id: c853ba5 2019-03-12 14:39:34,211+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-73) [] BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2019-03-12 14:39:34,604+08 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (ForkJoinPool-1-worker-3) [] Failed to destroy VM '5cad5c5f-5aab-46ec-a28e-d484abc0401d' because VM does not exist, ignoring 2019-03-12 14:39:34,605+08 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (ForkJoinPool-1-worker-3) [] FINISH, DestroyVDSCommand, return: , log id: c853ba5 2019-03-12 14:39:34,605+08 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-3) [] VM '5cad5c5f-5aab-46ec-a28e-d484abc0401d'(Win_2016_1) was unexpectedly detected as 'Down' on VDS '1bc9b9e9-1e90-4570-9930-08416d1927cc'(ohost3) (expected on 'f9014bc4-485c-4eb0-a9bc-42d13ed68f41') 2019-03-12 14:39:34,605+08 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-3) [] Migration of VM 'Win_2016_1' to host 'host3' failed: VM destroyed during the startup. 2019-03-12 14:39:34,615+08 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-10) [] VM '5cad5c5f-5aab-46ec-a28e-d484abc0401d'(Win_2016_1) moved from 'MigratingFrom' --> 'Up' 2019-03-12 14:39:34,615+08 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-10) [] Adding VM '5cad5c5f-5aab-46ec-a28e-d484abc0401d'(Win_2016_1) to re-run list 2019-03-12 14:39:34,621+08 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.VmsMonitoring] (ForkJoinPool-1-worker-10) [] Rerun VM '5cad5c5f-5aab-46ec-a28e-d484abc0401d'. Called from VDS 'host2.xxxx.com' 2019-03-12 14:39:34,752+08 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-53959) [] START, MigrateStatusVDSCommand(HostName = host2.xxxx.com, MigrateStatusVDSCommandParameters:{hostId='f9014bc4-485c-4eb0-a9bc-42d13ed68f41', vmId='5cad5c5f-5aab-46ec-a28e-d484abc0401d'}), log id: 7ded4ad7 2019-03-12 14:39:34,760+08 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-53959) [] FINISH, MigrateStatusVDSCommand, return: , log id: 7ded4ad7 2019-03-12 14:39:34,786+08 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-53959) [] EVENT_ID: VM_MIGRATION_TO_SERVER_FAILED(120), Migration failed (VM: Win_2016_1, Source: host2.xxxx.com, Destination: host3). Any help is greatly appreciated. regards, Bong SF

2 1

Re: VM has been paused due to a storage I/O error
by Strahil 19 Mar '19

19 Mar '19

This should not happen with replica 3 volumes. Are you sure you don't have a fluster brick out of sync/disconnected ? Best Regards, Strahil Nikolov

3 2

Where to find the hooks' print
by zodaoko＠gmail.com 19 Mar '19

19 Mar '19

Hi there, I created a before_set_num_of_cpus hook: # more /usr/libexec/vdsm/hooks/before_set_num_of_cpus/before.py #!/usr/bin/python import os import sys if os.environ.has_key('never_existed'): sys.stderr.write('cantsetcpu: before_cpu_set: cannot set cpu.\n') sys.exit(2) else: sys.stdout.write('hook ok.\n') sys.exit(0) But I cannot find the message "hook ok" in engine.log or vdsm.log, where to find it? Thank you very much. Thank you, -Zhen

1 0

Host affinity hard rule doesn't work
by zodaoko＠gmail.com 19 Mar '19

19 Mar '19

Hi there, Here is my setup: oVirt engine: 4.2.8 1. Create an affinity group as below: VM affinity rule: positive + enforcing Host affinity rule: disabled. VMs: 2 VMs added Hosts: No host selected. 2. Run the 2 VMs, they are running on the same host, say host1. 3. Change the affinity group's host affinity: Host affinity rule: positive + enforcing Hosts: host2 added. I expect that the 2 VMs can migrate to host2, but that never happen, is this expected? snippet of engine.log: 2019-03-13 07:47:05,747Z INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] Candidate host 'dub-svrfarm24' ('76b13e75-d01b-4dec-9298-1fad72b46525') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'VmAffinityGroups' (correlation id: null) 2019-03-13 07:47:05,747Z DEBUG [org.ovirt.engine.core.bll.scheduling.arem.AffinityRulesEnforcer] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] VM 822b37b7-5da3-453c-b775-d4192c2fdcae is NOT a viable candidate for solving the affinity group violation situation. 2019-03-13 07:47:05,747Z DEBUG [org.ovirt.engine.core.bll.scheduling.arem.AffinityRulesEnforcer] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] No vm to hosts soft-affinity group violation detected 2019-03-13 07:47:05,749Z DEBUG [org.ovirt.engine.core.bll.scheduling.arem.AffinityRulesEnforcer] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] No affinity group collision detected for cluster 8fe88b8c-966c-4c21-839d-e2437cc6b73d. Standing by. 2019-03-13 07:47:05,749Z DEBUG [org.ovirt.engine.core.bll.scheduling.arem.AffinityRulesEnforcer] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] No affinity group collision detected for cluster 3beac2ea-ed04-4f40-9ce3-5a9a67cebd8c. Standing by. 2019-03-13 07:47:05,750Z DEBUG [org.ovirt.engine.core.bll.scheduling.arem.AffinityRulesEnforcer] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] No affinity group collision detected for cluster da32d154-4303-11e9-9607-00163eaab080. Standing by. Thank you, -Zhen

3 4

Forum available
by Josep Manel Andrés Moscardó 18 Mar '19

18 Mar '19

Hi all, I am just wondering if anyone like me would like to have everything that is bump here in a forum, with all the benefits it brings (and people will still be able to subscribe and reply through email). Something like Discourse would be nice in my opinion. Best.

10 22

Cancel storage migration task?
by Alan G 18 Mar '19

18 Mar '19

Hi, I accidentally triggered a storage migration for a large vdisk that will take some hours to complete. Is there a way to cleanly cancel the task, such that the vdisk will remain on the original domain? Thanks, Alan

2 4