[ovirt-users] Re: Manual Migration not working and Dashboard broken after 4.3.4 update

Hi, For the dashboard: If ovirt-engine-dwh is still installed and running after upgrade (service ovirt-engine-dwhd restart) then can you please re-check the ovirt-engine-dwh.log file for errors? @Shirly Radco <sradco(a)redhat.com&gt; anything else to check? For the Migrate option, please attach again your browser console log snippet when you have the problem and also a screenshot of the error. Please also attach the engine log (the warnings you mentioned are not related to those issues). Thanks, Sharon On Tue, Jul 16, 2019 at 4:14 PM Neil <nwilson123(a)gmail.com&gt; wrote: > Hi Sharon, > > Thank you for coming back to me. > > Unfortunately I've upgraded to 4.3.5 today and both issues still persist. > I have also tried clearing all data out of my browser and re-logged back in. > > I see a new error though in my engine.log as below, however I still don't > see anything logged when I click the migrate button... > > 2019-07-16 15:01:19,600+02 WARN > [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-15) > [685e07c0-b76f-4093-afc9-7c3999ee4ae2] Field 'balloonEnabled' can not be > updated when status is 'Up' > 2019-07-16 15:01:19,601+02 WARN > [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-15) > [685e07c0-b76f-4093-afc9-7c3999ee4ae2] Field 'watchdog' can not be updated > when status is 'Up' > 2019-07-16 15:01:19,602+02 WARN > [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-15) > [685e07c0-b76f-4093-afc9-7c3999ee4ae2] Field 'rngDevice' can not be updated > when status is 'Up' > 2019-07-16 15:01:19,602+02 WARN > [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-15) > [685e07c0-b76f-4093-afc9-7c3999ee4ae2] Field 'soundDeviceEnabled' can not > be updated when status is 'Up' > 2019-07-16 15:01:19,603+02 WARN > [org.ovirt.engine.core.utils.ObjectIdentityChecker] (default task-15) > [685e07c0-b76f-4093-afc9-7c3999ee4ae2] Field 'consoleEnabled' can not be > updated when status is 'Up' > > Then in my vdsm.log I'm seeing the following error.... > > 2019-07-16 15:05:59,038+0200 WARN (qgapoller/3) > [virt.periodic.VmDispatcher] could not run <function <lambda> at > 0x7f00a00476e0> on ['ded20d05-f558-4e17-bf2d-e4907e1bbcde', > '8c93b301-b50d-4d3d-b6cb-54abb3d7f0bb', > '8d8571bf-a7ce-4e73-8d3e-fe1a2aab9b4b', > '2489c75f-2758-4d82-8338-12f02ff78afa', > '9a6561b8-5702-43dc-9e92-1dc5dfed4eef', > '523ad9ee-5738-42f2-9ee1-50727207e93b', > '84f4685b-39e1-4bc8-b8ab-755a2c325cb0', > '43c06f86-2e37-410b-84be-47e83052344a', > '6f44a02c-5de6-4002-992f-2c2c5feb2ee5', > '19844323-b3cc-441a-8d70-e45326848b10', > '77872f3d-c69f-48ab-992b-1d2765a38481'] (periodic:289) > > 2019-07-16 15:06:09,036+0200 WARN (qgapoller/2) > [virt.periodic.VmDispatcher] could not run <function <lambda> at > 0x7f00a00476e0> on ['ded20d05-f558-4e17-bf2d-e4907e1bbcde', > '8c93b301-b50d-4d3d-b6cb-54abb3d7f0bb', > '8d8571bf-a7ce-4e73-8d3e-fe1a2aab9b4b', > '2489c75f-2758-4d82-8338-12f02ff78afa', > '9a6561b8-5702-43dc-9e92-1dc5dfed4eef', > '523ad9ee-5738-42f2-9ee1-50727207e93b', > '84f4685b-39e1-4bc8-b8ab-755a2c325cb0', > '43c06f86-2e37-410b-84be-47e83052344a', > '6f44a02c-5de6-4002-992f-2c2c5feb2ee5', > '19844323-b3cc-441a-8d70-e45326848b10', > '77872f3d-c69f-48ab-992b-1d2765a38481'] (periodic:289) > > I'm not sure if this is related to either of the above issues though, but > I can attach the full log if needed. > > Please shout if there is anything else you think I can try doing. > > Thank you. > > Regards. > > Neil Wilson > > > > > On Mon, Jul 15, 2019 at 11:29 AM Sharon Gratch <sgratch(a)redhat.com&gt; > wrote: > >> Hi Neil, >> >> Regarding issue 1 (Dashboard): >> I recommend to upgrade to latest oVirt version 4.3.5, for this fix as >> well as other enhancements and bug fixes. >> For oVirt 4.3.5 installation / upgrade instructions: >> http://www.ovirt.org/release/4.3.5/ >> >> Regarding issue 2 (Manual Migrate dialog): >> If it will be reproduced after upgrading then please try to clean your >> browser caching before running the admin portal. It might help. >> >> Regards, >> Sharon >> >> On Thu, Jul 11, 2019 at 1:24 PM Neil <nwilson123(a)gmail.com&gt; wrote: >> >>> >>> Hi Sharon, >>> >>> Thanks for the assistance. >>> On Thu, Jul 11, 2019 at 11:58 AM Sharon Gratch <sgratch(a)redhat.com&gt; >>> wrote: >>> >>>> Hi, >>>> >>>> Regarding issue 1 (Dashboard): >>>> Did you upgrade the engine to 4.3.5? There was a bug fixed in version >>>> 4.3.4-5 https://bugzilla.redhat.com/show_bug.cgi?id=1713967 and it >>>> may be the same issue. >>>> >>> >>> >>> No I wasn't aware that there were updates, how do I obtain 4.3.4-5 is >>> there another repo available? >>> >>> Regarding issue 2 (Manual Migrate dialog): >>>> Can you please attach your browser console log and engine.log snippet >>>> when you have the problem? >>>> If you could take from the console log the actual REST API response, >>>> that would be great. >>>> The request will be something like >>>> <engine>/api/hosts?migration_target_of=... >>>> >>> >>> Please see attached text log for the browser console, I don't see any >>> REST API being logged, just a stack trace error. >>> The engine.log literally doesn't get updated when I click the Migrate >>> button so there isn't anything to share unfortunately. >>> >>> Please shout if you need further info. >>> >>> Thank you! >>> >>> >>> >>> >>>> >>>> >>>> On Thu, Jul 11, 2019 at 10:04 AM Neil <nwilson123(a)gmail.com&gt; wrote: >>>> >>>>> Hi everyone, >>>>> Just an update. >>>>> >>>>> I have both hosts upgraded to 4.3, I have upgraded my DC and cluster >>>>> to 4.3 and I'm still faced with the same problems. >>>>> >>>>> 1.) My Dashboard says the following "Error! Could not fetch dashboard >>>>> data. Please ensure that data warehouse is properly installed and >>>>> configured." >>>>> >>>>> 2.) When I click the Migrate button I get the error "Could not fetch >>>>> data needed for VM migrate operation" >>>>> >>>>> Upgrading my hosts resolved the "node status: DEGRADED" issue so at >>>>> least it's one issue down. >>>>> >>>>> I've done an engine-upgrade-check and a yum update on all my hosts >>>>> and engine and there are no further updates or patches waiting. >>>>> Nothing is logged in my engine.log when I click the Migrate button >>>>> either. >>>>> >>>>> Any ideas what to do or try for 1 and 2 above? >>>>> >>>>> Thank you. >>>>> >>>>> Regards. >>>>> >>>>> Neil Wilson. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Thu, Jul 11, 2019 at 8:27 AM Alex K <rightkicktech(a)gmail.com&gt; >>>>> wrote: >>>>> >>>>>> >>>>>> >>>>>> On Thu, Jul 11, 2019 at 7:57 AM Michal Skrivanek < >>>>>> michal.skrivanek(a)redhat.com&gt; wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> On 11 Jul 2019, at 06:34, Alex K <rightkicktech(a)gmail.com&gt; wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, Jul 9, 2019, 19:10 Michal Skrivanek < >>>>>>> michal.skrivanek(a)redhat.com&gt; wrote: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 9 Jul 2019, at 17:16, Strahil <hunter86_bg(a)yahoo.com&gt; wrote: >>>>>>>> >>>>>>>> I'm not sure, but I always thought that you need an agent for >>>>>>>> live migrations. >>>>>>>> >>>>>>>> >>>>>>>> You don’t. For snapshots, and other less important stuff like >>>>>>>> reporting IPs you do. In 4.3 you should be fine with qemu-ga only >>>>>>>> >>>>>>> I've seen resolving live migration issues by installing newer >>>>>>> versions of ovirt ga. >>>>>>> >>>>>>> >>>>>>> Hm, it shouldn’t make any difference whatsoever. Do you have any >>>>>>> concrete data? that would help. >>>>>>> >>>>>> That is some time ago when runnign 4.1. No data unfortunately. Also >>>>>> did not expect ovirt ga to affect migration, but experience showed me that >>>>>> it did. The only observation is that it affected only Windows VMs. Linux >>>>>> VMs never had an issue, regardless of ovirt ga. >>>>>> >>>>>>> You can always try installing either qemu-guest-agent or >>>>>>>> ovirt-guest-agent and check if live migration between hosts is possible. >>>>>>>> >>>>>>>> Have you set the new cluster/dc version ? >>>>>>>> >>>>>>>> Best Regards >>>>>>>> Strahil Nikolov >>>>>>>> On Jul 9, 2019 17:42, Neil <nwilson123(a)gmail.com&gt; wrote: >>>>>>>> >>>>>>>> I remember seeing the bug earlier but because it was closed >>>>>>>> thought it was unrelated, this appears to be it.... >>>>>>>> >>>>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=1670701 >>>>>>>> >>>>>>>> Perhaps I'm not understanding your question about the VM guest >>>>>>>> agent, but I don't have any guest agent currently installed on the VM, not >>>>>>>> sure if the output of my qemu-kvm process maybe answers this question?.... >>>>>>>> >>>>>>>> /usr/libexec/qemu-kvm -name >>>>>>>> guest=Headoffice.cbl-ho.local,debug-threads=on -S -object >>>>>>>> secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-1-Headoffice.cbl-ho.lo/master-key.aes >>>>>>>> -machine pc-i440fx-rhel7.3.0,accel=kvm,usb=off,dump-guest-core=off -cpu >>>>>>>> Broadwell,vme=on,f16c=on,rdrand=on,hypervisor=on,arat=on,xsaveopt=on,abm=on,rtm=on,hle=on >>>>>>>> -m 8192 -realtime mlock=off -smp 8,maxcpus=64,sockets=16,cores=4,threads=1 >>>>>>>> -numa node,nodeid=0,cpus=0-7,mem=8192 -uuid >>>>>>>> 9a6561b8-5702-43dc-9e92-1dc5dfed4eef -smbios >>>>>>>> type=1,manufacturer=oVirt,product=oVirt >>>>>>>> Node,version=7-3.1611.el7.centos,serial=4C4C4544-0034-5810-8033- >>>>>>>> >>>>>>>> >>>>>>> It’s 7.3, likely oVirt 4.1. Please upgrade... >>>>>>> >>>>>>> C2C04F4E4B32,uuid=9a6561b8-5702-43dc-9e92-1dc5dfed4eef >>>>>>>> -no-user-config -nodefaults -chardev >>>>>>>> socket,id=charmonitor,fd=31,server,nowait -mon >>>>>>>> chardev=charmonitor,id=monitor,mode=control -rtc >>>>>>>> base=2019-07-09T10:26:53,driftfix=slew -global >>>>>>>> kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -boot strict=on >>>>>>>> -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device >>>>>>>> virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x4 -device >>>>>>>> virtio-serial-pci,id=virtio-serial0,max_ports=16,bus=pci.0,addr=0x5 -drive >>>>>>>> if=none,id=drive-ide0-1-0,readonly=on -device >>>>>>>> ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -drive >>>>>>>> file=/rhev/data-center/59831b91-00a5-01e4-0294-000000000018/8a607f8a-542a-473c-bb18-25c05fe2a3d4/images/56e8240c-a172-4f52-b0c1-2bddc4f34f93/9f245467-d31d-4f5a-8037-7c5012a4aa84,format=qcow2,if=none,id=drive-virtio-disk0,serial=56e8240c-a172-4f52-b0c1-2bddc4f34f93,werror=stop,rerror=stop,cache=none,aio=native >>>>>>>> -device >>>>>>>> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1,write-cache=on >>>>>>>> -netdev tap,fd=33,id=hostnet0,vhost=on,vhostfd=34 -device >>>>>>>> virtio-net-pci,netdev=hostnet0,id=net0,mac=00:1a:4a:16:01:5b,bus=pci.0,addr=0x3 >>>>>>>> -chardev socket,id=charchannel0,fd=35,server,nowait -device >>>>>>>> virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.rhevm.vdsm >>>>>>>> -chardev socket,id=charchannel1,fd=36,server,nowait -device >>>>>>>> virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0 >>>>>>>> -chardev spicevmc,id=charchannel2,name=vdagent -device >>>>>>>> virtserialport,bus=virtio-serial0.0,nr=3,chardev=charchannel2,id=channel2,name=com.redhat.spice.0 >>>>>>>> -spice tls-port=5900,addr=10.0.1.11,x509-dir=/etc/pki/vdsm/libvirt-spice,tls-channel=default,tls-channel=main,tls-channel=display,tls-channel=inputs,tls-channel=cursor,tls-channel=playback,tls-channel=record,tls-channel=smartcard,tls-channel=usbredir,seamless-migration=on >>>>>>>> -device >>>>>>>> qxl-vga,id=video0,ram_size=67108864,vram_size=8388608,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pci.0,addr=0x2 >>>>>>>> -incoming defer -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 >>>>>>>> -object rng-random,id=objrng0,filename=/dev/urandom -device >>>>>>>> virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x8 -sandbox >>>>>>>> on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny >>>>>>>> -msg timestamp=on >>>>>>>> >>>>>>>> Please shout if you need further info. >>>>>>>> >>>>>>>> Thanks. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Jul 9, 2019 at 4:17 PM Strahil Nikolov < >>>>>>>> hunter86_bg(a)yahoo.com&gt; wrote: >>>>>>>> >>>>>>>> Shouldn't cause that problem. >>>>>>>> >>>>>>>> You have to find the bug in bugzilla and report a regression (if >>>>>>>> it's not closed) , or open a new one and report the regression. >>>>>>>> As far as I remember , only the dashboard was affected due to new >>>>>>>> features about vdo disk savings. >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Users mailing list -- users(a)ovirt.org >>>>>>>> To unsubscribe send an email to users-leave(a)ovirt.org >>>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>>> oVirt Code of Conduct: >>>>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>>>> List Archives: >>>>>>>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/IQCHU3VAIQQ... >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Users mailing list -- users(a)ovirt.org >>>>>>>> To unsubscribe send an email to users-leave(a)ovirt.org >>>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>>> oVirt Code of Conduct: >>>>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>>>> List Archives: >>>>>>>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/RVCCY6JWXWH... >>>>>>>> >>>>>>> _______________________________________________ >>>>>> Users mailing list -- users(a)ovirt.org >>>>>> To unsubscribe send an email to users-leave(a)ovirt.org >>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>> oVirt Code of Conduct: >>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>>> List Archives: >>>>>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/3OITNPMYSTE... >>>>>> >>>>> _______________________________________________ >>>>> Users mailing list -- users(a)ovirt.org >>>>> To unsubscribe send an email to users-leave(a)ovirt.org >>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>> oVirt Code of Conduct: >>>>> https://www.ovirt.org/community/about/community-guidelines/ >>>>> List Archives: >>>>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/AWMVWDDQMY2... >>>>> >>>>