- Users - oVirt List Archives

vm operation after host fence
by Kapetanakis Giannis 06 Sep '18

06 Sep '18

Hi, Last night we have an incident of a failed host. Engine issued a fence but did not restart the vms running on that node on other operational hosts. I'd like to know if this is normal or I can tune it somehow. Here are some relevant logs from engine: 2018-09-05 03:00:51,496+03 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (EE-ManagedThreadFactory-engine-Thread-827644) [] Host 'v3' is not responding. It will stay in Connecting state for a grace period of 63 seconds and after that an attempt to fence the host will be issued. 2018-09-05 03:01:11,945+03 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [] Failed to fetch vms info for host 'v3' - skipping VMs monitoring. 2018-09-05 03:01:48,028+03 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-827679) [] EVENT_ID: VM_SET_TO_UNKNOWN_STATUS(142), VM vm7 was set to the Unknown status. 2018-09-05 03:02:10,033+03 INFO [org.ovirt.engine.core.bll.pm.StopVdsCommand] (EE-ManagedThreadFactory-engine-Thread-827680) [30369e01] Power-Management: STOP of host 'v3' initiated. 2018-09-05 03:02:55,935+03 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-827680) [3adcac38] EVENT_ID: VM_WAS_SET_DOWN_DUE_TO_HOST_REBOOT_OR_MANUAL_FENCE(143), Vm vm7 was shut down due to v3 host reboot or manual fence 2018-09-05 03:02:56,018+03 INFO [org.ovirt.engine.core.bll.pm.StopVdsCommand] (EE-ManagedThreadFactory-engine-Thread-827680) [ea0f582] Power-Management: STOP host 'v3' succeeded. 2018-09-05 03:08:20,818+03 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [326878] EVENT_ID: VDS_DETECTED(13), Status of host v3 was set to Up. 2018-09-05 03:08:23,391+03 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [] VM '3b1262ef-7fff-40af-b85e-9fd01a4f422b'(vm7) was unexpectedly detected as 'Down' on VDS '4970369d-21c2-467d-9247-c73ca2d71b3e'(v3) (expected on 'null') As you can see, engine does a fence on node v3. vm7 as well as the others running on that node did not re-start. any tips? engine is ovirt-engine-4.2.5.3-1.el7.noarch and host is vdsm-4.20.35-1.el7.x86_64 best regards, Giannis

2 2

oVirt: Training, Webinars, Forums etc!!
by femi adegoke 06 Sep '18

06 Sep '18

Are there any plans for training's/webinars/workshops/documentation etc. Anything to help learn, process more info & get more comfortable with oVirt?

1 0

OVN and impact of missing connectivity with OVN Provider/Central
by Gianluca Cecchi 05 Sep '18

05 Sep '18

Hello, I have VM1 and VM2 with their vnics on OVN. They are running on the same host. Suppose this host (and so its OVN Controller) looses connectivity with the OVN Provider (that in my case runs on oVirt engine, that is an external server). Is it correct/expected that VM1 looses connectivity with VM2 until fixed? So, in other words, is the OVN Provider a sort of single point of failure (eg if I restart enigine in my case)? Thanks, Gianluca

2 2

Engine Setup Error
by Sakhi Hadebe 05 Sep '18

05 Sep '18

Hi, We are deploying the hosted engine on oVirt-Node-4.2.3.1 using the command "hosted-engine --deploy". After providing answers it runs the ansible script and hit the Error when creating glusterfs storage domain. Attached the screenshot of the ERROR. Please help.

5 17

problem with multipath.conf
by g.vasilopoulos＠uoc.gr 05 Sep '18

05 Sep '18

after upgrading to 4.20.39-1.el7 sas multipath stopped being detected I did a diff to the two files Is the current behaviour the correct one or the previous one ? I think sas multipath should also be detected no ? [root@g1-car0136 etc]# diff multipath.conf multipath.conf.201809031555 1c1 < # VDSM REVISION 1.6 --- > # VDSM REVISION 1.5 101,109d100 < } < < # Whitelist FCP and iSCSI devices. < blacklist { < protocol ".*" < } < < blacklist_exceptions { < protocol "(scsi:fcp|scsi:iscsi)"

3 4

ERROR: Malformed metadata for host 3
by info＠linuxfabrik.ch 05 Sep '18

05 Sep '18

Hi, we have a Hyperconverged oVirt Cluster with three machines. All three are running GlusterFS (Replica 3), and on two of them we are running the VMs (because they have better hardware). Now we wanted to add a new oVirt host. Due to some network related errors, adding this host was unsuccessful, so we removed it from GUI and also via "hosted-engine --clean-metadata --host-id=3 --force-clean" (as it was still listed using "hosted-engine --vm-status" after removal from the GUI). Inside the GUI and with "hosted-engine --vm-status", everything looks fine now, the third oVirt host is gone, and there are no problems at all. But in the /var/log/ovirt-hosted-engine-ha/agent.log, we now get every 15 minutes a "MainThread::ERROR::2018-09-05 10:20:17,190::hosted_engine::793::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(process_remote_metadata) Malformed metadata for host 3: received 0 of 512 expected bytes", on both of the remaining oVirt hosts. Can someone explain what is wrong? oVirt 4.2.2.6-1.el7.centos Thank you Markus

1 0

qemu-kvm memory leak in 4.2.5
by Hesham Ahmed 05 Sep '18

05 Sep '18

Starting oVirt 4.2.4 (also in 4.2.5 and maybe in 4.2.3) I am facing some sort of memory leak. The memory usage on the hosts keep increasing till it reaches somewhere around 97%. Putting the host in maintenance and back resolves it. The memory usage by the qemu-kvm processes is way above the defined VM memory for instance below is the memory usage of a VM: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 12271 qemu 20 0 35.4g 30.9g 8144 S 13.3 49.3 9433:15 qemu-kvm The VM memory settings are: Defined Memory: 8192 MB Physical Memory Guaranteed: 5461 MB This is for a 3 node hyperconverged cluster running on latest oVirt Node 4.2.5.

3 2

Local storage formatting
by Matt Simonsen 04 Sep '18

04 Sep '18

Hello, I'm running oVirt with several data centers, some with NFS storage and some with local storage. I had problems in the past with a large pool and local storage. The problem was nodectl showed the pool being too full (I think >80%), but it was only the images that made the pool "full" -- and this storage was carefully setup such that there was no chance it would actually fill. The LVs for oVirt itself were all under 20%, yet nodectl still reported the pool was too full. My solution so far has been to use our RAID card tools, so that sda is the oVirt node install, and sdb is for images. There are probably other good reasons for me to handle it this way, for example being able to use different RAID levels, but I'm hoping someone can confirm my partitioning below doesn't have some risk I'm now yet aware of. I setup a new volume group for images, as below: [root@node4-g8-h4 multipath]# pvs PV VG Fmt Attr PSize PFree /dev/mapper/3600508b1001c7e172160824d7b204c3b2 onn_node4-g8-h4 lvm2 a-- <119.00g <22.85g /dev/sdb1 data lvm2 a-- 1.13t <361.30g [root@node4-g8-h4 multipath]# vgs VG #PV #LV #SN Attr VSize VFree data 1 1 0 wz--n- 1.13t <361.30g onn_node4-g8-h4 1 13 0 wz--n- <119.00g <22.85g [root@node4-g8-h4 multipath]# lvs LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert images_main data -wi-ao---- 800.00g home onn_node4-g8-h4 Vwi-aotz-- 1.00g pool00 4.79 ovirt-node-ng-4.2.5.1-0.20180816.0 onn_node4-g8-h4 Vwi---tz-k 64.10g pool00 root ovirt-node-ng-4.2.5.1-0.20180816.0+1 onn_node4-g8-h4 Vwi---tz-- 64.10g pool00 ovirt-node-ng-4.2.5.1-0.20180816.0 ovirt-node-ng-4.2.6-0.20180903.0 onn_node4-g8-h4 Vri---tz-k 64.10g pool00 ovirt-node-ng-4.2.6-0.20180903.0+1 onn_node4-g8-h4 Vwi-aotz-- 64.10g pool00 ovirt-node-ng-4.2.6-0.20180903.0 4.83 pool00 onn_node4-g8-h4 twi-aotz-- 91.10g 8.94 0.49 root onn_node4-g8-h4 Vwi---tz-- 64.10g pool00 swap onn_node4-g8-h4 -wi-ao---- 4.00g tmp onn_node4-g8-h4 Vwi-aotz-- 1.00g pool00 4.87 var onn_node4-g8-h4 Vwi-aotz-- 15.00g pool00 3.31 var_crash onn_node4-g8-h4 Vwi-aotz-- 10.00g pool00 2.86 var_log onn_node4-g8-h4 Vwi-aotz-- 8.00g pool00 3.57 var_log_audit onn_node4-g8-h4 Vwi-aotz-- 2.00g pool00 4.89 The images_main is setup as "Block device for filesystems" with ext4. Is there any reason I should consider pool for thinly provisioned volumes? I don't need to over-allocate storage and it seems to me like a fixed partition is ideal. Please confirm or let me know if there's anything else I should consider. Thanks Matt

2 3

self-hosted engine lost password
by Rob Epping 04 Sep '18

04 Sep '18

Hello list, I'm trying to update our ovirt self-hosted engine to the latest version using the docs at https://www.ovirt.org/documentation/upgrade-guide/chap-Updates_between_ Minor_Releases/#updating-the-ovirt-self-hosted-engine-and-underlying- virtualization-hosts For this I need to logon to the engine VM, but none of the passwords we used seems to work. I can not seem to find information on how te reset passwords of the hosted engine, so I'm stuck. Am I doing something wrong or is access to the self-hosted engine required. When access is required, is there a way to reset the password? THNX && GRTNX, RobJE

2 2

Upgraded host, engine now won't boot
by Jim Kusznir 04 Sep '18

04 Sep '18

Hello: I saw that there were updates to my ovirt-4.2 3 node hyperconverged system, so I proceeded to apply them the usual way through the UI. At one point, the hosted engine was migrated to one of the upgraded hosts, and then went "unstable" on me. Now, the hosted engine appears to be crashed: It gets powered up, but it never boots up to the point where it responds to pings or allows logins. After a while, the hosted engine shows status (via console "hosted-engine --vm-status" command) "Powering Down". It stays there for a long time. I tried forcing a poweroff then powering it on, but again, it never gets up to where it will respond to pings. --vm-status shows bad health, but up. I tried running the hosted-engine --console command, but got: [root@ovirt1 ~]# hosted-engine --console The engine VM is running on this host Connected to domain HostedEngine Escape character is ^] error: internal error: cannot find character device <null> [root@ovirt1 ~]# I tried to run the hosted-engine --upgrade-appliance command, but it hangs at obtaining certificate (understandably, as the hosted-engine is not up). How do i recover from this? And what caused this? --Jim

2 10