- Users - oVirt List Archives

Re: Changing ISO domains
by Strahil 18 Sep '19

18 Sep '19

Yes. But check SELINUX and DAC permissions. Best Regards, Strahil NikolovOn Sep 17, 2019 18:04, Mark Steele <msteele(a)telvue.com> wrote: > > OK - so if I put them in the 1111-11111... directory with scp that should work? > > > *** > Mark Steele > CIO / VP Technical Operations | TelVue Corporation > TelVue - We Share Your Vision > 16000 Horizon Way, Suite 100 | Mt. Laurel, NJ 08054 > 800.885.8886 x128 | msteele(a)telvue.com | http://www.telvue.com > twitter: http://twitter.com/telvue | facebook: https://www.facebook.com/telvue > > > On Tue, Sep 17, 2019 at 10:57 AM Derek Atkins <derek(a)ihtfp.com> wrote: >> >> If you push down a level or two you'll see your .iso files. >> I just scp them directly in. >> -derek >> >> >> On Tue, September 17, 2019 10:52 am, Mark Steele wrote: >> > Additional information: >> > >> > The directory for the domain appear to have been created properly so I am >> > not clear on why the upload from the ovirt engine is failing: >> > drwxr-xr-x. 3 36 36 88 Sep 17 10:40 . >> > drwxr-xr-x. 5 36 36 126 Oct 17 2018 .. >> > drwxr-xr-x. 4 36 36 46 Sep 17 10:40 bdd2a547-dde9-4248-b2de-ae67063da8e4 >> > -rwxr-xr-x. 1 36 36 0 Sep 17 10:50 __DIRECT_IO_TEST__ >> > [root@phl-tevestore-01 iso-store]# >> > >> > *** >> > *Mark Steele* >> > CIO / VP Technical Operations | TelVue Corporation >> > TelVue - We Share Your Vision >> > 16000 Horizon Way, Suite 100 | Mt. Laurel, NJ 08054 >> > 800.885.8886 x128 | msteele(a)telvue.com | http://www.telvue.com >> > twitter: http://twitter.com/telvue | facebook: >> > https://www.facebook.com/telvue >> > >> > >> > On Tue, Sep 17, 2019 at 10:46 AM Mark Steele <msteele(a)telvue.com> wrote: >> > >> >> I was not using the uploader - I have the new domain active and attached >> >> now. >> >> >> >> I am attempting to upload the iso using the following command: >> >> >> >> engine-iso-uploader --iso-domain=phl-iso-03 upload >> >> ./windows-server-2012.iso >> >> >> >> Unfortunately I keep getting this error: >> >> >> >> Uploading, please wait... >> >> ERROR: mount.nfs: Connection timed out >> >> >> >> >> >> >> >> *** >> >> *Mark Steele* >> >> CIO / VP Technical Operations | TelVue Corporation >> >> TelVue - We Share Your Vision >> >> 16000 Horizon Way, Suite 100 | Mt. Laurel, NJ 08054 >> >> 800.885.8886 x128 | msteele(a)telvue.com | http://www.telvue.com >> >> twitter: http://twitter.com/telvue

1 0

Re: Disaster recovery active-active question
by Strahil 18 Sep '19

18 Sep '19

We use Active-Active Hp XP replication between 2 DCs for our VmWare Clusters. I guess you can use the same replication for oVirt or even Hyper-V. Best Regards, Strahil NikolovOn Sep 17, 2019 14:30, wodel youchi <wodel.youchi(a)gmail.com> wrote: > > Hi, > > On the active-active DR, in the documentation it is said : > "You require replicated storage that is writeable on both sites to allow virtual machines to migrate between sites and continue running on the site’s storage." > > I know that it's not a direct oVirt question, but if someone has implemented this ... > > Which type of storage that can offer both real time synchronization and read/write on both ends on the replicated volumes? > > > Regards.

1 0

GFS/GFS2
by marzoccafabio＠gmail.com 18 Sep '19

18 Sep '19

Is GFS/GFS2 supported by oVirt as shared storage?

3 3

FW: user role
by Staniforth, Paul 18 Sep '19

18 Sep '19

Hi Kim, If you add them to a group in AD then add the users and group to oVirt you can add the permissions to the group. Then add the VmCreator and DiskProfileUser to the Datacenter. Add the UserTemplateBasedVM permission to any templates you want them to be able to use Add the VnicProfileUser permission to any vNIC profiles they should have access ( make sure public isn't enabled on any vNIC profiles you don't want them to have access to) When they create a VM they will then get UserVmManager permission and the DiskOperator permissions on any disks. This means they should be able to do all 1 to 8 for their own VMs and also delete their VMs and Disks. Paul Staniforth To view the terms under which this email is distributed, please go to:- http://leedsbeckett.ac.uk/disclaimer/email/

1 0

Changing ISO domains
by Mark Steele 17 Sep '19

17 Sep '19

Hello, I recently removed our ISO domain from out cluster (oVirt Engine Version: 3.5.0.1-1.el6) because it was on a storage location we no longer want to use. I am now not clear on how to add the new ISO domain in. I created a new ISO domain type via storage and pointed to an NFS share on the new storage, however it does not behave as I anticipated (I cannot see ISO's when creating new VM's) I suspect I missed a step - how are new ISO domains added to a cluster? Mark *** *Mark Steele* CIO / VP Technical Operations | TelVue Corporation TelVue - We Share Your Vision 16000 Horizon Way, Suite 100 | Mt. Laurel, NJ 08054 800.885.8886 x128 | msteele(a)telvue.com | http://www.telvue.com twitter: http://twitter.com/telvue | facebook: https://www.facebook.com/telvue

3 9

Disaster recovery active-active question
by wodel youchi 17 Sep '19

17 Sep '19

Hi, On the active-active DR, in the documentation it is said : "You require replicated storage that is writeable on both sites to allow virtual machines to migrate between sites and continue running on the site’s storage." I know that it's not a direct oVirt question, but if someone has implemented this ... Which type of storage that can offer both real time synchronization and read/write on both ends on the replicated volumes? Regards.

1 0

Re: the virtual machine crushed and I cant shutdown the vm successfully
by Strahil 17 Sep '19

17 Sep '19

Have you checked the Hypervisor's logs. I would start with logs from libvirt and vdsm (host) and then the logs on your engine. What OS is your VM and the version of the VM itself (you can check in the UI). Best Regards, Starhil NikolovOn Sep 17, 2019 10:56, zhouhao(a)vip.friendtimes.net wrote: > > There is no useful information on Google. I can't solve this problem. I can only restart the ovirt-node > > ________________________________ > zhouhao(a)vip.friendtimes.net >> >> >> From: tedhima(a)yahoo.co.uk >> Date: 2019-09-17 15:30 >> To: users; devel; zhouhao(a)vip.friendtimes.net >> Subject: Re: [ovirt-users] the virtual machine crushed and I cant shutdown the vm successfully >> Probably related to XFS defragmentation, have you tried to google the error message? >> >> https://bugzilla.kernel.org/show_bug.cgi?id=73831 >> https://centos.org/forums/viewtopic.php?t=52412 >> https://blog.codecentric.de/en/2017/04/xfs-possible-memory-allocation-deadl… >> On Tuesday, 17 September 2019, 13:55:45 GMT+7, zhouhao(a)vip.friendtimes.net <zhouhao(a)vip.friendtimes.net> wrote: >> >> >>> >>> The trouble bothered me for a long time;Some times my vm will crush and I cant shutdown it,I had to reboot the ovirt-node to slove it; >>> the vm's error below >>> >>> The ovirt-node's error below >>> >>> The vm's threading on the ovirt-node, the IO ratio is 100% >>> >>> >>> and the vm's process change to defunct >>> I can not kill it,every time I had to shutdown the ovirt-node >>> >>> on the engine website ,the vm's status always in the way to shutdown ,even if I wait for it for hours; >>> <

1 0

Gluster: Bricks remove failed
by toslavik＠yandex.ru 17 Sep '19

17 Sep '19

Hi. There is an ovirt-hosted-engine on gluster volume engine Replicate replica count 3 Migrate to other drives. I do: gluster volume add-brick engine clemens:/gluster-bricks/engine tiberius:/gluster-bricks/engine octavius:/gluster-bricks/engine force volume add-brick: success gluster volume remove-brick engine tiberius:/engine/datastore clemens:/engine/datastore octavius:/engine/datastore start volume remove-brick start: success ID: dd9453d3-b688-4ed8-ad37-ba901615046c gluster volume remove-brick engine octavius:/engine/datastore status Node Rebalanced-files size scanned failures skipped status run time in h:m:s --------- ----------- ----------- ----------- ----------- ----------- ------------ -------------- localhost 7 50.0GB 34 1 0 completed 0:00:02 clemens 11 21.0MB 31 0 0 completed 0:00:02 tiberius 12 25.0MB 36 0 0 completed 0:00:02 gluster volume status engine Status of volume: engine Gluster process TCP Port RDMA Port Online Pid ------------------------------------------------------------------------------ Brick octavius:/engine/datastore 49156 0 Y 15669 Brick tiberius:/engine/datastore 49156 0 Y 15930 Brick clemens:/engine/datastore 49156 0 Y 16193 Brick clemens:/gluster-bricks/engine 49159 0 Y 6168 Brick tiberius:/gluster-bricks/engine 49163 0 Y 29524 Brick octavius:/gluster-bricks/engine 49159 0 Y 50056 Self-heal Daemon on localhost N/A N/A Y 50087 Self-heal Daemon on clemens N/A N/A Y 6263 Self-heal Daemon on tiberius N/A N/A Y 29583 Task Status of Volume engine ------------------------------------------------------------------------------ Task : Remove brick ID : dd9453d3-b688-4ed8-ad37-ba901615046c Removed bricks: tiberius:/engine/datastore clemens:/engine/datastore octavius:/engine/datastore Status : completed But the data did not migrate du -hs /gluster-bricks/engine/ /engine/datastore/ 49M /gluster-bricks/engine/ 20G /engine/datastore/ Can you give some advice?

2 3

oVirt 4.3 gluster hooks
by Staniforth, Paul 17 Sep '19

17 Sep '19

Hello, In an oVirt 4.3 cluster there are a number of gluster hooks out of sync. When I try to resolve any of the conflicts from the engine it comes back with operation cancelled with the error message "Error while executing action Update Gluster Hook: Internal Engine Error" In the engine log there are a number of errors including 2019-09-16 10:38:28,584+01 ERROR [org.ovirt.engine.core.vdsbroker.gluster.UpdateGlusterHookVDSCommand] (EE-ManagedThreadFactory-engine-Thread-2210463) [62ee29dd-5f80-47ad-8e5a-c97e7c3b7acb] Command 'UpdateGlusterHookVDSCommand(HostName = hostname@domainname, GlusterHookVDSParameters:{hostId='86571cea-a20c-4087-8bbf-103d26b3c4eb'})' execution failed: VDSGenericException: VDSErrorException: Failed to UpdateGlusterHookVDS, error = Internal JSON-RPC error: {'reason': "Attempt to call function: <bound method GlusterHook.update of <vdsm.gluster.apiwrapper.GlusterHook object at 0x7fb34421f510>> with arguments: (u'add-brick', u'PRE', u'28Quota-enable-root-xattr-heal.sh', u'IyEvYmluL3NoCgojIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjCiMjIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KIyMgVGhlIHNjcmlwdHMKIyMgSS4gICBhZGQtYnJpY2svcHJlL1MyOFF1b3RhLXJvb3QteGF0dHItaGVhbC5zaCAoaXRzZWxmKQojIyBJSS4gIGFkZC1icmljay9wb3N0L2Rpc2FibGVkLXJvb3QteGF0dHItaGVhbC5zaCBBTkQKIyMgY29sbGVjdGl2ZWx5IGFyY2hpZXZlcyB0aGUgam9iIG9mIGhlYWxpbmcgdGhlICdsaW1pdC1zZXQnIHhhdHRyIHVwb24KIyMgYWRkLWJyaWNrIHRvIHRoZSBnbHVzdGVyIHZvbHVtZS4KIyMKIyMgVGhpcyBzY3JpcHQgaXMgdGhlICdjb250cm9sbGluZycgc2NyaXB0LiBVcG9uIGFkZC1icmljayB0aGlzIHNjcmlwdCBlbmFibGVzCiMjIHRoZSBjb3JyZXNwb25kaW5nIHNjcmlwdCBiYXNlZCBvbiB0aGUgc3RhdHVzIG9mIHRoZSB2b2x1bWUuCiMjIElmIHZvbHVtZSBpcyBzdGFydGVkIC0gZW5hYmxlIGFkZC1icmljay9wb3N0IHNjcmlwdAojIyBlbHNlICAgICAgICAgICAgICAgICAtIGVuYWJsZSBzdGFydC9wb3N0IHNjcmlwdC4KIyMKIyMgVGhlIGVuYWJsaW5nIGFuZCBkaXNhYmxpbmcgb2YgYSBzY3JpcHQgaXMgYmFzZWQgb24gdGhlIGdsdXN0ZXJkJ3MgbG9naWMsCiMjIHRoYXQgaXQgb25seSBydW5zIHRoZSBzY3JpcHRzIHdoaWNoIHN0YXJ0cyBpdHMgbmFtZSB3aXRoICdTJy4gU28sCiMjIEVuYWJsZSAtIHN5bWxpbmsgdGhlIGZpbGUgdG8gJ1MnKi4KIyMgRGlzYWJsZS0gdW5saW5rIHN5bWxpbmsKIyMgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQojIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjCgpPUFRTUEVDPSJ2b2xuYW1lOix2ZXJzaW9uOixnZC13b3JrZGlyOix2b2x1bWUtb3A6IgpQUk9HTkFNRT0iUXVvdGEteGF0dHItaGVhbC1hZGQtYnJpY2stcHJlIgpWT0xfTkFNRT0KR0xVU1RFUkRfV09SS0RJUj0KVk9MVU1FX09QPQpWRVJTSU9OPQpFTkFCTEVEX05BTUU9IlMyOFF1b3RhLXJvb3QteGF0dHItaGVhbC5zaCIKRElTQUJMRURfTkFNRT0iZGlzYWJsZWQtcXVvdGEtcm9vdC14YXR0ci1oZWFsLnNoIgoKZW5hYmxlICgpCnsKICAgICAgICBsbiAtc2YgJERJU0FCTEVEX1NUQVRFICQxOwp9CgojIy0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQojIyBQYXJzZSB0aGUgYXJndW1lbnRzCiMjLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCkFSR1M9JChnZXRvcHQgLWwgJE9QVFNQRUMgIC1uYW1lICRQUk9HTkFNRSAkQCkKZXZhbCBzZXQgLS0gIiRBUkdTIgoKd2hpbGUgdHJ1ZTsKZG8KICAgIGNhc2UgJDEgaW4KICAgICAgICAtLXZvbG5hbWUpCiAgICAgICAgICAgIHNoaWZ0CiAgICAgICAgICAgIFZPTF9OQU1FPSQxCiAgICAgICAgICAgIDs7CiAgICAgICAgLS1nZC13b3JrZGlyKQogICAgICAgICAgICBzaGlmdAogICAgICAgICAgICBHTFVTVEVSRF9XT1JLRElSPSQxCiAgICAgICAgICAgIDs7CiAgICAgICAgLS12b2x1bWUtb3ApCiAgICAgICAgICAgIHNoaWZ0CiAgICAgICAgICAgIFZPTFVNRV9PUD0kMQogICAgICAgICAgICA7OwogICAgICAgIC0tdmVyc2lvbikKICAgICAgICAgICAgc2hpZnQKICAgICAgICAgICAgVkVSU0lPTj0kMQogICAgICAgICAgICA7OwogICAgICAgICopCiAgICAgICAgICAgIHNoaWZ0CiAgICAgICAgICAgIGJyZWFrCiAgICAgICAgICAgIDs7CiAgICBlc2FjCiAgICBzaGlmdApkb25lCiMjLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQoKRElTQUJMRURfU1RBVEU9IiRHTFVTVEVSRF9XT1JLRElSL2hvb2tzLyRWRVJTSU9OL2FkZC1icmljay9wb3N0LyRESVNBQkxFRF9OQU1FIgpFTkFCTEVEX1NUQVRFX1NUQVJUPSIkR0xVU1RFUkRfV09SS0RJUi9ob29rcy8kVkVSU0lPTi9zdGFydC9wb3N0LyRFTkFCTEVEX05BTUUiCkVOQUJMRURfU1RBVEVfQUREX0JSSUNLPSIkR0xVU1RFUkRfV09SS0RJUi9ob29rcy8kVkVSU0lPTi9hZGQtYnJpY2svcG9zdC8kRU5BQkxFRF9OQU1FIjsKCiMjIFdoeSB0byBwcm9jZWVkIGlmIHRoZSByZXF1aXJlZCBzY3JpcHQgaXRzZWxmIGlzIG5vdCBwcmVzZW50PwpscyAkRElTQUJMRURfU1RBVEU7CmlmIFsgMCAtbmUgJD8gXQp0aGVuCiAgICAgICAgZXhpdCAkPzsKZmkKCiMjIElzIHF1b3RhIGVuYWJsZWQ/CkZMQUc9YGNhdCAkR0xVU1RFUkRfV09SS0RJUi92b2xzLyRWT0xfTkFNRS9pbmZvIHwgZ3JlcCAiXmZlYXR1cmVzLnF1b3RhPSIgXAogICAgICB8IGF3ayAtRic9JyAne3ByaW50ICRORn0nYDsKaWYgWyAiJEZMQUciICE9ICJvbiIgXQp0aGVuCiAgICAgICAgZXhpdCAkRVhJVF9TVUNDRVNTOwpmaQoKIyMgSXMgdm9sdW1lIHN0YXJ0ZWQ/CkZMQUc9YGNhdCAkR0xVU1RFUkRfV09SS0RJUi92b2xzLyRWT0xfTkFNRS9pbmZvIHwgZ3JlcCAiXnN0YXR1cz0iIFwKICAgICAgfCBhd2sgLUYnPScgJ3twcmludCAkTkZ9J2A7CmlmIFsgIiRGTEFHIiAhPSAiMSIgXQp0aGVuCiAgICAgICAgZW5hYmxlICRFTkFCTEVEX1NUQVRFX1NUQVJUOwogICAgICAgIGV4aXQgJD8KZmkKCmVuYWJsZSAkRU5BQkxFRF9TVEFURV9BRERfQlJJQ0s7CmV4aXQgJD8K') error: update() takes exactly 6 arguments (5 given)"}, code = -32603 To view the terms under which this email is distributed, please go to:- http://leedsbeckett.ac.uk/disclaimer/email/ To view the terms under which this email is distributed, please go to:- http://leedsbeckett.ac.uk/disclaimer/email/

1 0

user role
by kim.kargaard＠noroff.no 17 Sep '19

17 Sep '19

Hi, We are an Higher Education Institution and have set up oVirt 4.3 as a Virtual Lab Environment, for both on-campus and distance learning students. We have integrated AD and can log in with AD credentials for users. However, I am struggling to find the right user role for normal student users. Basically, they need to be able to do the following: 1. Create a new VM and select from either a template or select an uploaded ISO 2. Start the VM 3. Restart the VM 4. Shutdown the VM 5. Snapshot the VM 6. Connect to the VM using either spice or vnc/novnc 7. Create a disk 8. Connect to the student network (only would be preferable here) as we have management networks that we dont want the students be be able to see or select (if possible) I have tested using the PowerUserRole, but that does not allow me to select VNC/noVNC or spice and does not allow me to install from an ISO, only a template. Any suggestions on role that I can give these users? Thanks. Kind regards Kim

3 2

Re: Does cluster upgrade wait for heal before proceeding to next host?
by Strahil 17 Sep '19

17 Sep '19

As far as I know, The UI allows the administrator to override the gluster status, although I always use that protection. In my opinion gluster health (especially in my setup - replica2 arbiter1) is very important. Best Regards, Strahil NikolovOn Sep 17, 2019 00:47, Jayme <jaymef(a)gmail.com> wrote: > > Strahil, yes this is similar to the approach I have used to upgrade my hci cluster. This thread is in regards to the ui cluster upgrade procedure. It fails to update after the first host is rebooted because the gluster volume is still in a healing state and the attempt to set the second host to maintenance fails because quorum is not met. > > On Mon, Sep 16, 2019 at 1:41 PM Strahil <hunter86_bg(a)yahoo.com> wrote: >> >> I keep reading this chain and I still don't get what/who should wait for the cluster to heal... >> Is there some kind of built-in autopatching feature? >> >> Here is my approach: >> 1. Set global maintenance >> 2. Power off the engine >> 3. Create a gluster snapshot of the engine's volume >> 4. Power on engine manually >> 5. Check engine status >> 6. Upgrade engine >> 7. Upgrade engine's OS >> 8. Reboot engine and check health >> 9. Remove global maintenance >> 10. Set a host into local maintenance (evacuate all VMs) >> 11. Use UI to patch the host (enable autoreboot) >> 12. When host is up - login and check gluster volumes' heal status >> 13. Remove maintenance for the host and repeate for the rest of the cluster. >> >> I realize that for large clusters this approach is tedious and an automatic approach can be scripted. >> >> Best Regards, >> Strahil Nikolov >> >> On Sep 16, 2019 11:02, Kaustav Majumder <kmajumde(a)redhat.com> wrote: >>> >>> Hi Jayme, >>> It would be great if you could raise a bug regarding the same. >>> >>> On Wed, Sep 11, 2019 at 5:05 PM Jayme <jaymef(a)gmail.com> wrote: >>>> >>>> This sounds similar to the issue I hit with the cluster upgrade process in my environment. I have large 2tb ssds and most of my vms are several hundred Gbs in size. The heal process after host reboot can take 5-10 minutes to complete. I may be able to address this with better gluster tuning. >>>> >>>> Either way the upgrade process should be aware of the heal status and wait for it to complete before attempting to move on to the next host. >>>> >>>> >>>> On Wed, Sep 11, 2019 at 3:53 AM Sahina Bose <sabose(a)redhat.com> wrote: >>>>> >>>>> >>>>> >>>>> On Fri, Aug 9, 2019 at 3:41 PM Martin Perina <mperina(a)redhat.com> wrote: >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Aug 8, 2019 at 10:25 AM Sandro Bonazzola <sbonazzo(a)redhat.com> wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> Il giorno mar 6 ago 2019 alle ore 23:17 Jayme <jaymef(a)gmail.com> ha scritto: >>>>>>>> >>>>>>>> I’m aware of the heal process but it’s unclear to me if the update continues to run while the volumes are healing and resumes when they are done. There doesn’t seem to be any indication in the ui (unless I’m mistaken) >>>>>>> >>>>>>> >>>>>>> Adding @Martin Perina , @Sahina Bose and

1 0

Re: Does cluster upgrade wait for heal before proceeding to next host?
by Strahil 16 Sep '19

16 Sep '19

I keep reading this chain and I still don't get what/who should wait for the cluster to heal... Is there some kind of built-in autopatching feature? Here is my approach: 1. Set global maintenance 2. Power off the engine 3. Create a gluster snapshot of the engine's volume 4. Power on engine manually 5. Check engine status 6. Upgrade engine 7. Upgrade engine's OS 8. Reboot engine and check health 9. Remove global maintenance 10. Set a host into local maintenance (evacuate all VMs) 11. Use UI to patch the host (enable autoreboot) 12. When host is up - login and check gluster volumes' heal status 13. Remove maintenance for the host and repeate for the rest of the cluster. I realize that for large clusters this approach is tedious and an automatic approach can be scripted. Best Regards, Strahil NikolovOn Sep 16, 2019 11:02, Kaustav Majumder <kmajumde(a)redhat.com> wrote: > > Hi Jayme, > It would be great if you could raise a bug regarding the same. > > On Wed, Sep 11, 2019 at 5:05 PM Jayme <jaymef(a)gmail.com> wrote: >> >> This sounds similar to the issue I hit with the cluster upgrade process in my environment. I have large 2tb ssds and most of my vms are several hundred Gbs in size. The heal process after host reboot can take 5-10 minutes to complete. I may be able to address this with better gluster tuning. >> >> Either way the upgrade process should be aware of the heal status and wait for it to complete before attempting to move on to the next host. >> >> >> On Wed, Sep 11, 2019 at 3:53 AM Sahina Bose <sabose(a)redhat.com> wrote: >>> >>> >>> >>> On Fri, Aug 9, 2019 at 3:41 PM Martin Perina <mperina(a)redhat.com> wrote: >>>> >>>> >>>> >>>> On Thu, Aug 8, 2019 at 10:25 AM Sandro Bonazzola <sbonazzo(a)redhat.com> wrote: >>>>> >>>>> >>>>> >>>>> Il giorno mar 6 ago 2019 alle ore 23:17 Jayme <jaymef(a)gmail.com> ha scritto: >>>>>> >>>>>> I’m aware of the heal process but it’s unclear to me if the update continues to run while the volumes are healing and resumes when they are done. There doesn’t seem to be any indication in the ui (unless I’m mistaken) >>>>> >>>>> >>>>> Adding @Martin Perina , @Sahina Bose and @Laura Wright on this, hyperconverged deployments using cluster upgrade command would probably need some improvement. >>>> >>>> >>>> The cluster upgrade process continues to the 2nd host after the 1st host becomes Up. If 2nd host then fails to switch to maintenance, we stop the upgrade process to prevent breakage. >>>> Sahina, is gluster healing process status exposed in RESTAPI? If so, does it makes sense to wait for healing to be finished before trying to move next host to maintenance? Or any other ideas how to improve? >>> >>> >>> I need to cross-check this, if we expose the heal count in the gluster bricks. Moving a host to maintenance does check if there are pending heal entries or possibility of quorum loss. And this would prevent the additional hosts to upgrade. >>> +Gobinda Das +Sachidananda URS >>> >>>>> >>>>> >>>>>> >>>>>> >>>>>> On Tue, Aug 6, 2019 at 6:06 PM Robert O'Kane <okane(a)khm.de> wrote: >>>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> Often(?), updates to a hypervisor that also has (provides) a Gluster >>>>>>> brick takes the hypervisor offline (updates often require a reboot). >>>>>>> >>>>>>> This reboot then makes the brick "out of sync" and it has to be resync'd. >>>>>>> >>>>>>> I find it a "feature" than another host that is also part of a gluster >>>>>>> domain can not be updated (rebooted) before all the bricks are updated >>>>>>> in order to guarantee there is not data loss. It is called Quorum, or? >>>>>>> >>>>>>> Always let the heal process end. Then the next update can start. >>>>>>> For me there is ALWAYS a healing time before Gluster is happy again. >>>>>>> >>>>>>> Cheers, >>>>>>> >>>>>>> Robert O'Kane >>>>>>> >>>>>>> >>>>>>> Am 06.08.2019 um 16:38 schrieb Shani Leviim: >>>>>>> > Hi Jayme, >>>>>>> > I can't recall such a healing time. >>>>>>> > Can you please retry and attach the engine & vdsm logs so we'll be smarter? >>>>>>> > >>>>>>> > *Regards, >>>>>>> > * >>>>>>> > *Shani Leviim >>>>>>> > * >>>>>>> > >>>>>>> > >>>>>>> > On Tue, Aug 6, 2019 at 5:24 PM Jayme <jaymef(a)gmail.com >>>>>>> > <mailto:jaymef@gmail.com>> wrote: >>>>>>> > >>>>>>> > I've yet to have cluster upgrade finish updating my three host HCI >>>>>>> > cluster. The most recent try was today moving from oVirt 4.3.3 to >>>>>>> > 4.3.5.5. The first host updates normally, but when it moves on to >>>>>>> > the second host it fails to put it in maintenance and the cluster >>>>>>> > upgrade stops. >>>>>>> > >>>>>>> > I suspect this is due to that fact that after my hosts are updated >>>>>>> > it takes 10 minutes or more for all volumes to sync/heal. I have >>>>>>> > 2Tb SSDs. >>>>>>> > >>>>>>> > Does the cluster upgrade process take heal time in to account before >>>>>>> > attempting to place the next host in maintenance to upgrade it? Or >>>>>>> > is there something else that may be at fault here, or perhaps a >>>>>>> > reason why the heal process takes 10 minutes after reboot to complete? >>>>>>> > _______________________________________________ >>>>>>> > Users mailing list -- users(a)ovirt.org <mailto:users@ovirt.org> >>>>>>> > To unsubscribe send an email to users-leave(a)ovirt.org >>>>>>> > <mailto:users-leave@ovirt.org> >>>>>>> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>> > oVirt Code of Conduct: >>>>>>> > https://www.ovirt.org/community/about/community-guidelines/ >>>>>>> > List Archives: >>>>>>> > https://lists.ovirt.org/archives/list/users@ovirt.org/message/5XM3QB3364ZYI… >>>>>>> > >>>>>>> > >>>>>>> > _______________________________________________ >>>>>>> > Users mailing list -- users(a)ovirt.org >>>>>>> > To unsubscribe send an email to users-leave(a)ovirt.org >>>>>>> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>> > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>>>>>> > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/GBX3L23MWGMTF… >>>>>>> > >>>>>>> >>>>>>> -- >>>>>>> Systems Administrator >>>>>>> Kunsthochschule für Medien Köln >>>>>>> Peter-Welter-Platz 2 >>>>>>> 50676 Köln >>>>>>> _______________________________________________ >>>>>>> Users mailing list -- users(a)ovirt.org >>>>>>> To unsubscribe send an email to users-leave(a)ovirt.org >>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>>>>>> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/OBAHFFFTDOI7L… >>>>>> >>>>>> _______________________________________________ >>>>>> Users mailing list -- users(a)ovirt.org >>>>>> To unsubscribe send an email to users-leave(a)ovirt.org >>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>>>>> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/T27ROHWZPJL47… >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Sandro Bonazzola >>>>> >>>>> MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV >>>>> >>>>> Red Hat EMEA >>>>> >>>>> sbonazzo(a)redhat.com >>>>> >>>>> Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. >>>> >>>> >>>> >>>> -- >>>> Martin Perina >>>> Manager, Software Engineering >>>> Red Hat Czech s.r.o. >> >> _______________________________________________ >> Users mailing list -- users(a)ovirt.org >> To unsubscribe send an email to users-leave(a)ovirt.org >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/I4KLDBPYBCQEM… > > > > -- > > Thanks, > > Kaustav Majumder

2 1

Cannot Activate iSCSI Storage Domain After Errors
by Clint Boggio 16 Sep '19

16 Sep '19

Good Day To all; oVirt 4.3.4.3-1.el7 6 Dell R710 oVirt Nodes 10 gig Infiniband iSCSI Standalone Engine After an error during scheduled automated VM backkup, the iSCSi datastore that the VMs beinng backed up lived on suddenly went inactive and refuse to activate. The backup job did not finish as the source datastore went inactive but the task did clear from the console after i restarted the ovirt-engine service. I have tried: 1. Restart the ovirt-engine service 2. Restarted the engine physical server. 3. Tried to migrate the VM disks to a different datastore (failure) The virtual machines that are currently living on the inactive datastore are still running and accessible. Trying to reboot one of those VM as a test fails as the VM will not come up due to the system believing that the datastore is inactive. From engin.log: 2019-09-15 17:09:08,787-05 INFO [org.ovirt.engine.core.bll.storage.disk.RemoveDiskCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-8) [fbdd9b7] Removing 'Wildix4-PBX_snapshot_metadata' disk 2019-09-15 17:09:13,399-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-commandCoordinator-Thread-8) [fbdd9b7] EVENT_ID: IRS_BROKER_COMMAND_FAILURE(10,803), VDSM command DeleteImageGroupVDS failed: General Storage Exception: ('5 [\' Logical volume d1c5f4da-9024-4599-9135-4dabafe86cf8/c5f64501-142b-4a38-9b9c-faf283a64b06 changed.\'] [\' WARNING: This metadata update is NOT backed up.\', \' WARNING: Combining activation change with other commands is not advised.\', \' /dev/mapper/36001405f448f60ecea7479f9d920ac51: Checksum error at offset 19198423553536\', " Couldn\'t read volume group metadata from /dev/mapper/36001405f448f60ecea7479f9d920ac51.", \' Metadata location on /dev/mapper/36001405f448f60ecea7479f9d920ac51 at 19198423553536 has invalid summary for VG.\', \' Failed to read metadata summary from /dev/mapper/36001405f448f60ecea7479f9d920ac51\', \' Failed to scan VG from /dev/mapper/36001405f448f60ecea7479f9d920ac51\', \' Volume group "d1c5f4da-9024-4599-9135-4dabafe86cf8" not found\', \' Cannot process volume group d1c5f4da-9024-4599-9135-4dabafe86cf8\']\nd1c5f4da-9024-4599-9135-4dabafe86cf8/{\'c5f64501-142b-4a38-9b9c-faf283a64b06\': ImgsPar(imgs=[\'aadbfed3-af54-4400-af79-65d09b74c687\'], parent=\'00000000-0000-0000-0000-000000000000\')}',) 2019-09-15 17:09:13,399-05 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-8) [fbdd9b7] IrsBroker::Failed::DeleteImageGroupVDS: IRSGenericException: IRSErrorException: Failed to DeleteImageGroupVDS, error = General Storage Exception: ('5 [\' Logical volume d1c5f4da-9024-4599-9135-4dabafe86cf8/c5f64501-142b-4a38-9b9c-faf283a64b06 changed.\'] [\' WARNING: This metadata update is NOT backed up.\', \' WARNING: Combining activation change with other commands is not advised.\', \' /dev/mapper/36001405f448f60ecea7479f9d920ac51: Checksum error at offset 19198423553536\', " Couldn\'t read volume group metadata from /dev/mapper/36001405f448f60ecea7479f9d920ac51.", \' Metadata location on /dev/mapper/36001405f448f60ecea7479f9d920ac51 at 19198423553536 has invalid summary for VG.\', \' Failed to read metadata summary from /dev/mapper/36001405f448f60ecea7479f9d920ac51\', \' Failed to scan VG from /dev/mapper/36001405f448f60ecea7479f9d920ac51\', \' Volume group "d1c5f4da-9024-4599-9135-4dabafe86cf8" not found\', \' Cannot process volume group d1c5f4da-9024-4599-9135-4dabafe86cf8\']\nd1c5f4da-9024-4599-9135-4dabafe86cf8/{\'c5f64501-142b-4a38-9b9c-faf283a64b06\': ImgsPar(imgs=[\'aadbfed3-af54-4400-af79-65d09b74c687\'], parent=\'00000000-0000-0000-0000-000000000000\')}',), code = 200 2019-09-15 17:09:13,463-05 ERROR [org.ovirt.engine.core.bll.storage.disk.image.RemoveImageCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-8) [fbdd9b7] Command 'org.ovirt.engine.core.bll.storage.disk.image.RemoveImageCommand' failed: EngineException: org.ovirt.engine.core.vdsbroker.irsbroker.IRSErrorException: IRSGenericException: IRSErrorException: Failed to DeleteImageGroupVDS, error = General Storage Exception: ('5 [\' Logical volume d1c5f4da-9024-4599-9135-4dabafe86cf8/c5f64501-142b-4a38-9b9c-faf283a64b06 changed.\'] [\' WARNING: This metadata update is NOT backed up.\', \' WARNING: Combining activation change with other commands is not advised.\', \' /dev/mapper/36001405f448f60ecea7479f9d920ac51: Checksum error at offset 19198423553536\', " Couldn\'t read volume group metadata from /dev/mapper/36001405f448f60ecea7479f9d920ac51.", \' Metadata location on /dev/mapper/36001405f448f60ecea7479f9d920ac51 at 19198423553536 has invalid summary for VG.\', \' Failed to read metadata summary from /dev/mapper/36001405f448f60ecea7479f9d920ac51\', \' Failed to scan VG from /dev/mapper/36001405f448f60ecea7479f9d920ac51\', \' Volume group "d1c5f4da-9024-4599-9135-4dabafe86cf8" not found\', \' Cannot process volume group d1c5f4da-9024-4599-9135-4dabafe86cf8\']\nd1c5f4da-9024-4599-9135-4dabafe86cf8/{\'c5f64501-142b-4a38-9b9c-faf283a64b06\': ImgsPar(imgs=[\'aadbfed3-af54-4400-af79-65d09b74c687\'], parent=\'00000000-0000-0000-0000-000000000000\')}',), code = 200 (Failed with error StorageException and code 200) <metadata xmlns:ns0="http://ovirt.org/vm/tune/1.0" xmlns:ovirt-vm="http://ovirt.org/vm/1.0"> Any help or input would be greaatly appreciated.

1 0

Re: Virt-Viewer issue
by Staniforth, Paul 16 Sep '19

16 Sep '19

Hello, you should use remote-viewer, it's part of the virt-viewer package. Paul Staniforth School of Built Environment Engineering and Computing. Leeds Beckett University Networked Systems Analyst, Research and Engineering Support. tel: +44 (0)113 28123754 email: p.staniforth(a)leedsbeckett.ac.uk ________________________________ From: Lozada, Agustin T <Agustin.Lozada(a)centerpointenergy.com> Sent: 13 September 2019 15:38 To: users(a)ovirt.org Subject: [ovirt-users] Virt-Viewer issue Im building my first VM in ovirt with gluster and Im getting this issue when I'm opening a console session using virt-viewer. Does anyone has encountered this issue? [cid:image001.png@01D56A15.FECFDAF0] To view the terms under which this email is distributed, please go to:- http://leedsbeckett.ac.uk/disclaimer/email/

1 0

Does cluster upgrade wait for heal before proceeding to next host?
by Jayme 16 Sep '19

16 Sep '19

I've yet to have cluster upgrade finish updating my three host HCI cluster. The most recent try was today moving from oVirt 4.3.3 to 4.3.5.5. The first host updates normally, but when it moves on to the second host it fails to put it in maintenance and the cluster upgrade stops. I suspect this is due to that fact that after my hosts are updated it takes 10 minutes or more for all volumes to sync/heal. I have 2Tb SSDs. Does the cluster upgrade process take heal time in to account before attempting to place the next host in maintenance to upgrade it? Or is there something else that may be at fault here, or perhaps a reason why the heal process takes 10 minutes after reboot to complete?

8 9

CentOS 6 in oVirt
by Dominik Holler 16 Sep '19

16 Sep '19

Hello, are the official CentOS 6 cloud images known to work in oVirt? I tried, but they seem not to work in oVirt [1], but on plain libvirt. Are there any pitfalls known during using them in oVirt? Thanks Dominik [1] https://gerrit.ovirt.org/#/c/101117/

2 2

VMs stuck at Migrating status in Dashboard- Trying to cancel migration will yield "cannot cancel migration for non-migrating vm"
by paul.christian.suba＠cevalogistics.com 16 Sep '19

16 Sep '19

Saw this problem initially on one host. Host was stuck in entering maintenance mode since it could not migrate a VM from itself. Manually shutdown that VM and also restarted the host. After restart, some VMs are stuck in a migrating status while trying to "migrate" to the rebooted host. oVirt Node 4.3.1

2 2

How do I make delta clones ? OVM (aka Xen) vs Ovirt
by Tim Tuck 15 Sep '19

15 Sep '19

Hi all, I'm new to Ovirt coming from a OVM ( aka Xen ) environment and I'm trying to work out how to do similar things. In my OVM environment I have VMs that are 100GB in size. These VMs are templates that are used for running customer workshops so we create and destroy them quite often. In OVM I can clone 10 x 100GB VM's in about 1 minute, the cloning process just creates a "delta" VM rather than copying the entire 100GB. Those VM's can then be run and the on-disk size of the delta VM grows as it runs and deviates from the master. The same goes for templates, creating a 100GB VM from a template takes only a few seconds. Creating a template from a VM only takes a few seconds. My problem is that I cant seem to do any of this using Ovirt. it appears that there is no way to tell the cloning process " I want 10" like I can in OVM - am I missing something here? In my tests I find that even if a "snapshot" is made, the entire VM is copied rather than creating a delta. Creating a VM from a template is no different in that it copies the entire VM. So... Is it possible for Ovirt to create multiple clones on request ? Can Ovirt do "delta" copies rather than copy the entire VM ? If Ovirt can do these things, how do I set it up to to do that ? Any help appreciated. thanks Tim --- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

2 1

Virt-Viewer issue
by Lozada, Agustin T 13 Sep '19

13 Sep '19

Im building my first VM in ovirt with gluster and Im getting this issue when I'm opening a console session using virt-viewer. Does anyone has encountered this issue? [cid:image001.png@01D56A15.FECFDAF0]

1 0

Re: VMs stuck at Migrating status in Dashboard- Trying to cancel migration will yield "cannot cancel migration for non-migrating vm"
by Strahil 13 Sep '19

13 Sep '19

Have you marked your host as 'recently rebooted' (or whatever was the name in the UI)? Best Regards, Strahil NikolovOn Sep 12, 2019 18:15, paul.christian.suba(a)cevalogistics.com wrote: > > Saw this problem initially on one host. Host was stuck in entering maintenance mode since it could not migrate a VM from itself. > Manually shutdown that VM and also restarted the host. > After restart, some VMs are stuck in a migrating status while trying to "migrate" to the rebooted host. > > oVirt Node 4.3.1 > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/QFMNWH63FQIVZ…

1 0

oVirt conference 2019 - Rome - October 4th
by Sandro Bonazzola 12 Sep '19

12 Sep '19

The oVirt Italian community <https://www.facebook.com/groups/ovirt.italia/> and Extraordy <https://www.extraordy.com/> in collaboration with Red Hat <https://www.redhat.com>, Oracle <https://www.oracle.com>, and Storware <https://storware.eu/> are pleased to invite you at oVirt Conference 2019 in Rome on October 4th 2019! Tickets are available at https://www.eventbrite.it/e/biglietti-ovirt-conference-2019-innovate-the-da… Thanks -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/>*Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

2 1

Re: VMs inaccessible from Ubuntu/Debian-based OS
by Strahil 12 Sep '19

12 Sep '19

Hi, I have just tested ovirt VNC console (.vv file) with remote-viewer and it works. Ensure that you have acees to hosts' ports 5900-5999 , as this is the VNC port. Otherwise you can configure noVNC, but it requires port 6100 (based on memory). Best Regards, Strahil NikolovOn Aug 30, 2019 20:22, Souvalioti Maria <souvaliotimaria(a)mail.com> wrote: > > Hello Strahil and thank you for your reply. > > Virt-viewer is installed but mozilla is trying to open the console.vv by remote-viewer. In my understanding virt-viewer and remote-viewer are somewhat the same? (Remote-viewer usin virt-viewer) Should I access the VM's console from the ternimal? I have tried changing the VM's graphics type from spice to spice+vnc but still nothing. All I get is a blank window and a message stating that it cannot access the graphic server. > > Thanks again for your help. > > Maria Souvalioti > > > Sent from BlueMail > On 30 Aug 2019, at 7:48 pm, Strahil <hunter86_bg(a)yahoo.com> wrote: >> >> You need virt-viewer installed on your ubuntu or change console type to spice. >> >> >> Best Regards, >> Strahil NikolovOn Aug 30, 2019 15:35, souvaliotimaria(a)mail.com wrote: >>> >>> >>> Hello all, >>> >>> I'm having an issue the past couple of days. I have tried anything I could find to solve this but with no success. >>> >>> I have a three node ovirt installation, glustered, hyperconverged and there i have a few VMs. The installation is for experimental reasons before we merge it in our DC. Anyway, though I can connect to the VMs' console from Fedora and Windows, I can't from Ubuntu. >>> >>> I have tried installing the browser-spice-plugin that's the corresponding package to spice-xpi and I got no results. >>> >>> I purged the browser-spice-plugin and then I installed the spice-client package, downloaded the spice-xpi from Fedora (FC19) (as instructed in https://www.ovirt.org/develop/infra/testing/spice.html) copied the libnsISpicec.so to /usr/lib/mozilla/plugins and made sure that xserver-xorg-video-qxl and spice-vdagent were installed and in their latest version, and still nothing. >>> >>> No matter what I have tried, I can't gain access to the console. No matter the browser I use, the message I get is "Unable to connect to the graphic server /tmp/mozilla_msouval0/console-1.vv". >>> >>> I have checked the logs and couldn't find anything useful, maybe I'm not checking the right logs? >>> I run tcpdump on both the node the VM is being hosted and the Ubuntu machine I'm using and though on the UBuntu I capture a few packets (6) on both sides but on the Ubuntu side there were 15 packets received by the filter. >>> >>> Could you please guide towards the right way to solve this? Could this be an ntp problem? >>> >>> ovirt node version: 4.2.3.1 >>> Workstation: Jessie >>> >>> (I thought maybe Jessie was too old, and I run the same steps from Ubuntu 14.04 and Ubuntu 16.04, but the problem still remains) >>> >>> Thank you in advance for any help. >>> Maria >>> ________________________________ >>> >>> Users mailing list -- users(a)ovirt.org >>> To unsubscribe send an email to users-leave(a)ovirt.org >>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/4VMMYEJNSHGUH…

3 2

Health Warning: Cinnamon toxic when adding EL7 nodes as oVirt hosts
by thomas＠hoberg.net 12 Sep '19

12 Sep '19

Of course a desktop on a virtualization host isn't really recommended, but actually, managing highly or even high-available supports services as VMs on a set of fat workstations under the control of oVirt can be rather useful... And I prefer Cinnamon so much over the other desktops, it gets installed almost first after setup... Which turns out to be a mistake with oVirt. Anyhow, to make a long journey of hair-pulling root-cause searching short: Installing Cinnamon from EPEL has Otopi not find rpmUtils for one reason or another, even if I can see it on both sides, the engine and the target host. Remove Cinnamon, it works again and you can re-install Cinnamon after. I didn't test re-deploys, but I'd expect them to fail. Don't know if this should be a bug report, since Cinnamon very unfortunately doesn't seem official just yet. This is what you may expect to see in the engine logs: 2019-08-22 09:49:24,152+0200 DEBUG otopi.context context._executeMethod:145 method exception Traceback (most recent call last): File "/tmp/ovirt-BAJG2FYLMu/pythonlib/otopi/context.py", line 132, in _executeMethod method['method']() File "/tmp/ovirt-BAJG2FYLMu/otopi-plugins/ovirt-host-deploy/kdump/packages.py", line 216, in _customization self._kexec_tools_version_supported() File "/tmp/ovirt-BAJG2FYLMu/otopi-plugins/ovirt-host-deploy/kdump/packages.py", line 143, in _kexec_tools_version_supported from rpmUtils.miscutils import compareEVR ModuleNotFoundError: No module named 'rpmUtils' 2019-08-22 09:49:24,152+0200 ERROR otopi.context context._executeMethod:154 Failed to execute stage 'Environment customization': No module named 'rpmUtils'

2 10

email
by Mustafa Taha 12 Sep '19

12 Sep '19

mustafa.taha(a)exalt.ps

1 0

Deploying the Self-Hosted Engine Loop
by mustafa.taha.mu95＠gmail.com 11 Sep '19

11 Sep '19

Hi i am trying to deploy self-hot with latest ovirt 4.3 on Centos 7.6.1810 but deploying -first stage of deploy- stick on following logs: [ INFO ] TASK [ovirt.hosted_engine_setup : Set new IPv4 subnet prefix] [ INFO ] skipping: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Search again with another prefix] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Define 3rd chunk] [ INFO ] skipping: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Set 3rd chunk] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Get ip route] [ INFO ] changed: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fail if can't find an available subnet] [ INFO ] skipping: [localhost] can anybody help ?

1 0

Locked Snapshot
by Peter Smith 11 Sep '19

11 Sep '19

Hi, I have been trying to get a backup solution working for my VM's, and have somehow managed to break the snapshot on one of my VM's. I'm still fairly new to oVirt, we're migrating from VMware. I have had this cluster running since May. The cluster: We are running oVirt on CentOS 7, with Hosted Engine. oVirt 4.3.5.4-1.el7 (according to the Help->About in the Admin Console) Hardware is 5 x Dell M620 Blades (300Gb internal storage for host OS) with ISCSI storage for the VM's (Dell M3660i), 10GB Network throughout. The VM in question is allocated 50Gb of storage and there is 461GB free in the storage domain. So I've been playing with bareos 18.2.5 with the ovirt plugin found here https://github.com/eurotux/bareos-contrib/tree/master/fd-plugins/ovirt-plug… I got bareos hooked up to ovirt. It created a snapshot, downloaded it, then at the end it failed to remove the snapshot. I though it may have been because I was looking round bareos while it was doing the backup. Went into oVirt deleted the snapshot no issues. Re ran the backup job in bareos, leaving bareos alone this time. The backup job failed again being unable to remove the snapshot. So I figured I'd go in and delete it, but I get an error saying "Cannot remove Snapshot: The following disks are locked: VM-NAME_vmdisk1. Please try again in a few minutes.". In the Admin console if I expand the disks on the snapshot it shows that the image is locked. And before you ask the bareos user is setup as a SuperUser. If I go to the Events from the menu on the left of the admin console I can see that (Sorry Names/IP's changed to protect the guilty): Sep 10, 2019, 10:42:39 AM Image Download with disk VM-NAME_vmdisk1 succeeded. Sep 10, 2019, 10:42:38 AM VDSM hyper2 command TeardownImageVDS failed: Cannot deactivate Logical Volume: ('General Storage Exception: ("5 [] [\' Logical volume 46a9dfd4-4871-46b2-a740-8fe430c70b4b/bbe315b9-e797-4cf8-b18b-6afeb33589cc in use.\', \' Logical volume 46a9dfd4-4871-46b2-a740-8fe430c70b4b/d3dfa2c2-45fa-4081-98c5-890dd7a5990a in use.\']\\n46a9dfd4-4871-46b2-a740-8fe430c70b4b/[\'bbe315b9-e797-4cf8-b18b-6afeb33589cc\', \'d3dfa2c2-45fa-4081-98c5-890dd7a5990a\']",)',) Sep 10, 2019, 10:22:09 AM Image Download with disk VM-NAME_vmdisk1 was initiated by bareos@internal-authz. Sep 10, 2019, 10:21:55 AM Snapshot 'VM-NAME-backup-1f952494-8245-479e-8d1d-9503d6fe8f9f' creation for VM 'VM-NAME' has been completed. Sep 10, 2019, 10:21:36 AM Snapshot 'VM-NAME-backup-1f952494-8245-479e-8d1d-9503d6fe8f9f' creation for VM 'VM-NAME' was initiated by bareos@internal-authz. Sep 10, 2019, 10:21:36 AM Backup of virtual machine 'VM-NAME using snapshot 'VM-NAME-backup-1f952494-8245-479e-8d1d-9503d6fe8f9f' is starting. Sep 10, 2019, 10:21:36 AM User bareos@internal-authz connecting from '10.*******' using session '****************' logged in. To me it looks like bareos tried to delete the snapshot whilst it was still downloading it, which failed as expected. Does anyone know how to fix/recover from this? If you need log files, please could you tell me which ones? Regards, Peter Smith

3 3

How to change Master Data Storage Domain
by Mark Steele 11 Sep '19

11 Sep '19

Good morning, I have a Storage Domain that I would like to retire and move that role to a new storage domain. Both Domains exist on my current Data Center and I have moved all disks from the existing Data (Master) domain to the new Data Domain. The only thing that is still associated with the Data (Master) domain are two OVF_STORE items: Alias Virtual Size Actual Size Allocation Policy Storage Domain Storage Type Creation Date Attached To Alignment Status Description OVF_STORE < 1 GB < 1 GB Preallocated phl-datastore NFS 2014-Nov-14, 20:00 Unknown OK OVF_STORE OVF_STORE < 1 GB < 1 GB Preallocated phl-datastore NFS 2014-Nov-14, 20:00 Unknown OK OVF_STORE What is the procedure for moving / removing these items and 'promoting' the other Data Domain to Master? Our current version is: oVirt Engine Version: 3.5.0.1-1.el6 (it's old but reliable) Best regards, *** *Mark Steele* CIO / VP Technical Operations | TelVue Corporation TelVue - We Share Your Vision 16000 Horizon Way, Suite 100 | Mt. Laurel, NJ 08054 800.885.8886 x128 | msteele(a)telvue.com | http://www.telvue.com twitter: http://twitter.com/telvue | facebook: https://www.facebook.com/telvue

3 4

ovirt with glusterfs data domain - very slow writing speed on Windows server virtual machine
by lyubomir.grancharov＠bottleshipvfx.com 11 Sep '19

11 Sep '19

Hi folks, I have been experimenting with oVirt cluster based on glusterfs for the past few days. (first-timer). The cluster is up and running and it consists of 4 nodes and has 4 replicas. When I try to deploy Windows Server VM I encounter the following issue: The disk of the VM has ok reading speed ( close to bare metal) but the writing speed is very slow. ( about 10 times slower than it is supposed to be). Can anyone give me any suggestion, please? Thanks in advance! Here are the settings of the glusterfs volume: Volume Name: bottle-volume Type: Replicate Volume ID: 869b8d1e-1266-4820-8dcd-4fea92346b90 Status: Started Snapshot Count: 0 Number of Bricks: 1 x 4 = 4 Transport-type: tcp Bricks: Brick1: cnode01.bottleship.local:/gluster_bricks/brick-cnode01/brick-cnode01 Brick2: cnode02.bottleship.local:/gluster_bricks/brick-cnode02/brick-cnode02 Brick3: cnode03.bottleship.local:/gluster_bricks/brick-cnode03/brick-cnode03 Brick4: cnode04.bottleship.local:/gluster_bricks/brick-cnode04/brick-cnode04 Options Reconfigured: network.ping-timeout: 30 cluster.granular-entry-heal: enable performance.strict-o-direct: on storage.owner-gid: 36 storage.owner-uid: 36 server.event-threads: 4 client.event-threads: 4 cluster.choose-local: off features.shard: on cluster.shd-wait-qlength: 10000 cluster.shd-max-threads: 8 cluster.locking-scheme: granular cluster.data-self-heal-algorithm: full cluster.server-quorum-type: server cluster.quorum-type: auto cluster.eager-lock: enable network.remote-dio: off performance.low-prio-threads: 32 performance.io-cache: off performance.read-ahead: off performance.quick-read: off auth.allow: * user.cifs: off transport.address-family: inet nfs.disable: on performance.client-io-threads: off server.allow-insecure: on Please let me know if you need any more configuration info or hardware specs. best, Lyubo

2 1

Procedure to replace out out of three hyperconverged nodes
by thomas＠hoberg.net 11 Sep '19

11 Sep '19

On my silent Atom based three node Hyperconverged journey I hit upon a snag: Evidently they are too slow for Ansible. The Gluster storage part went all great and perfect on fresh oVirt node images that I had configured to leave an empty partition instead of the standard /dev/sdb, but the HostedEngine setup part would then fail without any log-visible error while the transient VM HostedEngineLocal was supposed to be launched and the Wizard would just show "deployment failed" and go ahead and delete the VM. I then moved the SSD to a more powerful Xeon-D 1541 CPU and after some fiddling with the network (I miss good old eth0!), this also failed the deployment, but also failed to delete the temporary VM image, because that actually turned out to be running: I could even connect to its console and investigate the logs for any clues as to what might have gone wrong (nothing visible): Evidently Ansible was running out of patience just a tiny bit too early. And then I kicked it into high-gear with an i7-7700K again using the same SSD with a working three node Gluster all in sync, which still took what felt like an hour to creep through every step, but got it done, primary node on i7, secondary nodes on Atoms, with full migration capabilities etc. I then had to do some fiddling, because the HostedEngine had configued the Cluster CPU architecture to Skylake-Spectre, but after that I migrated it to an Atom node and was now ready to move the primary to the intended Atom hardware target. But at that point the overlay network has already been configured and evidently it's tied to the device name of the 10Gbit NIC on the i7 workstation and I haven't been able to make it work with the Atom. The Gluster runs fine, but the CPU node is reported "non-operational" and re-installation fails, because the ovirtmgmt network isn't properly configured. That specific issue may be seem way out of what oVirt should support, yet a HA-embedded edge platform may very well see nodes having to be replaced or renewed with as little interruption or downtime as possible, which is why I am asking the larger question: How can you replace a) a failed "burned" node or b) upgrade nodes while maintaining fault tolerance? The distinction in b) would be that it's a planned maneuver during normal operations without downtime. I'd want to do it pretty much like I have been playing with compute nodes, creating new ones, pushing VMs on them, pushing them out to other hosts, removing and replacing them seamlessly... Except that the Gluster nodes are special and much harder to replace, than a pure Gluster storage brick... from what I see I welcome any help - for fixing the network config in my limiping Atom 1:3 cluster - eliminating the need to fiddle with an i7 because of Ansible timing - ensuring long-term operability of a software defined datacenter with changing hardware

2 2

gluster
by mailing-ovirt＠qic.ca 11 Sep '19

11 Sep '19

Hi, I see options to deploy ovirt with gluster during the initial rollout, however I can't seem to find information as to how I can add it following a non gluster initial setup: GlusterFS Version: [N/A] Thanks Simon

4 5

VM crushed and cant shutdown successfully,I shoud to reboot the ovirt-node machine
by zhouhao＠vip.friendtimes.net 11 Sep '19

11 Sep '19

VM crushed and cant shutdown successfully,It always on the way to shutdown，even if I wait it for 10 hours；Everytimes I shoud to reboot the ovirt-node machine This comes up 4 times, It appes both on the ovirt-noe 4.3 and 4.2; This bug appers when my VM is running at high load（seems memory and disk's w/r） My storage domain is local on host; When the bug appers,it could cost my node's io,looks like the picture below；But sometimes it will not The vm's threading on the ovirt-node, the IO ratio is 100% Everytime the vm's process will change to defunct I can not kill it,I had to shutdown the ovirt-node on the engine website ,the vm's status always in the way to shutdown ,even if I wait for it for hours; It either failed or is shutting down and the "power off" cant shutdown the vm too. the vm's error below The ovirt-node's error below ---------------------------------------------------------------- The other infomation about my ovirt-node node-version： node hardware zhouhao(a)vip.friendtimes.net

1 0

Python to retrive the VM-Config from the snapshot
by Sven Achtelik 10 Sep '19

10 Sep '19

Hi All, I'm trying to build a custom solution for Backups using the python. Everything works and I can get my disk copies. The last thing is getting the VM config and I can't find any hint on how to get this done with python. Is there a way to actually get this information out easily ? Somehting like * Grab the wanted vm configuration from the needed snapshot - it'll be under initialization/configuration/data URL = SERVER:PORT/api/vms/VM_ID/snapshots/ID Method = GET (with All-Content:true header) I also have to mention that I'm not a profession programmer and work with what I can find as documentation or code examples. http://ovirt.github.io/ovirt-engine-sdk/4.1/ https://github.com/oVirt/ovirt-engine-sdk/tree/master/sdk/examples If some experienced python programmer could give a hint where to look at that would be great. Thank you, Sven

2 1

Re: Python to retrive the VM-Config from the snapshot
by Strahil 10 Sep '19

10 Sep '19

If the system is running you can use virsh dumpxml. Best Regards, Strahil NikolovOn Sep 10, 2019 16:12, Sven Achtelik <Sven.Achtelik(a)eps.aero> wrote: > > Hi All, > > > > I’m trying to build a custom solution for Backups using the python. Everything works and I can get my disk copies. The last thing is getting the VM config and I can’t find any hint on how to get this done with python. Is there a way to actually get this information out easily ? Somehting like > > Grab the wanted vm configuration from the needed snapshot - it’ll be under initialization/configuration/data > > URL = SERVER:PORT/api/vms/VM_ID/snapshots/ID Method = GET (with All-Content:true header) > > I also have to mention that I’m not a profession programmer and work with what I can find as documentation or code examples. > > http://ovirt.github.io/ovirt-engine-sdk/4.1/ > > https://github.com/oVirt/ovirt-engine-sdk/tree/master/sdk/examples > > If some experienced python programmer could give a hint where to look at that would be great. > > Thank you, > > Sven

1 0

nfs
by mailing-ovirt＠qic.ca 10 Sep '19

10 Sep '19

I`m trying to mount a nfs share. if I manually mount it from ssh, I can access it without issues. However when I do it from the web config, it keeps failing: Not sure how to solve that. Thanks Simon 2019-09-09 09:08:47,601-0400 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:47,610-0400 INFO (jsonrpc/3) [vdsm.api] START repoStats(domains=[u'4f4828c2-8f78-4566-a69d-a37cd73446d4']) from=::1,42394, task_id=6c1eeef7-ff8c-4d0d-be85-ced03aae4bc2 (api:48) 2019-09-09 09:08:47,610-0400 INFO (jsonrpc/3) [vdsm.api] FINISH repoStats return={u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.00101971', 'lastCheck': '0.4', 'valid': True}} from=::1,42394, task_id=6c1eeef7-ff8c-4d0d-be85-ced03aae4bc2 (api:54) 2019-09-09 09:08:47,611-0400 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:47,839-0400 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:47,846-0400 INFO (jsonrpc/2) [api.host] START getCapabilities() from=::1,42394 (api:48) 2019-09-09 09:08:48,149-0400 INFO (jsonrpc/2) [root] managedvolume not supported: Managed Volume Not Supported. Missing package os-brick.: ('Cannot import os_brick',) (caps:152) 2019-09-09 09:08:49,212-0400 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.01 seconds (__init__:312) 2019-09-09 09:08:49,263-0400 INFO (jsonrpc/2) [root] /usr/libexec/vdsm/hooks/after_get_caps/50_openstacknet: rc=0 err= (hooks:114) 2019-09-09 09:08:49,497-0400 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:49,657-0400 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:49,665-0400 INFO (jsonrpc/7) [vdsm.api] START repoStats(domains=[u'4f4828c2-8f78-4566-a69d-a37cd73446d4']) from=::1,42394, task_id=14b596bf-bc1c-412b-b1fc-8d53f17d88f7 (api:48) 2019-09-09 09:08:49,666-0400 INFO (jsonrpc/7) [vdsm.api] FINISH repoStats return={u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.00101971', 'lastCheck': '2.4', 'valid': True}} from=::1,42394, task_id=14b596bf-bc1c-412b-b1fc-8d53f17d88f7 (api:54) 2019-09-09 09:08:49,667-0400 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:49,800-0400 INFO (jsonrpc/2) [root] /usr/libexec/vdsm/hooks/after_get_caps/openstacknet_utils.py: rc=0 err= (hooks:114) 2019-09-09 09:08:50,464-0400 INFO (jsonrpc/2) [root] /usr/libexec/vdsm/hooks/after_get_caps/ovirt_provider_ovn_hook: rc=0 err= (hooks:114) 2019-09-09 09:08:50,467-0400 INFO (jsonrpc/2) [api.host] FINISH getCapabilities return={'status': {'message': 'Done', 'code': 0}, 'info': {u'HBAInventory': {u'iSCSI': [{u'InitiatorName': u'iqn.1994-05.com.redhat:d8c85fc0ab85'}], u'FC': []}, u'packages2': {u'kernel': {u'release': u'957.27.2.el7.x86_64', u'version': u'3.10.0'}, u'spice-server': {u'release': u'6.el7_6.1', u'version': u'0.14.0'}, u'librbd1': {u'release': u'4.el7', u'version': u'10.2.5'}, u'vdsm': {u'release': u'1.el7', u'version': u'4.30.24'}, u'qemu-kvm': {u'release': u'18.el7_6.7.1', u'version': u'2.12.0'}, u'openvswitch': {u'release': u'4.el7', u'version': u'2.11.0'}, u'libvirt': {u'release': u'10.el7_6.12', u'version': u'4.5.0'}, u'ovirt-hosted-engine-ha': {u'release': u'1.el7', u'version': u'2.3.3'}, u'qemu-img': {u'release': u'18.el7_6.7.1', u'version': u'2.12.0'}, u'mom': {u'release': u'1.el7.centos', u'version': u'0.5.12'}, u'glusterfs-cli': {u'release': u'1.el7', u'version': u'6.5'}}, u'numaNodeDistance': {u'1': [20, 10], u'0': [10, 20]}, u'cpuModel': u'Intel(R) Xeon(R) CPU E5-2640 v2 @ 2.00GHz', u'nestedVirtualization': False, u'liveMerge': u'true', u'hooks': {u'before_vm_start': {u'50_hostedengine': {u'md5': u'95c810cdcfe4195302a59574a5148289'}, u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'after_network_setup': {u'30_ethtool_options': {u'md5': u'ce1fbad7aa0389e3b06231219140bf0d'}}, u'after_vm_destroy': {u'delete_vhostuserclient_hook': {u'md5': u'c2f279cc9483a3f842f6c29df13994c1'}, u'50_vhostmd': {u'md5': u'bdf4802c0521cf1bae08f2b90a9559cf'}}, u'after_vm_start': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'ea0a5a715da8c1badbcda28e8b8fa00e'}}, u'after_device_migrate_destination': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'6226fbc4d1602994828a3904fc1b875d'}}, u'before_device_migrate_destination': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}}, u'after_device_create': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'6226fbc4d1602994828a3904fc1b875d'}}, u'after_nic_hotplug': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'6226fbc4d1602994828a3904fc1b875d'}}, u'before_vm_migrate_destination': {u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'before_vm_dehibernate': {u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'before_nic_hotplug': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}, u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'ec8d5d7ba063a109f749cd63c7a2b58d'}, u'20_ovirt_provider_ovn_vhostuser_hook': {u'md5': u'a8af653b7386c138b2e6e9738bd6b62c'}, u'10_ovirt_provider_ovn_hook': {u'md5': u'73822988042847bab1ea832a6b9fa837'}}, u'after_get_caps': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'5c3a9ab6e06e039bdd220c0216e45809'}, u'ovirt_provider_ovn_hook': {u'md5': u'1a6ce22288fc2466fcc339da252ef624'}}, u'before_network_setup': {u'50_fcoe': {u'md5': u'28c352339c8beef1e1b05c67d106d062'}}, u'before_device_create': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}, u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'ec8d5d7ba063a109f749cd63c7a2b58d'}, u'20_ovirt_provider_ovn_vhostuser_hook': {u'md5': u'a8af653b7386c138b2e6e9738bd6b62c'}, u'10_ovirt_provider_ovn_hook': {u'md5': u'73822988042847bab1ea832a6b9fa837'}}}, u'supportsIPv6': True, u'realtimeKernel': False, u'vmTypes': [u'kvm'], u'liveSnapshot': u'true', u'cpuThreads': u'32', u'kdumpStatus': 0, u'deferred_preallocation': True, u'networks': {u'ovirtmgmt': {u'iface': u'ovirtmgmt', u'ipv6autoconf': True, u'addr': u'192.168.20.101', u'dhcpv6': False, u'ipv6addrs': [], u'switch': u'legacy', u'bridged': True, u'southbound': u'em1', u'dhcpv4': True, u'netmask': u'255.255.255.0', u'ipv4defaultroute': True, u'stp': u'off', u'ipv4addrs': [u'192.168.20.101/24'], u'mtu': u'1500', u'ipv6gateway': u'::', u'gateway': u'192.168.20.1', u'ports': [u'em1']}}, u'kernelArgs': u'BOOT_IMAGE=/vmlinuz-3.10.0-957.27.2.el7.x86_64 root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rhgb quiet LANG=en_US.UTF-8', u'domain_versions': [0, 2, 3, 4, 5], u'bridges': {u'ovirtmgmt': {u'ipv6autoconf': True, u'addr': u'192.168.20.101', u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': True, u'netmask': u'255.255.255.0', u'ipv4defaultroute': True, u'stp': u'off', u'ipv4addrs': [u'192.168.20.101/24'], u'ipv6gateway': u'::', u'gateway': u'192.168.20.1', u'opts': {u'multicast_last_member_count': u'2', u'vlan_protocol': u'0x8100', u'hash_elasticity': u'4', u'multicast_query_response_interval': u'1000', u'group_fwd_mask': u'0x0', u'multicast_snooping': u'1', u'multicast_startup_query_interval': u'3125', u'hello_timer': u'0', u'multicast_querier_interval': u'25500', u'max_age': u'2000', u'hash_max': u'512', u'stp_state': u'0', u'topology_change_detected': u'0', u'priority': u'32768', u'multicast_igmp_version': u'2', u'multicast_membership_interval': u'26000', u'root_path_cost': u'0', u'root_port': u'0', u'multicast_stats_enabled': u'0', u'multicast_startup_query_count': u'2', u'nf_call_iptables': u'0', u'vlan_stats_enabled': u'0', u'hello_time': u'200', u'topology_change': u'0', u'bridge_id': u'8000.f8bc1234c8a8', u'topology_change_timer': u'0', u'ageing_time': u'30000', u'nf_call_ip6tables': u'0', u'multicast_mld_version': u'1', u'gc_timer': u'6672', u'root_id': u'8000.f8bc1234c8a8', u'nf_call_arptables': u'0', u'group_addr': u'1:80:c2:0:0:0', u'multicast_last_member_interval': u'100', u'default_pvid': u'1', u'multicast_query_interval': u'12500', u'multicast_query_use_ifaddr': u'0', u'tcn_timer': u'0', u'multicast_router': u'1', u'vlan_filtering': u'0', u'multicast_querier': u'0', u'forward_delay': u'0'}, u'ports': [u'em1']}}, u'uuid': u'4c4c4544-0046-3310-8034-c8c04f4a3032', u'onlineCpus': u'0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,1,3,5,7,9,11,13,15,17,19,21,23, 25,27,29,31', u'nameservers': [u'192.168.168.1'], u'nics': {u'em4': {u'ipv6autoconf': True, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:ab', u'ipv6gateway': u'::', u'gateway': u''}, u'idrac': {u'ipv6autoconf': True, u'addr': u'', u'speed': 0, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:37:d8:b1', u'ipv6gateway': u'::', u'gateway': u''}, u'em1': {u'ipv6autoconf': False, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:a8', u'ipv6gateway': u'::', u'gateway': u''}, u'em3': {u'ipv6autoconf': True, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:aa', u'ipv6gateway': u'::', u'gateway': u''}, u'em2': {u'ipv6autoconf': True, u'addr': u'', u'speed': 100, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:a9', u'ipv6gateway': u'::', u'gateway': u''}}, u'software_revision': u'1', u'hostdevPassthrough': u'false', u'clusterLevels': [u'4.1', u'4.2', u'4.3'], u'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,cl flush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,c onstant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfm perf,eagerfpu,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16 ,xtpr,pdcm,pcid,dca,sse4_1,sse4_2,x2apic,popcnt,tsc_deadline_timer,aes,xsave ,avx,f16c,rdrand,lahf_lm,ssbd,ibrs,ibpb,stibp,tpr_shadow,vnmi,flexpriority,e pt,vpid,fsgsbase,smep,erms,xsaveopt,dtherm,ida,arat,pln,pts,md_clear,spec_ct rl,intel_stibp,flush_l1d,model_Opteron_G2,model_kvm32,model_SandyBridge-IBRS ,model_Opteron_G1,model_core2duo,model_pentium,model_Westmere-IBRS,model_qem u64,model_kvm64,model_coreduo,model_Nehalem,model_Nehalem-IBRS,model_IvyBrid ge,model_Westmere,model_SandyBridge,model_IvyBridge-IBRS,model_qemu32,model_ Penryn,model_pentium2,model_pentium3,model_Conroe,model_486', u'kernelFeatures': {u'IBRS': 0, u'RETP': 1, u'SSBD': 3, u'PTI': 1}, u'ISCSIInitiatorName': u'iqn.1994-05.com.redhat:d8c85fc0ab85', u'netConfigDirty': u'False', u'supported_block_sizes': [512], u'selinux': {u'mode': u'1'}, u'autoNumaBalancing': 1, u'reservedMem': u'321', u'bondings': {}, u'software_version': u'4.30.24', u'supportedENGINEs': [u'4.1', u'4.2', u'4.3'], u'vncEncrypted': False, u'backupEnabled': False, u'cpuSpeed': u'1199.951', u'numaNodes': {u'1': {u'totalMemory': u'32768', u'cpus': [1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31]}, u'0': {u'totalMemory': u'32704', u'cpus': [0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30]}}, u'cpuSockets': u'2', u'vlans': {}, u'version_name': u'Snow Man', 'lastClientIface': 'lo', u'cpuCores': u'16', u'hostedEngineDeployed': True, u'hugepages': [1048576, 2048], u'guestOverhead': u'65', u'additionalFeatures': [u'libgfapi_supported'], u'openstack_binding_host_ids': {u'OPENSTACK_OVN': u'baremetal1.ledinfo.com', u'OPEN_VSWITCH': u'baremetal1.ledinfo.com', u'OVIRT_PROVIDER_OVN': u'c83bc734-603c-4a99-a5b9-5833ef1d90c7'}, u'kvmEnabled': u'true', u'memSize': u'64195', u'emulatedMachines': [u'pc-i440fx-rhel7.1.0', u'pc-q35-rhel7.3.0', u'rhel6.3.0', u'pc-i440fx-rhel7.5.0', u'pc-i440fx-rhel7.0.0', u'rhel6.1.0', u'pc-q35-rhel7.6.0', u'pc-i440fx-rhel7.4.0', u'rhel6.6.0', u'pc-q35-rhel7.5.0', u'rhel6.2.0', u'pc', u'pc-i440fx-rhel7.3.0', u'q35', u'pc-i440fx-rhel7.2.0', u'rhel6.4.0', u'pc-q35-rhel7.4.0', u'pc-i440fx-rhel7.6.0', u'rhel6.0.0', u'rhel6.5.0'], u'rngSources': [u'hwrng', u'random'], u'operatingSystem': {u'release': u'6.1810.2.el7.centos', u'pretty_name': u'CentOS Linux 7 (Core)', u'version': u'7', u'name': u'RHEL'}}} from=::1,42394 (api:54) 2019-09-09 09:08:50,493-0400 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.getCapabilities succeeded in 2.65 seconds (__init__:312) 2019-09-09 09:08:51,518-0400 INFO (jsonrpc/5) [api.host] START getAllVmStats() from=::1,42344 (api:48) 2019-09-09 09:08:51,518-0400 INFO (jsonrpc/5) [api.host] FINISH getAllVmStats return={'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} from=::1,42344 (api:54) 2019-09-09 09:08:51,519-0400 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:51,551-0400 INFO (jsonrpc/3) [vdsm.api] START connectStorageServer(domType=1, spUUID=u'00000000-0000-0000-0000-000000000000', conList=[{u'tpgt': u'1', u'id': u'00000000-0000-0000-0000-000000000000', u'connection': u'192.168.20.125:/ovirt', u'iqn': u'', u'user': u'', u'ipv6_enabled': u'false', u'protocol_version': u'auto', u'password': '********', u'port': u''}], options=None) from=::ffff:192.168.20.116,41974, flow_id=1b259177-b83c-4066-a045-b3a9db670a1e, task_id=3d0e32c4-288b-4be4-98ef-90fb676b57f2 (api:48) 2019-09-09 09:08:51,556-0400 INFO (jsonrpc/3) [storage.StorageServer.MountConnection] Creating directory u'/rhev/data-center/mnt/192.168.20.125:_ovirt' (storageServer:168) 2019-09-09 09:08:51,556-0400 INFO (jsonrpc/3) [storage.fileUtils] Creating directory: /rhev/data-center/mnt/192.168.20.125:_ovirt mode: None (fileUtils:199) 2019-09-09 09:08:51,556-0400 INFO (jsonrpc/3) [storage.Mount] mounting 192.168.20.125:/ovirt at /rhev/data-center/mnt/192.168.20.125:_ovirt (mount:204) 2019-09-09 09:08:51,684-0400 INFO (jsonrpc/3) [IOProcessClient] (Global) Starting client (__init__:308) 2019-09-09 09:08:51,705-0400 INFO (ioprocess/9120) [IOProcess] (Global) Starting ioprocess (__init__:434) 2019-09-09 09:08:51,739-0400 INFO (jsonrpc/3) [vdsm.api] FINISH connectStorageServer return={'statuslist': [{'status': 0, 'id': u'00000000-0000-0000-0000-000000000000'}]} from=::ffff:192.168.20.116,41974, flow_id=1b259177-b83c-4066-a045-b3a9db670a1e, task_id=3d0e32c4-288b-4be4-98ef-90fb676b57f2 (api:54) 2019-09-09 09:08:51,741-0400 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call StoragePool.connectStorageServer succeeded in 0.19 seconds (__init__:312) 2019-09-09 09:08:51,992-0400 INFO (jsonrpc/4) [vdsm.api] START connectStorageServer(domType=1, spUUID=u'00000000-0000-0000-0000-000000000000', conList=[{u'tpgt': u'1', u'id': u'1280de25-c248-41d3-8552-50ef964d0ee0', u'connection': u'192.168.20.125:/ovirt', u'iqn': u'', u'user': u'', u'ipv6_enabled': u'false', u'protocol_version': u'auto', u'password': '********', u'port': u''}], options=None) from=::ffff:192.168.20.116,41974, flow_id=360af944, task_id=44612f73-58c9-4597-9a29-c1e65e149c0a (api:48) 2019-09-09 09:08:51,999-0400 INFO (jsonrpc/4) [vdsm.api] FINISH connectStorageServer return={'statuslist': [{'status': 0, 'id': u'1280de25-c248-41d3-8552-50ef964d0ee0'}]} from=::ffff:192.168.20.116,41974, flow_id=360af944, task_id=44612f73-58c9-4597-9a29-c1e65e149c0a (api:54) 2019-09-09 09:08:52,000-0400 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call StoragePool.connectStorageServer succeeded in 0.01 seconds (__init__:312) 2019-09-09 09:08:52,010-0400 INFO (jsonrpc/6) [vdsm.api] START createStorageDomain(storageType=1, sdUUID=u'785a9a50-96c0-4cac-9c80-d0e2dd5bd162', domainName=u'ovirt-omv', typeSpecificArg=u'192.168.20.125:/ovirt', domClass=1, domVersion=u'5', block_size=512, max_hosts=250, options=None) from=::ffff:192.168.20.116,41974, flow_id=360af944, task_id=62008086-97f8-4986-b329-cc193b84fd95 (api:48) 2019-09-09 09:08:52,727-0400 WARN (jsonrpc/6) [storage.LVM] Reloading VGs failed (vgs=[u'785a9a50-96c0-4cac-9c80-d0e2dd5bd162'] rc=5 out=[] err=[' Volume group "785a9a50-96c0-4cac-9c80-d0e2dd5bd162" not found', ' Cannot process volume group 785a9a50-96c0-4cac-9c80-d0e2dd5bd162']) (lvm:470) 2019-09-09 09:08:52,731-0400 INFO (itmap/0) [IOProcessClient] (/baremetal3.ledinfo.com:_export) Starting client (__init__:308) 2019-09-09 09:08:52,751-0400 INFO (ioprocess/9141) [IOProcess] (/baremetal3.ledinfo.com:_export) Starting ioprocess (__init__:434) 2019-09-09 09:08:52,753-0400 INFO (itmap/1) [IOProcessClient] (/baremetal3.ledinfo.com:_import) Starting client (__init__:308) 2019-09-09 09:08:52,777-0400 INFO (itmap/2) [IOProcessClient] (/baremetal3.ledinfo.com:_ISO) Starting client (__init__:308) 2019-09-09 09:08:52,783-0400 INFO (ioprocess/9150) [IOProcess] (/baremetal3.ledinfo.com:_import) Starting ioprocess (__init__:434) 2019-09-09 09:08:52,802-0400 INFO (itmap/3) [IOProcessClient] (/192.168.20.125:_ovirt) Starting client (__init__:308) 2019-09-09 09:08:52,822-0400 INFO (ioprocess/9157) [IOProcess] (/baremetal3.ledinfo.com:_ISO) Starting ioprocess (__init__:434) 2019-09-09 09:08:52,828-0400 INFO (ioprocess/9160) [IOProcess] (/192.168.20.125:_ovirt) Starting ioprocess (__init__:434) 2019-09-09 09:08:52,831-0400 INFO (jsonrpc/6) [storage.StorageDomain] sdUUID=785a9a50-96c0-4cac-9c80-d0e2dd5bd162 domainName=ovirt-omv remotePath=192.168.20.125:/ovirt domClass=1, block_size=512, alignment=1048576 (nfsSD:86) 2019-09-09 09:08:52,866-0400 INFO (jsonrpc/6) [IOProcessClient] (785a9a50-96c0-4cac-9c80-d0e2dd5bd162) Starting client (__init__:308) 2019-09-09 09:08:52,889-0400 INFO (ioprocess/9174) [IOProcess] (785a9a50-96c0-4cac-9c80-d0e2dd5bd162) Starting ioprocess (__init__:434) 2019-09-09 09:08:52,895-0400 INFO (jsonrpc/6) [storage.StorageDomain] Creating domain metadata directory u'/rhev/data-center/mnt/192.168.20.125:_ovirt/785a9a50-96c0-4cac-9c80-d0e2dd 5bd162/dom_md' (fileSD:453) 2019-09-09 09:08:52,926-0400 INFO (jsonrpc/6) [storage.xlease] Formatting index for lockspace u'785a9a50-96c0-4cac-9c80-d0e2dd5bd162' (version=1) (xlease:691) 2019-09-09 09:08:53,047-0400 INFO (jsonrpc/6) [storage.StorageDomain] Creating domain images directory u'/rhev/data-center/mnt/192.168.20.125:_ovirt/785a9a50-96c0-4cac-9c80-d0e2dd 5bd162/images' (nfsSD:110) 2019-09-09 09:08:53,067-0400 INFO (jsonrpc/6) [storage.StorageDomain] Removing remnants of deleted images [] (fileSD:747) 2019-09-09 09:08:53,072-0400 ERROR (jsonrpc/6) [storage.initSANLock] Cannot initialize SANLock for domain 785a9a50-96c0-4cac-9c80-d0e2dd5bd162 (clusterlock:258) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/clusterlock.py", line 249, in initSANLock sdUUID, idsPath, align=alignment, sector=block_size) File "/usr/lib/python2.7/site-packages/vdsm/storage/compat.py", line 52, in decorator return func(*args, **kwargs) SanlockException: (19, 'Sanlock lockspace write failure', 'No such device') 2019-09-09 09:08:53,073-0400 INFO (jsonrpc/6) [vdsm.api] FINISH createStorageDomain error=Could not initialize cluster lock: () from=::ffff:192.168.20.116,41974, flow_id=360af944, task_id=62008086-97f8-4986-b329-cc193b84fd95 (api:52) 2019-09-09 09:08:53,074-0400 ERROR (jsonrpc/6) [storage.TaskManager.Task] (Task='62008086-97f8-4986-b329-cc193b84fd95') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "<string>", line 2, in createStorageDomain File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 2619, in createStorageDomain storageType, domVersion, block_size, alignment) File "/usr/lib/python2.7/site-packages/vdsm/storage/nfsSD.py", line 120, in create fsd.initSPMlease() File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 918, in initSPMlease return self._manifest.initDomainLock() File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 609, in initDomainLock self._domainLock.initLock(self.getDomainLease()) File "/usr/lib/python2.7/site-packages/vdsm/storage/clusterlock.py", line 303, in initLock block_size=self._block_size) File "/usr/lib/python2.7/site-packages/vdsm/storage/clusterlock.py", line 259, in initSANLock raise se.ClusterLockInitError() ClusterLockInitError: Could not initialize cluster lock: () 2019-09-09 09:08:53,075-0400 INFO (jsonrpc/6) [storage.TaskManager.Task] (Task='62008086-97f8-4986-b329-cc193b84fd95') aborting: Task is aborted: 'Could not initialize cluster lock: ()' - code 701 (task:1181) 2019-09-09 09:08:53,076-0400 ERROR (jsonrpc/6) [storage.Dispatcher] FINISH createStorageDomain error=Could not initialize cluster lock: () (dispatcher:83) 2019-09-09 09:08:53,077-0400 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call StorageDomain.create failed (error 701) in 1.07 seconds (__init__:312) 2019-09-09 09:08:53,425-0400 INFO (jsonrpc/0) [api.host] START getStats() from=::ffff:192.168.20.116,41974 (api:48) 2019-09-09 09:08:53,439-0400 INFO (jsonrpc/1) [vdsm.api] START disconnectStorageServer(domType=1, spUUID=u'00000000-0000-0000-0000-000000000000', conList=[{u'tpgt': u'1', u'id': u'1280de25-c248-41d3-8552-50ef964d0ee0', u'connection': u'192.168.20.125:/ovirt', u'iqn': u'', u'user': u'', u'ipv6_enabled': u'false', u'protocol_version': u'auto', u'password': '********', u'port': u''}], options=None) from=::ffff:192.168.20.116,41974, flow_id=e003809f-c7ef-4105-ac27-48be39b5c7b1, task_id=da5b627a-de84-4083-9454-aaecd6bb8aed (api:48) 2019-09-09 09:08:53,440-0400 INFO (jsonrpc/1) [storage.Mount] unmounting /rhev/data-center/mnt/192.168.20.125:_ovirt (mount:212) 2019-09-09 09:08:53,478-0400 INFO (jsonrpc/0) [vdsm.api] START repoStats(domains=()) from=::ffff:192.168.20.116,41974, task_id=42f53132-6ef6-4aff-9ab3-91d19c24bd93 (api:48) 2019-09-09 09:08:53,479-0400 INFO (jsonrpc/0) [vdsm.api] FINISH repoStats return={u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.00104514', 'lastCheck': '3.4', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000798507', 'lastCheck': '3.5', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000768283', 'lastCheck': '3.3', 'valid': True}} from=::ffff:192.168.20.116,41974, task_id=42f53132-6ef6-4aff-9ab3-91d19c24bd93 (api:54) 2019-09-09 09:08:53,481-0400 INFO (jsonrpc/0) [vdsm.api] START multipath_health() from=::ffff:192.168.20.116,41974, task_id=34ab20bd-b1ce-4bbe-ad1c-af0e78d38b1c (api:48) 2019-09-09 09:08:53,481-0400 INFO (jsonrpc/0) [vdsm.api] FINISH multipath_health return={} from=::ffff:192.168.20.116,41974, task_id=34ab20bd-b1ce-4bbe-ad1c-af0e78d38b1c (api:54) 2019-09-09 09:08:53,523-0400 INFO (jsonrpc/0) [api.host] FINISH getStats return={'status': {'message': 'Done', 'code': 0}, 'info': {'cpuStatistics': {'24': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.53'}, '25': {'cpuUser': '0.46', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.47'}, '26': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '27': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '20': {'cpuUser': '2.06', 'nodeIndex': 0, 'cpuSys': '0.40', 'cpuIdle': '97.54'}, '21': {'cpuUser': '0.99', 'nodeIndex': 1, 'cpuSys': '0.33', 'cpuIdle': '98.68'}, '22': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '23': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.74'}, '28': {'cpuUser': '0.99', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '98.88'}, '29': {'cpuUser': '0.46', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.47'}, '1': {'cpuUser': '2.85', 'nodeIndex': 1, 'cpuSys': '1.66', 'cpuIdle': '95.49'}, '0': {'cpuUser': '0.46', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.41'}, '3': {'cpuUser': '0.73', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.20'}, '2': {'cpuUser': '0.33', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.47'}, '5': {'cpuUser': '0.20', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.73'}, '4': {'cpuUser': '0.99', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '98.68'}, '7': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.74'}, '6': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '99.33'}, '9': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '8': {'cpuUser': '0.93', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '98.80'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.53'}, '13': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '12': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '15': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '14': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '17': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '16': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.87'}, '19': {'cpuUser': '1.79', 'nodeIndex': 1, 'cpuSys': '0.46', 'cpuIdle': '97.75'}, '18': {'cpuUser': '8.69', 'nodeIndex': 0, 'cpuSys': '1.72', 'cpuIdle': '89.59'}, '31': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '30': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.80'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31474'}, '0': {'memPercent': 7, 'memFree': '30445'}}, 'memShared': 0, 'haScore': 3400, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'vmCount': 0, 'memUsed': '3', 'storageDomains': {u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.00104514', 'lastCheck': '3.4', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000798507', 'lastCheck': '3.5', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000768283', 'lastCheck': '3.3', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'idrac': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'idrac', 'tx': '180', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '0', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '0', 'state': 'down'}, 'ovirtmgmt': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '1075026007', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '43585639790', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '618193132', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '618193132', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '0', 'state': 'down'}, 'em4': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em4', 'tx': '192', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034533.476725, 'rx': '38644112', 'state': 'up'}, 'br-int': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'br-int', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '0', 'state': 'down'}, 'em1': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em1', 'tx': '1115341150', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034533.476725, 'rx': '45376217511', 'state': 'up'}, 'genev_sys_6081': {'rxErrors': '0', 'txErrors': '8', 'speed': '1000', 'rxDropped': '0', 'name': 'genev_sys_6081', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '0', 'state': 'up'}, 'em3': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em3', 'tx': '192', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034533.476725, 'rx': '38644112', 'state': 'up'}, 'em2': {'rxErrors': '0', 'txErrors': '0', 'speed': '100', 'rxDropped': '0', 'name': 'em2', 'tx': '0', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034533.476725, 'rx': '16049108', 'state': 'up'}}, 'txDropped': '0', 'anonHugePages': '306', 'ksmPages': 100, 'elapsedTime': '223906.36', 'cpuLoad': '0.14', 'cpuSys': '0.24', 'diskStats': {'/var/log': {'free': '136508'}, '/var/run/vdsm/': {'free': '31856'}, '/tmp': {'free': '136508'}}, 'cpuUserVdsmd': '2.78', 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 63031, 'bootTime': '1567810572', 'haStats': {'active': True, 'configured': True, 'score': 3400, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'multipathHealth': {}, 'rxDropped': '0', 'outgoingVmMigrations': 0, 'swapTotal': 0, 'swapFree': 0, 'hugepages': defaultdict(<type 'dict'>, {1048576: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}, 2048: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}), 'dateTime': '2019-09-09T13:08:53 GMT', 'cpuUser': '0.75', 'memFree': 62775, 'cpuIdle': '99.01', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd': '1.06'}} from=::ffff:192.168.20.116,41974 (api:54) 2019-09-09 09:08:53,534-0400 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.11 seconds (__init__:312) 2019-09-09 09:08:54,001-0400 INFO (jsonrpc/1) [vdsm.api] FINISH disconnectStorageServer return={'statuslist': [{'status': 0, 'id': u'1280de25-c248-41d3-8552-50ef964d0ee0'}]} from=::ffff:192.168.20.116,41974, flow_id=e003809f-c7ef-4105-ac27-48be39b5c7b1, task_id=da5b627a-de84-4083-9454-aaecd6bb8aed (api:54) 2019-09-09 09:08:54,003-0400 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call StoragePool.disconnectStorageServer succeeded in 0.57 seconds (__init__:312) 2019-09-09 09:08:54,888-0400 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:54,895-0400 INFO (jsonrpc/2) [api.host] START getStats() from=::1,42394 (api:48) 2019-09-09 09:08:54,946-0400 INFO (jsonrpc/2) [vdsm.api] START repoStats(domains=()) from=::1,42394, task_id=f1acae4b-2563-4335-96c7-8b5097aa4127 (api:48) 2019-09-09 09:08:54,947-0400 INFO (jsonrpc/2) [vdsm.api] FINISH repoStats return={u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.00104514', 'lastCheck': '4.8', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000798507', 'lastCheck': '5.0', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000768283', 'lastCheck': '4.8', 'valid': True}} from=::1,42394, task_id=f1acae4b-2563-4335-96c7-8b5097aa4127 (api:54) 2019-09-09 09:08:54,948-0400 INFO (jsonrpc/2) [vdsm.api] START multipath_health() from=::1,42394, task_id=58483dc0-2a50-409a-b991-16dbb57360a7 (api:48) 2019-09-09 09:08:54,949-0400 INFO (jsonrpc/2) [vdsm.api] FINISH multipath_health return={} from=::1,42394, task_id=58483dc0-2a50-409a-b991-16dbb57360a7 (api:54) 2019-09-09 09:08:54,977-0400 INFO (jsonrpc/2) [api.host] FINISH getStats return={'status': {'message': 'Done', 'code': 0}, 'info': {'cpuStatistics': {'24': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.53'}, '25': {'cpuUser': '0.46', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.47'}, '26': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '27': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '20': {'cpuUser': '2.06', 'nodeIndex': 0, 'cpuSys': '0.40', 'cpuIdle': '97.54'}, '21': {'cpuUser': '0.99', 'nodeIndex': 1, 'cpuSys': '0.33', 'cpuIdle': '98.68'}, '22': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '23': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.74'}, '28': {'cpuUser': '0.99', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '98.88'}, '29': {'cpuUser': '0.46', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.47'}, '1': {'cpuUser': '2.85', 'nodeIndex': 1, 'cpuSys': '1.66', 'cpuIdle': '95.49'}, '0': {'cpuUser': '0.46', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.41'}, '3': {'cpuUser': '0.73', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.20'}, '2': {'cpuUser': '0.33', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.47'}, '5': {'cpuUser': '0.20', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.73'}, '4': {'cpuUser': '0.99', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '98.68'}, '7': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.74'}, '6': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '99.33'}, '9': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '8': {'cpuUser': '0.93', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '98.80'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.53'}, '13': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '12': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '15': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '14': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '17': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '16': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.87'}, '19': {'cpuUser': '1.79', 'nodeIndex': 1, 'cpuSys': '0.46', 'cpuIdle': '97.75'}, '18': {'cpuUser': '8.69', 'nodeIndex': 0, 'cpuSys': '1.72', 'cpuIdle': '89.59'}, '31': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '30': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.80'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31474'}, '0': {'memPercent': 7, 'memFree': '30445'}}, 'memShared': 0, 'haScore': 3400, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'vmCount': 0, 'memUsed': '3', 'storageDomains': {u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.00104514', 'lastCheck': '4.8', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000798507', 'lastCheck': '5.0', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000768283', 'lastCheck': '4.8', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'idrac': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'idrac', 'tx': '180', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '0', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '0', 'state': 'down'}, 'ovirtmgmt': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '1075046072', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '43586697397', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '618194231', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '618194231', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '0', 'state': 'down'}, 'em4': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em4', 'tx': '192', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034534.944025, 'rx': '38644112', 'state': 'up'}, 'br-int': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'br-int', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '0', 'state': 'down'}, 'em1': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em1', 'tx': '1115653922', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034534.944025, 'rx': '45377347900', 'state': 'up'}, 'genev_sys_6081': {'rxErrors': '0', 'txErrors': '8', 'speed': '1000', 'rxDropped': '0', 'name': 'genev_sys_6081', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '0', 'state': 'up'}, 'em3': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em3', 'tx': '192', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034534.944025, 'rx': '38644176', 'state': 'up'}, 'em2': {'rxErrors': '0', 'txErrors': '0', 'speed': '100', 'rxDropped': '0', 'name': 'em2', 'tx': '0', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034534.944025, 'rx': '16049108', 'state': 'up'}}, 'txDropped': '0', 'anonHugePages': '306', 'ksmPages': 100, 'elapsedTime': '223907.83', 'cpuLoad': '0.14', 'cpuSys': '0.24', 'diskStats': {'/var/log': {'free': '136508'}, '/var/run/vdsm/': {'free': '31856'}, '/tmp': {'free': '136508'}}, 'cpuUserVdsmd': '2.78', 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 63031, 'bootTime': '1567810572', 'haStats': {'active': True, 'configured': True, 'score': 3400, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'multipathHealth': {}, 'rxDropped': '0', 'outgoingVmMigrations': 0, 'swapTotal': 0, 'swapFree': 0, 'hugepages': defaultdict(<type 'dict'>, {1048576: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}, 2048: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}), 'dateTime': '2019-09-09T13:08:54 GMT', 'cpuUser': '0.75', 'memFree': 62775, 'cpuIdle': '99.01', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd': '1.06'}} from=::1,42394 (api:54) 2019-09-09 09:08:54,986-0400 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.09 seconds (__init__:312) 2019-09-09 09:08:55,842-0400 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:55,850-0400 INFO (jsonrpc/3) [api.virt] START getStats() from=::1,42394, vmId=aeed8192-5313-45d6-b9f5-fb2db09fb646 (api:48) 2019-09-09 09:08:55,850-0400 INFO (jsonrpc/3) [api] FINISH getStats error=Virtual machine does not exist: {'vmId': u'aeed8192-5313-45d6-b9f5-fb2db09fb646'} (api:129) 2019-09-09 09:08:55,850-0400 INFO (jsonrpc/3) [api.virt] FINISH getStats return={'status': {'message': "Virtual machine does not exist: {'vmId': u'aeed8192-5313-45d6-b9f5-fb2db09fb646'}", 'code': 1}} from=::1,42394, vmId=aeed8192-5313-45d6-b9f5-fb2db09fb646 (api:54) 2019-09-09 09:08:55,851-0400 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call VM.getStats failed (error 1) in 0.00 seconds (__init__:312) 2019-09-09 09:08:57,526-0400 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:57,534-0400 INFO (jsonrpc/6) [api.host] START getCapabilities() from=::1,42394 (api:48) 2019-09-09 09:08:57,627-0400 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:57,636-0400 INFO (jsonrpc/1) [vdsm.api] START repoStats(domains=[u'4f4828c2-8f78-4566-a69d-a37cd73446d4']) from=::1,42394, task_id=6da33ef3-de09-48a3-8ba9-6c31d5e2942b (api:48) 2019-09-09 09:08:57,636-0400 INFO (jsonrpc/1) [vdsm.api] FINISH repoStats return={u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000798507', 'lastCheck': '0.4', 'valid': True}} from=::1,42394, task_id=6da33ef3-de09-48a3-8ba9-6c31d5e2942b (api:54) 2019-09-09 09:08:57,637-0400 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:57,870-0400 INFO (jsonrpc/6) [root] managedvolume not supported: Managed Volume Not Supported. Missing package os-brick.: ('Cannot import os_brick',) (caps:152) 2019-09-09 09:08:58,884-0400 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:58,990-0400 INFO (jsonrpc/6) [root] /usr/libexec/vdsm/hooks/after_get_caps/50_openstacknet: rc=0 err= (hooks:114) 2019-09-09 09:08:59,048-0400 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:59,055-0400 INFO (jsonrpc/5) [vdsm.api] START repoStats(domains=[u'4f4828c2-8f78-4566-a69d-a37cd73446d4']) from=::1,42394, task_id=933c8ee8-2330-46ed-a609-38a519447156 (api:48) 2019-09-09 09:08:59,056-0400 INFO (jsonrpc/5) [vdsm.api] FINISH repoStats return={u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000798507', 'lastCheck': '1.8', 'valid': True}} from=::1,42394, task_id=933c8ee8-2330-46ed-a609-38a519447156 (api:54) 2019-09-09 09:08:59,057-0400 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:59,526-0400 INFO (jsonrpc/6) [root] /usr/libexec/vdsm/hooks/after_get_caps/openstacknet_utils.py: rc=0 err= (hooks:114) 2019-09-09 09:08:59,896-0400 INFO (jsonrpc/3) [api.host] START getAllVmStats() from=::ffff:192.168.20.116,41974 (api:48) 2019-09-09 09:08:59,897-0400 INFO (jsonrpc/3) [api.host] FINISH getAllVmStats return={'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} from=::ffff:192.168.20.116,41974 (api:54) 2019-09-09 09:08:59,898-0400 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:09:00,188-0400 INFO (periodic/0) [vdsm.api] START repoStats(domains=()) from=internal, task_id=aec0525d-393b-49fa-a0d4-96bc19f829e3 (api:48) 2019-09-09 09:09:00,190-0400 INFO (periodic/0) [vdsm.api] FINISH repoStats return={u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000991915', 'lastCheck': '0.1', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000985319', 'lastCheck': '0.2', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000768283', 'lastCheck': '3.1', 'valid': True}} from=internal, task_id=aec0525d-393b-49fa-a0d4-96bc19f829e3 (api:54) 2019-09-09 09:09:00,192-0400 INFO (jsonrpc/6) [root] /usr/libexec/vdsm/hooks/after_get_caps/ovirt_provider_ovn_hook: rc=0 err= (hooks:114) 2019-09-09 09:09:00,194-0400 INFO (jsonrpc/6) [api.host] FINISH getCapabilities return={'status': {'message': 'Done', 'code': 0}, 'info': {u'HBAInventory': {u'iSCSI': [{u'InitiatorName': u'iqn.1994-05.com.redhat:d8c85fc0ab85'}], u'FC': []}, u'packages2': {u'kernel': {u'release': u'957.27.2.el7.x86_64', u'version': u'3.10.0'}, u'spice-server': {u'release': u'6.el7_6.1', u'version': u'0.14.0'}, u'librbd1': {u'release': u'4.el7', u'version': u'10.2.5'}, u'vdsm': {u'release': u'1.el7', u'version': u'4.30.24'}, u'qemu-kvm': {u'release': u'18.el7_6.7.1', u'version': u'2.12.0'}, u'openvswitch': {u'release': u'4.el7', u'version': u'2.11.0'}, u'libvirt': {u'release': u'10.el7_6.12', u'version': u'4.5.0'}, u'ovirt-hosted-engine-ha': {u'release': u'1.el7', u'version': u'2.3.3'}, u'qemu-img': {u'release': u'18.el7_6.7.1', u'version': u'2.12.0'}, u'mom': {u'release': u'1.el7.centos', u'version': u'0.5.12'}, u'glusterfs-cli': {u'release': u'1.el7', u'version': u'6.5'}}, u'numaNodeDistance': {u'1': [20, 10], u'0': [10, 20]}, u'cpuModel': u'Intel(R) Xeon(R) CPU E5-2640 v2 @ 2.00GHz', u'nestedVirtualization': False, u'liveMerge': u'true', u'hooks': {u'before_vm_start': {u'50_hostedengine': {u'md5': u'95c810cdcfe4195302a59574a5148289'}, u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'after_network_setup': {u'30_ethtool_options': {u'md5': u'ce1fbad7aa0389e3b06231219140bf0d'}}, u'after_vm_destroy': {u'delete_vhostuserclient_hook': {u'md5': u'c2f279cc9483a3f842f6c29df13994c1'}, u'50_vhostmd': {u'md5': u'bdf4802c0521cf1bae08f2b90a9559cf'}}, u'after_vm_start': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'ea0a5a715da8c1badbcda28e8b8fa00e'}}, u'after_device_migrate_destination': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'6226fbc4d1602994828a3904fc1b875d'}}, u'before_device_migrate_destination': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}}, u'after_device_create': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'6226fbc4d1602994828a3904fc1b875d'}}, u'after_nic_hotplug': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'6226fbc4d1602994828a3904fc1b875d'}}, u'before_vm_migrate_destination': {u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'before_vm_dehibernate': {u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'before_nic_hotplug': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}, u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'ec8d5d7ba063a109f749cd63c7a2b58d'}, u'20_ovirt_provider_ovn_vhostuser_hook': {u'md5': u'a8af653b7386c138b2e6e9738bd6b62c'}, u'10_ovirt_provider_ovn_hook': {u'md5': u'73822988042847bab1ea832a6b9fa837'}}, u'after_get_caps': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'5c3a9ab6e06e039bdd220c0216e45809'}, u'ovirt_provider_ovn_hook': {u'md5': u'1a6ce22288fc2466fcc339da252ef624'}}, u'before_network_setup': {u'50_fcoe': {u'md5': u'28c352339c8beef1e1b05c67d106d062'}}, u'before_device_create': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}, u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'ec8d5d7ba063a109f749cd63c7a2b58d'}, u'20_ovirt_provider_ovn_vhostuser_hook': {u'md5': u'a8af653b7386c138b2e6e9738bd6b62c'}, u'10_ovirt_provider_ovn_hook': {u'md5': u'73822988042847bab1ea832a6b9fa837'}}}, u'supportsIPv6': True, u'realtimeKernel': False, u'vmTypes': [u'kvm'], u'liveSnapshot': u'true', u'cpuThreads': u'32', u'kdumpStatus': 0, u'deferred_preallocation': True, u'networks': {u'ovirtmgmt': {u'iface': u'ovirtmgmt', u'ipv6autoconf': True, u'addr': u'192.168.20.101', u'dhcpv6': False, u'ipv6addrs': [], u'switch': u'legacy', u'bridged': True, u'southbound': u'em1', u'dhcpv4': True, u'netmask': u'255.255.255.0', u'ipv4defaultroute': True, u'stp': u'off', u'ipv4addrs': [u'192.168.20.101/24'], u'mtu': u'1500', u'ipv6gateway': u'::', u'gateway': u'192.168.20.1', u'ports': [u'em1']}}, u'kernelArgs': u'BOOT_IMAGE=/vmlinuz-3.10.0-957.27.2.el7.x86_64 root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rhgb quiet LANG=en_US.UTF-8', u'domain_versions': [0, 2, 3, 4, 5], u'bridges': {u'ovirtmgmt': {u'ipv6autoconf': True, u'addr': u'192.168.20.101', u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': True, u'netmask': u'255.255.255.0', u'ipv4defaultroute': True, u'stp': u'off', u'ipv4addrs': [u'192.168.20.101/24'], u'ipv6gateway': u'::', u'gateway': u'192.168.20.1', u'opts': {u'multicast_last_member_count': u'2', u'vlan_protocol': u'0x8100', u'hash_elasticity': u'4', u'multicast_query_response_interval': u'1000', u'group_fwd_mask': u'0x0', u'multicast_snooping': u'1', u'multicast_startup_query_interval': u'3125', u'hello_timer': u'0', u'multicast_querier_interval': u'25500', u'max_age': u'2000', u'hash_max': u'512', u'stp_state': u'0', u'topology_change_detected': u'0', u'priority': u'32768', u'multicast_igmp_version': u'2', u'multicast_membership_interval': u'26000', u'root_path_cost': u'0', u'root_port': u'0', u'multicast_stats_enabled': u'0', u'multicast_startup_query_count': u'2', u'nf_call_iptables': u'0', u'vlan_stats_enabled': u'0', u'hello_time': u'200', u'topology_change': u'0', u'bridge_id': u'8000.f8bc1234c8a8', u'topology_change_timer': u'0', u'ageing_time': u'30000', u'nf_call_ip6tables': u'0', u'multicast_mld_version': u'1', u'gc_timer': u'5700', u'root_id': u'8000.f8bc1234c8a8', u'nf_call_arptables': u'0', u'group_addr': u'1:80:c2:0:0:0', u'multicast_last_member_interval': u'100', u'default_pvid': u'1', u'multicast_query_interval': u'12500', u'multicast_query_use_ifaddr': u'0', u'tcn_timer': u'0', u'multicast_router': u'1', u'vlan_filtering': u'0', u'multicast_querier': u'0', u'forward_delay': u'0'}, u'ports': [u'em1']}}, u'uuid': u'4c4c4544-0046-3310-8034-c8c04f4a3032', u'onlineCpus': u'0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,1,3,5,7,9,11,13,15,17,19,21,23, 25,27,29,31', u'nameservers': [u'192.168.168.1'], u'nics': {u'em4': {u'ipv6autoconf': True, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:ab', u'ipv6gateway': u'::', u'gateway': u''}, u'idrac': {u'ipv6autoconf': True, u'addr': u'', u'speed': 0, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:37:d8:b1', u'ipv6gateway': u'::', u'gateway': u''}, u'em1': {u'ipv6autoconf': False, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:a8', u'ipv6gateway': u'::', u'gateway': u''}, u'em3': {u'ipv6autoconf': True, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:aa', u'ipv6gateway': u'::', u'gateway': u''}, u'em2': {u'ipv6autoconf': True, u'addr': u'', u'speed': 100, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:a9', u'ipv6gateway': u'::', u'gateway': u''}}, u'software_revision': u'1', u'hostdevPassthrough': u'false', u'clusterLevels': [u'4.1', u'4.2', u'4.3'], u'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,cl flush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,c onstant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfm perf,eagerfpu,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16 ,xtpr,pdcm,pcid,dca,sse4_1,sse4_2,x2apic,popcnt,tsc_deadline_timer,aes,xsave ,avx,f16c,rdrand,lahf_lm,ssbd,ibrs,ibpb,stibp,tpr_shadow,vnmi,flexpriority,e pt,vpid,fsgsbase,smep,erms,xsaveopt,dtherm,ida,arat,pln,pts,md_clear,spec_ct rl,intel_stibp,flush_l1d,model_Opteron_G2,model_kvm32,model_SandyBridge-IBRS ,model_Opteron_G1,model_core2duo,model_pentium,model_Westmere-IBRS,model_qem u64,model_kvm64,model_coreduo,model_Nehalem,model_Nehalem-IBRS,model_IvyBrid ge,model_Westmere,model_SandyBridge,model_IvyBridge-IBRS,model_qemu32,model_ Penryn,model_pentium2,model_pentium3,model_Conroe,model_486', u'kernelFeatures': {u'IBRS': 0, u'RETP': 1, u'SSBD': 3, u'PTI': 1}, u'ISCSIInitiatorName': u'iqn.1994-05.com.redhat:d8c85fc0ab85', u'netConfigDirty': u'False', u'supported_block_sizes': [512], u'selinux': {u'mode': u'1'}, u'autoNumaBalancing': 1, u'reservedMem': u'321', u'bondings': {}, u'software_version': u'4.30.24', u'supportedENGINEs': [u'4.1', u'4.2', u'4.3'], u'vncEncrypted': False, u'backupEnabled': False, u'cpuSpeed': u'1199.951', u'numaNodes': {u'1': {u'totalMemory': u'32768', u'cpus': [1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31]}, u'0': {u'totalMemory': u'32704', u'cpus': [0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30]}}, u'cpuSockets': u'2', u'vlans': {}, u'version_name': u'Snow Man', 'lastClientIface': 'lo', u'cpuCores': u'16', u'hostedEngineDeployed': True, u'hugepages': [1048576, 2048], u'guestOverhead': u'65', u'additionalFeatures': [u'libgfapi_supported'], u'openstack_binding_host_ids': {u'OPENSTACK_OVN': u'baremetal1.ledinfo.com', u'OPEN_VSWITCH': u'baremetal1.ledinfo.com', u'OVIRT_PROVIDER_OVN': u'c83bc734-603c-4a99-a5b9-5833ef1d90c7'}, u'kvmEnabled': u'true', u'memSize': u'64195', u'emulatedMachines': [u'pc-i440fx-rhel7.1.0', u'pc-q35-rhel7.3.0', u'rhel6.3.0', u'pc-i440fx-rhel7.5.0', u'pc-i440fx-rhel7.0.0', u'rhel6.1.0', u'pc-q35-rhel7.6.0', u'pc-i440fx-rhel7.4.0', u'rhel6.6.0', u'pc-q35-rhel7.5.0', u'rhel6.2.0', u'pc', u'pc-i440fx-rhel7.3.0', u'q35', u'pc-i440fx-rhel7.2.0', u'rhel6.4.0', u'pc-q35-rhel7.4.0', u'pc-i440fx-rhel7.6.0', u'rhel6.0.0', u'rhel6.5.0'], u'rngSources': [u'hwrng', u'random'], u'operatingSystem': {u'release': u'6.1810.2.el7.centos', u'pretty_name': u'CentOS Linux 7 (Core)', u'version': u'7', u'name': u'RHEL'}}} from=::1,42394 (api:54) 2019-09-09 09:09:00,231-0400 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.getCapabilities succeeded in 2.69 seconds (__init__:312) 2019-09-09 09:09:05,021-0400 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:09:05,028-0400 INFO (jsonrpc/0) [api.host] START getStats() from=::1,42394 (api:48) 2019-09-09 09:09:05,077-0400 INFO (jsonrpc/0) [vdsm.api] START repoStats(domains=()) from=::1,42394, task_id=12e8837e-c9cd-45b9-aecb-477f86e12ec5 (api:48) 2019-09-09 09:09:05,078-0400 INFO (jsonrpc/0) [vdsm.api] FINISH repoStats return={u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000991915', 'lastCheck': '5.0', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000985319', 'lastCheck': '5.1', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.00093201', 'lastCheck': '4.8', 'valid': True}} from=::1,42394, task_id=12e8837e-c9cd-45b9-aecb-477f86e12ec5 (api:54) 2019-09-09 09:09:05,079-0400 INFO (jsonrpc/0) [vdsm.api] START multipath_health() from=::1,42394, task_id=42c12b61-4d82-4a06-b692-e11dfbb0bca6 (api:48) 2019-09-09 09:09:05,080-0400 INFO (jsonrpc/0) [vdsm.api] FINISH multipath_health return={} from=::1,42394, task_id=42c12b61-4d82-4a06-b692-e11dfbb0bca6 (api:54) 2019-09-09 09:09:05,106-0400 INFO (jsonrpc/0) [api.host] FINISH getStats return={'status': {'message': 'Done', 'code': 0}, 'info': {'cpuStatistics': {'24': {'cpuUser': '0.46', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.41'}, '25': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '26': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '27': {'cpuUser': '1.66', 'nodeIndex': 1, 'cpuSys': '0.40', 'cpuIdle': '97.94'}, '20': {'cpuUser': '1.66', 'nodeIndex': 0, 'cpuSys': '0.86', 'cpuIdle': '97.48'}, '21': {'cpuUser': '0.73', 'nodeIndex': 1, 'cpuSys': '0.40', 'cpuIdle': '98.87'}, '22': {'cpuUser': '0.93', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '98.80'}, '23': {'cpuUser': '0.93', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '98.94'}, '28': {'cpuUser': '1.00', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '98.73'}, '29': {'cpuUser': '0.60', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.27'}, '1': {'cpuUser': '4.31', 'nodeIndex': 1, 'cpuSys': '4.05', 'cpuIdle': '91.64'}, '0': {'cpuUser': '0.60', 'nodeIndex': 0, 'cpuSys': '0.40', 'cpuIdle': '99.00'}, '3': {'cpuUser': '0.27', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.66'}, '2': {'cpuUser': '7.23', 'nodeIndex': 0, 'cpuSys': '1.06', 'cpuIdle': '91.71'}, '5': {'cpuUser': '0.20', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.67'}, '4': {'cpuUser': '1.99', 'nodeIndex': 0, 'cpuSys': '0.86', 'cpuIdle': '97.15'}, '7': {'cpuUser': '0.27', 'nodeIndex': 1, 'cpuSys': '0.27', 'cpuIdle': '99.46'}, '6': {'cpuUser': '1.13', 'nodeIndex': 0, 'cpuSys': '0.60', 'cpuIdle': '98.27'}, '9': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.80'}, '8': {'cpuUser': '0.80', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '98.87'}, '11': {'cpuUser': '0.66', 'nodeIndex': 1, 'cpuSys': '0.40', 'cpuIdle': '98.94'}, '10': {'cpuUser': '0.33', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '99.40'}, '13': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '12': {'cpuUser': '1.33', 'nodeIndex': 0, 'cpuSys': '0.46', 'cpuIdle': '98.21'}, '15': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '14': {'cpuUser': '0.46', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '99.27'}, '17': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '16': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '19': {'cpuUser': '6.77', 'nodeIndex': 1, 'cpuSys': '1.59', 'cpuIdle': '91.64'}, '18': {'cpuUser': '14.40', 'nodeIndex': 0, 'cpuSys': '3.65', 'cpuIdle': '81.95'}, '31': {'cpuUser': '0.46', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.41'}, '30': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31468'}, '0': {'memPercent': 7, 'memFree': '30418'}}, 'memShared': 0, 'haScore': 3400, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'vmCount': 0, 'memUsed': '3', 'storageDomains': {u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000991915', 'lastCheck': '5.0', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000985319', 'lastCheck': '5.1', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.00093201', 'lastCheck': '4.8', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'idrac': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'idrac', 'tx': '180', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '0', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '0', 'state': 'down'}, 'ovirtmgmt': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '1075063579', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '43586979921', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '618221192', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '618221192', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '0', 'state': 'down'}, 'em4': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em4', 'tx': '192', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034545.075349, 'rx': '38652218', 'state': 'up'}, 'br-int': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'br-int', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '0', 'state': 'down'}, 'em1': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em1', 'tx': '1115671653', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034545.075349, 'rx': '45377642078', 'state': 'up'}, 'genev_sys_6081': {'rxErrors': '0', 'txErrors': '8', 'speed': '1000', 'rxDropped': '0', 'name': 'genev_sys_6081', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '0', 'state': 'up'}, 'em3': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em3', 'tx': '192', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034545.075349, 'rx': '38652218', 'state': 'up'}, 'em2': {'rxErrors': '0', 'txErrors': '0', 'speed': '100', 'rxDropped': '0', 'name': 'em2', 'tx': '0', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034545.075349, 'rx': '16049684', 'state': 'up'}}, 'txDropped': '0', 'anonHugePages': '320', 'ksmPages': 100, 'elapsedTime': '223917.96', 'cpuLoad': '0.13', 'cpuSys': '0.55', 'diskStats': {'/var/log': {'free': '136515'}, '/var/run/vdsm/': {'free': '31856'}, '/tmp': {'free': '136515'}}, 'cpuUserVdsmd': '4.25', 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 63032, 'bootTime': '1567810572', 'haStats': {'active': True, 'configured': True, 'score': 3400, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'multipathHealth': {}, 'rxDropped': '0', 'outgoingVmMigrations': 0, 'swapTotal': 0, 'swapFree': 0, 'hugepages': defaultdict(<type 'dict'>, {1048576: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}, 2048: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}), 'dateTime': '2019-09-09T13:09:05 GMT', 'cpuUser': '1.56', 'memFree': 62776, 'cpuIdle': '97.89', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd': '1.99'}} from=::1,42394 (api:54) 2019-09-09 09:09:05,115-0400 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.09 seconds (__init__:312)

2 2

ovirt-engine-extension-aaa-ldap-setup
by Rick A 09 Sep '19

09 Sep '19

Trying to get oVirt to use LDAP for user authentication. It seems to bind, but when I test it (Login Flow and Search) before applying the settings, I get the message below (I can confirm the credentials ARE valid). I also tried to set it up manually and still get the same result. I'm using the same information on Bugzilla and it works fine. I've seen similar threads here but without much answers. Anyone have any good links or docs I can view to get this set up? oVirt Node Version: 4.3.3.6-1.el7 API: <--Authn.InvokeCommands.AUTHENTICATE_CREDENTIALS profile='mydomain.com' result=CREDENTIALS_INVALID

3 5

network config change
by mailing-ovirt＠qic.ca 09 Sep '19

09 Sep '19

Greetings, So we have a single hosted engine ovirt box deployed. When this was deployed, the network requirements were poorly determined and there was no port teaming or any vlan configuration. now we need to change that. I have searched all over for any form of procedure or guide as to how we do this. in summary: I need to go from a setup that uses em1 as it's default device and gateway to a setup where em1 and 3 are bonded into bond0 and now use that instead. As for the vlan we could live with keeping the native vlan configured on the cisco side for management and only add extra ones for VMs. any assistance as to what I need to go through from a bash shell would be appreciated. thanks Simon

3 3

VM cant shutdown successfully,It's status always in the way to shutdown
by zhouhao＠vip.friendtimes.net 09 Sep '19

09 Sep '19

Sometimes on the engine website ,the vm's status always in the way to shutdown ,even if I wait for it for hours; It either failed or is shutting down and the "power off" cant shutdown the vm too. zhouhao(a)vip.friendtimes.net

2 2

Failure while import VM from export domain
by Timmi 08 Sep '19

08 Sep '19

Hi oVirt list, I'm planing to migrate my oVirt installation on a new HW. My current platform is running a 4.2.8. The new one a 4.3.5. I'm currently trying to import the VMs through the export domain. But some of my old VMs are failing. Which log file would be interesting to check? Also maybe there is a better way to copy more less all VMs from my current oVirt to the new one. Best regards Christoph

2 2

HP NetXen Incorporated NX3031 driver
by Leo David 07 Sep '19

07 Sep '19

Hi everyone, I have this bloody card installed on a node, and it seems that no driver can be loaded for it, "ip link sh" does not show it. Any recommendation about installing this network card on oVirt 4.2.8 ? lspci -v .... 05:00.0 Ethernet controller: NetXen Incorporated NX3031 Multifunction 1/10-Gigabit Server Adapter (rev 42) Subsystem: Hewlett-Packard Company NC522SFP Dual Port 10GbE Server Adapter Flags: fast devsel, IRQ 31 Memory at d9c00000 (64-bit, non-prefetchable) [size=2M] Memory at da000000 (64-bit, non-prefetchable) [size=32M] Expansion ROM at d8000000 [disabled] [size=64K] Capabilities: [40] MSI-X: Enable- Count=64 Masked- Capabilities: [80] Power Management version 3 Capabilities: [a0] MSI: Enable- Count=1/32 Maskable- 64bit+ Capabilities: [c0] Express Endpoint, MSI 00 Capabilities: [100] Advanced Error Reporting Capabilities: [140] Device Serial Number 59-69-46-61-6e-48-73-75 Kernel modules: netxen_nic 05:00.1 Ethernet controller: NetXen Incorporated NX3031 Multifunction 1/10-Gigabit Server Adapter (rev 42) Subsystem: Hewlett-Packard Company NC522SFP Dual Port 10GbE Server Adapter Flags: fast devsel, IRQ 33 Memory at d9e00000 (64-bit, non-prefetchable) [size=2M] Memory at dc000000 (64-bit, non-prefetchable) [size=32M] Capabilities: [40] MSI-X: Enable- Count=64 Masked- Capabilities: [80] Power Management version 3 Capabilities: [a0] MSI: Enable- Count=1/32 Maskable- 64bit+ Capabilities: [c0] Express Endpoint, MSI 00 Capabilities: [100] Advanced Error Reporting Capabilities: [140] Device Serial Number 59-69-46-61-6e-48-73-75 Kernel modules: netxen_nic dmesg | grep netxen [ 2.188078] netxen_nic 0000:05:00.0: 2MB memory map [ 2.188371] netxen_nic 0000:05:00.0: Timeout reached waiting for rom done [ 2.188442] netxen_nic 0000:05:00.0: Error getting board config info. [ 2.189488] netxen_nic: probe of 0000:05:00.0 failed with error -5 [ 2.190438] netxen_nic 0000:05:00.1: 2MB memory map [ 2.190741] netxen_nic 0000:05:00.1: Timeout reached waiting for rom done [ 2.190808] netxen_nic 0000:05:00.1: Error getting board config info. [ 2.190964] netxen_nic: probe of 0000:05:00.1 failed with error -5 ... Thank you very much ! Leo -- Best regards, Leo David

2 1

Windows server std 2019 64 bit support
by Alex K 06 Sep '19

06 Sep '19

Hi all, I am trying to find out if Windows server std 2019 64 bit is supported at ovirt 3.4. I have checked the following links: https://www.ovirt.org/documentation/vmm-guide/chap-Introduction.html https://access.redhat.com/articles/973133 https://www.windowsservercatalog.com/results.aspx?&bCatID=1521&cpID=24121&a… From the above I deduce that RHEV 4.3.5 supports it and ovirt does not include it in the supported list of guest OS. Also KVM on CentOS 7.7 seems to be supporting it as RHEL 7.7 is listed as supported platform. Have you tried to load Windows server std 2019 64 bit successfullyon ovirt 3.4? Thank you.

2 2

ovirt-guest: Network device hanging shortly after startup (after Upgrade to 4.3.5)
by julius.kempa＠astelon.ch 06 Sep '19

06 Sep '19

We have a some old guests (SLES 10.4) which has been worked corectly till 4.2.7 (Manager + Host) After Upgrade to 4.3.5, we have got a network issue, that the gues network device, not depending on used device type, stopts to work after some seconds.... after about 15-20 pings device is hanging. Sometimes, in the first time after the device hungs up, the message "no buffer space available" comes from ping. Does someone knows a solution for it? greetings Julius

3 3

[ANN] oVirt 4.3.6 Fifth Release Candidate is now available for testing
by Sandro Bonazzola 05 Sep '19

05 Sep '19

The oVirt Project is pleased to announce the availability of the oVirt 4.3.6 Fifth Release Candidate for testing, as of September 5th, 2019. This update is a release candidate of the sixth in a series of stabilization updates to the 4.3 series. This is pre-release software. This pre-release should not to be used in production. This release is available now on x86_64 architecture for: * Red Hat Enterprise Linux 7.7 or later (but <8) * CentOS Linux (or similar) 7.7 or later (but <8) This release supports Hypervisor Hosts on x86_64 and ppc64le architectures for: * Red Hat Enterprise Linux 7.7 or later (but <8) * CentOS Linux (or similar) 7.7 or later (but <8) * oVirt Node 4.3 (available for x86_64 only) has been built consuming CentOS 7.7 Candidate Release See the release notes [1] for known issues, new features and bugs fixed. Notes: - oVirt Appliance is already available - oVirt Node is already available based on CentOS 7.7 Candidate Release Additional Resources: * Read more about the oVirt 4.3.6 release highlights: http://www.ovirt.org/release/4.3.6/ * Get more oVirt Project updates on Twitter: https://twitter.com/ovirt * Check out the latest project news on the oVirt blog: http://www.ovirt.org/blog/ [1] http://www.ovirt.org/release/4.3.6/ [2] http://resources.ovirt.org/pub/ovirt-4.3-pre/iso/ -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/>*Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*

1 0

VM crushed and cant shutdown successfully,I shoud to reboot the ovirt-node machine
by zhouhao＠vip.friendtimes.net 05 Sep '19

05 Sep '19

VM crushed and cant shutdown successfully,It always on the way to shutdown，even if I wait it for 10 hours；Everytimes I shoud to reboot the ovirt-node machine This comes up 4 times, It appes both on the ovirt-noe 4.3 and 4.2; This bug appers when my VM is running at high load（seems memory and disk's w/r） My storage domain is local on host; When the bug appers,it could cost my node's io,looks like the picture below；But sometimes it will not The vm's threading on the ovirt-node, the IO ratio is 100% Everytime the vm's process will change to defunct I can not kill it,I had to shutdown the ovirt-node on the engine website ,the vm's status always in the way to shutdown ,even if I wait for it for hours; It either failed or is shutting down and the "power off" cant shutdown the vm too. the vm's error below The ovirt-node's error below ---------------------------------------------------------------- The other infomation about my ovirt-node node-version： node hardware zhouhao(a)vip.friendtimes.net

1 0

Re: [ANN] oVirt 4.3.6 Fourth Release Candidate is now available for testing
by Strahil 05 Sep '19

05 Sep '19

Hi Sandro, Can the update work with 7.6 ? I'm asking because CentOS 7.7 is still building and unavailable. Best Regards, Strahil NikolovOn Aug 29, 2019 10:47, Sandro Bonazzola <sbonazzo(a)redhat.com> wrote: > > The oVirt Project is pleased to announce the availability of the oVirt 4.3.6 Fourth Release Candidate for testing, as of August 29th, 2019. > > This update is a release candidate of the sixth in a series of stabilization updates to the 4.3 series. > This is pre-release software. This pre-release should not to be used in production. > > This release is available now on x86_64 architecture for: > * Red Hat Enterprise Linux 7.7 or later (but <8) > * CentOS Linux (or similar) 7.7 or later (but <8) > > This release supports Hypervisor Hosts on x86_64 and ppc64le architectures for: > * Red Hat Enterprise Linux 7.7 or later (but <8) > * CentOS Linux (or similar) 7.7 or later (but <8) > * oVirt Node 4.3 (available for x86_64 only) will be made available when CentOS 7.7 will be released. > > See the release notes [1] for known issues, new features and bugs fixed. > > Notes: > - oVirt Appliance is already available > - oVirt Node is not yet available, pending CentOS 7.7 release to be available > > Additional Resources: > * Read more about the oVirt 4.3.6 release highlights: http://www.ovirt.org/release/4.3.6/ > * Get more oVirt Project updates on Twitter: https://twitter.com/ovirt > * Check out the latest project news on the oVirt blog:http://www.ovirt.org/blog/ > > [1] http://www.ovirt.org/release/4.3.6/ > [2] http://resources.ovirt.org/pub/ovirt-4.3-pre/iso/ > > -- > > Sandro Bonazzola > > MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV > > Red Hat EMEA > > sbonazzo(a)redhat.com > > Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.

2 1

Re: Any real experiences with nested virtualisation?
by Strahil 05 Sep '19

05 Sep '19

I guess you want to do something like this one : http://www.chrisj.cloud/?q=node/8 As Ovirt is upstream of RHV , I believe that RDO and Ovirt have almost the same integration as in the post. Still,such setup requires a lot of knowledge about KVM, oVirt and OpenStack. Best Regards, Strahil NikolovOn Sep 4, 2019 21:36, thomas(a)hoberg.net wrote: > > First, it's a scenario the oVirt team seem to actually embrace themselves. > > I believe I have seen that type of deployment mentioned either in one of the forward looking blog posts or Red Hat summit videos. > > And then it simply makes a lot of sense to have those crucial management VMs that control the swarm of resources in a cloud run under a fault-tolerant VM management environment such as oVirt. > > As for nested virtualization, I can only provide some negative experience with oVirt. > > I've done nested virtualization with VMware, running ESX on VMWare Workstation and then some VMs underneath that ESX successfully. > > When I tried to do the same with oVirt, running an oVirt hyperconverged cluster as VMs on VMware workstation, I got reasonably far, but then every VM I tried to launch nested would just stop right at boot. > > It first happened with the hosted-engine, as that tries to start as a VM, but it happened more obviously when I tried to migrate VMs (just a plain Fedora 30 in this case) from a physical compute node running oVirt nodeOS to a virtualized compute node, running the same current oVirt nodeOS image on VMware Workstation (Windows 2016 host, if that matters). > > The machine would start the live-migration properly and then just freeze and stay frozen until I migrated it back or indeed to any other non-virtualized node. > > I haven't as yet tried to do that with nested KVM, which I believe might actually work: But it's on my things to test, so I'll report back, once I get around it. > > Actually I am wondering, a) how deep this nesting would be allowed to go and b) if the 'leaf' nodes at the last nesting level should actually have nesting support activated in their kernel parameters... > > Any insight here would be well appreciated. > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/6YDBQGIXUWTDB…

1 0

Any real experiences with nested virtualisation?
by tonyppe＠gmail.com 04 Sep '19

04 Sep '19

I have ovirt 4.3.5 and Openstack hosts. There are 2 x openstack compute and 2 x ovirt hosts with hosted manager. I think i can simplify the management of this if I were to virtualise the openstack compute nodes as VMs on ovirt. I wanted to reach out here to gauge any other users experiences with it if at all. From the research I have done it looks like from the VM perspective, migration must not be allowed. Any info / comments / feedback appreciated. Regards,

2 1

Re: Cannot Increase Hosted Engine VM Memory
by Dionysis K 04 Sep '19

04 Sep '19

Now i noticed another strange i deactivated the memory balloon option from the VM hosted engine and tried to update the values again with the same memory it still keeps the value of memory size at 5120 while the value of physical memory guaranteed it changed to what we desire at the 8192 value and then the error appears again in the vm engine i ckecked the engine log file it shows the same errors! is there are other logfile locations that can give us any clues ?

1 0

Re: Cannot Increase Hosted Engine VM Memory
by Douglas Duckworth 04 Sep '19

04 Sep '19

Yes, I do. Gold crown indeed. It's the "HostedEngine" as seen attached! Thanks, Douglas Duckworth, MSc, LFCS HPC System Administrator Scientific Computing Unit<https://scu.med.cornell.edu> Weill Cornell Medicine 1300 York Avenue New York, NY 10065 E: doug(a)med.cornell.edu<mailto:doug@med.cornell.edu> O: 212-746-6305 F: 212-746-8690 On Wed, Jan 23, 2019 at 12:02 PM Simone Tiraboschi <stirabos(a)redhat.com<mailto:stirabos@redhat.com>> wrote: On Wed, Jan 23, 2019 at 5:51 PM Douglas Duckworth <dod2014(a)med.cornell.edu<mailto:dod2014@med.cornell.edu>> wrote: Hi Simone Can I get help with this issue? Still cannot increase memory for Hosted Engine. From the logs it seams that the engine is trying to hotplug memory to the engine VM which is something it should not happen. The engine should simply update engine VM configuration in the OVF_STORE and require a reboot of the engine VM. Quick question, in the VM panel do you see a gold crown symbol on the Engine VM? Thanks, Douglas Duckworth, MSc, LFCS HPC System Administrator Scientific Computing Unit<https://scu.med.cornell.edu> Weill Cornell Medicine 1300 York Avenue New York, NY 10065 E: doug(a)med.cornell.edu<mailto:doug@med.cornell.edu> O: 212-746-6305 F: 212-746-8690 On Thu, Jan 17, 2019 at 8:08 AM Douglas Duckworth <dod2014(a)med.cornell.edu<mailto:dod2014@med.cornell.edu>> wrote: Sure, they're attached. In "first attempt" the error seems to be: 2019-01-17 07:49:24,795-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-29) [680f82b3-7612-4d91-afdc-43937aa298a2] EVENT_ID: FAILED_HOT_SET_MEMORY_NOT_DIVIDABLE(2,048), Failed to hot plug memory to VM HostedEngine. Amount of added memory (4000MiB) is not dividable by 256MiB. Followed by: 2019-01-17 07:49:24,814-05 WARN [org.ovirt.engine.core.bll.UpdateRngDeviceCommand] (default task-29) [26f5f3ed] Validation of action 'UpdateRngDevice' failed for user admin@internal-authz. Reasons: ACTION_TYPE_FAILED_VM_IS_RUNNING 2019-01-17 07:49:24,815-05 ERROR [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-29) [26f5f3ed] Updating RNG device of VM HostedEngine (adf14389-1563-4b1a-9af6-4b40370a825b) failed. Old RNG device = VmRngDevice:{id='VmDeviceId:{deviceId='6435b2b5-163c-4f0c-934e-7994da60dc89', vmId='adf14389-1563-4b1a-9af6-4b40370a825b'}', device='virtio', type='RNG', specParams='[source=urandom]', address='', managed='true', plugged='true', readOnly='false', deviceAlias='', customProperties='null', snapshotId='null', logicalName='null', hostDevice='null'}. New RNG device = VmRngDevice:{id='VmDeviceId:{deviceId='6435b2b5-163c-4f0c-934e-7994da60dc89', vmId='adf14389-1563-4b1a-9af6-4b40370a825b'}', device='virtio', type='RNG', specParams='[source=urandom]', address='', managed='true', plugged='true', readOnly='false', deviceAlias='', customProperties='null', snapshotId='null', logicalName='null', hostDevice='null'}. In "second attempt" I used values that are dividable by 256 MiB so that's no longer present. Though same error: 2019-01-17 07:56:59,795-05 INFO [org.ovirt.engine.core.vdsbroker.SetAmountOfMemoryVDSCommand] (default task-22) [7059a48f] START, SetAmountOfMemoryVDSCommand(HostName = ovirt-hv1.med.cornell.edu<http://ovirt-hv1.med.cornell.edu>, Params:{hostId='cdd5ffda-95c7-4ffa-ae40-be66f1d15c30', vmId='adf14389-1563-4b1a-9af6-4b40370a825b', memoryDevice='VmDevice:{id='VmDeviceId:{deviceId='7f7d97cc-c273-4033-af53-bc9033ea3abe', vmId='adf14389-1563-4b1a-9af6-4b40370a825b'}', device='memory', type='MEMORY', specParams='[node=0, size=2048]', address='', managed='true', plugged='true', readOnly='false', deviceAlias='', customProperties='null', snapshotId='null', logicalName='null', hostDevice='null'}', minAllocatedMem='6144'}), log id: 50873daa 2019-01-17 07:56:59,855-05 INFO [org.ovirt.engine.core.vdsbroker.SetAmountOfMemoryVDSCommand] (default task-22) [7059a48f] FINISH, SetAmountOfMemoryVDSCommand, log id: 50873daa 2019-01-17 07:56:59,862-05 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-22) [7059a48f] EVENT_ID: HOT_SET_MEMORY(2,039), Hotset memory: changed the amount of memory on VM HostedEngine from 4096 to 4096 2019-01-17 07:56:59,881-05 WARN [org.ovirt.engine.core.bll.UpdateRngDeviceCommand] (default task-22) [28fd4c82] Validation of action 'UpdateRngDevice' failed for user admin@internal-authz. Reasons: ACTION_TYPE_FAILED_VM_IS_RUNNING 2019-01-17 07:56:59,882-05 ERROR [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-22) [28fd4c82] Updating RNG device of VM HostedEngine (adf14389-1563-4b1a-9af6-4b40370a825b) failed. Old RNG device = VmRngDevice:{id='VmDeviceId:{deviceId='6435b2b5-163c-4f0c-934e-7994da60dc89', vmId='adf14389-1563-4b1a-9af6-4b40370a825b'}', device='virtio', type='RNG', specParams='[source=urandom]', address='', managed='true', plugged='true', readOnly='false', deviceAlias='', customProperties='null', snapshotId='null', logicalName='null', hostDevice='null'}. New RNG device = VmRngDevice:{id='VmDeviceId:{deviceId='6435b2b5-163c-4f0c-934e-7994da60dc89', vmId='adf14389-1563-4b1a-9af6-4b40370a825b'}', device='virtio', type='RNG', specParams='[source=urandom]', address='', managed='true', plugged='true', readOnly='false', deviceAlias='', customProperties='null', snapshotId='null', logicalName='null', hostDevice='null'}. This message repeats throughout engine.log: 2019-01-17 07:55:43,270-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-89) [] EVENT_ID: VM_MEMORY_UNDER_GUARANTEED_VALUE(148), VM HostedEngine on host ovirt-hv1.med.cornell.edu<http://ovirt-hv1.med.cornell.edu> was guaranteed 8192 MB but currently has 4224 MB As you can see attached the host has plenty of memory. Thank you Simone! Thanks, Douglas Duckworth, MSc, LFCS HPC System Administrator Scientific Computing Unit<https://scu.med.cornell.edu> Weill Cornell Medicine 1300 York Avenue New York, NY 10065 E: doug(a)med.cornell.edu<mailto:doug@med.cornell.edu> O: 212-746-6305 F: 212-746-8690 On Thu, Jan 17, 2019 at 5:09 AM Simone Tiraboschi <stirabos(a)redhat.com<mailto:stirabos@redhat.com>> wrote: On Wed, Jan 16, 2019 at 8:22 PM Douglas Duckworth <dod2014(a)med.cornell.edu<mailto:dod2014@med.cornell.edu>> wrote: Sorry for accidental send. Anyway I try to increase physical memory however it won't go above 4096MB. The hypervisor has 64GB. Do I need to modify this value with Hosted Engine offline? No, it's not required. Can you please attach your engine.log for the relevant time frame? Thanks, Douglas Duckworth, MSc, LFCS HPC System Administrator Scientific Computing Unit<https://scu.med.cornell.edu> Weill Cornell Medicine 1300 York Avenue New York, NY 10065 E: doug(a)med.cornell.edu<mailto:doug@med.cornell.edu> O: 212-746-6305 F: 212-746-8690 On Wed, Jan 16, 2019 at 1:58 PM Douglas Duckworth <dod2014(a)med.cornell.edu<mailto:dod2014@med.cornell.edu>> wrote: Hello I am trying to increase Hosted Engine physical memory above 4GB Thanks, Douglas Duckworth, MSc, LFCS HPC System Administrator Scientific Computing Unit<https://scu.med.cornell.edu> Weill Cornell Medicine 1300 York Avenue New York, NY 10065 E: doug(a)med.cornell.edu<mailto:doug@med.cornell.edu> O: 212-746-6305 F: 212-746-8690 _______________________________________________ Users mailing list -- users(a)ovirt.org<mailto:users@ovirt.org> To unsubscribe send an email to users-leave(a)ovirt.org<mailto:users-leave@ovirt.org> Privacy Statement: https://www.ovirt.org/site/privacy-policy/<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ovirt.org_site_pri…> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ovirt.org_communit…> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/WGSXQVVPJJ2CR…<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.ovirt.org_archiv…>

3 8

iSCSI Multipath/multiple gateways
by dan.poltawski＠tnp.net.uk 04 Sep '19

04 Sep '19

My iSCSI target (ceph-based) has multiple gateways, I’d like to ensure my hosted storage is aware of these such that each gateway can be rebooted for maintance without impacting service. What is the appropriate way to configure this so that each host knows of the multipath configuration? I do not have multiple network paths, just multiple gateways on the same network, so I think the multipath UI tool isn’t the way to do this. Any help/docs pointers appreciated. Regards, Dan

3 2

the virtual machine crushed and I cant shutdown the vm successfully
by zhouhao＠vip.friendtimes.net 04 Sep '19

04 Sep '19

Dear ovirt developers: The trouble bothered me for a long time;Some times my vm will crush and I cant shutdown it,I had to reboot the ovirt-node to slove it; the vm's error below The ovirt-node's error below The vm's threading on the ovirt-node, the IO ratio is 100% and the vm's process change to defunct I can not kill it,every time I had to shutdown the ovirt-node on the engine website ,the vm's status always in the way to shutdown ,even if I wait for it for hours; It either failed or is shutting down and the "power off" cant shutdown the vm too. ---------------------------------------------------------------- The other infomation about my ovirt-node node-version： node hardware zhouhao(a)vip.friendtimes.net

2 2