September 2019 - Users - oVirt List Archives

Deploying the Self-Hosted Engine Loop
by mustafa.taha.mu95＠gmail.com 11 Sep '19

11 Sep '19

Hi i am trying to deploy self-hot with latest ovirt 4.3 on Centos 7.6.1810 but deploying -first stage of deploy- stick on following logs: [ INFO ] TASK [ovirt.hosted_engine_setup : Set new IPv4 subnet prefix] [ INFO ] skipping: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Search again with another prefix] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Define 3rd chunk] [ INFO ] skipping: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Set 3rd chunk] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Get ip route] [ INFO ] changed: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fail if can't find an available subnet] [ INFO ] skipping: [localhost] can anybody help ?

1 0

Locked Snapshot
by Peter Smith 11 Sep '19

11 Sep '19

Hi, I have been trying to get a backup solution working for my VM's, and have somehow managed to break the snapshot on one of my VM's. I'm still fairly new to oVirt, we're migrating from VMware. I have had this cluster running since May. The cluster: We are running oVirt on CentOS 7, with Hosted Engine. oVirt 4.3.5.4-1.el7 (according to the Help->About in the Admin Console) Hardware is 5 x Dell M620 Blades (300Gb internal storage for host OS) with ISCSI storage for the VM's (Dell M3660i), 10GB Network throughout. The VM in question is allocated 50Gb of storage and there is 461GB free in the storage domain. So I've been playing with bareos 18.2.5 with the ovirt plugin found here https://github.com/eurotux/bareos-contrib/tree/master/fd-plugins/ovirt-plug… I got bareos hooked up to ovirt. It created a snapshot, downloaded it, then at the end it failed to remove the snapshot. I though it may have been because I was looking round bareos while it was doing the backup. Went into oVirt deleted the snapshot no issues. Re ran the backup job in bareos, leaving bareos alone this time. The backup job failed again being unable to remove the snapshot. So I figured I'd go in and delete it, but I get an error saying "Cannot remove Snapshot: The following disks are locked: VM-NAME_vmdisk1. Please try again in a few minutes.". In the Admin console if I expand the disks on the snapshot it shows that the image is locked. And before you ask the bareos user is setup as a SuperUser. If I go to the Events from the menu on the left of the admin console I can see that (Sorry Names/IP's changed to protect the guilty): Sep 10, 2019, 10:42:39 AM Image Download with disk VM-NAME_vmdisk1 succeeded. Sep 10, 2019, 10:42:38 AM VDSM hyper2 command TeardownImageVDS failed: Cannot deactivate Logical Volume: ('General Storage Exception: ("5 [] [\' Logical volume 46a9dfd4-4871-46b2-a740-8fe430c70b4b/bbe315b9-e797-4cf8-b18b-6afeb33589cc in use.\', \' Logical volume 46a9dfd4-4871-46b2-a740-8fe430c70b4b/d3dfa2c2-45fa-4081-98c5-890dd7a5990a in use.\']\\n46a9dfd4-4871-46b2-a740-8fe430c70b4b/[\'bbe315b9-e797-4cf8-b18b-6afeb33589cc\', \'d3dfa2c2-45fa-4081-98c5-890dd7a5990a\']",)',) Sep 10, 2019, 10:22:09 AM Image Download with disk VM-NAME_vmdisk1 was initiated by bareos@internal-authz. Sep 10, 2019, 10:21:55 AM Snapshot 'VM-NAME-backup-1f952494-8245-479e-8d1d-9503d6fe8f9f' creation for VM 'VM-NAME' has been completed. Sep 10, 2019, 10:21:36 AM Snapshot 'VM-NAME-backup-1f952494-8245-479e-8d1d-9503d6fe8f9f' creation for VM 'VM-NAME' was initiated by bareos@internal-authz. Sep 10, 2019, 10:21:36 AM Backup of virtual machine 'VM-NAME using snapshot 'VM-NAME-backup-1f952494-8245-479e-8d1d-9503d6fe8f9f' is starting. Sep 10, 2019, 10:21:36 AM User bareos@internal-authz connecting from '10.*******' using session '****************' logged in. To me it looks like bareos tried to delete the snapshot whilst it was still downloading it, which failed as expected. Does anyone know how to fix/recover from this? If you need log files, please could you tell me which ones? Regards, Peter Smith

3 3

How to change Master Data Storage Domain
by Mark Steele 11 Sep '19

11 Sep '19

Good morning, I have a Storage Domain that I would like to retire and move that role to a new storage domain. Both Domains exist on my current Data Center and I have moved all disks from the existing Data (Master) domain to the new Data Domain. The only thing that is still associated with the Data (Master) domain are two OVF_STORE items: Alias Virtual Size Actual Size Allocation Policy Storage Domain Storage Type Creation Date Attached To Alignment Status Description OVF_STORE < 1 GB < 1 GB Preallocated phl-datastore NFS 2014-Nov-14, 20:00 Unknown OK OVF_STORE OVF_STORE < 1 GB < 1 GB Preallocated phl-datastore NFS 2014-Nov-14, 20:00 Unknown OK OVF_STORE What is the procedure for moving / removing these items and 'promoting' the other Data Domain to Master? Our current version is: oVirt Engine Version: 3.5.0.1-1.el6 (it's old but reliable) Best regards, *** *Mark Steele* CIO / VP Technical Operations | TelVue Corporation TelVue - We Share Your Vision 16000 Horizon Way, Suite 100 | Mt. Laurel, NJ 08054 800.885.8886 x128 | msteele(a)telvue.com | http://www.telvue.com twitter: http://twitter.com/telvue | facebook: https://www.facebook.com/telvue

3 4

ovirt with glusterfs data domain - very slow writing speed on Windows server virtual machine
by lyubomir.grancharov＠bottleshipvfx.com 11 Sep '19

11 Sep '19

Hi folks, I have been experimenting with oVirt cluster based on glusterfs for the past few days. (first-timer). The cluster is up and running and it consists of 4 nodes and has 4 replicas. When I try to deploy Windows Server VM I encounter the following issue: The disk of the VM has ok reading speed ( close to bare metal) but the writing speed is very slow. ( about 10 times slower than it is supposed to be). Can anyone give me any suggestion, please? Thanks in advance! Here are the settings of the glusterfs volume: Volume Name: bottle-volume Type: Replicate Volume ID: 869b8d1e-1266-4820-8dcd-4fea92346b90 Status: Started Snapshot Count: 0 Number of Bricks: 1 x 4 = 4 Transport-type: tcp Bricks: Brick1: cnode01.bottleship.local:/gluster_bricks/brick-cnode01/brick-cnode01 Brick2: cnode02.bottleship.local:/gluster_bricks/brick-cnode02/brick-cnode02 Brick3: cnode03.bottleship.local:/gluster_bricks/brick-cnode03/brick-cnode03 Brick4: cnode04.bottleship.local:/gluster_bricks/brick-cnode04/brick-cnode04 Options Reconfigured: network.ping-timeout: 30 cluster.granular-entry-heal: enable performance.strict-o-direct: on storage.owner-gid: 36 storage.owner-uid: 36 server.event-threads: 4 client.event-threads: 4 cluster.choose-local: off features.shard: on cluster.shd-wait-qlength: 10000 cluster.shd-max-threads: 8 cluster.locking-scheme: granular cluster.data-self-heal-algorithm: full cluster.server-quorum-type: server cluster.quorum-type: auto cluster.eager-lock: enable network.remote-dio: off performance.low-prio-threads: 32 performance.io-cache: off performance.read-ahead: off performance.quick-read: off auth.allow: * user.cifs: off transport.address-family: inet nfs.disable: on performance.client-io-threads: off server.allow-insecure: on Please let me know if you need any more configuration info or hardware specs. best, Lyubo

2 1

Procedure to replace out out of three hyperconverged nodes
by thomas＠hoberg.net 11 Sep '19

11 Sep '19

On my silent Atom based three node Hyperconverged journey I hit upon a snag: Evidently they are too slow for Ansible. The Gluster storage part went all great and perfect on fresh oVirt node images that I had configured to leave an empty partition instead of the standard /dev/sdb, but the HostedEngine setup part would then fail without any log-visible error while the transient VM HostedEngineLocal was supposed to be launched and the Wizard would just show "deployment failed" and go ahead and delete the VM. I then moved the SSD to a more powerful Xeon-D 1541 CPU and after some fiddling with the network (I miss good old eth0!), this also failed the deployment, but also failed to delete the temporary VM image, because that actually turned out to be running: I could even connect to its console and investigate the logs for any clues as to what might have gone wrong (nothing visible): Evidently Ansible was running out of patience just a tiny bit too early. And then I kicked it into high-gear with an i7-7700K again using the same SSD with a working three node Gluster all in sync, which still took what felt like an hour to creep through every step, but got it done, primary node on i7, secondary nodes on Atoms, with full migration capabilities etc. I then had to do some fiddling, because the HostedEngine had configued the Cluster CPU architecture to Skylake-Spectre, but after that I migrated it to an Atom node and was now ready to move the primary to the intended Atom hardware target. But at that point the overlay network has already been configured and evidently it's tied to the device name of the 10Gbit NIC on the i7 workstation and I haven't been able to make it work with the Atom. The Gluster runs fine, but the CPU node is reported "non-operational" and re-installation fails, because the ovirtmgmt network isn't properly configured. That specific issue may be seem way out of what oVirt should support, yet a HA-embedded edge platform may very well see nodes having to be replaced or renewed with as little interruption or downtime as possible, which is why I am asking the larger question: How can you replace a) a failed "burned" node or b) upgrade nodes while maintaining fault tolerance? The distinction in b) would be that it's a planned maneuver during normal operations without downtime. I'd want to do it pretty much like I have been playing with compute nodes, creating new ones, pushing VMs on them, pushing them out to other hosts, removing and replacing them seamlessly... Except that the Gluster nodes are special and much harder to replace, than a pure Gluster storage brick... from what I see I welcome any help - for fixing the network config in my limiping Atom 1:3 cluster - eliminating the need to fiddle with an i7 because of Ansible timing - ensuring long-term operability of a software defined datacenter with changing hardware

2 2

gluster
by mailing-ovirt＠qic.ca 11 Sep '19

11 Sep '19

Hi, I see options to deploy ovirt with gluster during the initial rollout, however I can't seem to find information as to how I can add it following a non gluster initial setup: GlusterFS Version: [N/A] Thanks Simon

4 5

VM crushed and cant shutdown successfully,I shoud to reboot the ovirt-node machine
by zhouhao＠vip.friendtimes.net 11 Sep '19

11 Sep '19

VM crushed and cant shutdown successfully,It always on the way to shutdown，even if I wait it for 10 hours；Everytimes I shoud to reboot the ovirt-node machine This comes up 4 times, It appes both on the ovirt-noe 4.3 and 4.2; This bug appers when my VM is running at high load（seems memory and disk's w/r） My storage domain is local on host; When the bug appers,it could cost my node's io,looks like the picture below；But sometimes it will not The vm's threading on the ovirt-node, the IO ratio is 100% Everytime the vm's process will change to defunct I can not kill it,I had to shutdown the ovirt-node on the engine website ,the vm's status always in the way to shutdown ,even if I wait for it for hours; It either failed or is shutting down and the "power off" cant shutdown the vm too. the vm's error below The ovirt-node's error below ---------------------------------------------------------------- The other infomation about my ovirt-node node-version： node hardware zhouhao(a)vip.friendtimes.net

1 0

Python to retrive the VM-Config from the snapshot
by Sven Achtelik 10 Sep '19

10 Sep '19

Hi All, I'm trying to build a custom solution for Backups using the python. Everything works and I can get my disk copies. The last thing is getting the VM config and I can't find any hint on how to get this done with python. Is there a way to actually get this information out easily ? Somehting like * Grab the wanted vm configuration from the needed snapshot - it'll be under initialization/configuration/data URL = SERVER:PORT/api/vms/VM_ID/snapshots/ID Method = GET (with All-Content:true header) I also have to mention that I'm not a profession programmer and work with what I can find as documentation or code examples. http://ovirt.github.io/ovirt-engine-sdk/4.1/ https://github.com/oVirt/ovirt-engine-sdk/tree/master/sdk/examples If some experienced python programmer could give a hint where to look at that would be great. Thank you, Sven

2 1

Re: Python to retrive the VM-Config from the snapshot
by Strahil 10 Sep '19

10 Sep '19

If the system is running you can use virsh dumpxml. Best Regards, Strahil NikolovOn Sep 10, 2019 16:12, Sven Achtelik <Sven.Achtelik(a)eps.aero> wrote: > > Hi All, > > > > I’m trying to build a custom solution for Backups using the python. Everything works and I can get my disk copies. The last thing is getting the VM config and I can’t find any hint on how to get this done with python. Is there a way to actually get this information out easily ? Somehting like > > Grab the wanted vm configuration from the needed snapshot - it’ll be under initialization/configuration/data > > URL = SERVER:PORT/api/vms/VM_ID/snapshots/ID Method = GET (with All-Content:true header) > > I also have to mention that I’m not a profession programmer and work with what I can find as documentation or code examples. > > http://ovirt.github.io/ovirt-engine-sdk/4.1/ > > https://github.com/oVirt/ovirt-engine-sdk/tree/master/sdk/examples > > If some experienced python programmer could give a hint where to look at that would be great. > > Thank you, > > Sven

1 0

nfs
by mailing-ovirt＠qic.ca 10 Sep '19

10 Sep '19

I`m trying to mount a nfs share. if I manually mount it from ssh, I can access it without issues. However when I do it from the web config, it keeps failing: Not sure how to solve that. Thanks Simon 2019-09-09 09:08:47,601-0400 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:47,610-0400 INFO (jsonrpc/3) [vdsm.api] START repoStats(domains=[u'4f4828c2-8f78-4566-a69d-a37cd73446d4']) from=::1,42394, task_id=6c1eeef7-ff8c-4d0d-be85-ced03aae4bc2 (api:48) 2019-09-09 09:08:47,610-0400 INFO (jsonrpc/3) [vdsm.api] FINISH repoStats return={u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.00101971', 'lastCheck': '0.4', 'valid': True}} from=::1,42394, task_id=6c1eeef7-ff8c-4d0d-be85-ced03aae4bc2 (api:54) 2019-09-09 09:08:47,611-0400 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:47,839-0400 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:47,846-0400 INFO (jsonrpc/2) [api.host] START getCapabilities() from=::1,42394 (api:48) 2019-09-09 09:08:48,149-0400 INFO (jsonrpc/2) [root] managedvolume not supported: Managed Volume Not Supported. Missing package os-brick.: ('Cannot import os_brick',) (caps:152) 2019-09-09 09:08:49,212-0400 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.01 seconds (__init__:312) 2019-09-09 09:08:49,263-0400 INFO (jsonrpc/2) [root] /usr/libexec/vdsm/hooks/after_get_caps/50_openstacknet: rc=0 err= (hooks:114) 2019-09-09 09:08:49,497-0400 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:49,657-0400 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:49,665-0400 INFO (jsonrpc/7) [vdsm.api] START repoStats(domains=[u'4f4828c2-8f78-4566-a69d-a37cd73446d4']) from=::1,42394, task_id=14b596bf-bc1c-412b-b1fc-8d53f17d88f7 (api:48) 2019-09-09 09:08:49,666-0400 INFO (jsonrpc/7) [vdsm.api] FINISH repoStats return={u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.00101971', 'lastCheck': '2.4', 'valid': True}} from=::1,42394, task_id=14b596bf-bc1c-412b-b1fc-8d53f17d88f7 (api:54) 2019-09-09 09:08:49,667-0400 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:49,800-0400 INFO (jsonrpc/2) [root] /usr/libexec/vdsm/hooks/after_get_caps/openstacknet_utils.py: rc=0 err= (hooks:114) 2019-09-09 09:08:50,464-0400 INFO (jsonrpc/2) [root] /usr/libexec/vdsm/hooks/after_get_caps/ovirt_provider_ovn_hook: rc=0 err= (hooks:114) 2019-09-09 09:08:50,467-0400 INFO (jsonrpc/2) [api.host] FINISH getCapabilities return={'status': {'message': 'Done', 'code': 0}, 'info': {u'HBAInventory': {u'iSCSI': [{u'InitiatorName': u'iqn.1994-05.com.redhat:d8c85fc0ab85'}], u'FC': []}, u'packages2': {u'kernel': {u'release': u'957.27.2.el7.x86_64', u'version': u'3.10.0'}, u'spice-server': {u'release': u'6.el7_6.1', u'version': u'0.14.0'}, u'librbd1': {u'release': u'4.el7', u'version': u'10.2.5'}, u'vdsm': {u'release': u'1.el7', u'version': u'4.30.24'}, u'qemu-kvm': {u'release': u'18.el7_6.7.1', u'version': u'2.12.0'}, u'openvswitch': {u'release': u'4.el7', u'version': u'2.11.0'}, u'libvirt': {u'release': u'10.el7_6.12', u'version': u'4.5.0'}, u'ovirt-hosted-engine-ha': {u'release': u'1.el7', u'version': u'2.3.3'}, u'qemu-img': {u'release': u'18.el7_6.7.1', u'version': u'2.12.0'}, u'mom': {u'release': u'1.el7.centos', u'version': u'0.5.12'}, u'glusterfs-cli': {u'release': u'1.el7', u'version': u'6.5'}}, u'numaNodeDistance': {u'1': [20, 10], u'0': [10, 20]}, u'cpuModel': u'Intel(R) Xeon(R) CPU E5-2640 v2 @ 2.00GHz', u'nestedVirtualization': False, u'liveMerge': u'true', u'hooks': {u'before_vm_start': {u'50_hostedengine': {u'md5': u'95c810cdcfe4195302a59574a5148289'}, u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'after_network_setup': {u'30_ethtool_options': {u'md5': u'ce1fbad7aa0389e3b06231219140bf0d'}}, u'after_vm_destroy': {u'delete_vhostuserclient_hook': {u'md5': u'c2f279cc9483a3f842f6c29df13994c1'}, u'50_vhostmd': {u'md5': u'bdf4802c0521cf1bae08f2b90a9559cf'}}, u'after_vm_start': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'ea0a5a715da8c1badbcda28e8b8fa00e'}}, u'after_device_migrate_destination': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'6226fbc4d1602994828a3904fc1b875d'}}, u'before_device_migrate_destination': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}}, u'after_device_create': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'6226fbc4d1602994828a3904fc1b875d'}}, u'after_nic_hotplug': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'6226fbc4d1602994828a3904fc1b875d'}}, u'before_vm_migrate_destination': {u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'before_vm_dehibernate': {u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'before_nic_hotplug': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}, u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'ec8d5d7ba063a109f749cd63c7a2b58d'}, u'20_ovirt_provider_ovn_vhostuser_hook': {u'md5': u'a8af653b7386c138b2e6e9738bd6b62c'}, u'10_ovirt_provider_ovn_hook': {u'md5': u'73822988042847bab1ea832a6b9fa837'}}, u'after_get_caps': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'5c3a9ab6e06e039bdd220c0216e45809'}, u'ovirt_provider_ovn_hook': {u'md5': u'1a6ce22288fc2466fcc339da252ef624'}}, u'before_network_setup': {u'50_fcoe': {u'md5': u'28c352339c8beef1e1b05c67d106d062'}}, u'before_device_create': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}, u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'ec8d5d7ba063a109f749cd63c7a2b58d'}, u'20_ovirt_provider_ovn_vhostuser_hook': {u'md5': u'a8af653b7386c138b2e6e9738bd6b62c'}, u'10_ovirt_provider_ovn_hook': {u'md5': u'73822988042847bab1ea832a6b9fa837'}}}, u'supportsIPv6': True, u'realtimeKernel': False, u'vmTypes': [u'kvm'], u'liveSnapshot': u'true', u'cpuThreads': u'32', u'kdumpStatus': 0, u'deferred_preallocation': True, u'networks': {u'ovirtmgmt': {u'iface': u'ovirtmgmt', u'ipv6autoconf': True, u'addr': u'192.168.20.101', u'dhcpv6': False, u'ipv6addrs': [], u'switch': u'legacy', u'bridged': True, u'southbound': u'em1', u'dhcpv4': True, u'netmask': u'255.255.255.0', u'ipv4defaultroute': True, u'stp': u'off', u'ipv4addrs': [u'192.168.20.101/24'], u'mtu': u'1500', u'ipv6gateway': u'::', u'gateway': u'192.168.20.1', u'ports': [u'em1']}}, u'kernelArgs': u'BOOT_IMAGE=/vmlinuz-3.10.0-957.27.2.el7.x86_64 root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rhgb quiet LANG=en_US.UTF-8', u'domain_versions': [0, 2, 3, 4, 5], u'bridges': {u'ovirtmgmt': {u'ipv6autoconf': True, u'addr': u'192.168.20.101', u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': True, u'netmask': u'255.255.255.0', u'ipv4defaultroute': True, u'stp': u'off', u'ipv4addrs': [u'192.168.20.101/24'], u'ipv6gateway': u'::', u'gateway': u'192.168.20.1', u'opts': {u'multicast_last_member_count': u'2', u'vlan_protocol': u'0x8100', u'hash_elasticity': u'4', u'multicast_query_response_interval': u'1000', u'group_fwd_mask': u'0x0', u'multicast_snooping': u'1', u'multicast_startup_query_interval': u'3125', u'hello_timer': u'0', u'multicast_querier_interval': u'25500', u'max_age': u'2000', u'hash_max': u'512', u'stp_state': u'0', u'topology_change_detected': u'0', u'priority': u'32768', u'multicast_igmp_version': u'2', u'multicast_membership_interval': u'26000', u'root_path_cost': u'0', u'root_port': u'0', u'multicast_stats_enabled': u'0', u'multicast_startup_query_count': u'2', u'nf_call_iptables': u'0', u'vlan_stats_enabled': u'0', u'hello_time': u'200', u'topology_change': u'0', u'bridge_id': u'8000.f8bc1234c8a8', u'topology_change_timer': u'0', u'ageing_time': u'30000', u'nf_call_ip6tables': u'0', u'multicast_mld_version': u'1', u'gc_timer': u'6672', u'root_id': u'8000.f8bc1234c8a8', u'nf_call_arptables': u'0', u'group_addr': u'1:80:c2:0:0:0', u'multicast_last_member_interval': u'100', u'default_pvid': u'1', u'multicast_query_interval': u'12500', u'multicast_query_use_ifaddr': u'0', u'tcn_timer': u'0', u'multicast_router': u'1', u'vlan_filtering': u'0', u'multicast_querier': u'0', u'forward_delay': u'0'}, u'ports': [u'em1']}}, u'uuid': u'4c4c4544-0046-3310-8034-c8c04f4a3032', u'onlineCpus': u'0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,1,3,5,7,9,11,13,15,17,19,21,23, 25,27,29,31', u'nameservers': [u'192.168.168.1'], u'nics': {u'em4': {u'ipv6autoconf': True, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:ab', u'ipv6gateway': u'::', u'gateway': u''}, u'idrac': {u'ipv6autoconf': True, u'addr': u'', u'speed': 0, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:37:d8:b1', u'ipv6gateway': u'::', u'gateway': u''}, u'em1': {u'ipv6autoconf': False, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:a8', u'ipv6gateway': u'::', u'gateway': u''}, u'em3': {u'ipv6autoconf': True, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:aa', u'ipv6gateway': u'::', u'gateway': u''}, u'em2': {u'ipv6autoconf': True, u'addr': u'', u'speed': 100, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:a9', u'ipv6gateway': u'::', u'gateway': u''}}, u'software_revision': u'1', u'hostdevPassthrough': u'false', u'clusterLevels': [u'4.1', u'4.2', u'4.3'], u'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,cl flush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,c onstant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfm perf,eagerfpu,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16 ,xtpr,pdcm,pcid,dca,sse4_1,sse4_2,x2apic,popcnt,tsc_deadline_timer,aes,xsave ,avx,f16c,rdrand,lahf_lm,ssbd,ibrs,ibpb,stibp,tpr_shadow,vnmi,flexpriority,e pt,vpid,fsgsbase,smep,erms,xsaveopt,dtherm,ida,arat,pln,pts,md_clear,spec_ct rl,intel_stibp,flush_l1d,model_Opteron_G2,model_kvm32,model_SandyBridge-IBRS ,model_Opteron_G1,model_core2duo,model_pentium,model_Westmere-IBRS,model_qem u64,model_kvm64,model_coreduo,model_Nehalem,model_Nehalem-IBRS,model_IvyBrid ge,model_Westmere,model_SandyBridge,model_IvyBridge-IBRS,model_qemu32,model_ Penryn,model_pentium2,model_pentium3,model_Conroe,model_486', u'kernelFeatures': {u'IBRS': 0, u'RETP': 1, u'SSBD': 3, u'PTI': 1}, u'ISCSIInitiatorName': u'iqn.1994-05.com.redhat:d8c85fc0ab85', u'netConfigDirty': u'False', u'supported_block_sizes': [512], u'selinux': {u'mode': u'1'}, u'autoNumaBalancing': 1, u'reservedMem': u'321', u'bondings': {}, u'software_version': u'4.30.24', u'supportedENGINEs': [u'4.1', u'4.2', u'4.3'], u'vncEncrypted': False, u'backupEnabled': False, u'cpuSpeed': u'1199.951', u'numaNodes': {u'1': {u'totalMemory': u'32768', u'cpus': [1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31]}, u'0': {u'totalMemory': u'32704', u'cpus': [0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30]}}, u'cpuSockets': u'2', u'vlans': {}, u'version_name': u'Snow Man', 'lastClientIface': 'lo', u'cpuCores': u'16', u'hostedEngineDeployed': True, u'hugepages': [1048576, 2048], u'guestOverhead': u'65', u'additionalFeatures': [u'libgfapi_supported'], u'openstack_binding_host_ids': {u'OPENSTACK_OVN': u'baremetal1.ledinfo.com', u'OPEN_VSWITCH': u'baremetal1.ledinfo.com', u'OVIRT_PROVIDER_OVN': u'c83bc734-603c-4a99-a5b9-5833ef1d90c7'}, u'kvmEnabled': u'true', u'memSize': u'64195', u'emulatedMachines': [u'pc-i440fx-rhel7.1.0', u'pc-q35-rhel7.3.0', u'rhel6.3.0', u'pc-i440fx-rhel7.5.0', u'pc-i440fx-rhel7.0.0', u'rhel6.1.0', u'pc-q35-rhel7.6.0', u'pc-i440fx-rhel7.4.0', u'rhel6.6.0', u'pc-q35-rhel7.5.0', u'rhel6.2.0', u'pc', u'pc-i440fx-rhel7.3.0', u'q35', u'pc-i440fx-rhel7.2.0', u'rhel6.4.0', u'pc-q35-rhel7.4.0', u'pc-i440fx-rhel7.6.0', u'rhel6.0.0', u'rhel6.5.0'], u'rngSources': [u'hwrng', u'random'], u'operatingSystem': {u'release': u'6.1810.2.el7.centos', u'pretty_name': u'CentOS Linux 7 (Core)', u'version': u'7', u'name': u'RHEL'}}} from=::1,42394 (api:54) 2019-09-09 09:08:50,493-0400 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.getCapabilities succeeded in 2.65 seconds (__init__:312) 2019-09-09 09:08:51,518-0400 INFO (jsonrpc/5) [api.host] START getAllVmStats() from=::1,42344 (api:48) 2019-09-09 09:08:51,518-0400 INFO (jsonrpc/5) [api.host] FINISH getAllVmStats return={'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} from=::1,42344 (api:54) 2019-09-09 09:08:51,519-0400 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:51,551-0400 INFO (jsonrpc/3) [vdsm.api] START connectStorageServer(domType=1, spUUID=u'00000000-0000-0000-0000-000000000000', conList=[{u'tpgt': u'1', u'id': u'00000000-0000-0000-0000-000000000000', u'connection': u'192.168.20.125:/ovirt', u'iqn': u'', u'user': u'', u'ipv6_enabled': u'false', u'protocol_version': u'auto', u'password': '********', u'port': u''}], options=None) from=::ffff:192.168.20.116,41974, flow_id=1b259177-b83c-4066-a045-b3a9db670a1e, task_id=3d0e32c4-288b-4be4-98ef-90fb676b57f2 (api:48) 2019-09-09 09:08:51,556-0400 INFO (jsonrpc/3) [storage.StorageServer.MountConnection] Creating directory u'/rhev/data-center/mnt/192.168.20.125:_ovirt' (storageServer:168) 2019-09-09 09:08:51,556-0400 INFO (jsonrpc/3) [storage.fileUtils] Creating directory: /rhev/data-center/mnt/192.168.20.125:_ovirt mode: None (fileUtils:199) 2019-09-09 09:08:51,556-0400 INFO (jsonrpc/3) [storage.Mount] mounting 192.168.20.125:/ovirt at /rhev/data-center/mnt/192.168.20.125:_ovirt (mount:204) 2019-09-09 09:08:51,684-0400 INFO (jsonrpc/3) [IOProcessClient] (Global) Starting client (__init__:308) 2019-09-09 09:08:51,705-0400 INFO (ioprocess/9120) [IOProcess] (Global) Starting ioprocess (__init__:434) 2019-09-09 09:08:51,739-0400 INFO (jsonrpc/3) [vdsm.api] FINISH connectStorageServer return={'statuslist': [{'status': 0, 'id': u'00000000-0000-0000-0000-000000000000'}]} from=::ffff:192.168.20.116,41974, flow_id=1b259177-b83c-4066-a045-b3a9db670a1e, task_id=3d0e32c4-288b-4be4-98ef-90fb676b57f2 (api:54) 2019-09-09 09:08:51,741-0400 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call StoragePool.connectStorageServer succeeded in 0.19 seconds (__init__:312) 2019-09-09 09:08:51,992-0400 INFO (jsonrpc/4) [vdsm.api] START connectStorageServer(domType=1, spUUID=u'00000000-0000-0000-0000-000000000000', conList=[{u'tpgt': u'1', u'id': u'1280de25-c248-41d3-8552-50ef964d0ee0', u'connection': u'192.168.20.125:/ovirt', u'iqn': u'', u'user': u'', u'ipv6_enabled': u'false', u'protocol_version': u'auto', u'password': '********', u'port': u''}], options=None) from=::ffff:192.168.20.116,41974, flow_id=360af944, task_id=44612f73-58c9-4597-9a29-c1e65e149c0a (api:48) 2019-09-09 09:08:51,999-0400 INFO (jsonrpc/4) [vdsm.api] FINISH connectStorageServer return={'statuslist': [{'status': 0, 'id': u'1280de25-c248-41d3-8552-50ef964d0ee0'}]} from=::ffff:192.168.20.116,41974, flow_id=360af944, task_id=44612f73-58c9-4597-9a29-c1e65e149c0a (api:54) 2019-09-09 09:08:52,000-0400 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call StoragePool.connectStorageServer succeeded in 0.01 seconds (__init__:312) 2019-09-09 09:08:52,010-0400 INFO (jsonrpc/6) [vdsm.api] START createStorageDomain(storageType=1, sdUUID=u'785a9a50-96c0-4cac-9c80-d0e2dd5bd162', domainName=u'ovirt-omv', typeSpecificArg=u'192.168.20.125:/ovirt', domClass=1, domVersion=u'5', block_size=512, max_hosts=250, options=None) from=::ffff:192.168.20.116,41974, flow_id=360af944, task_id=62008086-97f8-4986-b329-cc193b84fd95 (api:48) 2019-09-09 09:08:52,727-0400 WARN (jsonrpc/6) [storage.LVM] Reloading VGs failed (vgs=[u'785a9a50-96c0-4cac-9c80-d0e2dd5bd162'] rc=5 out=[] err=[' Volume group "785a9a50-96c0-4cac-9c80-d0e2dd5bd162" not found', ' Cannot process volume group 785a9a50-96c0-4cac-9c80-d0e2dd5bd162']) (lvm:470) 2019-09-09 09:08:52,731-0400 INFO (itmap/0) [IOProcessClient] (/baremetal3.ledinfo.com:_export) Starting client (__init__:308) 2019-09-09 09:08:52,751-0400 INFO (ioprocess/9141) [IOProcess] (/baremetal3.ledinfo.com:_export) Starting ioprocess (__init__:434) 2019-09-09 09:08:52,753-0400 INFO (itmap/1) [IOProcessClient] (/baremetal3.ledinfo.com:_import) Starting client (__init__:308) 2019-09-09 09:08:52,777-0400 INFO (itmap/2) [IOProcessClient] (/baremetal3.ledinfo.com:_ISO) Starting client (__init__:308) 2019-09-09 09:08:52,783-0400 INFO (ioprocess/9150) [IOProcess] (/baremetal3.ledinfo.com:_import) Starting ioprocess (__init__:434) 2019-09-09 09:08:52,802-0400 INFO (itmap/3) [IOProcessClient] (/192.168.20.125:_ovirt) Starting client (__init__:308) 2019-09-09 09:08:52,822-0400 INFO (ioprocess/9157) [IOProcess] (/baremetal3.ledinfo.com:_ISO) Starting ioprocess (__init__:434) 2019-09-09 09:08:52,828-0400 INFO (ioprocess/9160) [IOProcess] (/192.168.20.125:_ovirt) Starting ioprocess (__init__:434) 2019-09-09 09:08:52,831-0400 INFO (jsonrpc/6) [storage.StorageDomain] sdUUID=785a9a50-96c0-4cac-9c80-d0e2dd5bd162 domainName=ovirt-omv remotePath=192.168.20.125:/ovirt domClass=1, block_size=512, alignment=1048576 (nfsSD:86) 2019-09-09 09:08:52,866-0400 INFO (jsonrpc/6) [IOProcessClient] (785a9a50-96c0-4cac-9c80-d0e2dd5bd162) Starting client (__init__:308) 2019-09-09 09:08:52,889-0400 INFO (ioprocess/9174) [IOProcess] (785a9a50-96c0-4cac-9c80-d0e2dd5bd162) Starting ioprocess (__init__:434) 2019-09-09 09:08:52,895-0400 INFO (jsonrpc/6) [storage.StorageDomain] Creating domain metadata directory u'/rhev/data-center/mnt/192.168.20.125:_ovirt/785a9a50-96c0-4cac-9c80-d0e2dd 5bd162/dom_md' (fileSD:453) 2019-09-09 09:08:52,926-0400 INFO (jsonrpc/6) [storage.xlease] Formatting index for lockspace u'785a9a50-96c0-4cac-9c80-d0e2dd5bd162' (version=1) (xlease:691) 2019-09-09 09:08:53,047-0400 INFO (jsonrpc/6) [storage.StorageDomain] Creating domain images directory u'/rhev/data-center/mnt/192.168.20.125:_ovirt/785a9a50-96c0-4cac-9c80-d0e2dd 5bd162/images' (nfsSD:110) 2019-09-09 09:08:53,067-0400 INFO (jsonrpc/6) [storage.StorageDomain] Removing remnants of deleted images [] (fileSD:747) 2019-09-09 09:08:53,072-0400 ERROR (jsonrpc/6) [storage.initSANLock] Cannot initialize SANLock for domain 785a9a50-96c0-4cac-9c80-d0e2dd5bd162 (clusterlock:258) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/clusterlock.py", line 249, in initSANLock sdUUID, idsPath, align=alignment, sector=block_size) File "/usr/lib/python2.7/site-packages/vdsm/storage/compat.py", line 52, in decorator return func(*args, **kwargs) SanlockException: (19, 'Sanlock lockspace write failure', 'No such device') 2019-09-09 09:08:53,073-0400 INFO (jsonrpc/6) [vdsm.api] FINISH createStorageDomain error=Could not initialize cluster lock: () from=::ffff:192.168.20.116,41974, flow_id=360af944, task_id=62008086-97f8-4986-b329-cc193b84fd95 (api:52) 2019-09-09 09:08:53,074-0400 ERROR (jsonrpc/6) [storage.TaskManager.Task] (Task='62008086-97f8-4986-b329-cc193b84fd95') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "<string>", line 2, in createStorageDomain File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 2619, in createStorageDomain storageType, domVersion, block_size, alignment) File "/usr/lib/python2.7/site-packages/vdsm/storage/nfsSD.py", line 120, in create fsd.initSPMlease() File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 918, in initSPMlease return self._manifest.initDomainLock() File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 609, in initDomainLock self._domainLock.initLock(self.getDomainLease()) File "/usr/lib/python2.7/site-packages/vdsm/storage/clusterlock.py", line 303, in initLock block_size=self._block_size) File "/usr/lib/python2.7/site-packages/vdsm/storage/clusterlock.py", line 259, in initSANLock raise se.ClusterLockInitError() ClusterLockInitError: Could not initialize cluster lock: () 2019-09-09 09:08:53,075-0400 INFO (jsonrpc/6) [storage.TaskManager.Task] (Task='62008086-97f8-4986-b329-cc193b84fd95') aborting: Task is aborted: 'Could not initialize cluster lock: ()' - code 701 (task:1181) 2019-09-09 09:08:53,076-0400 ERROR (jsonrpc/6) [storage.Dispatcher] FINISH createStorageDomain error=Could not initialize cluster lock: () (dispatcher:83) 2019-09-09 09:08:53,077-0400 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call StorageDomain.create failed (error 701) in 1.07 seconds (__init__:312) 2019-09-09 09:08:53,425-0400 INFO (jsonrpc/0) [api.host] START getStats() from=::ffff:192.168.20.116,41974 (api:48) 2019-09-09 09:08:53,439-0400 INFO (jsonrpc/1) [vdsm.api] START disconnectStorageServer(domType=1, spUUID=u'00000000-0000-0000-0000-000000000000', conList=[{u'tpgt': u'1', u'id': u'1280de25-c248-41d3-8552-50ef964d0ee0', u'connection': u'192.168.20.125:/ovirt', u'iqn': u'', u'user': u'', u'ipv6_enabled': u'false', u'protocol_version': u'auto', u'password': '********', u'port': u''}], options=None) from=::ffff:192.168.20.116,41974, flow_id=e003809f-c7ef-4105-ac27-48be39b5c7b1, task_id=da5b627a-de84-4083-9454-aaecd6bb8aed (api:48) 2019-09-09 09:08:53,440-0400 INFO (jsonrpc/1) [storage.Mount] unmounting /rhev/data-center/mnt/192.168.20.125:_ovirt (mount:212) 2019-09-09 09:08:53,478-0400 INFO (jsonrpc/0) [vdsm.api] START repoStats(domains=()) from=::ffff:192.168.20.116,41974, task_id=42f53132-6ef6-4aff-9ab3-91d19c24bd93 (api:48) 2019-09-09 09:08:53,479-0400 INFO (jsonrpc/0) [vdsm.api] FINISH repoStats return={u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.00104514', 'lastCheck': '3.4', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000798507', 'lastCheck': '3.5', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000768283', 'lastCheck': '3.3', 'valid': True}} from=::ffff:192.168.20.116,41974, task_id=42f53132-6ef6-4aff-9ab3-91d19c24bd93 (api:54) 2019-09-09 09:08:53,481-0400 INFO (jsonrpc/0) [vdsm.api] START multipath_health() from=::ffff:192.168.20.116,41974, task_id=34ab20bd-b1ce-4bbe-ad1c-af0e78d38b1c (api:48) 2019-09-09 09:08:53,481-0400 INFO (jsonrpc/0) [vdsm.api] FINISH multipath_health return={} from=::ffff:192.168.20.116,41974, task_id=34ab20bd-b1ce-4bbe-ad1c-af0e78d38b1c (api:54) 2019-09-09 09:08:53,523-0400 INFO (jsonrpc/0) [api.host] FINISH getStats return={'status': {'message': 'Done', 'code': 0}, 'info': {'cpuStatistics': {'24': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.53'}, '25': {'cpuUser': '0.46', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.47'}, '26': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '27': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '20': {'cpuUser': '2.06', 'nodeIndex': 0, 'cpuSys': '0.40', 'cpuIdle': '97.54'}, '21': {'cpuUser': '0.99', 'nodeIndex': 1, 'cpuSys': '0.33', 'cpuIdle': '98.68'}, '22': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '23': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.74'}, '28': {'cpuUser': '0.99', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '98.88'}, '29': {'cpuUser': '0.46', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.47'}, '1': {'cpuUser': '2.85', 'nodeIndex': 1, 'cpuSys': '1.66', 'cpuIdle': '95.49'}, '0': {'cpuUser': '0.46', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.41'}, '3': {'cpuUser': '0.73', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.20'}, '2': {'cpuUser': '0.33', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.47'}, '5': {'cpuUser': '0.20', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.73'}, '4': {'cpuUser': '0.99', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '98.68'}, '7': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.74'}, '6': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '99.33'}, '9': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '8': {'cpuUser': '0.93', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '98.80'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.53'}, '13': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '12': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '15': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '14': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '17': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '16': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.87'}, '19': {'cpuUser': '1.79', 'nodeIndex': 1, 'cpuSys': '0.46', 'cpuIdle': '97.75'}, '18': {'cpuUser': '8.69', 'nodeIndex': 0, 'cpuSys': '1.72', 'cpuIdle': '89.59'}, '31': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '30': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.80'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31474'}, '0': {'memPercent': 7, 'memFree': '30445'}}, 'memShared': 0, 'haScore': 3400, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'vmCount': 0, 'memUsed': '3', 'storageDomains': {u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.00104514', 'lastCheck': '3.4', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000798507', 'lastCheck': '3.5', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000768283', 'lastCheck': '3.3', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'idrac': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'idrac', 'tx': '180', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '0', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '0', 'state': 'down'}, 'ovirtmgmt': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '1075026007', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '43585639790', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '618193132', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '618193132', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '0', 'state': 'down'}, 'em4': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em4', 'tx': '192', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034533.476725, 'rx': '38644112', 'state': 'up'}, 'br-int': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'br-int', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '0', 'state': 'down'}, 'em1': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em1', 'tx': '1115341150', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034533.476725, 'rx': '45376217511', 'state': 'up'}, 'genev_sys_6081': {'rxErrors': '0', 'txErrors': '8', 'speed': '1000', 'rxDropped': '0', 'name': 'genev_sys_6081', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '0', 'state': 'up'}, 'em3': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em3', 'tx': '192', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034533.476725, 'rx': '38644112', 'state': 'up'}, 'em2': {'rxErrors': '0', 'txErrors': '0', 'speed': '100', 'rxDropped': '0', 'name': 'em2', 'tx': '0', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034533.476725, 'rx': '16049108', 'state': 'up'}}, 'txDropped': '0', 'anonHugePages': '306', 'ksmPages': 100, 'elapsedTime': '223906.36', 'cpuLoad': '0.14', 'cpuSys': '0.24', 'diskStats': {'/var/log': {'free': '136508'}, '/var/run/vdsm/': {'free': '31856'}, '/tmp': {'free': '136508'}}, 'cpuUserVdsmd': '2.78', 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 63031, 'bootTime': '1567810572', 'haStats': {'active': True, 'configured': True, 'score': 3400, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'multipathHealth': {}, 'rxDropped': '0', 'outgoingVmMigrations': 0, 'swapTotal': 0, 'swapFree': 0, 'hugepages': defaultdict(<type 'dict'>, {1048576: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}, 2048: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}), 'dateTime': '2019-09-09T13:08:53 GMT', 'cpuUser': '0.75', 'memFree': 62775, 'cpuIdle': '99.01', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd': '1.06'}} from=::ffff:192.168.20.116,41974 (api:54) 2019-09-09 09:08:53,534-0400 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.11 seconds (__init__:312) 2019-09-09 09:08:54,001-0400 INFO (jsonrpc/1) [vdsm.api] FINISH disconnectStorageServer return={'statuslist': [{'status': 0, 'id': u'1280de25-c248-41d3-8552-50ef964d0ee0'}]} from=::ffff:192.168.20.116,41974, flow_id=e003809f-c7ef-4105-ac27-48be39b5c7b1, task_id=da5b627a-de84-4083-9454-aaecd6bb8aed (api:54) 2019-09-09 09:08:54,003-0400 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call StoragePool.disconnectStorageServer succeeded in 0.57 seconds (__init__:312) 2019-09-09 09:08:54,888-0400 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:54,895-0400 INFO (jsonrpc/2) [api.host] START getStats() from=::1,42394 (api:48) 2019-09-09 09:08:54,946-0400 INFO (jsonrpc/2) [vdsm.api] START repoStats(domains=()) from=::1,42394, task_id=f1acae4b-2563-4335-96c7-8b5097aa4127 (api:48) 2019-09-09 09:08:54,947-0400 INFO (jsonrpc/2) [vdsm.api] FINISH repoStats return={u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.00104514', 'lastCheck': '4.8', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000798507', 'lastCheck': '5.0', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000768283', 'lastCheck': '4.8', 'valid': True}} from=::1,42394, task_id=f1acae4b-2563-4335-96c7-8b5097aa4127 (api:54) 2019-09-09 09:08:54,948-0400 INFO (jsonrpc/2) [vdsm.api] START multipath_health() from=::1,42394, task_id=58483dc0-2a50-409a-b991-16dbb57360a7 (api:48) 2019-09-09 09:08:54,949-0400 INFO (jsonrpc/2) [vdsm.api] FINISH multipath_health return={} from=::1,42394, task_id=58483dc0-2a50-409a-b991-16dbb57360a7 (api:54) 2019-09-09 09:08:54,977-0400 INFO (jsonrpc/2) [api.host] FINISH getStats return={'status': {'message': 'Done', 'code': 0}, 'info': {'cpuStatistics': {'24': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.53'}, '25': {'cpuUser': '0.46', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.47'}, '26': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '27': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '20': {'cpuUser': '2.06', 'nodeIndex': 0, 'cpuSys': '0.40', 'cpuIdle': '97.54'}, '21': {'cpuUser': '0.99', 'nodeIndex': 1, 'cpuSys': '0.33', 'cpuIdle': '98.68'}, '22': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '23': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.74'}, '28': {'cpuUser': '0.99', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '98.88'}, '29': {'cpuUser': '0.46', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.47'}, '1': {'cpuUser': '2.85', 'nodeIndex': 1, 'cpuSys': '1.66', 'cpuIdle': '95.49'}, '0': {'cpuUser': '0.46', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.41'}, '3': {'cpuUser': '0.73', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.20'}, '2': {'cpuUser': '0.33', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.47'}, '5': {'cpuUser': '0.20', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.73'}, '4': {'cpuUser': '0.99', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '98.68'}, '7': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.74'}, '6': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '99.33'}, '9': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '8': {'cpuUser': '0.93', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '98.80'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.53'}, '13': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '12': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '15': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '14': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '17': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '16': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.87'}, '19': {'cpuUser': '1.79', 'nodeIndex': 1, 'cpuSys': '0.46', 'cpuIdle': '97.75'}, '18': {'cpuUser': '8.69', 'nodeIndex': 0, 'cpuSys': '1.72', 'cpuIdle': '89.59'}, '31': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '30': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.80'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31474'}, '0': {'memPercent': 7, 'memFree': '30445'}}, 'memShared': 0, 'haScore': 3400, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'vmCount': 0, 'memUsed': '3', 'storageDomains': {u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.00104514', 'lastCheck': '4.8', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000798507', 'lastCheck': '5.0', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000768283', 'lastCheck': '4.8', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'idrac': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'idrac', 'tx': '180', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '0', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '0', 'state': 'down'}, 'ovirtmgmt': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '1075046072', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '43586697397', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '618194231', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '618194231', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '0', 'state': 'down'}, 'em4': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em4', 'tx': '192', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034534.944025, 'rx': '38644112', 'state': 'up'}, 'br-int': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'br-int', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '0', 'state': 'down'}, 'em1': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em1', 'tx': '1115653922', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034534.944025, 'rx': '45377347900', 'state': 'up'}, 'genev_sys_6081': {'rxErrors': '0', 'txErrors': '8', 'speed': '1000', 'rxDropped': '0', 'name': 'genev_sys_6081', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '0', 'state': 'up'}, 'em3': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em3', 'tx': '192', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034534.944025, 'rx': '38644176', 'state': 'up'}, 'em2': {'rxErrors': '0', 'txErrors': '0', 'speed': '100', 'rxDropped': '0', 'name': 'em2', 'tx': '0', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034534.944025, 'rx': '16049108', 'state': 'up'}}, 'txDropped': '0', 'anonHugePages': '306', 'ksmPages': 100, 'elapsedTime': '223907.83', 'cpuLoad': '0.14', 'cpuSys': '0.24', 'diskStats': {'/var/log': {'free': '136508'}, '/var/run/vdsm/': {'free': '31856'}, '/tmp': {'free': '136508'}}, 'cpuUserVdsmd': '2.78', 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 63031, 'bootTime': '1567810572', 'haStats': {'active': True, 'configured': True, 'score': 3400, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'multipathHealth': {}, 'rxDropped': '0', 'outgoingVmMigrations': 0, 'swapTotal': 0, 'swapFree': 0, 'hugepages': defaultdict(<type 'dict'>, {1048576: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}, 2048: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}), 'dateTime': '2019-09-09T13:08:54 GMT', 'cpuUser': '0.75', 'memFree': 62775, 'cpuIdle': '99.01', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd': '1.06'}} from=::1,42394 (api:54) 2019-09-09 09:08:54,986-0400 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.09 seconds (__init__:312) 2019-09-09 09:08:55,842-0400 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:55,850-0400 INFO (jsonrpc/3) [api.virt] START getStats() from=::1,42394, vmId=aeed8192-5313-45d6-b9f5-fb2db09fb646 (api:48) 2019-09-09 09:08:55,850-0400 INFO (jsonrpc/3) [api] FINISH getStats error=Virtual machine does not exist: {'vmId': u'aeed8192-5313-45d6-b9f5-fb2db09fb646'} (api:129) 2019-09-09 09:08:55,850-0400 INFO (jsonrpc/3) [api.virt] FINISH getStats return={'status': {'message': "Virtual machine does not exist: {'vmId': u'aeed8192-5313-45d6-b9f5-fb2db09fb646'}", 'code': 1}} from=::1,42394, vmId=aeed8192-5313-45d6-b9f5-fb2db09fb646 (api:54) 2019-09-09 09:08:55,851-0400 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call VM.getStats failed (error 1) in 0.00 seconds (__init__:312) 2019-09-09 09:08:57,526-0400 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:57,534-0400 INFO (jsonrpc/6) [api.host] START getCapabilities() from=::1,42394 (api:48) 2019-09-09 09:08:57,627-0400 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:57,636-0400 INFO (jsonrpc/1) [vdsm.api] START repoStats(domains=[u'4f4828c2-8f78-4566-a69d-a37cd73446d4']) from=::1,42394, task_id=6da33ef3-de09-48a3-8ba9-6c31d5e2942b (api:48) 2019-09-09 09:08:57,636-0400 INFO (jsonrpc/1) [vdsm.api] FINISH repoStats return={u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000798507', 'lastCheck': '0.4', 'valid': True}} from=::1,42394, task_id=6da33ef3-de09-48a3-8ba9-6c31d5e2942b (api:54) 2019-09-09 09:08:57,637-0400 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:57,870-0400 INFO (jsonrpc/6) [root] managedvolume not supported: Managed Volume Not Supported. Missing package os-brick.: ('Cannot import os_brick',) (caps:152) 2019-09-09 09:08:58,884-0400 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:58,990-0400 INFO (jsonrpc/6) [root] /usr/libexec/vdsm/hooks/after_get_caps/50_openstacknet: rc=0 err= (hooks:114) 2019-09-09 09:08:59,048-0400 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:59,055-0400 INFO (jsonrpc/5) [vdsm.api] START repoStats(domains=[u'4f4828c2-8f78-4566-a69d-a37cd73446d4']) from=::1,42394, task_id=933c8ee8-2330-46ed-a609-38a519447156 (api:48) 2019-09-09 09:08:59,056-0400 INFO (jsonrpc/5) [vdsm.api] FINISH repoStats return={u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000798507', 'lastCheck': '1.8', 'valid': True}} from=::1,42394, task_id=933c8ee8-2330-46ed-a609-38a519447156 (api:54) 2019-09-09 09:08:59,057-0400 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:59,526-0400 INFO (jsonrpc/6) [root] /usr/libexec/vdsm/hooks/after_get_caps/openstacknet_utils.py: rc=0 err= (hooks:114) 2019-09-09 09:08:59,896-0400 INFO (jsonrpc/3) [api.host] START getAllVmStats() from=::ffff:192.168.20.116,41974 (api:48) 2019-09-09 09:08:59,897-0400 INFO (jsonrpc/3) [api.host] FINISH getAllVmStats return={'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} from=::ffff:192.168.20.116,41974 (api:54) 2019-09-09 09:08:59,898-0400 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:09:00,188-0400 INFO (periodic/0) [vdsm.api] START repoStats(domains=()) from=internal, task_id=aec0525d-393b-49fa-a0d4-96bc19f829e3 (api:48) 2019-09-09 09:09:00,190-0400 INFO (periodic/0) [vdsm.api] FINISH repoStats return={u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000991915', 'lastCheck': '0.1', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000985319', 'lastCheck': '0.2', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000768283', 'lastCheck': '3.1', 'valid': True}} from=internal, task_id=aec0525d-393b-49fa-a0d4-96bc19f829e3 (api:54) 2019-09-09 09:09:00,192-0400 INFO (jsonrpc/6) [root] /usr/libexec/vdsm/hooks/after_get_caps/ovirt_provider_ovn_hook: rc=0 err= (hooks:114) 2019-09-09 09:09:00,194-0400 INFO (jsonrpc/6) [api.host] FINISH getCapabilities return={'status': {'message': 'Done', 'code': 0}, 'info': {u'HBAInventory': {u'iSCSI': [{u'InitiatorName': u'iqn.1994-05.com.redhat:d8c85fc0ab85'}], u'FC': []}, u'packages2': {u'kernel': {u'release': u'957.27.2.el7.x86_64', u'version': u'3.10.0'}, u'spice-server': {u'release': u'6.el7_6.1', u'version': u'0.14.0'}, u'librbd1': {u'release': u'4.el7', u'version': u'10.2.5'}, u'vdsm': {u'release': u'1.el7', u'version': u'4.30.24'}, u'qemu-kvm': {u'release': u'18.el7_6.7.1', u'version': u'2.12.0'}, u'openvswitch': {u'release': u'4.el7', u'version': u'2.11.0'}, u'libvirt': {u'release': u'10.el7_6.12', u'version': u'4.5.0'}, u'ovirt-hosted-engine-ha': {u'release': u'1.el7', u'version': u'2.3.3'}, u'qemu-img': {u'release': u'18.el7_6.7.1', u'version': u'2.12.0'}, u'mom': {u'release': u'1.el7.centos', u'version': u'0.5.12'}, u'glusterfs-cli': {u'release': u'1.el7', u'version': u'6.5'}}, u'numaNodeDistance': {u'1': [20, 10], u'0': [10, 20]}, u'cpuModel': u'Intel(R) Xeon(R) CPU E5-2640 v2 @ 2.00GHz', u'nestedVirtualization': False, u'liveMerge': u'true', u'hooks': {u'before_vm_start': {u'50_hostedengine': {u'md5': u'95c810cdcfe4195302a59574a5148289'}, u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'after_network_setup': {u'30_ethtool_options': {u'md5': u'ce1fbad7aa0389e3b06231219140bf0d'}}, u'after_vm_destroy': {u'delete_vhostuserclient_hook': {u'md5': u'c2f279cc9483a3f842f6c29df13994c1'}, u'50_vhostmd': {u'md5': u'bdf4802c0521cf1bae08f2b90a9559cf'}}, u'after_vm_start': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'ea0a5a715da8c1badbcda28e8b8fa00e'}}, u'after_device_migrate_destination': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'6226fbc4d1602994828a3904fc1b875d'}}, u'before_device_migrate_destination': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}}, u'after_device_create': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'6226fbc4d1602994828a3904fc1b875d'}}, u'after_nic_hotplug': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'6226fbc4d1602994828a3904fc1b875d'}}, u'before_vm_migrate_destination': {u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'before_vm_dehibernate': {u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'before_nic_hotplug': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}, u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'ec8d5d7ba063a109f749cd63c7a2b58d'}, u'20_ovirt_provider_ovn_vhostuser_hook': {u'md5': u'a8af653b7386c138b2e6e9738bd6b62c'}, u'10_ovirt_provider_ovn_hook': {u'md5': u'73822988042847bab1ea832a6b9fa837'}}, u'after_get_caps': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'5c3a9ab6e06e039bdd220c0216e45809'}, u'ovirt_provider_ovn_hook': {u'md5': u'1a6ce22288fc2466fcc339da252ef624'}}, u'before_network_setup': {u'50_fcoe': {u'md5': u'28c352339c8beef1e1b05c67d106d062'}}, u'before_device_create': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}, u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'ec8d5d7ba063a109f749cd63c7a2b58d'}, u'20_ovirt_provider_ovn_vhostuser_hook': {u'md5': u'a8af653b7386c138b2e6e9738bd6b62c'}, u'10_ovirt_provider_ovn_hook': {u'md5': u'73822988042847bab1ea832a6b9fa837'}}}, u'supportsIPv6': True, u'realtimeKernel': False, u'vmTypes': [u'kvm'], u'liveSnapshot': u'true', u'cpuThreads': u'32', u'kdumpStatus': 0, u'deferred_preallocation': True, u'networks': {u'ovirtmgmt': {u'iface': u'ovirtmgmt', u'ipv6autoconf': True, u'addr': u'192.168.20.101', u'dhcpv6': False, u'ipv6addrs': [], u'switch': u'legacy', u'bridged': True, u'southbound': u'em1', u'dhcpv4': True, u'netmask': u'255.255.255.0', u'ipv4defaultroute': True, u'stp': u'off', u'ipv4addrs': [u'192.168.20.101/24'], u'mtu': u'1500', u'ipv6gateway': u'::', u'gateway': u'192.168.20.1', u'ports': [u'em1']}}, u'kernelArgs': u'BOOT_IMAGE=/vmlinuz-3.10.0-957.27.2.el7.x86_64 root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rhgb quiet LANG=en_US.UTF-8', u'domain_versions': [0, 2, 3, 4, 5], u'bridges': {u'ovirtmgmt': {u'ipv6autoconf': True, u'addr': u'192.168.20.101', u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': True, u'netmask': u'255.255.255.0', u'ipv4defaultroute': True, u'stp': u'off', u'ipv4addrs': [u'192.168.20.101/24'], u'ipv6gateway': u'::', u'gateway': u'192.168.20.1', u'opts': {u'multicast_last_member_count': u'2', u'vlan_protocol': u'0x8100', u'hash_elasticity': u'4', u'multicast_query_response_interval': u'1000', u'group_fwd_mask': u'0x0', u'multicast_snooping': u'1', u'multicast_startup_query_interval': u'3125', u'hello_timer': u'0', u'multicast_querier_interval': u'25500', u'max_age': u'2000', u'hash_max': u'512', u'stp_state': u'0', u'topology_change_detected': u'0', u'priority': u'32768', u'multicast_igmp_version': u'2', u'multicast_membership_interval': u'26000', u'root_path_cost': u'0', u'root_port': u'0', u'multicast_stats_enabled': u'0', u'multicast_startup_query_count': u'2', u'nf_call_iptables': u'0', u'vlan_stats_enabled': u'0', u'hello_time': u'200', u'topology_change': u'0', u'bridge_id': u'8000.f8bc1234c8a8', u'topology_change_timer': u'0', u'ageing_time': u'30000', u'nf_call_ip6tables': u'0', u'multicast_mld_version': u'1', u'gc_timer': u'5700', u'root_id': u'8000.f8bc1234c8a8', u'nf_call_arptables': u'0', u'group_addr': u'1:80:c2:0:0:0', u'multicast_last_member_interval': u'100', u'default_pvid': u'1', u'multicast_query_interval': u'12500', u'multicast_query_use_ifaddr': u'0', u'tcn_timer': u'0', u'multicast_router': u'1', u'vlan_filtering': u'0', u'multicast_querier': u'0', u'forward_delay': u'0'}, u'ports': [u'em1']}}, u'uuid': u'4c4c4544-0046-3310-8034-c8c04f4a3032', u'onlineCpus': u'0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,1,3,5,7,9,11,13,15,17,19,21,23, 25,27,29,31', u'nameservers': [u'192.168.168.1'], u'nics': {u'em4': {u'ipv6autoconf': True, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:ab', u'ipv6gateway': u'::', u'gateway': u''}, u'idrac': {u'ipv6autoconf': True, u'addr': u'', u'speed': 0, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:37:d8:b1', u'ipv6gateway': u'::', u'gateway': u''}, u'em1': {u'ipv6autoconf': False, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:a8', u'ipv6gateway': u'::', u'gateway': u''}, u'em3': {u'ipv6autoconf': True, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:aa', u'ipv6gateway': u'::', u'gateway': u''}, u'em2': {u'ipv6autoconf': True, u'addr': u'', u'speed': 100, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:a9', u'ipv6gateway': u'::', u'gateway': u''}}, u'software_revision': u'1', u'hostdevPassthrough': u'false', u'clusterLevels': [u'4.1', u'4.2', u'4.3'], u'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,cl flush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,c onstant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfm perf,eagerfpu,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16 ,xtpr,pdcm,pcid,dca,sse4_1,sse4_2,x2apic,popcnt,tsc_deadline_timer,aes,xsave ,avx,f16c,rdrand,lahf_lm,ssbd,ibrs,ibpb,stibp,tpr_shadow,vnmi,flexpriority,e pt,vpid,fsgsbase,smep,erms,xsaveopt,dtherm,ida,arat,pln,pts,md_clear,spec_ct rl,intel_stibp,flush_l1d,model_Opteron_G2,model_kvm32,model_SandyBridge-IBRS ,model_Opteron_G1,model_core2duo,model_pentium,model_Westmere-IBRS,model_qem u64,model_kvm64,model_coreduo,model_Nehalem,model_Nehalem-IBRS,model_IvyBrid ge,model_Westmere,model_SandyBridge,model_IvyBridge-IBRS,model_qemu32,model_ Penryn,model_pentium2,model_pentium3,model_Conroe,model_486', u'kernelFeatures': {u'IBRS': 0, u'RETP': 1, u'SSBD': 3, u'PTI': 1}, u'ISCSIInitiatorName': u'iqn.1994-05.com.redhat:d8c85fc0ab85', u'netConfigDirty': u'False', u'supported_block_sizes': [512], u'selinux': {u'mode': u'1'}, u'autoNumaBalancing': 1, u'reservedMem': u'321', u'bondings': {}, u'software_version': u'4.30.24', u'supportedENGINEs': [u'4.1', u'4.2', u'4.3'], u'vncEncrypted': False, u'backupEnabled': False, u'cpuSpeed': u'1199.951', u'numaNodes': {u'1': {u'totalMemory': u'32768', u'cpus': [1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31]}, u'0': {u'totalMemory': u'32704', u'cpus': [0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30]}}, u'cpuSockets': u'2', u'vlans': {}, u'version_name': u'Snow Man', 'lastClientIface': 'lo', u'cpuCores': u'16', u'hostedEngineDeployed': True, u'hugepages': [1048576, 2048], u'guestOverhead': u'65', u'additionalFeatures': [u'libgfapi_supported'], u'openstack_binding_host_ids': {u'OPENSTACK_OVN': u'baremetal1.ledinfo.com', u'OPEN_VSWITCH': u'baremetal1.ledinfo.com', u'OVIRT_PROVIDER_OVN': u'c83bc734-603c-4a99-a5b9-5833ef1d90c7'}, u'kvmEnabled': u'true', u'memSize': u'64195', u'emulatedMachines': [u'pc-i440fx-rhel7.1.0', u'pc-q35-rhel7.3.0', u'rhel6.3.0', u'pc-i440fx-rhel7.5.0', u'pc-i440fx-rhel7.0.0', u'rhel6.1.0', u'pc-q35-rhel7.6.0', u'pc-i440fx-rhel7.4.0', u'rhel6.6.0', u'pc-q35-rhel7.5.0', u'rhel6.2.0', u'pc', u'pc-i440fx-rhel7.3.0', u'q35', u'pc-i440fx-rhel7.2.0', u'rhel6.4.0', u'pc-q35-rhel7.4.0', u'pc-i440fx-rhel7.6.0', u'rhel6.0.0', u'rhel6.5.0'], u'rngSources': [u'hwrng', u'random'], u'operatingSystem': {u'release': u'6.1810.2.el7.centos', u'pretty_name': u'CentOS Linux 7 (Core)', u'version': u'7', u'name': u'RHEL'}}} from=::1,42394 (api:54) 2019-09-09 09:09:00,231-0400 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.getCapabilities succeeded in 2.69 seconds (__init__:312) 2019-09-09 09:09:05,021-0400 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:09:05,028-0400 INFO (jsonrpc/0) [api.host] START getStats() from=::1,42394 (api:48) 2019-09-09 09:09:05,077-0400 INFO (jsonrpc/0) [vdsm.api] START repoStats(domains=()) from=::1,42394, task_id=12e8837e-c9cd-45b9-aecb-477f86e12ec5 (api:48) 2019-09-09 09:09:05,078-0400 INFO (jsonrpc/0) [vdsm.api] FINISH repoStats return={u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000991915', 'lastCheck': '5.0', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000985319', 'lastCheck': '5.1', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.00093201', 'lastCheck': '4.8', 'valid': True}} from=::1,42394, task_id=12e8837e-c9cd-45b9-aecb-477f86e12ec5 (api:54) 2019-09-09 09:09:05,079-0400 INFO (jsonrpc/0) [vdsm.api] START multipath_health() from=::1,42394, task_id=42c12b61-4d82-4a06-b692-e11dfbb0bca6 (api:48) 2019-09-09 09:09:05,080-0400 INFO (jsonrpc/0) [vdsm.api] FINISH multipath_health return={} from=::1,42394, task_id=42c12b61-4d82-4a06-b692-e11dfbb0bca6 (api:54) 2019-09-09 09:09:05,106-0400 INFO (jsonrpc/0) [api.host] FINISH getStats return={'status': {'message': 'Done', 'code': 0}, 'info': {'cpuStatistics': {'24': {'cpuUser': '0.46', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.41'}, '25': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '26': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '27': {'cpuUser': '1.66', 'nodeIndex': 1, 'cpuSys': '0.40', 'cpuIdle': '97.94'}, '20': {'cpuUser': '1.66', 'nodeIndex': 0, 'cpuSys': '0.86', 'cpuIdle': '97.48'}, '21': {'cpuUser': '0.73', 'nodeIndex': 1, 'cpuSys': '0.40', 'cpuIdle': '98.87'}, '22': {'cpuUser': '0.93', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '98.80'}, '23': {'cpuUser': '0.93', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '98.94'}, '28': {'cpuUser': '1.00', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '98.73'}, '29': {'cpuUser': '0.60', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.27'}, '1': {'cpuUser': '4.31', 'nodeIndex': 1, 'cpuSys': '4.05', 'cpuIdle': '91.64'}, '0': {'cpuUser': '0.60', 'nodeIndex': 0, 'cpuSys': '0.40', 'cpuIdle': '99.00'}, '3': {'cpuUser': '0.27', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.66'}, '2': {'cpuUser': '7.23', 'nodeIndex': 0, 'cpuSys': '1.06', 'cpuIdle': '91.71'}, '5': {'cpuUser': '0.20', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.67'}, '4': {'cpuUser': '1.99', 'nodeIndex': 0, 'cpuSys': '0.86', 'cpuIdle': '97.15'}, '7': {'cpuUser': '0.27', 'nodeIndex': 1, 'cpuSys': '0.27', 'cpuIdle': '99.46'}, '6': {'cpuUser': '1.13', 'nodeIndex': 0, 'cpuSys': '0.60', 'cpuIdle': '98.27'}, '9': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.80'}, '8': {'cpuUser': '0.80', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '98.87'}, '11': {'cpuUser': '0.66', 'nodeIndex': 1, 'cpuSys': '0.40', 'cpuIdle': '98.94'}, '10': {'cpuUser': '0.33', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '99.40'}, '13': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '12': {'cpuUser': '1.33', 'nodeIndex': 0, 'cpuSys': '0.46', 'cpuIdle': '98.21'}, '15': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '14': {'cpuUser': '0.46', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '99.27'}, '17': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '16': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '19': {'cpuUser': '6.77', 'nodeIndex': 1, 'cpuSys': '1.59', 'cpuIdle': '91.64'}, '18': {'cpuUser': '14.40', 'nodeIndex': 0, 'cpuSys': '3.65', 'cpuIdle': '81.95'}, '31': {'cpuUser': '0.46', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.41'}, '30': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31468'}, '0': {'memPercent': 7, 'memFree': '30418'}}, 'memShared': 0, 'haScore': 3400, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'vmCount': 0, 'memUsed': '3', 'storageDomains': {u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000991915', 'lastCheck': '5.0', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000985319', 'lastCheck': '5.1', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.00093201', 'lastCheck': '4.8', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'idrac': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'idrac', 'tx': '180', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '0', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '0', 'state': 'down'}, 'ovirtmgmt': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '1075063579', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '43586979921', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '618221192', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '618221192', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '0', 'state': 'down'}, 'em4': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em4', 'tx': '192', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034545.075349, 'rx': '38652218', 'state': 'up'}, 'br-int': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'br-int', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '0', 'state': 'down'}, 'em1': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em1', 'tx': '1115671653', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034545.075349, 'rx': '45377642078', 'state': 'up'}, 'genev_sys_6081': {'rxErrors': '0', 'txErrors': '8', 'speed': '1000', 'rxDropped': '0', 'name': 'genev_sys_6081', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '0', 'state': 'up'}, 'em3': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em3', 'tx': '192', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034545.075349, 'rx': '38652218', 'state': 'up'}, 'em2': {'rxErrors': '0', 'txErrors': '0', 'speed': '100', 'rxDropped': '0', 'name': 'em2', 'tx': '0', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034545.075349, 'rx': '16049684', 'state': 'up'}}, 'txDropped': '0', 'anonHugePages': '320', 'ksmPages': 100, 'elapsedTime': '223917.96', 'cpuLoad': '0.13', 'cpuSys': '0.55', 'diskStats': {'/var/log': {'free': '136515'}, '/var/run/vdsm/': {'free': '31856'}, '/tmp': {'free': '136515'}}, 'cpuUserVdsmd': '4.25', 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 63032, 'bootTime': '1567810572', 'haStats': {'active': True, 'configured': True, 'score': 3400, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'multipathHealth': {}, 'rxDropped': '0', 'outgoingVmMigrations': 0, 'swapTotal': 0, 'swapFree': 0, 'hugepages': defaultdict(<type 'dict'>, {1048576: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}, 2048: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}), 'dateTime': '2019-09-09T13:09:05 GMT', 'cpuUser': '1.56', 'memFree': 62776, 'cpuIdle': '97.89', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd': '1.99'}} from=::1,42394 (api:54) 2019-09-09 09:09:05,115-0400 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.09 seconds (__init__:312)

2 2