- Users - oVirt List Archives

Locked Snapshot
by Peter Smith 11 Sep '19

11 Sep '19

Hi, I have been trying to get a backup solution working for my VM's, and have somehow managed to break the snapshot on one of my VM's. I'm still fairly new to oVirt, we're migrating from VMware. I have had this cluster running since May. The cluster: We are running oVirt on CentOS 7, with Hosted Engine. oVirt 4.3.5.4-1.el7 (according to the Help->About in the Admin Console) Hardware is 5 x Dell M620 Blades (300Gb internal storage for host OS) with ISCSI storage for the VM's (Dell M3660i), 10GB Network throughout. The VM in question is allocated 50Gb of storage and there is 461GB free in the storage domain. So I've been playing with bareos 18.2.5 with the ovirt plugin found here https://github.com/eurotux/bareos-contrib/tree/master/fd-plugins/ovirt-plug… I got bareos hooked up to ovirt. It created a snapshot, downloaded it, then at the end it failed to remove the snapshot. I though it may have been because I was looking round bareos while it was doing the backup. Went into oVirt deleted the snapshot no issues. Re ran the backup job in bareos, leaving bareos alone this time. The backup job failed again being unable to remove the snapshot. So I figured I'd go in and delete it, but I get an error saying "Cannot remove Snapshot: The following disks are locked: VM-NAME_vmdisk1. Please try again in a few minutes.". In the Admin console if I expand the disks on the snapshot it shows that the image is locked. And before you ask the bareos user is setup as a SuperUser. If I go to the Events from the menu on the left of the admin console I can see that (Sorry Names/IP's changed to protect the guilty): Sep 10, 2019, 10:42:39 AM Image Download with disk VM-NAME_vmdisk1 succeeded. Sep 10, 2019, 10:42:38 AM VDSM hyper2 command TeardownImageVDS failed: Cannot deactivate Logical Volume: ('General Storage Exception: ("5 [] [\' Logical volume 46a9dfd4-4871-46b2-a740-8fe430c70b4b/bbe315b9-e797-4cf8-b18b-6afeb33589cc in use.\', \' Logical volume 46a9dfd4-4871-46b2-a740-8fe430c70b4b/d3dfa2c2-45fa-4081-98c5-890dd7a5990a in use.\']\\n46a9dfd4-4871-46b2-a740-8fe430c70b4b/[\'bbe315b9-e797-4cf8-b18b-6afeb33589cc\', \'d3dfa2c2-45fa-4081-98c5-890dd7a5990a\']",)',) Sep 10, 2019, 10:22:09 AM Image Download with disk VM-NAME_vmdisk1 was initiated by bareos@internal-authz. Sep 10, 2019, 10:21:55 AM Snapshot 'VM-NAME-backup-1f952494-8245-479e-8d1d-9503d6fe8f9f' creation for VM 'VM-NAME' has been completed. Sep 10, 2019, 10:21:36 AM Snapshot 'VM-NAME-backup-1f952494-8245-479e-8d1d-9503d6fe8f9f' creation for VM 'VM-NAME' was initiated by bareos@internal-authz. Sep 10, 2019, 10:21:36 AM Backup of virtual machine 'VM-NAME using snapshot 'VM-NAME-backup-1f952494-8245-479e-8d1d-9503d6fe8f9f' is starting. Sep 10, 2019, 10:21:36 AM User bareos@internal-authz connecting from '10.*******' using session '****************' logged in. To me it looks like bareos tried to delete the snapshot whilst it was still downloading it, which failed as expected. Does anyone know how to fix/recover from this? If you need log files, please could you tell me which ones? Regards, Peter Smith

3 3

How to change Master Data Storage Domain
by Mark Steele 11 Sep '19

11 Sep '19

Good morning, I have a Storage Domain that I would like to retire and move that role to a new storage domain. Both Domains exist on my current Data Center and I have moved all disks from the existing Data (Master) domain to the new Data Domain. The only thing that is still associated with the Data (Master) domain are two OVF_STORE items: Alias Virtual Size Actual Size Allocation Policy Storage Domain Storage Type Creation Date Attached To Alignment Status Description OVF_STORE < 1 GB < 1 GB Preallocated phl-datastore NFS 2014-Nov-14, 20:00 Unknown OK OVF_STORE OVF_STORE < 1 GB < 1 GB Preallocated phl-datastore NFS 2014-Nov-14, 20:00 Unknown OK OVF_STORE What is the procedure for moving / removing these items and 'promoting' the other Data Domain to Master? Our current version is: oVirt Engine Version: 3.5.0.1-1.el6 (it's old but reliable) Best regards, *** *Mark Steele* CIO / VP Technical Operations | TelVue Corporation TelVue - We Share Your Vision 16000 Horizon Way, Suite 100 | Mt. Laurel, NJ 08054 800.885.8886 x128 | msteele(a)telvue.com | http://www.telvue.com twitter: http://twitter.com/telvue | facebook: https://www.facebook.com/telvue

3 4

ovirt with glusterfs data domain - very slow writing speed on Windows server virtual machine
by lyubomir.grancharov＠bottleshipvfx.com 11 Sep '19

11 Sep '19

Hi folks, I have been experimenting with oVirt cluster based on glusterfs for the past few days. (first-timer). The cluster is up and running and it consists of 4 nodes and has 4 replicas. When I try to deploy Windows Server VM I encounter the following issue: The disk of the VM has ok reading speed ( close to bare metal) but the writing speed is very slow. ( about 10 times slower than it is supposed to be). Can anyone give me any suggestion, please? Thanks in advance! Here are the settings of the glusterfs volume: Volume Name: bottle-volume Type: Replicate Volume ID: 869b8d1e-1266-4820-8dcd-4fea92346b90 Status: Started Snapshot Count: 0 Number of Bricks: 1 x 4 = 4 Transport-type: tcp Bricks: Brick1: cnode01.bottleship.local:/gluster_bricks/brick-cnode01/brick-cnode01 Brick2: cnode02.bottleship.local:/gluster_bricks/brick-cnode02/brick-cnode02 Brick3: cnode03.bottleship.local:/gluster_bricks/brick-cnode03/brick-cnode03 Brick4: cnode04.bottleship.local:/gluster_bricks/brick-cnode04/brick-cnode04 Options Reconfigured: network.ping-timeout: 30 cluster.granular-entry-heal: enable performance.strict-o-direct: on storage.owner-gid: 36 storage.owner-uid: 36 server.event-threads: 4 client.event-threads: 4 cluster.choose-local: off features.shard: on cluster.shd-wait-qlength: 10000 cluster.shd-max-threads: 8 cluster.locking-scheme: granular cluster.data-self-heal-algorithm: full cluster.server-quorum-type: server cluster.quorum-type: auto cluster.eager-lock: enable network.remote-dio: off performance.low-prio-threads: 32 performance.io-cache: off performance.read-ahead: off performance.quick-read: off auth.allow: * user.cifs: off transport.address-family: inet nfs.disable: on performance.client-io-threads: off server.allow-insecure: on Please let me know if you need any more configuration info or hardware specs. best, Lyubo

2 1

Procedure to replace out out of three hyperconverged nodes
by thomas＠hoberg.net 11 Sep '19

11 Sep '19

On my silent Atom based three node Hyperconverged journey I hit upon a snag: Evidently they are too slow for Ansible. The Gluster storage part went all great and perfect on fresh oVirt node images that I had configured to leave an empty partition instead of the standard /dev/sdb, but the HostedEngine setup part would then fail without any log-visible error while the transient VM HostedEngineLocal was supposed to be launched and the Wizard would just show "deployment failed" and go ahead and delete the VM. I then moved the SSD to a more powerful Xeon-D 1541 CPU and after some fiddling with the network (I miss good old eth0!), this also failed the deployment, but also failed to delete the temporary VM image, because that actually turned out to be running: I could even connect to its console and investigate the logs for any clues as to what might have gone wrong (nothing visible): Evidently Ansible was running out of patience just a tiny bit too early. And then I kicked it into high-gear with an i7-7700K again using the same SSD with a working three node Gluster all in sync, which still took what felt like an hour to creep through every step, but got it done, primary node on i7, secondary nodes on Atoms, with full migration capabilities etc. I then had to do some fiddling, because the HostedEngine had configued the Cluster CPU architecture to Skylake-Spectre, but after that I migrated it to an Atom node and was now ready to move the primary to the intended Atom hardware target. But at that point the overlay network has already been configured and evidently it's tied to the device name of the 10Gbit NIC on the i7 workstation and I haven't been able to make it work with the Atom. The Gluster runs fine, but the CPU node is reported "non-operational" and re-installation fails, because the ovirtmgmt network isn't properly configured. That specific issue may be seem way out of what oVirt should support, yet a HA-embedded edge platform may very well see nodes having to be replaced or renewed with as little interruption or downtime as possible, which is why I am asking the larger question: How can you replace a) a failed "burned" node or b) upgrade nodes while maintaining fault tolerance? The distinction in b) would be that it's a planned maneuver during normal operations without downtime. I'd want to do it pretty much like I have been playing with compute nodes, creating new ones, pushing VMs on them, pushing them out to other hosts, removing and replacing them seamlessly... Except that the Gluster nodes are special and much harder to replace, than a pure Gluster storage brick... from what I see I welcome any help - for fixing the network config in my limiping Atom 1:3 cluster - eliminating the need to fiddle with an i7 because of Ansible timing - ensuring long-term operability of a software defined datacenter with changing hardware

2 2

gluster
by mailing-ovirt＠qic.ca 11 Sep '19

11 Sep '19

Hi, I see options to deploy ovirt with gluster during the initial rollout, however I can't seem to find information as to how I can add it following a non gluster initial setup: GlusterFS Version: [N/A] Thanks Simon

4 5

VM crushed and cant shutdown successfully,I shoud to reboot the ovirt-node machine
by zhouhao＠vip.friendtimes.net 11 Sep '19

11 Sep '19

VM crushed and cant shutdown successfully,It always on the way to shutdown，even if I wait it for 10 hours；Everytimes I shoud to reboot the ovirt-node machine This comes up 4 times, It appes both on the ovirt-noe 4.3 and 4.2; This bug appers when my VM is running at high load（seems memory and disk's w/r） My storage domain is local on host; When the bug appers,it could cost my node's io,looks like the picture below；But sometimes it will not The vm's threading on the ovirt-node, the IO ratio is 100% Everytime the vm's process will change to defunct I can not kill it,I had to shutdown the ovirt-node on the engine website ,the vm's status always in the way to shutdown ,even if I wait for it for hours; It either failed or is shutting down and the "power off" cant shutdown the vm too. the vm's error below The ovirt-node's error below ---------------------------------------------------------------- The other infomation about my ovirt-node node-version： node hardware zhouhao(a)vip.friendtimes.net

1 0

Python to retrive the VM-Config from the snapshot
by Sven Achtelik 10 Sep '19

10 Sep '19

Hi All, I'm trying to build a custom solution for Backups using the python. Everything works and I can get my disk copies. The last thing is getting the VM config and I can't find any hint on how to get this done with python. Is there a way to actually get this information out easily ? Somehting like * Grab the wanted vm configuration from the needed snapshot - it'll be under initialization/configuration/data URL = SERVER:PORT/api/vms/VM_ID/snapshots/ID Method = GET (with All-Content:true header) I also have to mention that I'm not a profession programmer and work with what I can find as documentation or code examples. http://ovirt.github.io/ovirt-engine-sdk/4.1/ https://github.com/oVirt/ovirt-engine-sdk/tree/master/sdk/examples If some experienced python programmer could give a hint where to look at that would be great. Thank you, Sven

2 1

Re: Python to retrive the VM-Config from the snapshot
by Strahil 10 Sep '19

10 Sep '19

If the system is running you can use virsh dumpxml. Best Regards, Strahil NikolovOn Sep 10, 2019 16:12, Sven Achtelik <Sven.Achtelik(a)eps.aero> wrote: > > Hi All, > > > > I’m trying to build a custom solution for Backups using the python. Everything works and I can get my disk copies. The last thing is getting the VM config and I can’t find any hint on how to get this done with python. Is there a way to actually get this information out easily ? Somehting like > > Grab the wanted vm configuration from the needed snapshot - it’ll be under initialization/configuration/data > > URL = SERVER:PORT/api/vms/VM_ID/snapshots/ID Method = GET (with All-Content:true header) > > I also have to mention that I’m not a profession programmer and work with what I can find as documentation or code examples. > > http://ovirt.github.io/ovirt-engine-sdk/4.1/ > > https://github.com/oVirt/ovirt-engine-sdk/tree/master/sdk/examples > > If some experienced python programmer could give a hint where to look at that would be great. > > Thank you, > > Sven

1 0

nfs
by mailing-ovirt＠qic.ca 10 Sep '19

10 Sep '19

I`m trying to mount a nfs share. if I manually mount it from ssh, I can access it without issues. However when I do it from the web config, it keeps failing: Not sure how to solve that. Thanks Simon 2019-09-09 09:08:47,601-0400 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:47,610-0400 INFO (jsonrpc/3) [vdsm.api] START repoStats(domains=[u'4f4828c2-8f78-4566-a69d-a37cd73446d4']) from=::1,42394, task_id=6c1eeef7-ff8c-4d0d-be85-ced03aae4bc2 (api:48) 2019-09-09 09:08:47,610-0400 INFO (jsonrpc/3) [vdsm.api] FINISH repoStats return={u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.00101971', 'lastCheck': '0.4', 'valid': True}} from=::1,42394, task_id=6c1eeef7-ff8c-4d0d-be85-ced03aae4bc2 (api:54) 2019-09-09 09:08:47,611-0400 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:47,839-0400 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:47,846-0400 INFO (jsonrpc/2) [api.host] START getCapabilities() from=::1,42394 (api:48) 2019-09-09 09:08:48,149-0400 INFO (jsonrpc/2) [root] managedvolume not supported: Managed Volume Not Supported. Missing package os-brick.: ('Cannot import os_brick',) (caps:152) 2019-09-09 09:08:49,212-0400 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.01 seconds (__init__:312) 2019-09-09 09:08:49,263-0400 INFO (jsonrpc/2) [root] /usr/libexec/vdsm/hooks/after_get_caps/50_openstacknet: rc=0 err= (hooks:114) 2019-09-09 09:08:49,497-0400 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:49,657-0400 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:49,665-0400 INFO (jsonrpc/7) [vdsm.api] START repoStats(domains=[u'4f4828c2-8f78-4566-a69d-a37cd73446d4']) from=::1,42394, task_id=14b596bf-bc1c-412b-b1fc-8d53f17d88f7 (api:48) 2019-09-09 09:08:49,666-0400 INFO (jsonrpc/7) [vdsm.api] FINISH repoStats return={u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.00101971', 'lastCheck': '2.4', 'valid': True}} from=::1,42394, task_id=14b596bf-bc1c-412b-b1fc-8d53f17d88f7 (api:54) 2019-09-09 09:08:49,667-0400 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:49,800-0400 INFO (jsonrpc/2) [root] /usr/libexec/vdsm/hooks/after_get_caps/openstacknet_utils.py: rc=0 err= (hooks:114) 2019-09-09 09:08:50,464-0400 INFO (jsonrpc/2) [root] /usr/libexec/vdsm/hooks/after_get_caps/ovirt_provider_ovn_hook: rc=0 err= (hooks:114) 2019-09-09 09:08:50,467-0400 INFO (jsonrpc/2) [api.host] FINISH getCapabilities return={'status': {'message': 'Done', 'code': 0}, 'info': {u'HBAInventory': {u'iSCSI': [{u'InitiatorName': u'iqn.1994-05.com.redhat:d8c85fc0ab85'}], u'FC': []}, u'packages2': {u'kernel': {u'release': u'957.27.2.el7.x86_64', u'version': u'3.10.0'}, u'spice-server': {u'release': u'6.el7_6.1', u'version': u'0.14.0'}, u'librbd1': {u'release': u'4.el7', u'version': u'10.2.5'}, u'vdsm': {u'release': u'1.el7', u'version': u'4.30.24'}, u'qemu-kvm': {u'release': u'18.el7_6.7.1', u'version': u'2.12.0'}, u'openvswitch': {u'release': u'4.el7', u'version': u'2.11.0'}, u'libvirt': {u'release': u'10.el7_6.12', u'version': u'4.5.0'}, u'ovirt-hosted-engine-ha': {u'release': u'1.el7', u'version': u'2.3.3'}, u'qemu-img': {u'release': u'18.el7_6.7.1', u'version': u'2.12.0'}, u'mom': {u'release': u'1.el7.centos', u'version': u'0.5.12'}, u'glusterfs-cli': {u'release': u'1.el7', u'version': u'6.5'}}, u'numaNodeDistance': {u'1': [20, 10], u'0': [10, 20]}, u'cpuModel': u'Intel(R) Xeon(R) CPU E5-2640 v2 @ 2.00GHz', u'nestedVirtualization': False, u'liveMerge': u'true', u'hooks': {u'before_vm_start': {u'50_hostedengine': {u'md5': u'95c810cdcfe4195302a59574a5148289'}, u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'after_network_setup': {u'30_ethtool_options': {u'md5': u'ce1fbad7aa0389e3b06231219140bf0d'}}, u'after_vm_destroy': {u'delete_vhostuserclient_hook': {u'md5': u'c2f279cc9483a3f842f6c29df13994c1'}, u'50_vhostmd': {u'md5': u'bdf4802c0521cf1bae08f2b90a9559cf'}}, u'after_vm_start': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'ea0a5a715da8c1badbcda28e8b8fa00e'}}, u'after_device_migrate_destination': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'6226fbc4d1602994828a3904fc1b875d'}}, u'before_device_migrate_destination': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}}, u'after_device_create': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'6226fbc4d1602994828a3904fc1b875d'}}, u'after_nic_hotplug': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'6226fbc4d1602994828a3904fc1b875d'}}, u'before_vm_migrate_destination': {u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'before_vm_dehibernate': {u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'before_nic_hotplug': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}, u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'ec8d5d7ba063a109f749cd63c7a2b58d'}, u'20_ovirt_provider_ovn_vhostuser_hook': {u'md5': u'a8af653b7386c138b2e6e9738bd6b62c'}, u'10_ovirt_provider_ovn_hook': {u'md5': u'73822988042847bab1ea832a6b9fa837'}}, u'after_get_caps': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'5c3a9ab6e06e039bdd220c0216e45809'}, u'ovirt_provider_ovn_hook': {u'md5': u'1a6ce22288fc2466fcc339da252ef624'}}, u'before_network_setup': {u'50_fcoe': {u'md5': u'28c352339c8beef1e1b05c67d106d062'}}, u'before_device_create': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}, u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'ec8d5d7ba063a109f749cd63c7a2b58d'}, u'20_ovirt_provider_ovn_vhostuser_hook': {u'md5': u'a8af653b7386c138b2e6e9738bd6b62c'}, u'10_ovirt_provider_ovn_hook': {u'md5': u'73822988042847bab1ea832a6b9fa837'}}}, u'supportsIPv6': True, u'realtimeKernel': False, u'vmTypes': [u'kvm'], u'liveSnapshot': u'true', u'cpuThreads': u'32', u'kdumpStatus': 0, u'deferred_preallocation': True, u'networks': {u'ovirtmgmt': {u'iface': u'ovirtmgmt', u'ipv6autoconf': True, u'addr': u'192.168.20.101', u'dhcpv6': False, u'ipv6addrs': [], u'switch': u'legacy', u'bridged': True, u'southbound': u'em1', u'dhcpv4': True, u'netmask': u'255.255.255.0', u'ipv4defaultroute': True, u'stp': u'off', u'ipv4addrs': [u'192.168.20.101/24'], u'mtu': u'1500', u'ipv6gateway': u'::', u'gateway': u'192.168.20.1', u'ports': [u'em1']}}, u'kernelArgs': u'BOOT_IMAGE=/vmlinuz-3.10.0-957.27.2.el7.x86_64 root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rhgb quiet LANG=en_US.UTF-8', u'domain_versions': [0, 2, 3, 4, 5], u'bridges': {u'ovirtmgmt': {u'ipv6autoconf': True, u'addr': u'192.168.20.101', u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': True, u'netmask': u'255.255.255.0', u'ipv4defaultroute': True, u'stp': u'off', u'ipv4addrs': [u'192.168.20.101/24'], u'ipv6gateway': u'::', u'gateway': u'192.168.20.1', u'opts': {u'multicast_last_member_count': u'2', u'vlan_protocol': u'0x8100', u'hash_elasticity': u'4', u'multicast_query_response_interval': u'1000', u'group_fwd_mask': u'0x0', u'multicast_snooping': u'1', u'multicast_startup_query_interval': u'3125', u'hello_timer': u'0', u'multicast_querier_interval': u'25500', u'max_age': u'2000', u'hash_max': u'512', u'stp_state': u'0', u'topology_change_detected': u'0', u'priority': u'32768', u'multicast_igmp_version': u'2', u'multicast_membership_interval': u'26000', u'root_path_cost': u'0', u'root_port': u'0', u'multicast_stats_enabled': u'0', u'multicast_startup_query_count': u'2', u'nf_call_iptables': u'0', u'vlan_stats_enabled': u'0', u'hello_time': u'200', u'topology_change': u'0', u'bridge_id': u'8000.f8bc1234c8a8', u'topology_change_timer': u'0', u'ageing_time': u'30000', u'nf_call_ip6tables': u'0', u'multicast_mld_version': u'1', u'gc_timer': u'6672', u'root_id': u'8000.f8bc1234c8a8', u'nf_call_arptables': u'0', u'group_addr': u'1:80:c2:0:0:0', u'multicast_last_member_interval': u'100', u'default_pvid': u'1', u'multicast_query_interval': u'12500', u'multicast_query_use_ifaddr': u'0', u'tcn_timer': u'0', u'multicast_router': u'1', u'vlan_filtering': u'0', u'multicast_querier': u'0', u'forward_delay': u'0'}, u'ports': [u'em1']}}, u'uuid': u'4c4c4544-0046-3310-8034-c8c04f4a3032', u'onlineCpus': u'0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,1,3,5,7,9,11,13,15,17,19,21,23, 25,27,29,31', u'nameservers': [u'192.168.168.1'], u'nics': {u'em4': {u'ipv6autoconf': True, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:ab', u'ipv6gateway': u'::', u'gateway': u''}, u'idrac': {u'ipv6autoconf': True, u'addr': u'', u'speed': 0, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:37:d8:b1', u'ipv6gateway': u'::', u'gateway': u''}, u'em1': {u'ipv6autoconf': False, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:a8', u'ipv6gateway': u'::', u'gateway': u''}, u'em3': {u'ipv6autoconf': True, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:aa', u'ipv6gateway': u'::', u'gateway': u''}, u'em2': {u'ipv6autoconf': True, u'addr': u'', u'speed': 100, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:a9', u'ipv6gateway': u'::', u'gateway': u''}}, u'software_revision': u'1', u'hostdevPassthrough': u'false', u'clusterLevels': [u'4.1', u'4.2', u'4.3'], u'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,cl flush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,c onstant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfm perf,eagerfpu,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16 ,xtpr,pdcm,pcid,dca,sse4_1,sse4_2,x2apic,popcnt,tsc_deadline_timer,aes,xsave ,avx,f16c,rdrand,lahf_lm,ssbd,ibrs,ibpb,stibp,tpr_shadow,vnmi,flexpriority,e pt,vpid,fsgsbase,smep,erms,xsaveopt,dtherm,ida,arat,pln,pts,md_clear,spec_ct rl,intel_stibp,flush_l1d,model_Opteron_G2,model_kvm32,model_SandyBridge-IBRS ,model_Opteron_G1,model_core2duo,model_pentium,model_Westmere-IBRS,model_qem u64,model_kvm64,model_coreduo,model_Nehalem,model_Nehalem-IBRS,model_IvyBrid ge,model_Westmere,model_SandyBridge,model_IvyBridge-IBRS,model_qemu32,model_ Penryn,model_pentium2,model_pentium3,model_Conroe,model_486', u'kernelFeatures': {u'IBRS': 0, u'RETP': 1, u'SSBD': 3, u'PTI': 1}, u'ISCSIInitiatorName': u'iqn.1994-05.com.redhat:d8c85fc0ab85', u'netConfigDirty': u'False', u'supported_block_sizes': [512], u'selinux': {u'mode': u'1'}, u'autoNumaBalancing': 1, u'reservedMem': u'321', u'bondings': {}, u'software_version': u'4.30.24', u'supportedENGINEs': [u'4.1', u'4.2', u'4.3'], u'vncEncrypted': False, u'backupEnabled': False, u'cpuSpeed': u'1199.951', u'numaNodes': {u'1': {u'totalMemory': u'32768', u'cpus': [1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31]}, u'0': {u'totalMemory': u'32704', u'cpus': [0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30]}}, u'cpuSockets': u'2', u'vlans': {}, u'version_name': u'Snow Man', 'lastClientIface': 'lo', u'cpuCores': u'16', u'hostedEngineDeployed': True, u'hugepages': [1048576, 2048], u'guestOverhead': u'65', u'additionalFeatures': [u'libgfapi_supported'], u'openstack_binding_host_ids': {u'OPENSTACK_OVN': u'baremetal1.ledinfo.com', u'OPEN_VSWITCH': u'baremetal1.ledinfo.com', u'OVIRT_PROVIDER_OVN': u'c83bc734-603c-4a99-a5b9-5833ef1d90c7'}, u'kvmEnabled': u'true', u'memSize': u'64195', u'emulatedMachines': [u'pc-i440fx-rhel7.1.0', u'pc-q35-rhel7.3.0', u'rhel6.3.0', u'pc-i440fx-rhel7.5.0', u'pc-i440fx-rhel7.0.0', u'rhel6.1.0', u'pc-q35-rhel7.6.0', u'pc-i440fx-rhel7.4.0', u'rhel6.6.0', u'pc-q35-rhel7.5.0', u'rhel6.2.0', u'pc', u'pc-i440fx-rhel7.3.0', u'q35', u'pc-i440fx-rhel7.2.0', u'rhel6.4.0', u'pc-q35-rhel7.4.0', u'pc-i440fx-rhel7.6.0', u'rhel6.0.0', u'rhel6.5.0'], u'rngSources': [u'hwrng', u'random'], u'operatingSystem': {u'release': u'6.1810.2.el7.centos', u'pretty_name': u'CentOS Linux 7 (Core)', u'version': u'7', u'name': u'RHEL'}}} from=::1,42394 (api:54) 2019-09-09 09:08:50,493-0400 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.getCapabilities succeeded in 2.65 seconds (__init__:312) 2019-09-09 09:08:51,518-0400 INFO (jsonrpc/5) [api.host] START getAllVmStats() from=::1,42344 (api:48) 2019-09-09 09:08:51,518-0400 INFO (jsonrpc/5) [api.host] FINISH getAllVmStats return={'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} from=::1,42344 (api:54) 2019-09-09 09:08:51,519-0400 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:51,551-0400 INFO (jsonrpc/3) [vdsm.api] START connectStorageServer(domType=1, spUUID=u'00000000-0000-0000-0000-000000000000', conList=[{u'tpgt': u'1', u'id': u'00000000-0000-0000-0000-000000000000', u'connection': u'192.168.20.125:/ovirt', u'iqn': u'', u'user': u'', u'ipv6_enabled': u'false', u'protocol_version': u'auto', u'password': '********', u'port': u''}], options=None) from=::ffff:192.168.20.116,41974, flow_id=1b259177-b83c-4066-a045-b3a9db670a1e, task_id=3d0e32c4-288b-4be4-98ef-90fb676b57f2 (api:48) 2019-09-09 09:08:51,556-0400 INFO (jsonrpc/3) [storage.StorageServer.MountConnection] Creating directory u'/rhev/data-center/mnt/192.168.20.125:_ovirt' (storageServer:168) 2019-09-09 09:08:51,556-0400 INFO (jsonrpc/3) [storage.fileUtils] Creating directory: /rhev/data-center/mnt/192.168.20.125:_ovirt mode: None (fileUtils:199) 2019-09-09 09:08:51,556-0400 INFO (jsonrpc/3) [storage.Mount] mounting 192.168.20.125:/ovirt at /rhev/data-center/mnt/192.168.20.125:_ovirt (mount:204) 2019-09-09 09:08:51,684-0400 INFO (jsonrpc/3) [IOProcessClient] (Global) Starting client (__init__:308) 2019-09-09 09:08:51,705-0400 INFO (ioprocess/9120) [IOProcess] (Global) Starting ioprocess (__init__:434) 2019-09-09 09:08:51,739-0400 INFO (jsonrpc/3) [vdsm.api] FINISH connectStorageServer return={'statuslist': [{'status': 0, 'id': u'00000000-0000-0000-0000-000000000000'}]} from=::ffff:192.168.20.116,41974, flow_id=1b259177-b83c-4066-a045-b3a9db670a1e, task_id=3d0e32c4-288b-4be4-98ef-90fb676b57f2 (api:54) 2019-09-09 09:08:51,741-0400 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call StoragePool.connectStorageServer succeeded in 0.19 seconds (__init__:312) 2019-09-09 09:08:51,992-0400 INFO (jsonrpc/4) [vdsm.api] START connectStorageServer(domType=1, spUUID=u'00000000-0000-0000-0000-000000000000', conList=[{u'tpgt': u'1', u'id': u'1280de25-c248-41d3-8552-50ef964d0ee0', u'connection': u'192.168.20.125:/ovirt', u'iqn': u'', u'user': u'', u'ipv6_enabled': u'false', u'protocol_version': u'auto', u'password': '********', u'port': u''}], options=None) from=::ffff:192.168.20.116,41974, flow_id=360af944, task_id=44612f73-58c9-4597-9a29-c1e65e149c0a (api:48) 2019-09-09 09:08:51,999-0400 INFO (jsonrpc/4) [vdsm.api] FINISH connectStorageServer return={'statuslist': [{'status': 0, 'id': u'1280de25-c248-41d3-8552-50ef964d0ee0'}]} from=::ffff:192.168.20.116,41974, flow_id=360af944, task_id=44612f73-58c9-4597-9a29-c1e65e149c0a (api:54) 2019-09-09 09:08:52,000-0400 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call StoragePool.connectStorageServer succeeded in 0.01 seconds (__init__:312) 2019-09-09 09:08:52,010-0400 INFO (jsonrpc/6) [vdsm.api] START createStorageDomain(storageType=1, sdUUID=u'785a9a50-96c0-4cac-9c80-d0e2dd5bd162', domainName=u'ovirt-omv', typeSpecificArg=u'192.168.20.125:/ovirt', domClass=1, domVersion=u'5', block_size=512, max_hosts=250, options=None) from=::ffff:192.168.20.116,41974, flow_id=360af944, task_id=62008086-97f8-4986-b329-cc193b84fd95 (api:48) 2019-09-09 09:08:52,727-0400 WARN (jsonrpc/6) [storage.LVM] Reloading VGs failed (vgs=[u'785a9a50-96c0-4cac-9c80-d0e2dd5bd162'] rc=5 out=[] err=[' Volume group "785a9a50-96c0-4cac-9c80-d0e2dd5bd162" not found', ' Cannot process volume group 785a9a50-96c0-4cac-9c80-d0e2dd5bd162']) (lvm:470) 2019-09-09 09:08:52,731-0400 INFO (itmap/0) [IOProcessClient] (/baremetal3.ledinfo.com:_export) Starting client (__init__:308) 2019-09-09 09:08:52,751-0400 INFO (ioprocess/9141) [IOProcess] (/baremetal3.ledinfo.com:_export) Starting ioprocess (__init__:434) 2019-09-09 09:08:52,753-0400 INFO (itmap/1) [IOProcessClient] (/baremetal3.ledinfo.com:_import) Starting client (__init__:308) 2019-09-09 09:08:52,777-0400 INFO (itmap/2) [IOProcessClient] (/baremetal3.ledinfo.com:_ISO) Starting client (__init__:308) 2019-09-09 09:08:52,783-0400 INFO (ioprocess/9150) [IOProcess] (/baremetal3.ledinfo.com:_import) Starting ioprocess (__init__:434) 2019-09-09 09:08:52,802-0400 INFO (itmap/3) [IOProcessClient] (/192.168.20.125:_ovirt) Starting client (__init__:308) 2019-09-09 09:08:52,822-0400 INFO (ioprocess/9157) [IOProcess] (/baremetal3.ledinfo.com:_ISO) Starting ioprocess (__init__:434) 2019-09-09 09:08:52,828-0400 INFO (ioprocess/9160) [IOProcess] (/192.168.20.125:_ovirt) Starting ioprocess (__init__:434) 2019-09-09 09:08:52,831-0400 INFO (jsonrpc/6) [storage.StorageDomain] sdUUID=785a9a50-96c0-4cac-9c80-d0e2dd5bd162 domainName=ovirt-omv remotePath=192.168.20.125:/ovirt domClass=1, block_size=512, alignment=1048576 (nfsSD:86) 2019-09-09 09:08:52,866-0400 INFO (jsonrpc/6) [IOProcessClient] (785a9a50-96c0-4cac-9c80-d0e2dd5bd162) Starting client (__init__:308) 2019-09-09 09:08:52,889-0400 INFO (ioprocess/9174) [IOProcess] (785a9a50-96c0-4cac-9c80-d0e2dd5bd162) Starting ioprocess (__init__:434) 2019-09-09 09:08:52,895-0400 INFO (jsonrpc/6) [storage.StorageDomain] Creating domain metadata directory u'/rhev/data-center/mnt/192.168.20.125:_ovirt/785a9a50-96c0-4cac-9c80-d0e2dd 5bd162/dom_md' (fileSD:453) 2019-09-09 09:08:52,926-0400 INFO (jsonrpc/6) [storage.xlease] Formatting index for lockspace u'785a9a50-96c0-4cac-9c80-d0e2dd5bd162' (version=1) (xlease:691) 2019-09-09 09:08:53,047-0400 INFO (jsonrpc/6) [storage.StorageDomain] Creating domain images directory u'/rhev/data-center/mnt/192.168.20.125:_ovirt/785a9a50-96c0-4cac-9c80-d0e2dd 5bd162/images' (nfsSD:110) 2019-09-09 09:08:53,067-0400 INFO (jsonrpc/6) [storage.StorageDomain] Removing remnants of deleted images [] (fileSD:747) 2019-09-09 09:08:53,072-0400 ERROR (jsonrpc/6) [storage.initSANLock] Cannot initialize SANLock for domain 785a9a50-96c0-4cac-9c80-d0e2dd5bd162 (clusterlock:258) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/clusterlock.py", line 249, in initSANLock sdUUID, idsPath, align=alignment, sector=block_size) File "/usr/lib/python2.7/site-packages/vdsm/storage/compat.py", line 52, in decorator return func(*args, **kwargs) SanlockException: (19, 'Sanlock lockspace write failure', 'No such device') 2019-09-09 09:08:53,073-0400 INFO (jsonrpc/6) [vdsm.api] FINISH createStorageDomain error=Could not initialize cluster lock: () from=::ffff:192.168.20.116,41974, flow_id=360af944, task_id=62008086-97f8-4986-b329-cc193b84fd95 (api:52) 2019-09-09 09:08:53,074-0400 ERROR (jsonrpc/6) [storage.TaskManager.Task] (Task='62008086-97f8-4986-b329-cc193b84fd95') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "<string>", line 2, in createStorageDomain File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 2619, in createStorageDomain storageType, domVersion, block_size, alignment) File "/usr/lib/python2.7/site-packages/vdsm/storage/nfsSD.py", line 120, in create fsd.initSPMlease() File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 918, in initSPMlease return self._manifest.initDomainLock() File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 609, in initDomainLock self._domainLock.initLock(self.getDomainLease()) File "/usr/lib/python2.7/site-packages/vdsm/storage/clusterlock.py", line 303, in initLock block_size=self._block_size) File "/usr/lib/python2.7/site-packages/vdsm/storage/clusterlock.py", line 259, in initSANLock raise se.ClusterLockInitError() ClusterLockInitError: Could not initialize cluster lock: () 2019-09-09 09:08:53,075-0400 INFO (jsonrpc/6) [storage.TaskManager.Task] (Task='62008086-97f8-4986-b329-cc193b84fd95') aborting: Task is aborted: 'Could not initialize cluster lock: ()' - code 701 (task:1181) 2019-09-09 09:08:53,076-0400 ERROR (jsonrpc/6) [storage.Dispatcher] FINISH createStorageDomain error=Could not initialize cluster lock: () (dispatcher:83) 2019-09-09 09:08:53,077-0400 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call StorageDomain.create failed (error 701) in 1.07 seconds (__init__:312) 2019-09-09 09:08:53,425-0400 INFO (jsonrpc/0) [api.host] START getStats() from=::ffff:192.168.20.116,41974 (api:48) 2019-09-09 09:08:53,439-0400 INFO (jsonrpc/1) [vdsm.api] START disconnectStorageServer(domType=1, spUUID=u'00000000-0000-0000-0000-000000000000', conList=[{u'tpgt': u'1', u'id': u'1280de25-c248-41d3-8552-50ef964d0ee0', u'connection': u'192.168.20.125:/ovirt', u'iqn': u'', u'user': u'', u'ipv6_enabled': u'false', u'protocol_version': u'auto', u'password': '********', u'port': u''}], options=None) from=::ffff:192.168.20.116,41974, flow_id=e003809f-c7ef-4105-ac27-48be39b5c7b1, task_id=da5b627a-de84-4083-9454-aaecd6bb8aed (api:48) 2019-09-09 09:08:53,440-0400 INFO (jsonrpc/1) [storage.Mount] unmounting /rhev/data-center/mnt/192.168.20.125:_ovirt (mount:212) 2019-09-09 09:08:53,478-0400 INFO (jsonrpc/0) [vdsm.api] START repoStats(domains=()) from=::ffff:192.168.20.116,41974, task_id=42f53132-6ef6-4aff-9ab3-91d19c24bd93 (api:48) 2019-09-09 09:08:53,479-0400 INFO (jsonrpc/0) [vdsm.api] FINISH repoStats return={u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.00104514', 'lastCheck': '3.4', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000798507', 'lastCheck': '3.5', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000768283', 'lastCheck': '3.3', 'valid': True}} from=::ffff:192.168.20.116,41974, task_id=42f53132-6ef6-4aff-9ab3-91d19c24bd93 (api:54) 2019-09-09 09:08:53,481-0400 INFO (jsonrpc/0) [vdsm.api] START multipath_health() from=::ffff:192.168.20.116,41974, task_id=34ab20bd-b1ce-4bbe-ad1c-af0e78d38b1c (api:48) 2019-09-09 09:08:53,481-0400 INFO (jsonrpc/0) [vdsm.api] FINISH multipath_health return={} from=::ffff:192.168.20.116,41974, task_id=34ab20bd-b1ce-4bbe-ad1c-af0e78d38b1c (api:54) 2019-09-09 09:08:53,523-0400 INFO (jsonrpc/0) [api.host] FINISH getStats return={'status': {'message': 'Done', 'code': 0}, 'info': {'cpuStatistics': {'24': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.53'}, '25': {'cpuUser': '0.46', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.47'}, '26': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '27': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '20': {'cpuUser': '2.06', 'nodeIndex': 0, 'cpuSys': '0.40', 'cpuIdle': '97.54'}, '21': {'cpuUser': '0.99', 'nodeIndex': 1, 'cpuSys': '0.33', 'cpuIdle': '98.68'}, '22': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '23': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.74'}, '28': {'cpuUser': '0.99', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '98.88'}, '29': {'cpuUser': '0.46', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.47'}, '1': {'cpuUser': '2.85', 'nodeIndex': 1, 'cpuSys': '1.66', 'cpuIdle': '95.49'}, '0': {'cpuUser': '0.46', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.41'}, '3': {'cpuUser': '0.73', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.20'}, '2': {'cpuUser': '0.33', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.47'}, '5': {'cpuUser': '0.20', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.73'}, '4': {'cpuUser': '0.99', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '98.68'}, '7': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.74'}, '6': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '99.33'}, '9': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '8': {'cpuUser': '0.93', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '98.80'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.53'}, '13': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '12': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '15': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '14': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '17': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '16': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.87'}, '19': {'cpuUser': '1.79', 'nodeIndex': 1, 'cpuSys': '0.46', 'cpuIdle': '97.75'}, '18': {'cpuUser': '8.69', 'nodeIndex': 0, 'cpuSys': '1.72', 'cpuIdle': '89.59'}, '31': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '30': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.80'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31474'}, '0': {'memPercent': 7, 'memFree': '30445'}}, 'memShared': 0, 'haScore': 3400, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'vmCount': 0, 'memUsed': '3', 'storageDomains': {u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.00104514', 'lastCheck': '3.4', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000798507', 'lastCheck': '3.5', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000768283', 'lastCheck': '3.3', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'idrac': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'idrac', 'tx': '180', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '0', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '0', 'state': 'down'}, 'ovirtmgmt': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '1075026007', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '43585639790', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '618193132', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '618193132', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '0', 'state': 'down'}, 'em4': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em4', 'tx': '192', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034533.476725, 'rx': '38644112', 'state': 'up'}, 'br-int': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'br-int', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '0', 'state': 'down'}, 'em1': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em1', 'tx': '1115341150', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034533.476725, 'rx': '45376217511', 'state': 'up'}, 'genev_sys_6081': {'rxErrors': '0', 'txErrors': '8', 'speed': '1000', 'rxDropped': '0', 'name': 'genev_sys_6081', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034533.476725, 'rx': '0', 'state': 'up'}, 'em3': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em3', 'tx': '192', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034533.476725, 'rx': '38644112', 'state': 'up'}, 'em2': {'rxErrors': '0', 'txErrors': '0', 'speed': '100', 'rxDropped': '0', 'name': 'em2', 'tx': '0', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034533.476725, 'rx': '16049108', 'state': 'up'}}, 'txDropped': '0', 'anonHugePages': '306', 'ksmPages': 100, 'elapsedTime': '223906.36', 'cpuLoad': '0.14', 'cpuSys': '0.24', 'diskStats': {'/var/log': {'free': '136508'}, '/var/run/vdsm/': {'free': '31856'}, '/tmp': {'free': '136508'}}, 'cpuUserVdsmd': '2.78', 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 63031, 'bootTime': '1567810572', 'haStats': {'active': True, 'configured': True, 'score': 3400, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'multipathHealth': {}, 'rxDropped': '0', 'outgoingVmMigrations': 0, 'swapTotal': 0, 'swapFree': 0, 'hugepages': defaultdict(<type 'dict'>, {1048576: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}, 2048: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}), 'dateTime': '2019-09-09T13:08:53 GMT', 'cpuUser': '0.75', 'memFree': 62775, 'cpuIdle': '99.01', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd': '1.06'}} from=::ffff:192.168.20.116,41974 (api:54) 2019-09-09 09:08:53,534-0400 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.11 seconds (__init__:312) 2019-09-09 09:08:54,001-0400 INFO (jsonrpc/1) [vdsm.api] FINISH disconnectStorageServer return={'statuslist': [{'status': 0, 'id': u'1280de25-c248-41d3-8552-50ef964d0ee0'}]} from=::ffff:192.168.20.116,41974, flow_id=e003809f-c7ef-4105-ac27-48be39b5c7b1, task_id=da5b627a-de84-4083-9454-aaecd6bb8aed (api:54) 2019-09-09 09:08:54,003-0400 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call StoragePool.disconnectStorageServer succeeded in 0.57 seconds (__init__:312) 2019-09-09 09:08:54,888-0400 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:54,895-0400 INFO (jsonrpc/2) [api.host] START getStats() from=::1,42394 (api:48) 2019-09-09 09:08:54,946-0400 INFO (jsonrpc/2) [vdsm.api] START repoStats(domains=()) from=::1,42394, task_id=f1acae4b-2563-4335-96c7-8b5097aa4127 (api:48) 2019-09-09 09:08:54,947-0400 INFO (jsonrpc/2) [vdsm.api] FINISH repoStats return={u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.00104514', 'lastCheck': '4.8', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000798507', 'lastCheck': '5.0', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000768283', 'lastCheck': '4.8', 'valid': True}} from=::1,42394, task_id=f1acae4b-2563-4335-96c7-8b5097aa4127 (api:54) 2019-09-09 09:08:54,948-0400 INFO (jsonrpc/2) [vdsm.api] START multipath_health() from=::1,42394, task_id=58483dc0-2a50-409a-b991-16dbb57360a7 (api:48) 2019-09-09 09:08:54,949-0400 INFO (jsonrpc/2) [vdsm.api] FINISH multipath_health return={} from=::1,42394, task_id=58483dc0-2a50-409a-b991-16dbb57360a7 (api:54) 2019-09-09 09:08:54,977-0400 INFO (jsonrpc/2) [api.host] FINISH getStats return={'status': {'message': 'Done', 'code': 0}, 'info': {'cpuStatistics': {'24': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.53'}, '25': {'cpuUser': '0.46', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.47'}, '26': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '27': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '20': {'cpuUser': '2.06', 'nodeIndex': 0, 'cpuSys': '0.40', 'cpuIdle': '97.54'}, '21': {'cpuUser': '0.99', 'nodeIndex': 1, 'cpuSys': '0.33', 'cpuIdle': '98.68'}, '22': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '23': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.74'}, '28': {'cpuUser': '0.99', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '98.88'}, '29': {'cpuUser': '0.46', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.47'}, '1': {'cpuUser': '2.85', 'nodeIndex': 1, 'cpuSys': '1.66', 'cpuIdle': '95.49'}, '0': {'cpuUser': '0.46', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.41'}, '3': {'cpuUser': '0.73', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.20'}, '2': {'cpuUser': '0.33', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.47'}, '5': {'cpuUser': '0.20', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.73'}, '4': {'cpuUser': '0.99', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '98.68'}, '7': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.74'}, '6': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '99.33'}, '9': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '8': {'cpuUser': '0.93', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '98.80'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.53'}, '13': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '12': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '15': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '14': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '17': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '16': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.87'}, '19': {'cpuUser': '1.79', 'nodeIndex': 1, 'cpuSys': '0.46', 'cpuIdle': '97.75'}, '18': {'cpuUser': '8.69', 'nodeIndex': 0, 'cpuSys': '1.72', 'cpuIdle': '89.59'}, '31': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '30': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.80'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31474'}, '0': {'memPercent': 7, 'memFree': '30445'}}, 'memShared': 0, 'haScore': 3400, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'vmCount': 0, 'memUsed': '3', 'storageDomains': {u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.00104514', 'lastCheck': '4.8', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000798507', 'lastCheck': '5.0', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000768283', 'lastCheck': '4.8', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'idrac': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'idrac', 'tx': '180', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '0', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '0', 'state': 'down'}, 'ovirtmgmt': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '1075046072', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '43586697397', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '618194231', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '618194231', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '0', 'state': 'down'}, 'em4': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em4', 'tx': '192', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034534.944025, 'rx': '38644112', 'state': 'up'}, 'br-int': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'br-int', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '0', 'state': 'down'}, 'em1': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em1', 'tx': '1115653922', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034534.944025, 'rx': '45377347900', 'state': 'up'}, 'genev_sys_6081': {'rxErrors': '0', 'txErrors': '8', 'speed': '1000', 'rxDropped': '0', 'name': 'genev_sys_6081', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034534.944025, 'rx': '0', 'state': 'up'}, 'em3': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em3', 'tx': '192', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034534.944025, 'rx': '38644176', 'state': 'up'}, 'em2': {'rxErrors': '0', 'txErrors': '0', 'speed': '100', 'rxDropped': '0', 'name': 'em2', 'tx': '0', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034534.944025, 'rx': '16049108', 'state': 'up'}}, 'txDropped': '0', 'anonHugePages': '306', 'ksmPages': 100, 'elapsedTime': '223907.83', 'cpuLoad': '0.14', 'cpuSys': '0.24', 'diskStats': {'/var/log': {'free': '136508'}, '/var/run/vdsm/': {'free': '31856'}, '/tmp': {'free': '136508'}}, 'cpuUserVdsmd': '2.78', 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 63031, 'bootTime': '1567810572', 'haStats': {'active': True, 'configured': True, 'score': 3400, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'multipathHealth': {}, 'rxDropped': '0', 'outgoingVmMigrations': 0, 'swapTotal': 0, 'swapFree': 0, 'hugepages': defaultdict(<type 'dict'>, {1048576: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}, 2048: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}), 'dateTime': '2019-09-09T13:08:54 GMT', 'cpuUser': '0.75', 'memFree': 62775, 'cpuIdle': '99.01', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd': '1.06'}} from=::1,42394 (api:54) 2019-09-09 09:08:54,986-0400 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.09 seconds (__init__:312) 2019-09-09 09:08:55,842-0400 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:55,850-0400 INFO (jsonrpc/3) [api.virt] START getStats() from=::1,42394, vmId=aeed8192-5313-45d6-b9f5-fb2db09fb646 (api:48) 2019-09-09 09:08:55,850-0400 INFO (jsonrpc/3) [api] FINISH getStats error=Virtual machine does not exist: {'vmId': u'aeed8192-5313-45d6-b9f5-fb2db09fb646'} (api:129) 2019-09-09 09:08:55,850-0400 INFO (jsonrpc/3) [api.virt] FINISH getStats return={'status': {'message': "Virtual machine does not exist: {'vmId': u'aeed8192-5313-45d6-b9f5-fb2db09fb646'}", 'code': 1}} from=::1,42394, vmId=aeed8192-5313-45d6-b9f5-fb2db09fb646 (api:54) 2019-09-09 09:08:55,851-0400 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call VM.getStats failed (error 1) in 0.00 seconds (__init__:312) 2019-09-09 09:08:57,526-0400 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:57,534-0400 INFO (jsonrpc/6) [api.host] START getCapabilities() from=::1,42394 (api:48) 2019-09-09 09:08:57,627-0400 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:57,636-0400 INFO (jsonrpc/1) [vdsm.api] START repoStats(domains=[u'4f4828c2-8f78-4566-a69d-a37cd73446d4']) from=::1,42394, task_id=6da33ef3-de09-48a3-8ba9-6c31d5e2942b (api:48) 2019-09-09 09:08:57,636-0400 INFO (jsonrpc/1) [vdsm.api] FINISH repoStats return={u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000798507', 'lastCheck': '0.4', 'valid': True}} from=::1,42394, task_id=6da33ef3-de09-48a3-8ba9-6c31d5e2942b (api:54) 2019-09-09 09:08:57,637-0400 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:57,870-0400 INFO (jsonrpc/6) [root] managedvolume not supported: Managed Volume Not Supported. Missing package os-brick.: ('Cannot import os_brick',) (caps:152) 2019-09-09 09:08:58,884-0400 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:58,990-0400 INFO (jsonrpc/6) [root] /usr/libexec/vdsm/hooks/after_get_caps/50_openstacknet: rc=0 err= (hooks:114) 2019-09-09 09:08:59,048-0400 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:59,055-0400 INFO (jsonrpc/5) [vdsm.api] START repoStats(domains=[u'4f4828c2-8f78-4566-a69d-a37cd73446d4']) from=::1,42394, task_id=933c8ee8-2330-46ed-a609-38a519447156 (api:48) 2019-09-09 09:08:59,056-0400 INFO (jsonrpc/5) [vdsm.api] FINISH repoStats return={u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000798507', 'lastCheck': '1.8', 'valid': True}} from=::1,42394, task_id=933c8ee8-2330-46ed-a609-38a519447156 (api:54) 2019-09-09 09:08:59,057-0400 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:08:59,526-0400 INFO (jsonrpc/6) [root] /usr/libexec/vdsm/hooks/after_get_caps/openstacknet_utils.py: rc=0 err= (hooks:114) 2019-09-09 09:08:59,896-0400 INFO (jsonrpc/3) [api.host] START getAllVmStats() from=::ffff:192.168.20.116,41974 (api:48) 2019-09-09 09:08:59,897-0400 INFO (jsonrpc/3) [api.host] FINISH getAllVmStats return={'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} from=::ffff:192.168.20.116,41974 (api:54) 2019-09-09 09:08:59,898-0400 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:09:00,188-0400 INFO (periodic/0) [vdsm.api] START repoStats(domains=()) from=internal, task_id=aec0525d-393b-49fa-a0d4-96bc19f829e3 (api:48) 2019-09-09 09:09:00,190-0400 INFO (periodic/0) [vdsm.api] FINISH repoStats return={u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000991915', 'lastCheck': '0.1', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000985319', 'lastCheck': '0.2', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000768283', 'lastCheck': '3.1', 'valid': True}} from=internal, task_id=aec0525d-393b-49fa-a0d4-96bc19f829e3 (api:54) 2019-09-09 09:09:00,192-0400 INFO (jsonrpc/6) [root] /usr/libexec/vdsm/hooks/after_get_caps/ovirt_provider_ovn_hook: rc=0 err= (hooks:114) 2019-09-09 09:09:00,194-0400 INFO (jsonrpc/6) [api.host] FINISH getCapabilities return={'status': {'message': 'Done', 'code': 0}, 'info': {u'HBAInventory': {u'iSCSI': [{u'InitiatorName': u'iqn.1994-05.com.redhat:d8c85fc0ab85'}], u'FC': []}, u'packages2': {u'kernel': {u'release': u'957.27.2.el7.x86_64', u'version': u'3.10.0'}, u'spice-server': {u'release': u'6.el7_6.1', u'version': u'0.14.0'}, u'librbd1': {u'release': u'4.el7', u'version': u'10.2.5'}, u'vdsm': {u'release': u'1.el7', u'version': u'4.30.24'}, u'qemu-kvm': {u'release': u'18.el7_6.7.1', u'version': u'2.12.0'}, u'openvswitch': {u'release': u'4.el7', u'version': u'2.11.0'}, u'libvirt': {u'release': u'10.el7_6.12', u'version': u'4.5.0'}, u'ovirt-hosted-engine-ha': {u'release': u'1.el7', u'version': u'2.3.3'}, u'qemu-img': {u'release': u'18.el7_6.7.1', u'version': u'2.12.0'}, u'mom': {u'release': u'1.el7.centos', u'version': u'0.5.12'}, u'glusterfs-cli': {u'release': u'1.el7', u'version': u'6.5'}}, u'numaNodeDistance': {u'1': [20, 10], u'0': [10, 20]}, u'cpuModel': u'Intel(R) Xeon(R) CPU E5-2640 v2 @ 2.00GHz', u'nestedVirtualization': False, u'liveMerge': u'true', u'hooks': {u'before_vm_start': {u'50_hostedengine': {u'md5': u'95c810cdcfe4195302a59574a5148289'}, u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'after_network_setup': {u'30_ethtool_options': {u'md5': u'ce1fbad7aa0389e3b06231219140bf0d'}}, u'after_vm_destroy': {u'delete_vhostuserclient_hook': {u'md5': u'c2f279cc9483a3f842f6c29df13994c1'}, u'50_vhostmd': {u'md5': u'bdf4802c0521cf1bae08f2b90a9559cf'}}, u'after_vm_start': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'ea0a5a715da8c1badbcda28e8b8fa00e'}}, u'after_device_migrate_destination': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'6226fbc4d1602994828a3904fc1b875d'}}, u'before_device_migrate_destination': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}}, u'after_device_create': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'6226fbc4d1602994828a3904fc1b875d'}}, u'after_nic_hotplug': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'6226fbc4d1602994828a3904fc1b875d'}}, u'before_vm_migrate_destination': {u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'before_vm_dehibernate': {u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'before_nic_hotplug': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}, u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'ec8d5d7ba063a109f749cd63c7a2b58d'}, u'20_ovirt_provider_ovn_vhostuser_hook': {u'md5': u'a8af653b7386c138b2e6e9738bd6b62c'}, u'10_ovirt_provider_ovn_hook': {u'md5': u'73822988042847bab1ea832a6b9fa837'}}, u'after_get_caps': {u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'5c3a9ab6e06e039bdd220c0216e45809'}, u'ovirt_provider_ovn_hook': {u'md5': u'1a6ce22288fc2466fcc339da252ef624'}}, u'before_network_setup': {u'50_fcoe': {u'md5': u'28c352339c8beef1e1b05c67d106d062'}}, u'before_device_create': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}, u'openstacknet_utils.py': {u'md5': u'1ed38ddf30f8a9c7574589e77e2c0b1f'}, u'50_openstacknet': {u'md5': u'ec8d5d7ba063a109f749cd63c7a2b58d'}, u'20_ovirt_provider_ovn_vhostuser_hook': {u'md5': u'a8af653b7386c138b2e6e9738bd6b62c'}, u'10_ovirt_provider_ovn_hook': {u'md5': u'73822988042847bab1ea832a6b9fa837'}}}, u'supportsIPv6': True, u'realtimeKernel': False, u'vmTypes': [u'kvm'], u'liveSnapshot': u'true', u'cpuThreads': u'32', u'kdumpStatus': 0, u'deferred_preallocation': True, u'networks': {u'ovirtmgmt': {u'iface': u'ovirtmgmt', u'ipv6autoconf': True, u'addr': u'192.168.20.101', u'dhcpv6': False, u'ipv6addrs': [], u'switch': u'legacy', u'bridged': True, u'southbound': u'em1', u'dhcpv4': True, u'netmask': u'255.255.255.0', u'ipv4defaultroute': True, u'stp': u'off', u'ipv4addrs': [u'192.168.20.101/24'], u'mtu': u'1500', u'ipv6gateway': u'::', u'gateway': u'192.168.20.1', u'ports': [u'em1']}}, u'kernelArgs': u'BOOT_IMAGE=/vmlinuz-3.10.0-957.27.2.el7.x86_64 root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rhgb quiet LANG=en_US.UTF-8', u'domain_versions': [0, 2, 3, 4, 5], u'bridges': {u'ovirtmgmt': {u'ipv6autoconf': True, u'addr': u'192.168.20.101', u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': True, u'netmask': u'255.255.255.0', u'ipv4defaultroute': True, u'stp': u'off', u'ipv4addrs': [u'192.168.20.101/24'], u'ipv6gateway': u'::', u'gateway': u'192.168.20.1', u'opts': {u'multicast_last_member_count': u'2', u'vlan_protocol': u'0x8100', u'hash_elasticity': u'4', u'multicast_query_response_interval': u'1000', u'group_fwd_mask': u'0x0', u'multicast_snooping': u'1', u'multicast_startup_query_interval': u'3125', u'hello_timer': u'0', u'multicast_querier_interval': u'25500', u'max_age': u'2000', u'hash_max': u'512', u'stp_state': u'0', u'topology_change_detected': u'0', u'priority': u'32768', u'multicast_igmp_version': u'2', u'multicast_membership_interval': u'26000', u'root_path_cost': u'0', u'root_port': u'0', u'multicast_stats_enabled': u'0', u'multicast_startup_query_count': u'2', u'nf_call_iptables': u'0', u'vlan_stats_enabled': u'0', u'hello_time': u'200', u'topology_change': u'0', u'bridge_id': u'8000.f8bc1234c8a8', u'topology_change_timer': u'0', u'ageing_time': u'30000', u'nf_call_ip6tables': u'0', u'multicast_mld_version': u'1', u'gc_timer': u'5700', u'root_id': u'8000.f8bc1234c8a8', u'nf_call_arptables': u'0', u'group_addr': u'1:80:c2:0:0:0', u'multicast_last_member_interval': u'100', u'default_pvid': u'1', u'multicast_query_interval': u'12500', u'multicast_query_use_ifaddr': u'0', u'tcn_timer': u'0', u'multicast_router': u'1', u'vlan_filtering': u'0', u'multicast_querier': u'0', u'forward_delay': u'0'}, u'ports': [u'em1']}}, u'uuid': u'4c4c4544-0046-3310-8034-c8c04f4a3032', u'onlineCpus': u'0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,1,3,5,7,9,11,13,15,17,19,21,23, 25,27,29,31', u'nameservers': [u'192.168.168.1'], u'nics': {u'em4': {u'ipv6autoconf': True, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:ab', u'ipv6gateway': u'::', u'gateway': u''}, u'idrac': {u'ipv6autoconf': True, u'addr': u'', u'speed': 0, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:37:d8:b1', u'ipv6gateway': u'::', u'gateway': u''}, u'em1': {u'ipv6autoconf': False, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:a8', u'ipv6gateway': u'::', u'gateway': u''}, u'em3': {u'ipv6autoconf': True, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:aa', u'ipv6gateway': u'::', u'gateway': u''}, u'em2': {u'ipv6autoconf': True, u'addr': u'', u'speed': 100, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'f8:bc:12:34:c8:a9', u'ipv6gateway': u'::', u'gateway': u''}}, u'software_revision': u'1', u'hostdevPassthrough': u'false', u'clusterLevels': [u'4.1', u'4.2', u'4.3'], u'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,cl flush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,c onstant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfm perf,eagerfpu,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16 ,xtpr,pdcm,pcid,dca,sse4_1,sse4_2,x2apic,popcnt,tsc_deadline_timer,aes,xsave ,avx,f16c,rdrand,lahf_lm,ssbd,ibrs,ibpb,stibp,tpr_shadow,vnmi,flexpriority,e pt,vpid,fsgsbase,smep,erms,xsaveopt,dtherm,ida,arat,pln,pts,md_clear,spec_ct rl,intel_stibp,flush_l1d,model_Opteron_G2,model_kvm32,model_SandyBridge-IBRS ,model_Opteron_G1,model_core2duo,model_pentium,model_Westmere-IBRS,model_qem u64,model_kvm64,model_coreduo,model_Nehalem,model_Nehalem-IBRS,model_IvyBrid ge,model_Westmere,model_SandyBridge,model_IvyBridge-IBRS,model_qemu32,model_ Penryn,model_pentium2,model_pentium3,model_Conroe,model_486', u'kernelFeatures': {u'IBRS': 0, u'RETP': 1, u'SSBD': 3, u'PTI': 1}, u'ISCSIInitiatorName': u'iqn.1994-05.com.redhat:d8c85fc0ab85', u'netConfigDirty': u'False', u'supported_block_sizes': [512], u'selinux': {u'mode': u'1'}, u'autoNumaBalancing': 1, u'reservedMem': u'321', u'bondings': {}, u'software_version': u'4.30.24', u'supportedENGINEs': [u'4.1', u'4.2', u'4.3'], u'vncEncrypted': False, u'backupEnabled': False, u'cpuSpeed': u'1199.951', u'numaNodes': {u'1': {u'totalMemory': u'32768', u'cpus': [1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31]}, u'0': {u'totalMemory': u'32704', u'cpus': [0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30]}}, u'cpuSockets': u'2', u'vlans': {}, u'version_name': u'Snow Man', 'lastClientIface': 'lo', u'cpuCores': u'16', u'hostedEngineDeployed': True, u'hugepages': [1048576, 2048], u'guestOverhead': u'65', u'additionalFeatures': [u'libgfapi_supported'], u'openstack_binding_host_ids': {u'OPENSTACK_OVN': u'baremetal1.ledinfo.com', u'OPEN_VSWITCH': u'baremetal1.ledinfo.com', u'OVIRT_PROVIDER_OVN': u'c83bc734-603c-4a99-a5b9-5833ef1d90c7'}, u'kvmEnabled': u'true', u'memSize': u'64195', u'emulatedMachines': [u'pc-i440fx-rhel7.1.0', u'pc-q35-rhel7.3.0', u'rhel6.3.0', u'pc-i440fx-rhel7.5.0', u'pc-i440fx-rhel7.0.0', u'rhel6.1.0', u'pc-q35-rhel7.6.0', u'pc-i440fx-rhel7.4.0', u'rhel6.6.0', u'pc-q35-rhel7.5.0', u'rhel6.2.0', u'pc', u'pc-i440fx-rhel7.3.0', u'q35', u'pc-i440fx-rhel7.2.0', u'rhel6.4.0', u'pc-q35-rhel7.4.0', u'pc-i440fx-rhel7.6.0', u'rhel6.0.0', u'rhel6.5.0'], u'rngSources': [u'hwrng', u'random'], u'operatingSystem': {u'release': u'6.1810.2.el7.centos', u'pretty_name': u'CentOS Linux 7 (Core)', u'version': u'7', u'name': u'RHEL'}}} from=::1,42394 (api:54) 2019-09-09 09:09:00,231-0400 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.getCapabilities succeeded in 2.69 seconds (__init__:312) 2019-09-09 09:09:05,021-0400 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2019-09-09 09:09:05,028-0400 INFO (jsonrpc/0) [api.host] START getStats() from=::1,42394 (api:48) 2019-09-09 09:09:05,077-0400 INFO (jsonrpc/0) [vdsm.api] START repoStats(domains=()) from=::1,42394, task_id=12e8837e-c9cd-45b9-aecb-477f86e12ec5 (api:48) 2019-09-09 09:09:05,078-0400 INFO (jsonrpc/0) [vdsm.api] FINISH repoStats return={u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000991915', 'lastCheck': '5.0', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000985319', 'lastCheck': '5.1', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.00093201', 'lastCheck': '4.8', 'valid': True}} from=::1,42394, task_id=12e8837e-c9cd-45b9-aecb-477f86e12ec5 (api:54) 2019-09-09 09:09:05,079-0400 INFO (jsonrpc/0) [vdsm.api] START multipath_health() from=::1,42394, task_id=42c12b61-4d82-4a06-b692-e11dfbb0bca6 (api:48) 2019-09-09 09:09:05,080-0400 INFO (jsonrpc/0) [vdsm.api] FINISH multipath_health return={} from=::1,42394, task_id=42c12b61-4d82-4a06-b692-e11dfbb0bca6 (api:54) 2019-09-09 09:09:05,106-0400 INFO (jsonrpc/0) [api.host] FINISH getStats return={'status': {'message': 'Done', 'code': 0}, 'info': {'cpuStatistics': {'24': {'cpuUser': '0.46', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.41'}, '25': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '26': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '27': {'cpuUser': '1.66', 'nodeIndex': 1, 'cpuSys': '0.40', 'cpuIdle': '97.94'}, '20': {'cpuUser': '1.66', 'nodeIndex': 0, 'cpuSys': '0.86', 'cpuIdle': '97.48'}, '21': {'cpuUser': '0.73', 'nodeIndex': 1, 'cpuSys': '0.40', 'cpuIdle': '98.87'}, '22': {'cpuUser': '0.93', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '98.80'}, '23': {'cpuUser': '0.93', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '98.94'}, '28': {'cpuUser': '1.00', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '98.73'}, '29': {'cpuUser': '0.60', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.27'}, '1': {'cpuUser': '4.31', 'nodeIndex': 1, 'cpuSys': '4.05', 'cpuIdle': '91.64'}, '0': {'cpuUser': '0.60', 'nodeIndex': 0, 'cpuSys': '0.40', 'cpuIdle': '99.00'}, '3': {'cpuUser': '0.27', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.66'}, '2': {'cpuUser': '7.23', 'nodeIndex': 0, 'cpuSys': '1.06', 'cpuIdle': '91.71'}, '5': {'cpuUser': '0.20', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.67'}, '4': {'cpuUser': '1.99', 'nodeIndex': 0, 'cpuSys': '0.86', 'cpuIdle': '97.15'}, '7': {'cpuUser': '0.27', 'nodeIndex': 1, 'cpuSys': '0.27', 'cpuIdle': '99.46'}, '6': {'cpuUser': '1.13', 'nodeIndex': 0, 'cpuSys': '0.60', 'cpuIdle': '98.27'}, '9': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.80'}, '8': {'cpuUser': '0.80', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '98.87'}, '11': {'cpuUser': '0.66', 'nodeIndex': 1, 'cpuSys': '0.40', 'cpuIdle': '98.94'}, '10': {'cpuUser': '0.33', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '99.40'}, '13': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '12': {'cpuUser': '1.33', 'nodeIndex': 0, 'cpuSys': '0.46', 'cpuIdle': '98.21'}, '15': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '14': {'cpuUser': '0.46', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '99.27'}, '17': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '16': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '19': {'cpuUser': '6.77', 'nodeIndex': 1, 'cpuSys': '1.59', 'cpuIdle': '91.64'}, '18': {'cpuUser': '14.40', 'nodeIndex': 0, 'cpuSys': '3.65', 'cpuIdle': '81.95'}, '31': {'cpuUser': '0.46', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.41'}, '30': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31468'}, '0': {'memPercent': 7, 'memFree': '30418'}}, 'memShared': 0, 'haScore': 3400, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'vmCount': 0, 'memUsed': '3', 'storageDomains': {u'a04efd69-f8ef-4c5a-9948-e640f7d61590': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000991915', 'lastCheck': '5.0', 'valid': True}, u'4f4828c2-8f78-4566-a69d-a37cd73446d4': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.000985319', 'lastCheck': '5.1', 'valid': True}, u'a5942a40-b278-46d9-9b9c-72efffeab1d9': {'code': 0, 'actual': True, 'version': 5, 'acquired': True, 'delay': '0.00093201', 'lastCheck': '4.8', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'idrac': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'idrac', 'tx': '180', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '0', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '0', 'state': 'down'}, 'ovirtmgmt': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '1075063579', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '43586979921', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '618221192', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '618221192', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '0', 'state': 'down'}, 'em4': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em4', 'tx': '192', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034545.075349, 'rx': '38652218', 'state': 'up'}, 'br-int': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'br-int', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '0', 'state': 'down'}, 'em1': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em1', 'tx': '1115671653', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034545.075349, 'rx': '45377642078', 'state': 'up'}, 'genev_sys_6081': {'rxErrors': '0', 'txErrors': '8', 'speed': '1000', 'rxDropped': '0', 'name': 'genev_sys_6081', 'tx': '0', 'txDropped': '0', 'duplex': 'unknown', 'sampleTime': 1568034545.075349, 'rx': '0', 'state': 'up'}, 'em3': {'rxErrors': '0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'em3', 'tx': '192', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034545.075349, 'rx': '38652218', 'state': 'up'}, 'em2': {'rxErrors': '0', 'txErrors': '0', 'speed': '100', 'rxDropped': '0', 'name': 'em2', 'tx': '0', 'txDropped': '0', 'duplex': 'full', 'sampleTime': 1568034545.075349, 'rx': '16049684', 'state': 'up'}}, 'txDropped': '0', 'anonHugePages': '320', 'ksmPages': 100, 'elapsedTime': '223917.96', 'cpuLoad': '0.13', 'cpuSys': '0.55', 'diskStats': {'/var/log': {'free': '136515'}, '/var/run/vdsm/': {'free': '31856'}, '/tmp': {'free': '136515'}}, 'cpuUserVdsmd': '4.25', 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 63032, 'bootTime': '1567810572', 'haStats': {'active': True, 'configured': True, 'score': 3400, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'multipathHealth': {}, 'rxDropped': '0', 'outgoingVmMigrations': 0, 'swapTotal': 0, 'swapFree': 0, 'hugepages': defaultdict(<type 'dict'>, {1048576: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}, 2048: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}), 'dateTime': '2019-09-09T13:09:05 GMT', 'cpuUser': '1.56', 'memFree': 62776, 'cpuIdle': '97.89', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd': '1.99'}} from=::1,42394 (api:54) 2019-09-09 09:09:05,115-0400 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.09 seconds (__init__:312)

2 2

ovirt-engine-extension-aaa-ldap-setup
by Rick A 09 Sep '19

09 Sep '19

Trying to get oVirt to use LDAP for user authentication. It seems to bind, but when I test it (Login Flow and Search) before applying the settings, I get the message below (I can confirm the credentials ARE valid). I also tried to set it up manually and still get the same result. I'm using the same information on Bugzilla and it works fine. I've seen similar threads here but without much answers. Anyone have any good links or docs I can view to get this set up? oVirt Node Version: 4.3.3.6-1.el7 API: <--Authn.InvokeCommands.AUTHENTICATE_CREDENTIALS profile='mydomain.com' result=CREDENTIALS_INVALID

3 5

network config change
by mailing-ovirt＠qic.ca 09 Sep '19

09 Sep '19

Greetings, So we have a single hosted engine ovirt box deployed. When this was deployed, the network requirements were poorly determined and there was no port teaming or any vlan configuration. now we need to change that. I have searched all over for any form of procedure or guide as to how we do this. in summary: I need to go from a setup that uses em1 as it's default device and gateway to a setup where em1 and 3 are bonded into bond0 and now use that instead. As for the vlan we could live with keeping the native vlan configured on the cisco side for management and only add extra ones for VMs. any assistance as to what I need to go through from a bash shell would be appreciated. thanks Simon

3 3

VM cant shutdown successfully,It's status always in the way to shutdown
by zhouhao＠vip.friendtimes.net 09 Sep '19

09 Sep '19

Sometimes on the engine website ,the vm's status always in the way to shutdown ,even if I wait for it for hours; It either failed or is shutting down and the "power off" cant shutdown the vm too. zhouhao(a)vip.friendtimes.net

2 2

Failure while import VM from export domain
by Timmi 08 Sep '19

08 Sep '19

Hi oVirt list, I'm planing to migrate my oVirt installation on a new HW. My current platform is running a 4.2.8. The new one a 4.3.5. I'm currently trying to import the VMs through the export domain. But some of my old VMs are failing. Which log file would be interesting to check? Also maybe there is a better way to copy more less all VMs from my current oVirt to the new one. Best regards Christoph

2 2

HP NetXen Incorporated NX3031 driver
by Leo David 07 Sep '19

07 Sep '19

Hi everyone, I have this bloody card installed on a node, and it seems that no driver can be loaded for it, "ip link sh" does not show it. Any recommendation about installing this network card on oVirt 4.2.8 ? lspci -v .... 05:00.0 Ethernet controller: NetXen Incorporated NX3031 Multifunction 1/10-Gigabit Server Adapter (rev 42) Subsystem: Hewlett-Packard Company NC522SFP Dual Port 10GbE Server Adapter Flags: fast devsel, IRQ 31 Memory at d9c00000 (64-bit, non-prefetchable) [size=2M] Memory at da000000 (64-bit, non-prefetchable) [size=32M] Expansion ROM at d8000000 [disabled] [size=64K] Capabilities: [40] MSI-X: Enable- Count=64 Masked- Capabilities: [80] Power Management version 3 Capabilities: [a0] MSI: Enable- Count=1/32 Maskable- 64bit+ Capabilities: [c0] Express Endpoint, MSI 00 Capabilities: [100] Advanced Error Reporting Capabilities: [140] Device Serial Number 59-69-46-61-6e-48-73-75 Kernel modules: netxen_nic 05:00.1 Ethernet controller: NetXen Incorporated NX3031 Multifunction 1/10-Gigabit Server Adapter (rev 42) Subsystem: Hewlett-Packard Company NC522SFP Dual Port 10GbE Server Adapter Flags: fast devsel, IRQ 33 Memory at d9e00000 (64-bit, non-prefetchable) [size=2M] Memory at dc000000 (64-bit, non-prefetchable) [size=32M] Capabilities: [40] MSI-X: Enable- Count=64 Masked- Capabilities: [80] Power Management version 3 Capabilities: [a0] MSI: Enable- Count=1/32 Maskable- 64bit+ Capabilities: [c0] Express Endpoint, MSI 00 Capabilities: [100] Advanced Error Reporting Capabilities: [140] Device Serial Number 59-69-46-61-6e-48-73-75 Kernel modules: netxen_nic dmesg | grep netxen [ 2.188078] netxen_nic 0000:05:00.0: 2MB memory map [ 2.188371] netxen_nic 0000:05:00.0: Timeout reached waiting for rom done [ 2.188442] netxen_nic 0000:05:00.0: Error getting board config info. [ 2.189488] netxen_nic: probe of 0000:05:00.0 failed with error -5 [ 2.190438] netxen_nic 0000:05:00.1: 2MB memory map [ 2.190741] netxen_nic 0000:05:00.1: Timeout reached waiting for rom done [ 2.190808] netxen_nic 0000:05:00.1: Error getting board config info. [ 2.190964] netxen_nic: probe of 0000:05:00.1 failed with error -5 ... Thank you very much ! Leo -- Best regards, Leo David

2 1

Windows server std 2019 64 bit support
by Alex K 06 Sep '19

06 Sep '19

Hi all, I am trying to find out if Windows server std 2019 64 bit is supported at ovirt 3.4. I have checked the following links: https://www.ovirt.org/documentation/vmm-guide/chap-Introduction.html https://access.redhat.com/articles/973133 https://www.windowsservercatalog.com/results.aspx?&bCatID=1521&cpID=24121&a… From the above I deduce that RHEV 4.3.5 supports it and ovirt does not include it in the supported list of guest OS. Also KVM on CentOS 7.7 seems to be supporting it as RHEL 7.7 is listed as supported platform. Have you tried to load Windows server std 2019 64 bit successfullyon ovirt 3.4? Thank you.

2 2

ovirt-guest: Network device hanging shortly after startup (after Upgrade to 4.3.5)
by julius.kempa＠astelon.ch 06 Sep '19

06 Sep '19

We have a some old guests (SLES 10.4) which has been worked corectly till 4.2.7 (Manager + Host) After Upgrade to 4.3.5, we have got a network issue, that the gues network device, not depending on used device type, stopts to work after some seconds.... after about 15-20 pings device is hanging. Sometimes, in the first time after the device hungs up, the message "no buffer space available" comes from ping. Does someone knows a solution for it? greetings Julius

3 3

[ANN] oVirt 4.3.6 Fifth Release Candidate is now available for testing
by Sandro Bonazzola 05 Sep '19

05 Sep '19

The oVirt Project is pleased to announce the availability of the oVirt 4.3.6 Fifth Release Candidate for testing, as of September 5th, 2019. This update is a release candidate of the sixth in a series of stabilization updates to the 4.3 series. This is pre-release software. This pre-release should not to be used in production. This release is available now on x86_64 architecture for: * Red Hat Enterprise Linux 7.7 or later (but <8) * CentOS Linux (or similar) 7.7 or later (but <8) This release supports Hypervisor Hosts on x86_64 and ppc64le architectures for: * Red Hat Enterprise Linux 7.7 or later (but <8) * CentOS Linux (or similar) 7.7 or later (but <8) * oVirt Node 4.3 (available for x86_64 only) has been built consuming CentOS 7.7 Candidate Release See the release notes [1] for known issues, new features and bugs fixed. Notes: - oVirt Appliance is already available - oVirt Node is already available based on CentOS 7.7 Candidate Release Additional Resources: * Read more about the oVirt 4.3.6 release highlights: http://www.ovirt.org/release/4.3.6/ * Get more oVirt Project updates on Twitter: https://twitter.com/ovirt * Check out the latest project news on the oVirt blog: http://www.ovirt.org/blog/ [1] http://www.ovirt.org/release/4.3.6/ [2] http://resources.ovirt.org/pub/ovirt-4.3-pre/iso/ -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/>*Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*

1 0

VM crushed and cant shutdown successfully,I shoud to reboot the ovirt-node machine
by zhouhao＠vip.friendtimes.net 05 Sep '19

05 Sep '19

VM crushed and cant shutdown successfully,It always on the way to shutdown，even if I wait it for 10 hours；Everytimes I shoud to reboot the ovirt-node machine This comes up 4 times, It appes both on the ovirt-noe 4.3 and 4.2; This bug appers when my VM is running at high load（seems memory and disk's w/r） My storage domain is local on host; When the bug appers,it could cost my node's io,looks like the picture below；But sometimes it will not The vm's threading on the ovirt-node, the IO ratio is 100% Everytime the vm's process will change to defunct I can not kill it,I had to shutdown the ovirt-node on the engine website ,the vm's status always in the way to shutdown ,even if I wait for it for hours; It either failed or is shutting down and the "power off" cant shutdown the vm too. the vm's error below The ovirt-node's error below ---------------------------------------------------------------- The other infomation about my ovirt-node node-version： node hardware zhouhao(a)vip.friendtimes.net

1 0

Re: [ANN] oVirt 4.3.6 Fourth Release Candidate is now available for testing
by Strahil 05 Sep '19

05 Sep '19

Hi Sandro, Can the update work with 7.6 ? I'm asking because CentOS 7.7 is still building and unavailable. Best Regards, Strahil NikolovOn Aug 29, 2019 10:47, Sandro Bonazzola <sbonazzo(a)redhat.com> wrote: > > The oVirt Project is pleased to announce the availability of the oVirt 4.3.6 Fourth Release Candidate for testing, as of August 29th, 2019. > > This update is a release candidate of the sixth in a series of stabilization updates to the 4.3 series. > This is pre-release software. This pre-release should not to be used in production. > > This release is available now on x86_64 architecture for: > * Red Hat Enterprise Linux 7.7 or later (but <8) > * CentOS Linux (or similar) 7.7 or later (but <8) > > This release supports Hypervisor Hosts on x86_64 and ppc64le architectures for: > * Red Hat Enterprise Linux 7.7 or later (but <8) > * CentOS Linux (or similar) 7.7 or later (but <8) > * oVirt Node 4.3 (available for x86_64 only) will be made available when CentOS 7.7 will be released. > > See the release notes [1] for known issues, new features and bugs fixed. > > Notes: > - oVirt Appliance is already available > - oVirt Node is not yet available, pending CentOS 7.7 release to be available > > Additional Resources: > * Read more about the oVirt 4.3.6 release highlights: http://www.ovirt.org/release/4.3.6/ > * Get more oVirt Project updates on Twitter: https://twitter.com/ovirt > * Check out the latest project news on the oVirt blog:http://www.ovirt.org/blog/ > > [1] http://www.ovirt.org/release/4.3.6/ > [2] http://resources.ovirt.org/pub/ovirt-4.3-pre/iso/ > > -- > > Sandro Bonazzola > > MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV > > Red Hat EMEA > > sbonazzo(a)redhat.com > > Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.

2 1

Re: Any real experiences with nested virtualisation?
by Strahil 05 Sep '19

05 Sep '19

I guess you want to do something like this one : http://www.chrisj.cloud/?q=node/8 As Ovirt is upstream of RHV , I believe that RDO and Ovirt have almost the same integration as in the post. Still,such setup requires a lot of knowledge about KVM, oVirt and OpenStack. Best Regards, Strahil NikolovOn Sep 4, 2019 21:36, thomas(a)hoberg.net wrote: > > First, it's a scenario the oVirt team seem to actually embrace themselves. > > I believe I have seen that type of deployment mentioned either in one of the forward looking blog posts or Red Hat summit videos. > > And then it simply makes a lot of sense to have those crucial management VMs that control the swarm of resources in a cloud run under a fault-tolerant VM management environment such as oVirt. > > As for nested virtualization, I can only provide some negative experience with oVirt. > > I've done nested virtualization with VMware, running ESX on VMWare Workstation and then some VMs underneath that ESX successfully. > > When I tried to do the same with oVirt, running an oVirt hyperconverged cluster as VMs on VMware workstation, I got reasonably far, but then every VM I tried to launch nested would just stop right at boot. > > It first happened with the hosted-engine, as that tries to start as a VM, but it happened more obviously when I tried to migrate VMs (just a plain Fedora 30 in this case) from a physical compute node running oVirt nodeOS to a virtualized compute node, running the same current oVirt nodeOS image on VMware Workstation (Windows 2016 host, if that matters). > > The machine would start the live-migration properly and then just freeze and stay frozen until I migrated it back or indeed to any other non-virtualized node. > > I haven't as yet tried to do that with nested KVM, which I believe might actually work: But it's on my things to test, so I'll report back, once I get around it. > > Actually I am wondering, a) how deep this nesting would be allowed to go and b) if the 'leaf' nodes at the last nesting level should actually have nesting support activated in their kernel parameters... > > Any insight here would be well appreciated. > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/6YDBQGIXUWTDB…

1 0

Any real experiences with nested virtualisation?
by tonyppe＠gmail.com 04 Sep '19

04 Sep '19

I have ovirt 4.3.5 and Openstack hosts. There are 2 x openstack compute and 2 x ovirt hosts with hosted manager. I think i can simplify the management of this if I were to virtualise the openstack compute nodes as VMs on ovirt. I wanted to reach out here to gauge any other users experiences with it if at all. From the research I have done it looks like from the VM perspective, migration must not be allowed. Any info / comments / feedback appreciated. Regards,

2 1

Re: Cannot Increase Hosted Engine VM Memory
by Dionysis K 04 Sep '19

04 Sep '19

Now i noticed another strange i deactivated the memory balloon option from the VM hosted engine and tried to update the values again with the same memory it still keeps the value of memory size at 5120 while the value of physical memory guaranteed it changed to what we desire at the 8192 value and then the error appears again in the vm engine i ckecked the engine log file it shows the same errors! is there are other logfile locations that can give us any clues ?

1 0

Re: Cannot Increase Hosted Engine VM Memory
by Douglas Duckworth 04 Sep '19

04 Sep '19

Yes, I do. Gold crown indeed. It's the "HostedEngine" as seen attached! Thanks, Douglas Duckworth, MSc, LFCS HPC System Administrator Scientific Computing Unit<https://scu.med.cornell.edu> Weill Cornell Medicine 1300 York Avenue New York, NY 10065 E: doug(a)med.cornell.edu<mailto:doug@med.cornell.edu> O: 212-746-6305 F: 212-746-8690 On Wed, Jan 23, 2019 at 12:02 PM Simone Tiraboschi <stirabos(a)redhat.com<mailto:stirabos@redhat.com>> wrote: On Wed, Jan 23, 2019 at 5:51 PM Douglas Duckworth <dod2014(a)med.cornell.edu<mailto:dod2014@med.cornell.edu>> wrote: Hi Simone Can I get help with this issue? Still cannot increase memory for Hosted Engine. From the logs it seams that the engine is trying to hotplug memory to the engine VM which is something it should not happen. The engine should simply update engine VM configuration in the OVF_STORE and require a reboot of the engine VM. Quick question, in the VM panel do you see a gold crown symbol on the Engine VM? Thanks, Douglas Duckworth, MSc, LFCS HPC System Administrator Scientific Computing Unit<https://scu.med.cornell.edu> Weill Cornell Medicine 1300 York Avenue New York, NY 10065 E: doug(a)med.cornell.edu<mailto:doug@med.cornell.edu> O: 212-746-6305 F: 212-746-8690 On Thu, Jan 17, 2019 at 8:08 AM Douglas Duckworth <dod2014(a)med.cornell.edu<mailto:dod2014@med.cornell.edu>> wrote: Sure, they're attached. In "first attempt" the error seems to be: 2019-01-17 07:49:24,795-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-29) [680f82b3-7612-4d91-afdc-43937aa298a2] EVENT_ID: FAILED_HOT_SET_MEMORY_NOT_DIVIDABLE(2,048), Failed to hot plug memory to VM HostedEngine. Amount of added memory (4000MiB) is not dividable by 256MiB. Followed by: 2019-01-17 07:49:24,814-05 WARN [org.ovirt.engine.core.bll.UpdateRngDeviceCommand] (default task-29) [26f5f3ed] Validation of action 'UpdateRngDevice' failed for user admin@internal-authz. Reasons: ACTION_TYPE_FAILED_VM_IS_RUNNING 2019-01-17 07:49:24,815-05 ERROR [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-29) [26f5f3ed] Updating RNG device of VM HostedEngine (adf14389-1563-4b1a-9af6-4b40370a825b) failed. Old RNG device = VmRngDevice:{id='VmDeviceId:{deviceId='6435b2b5-163c-4f0c-934e-7994da60dc89', vmId='adf14389-1563-4b1a-9af6-4b40370a825b'}', device='virtio', type='RNG', specParams='[source=urandom]', address='', managed='true', plugged='true', readOnly='false', deviceAlias='', customProperties='null', snapshotId='null', logicalName='null', hostDevice='null'}. New RNG device = VmRngDevice:{id='VmDeviceId:{deviceId='6435b2b5-163c-4f0c-934e-7994da60dc89', vmId='adf14389-1563-4b1a-9af6-4b40370a825b'}', device='virtio', type='RNG', specParams='[source=urandom]', address='', managed='true', plugged='true', readOnly='false', deviceAlias='', customProperties='null', snapshotId='null', logicalName='null', hostDevice='null'}. In "second attempt" I used values that are dividable by 256 MiB so that's no longer present. Though same error: 2019-01-17 07:56:59,795-05 INFO [org.ovirt.engine.core.vdsbroker.SetAmountOfMemoryVDSCommand] (default task-22) [7059a48f] START, SetAmountOfMemoryVDSCommand(HostName = ovirt-hv1.med.cornell.edu<http://ovirt-hv1.med.cornell.edu>, Params:{hostId='cdd5ffda-95c7-4ffa-ae40-be66f1d15c30', vmId='adf14389-1563-4b1a-9af6-4b40370a825b', memoryDevice='VmDevice:{id='VmDeviceId:{deviceId='7f7d97cc-c273-4033-af53-bc9033ea3abe', vmId='adf14389-1563-4b1a-9af6-4b40370a825b'}', device='memory', type='MEMORY', specParams='[node=0, size=2048]', address='', managed='true', plugged='true', readOnly='false', deviceAlias='', customProperties='null', snapshotId='null', logicalName='null', hostDevice='null'}', minAllocatedMem='6144'}), log id: 50873daa 2019-01-17 07:56:59,855-05 INFO [org.ovirt.engine.core.vdsbroker.SetAmountOfMemoryVDSCommand] (default task-22) [7059a48f] FINISH, SetAmountOfMemoryVDSCommand, log id: 50873daa 2019-01-17 07:56:59,862-05 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-22) [7059a48f] EVENT_ID: HOT_SET_MEMORY(2,039), Hotset memory: changed the amount of memory on VM HostedEngine from 4096 to 4096 2019-01-17 07:56:59,881-05 WARN [org.ovirt.engine.core.bll.UpdateRngDeviceCommand] (default task-22) [28fd4c82] Validation of action 'UpdateRngDevice' failed for user admin@internal-authz. Reasons: ACTION_TYPE_FAILED_VM_IS_RUNNING 2019-01-17 07:56:59,882-05 ERROR [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-22) [28fd4c82] Updating RNG device of VM HostedEngine (adf14389-1563-4b1a-9af6-4b40370a825b) failed. Old RNG device = VmRngDevice:{id='VmDeviceId:{deviceId='6435b2b5-163c-4f0c-934e-7994da60dc89', vmId='adf14389-1563-4b1a-9af6-4b40370a825b'}', device='virtio', type='RNG', specParams='[source=urandom]', address='', managed='true', plugged='true', readOnly='false', deviceAlias='', customProperties='null', snapshotId='null', logicalName='null', hostDevice='null'}. New RNG device = VmRngDevice:{id='VmDeviceId:{deviceId='6435b2b5-163c-4f0c-934e-7994da60dc89', vmId='adf14389-1563-4b1a-9af6-4b40370a825b'}', device='virtio', type='RNG', specParams='[source=urandom]', address='', managed='true', plugged='true', readOnly='false', deviceAlias='', customProperties='null', snapshotId='null', logicalName='null', hostDevice='null'}. This message repeats throughout engine.log: 2019-01-17 07:55:43,270-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-89) [] EVENT_ID: VM_MEMORY_UNDER_GUARANTEED_VALUE(148), VM HostedEngine on host ovirt-hv1.med.cornell.edu<http://ovirt-hv1.med.cornell.edu> was guaranteed 8192 MB but currently has 4224 MB As you can see attached the host has plenty of memory. Thank you Simone! Thanks, Douglas Duckworth, MSc, LFCS HPC System Administrator Scientific Computing Unit<https://scu.med.cornell.edu> Weill Cornell Medicine 1300 York Avenue New York, NY 10065 E: doug(a)med.cornell.edu<mailto:doug@med.cornell.edu> O: 212-746-6305 F: 212-746-8690 On Thu, Jan 17, 2019 at 5:09 AM Simone Tiraboschi <stirabos(a)redhat.com<mailto:stirabos@redhat.com>> wrote: On Wed, Jan 16, 2019 at 8:22 PM Douglas Duckworth <dod2014(a)med.cornell.edu<mailto:dod2014@med.cornell.edu>> wrote: Sorry for accidental send. Anyway I try to increase physical memory however it won't go above 4096MB. The hypervisor has 64GB. Do I need to modify this value with Hosted Engine offline? No, it's not required. Can you please attach your engine.log for the relevant time frame? Thanks, Douglas Duckworth, MSc, LFCS HPC System Administrator Scientific Computing Unit<https://scu.med.cornell.edu> Weill Cornell Medicine 1300 York Avenue New York, NY 10065 E: doug(a)med.cornell.edu<mailto:doug@med.cornell.edu> O: 212-746-6305 F: 212-746-8690 On Wed, Jan 16, 2019 at 1:58 PM Douglas Duckworth <dod2014(a)med.cornell.edu<mailto:dod2014@med.cornell.edu>> wrote: Hello I am trying to increase Hosted Engine physical memory above 4GB Thanks, Douglas Duckworth, MSc, LFCS HPC System Administrator Scientific Computing Unit<https://scu.med.cornell.edu> Weill Cornell Medicine 1300 York Avenue New York, NY 10065 E: doug(a)med.cornell.edu<mailto:doug@med.cornell.edu> O: 212-746-6305 F: 212-746-8690 _______________________________________________ Users mailing list -- users(a)ovirt.org<mailto:users@ovirt.org> To unsubscribe send an email to users-leave(a)ovirt.org<mailto:users-leave@ovirt.org> Privacy Statement: https://www.ovirt.org/site/privacy-policy/<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ovirt.org_site_pri…> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ovirt.org_communit…> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/WGSXQVVPJJ2CR…<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.ovirt.org_archiv…>

3 8

iSCSI Multipath/multiple gateways
by dan.poltawski＠tnp.net.uk 04 Sep '19

04 Sep '19

My iSCSI target (ceph-based) has multiple gateways, I’d like to ensure my hosted storage is aware of these such that each gateway can be rebooted for maintance without impacting service. What is the appropriate way to configure this so that each host knows of the multipath configuration? I do not have multiple network paths, just multiple gateways on the same network, so I think the multipath UI tool isn’t the way to do this. Any help/docs pointers appreciated. Regards, Dan

3 2

the virtual machine crushed and I cant shutdown the vm successfully
by zhouhao＠vip.friendtimes.net 04 Sep '19

04 Sep '19

Dear ovirt developers: The trouble bothered me for a long time;Some times my vm will crush and I cant shutdown it,I had to reboot the ovirt-node to slove it; the vm's error below The ovirt-node's error below The vm's threading on the ovirt-node, the IO ratio is 100% and the vm's process change to defunct I can not kill it,every time I had to shutdown the ovirt-node on the engine website ,the vm's status always in the way to shutdown ,even if I wait for it for hours; It either failed or is shutting down and the "power off" cant shutdown the vm too. ---------------------------------------------------------------- The other infomation about my ovirt-node node-version： node hardware zhouhao(a)vip.friendtimes.net

2 2