source vdsm.log:

2019-01-03 09:10:06,625+0800 INFO  (jsonrpc/5) [api.host] START getAllVmStats() from=::1,35448 (api:46)
2019-01-03 09:10:06,627+0800 INFO  (jsonrpc/5) [api.host] FINISH getAllVmStats return={'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} from=::1,35448 (api:52)
2019-01-03 09:10:06,627+0800 INFO  (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:573)
2019-01-03 09:10:06,633+0800 INFO  (jsonrpc/4) [api.host] START getAllVmIoTunePolicies() from=::1,35448 (api:46)
2019-01-03 09:10:06,633+0800 INFO  (jsonrpc/4) [api.host] FINISH getAllVmIoTunePolicies return={'status': {'message': 'Done', 'code': 0}, 'io_tune_policies_dict': {'eff7f697-8a07-46e5-a631-a1011a0eb836': {'policy': [], 'current_values': [{'ioTune': {'write_bytes_sec': 0L, 'total_iops_sec': 0L, 'read_iops_sec': 0L, 'read_bytes_sec': 0L, 'write_iops_sec': 0L, 'total_bytes_sec': 0L}, 'path': '/rhev/data-center/mnt/blockSD/f3c9971c-6b2d-435e-907c-b0d337e30cd2/images/d4b2c6ab-bd0a-44b3-9d07-48e5dc3f08b7/d82f2a5b-e1b7-4054-904f-9351c7446cf8', 'name': 'sda'}]}}} from=::1,35448 (api:52)
2019-01-03 09:10:06,633+0800 INFO  (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmIoTunePolicies succeeded in 0.00 seconds (__init__:573)
2019-01-03 09:10:07,228+0800 INFO  (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:573)
2019-01-03 09:10:07,231+0800 INFO  (jsonrpc/2) [api.virt] START getStats() from=::1,35438, vmId=c4d8cab1-54f3-4650-b0b4-bd07e3b8e310 (api:46)
2019-01-03 09:10:07,231+0800 INFO  (jsonrpc/2) [api] FINISH getStats error=Virtual machine does not exist: {'vmId': u'c4d8cab1-54f3-4650-b0b4-bd07e3b8e310'} (api:127)
2019-01-03 09:10:07,231+0800 INFO  (jsonrpc/2) [api.virt] FINISH getStats return={'status': {'message': "Virtual machine does not exist: {'vmId': u'c4d8cab1-54f3-4650-b0b4-bd07e3b8e310'}", 'code': 1}} from=::1,35438, vmId=c4d8cab1-54f3-4650-b0b4-bd07e3b8e310 (api:52)
2019-01-03 09:10:07,231+0800 INFO  (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call VM.getStats failed (error 1) in 0.00 seconds (__init__:573)
2019-01-03 09:10:07,370+0800 INFO  (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:573)
2019-01-03 09:10:07,373+0800 INFO  (jsonrpc/3) [api.host] START getStats() from=::1,35438 (api:46)
2019-01-03 09:10:07,374+0800 INFO  (jsonrpc/3) [vdsm.api] START repoStats(domains=()) from=::1,35438, task_id=4f5c57f9-a1df-49fe-bb35-7a5deffeb060 (api:46)
2019-01-03 09:10:07,374+0800 INFO  (jsonrpc/3) [vdsm.api] FINISH repoStats return={u'f3c9971c-6b2d-435e-907c-b0d337e30cd2': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.00102845', 'lastCheck': '0.1', 'valid': True}, u'52c9a48f-00e6-404c-920e-dd01e7738cda': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.001025', 'lastCheck': '3.1', 'valid': True}} from=::1,35438, task_id=4f5c57f9-a1df-49fe-bb35-7a5deffeb060 (api:52)
2019-01-03 09:10:07,374+0800 INFO  (jsonrpc/3) [vdsm.api] START multipath_health() from=::1,35438, task_id=84e20012-8e90-4705-8e1f-aab793654bdc (api:46)
2019-01-03 09:10:07,374+0800 INFO  (jsonrpc/3) [vdsm.api] FINISH multipath_health return={} from=::1,35438, task_id=84e20012-8e90-4705-8e1f-aab793654bdc (api:52)
2019-01-03 09:10:07,386+0800 INFO  (jsonrpc/3) [api.host] FINISH getStats return={'status': {'message': 'Done', 'code': 0}, 'info': {'cpuStatistics': {'11': {'cpuUser': '2.46', 'nodeIndex': 0, 'cpuSys': '0.53', 'cpuIdle': '97.01'}, '10': {'cpuUser': '1.06', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '98.67'}, '1': {'cpuUser': '1.06', 'nodeIndex': 0, 'cpuSys': '1.00', 'cpuIdle': '97.94'}, '0': {'cpuUser': '1.20', 'nodeIndex': 0, 'cpuSys': '0.93', 'cpuIdle': '97.87'}, '3': {'cpuUser': '0.93', 'nodeIndex': 0, 'cpuSys': '0.53', 'cpuIdle': '98.54'}, '2': {'cpuUser': '2.13', 'nodeIndex': 0, 'cpuSys': '0.66', 'cpuIdle': '97.21'}, '5': {'cpuUser': '1.53', 'nodeIndex': 0, 'cpuSys': '0.80', 'cpuIdle': '97.67'}, '4': {'cpuUser': '1.26', 'nodeIndex': 0, 'cpuSys': '0.86', 'cpuIdle': '97.88'}, '7': {'cpuUser': '0.73', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '99.00'}, '6': {'cpuUser': '0.53', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '99.14'}, '9': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '8': {'cpuUser': '2.46', 'nodeIndex': 0, 'cpuSys': '0.40', 'cpuIdle': '97.14'}}, 'numaNodeMemFree': {'0': {'memPercent': 34, 'memFree': '21359'}}, 'memShared': 0, 'haScore': 3400, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'vmCount': 1, 'memUsed': '31', 'storageDomains': {u'f3c9971c-6b2d-435e-907c-b0d337e30cd2': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.00102845', 'lastCheck': '0.1', 'valid': True}, u'52c9a48f-00e6-404c-920e-dd01e7738cda': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.001025', 'lastCheck': '3.1', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'eno2': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477796.5536, 'name': 'eno2', 'tx': '477564543', 'txDropped': '0', 'rx': '1928549178', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'eno1': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477796.5536, 'name': 'eno1', 'tx': '1908960485', 'txDropped': '0', 'rx': '1556808601', 'rxErrors': '4', 'speed': '1000', 'rxDropped': '0'}, 'ovirtmgmt': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477796.5536, 'name': 'ovirtmgmt', 'tx': '685727596', 'txDropped': '0', 'rx': '1298795728', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'lo': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477796.5536, 'name': 'lo', 'tx': '874675706451', 'txDropped': '0', 'rx': '874675706451', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'ens6f0': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477796.5536, 'name': 'ens6f0', 'tx': '2075274809499', 'txDropped': '0', 'rx': '646321835504', 'rxErrors': '30', 'speed': '10000', 'rxDropped': '0'}, 'ens6f1': {'txErrors': '0', 'state': 'down', 'sampleTime': 1546477796.5536, 'name': 'ens6f1', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'vm': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477796.5536, 'name': 'vm', 'tx': '1845465539', 'txDropped': '0', 'rx': '1243143624', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'vnet0': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477796.5536, 'name': 'vnet0', 'tx': '87922091', 'txDropped': '0', 'rx': '6887881', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, ';vdsmdummy;': {'txErrors': '0', 'state': 'down', 'sampleTime': 1546477796.5536, 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'br-int': {'txErrors': '0', 'state': 'down', 'sampleTime': 1546477796.5536, 'name': 'br-int', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'genev_sys_6081': {'txErrors': '8', 'state': 'up', 'sampleTime': 1546477796.5536, 'name': 'genev_sys_6081', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'ovs-system': {'txErrors': '0', 'state': 'down', 'sampleTime': 1546477796.5536, 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}}, 'txDropped': '0', 'anonHugePages': '5684', 'ksmPages': 100, 'elapsedTime': '1103588.58', 'cpuLoad': '0.26', 'cpuSys': '0.56', 'diskStats': {'/var/log': {'free': '70446'}, '/var/run/vdsm/': {'free': '15845'}, '/tmp': {'free': '70446'}}, 'cpuUserVdsmd': '0.86', 'netConfigDirty': 'False', 'memCommitted': 4161, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 21858, 'bootTime': '1545374161', 'haStats': {'active': True, 'configured': True, 'score': 3400, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'multipathHealth': {}, 'rxDropped': '0', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'hugepages': defaultdict(<type 'dict'>, {1048576: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}, 2048: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}), 'dateTime': '2019-01-03T01:10:07 GMT', 'cpuUser': '1.29', 'memFree': 22162, 'cpuIdle': '98.15', 'vmActive': 1, 'v2vJobs': {}, 'cpuSysVdsmd': '0.33'}} from=::1,35438 (api:52)
2019-01-03 09:10:07,387+0800 INFO  (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds (__init__:573)
2019-01-03 09:10:07,660+0800 INFO  (jsonrpc/0) [api.host] START getStats() from=::ffff:192.168.128.88,59098 (api:46)
2019-01-03 09:10:07,661+0800 INFO  (jsonrpc/0) [vdsm.api] START repoStats(domains=()) from=::ffff:192.168.128.88,59098, task_id=6cd450cd-51c0-4d11-8fde-dca66f0608bb (api:46)
2019-01-03 09:10:07,661+0800 INFO  (jsonrpc/0) [vdsm.api] FINISH repoStats return={u'f3c9971c-6b2d-435e-907c-b0d337e30cd2': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.00102845', 'lastCheck': '0.4', 'valid': True}, u'52c9a48f-00e6-404c-920e-dd01e7738cda': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.001025', 'lastCheck': '3.4', 'valid': True}} from=::ffff:192.168.128.88,59098, task_id=6cd450cd-51c0-4d11-8fde-dca66f0608bb (api:52)
2019-01-03 09:10:07,661+0800 INFO  (jsonrpc/0) [vdsm.api] START multipath_health() from=::ffff:192.168.128.88,59098, task_id=7c475633-f82d-4bbd-bb5f-1afd94f71350 (api:46)
2019-01-03 09:10:07,661+0800 INFO  (jsonrpc/0) [vdsm.api] FINISH multipath_health return={} from=::ffff:192.168.128.88,59098, task_id=7c475633-f82d-4bbd-bb5f-1afd94f71350 (api:52)
2019-01-03 09:10:07,672+0800 INFO  (jsonrpc/0) [api.host] FINISH getStats return={'status': {'message': 'Done', 'code': 0}, 'info': {'cpuStatistics': {'11': {'cpuUser': '2.46', 'nodeIndex': 0, 'cpuSys': '0.53', 'cpuIdle': '97.01'}, '10': {'cpuUser': '1.06', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '98.67'}, '1': {'cpuUser': '1.06', 'nodeIndex': 0, 'cpuSys': '1.00', 'cpuIdle': '97.94'}, '0': {'cpuUser': '1.20', 'nodeIndex': 0, 'cpuSys': '0.93', 'cpuIdle': '97.87'}, '3': {'cpuUser': '0.93', 'nodeIndex': 0, 'cpuSys': '0.53', 'cpuIdle': '98.54'}, '2': {'cpuUser': '2.13', 'nodeIndex': 0, 'cpuSys': '0.66', 'cpuIdle': '97.21'}, '5': {'cpuUser': '1.53', 'nodeIndex': 0, 'cpuSys': '0.80', 'cpuIdle': '97.67'}, '4': {'cpuUser': '1.26', 'nodeIndex': 0, 'cpuSys': '0.86', 'cpuIdle': '97.88'}, '7': {'cpuUser': '0.73', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '99.00'}, '6': {'cpuUser': '0.53', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '99.14'}, '9': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '8': {'cpuUser': '2.46', 'nodeIndex': 0, 'cpuSys': '0.40', 'cpuIdle': '97.14'}}, 'numaNodeMemFree': {'0': {'memPercent': 34, 'memFree': '21359'}}, 'memShared': 0, 'haScore': 3400, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'vmCount': 1, 'memUsed': '31', 'storageDomains': {u'f3c9971c-6b2d-435e-907c-b0d337e30cd2': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.00102845', 'lastCheck': '0.4', 'valid': True}, u'52c9a48f-00e6-404c-920e-dd01e7738cda': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.001025', 'lastCheck': '3.4', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'eno2': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477796.5536, 'name': 'eno2', 'tx': '477564543', 'txDropped': '0', 'rx': '1928549178', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'eno1': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477796.5536, 'name': 'eno1', 'tx': '1908960485', 'txDropped': '0', 'rx': '1556808601', 'rxErrors': '4', 'speed': '1000', 'rxDropped': '0'}, 'ovirtmgmt': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477796.5536, 'name': 'ovirtmgmt', 'tx': '685727596', 'txDropped': '0', 'rx': '1298795728', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'lo': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477796.5536, 'name': 'lo', 'tx': '874675706451', 'txDropped': '0', 'rx': '874675706451', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'ens6f0': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477796.5536, 'name': 'ens6f0', 'tx': '2075274809499', 'txDropped': '0', 'rx': '646321835504', 'rxErrors': '30', 'speed': '10000', 'rxDropped': '0'}, 'ens6f1': {'txErrors': '0', 'state': 'down', 'sampleTime': 1546477796.5536, 'name': 'ens6f1', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'vm': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477796.5536, 'name': 'vm', 'tx': '1845465539', 'txDropped': '0', 'rx': '1243143624', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'vnet0': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477796.5536, 'name': 'vnet0', 'tx': '87922091', 'txDropped': '0', 'rx': '6887881', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, ';vdsmdummy;': {'txErrors': '0', 'state': 'down', 'sampleTime': 1546477796.5536, 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'br-int': {'txErrors': '0', 'state': 'down', 'sampleTime': 1546477796.5536, 'name': 'br-int', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'genev_sys_6081': {'txErrors': '8', 'state': 'up', 'sampleTime': 1546477796.5536, 'name': 'genev_sys_6081', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'ovs-system': {'txErrors': '0', 'state': 'down', 'sampleTime': 1546477796.5536, 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}}, 'txDropped': '0', 'anonHugePages': '5684', 'ksmPages': 100, 'elapsedTime': '1103588.87', 'cpuLoad': '0.26', 'cpuSys': '0.56', 'diskStats': {'/var/log': {'free': '70446'}, '/var/run/vdsm/': {'free': '15845'}, '/tmp': {'free': '70446'}}, 'cpuUserVdsmd': '0.86', 'netConfigDirty': 'False', 'memCommitted': 4161, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 21858, 'bootTime': '1545374161', 'haStats': {'active': True, 'configured': True, 'score': 3400, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'multipathHealth': {}, 'rxDropped': '0', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'hugepages': defaultdict(<type 'dict'>, {1048576: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}, 2048: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}), 'dateTime': '2019-01-03T01:10:07 GMT', 'cpuUser': '1.29', 'memFree': 22162, 'cpuIdle': '98.15', 'vmActive': 1, 'v2vJobs': {}, 'cpuSysVdsmd': '0.33'}} from=::ffff:192.168.128.88,59098 (api:52)
2019-01-03 09:10:07,673+0800 INFO  (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds (__init__:573)
2019-01-03 09:10:08,311+0800 INFO  (jsonrpc/7) [root] /usr/libexec/vdsm/hooks/after_get_caps/50_openstacknet: rc=0 err= (hooks:110)
2019-01-03 09:10:08,445+0800 INFO  (jsonrpc/5) [api.virt] START migrate(params={u'incomingLimit': 2, u'src': u'horeb66', u'dstqemu': u'192.168.128.78', u'autoConverge': u'true', u'tunneled': u'false', u'enableGuestEvents': True, u'dst': u'horeb65:54321', u'convergenceSchedule': {u'init': [{u'params': [u'100'], u'name': u'setDowntime'}], u'stalling': [{u'action': {u'params': [u'150'], u'name': u'setDowntime'}, u'limit': 1}, {u'action': {u'params': [u'200'], u'name': u'setDowntime'}, u'limit': 2}, {u'action': {u'params': [u'300'], u'name': u'setDowntime'}, u'limit': 3}, {u'action': {u'params': [u'400'], u'name': u'setDowntime'}, u'limit': 4}, {u'action': {u'params': [u'500'], u'name': u'setDowntime'}, u'limit': 6}, {u'action': {u'params': [], u'name': u'abort'}, u'limit': -1}]}, u'vmId': u'eff7f697-8a07-46e5-a631-a1011a0eb836', u'abortOnError': u'true', u'outgoingLimit': 2, u'compressed': u'false', u'maxBandwidth': 500, u'method': u'online'}) from=::ffff:192.168.128.88,59098, flow_id=658e335e-7d3e-438b-b1ff-bdf940ba5459, vmId=eff7f697-8a07-46e5-a631-a1011a0eb836 (api:46)
2019-01-03 09:10:08,446+0800 INFO  (jsonrpc/5) [api.virt] FINISH migrate return={'status': {'message': 'Migration in progress', 'code': 0}, 'progress': 0} from=::ffff:192.168.128.88,59098, flow_id=658e335e-7d3e-438b-b1ff-bdf940ba5459, vmId=eff7f697-8a07-46e5-a631-a1011a0eb836 (api:52)
2019-01-03 09:10:08,447+0800 INFO  (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call VM.migrate succeeded in 0.00 seconds (__init__:573)
2019-01-03 09:10:08,449+0800 ERROR (migsrc/eff7f697) [virt.vm] (vmId='eff7f697-8a07-46e5-a631-a1011a0eb836') [Errno -2] Name or service not known (migration:290)
2019-01-03 09:10:08,450+0800 ERROR (migsrc/eff7f697) [virt.vm] (vmId='eff7f697-8a07-46e5-a631-a1011a0eb836') Failed to migrate (migration:455)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/virt/migration.py", line 409, in _regular_run
    self._setupVdsConnection()
  File "/usr/lib/python2.7/site-packages/vdsm/virt/migration.py", line 233, in _setupVdsConnection
    client = self._createClient(port)
  File "/usr/lib/python2.7/site-packages/vdsm/virt/migration.py", line 221, in _createClient
    client_socket = utils.create_connected_socket(host, int(port), sslctx)
  File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 433, in create_connected_socket
    socket.AF_UNSPEC, socket.SOCK_STREAM)
gaierror: [Errno -2] Name or service not known
2019-01-03 09:10:08,526+0800 INFO  (jsonrpc/7) [root] /usr/libexec/vdsm/hooks/after_get_caps/openstacknet_utils.py: rc=0 err= (hooks:110)
2019-01-03 09:10:08,718+0800 INFO  (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:573)
2019-01-03 09:10:08,720+0800 INFO  (jsonrpc/6) [api.virt] START getStats() from=::1,35438, vmId=c4d8cab1-54f3-4650-b0b4-bd07e3b8e310 (api:46)
2019-01-03 09:10:08,721+0800 INFO  (jsonrpc/6) [api] FINISH getStats error=Virtual machine does not exist: {'vmId': u'c4d8cab1-54f3-4650-b0b4-bd07e3b8e310'} (api:127)
2019-01-03 09:10:08,721+0800 INFO  (jsonrpc/6) [api.virt] FINISH getStats return={'status': {'message': "Virtual machine does not exist: {'vmId': u'c4d8cab1-54f3-4650-b0b4-bd07e3b8e310'}", 'code': 1}} from=::1,35438, vmId=c4d8cab1-54f3-4650-b0b4-bd07e3b8e310 (api:52)
2019-01-03 09:10:08,721+0800 INFO  (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call VM.getStats failed (error 1) in 0.00 seconds (__init__:573)
2019-01-03 09:10:08,768+0800 INFO  (jsonrpc/7) [root] /usr/libexec/vdsm/hooks/after_get_caps/ovirt_provider_ovn_hook: rc=0 err= (hooks:110)
2019-01-03 09:10:08,769+0800 INFO  (jsonrpc/7) [api.host] FINISH getCapabilities return={'status': {'message': 'Done', 'code': 0}, 'info': {u'HBAInventory': {u'iSCSI': [{u'InitiatorName': u'iqn.1994-05.com.redhat:1c9838d4285'}], u'FC': []}, u'packages2': {u'kernel': {u'release': u'862.14.4.el7.x86_64', u'version': u'3.10.0'}, u'spice-server': {u'release': u'2.el7_5.5', u'version': u'0.14.0'}, u'librbd1': {u'release': u'0.el7', u'version': u'12.2.10'}, u'vdsm': {u'release': u'1.el7', u'version': u'4.20.39.1'}, u'qemu-kvm': {u'release': u'21.el7_5.4.1', u'version': u'2.10.0'}, u'openvswitch': {u'release': u'4.el7', u'version': u'2.9.0'}, u'libvirt': {u'release': u'14.el7_5.8', u'version': u'3.9.0'}, u'ovirt-hosted-engine-ha': {u'release': u'1.el7', u'version': u'2.2.16'}, u'qemu-img': {u'release': u'21.el7_5.4.1', u'version': u'2.10.0'}, u'mom': {u'release': u'1.el7.centos', u'version': u'0.5.12'}, u'glusterfs-cli': {u'release': u'1.el7', u'version': u'3.12.14'}}, u'numaNodeDistance': {u'0': [10]}, u'cpuModel': u'Intel(R) Xeon(R) CPU E5-2430 v2 @ 2.50GHz', u'nestedVirtualization': False, u'liveMerge': u'true', u'hooks': {u'after_vm_start': {u'openstacknet_utils.py': {u'md5': u'640ef8b473cbc16b102c5ebf57ea533f'}, u'50_openstacknet': {u'md5': u'ea0a5a715da8c1badbcda28e8b8fa00e'}}, u'after_network_setup': {u'30_ethtool_options': {u'md5': u'f04c2ca5dce40663e2ed69806eea917c'}}, u'before_vm_start': {u'50_hostedengine': {u'md5': u'95c810cdcfe4195302a59574a5148289'}, u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'after_device_migrate_destination': {u'openstacknet_utils.py': {u'md5': u'640ef8b473cbc16b102c5ebf57ea533f'}, u'50_openstacknet': {u'md5': u'f93c12af7454bbba4c1ef445c2bc9860'}}, u'before_device_migrate_destination': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}}, u'before_device_create': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}, u'openstacknet_utils.py': {u'md5': u'640ef8b473cbc16b102c5ebf57ea533f'}, u'50_openstacknet': {u'md5': u'0438de5ff9b6bf8d3160804ff71bf827'}, u'20_ovirt_provider_ovn_vhostuser_hook': {u'md5': u'6092f45748b23644101c04ba025ddec5'}, u'10_ovirt_provider_ovn_hook': {u'md5': u'ada7e4e757fd0241b682b5cc6a545fc8'}}, u'after_device_create': {u'openstacknet_utils.py': {u'md5': u'640ef8b473cbc16b102c5ebf57ea533f'}, u'50_openstacknet': {u'md5': u'f93c12af7454bbba4c1ef445c2bc9860'}}, u'after_nic_hotplug': {u'openstacknet_utils.py': {u'md5': u'640ef8b473cbc16b102c5ebf57ea533f'}, u'50_openstacknet': {u'md5': u'f93c12af7454bbba4c1ef445c2bc9860'}}, u'before_vm_migrate_destination': {u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'before_vm_dehibernate': {u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'before_nic_hotplug': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}, u'openstacknet_utils.py': {u'md5': u'640ef8b473cbc16b102c5ebf57ea533f'}, u'50_openstacknet': {u'md5': u'0438de5ff9b6bf8d3160804ff71bf827'}, u'20_ovirt_provider_ovn_vhostuser_hook': {u'md5': u'6092f45748b23644101c04ba025ddec5'}, u'10_ovirt_provider_ovn_hook': {u'md5': u'ada7e4e757fd0241b682b5cc6a545fc8'}}, u'after_get_caps': {u'openstacknet_utils.py': {u'md5': u'640ef8b473cbc16b102c5ebf57ea533f'}, u'50_openstacknet': {u'md5': u'0e00c63ab44a952e722209ea31fd7a71'}, u'ovirt_provider_ovn_hook': {u'md5': u'4c4b1d2d5460e6a65114ae36cb775df6'}}, u'before_network_setup': {u'50_fcoe': {u'md5': u'28c352339c8beef1e1b05c67d106d062'}}, u'after_vm_destroy': {u'delete_vhostuserclient_hook': {u'md5': u'cb68e366da312b942a9a8518f3174df9'}, u'50_vhostmd': {u'md5': u'd70f7ee0453f632e87c09a157ff8ff66'}}}, u'supportsIPv6': True, u'realtimeKernel': False, u'vmTypes': [u'kvm'], u'liveSnapshot': u'true', u'cpuThreads': u'12', u'kdumpStatus': 0, u'networks': {u'ovirtmgmt': {u'iface': u'ovirtmgmt', u'ipv6autoconf': True, u'addr': u'192.168.128.77', u'dhcpv6': False, u'ipv6addrs': [], u'switch': u'legacy', u'bridged': True, u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'255.255.255.0', u'ipv4defaultroute': False, u'stp': u'off', u'ipv4addrs': [u'192.168.128.77/24'], u'ipv6gateway': u'::', u'gateway': u'', u'ports': [u'eno2', u'vnet0']}, u'vm': {u'iface': u'vm', u'ipv6autoconf': False, u'addr': u'192.168.1.66', u'dhcpv6': False, u'ipv6addrs': [], u'switch': u'legacy', u'bridged': True, u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'255.255.255.0', u'ipv4defaultroute': True, u'stp': u'off', u'ipv4addrs': [u'192.168.1.66/24'], u'ipv6gateway': u'::', u'gateway': u'192.168.1.1', u'ports': [u'eno1']}}, u'kernelArgs': u'BOOT_IMAGE=/vmlinuz-3.10.0-862.14.4.el7.x86_64 root=UUID=477d201f-3595-4d61-8f06-2ca97f3dadc0 ro crashkernel=auto rhgb quiet LANG=en_US.UTF-8', u'bridges': {u'ovirtmgmt': {u'ipv6autoconf': True, u'addr': u'192.168.128.77', u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'255.255.255.0', u'ipv4defaultroute': False, u'stp': u'off', u'ipv4addrs': [u'192.168.128.77/24'], u'ipv6gateway': u'::', u'gateway': u'', u'opts': {u'multicast_last_member_count': u'2', u'vlan_protocol': u'0x8100', u'hash_elasticity': u'4', u'multicast_query_response_interval': u'1000', u'group_fwd_mask': u'0x0', u'multicast_snooping': u'1', u'multicast_startup_query_interval': u'3125', u'hello_timer': u'0', u'multicast_querier_interval': u'25500', u'max_age': u'2000', u'hash_max': u'512', u'stp_state': u'0', u'topology_change_detected': u'0', u'priority': u'32768', u'multicast_igmp_version': u'2', u'multicast_membership_interval': u'26000', u'root_path_cost': u'0', u'root_port': u'0', u'multicast_stats_enabled': u'0', u'multicast_startup_query_count': u'2', u'nf_call_iptables': u'0', u'vlan_stats_enabled': u'0', u'hello_time': u'200', u'topology_change': u'0', u'bridge_id': u'8000.001e67da4401', u'topology_change_timer': u'0', u'ageing_time': u'30000', u'nf_call_ip6tables': u'0', u'multicast_mld_version': u'1', u'gc_timer': u'2918', u'root_id': u'8000.001e67da4401', u'nf_call_arptables': u'0', u'group_addr': u'1:80:c2:0:0:0', u'multicast_last_member_interval': u'100', u'default_pvid': u'1', u'multicast_query_interval': u'12500', u'multicast_query_use_ifaddr': u'0', u'tcn_timer': u'0', u'multicast_router': u'1', u'vlan_filtering': u'0', u'multicast_querier': u'0', u'forward_delay': u'0'}, u'ports': [u'eno2', u'vnet0']}, u'vm': {u'ipv6autoconf': False, u'addr': u'192.168.1.66', u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'255.255.255.0', u'ipv4defaultroute': True, u'stp': u'off', u'ipv4addrs': [u'192.168.1.66/24'], u'ipv6gateway': u'::', u'gateway': u'192.168.1.1', u'opts': {u'multicast_last_member_count': u'2', u'vlan_protocol': u'0x8100', u'hash_elasticity': u'4', u'multicast_query_response_interval': u'1000', u'group_fwd_mask': u'0x0', u'multicast_snooping': u'1', u'multicast_startup_query_interval': u'3125', u'hello_timer': u'0', u'multicast_querier_interval': u'25500', u'max_age': u'2000', u'hash_max': u'512', u'stp_state': u'0', u'topology_change_detected': u'0', u'priority': u'32768', u'multicast_igmp_version': u'2', u'multicast_membership_interval': u'26000', u'root_path_cost': u'0', u'root_port': u'0', u'multicast_stats_enabled': u'0', u'multicast_startup_query_count': u'2', u'nf_call_iptables': u'0', u'vlan_stats_enabled': u'0', u'hello_time': u'200', u'topology_change': u'0', u'bridge_id': u'8000.001e67da4400', u'topology_change_timer': u'0', u'ageing_time': u'30000', u'nf_call_ip6tables': u'0', u'multicast_mld_version': u'1', u'gc_timer': u'2918', u'root_id': u'8000.001e67da4400', u'nf_call_arptables': u'0', u'group_addr': u'1:80:c2:0:0:0', u'multicast_last_member_interval': u'100', u'default_pvid': u'1', u'multicast_query_interval': u'12500', u'multicast_query_use_ifaddr': u'0', u'tcn_timer': u'0', u'multicast_router': u'1', u'vlan_filtering': u'0', u'multicast_querier': u'0', u'forward_delay': u'0'}, u'ports': [u'eno1']}}, u'uuid': u'BE83EC95-ECA4-E411-BBFA-001E67DA4400', u'onlineCpus': u'0,1,2,3,4,5,6,7,8,9,10,11', u'nameservers': [u'192.168.1.1'], u'nics': {u'eno1': {u'ipv6autoconf': False, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'00:1e:67:da:44:00', u'ipv6gateway': u'::', u'gateway': u''}, u'ens6f0': {u'ipv6autoconf': True, u'addr': u'192.168.10.66', u'speed': 10000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'9000', u'dhcpv4': False, u'netmask': u'255.255.255.0', u'ipv4defaultroute': False, u'ipv4addrs': [u'192.168.10.66/24'], u'hwaddr': u'90:e2:ba:92:bd:58', u'ipv6gateway': u'fe80::1:1', u'gateway': u''}, u'ens6f1': {u'ipv6autoconf': True, u'addr': u'', u'speed': 0, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'90:e2:ba:92:bd:5a', u'ipv6gateway': u'::', u'gateway': u''}, u'eno2': {u'ipv6autoconf': False, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'00:1e:67:da:44:01', u'ipv6gateway': u'::', u'gateway': u''}}, u'software_revision': u'1', u'hostdevPassthrough': u'false', u'clusterLevels': [u'3.6', u'4.0', u'4.1', u'4.2'], u'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfmperf,eagerfpu,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,dca,sse4_1,sse4_2,x2apic,popcnt,tsc_deadline_timer,aes,xsave,avx,f16c,rdrand,lahf_lm,epb,intel_ppin,tpr_shadow,vnmi,flexpriority,ept,vpid,fsgsbase,smep,erms,xsaveopt,dtherm,ida,arat,pln,pts,model_n270,model_coreduo,model_Nehalem,model_core2duo,model_IvyBridge,model_Penryn,model_Westmere,model_SandyBridge,model_Conroe', u'kernelFeatures': {u'RETP': 1, u'IBRS': 0, u'PTI': 1}, u'ISCSIInitiatorName': u'iqn.1994-05.com.redhat:1c9838d4285', u'netConfigDirty': u'False', u'selinux': {u'mode': u'-1'}, u'autoNumaBalancing': 0, u'reservedMem': u'321', u'containers': False, u'bondings': {}, u'software_version': u'4.20', u'supportedENGINEs': [u'3.6', u'4.0', u'4.1', u'4.2'], u'cpuSpeed': u'2904.968', u'numaNodes': {u'0': {u'totalMemory': u'31902', u'cpus': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]}}, u'cpuSockets': u'1', u'vlans': {}, u'version_name': u'Snow Man', 'lastClientIface': 'lo', u'cpuCores': u'6', u'hostedEngineDeployed': True, u'hugepages': [1048576, 2048], u'guestOverhead': u'65', u'additionalFeatures': [u'libgfapi_supported'], u'openstack_binding_host_ids': {u'OVIRT_PROVIDER_OVN': u'0052edeb-c48c-4b70-aa75-7f068f045ad7'}, u'kvmEnabled': u'true', u'memSize': u'31902', u'emulatedMachines': [u'pc-i440fx-rhel7.1.0', u'pc-q35-rhel7.3.0', u'rhel6.3.0', u'pc-i440fx-rhel7.5.0', u'pc-i440fx-rhel7.0.0', u'rhel6.1.0', u'pc-i440fx-rhel7.4.0', u'rhel6.6.0', u'pc-q35-rhel7.5.0', u'rhel6.2.0', u'pc', u'pc-i440fx-rhel7.3.0', u'q35', u'pc-i440fx-rhel7.2.0', u'rhel6.4.0', u'pc-q35-rhel7.4.0', u'rhel6.0.0', u'rhel6.5.0'], u'rngSources': [u'hwrng', u'random'], u'operatingSystem': {u'release': u'5.1804.el7.centos', u'pretty_name': u'CentOS Linux 7 (Core)', u'version': u'7', u'name': u'RHEL'}}} from=::1,35438 (api:52)
2019-01-03 09:10:08,782+0800 INFO  (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.getCapabilities succeeded in 2.94 seconds (__init__:573)
2019-01-03 09:10:11,630+0800 INFO  (periodic/2) [vdsm.api] START repoStats(domains=()) from=internal, task_id=615fe1c2-4d34-4fdf-a13e-71df6a8fa77b (api:46)
2019-01-03 09:10:11,630+0800 INFO  (periodic/2) [vdsm.api] FINISH repoStats return={u'f3c9971c-6b2d-435e-907c-b0d337e30cd2': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.00102845', 'lastCheck': '4.4', 'valid': True}, u'52c9a48f-00e6-404c-920e-dd01e7738cda': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.001025', 'lastCheck': '7.4', 'valid': True}} from=internal, task_id=615fe1c2-4d34-4fdf-a13e-71df6a8fa77b (api:52)
2019-01-03 09:10:11,631+0800 INFO  (periodic/2) [vdsm.api] START multipath_health() from=internal, task_id=8855dd8d-35f4-4a1b-be18-a43c73b6941e (api:46)
2019-01-03 09:10:11,631+0800 INFO  (periodic/2) [vdsm.api] FINISH multipath_health return={} from=internal, task_id=8855dd8d-35f4-4a1b-be18-a43c73b6941e (api:52)
2019-01-03 09:10:11,775+0800 INFO  (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:573)
2019-01-03 09:10:11,778+0800 INFO  (jsonrpc/1) [vdsm.api] START repoStats(domains=[u'52c9a48f-00e6-404c-920e-dd01e7738cda']) from=::1,35438, task_id=79acc9d4-bee9-4b49-b5b3-561bafac2e3a (api:46)
2019-01-03 09:10:11,779+0800 INFO  (jsonrpc/1) [vdsm.api] FINISH repoStats return={u'52c9a48f-00e6-404c-920e-dd01e7738cda': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.001025', 'lastCheck': '7.5', 'valid': True}} from=::1,35438, task_id=79acc9d4-bee9-4b49-b5b3-561bafac2e3a (api:52)
2019-01-03 09:10:11,779+0800 INFO  (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.getStorageRepoStats succeeded in 0.01 seconds (__init__:573)
2019-01-03 09:10:14,715+0800 INFO  (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:573)
2019-01-03 09:10:14,850+0800 INFO  (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:573)
2019-01-03 09:10:14,854+0800 INFO  (jsonrpc/5) [vdsm.api] START repoStats(domains=[u'52c9a48f-00e6-404c-920e-dd01e7738cda']) from=::1,35438, task_id=f00ce3ce-8dd1-4189-b139-ffb5e1ac2c0e (api:46)
2019-01-03 09:10:14,854+0800 INFO  (jsonrpc/5) [vdsm.api] FINISH repoStats return={u'52c9a48f-00e6-404c-920e-dd01e7738cda': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000774137', 'lastCheck': '0.6', 'valid': True}} from=::1,35438, task_id=f00ce3ce-8dd1-4189-b139-ffb5e1ac2c0e (api:52)
2019-01-03 09:10:14,854+0800 INFO  (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:573)
2019-01-03 09:10:14,858+0800 INFO  (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:573)
2019-01-03 09:10:15,817+0800 INFO  (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:573)
2019-01-03 09:10:15,820+0800 INFO  (jsonrpc/7) [api.host] START getCapabilities() from=::1,35438 (api:46)
2019-01-03 09:10:17,189+0800 INFO  (jsonrpc/2) [api.host] START getAllVmStats() from=::ffff:192.168.128.88,59098 (api:46)
2019-01-03 09:10:17,190+0800 INFO  (jsonrpc/2) [api.host] FINISH getAllVmStats return={'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} from=::ffff:192.168.128.88,59098 (api:52)
2019-01-03 09:10:17,190+0800 INFO  (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.01 seconds (__init__:573)
2019-01-03 09:10:17,235+0800 INFO  (jsonrpc/1) [api.virt] START getMigrationStatus() from=::ffff:192.168.128.88,59098, vmId=eff7f697-8a07-46e5-a631-a1011a0eb836 (api:46)
2019-01-03 09:10:17,236+0800 INFO  (jsonrpc/1) [api.virt] FINISH getMigrationStatus return={'status': {'message': 'Done', 'code': 0}, 'migrationStats': {'status': {'message': 'Fatal error during migration', 'code': 12}, 'progress': 0}} from=::ffff:192.168.128.88,59098, vmId=eff7f697-8a07-46e5-a631-a1011a0eb836 (api:52)
2019-01-03 09:10:17,236+0800 INFO  (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call VM.getMigrationStatus succeeded in 0.00 seconds (__init__:573)
2019-01-03 09:10:17,413+0800 INFO  (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:573)
2019-01-03 09:10:17,416+0800 INFO  (jsonrpc/0) [api.host] START getStats() from=::1,35438 (api:46)
2019-01-03 09:10:17,417+0800 INFO  (jsonrpc/0) [vdsm.api] START repoStats(domains=()) from=::1,35438, task_id=83ea7d93-1b32-469e-b0b5-2b885d8722cf (api:46)
2019-01-03 09:10:17,417+0800 INFO  (jsonrpc/0) [vdsm.api] FINISH repoStats return={u'f3c9971c-6b2d-435e-907c-b0d337e30cd2': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.00102407', 'lastCheck': '0.2', 'valid': True}, u'52c9a48f-00e6-404c-920e-dd01e7738cda': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000774137', 'lastCheck': '3.2', 'valid': True}} from=::1,35438, task_id=83ea7d93-1b32-469e-b0b5-2b885d8722cf (api:52)
2019-01-03 09:10:17,417+0800 INFO  (jsonrpc/0) [vdsm.api] START multipath_health() from=::1,35438, task_id=465f5792-be39-4724-b3c5-6619492534c9 (api:46)
2019-01-03 09:10:17,417+0800 INFO  (jsonrpc/0) [vdsm.api] FINISH multipath_health return={} from=::1,35438, task_id=465f5792-be39-4724-b3c5-6619492534c9 (api:52)
2019-01-03 09:10:17,429+0800 INFO  (jsonrpc/0) [api.host] FINISH getStats return={'status': {'message': 'Done', 'code': 0}, 'info': {'cpuStatistics': {'11': {'cpuUser': '2.86', 'nodeIndex': 0, 'cpuSys': '0.73', 'cpuIdle': '96.41'}, '10': {'cpuUser': '1.39', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '98.28'}, '1': {'cpuUser': '1.93', 'nodeIndex': 0, 'cpuSys': '1.46', 'cpuIdle': '96.61'}, '0': {'cpuUser': '1.06', 'nodeIndex': 0, 'cpuSys': '0.60', 'cpuIdle': '98.34'}, '3': {'cpuUser': '0.73', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '98.94'}, '2': {'cpuUser': '2.13', 'nodeIndex': 0, 'cpuSys': '0.86', 'cpuIdle': '97.01'}, '5': {'cpuUser': '1.06', 'nodeIndex': 0, 'cpuSys': '0.53', 'cpuIdle': '98.41'}, '4': {'cpuUser': '0.53', 'nodeIndex': 0, 'cpuSys': '0.60', 'cpuIdle': '98.87'}, '7': {'cpuUser': '1.06', 'nodeIndex': 0, 'cpuSys': '0.46', 'cpuIdle': '98.48'}, '6': {'cpuUser': '0.33', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.47'}, '9': {'cpuUser': '0.93', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '98.87'}, '8': {'cpuUser': '5.45', 'nodeIndex': 0, 'cpuSys': '0.66', 'cpuIdle': '93.89'}}, 'numaNodeMemFree': {'0': {'memPercent': 34, 'memFree': '21355'}}, 'memShared': 0, 'haScore': 3400, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'vmCount': 1, 'memUsed': '31', 'storageDomains': {u'f3c9971c-6b2d-435e-907c-b0d337e30cd2': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.00102407', 'lastCheck': '0.2', 'valid': True}, u'52c9a48f-00e6-404c-920e-dd01e7738cda': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000774137', 'lastCheck': '3.2', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'eno2': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477811.610592, 'name': 'eno2', 'tx': '477576468', 'txDropped': '0', 'rx': '1928559759', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'eno1': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477811.610592, 'name': 'eno1', 'tx': '1908986682', 'txDropped': '0', 'rx': '1556817423', 'rxErrors': '4', 'speed': '1000', 'rxDropped': '0'}, 'ovirtmgmt': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477811.610592, 'name': 'ovirtmgmt', 'tx': '685738386', 'txDropped': '0', 'rx': '1298804469', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'lo': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477811.610592, 'name': 'lo', 'tx': '874688360632', 'txDropped': '0', 'rx': '874688360632', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'ens6f0': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477811.610592, 'name': 'ens6f0', 'tx': '2075304447646', 'txDropped': '0', 'rx': '646331235009', 'rxErrors': '30', 'speed': '10000', 'rxDropped': '0'}, 'ens6f1': {'txErrors': '0', 'state': 'down', 'sampleTime': 1546477811.610592, 'name': 'ens6f1', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'vm': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477811.610592, 'name': 'vm', 'tx': '1845490878', 'txDropped': '0', 'rx': '1243150990', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'vnet0': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477811.610592, 'name': 'vnet0', 'tx': '87929548', 'txDropped': '0', 'rx': '6888752', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, ';vdsmdummy;': {'txErrors': '0', 'state': 'down', 'sampleTime': 1546477811.610592, 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'br-int': {'txErrors': '0', 'state': 'down', 'sampleTime': 1546477811.610592, 'name': 'br-int', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'genev_sys_6081': {'txErrors': '8', 'state': 'up', 'sampleTime': 1546477811.610592, 'name': 'genev_sys_6081', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'ovs-system': {'txErrors': '0', 'state': 'down', 'sampleTime': 1546477811.610592, 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}}, 'txDropped': '0', 'anonHugePages': '5672', 'ksmPages': 100, 'elapsedTime': '1103598.62', 'cpuLoad': '0.40', 'cpuSys': '0.59', 'diskStats': {'/var/log': {'free': '70446'}, '/var/run/vdsm/': {'free': '15845'}, '/tmp': {'free': '70446'}}, 'cpuUserVdsmd': '1.06', 'netConfigDirty': 'False', 'memCommitted': 4161, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 21862, 'bootTime': '1545374161', 'haStats': {'active': True, 'configured': True, 'score': 3400, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'multipathHealth': {}, 'rxDropped': '0', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'hugepages': defaultdict(<type 'dict'>, {1048576: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}, 2048: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}), 'dateTime': '2019-01-03T01:10:17 GMT', 'cpuUser': '1.63', 'memFree': 22166, 'cpuIdle': '97.78', 'vmActive': 1, 'v2vJobs': {}, 'cpuSysVdsmd': '0.53'}} from=::1,35438 (api:52)
2019-01-03 09:10:17,430+0800 INFO  (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.02 seconds (__init__:573)
2019-01-03 09:10:18,291+0800 INFO  (jsonrpc/7) [root] /usr/libexec/vdsm/hooks/after_get_caps/50_openstacknet: rc=0 err= (hooks:110)

destination vdsm.log:

2019-01-03 09:10:05,920+0800 INFO  (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:573)
2019-01-03 09:10:07,288+0800 INFO  (jsonrpc/1) [api.host] START getAllVmStats() from=::1,56728 (api:46)
2019-01-03 09:10:07,289+0800 INFO  (jsonrpc/1) [throttled] Current getAllVmStats: {} (throttledlog:103)
2019-01-03 09:10:07,289+0800 INFO  (jsonrpc/1) [api.host] FINISH getAllVmStats return={'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} from=::1,56728 (api:52)
2019-01-03 09:10:07,289+0800 INFO  (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:573)
2019-01-03 09:10:07,660+0800 INFO  (jsonrpc/0) [api.host] START getStats() from=::ffff:192.168.128.88,38760 (api:46)
2019-01-03 09:10:07,661+0800 INFO  (jsonrpc/0) [vdsm.api] START repoStats(domains=()) from=::ffff:192.168.128.88,38760, task_id=954d8da4-e71d-4cac-819e-8494dbc1b938 (api:46)
2019-01-03 09:10:07,661+0800 INFO  (jsonrpc/0) [vdsm.api] FINISH repoStats return={u'f3c9971c-6b2d-435e-907c-b0d337e30cd2': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.00133207', 'lastCheck': '6.3', 'valid': True}, u'52c9a48f-00e6-404c-920e-dd01e7738cda': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000834762', 'lastCheck': '6.8', 'valid': True}} from=::ffff:192.168.128.88,38760, task_id=954d8da4-e71d-4cac-819e-8494dbc1b938 (api:52)
2019-01-03 09:10:07,662+0800 INFO  (jsonrpc/0) [vdsm.api] START multipath_health() from=::ffff:192.168.128.88,38760, task_id=898f6f0c-c6b1-4212-bb94-f9c75a3ca8cd (api:46)
2019-01-03 09:10:07,662+0800 INFO  (jsonrpc/0) [vdsm.api] FINISH multipath_health return={} from=::ffff:192.168.128.88,38760, task_id=898f6f0c-c6b1-4212-bb94-f9c75a3ca8cd (api:52)
2019-01-03 09:10:07,672+0800 INFO  (jsonrpc/0) [api.host] FINISH getStats return={'status': {'message': 'Done', 'code': 0}, 'info': {'cpuStatistics': {'11': {'cpuUser': '1.46', 'nodeIndex': 0, 'cpuSys': '0.47', 'cpuIdle': '98.07'}, '10': {'cpuUser': '1.00', 'nodeIndex': 0, 'cpuSys': '0.47', 'cpuIdle': '98.53'}, '1': {'cpuUser': '2.19', 'nodeIndex': 0, 'cpuSys': '1.66', 'cpuIdle': '96.15'}, '0': {'cpuUser': '0.33', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '99.34'}, '3': {'cpuUser': '0.93', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '98.74'}, '2': {'cpuUser': '1.20', 'nodeIndex': 0, 'cpuSys': '0.40', 'cpuIdle': '98.40'}, '5': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '99.27'}, '4': {'cpuUser': '0.93', 'nodeIndex': 0, 'cpuSys': '0.60', 'cpuIdle': '98.47'}, '7': {'cpuUser': '1.13', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '98.54'}, '6': {'cpuUser': '0.93', 'nodeIndex': 0, 'cpuSys': '0.53', 'cpuIdle': '98.54'}, '9': {'cpuUser': '0.80', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.00'}, '8': {'cpuUser': '1.79', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '97.88'}}, 'numaNodeMemFree': {'0': {'memPercent': 32, 'memFree': '21876'}}, 'memShared': 0, 'haScore': 3400, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'vmCount': 0, 'memUsed': '18', 'storageDomains': {u'f3c9971c-6b2d-435e-907c-b0d337e30cd2': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.00133207', 'lastCheck': '6.3', 'valid': True}, u'52c9a48f-00e6-404c-920e-dd01e7738cda': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000834762', 'lastCheck': '6.8', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'eno2': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477804.078918, 'name': 'eno2', 'tx': '3572650999', 'txDropped': '0', 'rx': '5148132835', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'eno1': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477804.078918, 'name': 'eno1', 'tx': '691406028', 'txDropped': '0', 'rx': '1668370433', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'ovirtmgmt': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477804.078918, 'name': 'ovirtmgmt', 'tx': '676933032', 'txDropped': '0', 'rx': '1357843797', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'lo': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477804.078918, 'name': 'lo', 'tx': '669904485321', 'txDropped': '0', 'rx': '669904485321', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'ens6f0': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477804.078918, 'name': 'ens6f0', 'tx': '688066944071', 'txDropped': '0', 'rx': '1836942524354', 'rxErrors': '30', 'speed': '10000', 'rxDropped': '0'}, 'ens6f1': {'txErrors': '0', 'state': 'down', 'sampleTime': 1546477804.078918, 'name': 'ens6f1', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'vm': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477804.078918, 'name': 'vm', 'tx': '3428346367', 'txDropped': '0', 'rx': '4713392702', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, ';vdsmdummy;': {'txErrors': '0', 'state': 'down', 'sampleTime': 1546477804.078918, 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'br-int': {'txErrors': '0', 'state': 'down', 'sampleTime': 1546477804.078918, 'name': 'br-int', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'genev_sys_6081': {'txErrors': '8', 'state': 'up', 'sampleTime': 1546477804.078918, 'name': 'genev_sys_6081', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'ovs-system': {'txErrors': '0', 'state': 'down', 'sampleTime': 1546477804.078918, 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}}, 'txDropped': '0', 'anonHugePages': '1650', 'ksmPages': 100, 'elapsedTime': '1103600.57', 'cpuLoad': '0.17', 'cpuSys': '0.49', 'diskStats': {'/var/log': {'free': '57256'}, '/var/run/vdsm/': {'free': '15845'}, '/tmp': {'free': '57256'}}, 'cpuUserVdsmd': '1.46', 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 26062, 'bootTime': '1545374170', 'haStats': {'active': True, 'configured': True, 'score': 3400, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'multipathHealth': {}, 'rxDropped': '0', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'hugepages': defaultdict(<type 'dict'>, {1048576: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}, 2048: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}), 'dateTime': '2019-01-03T01:10:07 GMT', 'cpuUser': '1.09', 'memFree': 26318, 'cpuIdle': '98.42', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd': '0.60'}} from=::ffff:192.168.128.88,38760 (api:52)
2019-01-03 09:10:07,673+0800 INFO  (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.02 seconds (__init__:573)
2019-01-03 09:10:10,127+0800 INFO  (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:573)
2019-01-03 09:10:10,130+0800 INFO  (jsonrpc/3) [api.virt] START getStats() from=::1,56722, vmId=c4d8cab1-54f3-4650-b0b4-bd07e3b8e310 (api:46)
2019-01-03 09:10:10,130+0800 INFO  (jsonrpc/3) [api] FINISH getStats error=Virtual machine does not exist: {'vmId': u'c4d8cab1-54f3-4650-b0b4-bd07e3b8e310'} (api:127)
2019-01-03 09:10:10,130+0800 INFO  (jsonrpc/3) [api.virt] FINISH getStats return={'status': {'message': "Virtual machine does not exist: {'vmId': u'c4d8cab1-54f3-4650-b0b4-bd07e3b8e310'}", 'code': 1}} from=::1,56722, vmId=c4d8cab1-54f3-4650-b0b4-bd07e3b8e310 (api:52)
2019-01-03 09:10:10,130+0800 INFO  (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call VM.getStats failed (error 1) in 0.00 seconds (__init__:573)
2019-01-03 09:10:12,022+0800 INFO  (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:573)
2019-01-03 09:10:12,025+0800 INFO  (jsonrpc/5) [vdsm.api] START repoStats(domains=[u'52c9a48f-00e6-404c-920e-dd01e7738cda']) from=::1,56722, task_id=057e9ebb-8e99-45e3-a132-a778bd721444 (api:46)
2019-01-03 09:10:12,025+0800 INFO  (jsonrpc/5) [vdsm.api] FINISH repoStats return={u'52c9a48f-00e6-404c-920e-dd01e7738cda': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.0010759', 'lastCheck': '1.1', 'valid': True}} from=::1,56722, task_id=057e9ebb-8e99-45e3-a132-a778bd721444 (api:52)
2019-01-03 09:10:12,025+0800 INFO  (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:573)
2019-01-03 09:10:12,090+0800 INFO  (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:573)
2019-01-03 09:10:12,094+0800 INFO  (jsonrpc/6) [api.host] START getStats() from=::1,56722 (api:46)
2019-01-03 09:10:12,094+0800 INFO  (jsonrpc/6) [vdsm.api] START repoStats(domains=()) from=::1,56722, task_id=89c3df5e-7d12-4b24-aa32-12d51ab5941d (api:46)
2019-01-03 09:10:12,094+0800 INFO  (jsonrpc/6) [vdsm.api] FINISH repoStats return={u'f3c9971c-6b2d-435e-907c-b0d337e30cd2': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.00119531', 'lastCheck': '0.8', 'valid': True}, u'52c9a48f-00e6-404c-920e-dd01e7738cda': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.0010759', 'lastCheck': '1.2', 'valid': True}} from=::1,56722, task_id=89c3df5e-7d12-4b24-aa32-12d51ab5941d (api:52)
2019-01-03 09:10:12,095+0800 INFO  (jsonrpc/6) [vdsm.api] START multipath_health() from=::1,56722, task_id=9bd48f6d-8687-4a7c-855e-a1d83207c112 (api:46)
2019-01-03 09:10:12,095+0800 INFO  (jsonrpc/6) [vdsm.api] FINISH multipath_health return={} from=::1,56722, task_id=9bd48f6d-8687-4a7c-855e-a1d83207c112 (api:52)
2019-01-03 09:10:12,105+0800 INFO  (jsonrpc/6) [api.host] FINISH getStats return={'status': {'message': 'Done', 'code': 0}, 'info': {'cpuStatistics': {'11': {'cpuUser': '1.46', 'nodeIndex': 0, 'cpuSys': '0.47', 'cpuIdle': '98.07'}, '10': {'cpuUser': '1.00', 'nodeIndex': 0, 'cpuSys': '0.47', 'cpuIdle': '98.53'}, '1': {'cpuUser': '2.19', 'nodeIndex': 0, 'cpuSys': '1.66', 'cpuIdle': '96.15'}, '0': {'cpuUser': '0.33', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '99.34'}, '3': {'cpuUser': '0.93', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '98.74'}, '2': {'cpuUser': '1.20', 'nodeIndex': 0, 'cpuSys': '0.40', 'cpuIdle': '98.40'}, '5': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '99.27'}, '4': {'cpuUser': '0.93', 'nodeIndex': 0, 'cpuSys': '0.60', 'cpuIdle': '98.47'}, '7': {'cpuUser': '1.13', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '98.54'}, '6': {'cpuUser': '0.93', 'nodeIndex': 0, 'cpuSys': '0.53', 'cpuIdle': '98.54'}, '9': {'cpuUser': '0.80', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.00'}, '8': {'cpuUser': '1.79', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '97.88'}}, 'numaNodeMemFree': {'0': {'memPercent': 32, 'memFree': '21876'}}, 'memShared': 0, 'haScore': 3400, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'vmCount': 0, 'memUsed': '18', 'storageDomains': {u'f3c9971c-6b2d-435e-907c-b0d337e30cd2': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.00119531', 'lastCheck': '0.8', 'valid': True}, u'52c9a48f-00e6-404c-920e-dd01e7738cda': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.0010759', 'lastCheck': '1.2', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'eno2': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477804.078918, 'name': 'eno2', 'tx': '3572650999', 'txDropped': '0', 'rx': '5148132835', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'eno1': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477804.078918, 'name': 'eno1', 'tx': '691406028', 'txDropped': '0', 'rx': '1668370433', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'ovirtmgmt': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477804.078918, 'name': 'ovirtmgmt', 'tx': '676933032', 'txDropped': '0', 'rx': '1357843797', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'lo': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477804.078918, 'name': 'lo', 'tx': '669904485321', 'txDropped': '0', 'rx': '669904485321', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'ens6f0': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477804.078918, 'name': 'ens6f0', 'tx': '688066944071', 'txDropped': '0', 'rx': '1836942524354', 'rxErrors': '30', 'speed': '10000', 'rxDropped': '0'}, 'ens6f1': {'txErrors': '0', 'state': 'down', 'sampleTime': 1546477804.078918, 'name': 'ens6f1', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'vm': {'txErrors': '0', 'state': 'up', 'sampleTime': 1546477804.078918, 'name': 'vm', 'tx': '3428346367', 'txDropped': '0', 'rx': '4713392702', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, ';vdsmdummy;': {'txErrors': '0', 'state': 'down', 'sampleTime': 1546477804.078918, 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'br-int': {'txErrors': '0', 'state': 'down', 'sampleTime': 1546477804.078918, 'name': 'br-int', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'genev_sys_6081': {'txErrors': '8', 'state': 'up', 'sampleTime': 1546477804.078918, 'name': 'genev_sys_6081', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'ovs-system': {'txErrors': '0', 'state': 'down', 'sampleTime': 1546477804.078918, 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}}, 'txDropped': '0', 'anonHugePages': '1650', 'ksmPages': 100, 'elapsedTime': '1103605.01', 'cpuLoad': '0.17', 'cpuSys': '0.49', 'diskStats': {'/var/log': {'free': '57256'}, '/var/run/vdsm/': {'free': '15845'}, '/tmp': {'free': '57256'}}, 'cpuUserVdsmd': '1.46', 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 26065, 'bootTime': '1545374170', 'haStats': {'active': True, 'configured': True, 'score': 3400, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'multipathHealth': {}, 'rxDropped': '0', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'hugepages': defaultdict(<type 'dict'>, {1048576: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}, 2048: {'resv_hugepages': 0, 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}), 'dateTime': '2019-01-03T01:10:12 GMT', 'cpuUser': '1.09', 'memFree': 26321, 'cpuIdle': '98.42', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd': '0.60'}} from=::1,56722 (api:52)
2019-01-03 09:10:12,106+0800 INFO  (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds (__init__:573)
2019-01-03 09:10:12,583+0800 INFO  (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:573)
2019-01-03 09:10:12,586+0800 INFO  (jsonrpc/0) [api.host] START getCapabilities() from=::1,56722 (api:46)
2019-01-03 09:10:14,540+0800 INFO  (jsonrpc/4) [vdsm.api] START getSpmStatus(spUUID=u'82dc8592-f44a-11e8-a0ce-00163e377976', options=None) from=::ffff:192.168.128.88,38760, task_id=c2562ac4-4275-4968-9cd4-82d2ed066340 (api:46)
2019-01-03 09:10:14,544+0800 INFO  (jsonrpc/4) [vdsm.api] FINISH getSpmStatus return={'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 13L}} from=::ffff:192.168.128.88,38760, task_id=c2562ac4-4275-4968-9cd4-82d2ed066340 (api:52)
2019-01-03 09:10:14,544+0800 INFO  (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call StoragePool.getSpmStatus succeeded in 0.01 seconds (__init__:573)
2019-01-03 09:10:14,599+0800 INFO  (jsonrpc/3) [vdsm.api] START getStoragePoolInfo(spUUID=u'82dc8592-f44a-11e8-a0ce-00163e377976', options=None) from=::ffff:192.168.128.88,38768, task_id=e6f6ee42-ab09-415d-a501-84acd9ad591a (api:46)
2019-01-03 09:10:14,604+0800 INFO  (jsonrpc/3) [vdsm.api] FINISH getStoragePoolInfo return={'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 13L, 'domains': u'f3c9971c-6b2d-435e-907c-b0d337e30cd2:Active,52c9a48f-00e6-404c-920e-dd01e7738cda:Active', 'master_uuid': '52c9a48f-00e6-404c-920e-dd01e7738cda', 'version': '4', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'f3c9971c-6b2d-435e-907c-b0d337e30cd2': {'status': u'Active', 'diskfree': '2126143029248', 'isoprefix': '', 'alerts': [], 'disktotal': '2147080994816', 'version': 4}, u'52c9a48f-00e6-404c-920e-dd01e7738cda': {'status': u'Active', 'diskfree': '25232932864', 'isoprefix': '', 'alerts': [], 'disktotal': '106971529216', 'version': 4}}} from=::ffff:192.168.128.88,38768, task_id=e6f6ee42-ab09-415d-a501-84acd9ad591a (api:52)
2019-01-03 09:10:14,604+0800 INFO  (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call StoragePool.getInfo succeeded in 0.01 seconds (__init__:573)
2019-01-03 09:10:15,063+0800 INFO  (jsonrpc/0) [root] /usr/libexec/vdsm/hooks/after_get_caps/50_openstacknet: rc=0 err= (hooks:110)
2019-01-03 09:10:15,279+0800 INFO  (jsonrpc/0) [root] /usr/libexec/vdsm/hooks/after_get_caps/openstacknet_utils.py: rc=0 err= (hooks:110)
2019-01-03 09:10:15,524+0800 INFO  (jsonrpc/0) [root] /usr/libexec/vdsm/hooks/after_get_caps/ovirt_provider_ovn_hook: rc=0 err= (hooks:110)
2019-01-03 09:10:15,525+0800 INFO  (jsonrpc/0) [api.host] FINISH getCapabilities return={'status': {'message': 'Done', 'code': 0}, 'info': {u'HBAInventory': {u'iSCSI': [{u'InitiatorName': u'iqn.1994-05.com.redhat:d9402c44e2a3'}], u'FC': []}, u'packages2': {u'kernel': {u'release': u'862.14.4.el7.x86_64', u'version': u'3.10.0'}, u'spice-server': {u'release': u'2.el7_5.5', u'version': u'0.14.0'}, u'librbd1': {u'release': u'0.el7', u'version': u'12.2.10'}, u'vdsm': {u'release': u'1.el7', u'version': u'4.20.39.1'}, u'qemu-kvm': {u'release': u'21.el7_5.4.1', u'version': u'2.10.0'}, u'openvswitch': {u'release': u'4.el7', u'version': u'2.9.0'}, u'libvirt': {u'release': u'14.el7_5.8', u'version': u'3.9.0'}, u'ovirt-hosted-engine-ha': {u'release': u'1.el7', u'version': u'2.2.16'}, u'qemu-img': {u'release': u'21.el7_5.4.1', u'version': u'2.10.0'}, u'mom': {u'release': u'1.el7.centos', u'version': u'0.5.12'}, u'glusterfs-cli': {u'release': u'1.el7', u'version': u'3.12.14'}}, u'numaNodeDistance': {u'0': [10]}, u'cpuModel': u'Intel(R) Xeon(R) CPU E5-2430 v2 @ 2.50GHz', u'nestedVirtualization': False, u'liveMerge': u'true', u'hooks': {u'after_vm_start': {u'openstacknet_utils.py': {u'md5': u'640ef8b473cbc16b102c5ebf57ea533f'}, u'50_openstacknet': {u'md5': u'ea0a5a715da8c1badbcda28e8b8fa00e'}}, u'after_network_setup': {u'30_ethtool_options': {u'md5': u'f04c2ca5dce40663e2ed69806eea917c'}}, u'before_vm_start': {u'50_hostedengine': {u'md5': u'95c810cdcfe4195302a59574a5148289'}, u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'after_device_migrate_destination': {u'openstacknet_utils.py': {u'md5': u'640ef8b473cbc16b102c5ebf57ea533f'}, u'50_openstacknet': {u'md5': u'f93c12af7454bbba4c1ef445c2bc9860'}}, u'before_device_migrate_destination': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}}, u'before_device_create': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}, u'openstacknet_utils.py': {u'md5': u'640ef8b473cbc16b102c5ebf57ea533f'}, u'50_openstacknet': {u'md5': u'0438de5ff9b6bf8d3160804ff71bf827'}, u'20_ovirt_provider_ovn_vhostuser_hook': {u'md5': u'6092f45748b23644101c04ba025ddec5'}, u'10_ovirt_provider_ovn_hook': {u'md5': u'ada7e4e757fd0241b682b5cc6a545fc8'}}, u'after_device_create': {u'openstacknet_utils.py': {u'md5': u'640ef8b473cbc16b102c5ebf57ea533f'}, u'50_openstacknet': {u'md5': u'f93c12af7454bbba4c1ef445c2bc9860'}}, u'after_nic_hotplug': {u'openstacknet_utils.py': {u'md5': u'640ef8b473cbc16b102c5ebf57ea533f'}, u'50_openstacknet': {u'md5': u'f93c12af7454bbba4c1ef445c2bc9860'}}, u'before_vm_migrate_destination': {u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'before_vm_dehibernate': {u'50_vhostmd': {u'md5': u'9206bc390bcbf208b06a8e899581be2d'}}, u'before_nic_hotplug': {u'50_vmfex': {u'md5': u'49caba1a5faadd8efacef966f79bc30a'}, u'openstacknet_utils.py': {u'md5': u'640ef8b473cbc16b102c5ebf57ea533f'}, u'50_openstacknet': {u'md5': u'0438de5ff9b6bf8d3160804ff71bf827'}, u'20_ovirt_provider_ovn_vhostuser_hook': {u'md5': u'6092f45748b23644101c04ba025ddec5'}, u'10_ovirt_provider_ovn_hook': {u'md5': u'ada7e4e757fd0241b682b5cc6a545fc8'}}, u'after_get_caps': {u'openstacknet_utils.py': {u'md5': u'640ef8b473cbc16b102c5ebf57ea533f'}, u'50_openstacknet': {u'md5': u'0e00c63ab44a952e722209ea31fd7a71'}, u'ovirt_provider_ovn_hook': {u'md5': u'4c4b1d2d5460e6a65114ae36cb775df6'}}, u'before_network_setup': {u'50_fcoe': {u'md5': u'28c352339c8beef1e1b05c67d106d062'}}, u'after_vm_destroy': {u'delete_vhostuserclient_hook': {u'md5': u'cb68e366da312b942a9a8518f3174df9'}, u'50_vhostmd': {u'md5': u'd70f7ee0453f632e87c09a157ff8ff66'}}}, u'supportsIPv6': True, u'realtimeKernel': False, u'vmTypes': [u'kvm'], u'liveSnapshot': u'true', u'cpuThreads': u'12', u'kdumpStatus': 0, u'networks': {u'ovirtmgmt': {u'iface': u'ovirtmgmt', u'ipv6autoconf': True, u'addr': u'192.168.128.78', u'dhcpv6': False, u'ipv6addrs': [], u'switch': u'legacy', u'bridged': True, u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'255.255.255.0', u'ipv4defaultroute': False, u'stp': u'off', u'ipv4addrs': [u'192.168.128.78/24'], u'ipv6gateway': u'::', u'gateway': u'', u'ports': [u'eno1']}, u'vm': {u'iface': u'vm', u'ipv6autoconf': True, u'addr': u'192.168.1.65', u'dhcpv6': False, u'ipv6addrs': [], u'switch': u'legacy', u'bridged': True, u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'255.255.255.0', u'ipv4defaultroute': True, u'stp': u'off', u'ipv4addrs': [u'192.168.1.65/24'], u'ipv6gateway': u'::', u'gateway': u'192.168.1.1', u'ports': [u'eno2']}}, u'kernelArgs': u'BOOT_IMAGE=/vmlinuz-3.10.0-862.14.4.el7.x86_64 root=UUID=2b3023f0-afd1-4e95-9267-5a866b7a0071 ro crashkernel=auto rhgb quiet LANG=en_US.UTF-8', u'bridges': {u'ovirtmgmt': {u'ipv6autoconf': True, u'addr': u'192.168.128.78', u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'255.255.255.0', u'ipv4defaultroute': False, u'stp': u'off', u'ipv4addrs': [u'192.168.128.78/24'], u'ipv6gateway': u'::', u'gateway': u'', u'opts': {u'multicast_last_member_count': u'2', u'vlan_protocol': u'0x8100', u'hash_elasticity': u'4', u'multicast_query_response_interval': u'1000', u'group_fwd_mask': u'0x0', u'multicast_snooping': u'1', u'multicast_startup_query_interval': u'3125', u'hello_timer': u'0', u'multicast_querier_interval': u'25500', u'max_age': u'2000', u'hash_max': u'512', u'stp_state': u'0', u'topology_change_detected': u'0', u'priority': u'32768', u'multicast_igmp_version': u'2', u'multicast_membership_interval': u'26000', u'root_path_cost': u'0', u'root_port': u'0', u'multicast_stats_enabled': u'0', u'multicast_startup_query_count': u'2', u'nf_call_iptables': u'0', u'vlan_stats_enabled': u'0', u'hello_time': u'200', u'topology_change': u'0', u'bridge_id': u'8000.001e67da42f4', u'topology_change_timer': u'0', u'ageing_time': u'30000', u'nf_call_ip6tables': u'0', u'multicast_mld_version': u'1', u'gc_timer': u'2259', u'root_id': u'8000.001e67da42f4', u'nf_call_arptables': u'0', u'group_addr': u'1:80:c2:0:0:0', u'multicast_last_member_interval': u'100', u'default_pvid': u'1', u'multicast_query_interval': u'12500', u'multicast_query_use_ifaddr': u'0', u'tcn_timer': u'0', u'multicast_router': u'1', u'vlan_filtering': u'0', u'multicast_querier': u'0', u'forward_delay': u'0'}, u'ports': [u'eno1']}, u'vm': {u'ipv6autoconf': True, u'addr': u'192.168.1.65', u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'255.255.255.0', u'ipv4defaultroute': True, u'stp': u'off', u'ipv4addrs': [u'192.168.1.65/24'], u'ipv6gateway': u'::', u'gateway': u'192.168.1.1', u'opts': {u'multicast_last_member_count': u'2', u'vlan_protocol': u'0x8100', u'hash_elasticity': u'4', u'multicast_query_response_interval': u'1000', u'group_fwd_mask': u'0x0', u'multicast_snooping': u'1', u'multicast_startup_query_interval': u'3125', u'hello_timer': u'0', u'multicast_querier_interval': u'25500', u'max_age': u'2000', u'hash_max': u'512', u'stp_state': u'0', u'topology_change_detected': u'0', u'priority': u'32768', u'multicast_igmp_version': u'2', u'multicast_membership_interval': u'26000', u'root_path_cost': u'0', u'root_port': u'0', u'multicast_stats_enabled': u'0', u'multicast_startup_query_count': u'2', u'nf_call_iptables': u'0', u'vlan_stats_enabled': u'0', u'hello_time': u'200', u'topology_change': u'0', u'bridge_id': u'8000.001e67da42f5', u'topology_change_timer': u'0', u'ageing_time': u'30000', u'nf_call_ip6tables': u'0', u'multicast_mld_version': u'1', u'gc_timer': u'2259', u'root_id': u'8000.001e67da42f5', u'nf_call_arptables': u'0', u'group_addr': u'1:80:c2:0:0:0', u'multicast_last_member_interval': u'100', u'default_pvid': u'1', u'multicast_query_interval': u'12500', u'multicast_query_use_ifaddr': u'0', u'tcn_timer': u'0', u'multicast_router': u'1', u'vlan_filtering': u'0', u'multicast_querier': u'0', u'forward_delay': u'0'}, u'ports': [u'eno2']}}, u'uuid': u'760CF86F-6CA3-E411-BBFA-001E67DA42F4', u'onlineCpus': u'0,1,2,3,4,5,6,7,8,9,10,11', u'nameservers': [], u'nics': {u'eno1': {u'ipv6autoconf': False, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'00:1e:67:da:42:f4', u'ipv6gateway': u'::', u'gateway': u''}, u'ens6f0': {u'ipv6autoconf': True, u'addr': u'192.168.10.65', u'speed': 10000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'9000', u'dhcpv4': False, u'netmask': u'255.255.255.0', u'ipv4defaultroute': False, u'ipv4addrs': [u'192.168.10.65/24'], u'hwaddr': u'90:e2:ba:92:bd:08', u'ipv6gateway': u'fe80::1:1', u'gateway': u''}, u'ens6f1': {u'ipv6autoconf': True, u'addr': u'', u'speed': 0, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'90:e2:ba:92:bd:0a', u'ipv6gateway': u'::', u'gateway': u''}, u'eno2': {u'ipv6autoconf': False, u'addr': u'', u'speed': 1000, u'dhcpv6': False, u'ipv6addrs': [], u'mtu': u'1500', u'dhcpv4': False, u'netmask': u'', u'ipv4defaultroute': False, u'ipv4addrs': [], u'hwaddr': u'00:1e:67:da:42:f5', u'ipv6gateway': u'::', u'gateway': u''}}, u'software_revision': u'1', u'hostdevPassthrough': u'false', u'clusterLevels': [u'3.6', u'4.0', u'4.1', u'4.2'], u'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfmperf,eagerfpu,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,dca,sse4_1,sse4_2,x2apic,popcnt,tsc_deadline_timer,aes,xsave,avx,f16c,rdrand,lahf_lm,epb,intel_ppin,tpr_shadow,vnmi,flexpriority,ept,vpid,fsgsbase,smep,erms,xsaveopt,dtherm,ida,arat,pln,pts,model_n270,model_coreduo,model_Nehalem,model_core2duo,model_IvyBridge,model_Penryn,model_Westmere,model_SandyBridge,model_Conroe', u'kernelFeatures': {u'RETP': 1, u'IBRS': 0, u'PTI': 1}, u'ISCSIInitiatorName': u'iqn.1994-05.com.redhat:d9402c44e2a3', u'netConfigDirty': u'False', u'selinux': {u'mode': u'-1'}, u'autoNumaBalancing': 0, u'reservedMem': u'321', u'containers': False, u'bondings': {}, u'software_version': u'4.20', u'supportedENGINEs': [u'3.6', u'4.0', u'4.1', u'4.2'], u'cpuSpeed': u'2799.987', u'numaNodes': {u'0': {u'totalMemory': u'31902', u'cpus': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]}}, u'cpuSockets': u'1', u'vlans': {}, u'version_name': u'Snow Man', 'lastClientIface': 'lo', u'cpuCores': u'6', u'hostedEngineDeployed': True, u'hugepages': [1048576, 2048], u'guestOverhead': u'65', u'additionalFeatures': [u'libgfapi_supported'], u'openstack_binding_host_ids': {u'OVIRT_PROVIDER_OVN': u'9549cea1-37dd-4e4b-b34a-62d70e3479bb'}, u'kvmEnabled': u'true', u'memSize': u'31902', u'emulatedMachines': [u'pc-i440fx-rhel7.1.0', u'pc-q35-rhel7.3.0', u'rhel6.3.0', u'pc-i440fx-rhel7.5.0', u'pc-i440fx-rhel7.0.0', u'rhel6.1.0', u'pc-i440fx-rhel7.4.0', u'rhel6.6.0', u'pc-q35-rhel7.5.0', u'rhel6.2.0', u'pc', u'pc-i440fx-rhel7.3.0', u'q35', u'pc-i440fx-rhel7.2.0', u'rhel6.4.0', u'pc-q35-rhel7.4.0', u'rhel6.0.0', u'rhel6.5.0'], u'rngSources': [u'hwrng', u'random'], u'operatingSystem': {u'release': u'5.1804.el7.centos', u'pretty_name': u'CentOS Linux 7 (Core)', u'version': u'7', u'name': u'RHEL'}}} from=::1,56722 (api:52)
2019-01-03 09:10:15,537+0800 INFO  (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Host.getCapabilities succeeded in 2.95 seconds (__init__:573)
2019-01-03 09:10:16,065+0800 INFO  (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:573)
2019-01-03 09:10:16,201+0800 INFO  (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:573)
2019-01-03 09:10:16,205+0800 INFO  (jsonrpc/7) [vdsm.api] START repoStats(domains=[u'52c9a48f-00e6-404c-920e-dd01e7738cda']) from=::1,56722, task_id=6e6920ee-20ad-43b3-9699-5c566f4fa76d (api:46)
2019-01-03 09:10:16,205+0800 INFO  (jsonrpc/7) [vdsm.api] FINISH repoStats return={u'52c9a48f-00e6-404c-920e-dd01e7738cda': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.0010759', 'lastCheck': '5.3', 'valid': True}} from=::1,56722, task_id=6e6920ee-20ad-43b3-9699-5c566f4fa76d (api:52)
2019-01-03 09:10:16,205+0800 INFO  (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:573)
2019-01-03 09:10:16,210+0800 INFO  (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:573)
2019-01-03 09:10:17,389+0800 INFO  (jsonrpc/1) [api.host] START getAllVmStats() from=::ffff:192.168.128.88,38760 (api:46)
2019-01-03 09:10:17,389+0800 INFO  (jsonrpc/1) [api.host] FINISH getAllVmStats return={'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} from=::ffff:192.168.128.88,38760 (api:52)
2019-01-03 09:10:17,389+0800 INFO  (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:573)
2019-01-03 09:10:18,191+0800 INFO  (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:573)



ÔÚ 2019-01-02 14:27:31£¬"Michal Skrivanek" <michal.skrivanek@redhat.com> дµÀ£º


On 2 Jan 2019, at 03:02, ¶­ÇàÁú <ddqlo@126.com> wrote:

Hi all,
        I have an ovirt4.2 environment of 3 hosts. Now all vms in this environment could not be migrated. But all the vms could be started on all 3 hosts. Anyone can help? Thanks a lot!

can you add source and destination vdsm.log please?


engine logs:

2019-01-02 09:41:26,868+08 INFO  [org.ovirt.engine.core.bll.MigrateVmCommand] (default task-9) [3eed5f0e-aaf5-4dce-bf30-2c49e09ab30d] Lock Acquired to object 'EngineLock:{exclusiveLocks='[eff7f697-8a07-46e5-a631-a1011a0eb836=VM]', sharedLocks=''}'
2019-01-02 09:41:26,978+08 INFO  [org.ovirt.engine.core.bll.MigrateVmCommand] (EE-ManagedThreadFactory-engine-Thread-168938) [3eed5f0e-aaf5-4dce-bf30-2c49e09ab30d] Running command: MigrateVmCommand internal: false. Entities affected :  ID: eff7f697-8a07-46e5-a631-a1011a0eb836 Type: VMAction group MIGRATE_VM with role type USER
2019-01-02 09:41:27,019+08 INFO  [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (EE-ManagedThreadFactory-engine-Thread-168938) [3eed5f0e-aaf5-4dce-bf30-2c49e09ab30d] START, MigrateVDSCommand( MigrateVDSCommandParameters:{hostId='0aff0075-4b41-4f37-98de-7433a17cd47e', vmId='eff7f697-8a07-46e5-a631-a1011a0eb836', srcHost='horeb66', dstVdsId='5bb18f6e-9c7e-4afd-92de-f6482bf752e5', dstHost='horeb65:54321', migrationMethod='ONLINE', tunnelMigration='false', migrationDowntime='0', autoConverge='true', migrateCompressed='false', consoleAddress='null', maxBandwidth='500', enableGuestEvents='true', maxIncomingMigrations='2', maxOutgoingMigrations='2', convergenceSchedule='[init=[{name=setDowntime, params=[100]}], stalling=[{limit=1, action={name=setDowntime, params=[150]}}, {limit=2, action={name=setDowntime, params=[200]}}, {limit=3, action={name=setDowntime, params=[300]}}, {limit=4, action={name=setDowntime, params=[400]}}, {limit=6, action={name=setDowntime, params=[500]}}, {limit=-1, action={name=abort, params=[]}}]]', dstQemu='192.168.128.78'}), log id: 1bd72db2
2019-01-02 09:41:27,019+08 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (EE-ManagedThreadFactory-engine-Thread-168938) [3eed5f0e-aaf5-4dce-bf30-2c49e09ab30d] START, MigrateBrokerVDSCommand(HostName = horeb66, MigrateVDSCommandParameters:{hostId='0aff0075-4b41-4f37-98de-7433a17cd47e', vmId='eff7f697-8a07-46e5-a631-a1011a0eb836', srcHost='horeb66', dstVdsId='5bb18f6e-9c7e-4afd-92de-f6482bf752e5', dstHost='horeb65:54321', migrationMethod='ONLINE', tunnelMigration='false', migrationDowntime='0', autoConverge='true', migrateCompressed='false', consoleAddress='null', maxBandwidth='500', enableGuestEvents='true', maxIncomingMigrations='2', maxOutgoingMigrations='2', convergenceSchedule='[init=[{name=setDowntime, params=[100]}], stalling=[{limit=1, action={name=setDowntime, params=[150]}}, {limit=2, action={name=setDowntime, params=[200]}}, {limit=3, action={name=setDowntime, params=[300]}}, {limit=4, action={name=setDowntime, params=[400]}}, {limit=6, action={name=setDowntime, params=[500]}}, {limit=-1, action={name=abort, params=[]}}]]', dstQemu='192.168.128.78'}), log id: 380b8d38
2019-01-02 09:41:27,025+08 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (EE-ManagedThreadFactory-engine-Thread-168938) [3eed5f0e-aaf5-4dce-bf30-2c49e09ab30d] FINISH, MigrateBrokerVDSCommand, log id: 380b8d38
2019-01-02 09:41:27,029+08 INFO  [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (EE-ManagedThreadFactory-engine-Thread-168938) [3eed5f0e-aaf5-4dce-bf30-2c49e09ab30d] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 1bd72db2
2019-01-02 09:41:27,036+08 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-168938) [3eed5f0e-aaf5-4dce-bf30-2c49e09ab30d] EVENT_ID: VM_MIGRATION_START(62), Migration started (VM: win7, Source: horeb66, Destination: horeb65, User: admin@internal-authz). 
2019-01-02 09:41:41,557+08 INFO  [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] VM 'eff7f697-8a07-46e5-a631-a1011a0eb836'(win7) moved from 'MigratingFrom' --> 'Up'
2019-01-02 09:41:41,557+08 INFO  [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] Adding VM 'eff7f697-8a07-46e5-a631-a1011a0eb836'(win7) to re-run list
2019-01-02 09:41:41,567+08 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.VmsMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] Rerun VM 'eff7f697-8a07-46e5-a631-a1011a0eb836'. Called from VDS 'horeb66'
2019-01-02 09:41:41,570+08 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-168945) [] START, MigrateStatusVDSCommand(HostName = horeb66, MigrateStatusVDSCommandParameters:{hostId='0aff0075-4b41-4f37-98de-7433a17cd47e', vmId='eff7f697-8a07-46e5-a631-a1011a0eb836'}), log id: 4ed2923c
2019-01-02 09:41:41,573+08 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-168945) [] FINISH, MigrateStatusVDSCommand, log id: 4ed2923c
2019-01-02 09:41:41,583+08 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-168945) [] EVENT_ID: VM_MIGRATION_TRYING_RERUN(128), Failed to migrate VM win7 to Host horeb65 . Trying to migrate to another Host.
2019-01-02 09:41:41,642+08 INFO  [org.ovirt.engine.core.bll.MigrateVmCommand] (EE-ManagedThreadFactory-engine-Thread-168945) [] Running command: MigrateVmCommand internal: false. Entities affected :  ID: eff7f697-8a07-46e5-a631-a1011a0eb836 Type: VMAction group MIGRATE_VM with role type USER
2019-01-02 09:41:41,671+08 INFO  [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (EE-ManagedThreadFactory-engine-Thread-168945) [] START, MigrateVDSCommand( MigrateVDSCommandParameters:{hostId='0aff0075-4b41-4f37-98de-7433a17cd47e', vmId='eff7f697-8a07-46e5-a631-a1011a0eb836', srcHost='horeb66', dstVdsId='20786f47-87fe-4ef6-be82-b580e5d0a350', dstHost='horeb67:54321', migrationMethod='ONLINE', tunnelMigration='false', migrationDowntime='0', autoConverge='true', migrateCompressed='false', consoleAddress='null', maxBandwidth='500', enableGuestEvents='true', maxIncomingMigrations='2', maxOutgoingMigrations='2', convergenceSchedule='[init=[{name=setDowntime, params=[100]}], stalling=[{limit=1, action={name=setDowntime, params=[150]}}, {limit=2, action={name=setDowntime, params=[200]}}, {limit=3, action={name=setDowntime, params=[300]}}, {limit=4, action={name=setDowntime, params=[400]}}, {limit=6, action={name=setDowntime, params=[500]}}, {limit=-1, action={name=abort, params=[]}}]]', dstQemu='192.168.128.76'}), log id: 4611863
2019-01-02 09:41:41,672+08 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (EE-ManagedThreadFactory-engine-Thread-168945) [] START, MigrateBrokerVDSCommand(HostName = horeb66, MigrateVDSCommandParameters:{hostId='0aff0075-4b41-4f37-98de-7433a17cd47e', vmId='eff7f697-8a07-46e5-a631-a1011a0eb836', srcHost='horeb66', dstVdsId='20786f47-87fe-4ef6-be82-b580e5d0a350', dstHost='horeb67:54321', migrationMethod='ONLINE', tunnelMigration='false', migrationDowntime='0', autoConverge='true', migrateCompressed='false', consoleAddress='null', maxBandwidth='500', enableGuestEvents='true', maxIncomingMigrations='2', maxOutgoingMigrations='2', convergenceSchedule='[init=[{name=setDowntime, params=[100]}], stalling=[{limit=1, action={name=setDowntime, params=[150]}}, {limit=2, action={name=setDowntime, params=[200]}}, {limit=3, action={name=setDowntime, params=[300]}}, {limit=4, action={name=setDowntime, params=[400]}}, {limit=6, action={name=setDowntime, params=[500]}}, {limit=-1, action={name=abort, params=[]}}]]', dstQemu='192.168.128.76'}), log id: 7d20b357
2019-01-02 09:41:41,677+08 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (EE-ManagedThreadFactory-engine-Thread-168945) [] FINISH, MigrateBrokerVDSCommand, log id: 7d20b357
2019-01-02 09:41:41,682+08 INFO  [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (EE-ManagedThreadFactory-engine-Thread-168945) [] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 4611863
2019-01-02 09:41:41,686+08 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-168945) [] EVENT_ID: VM_MIGRATION_START(62), Migration started (VM: win7, Source: horeb66, Destination: horeb67, User: admin@internal-authz). 
2019-01-02 09:41:56,575+08 INFO  [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-45) [] VM 'eff7f697-8a07-46e5-a631-a1011a0eb836'(win7) moved from 'MigratingFrom' --> 'Up'
2019-01-02 09:41:56,575+08 INFO  [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-45) [] Adding VM 'eff7f697-8a07-46e5-a631-a1011a0eb836'(win7) to re-run list
2019-01-02 09:41:56,584+08 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.VmsMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-45) [] Rerun VM 'eff7f697-8a07-46e5-a631-a1011a0eb836'. Called from VDS 'horeb66'
2019-01-02 09:41:56,625+08 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-168952) [] START, MigrateStatusVDSCommand(HostName = horeb66, MigrateStatusVDSCommandParameters:{hostId='0aff0075-4b41-4f37-98de-7433a17cd47e', vmId='eff7f697-8a07-46e5-a631-a1011a0eb836'}), log id: 4ded4ce9
2019-01-02 09:41:56,628+08 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-168952) [] FINISH, MigrateStatusVDSCommand, log id: 4ded4ce9
2019-01-02 09:41:56,638+08 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-168952) [] EVENT_ID: VM_MIGRATION_TRYING_RERUN(128), Failed to migrate VM win7 to Host horeb67 . Trying to migrate to another Host.
2019-01-02 09:41:56,695+08 WARN  [org.ovirt.engine.core.bll.MigrateVmCommand] (EE-ManagedThreadFactory-engine-Thread-168952) [] Validation of action 'MigrateVm' failed for user admin@internal-authz. Reasons: VAR__ACTION__MIGRATE,VAR__TYPE__VM,VAR__ACTION__MIGRATE,VAR__TYPE__VM,VAR__ACTION__MIGRATE,VAR__TYPE__VM,VAR__ACTION__MIGRATE,VAR__TYPE__VM,SCHEDULING_NO_HOSTS
2019-01-02 09:41:56,696+08 INFO  [org.ovirt.engine.core.bll.MigrateVmCommand] (EE-ManagedThreadFactory-engine-Thread-168952) [] Lock freed to object 'EngineLock:{exclusiveLocks='[eff7f697-8a07-46e5-a631-a1011a0eb836=VM]', sharedLocks=''}'
2019-01-02 09:41:56,706+08 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-168952) [] EVENT_ID: VM_MIGRATION_NO_VDS_TO_MIGRATE_TO(166), No available host was found to migrate VM win7 to.
2019-01-02 09:41:56,709+08 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-168952) [] EVENT_ID: VM_MIGRATION_FAILED(65), Migration failed  (VM: win7, Source: horeb66).



 
_______________________________________________
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/IC636YTVOKOKC2ULFXHL363OXZCUTSUZ/