[ovirt-users] Attach Export Domain to another Data Center - Failure
Fred Rolland
frolland at redhat.com
Wed Mar 21 11:29:54 UTC 2018
node11 fails to mount the export domain from node 10.
You can try manually to see if you have access from node 11.
Look at this page to debug the NFS connection:
https://www.ovirt.org/documentation/how-to/troubleshooting/troubleshooting-nfs-storage-issues/
On Wed, Mar 21, 2018 at 12:04 PM, Andrei Verovski <andreil1 at starlett.lv>
wrote:
> Hi,
>
>
> Errors occurred at 10.43 AM and 10.46 AM
> node00.starlett.lv - 192.168.0.4 - oVirt host engine (separate PC, not
> hosted)
> node10.starlett.lv - 192.168.0.5 - host #1 of DC #1, export domain from
> which was detached from DC #1
> node11.starlett.lv - 192.168.0.6 - host #1 of DC #2,
>
> Logs from DC#2 node11, to which I’m trying to attach export domain located
> at NFS share node10.
>
> *grep 570ec5d9-fff5-4656-afbd-90b3207a616e*
>>
> within vdsm.log returned nothing, so I did
>
>
> *grep -n 10:43 vdsm.log | tail -1000*
> 1011:2018-03-21 06:10:43,077+0200 INFO (jsonrpc/3) [api.host] START
> getAllVmStats() from=::1,36114 (api:46)
> 1012:2018-03-21 06:10:43,077+0200 INFO (jsonrpc/3) [api.host] FINISH
> getAllVmStats return={'status': {'message': 'Done', 'code': 0},
> 'statsList': (suppressed)} from=::1,36114 (api:52)
> 1013:2018-03-21 06:10:43,077+0200 INFO (jsonrpc/3)
> [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.01
> seconds (__init__:573)
> 1014:2018-03-21 06:10:43,868+0200 INFO (jsonrpc/7) [api.host] START
> getAllVmStats() from=::ffff:192.168.0.4,49914 (api:46)
> 1015:2018-03-21 06:10:43,868+0200 INFO (jsonrpc/7) [api.host] FINISH
> getAllVmStats return={'status': {'message': 'Done', 'code': 0},
> 'statsList': (suppressed)} from=::ffff:192.168.0.4,49914 (api:52)
> 1016:2018-03-21 06:10:43,868+0200 INFO (jsonrpc/7)
> [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
> seconds (__init__:573)
> 19729:2018-03-21 09:10:43,641+0200 INFO (jsonrpc/5) [api.host] START
> getAllVmStats() from=::1,36114 (api:46)
> 19730:2018-03-21 09:10:43,641+0200 INFO (jsonrpc/5) [api.host] FINISH
> getAllVmStats return={'status': {'message': 'Done', 'code': 0},
> 'statsList': (suppressed)} from=::1,36114 (api:52)
> 19731:2018-03-21 09:10:43,642+0200 INFO (jsonrpc/5)
> [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
> seconds (__init__:573)
> 29302:2018-03-21 10:43:00,085+0200 INFO (periodic/2) [vdsm.api] START
> repoStats(domains=()) from=internal, task_id=7b690c8a-5470-44da-a8f3-7e7e9b018e88
> (api:46)
> 29303:2018-03-21 10:43:00,085+0200 INFO (periodic/2) [vdsm.api] FINISH
> repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
> 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000477038',
> 'lastCheck': '0.5', 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '0.000498252', 'lastCheck': '0.4', 'valid': True}} from=internal,
> task_id=7b690c8a-5470-44da-a8f3-7e7e9b018e88 (api:52)
> 29304:2018-03-21 10:43:00,086+0200 INFO (periodic/2) [vdsm.api] START
> multipath_health() from=internal, task_id=9a625d43-a03c-429c-856f-9aa8e5ff65b5
> (api:46)
> 29305:2018-03-21 10:43:00,086+0200 INFO (periodic/2) [vdsm.api] FINISH
> multipath_health return={} from=internal, task_id=9a625d43-a03c-429c-856f-9aa8e5ff65b5
> (api:52)
> 29306:2018-03-21 10:43:05,382+0200 INFO (jsonrpc/5) [api.host] START
> getAllVmStats() from=::ffff:192.168.0.4,49914 (api:46)
> 29307:2018-03-21 10:43:05,383+0200 INFO (jsonrpc/5) [api.host] FINISH
> getAllVmStats return={'status': {'message': 'Done', 'code': 0},
> 'statsList': (suppressed)} from=::ffff:192.168.0.4,49914 (api:52)
> 29308:2018-03-21 10:43:05,383+0200 INFO (jsonrpc/5)
> [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
> seconds (__init__:573)
> 29309:2018-03-21 10:43:06,248+0200 INFO (jsonrpc/3) [api.host] START
> getStats() from=::ffff:192.168.0.4,49914 (api:46)
> 29310:2018-03-21 10:43:06,249+0200 INFO (jsonrpc/3) [vdsm.api] START
> repoStats(domains=()) from=::ffff:192.168.0.4,49914,
> task_id=f80f9bb8-2e84-4f45-ab58-09a88e808cf3 (api:46)
> 29311:2018-03-21 10:43:06,250+0200 INFO (jsonrpc/3) [vdsm.api] FINISH
> repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
> 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000477038',
> 'lastCheck': '6.7', 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '0.000498252', 'lastCheck': '6.6', 'valid': True}}
> from=::ffff:192.168.0.4,49914, task_id=f80f9bb8-2e84-4f45-ab58-09a88e808cf3
> (api:52)
> 29312:2018-03-21 10:43:06,250+0200 INFO (jsonrpc/3) [vdsm.api] START
> multipath_health() from=::ffff:192.168.0.4,49914,
> task_id=ddd7064d-cd12-43bf-a177-b2a93cc9035e (api:46)
> 29313:2018-03-21 10:43:06,250+0200 INFO (jsonrpc/3) [vdsm.api] FINISH
> multipath_health return={} from=::ffff:192.168.0.4,49914,
> task_id=ddd7064d-cd12-43bf-a177-b2a93cc9035e (api:52)
> 29314:2018-03-21 10:43:06,256+0200 INFO (jsonrpc/3) [api.host] FINISH
> getStats return={'status': {'message': 'Done', 'code': 0}, 'info':
> {'cpuStatistics': {'11': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys':
> '0.27', 'cpuIdle': '99.60'}, '10': {'cpuUser': '0.13', 'nodeIndex': 0,
> 'cpuSys': '0.40', 'cpuIdle': '99.47'}, '13': {'cpuUser': '0.07',
> 'nodeIndex': 1, 'cpuSys': '0.20', 'cpuIdle': '99.73'}, '12': {'cpuUser':
> '0.13', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '15':
> {'cpuUser': '0.20', 'nodeIndex': 1, 'cpuSys': '0.47', 'cpuIdle': '99.33'},
> '14': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle':
> '99.60'}, '1': {'cpuUser': '1.13', 'nodeIndex': 1, 'cpuSys': '1.60',
> 'cpuIdle': '97.27'}, '0': {'cpuUser': '0.47', 'nodeIndex': 0, 'cpuSys':
> '0.27', 'cpuIdle': '99.26'}, '3': {'cpuUser': '0.07', 'nodeIndex': 1,
> 'cpuSys': '0.20', 'cpuIdle': '99.73'}, '2': {'cpuUser': '0.33',
> 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.54'}, '5': {'cpuUser':
> '0.13', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.74'}, '4':
> {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.67'},
> '7': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.60', 'cpuIdle':
> '99.27'}, '6': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.33',
> 'cpuIdle': '99.54'}, '9': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys':
> '0.27', 'cpuIdle': '99.66'}, '8': {'cpuUser': '0.13', 'nodeIndex': 0,
> 'cpuSys': '0.33', 'cpuIdle': '99.54'}}, 'numaNodeMemFree': {'1':
> {'memPercent': 7, 'memFree': '15142'}, '0': {'memPercent': 5, 'memFree':
> '15349'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes':
> True, 'vmCount': 0, 'memUsed': '3', 'storageDomains':
> {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0, 'actual': True,
> 'version': 4, 'acquired': True, 'delay': '0.000477038', 'lastCheck': '6.7',
> 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f': {'code': 0,
> 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000498252',
> 'lastCheck': '6.6', 'valid': True}}, 'incomingVmMigrations': 0, 'network':
> {'enp3s0f0': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621780.072418, 'name': 'enp3s0f0', 'tx': '1091287978', 'txDropped':
> '0', 'rx': '11914868563', 'rxErrors': '0', 'speed': '100', 'rxDropped':
> '1439'}, 'ovirtmgmt': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621780.072418, 'name': 'ovirtmgmt', 'tx': '1048639112', 'txDropped':
> '0', 'rx': '11615209092', 'rxErrors': '0', 'speed': '1000', 'rxDropped':
> '0'}, 'lo': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621780.072418, 'name': 'lo', 'tx': '58057075945', 'txDropped': '0',
> 'rx': '58057075945', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'},
> 'enp3s0f1': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621780.072418, 'name': 'enp3s0f1', 'tx': '0', 'txDropped': '0', 'rx':
> '66839', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'},
> ';vdsmdummy;': {'txErrors': '0', 'state': 'down', 'sampleTime':
> 1521621780.072418, 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0',
> 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'enp4s0f0':
> {'txErrors': '0', 'state': 'down', 'sampleTime': 1521621780.072418, 'name':
> 'enp4s0f0', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0',
> 'speed': '1000', 'rxDropped': '0'}, 'enp4s0f1': {'txErrors': '0', 'state':
> 'down', 'sampleTime': 1521621780.072418, 'name': 'enp4s0f1', 'tx': '0',
> 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped':
> '0'}}, 'txDropped': '0', 'anonHugePages': '204', 'ksmPages': 100,
> 'elapsedTime': '76345.09', 'cpuLoad': '0.16', 'cpuSys': '0.36',
> 'diskStats': {'/var/log': {'free': '7344'}, '/var/run/vdsm/': {'free':
> '16060'}, '/tmp': {'free': '906'}}, 'cpuUserVdsmd': '1.07',
> 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState': False,
> 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 31081, 'bootTime':
> '1521545382', 'haStats': {'active': False, 'configured': False, 'score': 0,
> 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus':
> 'active', 'multipathHealth': {}, 'rxDropped': '1439',
> 'outgoingVmMigrations': 0, 'swapTotal': 12287, 'swapFree': 12287,
> 'hugepages': defaultdict(<type 'dict'>, {2048: {'resv_hugepages': 0,
> 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0,
> 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0},
> 1048576: {'resv_hugepages': 0, 'free_hugepages': 0,
> 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages':
> 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}), 'dateTime':
> '2018-03-21T08:43:06 GMT', 'cpuUser': '0.21', 'memFree': 31337, 'cpuIdle':
> '99.43', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd': '0.93'}}
> from=::ffff:192.168.0.4,49914 (api:52)
> 29315:2018-03-21 10:43:06,258+0200 INFO (jsonrpc/3)
> [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds
> (__init__:573)
> 29316:2018-03-21 10:43:06,632+0200 INFO (jsonrpc/0) [api.host] START
> getAllVmStats() from=::1,36114 (api:46)
> 29317:2018-03-21 10:43:06,632+0200 INFO (jsonrpc/0) [api.host] FINISH
> getAllVmStats return={'status': {'message': 'Done', 'code': 0},
> 'statsList': (suppressed)} from=::1,36114 (api:52)
> 29318:2018-03-21 10:43:06,633+0200 INFO (jsonrpc/0)
> [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
> seconds (__init__:573)
> 29319:2018-03-21 10:43:07,415+0200 INFO (jsonrpc/4) [vdsm.api] START
> getSpmStatus(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
> options=None) from=::ffff:192.168.0.4,49914, task_id=276c13c7-61ec-4782-a0d9-5cd0f0b0a729
> (api:46)
> 29320:2018-03-21 10:43:07,420+0200 INFO (jsonrpc/4) [vdsm.api] FINISH
> getSpmStatus return={'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver':
> 6L}} from=::ffff:192.168.0.4,49914, task_id=276c13c7-61ec-4782-a0d9-5cd0f0b0a729
> (api:52)
> 29321:2018-03-21 10:43:07,420+0200 INFO (jsonrpc/4)
> [jsonrpc.JsonRpcServer] RPC call StoragePool.getSpmStatus succeeded in 0.01
> seconds (__init__:573)
> 29322:2018-03-21 10:43:07,427+0200 INFO (jsonrpc/6) [vdsm.api] START
> getStoragePoolInfo(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
> options=None) from=::ffff:192.168.0.4,49920, task_id=344b18df-a428-429a-b49f-381f901d1618
> (api:46)
> 29323:2018-03-21 10:43:07,434+0200 INFO (jsonrpc/6) [vdsm.api] FINISH
> getStoragePoolInfo return={'info': {'name': 'No Description', 'isoprefix':
> u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
> nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
> 11111111-1111-1111-1111-111111111111', 'pool_status': 'connected',
> 'lver': 6L, 'domains': u'ef184b28-1dbc-45ed-b0b3-
> 85e780cce5d8:Active,8a2c304b-c8ae-438b-af54-fc8797ea149f:Active',
> 'master_uuid': u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8', 'version': '4',
> 'spm_id': 1, 'type': 'NFS', 'master_ver': 3}, 'dominfo':
> {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'status': u'Active',
> 'diskfree': '1868692455424', 'isoprefix': '', 'alerts': [], 'disktotal':
> '1968811540480', 'version': 4}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'status': u'Active', 'diskfree': '1868692455424', 'isoprefix':
> u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
> nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
> 11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal':
> '1968811540480', 'version': 0}}} from=::ffff:192.168.0.4,49920,
> task_id=344b18df-a428-429a-b49f-381f901d1618 (api:52)
> 29324:2018-03-21 10:43:07,435+0200 INFO (jsonrpc/6)
> [jsonrpc.JsonRpcServer] RPC call StoragePool.getInfo succeeded in 0.01
> seconds (__init__:573)
> 29325:2018-03-21 10:43:13,839+0200 WARN (vdsm.Scheduler) [Executor]
> Worker blocked: <Worker name=jsonrpc/1 running <Task <JsonRpcTask
> {'params': {u'connectionParams': [{u'id': u'461f65a9-3a81-4f3f-a46d-c5ed12520524',
> u'connection': u'node10.starlett.lv:/vmdata/nfs/exports', u'iqn': u'',
> u'user': u'', u'tpgt': u'1', u'protocol_version': u'auto', u'password':
> '********', u'port': u''}], u'storagepoolID': u'00000000-0000-0000-0000-000000000000',
> u'domainType': 1}, 'jsonrpc': '2.0', 'method': u'StoragePool.connectStorageServer',
> 'id': u'5f98dfd5-998f-44a6-8594-e5f40144b0ed'} at 0x4076150> timeout=60,
> duration=120 at 0x402fe50> task#=3754 at 0x353e690>, traceback:
> 29376:2018-03-21 10:43:15,104+0200 INFO (periodic/2) [vdsm.api] START
> repoStats(domains=()) from=internal, task_id=672a883e-1d60-4981-9eee-bdcac2ec30c1
> (api:46)
> 29377:2018-03-21 10:43:15,105+0200 INFO (periodic/2) [vdsm.api] FINISH
> repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
> 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000488777',
> 'lastCheck': '5.6', 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '0.000481962', 'lastCheck': '5.4', 'valid': True}} from=internal,
> task_id=672a883e-1d60-4981-9eee-bdcac2ec30c1 (api:52)
> 29378:2018-03-21 10:43:15,105+0200 INFO (periodic/2) [vdsm.api] START
> multipath_health() from=internal, task_id=55f1b797-da2a-4b07-affe-5b9e41a74307
> (api:46)
> 29379:2018-03-21 10:43:15,106+0200 INFO (periodic/2) [vdsm.api] FINISH
> multipath_health return={} from=internal, task_id=55f1b797-da2a-4b07-affe-5b9e41a74307
> (api:52)
> 29380:2018-03-21 10:43:17,645+0200 INFO (jsonrpc/7) [vdsm.api] START
> getSpmStatus(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
> options=None) from=::ffff:192.168.0.4,49914, task_id=52a6ddeb-3c20-4f3f-b27d-94d19580b473
> (api:46)
> 29381:2018-03-21 10:43:17,649+0200 INFO (jsonrpc/7) [vdsm.api] FINISH
> getSpmStatus return={'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver':
> 6L}} from=::ffff:192.168.0.4,49914, task_id=52a6ddeb-3c20-4f3f-b27d-94d19580b473
> (api:52)
> 29382:2018-03-21 10:43:17,650+0200 INFO (jsonrpc/7)
> [jsonrpc.JsonRpcServer] RPC call StoragePool.getSpmStatus succeeded in 0.01
> seconds (__init__:573)
> 29383:2018-03-21 10:43:17,656+0200 INFO (jsonrpc/2) [vdsm.api] START
> getStoragePoolInfo(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
> options=None) from=::ffff:192.168.0.4,49920, task_id=1a103883-6a6a-4d61-ade6-96559591ddb0
> (api:46)
> 29384:2018-03-21 10:43:17,663+0200 INFO (jsonrpc/2) [vdsm.api] FINISH
> getStoragePoolInfo return={'info': {'name': 'No Description', 'isoprefix':
> u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
> nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
> 11111111-1111-1111-1111-111111111111', 'pool_status': 'connected',
> 'lver': 6L, 'domains': u'ef184b28-1dbc-45ed-b0b3-
> 85e780cce5d8:Active,8a2c304b-c8ae-438b-af54-fc8797ea149f:Active',
> 'master_uuid': u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8', 'version': '4',
> 'spm_id': 1, 'type': 'NFS', 'master_ver': 3}, 'dominfo':
> {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'status': u'Active',
> 'diskfree': '1868692455424', 'isoprefix': '', 'alerts': [], 'disktotal':
> '1968811540480', 'version': 4}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'status': u'Active', 'diskfree': '1868692455424', 'isoprefix':
> u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
> nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
> 11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal':
> '1968811540480', 'version': 0}}} from=::ffff:192.168.0.4,49920,
> task_id=1a103883-6a6a-4d61-ade6-96559591ddb0 (api:52)
> 29385:2018-03-21 10:43:17,663+0200 INFO (jsonrpc/2)
> [jsonrpc.JsonRpcServer] RPC call StoragePool.getInfo succeeded in 0.01
> seconds (__init__:573)
> 29386:2018-03-21 10:43:18,882+0200 ERROR (jsonrpc/1) [storage.HSM] Could
> not connect to storageServer (hsm:2407)
> 29406:2018-03-21 10:43:18,883+0200 INFO (jsonrpc/1) [vdsm.api] FINISH
> connectStorageServer return={'statuslist': [{'status': 477, 'id':
> u'461f65a9-3a81-4f3f-a46d-c5ed12520524'}]} from=::ffff:192.168.0.4,49914,
> flow_id=4a53b512, task_id=e598cbe0-cde8-4c73-b526-4398df05e67f (api:52)
> 29407:2018-03-21 10:43:18,883+0200 INFO (jsonrpc/1)
> [jsonrpc.JsonRpcServer] RPC call StoragePool.connectStorageServer
> succeeded in 125.04 seconds (__init__:573)
> 29408:2018-03-21 10:43:20,387+0200 INFO (jsonrpc/5) [api.host] START
> getAllVmStats() from=::ffff:192.168.0.4,49914 (api:46)
> 29409:2018-03-21 10:43:20,388+0200 INFO (jsonrpc/5) [api.host] FINISH
> getAllVmStats return={'status': {'message': 'Done', 'code': 0},
> 'statsList': (suppressed)} from=::ffff:192.168.0.4,49914 (api:52)
> 29410:2018-03-21 10:43:20,388+0200 INFO (jsonrpc/5)
> [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.01
> seconds (__init__:573)
> 29411:2018-03-21 10:43:21,497+0200 INFO (jsonrpc/3) [api.host] START
> getStats() from=::ffff:192.168.0.4,49914 (api:46)
> 29412:2018-03-21 10:43:21,498+0200 INFO (jsonrpc/3) [vdsm.api] START
> repoStats(domains=()) from=::ffff:192.168.0.4,49914,
> task_id=a7469c1f-3d6d-42e7-8798-5c5335da4a9b (api:46)
> 29413:2018-03-21 10:43:21,498+0200 INFO (jsonrpc/3) [vdsm.api] FINISH
> repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
> 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000445096',
> 'lastCheck': '1.9', 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '0.000296586', 'lastCheck': '1.8', 'valid': True}}
> from=::ffff:192.168.0.4,49914, task_id=a7469c1f-3d6d-42e7-8798-5c5335da4a9b
> (api:52)
> 29414:2018-03-21 10:43:21,499+0200 INFO (jsonrpc/3) [vdsm.api] START
> multipath_health() from=::ffff:192.168.0.4,49914,
> task_id=9fa5dae3-e2bd-4186-9a1d-afb0aa6e2511 (api:46)
> 29415:2018-03-21 10:43:21,499+0200 INFO (jsonrpc/3) [vdsm.api] FINISH
> multipath_health return={} from=::ffff:192.168.0.4,49914,
> task_id=9fa5dae3-e2bd-4186-9a1d-afb0aa6e2511 (api:52)
> 29416:2018-03-21 10:43:21,505+0200 INFO (jsonrpc/3) [api.host] FINISH
> getStats return={'status': {'message': 'Done', 'code': 0}, 'info':
> {'cpuStatistics': {'11': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys':
> '0.07', 'cpuIdle': '99.86'}, '10': {'cpuUser': '0.07', 'nodeIndex': 0,
> 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '13': {'cpuUser': '0.07',
> 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '12': {'cpuUser':
> '0.33', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.47'}, '15':
> {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.87'},
> '14': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle':
> '99.60'}, '1': {'cpuUser': '1.00', 'nodeIndex': 1, 'cpuSys': '1.00',
> 'cpuIdle': '98.00'}, '0': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys':
> '0.07', 'cpuIdle': '99.86'}, '3': {'cpuUser': '0.00', 'nodeIndex': 1,
> 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '2': {'cpuUser': '0.27',
> 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.66'}, '5': {'cpuUser':
> '0.07', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '4':
> {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'},
> '7': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.27', 'cpuIdle':
> '99.66'}, '6': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07',
> 'cpuIdle': '99.86'}, '9': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys':
> '0.13', 'cpuIdle': '99.80'}, '8': {'cpuUser': '0.00', 'nodeIndex': 0,
> 'cpuSys': '0.07', 'cpuIdle': '99.93'}}, 'numaNodeMemFree': {'1':
> {'memPercent': 7, 'memFree': '15142'}, '0': {'memPercent': 5, 'memFree':
> '15351'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes':
> True, 'vmCount': 0, 'memUsed': '3', 'storageDomains':
> {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0, 'actual': True,
> 'version': 4, 'acquired': True, 'delay': '0.000445096', 'lastCheck': '1.9',
> 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f': {'code': 0,
> 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000296586',
> 'lastCheck': '1.8', 'valid': True}}, 'incomingVmMigrations': 0, 'network':
> {'enp3s0f0': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621795.091263, 'name': 'enp3s0f0', 'tx': '1091295628', 'txDropped':
> '0', 'rx': '11914875210', 'rxErrors': '0', 'speed': '100', 'rxDropped':
> '1442'}, 'ovirtmgmt': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621795.091263, 'name': 'ovirtmgmt', 'tx': '1048646480', 'txDropped':
> '0', 'rx': '11615214750', 'rxErrors': '0', 'speed': '1000', 'rxDropped':
> '0'}, 'lo': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621795.091263, 'name': 'lo', 'tx': '58066397668', 'txDropped': '0',
> 'rx': '58066397668', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'},
> 'enp3s0f1': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621795.091263, 'name': 'enp3s0f1', 'tx': '0', 'txDropped': '0', 'rx':
> '66839', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'},
> ';vdsmdummy;': {'txErrors': '0', 'state': 'down', 'sampleTime':
> 1521621795.091263, 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0',
> 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'enp4s0f0':
> {'txErrors': '0', 'state': 'down', 'sampleTime': 1521621795.091263, 'name':
> 'enp4s0f0', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0',
> 'speed': '1000', 'rxDropped': '0'}, 'enp4s0f1': {'txErrors': '0', 'state':
> 'down', 'sampleTime': 1521621795.091263, 'name': 'enp4s0f1', 'tx': '0',
> 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped':
> '0'}}, 'txDropped': '0', 'anonHugePages': '204', 'ksmPages': 100,
> 'elapsedTime': '76360.34', 'cpuLoad': '0.15', 'cpuSys': '0.18',
> 'diskStats': {'/var/log': {'free': '7344'}, '/var/run/vdsm/': {'free':
> '16060'}, '/tmp': {'free': '906'}}, 'cpuUserVdsmd': '1.07',
> 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState': False,
> 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 31031, 'bootTime':
> '1521545382', 'haStats': {'active': False, 'configured': False, 'score': 0,
> 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus':
> 'active', 'multipathHealth': {}, 'rxDropped': '1442',
> 'outgoingVmMigrations': 0, 'swapTotal': 12287, 'swapFree': 12287,
> 'hugepages': defaultdict(<type 'dict'>, {2048: {'resv_hugepages': 0,
> 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0,
> 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0},
> 1048576: {'resv_hugepages': 0, 'free_hugepages': 0,
> 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages':
> 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}), 'dateTime':
> '2018-03-21T08:43:21 GMT', 'cpuUser': '0.15', 'memFree': 31287, 'cpuIdle':
> '99.67', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd': '0.67'}}
> from=::ffff:192.168.0.4,49914 (api:52)
> 29417:2018-03-21 10:43:21,508+0200 INFO (jsonrpc/3)
> [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.02 seconds
> (__init__:573)
> 29418:2018-03-21 10:43:21,651+0200 INFO (jsonrpc/0) [api.host] START
> getAllVmStats() from=::1,36114 (api:46)
> 29419:2018-03-21 10:43:21,651+0200 INFO (jsonrpc/0) [api.host] FINISH
> getAllVmStats return={'status': {'message': 'Done', 'code': 0},
> 'statsList': (suppressed)} from=::1,36114 (api:52)
> 29420:2018-03-21 10:43:21,652+0200 INFO (jsonrpc/0)
> [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
> seconds (__init__:573)
> 29421:2018-03-21 10:43:27,764+0200 INFO (jsonrpc/4) [vdsm.api] START
> getSpmStatus(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
> options=None) from=::ffff:192.168.0.4,49914, task_id=46d19cab-8903-4c52-9bb2-4dd8f370997e
> (api:46)
> 29422:2018-03-21 10:43:27,769+0200 INFO (jsonrpc/4) [vdsm.api] FINISH
> getSpmStatus return={'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver':
> 6L}} from=::ffff:192.168.0.4,49914, task_id=46d19cab-8903-4c52-9bb2-4dd8f370997e
> (api:52)
> 29423:2018-03-21 10:43:27,770+0200 INFO (jsonrpc/4)
> [jsonrpc.JsonRpcServer] RPC call StoragePool.getSpmStatus succeeded in 0.01
> seconds (__init__:573)
> 29424:2018-03-21 10:43:27,815+0200 INFO (jsonrpc/6) [vdsm.api] START
> getStoragePoolInfo(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
> options=None) from=::ffff:192.168.0.4,49920, task_id=290acc73-8f25-4fd7-9263-7bd05d46430a
> (api:46)
> 29425:2018-03-21 10:43:27,822+0200 INFO (jsonrpc/6) [vdsm.api] FINISH
> getStoragePoolInfo return={'info': {'name': 'No Description', 'isoprefix':
> u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
> nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
> 11111111-1111-1111-1111-111111111111', 'pool_status': 'connected',
> 'lver': 6L, 'domains': u'ef184b28-1dbc-45ed-b0b3-
> 85e780cce5d8:Active,8a2c304b-c8ae-438b-af54-fc8797ea149f:Active',
> 'master_uuid': u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8', 'version': '4',
> 'spm_id': 1, 'type': 'NFS', 'master_ver': 3}, 'dominfo':
> {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'status': u'Active',
> 'diskfree': '1868692455424', 'isoprefix': '', 'alerts': [], 'disktotal':
> '1968811540480', 'version': 4}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'status': u'Active', 'diskfree': '1868692455424', 'isoprefix':
> u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
> nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
> 11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal':
> '1968811540480', 'version': 0}}} from=::ffff:192.168.0.4,49920,
> task_id=290acc73-8f25-4fd7-9263-7bd05d46430a (api:52)
> 29426:2018-03-21 10:43:27,823+0200 INFO (jsonrpc/6)
> [jsonrpc.JsonRpcServer] RPC call StoragePool.getInfo succeeded in 0.01
> seconds (__init__:573)
> 29427:2018-03-21 10:43:30,121+0200 INFO (periodic/3) [vdsm.api] START
> repoStats(domains=()) from=internal, task_id=0abdebcf-2b0b-4d33-b552-c0f2ed3e567b
> (api:46)
> 29428:2018-03-21 10:43:30,122+0200 INFO (periodic/3) [vdsm.api] FINISH
> repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
> 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000402077',
> 'lastCheck': '0.6', 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '0.000344445', 'lastCheck': '0.5', 'valid': True}} from=internal,
> task_id=0abdebcf-2b0b-4d33-b552-c0f2ed3e567b (api:52)
> 29429:2018-03-21 10:43:30,122+0200 INFO (periodic/3) [vdsm.api] START
> multipath_health() from=internal, task_id=acde7e7d-ab4a-4811-934f-4d312da33208
> (api:46)
> 29430:2018-03-21 10:43:30,123+0200 INFO (periodic/3) [vdsm.api] FINISH
> multipath_health return={} from=internal, task_id=acde7e7d-ab4a-4811-934f-4d312da33208
> (api:52)
> 29431:2018-03-21 10:43:35,393+0200 INFO (jsonrpc/7) [api.host] START
> getAllVmStats() from=::ffff:192.168.0.4,49914 (api:46)
> 29432:2018-03-21 10:43:35,394+0200 INFO (jsonrpc/7) [api.host] FINISH
> getAllVmStats return={'status': {'message': 'Done', 'code': 0},
> 'statsList': (suppressed)} from=::ffff:192.168.0.4,49914 (api:52)
> 29433:2018-03-21 10:43:35,394+0200 INFO (jsonrpc/7)
> [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
> seconds (__init__:573)
> 29434:2018-03-21 10:43:36,671+0200 INFO (jsonrpc/2) [api.host] START
> getAllVmStats() from=::1,36114 (api:46)
> 29435:2018-03-21 10:43:36,671+0200 INFO (jsonrpc/2) [api.host] FINISH
> getAllVmStats return={'status': {'message': 'Done', 'code': 0},
> 'statsList': (suppressed)} from=::1,36114 (api:52)
> 29436:2018-03-21 10:43:36,672+0200 INFO (jsonrpc/2)
> [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
> seconds (__init__:573)
> 29437:2018-03-21 10:43:37,356+0200 INFO (jsonrpc/1) [api.host] START
> getStats() from=::ffff:192.168.0.4,49914 (api:46)
> 29438:2018-03-21 10:43:37,357+0200 INFO (jsonrpc/1) [vdsm.api] START
> repoStats(domains=()) from=::ffff:192.168.0.4,49914,
> task_id=bc14e253-d3d4-4cc3-9ae1-c0e6ac6e37f7 (api:46)
> 29439:2018-03-21 10:43:37,357+0200 INFO (jsonrpc/1) [vdsm.api] FINISH
> repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
> 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000402077',
> 'lastCheck': '7.8', 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '0.000344445', 'lastCheck': '7.7', 'valid': True}}
> from=::ffff:192.168.0.4,49914, task_id=bc14e253-d3d4-4cc3-9ae1-c0e6ac6e37f7
> (api:52)
> 29440:2018-03-21 10:43:37,358+0200 INFO (jsonrpc/1) [vdsm.api] START
> multipath_health() from=::ffff:192.168.0.4,49914,
> task_id=23c698b8-d5bd-4746-b738-97b10f3e4bfb (api:46)
> 29441:2018-03-21 10:43:37,358+0200 INFO (jsonrpc/1) [vdsm.api] FINISH
> multipath_health return={} from=::ffff:192.168.0.4,49914,
> task_id=23c698b8-d5bd-4746-b738-97b10f3e4bfb (api:52)
> 29442:2018-03-21 10:43:37,365+0200 INFO (jsonrpc/1) [api.host] FINISH
> getStats return={'status': {'message': 'Done', 'code': 0}, 'info':
> {'cpuStatistics': {'11': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys':
> '0.07', 'cpuIdle': '99.86'}, '10': {'cpuUser': '3.06', 'nodeIndex': 0,
> 'cpuSys': '0.80', 'cpuIdle': '96.14'}, '13': {'cpuUser': '0.00',
> 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '12': {'cpuUser':
> '0.07', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.73'}, '15':
> {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.87'},
> '14': {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.40', 'cpuIdle':
> '99.33'}, '1': {'cpuUser': '1.13', 'nodeIndex': 1, 'cpuSys': '1.33',
> 'cpuIdle': '97.54'}, '0': {'cpuUser': '0.47', 'nodeIndex': 0, 'cpuSys':
> '0.33', 'cpuIdle': '99.20'}, '3': {'cpuUser': '0.00', 'nodeIndex': 1,
> 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '2': {'cpuUser': '0.67',
> 'nodeIndex': 0, 'cpuSys': '0.40', 'cpuIdle': '98.93'}, '5': {'cpuUser':
> '0.13', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.74'}, '4':
> {'cpuUser': '0.47', 'nodeIndex': 0, 'cpuSys': '0.40', 'cpuIdle': '99.13'},
> '7': {'cpuUser': '0.33', 'nodeIndex': 1, 'cpuSys': '0.73', 'cpuIdle':
> '98.94'}, '6': {'cpuUser': '5.39', 'nodeIndex': 0, 'cpuSys': '0.47',
> 'cpuIdle': '94.14'}, '9': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys':
> '0.27', 'cpuIdle': '99.66'}, '8': {'cpuUser': '0.53', 'nodeIndex': 0,
> 'cpuSys': '0.40', 'cpuIdle': '99.07'}}, 'numaNodeMemFree': {'1':
> {'memPercent': 7, 'memFree': '15143'}, '0': {'memPercent': 5, 'memFree':
> '15348'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes':
> True, 'vmCount': 0, 'memUsed': '3', 'storageDomains':
> {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0, 'actual': True,
> 'version': 4, 'acquired': True, 'delay': '0.000402077', 'lastCheck': '7.8',
> 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f': {'code': 0,
> 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000344445',
> 'lastCheck': '7.7', 'valid': True}}, 'incomingVmMigrations': 0, 'network':
> {'enp3s0f0': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621810.108511, 'name': 'enp3s0f0', 'tx': '1091305295', 'txDropped':
> '0', 'rx': '11914880953', 'rxErrors': '0', 'speed': '100', 'rxDropped':
> '1442'}, 'ovirtmgmt': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621810.108511, 'name': 'ovirtmgmt', 'tx': '1048655897', 'txDropped':
> '0', 'rx': '11615220341', 'rxErrors': '0', 'speed': '1000', 'rxDropped':
> '0'}, 'lo': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621810.108511, 'name': 'lo', 'tx': '58078891247', 'txDropped': '0',
> 'rx': '58078891247', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'},
> 'enp3s0f1': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621810.108511, 'name': 'enp3s0f1', 'tx': '0', 'txDropped': '0', 'rx':
> '66839', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'},
> ';vdsmdummy;': {'txErrors': '0', 'state': 'down', 'sampleTime':
> 1521621810.108511, 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0',
> 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'enp4s0f0':
> {'txErrors': '0', 'state': 'down', 'sampleTime': 1521621810.108511, 'name':
> 'enp4s0f0', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0',
> 'speed': '1000', 'rxDropped': '0'}, 'enp4s0f1': {'txErrors': '0', 'state':
> 'down', 'sampleTime': 1521621810.108511, 'name': 'enp4s0f1', 'tx': '0',
> 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped':
> '0'}}, 'txDropped': '0', 'anonHugePages': '204', 'ksmPages': 100,
> 'elapsedTime': '76376.19', 'cpuLoad': '0.16', 'cpuSys': '0.39',
> 'diskStats': {'/var/log': {'free': '7344'}, '/var/run/vdsm/': {'free':
> '16060'}, '/tmp': {'free': '906'}}, 'cpuUserVdsmd': '1.00',
> 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState': False,
> 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 31080, 'bootTime':
> '1521545382', 'haStats': {'active': False, 'configured': False, 'score': 0,
> 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus':
> 'active', 'multipathHealth': {}, 'rxDropped': '1442',
> 'outgoingVmMigrations': 0, 'swapTotal': 12287, 'swapFree': 12287,
> 'hugepages': defaultdict(<type 'dict'>, {2048: {'resv_hugepages': 0,
> 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0,
> 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0},
> 1048576: {'resv_hugepages': 0, 'free_hugepages': 0,
> 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages':
> 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}), 'dateTime':
> '2018-03-21T08:43:37 GMT', 'cpuUser': '0.79', 'memFree': 31336, 'cpuIdle':
> '98.82', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd': '0.73'}}
> from=::ffff:192.168.0.4,49914 (api:52)
> 29443:2018-03-21 10:43:37,367+0200 INFO (jsonrpc/1)
> [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds
> (__init__:573)
> 29444:2018-03-21 10:43:38,124+0200 INFO (jsonrpc/5) [vdsm.api] START
> getSpmStatus(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
> options=None) from=::ffff:192.168.0.4,49914, task_id=9d2a82dd-c76c-4eb5-a1e1-b49bf004384e
> (api:46)
> 29445:2018-03-21 10:43:38,128+0200 INFO (jsonrpc/5) [vdsm.api] FINISH
> getSpmStatus return={'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver':
> 6L}} from=::ffff:192.168.0.4,49914, task_id=9d2a82dd-c76c-4eb5-a1e1-b49bf004384e
> (api:52)
> 29446:2018-03-21 10:43:38,129+0200 INFO (jsonrpc/5)
> [jsonrpc.JsonRpcServer] RPC call StoragePool.getSpmStatus succeeded in 0.01
> seconds (__init__:573)
> 29447:2018-03-21 10:43:38,171+0200 INFO (jsonrpc/3) [vdsm.api] START
> getStoragePoolInfo(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
> options=None) from=::ffff:192.168.0.4,49920, task_id=62d44b0b-20b2-47ae-bfb8-d4717e301b69
> (api:46)
> 29448:2018-03-21 10:43:38,177+0200 INFO (jsonrpc/3) [vdsm.api] FINISH
> getStoragePoolInfo return={'info': {'name': 'No Description', 'isoprefix':
> u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
> nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
> 11111111-1111-1111-1111-111111111111', 'pool_status': 'connected',
> 'lver': 6L, 'domains': u'ef184b28-1dbc-45ed-b0b3-
> 85e780cce5d8:Active,8a2c304b-c8ae-438b-af54-fc8797ea149f:Active',
> 'master_uuid': u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8', 'version': '4',
> 'spm_id': 1, 'type': 'NFS', 'master_ver': 3}, 'dominfo':
> {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'status': u'Active',
> 'diskfree': '1868692455424', 'isoprefix': '', 'alerts': [], 'disktotal':
> '1968811540480', 'version': 4}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'status': u'Active', 'diskfree': '1868692455424', 'isoprefix':
> u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
> nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
> 11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal':
> '1968811540480', 'version': 0}}} from=::ffff:192.168.0.4,49920,
> task_id=62d44b0b-20b2-47ae-bfb8-d4717e301b69 (api:52)
> 29449:2018-03-21 10:43:38,178+0200 INFO (jsonrpc/3)
> [jsonrpc.JsonRpcServer] RPC call StoragePool.getInfo succeeded in 0.01
> seconds (__init__:573)
> 29450:2018-03-21 10:43:45,147+0200 INFO (periodic/2) [vdsm.api] START
> repoStats(domains=()) from=internal, task_id=49c33285-a222-464d-8984-4e75a5c6354a
> (api:46)
> 29451:2018-03-21 10:43:45,148+0200 INFO (periodic/2) [vdsm.api] FINISH
> repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
> 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000454102',
> 'lastCheck': '5.6', 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '0.000347381', 'lastCheck': '5.5', 'valid': True}} from=internal,
> task_id=49c33285-a222-464d-8984-4e75a5c6354a (api:52)
> 29452:2018-03-21 10:43:45,148+0200 INFO (periodic/2) [vdsm.api] START
> multipath_health() from=internal, task_id=eed6b564-76b1-4f9b-80a2-1f0f6d9bbe32
> (api:46)
> 29453:2018-03-21 10:43:45,149+0200 INFO (periodic/2) [vdsm.api] FINISH
> multipath_health return={} from=internal, task_id=eed6b564-76b1-4f9b-80a2-1f0f6d9bbe32
> (api:52)
> 29454:2018-03-21 10:43:48,405+0200 INFO (jsonrpc/0) [vdsm.api] START
> getSpmStatus(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
> options=None) from=::ffff:192.168.0.4,49914, task_id=84651a37-7d13-41a9-9dcf-2169a9c46fc5
> (api:46)
> 29455:2018-03-21 10:43:48,410+0200 INFO (jsonrpc/0) [vdsm.api] FINISH
> getSpmStatus return={'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver':
> 6L}} from=::ffff:192.168.0.4,49914, task_id=84651a37-7d13-41a9-9dcf-2169a9c46fc5
> (api:52)
> 29456:2018-03-21 10:43:48,411+0200 INFO (jsonrpc/0)
> [jsonrpc.JsonRpcServer] RPC call StoragePool.getSpmStatus succeeded in 0.01
> seconds (__init__:573)
> 29457:2018-03-21 10:43:48,417+0200 INFO (jsonrpc/4) [vdsm.api] START
> getStoragePoolInfo(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
> options=None) from=::ffff:192.168.0.4,49920, task_id=4ebd1d1a-11ac-41ee-b459-7e45f008776b
> (api:46)
> 29458:2018-03-21 10:43:48,423+0200 INFO (jsonrpc/4) [vdsm.api] FINISH
> getStoragePoolInfo return={'info': {'name': 'No Description', 'isoprefix':
> u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
> nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
> 11111111-1111-1111-1111-111111111111', 'pool_status': 'connected',
> 'lver': 6L, 'domains': u'ef184b28-1dbc-45ed-b0b3-
> 85e780cce5d8:Active,8a2c304b-c8ae-438b-af54-fc8797ea149f:Active',
> 'master_uuid': u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8', 'version': '4',
> 'spm_id': 1, 'type': 'NFS', 'master_ver': 3}, 'dominfo':
> {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'status': u'Active',
> 'diskfree': '1868692455424', 'isoprefix': '', 'alerts': [], 'disktotal':
> '1968811540480', 'version': 4}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'status': u'Active', 'diskfree': '1868692455424', 'isoprefix':
> u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
> nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
> 11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal':
> '1968811540480', 'version': 0}}} from=::ffff:192.168.0.4,49920,
> task_id=4ebd1d1a-11ac-41ee-b459-7e45f008776b (api:52)
> 29459:2018-03-21 10:43:48,424+0200 INFO (jsonrpc/4)
> [jsonrpc.JsonRpcServer] RPC call StoragePool.getInfo succeeded in 0.01
> seconds (__init__:573)
> 29460:2018-03-21 10:43:50,399+0200 INFO (jsonrpc/6) [api.host] START
> getAllVmStats() from=::ffff:192.168.0.4,49914 (api:46)
> 29461:2018-03-21 10:43:50,400+0200 INFO (jsonrpc/6) [api.host] FINISH
> getAllVmStats return={'status': {'message': 'Done', 'code': 0},
> 'statsList': (suppressed)} from=::ffff:192.168.0.4,49914 (api:52)
> 29462:2018-03-21 10:43:50,400+0200 INFO (jsonrpc/6)
> [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
> seconds (__init__:573)
> 29463:2018-03-21 10:43:51,694+0200 INFO (jsonrpc/7) [api.host] START
> getAllVmStats() from=::1,36114 (api:46)
> 29464:2018-03-21 10:43:51,694+0200 INFO (jsonrpc/7) [api.host] FINISH
> getAllVmStats return={'status': {'message': 'Done', 'code': 0},
> 'statsList': (suppressed)} from=::1,36114 (api:52)
> 29465:2018-03-21 10:43:51,695+0200 INFO (jsonrpc/7)
> [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
> seconds (__init__:573)
> 29466:2018-03-21 10:43:52,516+0200 INFO (jsonrpc/2) [api.host] START
> getStats() from=::ffff:192.168.0.4,49914 (api:46)
> 29467:2018-03-21 10:43:52,517+0200 INFO (jsonrpc/2) [vdsm.api] START
> repoStats(domains=()) from=::ffff:192.168.0.4,49914,
> task_id=0a88f0dc-eccf-4d24-9ac2-22e079a1480a (api:46)
> 29468:2018-03-21 10:43:52,517+0200 INFO (jsonrpc/2) [vdsm.api] FINISH
> repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
> 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000487937',
> 'lastCheck': '3.0', 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '0.000455956', 'lastCheck': '2.9', 'valid': True}}
> from=::ffff:192.168.0.4,49914, task_id=0a88f0dc-eccf-4d24-9ac2-22e079a1480a
> (api:52)
> 29469:2018-03-21 10:43:52,518+0200 INFO (jsonrpc/2) [vdsm.api] START
> multipath_health() from=::ffff:192.168.0.4,49914,
> task_id=a7a44ff7-a137-41b6-9ad5-2c0b32ab79ec (api:46)
> 29470:2018-03-21 10:43:52,518+0200 INFO (jsonrpc/2) [vdsm.api] FINISH
> multipath_health return={} from=::ffff:192.168.0.4,49914,
> task_id=a7a44ff7-a137-41b6-9ad5-2c0b32ab79ec (api:52)
> 29471:2018-03-21 10:43:52,524+0200 INFO (jsonrpc/2) [api.host] FINISH
> getStats return={'status': {'message': 'Done', 'code': 0}, 'info':
> {'cpuStatistics': {'11': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys':
> '0.13', 'cpuIdle': '99.80'}, '10': {'cpuUser': '0.00', 'nodeIndex': 0,
> 'cpuSys': '0.13', 'cpuIdle': '99.87'}, '13': {'cpuUser': '0.00',
> 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '12': {'cpuUser':
> '0.00', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.87'}, '15':
> {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.86'},
> '14': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle':
> '99.87'}, '1': {'cpuUser': '1.13', 'nodeIndex': 1, 'cpuSys': '1.13',
> 'cpuIdle': '97.74'}, '0': {'cpuUser': '0.33', 'nodeIndex': 0, 'cpuSys':
> '0.20', 'cpuIdle': '99.47'}, '3': {'cpuUser': '0.07', 'nodeIndex': 1,
> 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '2': {'cpuUser': '0.00',
> 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.87'}, '5': {'cpuUser':
> '0.00', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.87'}, '4':
> {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.60'},
> '7': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.33', 'cpuIdle':
> '99.67'}, '6': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.13',
> 'cpuIdle': '99.87'}, '9': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys':
> '0.13', 'cpuIdle': '99.80'}, '8': {'cpuUser': '0.07', 'nodeIndex': 0,
> 'cpuSys': '0.13', 'cpuIdle': '99.80'}}, 'numaNodeMemFree': {'1':
> {'memPercent': 7, 'memFree': '15143'}, '0': {'memPercent': 5, 'memFree':
> '15349'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes':
> True, 'vmCount': 0, 'memUsed': '3', 'storageDomains':
> {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0, 'actual': True,
> 'version': 4, 'acquired': True, 'delay': '0.000487937', 'lastCheck': '3.0',
> 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f': {'code': 0,
> 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000455956',
> 'lastCheck': '2.9', 'valid': True}}, 'incomingVmMigrations': 0, 'network':
> {'enp3s0f0': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621825.13588, 'name': 'enp3s0f0', 'tx': '1091312739', 'txDropped': '0',
> 'rx': '11914899051', 'rxErrors': '0', 'speed': '100', 'rxDropped': '1443'},
> 'ovirtmgmt': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621825.13588, 'name': 'ovirtmgmt', 'tx': '1048663107', 'txDropped':
> '0', 'rx': '11615236520', 'rxErrors': '0', 'speed': '1000', 'rxDropped':
> '0'}, 'lo': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621825.13588, 'name': 'lo', 'tx': '58090295494', 'txDropped': '0',
> 'rx': '58090295494', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'},
> 'enp3s0f1': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621825.13588, 'name': 'enp3s0f1', 'tx': '0', 'txDropped': '0', 'rx':
> '66839', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'},
> ';vdsmdummy;': {'txErrors': '0', 'state': 'down', 'sampleTime':
> 1521621825.13588, 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'rx':
> '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'enp4s0f0':
> {'txErrors': '0', 'state': 'down', 'sampleTime': 1521621825.13588, 'name':
> 'enp4s0f0', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0',
> 'speed': '1000', 'rxDropped': '0'}, 'enp4s0f1': {'txErrors': '0', 'state':
> 'down', 'sampleTime': 1521621825.13588, 'name': 'enp4s0f1', 'tx': '0',
> 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped':
> '0'}}, 'txDropped': '0', 'anonHugePages': '204', 'ksmPages': 100,
> 'elapsedTime': '76391.35', 'cpuLoad': '0.15', 'cpuSys': '0.20',
> 'diskStats': {'/var/log': {'free': '7344'}, '/var/run/vdsm/': {'free':
> '16060'}, '/tmp': {'free': '906'}}, 'cpuUserVdsmd': '1.13',
> 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState': False,
> 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 31080, 'bootTime':
> '1521545382', 'haStats': {'active': False, 'configured': False, 'score': 0,
> 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus':
> 'active', 'multipathHealth': {}, 'rxDropped': '1443',
> 'outgoingVmMigrations': 0, 'swapTotal': 12287, 'swapFree': 12287,
> 'hugepages': defaultdict(<type 'dict'>, {2048: {'resv_hugepages': 0,
> 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0,
> 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0},
> 1048576: {'resv_hugepages': 0, 'free_hugepages': 0,
> 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages':
> 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}), 'dateTime':
> '2018-03-21T08:43:52 GMT', 'cpuUser': '0.14', 'memFree': 31336, 'cpuIdle':
> '99.66', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd': '0.73'}}
> from=::ffff:192.168.0.4,49914 (api:52)
> 29472:2018-03-21 10:43:52,526+0200 INFO (jsonrpc/2)
> [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds
> (__init__:573)
> 29473:2018-03-21 10:43:58,686+0200 INFO (jsonrpc/1) [vdsm.api] START
> getSpmStatus(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
> options=None) from=::ffff:192.168.0.4,49914, task_id=d6d29cf6-c083-4639-91a2-f062eae4e629
> (api:46)
> 29474:2018-03-21 10:43:58,690+0200 INFO (jsonrpc/1) [vdsm.api] FINISH
> getSpmStatus return={'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver':
> 6L}} from=::ffff:192.168.0.4,49914, task_id=d6d29cf6-c083-4639-91a2-f062eae4e629
> (api:52)
> 29475:2018-03-21 10:43:58,690+0200 INFO (jsonrpc/1)
> [jsonrpc.JsonRpcServer] RPC call StoragePool.getSpmStatus succeeded in 0.01
> seconds (__init__:573)
> 29476:2018-03-21 10:43:58,697+0200 INFO (jsonrpc/5) [vdsm.api] START
> getStoragePoolInfo(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
> options=None) from=::ffff:192.168.0.4,49920, task_id=9dbd91df-dd9c-469c-8308-bfbfc228937c
> (api:46)
> 29477:2018-03-21 10:43:58,703+0200 INFO (jsonrpc/5) [vdsm.api] FINISH
> getStoragePoolInfo return={'info': {'name': 'No Description', 'isoprefix':
> u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
> nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
> 11111111-1111-1111-1111-111111111111', 'pool_status': 'connected',
> 'lver': 6L, 'domains': u'ef184b28-1dbc-45ed-b0b3-
> 85e780cce5d8:Active,8a2c304b-c8ae-438b-af54-fc8797ea149f:Active',
> 'master_uuid': u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8', 'version': '4',
> 'spm_id': 1, 'type': 'NFS', 'master_ver': 3}, 'dominfo':
> {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'status': u'Active',
> 'diskfree': '1868692455424', 'isoprefix': '', 'alerts': [], 'disktotal':
> '1968811540480', 'version': 4}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'status': u'Active', 'diskfree': '1868692455424', 'isoprefix':
> u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
> nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
> 11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal':
> '1968811540480', 'version': 0}}} from=::ffff:192.168.0.4,49920,
> task_id=9dbd91df-dd9c-469c-8308-bfbfc228937c (api:52)
> 29478:2018-03-21 10:43:58,704+0200 INFO (jsonrpc/5)
> [jsonrpc.JsonRpcServer] RPC call StoragePool.getInfo succeeded in 0.01
> seconds (__init__:573)
>
>
>
> *grep -n 10:46 vdsm.log | tail -1000*
> 1017:2018-03-21 06:10:46,882+0200 INFO (jsonrpc/2) [api.host] START
> getStats() from=::ffff:192.168.0.4,49914 (api:46)
> 1018:2018-03-21 06:10:46,883+0200 INFO (jsonrpc/2) [vdsm.api] START
> repoStats(domains=()) from=::ffff:192.168.0.4,49914,
> task_id=6e045b00-88b2-469b-80d6-af38303f8c32 (api:46)
> 1019:2018-03-21 06:10:46,883+0200 INFO (jsonrpc/2) [vdsm.api] FINISH
> repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
> 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000332785',
> 'lastCheck': '7.3', 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '0.000203914', 'lastCheck': '5.6', 'valid': True}}
> from=::ffff:192.168.0.4,49914, task_id=6e045b00-88b2-469b-80d6-af38303f8c32
> (api:52)
> 1020:2018-03-21 06:10:46,884+0200 INFO (jsonrpc/2) [vdsm.api] START
> multipath_health() from=::ffff:192.168.0.4,49914,
> task_id=475a5b40-4a43-42ee-9524-2bdd220aed83 (api:46)
> 1021:2018-03-21 06:10:46,884+0200 INFO (jsonrpc/2) [vdsm.api] FINISH
> multipath_health return={} from=::ffff:192.168.0.4,49914,
> task_id=475a5b40-4a43-42ee-9524-2bdd220aed83 (api:52)
> 1022:2018-03-21 06:10:46,889+0200 INFO (jsonrpc/2) [api.host] FINISH
> getStats return={'status': {'message': 'Done', 'code': 0}, 'info':
> {'cpuStatistics': {'11': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys':
> '0.20', 'cpuIdle': '99.73'}, '10': {'cpuUser': '0.13', 'nodeIndex': 0,
> 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '13': {'cpuUser': '0.07',
> 'nodeIndex': 1, 'cpuSys': '0.27', 'cpuIdle': '99.66'}, '12': {'cpuUser':
> '0.33', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.47'}, '15':
> {'cpuUser': '0.33', 'nodeIndex': 1, 'cpuSys': '0.27', 'cpuIdle': '99.40'},
> '14': {'cpuUser': '0.47', 'nodeIndex': 0, 'cpuSys': '0.40', 'cpuIdle':
> '99.13'}, '1': {'cpuUser': '3.00', 'nodeIndex': 1, 'cpuSys': '1.40',
> 'cpuIdle': '95.60'}, '0': {'cpuUser': '0.73', 'nodeIndex': 0, 'cpuSys':
> '0.20', 'cpuIdle': '99.07'}, '3': {'cpuUser': '0.07', 'nodeIndex': 1,
> 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '2': {'cpuUser': '0.07',
> 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '99.66'}, '5': {'cpuUser':
> '0.07', 'nodeIndex': 1, 'cpuSys': '0.20', 'cpuIdle': '99.73'}, '4':
> {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '99.54'},
> '7': {'cpuUser': '0.80', 'nodeIndex': 1, 'cpuSys': '0.80', 'cpuIdle':
> '98.40'}, '6': {'cpuUser': '5.46', 'nodeIndex': 0, 'cpuSys': '0.27',
> 'cpuIdle': '94.27'}, '9': {'cpuUser': '0.20', 'nodeIndex': 1, 'cpuSys':
> '0.27', 'cpuIdle': '99.53'}, '8': {'cpuUser': '1.27', 'nodeIndex': 0,
> 'cpuSys': '0.47', 'cpuIdle': '98.26'}}, 'numaNodeMemFree': {'1':
> {'memPercent': 7, 'memFree': '15128'}, '0': {'memPercent': 5, 'memFree':
> '15366'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes':
> True, 'vmCount': 0, 'memUsed': '3', 'storageDomains':
> {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0, 'actual': True,
> 'version': 4, 'acquired': True, 'delay': '0.000332785', 'lastCheck': '7.3',
> 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f': {'code': 0,
> 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000203914',
> 'lastCheck': '5.6', 'valid': True}}, 'incomingVmMigrations': 0, 'network':
> {'enp3s0f0': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521605437.203776, 'name': 'enp3s0f0', 'tx': '1082264809', 'txDropped':
> '0', 'rx': '11907188594', 'rxErrors': '0', 'speed': '100', 'rxDropped':
> '289'}, 'ovirtmgmt': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521605437.203776, 'name': 'ovirtmgmt', 'tx': '1039869554', 'txDropped':
> '0', 'rx': '11608697383', 'rxErrors': '0', 'speed': '1000', 'rxDropped':
> '0'}, 'lo': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521605437.203776, 'name': 'lo', 'tx': '45587320171', 'txDropped': '0',
> 'rx': '45587320171', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'},
> 'enp3s0f1': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521605437.203776, 'name': 'enp3s0f1', 'tx': '0', 'txDropped': '0', 'rx':
> '53620', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'},
> ';vdsmdummy;': {'txErrors': '0', 'state': 'down', 'sampleTime':
> 1521605437.203776, 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0',
> 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'enp4s0f0':
> {'txErrors': '0', 'state': 'down', 'sampleTime': 1521605437.203776, 'name':
> 'enp4s0f0', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0',
> 'speed': '1000', 'rxDropped': '0'}, 'enp4s0f1': {'txErrors': '0', 'state':
> 'down', 'sampleTime': 1521605437.203776, 'name': 'enp4s0f1', 'tx': '0',
> 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped':
> '0'}}, 'txDropped': '0', 'anonHugePages': '204', 'ksmPages': 100,
> 'elapsedTime': '60005.72', 'cpuLoad': '0.06', 'cpuSys': '0.38',
> 'diskStats': {'/var/log': {'free': '7357'}, '/var/run/vdsm/': {'free':
> '16060'}, '/tmp': {'free': '906'}}, 'cpuUserVdsmd': '1.07',
> 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState': False,
> 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 31084, 'bootTime':
> '1521545382', 'haStats': {'active': False, 'configured': False, 'score': 0,
> 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus':
> 'active', 'multipathHealth': {}, 'rxDropped': '289',
> 'outgoingVmMigrations': 0, 'swapTotal': 12287, 'swapFree': 12287,
> 'hugepages': defaultdict(<type 'dict'>, {2048: {'resv_hugepages': 0,
> 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0,
> 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0},
> 1048576: {'resv_hugepages': 0, 'free_hugepages': 0,
> 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages':
> 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}), 'dateTime':
> '2018-03-21T04:10:46 GMT', 'cpuUser': '0.83', 'memFree': 31340, 'cpuIdle':
> '98.79', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd': '0.80'}}
> from=::ffff:192.168.0.4,49914 (api:52)
> 1023:2018-03-21 06:10:46,891+0200 INFO (jsonrpc/2)
> [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds
> (__init__:573)
> 13522:2018-03-21 08:10:46,827+0200 INFO (jsonrpc/7) [api.host] START
> getAllVmStats() from=::ffff:192.168.0.4,49914 (api:46)
> 13523:2018-03-21 08:10:46,828+0200 INFO (jsonrpc/7) [api.host] FINISH
> getAllVmStats return={'status': {'message': 'Done', 'code': 0},
> 'statsList': (suppressed)} from=::ffff:192.168.0.4,49914 (api:52)
> 13524:2018-03-21 08:10:46,828+0200 INFO (jsonrpc/7)
> [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
> seconds (__init__:573)
> 29734:2018-03-21 10:46:00,338+0200 INFO (periodic/2) [vdsm.api] START
> repoStats(domains=()) from=internal, task_id=af099ce6-0f44-461b-9e8e-99de63b3884f
> (api:46)
> 29735:2018-03-21 10:46:00,339+0200 INFO (periodic/2) [vdsm.api] FINISH
> repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
> 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000470245',
> 'lastCheck': '0.8', 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '0.000476935', 'lastCheck': '0.7', 'valid': True}} from=internal,
> task_id=af099ce6-0f44-461b-9e8e-99de63b3884f (api:52)
> 29736:2018-03-21 10:46:00,339+0200 INFO (periodic/2) [vdsm.api] START
> multipath_health() from=internal, task_id=4a5709d9-e322-44eb-a8a4-bbe3f5e5e1bd
> (api:46)
> 29737:2018-03-21 10:46:00,340+0200 INFO (periodic/2) [vdsm.api] FINISH
> multipath_health return={} from=internal, task_id=4a5709d9-e322-44eb-a8a4-bbe3f5e5e1bd
> (api:52)
> 29738:2018-03-21 10:46:04,637+0200 INFO (jsonrpc/7) [vdsm.api] START
> getSpmStatus(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
> options=None) from=::ffff:192.168.0.4,49914, task_id=5b9300f7-f8c4-4f63-a49c-45ca8119d3b9
> (api:46)
> 29739:2018-03-21 10:46:04,642+0200 INFO (jsonrpc/7) [vdsm.api] FINISH
> getSpmStatus return={'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver':
> 6L}} from=::ffff:192.168.0.4,49914, task_id=5b9300f7-f8c4-4f63-a49c-45ca8119d3b9
> (api:52)
> 29740:2018-03-21 10:46:04,642+0200 INFO (jsonrpc/7)
> [jsonrpc.JsonRpcServer] RPC call StoragePool.getSpmStatus succeeded in 0.01
> seconds (__init__:573)
> 29741:2018-03-21 10:46:04,648+0200 INFO (jsonrpc/1) [vdsm.api] START
> getStoragePoolInfo(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
> options=None) from=::ffff:192.168.0.4,49920, task_id=f4db5bfa-e131-4f9a-baaa-e7555c996ab0
> (api:46)
> 29742:2018-03-21 10:46:04,655+0200 INFO (jsonrpc/1) [vdsm.api] FINISH
> getStoragePoolInfo return={'info': {'name': 'No Description', 'isoprefix':
> u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
> nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
> 11111111-1111-1111-1111-111111111111', 'pool_status': 'connected',
> 'lver': 6L, 'domains': u'ef184b28-1dbc-45ed-b0b3-
> 85e780cce5d8:Active,8a2c304b-c8ae-438b-af54-fc8797ea149f:Active',
> 'master_uuid': u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8', 'version': '4',
> 'spm_id': 1, 'type': 'NFS', 'master_ver': 3}, 'dominfo':
> {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'status': u'Active',
> 'diskfree': '1868692455424', 'isoprefix': '', 'alerts': [], 'disktotal':
> '1968811540480', 'version': 4}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'status': u'Active', 'diskfree': '1868692455424', 'isoprefix':
> u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
> nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
> 11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal':
> '1968811540480', 'version': 0}}} from=::ffff:192.168.0.4,49920,
> task_id=f4db5bfa-e131-4f9a-baaa-e7555c996ab0 (api:52)
> 29743:2018-03-21 10:46:04,656+0200 INFO (jsonrpc/1)
> [jsonrpc.JsonRpcServer] RPC call StoragePool.getInfo succeeded in 0.00
> seconds (__init__:573)
> 29744:2018-03-21 10:46:05,451+0200 INFO (jsonrpc/5) [api.host] START
> getAllVmStats() from=::ffff:192.168.0.4,49914 (api:46)
> 29745:2018-03-21 10:46:05,452+0200 INFO (jsonrpc/5) [api.host] FINISH
> getAllVmStats return={'status': {'message': 'Done', 'code': 0},
> 'statsList': (suppressed)} from=::ffff:192.168.0.4,49914 (api:52)
> 29746:2018-03-21 10:46:05,452+0200 INFO (jsonrpc/5)
> [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
> seconds (__init__:573)
> 29747:2018-03-21 10:46:06,903+0200 INFO (jsonrpc/3) [api.host] START
> getAllVmStats() from=::1,36114 (api:46)
> 29748:2018-03-21 10:46:06,904+0200 INFO (jsonrpc/3) [api.host] FINISH
> getAllVmStats return={'status': {'message': 'Done', 'code': 0},
> 'statsList': (suppressed)} from=::1,36114 (api:52)
> 29749:2018-03-21 10:46:06,904+0200 INFO (jsonrpc/3)
> [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
> seconds (__init__:573)
> 29750:2018-03-21 10:46:08,791+0200 WARN (vdsm.Scheduler) [Executor]
> Worker blocked: <Worker name=jsonrpc/6 running <Task <JsonRpcTask
> {'params': {u'connectionParams': [{u'id': u'461f65a9-3a81-4f3f-a46d-c5ed12520524',
> u'connection': u'node10.starlett.lv:/vmdata/nfs/exports', u'iqn': u'',
> u'user': u'', u'tpgt': u'1', u'protocol_version': u'auto', u'password':
> '********', u'port': u''}], u'storagepoolID': u'00000000-0000-0000-0000-000000000000',
> u'domainType': 1}, 'jsonrpc': '2.0', 'method': u'StoragePool.connectStorageServer',
> 'id': u'71143eef-aac6-4996-9783-3e0e3da180c3'} at 0x3604fd0> timeout=60,
> duration=120 at 0x3604e10> task#=3762 at 0x3541650>, traceback:
> 29801:2018-03-21 10:46:12,852+0200 INFO (jsonrpc/0) [api.host] START
> getStats() from=::ffff:192.168.0.4,49914 (api:46)
> 29802:2018-03-21 10:46:12,853+0200 INFO (jsonrpc/0) [vdsm.api] START
> repoStats(domains=()) from=::ffff:192.168.0.4,49914,
> task_id=391fdab9-d3c2-4075-9c34-177a07a21ec3 (api:46)
> 29803:2018-03-21 10:46:12,854+0200 INFO (jsonrpc/0) [vdsm.api] FINISH
> repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
> 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000492277',
> 'lastCheck': '3.3', 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '0.000479153', 'lastCheck': '3.2', 'valid': True}}
> from=::ffff:192.168.0.4,49914, task_id=391fdab9-d3c2-4075-9c34-177a07a21ec3
> (api:52)
> 29804:2018-03-21 10:46:12,854+0200 INFO (jsonrpc/0) [vdsm.api] START
> multipath_health() from=::ffff:192.168.0.4,49914,
> task_id=7e53adbc-3974-4f51-bfc8-8fbd3ca6b749 (api:46)
> 29805:2018-03-21 10:46:12,854+0200 INFO (jsonrpc/0) [vdsm.api] FINISH
> multipath_health return={} from=::ffff:192.168.0.4,49914,
> task_id=7e53adbc-3974-4f51-bfc8-8fbd3ca6b749 (api:52)
> 29806:2018-03-21 10:46:12,861+0200 INFO (jsonrpc/0) [api.host] FINISH
> getStats return={'status': {'message': 'Done', 'code': 0}, 'info':
> {'cpuStatistics': {'11': {'cpuUser': '0.47', 'nodeIndex': 1, 'cpuSys':
> '0.40', 'cpuIdle': '99.13'}, '10': {'cpuUser': '0.13', 'nodeIndex': 0,
> 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '13': {'cpuUser': '0.07',
> 'nodeIndex': 1, 'cpuSys': '0.20', 'cpuIdle': '99.73'}, '12': {'cpuUser':
> '0.20', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '99.53'}, '15':
> {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.27', 'cpuIdle': '99.60'},
> '14': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle':
> '99.54'}, '1': {'cpuUser': '1.00', 'nodeIndex': 1, 'cpuSys': '1.33',
> 'cpuIdle': '97.67'}, '0': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys':
> '0.27', 'cpuIdle': '99.60'}, '3': {'cpuUser': '0.07', 'nodeIndex': 1,
> 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '2': {'cpuUser': '0.20',
> 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle': '99.47'}, '5': {'cpuUser':
> '0.07', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.80'}, '4':
> {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.73'},
> '7': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.40', 'cpuIdle':
> '99.53'}, '6': {'cpuUser': '0.47', 'nodeIndex': 0, 'cpuSys': '0.40',
> 'cpuIdle': '99.13'}, '9': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys':
> '0.20', 'cpuIdle': '99.73'}, '8': {'cpuUser': '0.13', 'nodeIndex': 0,
> 'cpuSys': '0.33', 'cpuIdle': '99.54'}}, 'numaNodeMemFree': {'1':
> {'memPercent': 7, 'memFree': '15142'}, '0': {'memPercent': 5, 'memFree':
> '15348'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes':
> True, 'vmCount': 0, 'memUsed': '3', 'storageDomains':
> {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0, 'actual': True,
> 'version': 4, 'acquired': True, 'delay': '0.000492277', 'lastCheck': '3.3',
> 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f': {'code': 0,
> 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000479153',
> 'lastCheck': '3.2', 'valid': True}}, 'incomingVmMigrations': 0, 'network':
> {'enp3s0f0': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621960.326184, 'name': 'enp3s0f0', 'tx': '1091387790', 'txDropped':
> '0', 'rx': '11914959714', 'rxErrors': '0', 'speed': '100', 'rxDropped':
> '1451'}, 'ovirtmgmt': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621960.326184, 'name': 'ovirtmgmt', 'tx': '1048735814', 'txDropped':
> '0', 'rx': '11615289173', 'rxErrors': '0', 'speed': '1000', 'rxDropped':
> '0'}, 'lo': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621960.326184, 'name': 'lo', 'tx': '58192068637', 'txDropped': '0',
> 'rx': '58192068637', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'},
> 'enp3s0f1': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621960.326184, 'name': 'enp3s0f1', 'tx': '0', 'txDropped': '0', 'rx':
> '66839', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'},
> ';vdsmdummy;': {'txErrors': '0', 'state': 'down', 'sampleTime':
> 1521621960.326184, 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0',
> 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'enp4s0f0':
> {'txErrors': '0', 'state': 'down', 'sampleTime': 1521621960.326184, 'name':
> 'enp4s0f0', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0',
> 'speed': '1000', 'rxDropped': '0'}, 'enp4s0f1': {'txErrors': '0', 'state':
> 'down', 'sampleTime': 1521621960.326184, 'name': 'enp4s0f1', 'tx': '0',
> 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped':
> '0'}}, 'txDropped': '0', 'anonHugePages': '204', 'ksmPages': 100,
> 'elapsedTime': '76531.69', 'cpuLoad': '0.17', 'cpuSys': '0.32',
> 'diskStats': {'/var/log': {'free': '7344'}, '/var/run/vdsm/': {'free':
> '16060'}, '/tmp': {'free': '906'}}, 'cpuUserVdsmd': '1.00',
> 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState': False,
> 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 31081, 'bootTime':
> '1521545382', 'haStats': {'active': False, 'configured': False, 'score': 0,
> 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus':
> 'active', 'multipathHealth': {}, 'rxDropped': '1451',
> 'outgoingVmMigrations': 0, 'swapTotal': 12287, 'swapFree': 12287,
> 'hugepages': defaultdict(<type 'dict'>, {2048: {'resv_hugepages': 0,
> 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0,
> 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0},
> 1048576: {'resv_hugepages': 0, 'free_hugepages': 0,
> 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages':
> 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}), 'dateTime':
> '2018-03-21T08:46:12 GMT', 'cpuUser': '0.21', 'memFree': 31337, 'cpuIdle':
> '99.46', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd': '0.80'}}
> from=::ffff:192.168.0.4,49914 (api:52)
> 29807:2018-03-21 10:46:12,863+0200 INFO (jsonrpc/0)
> [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds
> (__init__:573)
> 29808:2018-03-21 10:46:13,829+0200 ERROR (jsonrpc/6) [storage.HSM] Could
> not connect to storageServer (hsm:2407)
> 29828:2018-03-21 10:46:13,829+0200 INFO (jsonrpc/6) [vdsm.api] FINISH
> connectStorageServer return={'statuslist': [{'status': 477, 'id':
> u'461f65a9-3a81-4f3f-a46d-c5ed12520524'}]} from=::ffff:192.168.0.4,49914,
> flow_id=1435fc81, task_id=0a828d2c-d9f4-4f83-a9e9-7393159d5323 (api:52)
> 29829:2018-03-21 10:46:13,830+0200 INFO (jsonrpc/6)
> [jsonrpc.JsonRpcServer] RPC call StoragePool.connectStorageServer
> succeeded in 125.04 seconds (__init__:573)
> 29830:2018-03-21 10:46:14,767+0200 INFO (jsonrpc/4) [vdsm.api] START
> getSpmStatus(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
> options=None) from=::ffff:192.168.0.4,49914, task_id=49cd2ae6-7943-47c9-ad86-9e0b7e58bca3
> (api:46)
> 29831:2018-03-21 10:46:14,772+0200 INFO (jsonrpc/4) [vdsm.api] FINISH
> getSpmStatus return={'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver':
> 6L}} from=::ffff:192.168.0.4,49914, task_id=49cd2ae6-7943-47c9-ad86-9e0b7e58bca3
> (api:52)
> 29832:2018-03-21 10:46:14,773+0200 INFO (jsonrpc/4)
> [jsonrpc.JsonRpcServer] RPC call StoragePool.getSpmStatus succeeded in 0.01
> seconds (__init__:573)
> 29833:2018-03-21 10:46:14,815+0200 INFO (jsonrpc/2) [vdsm.api] START
> getStoragePoolInfo(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
> options=None) from=::ffff:192.168.0.4,49920, task_id=19909616-efb4-490d-982c-66eda9ca4381
> (api:46)
> 29834:2018-03-21 10:46:14,822+0200 INFO (jsonrpc/2) [vdsm.api] FINISH
> getStoragePoolInfo return={'info': {'name': 'No Description', 'isoprefix':
> u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
> nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
> 11111111-1111-1111-1111-111111111111', 'pool_status': 'connected',
> 'lver': 6L, 'domains': u'ef184b28-1dbc-45ed-b0b3-
> 85e780cce5d8:Active,8a2c304b-c8ae-438b-af54-fc8797ea149f:Active',
> 'master_uuid': u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8', 'version': '4',
> 'spm_id': 1, 'type': 'NFS', 'master_ver': 3}, 'dominfo':
> {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'status': u'Active',
> 'diskfree': '1868692455424', 'isoprefix': '', 'alerts': [], 'disktotal':
> '1968811540480', 'version': 4}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'status': u'Active', 'diskfree': '1868692455424', 'isoprefix':
> u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
> nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
> 11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal':
> '1968811540480', 'version': 0}}} from=::ffff:192.168.0.4,49920,
> task_id=19909616-efb4-490d-982c-66eda9ca4381 (api:52)
> 29835:2018-03-21 10:46:14,823+0200 INFO (jsonrpc/2)
> [jsonrpc.JsonRpcServer] RPC call StoragePool.getInfo succeeded in 0.01
> seconds (__init__:573)
> 29836:2018-03-21 10:46:15,360+0200 INFO (periodic/2) [vdsm.api] START
> repoStats(domains=()) from=internal, task_id=da74ea9f-164b-450d-b836-6818caa3fdc5
> (api:46)
> 29837:2018-03-21 10:46:15,360+0200 INFO (periodic/2) [vdsm.api] FINISH
> repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
> 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000492277',
> 'lastCheck': '5.8', 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '0.000479153', 'lastCheck': '5.7', 'valid': True}} from=internal,
> task_id=da74ea9f-164b-450d-b836-6818caa3fdc5 (api:52)
> 29838:2018-03-21 10:46:15,361+0200 INFO (periodic/2) [vdsm.api] START
> multipath_health() from=internal, task_id=06245e2b-e8ca-41c4-90bf-4294d0c699b8
> (api:46)
> 29839:2018-03-21 10:46:15,361+0200 INFO (periodic/2) [vdsm.api] FINISH
> multipath_health return={} from=internal, task_id=06245e2b-e8ca-41c4-90bf-4294d0c699b8
> (api:52)
> 29840:2018-03-21 10:46:20,456+0200 INFO (jsonrpc/7) [api.host] START
> getAllVmStats() from=::ffff:192.168.0.4,49914 (api:46)
> 29841:2018-03-21 10:46:20,457+0200 INFO (jsonrpc/7) [api.host] FINISH
> getAllVmStats return={'status': {'message': 'Done', 'code': 0},
> 'statsList': (suppressed)} from=::ffff:192.168.0.4,49914 (api:52)
> 29842:2018-03-21 10:46:20,457+0200 INFO (jsonrpc/7)
> [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.01
> seconds (__init__:573)
> 29843:2018-03-21 10:46:21,925+0200 INFO (jsonrpc/1) [api.host] START
> getAllVmStats() from=::1,36114 (api:46)
> 29844:2018-03-21 10:46:21,926+0200 INFO (jsonrpc/1) [api.host] FINISH
> getAllVmStats return={'status': {'message': 'Done', 'code': 0},
> 'statsList': (suppressed)} from=::1,36114 (api:52)
> 29845:2018-03-21 10:46:21,926+0200 INFO (jsonrpc/1)
> [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
> seconds (__init__:573)
> 29846:2018-03-21 10:46:24,930+0200 INFO (jsonrpc/5) [vdsm.api] START
> getSpmStatus(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
> options=None) from=::ffff:192.168.0.4,49914, task_id=d964535d-e5eb-42bc-b324-432a05c364da
> (api:46)
> 29847:2018-03-21 10:46:24,935+0200 INFO (jsonrpc/5) [vdsm.api] FINISH
> getSpmStatus return={'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver':
> 6L}} from=::ffff:192.168.0.4,49914, task_id=d964535d-e5eb-42bc-b324-432a05c364da
> (api:52)
> 29848:2018-03-21 10:46:24,935+0200 INFO (jsonrpc/5)
> [jsonrpc.JsonRpcServer] RPC call StoragePool.getSpmStatus succeeded in 0.00
> seconds (__init__:573)
> 29849:2018-03-21 10:46:24,987+0200 INFO (jsonrpc/3) [vdsm.api] START
> getStoragePoolInfo(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
> options=None) from=::ffff:192.168.0.4,49920, task_id=167dcd23-04b5-4de3-9c81-7d25ba56407a
> (api:46)
> 29850:2018-03-21 10:46:25,022+0200 INFO (jsonrpc/3) [vdsm.api] FINISH
> getStoragePoolInfo return={'info': {'name': 'No Description', 'isoprefix':
> u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
> nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
> 11111111-1111-1111-1111-111111111111', 'pool_status': 'connected',
> 'lver': 6L, 'domains': u'ef184b28-1dbc-45ed-b0b3-
> 85e780cce5d8:Active,8a2c304b-c8ae-438b-af54-fc8797ea149f:Active',
> 'master_uuid': u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8', 'version': '4',
> 'spm_id': 1, 'type': 'NFS', 'master_ver': 3}, 'dominfo':
> {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'status': u'Active',
> 'diskfree': '1868692455424', 'isoprefix': '', 'alerts': [], 'disktotal':
> '1968811540480', 'version': 4}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'status': u'Active', 'diskfree': '1868692455424', 'isoprefix':
> u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
> nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
> 11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal':
> '1968811540480', 'version': 0}}} from=::ffff:192.168.0.4,49920,
> task_id=167dcd23-04b5-4de3-9c81-7d25ba56407a (api:52)
> 29851:2018-03-21 10:46:25,023+0200 INFO (jsonrpc/3)
> [jsonrpc.JsonRpcServer] RPC call StoragePool.getInfo succeeded in 0.03
> seconds (__init__:573)
> 29852:2018-03-21 10:46:28,010+0200 INFO (jsonrpc/0) [api.host] START
> getStats() from=::ffff:192.168.0.4,49914 (api:46)
> 29853:2018-03-21 10:46:28,011+0200 INFO (jsonrpc/0) [vdsm.api] START
> repoStats(domains=()) from=::ffff:192.168.0.4,49914,
> task_id=011616ce-53c9-4cfc-8b29-247b1be03409 (api:46)
> 29854:2018-03-21 10:46:28,011+0200 INFO (jsonrpc/0) [vdsm.api] FINISH
> repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
> 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000218955',
> 'lastCheck': '8.4', 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '0.000351741', 'lastCheck': '8.4', 'valid': True}}
> from=::ffff:192.168.0.4,49914, task_id=011616ce-53c9-4cfc-8b29-247b1be03409
> (api:52)
> 29855:2018-03-21 10:46:28,012+0200 INFO (jsonrpc/0) [vdsm.api] START
> multipath_health() from=::ffff:192.168.0.4,49914,
> task_id=85b42a8d-1bf1-495d-ad32-a1f9710a6468 (api:46)
> 29856:2018-03-21 10:46:28,012+0200 INFO (jsonrpc/0) [vdsm.api] FINISH
> multipath_health return={} from=::ffff:192.168.0.4,49914,
> task_id=85b42a8d-1bf1-495d-ad32-a1f9710a6468 (api:52)
> 29857:2018-03-21 10:46:28,018+0200 INFO (jsonrpc/0) [api.host] FINISH
> getStats return={'status': {'message': 'Done', 'code': 0}, 'info':
> {'cpuStatistics': {'11': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys':
> '0.20', 'cpuIdle': '99.67'}, '10': {'cpuUser': '1.73', 'nodeIndex': 0,
> 'cpuSys': '0.53', 'cpuIdle': '97.74'}, '13': {'cpuUser': '0.20',
> 'nodeIndex': 1, 'cpuSys': '0.27', 'cpuIdle': '99.53'}, '12': {'cpuUser':
> '0.07', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.73'}, '15':
> {'cpuUser': '0.67', 'nodeIndex': 1, 'cpuSys': '0.27', 'cpuIdle': '99.06'},
> '14': {'cpuUser': '0.20', 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle':
> '99.47'}, '1': {'cpuUser': '1.26', 'nodeIndex': 1, 'cpuSys': '1.33',
> 'cpuIdle': '97.41'}, '0': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys':
> '0.07', 'cpuIdle': '99.53'}, '3': {'cpuUser': '0.07', 'nodeIndex': 1,
> 'cpuSys': '0.13', 'cpuIdle': '99.80'}, '2': {'cpuUser': '0.00',
> 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '99.73'}, '5': {'cpuUser':
> '0.00', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.87'}, '4':
> {'cpuUser': '13.78', 'nodeIndex': 0, 'cpuSys': '0.80', 'cpuIdle': '85.42'},
> '7': {'cpuUser': '0.47', 'nodeIndex': 1, 'cpuSys': '0.67', 'cpuIdle':
> '98.86'}, '6': {'cpuUser': '1.07', 'nodeIndex': 0, 'cpuSys': '0.27',
> 'cpuIdle': '98.66'}, '9': {'cpuUser': '1.20', 'nodeIndex': 1, 'cpuSys':
> '0.27', 'cpuIdle': '98.53'}, '8': {'cpuUser': '0.80', 'nodeIndex': 0,
> 'cpuSys': '0.33', 'cpuIdle': '98.87'}}, 'numaNodeMemFree': {'1':
> {'memPercent': 7, 'memFree': '15136'}, '0': {'memPercent': 5, 'memFree':
> '15357'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes':
> True, 'vmCount': 0, 'memUsed': '3', 'storageDomains':
> {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0, 'actual': True,
> 'version': 4, 'acquired': True, 'delay': '0.000218955', 'lastCheck': '8.4',
> 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f': {'code': 0,
> 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000351741',
> 'lastCheck': '8.4', 'valid': True}}, 'incomingVmMigrations': 0, 'network':
> {'enp3s0f0': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621975.346633, 'name': 'enp3s0f0', 'tx': '1091397599', 'txDropped':
> '0', 'rx': '11914966591', 'rxErrors': '0', 'speed': '100', 'rxDropped':
> '1454'}, 'ovirtmgmt': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621975.346633, 'name': 'ovirtmgmt', 'tx': '1048745347', 'txDropped':
> '0', 'rx': '11615294897', 'rxErrors': '0', 'speed': '1000', 'rxDropped':
> '0'}, 'lo': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621975.346633, 'name': 'lo', 'tx': '58205583384', 'txDropped': '0',
> 'rx': '58205583384', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'},
> 'enp3s0f1': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621975.346633, 'name': 'enp3s0f1', 'tx': '0', 'txDropped': '0', 'rx':
> '66839', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'},
> ';vdsmdummy;': {'txErrors': '0', 'state': 'down', 'sampleTime':
> 1521621975.346633, 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0',
> 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'enp4s0f0':
> {'txErrors': '0', 'state': 'down', 'sampleTime': 1521621975.346633, 'name':
> 'enp4s0f0', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0',
> 'speed': '1000', 'rxDropped': '0'}, 'enp4s0f1': {'txErrors': '0', 'state':
> 'down', 'sampleTime': 1521621975.346633, 'name': 'enp4s0f1', 'tx': '0',
> 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped':
> '0'}}, 'txDropped': '0', 'anonHugePages': '204', 'ksmPages': 100,
> 'elapsedTime': '76546.85', 'cpuLoad': '0.18', 'cpuSys': '0.39',
> 'diskStats': {'/var/log': {'free': '7344'}, '/var/run/vdsm/': {'free':
> '16060'}, '/tmp': {'free': '906'}}, 'cpuUserVdsmd': '1.20',
> 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState': False,
> 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 31031, 'bootTime':
> '1521545382', 'haStats': {'active': False, 'configured': False, 'score': 0,
> 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus':
> 'active', 'multipathHealth': {}, 'rxDropped': '1454',
> 'outgoingVmMigrations': 0, 'swapTotal': 12287, 'swapFree': 12287,
> 'hugepages': defaultdict(<type 'dict'>, {2048: {'resv_hugepages': 0,
> 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0,
> 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0},
> 1048576: {'resv_hugepages': 0, 'free_hugepages': 0,
> 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages':
> 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}), 'dateTime':
> '2018-03-21T08:46:28 GMT', 'cpuUser': '1.38', 'memFree': 31287, 'cpuIdle':
> '98.23', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd': '0.87'}}
> from=::ffff:192.168.0.4,49914 (api:52)
> 29858:2018-03-21 10:46:28,020+0200 INFO (jsonrpc/0)
> [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds
> (__init__:573)
> 29859:2018-03-21 10:46:29,667+0200 INFO (itmap/0) [IOProcessClient]
> Starting client ioprocess-575 (__init__:308)
> 29860:2018-03-21 10:46:29,700+0200 INFO (itmap/1) [IOProcessClient]
> Starting client ioprocess-576 (__init__:308)
> 29861:2018-03-21 10:46:29,714+0200 INFO (ioprocess/1711202) [IOProcess]
> Starting ioprocess (__init__:437)
> 29862:2018-03-21 10:46:29,725+0200 INFO (ioprocess/1711208) [IOProcess]
> Starting ioprocess (__init__:437)
> 29863:2018-03-21 10:46:30,375+0200 INFO (periodic/0) [vdsm.api] START
> repoStats(domains=()) from=internal, task_id=bdacf1c0-3fcb-43b1-82e2-a041a26d0d7f
> (api:46)
> 29864:2018-03-21 10:46:30,376+0200 INFO (periodic/0) [vdsm.api] FINISH
> repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
> 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.00031111',
> 'lastCheck': '0.8', 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '0.00045594', 'lastCheck': '0.6', 'valid': True}} from=internal,
> task_id=bdacf1c0-3fcb-43b1-82e2-a041a26d0d7f (api:52)
> 29865:2018-03-21 10:46:30,376+0200 INFO (periodic/0) [vdsm.api] START
> multipath_health() from=internal, task_id=3d3cad70-e5c8-49b2-9f5d-92a249cc102d
> (api:46)
> 29866:2018-03-21 10:46:30,377+0200 INFO (periodic/0) [vdsm.api] FINISH
> multipath_health return={} from=internal, task_id=3d3cad70-e5c8-49b2-9f5d-92a249cc102d
> (api:52)
> 29867:2018-03-21 10:46:35,108+0200 INFO (jsonrpc/6) [vdsm.api] START
> getSpmStatus(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
> options=None) from=::ffff:192.168.0.4,49914, task_id=304d7e7c-03fc-44d6-be32-d083ada09b30
> (api:46)
> 29868:2018-03-21 10:46:35,114+0200 INFO (jsonrpc/6) [vdsm.api] FINISH
> getSpmStatus return={'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver':
> 6L}} from=::ffff:192.168.0.4,49914, task_id=304d7e7c-03fc-44d6-be32-d083ada09b30
> (api:52)
> 29869:2018-03-21 10:46:35,114+0200 INFO (jsonrpc/6)
> [jsonrpc.JsonRpcServer] RPC call StoragePool.getSpmStatus succeeded in 0.00
> seconds (__init__:573)
> 29870:2018-03-21 10:46:35,170+0200 INFO (jsonrpc/4) [vdsm.api] START
> getStoragePoolInfo(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
> options=None) from=::ffff:192.168.0.4,49920, task_id=76736a93-d4c4-4b2f-9848-7b8731fe2b67
> (api:46)
> 29871:2018-03-21 10:46:35,176+0200 INFO (jsonrpc/4) [vdsm.api] FINISH
> getStoragePoolInfo return={'info': {'name': 'No Description', 'isoprefix':
> u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
> nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
> 11111111-1111-1111-1111-111111111111', 'pool_status': 'connected',
> 'lver': 6L, 'domains': u'ef184b28-1dbc-45ed-b0b3-
> 85e780cce5d8:Active,8a2c304b-c8ae-438b-af54-fc8797ea149f:Active',
> 'master_uuid': u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8', 'version': '4',
> 'spm_id': 1, 'type': 'NFS', 'master_ver': 3}, 'dominfo':
> {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'status': u'Active',
> 'diskfree': '1868692455424', 'isoprefix': '', 'alerts': [], 'disktotal':
> '1968811540480', 'version': 4}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'status': u'Active', 'diskfree': '1868692455424', 'isoprefix':
> u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
> nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
> 11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal':
> '1968811540480', 'version': 0}}} from=::ffff:192.168.0.4,49920,
> task_id=76736a93-d4c4-4b2f-9848-7b8731fe2b67 (api:52)
> 29872:2018-03-21 10:46:35,177+0200 INFO (jsonrpc/4)
> [jsonrpc.JsonRpcServer] RPC call StoragePool.getInfo succeeded in 0.01
> seconds (__init__:573)
> 29873:2018-03-21 10:46:35,462+0200 INFO (jsonrpc/2) [api.host] START
> getAllVmStats() from=::ffff:192.168.0.4,49914 (api:46)
> 29874:2018-03-21 10:46:35,462+0200 INFO (jsonrpc/2) [api.host] FINISH
> getAllVmStats return={'status': {'message': 'Done', 'code': 0},
> 'statsList': (suppressed)} from=::ffff:192.168.0.4,49914 (api:52)
> 29875:2018-03-21 10:46:35,463+0200 INFO (jsonrpc/2)
> [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
> seconds (__init__:573)
> 29876:2018-03-21 10:46:36,948+0200 INFO (jsonrpc/7) [api.host] START
> getAllVmStats() from=::1,36114 (api:46)
> 29877:2018-03-21 10:46:36,948+0200 INFO (jsonrpc/7) [api.host] FINISH
> getAllVmStats return={'status': {'message': 'Done', 'code': 0},
> 'statsList': (suppressed)} from=::1,36114 (api:52)
> 29878:2018-03-21 10:46:36,949+0200 INFO (jsonrpc/7)
> [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
> seconds (__init__:573)
> 29879:2018-03-21 10:46:43,179+0200 INFO (jsonrpc/1) [api.host] START
> getStats() from=::ffff:192.168.0.4,49914 (api:46)
> 29880:2018-03-21 10:46:43,180+0200 INFO (jsonrpc/1) [vdsm.api] START
> repoStats(domains=()) from=::ffff:192.168.0.4,49914,
> task_id=83bbc95c-6d59-4eec-82b1-e8993be28759 (api:46)
> 29881:2018-03-21 10:46:43,181+0200 INFO (jsonrpc/1) [vdsm.api] FINISH
> repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
> 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000486101',
> 'lastCheck': '3.6', 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '0.00043444', 'lastCheck': '3.4', 'valid': True}}
> from=::ffff:192.168.0.4,49914, task_id=83bbc95c-6d59-4eec-82b1-e8993be28759
> (api:52)
> 29882:2018-03-21 10:46:43,181+0200 INFO (jsonrpc/1) [vdsm.api] START
> multipath_health() from=::ffff:192.168.0.4,49914,
> task_id=9b781324-b50b-49a9-bf31-5eca835dc75c (api:46)
> 29883:2018-03-21 10:46:43,181+0200 INFO (jsonrpc/1) [vdsm.api] FINISH
> multipath_health return={} from=::ffff:192.168.0.4,49914,
> task_id=9b781324-b50b-49a9-bf31-5eca835dc75c (api:52)
> 29884:2018-03-21 10:46:43,188+0200 INFO (jsonrpc/1) [api.host] FINISH
> getStats return={'status': {'message': 'Done', 'code': 0}, 'info':
> {'cpuStatistics': {'11': {'cpuUser': '0.40', 'nodeIndex': 1, 'cpuSys':
> '0.27', 'cpuIdle': '99.33'}, '10': {'cpuUser': '0.60', 'nodeIndex': 0,
> 'cpuSys': '0.47', 'cpuIdle': '98.93'}, '13': {'cpuUser': '0.00',
> 'nodeIndex': 1, 'cpuSys': '0.27', 'cpuIdle': '99.73'}, '12': {'cpuUser':
> '1.46', 'nodeIndex': 0, 'cpuSys': '0.60', 'cpuIdle': '97.94'}, '15':
> {'cpuUser': '0.20', 'nodeIndex': 1, 'cpuSys': '0.33', 'cpuIdle': '99.47'},
> '14': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.47', 'cpuIdle':
> '99.13'}, '1': {'cpuUser': '1.33', 'nodeIndex': 1, 'cpuSys': '1.93',
> 'cpuIdle': '96.74'}, '0': {'cpuUser': '0.60', 'nodeIndex': 0, 'cpuSys':
> '0.27', 'cpuIdle': '99.13'}, '3': {'cpuUser': '0.20', 'nodeIndex': 1,
> 'cpuSys': '0.40', 'cpuIdle': '99.40'}, '2': {'cpuUser': '6.99',
> 'nodeIndex': 0, 'cpuSys': '0.40', 'cpuIdle': '92.61'}, '5': {'cpuUser':
> '0.13', 'nodeIndex': 1, 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '4':
> {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '99.46'},
> '7': {'cpuUser': '0.47', 'nodeIndex': 1, 'cpuSys': '0.60', 'cpuIdle':
> '98.93'}, '6': {'cpuUser': '0.53', 'nodeIndex': 0, 'cpuSys': '0.27',
> 'cpuIdle': '99.20'}, '9': {'cpuUser': '0.47', 'nodeIndex': 1, 'cpuSys':
> '0.33', 'cpuIdle': '99.20'}, '8': {'cpuUser': '0.07', 'nodeIndex': 0,
> 'cpuSys': '0.13', 'cpuIdle': '99.80'}}, 'numaNodeMemFree': {'1':
> {'memPercent': 7, 'memFree': '15131'}, '0': {'memPercent': 5, 'memFree':
> '15357'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes':
> True, 'vmCount': 0, 'memUsed': '3', 'storageDomains':
> {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0, 'actual': True,
> 'version': 4, 'acquired': True, 'delay': '0.000486101', 'lastCheck': '3.6',
> 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f': {'code': 0,
> 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.00043444',
> 'lastCheck': '3.4', 'valid': True}}, 'incomingVmMigrations': 0, 'network':
> {'enp3s0f0': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621990.363726, 'name': 'enp3s0f0', 'tx': '1091404888', 'txDropped':
> '0', 'rx': '11914970667', 'rxErrors': '0', 'speed': '100', 'rxDropped':
> '1455'}, 'ovirtmgmt': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621990.363726, 'name': 'ovirtmgmt', 'tx': '1048752406', 'txDropped':
> '0', 'rx': '11615298347', 'rxErrors': '0', 'speed': '1000', 'rxDropped':
> '0'}, 'lo': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621990.363726, 'name': 'lo', 'tx': '58215969455', 'txDropped': '0',
> 'rx': '58215969455', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'},
> 'enp3s0f1': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521621990.363726, 'name': 'enp3s0f1', 'tx': '0', 'txDropped': '0', 'rx':
> '66839', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'},
> ';vdsmdummy;': {'txErrors': '0', 'state': 'down', 'sampleTime':
> 1521621990.363726, 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0',
> 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'enp4s0f0':
> {'txErrors': '0', 'state': 'down', 'sampleTime': 1521621990.363726, 'name':
> 'enp4s0f0', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0',
> 'speed': '1000', 'rxDropped': '0'}, 'enp4s0f1': {'txErrors': '0', 'state':
> 'down', 'sampleTime': 1521621990.363726, 'name': 'enp4s0f1', 'tx': '0',
> 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped':
> '0'}}, 'txDropped': '0', 'anonHugePages': '206', 'ksmPages': 100,
> 'elapsedTime': '76562.02', 'cpuLoad': '0.18', 'cpuSys': '0.45',
> 'diskStats': {'/var/log': {'free': '7344'}, '/var/run/vdsm/': {'free':
> '16060'}, '/tmp': {'free': '906'}}, 'cpuUserVdsmd': '1.27',
> 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState': False,
> 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 31080, 'bootTime':
> '1521545382', 'haStats': {'active': False, 'configured': False, 'score': 0,
> 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus':
> 'active', 'multipathHealth': {}, 'rxDropped': '1455',
> 'outgoingVmMigrations': 0, 'swapTotal': 12287, 'swapFree': 12287,
> 'hugepages': defaultdict(<type 'dict'>, {2048: {'resv_hugepages': 0,
> 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0,
> 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0},
> 1048576: {'resv_hugepages': 0, 'free_hugepages': 0,
> 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages':
> 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}), 'dateTime':
> '2018-03-21T08:46:43 GMT', 'cpuUser': '0.87', 'memFree': 31336, 'cpuIdle':
> '98.68', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd': '1.20'}}
> from=::ffff:192.168.0.4,49914 (api:52)
> 29885:2018-03-21 10:46:43,190+0200 INFO (jsonrpc/1)
> [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds
> (__init__:573)
> 29886:2018-03-21 10:46:45,288+0200 INFO (jsonrpc/5) [vdsm.api] START
> getSpmStatus(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
> options=None) from=::ffff:192.168.0.4,49914, task_id=ffe57349-2532-4b64-b32c-8a558273d8e5
> (api:46)
> 29887:2018-03-21 10:46:45,292+0200 INFO (jsonrpc/5) [vdsm.api] FINISH
> getSpmStatus return={'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver':
> 6L}} from=::ffff:192.168.0.4,49914, task_id=ffe57349-2532-4b64-b32c-8a558273d8e5
> (api:52)
> 29888:2018-03-21 10:46:45,293+0200 INFO (jsonrpc/5)
> [jsonrpc.JsonRpcServer] RPC call StoragePool.getSpmStatus succeeded in 0.00
> seconds (__init__:573)
> 29889:2018-03-21 10:46:45,298+0200 INFO (jsonrpc/3) [vdsm.api] START
> getStoragePoolInfo(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
> options=None) from=::ffff:192.168.0.4,49920, task_id=b2ea28f0-abf0-4258-8152-6035d93aa5dc
> (api:46)
> 29890:2018-03-21 10:46:45,305+0200 INFO (jsonrpc/3) [vdsm.api] FINISH
> getStoragePoolInfo return={'info': {'name': 'No Description', 'isoprefix':
> u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
> nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
> 11111111-1111-1111-1111-111111111111', 'pool_status': 'connected',
> 'lver': 6L, 'domains': u'ef184b28-1dbc-45ed-b0b3-
> 85e780cce5d8:Active,8a2c304b-c8ae-438b-af54-fc8797ea149f:Active',
> 'master_uuid': u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8', 'version': '4',
> 'spm_id': 1, 'type': 'NFS', 'master_ver': 3}, 'dominfo':
> {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'status': u'Active',
> 'diskfree': '1868692455424', 'isoprefix': '', 'alerts': [], 'disktotal':
> '1968811540480', 'version': 4}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'status': u'Active', 'diskfree': '1868692455424', 'isoprefix':
> u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
> nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
> 11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal':
> '1968811540480', 'version': 0}}} from=::ffff:192.168.0.4,49920,
> task_id=b2ea28f0-abf0-4258-8152-6035d93aa5dc (api:52)
> 29891:2018-03-21 10:46:45,306+0200 INFO (jsonrpc/3)
> [jsonrpc.JsonRpcServer] RPC call StoragePool.getInfo succeeded in 0.00
> seconds (__init__:573)
> 29892:2018-03-21 10:46:45,395+0200 INFO (periodic/2) [vdsm.api] START
> repoStats(domains=()) from=internal, task_id=4010bf72-5c86-4daf-a8f9-12ec229a86e4
> (api:46)
> 29893:2018-03-21 10:46:45,395+0200 INFO (periodic/2) [vdsm.api] FINISH
> repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
> 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000486101',
> 'lastCheck': '5.8', 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '0.00043444', 'lastCheck': '5.7', 'valid': True}} from=internal,
> task_id=4010bf72-5c86-4daf-a8f9-12ec229a86e4 (api:52)
> 29894:2018-03-21 10:46:45,396+0200 INFO (periodic/2) [vdsm.api] START
> multipath_health() from=internal, task_id=37a1132f-452b-4805-9cc0-b8e74e8d6b02
> (api:46)
> 29895:2018-03-21 10:46:45,396+0200 INFO (periodic/2) [vdsm.api] FINISH
> multipath_health return={} from=internal, task_id=37a1132f-452b-4805-9cc0-b8e74e8d6b02
> (api:52)
> 29896:2018-03-21 10:46:50,467+0200 INFO (jsonrpc/0) [api.host] START
> getAllVmStats() from=::ffff:192.168.0.4,49914 (api:46)
> 29897:2018-03-21 10:46:50,468+0200 INFO (jsonrpc/0) [throttled] Current
> getAllVmStats: {} (throttledlog:103)
> 29898:2018-03-21 10:46:50,468+0200 INFO (jsonrpc/0) [api.host] FINISH
> getAllVmStats return={'status': {'message': 'Done', 'code': 0},
> 'statsList': (suppressed)} from=::ffff:192.168.0.4,49914 (api:52)
> 29899:2018-03-21 10:46:50,468+0200 INFO (jsonrpc/0)
> [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.01
> seconds (__init__:573)
> 29900:2018-03-21 10:46:51,967+0200 INFO (jsonrpc/6) [api.host] START
> getAllVmStats() from=::1,36114 (api:46)
> 29901:2018-03-21 10:46:51,968+0200 INFO (jsonrpc/6) [api.host] FINISH
> getAllVmStats return={'status': {'message': 'Done', 'code': 0},
> 'statsList': (suppressed)} from=::1,36114 (api:52)
> 29902:2018-03-21 10:46:51,968+0200 INFO (jsonrpc/6)
> [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
> seconds (__init__:573)
> 29903:2018-03-21 10:46:55,648+0200 INFO (jsonrpc/4) [vdsm.api] START
> getSpmStatus(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
> options=None) from=::ffff:192.168.0.4,49914, task_id=a975635f-59d7-4f40-b9a7-a5ea7150368e
> (api:46)
> 29904:2018-03-21 10:46:55,653+0200 INFO (jsonrpc/4) [vdsm.api] FINISH
> getSpmStatus return={'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver':
> 6L}} from=::ffff:192.168.0.4,49914, task_id=a975635f-59d7-4f40-b9a7-a5ea7150368e
> (api:52)
> 29905:2018-03-21 10:46:55,653+0200 INFO (jsonrpc/4)
> [jsonrpc.JsonRpcServer] RPC call StoragePool.getSpmStatus succeeded in 0.00
> seconds (__init__:573)
> 29906:2018-03-21 10:46:55,659+0200 INFO (jsonrpc/2) [vdsm.api] START
> getStoragePoolInfo(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
> options=None) from=::ffff:192.168.0.4,49920, task_id=97d4ae6c-3719-4149-85fd-28e924ad8883
> (api:46)
> 29907:2018-03-21 10:46:55,666+0200 INFO (jsonrpc/2) [vdsm.api] FINISH
> getStoragePoolInfo return={'info': {'name': 'No Description', 'isoprefix':
> u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
> nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
> 11111111-1111-1111-1111-111111111111', 'pool_status': 'connected',
> 'lver': 6L, 'domains': u'ef184b28-1dbc-45ed-b0b3-
> 85e780cce5d8:Active,8a2c304b-c8ae-438b-af54-fc8797ea149f:Active',
> 'master_uuid': u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8', 'version': '4',
> 'spm_id': 1, 'type': 'NFS', 'master_ver': 3}, 'dominfo':
> {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'status': u'Active',
> 'diskfree': '1868692455424', 'isoprefix': '', 'alerts': [], 'disktotal':
> '1968811540480', 'version': 4}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'status': u'Active', 'diskfree': '1868692455424', 'isoprefix':
> u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
> nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
> 11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal':
> '1968811540480', 'version': 0}}} from=::ffff:192.168.0.4,49920,
> task_id=97d4ae6c-3719-4149-85fd-28e924ad8883 (api:52)
> 29908:2018-03-21 10:46:55,667+0200 INFO (jsonrpc/2)
> [jsonrpc.JsonRpcServer] RPC call StoragePool.getInfo succeeded in 0.00
> seconds (__init__:573)
> 29909:2018-03-21 10:46:59,473+0200 INFO (jsonrpc/7) [api.host] START
> getStats() from=::ffff:192.168.0.4,49914 (api:46)
> 29910:2018-03-21 10:46:59,474+0200 INFO (jsonrpc/7) [vdsm.api] START
> repoStats(domains=()) from=::ffff:192.168.0.4,49914,
> task_id=177b551b-9bd3-45a4-84d0-5244630c08e5 (api:46)
> 29911:2018-03-21 10:46:59,474+0200 INFO (jsonrpc/7) [vdsm.api] FINISH
> repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
> 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000470906',
> 'lastCheck': '9.9', 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
> {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
> '0.000334928', 'lastCheck': '9.7', 'valid': True}}
> from=::ffff:192.168.0.4,49914, task_id=177b551b-9bd3-45a4-84d0-5244630c08e5
> (api:52)
> 29912:2018-03-21 10:46:59,475+0200 INFO (jsonrpc/7) [vdsm.api] START
> multipath_health() from=::ffff:192.168.0.4,49914,
> task_id=30753d5c-9bfd-4b0f-ac83-e32278d82b63 (api:46)
> 29913:2018-03-21 10:46:59,475+0200 INFO (jsonrpc/7) [vdsm.api] FINISH
> multipath_health return={} from=::ffff:192.168.0.4,49914,
> task_id=30753d5c-9bfd-4b0f-ac83-e32278d82b63 (api:52)
> 29914:2018-03-21 10:46:59,482+0200 INFO (jsonrpc/7) [api.host] FINISH
> getStats return={'status': {'message': 'Done', 'code': 0}, 'info':
> {'cpuStatistics': {'11': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys':
> '0.13', 'cpuIdle': '99.80'}, '10': {'cpuUser': '0.07', 'nodeIndex': 0,
> 'cpuSys': '0.13', 'cpuIdle': '99.80'}, '13': {'cpuUser': '0.07',
> 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '12': {'cpuUser':
> '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '15':
> {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.87'},
> '14': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle':
> '99.73'}, '1': {'cpuUser': '1.07', 'nodeIndex': 1, 'cpuSys': '1.07',
> 'cpuIdle': '97.86'}, '0': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys':
> '0.07', 'cpuIdle': '99.86'}, '3': {'cpuUser': '0.07', 'nodeIndex': 1,
> 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '2': {'cpuUser': '0.00',
> 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.87'}, '5': {'cpuUser':
> '0.07', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.80'}, '4':
> {'cpuUser': '0.47', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '99.26'},
> '7': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.47', 'cpuIdle':
> '99.53'}, '6': {'cpuUser': '0.20', 'nodeIndex': 0, 'cpuSys': '0.07',
> 'cpuIdle': '99.73'}, '9': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys':
> '0.13', 'cpuIdle': '99.80'}, '8': {'cpuUser': '0.00', 'nodeIndex': 0,
> 'cpuSys': '0.07', 'cpuIdle': '99.93'}}, 'numaNodeMemFree': {'1':
> {'memPercent': 7, 'memFree': '15125'}, '0': {'memPercent': 5, 'memFree':
> '15364'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes':
> True, 'vmCount': 0, 'memUsed': '3', 'storageDomains':
> {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0, 'actual': True,
> 'version': 4, 'acquired': True, 'delay': '0.000470906', 'lastCheck': '9.9',
> 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f': {'code': 0,
> 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000334928',
> 'lastCheck': '9.7', 'valid': True}}, 'incomingVmMigrations': 0, 'network':
> {'enp3s0f0': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521622005.383289, 'name': 'enp3s0f0', 'tx': '1091412332', 'txDropped':
> '0', 'rx': '11914976557', 'rxErrors': '0', 'speed': '100', 'rxDropped':
> '1456'}, 'ovirtmgmt': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521622005.383289, 'name': 'ovirtmgmt', 'tx': '1048761326', 'txDropped':
> '0', 'rx': '11615304242', 'rxErrors': '0', 'speed': '1000', 'rxDropped':
> '0'}, 'lo': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521622005.383289, 'name': 'lo', 'tx': '58228460774', 'txDropped': '0',
> 'rx': '58228460774', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'},
> 'enp3s0f1': {'txErrors': '0', 'state': 'up', 'sampleTime':
> 1521622005.383289, 'name': 'enp3s0f1', 'tx': '0', 'txDropped': '0', 'rx':
> '66839', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'},
> ';vdsmdummy;': {'txErrors': '0', 'state': 'down', 'sampleTime':
> 1521622005.383289, 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0',
> 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'enp4s0f0':
> {'txErrors': '0', 'state': 'down', 'sampleTime': 1521622005.383289, 'name':
> 'enp4s0f0', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0',
> 'speed': '1000', 'rxDropped': '0'}, 'enp4s0f1': {'txErrors': '0', 'state':
> 'down', 'sampleTime': 1521622005.383289, 'name': 'enp4s0f1', 'tx': '0',
> 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped':
> '0'}}, 'txDropped': '0', 'anonHugePages': '208', 'ksmPages': 100,
> 'elapsedTime': '76578.31', 'cpuLoad': '0.17', 'cpuSys': '0.20',
> 'diskStats': {'/var/log': {'free': '7344'}, '/var/run/vdsm/': {'free':
> '16060'}, '/tmp': {'free': '906'}}, 'cpuUserVdsmd': '1.13',
> 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState': False,
> 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 31078, 'bootTime':
> '1521545382', 'haStats': {'active': False, 'configured': False, 'score': 0,
> 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus':
> 'active', 'multipathHealth': {}, 'rxDropped': '1456',
> 'outgoingVmMigrations': 0, 'swapTotal': 12287, 'swapFree': 12287,
> 'hugepages': defaultdict(<type 'dict'>, {2048: {'resv_hugepages': 0,
> 'free_hugepages': 0, 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0,
> 'vm.free_hugepages': 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0},
> 1048576: {'resv_hugepages': 0, 'free_hugepages': 0,
> 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0, 'vm.free_hugepages':
> 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}), 'dateTime':
> '2018-03-21T08:46:59 GMT', 'cpuUser': '0.15', 'memFree': 31334, 'cpuIdle':
> '99.65', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd': '0.60'}}
> from=::ffff:192.168.0.4,49914 (api:52)
> 29915:2018-03-21 10:46:59,484+0200 INFO (jsonrpc/7)
> [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds
> (__init__:573)
>
>
> On 21 Mar 2018, at 11:33, Fred Rolland <frolland at redhat.com> wrote:
>
> Can you provide the vdsm logs from the host.
> It looks the vdsm failed to connect to the server.
>
> On Wed, Mar 21, 2018 at 11:21 AM, Andrei Verovski <andreil1 at starlett.lv>
> wrote:
>
>> Hi,
>>
>> I have 2-host oVirt setup with 2 Data Centers, one with local storage
>> domain (DC #1) for VMs + Export domain on NFS, another with all NFS shared
>> (DC #2).
>> Trying to export VMs from DC #1 to DC #2.
>> VMs are exported to DC #1 export domain (NFS), then domain put into
>> maintenance mode and detached from DC #1.
>>
>> Unfortunately, attaching it to DC #2 failed. Logs attached. Tried to run
>> this command twice.
>> Workaround are possible in order to accomplish this task, yet it would be
>> better to do in a way as it was designed.
>> Thanks.
>>
>>
>> 2018-03-21 10:43:24,024+02 ERROR [org.ovirt.engine.core.dal.dbb
>> roker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-128904)
>> [570ec5d9-fff5-4656-afbd-90b3207a616e] EVENT_ID:
>> USER_ATTACH_STORAGE_DOMAIN_TO_POOL_FAILED(963), Failed to attach Storage
>> Domain node10-NFS-EXPORTS to Data Center StrDataCenter11. (User:
>> admin at internal-authz)
>> 2018-03-21 10:46:16,512+02 ERROR [org.ovirt.engine.core.dal.dbb
>> roker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-128956)
>> [1435fc81] EVENT_ID: VDS_STORAGES_CONNECTION_FAILED(188), Failed to
>> connect Host node11 to the Storage Domains node10-NFS-EXPORTS.
>> 2018-03-21 10:46:16,651+02 ERROR [org.ovirt.engine.core.dal.dbb
>> roker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-128955)
>> [a81ffa4a-5a58-41a0-888a-f0edc321609b] EVENT_ID:
>> USER_ATTACH_STORAGE_DOMAIN_TO_POOL_FAILED(963), Failed to attach Storage
>> Domain node10-NFS-EXPORTS to Data Center StrDataCenter11. (User:
>> admin at internal-authz)
>>
>>
>> *tail -n 1000 engine.log | grep 570ec5d9-fff5-4656-afbd-90b3207a616e*
>> 2018-03-21 10:41:14,643+02 INFO [org.ovirt.engine.core.bll.sto
>> rage.domain.AttachStorageDomainToPoolCommand] (default task-2)
>> [570ec5d9-fff5-4656-afbd-90b3207a616e] Lock Acquired to object
>> 'EngineLock:{exclusiveLocks='[1d7208ce-d3a1-4406-9638-fe7051562994=STORAGE]',
>> sharedLocks=''}'
>> 2018-03-21 10:41:16,129+02 INFO [org.ovirt.engine.core.bll.sto
>> rage.domain.AttachStorageDomainToPoolCommand]
>> (EE-ManagedThreadFactory-engine-Thread-128904)
>> [570ec5d9-fff5-4656-afbd-90b3207a616e] Running command:
>> AttachStorageDomainToPoolCommand internal: false. Entities affected :
>> ID: 1d7208ce-d3a1-4406-9638-fe7051562994 Type: StorageAction group
>> MANIPULATE_STORAGE_DOMAIN with role type ADMIN, ID:
>> 80cc922f-8dea-4fed-b951-1060ba116ad5 Type: StoragePoolAction group
>> MANIPULATE_STORAGE_DOMAIN with role type ADMIN
>> 2018-03-21 10:43:23,564+02 ERROR [org.ovirt.engine.core.bll.sto
>> rage.domain.AttachStorageDomainToPoolCommand]
>> (EE-ManagedThreadFactory-engine-Thread-128904)
>> [570ec5d9-fff5-4656-afbd-90b3207a616e] Cannot connect storage connection
>> server, aborting attach storage domain operation.
>> 2018-03-21 10:43:23,567+02 INFO [org.ovirt.engine.core.bll.CommandCompensator]
>> (EE-ManagedThreadFactory-engine-Thread-128904)
>> [570ec5d9-fff5-4656-afbd-90b3207a616e] Command
>> [id=921ca7cd-4f93-46aa-8de2-91b13b8f96cb]: Compensating NEW_ENTITY_ID of
>> org.ovirt.engine.core.common.businessentities.StoragePoolIsoMap;
>> snapshot: StoragePoolIsoMapId:{storagePoolId='80cc922f-8dea-4fed-b951-1060ba116ad5',
>> storageId='1d7208ce-d3a1-4406-9638-fe7051562994'}.
>> 2018-03-21 10:43:24,024+02 ERROR [org.ovirt.engine.core.dal.dbb
>> roker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-128904)
>> [570ec5d9-fff5-4656-afbd-90b3207a616e] EVENT_ID:
>> USER_ATTACH_STORAGE_DOMAIN_TO_POOL_FAILED(963), Failed to attach Storage
>> Domain node10-NFS-EXPORTS to Data Center StrDataCenter11. (User:
>> admin at internal-authz)
>> 2018-03-21 10:43:24,114+02 INFO [org.ovirt.engine.core.bll.sto
>> rage.domain.AttachStorageDomainToPoolCommand]
>> (EE-ManagedThreadFactory-engine-Thread-128904)
>> [570ec5d9-fff5-4656-afbd-90b3207a616e] Lock freed to object
>> 'EngineLock:{exclusiveLocks='[1d7208ce-d3a1-4406-9638-fe7051562994=STORAGE]',
>> sharedLocks=''}'
>>
>>
>>
>> *[root at node00 ovirt-engine]# tail -n 1000 engine.log | grep
>> a81ffa4a-5a58-41a0-888a-f0edc321609b*
>> 2018-03-21 10:44:11,025+02 INFO [org.ovirt.engine.core.bll.sto
>> rage.domain.AttachStorageDomainToPoolCommand] (default task-16)
>> [a81ffa4a-5a58-41a0-888a-f0edc321609b] Lock Acquired to object
>> 'EngineLock:{exclusiveLocks='[1d7208ce-d3a1-4406-9638-fe7051562994=STORAGE]',
>> sharedLocks=''}'
>> 2018-03-21 10:44:11,236+02 INFO [org.ovirt.engine.core.bll.sto
>> rage.domain.AttachStorageDomainToPoolCommand]
>> (EE-ManagedThreadFactory-engine-Thread-128955)
>> [a81ffa4a-5a58-41a0-888a-f0edc321609b] Running command:
>> AttachStorageDomainToPoolCommand internal: false. Entities affected :
>> ID: 1d7208ce-d3a1-4406-9638-fe7051562994 Type: StorageAction group
>> MANIPULATE_STORAGE_DOMAIN with role type ADMIN, ID:
>> 80cc922f-8dea-4fed-b951-1060ba116ad5 Type: StoragePoolAction group
>> MANIPULATE_STORAGE_DOMAIN with role type ADMIN
>> 2018-03-21 10:46:16,567+02 ERROR [org.ovirt.engine.core.bll.sto
>> rage.domain.AttachStorageDomainToPoolCommand]
>> (EE-ManagedThreadFactory-engine-Thread-128955)
>> [a81ffa4a-5a58-41a0-888a-f0edc321609b] Cannot connect storage connection
>> server, aborting attach storage domain operation.
>> 2018-03-21 10:46:16,568+02 INFO [org.ovirt.engine.core.bll.CommandCompensator]
>> (EE-ManagedThreadFactory-engine-Thread-128955)
>> [a81ffa4a-5a58-41a0-888a-f0edc321609b] Command
>> [id=b5c25100-1a8a-4db0-9509-99cfa60995b1]: Compensating NEW_ENTITY_ID of
>> org.ovirt.engine.core.common.businessentities.StoragePoolIsoMap;
>> snapshot: StoragePoolIsoMapId:{storagePoolId='80cc922f-8dea-4fed-b951-1060ba116ad5',
>> storageId='1d7208ce-d3a1-4406-9638-fe7051562994'}.
>> 2018-03-21 10:46:16,651+02 ERROR [org.ovirt.engine.core.dal.dbb
>> roker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-128955)
>> [a81ffa4a-5a58-41a0-888a-f0edc321609b] EVENT_ID:
>> USER_ATTACH_STORAGE_DOMAIN_TO_POOL_FAILED(963), Failed to attach Storage
>> Domain node10-NFS-EXPORTS to Data Center StrDataCenter11. (User:
>> admin at internal-authz)
>> 2018-03-21 10:46:16,681+02 INFO [org.ovirt.engine.core.bll.sto
>> rage.domain.AttachStorageDomainToPoolCommand]
>> (EE-ManagedThreadFactory-engine-Thread-128955)
>> [a81ffa4a-5a58-41a0-888a-f0edc321609b] Lock freed to object
>> 'EngineLock:{exclusiveLocks='[1d7208ce-d3a1-4406-9638-fe7051562994=STORAGE]',
>> sharedLocks='’}'
>>
>>
>> _______________________________________________
>> Users mailing list
>> Users at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20180321/9b45cd3c/attachment.html>
More information about the Users
mailing list