Re: [ovirt-users] Attach Export Domain to another Data Center - Failure

Wednesday, 21 March 2018

node11 fails to mount the export domain from node 10.

You can try manually to see if you have access from node 11.
Look at this page to debug the NFS connection:
https://www.ovirt.org/documentation/how-to/troubleshooting/troubleshootin...

On Wed, Mar 21, 2018 at 12:04 PM, Andrei Verovski <andreil1(a)starlett.lv&gt;
wrote:

...
 Hi,

 Errors occurred at 10.43 AM and 10.46 AM
 node00.starlett.lv - 192.168.0.4 - oVirt host engine (separate PC, not
 hosted)
 node10.starlett.lv - 192.168.0.5 - host #1 of DC #1, export domain from
 which was detached from DC #1
 node11.starlett.lv - 192.168.0.6 - host #1 of DC #2,

 Logs from DC#2 node11, to which I’m trying to attach export domain located
 at NFS share node10.

 *grep 570ec5d9-fff5-4656-afbd-90b3207a616e*
>
 within vdsm.log returned nothing, so I did

 *grep -n 10:43 vdsm.log | tail -1000*
 1011:2018-03-21 06:10:43,077+0200 INFO  (jsonrpc/3) [api.host] START
 getAllVmStats() from=::1,36114 (api:46)
 1012:2018-03-21 06:10:43,077+0200 INFO  (jsonrpc/3) [api.host] FINISH
 getAllVmStats return={'status': {'message': 'Done',
'code': 0},
 'statsList': (suppressed)} from=::1,36114 (api:52)
 1013:2018-03-21 06:10:43,077+0200 INFO  (jsonrpc/3)
 [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.01
 seconds (__init__:573)
 1014:2018-03-21 06:10:43,868+0200 INFO  (jsonrpc/7) [api.host] START
 getAllVmStats() from=::ffff:192.168.0.4,49914 (api:46)
 1015:2018-03-21 06:10:43,868+0200 INFO  (jsonrpc/7) [api.host] FINISH
 getAllVmStats return={'status': {'message': 'Done',
'code': 0},
 'statsList': (suppressed)} from=::ffff:192.168.0.4,49914 (api:52)
 1016:2018-03-21 06:10:43,868+0200 INFO  (jsonrpc/7)
 [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
 seconds (__init__:573)
 19729:2018-03-21 09:10:43,641+0200 INFO  (jsonrpc/5) [api.host] START
 getAllVmStats() from=::1,36114 (api:46)
 19730:2018-03-21 09:10:43,641+0200 INFO  (jsonrpc/5) [api.host] FINISH
 getAllVmStats return={'status': {'message': 'Done',
'code': 0},
 'statsList': (suppressed)} from=::1,36114 (api:52)
 19731:2018-03-21 09:10:43,642+0200 INFO  (jsonrpc/5)
 [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
 seconds (__init__:573)
 29302:2018-03-21 10:43:00,085+0200 INFO  (periodic/2) [vdsm.api] START
 repoStats(domains=()) from=internal, task_id=7b690c8a-5470-44da-a8f3-7e7e9b018e88
 (api:46)
 29303:2018-03-21 10:43:00,085+0200 INFO  (periodic/2) [vdsm.api] FINISH
 repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
 'actual': True, 'version': 4, 'acquired': True, 'delay':
'0.000477038',
 'lastCheck': '0.5', 'valid': True},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '0.000498252', 'lastCheck': '0.4', 'valid': True}}
from=internal,
 task_id=7b690c8a-5470-44da-a8f3-7e7e9b018e88 (api:52)
 29304:2018-03-21 10:43:00,086+0200 INFO  (periodic/2) [vdsm.api] START
 multipath_health() from=internal, task_id=9a625d43-a03c-429c-856f-9aa8e5ff65b5
 (api:46)
 29305:2018-03-21 10:43:00,086+0200 INFO  (periodic/2) [vdsm.api] FINISH
 multipath_health return={} from=internal, task_id=9a625d43-a03c-429c-856f-9aa8e5ff65b5
 (api:52)
 29306:2018-03-21 10:43:05,382+0200 INFO  (jsonrpc/5) [api.host] START
 getAllVmStats() from=::ffff:192.168.0.4,49914 (api:46)
 29307:2018-03-21 10:43:05,383+0200 INFO  (jsonrpc/5) [api.host] FINISH
 getAllVmStats return={'status': {'message': 'Done',
'code': 0},
 'statsList': (suppressed)} from=::ffff:192.168.0.4,49914 (api:52)
 29308:2018-03-21 10:43:05,383+0200 INFO  (jsonrpc/5)
 [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
 seconds (__init__:573)
 29309:2018-03-21 10:43:06,248+0200 INFO  (jsonrpc/3) [api.host] START
 getStats() from=::ffff:192.168.0.4,49914 (api:46)
 29310:2018-03-21 10:43:06,249+0200 INFO  (jsonrpc/3) [vdsm.api] START
 repoStats(domains=()) from=::ffff:192.168.0.4,49914,
 task_id=f80f9bb8-2e84-4f45-ab58-09a88e808cf3 (api:46)
 29311:2018-03-21 10:43:06,250+0200 INFO  (jsonrpc/3) [vdsm.api] FINISH
 repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
 'actual': True, 'version': 4, 'acquired': True, 'delay':
'0.000477038',
 'lastCheck': '6.7', 'valid': True},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '0.000498252', 'lastCheck': '6.6', 'valid': True}}
 from=::ffff:192.168.0.4,49914, task_id=f80f9bb8-2e84-4f45-ab58-09a88e808cf3
 (api:52)
 29312:2018-03-21 10:43:06,250+0200 INFO  (jsonrpc/3) [vdsm.api] START
 multipath_health() from=::ffff:192.168.0.4,49914,
 task_id=ddd7064d-cd12-43bf-a177-b2a93cc9035e (api:46)
 29313:2018-03-21 10:43:06,250+0200 INFO  (jsonrpc/3) [vdsm.api] FINISH
 multipath_health return={} from=::ffff:192.168.0.4,49914,
 task_id=ddd7064d-cd12-43bf-a177-b2a93cc9035e (api:52)
 29314:2018-03-21 10:43:06,256+0200 INFO  (jsonrpc/3) [api.host] FINISH
 getStats return={'status': {'message': 'Done', 'code':
0}, 'info':
 {'cpuStatistics': {'11': {'cpuUser': '0.13',
'nodeIndex': 1, 'cpuSys':
 '0.27', 'cpuIdle': '99.60'}, '10': {'cpuUser':
'0.13', 'nodeIndex': 0,
 'cpuSys': '0.40', 'cpuIdle': '99.47'}, '13':
{'cpuUser': '0.07',
 'nodeIndex': 1, 'cpuSys': '0.20', 'cpuIdle':
'99.73'}, '12': {'cpuUser':
 '0.13', 'nodeIndex': 0, 'cpuSys': '0.20',
'cpuIdle': '99.67'}, '15':
 {'cpuUser': '0.20', 'nodeIndex': 1, 'cpuSys':
'0.47', 'cpuIdle': '99.33'},
 '14': {'cpuUser': '0.13', 'nodeIndex': 0,
'cpuSys': '0.27', 'cpuIdle':
 '99.60'}, '1': {'cpuUser': '1.13', 'nodeIndex':
1, 'cpuSys': '1.60',
 'cpuIdle': '97.27'}, '0': {'cpuUser': '0.47',
'nodeIndex': 0, 'cpuSys':
 '0.27', 'cpuIdle': '99.26'}, '3': {'cpuUser':
'0.07', 'nodeIndex': 1,
 'cpuSys': '0.20', 'cpuIdle': '99.73'}, '2':
{'cpuUser': '0.33',
 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle':
'99.54'}, '5': {'cpuUser':
 '0.13', 'nodeIndex': 1, 'cpuSys': '0.13',
'cpuIdle': '99.74'}, '4':
 {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys':
'0.20', 'cpuIdle': '99.67'},
 '7': {'cpuUser': '0.13', 'nodeIndex': 1,
'cpuSys': '0.60', 'cpuIdle':
 '99.27'}, '6': {'cpuUser': '0.13', 'nodeIndex':
0, 'cpuSys': '0.33',
 'cpuIdle': '99.54'}, '9': {'cpuUser': '0.07',
'nodeIndex': 1, 'cpuSys':
 '0.27', 'cpuIdle': '99.66'}, '8': {'cpuUser':
'0.13', 'nodeIndex': 0,
 'cpuSys': '0.33', 'cpuIdle': '99.54'}},
'numaNodeMemFree': {'1':
 {'memPercent': 7, 'memFree': '15142'}, '0':
{'memPercent': 5, 'memFree':
 '15349'}}, 'memShared': 0, 'thpState': 'always',
'ksmMergeAcrossNodes':
 True, 'vmCount': 0, 'memUsed': '3', 'storageDomains':
 {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0, 'actual':
True,
 'version': 4, 'acquired': True, 'delay': '0.000477038',
'lastCheck': '6.7',
 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f': {'code':
0,
 'actual': True, 'version': 0, 'acquired': True, 'delay':
'0.000498252',
 'lastCheck': '6.6', 'valid': True}},
'incomingVmMigrations': 0, 'network':
 {'enp3s0f0': {'txErrors': '0', 'state': 'up',
'sampleTime':
 1521621780.072418, 'name': 'enp3s0f0', 'tx':
'1091287978', 'txDropped':
 '0', 'rx': '11914868563', 'rxErrors': '0',
'speed': '100', 'rxDropped':
 '1439'}, 'ovirtmgmt': {'txErrors': '0', 'state':
'up', 'sampleTime':
 1521621780.072418, 'name': 'ovirtmgmt', 'tx':
'1048639112', 'txDropped':
 '0', 'rx': '11615209092', 'rxErrors': '0',
'speed': '1000', 'rxDropped':
 '0'}, 'lo': {'txErrors': '0', 'state':
'up', 'sampleTime':
 1521621780.072418, 'name': 'lo', 'tx': '58057075945',
'txDropped': '0',
 'rx': '58057075945', 'rxErrors': '0', 'speed':
'1000', 'rxDropped': '0'},
 'enp3s0f1': {'txErrors': '0', 'state': 'up',
'sampleTime':
 1521621780.072418, 'name': 'enp3s0f1', 'tx': '0',
'txDropped': '0', 'rx':
 '66839', 'rxErrors': '0', 'speed': '1000',
'rxDropped': '0'},
 ';vdsmdummy;': {'txErrors': '0', 'state': 'down',
'sampleTime':
 1521621780.072418, 'name': ';vdsmdummy;', 'tx': '0',
'txDropped': '0',
 'rx': '0', 'rxErrors': '0', 'speed':
'1000', 'rxDropped': '0'}, 'enp4s0f0':
 {'txErrors': '0', 'state': 'down', 'sampleTime':
1521621780.072418, 'name':
 'enp4s0f0', 'tx': '0', 'txDropped': '0',
'rx': '0', 'rxErrors': '0',
 'speed': '1000', 'rxDropped': '0'}, 'enp4s0f1':
{'txErrors': '0', 'state':
 'down', 'sampleTime': 1521621780.072418, 'name':
'enp4s0f1', 'tx': '0',
 'txDropped': '0', 'rx': '0', 'rxErrors':
'0', 'speed': '1000', 'rxDropped':
 '0'}}, 'txDropped': '0', 'anonHugePages': '204',
'ksmPages': 100,
 'elapsedTime': '76345.09', 'cpuLoad': '0.16',
'cpuSys': '0.36',
 'diskStats': {'/var/log': {'free': '7344'},
'/var/run/vdsm/': {'free':
 '16060'}, '/tmp': {'free': '906'}},
'cpuUserVdsmd': '1.07',
 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState':
False,
 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 31081,
'bootTime':
 '1521545382', 'haStats': {'active': False, 'configured':
False, 'score': 0,
 'localMaintenance': False, 'globalMaintenance': False},
'momStatus':
 'active', 'multipathHealth': {}, 'rxDropped': '1439',
 'outgoingVmMigrations': 0, 'swapTotal': 12287, 'swapFree':
12287,
 'hugepages': defaultdict(<type 'dict'>, {2048:
{'resv_hugepages': 0,
 'free_hugepages': 0, 'nr_overcommit_hugepages': 0,
'surplus_hugepages': 0,
 'vm.free_hugepages': 0, 'nr_hugepages': 0,
'nr_hugepages_mempolicy': 0},
 1048576: {'resv_hugepages': 0, 'free_hugepages': 0,
 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0,
'vm.free_hugepages':
 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}),
'dateTime':
 '2018-03-21T08:43:06 GMT', 'cpuUser': '0.21', 'memFree':
31337, 'cpuIdle':
 '99.43', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd':
'0.93'}}
 from=::ffff:192.168.0.4,49914 (api:52)
 29315:2018-03-21 10:43:06,258+0200 INFO  (jsonrpc/3)
 [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds
 (__init__:573)
 29316:2018-03-21 10:43:06,632+0200 INFO  (jsonrpc/0) [api.host] START
 getAllVmStats() from=::1,36114 (api:46)
 29317:2018-03-21 10:43:06,632+0200 INFO  (jsonrpc/0) [api.host] FINISH
 getAllVmStats return={'status': {'message': 'Done',
'code': 0},
 'statsList': (suppressed)} from=::1,36114 (api:52)
 29318:2018-03-21 10:43:06,633+0200 INFO  (jsonrpc/0)
 [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
 seconds (__init__:573)
 29319:2018-03-21 10:43:07,415+0200 INFO  (jsonrpc/4) [vdsm.api] START
 getSpmStatus(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
 options=None) from=::ffff:192.168.0.4,49914,
task_id=276c13c7-61ec-4782-a0d9-5cd0f0b0a729
 (api:46)
 29320:2018-03-21 10:43:07,420+0200 INFO  (jsonrpc/4) [vdsm.api] FINISH
 getSpmStatus return={'spm_st': {'spmId': 1, 'spmStatus':
'SPM', 'spmLver':
 6L}} from=::ffff:192.168.0.4,49914, task_id=276c13c7-61ec-4782-a0d9-5cd0f0b0a729
 (api:52)
 29321:2018-03-21 10:43:07,420+0200 INFO  (jsonrpc/4)
 [jsonrpc.JsonRpcServer] RPC call StoragePool.getSpmStatus succeeded in 0.01
 seconds (__init__:573)
 29322:2018-03-21 10:43:07,427+0200 INFO  (jsonrpc/6) [vdsm.api] START
 getStoragePoolInfo(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
 options=None) from=::ffff:192.168.0.4,49920,
task_id=344b18df-a428-429a-b49f-381f901d1618
 (api:46)
 29323:2018-03-21 10:43:07,434+0200 INFO  (jsonrpc/6) [vdsm.api] FINISH
 getStoragePoolInfo return={'info': {'name': 'No Description',
'isoprefix':
 u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
 nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
 11111111-1111-1111-1111-111111111111', 'pool_status': 'connected',
 'lver': 6L, 'domains': u'ef184b28-1dbc-45ed-b0b3-
 85e780cce5d8:Active,8a2c304b-c8ae-438b-af54-fc8797ea149f:Active',
 'master_uuid': u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8',
'version': '4',
 'spm_id': 1, 'type': 'NFS', 'master_ver': 3},
'dominfo':
 {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'status': u'Active',
 'diskfree': '1868692455424', 'isoprefix': '',
'alerts': [], 'disktotal':
 '1968811540480', 'version': 4},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'status': u'Active', 'diskfree': '1868692455424',
'isoprefix':
 u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
 nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
 11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal':
 '1968811540480', 'version': 0}}} from=::ffff:192.168.0.4,49920,
 task_id=344b18df-a428-429a-b49f-381f901d1618 (api:52)
 29324:2018-03-21 10:43:07,435+0200 INFO  (jsonrpc/6)
 [jsonrpc.JsonRpcServer] RPC call StoragePool.getInfo succeeded in 0.01
 seconds (__init__:573)
 29325:2018-03-21 10:43:13,839+0200 WARN  (vdsm.Scheduler) [Executor]
 Worker blocked: <Worker name=jsonrpc/1 running <Task <JsonRpcTask
 {'params': {u'connectionParams': [{u'id':
u'461f65a9-3a81-4f3f-a46d-c5ed12520524',
 u'connection': u'node10.starlett.lv:/vmdata/nfs/exports', u'iqn':
u'',
 u'user': u'', u'tpgt': u'1', u'protocol_version':
u'auto', u'password':
 '********', u'port': u''}], u'storagepoolID':
u'00000000-0000-0000-0000-000000000000',
 u'domainType': 1}, 'jsonrpc': '2.0', 'method':
u'StoragePool.connectStorageServer',
 'id': u'5f98dfd5-998f-44a6-8594-e5f40144b0ed'} at 0x4076150>
timeout=60,
 duration=120 at 0x402fe50> task#=3754 at 0x353e690>, traceback:
 29376:2018-03-21 10:43:15,104+0200 INFO  (periodic/2) [vdsm.api] START
 repoStats(domains=()) from=internal, task_id=672a883e-1d60-4981-9eee-bdcac2ec30c1
 (api:46)
 29377:2018-03-21 10:43:15,105+0200 INFO  (periodic/2) [vdsm.api] FINISH
 repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
 'actual': True, 'version': 4, 'acquired': True, 'delay':
'0.000488777',
 'lastCheck': '5.6', 'valid': True},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '0.000481962', 'lastCheck': '5.4', 'valid': True}}
from=internal,
 task_id=672a883e-1d60-4981-9eee-bdcac2ec30c1 (api:52)
 29378:2018-03-21 10:43:15,105+0200 INFO  (periodic/2) [vdsm.api] START
 multipath_health() from=internal, task_id=55f1b797-da2a-4b07-affe-5b9e41a74307
 (api:46)
 29379:2018-03-21 10:43:15,106+0200 INFO  (periodic/2) [vdsm.api] FINISH
 multipath_health return={} from=internal, task_id=55f1b797-da2a-4b07-affe-5b9e41a74307
 (api:52)
 29380:2018-03-21 10:43:17,645+0200 INFO  (jsonrpc/7) [vdsm.api] START
 getSpmStatus(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
 options=None) from=::ffff:192.168.0.4,49914,
task_id=52a6ddeb-3c20-4f3f-b27d-94d19580b473
 (api:46)
 29381:2018-03-21 10:43:17,649+0200 INFO  (jsonrpc/7) [vdsm.api] FINISH
 getSpmStatus return={'spm_st': {'spmId': 1, 'spmStatus':
'SPM', 'spmLver':
 6L}} from=::ffff:192.168.0.4,49914, task_id=52a6ddeb-3c20-4f3f-b27d-94d19580b473
 (api:52)
 29382:2018-03-21 10:43:17,650+0200 INFO  (jsonrpc/7)
 [jsonrpc.JsonRpcServer] RPC call StoragePool.getSpmStatus succeeded in 0.01
 seconds (__init__:573)
 29383:2018-03-21 10:43:17,656+0200 INFO  (jsonrpc/2) [vdsm.api] START
 getStoragePoolInfo(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
 options=None) from=::ffff:192.168.0.4,49920,
task_id=1a103883-6a6a-4d61-ade6-96559591ddb0
 (api:46)
 29384:2018-03-21 10:43:17,663+0200 INFO  (jsonrpc/2) [vdsm.api] FINISH
 getStoragePoolInfo return={'info': {'name': 'No Description',
'isoprefix':
 u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
 nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
 11111111-1111-1111-1111-111111111111', 'pool_status': 'connected',
 'lver': 6L, 'domains': u'ef184b28-1dbc-45ed-b0b3-
 85e780cce5d8:Active,8a2c304b-c8ae-438b-af54-fc8797ea149f:Active',
 'master_uuid': u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8',
'version': '4',
 'spm_id': 1, 'type': 'NFS', 'master_ver': 3},
'dominfo':
 {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'status': u'Active',
 'diskfree': '1868692455424', 'isoprefix': '',
'alerts': [], 'disktotal':
 '1968811540480', 'version': 4},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'status': u'Active', 'diskfree': '1868692455424',
'isoprefix':
 u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
 nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
 11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal':
 '1968811540480', 'version': 0}}} from=::ffff:192.168.0.4,49920,
 task_id=1a103883-6a6a-4d61-ade6-96559591ddb0 (api:52)
 29385:2018-03-21 10:43:17,663+0200 INFO  (jsonrpc/2)
 [jsonrpc.JsonRpcServer] RPC call StoragePool.getInfo succeeded in 0.01
 seconds (__init__:573)
 29386:2018-03-21 10:43:18,882+0200 ERROR (jsonrpc/1) [storage.HSM] Could
 not connect to storageServer (hsm:2407)
 29406:2018-03-21 10:43:18,883+0200 INFO  (jsonrpc/1) [vdsm.api] FINISH
 connectStorageServer return={'statuslist': [{'status': 477,
'id':
 u'461f65a9-3a81-4f3f-a46d-c5ed12520524'}]} from=::ffff:192.168.0.4,49914,
 flow_id=4a53b512, task_id=e598cbe0-cde8-4c73-b526-4398df05e67f (api:52)
 29407:2018-03-21 10:43:18,883+0200 INFO  (jsonrpc/1)
 [jsonrpc.JsonRpcServer] RPC call StoragePool.connectStorageServer
 succeeded in 125.04 seconds (__init__:573)
 29408:2018-03-21 10:43:20,387+0200 INFO  (jsonrpc/5) [api.host] START
 getAllVmStats() from=::ffff:192.168.0.4,49914 (api:46)
 29409:2018-03-21 10:43:20,388+0200 INFO  (jsonrpc/5) [api.host] FINISH
 getAllVmStats return={'status': {'message': 'Done',
'code': 0},
 'statsList': (suppressed)} from=::ffff:192.168.0.4,49914 (api:52)
 29410:2018-03-21 10:43:20,388+0200 INFO  (jsonrpc/5)
 [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.01
 seconds (__init__:573)
 29411:2018-03-21 10:43:21,497+0200 INFO  (jsonrpc/3) [api.host] START
 getStats() from=::ffff:192.168.0.4,49914 (api:46)
 29412:2018-03-21 10:43:21,498+0200 INFO  (jsonrpc/3) [vdsm.api] START
 repoStats(domains=()) from=::ffff:192.168.0.4,49914,
 task_id=a7469c1f-3d6d-42e7-8798-5c5335da4a9b (api:46)
 29413:2018-03-21 10:43:21,498+0200 INFO  (jsonrpc/3) [vdsm.api] FINISH
 repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
 'actual': True, 'version': 4, 'acquired': True, 'delay':
'0.000445096',
 'lastCheck': '1.9', 'valid': True},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '0.000296586', 'lastCheck': '1.8', 'valid': True}}
 from=::ffff:192.168.0.4,49914, task_id=a7469c1f-3d6d-42e7-8798-5c5335da4a9b
 (api:52)
 29414:2018-03-21 10:43:21,499+0200 INFO  (jsonrpc/3) [vdsm.api] START
 multipath_health() from=::ffff:192.168.0.4,49914,
 task_id=9fa5dae3-e2bd-4186-9a1d-afb0aa6e2511 (api:46)
 29415:2018-03-21 10:43:21,499+0200 INFO  (jsonrpc/3) [vdsm.api] FINISH
 multipath_health return={} from=::ffff:192.168.0.4,49914,
 task_id=9fa5dae3-e2bd-4186-9a1d-afb0aa6e2511 (api:52)
 29416:2018-03-21 10:43:21,505+0200 INFO  (jsonrpc/3) [api.host] FINISH
 getStats return={'status': {'message': 'Done', 'code':
0}, 'info':
 {'cpuStatistics': {'11': {'cpuUser': '0.07',
'nodeIndex': 1, 'cpuSys':
 '0.07', 'cpuIdle': '99.86'}, '10': {'cpuUser':
'0.07', 'nodeIndex': 0,
 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '13':
{'cpuUser': '0.07',
 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle':
'99.86'}, '12': {'cpuUser':
 '0.33', 'nodeIndex': 0, 'cpuSys': '0.20',
'cpuIdle': '99.47'}, '15':
 {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys':
'0.13', 'cpuIdle': '99.87'},
 '14': {'cpuUser': '0.07', 'nodeIndex': 0,
'cpuSys': '0.33', 'cpuIdle':
 '99.60'}, '1': {'cpuUser': '1.00', 'nodeIndex':
1, 'cpuSys': '1.00',
 'cpuIdle': '98.00'}, '0': {'cpuUser': '0.07',
'nodeIndex': 0, 'cpuSys':
 '0.07', 'cpuIdle': '99.86'}, '3': {'cpuUser':
'0.00', 'nodeIndex': 1,
 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '2':
{'cpuUser': '0.27',
 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle':
'99.66'}, '5': {'cpuUser':
 '0.07', 'nodeIndex': 1, 'cpuSys': '0.07',
'cpuIdle': '99.86'}, '4':
 {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys':
'0.07', 'cpuIdle': '99.86'},
 '7': {'cpuUser': '0.07', 'nodeIndex': 1,
'cpuSys': '0.27', 'cpuIdle':
 '99.66'}, '6': {'cpuUser': '0.07', 'nodeIndex':
0, 'cpuSys': '0.07',
 'cpuIdle': '99.86'}, '9': {'cpuUser': '0.07',
'nodeIndex': 1, 'cpuSys':
 '0.13', 'cpuIdle': '99.80'}, '8': {'cpuUser':
'0.00', 'nodeIndex': 0,
 'cpuSys': '0.07', 'cpuIdle': '99.93'}},
'numaNodeMemFree': {'1':
 {'memPercent': 7, 'memFree': '15142'}, '0':
{'memPercent': 5, 'memFree':
 '15351'}}, 'memShared': 0, 'thpState': 'always',
'ksmMergeAcrossNodes':
 True, 'vmCount': 0, 'memUsed': '3', 'storageDomains':
 {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0, 'actual':
True,
 'version': 4, 'acquired': True, 'delay': '0.000445096',
'lastCheck': '1.9',
 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f': {'code':
0,
 'actual': True, 'version': 0, 'acquired': True, 'delay':
'0.000296586',
 'lastCheck': '1.8', 'valid': True}},
'incomingVmMigrations': 0, 'network':
 {'enp3s0f0': {'txErrors': '0', 'state': 'up',
'sampleTime':
 1521621795.091263, 'name': 'enp3s0f0', 'tx':
'1091295628', 'txDropped':
 '0', 'rx': '11914875210', 'rxErrors': '0',
'speed': '100', 'rxDropped':
 '1442'}, 'ovirtmgmt': {'txErrors': '0', 'state':
'up', 'sampleTime':
 1521621795.091263, 'name': 'ovirtmgmt', 'tx':
'1048646480', 'txDropped':
 '0', 'rx': '11615214750', 'rxErrors': '0',
'speed': '1000', 'rxDropped':
 '0'}, 'lo': {'txErrors': '0', 'state':
'up', 'sampleTime':
 1521621795.091263, 'name': 'lo', 'tx': '58066397668',
'txDropped': '0',
 'rx': '58066397668', 'rxErrors': '0', 'speed':
'1000', 'rxDropped': '0'},
 'enp3s0f1': {'txErrors': '0', 'state': 'up',
'sampleTime':
 1521621795.091263, 'name': 'enp3s0f1', 'tx': '0',
'txDropped': '0', 'rx':
 '66839', 'rxErrors': '0', 'speed': '1000',
'rxDropped': '0'},
 ';vdsmdummy;': {'txErrors': '0', 'state': 'down',
'sampleTime':
 1521621795.091263, 'name': ';vdsmdummy;', 'tx': '0',
'txDropped': '0',
 'rx': '0', 'rxErrors': '0', 'speed':
'1000', 'rxDropped': '0'}, 'enp4s0f0':
 {'txErrors': '0', 'state': 'down', 'sampleTime':
1521621795.091263, 'name':
 'enp4s0f0', 'tx': '0', 'txDropped': '0',
'rx': '0', 'rxErrors': '0',
 'speed': '1000', 'rxDropped': '0'}, 'enp4s0f1':
{'txErrors': '0', 'state':
 'down', 'sampleTime': 1521621795.091263, 'name':
'enp4s0f1', 'tx': '0',
 'txDropped': '0', 'rx': '0', 'rxErrors':
'0', 'speed': '1000', 'rxDropped':
 '0'}}, 'txDropped': '0', 'anonHugePages': '204',
'ksmPages': 100,
 'elapsedTime': '76360.34', 'cpuLoad': '0.15',
'cpuSys': '0.18',
 'diskStats': {'/var/log': {'free': '7344'},
'/var/run/vdsm/': {'free':
 '16060'}, '/tmp': {'free': '906'}},
'cpuUserVdsmd': '1.07',
 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState':
False,
 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 31031,
'bootTime':
 '1521545382', 'haStats': {'active': False, 'configured':
False, 'score': 0,
 'localMaintenance': False, 'globalMaintenance': False},
'momStatus':
 'active', 'multipathHealth': {}, 'rxDropped': '1442',
 'outgoingVmMigrations': 0, 'swapTotal': 12287, 'swapFree':
12287,
 'hugepages': defaultdict(<type 'dict'>, {2048:
{'resv_hugepages': 0,
 'free_hugepages': 0, 'nr_overcommit_hugepages': 0,
'surplus_hugepages': 0,
 'vm.free_hugepages': 0, 'nr_hugepages': 0,
'nr_hugepages_mempolicy': 0},
 1048576: {'resv_hugepages': 0, 'free_hugepages': 0,
 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0,
'vm.free_hugepages':
 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}),
'dateTime':
 '2018-03-21T08:43:21 GMT', 'cpuUser': '0.15', 'memFree':
31287, 'cpuIdle':
 '99.67', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd':
'0.67'}}
 from=::ffff:192.168.0.4,49914 (api:52)
 29417:2018-03-21 10:43:21,508+0200 INFO  (jsonrpc/3)
 [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.02 seconds
 (__init__:573)
 29418:2018-03-21 10:43:21,651+0200 INFO  (jsonrpc/0) [api.host] START
 getAllVmStats() from=::1,36114 (api:46)
 29419:2018-03-21 10:43:21,651+0200 INFO  (jsonrpc/0) [api.host] FINISH
 getAllVmStats return={'status': {'message': 'Done',
'code': 0},
 'statsList': (suppressed)} from=::1,36114 (api:52)
 29420:2018-03-21 10:43:21,652+0200 INFO  (jsonrpc/0)
 [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
 seconds (__init__:573)
 29421:2018-03-21 10:43:27,764+0200 INFO  (jsonrpc/4) [vdsm.api] START
 getSpmStatus(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
 options=None) from=::ffff:192.168.0.4,49914,
task_id=46d19cab-8903-4c52-9bb2-4dd8f370997e
 (api:46)
 29422:2018-03-21 10:43:27,769+0200 INFO  (jsonrpc/4) [vdsm.api] FINISH
 getSpmStatus return={'spm_st': {'spmId': 1, 'spmStatus':
'SPM', 'spmLver':
 6L}} from=::ffff:192.168.0.4,49914, task_id=46d19cab-8903-4c52-9bb2-4dd8f370997e
 (api:52)
 29423:2018-03-21 10:43:27,770+0200 INFO  (jsonrpc/4)
 [jsonrpc.JsonRpcServer] RPC call StoragePool.getSpmStatus succeeded in 0.01
 seconds (__init__:573)
 29424:2018-03-21 10:43:27,815+0200 INFO  (jsonrpc/6) [vdsm.api] START
 getStoragePoolInfo(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
 options=None) from=::ffff:192.168.0.4,49920,
task_id=290acc73-8f25-4fd7-9263-7bd05d46430a
 (api:46)
 29425:2018-03-21 10:43:27,822+0200 INFO  (jsonrpc/6) [vdsm.api] FINISH
 getStoragePoolInfo return={'info': {'name': 'No Description',
'isoprefix':
 u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
 nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
 11111111-1111-1111-1111-111111111111', 'pool_status': 'connected',
 'lver': 6L, 'domains': u'ef184b28-1dbc-45ed-b0b3-
 85e780cce5d8:Active,8a2c304b-c8ae-438b-af54-fc8797ea149f:Active',
 'master_uuid': u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8',
'version': '4',
 'spm_id': 1, 'type': 'NFS', 'master_ver': 3},
'dominfo':
 {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'status': u'Active',
 'diskfree': '1868692455424', 'isoprefix': '',
'alerts': [], 'disktotal':
 '1968811540480', 'version': 4},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'status': u'Active', 'diskfree': '1868692455424',
'isoprefix':
 u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
 nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
 11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal':
 '1968811540480', 'version': 0}}} from=::ffff:192.168.0.4,49920,
 task_id=290acc73-8f25-4fd7-9263-7bd05d46430a (api:52)
 29426:2018-03-21 10:43:27,823+0200 INFO  (jsonrpc/6)
 [jsonrpc.JsonRpcServer] RPC call StoragePool.getInfo succeeded in 0.01
 seconds (__init__:573)
 29427:2018-03-21 10:43:30,121+0200 INFO  (periodic/3) [vdsm.api] START
 repoStats(domains=()) from=internal, task_id=0abdebcf-2b0b-4d33-b552-c0f2ed3e567b
 (api:46)
 29428:2018-03-21 10:43:30,122+0200 INFO  (periodic/3) [vdsm.api] FINISH
 repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
 'actual': True, 'version': 4, 'acquired': True, 'delay':
'0.000402077',
 'lastCheck': '0.6', 'valid': True},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '0.000344445', 'lastCheck': '0.5', 'valid': True}}
from=internal,
 task_id=0abdebcf-2b0b-4d33-b552-c0f2ed3e567b (api:52)
 29429:2018-03-21 10:43:30,122+0200 INFO  (periodic/3) [vdsm.api] START
 multipath_health() from=internal, task_id=acde7e7d-ab4a-4811-934f-4d312da33208
 (api:46)
 29430:2018-03-21 10:43:30,123+0200 INFO  (periodic/3) [vdsm.api] FINISH
 multipath_health return={} from=internal, task_id=acde7e7d-ab4a-4811-934f-4d312da33208
 (api:52)
 29431:2018-03-21 10:43:35,393+0200 INFO  (jsonrpc/7) [api.host] START
 getAllVmStats() from=::ffff:192.168.0.4,49914 (api:46)
 29432:2018-03-21 10:43:35,394+0200 INFO  (jsonrpc/7) [api.host] FINISH
 getAllVmStats return={'status': {'message': 'Done',
'code': 0},
 'statsList': (suppressed)} from=::ffff:192.168.0.4,49914 (api:52)
 29433:2018-03-21 10:43:35,394+0200 INFO  (jsonrpc/7)
 [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
 seconds (__init__:573)
 29434:2018-03-21 10:43:36,671+0200 INFO  (jsonrpc/2) [api.host] START
 getAllVmStats() from=::1,36114 (api:46)
 29435:2018-03-21 10:43:36,671+0200 INFO  (jsonrpc/2) [api.host] FINISH
 getAllVmStats return={'status': {'message': 'Done',
'code': 0},
 'statsList': (suppressed)} from=::1,36114 (api:52)
 29436:2018-03-21 10:43:36,672+0200 INFO  (jsonrpc/2)
 [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
 seconds (__init__:573)
 29437:2018-03-21 10:43:37,356+0200 INFO  (jsonrpc/1) [api.host] START
 getStats() from=::ffff:192.168.0.4,49914 (api:46)
 29438:2018-03-21 10:43:37,357+0200 INFO  (jsonrpc/1) [vdsm.api] START
 repoStats(domains=()) from=::ffff:192.168.0.4,49914,
 task_id=bc14e253-d3d4-4cc3-9ae1-c0e6ac6e37f7 (api:46)
 29439:2018-03-21 10:43:37,357+0200 INFO  (jsonrpc/1) [vdsm.api] FINISH
 repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
 'actual': True, 'version': 4, 'acquired': True, 'delay':
'0.000402077',
 'lastCheck': '7.8', 'valid': True},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '0.000344445', 'lastCheck': '7.7', 'valid': True}}
 from=::ffff:192.168.0.4,49914, task_id=bc14e253-d3d4-4cc3-9ae1-c0e6ac6e37f7
 (api:52)
 29440:2018-03-21 10:43:37,358+0200 INFO  (jsonrpc/1) [vdsm.api] START
 multipath_health() from=::ffff:192.168.0.4,49914,
 task_id=23c698b8-d5bd-4746-b738-97b10f3e4bfb (api:46)
 29441:2018-03-21 10:43:37,358+0200 INFO  (jsonrpc/1) [vdsm.api] FINISH
 multipath_health return={} from=::ffff:192.168.0.4,49914,
 task_id=23c698b8-d5bd-4746-b738-97b10f3e4bfb (api:52)
 29442:2018-03-21 10:43:37,365+0200 INFO  (jsonrpc/1) [api.host] FINISH
 getStats return={'status': {'message': 'Done', 'code':
0}, 'info':
 {'cpuStatistics': {'11': {'cpuUser': '0.07',
'nodeIndex': 1, 'cpuSys':
 '0.07', 'cpuIdle': '99.86'}, '10': {'cpuUser':
'3.06', 'nodeIndex': 0,
 'cpuSys': '0.80', 'cpuIdle': '96.14'}, '13':
{'cpuUser': '0.00',
 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle':
'99.93'}, '12': {'cpuUser':
 '0.07', 'nodeIndex': 0, 'cpuSys': '0.20',
'cpuIdle': '99.73'}, '15':
 {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys':
'0.13', 'cpuIdle': '99.87'},
 '14': {'cpuUser': '0.27', 'nodeIndex': 0,
'cpuSys': '0.40', 'cpuIdle':
 '99.33'}, '1': {'cpuUser': '1.13', 'nodeIndex':
1, 'cpuSys': '1.33',
 'cpuIdle': '97.54'}, '0': {'cpuUser': '0.47',
'nodeIndex': 0, 'cpuSys':
 '0.33', 'cpuIdle': '99.20'}, '3': {'cpuUser':
'0.00', 'nodeIndex': 1,
 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '2':
{'cpuUser': '0.67',
 'nodeIndex': 0, 'cpuSys': '0.40', 'cpuIdle':
'98.93'}, '5': {'cpuUser':
 '0.13', 'nodeIndex': 1, 'cpuSys': '0.13',
'cpuIdle': '99.74'}, '4':
 {'cpuUser': '0.47', 'nodeIndex': 0, 'cpuSys':
'0.40', 'cpuIdle': '99.13'},
 '7': {'cpuUser': '0.33', 'nodeIndex': 1,
'cpuSys': '0.73', 'cpuIdle':
 '98.94'}, '6': {'cpuUser': '5.39', 'nodeIndex':
0, 'cpuSys': '0.47',
 'cpuIdle': '94.14'}, '9': {'cpuUser': '0.07',
'nodeIndex': 1, 'cpuSys':
 '0.27', 'cpuIdle': '99.66'}, '8': {'cpuUser':
'0.53', 'nodeIndex': 0,
 'cpuSys': '0.40', 'cpuIdle': '99.07'}},
'numaNodeMemFree': {'1':
 {'memPercent': 7, 'memFree': '15143'}, '0':
{'memPercent': 5, 'memFree':
 '15348'}}, 'memShared': 0, 'thpState': 'always',
'ksmMergeAcrossNodes':
 True, 'vmCount': 0, 'memUsed': '3', 'storageDomains':
 {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0, 'actual':
True,
 'version': 4, 'acquired': True, 'delay': '0.000402077',
'lastCheck': '7.8',
 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f': {'code':
0,
 'actual': True, 'version': 0, 'acquired': True, 'delay':
'0.000344445',
 'lastCheck': '7.7', 'valid': True}},
'incomingVmMigrations': 0, 'network':
 {'enp3s0f0': {'txErrors': '0', 'state': 'up',
'sampleTime':
 1521621810.108511, 'name': 'enp3s0f0', 'tx':
'1091305295', 'txDropped':
 '0', 'rx': '11914880953', 'rxErrors': '0',
'speed': '100', 'rxDropped':
 '1442'}, 'ovirtmgmt': {'txErrors': '0', 'state':
'up', 'sampleTime':
 1521621810.108511, 'name': 'ovirtmgmt', 'tx':
'1048655897', 'txDropped':
 '0', 'rx': '11615220341', 'rxErrors': '0',
'speed': '1000', 'rxDropped':
 '0'}, 'lo': {'txErrors': '0', 'state':
'up', 'sampleTime':
 1521621810.108511, 'name': 'lo', 'tx': '58078891247',
'txDropped': '0',
 'rx': '58078891247', 'rxErrors': '0', 'speed':
'1000', 'rxDropped': '0'},
 'enp3s0f1': {'txErrors': '0', 'state': 'up',
'sampleTime':
 1521621810.108511, 'name': 'enp3s0f1', 'tx': '0',
'txDropped': '0', 'rx':
 '66839', 'rxErrors': '0', 'speed': '1000',
'rxDropped': '0'},
 ';vdsmdummy;': {'txErrors': '0', 'state': 'down',
'sampleTime':
 1521621810.108511, 'name': ';vdsmdummy;', 'tx': '0',
'txDropped': '0',
 'rx': '0', 'rxErrors': '0', 'speed':
'1000', 'rxDropped': '0'}, 'enp4s0f0':
 {'txErrors': '0', 'state': 'down', 'sampleTime':
1521621810.108511, 'name':
 'enp4s0f0', 'tx': '0', 'txDropped': '0',
'rx': '0', 'rxErrors': '0',
 'speed': '1000', 'rxDropped': '0'}, 'enp4s0f1':
{'txErrors': '0', 'state':
 'down', 'sampleTime': 1521621810.108511, 'name':
'enp4s0f1', 'tx': '0',
 'txDropped': '0', 'rx': '0', 'rxErrors':
'0', 'speed': '1000', 'rxDropped':
 '0'}}, 'txDropped': '0', 'anonHugePages': '204',
'ksmPages': 100,
 'elapsedTime': '76376.19', 'cpuLoad': '0.16',
'cpuSys': '0.39',
 'diskStats': {'/var/log': {'free': '7344'},
'/var/run/vdsm/': {'free':
 '16060'}, '/tmp': {'free': '906'}},
'cpuUserVdsmd': '1.00',
 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState':
False,
 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 31080,
'bootTime':
 '1521545382', 'haStats': {'active': False, 'configured':
False, 'score': 0,
 'localMaintenance': False, 'globalMaintenance': False},
'momStatus':
 'active', 'multipathHealth': {}, 'rxDropped': '1442',
 'outgoingVmMigrations': 0, 'swapTotal': 12287, 'swapFree':
12287,
 'hugepages': defaultdict(<type 'dict'>, {2048:
{'resv_hugepages': 0,
 'free_hugepages': 0, 'nr_overcommit_hugepages': 0,
'surplus_hugepages': 0,
 'vm.free_hugepages': 0, 'nr_hugepages': 0,
'nr_hugepages_mempolicy': 0},
 1048576: {'resv_hugepages': 0, 'free_hugepages': 0,
 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0,
'vm.free_hugepages':
 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}),
'dateTime':
 '2018-03-21T08:43:37 GMT', 'cpuUser': '0.79', 'memFree':
31336, 'cpuIdle':
 '98.82', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd':
'0.73'}}
 from=::ffff:192.168.0.4,49914 (api:52)
 29443:2018-03-21 10:43:37,367+0200 INFO  (jsonrpc/1)
 [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds
 (__init__:573)
 29444:2018-03-21 10:43:38,124+0200 INFO  (jsonrpc/5) [vdsm.api] START
 getSpmStatus(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
 options=None) from=::ffff:192.168.0.4,49914,
task_id=9d2a82dd-c76c-4eb5-a1e1-b49bf004384e
 (api:46)
 29445:2018-03-21 10:43:38,128+0200 INFO  (jsonrpc/5) [vdsm.api] FINISH
 getSpmStatus return={'spm_st': {'spmId': 1, 'spmStatus':
'SPM', 'spmLver':
 6L}} from=::ffff:192.168.0.4,49914, task_id=9d2a82dd-c76c-4eb5-a1e1-b49bf004384e
 (api:52)
 29446:2018-03-21 10:43:38,129+0200 INFO  (jsonrpc/5)
 [jsonrpc.JsonRpcServer] RPC call StoragePool.getSpmStatus succeeded in 0.01
 seconds (__init__:573)
 29447:2018-03-21 10:43:38,171+0200 INFO  (jsonrpc/3) [vdsm.api] START
 getStoragePoolInfo(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
 options=None) from=::ffff:192.168.0.4,49920,
task_id=62d44b0b-20b2-47ae-bfb8-d4717e301b69
 (api:46)
 29448:2018-03-21 10:43:38,177+0200 INFO  (jsonrpc/3) [vdsm.api] FINISH
 getStoragePoolInfo return={'info': {'name': 'No Description',
'isoprefix':
 u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
 nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
 11111111-1111-1111-1111-111111111111', 'pool_status': 'connected',
 'lver': 6L, 'domains': u'ef184b28-1dbc-45ed-b0b3-
 85e780cce5d8:Active,8a2c304b-c8ae-438b-af54-fc8797ea149f:Active',
 'master_uuid': u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8',
'version': '4',
 'spm_id': 1, 'type': 'NFS', 'master_ver': 3},
'dominfo':
 {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'status': u'Active',
 'diskfree': '1868692455424', 'isoprefix': '',
'alerts': [], 'disktotal':
 '1968811540480', 'version': 4},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'status': u'Active', 'diskfree': '1868692455424',
'isoprefix':
 u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
 nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
 11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal':
 '1968811540480', 'version': 0}}} from=::ffff:192.168.0.4,49920,
 task_id=62d44b0b-20b2-47ae-bfb8-d4717e301b69 (api:52)
 29449:2018-03-21 10:43:38,178+0200 INFO  (jsonrpc/3)
 [jsonrpc.JsonRpcServer] RPC call StoragePool.getInfo succeeded in 0.01
 seconds (__init__:573)
 29450:2018-03-21 10:43:45,147+0200 INFO  (periodic/2) [vdsm.api] START
 repoStats(domains=()) from=internal, task_id=49c33285-a222-464d-8984-4e75a5c6354a
 (api:46)
 29451:2018-03-21 10:43:45,148+0200 INFO  (periodic/2) [vdsm.api] FINISH
 repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
 'actual': True, 'version': 4, 'acquired': True, 'delay':
'0.000454102',
 'lastCheck': '5.6', 'valid': True},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '0.000347381', 'lastCheck': '5.5', 'valid': True}}
from=internal,
 task_id=49c33285-a222-464d-8984-4e75a5c6354a (api:52)
 29452:2018-03-21 10:43:45,148+0200 INFO  (periodic/2) [vdsm.api] START
 multipath_health() from=internal, task_id=eed6b564-76b1-4f9b-80a2-1f0f6d9bbe32
 (api:46)
 29453:2018-03-21 10:43:45,149+0200 INFO  (periodic/2) [vdsm.api] FINISH
 multipath_health return={} from=internal, task_id=eed6b564-76b1-4f9b-80a2-1f0f6d9bbe32
 (api:52)
 29454:2018-03-21 10:43:48,405+0200 INFO  (jsonrpc/0) [vdsm.api] START
 getSpmStatus(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
 options=None) from=::ffff:192.168.0.4,49914,
task_id=84651a37-7d13-41a9-9dcf-2169a9c46fc5
 (api:46)
 29455:2018-03-21 10:43:48,410+0200 INFO  (jsonrpc/0) [vdsm.api] FINISH
 getSpmStatus return={'spm_st': {'spmId': 1, 'spmStatus':
'SPM', 'spmLver':
 6L}} from=::ffff:192.168.0.4,49914, task_id=84651a37-7d13-41a9-9dcf-2169a9c46fc5
 (api:52)
 29456:2018-03-21 10:43:48,411+0200 INFO  (jsonrpc/0)
 [jsonrpc.JsonRpcServer] RPC call StoragePool.getSpmStatus succeeded in 0.01
 seconds (__init__:573)
 29457:2018-03-21 10:43:48,417+0200 INFO  (jsonrpc/4) [vdsm.api] START
 getStoragePoolInfo(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
 options=None) from=::ffff:192.168.0.4,49920,
task_id=4ebd1d1a-11ac-41ee-b459-7e45f008776b
 (api:46)
 29458:2018-03-21 10:43:48,423+0200 INFO  (jsonrpc/4) [vdsm.api] FINISH
 getStoragePoolInfo return={'info': {'name': 'No Description',
'isoprefix':
 u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
 nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
 11111111-1111-1111-1111-111111111111', 'pool_status': 'connected',
 'lver': 6L, 'domains': u'ef184b28-1dbc-45ed-b0b3-
 85e780cce5d8:Active,8a2c304b-c8ae-438b-af54-fc8797ea149f:Active',
 'master_uuid': u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8',
'version': '4',
 'spm_id': 1, 'type': 'NFS', 'master_ver': 3},
'dominfo':
 {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'status': u'Active',
 'diskfree': '1868692455424', 'isoprefix': '',
'alerts': [], 'disktotal':
 '1968811540480', 'version': 4},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'status': u'Active', 'diskfree': '1868692455424',
'isoprefix':
 u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
 nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
 11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal':
 '1968811540480', 'version': 0}}} from=::ffff:192.168.0.4,49920,
 task_id=4ebd1d1a-11ac-41ee-b459-7e45f008776b (api:52)
 29459:2018-03-21 10:43:48,424+0200 INFO  (jsonrpc/4)
 [jsonrpc.JsonRpcServer] RPC call StoragePool.getInfo succeeded in 0.01
 seconds (__init__:573)
 29460:2018-03-21 10:43:50,399+0200 INFO  (jsonrpc/6) [api.host] START
 getAllVmStats() from=::ffff:192.168.0.4,49914 (api:46)
 29461:2018-03-21 10:43:50,400+0200 INFO  (jsonrpc/6) [api.host] FINISH
 getAllVmStats return={'status': {'message': 'Done',
'code': 0},
 'statsList': (suppressed)} from=::ffff:192.168.0.4,49914 (api:52)
 29462:2018-03-21 10:43:50,400+0200 INFO  (jsonrpc/6)
 [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
 seconds (__init__:573)
 29463:2018-03-21 10:43:51,694+0200 INFO  (jsonrpc/7) [api.host] START
 getAllVmStats() from=::1,36114 (api:46)
 29464:2018-03-21 10:43:51,694+0200 INFO  (jsonrpc/7) [api.host] FINISH
 getAllVmStats return={'status': {'message': 'Done',
'code': 0},
 'statsList': (suppressed)} from=::1,36114 (api:52)
 29465:2018-03-21 10:43:51,695+0200 INFO  (jsonrpc/7)
 [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
 seconds (__init__:573)
 29466:2018-03-21 10:43:52,516+0200 INFO  (jsonrpc/2) [api.host] START
 getStats() from=::ffff:192.168.0.4,49914 (api:46)
 29467:2018-03-21 10:43:52,517+0200 INFO  (jsonrpc/2) [vdsm.api] START
 repoStats(domains=()) from=::ffff:192.168.0.4,49914,
 task_id=0a88f0dc-eccf-4d24-9ac2-22e079a1480a (api:46)
 29468:2018-03-21 10:43:52,517+0200 INFO  (jsonrpc/2) [vdsm.api] FINISH
 repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
 'actual': True, 'version': 4, 'acquired': True, 'delay':
'0.000487937',
 'lastCheck': '3.0', 'valid': True},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '0.000455956', 'lastCheck': '2.9', 'valid': True}}
 from=::ffff:192.168.0.4,49914, task_id=0a88f0dc-eccf-4d24-9ac2-22e079a1480a
 (api:52)
 29469:2018-03-21 10:43:52,518+0200 INFO  (jsonrpc/2) [vdsm.api] START
 multipath_health() from=::ffff:192.168.0.4,49914,
 task_id=a7a44ff7-a137-41b6-9ad5-2c0b32ab79ec (api:46)
 29470:2018-03-21 10:43:52,518+0200 INFO  (jsonrpc/2) [vdsm.api] FINISH
 multipath_health return={} from=::ffff:192.168.0.4,49914,
 task_id=a7a44ff7-a137-41b6-9ad5-2c0b32ab79ec (api:52)
 29471:2018-03-21 10:43:52,524+0200 INFO  (jsonrpc/2) [api.host] FINISH
 getStats return={'status': {'message': 'Done', 'code':
0}, 'info':
 {'cpuStatistics': {'11': {'cpuUser': '0.07',
'nodeIndex': 1, 'cpuSys':
 '0.13', 'cpuIdle': '99.80'}, '10': {'cpuUser':
'0.00', 'nodeIndex': 0,
 'cpuSys': '0.13', 'cpuIdle': '99.87'}, '13':
{'cpuUser': '0.00',
 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle':
'99.93'}, '12': {'cpuUser':
 '0.00', 'nodeIndex': 0, 'cpuSys': '0.13',
'cpuIdle': '99.87'}, '15':
 {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys':
'0.07', 'cpuIdle': '99.86'},
 '14': {'cpuUser': '0.00', 'nodeIndex': 0,
'cpuSys': '0.13', 'cpuIdle':
 '99.87'}, '1': {'cpuUser': '1.13', 'nodeIndex':
1, 'cpuSys': '1.13',
 'cpuIdle': '97.74'}, '0': {'cpuUser': '0.33',
'nodeIndex': 0, 'cpuSys':
 '0.20', 'cpuIdle': '99.47'}, '3': {'cpuUser':
'0.07', 'nodeIndex': 1,
 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '2':
{'cpuUser': '0.00',
 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle':
'99.87'}, '5': {'cpuUser':
 '0.00', 'nodeIndex': 1, 'cpuSys': '0.13',
'cpuIdle': '99.87'}, '4':
 {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys':
'0.13', 'cpuIdle': '99.60'},
 '7': {'cpuUser': '0.00', 'nodeIndex': 1,
'cpuSys': '0.33', 'cpuIdle':
 '99.67'}, '6': {'cpuUser': '0.00', 'nodeIndex':
0, 'cpuSys': '0.13',
 'cpuIdle': '99.87'}, '9': {'cpuUser': '0.07',
'nodeIndex': 1, 'cpuSys':
 '0.13', 'cpuIdle': '99.80'}, '8': {'cpuUser':
'0.07', 'nodeIndex': 0,
 'cpuSys': '0.13', 'cpuIdle': '99.80'}},
'numaNodeMemFree': {'1':
 {'memPercent': 7, 'memFree': '15143'}, '0':
{'memPercent': 5, 'memFree':
 '15349'}}, 'memShared': 0, 'thpState': 'always',
'ksmMergeAcrossNodes':
 True, 'vmCount': 0, 'memUsed': '3', 'storageDomains':
 {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0, 'actual':
True,
 'version': 4, 'acquired': True, 'delay': '0.000487937',
'lastCheck': '3.0',
 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f': {'code':
0,
 'actual': True, 'version': 0, 'acquired': True, 'delay':
'0.000455956',
 'lastCheck': '2.9', 'valid': True}},
'incomingVmMigrations': 0, 'network':
 {'enp3s0f0': {'txErrors': '0', 'state': 'up',
'sampleTime':
 1521621825.13588, 'name': 'enp3s0f0', 'tx': '1091312739',
'txDropped': '0',
 'rx': '11914899051', 'rxErrors': '0', 'speed':
'100', 'rxDropped': '1443'},
 'ovirtmgmt': {'txErrors': '0', 'state': 'up',
'sampleTime':
 1521621825.13588, 'name': 'ovirtmgmt', 'tx':
'1048663107', 'txDropped':
 '0', 'rx': '11615236520', 'rxErrors': '0',
'speed': '1000', 'rxDropped':
 '0'}, 'lo': {'txErrors': '0', 'state':
'up', 'sampleTime':
 1521621825.13588, 'name': 'lo', 'tx': '58090295494',
'txDropped': '0',
 'rx': '58090295494', 'rxErrors': '0', 'speed':
'1000', 'rxDropped': '0'},
 'enp3s0f1': {'txErrors': '0', 'state': 'up',
'sampleTime':
 1521621825.13588, 'name': 'enp3s0f1', 'tx': '0',
'txDropped': '0', 'rx':
 '66839', 'rxErrors': '0', 'speed': '1000',
'rxDropped': '0'},
 ';vdsmdummy;': {'txErrors': '0', 'state': 'down',
'sampleTime':
 1521621825.13588, 'name': ';vdsmdummy;', 'tx': '0',
'txDropped': '0', 'rx':
 '0', 'rxErrors': '0', 'speed': '1000',
'rxDropped': '0'}, 'enp4s0f0':
 {'txErrors': '0', 'state': 'down', 'sampleTime':
1521621825.13588, 'name':
 'enp4s0f0', 'tx': '0', 'txDropped': '0',
'rx': '0', 'rxErrors': '0',
 'speed': '1000', 'rxDropped': '0'}, 'enp4s0f1':
{'txErrors': '0', 'state':
 'down', 'sampleTime': 1521621825.13588, 'name':
'enp4s0f1', 'tx': '0',
 'txDropped': '0', 'rx': '0', 'rxErrors':
'0', 'speed': '1000', 'rxDropped':
 '0'}}, 'txDropped': '0', 'anonHugePages': '204',
'ksmPages': 100,
 'elapsedTime': '76391.35', 'cpuLoad': '0.15',
'cpuSys': '0.20',
 'diskStats': {'/var/log': {'free': '7344'},
'/var/run/vdsm/': {'free':
 '16060'}, '/tmp': {'free': '906'}},
'cpuUserVdsmd': '1.13',
 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState':
False,
 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 31080,
'bootTime':
 '1521545382', 'haStats': {'active': False, 'configured':
False, 'score': 0,
 'localMaintenance': False, 'globalMaintenance': False},
'momStatus':
 'active', 'multipathHealth': {}, 'rxDropped': '1443',
 'outgoingVmMigrations': 0, 'swapTotal': 12287, 'swapFree':
12287,
 'hugepages': defaultdict(<type 'dict'>, {2048:
{'resv_hugepages': 0,
 'free_hugepages': 0, 'nr_overcommit_hugepages': 0,
'surplus_hugepages': 0,
 'vm.free_hugepages': 0, 'nr_hugepages': 0,
'nr_hugepages_mempolicy': 0},
 1048576: {'resv_hugepages': 0, 'free_hugepages': 0,
 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0,
'vm.free_hugepages':
 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}),
'dateTime':
 '2018-03-21T08:43:52 GMT', 'cpuUser': '0.14', 'memFree':
31336, 'cpuIdle':
 '99.66', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd':
'0.73'}}
 from=::ffff:192.168.0.4,49914 (api:52)
 29472:2018-03-21 10:43:52,526+0200 INFO  (jsonrpc/2)
 [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds
 (__init__:573)
 29473:2018-03-21 10:43:58,686+0200 INFO  (jsonrpc/1) [vdsm.api] START
 getSpmStatus(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
 options=None) from=::ffff:192.168.0.4,49914,
task_id=d6d29cf6-c083-4639-91a2-f062eae4e629
 (api:46)
 29474:2018-03-21 10:43:58,690+0200 INFO  (jsonrpc/1) [vdsm.api] FINISH
 getSpmStatus return={'spm_st': {'spmId': 1, 'spmStatus':
'SPM', 'spmLver':
 6L}} from=::ffff:192.168.0.4,49914, task_id=d6d29cf6-c083-4639-91a2-f062eae4e629
 (api:52)
 29475:2018-03-21 10:43:58,690+0200 INFO  (jsonrpc/1)
 [jsonrpc.JsonRpcServer] RPC call StoragePool.getSpmStatus succeeded in 0.01
 seconds (__init__:573)
 29476:2018-03-21 10:43:58,697+0200 INFO  (jsonrpc/5) [vdsm.api] START
 getStoragePoolInfo(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
 options=None) from=::ffff:192.168.0.4,49920,
task_id=9dbd91df-dd9c-469c-8308-bfbfc228937c
 (api:46)
 29477:2018-03-21 10:43:58,703+0200 INFO  (jsonrpc/5) [vdsm.api] FINISH
 getStoragePoolInfo return={'info': {'name': 'No Description',
'isoprefix':
 u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
 nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
 11111111-1111-1111-1111-111111111111', 'pool_status': 'connected',
 'lver': 6L, 'domains': u'ef184b28-1dbc-45ed-b0b3-
 85e780cce5d8:Active,8a2c304b-c8ae-438b-af54-fc8797ea149f:Active',
 'master_uuid': u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8',
'version': '4',
 'spm_id': 1, 'type': 'NFS', 'master_ver': 3},
'dominfo':
 {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'status': u'Active',
 'diskfree': '1868692455424', 'isoprefix': '',
'alerts': [], 'disktotal':
 '1968811540480', 'version': 4},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'status': u'Active', 'diskfree': '1868692455424',
'isoprefix':
 u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
 nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
 11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal':
 '1968811540480', 'version': 0}}} from=::ffff:192.168.0.4,49920,
 task_id=9dbd91df-dd9c-469c-8308-bfbfc228937c (api:52)
 29478:2018-03-21 10:43:58,704+0200 INFO  (jsonrpc/5)
 [jsonrpc.JsonRpcServer] RPC call StoragePool.getInfo succeeded in 0.01
 seconds (__init__:573)

 *grep -n 10:46 vdsm.log | tail -1000*
 1017:2018-03-21 06:10:46,882+0200 INFO  (jsonrpc/2) [api.host] START
 getStats() from=::ffff:192.168.0.4,49914 (api:46)
 1018:2018-03-21 06:10:46,883+0200 INFO  (jsonrpc/2) [vdsm.api] START
 repoStats(domains=()) from=::ffff:192.168.0.4,49914,
 task_id=6e045b00-88b2-469b-80d6-af38303f8c32 (api:46)
 1019:2018-03-21 06:10:46,883+0200 INFO  (jsonrpc/2) [vdsm.api] FINISH
 repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
 'actual': True, 'version': 4, 'acquired': True, 'delay':
'0.000332785',
 'lastCheck': '7.3', 'valid': True},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '0.000203914', 'lastCheck': '5.6', 'valid': True}}
 from=::ffff:192.168.0.4,49914, task_id=6e045b00-88b2-469b-80d6-af38303f8c32
 (api:52)
 1020:2018-03-21 06:10:46,884+0200 INFO  (jsonrpc/2) [vdsm.api] START
 multipath_health() from=::ffff:192.168.0.4,49914,
 task_id=475a5b40-4a43-42ee-9524-2bdd220aed83 (api:46)
 1021:2018-03-21 06:10:46,884+0200 INFO  (jsonrpc/2) [vdsm.api] FINISH
 multipath_health return={} from=::ffff:192.168.0.4,49914,
 task_id=475a5b40-4a43-42ee-9524-2bdd220aed83 (api:52)
 1022:2018-03-21 06:10:46,889+0200 INFO  (jsonrpc/2) [api.host] FINISH
 getStats return={'status': {'message': 'Done', 'code':
0}, 'info':
 {'cpuStatistics': {'11': {'cpuUser': '0.07',
'nodeIndex': 1, 'cpuSys':
 '0.20', 'cpuIdle': '99.73'}, '10': {'cpuUser':
'0.13', 'nodeIndex': 0,
 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '13':
{'cpuUser': '0.07',
 'nodeIndex': 1, 'cpuSys': '0.27', 'cpuIdle':
'99.66'}, '12': {'cpuUser':
 '0.33', 'nodeIndex': 0, 'cpuSys': '0.20',
'cpuIdle': '99.47'}, '15':
 {'cpuUser': '0.33', 'nodeIndex': 1, 'cpuSys':
'0.27', 'cpuIdle': '99.40'},
 '14': {'cpuUser': '0.47', 'nodeIndex': 0,
'cpuSys': '0.40', 'cpuIdle':
 '99.13'}, '1': {'cpuUser': '3.00', 'nodeIndex':
1, 'cpuSys': '1.40',
 'cpuIdle': '95.60'}, '0': {'cpuUser': '0.73',
'nodeIndex': 0, 'cpuSys':
 '0.20', 'cpuIdle': '99.07'}, '3': {'cpuUser':
'0.07', 'nodeIndex': 1,
 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '2':
{'cpuUser': '0.07',
 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle':
'99.66'}, '5': {'cpuUser':
 '0.07', 'nodeIndex': 1, 'cpuSys': '0.20',
'cpuIdle': '99.73'}, '4':
 {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys':
'0.33', 'cpuIdle': '99.54'},
 '7': {'cpuUser': '0.80', 'nodeIndex': 1,
'cpuSys': '0.80', 'cpuIdle':
 '98.40'}, '6': {'cpuUser': '5.46', 'nodeIndex':
0, 'cpuSys': '0.27',
 'cpuIdle': '94.27'}, '9': {'cpuUser': '0.20',
'nodeIndex': 1, 'cpuSys':
 '0.27', 'cpuIdle': '99.53'}, '8': {'cpuUser':
'1.27', 'nodeIndex': 0,
 'cpuSys': '0.47', 'cpuIdle': '98.26'}},
'numaNodeMemFree': {'1':
 {'memPercent': 7, 'memFree': '15128'}, '0':
{'memPercent': 5, 'memFree':
 '15366'}}, 'memShared': 0, 'thpState': 'always',
'ksmMergeAcrossNodes':
 True, 'vmCount': 0, 'memUsed': '3', 'storageDomains':
 {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0, 'actual':
True,
 'version': 4, 'acquired': True, 'delay': '0.000332785',
'lastCheck': '7.3',
 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f': {'code':
0,
 'actual': True, 'version': 0, 'acquired': True, 'delay':
'0.000203914',
 'lastCheck': '5.6', 'valid': True}},
'incomingVmMigrations': 0, 'network':
 {'enp3s0f0': {'txErrors': '0', 'state': 'up',
'sampleTime':
 1521605437.203776, 'name': 'enp3s0f0', 'tx':
'1082264809', 'txDropped':
 '0', 'rx': '11907188594', 'rxErrors': '0',
'speed': '100', 'rxDropped':
 '289'}, 'ovirtmgmt': {'txErrors': '0', 'state':
'up', 'sampleTime':
 1521605437.203776, 'name': 'ovirtmgmt', 'tx':
'1039869554', 'txDropped':
 '0', 'rx': '11608697383', 'rxErrors': '0',
'speed': '1000', 'rxDropped':
 '0'}, 'lo': {'txErrors': '0', 'state':
'up', 'sampleTime':
 1521605437.203776, 'name': 'lo', 'tx': '45587320171',
'txDropped': '0',
 'rx': '45587320171', 'rxErrors': '0', 'speed':
'1000', 'rxDropped': '0'},
 'enp3s0f1': {'txErrors': '0', 'state': 'up',
'sampleTime':
 1521605437.203776, 'name': 'enp3s0f1', 'tx': '0',
'txDropped': '0', 'rx':
 '53620', 'rxErrors': '0', 'speed': '1000',
'rxDropped': '0'},
 ';vdsmdummy;': {'txErrors': '0', 'state': 'down',
'sampleTime':
 1521605437.203776, 'name': ';vdsmdummy;', 'tx': '0',
'txDropped': '0',
 'rx': '0', 'rxErrors': '0', 'speed':
'1000', 'rxDropped': '0'}, 'enp4s0f0':
 {'txErrors': '0', 'state': 'down', 'sampleTime':
1521605437.203776, 'name':
 'enp4s0f0', 'tx': '0', 'txDropped': '0',
'rx': '0', 'rxErrors': '0',
 'speed': '1000', 'rxDropped': '0'}, 'enp4s0f1':
{'txErrors': '0', 'state':
 'down', 'sampleTime': 1521605437.203776, 'name':
'enp4s0f1', 'tx': '0',
 'txDropped': '0', 'rx': '0', 'rxErrors':
'0', 'speed': '1000', 'rxDropped':
 '0'}}, 'txDropped': '0', 'anonHugePages': '204',
'ksmPages': 100,
 'elapsedTime': '60005.72', 'cpuLoad': '0.06',
'cpuSys': '0.38',
 'diskStats': {'/var/log': {'free': '7357'},
'/var/run/vdsm/': {'free':
 '16060'}, '/tmp': {'free': '906'}},
'cpuUserVdsmd': '1.07',
 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState':
False,
 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 31084,
'bootTime':
 '1521545382', 'haStats': {'active': False, 'configured':
False, 'score': 0,
 'localMaintenance': False, 'globalMaintenance': False},
'momStatus':
 'active', 'multipathHealth': {}, 'rxDropped': '289',
 'outgoingVmMigrations': 0, 'swapTotal': 12287, 'swapFree':
12287,
 'hugepages': defaultdict(<type 'dict'>, {2048:
{'resv_hugepages': 0,
 'free_hugepages': 0, 'nr_overcommit_hugepages': 0,
'surplus_hugepages': 0,
 'vm.free_hugepages': 0, 'nr_hugepages': 0,
'nr_hugepages_mempolicy': 0},
 1048576: {'resv_hugepages': 0, 'free_hugepages': 0,
 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0,
'vm.free_hugepages':
 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}),
'dateTime':
 '2018-03-21T04:10:46 GMT', 'cpuUser': '0.83', 'memFree':
31340, 'cpuIdle':
 '98.79', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd':
'0.80'}}
 from=::ffff:192.168.0.4,49914 (api:52)
 1023:2018-03-21 06:10:46,891+0200 INFO  (jsonrpc/2)
 [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds
 (__init__:573)
 13522:2018-03-21 08:10:46,827+0200 INFO  (jsonrpc/7) [api.host] START
 getAllVmStats() from=::ffff:192.168.0.4,49914 (api:46)
 13523:2018-03-21 08:10:46,828+0200 INFO  (jsonrpc/7) [api.host] FINISH
 getAllVmStats return={'status': {'message': 'Done',
'code': 0},
 'statsList': (suppressed)} from=::ffff:192.168.0.4,49914 (api:52)
 13524:2018-03-21 08:10:46,828+0200 INFO  (jsonrpc/7)
 [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
 seconds (__init__:573)
 29734:2018-03-21 10:46:00,338+0200 INFO  (periodic/2) [vdsm.api] START
 repoStats(domains=()) from=internal, task_id=af099ce6-0f44-461b-9e8e-99de63b3884f
 (api:46)
 29735:2018-03-21 10:46:00,339+0200 INFO  (periodic/2) [vdsm.api] FINISH
 repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
 'actual': True, 'version': 4, 'acquired': True, 'delay':
'0.000470245',
 'lastCheck': '0.8', 'valid': True},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '0.000476935', 'lastCheck': '0.7', 'valid': True}}
from=internal,
 task_id=af099ce6-0f44-461b-9e8e-99de63b3884f (api:52)
 29736:2018-03-21 10:46:00,339+0200 INFO  (periodic/2) [vdsm.api] START
 multipath_health() from=internal, task_id=4a5709d9-e322-44eb-a8a4-bbe3f5e5e1bd
 (api:46)
 29737:2018-03-21 10:46:00,340+0200 INFO  (periodic/2) [vdsm.api] FINISH
 multipath_health return={} from=internal, task_id=4a5709d9-e322-44eb-a8a4-bbe3f5e5e1bd
 (api:52)
 29738:2018-03-21 10:46:04,637+0200 INFO  (jsonrpc/7) [vdsm.api] START
 getSpmStatus(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
 options=None) from=::ffff:192.168.0.4,49914,
task_id=5b9300f7-f8c4-4f63-a49c-45ca8119d3b9
 (api:46)
 29739:2018-03-21 10:46:04,642+0200 INFO  (jsonrpc/7) [vdsm.api] FINISH
 getSpmStatus return={'spm_st': {'spmId': 1, 'spmStatus':
'SPM', 'spmLver':
 6L}} from=::ffff:192.168.0.4,49914, task_id=5b9300f7-f8c4-4f63-a49c-45ca8119d3b9
 (api:52)
 29740:2018-03-21 10:46:04,642+0200 INFO  (jsonrpc/7)
 [jsonrpc.JsonRpcServer] RPC call StoragePool.getSpmStatus succeeded in 0.01
 seconds (__init__:573)
 29741:2018-03-21 10:46:04,648+0200 INFO  (jsonrpc/1) [vdsm.api] START
 getStoragePoolInfo(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
 options=None) from=::ffff:192.168.0.4,49920,
task_id=f4db5bfa-e131-4f9a-baaa-e7555c996ab0
 (api:46)
 29742:2018-03-21 10:46:04,655+0200 INFO  (jsonrpc/1) [vdsm.api] FINISH
 getStoragePoolInfo return={'info': {'name': 'No Description',
'isoprefix':
 u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
 nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
 11111111-1111-1111-1111-111111111111', 'pool_status': 'connected',
 'lver': 6L, 'domains': u'ef184b28-1dbc-45ed-b0b3-
 85e780cce5d8:Active,8a2c304b-c8ae-438b-af54-fc8797ea149f:Active',
 'master_uuid': u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8',
'version': '4',
 'spm_id': 1, 'type': 'NFS', 'master_ver': 3},
'dominfo':
 {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'status': u'Active',
 'diskfree': '1868692455424', 'isoprefix': '',
'alerts': [], 'disktotal':
 '1968811540480', 'version': 4},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'status': u'Active', 'diskfree': '1868692455424',
'isoprefix':
 u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
 nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
 11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal':
 '1968811540480', 'version': 0}}} from=::ffff:192.168.0.4,49920,
 task_id=f4db5bfa-e131-4f9a-baaa-e7555c996ab0 (api:52)
 29743:2018-03-21 10:46:04,656+0200 INFO  (jsonrpc/1)
 [jsonrpc.JsonRpcServer] RPC call StoragePool.getInfo succeeded in 0.00
 seconds (__init__:573)
 29744:2018-03-21 10:46:05,451+0200 INFO  (jsonrpc/5) [api.host] START
 getAllVmStats() from=::ffff:192.168.0.4,49914 (api:46)
 29745:2018-03-21 10:46:05,452+0200 INFO  (jsonrpc/5) [api.host] FINISH
 getAllVmStats return={'status': {'message': 'Done',
'code': 0},
 'statsList': (suppressed)} from=::ffff:192.168.0.4,49914 (api:52)
 29746:2018-03-21 10:46:05,452+0200 INFO  (jsonrpc/5)
 [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
 seconds (__init__:573)
 29747:2018-03-21 10:46:06,903+0200 INFO  (jsonrpc/3) [api.host] START
 getAllVmStats() from=::1,36114 (api:46)
 29748:2018-03-21 10:46:06,904+0200 INFO  (jsonrpc/3) [api.host] FINISH
 getAllVmStats return={'status': {'message': 'Done',
'code': 0},
 'statsList': (suppressed)} from=::1,36114 (api:52)
 29749:2018-03-21 10:46:06,904+0200 INFO  (jsonrpc/3)
 [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
 seconds (__init__:573)
 29750:2018-03-21 10:46:08,791+0200 WARN  (vdsm.Scheduler) [Executor]
 Worker blocked: <Worker name=jsonrpc/6 running <Task <JsonRpcTask
 {'params': {u'connectionParams': [{u'id':
u'461f65a9-3a81-4f3f-a46d-c5ed12520524',
 u'connection': u'node10.starlett.lv:/vmdata/nfs/exports', u'iqn':
u'',
 u'user': u'', u'tpgt': u'1', u'protocol_version':
u'auto', u'password':
 '********', u'port': u''}], u'storagepoolID':
u'00000000-0000-0000-0000-000000000000',
 u'domainType': 1}, 'jsonrpc': '2.0', 'method':
u'StoragePool.connectStorageServer',
 'id': u'71143eef-aac6-4996-9783-3e0e3da180c3'} at 0x3604fd0>
timeout=60,
 duration=120 at 0x3604e10> task#=3762 at 0x3541650>, traceback:
 29801:2018-03-21 10:46:12,852+0200 INFO  (jsonrpc/0) [api.host] START
 getStats() from=::ffff:192.168.0.4,49914 (api:46)
 29802:2018-03-21 10:46:12,853+0200 INFO  (jsonrpc/0) [vdsm.api] START
 repoStats(domains=()) from=::ffff:192.168.0.4,49914,
 task_id=391fdab9-d3c2-4075-9c34-177a07a21ec3 (api:46)
 29803:2018-03-21 10:46:12,854+0200 INFO  (jsonrpc/0) [vdsm.api] FINISH
 repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
 'actual': True, 'version': 4, 'acquired': True, 'delay':
'0.000492277',
 'lastCheck': '3.3', 'valid': True},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '0.000479153', 'lastCheck': '3.2', 'valid': True}}
 from=::ffff:192.168.0.4,49914, task_id=391fdab9-d3c2-4075-9c34-177a07a21ec3
 (api:52)
 29804:2018-03-21 10:46:12,854+0200 INFO  (jsonrpc/0) [vdsm.api] START
 multipath_health() from=::ffff:192.168.0.4,49914,
 task_id=7e53adbc-3974-4f51-bfc8-8fbd3ca6b749 (api:46)
 29805:2018-03-21 10:46:12,854+0200 INFO  (jsonrpc/0) [vdsm.api] FINISH
 multipath_health return={} from=::ffff:192.168.0.4,49914,
 task_id=7e53adbc-3974-4f51-bfc8-8fbd3ca6b749 (api:52)
 29806:2018-03-21 10:46:12,861+0200 INFO  (jsonrpc/0) [api.host] FINISH
 getStats return={'status': {'message': 'Done', 'code':
0}, 'info':
 {'cpuStatistics': {'11': {'cpuUser': '0.47',
'nodeIndex': 1, 'cpuSys':
 '0.40', 'cpuIdle': '99.13'}, '10': {'cpuUser':
'0.13', 'nodeIndex': 0,
 'cpuSys': '0.20', 'cpuIdle': '99.67'}, '13':
{'cpuUser': '0.07',
 'nodeIndex': 1, 'cpuSys': '0.20', 'cpuIdle':
'99.73'}, '12': {'cpuUser':
 '0.20', 'nodeIndex': 0, 'cpuSys': '0.27',
'cpuIdle': '99.53'}, '15':
 {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys':
'0.27', 'cpuIdle': '99.60'},
 '14': {'cpuUser': '0.13', 'nodeIndex': 0,
'cpuSys': '0.33', 'cpuIdle':
 '99.54'}, '1': {'cpuUser': '1.00', 'nodeIndex':
1, 'cpuSys': '1.33',
 'cpuIdle': '97.67'}, '0': {'cpuUser': '0.13',
'nodeIndex': 0, 'cpuSys':
 '0.27', 'cpuIdle': '99.60'}, '3': {'cpuUser':
'0.07', 'nodeIndex': 1,
 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '2':
{'cpuUser': '0.20',
 'nodeIndex': 0, 'cpuSys': '0.33', 'cpuIdle':
'99.47'}, '5': {'cpuUser':
 '0.07', 'nodeIndex': 1, 'cpuSys': '0.13',
'cpuIdle': '99.80'}, '4':
 {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys':
'0.20', 'cpuIdle': '99.73'},
 '7': {'cpuUser': '0.07', 'nodeIndex': 1,
'cpuSys': '0.40', 'cpuIdle':
 '99.53'}, '6': {'cpuUser': '0.47', 'nodeIndex':
0, 'cpuSys': '0.40',
 'cpuIdle': '99.13'}, '9': {'cpuUser': '0.07',
'nodeIndex': 1, 'cpuSys':
 '0.20', 'cpuIdle': '99.73'}, '8': {'cpuUser':
'0.13', 'nodeIndex': 0,
 'cpuSys': '0.33', 'cpuIdle': '99.54'}},
'numaNodeMemFree': {'1':
 {'memPercent': 7, 'memFree': '15142'}, '0':
{'memPercent': 5, 'memFree':
 '15348'}}, 'memShared': 0, 'thpState': 'always',
'ksmMergeAcrossNodes':
 True, 'vmCount': 0, 'memUsed': '3', 'storageDomains':
 {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0, 'actual':
True,
 'version': 4, 'acquired': True, 'delay': '0.000492277',
'lastCheck': '3.3',
 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f': {'code':
0,
 'actual': True, 'version': 0, 'acquired': True, 'delay':
'0.000479153',
 'lastCheck': '3.2', 'valid': True}},
'incomingVmMigrations': 0, 'network':
 {'enp3s0f0': {'txErrors': '0', 'state': 'up',
'sampleTime':
 1521621960.326184, 'name': 'enp3s0f0', 'tx':
'1091387790', 'txDropped':
 '0', 'rx': '11914959714', 'rxErrors': '0',
'speed': '100', 'rxDropped':
 '1451'}, 'ovirtmgmt': {'txErrors': '0', 'state':
'up', 'sampleTime':
 1521621960.326184, 'name': 'ovirtmgmt', 'tx':
'1048735814', 'txDropped':
 '0', 'rx': '11615289173', 'rxErrors': '0',
'speed': '1000', 'rxDropped':
 '0'}, 'lo': {'txErrors': '0', 'state':
'up', 'sampleTime':
 1521621960.326184, 'name': 'lo', 'tx': '58192068637',
'txDropped': '0',
 'rx': '58192068637', 'rxErrors': '0', 'speed':
'1000', 'rxDropped': '0'},
 'enp3s0f1': {'txErrors': '0', 'state': 'up',
'sampleTime':
 1521621960.326184, 'name': 'enp3s0f1', 'tx': '0',
'txDropped': '0', 'rx':
 '66839', 'rxErrors': '0', 'speed': '1000',
'rxDropped': '0'},
 ';vdsmdummy;': {'txErrors': '0', 'state': 'down',
'sampleTime':
 1521621960.326184, 'name': ';vdsmdummy;', 'tx': '0',
'txDropped': '0',
 'rx': '0', 'rxErrors': '0', 'speed':
'1000', 'rxDropped': '0'}, 'enp4s0f0':
 {'txErrors': '0', 'state': 'down', 'sampleTime':
1521621960.326184, 'name':
 'enp4s0f0', 'tx': '0', 'txDropped': '0',
'rx': '0', 'rxErrors': '0',
 'speed': '1000', 'rxDropped': '0'}, 'enp4s0f1':
{'txErrors': '0', 'state':
 'down', 'sampleTime': 1521621960.326184, 'name':
'enp4s0f1', 'tx': '0',
 'txDropped': '0', 'rx': '0', 'rxErrors':
'0', 'speed': '1000', 'rxDropped':
 '0'}}, 'txDropped': '0', 'anonHugePages': '204',
'ksmPages': 100,
 'elapsedTime': '76531.69', 'cpuLoad': '0.17',
'cpuSys': '0.32',
 'diskStats': {'/var/log': {'free': '7344'},
'/var/run/vdsm/': {'free':
 '16060'}, '/tmp': {'free': '906'}},
'cpuUserVdsmd': '1.00',
 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState':
False,
 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 31081,
'bootTime':
 '1521545382', 'haStats': {'active': False, 'configured':
False, 'score': 0,
 'localMaintenance': False, 'globalMaintenance': False},
'momStatus':
 'active', 'multipathHealth': {}, 'rxDropped': '1451',
 'outgoingVmMigrations': 0, 'swapTotal': 12287, 'swapFree':
12287,
 'hugepages': defaultdict(<type 'dict'>, {2048:
{'resv_hugepages': 0,
 'free_hugepages': 0, 'nr_overcommit_hugepages': 0,
'surplus_hugepages': 0,
 'vm.free_hugepages': 0, 'nr_hugepages': 0,
'nr_hugepages_mempolicy': 0},
 1048576: {'resv_hugepages': 0, 'free_hugepages': 0,
 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0,
'vm.free_hugepages':
 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}),
'dateTime':
 '2018-03-21T08:46:12 GMT', 'cpuUser': '0.21', 'memFree':
31337, 'cpuIdle':
 '99.46', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd':
'0.80'}}
 from=::ffff:192.168.0.4,49914 (api:52)
 29807:2018-03-21 10:46:12,863+0200 INFO  (jsonrpc/0)
 [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds
 (__init__:573)
 29808:2018-03-21 10:46:13,829+0200 ERROR (jsonrpc/6) [storage.HSM] Could
 not connect to storageServer (hsm:2407)
 29828:2018-03-21 10:46:13,829+0200 INFO  (jsonrpc/6) [vdsm.api] FINISH
 connectStorageServer return={'statuslist': [{'status': 477,
'id':
 u'461f65a9-3a81-4f3f-a46d-c5ed12520524'}]} from=::ffff:192.168.0.4,49914,
 flow_id=1435fc81, task_id=0a828d2c-d9f4-4f83-a9e9-7393159d5323 (api:52)
 29829:2018-03-21 10:46:13,830+0200 INFO  (jsonrpc/6)
 [jsonrpc.JsonRpcServer] RPC call StoragePool.connectStorageServer
 succeeded in 125.04 seconds (__init__:573)
 29830:2018-03-21 10:46:14,767+0200 INFO  (jsonrpc/4) [vdsm.api] START
 getSpmStatus(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
 options=None) from=::ffff:192.168.0.4,49914,
task_id=49cd2ae6-7943-47c9-ad86-9e0b7e58bca3
 (api:46)
 29831:2018-03-21 10:46:14,772+0200 INFO  (jsonrpc/4) [vdsm.api] FINISH
 getSpmStatus return={'spm_st': {'spmId': 1, 'spmStatus':
'SPM', 'spmLver':
 6L}} from=::ffff:192.168.0.4,49914, task_id=49cd2ae6-7943-47c9-ad86-9e0b7e58bca3
 (api:52)
 29832:2018-03-21 10:46:14,773+0200 INFO  (jsonrpc/4)
 [jsonrpc.JsonRpcServer] RPC call StoragePool.getSpmStatus succeeded in 0.01
 seconds (__init__:573)
 29833:2018-03-21 10:46:14,815+0200 INFO  (jsonrpc/2) [vdsm.api] START
 getStoragePoolInfo(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
 options=None) from=::ffff:192.168.0.4,49920,
task_id=19909616-efb4-490d-982c-66eda9ca4381
 (api:46)
 29834:2018-03-21 10:46:14,822+0200 INFO  (jsonrpc/2) [vdsm.api] FINISH
 getStoragePoolInfo return={'info': {'name': 'No Description',
'isoprefix':
 u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
 nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
 11111111-1111-1111-1111-111111111111', 'pool_status': 'connected',
 'lver': 6L, 'domains': u'ef184b28-1dbc-45ed-b0b3-
 85e780cce5d8:Active,8a2c304b-c8ae-438b-af54-fc8797ea149f:Active',
 'master_uuid': u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8',
'version': '4',
 'spm_id': 1, 'type': 'NFS', 'master_ver': 3},
'dominfo':
 {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'status': u'Active',
 'diskfree': '1868692455424', 'isoprefix': '',
'alerts': [], 'disktotal':
 '1968811540480', 'version': 4},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'status': u'Active', 'diskfree': '1868692455424',
'isoprefix':
 u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
 nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
 11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal':
 '1968811540480', 'version': 0}}} from=::ffff:192.168.0.4,49920,
 task_id=19909616-efb4-490d-982c-66eda9ca4381 (api:52)
 29835:2018-03-21 10:46:14,823+0200 INFO  (jsonrpc/2)
 [jsonrpc.JsonRpcServer] RPC call StoragePool.getInfo succeeded in 0.01
 seconds (__init__:573)
 29836:2018-03-21 10:46:15,360+0200 INFO  (periodic/2) [vdsm.api] START
 repoStats(domains=()) from=internal, task_id=da74ea9f-164b-450d-b836-6818caa3fdc5
 (api:46)
 29837:2018-03-21 10:46:15,360+0200 INFO  (periodic/2) [vdsm.api] FINISH
 repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
 'actual': True, 'version': 4, 'acquired': True, 'delay':
'0.000492277',
 'lastCheck': '5.8', 'valid': True},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '0.000479153', 'lastCheck': '5.7', 'valid': True}}
from=internal,
 task_id=da74ea9f-164b-450d-b836-6818caa3fdc5 (api:52)
 29838:2018-03-21 10:46:15,361+0200 INFO  (periodic/2) [vdsm.api] START
 multipath_health() from=internal, task_id=06245e2b-e8ca-41c4-90bf-4294d0c699b8
 (api:46)
 29839:2018-03-21 10:46:15,361+0200 INFO  (periodic/2) [vdsm.api] FINISH
 multipath_health return={} from=internal, task_id=06245e2b-e8ca-41c4-90bf-4294d0c699b8
 (api:52)
 29840:2018-03-21 10:46:20,456+0200 INFO  (jsonrpc/7) [api.host] START
 getAllVmStats() from=::ffff:192.168.0.4,49914 (api:46)
 29841:2018-03-21 10:46:20,457+0200 INFO  (jsonrpc/7) [api.host] FINISH
 getAllVmStats return={'status': {'message': 'Done',
'code': 0},
 'statsList': (suppressed)} from=::ffff:192.168.0.4,49914 (api:52)
 29842:2018-03-21 10:46:20,457+0200 INFO  (jsonrpc/7)
 [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.01
 seconds (__init__:573)
 29843:2018-03-21 10:46:21,925+0200 INFO  (jsonrpc/1) [api.host] START
 getAllVmStats() from=::1,36114 (api:46)
 29844:2018-03-21 10:46:21,926+0200 INFO  (jsonrpc/1) [api.host] FINISH
 getAllVmStats return={'status': {'message': 'Done',
'code': 0},
 'statsList': (suppressed)} from=::1,36114 (api:52)
 29845:2018-03-21 10:46:21,926+0200 INFO  (jsonrpc/1)
 [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
 seconds (__init__:573)
 29846:2018-03-21 10:46:24,930+0200 INFO  (jsonrpc/5) [vdsm.api] START
 getSpmStatus(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
 options=None) from=::ffff:192.168.0.4,49914,
task_id=d964535d-e5eb-42bc-b324-432a05c364da
 (api:46)
 29847:2018-03-21 10:46:24,935+0200 INFO  (jsonrpc/5) [vdsm.api] FINISH
 getSpmStatus return={'spm_st': {'spmId': 1, 'spmStatus':
'SPM', 'spmLver':
 6L}} from=::ffff:192.168.0.4,49914, task_id=d964535d-e5eb-42bc-b324-432a05c364da
 (api:52)
 29848:2018-03-21 10:46:24,935+0200 INFO  (jsonrpc/5)
 [jsonrpc.JsonRpcServer] RPC call StoragePool.getSpmStatus succeeded in 0.00
 seconds (__init__:573)
 29849:2018-03-21 10:46:24,987+0200 INFO  (jsonrpc/3) [vdsm.api] START
 getStoragePoolInfo(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
 options=None) from=::ffff:192.168.0.4,49920,
task_id=167dcd23-04b5-4de3-9c81-7d25ba56407a
 (api:46)
 29850:2018-03-21 10:46:25,022+0200 INFO  (jsonrpc/3) [vdsm.api] FINISH
 getStoragePoolInfo return={'info': {'name': 'No Description',
'isoprefix':
 u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
 nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
 11111111-1111-1111-1111-111111111111', 'pool_status': 'connected',
 'lver': 6L, 'domains': u'ef184b28-1dbc-45ed-b0b3-
 85e780cce5d8:Active,8a2c304b-c8ae-438b-af54-fc8797ea149f:Active',
 'master_uuid': u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8',
'version': '4',
 'spm_id': 1, 'type': 'NFS', 'master_ver': 3},
'dominfo':
 {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'status': u'Active',
 'diskfree': '1868692455424', 'isoprefix': '',
'alerts': [], 'disktotal':
 '1968811540480', 'version': 4},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'status': u'Active', 'diskfree': '1868692455424',
'isoprefix':
 u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
 nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
 11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal':
 '1968811540480', 'version': 0}}} from=::ffff:192.168.0.4,49920,
 task_id=167dcd23-04b5-4de3-9c81-7d25ba56407a (api:52)
 29851:2018-03-21 10:46:25,023+0200 INFO  (jsonrpc/3)
 [jsonrpc.JsonRpcServer] RPC call StoragePool.getInfo succeeded in 0.03
 seconds (__init__:573)
 29852:2018-03-21 10:46:28,010+0200 INFO  (jsonrpc/0) [api.host] START
 getStats() from=::ffff:192.168.0.4,49914 (api:46)
 29853:2018-03-21 10:46:28,011+0200 INFO  (jsonrpc/0) [vdsm.api] START
 repoStats(domains=()) from=::ffff:192.168.0.4,49914,
 task_id=011616ce-53c9-4cfc-8b29-247b1be03409 (api:46)
 29854:2018-03-21 10:46:28,011+0200 INFO  (jsonrpc/0) [vdsm.api] FINISH
 repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
 'actual': True, 'version': 4, 'acquired': True, 'delay':
'0.000218955',
 'lastCheck': '8.4', 'valid': True},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '0.000351741', 'lastCheck': '8.4', 'valid': True}}
 from=::ffff:192.168.0.4,49914, task_id=011616ce-53c9-4cfc-8b29-247b1be03409
 (api:52)
 29855:2018-03-21 10:46:28,012+0200 INFO  (jsonrpc/0) [vdsm.api] START
 multipath_health() from=::ffff:192.168.0.4,49914,
 task_id=85b42a8d-1bf1-495d-ad32-a1f9710a6468 (api:46)
 29856:2018-03-21 10:46:28,012+0200 INFO  (jsonrpc/0) [vdsm.api] FINISH
 multipath_health return={} from=::ffff:192.168.0.4,49914,
 task_id=85b42a8d-1bf1-495d-ad32-a1f9710a6468 (api:52)
 29857:2018-03-21 10:46:28,018+0200 INFO  (jsonrpc/0) [api.host] FINISH
 getStats return={'status': {'message': 'Done', 'code':
0}, 'info':
 {'cpuStatistics': {'11': {'cpuUser': '0.13',
'nodeIndex': 1, 'cpuSys':
 '0.20', 'cpuIdle': '99.67'}, '10': {'cpuUser':
'1.73', 'nodeIndex': 0,
 'cpuSys': '0.53', 'cpuIdle': '97.74'}, '13':
{'cpuUser': '0.20',
 'nodeIndex': 1, 'cpuSys': '0.27', 'cpuIdle':
'99.53'}, '12': {'cpuUser':
 '0.07', 'nodeIndex': 0, 'cpuSys': '0.20',
'cpuIdle': '99.73'}, '15':
 {'cpuUser': '0.67', 'nodeIndex': 1, 'cpuSys':
'0.27', 'cpuIdle': '99.06'},
 '14': {'cpuUser': '0.20', 'nodeIndex': 0,
'cpuSys': '0.33', 'cpuIdle':
 '99.47'}, '1': {'cpuUser': '1.26', 'nodeIndex':
1, 'cpuSys': '1.33',
 'cpuIdle': '97.41'}, '0': {'cpuUser': '0.40',
'nodeIndex': 0, 'cpuSys':
 '0.07', 'cpuIdle': '99.53'}, '3': {'cpuUser':
'0.07', 'nodeIndex': 1,
 'cpuSys': '0.13', 'cpuIdle': '99.80'}, '2':
{'cpuUser': '0.00',
 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle':
'99.73'}, '5': {'cpuUser':
 '0.00', 'nodeIndex': 1, 'cpuSys': '0.13',
'cpuIdle': '99.87'}, '4':
 {'cpuUser': '13.78', 'nodeIndex': 0, 'cpuSys':
'0.80', 'cpuIdle': '85.42'},
 '7': {'cpuUser': '0.47', 'nodeIndex': 1,
'cpuSys': '0.67', 'cpuIdle':
 '98.86'}, '6': {'cpuUser': '1.07', 'nodeIndex':
0, 'cpuSys': '0.27',
 'cpuIdle': '98.66'}, '9': {'cpuUser': '1.20',
'nodeIndex': 1, 'cpuSys':
 '0.27', 'cpuIdle': '98.53'}, '8': {'cpuUser':
'0.80', 'nodeIndex': 0,
 'cpuSys': '0.33', 'cpuIdle': '98.87'}},
'numaNodeMemFree': {'1':
 {'memPercent': 7, 'memFree': '15136'}, '0':
{'memPercent': 5, 'memFree':
 '15357'}}, 'memShared': 0, 'thpState': 'always',
'ksmMergeAcrossNodes':
 True, 'vmCount': 0, 'memUsed': '3', 'storageDomains':
 {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0, 'actual':
True,
 'version': 4, 'acquired': True, 'delay': '0.000218955',
'lastCheck': '8.4',
 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f': {'code':
0,
 'actual': True, 'version': 0, 'acquired': True, 'delay':
'0.000351741',
 'lastCheck': '8.4', 'valid': True}},
'incomingVmMigrations': 0, 'network':
 {'enp3s0f0': {'txErrors': '0', 'state': 'up',
'sampleTime':
 1521621975.346633, 'name': 'enp3s0f0', 'tx':
'1091397599', 'txDropped':
 '0', 'rx': '11914966591', 'rxErrors': '0',
'speed': '100', 'rxDropped':
 '1454'}, 'ovirtmgmt': {'txErrors': '0', 'state':
'up', 'sampleTime':
 1521621975.346633, 'name': 'ovirtmgmt', 'tx':
'1048745347', 'txDropped':
 '0', 'rx': '11615294897', 'rxErrors': '0',
'speed': '1000', 'rxDropped':
 '0'}, 'lo': {'txErrors': '0', 'state':
'up', 'sampleTime':
 1521621975.346633, 'name': 'lo', 'tx': '58205583384',
'txDropped': '0',
 'rx': '58205583384', 'rxErrors': '0', 'speed':
'1000', 'rxDropped': '0'},
 'enp3s0f1': {'txErrors': '0', 'state': 'up',
'sampleTime':
 1521621975.346633, 'name': 'enp3s0f1', 'tx': '0',
'txDropped': '0', 'rx':
 '66839', 'rxErrors': '0', 'speed': '1000',
'rxDropped': '0'},
 ';vdsmdummy;': {'txErrors': '0', 'state': 'down',
'sampleTime':
 1521621975.346633, 'name': ';vdsmdummy;', 'tx': '0',
'txDropped': '0',
 'rx': '0', 'rxErrors': '0', 'speed':
'1000', 'rxDropped': '0'}, 'enp4s0f0':
 {'txErrors': '0', 'state': 'down', 'sampleTime':
1521621975.346633, 'name':
 'enp4s0f0', 'tx': '0', 'txDropped': '0',
'rx': '0', 'rxErrors': '0',
 'speed': '1000', 'rxDropped': '0'}, 'enp4s0f1':
{'txErrors': '0', 'state':
 'down', 'sampleTime': 1521621975.346633, 'name':
'enp4s0f1', 'tx': '0',
 'txDropped': '0', 'rx': '0', 'rxErrors':
'0', 'speed': '1000', 'rxDropped':
 '0'}}, 'txDropped': '0', 'anonHugePages': '204',
'ksmPages': 100,
 'elapsedTime': '76546.85', 'cpuLoad': '0.18',
'cpuSys': '0.39',
 'diskStats': {'/var/log': {'free': '7344'},
'/var/run/vdsm/': {'free':
 '16060'}, '/tmp': {'free': '906'}},
'cpuUserVdsmd': '1.20',
 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState':
False,
 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 31031,
'bootTime':
 '1521545382', 'haStats': {'active': False, 'configured':
False, 'score': 0,
 'localMaintenance': False, 'globalMaintenance': False},
'momStatus':
 'active', 'multipathHealth': {}, 'rxDropped': '1454',
 'outgoingVmMigrations': 0, 'swapTotal': 12287, 'swapFree':
12287,
 'hugepages': defaultdict(<type 'dict'>, {2048:
{'resv_hugepages': 0,
 'free_hugepages': 0, 'nr_overcommit_hugepages': 0,
'surplus_hugepages': 0,
 'vm.free_hugepages': 0, 'nr_hugepages': 0,
'nr_hugepages_mempolicy': 0},
 1048576: {'resv_hugepages': 0, 'free_hugepages': 0,
 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0,
'vm.free_hugepages':
 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}),
'dateTime':
 '2018-03-21T08:46:28 GMT', 'cpuUser': '1.38', 'memFree':
31287, 'cpuIdle':
 '98.23', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd':
'0.87'}}
 from=::ffff:192.168.0.4,49914 (api:52)
 29858:2018-03-21 10:46:28,020+0200 INFO  (jsonrpc/0)
 [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds
 (__init__:573)
 29859:2018-03-21 10:46:29,667+0200 INFO  (itmap/0) [IOProcessClient]
 Starting client ioprocess-575 (__init__:308)
 29860:2018-03-21 10:46:29,700+0200 INFO  (itmap/1) [IOProcessClient]
 Starting client ioprocess-576 (__init__:308)
 29861:2018-03-21 10:46:29,714+0200 INFO  (ioprocess/1711202) [IOProcess]
 Starting ioprocess (__init__:437)
 29862:2018-03-21 10:46:29,725+0200 INFO  (ioprocess/1711208) [IOProcess]
 Starting ioprocess (__init__:437)
 29863:2018-03-21 10:46:30,375+0200 INFO  (periodic/0) [vdsm.api] START
 repoStats(domains=()) from=internal, task_id=bdacf1c0-3fcb-43b1-82e2-a041a26d0d7f
 (api:46)
 29864:2018-03-21 10:46:30,376+0200 INFO  (periodic/0) [vdsm.api] FINISH
 repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
 'actual': True, 'version': 4, 'acquired': True, 'delay':
'0.00031111',
 'lastCheck': '0.8', 'valid': True},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '0.00045594', 'lastCheck': '0.6', 'valid': True}}
from=internal,
 task_id=bdacf1c0-3fcb-43b1-82e2-a041a26d0d7f (api:52)
 29865:2018-03-21 10:46:30,376+0200 INFO  (periodic/0) [vdsm.api] START
 multipath_health() from=internal, task_id=3d3cad70-e5c8-49b2-9f5d-92a249cc102d
 (api:46)
 29866:2018-03-21 10:46:30,377+0200 INFO  (periodic/0) [vdsm.api] FINISH
 multipath_health return={} from=internal, task_id=3d3cad70-e5c8-49b2-9f5d-92a249cc102d
 (api:52)
 29867:2018-03-21 10:46:35,108+0200 INFO  (jsonrpc/6) [vdsm.api] START
 getSpmStatus(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
 options=None) from=::ffff:192.168.0.4,49914,
task_id=304d7e7c-03fc-44d6-be32-d083ada09b30
 (api:46)
 29868:2018-03-21 10:46:35,114+0200 INFO  (jsonrpc/6) [vdsm.api] FINISH
 getSpmStatus return={'spm_st': {'spmId': 1, 'spmStatus':
'SPM', 'spmLver':
 6L}} from=::ffff:192.168.0.4,49914, task_id=304d7e7c-03fc-44d6-be32-d083ada09b30
 (api:52)
 29869:2018-03-21 10:46:35,114+0200 INFO  (jsonrpc/6)
 [jsonrpc.JsonRpcServer] RPC call StoragePool.getSpmStatus succeeded in 0.00
 seconds (__init__:573)
 29870:2018-03-21 10:46:35,170+0200 INFO  (jsonrpc/4) [vdsm.api] START
 getStoragePoolInfo(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
 options=None) from=::ffff:192.168.0.4,49920,
task_id=76736a93-d4c4-4b2f-9848-7b8731fe2b67
 (api:46)
 29871:2018-03-21 10:46:35,176+0200 INFO  (jsonrpc/4) [vdsm.api] FINISH
 getStoragePoolInfo return={'info': {'name': 'No Description',
'isoprefix':
 u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
 nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
 11111111-1111-1111-1111-111111111111', 'pool_status': 'connected',
 'lver': 6L, 'domains': u'ef184b28-1dbc-45ed-b0b3-
 85e780cce5d8:Active,8a2c304b-c8ae-438b-af54-fc8797ea149f:Active',
 'master_uuid': u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8',
'version': '4',
 'spm_id': 1, 'type': 'NFS', 'master_ver': 3},
'dominfo':
 {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'status': u'Active',
 'diskfree': '1868692455424', 'isoprefix': '',
'alerts': [], 'disktotal':
 '1968811540480', 'version': 4},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'status': u'Active', 'diskfree': '1868692455424',
'isoprefix':
 u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
 nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
 11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal':
 '1968811540480', 'version': 0}}} from=::ffff:192.168.0.4,49920,
 task_id=76736a93-d4c4-4b2f-9848-7b8731fe2b67 (api:52)
 29872:2018-03-21 10:46:35,177+0200 INFO  (jsonrpc/4)
 [jsonrpc.JsonRpcServer] RPC call StoragePool.getInfo succeeded in 0.01
 seconds (__init__:573)
 29873:2018-03-21 10:46:35,462+0200 INFO  (jsonrpc/2) [api.host] START
 getAllVmStats() from=::ffff:192.168.0.4,49914 (api:46)
 29874:2018-03-21 10:46:35,462+0200 INFO  (jsonrpc/2) [api.host] FINISH
 getAllVmStats return={'status': {'message': 'Done',
'code': 0},
 'statsList': (suppressed)} from=::ffff:192.168.0.4,49914 (api:52)
 29875:2018-03-21 10:46:35,463+0200 INFO  (jsonrpc/2)
 [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
 seconds (__init__:573)
 29876:2018-03-21 10:46:36,948+0200 INFO  (jsonrpc/7) [api.host] START
 getAllVmStats() from=::1,36114 (api:46)
 29877:2018-03-21 10:46:36,948+0200 INFO  (jsonrpc/7) [api.host] FINISH
 getAllVmStats return={'status': {'message': 'Done',
'code': 0},
 'statsList': (suppressed)} from=::1,36114 (api:52)
 29878:2018-03-21 10:46:36,949+0200 INFO  (jsonrpc/7)
 [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
 seconds (__init__:573)
 29879:2018-03-21 10:46:43,179+0200 INFO  (jsonrpc/1) [api.host] START
 getStats() from=::ffff:192.168.0.4,49914 (api:46)
 29880:2018-03-21 10:46:43,180+0200 INFO  (jsonrpc/1) [vdsm.api] START
 repoStats(domains=()) from=::ffff:192.168.0.4,49914,
 task_id=83bbc95c-6d59-4eec-82b1-e8993be28759 (api:46)
 29881:2018-03-21 10:46:43,181+0200 INFO  (jsonrpc/1) [vdsm.api] FINISH
 repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
 'actual': True, 'version': 4, 'acquired': True, 'delay':
'0.000486101',
 'lastCheck': '3.6', 'valid': True},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '0.00043444', 'lastCheck': '3.4', 'valid': True}}
 from=::ffff:192.168.0.4,49914, task_id=83bbc95c-6d59-4eec-82b1-e8993be28759
 (api:52)
 29882:2018-03-21 10:46:43,181+0200 INFO  (jsonrpc/1) [vdsm.api] START
 multipath_health() from=::ffff:192.168.0.4,49914,
 task_id=9b781324-b50b-49a9-bf31-5eca835dc75c (api:46)
 29883:2018-03-21 10:46:43,181+0200 INFO  (jsonrpc/1) [vdsm.api] FINISH
 multipath_health return={} from=::ffff:192.168.0.4,49914,
 task_id=9b781324-b50b-49a9-bf31-5eca835dc75c (api:52)
 29884:2018-03-21 10:46:43,188+0200 INFO  (jsonrpc/1) [api.host] FINISH
 getStats return={'status': {'message': 'Done', 'code':
0}, 'info':
 {'cpuStatistics': {'11': {'cpuUser': '0.40',
'nodeIndex': 1, 'cpuSys':
 '0.27', 'cpuIdle': '99.33'}, '10': {'cpuUser':
'0.60', 'nodeIndex': 0,
 'cpuSys': '0.47', 'cpuIdle': '98.93'}, '13':
{'cpuUser': '0.00',
 'nodeIndex': 1, 'cpuSys': '0.27', 'cpuIdle':
'99.73'}, '12': {'cpuUser':
 '1.46', 'nodeIndex': 0, 'cpuSys': '0.60',
'cpuIdle': '97.94'}, '15':
 {'cpuUser': '0.20', 'nodeIndex': 1, 'cpuSys':
'0.33', 'cpuIdle': '99.47'},
 '14': {'cpuUser': '0.40', 'nodeIndex': 0,
'cpuSys': '0.47', 'cpuIdle':
 '99.13'}, '1': {'cpuUser': '1.33', 'nodeIndex':
1, 'cpuSys': '1.93',
 'cpuIdle': '96.74'}, '0': {'cpuUser': '0.60',
'nodeIndex': 0, 'cpuSys':
 '0.27', 'cpuIdle': '99.13'}, '3': {'cpuUser':
'0.20', 'nodeIndex': 1,
 'cpuSys': '0.40', 'cpuIdle': '99.40'}, '2':
{'cpuUser': '6.99',
 'nodeIndex': 0, 'cpuSys': '0.40', 'cpuIdle':
'92.61'}, '5': {'cpuUser':
 '0.13', 'nodeIndex': 1, 'cpuSys': '0.20',
'cpuIdle': '99.67'}, '4':
 {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys':
'0.27', 'cpuIdle': '99.46'},
 '7': {'cpuUser': '0.47', 'nodeIndex': 1,
'cpuSys': '0.60', 'cpuIdle':
 '98.93'}, '6': {'cpuUser': '0.53', 'nodeIndex':
0, 'cpuSys': '0.27',
 'cpuIdle': '99.20'}, '9': {'cpuUser': '0.47',
'nodeIndex': 1, 'cpuSys':
 '0.33', 'cpuIdle': '99.20'}, '8': {'cpuUser':
'0.07', 'nodeIndex': 0,
 'cpuSys': '0.13', 'cpuIdle': '99.80'}},
'numaNodeMemFree': {'1':
 {'memPercent': 7, 'memFree': '15131'}, '0':
{'memPercent': 5, 'memFree':
 '15357'}}, 'memShared': 0, 'thpState': 'always',
'ksmMergeAcrossNodes':
 True, 'vmCount': 0, 'memUsed': '3', 'storageDomains':
 {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0, 'actual':
True,
 'version': 4, 'acquired': True, 'delay': '0.000486101',
'lastCheck': '3.6',
 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f': {'code':
0,
 'actual': True, 'version': 0, 'acquired': True, 'delay':
'0.00043444',
 'lastCheck': '3.4', 'valid': True}},
'incomingVmMigrations': 0, 'network':
 {'enp3s0f0': {'txErrors': '0', 'state': 'up',
'sampleTime':
 1521621990.363726, 'name': 'enp3s0f0', 'tx':
'1091404888', 'txDropped':
 '0', 'rx': '11914970667', 'rxErrors': '0',
'speed': '100', 'rxDropped':
 '1455'}, 'ovirtmgmt': {'txErrors': '0', 'state':
'up', 'sampleTime':
 1521621990.363726, 'name': 'ovirtmgmt', 'tx':
'1048752406', 'txDropped':
 '0', 'rx': '11615298347', 'rxErrors': '0',
'speed': '1000', 'rxDropped':
 '0'}, 'lo': {'txErrors': '0', 'state':
'up', 'sampleTime':
 1521621990.363726, 'name': 'lo', 'tx': '58215969455',
'txDropped': '0',
 'rx': '58215969455', 'rxErrors': '0', 'speed':
'1000', 'rxDropped': '0'},
 'enp3s0f1': {'txErrors': '0', 'state': 'up',
'sampleTime':
 1521621990.363726, 'name': 'enp3s0f1', 'tx': '0',
'txDropped': '0', 'rx':
 '66839', 'rxErrors': '0', 'speed': '1000',
'rxDropped': '0'},
 ';vdsmdummy;': {'txErrors': '0', 'state': 'down',
'sampleTime':
 1521621990.363726, 'name': ';vdsmdummy;', 'tx': '0',
'txDropped': '0',
 'rx': '0', 'rxErrors': '0', 'speed':
'1000', 'rxDropped': '0'}, 'enp4s0f0':
 {'txErrors': '0', 'state': 'down', 'sampleTime':
1521621990.363726, 'name':
 'enp4s0f0', 'tx': '0', 'txDropped': '0',
'rx': '0', 'rxErrors': '0',
 'speed': '1000', 'rxDropped': '0'}, 'enp4s0f1':
{'txErrors': '0', 'state':
 'down', 'sampleTime': 1521621990.363726, 'name':
'enp4s0f1', 'tx': '0',
 'txDropped': '0', 'rx': '0', 'rxErrors':
'0', 'speed': '1000', 'rxDropped':
 '0'}}, 'txDropped': '0', 'anonHugePages': '206',
'ksmPages': 100,
 'elapsedTime': '76562.02', 'cpuLoad': '0.18',
'cpuSys': '0.45',
 'diskStats': {'/var/log': {'free': '7344'},
'/var/run/vdsm/': {'free':
 '16060'}, '/tmp': {'free': '906'}},
'cpuUserVdsmd': '1.27',
 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState':
False,
 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 31080,
'bootTime':
 '1521545382', 'haStats': {'active': False, 'configured':
False, 'score': 0,
 'localMaintenance': False, 'globalMaintenance': False},
'momStatus':
 'active', 'multipathHealth': {}, 'rxDropped': '1455',
 'outgoingVmMigrations': 0, 'swapTotal': 12287, 'swapFree':
12287,
 'hugepages': defaultdict(<type 'dict'>, {2048:
{'resv_hugepages': 0,
 'free_hugepages': 0, 'nr_overcommit_hugepages': 0,
'surplus_hugepages': 0,
 'vm.free_hugepages': 0, 'nr_hugepages': 0,
'nr_hugepages_mempolicy': 0},
 1048576: {'resv_hugepages': 0, 'free_hugepages': 0,
 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0,
'vm.free_hugepages':
 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}),
'dateTime':
 '2018-03-21T08:46:43 GMT', 'cpuUser': '0.87', 'memFree':
31336, 'cpuIdle':
 '98.68', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd':
'1.20'}}
 from=::ffff:192.168.0.4,49914 (api:52)
 29885:2018-03-21 10:46:43,190+0200 INFO  (jsonrpc/1)
 [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds
 (__init__:573)
 29886:2018-03-21 10:46:45,288+0200 INFO  (jsonrpc/5) [vdsm.api] START
 getSpmStatus(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
 options=None) from=::ffff:192.168.0.4,49914,
task_id=ffe57349-2532-4b64-b32c-8a558273d8e5
 (api:46)
 29887:2018-03-21 10:46:45,292+0200 INFO  (jsonrpc/5) [vdsm.api] FINISH
 getSpmStatus return={'spm_st': {'spmId': 1, 'spmStatus':
'SPM', 'spmLver':
 6L}} from=::ffff:192.168.0.4,49914, task_id=ffe57349-2532-4b64-b32c-8a558273d8e5
 (api:52)
 29888:2018-03-21 10:46:45,293+0200 INFO  (jsonrpc/5)
 [jsonrpc.JsonRpcServer] RPC call StoragePool.getSpmStatus succeeded in 0.00
 seconds (__init__:573)
 29889:2018-03-21 10:46:45,298+0200 INFO  (jsonrpc/3) [vdsm.api] START
 getStoragePoolInfo(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
 options=None) from=::ffff:192.168.0.4,49920,
task_id=b2ea28f0-abf0-4258-8152-6035d93aa5dc
 (api:46)
 29890:2018-03-21 10:46:45,305+0200 INFO  (jsonrpc/3) [vdsm.api] FINISH
 getStoragePoolInfo return={'info': {'name': 'No Description',
'isoprefix':
 u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
 nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
 11111111-1111-1111-1111-111111111111', 'pool_status': 'connected',
 'lver': 6L, 'domains': u'ef184b28-1dbc-45ed-b0b3-
 85e780cce5d8:Active,8a2c304b-c8ae-438b-af54-fc8797ea149f:Active',
 'master_uuid': u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8',
'version': '4',
 'spm_id': 1, 'type': 'NFS', 'master_ver': 3},
'dominfo':
 {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'status': u'Active',
 'diskfree': '1868692455424', 'isoprefix': '',
'alerts': [], 'disktotal':
 '1968811540480', 'version': 4},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'status': u'Active', 'diskfree': '1868692455424',
'isoprefix':
 u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
 nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
 11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal':
 '1968811540480', 'version': 0}}} from=::ffff:192.168.0.4,49920,
 task_id=b2ea28f0-abf0-4258-8152-6035d93aa5dc (api:52)
 29891:2018-03-21 10:46:45,306+0200 INFO  (jsonrpc/3)
 [jsonrpc.JsonRpcServer] RPC call StoragePool.getInfo succeeded in 0.00
 seconds (__init__:573)
 29892:2018-03-21 10:46:45,395+0200 INFO  (periodic/2) [vdsm.api] START
 repoStats(domains=()) from=internal, task_id=4010bf72-5c86-4daf-a8f9-12ec229a86e4
 (api:46)
 29893:2018-03-21 10:46:45,395+0200 INFO  (periodic/2) [vdsm.api] FINISH
 repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
 'actual': True, 'version': 4, 'acquired': True, 'delay':
'0.000486101',
 'lastCheck': '5.8', 'valid': True},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '0.00043444', 'lastCheck': '5.7', 'valid': True}}
from=internal,
 task_id=4010bf72-5c86-4daf-a8f9-12ec229a86e4 (api:52)
 29894:2018-03-21 10:46:45,396+0200 INFO  (periodic/2) [vdsm.api] START
 multipath_health() from=internal, task_id=37a1132f-452b-4805-9cc0-b8e74e8d6b02
 (api:46)
 29895:2018-03-21 10:46:45,396+0200 INFO  (periodic/2) [vdsm.api] FINISH
 multipath_health return={} from=internal, task_id=37a1132f-452b-4805-9cc0-b8e74e8d6b02
 (api:52)
 29896:2018-03-21 10:46:50,467+0200 INFO  (jsonrpc/0) [api.host] START
 getAllVmStats() from=::ffff:192.168.0.4,49914 (api:46)
 29897:2018-03-21 10:46:50,468+0200 INFO  (jsonrpc/0) [throttled] Current
 getAllVmStats: {} (throttledlog:103)
 29898:2018-03-21 10:46:50,468+0200 INFO  (jsonrpc/0) [api.host] FINISH
 getAllVmStats return={'status': {'message': 'Done',
'code': 0},
 'statsList': (suppressed)} from=::ffff:192.168.0.4,49914 (api:52)
 29899:2018-03-21 10:46:50,468+0200 INFO  (jsonrpc/0)
 [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.01
 seconds (__init__:573)
 29900:2018-03-21 10:46:51,967+0200 INFO  (jsonrpc/6) [api.host] START
 getAllVmStats() from=::1,36114 (api:46)
 29901:2018-03-21 10:46:51,968+0200 INFO  (jsonrpc/6) [api.host] FINISH
 getAllVmStats return={'status': {'message': 'Done',
'code': 0},
 'statsList': (suppressed)} from=::1,36114 (api:52)
 29902:2018-03-21 10:46:51,968+0200 INFO  (jsonrpc/6)
 [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00
 seconds (__init__:573)
 29903:2018-03-21 10:46:55,648+0200 INFO  (jsonrpc/4) [vdsm.api] START
 getSpmStatus(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
 options=None) from=::ffff:192.168.0.4,49914,
task_id=a975635f-59d7-4f40-b9a7-a5ea7150368e
 (api:46)
 29904:2018-03-21 10:46:55,653+0200 INFO  (jsonrpc/4) [vdsm.api] FINISH
 getSpmStatus return={'spm_st': {'spmId': 1, 'spmStatus':
'SPM', 'spmLver':
 6L}} from=::ffff:192.168.0.4,49914, task_id=a975635f-59d7-4f40-b9a7-a5ea7150368e
 (api:52)
 29905:2018-03-21 10:46:55,653+0200 INFO  (jsonrpc/4)
 [jsonrpc.JsonRpcServer] RPC call StoragePool.getSpmStatus succeeded in 0.00
 seconds (__init__:573)
 29906:2018-03-21 10:46:55,659+0200 INFO  (jsonrpc/2) [vdsm.api] START
 getStoragePoolInfo(spUUID=u'80cc922f-8dea-4fed-b951-1060ba116ad5',
 options=None) from=::ffff:192.168.0.4,49920,
task_id=97d4ae6c-3719-4149-85fd-28e924ad8883
 (api:46)
 29907:2018-03-21 10:46:55,666+0200 INFO  (jsonrpc/2) [vdsm.api] FINISH
 getStoragePoolInfo return={'info': {'name': 'No Description',
'isoprefix':
 u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
 nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
 11111111-1111-1111-1111-111111111111', 'pool_status': 'connected',
 'lver': 6L, 'domains': u'ef184b28-1dbc-45ed-b0b3-
 85e780cce5d8:Active,8a2c304b-c8ae-438b-af54-fc8797ea149f:Active',
 'master_uuid': u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8',
'version': '4',
 'spm_id': 1, 'type': 'NFS', 'master_ver': 3},
'dominfo':
 {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'status': u'Active',
 'diskfree': '1868692455424', 'isoprefix': '',
'alerts': [], 'disktotal':
 '1968811540480', 'version': 4},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'status': u'Active', 'diskfree': '1868692455424',
'isoprefix':
 u'/rhev/data-center/mnt/node11.starlett.lv:_vmraid_
 nfs_iso/8a2c304b-c8ae-438b-af54-fc8797ea149f/images/
 11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal':
 '1968811540480', 'version': 0}}} from=::ffff:192.168.0.4,49920,
 task_id=97d4ae6c-3719-4149-85fd-28e924ad8883 (api:52)
 29908:2018-03-21 10:46:55,667+0200 INFO  (jsonrpc/2)
 [jsonrpc.JsonRpcServer] RPC call StoragePool.getInfo succeeded in 0.00
 seconds (__init__:573)
 29909:2018-03-21 10:46:59,473+0200 INFO  (jsonrpc/7) [api.host] START
 getStats() from=::ffff:192.168.0.4,49914 (api:46)
 29910:2018-03-21 10:46:59,474+0200 INFO  (jsonrpc/7) [vdsm.api] START
 repoStats(domains=()) from=::ffff:192.168.0.4,49914,
 task_id=177b551b-9bd3-45a4-84d0-5244630c08e5 (api:46)
 29911:2018-03-21 10:46:59,474+0200 INFO  (jsonrpc/7) [vdsm.api] FINISH
 repoStats return={u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0,
 'actual': True, 'version': 4, 'acquired': True, 'delay':
'0.000470906',
 'lastCheck': '9.9', 'valid': True},
u'8a2c304b-c8ae-438b-af54-fc8797ea149f':
 {'code': 0, 'actual': True, 'version': 0, 'acquired':
True, 'delay':
 '0.000334928', 'lastCheck': '9.7', 'valid': True}}
 from=::ffff:192.168.0.4,49914, task_id=177b551b-9bd3-45a4-84d0-5244630c08e5
 (api:52)
 29912:2018-03-21 10:46:59,475+0200 INFO  (jsonrpc/7) [vdsm.api] START
 multipath_health() from=::ffff:192.168.0.4,49914,
 task_id=30753d5c-9bfd-4b0f-ac83-e32278d82b63 (api:46)
 29913:2018-03-21 10:46:59,475+0200 INFO  (jsonrpc/7) [vdsm.api] FINISH
 multipath_health return={} from=::ffff:192.168.0.4,49914,
 task_id=30753d5c-9bfd-4b0f-ac83-e32278d82b63 (api:52)
 29914:2018-03-21 10:46:59,482+0200 INFO  (jsonrpc/7) [api.host] FINISH
 getStats return={'status': {'message': 'Done', 'code':
0}, 'info':
 {'cpuStatistics': {'11': {'cpuUser': '0.07',
'nodeIndex': 1, 'cpuSys':
 '0.13', 'cpuIdle': '99.80'}, '10': {'cpuUser':
'0.07', 'nodeIndex': 0,
 'cpuSys': '0.13', 'cpuIdle': '99.80'}, '13':
{'cpuUser': '0.07',
 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle':
'99.86'}, '12': {'cpuUser':
 '0.00', 'nodeIndex': 0, 'cpuSys': '0.07',
'cpuIdle': '99.93'}, '15':
 {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys':
'0.13', 'cpuIdle': '99.87'},
 '14': {'cpuUser': '0.07', 'nodeIndex': 0,
'cpuSys': '0.20', 'cpuIdle':
 '99.73'}, '1': {'cpuUser': '1.07', 'nodeIndex':
1, 'cpuSys': '1.07',
 'cpuIdle': '97.86'}, '0': {'cpuUser': '0.07',
'nodeIndex': 0, 'cpuSys':
 '0.07', 'cpuIdle': '99.86'}, '3': {'cpuUser':
'0.07', 'nodeIndex': 1,
 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '2':
{'cpuUser': '0.00',
 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle':
'99.87'}, '5': {'cpuUser':
 '0.07', 'nodeIndex': 1, 'cpuSys': '0.13',
'cpuIdle': '99.80'}, '4':
 {'cpuUser': '0.47', 'nodeIndex': 0, 'cpuSys':
'0.27', 'cpuIdle': '99.26'},
 '7': {'cpuUser': '0.00', 'nodeIndex': 1,
'cpuSys': '0.47', 'cpuIdle':
 '99.53'}, '6': {'cpuUser': '0.20', 'nodeIndex':
0, 'cpuSys': '0.07',
 'cpuIdle': '99.73'}, '9': {'cpuUser': '0.07',
'nodeIndex': 1, 'cpuSys':
 '0.13', 'cpuIdle': '99.80'}, '8': {'cpuUser':
'0.00', 'nodeIndex': 0,
 'cpuSys': '0.07', 'cpuIdle': '99.93'}},
'numaNodeMemFree': {'1':
 {'memPercent': 7, 'memFree': '15125'}, '0':
{'memPercent': 5, 'memFree':
 '15364'}}, 'memShared': 0, 'thpState': 'always',
'ksmMergeAcrossNodes':
 True, 'vmCount': 0, 'memUsed': '3', 'storageDomains':
 {u'ef184b28-1dbc-45ed-b0b3-85e780cce5d8': {'code': 0, 'actual':
True,
 'version': 4, 'acquired': True, 'delay': '0.000470906',
'lastCheck': '9.9',
 'valid': True}, u'8a2c304b-c8ae-438b-af54-fc8797ea149f': {'code':
0,
 'actual': True, 'version': 0, 'acquired': True, 'delay':
'0.000334928',
 'lastCheck': '9.7', 'valid': True}},
'incomingVmMigrations': 0, 'network':
 {'enp3s0f0': {'txErrors': '0', 'state': 'up',
'sampleTime':
 1521622005.383289, 'name': 'enp3s0f0', 'tx':
'1091412332', 'txDropped':
 '0', 'rx': '11914976557', 'rxErrors': '0',
'speed': '100', 'rxDropped':
 '1456'}, 'ovirtmgmt': {'txErrors': '0', 'state':
'up', 'sampleTime':
 1521622005.383289, 'name': 'ovirtmgmt', 'tx':
'1048761326', 'txDropped':
 '0', 'rx': '11615304242', 'rxErrors': '0',
'speed': '1000', 'rxDropped':
 '0'}, 'lo': {'txErrors': '0', 'state':
'up', 'sampleTime':
 1521622005.383289, 'name': 'lo', 'tx': '58228460774',
'txDropped': '0',
 'rx': '58228460774', 'rxErrors': '0', 'speed':
'1000', 'rxDropped': '0'},
 'enp3s0f1': {'txErrors': '0', 'state': 'up',
'sampleTime':
 1521622005.383289, 'name': 'enp3s0f1', 'tx': '0',
'txDropped': '0', 'rx':
 '66839', 'rxErrors': '0', 'speed': '1000',
'rxDropped': '0'},
 ';vdsmdummy;': {'txErrors': '0', 'state': 'down',
'sampleTime':
 1521622005.383289, 'name': ';vdsmdummy;', 'tx': '0',
'txDropped': '0',
 'rx': '0', 'rxErrors': '0', 'speed':
'1000', 'rxDropped': '0'}, 'enp4s0f0':
 {'txErrors': '0', 'state': 'down', 'sampleTime':
1521622005.383289, 'name':
 'enp4s0f0', 'tx': '0', 'txDropped': '0',
'rx': '0', 'rxErrors': '0',
 'speed': '1000', 'rxDropped': '0'}, 'enp4s0f1':
{'txErrors': '0', 'state':
 'down', 'sampleTime': 1521622005.383289, 'name':
'enp4s0f1', 'tx': '0',
 'txDropped': '0', 'rx': '0', 'rxErrors':
'0', 'speed': '1000', 'rxDropped':
 '0'}}, 'txDropped': '0', 'anonHugePages': '208',
'ksmPages': 100,
 'elapsedTime': '76578.31', 'cpuLoad': '0.17',
'cpuSys': '0.20',
 'diskStats': {'/var/log': {'free': '7344'},
'/var/run/vdsm/': {'free':
 '16060'}, '/tmp': {'free': '906'}},
'cpuUserVdsmd': '1.13',
 'netConfigDirty': 'False', 'memCommitted': 0, 'ksmState':
False,
 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 31078,
'bootTime':
 '1521545382', 'haStats': {'active': False, 'configured':
False, 'score': 0,
 'localMaintenance': False, 'globalMaintenance': False},
'momStatus':
 'active', 'multipathHealth': {}, 'rxDropped': '1456',
 'outgoingVmMigrations': 0, 'swapTotal': 12287, 'swapFree':
12287,
 'hugepages': defaultdict(<type 'dict'>, {2048:
{'resv_hugepages': 0,
 'free_hugepages': 0, 'nr_overcommit_hugepages': 0,
'surplus_hugepages': 0,
 'vm.free_hugepages': 0, 'nr_hugepages': 0,
'nr_hugepages_mempolicy': 0},
 1048576: {'resv_hugepages': 0, 'free_hugepages': 0,
 'nr_overcommit_hugepages': 0, 'surplus_hugepages': 0,
'vm.free_hugepages':
 0, 'nr_hugepages': 0, 'nr_hugepages_mempolicy': 0}}),
'dateTime':
 '2018-03-21T08:46:59 GMT', 'cpuUser': '0.15', 'memFree':
31334, 'cpuIdle':
 '99.65', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd':
'0.60'}}
 from=::ffff:192.168.0.4,49914 (api:52)
 29915:2018-03-21 10:46:59,484+0200 INFO  (jsonrpc/7)
 [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds
 (__init__:573)

 On 21 Mar 2018, at 11:33, Fred Rolland <frolland(a)redhat.com&gt; wrote:

 Can you provide the vdsm logs from the host.
 It looks the vdsm failed to connect to the server.

 On Wed, Mar 21, 2018 at 11:21 AM, Andrei Verovski <andreil1(a)starlett.lv&gt;
 wrote:

> Hi,
>
> I have 2-host oVirt setup with 2 Data Centers, one with local storage
> domain (DC #1) for VMs + Export domain on NFS, another with all NFS shared
> (DC #2).
> Trying to export VMs from DC #1 to DC #2.
> VMs are exported to DC #1 export domain (NFS), then domain put into
> maintenance mode and detached from DC #1.
>
> Unfortunately, attaching it to DC #2 failed. Logs attached. Tried to run
> this command twice.
> Workaround are possible in order to accomplish this task, yet it would be
> better to do in a way as it was designed.
> Thanks.
>
>
> 2018-03-21 10:43:24,024+02 ERROR [org.ovirt.engine.core.dal.dbb
> roker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engine-Thread-128904)
> [570ec5d9-fff5-4656-afbd-90b3207a616e] EVENT_ID:
> USER_ATTACH_STORAGE_DOMAIN_TO_POOL_FAILED(963), Failed to attach Storage
> Domain node10-NFS-EXPORTS to Data Center StrDataCenter11. (User:
> admin@internal-authz)
> 2018-03-21 10:46:16,512+02 ERROR [org.ovirt.engine.core.dal.dbb
> roker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engine-Thread-128956)
> [1435fc81] EVENT_ID: VDS_STORAGES_CONNECTION_FAILED(188), Failed to
> connect Host node11 to the Storage Domains node10-NFS-EXPORTS.
> 2018-03-21 10:46:16,651+02 ERROR [org.ovirt.engine.core.dal.dbb
> roker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engine-Thread-128955)
> [a81ffa4a-5a58-41a0-888a-f0edc321609b] EVENT_ID:
> USER_ATTACH_STORAGE_DOMAIN_TO_POOL_FAILED(963), Failed to attach Storage
> Domain node10-NFS-EXPORTS to Data Center StrDataCenter11. (User:
> admin@internal-authz)
>
>
> *tail -n 1000 engine.log | grep 570ec5d9-fff5-4656-afbd-90b3207a616e*
> 2018-03-21 10:41:14,643+02 INFO  [org.ovirt.engine.core.bll.sto
> rage.domain.AttachStorageDomainToPoolCommand] (default task-2)
> [570ec5d9-fff5-4656-afbd-90b3207a616e] Lock Acquired to object
>
'EngineLock:{exclusiveLocks='[1d7208ce-d3a1-4406-9638-fe7051562994=STORAGE]',
> sharedLocks=''}'
> 2018-03-21 10:41:16,129+02 INFO  [org.ovirt.engine.core.bll.sto
> rage.domain.AttachStorageDomainToPoolCommand]
> (EE-ManagedThreadFactory-engine-Thread-128904)
> [570ec5d9-fff5-4656-afbd-90b3207a616e] Running command:
> AttachStorageDomainToPoolCommand internal: false. Entities affected :
> ID: 1d7208ce-d3a1-4406-9638-fe7051562994 Type: StorageAction group
> MANIPULATE_STORAGE_DOMAIN with role type ADMIN,  ID:
> 80cc922f-8dea-4fed-b951-1060ba116ad5 Type: StoragePoolAction group
> MANIPULATE_STORAGE_DOMAIN with role type ADMIN
> 2018-03-21 10:43:23,564+02 ERROR [org.ovirt.engine.core.bll.sto
> rage.domain.AttachStorageDomainToPoolCommand]
> (EE-ManagedThreadFactory-engine-Thread-128904)
> [570ec5d9-fff5-4656-afbd-90b3207a616e] Cannot connect storage connection
> server, aborting attach storage domain operation.
> 2018-03-21 10:43:23,567+02 INFO  [org.ovirt.engine.core.bll.CommandCompensator]
> (EE-ManagedThreadFactory-engine-Thread-128904)
> [570ec5d9-fff5-4656-afbd-90b3207a616e] Command
> [id=921ca7cd-4f93-46aa-8de2-91b13b8f96cb]: Compensating NEW_ENTITY_ID of
> org.ovirt.engine.core.common.businessentities.StoragePoolIsoMap;
> snapshot:
StoragePoolIsoMapId:{storagePoolId='80cc922f-8dea-4fed-b951-1060ba116ad5',
> storageId='1d7208ce-d3a1-4406-9638-fe7051562994'}.
> 2018-03-21 10:43:24,024+02 ERROR [org.ovirt.engine.core.dal.dbb
> roker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engine-Thread-128904)
> [570ec5d9-fff5-4656-afbd-90b3207a616e] EVENT_ID:
> USER_ATTACH_STORAGE_DOMAIN_TO_POOL_FAILED(963), Failed to attach Storage
> Domain node10-NFS-EXPORTS to Data Center StrDataCenter11. (User:
> admin@internal-authz)
> 2018-03-21 10:43:24,114+02 INFO  [org.ovirt.engine.core.bll.sto
> rage.domain.AttachStorageDomainToPoolCommand]
> (EE-ManagedThreadFactory-engine-Thread-128904)
> [570ec5d9-fff5-4656-afbd-90b3207a616e] Lock freed to object
>
'EngineLock:{exclusiveLocks='[1d7208ce-d3a1-4406-9638-fe7051562994=STORAGE]',
> sharedLocks=''}'
>
>
>
> *[root@node00 ovirt-engine]# tail -n 1000 engine.log | grep
> a81ffa4a-5a58-41a0-888a-f0edc321609b*
> 2018-03-21 10:44:11,025+02 INFO  [org.ovirt.engine.core.bll.sto
> rage.domain.AttachStorageDomainToPoolCommand] (default task-16)
> [a81ffa4a-5a58-41a0-888a-f0edc321609b] Lock Acquired to object
>
'EngineLock:{exclusiveLocks='[1d7208ce-d3a1-4406-9638-fe7051562994=STORAGE]',
> sharedLocks=''}'
> 2018-03-21 10:44:11,236+02 INFO  [org.ovirt.engine.core.bll.sto
> rage.domain.AttachStorageDomainToPoolCommand]
> (EE-ManagedThreadFactory-engine-Thread-128955)
> [a81ffa4a-5a58-41a0-888a-f0edc321609b] Running command:
> AttachStorageDomainToPoolCommand internal: false. Entities affected :
> ID: 1d7208ce-d3a1-4406-9638-fe7051562994 Type: StorageAction group
> MANIPULATE_STORAGE_DOMAIN with role type ADMIN,  ID:
> 80cc922f-8dea-4fed-b951-1060ba116ad5 Type: StoragePoolAction group
> MANIPULATE_STORAGE_DOMAIN with role type ADMIN
> 2018-03-21 10:46:16,567+02 ERROR [org.ovirt.engine.core.bll.sto
> rage.domain.AttachStorageDomainToPoolCommand]
> (EE-ManagedThreadFactory-engine-Thread-128955)
> [a81ffa4a-5a58-41a0-888a-f0edc321609b] Cannot connect storage connection
> server, aborting attach storage domain operation.
> 2018-03-21 10:46:16,568+02 INFO  [org.ovirt.engine.core.bll.CommandCompensator]
> (EE-ManagedThreadFactory-engine-Thread-128955)
> [a81ffa4a-5a58-41a0-888a-f0edc321609b] Command
> [id=b5c25100-1a8a-4db0-9509-99cfa60995b1]: Compensating NEW_ENTITY_ID of
> org.ovirt.engine.core.common.businessentities.StoragePoolIsoMap;
> snapshot:
StoragePoolIsoMapId:{storagePoolId='80cc922f-8dea-4fed-b951-1060ba116ad5',
> storageId='1d7208ce-d3a1-4406-9638-fe7051562994'}.
> 2018-03-21 10:46:16,651+02 ERROR [org.ovirt.engine.core.dal.dbb
> roker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engine-Thread-128955)
> [a81ffa4a-5a58-41a0-888a-f0edc321609b] EVENT_ID:
> USER_ATTACH_STORAGE_DOMAIN_TO_POOL_FAILED(963), Failed to attach Storage
> Domain node10-NFS-EXPORTS to Data Center StrDataCenter11. (User:
> admin@internal-authz)
> 2018-03-21 10:46:16,681+02 INFO  [org.ovirt.engine.core.bll.sto
> rage.domain.AttachStorageDomainToPoolCommand]
> (EE-ManagedThreadFactory-engine-Thread-128955)
> [a81ffa4a-5a58-41a0-888a-f0edc321609b] Lock freed to object
>
'EngineLock:{exclusiveLocks='[1d7208ce-d3a1-4406-9638-fe7051562994=STORAGE]',
> sharedLocks='’}'
>
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

Re: [ovirt-users] Attach Export Domain to another Data Center - Failure