Thread-12::DEBUG::2016-11-04 14:01:04,263::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:01:04,313::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:01:04,314::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000392948 s, 1.5 MB/s\n') elapsed=0.06 Thread-12::DEBUG::2016-11-04 14:01:04,344::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:01:04,355::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:01:04,371::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000384935 s, 917 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:01:04,373::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:01:04,395::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000428481 s, 803 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:01:04,443::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:01:04,443::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000645057 s, 532 kB/s\n') elapsed=0.07 jsonrpc.Executor/4::DEBUG::2016-11-04 14:01:04,996::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getAllVmStats' in bridge with {} jsonrpc.Executor/4::DEBUG::2016-11-04 14:01:04,998::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getAllVmStats' in bridge with (suppressed) jsonrpc.Executor/4::INFO::2016-11-04 14:01:04,999::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getAllVmStats succeeded in 0.00 seconds jsonrpc.Executor/5::DEBUG::2016-11-04 14:01:05,152::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'VM.getMigrationStatus' in bridge with {u'vmID': u'cddd2019-5baf-4321-aa8b-130b2889d1b0'} jsonrpc.Executor/5::DEBUG::2016-11-04 14:01:05,153::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'VM.getMigrationStatus' in bridge with {'status': {'message': 'Fatal error during migration', 'code': 12}, 'progress': 0} jsonrpc.Executor/5::INFO::2016-11-04 14:01:05,154::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call VM.getMigrationStatus succeeded in 0.01 seconds periodic/0::DEBUG::2016-11-04 14:01:06,595::task::597::Storage.TaskManager.Task::(_updateState) Task=`a0325f46-14ea-4111-b36a-ab3c0ddef971`::moving from state init -> state preparing periodic/0::INFO::2016-11-04 14:01:06,595::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) periodic/0::INFO::2016-11-04 14:01:06,595::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000384935', 'lastCheck': '2.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000645057', 'lastCheck': '2.2', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000428481', 'lastCheck': '2.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000392948', 'lastCheck': '2.3', 'valid': True}} periodic/0::DEBUG::2016-11-04 14:01:06,595::task::1193::Storage.TaskManager.Task::(prepare) Task=`a0325f46-14ea-4111-b36a-ab3c0ddef971`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000384935', 'lastCheck': '2.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000645057', 'lastCheck': '2.2', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000428481', 'lastCheck': '2.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000392948', 'lastCheck': '2.3', 'valid': True}} periodic/0::DEBUG::2016-11-04 14:01:06,595::task::597::Storage.TaskManager.Task::(_updateState) Task=`a0325f46-14ea-4111-b36a-ab3c0ddef971`::moving from state preparing -> state finished periodic/0::DEBUG::2016-11-04 14:01:06,595::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/0::DEBUG::2016-11-04 14:01:06,596::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/0::DEBUG::2016-11-04 14:01:06,596::task::995::Storage.TaskManager.Task::(_decref) Task=`a0325f46-14ea-4111-b36a-ab3c0ddef971`::ref 0 aborting False Reactor thread::INFO::2016-11-04 14:01:13,890::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58188 Reactor thread::DEBUG::2016-11-04 14:01:13,899::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:01:13,899::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58188 Reactor thread::DEBUG::2016-11-04 14:01:13,899::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58188) BindingXMLRPC::INFO::2016-11-04 14:01:13,899::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58188 Thread-1030::INFO::2016-11-04 14:01:13,900::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58188 started Thread-1030::DEBUG::2016-11-04 14:01:13,901::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call getAllVmStats with () {} Thread-1030::DEBUG::2016-11-04 14:01:13,902::bindingxmlrpc::1255::vds::(wrapper) return getAllVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} Thread-1030::INFO::2016-11-04 14:01:13,902::bindingxmlrpc::1282::vds::(wrapper) RPC call getAllVmStats finished (code=0) in 0.00 seconds Thread-1030::INFO::2016-11-04 14:01:13,908::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58188 stopped Reactor thread::INFO::2016-11-04 14:01:13,909::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58190 Reactor thread::DEBUG::2016-11-04 14:01:13,915::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:01:13,915::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58190 Reactor thread::DEBUG::2016-11-04 14:01:13,915::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58190) BindingXMLRPC::INFO::2016-11-04 14:01:13,915::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58190 Thread-1031::INFO::2016-11-04 14:01:13,916::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58190 started Thread-1031::DEBUG::2016-11-04 14:01:13,916::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call vmGetIoTunePolicy with ('cddd2019-5baf-4321-aa8b-130b2889d1b0',) {} Thread-1031::DEBUG::2016-11-04 14:01:13,917::bindingxmlrpc::1255::vds::(wrapper) return vmGetIoTunePolicy with {'ioTunePolicy': [], 'status': {'message': 'Done', 'code': 0}} Thread-1031::INFO::2016-11-04 14:01:13,917::bindingxmlrpc::1282::vds::(wrapper) RPC call vmGetIoTunePolicy finished (code=0) in 0.00 seconds Thread-1031::INFO::2016-11-04 14:01:13,918::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58190 stopped Thread-12::DEBUG::2016-11-04 14:01:14,263::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:01:14,313::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:01:14,313::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000343793 s, 1.8 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:01:14,344::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:01:14,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:01:14,370::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000466087 s, 757 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:01:14,372::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:01:14,391::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000464873 s, 740 kB/s\n') elapsed=0.03 Thread-12::DEBUG::2016-11-04 14:01:14,402::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000528587 s, 649 kB/s\n') elapsed=0.03 jsonrpc.Executor/6::DEBUG::2016-11-04 14:01:14,663::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getStats' in bridge with {} jsonrpc.Executor/6::DEBUG::2016-11-04 14:01:14,664::task::597::Storage.TaskManager.Task::(_updateState) Task=`f12f80a4-faea-4b0e-b45a-ed9b45d920fc`::moving from state init -> state preparing jsonrpc.Executor/6::INFO::2016-11-04 14:01:14,664::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) jsonrpc.Executor/6::INFO::2016-11-04 14:01:14,664::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000466087', 'lastCheck': '0.3', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000528587', 'lastCheck': '0.3', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000464873', 'lastCheck': '0.3', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000343793', 'lastCheck': '0.4', 'valid': True}} jsonrpc.Executor/6::DEBUG::2016-11-04 14:01:14,664::task::1193::Storage.TaskManager.Task::(prepare) Task=`f12f80a4-faea-4b0e-b45a-ed9b45d920fc`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000466087', 'lastCheck': '0.3', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000528587', 'lastCheck': '0.3', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000464873', 'lastCheck': '0.3', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000343793', 'lastCheck': '0.4', 'valid': True}} jsonrpc.Executor/6::DEBUG::2016-11-04 14:01:14,665::task::597::Storage.TaskManager.Task::(_updateState) Task=`f12f80a4-faea-4b0e-b45a-ed9b45d920fc`::moving from state preparing -> state finished jsonrpc.Executor/6::DEBUG::2016-11-04 14:01:14,665::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} jsonrpc.Executor/6::DEBUG::2016-11-04 14:01:14,665::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} jsonrpc.Executor/6::DEBUG::2016-11-04 14:01:14,665::task::995::Storage.TaskManager.Task::(_decref) Task=`f12f80a4-faea-4b0e-b45a-ed9b45d920fc`::ref 0 aborting False jsonrpc.Executor/6::DEBUG::2016-11-04 14:01:14,672::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getStats' in bridge with {'cpuStatistics': {'20': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '21': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '22': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '23': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '1': {'cpuUser': '14.87', 'nodeIndex': 0, 'cpuSys': '1.20', 'cpuIdle': '83.93'}, '0': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '3': {'cpuUser': '0.20', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.67'}, '2': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '5': {'cpuUser': '31.88', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '67.85'}, '4': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '7': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '6': {'cpuUser': '0.40', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.53'}, '9': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '8': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '11': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '10': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '13': {'cpuUser': '0.47', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '99.26'}, '12': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '15': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '14': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '17': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '16': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '19': {'cpuUser': '0.33', 'nodeIndex': 1, 'cpuSys': '0.20', 'cpuIdle': '99.47'}, '18': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31505'}, '0': {'memPercent': 5, 'memFree': '70581'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'rxRate': '0.05', 'vmCount': 1, 'memUsed': '3', 'storageDomains': {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000466087', 'lastCheck': '0.3', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000528587', 'lastCheck': '0.3', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000464873', 'lastCheck': '0.3', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000343793', 'lastCheck': '0.4', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'eno1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '11618', 'name': 'eno1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286066.527091, 'rx': '5200015', 'state': 'up'}, 'ovirtmgmt': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.5', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '31260771', 'txDropped': '0', 'sampleTime': 1478286066.527091, 'rx': '711049672', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '6470074', 'txDropped': '0', 'sampleTime': 1478286066.527091, 'rx': '6470074', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286066.527091, 'rx': '0', 'state': 'down'}, 'eno2': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno2', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286066.527091, 'rx': '0', 'state': 'down'}, 'vnet0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'vnet0', 'tx': '355718', 'txDropped': '0', 'sampleTime': 1478286066.527091, 'rx': '98322', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286066.527091, 'rx': '0', 'state': 'down'}, 'vdsmbr_AoB33GL7': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '4585', 'name': 'vdsmbr_AoB33GL7', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286066.527091, 'rx': '0', 'state': 'down'}, 'eno3': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno3', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286066.527091, 'rx': '0', 'state': 'down'}, 'eno4': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno4', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286066.527091, 'rx': '0', 'state': 'down'}, 'ens2f0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.1', 'txErrors': '0', 'speed': '10000', 'rxDropped': '0', 'name': 'ens2f0', 'tx': '31969908', 'txDropped': '0', 'sampleTime': 1478286066.527091, 'rx': '739049324', 'state': 'up'}, 'ens2f1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ens2f1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286066.527091, 'rx': '0', 'state': 'down'}}, 'txDropped': '0', 'cpuUser': '2.03', 'ksmPages': 100, 'elapsedTime': '6158.22', 'cpuLoad': '0.03', 'cpuSys': '0.12', 'diskStats': {'/var/log': {'free': '15167'}, '/var/log/core': {'free': '15167'}, '/var/run/vdsm/': {'free': '52254'}, '/tmp': {'free': '1640028'}}, 'cpuUserVdsmd': '14.80', 'netConfigDirty': 'False', 'memCommitted': 4161, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 98632, 'txRate': '0.00', 'bootTime': '1478279889', 'haStats': {'active': False, 'configured': False, 'score': 0, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'rxDropped': '16203', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'dateTime': '2016-11-04T19:01:14 GMT', 'anonHugePages': '996', 'memFree': 102311, 'cpuIdle': '97.85', 'vmActive': 1, 'v2vJobs': {}, 'cpuSysVdsmd': '0.73'} jsonrpc.Executor/6::INFO::2016-11-04 14:01:14,673::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getStats succeeded in 0.02 seconds jsonrpc.Executor/0::DEBUG::2016-11-04 14:01:20,145::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getAllVmStats' in bridge with {} jsonrpc.Executor/0::DEBUG::2016-11-04 14:01:20,146::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getAllVmStats' in bridge with (suppressed) jsonrpc.Executor/0::INFO::2016-11-04 14:01:20,146::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getAllVmStats succeeded in 0.00 seconds periodic/1::DEBUG::2016-11-04 14:01:21,595::task::597::Storage.TaskManager.Task::(_updateState) Task=`a721ad19-b931-4bdc-bbd4-1631d12186fc`::moving from state init -> state preparing periodic/1::INFO::2016-11-04 14:01:21,596::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) periodic/1::INFO::2016-11-04 14:01:21,596::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000466087', 'lastCheck': '4.1', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000528587', 'lastCheck': '4.1', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000464873', 'lastCheck': '4.5', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000343793', 'lastCheck': '4.4', 'valid': True}} periodic/1::DEBUG::2016-11-04 14:01:21,596::task::1193::Storage.TaskManager.Task::(prepare) Task=`a721ad19-b931-4bdc-bbd4-1631d12186fc`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000466087', 'lastCheck': '4.1', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000528587', 'lastCheck': '4.1', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000464873', 'lastCheck': '4.5', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000343793', 'lastCheck': '4.4', 'valid': True}} periodic/1::DEBUG::2016-11-04 14:01:21,596::task::597::Storage.TaskManager.Task::(_updateState) Task=`a721ad19-b931-4bdc-bbd4-1631d12186fc`::moving from state preparing -> state finished periodic/1::DEBUG::2016-11-04 14:01:21,596::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/1::DEBUG::2016-11-04 14:01:21,596::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/1::DEBUG::2016-11-04 14:01:21,596::task::995::Storage.TaskManager.Task::(_decref) Task=`a721ad19-b931-4bdc-bbd4-1631d12186fc`::ref 0 aborting False Thread-12::DEBUG::2016-11-04 14:01:24,273::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:01:24,324::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:01:24,324::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000401493 s, 1.5 MB/s\n') elapsed=0.06 Thread-12::DEBUG::2016-11-04 14:01:24,344::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:01:24,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:01:24,373::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000448793 s, 787 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:01:24,374::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:01:24,396::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000502695 s, 684 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:01:24,407::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000497434 s, 690 kB/s\n') elapsed=0.03 Reactor thread::INFO::2016-11-04 14:01:28,950::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58192 Reactor thread::DEBUG::2016-11-04 14:01:28,956::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:01:28,956::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58192 Reactor thread::DEBUG::2016-11-04 14:01:28,956::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58192) BindingXMLRPC::INFO::2016-11-04 14:01:28,957::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58192 Thread-1032::INFO::2016-11-04 14:01:28,957::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58192 started Thread-1032::DEBUG::2016-11-04 14:01:28,958::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call getAllVmStats with () {} Thread-1032::DEBUG::2016-11-04 14:01:28,959::bindingxmlrpc::1255::vds::(wrapper) return getAllVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} Thread-1032::INFO::2016-11-04 14:01:28,959::bindingxmlrpc::1282::vds::(wrapper) RPC call getAllVmStats finished (code=0) in 0.00 seconds Thread-1032::INFO::2016-11-04 14:01:28,964::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58192 stopped Reactor thread::INFO::2016-11-04 14:01:28,965::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58194 Reactor thread::DEBUG::2016-11-04 14:01:28,971::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:01:28,972::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58194 Reactor thread::DEBUG::2016-11-04 14:01:28,972::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58194) BindingXMLRPC::INFO::2016-11-04 14:01:28,972::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58194 Thread-1033::INFO::2016-11-04 14:01:28,972::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58194 started Thread-1033::DEBUG::2016-11-04 14:01:28,973::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call vmGetIoTunePolicy with ('cddd2019-5baf-4321-aa8b-130b2889d1b0',) {} Thread-1033::DEBUG::2016-11-04 14:01:28,973::bindingxmlrpc::1255::vds::(wrapper) return vmGetIoTunePolicy with {'ioTunePolicy': [], 'status': {'message': 'Done', 'code': 0}} Thread-1033::INFO::2016-11-04 14:01:28,973::bindingxmlrpc::1282::vds::(wrapper) RPC call vmGetIoTunePolicy finished (code=0) in 0.01 seconds Thread-1033::INFO::2016-11-04 14:01:28,974::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58194 stopped jsonrpc.Executor/2::DEBUG::2016-11-04 14:01:30,949::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getStats' in bridge with {} jsonrpc.Executor/2::DEBUG::2016-11-04 14:01:30,950::task::597::Storage.TaskManager.Task::(_updateState) Task=`bfeb9f55-aca1-4aba-8e7b-01edd96c8754`::moving from state init -> state preparing jsonrpc.Executor/2::INFO::2016-11-04 14:01:30,951::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) jsonrpc.Executor/2::INFO::2016-11-04 14:01:30,951::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000448793', 'lastCheck': '3.5', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000497434', 'lastCheck': '3.5', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000502695', 'lastCheck': '3.8', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000401493', 'lastCheck': '3.8', 'valid': True}} jsonrpc.Executor/2::DEBUG::2016-11-04 14:01:30,951::task::1193::Storage.TaskManager.Task::(prepare) Task=`bfeb9f55-aca1-4aba-8e7b-01edd96c8754`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000448793', 'lastCheck': '3.5', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000497434', 'lastCheck': '3.5', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000502695', 'lastCheck': '3.8', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000401493', 'lastCheck': '3.8', 'valid': True}} jsonrpc.Executor/2::DEBUG::2016-11-04 14:01:30,951::task::597::Storage.TaskManager.Task::(_updateState) Task=`bfeb9f55-aca1-4aba-8e7b-01edd96c8754`::moving from state preparing -> state finished jsonrpc.Executor/2::DEBUG::2016-11-04 14:01:30,951::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} jsonrpc.Executor/2::DEBUG::2016-11-04 14:01:30,951::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} jsonrpc.Executor/2::DEBUG::2016-11-04 14:01:30,951::task::995::Storage.TaskManager.Task::(_decref) Task=`bfeb9f55-aca1-4aba-8e7b-01edd96c8754`::ref 0 aborting False jsonrpc.Executor/2::DEBUG::2016-11-04 14:01:30,958::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getStats' in bridge with {'cpuStatistics': {'20': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '21': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '22': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '23': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '1': {'cpuUser': '0.47', 'nodeIndex': 0, 'cpuSys': '0.60', 'cpuIdle': '98.93'}, '0': {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.66'}, '3': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.87'}, '2': {'cpuUser': '0.20', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.67'}, '5': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '4': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '7': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '6': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '9': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '8': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '13': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.40'}, '12': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '15': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '14': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '17': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '16': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '19': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '18': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31507'}, '0': {'memPercent': 5, 'memFree': '70603'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'rxRate': '0.01', 'vmCount': 1, 'memUsed': '3', 'storageDomains': {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000448793', 'lastCheck': '3.5', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000497434', 'lastCheck': '3.5', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000502695', 'lastCheck': '3.8', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000401493', 'lastCheck': '3.8', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'eno1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '11646', 'name': 'eno1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286081.532163, 'rx': '5212521', 'state': 'up'}, 'ovirtmgmt': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.1', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '31306047', 'txDropped': '0', 'sampleTime': 1478286081.532163, 'rx': '712111796', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '6488320', 'txDropped': '0', 'sampleTime': 1478286081.532163, 'rx': '6488320', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286081.532163, 'rx': '0', 'state': 'down'}, 'eno2': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno2', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286081.532163, 'rx': '0', 'state': 'down'}, 'vnet0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'vnet0', 'tx': '357062', 'txDropped': '0', 'sampleTime': 1478286081.532163, 'rx': '98322', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286081.532163, 'rx': '0', 'state': 'down'}, 'vdsmbr_AoB33GL7': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '4599', 'name': 'vdsmbr_AoB33GL7', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286081.532163, 'rx': '0', 'state': 'down'}, 'eno3': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno3', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286081.532163, 'rx': '0', 'state': 'down'}, 'eno4': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno4', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286081.532163, 'rx': '0', 'state': 'down'}, 'ens2f0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '10000', 'rxDropped': '0', 'name': 'ens2f0', 'tx': '32016379', 'txDropped': '0', 'sampleTime': 1478286081.532163, 'rx': '740158117', 'state': 'up'}, 'ens2f1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ens2f1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286081.532163, 'rx': '0', 'state': 'down'}}, 'txDropped': '0', 'cpuUser': '0.06', 'ksmPages': 100, 'elapsedTime': '6174.50', 'cpuLoad': '0.03', 'cpuSys': '0.06', 'diskStats': {'/var/log': {'free': '15189'}, '/var/log/core': {'free': '15189'}, '/var/run/vdsm/': {'free': '52254'}, '/tmp': {'free': '1640028'}}, 'cpuUserVdsmd': '0.33', 'netConfigDirty': 'False', 'memCommitted': 4161, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 98633, 'txRate': '0.00', 'bootTime': '1478279889', 'haStats': {'active': False, 'configured': False, 'score': 0, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'rxDropped': '16245', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'dateTime': '2016-11-04T19:01:30 GMT', 'anonHugePages': '904', 'memFree': 102314, 'cpuIdle': '99.88', 'vmActive': 1, 'v2vJobs': {}, 'cpuSysVdsmd': '0.40'} jsonrpc.Executor/2::INFO::2016-11-04 14:01:30,959::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getStats succeeded in 0.01 seconds Thread-12::DEBUG::2016-11-04 14:01:34,268::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:01:34,314::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:01:34,314::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000450809 s, 1.3 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:01:34,344::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:01:34,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:01:34,372::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000452594 s, 780 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:01:34,374::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:01:34,398::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000507122 s, 678 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:01:34,444::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:01:34,444::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000582472 s, 589 kB/s\n') elapsed=0.07 jsonrpc.Executor/7::DEBUG::2016-11-04 14:01:35,289::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getAllVmStats' in bridge with {} jsonrpc.Executor/7::DEBUG::2016-11-04 14:01:35,290::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getAllVmStats' in bridge with (suppressed) jsonrpc.Executor/7::INFO::2016-11-04 14:01:35,291::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getAllVmStats succeeded in 0.00 seconds periodic/0::DEBUG::2016-11-04 14:01:36,457::task::597::Storage.TaskManager.Task::(_updateState) Task=`3ddd90cf-b1c6-4954-a371-6c307473922d`::moving from state init -> state preparing periodic/0::INFO::2016-11-04 14:01:36,457::logUtils::49::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'bfcabdd8-3aa7-4241-91c3-5db0e843e476', spUUID=u'00000001-0001-0001-0001-000000000366', imgUUID=u'b2109025-42be-4c40-ac2b-26baf8c304e5', volUUID=u'3153e04b-729c-44b2-b31a-0b2547ad3a77', options=None) periodic/0::INFO::2016-11-04 14:01:36,459::logUtils::52::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '17503478272', 'apparentsize': '21474836480'} periodic/0::DEBUG::2016-11-04 14:01:36,460::task::1193::Storage.TaskManager.Task::(prepare) Task=`3ddd90cf-b1c6-4954-a371-6c307473922d`::finished: {'truesize': '17503478272', 'apparentsize': '21474836480'} periodic/0::DEBUG::2016-11-04 14:01:36,460::task::597::Storage.TaskManager.Task::(_updateState) Task=`3ddd90cf-b1c6-4954-a371-6c307473922d`::moving from state preparing -> state finished periodic/0::DEBUG::2016-11-04 14:01:36,460::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/0::DEBUG::2016-11-04 14:01:36,460::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/0::DEBUG::2016-11-04 14:01:36,460::task::995::Storage.TaskManager.Task::(_decref) Task=`3ddd90cf-b1c6-4954-a371-6c307473922d`::ref 0 aborting False periodic/1::DEBUG::2016-11-04 14:01:36,593::task::597::Storage.TaskManager.Task::(_updateState) Task=`c4341b15-a154-4f18-840c-983c43323c97`::moving from state init -> state preparing periodic/1::INFO::2016-11-04 14:01:36,593::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) periodic/1::INFO::2016-11-04 14:01:36,594::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000452594', 'lastCheck': '2.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000582472', 'lastCheck': '2.1', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000507122', 'lastCheck': '2.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000450809', 'lastCheck': '2.3', 'valid': True}} periodic/1::DEBUG::2016-11-04 14:01:36,594::task::1193::Storage.TaskManager.Task::(prepare) Task=`c4341b15-a154-4f18-840c-983c43323c97`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000452594', 'lastCheck': '2.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000582472', 'lastCheck': '2.1', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000507122', 'lastCheck': '2.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000450809', 'lastCheck': '2.3', 'valid': True}} periodic/1::DEBUG::2016-11-04 14:01:36,594::task::597::Storage.TaskManager.Task::(_updateState) Task=`c4341b15-a154-4f18-840c-983c43323c97`::moving from state preparing -> state finished periodic/1::DEBUG::2016-11-04 14:01:36,594::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/1::DEBUG::2016-11-04 14:01:36,594::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/1::DEBUG::2016-11-04 14:01:36,594::task::995::Storage.TaskManager.Task::(_decref) Task=`c4341b15-a154-4f18-840c-983c43323c97`::ref 0 aborting False Reactor thread::INFO::2016-11-04 14:01:43,990::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58196 Reactor thread::DEBUG::2016-11-04 14:01:43,996::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:01:43,996::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58196 Reactor thread::DEBUG::2016-11-04 14:01:43,997::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58196) BindingXMLRPC::INFO::2016-11-04 14:01:43,997::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58196 Thread-1034::INFO::2016-11-04 14:01:43,997::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58196 started Thread-1034::DEBUG::2016-11-04 14:01:43,998::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call getAllVmStats with () {} Thread-1034::DEBUG::2016-11-04 14:01:43,998::bindingxmlrpc::1255::vds::(wrapper) return getAllVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} Thread-1034::INFO::2016-11-04 14:01:43,999::bindingxmlrpc::1282::vds::(wrapper) RPC call getAllVmStats finished (code=0) in 0.00 seconds Thread-1034::INFO::2016-11-04 14:01:44,004::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58196 stopped Reactor thread::INFO::2016-11-04 14:01:44,005::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58198 Reactor thread::DEBUG::2016-11-04 14:01:44,011::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:01:44,011::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58198 Reactor thread::DEBUG::2016-11-04 14:01:44,012::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58198) BindingXMLRPC::INFO::2016-11-04 14:01:44,012::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58198 Thread-1035::INFO::2016-11-04 14:01:44,012::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58198 started Thread-1035::DEBUG::2016-11-04 14:01:44,012::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call vmGetIoTunePolicy with ('cddd2019-5baf-4321-aa8b-130b2889d1b0',) {} Thread-1035::DEBUG::2016-11-04 14:01:44,013::bindingxmlrpc::1255::vds::(wrapper) return vmGetIoTunePolicy with {'ioTunePolicy': [], 'status': {'message': 'Done', 'code': 0}} Thread-1035::INFO::2016-11-04 14:01:44,013::bindingxmlrpc::1282::vds::(wrapper) RPC call vmGetIoTunePolicy finished (code=0) in 0.01 seconds Thread-1035::INFO::2016-11-04 14:01:44,014::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58198 stopped Thread-12::DEBUG::2016-11-04 14:01:44,264::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:01:44,314::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:01:44,314::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.00046194 s, 1.3 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:01:44,345::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:01:44,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:01:44,371::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000421672 s, 837 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:01:44,375::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:01:44,397::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000492752 s, 698 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:01:44,407::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000470782 s, 729 kB/s\n') elapsed=0.03 jsonrpc.Executor/1::DEBUG::2016-11-04 14:01:46,288::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getStats' in bridge with {} jsonrpc.Executor/1::DEBUG::2016-11-04 14:01:46,290::task::597::Storage.TaskManager.Task::(_updateState) Task=`8bbada21-e20b-44fc-a753-55ea26b6ec73`::moving from state init -> state preparing jsonrpc.Executor/1::INFO::2016-11-04 14:01:46,290::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) jsonrpc.Executor/1::INFO::2016-11-04 14:01:46,290::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000421672', 'lastCheck': '1.9', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000470782', 'lastCheck': '1.9', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000492752', 'lastCheck': '1.9', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00046194', 'lastCheck': '2.0', 'valid': True}} jsonrpc.Executor/1::DEBUG::2016-11-04 14:01:46,290::task::1193::Storage.TaskManager.Task::(prepare) Task=`8bbada21-e20b-44fc-a753-55ea26b6ec73`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000421672', 'lastCheck': '1.9', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000470782', 'lastCheck': '1.9', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000492752', 'lastCheck': '1.9', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00046194', 'lastCheck': '2.0', 'valid': True}} jsonrpc.Executor/1::DEBUG::2016-11-04 14:01:46,290::task::597::Storage.TaskManager.Task::(_updateState) Task=`8bbada21-e20b-44fc-a753-55ea26b6ec73`::moving from state preparing -> state finished jsonrpc.Executor/1::DEBUG::2016-11-04 14:01:46,290::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} jsonrpc.Executor/1::DEBUG::2016-11-04 14:01:46,291::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} jsonrpc.Executor/1::DEBUG::2016-11-04 14:01:46,291::task::995::Storage.TaskManager.Task::(_decref) Task=`8bbada21-e20b-44fc-a753-55ea26b6ec73`::ref 0 aborting False jsonrpc.Executor/1::DEBUG::2016-11-04 14:01:46,298::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getStats' in bridge with {'cpuStatistics': {'20': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '21': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '22': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '23': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '1': {'cpuUser': '0.47', 'nodeIndex': 0, 'cpuSys': '1.00', 'cpuIdle': '98.53'}, '0': {'cpuUser': '0.33', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.60'}, '3': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '2': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '5': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '4': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '7': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '6': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '9': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '8': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '13': {'cpuUser': '0.47', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.33'}, '12': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '15': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '14': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '17': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '16': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '19': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '18': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31507'}, '0': {'memPercent': 5, 'memFree': '70603'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'rxRate': '0.01', 'vmCount': 1, 'memUsed': '3', 'storageDomains': {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000421672', 'lastCheck': '1.9', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000470782', 'lastCheck': '1.9', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000492752', 'lastCheck': '1.9', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00046194', 'lastCheck': '2.0', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'eno1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '11674', 'name': 'eno1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286096.527162, 'rx': '5226514', 'state': 'up'}, 'ovirtmgmt': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.1', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '31337837', 'txDropped': '0', 'sampleTime': 1478286096.527162, 'rx': '713181306', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '6506534', 'txDropped': '0', 'sampleTime': 1478286096.527162, 'rx': '6506534', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286096.527162, 'rx': '0', 'state': 'down'}, 'eno2': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno2', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286096.527162, 'rx': '0', 'state': 'down'}, 'vnet0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'vnet0', 'tx': '358210', 'txDropped': '0', 'sampleTime': 1478286096.527162, 'rx': '98322', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286096.527162, 'rx': '0', 'state': 'down'}, 'vdsmbr_AoB33GL7': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '4615', 'name': 'vdsmbr_AoB33GL7', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286096.527162, 'rx': '0', 'state': 'down'}, 'eno3': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno3', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286096.527162, 'rx': '0', 'state': 'down'}, 'eno4': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno4', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286096.527162, 'rx': '0', 'state': 'down'}, 'ens2f0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '10000', 'rxDropped': '0', 'name': 'ens2f0', 'tx': '32048763', 'txDropped': '0', 'sampleTime': 1478286096.527162, 'rx': '741267804', 'state': 'up'}, 'ens2f1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ens2f1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286096.527162, 'rx': '0', 'state': 'down'}}, 'txDropped': '0', 'cpuUser': '0.07', 'ksmPages': 100, 'elapsedTime': '6189.84', 'cpuLoad': '0.03', 'cpuSys': '0.07', 'diskStats': {'/var/log': {'free': '15189'}, '/var/log/core': {'free': '15189'}, '/var/run/vdsm/': {'free': '52254'}, '/tmp': {'free': '1640028'}}, 'cpuUserVdsmd': '0.40', 'netConfigDirty': 'False', 'memCommitted': 4161, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 98634, 'txRate': '0.00', 'bootTime': '1478279889', 'haStats': {'active': False, 'configured': False, 'score': 0, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'rxDropped': '16289', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'dateTime': '2016-11-04T19:01:46 GMT', 'anonHugePages': '904', 'memFree': 102314, 'cpuIdle': '99.87', 'vmActive': 1, 'v2vJobs': {}, 'cpuSysVdsmd': '0.67'} jsonrpc.Executor/1::INFO::2016-11-04 14:01:46,299::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getStats succeeded in 0.01 seconds jsonrpc.Executor/3::DEBUG::2016-11-04 14:01:50,399::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getAllVmStats' in bridge with {} jsonrpc.Executor/3::DEBUG::2016-11-04 14:01:50,401::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getAllVmStats' in bridge with (suppressed) jsonrpc.Executor/3::INFO::2016-11-04 14:01:50,401::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getAllVmStats succeeded in 0.00 seconds periodic/2::DEBUG::2016-11-04 14:01:51,598::task::597::Storage.TaskManager.Task::(_updateState) Task=`3e4a79ca-31ef-44fe-9723-16b80ec7a03b`::moving from state init -> state preparing periodic/2::INFO::2016-11-04 14:01:51,598::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) periodic/2::INFO::2016-11-04 14:01:51,598::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000421672', 'lastCheck': '4.1', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000470782', 'lastCheck': '4.1', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000492752', 'lastCheck': '4.5', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00046194', 'lastCheck': '4.4', 'valid': True}} periodic/2::DEBUG::2016-11-04 14:01:51,598::task::1193::Storage.TaskManager.Task::(prepare) Task=`3e4a79ca-31ef-44fe-9723-16b80ec7a03b`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000421672', 'lastCheck': '4.1', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000470782', 'lastCheck': '4.1', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000492752', 'lastCheck': '4.5', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00046194', 'lastCheck': '4.4', 'valid': True}} periodic/2::DEBUG::2016-11-04 14:01:51,598::task::597::Storage.TaskManager.Task::(_updateState) Task=`3e4a79ca-31ef-44fe-9723-16b80ec7a03b`::moving from state preparing -> state finished periodic/2::DEBUG::2016-11-04 14:01:51,599::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/2::DEBUG::2016-11-04 14:01:51,599::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/2::DEBUG::2016-11-04 14:01:51,599::task::995::Storage.TaskManager.Task::(_decref) Task=`3e4a79ca-31ef-44fe-9723-16b80ec7a03b`::ref 0 aborting False Thread-12::DEBUG::2016-11-04 14:01:54,268::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:01:54,314::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:01:54,314::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000365011 s, 1.7 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:01:54,344::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:01:54,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:01:54,372::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000479562 s, 736 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:01:54,373::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:01:54,397::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.00053837 s, 639 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:01:54,407::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000458762 s, 748 kB/s\n') elapsed=0.03 Reactor thread::INFO::2016-11-04 14:01:59,046::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58200 Reactor thread::DEBUG::2016-11-04 14:01:59,052::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:01:59,052::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58200 Reactor thread::DEBUG::2016-11-04 14:01:59,053::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58200) BindingXMLRPC::INFO::2016-11-04 14:01:59,053::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58200 Thread-1036::INFO::2016-11-04 14:01:59,053::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58200 started Thread-1036::DEBUG::2016-11-04 14:01:59,054::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call getAllVmStats with () {} Thread-1036::DEBUG::2016-11-04 14:01:59,055::bindingxmlrpc::1255::vds::(wrapper) return getAllVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} Thread-1036::INFO::2016-11-04 14:01:59,055::bindingxmlrpc::1282::vds::(wrapper) RPC call getAllVmStats finished (code=0) in 0.00 seconds Thread-1036::INFO::2016-11-04 14:01:59,061::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58200 stopped Reactor thread::INFO::2016-11-04 14:01:59,062::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58202 Reactor thread::DEBUG::2016-11-04 14:01:59,068::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:01:59,068::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58202 Reactor thread::DEBUG::2016-11-04 14:01:59,068::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58202) BindingXMLRPC::INFO::2016-11-04 14:01:59,069::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58202 Thread-1037::INFO::2016-11-04 14:01:59,069::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58202 started Thread-1037::DEBUG::2016-11-04 14:01:59,069::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call vmGetIoTunePolicy with ('cddd2019-5baf-4321-aa8b-130b2889d1b0',) {} Thread-1037::DEBUG::2016-11-04 14:01:59,070::bindingxmlrpc::1255::vds::(wrapper) return vmGetIoTunePolicy with {'ioTunePolicy': [], 'status': {'message': 'Done', 'code': 0}} Thread-1037::INFO::2016-11-04 14:01:59,070::bindingxmlrpc::1282::vds::(wrapper) RPC call vmGetIoTunePolicy finished (code=0) in 0.00 seconds Thread-1037::INFO::2016-11-04 14:01:59,071::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58202 stopped jsonrpc.Executor/4::DEBUG::2016-11-04 14:02:02,536::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getStats' in bridge with {} jsonrpc.Executor/4::DEBUG::2016-11-04 14:02:02,537::task::597::Storage.TaskManager.Task::(_updateState) Task=`cb52dcff-3446-48bd-adfe-8601b1823b3d`::moving from state init -> state preparing jsonrpc.Executor/4::INFO::2016-11-04 14:02:02,537::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) jsonrpc.Executor/4::INFO::2016-11-04 14:02:02,537::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000479562', 'lastCheck': '5.0', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000458762', 'lastCheck': '5.0', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.00053837', 'lastCheck': '5.4', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000365011', 'lastCheck': '5.4', 'valid': True}} jsonrpc.Executor/4::DEBUG::2016-11-04 14:02:02,537::task::1193::Storage.TaskManager.Task::(prepare) Task=`cb52dcff-3446-48bd-adfe-8601b1823b3d`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000479562', 'lastCheck': '5.0', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000458762', 'lastCheck': '5.0', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.00053837', 'lastCheck': '5.4', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000365011', 'lastCheck': '5.4', 'valid': True}} jsonrpc.Executor/4::DEBUG::2016-11-04 14:02:02,538::task::597::Storage.TaskManager.Task::(_updateState) Task=`cb52dcff-3446-48bd-adfe-8601b1823b3d`::moving from state preparing -> state finished jsonrpc.Executor/4::DEBUG::2016-11-04 14:02:02,538::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} jsonrpc.Executor/4::DEBUG::2016-11-04 14:02:02,538::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} jsonrpc.Executor/4::DEBUG::2016-11-04 14:02:02,538::task::995::Storage.TaskManager.Task::(_decref) Task=`cb52dcff-3446-48bd-adfe-8601b1823b3d`::ref 0 aborting False jsonrpc.Executor/4::DEBUG::2016-11-04 14:02:02,545::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getStats' in bridge with {'cpuStatistics': {'20': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '21': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '22': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '23': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '1': {'cpuUser': '0.67', 'nodeIndex': 0, 'cpuSys': '0.53', 'cpuIdle': '98.80'}, '0': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '3': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '2': {'cpuUser': '0.60', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.33'}, '5': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '4': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '7': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '6': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '9': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '8': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.13', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '13': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.47'}, '12': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '15': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '14': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '17': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '16': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '19': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '18': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31488'}, '0': {'memPercent': 5, 'memFree': '70621'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'rxRate': '0.07', 'vmCount': 1, 'memUsed': '3', 'storageDomains': {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000479562', 'lastCheck': '5.0', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000458762', 'lastCheck': '5.0', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.00053837', 'lastCheck': '5.4', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000365011', 'lastCheck': '5.4', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'eno1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '11702', 'name': 'eno1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286111.53191, 'rx': '5239661', 'state': 'up'}, 'ovirtmgmt': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.7', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '31500811', 'txDropped': '0', 'sampleTime': 1478286111.53191, 'rx': '725805765', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '6524748', 'txDropped': '0', 'sampleTime': 1478286111.53191, 'rx': '6524748', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286111.53191, 'rx': '0', 'state': 'down'}, 'eno2': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno2', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286111.53191, 'rx': '0', 'state': 'down'}, 'vnet0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'vnet0', 'tx': '361115', 'txDropped': '0', 'sampleTime': 1478286111.53191, 'rx': '100590', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286111.53191, 'rx': '0', 'state': 'down'}, 'vdsmbr_AoB33GL7': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '4624', 'name': 'vdsmbr_AoB33GL7', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286111.53191, 'rx': '0', 'state': 'down'}, 'eno3': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno3', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286111.53191, 'rx': '0', 'state': 'down'}, 'eno4': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno4', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286111.53191, 'rx': '0', 'state': 'down'}, 'ens2f0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.1', 'txErrors': '0', 'speed': '10000', 'rxDropped': '0', 'name': 'ens2f0', 'tx': '32214474', 'txDropped': '0', 'sampleTime': 1478286111.53191, 'rx': '754373046', 'state': 'up'}, 'ens2f1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ens2f1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286111.53191, 'rx': '0', 'state': 'down'}}, 'txDropped': '0', 'cpuUser': '0.09', 'ksmPages': 100, 'elapsedTime': '6206.09', 'cpuLoad': '0.03', 'cpuSys': '0.05', 'diskStats': {'/var/log': {'free': '15189'}, '/var/log/core': {'free': '15189'}, '/var/run/vdsm/': {'free': '52254'}, '/tmp': {'free': '1640028'}}, 'cpuUserVdsmd': '0.40', 'netConfigDirty': 'False', 'memCommitted': 4161, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 98634, 'txRate': '0.00', 'bootTime': '1478279889', 'haStats': {'active': False, 'configured': False, 'score': 0, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'rxDropped': '16326', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'dateTime': '2016-11-04T19:02:02 GMT', 'anonHugePages': '906', 'memFree': 102315, 'cpuIdle': '99.86', 'vmActive': 1, 'v2vJobs': {}, 'cpuSysVdsmd': '0.33'} jsonrpc.Executor/4::INFO::2016-11-04 14:02:02,546::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getStats succeeded in 0.01 seconds Thread-12::DEBUG::2016-11-04 14:02:04,267::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:02:04,313::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:02:04,313::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.00031919 s, 1.9 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:02:04,344::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:02:04,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:02:04,368::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000441777 s, 799 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:02:04,372::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:02:04,394::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000447784 s, 768 kB/s\n') elapsed=0.03 Thread-12::DEBUG::2016-11-04 14:02:04,405::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000367921 s, 932 kB/s\n') elapsed=0.04 jsonrpc.Executor/5::DEBUG::2016-11-04 14:02:05,699::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getAllVmStats' in bridge with {} jsonrpc.Executor/5::DEBUG::2016-11-04 14:02:05,700::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getAllVmStats' in bridge with (suppressed) jsonrpc.Executor/5::INFO::2016-11-04 14:02:05,701::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getAllVmStats succeeded in 0.00 seconds periodic/3::DEBUG::2016-11-04 14:02:06,592::task::597::Storage.TaskManager.Task::(_updateState) Task=`3f694493-24d4-4717-8ccf-55b2d282517e`::moving from state init -> state preparing periodic/3::INFO::2016-11-04 14:02:06,593::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) periodic/3::INFO::2016-11-04 14:02:06,593::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000441777', 'lastCheck': '2.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000367921', 'lastCheck': '2.2', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000447784', 'lastCheck': '2.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00031919', 'lastCheck': '2.3', 'valid': True}} periodic/3::DEBUG::2016-11-04 14:02:06,593::task::1193::Storage.TaskManager.Task::(prepare) Task=`3f694493-24d4-4717-8ccf-55b2d282517e`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000441777', 'lastCheck': '2.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000367921', 'lastCheck': '2.2', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000447784', 'lastCheck': '2.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00031919', 'lastCheck': '2.3', 'valid': True}} periodic/3::DEBUG::2016-11-04 14:02:06,593::task::597::Storage.TaskManager.Task::(_updateState) Task=`3f694493-24d4-4717-8ccf-55b2d282517e`::moving from state preparing -> state finished periodic/3::DEBUG::2016-11-04 14:02:06,593::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/3::DEBUG::2016-11-04 14:02:06,593::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/3::DEBUG::2016-11-04 14:02:06,593::task::995::Storage.TaskManager.Task::(_decref) Task=`3f694493-24d4-4717-8ccf-55b2d282517e`::ref 0 aborting False Thread-114::DEBUG::2016-11-04 14:02:07,123::monitor::471::Storage.Monitor::(_refreshDomain) Refreshing domain 9d32932b-0e43-4377-b910-35fe2830c9e9 Thread-1038::DEBUG::2016-11-04 14:02:07,124::outOfProcess::87::Storage.oop::(getProcessPool) Creating ioprocess /rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02 Thread-1038::INFO::2016-11-04 14:02:07,124::__init__::325::IOProcessClient::(__init__) Starting client ioprocess-69 Thread-1038::DEBUG::2016-11-04 14:02:07,124::__init__::334::IOProcessClient::(_run) Starting ioprocess for client ioprocess-69 Thread-1038::DEBUG::2016-11-04 14:02:07,135::__init__::386::IOProcessClient::(_startCommunication) Starting communication thread for client ioprocess-69 Thread-1038::DEBUG::2016-11-04 14:02:07,136::__init__::398::IOProcessClient::(_startCommunication) Communication thread for client ioprocess-69 started Thread-1039::DEBUG::2016-11-04 14:02:07,137::outOfProcess::87::Storage.oop::(getProcessPool) Creating ioprocess /rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01 Thread-1039::INFO::2016-11-04 14:02:07,137::__init__::325::IOProcessClient::(__init__) Starting client ioprocess-70 Thread-1039::DEBUG::2016-11-04 14:02:07,137::__init__::334::IOProcessClient::(_run) Starting ioprocess for client ioprocess-70 Thread-1039::DEBUG::2016-11-04 14:02:07,147::__init__::386::IOProcessClient::(_startCommunication) Starting communication thread for client ioprocess-70 ioprocess communication (8832)::INFO::2016-11-04 14:02:07,147::__init__::447::IOProcess::(_processLogs) Starting ioprocess ioprocess communication (8832)::INFO::2016-11-04 14:02:07,147::__init__::447::IOProcess::(_processLogs) Starting ioprocess Thread-1039::DEBUG::2016-11-04 14:02:07,148::__init__::398::IOProcessClient::(_startCommunication) Communication thread for client ioprocess-70 started Thread-1040::DEBUG::2016-11-04 14:02:07,149::outOfProcess::87::Storage.oop::(getProcessPool) Creating ioprocess /rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports Thread-1040::INFO::2016-11-04 14:02:07,149::__init__::325::IOProcessClient::(__init__) Starting client ioprocess-71 Thread-1040::DEBUG::2016-11-04 14:02:07,149::__init__::334::IOProcessClient::(_run) Starting ioprocess for client ioprocess-71 Thread-1040::DEBUG::2016-11-04 14:02:07,159::__init__::386::IOProcessClient::(_startCommunication) Starting communication thread for client ioprocess-71 ioprocess communication (8839)::INFO::2016-11-04 14:02:07,160::__init__::447::IOProcess::(_processLogs) Starting ioprocess ioprocess communication (8839)::INFO::2016-11-04 14:02:07,160::__init__::447::IOProcess::(_processLogs) Starting ioprocess Thread-1040::DEBUG::2016-11-04 14:02:07,161::__init__::398::IOProcessClient::(_startCommunication) Communication thread for client ioprocess-71 started Thread-1041::DEBUG::2016-11-04 14:02:07,161::outOfProcess::87::Storage.oop::(getProcessPool) Creating ioprocess /rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos Thread-1041::INFO::2016-11-04 14:02:07,161::__init__::325::IOProcessClient::(__init__) Starting client ioprocess-72 Thread-1041::DEBUG::2016-11-04 14:02:07,161::__init__::334::IOProcessClient::(_run) Starting ioprocess for client ioprocess-72 Thread-117::DEBUG::2016-11-04 14:02:07,171::monitor::471::Storage.Monitor::(_refreshDomain) Refreshing domain d129a955-0810-4cb1-9ef5-86cca703a56d Thread-1041::DEBUG::2016-11-04 14:02:07,172::__init__::386::IOProcessClient::(_startCommunication) Starting communication thread for client ioprocess-72 ioprocess communication (8847)::INFO::2016-11-04 14:02:07,173::__init__::447::IOProcess::(_processLogs) Starting ioprocess ioprocess communication (8847)::INFO::2016-11-04 14:02:07,173::__init__::447::IOProcess::(_processLogs) Starting ioprocess Thread-1041::DEBUG::2016-11-04 14:02:07,174::__init__::398::IOProcessClient::(_startCommunication) Communication thread for client ioprocess-72 started ioprocess communication (8854)::INFO::2016-11-04 14:02:07,175::__init__::447::IOProcess::(_processLogs) Starting ioprocess ioprocess communication (8854)::INFO::2016-11-04 14:02:07,175::__init__::447::IOProcess::(_processLogs) Starting ioprocess Thread-114::DEBUG::2016-11-04 14:02:07,178::fileSD::159::Storage.StorageDomainManifest::(__init__) Reading domain in path /rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9 Thread-114::DEBUG::2016-11-04 14:02:07,178::persistent::194::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend Thread-117::DEBUG::2016-11-04 14:02:07,180::fileSD::159::Storage.StorageDomainManifest::(__init__) Reading domain in path /rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d Thread-117::DEBUG::2016-11-04 14:02:07,180::persistent::194::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend Thread-114::DEBUG::2016-11-04 14:02:07,181::persistent::236::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=iso01', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=00000001-0001-0001-0001-000000000366', 'REMOTE_PATH=labvmfst01.lab.ingram.io:/data1/ovirt_isos', 'ROLE=Regular', 'SDUUID=9d32932b-0e43-4377-b910-35fe2830c9e9', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=bb17f06823c8b5503680d68fe547b6adfbafe1f2'] Thread-117::DEBUG::2016-11-04 14:02:07,182::persistent::236::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=BubbaDosVM_NFS', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=6', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=d129a955-0810-4cb1-9ef5-86cca703a56d:Active,96cf2ee2-cbca-465c-a0b1-179d53c69d3e:Active,01cfff0b-6e88-4417-9462-071571f8fd00:Active,8d104335-a03c-4b78-9937-7b63d2e3b6c0:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=00000001-0001-0001-0001-000000000366', 'REMOTE_PATH=10.50.3.103:/data1/ovirt_data02', 'ROLE=Master', 'SDUUID=d129a955-0810-4cb1-9ef5-86cca703a56d', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=91314f0f753b172694f0dde6d9d0f1a3e6acfed4'] Thread-117::DEBUG::2016-11-04 14:02:07,186::fileSD::679::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] Thread-117::INFO::2016-11-04 14:02:07,186::sd::604::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace d129a955-0810-4cb1-9ef5-86cca703a56d_imageNS already registered Thread-117::INFO::2016-11-04 14:02:07,186::sd::612::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace d129a955-0810-4cb1-9ef5-86cca703a56d_volumeNS already registered Thread-114::DEBUG::2016-11-04 14:02:07,210::fileSD::679::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] Thread-114::INFO::2016-11-04 14:02:07,211::sd::604::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 9d32932b-0e43-4377-b910-35fe2830c9e9_imageNS already registered Thread-114::INFO::2016-11-04 14:02:07,211::sd::612::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 9d32932b-0e43-4377-b910-35fe2830c9e9_volumeNS already registered Thread-111::DEBUG::2016-11-04 14:02:07,490::monitor::471::Storage.Monitor::(_refreshDomain) Refreshing domain 9bee9944-821e-4ec7-8513-e92131d2ad76 Thread-112::DEBUG::2016-11-04 14:02:07,490::monitor::471::Storage.Monitor::(_refreshDomain) Refreshing domain bfcabdd8-3aa7-4241-91c3-5db0e843e476 Thread-111::DEBUG::2016-11-04 14:02:07,495::fileSD::159::Storage.StorageDomainManifest::(__init__) Reading domain in path /rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76 Thread-111::DEBUG::2016-11-04 14:02:07,495::persistent::194::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend Thread-112::DEBUG::2016-11-04 14:02:07,497::fileSD::159::Storage.StorageDomainManifest::(__init__) Reading domain in path /rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476 Thread-112::DEBUG::2016-11-04 14:02:07,497::persistent::194::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend Thread-111::DEBUG::2016-11-04 14:02:07,498::persistent::236::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Backup', 'DESCRIPTION=export01', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=00000001-0001-0001-0001-000000000366', 'REMOTE_PATH=labvmfst01.lab.ingram.io:/data1/ovirt_exports', 'ROLE=Regular', 'SDUUID=9bee9944-821e-4ec7-8513-e92131d2ad76', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=02f10a5773e9d4d28d2721049d6d9514961cff10'] Thread-112::DEBUG::2016-11-04 14:02:07,499::persistent::236::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=BubbaUnoVM_NFS', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=00000001-0001-0001-0001-000000000366', 'REMOTE_PATH=10.50.3.102:/data1/ovirt_data01', 'ROLE=Regular', 'SDUUID=bfcabdd8-3aa7-4241-91c3-5db0e843e476', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=bf7f8dc945380336b3966f35a029b33939f57ecd'] Thread-112::DEBUG::2016-11-04 14:02:07,515::fileSD::679::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] Thread-112::INFO::2016-11-04 14:02:07,515::sd::604::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace bfcabdd8-3aa7-4241-91c3-5db0e843e476_imageNS already registered Thread-112::INFO::2016-11-04 14:02:07,516::sd::612::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace bfcabdd8-3aa7-4241-91c3-5db0e843e476_volumeNS already registered Thread-111::DEBUG::2016-11-04 14:02:07,524::fileSD::679::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] Thread-111::INFO::2016-11-04 14:02:07,524::sd::604::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 9bee9944-821e-4ec7-8513-e92131d2ad76_imageNS already registered Thread-111::INFO::2016-11-04 14:02:07,524::sd::612::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 9bee9944-821e-4ec7-8513-e92131d2ad76_volumeNS already registered Reactor thread::INFO::2016-11-04 14:02:14,104::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58204 Reactor thread::DEBUG::2016-11-04 14:02:14,110::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:02:14,110::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58204 Reactor thread::DEBUG::2016-11-04 14:02:14,110::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58204) BindingXMLRPC::INFO::2016-11-04 14:02:14,110::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58204 Thread-1054::INFO::2016-11-04 14:02:14,110::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58204 started Thread-1054::DEBUG::2016-11-04 14:02:14,111::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call getAllVmStats with () {} Thread-1054::DEBUG::2016-11-04 14:02:14,112::bindingxmlrpc::1255::vds::(wrapper) return getAllVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} Thread-1054::INFO::2016-11-04 14:02:14,112::bindingxmlrpc::1282::vds::(wrapper) RPC call getAllVmStats finished (code=0) in 0.00 seconds Thread-1054::INFO::2016-11-04 14:02:14,118::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58204 stopped Reactor thread::INFO::2016-11-04 14:02:14,119::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58206 Reactor thread::DEBUG::2016-11-04 14:02:14,125::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:02:14,125::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58206 Reactor thread::DEBUG::2016-11-04 14:02:14,125::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58206) BindingXMLRPC::INFO::2016-11-04 14:02:14,125::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58206 Thread-1055::INFO::2016-11-04 14:02:14,125::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58206 started Thread-1055::DEBUG::2016-11-04 14:02:14,126::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call vmGetIoTunePolicy with ('cddd2019-5baf-4321-aa8b-130b2889d1b0',) {} Thread-1055::DEBUG::2016-11-04 14:02:14,127::bindingxmlrpc::1255::vds::(wrapper) return vmGetIoTunePolicy with {'ioTunePolicy': [], 'status': {'message': 'Done', 'code': 0}} Thread-1055::INFO::2016-11-04 14:02:14,127::bindingxmlrpc::1282::vds::(wrapper) RPC call vmGetIoTunePolicy finished (code=0) in 0.00 seconds Thread-1055::INFO::2016-11-04 14:02:14,127::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58206 stopped Thread-12::DEBUG::2016-11-04 14:02:14,259::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:02:14,304::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:02:14,304::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.00035836 s, 1.7 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:02:14,345::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:02:14,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:02:14,371::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000412309 s, 856 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:02:14,374::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:02:14,399::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000483478 s, 712 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:02:14,408::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000419251 s, 818 kB/s\n') elapsed=0.03 jsonrpc.Executor/6::DEBUG::2016-11-04 14:02:18,770::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getStats' in bridge with {} jsonrpc.Executor/6::DEBUG::2016-11-04 14:02:18,771::task::597::Storage.TaskManager.Task::(_updateState) Task=`62d0a4a3-f30a-4996-ab00-0ba41d7c687c`::moving from state init -> state preparing jsonrpc.Executor/6::INFO::2016-11-04 14:02:18,771::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) jsonrpc.Executor/6::INFO::2016-11-04 14:02:18,771::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000412309', 'lastCheck': '1.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000419251', 'lastCheck': '1.3', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000483478', 'lastCheck': '1.6', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00035836', 'lastCheck': '1.6', 'valid': True}} jsonrpc.Executor/6::DEBUG::2016-11-04 14:02:18,771::task::1193::Storage.TaskManager.Task::(prepare) Task=`62d0a4a3-f30a-4996-ab00-0ba41d7c687c`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000412309', 'lastCheck': '1.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000419251', 'lastCheck': '1.3', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000483478', 'lastCheck': '1.6', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00035836', 'lastCheck': '1.6', 'valid': True}} jsonrpc.Executor/6::DEBUG::2016-11-04 14:02:18,772::task::597::Storage.TaskManager.Task::(_updateState) Task=`62d0a4a3-f30a-4996-ab00-0ba41d7c687c`::moving from state preparing -> state finished jsonrpc.Executor/6::DEBUG::2016-11-04 14:02:18,772::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} jsonrpc.Executor/6::DEBUG::2016-11-04 14:02:18,772::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} jsonrpc.Executor/6::DEBUG::2016-11-04 14:02:18,772::task::995::Storage.TaskManager.Task::(_decref) Task=`62d0a4a3-f30a-4996-ab00-0ba41d7c687c`::ref 0 aborting False jsonrpc.Executor/6::DEBUG::2016-11-04 14:02:18,779::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getStats' in bridge with {'cpuStatistics': {'20': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '21': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '22': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '23': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '1': {'cpuUser': '0.87', 'nodeIndex': 0, 'cpuSys': '1.07', 'cpuIdle': '98.06'}, '0': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '3': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.80'}, '2': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '5': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '4': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.74'}, '7': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '6': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '9': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '8': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '13': {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.60'}, '12': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '15': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '14': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '17': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '16': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '19': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '18': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '99.93'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31480'}, '0': {'memPercent': 5, 'memFree': '70631'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'rxRate': '0.01', 'vmCount': 1, 'memUsed': '3', 'storageDomains': {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000412309', 'lastCheck': '1.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000419251', 'lastCheck': '1.3', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000483478', 'lastCheck': '1.6', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00035836', 'lastCheck': '1.6', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'eno1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '11732', 'name': 'eno1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286126.526584, 'rx': '5251617', 'state': 'up'}, 'ovirtmgmt': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.1', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '31558695', 'txDropped': '0', 'sampleTime': 1478286126.526584, 'rx': '727942211', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '6542962', 'txDropped': '0', 'sampleTime': 1478286126.526584, 'rx': '6542962', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286126.526584, 'rx': '0', 'state': 'down'}, 'eno2': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno2', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286126.526584, 'rx': '0', 'state': 'down'}, 'vnet0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'vnet0', 'tx': '361715', 'txDropped': '0', 'sampleTime': 1478286126.526584, 'rx': '100632', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286126.526584, 'rx': '0', 'state': 'down'}, 'vdsmbr_AoB33GL7': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '4633', 'name': 'vdsmbr_AoB33GL7', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286126.526584, 'rx': '0', 'state': 'down'}, 'eno3': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno3', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286126.526584, 'rx': '0', 'state': 'down'}, 'eno4': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno4', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286126.526584, 'rx': '0', 'state': 'down'}, 'ens2f0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '10000', 'rxDropped': '0', 'name': 'ens2f0', 'tx': '32273382', 'txDropped': '0', 'sampleTime': 1478286126.526584, 'rx': '756597719', 'state': 'up'}, 'ens2f1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ens2f1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286126.526584, 'rx': '0', 'state': 'down'}}, 'txDropped': '0', 'cpuUser': '0.07', 'ksmPages': 100, 'elapsedTime': '6222.32', 'cpuLoad': '0.03', 'cpuSys': '0.07', 'diskStats': {'/var/log': {'free': '15189'}, '/var/log/core': {'free': '15189'}, '/var/run/vdsm/': {'free': '52254'}, '/tmp': {'free': '1640028'}}, 'cpuUserVdsmd': '0.47', 'netConfigDirty': 'False', 'memCommitted': 4161, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 98632, 'txRate': '0.00', 'bootTime': '1478279889', 'haStats': {'active': False, 'configured': False, 'score': 0, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'rxDropped': '16365', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'dateTime': '2016-11-04T19:02:18 GMT', 'anonHugePages': '902', 'memFree': 102313, 'cpuIdle': '99.86', 'vmActive': 1, 'v2vJobs': {}, 'cpuSysVdsmd': '0.53'} jsonrpc.Executor/6::INFO::2016-11-04 14:02:18,780::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getStats succeeded in 0.01 seconds periodic/0::DEBUG::2016-11-04 14:02:21,597::task::597::Storage.TaskManager.Task::(_updateState) Task=`082cd735-507d-43f7-8f2c-a525092de4ef`::moving from state init -> state preparing periodic/0::INFO::2016-11-04 14:02:21,598::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) periodic/0::INFO::2016-11-04 14:02:21,598::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000412309', 'lastCheck': '4.1', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000419251', 'lastCheck': '4.1', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000483478', 'lastCheck': '4.4', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00035836', 'lastCheck': '4.4', 'valid': True}} periodic/0::DEBUG::2016-11-04 14:02:21,598::task::1193::Storage.TaskManager.Task::(prepare) Task=`082cd735-507d-43f7-8f2c-a525092de4ef`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000412309', 'lastCheck': '4.1', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000419251', 'lastCheck': '4.1', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000483478', 'lastCheck': '4.4', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00035836', 'lastCheck': '4.4', 'valid': True}} periodic/0::DEBUG::2016-11-04 14:02:21,598::task::597::Storage.TaskManager.Task::(_updateState) Task=`082cd735-507d-43f7-8f2c-a525092de4ef`::moving from state preparing -> state finished periodic/0::DEBUG::2016-11-04 14:02:21,598::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/0::DEBUG::2016-11-04 14:02:21,598::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/0::DEBUG::2016-11-04 14:02:21,598::task::995::Storage.TaskManager.Task::(_decref) Task=`082cd735-507d-43f7-8f2c-a525092de4ef`::ref 0 aborting False jsonrpc.Executor/0::DEBUG::2016-11-04 14:02:21,761::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getAllVmStats' in bridge with {} jsonrpc.Executor/0::DEBUG::2016-11-04 14:02:21,762::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getAllVmStats' in bridge with (suppressed) jsonrpc.Executor/0::INFO::2016-11-04 14:02:21,763::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getAllVmStats succeeded in 0.01 seconds Thread-12::DEBUG::2016-11-04 14:02:24,269::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:02:24,314::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:02:24,314::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000239874 s, 2.5 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:02:24,345::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:02:24,355::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:02:24,374::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000318474 s, 1.1 MB/s\n') elapsed=0.03 Thread-12::DEBUG::2016-11-04 14:02:24,376::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:02:24,402::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000393032 s, 875 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:02:24,410::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000451865 s, 759 kB/s\n') elapsed=0.03 Reactor thread::INFO::2016-11-04 14:02:29,156::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58208 Reactor thread::DEBUG::2016-11-04 14:02:29,162::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:02:29,162::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58208 Reactor thread::DEBUG::2016-11-04 14:02:29,162::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58208) BindingXMLRPC::INFO::2016-11-04 14:02:29,163::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58208 Thread-1056::INFO::2016-11-04 14:02:29,163::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58208 started Thread-1056::DEBUG::2016-11-04 14:02:29,163::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call getAllVmStats with () {} Thread-1056::DEBUG::2016-11-04 14:02:29,164::bindingxmlrpc::1255::vds::(wrapper) return getAllVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} Thread-1056::INFO::2016-11-04 14:02:29,165::bindingxmlrpc::1282::vds::(wrapper) RPC call getAllVmStats finished (code=0) in 0.00 seconds Thread-1056::INFO::2016-11-04 14:02:29,170::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58208 stopped Reactor thread::INFO::2016-11-04 14:02:29,171::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58210 Reactor thread::DEBUG::2016-11-04 14:02:29,177::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:02:29,177::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58210 Reactor thread::DEBUG::2016-11-04 14:02:29,178::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58210) BindingXMLRPC::INFO::2016-11-04 14:02:29,178::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58210 Thread-1057::INFO::2016-11-04 14:02:29,178::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58210 started Thread-1057::DEBUG::2016-11-04 14:02:29,178::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call vmGetIoTunePolicy with ('cddd2019-5baf-4321-aa8b-130b2889d1b0',) {} Thread-1057::DEBUG::2016-11-04 14:02:29,179::bindingxmlrpc::1255::vds::(wrapper) return vmGetIoTunePolicy with {'ioTunePolicy': [], 'status': {'message': 'Done', 'code': 0}} Thread-1057::INFO::2016-11-04 14:02:29,179::bindingxmlrpc::1282::vds::(wrapper) RPC call vmGetIoTunePolicy finished (code=0) in 0.00 seconds Thread-1057::INFO::2016-11-04 14:02:29,180::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58210 stopped Thread-12::DEBUG::2016-11-04 14:02:34,271::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:02:34,324::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:02:34,324::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000454775 s, 1.3 MB/s\n') elapsed=0.06 Thread-12::DEBUG::2016-11-04 14:02:34,344::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:02:34,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:02:34,372::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000415649 s, 849 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:02:34,374::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:02:34,398::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000395022 s, 871 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:02:34,408::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.00049154 s, 698 kB/s\n') elapsed=0.03 jsonrpc.Executor/2::DEBUG::2016-11-04 14:02:34,990::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getStats' in bridge with {} jsonrpc.Executor/2::DEBUG::2016-11-04 14:02:34,992::task::597::Storage.TaskManager.Task::(_updateState) Task=`30a135e3-d705-4eea-a851-fe3bb1d204ec`::moving from state init -> state preparing jsonrpc.Executor/2::INFO::2016-11-04 14:02:34,992::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) jsonrpc.Executor/2::INFO::2016-11-04 14:02:34,992::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000415649', 'lastCheck': '0.6', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00049154', 'lastCheck': '0.6', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000395022', 'lastCheck': '0.6', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000454775', 'lastCheck': '0.7', 'valid': True}} jsonrpc.Executor/2::DEBUG::2016-11-04 14:02:34,992::task::1193::Storage.TaskManager.Task::(prepare) Task=`30a135e3-d705-4eea-a851-fe3bb1d204ec`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000415649', 'lastCheck': '0.6', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00049154', 'lastCheck': '0.6', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000395022', 'lastCheck': '0.6', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000454775', 'lastCheck': '0.7', 'valid': True}} jsonrpc.Executor/2::DEBUG::2016-11-04 14:02:34,992::task::597::Storage.TaskManager.Task::(_updateState) Task=`30a135e3-d705-4eea-a851-fe3bb1d204ec`::moving from state preparing -> state finished jsonrpc.Executor/2::DEBUG::2016-11-04 14:02:34,992::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} jsonrpc.Executor/2::DEBUG::2016-11-04 14:02:34,992::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} jsonrpc.Executor/2::DEBUG::2016-11-04 14:02:34,993::task::995::Storage.TaskManager.Task::(_decref) Task=`30a135e3-d705-4eea-a851-fe3bb1d204ec`::ref 0 aborting False jsonrpc.Executor/2::DEBUG::2016-11-04 14:02:35,000::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getStats' in bridge with {'cpuStatistics': {'20': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '21': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '22': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '23': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '1': {'cpuUser': '0.87', 'nodeIndex': 0, 'cpuSys': '0.93', 'cpuIdle': '98.20'}, '0': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '3': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '2': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '5': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '4': {'cpuUser': '0.20', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.73'}, '7': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '6': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '9': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '8': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '13': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.47'}, '12': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '15': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '14': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '17': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '16': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '19': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '18': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31479'}, '0': {'memPercent': 5, 'memFree': '70631'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'rxRate': '0.01', 'vmCount': 1, 'memUsed': '3', 'storageDomains': {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000415649', 'lastCheck': '0.6', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00049154', 'lastCheck': '0.6', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000395022', 'lastCheck': '0.6', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000454775', 'lastCheck': '0.7', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'eno1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '11760', 'name': 'eno1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286141.531415, 'rx': '5264094', 'state': 'up'}, 'ovirtmgmt': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.1', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '31596347', 'txDropped': '0', 'sampleTime': 1478286141.531415, 'rx': '729011894', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '6561208', 'txDropped': '0', 'sampleTime': 1478286141.531415, 'rx': '6561208', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286141.531415, 'rx': '0', 'state': 'down'}, 'eno2': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno2', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286141.531415, 'rx': '0', 'state': 'down'}, 'vnet0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'vnet0', 'tx': '362892', 'txDropped': '0', 'sampleTime': 1478286141.531415, 'rx': '100632', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286141.531415, 'rx': '0', 'state': 'down'}, 'vdsmbr_AoB33GL7': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '4651', 'name': 'vdsmbr_AoB33GL7', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286141.531415, 'rx': '0', 'state': 'down'}, 'eno3': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno3', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286141.531415, 'rx': '0', 'state': 'down'}, 'eno4': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno4', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286141.531415, 'rx': '0', 'state': 'down'}, 'ens2f0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '10000', 'rxDropped': '0', 'name': 'ens2f0', 'tx': '32311635', 'txDropped': '0', 'sampleTime': 1478286141.531415, 'rx': '757714422', 'state': 'up'}, 'ens2f1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ens2f1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286141.531415, 'rx': '0', 'state': 'down'}}, 'txDropped': '0', 'cpuUser': '0.07', 'ksmPages': 100, 'elapsedTime': '6238.54', 'cpuLoad': '0.02', 'cpuSys': '0.06', 'diskStats': {'/var/log': {'free': '15189'}, '/var/log/core': {'free': '15189'}, '/var/run/vdsm/': {'free': '52254'}, '/tmp': {'free': '1640028'}}, 'cpuUserVdsmd': '0.47', 'netConfigDirty': 'False', 'memCommitted': 4161, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 98632, 'txRate': '0.00', 'bootTime': '1478279889', 'haStats': {'active': False, 'configured': False, 'score': 0, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'rxDropped': '16411', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'dateTime': '2016-11-04T19:02:34 GMT', 'anonHugePages': '902', 'memFree': 102313, 'cpuIdle': '99.87', 'vmActive': 1, 'v2vJobs': {}, 'cpuSysVdsmd': '0.53'} jsonrpc.Executor/2::INFO::2016-11-04 14:02:35,001::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getStats succeeded in 0.01 seconds periodic/3::DEBUG::2016-11-04 14:02:36,456::task::597::Storage.TaskManager.Task::(_updateState) Task=`f82700d5-1c35-484a-879a-4cea505e0507`::moving from state init -> state preparing periodic/3::INFO::2016-11-04 14:02:36,456::logUtils::49::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'bfcabdd8-3aa7-4241-91c3-5db0e843e476', spUUID=u'00000001-0001-0001-0001-000000000366', imgUUID=u'b2109025-42be-4c40-ac2b-26baf8c304e5', volUUID=u'3153e04b-729c-44b2-b31a-0b2547ad3a77', options=None) periodic/3::INFO::2016-11-04 14:02:36,458::logUtils::52::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '17503478272', 'apparentsize': '21474836480'} periodic/3::DEBUG::2016-11-04 14:02:36,458::task::1193::Storage.TaskManager.Task::(prepare) Task=`f82700d5-1c35-484a-879a-4cea505e0507`::finished: {'truesize': '17503478272', 'apparentsize': '21474836480'} periodic/3::DEBUG::2016-11-04 14:02:36,458::task::597::Storage.TaskManager.Task::(_updateState) Task=`f82700d5-1c35-484a-879a-4cea505e0507`::moving from state preparing -> state finished periodic/3::DEBUG::2016-11-04 14:02:36,458::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/3::DEBUG::2016-11-04 14:02:36,458::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/3::DEBUG::2016-11-04 14:02:36,458::task::995::Storage.TaskManager.Task::(_decref) Task=`f82700d5-1c35-484a-879a-4cea505e0507`::ref 0 aborting False periodic/0::DEBUG::2016-11-04 14:02:36,592::task::597::Storage.TaskManager.Task::(_updateState) Task=`3a036150-8ae1-4114-806d-115ff40c1011`::moving from state init -> state preparing periodic/0::INFO::2016-11-04 14:02:36,592::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) periodic/0::INFO::2016-11-04 14:02:36,592::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000415649', 'lastCheck': '2.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00049154', 'lastCheck': '2.2', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000395022', 'lastCheck': '2.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000454775', 'lastCheck': '2.3', 'valid': True}} periodic/0::DEBUG::2016-11-04 14:02:36,592::task::1193::Storage.TaskManager.Task::(prepare) Task=`3a036150-8ae1-4114-806d-115ff40c1011`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000415649', 'lastCheck': '2.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00049154', 'lastCheck': '2.2', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000395022', 'lastCheck': '2.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000454775', 'lastCheck': '2.3', 'valid': True}} periodic/0::DEBUG::2016-11-04 14:02:36,592::task::597::Storage.TaskManager.Task::(_updateState) Task=`3a036150-8ae1-4114-806d-115ff40c1011`::moving from state preparing -> state finished periodic/0::DEBUG::2016-11-04 14:02:36,592::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/0::DEBUG::2016-11-04 14:02:36,592::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/0::DEBUG::2016-11-04 14:02:36,592::task::995::Storage.TaskManager.Task::(_decref) Task=`3a036150-8ae1-4114-806d-115ff40c1011`::ref 0 aborting False jsonrpc.Executor/7::DEBUG::2016-11-04 14:02:37,056::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getAllVmStats' in bridge with {} jsonrpc.Executor/7::DEBUG::2016-11-04 14:02:37,058::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getAllVmStats' in bridge with (suppressed) jsonrpc.Executor/7::INFO::2016-11-04 14:02:37,058::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getAllVmStats succeeded in 0.00 seconds Reactor thread::INFO::2016-11-04 14:02:44,211::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58212 Reactor thread::DEBUG::2016-11-04 14:02:44,218::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:02:44,218::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58212 Reactor thread::DEBUG::2016-11-04 14:02:44,218::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58212) BindingXMLRPC::INFO::2016-11-04 14:02:44,219::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58212 Thread-1058::INFO::2016-11-04 14:02:44,219::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58212 started Thread-1058::DEBUG::2016-11-04 14:02:44,219::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call getAllVmStats with () {} Thread-1058::DEBUG::2016-11-04 14:02:44,220::bindingxmlrpc::1255::vds::(wrapper) return getAllVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} Thread-1058::INFO::2016-11-04 14:02:44,220::bindingxmlrpc::1282::vds::(wrapper) RPC call getAllVmStats finished (code=0) in 0.00 seconds Thread-1058::INFO::2016-11-04 14:02:44,226::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58212 stopped Reactor thread::INFO::2016-11-04 14:02:44,227::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58214 Reactor thread::DEBUG::2016-11-04 14:02:44,233::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:02:44,233::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58214 Reactor thread::DEBUG::2016-11-04 14:02:44,233::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58214) BindingXMLRPC::INFO::2016-11-04 14:02:44,233::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58214 Thread-1059::INFO::2016-11-04 14:02:44,233::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58214 started Thread-1059::DEBUG::2016-11-04 14:02:44,234::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call vmGetIoTunePolicy with ('cddd2019-5baf-4321-aa8b-130b2889d1b0',) {} Thread-1059::DEBUG::2016-11-04 14:02:44,235::bindingxmlrpc::1255::vds::(wrapper) return vmGetIoTunePolicy with {'ioTunePolicy': [], 'status': {'message': 'Done', 'code': 0}} Thread-1059::INFO::2016-11-04 14:02:44,235::bindingxmlrpc::1282::vds::(wrapper) RPC call vmGetIoTunePolicy finished (code=0) in 0.00 seconds Thread-1059::INFO::2016-11-04 14:02:44,235::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58214 stopped Thread-12::DEBUG::2016-11-04 14:02:44,268::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:02:44,314::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:02:44,314::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000421349 s, 1.4 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:02:44,344::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:02:44,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:02:44,371::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000470032 s, 751 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:02:44,373::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:02:44,395::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000487185 s, 706 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:02:44,406::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000486319 s, 705 kB/s\n') elapsed=0.04 jsonrpc.Executor/1::DEBUG::2016-11-04 14:02:51,143::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getStats' in bridge with {} jsonrpc.Executor/1::DEBUG::2016-11-04 14:02:51,144::task::597::Storage.TaskManager.Task::(_updateState) Task=`c04c100b-336c-4710-820c-309fcd9b595c`::moving from state init -> state preparing jsonrpc.Executor/1::INFO::2016-11-04 14:02:51,144::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) jsonrpc.Executor/1::INFO::2016-11-04 14:02:51,144::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000470032', 'lastCheck': '3.6', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000486319', 'lastCheck': '3.6', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000487185', 'lastCheck': '3.9', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000421349', 'lastCheck': '3.9', 'valid': True}} jsonrpc.Executor/1::DEBUG::2016-11-04 14:02:51,144::task::1193::Storage.TaskManager.Task::(prepare) Task=`c04c100b-336c-4710-820c-309fcd9b595c`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000470032', 'lastCheck': '3.6', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000486319', 'lastCheck': '3.6', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000487185', 'lastCheck': '3.9', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000421349', 'lastCheck': '3.9', 'valid': True}} jsonrpc.Executor/1::DEBUG::2016-11-04 14:02:51,144::task::597::Storage.TaskManager.Task::(_updateState) Task=`c04c100b-336c-4710-820c-309fcd9b595c`::moving from state preparing -> state finished jsonrpc.Executor/1::DEBUG::2016-11-04 14:02:51,145::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} jsonrpc.Executor/1::DEBUG::2016-11-04 14:02:51,145::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} jsonrpc.Executor/1::DEBUG::2016-11-04 14:02:51,145::task::995::Storage.TaskManager.Task::(_decref) Task=`c04c100b-336c-4710-820c-309fcd9b595c`::ref 0 aborting False jsonrpc.Executor/1::DEBUG::2016-11-04 14:02:51,152::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getStats' in bridge with {'cpuStatistics': {'20': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '21': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '22': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '23': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '1': {'cpuUser': '0.60', 'nodeIndex': 0, 'cpuSys': '1.07', 'cpuIdle': '98.33'}, '0': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.87'}, '3': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '2': {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.66'}, '5': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '4': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '7': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '6': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '9': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '8': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '13': {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.66'}, '12': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '15': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '14': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '17': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '16': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '19': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '18': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31475'}, '0': {'memPercent': 5, 'memFree': '70634'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'rxRate': '0.01', 'vmCount': 1, 'memUsed': '3', 'storageDomains': {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000470032', 'lastCheck': '3.6', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000486319', 'lastCheck': '3.6', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000487185', 'lastCheck': '3.9', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000421349', 'lastCheck': '3.9', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'eno1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '11788', 'name': 'eno1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286156.525699, 'rx': '5276036', 'state': 'up'}, 'ovirtmgmt': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.1', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '31636097', 'txDropped': '0', 'sampleTime': 1478286156.525699, 'rx': '730084144', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '6579422', 'txDropped': '0', 'sampleTime': 1478286156.525699, 'rx': '6579422', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286156.525699, 'rx': '0', 'state': 'down'}, 'eno2': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno2', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286156.525699, 'rx': '0', 'state': 'down'}, 'vnet0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'vnet0', 'tx': '364462', 'txDropped': '0', 'sampleTime': 1478286156.525699, 'rx': '100632', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286156.525699, 'rx': '0', 'state': 'down'}, 'vdsmbr_AoB33GL7': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '4673', 'name': 'vdsmbr_AoB33GL7', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286156.525699, 'rx': '0', 'state': 'down'}, 'eno3': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno3', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286156.525699, 'rx': '0', 'state': 'down'}, 'eno4': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno4', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286156.525699, 'rx': '0', 'state': 'down'}, 'ens2f0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '10000', 'rxDropped': '0', 'name': 'ens2f0', 'tx': '32352317', 'txDropped': '0', 'sampleTime': 1478286156.525699, 'rx': '758824766', 'state': 'up'}, 'ens2f1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ens2f1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286156.525699, 'rx': '0', 'state': 'down'}}, 'txDropped': '0', 'cpuUser': '0.06', 'ksmPages': 100, 'elapsedTime': '6254.70', 'cpuLoad': '0.02', 'cpuSys': '0.06', 'diskStats': {'/var/log': {'free': '15189'}, '/var/log/core': {'free': '15189'}, '/var/run/vdsm/': {'free': '52254'}, '/tmp': {'free': '1640028'}}, 'cpuUserVdsmd': '0.47', 'netConfigDirty': 'False', 'memCommitted': 4161, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 98633, 'txRate': '0.00', 'bootTime': '1478279889', 'haStats': {'active': False, 'configured': False, 'score': 0, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'rxDropped': '16461', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'dateTime': '2016-11-04T19:02:51 GMT', 'anonHugePages': '902', 'memFree': 102314, 'cpuIdle': '99.88', 'vmActive': 1, 'v2vJobs': {}, 'cpuSysVdsmd': '0.60'} jsonrpc.Executor/1::INFO::2016-11-04 14:02:51,153::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getStats succeeded in 0.02 seconds periodic/2::DEBUG::2016-11-04 14:02:51,596::task::597::Storage.TaskManager.Task::(_updateState) Task=`994ba98b-fe81-4454-9baa-b0841e5259ca`::moving from state init -> state preparing periodic/2::INFO::2016-11-04 14:02:51,596::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) periodic/2::INFO::2016-11-04 14:02:51,596::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000470032', 'lastCheck': '4.1', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000486319', 'lastCheck': '4.1', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000487185', 'lastCheck': '4.4', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000421349', 'lastCheck': '4.4', 'valid': True}} periodic/2::DEBUG::2016-11-04 14:02:51,596::task::1193::Storage.TaskManager.Task::(prepare) Task=`994ba98b-fe81-4454-9baa-b0841e5259ca`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000470032', 'lastCheck': '4.1', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000486319', 'lastCheck': '4.1', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000487185', 'lastCheck': '4.4', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000421349', 'lastCheck': '4.4', 'valid': True}} periodic/2::DEBUG::2016-11-04 14:02:51,596::task::597::Storage.TaskManager.Task::(_updateState) Task=`994ba98b-fe81-4454-9baa-b0841e5259ca`::moving from state preparing -> state finished periodic/2::DEBUG::2016-11-04 14:02:51,597::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/2::DEBUG::2016-11-04 14:02:51,597::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/2::DEBUG::2016-11-04 14:02:51,597::task::995::Storage.TaskManager.Task::(_decref) Task=`994ba98b-fe81-4454-9baa-b0841e5259ca`::ref 0 aborting False jsonrpc.Executor/3::DEBUG::2016-11-04 14:02:53,121::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getAllVmStats' in bridge with {} jsonrpc.Executor/3::DEBUG::2016-11-04 14:02:53,122::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getAllVmStats' in bridge with (suppressed) jsonrpc.Executor/3::INFO::2016-11-04 14:02:53,123::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getAllVmStats succeeded in 0.01 seconds Thread-12::DEBUG::2016-11-04 14:02:54,267::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:02:54,313::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:02:54,314::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000479742 s, 1.3 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:02:54,344::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:02:54,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:02:54,372::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000464082 s, 761 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:02:54,375::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:02:54,400::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000512016 s, 672 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:02:54,443::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:02:54,444::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.00054311 s, 632 kB/s\n') elapsed=0.07 Reactor thread::INFO::2016-11-04 14:02:59,264::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58216 Reactor thread::DEBUG::2016-11-04 14:02:59,271::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:02:59,271::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58216 Reactor thread::DEBUG::2016-11-04 14:02:59,271::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58216) BindingXMLRPC::INFO::2016-11-04 14:02:59,271::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58216 Thread-1060::INFO::2016-11-04 14:02:59,272::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58216 started Thread-1060::DEBUG::2016-11-04 14:02:59,272::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call getAllVmStats with () {} Thread-1060::DEBUG::2016-11-04 14:02:59,274::bindingxmlrpc::1255::vds::(wrapper) return getAllVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} Thread-1060::INFO::2016-11-04 14:02:59,274::bindingxmlrpc::1282::vds::(wrapper) RPC call getAllVmStats finished (code=0) in 0.01 seconds Thread-1060::INFO::2016-11-04 14:02:59,279::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58216 stopped Reactor thread::INFO::2016-11-04 14:02:59,280::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58218 Reactor thread::DEBUG::2016-11-04 14:02:59,286::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:02:59,286::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58218 Reactor thread::DEBUG::2016-11-04 14:02:59,287::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58218) BindingXMLRPC::INFO::2016-11-04 14:02:59,287::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58218 Thread-1061::INFO::2016-11-04 14:02:59,287::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58218 started Thread-1061::DEBUG::2016-11-04 14:02:59,287::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call vmGetIoTunePolicy with ('cddd2019-5baf-4321-aa8b-130b2889d1b0',) {} Thread-1061::DEBUG::2016-11-04 14:02:59,288::bindingxmlrpc::1255::vds::(wrapper) return vmGetIoTunePolicy with {'ioTunePolicy': [], 'status': {'message': 'Done', 'code': 0}} Thread-1061::INFO::2016-11-04 14:02:59,288::bindingxmlrpc::1282::vds::(wrapper) RPC call vmGetIoTunePolicy finished (code=0) in 0.00 seconds Thread-1061::INFO::2016-11-04 14:02:59,289::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58218 stopped Thread-12::DEBUG::2016-11-04 14:03:04,264::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:03:04,314::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:03:04,314::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000435236 s, 1.4 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:03:04,345::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:03:04,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:03:04,372::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000407565 s, 866 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:03:04,375::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:03:04,400::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000390698 s, 880 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:03:04,409::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000472804 s, 725 kB/s\n') elapsed=0.03 periodic/1::DEBUG::2016-11-04 14:03:06,587::task::597::Storage.TaskManager.Task::(_updateState) Task=`0e64b99a-8954-4f5b-b997-4698be590fe7`::moving from state init -> state preparing periodic/1::INFO::2016-11-04 14:03:06,588::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) periodic/1::INFO::2016-11-04 14:03:06,588::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000407565', 'lastCheck': '2.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000472804', 'lastCheck': '2.2', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000390698', 'lastCheck': '2.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000435236', 'lastCheck': '2.3', 'valid': True}} periodic/1::DEBUG::2016-11-04 14:03:06,588::task::1193::Storage.TaskManager.Task::(prepare) Task=`0e64b99a-8954-4f5b-b997-4698be590fe7`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000407565', 'lastCheck': '2.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000472804', 'lastCheck': '2.2', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000390698', 'lastCheck': '2.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000435236', 'lastCheck': '2.3', 'valid': True}} periodic/1::DEBUG::2016-11-04 14:03:06,588::task::597::Storage.TaskManager.Task::(_updateState) Task=`0e64b99a-8954-4f5b-b997-4698be590fe7`::moving from state preparing -> state finished periodic/1::DEBUG::2016-11-04 14:03:06,588::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/1::DEBUG::2016-11-04 14:03:06,588::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/1::DEBUG::2016-11-04 14:03:06,588::task::995::Storage.TaskManager.Task::(_decref) Task=`0e64b99a-8954-4f5b-b997-4698be590fe7`::ref 0 aborting False jsonrpc.Executor/4::DEBUG::2016-11-04 14:03:07,228::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getStats' in bridge with {} jsonrpc.Executor/4::DEBUG::2016-11-04 14:03:07,229::task::597::Storage.TaskManager.Task::(_updateState) Task=`657b8259-8c85-48d8-8107-8e70c0f5d5da`::moving from state init -> state preparing jsonrpc.Executor/4::INFO::2016-11-04 14:03:07,229::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) jsonrpc.Executor/4::INFO::2016-11-04 14:03:07,230::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000407565', 'lastCheck': '2.9', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000472804', 'lastCheck': '2.8', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000390698', 'lastCheck': '0.0', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000435236', 'lastCheck': '0.0', 'valid': True}} jsonrpc.Executor/4::DEBUG::2016-11-04 14:03:07,230::task::1193::Storage.TaskManager.Task::(prepare) Task=`657b8259-8c85-48d8-8107-8e70c0f5d5da`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000407565', 'lastCheck': '2.9', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000472804', 'lastCheck': '2.8', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000390698', 'lastCheck': '0.0', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000435236', 'lastCheck': '0.0', 'valid': True}} jsonrpc.Executor/4::DEBUG::2016-11-04 14:03:07,230::task::597::Storage.TaskManager.Task::(_updateState) Task=`657b8259-8c85-48d8-8107-8e70c0f5d5da`::moving from state preparing -> state finished jsonrpc.Executor/4::DEBUG::2016-11-04 14:03:07,230::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} jsonrpc.Executor/4::DEBUG::2016-11-04 14:03:07,230::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} jsonrpc.Executor/4::DEBUG::2016-11-04 14:03:07,230::task::995::Storage.TaskManager.Task::(_decref) Task=`657b8259-8c85-48d8-8107-8e70c0f5d5da`::ref 0 aborting False jsonrpc.Executor/4::DEBUG::2016-11-04 14:03:07,236::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getStats' in bridge with {'cpuStatistics': {'20': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '21': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '22': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '23': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '1': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '1.07', 'cpuIdle': '98.53'}, '0': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '3': {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.66'}, '2': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.74'}, '5': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '4': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '7': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '6': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '9': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '8': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '13': {'cpuUser': '0.47', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.33'}, '12': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '15': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '14': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '17': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '16': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '19': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '18': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31475'}, '0': {'memPercent': 5, 'memFree': '70634'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'rxRate': '0.01', 'vmCount': 1, 'memUsed': '3', 'storageDomains': {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000407565', 'lastCheck': '2.9', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000472804', 'lastCheck': '2.8', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000390698', 'lastCheck': '0.0', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000435236', 'lastCheck': '0.0', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'eno1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '11844', 'name': 'eno1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286186.524995, 'rx': '5300683', 'state': 'up'}, 'ovirtmgmt': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.1', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '31699913', 'txDropped': '0', 'sampleTime': 1478286186.524995, 'rx': '734352015', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '6615850', 'txDropped': '0', 'sampleTime': 1478286186.524995, 'rx': '6615850', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286186.524995, 'rx': '0', 'state': 'down'}, 'eno2': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno2', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286186.524995, 'rx': '0', 'state': 'down'}, 'vnet0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'vnet0', 'tx': '365725', 'txDropped': '0', 'sampleTime': 1478286186.524995, 'rx': '100632', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286186.524995, 'rx': '0', 'state': 'down'}, 'vdsmbr_AoB33GL7': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '4691', 'name': 'vdsmbr_AoB33GL7', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286186.524995, 'rx': '0', 'state': 'down'}, 'eno3': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno3', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286186.524995, 'rx': '0', 'state': 'down'}, 'eno4': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno4', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286186.524995, 'rx': '0', 'state': 'down'}, 'ens2f0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '10000', 'rxDropped': '0', 'name': 'ens2f0', 'tx': '32416594', 'txDropped': '0', 'sampleTime': 1478286186.524995, 'rx': '763264903', 'state': 'up'}, 'ens2f1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ens2f1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286186.524995, 'rx': '0', 'state': 'down'}}, 'txDropped': '0', 'cpuUser': '0.06', 'ksmPages': 100, 'elapsedTime': '6270.78', 'cpuLoad': '0.02', 'cpuSys': '0.07', 'diskStats': {'/var/log': {'free': '15189'}, '/var/log/core': {'free': '15189'}, '/var/run/vdsm/': {'free': '52254'}, '/tmp': {'free': '1640028'}}, 'cpuUserVdsmd': '0.33', 'netConfigDirty': 'False', 'memCommitted': 4161, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 98633, 'txRate': '0.00', 'bootTime': '1478279889', 'haStats': {'active': False, 'configured': False, 'score': 0, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'rxDropped': '16535', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'dateTime': '2016-11-04T19:03:07 GMT', 'anonHugePages': '902', 'memFree': 102313, 'cpuIdle': '99.87', 'vmActive': 1, 'v2vJobs': {}, 'cpuSysVdsmd': '0.67'} jsonrpc.Executor/4::INFO::2016-11-04 14:03:07,237::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getStats succeeded in 0.01 seconds Thread-112::DEBUG::2016-11-04 14:03:07,530::outOfProcess::77::Storage.oop::(cleanIdleIOProcesses) Removing idle ioprocess /rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01 Thread-112::DEBUG::2016-11-04 14:03:07,530::outOfProcess::77::Storage.oop::(cleanIdleIOProcesses) Removing idle ioprocess /rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports Thread-112::DEBUG::2016-11-04 14:03:07,530::outOfProcess::77::Storage.oop::(cleanIdleIOProcesses) Removing idle ioprocess /rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02 Thread-112::DEBUG::2016-11-04 14:03:07,530::outOfProcess::77::Storage.oop::(cleanIdleIOProcesses) Removing idle ioprocess /rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos Thread-112::INFO::2016-11-04 14:03:07,530::__init__::593::IOProcessClient::(close) Closing client ioprocess-72 Thread-112::INFO::2016-11-04 14:03:07,530::__init__::593::IOProcessClient::(close) Closing client ioprocess-69 Thread-112::INFO::2016-11-04 14:03:07,531::__init__::593::IOProcessClient::(close) Closing client ioprocess-71 Thread-112::INFO::2016-11-04 14:03:07,531::__init__::593::IOProcessClient::(close) Closing client ioprocess-70 jsonrpc.Executor/5::DEBUG::2016-11-04 14:03:08,272::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getAllVmStats' in bridge with {} jsonrpc.Executor/5::DEBUG::2016-11-04 14:03:08,273::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getAllVmStats' in bridge with (suppressed) jsonrpc.Executor/5::INFO::2016-11-04 14:03:08,274::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getAllVmStats succeeded in 0.01 seconds Thread-12::DEBUG::2016-11-04 14:03:14,270::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Reactor thread::INFO::2016-11-04 14:03:14,314::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58220 Reactor thread::DEBUG::2016-11-04 14:03:14,320::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:03:14,320::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58220 Reactor thread::DEBUG::2016-11-04 14:03:14,320::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58220) BindingXMLRPC::INFO::2016-11-04 14:03:14,320::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58220 Thread-1062::INFO::2016-11-04 14:03:14,321::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58220 started Thread-1062::DEBUG::2016-11-04 14:03:14,321::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call getAllVmStats with () {} Thread-1062::DEBUG::2016-11-04 14:03:14,322::bindingxmlrpc::1255::vds::(wrapper) return getAllVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} Thread-1062::INFO::2016-11-04 14:03:14,322::bindingxmlrpc::1282::vds::(wrapper) RPC call getAllVmStats finished (code=0) in 0.00 seconds Thread-12::DEBUG::2016-11-04 14:03:14,323::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:03:14,323::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000366973 s, 1.6 MB/s\n') elapsed=0.06 Thread-1062::INFO::2016-11-04 14:03:14,328::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58220 stopped Reactor thread::INFO::2016-11-04 14:03:14,330::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58222 Reactor thread::DEBUG::2016-11-04 14:03:14,335::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:03:14,336::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58222 Reactor thread::DEBUG::2016-11-04 14:03:14,336::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58222) BindingXMLRPC::INFO::2016-11-04 14:03:14,336::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58222 Thread-1063::INFO::2016-11-04 14:03:14,336::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58222 started Thread-1063::DEBUG::2016-11-04 14:03:14,337::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call vmGetIoTunePolicy with ('cddd2019-5baf-4321-aa8b-130b2889d1b0',) {} Thread-1063::DEBUG::2016-11-04 14:03:14,337::bindingxmlrpc::1255::vds::(wrapper) return vmGetIoTunePolicy with {'ioTunePolicy': [], 'status': {'message': 'Done', 'code': 0}} Thread-1063::INFO::2016-11-04 14:03:14,337::bindingxmlrpc::1282::vds::(wrapper) RPC call vmGetIoTunePolicy finished (code=0) in 0.00 seconds Thread-1063::INFO::2016-11-04 14:03:14,338::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58222 stopped Thread-12::DEBUG::2016-11-04 14:03:14,343::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:03:14,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:03:14,370::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000380521 s, 928 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:03:14,373::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:03:14,399::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000289939 s, 1.2 MB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:03:14,407::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000422236 s, 812 kB/s\n') elapsed=0.04 periodic/3::DEBUG::2016-11-04 14:03:21,596::task::597::Storage.TaskManager.Task::(_updateState) Task=`838c5ed7-1c39-4dbe-a449-6e73dd66caa5`::moving from state init -> state preparing periodic/3::INFO::2016-11-04 14:03:21,596::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) periodic/3::INFO::2016-11-04 14:03:21,597::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000380521', 'lastCheck': '4.1', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000422236', 'lastCheck': '4.1', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000289939', 'lastCheck': '4.4', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000366973', 'lastCheck': '4.4', 'valid': True}} periodic/3::DEBUG::2016-11-04 14:03:21,597::task::1193::Storage.TaskManager.Task::(prepare) Task=`838c5ed7-1c39-4dbe-a449-6e73dd66caa5`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000380521', 'lastCheck': '4.1', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000422236', 'lastCheck': '4.1', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000289939', 'lastCheck': '4.4', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000366973', 'lastCheck': '4.4', 'valid': True}} periodic/3::DEBUG::2016-11-04 14:03:21,597::task::597::Storage.TaskManager.Task::(_updateState) Task=`838c5ed7-1c39-4dbe-a449-6e73dd66caa5`::moving from state preparing -> state finished periodic/3::DEBUG::2016-11-04 14:03:21,597::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/3::DEBUG::2016-11-04 14:03:21,597::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/3::DEBUG::2016-11-04 14:03:21,597::task::995::Storage.TaskManager.Task::(_decref) Task=`838c5ed7-1c39-4dbe-a449-6e73dd66caa5`::ref 0 aborting False jsonrpc.Executor/6::DEBUG::2016-11-04 14:03:22,489::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getStats' in bridge with {} jsonrpc.Executor/6::DEBUG::2016-11-04 14:03:22,490::task::597::Storage.TaskManager.Task::(_updateState) Task=`7cd9562b-5b39-4523-a169-9193f0b37a16`::moving from state init -> state preparing jsonrpc.Executor/6::INFO::2016-11-04 14:03:22,490::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) jsonrpc.Executor/6::INFO::2016-11-04 14:03:22,490::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000380521', 'lastCheck': '4.9', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000422236', 'lastCheck': '5.0', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000289939', 'lastCheck': '5.3', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000366973', 'lastCheck': '5.3', 'valid': True}} jsonrpc.Executor/6::DEBUG::2016-11-04 14:03:22,490::task::1193::Storage.TaskManager.Task::(prepare) Task=`7cd9562b-5b39-4523-a169-9193f0b37a16`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000380521', 'lastCheck': '4.9', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000422236', 'lastCheck': '5.0', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000289939', 'lastCheck': '5.3', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000366973', 'lastCheck': '5.3', 'valid': True}} jsonrpc.Executor/6::DEBUG::2016-11-04 14:03:22,490::task::597::Storage.TaskManager.Task::(_updateState) Task=`7cd9562b-5b39-4523-a169-9193f0b37a16`::moving from state preparing -> state finished jsonrpc.Executor/6::DEBUG::2016-11-04 14:03:22,490::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} jsonrpc.Executor/6::DEBUG::2016-11-04 14:03:22,490::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} jsonrpc.Executor/6::DEBUG::2016-11-04 14:03:22,490::task::995::Storage.TaskManager.Task::(_decref) Task=`7cd9562b-5b39-4523-a169-9193f0b37a16`::ref 0 aborting False jsonrpc.Executor/6::DEBUG::2016-11-04 14:03:22,497::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getStats' in bridge with {'cpuStatistics': {'20': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '21': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '22': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '23': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '1': {'cpuUser': '0.47', 'nodeIndex': 0, 'cpuSys': '0.53', 'cpuIdle': '99.00'}, '0': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '3': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '2': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '5': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '4': {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.66'}, '7': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '6': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '9': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '8': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '13': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.47'}, '12': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '15': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '14': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '17': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '16': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '19': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '18': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31477'}, '0': {'memPercent': 5, 'memFree': '70635'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'rxRate': '0.00', 'vmCount': 1, 'memUsed': '3', 'storageDomains': {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000380521', 'lastCheck': '4.9', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000422236', 'lastCheck': '5.0', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000289939', 'lastCheck': '5.3', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000366973', 'lastCheck': '5.3', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'eno1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '11872', 'name': 'eno1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286201.529684, 'rx': '5313323', 'state': 'up'}, 'ovirtmgmt': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '31718909', 'txDropped': '0', 'sampleTime': 1478286201.529684, 'rx': '734362035', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '6634064', 'txDropped': '0', 'sampleTime': 1478286201.529684, 'rx': '6634064', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286201.529684, 'rx': '0', 'state': 'down'}, 'eno2': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno2', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286201.529684, 'rx': '0', 'state': 'down'}, 'vnet0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'vnet0', 'tx': '366565', 'txDropped': '0', 'sampleTime': 1478286201.529684, 'rx': '100632', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286201.529684, 'rx': '0', 'state': 'down'}, 'vdsmbr_AoB33GL7': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '4705', 'name': 'vdsmbr_AoB33GL7', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286201.529684, 'rx': '0', 'state': 'down'}, 'eno3': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno3', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286201.529684, 'rx': '0', 'state': 'down'}, 'eno4': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno4', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286201.529684, 'rx': '0', 'state': 'down'}, 'ens2f0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '10000', 'rxDropped': '0', 'name': 'ens2f0', 'tx': '32436059', 'txDropped': '0', 'sampleTime': 1478286201.529684, 'rx': '763275941', 'state': 'up'}, 'ens2f1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ens2f1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286201.529684, 'rx': '0', 'state': 'down'}}, 'txDropped': '0', 'cpuUser': '0.07', 'ksmPages': 100, 'elapsedTime': '6286.04', 'cpuLoad': '0.02', 'cpuSys': '0.04', 'diskStats': {'/var/log': {'free': '15189'}, '/var/log/core': {'free': '15189'}, '/var/run/vdsm/': {'free': '52254'}, '/tmp': {'free': '1640028'}}, 'cpuUserVdsmd': '0.40', 'netConfigDirty': 'False', 'memCommitted': 4161, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 98635, 'txRate': '0.00', 'bootTime': '1478279889', 'haStats': {'active': False, 'configured': False, 'score': 0, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'rxDropped': '16577', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'dateTime': '2016-11-04T19:03:22 GMT', 'anonHugePages': '902', 'memFree': 102316, 'cpuIdle': '99.89', 'vmActive': 1, 'v2vJobs': {}, 'cpuSysVdsmd': '0.33'} jsonrpc.Executor/6::INFO::2016-11-04 14:03:22,498::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getStats succeeded in 0.01 seconds Thread-12::DEBUG::2016-11-04 14:03:24,267::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:03:24,314::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:03:24,314::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000397177 s, 1.5 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:03:24,345::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:03:24,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:03:24,373::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000377942 s, 934 kB/s\n') elapsed=0.03 Thread-12::DEBUG::2016-11-04 14:03:24,374::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 jsonrpc.Executor/0::DEBUG::2016-11-04 14:03:24,402::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getAllVmStats' in bridge with {} jsonrpc.Executor/0::DEBUG::2016-11-04 14:03:24,406::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getAllVmStats' in bridge with (suppressed) jsonrpc.Executor/0::INFO::2016-11-04 14:03:24,407::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getAllVmStats succeeded in 0.01 seconds Thread-12::DEBUG::2016-11-04 14:03:24,408::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000543858 s, 633 kB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:03:24,416::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.00042449 s, 808 kB/s\n') elapsed=0.04 Reactor thread::INFO::2016-11-04 14:03:29,366::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58224 Reactor thread::DEBUG::2016-11-04 14:03:29,372::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:03:29,372::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58224 Reactor thread::DEBUG::2016-11-04 14:03:29,373::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58224) BindingXMLRPC::INFO::2016-11-04 14:03:29,373::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58224 Thread-1064::INFO::2016-11-04 14:03:29,373::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58224 started Thread-1064::DEBUG::2016-11-04 14:03:29,373::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call getAllVmStats with () {} Thread-1064::DEBUG::2016-11-04 14:03:29,374::bindingxmlrpc::1255::vds::(wrapper) return getAllVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} Thread-1064::INFO::2016-11-04 14:03:29,375::bindingxmlrpc::1282::vds::(wrapper) RPC call getAllVmStats finished (code=0) in 0.00 seconds Thread-1064::INFO::2016-11-04 14:03:29,380::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58224 stopped Reactor thread::INFO::2016-11-04 14:03:29,414::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58226 Reactor thread::DEBUG::2016-11-04 14:03:29,420::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:03:29,420::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58226 Reactor thread::DEBUG::2016-11-04 14:03:29,421::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58226) BindingXMLRPC::INFO::2016-11-04 14:03:29,421::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58226 Thread-1065::INFO::2016-11-04 14:03:29,421::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58226 started Thread-1065::DEBUG::2016-11-04 14:03:29,421::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call vmGetIoTunePolicy with ('cddd2019-5baf-4321-aa8b-130b2889d1b0',) {} Thread-1065::DEBUG::2016-11-04 14:03:29,422::bindingxmlrpc::1255::vds::(wrapper) return vmGetIoTunePolicy with {'ioTunePolicy': [], 'status': {'message': 'Done', 'code': 0}} Thread-1065::INFO::2016-11-04 14:03:29,422::bindingxmlrpc::1282::vds::(wrapper) RPC call vmGetIoTunePolicy finished (code=0) in 0.00 seconds Thread-1065::INFO::2016-11-04 14:03:29,423::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58226 stopped Thread-12::DEBUG::2016-11-04 14:03:34,265::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:03:34,313::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:03:34,314::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000455075 s, 1.3 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:03:34,344::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:03:34,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:03:34,371::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000514986 s, 685 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:03:34,373::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:03:34,394::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000516028 s, 667 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:03:34,443::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:03:34,443::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000586496 s, 585 kB/s\n') elapsed=0.07 periodic/1::DEBUG::2016-11-04 14:03:36,454::task::597::Storage.TaskManager.Task::(_updateState) Task=`86328956-cd28-405c-9d5c-26a54b604bdc`::moving from state init -> state preparing periodic/1::INFO::2016-11-04 14:03:36,455::logUtils::49::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'bfcabdd8-3aa7-4241-91c3-5db0e843e476', spUUID=u'00000001-0001-0001-0001-000000000366', imgUUID=u'b2109025-42be-4c40-ac2b-26baf8c304e5', volUUID=u'3153e04b-729c-44b2-b31a-0b2547ad3a77', options=None) periodic/1::INFO::2016-11-04 14:03:36,457::logUtils::52::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '17503478272', 'apparentsize': '21474836480'} periodic/1::DEBUG::2016-11-04 14:03:36,457::task::1193::Storage.TaskManager.Task::(prepare) Task=`86328956-cd28-405c-9d5c-26a54b604bdc`::finished: {'truesize': '17503478272', 'apparentsize': '21474836480'} periodic/1::DEBUG::2016-11-04 14:03:36,457::task::597::Storage.TaskManager.Task::(_updateState) Task=`86328956-cd28-405c-9d5c-26a54b604bdc`::moving from state preparing -> state finished periodic/1::DEBUG::2016-11-04 14:03:36,458::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/1::DEBUG::2016-11-04 14:03:36,458::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/1::DEBUG::2016-11-04 14:03:36,458::task::995::Storage.TaskManager.Task::(_decref) Task=`86328956-cd28-405c-9d5c-26a54b604bdc`::ref 0 aborting False periodic/3::DEBUG::2016-11-04 14:03:36,591::task::597::Storage.TaskManager.Task::(_updateState) Task=`fe01efcd-a8fc-4d64-bed4-e470b3bd2dcf`::moving from state init -> state preparing periodic/3::INFO::2016-11-04 14:03:36,591::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) periodic/3::INFO::2016-11-04 14:03:36,591::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000514986', 'lastCheck': '2.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000586496', 'lastCheck': '2.1', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000516028', 'lastCheck': '2.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000455075', 'lastCheck': '2.3', 'valid': True}} periodic/3::DEBUG::2016-11-04 14:03:36,591::task::1193::Storage.TaskManager.Task::(prepare) Task=`fe01efcd-a8fc-4d64-bed4-e470b3bd2dcf`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000514986', 'lastCheck': '2.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000586496', 'lastCheck': '2.1', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000516028', 'lastCheck': '2.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000455075', 'lastCheck': '2.3', 'valid': True}} periodic/3::DEBUG::2016-11-04 14:03:36,591::task::597::Storage.TaskManager.Task::(_updateState) Task=`fe01efcd-a8fc-4d64-bed4-e470b3bd2dcf`::moving from state preparing -> state finished periodic/3::DEBUG::2016-11-04 14:03:36,591::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/3::DEBUG::2016-11-04 14:03:36,591::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/3::DEBUG::2016-11-04 14:03:36,591::task::995::Storage.TaskManager.Task::(_decref) Task=`fe01efcd-a8fc-4d64-bed4-e470b3bd2dcf`::ref 0 aborting False jsonrpc.Executor/2::DEBUG::2016-11-04 14:03:38,136::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getStats' in bridge with {} jsonrpc.Executor/2::DEBUG::2016-11-04 14:03:38,137::task::597::Storage.TaskManager.Task::(_updateState) Task=`d652ba85-e468-45f8-8dd3-7779c7d17878`::moving from state init -> state preparing jsonrpc.Executor/2::INFO::2016-11-04 14:03:38,137::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) jsonrpc.Executor/2::INFO::2016-11-04 14:03:38,138::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000514986', 'lastCheck': '0.6', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000586496', 'lastCheck': '0.6', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000516028', 'lastCheck': '0.9', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000455075', 'lastCheck': '0.9', 'valid': True}} jsonrpc.Executor/2::DEBUG::2016-11-04 14:03:38,138::task::1193::Storage.TaskManager.Task::(prepare) Task=`d652ba85-e468-45f8-8dd3-7779c7d17878`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000514986', 'lastCheck': '0.6', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000586496', 'lastCheck': '0.6', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000516028', 'lastCheck': '0.9', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000455075', 'lastCheck': '0.9', 'valid': True}} jsonrpc.Executor/2::DEBUG::2016-11-04 14:03:38,138::task::597::Storage.TaskManager.Task::(_updateState) Task=`d652ba85-e468-45f8-8dd3-7779c7d17878`::moving from state preparing -> state finished jsonrpc.Executor/2::DEBUG::2016-11-04 14:03:38,138::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} jsonrpc.Executor/2::DEBUG::2016-11-04 14:03:38,138::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} jsonrpc.Executor/2::DEBUG::2016-11-04 14:03:38,138::task::995::Storage.TaskManager.Task::(_decref) Task=`d652ba85-e468-45f8-8dd3-7779c7d17878`::ref 0 aborting False jsonrpc.Executor/2::DEBUG::2016-11-04 14:03:38,145::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getStats' in bridge with {'cpuStatistics': {'20': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '21': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '22': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '23': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '1': {'cpuUser': '0.47', 'nodeIndex': 0, 'cpuSys': '1.07', 'cpuIdle': '98.46'}, '0': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '3': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '2': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '5': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '4': {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.66'}, '7': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '6': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '9': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '8': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '13': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.47'}, '12': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '15': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '14': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '17': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '16': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '19': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '18': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31476'}, '0': {'memPercent': 5, 'memFree': '70635'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'rxRate': '0.01', 'vmCount': 1, 'memUsed': '3', 'storageDomains': {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000514986', 'lastCheck': '0.6', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000586496', 'lastCheck': '0.6', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000516028', 'lastCheck': '0.9', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000455075', 'lastCheck': '0.9', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'eno1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '11900', 'name': 'eno1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286216.52452, 'rx': '5325901', 'state': 'up'}, 'ovirtmgmt': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.1', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '31793783', 'txDropped': '0', 'sampleTime': 1478286216.52452, 'rx': '736491003', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '6652262', 'txDropped': '0', 'sampleTime': 1478286216.52452, 'rx': '6652262', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286216.52452, 'rx': '0', 'state': 'down'}, 'eno2': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno2', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286216.52452, 'rx': '0', 'state': 'down'}, 'vnet0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'vnet0', 'tx': '366805', 'txDropped': '0', 'sampleTime': 1478286216.52452, 'rx': '100632', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286216.52452, 'rx': '0', 'state': 'down'}, 'vdsmbr_AoB33GL7': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '4709', 'name': 'vdsmbr_AoB33GL7', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286216.52452, 'rx': '0', 'state': 'down'}, 'eno3': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno3', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286216.52452, 'rx': '0', 'state': 'down'}, 'eno4': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno4', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286216.52452, 'rx': '0', 'state': 'down'}, 'ens2f0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '10000', 'rxDropped': '0', 'name': 'ens2f0', 'tx': '32512921', 'txDropped': '0', 'sampleTime': 1478286216.52452, 'rx': '765493097', 'state': 'up'}, 'ens2f1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ens2f1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286216.52452, 'rx': '0', 'state': 'down'}}, 'txDropped': '0', 'cpuUser': '0.06', 'ksmPages': 100, 'elapsedTime': '6301.69', 'cpuLoad': '0.02', 'cpuSys': '0.06', 'diskStats': {'/var/log': {'free': '15189'}, '/var/log/core': {'free': '15189'}, '/var/run/vdsm/': {'free': '52254'}, '/tmp': {'free': '1640028'}}, 'cpuUserVdsmd': '0.47', 'netConfigDirty': 'False', 'memCommitted': 4161, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 98635, 'txRate': '0.00', 'bootTime': '1478279889', 'haStats': {'active': False, 'configured': False, 'score': 0, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'rxDropped': '16609', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'dateTime': '2016-11-04T19:03:38 GMT', 'anonHugePages': '902', 'memFree': 102316, 'cpuIdle': '99.87', 'vmActive': 1, 'v2vJobs': {}, 'cpuSysVdsmd': '0.60'} jsonrpc.Executor/2::INFO::2016-11-04 14:03:38,146::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getStats succeeded in 0.01 seconds jsonrpc.Executor/7::DEBUG::2016-11-04 14:03:40,457::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getAllVmStats' in bridge with {} jsonrpc.Executor/7::DEBUG::2016-11-04 14:03:40,458::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getAllVmStats' in bridge with (suppressed) jsonrpc.Executor/7::INFO::2016-11-04 14:03:40,459::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getAllVmStats succeeded in 0.00 seconds Thread-12::DEBUG::2016-11-04 14:03:44,263::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:03:44,313::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:03:44,314::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000459226 s, 1.3 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:03:44,344::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:03:44,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:03:44,370::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000451093 s, 783 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:03:44,373::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:03:44,393::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000536908 s, 641 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:03:44,405::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000399847 s, 858 kB/s\n') elapsed=0.04 Reactor thread::INFO::2016-11-04 14:03:44,427::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58228 Reactor thread::DEBUG::2016-11-04 14:03:44,433::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:03:44,434::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58228 Reactor thread::DEBUG::2016-11-04 14:03:44,434::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58228) BindingXMLRPC::INFO::2016-11-04 14:03:44,434::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58228 Thread-1066::INFO::2016-11-04 14:03:44,434::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58228 started Thread-1066::DEBUG::2016-11-04 14:03:44,435::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call getAllVmStats with () {} Thread-1066::DEBUG::2016-11-04 14:03:44,436::bindingxmlrpc::1255::vds::(wrapper) return getAllVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} Thread-1066::INFO::2016-11-04 14:03:44,436::bindingxmlrpc::1282::vds::(wrapper) RPC call getAllVmStats finished (code=0) in 0.00 seconds Thread-1066::INFO::2016-11-04 14:03:44,442::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58228 stopped Reactor thread::INFO::2016-11-04 14:03:44,443::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58230 Reactor thread::DEBUG::2016-11-04 14:03:44,449::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:03:44,449::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58230 Reactor thread::DEBUG::2016-11-04 14:03:44,449::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58230) BindingXMLRPC::INFO::2016-11-04 14:03:44,450::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58230 Thread-1067::INFO::2016-11-04 14:03:44,450::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58230 started Thread-1067::DEBUG::2016-11-04 14:03:44,450::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call vmGetIoTunePolicy with ('cddd2019-5baf-4321-aa8b-130b2889d1b0',) {} Thread-1067::DEBUG::2016-11-04 14:03:44,451::bindingxmlrpc::1255::vds::(wrapper) return vmGetIoTunePolicy with {'ioTunePolicy': [], 'status': {'message': 'Done', 'code': 0}} Thread-1067::INFO::2016-11-04 14:03:44,451::bindingxmlrpc::1282::vds::(wrapper) RPC call vmGetIoTunePolicy finished (code=0) in 0.00 seconds Thread-1067::INFO::2016-11-04 14:03:44,452::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58230 stopped periodic/2::DEBUG::2016-11-04 14:03:51,596::task::597::Storage.TaskManager.Task::(_updateState) Task=`1d6c2e67-6c7b-486d-9092-45f4e6c7befa`::moving from state init -> state preparing periodic/2::INFO::2016-11-04 14:03:51,596::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) periodic/2::INFO::2016-11-04 14:03:51,596::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000451093', 'lastCheck': '4.0', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000399847', 'lastCheck': '4.1', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000536908', 'lastCheck': '4.4', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000459226', 'lastCheck': '4.4', 'valid': True}} periodic/2::DEBUG::2016-11-04 14:03:51,596::task::1193::Storage.TaskManager.Task::(prepare) Task=`1d6c2e67-6c7b-486d-9092-45f4e6c7befa`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000451093', 'lastCheck': '4.0', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000399847', 'lastCheck': '4.1', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000536908', 'lastCheck': '4.4', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000459226', 'lastCheck': '4.4', 'valid': True}} periodic/2::DEBUG::2016-11-04 14:03:51,596::task::597::Storage.TaskManager.Task::(_updateState) Task=`1d6c2e67-6c7b-486d-9092-45f4e6c7befa`::moving from state preparing -> state finished periodic/2::DEBUG::2016-11-04 14:03:51,596::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/2::DEBUG::2016-11-04 14:03:51,596::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/2::DEBUG::2016-11-04 14:03:51,596::task::995::Storage.TaskManager.Task::(_decref) Task=`1d6c2e67-6c7b-486d-9092-45f4e6c7befa`::ref 0 aborting False jsonrpc.Executor/1::DEBUG::2016-11-04 14:03:53,700::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getStats' in bridge with {} jsonrpc.Executor/1::DEBUG::2016-11-04 14:03:53,701::task::597::Storage.TaskManager.Task::(_updateState) Task=`38484da1-5663-4154-82f6-b24512be9077`::moving from state init -> state preparing jsonrpc.Executor/1::INFO::2016-11-04 14:03:53,701::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) jsonrpc.Executor/1::INFO::2016-11-04 14:03:53,701::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000451093', 'lastCheck': '6.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000399847', 'lastCheck': '6.2', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000536908', 'lastCheck': '6.5', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000459226', 'lastCheck': '6.5', 'valid': True}} jsonrpc.Executor/1::DEBUG::2016-11-04 14:03:53,701::task::1193::Storage.TaskManager.Task::(prepare) Task=`38484da1-5663-4154-82f6-b24512be9077`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000451093', 'lastCheck': '6.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000399847', 'lastCheck': '6.2', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000536908', 'lastCheck': '6.5', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000459226', 'lastCheck': '6.5', 'valid': True}} jsonrpc.Executor/1::DEBUG::2016-11-04 14:03:53,702::task::597::Storage.TaskManager.Task::(_updateState) Task=`38484da1-5663-4154-82f6-b24512be9077`::moving from state preparing -> state finished jsonrpc.Executor/1::DEBUG::2016-11-04 14:03:53,702::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} jsonrpc.Executor/1::DEBUG::2016-11-04 14:03:53,702::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} jsonrpc.Executor/1::DEBUG::2016-11-04 14:03:53,702::task::995::Storage.TaskManager.Task::(_decref) Task=`38484da1-5663-4154-82f6-b24512be9077`::ref 0 aborting False jsonrpc.Executor/1::DEBUG::2016-11-04 14:03:53,708::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getStats' in bridge with {'cpuStatistics': {'20': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '21': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '22': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '23': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '1': {'cpuUser': '0.60', 'nodeIndex': 0, 'cpuSys': '0.60', 'cpuIdle': '98.80'}, '0': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '3': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '2': {'cpuUser': '0.87', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.06'}, '5': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '4': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '7': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '6': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '9': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '8': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '13': {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.66'}, '12': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '15': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '14': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '17': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '16': {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.73'}, '19': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '18': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31475'}, '0': {'memPercent': 5, 'memFree': '70637'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'rxRate': '0.02', 'vmCount': 1, 'memUsed': '3', 'storageDomains': {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000451093', 'lastCheck': '6.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000399847', 'lastCheck': '6.2', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000536908', 'lastCheck': '6.5', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000459226', 'lastCheck': '6.5', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'eno1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '11930', 'name': 'eno1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286231.529129, 'rx': '5337462', 'state': 'up'}, 'ovirtmgmt': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.2', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '31847573', 'txDropped': '0', 'sampleTime': 1478286231.529129, 'rx': '740010009', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '6670492', 'txDropped': '0', 'sampleTime': 1478286231.529129, 'rx': '6670492', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286231.529129, 'rx': '0', 'state': 'down'}, 'eno2': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno2', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286231.529129, 'rx': '0', 'state': 'down'}, 'vnet0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'vnet0', 'tx': '369553', 'txDropped': '0', 'sampleTime': 1478286231.529129, 'rx': '102900', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286231.529129, 'rx': '0', 'state': 'down'}, 'vdsmbr_AoB33GL7': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '4717', 'name': 'vdsmbr_AoB33GL7', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286231.529129, 'rx': '0', 'state': 'down'}, 'eno3': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno3', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286231.529129, 'rx': '0', 'state': 'down'}, 'eno4': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno4', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286231.529129, 'rx': '0', 'state': 'down'}, 'ens2f0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '10000', 'rxDropped': '0', 'name': 'ens2f0', 'tx': '32569448', 'txDropped': '0', 'sampleTime': 1478286231.529129, 'rx': '769151283', 'state': 'up'}, 'ens2f1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ens2f1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286231.529129, 'rx': '0', 'state': 'down'}}, 'txDropped': '0', 'cpuUser': '0.09', 'ksmPages': 100, 'elapsedTime': '6317.25', 'cpuLoad': '0.02', 'cpuSys': '0.04', 'diskStats': {'/var/log': {'free': '15189'}, '/var/log/core': {'free': '15189'}, '/var/run/vdsm/': {'free': '52254'}, '/tmp': {'free': '1640028'}}, 'cpuUserVdsmd': '0.40', 'netConfigDirty': 'False', 'memCommitted': 4161, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 98635, 'txRate': '0.00', 'bootTime': '1478279889', 'haStats': {'active': False, 'configured': False, 'score': 0, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'rxDropped': '16647', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'dateTime': '2016-11-04T19:03:53 GMT', 'anonHugePages': '902', 'memFree': 102315, 'cpuIdle': '99.86', 'vmActive': 1, 'v2vJobs': {}, 'cpuSysVdsmd': '0.40'} jsonrpc.Executor/1::INFO::2016-11-04 14:03:53,709::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getStats succeeded in 0.01 seconds Thread-12::DEBUG::2016-11-04 14:03:54,266::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:03:54,313::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:03:54,313::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000484481 s, 1.2 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:03:54,344::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:03:54,353::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:03:54,371::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000353976 s, 997 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:03:54,374::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:03:54,398::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000512147 s, 672 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:03:54,407::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000376294 s, 912 kB/s\n') elapsed=0.03 jsonrpc.Executor/3::DEBUG::2016-11-04 14:03:55,517::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getAllVmStats' in bridge with {} jsonrpc.Executor/3::DEBUG::2016-11-04 14:03:55,518::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getAllVmStats' in bridge with (suppressed) jsonrpc.Executor/3::INFO::2016-11-04 14:03:55,519::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getAllVmStats succeeded in 0.00 seconds Reactor thread::INFO::2016-11-04 14:03:59,482::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58232 Reactor thread::DEBUG::2016-11-04 14:03:59,488::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:03:59,488::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58232 Reactor thread::DEBUG::2016-11-04 14:03:59,489::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58232) BindingXMLRPC::INFO::2016-11-04 14:03:59,489::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58232 Thread-1068::INFO::2016-11-04 14:03:59,489::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58232 started Thread-1068::DEBUG::2016-11-04 14:03:59,490::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call getAllVmStats with () {} Thread-1068::DEBUG::2016-11-04 14:03:59,491::bindingxmlrpc::1255::vds::(wrapper) return getAllVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} Thread-1068::INFO::2016-11-04 14:03:59,491::bindingxmlrpc::1282::vds::(wrapper) RPC call getAllVmStats finished (code=0) in 0.00 seconds Thread-1068::INFO::2016-11-04 14:03:59,496::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58232 stopped Reactor thread::INFO::2016-11-04 14:03:59,497::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58234 Reactor thread::DEBUG::2016-11-04 14:03:59,503::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:03:59,503::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58234 Reactor thread::DEBUG::2016-11-04 14:03:59,503::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58234) BindingXMLRPC::INFO::2016-11-04 14:03:59,504::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58234 Thread-1069::INFO::2016-11-04 14:03:59,504::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58234 started Thread-1069::DEBUG::2016-11-04 14:03:59,504::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call vmGetIoTunePolicy with ('cddd2019-5baf-4321-aa8b-130b2889d1b0',) {} Thread-1069::DEBUG::2016-11-04 14:03:59,505::bindingxmlrpc::1255::vds::(wrapper) return vmGetIoTunePolicy with {'ioTunePolicy': [], 'status': {'message': 'Done', 'code': 0}} Thread-1069::INFO::2016-11-04 14:03:59,505::bindingxmlrpc::1282::vds::(wrapper) RPC call vmGetIoTunePolicy finished (code=0) in 0.00 seconds Thread-1069::INFO::2016-11-04 14:03:59,506::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58234 stopped Thread-12::DEBUG::2016-11-04 14:04:04,268::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:04:04,313::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:04:04,313::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000466977 s, 1.3 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:04:04,344::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:04:04,353::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:04:04,370::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000475521 s, 742 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:04:04,373::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:04:04,396::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000512826 s, 671 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:04:04,405::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000374777 s, 915 kB/s\n') elapsed=0.04 periodic/0::DEBUG::2016-11-04 14:04:06,590::task::597::Storage.TaskManager.Task::(_updateState) Task=`ec5fd6b8-3bc1-481a-ae19-20ca8ed2f380`::moving from state init -> state preparing periodic/0::INFO::2016-11-04 14:04:06,590::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) periodic/0::INFO::2016-11-04 14:04:06,591::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000475521', 'lastCheck': '2.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000374777', 'lastCheck': '2.2', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000512826', 'lastCheck': '2.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000466977', 'lastCheck': '2.3', 'valid': True}} periodic/0::DEBUG::2016-11-04 14:04:06,591::task::1193::Storage.TaskManager.Task::(prepare) Task=`ec5fd6b8-3bc1-481a-ae19-20ca8ed2f380`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000475521', 'lastCheck': '2.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000374777', 'lastCheck': '2.2', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000512826', 'lastCheck': '2.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000466977', 'lastCheck': '2.3', 'valid': True}} periodic/0::DEBUG::2016-11-04 14:04:06,591::task::597::Storage.TaskManager.Task::(_updateState) Task=`ec5fd6b8-3bc1-481a-ae19-20ca8ed2f380`::moving from state preparing -> state finished periodic/0::DEBUG::2016-11-04 14:04:06,591::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/0::DEBUG::2016-11-04 14:04:06,591::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/0::DEBUG::2016-11-04 14:04:06,591::task::995::Storage.TaskManager.Task::(_decref) Task=`ec5fd6b8-3bc1-481a-ae19-20ca8ed2f380`::ref 0 aborting False jsonrpc.Executor/4::DEBUG::2016-11-04 14:04:09,664::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getStats' in bridge with {} jsonrpc.Executor/4::DEBUG::2016-11-04 14:04:09,665::task::597::Storage.TaskManager.Task::(_updateState) Task=`76c50c03-ce10-4157-a115-0815bd98d5ef`::moving from state init -> state preparing jsonrpc.Executor/4::INFO::2016-11-04 14:04:09,665::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) jsonrpc.Executor/4::INFO::2016-11-04 14:04:09,665::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000475521', 'lastCheck': '2.1', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000374777', 'lastCheck': '2.1', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000512826', 'lastCheck': '2.4', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000466977', 'lastCheck': '2.4', 'valid': True}} jsonrpc.Executor/4::DEBUG::2016-11-04 14:04:09,665::task::1193::Storage.TaskManager.Task::(prepare) Task=`76c50c03-ce10-4157-a115-0815bd98d5ef`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000475521', 'lastCheck': '2.1', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000374777', 'lastCheck': '2.1', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000512826', 'lastCheck': '2.4', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000466977', 'lastCheck': '2.4', 'valid': True}} jsonrpc.Executor/4::DEBUG::2016-11-04 14:04:09,665::task::597::Storage.TaskManager.Task::(_updateState) Task=`76c50c03-ce10-4157-a115-0815bd98d5ef`::moving from state preparing -> state finished jsonrpc.Executor/4::DEBUG::2016-11-04 14:04:09,665::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} jsonrpc.Executor/4::DEBUG::2016-11-04 14:04:09,665::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} jsonrpc.Executor/4::DEBUG::2016-11-04 14:04:09,666::task::995::Storage.TaskManager.Task::(_decref) Task=`76c50c03-ce10-4157-a115-0815bd98d5ef`::ref 0 aborting False jsonrpc.Executor/4::DEBUG::2016-11-04 14:04:09,672::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getStats' in bridge with {'cpuStatistics': {'20': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '21': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '22': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '23': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '1': {'cpuUser': '0.47', 'nodeIndex': 0, 'cpuSys': '1.00', 'cpuIdle': '98.53'}, '0': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '3': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '2': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '5': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '4': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '7': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '6': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '9': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '8': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '13': {'cpuUser': '0.47', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.40'}, '12': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '15': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '14': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '17': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '16': {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.73'}, '19': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '18': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31474'}, '0': {'memPercent': 5, 'memFree': '70637'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'rxRate': '0.01', 'vmCount': 1, 'memUsed': '3', 'storageDomains': {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000475521', 'lastCheck': '2.1', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000374777', 'lastCheck': '2.1', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000512826', 'lastCheck': '2.4', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000466977', 'lastCheck': '2.4', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'eno1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '11958', 'name': 'eno1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286246.5238, 'rx': '5349663', 'state': 'up'}, 'ovirtmgmt': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.1', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '31880855', 'txDropped': '0', 'sampleTime': 1478286246.5238, 'rx': '742130327', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '6688722', 'txDropped': '0', 'sampleTime': 1478286246.5238, 'rx': '6688722', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286246.5238, 'rx': '0', 'state': 'down'}, 'eno2': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno2', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286246.5238, 'rx': '0', 'state': 'down'}, 'vnet0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'vnet0', 'tx': '369973', 'txDropped': '0', 'sampleTime': 1478286246.5238, 'rx': '102942', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286246.5238, 'rx': '0', 'state': 'down'}, 'vdsmbr_AoB33GL7': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '4723', 'name': 'vdsmbr_AoB33GL7', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286246.5238, 'rx': '0', 'state': 'down'}, 'eno3': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno3', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286246.5238, 'rx': '0', 'state': 'down'}, 'eno4': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno4', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286246.5238, 'rx': '0', 'state': 'down'}, 'ens2f0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '10000', 'rxDropped': '0', 'name': 'ens2f0', 'tx': '32602896', 'txDropped': '0', 'sampleTime': 1478286246.5238, 'rx': '771357373', 'state': 'up'}, 'ens2f1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ens2f1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286246.5238, 'rx': '0', 'state': 'down'}}, 'txDropped': '0', 'cpuUser': '0.06', 'ksmPages': 100, 'elapsedTime': '6333.22', 'cpuLoad': '0.01', 'cpuSys': '0.06', 'diskStats': {'/var/log': {'free': '15189'}, '/var/log/core': {'free': '15189'}, '/var/run/vdsm/': {'free': '52254'}, '/tmp': {'free': '1640028'}}, 'cpuUserVdsmd': '0.40', 'netConfigDirty': 'False', 'memCommitted': 4161, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 98634, 'txRate': '0.00', 'bootTime': '1478279889', 'haStats': {'active': False, 'configured': False, 'score': 0, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'rxDropped': '16681', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'dateTime': '2016-11-04T19:04:09 GMT', 'anonHugePages': '902', 'memFree': 102315, 'cpuIdle': '99.87', 'vmActive': 1, 'v2vJobs': {}, 'cpuSysVdsmd': '0.53'} jsonrpc.Executor/4::INFO::2016-11-04 14:04:09,673::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getStats succeeded in 0.01 seconds jsonrpc.Executor/5::DEBUG::2016-11-04 14:04:11,557::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getAllVmStats' in bridge with {} jsonrpc.Executor/5::DEBUG::2016-11-04 14:04:11,558::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getAllVmStats' in bridge with (suppressed) jsonrpc.Executor/5::INFO::2016-11-04 14:04:11,558::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getAllVmStats succeeded in 0.00 seconds Thread-12::DEBUG::2016-11-04 14:04:14,266::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:04:14,314::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:04:14,314::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000302886 s, 2.0 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:04:14,345::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:04:14,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:04:14,372::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000463085 s, 762 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:04:14,374::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:04:14,397::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000416953 s, 825 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:04:14,407::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000359891 s, 953 kB/s\n') elapsed=0.03 Reactor thread::INFO::2016-11-04 14:04:14,537::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58236 Reactor thread::DEBUG::2016-11-04 14:04:14,543::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:04:14,543::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58236 Reactor thread::DEBUG::2016-11-04 14:04:14,543::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58236) BindingXMLRPC::INFO::2016-11-04 14:04:14,544::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58236 Thread-1070::INFO::2016-11-04 14:04:14,544::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58236 started Thread-1070::DEBUG::2016-11-04 14:04:14,544::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call getAllVmStats with () {} Thread-1070::DEBUG::2016-11-04 14:04:14,545::bindingxmlrpc::1255::vds::(wrapper) return getAllVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} Thread-1070::INFO::2016-11-04 14:04:14,546::bindingxmlrpc::1282::vds::(wrapper) RPC call getAllVmStats finished (code=0) in 0.00 seconds Thread-1070::INFO::2016-11-04 14:04:14,551::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58236 stopped Reactor thread::INFO::2016-11-04 14:04:14,552::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58238 Reactor thread::DEBUG::2016-11-04 14:04:14,558::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:04:14,558::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58238 Reactor thread::DEBUG::2016-11-04 14:04:14,559::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58238) BindingXMLRPC::INFO::2016-11-04 14:04:14,559::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58238 Thread-1071::INFO::2016-11-04 14:04:14,559::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58238 started Thread-1071::DEBUG::2016-11-04 14:04:14,559::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call vmGetIoTunePolicy with ('cddd2019-5baf-4321-aa8b-130b2889d1b0',) {} Thread-1071::DEBUG::2016-11-04 14:04:14,560::bindingxmlrpc::1255::vds::(wrapper) return vmGetIoTunePolicy with {'ioTunePolicy': [], 'status': {'message': 'Done', 'code': 0}} Thread-1071::INFO::2016-11-04 14:04:14,560::bindingxmlrpc::1282::vds::(wrapper) RPC call vmGetIoTunePolicy finished (code=0) in 0.00 seconds Thread-1071::INFO::2016-11-04 14:04:14,561::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58238 stopped periodic/1::DEBUG::2016-11-04 14:04:21,594::task::597::Storage.TaskManager.Task::(_updateState) Task=`76c50c67-8b59-47e0-a1cf-7918354498ca`::moving from state init -> state preparing periodic/1::INFO::2016-11-04 14:04:21,594::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) periodic/1::INFO::2016-11-04 14:04:21,594::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000463085', 'lastCheck': '4.0', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000359891', 'lastCheck': '4.0', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000416953', 'lastCheck': '4.4', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000302886', 'lastCheck': '4.3', 'valid': True}} periodic/1::DEBUG::2016-11-04 14:04:21,595::task::1193::Storage.TaskManager.Task::(prepare) Task=`76c50c67-8b59-47e0-a1cf-7918354498ca`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000463085', 'lastCheck': '4.0', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000359891', 'lastCheck': '4.0', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000416953', 'lastCheck': '4.4', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000302886', 'lastCheck': '4.3', 'valid': True}} periodic/1::DEBUG::2016-11-04 14:04:21,595::task::597::Storage.TaskManager.Task::(_updateState) Task=`76c50c67-8b59-47e0-a1cf-7918354498ca`::moving from state preparing -> state finished periodic/1::DEBUG::2016-11-04 14:04:21,595::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/1::DEBUG::2016-11-04 14:04:21,595::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/1::DEBUG::2016-11-04 14:04:21,595::task::995::Storage.TaskManager.Task::(_decref) Task=`76c50c67-8b59-47e0-a1cf-7918354498ca`::ref 0 aborting False Thread-12::DEBUG::2016-11-04 14:04:24,265::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:04:24,314::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:04:24,314::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000341658 s, 1.8 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:04:24,345::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:04:24,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:04:24,371::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000471494 s, 749 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:04:24,373::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:04:24,396::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000546067 s, 630 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:04:24,405::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000386682 s, 887 kB/s\n') elapsed=0.03 jsonrpc.Executor/6::DEBUG::2016-11-04 14:04:25,888::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getStats' in bridge with {} jsonrpc.Executor/6::DEBUG::2016-11-04 14:04:25,889::task::597::Storage.TaskManager.Task::(_updateState) Task=`2119af32-1b66-4b17-9b04-61c5338dc93c`::moving from state init -> state preparing jsonrpc.Executor/6::INFO::2016-11-04 14:04:25,889::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) jsonrpc.Executor/6::INFO::2016-11-04 14:04:25,890::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000471494', 'lastCheck': '1.5', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000386682', 'lastCheck': '1.5', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000546067', 'lastCheck': '1.5', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000341658', 'lastCheck': '1.6', 'valid': True}} jsonrpc.Executor/6::DEBUG::2016-11-04 14:04:25,890::task::1193::Storage.TaskManager.Task::(prepare) Task=`2119af32-1b66-4b17-9b04-61c5338dc93c`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000471494', 'lastCheck': '1.5', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000386682', 'lastCheck': '1.5', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000546067', 'lastCheck': '1.5', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000341658', 'lastCheck': '1.6', 'valid': True}} jsonrpc.Executor/6::DEBUG::2016-11-04 14:04:25,890::task::597::Storage.TaskManager.Task::(_updateState) Task=`2119af32-1b66-4b17-9b04-61c5338dc93c`::moving from state preparing -> state finished jsonrpc.Executor/6::DEBUG::2016-11-04 14:04:25,890::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} jsonrpc.Executor/6::DEBUG::2016-11-04 14:04:25,890::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} jsonrpc.Executor/6::DEBUG::2016-11-04 14:04:25,890::task::995::Storage.TaskManager.Task::(_decref) Task=`2119af32-1b66-4b17-9b04-61c5338dc93c`::ref 0 aborting False jsonrpc.Executor/6::DEBUG::2016-11-04 14:04:25,897::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getStats' in bridge with {'cpuStatistics': {'20': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '21': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '22': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '23': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '1': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.53', 'cpuIdle': '99.07'}, '0': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '3': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '2': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '5': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '4': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '7': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '6': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '9': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '8': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '11': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '10': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '13': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '12': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '15': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '14': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '17': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '16': {'cpuUser': '0.20', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.73'}, '19': {'cpuUser': '0.20', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.73'}, '18': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.86'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31474'}, '0': {'memPercent': 5, 'memFree': '70637'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'rxRate': '0.00', 'vmCount': 1, 'memUsed': '3', 'storageDomains': {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000471494', 'lastCheck': '1.5', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000386682', 'lastCheck': '1.5', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000546067', 'lastCheck': '1.5', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000341658', 'lastCheck': '1.6', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'eno1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '11986', 'name': 'eno1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286261.528101, 'rx': '5362897', 'state': 'up'}, 'ovirtmgmt': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '31907961', 'txDropped': '0', 'sampleTime': 1478286261.528101, 'rx': '742140143', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '6706952', 'txDropped': '0', 'sampleTime': 1478286261.528101, 'rx': '6706952', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286261.528101, 'rx': '0', 'state': 'down'}, 'eno2': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno2', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286261.528101, 'rx': '0', 'state': 'down'}, 'vnet0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'vnet0', 'tx': '370693', 'txDropped': '0', 'sampleTime': 1478286261.528101, 'rx': '102942', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286261.528101, 'rx': '0', 'state': 'down'}, 'vdsmbr_AoB33GL7': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '4735', 'name': 'vdsmbr_AoB33GL7', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286261.528101, 'rx': '0', 'state': 'down'}, 'eno3': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno3', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286261.528101, 'rx': '0', 'state': 'down'}, 'eno4': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno4', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286261.528101, 'rx': '0', 'state': 'down'}, 'ens2f0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '10000', 'rxDropped': '0', 'name': 'ens2f0', 'tx': '32630735', 'txDropped': '0', 'sampleTime': 1478286261.528101, 'rx': '771368207', 'state': 'up'}, 'ens2f1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ens2f1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286261.528101, 'rx': '0', 'state': 'down'}}, 'txDropped': '0', 'cpuUser': '0.06', 'ksmPages': 100, 'elapsedTime': '6349.44', 'cpuLoad': '0.01', 'cpuSys': '0.05', 'diskStats': {'/var/log': {'free': '15189'}, '/var/log/core': {'free': '15189'}, '/var/run/vdsm/': {'free': '52254'}, '/tmp': {'free': '1640028'}}, 'cpuUserVdsmd': '0.40', 'netConfigDirty': 'False', 'memCommitted': 4161, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 98634, 'txRate': '0.00', 'bootTime': '1478279889', 'haStats': {'active': False, 'configured': False, 'score': 0, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'rxDropped': '16721', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'dateTime': '2016-11-04T19:04:25 GMT', 'anonHugePages': '902', 'memFree': 102315, 'cpuIdle': '99.89', 'vmActive': 1, 'v2vJobs': {}, 'cpuSysVdsmd': '0.40'} jsonrpc.Executor/6::INFO::2016-11-04 14:04:25,898::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getStats succeeded in 0.01 seconds jsonrpc.Executor/0::DEBUG::2016-11-04 14:04:27,611::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getAllVmStats' in bridge with {} jsonrpc.Executor/0::DEBUG::2016-11-04 14:04:27,612::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getAllVmStats' in bridge with (suppressed) jsonrpc.Executor/0::INFO::2016-11-04 14:04:27,613::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getAllVmStats succeeded in 0.00 seconds Reactor thread::INFO::2016-11-04 14:04:29,595::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58240 Reactor thread::DEBUG::2016-11-04 14:04:29,601::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:04:29,602::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58240 Reactor thread::DEBUG::2016-11-04 14:04:29,602::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58240) BindingXMLRPC::INFO::2016-11-04 14:04:29,602::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58240 Thread-1072::INFO::2016-11-04 14:04:29,603::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58240 started Thread-1072::DEBUG::2016-11-04 14:04:29,603::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call getAllVmStats with () {} Thread-1072::DEBUG::2016-11-04 14:04:29,604::bindingxmlrpc::1255::vds::(wrapper) return getAllVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} Thread-1072::INFO::2016-11-04 14:04:29,604::bindingxmlrpc::1282::vds::(wrapper) RPC call getAllVmStats finished (code=0) in 0.00 seconds Thread-1072::INFO::2016-11-04 14:04:29,610::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58240 stopped Reactor thread::INFO::2016-11-04 14:04:29,611::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58242 Reactor thread::DEBUG::2016-11-04 14:04:29,617::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:04:29,617::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58242 Reactor thread::DEBUG::2016-11-04 14:04:29,617::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58242) BindingXMLRPC::INFO::2016-11-04 14:04:29,617::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58242 Thread-1073::INFO::2016-11-04 14:04:29,618::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58242 started Thread-1073::DEBUG::2016-11-04 14:04:29,618::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call vmGetIoTunePolicy with ('cddd2019-5baf-4321-aa8b-130b2889d1b0',) {} Thread-1073::DEBUG::2016-11-04 14:04:29,619::bindingxmlrpc::1255::vds::(wrapper) return vmGetIoTunePolicy with {'ioTunePolicy': [], 'status': {'message': 'Done', 'code': 0}} Thread-1073::INFO::2016-11-04 14:04:29,619::bindingxmlrpc::1282::vds::(wrapper) RPC call vmGetIoTunePolicy finished (code=0) in 0.00 seconds Thread-1073::INFO::2016-11-04 14:04:29,619::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58242 stopped Thread-12::DEBUG::2016-11-04 14:04:34,266::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:04:34,313::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:04:34,314::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000362253 s, 1.7 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:04:34,344::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:04:34,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:04:34,370::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000427858 s, 825 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:04:34,374::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:04:34,397::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000576044 s, 597 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:04:34,406::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000360426 s, 952 kB/s\n') elapsed=0.03 periodic/0::DEBUG::2016-11-04 14:04:36,463::task::597::Storage.TaskManager.Task::(_updateState) Task=`083418a5-972a-490f-b4e6-a0b7a4f44514`::moving from state init -> state preparing periodic/0::INFO::2016-11-04 14:04:36,463::logUtils::49::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'bfcabdd8-3aa7-4241-91c3-5db0e843e476', spUUID=u'00000001-0001-0001-0001-000000000366', imgUUID=u'b2109025-42be-4c40-ac2b-26baf8c304e5', volUUID=u'3153e04b-729c-44b2-b31a-0b2547ad3a77', options=None) periodic/0::INFO::2016-11-04 14:04:36,465::logUtils::52::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '17503478272', 'apparentsize': '21474836480'} periodic/0::DEBUG::2016-11-04 14:04:36,466::task::1193::Storage.TaskManager.Task::(prepare) Task=`083418a5-972a-490f-b4e6-a0b7a4f44514`::finished: {'truesize': '17503478272', 'apparentsize': '21474836480'} periodic/0::DEBUG::2016-11-04 14:04:36,466::task::597::Storage.TaskManager.Task::(_updateState) Task=`083418a5-972a-490f-b4e6-a0b7a4f44514`::moving from state preparing -> state finished periodic/0::DEBUG::2016-11-04 14:04:36,466::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/0::DEBUG::2016-11-04 14:04:36,466::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/0::DEBUG::2016-11-04 14:04:36,466::task::995::Storage.TaskManager.Task::(_decref) Task=`083418a5-972a-490f-b4e6-a0b7a4f44514`::ref 0 aborting False periodic/1::DEBUG::2016-11-04 14:04:36,599::task::597::Storage.TaskManager.Task::(_updateState) Task=`8d949fb4-6455-41be-b813-4986e36d8194`::moving from state init -> state preparing periodic/1::INFO::2016-11-04 14:04:36,599::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) periodic/1::INFO::2016-11-04 14:04:36,599::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000427858', 'lastCheck': '2.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000360426', 'lastCheck': '2.2', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000576044', 'lastCheck': '2.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000362253', 'lastCheck': '2.3', 'valid': True}} periodic/1::DEBUG::2016-11-04 14:04:36,599::task::1193::Storage.TaskManager.Task::(prepare) Task=`8d949fb4-6455-41be-b813-4986e36d8194`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000427858', 'lastCheck': '2.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000360426', 'lastCheck': '2.2', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000576044', 'lastCheck': '2.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000362253', 'lastCheck': '2.3', 'valid': True}} periodic/1::DEBUG::2016-11-04 14:04:36,600::task::597::Storage.TaskManager.Task::(_updateState) Task=`8d949fb4-6455-41be-b813-4986e36d8194`::moving from state preparing -> state finished periodic/1::DEBUG::2016-11-04 14:04:36,600::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/1::DEBUG::2016-11-04 14:04:36,600::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/1::DEBUG::2016-11-04 14:04:36,600::task::995::Storage.TaskManager.Task::(_decref) Task=`8d949fb4-6455-41be-b813-4986e36d8194`::ref 0 aborting False jsonrpc.Executor/2::DEBUG::2016-11-04 14:04:41,911::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getStats' in bridge with {} jsonrpc.Executor/2::DEBUG::2016-11-04 14:04:41,912::task::597::Storage.TaskManager.Task::(_updateState) Task=`3905aff3-c3bc-4a13-80c4-5538e5bea408`::moving from state init -> state preparing jsonrpc.Executor/2::INFO::2016-11-04 14:04:41,912::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) jsonrpc.Executor/2::INFO::2016-11-04 14:04:41,913::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000427858', 'lastCheck': '4.4', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000360426', 'lastCheck': '4.4', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000576044', 'lastCheck': '4.7', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000362253', 'lastCheck': '4.7', 'valid': True}} jsonrpc.Executor/2::DEBUG::2016-11-04 14:04:41,913::task::1193::Storage.TaskManager.Task::(prepare) Task=`3905aff3-c3bc-4a13-80c4-5538e5bea408`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000427858', 'lastCheck': '4.4', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000360426', 'lastCheck': '4.4', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000576044', 'lastCheck': '4.7', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000362253', 'lastCheck': '4.7', 'valid': True}} jsonrpc.Executor/2::DEBUG::2016-11-04 14:04:41,913::task::597::Storage.TaskManager.Task::(_updateState) Task=`3905aff3-c3bc-4a13-80c4-5538e5bea408`::moving from state preparing -> state finished jsonrpc.Executor/2::DEBUG::2016-11-04 14:04:41,913::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} jsonrpc.Executor/2::DEBUG::2016-11-04 14:04:41,913::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} jsonrpc.Executor/2::DEBUG::2016-11-04 14:04:41,913::task::995::Storage.TaskManager.Task::(_decref) Task=`3905aff3-c3bc-4a13-80c4-5538e5bea408`::ref 0 aborting False jsonrpc.Executor/2::DEBUG::2016-11-04 14:04:41,920::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getStats' in bridge with {'cpuStatistics': {'20': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '21': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '22': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '23': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '1': {'cpuUser': '0.47', 'nodeIndex': 0, 'cpuSys': '1.00', 'cpuIdle': '98.53'}, '0': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.74'}, '3': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '2': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '5': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '4': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '7': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '6': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '9': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '8': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '13': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '12': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '15': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '14': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '17': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '16': {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.73'}, '19': {'cpuUser': '0.40', 'nodeIndex': 1, 'cpuSys': '0.20', 'cpuIdle': '99.40'}, '18': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31474'}, '0': {'memPercent': 5, 'memFree': '70637'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'rxRate': '0.01', 'vmCount': 1, 'memUsed': '3', 'storageDomains': {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000427858', 'lastCheck': '4.4', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000360426', 'lastCheck': '4.4', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000576044', 'lastCheck': '4.7', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000362253', 'lastCheck': '4.7', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'eno1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '12014', 'name': 'eno1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286276.532962, 'rx': '5376559', 'state': 'up'}, 'ovirtmgmt': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.1', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '31941869', 'txDropped': '0', 'sampleTime': 1478286276.532962, 'rx': '744261025', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '6725166', 'txDropped': '0', 'sampleTime': 1478286276.532962, 'rx': '6725166', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286276.532962, 'rx': '0', 'state': 'down'}, 'eno2': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno2', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286276.532962, 'rx': '0', 'state': 'down'}, 'vnet0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'vnet0', 'tx': '371309', 'txDropped': '0', 'sampleTime': 1478286276.532962, 'rx': '102942', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286276.532962, 'rx': '0', 'state': 'down'}, 'vdsmbr_AoB33GL7': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '4744', 'name': 'vdsmbr_AoB33GL7', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286276.532962, 'rx': '0', 'state': 'down'}, 'eno3': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno3', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286276.532962, 'rx': '0', 'state': 'down'}, 'eno4': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno4', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286276.532962, 'rx': '0', 'state': 'down'}, 'ens2f0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '10000', 'rxDropped': '0', 'name': 'ens2f0', 'tx': '32665311', 'txDropped': '0', 'sampleTime': 1478286276.532962, 'rx': '773575376', 'state': 'up'}, 'ens2f1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ens2f1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286276.532962, 'rx': '0', 'state': 'down'}}, 'txDropped': '0', 'cpuUser': '0.06', 'ksmPages': 100, 'elapsedTime': '6365.46', 'cpuLoad': '0.01', 'cpuSys': '0.07', 'diskStats': {'/var/log': {'free': '15189'}, '/var/log/core': {'free': '15189'}, '/var/run/vdsm/': {'free': '52254'}, '/tmp': {'free': '1640028'}}, 'cpuUserVdsmd': '0.40', 'netConfigDirty': 'False', 'memCommitted': 4161, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 98635, 'txRate': '0.00', 'bootTime': '1478279889', 'haStats': {'active': False, 'configured': False, 'score': 0, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'rxDropped': '16758', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'dateTime': '2016-11-04T19:04:41 GMT', 'anonHugePages': '902', 'memFree': 102316, 'cpuIdle': '99.87', 'vmActive': 1, 'v2vJobs': {}, 'cpuSysVdsmd': '0.60'} jsonrpc.Executor/2::INFO::2016-11-04 14:04:41,921::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getStats succeeded in 0.01 seconds jsonrpc.Executor/7::DEBUG::2016-11-04 14:04:43,655::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getAllVmStats' in bridge with {} jsonrpc.Executor/7::DEBUG::2016-11-04 14:04:43,656::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getAllVmStats' in bridge with (suppressed) jsonrpc.Executor/7::INFO::2016-11-04 14:04:43,656::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getAllVmStats succeeded in 0.00 seconds Thread-12::DEBUG::2016-11-04 14:04:44,267::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:04:44,313::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:04:44,314::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000418349 s, 1.4 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:04:44,344::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:04:44,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:04:44,371::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000466524 s, 757 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:04:44,374::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:04:44,398::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000483457 s, 712 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:04:44,407::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000478529 s, 717 kB/s\n') elapsed=0.03 Reactor thread::INFO::2016-11-04 14:04:44,652::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58244 Reactor thread::DEBUG::2016-11-04 14:04:44,658::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:04:44,658::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58244 Reactor thread::DEBUG::2016-11-04 14:04:44,659::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58244) BindingXMLRPC::INFO::2016-11-04 14:04:44,659::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58244 Thread-1074::INFO::2016-11-04 14:04:44,659::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58244 started Thread-1074::DEBUG::2016-11-04 14:04:44,660::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call getAllVmStats with () {} Thread-1074::DEBUG::2016-11-04 14:04:44,661::bindingxmlrpc::1255::vds::(wrapper) return getAllVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} Thread-1074::INFO::2016-11-04 14:04:44,661::bindingxmlrpc::1282::vds::(wrapper) RPC call getAllVmStats finished (code=0) in 0.00 seconds Thread-1074::INFO::2016-11-04 14:04:44,667::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58244 stopped Reactor thread::INFO::2016-11-04 14:04:44,668::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58246 Reactor thread::DEBUG::2016-11-04 14:04:44,674::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:04:44,675::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58246 Reactor thread::DEBUG::2016-11-04 14:04:44,675::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58246) BindingXMLRPC::INFO::2016-11-04 14:04:44,675::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58246 Thread-1075::INFO::2016-11-04 14:04:44,675::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58246 started Thread-1075::DEBUG::2016-11-04 14:04:44,676::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call vmGetIoTunePolicy with ('cddd2019-5baf-4321-aa8b-130b2889d1b0',) {} Thread-1075::DEBUG::2016-11-04 14:04:44,676::bindingxmlrpc::1255::vds::(wrapper) return vmGetIoTunePolicy with {'ioTunePolicy': [], 'status': {'message': 'Done', 'code': 0}} Thread-1075::INFO::2016-11-04 14:04:44,677::bindingxmlrpc::1282::vds::(wrapper) RPC call vmGetIoTunePolicy finished (code=0) in 0.00 seconds Thread-1075::INFO::2016-11-04 14:04:44,677::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58246 stopped periodic/2::DEBUG::2016-11-04 14:04:51,594::task::597::Storage.TaskManager.Task::(_updateState) Task=`ed23cbc5-d029-42d8-b411-42dace6abfa2`::moving from state init -> state preparing periodic/2::INFO::2016-11-04 14:04:51,594::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) periodic/2::INFO::2016-11-04 14:04:51,594::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000466524', 'lastCheck': '4.0', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000478529', 'lastCheck': '4.0', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000483457', 'lastCheck': '4.3', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000418349', 'lastCheck': '4.3', 'valid': True}} periodic/2::DEBUG::2016-11-04 14:04:51,594::task::1193::Storage.TaskManager.Task::(prepare) Task=`ed23cbc5-d029-42d8-b411-42dace6abfa2`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000466524', 'lastCheck': '4.0', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000478529', 'lastCheck': '4.0', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000483457', 'lastCheck': '4.3', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000418349', 'lastCheck': '4.3', 'valid': True}} periodic/2::DEBUG::2016-11-04 14:04:51,595::task::597::Storage.TaskManager.Task::(_updateState) Task=`ed23cbc5-d029-42d8-b411-42dace6abfa2`::moving from state preparing -> state finished periodic/2::DEBUG::2016-11-04 14:04:51,595::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/2::DEBUG::2016-11-04 14:04:51,595::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/2::DEBUG::2016-11-04 14:04:51,595::task::995::Storage.TaskManager.Task::(_decref) Task=`ed23cbc5-d029-42d8-b411-42dace6abfa2`::ref 0 aborting False Thread-12::DEBUG::2016-11-04 14:04:54,268::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:04:54,314::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:04:54,314::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000382744 s, 1.6 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:04:54,344::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:04:54,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:04:54,371::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000402915 s, 876 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:04:54,374::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:04:54,398::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000459048 s, 749 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:04:54,407::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000381645 s, 899 kB/s\n') elapsed=0.03 jsonrpc.Executor/1::DEBUG::2016-11-04 14:04:57,186::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getStats' in bridge with {} jsonrpc.Executor/1::DEBUG::2016-11-04 14:04:57,187::task::597::Storage.TaskManager.Task::(_updateState) Task=`4d308101-4da4-4d74-9d48-731e86cdae11`::moving from state init -> state preparing jsonrpc.Executor/1::INFO::2016-11-04 14:04:57,187::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) jsonrpc.Executor/1::INFO::2016-11-04 14:04:57,187::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000402915', 'lastCheck': '2.8', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000381645', 'lastCheck': '2.8', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000459048', 'lastCheck': '2.8', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000382744', 'lastCheck': '2.9', 'valid': True}} jsonrpc.Executor/1::DEBUG::2016-11-04 14:04:57,188::task::1193::Storage.TaskManager.Task::(prepare) Task=`4d308101-4da4-4d74-9d48-731e86cdae11`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000402915', 'lastCheck': '2.8', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000381645', 'lastCheck': '2.8', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000459048', 'lastCheck': '2.8', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000382744', 'lastCheck': '2.9', 'valid': True}} jsonrpc.Executor/1::DEBUG::2016-11-04 14:04:57,188::task::597::Storage.TaskManager.Task::(_updateState) Task=`4d308101-4da4-4d74-9d48-731e86cdae11`::moving from state preparing -> state finished jsonrpc.Executor/1::DEBUG::2016-11-04 14:04:57,188::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} jsonrpc.Executor/1::DEBUG::2016-11-04 14:04:57,188::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} jsonrpc.Executor/1::DEBUG::2016-11-04 14:04:57,188::task::995::Storage.TaskManager.Task::(_decref) Task=`4d308101-4da4-4d74-9d48-731e86cdae11`::ref 0 aborting False jsonrpc.Executor/1::DEBUG::2016-11-04 14:04:57,195::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getStats' in bridge with {'cpuStatistics': {'20': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '21': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '22': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '23': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '1': {'cpuUser': '0.47', 'nodeIndex': 0, 'cpuSys': '0.60', 'cpuIdle': '98.93'}, '0': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '3': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '2': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.74'}, '5': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '4': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '7': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '6': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '9': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '8': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '13': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '12': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '15': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '14': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '17': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '16': {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.66'}, '19': {'cpuUser': '0.40', 'nodeIndex': 1, 'cpuSys': '0.20', 'cpuIdle': '99.40'}, '18': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31490'}, '0': {'memPercent': 5, 'memFree': '70621'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'rxRate': '0.01', 'vmCount': 1, 'memUsed': '3', 'storageDomains': {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000402915', 'lastCheck': '2.8', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000381645', 'lastCheck': '2.8', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000459048', 'lastCheck': '2.8', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000382744', 'lastCheck': '2.9', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'eno1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '12042', 'name': 'eno1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286291.527772, 'rx': '5388183', 'state': 'up'}, 'ovirtmgmt': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.1', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '31975445', 'txDropped': '0', 'sampleTime': 1478286291.527772, 'rx': '746381025', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '6743412', 'txDropped': '0', 'sampleTime': 1478286291.527772, 'rx': '6743412', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286291.527772, 'rx': '0', 'state': 'down'}, 'eno2': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno2', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286291.527772, 'rx': '0', 'state': 'down'}, 'vnet0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'vnet0', 'tx': '371969', 'txDropped': '0', 'sampleTime': 1478286291.527772, 'rx': '102942', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286291.527772, 'rx': '0', 'state': 'down'}, 'vdsmbr_AoB33GL7': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '4755', 'name': 'vdsmbr_AoB33GL7', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286291.527772, 'rx': '0', 'state': 'down'}, 'eno3': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno3', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286291.527772, 'rx': '0', 'state': 'down'}, 'eno4': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno4', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286291.527772, 'rx': '0', 'state': 'down'}, 'ens2f0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '10000', 'rxDropped': '0', 'name': 'ens2f0', 'tx': '32699084', 'txDropped': '0', 'sampleTime': 1478286291.527772, 'rx': '775780877', 'state': 'up'}, 'ens2f1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ens2f1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286291.527772, 'rx': '0', 'state': 'down'}}, 'txDropped': '0', 'cpuUser': '0.06', 'ksmPages': 100, 'elapsedTime': '6380.74', 'cpuLoad': '0.01', 'cpuSys': '0.04', 'diskStats': {'/var/log': {'free': '15189'}, '/var/log/core': {'free': '15189'}, '/var/run/vdsm/': {'free': '52254'}, '/tmp': {'free': '1640028'}}, 'cpuUserVdsmd': '0.40', 'netConfigDirty': 'False', 'memCommitted': 4161, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 98635, 'txRate': '0.00', 'bootTime': '1478279889', 'haStats': {'active': False, 'configured': False, 'score': 0, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'rxDropped': '16797', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'dateTime': '2016-11-04T19:04:57 GMT', 'anonHugePages': '902', 'memFree': 102315, 'cpuIdle': '99.89', 'vmActive': 1, 'v2vJobs': {}, 'cpuSysVdsmd': '0.33'} jsonrpc.Executor/1::INFO::2016-11-04 14:04:57,196::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getStats succeeded in 0.01 seconds Reactor thread::INFO::2016-11-04 14:04:59,707::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58248 Reactor thread::DEBUG::2016-11-04 14:04:59,713::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:04:59,714::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58248 Reactor thread::DEBUG::2016-11-04 14:04:59,714::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58248) BindingXMLRPC::INFO::2016-11-04 14:04:59,714::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58248 Thread-1076::INFO::2016-11-04 14:04:59,714::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58248 started Thread-1076::DEBUG::2016-11-04 14:04:59,715::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call getAllVmStats with () {} Thread-1076::DEBUG::2016-11-04 14:04:59,716::bindingxmlrpc::1255::vds::(wrapper) return getAllVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} Thread-1076::INFO::2016-11-04 14:04:59,716::bindingxmlrpc::1282::vds::(wrapper) RPC call getAllVmStats finished (code=0) in 0.00 seconds Thread-1076::INFO::2016-11-04 14:04:59,721::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58248 stopped Reactor thread::INFO::2016-11-04 14:04:59,722::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58250 Reactor thread::DEBUG::2016-11-04 14:04:59,728::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:04:59,729::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58250 Reactor thread::DEBUG::2016-11-04 14:04:59,729::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58250) BindingXMLRPC::INFO::2016-11-04 14:04:59,729::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58250 Thread-1077::INFO::2016-11-04 14:04:59,729::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58250 started Thread-1077::DEBUG::2016-11-04 14:04:59,730::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call vmGetIoTunePolicy with ('cddd2019-5baf-4321-aa8b-130b2889d1b0',) {} Thread-1077::DEBUG::2016-11-04 14:04:59,730::bindingxmlrpc::1255::vds::(wrapper) return vmGetIoTunePolicy with {'ioTunePolicy': [], 'status': {'message': 'Done', 'code': 0}} Thread-1077::INFO::2016-11-04 14:04:59,730::bindingxmlrpc::1282::vds::(wrapper) RPC call vmGetIoTunePolicy finished (code=0) in 0.00 seconds Thread-1077::INFO::2016-11-04 14:04:59,731::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58250 stopped jsonrpc.Executor/3::DEBUG::2016-11-04 14:04:59,761::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getAllVmStats' in bridge with {} jsonrpc.Executor/3::DEBUG::2016-11-04 14:04:59,762::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getAllVmStats' in bridge with (suppressed) jsonrpc.Executor/3::INFO::2016-11-04 14:04:59,762::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getAllVmStats succeeded in 0.00 seconds Thread-12::DEBUG::2016-11-04 14:05:04,267::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:05:04,313::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:05:04,313::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000414667 s, 1.5 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:05:04,344::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:05:04,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:05:04,370::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000489221 s, 722 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:05:04,373::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:05:04,397::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000516194 s, 666 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:05:04,408::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000400286 s, 857 kB/s\n') elapsed=0.04 periodic/3::DEBUG::2016-11-04 14:05:06,599::task::597::Storage.TaskManager.Task::(_updateState) Task=`b51ab920-b1bd-47a0-9f3e-481a1e2e9a81`::moving from state init -> state preparing periodic/3::INFO::2016-11-04 14:05:06,599::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) periodic/3::INFO::2016-11-04 14:05:06,599::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000489221', 'lastCheck': '2.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000400286', 'lastCheck': '2.2', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000516194', 'lastCheck': '2.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000414667', 'lastCheck': '2.3', 'valid': True}} periodic/3::DEBUG::2016-11-04 14:05:06,599::task::1193::Storage.TaskManager.Task::(prepare) Task=`b51ab920-b1bd-47a0-9f3e-481a1e2e9a81`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000489221', 'lastCheck': '2.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000400286', 'lastCheck': '2.2', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000516194', 'lastCheck': '2.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000414667', 'lastCheck': '2.3', 'valid': True}} periodic/3::DEBUG::2016-11-04 14:05:06,600::task::597::Storage.TaskManager.Task::(_updateState) Task=`b51ab920-b1bd-47a0-9f3e-481a1e2e9a81`::moving from state preparing -> state finished periodic/3::DEBUG::2016-11-04 14:05:06,600::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/3::DEBUG::2016-11-04 14:05:06,600::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/3::DEBUG::2016-11-04 14:05:06,600::task::995::Storage.TaskManager.Task::(_decref) Task=`b51ab920-b1bd-47a0-9f3e-481a1e2e9a81`::ref 0 aborting False jsonrpc.Executor/4::DEBUG::2016-11-04 14:05:13,423::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getStats' in bridge with {} jsonrpc.Executor/4::DEBUG::2016-11-04 14:05:13,425::task::597::Storage.TaskManager.Task::(_updateState) Task=`a6d71a05-1b6b-4b6c-8808-413901d060a1`::moving from state init -> state preparing jsonrpc.Executor/4::INFO::2016-11-04 14:05:13,425::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) jsonrpc.Executor/4::INFO::2016-11-04 14:05:13,425::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000489221', 'lastCheck': '5.9', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000400286', 'lastCheck': '5.9', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000516194', 'lastCheck': '6.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000414667', 'lastCheck': '6.2', 'valid': True}} jsonrpc.Executor/4::DEBUG::2016-11-04 14:05:13,425::task::1193::Storage.TaskManager.Task::(prepare) Task=`a6d71a05-1b6b-4b6c-8808-413901d060a1`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000489221', 'lastCheck': '5.9', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000400286', 'lastCheck': '5.9', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000516194', 'lastCheck': '6.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000414667', 'lastCheck': '6.2', 'valid': True}} jsonrpc.Executor/4::DEBUG::2016-11-04 14:05:13,425::task::597::Storage.TaskManager.Task::(_updateState) Task=`a6d71a05-1b6b-4b6c-8808-413901d060a1`::moving from state preparing -> state finished jsonrpc.Executor/4::DEBUG::2016-11-04 14:05:13,425::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} jsonrpc.Executor/4::DEBUG::2016-11-04 14:05:13,425::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} jsonrpc.Executor/4::DEBUG::2016-11-04 14:05:13,425::task::995::Storage.TaskManager.Task::(_decref) Task=`a6d71a05-1b6b-4b6c-8808-413901d060a1`::ref 0 aborting False jsonrpc.Executor/4::DEBUG::2016-11-04 14:05:13,434::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getStats' in bridge with {'cpuStatistics': {'20': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '21': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '22': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '23': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '1': {'cpuUser': '0.53', 'nodeIndex': 0, 'cpuSys': '1.00', 'cpuIdle': '98.47'}, '0': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '3': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '2': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.87'}, '5': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '4': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '7': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '6': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '9': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '8': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '13': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '12': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '15': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '14': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '17': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '16': {'cpuUser': '0.20', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.80'}, '19': {'cpuUser': '0.40', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.47'}, '18': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31493'}, '0': {'memPercent': 5, 'memFree': '70619'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'rxRate': '0.01', 'vmCount': 1, 'memUsed': '3', 'storageDomains': {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000489221', 'lastCheck': '5.9', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000400286', 'lastCheck': '5.9', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000516194', 'lastCheck': '6.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000414667', 'lastCheck': '6.2', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'eno1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '12070', 'name': 'eno1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286306.53247, 'rx': '5399946', 'state': 'up'}, 'ovirtmgmt': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.1', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '32014351', 'txDropped': '0', 'sampleTime': 1478286306.53247, 'rx': '747444691', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '6761610', 'txDropped': '0', 'sampleTime': 1478286306.53247, 'rx': '6761610', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286306.53247, 'rx': '0', 'state': 'down'}, 'eno2': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno2', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286306.53247, 'rx': '0', 'state': 'down'}, 'vnet0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'vnet0', 'tx': '373359', 'txDropped': '0', 'sampleTime': 1478286306.53247, 'rx': '102942', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286306.53247, 'rx': '0', 'state': 'down'}, 'vdsmbr_AoB33GL7': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '4775', 'name': 'vdsmbr_AoB33GL7', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286306.53247, 'rx': '0', 'state': 'down'}, 'eno3': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno3', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286306.53247, 'rx': '0', 'state': 'down'}, 'eno4': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno4', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286306.53247, 'rx': '0', 'state': 'down'}, 'ens2f0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '10000', 'rxDropped': '0', 'name': 'ens2f0', 'tx': '32738848', 'txDropped': '0', 'sampleTime': 1478286306.53247, 'rx': '776891180', 'state': 'up'}, 'ens2f1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ens2f1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286306.53247, 'rx': '0', 'state': 'down'}}, 'txDropped': '0', 'cpuUser': '0.07', 'ksmPages': 100, 'elapsedTime': '6396.98', 'cpuLoad': '0.01', 'cpuSys': '0.06', 'diskStats': {'/var/log': {'free': '15189'}, '/var/log/core': {'free': '15189'}, '/var/run/vdsm/': {'free': '52254'}, '/tmp': {'free': '1640028'}}, 'cpuUserVdsmd': '0.40', 'netConfigDirty': 'False', 'memCommitted': 4161, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 98631, 'txRate': '0.00', 'bootTime': '1478279889', 'haStats': {'active': False, 'configured': False, 'score': 0, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'rxDropped': '16845', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'dateTime': '2016-11-04T19:05:13 GMT', 'anonHugePages': '902', 'memFree': 102312, 'cpuIdle': '99.87', 'vmActive': 1, 'v2vJobs': {}, 'cpuSysVdsmd': '0.60'} jsonrpc.Executor/4::INFO::2016-11-04 14:05:13,435::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getStats succeeded in 0.02 seconds Thread-12::DEBUG::2016-11-04 14:05:14,268::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:05:14,324::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:05:14,324::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.00031418 s, 1.9 MB/s\n') elapsed=0.06 Thread-12::DEBUG::2016-11-04 14:05:14,344::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:05:14,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:05:14,370::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.0004865 s, 726 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:05:14,372::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:05:14,394::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000467423 s, 736 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:05:14,404::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.00042935 s, 799 kB/s\n') elapsed=0.04 Reactor thread::INFO::2016-11-04 14:05:14,764::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58252 Reactor thread::DEBUG::2016-11-04 14:05:14,770::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:05:14,771::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58252 Reactor thread::DEBUG::2016-11-04 14:05:14,771::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58252) BindingXMLRPC::INFO::2016-11-04 14:05:14,771::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58252 Thread-1078::INFO::2016-11-04 14:05:14,772::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58252 started Thread-1078::DEBUG::2016-11-04 14:05:14,772::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call getAllVmStats with () {} Thread-1078::DEBUG::2016-11-04 14:05:14,773::bindingxmlrpc::1255::vds::(wrapper) return getAllVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} Thread-1078::INFO::2016-11-04 14:05:14,773::bindingxmlrpc::1282::vds::(wrapper) RPC call getAllVmStats finished (code=0) in 0.01 seconds Thread-1078::INFO::2016-11-04 14:05:14,779::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58252 stopped Reactor thread::INFO::2016-11-04 14:05:14,780::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58254 Reactor thread::DEBUG::2016-11-04 14:05:14,786::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:05:14,786::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58254 Reactor thread::DEBUG::2016-11-04 14:05:14,786::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58254) BindingXMLRPC::INFO::2016-11-04 14:05:14,787::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58254 Thread-1079::INFO::2016-11-04 14:05:14,787::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58254 started Thread-1079::DEBUG::2016-11-04 14:05:14,787::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call vmGetIoTunePolicy with ('cddd2019-5baf-4321-aa8b-130b2889d1b0',) {} Thread-1079::DEBUG::2016-11-04 14:05:14,788::bindingxmlrpc::1255::vds::(wrapper) return vmGetIoTunePolicy with {'ioTunePolicy': [], 'status': {'message': 'Done', 'code': 0}} Thread-1079::INFO::2016-11-04 14:05:14,788::bindingxmlrpc::1282::vds::(wrapper) RPC call vmGetIoTunePolicy finished (code=0) in 0.00 seconds Thread-1079::INFO::2016-11-04 14:05:14,789::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58254 stopped jsonrpc.Executor/5::DEBUG::2016-11-04 14:05:15,416::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getAllVmStats' in bridge with {} jsonrpc.Executor/5::DEBUG::2016-11-04 14:05:15,417::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getAllVmStats' in bridge with (suppressed) jsonrpc.Executor/5::INFO::2016-11-04 14:05:15,417::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getAllVmStats succeeded in 0.00 seconds periodic/0::DEBUG::2016-11-04 14:05:21,599::task::597::Storage.TaskManager.Task::(_updateState) Task=`f7677294-ed85-48a8-8ff7-a92a5e294427`::moving from state init -> state preparing periodic/0::INFO::2016-11-04 14:05:21,600::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) periodic/0::INFO::2016-11-04 14:05:21,600::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.0004865', 'lastCheck': '4.0', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00042935', 'lastCheck': '4.0', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000467423', 'lastCheck': '4.3', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00031418', 'lastCheck': '4.3', 'valid': True}} periodic/0::DEBUG::2016-11-04 14:05:21,600::task::1193::Storage.TaskManager.Task::(prepare) Task=`f7677294-ed85-48a8-8ff7-a92a5e294427`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.0004865', 'lastCheck': '4.0', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00042935', 'lastCheck': '4.0', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000467423', 'lastCheck': '4.3', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00031418', 'lastCheck': '4.3', 'valid': True}} periodic/0::DEBUG::2016-11-04 14:05:21,600::task::597::Storage.TaskManager.Task::(_updateState) Task=`f7677294-ed85-48a8-8ff7-a92a5e294427`::moving from state preparing -> state finished periodic/0::DEBUG::2016-11-04 14:05:21,600::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/0::DEBUG::2016-11-04 14:05:21,600::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/0::DEBUG::2016-11-04 14:05:21,600::task::995::Storage.TaskManager.Task::(_decref) Task=`f7677294-ed85-48a8-8ff7-a92a5e294427`::ref 0 aborting False Thread-12::DEBUG::2016-11-04 14:05:24,264::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:05:24,314::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:05:24,314::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000310414 s, 1.9 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:05:24,344::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:05:24,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:05:24,370::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000444634 s, 794 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:05:24,373::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:05:24,395::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000467911 s, 735 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:05:24,443::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:05:24,444::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000353975 s, 969 kB/s\n') elapsed=0.07 jsonrpc.Executor/6::DEBUG::2016-11-04 14:05:29,649::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getStats' in bridge with {} jsonrpc.Executor/6::DEBUG::2016-11-04 14:05:29,650::task::597::Storage.TaskManager.Task::(_updateState) Task=`26d146a2-8e6b-4a12-aa17-8c16ee095d9d`::moving from state init -> state preparing jsonrpc.Executor/6::INFO::2016-11-04 14:05:29,651::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) jsonrpc.Executor/6::INFO::2016-11-04 14:05:29,651::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000444634', 'lastCheck': '2.1', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000353975', 'lastCheck': '2.1', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000467911', 'lastCheck': '2.4', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000310414', 'lastCheck': '2.4', 'valid': True}} jsonrpc.Executor/6::DEBUG::2016-11-04 14:05:29,651::task::1193::Storage.TaskManager.Task::(prepare) Task=`26d146a2-8e6b-4a12-aa17-8c16ee095d9d`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000444634', 'lastCheck': '2.1', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000353975', 'lastCheck': '2.1', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000467911', 'lastCheck': '2.4', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000310414', 'lastCheck': '2.4', 'valid': True}} jsonrpc.Executor/6::DEBUG::2016-11-04 14:05:29,651::task::597::Storage.TaskManager.Task::(_updateState) Task=`26d146a2-8e6b-4a12-aa17-8c16ee095d9d`::moving from state preparing -> state finished jsonrpc.Executor/6::DEBUG::2016-11-04 14:05:29,651::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} jsonrpc.Executor/6::DEBUG::2016-11-04 14:05:29,651::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} jsonrpc.Executor/6::DEBUG::2016-11-04 14:05:29,651::task::995::Storage.TaskManager.Task::(_decref) Task=`26d146a2-8e6b-4a12-aa17-8c16ee095d9d`::ref 0 aborting False jsonrpc.Executor/6::DEBUG::2016-11-04 14:05:29,659::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getStats' in bridge with {'cpuStatistics': {'20': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '21': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '22': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '23': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '1': {'cpuUser': '0.47', 'nodeIndex': 0, 'cpuSys': '0.67', 'cpuIdle': '98.86'}, '0': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '3': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.47', 'cpuIdle': '99.46'}, '2': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '5': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '4': {'cpuUser': '0.33', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.54'}, '7': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '6': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '9': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '8': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '13': {'cpuUser': '1.13', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '98.67'}, '12': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '15': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '14': {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.53'}, '17': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '16': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '19': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.80'}, '18': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31492'}, '0': {'memPercent': 5, 'memFree': '70616'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'rxRate': '0.01', 'vmCount': 1, 'memUsed': '3', 'storageDomains': {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000444634', 'lastCheck': '2.1', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000353975', 'lastCheck': '2.1', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000467911', 'lastCheck': '2.4', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000310414', 'lastCheck': '2.4', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'eno1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '12098', 'name': 'eno1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286321.527524, 'rx': '5413202', 'state': 'up'}, 'ovirtmgmt': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.1', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '32047899', 'txDropped': '0', 'sampleTime': 1478286321.527524, 'rx': '748518284', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '6779824', 'txDropped': '0', 'sampleTime': 1478286321.527524, 'rx': '6779824', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286321.527524, 'rx': '0', 'state': 'down'}, 'eno2': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno2', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286321.527524, 'rx': '0', 'state': 'down'}, 'vnet0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'vnet0', 'tx': '374869', 'txDropped': '0', 'sampleTime': 1478286321.527524, 'rx': '102942', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286321.527524, 'rx': '0', 'state': 'down'}, 'vdsmbr_AoB33GL7': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '4797', 'name': 'vdsmbr_AoB33GL7', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286321.527524, 'rx': '0', 'state': 'down'}, 'eno3': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno3', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286321.527524, 'rx': '0', 'state': 'down'}, 'eno4': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno4', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286321.527524, 'rx': '0', 'state': 'down'}, 'ens2f0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '10000', 'rxDropped': '0', 'name': 'ens2f0', 'tx': '32772931', 'txDropped': '0', 'sampleTime': 1478286321.527524, 'rx': '778003530', 'state': 'up'}, 'ens2f1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ens2f1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286321.527524, 'rx': '0', 'state': 'down'}}, 'txDropped': '0', 'cpuUser': '0.11', 'ksmPages': 100, 'elapsedTime': '6413.20', 'cpuLoad': '0.05', 'cpuSys': '0.09', 'diskStats': {'/var/log': {'free': '15189'}, '/var/log/core': {'free': '15189'}, '/var/run/vdsm/': {'free': '52254'}, '/tmp': {'free': '1640028'}}, 'cpuUserVdsmd': '0.40', 'netConfigDirty': 'False', 'memCommitted': 4161, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 98632, 'txRate': '0.00', 'bootTime': '1478279889', 'haStats': {'active': False, 'configured': False, 'score': 0, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'rxDropped': '16895', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'dateTime': '2016-11-04T19:05:29 GMT', 'anonHugePages': '902', 'memFree': 102313, 'cpuIdle': '99.80', 'vmActive': 1, 'v2vJobs': {}, 'cpuSysVdsmd': '0.40'} jsonrpc.Executor/6::INFO::2016-11-04 14:05:29,660::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getStats succeeded in 0.01 seconds Reactor thread::INFO::2016-11-04 14:05:29,818::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58256 Reactor thread::DEBUG::2016-11-04 14:05:29,824::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:05:29,824::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58256 Reactor thread::DEBUG::2016-11-04 14:05:29,824::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58256) BindingXMLRPC::INFO::2016-11-04 14:05:29,824::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58256 Thread-1080::INFO::2016-11-04 14:05:29,825::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58256 started Thread-1080::DEBUG::2016-11-04 14:05:29,825::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call getAllVmStats with () {} Thread-1080::DEBUG::2016-11-04 14:05:29,826::bindingxmlrpc::1255::vds::(wrapper) return getAllVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} Thread-1080::INFO::2016-11-04 14:05:29,826::bindingxmlrpc::1282::vds::(wrapper) RPC call getAllVmStats finished (code=0) in 0.00 seconds Thread-1080::INFO::2016-11-04 14:05:29,831::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58256 stopped Reactor thread::INFO::2016-11-04 14:05:29,889::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58258 Reactor thread::DEBUG::2016-11-04 14:05:29,895::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:05:29,896::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58258 Reactor thread::DEBUG::2016-11-04 14:05:29,896::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58258) BindingXMLRPC::INFO::2016-11-04 14:05:29,896::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58258 Thread-1081::INFO::2016-11-04 14:05:29,896::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58258 started Thread-1081::DEBUG::2016-11-04 14:05:29,897::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call vmGetIoTunePolicy with ('cddd2019-5baf-4321-aa8b-130b2889d1b0',) {} Thread-1081::DEBUG::2016-11-04 14:05:29,897::bindingxmlrpc::1255::vds::(wrapper) return vmGetIoTunePolicy with {'ioTunePolicy': [], 'status': {'message': 'Done', 'code': 0}} Thread-1081::INFO::2016-11-04 14:05:29,897::bindingxmlrpc::1282::vds::(wrapper) RPC call vmGetIoTunePolicy finished (code=0) in 0.00 seconds Thread-1081::INFO::2016-11-04 14:05:29,898::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58258 stopped jsonrpc.Executor/0::DEBUG::2016-11-04 14:05:31,454::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getAllVmStats' in bridge with {} jsonrpc.Executor/0::DEBUG::2016-11-04 14:05:31,455::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getAllVmStats' in bridge with (suppressed) jsonrpc.Executor/0::INFO::2016-11-04 14:05:31,455::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getAllVmStats succeeded in 0.00 seconds Thread-12::DEBUG::2016-11-04 14:05:34,264::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:05:34,314::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:05:34,314::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.0003911 s, 1.5 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:05:34,345::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:05:34,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:05:34,371::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000379826 s, 929 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:05:34,372::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:05:34,395::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000387966 s, 887 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:05:34,444::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:05:34,444::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000645916 s, 531 kB/s\n') elapsed=0.08 periodic/3::DEBUG::2016-11-04 14:05:36,463::task::597::Storage.TaskManager.Task::(_updateState) Task=`eeb0c512-5171-4bcb-8e5a-d9e6491d8248`::moving from state init -> state preparing periodic/3::INFO::2016-11-04 14:05:36,463::logUtils::49::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'bfcabdd8-3aa7-4241-91c3-5db0e843e476', spUUID=u'00000001-0001-0001-0001-000000000366', imgUUID=u'b2109025-42be-4c40-ac2b-26baf8c304e5', volUUID=u'3153e04b-729c-44b2-b31a-0b2547ad3a77', options=None) periodic/3::INFO::2016-11-04 14:05:36,466::logUtils::52::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '17503478272', 'apparentsize': '21474836480'} periodic/3::DEBUG::2016-11-04 14:05:36,466::task::1193::Storage.TaskManager.Task::(prepare) Task=`eeb0c512-5171-4bcb-8e5a-d9e6491d8248`::finished: {'truesize': '17503478272', 'apparentsize': '21474836480'} periodic/3::DEBUG::2016-11-04 14:05:36,466::task::597::Storage.TaskManager.Task::(_updateState) Task=`eeb0c512-5171-4bcb-8e5a-d9e6491d8248`::moving from state preparing -> state finished periodic/3::DEBUG::2016-11-04 14:05:36,466::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/3::DEBUG::2016-11-04 14:05:36,466::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/3::DEBUG::2016-11-04 14:05:36,466::task::995::Storage.TaskManager.Task::(_decref) Task=`eeb0c512-5171-4bcb-8e5a-d9e6491d8248`::ref 0 aborting False periodic/0::DEBUG::2016-11-04 14:05:36,596::task::597::Storage.TaskManager.Task::(_updateState) Task=`6139be29-5d89-493f-ae05-4c2140ef90af`::moving from state init -> state preparing periodic/0::INFO::2016-11-04 14:05:36,597::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) periodic/0::INFO::2016-11-04 14:05:36,597::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000379826', 'lastCheck': '2.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000645916', 'lastCheck': '2.2', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000387966', 'lastCheck': '2.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.0003911', 'lastCheck': '2.3', 'valid': True}} periodic/0::DEBUG::2016-11-04 14:05:36,597::task::1193::Storage.TaskManager.Task::(prepare) Task=`6139be29-5d89-493f-ae05-4c2140ef90af`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000379826', 'lastCheck': '2.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000645916', 'lastCheck': '2.2', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000387966', 'lastCheck': '2.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.0003911', 'lastCheck': '2.3', 'valid': True}} periodic/0::DEBUG::2016-11-04 14:05:36,597::task::597::Storage.TaskManager.Task::(_updateState) Task=`6139be29-5d89-493f-ae05-4c2140ef90af`::moving from state preparing -> state finished periodic/0::DEBUG::2016-11-04 14:05:36,597::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/0::DEBUG::2016-11-04 14:05:36,597::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/0::DEBUG::2016-11-04 14:05:36,597::task::995::Storage.TaskManager.Task::(_decref) Task=`6139be29-5d89-493f-ae05-4c2140ef90af`::ref 0 aborting False Thread-12::DEBUG::2016-11-04 14:05:44,258::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:05:44,304::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:05:44,304::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000391294 s, 1.5 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:05:44,345::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:05:44,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:05:44,372::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000483559 s, 730 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:05:44,373::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:05:44,395::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000524842 s, 655 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:05:44,443::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:05:44,443::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000682512 s, 503 kB/s\n') elapsed=0.07 Reactor thread::INFO::2016-11-04 14:05:44,873::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58260 Reactor thread::DEBUG::2016-11-04 14:05:44,879::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:05:44,879::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58260 Reactor thread::DEBUG::2016-11-04 14:05:44,879::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58260) BindingXMLRPC::INFO::2016-11-04 14:05:44,879::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58260 Thread-1082::INFO::2016-11-04 14:05:44,880::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58260 started Thread-1082::DEBUG::2016-11-04 14:05:44,880::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call getAllVmStats with () {} Thread-1082::DEBUG::2016-11-04 14:05:44,881::bindingxmlrpc::1255::vds::(wrapper) return getAllVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} Thread-1082::INFO::2016-11-04 14:05:44,882::bindingxmlrpc::1282::vds::(wrapper) RPC call getAllVmStats finished (code=0) in 0.00 seconds Thread-1082::INFO::2016-11-04 14:05:44,887::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58260 stopped Reactor thread::INFO::2016-11-04 14:05:44,914::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58262 Reactor thread::DEBUG::2016-11-04 14:05:44,920::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:05:44,920::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58262 Reactor thread::DEBUG::2016-11-04 14:05:44,920::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58262) BindingXMLRPC::INFO::2016-11-04 14:05:44,921::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58262 Thread-1083::INFO::2016-11-04 14:05:44,921::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58262 started Thread-1083::DEBUG::2016-11-04 14:05:44,921::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call vmGetIoTunePolicy with ('cddd2019-5baf-4321-aa8b-130b2889d1b0',) {} Thread-1083::DEBUG::2016-11-04 14:05:44,922::bindingxmlrpc::1255::vds::(wrapper) return vmGetIoTunePolicy with {'ioTunePolicy': [], 'status': {'message': 'Done', 'code': 0}} Thread-1083::INFO::2016-11-04 14:05:44,922::bindingxmlrpc::1282::vds::(wrapper) RPC call vmGetIoTunePolicy finished (code=0) in 0.00 seconds Thread-1083::INFO::2016-11-04 14:05:44,923::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58262 stopped jsonrpc.Executor/2::DEBUG::2016-11-04 14:05:45,889::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getStats' in bridge with {} jsonrpc.Executor/2::DEBUG::2016-11-04 14:05:45,890::task::597::Storage.TaskManager.Task::(_updateState) Task=`cc089a62-f866-4baa-8563-a020bb3ca05b`::moving from state init -> state preparing jsonrpc.Executor/2::INFO::2016-11-04 14:05:45,890::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) jsonrpc.Executor/2::INFO::2016-11-04 14:05:45,890::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000483559', 'lastCheck': '1.5', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000682512', 'lastCheck': '1.4', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000524842', 'lastCheck': '1.5', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000391294', 'lastCheck': '1.6', 'valid': True}} jsonrpc.Executor/2::DEBUG::2016-11-04 14:05:45,890::task::1193::Storage.TaskManager.Task::(prepare) Task=`cc089a62-f866-4baa-8563-a020bb3ca05b`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000483559', 'lastCheck': '1.5', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000682512', 'lastCheck': '1.4', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000524842', 'lastCheck': '1.5', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000391294', 'lastCheck': '1.6', 'valid': True}} jsonrpc.Executor/2::DEBUG::2016-11-04 14:05:45,890::task::597::Storage.TaskManager.Task::(_updateState) Task=`cc089a62-f866-4baa-8563-a020bb3ca05b`::moving from state preparing -> state finished jsonrpc.Executor/2::DEBUG::2016-11-04 14:05:45,891::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} jsonrpc.Executor/2::DEBUG::2016-11-04 14:05:45,891::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} jsonrpc.Executor/2::DEBUG::2016-11-04 14:05:45,891::task::995::Storage.TaskManager.Task::(_decref) Task=`cc089a62-f866-4baa-8563-a020bb3ca05b`::ref 0 aborting False jsonrpc.Executor/2::DEBUG::2016-11-04 14:05:45,898::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getStats' in bridge with {'cpuStatistics': {'20': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '21': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '22': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '23': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '1': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.93', 'cpuIdle': '98.67'}, '0': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.74'}, '3': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '2': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '5': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '4': {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.73'}, '7': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '6': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '9': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '8': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '13': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.47'}, '12': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '15': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '14': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '17': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '16': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '19': {'cpuUser': '0.20', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.73'}, '18': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31491'}, '0': {'memPercent': 5, 'memFree': '70617'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'rxRate': '0.01', 'vmCount': 1, 'memUsed': '3', 'storageDomains': {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000483559', 'lastCheck': '1.5', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000682512', 'lastCheck': '1.4', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000524842', 'lastCheck': '1.5', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000391294', 'lastCheck': '1.6', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'eno1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '12128', 'name': 'eno1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286336.532133, 'rx': '5425396', 'state': 'up'}, 'ovirtmgmt': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.1', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '32087559', 'txDropped': '0', 'sampleTime': 1478286336.532133, 'rx': '750645016', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '6798022', 'txDropped': '0', 'sampleTime': 1478286336.532133, 'rx': '6798022', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286336.532133, 'rx': '0', 'state': 'down'}, 'eno2': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno2', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286336.532133, 'rx': '0', 'state': 'down'}, 'vnet0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'vnet0', 'tx': '375049', 'txDropped': '0', 'sampleTime': 1478286336.532133, 'rx': '102942', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286336.532133, 'rx': '0', 'state': 'down'}, 'vdsmbr_AoB33GL7': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '4800', 'name': 'vdsmbr_AoB33GL7', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286336.532133, 'rx': '0', 'state': 'down'}, 'eno3': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno3', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286336.532133, 'rx': '0', 'state': 'down'}, 'eno4': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno4', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286336.532133, 'rx': '0', 'state': 'down'}, 'ens2f0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '10000', 'rxDropped': '0', 'name': 'ens2f0', 'tx': '32813259', 'txDropped': '0', 'sampleTime': 1478286336.532133, 'rx': '780215955', 'state': 'up'}, 'ens2f1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ens2f1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286336.532133, 'rx': '0', 'state': 'down'}}, 'txDropped': '0', 'cpuUser': '0.07', 'ksmPages': 100, 'elapsedTime': '6429.44', 'cpuLoad': '0.06', 'cpuSys': '0.07', 'diskStats': {'/var/log': {'free': '15189'}, '/var/log/core': {'free': '15189'}, '/var/run/vdsm/': {'free': '52254'}, '/tmp': {'free': '1640028'}}, 'cpuUserVdsmd': '0.40', 'netConfigDirty': 'False', 'memCommitted': 4161, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 98628, 'txRate': '0.00', 'bootTime': '1478279889', 'haStats': {'active': False, 'configured': False, 'score': 0, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'rxDropped': '16928', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'dateTime': '2016-11-04T19:05:45 GMT', 'anonHugePages': '902', 'memFree': 102309, 'cpuIdle': '99.86', 'vmActive': 1, 'v2vJobs': {}, 'cpuSysVdsmd': '0.60'} jsonrpc.Executor/2::INFO::2016-11-04 14:05:45,899::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getStats succeeded in 0.01 seconds jsonrpc.Executor/7::DEBUG::2016-11-04 14:05:47,520::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getAllVmStats' in bridge with {} jsonrpc.Executor/7::DEBUG::2016-11-04 14:05:47,522::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getAllVmStats' in bridge with (suppressed) jsonrpc.Executor/7::INFO::2016-11-04 14:05:47,522::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getAllVmStats succeeded in 0.00 seconds periodic/2::DEBUG::2016-11-04 14:05:51,599::task::597::Storage.TaskManager.Task::(_updateState) Task=`73d5f9ea-2d5d-4564-8224-98df51e54997`::moving from state init -> state preparing periodic/2::INFO::2016-11-04 14:05:51,600::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) periodic/2::INFO::2016-11-04 14:05:51,600::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000483559', 'lastCheck': '4.0', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000682512', 'lastCheck': '4.0', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000524842', 'lastCheck': '4.3', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000391294', 'lastCheck': '4.3', 'valid': True}} periodic/2::DEBUG::2016-11-04 14:05:51,600::task::1193::Storage.TaskManager.Task::(prepare) Task=`73d5f9ea-2d5d-4564-8224-98df51e54997`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000483559', 'lastCheck': '4.0', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000682512', 'lastCheck': '4.0', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000524842', 'lastCheck': '4.3', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000391294', 'lastCheck': '4.3', 'valid': True}} periodic/2::DEBUG::2016-11-04 14:05:51,600::task::597::Storage.TaskManager.Task::(_updateState) Task=`73d5f9ea-2d5d-4564-8224-98df51e54997`::moving from state preparing -> state finished periodic/2::DEBUG::2016-11-04 14:05:51,600::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/2::DEBUG::2016-11-04 14:05:51,600::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/2::DEBUG::2016-11-04 14:05:51,600::task::995::Storage.TaskManager.Task::(_decref) Task=`73d5f9ea-2d5d-4564-8224-98df51e54997`::ref 0 aborting False Thread-12::DEBUG::2016-11-04 14:05:54,263::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:05:54,313::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:05:54,313::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000394215 s, 1.5 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:05:54,344::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:05:54,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:05:54,370::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000456438 s, 773 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:05:54,372::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:05:54,396::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000589195 s, 584 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:05:54,404::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000449944 s, 762 kB/s\n') elapsed=0.04 Reactor thread::INFO::2016-11-04 14:05:59,927::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58264 Reactor thread::DEBUG::2016-11-04 14:05:59,933::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:05:59,933::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58264 Reactor thread::DEBUG::2016-11-04 14:05:59,933::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58264) BindingXMLRPC::INFO::2016-11-04 14:05:59,934::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58264 Thread-1084::INFO::2016-11-04 14:05:59,934::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58264 started Thread-1084::DEBUG::2016-11-04 14:05:59,934::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call getAllVmStats with () {} Thread-1084::DEBUG::2016-11-04 14:05:59,935::bindingxmlrpc::1255::vds::(wrapper) return getAllVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} Thread-1084::INFO::2016-11-04 14:05:59,936::bindingxmlrpc::1282::vds::(wrapper) RPC call getAllVmStats finished (code=0) in 0.00 seconds Thread-1084::INFO::2016-11-04 14:05:59,942::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58264 stopped Reactor thread::INFO::2016-11-04 14:05:59,943::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58266 Reactor thread::DEBUG::2016-11-04 14:05:59,949::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:05:59,949::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58266 Reactor thread::DEBUG::2016-11-04 14:05:59,949::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58266) BindingXMLRPC::INFO::2016-11-04 14:05:59,949::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58266 Thread-1085::INFO::2016-11-04 14:05:59,950::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58266 started Thread-1085::DEBUG::2016-11-04 14:05:59,950::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call vmGetIoTunePolicy with ('cddd2019-5baf-4321-aa8b-130b2889d1b0',) {} Thread-1085::DEBUG::2016-11-04 14:05:59,951::bindingxmlrpc::1255::vds::(wrapper) return vmGetIoTunePolicy with {'ioTunePolicy': [], 'status': {'message': 'Done', 'code': 0}} Thread-1085::INFO::2016-11-04 14:05:59,951::bindingxmlrpc::1282::vds::(wrapper) RPC call vmGetIoTunePolicy finished (code=0) in 0.00 seconds Thread-1085::INFO::2016-11-04 14:05:59,952::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58266 stopped jsonrpc.Executor/1::DEBUG::2016-11-04 14:06:01,934::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getStats' in bridge with {} jsonrpc.Executor/1::DEBUG::2016-11-04 14:06:01,935::task::597::Storage.TaskManager.Task::(_updateState) Task=`9024263b-0d00-4a81-8f8f-c063548825a8`::moving from state init -> state preparing jsonrpc.Executor/1::INFO::2016-11-04 14:06:01,935::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) jsonrpc.Executor/1::INFO::2016-11-04 14:06:01,935::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000456438', 'lastCheck': '4.4', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000449944', 'lastCheck': '4.4', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000589195', 'lastCheck': '4.7', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000394215', 'lastCheck': '4.6', 'valid': True}} jsonrpc.Executor/1::DEBUG::2016-11-04 14:06:01,935::task::1193::Storage.TaskManager.Task::(prepare) Task=`9024263b-0d00-4a81-8f8f-c063548825a8`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000456438', 'lastCheck': '4.4', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000449944', 'lastCheck': '4.4', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000589195', 'lastCheck': '4.7', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000394215', 'lastCheck': '4.6', 'valid': True}} jsonrpc.Executor/1::DEBUG::2016-11-04 14:06:01,936::task::597::Storage.TaskManager.Task::(_updateState) Task=`9024263b-0d00-4a81-8f8f-c063548825a8`::moving from state preparing -> state finished jsonrpc.Executor/1::DEBUG::2016-11-04 14:06:01,936::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} jsonrpc.Executor/1::DEBUG::2016-11-04 14:06:01,936::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} jsonrpc.Executor/1::DEBUG::2016-11-04 14:06:01,936::task::995::Storage.TaskManager.Task::(_decref) Task=`9024263b-0d00-4a81-8f8f-c063548825a8`::ref 0 aborting False jsonrpc.Executor/1::DEBUG::2016-11-04 14:06:01,943::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getStats' in bridge with {'cpuStatistics': {'20': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '21': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '22': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '23': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.80'}, '1': {'cpuUser': '0.60', 'nodeIndex': 0, 'cpuSys': '0.60', 'cpuIdle': '98.80'}, '0': {'cpuUser': '0.53', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.40'}, '3': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '2': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '5': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '4': {'cpuUser': '0.33', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.54'}, '7': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '6': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '9': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '8': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '13': {'cpuUser': '0.53', 'nodeIndex': 0, 'cpuSys': '0.27', 'cpuIdle': '99.20'}, '12': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '15': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '14': {'cpuUser': '0.20', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.73'}, '17': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '16': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '19': {'cpuUser': '0.20', 'nodeIndex': 1, 'cpuSys': '0.13', 'cpuIdle': '99.67'}, '18': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31490'}, '0': {'memPercent': 5, 'memFree': '70616'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'rxRate': '0.01', 'vmCount': 1, 'memUsed': '3', 'storageDomains': {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000456438', 'lastCheck': '4.4', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000449944', 'lastCheck': '4.4', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000589195', 'lastCheck': '4.7', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000394215', 'lastCheck': '4.6', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'eno1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '12156', 'name': 'eno1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286351.527478, 'rx': '5438109', 'state': 'up'}, 'ovirtmgmt': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.1', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '32124339', 'txDropped': '0', 'sampleTime': 1478286351.527478, 'rx': '752769990', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '6816268', 'txDropped': '0', 'sampleTime': 1478286351.527478, 'rx': '6816268', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286351.527478, 'rx': '0', 'state': 'down'}, 'eno2': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno2', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286351.527478, 'rx': '0', 'state': 'down'}, 'vnet0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'vnet0', 'tx': '378097', 'txDropped': '0', 'sampleTime': 1478286351.527478, 'rx': '105210', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286351.527478, 'rx': '0', 'state': 'down'}, 'vdsmbr_AoB33GL7': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '4813', 'name': 'vdsmbr_AoB33GL7', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286351.527478, 'rx': '0', 'state': 'down'}, 'eno3': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno3', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286351.527478, 'rx': '0', 'state': 'down'}, 'eno4': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno4', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286351.527478, 'rx': '0', 'state': 'down'}, 'ens2f0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '10000', 'rxDropped': '0', 'name': 'ens2f0', 'tx': '32852504', 'txDropped': '0', 'sampleTime': 1478286351.527478, 'rx': '782430216', 'state': 'up'}, 'ens2f1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ens2f1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286351.527478, 'rx': '0', 'state': 'down'}}, 'txDropped': '0', 'cpuUser': '0.12', 'ksmPages': 100, 'elapsedTime': '6445.49', 'cpuLoad': '0.06', 'cpuSys': '0.08', 'diskStats': {'/var/log': {'free': '15189'}, '/var/log/core': {'free': '15189'}, '/var/run/vdsm/': {'free': '52254'}, '/tmp': {'free': '1640028'}}, 'cpuUserVdsmd': '0.40', 'netConfigDirty': 'False', 'memCommitted': 4161, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 98630, 'txRate': '0.00', 'bootTime': '1478279889', 'haStats': {'active': False, 'configured': False, 'score': 0, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'rxDropped': '16969', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'dateTime': '2016-11-04T19:06:01 GMT', 'anonHugePages': '902', 'memFree': 102311, 'cpuIdle': '99.81', 'vmActive': 1, 'v2vJobs': {}, 'cpuSysVdsmd': '0.33'} jsonrpc.Executor/1::INFO::2016-11-04 14:06:01,944::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getStats succeeded in 0.01 seconds jsonrpc.Executor/3::DEBUG::2016-11-04 14:06:02,676::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getAllVmStats' in bridge with {} jsonrpc.Executor/3::DEBUG::2016-11-04 14:06:02,677::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getAllVmStats' in bridge with (suppressed) jsonrpc.Executor/3::INFO::2016-11-04 14:06:02,677::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getAllVmStats succeeded in 0.00 seconds Thread-12::DEBUG::2016-11-04 14:06:04,265::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:06:04,313::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:06:04,313::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000445267 s, 1.4 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:06:04,344::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:06:04,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:06:04,372::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000362235 s, 975 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:06:04,374::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:06:04,398::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000491378 s, 700 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:06:04,407::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000436739 s, 785 kB/s\n') elapsed=0.03 periodic/1::DEBUG::2016-11-04 14:06:06,599::task::597::Storage.TaskManager.Task::(_updateState) Task=`d8b40bec-3693-4d19-b47d-2c00381173cb`::moving from state init -> state preparing periodic/1::INFO::2016-11-04 14:06:06,599::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) periodic/1::INFO::2016-11-04 14:06:06,599::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000362235', 'lastCheck': '2.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000436739', 'lastCheck': '2.2', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000491378', 'lastCheck': '2.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000445267', 'lastCheck': '2.3', 'valid': True}} periodic/1::DEBUG::2016-11-04 14:06:06,599::task::1193::Storage.TaskManager.Task::(prepare) Task=`d8b40bec-3693-4d19-b47d-2c00381173cb`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000362235', 'lastCheck': '2.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000436739', 'lastCheck': '2.2', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000491378', 'lastCheck': '2.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000445267', 'lastCheck': '2.3', 'valid': True}} periodic/1::DEBUG::2016-11-04 14:06:06,599::task::597::Storage.TaskManager.Task::(_updateState) Task=`d8b40bec-3693-4d19-b47d-2c00381173cb`::moving from state preparing -> state finished periodic/1::DEBUG::2016-11-04 14:06:06,599::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/1::DEBUG::2016-11-04 14:06:06,600::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/1::DEBUG::2016-11-04 14:06:06,600::task::995::Storage.TaskManager.Task::(_decref) Task=`d8b40bec-3693-4d19-b47d-2c00381173cb`::ref 0 aborting False Thread-12::DEBUG::2016-11-04 14:06:14,268::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:06:14,313::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:06:14,313::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000428546 s, 1.4 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:06:14,344::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:06:14,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:06:14,371::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000468257 s, 754 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:06:14,373::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:06:14,397::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000489551 s, 703 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:06:14,407::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000447534 s, 766 kB/s\n') elapsed=0.04 Reactor thread::INFO::2016-11-04 14:06:14,971::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58268 Reactor thread::DEBUG::2016-11-04 14:06:14,989::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:06:14,990::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58268 Reactor thread::DEBUG::2016-11-04 14:06:14,990::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58268) BindingXMLRPC::INFO::2016-11-04 14:06:14,990::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58268 Thread-1086::INFO::2016-11-04 14:06:14,990::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58268 started Thread-1086::DEBUG::2016-11-04 14:06:14,991::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call getAllVmStats with () {} Thread-1086::DEBUG::2016-11-04 14:06:14,992::bindingxmlrpc::1255::vds::(wrapper) return getAllVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} Thread-1086::INFO::2016-11-04 14:06:14,992::bindingxmlrpc::1282::vds::(wrapper) RPC call getAllVmStats finished (code=0) in 0.00 seconds Thread-1086::INFO::2016-11-04 14:06:14,998::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58268 stopped Reactor thread::INFO::2016-11-04 14:06:14,999::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58270 Reactor thread::DEBUG::2016-11-04 14:06:15,005::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:06:15,005::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58270 Reactor thread::DEBUG::2016-11-04 14:06:15,005::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58270) BindingXMLRPC::INFO::2016-11-04 14:06:15,005::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58270 Thread-1087::INFO::2016-11-04 14:06:15,006::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58270 started Thread-1087::DEBUG::2016-11-04 14:06:15,006::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call vmGetIoTunePolicy with ('cddd2019-5baf-4321-aa8b-130b2889d1b0',) {} Thread-1087::DEBUG::2016-11-04 14:06:15,007::bindingxmlrpc::1255::vds::(wrapper) return vmGetIoTunePolicy with {'ioTunePolicy': [], 'status': {'message': 'Done', 'code': 0}} Thread-1087::INFO::2016-11-04 14:06:15,007::bindingxmlrpc::1282::vds::(wrapper) RPC call vmGetIoTunePolicy finished (code=0) in 0.00 seconds Thread-1087::INFO::2016-11-04 14:06:15,008::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58270 stopped jsonrpc.Executor/4::DEBUG::2016-11-04 14:06:17,426::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getStats' in bridge with {} jsonrpc.Executor/4::DEBUG::2016-11-04 14:06:17,427::task::597::Storage.TaskManager.Task::(_updateState) Task=`3464ab7f-efc4-4a33-bb26-fb6b87f1ff3a`::moving from state init -> state preparing jsonrpc.Executor/4::INFO::2016-11-04 14:06:17,427::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) jsonrpc.Executor/4::INFO::2016-11-04 14:06:17,427::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000468257', 'lastCheck': '3.1', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000447534', 'lastCheck': '3.0', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000489551', 'lastCheck': '0.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000428546', 'lastCheck': '0.1', 'valid': True}} jsonrpc.Executor/4::DEBUG::2016-11-04 14:06:17,427::task::1193::Storage.TaskManager.Task::(prepare) Task=`3464ab7f-efc4-4a33-bb26-fb6b87f1ff3a`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000468257', 'lastCheck': '3.1', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000447534', 'lastCheck': '3.0', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000489551', 'lastCheck': '0.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000428546', 'lastCheck': '0.1', 'valid': True}} jsonrpc.Executor/4::DEBUG::2016-11-04 14:06:17,427::task::597::Storage.TaskManager.Task::(_updateState) Task=`3464ab7f-efc4-4a33-bb26-fb6b87f1ff3a`::moving from state preparing -> state finished jsonrpc.Executor/4::DEBUG::2016-11-04 14:06:17,428::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} jsonrpc.Executor/4::DEBUG::2016-11-04 14:06:17,428::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} jsonrpc.Executor/4::DEBUG::2016-11-04 14:06:17,428::task::995::Storage.TaskManager.Task::(_decref) Task=`3464ab7f-efc4-4a33-bb26-fb6b87f1ff3a`::ref 0 aborting False jsonrpc.Executor/4::DEBUG::2016-11-04 14:06:17,435::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getStats' in bridge with {'cpuStatistics': {'20': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '21': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '22': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '23': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '1': {'cpuUser': '0.53', 'nodeIndex': 0, 'cpuSys': '1.00', 'cpuIdle': '98.47'}, '0': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '3': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '2': {'cpuUser': '0.20', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.73'}, '5': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '4': {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.66'}, '7': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '6': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '9': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '8': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '13': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.40'}, '12': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '15': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '14': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '17': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '16': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '19': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '18': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31490'}, '0': {'memPercent': 5, 'memFree': '70616'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'rxRate': '0.00', 'vmCount': 1, 'memUsed': '3', 'storageDomains': {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000468257', 'lastCheck': '3.1', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000447534', 'lastCheck': '3.0', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000489551', 'lastCheck': '0.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000428546', 'lastCheck': '0.1', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'eno1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '12184', 'name': 'eno1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286366.532322, 'rx': '5449789', 'state': 'up'}, 'ovirtmgmt': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '32143431', 'txDropped': '0', 'sampleTime': 1478286366.532322, 'rx': '752780726', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '6834466', 'txDropped': '0', 'sampleTime': 1478286366.532322, 'rx': '6834466', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286366.532322, 'rx': '0', 'state': 'down'}, 'eno2': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno2', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286366.532322, 'rx': '0', 'state': 'down'}, 'vnet0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'vnet0', 'tx': '378577', 'txDropped': '0', 'sampleTime': 1478286366.532322, 'rx': '105252', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286366.532322, 'rx': '0', 'state': 'down'}, 'vdsmbr_AoB33GL7': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '4820', 'name': 'vdsmbr_AoB33GL7', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286366.532322, 'rx': '0', 'state': 'down'}, 'eno3': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno3', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286366.532322, 'rx': '0', 'state': 'down'}, 'eno4': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno4', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286366.532322, 'rx': '0', 'state': 'down'}, 'ens2f0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '10000', 'rxDropped': '0', 'name': 'ens2f0', 'tx': '32872034', 'txDropped': '0', 'sampleTime': 1478286366.532322, 'rx': '782441845', 'state': 'up'}, 'ens2f1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ens2f1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286366.532322, 'rx': '0', 'state': 'down'}}, 'txDropped': '0', 'cpuUser': '0.06', 'ksmPages': 100, 'elapsedTime': '6460.98', 'cpuLoad': '0.06', 'cpuSys': '0.06', 'diskStats': {'/var/log': {'free': '15189'}, '/var/log/core': {'free': '15189'}, '/var/run/vdsm/': {'free': '52254'}, '/tmp': {'free': '1640028'}}, 'cpuUserVdsmd': '0.40', 'netConfigDirty': 'False', 'memCommitted': 4161, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 98630, 'txRate': '0.00', 'bootTime': '1478279889', 'haStats': {'active': False, 'configured': False, 'score': 0, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'rxDropped': '17004', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'dateTime': '2016-11-04T19:06:17 GMT', 'anonHugePages': '902', 'memFree': 102311, 'cpuIdle': '99.88', 'vmActive': 1, 'v2vJobs': {}, 'cpuSysVdsmd': '0.60'} jsonrpc.Executor/4::INFO::2016-11-04 14:06:17,436::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getStats succeeded in 0.01 seconds jsonrpc.Executor/5::DEBUG::2016-11-04 14:06:18,735::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getAllVmStats' in bridge with {} jsonrpc.Executor/5::DEBUG::2016-11-04 14:06:18,736::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getAllVmStats' in bridge with (suppressed) jsonrpc.Executor/5::INFO::2016-11-04 14:06:18,736::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getAllVmStats succeeded in 0.00 seconds periodic/3::DEBUG::2016-11-04 14:06:21,593::task::597::Storage.TaskManager.Task::(_updateState) Task=`1b5864a3-0a04-424f-b00e-334ddff47e99`::moving from state init -> state preparing periodic/3::INFO::2016-11-04 14:06:21,593::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) periodic/3::INFO::2016-11-04 14:06:21,594::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000468257', 'lastCheck': '4.0', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000447534', 'lastCheck': '4.0', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000489551', 'lastCheck': '4.3', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000428546', 'lastCheck': '4.3', 'valid': True}} periodic/3::DEBUG::2016-11-04 14:06:21,594::task::1193::Storage.TaskManager.Task::(prepare) Task=`1b5864a3-0a04-424f-b00e-334ddff47e99`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000468257', 'lastCheck': '4.0', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000447534', 'lastCheck': '4.0', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000489551', 'lastCheck': '4.3', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000428546', 'lastCheck': '4.3', 'valid': True}} periodic/3::DEBUG::2016-11-04 14:06:21,594::task::597::Storage.TaskManager.Task::(_updateState) Task=`1b5864a3-0a04-424f-b00e-334ddff47e99`::moving from state preparing -> state finished periodic/3::DEBUG::2016-11-04 14:06:21,594::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/3::DEBUG::2016-11-04 14:06:21,594::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/3::DEBUG::2016-11-04 14:06:21,594::task::995::Storage.TaskManager.Task::(_decref) Task=`1b5864a3-0a04-424f-b00e-334ddff47e99`::ref 0 aborting False Thread-12::DEBUG::2016-11-04 14:06:24,267::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:06:24,313::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:06:24,314::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000466099 s, 1.3 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:06:24,344::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:06:24,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:06:24,372::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000480088 s, 735 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:06:24,373::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:06:24,396::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000503352 s, 683 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:06:24,406::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000435854 s, 787 kB/s\n') elapsed=0.03 Reactor thread::INFO::2016-11-04 14:06:30,034::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58272 Reactor thread::DEBUG::2016-11-04 14:06:30,043::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:06:30,043::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58272 Reactor thread::DEBUG::2016-11-04 14:06:30,043::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58272) BindingXMLRPC::INFO::2016-11-04 14:06:30,043::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58272 Thread-1088::INFO::2016-11-04 14:06:30,044::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58272 started Thread-1088::DEBUG::2016-11-04 14:06:30,044::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call getAllVmStats with () {} Thread-1088::DEBUG::2016-11-04 14:06:30,045::bindingxmlrpc::1255::vds::(wrapper) return getAllVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} Thread-1088::INFO::2016-11-04 14:06:30,045::bindingxmlrpc::1282::vds::(wrapper) RPC call getAllVmStats finished (code=0) in 0.00 seconds Thread-1088::INFO::2016-11-04 14:06:30,051::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58272 stopped Reactor thread::INFO::2016-11-04 14:06:30,052::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58274 Reactor thread::DEBUG::2016-11-04 14:06:30,058::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:06:30,059::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58274 Reactor thread::DEBUG::2016-11-04 14:06:30,059::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58274) BindingXMLRPC::INFO::2016-11-04 14:06:30,059::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58274 Thread-1089::INFO::2016-11-04 14:06:30,059::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58274 started Thread-1089::DEBUG::2016-11-04 14:06:30,059::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call vmGetIoTunePolicy with ('cddd2019-5baf-4321-aa8b-130b2889d1b0',) {} Thread-1089::DEBUG::2016-11-04 14:06:30,060::bindingxmlrpc::1255::vds::(wrapper) return vmGetIoTunePolicy with {'ioTunePolicy': [], 'status': {'message': 'Done', 'code': 0}} Thread-1089::INFO::2016-11-04 14:06:30,060::bindingxmlrpc::1282::vds::(wrapper) RPC call vmGetIoTunePolicy finished (code=0) in 0.00 seconds Thread-1089::INFO::2016-11-04 14:06:30,061::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58274 stopped jsonrpc.Executor/6::DEBUG::2016-11-04 14:06:33,139::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getStats' in bridge with {} jsonrpc.Executor/6::DEBUG::2016-11-04 14:06:33,140::task::597::Storage.TaskManager.Task::(_updateState) Task=`7a4018d9-a4b4-4a8d-b1a4-01002a836618`::moving from state init -> state preparing jsonrpc.Executor/6::INFO::2016-11-04 14:06:33,140::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) jsonrpc.Executor/6::INFO::2016-11-04 14:06:33,140::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000480088', 'lastCheck': '5.6', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000435854', 'lastCheck': '5.6', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000503352', 'lastCheck': '5.9', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000466099', 'lastCheck': '5.8', 'valid': True}} jsonrpc.Executor/6::DEBUG::2016-11-04 14:06:33,140::task::1193::Storage.TaskManager.Task::(prepare) Task=`7a4018d9-a4b4-4a8d-b1a4-01002a836618`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000480088', 'lastCheck': '5.6', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000435854', 'lastCheck': '5.6', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000503352', 'lastCheck': '5.9', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000466099', 'lastCheck': '5.8', 'valid': True}} jsonrpc.Executor/6::DEBUG::2016-11-04 14:06:33,140::task::597::Storage.TaskManager.Task::(_updateState) Task=`7a4018d9-a4b4-4a8d-b1a4-01002a836618`::moving from state preparing -> state finished jsonrpc.Executor/6::DEBUG::2016-11-04 14:06:33,140::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} jsonrpc.Executor/6::DEBUG::2016-11-04 14:06:33,141::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} jsonrpc.Executor/6::DEBUG::2016-11-04 14:06:33,141::task::995::Storage.TaskManager.Task::(_decref) Task=`7a4018d9-a4b4-4a8d-b1a4-01002a836618`::ref 0 aborting False jsonrpc.Executor/6::DEBUG::2016-11-04 14:06:33,148::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getStats' in bridge with {'cpuStatistics': {'20': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '21': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '22': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '23': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '1': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.53', 'cpuIdle': '99.07'}, '0': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '3': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '2': {'cpuUser': '0.20', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.80'}, '5': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '4': {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.73'}, '7': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '6': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '9': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '8': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '13': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.47'}, '12': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '15': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '14': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '17': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '16': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '19': {'cpuUser': '0.07', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '18': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31489'}, '0': {'memPercent': 5, 'memFree': '70619'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'rxRate': '0.01', 'vmCount': 1, 'memUsed': '3', 'storageDomains': {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000480088', 'lastCheck': '5.6', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000435854', 'lastCheck': '5.6', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000503352', 'lastCheck': '5.9', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000466099', 'lastCheck': '5.8', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'eno1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '12212', 'name': 'eno1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286381.527218, 'rx': '5462779', 'state': 'up'}, 'ovirtmgmt': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.1', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '32183277', 'txDropped': '0', 'sampleTime': 1478286381.527218, 'rx': '754898384', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '6852696', 'txDropped': '0', 'sampleTime': 1478286381.527218, 'rx': '6852696', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286381.527218, 'rx': '0', 'state': 'down'}, 'eno2': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno2', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286381.527218, 'rx': '0', 'state': 'down'}, 'vnet0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'vnet0', 'tx': '379117', 'txDropped': '0', 'sampleTime': 1478286381.527218, 'rx': '105252', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286381.527218, 'rx': '0', 'state': 'down'}, 'vdsmbr_AoB33GL7': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '4829', 'name': 'vdsmbr_AoB33GL7', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286381.527218, 'rx': '0', 'state': 'down'}, 'eno3': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno3', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286381.527218, 'rx': '0', 'state': 'down'}, 'eno4': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno4', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286381.527218, 'rx': '0', 'state': 'down'}, 'ens2f0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '10000', 'rxDropped': '0', 'name': 'ens2f0', 'tx': '32912613', 'txDropped': '0', 'sampleTime': 1478286381.527218, 'rx': '784646453', 'state': 'up'}, 'ens2f1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ens2f1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286381.527218, 'rx': '0', 'state': 'down'}}, 'txDropped': '0', 'cpuUser': '0.07', 'ksmPages': 100, 'elapsedTime': '6476.69', 'cpuLoad': '0.05', 'cpuSys': '0.04', 'diskStats': {'/var/log': {'free': '15189'}, '/var/log/core': {'free': '15189'}, '/var/run/vdsm/': {'free': '52254'}, '/tmp': {'free': '1640028'}}, 'cpuUserVdsmd': '0.40', 'netConfigDirty': 'False', 'memCommitted': 4161, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 98631, 'txRate': '0.00', 'bootTime': '1478279889', 'haStats': {'active': False, 'configured': False, 'score': 0, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'rxDropped': '17041', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'dateTime': '2016-11-04T19:06:33 GMT', 'anonHugePages': '902', 'memFree': 102312, 'cpuIdle': '99.89', 'vmActive': 1, 'v2vJobs': {}, 'cpuSysVdsmd': '0.33'} jsonrpc.Executor/6::INFO::2016-11-04 14:06:33,149::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getStats succeeded in 0.01 seconds Thread-12::DEBUG::2016-11-04 14:06:34,266::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:06:34,313::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:06:34,313::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.00042259 s, 1.4 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:06:34,344::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:06:34,354::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:06:34,371::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000461919 s, 764 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:06:34,373::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:06:34,397::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000548949 s, 627 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:06:34,405::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000482522 s, 711 kB/s\n') elapsed=0.04 jsonrpc.Executor/0::DEBUG::2016-11-04 14:06:34,814::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getAllVmStats' in bridge with {} jsonrpc.Executor/0::DEBUG::2016-11-04 14:06:34,816::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getAllVmStats' in bridge with (suppressed) jsonrpc.Executor/0::INFO::2016-11-04 14:06:34,817::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getAllVmStats succeeded in 0.00 seconds periodic/2::DEBUG::2016-11-04 14:06:36,462::task::597::Storage.TaskManager.Task::(_updateState) Task=`687268fe-dd08-4b7e-b99b-55795dcdab99`::moving from state init -> state preparing periodic/2::INFO::2016-11-04 14:06:36,462::logUtils::49::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID=u'bfcabdd8-3aa7-4241-91c3-5db0e843e476', spUUID=u'00000001-0001-0001-0001-000000000366', imgUUID=u'b2109025-42be-4c40-ac2b-26baf8c304e5', volUUID=u'3153e04b-729c-44b2-b31a-0b2547ad3a77', options=None) periodic/2::INFO::2016-11-04 14:06:36,464::logUtils::52::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '17503478272', 'apparentsize': '21474836480'} periodic/2::DEBUG::2016-11-04 14:06:36,464::task::1193::Storage.TaskManager.Task::(prepare) Task=`687268fe-dd08-4b7e-b99b-55795dcdab99`::finished: {'truesize': '17503478272', 'apparentsize': '21474836480'} periodic/2::DEBUG::2016-11-04 14:06:36,464::task::597::Storage.TaskManager.Task::(_updateState) Task=`687268fe-dd08-4b7e-b99b-55795dcdab99`::moving from state preparing -> state finished periodic/2::DEBUG::2016-11-04 14:06:36,464::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/2::DEBUG::2016-11-04 14:06:36,464::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/2::DEBUG::2016-11-04 14:06:36,464::task::995::Storage.TaskManager.Task::(_decref) Task=`687268fe-dd08-4b7e-b99b-55795dcdab99`::ref 0 aborting False periodic/3::DEBUG::2016-11-04 14:06:36,598::task::597::Storage.TaskManager.Task::(_updateState) Task=`3f325390-b516-430d-9183-4f215e29d93d`::moving from state init -> state preparing periodic/3::INFO::2016-11-04 14:06:36,598::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) periodic/3::INFO::2016-11-04 14:06:36,598::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000461919', 'lastCheck': '2.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000482522', 'lastCheck': '2.2', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000548949', 'lastCheck': '2.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00042259', 'lastCheck': '2.3', 'valid': True}} periodic/3::DEBUG::2016-11-04 14:06:36,598::task::1193::Storage.TaskManager.Task::(prepare) Task=`3f325390-b516-430d-9183-4f215e29d93d`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000461919', 'lastCheck': '2.2', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000482522', 'lastCheck': '2.2', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000548949', 'lastCheck': '2.2', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00042259', 'lastCheck': '2.3', 'valid': True}} periodic/3::DEBUG::2016-11-04 14:06:36,598::task::597::Storage.TaskManager.Task::(_updateState) Task=`3f325390-b516-430d-9183-4f215e29d93d`::moving from state preparing -> state finished periodic/3::DEBUG::2016-11-04 14:06:36,599::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/3::DEBUG::2016-11-04 14:06:36,599::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/3::DEBUG::2016-11-04 14:06:36,599::task::995::Storage.TaskManager.Task::(_decref) Task=`3f325390-b516-430d-9183-4f215e29d93d`::ref 0 aborting False Thread-12::DEBUG::2016-11-04 14:06:44,266::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:06:44,313::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:06:44,313::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000316188 s, 1.9 MB/s\n') elapsed=0.05 Thread-12::DEBUG::2016-11-04 14:06:44,344::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:06:44,353::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:06:44,371::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000391054 s, 903 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:06:44,373::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:06:44,397::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000339122 s, 1.0 MB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:06:44,407::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.000466648 s, 735 kB/s\n') elapsed=0.03 Reactor thread::INFO::2016-11-04 14:06:45,086::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58276 Reactor thread::DEBUG::2016-11-04 14:06:45,100::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:06:45,100::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58276 Reactor thread::DEBUG::2016-11-04 14:06:45,100::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58276) BindingXMLRPC::INFO::2016-11-04 14:06:45,101::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58276 Thread-1090::INFO::2016-11-04 14:06:45,101::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58276 started Thread-1090::DEBUG::2016-11-04 14:06:45,102::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call getAllVmStats with () {} Thread-1090::DEBUG::2016-11-04 14:06:45,103::bindingxmlrpc::1255::vds::(wrapper) return getAllVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} Thread-1090::INFO::2016-11-04 14:06:45,103::bindingxmlrpc::1282::vds::(wrapper) RPC call getAllVmStats finished (code=0) in 0.01 seconds Thread-1090::INFO::2016-11-04 14:06:45,108::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58276 stopped Reactor thread::INFO::2016-11-04 14:06:45,109::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.98:58278 Reactor thread::DEBUG::2016-11-04 14:06:45,115::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:06:45,116::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::ffff:10.50.3.98:58278 Reactor thread::DEBUG::2016-11-04 14:06:45,116::bindingxmlrpc::1302::XmlDetector::(handle_socket) xml over http detected from ('::ffff:10.50.3.98', 58278) BindingXMLRPC::INFO::2016-11-04 14:06:45,116::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::ffff:10.50.3.98:58278 Thread-1091::INFO::2016-11-04 14:06:45,116::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58278 started Thread-1091::DEBUG::2016-11-04 14:06:45,117::bindingxmlrpc::1247::vds::(wrapper) client [::ffff:10.50.3.98]::call vmGetIoTunePolicy with ('cddd2019-5baf-4321-aa8b-130b2889d1b0',) {} Thread-1091::DEBUG::2016-11-04 14:06:45,117::bindingxmlrpc::1255::vds::(wrapper) return vmGetIoTunePolicy with {'ioTunePolicy': [], 'status': {'message': 'Done', 'code': 0}} Thread-1091::INFO::2016-11-04 14:06:45,117::bindingxmlrpc::1282::vds::(wrapper) RPC call vmGetIoTunePolicy finished (code=0) in 0.00 seconds Thread-1091::INFO::2016-11-04 14:06:45,118::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::ffff:10.50.3.98:58278 stopped jsonrpc.Executor/2::DEBUG::2016-11-04 14:06:46,634::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'VM.migrate' in bridge with {u'params': {u'tunneled': u'false', u'dstqemu': u'10.50.3.97', u'autoConverge': u'true', u'src': u'labvmhostt04.lab.ingram.io', u'enableGuestEvents': True, u'dst': u'labvmhostt05.lab.ingram.io:54321', u'convergenceSchedule': {u'init': [{u'params': [u'100'], u'name': u'setDowntime'}], u'stalling': [{u'action': {u'params': [u'150'], u'name': u'setDowntime'}, u'limit': 1}, {u'action': {u'params': [u'200'], u'name': u'setDowntime'}, u'limit': 2}, {u'action': {u'params': [u'300'], u'name': u'setDowntime'}, u'limit': 3}, {u'action': {u'params': [u'400'], u'name': u'setDowntime'}, u'limit': 4}, {u'action': {u'params': [u'500'], u'name': u'setDowntime'}, u'limit': 6}, {u'action': {u'params': [u'5000'], u'name': u'setDowntime'}, u'limit': -1}, {u'action': {u'params': [], u'name': u'abort'}, u'limit': -1}]}, u'vmId': u'cddd2019-5baf-4321-aa8b-130b2889d1b0', u'abortOnError': u'true', u'compressed': u'true', u'maxBandwidth': 10000, u'method': u'online'}, u'vmID': u'cddd2019-5baf-4321-aa8b-130b2889d1b0'} jsonrpc.Executor/2::DEBUG::2016-11-04 14:06:46,636::API::540::vds::(migrate) {u'tunneled': u'false', u'dstqemu': u'10.50.3.97', u'autoConverge': u'true', u'src': u'labvmhostt04.lab.ingram.io', u'enableGuestEvents': True, u'dst': u'labvmhostt05.lab.ingram.io:54321', u'convergenceSchedule': {u'init': [{u'params': [u'100'], u'name': u'setDowntime'}], u'stalling': [{u'action': {u'params': [u'150'], u'name': u'setDowntime'}, u'limit': 1}, {u'action': {u'params': [u'200'], u'name': u'setDowntime'}, u'limit': 2}, {u'action': {u'params': [u'300'], u'name': u'setDowntime'}, u'limit': 3}, {u'action': {u'params': [u'400'], u'name': u'setDowntime'}, u'limit': 4}, {u'action': {u'params': [u'500'], u'name': u'setDowntime'}, u'limit': 6}, {u'action': {u'params': [u'5000'], u'name': u'setDowntime'}, u'limit': -1}, {u'action': {u'params': [], u'name': u'abort'}, u'limit': -1}]}, u'vmId': u'cddd2019-5baf-4321-aa8b-130b2889d1b0', u'abortOnError': u'true', u'compressed': u'true', u'maxBandwidth': 10000, u'method': u'online'} jsonrpc.Executor/2::DEBUG::2016-11-04 14:06:46,637::migration::136::virt.vm::(__init__) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::convergence schedule set to: {u'init': [{u'params': [u'100'], u'name': u'setDowntime'}], u'stalling': [{u'action': {u'params': [u'150'], u'name': u'setDowntime'}, u'limit': 1}, {u'action': {u'params': [u'200'], u'name': u'setDowntime'}, u'limit': 2}, {u'action': {u'params': [u'300'], u'name': u'setDowntime'}, u'limit': 3}, {u'action': {u'params': [u'400'], u'name': u'setDowntime'}, u'limit': 4}, {u'action': {u'params': [u'500'], u'name': u'setDowntime'}, u'limit': 6}, {u'action': {u'params': [u'5000'], u'name': u'setDowntime'}, u'limit': -1}, {u'action': {u'params': [], u'name': u'abort'}, u'limit': -1}]} jsonrpc.Executor/2::DEBUG::2016-11-04 14:06:46,640::__init__::208::jsonrpc.Notification::(emit) Sending event {"params": {"notify_time": 4301184910, "cddd2019-5baf-4321-aa8b-130b2889d1b0": {"status": "Migration Source"}}, "jsonrpc": "2.0", "method": "|virt|VM_status|cddd2019-5baf-4321-aa8b-130b2889d1b0"} jsonrpc.Executor/2::DEBUG::2016-11-04 14:06:46,640::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'VM.migrate' in bridge with True jsonrpc.Executor/2::INFO::2016-11-04 14:06:46,640::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call VM.migrate succeeded in 0.00 seconds JsonRpc (StompReactor)::DEBUG::2016-11-04 14:06:46,649::stomp::488::yajsonrpc.protocols.stomp.AsyncClient::(_process_connected) Stomp connection established jsonrpc.Executor/7::DEBUG::2016-11-04 14:06:48,657::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getStats' in bridge with {} jsonrpc.Executor/7::DEBUG::2016-11-04 14:06:48,658::task::597::Storage.TaskManager.Task::(_updateState) Task=`387b6af7-c01c-4d1c-9ac0-c596577e5508`::moving from state init -> state preparing jsonrpc.Executor/7::INFO::2016-11-04 14:06:48,659::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) jsonrpc.Executor/7::INFO::2016-11-04 14:06:48,659::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000391054', 'lastCheck': '1.1', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000466648', 'lastCheck': '1.1', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000339122', 'lastCheck': '1.4', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000316188', 'lastCheck': '1.3', 'valid': True}} jsonrpc.Executor/7::DEBUG::2016-11-04 14:06:48,659::task::1193::Storage.TaskManager.Task::(prepare) Task=`387b6af7-c01c-4d1c-9ac0-c596577e5508`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000391054', 'lastCheck': '1.1', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000466648', 'lastCheck': '1.1', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000339122', 'lastCheck': '1.4', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000316188', 'lastCheck': '1.3', 'valid': True}} jsonrpc.Executor/7::DEBUG::2016-11-04 14:06:48,659::task::597::Storage.TaskManager.Task::(_updateState) Task=`387b6af7-c01c-4d1c-9ac0-c596577e5508`::moving from state preparing -> state finished jsonrpc.Executor/7::DEBUG::2016-11-04 14:06:48,659::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} jsonrpc.Executor/7::DEBUG::2016-11-04 14:06:48,659::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} jsonrpc.Executor/7::DEBUG::2016-11-04 14:06:48,659::task::995::Storage.TaskManager.Task::(_decref) Task=`387b6af7-c01c-4d1c-9ac0-c596577e5508`::ref 0 aborting False jsonrpc.Executor/7::DEBUG::2016-11-04 14:06:48,666::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getStats' in bridge with {'cpuStatistics': {'20': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '21': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '22': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '23': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '1': {'cpuUser': '0.47', 'nodeIndex': 0, 'cpuSys': '1.00', 'cpuIdle': '98.53'}, '0': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '3': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '2': {'cpuUser': '0.20', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.67'}, '5': {'cpuUser': '0.27', 'nodeIndex': 0, 'cpuSys': '0.13', 'cpuIdle': '99.60'}, '4': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.87'}, '7': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '6': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '9': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '8': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '11': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '13': {'cpuUser': '0.40', 'nodeIndex': 0, 'cpuSys': '0.20', 'cpuIdle': '99.40'}, '12': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '15': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '14': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '17': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '16': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '19': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '18': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}}, 'numaNodeMemFree': {'1': {'memPercent': 4, 'memFree': '31488'}, '0': {'memPercent': 5, 'memFree': '70618'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'rxRate': '0.01', 'vmCount': 1, 'memUsed': '3', 'storageDomains': {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000391054', 'lastCheck': '1.1', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000466648', 'lastCheck': '1.1', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000339122', 'lastCheck': '1.4', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000316188', 'lastCheck': '1.3', 'valid': True}}, 'incomingVmMigrations': 0, 'network': {'eno1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '12240', 'name': 'eno1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286396.531961, 'rx': '5476271', 'state': 'up'}, 'ovirtmgmt': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.1', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovirtmgmt', 'tx': '32216949', 'txDropped': '0', 'sampleTime': 1478286396.531961, 'rx': '757019396', 'state': 'up'}, 'lo': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'lo', 'tx': '6870910', 'txDropped': '0', 'sampleTime': 1478286396.531961, 'rx': '6870910', 'state': 'up'}, 'ovs-system': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ovs-system', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286396.531961, 'rx': '0', 'state': 'down'}, 'eno2': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno2', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286396.531961, 'rx': '0', 'state': 'down'}, 'vnet0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'vnet0', 'tx': '379357', 'txDropped': '0', 'sampleTime': 1478286396.531961, 'rx': '105252', 'state': 'up'}, ';vdsmdummy;': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286396.531961, 'rx': '0', 'state': 'down'}, 'vdsmbr_AoB33GL7': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '4833', 'name': 'vdsmbr_AoB33GL7', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286396.531961, 'rx': '0', 'state': 'down'}, 'eno3': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno3', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286396.531961, 'rx': '0', 'state': 'down'}, 'eno4': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'eno4', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286396.531961, 'rx': '0', 'state': 'down'}, 'ens2f0': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '10000', 'rxDropped': '0', 'name': 'ens2f0', 'tx': '32946953', 'txDropped': '0', 'sampleTime': 1478286396.531961, 'rx': '786853805', 'state': 'up'}, 'ens2f1': {'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'speed': '1000', 'rxDropped': '0', 'name': 'ens2f1', 'tx': '0', 'txDropped': '0', 'sampleTime': 1478286396.531961, 'rx': '0', 'state': 'down'}}, 'txDropped': '0', 'cpuUser': '0.06', 'ksmPages': 100, 'elapsedTime': '6492.21', 'cpuLoad': '0.05', 'cpuSys': '0.07', 'diskStats': {'/var/log': {'free': '15189'}, '/var/log/core': {'free': '15189'}, '/var/run/vdsm/': {'free': '52254'}, '/tmp': {'free': '1640028'}}, 'cpuUserVdsmd': '0.47', 'netConfigDirty': 'False', 'memCommitted': 4161, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 98594, 'txRate': '0.00', 'bootTime': '1478279889', 'haStats': {'active': False, 'configured': False, 'score': 0, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'rxDropped': '17073', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'dateTime': '2016-11-04T19:06:48 GMT', 'anonHugePages': '902', 'memFree': 102274, 'cpuIdle': '99.87', 'vmActive': 1, 'v2vJobs': {}, 'cpuSysVdsmd': '0.60'} jsonrpc.Executor/7::INFO::2016-11-04 14:06:48,667::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getStats succeeded in 0.01 seconds Thread-1092::DEBUG::2016-11-04 14:06:48,820::migration::188::virt.vm::(_setupVdsConnection) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::Initiating connection with destination Thread-1092::DEBUG::2016-11-04 14:06:48,821::stompreactor::408::jsonrpc.AsyncoreClient::(send) Sending response Thread-1092::DEBUG::2016-11-04 14:06:48,823::migration::200::virt.vm::(_setupVdsConnection) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::Destination server is: labvmhostt05.lab.ingram.io:54321 Thread-1092::DEBUG::2016-11-04 14:06:48,825::migration::246::virt.vm::(_prepareGuest) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::Migration started Thread-1092::DEBUG::2016-11-04 14:06:48,838::guestagent::502::virt.vm::(send_lifecycle_event) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::send_lifecycle_event before_migration called Thread-1092::DEBUG::2016-11-04 14:06:48,859::migration::353::virt.vm::(run) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::migration semaphore acquired after 2 seconds Thread-1092::DEBUG::2016-11-04 14:06:48,873::stompreactor::408::jsonrpc.AsyncoreClient::(send) Sending response Thread-1092::INFO::2016-11-04 14:06:48,923::migration::407::virt.vm::(_startUnderlyingMigration) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::Creation of destination VM took: 0 seconds Thread-1092::INFO::2016-11-04 14:06:48,923::migration::429::virt.vm::(_startUnderlyingMigration) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::starting migration to qemu+tls://labvmhostt05.lab.ingram.io/system with miguri tcp://10.50.3.97 Thread-1092::DEBUG::2016-11-04 14:06:48,923::migration::494::virt.vm::(_perform_with_conv_schedule) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::performing migration with conv schedule Thread-1093::DEBUG::2016-11-04 14:06:48,924::migration::620::virt.vm::(run) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::starting migration monitor thread Thread-1093::DEBUG::2016-11-04 14:06:48,924::migration::739::virt.vm::(_execute_action_with_params) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::Setting downtime to 100 Thread-1092::DEBUG::2016-11-04 14:06:49,026::migration::715::virt.vm::(stop) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::stopping migration monitor thread Thread-1092::ERROR::2016-11-04 14:06:49,026::migration::252::virt.vm::(_recover) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::Cannot get interface MTU on 'vdsmbr_AoB33GL7': No such device Thread-1092::DEBUG::2016-11-04 14:06:49,027::stompreactor::408::jsonrpc.AsyncoreClient::(send) Sending response Thread-1093::DEBUG::2016-11-04 14:06:49,027::migration::629::virt.vm::(run) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::stopped migration monitor thread Thread-1092::DEBUG::2016-11-04 14:06:49,034::guestagent::502::virt.vm::(send_lifecycle_event) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::send_lifecycle_event after_migration called Thread-1092::DEBUG::2016-11-04 14:06:49,047::__init__::208::jsonrpc.Notification::(emit) Sending event {"params": {"notify_time": 4301187320, "cddd2019-5baf-4321-aa8b-130b2889d1b0": {"status": "Migration Source"}}, "jsonrpc": "2.0", "method": "|virt|VM_status|cddd2019-5baf-4321-aa8b-130b2889d1b0"} Thread-1092::ERROR::2016-11-04 14:06:49,047::migration::381::virt.vm::(run) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::Failed to migrate Traceback (most recent call last): File "/usr/share/vdsm/virt/migration.py", line 363, in run self._startUnderlyingMigration(time.time()) File "/usr/share/vdsm/virt/migration.py", line 436, in _startUnderlyingMigration self._perform_with_conv_schedule(duri, muri) File "/usr/share/vdsm/virt/migration.py", line 496, in _perform_with_conv_schedule self._perform_migration(duri, muri) File "/usr/share/vdsm/virt/migration.py", line 476, in _perform_migration self._vm._dom.migrateToURI3(duri, params, flags) File "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py", line 69, in f ret = attr(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line 123, in wrapper ret = f(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 916, in wrapper return func(inst, *args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1836, in migrateToURI3 if ret == -1: raise libvirtError ('virDomainMigrateToURI3() failed', dom=self) libvirtError: Cannot get interface MTU on 'vdsmbr_AoB33GL7': No such device jsonrpc.Executor/1::DEBUG::2016-11-04 14:06:50,933::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getAllVmStats' in bridge with {} jsonrpc.Executor/1::DEBUG::2016-11-04 14:06:50,934::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getAllVmStats' in bridge with (suppressed) jsonrpc.Executor/1::INFO::2016-11-04 14:06:50,934::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getAllVmStats succeeded in 0.00 seconds periodic/1::DEBUG::2016-11-04 14:06:51,592::task::597::Storage.TaskManager.Task::(_updateState) Task=`1a2a2013-0955-42fd-9b4d-f8f421622ae4`::moving from state init -> state preparing periodic/1::INFO::2016-11-04 14:06:51,592::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None) periodic/1::INFO::2016-11-04 14:06:51,592::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000391054', 'lastCheck': '4.0', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000466648', 'lastCheck': '4.0', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000339122', 'lastCheck': '4.3', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000316188', 'lastCheck': '4.3', 'valid': True}} periodic/1::DEBUG::2016-11-04 14:06:51,593::task::1193::Storage.TaskManager.Task::(prepare) Task=`1a2a2013-0955-42fd-9b4d-f8f421622ae4`::finished: {u'9bee9944-821e-4ec7-8513-e92131d2ad76': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000391054', 'lastCheck': '4.0', 'valid': True}, u'bfcabdd8-3aa7-4241-91c3-5db0e843e476': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000466648', 'lastCheck': '4.0', 'valid': True}, u'9d32932b-0e43-4377-b910-35fe2830c9e9': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000339122', 'lastCheck': '4.3', 'valid': True}, u'd129a955-0810-4cb1-9ef5-86cca703a56d': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.000316188', 'lastCheck': '4.3', 'valid': True}} periodic/1::DEBUG::2016-11-04 14:06:51,593::task::597::Storage.TaskManager.Task::(_updateState) Task=`1a2a2013-0955-42fd-9b4d-f8f421622ae4`::moving from state preparing -> state finished periodic/1::DEBUG::2016-11-04 14:06:51,593::resourceManager::952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} periodic/1::DEBUG::2016-11-04 14:06:51,593::resourceManager::989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} periodic/1::DEBUG::2016-11-04 14:06:51,593::task::995::Storage.TaskManager.Task::(_decref) Task=`1a2a2013-0955-42fd-9b4d-f8f421622ae4`::ref 0 aborting False jsonrpc.Executor/3::DEBUG::2016-11-04 14:06:52,016::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'VM.getMigrationStatus' in bridge with {u'vmID': u'cddd2019-5baf-4321-aa8b-130b2889d1b0'} jsonrpc.Executor/3::DEBUG::2016-11-04 14:06:52,017::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'VM.getMigrationStatus' in bridge with {'status': {'message': 'Fatal error during migration', 'code': 12}, 'progress': 0} jsonrpc.Executor/3::INFO::2016-11-04 14:06:52,017::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call VM.getMigrationStatus succeeded in 0.00 seconds jsonrpc.Executor/4::DEBUG::2016-11-04 14:06:53,401::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'VM.migrate' in bridge with {u'params': {u'tunneled': u'false', u'dstqemu': u'10.50.3.100', u'autoConverge': u'true', u'src': u'labvmhostt04.lab.ingram.io', u'enableGuestEvents': True, u'dst': u'labvmhostt01.lab.ingram.io:54321', u'convergenceSchedule': {u'init': [{u'params': [u'100'], u'name': u'setDowntime'}], u'stalling': [{u'action': {u'params': [u'150'], u'name': u'setDowntime'}, u'limit': 1}, {u'action': {u'params': [u'200'], u'name': u'setDowntime'}, u'limit': 2}, {u'action': {u'params': [u'300'], u'name': u'setDowntime'}, u'limit': 3}, {u'action': {u'params': [u'400'], u'name': u'setDowntime'}, u'limit': 4}, {u'action': {u'params': [u'500'], u'name': u'setDowntime'}, u'limit': 6}, {u'action': {u'params': [u'5000'], u'name': u'setDowntime'}, u'limit': -1}, {u'action': {u'params': [], u'name': u'abort'}, u'limit': -1}]}, u'vmId': u'cddd2019-5baf-4321-aa8b-130b2889d1b0', u'abortOnError': u'true', u'compressed': u'true', u'maxBandwidth': 10000, u'method': u'online'}, u'vmID': u'cddd2019-5baf-4321-aa8b-130b2889d1b0'} jsonrpc.Executor/4::DEBUG::2016-11-04 14:06:53,402::API::540::vds::(migrate) {u'tunneled': u'false', u'dstqemu': u'10.50.3.100', u'autoConverge': u'true', u'src': u'labvmhostt04.lab.ingram.io', u'enableGuestEvents': True, u'dst': u'labvmhostt01.lab.ingram.io:54321', u'convergenceSchedule': {u'init': [{u'params': [u'100'], u'name': u'setDowntime'}], u'stalling': [{u'action': {u'params': [u'150'], u'name': u'setDowntime'}, u'limit': 1}, {u'action': {u'params': [u'200'], u'name': u'setDowntime'}, u'limit': 2}, {u'action': {u'params': [u'300'], u'name': u'setDowntime'}, u'limit': 3}, {u'action': {u'params': [u'400'], u'name': u'setDowntime'}, u'limit': 4}, {u'action': {u'params': [u'500'], u'name': u'setDowntime'}, u'limit': 6}, {u'action': {u'params': [u'5000'], u'name': u'setDowntime'}, u'limit': -1}, {u'action': {u'params': [], u'name': u'abort'}, u'limit': -1}]}, u'vmId': u'cddd2019-5baf-4321-aa8b-130b2889d1b0', u'abortOnError': u'true', u'compressed': u'true', u'maxBandwidth': 10000, u'method': u'online'} jsonrpc.Executor/4::DEBUG::2016-11-04 14:06:53,403::migration::136::virt.vm::(__init__) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::convergence schedule set to: {u'init': [{u'params': [u'100'], u'name': u'setDowntime'}], u'stalling': [{u'action': {u'params': [u'150'], u'name': u'setDowntime'}, u'limit': 1}, {u'action': {u'params': [u'200'], u'name': u'setDowntime'}, u'limit': 2}, {u'action': {u'params': [u'300'], u'name': u'setDowntime'}, u'limit': 3}, {u'action': {u'params': [u'400'], u'name': u'setDowntime'}, u'limit': 4}, {u'action': {u'params': [u'500'], u'name': u'setDowntime'}, u'limit': 6}, {u'action': {u'params': [u'5000'], u'name': u'setDowntime'}, u'limit': -1}, {u'action': {u'params': [], u'name': u'abort'}, u'limit': -1}]} jsonrpc.Executor/4::DEBUG::2016-11-04 14:06:53,405::__init__::208::jsonrpc.Notification::(emit) Sending event {"params": {"notify_time": 4301191680, "cddd2019-5baf-4321-aa8b-130b2889d1b0": {"status": "Migration Source"}}, "jsonrpc": "2.0", "method": "|virt|VM_status|cddd2019-5baf-4321-aa8b-130b2889d1b0"} jsonrpc.Executor/4::DEBUG::2016-11-04 14:06:53,405::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'VM.migrate' in bridge with True jsonrpc.Executor/4::INFO::2016-11-04 14:06:53,405::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call VM.migrate succeeded in 0.01 seconds JsonRpc (StompReactor)::DEBUG::2016-11-04 14:06:53,414::stomp::488::yajsonrpc.protocols.stomp.AsyncClient::(_process_connected) Stomp connection established Thread-12::DEBUG::2016-11-04 14:06:54,267::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.01 Thread-12::DEBUG::2016-11-04 14:06:54,343::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:06:54,344::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.103:_data1_ovirt__data02/d129a955-0810-4cb1-9ef5-86cca703a56d/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n605 bytes (605 B) copied, 0.000345215 s, 1.8 MB/s\n') elapsed=0.08 Thread-12::DEBUG::2016-11-04 14:06:54,344::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:06:54,356::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:06:54,371::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__exports/9bee9944-821e-4ec7-8513-e92131d2ad76/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n353 bytes (353 B) copied, 0.000473154 s, 746 kB/s\n') elapsed=0.02 Thread-12::DEBUG::2016-11-04 14:06:54,372::check::296::storage.check::(_start_process) START check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' cmd=['/usr/bin/taskset', '--cpu-list', '0-23', '/usr/bin/dd', u'if=/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata', 'of=/dev/null', 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 Thread-12::DEBUG::2016-11-04 14:06:54,395::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/labvmfst01.lab.ingram.io:_data1_ovirt__isos/9d32932b-0e43-4377-b910-35fe2830c9e9/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n344 bytes (344 B) copied, 0.000522554 s, 658 kB/s\n') elapsed=0.04 Thread-12::DEBUG::2016-11-04 14:06:54,493::asyncevent::564::storage.asyncevent::(reap) Process terminated (count=1) Thread-12::DEBUG::2016-11-04 14:06:54,494::check::327::storage.check::(_check_completed) FINISH check u'/rhev/data-center/mnt/10.50.3.102:_data1_ovirt__data01/bfcabdd8-3aa7-4241-91c3-5db0e843e476/dom_md/metadata' rc=0 err=bytearray(b'0+1 records in\n0+1 records out\n343 bytes (343 B) copied, 0.00038934 s, 881 kB/s\n') elapsed=0.13 Thread-1094::DEBUG::2016-11-04 14:06:55,605::migration::188::virt.vm::(_setupVdsConnection) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::Initiating connection with destination Thread-1094::DEBUG::2016-11-04 14:06:55,606::stompreactor::408::jsonrpc.AsyncoreClient::(send) Sending response Thread-1094::DEBUG::2016-11-04 14:06:55,608::migration::200::virt.vm::(_setupVdsConnection) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::Destination server is: labvmhostt01.lab.ingram.io:54321 Thread-1094::DEBUG::2016-11-04 14:06:55,610::migration::246::virt.vm::(_prepareGuest) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::Migration started Thread-1094::DEBUG::2016-11-04 14:06:55,623::guestagent::502::virt.vm::(send_lifecycle_event) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::send_lifecycle_event before_migration called Thread-1094::DEBUG::2016-11-04 14:06:55,671::migration::353::virt.vm::(run) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::migration semaphore acquired after 2 seconds Thread-1094::DEBUG::2016-11-04 14:06:55,685::stompreactor::408::jsonrpc.AsyncoreClient::(send) Sending response Thread-1094::INFO::2016-11-04 14:06:55,736::migration::407::virt.vm::(_startUnderlyingMigration) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::Creation of destination VM took: 0 seconds Thread-1094::INFO::2016-11-04 14:06:55,736::migration::429::virt.vm::(_startUnderlyingMigration) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::starting migration to qemu+tls://labvmhostt01.lab.ingram.io/system with miguri tcp://10.50.3.100 Thread-1094::DEBUG::2016-11-04 14:06:55,736::migration::494::virt.vm::(_perform_with_conv_schedule) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::performing migration with conv schedule Thread-1095::DEBUG::2016-11-04 14:06:55,737::migration::620::virt.vm::(run) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::starting migration monitor thread Thread-1095::DEBUG::2016-11-04 14:06:55,737::migration::739::virt.vm::(_execute_action_with_params) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::Setting downtime to 100 Thread-1094::DEBUG::2016-11-04 14:06:55,819::migration::715::virt.vm::(stop) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::stopping migration monitor thread Thread-1094::ERROR::2016-11-04 14:06:55,819::migration::252::virt.vm::(_recover) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::Cannot get interface MTU on 'vdsmbr_AoB33GL7': No such device Thread-1094::DEBUG::2016-11-04 14:06:55,819::stompreactor::408::jsonrpc.AsyncoreClient::(send) Sending response Thread-1095::DEBUG::2016-11-04 14:06:55,819::migration::629::virt.vm::(run) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::stopped migration monitor thread Thread-1094::DEBUG::2016-11-04 14:06:55,826::guestagent::502::virt.vm::(send_lifecycle_event) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::send_lifecycle_event after_migration called Thread-1094::DEBUG::2016-11-04 14:06:55,838::__init__::208::jsonrpc.Notification::(emit) Sending event {"params": {"notify_time": 4301194110, "cddd2019-5baf-4321-aa8b-130b2889d1b0": {"status": "Migration Source"}}, "jsonrpc": "2.0", "method": "|virt|VM_status|cddd2019-5baf-4321-aa8b-130b2889d1b0"} Thread-1094::ERROR::2016-11-04 14:06:55,838::migration::381::virt.vm::(run) vmId=`cddd2019-5baf-4321-aa8b-130b2889d1b0`::Failed to migrate Traceback (most recent call last): File "/usr/share/vdsm/virt/migration.py", line 363, in run self._startUnderlyingMigration(time.time()) File "/usr/share/vdsm/virt/migration.py", line 436, in _startUnderlyingMigration self._perform_with_conv_schedule(duri, muri) File "/usr/share/vdsm/virt/migration.py", line 496, in _perform_with_conv_schedule self._perform_migration(duri, muri) File "/usr/share/vdsm/virt/migration.py", line 476, in _perform_migration self._vm._dom.migrateToURI3(duri, params, flags) File "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py", line 69, in f ret = attr(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line 123, in wrapper ret = f(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 916, in wrapper return func(inst, *args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1836, in migrateToURI3 if ret == -1: raise libvirtError ('virDomainMigrateToURI3() failed', dom=self) libvirtError: Cannot get interface MTU on 'vdsmbr_AoB33GL7': No such device Reactor thread::INFO::2016-11-04 14:06:57,444::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepting connection from ::ffff:10.50.3.97:35072 Reactor thread::DEBUG::2016-11-04 14:06:57,450::protocoldetector::85::ProtocolDetector.Detector::(__init__) Using required_size=11 Reactor thread::INFO::2016-11-04 14:06:57,450::protocoldetector::121::ProtocolDetector.Detector::(handle_read) Detected protocol stomp from ::ffff:10.50.3.97:35072 Reactor thread::INFO::2016-11-04 14:06:57,451::stompreactor::101::Broker.StompAdapter::(_cmd_connect) Processing CONNECT request Reactor thread::DEBUG::2016-11-04 14:06:57,451::stompreactor::492::protocoldetector.StompDetector::(handle_socket) Stomp detected from ('::ffff:10.50.3.97', 35072) JsonRpc (StompReactor)::INFO::2016-11-04 14:06:57,451::stompreactor::128::Broker.StompAdapter::(_cmd_subscribe) Subscribe command received jsonrpc.Executor/5::DEBUG::2016-11-04 14:06:58,702::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.ping' in bridge with {} jsonrpc.Executor/5::DEBUG::2016-11-04 14:06:58,702::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.ping' in bridge with True jsonrpc.Executor/5::INFO::2016-11-04 14:06:58,702::__init__::512::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.ping succeeded in 0.00 seconds jsonrpc.Executor/6::DEBUG::2016-11-04 14:06:58,721::__init__::529::jsonrpc.JsonRpcServer::(_handle_request) Calling 'VM.migrationCreate' in bridge with {u'vmID': u'15f6581b-da91-4f9d-ad3f-4c78ada4e334', u'params': {u'guestFQDN': u'', u'acpiEnable': u'true', u'emulatedMachine': u'pc-i440fx-rhel7.2.0', u'afterMigrationStatus': u'', u'enableGuestEvents': True, u'vmId': u'15f6581b-da91-4f9d-ad3f-4c78ada4e334', u'elapsedTimeOffset': 3197.7830469608307, u'memGuaranteedSize': 2730, u'transparentHugePages': u'true', u'displayPort': u'5900', u'displaySecurePort': u'-1', u'timeOffset': u'0', u'cpuType': u'SandyBridge', u'custom': {}, u'pauseCode': u'NOERR', u'migrationDest': u'libvirt', u'guestNumaNodes': [{u'nodeIndex': 0, u'cpus': u'0,1', u'memory': u'4096'}], u'smartcardEnable': u'false', u'smp': u'2', u'vmType': u'kvm', u'_srcDomXML': u'\n testbunny\n 15f6581b-da91-4f9d-ad3f-4c78ada4e334\n \n \n \n 4294967296\n 4194304\n 4194304\n 16\n \n 1020\n \n \n /machine\n \n \n \n oVirt\n oVirt Node\n 7-2.1511.el7.centos.2.10\n 4C4C4544-0034-3010-8048-B9C04F514432\n 15f6581b-da91-4f9d-ad3f-4c78ada4e334\n \n \n \n hvm\n \n \n \n \n \n \n SandyBridge\n \n \n \n \n \n \n \n \n \n \n destroy\n restart\n destroy\n \n /usr/libexec/qemu-kvm\n \n \n \n \n \n \n \n \n \n \n
\n \n \n \n
\n \n \n \n
\n \n \n \n
\n \n \n \n \n \n \n
\n \n \n \n \n \n \n \n \n \n \n \n \n
\n \n \n \n \n \n
\n \n \n \n \n \n
\n \n \n \n \n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n \n