5:22 a.m.
This is a multi-part message in MIME format.
--------------060401020402070903050401
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Hi,
I'm adding two twin nodes to a cluster and I was able to add the first
while the second fails as it removes the iscsi sessions and don't bring
them back and thus fails with the message :
Host Dell4 cannot access one of the Storage Domains attached to the Data
Center Default. Setting Host state to Non-Operational.
Here is the vdsm.log entries for one activationintent:
Thread-20::DEBUG::2013-12-04
09:02:41,760::task::579::TaskManager.Task::(_updateState)
Task=`e3529e49-3083-422e-b2e6-c7d8c47b886a`::moving from state init ->
state preparing
Thread-20::INFO::2013-12-04
09:02:41,760::logUtils::44::dispatcher::(wrapper) Run and protect:
repoStats(options=None)
Thread-20::INFO::2013-12-04
09:02:41,760::logUtils::47::dispatcher::(wrapper) Run and protect:
repoStats, Return response: {}
Thread-20::DEBUG::2013-12-04
09:02:41,760::task::1168::TaskManager.Task::(prepare)
Task=`e3529e49-3083-422e-b2e6-c7d8c47b886a`::finished: {}
Thread-20::DEBUG::2013-12-04
09:02:41,761::task::579::TaskManager.Task::(_updateState)
Task=`e3529e49-3083-422e-b2e6-c7d8c47b886a`::moving from state preparing
-> state finish
ed
Thread-20::DEBUG::2013-12-04
09:02:41,761::resourceManager::939::ResourceManager.Owner::(releaseAll)
Owner.releaseAll requests {} resources {}
Thread-20::DEBUG::2013-12-04
09:02:41,761::resourceManager::976::ResourceManager.Owner::(cancelAll)
Owner.cancelAll requests {}
Thread-20::DEBUG::2013-12-04
09:02:41,761::task::974::TaskManager.Task::(_decref)
Task=`e3529e49-3083-422e-b2e6-c7d8c47b886a`::ref 0 aborting False
Thread-22::DEBUG::2013-12-04
09:02:41,814::BindingXMLRPC::177::vds::(wrapper) client [192.168.128.79]
Thread-22::DEBUG::2013-12-04
09:02:41,815::task::579::TaskManager.Task::(_updateState)
Task=`337b1ad3-2559-450c-9133-4fb733e6b5da`::moving from state init ->
state preparing
Thread-22::INFO::2013-12-04
09:02:41,815::logUtils::44::dispatcher::(wrapper) Run and protect:
disconnectStoragePool(spUUID='5849b030-626e-47cb-ad90-3ce782d831b3',
hostID=5, s
csiKey='5849b030-626e-47cb-ad90-3ce782d831b3', remove=False, options=None)
Thread-22::WARNING::2013-12-04
09:02:41,815::hsm::1094::Storage.HSM::(disconnectStoragePool) disconnect
sp: 5849b030-626e-47cb-ad90-3ce782d831b3 failed. Known pools {}
Thread-22::INFO::2013-12-04
09:02:41,815::logUtils::47::dispatcher::(wrapper) Run and protect:
disconnectStoragePool, Return response: None
Thread-22::DEBUG::2013-12-04
09:02:41,815::task::1168::TaskManager.Task::(prepare)
Task=`337b1ad3-2559-450c-9133-4fb733e6b5da`::finished: None
Thread-22::DEBUG::2013-12-04
09:02:41,816::task::579::TaskManager.Task::(_updateState)
Task=`337b1ad3-2559-450c-9133-4fb733e6b5da`::moving from state preparing
-> state finish
ed
Thread-22::DEBUG::2013-12-04
09:02:41,816::resourceManager::939::ResourceManager.Owner::(releaseAll)
Owner.releaseAll requests {} resources {}
Thread-22::DEBUG::2013-12-04
09:02:41,816::resourceManager::976::ResourceManager.Owner::(cancelAll)
Owner.cancelAll requests {}
Thread-22::DEBUG::2013-12-04
09:02:41,816::task::974::TaskManager.Task::(_decref)
Task=`337b1ad3-2559-450c-9133-4fb733e6b5da`::ref 0 aborting False
Thread-23::DEBUG::2013-12-04
09:02:41,872::BindingXMLRPC::177::vds::(wrapper) client [192.168.128.79]
Thread-23::DEBUG::2013-12-04
09:02:41,872::task::579::TaskManager.Task::(_updateState)
Task=`155824b5-48c3-4eb2-90fb-d90e84f2900f`::moving from state init ->
state preparing
Thread-23::INFO::2013-12-04
09:02:41,873::logUtils::44::dispatcher::(wrapper) Run and protect:
disconnectStorageServer(domType=3,
spUUID='5849b030-626e-47cb-ad90-3ce782d831b3', conList=[{'connection':
'192.168.130.10', 'iqn': 'iqn.1992-08.com.netapp:sn.135056614',
'portal': '1000', 'user': '', 'password':
'******', 'id':
'478b35df-da9a-4cb0-847f-2f61f17a02f7', 'port': '3260'},
{'connection':
'192.168.131.10', 'iqn': 'iqn.1992-08.com.netapp:sn.135056614',
'portal': '1001', 'user': '', 'password':
'******', 'id':
'717e3505-3290-421b-bb00-25e046f63362', 'port': '3260'}],
options=None)
Thread-23::DEBUG::2013-12-04
09:02:41,876::misc::809::SamplingMethod::(__call__) Trying to enter
sampling method (storage.sdc.refreshStorage)
Thread-23::DEBUG::2013-12-04
09:02:41,876::misc::811::SamplingMethod::(__call__) Got in to sampling
method
Thread-23::DEBUG::2013-12-04
09:02:41,877::misc::809::SamplingMethod::(__call__) Trying to enter
sampling method (storage.iscsi.rescan)
Thread-23::DEBUG::2013-12-04
09:02:41,877::misc::811::SamplingMethod::(__call__) Got in to sampling
method
Thread-23::DEBUG::2013-12-04
09:02:41,877::iscsiadm::91::Storage.Misc.excCmd::(_runCmd)
'/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None)
Thread-23::DEBUG::2013-12-04
09:02:41,896::iscsiadm::91::Storage.Misc.excCmd::(_runCmd) FAILED: <err>
= 'iscsiadm: No session found.\n'; <rc> = 21
Thread-23::DEBUG::2013-12-04
09:02:41,896::misc::819::SamplingMethod::(__call__) Returning last result
Thread-23::DEBUG::2013-12-04
09:02:44,915::multipath::112::Storage.Misc.excCmd::(rescan)
'/usr/bin/sudo -n /sbin/multipath -r' (cwd None)
Thread-23::DEBUG::2013-12-04
09:02:44,966::multipath::112::Storage.Misc.excCmd::(rescan) SUCCESS:
<err> = ''; <rc> = 0
Thread-23::DEBUG::2013-12-04
09:02:44,967::lvm::510::OperationMutex::(_invalidateAllPvs) Operation
'lvm invalidate operation' got the operation mutex
Thread-23::DEBUG::2013-12-04
09:02:44,967::lvm::512::OperationMutex::(_invalidateAllPvs) Operation
'lvm invalidate operation' released the operation mutex
Thread-23::DEBUG::2013-12-04
09:02:44,968::lvm::521::OperationMutex::(_invalidateAllVgs) Operation
'lvm invalidate operation' got the operation mutex
Thread-23::DEBUG::2013-12-04
09:02:44,968::lvm::523::OperationMutex::(_invalidateAllVgs) Operation
'lvm invalidate operation' released the operation mutex
Thread-23::DEBUG::2013-12-04
09:02:44,968::lvm::541::OperationMutex::(_invalidateAllLvs) Operation
'lvm invalidate operation' got the operation mutex
Thread-23::DEBUG::2013-12-04
09:02:44,968::lvm::543::OperationMutex::(_invalidateAllLvs) Operation
'lvm invalidate operation' released the operation mutex
Thread-23::DEBUG::2013-12-04
09:02:44,968::misc::819::SamplingMethod::(__call__) Returning last result
Thread-23::INFO::2013-12-04
09:02:44,969::logUtils::47::dispatcher::(wrapper) Run and protect:
disconnectStorageServer, Return response: {'statuslist': [{'status': 0,
'id': '478b35df-da9a-4cb0-847f-2f61f17a02f7'}, {'status': 0,
'id':
'717e3505-3290-421b-bb00-25e046f63362'}]}
Thread-23::DEBUG::2013-12-04
09:02:44,969::task::1168::TaskManager.Task::(prepare)
Task=`155824b5-48c3-4eb2-90fb-d90e84f2900f`::finished: {'statuslist':
[{'status': 0, 'id': '478b35df-da9a-4cb0-847f-2f61f17a02f7'},
{'status':
0, 'id': '717e3505-3290-421b-bb00-25e046f63362'}]}
Thread-23::DEBUG::2013-12-04
09:02:44,969::task::579::TaskManager.Task::(_updateState)
Task=`155824b5-48c3-4eb2-90fb-d90e84f2900f`::moving from state preparing
-> state finished
Thread-23::DEBUG::2013-12-04
09:02:44,969::resourceManager::939::ResourceManager.Owner::(releaseAll)
Owner.releaseAll requests {} resources {}
Thread-23::DEBUG::2013-12-04
09:02:44,969::resourceManager::976::ResourceManager.Owner::(cancelAll)
Owner.cancelAll requests {}
Thread-23::DEBUG::2013-12-04
09:02:44,970::task::974::TaskManager.Task::(_decref)
Task=`155824b5-48c3-4eb2-90fb-d90e84f2900f`::ref 0 aborting False
Regards,
--------------060401020402070903050401
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
<html>
<head>
<meta http-equiv="content-type" content="text/html;
charset=ISO-8859-1">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi,<br>
<br>
I'm adding two twin nodes to a cluster and I was able to add the
first while the second fails as it removes the iscsi sessions and
don't bring them back and thus fails with the message :<br>
<br>
<div title="" tabindex="0" style="outline-style:
none;"
__gwt_cell="cell-gwt-uid-303550">
<div id="gwt-uid-3368_col2_row2">Host Dell4 cannot access one of
the Storage Domains attached to the Data Center Default. Setting
Host state to Non-Operational.<br>
<br>
<small><big>Here is the vdsm.log entries for one
activation<big>
</big></big></small>intent:</div>
</div>
<br>
<br>
Thread-20::DEBUG::2013-12-04
09:02:41,760::task::579::TaskManager.Task::(_updateState)
Task=`e3529e49-3083-422e-b2e6-c7d8c47b886a`::moving from state init
-> state preparing<br>
Thread-20::<a class="moz-txt-link-freetext"
href="INFO::2013-12-04">INFO::2013-12-04</a>
09:02:41,760::logUtils::44::dispatcher::(wrapper) Run and protect:
repoStats(options=None)<br>
Thread-20::<a class="moz-txt-link-freetext"
href="INFO::2013-12-04">INFO::2013-12-04</a>
09:02:41,760::logUtils::47::dispatcher::(wrapper) Run and protect:
repoStats, Return response: {}<br>
Thread-20::DEBUG::2013-12-04
09:02:41,760::task::1168::TaskManager.Task::(prepare)
Task=`e3529e49-3083-422e-b2e6-c7d8c47b886a`::finished: {}<br>
Thread-20::DEBUG::2013-12-04
09:02:41,761::task::579::TaskManager.Task::(_updateState)
Task=`e3529e49-3083-422e-b2e6-c7d8c47b886a`::moving from state
preparing -> state finish<br>
ed<br>
Thread-20::DEBUG::2013-12-04
09:02:41,761::resourceManager::939::ResourceManager.Owner::(releaseAll)
Owner.releaseAll requests {} resources {}<br>
Thread-20::DEBUG::2013-12-04
09:02:41,761::resourceManager::976::ResourceManager.Owner::(cancelAll)
Owner.cancelAll requests {}<br>
Thread-20::DEBUG::2013-12-04
09:02:41,761::task::974::TaskManager.Task::(_decref)
Task=`e3529e49-3083-422e-b2e6-c7d8c47b886a`::ref 0 aborting False<br>
Thread-22::DEBUG::2013-12-04
09:02:41,814::BindingXMLRPC::177::vds::(wrapper) client
[192.168.128.79]<br>
Thread-22::DEBUG::2013-12-04
09:02:41,815::task::579::TaskManager.Task::(_updateState)
Task=`337b1ad3-2559-450c-9133-4fb733e6b5da`::moving from state init
-> state preparing<br>
Thread-22::<a class="moz-txt-link-freetext"
href="INFO::2013-12-04">INFO::2013-12-04</a>
09:02:41,815::logUtils::44::dispatcher::(wrapper) Run and protect:
disconnectStoragePool(spUUID='5849b030-626e-47cb-ad90-3ce782d831b3',
hostID=5, s<br>
csiKey='5849b030-626e-47cb-ad90-3ce782d831b3', remove=False,
options=None)<br>
Thread-22::WARNING::2013-12-04
09:02:41,815::hsm::1094::Storage.HSM::(disconnectStoragePool)
disconnect sp: 5849b030-626e-47cb-ad90-3ce782d831b3 failed. Known
pools {}<br>
Thread-22::<a class="moz-txt-link-freetext"
href="INFO::2013-12-04">INFO::2013-12-04</a>
09:02:41,815::logUtils::47::dispatcher::(wrapper) Run and protect:
disconnectStoragePool, Return response: None<br>
Thread-22::DEBUG::2013-12-04
09:02:41,815::task::1168::TaskManager.Task::(prepare)
Task=`337b1ad3-2559-450c-9133-4fb733e6b5da`::finished: None<br>
Thread-22::DEBUG::2013-12-04
09:02:41,816::task::579::TaskManager.Task::(_updateState)
Task=`337b1ad3-2559-450c-9133-4fb733e6b5da`::moving from state
preparing -> state finish<br>
ed<br>
Thread-22::DEBUG::2013-12-04
09:02:41,816::resourceManager::939::ResourceManager.Owner::(releaseAll)
Owner.releaseAll requests {} resources {}<br>
Thread-22::DEBUG::2013-12-04
09:02:41,816::resourceManager::976::ResourceManager.Owner::(cancelAll)
Owner.cancelAll requests {}<br>
Thread-22::DEBUG::2013-12-04
09:02:41,816::task::974::TaskManager.Task::(_decref)
Task=`337b1ad3-2559-450c-9133-4fb733e6b5da`::ref 0 aborting False<br>
Thread-23::DEBUG::2013-12-04
09:02:41,872::BindingXMLRPC::177::vds::(wrapper) client
[192.168.128.79]<br>
Thread-23::DEBUG::2013-12-04
09:02:41,872::task::579::TaskManager.Task::(_updateState)
Task=`155824b5-48c3-4eb2-90fb-d90e84f2900f`::moving from state init
-> state preparing<br>
Thread-23::<a class="moz-txt-link-freetext"
href="INFO::2013-12-04">INFO::2013-12-04</a>
09:02:41,873::logUtils::44::dispatcher::(wrapper) Run and protect:
disconnectStorageServer(domType=3,
spUUID='5849b030-626e-47cb-ad90-3ce782d831b3',
conList=[{'connection': '192.168.130.10', 'iqn':
'iqn.1992-08.com.netapp:sn.135056614', 'portal': '1000',
'user': '',
'password': '******', 'id':
'478b35df-da9a-4cb0-847f-2f61f17a02f7',
'port': '3260'}, {'connection': '192.168.131.10',
'iqn':
'iqn.1992-08.com.netapp:sn.135056614', 'portal': '1001',
'user': '',
'password': '******', 'id':
'717e3505-3290-421b-bb00-25e046f63362',
'port': '3260'}], options=None)<br>
Thread-23::DEBUG::2013-12-04
09:02:41,876::misc::809::SamplingMethod::(__call__) Trying to enter
sampling method (storage.sdc.refreshStorage)<br>
Thread-23::DEBUG::2013-12-04
09:02:41,876::misc::811::SamplingMethod::(__call__) Got in to
sampling method<br>
Thread-23::DEBUG::2013-12-04
09:02:41,877::misc::809::SamplingMethod::(__call__) Trying to enter
sampling method (storage.iscsi.rescan)<br>
Thread-23::DEBUG::2013-12-04
09:02:41,877::misc::811::SamplingMethod::(__call__) Got in to
sampling method<br>
Thread-23::DEBUG::2013-12-04
09:02:41,877::iscsiadm::91::Storage.Misc.excCmd::(_runCmd)
'/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None)<br>
Thread-23::DEBUG::2013-12-04
09:02:41,896::iscsiadm::91::Storage.Misc.excCmd::(_runCmd) FAILED:
<err> = 'iscsiadm: No session found.\n'; <rc> =
21<br>
Thread-23::DEBUG::2013-12-04
09:02:41,896::misc::819::SamplingMethod::(__call__) Returning last
result<br>
Thread-23::DEBUG::2013-12-04
09:02:44,915::multipath::112::Storage.Misc.excCmd::(rescan)
'/usr/bin/sudo -n /sbin/multipath -r' (cwd None)<br>
Thread-23::DEBUG::2013-12-04
09:02:44,966::multipath::112::Storage.Misc.excCmd::(rescan) SUCCESS:
<err> = ''; <rc> = 0<br>
Thread-23::DEBUG::2013-12-04
09:02:44,967::lvm::510::OperationMutex::(_invalidateAllPvs)
Operation 'lvm invalidate operation' got the operation mutex<br>
Thread-23::DEBUG::2013-12-04
09:02:44,967::lvm::512::OperationMutex::(_invalidateAllPvs)
Operation 'lvm invalidate operation' released the operation mutex<br>
Thread-23::DEBUG::2013-12-04
09:02:44,968::lvm::521::OperationMutex::(_invalidateAllVgs)
Operation 'lvm invalidate operation' got the operation mutex<br>
Thread-23::DEBUG::2013-12-04
09:02:44,968::lvm::523::OperationMutex::(_invalidateAllVgs)
Operation 'lvm invalidate operation' released the operation mutex<br>
Thread-23::DEBUG::2013-12-04
09:02:44,968::lvm::541::OperationMutex::(_invalidateAllLvs)
Operation 'lvm invalidate operation' got the operation mutex<br>
Thread-23::DEBUG::2013-12-04
09:02:44,968::lvm::543::OperationMutex::(_invalidateAllLvs)
Operation 'lvm invalidate operation' released the operation mutex<br>
Thread-23::DEBUG::2013-12-04
09:02:44,968::misc::819::SamplingMethod::(__call__) Returning last
result<br>
Thread-23::<a class="moz-txt-link-freetext"
href="INFO::2013-12-04">INFO::2013-12-04</a>
09:02:44,969::logUtils::47::dispatcher::(wrapper) Run and protect:
disconnectStorageServer, Return response: {'statuslist': [{'status':
0, 'id': '478b35df-da9a-4cb0-847f-2f61f17a02f7'}, {'status':
0,
'id': '717e3505-3290-421b-bb00-25e046f63362'}]}<br>
Thread-23::DEBUG::2013-12-04
09:02:44,969::task::1168::TaskManager.Task::(prepare)
Task=`155824b5-48c3-4eb2-90fb-d90e84f2900f`::finished:
{'statuslist': [{'status': 0, 'id':
'478b35df-da9a-4cb0-847f-2f61f17a02f7'}, {'status': 0, 'id':
'717e3505-3290-421b-bb00-25e046f63362'}]}<br>
Thread-23::DEBUG::2013-12-04
09:02:44,969::task::579::TaskManager.Task::(_updateState)
Task=`155824b5-48c3-4eb2-90fb-d90e84f2900f`::moving from state
preparing -> state finished<br>
Thread-23::DEBUG::2013-12-04
09:02:44,969::resourceManager::939::ResourceManager.Owner::(releaseAll)
Owner.releaseAll requests {} resources {}<br>
Thread-23::DEBUG::2013-12-04
09:02:44,969::resourceManager::976::ResourceManager.Owner::(cancelAll)
Owner.cancelAll requests {}<br>
Thread-23::DEBUG::2013-12-04
09:02:44,970::task::974::TaskManager.Task::(_decref)
Task=`155824b5-48c3-4eb2-90fb-d90e84f2900f`::ref 0 aborting False<br>
<br>
Regards,<br>
<br>
<br>
</body>
</html>
--------------060401020402070903050401--
5:30 a.m.
the below is not the reason for the failure.
please attach full vdsm, engine, and host installation logs
Also, please run the below from the host you are trying to add (if vdsm
is installed there already):
vdsClient -s 0 getStorageDeviceList
Dafna
On 12/04/2013 11:22 AM, Juan Pablo Lorier wrote:
--
Dafna Ron
11:13 a.m.
Hi Defna,
Thanks for the reply. Here are the logs:
https://drive.google.com/file/d/0B9OrU8RK9m26SUZ1bXVFNUFwV1k/edit?usp=sha...
https://drive.google.com/file/d/0B9OrU8RK9m26eTZ3N0ZrUzZWZTQ/edit?usp=sha...
I couldn't get the output of vdsClient -s 0 getStorageDeviceList as
getStorageDeviceList is not a legal parameter for vdsclient
(vdsm-cli.4.13.0-11.el6)
I send them just to help see what may have happened as I rebooted the
server and I was able to activate it.
Regards,
On 04/12/13 09:30, Dafna Ron wrote:
4120
days inactive
4120
days old
3 comments
2 participants
participants (2)
-
Dafna Ron -
Juan Pablo Lorier