Re: [Users] How to change storage domain ip address

i have 2 hosts , one is up and the other is in Non Operational status node1 vdsm.log: Thread-77492::DEBUG::2012-05-29 17:58:39,927::clientIF::261::Storage.Dispatcher.Protect::(wrapper) [172.30.0.229] Thread-77492::DEBUG::2012-05-29 17:58:39,929::task::588::TaskManager.Task::(_updateState) Task=`9e61d75e-5673-4c01-a8ad-99fc737398de`::moving from state init -> state preparing Thread-77492::INFO::2012-05-29 17:58:39,930::logUtils::37::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='524a7003-edec-4f52-a38e-b15cadfbe3ef', options=None) Thread-77492::ERROR::2012-05-29 17:58:39,930::task::855::TaskManager.Task::(_setError) Task=`9e61d75e-5673-4c01-a8ad-99fc737398de`::Unexpected error Traceback (most recent call last): File "/usr/share/vdsm/storage/task.py", line 863, in _run File "/usr/share/vdsm/logUtils.py", line 38, in wrapper File "/usr/share/vdsm/storage/hsm.py", line 438, in getSpmStatus File "/usr/share/vdsm/storage/hsm.py", line 186, in getPool StoragePoolUnknown: Unknown pool id, pool not connected: ('524a7003-edec-4f52-a38e-b15cadfbe3ef',) Thread-77492::DEBUG::2012-05-29 17:58:39,931::task::874::TaskManager.Task::(_run) Task=`9e61d75e-5673-4c01-a8ad-99fc737398de`::Task._run: 9e61d75e-5673-4c01-a8ad-99fc737398de ('524a7003-edec-4f52-a38e-b15cadfbe3ef',) {} failed - stopping task Thread-77492::DEBUG::2012-05-29 17:58:39,932::task::1201::TaskManager.Task::(stop) Task=`9e61d75e-5673-4c01-a8ad-99fc737398de`::stopping in state preparing (force False) Thread-77492::DEBUG::2012-05-29 17:58:39,933::task::980::TaskManager.Task::(_decref) Task=`9e61d75e-5673-4c01-a8ad-99fc737398de`::ref 1 aborting True Thread-77492::INFO::2012-05-29 17:58:39,933::task::1159::TaskManager.Task::(prepare) Task=`9e61d75e-5673-4c01-a8ad-99fc737398de`::aborting: Task is aborted: 'Unknown pool id, pool not connected' - code 309 Thread-77492::DEBUG::2012-05-29 17:58:39,934::task::1164::TaskManager.Task::(prepare) Task=`9e61d75e-5673-4c01-a8ad-99fc737398de`::Prepare: aborted: Unknown pool id, pool not connected Thread-77492::DEBUG::2012-05-29 17:58:39,935::task::980::TaskManager.Task::(_decref) Task=`9e61d75e-5673-4c01-a8ad-99fc737398de`::ref 0 aborting True Thread-77492::DEBUG::2012-05-29 17:58:39,935::task::915::TaskManager.Task::(_doAbort) Task=`9e61d75e-5673-4c01-a8ad-99fc737398de`::Task._doAbort: force False Thread-77492::DEBUG::2012-05-29 17:58:39,936::resourceManager::841::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-77492::DEBUG::2012-05-29 17:58:39,937::task::588::TaskManager.Task::(_updateState) Task=`9e61d75e-5673-4c01-a8ad-99fc737398de`::moving from state preparing -> state aborting Thread-77492::DEBUG::2012-05-29 17:58:39,937::task::537::TaskManager.Task::(__state_aborting) Task=`9e61d75e-5673-4c01-a8ad-99fc737398de`::_aborting: recover policy none Thread-77492::DEBUG::2012-05-29 17:58:39,938::task::588::TaskManager.Task::(_updateState) Task=`9e61d75e-5673-4c01-a8ad-99fc737398de`::moving from state aborting -> state failed Thread-77492::DEBUG::2012-05-29 17:58:39,939::resourceManager::806::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} Thread-77492::DEBUG::2012-05-29 17:58:39,939::resourceManager::841::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-77492::ERROR::2012-05-29 17:58:39,940::dispatcher::90::Storage.Dispatcher.Protect::(run) {'status': {'message': "Unknown pool id, pool not connected: ('524a7003-edec-4f52-a38e-b15cadfbe3ef',)", 'code': 309}} Thread-77493::DEBUG::2012-05-29 17:58:39,964::clientIF::261::Storage.Dispatcher.Protect::(wrapper) [172.30.0.229] Thread-77493::DEBUG::2012-05-29 17:58:39,965::task::588::TaskManager.Task::(_updateState) Task=`47053840-dcdd-4c64-8788-e730ca0e87cb`::moving from state init -> state preparing Thread-77493::INFO::2012-05-29 17:58:39,966::logUtils::37::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID='524a7003-edec-4f52-a38e-b15cadfbe3ef', hostID=1, scsiKey='524a7003-edec-4f52-a38e-b15cadfbe3ef', msdUUID='5e2ac537-6a73-4faf-8379-68f3ff26a75d', masterVersion=1, options=None) Thread-77493::DEBUG::2012-05-29 17:58:39,967::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.524a7003-edec-4f52-a38e-b15cadfbe3ef`ReqID=`64a4665d-f6e7-4f70-980d-035020fed461`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '747' at '_connectStoragePool' Thread-77493::DEBUG::2012-05-29 17:58:39,968::resourceManager::483::ResourceManager::(registerResource) Trying to register resource 'Storage.524a7003-edec-4f52-a38e-b15cadfbe3ef' for lock type 'exclusive' Thread-77493::DEBUG::2012-05-29 17:58:39,969::resourceManager::525::ResourceManager::(registerResource) Resource 'Storage.524a7003-edec-4f52-a38e-b15cadfbe3ef' is free. Now locking as 'exclusive' (1 active user) Thread-77493::DEBUG::2012-05-29 17:58:39,970::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.524a7003-edec-4f52-a38e-b15cadfbe3ef`ReqID=`64a4665d-f6e7-4f70-980d-035020fed461`::Granted request Thread-77493::INFO::2012-05-29 17:58:39,971::sp::608::Storage.StoragePool::(connect) Connect host #1 to the storage pool 524a7003-edec-4f52-a38e-b15cadfbe3ef with master domain: 5e2ac537-6a73-4faf-8379-68f3ff26a75d (ver = 1) Thread-77493::DEBUG::2012-05-29 17:58:39,972::lvm::460::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex Thread-77493::DEBUG::2012-05-29 17:58:39,973::lvm::462::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex Thread-77493::DEBUG::2012-05-29 17:58:39,974::lvm::472::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex Thread-77493::DEBUG::2012-05-29 17:58:39,975::lvm::474::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex Thread-77493::DEBUG::2012-05-29 17:58:39,976::lvm::493::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex Thread-77493::DEBUG::2012-05-29 17:58:39,977::lvm::495::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex Thread-77493::DEBUG::2012-05-29 17:58:39,978::resourceManager::535::ResourceManager::(releaseResource) Trying to release resource 'Storage.524a7003-edec-4f52-a38e-b15cadfbe3ef' Thread-77493::DEBUG::2012-05-29 17:58:39,978::resourceManager::550::ResourceManager::(releaseResource) Released resource 'Storage.524a7003-edec-4f52-a38e-b15cadfbe3ef' (0 active users) Thread-77493::DEBUG::2012-05-29 17:58:39,979::resourceManager::555::ResourceManager::(releaseResource) Resource 'Storage.524a7003-edec-4f52-a38e-b15cadfbe3ef' is free, finding out if anyone is waiting for it. Thread-77493::DEBUG::2012-05-29 17:58:39,980::resourceManager::562::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.524a7003-edec-4f52-a38e-b15cadfbe3ef', Clearing records. Thread-77493::ERROR::2012-05-29 17:58:39,981::task::855::TaskManager.Task::(_setError) Task=`47053840-dcdd-4c64-8788-e730ca0e87cb`::Unexpected error Traceback (most recent call last): File "/usr/share/vdsm/storage/task.py", line 863, in _run File "/usr/share/vdsm/logUtils.py", line 38, in wrapper File "/usr/share/vdsm/storage/hsm.py", line 721, in connectStoragePool File "/usr/share/vdsm/storage/hsm.py", line 763, in _connectStoragePool File "/usr/share/vdsm/storage/sp.py", line 624, in connect File "/usr/share/vdsm/storage/sp.py", line 1097, in __rebuild File "/usr/share/vdsm/storage/sp.py", line 1437, in getMasterDomain File "/usr/share/vdsm/storage/sd.py", line 656, in isMaster File "/usr/share/vdsm/storage/sd.py", line 616, in getMetaParam File "/usr/share/vdsm/storage/persistentDict.py", line 75, in __getitem__ File "/usr/share/vdsm/storage/persistentDict.py", line 185, in __getitem__ KeyError: 'ROLE' Thread-77493::DEBUG::2012-05-29 17:58:39,981::task::874::TaskManager.Task::(_run) Task=`47053840-dcdd-4c64-8788-e730ca0e87cb`::Task._run: 47053840-dcdd-4c64-8788-e730ca0e87cb ('524a7003-edec-4f52-a38e-b15cadfbe3ef', 1, '524a7003-edec-4f52-a38e-b15cadfbe3ef', '5e2ac537-6a73-4faf-8379-68f3ff26a75d', 1) {} failed - stopping task Thread-77493::DEBUG::2012-05-29 17:58:39,981::task::1201::TaskManager.Task::(stop) Task=`47053840-dcdd-4c64-8788-e730ca0e87cb`::stopping in state preparing (force False) Thread-77493::DEBUG::2012-05-29 17:58:39,982::task::980::TaskManager.Task::(_decref) Task=`47053840-dcdd-4c64-8788-e730ca0e87cb`::ref 1 aborting True Thread-77493::INFO::2012-05-29 17:58:39,982::task::1159::TaskManager.Task::(prepare) Task=`47053840-dcdd-4c64-8788-e730ca0e87cb`::aborting: Task is aborted: "'ROLE'" - code 100 Thread-77493::DEBUG::2012-05-29 17:58:39,982::task::1164::TaskManager.Task::(prepare) Task=`47053840-dcdd-4c64-8788-e730ca0e87cb`::Prepare: aborted: 'ROLE' Thread-77493::DEBUG::2012-05-29 17:58:39,982::task::980::TaskManager.Task::(_decref) Task=`47053840-dcdd-4c64-8788-e730ca0e87cb`::ref 0 aborting True Thread-77493::DEBUG::2012-05-29 17:58:39,982::task::915::TaskManager.Task::(_doAbort) Task=`47053840-dcdd-4c64-8788-e730ca0e87cb`::Task._doAbort: force False Thread-77493::DEBUG::2012-05-29 17:58:39,983::resourceManager::841::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-77493::DEBUG::2012-05-29 17:58:39,983::task::588::TaskManager.Task::(_updateState) Task=`47053840-dcdd-4c64-8788-e730ca0e87cb`::moving from state preparing -> state aborting Thread-77493::DEBUG::2012-05-29 17:58:39,983::task::537::TaskManager.Task::(__state_aborting) Task=`47053840-dcdd-4c64-8788-e730ca0e87cb`::_aborting: recover policy none Thread-77493::DEBUG::2012-05-29 17:58:39,983::task::588::TaskManager.Task::(_updateState) Task=`47053840-dcdd-4c64-8788-e730ca0e87cb`::moving from state aborting -> state failed Thread-77493::DEBUG::2012-05-29 17:58:39,983::resourceManager::806::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} Thread-77493::DEBUG::2012-05-29 17:58:39,984::resourceManager::841::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-77493::ERROR::2012-05-29 17:58:39,984::dispatcher::93::Storage.Dispatcher.Protect::(run) 'ROLE' Traceback (most recent call last): File "/usr/share/vdsm/storage/dispatcher.py", line 85, in run File "/usr/share/vdsm/storage/task.py", line 1166, in prepare KeyError: 'ROLE' node2 vdsm.log Thread-163903::DEBUG::2012-05-29 17:58:51,732::clientIF::76::vds::(wrapper) [172.30.0.229]::call getVdsCapabilities with () {} Thread-163903::DEBUG::2012-05-29 17:58:51,774::utils::595::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-kvm' (cwd None) Thread-163903::DEBUG::2012-05-29 17:58:51,816::utils::595::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0 Thread-163903::DEBUG::2012-05-29 17:58:51,818::utils::595::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-img' (cwd None) Thread-163903::DEBUG::2012-05-29 17:58:51,863::utils::595::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0 Thread-163903::DEBUG::2012-05-29 17:58:51,865::utils::595::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" vdsm' (cwd None) Thread-163903::DEBUG::2012-05-29 17:58:51,912::utils::595::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0 Thread-163903::DEBUG::2012-05-29 17:58:51,913::utils::595::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" spice-server' (cwd None) Thread-163903::DEBUG::2012-05-29 17:58:51,948::utils::595::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0 Thread-163903::DEBUG::2012-05-29 17:58:51,950::utils::595::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" libvirt' (cwd None) Thread-163903::DEBUG::2012-05-29 17:58:51,989::utils::595::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0 Thread-163903::DEBUG::2012-05-29 17:58:51,992::clientIF::81::vds::(wrapper) return getVdsCapabilities with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:80e221f0efc2'}], 'FC': []}, 'packages2': {'kernel': {'release': '2.fc16.x86_64', 'buildtime': 1328299688.0, 'version': '3.2.3'}, 'spice-server': {'release': '1.fc16', 'buildtime': '1327339129', 'version': '0.10.1'}, 'vdsm': {'release': '0.fc16', 'buildtime': '1327521056', 'version': '4.9.3.2'}, 'qemu-kvm': {'release': '3.fc16', 'buildtime': '1321651456', 'version': '0.15.1'}, 'libvirt': {'release': '4.fc16', 'buildtime': '1324326688', 'version': '0.9.6'}, 'qemu-img': {'release': '3.fc16', 'buildtime': '1321651456', 'version': '0.15.1'}}, 'cpuModel': 'Six-Core AMD Opteron(tm) Processor 2435', 'hooks': {}, 'vmTypes': ['kvm'], 'supportedProtocols': ['2.2', '2.3'], 'networks': {'ovirtmgmt': {'addr': 'xxx.xxx.xxx.xxx', 'cfg': {'IPV6FORWARDING': 'no', 'IPV6INIT': 'no', 'SKIPLIBVIRT': 'True', 'IPADDR': 'xxx.xxx.xxx.xxx', 'PEERDNS': 'no', 'GATEWAY': 'xxx.xxx.xxx.xxx', 'DELAY': '0', 'IPV6_AUTOCONF': 'no', 'NETMASK': '255.255.254.0', 'BOOTPROTO': 'static', 'DEVICE': 'ovirtmgmt', 'PEERNTP': 'yes', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'ports': ['eth0'], 'netmask': 'xxx.xxx.xxx.xxx'em2': {'addr': 'xxx.xx.xxx.xxx', 'cfg': {'IPADDR': 'xxx.xxx.xxx.xxx', 'DELAY': '0', 'NETMASK': '255.255.255.0', 'STP': 'no', 'DEVICE': 'em2', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'ports': ['eth1'], 'netmask': '255.255.255.0', 'stp': 'off', 'gateway': '0.0.0.0'}}, 'uuid': '44454C4C-5900-105A-804B-B7C04F563258_00:1D:09:FD:8B:80', 'lastClientIface': 'ovirtmgmt', 'nics': {'eth1': {'hwaddr': '00:1D:09:FD:8B:82', 'netmask': '', 'speed': 1000, 'addr': ''}, 'eth0': {'hwaddr': '00:1D:09:FD:8B:80', 'netmask': '', 'speed': 1000, 'addr': ''}}, 'software_revision': '0', 'management_ip': '', 'clusterLevels': ['3.0'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,mmx,fxsr,sse,sse2,ht,syscall,nx,mmxext,fxsr_opt,pdpe1gb,rdtscp,lm,3dnowext,3dnow,constant_tsc,rep_good,nopl,nonstop_tsc,extd_apicid,pni,monitor,cx16,popcnt,lahf_lm,cmp_legacy,svm,extapic,cr8_legacy,abm,sse4a,misalignsse,3dnowprefetch,osvw,ibs,skinit,wdt,npt,lbrv,svm_lock,nrip_save,pausefilter,model_486,model_pentium,model_pentium2,model_pentium3,model_pentiumpro,model_qemu32,model_coreduo,model_qemu64,model_phenom,model_athlon,model_Opteron_G1,model_Opteron_G2,model_Opteron_G3', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:80e221f0efc2', 'memSize': '32109', 'reservedMem': '321', 'bondings': {'bond4': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}, 'bond0': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}, 'bond1': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}, 'bond2': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}, 'bond3': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}}, 'software_version': '4.9', 'cpuSpeed': '800.000', 'cpuSockets': '2', 'vlans': {}, 'cpuCores': '12', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0'], 'version_name': 'Snow Man', 'emulatedMachines': [u'pc-0.14', u'pc', u'fedora-13', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc', u'pc-0.14', u'pc', u'fedora-13', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc'], 'operatingSystem': {'release': '1', 'version': '16', 'name': 'oVirt Node'}, 'lastClient': '172.30.0.229'}} Thread-163904::DEBUG::2012-05-29 17:58:52,549::clientIF::76::vds::(wrapper) [172.30.0.229]::call getVdsCapabilities with () {} Thread-163904::DEBUG::2012-05-29 17:58:52,586::utils::595::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-kvm' (cwd None) Thread-163904::DEBUG::2012-05-29 17:58:52,633::utils::595::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0 Thread-163904::DEBUG::2012-05-29 17:58:52,635::utils::595::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" qemu-img' (cwd None) Thread-163904::DEBUG::2012-05-29 17:58:52,682::utils::595::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0 Thread-163904::DEBUG::2012-05-29 17:58:52,684::utils::595::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" vdsm' (cwd None) Thread-163904::DEBUG::2012-05-29 17:58:52,725::utils::595::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0 Thread-163904::DEBUG::2012-05-29 17:58:52,726::utils::595::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" spice-server' (cwd None) Thread-163904::DEBUG::2012-05-29 17:58:52,773::utils::595::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0 Thread-163904::DEBUG::2012-05-29 17:58:52,774::utils::595::Storage.Misc.excCmd::(execCmd) '/bin/rpm -q --qf "%{NAME}\t%{VERSION}\t%{RELEASE}\t%{BUILDTIME}\n" libvirt' (cwd None) Thread-163904::DEBUG::2012-05-29 17:58:52,823::utils::595::Storage.Misc.excCmd::(execCmd) SUCCESS: <err> = ''; <rc> = 0 Thread-163904::DEBUG::2012-05-29 17:58:52,826::clientIF::81::vds::(wrapper) return getVdsCapabilities with {'status': {'message': 'Done', 'code': 0}, 'info': {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:80e221f0efc2'}], 'FC': []}, 'packages2': {'kernel': {'release': '2.fc16.x86_64', 'buildtime': 1328299688.0, 'version': '3.2.3'}, 'spice-server': {'release': '1.fc16', 'buildtime': '1327339129', 'version': '0.10.1'}, 'vdsm': {'release': '0.fc16', 'buildtime': '1327521056', 'version': '4.9.3.2'}, 'qemu-kvm': {'release': '3.fc16', 'buildtime': '1321651456', 'version': '0.15.1'}, 'libvirt': {'release': '4.fc16', 'buildtime': '1324326688', 'version': '0.9.6'}, 'qemu-img': {'release': '3.fc16', 'buildtime': '1321651456', 'version': '0.15.1'}}, 'cpuModel': 'Six-Core AMD Opteron(tm) Processor 2435', 'hooks': {}, 'vmTypes': ['kvm'], 'supportedProtocols': ['2.2', '2.3'], 'networks': {'ovirtmgmt': {'addr': 'xxx.xxx.xxx.xxx', 'cfg': {'IPV6FORWARDING': 'no', 'IPV6INIT': 'no', 'SKIPLIBVIRT': 'True', 'IPADDR': 'xxx.xxx.xxx.xxx', 'PEERDNS': 'no', 'GATEWAY': 'xxx.xxx.xxx.xxx', 'DELAY': '0', 'IPV6_AUTOCONF': 'no', 'NETMASK': '255.255.254.0', 'BOOTPROTO': 'static', 'DEVICE': 'ovirtmgmt', 'PEERNTP': 'yes', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'ports': ['eth0'], 'netmask': '255.255.254.0', 'stp': 'off', 'gateway': 'xxx.xxx.xxx.xxx'}, 'em2': {'addr': 'xxx.xxx.xxx.xxx', 'cfg': {'IPADDR': 'xxx.xxx.xxx.xxx', 'DELAY': '0', 'NETMASK': '255.255.255.0', 'STP': 'no', 'DEVICE': 'em2', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'ports': ['eth1'], 'netmask': '255.255.255.0', 'stp': 'off', 'gateway': '0.0.0.0'}}, 'uuid': '44454C4C-5900-105A-804B-B7C04F563258_00:1D:09:FD:8B:80', 'lastClientIface': 'ovirtmgmt', 'nics': {'eth1': {'hwaddr': '00:1D:09:FD:8B:82', 'netmask': '', 'speed': 1000, 'addr': ''}, 'eth0': {'hwaddr': '00:1D:09:FD:8B:80', 'netmask': '', 'speed': 1000, 'addr': ''}}, 'software_revision': '0', 'management_ip': '', 'clusterLevels': ['3.0'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,mmx,fxsr,sse,sse2,ht,syscall,nx,mmxext,fxsr_opt,pdpe1gb,rdtscp,lm,3dnowext,3dnow,constant_tsc,rep_good,nopl,nonstop_tsc,extd_apicid,pni,monitor,cx16,popcnt,lahf_lm,cmp_legacy,svm,extapic,cr8_legacy,abm,sse4a,misalignsse,3dnowprefetch,osvw,ibs,skinit,wdt,npt,lbrv,svm_lock,nrip_save,pausefilter,model_486,model_pentium,model_pentium2,model_pentium3,model_pentiumpro,model_qemu32,model_coreduo,model_qemu64,model_phenom,model_athlon,model_Opteron_G1,model_Opteron_G2,model_Opteron_G3', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:80e221f0efc2', 'memSize': '32109', 'reservedMem': '321', 'bondings': {'bond4': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}, 'bond0': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}, 'bond1': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}, 'bond2': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}, 'bond3': {'hwaddr': '00:00:00:00:00:00', 'cfg': {}, 'netmask': '', 'addr': '', 'slaves': []}}, 'software_version': '4.9', 'cpuSpeed': '800.000', 'cpuSockets': '2', 'vlans': {}, 'cpuCores': '12', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0'], 'version_name': 'Snow Man', 'emulatedMachines': [u'pc-0.14', u'pc', u'fedora-13', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc', u'pc-0.14', u'pc', u'fedora-13', u'pc-0.13', u'pc-0.12', u'pc-0.11', u'pc-0.10', u'isapc'], 'operatingSystem': {'release': '1', 'version': '16', 'name': 'oVirt Node'}, 'lastClient': '172.30.0.229'}} Thread-163906::DEBUG::2012-05-29 17:58:53,014::clientIF::261::Storage.Dispatcher.Protect::(wrapper) [172.30.0.229] Thread-163906::DEBUG::2012-05-29 17:58:53,015::task::588::TaskManager.Task::(_updateState) Task=`844eca1f-ec6f-4e3b-ad97-e31939cb96d3`::moving from state init -> state preparing Thread-163906::INFO::2012-05-29 17:58:53,016::logUtils::37::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID='524a7003-edec-4f52-a38e-b15cadfbe3ef', hostID=2, scsiKey='524a7003-edec-4f52-a38e-b15cadfbe3ef', msdUUID='5e2ac537-6a73-4faf-8379-68f3ff26a75d', masterVersion=1, options=None) Thread-163906::DEBUG::2012-05-29 17:58:53,018::resourceManager::175::ResourceManager.Request::(__init__) ResName=`Storage.524a7003-edec-4f52-a38e-b15cadfbe3ef`ReqID=`466be789-4790-457e-a752-751b4c75f9b4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '747' at '_connectStoragePool' Thread-163906::DEBUG::2012-05-29 17:58:53,018::resourceManager::483::ResourceManager::(registerResource) Trying to register resource 'Storage.524a7003-edec-4f52-a38e-b15cadfbe3ef' for lock type 'exclusive' Thread-163906::DEBUG::2012-05-29 17:58:53,019::resourceManager::525::ResourceManager::(registerResource) Resource 'Storage.524a7003-edec-4f52-a38e-b15cadfbe3ef' is free. Now locking as 'exclusive' (1 active user) Thread-163906::DEBUG::2012-05-29 17:58:53,020::resourceManager::212::ResourceManager.Request::(grant) ResName=`Storage.524a7003-edec-4f52-a38e-b15cadfbe3ef`ReqID=`466be789-4790-457e-a752-751b4c75f9b4`::Granted request Thread-163906::INFO::2012-05-29 17:58:53,021::sp::608::Storage.StoragePool::(connect) Connect host #2 to the storage pool 524a7003-edec-4f52-a38e-b15cadfbe3ef with master domain: 5e2ac537-6a73-4faf-8379-68f3ff26a75d (ver = 1) Thread-163906::DEBUG::2012-05-29 17:58:53,021::lvm::460::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex Thread-163906::DEBUG::2012-05-29 17:58:53,021::lvm::462::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex Thread-163906::DEBUG::2012-05-29 17:58:53,022::lvm::472::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex Thread-163906::DEBUG::2012-05-29 17:58:53,022::lvm::474::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex Thread-163906::DEBUG::2012-05-29 17:58:53,022::lvm::493::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex Thread-163906::DEBUG::2012-05-29 17:58:53,022::lvm::495::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex Thread-163906::DEBUG::2012-05-29 17:58:53,023::resourceManager::535::ResourceManager::(releaseResource) Trying to release resource 'Storage.524a7003-edec-4f52-a38e-b15cadfbe3ef' Thread-163906::DEBUG::2012-05-29 17:58:53,023::resourceManager::550::ResourceManager::(releaseResource) Released resource 'Storage.524a7003-edec-4f52-a38e-b15cadfbe3ef' (0 active users) Thread-163906::DEBUG::2012-05-29 17:58:53,023::resourceManager::555::ResourceManager::(releaseResource) Resource 'Storage.524a7003-edec-4f52-a38e-b15cadfbe3ef' is free, finding out if anyone is waiting for it. Thread-163906::DEBUG::2012-05-29 17:58:53,023::resourceManager::562::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.524a7003-edec-4f52-a38e-b15cadfbe3ef', Clearing records. Thread-163906::ERROR::2012-05-29 17:58:53,024::task::855::TaskManager.Task::(_setError) Task=`844eca1f-ec6f-4e3b-ad97-e31939cb96d3`::Unexpected error Traceback (most recent call last): File "/usr/share/vdsm/storage/task.py", line 863, in _run File "/usr/share/vdsm/logUtils.py", line 38, in wrapper File "/usr/share/vdsm/storage/hsm.py", line 721, in connectStoragePool File "/usr/share/vdsm/storage/hsm.py", line 763, in _connectStoragePool File "/usr/share/vdsm/storage/sp.py", line 624, in connect File "/usr/share/vdsm/storage/sp.py", line 1097, in __rebuild File "/usr/share/vdsm/storage/sp.py", line 1437, in getMasterDomain File "/usr/share/vdsm/storage/sd.py", line 656, in isMaster File "/usr/share/vdsm/storage/sd.py", line 616, in getMetaParam File "/usr/share/vdsm/storage/persistentDict.py", line 75, in __getitem__ File "/usr/share/vdsm/storage/persistentDict.py", line 185, in __getitem__ KeyError: 'ROLE' Thread-163906::DEBUG::2012-05-29 17:58:53,024::task::874::TaskManager.Task::(_run) Task=`844eca1f-ec6f-4e3b-ad97-e31939cb96d3`::Task._run: 844eca1f-ec6f-4e3b-ad97-e31939cb96d3 ('524a7003-edec-4f52-a38e-b15cadfbe3ef', 2, '524a7003-edec-4f52-a38e-b15cadfbe3ef', '5e2ac537-6a73-4faf-8379-68f3ff26a75d', 1) {} failed - stopping task Thread-163906::DEBUG::2012-05-29 17:58:53,024::task::1201::TaskManager.Task::(stop) Task=`844eca1f-ec6f-4e3b-ad97-e31939cb96d3`::stopping in state preparing (force False) Thread-163906::DEBUG::2012-05-29 17:58:53,024::task::980::TaskManager.Task::(_decref) Task=`844eca1f-ec6f-4e3b-ad97-e31939cb96d3`::ref 1 aborting True Thread-163906::INFO::2012-05-29 17:58:53,025::task::1159::TaskManager.Task::(prepare) Task=`844eca1f-ec6f-4e3b-ad97-e31939cb96d3`::aborting: Task is aborted: "'ROLE'" - code 100 Thread-163906::DEBUG::2012-05-29 17:58:53,025::task::1164::TaskManager.Task::(prepare) Task=`844eca1f-ec6f-4e3b-ad97-e31939cb96d3`::Prepare: aborted: 'ROLE' Thread-163906::DEBUG::2012-05-29 17:58:53,025::task::980::TaskManager.Task::(_decref) Task=`844eca1f-ec6f-4e3b-ad97-e31939cb96d3`::ref 0 aborting True Thread-163906::DEBUG::2012-05-29 17:58:53,025::task::915::TaskManager.Task::(_doAbort) Task=`844eca1f-ec6f-4e3b-ad97-e31939cb96d3`::Task._doAbort: force False Thread-163906::DEBUG::2012-05-29 17:58:53,025::resourceManager::841::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-163906::DEBUG::2012-05-29 17:58:53,026::task::588::TaskManager.Task::(_updateState) Task=`844eca1f-ec6f-4e3b-ad97-e31939cb96d3`::moving from state preparing -> state aborting Thread-163906::DEBUG::2012-05-29 17:58:53,026::task::537::TaskManager.Task::(__state_aborting) Task=`844eca1f-ec6f-4e3b-ad97-e31939cb96d3`::_aborting: recover policy none Thread-163906::DEBUG::2012-05-29 17:58:53,026::task::588::TaskManager.Task::(_updateState) Task=`844eca1f-ec6f-4e3b-ad97-e31939cb96d3`::moving from state aborting -> state failed Thread-163906::DEBUG::2012-05-29 17:58:53,026::resourceManager::806::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} Thread-163906::DEBUG::2012-05-29 17:58:53,026::resourceManager::841::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-163906::ERROR::2012-05-29 17:58:53,026::dispatcher::93::Storage.Dispatcher.Protect::(run) 'ROLE' Traceback (most recent call last): File "/usr/share/vdsm/storage/dispatcher.py", line 85, in run File "/usr/share/vdsm/storage/task.py", line 1166, in prepare KeyError: 'ROLE' On 30 May, 2012, at 1:34 AM, Haim Ateya wrote: