<div dir="ltr">I also have another setup and i used ovirt-engine-3.5.0.1-1.el6.noarch and vdsm-4.16.7-1.gitdb83943.el6.x86_64 but i still encounter the same error.<div><br></div><div>Here is the vdsm.log</div><div><br></div><div><div>MainThread::DEBUG::2014-10-23 18:20:33,717::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'Storage'</div><div>MainThread::DEBUG::2014-10-23 18:20:33,717::threadPool::35::Storage.ThreadPool::(__init__) Enter - numThreads: 10, waitTimeout: 3, maxTasks: 500</div><div>MainThread::DEBUG::2014-10-23 18:20:33,722::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt</div><div>MainThread::DEBUG::2014-10-23 18:20:33,844::supervdsm::77::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm</div><div>MainThread::DEBUG::2014-10-23 18:20:33,870::multipath::151::Storage.Multipath::(isEnabled) multipath Defaulting to False</div><div>MainThread::DEBUG::2014-10-23 18:20:33,871::misc::456::Storage.Misc::(rotateFiles) dir: /etc, prefixName: multipath.conf, versions: 5</div><div>MainThread::DEBUG::2014-10-23 18:20:33,871::misc::477::Storage.Misc::(rotateFiles) versions found: []</div><div>MainThread::DEBUG::2014-10-23 18:20:33,872::multipath::169::Storage.Misc.excCmd::(setupMultipath) /usr/bin/sudo -n /bin/cp /tmp/tmp2_UKnY /etc/multipath.conf (cwd None)</div><div>MainThread::DEBUG::2014-10-23 18:20:33,888::multipath::169::Storage.Misc.excCmd::(setupMultipath) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-23 18:20:33,890::multipath::175::Storage.Misc.excCmd::(setupMultipath) /usr/bin/sudo -n /sbin/multipath -F (cwd None)</div><div>MainThread::DEBUG::2014-10-23 18:20:33,907::multipath::175::Storage.Misc.excCmd::(setupMultipath) FAILED: <err> = ''; <rc> = 1</div><div>MainThread::DEBUG::2014-10-23 18:20:33,907::multipath::178::Storage.Misc.excCmd::(setupMultipath) /usr/bin/sudo -n /usr/bin/vdsm-tool service-reload multipathd (cwd None)</div><div>MainThread::DEBUG::2014-10-23 18:20:54,163::multipath::178::Storage.Misc.excCmd::(setupMultipath) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-23 18:20:54,163::hsm::418::Storage.Misc.excCmd::(__validateLvmLockingType) /usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type (cwd None)</div><div>MainThread::DEBUG::2014-10-23 18:20:54,331::hsm::418::Storage.Misc.excCmd::(__validateLvmLockingType) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-23 18:20:54,332::hsm::442::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center'</div><div>MainThread::DEBUG::2014-10-23 18:20:54,333::hsm::474::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt']</div><div>MainThread::DEBUG::2014-10-23 18:20:54,334::hsm::475::Storage.HSM::(__cleanStorageRepository) Mount list: []</div><div>MainThread::DEBUG::2014-10-23 18:20:54,334::hsm::477::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers</div><div>MainThread::DEBUG::2014-10-23 18:20:54,338::hsm::520::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center'</div><div>storageRefresh::DEBUG::2014-10-23 18:20:54,339::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage)</div><div>MainThread::INFO::2014-10-23 18:20:54,341::dispatcher::46::Storage.Dispatcher::(__init__) Starting StorageDispatcher...</div><div>MainThread::INFO::2014-10-24 14:24:27,283::vdsm::131::vds::(run) (PID: 3372) I am the actual vdsm 4.16.7-1.gitdb83943.el6 <a href="http://ovirt-node-mk2-05.i.gov.ph">ovirt-node-mk2-05.i.gov.ph</a> (2.6.32-431.29.2.el6.x86_64)</div><div>MainThread::DEBUG::2014-10-24 14:24:27,284::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'Storage'</div><div>MainThread::DEBUG::2014-10-24 14:24:27,285::threadPool::35::Storage.ThreadPool::(__init__) Enter - numThreads: 10, waitTimeout: 3, maxTasks: 500</div><div>MainThread::DEBUG::2014-10-24 14:24:27,288::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt</div><div>MainThread::WARNING::2014-10-24 14:24:27,289::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt already exists</div><div>MainThread::DEBUG::2014-10-24 14:24:27,394::supervdsm::77::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm</div><div>MainThread::DEBUG::2014-10-24 14:24:27,418::multipath::142::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving</div><div>MainThread::DEBUG::2014-10-24 14:24:27,418::hsm::418::Storage.Misc.excCmd::(__validateLvmLockingType) /usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type (cwd None)</div><div>MainThread::DEBUG::2014-10-24 14:24:27,604::hsm::418::Storage.Misc.excCmd::(__validateLvmLockingType) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-24 14:24:27,604::hsm::442::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center'</div><div>MainThread::DEBUG::2014-10-24 14:24:27,607::hsm::474::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt']</div><div>MainThread::DEBUG::2014-10-24 14:24:27,607::hsm::475::Storage.HSM::(__cleanStorageRepository) Mount list: []</div><div>MainThread::DEBUG::2014-10-24 14:24:27,607::hsm::477::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers</div><div>MainThread::DEBUG::2014-10-24 14:24:27,611::hsm::520::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center'</div><div>MainThread::INFO::2014-10-24 14:24:27,614::dispatcher::46::Storage.Dispatcher::(__init__) Starting StorageDispatcher...</div><div>storageRefresh::DEBUG::2014-10-24 14:24:27,614::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage)</div><div>storageRefresh::DEBUG::2014-10-24 14:24:47,671::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method</div><div>storageRefresh::DEBUG::2014-10-24 14:24:47,673::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan)</div><div>storageRefresh::DEBUG::2014-10-24 14:24:47,679::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method</div><div>storageRefresh::DEBUG::2014-10-24 14:24:47,680::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds</div><div>storageRefresh::DEBUG::2014-10-24 14:24:47,684::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None)</div><div>MainThread::DEBUG::2014-10-24 14:24:47,761::task::595::Storage.TaskManager.Task::(_updateState) Task=`377cdbc9-9686-4f97-a846-2a2a0f87053f`::moving from state init -> state preparing</div><div>MainThread::INFO::2014-10-24 14:24:47,761::logUtils::44::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback(callbackFunc=<bound method clientIF.contEIOVms of <clientIF.clientIF object at 0x2d4a690>>)</div><div>MainThread::INFO::2014-10-24 14:24:47,761::logUtils::47::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback, Return response: None</div><div>MainThread::DEBUG::2014-10-24 14:24:47,761::task::1191::Storage.TaskManager.Task::(prepare) Task=`377cdbc9-9686-4f97-a846-2a2a0f87053f`::finished: None</div><div>MainThread::DEBUG::2014-10-24 14:24:47,761::task::595::Storage.TaskManager.Task::(_updateState) Task=`377cdbc9-9686-4f97-a846-2a2a0f87053f`::moving from state preparing -> state finished</div><div>MainThread::DEBUG::2014-10-24 14:24:47,762::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}</div><div>MainThread::DEBUG::2014-10-24 14:24:47,762::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}</div><div>MainThread::DEBUG::2014-10-24 14:24:47,762::task::993::Storage.TaskManager.Task::(_decref) Task=`377cdbc9-9686-4f97-a846-2a2a0f87053f`::ref 0 aborting False</div><div>MainThread::INFO::2014-10-24 14:24:47,763::momIF::49::MOM::(__init__) Starting up MOM</div><div>MainThread::INFO::2014-10-24 14:24:47,767::vmchannels::192::vds::(settimeout) Setting channels' timeout to 30 seconds.</div><div>VM Channels Listener::INFO::2014-10-24 14:24:47,769::vmchannels::175::vds::(run) Starting VM channels listener thread.</div><div>clientIFinit::DEBUG::2014-10-24 14:24:47,772::libvirtconnection::150::root::(get) trying to connect libvirt</div><div>MainThread::DEBUG::2014-10-24 14:24:47,783::protocoldetector::131::vds.MultiProtocolAcceptor::(add_detector) adding detector: <rpc.BindingXMLRPC.XmlDetector instance at 0x2e439e0></div><div>MainThread::DEBUG::2014-10-24 14:24:47,876::protocoldetector::131::vds.MultiProtocolAcceptor::(add_detector) adding detector: <yajsonrpc.stompReactor.StompDetector instance at 0x2fca050></div><div>JsonRpcServer::DEBUG::2014-10-24 14:24:47,880::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request</div><div>Detector thread::DEBUG::2014-10-24 14:24:47,883::protocoldetector::70::vds.MultiProtocolAcceptor::(serve_forever) Acceptor running</div><div>Detector thread::DEBUG::2014-10-24 14:24:47,884::protocoldetector::72::vds.MultiProtocolAcceptor::(serve_forever) Using required_size=7</div><div>storageRefresh::DEBUG::2014-10-24 14:24:49,733::misc::751::Storage.SamplingMethod::(__call__) Returning last result</div><div>storageRefresh::DEBUG::2014-10-24 14:24:49,770::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None)</div></div><div><br></div><div><br></div><div><br></div><div>and Here is the supervdsm.log</div><div><br></div><div><div>MainThread::DEBUG::2014-10-23 18:17:50,485::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-23 18:17:50,486::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-23 18:17:50,516::supervdsmServer::411::SuperVdsm.Server::(main) Making sure I'm root - SuperVdsm</div><div>MainThread::DEBUG::2014-10-23 18:17:50,517::supervdsmServer::420::SuperVdsm.Server::(main) Parsing cmd args</div><div>MainThread::DEBUG::2014-10-23 18:17:50,517::supervdsmServer::423::SuperVdsm.Server::(main) Cleaning old socket /var/run/vdsm/svdsm.sock</div><div>MainThread::DEBUG::2014-10-23 18:17:50,517::supervdsmServer::427::SuperVdsm.Server::(main) Setting up keep alive thread</div><div>MainThread::DEBUG::2014-10-23 18:17:50,517::supervdsmServer::433::SuperVdsm.Server::(main) Creating remote object manager</div><div>MainThread::DEBUG::2014-10-23 18:17:50,519::fileUtils::192::Storage.fileUtils::(chown) Changing owner for /var/run/vdsm/svdsm.sock, to (36:36)</div><div>MainThread::DEBUG::2014-10-23 18:17:50,519::supervdsmServer::444::SuperVdsm.Server::(main) Started serving super vdsm object</div><div>sourceRoute::DEBUG::2014-10-23 18:17:50,519::sourceroutethread::75::root::(_subscribeToInotifyLoop) sourceRouteThread.subscribeToInotifyLoop started</div><div>MainThread::DEBUG::2014-10-23 18:20:12,945::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-23 18:20:12,945::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-23 18:20:12,945::vdsm-restore-net-config::55::root::(unified_restoration) Removing all networks ({}) and bonds ({}) in running config.</div><div>MainThread::DEBUG::2014-10-23 18:20:12,945::libvirtconnection::150::root::(get) trying to connect libvirt</div><div>MainThread::DEBUG::2014-10-23 18:20:12,959::utils::738::root::(execCmd) /sbin/ip route show to <a href="http://0.0.0.0/0">0.0.0.0/0</a> table all (cwd None)</div><div>MainThread::DEBUG::2014-10-23 18:20:12,961::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-23 18:20:12,962::utils::738::root::(execCmd) /sbin/ip route show to <a href="http://0.0.0.0/0">0.0.0.0/0</a> table all (cwd None)</div><div>MainThread::DEBUG::2014-10-23 18:20:12,964::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-23 18:20:12,974::utils::738::root::(execCmd) /usr/bin/tee /sys/class/net/bonding_masters (cwd None)</div><div>MainThread::DEBUG::2014-10-23 18:20:12,976::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-23 18:20:12,976::utils::738::root::(execCmd) /usr/bin/tee /sys/class/net/L5P6SkRXZIa3zqz/bonding/mode (cwd None)</div><div>MainThread::DEBUG::2014-10-23 18:20:12,979::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-23 18:20:12,980::utils::738::root::(execCmd) /usr/bin/tee /sys/class/net/L5P6SkRXZIa3zqz/bonding/mode (cwd None)</div><div>MainThread::DEBUG::2014-10-23 18:20:12,982::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-23 18:20:12,983::utils::738::root::(execCmd) /usr/bin/tee /sys/class/net/L5P6SkRXZIa3zqz/bonding/mode (cwd None)</div><div>MainThread::DEBUG::2014-10-23 18:20:12,985::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-23 18:20:12,986::utils::738::root::(execCmd) /usr/bin/tee /sys/class/net/L5P6SkRXZIa3zqz/bonding/mode (cwd None)</div><div>MainThread::DEBUG::2014-10-23 18:20:12,989::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-23 18:20:12,990::utils::738::root::(execCmd) /usr/bin/tee /sys/class/net/L5P6SkRXZIa3zqz/bonding/mode (cwd None)</div><div>MainThread::DEBUG::2014-10-23 18:20:12,992::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-23 18:20:12,993::utils::738::root::(execCmd) /usr/bin/tee /sys/class/net/L5P6SkRXZIa3zqz/bonding/mode (cwd None)</div><div>MainThread::DEBUG::2014-10-23 18:20:12,995::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-23 18:20:12,996::utils::738::root::(execCmd) /usr/bin/tee /sys/class/net/L5P6SkRXZIa3zqz/bonding/mode (cwd None)</div><div>MainThread::DEBUG::2014-10-23 18:20:12,997::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-23 18:20:12,999::utils::738::root::(execCmd) /usr/bin/tee /sys/class/net/bonding_masters (cwd None)</div><div>MainThread::DEBUG::2014-10-23 18:20:13,035::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-23 18:20:13,036::api::621::setupNetworks::(setupNetworks) Setting up network according to configuration: networks:{}, bondings:{}, options:{'_inRollback': True, 'connectivityCheck': False}</div><div>MainThread::DEBUG::2014-10-23 18:20:13,036::api::625::root::(setupNetworks) Validating configuration</div><div>MainThread::DEBUG::2014-10-23 18:20:13,040::api::637::setupNetworks::(setupNetworks) Applying...</div><div>MainThread::DEBUG::2014-10-23 18:20:13,040::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-23 18:20:13,040::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-23 18:20:13,041::utils::738::root::(execCmd) /sbin/ip route show to <a href="http://0.0.0.0/0">0.0.0.0/0</a> table all (cwd None)</div><div>MainThread::DEBUG::2014-10-23 18:20:13,043::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-23 18:20:13,053::utils::738::root::(execCmd) /sbin/ip route show to <a href="http://0.0.0.0/0">0.0.0.0/0</a> table all (cwd None)</div><div>MainThread::DEBUG::2014-10-23 18:20:13,055::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::INFO::2014-10-23 18:20:13,063::netconfpersistence::158::root::(_clearDisk) Clearing /var/run/vdsm/netconf/nets/ and /var/run/vdsm/netconf/bonds/</div><div>MainThread::DEBUG::2014-10-23 18:20:13,063::netconfpersistence::166::root::(_clearDisk) No existent config to clear.</div><div>MainThread::INFO::2014-10-23 18:20:13,063::netconfpersistence::182::root::(save) Saved new config RunningConfig({}, {}) to /var/run/vdsm/netconf/nets/ and /var/run/vdsm/netconf/bonds/</div><div>MainThread::DEBUG::2014-10-23 18:20:13,065::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-23 18:20:13,065::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-23 18:20:13,116::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-23 18:20:13,116::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-23 18:20:13,117::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-23 18:20:13,117::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-23 18:20:13,120::vdsm-restore-net-config::68::root::(unified_restoration) Calling setupNetworks with networks ({}) and bond ({}).</div><div>MainThread::DEBUG::2014-10-23 18:20:13,121::utils::738::root::(execCmd) /sbin/ip route show to <a href="http://0.0.0.0/0">0.0.0.0/0</a> table all (cwd None)</div><div>MainThread::DEBUG::2014-10-23 18:20:13,123::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-23 18:20:13,124::utils::738::root::(execCmd) /sbin/ip route show to <a href="http://0.0.0.0/0">0.0.0.0/0</a> table all (cwd None)</div><div>MainThread::DEBUG::2014-10-23 18:20:13,126::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-23 18:20:13,135::api::621::setupNetworks::(setupNetworks) Setting up network according to configuration: networks:{}, bondings:{}, options:{'_inRollback': True, 'connectivityCheck': False}</div><div>MainThread::DEBUG::2014-10-23 18:20:13,135::api::625::root::(setupNetworks) Validating configuration</div><div>MainThread::DEBUG::2014-10-23 18:20:13,139::api::637::setupNetworks::(setupNetworks) Applying...</div><div>MainThread::DEBUG::2014-10-23 18:20:13,139::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-23 18:20:13,139::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-23 18:20:13,139::utils::738::root::(execCmd) /sbin/ip route show to <a href="http://0.0.0.0/0">0.0.0.0/0</a> table all (cwd None)</div><div>MainThread::DEBUG::2014-10-23 18:20:13,141::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-23 18:20:13,150::utils::738::root::(execCmd) /sbin/ip route show to <a href="http://0.0.0.0/0">0.0.0.0/0</a> table all (cwd None)</div><div>MainThread::DEBUG::2014-10-23 18:20:13,153::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::INFO::2014-10-23 18:20:13,161::netconfpersistence::158::root::(_clearDisk) Clearing /var/run/vdsm/netconf/nets/ and /var/run/vdsm/netconf/bonds/</div><div>MainThread::DEBUG::2014-10-23 18:20:13,161::netconfpersistence::166::root::(_clearDisk) No existent config to clear.</div><div>MainThread::INFO::2014-10-23 18:20:13,161::netconfpersistence::182::root::(save) Saved new config RunningConfig({}, {}) to /var/run/vdsm/netconf/nets/ and /var/run/vdsm/netconf/bonds/</div><div>MainProcess::DEBUG::2014-10-23 18:20:33,869::supervdsmServer::101::SuperVdsm.ServerCallback::(wrapper) call readMultipathConf with () {}</div><div>MainProcess::DEBUG::2014-10-23 18:20:33,870::supervdsmServer::108::SuperVdsm.ServerCallback::(wrapper) return readMultipathConf with ['# multipath.conf written by anaconda', '', 'defaults {', '\tuser_friendly_names yes', '}', 'blacklist {', '\tdevnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"', '\tdevnode "^hd[a-z]"', '\tdevnode "^dcssblk[0-9]*"', '\tdevice {', '\t\tvendor "DGC"', '\t\tproduct "LUNZ"', '\t}', '\tdevice {', '\t\tvendor "IBM"', '\t\tproduct "S/390.*"', '\t}', "\t# don't count normal SATA devices as multipaths", '\tdevice {', '\t\tvendor "ATA"', '\t}', "\t# don't count 3ware devices as multipaths", '\tdevice {', '\t\tvendor "3ware"', '\t}', '\tdevice {', '\t\tvendor "AMCC"', '\t}', '\t# nor highpoint devices', '\tdevice {', '\t\tvendor "HPT"', '\t}', '\twwid "F0102BJF"', '\tdevice {', '\t\tvendor IBM_SATA', '\t\tproduct DEVICE_81Y3658', '\t}', '\twwid "*"', '}', 'blacklist_exceptions {', '\twwid "3600507680281077d680000000000001c"', '}', 'multipaths {', '\tmultipath {', '\t\tuid 0', '\t\tgid 0', '\t\twwid "3600507680281077d680000000000001c"', '\t\tmode 0600', '\t}', '}']</div><div>MainThread::DEBUG::2014-10-24 14:21:44,088::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-24 14:21:44,088::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-24 14:21:44,128::supervdsmServer::411::SuperVdsm.Server::(main) Making sure I'm root - SuperVdsm</div><div>MainThread::DEBUG::2014-10-24 14:21:44,129::supervdsmServer::420::SuperVdsm.Server::(main) Parsing cmd args</div><div>MainThread::DEBUG::2014-10-24 14:21:44,129::supervdsmServer::423::SuperVdsm.Server::(main) Cleaning old socket /var/run/vdsm/svdsm.sock</div><div>MainThread::DEBUG::2014-10-24 14:21:44,129::supervdsmServer::427::SuperVdsm.Server::(main) Setting up keep alive thread</div><div>MainThread::DEBUG::2014-10-24 14:21:44,129::supervdsmServer::433::SuperVdsm.Server::(main) Creating remote object manager</div><div>MainThread::DEBUG::2014-10-24 14:21:44,131::fileUtils::192::Storage.fileUtils::(chown) Changing owner for /var/run/vdsm/svdsm.sock, to (36:36)</div><div>MainThread::DEBUG::2014-10-24 14:21:44,131::supervdsmServer::444::SuperVdsm.Server::(main) Started serving super vdsm object</div><div>sourceRoute::DEBUG::2014-10-24 14:21:44,131::sourceroutethread::75::root::(_subscribeToInotifyLoop) sourceRouteThread.subscribeToInotifyLoop started</div><div>MainThread::DEBUG::2014-10-24 14:24:06,571::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-24 14:24:06,571::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-24 14:24:06,571::vdsm-restore-net-config::55::root::(unified_restoration) Removing all networks ({}) and bonds ({}) in running config.</div><div>MainThread::DEBUG::2014-10-24 14:24:06,571::libvirtconnection::150::root::(get) trying to connect libvirt</div><div>MainThread::DEBUG::2014-10-24 14:24:06,591::utils::738::root::(execCmd) /sbin/ip route show to <a href="http://0.0.0.0/0">0.0.0.0/0</a> table all (cwd None)</div><div>MainThread::DEBUG::2014-10-24 14:24:06,594::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-24 14:24:06,595::utils::738::root::(execCmd) /sbin/ip route show to <a href="http://0.0.0.0/0">0.0.0.0/0</a> table all (cwd None)</div><div>MainThread::DEBUG::2014-10-24 14:24:06,597::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-24 14:24:06,607::utils::738::root::(execCmd) /usr/bin/tee /sys/class/net/bonding_masters (cwd None)</div><div>MainThread::DEBUG::2014-10-24 14:24:06,610::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-24 14:24:06,610::utils::738::root::(execCmd) /usr/bin/tee /sys/class/net/WUR1ZiCBIS9eo8U/bonding/mode (cwd None)</div><div>MainThread::DEBUG::2014-10-24 14:24:06,612::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-24 14:24:06,613::utils::738::root::(execCmd) /usr/bin/tee /sys/class/net/WUR1ZiCBIS9eo8U/bonding/mode (cwd None)</div><div>MainThread::DEBUG::2014-10-24 14:24:06,615::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-24 14:24:06,616::utils::738::root::(execCmd) /usr/bin/tee /sys/class/net/WUR1ZiCBIS9eo8U/bonding/mode (cwd None)</div><div>MainThread::DEBUG::2014-10-24 14:24:06,618::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-24 14:24:06,619::utils::738::root::(execCmd) /usr/bin/tee /sys/class/net/WUR1ZiCBIS9eo8U/bonding/mode (cwd None)</div><div>MainThread::DEBUG::2014-10-24 14:24:06,621::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-24 14:24:06,622::utils::738::root::(execCmd) /usr/bin/tee /sys/class/net/WUR1ZiCBIS9eo8U/bonding/mode (cwd None)</div><div>MainThread::DEBUG::2014-10-24 14:24:06,624::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-24 14:24:06,625::utils::738::root::(execCmd) /usr/bin/tee /sys/class/net/WUR1ZiCBIS9eo8U/bonding/mode (cwd None)</div><div>MainThread::DEBUG::2014-10-24 14:24:06,626::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-24 14:24:06,628::utils::738::root::(execCmd) /usr/bin/tee /sys/class/net/WUR1ZiCBIS9eo8U/bonding/mode (cwd None)</div><div>MainThread::DEBUG::2014-10-24 14:24:06,630::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-24 14:24:06,631::utils::738::root::(execCmd) /usr/bin/tee /sys/class/net/bonding_masters (cwd None)</div><div>MainThread::DEBUG::2014-10-24 14:24:06,671::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-24 14:24:06,672::api::621::setupNetworks::(setupNetworks) Setting up network according to configuration: networks:{}, bondings:{}, options:{'_inRollback': True, 'connectivityCheck': False}</div><div>MainThread::DEBUG::2014-10-24 14:24:06,672::api::625::root::(setupNetworks) Validating configuration</div><div>MainThread::DEBUG::2014-10-24 14:24:06,676::api::637::setupNetworks::(setupNetworks) Applying...</div><div>MainThread::DEBUG::2014-10-24 14:24:06,676::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-24 14:24:06,676::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-24 14:24:06,676::utils::738::root::(execCmd) /sbin/ip route show to <a href="http://0.0.0.0/0">0.0.0.0/0</a> table all (cwd None)</div><div>MainThread::DEBUG::2014-10-24 14:24:06,678::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-24 14:24:06,688::utils::738::root::(execCmd) /sbin/ip route show to <a href="http://0.0.0.0/0">0.0.0.0/0</a> table all (cwd None)</div><div>MainThread::DEBUG::2014-10-24 14:24:06,690::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::INFO::2014-10-24 14:24:06,699::netconfpersistence::158::root::(_clearDisk) Clearing /var/run/vdsm/netconf/nets/ and /var/run/vdsm/netconf/bonds/</div><div>MainThread::DEBUG::2014-10-24 14:24:06,699::netconfpersistence::166::root::(_clearDisk) No existent config to clear.</div><div>MainThread::INFO::2014-10-24 14:24:06,699::netconfpersistence::182::root::(save) Saved new config RunningConfig({}, {}) to /var/run/vdsm/netconf/nets/ and /var/run/vdsm/netconf/bonds/</div><div>MainThread::DEBUG::2014-10-24 14:24:06,700::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-24 14:24:06,700::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-24 14:24:06,705::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-24 14:24:06,705::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-24 14:24:06,706::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-24 14:24:06,706::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-24 14:24:06,709::vdsm-restore-net-config::68::root::(unified_restoration) Calling setupNetworks with networks ({}) and bond ({}).</div><div>MainThread::DEBUG::2014-10-24 14:24:06,710::utils::738::root::(execCmd) /sbin/ip route show to <a href="http://0.0.0.0/0">0.0.0.0/0</a> table all (cwd None)</div><div>MainThread::DEBUG::2014-10-24 14:24:06,712::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-24 14:24:06,713::utils::738::root::(execCmd) /sbin/ip route show to <a href="http://0.0.0.0/0">0.0.0.0/0</a> table all (cwd None)</div><div>MainThread::DEBUG::2014-10-24 14:24:06,715::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-24 14:24:06,724::api::621::setupNetworks::(setupNetworks) Setting up network according to configuration: networks:{}, bondings:{}, options:{'_inRollback': True, 'connectivityCheck': False}</div><div>MainThread::DEBUG::2014-10-24 14:24:06,724::api::625::root::(setupNetworks) Validating configuration</div><div>MainThread::DEBUG::2014-10-24 14:24:06,727::api::637::setupNetworks::(setupNetworks) Applying...</div><div>MainThread::DEBUG::2014-10-24 14:24:06,728::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-24 14:24:06,728::netconfpersistence::134::root::(_getConfigs) Non-existing config set.</div><div>MainThread::DEBUG::2014-10-24 14:24:06,728::utils::738::root::(execCmd) /sbin/ip route show to <a href="http://0.0.0.0/0">0.0.0.0/0</a> table all (cwd None)</div><div>MainThread::DEBUG::2014-10-24 14:24:06,730::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::DEBUG::2014-10-24 14:24:06,740::utils::738::root::(execCmd) /sbin/ip route show to <a href="http://0.0.0.0/0">0.0.0.0/0</a> table all (cwd None)</div><div>MainThread::DEBUG::2014-10-24 14:24:06,742::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0</div><div>MainThread::INFO::2014-10-24 14:24:06,750::netconfpersistence::158::root::(_clearDisk) Clearing /var/run/vdsm/netconf/nets/ and /var/run/vdsm/netconf/bonds/</div><div>MainThread::DEBUG::2014-10-24 14:24:06,750::netconfpersistence::166::root::(_clearDisk) No existent config to clear.</div><div>MainThread::INFO::2014-10-24 14:24:06,750::netconfpersistence::182::root::(save) Saved new config RunningConfig({}, {}) to /var/run/vdsm/netconf/nets/ and /var/run/vdsm/netconf/bonds/</div><div>MainProcess::DEBUG::2014-10-24 14:24:27,417::supervdsmServer::101::SuperVdsm.ServerCallback::(wrapper) call readMultipathConf with () {}</div><div>MainProcess::DEBUG::2014-10-24 14:24:27,417::supervdsmServer::108::SuperVdsm.ServerCallback::(wrapper) return readMultipathConf with ['# RHEV REVISION 1.0', '', 'defaults {', ' polling_interval 5', ' getuid_callout "/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/%n"', ' no_path_retry fail', ' user_friendly_names no', ' flush_on_last_del yes', ' fast_io_fail_tmo 5', ' dev_loss_tmo 30', ' max_fds 4096', '}', '', 'devices {', 'device {', ' vendor "HITACHI"', ' product "DF.*"', ' getuid_callout "/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/%n"', '}', 'device {', ' vendor "COMPELNT"', ' product "Compellent Vol"', ' no_path_retry fail', '}', '}']</div><div>MainProcess|storageRefresh::DEBUG::2014-10-24 14:24:49,745::supervdsmServer::101::SuperVdsm.ServerCallback::(wrapper) call hbaRescan with () {}</div><div>MainProcess|storageRefresh::INFO::2014-10-24 14:24:49,746::hba::54::Storage.HBA::(rescan) Rescanning HBAs</div><div>MainProcess|storageRefresh::DEBUG::2014-10-24 14:24:49,747::hba::56::Storage.HBA::(rescan) Issuing lip /sys/class/fc_host/host1/issue_lip</div><div>MainProcess|storageRefresh::DEBUG::2014-10-24 14:24:49,758::hba::56::Storage.HBA::(rescan) Issuing lip /sys/class/fc_host/host2/issue_lip</div><div>MainProcess|storageRefresh::DEBUG::2014-10-24 14:24:49,769::supervdsmServer::108::SuperVdsm.ServerCallback::(wrapper) return hbaRescan with None</div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Oct 24, 2014 at 6:47 PM, Dan Kenigsberg <span dir="ltr"><<a href="mailto:danken@redhat.com" target="_blank">danken@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On Fri, Oct 24, 2014 at 03:01:40PM +0800, John Michael Mercado wrote:<br>
> Hi All,<br>
><br>
</span>> I need your help. Everytime the vdsm start, the server become *read-only<br>
> file system*. My OS installed in SAN storage.<br>
<br>
Could you share more details about your platfor? Is it el6?<br>
Which vdsm version is running there? Did it work better with a previous<br>
version?<br>
<br>
Do you have anything in /var/log/vdsm/vdsm.log? and supervdsm.log?<br>
<br>
Vdsm may be restoring the host network configuration to some outdated<br>
values that break the connection to you SAN storage.<br>
<span class="HOEnZb"><font color="#888888"><br>
Dan.<br>
</font></span></blockquote></div><br></div>