I was able to get iSCSI over TCP working...but now the task of adding the LUN to the GUI has been stuck at the "spinning" icon for about 20 minutes.

I see these entries in vdsm.log over and over with the Task value changing:

Thread-14::DEBUG::2014-10-21 14:16:50,086::task::595::TaskManager.Task::(_updateState) Task=`ebcd8e0a-54b1-43d2-92a2-ed9fd62d00fa`::moving from state init -> state preparing
Thread-14::INFO::2014-10-21 14:16:50,086::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
Thread-14::INFO::2014-10-21 14:16:50,086::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {}
Thread-14::DEBUG::2014-10-21 14:16:50,087::task::1185::TaskManager.Task::(prepare) Task=`ebcd8e0a-54b1-43d2-92a2-ed9fd62d00fa`::finished: {}
Thread-14::DEBUG::2014-10-21 14:16:50,087::task::595::TaskManager.Task::(_updateState) Task=`ebcd8e0a-54b1-43d2-92a2-ed9fd62d00fa`::moving from state preparing -> state finished
Thread-14::DEBUG::2014-10-21 14:16:50,087::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
Thread-14::DEBUG::2014-10-21 14:16:50,087::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
Thread-14::DEBUG::2014-10-21 14:16:50,087::task::990::TaskManager.Task::(_decref) Task=`ebcd8e0a-54b1-43d2-92a2-ed9fd62d00fa`::ref 0 aborting False

What is there I can do to get my storage back online?  Right now my iSCSI is master (something I did not want) which is odd considering the NFS data domain was added as master when I setup oVirt.  Nothing will come back until I get the master domain online and unsure what to do now.

Thanks,
- Trey

On Tue, Oct 21, 2014 at 12:58 PM, Trey Dockendorf <treydock@gmail.com> wrote:
I had a catastrophic failure of the IB switch that was used by all my storage domains.  I had one data domain that was NFS and one that was iSCSI. I managed to get the iSCSI LUN detached using the docs [1] but now I noticed that somehow my master domain went from the NFS domain to the iSCSI domain and I'm unable to switch them back.

How does one change the master?  Right now I am having issues getting iSCSI over TCP to work, so am sort of stuck with 30 VMs down and an entire cluster inaccessible.

Thanks,
- Trey