- Users - oVirt List Archives

macvlan + IPv6
by Jay Turner 29 Feb '16

29 Feb '16

Hoping someone can help with a problem my team is seeing under oVirt. We are making heavy use of macvlan interfaces (in VEPA mode) on-top of virtual functions, under VMs being managed by oVirt. In this scenario IPv6 is not playing nicely, with no traffic going through, and messages about neighbor solicitation. There are some pointers out there indicating the issue stems from the fact IPv6 utilizing multicast for neighbor solicitation, but nothing we have tried seems to work around this issue. The problem is made all the most bizarre by the fact that on the same hardware libvirt + virt-manager works perfectly fine. I have looked at the corresponding xml for the guests, and nothing seems to point to the underlying cause for oVirt to fail, but libvirt to succeed. * Intel XL710 40G NICs (i40e/i40evf drivers) * CentOS 7.1 (fully up-to-date) * oVirt 3.6 * libvirt-1.2.17 * virt-manager-1.2.1-8 Thanks for any pointers folks can provide. - jkt

3 10

Snapshot Locked for an extended period of time
by Simon Hallam 28 Feb '16

28 Feb '16

Hi All, I’ve recently written a script which: 1. Takes a snapshot with memory. 2. Creates a VM from that snapshot. 3. Exports the snapshot to the export domain. 4. Deletes the snapshot. 5. Deletes the cloned VM. Whilst running the script for the first time last night, the script timed out whilst deleting the snapshot on the VM at step 4. Now the logs are filled with: 2016-02-25 10:32:53,272 INFO [org.ovirt.engine.core.bll.RemoveSnapshotCommandCallback] (DefaultQuartzScheduler_Worker-25) [4559dd29] Waiting on Live Merge child commands to complete 2016-02-25 10:32:53,332 INFO [org.ovirt.engine.core.bll.RemoveSnapshotSingleDiskLiveCommand] (DefaultQuartzScheduler_Worker-25) [225b841f] Waiting on Live Merge command step MERGE to complete 2016-02-25 10:32:53,335 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (DefaultQuartzScheduler_Worker-25) [4559dd29] Waiting on merge command to complete 2016-02-25 10:32:55,331 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-19) [4559dd29] VM job c1064551-022f-4549-a9d4-bb915e177e80: In progress (no change) 2016-02-25 10:32:58,405 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-37) [4559dd29] VM job c1064551-022f-4549-a9d4-bb915e177e80: In progress, updating 2016-02-25 10:33:01,521 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-56) [4559dd29] VM job c1064551-022f-4549-a9d4-bb915e177e80: In progress (no change) 2016-02-25 10:33:03,337 INFO [org.ovirt.engine.core.bll.RemoveSnapshotCommandCallback] (DefaultQuartzScheduler_Worker-12) [4559dd29] Waiting on Live Merge child commands to complete 2016-02-25 10:33:03,350 INFO [org.ovirt.engine.core.bll.RemoveSnapshotSingleDiskLiveCommand] (DefaultQuartzScheduler_Worker-12) [225b841f] Waiting on Live Merge command step MERGE to complete 2016-02-25 10:33:03,352 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (DefaultQuartzScheduler_Worker-12) [4559dd29] Waiting on merge command to complete 2016-02-25 10:33:04,678 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-22) [4559dd29] VM job c1064551-022f-4549-a9d4-bb915e177e80: In progress (no change) 2016-02-25 10:33:07,852 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-11) [4559dd29] VM job c1064551-022f-4549-a9d4-bb915e177e80: In progress (no change) 2016-02-25 10:33:10,993 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-44) [4559dd29] VM job c1064551-022f-4549-a9d4-bb915e177e80: In progress, updating 2016-02-25 10:33:13,354 INFO [org.ovirt.engine.core.bll.RemoveSnapshotCommandCallback] (DefaultQuartzScheduler_Worker-14) [4559dd29] Waiting on Live Merge child commands to complete I have attached the log of when the snapshot started deleting (2016-02- 24 23:51:17,094). Any idea what's gone wrong? What are my next steps to clear this? Cheers, Simon Please visit our new website at www.pml.ac.uk and follow us on Twitter @PlymouthMarine Winner of the Environment & Conservation category, the Charity Awards 2014. Plymouth Marine Laboratory (PML) is a company limited by guarantee registered in England & Wales, company number 4178503. Registered Charity No. 1091222. Registered Office: Prospect Place, The Hoe, Plymouth PL1 3DH, UK. This message is private and confidential. If you have received this message in error, please notify the sender and remove it from your system. You are reminded that e-mail communications are not secure and may contain viruses; PML accepts no liability for any loss or damage which may be caused by viruses.

3 4

Cannot delete snapshot or export VM after adding disk
by gregor 28 Feb '16

28 Feb '16

Hi, after the following workflow I can't remove or create a snapshot neither export the VM. - Create a VM - Add a disk with 1GB - Make a snapshot - Add another disk with 1GB - Export the VM Now you I get the following error in the events list: -------------------- VDSM host01 command failed: Error during destination image manipulation -------------------- Did I understand something wrong or is there a bug? Besides: The web-interface is extremely slow and copy&paste the error messages are painful. regards gregor

2 2

[hosted-engine] admin@internal password for additional nodes?
by Wee Sritippho 28 Feb '16

28 Feb '16

Hi, I'm trying to deploy a 2nd host to my hosted-engine environment, but at some point, the setup ask me to type a password for admin@internal again. Do I need to type the same password that I choose when deploying the 1st host? If not, would it replace the old password? Thank you, Wee --- ซอฟต์แวร์ Avast แอนตี้ไวรัสตรวจสอบหาไวรัสจากอีเมลนี้แล้ว https://www.avast.com/antivirus

2 1

DevConf.CZ 2016 - my experience
by Yaniv Kaul 28 Feb '16

28 Feb '16

On the first weekend of February I've had the pleaseure of attending DevConf.CZ 2016, which took place in the wonderful city of Brno, CZ. Compared to FOSDEM, it's a more relaxed, young and vibrant conference, but it was just as fun and rewarding from my perspective. Here's a disorganized personal summary: - Met community members as well as customers and Red Hat TAMs (Technical Account Managers) from all over Europe. - Attended a presentation on a research to understand theoretical best possible improvements to migration convergence performance[3]. I hope to see some of it in coming versions of QEMU, while we work on other improvements to this critical feature. - Attended a presentation on the status of ARM on Fedora and CentOS[14]. Seems like a great positive effort, but the road is still long for complete support. Would be cool to support ARM based hosts running virtual machines! - Attended a presentation on native vs. threads implementation performance in QEMU[4]. The results were not conclusive, though some improvements in native brought it close to or better in some cases to threads. Seems that our default of heuristic for threads in file-based storage and native in block based storage is fine for the time being, but we'll watch closely for developments in this area. - Attended a presentation on debugging qemu (mainly) in OpenStack[5] - I think (and asked) if libvirt can do a better job - in saving in a cyclic log the QMP commands sent and the response from the guest and dump it in case of a guest crash. - Attended a Cockpit hackfest[9] - as we intend to use Cockpit as our UI for RHEVH Next Generation Node, it was great to present our use case and exchange thoughts, ideas and directions. Few bugs were filed during the hackfest per our comments. - Attended 'Dockerizing JBoss Applications' presentation[10]. Great work done by the JBoss team to improve and streamline packaging of JBoss into containers. - Attended 'Fedora Upstream Testing' presentation[13]. I disscussed with the presentor the possibiliy to run oVirt with Lago[17] in Fedora, in a CI way, by monitoring distgit changes for relevant packages (such as lvm2, device-mapper*, libvirt and others. - Attended 'High Performance VMs in OpenStack'[12] - looks like oVirt is already doing a lot of what OpenStack are working to achieve high-performance from KVM! - Preached on running Lago with OpenStack, Gluster and Cockpit. At least the first two items look really promising and I'm looking forward to a collaboration in those efforts. I might just try to do the Gluster one myself. - Having used the oVirt Live[18] USB DoK image in both FOSDEM and DevConf, I've found several items where we can improve in and filed relevant RFEs for them[15][16]. - The oVirt team has delivered numerous presentations in the virtualization track[1][2][6] - and the Gluster team has additionally provided more presentations[7][8], which were all well received. I'd like to thank Red Hat's OSAS team members Mikey and Brian for once again, just one week after FOSDEM, leading the effort of representing the oVirt project and community in this event. Y. [1] https://www.youtube.com/watch?v=cQqJEiK7-Ug - Smart VM Scheduling - Martin Sivák [2] https://www.youtube.com/watch?v=V1JQtmdleaM - Host fencing in oVirt - Fixing the unknown and allowing VMs to be highly available - Martin Peřina [3] https://www.youtube.com/watch?v=XkMIMJKJeTY - in-depth look of virtual machine migration algorithms - Marcelo Tosatti [4] https://www.youtube.com/watch?v=Jx93riUF5_I - Qemu Disk I/O: Which performs better, Native or Threads? - Pradeep K Surisetty [5] https://www.youtube.com/watch?v=Dd2AGGMWXQM - Debugging the Virtualization Layer (libvirt and QEMU) in OpenStack - Kashyap Chamarthy [6] https://www.youtube.com/watch?v=4CbHTAkVDZo - Ceph integration with oVirt using Cinder - Nir Soffer [7] https://www.youtube.com/watch?v=XudYwEWQF7U - oVirt and Gluster Hyperconvergence - Ramesh Nachimuthu [8] https://www.youtube.com/watch?v=TczVVCbm8NE - Improvements in gluster for virtualization usecase - Prasanna Kumar Kalever [9] https://www.youtube.com/watch?v=TNDe90WSZow - Cockpit Hackfest - Dominik Perpeet, Marius Vollmer, Peter Volpe, Stef Walter [10] https://www.youtube.com/watch?v=NpyEoFlDzOQ - Dockerizing JBoss Products - David Becvarik [11] https://www.youtube.com/watch?v=rqavfmPAt7o - Avocado and Jenkins: Test Automation and CI - Lukáš Doktor, Yash Mankad [12] https://www.youtube.com/watch?v=9J_sEdlpIlQ - High performance VMs in OpenStack - Nikola Dipanov [13] https://www.youtube.com/watch?v=15-yXOJuonQ - Upstream First Testing - Tim Flink [14] https://www.youtube.com/watch?v=q4gU87wdRtI - Growing the ARM server ecosystem - Jim Perrin [15] https://bugzilla.redhat.com/show_bug.cgi?id=1307261 - [RFE] register .vv files so they'll be opened automatically with remote-viewer [16] https://bugzilla.redhat.com/show_bug.cgi?id=1307262 - [RFE] Disable power management (display and computer) [17] https://github.com/lago-project/lago

2 2

Master Doamin - Ovirt 3.6 - hoted engine
by Dariusz Kryszak 26 Feb '16

26 Feb '16

Hi folks, I have a question about master domain when I'm using hosted engine deployment. At the beginning I've made deployment on NUC (small home installation) with hosted engine on the nfs share from NUC host. I've configured FS gluster on the same machine and used it for master domain and iso domain. Lets say All-in-ONE. After reboot happened something strange. Log says that master domain is not available and has to become on the hosted_storage. This is not ok in my opinion. I know that behavior because master doamin is not available, has been migrated to other shareable (in this case hosted_domain is nfs ). Do you thing, that should be locked in this particular case means when available is only hosted_storage? Right now it is not possible to change this situation because hosted engine resides on the hosted_storage. I Can't migrate it. jezyk

3 7

OVirt Node / RHEV-H and Multiple NICs/bond
by Christopher Young 26 Feb '16

26 Feb '16

So, I have what I think should be a standard setup where I have dedicated NICs (to be bonded via LACP for storage) and well are NICs for various VLANS, etc. As typical, I have the main system interfaces for the usual system IPs (em1 in this case). A couple of observations (and a "chicken and the egg problem"): #1. The RHEV-H/Ovirt-Node interface doesn't allow you to configure more than one interface. Why is this? #2. This prevents me from bringing up an interface for access to my Netapp SAN (which I keep on separate networking/VLANs for best-practices purposes) If I'm unable to bring up a regular system interface AND an interface for my storage, then how am I going to be able to install a RHEV-M (engine) hosted-engine VM since I would be either unable to have an interface for this VM's IP AND be able to connect to my storage network. In short, I'm confused. I see this as a very standard enterprise setup so I feel like I must be missing something obvious. If someone could educate me, I'd really appreciate it. Thanks, CHris

2 3

Shrinking a storage domain
by Troels Arvin 26 Feb '16

26 Feb '16

Hello, I'm moving some storage from one SAN storage unit to another SAN storage unit (i.e.: block device storage). So I have two RHEV storage domains: SD1 SD2 We need to move most of SD1s data to SD2. Live storage migration for a guest works fine: It's easy to migrate a disk from being in SD1 to SD2. But now that I have released around half of SD1, I would like to let go of a LUN which is part of SD1. How is that done? (If this were a stand- alone Linux box, I would use vgreduce to release a PV and then remove the PV from the server.) -- Regards, Troels Arvin

2 1

Guest/User permissions
by Colin Coe 26 Feb '16

26 Feb '16

Hi all I've been asked to produce a matrix of guests (VMs) and the users who have rights on them. Just trying to save myself some effort and see if anyone's already done this and is willing to share? Thanks CC

1 0

One host failed to attach one of Storage Domains after reboot of all hosts
by Giuseppe Berellini 25 Feb '16

25 Feb '16

--_000_03FD1DFE37AC0E47A173AC5A0835B1C1090AD30B8CSKAWNE25CEXpt_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi, At the beginning of february I successfully installed oVirt 3.6.2 (with hos= ted engine) on 3 hosts, which are using 1 storage server with GlusterFS. 2 hosts (with Intel CPU) are using HA and are hosting the engine; the 3rd h= ost (AMD CPU) was added as host from oVirt web administration panel, withou= t hosted engine deployment (I don't want the engine running on this host). About 10 days ago I tried to reboot my oVirt environment (i.e. going to glo= bal maintenance, shutting down the engine, turning off all the hosts, start= ing them again, then setting maintenance mode to "none"). After the reboot, everything was fine with the Intel hosts and the hosted e= ngine, but AMD host (the one without HA) was not operational. I tryed to activate it, buti t failed with the following error: "Host failed to attach one of Storage Domains attached to it." If I log into my AMD host and I check the logs, I see that the storage doma= in which is not mounted is the one of the hosted engine (but this could be = correct, since this host won't run the hosted engine). >From /var/log/vdsm/vdsm.log: Thread-29::DEBUG::2016-02-25 11:44:01,157::monitor::322::Storage.Monitor::(= _produceDomain) Producing domain 6fb10a49-5f1c-4bd4-9ff7-b7e33c1125cd Thread-29::ERROR::2016-02-25 11:44:01,158::sdc::139::Storage.StorageDomainC= ache::(_findDomain) looking for unfetched domain 6fb10a49-5f1c-4bd4-9ff7-b7= e33c1125cd Thread-29::ERROR::2016-02-25 11:44:01,158::sdc::156::Storage.StorageDomainC= ache::(_findUnfetchedDomain) looking for domain 6fb10a49-5f1c-4bd4-9ff7-b7e= 33c1125cd Thread-29::DEBUG::2016-02-25 11:44:01,159::lvm::370::Storage.OperationMutex= ::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex Thread-29::DEBUG::2016-02-25 11:44:01,159::lvm::290::Storage.Misc.excCmd::(= cmd) /usr/bin/taskset --cpu-list 0-63 /usr/bin/sudo -n /usr/sbin/lvm vgs --= config ' devices { preferred_names =3D ["^/dev/mapper/"] ignore_suspended_d= evices=3D1 write_cache_state=3D0 disable_after_error_count=3D3 filter =3D [= '\''r|.*|'\'' ] } global { locking_type=3D1 prioritise_write_locks=3D1 = wait_for_locks=3D1 use_lvmetad=3D0 } backup { retain_min =3D 50 retain= _days =3D 0 } ' --noheadings --units b --nosuffix --separator '|' --ignores= kippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_cou= nt,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 6fb10a49-5f1c-4bd= 4-9ff7-b7e33c1125cd (cwd None) Thread-29::DEBUG::2016-02-25 11:44:01,223::lvm::290::Storage.Misc.excCmd::(= cmd) FAILED: <err> =3D ' WARNING: lvmetad is running but disabled. Restart= lvmetad before enabling it!\n Volume group "6fb10a49-5f1c-4bd4-9ff7-b7e33= c1125cd" not found\n Cannot process volume group 6fb10a49-5f1c-4bd4-9ff7-b= 7e33c1125cd\n'; <rc> =3D 5 Thread-29::WARNING::2016-02-25 11:44:01,225::lvm::375::Storage.LVM::(_reloa= dvgs) lvm vgs failed: 5 [] [' WARNING: lvmetad is running but disabled. Re= start lvmetad before enabling it!', ' Volume group "6fb10a49-5f1c-4bd4-9ff= 7-b7e33c1125cd" not found', ' Cannot process volume group 6fb10a49-5f1c-4b= d4-9ff7-b7e33c1125cd'] Thread-29::DEBUG::2016-02-25 11:44:01,225::lvm::415::Storage.OperationMutex= ::(_reloadvgs) Operation 'lvm reload operation' released the operation mute= x Thread-29::ERROR::2016-02-25 11:44:01,245::sdc::145::Storage.StorageDomainC= ache::(_findDomain) domain 6fb10a49-5f1c-4bd4-9ff7-b7e33c1125cd not found Traceback (most recent call last): File "/usr/share/vdsm/storage/sdc.py", line 143, in _findDomain dom =3D findMethod(sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 173, in _findUnfetchedDomain raise se.StorageDomainDoesNotExist(sdUUID) StorageDomainDoesNotExist: Storage domain does not exist: (u'6fb10a49-5f1c-= 4bd4-9ff7-b7e33c1125cd',) Thread-29::ERROR::2016-02-25 11:44:01,246::monitor::276::Storage.Monitor::(= _monitorDomain) Error monitoring domain 6fb10a49-5f1c-4bd4-9ff7-b7e33c1125c= d Traceback (most recent call last): File "/usr/share/vdsm/storage/monitor.py", line 264, in _monitorDomain self._produceDomain() File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 767, in wrapp= er value =3D meth(self, *a, **kw) File "/usr/share/vdsm/storage/monitor.py", line 323, in _produceDomain self.domain =3D sdCache.produce(self.sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 100, in produce domain.getRealDomain() File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain return self._cache._realProduce(self._sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 124, in _realProduce domain =3D self._findDomain(sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 143, in _findDomain dom =3D findMethod(sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 173, in _findUnfetchedDomain raise se.StorageDomainDoesNotExist(sdUUID) StorageDomainDoesNotExist: Storage domain does not exist: (u'6fb10a49-5f1c-= 4bd4-9ff7-b7e33c1125cd',) jsonrpc.Executor/0::DEBUG::2016-02-25 11:44:03,292::task::595::Storage.Task= Manager.Task::(_updateState) Task=3D`2862ba96-8080-4e74-a55a-cdf93326631a`:= :moving from state init -> state preparing jsonrpc.Executor/0::INFO::2016-02-25 11:44:03,293::logUtils::48::dispatcher= ::(wrapper) Run and protect: repoStats(options=3DNone) jsonrpc.Executor/0::INFO::2016-02-25 11:44:03,293::logUtils::51::dispatcher= ::(wrapper) Run and protect: repoStats, Return response: {u'5f7991ba-fdf8-4= b40-9974-c7adcd4da879': {'code': 0, 'actual': True, 'version': 3, 'acquired= ': True, 'delay': '0.00056349', 'lastCheck': '7.7', 'valid': True}, u'6fb10= a49-5f1c-4bd4-9ff7-b7e33c1125cd': {'code': 358, 'actual': True, 'version': = -1, 'acquired': False, 'delay': '0', 'lastCheck': '2.0', 'valid': False}, u= '5efea9c7-c4ec-44d4-a283-060d4c83303c': {'code': 0, 'actual': True, 'versio= n': 0, 'acquired': True, 'delay': '0.000561865', 'lastCheck': '8.4', 'valid= ': True}, u'e84c6a1a-9f82-4fa6-9a3b-0b0bc0330d9a': {'code': 0, 'actual': Tr= ue, 'version': 3, 'acquired': True, 'delay': '0.000227759', 'lastCheck': '8= .7', 'valid': True}} jsonrpc.Executor/0::DEBUG::2016-02-25 11:44:03,294::task::1191::Storage.Tas= kManager.Task::(prepare) Task=3D`2862ba96-8080-4e74-a55a-cdf93326631a`::fin= ished: {u'5f7991ba-fdf8-4b40-9974-c7adcd4da879': {'code': 0, 'actual': True= , 'version': 3, 'acquired': True, 'delay': '0.00056349', 'lastCheck': '7.7'= , 'valid': True}, u'6fb10a49-5f1c-4bd4-9ff7-b7e33c1125cd': {'code': 358, 'a= ctual': True, 'version': -1, 'acquired': False, 'delay': '0', 'lastCheck': = '2.0', 'valid': False}, u'5efea9c7-c4ec-44d4-a283-060d4c83303c': {'code': 0= , 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000561865', '= lastCheck': '8.4', 'valid': True}, u'e84c6a1a-9f82-4fa6-9a3b-0b0bc0330d9a':= {'code': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00= 0227759', 'lastCheck': '8.7', 'valid': True}} jsonrpc.Executor/0::DEBUG::2016-02-25 11:44:03,294::task::595::Storage.Task= Manager.Task::(_updateState) Task=3D`2862ba96-8080-4e74-a55a-cdf93326631a`:= :moving from state preparing -> state finished jsonrpc.Executor/0::DEBUG::2016-02-25 11:44:03,294::resourceManager::940::S= torage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} res= ources {} jsonrpc.Executor/0::DEBUG::2016-02-25 11:44:03,295::resourceManager::977::S= torage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} jsonrpc.Executor/0::DEBUG::2016-02-25 11:44:03,295::task::993::Storage.Task= Manager.Task::(_decref) Task=3D`2862ba96-8080-4e74-a55a-cdf93326631a`::ref = 0 aborting False Thread-30::DEBUG::2016-02-25 11:44:04,603::fileSD::173::Storage.Misc.excCmd= ::(getReadDelay) /usr/bin/taskset --cpu-list 0-63 /usr/bin/dd if=3D/rhev/da= ta-center/mnt/glusterSD/srv-stor-01-ib0:_ssd-pcie/e84c6a1a-9f82-4fa6-9a3b-0= b0bc0330d9a/dom_md/metadata iflag=3Ddirect of=3D/dev/null bs=3D4096 count= =3D1 (cwd None) Thread-30::DEBUG::2016-02-25 11:44:04,630::fileSD::173::Storage.Misc.excCmd= ::(getReadDelay) SUCCESS: <err> =3D '0+1 records in\n0+1 records out\n336 b= ytes (336 B) copied, 0.000286148 s, 1.2 MB/s\n'; <rc> =3D 0 Thread-31::DEBUG::2016-02-25 11:44:04,925::fileSD::173::Storage.Misc.excCmd= ::(getReadDelay) /usr/bin/taskset --cpu-list 0-63 /usr/bin/dd if=3D/rhev/da= ta-center/mnt/glusterSD/srv-stor-01-ib0:_isodomain/5efea9c7-c4ec-44d4-a283-= 060d4c83303c/dom_md/metadata iflag=3Ddirect of=3D/dev/null bs=3D4096 count= =3D1 (cwd None) Thread-31::DEBUG::2016-02-25 11:44:04,950::fileSD::173::Storage.Misc.excCmd= ::(getReadDelay) SUCCESS: <err> =3D '0+1 records in\n0+1 records out\n339 b= ytes (339 B) copied, 0.0005884 s, 576 kB/s\n'; <rc> =3D 0 Thread-28::DEBUG::2016-02-25 11:44:05,583::fileSD::173::Storage.Misc.excCmd= ::(getReadDelay) /usr/bin/taskset --cpu-list 0-63 /usr/bin/dd if=3D/rhev/da= ta-center/mnt/glusterSD/srv-stor-01-ib0:_virtualmachines/5f7991ba-fdf8-4b40= -9974-c7adcd4da879/dom_md/metadata iflag=3Ddirect of=3D/dev/null bs=3D4096 = count=3D1 (cwd None) Thread-28::DEBUG::2016-02-25 11:44:05,606::fileSD::173::Storage.Misc.excCmd= ::(getReadDelay) SUCCESS: <err> =3D '0+1 records in\n0+1 records out\n482 b= ytes (482 B) copied, 0.000637557 s, 756 kB/s\n'; <rc> =3D 0 Other commands (executed on the host having the problem) which probably giv= e useful information: # vdsClient -s 0 getConnectedStoragePoolsList 00000001-0001-0001-0001-00000000020e # vdsClient -s 0 getStoragePoolInfo 00000001-0001-0001-0001-00000000020e name =3D No Description isoprefix =3D /rhev/data-center/mnt/glusterSD/srv-stor-01-ib0:_isod= omain/5efea9c7-c4ec-44d4-a283-060d4c83303c/images/11111111-1111-1111-1111-1= 11111111111 pool_status =3D connected lver =3D 6 spm_id =3D 2 master_uuid =3D 5f7991ba-fdf8-4b40-9974-c7adcd4da879 version =3D 3 domains =3D 5f7991ba-fdf8-4b40-9974-c7adcd4da879:Active,6fb10a49-5f= 1c-4bd4-9ff7-b7e33c1125cd:Active,5efea9c7-c4ec-44d4-a283-060d4c83303c:Activ= e,e84c6a1a-9f82-4fa6-9a3b-0b0bc0330d9a:Active type =3D GLUSTERFS master_ver =3D 1 5f7991ba-fdf8-4b40-9974-c7adcd4da879 =3D {'status': 'Active', 'disk= free': '6374172262400', 'isoprefix': '', 'alerts': [], 'disktotal': '699543= 6371968', 'version': 3} 6fb10a49-5f1c-4bd4-9ff7-b7e33c1125cd =3D {'status': 'Active', 'isop= refix': '', 'alerts': [], 'version': -1} e84c6a1a-9f82-4fa6-9a3b-0b0bc0330d9a =3D {'status': 'Active', 'disk= free': '224145833984', 'isoprefix': '', 'alerts': [], 'disktotal': '2363171= 79904', 'version': 3} 5efea9c7-c4ec-44d4-a283-060d4c83303c =3D {'status': 'Active', 'disk= free': '6374172262400', 'isoprefix': '/rhev/data-center/mnt/glusterSD/srv-s= tor-01-ib0:_isodomain/5efea9c7-c4ec-44d4-a283-060d4c83303c/images/11111111-= 1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '6995436371968', '= version': 0} # vdsClient -s 0 getStorageDomainInfo 6fb10a49-5f1c-4bd4-9ff7-b7e33c1125cd Storage domain does not exist: ('6fb10a49-5f1c-4bd4-9ff7-b7e33c1125cd',) If I run this last command on one of the working hosts: # vdsClient -s 0 getStorageDomainInfo 6fb10a49-5f1c-4bd4-9ff7-b7e33c1125cd uuid =3D 6fb10a49-5f1c-4bd4-9ff7-b7e33c1125cd version =3D 3 role =3D Regular remotePath =3D srv-stor-01-ib0:/ovirtengine type =3D GLUSTERFS class =3D Data pool =3D ['00000001-0001-0001-0001-00000000020e'] name =3D hosted_storage (please note: this is the storage domain used for my hosted engine) If I run "mount" on my AMD host (the one with the problem): # mount ... srv-stor-01-ib0:/virtualmachines on /rhev/data-center/mnt/glusterSD/srv-sto= r-01-ib0:_virtualmachines type fuse.glusterfs (rw,relatime,user_id=3D0,grou= p_id=3D0,default_permissions,allow_other,max_read=3D131072) srv-stor-01-ib0:/isodomain on /rhev/data-center/mnt/glusterSD/srv-stor-01-i= b0:_isodomain type fuse.glusterfs (rw,relatime,user_id=3D0,group_id=3D0,def= ault_permissions,allow_other,max_read=3D131072) srv-stor-01-ib0:/ssd-pcie on /rhev/data-center/mnt/glusterSD/srv-stor-01-ib= 0:_ssd-pcie type fuse.glusterfs (rw,relatime,user_id=3D0,group_id=3D0,defau= lt_permissions,allow_other,max_read=3D131072) tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=3D13185796k,= mode=3D700) If I run "mount" on one of the Intel hosts (currently working): # mount ... srv-stor-01-ib0:/ovirtengine on /rhev/data-center/mnt/glusterSD/srv-stor-01= -ib0:_ovirtengine type fuse.glusterfs (rw,relatime,user_id=3D0,group_id=3D0= ,default_permissions,allow_other,max_read=3D131072) srv-stor-01-ib0:/virtualmachines on /rhev/data-center/mnt/glusterSD/srv-sto= r-01-ib0:_virtualmachines type fuse.glusterfs (rw,relatime,user_id=3D0,grou= p_id=3D0,default_permissions,allow_other,max_read=3D131072) srv-stor-01-ib0:/isodomain on /rhev/data-center/mnt/glusterSD/srv-stor-01-i= b0:_isodomain type fuse.glusterfs (rw,relatime,user_id=3D0,group_id=3D0,def= ault_permissions,allow_other,max_read=3D131072) srv-stor-01-ib0:/ssd-pcie on /rhev/data-center/mnt/glusterSD/srv-stor-01-ib= 0:_ssd-pcie type fuse.glusterfs (rw,relatime,user_id=3D0,group_id=3D0,defau= lt_permissions,allow_other,max_read=3D131072) tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=3D3272288k,m= ode=3D700) The only difference in "mount" is that the hosted-engine storage domain is = not loaded on the host which should not run the engine. The other domains a= re mounted correctly. What could I do to solve this issue? Best regards, Giuseppe -- Giuseppe Berellini PTV SISTeMA www.sistemaits.com<http://www.sistemaits.com/> facebook.com/sistemaits<https://www.facebook.com/sistemaits> linkedin.com/SISTeMA<https://www.linkedin.com/company/sistema-soluzioni-per= -l-ingegneria-dei-sistemi-di-trasporto-e-l-infomobilit-s-r-l-> --_000_03FD1DFE37AC0E47A173AC5A0835B1C1090AD30B8CSKAWNE25CEXpt_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable <html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr= osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" = xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:= //www.w3.org/TR/REC-html40"><head><meta http-equiv=3DContent-Type content= =3D"text/html; charset=3Dus-ascii"><meta name=3DGenerator content=3D"Micros= oft Word 15 (filtered medium)"><style></style></head><body lang=3DIT link=3D"#0563C1" v= link=3D"#954F72"><div class=3DWordSection1>Hi,<o:p></o= :p><o:p> </o:p>At th= e beginning of february I successfully installed oVirt 3.6.2 (with hosted e= ngine) on 3 hosts, which are using 1 storage server with GlusterFS.<o:p></o= :p>2 hosts (with Intel CPU) are using HA and are h= osting the engine; the 3rd host (AMD CPU) was added as host from oVirt web = administration panel, without hosted engine deployment (I don’t want = the engine running on this host).<o:p></o:p><o:p>&= nbsp;</o:p>About 10 days ago I tried to reboot my = oVirt environment (i.e. going to global maintenance, shutting down the engi= ne, turning off all the hosts, starting them again, then setting maintenanc= e mode to “none”).<o:p></o:p>After the= reboot, everything was fine with the Intel hosts and the hosted engine, bu= t AMD host (the one without HA) was not operational.<o:p></o:p>I tryed to activate it, buti t failed with the following error= :<o:p></o:p>      &n= bsp; “Host failed to attach one of Storage Domains attached to it.&#8= 221;<o:p></o:p><o:p> </o:p>If I log into my AMD host and I check the logs, I see that the stora= ge domain which is not mounted is the one of the hosted engine (but this co= uld be correct, since this host won’t run the hosted engine).<o:p></o= :p><o:p> </o:p>From = /var/log/vdsm/vdsm.log:<o:p></o:p><o:p> </o:p= >Thread-= 29::DEBUG::2016-02-25 11:44:01,157::monitor::322::Storage.Monitor::(_produc= eDomain) Producing domain 6fb10a49-5f1c-4bd4-9ff7-b7e33c1125cd<o:p></o:p></= span>Thr= ead-29::ERROR::2016-02-25 11:44:01,158::sdc::139::Storage.StorageDomainCach= e::(_findDomain) looking for unfetched domain 6fb10a49-5f1c-4bd4-9ff7-b7e33= c1125cd<o:p></o:p>Thread-29::ERROR::2016-02-25 11:44:01,158::sdc::156::Stora= ge.StorageDomainCache::(_findUnfetchedDomain) looking for domain 6fb10a49-5= f1c-4bd4-9ff7-b7e33c1125cd<o:p></o:p>Thread-29::DEBUG::2016-02-25 11:44:01,1= 59::lvm::370::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload op= eration' got the operation mutex<o:p></o:p>= Thread-29::DEBUG::2016-02-25 11:4= 4:01,159::lvm::290::Storage.Misc.excCmd::(cmd) /usr/bin/taskset --cpu-list = 0-63 /usr/bin/sudo -n /usr/sbin/lvm vgs --config ' devices { preferred_name= s =3D ["^/dev/mapper/"] ignore_suspended_devices=3D1 write_cache_= state=3D0 disable_after_error_count=3D3 filter =3D [ '\''r|.*|'\'' ] }&nbsp= ; global {  locking_type=3D1  prioritise_write_locks=3D1  wa= it_for_locks=3D1  use_lvmetad=3D0 }  backup {  retain_min = =3D 50  retain_days =3D 0 } ' --noheadings --units b --nosuffix --sepa= rator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,ex= tent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_nam= e 6fb10a49-5f1c-4bd4-9ff7-b7e33c1125cd (cwd None)<o:p></o:p>Thread-29::DEBUG= ::2016-02-25 11:44:01,223::lvm::290::Storage.Misc.excCmd::(cmd) FAILED: &lt= ;err> =3D '  WARNING: lvmetad is running but disabled. Restart lvme= tad before enabling it!\n  Volume group "6fb10a49-5f1c-4bd4-9ff7-= b7e33c1125cd" not found\n  Cannot process volume group 6fb10a49-5= f1c-4bd4-9ff7-b7e33c1125cd\n'; <rc> =3D 5<o:p></o:p>Thread-29::WARNING= ::2016-02-25 11:44:01,225::lvm::375::Storage.LVM::(_reloadvgs) lvm vgs fail= ed: 5 [] ['  WARNING: lvmetad is running but disabled. Restart lvmetad= before enabling it!', '  Volume group "6fb10a49-5f1c-4bd4-9ff7-b= 7e33c1125cd" not found', '  Cannot process volume group 6fb10a49-= 5f1c-4bd4-9ff7-b7e33c1125cd']<o:p></o:p><sp= an style=3D'font-family:"Courier New"'>Thread-29::DEBUG::2016-02-25 11:44:0= 1,225::lvm::415::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload= operation' released the operation mutex<o:p></o:p>Thread-29::ERROR::2016-02= -25 11:44:01,245::sdc::145::Storage.StorageDomainCache::(_findDomain) domai= n 6fb10a49-5f1c-4bd4-9ff7-b7e33c1125cd not found<o:p></o:p>Traceback (most r= ecent call last):<o:p></o:p>  File "/usr/share/vdsm/storage/sdc.py= ", line 143, in _findDomain<o:p></o:p>=     dom =3D findMe= thod(sdUUID)<o:p></o:p>  File "/usr/share/vdsm/storage/sdc.py&quot= ;, line 173, in _findUnfetchedDomain<o:p></o:p>    raise se.S= torageDomainDoesNotExist(sdUUID)<o:p></o:p>= StorageDomainDoesNotExist: Storag= e domain does not exist: (u'6fb10a49-5f1c-4bd4-9ff7-b7e33c1125cd',)<o:p></o= :p>Thread-29::ERROR::2016-02-25 11:44:01,246::monitor::276::Storage.Monitor:= :(_monitorDomain) Error monitoring domain 6fb10a49-5f1c-4bd4-9ff7-b7e33c112= 5cd<o:p></o:p>Traceback (most recent call last):<o:p></o:p>  File "= /usr/share/vdsm/storage/monitor.py", line 264, in _monitorDomain<o:p><= /o:p>    self._produceDomain()<o:p></o:p>  File "/u= sr/lib/python2.7/site-packages/vdsm/utils.py", line 767, in wrapper<o:= p></o:p>    value =3D meth(self, *a, **kw)<o:p></o:p><= /p>  Fi= le "/usr/share/vdsm/storage/monitor.py", line 323, in _produceDom= ain<o:p></o:p>    self.domain =3D sdCache.produce(self.sdUUID= )<o:p></o:p>  File "/usr/share/vdsm/storage/sdc.py", line 100= , in produce<o:p></o:p>    domain.getRealDomain()<o:p></o:p><= /span>&n= bsp; File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDo= main<o:p></o:p>    return self._cache._realProduce(self._sdUU= ID)<o:p></o:p>  File "/usr/share/vdsm/storage/sdc.py", line 1= 24, in _realProduce<o:p></o:p>    domain =3D self._findDoma= in(sdUUID)<o:p></o:p>  File "/usr/share/vdsm/storage/sdc.py",= line 143, in _findDomain<o:p></o:p>    dom =3D findMethod(sd= UUID)<o:p></o:p>  File "/usr/share/vdsm/storage/sdc.py", line= 173, in _findUnfetchedDomain<o:p></o:p><sp= an style=3D'font-family:"Courier New"'>    raise se.StorageD= omainDoesNotExist(sdUUID)<o:p></o:p>StorageDomainDoesNotExist: Storage domai= n does not exist: (u'6fb10a49-5f1c-4bd4-9ff7-b7e33c1125cd',)<o:p></o:p></sp= an>jsonr= pc.Executor/0::DEBUG::2016-02-25 11:44:03,292::task::595::Storage.TaskManag= er.Task::(_updateState) Task=3D`2862ba96-8080-4e74-a55a-cdf93326631a`::movi= ng from state init -> state preparing<o:p></o:p>jsonrpc.Executor/0::INFO:= :2016-02-25 11:44:03,293::logUtils::48::dispatcher::(wrapper) Run and prote= ct: repoStats(options=3DNone)<o:p></o:p><sp= an style=3D'font-family:"Courier New"'>jsonrpc.Executor/0::INFO::2016-02-25= 11:44:03,293::logUtils::51::dispatcher::(wrapper) Run and protect: repoSta= ts, Return response: {u'5f7991ba-fdf8-4b40-9974-c7adcd4da879': {'code': 0, = 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00056349', 'las= tCheck': '7.7', 'valid': True}, u'6fb10a49-5f1c-4bd4-9ff7-b7e33c1125cd': {'= code': 358, 'actual': True, 'version': -1, 'acquired': False, 'delay': '0',= 'lastCheck': '2.0', 'valid': False}, u'5efea9c7-c4ec-44d4-a283-060d4c83303= c': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0= .000561865', 'lastCheck': '8.4', 'valid': True}, u'e84c6a1a-9f82-4fa6-9a3b-= 0b0bc0330d9a': {'code': 0, 'actual': True, 'version': 3, 'acquired': True, = 'delay': '0.000227759', 'lastCheck': '8.7', 'valid': True}}<o:p></o:p></spa= n>jsonrp= c.Executor/0::DEBUG::2016-02-25 11:44:03,294::task::1191::Storage.TaskManag= er.Task::(prepare) Task=3D`2862ba96-8080-4e74-a55a-cdf93326631a`::finished:= {u'5f7991ba-fdf8-4b40-9974-c7adcd4da879': {'code': 0, 'actual': True, 'ver= sion': 3, 'acquired': True, 'delay': '0.00056349', 'lastCheck': '7.7', 'val= id': True}, u'6fb10a49-5f1c-4bd4-9ff7-b7e33c1125cd': {'code': 358, 'actual'= : True, 'version': -1, 'acquired': False, 'delay': '0', 'lastCheck': '2.0',= 'valid': False}, u'5efea9c7-c4ec-44d4-a283-060d4c83303c': {'code': 0, 'act= ual': True, 'version': 0, 'acquired': True, 'delay': '0.000561865', 'lastCh= eck': '8.4', 'valid': True}, u'e84c6a1a-9f82-4fa6-9a3b-0b0bc0330d9a': {'cod= e': 0, 'actual': True, 'version': 3, 'acquired': True, 'delay': '0.00022775= 9', 'lastCheck': '8.7', 'valid': True}}<o:p></o:p>jsonrpc.Executor/0::DEBUG:= :2016-02-25 11:44:03,294::task::595::Storage.TaskManager.Task::(_updateStat= e) Task=3D`2862ba96-8080-4e74-a55a-cdf93326631a`::moving from state prepari= ng -> state finished<o:p></o:p>jsonrpc.Executor/0::DEBUG::2016-02-25 11:4= 4:03,294::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll)= Owner.releaseAll requests {} resources {}<o:p></o:p>jsonrpc.Executor/0::DEB= UG::2016-02-25 11:44:03,295::resourceManager::977::Storage.ResourceManager.= Owner::(cancelAll) Owner.cancelAll requests {}<o:p></o:p>jsonrpc.Executor/0:= :DEBUG::2016-02-25 11:44:03,295::task::993::Storage.TaskManager.Task::(_dec= ref) Task=3D`2862ba96-8080-4e74-a55a-cdf93326631a`::ref 0 aborting False<o:= p></o:p>Thread-30::DEBUG::2016-02-25 11:44:04,603::fileSD::173::Storage.Misc= .excCmd::(getReadDelay) /usr/bin/taskset --cpu-list 0-63 /usr/bin/dd if=3D/= rhev/data-center/mnt/glusterSD/srv-stor-01-ib0:_ssd-pcie/e84c6a1a-9f82-4fa6= -9a3b-0b0bc0330d9a/dom_md/metadata iflag=3Ddirect of=3D/dev/null bs=3D4096 = count=3D1 (cwd None)<o:p></o:p>Thread-30::DEBUG::2016-02-25 11:44:04,630::f= ileSD::173::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> =3D '0= +1 records in\n0+1 records out\n336 bytes (336 B) copied, 0.000286148 s, 1.= 2 MB/s\n'; <rc> =3D 0<o:p></o:p><span= style=3D'font-family:"Courier New"'>Thread-31::DEBUG::2016-02-25 11:44:04,= 925::fileSD::173::Storage.Misc.excCmd::(getReadDelay) /usr/bin/taskset --cp= u-list 0-63 /usr/bin/dd if=3D/rhev/data-center/mnt/glusterSD/srv-stor-01-ib= 0:_isodomain/5efea9c7-c4ec-44d4-a283-060d4c83303c/dom_md/metadata iflag=3Dd= irect of=3D/dev/null bs=3D4096 count=3D1 (cwd None)<o:p></o:p><p= class=3DMsoNormal>Thread-31::DEB= UG::2016-02-25 11:44:04,950::fileSD::173::Storage.Misc.excCmd::(getReadDela= y) SUCCESS: <err> =3D '0+1 records in\n0+1 records out\n339 bytes (33= 9 B) copied, 0.0005884 s, 576 kB/s\n'; <rc> =3D 0<o:p></o:p></= p>Thread-28:= :DEBUG::2016-02-25 11:44:05,583::fileSD::173::Storage.Misc.excCmd::(getRead= Delay) /usr/bin/taskset --cpu-list 0-63 /usr/bin/dd if=3D/rhev/data-center/= mnt/glusterSD/srv-stor-01-ib0:_virtualmachines/5f7991ba-fdf8-4b40-9974-c7ad= cd4da879/dom_md/metadata iflag=3Ddirect of=3D/dev/null bs=3D4096 count=3D1 = (cwd None)<o:p></o:p>Thread-28::DEBUG::2016-02-25 11:44:05,606::fileSD::173:= :Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> =3D '0+1 records = in\n0+1 records out\n482 bytes (482 B) copied, 0.000637557 s, 756 kB/s\n'; = <rc> =3D 0<o:p></o:p><o:p> </o:p= ><o:p> </o:p><o:p>&n= bsp;</o:p>Other commands (executed on the host hav= ing the problem) which probably give useful information:<o:p></o:p># vdsClient -s 0= getConnectedStoragePoolsList<o:p></o:p><sp= an style=3D'font-family:"Courier New"'>00000001-0001-0001-0001-00000000020e= <o:p></o:p><o:p> </o:p># vdsClient -s 0 getStoragePoolInfo 00000001-0001= -0001-0001-00000000020e<o:p></o:p>       = name =3D No Description<o:p></o:p>      &nbsp= ; isoprefix =3D /rhev/data-center/mnt/glusterSD/srv-stor-01-ib0:_isodomain/= 5efea9c7-c4ec-44d4-a283-060d4c83303c/images/11111111-1111-1111-1111-1111111= 11111<o:p></o:p>        pool_status =3D c= onnected<o:p></o:p>        lver =3D 6<o:p= ></o:p>        spm_id =3D 2<o:p></o:p></s= pan>&nbs= p;       master_uuid =3D 5f7991ba-fdf8-4b40-9= 974-c7adcd4da879<o:p></o:p>        versio= n =3D 3<o:p></o:p>        domains =3D 5f7= 991ba-fdf8-4b40-9974-c7adcd4da879:Active,6fb10a49-5f1c-4bd4-9ff7-b7e33c1125= cd:Active,5efea9c7-c4ec-44d4-a283-060d4c83303c:Active,e84c6a1a-9f82-4fa6-9a= 3b-0b0bc0330d9a:Active<o:p></o:p>        = type =3D GLUSTERFS<o:p></o:p>        mast= er_ver =3D 1<o:p></o:p>        5f7991ba-f= df8-4b40-9974-c7adcd4da879 =3D {'status': 'Active', 'diskfree': '6374172262= 400', 'isoprefix': '', 'alerts': [], 'disktotal': '6995436371968', 'version= ': 3}<o:p></o:p>        6fb10a49-5f1c-4bd= 4-9ff7-b7e33c1125cd =3D {'status': 'Active', 'isoprefix': '', 'alerts': [],= 'version': -1}<o:p></o:p>        e84c6a1= a-9f82-4fa6-9a3b-0b0bc0330d9a =3D {'status': 'Active', 'diskfree': '2241458= 33984', 'isoprefix': '', 'alerts': [], 'disktotal': '236317179904', 'versio= n': 3}<o:p></o:p>        5efea9c7-c4ec-44= d4-a283-060d4c83303c =3D {'status': 'Active', 'diskfree': '6374172262400', = 'isoprefix': '/rhev/data-center/mnt/glusterSD/srv-stor-01-ib0:_isodomain/5e= fea9c7-c4ec-44d4-a283-060d4c83303c/images/11111111-1111-1111-1111-111111111= 111', 'alerts': [], 'disktotal': '6995436371968', 'version': 0}<o:p></o:p><= /span><o= :p> </o:p># vdsClient -s 0 getStorageDomainInfo 6fb10a49-5f1c-4bd4-9ff7= -b7e33c1125cd<o:p></o:p>Storage domain does not exist: ('6fb10a49-5f1c-4bd4-= 9ff7-b7e33c1125cd',)<o:p></o:p><o:p> <= /o:p>If I run this last command on one of the work= ing hosts:<o:p></o:p># vdsClient -s 0 getStorageDomainInfo 6fb10a49-5f1c-4bd4-9ff7-= b7e33c1125cd<o:p></o:p>        uuid =3D 6= fb10a49-5f1c-4bd4-9ff7-b7e33c1125cd<o:p></o:p>     =    version =3D 3<o:p></o:p>      &nb= sp; role =3D Regular<o:p></o:p>        r= emotePath =3D srv-stor-01-ib0:/ovirtengine<o:p></o:p>   &nbsp= ;    type =3D GLUSTERFS<o:p></o:p>    &nb= sp;   class =3D Data<o:p></o:p><s= pan style=3D'font-family:"Courier New"'>     &nbsp= ;  pool =3D ['00000001-0001-0001-0001-00000000020e']<o:p></o:p>=  &n= bsp;      name =3D hosted_storage<o:p></o:p></span= ><o:p> </o:p>(please= note: this is the storage domain used for my hosted engine)<o:p></o:p>= <o:p> </o:p><o:p> <= /o:p><o:p> </o:p>If = I run “mount” on my AMD host (the one with the problem):<o:p></= o:p># mo= unt<o:p></o:p>...<o:p></o:p>srv-stor-01-ib0:/virtualmachines on /rhev/data-ce= nter/mnt/glusterSD/srv-stor-01-ib0:_virtualmachines type fuse.glusterfs (rw= ,relatime,user_id=3D0,group_id=3D0,default_permissions,allow_other,max_read= =3D131072)<o:p></o:p>srv-stor-01-ib0:/isodomain on /rhev/data-center/mnt/glu= sterSD/srv-stor-01-ib0:_isodomain type fuse.glusterfs (rw,relatime,user_id= =3D0,group_id=3D0,default_permissions,allow_other,max_read=3D131072)<o:p></= o:p>srv-stor-01-ib0:/ssd-pcie on /rhev/data-center/mnt/glusterSD/srv-stor-01= -ib0:_ssd-pcie type fuse.glusterfs (rw,relatime,user_id=3D0,group_id=3D0,de= fault_permissions,allow_other,max_read=3D131072)<o:p></o:p>tmpfs on /run/use= r/0 type tmpfs (rw,nosuid,nodev,relatime,size=3D13185796k,mode=3D700)<o:p><= /o:p><o:p> </o:p>If I run “mount&= #8221; on one of the Intel hosts (currently working):<o:p></o:p># mount<o:p></o:p><= /span>..= .<o:p></o:p>srv-stor-01-ib0:/ovirtengine on /rhev/data-center/mnt/glusterSD/= srv-stor-01-ib0:_ovirtengine type fuse.glusterfs (rw,relatime,user_id=3D0,g= roup_id=3D0,default_permissions,allow_other,max_read=3D131072)<o:p></o:p></= span>srv= -stor-01-ib0:/virtualmachines on /rhev/data-center/mnt/glusterSD/srv-stor-0= 1-ib0:_virtualmachines type fuse.glusterfs (rw,relatime,user_id=3D0,group_i= d=3D0,default_permissions,allow_other,max_read=3D131072)<o:p></o:p><= /p>srv-stor-= 01-ib0:/isodomain on /rhev/data-center/mnt/glusterSD/srv-stor-01-ib0:_isodo= main type fuse.glusterfs (rw,relatime,user_id=3D0,group_id=3D0,default_perm= issions,allow_other,max_read=3D131072)<o:p></o:p>srv-stor-01-ib0:/ssd-pcie o= n /rhev/data-center/mnt/glusterSD/srv-stor-01-ib0:_ssd-pcie type fuse.glust= erfs (rw,relatime,user_id=3D0,group_id=3D0,default_permissions,allow_other,= max_read=3D131072)<o:p></o:p>tmpfs on /run/user/0 type tmpfs (rw,nosuid,node= v,relatime,size=3D3272288k,mode=3D700)<o:p></o:p><o:p> </o:p></p= >The only difference in “mount” is that th= e hosted-engine storage domain is not loaded on the host which should not r= un the engine. The other domains are mounted correctly.<o:p></o:p><o:p> </o:p>What could I do t= o solve this issue?<o:p></o:p><o:p> </o:p></p= >Best regards,<o:p></o:p>&nbs= p;       Giuseppe<o:p></o:p><o:p> </o:p><o:p> </o:p>-- Giuseppe Berellini<o:p></o:p>PTV SISTeMA<o:p></o:p><a hr= ef=3D"http://www.sistemaits.com/">www.sistemaits.com</a><o:p></o:p><a href=3D"https://www.facebook.com/sistemaits">facebook.com/sistemaits</a><= br><a href=3D"https://www.linkedin.com/company/sistema-soluzioni-per-l-inge= gneria-dei-sistemi-di-trasporto-e-l-infomobilit-s-r-l-">linkedin.com/SISTeMA</a>  </sp= an><o:p></o:p></div></body></html>= --_000_03FD1DFE37AC0E47A173AC5A0835B1C1090AD30B8CSKAWNE25CEXpt_--

1 1