Can we have the gluster mount logs and brick logs to check if it's the same issue? On Sun, Jun 4, 2017 at 11:21 PM, Abi Askushi <rightkicktech@gmail.com> wrote:
I clean installed everything and ran into the same. I then ran gdeploy and encountered the same issue when deploying engine. Seems that gluster (?) doesn't like 4K sector drives. I am not sure if it has to do with alignment. The weird thing is that gluster volumes are all ok, replicating normally and no split brain is reported.
The solution to the mentioned bug (1386443 <https://bugzilla.redhat.com/show_bug.cgi?id=1386443>) was to format with 512 sector size, which for my case is not an option:
mkfs.xfs -f -i size=512 -s size=512 /dev/gluster/engine illegal sector size 512; hw sector is 4096
Is there any workaround to address this?
Thanx, Alex
On Sun, Jun 4, 2017 at 5:48 PM, Abi Askushi <rightkicktech@gmail.com> wrote:
Hi Maor,
My disk are of 4K block size and from this bug seems that gluster replica needs 512B block size. Is there a way to make gluster function with 4K drives?
Thank you!
On Sun, Jun 4, 2017 at 2:34 PM, Maor Lipchuk <mlipchuk@redhat.com> wrote:
Hi Alex,
I saw a bug that might be related to the issue you encountered at https://bugzilla.redhat.com/show_bug.cgi?id=1386443
Sahina, maybe you have any advise? Do you think that BZ1386443is related?
Regards, Maor
Hi All,
I have installed successfully several times oVirt (version 4.1) with 3 nodes on top glusterfs.
This time, when trying to configure the same setup, I am facing the following issue which doesn't seem to go away. During installation i get the error:
Failed to execute stage 'Misc configuration': Cannot acquire host id: (u'a5a6b0e7-fc3f-4838-8e26-c8b4d5e5e922', SanlockException(22, 'Sanlock lockspace add failure', 'Invalid argument'))
The only different in this setup is that instead of standard
have GPT partitioning and the disks have 4K block size instead of 512.
The /var/log/sanlock.log has the following lines:
2017-06-03 19:21:15+0200 23450 [943]: s9 lockspace ba6bd862-c2b8-46e7-b2c8-91e4a5bb2047:250:/rhev/data-center/m nt/_var_lib_ovirt-hosted-engin-setup_tmptjkIDI/ba6bd862- c2b8-46e7-b2c8-91e4a5bb2047/dom_md/ids:0 2017-06-03 19:21:36+0200 23471 [944]: s9:r5 resource ba6bd862-c2b8-46e7-b2c8-91e4a5bb2047:SDM:/rhev/data-center/m nt/_var_lib_ovirt-hosted-engine-setup_tmptjkIDI/ba6bd862- c2b8-46e7-b2c8-91e4a5bb2047/dom_md/leases:1048576 for 2,9,23040 2017-06-03 19:21:36+0200 23471 [943]: s10 lockspace a5a6b0e7-fc3f-4838-8e26-c8b4d5e5e922:250:/rhev/data-center/m nt/glusterSD/10.100.100.1:_engine/a5a6b0e7-fc3f-4838-8e26- c8b4d5e5e922/dom_md/ids:0 2017-06-03 19:21:36+0200 23471 [23522]: a5a6b0e7 aio collect RD 0x7f59b00008c0:0x7f59b00008d0:0x7f59b0101000 result -22:0 match res 2017-06-03 19:21:36+0200 23471 [23522]: read_sectors delta_leader offset 127488 rv -22 /rhev/data-center/mnt/glusterSD/10.100.100.1:_engine/a5a6b0e 7-fc3f-4838-8e26-c8b4d5e5e922/dom_md/ids 2017-06-03 19:21:37+0200 23472 [930]: s9 host 250 1 23450 88c2244c-a782-40ed-9560-6cfa4d46f853.v0.neptune 2017-06-03 19:21:37+0200 23472 [943]: s10 add_lockspace fail result -22
And /var/log/vdsm/vdsm.log says:
2017-06-03 19:19:38,176+0200 WARN (jsonrpc/3) [storage.StorageServer.MountConnection] Using user specified backup-volfile-servers option (storageServer:253) 2017-06-03 19:21:12,379+0200 WARN (periodic/1) [throttled] MOM not available. (throttledlog:105) 2017-06-03 19:21:12,380+0200 WARN (periodic/1) [throttled] MOM not available, KSM stats will be missing. (throttledlog:105) 2017-06-03 19:21:14,714+0200 WARN (jsonrpc/1) [storage.StorageServer.MountConnection] Using user specified backup-volfile-servers option (storageServer:253) 2017-06-03 19:21:15,515+0200 ERROR (jsonrpc/4) [storage.initSANLock] Cannot initialize SANLock for domain a5a6b0e7-fc3f-4838-8e26-c8b4d5e5e922 (clusterlock:238) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/clusterlock.py",
234, in initSANLock sanlock.init_lockspace(sdUUID, idsPath) SanlockException: (107, 'Sanlock lockspace init failure', 'Transport endpoint is not connected') 2017-06-03 19:21:15,515+0200 WARN (jsonrpc/4) [storage.StorageDomainManifest] lease did not initialize successfully (sd:557) Traceback (most recent call last): File "/usr/share/vdsm/storage/sd.py", line 552, in initDomainLock self._domainLock.initLock(self.getDomainLease()) File "/usr/lib/python2.7/site-packages/vdsm/storage/clusterlock.py",
271, in initLock initSANLock(self._sdUUID, self._idsPath, lease) File "/usr/lib/python2.7/site-packages/vdsm/storage/clusterlock.py",
239, in initSANLock raise se.ClusterLockInitError() ClusterLockInitError: Could not initialize cluster lock: () 2017-06-03 19:21:37,867+0200 ERROR (jsonrpc/2) [storage.StoragePool] Create pool hosted_datacenter canceled (sp:655) Traceback (most recent call last): File "/usr/share/vdsm/storage/sp.py", line 652, in create self.attachSD(sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py",
79, in wrapper return method(self, *args, **kwargs) File "/usr/share/vdsm/storage/sp.py", line 971, in attachSD dom.acquireHostId(self.id) File "/usr/share/vdsm/storage/sd.py", line 790, in acquireHostId self._manifest.acquireHostId(hostId, async) File "/usr/share/vdsm/storage/sd.py", line 449, in acquireHostId self._domainLock.acquireHostId(hostId, async) File "/usr/lib/python2.7/site-packages/vdsm/storage/clusterlock.py",
297, in acquireHostId raise se.AcquireHostIdFailure(self._sdUUID, e) AcquireHostIdFailure: Cannot acquire host id: (u'a5a6b0e7-fc3f-4838-8e26-c8b4d5e5e922', SanlockException(22, 'Sanlock lockspace add failure', 'Invalid argument')) 2017-06-03 19:21:37,870+0200 ERROR (jsonrpc/2) [storage.StoragePool] Domain ba6bd862-c2b8-46e7-b2c8-91e4a5bb2047 detach from MSD ba6bd862-c2b8-46e7-b2c8-91e4a5bb2047 Ver 1 failed. (sp:528) Traceback (most recent call last): File "/usr/share/vdsm/storage/sp.py", line 525, in __cleanupDomains self.detachSD(sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py",
79, in wrapper return method(self, *args, **kwargs) File "/usr/share/vdsm/storage/sp.py", line 1046, in detachSD raise se.CannotDetachMasterStorageDomain(sdUUID) CannotDetachMasterStorageDomain: Illegal action: (u'ba6bd862-c2b8-46e7-b2c8-91e4a5bb2047',) 2017-06-03 19:21:37,872+0200 ERROR (jsonrpc/2) [storage.StoragePool] Domain a5a6b0e7-fc3f-4838-8e26-c8b4d5e5e922 detach from MSD ba6bd862-c2b8-46e7-b2c8-91e4a5bb2047 Ver 1 failed. (sp:528) Traceback (most recent call last): File "/usr/share/vdsm/storage/sp.py", line 525, in __cleanupDomains self.detachSD(sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py",
79, in wrapper return method(self, *args, **kwargs) File "/usr/share/vdsm/storage/sp.py", line 1043, in detachSD self.validateAttachedDomain(dom) File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py",
79, in wrapper return method(self, *args, **kwargs) File "/usr/share/vdsm/storage/sp.py", line 542, in validateAttachedDomain self.validatePoolSD(dom.sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py",
79, in wrapper return method(self, *args, **kwargs) File "/usr/share/vdsm/storage/sp.py", line 535, in validatePoolSD raise se.StorageDomainNotMemberOfPool(self.spUUID, sdUUID) StorageDomainNotMemberOfPool: Domain is not member in pool: u'pool=a1e7e9dd-0cf4-41ae-ba13-36297ed66309, domain=a5a6b0e7-fc3f-4838-8e26-c8b4d5e5e922' 2017-06-03 19:21:40,063+0200 ERROR (jsonrpc/2) [storage.TaskManager.Task] (Task='a2476a33-26f8-4ebd-876d-02fe5d13ef78') Unexpected error (task:870) Traceback (most recent call last): File "/usr/share/vdsm/storage/task.py", line 877, in _run return fn(*args, **kargs) File "/usr/lib/python2.7/site-packages/vdsm/logUtils.py", line 52, in wrapper res = f(*args, **kwargs) File "/usr/share/vdsm/storage/hsm.py", line 959, in createStoragePool leaseParams) File "/usr/share/vdsm/storage/sp.py", line 652, in create self.attachSD(sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py",
79, in wrapper return method(self, *args, **kwargs) File "/usr/share/vdsm/storage/sp.py", line 971, in attachSD dom.acquireHostId(self.id) File "/usr/share/vdsm/storage/sd.py", line 790, in acquireHostId self._manifest.acquireHostId(hostId, async) File "/usr/share/vdsm/storage/sd.py", line 449, in acquireHostId self._domainLock.acquireHostId(hostId, async) File "/usr/lib/python2.7/site-packages/vdsm/storage/clusterlock.py",
297, in acquireHostId raise se.AcquireHostIdFailure(self._sdUUID, e) AcquireHostIdFailure: Cannot acquire host id: (u'a5a6b0e7-fc3f-4838-8e26-c8b4d5e5e922', SanlockException(22, 'Sanlock lockspace add failure', 'Invalid argument')) 2017-06-03 19:21:40,067+0200 ERROR (jsonrpc/2) [storage.Dispatcher] {'status': {'message': "Cannot acquire host id: (u'a5a6b0e7-fc3f-4838-8e26-c8b4d5e5e922', SanlockException(22, 'Sanlock lockspace add failure', 'Invalid argument'))", 'code': 661}} (dispatcher:77)
The gluster volume prepared for engine storage is online and no split brain is reported. I don't understand what needs to be done to overcome
On Sat, Jun 3, 2017 at 8:45 PM, Abi Askushi <rightkicktech@gmail.com> wrote: partitioning i line line line line line line line line line line line this. Any
idea will be appreciated.
Thank you, Alex
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users