[ovirt-users] ovirt 3.6.6 and gluster 3.7.13
Scott
romracer at gmail.com
Thu Jul 21 13:18:43 UTC 2016
I get similar problems with oVirt 4.0.1 and hosted engine. After upgrading
all my hosts to Gluster 3.7.13 (client and server), I get the following:
$ sudo hosted-engine --set-maintenance --mode=none
Traceback (most recent call last):
File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_setup/set_maintenance.py",
line 73, in <module>
if not maintenance.set_mode(sys.argv[1]):
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_setup/set_maintenance.py",
line 61, in set_mode
value=m_global,
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py",
line 259, in set_maintenance_mode
str(value))
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py",
line 204, in set_global_md_flag
all_stats = broker.get_stats_from_storage(service)
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
line 232, in get_stats_from_storage
result = self._checked_communicate(request)
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
line 260, in _checked_communicate
.format(message or response))
ovirt_hosted_engine_ha.lib.exceptions.RequestError: Request failed: failed
to read metadata: [Errno 1] Operation not permitted
If I only upgrade one host, then things will continue to work but my nodes
are constantly healing shards. My logs are also flooded with:
[2016-07-21 13:15:14.137734] W [fuse-bridge.c:2227:fuse_readv_cbk]
0-glusterfs-fuse: 274714: READ => -1 gfid=4
41f2789-f6b1-4918-a280-1b9905a11429 fd=0x7f19bc0041d0 (Operation not
permitted)
The message "W [MSGID: 114031] [client-rpc-fops.c:3050:client3_3_readv_cbk]
0-data-client-0: remote operation failed [Operation not permitted]"
repeated 6 times between [2016-07-21 13:13:24.134985] and [2016-07-21
13:15:04.132226]
The message "W [MSGID: 114031] [client-rpc-fops.c:3050:client3_3_readv_cbk]
0-data-client-1: remote operation failed [Operation not permitted]"
repeated 8 times between [2016-07-21 13:13:34.133116] and [2016-07-21
13:15:14.137178]
The message "W [MSGID: 114031] [client-rpc-fops.c:3050:client3_3_readv_cbk]
0-data-client-2: remote operation failed [Operation not permitted]"
repeated 7 times between [2016-07-21 13:13:24.135071] and [2016-07-21
13:15:14.137666]
[2016-07-21 13:15:24.134647] W [MSGID: 114031]
[client-rpc-fops.c:3050:client3_3_readv_cbk] 0-data-client-0: remote
operation failed [Operation not permitted]
[2016-07-21 13:15:24.134764] W [MSGID: 114031]
[client-rpc-fops.c:3050:client3_3_readv_cbk] 0-data-client-2: remote
operation failed [Operation not permitted]
[2016-07-21 13:15:24.134793] W [fuse-bridge.c:2227:fuse_readv_cbk]
0-glusterfs-fuse: 274741: READ => -1
gfid=441f2789-f6b1-4918-a280-1b9905a11429 fd=0x7f19bc0038f4 (Operation not
permitted)
[2016-07-21 13:15:34.135413] W [fuse-bridge.c:2227:fuse_readv_cbk]
0-glusterfs-fuse: 274756: READ => -1
gfid=441f2789-f6b1-4918-a280-1b9905a11429 fd=0x7f19bc0041d0 (Operation not
permitted)
[2016-07-21 13:15:44.141062] W [fuse-bridge.c:2227:fuse_readv_cbk]
0-glusterfs-fuse: 274818: READ => -1
gfid=441f2789-f6b1-4918-a280-1b9905a11429 fd=0x7f19bc0038f4 (Operation not
permitted)
[2016-07-21 13:15:54.133582] W [MSGID: 114031]
[client-rpc-fops.c:3050:client3_3_readv_cbk] 0-data-client-1: remote
operation failed [Operation not permitted]
[2016-07-21 13:15:54.133629] W [fuse-bridge.c:2227:fuse_readv_cbk]
0-glusterfs-fuse: 274853: READ => -1
gfid=441f2789-f6b1-4918-a280-1b9905a11429 fd=0x7f19bc0036d8 (Operation not
permitted)
[2016-07-21 13:16:04.133666] W [fuse-bridge.c:2227:fuse_readv_cbk]
0-glusterfs-fuse: 274879: READ => -1
gfid=441f2789-f6b1-4918-a280-1b9905a11429 fd=0x7f19bc0041d0 (Operation not
permitted)
[2016-07-21 13:16:14.134954] W [fuse-bridge.c:2227:fuse_readv_cbk]
0-glusterfs-fuse: 274894: READ => -1
gfid=441f2789-f6b1-4918-a280-1b9905a11429 fd=0x7f19bc0036d8 (Operation not
permitted)
Scott
On Thu, Jul 21, 2016 at 6:57 AM Frank Rothenstein <
f.rothenstein at bodden-kliniken.de> wrote:
> Hey Devid,
>
> I have the very same problem on my test-cluster, despite on running ovirt
> 4.0.
> If you access your volumes via NFS all is fine, problem is FUSE. I stayed
> on 3.7.13, but have no solution yet, now I use NFS.
>
> Frank
>
> Am Donnerstag, den 21.07.2016, 04:28 -0500 schrieb David Gossage:
>
> Anyone running one of recent 3.6.x lines and gluster using 3.7.13? I am
> looking to upgrade gluster from 3.7.11->3.7.13 for some bug fixes, but have
> been told by users on gluster mail list due to some gluster changes I'd
> need to change the disk parameters to use writeback cache. Something to do
> with aio support being removed.
>
> I believe this could be done with custom parameters? But I believe strage
> tests are done using dd and would they fail with current settings then?
> Last upgrade to 3.7.13 I had to rollback to 3.7.11 due to stability isues
> where gluster storage would go into down state and always show N/A as space
> available/used. Even if hosts saw storage still and VM's were running on
> it on all 3 hosts.
>
> Saw a lot of messages like these that went away once gluster rollback
> finished
>
> [2016-07-09 15:27:46.935694] I [fuse-bridge.c:4083:fuse_init]
> 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.22 kernel
> 7.22
> [2016-07-09 15:27:49.555466] W [MSGID: 114031]
> [client-rpc-fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-1: remote
> operation failed [Operation not permitted]
> [2016-07-09 15:27:49.556574] W [MSGID: 114031]
> [client-rpc-fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-0: remote
> operation failed [Operation not permitted]
> [2016-07-09 15:27:49.556659] W [fuse-bridge.c:2227:fuse_readv_cbk]
> 0-glusterfs-fuse: 80: READ => -1 gfid=deb61291-5176-4b81-8315-3f1cf8e3534d
> fd=0x7f5224002f68 (Operation not permitted)
> [2016-07-09 15:27:59.612477] W [MSGID: 114031]
> [client-rpc-fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-1: remote
> operation failed [Operation not permitted]
> [2016-07-09 15:27:59.613700] W [MSGID: 114031]
> [client-rpc-fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-0: remote
> operation failed [Operation not permitted]
> [2016-07-09 15:27:59.613781] W [fuse-bridge.c:2227:fuse_readv_cbk]
> 0-glusterfs-fuse: 168: READ => -1 gfid=deb61291-5176-4b81-8315-3f1cf8e3534d
> fd=0x7f5224002f68 (Operation not permitted)
>
> *David Gossage*
> *Carousel Checks Inc. | System Administrator*
> *Office* 708.613.2284
>
> _______________________________________________
> Users mailing listUsers at ovirt.orghttp://lists.ovirt.org/mailman/listinfo/users
>
>
>
> ------------------------------
>
>
>
>
> ______________________________________________________________________________
> BODDEN-KLINIKEN Ribnitz-Damgarten GmbH
> Sandhufe 2
> 18311 Ribnitz-Damgarten
>
> Telefon: 03821-700-0
> Fax: 03821-700-240
>
> E-Mail: info at bodden-kliniken.de Internet: http://www.bodden-kliniken.de
>
>
> Sitz: Ribnitz-Damgarten, Amtsgericht: Stralsund, HRB 2919, Steuer-Nr.: 079/133/40188
>
> Aufsichtsratsvorsitzende: Carmen Schröter, Geschäftsführer: Dr. Falko Milski
>
>
> Der Inhalt dieser E-Mail ist ausschließlich für den bezeichneten Adressaten bestimmt. Wenn Sie nicht der vorge-
>
> sehene Adressat dieser E-Mail oder dessen Vertreter sein sollten, beachten Sie bitte, dass jede Form der Veröf-
>
> fentlichung, Vervielfältigung oder Weitergabe des Inhalts dieser E-Mail unzulässig ist. Wir bitten Sie, sofort den
> Absender zu informieren und die E-Mail zu löschen.
>
>
> Bodden-Kliniken Ribnitz-Damgarten GmbH 2016
> *** Virenfrei durch Kerio Mail Server und Sophos Antivirus ***
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20160721/b7984f70/attachment-0001.html>
More information about the Users
mailing list