On Wed, Jul 5, 2017 at 3:10 AM, Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
On Tue, Jul 4, 2017 at 2:57 PM, Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:

No, it's not. One option is to update glusterfs packages to 3.10.

Is it supported throughout oVirt to use CentOS Storage SIG packages instead of ovirt provided ones? I imagine you mean it, correct?

If this is a case, would I have to go with Gluster 3.9 (non LTS)

Or Gluster 3.10 (LTS)

I suppose the latter...
Any problem then with updates of oVirt itself, eg going through 4.1.2 to 4.1.3?

Thanks
Gianluca

Is 3.9 version of Gluster packages provided when updating to upcoming 4.1.3, perhaps?

Never mind, I will verify. At the end this is a test system.
I put the nodes in maintenance one by one and then installed glusterfs 3.10 with;

yum install centos-release-gluster
yum update

All were able to self heal then and I see the 4 storage domains (engine, data, iso, export) up and running.
See some notes at the end of the e-mail.
Now I'm ready to test the change of gluster network traffic.

In my case the current hostnames that are also matching the ovirtmgmt network are ovirt0N.localdomain.com with N=1,2,3

On my vlan2, defined as gluster network role in the cluster, I have defined (on each node /etc/hosts file) the hostnames

10.10.2.102 gl01.localdomain.local gl01
10.10.2.103 gl02.localdomain.local gl02
10.10.2.104 gl03.localdomain.local gl03

I need more details about command to run:

Currently I have

[root@ovirt03 ~]# gluster peer status
Number of Peers: 2

Hostname: ovirt01.localdomain.local
Uuid: e9717281-a356-42aa-a579-a4647a29a0bc
State: Peer in Cluster (Connected)
Other names:
10.10.2.102

Hostname: ovirt02.localdomain.local
Uuid: b89311fe-257f-4e44-8e15-9bff6245d689
State: Peer in Cluster (Connected)
Other names:
10.10.2.103

Suppose I start form export volume, that has these info:

[root@ovirt03 ~]# gluster volume info export
 
Volume Name: export
Type: Replicate
Volume ID: b00e5839-becb-47e7-844f-6ce6ce1b7153
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x (2 + 1) = 3
Transport-type: tcp
Bricks:
Brick1: ovirt01.localdomain.local:/gluster/brick3/export
Brick2: ovirt02.localdomain.local:/gluster/brick3/export
Brick3: ovirt03.localdomain.local:/gluster/brick3/export (arbiter)
...

then the commands I need to run would be:

gluster volume reset-brick export ovirt01.localdomain.local:/gluster/brick3/export start
gluster volume reset-brick export ovirt01.localdomain.local:/gluster/brick3/export gl01.localdomain.local:/gluster/brick3/export commit force

Correct?

Yes, correct. gl01.localdomain.local should resolve correctly on all 3 nodes.


Is it sufficient to run it on a single node? And then on the same node, to run also for the other bricks of the same volume:

Yes, it is sufficient to run on single node. You can run the reset-brick for all bricks from same node.
 

gluster volume reset-brick export ovirt02.localdomain.local:/gluster/brick3/export start
gluster volume reset-brick export ovirt02.localdomain.local:/gluster/brick3/export gl02.localdomain.local:/gluster/brick3/export commit force

and

gluster volume reset-brick export ovirt03.localdomain.local:/gluster/brick3/export start
gluster volume reset-brick export ovirt03.localdomain.local:/gluster/brick3/export gl03.localdomain.local:/gluster/brick3/export commit force

Correct? Do I have to wait self-heal after each commit command, before proceeding with the other ones?

Ideally, gluster should recognize this as same brick as before, and heal will not be needed. Please confirm that it is indeed the case before proceeding
 

Thanks in advance for input so that I can test it.

Gianluca


NOTE: during the update of gluster packages from 3.8 to 3.10 I got these:

warning: /var/lib/glusterd/vols/engine/engine.ovirt01.localdomain.local.gluster-brick1-engine.vol saved as /var/lib/glusterd/vols/engine/engine.ovirt01.localdomain.local.gluster-brick1-engine.vol.rpmsave
warning: /var/lib/glusterd/vols/engine/engine.ovirt02.localdomain.local.gluster-brick1-engine.vol saved as /var/lib/glusterd/vols/engine/engine.ovirt02.localdomain.local.gluster-brick1-engine.vol.rpmsave
warning: /var/lib/glusterd/vols/engine/engine.ovirt03.localdomain.local.gluster-brick1-engine.vol saved as /var/lib/glusterd/vols/engine/engine.ovirt03.localdomain.local.gluster-brick1-engine.vol.rpmsave
warning: /var/lib/glusterd/vols/engine/trusted-engine.tcp-fuse.vol saved as /var/lib/glusterd/vols/engine/trusted-engine.tcp-fuse.vol.rpmsave
warning: /var/lib/glusterd/vols/engine/engine.tcp-fuse.vol saved as /var/lib/glusterd/vols/engine/engine.tcp-fuse.vol.rpmsave
warning: /var/lib/glusterd/vols/data/data.ovirt01.localdomain.local.gluster-brick2-data.vol saved as /var/lib/glusterd/vols/data/data.ovirt01.localdomain.local.gluster-brick2-data.vol.rpmsave
warning: /var/lib/glusterd/vols/data/data.ovirt02.localdomain.local.gluster-brick2-data.vol saved as /var/lib/glusterd/vols/data/data.ovirt02.localdomain.local.gluster-brick2-data.vol.rpmsave
warning: /var/lib/glusterd/vols/data/data.ovirt03.localdomain.local.gluster-brick2-data.vol saved as /var/lib/glusterd/vols/data/data.ovirt03.localdomain.local.gluster-brick2-data.vol.rpmsave
warning: /var/lib/glusterd/vols/data/trusted-data.tcp-fuse.vol saved as /var/lib/glusterd/vols/data/trusted-data.tcp-fuse.vol.rpmsave
warning: /var/lib/glusterd/vols/data/data.tcp-fuse.vol saved as /var/lib/glusterd/vols/data/data.tcp-fuse.vol.rpmsave
warning: /var/lib/glusterd/vols/export/export.ovirt01.localdomain.local.gluster-brick3-export.vol saved as /var/lib/glusterd/vols/export/export.ovirt01.localdomain.local.gluster-brick3-export.vol.rpmsave
warning: /var/lib/glusterd/vols/export/export.ovirt02.localdomain.local.gluster-brick3-export.vol saved as /var/lib/glusterd/vols/export/export.ovirt02.localdomain.local.gluster-brick3-export.vol.rpmsave
warning: /var/lib/glusterd/vols/export/export.ovirt03.localdomain.local.gluster-brick3-export.vol saved as /var/lib/glusterd/vols/export/export.ovirt03.localdomain.local.gluster-brick3-export.vol.rpmsave
warning: /var/lib/glusterd/vols/export/trusted-export.tcp-fuse.vol saved as /var/lib/glusterd/vols/export/trusted-export.tcp-fuse.vol.rpmsave
warning: /var/lib/glusterd/vols/export/export.tcp-fuse.vol saved as /var/lib/glusterd/vols/export/export.tcp-fuse.vol.rpmsave
warning: /var/lib/glusterd/vols/iso/iso.ovirt01.localdomain.local.gluster-brick4-iso.vol saved as /var/lib/glusterd/vols/iso/iso.ovirt01.localdomain.local.gluster-brick4-iso.vol.rpmsave
warning: /var/lib/glusterd/vols/iso/iso.ovirt02.localdomain.local.gluster-brick4-iso.vol saved as /var/lib/glusterd/vols/iso/iso.ovirt02.localdomain.local.gluster-brick4-iso.vol.rpmsave
warning: /var/lib/glusterd/vols/iso/iso.ovirt03.localdomain.local.gluster-brick4-iso.vol saved as /var/lib/glusterd/vols/iso/iso.ovirt03.localdomain.local.gluster-brick4-iso.vol.rpmsave
warning: /var/lib/glusterd/vols/iso/trusted-iso.tcp-fuse.vol saved as /var/lib/glusterd/vols/iso/trusted-iso.tcp-fuse.vol.rpmsave
warning: /var/lib/glusterd/vols/iso/iso.tcp-fuse.vol saved as /var/lib/glusterd/vols/iso/iso.tcp-fuse.vol.rpmsave
  Installing : python2-gluster-3.10.3-1.el7.x86_64                                                             9/20
  Installing : python-prettytable-0.7.2-2.el7.centos.noarch                                                   10/20
  Updating   : glusterfs-geo-replication-3.10.3-1.el7.x86_64                                                  11/20
Warning: glusterd.service changed on disk. Run 'systemctl daemon-reload' to reload units.
  Cleanup    : glusterfs-geo-replication-3.8.13-1.el7.x86_64                                                  12/20
Warning: glusterd.service changed on disk. Run 'systemctl daemon-reload' to reload units.

For each volume the differences were these:

[root@ovirt02 engine]# diff engine.ovirt01.localdomain.local.gluster-brick1-engine.vol engine.ovirt01.localdomain.local.gluster-brick1-engine.vol.rpmsave
19,20c19,20
<     option sql-db-wal-autocheckpoint 25000
<     option sql-db-cachesize 12500
---
>     option sql-db-wal-autocheckpoint 1000
>     option sql-db-cachesize 1000
127c127
< volume engine-io-stats
---
> volume /gluster/brick1/engine
132d131
<     option unique-id /gluster/brick1/engine
136c135
< volume /gluster/brick1/engine
---
> volume engine-decompounder
138c137
<     subvolumes engine-io-stats
---
>     subvolumes /gluster/brick1/engine
149c148
<     subvolumes /gluster/brick1/engine
---
>     subvolumes engine-decompounder
[root@ovirt02 engine]#


[root@ovirt02 engine]# diff trusted-engine.tcp-fuse.vol trusted-engine.tcp-fuse.vol.rpmsave
39d38
<     option use-compound-fops off
70,72d68
<     option rda-cache-limit 10MB
<     option rda-request-size 131072
<     option parallel-readdir off
[root@ovirt02 engine]#



[root@ovirt02 engine]# diff engine.tcp-fuse.vol engine.tcp-fuse.vol.rpmsave
33d32
<     option use-compound-fops off
64,66d62
<     option rda-cache-limit 10MB
<     option rda-request-size 131072
<     option parallel-readdir off
[root@ovirt02 engine]#


The message related to glusterd service was misleading, because actually I verified that the file /usr/lib/systemd/system/glusterd.service was the same as before.