<div dir="ltr"><div class="gmail_extra"><br><div class="gmail_quote">2017-07-20 11:34 GMT+02:00 Ravishankar N <span dir="ltr">&lt;<a href="mailto:ravishankar@redhat.com" target="_blank">ravishankar@redhat.com</a>&gt;</span>:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
  
    
  
  <div bgcolor="#FFFFFF"><span class="gmail-m_7883186325552308780gmail-">
    <br>
    <div class="gmail-m_7883186325552308780gmail-m_9132661499198415129moz-cite-prefix">Could you check if the self-heal daemon on all nodes is connected to
    the 3 bricks? You will need to check the glustershd.log for that.<br></div></span>
    If it is not connected, try restarting the shd using `gluster volume
    start engine force`, then launch the heal command like you did
    earlier and see if heals happen.<br>
    <br></div></blockquote><div><br></div><div>I&#39;ve executed the command on all 3 nodes (Know is enougth only one) , after that the &quot;heal&quot; command report elements between 6 and 10 ... (sometime 6, sometime 8, sometime 10)</div><div><br></div><div><br></div><div>Log on glustershd.log don&#39;t say anything :</div><div><br></div></div></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div class="gmail_extra"><div class="gmail_quote"><div><div><i>[2017-07-20 09:58:46.573079] I [MSGID: 108026] [afr-self-heal-common.c:1254:<wbr>afr_log_selfheal] 0-engine-replicate-0: Completed data selfheal on e6dfd556-340b-4b76-b47b-<wbr>7b6f5bd74327. sources=[0] 1  sinks=2</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>[2017-07-20 09:59:22.995003] I [MSGID: 108026] [afr-self-heal-metadata.c:51:_<wbr>_afr_selfheal_metadata_do] 0-engine-replicate-0: performing metadata selfheal on f05b9742-2771-484a-85fc-<wbr>5b6974bcef81</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>[2017-07-20 09:59:22.999372] I [MSGID: 108026] [afr-self-heal-common.c:1254:<wbr>afr_log_selfheal] 0-engine-replicate-0: Completed metadata selfheal on f05b9742-2771-484a-85fc-<wbr>5b6974bcef81. sources=[0] 1  sinks=2</i></div></div></div></div></blockquote><div class="gmail_extra"><div class="gmail_quote"><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF">
    If it doesn&#39;t, please provide the getfattr outputs of the 12 files
    from all 3 nodes using `getfattr -d -m . -e hex <i>/gluster/engine/brick/</i>path-to-<wbr>file`
    ?<br>
    <br></div></blockquote><div><br></div></div></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div class="gmail_extra"><div class="gmail_quote"><div><div><b><i>NODE01:</i></b></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>getfattr: Removing leading &#39;/&#39; from absolute path names</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i># file: gluster/engine/brick/.shard/<wbr>8aa74564-6740-403e-ad51-<wbr>f56d9ca5d7a7.68</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-1=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-2=<wbr>0x000000120000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.bit-rot.version=<wbr>0x090000000000000059647d5b0004<wbr>47e9</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.gfid=<wbr>0xe3565b5014954e5bae883bceca47<wbr>b7d9</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i><br></i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>getfattr: Removing leading &#39;/&#39; from absolute path names</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i># file: gluster/engine/brick/.shard/<wbr>8aa74564-6740-403e-ad51-<wbr>f56d9ca5d7a7.48</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-1=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-2=<wbr>0x0000000e0000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.bit-rot.version=<wbr>0x090000000000000059647d5b0004<wbr>47e9</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.gfid=<wbr>0x676067891f344c1586b8c0d05b07<wbr>f187</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i><br></i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>getfattr: Removing leading &#39;/&#39; from absolute path names</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i># file: gluster/engine/brick/8f215dd2-<wbr>8531-4a4f-b6ed-ea789dd8821b/<wbr>images/19d71267-52a4-42a3-<wbr>bb1e-e3145361c0c2/7a215635-<wbr>02f3-47db-80db-8b689c6a8f01</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-1=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-2=<wbr>0x000000550000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.bit-rot.version=<wbr>0x090000000000000059647d5b0004<wbr>47e9</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.gfid=<wbr>0x8aa745646740403ead51f56d9ca5<wbr>d7a7</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.glusterfs.shard.block-<wbr>size=0x0000000020000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.glusterfs.shard.file-<wbr>size=<wbr>0x0000000c80000000000000000000<wbr>00000000000000d4f2290000000000<wbr>000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i><br></i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>getfattr: Removing leading &#39;/&#39; from absolute path names</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i># file: gluster/engine/brick/.shard/<wbr>8aa74564-6740-403e-ad51-<wbr>f56d9ca5d7a7.60</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-1=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-2=<wbr>0x000000070000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.bit-rot.version=<wbr>0x090000000000000059647d5b0004<wbr>47e9</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.gfid=<wbr>0x4e33ac33dddb4e29b4a351770b81<wbr>166a</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i><br></i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>getfattr: Removing leading &#39;/&#39; from absolute path names</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i># file: gluster/engine/brick/8f215dd2-<wbr>8531-4a4f-b6ed-ea789dd8821b/<wbr>dom_md/ids</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-1=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-2=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.bit-rot.version=<wbr>0x0f0000000000000059647d5b0004<wbr>47e9</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.gfid=<wbr>0x2581cb9ac2b74bd9ac17a09bd2f0<wbr>01b3</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.glusterfs.shard.block-<wbr>size=0x0000000020000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.glusterfs.shard.file-<wbr>size=<wbr>0x0000000000100000000000000000<wbr>000000000000000008000000000000<wbr>000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i><br></i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>getfattr: Removing leading &#39;/&#39; from absolute path names</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i># file: gluster/engine/brick/__DIRECT_<wbr>IO_TEST__</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-1=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-2=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.gfid=<wbr>0xf05b97422771484a85fc5b6974bc<wbr>ef81</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.glusterfs.shard.block-<wbr>size=0x0000000020000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.glusterfs.shard.file-<wbr>size=<wbr>0x0000000000000000000000000000<wbr>000000000000000000000000000000<wbr>000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i><br></i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>getfattr: Removing leading &#39;/&#39; from absolute path names</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i># file: gluster/engine/brick/8f215dd2-<wbr>8531-4a4f-b6ed-ea789dd8821b/<wbr>images/88d41053-a257-4272-<wbr>9e2e-2f3de0743b81/6573ed08-<wbr>d3ed-4d12-9227-2c95941e1ad6</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-1=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-2=<wbr>0x000000010000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.bit-rot.version=<wbr>0x0f0000000000000059647d5b0004<wbr>47e9</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.gfid=<wbr>0xe6dfd556340b4b76b47b7b6f5bd7<wbr>4327</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.glusterfs.shard.block-<wbr>size=0x0000000020000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.glusterfs.shard.file-<wbr>size=<wbr>0x0000000000100000000000000000<wbr>000000000000000008000000000000<wbr>000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i><br></i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>getfattr: Removing leading &#39;/&#39; from absolute path names</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i># file: gluster/engine/brick/.shard/<wbr>8aa74564-6740-403e-ad51-<wbr>f56d9ca5d7a7.64</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-1=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-2=<wbr>0x0000000a0000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.bit-rot.version=<wbr>0x090000000000000059647d5b0004<wbr>47e9</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.gfid=<wbr>0x9ef88647cfe64a35a38ca5173c9e<wbr>8fc0</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><i><br></i></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><b><i><br></i></b></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><b><i>NODE02:</i></b></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>getfattr: Removing leading &#39;/&#39; from absolute path names</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i># file: gluster/engine/brick/.shard/<wbr>8aa74564-6740-403e-ad51-<wbr>f56d9ca5d7a7.68</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-0=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-2=<wbr>0x0000001a0000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.bit-rot.version=<wbr>0x08000000000000005965ede0000c<wbr>352d</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.gfid=<wbr>0xe3565b5014954e5bae883bceca47<wbr>b7d9</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i><br></i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>getfattr: Removing leading &#39;/&#39; from absolute path names</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i># file: gluster/engine/brick/.shard/<wbr>8aa74564-6740-403e-ad51-<wbr>f56d9ca5d7a7.48</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.dirty=<wbr>0x000000010000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-0=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-2=<wbr>0x0000000c0000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.bit-rot.version=<wbr>0x08000000000000005965ede0000c<wbr>352d</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.gfid=<wbr>0x676067891f344c1586b8c0d05b07<wbr>f187</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i><br></i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>getfattr: Removing leading &#39;/&#39; from absolute path names</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i># file: gluster/engine/brick/8f215dd2-<wbr>8531-4a4f-b6ed-ea789dd8821b/<wbr>images/19d71267-52a4-42a3-<wbr>bb1e-e3145361c0c2/7a215635-<wbr>02f3-47db-80db-8b689c6a8f01</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-0=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-1=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-2=<wbr>0x0000008e0000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.bit-rot.version=<wbr>0x08000000000000005965ede0000c<wbr>352d</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.gfid=<wbr>0x8aa745646740403ead51f56d9ca5<wbr>d7a7</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.glusterfs.shard.block-<wbr>size=0x0000000020000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.glusterfs.shard.file-<wbr>size=<wbr>0x0000000c80000000000000000000<wbr>00000000000000d4f2290000000000<wbr>000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i><br></i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>getfattr: Removing leading &#39;/&#39; from absolute path names</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i># file: gluster/engine/brick/.shard/<wbr>8aa74564-6740-403e-ad51-<wbr>f56d9ca5d7a7.60</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-0=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-2=<wbr>0x000000090000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.bit-rot.version=<wbr>0x08000000000000005965ede0000c<wbr>352d</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.gfid=<wbr>0x4e33ac33dddb4e29b4a351770b81<wbr>166a</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i><br></i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>getfattr: Removing leading &#39;/&#39; from absolute path names</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i># file: gluster/engine/brick/8f215dd2-<wbr>8531-4a4f-b6ed-ea789dd8821b/<wbr>dom_md/ids</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-0=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-2=<wbr>0x000000010000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.bit-rot.version=<wbr>0x08000000000000005965ede0000c<wbr>352d</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.gfid=<wbr>0x2581cb9ac2b74bd9ac17a09bd2f0<wbr>01b3</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.glusterfs.shard.block-<wbr>size=0x0000000020000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.glusterfs.shard.file-<wbr>size=<wbr>0x0000000000100000000000000000<wbr>000000000000000008000000000000<wbr>000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i><br></i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>getfattr: Removing leading &#39;/&#39; from absolute path names</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i># file: gluster/engine/brick/__DIRECT_<wbr>IO_TEST__</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-0=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-2=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.gfid=<wbr>0xf05b97422771484a85fc5b6974bc<wbr>ef81</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.glusterfs.shard.block-<wbr>size=0x0000000020000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.glusterfs.shard.file-<wbr>size=<wbr>0x0000000000000000000000000000<wbr>000000000000000000000000000000<wbr>000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i><br></i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>getfattr: Removing leading &#39;/&#39; from absolute path names</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i># file: gluster/engine/brick/8f215dd2-<wbr>8531-4a4f-b6ed-ea789dd8821b/<wbr>images/88d41053-a257-4272-<wbr>9e2e-2f3de0743b81/6573ed08-<wbr>d3ed-4d12-9227-2c95941e1ad6</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-0=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-2=<wbr>0x000000020000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.bit-rot.version=<wbr>0x08000000000000005965ede0000c<wbr>352d</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.gfid=<wbr>0xe6dfd556340b4b76b47b7b6f5bd7<wbr>4327</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.glusterfs.shard.block-<wbr>size=0x0000000020000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.glusterfs.shard.file-<wbr>size=<wbr>0x0000000000100000000000000000<wbr>000000000000000008000000000000<wbr>000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i><br></i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>getfattr: Removing leading &#39;/&#39; from absolute path names</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i># file: gluster/engine/brick/.shard/<wbr>8aa74564-6740-403e-ad51-<wbr>f56d9ca5d7a7.64</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-0=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.engine-client-2=<wbr>0x000000120000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.bit-rot.version=<wbr>0x08000000000000005965ede0000c<wbr>352d</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.gfid=<wbr>0x9ef88647cfe64a35a38ca5173c9e<wbr>8fc0</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><i><br></i></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><i><br></i></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><i><br></i></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><i><br></i></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i><b>NODE04</b>:</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>getfattr: Removing leading &#39;/&#39; from absolute path names</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i># file: gluster/engine/brick/.shard/<wbr>8aa74564-6740-403e-ad51-<wbr>f56d9ca5d7a7.68</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>security.selinux=<wbr>0x73797374656d5f753a6f626a6563<wbr>745f723a756e6c6162656c65645f74<wbr>3a733000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.bit-rot.version=<wbr>0x050000000000000059662c390006<wbr>b836</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.gfid=<wbr>0xe3565b5014954e5bae883bceca47<wbr>b7d9</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i><br></i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>getfattr: Removing leading &#39;/&#39; from absolute path names</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i># file: gluster/engine/brick/.shard/<wbr>8aa74564-6740-403e-ad51-<wbr>f56d9ca5d7a7.48</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>security.selinux=<wbr>0x73797374656d5f753a6f626a6563<wbr>745f723a756e6c6162656c65645f74<wbr>3a733000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.bit-rot.version=<wbr>0x050000000000000059662c390006<wbr>b836</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.gfid=<wbr>0x676067891f344c1586b8c0d05b07<wbr>f187</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i><br></i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>getfattr: Removing leading &#39;/&#39; from absolute path names</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i># file: gluster/engine/brick/8f215dd2-<wbr>8531-4a4f-b6ed-ea789dd8821b/<wbr>images/19d71267-52a4-42a3-<wbr>bb1e-e3145361c0c2/7a215635-<wbr>02f3-47db-80db-8b689c6a8f01</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>security.selinux=<wbr>0x73797374656d5f753a6f626a6563<wbr>745f723a756e6c6162656c65645f74<wbr>3a733000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.bit-rot.version=<wbr>0x050000000000000059662c390006<wbr>b836</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.gfid=<wbr>0x8aa745646740403ead51f56d9ca5<wbr>d7a7</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.glusterfs.shard.block-<wbr>size=0x0000000020000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.glusterfs.shard.file-<wbr>size=<wbr>0x0000000c80000000000000000000<wbr>00000000000000d4f2290000000000<wbr>000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i><br></i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>getfattr: Removing leading &#39;/&#39; from absolute path names</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i># file: gluster/engine/brick/.shard/<wbr>8aa74564-6740-403e-ad51-<wbr>f56d9ca5d7a7.60</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>security.selinux=<wbr>0x73797374656d5f753a6f626a6563<wbr>745f723a756e6c6162656c65645f74<wbr>3a733000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.bit-rot.version=<wbr>0x050000000000000059662c390006<wbr>b836</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.gfid=<wbr>0x4e33ac33dddb4e29b4a351770b81<wbr>166a</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i><br></i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>getfattr: Removing leading &#39;/&#39; from absolute path names</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i># file: gluster/engine/brick/8f215dd2-<wbr>8531-4a4f-b6ed-ea789dd8821b/<wbr>dom_md/ids</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>security.selinux=<wbr>0x73797374656d5f753a6f626a6563<wbr>745f723a756e6c6162656c65645f74<wbr>3a733000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.bit-rot.version=<wbr>0x050000000000000059662c390006<wbr>b836</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.gfid=<wbr>0x2581cb9ac2b74bd9ac17a09bd2f0<wbr>01b3</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.glusterfs.shard.block-<wbr>size=0x0000000020000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.glusterfs.shard.file-<wbr>size=<wbr>0x0000000000100000000000000000<wbr>000000000000000008000000000000<wbr>000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i><br></i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>getfattr: Removing leading &#39;/&#39; from absolute path names</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i># file: gluster/engine/brick/__DIRECT_<wbr>IO_TEST__</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>security.selinux=<wbr>0x73797374656d5f753a6f626a6563<wbr>745f723a756e6c6162656c65645f74<wbr>3a733000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.bit-rot.version=<wbr>0x0200000000000000596484e20006<wbr>237b</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.gfid=<wbr>0xf05b97422771484a85fc5b6974bc<wbr>ef81</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.glusterfs.shard.block-<wbr>size=0x0000000020000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.glusterfs.shard.file-<wbr>size=<wbr>0x0000000000000000000000000000<wbr>000000000000000000000000000000<wbr>000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i><br></i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>getfattr: Removing leading &#39;/&#39; from absolute path names</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i># file: gluster/engine/brick/8f215dd2-<wbr>8531-4a4f-b6ed-ea789dd8821b/<wbr>images/88d41053-a257-4272-<wbr>9e2e-2f3de0743b81/6573ed08-<wbr>d3ed-4d12-9227-2c95941e1ad6</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>security.selinux=<wbr>0x73797374656d5f753a6f626a6563<wbr>745f723a756e6c6162656c65645f74<wbr>3a733000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.afr.dirty=<wbr>0x000000000000000000000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.bit-rot.version=<wbr>0x050000000000000059662c390006<wbr>b836</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.gfid=<wbr>0xe6dfd556340b4b76b47b7b6f5bd7<wbr>4327</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.glusterfs.shard.block-<wbr>size=0x0000000020000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.glusterfs.shard.file-<wbr>size=<wbr>0x0000000000100000000000000000<wbr>000000000000000008000000000000<wbr>000000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i><br></i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>getfattr: Removing leading &#39;/&#39; from absolute path names</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i># file: gluster/engine/brick/.shard/<wbr>8aa74564-6740-403e-ad51-<wbr>f56d9ca5d7a7.64</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>security.selinux=<wbr>0x73797374656d5f753a6f626a6563<wbr>745f723a756e6c6162656c65645f74<wbr>3a733000</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.bit-rot.version=<wbr>0x050000000000000059662c390006<wbr>b836</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>trusted.gfid=<wbr>0x9ef88647cfe64a35a38ca5173c9e<wbr>8fc0</i></div></div></div></div></blockquote><div class="gmail_extra"><div class="gmail_quote"><div><br></div><div><br></div><div><br></div><div><br></div><div>hum.... Is selinux the problem? but on node04 was disabled (AFTER GLUSTER JOIN, I hope to remember) ... You think I needs to relabel? how?</div><div><br></div></div></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div class="gmail_extra"><div class="gmail_quote"><div><div><i>[root@node01 ~]# sestatus</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>SELinux status:                 disabled</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><i><br></i></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>[root@node02 ~]# sestatus</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>SELinux status:                 disabled</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><i><br></i></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>[root@node04 ~]# sestatus</i></div></div></div></div><div class="gmail_extra"><div class="gmail_quote"><div><div><i>SELinux status:                 disabled</i></div></div></div></div></blockquote><div class="gmail_extra"><div class="gmail_quote"><div><br></div><div><br></div><div>Thank you </div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF">
    Thanks,<br>
    Ravi<div><div class="gmail-m_7883186325552308780gmail-h5"><br>
    <br>
    <blockquote type="cite">
      <div dir="ltr">
        <div class="gmail_extra">
          <div class="gmail_quote">
            <div> </div>
            <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
              <div bgcolor="#FFFFFF"> 2. Are these 12 files also present
                in the 3rd data brick?<br>
              </div>
            </blockquote>
            <div><br>
            </div>
            <div>I&#39;ve checked right now: all files exists in all 3
              nodes </div>
            <div> </div>
            <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
              <div bgcolor="#FFFFFF"> 3. Can you provide the output of
                `gluster volume info` for the this volume?</div>
            </blockquote>
            <div><br>
            </div>
            <div><br>
            </div>
          </div>
        </div>
        <blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>Volume Name: engine</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>Type: Replicate</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>Volume ID: d19c19e3-910d-437b-8ba7-4f2a23<wbr>d17515</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>Status: Started</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>Snapshot Count: 0</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>Number of Bricks: 1 x 3 = 3</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>Transport-type: tcp</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>Bricks:</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>Brick1: node01:/gluster/engine/brick</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>Brick2: node02:/gluster/engine/brick</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>Brick3: node04:/gluster/engine/brick</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>Options Reconfigured:</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>nfs.disable: on</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>performance.readdir-ahead: on</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>transport.address-family: inet</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>storage.owner-uid: 36</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>performance.quick-read: off</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>performance.read-ahead: off</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>performance.io-cache: off</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>performance.stat-prefetch: off</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>performance.low-prio-threads: 32</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>network.remote-dio: off</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>cluster.eager-lock: enable</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>cluster.quorum-type: auto</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>cluster.server-quorum-type: server</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>cluster.data-self-heal-algorit<wbr>hm: full</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>cluster.locking-scheme: granular</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>cluster.shd-max-threads: 8</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>cluster.shd-wait-qlength: 10000</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>features.shard: on</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>user.cifs: off</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>storage.owner-gid: 36</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>features.shard-block-size: 512MB</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>network.ping-timeout: 30</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>performance.strict-o-direct: on</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>cluster.granular-entry-heal: on</i></div>
              </div>
            </div>
          </div>
          <div class="gmail_extra">
            <div class="gmail_quote">
              <div>
                <div><i>auth.allow: *</i></div>
              </div>
            </div>
          </div>
        </blockquote>
        <div class="gmail_extra">
          <div class="gmail_quote">
            <div>
              <div>          server.allow-insecure: on</div>
            </div>
            <div><br>
            </div>
            <div><br>
            </div>
            <div><br>
            </div>
            <div> </div>
            <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
              <div bgcolor="#FFFFFF"><span class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-"><br>
                  <blockquote type="cite">
                    <div class="gmail_extra">
                      <div class="gmail_quote">
                        <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                          <div dir="ltr">
                            <div>Some extra info:</div>
                            <div><br>
                            </div>
                            <div>We have recently changed the gluster
                              from: 2 (full repliacated) + 1 arbiter to
                              3 full replicated cluster </div>
                          </div>
                        </blockquote>
                      </div>
                    </div>
                  </blockquote>
                  <br>
                </span> Just curious, how did you do this?
                `remove-brick` of arbiter brick  followed by an
                `add-brick` to increase to replica-3?<br>
                <br>
              </div>
            </blockquote>
            <div><br>
            </div>
            <div>Yes</div>
            <br>
            <div>
              <div class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-nginx" style="display:block;overflow-x:auto;padding:0.5em;color:rgb(51,51,51);background:rgb(248,248,248);font-family:monospace">
                <div><br>
                </div>
                <div>
                  <div><span class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-attribute">#gluster</span>
                    volume remove-brick engine replica <span class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-number" style="color:rgb(136,0,0)">2</span>
                    node03:/gluster/data/brick force <b>(OK!)</b></div>
                  <div><br>
                  </div>
                  <div>#gluster volume heal engine <span class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-literal" style="color:rgb(120,169,96)">info</span> <b>(<span class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-literal" style="color:rgb(120,169,96)">no</span>
                      entries!)</b></div>
                  <div><br>
                  </div>
                  <div>#gluster volume add-brick engine replica <span class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-number" style="color:rgb(136,0,0)">3</span>
                    node04:/gluster/engine/brick <b>(OK!)</b></div>
                  <div><br>
                  </div>
                  <div><b>After some minutes</b></div>
                  <div><br>
                  </div>
                  <div>[root<span class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-variable" style="color:rgb(188,96,96)">@node01</span> ~]<span class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-comment" style="color:rgb(136,136,136)">#  gluster volume
                      heal engine info</span></div>
                  <div><span class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-comment" style="color:rgb(136,136,136)">Brick
                      node01:/gluster/engine/brick</span></div>
                  <div><span class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-comment" style="color:rgb(136,136,136)">Status: Connected</span></div>
                  <div><span class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-comment" style="color:rgb(136,136,136)">Number of entries:
                      0</span></div>
                  <div><br>
                  </div>
                  <div><span class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-comment" style="color:rgb(136,136,136)">Brick
                      node02:/gluster/engine/brick</span></div>
                  <div><span class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-comment" style="color:rgb(136,136,136)">Status: Connected</span></div>
                  <div><span class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-comment" style="color:rgb(136,136,136)">Number of entries:
                      0</span></div>
                  <div><br>
                  </div>
                  <div><span class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-comment" style="color:rgb(136,136,136)">Brick
                      node04:/gluster/engine/brick</span></div>
                  <div><span class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-comment" style="color:rgb(136,136,136)">Status: Connected</span></div>
                  <div><span class="gmail-m_7883186325552308780gmail-m_9132661499198415129gmail-hljs-comment" style="color:rgb(136,136,136)">Number of entries:
                      0</span></div>
                </div>
              </div>
            </div>
            <br>
            <div> </div>
            <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
              <div bgcolor="#FFFFFF"> Thanks,<br>
                Ravi</div>
            </blockquote>
            <div><br>
            </div>
            <div>Another extra info (I don&#39;t know if this can be the
              problem): Five days ago A black out has suddenly shut down
              the networks switch (also gluster network) of node 03 and
              04 ... But I don&#39;t know this problem is in place after
              this black out  </div>
            <div><br>
            </div>
            <div>Thank you!</div>
          </div>
          <div><br>
          </div>
        </div>
      </div>
    </blockquote>
    <br>
  </div></div></div>

</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail-m_7883186325552308780gmail_signature">Linux User: 369739 <a href="http://counter.li.org" target="_blank">http://counter.li.org</a></div>
</div></div>