------=_Part_3055420_61788721.1437686254896 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit As far as I can see from the logs you removed 3 bricks. Can you confirm? Thanks in advance, Raz Tamir ratamir@redhat.com RedHat Israel RHEV-M QE Storage team ----- Original Message ----- From: "Stephen Repetski" <srepetsk@srepetsk.net> To: "Raz Tamir" <ratamir@redhat.com> Cc: "users" <users@ovirt.org> Sent: Friday, July 24, 2015 12:01:16 AM Subject: Re: [ovirt-users] oVirt not starting primary storage domain Hi Raz: I'm using vdsm-4.16.14-0.el6.x86_64 with glusterfs-3.6.2-1.el6.x86_64 on oVirt 3.5.2. I removed the brick with: gluster remove-brick store1 replica 3 $1 $2 $3 start; gluster remove-brick store1 replica 3 $1 $2 $3 commit. Between the two commands I used the 'status' option to verify that all nodes were marked as 'completed' before running the 'commit' one. Also, the two log files you requested are available here: http://srepetsk.net/files/engine.log.20150723 && http://srepetsk.net/files/etc-glusterfs-glusterd.vol.log.20150723 The gluster log file is from one of the servers from a different brick in the primary (aka "store1") datacenter/gluster volume, so it was and still is in the volume. Thanks, Stephen Stephen Repetski Rochester Institute of Technology '13 | http://srepetsk.net On Thu, Jul 23, 2015 at 4:28 PM, Raz Tamir < ratamir@redhat.com > wrote: Hi Stephen, 1) Can you please provide the vdsm and gluster versions? 2) How you removed the brick? 3) Can you please attach the glusterfs log located under /var/log ? * Just for info - there is no support for gluster if the volume is not a 3-way replica Thanks in advance, Raz Tamir ratamir@redhat.com RedHat Israel RHEV-M QE Storage team From: "Stephen Repetski" < srepetsk@srepetsk.net > To: "users" < users@ovirt.org > Sent: Thursday, July 23, 2015 11:08:57 PM Subject: [ovirt-users] oVirt not starting primary storage domain Hi all, I recently made a change with the gluster volume backing my primary storage domain (removed 1 of the 3 bricks, each w/ 3x replication), and now oVirt fails to activate the primary storage domain. After attempting to start the domain the engine goes through and does its various commications with VDSM, but then fails out with a "Sanlock resource read failure" - https://gist.githubusercontent.com/srepetsk/83ef13ddcf1e690a398e/raw/ada362a... Is there a way to figure out more on what this SpmStatusVDS error is and what might be causing it? Thanks, Stephen Stephen Repetski _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users ------=_Part_3055420_61788721.1437686254896 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable <html><body><div style=3D"font-family: trebuchet ms,sans-serif; font-size: = 12pt; color: #000000"><div>As far as I can see from the logs you removed 3 = bricks. Can you confirm?<br></div><div><br></div><div><span name=3D"x"></sp= an><div><br></div><div><br></div><div><br>Thanks in advance,<br>Raz Tamir<b= r>ratamir@redhat.com<br>RedHat Israel</div><div>RHEV-M QE Storage team<br><= /div><span name=3D"x"></span><br></div><hr id=3D"zwchr"><div style=3D"color= :#000;font-weight:normal;font-style:normal;text-decoration:none;font-family= :Helvetica,Arial,sans-serif;font-size:12pt;"><b>From: </b>"Stephen Repetski= " <srepetsk@srepetsk.net><br><b>To: </b>"Raz Tamir" <ratamir@redha= t.com><br><b>Cc: </b>"users" <users@ovirt.org><br><b>Sent: </b>Fri= day, July 24, 2015 12:01:16 AM<br><b>Subject: </b>Re: [ovirt-users] oVirt n= ot starting primary storage domain<br><div><br></div><div dir=3D"ltr">Hi Ra= z:<div><br></div><div>I'm using vdsm-4.16.14-0.el6.x86_64 with gl= usterfs-3.6.2-1.el6.x86_64 on oVirt 3.5.2.</div><div><br></div><div>I remov= ed the brick with: gluster remove-brick store1 replica 3 $1 $2 $3 start; gl= uster remove-brick store1 replica 3 $1 $2 $3 commit. Between the two comman= ds I used the 'status' option to verify that all nodes were marked as 'comp= leted' before running the 'commit' one.</div><div><br></div><div>Also, the = two log files you requested are available here:</div><div><a href=3D"http:/= /srepetsk.net/files/engine.log.20150723" target=3D"_blank">http://srepetsk.= net/files/engine.log.20150723</a> && <a href=3D"http://srepets= k.net/files/etc-glusterfs-glusterd.vol.log.20150723" target=3D"_blank">http= ://srepetsk.net/files/etc-glusterfs-glusterd.vol.log.20150723</a><br></div>= <div>The gluster log file is from one of the servers from a different brick= in the primary (aka "store1") datacenter/gluster volume, so it was and sti= ll is in the volume.</div><div><br></div><div><br></div><div>Thanks,</div><= div>Stephen</div><div><br></div></div><div class=3D"gmail_extra"><br clear= =3D"all"><div><div class=3D"gmail_signature"><div dir=3D"ltr"><b style=3D"c= olor:rgb(102,102,102)">Stephen Repetski</b><br style=3D"color:rgb(102,102,1= 02)"><span style=3D"color:rgb(102,102,102)">Rochester Institute of Technolo= gy '13 | </span><a style=3D"color:rgb(102,102,102)" href=3D"http://srepetsk= .net" target=3D"_blank">http://srepetsk.net</a><br></div></div></div> <br><div class=3D"gmail_quote">On Thu, Jul 23, 2015 at 4:28 PM, Raz Tamir <= span dir=3D"ltr"><<a href=3D"mailto:ratamir@redhat.com" target=3D"_blank= ">ratamir@redhat.com</a>></span> wrote:<br><blockquote class=3D"gmail_qu= ote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex= "><div><div style=3D"font-family:trebuchet ms,sans-serif;font-size:12pt;col= or:#000000"><div>Hi <span style=3D"font-family:Helvetica,Arial,sans-se= rif;font-size:16.3636360168457px">Stephen,</span><br></div><div><span style= =3D"font-family:Helvetica,Arial,sans-serif;font-size:16.3636360168457px">1)= Can you please provide the vdsm and gluster versions?</span></div><div><sp= an style=3D"font-family:Helvetica,Arial,sans-serif;font-size:16.36363601684= 57px">2) How you removed the brick?</span></div><div><span style=3D"font-fa= mily:Helvetica,Arial,sans-serif;font-size:16.3636360168457px">3) Can you pl= ease attach the glusterfs log located under /var/log ?</span></div><div><sp= an style=3D"font-family:Helvetica,Arial,sans-serif;font-size:16.36363601684= 57px"><br></span></div><div><span style=3D"font-family:Helvetica,Arial,sans= -serif;font-size:16.3636360168457px">* Just for info - there is no support = for gluster if the volume is not a 3-way replica</span></div><div><br></div=
<div><span></span><div><br></div><div><br></div><div><br>Thanks in advance= ,<br>Raz Tamir<br><a href=3D"mailto:ratamir@redhat.com" target=3D"_blank">r= atamir@redhat.com</a><br>RedHat Israel</div><div>RHEV-M QE Storage team<br>= </div><span></span><br></div><hr><div style=3D"color:#000;font-weight:norma= l;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-s= erif;font-size:12pt"><b>From: </b>"Stephen Repetski" <<a href=3D"mailto:= srepetsk@srepetsk.net" target=3D"_blank">srepetsk@srepetsk.net</a>><br><= b>To: </b>"users" <<a href=3D"mailto:users@ovirt.org" target=3D"_blank">= users@ovirt.org</a>><br><b>Sent: </b>Thursday, July 23, 2015 11:08:57 PM= <br><b>Subject: </b>[ovirt-users] oVirt not starting primary storage domain= <div><div class=3D"h5"><br><div><br></div><div dir=3D"ltr">Hi all,<div><br>= </div><div>I recently made a change with the gluster volume backing my prim= ary storage domain (removed 1 of the 3 bricks, each w/ 3x replication), and= now oVirt fails to activate the primary storage domain. After attempting t= o start the domain the engine goes through and does its various commication= s with VDSM, but then fails out with a "Sanlock resource read failure" -&nb= sp;<a href=3D"https://gist.githubusercontent.com/srepetsk/83ef13ddcf1e690a3= 98e/raw/ada362ac43ae71984a90979a676f2738648ac4ac/gistfile1.txt" target=3D"_= blank">https://gist.githubusercontent.com/srepetsk/83ef13ddcf1e690a398e/raw= /ada362ac43ae71984a90979a676f2738648ac4ac/gistfile1.txt</a></div><div><br><= /div><div>Is there a way to figure out more on what this SpmStatusVDS error= is and what might be causing it?</div><div><br></div><div>Thanks,</div><di= v>Stephen</div><div><br clear=3D"all"><div><div><div dir=3D"ltr"><b style= =3D"color:rgb(102,102,102)">Stephen Repetski</b><br></div></div></div> </div></div> <br></div></div>_______________________________________________<br>Users ma= iling list<br><a href=3D"mailto:Users@ovirt.org" target=3D"_blank">Users@ov= irt.org</a><br><a href=3D"http://lists.ovirt.org/mailman/listinfo/users" ta= rget=3D"_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br></div>= <div><br></div></div></div></blockquote></div><br></div> </div><div><br></div></div></body></html> ------=_Part_3055420_61788721.1437686254896--