<div dir="ltr"><br><br><div class="gmail_quote"><div dir="ltr">On Sun, Aug 14, 2016 at 10:04 PM Nir Soffer <<a href="mailto:nsoffer@redhat.com">nsoffer@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Sun, Aug 14, 2016 at 8:10 PM, Siavash Safi <<a href="mailto:siavash.safi@gmail.com" target="_blank">siavash.safi@gmail.com</a>> wrote:<br>
><br>
><br>
> On Sun, Aug 14, 2016 at 8:07 PM Nir Soffer <<a href="mailto:nsoffer@redhat.com" target="_blank">nsoffer@redhat.com</a>> wrote:<br>
>><br>
>> On Sun, Aug 14, 2016 at 5:55 PM, Siavash Safi <<a href="mailto:siavash.safi@gmail.com" target="_blank">siavash.safi@gmail.com</a>><br>
>> wrote:<br>
>> > Hi,<br>
>> ><br>
>> > An unknown bug broke our gluster storage (dom_md/ids is corrupted) and<br>
>> > oVirt<br>
>> > no longer activates the storage(I tried to recover it using the similar<br>
>> > issues reported in mailing list but it didn't work).<br>
>><br>
>> Can you explain what you did?<br>
><br>
> cd /mnt/4697fbde-45fb-4f91-ac4c-5516bc59f683/dom_md/<br>
> rm ids<br>
> touch ids<br>
> sanlock direct init -s 4697fbde-45fb-4f91-ac4c-5516bc59f683:0:ids:1048576<br>
<br>
The offset parameter should be 0, not 1048576:<br>
<br>
sanlock direct init -s 4697fbde-45fb-4f91-ac4c-5516bc59f683:0:ids:0<br>
<br>
See<br>
<a href="http://lists.ovirt.org/pipermail/users/2016-February/038046.html" rel="noreferrer" target="_blank">http://lists.ovirt.org/pipermail/users/2016-February/038046.html</a><br>
<br>
Please retry this.<br>
<br></blockquote><div>I didn't know what the number at the end of the sring does ;)</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Also, are you using replica 3? These issues typically happened when people<br>
used replica 2 gluster volumes.<br></blockquote><div>Actually we removed one of the broken nodes from gluster and tried to setup local storage.</div><div>I wiped the storage and added the bricks back to gluster.</div><div><br></div><div>Thanks Nir, recreating the ids file with correct offset and resizing gluster to replica 3 fixed the issue :)</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
>> The best way to fix this is to initialize the corrupt id file and<br>
>> activate the domain.<br>
><br>
> This would be great!<br>
>><br>
>><br>
>><br>
>> > As I checked VM disk images are still accessible when I mount the<br>
>> > gluster<br>
>> > storage manually.<br>
>> > How can we manually move the VM disk images to local storage? (oVirt<br>
>> > complains about gluster storage being inactive when using the web<br>
>> > interface<br>
>> > for move/copy)<br>
>><br>
>> You can easily copy the images to another file based storage (nfs,<br>
>> gluster) like this:<br>
>><br>
>> 1. activate other storage domain using engine<br>
>> 2. mount gluster domain manually<br>
>> 3. copy the image from gluster domain to the other domain:<br>
>><br>
>> cp -r gluster-domain-mountpoint/images/image-uuid<br>
>> /rhev/data-center/mnt/server:_path/other-domain-uuid/images/<br>
>><br>
>> But the images will not be available since engine does know them.<br>
>> Maybe this can be<br>
>> fixed by modifying engine database.<br>
>><br>
> How complicated is it?<br>
<br>
I never tried this, lets try the simple way first.<br>
<br>
>> Another solution (if you are using ovirt 4.0), is to upload the images<br>
>> to a new disk,<br>
>> and attach the disk to the vm instead of the missing disk.<br>
><br>
> We are running 3.6<br>
<br>
Maybe consider an upgrade?<br>
<br>
Nir<br>
</blockquote></div></div>