This is a multi-part message in MIME format. --------------050507020202060701050903 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Sure that ovirt is not directly concerned but this has been done=20 following the foreman integration feature. So a warning doing this in a=20 fc environnement could be very preventive. My question was rather : now is there a possibility to recover this=20 disk? After all, it is only a LVM disk with one VG per LUN and as much=20 LV as much as existing VMs. Recovering LVM metadata might be enough? Le 21/06/2015 20:23, Dan Yasny a =C3=A9crit :
Anaconda and Kickstart are dangerous in FC environments and have been=20 known to wipe all kinds of data if zoning wasn't properly done prior=20 to os deployment to new hosts.
Nothing to do with oVirt, it's a common mistake people make at least=20 once before they step on this specific rake.
On Jun 21, 2015 2:09 PM, "Nathana=C3=ABl Blanchet" <blanchet@abes.fr=20 <mailto:blanchet@abes.fr>> wrote:
In a traditional installation of an ovirt host, it seems that provisionning the OS is the first step before talking about vdsm. I didn't go further, the problem is that this host saw the production LUN and the kickstart took all the lun into the same VG with a XFS format. Yes it is an error from myself for not having dissociated the lun. Yes it is an error from myself for not having taken care of having snapshot lun level backup. But now what should I do? My idea was to use pvreduce to unlabel the LUN, and then using the /etc/lvm/backup/lun for recovering physical volume metadata on that lun (http://www.tldp.org/HOWTO/LVM-HOWTO/recovermetadata.html)
Le 21/06/2015 12:02, mots a =C3=A9crit :
Let me get this straight:
You added a new host and instead of letting VDSM manage the VM storage you added it manually, completely independent from oVirt, during an unattended installation of the host OS? In a production environment?
Why? -----Urspr=C3=BCngliche Nachricht-----
Von:Nathana=C3=ABl Blanchet <blanchet@abes.fr <mailto:blanchet@abes.fr>> Gesendet: Fre 19 Juni 2015 14:09 An: users@ovirt.org <mailto:users@ovirt.org> Betreff: [ovirt-users] disaster recovery
Hello all,
Here is what can happen as a nightmare for a sysadmin:
I installed a new host to my pool of pre existing ovirt hosts with a kickstart file and a default partitionning. I used to do this with vms that usually get one local disk. But.. This time this host was attached to several existing ovirt (production) lun and anaconda gathered all existing disk (local and lun) into the same VG (lv_home) et formatted them with XFS.... In the webadmin, the domain storage became unvailaible, vms were still up (thankgoodness), but it was impossible to interact with them. If I stopped them, it was impossible to reboot them. If I launch lvs command, some hosts can still see the ovirt LV, but other see only the lv_home while /dev/[idofvms] are still present. So it was my chance that vms were still present, and I began to export them with a tar at file system level so as to import them in a new domain storage.
It seems that data are still present because the vms are still running. So my question is really : instead of this difficult export step, is there a way to recover the initial format of the ovirt lun so as to make the lvm index come back on the disk?
Any help or disaster experience would be much appreciated.
--=20 Nathana=C3=ABl Blanchet
Supervision r=C3=A9seau P=C3=B4le Infrastrutures Informatiques 227 avenue Professeur-Jean-Louis-Viala 34193 MONTPELLIER CEDEX 5 T=C3=A9l. 33 (0)4 67 54 84 55 Fax 33 (0)4 67 54 84 14 blanchet@abes.fr <mailto:blanchet@abes.fr>
_______________________________________________ Users mailing list Users@ovirt.org <mailto:Users@ovirt.org> http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org <mailto:Users@ovirt.org> http://lists.ovirt.org/mailman/listinfo/users
--=20 Nathana=C3=ABl Blanchet Supervision r=C3=A9seau P=C3=B4le Infrastrutures Informatiques 227 avenue Professeur-Jean-Louis-Viala 34193 MONTPELLIER CEDEX 5 =09 T=C3=A9l. 33 (0)4 67 54 84 55 Fax 33 (0)4 67 54 84 14 blanchet@abes.fr --------------050507020202060701050903 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable <html> <head> <meta content=3D"text/html; charset=3Dutf-8" http-equiv=3D"Content-Ty= pe"> </head> <body text=3D"#000000" bgcolor=3D"#FFFFFF"> Sure that ovirt is not directly concerned but this has been done following the foreman integration feature. So a warning doing this in a fc environnement could be very preventive.<br> My question was rather : now is there a possibility to recover this disk? After all, it is only a LVM disk with one VG per LUN and as much LV as much as existing VMs. Recovering LVM metadata might be enough?<br> <br> <div class=3D"moz-cite-prefix">Le 21/06/2015 20:23, Dan Yasny a =C3=A9crit=C2=A0:<br> </div> <blockquote cite=3D"mid:CALLXwb6H_apELuw=3Dh+uRA9er_Mn15Kw-46f=3DFx5VsGoTSdyJ6w@mail.= gmail.com" type=3D"cite"> <p dir=3D"ltr">Anaconda and Kickstart are dangerous in FC environments and have been known to wipe all kinds of data if zoning wasn't properly done prior to os deployment to new hosts. </p> <p dir=3D"ltr">Nothing to do with oVirt, it's a common mistake people make at least once before they step on this specific rake.</p> <div class=3D"gmail_quote">On Jun 21, 2015 2:09 PM, "Nathana=C3=ABl Blanchet" <<a moz-do-not-send=3D"true" href=3D"mailto:blanchet@abes.fr">blanchet@abes.fr</a>> wrote= :<br type=3D"attribution"> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">In a traditional installation of an ovirt host, it seems that provisionning the OS is the first step before talking about vdsm. I didn't go further, the problem is that this host saw the production LUN and the kickstart took all the lun into the same VG with a XFS format.<br> Yes it is an error from myself for not having dissociated the lun.<br> Yes it is an error from myself for not having taken care of having snapshot lun level backup.<br> But now what should I do?<br> My idea was to use pvreduce to unlabel the LUN, and then using the /etc/lvm/backup/lun for recovering physical volume metadata on that lun (<a moz-do-not-send=3D"true" href=3D"http://www.tldp.org/HOWTO/LVM-HOWTO/recovermetadata.h= tml" rel=3D"noreferrer" target=3D"_blank">http://www.tldp.org/HOWT= O/LVM-HOWTO/recovermetadata.html</a>)<br> <br> Le 21/06/2015 12:02, mots a =C3=A9crit :<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> Let me get this straight:<br> <br> You added a new host and instead of letting VDSM manage the VM storage you added it manually, completely independent from oVirt, during an unattended installation of the host OS? In a production environment?<br> <br> Why?<br> =C2=A0 -----Urspr=C3=BCngliche Nachricht-----<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> Von:Nathana=C3=ABl Blanchet <<a moz-do-not-send=3D"true" href=3D"mailto:blanchet@abes.fr" target=3D"_blank">blanch= et@abes.fr</a>><br> Gesendet: Fre 19 Juni 2015 14:09<br> An: <a moz-do-not-send=3D"true" href=3D"mailto:users@ovirt.org" target=3D"_blank">users@o= virt.org</a><br> Betreff: [ovirt-users] disaster recovery<br> <br> Hello all,<br> <br> Here is what can happen as a nightmare for a sysadmin:<br> <br> I installed a new host to my pool of pre existing ovirt hosts with a<br> kickstart file and a default partitionning. I used to do this with vms<br> that usually get one local disk.<br> But..<br> This time this host was attached to several existing ovirt (production)<br> lun and anaconda gathered all existing disk (local and lun) into the<br> same VG (lv_home) et formatted them with XFS....<br> In the webadmin, the domain storage became unvailaible, vms were still<br> up (thankgoodness), but it was impossible to interact with them. If I<br> stopped them, it was impossible to reboot them. If I launch lvs command,<br> some hosts can still see the ovirt LV, but other see only the lv_home<br> while /dev/[idofvms] are still present.<br> So it was my chance that vms were still present, and I began to export<br> them with a tar at file system level so as to import them in a new<br> domain storage.<br> <br> It seems that data are still present because the vms are still running.<br> So my question is really : instead of this difficult export step, is<br> there a way to recover the initial format of the ovirt lun so as to make<br> the lvm index come back on the disk?<br> <br> Any help or disaster experience would be much appreciated.<= br> <br> -- <br> Nathana=C3=ABl Blanchet<br> <br> Supervision r=C3=A9seau<br> P=C3=B4le Infrastrutures Informatiques<br> 227 avenue Professeur-Jean-Louis-Viala<br> 34193 MONTPELLIER CEDEX 5=C2=A0 =C2=A0 =C2=A0 =C2=A0<br> T=C3=A9l. 33 (0)4 67 54 84 55<br> Fax=C2=A0 33 (0)4 67 54 84 14<br> <a moz-do-not-send=3D"true" href=3D"mailto:blanchet@abes.fr= " target=3D"_blank">blanchet@abes.fr</a><br> <br> _______________________________________________<br> Users mailing list<br> <a moz-do-not-send=3D"true" href=3D"mailto:Users@ovirt.org" target=3D"_blank">Users@ovirt.org</a><br> <a moz-do-not-send=3D"true" href=3D"http://lists.ovirt.org/mailman/listinfo/users" rel=3D"noreferrer" target=3D"_blank">http://lists.ovirt.o= rg/mailman/listinfo/users</a><br> <br> </blockquote> </blockquote> <br> _______________________________________________<br> Users mailing list<br> <a moz-do-not-send=3D"true" href=3D"mailto:Users@ovirt.org" target=3D"_blank">Users@ovirt.org</a><br> <a moz-do-not-send=3D"true" href=3D"http://lists.ovirt.org/mailman/listinfo/users" rel=3D"noreferrer" target=3D"_blank">http://lists.ovirt.org/m= ailman/listinfo/users</a><br> </blockquote> </div> </blockquote> <br> <pre class=3D"moz-signature" cols=3D"72">--=20 Nathana=C3=ABl Blanchet Supervision r=C3=A9seau P=C3=B4le Infrastrutures Informatiques 227 avenue Professeur-Jean-Louis-Viala 34193 MONTPELLIER CEDEX 5 =09 T=C3=A9l. 33 (0)4 67 54 84 55 Fax 33 (0)4 67 54 84 14 <a class=3D"moz-txt-link-abbreviated" href=3D"mailto:blanchet@abes.fr">bl= anchet@abes.fr</a> </pre> </body> </html> --------------050507020202060701050903--