5:02 a.m.
------=_Part_90_1900995351.1434880991903
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Let me get this straight:
You added a new host and instead of letting VDSM manage the VM storage you added it
manually, completely independent from oVirt, during an unattended installation of the host
OS=3F In a production environment=3F
Why=3F
=20
-----Urspr=C3=BCngliche Nachricht-----
Von:Nathana=C3=ABl Blanchet <blanchet(a)abes.fr>
Gesendet: Fre 19 Juni 2015 14:09
An: users(a)ovirt.org
Betreff: [ovirt-users] disaster recovery
=20
Hello all,
=20
Here is what can happen as a nightmare for a sysadmin:
=20
I installed a new host to my pool of pre existing ovirt hosts with a=20
kickstart file and a default partitionning. I used to do this with vms=20
that usually get one local disk.
But..
This time this host was attached to several existing ovirt (production)=20
lun and anaconda gathered all existing disk (local and lun) into the=20
same VG (lv_home) et formatted them with XFS....
In the webadmin, the domain storage became unvailaible, vms were still=20
up (thankgoodness), but it was impossible to interact with them. If I=20
stopped them, it was impossible to reboot them. If I launch lvs command,=20
some hosts can still see the ovirt LV, but other see only the lv_home=20
while /dev/[idofvms] are still present.
So it was my chance that vms were still present, and I began to export=20
them with a tar at file system level so as to import them in a new=20
domain storage.
=20
It seems that data are still present because the vms are still running.
So my question is really : instead of this difficult export step, is=20
there a way to recover the initial format of the ovirt lun so as to make=20
the lvm index come back on the disk=3F
=20
Any help or disaster experience would be much appreciated.
=20
--=20
Nathana=C3=ABl Blanchet
=20
Supervision r=C3=A9seau
P=C3=B4le Infrastrutures Informatiques
227 avenue Professeur-Jean-Louis-Viala
34193 MONTPELLIER CEDEX 5 =09
T=C3=A9l. 33 (0)4 67 54 84 55
Fax 33 (0)4 67 54 84 14
blanchet(a)abes.fr =20
=20
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
=20
------=_Part_90_1900995351.1434880991903
Content-Type: application/pgp-signature; name=signature.asc
Content-Transfer-Encoding: 7bit
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"
-----BEGIN PGP SIGNATURE-----
Version: CIPHERMAIL (2.9.0-0)
iQIcBAEBCAAGBQJVhovfAAoJEJ44dql+IcLKmV4QAIVOz1GZYMx5TM1Sd+By3J5W
9UHwzcsM11eFol8QI5iiQZaFrZvAUETTPsOlnelB9CpUkFv4auFjgsvedTRhplU8
41YdWMfV7YzFDX2Jd2Ou3kiV6SXc40P9LwM5rMAmoip8IhgND7jtue7NVkmKsuvO
HSLKb7AvG4+lFwLEuPJlRB0Xe1aSbEbrlHF+G2KhUC4i2fclHePqgBFUrbqGUuiV
p0uov92/TVuJSkahll3dWPkUGGRG9Z/gOVwZ2yUv1FnDpPns+KtwsQ0rtIWBp69o
u88ntyMw4BP+Sq0haX0kUjFDELFUw52Q0YSw6800I+hQbGQGYh3kLTm69rsVI2sr
Pqr7qkH8ELEQb+iXRaWqNV+P2onM8ni2WaX51yuHAonC+3CT31qYrDAiFCzAZ9DD
fV1hHI0a+tGLrvju4jMyplzDchAfAUaKOAyP0EUyKKcljAQcBJcCWiireYfWM4IU
39nhb8DRvBQWovUjz6nvdAERNtM+R1ZsONKQYdrgAD6AzZrIvTF8feJnMzErdHEo
HNSopYQZxv6gZkafnFG5aCA/SfvwKtzfP57hrVgr6RnOYApyHoTH05ii1iaKErWA
5FHzs1tJ7k7RnKcsswNVMijQJ4Qo5YdMyLeqh5QFWuQmEMWvrm85Uc451rAOxHJJ
hVZ22QUIBejbQpXgM2ds
=QG8e
-----END PGP SIGNATURE-----
------=_Part_90_1900995351.1434880991903--
1:09 p.m.
New subject: disaster recovery
In a traditional installation of an ovirt host, it seems that
provisionning the OS is the first step before talking about vdsm. I
didn't go further, the problem is that this host saw the production LUN
and the kickstart took all the lun into the same VG with a XFS format.
Yes it is an error from myself for not having dissociated the lun.
Yes it is an error from myself for not having taken care of having
snapshot lun level backup.
But now what should I do?
My idea was to use pvreduce to unlabel the LUN, and then using the
/etc/lvm/backup/lun for recovering physical volume metadata on that lun
(http://www.tldp.org/HOWTO/LVM-HOWTO/recovermetadata.html)
Le 21/06/2015 12:02, mots a écrit :
Let me get this straight:
You added a new host and instead of letting VDSM manage the VM storage you added it
manually, completely independent from oVirt, during an unattended installation of the host
OS? In a production environment?
Why?
-----Ursprüngliche Nachricht-----
> Von:Nathanaël Blanchet <blanchet(a)abes.fr>
> Gesendet: Fre 19 Juni 2015 14:09
> An: users(a)ovirt.org
> Betreff: [ovirt-users] disaster recovery
>
> Hello all,
>
> Here is what can happen as a nightmare for a sysadmin:
>
> I installed a new host to my pool of pre existing ovirt hosts with a
> kickstart file and a default partitionning. I used to do this with vms
> that usually get one local disk.
> But..
> This time this host was attached to several existing ovirt (production)
> lun and anaconda gathered all existing disk (local and lun) into the
> same VG (lv_home) et formatted them with XFS....
> In the webadmin, the domain storage became unvailaible, vms were still
> up (thankgoodness), but it was impossible to interact with them. If I
> stopped them, it was impossible to reboot them. If I launch lvs command,
> some hosts can still see the ovirt LV, but other see only the lv_home
> while /dev/[idofvms] are still present.
> So it was my chance that vms were still present, and I began to export
> them with a tar at file system level so as to import them in a new
> domain storage.
>
> It seems that data are still present because the vms are still running.
> So my question is really : instead of this difficult export step, is
> there a way to recover the initial format of the ovirt lun so as to make
> the lvm index come back on the disk?
>
> Any help or disaster experience would be much appreciated.
>
> --
> Nathanaël Blanchet
>
> Supervision réseau
> Pôle Infrastrutures Informatiques
> 227 avenue Professeur-Jean-Louis-Viala
> 34193 MONTPELLIER CEDEX 5
> Tél. 33 (0)4 67 54 84 55
> Fax 33 (0)4 67 54 84 14
> blanchet(a)abes.fr
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
1:23 p.m.
New subject: disaster recovery
Anaconda and Kickstart are dangerous in FC environments and have been known
to wipe all kinds of data if zoning wasn't properly done prior to os
deployment to new hosts.
Nothing to do with oVirt, it's a common mistake people make at least once
before they step on this specific rake.
On Jun 21, 2015 2:09 PM, "Nathanaël Blanchet" <blanchet(a)abes.fr> wrote:
In a traditional installation of an ovirt host, it seems that
provisionning the OS is the first step before talking about vdsm. I didn't
go further, the problem is that this host saw the production LUN and the
kickstart took all the lun into the same VG with a XFS format.
Yes it is an error from myself for not having dissociated the lun.
Yes it is an error from myself for not having taken care of having
snapshot lun level backup.
But now what should I do?
My idea was to use pvreduce to unlabel the LUN, and then using the
/etc/lvm/backup/lun for recovering physical volume metadata on that lun (
http://www.tldp.org/HOWTO/LVM-HOWTO/recovermetadata.html)
Le 21/06/2015 12:02, mots a écrit :
> Let me get this straight:
>
> You added a new host and instead of letting VDSM manage the VM storage
> you added it manually, completely independent from oVirt, during an
> unattended installation of the host OS? In a production environment?
>
> Why?
> -----Ursprüngliche Nachricht-----
>
>> Von:Nathanaël Blanchet <blanchet(a)abes.fr>
>> Gesendet: Fre 19 Juni 2015 14:09
>> An: users(a)ovirt.org
>> Betreff: [ovirt-users] disaster recovery
>>
>> Hello all,
>>
>> Here is what can happen as a nightmare for a sysadmin:
>>
>> I installed a new host to my pool of pre existing ovirt hosts with a
>> kickstart file and a default partitionning. I used to do this with vms
>> that usually get one local disk.
>> But..
>> This time this host was attached to several existing ovirt (production)
>> lun and anaconda gathered all existing disk (local and lun) into the
>> same VG (lv_home) et formatted them with XFS....
>> In the webadmin, the domain storage became unvailaible, vms were still
>> up (thankgoodness), but it was impossible to interact with them. If I
>> stopped them, it was impossible to reboot them. If I launch lvs command,
>> some hosts can still see the ovirt LV, but other see only the lv_home
>> while /dev/[idofvms] are still present.
>> So it was my chance that vms were still present, and I began to export
>> them with a tar at file system level so as to import them in a new
>> domain storage.
>>
>> It seems that data are still present because the vms are still running.
>> So my question is really : instead of this difficult export step, is
>> there a way to recover the initial format of the ovirt lun so as to make
>> the lvm index come back on the disk?
>>
>> Any help or disaster experience would be much appreciated.
>>
>> --
>> Nathanaël Blanchet
>>
>> Supervision réseau
>> Pôle Infrastrutures Informatiques
>> 227 avenue Professeur-Jean-Louis-Viala
>> 34193 MONTPELLIER CEDEX 5
>> Tél. 33 (0)4 67 54 84 55
>> Fax 33 (0)4 67 54 84 14
>> blanchet(a)abes.fr
>>
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>>
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
1:26 a.m.
New subject: disaster recovery
This is a multi-part message in MIME format.
--------------050507020202060701050903
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable
Sure that ovirt is not directly concerned but this has been done=20
following the foreman integration feature. So a warning doing this in a=20
fc environnement could be very preventive.
My question was rather : now is there a possibility to recover this=20
disk? After all, it is only a LVM disk with one VG per LUN and as much=20
LV as much as existing VMs. Recovering LVM metadata might be enough?
Le 21/06/2015 20:23, Dan Yasny a =C3=A9crit :
Anaconda and Kickstart are dangerous in FC environments and have been=20
known to wipe all kinds of data if zoning wasn't properly done prior=20
to os deployment to new hosts.
Nothing to do with oVirt, it's a common mistake people make at least=20
once before they step on this specific rake.
On Jun 21, 2015 2:09 PM, "Nathana=C3=ABl Blanchet" <blanchet(a)abes.fr=20
<mailto:blanchet@abes.fr>> wrote:
In a traditional installation of an ovirt host, it seems that
provisionning the OS is the first step before talking about vdsm.
I didn't go further, the problem is that this host saw the
production LUN and the kickstart took all the lun into the same VG
with a XFS format.
Yes it is an error from myself for not having dissociated the lun.
Yes it is an error from myself for not having taken care of having
snapshot lun level backup.
But now what should I do?
My idea was to use pvreduce to unlabel the LUN, and then using the
/etc/lvm/backup/lun for recovering physical volume metadata on
that lun (http://www.tldp.org/HOWTO/LVM-HOWTO/recovermetadata.html)
Le 21/06/2015 12:02, mots a =C3=A9crit :
Let me get this straight:
You added a new host and instead of letting VDSM manage the VM
storage you added it manually, completely independent from
oVirt, during an unattended installation of the host OS? In a
production environment?
Why?
-----Urspr=C3=BCngliche Nachricht-----
Von:Nathana=C3=ABl Blanchet <blanchet(a)abes.fr
<mailto:blanchet@abes.fr>>
Gesendet: Fre 19 Juni 2015 14:09
An: users(a)ovirt.org <mailto:users@ovirt.org>
Betreff: [ovirt-users] disaster recovery
Hello all,
Here is what can happen as a nightmare for a sysadmin:
I installed a new host to my pool of pre existing ovirt
hosts with a
kickstart file and a default partitionning. I used to do
this with vms
that usually get one local disk.
But..
This time this host was attached to several existing ovirt
(production)
lun and anaconda gathered all existing disk (local and
lun) into the
same VG (lv_home) et formatted them with XFS....
In the webadmin, the domain storage became unvailaible,
vms were still
up (thankgoodness), but it was impossible to interact with
them. If I
stopped them, it was impossible to reboot them. If I
launch lvs command,
some hosts can still see the ovirt LV, but other see only
the lv_home
while /dev/[idofvms] are still present.
So it was my chance that vms were still present, and I
began to export
them with a tar at file system level so as to import them
in a new
domain storage.
It seems that data are still present because the vms are
still running.
So my question is really : instead of this difficult
export step, is
there a way to recover the initial format of the ovirt lun
so as to make
the lvm index come back on the disk?
Any help or disaster experience would be much appreciated.
--=20
Nathana=C3=ABl Blanchet
Supervision r=C3=A9seau
P=C3=B4le Infrastrutures Informatiques
227 avenue Professeur-Jean-Louis-Viala
34193 MONTPELLIER CEDEX 5
T=C3=A9l. 33 (0)4 67 54 84 55
Fax 33 (0)4 67 54 84 14
blanchet(a)abes.fr <mailto:blanchet@abes.fr>
_______________________________________________
Users mailing list
Users(a)ovirt.org <mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users(a)ovirt.org <mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users
--=20
Nathana=C3=ABl Blanchet
Supervision r=C3=A9seau
P=C3=B4le Infrastrutures Informatiques
227 avenue Professeur-Jean-Louis-Viala
34193 MONTPELLIER CEDEX 5 =09
T=C3=A9l. 33 (0)4 67 54 84 55
Fax 33 (0)4 67 54 84 14
blanchet(a)abes.fr
--------------050507020202060701050903
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable
<html>
<head>
<meta content=3D"text/html; charset=3Dutf-8"
http-equiv=3D"Content-Ty=
pe">
</head>
<body text=3D"#000000" bgcolor=3D"#FFFFFF">
Sure that ovirt is not directly concerned but this has been done
following the foreman integration feature. So a warning doing this
in a fc environnement could be very preventive.<br>
My question was rather : now is there a possibility to recover this
disk? After all, it is only a LVM disk with one VG per LUN and as
much LV as much as existing VMs. Recovering LVM metadata might be
enough?<br>
<br>
<div class=3D"moz-cite-prefix">Le 21/06/2015 20:23, Dan Yasny a
=C3=A9crit=C2=A0:<br>
</div>
<blockquote
cite=3D"mid:CALLXwb6H_apELuw=3Dh+uRA9er_Mn15Kw-46f=3DFx5VsGoTSdyJ6w@mail.=
gmail.com"
type=3D"cite">
<p dir=3D"ltr">Anaconda and Kickstart are dangerous in FC
environments and have been known to wipe all kinds of data if
zoning wasn't properly done prior to os deployment to new hosts.
</p>
<p dir=3D"ltr">Nothing to do with oVirt, it's a common mistake
people make at least once before they step on this specific
rake.</p>
<div class=3D"gmail_quote">On Jun 21, 2015 2:09 PM,
"Nathana=C3=ABl
Blanchet" <<a moz-do-not-send=3D"true"
href=3D"mailto:blanchet@abes.fr">blanchet@abes.fr</a>>
wrote=
:<br
type=3D"attribution">
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">In a
traditional installation of an ovirt host, it seems that
provisionning the OS is the first step before talking about
vdsm. I didn't go further, the problem is that this host saw
the production LUN and the kickstart took all the lun into the
same VG with a XFS format.<br>
Yes it is an error from myself for not having dissociated the
lun.<br>
Yes it is an error from myself for not having taken care of
having snapshot lun level backup.<br>
But now what should I do?<br>
My idea was to use pvreduce to unlabel the LUN, and then using
the /etc/lvm/backup/lun for recovering physical volume
metadata on that lun (<a moz-do-not-send=3D"true"
href=3D"http://www.tldp.org/HOWTO/LVM-HOWTO/recovermetadata.h=
tml"
rel=3D"noreferrer"
target=3D"_blank">http://www.tldp.org/HOWT=
O/LVM-HOWTO/recovermetadata.html</a>)<br>
<br>
Le 21/06/2015 12:02, mots a =C3=A9crit :<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
Let me get this straight:<br>
<br>
You added a new host and instead of letting VDSM manage the
VM storage you added it manually, completely independent
from oVirt, during an unattended installation of the host
OS? In a production environment?<br>
<br>
Why?<br>
=C2=A0 -----Urspr=C3=BCngliche Nachricht-----<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
Von:Nathana=C3=ABl Blanchet <<a
moz-do-not-send=3D"true"
href=3D"mailto:blanchet@abes.fr"
target=3D"_blank">blanch=
et(a)abes.fr</a>&gt;<br>
Gesendet: Fre 19 Juni 2015 14:09<br>
An: <a moz-do-not-send=3D"true"
href=3D"mailto:users@ovirt.org"
target=3D"_blank">users@o=
virt.org</a><br>
Betreff: [ovirt-users] disaster recovery<br>
<br>
Hello all,<br>
<br>
Here is what can happen as a nightmare for a sysadmin:<br>
<br>
I installed a new host to my pool of pre existing ovirt
hosts with a<br>
kickstart file and a default partitionning. I used to do
this with vms<br>
that usually get one local disk.<br>
But..<br>
This time this host was attached to several existing ovirt
(production)<br>
lun and anaconda gathered all existing disk (local and
lun) into the<br>
same VG (lv_home) et formatted them with XFS....<br>
In the webadmin, the domain storage became unvailaible,
vms were still<br>
up (thankgoodness), but it was impossible to interact with
them. If I<br>
stopped them, it was impossible to reboot them. If I
launch lvs command,<br>
some hosts can still see the ovirt LV, but other see only
the lv_home<br>
while /dev/[idofvms] are still present.<br>
So it was my chance that vms were still present, and I
began to export<br>
them with a tar at file system level so as to import them
in a new<br>
domain storage.<br>
<br>
It seems that data are still present because the vms are
still running.<br>
So my question is really : instead of this difficult
export step, is<br>
there a way to recover the initial format of the ovirt lun
so as to make<br>
the lvm index come back on the disk?<br>
<br>
Any help or disaster experience would be much appreciated.<=
br>
<br>
-- <br>
Nathana=C3=ABl Blanchet<br>
<br>
Supervision r=C3=A9seau<br>
P=C3=B4le Infrastrutures Informatiques<br>
227 avenue Professeur-Jean-Louis-Viala<br>
34193 MONTPELLIER CEDEX 5=C2=A0 =C2=A0 =C2=A0 =C2=A0<br>
T=C3=A9l. 33 (0)4 67 54 84 55<br>
Fax=C2=A0 33 (0)4 67 54 84 14<br>
<a moz-do-not-send=3D"true"
href=3D"mailto:blanchet@abes.fr=
"
target=3D"_blank">blanchet(a)abes.fr</a><br>
<br>
_______________________________________________<br>
Users mailing list<br>
<a moz-do-not-send=3D"true"
href=3D"mailto:Users@ovirt.org"
target=3D"_blank">Users(a)ovirt.org</a><br>
<a moz-do-not-send=3D"true"
href=3D"http://lists.ovirt.org/mailman/listinfo/users"
rel=3D"noreferrer"
target=3D"_blank">http://lists.ovirt.o=
rg/mailman/listinfo/users</a><br>
<br>
</blockquote>
</blockquote>
<br>
_______________________________________________<br>
Users mailing list<br>
<a moz-do-not-send=3D"true"
href=3D"mailto:Users@ovirt.org"
target=3D"_blank">Users(a)ovirt.org</a><br>
<a moz-do-not-send=3D"true"
href=3D"http://lists.ovirt.org/mailman/listinfo/users"
rel=3D"noreferrer"
target=3D"_blank">http://lists.ovirt.org/m=
ailman/listinfo/users</a><br>
</blockquote>
</div>
</blockquote>
<br>
<pre class=3D"moz-signature" cols=3D"72">--=20
Nathana=C3=ABl Blanchet
Supervision r=C3=A9seau
P=C3=B4le Infrastrutures Informatiques
227 avenue Professeur-Jean-Louis-Viala
34193 MONTPELLIER CEDEX 5 =09
T=C3=A9l. 33 (0)4 67 54 84 55
Fax 33 (0)4 67 54 84 14
<a class=3D"moz-txt-link-abbreviated"
href=3D"mailto:blanchet@abes.fr">bl=
anchet(a)abes.fr</a> </pre>
</body>
</html>
--------------050507020202060701050903--
3470
days inactive
3471
days old
3 comments
3 participants
participants (3)
-
Dan Yasny -
mots -
Nathanaël Blanchet