3:47 p.m.
The Power management is configured correctly. And as long as the host who
loses his storage isn't the SPM, there is no problem.
If I can make it work that, when the VM is pauzed it's get switched of and
(HA-way) reboots itself. I'm perfectly happy :-).
Kind regards,
---------- Forwarded message ----------
From: Koen Vanoppen <vanoppen.koen(a)gmail.com>
Date: 2014-04-11 14:47 GMT+02:00
Subject: Re: [ovirt-users] [Users] HA
To: Michal Skrivanek <michal.skrivanek(a)redhat.com>
The Power management is configured correctly. And as long as the host who
loses his storage isn't the SPM, there is no problem.
If I can make it work that, when the VM is pauzed it's get switched of and
(HA-way) reboots itself. I'm perfectly happy :-).
Kind regards,
2014-04-11 9:37 GMT+02:00 Michal Skrivanek <michal.skrivanek(a)redhat.com>:
On 11 Apr 2014, at 09:00, Koen Vanoppen wrote:
Hi All,
Any news about this? DSM hook or anything?
Thanx!
Kind regards
2014-04-09 9:37 GMT+02:00 Omer Frenkel <ofrenkel(a)redhat.com>:
>
>
> ----- Original Message -----
> > From: "Koen Vanoppen" <vanoppen.koen(a)gmail.com>
> > To: users(a)ovirt.org
> > Sent: Tuesday, April 8, 2014 3:41:02 PM
> > Subject: Re: [Users] HA
> >
> > Or with other words, the SPM and the VM should move almost immediate
> after
> > the storage connections on the hypervisor are gone. I know, I'm asking
> to
> > much maybe, but we would be very happy :-) :-).
> >
> > So sketch:
> >
> > Mercury1 SPM
> > Mercury 2
> >
> > Mercury1 loses both fibre connections --> goes in non-operational and
> the VM
> > goes in paused state and stays this way, until I manually reboot the
> host so
> > it fences.
> >
> > What I would like is that when mercury 1 loses both fibre connections.
> He
> > fences immediate so the VM's are moved also almost instantly... If this
> is
> > possible... :-)
> >
> > Kind regards and thanks for all the help!
> >
>
> Michal, is there a vdsm hook for vm moved to pause?
> if so, you could send KILL to it, and engine will identify vm was
> killed+HA,
> so it will be restarted, and no need to reboot the host, it will stay in
> non-operational until storage is fixed.
>
you have to differentiate - if only the VMs would be paused, yes, you can
do anything (also change the err reporting policy to not pause the VM)
but if the host becomes non-operational then it simply doesn't work, vdsm
got stuck somewhere (often in get blk device stats)
proper power management config should fence it
Thanks,
michal
> >
> >
> > 2014-04-08 14:26 GMT+02:00 Koen Vanoppen < vanoppen.koen(a)gmail.com > :
> >
> >
> >
> > Ok,
> > Thanx already for all the help. I adapted some things for quicker
> respons:
> > engine-config --get FenceQuietTimeBetweenOperationsInSec-->180
> > engine-config --set FenceQuietTimeBetweenOperationsInSec=60
> >
> > engine-config --get StorageDomainFalureTimeoutInMinutes-->180
> > engine-config --set StorageDomainFalureTimeoutInMinutes=1
> >
> > engine-config --get SpmCommandFailOverRetries-->5
> > engine-config --set SpmCommandFailOverRetries
> >
> > engine-config --get SPMFailOverAttempts-->3
> > engine-config --set SPMFailOverAttempts=1
> >
> > engine-config --get NumberOfFailedRunsOnVds-->3
> > engine-config --set NumberOfFailedRunsOnVds=1
> >
> > engine-config --get vdsTimeout-->180
> > engine-config --set vdsTimeout=30
> >
> > engine-config --get VDSAttemptsToResetCount-->2
> > engine-config --set VDSAttemptsToResetCount=1
> >
> > engine-config --get TimeoutToResetVdsInSeconds-->60
> > engine-config --set TimeoutToResetVdsInSeconds=30
> >
> > Now the result of this is that when the VM is not running on the SPM
> that it
> > will migrate before going in pause mode.
> > But when we tried it, when the vm is running on the SPM, it get's in
> paused
> > mode (for safety reasons, I know ;-) ). And stays there until the host
> gets
> > MANUALLY fenced by rebooting it. So now my question is... How can I
> make the
> > hypervisor fence (so reboots, so vm is moved) quicker?
> >
> > Kind regards,
> >
> > Koen
> >
> >
> > 2014-04-04 16:28 GMT+02:00 Koen Vanoppen < vanoppen.koen(a)gmail.com > :
> >
> >
> >
> >
> >
> > Ja das waar. Maar was aan't rijden... Dus ik stuur maar door dan :-).
> Ik heb
> > reeds de time out aangepast. Die stond op 5 min voor hij den time out
> ging
> > geven. Staat nu op 2 min
> > On Apr 4, 2014 4:14 PM, "David Van Zeebroeck" <
> > david.van.zeebroeck(a)brusselsairport.be > wrote:
> >
> >
> >
> >
> >
> >
> >
> >
> > Ik heb ze ook he
> >
> >
> >
> > Maar normaal had de fencing moeten werken als ik het zo lees
> >
> > Dus daar is ergens iets verkeerd gelopen zo te lezen
> >
> >
> >
> > From: Koen Vanoppen [mailto: vanoppen.koen(a)gmail.com ]
> > Sent: vrijdag 4 april 2014 16:07
> > To: David Van Zeebroeck
> > Subject: Fwd: Re: [Users] HA
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > David Van Zeebroeck
> >
> > Product Manager Unix Infrastructure
> >
> > Information & Communication Technology
> >
> > Brussels Airport Company
> >
> > T +32 (0)2 753 66 24
> >
> > M +32 (0)497 02 17 31
> >
> > david.van.zeebroeck(a)brusselsairport.be
> >
> >
> >
> > www.brusselsairport.be
> >
> >
> >
> >
> >
> >
> >
> > FOLLOW US ON:
> >
> >
> >
> >
> >
> >
> >
> > Company Info
> >
> >
> >
> >
> >
> >
> > ---------- Forwarded message ----------
> > From: "Michal Skrivanek" < michal.skrivanek(a)redhat.com >
> > Date: Apr 4, 2014 3:39 PM
> > Subject: Re: [Users] HA
> > To: "Koen Vanoppen" < vanoppen.koen(a)gmail.com >
> > Cc: "ovirt-users Users" < users(a)ovirt.org >
> >
> >
> >
> >
> >
> >
> > On 4 Apr 2014, at 15:14, Sander Grendelman wrote:
> >
> >
> >
> >
> >
> >
> > Do you have power management configured?
> >
> >
> > Was the "failed" host fenced/rebooted?
> >
> >
> >
> >
> >
> > On Fri, Apr 4, 2014 at 2:21 PM, Koen Vanoppen < vanoppen.koen(a)gmail.com>
> > wrote:
> >
> >
> > So... It is possible for a fully automatic migration of the VM to
> another
> > hypervisor in case Storage connection fails?
> >
> >
> > How can we make this happen? Because for the moment, when we tested the
> > situation they stayed in pause state.
> >
> >
> > (Test situation:
> >
> > * Unplug the 2 fibre cables from the hypervisor
> > * VM's go in pause state
> > * VM's stayed in pause state until the failure was solved
> >
> >
> >
> >
> >
> > as said before, it's not safe hence we (try to) not migrate them.
> >
> >
> > They only get paused when they actually access the storage which may
> not be
> > always the case. I.e. the storage connection is severed, host deemed
> > NonOperational and VMs are getting migrated from it, then some of them
> will
> > succeed if they didn't access that "bad" storage … the paused VMs
will
> > remain (mostly, it can still happen that they appear paused migrated on
> > other host when the disk access occurs only at the last stage of
> migration)
> >
> >
> >
> >
> >
> >
> >
> >
> > so in other words, if you want to migrate the VMs without interruption
> it's
> > not sometimes possible
> >
> >
> > if you are fine with the VMs restarted in short time on other host then
> power
> > management/fencing will help here
> >
> >
> >
> >
> >
> > Thanks,
> >
> >
> > michal
> >
> >
> >
> >
> >
> >
> > )
> >
> >
> >
> >
> > They only returned when we restored the fiber connection to the
> Hypervisor…
> >
> >
> >
> >
> >
> > yes, since 3.3 we have the autoresume feature
> >
> >
> >
> >
> >
> > Thanks,
> >
> >
> > michal
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Kind Regards,
> >
> > Koen
> >
> >
> >
> >
> >
> >
> >
> >
> > 2014-04-04 13:52 GMT+02:00 Koen Vanoppen < vanoppen.koen(a)gmail.com >:
> >
> >
> > So... It is possible for a fully automatic migration of the VM to
> another
> > hypervisor in case Storage connection fails?
> >
> >
> > How can we make this happen? Because for the moment, when we tested the
> > situation they stayed in pause state.
> >
> >
> > (Test situation:
> >
> > * Unplug the 2 fibre cables from the hypervisor
> > * VM's go in pause state
> > * VM's stayed in pause state until the failure was solved
> >
> >
> > )
> >
> >
> >
> >
> > They only returned when we restored the fiber connection to the
> Hypervisor...
> >
> >
> > Kind Regards,
> >
> > Koen
> >
> >
> >
> >
> >
> > 2014-04-03 16:53 GMT+02:00 Koen Vanoppen < vanoppen.koen(a)gmail.com >:
> >
> >
> >
> >
> >
> >
> >
> > ---------- Forwarded message ----------
> > From: "Doron Fediuck" < dfediuck(a)redhat.com >
> > Date: Apr 3, 2014 4:51 PM
> > Subject: Re: [Users] HA
> >
> >
> > To: "Koen Vanoppen" < vanoppen.koen(a)gmail.com >
> > Cc: "Omer Frenkel" < ofrenkel(a)redhat.com >, < users(a)ovirt.org
>,
> "Federico
> > Simoncelli" < fsimonce(a)redhat.com >, "Allon Mureinik" <
> amureini(a)redhat.com
> > >
> >
> >
> >
> > ----- Original Message -----
> > > From: "Koen Vanoppen" < vanoppen.koen(a)gmail.com >
> > > To: "Omer Frenkel" < ofrenkel(a)redhat.com >,
users(a)ovirt.org
> > > Sent: Wednesday, April 2, 2014 4:17:36 PM
> > > Subject: Re: [Users] HA
> > >
> > > Yes, indeed. I meant not-operational. Sorry.
> > > So, if I understand this correctly. When we ever come in a situation
> that
> > > we
> > > loose both storage connections on our hypervisor, we will have to
> manually
> > > restore the connections first?
> > >
> > > And thanx for the tip for speeding up thins :-).
> > >
> > > Kind regards,
> > >
> > > Koen
> > >
> > >
> > > 2014-04-02 15:14 GMT+02:00 Omer Frenkel < ofrenkel(a)redhat.com > :
> > >
> > >
> > >
> > >
> > >
> > > ----- Original Message -----
> > > > From: "Koen Vanoppen" < vanoppen.koen(a)gmail.com >
> > > > To: users(a)ovirt.org
> > > > Sent: Wednesday, April 2, 2014 4:07:19 PM
> > > > Subject: [Users] HA
> > > >
> > > > Dear All,
> > > >
> > > > Due our acceptance testing, we discovered something. (Document will
> > > > follow).
> > > > When we disable one fiber path, no problem multipath finds it way no
> > > > pings
> > > > are lost.
> > > > BUT when we disabled both the fiber paths (so one of the storage
> domain
> > > > is
> > > > gone on this host, but still available on the other host), vms go in
> > > > paused
> > > > mode... He chooses a new SPM (can we speed this up?), put's the
> host in
> > > > non-responsive (can we speed this up, more important) and the
VM's
> stay
> > > > on
> > > > Paused mode... I would expect that they would be migrated (yes, HA
> is
> > >
> > > i guess you mean the host moves to not-operational (in contrast to
> > > non-responsive)?
> > > if so, the engine will not migrate vms that are paused to do io error,
> > > because of data corruption risk.
> > >
> > > to speed up you can look at the storage domain monitoring timeout:
> > > engine-config --get StorageDomainFalureTimeoutInMinutes
> > >
> > >
> > > > enabled) to the other host and reboot there... Any solution? We are
> still
> > > > using oVirt 3.3.1 , but we are planning a upgrade to 3.4 after the
> easter
> > > > holiday.
> > > >
> > > > Kind Regards,
> > > >
> > > > Koen
> > > >
> >
> > Hi Koen,
> > Resuming from paused due to io issues is supported (adding relevant
> folks).
> > Regardless, if you did not define power management, you should manually
> > approve
> > source host was rebooted in order for migration to proceed. Otherwise
> we risk
> > split-brain scenario.
> >
> > Doron
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > _______________________________________________
> > Users mailing list
> > Users(a)ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/users
> >
> >
> >
> >
> >
> > _______________________________________________
> > Users mailing list
> > Users(a)ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/users
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > _______________________________________________
> > Users mailing list
> > Users(a)ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/users
> >
>
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
4:11 p.m.
New subject: [Users] HA
--Apple-Mail=_E20BF385-49FA-412A-A7EF-BBE6B4534F92
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=windows-1252
On 11 Apr 2014, at 14:47, Koen Vanoppen wrote:
The Power management is configured correctly. And as long as the host
=
who loses his storage isn't the SPM, there is no problem.
ah, I see
If I can make it work that, when the VM is pauzed it's get
switched of =
and (HA-way) reboots itself. I'm perfectly happy :-).
I'm not entirely sure that the after_vm_pause() hook gets invoked in =
this case. It was not intended for involuntary pause=85but give it a =
try!:)
otherwise =85.well, you can always do a periodic query=85not very =
effective though
Thanks,
michal
>=20
> Kind regards,
>=20
>=20
> ---------- Forwarded message ----------
> From: Koen Vanoppen <vanoppen.koen(a)gmail.com>
> Date: 2014-04-11 14:47 GMT+02:00
> Subject: Re: [ovirt-users] [Users] HA
> To: Michal Skrivanek <michal.skrivanek(a)redhat.com>
>=20
>=20
The Power management is configured correctly. And as long as the host
=
who loses his storage isn't the SPM, there is no problem.
If I can make it work that, when the VM is pauzed it's get
switched of =
and (HA-way) reboots itself. I'm perfectly happy :-).
=20
Kind regards,
=20
=20
=20
=20
2014-04-11 9:37 GMT+02:00 Michal Skrivanek =
<michal.skrivanek(a)redhat.com>:
=20
=20
On 11 Apr 2014, at 09:00, Koen Vanoppen wrote:
=20
> Hi All,
>=20
> Any news about this? DSM hook or anything?
> Thanx!
>=20
> Kind regards
>=20
>=20
> 2014-04-09 9:37 GMT+02:00 Omer Frenkel <ofrenkel(a)redhat.com>:
>=20
>=20
> ----- Original Message -----
> > From: "Koen Vanoppen" <vanoppen.koen(a)gmail.com>
> > To: users(a)ovirt.org
> > Sent: Tuesday, April 8, 2014 3:41:02 PM
> > Subject: Re: [Users] HA
> >
> > Or with other words, the SPM and the VM should move almost =
immediate
after
> > the storage connections on the hypervisor are gone. I know,
I'm =
asking to
> > much maybe, but we would be very happy :-) :-).
> >
> > So sketch:
> >
> > Mercury1 SPM
> > Mercury 2
> >
> > Mercury1 loses both fibre connections --> goes in non-operational =
and
the VM
> > goes in paused state and stays this way, until I manually
reboot =
the host so
> > it fences.
> >
> > What I would like is that when mercury 1 loses both fibre =
connections.
He
> > fences immediate so the VM's are moved also almost
instantly... If =
this is
> > possible... :-)
> >
> > Kind regards and thanks for all the help!
> >
>=20
> Michal, is there a vdsm hook for vm moved to pause?
> if so, you could send KILL to it, and engine will identify vm was =
killed+HA,
> so it will be restarted, and no need to reboot the host, it will
stay =
in non-operational until storage is fixed.
=20
you have to differentiate - if only the VMs would be paused, yes, you =
can do
anything (also change the err reporting policy to not pause the =
VM)
but if the host becomes non-operational then it simply doesn't
work, =
vdsm got stuck somewhere (often in get blk device stats)
proper power management config should fence it
=20
Thanks,
michal
=20
>=20
> >
> >
> > 2014-04-08 14:26 GMT+02:00 Koen Vanoppen < vanoppen.koen(a)gmail.com =
:
> >
> >
> >
> > Ok,
> > Thanx already for all the help. I adapted some things for quicker =
respons:
> > engine-config --get
FenceQuietTimeBetweenOperationsInSec-->180
> > engine-config --set FenceQuietTimeBetweenOperationsInSec=3D60
> >
> > engine-config --get StorageDomainFalureTimeoutInMinutes-->180
> > engine-config --set StorageDomainFalureTimeoutInMinutes=3D1
> >
> > engine-config --get SpmCommandFailOverRetries-->5
> > engine-config --set SpmCommandFailOverRetries
> >
> > engine-config --get SPMFailOverAttempts-->3
> > engine-config --set SPMFailOverAttempts=3D1
> >
> > engine-config --get NumberOfFailedRunsOnVds-->3
> > engine-config --set NumberOfFailedRunsOnVds=3D1
> >
> > engine-config --get vdsTimeout-->180
> > engine-config --set vdsTimeout=3D30
> >
> > engine-config --get VDSAttemptsToResetCount-->2
> > engine-config --set VDSAttemptsToResetCount=3D1
> >
> > engine-config --get TimeoutToResetVdsInSeconds-->60
> > engine-config --set TimeoutToResetVdsInSeconds=3D30
> >
> > Now the result of this is that when the VM is not running on the =
SPM
that it
> > will migrate before going in pause mode.
> > But when we tried it, when the vm is running on the SPM, it get's =
in
paused
> > mode (for safety reasons, I know ;-) ). And stays there
until the =
host gets
> > MANUALLY fenced by rebooting it. So now my question is...
How can I =
make the
> > hypervisor fence (so reboots, so vm is moved) quicker?
> >
> > Kind regards,
> >
> > Koen
> >
> >
> > 2014-04-04 16:28 GMT+02:00 Koen Vanoppen < vanoppen.koen(a)gmail.com =
:
> >
> >
> >
> >
> >
> > Ja das waar. Maar was aan't rijden... Dus ik stuur maar door dan =
:-). Ik heb
> > reeds de time out aangepast. Die stond op 5 min voor hij den
time =
out ging
> > geven. Staat nu op 2 min
> > On Apr 4, 2014 4:14 PM, "David Van Zeebroeck" <
> > david.van.zeebroeck(a)brusselsairport.be > wrote:
> >
> >
> >
> >
> >
> >
> >
> >
> > Ik heb ze ook he
> >
> >
> >
> > Maar normaal had de fencing moeten werken als ik het zo lees
> >
> > Dus daar is ergens iets verkeerd gelopen zo te lezen
> >
> >
> >
> > From: Koen Vanoppen [mailto: vanoppen.koen(a)gmail.com ]
> > Sent: vrijdag 4 april 2014 16:07
> > To: David Van Zeebroeck
> > Subject: Fwd: Re: [Users] HA
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > David Van Zeebroeck
> >
> > Product Manager Unix Infrastructure
> >
> > Information & Communication Technology
> >
> > Brussels Airport Company
> >
> > T +32 (0)2 753 66 24
> >
> > M +32 (0)497 02 17 31
> >
> > david.van.zeebroeck(a)brusselsairport.be
> >
> >
> >
> > www.brusselsairport.be
> >
> >
> >
> >
> >
> >
> >
> > FOLLOW US ON:
> >
> >
> >
> >
> >
> >
> >
> > Company Info
> >
> >
> >
> >
> >
> >
> > ---------- Forwarded message ----------
> > From: "Michal Skrivanek" < michal.skrivanek(a)redhat.com >
> > Date: Apr 4, 2014 3:39 PM
> > Subject: Re: [Users] HA
> > To: "Koen Vanoppen" < vanoppen.koen(a)gmail.com >
> > Cc: "ovirt-users Users" < users(a)ovirt.org >
> >
> >
> >
> >
> >
> >
> > On 4 Apr 2014, at 15:14, Sander Grendelman wrote:
> >
> >
> >
> >
> >
> >
> > Do you have power management configured?
> >
> >
> > Was the "failed" host fenced/rebooted?
> >
> >
> >
> >
> >
> > On Fri, Apr 4, 2014 at 2:21 PM, Koen Vanoppen < =
vanoppen.koen(a)gmail.com >
> > wrote:
> >
> >
> > So... It is possible for a fully automatic migration of the VM to =
another
> > hypervisor in case Storage connection fails?
> >
> >
> > How can we make this happen? Because for the moment, when we tested =
the
> > situation they stayed in pause state.
> >
> >
> > (Test situation:
> >
> > * Unplug the 2 fibre cables from the hypervisor
> > * VM's go in pause state
> > * VM's stayed in pause state until the failure was solved
> >
> >
> >
> >
> >
> > as said before, it's not safe hence we (try to) not migrate them.
> >
> >
> > They only get paused when they actually access the storage which =
may not
be
> > always the case. I.e. the storage connection is severed,
host =
deemed
> > NonOperational and VMs are getting migrated from it, then
some of =
them will
> > succeed if they didn't access that "bad"
storage =85 the paused VMs =
will
> > remain (mostly, it can still happen that they appear paused
=
migrated on
> > other host when the disk access occurs only at the last
stage of =
migration)
> >
> >
> >
> >
> >
> >
> >
> >
> > so in other words, if you want to migrate the VMs without =
interruption
it's
> > not sometimes possible
> >
> >
> > if you are fine with the VMs restarted in short time on other host =
then
power
> > management/fencing will help here
> >
> >
> >
> >
> >
> > Thanks,
> >
> >
> > michal
> >
> >
> >
> >
> >
> >
> > )
> >
> >
> >
> >
> > They only returned when we restored the fiber connection to the =
Hypervisor=85
> >
> >
> >
> >
> >
> > yes, since 3.3 we have the autoresume feature
> >
> >
> >
> >
> >
> > Thanks,
> >
> >
> > michal
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Kind Regards,
> >
> > Koen
> >
> >
> >
> >
> >
> >
> >
> >
> > 2014-04-04 13:52 GMT+02:00 Koen Vanoppen < vanoppen.koen(a)gmail.com =
:
> >
> >
> > So... It is possible for a fully automatic migration of the VM to =
another
> > hypervisor in case Storage connection fails?
> >
> >
> > How can we make this happen? Because for the moment, when we tested =
the
> > situation they stayed in pause state.
> >
> >
> > (Test situation:
> >
> > * Unplug the 2 fibre cables from the hypervisor
> > * VM's go in pause state
> > * VM's stayed in pause state until the failure was solved
> >
> >
> > )
> >
> >
> >
> >
> > They only returned when we restored the fiber connection to the =
Hypervisor...
> >
> >
> > Kind Regards,
> >
> > Koen
> >
> >
> >
> >
> >
> > 2014-04-03 16:53 GMT+02:00 Koen Vanoppen < vanoppen.koen(a)gmail.com =
:
> >
> >
> >
> >
> >
> >
> >
> > ---------- Forwarded message ----------
> > From: "Doron Fediuck" < dfediuck(a)redhat.com >
> > Date: Apr 3, 2014 4:51 PM
> > Subject: Re: [Users] HA
> >
> >
> > To: "Koen Vanoppen" < vanoppen.koen(a)gmail.com >
> > Cc: "Omer Frenkel" < ofrenkel(a)redhat.com >, < users(a)ovirt.org
>, =
"Federico
> > Simoncelli" < fsimonce(a)redhat.com >, "Allon
Mureinik" < =
amureini(a)redhat.com
> > >
> >
> >
> >
> > ----- Original Message -----
> > > From: "Koen Vanoppen" < vanoppen.koen(a)gmail.com >
> > > To: "Omer Frenkel" < ofrenkel(a)redhat.com >,
users(a)ovirt.org
> > > Sent: Wednesday, April 2, 2014 4:17:36 PM
> > > Subject: Re: [Users] HA
> > >
> > > Yes, indeed. I meant not-operational. Sorry.
> > > So, if I understand this correctly. When we ever come in a =
situation that
> > > we
> > > loose both storage connections on our hypervisor, we will have to =
manually
> > > restore the connections first?
> > >
> > > And thanx for the tip for speeding up thins :-).
> > >
> > > Kind regards,
> > >
> > > Koen
> > >
> > >
> > > 2014-04-02 15:14 GMT+02:00 Omer Frenkel < ofrenkel(a)redhat.com > :
> > >
> > >
> > >
> > >
> > >
> > > ----- Original Message -----
> > > > From: "Koen Vanoppen" < vanoppen.koen(a)gmail.com >
> > > > To: users(a)ovirt.org
> > > > Sent: Wednesday, April 2, 2014 4:07:19 PM
> > > > Subject: [Users] HA
> > > >
> > > > Dear All,
> > > >
> > > > Due our acceptance testing, we discovered something. (Document =
will
> > > > follow).
> > > > When we disable one fiber path, no problem multipath finds it =
way no
> > > > pings
> > > > are lost.
> > > > BUT when we disabled both the fiber paths (so one of the =
storage domain
> > > > is
> > > > gone on this host, but still available on the other host), vms =
go in
> > > > paused
> > > > mode... He chooses a new SPM (can we speed this up?), put's the =
host in
> > > > non-responsive (can we speed this up, more
important) and the =
VM's stay
> > > > on
> > > > Paused mode... I would expect that they would be migrated (yes, =
HA is
> > >
> > > i guess you mean the host moves to not-operational (in contrast =
to
> > > non-responsive)?
> > > if so, the engine will not migrate vms that are paused to do io =
error,
> > > because of data corruption risk.
> > >
> > > to speed up you can look at the storage domain monitoring =
timeout:
> > > engine-config --get
StorageDomainFalureTimeoutInMinutes
> > >
> > >
> > > > enabled) to the other host and reboot there... Any solution? We =
are still
> > > > using oVirt 3.3.1 , but we are planning a upgrade
to 3.4 after =
the easter
> > > > holiday.
> > > >
> > > > Kind Regards,
> > > >
> > > > Koen
> > > >
> >
> > Hi Koen,
> > Resuming from paused due to io issues is supported (adding relevant =
folks).
> > Regardless, if you did not define power management, you
should =
manually
> > approve
> > source host was rebooted in order for migration to proceed. =
Otherwise we
risk
> > split-brain scenario.
> >
> > Doron
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > _______________________________________________
> > Users mailing list
> > Users(a)ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/users
> >
> >
> >
> >
> >
> > _______________________________________________
> > Users mailing list
> > Users(a)ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/users
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > _______________________________________________
> > Users mailing list
> > Users(a)ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/users
> >
>=20
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
=20
=20
=20
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
--Apple-Mail=_E20BF385-49FA-412A-A7EF-BBE6B4534F92
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
charset=windows-1252
<html><head></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
"><br><div><div>On 11 Apr 2014, at 14:47, Koen Vanoppen
wrote:</div><br =
class=3D"Apple-interchange-newline"><blockquote
type=3D"cite"><div =
dir=3D"ltr"><div><div>The Power management is configured
correctly. And =
as long as the host who loses his storage isn't the SPM, there is no =
problem.<br></div></div></div></blockquote><div><br></div>ah,
I =
see</div><div><br><blockquote type=3D"cite"><div
dir=3D"ltr"><div>If I =
can make it work that, when the VM is pauzed it's get switched of and =
(HA-way) reboots itself. I'm perfectly happy =
:-).<br></div></div></blockquote><div><br></div>I'm
not entirely sure =
that the after_vm_pause() hook gets invoked in this case. It was not =
intended for involuntary pause=85but give it a =
try!:)</div><div>otherwise =85.well, you can always do a periodic =
query=85not very effective =
though</div><div><br></div><div>Thanks,</div><div>michal</div><div><br><bl=
ockquote type=3D"cite"><div dir=3D"ltr"><div>
<br></div>Kind regards,<br><br><br><div
class=3D"gmail_quote">---------- =
Forwarded message ----------<br>From: <b
class=3D"gmail_sendername">Koen =
Vanoppen</b> <span dir=3D"ltr"><<a =
href=3D"mailto:vanoppen.koen@gmail.com">vanoppen.koen@gmail.com</a>></s=
pan><br>
Date: 2014-04-11 14:47 GMT+02:00<br>Subject: Re: [ovirt-users] [Users] =
HA<br>To: Michal Skrivanek <<a =
href=3D"mailto:michal.skrivanek@redhat.com">michal.skrivanek@redhat.com</a=
><br><br><br><div
dir=3D"ltr"><div><div>The Power management is =
configured
correctly. And as long as the host who loses his storage =
isn't the SPM, there is no problem.<br>
</div>If I can make it work that, when the VM is pauzed it's get =
switched of and (HA-way) reboots itself. I'm perfectly happy :-).<br>
<br></div>Kind regards,<br><br><br></div><div =
class=3D"gmail_extra"><br><br><div
class=3D"gmail_quote">2014-04-11 9:37 =
GMT+02:00 Michal Skrivanek <span dir=3D"ltr"><<a =
href=3D"mailto:michal.skrivanek@redhat.com" =
target=3D"_blank">michal.skrivanek@redhat.com</a>></span>:<div>
<div class=3D"h5"><br>
<blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px =
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div =
style=3D"word-wrap:break-word"><br><div><div><div><div>On
11 Apr 2014, =
at 09:00, Koen Vanoppen wrote:</div>
<br><blockquote type=3D"cite"><div
dir=3D"ltr"><div><div>Hi =
All,<br><br>Any news about this? DSM hook or =
anything?<br></div>Thanx!<br><br></div>Kind
regards<br></div><div =
class=3D"gmail_extra"><br><br><div
class=3D"gmail_quote">
2014-04-09 9:37 GMT+02:00 Omer Frenkel <span dir=3D"ltr"><<a =
href=3D"mailto:ofrenkel@redhat.com" =
target=3D"_blank">ofrenkel@redhat.com</a>></span>:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px =
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex"><div><br>
<br>
----- Original Message -----<br>
> From: "Koen Vanoppen" <<a
href=3D"mailto:vanoppen.koen@gmail.com" =
target=3D"_blank">vanoppen.koen(a)gmail.com</a>&gt;<br>
> To: <a href=3D"mailto:users@ovirt.org" =
target=3D"_blank">users(a)ovirt.org</a><br>
</div><div>> Sent: Tuesday, April 8, 2014 3:41:02 PM<br>
> Subject: Re: [Users] HA<br>
><br>
</div><div>> Or with other words, the SPM and the VM should move =
almost immediate after<br>
> the storage connections on the hypervisor are gone. I know, I'm =
asking to<br>
> much maybe, but we would be very happy :-) :-).<br>
><br>
> So sketch:<br>
><br>
> Mercury1 SPM<br>
> Mercury 2<br>
><br>
> Mercury1 loses both fibre connections --> goes in =
non-operational and the VM<br>
> goes in paused state and stays this way, until I manually reboot =
the host so<br>
> it fences.<br>
><br>
> What I would like is that when mercury 1 loses both fibre =
connections. He<br>
> fences immediate so the VM's are moved also almost instantly... If =
this is<br>
> possible... :-)<br>
><br>
> Kind regards and thanks for all the help!<br>
><br>
<br>
</div>Michal, is there a vdsm hook for vm moved to pause?<br>
if so, you could send KILL to it, and engine will identify vm was =
killed+HA,<br>
so it will be restarted, and no need to reboot the host, it will stay in =
non-operational until storage is =
fixed.<br></blockquote></div></div></blockquote><div><br></div></div></div=
you have to differentiate - if only the VMs would be paused, yes, you
=
can do anything (also change the err reporting policy to not pause the =
VM)</div>
<div>but if the host becomes non-operational then it simply doesn't =
work, vdsm got stuck somewhere (often in get blk device =
stats)</div><div>proper power management config should fence =
it</div><div><br></div><div>Thanks,</div>
<div>michal</div><div><div><div><br><blockquote
type=3D"cite"><div =
class=3D"gmail_extra"><div
class=3D"gmail_quote"><blockquote =
class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px =
solid rgb(204,204,204);padding-left:1ex">
<div><div><br>
><br>
><br>
> 2014-04-08 14:26 GMT+02:00 Koen Vanoppen < <a =
href=3D"mailto:vanoppen.koen@gmail.com" =
target=3D"_blank">vanoppen.koen(a)gmail.com</a> > :<br>
><br>
><br>
><br>
> Ok,<br>
> Thanx already for all the help. I adapted some things for quicker =
respons:<br>
> engine-config --get =
FenceQuietTimeBetweenOperationsInSec-->180<br>
> engine-config --set FenceQuietTimeBetweenOperationsInSec=3D60<br>
><br>
> engine-config --get =
StorageDomainFalureTimeoutInMinutes-->180<br>
> engine-config --set StorageDomainFalureTimeoutInMinutes=3D1<br>
><br>
> engine-config --get SpmCommandFailOverRetries-->5<br>
> engine-config --set SpmCommandFailOverRetries<br>
><br>
> engine-config --get SPMFailOverAttempts-->3<br>
> engine-config --set SPMFailOverAttempts=3D1<br>
><br>
> engine-config --get NumberOfFailedRunsOnVds-->3<br>
> engine-config --set NumberOfFailedRunsOnVds=3D1<br>
><br>
> engine-config --get vdsTimeout-->180<br>
> engine-config --set vdsTimeout=3D30<br>
><br>
> engine-config --get VDSAttemptsToResetCount-->2<br>
> engine-config --set VDSAttemptsToResetCount=3D1<br>
><br>
> engine-config --get TimeoutToResetVdsInSeconds-->60<br>
> engine-config --set TimeoutToResetVdsInSeconds=3D30<br>
><br>
> Now the result of this is that when the VM is not running on the =
SPM that it<br>
> will migrate before going in pause mode.<br>
> But when we tried it, when the vm is running on the SPM, it get's =
in paused<br>
> mode (for safety reasons, I know ;-) ). And stays there until the =
host gets<br>
> MANUALLY fenced by rebooting it. So now my question is... How can I =
make the<br>
> hypervisor fence (so reboots, so vm is moved) quicker?<br>
><br>
> Kind regards,<br>
><br>
> Koen<br>
><br>
><br>
> 2014-04-04 16:28 GMT+02:00 Koen Vanoppen < <a =
href=3D"mailto:vanoppen.koen@gmail.com" =
target=3D"_blank">vanoppen.koen(a)gmail.com</a> > :<br>
><br>
><br>
><br>
><br>
><br>
> Ja das waar. Maar was aan't rijden... Dus ik stuur maar door dan =
:-). Ik heb<br>
> reeds de time out aangepast. Die stond op 5 min voor hij den time =
out ging<br>
> geven. Staat nu op 2 min<br>
> On Apr 4, 2014 4:14 PM, "David Van Zeebroeck" <<br>
> <a href=3D"mailto:david.van.zeebroeck@brusselsairport.be" =
target=3D"_blank">david.van.zeebroeck(a)brusselsairport.be</a> >
=
wrote:<br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
> Ik heb ze ook he<br>
><br>
><br>
><br>
> Maar normaal had de fencing moeten werken als ik het zo lees<br>
><br>
> Dus daar is ergens iets verkeerd gelopen zo te lezen<br>
><br>
><br>
><br>
> From: Koen Vanoppen [mailto: <a =
href=3D"mailto:vanoppen.koen@gmail.com" =
target=3D"_blank">vanoppen.koen(a)gmail.com</a> ]<br>
> Sent: vrijdag 4 april 2014 16:07<br>
> To: David Van Zeebroeck<br>
> Subject: Fwd: Re: [Users] HA<br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
> David Van Zeebroeck<br>
><br>
> Product Manager Unix Infrastructure<br>
><br>
> Information & Communication Technology<br>
><br>
> Brussels Airport Company<br>
><br>
> T <a href=3D"tel:%2B32%20%280%292%20753%2066%2024" =
value=3D"+3227536624" target=3D"_blank">+32 (0)2 753 66
24</a><br>
><br>
> M <a href=3D"tel:%2B32%20%280%29497%2002%2017%2031" =
value=3D"+32497021731" target=3D"_blank">+32 (0)497 02 17
31</a><br>
><br>
> <a href=3D"mailto:david.van.zeebroeck@brusselsairport.be" =
target=3D"_blank">david.van.zeebroeck(a)brusselsairport.be</a><br>
><br>
><br>
><br>
> <a href=3D"http://www.brusselsairport.be/" =
target=3D"_blank">www.brusselsairport.be</a><br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
</div></div>> FOLLOW US ON:<br>
<div>><br>
><br>
><br>
><br>
><br>
><br>
><br>
> Company Info<br>
><br>
><br>
><br>
><br>
><br>
><br>
> ---------- Forwarded message ----------<br>
> From: "Michal Skrivanek" < <a =
href=3D"mailto:michal.skrivanek@redhat.com" =
target=3D"_blank">michal.skrivanek(a)redhat.com</a> ><br>
> Date: Apr 4, 2014 3:39 PM<br>
> Subject: Re: [Users] HA<br>
> To: "Koen Vanoppen" < <a
href=3D"mailto:vanoppen.koen@gmail.com" =
target=3D"_blank">vanoppen.koen(a)gmail.com</a> ><br>
> Cc: "ovirt-users Users" < <a
href=3D"mailto:users@ovirt.org" =
target=3D"_blank">users(a)ovirt.org</a> ><br>
><br>
><br>
><br>
><br>
><br>
><br>
> On 4 Apr 2014, at 15:14, Sander Grendelman wrote:<br>
><br>
><br>
><br>
><br>
><br>
><br>
> Do you have power management configured?<br>
><br>
><br>
> Was the "failed" host fenced/rebooted?<br>
><br>
><br>
><br>
><br>
><br>
> On Fri, Apr 4, 2014 at 2:21 PM, Koen Vanoppen < <a =
href=3D"mailto:vanoppen.koen@gmail.com" =
target=3D"_blank">vanoppen.koen(a)gmail.com</a> ><br>
> wrote:<br>
><br>
><br>
> So... It is possible for a fully automatic migration of the VM to =
another<br>
> hypervisor in case Storage connection fails?<br>
><br>
><br>
> How can we make this happen? Because for the moment, when we tested =
the<br>
> situation they stayed in pause state.<br>
><br>
><br>
> (Test situation:<br>
><br>
</div>> * Unplug the 2 fibre cables from the =
hypervisor<br>
> * VM's go in pause state<br>
> * VM's stayed in pause state until the failure was =
solved<br>
<div><div>><br>
><br>
><br>
><br>
><br>
> as said before, it's not safe hence we (try to) not migrate =
them.<br>
><br>
><br>
> They only get paused when they actually access the storage which =
may not be<br>
> always the case. I.e. the storage connection is severed, host =
deemed<br>
> NonOperational and VMs are getting migrated from it, then some of =
them will<br>
> succeed if they didn't access that "bad" storage =85 the paused VMs
=
will<br>
> remain (mostly, it can still happen that they appear paused =
migrated on<br>
> other host when the disk access occurs only at the last stage of =
migration)<br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
> so in other words, if you want to migrate the VMs without =
interruption it's<br>
> not sometimes possible<br>
><br>
><br>
> if you are fine with the VMs restarted in short time on other host =
then power<br>
> management/fencing will help here<br>
><br>
><br>
><br>
><br>
><br>
> Thanks,<br>
><br>
><br>
> michal<br>
><br>
><br>
><br>
><br>
><br>
><br>
> )<br>
><br>
><br>
><br>
><br>
> They only returned when we restored the fiber connection to the =
Hypervisor=85<br>
><br>
><br>
><br>
><br>
><br>
> yes, since 3.3 we have the autoresume feature<br>
><br>
><br>
><br>
><br>
><br>
> Thanks,<br>
><br>
><br>
> michal<br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
> Kind Regards,<br>
><br>
> Koen<br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
> 2014-04-04 13:52 GMT+02:00 Koen Vanoppen < <a =
href=3D"mailto:vanoppen.koen@gmail.com" =
target=3D"_blank">vanoppen.koen(a)gmail.com</a> >:<br>
><br>
><br>
> So... It is possible for a fully automatic migration of the VM to =
another<br>
> hypervisor in case Storage connection fails?<br>
><br>
><br>
> How can we make this happen? Because for the moment, when we tested =
the<br>
> situation they stayed in pause state.<br>
><br>
><br>
> (Test situation:<br>
><br>
</div></div>> * Unplug the 2 fibre cables from
the =
hypervisor<br>
> * VM's go in pause state<br>
> * VM's stayed in pause state until the failure was =
solved<br>
<div><div>><br>
><br>
> )<br>
><br>
><br>
><br>
><br>
> They only returned when we restored the fiber connection to the =
Hypervisor...<br>
><br>
><br>
> Kind Regards,<br>
><br>
> Koen<br>
><br>
><br>
><br>
><br>
><br>
> 2014-04-03 16:53 GMT+02:00 Koen Vanoppen < <a =
href=3D"mailto:vanoppen.koen@gmail.com" =
target=3D"_blank">vanoppen.koen(a)gmail.com</a> >:<br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
> ---------- Forwarded message ----------<br>
> From: "Doron Fediuck" < <a
href=3D"mailto:dfediuck@redhat.com" =
target=3D"_blank">dfediuck(a)redhat.com</a> ><br>
> Date: Apr 3, 2014 4:51 PM<br>
> Subject: Re: [Users] HA<br>
><br>
><br>
> To: "Koen Vanoppen" < <a
href=3D"mailto:vanoppen.koen@gmail.com" =
target=3D"_blank">vanoppen.koen(a)gmail.com</a> ><br>
> Cc: "Omer Frenkel" < <a
href=3D"mailto:ofrenkel@redhat.com" =
target=3D"_blank">ofrenkel(a)redhat.com</a> >, < <a =
href=3D"mailto:users@ovirt.org"
target=3D"_blank">users(a)ovirt.org</a> =
>, "Federico<br>
> Simoncelli" < <a href=3D"mailto:fsimonce@redhat.com" =
target=3D"_blank">fsimonce(a)redhat.com</a> >, "Allon
Mureinik" < <a =
href=3D"mailto:amureini@redhat.com" =
target=3D"_blank">amureini(a)redhat.com</a><br>
> ><br>
><br>
><br>
><br>
> ----- Original Message -----<br>
> > From: "Koen Vanoppen" < <a =
href=3D"mailto:vanoppen.koen@gmail.com" =
target=3D"_blank">vanoppen.koen(a)gmail.com</a> ><br>
> > To: "Omer Frenkel" < <a
href=3D"mailto:ofrenkel@redhat.com" =
target=3D"_blank">ofrenkel(a)redhat.com</a> >, <a =
href=3D"mailto:users@ovirt.org"
target=3D"_blank">users(a)ovirt.org</a><br>
> > Sent: Wednesday, April 2, 2014 4:17:36 PM<br>
> > Subject: Re: [Users] HA<br>
> ><br>
> > Yes, indeed. I meant not-operational. Sorry.<br>
> > So, if I understand this correctly. When we ever come in a =
situation that<br>
> > we<br>
> > loose both storage connections on our hypervisor, we will have =
to manually<br>
> > restore the connections first?<br>
> ><br>
> > And thanx for the tip for speeding up thins :-).<br>
> ><br>
> > Kind regards,<br>
> ><br>
> > Koen<br>
> ><br>
> ><br>
> > 2014-04-02 15:14 GMT+02:00 Omer Frenkel < <a =
href=3D"mailto:ofrenkel@redhat.com" =
target=3D"_blank">ofrenkel(a)redhat.com</a> > :<br>
> ><br>
> ><br>
> ><br>
> ><br>
> ><br>
> > ----- Original Message -----<br>
> > > From: "Koen Vanoppen" < <a =
href=3D"mailto:vanoppen.koen@gmail.com" =
target=3D"_blank">vanoppen.koen(a)gmail.com</a> ><br>
> > > To: <a href=3D"mailto:users@ovirt.org" =
target=3D"_blank">users(a)ovirt.org</a><br>
> > > Sent: Wednesday, April 2, 2014 4:07:19 PM<br>
> > > Subject: [Users] HA<br>
> > ><br>
> > > Dear All,<br>
> > ><br>
> > > Due our acceptance testing, we discovered something. =
(Document will<br>
> > > follow).<br>
> > > When we disable one fiber path, no problem multipath =
finds it way no<br>
> > > pings<br>
> > > are lost.<br>
> > > BUT when we disabled both the fiber paths (so one of the =
storage domain<br>
> > > is<br>
> > > gone on this host, but still available on the other =
host), vms go in<br>
> > > paused<br>
> > > mode... He chooses a new SPM (can we speed this up?), =
put's the host in<br>
> > > non-responsive (can we speed this up, more important) and =
the VM's stay<br>
> > > on<br>
> > > Paused mode... I would expect that they would be migrated =
(yes, HA is<br>
> ><br>
> > i guess you mean the host moves to not-operational (in =
contrast to<br>
> > non-responsive)?<br>
> > if so, the engine will not migrate vms that are paused to do =
io error,<br>
> > because of data corruption risk.<br>
> ><br>
> > to speed up you can look at the storage domain monitoring =
timeout:<br>
> > engine-config --get StorageDomainFalureTimeoutInMinutes<br>
> ><br>
> ><br>
> > > enabled) to the other host and reboot there... Any =
solution? We are still<br>
> > > using oVirt 3.3.1 , but we are planning a upgrade to 3.4 =
after the easter<br>
> > > holiday.<br>
> > ><br>
> > > Kind Regards,<br>
> > ><br>
> > > Koen<br>
> > ><br>
><br>
> Hi Koen,<br>
> Resuming from paused due to io issues is supported (adding relevant =
folks).<br>
> Regardless, if you did not define power management, you should =
manually<br>
> approve<br>
> source host was rebooted in order for migration to proceed. =
Otherwise we risk<br>
> split-brain scenario.<br>
><br>
> Doron<br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
> _______________________________________________<br>
> Users mailing list<br>
> <a href=3D"mailto:Users@ovirt.org" =
target=3D"_blank">Users(a)ovirt.org</a><br>
> <a href=3D"http://lists.ovirt.org/mailman/listinfo/users" =
target=3D"_blank">http://lists.ovirt.org/mailman/listinfo/us...
><br>
><br>
><br>
><br>
><br>
> _______________________________________________<br>
> Users mailing list<br>
> <a href=3D"mailto:Users@ovirt.org" =
target=3D"_blank">Users(a)ovirt.org</a><br>
> <a href=3D"http://lists.ovirt.org/mailman/listinfo/users" =
target=3D"_blank">http://lists.ovirt.org/mailman/listinfo/us...
><br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
> _______________________________________________<br>
> Users mailing list<br>
> <a href=3D"mailto:Users@ovirt.org" =
target=3D"_blank">Users(a)ovirt.org</a><br>
> <a href=3D"http://lists.ovirt.org/mailman/listinfo/users" =
target=3D"_blank">http://lists.ovirt.org/mailman/listinfo/us...
><br>
</div></div></blockquote></div><br></div>
_______________________________________________<br>Users mailing =
list<br><a href=3D"mailto:Users@ovirt.org" =
target=3D"_blank">Users(a)ovirt.org</a><br><a =
href=3D"http://lists.ovirt.org/mailman/listinfo/users" =
target=3D"_blank">http://lists.ovirt.org/mailman/listinfo/us...
=
</blockquote></div><br></div></div></div></blockquote></div></div></div><b=
r></div>
</div><br></div>
_______________________________________________<br>Users mailing =
list<br><a =
href=3D"mailto:Users@ovirt.org">Users@ovirt.org</a><br>http://lists.ovirt.=
org/mailman/listinfo/users<br></blockquote></div><br></body></html>=
--Apple-Mail=_E20BF385-49FA-412A-A7EF-BBE6B4534F92--
3878
days inactive
3878
days old
1 comments
2 participants
participants (2)
-
Koen Vanoppen -
Michal Skrivanek