5:55 a.m.
Hello all,
It seems that oVirt 3.6 is still using FUSE to access GlusterFS storage domains instead of
using QEMU driver (libgfapi). As far as I know libgfapi support should be available in
Libvirt and QEMU packages provided in CentOS 7. Is there any workarounds how to use
libgfapi with oVirt before it’s officially available?
Best regards,
Samuli Heinonen
9:04 a.m.
On Sat, Mar 12, 2016 at 1:55 PM, Samuli Heinonen <samppah(a)neutraali.net> wrote:
Hello all,
It seems that oVirt 3.6 is still using FUSE to access GlusterFS storage domains instead
of using QEMU driver (libgfapi). As far as I know libgfapi support should be available in
Libvirt and QEMU packages provided in CentOS 7.
We started to work this during 3.6 development, but the work was
suspended because
libvirt and qemu do not support multiple gluster servers [1]. This
means that if your single
server is down, you will not be able to connect to gluster.
Recently Niels suggested that we use DNS for this purpose - if the DNS
return multiple
servers, libgafpi should be able to failover to one of these servers,
so connecting with
single server address should good as multiple server support in libvirt or qemu.
The changes needed to support this are not big as you can see in [2],
[3]. However the work
was not completed and I don't know if it will completed for 4.0.
Is there any workarounds how to use libgfapi with oVirt before it’s
officially available?
I don't know about any.
[1] https://bugzilla.redhat.com/1247521
[2] https://gerrit.ovirt.org/44061
[3] https://gerrit.ovirt.org/33768
Nir
10:14 a.m.
--6zipuvUKAEymKn2g
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Sat, Mar 12, 2016 at 05:04:16PM +0200, Nir Soffer wrote:
On Sat, Mar 12, 2016 at 1:55 PM, Samuli Heinonen
<samppah(a)neutraali.net> =
wrote:
> Hello all,
>
> It seems that oVirt 3.6 is still using FUSE to access GlusterFS storage=
domains instead of using QEMU driver (libgfapi). As far as I know libgfapi=
support should be available in Libvirt and QEMU packages provided in CentO=
S 7.
=20
We started to work this during 3.6 development, but the work was
suspended because
libvirt and qemu do not support multiple gluster servers [1]. This
means that if your single
server is down, you will not be able to connect to gluster.
=20
Recently Niels suggested that we use DNS for this purpose - if the DNS
return multiple
servers, libgafpi should be able to failover to one of these servers,
so connecting with
single server address should good as multiple server support in libvirt o=
r qemu.
And in case the local oVirt Node is part of the Gluster Trusted Storage
Pool (aka running GlusterD), qemu can use "localhost" to connect to the
storage too. It is only the initial connection that would benefit from
the added fail-over by multiple hosts. Once the connection is
established, qemu/libgfapi will connect to all the bricks that
participate in the volume. That means that only starting or attaching a
new disk to a running VM is impacted when the gluster:// URL is used
with a storage server that is down. In case oVirt/VDSM knows what
storage servers are up, it could even select one of those and not use a
server that is down.
I've left a similar note in [1], maybe it encourages to start with a
"single host" solution. Extending it for multiple hostnames should then
be pretty simple, and it allows us to start furter testing and doing
other integration bits.
And in case someone cares about (raw) sparse files (not possible over
FUSE, only with Linux 4.5), glusterfs-3.8 will provide a huge
improvement. A qemu patch for utilizing it is under review at [4].
HTH,
Niels
The changes needed to support this are not big as you can see in
[2],
[3]. However the work
was not completed and I don't know if it will completed for 4.0.
=20
> Is there any workarounds how to use libgfapi with oVirt before it=E2=80=
=99s
> officially available?
=20
I don't know about any.
=20
[1] https://bugzilla.redhat.com/1247521
[2] https://gerrit.ovirt.org/44061
[3] https://gerrit.ovirt.org/33768
[4] http://lists.nongnu.org/archive/html/qemu-block/2016-03/msg00288.html
=20
Nir
--6zipuvUKAEymKn2g
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQIcBAEBAgAGBQJW5EBTAAoJECXo5AApwsWz3LMQAM6Qp9Jh5yC1WdSPgOkvSfqS
ZPwoynbaEaUx2LDubf1q+zWd4CXR2jXk7Chpt++KvpfutLviNSg3jwBiV/kxns/H
Rp9WgzmORpuTz5uVbERPzU9aGzPXX3ImThGHyIEDfoSX/iD0/RtGV+uM8M8QLghm
VqjOCDR6CyJ+bfl5lF5tEgwz4oKp+vseYdQu9fEcEmv3AIHj/SK8m4xKEs2WqrLn
r+oK1++H9Gh+QzvNoLVZVuU0CFUZ1Wu03Nckkg6fjSY39YBLvagjkoPWekRCx6q4
XF68gR95OIhXdVfwBX3wS30skjGpWufGfDMirIT+guOunEd6oM842LXvKEtvYPvL
J64mQqAV5HzJa6l4KdXTxBIrKLcJoiIsjwEUWFBft1ofcSguWOU9J9NvRvl+mPLD
fGVUzbrQptwUTaLPDbPZrvwQpi4GaA0P41F0rWeb+jGFxz7GMxzKUBlylpxJXzzV
u2CPcToIJ2uqul0ow7kBOoE2Eoyo2bcG2FJjy6vFelUTe1VGbwhpjlnHsfNgE2BM
A/tRfEElHcdq+VylE0uiNBbjMTAC5C2KJkJcLELS5ehDJYuN4dI9i3/K+9G+9c/1
niFbFyhWCBhZoO+DhBaeuXoVi1tRXQAXT15ahUqs/dTXBNhupVZJ53rDoy6/yX5L
RpNKqteW1LY9e897/WS0
=ISoH
-----END PGP SIGNATURE-----
--6zipuvUKAEymKn2g--
3:17 p.m.
On 12 Mar 2016, at 17:04, Nir Soffer <nsoffer(a)redhat.com>
wrote:
On Sat, Mar 12, 2016 at 1:55 PM, Samuli Heinonen <samppah(a)neutraali.net> wrote:
> Hello all,
>
> It seems that oVirt 3.6 is still using FUSE to access GlusterFS storage domains
instead of using QEMU driver (libgfapi). As far as I know libgfapi support should be
available in Libvirt and QEMU packages provided in CentOS 7.
We started to work this during 3.6 development, but the work was
suspended because
libvirt and qemu do not support multiple gluster servers [1]. This
means that if your single
server is down, you will not be able to connect to glister.
Since this is only used to fetch volume information when connecting to Gluster volume I
don’t think it should be treated as blocking issue. If we lose one storage server that’s a
problem we have to fix as soon as possible anyway. Even then hypervisors are already
connected to other storage servers and there is no need to fetch volume information again.
Also there are other ways to work around this like having a separate hostname that is used
to fetch volume information and which can then be pointed to server that’s up.
It would be great if libgfapi could be available even as selectable option so it could be
tested in real world situation.
Recently Niels suggested that we use DNS for this purpose - if the
DNS
return multiple
servers, libgafpi should be able to failover to one of these servers,
so connecting with
single server address should good as multiple server support in libvirt or qemu.
The changes needed to support this are not big as you can see in [2],
[3]. However the work
was not completed and I don't know if it will completed for 4.0.
Thank you for these links. I’ll see if this is something we can try out in our test
environment
Best regards,
Samuli
> Is there any workarounds how to use libgfapi with oVirt before it’s officially
available?
I don't know about any.
[1] https://bugzilla.redhat.com/1247521
[2] https://gerrit.ovirt.org/44061
[3] https://gerrit.ovirt.org/33768
Nir
5:34 a.m.
On Sat, Mar 12, 2016 at 11:17 PM, Samuli Heinonen <samppah(a)neutraali.net> wrote:
> On 12 Mar 2016, at 17:04, Nir Soffer <nsoffer(a)redhat.com> wrote:
>
> On Sat, Mar 12, 2016 at 1:55 PM, Samuli Heinonen <samppah(a)neutraali.net>
wrote:
>> Hello all,
>>
>> It seems that oVirt 3.6 is still using FUSE to access GlusterFS storage domains
instead of using QEMU driver (libgfapi). As far as I know libgfapi support should be
available in Libvirt and QEMU packages provided in CentOS 7.
>
> We started to work this during 3.6 development, but the work was
> suspended because
> libvirt and qemu do not support multiple gluster servers [1]. This
> means that if your single
> server is down, you will not be able to connect to glister.
Since this is only used to fetch volume information when connecting to Gluster volume I
don’t think it should be treated as blocking issue. If we lose one storage server that’s a
problem we have to fix as soon as possible anyway. Even then hypervisors are already
connected to other storage servers and there is no need to fetch volume information again.
Also there are other ways to work around this like having a separate hostname that is used
to fetch volume information and which can then be pointed to server that’s up.
Multiple servers are used to connect to gluster. If you are using only
one server address
and this server is not reachable, your connection will fail, and you
will not able to start
a vm.
The solution is either manage server addresses in engine (like iscsi
targets), and pass this
to vdsm, or use special DNS name that point to all servers.
The first option is not possible yet since libvirt can accept only one
host in the disk
description.
It would be great if libgfapi could be available even as selectable option so it could be
tested in real world situation.
> Recently Niels suggested that we use DNS for this purpose - if the DNS
> return multiple
> servers, libgafpi should be able to failover to one of these servers,
> so connecting with
> single server address should good as multiple server support in libvirt or qemu.
>
> The changes needed to support this are not big as you can see in [2],
> [3]. However the work
> was not completed and I don't know if it will completed for 4.0.
Thank you for these links. I’ll see if this is something we can try out in our test
environment
The patches not complete, so I don't think they will be useful in the
current state.
They can be used as a starting point for someone who want to complete this work.
Nir
2965
days inactive
2966
days old
4 comments
3 participants
participants (3)
-
Niels de Vos -
Nir Soffer -
Samuli Heinonen