10:25 a.m.
On Fri, Nov 29, 2013 at 11:48 AM, Dan Kenigsberg wrote:
On Fri, Nov 29, 2013 at 04:04:03PM +0530, Vijay Bellur wrote:
>
> There are two ways in which GlusterFS can be used as a storage domain:
>
> a) Use gluster native/fuse access with POSIXFS
>
> b) Use the gluster native storage domain to bypass fuse (with
> libgfapi). We are currently addressing an issue in libvirt
> (https://bugzilla.redhat.com/show_bug.cgi?id=1017289) to enable
> snapshot support with libgfapi. Once this is addressed, we will have
> libgfapi support in the native storage domain.
It won't be as immediate, since there's a required fix on Vdsm's side
(Bug 1022961 - Running a VM from a gluster domain uses mount instead of
gluster URI)
> Till then, fuse would
> be used with native storage domain. You can find more details about
> native storage domain here:
>
> http://www.ovirt.org/Features/GlusterFS_Storage_Domain
rg/mailman/listinfo/users
Hello,
revamping here....
I'm using oVirt 3.3.3 beta1 after upgrade from 3.3.2 on fedora 19
ovirt beta repo
It seems bug referred by Vijay (1017289) is still marked as
"assigned", but actually it is for rhel 6.
Bug referred by Dan (1022961) is marked as "blocked" but I don't see
any particular updates since late november. But it is for rhevm, so I
think it is for rhel 6...
So what is the situation for fedora 19 and oVirt in upcoming 3.3.3?
And upcoming fedora 19/20 and 3.4?
I ask because I see that in my qemu command line generated by oVirt there is:
for virtio (virtio-blk) disk
-drive
file=/rhev/data-center/mnt/glusterSD/node01.mydomain:gvdata/d0b96d4a-62aa-4e9f-b50e-f7a0cb5be291/images/a5e4f67b-50b5-4740-9990-39deb8812445/53408cb0-bcd4-40de-bc69-89d59b7b5bc2,if=none,id=drive-virtio-disk0,format=raw,serial=a5e4f67b-50b5-4740-9990-39deb8812445,cache=none,werror=stop,rerror=stop,aio=threads
for virtio-scsi disk
-drive
file=/rhev/data-center/mnt/glusterSD/node01.mydomain:gvdata/d0b96d4a-62aa-4e9f-b50e-f7a0cb5be291/images/c1477133-6b06-480d-a233-1dae08daf8b3/c2a82c64-9dee-42bb-acf2-65b8081f2edf,if=none,id=drive-scsi0-0-0-0,format=raw,serial=c1477133-6b06-480d-a233-1dae08daf8b3,cache=none,werror=stop,rerror=stop,aio=threads
So it is referred as mount point and not gluster:// way
Also, the output of "mount" command on hypervisors shows:
node01.mydomain:gvdata on
/rhev/data-center/mnt/glusterSD/node01.mydomain:gvdata type
fuse.glusterfs
(rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)
and it seems indeed fuse mount, so not using libgfapi...
output of
"lsof -Pp pid" where pid is the qemu process shows libgfapi....:
qemu-syst 2057 qemu mem REG 253,0 99440
541417 /usr/lib64/libgfapi.so.0.0.0
(btw: strange version 0.0.0 for a release.. not so reassuring ;-)
# ll /usr/lib64/libgfapi.so.0*
lrwxrwxrwx. 1 root root 17 Jan 7 12:45 /usr/lib64/libgfapi.so.0 ->
libgfapi.so.0.0.0
-rwxr-xr-x. 1 root root 99440 Jan 3 13:35 /usr/lib64/libgfapi.so.0.0.0
)
At page http://www.gluster.org/category/qemu/ there is a schema about
mount types and benchmarks
1) FUSE mount
2) GlusterFS block driver in QEMU (FUSE bypass)
3) Base (VM image accessed directly from brick)
(
qemu-system-x86_64 –enable-kvm –nographic -smp 4 -m 2048 -drive
file=/test/F17,if=virtio,cache=none => /test is brick directory
)
I have not understood if we are in Base (best performance) or FUSE
(worst performance)
thanks in advance for clarifications and possible roadmaps...
Gianluca
12:45 p.m.
New subject: [Users] Gluster storage
On Fri, Jan 17, 2014 at 08:25:05AM +0100, Gianluca Cecchi wrote:
On Fri, Nov 29, 2013 at 11:48 AM, Dan Kenigsberg wrote:
> On Fri, Nov 29, 2013 at 04:04:03PM +0530, Vijay Bellur wrote:
>
>>
>> There are two ways in which GlusterFS can be used as a storage domain:
>>
>> a) Use gluster native/fuse access with POSIXFS
>>
>> b) Use the gluster native storage domain to bypass fuse (with
>> libgfapi). We are currently addressing an issue in libvirt
>> (https://bugzilla.redhat.com/show_bug.cgi?id=1017289) to enable
>> snapshot support with libgfapi. Once this is addressed, we will have
>> libgfapi support in the native storage domain.
>
> It won't be as immediate, since there's a required fix on Vdsm's side
> (Bug 1022961 - Running a VM from a gluster domain uses mount instead of
> gluster URI)
>
>> Till then, fuse would
>> be used with native storage domain. You can find more details about
>> native storage domain here:
>>
>> http://www.ovirt.org/Features/GlusterFS_Storage_Domain
rg/mailman/listinfo/users
Hello,
revamping here....
I'm using oVirt 3.3.3 beta1 after upgrade from 3.3.2 on fedora 19
ovirt beta repo
It seems bug referred by Vijay (1017289) is still marked as
"assigned", but actually it is for rhel 6.
Bug referred by Dan (1022961) is marked as "blocked" but I don't see
any particular updates since late november. But it is for rhevm, so I
think it is for rhel 6...
So what is the situation for fedora 19 and oVirt in upcoming 3.3.3?
And upcoming fedora 19/20 and 3.4?
Neither bug has been fixed as of yet, I'm afraid.
You should seem them change state to POST when there's code to be seen.
Dan.
12:58 p.m.
New subject: [Users] Gluster storage
On 01/17/2014 12:55 PM, Gianluca Cecchi wrote:
On Fri, Nov 29, 2013 at 11:48 AM, Dan Kenigsberg wrote:
> On Fri, Nov 29, 2013 at 04:04:03PM +0530, Vijay Bellur wrote:
>
>>
>> There are two ways in which GlusterFS can be used as a storage domain:
>>
>> a) Use gluster native/fuse access with POSIXFS
>>
>> b) Use the gluster native storage domain to bypass fuse (with
>> libgfapi). We are currently addressing an issue in libvirt
>> (https://bugzilla.redhat.com/show_bug.cgi?id=1017289) to enable
>> snapshot support with libgfapi. Once this is addressed, we will have
>> libgfapi support in the native storage domain.
>
> It won't be as immediate, since there's a required fix on Vdsm's side
> (Bug 1022961 - Running a VM from a gluster domain uses mount instead of
> gluster URI)
>
>> Till then, fuse would
>> be used with native storage domain. You can find more details about
>> native storage domain here:
>>
>> http://www.ovirt.org/Features/GlusterFS_Storage_Domain
rg/mailman/listinfo/users
Hello,
revamping here....
I'm using oVirt 3.3.3 beta1 after upgrade from 3.3.2 on fedora 19
ovirt beta repo
It seems bug referred by Vijay (1017289) is still marked as
"assigned", but actually it is for rhel 6.
Bug referred by Dan (1022961) is marked as "blocked" but I don't see
any particular updates since late november. But it is for rhevm, so I
think it is for rhel 6...
So what is the situation for fedora 19 and oVirt in upcoming 3.3.3?
And upcoming fedora 19/20 and 3.4?
I ask because I see that in my qemu command line generated by oVirt there is:
for virtio (virtio-blk) disk
-drive
file=/rhev/data-center/mnt/glusterSD/node01.mydomain:gvdata/d0b96d4a-62aa-4e9f-b50e-f7a0cb5be291/images/a5e4f67b-50b5-4740-9990-39deb8812445/53408cb0-bcd4-40de-bc69-89d59b7b5bc2,if=none,id=drive-virtio-disk0,format=raw,serial=a5e4f67b-50b5-4740-9990-39deb8812445,cache=none,werror=stop,rerror=stop,aio=threads
for virtio-scsi disk
-drive
file=/rhev/data-center/mnt/glusterSD/node01.mydomain:gvdata/d0b96d4a-62aa-4e9f-b50e-f7a0cb5be291/images/c1477133-6b06-480d-a233-1dae08daf8b3/c2a82c64-9dee-42bb-acf2-65b8081f2edf,if=none,id=drive-scsi0-0-0-0,format=raw,serial=c1477133-6b06-480d-a233-1dae08daf8b3,cache=none,werror=stop,rerror=stop,aio=threads
So it is referred as mount point and not gluster:// way
Also, the output of "mount" command on hypervisors shows:
node01.mydomain:gvdata on
/rhev/data-center/mnt/glusterSD/node01.mydomain:gvdata type
fuse.glusterfs
(rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)
and it seems indeed fuse mount, so not using libgfapi...
output of
"lsof -Pp pid" where pid is the qemu process shows libgfapi....:
qemu-syst 2057 qemu mem REG 253,0 99440
541417 /usr/lib64/libgfapi.so.0.0.0
(btw: strange version 0.0.0 for a release.. not so reassuring ;-)
libgfapi uses a different internal versioning scheme and hence
compatibility can be handled. However there is work happening in
glusterfs upstream to change the version scheme for libraries and have
it compatible with libtool guidelines.
# ll /usr/lib64/libgfapi.so.0*
lrwxrwxrwx. 1 root root 17 Jan 7 12:45 /usr/lib64/libgfapi.so.0 ->
libgfapi.so.0.0.0
-rwxr-xr-x. 1 root root 99440 Jan 3 13:35 /usr/lib64/libgfapi.so.0.0.0
)
At page http://www.gluster.org/category/qemu/ there is a schema about
mount types and benchmarks
1) FUSE mount
2) GlusterFS block driver in QEMU (FUSE bypass)
3) Base (VM image accessed directly from brick)
(
qemu-system-x86_64 –enable-kvm –nographic -smp 4 -m 2048 -drive
file=/test/F17,if=virtio,cache=none => /test is brick directory
)
I have not understood if we are in Base (best performance) or FUSE
(worst performance)
In this particular blog post, base refers to local disk storage and the
FUSE performance was measured before a set of optimizations were done in
GlusterFS. The "Optimize for Virt" action in oVirt ensures that these
optimizations are enabled. Hence, even with FUSE the performance for VM
image storage would be somewhere between the worst and best cases.
thanks in advance for clarifications and possible roadmaps...
If you are interested in seeing any new feature related to the
oVirt-Gluster integration, please feel free to let us know as part of
the GlusterFS 3.6 planning process [1] which is ongoing now.
- Vijay
[1] http://www.gluster.org/community/documentation/index.php/Planning36
3963
days inactive
3963
days old
2 comments
3 participants
participants (3)
-
Dan Kenigsberg -
Gianluca Cecchi -
Vijay Bellur