Wishlist - Mix gluster and local storage in same data center
Hello all... Would like to be able to use both local storage and gluster within same host / data center. Wondering if this is something being worked on?
------=_Part_76927_1654056605.1446391332174 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit No... when creating DC you choose shared/local (where gluster is shared) You can mix shared (gluster/iscsi/nfs etc) but not local. Regards, __________________________________________________ Aharon Canan ----- Original Message -----
From: "Liam Curtis" <lcurtis@datto.com> To: Users@ovirt.org Sent: Sunday, November 1, 2015 5:17:58 PM Subject: [ovirt-users] Wishlist - Mix gluster and local storage in same data center
Hello all...
Would like to be able to use both local storage and gluster within same host / data center.
Wondering if this is something being worked on?
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
------=_Part_76927_1654056605.1446391332174 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable <html><body><div style=3D"font-family: comic sans ms,comic sans,sans-serif;= font-size: 10pt; color: #000000"><div>No...</div><div><br></div><div>when = creating DC you choose shared/local (where gluster is shared) <br></div><di= v><br></div><div>You can mix shared (gluster/iscsi/nfs etc) but not local.<= br></div><div><br></div><div><br></div><div><span name=3D"x"></span><br><di= v><br></div>Regards,<br>__________________________________________________<= br>Aharon Canan<span name=3D"x"></span><br></div><div><br></div><hr id=3D"z= wchr"><blockquote style=3D"border-left:2px solid #1010FF;margin-left:5px;pa= dding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decorat= ion:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><b>From: <= /b>"Liam Curtis" <lcurtis@datto.com><br><b>To: </b>Users@ovirt.org<br=
<b>Sent: </b>Sunday, November 1, 2015 5:17:58 PM<br><b>Subject: </b>[ovirt= -users] Wishlist - Mix gluster and local storage in same data &n= bsp; center<br><div><br></div><div dir=3D"ltr"= Hello all...<div class=3D"gmail_signature"><div dir=3D"ltr"><div><div dir= =3D"ltr"><div><div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"></div></di= v></div></div></div></div></div></div> <div><br></div><div>Would like to be able to use both local storage and glu= ster within same host / data center. </div><div><br></div><div>Wonderi= ng if this is something being worked on? <br></div></div> <br>_______________________________________________<br>Users mailing list<b= r>Users@ovirt.org<br>http://lists.ovirt.org/mailman/listinfo/users<br></blo= ckquote><div><br></div></div></body></html> ------=_Part_76927_1654056605.1446391332174--
Any chance this will change? It is a severe limitation to not be able to use local storage available to a host as that is often very fast storage. On Sun, Nov 1, 2015 at 10:22 AM, Aharon Canan <acanan@redhat.com> wrote:
No...
when creating DC you choose shared/local (where gluster is shared)
You can mix shared (gluster/iscsi/nfs etc) but not local.
Regards, __________________________________________________ Aharon Canan
------------------------------
*From: *"Liam Curtis" <lcurtis@datto.com> *To: *Users@ovirt.org *Sent: *Sunday, November 1, 2015 5:17:58 PM *Subject: *[ovirt-users] Wishlist - Mix gluster and local storage in same data center
Hello all...
Would like to be able to use both local storage and gluster within same host / data center.
Wondering if this is something being worked on?
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-- *Liam Curtis* Manager of Systems EngineeringDatto, Inc.(203) 529-4949 x228 www.datto.com <http://datto.com/datto-signature/> Join the conversation! [image: Facebook] <http://www.facebook.com/dattoinc> [image: Twitter] <https://twitter.com/datto> [image: LinkedIn] <https://www.linkedin.com/company/datto-inc.> [image: Blog RSS] <http://blog.datto.com/blog> [image: YouTube] <http://www.youtube.com/user/DattoInc/featured> [image: Google Plus Page] <https://plus.google.com/u/0/108292366419623632143/posts>
On Tue, Nov 3, 2015 at 7:48 AM, Liam Curtis <lcurtis@datto.com> wrote:
Any chance this will change? It is a severe limitation to not be able to use local storage available to a host as that is often very fast storage.
This will couple the vm to this host, so you cannot run it to another host. If this host is down, you cannot run the vm on any other host, since your storage is gone. Don't you think this is a severe limitation as well? The current system is focused on high availability; all hosts must be able to access all storage domains in the data center, so you can run any vm on all hosts on the data center. If you don't need to migrate vms or manage shared storage, you can use virt-manager to run vms on specific hosts. Nir
On Sun, Nov 1, 2015 at 10:22 AM, Aharon Canan <acanan@redhat.com> wrote:
No...
when creating DC you choose shared/local (where gluster is shared)
You can mix shared (gluster/iscsi/nfs etc) but not local.
Regards, __________________________________________________ Aharon Canan
________________________________
From: "Liam Curtis" <lcurtis@datto.com> To: Users@ovirt.org Sent: Sunday, November 1, 2015 5:17:58 PM Subject: [ovirt-users] Wishlist - Mix gluster and local storage in same data center
Hello all...
Would like to be able to use both local storage and gluster within same host / data center.
Wondering if this is something being worked on?
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
--
Liam Curtis Manager of Systems Engineering Datto, Inc. (203) 529-4949 x228 www.datto.com
Join the conversation!
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Once upon a time, Nir Soffer <nsoffer@redhat.com> said:
On Tue, Nov 3, 2015 at 7:48 AM, Liam Curtis <lcurtis@datto.com> wrote:
Any chance this will change? It is a severe limitation to not be able to use local storage available to a host as that is often very fast storage.
This will couple the vm to this host, so you cannot run it to another host. If this host is down, you cannot run the vm on any other host, since your storage is gone.
Don't you think this is a severe limitation as well?
There are many things that tie a VM to a host, like USB device passthrough, but that's not reason to remove all such support from oVirt, is it? In my case, I'd like to mix iSCSI and local storage, because I have a couple of systems that need higher disk I/O that I'd like to put on my shared storage. The two systems are redundant to each other, so that is taken care of at a different layer. The two systems don't however consume all the resources of the host machines (lots of CPU and RAM available). I'd like to make them nodes in my oVirt cluster, so those resources can be used for other VMs (that are on shared storage for that level of HA), but I can't do that (at least as far as I know, with oVirt 3.5). I thought that had been mentioned as a feature for 3.6, but I don't see it anywhere in the features or release notes, so I assume that functionality is still not available. -- Chris Adams <cma@cmadams.net>
On Saturday, November 07, 2015 02:02:07 PM Chris Adams wrote:
I thought that had been mentioned as a feature for 3.6, but I don't see it anywhere in the features or release notes, so I assume that functionality is still not available.
I've tried mixing local and shared storage (POSIX compliant FS storage domain that is pointing to a local ext4 partition and FC/iSCSI storage domains) on a shared datacenter type. It allows me to do that (after some juggling) but on 3.5 and 3.6 both, it kicks all the hosts out of the cluster that can't reach that local partition (except the host that contains that partition). This is expected behaviour, on a shared datacenter type, every defined storage domain must be available to every host in the cluster. At one point in time, during testing of 3.6-rc's, I had a situation that host that can't reach the local partition wasn't evicted from the cluster, and VM that was on that partition was happily running. I've most probably hit a bug, and that happened on my first attempt to mess with the POSIX compliant storage domain, so I thought at first that it was a feature for the 3.6 :) After reading comments from Nir and Aharon, and after digging through the docs, mixing local and shared storage domains in the shared datacenter is not possible, at least not until SPM concept is removed from oVirt. In any case, it's always an option to present local storage as NFS, or to just utilize local storage in a better way (as shared and redundant, not always more efficient regarding available storage space) through gluster if you can satisfy replica 3 requirement. Kind regards, Ivan
There are many things that tie a VM to a host, like USB device passthrough, but that's not reason to remove all such support from oVirt, is it? In my case, I'd like to mix iSCSI and local storage, because I have a couple of systems that need higher disk I/O that I'd like to put on my shared storage. The two systems are redundant to each other, so that is taken care of at a different layer. The two systems don't however consume all the resources of the host machines (lots of CPU and RAM available). I'd like to make them nodes in my oVirt cluster, so those resources can be used for other VMs (that are on shared storage for that level of HA), but I can't do that (at least as far as I know, with oVirt 3.5). I thought that had been mentioned as a feature for 3.6, but I don't see it anywhere in the features or release notes, so I assume that functionality is still not available. One thing that would render this whole issue moot is being able to use local fast storage on the hypervisor hosts, ie SSD or 3D-Xpoint drives in LVM-Cache to accelerate IOPS on shared storage. The underlying stuff is already there, and it works. I've been using LVM cache for while in Centos 7. Geting this to work in oVirt would be a killer feature. Alex -- This message is intended only for the addressee and may contain confidential information. Unless you are that person, you may not disclose its contents or use it in any way and are requested to delete the message along with any attachments and notify us immediately. This email is not intended to, nor should it be taken to, constitute advice. The information provided is correct to our knowledge & belief and must not be used as a substitute for obtaining tax, regulatory, investment, legal or any other appropriate advice. "Transact" is operated by Integrated Financial Arrangements Ltd. 29 Clement's Lane, London EC4N 7AE. Tel: (020) 7608 4900 Fax: (020) 7608 5300. (Registered office: as above; Registered in England and Wales under number: 3727592). Authorised and regulated by the Financial Conduct Authority (entered on the Financial Services Register; no. 190856).
Messed up the quote. removed it for clarity:
One thing that would render this whole issue moot is being able to use local fast storage on the hypervisor hosts, ie SSD or 3D-Xpoint drives in LVM-Cache to accelerate IOPS on shared storage.
The underlying stuff is already there, and it works. I've been using LVM cache for while in Centos 7.
Geting this to work in oVirt would be a killer feature.
Alex -- This message is intended only for the addressee and may contain confidential information. Unless you are that person, you may not disclose its contents or use it in any way and are requested to delete the message along with any attachments and notify us immediately. This email is not intended to, nor should it be taken to, constitute advice. The information provided is correct to our knowledge & belief and must not be used as a substitute for obtaining tax, regulatory, investment, legal or any other appropriate advice.
"Transact" is operated by Integrated Financial Arrangements Ltd. 29 Clement's Lane, London EC4N 7AE. Tel: (020) 7608 4900 Fax: (020) 7608 5300. (Registered office: as above; Registered in England and Wales under number: 3727592). Authorised and regulated by the Financial Conduct Authority (entered on the Financial Services Register; no. 190856). _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users -- This message is intended only for the addressee and may contain confidential information. Unless you are that person, you may not disclose its contents or use it in any way and are requested to delete the message along with any attachments and notify us immediately. This email is not intended to, nor should it be taken to, constitute advice. The information provided is correct to our knowledge & belief and must not be used as a substitute for obtaining tax, regulatory, investment, legal or any other appropriate advice.
"Transact" is operated by Integrated Financial Arrangements Ltd. 29 Clement's Lane, London EC4N 7AE. Tel: (020) 7608 4900 Fax: (020) 7608 5300. (Registered office: as above; Registered in England and Wales under number: 3727592). Authorised and regulated by the Financial Conduct Authority (entered on the Financial Services Register; no. 190856).
-- This message is intended only for the addressee and may contain confidential information. Unless you are that person, you may not disclose its contents or use it in any way and are requested to delete the message along with any attachments and notify us immediately. This email is not intended to, nor should it be taken to, constitute advice. The information provided is correct to our knowledge & belief and must not be used as a substitute for obtaining tax, regulatory, investment, legal or any other appropriate advice. "Transact" is operated by Integrated Financial Arrangements Ltd. 29 Clement's Lane, London EC4N 7AE. Tel: (020) 7608 4900 Fax: (020) 7608 5300. (Registered office: as above; Registered in England and Wales under number: 3727592). Authorised and regulated by the Financial Conduct Authority (entered on the Financial Services Register; no. 190856).
On 3 November 2015 at 07:48, Liam Curtis <lcurtis@datto.com> wrote:
Any chance this will change? It is a severe limitation to not be able to use local storage available to a host as that is often very fast storage.
It is not completely impossible to mix shared and local storage. There is a VDSM hook that allows you to attach some local storage to a VM, stored on a shared storage, for temporary use. We actually plan to try and use it in the oVirt CI infrastructure to speed up builds and other random-I/O-intensive operations. Anton Marchukov from the oVirt infra team could give you more details about that.
On Sun, Nov 1, 2015 at 10:22 AM, Aharon Canan <acanan@redhat.com> wrote:
No...
when creating DC you choose shared/local (where gluster is shared)
You can mix shared (gluster/iscsi/nfs etc) but not local.
Regards, __________________________________________________ Aharon Canan
------------------------------
*From: *"Liam Curtis" <lcurtis@datto.com> *To: *Users@ovirt.org *Sent: *Sunday, November 1, 2015 5:17:58 PM *Subject: *[ovirt-users] Wishlist - Mix gluster and local storage in same data center
Hello all...
Would like to be able to use both local storage and gluster within same host / data center.
Wondering if this is something being worked on?
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
--
*Liam Curtis* Manager of Systems EngineeringDatto, Inc.(203) 529-4949 x228 www.datto.com
<http://datto.com/datto-signature/> Join the conversation! [image: Facebook] <http://www.facebook.com/dattoinc> [image: Twitter] <https://twitter.com/datto> [image: LinkedIn] <https://www.linkedin.com/company/datto-inc.> [image: Blog RSS] <http://blog.datto.com/blog> [image: YouTube] <http://www.youtube.com/user/DattoInc/featured> [image: Google Plus Page] <https://plus.google.com/u/0/108292366419623632143/posts>
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-- *Barak Korren* bkorren@redhat.com RHEV-CI Team
On Sun, Nov 8, 2015 at 9:22 AM, Barak Korren <bkorren@redhat.com> wrote:
On 3 November 2015 at 07:48, Liam Curtis <lcurtis@datto.com> wrote:
Any chance this will change? It is a severe limitation to not be able to use local storage available to a host as that is often very fast storage.
It is not completely impossible to mix shared and local storage. There is a VDSM hook that allows you to attach some local storage to a VM, stored on a shared storage, for temporary use. We actually plan to try and use it in the oVirt CI infrastructure to speed up builds and other random-I/O-intensive operations. Anton Marchukov from the oVirt infra team could give you more details about that.
Please document this somewhere in the wiki as well.
On Sun, Nov 1, 2015 at 10:22 AM, Aharon Canan <acanan@redhat.com> wrote:
No...
when creating DC you choose shared/local (where gluster is shared)
You can mix shared (gluster/iscsi/nfs etc) but not local.
Regards, __________________________________________________ Aharon Canan
------------------------------
*From: *"Liam Curtis" <lcurtis@datto.com> *To: *Users@ovirt.org *Sent: *Sunday, November 1, 2015 5:17:58 PM *Subject: *[ovirt-users] Wishlist - Mix gluster and local storage in same data center
Hello all...
Would like to be able to use both local storage and gluster within same host / data center.
Wondering if this is something being worked on?
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
--
*Liam Curtis* Manager of Systems EngineeringDatto, Inc.(203) 529-4949 x228 www.datto.com
<http://datto.com/datto-signature/> Join the conversation! [image: Facebook] <http://www.facebook.com/dattoinc> [image: Twitter] <https://twitter.com/datto> [image: LinkedIn] <https://www.linkedin.com/company/datto-inc.> [image: Blog RSS] <http://blog.datto.com/blog> [image: YouTube] <http://www.youtube.com/user/DattoInc/featured> [image: Google Plus Page] <https://plus.google.com/u/0/108292366419623632143/posts>
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-- *Barak Korren* bkorren@redhat.com RHEV-CI Team
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com
On 9 November 2015 at 09:18, Sandro Bonazzola <sbonazzo@redhat.com> wrote:
On Sun, Nov 8, 2015 at 9:22 AM, Barak Korren <bkorren@redhat.com> wrote:
On 3 November 2015 at 07:48, Liam Curtis <lcurtis@datto.com> wrote:
Any chance this will change? It is a severe limitation to not be able to use local storage available to a host as that is often very fast storage.
It is not completely impossible to mix shared and local storage. There is a VDSM hook that allows you to attach some local storage to a VM, stored on a shared storage, for temporary use. We actually plan to try and use it in the oVirt CI infrastructure to speed up builds and other random-I/O-intensive operations. Anton Marchukov from the oVirt infra team could give you more details about that.
Please document this somewhere in the wiki as well.
There is this: http://www.ovirt.org/VDSM-Hooks/scratchpad
On Sun, Nov 1, 2015 at 10:22 AM, Aharon Canan <acanan@redhat.com> wrote:
No...
when creating DC you choose shared/local (where gluster is shared)
You can mix shared (gluster/iscsi/nfs etc) but not local.
Regards, __________________________________________________ Aharon Canan
------------------------------
*From: *"Liam Curtis" <lcurtis@datto.com> *To: *Users@ovirt.org *Sent: *Sunday, November 1, 2015 5:17:58 PM *Subject: *[ovirt-users] Wishlist - Mix gluster and local storage in same data center
Hello all...
Would like to be able to use both local storage and gluster within same host / data center.
Wondering if this is something being worked on?
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
--
*Liam Curtis* Manager of Systems EngineeringDatto, Inc.(203) 529-4949 x228 www.datto.com
<http://datto.com/datto-signature/> Join the conversation! [image: Facebook] <http://www.facebook.com/dattoinc> [image: Twitter] <https://twitter.com/datto> [image: LinkedIn] <https://www.linkedin.com/company/datto-inc.> [image: Blog RSS] <http://blog.datto.com/blog> [image: YouTube] <http://www.youtube.com/user/DattoInc/featured> [image: Google Plus Page] <https://plus.google.com/u/0/108292366419623632143/posts>
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-- *Barak Korren* bkorren@redhat.com RHEV-CI Team
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com
-- *Barak Korren* bkorren@redhat.com RHEV-CI Team
Thanks for this info! Would be great if the temporary storage could ultimately be made to be persistent
Hello Liam. Well. If you have a look on hook's code than you can make it persistent by removing the code that recreates the storage when VM is started. However than you have to make sure that you either do not relocate VM to another host or purge the local storage image if you do. If you come up with general solution for this or ideas, let me know. So far we are experimenting with using that local storage for CI slaves. In this case it is not a problem for it not be permanent. Anton. On Wed, Nov 11, 2015 at 12:02 AM, Liam Curtis <lcurtis@datto.com> wrote:
Thanks for this info! Would be great if the temporary storage could ultimately be made to be persistent
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-- Anton Marchukov Senior Software Engineer - RHEV CI - Red Hat
On 11 November 2015 at 01:02, Liam Curtis <lcurtis@datto.com> wrote:
Thanks for this info! Would be great if the temporary storage could ultimately be made to be persistent
Having persistent data on local storage will make your life very hard when the time comes around to secure and back up that important data. I think you can enjoy both worlds by using temporary local storage for I/O-intensive calculation, and then store the important final results to a central storage you can easily secure and back up. If what you want to do is pool together your local storage to get something that looks like a cheap central storage, then you are in the realm of ceph/gluster/hyper-converge. There is a performance and complexity overhead to using those technologies. (And not everything is supported by oVirt yet) -- Barak Korren bkorren@redhat.com RHEV-CI Team
Great..thanks all very much for insight and advice. Keep up the great work. This is a very vibrant and supportive community. On Wed, Nov 11, 2015 at 4:51 AM, Barak Korren <bkorren@redhat.com> wrote:
On 11 November 2015 at 01:02, Liam Curtis <lcurtis@datto.com> wrote:
Thanks for this info! Would be great if the temporary storage could ultimately be made to be persistent
Having persistent data on local storage will make your life very hard when the time comes around to secure and back up that important data. I think you can enjoy both worlds by using temporary local storage for I/O-intensive calculation, and then store the important final results to a central storage you can easily secure and back up. If what you want to do is pool together your local storage to get something that looks like a cheap central storage, then you are in the realm of ceph/gluster/hyper-converge. There is a performance and complexity overhead to using those technologies. (And not everything is supported by oVirt yet)
-- Barak Korren bkorren@redhat.com RHEV-CI Team
participants (9)
-
Aharon Canan -
Alex Crow -
Anton Marchukov -
Barak Korren -
Chris Adams -
Ivan Bulatovic -
Liam Curtis -
Nir Soffer -
Sandro Bonazzola