From piotr.szubiakowski at nask.pl Thu Jul 11 08:49:59 2013 Content-Type: multipart/mixed; boundary="===============7291990903484157030==" MIME-Version: 1.0 From: Piotr Szubiakowski To: users at ovirt.org Subject: Re: [Users] which file system for shared disk? Date: Thu, 11 Jul 2013 14:49:40 +0200 Message-ID: <51DEA9E4.3090709@nask.pl> In-Reply-To: 51DE8FC1.4020700@gmail.com --===============7291990903484157030== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable This is a multi-part message in MIME format. --------------040107040309070106020800 Content-Type: text/plain; charset=3DISO-8859-1; format=3Dflowed Content-Transfer-Encoding: 7bit > On 07/11/2013 02:45 PM, Piotr Szubiakowski wrote: >> >> >>> On 07/11/2013 12:52 PM, Piotr Szubiakowski wrote: >>>> >>>>>> The way that oVirt manage storage domains accessed via FC is very = >>>>>> smart. >>>>>> There is separate logical volume for each virtual disk. But I = >>>>>> think that >>>>>> logical volume at the same time could be "touched" only by one = >>>>>> host. Is >>>>>> it possible that two host access read/write the same logical = >>>>>> volume and >>>>>> there is no data corruption? >>>>> >>>>> hence a shared disk over block storage using LVM must be = >>>>> pre-allocated, so no LV changes (lv extend) would be needed. >>>>> (also, it cannot have snapshots, since it would become qcow) >>>> >>>> Ok, but this is the hypervisor view. For a guest OS this LV is = >>>> normal RAW block device. I wonder if someone test this feature and = >>>> access shared disk from many VM's at the same time? >>> I did a brief tests - 3 vms, shared disk, cman/pacemaker + gfs2 - = >>> had no problems using it, no data corruption. although that was only = >>> basic tests, like create/move/delete files, no extensive = >>> usage/stress or something like that. >> >> Thanks Yuri, >> Is fencing handled by the cman/pacemaker? I read about the fanced = >> daemon and it seems to be difficult to use it together with oVirt. >> >> Piotr > yea, fencing handled by pacemaker, stonith resource fence_rhev - = > parameters according to man page, with one exception - had to add = > /pcmk_host_list=3D"vmname" /additional parameter, so pacemaker would = > know that this stonith device belongs to that VM. It's great news for me! Many thanks, Piotr --------------040107040309070106020800 Content-Type: text/html; charset=3DISO-8859-1 Content-Transfer-Encoding: 7bit
On 07/11/2013 02:45 PM, Piotr Szubiakowski wrote:

On 07/11/2013 12:52 PM, Piotr Szubiakowski wrote:

The way that oVirt manage storage domains accessed via FC is very smart.
There is separate logical volume for each virtual disk. But I think that
logical volume at the same time could be "touched" only by one host. Is
it possible that two host access read/write the same logical volume and
there is no data corruption?

hence a shared disk over block storage using LVM must be pre-allocated, so no LV changes (lv extend) would be needed.
(also, it cannot have snapshots, since it would become qcow)

Ok, but this is the hypervisor view. For a guest OS this LV is normal RAW block device. I wonder if someone test this feature and access shared disk from many VM's at the same time?
I did a brief tests - 3 vms, shared disk, cman/pacemaker + gfs2 - had no problems using it, no data corruption. although that was only basic tests, like create/move/delete files, no extensive usage/stress or something like that.

Thanks Yuri,
Is fencing handled by the cman/pacemaker? I read about the fanced daemon and it seems to be difficult to use it together with oVirt.

Piotr
yea, fencing handled by pacemaker, stonith resource fence_rhev - parameters according to man page, with one exception - had to add pcmk_host_list=3D"vmname" additional parameter, so pacemaker would know that this stonith device belongs to that VM.
It's great news for me!

Many thanks,
Piotr
--------------040107040309070106020800-- --===============7291990903484157030== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" VGhpcyBpcyBhIG11bHRpLXBhcnQgbWVzc2FnZSBpbiBNSU1FIGZvcm1hdC4KLS0tLS0tLS0tLS0t LS0wNDAxMDcwNDAzMDkwNzAxMDYwMjA4MDAKQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFy c2V0PUlTTy04ODU5LTE7IGZvcm1hdD1mbG93ZWQKQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZzog N2JpdAoKCj4gT24gMDcvMTEvMjAxMyAwMjo0NSBQTSwgUGlvdHIgU3p1Ymlha293c2tpIHdyb3Rl Ogo+Pgo+Pgo+Pj4gT24gMDcvMTEvMjAxMyAxMjo1MiBQTSwgUGlvdHIgU3p1Ymlha293c2tpIHdy b3RlOgo+Pj4+Cj4+Pj4+PiBUaGUgd2F5IHRoYXQgb1ZpcnQgbWFuYWdlIHN0b3JhZ2UgZG9tYWlu cyBhY2Nlc3NlZCB2aWEgRkMgaXMgdmVyeSAKPj4+Pj4+IHNtYXJ0Lgo+Pj4+Pj4gVGhlcmUgaXMg c2VwYXJhdGUgbG9naWNhbCB2b2x1bWUgZm9yIGVhY2ggdmlydHVhbCBkaXNrLiBCdXQgSSAKPj4+ Pj4+IHRoaW5rIHRoYXQKPj4+Pj4+IGxvZ2ljYWwgdm9sdW1lIGF0IHRoZSBzYW1lIHRpbWUgY291 bGQgYmUgInRvdWNoZWQiIG9ubHkgYnkgb25lIAo+Pj4+Pj4gaG9zdC4gSXMKPj4+Pj4+IGl0IHBv c3NpYmxlIHRoYXQgdHdvIGhvc3QgYWNjZXNzIHJlYWQvd3JpdGUgdGhlIHNhbWUgbG9naWNhbCAK Pj4+Pj4+IHZvbHVtZSBhbmQKPj4+Pj4+IHRoZXJlIGlzIG5vIGRhdGEgY29ycnVwdGlvbj8KPj4+ Pj4KPj4+Pj4gaGVuY2UgYSBzaGFyZWQgZGlzayBvdmVyIGJsb2NrIHN0b3JhZ2UgdXNpbmcgTFZN IG11c3QgYmUgCj4+Pj4+IHByZS1hbGxvY2F0ZWQsIHNvIG5vIExWIGNoYW5nZXMgKGx2IGV4dGVu ZCkgd291bGQgYmUgbmVlZGVkLgo+Pj4+PiAoYWxzbywgaXQgY2Fubm90IGhhdmUgc25hcHNob3Rz LCBzaW5jZSBpdCB3b3VsZCBiZWNvbWUgcWNvdykKPj4+Pgo+Pj4+IE9rLCBidXQgdGhpcyBpcyB0 aGUgaHlwZXJ2aXNvciB2aWV3LiBGb3IgYSBndWVzdCBPUyB0aGlzIExWIGlzIAo+Pj4+IG5vcm1h bCBSQVcgYmxvY2sgZGV2aWNlLiBJIHdvbmRlciBpZiBzb21lb25lIHRlc3QgdGhpcyBmZWF0dXJl IGFuZCAKPj4+PiBhY2Nlc3Mgc2hhcmVkIGRpc2sgZnJvbSBtYW55IFZNJ3MgYXQgdGhlIHNhbWUg dGltZT8KPj4+IEkgZGlkIGEgYnJpZWYgdGVzdHMgLSAzIHZtcywgc2hhcmVkIGRpc2ssIGNtYW4v cGFjZW1ha2VyICsgZ2ZzMiAtIAo+Pj4gaGFkIG5vIHByb2JsZW1zIHVzaW5nIGl0LCBubyBkYXRh IGNvcnJ1cHRpb24uIGFsdGhvdWdoIHRoYXQgd2FzIG9ubHkgCj4+PiBiYXNpYyB0ZXN0cywgbGlr ZSBjcmVhdGUvbW92ZS9kZWxldGUgZmlsZXMsIG5vIGV4dGVuc2l2ZSAKPj4+IHVzYWdlL3N0cmVz cyBvciBzb21ldGhpbmcgbGlrZSB0aGF0Lgo+Pgo+PiBUaGFua3MgWXVyaSwKPj4gSXMgZmVuY2lu ZyBoYW5kbGVkIGJ5IHRoZSBjbWFuL3BhY2VtYWtlcj8gSSByZWFkIGFib3V0IHRoZSBmYW5jZWQg Cj4+IGRhZW1vbiBhbmQgaXQgc2VlbXMgdG8gYmUgZGlmZmljdWx0IHRvIHVzZSBpdCB0b2dldGhl ciB3aXRoIG9WaXJ0Lgo+Pgo+PiBQaW90cgo+IHllYSwgZmVuY2luZyBoYW5kbGVkIGJ5IHBhY2Vt YWtlciwgc3Rvbml0aCByZXNvdXJjZSBmZW5jZV9yaGV2IC0gCj4gcGFyYW1ldGVycyBhY2NvcmRp bmcgdG8gbWFuIHBhZ2UsIHdpdGggb25lIGV4Y2VwdGlvbiAtIGhhZCB0byBhZGQgCj4gL3BjbWtf aG9zdF9saXN0PSJ2bW5hbWUiIC9hZGRpdGlvbmFsIHBhcmFtZXRlciwgc28gcGFjZW1ha2VyIHdv dWxkIAo+IGtub3cgdGhhdCB0aGlzIHN0b25pdGggZGV2aWNlIGJlbG9uZ3MgdG8gdGhhdCBWTS4K SXQncyBncmVhdCBuZXdzIGZvciBtZSEKCk1hbnkgdGhhbmtzLApQaW90cgoKLS0tLS0tLS0tLS0t LS0wNDAxMDcwNDAzMDkwNzAxMDYwMjA4MDAKQ29udGVudC1UeXBlOiB0ZXh0L2h0bWw7IGNoYXJz ZXQ9SVNPLTg4NTktMQpDb250ZW50LVRyYW5zZmVyLUVuY29kaW5nOiA3Yml0Cgo8IURPQ1RZUEUg SFRNTCBQVUJMSUMgIi0vL1czQy8vRFREIEhUTUwgNC4wMSBUcmFuc2l0aW9uYWwvL0VOIj4KPGh0 bWw+CiAgPGhlYWQ+CiAgICA8bWV0YSBjb250ZW50PSJ0ZXh0L2h0bWw7IGNoYXJzZXQ9SVNPLTg4 NTktMSIKICAgICAgaHR0cC1lcXVpdj0iQ29udGVudC1UeXBlIj4KICA8L2hlYWQ+CiAgPGJvZHkg dGV4dD0iIzAwMDAwMCIgYmdjb2xvcj0iI2ZmZmZmZiI+CiAgICA8YnI+CiAgICA8YmxvY2txdW90 ZSBjaXRlPSJtaWQ6NTFERThGQzEuNDAyMDcwMEBnbWFpbC5jb20iIHR5cGU9ImNpdGUiPgogICAg ICA8bWV0YSBjb250ZW50PSJ0ZXh0L2h0bWw7IGNoYXJzZXQ9SVNPLTg4NTktMSIKICAgICAgICBo dHRwLWVxdWl2PSJDb250ZW50LVR5cGUiPgogICAgICA8ZGl2IGNsYXNzPSJtb3otY2l0ZS1wcmVm aXgiPk9uIDA3LzExLzIwMTMgMDI6NDUgUE0sIFBpb3RyCiAgICAgICAgU3p1Ymlha293c2tpIHdy b3RlOjxicj4KICAgICAgPC9kaXY+CiAgICAgIDxibG9ja3F1b3RlIGNpdGU9Im1pZDo1MURFOENE NS4yMDMwODA2QG5hc2sucGwiIHR5cGU9ImNpdGUiPiA8YnI+CiAgICAgICAgPGJyPgogICAgICAg IDxibG9ja3F1b3RlIHR5cGU9ImNpdGUiPk9uIDA3LzExLzIwMTMgMTI6NTIgUE0sIFBpb3RyCiAg ICAgICAgICBTenViaWFrb3dza2kgd3JvdGU6IDxicj4KICAgICAgICAgIDxibG9ja3F1b3RlIHR5 cGU9ImNpdGUiPiA8YnI+CiAgICAgICAgICAgIDxibG9ja3F1b3RlIHR5cGU9ImNpdGUiPgogICAg ICAgICAgICAgIDxibG9ja3F1b3RlIHR5cGU9ImNpdGUiPlRoZSB3YXkgdGhhdCBvVmlydCBtYW5h Z2Ugc3RvcmFnZQogICAgICAgICAgICAgICAgZG9tYWlucyBhY2Nlc3NlZCB2aWEgRkMgaXMgdmVy eSBzbWFydC4gPGJyPgogICAgICAgICAgICAgICAgVGhlcmUgaXMgc2VwYXJhdGUgbG9naWNhbCB2 b2x1bWUgZm9yIGVhY2ggdmlydHVhbCBkaXNrLgogICAgICAgICAgICAgICAgQnV0IEkgdGhpbmsg dGhhdCA8YnI+CiAgICAgICAgICAgICAgICBsb2dpY2FsIHZvbHVtZSBhdCB0aGUgc2FtZSB0aW1l IGNvdWxkIGJlICJ0b3VjaGVkIiBvbmx5CiAgICAgICAgICAgICAgICBieSBvbmUgaG9zdC4gSXMg PGJyPgogICAgICAgICAgICAgICAgaXQgcG9zc2libGUgdGhhdCB0d28gaG9zdCBhY2Nlc3MgcmVh ZC93cml0ZSB0aGUgc2FtZQogICAgICAgICAgICAgICAgbG9naWNhbCB2b2x1bWUgYW5kIDxicj4K ICAgICAgICAgICAgICAgIHRoZXJlIGlzIG5vIGRhdGEgY29ycnVwdGlvbj8gPGJyPgogICAgICAg ICAgICAgIDwvYmxvY2txdW90ZT4KICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAgICAgaGVu Y2UgYSBzaGFyZWQgZGlzayBvdmVyIGJsb2NrIHN0b3JhZ2UgdXNpbmcgTFZNIG11c3QgYmUKICAg ICAgICAgICAgICBwcmUtYWxsb2NhdGVkLCBzbyBubyBMViBjaGFuZ2VzIChsdiBleHRlbmQpIHdv dWxkIGJlCiAgICAgICAgICAgICAgbmVlZGVkLiA8YnI+CiAgICAgICAgICAgICAgKGFsc28sIGl0 IGNhbm5vdCBoYXZlIHNuYXBzaG90cywgc2luY2UgaXQgd291bGQgYmVjb21lCiAgICAgICAgICAg ICAgcWNvdykgPGJyPgogICAgICAgICAgICA8L2Jsb2NrcXVvdGU+CiAgICAgICAgICAgIDxicj4K ICAgICAgICAgICAgT2ssIGJ1dCB0aGlzIGlzIHRoZSBoeXBlcnZpc29yIHZpZXcuIEZvciBhIGd1 ZXN0IE9TIHRoaXMgTFYKICAgICAgICAgICAgaXMgbm9ybWFsIFJBVyBibG9jayBkZXZpY2UuIEkg d29uZGVyIGlmIHNvbWVvbmUgdGVzdCB0aGlzCiAgICAgICAgICAgIGZlYXR1cmUgYW5kIGFjY2Vz cyBzaGFyZWQgZGlzayBmcm9tIG1hbnkgVk0ncyBhdCB0aGUgc2FtZQogICAgICAgICAgICB0aW1l PyA8YnI+CiAgICAgICAgICA8L2Jsb2NrcXVvdGU+CiAgICAgICAgICBJIGRpZCBhIGJyaWVmIHRl c3RzIC0gMyB2bXMsIHNoYXJlZCBkaXNrLCBjbWFuL3BhY2VtYWtlciArCiAgICAgICAgICBnZnMy IC0gaGFkIG5vIHByb2JsZW1zIHVzaW5nIGl0LCBubyBkYXRhIGNvcnJ1cHRpb24uIGFsdGhvdWdo CiAgICAgICAgICB0aGF0IHdhcyBvbmx5IGJhc2ljIHRlc3RzLCBsaWtlIGNyZWF0ZS9tb3ZlL2Rl bGV0ZSBmaWxlcywgbm8KICAgICAgICAgIGV4dGVuc2l2ZSB1c2FnZS9zdHJlc3Mgb3Igc29tZXRo aW5nIGxpa2UgdGhhdC4gPGJyPgogICAgICAgIDwvYmxvY2txdW90ZT4KICAgICAgICA8YnI+CiAg ICAgICAgVGhhbmtzIFl1cmksIDxicj4KICAgICAgICBJcyBmZW5jaW5nIGhhbmRsZWQgYnkgdGhl IGNtYW4vcGFjZW1ha2VyPyBJIHJlYWQgYWJvdXQgdGhlCiAgICAgICAgZmFuY2VkIGRhZW1vbiBh bmQgaXQgc2VlbXMgdG8gYmUgZGlmZmljdWx0IHRvIHVzZSBpdCB0b2dldGhlcgogICAgICAgIHdp dGggb1ZpcnQuIDxicj4KICAgICAgICA8YnI+CiAgICAgICAgUGlvdHIgPGJyPgogICAgICA8L2Js b2NrcXVvdGU+CiAgICAgIHllYSwgZmVuY2luZyBoYW5kbGVkIGJ5IHBhY2VtYWtlciwgc3Rvbml0 aCByZXNvdXJjZSBmZW5jZV9yaGV2IC0KICAgICAgcGFyYW1ldGVycyBhY2NvcmRpbmcgdG8gbWFu IHBhZ2UsIHdpdGggb25lIGV4Y2VwdGlvbiAtIGhhZCB0byBhZGQKICAgICAgPGVtPnBjbWtfaG9z dF9saXN0PSJ2bW5hbWUiIDwvZW0+YWRkaXRpb25hbCBwYXJhbWV0ZXIsIHNvCiAgICAgIHBhY2Vt YWtlciB3b3VsZCBrbm93IHRoYXQgdGhpcyBzdG9uaXRoIGRldmljZSBiZWxvbmdzIHRvIHRoYXQg Vk0uPGJyPgogICAgPC9ibG9ja3F1b3RlPgogICAgSXQncyBncmVhdCBuZXdzIGZvciBtZSE8YnI+ CiAgICA8YnI+CiAgICBNYW55IHRoYW5rcyw8YnI+CiAgICBQaW90cjxicj4KICA8L2JvZHk+Cjwv aHRtbD4KCi0tLS0tLS0tLS0tLS0tMDQwMTA3MDQwMzA5MDcwMTA2MDIwODAwLS0K --===============7291990903484157030==--