From nicolas at ecarnot.net Thu Feb 8 13:38:57 2018 Content-Type: multipart/mixed; boundary="===============0275680792605921347==" MIME-Version: 1.0 From: Nicolas Ecarnot To: users at ovirt.org Subject: Re: [ovirt-users] qcow2 images corruption Date: Thu, 08 Feb 2018 14:38:54 +0100 Message-ID: <1e09b91d-1bed-7354-7b7d-8b0dfdac1c4c@ecarnot.net> In-Reply-To: CAJgorsb_we+ZEu=m1aeh=e_jaiXqf4P09SyEHVdC8V-S+ZoTFg@mail.gmail.com --===============0275680792605921347== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable This is a multi-part message in MIME format. --------------1BC2811328971F9BAD7C7EDB Content-Type: text/plain; charset=3Dutf-8; format=3Dflowed Content-Transfer-Encoding: 8bit Le 08/02/2018 =C3=A0 13:59, Yaniv Kaul a =C3=A9crit=C2=A0: > > > On Feb 7, 2018 7:08 PM, "Nicolas Ecarnot" > wrote: > > Hello, > > TL; DR : qcow2 images keep getting corrupted. Any workaround? > > Long version: > This discussion has already been launched by me on the oVirt and > on qemu-block mailing list, under similar circumstances but I > learned further things since months and here are some informations : > > - We are using 2 oVirt 3.6.7.5-1.el7.centos datacenters, using > CentOS 7.{2,3} hosts > - Hosts : > =C2=A0 - CentOS 7.2 1511 : > =C2=A0 =C2=A0 - Kernel =3D 3.10.0 327 > =C2=A0 =C2=A0 - KVM : 2.3.0-31 > =C2=A0 =C2=A0 - libvirt : 1.2.17 > =C2=A0 =C2=A0 - vdsm : 4.17.32-1 > =C2=A0 - CentOS 7.3 1611 : > =C2=A0 =C2=A0 - Kernel 3.10.0 514 > =C2=A0 =C2=A0 - KVM : 2.3.0-31 > =C2=A0 =C2=A0 - libvirt 2.0.0-10 > =C2=A0 =C2=A0 - vdsm : 4.17.32-1 > > > All are somewhat old releases. I suggest upgrading to the latest RHEL = > and qemu-kvm bits. > > Later on, upgrade oVirt. > Y. Hello Yaniv, We could discuss for hours about the fact that CentOS 7.3 was released = in January 2017, thus not that old. And also discuss for hours explaining the gap between developers' will = to push their freshest releases and the curb we - industry users - put = on adopting such new versions. In my case, the virtualization = infrastructure is just one of the +30 domains I have to master everyday, = and the more stable the better. In the setup described previously, the qemu qcow2 images were correct, = then not. We did not change anything. We have to find a workaround and = we need your expertise. Not understanding the cause of the corruption threatens us to the same = situation in oVirt 4.2. -- = Nicolas Ecarnot --------------1BC2811328971F9BAD7C7EDB Content-Type: text/html; charset=3Dutf-8 Content-Transfer-Encoding: 8bit
Le 08/02/2018 =C3=A0 13:59, Yaniv Kaul a =C3=A9crit=C2=A0:


On Feb 7, 2018 7:08 PM, "Nicolas Ecarnot" <nicolas(a)ecarnot.net> wrote:
Hello,

TL; DR : qcow2 images keep getting corrupted. Any workaround?

Long version:
This discussion has already been launched by me on the oVirt and on qemu-block mailing list, under similar circumstances but I learned further things since months and here are some informations :

- We are using 2 oVirt 3.6.7.5-1.el7.centos datacenters, using CentOS 7.{2,3} hosts
- Hosts :
=C2=A0 - CentOS 7.2 1511 :
=C2=A0 =C2=A0 - Kernel =3D 3.10.0 327
=C2=A0 =C2=A0 - KVM : 2.3.0-31
=C2=A0 =C2=A0 - libvirt : 1.2.17
=C2=A0 =C2=A0 - vdsm : 4.17.32-1
=C2=A0 - CentOS 7.3 1611 :
=C2=A0 =C2=A0 - Kernel 3.10.0 514
=C2=A0 =C2=A0 - KVM : 2.3.0-31
=C2=A0 =C2=A0 - libvirt 2.0.0-10
=C2=A0 =C2=A0 - vdsm : 4.17.32-1

All are somewhat old releases. I suggest upgrading to the latest RHEL and qemu-kvm bits.=C2=A0

Later on, upgrade oVirt.=C2=A0
Y.=C2=A0
Hello Yaniv,

We could discuss for hours about the fact that CentOS 7.3 was released in January 2017, thus not that old.
And also discuss for hours explaining the gap between developers' will to push their freshest releases and the curb we - industry users - put on adopting such new versions. In my case, the virtualization infrastructure is just one of the +30 domains I have to master everyday, and the more stable the better.
In the setup described previously, the qemu qcow2 images were correct, then not. We did not change anything. We have to find a workaround and we need your expertise.

Not understanding the cause of the corruption threatens us to the same situation in oVirt 4.2.

--
Nicolas Ecarnot
--------------1BC2811328971F9BAD7C7EDB-- --===============0275680792605921347== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" VGhpcyBpcyBhIG11bHRpLXBhcnQgbWVzc2FnZSBpbiBNSU1FIGZvcm1hdC4KLS0tLS0tLS0tLS0t LS0xQkMyODExMzI4OTcxRjlCQUQ3QzdFREIKQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFy c2V0PXV0Zi04OyBmb3JtYXQ9Zmxvd2VkCkNvbnRlbnQtVHJhbnNmZXItRW5jb2Rpbmc6IDhiaXQK CkxlIDA4LzAyLzIwMTggw6AgMTM6NTksIFlhbml2IEthdWwgYSDDqWNyaXTCoDoKPgo+Cj4gT24g RmViIDcsIDIwMTggNzowOCBQTSwgIk5pY29sYXMgRWNhcm5vdCIgPG5pY29sYXNAZWNhcm5vdC5u ZXQgCj4gPG1haWx0bzpuaWNvbGFzQGVjYXJub3QubmV0Pj4gd3JvdGU6Cj4KPiAgICAgSGVsbG8s Cj4KPiAgICAgVEw7IERSIDogcWNvdzIgaW1hZ2VzIGtlZXAgZ2V0dGluZyBjb3JydXB0ZWQuIEFu eSB3b3JrYXJvdW5kPwo+Cj4gICAgIExvbmcgdmVyc2lvbjoKPiAgICAgVGhpcyBkaXNjdXNzaW9u IGhhcyBhbHJlYWR5IGJlZW4gbGF1bmNoZWQgYnkgbWUgb24gdGhlIG9WaXJ0IGFuZAo+ICAgICBv biBxZW11LWJsb2NrIG1haWxpbmcgbGlzdCwgdW5kZXIgc2ltaWxhciBjaXJjdW1zdGFuY2VzIGJ1 dCBJCj4gICAgIGxlYXJuZWQgZnVydGhlciB0aGluZ3Mgc2luY2UgbW9udGhzIGFuZCBoZXJlIGFy ZSBzb21lIGluZm9ybWF0aW9ucyA6Cj4KPiAgICAgLSBXZSBhcmUgdXNpbmcgMiBvVmlydCAzLjYu Ny41LTEuZWw3LmNlbnRvcyBkYXRhY2VudGVycywgdXNpbmcKPiAgICAgQ2VudE9TIDcuezIsM30g aG9zdHMKPiAgICAgLSBIb3N0cyA6Cj4gICAgIMKgIC0gQ2VudE9TIDcuMiAxNTExIDoKPiAgICAg wqAgwqAgLSBLZXJuZWwgPSAzLjEwLjAgMzI3Cj4gICAgIMKgIMKgIC0gS1ZNIDogMi4zLjAtMzEK PiAgICAgwqAgwqAgLSBsaWJ2aXJ0IDogMS4yLjE3Cj4gICAgIMKgIMKgIC0gdmRzbSA6IDQuMTcu MzItMQo+ICAgICDCoCAtIENlbnRPUyA3LjMgMTYxMSA6Cj4gICAgIMKgIMKgIC0gS2VybmVsIDMu MTAuMCA1MTQKPiAgICAgwqAgwqAgLSBLVk0gOiAyLjMuMC0zMQo+ICAgICDCoCDCoCAtIGxpYnZp cnQgMi4wLjAtMTAKPiAgICAgwqAgwqAgLSB2ZHNtIDogNC4xNy4zMi0xCj4KPgo+IEFsbCBhcmUg c29tZXdoYXQgb2xkIHJlbGVhc2VzLiBJIHN1Z2dlc3QgdXBncmFkaW5nIHRvIHRoZSBsYXRlc3Qg UkhFTCAKPiBhbmQgcWVtdS1rdm0gYml0cy4KPgo+IExhdGVyIG9uLCB1cGdyYWRlIG9WaXJ0Lgo+ IFkuCkhlbGxvIFlhbml2LAoKV2UgY291bGQgZGlzY3VzcyBmb3IgaG91cnMgYWJvdXQgdGhlIGZh Y3QgdGhhdCBDZW50T1MgNy4zIHdhcyByZWxlYXNlZCAKaW4gSmFudWFyeSAyMDE3LCB0aHVzIG5v dCB0aGF0IG9sZC4KQW5kIGFsc28gZGlzY3VzcyBmb3IgaG91cnMgZXhwbGFpbmluZyB0aGUgZ2Fw IGJldHdlZW4gZGV2ZWxvcGVycycgd2lsbCAKdG8gcHVzaCB0aGVpciBmcmVzaGVzdCByZWxlYXNl cyBhbmQgdGhlIGN1cmIgd2UgLSBpbmR1c3RyeSB1c2VycyAtIHB1dCAKb24gYWRvcHRpbmcgc3Vj aCBuZXcgdmVyc2lvbnMuIEluIG15IGNhc2UsIHRoZSB2aXJ0dWFsaXphdGlvbiAKaW5mcmFzdHJ1 Y3R1cmUgaXMganVzdCBvbmUgb2YgdGhlICszMCBkb21haW5zIEkgaGF2ZSB0byBtYXN0ZXIgZXZl cnlkYXksIAphbmQgdGhlIG1vcmUgc3RhYmxlIHRoZSBiZXR0ZXIuCkluIHRoZSBzZXR1cCBkZXNj cmliZWQgcHJldmlvdXNseSwgdGhlIHFlbXUgcWNvdzIgaW1hZ2VzIHdlcmUgY29ycmVjdCwgCnRo ZW4gbm90LiBXZSBkaWQgbm90IGNoYW5nZSBhbnl0aGluZy4gV2UgaGF2ZSB0byBmaW5kIGEgd29y a2Fyb3VuZCBhbmQgCndlIG5lZWQgeW91ciBleHBlcnRpc2UuCgpOb3QgdW5kZXJzdGFuZGluZyB0 aGUgY2F1c2Ugb2YgdGhlIGNvcnJ1cHRpb24gdGhyZWF0ZW5zIHVzIHRvIHRoZSBzYW1lIApzaXR1 YXRpb24gaW4gb1ZpcnQgNC4yLgoKLS0gCk5pY29sYXMgRWNhcm5vdAoKLS0tLS0tLS0tLS0tLS0x QkMyODExMzI4OTcxRjlCQUQ3QzdFREIKQ29udGVudC1UeXBlOiB0ZXh0L2h0bWw7IGNoYXJzZXQ9 dXRmLTgKQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZzogOGJpdAoKPGh0bWw+CiAgPGhlYWQ+CiAg ICA8bWV0YSBodHRwLWVxdWl2PSJDb250ZW50LVR5cGUiIGNvbnRlbnQ9InRleHQvaHRtbDsgY2hh cnNldD11dGYtOCI+CiAgPC9oZWFkPgogIDxib2R5IHRleHQ9IiMwMDAwMDAiIGJnY29sb3I9IiNG RkZGRkYiPgogICAgPGRpdiBjbGFzcz0ibW96LWNpdGUtcHJlZml4Ij5MZSAwOC8wMi8yMDE4IMOg IDEzOjU5LCBZYW5pdiBLYXVsIGEKICAgICAgw6ljcml0wqA6PGJyPgogICAgPC9kaXY+CiAgICA8 YmxvY2txdW90ZSB0eXBlPSJjaXRlIgpjaXRlPSJtaWQ6Q0FKZ29yc2Jfd2UrWkV1PW0xYWVoPWVf amFpWHFmNFAwOVN5RUhWZEM4Vi1TK1pvVEZnQG1haWwuZ21haWwuY29tIj4KICAgICAgPGRpdiBk aXI9ImF1dG8iPgogICAgICAgIDxkaXY+PGJyPgogICAgICAgICAgPGRpdiBjbGFzcz0iZ21haWxf ZXh0cmEiPjxicj4KICAgICAgICAgICAgPGRpdiBjbGFzcz0iZ21haWxfcXVvdGUiPk9uIEZlYiA3 LCAyMDE4IDc6MDggUE0sICJOaWNvbGFzCiAgICAgICAgICAgICAgRWNhcm5vdCIgJmx0OzxhIGhy ZWY9Im1haWx0bzpuaWNvbGFzQGVjYXJub3QubmV0IgogICAgICAgICAgICAgICAgbW96LWRvLW5v dC1zZW5kPSJ0cnVlIj5uaWNvbGFzQGVjYXJub3QubmV0PC9hPiZndDsKICAgICAgICAgICAgICB3 cm90ZTo8YnIgdHlwZT0iYXR0cmlidXRpb24iPgogICAgICAgICAgICAgIDxibG9ja3F1b3RlIGNs YXNzPSJxdW90ZSIgc3R5bGU9Im1hcmdpbjowIDAgMAogICAgICAgICAgICAgICAgLjhleDtib3Jk ZXItbGVmdDoxcHggI2NjYyBzb2xpZDtwYWRkaW5nLWxlZnQ6MWV4Ij5IZWxsbyw8YnI+CiAgICAg ICAgICAgICAgICA8YnI+CiAgICAgICAgICAgICAgICBUTDsgRFIgOiBxY293MiBpbWFnZXMga2Vl cCBnZXR0aW5nIGNvcnJ1cHRlZC4gQW55CiAgICAgICAgICAgICAgICB3b3JrYXJvdW5kPzxicj4K ICAgICAgICAgICAgICAgIDxicj4KICAgICAgICAgICAgICAgIExvbmcgdmVyc2lvbjo8YnI+CiAg ICAgICAgICAgICAgICBUaGlzIGRpc2N1c3Npb24gaGFzIGFscmVhZHkgYmVlbiBsYXVuY2hlZCBi eSBtZSBvbiB0aGUKICAgICAgICAgICAgICAgIG9WaXJ0IGFuZCBvbiBxZW11LWJsb2NrIG1haWxp bmcgbGlzdCwgdW5kZXIgc2ltaWxhcgogICAgICAgICAgICAgICAgY2lyY3Vtc3RhbmNlcyBidXQg SSBsZWFybmVkIGZ1cnRoZXIgdGhpbmdzIHNpbmNlIG1vbnRocwogICAgICAgICAgICAgICAgYW5k IGhlcmUgYXJlIHNvbWUgaW5mb3JtYXRpb25zIDo8YnI+CiAgICAgICAgICAgICAgICA8YnI+CiAg ICAgICAgICAgICAgICAtIFdlIGFyZSB1c2luZyAyIG9WaXJ0IDMuNi43LjUtMS5lbDcuY2VudG9z IGRhdGFjZW50ZXJzLAogICAgICAgICAgICAgICAgdXNpbmcgQ2VudE9TIDcuezIsM30gaG9zdHM8 YnI+CiAgICAgICAgICAgICAgICAtIEhvc3RzIDo8YnI+CiAgICAgICAgICAgICAgICDCoCAtIENl bnRPUyA3LjIgMTUxMSA6PGJyPgogICAgICAgICAgICAgICAgwqAgwqAgLSBLZXJuZWwgPSAzLjEw LjAgMzI3PGJyPgogICAgICAgICAgICAgICAgwqAgwqAgLSBLVk0gOiAyLjMuMC0zMTxicj4KICAg ICAgICAgICAgICAgIMKgIMKgIC0gbGlidmlydCA6IDEuMi4xNzxicj4KICAgICAgICAgICAgICAg IMKgIMKgIC0gdmRzbSA6IDQuMTcuMzItMTxicj4KICAgICAgICAgICAgICAgIMKgIC0gQ2VudE9T IDcuMyAxNjExIDo8YnI+CiAgICAgICAgICAgICAgICDCoCDCoCAtIEtlcm5lbCAzLjEwLjAgNTE0 PGJyPgogICAgICAgICAgICAgICAgwqAgwqAgLSBLVk0gOiAyLjMuMC0zMTxicj4KICAgICAgICAg ICAgICAgIMKgIMKgIC0gbGlidmlydCAyLjAuMC0xMDxicj4KICAgICAgICAgICAgICAgIMKgIMKg IC0gdmRzbSA6IDQuMTcuMzItMTxicj4KICAgICAgICAgICAgICA8L2Jsb2NrcXVvdGU+CiAgICAg ICAgICAgIDwvZGl2PgogICAgICAgICAgPC9kaXY+CiAgICAgICAgPC9kaXY+CiAgICAgICAgPGRp diBkaXI9ImF1dG8iPjxicj4KICAgICAgICA8L2Rpdj4KICAgICAgICA8ZGl2IGRpcj0iYXV0byI+ QWxsIGFyZSBzb21ld2hhdCBvbGQgcmVsZWFzZXMuIEkgc3VnZ2VzdAogICAgICAgICAgdXBncmFk aW5nIHRvIHRoZSBsYXRlc3QgUkhFTCBhbmQgcWVtdS1rdm0gYml0cy7CoDwvZGl2PgogICAgICAg IDxkaXYgZGlyPSJhdXRvIj48YnI+CiAgICAgICAgPC9kaXY+CiAgICAgICAgPGRpdiBkaXI9ImF1 dG8iPkxhdGVyIG9uLCB1cGdyYWRlIG9WaXJ0LsKgPC9kaXY+CiAgICAgICAgPGRpdiBkaXI9ImF1 dG8iPlkuwqA8L2Rpdj4KICAgICAgPC9kaXY+CiAgICA8L2Jsb2NrcXVvdGU+CiAgICBIZWxsbyBZ YW5pdiw8YnI+CiAgICA8YnI+CiAgICBXZSBjb3VsZCBkaXNjdXNzIGZvciBob3VycyBhYm91dCB0 aGUgZmFjdCB0aGF0IENlbnRPUyA3LjMgd2FzCiAgICByZWxlYXNlZCBpbiBKYW51YXJ5IDIwMTcs IHRodXMgbm90IHRoYXQgb2xkLjxicj4KICAgIEFuZCBhbHNvIGRpc2N1c3MgZm9yIGhvdXJzIGV4 cGxhaW5pbmcgdGhlIGdhcCBiZXR3ZWVuIGRldmVsb3BlcnMnCiAgICB3aWxsIHRvIHB1c2ggdGhl aXIgZnJlc2hlc3QgcmVsZWFzZXMgYW5kIHRoZSBjdXJiIHdlIC0gaW5kdXN0cnkKICAgIHVzZXJz IC0gcHV0IG9uIGFkb3B0aW5nIHN1Y2ggbmV3IHZlcnNpb25zLiBJbiBteSBjYXNlLCB0aGUKICAg IHZpcnR1YWxpemF0aW9uIGluZnJhc3RydWN0dXJlIGlzIGp1c3Qgb25lIG9mIHRoZSArMzAgZG9t YWlucyBJIGhhdmUKICAgIHRvIG1hc3RlciBldmVyeWRheSwgYW5kIHRoZSBtb3JlIHN0YWJsZSB0 aGUgYmV0dGVyLjxicj4KICAgIEluIHRoZSBzZXR1cCBkZXNjcmliZWQgcHJldmlvdXNseSwgdGhl IHFlbXUgcWNvdzIgaW1hZ2VzIHdlcmUKICAgIGNvcnJlY3QsIHRoZW4gbm90LiBXZSBkaWQgbm90 IGNoYW5nZSBhbnl0aGluZy4gV2UgaGF2ZSB0byBmaW5kIGEKICAgIHdvcmthcm91bmQgYW5kIHdl IG5lZWQgeW91ciBleHBlcnRpc2UuPGJyPgogICAgPGJyPgogICAgTm90IHVuZGVyc3RhbmRpbmcg dGhlIGNhdXNlIG9mIHRoZSBjb3JydXB0aW9uIHRocmVhdGVucyB1cyB0byB0aGUKICAgIHNhbWUg c2l0dWF0aW9uIGluIG9WaXJ0IDQuMi48YnI+CiAgICA8YnI+CiAgICAtLSA8YnI+CiAgICBOaWNv bGFzIEVjYXJub3Q8YnI+CiAgPC9ib2R5Pgo8L2h0bWw+CgotLS0tLS0tLS0tLS0tLTFCQzI4MTEz Mjg5NzFGOUJBRDdDN0VEQi0tCg== --===============0275680792605921347==--