From riehecky at fnal.gov Tue Dec 20 14:54:59 2016 Content-Type: multipart/mixed; boundary="===============7616020719386533876==" MIME-Version: 1.0 From: Pat Riehecky To: users at ovirt.org Subject: Re: [ovirt-users] Adding Disk stuck? Date: Tue, 20 Dec 2016 08:54:49 -0600 Message-ID: <715bf3e0-ea95-e0b8-c9a7-10dbad0411fd@fnal.gov> In-Reply-To: CAOpfNcVLqOVHGBf7Q5awiFfRvyfzKWWxmWqWT4onxhw3T5qgqg@mail.gmail.com --===============7616020719386533876== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable --------------7AC572FD7B30942020C6F98C Content-Type: text/plain; charset=3D"utf-8"; format=3Dflowed Content-Transfer-Encoding: 7bit They were fairly large and getting filtered out. So I setup a github = repo containing the log files. https://github.com/jcpunk/logs Pat On 12/19/2016 10:10 AM, Elad Ben Aharon wrote: > Hi, can you please provide engine.log? > > On Mon, Dec 19, 2016 at 5:06 PM, Pat Riehecky > wrote: > > Last Friday I started a job to add 1 new disk to each of 4 VMs - > total of 4 disks each 100G. > > It seems to still be running, but no host shows an obvious IO load. > State is > > Adding Disk (hour glass) > -> Validating (green check mark) > -> Executing (hour glass) > ->-> Creating Volume (green check mark) > > I checked in with: > /usr/share/ovirt-engine/setup/dbutils/taskcleaner.sh > > and it didn't show anything interesting. > > The VMs themselves show the disks are there, but the VMs are still > locked by the disk processes. > > Ideas? > > Pat > > -- = > Pat Riehecky > > Fermi National Accelerator Laboratory > www.fnal.gov > www.scientificlinux.org > > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > > > --------------7AC572FD7B30942020C6F98C Content-Type: text/html; charset=3D"utf-8" Content-Transfer-Encoding: 8bit They were fairly large and getting filtered out.=C2=A0 So I setup a github repo containing the log files.

https://github.com/jcpunk/logs

Pat

On 12/19/2016 10:10 AM, Elad Ben Aharon wrote:

Hi, can you please provide engine.log?

On Mon, Dec 19, 2016 at 5:06 PM, Pat Riehecky <rieheck= y(a)fnal.gov> wrote:

Last Friday I started a job to add 1 new disk to each of 4 VMs - total of 4 disks each 100G.

It seems to still be running, but no host shows an obvious IO load.
State is

Adding Disk (hour glass)
-> Validating (green check mark)
-> Executing (hour glass)
->-> Creating Volume (green check mark)

I checked in with:
/usr/share/ovirt-engine/setup/dbutils/taskcleaner.sh

and it didn't show anything interesting.

The VMs themselves show the disks are there, but the VMs are still locked by the disk processes.

Ideas?

Pat

--
Pat Riehecky

Fermi National Accelerator Laboratory
www.fnal.gov
www.scientificlinux.org

_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.o= rg/mailman/listinfo/users

--------------7AC572FD7B30942020C6F98C-- --===============7616020719386533876== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" LS0tLS0tLS0tLS0tLS03QUM1NzJGRDdCMzA5NDIwMjBDNkY5OEMKQ29udGVudC1UeXBlOiB0ZXh0 L3BsYWluOyBjaGFyc2V0PSJ1dGYtOCI7IGZvcm1hdD1mbG93ZWQKQ29udGVudC1UcmFuc2Zlci1F bmNvZGluZzogN2JpdAoKVGhleSB3ZXJlIGZhaXJseSBsYXJnZSBhbmQgZ2V0dGluZyBmaWx0ZXJl ZCBvdXQuICBTbyBJIHNldHVwIGEgZ2l0aHViIApyZXBvIGNvbnRhaW5pbmcgdGhlIGxvZyBmaWxl cy4KCmh0dHBzOi8vZ2l0aHViLmNvbS9qY3B1bmsvbG9ncwoKUGF0CgoKT24gMTIvMTkvMjAxNiAx MDoxMCBBTSwgRWxhZCBCZW4gQWhhcm9uIHdyb3RlOgo+IEhpLCBjYW4geW91IHBsZWFzZSBwcm92 aWRlIGVuZ2luZS5sb2c/Cj4KPiBPbiBNb24sIERlYyAxOSwgMjAxNiBhdCA1OjA2IFBNLCBQYXQg UmllaGVja3kgPHJpZWhlY2t5QGZuYWwuZ292IAo+IDxtYWlsdG86cmllaGVja3lAZm5hbC5nb3Y+ PiB3cm90ZToKPgo+ICAgICBMYXN0IEZyaWRheSBJIHN0YXJ0ZWQgYSBqb2IgdG8gYWRkIDEgbmV3 IGRpc2sgdG8gZWFjaCBvZiA0IFZNcyAtCj4gICAgIHRvdGFsIG9mIDQgZGlza3MgZWFjaCAxMDBH Lgo+Cj4gICAgIEl0IHNlZW1zIHRvIHN0aWxsIGJlIHJ1bm5pbmcsIGJ1dCBubyBob3N0IHNob3dz IGFuIG9idmlvdXMgSU8gbG9hZC4KPiAgICAgU3RhdGUgaXMKPgo+ICAgICBBZGRpbmcgRGlzayAo aG91ciBnbGFzcykKPiAgICAgLT4gVmFsaWRhdGluZyAoZ3JlZW4gY2hlY2sgbWFyaykKPiAgICAg LT4gRXhlY3V0aW5nIChob3VyIGdsYXNzKQo+ICAgICAtPi0+IENyZWF0aW5nIFZvbHVtZSAoZ3Jl ZW4gY2hlY2sgbWFyaykKPgo+ICAgICBJIGNoZWNrZWQgaW4gd2l0aDoKPiAgICAgL3Vzci9zaGFy ZS9vdmlydC1lbmdpbmUvc2V0dXAvZGJ1dGlscy90YXNrY2xlYW5lci5zaAo+Cj4gICAgIGFuZCBp dCBkaWRuJ3Qgc2hvdyBhbnl0aGluZyBpbnRlcmVzdGluZy4KPgo+ICAgICBUaGUgVk1zIHRoZW1z ZWx2ZXMgc2hvdyB0aGUgZGlza3MgYXJlIHRoZXJlLCBidXQgdGhlIFZNcyBhcmUgc3RpbGwKPiAg ICAgbG9ja2VkIGJ5IHRoZSBkaXNrIHByb2Nlc3Nlcy4KPgo+ICAgICBJZGVhcz8KPgo+ICAgICBQ YXQKPgo+ICAgICAtLSAKPiAgICAgUGF0IFJpZWhlY2t5Cj4KPiAgICAgRmVybWkgTmF0aW9uYWwg QWNjZWxlcmF0b3IgTGFib3JhdG9yeQo+ICAgICB3d3cuZm5hbC5nb3YgPGh0dHA6Ly93d3cuZm5h bC5nb3Y+Cj4gICAgIHd3dy5zY2llbnRpZmljbGludXgub3JnIDxodHRwOi8vd3d3LnNjaWVudGlm aWNsaW51eC5vcmc+Cj4KPiAgICAgX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX18KPiAgICAgVXNlcnMgbWFpbGluZyBsaXN0Cj4gICAgIFVzZXJzQG92aXJ0Lm9y ZyA8bWFpbHRvOlVzZXJzQG92aXJ0Lm9yZz4KPiAgICAgaHR0cDovL2xpc3RzLm92aXJ0Lm9yZy9t YWlsbWFuL2xpc3RpbmZvL3VzZXJzCj4gICAgIDxodHRwOi8vbGlzdHMub3ZpcnQub3JnL21haWxt YW4vbGlzdGluZm8vdXNlcnM+Cj4KPgoKCi0tLS0tLS0tLS0tLS0tN0FDNTcyRkQ3QjMwOTQyMDIw QzZGOThDCkNvbnRlbnQtVHlwZTogdGV4dC9odG1sOyBjaGFyc2V0PSJ1dGYtOCIKQ29udGVudC1U cmFuc2Zlci1FbmNvZGluZzogOGJpdAoKPGh0bWw+CiAgPGhlYWQ+CiAgICA8bWV0YSBjb250ZW50 PSJ0ZXh0L2h0bWw7IGNoYXJzZXQ9dXRmLTgiIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSI+CiAg PC9oZWFkPgogIDxib2R5IGJnY29sb3I9IiNGRkZGRkYiIHRleHQ9IiMwMDAwMDAiPgogICAgVGhl eSB3ZXJlIGZhaXJseSBsYXJnZSBhbmQgZ2V0dGluZyBmaWx0ZXJlZCBvdXQuwqAgU28gSSBzZXR1 cCBhCiAgICBnaXRodWIgcmVwbyBjb250YWluaW5nIHRoZSBsb2cgZmlsZXMuPGJyPgogICAgPGJy PgogICAgPGEgY2xhc3M9Im1vei10eHQtbGluay1mcmVldGV4dCIgaHJlZj0iaHR0cHM6Ly9naXRo dWIuY29tL2pjcHVuay9sb2dzIj5odHRwczovL2dpdGh1Yi5jb20vamNwdW5rL2xvZ3M8L2E+PGJy PgogICAgPGJyPgogICAgUGF0PGJyPgogICAgPGJyPgogICAgPGJyPgogICAgPGRpdiBjbGFzcz0i bW96LWNpdGUtcHJlZml4Ij5PbiAxMi8xOS8yMDE2IDEwOjEwIEFNLCBFbGFkIEJlbiBBaGFyb24K ICAgICAgd3JvdGU6PGJyPgogICAgPC9kaXY+CiAgICA8YmxvY2txdW90ZQpjaXRlPSJtaWQ6Q0FP cGZOY1ZMcU9WSEdCZjdRNWF3aUZmUnZ5ZnpLV1d4bVdxV1Q0b254aHczVDVxZ3FnQG1haWwuZ21h aWwuY29tIgogICAgICB0eXBlPSJjaXRlIj4KICAgICAgPG1ldGEgaHR0cC1lcXVpdj0iQ29udGVu dC1UeXBlIiBjb250ZW50PSJ0ZXh0L2h0bWw7IGNoYXJzZXQ9dXRmLTgiPgogICAgICA8ZGl2IGRp cj0ibHRyIj5IaSwgY2FuIHlvdSBwbGVhc2UgcHJvdmlkZSBlbmdpbmUubG9nPzxicj4KICAgICAg PC9kaXY+CiAgICAgIDxkaXYgY2xhc3M9ImdtYWlsX2V4dHJhIj48YnI+CiAgICAgICAgPGRpdiBj bGFzcz0iZ21haWxfcXVvdGUiPk9uIE1vbiwgRGVjIDE5LCAyMDE2IGF0IDU6MDYgUE0sIFBhdAog ICAgICAgICAgUmllaGVja3kgPHNwYW4gZGlyPSJsdHIiPiZsdDs8YSBtb3otZG8tbm90LXNlbmQ9 InRydWUiCiAgICAgICAgICAgICAgaHJlZj0ibWFpbHRvOnJpZWhlY2t5QGZuYWwuZ292IiB0YXJn ZXQ9Il9ibGFuayI+cmllaGVja3lAZm5hbC5nb3Y8L2E+Jmd0Ozwvc3Bhbj4KICAgICAgICAgIHdy b3RlOjxicj4KICAgICAgICAgIDxibG9ja3F1b3RlIGNsYXNzPSJnbWFpbF9xdW90ZSIgc3R5bGU9 Im1hcmdpbjowIDAgMAogICAgICAgICAgICAuOGV4O2JvcmRlci1sZWZ0OjFweCAjY2NjIHNvbGlk O3BhZGRpbmctbGVmdDoxZXgiPkxhc3QKICAgICAgICAgICAgRnJpZGF5IEkgc3RhcnRlZCBhIGpv YiB0byBhZGQgMSBuZXcgZGlzayB0byBlYWNoIG9mIDQgVk1zIC0KICAgICAgICAgICAgdG90YWwg b2YgNCBkaXNrcyBlYWNoIDEwMEcuPGJyPgogICAgICAgICAgICA8YnI+CiAgICAgICAgICAgIEl0 IHNlZW1zIHRvIHN0aWxsIGJlIHJ1bm5pbmcsIGJ1dCBubyBob3N0IHNob3dzIGFuIG9idmlvdXMK ICAgICAgICAgICAgSU8gbG9hZC48YnI+CiAgICAgICAgICAgIFN0YXRlIGlzPGJyPgogICAgICAg ICAgICA8YnI+CiAgICAgICAgICAgIEFkZGluZyBEaXNrIChob3VyIGdsYXNzKTxicj4KICAgICAg ICAgICAgLSZndDsgVmFsaWRhdGluZyAoZ3JlZW4gY2hlY2sgbWFyayk8YnI+CiAgICAgICAgICAg IC0mZ3Q7IEV4ZWN1dGluZyAoaG91ciBnbGFzcyk8YnI+CiAgICAgICAgICAgIC0mZ3Q7LSZndDsg Q3JlYXRpbmcgVm9sdW1lIChncmVlbiBjaGVjayBtYXJrKTxicj4KICAgICAgICAgICAgPGJyPgog ICAgICAgICAgICBJIGNoZWNrZWQgaW4gd2l0aDo8YnI+CiAgICAgICAgICAgIC91c3Ivc2hhcmUv b3ZpcnQtZW5naW5lL3NldHVwLzx3YnI+ZGJ1dGlscy90YXNrY2xlYW5lci5zaDxicj4KICAgICAg ICAgICAgPGJyPgogICAgICAgICAgICBhbmQgaXQgZGlkbid0IHNob3cgYW55dGhpbmcgaW50ZXJl c3RpbmcuPGJyPgogICAgICAgICAgICA8YnI+CiAgICAgICAgICAgIFRoZSBWTXMgdGhlbXNlbHZl cyBzaG93IHRoZSBkaXNrcyBhcmUgdGhlcmUsIGJ1dCB0aGUgVk1zIGFyZQogICAgICAgICAgICBz dGlsbCBsb2NrZWQgYnkgdGhlIGRpc2sgcHJvY2Vzc2VzLjxicj4KICAgICAgICAgICAgPGJyPgog ICAgICAgICAgICBJZGVhcz88c3BhbiBjbGFzcz0iSE9FblpiIj48Zm9udCBjb2xvcj0iIzg4ODg4 OCI+PGJyPgogICAgICAgICAgICAgICAgPGJyPgogICAgICAgICAgICAgICAgUGF0PGJyPgogICAg ICAgICAgICAgICAgPGJyPgogICAgICAgICAgICAgICAgLS0gPGJyPgogICAgICAgICAgICAgICAg UGF0IFJpZWhlY2t5PGJyPgogICAgICAgICAgICAgICAgPGJyPgogICAgICAgICAgICAgICAgRmVy bWkgTmF0aW9uYWwgQWNjZWxlcmF0b3IgTGFib3JhdG9yeTxicj4KICAgICAgICAgICAgICAgIDxh IG1vei1kby1ub3Qtc2VuZD0idHJ1ZSIgaHJlZj0iaHR0cDovL3d3dy5mbmFsLmdvdiIKICAgICAg ICAgICAgICAgICAgcmVsPSJub3JlZmVycmVyIiB0YXJnZXQ9Il9ibGFuayI+d3d3LmZuYWwuZ292 PC9hPjxicj4KICAgICAgICAgICAgICAgIDxhIG1vei1kby1ub3Qtc2VuZD0idHJ1ZSIKICAgICAg ICAgICAgICAgICAgaHJlZj0iaHR0cDovL3d3dy5zY2llbnRpZmljbGludXgub3JnIiByZWw9Im5v cmVmZXJyZXIiCiAgICAgICAgICAgICAgICAgIHRhcmdldD0iX2JsYW5rIj53d3cuc2NpZW50aWZp Y2xpbnV4Lm9yZzwvYT48YnI+CiAgICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAgICAgICBf X19fX19fX19fX19fX19fX19fX19fX19fX19fX188d2JyPl9fX19fX19fX19fX19fX19fPGJyPgog ICAgICAgICAgICAgICAgVXNlcnMgbWFpbGluZyBsaXN0PGJyPgogICAgICAgICAgICAgICAgPGEg bW96LWRvLW5vdC1zZW5kPSJ0cnVlIiBocmVmPSJtYWlsdG86VXNlcnNAb3ZpcnQub3JnIgogICAg ICAgICAgICAgICAgICB0YXJnZXQ9Il9ibGFuayI+VXNlcnNAb3ZpcnQub3JnPC9hPjxicj4KICAg ICAgICAgICAgICAgIDxhIG1vei1kby1ub3Qtc2VuZD0idHJ1ZSIKICAgICAgICAgICAgICAgICAg aHJlZj0iaHR0cDovL2xpc3RzLm92aXJ0Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL3VzZXJzIgogICAg ICAgICAgICAgICAgICByZWw9Im5vcmVmZXJyZXIiIHRhcmdldD0iX2JsYW5rIj5odHRwOi8vbGlz dHMub3ZpcnQub3JnL21haWxtYW48d2JyPi9saXN0aW5mby91c2VyczwvYT48YnI+CiAgICAgICAg ICAgICAgPC9mb250Pjwvc3Bhbj48L2Jsb2NrcXVvdGU+CiAgICAgICAgPC9kaXY+CiAgICAgICAg PGJyPgogICAgICA8L2Rpdj4KICAgIDwvYmxvY2txdW90ZT4KICAgIDxicj4KICA8L2JvZHk+Cjwv aHRtbD4KCi0tLS0tLS0tLS0tLS0tN0FDNTcyRkQ3QjMwOTQyMDIwQzZGOThDLS0K --===============7616020719386533876==--