January 2018 - Users - oVirt List Archives

How to manage virtual networks in oVirt 4.2?
by Dmitry Semenov 23 Jan '18

23 Jan '18

I use OVN with linuxbridge. Where and how I may adjust routing, NAT etc. for virtual networks? ManageIQ doesn't see them (probably it is devoted that my switch type should be OVS) :( -- Best regards,

2 1

Found some bugs with NFS.
by Sergey Kulikov 23 Jan '18

23 Jan '18

This is test environment, running Centos 7.4, oVirt 4.2.0, kernel 3.10.0-693.11.6.el7.x86_64 (3.10.0-693.11.1 and 3.10.0-693 have same bugs) 1. Can't force NFS to 4.0. Some time ago, I've set my NFS version for all storage domains to V4, because there was a bug with Netapp data ontap 8.x and RHEL, using NFS 4.1(NFS mount started to hang after a while, STATEID problems) v4 on centos 7.2 and 7.3 was mounting NFS as 4.0, so there were no problems related to NFS, after some time Centos 7.4 was released, and I've noticed that mount points started to hang again, NFS was mounted with vers=4.1, and it's not possible to change to 4.0, both options "V4" and "V4.1" mounts as 4.1. Looks like V4 option is system default version for 4.X, and as I know it was changed in Centos 7.4 from 4.0 to 4.1, maybe 4.0 option should be added to force 4.0 version? because adding vers=/nfsvers= in "Additional mount options" is denied by ovirt. I know, I can turn it off on netapp side, but there may be situations where storage is out of control. And 4.0 version can't be set on ovirt side. 2. This bug isn't directly related to ovirt, but affects it. Don't really shure that this is right place to report. As I've said before there were bug with NFS 4.1, Netapp data ontap 8 and RHEL 7.x, but it was fixed in otap 9.x, Now we have 9.x ONTAP on Netapp and it brought new bugs with RHEL 7.4 :D After updating to centos 7.4 nfs domains in ovirt started to hang\lock again, This happens randomly, on random hosts, after few days of uptime, entire datacenter goes offline, hosts down, storage domains down, some vms in UP and some in unknown state, but actually VMs are working, HostedEngine also working, but I can't control the environment. There are many hanging ioprocess(>1300) and vdsm processes(>1300) on some hosts, also there are some dd commands, that are checking storage hanging: ├─vdsmd─┬─2*[dd] │ ├─1304*[ioprocess───{ioprocess}] │ ├─12*[ioprocess───4*[{ioprocess}]] │ └─1365*[{vdsmd}] vdsm 19470 0.0 0.0 4360 348 ? D< Jan21 0:00 /usr/bin/dd if=/rhev/data-center/mnt/10.xx.xx.xx:_test__nfs__sas_iso/6cd147b4-8039-4f8a-8aa7-5fd444454d81/dom_md/metadata of=/dev/null bs=4096 count=1 iflag=direct vdsm 40707 0.0 0.0 4360 348 ? D< 00:44 0:00 /usr/bin/dd if=/rhev/data-center/mnt/10.xx.xx.xx:_test__nfs__sas_export/58d9e2c2-8fef-4abc-be13-a273d6af320f/dom_md/metadata of=/dev/null bs=4096 count=1 iflag=direct vdsm is hanging at 100% cpu load If I'll try to ls this files ls will hang. I've made some dump of traffic, so looks like problem with STATID, I've found 2 issues on RedHat web site, but they aren't publically available, so i can't read the solution: https://access.redhat.com/solutions/3214331 (in my case I have STATEID test) https://access.redhat.com/solutions/3164451 (in my case there is no manager thread) But it looks' that I've another issue with stateid, According to dumps my hosts are sending: TEST_STATEID netapp reply is: Status: NFS4ERR_BAD_STATEID (10025) After this host sends: Network File System, Ops(5): SEQUENCE, PUTFH, OPEN, ACCESS, GETATTR Reply: V4 Reply (Call In 17) OPEN StateID: 0xa205 Request: V4 Call (Reply In 22) READ StateID: 0xca5f Offset: 0 Len: 4096 Reply: V4 Reply (Call In 19) READ Status: NFS4ERR_BAD_STATEID Entire conversaion looks like: No. Time Source Destination Protocol Length Info 1 0.000000 10._host_ 10._netapp_ NFS 238 V4 Call (Reply In 2) TEST_STATEID 2 0.000251 10._netapp_ 10._host_ NFS 170 V4 Reply (Call In 1) TEST_STATEID (here is Status: NFS4ERR_BAD_STATEID (10025)) 3 0.000352 10._host_ 10._netapp_ NFS 338 V4 Call (Reply In 4) OPEN DH: 0xa2c3ad28/ 4 0.000857 10._netapp_ 10._host_ NFS 394 V4 Reply (Call In 3) OPEN StateID: 0xa205 5 0.000934 10._host_ 10._netapp_ NFS 302 V4 Call (Reply In 8) READ StateID: 0xca5f Offset: 0 Len: 4096 6 0.000964 10._host_ 10._netapp_ NFS 302 V4 Call (Reply In 9) READ StateID: 0xca5f Offset: 0 Len: 4096 7 0.001133 10._netapp_ 10._host_ TCP 70 2049 → 683 [ACK] Seq=425 Ack=901 Win=10240 Len=0 TSval=225608100 TSecr=302215289 8 0.001258 10._netapp_ 10._host_ NFS 170 V4 Reply (Call In 5) READ Status: NFS4ERR_BAD_STATEID 9 0.001320 10._netapp_ 10._host_ NFS 170 V4 Reply (Call In 6) READ Status: NFS4ERR_BAD_STATEID Sometimes clearing locks on netapp(vserver locks break) and killing dd\ioprocess will help for a while. Right now I've my test setup in this state, looks like lock problem is always with metadata\disk check, but not domain itself, I can read and write other files in this mountpoint from the same host. Hosts have 3.10.0-693.11.6.el7.x86_64 kernel, ovirt 4.2.0 can't find out If it's Netapp or Centos bug. If somebody wants to look closer on dumps, I can mail them directly. --

2 3

Official Hyperconverged Gluster oVirt upgrade procedure?
by Hanson 23 Jan '18

23 Jan '18

Hi Guys, Just wondering if we have an updated manual or whats the current procedure for upgrading the nodes in a hyperconverged ovirt gluster pool? Ie Nodes run 4.0 oVirt, as well as GlusterFS, and hosted-engine running in a gluster storage domain. Put node in maintenance mode and disable glusterfs from ovirt gui, run yum update? Thanks!

4 4

noVNC console is not work
by Pym 23 Jan '18

23 Jan '18

------=_Part_174803_217249912.1516621712629 Content-Type: multipart/alternative; boundary="----=_Part_174804_192097289.1516621712629" ------=_Part_174804_192097289.1516621712629 Content-Type: text/plain; charset=GBK Content-Transfer-Encoding: base64 SGk6CgoKSSdtIGluc3RhbGxpbmcgb3ZpcnQtNC4yIHRocm91Z2ggdGhlIHNvdXJjZSBjb2RlLCBh bmQgbm93IEkgd2FudCB0byBzZWUgdGhlIHZpcnR1YWwgbWFjaGluZSB0aHJvdWdoIHRoZSB3ZWIu CgoKSSBzaGFsbCBiZSBjYXJyaWVkIG91dCBpbiBhY2NvcmRhbmNlIHdpdGggdGhlIGh0dHBzOi8v d3d3Lm92aXJ0Lm9yZy9kZXZlbG9wL3JlbGVhc2UtbWFuYWdlbWVudC9mZWF0dXJlcy92aXJ0L25v dm5jLWNvbnNvbGUvI3VzaW5nLW5vdm5jIG9wZXJhdGlvbi4KCgpUaGUgc3BlY2lmaWMgY29uZmln dXJhdGlvbiBpcyBzaG93biBpbiB0aGUgZmlndXJlCgoKCgpXaGVuIEkgY2xpY2sgb24gdGhlIENv bnNvbGUgZnVuY3Rpb24gb2YgdGhlIHZpcnR1YWwgbWFjaGluZSwgdGhlIG5ldyBwYWdlIHdpbGwg YWx3YXlzIGRpc3BsYXkgTG9hZGluZy4KCgpUaGUgdXJsIGZvciB0aGUgbmV3IHBhZ2UgaXM6aHR0 cHM6Ly8xMC4xMDAuMjE5LjM3Ojg0NDMvb3ZpcnQtZW5naW5lL3NlcnZpY2VzL25vdm5jLW1haW4u anNwP2hvc3Q9dGNoeXAuZWNyLmNvbSZwb3J0PTYxMDAmdGl0bGU9dm0tMyUyMC0lMjBub1ZOQy4K CgpFeGN1c2UgbWUsIGlzIHRoZXJlIGFueSBvdGhlciBjb25maWd1cmF0aW9uIHRoYXQgbmVlZHMg dG8gYmUgY2hhbmdlZD8KCgpUaGFuayB5b3UuCgoKCgoKCgoKIA== ------=_Part_174804_192097289.1516621712629 Content-Type: text/html; charset=GBK Content-Transfer-Encoding: base64 PGRpdiBzdHlsZT0ibGluZS1oZWlnaHQ6MS43O2NvbG9yOiMwMDAwMDA7Zm9udC1zaXplOjE0cHg7 Zm9udC1mYW1pbHk6QXJpYWwiPjxkaXYgc3R5bGU9ImxpbmUtaGVpZ2h0OiAxLjc7Ij48ZGl2IHN0 eWxlPSJjb2xvcjogcmdiKDAsIDAsIDApOyBmb250LWZhbWlseTogQXJpYWw7IGZvbnQtc2l6ZTog MTRweDsiPkhpOjwvZGl2PjxkaXYgc3R5bGU9ImNvbG9yOiByZ2IoMCwgMCwgMCk7IGZvbnQtZmFt aWx5OiBBcmlhbDsgZm9udC1zaXplOiAxNHB4OyI+PGJyPjwvZGl2PjxkaXYgc3R5bGU9IiI+SSdt IGluc3RhbGxpbmcgb3ZpcnQtNC4yIHRocm91Z2ggdGhlIHNvdXJjZSBjb2RlLCBhbmQgbm93IEkg d2FudCB0byBzZWUgdGhlIHZpcnR1YWwgbWFjaGluZSB0aHJvdWdoIHRoZSB3ZWIuPC9kaXY+PGRp diBzdHlsZT0iY29sb3I6IHJnYigwLCAwLCAwKTsgZm9udC1mYW1pbHk6IEFyaWFsOyBmb250LXNp emU6IDE0cHg7Ij48YnI+PC9kaXY+PGRpdiBzdHlsZT0iY29sb3I6IHJnYigwLCAwLCAwKTsgZm9u dC1mYW1pbHk6IEFyaWFsOyBmb250LXNpemU6IDE0cHg7Ij5JIHNoYWxsIGJlIGNhcnJpZWQgb3V0 IGluIGFjY29yZGFuY2Ugd2l0aCB0aGUgaHR0cHM6Ly93d3cub3ZpcnQub3JnL2RldmVsb3AvcmVs ZWFzZS1tYW5hZ2VtZW50L2ZlYXR1cmVzL3ZpcnQvbm92bmMtY29uc29sZS8jdXNpbmctbm92bmMg b3BlcmF0aW9uLjwvZGl2PjxkaXYgc3R5bGU9ImNvbG9yOiByZ2IoMCwgMCwgMCk7IGZvbnQtZmFt aWx5OiBBcmlhbDsgZm9udC1zaXplOiAxNHB4OyI+PGJyPjwvZGl2PjxkaXYgc3R5bGU9ImNvbG9y OiByZ2IoMCwgMCwgMCk7IGZvbnQtZmFtaWx5OiBBcmlhbDsgZm9udC1zaXplOiAxNHB4OyI+VGhl IHNwZWNpZmljIGNvbmZpZ3VyYXRpb24gaXMgc2hvd24gaW4gdGhlIGZpZ3VyZTwvZGl2PjxkaXYg c3R5bGU9ImNvbG9yOiByZ2IoMCwgMCwgMCk7IGZvbnQtZmFtaWx5OiBBcmlhbDsgZm9udC1zaXpl OiAxNHB4OyI+PGJyPjwvZGl2PjxkaXYgc3R5bGU9ImNvbG9yOiByZ2IoMCwgMCwgMCk7IGZvbnQt ZmFtaWx5OiBBcmlhbDsgZm9udC1zaXplOiAxNHB4OyI+PGltZyBzcmM9ImNpZDo2ODMwMDYzZiQy JDE2MTFkYjJjY2Y0JENvcmVtYWlsJHB5bTA5MTQkMTYzLmNvbSIgb3Jnd2lkdGg9IjUwMCIgb3Jn aGVpZ2h0PSIyNTkiIGRhdGEtaW1hZ2U9IjEiIHN0eWxlPSJ3aWR0aDogNTAwcHg7IGhlaWdodDog MjU5cHg7Ij48L2Rpdj48ZGl2IHN0eWxlPSJjb2xvcjogcmdiKDAsIDAsIDApOyBmb250LWZhbWls eTogQXJpYWw7IGZvbnQtc2l6ZTogMTRweDsiPjxicj48L2Rpdj48ZGl2IHN0eWxlPSJjb2xvcjog cmdiKDAsIDAsIDApOyBmb250LWZhbWlseTogQXJpYWw7IGZvbnQtc2l6ZTogMTRweDsiPldoZW4g SSBjbGljayBvbiB0aGUgQ29uc29sZSBmdW5jdGlvbiBvZiB0aGUgdmlydHVhbCBtYWNoaW5lLCB0 aGUgbmV3IHBhZ2Ugd2lsbCBhbHdheXMgZGlzcGxheSBMb2FkaW5nLjwvZGl2PjxkaXYgc3R5bGU9 ImNvbG9yOiByZ2IoMCwgMCwgMCk7IGZvbnQtZmFtaWx5OiBBcmlhbDsgZm9udC1zaXplOiAxNHB4 OyI+PGJyPjwvZGl2PjxkaXYgc3R5bGU9ImNvbG9yOiByZ2IoMCwgMCwgMCk7IGZvbnQtZmFtaWx5 OiBBcmlhbDsgZm9udC1zaXplOiAxNHB4OyI+VGhlIHVybCBmb3IgdGhlIG5ldyBwYWdlIGlzOjxh IGhyZWY9Imh0dHBzOi8vMTAuMTAwLjIxOS4zNzo4NDQzL292aXJ0LWVuZ2luZS9zZXJ2aWNlcy9u b3ZuYy1tYWluLmpzcD9ob3N0PXRjaHlwLmVjci5jb20mYW1wO3BvcnQ9NjEwMCZhbXA7dGl0bGU9 dm0tMyUyMC0lMjBub1ZOQy4iIF9zcmM9Imh0dHBzOi8vMTAuMTAwLjIxOS4zNzo4NDQzL292aXJ0 LWVuZ2luZS9zZXJ2aWNlcy9ub3ZuYy1tYWluLmpzcD9ob3N0PXRjaHlwLmVjci5jb20mYW1wO3Bv cnQ9NjEwMCZhbXA7dGl0bGU9dm0tMyUyMC0lMjBub1ZOQy4iPmh0dHBzOi8vMTAuMTAwLjIxOS4z Nzo4NDQzL292aXJ0LWVuZ2luZS9zZXJ2aWNlcy9ub3ZuYy1tYWluLmpzcD9ob3N0PXRjaHlwLmVj ci5jb20mYW1wO3BvcnQ9NjEwMCZhbXA7dGl0bGU9dm0tMyUyMC0lMjBub1ZOQy48L2E+IDwvZGl2 PjxkaXYgc3R5bGU9ImNvbG9yOiByZ2IoMCwgMCwgMCk7IGZvbnQtZmFtaWx5OiBBcmlhbDsgZm9u dC1zaXplOiAxNHB4OyI+PGJyPjwvZGl2PjxkaXYgc3R5bGU9ImNvbG9yOiByZ2IoMCwgMCwgMCk7 IGZvbnQtZmFtaWx5OiBBcmlhbDsgZm9udC1zaXplOiAxNHB4OyI+RXhjdXNlIG1lLCBpcyB0aGVy ZSBhbnkgb3RoZXIgY29uZmlndXJhdGlvbiB0aGF0IG5lZWRzIHRvIGJlIGNoYW5nZWQ/PC9kaXY+ PGRpdiBzdHlsZT0iY29sb3I6IHJnYigwLCAwLCAwKTsgZm9udC1mYW1pbHk6IEFyaWFsOyBmb250 LXNpemU6IDE0cHg7Ij48YnI+PC9kaXY+PGRpdiBzdHlsZT0iY29sb3I6IHJnYigwLCAwLCAwKTsg Zm9udC1mYW1pbHk6IEFyaWFsOyBmb250LXNpemU6IDE0cHg7Ij5UaGFuayB5b3UuPC9kaXY+PGRp diBzdHlsZT0iY29sb3I6IHJnYigwLCAwLCAwKTsgZm9udC1mYW1pbHk6IEFyaWFsOyBmb250LXNp emU6IDE0cHg7Ij48YnI+PC9kaXY+PGRpdiBzdHlsZT0iY29sb3I6IHJnYigwLCAwLCAwKTsgZm9u dC1mYW1pbHk6IEFyaWFsOyBmb250LXNpemU6IDE0cHg7Ij48YnI+PC9kaXY+PC9kaXY+PGJyPjxi cj48c3BhbiB0aXRsZT0ibmV0ZWFzZWZvb3RlciI+PHA+Jm5ic3A7PC9wPjwvc3Bhbj48L2Rpdj48 YnI+PGJyPjxzcGFuIHRpdGxlPSJuZXRlYXNlZm9vdGVyIj48cD4mbmJzcDs8L3A+PC9zcGFuPg== ------=_Part_174804_192097289.1516621712629-- ------=_Part_174803_217249912.1516621712629 Content-Type: image/png; name="image.png" Content-Transfer-Encoding: base64 Content-Disposition: inline; filename="image.png" Content-ID: <6830063f$2$1611db2ccf4$Coremail$pym0914$163.com> iVBORw0KGgoAAAANSUhEUgAAAfQAAAEDCAYAAADDQfYrAAAgAElEQVR4Ae3dD1RUV54v+q+JFuk0 aGJ5ozBRiNp4k1WkmSla/rSzxGQsfAw4fSnX5cJbq8Uh4nQGtANObE2jIq2GrBSOkR4f2K6Ud96D ME9Y3cJiSTkdcY0BScKEGeqZhjYKOgOxhyKJmHQoTXhrn6pTdeofVWqBBXxPL7pOnbPPPr/92RV/ Z+9zCua8/fbb49euXcOuXbvAhQIUoAAFKECB6SnwyLfffotvvvlmekbPqClAAQpQgAIUkAQeuXv3 LhM6PwwUoAAFKECBaS7wiBidi6TOhQIUoAAFKECB6SsgjdDFtDsXClCAAhSgAAWmr4A0Quc99Onb gYycAhSgAAUoIASkETqn3PlhoAAFKEABCkxvAT7lPr37j9FTgAIUoAAFJAHeQ+cHgQIUoAAFKDAD BB4ZHx8HH4qbAT3JJlCAAhSgwKwWeOSzzz7D7du3ZzUCG08BClCAAhSY7gJzlyxZgq+++mq6t4Px U4ACFKAABWa1gDTlLqbduVCAAhSgAAUoMH0FHpm+oTNyClCAAhSgQGgLTPT3UsTvgAnmM2zSCD20 ORgdBShAAQpQYPoJ3LlzB62trTh9+jTGxsZcGiBuddfX1+Of//mfg/br12fdCH3k7KtYt24dXj07 4oIbEm9uv4/XM9Yh4/X3wccUQ6JHGAQFKECB+xYQt7PFL277l3/5F/z61792JHWRzMX79vZ26Y+j Beu299z7jtTPgdbr76Kmxoiz793Al5iHJ1esxgt5BShYswwqP8eG9u4RdJ+uQk3tRXz82R1g3pNY sfoF5BYU4IVl99Cy25dxpvYKVuZuxHPhcotVWLhyBVYuC5/mRnJ7+EoBClBg9gqoVCqkpaVJAGKk Lpa/+Iu/kEbtIpn/5V/+pfR+3rx5QUGalIRuvVyFl/62ATfwXTy7To/48E9x8d33cHmkcJonqk9x 9tUfo+KDO5j35A+QmbMSuHIWZ99rQPl77+PKL3+FgucCS+q336/FkTordm3aiOfkrgyPR8Hf/0p+ x1cKUIACFJjmAu5JXYzWxRLsZC7qnIQp909xtkok88XQH2nEP+wtREHxL/C/m1vx9xuX2LtmBN21 P8ePM9ZJ098ZP/45arudU+CXq9Zh3f+qRfeV03g1S2cr83IlLn5qPxyux+uyXsLPay/bp6ld97nX LdfgfFWW1yHr5UqcvW517las3b5YIyVz/GAX/nfjGyguKEDxG414p3Qd5uEG6mov2mIYOYtX163D S6ffx8XKl5Gxbh10Wa862jhy8XX89PX3AHyACr0weBXSHQD7cetePQubhhXXz1biZbuBLutlVJ69 Dkd09vIH3r3iOM86XRZePX3FUWakuxY//3GGZCj2vfTzWlzmfL6iV7lKAQpQYHIFRFL/8z//czz7 7LPSiTQaDdasWYNgjczl6IOf0Ee6cfFjACtysTFeOVpVOUbnV2p/ildOvIeRlZnIycnEypH3cOKV n6LmsiNVATdP4JWX38XCDZug/+FSfPlxE0pr7Anzci1ePfEerKtfQcWRUhSuWQjrwoUQM9cB1S23 3lH+fYRvLMWRikKsvt2Eipeq0K0IRS5+vfu8tJq56QXIlyZiw8IXNmKjWHnvMq7IhQF88sufw3j7 OWzUr8Oy2x/gxKtVuHgbUC1bjfhlouAK6EuP4MiRPMQ7pt2dFdy++DpeqmjClYVroM/RY83CK2iq eAk/f9d58SNKny/fipqR57AxJw3P4jN88Muf47QUyGXUvnoC71lX45WKIygtXIOF1oVY6OVczrNy jQIUoAAFgikg7pm3tLTg448/xlNPPQWz2Sy9d39Q7kHPOSlT7lJQC1VSgvUM8DLeNd4AkIm9bxRj tcj5LyzB/9p6AnVnu5H33GrHIT/82Rv42QvhgDUe1rRdaLp92zHyFIVuXnkf160F2FD8hi2hIvC6 bSeRy+tRUPAC4gE8V3gFrbua8G53IeKl4BzhOFZU4coLFbFZfn8dnypz7YpC7N27ESJ3b1h2HZuP nMe73cVYsyYeKxeKjL8QK+PjES/WPZYRvH/mPO5gBV7ZuxcbRSW5z8GaWY73ai/i+gu2eqXDFm/F 3l/kYiWA60uuYPORTzCivCC5eQXvX7eiYEMx3pCuPDxOxg0UoAAFKDAJAsoH4MQ0uxipnz9/XrqP Lk73ox/9CGFhYUE5c/BH6HJYVrgkX3kzRj7F9TsAfvAcVsp5cOFCKemJbKicDV6yxD6UVLk9JPZc AX61KxPPjryHX+7ajEzdS3hdzMffQ91SPHJ5NOCVdbbp/7RdTdKuT0eUkTiil1ast5XZUmyS3y/D EmVyVlzUhKtsO257HOtat/Pdp/j0sngXj5XSaB5A+BLbzMAn112csGwh5NPK57HV8xwKfrULmc+O 4L1f7sLmTB1eev0iHHcunCfjGgUoQAEKBFngj3/8I86cOSM9zS7fMw8PD5celBMPy4n76b/5zW9g tco55MECCH5CX7gS8YsB/Fst3lXOP8txLlyCZeKBvg8u44rchpERXBf7l9imzeWivl9VWLahGP/Q 3Iq6I3o8e+cTtB44g8v3WrdcfulW/LKpCU2Kn70vyCnSGcWy+HXSm6bT77okxU/P1uKM2PPD56RR suOIEasj8Y6MSC1EuMfo3lHabWUJlkhPy3Xjiu1Q4PantvOuWOZj9sOtCjF3sGwDiv+hGa11R6B/ 9g4+aT2AM9KFgmdZbqEABShAgeAJiJF3ZGQkMjIyXJ5mlx+UE0l96dKlmDs3OJPlwanFpf0rsaHg BzCWf4ATL2fh4poXEL9wBJe7u6Ha9Cu8seE5bCh8FnVHmnDgVWDjc8DlM024iaXI2RDvmLx2qdL9 zeVavHz6NjZuWI0lt1WAuEBYuQQLca91P4cX8pai7oQRlZUq5G5ciYUj1/F+twprijc4nz63nz98 TQF2/eAiKj6owI+zzmDNCyL4d3Hx489wZ94K/G3eC66J9pMqHDhwHasXiunzm8C8dXjBfrN8iRh1 f9CN01VVuL5kCVZv2iRN+TubuhBrNmVi3gdNqDpwANdXL8TI+2fwHubhB7lrbDMazsI+1i6j9uXT uL1xA1YvuQ2VDcp1FsHHkdxMAQpQgAIPJvDII49g7dq1XisRST0zM9PrvvvdOAkJXTwk9gbeWXga VTW1uHi+AR/bv4e+yf497WUb38AJVQ0qq5pQ92/Ad5f+EFt/UYzcAL/yhfBwqK7UomJXHUQ2f/LZ TJTu3Wibjr7Hulfm/gNOLKxBZU0Nys/L3ysvsN+Td2ddgg1vvIMl9u+hn2/42PY99B9uRWFxrue9 8MxC5OIsKhs+hvXJH+Bv3/gZ1tjvIsRvKsW6d1/H+fMNuP7kOjy3yf1cgGp1Md45sgSvHzCioU58 Ve5ZZO76GQq9zB54Hi22hCNcdQW1FbsgSYnjS/fC8WUD7wdxKwUoQAEKTEOBORUVFePXrl2DwWCY huGHaMji62T6Cnzwg11oeGOD4/52iEbLsChAAQpQYAYIBP8e+gxAYRMoQAEKUIAC002ACX269Rjj pQAFKEABCngRmJR76F7OM7s2LdyAN85vmF1tZmspQAEKUOChCnCE/lD5eXIKUIACFKBAcASY0IPj yFooQAEKUIACD1WACf2h8vPkFKAABShAgeAIMKEHx5G1UIACFKAABR6qABP6Q+XnySlAAQpQgALB EWBCD44ja6EABShAAQo8VAEm9IfKz5NTgAIUoAAFgiPwyNDQEIaHh4NTG2uhAAUoQAEKUOChCMxd tGgRbt/2/be/H0pUPCkFKEABClCAAvckwCn3e+JiYQpQgAIUoEBoCjChh2a/MCoKUIACFKDAPQkw od8TFwtTgAIUoAAFQlPA8cdZHn/88dCMkFFRgAIUoAAFKOBXgCN0v0QsQAEKUIACFAh9ASb00O8j RkgBClCAAhTwK8CE7peIBShAAQpQgAKhL8CEHvp9xAgpQAEKUIACfgWY0P0SsQAFKEABClAg9AWY 0EO/jxghBShAAQpQwK8AE7pfIhagAAUoQAEKhL4AE3ro9xEjpAAFKEABCvgVYEL3S8QCFKAABShA gdAXYEIP/T5ihBSgAAUoQAG/AkzofolYgAIUoAAFKBD6Akzood9HjJACFKAABSjgV4AJ3S8RC1CA AhSgAAVCX4AJPfT7iBFSgAIUoAAF/AowofslYgEKUIACFKBA6AswoYd+HzFCClCAAhSggF8BJnS/ RCxAAQpQgAIUCH0BJvTQ7yNGSAEKUIACFPArwITul4gFKEABClCAAqEvELSEPtxlRIk+FQkJCUhI 1aPE0I5hf+0fbkJRQgKKmvyW9FeT3/3u8W3b04DeMb+HTUqBHkMCEhIM6Lnn2gfRdigPqcI4JQ2H 2kfvuQaXA3oMSEgowuTyj6G/1eD8bKSkocjQjkEpkGE0FSUgwXDvEi7t4BsKUIACFMDc4eFhfP75 5w9GMdyEsm1V6IrVIWdzJFSjfWhDGCIerNYAjx7DYI8ZFrUGcVFhXo8ZbduDjTtNgCK+5gEgwntx r3WExMaeOuxpNEOdtQ8GbQRUcQ8qrIJq0hsWhrGhLgyt0mNfSSxGG2pQWbcde2LOwKifivNPegN5 AgpQgAIhITD373vHgZEHjGWoDx0AknOKUZK5SKqs6AGrDPjwsS4Yt2zH0L6zOOY1ofeiwWCCNTIH 1W+XQGtP4lMWX8AN8V9wuL8fVgCpmZnQxvkv77dEZAy0sCLK1mV+i99vgVV5taiVD461ouNCGboG LQDiEBUDaGPV8l6+UoACFKDAfQoEZ8o9RgudCug4vBPH2vvhOhE8iHZDEdJSxDRzCtLyDqHNNt/q GfJYLxr22KeUE1KhL6lVTIsr60lAqn4/2vq7YNiyE40AOso2eJ8+HuxDx5DIgjpHMnc98TC6jK7x NfXLc/E9MCQkYE9TO2qL0pCSkICUtCLUOubqbVPgjrbllqDBvm+svwmH8hS3IIxdPm9BjPU2YI+y bG0v5AjkWEd7jHjNIC6bgLotCUgoapLqE7cSitJSpFsdKWl5ONTU7zhWmto/1I7+tkPIS/Vya2NR Jo59WAItRtFaIqbxD6FLPnGvEZkJCcg09gLSrZFMGFvb7G1KQVpRLXr622Gwx506Ub/KjcAYBrs6 0IVo5KTGSlvjSj5EdWaUowRXKEABClDg/gSCk9AjUrG7uhhalRmntm9CWu5+yEmx11iE7Q1ATvUZ nD9TjZyIZuwsqkW/R7yjaCvbgsO9sdh9+izOnt4NbX8lthxuky4QpHrquhCp34e33toHfUoyYiMj octIlmqKzTmI6uoCaN1noS196BKzB7GRHmcUG3qN27CtqguR4viDhdCMNqIs1+BMbABMZQb0pe7D sX1ZiLR0oLKsWYp/tK0KOxvN0BS+herqN1Go1SImJgwYbUNZbhkaR1Ox+623sC8zAh1V27CzwcuV jCi75TB6Y3fj9NmzOL1bi/7KLTjc5npZpFJrkZVpS4Jri6tRXaBFRK8R28StjsgcHKw+iELNKBrL cmFwZGUAzTuRu+cCkJIDfZyvoXgEUnOzoLI2otZ+3t6OBgxBgzzdKrvbEKrKjIjIMeBgTiwsHZXY ssmA0cxDOLhZA6u5EXsafN8Ltz038ENsfK0X+reqURQ33e53eP34cCMFKECBkBGYG6xIIuJyUd2W id4mA/aUNaOsKAaxTcnoaBgArAOo2rIRVY6TdaBnOBcxjvcARrtgMokJ5Ua8tkmMue3LQBf690ei S9QTuxv7SzKl41JSbPuj4myjO3WsFlqtl4QVEQORBjv6xDDdfX+vLT5Rb1GaVG+k9QIulDWirW+P c1o7uQCF+hQsQizyTI0o67DNQsTEJiNZZcKFqjJYMwqQk5spzQKMdpggmpJVshv6lDAgJQx9ddtQ 19aFYb3raHS0y1YWja/Btdn92J/qnFcPi4pzTE1HxWml2HqNDRhALHbvL0KawIy04sKFMjS29WGP PCdvjUXJaSP0LtgyrvM1TKtHQWQjqkztGE2LQnPNELC2EDoRrv2ZRVVOCYrS4gCtHs11ZnTk7MF+ vRZAGMyntqGurx/Dw/0o21Am3YIBcvD2hyUQrYjRV6M6dQyDbTU4vH0juorfRm2ufLHgjINrFKAA BShwfwJBS+i200dgVeZ+7Olrxra6LvQN20bPiC7E20a9SwJXiZG0t4fbdQdxZneK4oE6FSLQL42y 76uJkTHQqIC+ti70lsThvlJITJTHpYAUS1Qmjp3XoretDadOGbB9kwEZb7ai5D4C1R08g90piukF Ceg+KvI4JA6xfpK57ZBV0OVpUHW4GaaGSDRYVcjKTVX0A6CNcZ/lkB+pk18BRGhRUF2NPKlStaPP I2K00Ir75Vr7xU1HH4ZzV3l39WgDN1CAAhSggD+BoCT04aYibGqIQKY2EipY0NUAIFqL2EWrAH00 qqpqYDBGoihlETA2iPbROBSlKZKXiDJCC51OBZOpBjXaCGSKqevhfvQuWotcbQziklVAowH7DSoU JANdzV2ILd6PNHsL+zsuoF2tAmIykaIcBIdpkVuiQePhKmzJ7YM+WcQ4iv4hLXYfSkOyFJ8B+4+F oyjWglpjF6DKgv0W74R+w23HcGpUi9SoVdDnZKCnrBGWsTFEpOigU5nQaDiMVdBB1VEDQaJJ1UoJ TMwVyEuE1lbWVFMDbUQmbM3uxaK1uT7u+ctHAquS9YiuqoJh/zGEF8XCUmtEF1TICiR4ZzWOtShd DtYefg0NDbGwRhZALz9B6CgRwEpYFOK0yg7ogbGoCSpdKlZFjaK34ZRkoYqMcrlYCKBmFqEABShA gQkEgpLQw8Toy1qHhlMWWBGO6LWFqN6dZxsN51WjGodxyPgatp0CEB6NtSXHvIQUgdR9tdgXvh9V hu1otgIqtQY5h3TSlK62pBb7sB9VDWXYXqeCWpODQ+I2bFwODupMKDMdxvb2aBQfc0vo9une0yoD 9lc1o+6UVYpBq0+DOHyVMr7bot4s7DM4n4b3EqhjU1gE0HN4J+osUrDQZL2J3WliWj8V+2r3IXx/ FQzbm2EVbS6sxm636Xapogi3srC3TTTb37IqD9XVwOFDRry27bbklbXPgJL7ScTiXBGpyM1SYVtj HzS737y/2QyPmNVQq8yoOtwIwST1f85bKCnRSv4exbmBAhSgAAXuS2AO/o8d4xj5T4xf+n/vqwIe RAEKUIACFKDAwxcIzlPuD78djIACFKAABSgwqwWY0Gd197PxFKAABSgwUwSY0GdKT7IdFKAABSgw qwWY0Gd197PxFKAABSgwUwSY0GdKT7IdFKAABSgwqwWY0Gd197PxFKAABSgwUwSY0GdKT7IdFKAA BSgwqwWY0Gd197PxFKAABSgwUwSY0GdKT7IdFKAABSgwqwWY0Gd197PxFKAABSgwUwSY0GdKT7Id FKAABSgwqwWY0Gd197PxFKAABSgwUwSY0GdKT7IdFKAABSgwqwWY0Gd197PxFKAABSgwUwSY0GdK T7IdFKAABSgwqwWY0Gd197PxFKAABSgwUwSY0GdKT7IdFKAABSgwqwXm/uKH/w39/V9OOcKtW7fw xRdfYGxsTDp3WFgYFixYgPnz5wcWS08tKlpuOMvOj0feT9Kw2H07liJ9Vy7ipJI9qK3oQZzjvdgo trVAURPmx+fhJ2l/8NgO+RzOswZ17fr16/iP//gPCBuxCIulS5dKP35PZK5EokmHzmKNS1FzZSJO xLbgaMwpJOYPoLTlKDLUziKW5h04FXMUzsMsaN6RjvJLzjLIPulRr2LvA68+6GfhZutxGLttZrZg lH3+wOFNYgWen70H/4x5+4xPYhNYNQUoEDICc6c6kvHxcdy8eRPffPMNwsPD8eSTT0ohWK1WjI6O 4quvvsLixYsxZ84c36HdbMXxFiB91y57onYrujQdu3JtKVz6x/54K54Syd6tmJzMv4jPw640971/ AFwuBjwODtqGb7/9Ft3d3ZLJkiVLsHz5cqnuL7/8EoODgxgeHsb3v/99PPLIBBMqGh2y800wF2vg TOlmmOqTsL5FDQwBSUlA+SkzMpzZ260N9mQefRKdR521uBUK2tugfBbs0dguwux9KC7qfPZ50MIP UkWuFx/S57WiVnERGqTTsBoKUGDGCzxisVikkfJUtVRO5iKRi1G5SFLi57HHHpOS+507dzA0NDRx OH+w4NZ8NZ6auJS0d3HaGiy9dRXdNz0L99S2QCTzn3gkc8+yk7nlo48+kpL5M888I43K586dC/Ej ZixWrFgBcbEjEv7Eiwa67HqYzIpSZhPqk9YjWR6Rr9+K0oF8VCrLKIpbmg+gXCRznwlfUTgIq0H5 LHiLIy4OS29ZIC7Jpma5idbaVnj5iN3z6Ren/QTpS2/gYmswanM7fU8tanvctvEtBSgwYwSmdIT+ +eefS1PsCxculBKYN8UnnnhCGpGKsmLd6yL+wW5pQWNrfIDJeAGi3AfguInBL+Zj+YseO7yecrI2 Xrt2DWIkLhK3uP0gEndfXx/++Mc/Ij4+XhqZP/300/j9738PUVYkfV+LRpeNfJMZxRrb6NpsqkfS +hbI+fxSH3B0byl2pFfC3FmsGMnbahzqu4Rs3VFf1Qd1e9A+C16iutl6ETeWrkGufV9PbQUcd2cU szc9tccxGLccV1u6IU3Yi1sqWUCj0f7ebYbGpR7HvptoPW6EmPHvruiGY8pczCI56pmP+LyfINDr xri4pWi52I2bsM0quZxXEb/7rYal6btgn5hyqEjHfhGPvDUWGCWEClS0AHJZl7odbQIg4v8tsPyL bqltokKXWRDHGbhCAQqEisCUJvSRkRF897vf9ZnMZRQxWv/ss898J3TEIXdXHMQ/RhUVYmbcOcUu 1yG/Sv+4z1+OF+UNjtc/wHILjmTn2OyycgMtFRVosW+bjH/QPvnkEyxatEgahf/7v/87WlrkswED AwMQMxbPP/+8NFoXZSdK6HCZdldMtyvbpM7A3tIdSK80u43ELegfUBac3PXgfRZscd7qNkLkU2lR fB5E0ruozsOuXNuFm3h/vPUp+4XgLXRfBPJ27ZJux4jPk7Ex3uW9GCnHpS2GOK4F6di1y/NWTtpP 0mE5PogXHbd1elBrtGCN45aQuK9dix6XZzfssXp7eUoN+UkS3/H/Ab+9uhx5u7zdShKV2i40ri53 tj1v8Dh+G/UTR9KfqE2S1o2rAge7pDeiDUbURnleNHhrArdRgAJTLzClCV2MOsWDXuLe6UTLvHnz IG4F+Fvicm330MU/TBXHB20PxYmDbrSgQgxDxOLzQbanoJb/1bSV9PL/rvc3vRR44E3iwcBly5ZJ 9fT0eM6HipG5SOiPP/44rl696ud8Yto9HyazGH2bUJ+9FZ3y8FxxpDpjK7ITT6B581EkO7arERMN 9DneT+5KsD8L8sWW+CwYHR+dm+i+egu3bimSvWjWUjEZb0vwS9c4E+JT6vmYr453PGsh3tsWUQ8Q n2VL5mKbdCunokea1rfVZC8qXnp6cEP8T3ExCMyH+iYQ51FYcZy8Km4pSZeaE8Ufh7gFLTBWdDtG 2/LhwBe4eNwILJ/odlIAbZq/HPGOeOPwYvxFNA4G2ghnNFyjAAWmRmBKE/pkNcnjH1fFCM33ORcj asEtXBQ31wOdC/Vd2X3v8Xdxo6xYPDznb5Gn3XWoR7au2EdxDYpPRiPxQDNOrncWiYxNQr1iyt65 Z/qsLU7LQvzxRrTejLN3671NdwetpQF9Br2frafnBuYvf9F+YeE7fvmCVpqpUkyji1qXZ+UBjcIh 8Kl+79FwKwUoMF0EJnhsOvhN+M53viNNIfurWTwEJqbmA17EiCjAh+SUdca9GA90Gx/qg0JixkLc QxdLXJxzBCjH+b3vfU9aFU//i4fk/C5i2r3+BE4MZEM30YPqmmKcjC5HvuL7adLIvT4fO5odQ1y/ p7vfApP2WcBipK1ZgO5G8ZDaYsQvh339fiMVx9nr+a1zBsV2K0fxYKbyITzxjMeNlvv6XNnuaS/F GukiM7D4RWLflb4UX4jRs7SIZ0YWIy1rOa4aa+GMGooygbSpG84m9+C33bewwPNhFPs5+UIBCjxs gSkdoYuH4cRUurhHPtEikpe4r+xz8fpdc+fUqc/j3HcsTsNP8oDjRtuDQrbdyhGR6z1039P37hUH /l488Cae6hcPAIqpdbGIafavv/5aSvDyNvHVNVHW/yKm3S8hH1vh7/E2TfFJZNfnK6rUoLjzJCoT 05FY7tycfbJT8T115/YHWQvaZ8FbEHEvIv6iEb/tSUOueGpc3Bt33GB3PhDm7VBf26Snz6VnNuRn HMTtGPkzF4e4pS225y3st3hy8wZdP1c+b/14+Yw56hVT+77id/8Ou4hHzI8rnu0Xn+/0WlRU1ALi /r24ujHabj9ID8XZ63bcnpIeipPbJG5XLQUuOv/bkG5reF5z+iLjdgpQYIoF5rzyyivj4peZ/NM/ /dOUnPrGjRvSA2DiH3Rvi0j44uts4heqzIZFPNne2dmJu3fv+nzgTdw7F88VJCYmSjYzxYWfhRDu SfGUeyOQ5XjQL4RjZWgUoIAkMKVT7uKMYpQpP/QmHowSv2BG/Ih1OZkHNhKdGT0oLl7EVPujjz6K K1euQHydS9xyED9iXWwTXqKMKDuTFn4WZlJvsi0UoMDDFpjSKXfRWPEb4MRT3eJraSJhia8viUXc MxfT7D6/e/6wpSbx/KLN4jvn4mtqYvpd+atfIyMjER0dHfivxJ3EOINdNT8LwRZlfRSgwGwWmPIp 99mM7a/tYvpd3DsX0+9iEb8tTjxvMNNG5v4cuJ8CFKAABe5dYMpH6Pce4uw5QiRuJu/Z099sKQUo QIFgCkz5PfRgBs+6KEABClCAAhSwCcxVq9XSXzkjCAUoQAEKUIAC01eAI/Tp23eMnAIUoAAFKOAQ eET8whLxtDkXClCAAhSgAAWmr8Bc8bUo8deNouUAACAASURBVGQ1Fwq4CNy2oPv3X0Jc6j3x1CLE /8njLrv5hgIUoAAFQksgqFPuw11GlOhTkZCQgIRUPbbtaUDv2MNpcI8hAQkJBpffYx1IJLbjitA0 HEjpECgz1gVDagJS97djNBjh3B3Gr9/uxpJfXsfPur9C9+9H8bP/uxdL/v7/w6+v3wnGGVgHBShA AQpMgkDQEvpo2x5s3FaFjrAU5GzejM26KPRbgIiZ9cvNJqEL7q3KscEedPUMwnGdFKaGJmUtUpKj EHFvVXkpbcE7J27gf/xhHOLPfKQm/Hf89H8uQKr469pjVvyPut/hnf+0fUfey8HcRAEKUIACD1Eg SN9D70WDwQRrZA6q3y6B1p7Eix5iw2bmqcfQZdyC7UP7cPZYFGzMMUg7ZEBaEBr8aet/IueWs6L/ q7UbZ387jt85Nt3FT1tuYMPWZ/CEYxtXKEABClAgFASCM0If7EPHkBjS6RzJ3L1xYjq+KC1Fmo5P ScvDoaZ++yizB4aEBOxpakdtURpSEhKQklaEWsdc/SDaDuUhLUVMoacgLbcEDfZ9Y/1NOJTnnOIv MXbB10z5WG8D9ijL1vY6R7nuwSreS1Pwe5rQXltkiyElDUXSsaNoLUlAQsohdMnD5V4jMhMSkGns lWqYML6xXjTsyUOquD0h2pVXix5Rz3AXjpXoFdsPoW1QVDeKLsMW7GwE0FGGDQkJKJLuC9j8EgzO P5J5f9afoe3qN46W/1V8DPp3xqPtp/8NP3VsBW6O3Ebb5P91VcUZuUoBClCAAoEIBCehW/rQBSA5 NtL7OXuN2LatCl2ROThYfRCFmlE0luXC4MiEgKnMgL7UfTi2LwuRlg5UljWjX6SxtirsbDRDU/gW qqvfRKFWi5iYMLEDZbllaBxNxe633sK+zAh0VG3DzgYp+7nGIcpuOYze2N04ffYsTu/Wor9yCw63 BXjX2VQGQ18q9h3bh6xICzoqy9DcH4HU3CyorI2otdfT29GAIWiQp1vlJ75RtJVtwWHTKLTFB1F9 sARrdasQK4bci8IRpdZjd3U1qt8qQKy5ETur2jAKFSJ1GUgWLYsVjtUo0HqZZL9v6y/Rrxid918d wtnrroy2d3fxO8Vf6PRWgtsoQAEKUGDqBYIz5R4Rg1gxcOwTw3TPv2MuEt0AYrF7fxHSYgBEWnHh Qhka2/qwR2tvdHIBCvUpWIRY5JkaUdbRLz3kFRObjGSVCReqymDNKEBObqY0CzDaYYLJCmSV7IY+ JQxICUNf3TbUtXVhWB/lIjnaZSuLxtewSYxw7ctAVz/2pwbyB56TUVCoR8oiIDbPhMayDvSPAmFa PQoiG1FlasdoWhSaa4aAtYXQRYl8PkF8ugiYbMHjUG6KNHUuM4z196Pf0gxTUSUsVnugo6MYQxii 4uIgtUwdC61W60UauH/rNXjsUQD2Qfq/3RpD9xfABhnL8fooYuY73nCFAhSgAAVCRCA4I/TIGGhU ANq6YJtsvo/WxUR5TVCIysSx82fw/+zbjAizAds3pWF/oCNrtzB0B8/g/Pnzzp/CQJK5qCQGUZ7X KQBWQZenAS40w9TQhAarClm5qQ/wcFov6opeQ12fBrtPv4cPP3wbOW5tCMpbr9YLEL/YWfv354ch foF4/xjil87F9+Vdj4bhv/+J/IavFKAABSgQKgLBSehhWuSWaIChKmzJ3QPDsWM4duwQSva0Sve0 VyXrEY0+GPYfQ2tXK2oNRnRBhaxUMa6feBluOwaDqR+3F62CPicDkbDCMjaGCK0OOhXQaDiMhvZ2 NBmOoQGAJtVz5CqXNdXUwNTTh76+PrSbmtAX4Iz7RBFG6XKwFh1oaDDDGlkAvf2JQPmcXuOLiIVW AxE89tS2oqu1AYe2GV2+Yjc6ZEZ7QwPavZ28vwMXRJvbPW8v3L/1Y0h9IQLZ9vPFLI/EhmXizSJs ePE7EBMrYsn+06cQb1/nCwUoQAEKhI5AcKbcxRhWX43TKgP2VzWj7pQVCI+GVp9mexJ7VR6qq4HD h4x4bdttqNQaZO0zoER+HH4Cj7AIoOfwTtSJ+WeVGpqsN7E7TQyXU7Gvdh/C91fBsL0Z1vBorC2s xm636Xap6gi3slBBrcnBId0EJw50V0QqcrNU2NbYB83uN7FKPs79nG7x6d+sxujhQzBWvoYLCEf0 2hLkiRF/SRba9jeibJsJmpxD2FPch20dcqVxyDmog6nMhMPb2xFdfAyZKfI+++sDWONPVsKY/jt8 2vJHXOoZQOrHA/ZKbU+6p33vKRhffNLthHxLAQpQgAKhIDCnoqJi/Nq1azh+/HgoxMMYQkHgi5v4 deswjIN3bL8pbsF38DcvRmHDMi8P4YVCvIyBAhSgAAUQtBE6LWeQwILF+NH/XIwfzaAmsSkUoAAF ZrpAcO6hz3Qlto8CFKAABSgQ4gJM6CHeQQyPAhSgAAUoEIgAE3ogSixDAQpQgAIUCHEBJvQQ7yCG RwEKUIACFAhEgAk9ECWWoQAFKEABCoS4ABN6iHcQw6MABShAAQoEIsCEHogSy1CAAhSgAAVCXIAJ PcQ7iOFRgAIUoAAFAhFgQg9EiWUoQAEKUIACIS7AhB7iHcTwKEABClCAAoEIMKEHosQyFKAABShA gRAXYEIP8Q5ieBSgAAUoQIFABJjQA1FiGQpQgAIUoECICzChh3gHMTwKUIACFKBAIAJM6IEosQwF KEABClAgxAWY0EO8gxgeBShAAQpQIBCBh5bQr1+/jvb2dpw9e1b6Ees3btwIJGYAFjTvSERipdmt vBmVO5phcdsaCm8tzTuwozkUIwsFHcZAAQpQgAIPKjDlCf3bb7/Fv/7rv+LmzZtYsmQJ4uPjpZ/F ixdjcHAQH330EUQZ/0sSkgZOIDg50ozKxB1Bqst/5CK5e16M+D+OJShAAQpQgAK+BKY8oYuE/c03 3+CZZ57B/PnzMXfuXOlnwYIFWLFiBaxWK7q7u33Fq9geja1bo1F+4H5G5BY0VyqP06C48ygy1Irq J3FVnXEUncWaBzyDexsesDoeTgEKUIAC01pgShP6tWvX8OWXXyIqKgpjY2Nef55++mncunULouzE ywD6I4txMrocB3wM06WRcGIiEqUfeQQupuvTUV5fjnSxXZqiFyP0SpilqXy5nP3slmbskKfxxbpH fV6iNFfazynO7VafuGHgPv3uUl7EYVvMlTvQ3OytLm9t8BIHN1GAAhSgwKwRmDuVLf3kk0+waNEi aRQ+0XnFaF2UFaN4f4um+CSiEw+gOdlzhC2NhDPsNYikecqMjGINMo6eRN+Ofmw+mgHboFxOoWpk bI12lBNHmk+VI3prJ9QwozK9D1s7O3FUqtJ+EdBZDJextkj6+cDJzk6X7T7vnovyJ2LR0inOIWV7 6QJi71ER+CWUn1vv2CcuBNJ9tsHeTr5QgAIUoMCsFJjShP7FF19g2bJlfqEff/xxXL161W85WwEN NpfCnuhcD5ESYPkl58ZsnXPd15pGh+x8E8zFGmhghqk+G7pikdlNqBf/S6xXHJmEWAugUUzVWzrO AaV7XZK54gCPVVH+0qVLSE8sV+zLxpD9XfZW+aIDUCevR1KfohhXKUABClCAAnaBKU3o4+PjAcMH 9mCcrTp1xl6U7khHpfmks35zJdKl0e1R28hXjNBNzt2+18QFwgmcMgMamDBQutmZnLNPBuHet+eZ k0pbcNTLDXx53sDzCG6hAAUoQAEKuApM6T108RCcuIfub/nqq68gpt0DX9TI2FuKgfx8yONnS/8A EB1jn1IXA2x5j73WS32OUbD7ecRIeOBEJSpPDGB9sn34LUbu9fnw+Kac28HiWJSfctwHd9vt8dZW /sD9PWE/QRs8TsQNFKAABSgwowWmNKGLB95GRkb8gg4PD0OUvadFnYG9pUmOQ6RR+0C+4+E0U2wp nHs10GXXI9/xUJzjMNuKOhnrpQn29ZDzuRivF7eIiwb5ITv5gTr3YzNw9CRsdbs8jOdWTn6rFuWj UZ6uqNffVYN0rJ82yPXzlQIUoAAFZoXAnIqKinHxRPnx48cnvcHiyfbOzk7cvXvX5wNv4t75vHnz pEQcFhY26THxBBSgAAUoQIGZIDClI3SRoOPi4vDoo4/iypUr+Pzzz6Un3sV3z8W62CaSuSjDZD4T Pl5sAwUoQAEKTJXAlD4UJxr1xBNPSL8ZbmBgAENDQ9J3zsV2cX89MjIS0dHR0vpUAfA8FKAABShA gZkgMOUJfc6cOdIDb6tWrZKSt5h+F4v4jXGPPfYYR+Yz4VPFNlCAAhSgwJQLTHlCl1soptQ5rS5r 8JUCFKAABSjwYAJTeg/9wULl0RSgAAUoQAEK+BJgQvclw+0UoAAFKECBaSTAhD6NOouhUoACFKAA BXwJMKH7kuF2ClCAAhSgwDQSYEKfRp3FUClAAQpQgAK+BJjQfclwOwUoQAEKUGAaCTChT6POYqgU oAAFKEABXwJM6L5kuJ0CFKAABSgwjQSY0KdRZzFUClCAAhSggC8BJnRfMtxOAQpQgAIUmEYCTOjT qLMYKgUoQAEKUMCXABO6LxlupwAFKEABCkwjgYeW0K9fv4729nacPXtW+hHrN27cmFw6cyUSK80e 5zBXJmJHswWAGZWJ8rqimKUZO9yOszTvQGJiouKnEp41K+rgKgUoQAEKUGASBab8r619++236O7u xjfffIMlS5Zg+fLlUvO+/PJLDA4OYnh4GN///vfxyCOTcK2h0SE73wRzsQYaB6oZpvokrG9RAxgC kpKA8lMwZxQryjgKSysimaeXR+Nk51GfZVyP4DsKUIACFKDA5ApMQtacOOCPPvpISubPPPMM5s+f L/0ddPG30BcsWIAVK1bAarVKCX/iWu53rwa67HqYlENpswn1SeuRLPK5tKzH1tIB5LuNyOW9sDTj gJTMfSd8R1muUIACFKAABaZIYEoT+rVr1yBG4lFRURgbG0NnZyf+8R//ETU1NXj//felbU8//TRu 3boFUdb3YkbljmY0VyqmvHc0Q0ya2xYLmnd436fRZaNekdHNpnokrU+GI59f6gMy9qJ0IB9ec/pQ Hy5l6zgyl6n5SgEKUIACISEwpQn9k08+kUblYhT+4Ycf4syZM/jd736HgYEB/OY3v5G2iX1itC7K TrhcKse52BbpoqCzswWlKMcp+8jbXJmOc+vlfZ04GV2OdDk7i2n3epP9frd9ut05PLefUo2MvaUY yPe8L27pH5gwLO6kAAUoQAEKPAyBKU3oX3zxBR5//HGpnT09PR7t/f3vfy9tE2VE2YmXbGzNkMfV aiSvT7IXF0lauQ/QbC5F0kC/fQSvmHYX0+3ZW+GoRnlCdQa2ZtfjhPSwnHOHOiba+YZrFKAABShA gRARmNKEPj4+HnCzxcNzk7XI0+5iuj1b53w8zv18muKTiC4/gOYhxZ7IWCQ5RviK7VylAAUoQAEK PESBKU3o4iE4cQ9dLHFxcR7N/t73vidt++qrr6Rpd48CAW2wjcCVI2vzqXJcio5x3ieXpt1P4MRA NibI5wA0KD4ZjfL8clySz20fuee73LOXd/KVAhSgAAUo8HAEpvRra+KBt6GhITzxxBN4/vnnpRaL afavv/5aSvDyNvHVNVH2fhdNcQvW70hHYrm9hqRStBxVjsRF0r+EfGzFUX8n0RTjZHY98hXlNMWd OFmZiHTHCQBkn0RnsfIcigO4SgEKUIACFJhkgTkVFRXj4ony48ePT/Kp4Hiy/e7duxBfW/O2XL16 FfPmzZN+YUtYWJi3ItxGAQpQgAIUoICbwJROuYsELabaH330UVy5cgWff/659L1z8WS7WBfbRDIX ZZjM3XqKbylAAQpQgAITCEzplLuIQ0y3x8fHS19VE9Pv4jvnYhH31yMjIxEdHS2tTxAzd1GAAhSg AAUo4CYw5Ql9zpw50gNvq1atkpK3mH4Xi/htcY899hhH5m4dxLcUoAAFKECBQASmPKHLQYkpdU6r yxp8pQAFKEABCjyYwJTeQ3+wUHk0BShAAQpQgAK+BJjQfclwOwUoQAEKUGAaCTChT6POYqgUoAAF KEABXwJM6L5kuJ0CFKAABSgwjQSY0KdRZzFUClCAAhSggC+BICT0YTQVJSAhxQCXv5823ISihASk GHoA+3pCQgKKmoadsdi3u2wb60fToTykpSQgISEFabklMLQrjnEeDQx3wViiR2qCvWzeHtR2+Sir PG6i9cEmbEtJQW5t/0SluI8CFKAABSgQUgJBSOiLkKxLBqxt6Op1tm20qw0dUEGvc/0jLB2Ha9A+ 5iznutYLY+4mlDX2IyZjMzZv1kOrMmMsLMK1mHg32ISijdtQ1RGGlBxRNgMx/SZUbtuIPW2jnuW9 bhnDYE8XegYVAUXEQJOcjOQ4+U+zej2QGylAAQpQgAIhJRCU76Ev0qZCg8MwmfuRtyoGwCi6LlwA VDmQ8rl90BybnIzRjkYY6vRIyVvlATHaWoOqASB5Xy2OZUbZ95d4lBMbeuoOo8MaiZzqt1Gitf/O 9zwtSta9BpOhAZtT8+B5Breqxrpg3LIdQ/vO4liUvY6IOBQZDG4F+ZYCFKAABSgQ2gJBGKEDiEpG pgboa+iCNFE91oN2E6DS66Acn6t1hShcCwxUVUE58y4T9ZkvAEiGLllO5vIe99d+9HVZAaQiVU7m okhECjJ1AIa60C8uIuxT+vtrm3AoL9U2hV9Ui14xIB/tgmHLTjQC6CjbgISEIltM7rcB7LcAbNP6 qdCXGOGY1ZfKZsLY2mavPwGpeYfQNmiPd6wXtY5bAqnQbzsEX3cP3FvI9xSgAAUoQIF7EQhOQkcU tJmxQF8beoaBsa42NHqZbgfUSCsoRDQ6UHOqB4qJbpF9MShdDcQgZpG/Joyiv89WRuVSNAJq+0y5 SPfy0lzVgIgcAw5ujsVoRyV21vUCqkjoMpKlIrE5B1FdXQCtx8z+KNrKclHWOIrU3W/hrX2ZiOio wradDZBzNjCEqjKjVP9bhcmwmhux09guta23bicqLwC6g9Wort4Nfaw2gLbJUfOVAhSgAAUoELhA kBI6EKPVIxYd6DCPoq+jGVBlIDXWSyCrclCSpcJQnQHNA8r9EVBHivf9ttG1cpfHegRi7HUrE7eY 6rdYbIWViT62ZD+K0rRIKyqBXhrA92E4LApxcbaZAHWsFlptHORZd8fpRrtgMlmBrBLs1qcgJbME RaICc5tzlA5xZ6FEqj8lr0CqH0MWiLv4MdpURGMAjYePoaHdirjNafA39+A4N1coQAEKUIAC9yAQ tISOGC30sYCpvQamNitUGWnQeP1z5mFIKdiNZJhhaBCPzclLGGJWacQEONq6/D3UFoNYrTiyAa3K J+xG29FkAhDpOhIeHXWdC5DPGKxXbYx0JeJRXVhcCRrOn0Z1SRwsDWXYskEPPjzvwcQNFKAABSgQ BIGgPBRniyMGWpHRjW1oG1IhI00Dr/lcFF6UicJCI/7PKhNsd81tNURlFCDLsB2Nr2ViW5cemgjA OtQFdY4Recqb8QDicnYjuaEMjTu34LY+GZGqUZgbGtEFFXQlepcH4oZqDsOgzkFsXx0apHwfC+Ws fn/HBbSrVUBMJlKcVxhAhBY6nQqmRgMOrwJ0qg7UiAo0qdCKCvx8Q66nYQ96wvVYFZWKzXn96Krq gsXftUoQOpVVUIACFKDA7BMIYkIHYjQ6RA5VYQhZSPU+PHcIr8opQVbNdjQq58zDUrDnTDWiDh+C sfGUlJzVscko9PYNsqhMHDsTBaMoW3cKt6GCWqNDcVExct1uhmv1qRhrOIwysxVq3W4cy7E//x6X g4M6E8pMh7G9PRrFxzKR4jLYjkDqvlrsC9+PKsN2NFvDEb22ENW79QFNnatVY2g4vA0DtwGER2Nt 8dsocLswcYBwhQIUoAAFKPAAAnMqKirGr127huPHjz9ANSF6qHgKfUMZIL6Wlqkck4dovAyLAhSg AAUocJ8CwbuHfp8B8DAKUIACFKAABR5cgAn9wQ1ZAwUoQAEKUOChCwT1HvpDb417AIsycezDTPet fE8BClCAAhSYcQIcoc+4LmWDKEABClBgNgowoc/GXmebKUABClBgxgkwoc+4LmWDKEABClBgNgow oc/GXmebKUABClBgxgkwoc+4LmWDKEABClBgNgowoc/GXmebKUABClBgxgkwoc+4LmWDKEABClBg Ngowoc/GXmebKUABClBgxgkwoc+4LmWDKEABClBgNgowoc/GXmebKUABClBgxgkwoc+4LmWDKEAB ClBgNgpM04RuQfOORCTuaIbFo9fMqExMRGKl2WPPQ9lgacYOr3EGHo2leQd2NHu2NPAa7CUDjOVe z3ev5e857hlxgPhcVuKBPpUB9t9kcpkrExEq/2lNZjtZNwWmo8A0TeiCOglJOIcO9zxnNmEgKSm0 +0L8wxzsfxUno87QVpwR0ZkrdyAY12pODDMqH/AC0lkX1yhAgekkMI0TOrB+fTTOuWR0C5pPDEjb p6YTLGiu9DZLMIlntzSjMrgZYBKDvb+qzZUPOJIVpzVXTjCSDH6/BSXme+IKfhvu6fQsTAEKhJzA lP75VKvVio8++ggWiwXffPONV4xHH30UCxcuxJ/92Z9BpVJ5LePYmKxDdPopmDOKoREbLR04h/XY G9MH9DlKQUwT5tfL77NxstNWXkwVn8J6DJSX45K0OwmlLUeRoZbLKl7FCDhdLgdkn2xB7Il0lIsD 68uBpFK0HI3BqR392Ly1D+nihNkn0blZUYefVV9xOg5TxGA7ZQuOeg1WPkLcmrDHKOY0srMBxMo7 AUV9YsbDZ9sdR4hp43w4KKU2Z9j29p3CjsR6u6NoeieKpU6xJddEZwco9onRpKvXSeTb+qpenEXu K9d2SK7FQGWiCTp7X4oghJ9J14liVMJ2vnopVpdYoKjL0W8ZULtYiO70Zas4Xpw0+yS8xYzKHejf vBV96cJLtEPnUPS3Ij6X6dIHS5T01i+KGJRtmKhis2wiCsmu4iPg/t/ARH2XDekjNNF5uI8CFHh4 AhUVFeN/8zd/Mz4VS0dHx/ilS5fGLRbLhD+iTHt7+wQhDY83bd8+3jQ8Pt5jWD1u6LEVHW7ablvv MYyvlje61SLKbBcHjo+Pi/XV25vGbe/GRWXjq1cbxu3VKY7sGTestp1PsVEcMG5QHi/er17teu7h pvHt9jLS+cR+lx9v57PFpoxTXhcxOtblNrjUJ+q31SmdT+Hg2l4Rq/LczveinPIccpvFdkV18mab o7Iu4Si7iPYr90nvZUsvXuOib5Vx2frYMx5nvHIgXj8L8k6XV2/9JsckCtri8tZWcQ7PWLzHLPeD 7dTKeO3tdus3z3rtn0k5EMVnyeOzJ3123T9b9ja5HCd9aNw+kwpvZd9JDoG5uPDyDQUo8FAEpnSE Pjw8jKSkJHz77bceVzD9/f2oqalBQUEBVqxYgUuXbGNmj4JuGzS6bOSbzCjWRKLjXDR0R8VQzbWQ 68hXGkw7CiStT4ZjQK7ZjNKkA+i3ABrHRlGfCfXZW9Gp3OaowX0lGycdQ1PXfeqMo+gUA1oxGjwV g6Nu5SaK07Um5zvfdVrQfA4o3SsPkwF18noknbMfK9ok/pfoGG9Lo8FYCxDprN5lTRw/kJ6IRPvI XMmRVLrZNksijtDokI1+6VhLhxSEc586A1uzy2EaAmzwvr1sJzfDVJ+NrYHhu8Qb8BuP/tVgc2kS Dnh+EO4pluyT9pkjj0A0KO4UMwliVkGM5F1nhVxH6GJAHcDoXlOMzk6pRtusx9EMx+fa0nxO+u8p PbFcEUk2RBeIvvbVd56few102YBJUQtXKUCB0BGY0oT+9ddfY2xszKP1X331FSorK/Ff//Vf+MUv foH9+/dDlA1oEckj3wTz5lici9ZB5HPlIv5xzMdJdHbaEpt4f0BZIETWH0qc4paA20WF4HB/ztBB pM7AUXFFIi5IEhNxSTnl7ijElQcSMFci/dx6tHQetSVkMVUehAzq6xaCz75+oEbwYApQ4GEITOlD cXfu3IG4j+7+c+TIEdy8eVMauaempmLBggUQZQNbxGhqAPnp5YjWOUej8rFDfZeQFCuPOS3oOOc6 8r9Ufso5oDefQvmlaMQoh56iInHRUH/C+9PIl/qkkY58vvt99Renst5LfWJs5W9RI3k9UH7KOV0h RsuO1kttyp/gwbEJ6pcSewtK4b/tYlQPpbGlGSfqk+DoEq+nGZBmSWy7xKiwHie8PgioLCdG8q6V DYgRtq9F2W8e/WvGqfJLiPb8IAQYi6+T+t9u6R8AomMco2uze6OUVSjboNzutm7rgwPeP79uZV3e enGZKByXY/mGAhSYcoEpTehy68SI/O2334Z4feedd9Db2yvt+tM//VP81V/9lVws4FdpKhnZ8JLP oSk+iejydCSK76YnHgDWiwfDnEtSNnBC2peIxPwBlLZ4mybVoLhlPc6J6WZ7Wdu3zmzJJl9su5ev ComE6DYy9henI2J7IhZxuHw33Uud6oy9KB3Id8R8AOvhbL1oUykG8p1t8tcGMYsgtz8xMR3n1ium 2R0Buq2IuE4CkpFwEhdeJ12nmF2PsF+ISNa2p901xS1Yf07uQ/l3DIgLOaDc0ScmxJY6v65oS2K2 Yzy/Iejeb+79m4+B0hbnQ32KAL3H4hmz4pAJVzXFrhbufWaKLYWzVcqq3Nsg79OgWDHdLm2V+iBa YSUbysf4etWg+KTyOFdjX0dxOwUo8HAE5oiH4q5du4bjx49PegR1dXV4/vnn8eabb+L69etYtGgR xH11sSxduhR/93d/h8cff1x6L+6h5+TkTGpMtun3vX6eFJ/UEFg5BShAAQpQICgCUz5CFwn76aef loKXk7nYtmXLFkcyD0rLWAkFKEABClBgFglMeUIXtn/913+NlJQUB7NI5suWLXO85woFKEABClCA AvcmMKVPuc+d6zydSOpiUavVEPfOucTaiwAAAw1JREFU3RfxC2YmexFf+XJ/Kn6yz8n6KUABClCA ApMh4Mywk1G7W51PPPEErly5gpUrV0p75KTuVkwq8+STT7pv5nsKUIACFKAABXwITGlCj4uLQ09P Dz744AOvv1xGxChG5iLxi4fnuFCAAhSgAAUoEJjAlCZ08TvaExIScPfu3QmjE0k9PDx8wjLcSQEK UIACFKCAU2BKE7r4Yyt+/+CKMzauUYACFKAABSgQoMBDeco9wNhYjAIUoAAFKECBAAWY0AOEYjEK UIACFKBAKAswoYdy7zA2ClCAAhSgQIACTOgBQrEYBShAAQpQIJQFmNBDuXcYGwUoQAEKUCBAASb0 AKFYjAIUoAAFKBDKAkzoodw7jI0CFKAABSgQoAATeoBQLEYBClCAAhQIZQEm9FDuHcZGAQpQgAIU CFCACT1AKBajAAUoQAEKhLIAE3oo9w5jowAFKEABCgQowIQeIBSLUYACFKAABUJZgAk9lHuHsVGA AhSgAAUCFGBCDxCKxShAAQpQgAKhLMCEHsq9w9goQAEKUIACAQowoQcIxWIUoAAFKECBUBZgQg/l 3mFsFKAABShAgQAFmNADhGIxClCAAhSgQCgLMKGHcu8wNgpQgAIUoECAAkzoAUKxGAUoQAEKUCCU BZjQQ7l3GBsFKEABClAgQAEm9AChWIwCFKAABSgQygJM6KHcO4yNAhSgAAUoEKAAE3qAUCxGAQpQ gAIUCGUBJvRQ7h3GRgEKUIACFAhQgAk9QCgWowAFKEABCoSyABN6KPcOY6MABShAAQoEKMCEHiAU i1GAAhSgAAVCWYAJPZR7h7FRgAIUoAAFAhRgQg8QisUoQAEKUIACoSzAhB7KvcPYKEABClCAAgEK MKEHCMViFKAABShAgVAWYEIP5d5hbBSgAAUoQIEABZjQA4RiMQpQgAIUoEAoCzChh3LvMDYKUIAC FKBAgAJM6AFCsRgFKEABClAglAWY0EO5dxgbBShAAQpQIEABJvQAoViMAhSgAAUoEMoCTOih3DuM jQIUoAAFKBCgwP8PWejLHQMXwwgAAAAASUVORK5CYII= ------=_Part_174803_217249912.1516621712629--

4 5

ovirt-websoclket-proxy: handler exception: Incorrect padding
by Peter Hudec 23 Jan '18

23 Jan '18

Hi all, I'm using VNC/noVNC for accessing the servers console in case of emergency. The console works fine, but I have got repeadly warning on the screen, that the wss://<host>:6100 could bot be connected with 3 recommendations - process is running /the console works/ - I'm able to connect the 6100 port by teknet form my pc - the certificate is imported In the /var/log/messages on HE I have found repeating message when the dialog appears. Jan 23 12:33:32 dipovirt-engine02 ovirt-websocket-proxy.py: ovirt-websocket-proxy[356] INFO msg:871 handler exception: Incorrect padding Nothing more is logged. The popus is quite annoing, since it appears every few seconds. my side: Mac OS X 10.13.2 FireFox 57.0.4 tested also on Ubuntu and Firefox 57.0.4 and works good,no exception. Any help or advise to figure what's wrong? regards Peter -- *Peter Hudec* Infraštruktúrny architekt phudec(a)cnc.sk <mailto:phudec@cnc.sk> *CNC, a.s.* Borská 6, 841 04 Bratislava Recepcia: +421 2 35 000 100 Mobil:+421 905 997 203 *www.cnc.sk* <http:///www.cnc.sk>

1 4

oVirt 4.1 change ovirtmgmt to enable VLAN tagging
by Staniforth, Paul 23 Jan '18

23 Jan '18

--_000_15166378166581144leedsbeckettacuk_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hello, We have an oVirt 4.1 cluster of 3 ovirt-nodes that currently on t= he management network don't have VLAN tagging enabled, we would like to ena= ble VLAN tagging and so would be grateful of anyone had an information or e= xperience doing this. The management engine is running on a separate host and the 3 hosts are run= ning oVirt Node 4.1.6 Thanks, Paul S. To view the terms under which this email is distributed, please go to:- http://disclaimer.leedsbeckett.ac.uk/disclaimer/disclaimer.html --_000_15166378166581144leedsbeckettacuk_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable <html> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Diso-8859-= 1"> <style type=3D"text/css" style=3D"display:none"></style> </head> <body dir=3D"ltr" style=3D"font-size:12pt;color:#000000;background-color:#F= FFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;"> <p>Hello,</p> <p>         We have an oVirt 4.1  cluster of = 3 ovirt-nodes that currently on the management network don't have VLAN tagg= ing enabled, we would like to enable VLAN tagging and so would be grat= eful of anyone had an information or experience doing this.</p> <p><br> </p> <p>The management engine is running on a separate host and the 3 hosts are = running oVirt Node 4.1.6</p> <p><br> </p> <p>Thanks,</p> <p>             Paul S. <br> </p> To view the terms under which this email is distributed, please go to:- <br= > <a href=3D"http://disclaimer.leedsbeckett.ac.uk/disclaimer/disclaimer.html"= target=3D"_blank">http://disclaimer.leedsbeckett.ac.uk/disclaimer/disclaim= er.html</a> <p></p> </body> </html> --_000_15166378166581144leedsbeckettacuk_--

2 4

q: lldpad.service needed for oVirt node ?
by Andrei V 23 Jan '18

23 Jan '18

Hi, Does lldpad.service (Link Layer Discovery Protocol) needed for oVirt node ? Thanks

3 5

Re: [ovirt-users] Windows Safely Remove Hardware
by Стаценко Константин Юрьевич 23 Jan '18

23 Jan '18

--_000_1900dcb19fde49ffb06cfc51ed8c6e14msk1exchmb07interraoru_ Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: quoted-printable Anyone ? I don't believe that this is a normal behavior when users can eject scsi co= ntroller or nic from virtual machine and make it unusable. The device capabilities is 6 by default in windows registry and to disable = device ejects we need to set it to the value of 2. But this value reverts back to 6 after VM restart. Maybe it is possible to make this value of 2 by default, so the device cann= ot be ejected ? From: =F3=D4=C1=C3=C5=CE=CB=CF =EB=CF=CE=D3=D4=C1=CE=D4=C9=CE =E0=D2=D8=C5= =D7=C9=DE Sent: Monday, January 22, 2018 11:30 AM To: users <users(a)ovirt.org> Subject: Windows Safely Remove Hardware Hello! What is the best way to remove options like eject virtual machine disks/con= trollers or nics from "Safely Remove Hardware" in Windows OS ? Currently running oVirt 4.1.8. Thanks. --_000_1900dcb19fde49ffb06cfc51ed8c6e14msk1exchmb07interraoru_ Content-Type: text/html; charset="koi8-r" Content-Transfer-Encoding: quoted-printable <html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr= osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" = xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:= //www.w3.org/TR/REC-html40"> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dkoi8-r"> <meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered medium)"> <style></style> </head> <body lang=3D"RU" link=3D"#0563C1" vlink=3D"#954F72"> <div class=3D"WordSection1"> <p class=3D"MsoNormal"><span lang=3D"EN-US" style=3D"color:#1F497D">Anyone = ?<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US" style=3D"color:#1F497D">I don&#= 8217;t believe that this is a normal behavior when users can eject scsi con= troller or nic from virtual machine and make it unusable.<o:p></o:p></span>= </p> <p class=3D"MsoNormal"><span lang=3D"EN-US" style=3D"color:#1F497D">The dev= ice capabilities is 6 by default in windows registry and to disable device = ejects we need to set it to the value of 2.<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US" style=3D"color:#1F497D">But thi= s value reverts back to 6 after VM restart.<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US" style=3D"color:#1F497D">Maybe i= t is possible to make this value of 2 by default, so the device cannot be e= jected ?<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US" style=3D"color:#1F497D"><o:p>&n= bsp;</o:p></span></p> <div> <div style=3D"border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm = 0cm 0cm"> <p class=3D"MsoNormal"><b><span lang=3D"EN-US" style=3D"mso-fareast-languag= e:RU">From:</span></b><span lang=3D"EN-US" style=3D"mso-fareast-language:RU= "> </span><span style=3D"mso-fareast-language:RU">=F3=D4=C1=C3=C5=CE=CB=CF</sp= an><span style=3D"mso-fareast-language:RU"> </span><span style=3D"mso-fareast-language:RU">=EB=CF=CE=D3=D4=C1=CE=D4=C9= =CE</span><span style=3D"mso-fareast-language:RU"> </span><span style=3D"mso-fareast-language:RU">=E0=D2=D8=C5=D7=C9=DE</span>= <span lang=3D"EN-US" style=3D"mso-fareast-language:RU"> <br> <b>Sent:</b> Monday, January 22, 2018 11:30 AM<br> <b>To:</b> users <users(a)ovirt.org><br> <b>Subject:</b> Windows Safely Remove Hardware<o:p></o:p></span></p> </div> </div> <p class=3D"MsoNormal"><span lang=3D"EN-US"><o:p> </o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Hello!<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">What is the best way to remove = options like eject virtual machine disks/controllers or nics from “Sa= fely Remove Hardware” in Windows OS ?<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Currently running oVirt 4.1.8.<= o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Thanks.<o:p></o:p></span></p> </div> </body> </html> --_000_1900dcb19fde49ffb06cfc51ed8c6e14msk1exchmb07interraoru_--

1 0

Snapshot Disk fails to connect to BackupApplianceVM - Failed to HotPlugDiskVDS
by Zip 23 Jan '18

23 Jan '18

> This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. --B_3599508108_124784952 Content-type: text/plain; charset="ISO-8859-1" Content-transfer-encoding: quoted-printable Okay, when I do a snapshot, and then mount the snapshot to a BackupApplianceVM, all works using NFS4 as my Storage Domain for the VM being backed up. If I change that disk storage on the VM to be backed up to my iSCSI storage, my script breaks and I see these errors in the oVirt Engine logs: --- 2018-01-22 23:10:36,249-06 INFO [org.ovirt.engine.core.bll.storage.disk.AttachDiskToVmCommand] (default task-58) [948cc433-45ed-46cc-b920-b4d9627bf99b] Lock Acquired to object 'EngineLock:{exclusiveLocks=3D'[dcabf886-22b2-4239-b57a-559d9b5b4c6d=3DDISK]', sharedLocks=3D''}' 2018-01-22 23:10:36,296-06 INFO [org.ovirt.engine.core.bll.storage.disk.AttachDiskToVmCommand] (default task-58) [948cc433-45ed-46cc-b920-b4d9627bf99b] Running command: AttachDiskToVmCommand internal: false. Entities affected : ID: 3c78dd5a-2982-4e35-ad9b-004394970dea Type: VMAction group CONFIGURE_VM_STORAGE with role type USER, ID: dcabf886-22b2-4239-b57a-559d9b5b4c6d Type: DiskAction group ATTACH_DISK wit= h role type USER 2018-01-22 23:10:36,300-06 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HotPlugDiskVDSCommand] (default task-58) [948cc433-45ed-46cc-b920-b4d9627bf99b] START, HotPlugDiskVDSCommand(HostName =3D ovirtnodea.mydomain.com, HotPlugDiskVDSParameters:{hostId=3D'34591e56-fd25-48dd-bf56-76b7b15e3721', vmId=3D'3c78dd5a-2982-4e35-ad9b-004394970dea', diskId=3D'dcabf886-22b2-4239-b57a-559d9b5b4c6d', addressMap=3D'null'}), log id: 1bb4b632 2018-01-22 23:10:37,903-06 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HotPlugDiskVDSCommand] (default task-58) [948cc433-45ed-46cc-b920-b4d9627bf99b] Failed in 'HotPlugDiskVDS' method 2018-01-22 23:10:37,907-06 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-58) [948cc433-45ed-46cc-b920-b4d9627bf99b] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM ovirtnodea.mydomain.com command HotPlugDiskVDS failed: unsupported configuration: native I/O needs either n= o disk cache or directsync cache mode, QEMU will fallback to aio=3Dthreads 2018-01-22 23:10:37,907-06 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HotPlugDiskVDSCommand] (default task-58) [948cc433-45ed-46cc-b920-b4d9627bf99b] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.HotPlugDiskVDSCommand' return value 'StatusOnlyReturn [status=3DStatus [code=3D45, message=3Dunsupported configuration: native I/O needs either no disk cache or directsync cache mode, QEMU will fallback to aio=3Dthreads]]' 2018-01-22 23:10:37,907-06 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HotPlugDiskVDSCommand] (default task-58) [948cc433-45ed-46cc-b920-b4d9627bf99b] HostName =3D ovirtnodea.mydomain.com 2018-01-22 23:10:37,907-06 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HotPlugDiskVDSCommand] (default task-58) [948cc433-45ed-46cc-b920-b4d9627bf99b] Command 'HotPlugDiskVDSCommand(HostName =3D ovirtnodea.mydomain.com, HotPlugDiskVDSParameters:{hostId=3D'34591e56-fd25-48dd-bf56-76b7b15e3721', vmId=3D'3c78dd5a-2982-4e35-ad9b-004394970dea', diskId=3D'dcabf886-22b2-4239-b57a-559d9b5b4c6d', addressMap=3D'null'})' execution failed: VDSGenericException: VDSErrorException: Failed to HotPlugDiskVDS, error =3D unsupported configuration: native I/O needs either no disk cache or directsync cache mode, QEMU will fallback to aio=3Dthreads, code =3D 45 2018-01-22 23:10:37,907-06 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HotPlugDiskVDSCommand] (default task-58) [948cc433-45ed-46cc-b920-b4d9627bf99b] FINISH, HotPlugDiskVDSCommand, log id: 1bb4b632 2018-01-22 23:10:37,907-06 ERROR [org.ovirt.engine.core.bll.storage.disk.AttachDiskToVmCommand] (default task-58) [948cc433-45ed-46cc-b920-b4d9627bf99b] Command 'org.ovirt.engine.core.bll.storage.disk.AttachDiskToVmCommand' failed: EngineException:=20 org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to HotPlugDiskVDS, error =3D unsupported configuration: native I/O needs either no disk cache or directsync cache mode, QEMU will fallback to aio=3Dthreads, code =3D 45 (Failed with error FailedToPlugDisk and code 45) 2018-01-22 23:10:37,910-06 ERROR [org.ovirt.engine.core.bll.storage.disk.AttachDiskToVmCommand] (default task-58) [948cc433-45ed-46cc-b920-b4d9627bf99b] Transaction rolled-back for command 'org.ovirt.engine.core.bll.storage.disk.AttachDiskToVmCommand'. 2018-01-22 23:10:37,954-06 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-58) [948cc433-45ed-46cc-b920-b4d9627bf99b] EVENT_ID: USER_FAILED_ATTACH_DISK_TO_VM(2,017), Failed to attach Disk TEST1234_Disk1 to VM SimpleBackup (User: admin@internal-authz). 2018-01-22 23:10:37,955-06 INFO [org.ovirt.engine.core.bll.storage.disk.AttachDiskToVmCommand] (default task-58) [948cc433-45ed-46cc-b920-b4d9627bf99b] Lock freed to object 'EngineLock:{exclusiveLocks=3D'[dcabf886-22b2-4239-b57a-559d9b5b4c6d=3DDISK]', sharedLocks=3D''}' 2018-01-22 23:10:37,955-06 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-58) [] Operation Failed: [Failed to hot-plug disk] --- I see the "Failed to HotPlugDiskVDS, error =3D unsupported configuration: native I/O needs either no disk cache or directsync cache mode, QEMU will fallback to aio=3Dthreads, code=B2 But I have no clue how to fix that using oVirt 4.2 on CentOS 7. Fresh install and all up to date. Thanks for any help. BTW, current script I am writing is here: https://github.com/zipurman/oVIRT_Simple_Backup Irc.oftc.net #ovirt zipur --B_3599508108_124784952 Content-type: text/html; charset="ISO-8859-1" Content-transfer-encoding: quoted-printable <html><head></head><body style=3D"word-wrap: break-word; -webkit-nbsp-mode: s= pace; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size:= 14px; font-family: Calibri, sans-serif;"><div><div>Okay, when I do a snapsh= ot, and then mount the snapshot to a Backup<span style=3D"font-style: italic;"= >Appliance</span>VM, all works using NFS4 as my Storage Domain for the VM be= ing backed up. If I change that disk storage on the VM to be backed up to my= iSCSI storage, my script breaks and I see these errors in the oVirt Engine = logs:</div><div><br></div><div>---</div><div><div>2018-01-22 23:10:36,249-06= INFO  [org.ovirt.engine.core.bll.storage.disk.AttachDiskToVmCommand] (= default task-58) [948cc433-45ed-46cc-b920-b4d9627bf99b] Lock Acquired to obj= ect 'EngineLock:{exclusiveLocks=3D'[dcabf886-22b2-4239-b57a-559d9b5b4c6d=3DDISK]= ', sharedLocks=3D''}'</div><div><br></div><div>2018-01-22 23:10:36,296-06 INFO=  [org.ovirt.engine.core.bll.storage.disk.AttachDiskToVmCommand] (defau= lt task-58) [948cc433-45ed-46cc-b920-b4d9627bf99b] Running command: AttachDi= skToVmCommand internal: false. Entities affected :  ID: 3c78dd5a-2982-4= e35-ad9b-004394970dea Type: VMAction group CONFIGURE_VM_STORAGE with role ty= pe USER,  ID: dcabf886-22b2-4239-b57a-559d9b5b4c6d Type: DiskAction gro= up ATTACH_DISK with role type USER</div><div><br></div><div>2018-01-22 23:10= :36,300-06 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.HotPlugDisk= VDSCommand] (default task-58) [948cc433-45ed-46cc-b920-b4d9627bf99b] START, = HotPlugDiskVDSCommand(HostName =3D ovirtnodea.mydomain.com, HotPlugDiskVDSPara= meters:{hostId=3D'34591e56-fd25-48dd-bf56-76b7b15e3721', vmId=3D'3c78dd5a-2982-4= e35-ad9b-004394970dea', diskId=3D'dcabf886-22b2-4239-b57a-559d9b5b4c6d', addre= ssMap=3D'null'}), log id: 1bb4b632</div><div><br></div><div>2018-01-22 23:10:3= 7,903-06 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HotPlugDiskVDSComm= and] (default task-58) [948cc433-45ed-46cc-b920-b4d9627bf99b] Failed in 'Hot= PlugDiskVDS' method</div><div><br></div><div>2018-01-22 23:10:37,907-06 ERRO= R [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (de= fault task-58) [948cc433-45ed-46cc-b920-b4d9627bf99b] EVENT_ID: VDS_BROKER_C= OMMAND_FAILURE(10,802), VDSM ovirtnodea.mydomain.com command HotPl= ugDiskVDS failed: unsupported configuration: native I/O needs either no disk= cache or directsync cache mode, QEMU will fallback to aio=3Dthreads</div><div= ><br></div><div>2018-01-22 23:10:37,907-06 INFO  [org.ovirt.engine.core= .vdsbroker.vdsbroker.HotPlugDiskVDSCommand] (default task-58) [948cc433-45ed= -46cc-b920-b4d9627bf99b] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.= HotPlugDiskVDSCommand' return value 'StatusOnlyReturn [status=3DStatus [code=3D4= 5, message=3Dunsupported configuration: native I/O needs either no disk cache = or directsync cache mode, QEMU will fallback to aio=3Dthreads]]'</div><div><br= ></div><div>2018-01-22 23:10:37,907-06 INFO  [org.ovirt.engine.core.vds= broker.vdsbroker.HotPlugDiskVDSCommand] (default task-58) [948cc433-45ed-46c= c-b920-b4d9627bf99b] HostName =3D ovirtnodea.mydomain.com</div><div><br></div>= <div>2018-01-22 23:10:37,907-06 ERROR [org.ovirt.engine.core.vdsbroker.vdsbr= oker.HotPlugDiskVDSCommand] (default task-58) [948cc433-45ed-46cc-b920-b4d96= 27bf99b] Command 'HotPlugDiskVDSCommand(HostName =3D ovirtnodea.mydomain.= com, HotPlugDiskVDSParameters:{hostId=3D'34591e56-fd25-48dd-bf56-76b7b15e3721'= , vmId=3D'3c78dd5a-2982-4e35-ad9b-004394970dea', diskId=3D'dcabf886-22b2-4239-b5= 7a-559d9b5b4c6d', addressMap=3D'null'})' execution failed: VDSGenericException= : VDSErrorException: Failed to HotPlugDiskVDS, error =3D unsupported configura= tion: native I/O needs either no disk cache or directsync cache mode, QEMU w= ill fallback to aio=3Dthreads, code =3D 45</div><div><br></div><div>2018-01-22 2= 3:10:37,907-06 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.HotPlug= DiskVDSCommand] (default task-58) [948cc433-45ed-46cc-b920-b4d9627bf99b] FIN= ISH, HotPlugDiskVDSCommand, log id: 1bb4b632</div><div><br></div><div>2018-0= 1-22 23:10:37,907-06 ERROR [org.ovirt.engine.core.bll.storage.disk.AttachDis= kToVmCommand] (default task-58) [948cc433-45ed-46cc-b920-b4d9627bf99b] Comma= nd 'org.ovirt.engine.core.bll.storage.disk.AttachDiskToVmCommand' failed: En= gineException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: = VDSGenericException: VDSErrorException: Failed to HotPlugDiskVDS, error =3D un= supported configuration: native I/O needs either no disk cache or directsync= cache mode, QEMU will fallback to aio=3Dthreads, code =3D 45 (Failed with error= FailedToPlugDisk and code 45)</div><div><br></div><div>2018-01-22 23:10:37,= 910-06 ERROR [org.ovirt.engine.core.bll.storage.disk.AttachDiskToVmCommand] = (default task-58) [948cc433-45ed-46cc-b920-b4d9627bf99b] Transaction rolled-= back for command 'org.ovirt.engine.core.bll.storage.disk.AttachDiskToVmComma= nd'.</div><div><br></div><div>2018-01-22 23:10:37,954-06 ERROR [org.ovirt.en= gine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-58) = [948cc433-45ed-46cc-b920-b4d9627bf99b] EVENT_ID: USER_FAILED_ATTACH_DISK_TO_= VM(2,017), Failed to attach Disk TEST1234_Disk1 to VM SimpleBackup (User: ad= min@internal-authz).</div><div><br></div><div>2018-01-22 23:10:37,955-06 INF= O  [org.ovirt.engine.core.bll.storage.disk.AttachDiskToVmCommand] (defa= ult task-58) [948cc433-45ed-46cc-b920-b4d9627bf99b] Lock freed to object 'En= gineLock:{exclusiveLocks=3D'[dcabf886-22b2-4239-b57a-559d9b5b4c6d=3DDISK]', shar= edLocks=3D''}'</div><div><br></div><div>2018-01-22 23:10:37,955-06 ERROR [org.= ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-58)= [] Operation Failed: [Failed to hot-plug disk]</div></div><div>---</div><di= v><br></div><div><br></div><div>I see the "Failed to HotPlugDiskVDS, error =3D= unsupported configuration: native I/O needs either no disk cache or directs= ync cache mode, QEMU will fallback to aio=3Dthreads, code”</div><div><br= ></div><div>But I have no clue how to fix that using oVirt 4.2 on CentOS 7. = Fresh install and all up to date.</div><div><br></div><div>Thanks for any he= lp.</div><div><br></div><div>BTW, current script I am writing is here: = <a href=3D"https://github.com/zipurman/oVIRT_Simple_Backup">https://github.com= /zipurman/oVIRT_Simple_Backup</a> </div><div><br></div><div>Irc.oftc.ne= t #ovirt</div><div>zipur</div><br></div><div><span class=3D"Apple-style-span" = style=3D"font-size: 12px; font-family: Verdana, Helvetica, Arial;"><font color= =3D"#808080"><br></font></span></div></body></html> --B_3599508108_124784952--

1 0

Uploading big raw disks into storage
by Gabriel Stein 22 Jan '18

22 Jan '18

Hello! I'm still migrating the VMs from Proxmox to oVirt, migrating all disks, one by one. Google Chrome doesn't accept anymore the certificates from ovirt-engine, firefox it's ok. Error(I receive that direct from start from upload on google chrome, on firefox works): *Unable to upload image to disk a-b-c due to a network error. Make sure ovirt-imageio-proxy service is installed and configured, and ovirt-engine's certificate is registered as a valid CA in the browser. The certificate can be fetched from https://<engine_url>/ovirt-engine/services/pki-resource?resource=ca-certificate&format=X509-PEM-CA* BUT! When I'm uploading big disks(using firefox), such > 100GB, ovirt-engine interrupts the process, giving the same error above. I think that is a kind of timeout and the error doesn't show the real problem. On engine.log is the same error. on image-proxy.log: (Thread-5834) WARNING 2018-01-22 10:46:07,708 web:89:web:(log_response) 1.1.1.1 - PUT /d69d1a59-ed5c-4097-a9af-5e2d36299181 401 360 (0.00s) (Thread-5835) INFO 2018-01-22 10:46:10,716 web:89:web:(log_response) 1.1.1.1 - OPTIONS /d69d1a59-ed5c-4097-a9af-5e2d36299181 204 0 (0.00s) (Thread-5836) ERROR 2018-01-22 10:46:10,722 session:314:root:(_decode_ovirt_ticket) Failed to verify proxy ticket: Ticket life time expired (Thread-5836) ERROR 2018-01-22 10:46:10,722 session:137:root:(start_session) Error starting session: Unable to verify proxy ticket Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/session.py", line 135, in start_session session_id = _create_update_session(ticket, session_id) File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/session.py", line 194, in _create_update_session ticket_vars = _decode_proxy_ticket(authorization) File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/session.py", line 237, in _decode_proxy_ticket payload = _decode_ovirt_ticket(ticket) File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/session.py", line 315, in _decode_ovirt_ticket raise ValueError("Unable to verify proxy ticket") ValueError: Unable to verify proxy ticket (Thread-5836) WARNING 2018-01-22 10:46:10,722 web:89:web:(log_response) 1.1.1.1 - PUT /d69d1a59-ed5c-4097-a9af-5e2d36299181 401 360 (0.00s) (Thread-5837) INFO 2018-01-22 10:46:13,731 web:89:web:(log_response) 1.1.1.1 - OPTIONS /d69d1a59-ed5c-4097-a9af-5e2d36299181 204 0 (0.00s) (Thread-5838) ERROR 2018-01-22 10:46:13,737 session:314:root:(_decode_ovirt_ticket) Failed to verify proxy ticket: Ticket life time expired (Thread-5838) ERROR 2018-01-22 10:46:13,737 session:137:root:(start_session) Error starting session: Unable to verify proxy ticket Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/session.py", line 135, in start_session session_id = _create_update_session(ticket, session_id) File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/session.py", line 194, in _create_update_session ticket_vars = _decode_proxy_ticket(authorization) File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/session.py", line 237, in _decode_proxy_ticket payload = _decode_ovirt_ticket(ticket) File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/session.py", line 315, in _decode_ovirt_ticket raise ValueError("Unable to verify proxy ticket") ValueError: Unable to verify proxy ticket (Thread-5838) WARNING 2018-01-22 10:46:13,738 web:89:web:(log_response) 1.1.1.1 - PUT /d69d1a59-ed5c-4097-a9af-5e2d36299181 401 360 (0.00s) (Thread-5839) INFO 2018-01-22 10:46:16,745 web:89:web:(log_response) 1.1.1.1 - OPTIONS /d69d1a59-ed5c-4097-a9af-5e2d36299181 204 0 (0.00s) (Thread-5840) ERROR 2018-01-22 10:46:16,750 session:314:root:(_decode_ovirt_ticket) Failed to verify proxy ticket: Ticket life time expired (Thread-5840) ERROR 2018-01-22 10:46:16,750 session:137:root:(start_session) Error starting session: Unable to verify proxy ticket There is a bugzilla bug, but it's confusing, I'm already using the version 4.2 and the error persists. Bug: https://bugzilla.redhat.com/show_bug.cgi?id=1514887(no, I'm not restaring the daemon, the it seems something similar....) A thread on version 4.05: http://lists.ovirt.org/pipermail/users/2016-November/077706.html I need some help... Thanks in Advance! Best Regards, Gabriel Gabriel Stein ------------------------------ Gabriel Ferraz Stein Tel.: +49 (0) 170 2881531

3 4

Hosted-engine without mac-spoofing
by Matteo Capuano 22 Jan '18

22 Jan '18

Hi folks, i'm trying to install oVirt hyperconverged on Google Cloud Engine (GCE) to create a lab environment. Nested virtualization on GCE is in public beta, it seems to work fine but there's a catch: i cannot enable mac-spoofing on the GCE instances' nics. So here's my question: is there a way to install hosted-engine on a network there you can't enabling mac-spoofing? Thank you Cheers Matteo

3 3

Re: [ovirt-users] oVirt 4.1 change ovirtmgmt to enable VLAN tagging
by Staniforth, Paul 22 Jan '18

22 Jan '18

--_000_14955b19e8134aa58fc7371f58868590emailandroidcom_ Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 VGhhbmtzIEx1Y2EsDQogICAgICAgICAgICAgICAgICAgICAgICAgSSdsbCB0YWxrIHRvIG91ciBu ZXR3b3JrIHRlYW0uDQpQYXVsIFMuDQoNCk9uIDIyIEphbiAyMDE4IDE5OjEwLCBMdWNhICdyZW1p eF90aicgTG9yZW56ZXR0byA8bG9yZW56ZXR0by5sdWNhQGdtYWlsLmNvbT4gd3JvdGU6DQoNCg0K SWwgMjIgZ2VuIDIwMTggNzo0OSBQTSwgIlN0YW5pZm9ydGgsIFBhdWwiIDxQLlN0YW5pZm9ydGhA bGVlZHNiZWNrZXR0LmFjLnVrPG1haWx0bzpQLlN0YW5pZm9ydGhAbGVlZHNiZWNrZXR0LmFjLnVr Pj4gaGEgc2NyaXR0bzoNCg0KSGVsbG8sDQoNCiAgICAgICAgIFdlIGhhdmUgYW4gb1ZpcnQgNC4x ICBjbHVzdGVyIG9mIDMgb3ZpcnQtbm9kZXMgdGhhdCBjdXJyZW50bHkgb24gdGhlIG1hbmFnZW1l bnQgbmV0d29yayBkb24ndCBoYXZlIFZMQU4gdGFnZ2luZyBlbmFibGVkLCB3ZSB3b3VsZCBsaWtl IHRvIGVuYWJsZSBWTEFOIHRhZ2dpbmcgYW5kIHNvIHdvdWxkIGJlIGdyYXRlZnVsIG9mIGFueW9u ZSBoYWQgYW4gaW5mb3JtYXRpb24gb3IgZXhwZXJpZW5jZSBkb2luZyB0aGlzLg0KDQoNClRoZSBt YW5hZ2VtZW50IGVuZ2luZSBpcyBydW5uaW5nIG9uIGEgc2VwYXJhdGUgaG9zdCBhbmQgdGhlIDMg aG9zdHMgYXJlIHJ1bm5pbmcgb1ZpcnQgTm9kZSA0LjEuNg0KDQpIaSBQYXVsLCBpIGRpZCB0aGUg c2FtZSB3aXRoIHRoZSBzYW1lIHJlbGVhc2UuDQoNClRoZSB0cmljayB3YXMgdG8gc2V0IHRoZSB0 YWdnaW5nIGFsc28gb2YgdGhlIGRlZmF1bHQgdmxhbiBvbiB0aGUgc3dpdGNoIGR1cmluZyB0aGUg Y2hhbmdlIGZyb20gdW50YWdnZWQgdG8gdGFnZ2VkLg0KDQpJbiBleGFtcGxlLCBpZiB5b3UgaGF2 ZSB0byBtb3ZlIHlvdXIgb3ZpcnRtZ210IHRvIHZsYW4gMTAsIHlvdSBjYW4gc2V0IHlvdXIgcG9y dCBmb3Igc2VydmluZyB0aGUgbmV0d29yayBib3RoIG5hdGl2ZSBhbmQgdGFnZ2VkIHNvIHdoaWxl IGNoYW5naW5nIHRoZSBvdmlydG1nbXQgc2V0dXAgeW91IHdpbGwgbm90IGxvc2UgdGhlIGNvbm5l Y3Rpdml0eS4NCg0KTHVjYQ0KDQpUbyB2aWV3IHRoZSB0ZXJtcyB1bmRlciB3aGljaCB0aGlzIGVt YWlsIGlzIGRpc3RyaWJ1dGVkLCBwbGVhc2UgZ28gdG86LQ0KaHR0cDovL2Rpc2NsYWltZXIubGVl ZHNiZWNrZXR0LmFjLnVrL2Rpc2NsYWltZXIvZGlzY2xhaW1lci5odG1sDQo= --_000_14955b19e8134aa58fc7371f58868590emailandroidcom_ Content-Type: text/html; charset="utf-8" Content-ID: <E03D7FE978BC7F459C7E38BCCE487EF5(a)leedsbeckett.ac.uk> Content-Transfer-Encoding: base64 PGh0bWw+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIgY29udGVudD0i dGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjwvaGVhZD4NCjxib2R5Pg0KPGRpdiBkaXI9ImF1 dG8iPg0KPGRpdj5UaGFua3MgTHVjYSwNCjxkaXYgZGlyPSJhdXRvIj4mbmJzcDsgJm5ic3A7ICZu YnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5i c3A7ICZuYnNwOyAmbmJzcDtJJ2xsIHRhbGsgdG8gb3VyIG5ldHdvcmsgdGVhbS48L2Rpdj4NClBh dWwgUy48YnI+DQo8ZGl2IGNsYXNzPSJnbWFpbF9leHRyYSI+PGJyPg0KPGRpdiBjbGFzcz0iZ21h aWxfcXVvdGUiPk9uIDIyIEphbiAyMDE4IDE5OjEwLCBMdWNhICdyZW1peF90aicgTG9yZW56ZXR0 byAmbHQ7bG9yZW56ZXR0by5sdWNhQGdtYWlsLmNvbSZndDsgd3JvdGU6PGJyIHR5cGU9ImF0dHJp YnV0aW9uIj4NCjxibG9ja3F1b3RlIGNsYXNzPSJxdW90ZSIgc3R5bGU9Im1hcmdpbjowIDAgMCAu OGV4O2JvcmRlci1sZWZ0OjFweCAjY2NjIHNvbGlkO3BhZGRpbmctbGVmdDoxZXgiPg0KPGRpdj4N CjxkaXYgZGlyPSJhdXRvIj48YnI+DQo8ZGl2IGRpcj0iYXV0byI+PGJyPg0KPGRpdiBjbGFzcz0i ZWxpZGVkLXRleHQiPklsIDIyIGdlbiAyMDE4IDc6NDkgUE0sICZxdW90O1N0YW5pZm9ydGgsIFBh dWwmcXVvdDsgJmx0OzxhIGhyZWY9Im1haWx0bzpQLlN0YW5pZm9ydGhAbGVlZHNiZWNrZXR0LmFj LnVrIj5QLlN0YW5pZm9ydGhAbGVlZHNiZWNrZXR0LmFjLjx3YnI+dWs8L2E+Jmd0OyBoYSBzY3Jp dHRvOjxicj4NCjxibG9ja3F1b3RlIHN0eWxlPSJtYXJnaW46MCAwIDAgMC44ZXg7Ym9yZGVyLWxl ZnQ6MXB4ICNjY2Mgc29saWQ7cGFkZGluZy1sZWZ0OjFleCI+DQo8ZGl2IGRpcj0ibHRyIiBzdHls ZT0iZm9udC1zaXplOjEycHQ7Y29sb3I6IzAwMDAwMDtiYWNrZ3JvdW5kLWNvbG9yOiNmZmZmZmY7 Zm9udC1mYW1pbHk6J2NhbGlicmknICwgJ2FyaWFsJyAsICdoZWx2ZXRpY2EnICwgc2Fucy1zZXJp ZiI+DQo8cD5IZWxsbyw8L3A+DQo8cD4mbmJzcDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7 V2UgaGF2ZSBhbiBvVmlydCA0LjEgJm5ic3A7Y2x1c3RlciBvZiAzIG92aXJ0LW5vZGVzIHRoYXQg Y3VycmVudGx5IG9uIHRoZSBtYW5hZ2VtZW50IG5ldHdvcmsgZG9uJ3QgaGF2ZSBWTEFOIHRhZ2dp bmcgZW5hYmxlZCwgd2Ugd291bGQgbGlrZSB0byBlbmFibGUmbmJzcDtWTEFOIHRhZ2dpbmcgYW5k IHNvIHdvdWxkIGJlIGdyYXRlZnVsIG9mIGFueW9uZSBoYWQgYW4gaW5mb3JtYXRpb24gb3IgZXhw ZXJpZW5jZSBkb2luZyB0aGlzLjwvcD4NCjxwPjxicj4NCjwvcD4NCjxwPlRoZSBtYW5hZ2VtZW50 IGVuZ2luZSBpcyBydW5uaW5nIG9uIGEgc2VwYXJhdGUgaG9zdCBhbmQgdGhlIDMgaG9zdHMgYXJl IHJ1bm5pbmcgb1ZpcnQgTm9kZSA0LjEuNjwvcD4NCjwvZGl2Pg0KPC9ibG9ja3F1b3RlPg0KPC9k aXY+DQo8L2Rpdj4NCjxkaXYgZGlyPSJhdXRvIj48YnI+DQo8L2Rpdj4NCjxkaXYgZGlyPSJhdXRv Ij5IaSBQYXVsLCBpIGRpZCB0aGUgc2FtZSB3aXRoIHRoZSBzYW1lIHJlbGVhc2UuPC9kaXY+DQo8 ZGl2IGRpcj0iYXV0byI+PGJyPg0KPC9kaXY+DQo8ZGl2IGRpcj0iYXV0byI+VGhlIHRyaWNrIHdh cyB0byBzZXQgdGhlIHRhZ2dpbmcgYWxzbyBvZiB0aGUgZGVmYXVsdCB2bGFuIG9uIHRoZSBzd2l0 Y2ggZHVyaW5nIHRoZSBjaGFuZ2UgZnJvbSB1bnRhZ2dlZCB0byB0YWdnZWQuPC9kaXY+DQo8ZGl2 IGRpcj0iYXV0byI+PGJyPg0KPC9kaXY+DQo8ZGl2IGRpcj0iYXV0byI+SW4gZXhhbXBsZSwgaWYg eW91IGhhdmUgdG8gbW92ZSB5b3VyIG92aXJ0bWdtdCB0byB2bGFuIDEwLCB5b3UgY2FuIHNldCB5 b3VyIHBvcnQgZm9yIHNlcnZpbmcgdGhlIG5ldHdvcmsgYm90aCBuYXRpdmUgYW5kIHRhZ2dlZCBz byB3aGlsZSBjaGFuZ2luZyB0aGUgb3ZpcnRtZ210IHNldHVwIHlvdSB3aWxsIG5vdCBsb3NlIHRo ZSBjb25uZWN0aXZpdHkuPC9kaXY+DQo8ZGl2IGRpcj0iYXV0byI+PGJyPg0KPC9kaXY+DQo8ZGl2 IGRpcj0iYXV0byI+THVjYTwvZGl2Pg0KPGRpdiBkaXI9ImF1dG8iPjwvZGl2Pg0KPC9kaXY+DQo8 L2Rpdj4NCjwvYmxvY2txdW90ZT4NCjwvZGl2Pg0KPGJyPg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2 Pg0KVG8gdmlldyB0aGUgdGVybXMgdW5kZXIgd2hpY2ggdGhpcyBlbWFpbCBpcyBkaXN0cmlidXRl ZCwgcGxlYXNlIGdvIHRvOi0gPGJyPg0KPGEgaHJlZj0iaHR0cDovL2Rpc2NsYWltZXIubGVlZHNi ZWNrZXR0LmFjLnVrL2Rpc2NsYWltZXIvZGlzY2xhaW1lci5odG1sIiB0YXJnZXQ9Il9ibGFuayI+ aHR0cDovL2Rpc2NsYWltZXIubGVlZHNiZWNrZXR0LmFjLnVrL2Rpc2NsYWltZXIvZGlzY2xhaW1l ci5odG1sPC9hPg0KPHA+PC9wPg0KPC9ib2R5Pg0KPC9odG1sPg0K --_000_14955b19e8134aa58fc7371f58868590emailandroidcom_--

1 0

Re: [ovirt-users] oVirt 4.2 - Removed disk remains in VM OS with hooks?
by Yaniv Kaul 22 Jan '18

22 Jan '18

On Jan 22, 2018 5:32 PM, "Zip" <plord(a)intricatenetworks.com> wrote: I am having an issue where when I use the REST API to connect a snapshot > from another VM to a Backup-Appliance-VM, after the clone when I remove the > Disk and delete the Snapshot, the disk remains in the Backup-Appliance-VM > as /dev/sdb ot /dev/vdb. > > If I reboot the Bakup-Applicance-VM the disk disappears. > > If I manually remove the disk by "echo 1 > /sys/block/sdb/device/delete” > the disk will disappear, but if I rescan the scsi bus, it is found and > shows up again in the VM OS, but the oVirt WebGUI does NOT show it as > connected. > -- The first part is expected - the 2nd isn't. What are you referring to as the first and second part? I understand why the disk is seen after it was detached (stale device), not why it comes back after rescan, which seems to suggest it is not detached properly. > > I am also not able to attach any other disks as it complains of : > > HotPlugDiskVDS failed: internal error: unable to execute QEMU command > '__com.redhat_drive_add': Duplicate ID 'drive-scsi0-0-0-2' for drive > > I did see that others in the past have gotten around this issue by > rebooting the Backup-Appliance-VM and then continuing on with the next VM > backup and looping through backup-reboot-backup-reboot-etc. > > Anyone have an idea on how to solve this issue and remove the hooks from > the guest OS? > > Steps to reproduce this issue: > > > 1. Create a backup appliance VM to be used for the backup script > execution > 2. Currently I have the Vms set to virtio with threaded I/O enabled. > Also tried virtio_scsi with same result. > 3. Using REST API – make snapshot of target VM > 4. Using REST API – fetch vm metadata > 5. Using REST API – attach the snapshot/disk to the Backup-Appliance-VM > 6. dd the drive to backup folder > 7. Using REST API – remove the disk from the Backup-Appliance-VM > 8. Using REST API – delete the snapshot > 9. ** Check the guest OS of the Backup-Appliance-VM and the mounted > drive from the backup above still appears and behaves as mentioned in > comments above. > > > — There are many details missing, including versions of everything used, but logs would be most helpful here. Versions for oVirt are all the most recent. This is a fresh install of the Hosted Engine. I will just script the backup to cycle through sdb, sdc, sdd, …. Szzzz, just seems odd that once a disk is detached and a snapshot deleted, that the Backup_appliance-VM can still access the drive/snapshot? Unrelated note - do NOT use /dev/sdX to enumerate them. Especially on SCSI bus, probing is done in parallel and they may have a different name next time. Use /dev/disk/by-id paths. A second issue is that the above wont work when I have the Vms running on > MPIO iSCSI storage, so for testing I have moved to NFS4. Anyone have ideas > about either issue, I’d love to hear ;) > — Same - logs would be helpful here. I will continue to dig through this issue and will post logs if stuck. I just wanted to know if there was anything obvious that I should be doing differently with iSCSI vs NFS with mounting disks/snapshots. Nope. Y. Zip — Y. > > Thanks > > Irc.oftc.net #ovirt > zipur > > > > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > >

2 1

oVirt home lab hardware
by Abdurrahman A. Ibrahim 22 Jan '18

22 Jan '18

Hello, I am planning to buy home lab hardware to be used by oVirt. Any recommendations for used hardware i can buy from eBay for example? Also, have you tried oVirt on Intel NUC or any other SMB servers before? Thanks, Ab

6 6

New post on oVirt blog: Build oVirt Reports Using Grafana
by Shirly Radco 22 Jan '18

22 Jan '18

Hello everyone, A new oVirt blog post has been published, on how to build oVirt reports using Grafana. This allows connecting to oVirt DWH and create dashboards for System, Hosts, VMs, Storage domains etc. See https://ovirt.org/blog/2018/01/ovirt-report-using-grafana/ for the full post. For more information you can contact me. Best regards, -- SHIRLY RADCO BI SeNIOR SOFTWARE ENGINEER Red Hat Israel <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

2 1

oVirt 4.2 - Removed disk remains in VM OS with hooks?
by Zip 22 Jan '18

22 Jan '18

> This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. --B_3599421680_119924947 Content-type: text/plain; charset="ISO-8859-1" Content-transfer-encoding: quoted-printable I am having an issue where when I use the REST API to connect a snapshot from another VM to a Backup-Appliance-VM, after the clone when I remove the Disk and delete the Snapshot, the disk remains in the Backup-Appliance-VM a= s /dev/sdb ot /dev/vdb. If I reboot the Bakup-Applicance-VM the disk disappears. If I manually remove the disk by "echo 1 > /sys/block/sdb/device/delete=B2 th= e disk will disappear, but if I rescan the scsi bus, it is found and shows up again in the VM OS, but the oVirt WebGUI does NOT show it as connected. I am also not able to attach any other disks as it complains of : HotPlugDiskVDS failed: internal error: unable to execute QEMU command '__com.redhat_drive_add': Duplicate ID 'drive-scsi0-0-0-2' for drive I did see that others in the past have gotten around this issue by rebootin= g the Backup-Appliance-VM and then continuing on with the next VM backup and looping through backup-reboot-backup-reboot-etc. Anyone have an idea on how to solve this issue and remove the hooks from th= e guest OS? Steps to reproduce this issue: 1. Create a backup appliance VM to be used for the backup script execution 2. Currently I have the Vms set to virtio with threaded I/O enabled. Also tried virtio_scsi with same result. 3. Using REST API =AD make snapshot of target VM 4. Using REST API =AD fetch vm metadata 5. Using REST API =AD attach the snapshot/disk to the Backup-Appliance-VM 6. dd the drive to backup folder 7. Using REST API =AD remove the disk from the Backup-Appliance-VM 8. Using REST API =AD delete the snapshot 9. ** Check the guest OS of the Backup-Appliance-VM and the mounted drive from the backup above still appears and behaves as mentioned in comments above. A second issue is that the above wont work when I have the Vms running on MPIO iSCSI storage, so for testing I have moved to NFS4. Anyone have ideas about either issue, I=B9d love to hear ;) Thanks Irc.oftc.net #ovirt zipur --B_3599421680_119924947 Content-type: text/html; charset="ISO-8859-1" Content-transfer-encoding: quoted-printable <html><head></head><body style=3D"word-wrap: break-word; -webkit-nbsp-mode: s= pace; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size:= 14px; font-family: Calibri, sans-serif;"><div>I am having an issue where wh= en I use the REST API to connect a snapshot from another VM to a Backup-Appl= iance-VM, after the clone when I remove the Disk and delete the Snapshot, th= e disk remains in the Backup-Appliance-VM as /dev/sdb ot /dev/vdb. </di= v><div><br></div><div>If I reboot the Bakup-Applicance-VM the disk disappear= s.</div><div><br></div><div>If I manually remove the disk by "echo 1 > /s= ys/block/sdb/device/delete” the disk will disappear, but if I rescan t= he scsi bus, it is found and shows up again in the VM OS, but the oVirt WebG= UI does NOT show it as connected.</div><div><br></div><div>I am also not abl= e to attach any other disks as it complains of :</div><div><br></div><div><s= pan class=3D"Apple-tab-span" style=3D"white-space:pre"> </span>HotPlugDiskVDS fa= iled: internal error: unable to execute QEMU command '__com.redhat_drive_add= ': Duplicate ID 'drive-scsi0-0-0-2' for drive</div><div><br></div><div>I did= see that others in the past have gotten around this issue by rebooting the = Backup-Appliance-VM and then continuing on with the next VM backup and loopi= ng through backup-reboot-backup-reboot-etc. </div><div><br></div><div>A= nyone have an idea on how to solve this issue and remove the hooks from the = guest OS?</div><div><br></div><div>Steps to reproduce this issue:</div><div>= <br></div><ol><li>Create a backup appliance VM to be used for the backup scr= ipt execution</li><li>Currently I have the Vms set to virtio with threaded I= /O enabled. Also tried virtio_scsi with same result.</li><li>Using REST API = – make snapshot of target VM</li><li>Using REST API – fetch vm m= etadata</li><li>Using REST API – attach the snapshot/disk to the Backu= p-Appliance-VM</li><li>dd the drive to backup folder</li><li>Using REST API = – remove the disk from the Backup-Appliance-VM</li><li>Using REST API = – delete the snapshot</li><li>** Check the guest OS of the Backup-Appl= iance-VM and the mounted drive from the backup above still appears and behav= es as mentioned in comments above.</li></ol><div><br></div><div>A second iss= ue is that the above wont work when I have the Vms running on MPIO iSCSI sto= rage, so for testing I have moved to NFS4. Anyone have ideas about either is= sue, I’d love to hear ;)</div><div><br></div><div>Thanks</div><div><br= ></div><div>Irc.oftc.net #ovirt</div><div>zipur</div><div><br></div><br></bo= dy></html> --B_3599421680_119924947--

2 2

Last week for participating to oVirt 2018 Survey!
by Sandro Bonazzola 22 Jan '18

22 Jan '18

As we continue to develop oVirt 4.2 and future releases, the Development and Integration teams at Red Hat would value insights on how you are deploying the oVirt environment. If you haven't already done, please help us to hit the mark by completing this short survey. Survey will close on February 1st. Here's the link to the survey: https://goo.gl/forms/cAKWAR8RD7rGrVhE2 Thanks, -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

1 0

VirtIO-SCSI and viodiskcache custom property
by Matthias Leopold 22 Jan '18

22 Jan '18

Hi, is there a reason why the viodiskcache custom property isn't honored when using VirtIO-SCSI? On a Cinder (Ceph) disk "viodiskcache=writeback" is ignored with VirtIO-SCSI and honored when using VirtIO. On an iSCSI disk "viodiskcache=writeback" is ignored with VirtIO-SCSI and the VM can't be started when using VirtIO with "unsupported configuration: native I/O needs either no disk cache or directsync cache mode, QEMU will fallback to aio=threads" We actually want to use "viodiskcache=writeback" with Cinder (Ceph) disks. oVirt version: 4.1.8 Thanks Matthias

3 4

Configuration of FCoE in oVirt 4.2 on HP BladeSystem c7000
by Gunder Johansen 22 Jan '18

22 Jan '18

------=_Part_3635619_1374417168.1515792280983 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit We are running oVirt 4.2 on a HP BladeSystem c7000 with six BL460 gen 9 blade servers, all running oVirt Node 4.2 distro. We want to configure FCoE for the storage domains, but I find no places in the GUI where I can define the logical network. I guess the documentation on oVirt site (https://www.ovirt.org/documentation/admin-guide/chap-Logical_Networks/) is outdated with respect to FCoE. Can anyone tell us how to configure FCoE in oVirt 4.2? Thanks in advance for your help. ------=_Part_3635619_1374417168.1515792280983 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <html><head></head><body><div style=3D"color:#000; background-color:#fff; f= ont-family:Courier New, courier, monaco, monospace, sans-serif;font-size:13= px"><div style=3D"margin-bottom: 6px; font-family: Helvetica, Arial, sans-s= erif; color: rgb(29, 33, 41); font-size: 14px;" id=3D"yui_3_16_0_ym19_1_151= 5490309249_45448" dir=3D"ltr">We are running oVirt 4.2 on a HP BladeSystem = c7000 with six BL460 gen 9 blade servers, all running oVirt Node 4.2 distro= . We want to configure FCoE for the storage domains, but I find no places i= n the GUI where I can define the logical network. I guess the documentation= on oVirt site (https://www.ovirt.org/documentation/admin-guide/chap-Logica= l_Networks/) is outdated with respect to FCoE. Can anyone tell us how to co= nfigure FCoE in oVirt 4.2?</div><div id=3D"yui_3_16_0_ym19_1_1515490309249_= 45475"><br></div><div style=3D"margin-top: 6px; display: inline; font-famil= y: Helvetica, Arial, sans-serif; color: rgb(29, 33, 41); font-size: 14px;" = dir=3D"ltr" id=3D"yui_3_16_0_ym19_1_1515490309249_45449">Thanks in advance = for your help.</div></div></body></html> ------=_Part_3635619_1374417168.1515792280983--

4 4

Re: [ovirt-users] hosted-engine unknow stale-data
by Artem Tambovskiy 22 Jan '18

22 Jan '18

Explored logs on both hosts. broker.log shows no errors. agent.log looking not good: on host1 (which running hosted engine) : MainThread::ERROR::2018-01-12 21:51:03,883::agent::205::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 191, in _run_agent return action(he) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 64, in action_proper return he.start_monitoring() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 411, in start_monitoring self._initialize_sanlock() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 749, in _initialize_sanlock "Failed to initialize sanlock, the number of errors has" SanlockInitializationError: Failed to initialize sanlock, the number of errors has exceeded the limit MainThread::ERROR::2018-01-12 21:51:03,884::agent::206::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Trying to restart agent MainThread::WARNING::2018-01-12 21:51:08,889::agent::209::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Restarting agent, attempt '1' MainThread::INFO::2018-01-12 21:51:08,919::hosted_engine::242::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_hostname) Found certificate common name: ovirt1.telia.ru MainThread::INFO::2018-01-12 21:51:08,921::hosted_engine::604::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm) Initializing VDSM MainThread::INFO::2018-01-12 21:51:11,398::hosted_engine::630::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Connecting the storage MainThread::INFO::2018-01-12 21:51:11,399::storage_server::220::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(validate_storage_server) Validating storage server MainThread::INFO::2018-01-12 21:51:13,725::storage_server::239::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server MainThread::INFO::2018-01-12 21:51:18,390::storage_server::246::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server MainThread::INFO::2018-01-12 21:51:18,423::storage_server::253::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Refreshing the storage domain MainThread::INFO::2018-01-12 21:51:18,689::hosted_engine::663::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Preparing images MainThread::INFO::2018-01-12 21:51:18,690::image::126::ovirt_hosted_engine_ha.lib.image.Image::(prepare_images) Preparing images MainThread::INFO::2018-01-12 21:51:21,895::hosted_engine::666::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Refreshing vm.conf MainThread::INFO::2018-01-12 21:51:21,895::config::493::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_vm_conf) Reloading vm.conf from the shared storage domain MainThread::INFO::2018-01-12 21:51:21,896::config::416::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Trying to get a fresher copy of vm configuration from the OVF_STORE MainThread::INFO::2018-01-12 21:51:21,896::ovf_store::132::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) Extracting Engine VM OVF from the OVF_STORE MainThread::INFO::2018-01-12 21:51:21,897::ovf_store::134::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) OVF_STORE volume path: /var/run/vdsm/storage/4a7f8717-9bb0-4d80-8016-498fa4b88162/5cabd8e1-5f4b-469e-becc-227469e03f5c/8048cbd7-77e2-4805-9af4-d109fa36dfcf MainThread::INFO::2018-01-12 21:51:21,915::config::435::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Found an OVF for HE VM, trying to convert MainThread::INFO::2018-01-12 21:51:21,918::config::440::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Got vm.conf from OVF_STORE MainThread::INFO::2018-01-12 21:51:21,919::hosted_engine::509::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_broker) Initializing ha-broker connection MainThread::INFO::2018-01-12 21:51:21,919::brokerlink::130::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) Starting monitor ping, options {'addr': '80.239.162.97'} MainThread::INFO::2018-01-12 21:51:21,922::brokerlink::141::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) Success, id 140547104457680 MainThread::INFO::2018-01-12 21:51:21,922::brokerlink::130::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) Starting monitor mgmt-bridge, options {'use_ssl': 'true', 'bridge_name': 'ovirtmgmt', 'address': '0'} MainThread::INFO::2018-01-12 21:51:21,936::brokerlink::141::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) Success, id 140547104458064 MainThread::INFO::2018-01-12 21:51:21,936::brokerlink::130::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) Starting monitor mem-free, options {'use_ssl': 'true', 'address': '0'} MainThread::INFO::2018-01-12 21:51:21,938::brokerlink::141::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) Success, id 140547104458448 MainThread::INFO::2018-01-12 21:51:21,939::brokerlink::130::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) Starting monitor cpu-load-no-engine, options {'use_ssl': 'true', 'vm_uuid': 'b366e466-b0ea-4a09-866b-d0248d7523a6', 'address': '0'} MainThread::INFO::2018-01-12 21:51:21,940::brokerlink::141::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) Success, id 140547104457552 MainThread::INFO::2018-01-12 21:51:21,941::brokerlink::130::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) Starting monitor engine-health, options {'use_ssl': 'true', 'vm_uuid': 'b366e466-b0ea-4a09-866b-d0248d7523a6', 'address': '0'} MainThread::INFO::2018-01-12 21:51:21,942::brokerlink::141::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) Success, id 140547104459792 MainThread::INFO::2018-01-12 21:51:26,951::brokerlink::179::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(set_storage_domain) Success, id 140546772847056 MainThread::INFO::2018-01-12 21:51:26,952::hosted_engine::601::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_broker) Broker initialized, all submonitors started MainThread::INFO::2018-01-12 21:51:27,049::hosted_engine::704::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_sanlock) Ensuring lease for lockspace hosted-engine, host id 1 is acquired (file: /var/run/vdsm/storage/4a7f8717-9bb0-4d80-8016-498fa4b88162/093faa75-5e33-4559-84fa-1f1f8d48153b/911c7637-b49d-463e-b186-23b404e50769) MainThread::INFO::2018-01-12 21:53:48,067::hosted_engine::745::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_sanlock) Failed to acquire the lock. Waiting '5's before the next attempt MainThread::INFO::2018-01-12 21:56:14,088::hosted_engine::745::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_sanlock) Failed to acquire the lock. Waiting '5's before the next attempt MainThread::INFO::2018-01-12 21:58:40,111::hosted_engine::745::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_sanlock) Failed to acquire the lock. Waiting '5's before the next attempt MainThread::INFO::2018-01-12 22:01:06,133::hosted_engine::745::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_sanlock) Failed to acquire the lock. Waiting '5's before the next attempt agent.log from second host MainThread::INFO::2018-01-12 22:01:37,241::hosted_engine::630::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Connecting the storage MainThread::INFO::2018-01-12 22:01:37,242::storage_server::220::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(validate_storage_server) Validating storage server MainThread::INFO::2018-01-12 22:01:39,540::hosted_engine::639::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Storage domain reported as valid and reconnect is not forced. MainThread::INFO::2018-01-12 22:01:41,939::hosted_engine::453::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUnexpectedlyDown (score: 0) MainThread::INFO::2018-01-12 22:01:52,150::config::493::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_vm_conf) Reloading vm.conf from the shared storage domain MainThread::INFO::2018-01-12 22:01:52,150::config::416::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Trying to get a fresher copy of vm configuration from the OVF_STORE MainThread::INFO::2018-01-12 22:01:52,151::ovf_store::132::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) Extracting Engine VM OVF from the OVF_STORE MainThread::INFO::2018-01-12 22:01:52,153::ovf_store::134::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) OVF_STORE volume path: /var/run/vdsm/storage/4a7f8717-9bb0-4d80-8016-498fa4b88162/5cabd8e1-5f4b-469e-becc-227469e03f5c/8048cbd7-77e2-4805-9af4-d109fa36dfcf MainThread::INFO::2018-01-12 22:01:52,174::config::435::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Found an OVF for HE VM, trying to convert MainThread::INFO::2018-01-12 22:01:52,179::config::440::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Got vm.conf from OVF_STORE MainThread::INFO::2018-01-12 22:01:52,189::hosted_engine::604::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm) Initializing VDSM MainThread::INFO::2018-01-12 22:01:54,586::hosted_engine::630::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Connecting the storage MainThread::INFO::2018-01-12 22:01:54,587::storage_server::220::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(validate_storage_server) Validating storage server MainThread::INFO::2018-01-12 22:01:56,903::hosted_engine::639::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Storage domain reported as valid and reconnect is not forced. MainThread::INFO::2018-01-12 22:01:59,299::states::682::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(score) Score is 0 due to unexpected vm shutdown at Fri Jan 12 21:57:48 2018 MainThread::INFO::2018-01-12 22:01:59,299::hosted_engine::453::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUnexpectedlyDown (score: 0) MainThread::INFO::2018-01-12 22:02:09,659::config::493::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_vm_conf) Reloading vm.conf from the shared storage domain MainThread::INFO::2018-01-12 22:02:09,659::config::416::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Trying to get a fresher copy of vm configuration from the OVF_STORE MainThread::INFO::2018-01-12 22:02:09,660::ovf_store::132::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) Extracting Engine VM OVF from the OVF_STORE MainThread::INFO::2018-01-12 22:02:09,663::ovf_store::134::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) OVF_STORE volume path: /var/run/vdsm/storage/4a7f8717-9bb0-4d80-8016-498fa4b88162/5cabd8e1-5f4b-469e-becc-227469e03f5c/8048cbd7-77e2-4805-9af4-d109fa36dfcf MainThread::INFO::2018-01-12 22:02:09,683::config::435::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Found an OVF for HE VM, trying to convert MainThread::INFO::2018-01-12 22:02:09,688::config::440::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Got vm.conf from OVF_STORE MainThread::INFO::2018-01-12 22:02:09,698::hosted_engine::604::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm) Initializing VDSM MainThread::INFO::2018-01-12 22:02:12,112::hosted_engine::630::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Connecting the storage MainThread::INFO::2018-01-12 22:02:12,113::storage_server::220::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(validate_storage_server) Validating storage server MainThread::INFO::2018-01-12 22:02:14,444::hosted_engine::639::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Storage domain reported as valid and reconnect is not forced. MainThread::INFO::2018-01-12 22:02:16,859::states::682::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(score) Score is 0 due to unexpected vm shutdown at Fri Jan 12 21:57:47 2018 MainThread::INFO::2018-01-12 22:02:16,859::hosted_engine::453::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUnexpectedlyDown (score: 0) MainThread::INFO::2018-01-12 22:02:27,100::config::493::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_vm_conf) Reloading vm.conf from the shared storage domain MainThread::INFO::2018-01-12 22:02:27,100::config::416::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Trying to get a fresher copy of vm configuration from the OVF_STORE MainThread::INFO::2018-01-12 22:02:27,101::ovf_store::132::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) Extracting Engine VM OVF from the OVF_STORE MainThread::INFO::2018-01-12 22:02:27,103::ovf_store::134::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngineVMOVF) OVF_STORE volume path: /var/run/vdsm/storage/4a7f8717-9bb0-4d80-8016-498fa4b88162/5cabd8e1-5f4b-469e-becc-227469e03f5c/8048cbd7-77e2-4805-9af4-d109fa36dfcf MainThread::INFO::2018-01-12 22:02:27,125::config::435::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Found an OVF for HE VM, trying to convert MainThread::INFO::2018-01-12 22:02:27,129::config::440::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Got vm.conf from OVF_STORE MainThread::INFO::2018-01-12 22:02:27,130::states::667::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine down, local host does not have best score MainThread::INFO::2018-01-12 22:02:27,139::hosted_engine::604::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm) Initializing VDSM MainThread::INFO::2018-01-12 22:02:29,584::hosted_engine::630::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Connecting the storage MainThread::INFO::2018-01-12 22:02:29,586::storage_server::220::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(validate_storage_server) Validating storage server Any suggestions how to resolve this . regards, Artem On Fri, Jan 12, 2018 at 7:08 PM, Artem Tambovskiy < artem.tambovskiy(a)gmail.com> wrote: > Trying to fix one thing I broke another :( > > I fixed mnt_options for hosted engine storage domain and installed latest > security patches to my hosts and hosted engine. All VM's up and running, > but hosted_engine --vm-status reports about issues: > > [root@ovirt1 ~]# hosted-engine --vm-status > > > --== Host 1 status ==-- > > conf_on_shared_storage : True > Status up-to-date : False > Hostname : ovirt2 > Host ID : 1 > Engine status : unknown stale-data > Score : 0 > stopped : False > Local maintenance : False > crc32 : 193164b8 > local_conf_timestamp : 8350 > Host timestamp : 8350 > Extra metadata (valid at timestamp): > metadata_parse_version=1 > metadata_feature_version=1 > timestamp=8350 (Fri Jan 12 19:03:54 2018) > host-id=1 > score=0 > vm_conf_refresh_time=8350 (Fri Jan 12 19:03:54 2018) > conf_on_shared_storage=True > maintenance=False > state=EngineUnexpectedlyDown > stopped=False > timeout=Thu Jan 1 05:24:43 1970 > > > --== Host 2 status ==-- > > conf_on_shared_storage : True > Status up-to-date : False > Hostname : ovirt1.telia.ru > Host ID : 2 > Engine status : unknown stale-data > Score : 0 > stopped : True > Local maintenance : False > crc32 : c7037c03 > local_conf_timestamp : 7530 > Host timestamp : 7530 > Extra metadata (valid at timestamp): > metadata_parse_version=1 > metadata_feature_version=1 > timestamp=7530 (Fri Jan 12 16:10:12 2018) > host-id=2 > score=0 > vm_conf_refresh_time=7530 (Fri Jan 12 16:10:12 2018) > conf_on_shared_storage=True > maintenance=False > state=AgentStopped > stopped=True > [root@ovirt1 ~]# > > > > from second host situation looks a bit different: > > > [root@ovirt2 ~]# hosted-engine --vm-status > > > --== Host 1 status ==-- > > conf_on_shared_storage : True > Status up-to-date : True > Hostname : ovirt2 > Host ID : 1 > Engine status : {"reason": "vm not running on this > host", "health": "bad", "vm": "down", "detail": "unknown"} > Score : 0 > stopped : False > Local maintenance : False > crc32 : 78eabdb6 > local_conf_timestamp : 8403 > Host timestamp : 8402 > Extra metadata (valid at timestamp): > metadata_parse_version=1 > metadata_feature_version=1 > timestamp=8402 (Fri Jan 12 19:04:47 2018) > host-id=1 > score=0 > vm_conf_refresh_time=8403 (Fri Jan 12 19:04:47 2018) > conf_on_shared_storage=True > maintenance=False > state=EngineUnexpectedlyDown > stopped=False > timeout=Thu Jan 1 05:24:43 1970 > > > --== Host 2 status ==-- > > conf_on_shared_storage : True > Status up-to-date : False > Hostname : ovirt1.telia.ru > Host ID : 2 > Engine status : unknown stale-data > Score : 0 > stopped : True > Local maintenance : False > crc32 : c7037c03 > local_conf_timestamp : 7530 > Host timestamp : 7530 > Extra metadata (valid at timestamp): > metadata_parse_version=1 > metadata_feature_version=1 > timestamp=7530 (Fri Jan 12 16:10:12 2018) > host-id=2 > score=0 > vm_conf_refresh_time=7530 (Fri Jan 12 16:10:12 2018) > conf_on_shared_storage=True > maintenance=False > state=AgentStopped > stopped=True > > > WebGUI shows that engine running on host ovirt1. > Gluster looks fine > [root@ovirt1 ~]# gluster volume status engine > Status of volume: engine > Gluster process TCP Port RDMA Port Online > Pid > ------------------------------------------------------------ > ------------------ > Brick ovirt1.telia.ru:/oVirt/engine 49169 0 Y > 3244 > Brick ovirt2.telia.ru:/oVirt/engine 49179 0 Y > 20372 > Brick ovirt3.telia.ru:/oVirt/engine 49206 0 Y > 16609 > Self-heal Daemon on localhost N/A N/A Y > 117868 > Self-heal Daemon on ovirt2.telia.ru N/A N/A Y > 20521 > Self-heal Daemon on ovirt3 N/A N/A Y > 25093 > > Task Status of Volume engine > ------------------------------------------------------------ > ------------------ > There are no active volume tasks > > How to resolve this issue? > > > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > >

4 12

Windows Safely Remove Hardware
by Стаценко Константин Юрьевич 22 Jan '18

22 Jan '18

--_000_3146965f26a84fbda26a43c3f5f55606msk1exchmb07interraoru_ Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: quoted-printable Hello! What is the best way to remove options like eject virtual machine disks/con= trollers or nics from "Safely Remove Hardware" in Windows OS ? Currently running oVirt 4.1.8. Thanks. --_000_3146965f26a84fbda26a43c3f5f55606msk1exchmb07interraoru_ Content-Type: text/html; charset="koi8-r" Content-Transfer-Encoding: quoted-printable <html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr= osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" = xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:= //www.w3.org/TR/REC-html40"> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dkoi8-r"> <meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered medium)"> <style></style> </head> <body lang=3D"RU" link=3D"#0563C1" vlink=3D"#954F72"> <div class=3D"WordSection1"> <p class=3D"MsoNormal"><span lang=3D"EN-US">Hello!<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">What is the best way to remove = options like eject virtual machine disks/controllers or nics from “Sa= fely Remove Hardware” in Windows OS ?<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Currently running oVirt 4.1.8.<= o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"EN-US">Thanks.<o:p></o:p></span></p> </div> </body> </html> --_000_3146965f26a84fbda26a43c3f5f55606msk1exchmb07interraoru_--

1 0

Re: [ovirt-users] Q: Optimal settings for DB hosting
by Yaniv Kaul 22 Jan '18

22 Jan '18

On Jan 19, 2018 10:52 AM, "andreil1" <andreil1(a)starlett.lv> wrote: Hi ! What is optimal setting for oVirt KVM guests for database hosting on Xeon server (2 x Xeon 4-core each) (in my case this is Firebird based accounting/stock control system with several clients active)? 1st of course its preallocated disk image. VirtO-SCSI enabled - It's not clear that virtio-scsi is faster than virtio-blk in all cases. Test. - What's the backend storage? Migration disabled. Balloning disabled. CPU shares disabled Pass-through host CPU enabled What about NUMA and pinning? What should be other CPU settings? For example, Xeon have 2 threads per core, should I set in oVirt 1 or 2 threads per virtual CPU? IO Threads on or off? On. Any idea of NUMA settings ? Indeed. + Huge pages, in both host and guest. Node running 4 VMs total, CPU load is quite low, RAM is enough to preallocate for each VM + 4GB for node itself. In short, use high performance VM. See ovirt.org feature page. Y. Thanks in advance ! Andrei _______________________________________________ Users mailing list Users(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/users

2 3

Questions about converged infrastructure setup and glusterFS sizing/performance
by Jayme 22 Jan '18

22 Jan '18

I am attempting to narrow down choices for storage in a new oVirt build that will eventually be used for a mix of dev and production servers. My current space usage excluding backups sits at about only 1TB so I figure 3-5 TB would be more than enough for VM storage only + some room to grow. There will be around 24 linux VMs total but 80% of them are VERY low usage and low spec servers. I've been considering a 3 host hyperconverged oVirt setup, replica 3 arbiter 1 setup with a disaster recovery plan to replicate the gluster volume to a separate server. I would of course do additional incremental backups to an alternate server as well probably with rsync or some other method. Some questions: 1. Is it recommended to use SSDs for glusterFS or can the performance of regular server/sas drives be sufficient enough performance. If using SSDs is it recommended to use enterprise SSDs are consumer SSDs good enough due to the redundancy of glusterFS? I would love to hear of any use cases from any of you regarding hardware specs you used in hyperconverged setups and what level of performance you are seeing. 2. Is it recommended to RAID the drives that form the gluster bricks? If so what raid level? 3. How do I calculate how much space will be usable in a replicate 3 arbiter 1 configuration? Will it be 75% of total drive capacity minus what I lose from raid (if I raid the drives)? 4. For replication of the gluster volume, is it possible for me to replicate the entire volume to a single drive/raid array in an alternate server or does the replicated volume need to match the configuration of the main glusterFS volume (i.e. same amount of drives/configuration etc). 5. Has the meltdown bug caused or expected to cause major issues with oVirt hyperconverged setup due to performance loss from the patches. I've been reading articles suggesting up to 30% performance loss on some converged/storage setups due to how CPU intensive converged setups are. Thanks in advance!

2 1

OVS not running / logwatch error after upgrade from 4.0.6 to 4.1.8
by Derek Atkins 22 Jan '18

22 Jan '18

Hi, I recently upgraded my 1-host ovirt deployment from 4.0.6 to 4.1.8. Since then, the host has been reporting a cron.daily error: /etc/cron.daily/logrotate: logrotate_script: line 4: cd: /var/run/openvswitch: No such file or directory This isn't surprising, since: # systemctl status openvswitch ● openvswitch.service - Open vSwitch Loaded: loaded (/usr/lib/systemd/system/openvswitch.service; disabled; vendor preset: disabled) Active: inactive (dead) The host was just upgraded by "yum update". Was there anything special that needed to happen after the update? Do I *NEED* OVS running? The VMs all seem to be behaving properly. Thanks, -derek -- Derek Atkins 617-623-3745 derek(a)ihtfp.com www.ihtfp.com Computer and Internet Security Consultant

2 3

New blog on ovirt.org - Build oVirt Reports Using Grafana
by Yaniv Kaul 21 Jan '18

21 Jan '18

Grafana <https://grafana.com/>, The open platform for beautiful analytics and monitoring, recently added support for PostgreSQL <http://docs.grafana.org/features/datasources/postgres/>. It in now possible to connect Grafana to oVirt DWH <https://www.ovirt.org/documentation/how-to/reports/dwh/>, in order to visualize and monitor the oVirt environment. Read more @ https://ovirt.org/blog/2018/01/ovirt-report-using-grafana/

1 0

oVirt storage access failure from host
by Alex K 19 Jan '18

19 Jan '18

Hi All, I have a 3 server ovirt 4.1 selft hosted setup with gluster replica 3. I see that suddenly one of the hosts reported as unresponsive and at same time the /var/log/messages logged: ovirt-ha-broker ovirt_hosted_engine_ha.broker.listener.ConnectionHandler ERROR Error handling request, data: 'set-storage-domain FilesystemBackend dom_type=glusterfs sd_uuid=ad7b9e2a-7ae3-46ad-9429-5f5ef452eac8'#012Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/listener.py", line 166, in handle#012 data)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/listener.py", line 299, in _dispatch#012 .set_storage_domain(client, sd_type, **options)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py", line 66, in set_storage_domain#012 self._backends[client].connect()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/storage_backends.py", line 462, in connect#012 self._dom_type)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/storage_backends.py", line 107, in get_domain_path#012 " in {1}".format(sd_uuid, parent))#012BackendFailureException: path to storage domain ad7b9e2a-7ae3-46ad-9429-5f5ef452eac8 not found in /rhev/data-center/mnt/glusterSD Jan 15 11:04:56 v1 journal: vdsm root ERROR failed to retrieve Hosted Engine HA info#012Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 231, in _getHaInfo#012 stats = instance.get_all_stats()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 103, in get_all_stats#012 self._configure_broker_conn(broker)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 180, in _configure_broker_conn#012 dom_type=dom_type)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 177, in set_storage_domain#012 .format(sd_type, options, e))#012RequestError: Failed to set storage domain FilesystemBackend, options {'dom_type': 'glusterfs', 'sd_uuid': 'ad7b9e2a-7ae3-46ad-9429-5f5ef452eac8'}: Request failed: <class 'ovirt_hosted_engine_ha.lib.storage_backends.BackendFailureException'> At VDSM logs i see the following continuously logged: [jsonrpc.JsonRpcServer] RPC call VM.getStats failed (error 1) in 0.00 seconds (__init__:539) No errors seen at gluster at same time frame. Any hints on what is causing this issue? It seems a storage access issue but gluster was up and volumes ok. The VMs that I am running on top are Windows 10 and Windows 2016 64 bit. Thanx, Alex

2 2

correct settings for gluster based storage domain
by Artem Tambovskiy 19 Jan '18

19 Jan '18

I'm still troubleshooting the my oVirt 4.1.8 cluster and idea came to my mind that I have an issue with storage settings for hosted_engine storage domain. But in general if I have a 2 ovirt nodes running gluster + 3rd host as arbiter, how the settings should looks like? lets say I have a 3 nodes: ovirt1.domain.com (gluster + ovirt) ovirt2.domain.com (gluster + ovirt) ovirt3.domain.com (gluster) How the correct storage domain config should looks like? Option 1: /etc/ovirt-hosted-engine/hosted-engine.conf .... storage=ovirt1.domain.com:/engine mnt_options=backup-volfile-servers=ovirt2.domain.com:ovirt3.domain.com Option 2: /etc/ovirt-hosted-engine/hosted-engine.conf .... storage=localhost:/engine mnt_options=backup-volfile-servers=ovirt1.domain.com:ovirt2.domain.com:o virt3.domain.com Option 3: Setup a DNS record gluster.domain.com pointing to IP addresses of gluster nodes /etc/ovirt-hosted-engine/hosted-engine.conf .... storage=gluster.domain.com:/engine mnt_options= Of course its related not only to hosted engine domain, but to all gluster based storage domains. Thank you in advance! Regards, Artem

2 2

[ANN] oVirt 4.2.1 Second Release Candidate is now available
by Sandro Bonazzola 19 Jan '18

19 Jan '18

The oVirt Project is pleased to announce the availability of the oVirt 4.2.1 Second Release Candidate, as of January 18th, 2017 This update is a release candidate of the second in a series of stabilization updates to the 4.2 series. This is pre-release software. This pre-release should not to be used in production. [WARNING] right after we finished to compose the release candidate we discovered a regression in a disaster recovery flow causing wrong MAC address to be assigned to re-imported VMs. This release is available now for: * Red Hat Enterprise Linux 7.4 or later * CentOS Linux (or similar) 7.4 or later This release supports Hypervisor Hosts running: * Red Hat Enterprise Linux 7.4 or later * CentOS Linux (or similar) 7.4 or later * oVirt Node 4.2 See the release notes [1] for installation / upgrade instructions and a list of new features and bugs fixed. Notes: - oVirt Appliance is already available - oVirt Node will be available soon [2] Additional Resources: * Read more about the oVirt 4.2.1 release highlights: http://www.ovirt.org/release/4.2.1/ * Get more oVirt Project updates on Twitter: https://twitter.com/ovirt * Check out the latest project news on the oVirt blog: http://www.ovirt.org/blog/ [1] http://www.ovirt.org/release/4.2.1/ [2] http://resources.ovirt.org/pub/ovirt-4.2-pre/iso/ -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

2 2

Re: [ovirt-users] Problems with some vms
by Endre Karlson 19 Jan '18

19 Jan '18

One brick was at a point down for replacement. It has been replaced and all vols are up Status of volume: data Gluster process TCP Port RDMA Port Online Pid ------------------------------------------------------------------------------ Brick ovirt0:/gluster/brick3/data 49152 0 Y 22467 Brick ovirt2:/gluster/brick3/data 49152 0 Y 20736 Brick ovirt3:/gluster/brick3/data 49152 0 Y 23148 Brick ovirt0:/gluster/brick4/data 49153 0 Y 22497 Brick ovirt2:/gluster/brick4/data 49153 0 Y 20742 Brick ovirt3:/gluster/brick4/data 49153 0 Y 23158 Brick ovirt0:/gluster/brick5/data 49154 0 Y 22473 Brick ovirt2:/gluster/brick5/data 49154 0 Y 20748 Brick ovirt3:/gluster/brick5/data 49154 0 Y 23156 Brick ovirt0:/gluster/brick6/data 49155 0 Y 22479 Brick ovirt2:/gluster/brick6_1/data 49161 0 Y 21203 Brick ovirt3:/gluster/brick6/data 49155 0 Y 23157 Brick ovirt0:/gluster/brick7/data 49156 0 Y 22485 Brick ovirt2:/gluster/brick7/data 49156 0 Y 20763 Brick ovirt3:/gluster/brick7/data 49156 0 Y 23155 Brick ovirt0:/gluster/brick8/data 49157 0 Y 22491 Brick ovirt2:/gluster/brick8/data 49157 0 Y 20771 Brick ovirt3:/gluster/brick8/data 49157 0 Y 23154 Self-heal Daemon on localhost N/A N/A Y 23238 Bitrot Daemon on localhost N/A N/A Y 24870 Scrubber Daemon on localhost N/A N/A Y 24889 Self-heal Daemon on ovirt2 N/A N/A Y 24271 Bitrot Daemon on ovirt2 N/A N/A Y 24856 Scrubber Daemon on ovirt2 N/A N/A Y 24866 Self-heal Daemon on ovirt0 N/A N/A Y 29409 Bitrot Daemon on ovirt0 N/A N/A Y 5457 Scrubber Daemon on ovirt0 N/A N/A Y 5468 Task Status of Volume data ------------------------------------------------------------------------------ There are no active volume tasks Status of volume: engine Gluster process TCP Port RDMA Port Online Pid ------------------------------------------------------------------------------ Brick ovirt0:/gluster/brick1/engine 49158 0 Y 22511 Brick ovirt2:/gluster/brick1/engine 49158 0 Y 20780 Brick ovirt3:/gluster/brick1/engine 49158 0 Y 23199 Self-heal Daemon on localhost N/A N/A Y 23238 Self-heal Daemon on ovirt0 N/A N/A Y 29409 Self-heal Daemon on ovirt2 N/A N/A Y 24271 Task Status of Volume engine ------------------------------------------------------------------------------ There are no active volume tasks Status of volume: iso Gluster process TCP Port RDMA Port Online Pid ------------------------------------------------------------------------------ Brick ovirt0:/gluster/brick2/iso 49159 0 Y 22520 Brick ovirt2:/gluster/brick2/iso 49159 0 Y 20789 Brick ovirt3:/gluster/brick2/iso 49159 0 Y 23208 NFS Server on localhost N/A N/A N N/A Self-heal Daemon on localhost N/A N/A Y 23238 NFS Server on ovirt2 N/A N/A N N/A Self-heal Daemon on ovirt2 N/A N/A Y 24271 NFS Server on ovirt0 N/A N/A N N/A Self-heal Daemon on ovirt0 N/A N/A Y 29409 Task Status of Volume iso ------------------------------------------------------------------------------ There are no active volume tasks 2018-01-17 8:13 GMT+01:00 Gobinda Das <godas(a)redhat.com>: > Hi, > I can see some error in log: > [2018-01-14 11:19:49.886571] E [socket.c:2309:socket_connect_finish] > 0-engine-client-0: connection to 10.2.0.120:24007 failed (Connection > timed out) > [2018-01-14 11:20:05.630669] E [socket.c:2309:socket_connect_finish] > 0-engine-client-0: connection to 10.2.0.120:24007 failed (Connection > timed out) > [2018-01-14 12:01:09.089925] E [MSGID: 114058] > [client-handshake.c:1527:client_query_portmap_cbk] 0-engine-client-0: > failed to get the port number for remote subvolume. Please run 'gluster > volume status' on server to see if brick process is running. > [2018-01-14 12:01:09.090048] I [MSGID: 114018] > [client.c:2280:client_rpc_notify] 0-engine-client-0: disconnected from > engine-client-0. Client process will keep trying to connect to glusterd > until brick's port is available > > Can you please check gluster volume status and see if all bricks are up? > > On Wed, Jan 17, 2018 at 12:24 PM, Endre Karlson <endre.karlson(a)gmail.com> > wrote: > >> It's there now for each of the hosts. ovirt1 is not in service yet. >> >> 2018-01-17 5:52 GMT+01:00 Gobinda Das <godas(a)redhat.com>: >> >>> In the above url only data and iso mnt log present,But there is no >>> engine and vmstore mount log. >>> >>> On Wed, Jan 17, 2018 at 1:26 AM, Endre Karlson <endre.karlson(a)gmail.com> >>> wrote: >>> >>>> Hi, all logs are located here: https://www.dropbox.com/ >>>> sh/3qzmwe76rkt09fk/AABzM9rJKbH5SBPWc31Npxhma?dl=0 for the mounts >>>> >>>> additionally we replaced a broken disk that is now resynced. >>>> >>>> 2018-01-15 11:17 GMT+01:00 Gobinda Das <godas(a)redhat.com>: >>>> >>>>> Hi Endre, >>>>> Mount logs will be in below format inside /var/log/glusterfs : >>>>> >>>>> /var/log/glusterfs/rhev-data-center-mnt-glusterSD-*\:_engine.log >>>>> /var/log/glusterfs/rhev-data-center-mnt-glusterSD-*\:_data.log >>>>> /var/log/glusterfs/rhev-data-center-mnt-glusterSD-*\:_vmstore.log >>>>> >>>>> On Mon, Jan 15, 2018 at 11:57 AM, Endre Karlson < >>>>> endre.karlson(a)gmail.com> wrote: >>>>> >>>>>> Hi. >>>>>> >>>>>> What are the gluster mount logs ? >>>>>> >>>>>> I have these gluster logs. >>>>>> cli.log etc-glusterfs-glusterd.vol.log >>>>>> glfsheal-engine.log glusterd.log nfs.log >>>>>> rhev-data-center-mnt-glusterSD-ovirt0:_engine.log >>>>>> rhev-data-center-mnt-glusterSD-ovirt3:_iso.log >>>>>> cmd_history.log glfsheal-data.log glfsheal-iso.log >>>>>> glustershd.log rhev-data-center-mnt-glusterSD-ovirt0:_data.log >>>>>> rhev-data-center-mnt-glusterSD-ovirt0:_iso.log statedump.log >>>>>> >>>>>> >>>>>> I am running version >>>>>> glusterfs-server-3.12.4-1.el7.x86_64 >>>>>> glusterfs-geo-replication-3.12.4-1.el7.x86_64 >>>>>> libvirt-daemon-driver-storage-gluster-3.2.0-14.el7_4.7.x86_64 >>>>>> glusterfs-libs-3.12.4-1.el7.x86_64 >>>>>> glusterfs-api-3.12.4-1.el7.x86_64 >>>>>> python2-gluster-3.12.4-1.el7.x86_64 >>>>>> glusterfs-client-xlators-3.12.4-1.el7.x86_64 >>>>>> glusterfs-cli-3.12.4-1.el7.x86_64 >>>>>> glusterfs-events-3.12.4-1.el7.x86_64 >>>>>> glusterfs-rdma-3.12.4-1.el7.x86_64 >>>>>> vdsm-gluster-4.20.9.3-1.el7.centos.noarch >>>>>> glusterfs-3.12.4-1.el7.x86_64 >>>>>> glusterfs-fuse-3.12.4-1.el7.x86_64 >>>>>> >>>>>> // Endre >>>>>> >>>>>> 2018-01-15 6:11 GMT+01:00 Gobinda Das <godas(a)redhat.com>: >>>>>> >>>>>>> Hi Endre, >>>>>>> Can you please provide glusterfs mount logs? >>>>>>> >>>>>>> On Mon, Jan 15, 2018 at 6:16 AM, Darrell Budic < >>>>>>> budic(a)onholyground.com> wrote: >>>>>>> >>>>>>>> What version of gluster are you running? I’ve seen a few of these >>>>>>>> since moving my storage cluster to 12.3, but still haven’t been able to >>>>>>>> determine what’s causing it. Seems to be happening most often on VMs that >>>>>>>> haven’t been switches over to libgfapi mounts yet, but even one of those >>>>>>>> has paused once so far. They generally restart fine from the GUI, and >>>>>>>> nothing seems to need healing. >>>>>>>> >>>>>>>> ------------------------------ >>>>>>>> *From:* Endre Karlson <endre.karlson(a)gmail.com> >>>>>>>> *Subject:* [ovirt-users] Problems with some vms >>>>>>>> *Date:* January 14, 2018 at 12:55:45 PM CST >>>>>>>> *To:* users >>>>>>>> >>>>>>>> Hi, we are getting some errors with some of our vms in a 3 node >>>>>>>> server setup. >>>>>>>> >>>>>>>> 2018-01-14 15:01:44,015+0100 INFO (libvirt/events) [virt.vm] >>>>>>>> (vmId='2c34f52d-140b-4dbe-a4bd-d2cb467b0b7c') abnormal vm stop >>>>>>>> device virtio-disk0 error eother (vm:4880) >>>>>>>> >>>>>>>> We are running glusterfs for shared storage. >>>>>>>> >>>>>>>> I have tried setting global maintenance on the first server and >>>>>>>> then issuing a 'hosted-engine --vm-start' but that leads to nowhere. >>>>>>>> _______________________________________________ >>>>>>>> Users mailing list >>>>>>>> Users(a)ovirt.org >>>>>>>> http://lists.ovirt.org/mailman/listinfo/users >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Users mailing list >>>>>>>> Users(a)ovirt.org >>>>>>>> http://lists.ovirt.org/mailman/listinfo/users >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Thanks, >>>>>>> Gobinda >>>>>>> +91-9019047912 <+91%2090190%2047912> >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Thanks, >>>>> Gobinda >>>>> +91-9019047912 <+91%2090190%2047912> >>>>> >>>> >>>> >>> >>> >>> -- >>> Thanks, >>> Gobinda >>> +91-9019047912 <+91%2090190%2047912> >>> >> >> > > > -- > Thanks, > Gobinda > +91-9019047912 <+91%2090190%2047912> >

1 1

Re: [ovirt-users] oVirt home lab hardware
by Abdurrahman A. Ibrahim 19 Jan '18

19 Jan '18

One more thing, Do you have power switch? Best regards, Ab On 19 Jan 2018 10:44 am, "Abdurrahman A. Ibrahim" <a.rahman.attia(a)gmail.com> wrote: > Thank you Jayme for your reply, > > I have some concerns regarding space, noise and power consumption. > > Would you mind to share your experience regarding those three parameters > with us? > > Best regards, > Ab > > > > > On 18 Jan 2018 10:23 pm, "Jayme" <jaymef(a)gmail.com> wrote: > > For rackmount Dell R710's are fairly popular for home labs, they have good > specs and can be found at reasonable prices on ebay. > > On Thu, Jan 18, 2018 at 4:52 PM, Abdurrahman A. Ibrahim < > a.rahman.attia(a)gmail.com> wrote: > >> Hello, >> >> I am planning to buy home lab hardware to be used by oVirt. >> >> Any recommendations for used hardware i can buy from eBay for example? >> Also, have you tried oVirt on Intel NUC or any other SMB servers before? >> >> Thanks, >> Ab >> >> _______________________________________________ >> Users mailing list >> Users(a)ovirt.org >> http://lists.ovirt.org/mailman/listinfo/users >> >> > >

1 0

Q: Optimal settings for DB hosting
by andreil1 19 Jan '18

19 Jan '18

Hi ! What is optimal setting for oVirt KVM guests for database hosting on Xeon server (2 x Xeon 4-core each) (in my case this is Firebird based accounting/stock control system with several clients active)? 1st of course its preallocated disk image. VirtO-SCSI enabled Migration disabled. Balloning disabled. CPU shares disabled Pass-through host CPU enabled What should be other CPU settings? For example, Xeon have 2 threads per core, should I set in oVirt 1 or 2 threads per virtual CPU? IO Threads on or off? Any idea of NUMA settings ? Node running 4 VMs total, CPU load is quite low, RAM is enough to preallocate for each VM + 4GB for node itself. Thanks in advance ! Andrei

1 0

web 404 after reinstall ovirt-engine
by 董青龙 19 Jan '18

19 Jan '18

------=_Part_69969_201422119.1516342255693 Content-Type: text/plain; charset=GBK Content-Transfer-Encoding: base64 SGksIGFsbAogICAgICAgIEkgaW5zdGFsbGVkIG92aXJ0LWVuZ2luZSA0LjEuOC4yIGZvciB0aGUg c2Vjb25kIHRpbWUgYW5kIEkgZ290ICJzdWNjZXNzZnVsIiBhZnRlciBJIGV4Y3V0ZWQgImVuZ2lu ZS1zZXR1cCIuIEJ1dCBJIGdvdCAiNDA0IiB3aGVuIEkgdHJpZWQgdG8gYWNjZXNzIHdlYmFkbWlu IHBvcnRhbCB1c2luZyAiaHR0cHM6Ly9GUUROL292aXJ0LWVuZ2luZSIuIEJ5IHRoZSB3YXksIEkg Y291bGQgYWNjZXNzIHRoZSB3ZWIgYWZ0ZXIgSSBpbnN0YWxsZWQgb3ZpcnQtZW5naW5lIGZvciB0 aGUgZmlyc3QgdGltZS4gVGhlbiBJIGV4Y3V0ZWQgImVuZ2luZS1jbGVhbnVwIiBhbmQgInl1bSBy ZW1vdmUgb3ZpcnQtZW5naW5lIiBhbmQgaW5zdGFsbGVkIG92aXJ0LWVuZ2luZSBmb3IgdGhlIHNl Y29uZCB0aW1lLiBJIGFsc28gdHJpZWQgdG8gcmVtb3ZlIG92aXJ0LWVuZ2luZS1kd2ggYW5kIHBv c3RncmVzcWwgYnV0IGFmdGVyIEkgcmVpbnN0YWxsZWQgb3ZpcnQtZW5naW5lIEkgc3RpbGwgZ290 ICI0MDQiLgogICAgICAgIENhbiBJIGZpeCB0aGlzIHByb2JsZW0/IEhvcGUgc29tZSBjYW4gaGVs cCwgdGhhbmtzIQogICAgICAgIEhlcmUgYXJlIHNvbWUgbG9ncyBpbiAiL3Zhci9sb2cvb3ZpcnQt ZW5naW5lL2VuZ2luZS5sb2ciOgouLi4KMjAxOC0wMS0xOSAxMjo1NjowNiwyODgrMDggRVJST1Ig W29yZy5vdmlydC5lbmdpbmUudWkuZnJvbnRlbmQuc2VydmVyLmRhc2hib2FyZC5EYXNoYm9hcmRE YXRhU2VydmxldF0gKFNlcnZlclNlcnZpY2UgVGhyZWFkIFBvb2wgLS0gNjEpIFtdIENvdWxkIG5v dCBhY2Nlc3MgZW5naW5lJ3MgRFdIIGNvbmZpZ3VyYXRpb24gdGFibGU6IGphdmEuc3FsLlNRTEV4 Y2VwdGlvbjogamF2YXgucmVzb3VyY2UuUmVzb3VyY2VFeGNlcHRpb246IElKMDAwNDUzOiBVbmFi bGUgdG8gZ2V0IG1hbmFnZWQgY29ubmVjdGlvbiBmb3IgamF2YTovRU5HSU5FRGF0YVNvdXJjZQou Li4KQ2F1c2VkIGJ5OiBqYXZheC5yZXNvdXJjZS5SZXNvdXJjZUV4Y2VwdGlvbjogSUowMDA0NTM6 IFVuYWJsZSB0byBnZXQgbWFuYWdlZCBjb25uZWN0aW9uIGZvciBqYXZhOi9FTkdJTkVEYXRhU291 cmNlCi4uLgpDYXVzZWQgYnk6IGphdmF4LnJlc291cmNlLlJlc291cmNlRXhjZXB0aW9uOiBJSjAz MTA4NDogVW5hYmxlIHRvIGNyZWF0ZSBjb25uZWN0aW9uCi4uLgpDYXVzZWQgYnk6IG9yZy5wb3N0 Z3Jlc3FsLnV0aWwuUFNRTEV4Y2VwdGlvbjogRkFUQUw6IHBhc3N3b3JkIGF1dGhlbnRpY2F0aW9u IGZhaWxlZCBmb3IgdXNlciAiZW5naW5lIgouLi4KMjAxOC0wMS0xOSAxMjo1NjowNiwyOTIrMDgg V0FSTiAgW29yZy5vdmlydC5lbmdpbmUudWkuZnJvbnRlbmQuc2VydmVyLmRhc2hib2FyZC5EYXNo Ym9hcmREYXRhU2VydmxldF0gKFNlcnZlclNlcnZpY2UgVGhyZWFkIFBvb2wgLS0gNjEpIFtdIE5v IHZhbGlkIERXSCBjb25maWd1cmF0aW9ucyB3ZXJlIGZvdW5kLCBhc3N1bWluZyBEV0ggZGF0YWJh c2UgaXNuJ3Qgc2V0dXAuCjIwMTgtMDEtMTkgMTI6NTY6MDYsMjkyKzA4IElORk8gIFtvcmcub3Zp cnQuZW5naW5lLnVpLmZyb250ZW5kLnNlcnZlci5kYXNoYm9hcmQuRGFzaGJvYXJkRGF0YVNlcnZs ZXRdIChTZXJ2ZXJTZXJ2aWNlIFRocmVhZCBQb29sIC0tIDYxKSBbXSBEYXNoYm9hcmQgREIgcXVl cnkgY2FjaGUgaGFzIGJlZW4gZGlzYWJsZWQu ------=_Part_69969_201422119.1516342255693 Content-Type: text/html; charset=GBK Content-Transfer-Encoding: base64 PGRpdiBzdHlsZT0ibGluZS1oZWlnaHQ6MS43O2NvbG9yOiMwMDAwMDA7Zm9udC1zaXplOjE0cHg7 Zm9udC1mYW1pbHk6QXJpYWwiPjxkaXY+SGksIGFsbDwvZGl2PjxkaXY+Jm5ic3A7ICZuYnNwOyAm bmJzcDsgJm5ic3A7IEkgaW5zdGFsbGVkIG92aXJ0LWVuZ2luZSA0LjEuOC4yIGZvciB0aGUgc2Vj b25kIHRpbWUgYW5kIEkgZ290ICJzdWNjZXNzZnVsIiBhZnRlciBJIGV4Y3V0ZWQgImVuZ2luZS1z ZXR1cCIuIEJ1dCBJIGdvdCAiNDA0IiB3aGVuIEkgdHJpZWQgdG8gYWNjZXNzIHdlYmFkbWluIHBv cnRhbCB1c2luZyAiaHR0cHM6Ly9GUUROL292aXJ0LWVuZ2luZSIuIEJ5IHRoZSB3YXksIEkgY291 bGQgYWNjZXNzIHRoZSB3ZWIgYWZ0ZXIgSSBpbnN0YWxsZWQgb3ZpcnQtZW5naW5lIGZvciB0aGUg Zmlyc3QgdGltZS4gVGhlbiBJIGV4Y3V0ZWQgImVuZ2luZS1jbGVhbnVwIiBhbmQgInl1bSByZW1v dmUgb3ZpcnQtZW5naW5lIiBhbmQgaW5zdGFsbGVkIG92aXJ0LWVuZ2luZSBmb3IgdGhlIHNlY29u ZCB0aW1lLiBJIGFsc28gdHJpZWQgdG8gcmVtb3ZlIG92aXJ0LWVuZ2luZS1kd2ggYW5kIHBvc3Rn cmVzcWwgYnV0IGFmdGVyIEkgcmVpbnN0YWxsZWQgb3ZpcnQtZW5naW5lIEkgc3RpbGwgZ290ICI0 MDQiLjwvZGl2PjxkaXY+Jm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7IENhbiBJIGZpeCB0aGlz IHByb2JsZW0/IEhvcGUgc29tZSBjYW4gaGVscCwgdGhhbmtzITwvZGl2PjxkaXY+Jm5ic3A7ICZu YnNwOyAmbmJzcDsgJm5ic3A7IEhlcmUgYXJlIHNvbWUgbG9ncyBpbiAiL3Zhci9sb2cvb3ZpcnQt ZW5naW5lL2VuZ2luZS5sb2ciOjwvZGl2PjxkaXY+Li4uPC9kaXY+PGRpdj4yMDE4LTAxLTE5IDEy OjU2OjA2LDI4OCswOCBFUlJPUiBbb3JnLm92aXJ0LmVuZ2luZS51aS5mcm9udGVuZC5zZXJ2ZXIu ZGFzaGJvYXJkLkRhc2hib2FyZERhdGFTZXJ2bGV0XSAoU2VydmVyU2VydmljZSBUaHJlYWQgUG9v bCAtLSA2MSkgW10gQ291bGQgbm90IGFjY2VzcyBlbmdpbmUncyBEV0ggY29uZmlndXJhdGlvbiB0 YWJsZTogamF2YS5zcWwuU1FMRXhjZXB0aW9uOiBqYXZheC5yZXNvdXJjZS5SZXNvdXJjZUV4Y2Vw dGlvbjogSUowMDA0NTM6IFVuYWJsZSB0byBnZXQgbWFuYWdlZCBjb25uZWN0aW9uIGZvciBqYXZh Oi9FTkdJTkVEYXRhU291cmNlPC9kaXY+PGRpdj4uLi48L2Rpdj48ZGl2PkNhdXNlZCBieTogamF2 YXgucmVzb3VyY2UuUmVzb3VyY2VFeGNlcHRpb246IElKMDAwNDUzOiBVbmFibGUgdG8gZ2V0IG1h bmFnZWQgY29ubmVjdGlvbiBmb3IgamF2YTovRU5HSU5FRGF0YVNvdXJjZTwvZGl2PjxkaXY+Li4u PC9kaXY+PGRpdj5DYXVzZWQgYnk6IGphdmF4LnJlc291cmNlLlJlc291cmNlRXhjZXB0aW9uOiBJ SjAzMTA4NDogVW5hYmxlIHRvIGNyZWF0ZSBjb25uZWN0aW9uPC9kaXY+PGRpdj4uLi48L2Rpdj48 ZGl2PkNhdXNlZCBieTogb3JnLnBvc3RncmVzcWwudXRpbC5QU1FMRXhjZXB0aW9uOiBGQVRBTDog cGFzc3dvcmQgYXV0aGVudGljYXRpb24gZmFpbGVkIGZvciB1c2VyICJlbmdpbmUiPC9kaXY+PGRp dj4uLi48L2Rpdj48ZGl2PjIwMTgtMDEtMTkgMTI6NTY6MDYsMjkyKzA4IFdBUk4gJm5ic3A7W29y Zy5vdmlydC5lbmdpbmUudWkuZnJvbnRlbmQuc2VydmVyLmRhc2hib2FyZC5EYXNoYm9hcmREYXRh U2VydmxldF0gKFNlcnZlclNlcnZpY2UgVGhyZWFkIFBvb2wgLS0gNjEpIFtdIE5vIHZhbGlkIERX SCBjb25maWd1cmF0aW9ucyB3ZXJlIGZvdW5kLCBhc3N1bWluZyBEV0ggZGF0YWJhc2UgaXNuJ3Qg c2V0dXAuPC9kaXY+PGRpdj4yMDE4LTAxLTE5IDEyOjU2OjA2LDI5MiswOCBJTkZPICZuYnNwO1tv cmcub3ZpcnQuZW5naW5lLnVpLmZyb250ZW5kLnNlcnZlci5kYXNoYm9hcmQuRGFzaGJvYXJkRGF0 YVNlcnZsZXRdIChTZXJ2ZXJTZXJ2aWNlIFRocmVhZCBQb29sIC0tIDYxKSBbXSBEYXNoYm9hcmQg REIgcXVlcnkgY2FjaGUgaGFzIGJlZW4gZGlzYWJsZWQuPC9kaXY+PC9kaXY+PGJyPjxicj48c3Bh biB0aXRsZT0ibmV0ZWFzZWZvb3RlciI+PHA+Jm5ic3A7PC9wPjwvc3Bhbj4= ------=_Part_69969_201422119.1516342255693--

3 4

The future of backups in oVirt 4.2+
by Jayme 19 Jan '18

19 Jan '18

I've been running a non-production oVirt setup for a while and plan on building a more robust oVirt setup for eventual production use. Part of that planning of course is backup/disaster recovery options. I've been playing around with a few options to backup oVirt, I'm sure most of you are aware of them. The github webfix it python scripts, and starting to look at bacchus as well although have not set it up or tested bacchus yet. The webfix it python script is fairly simple to setup and seems to work ok as intended, even on 4.2, but it's definitely clunky in terms of having to snapshot, clone, export. A lot of resource usage. I assume bacchus most likely uses the same method of snapshot/clone/export as well (due to what was available in oVirt at the time these scripts were written). My concerns with using something like these are: 1. It's using old 3x api -- which is fine for now but probably not with oVirt 4.3 2. Inefficient / resource intensive 3. Using export domain which is depreciated >From what I've read in oVirt documentation the Export domain is depreciated. I assume that this means that instead of a export domain you could instead create a regular data domain for backups that can be detached and attached to another environment, is that the correct assumption? I've also read that it might be possible to skip the cloning of the VM part and export snapshots directly. Is this now possible in 4.2 or will be in the future? if it is possible to perform VM backups without cloning to a new VM first is anyone aware of any scripts/software that is available now which takes advantage of that? Essentially all I want is a simple solution to backup my VMs to separate NFS storage. If for some reason my main storage crashes I then have the option to connect that backup NFS storage to a secondary stand alone oVirt instance and run my VMs from there until the primary oVirt instance is repaired. What I want to avoid is implementing an older more inefficient solution that might work for a while but will eventually no longer work due to ovIrt updates in the future. I know options like active-active failover and georeplication exist but that may be too complex for my needs although I would be interested in hearing about how some of you may be implementing these features. In summary, I'd trying to figure out what the best backup option may be going forward with oVirt in the future so I can implement the best option from the start rather than having to change it all around in the near future. Thanks!

4 6

Zombie kworker process on node took 100% CPU load
by Andrei V 18 Jan '18

18 Jan '18

Hi, I've got strange problem with oVirt Node 4.2 installed on CentOS 7 with mainline 4.14 kernel & HP ProLiant ML350. Kworker process loaded 1 cpu CPU core at 100%, and since it didn't finish even after several hours, it was clearly a zombie. I logged into each VM, and powered them off with "shutdown now". From SPICE console it was seen VMs halted. However, in oVirt engine web they have been marked with tripple green down arrow. An attempt to shutdown them completely with oVirt button resulted in error message something like "Can't shutdown VM because highly available and can't be moved". Sorry, I forgot to take screenshot. Its was also impossible to shutdown the node correctly, process freezes. After restart, everything seem to work correctly. Anyone have any idea how to cope with this kind of problem? Thanks in advance Andrei

1 0

docker, kubernetes and ovirt
by Nathanaël Blanchet 18 Jan '18

18 Jan '18

Hi all, Regarding to this video : https://www.youtube.com/watch?v=JyyST4ZKne8 and ovedou.blogspot.fr/2014/03/running-docker-container-in-ovirt.html, it appears that some stuff has been done in integrated docker into ovirt. I'm interested into these two UI plugins : docker-resources (I found it in samples-uiplugins) and the kubernetes one that I didn't find anywhere. Can anyone tell me if the development of that projects are definitively stopped and the reason why? -- Nathanaël Blanchet Supervision réseau Pôle Infrastrutures Informatiques 227 avenue Professeur-Jean-Louis-Viala 34193 MONTPELLIER CEDEX 5 Tél. 33 (0)4 67 54 84 55 Fax 33 (0)4 67 54 84 14 blanchet(a)abes.fr

3 3

oVirt Node 4.1.8 -> 4.2 upgrade
by Ed Stout 18 Jan '18

18 Jan '18

Good Morning, Perhaps someone could clarify for me, please - to update oVirt Nodes from 4.1.8 -> 4.2, should I just be able to yum install the 4.2 release file, then yum update? Or, do I have to start with the 4.2 ISO then update from there? When I try the former it fails with dependency issues for the nodes, I did however manage to update a self hosted appliance engine to 4.2. I had a read through the mailing list archive and couldn't find similar issues, so perhaps I'm doing something wrong. --> Processing Dependency: lshw for package: vdsm-4.20.9.3-1.el7.centos.x86_64 --> Finished Dependency Resolution Error: Package: ovirt-hosted-engine-setup-2.2.3-1.el7.centos.noarch (ovirt-4.2) Requires: libvirt-daemon-config-network Error: Package: ovirt-hosted-engine-setup-2.2.3-1.el7.centos.noarch (ovirt-4.2) Requires: virt-install Error: Package: glusterfs-geo-replication-3.12.3-1.el7.x86_64 (ovirt-4.2-centos-gluster312) Requires: python-prettytable Error: Package: vdsm-4.20.9.3-1.el7.centos.x86_64 (ovirt-4.2) Requires: lshw Error: Package: glusterfs-events-3.12.3-1.el7.x86_64 (ovirt-4.2-centos-gluster312) Requires: python-prettytable You could try using --skip-broken to work around the problem You could try running: rpm -Va --nofiles --nodigest OS Version: RHEL - 7 - 4.1708.el7.centos OS Description: oVirt Node 4.1.8 Kernel Version: 3.10.0 - 693.11.1.el7.x86_64 KVM Version: 2.9.0 - 16.el7_4.8.1 LIBVIRT Version: libvirt-3.2.0-14.el7_4.5 VDSM Version: vdsm-4.19.43-1.el7.centos SPICE Version: 0.12.8 - 2.el7.1 GlusterFS Version: glusterfs-3.8.15-2.el7 CEPH Version: librbd1-0.94.5-2.el7 Cheers, Ed

5 8

gluster volume permission denied
by Bill James 18 Jan '18

18 Jan '18

I have one node in our cluster that has problems when migrating a VM if the VM is hosted on a gluster volume. If the VM is hosted on a NFS volume it migrates fine. /var/log/messages: Jan 16 14:36:09 ovirt1 libvirtd: 2018-01-16 22:36:09.769+0000: 9700: error : qemuProcessReportLogError:1862 : internal error: process exited while connecting to monitor: 2018-01-16T22:36:09.587405Z qemu-kvm: -drive file=/rhev/data-center/8b6303b3-79c6-4633-ae21-71b15ed00675/67b4d9aa-f174-436a-b5a1-ec7cee5f2edb/images/60 3271e4-3089-4779-aa1c-faf280fabd17/7c7f7726-4ab0-4047-9b91-e0817a139e90,format=raw,if=none,id=drive-virtio- disk0,serial=603271e4-3089-4779-aa1c-faf280fabd17,cache=none,werror=stop,rerror=stop,aio=threads: Could not open '/rhev/data-center/8b6303b3-79c6-4633-ae21-71b15ed00675/67b4d9aa-f174-436a-b5a1-ec7cee5f2edb/images/603271e4-3089-4779-aa1c-faf280fabd17/7c7f7726-4ab0-4047-9b91-e0817a139e90': Permission denied (libvirt/csapi1.test.j2noc.com.log says same thing) [root@ovirt1 test log]# ls -hl /rhev/data-center/8b6303b3-79c6-4633-ae21-71b15ed00675/67b4d9aa-f174-436a-b5a1-ec7cee5f2edb/images/603271e4-3089-4779-aa1c-faf280fabd17/7c7f7726-4ab0-4047-9b91-e0817a139e90 -rw-rw---- 1 vdsm kvm 20G Jan 16 15:32 /rhev/data-center/8b6303b3-79c6-4633-ae21-71b15ed00675/67b4d9aa-f174-436a-b5a1-ec7cee5f2edb/images/603271e4-3089-4779-aa1c-faf280fabd17/7c7f7726-4ab0-4047-9b91-e0817a139e90 lrwxrwxrwx 1 vdsm kvm 98 Jan 10 21:21 67b4d9aa-f174-436a-b5a1-ec7cee5f2edb -> /rhev/data-center/mnt/glusterSD/ovirt3-ks.test.j2noc.com:_gv3/67b4d9aa-f174-436a-b5a1-ec7cee5f2edb engine.log: 2018-01-16 14:36:18,846-08 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (o rg.ovirt.thread.pool-6-thread-34) [78d8b38c] EVENT_ID: VM_MIGRATION_TRYING_RERUN(128), Correlation ID: e39 9da53-cd31-401c-bf63-16b118b8884c, Job ID: 4db6bd87-0da4-4358-acae-dc453c813969, Call Stack: null, Custom ID: null, Custom Event ID: -1, Message: Failed to migrate VM csapi1.test.j2noc.com to Host ovirt1.test.j2n oc.com . Trying to migrate to another Host. ovirt-engine-tools-4.1.8.2-1.el7.centos.noarch glusterfs-3.8.15-2.el7.x86_64 vdsm-4.19.43-1.el7.centos.x86_64 libvirt-daemon-3.2.0-14.el7_4.7.x86_64 Let me know what logs would be useful in troubleshooting this. Thanks!

2 2

4.1.8: Run Once does not allow me to choose a CD Image to attach
by Derek Atkins 18 Jan '18

18 Jan '18

Hi, I upgraded from 4.0.6 to 4.1.8. I had created a VM in 4.0.6 but had never started it nor had I ever run it before I upgraded. I then upgraded to 4.1.8 (which went very smoothly), but then I tried to use the Run Once function to boot up and attach an ISO Image. However when I select Run Once, then Boot Options, the "Attach CD" option is unchecked and wont let me select it like I could in 4.0.6. If I try to select the checkbox I see the popup flash but the box remains unchecked. Is this a bug/regression in 4.1.8? I did clear my browser cache thinking that might be it, but no, that didn't help. The only thing I could do was Edit the VM, go to Boot options, select it there (which I could do) and then choose the ISO. THEN I could run it with the CD attached. Thanks, -derek -- Derek Atkins 617-623-3745 derek(a)ihtfp.com www.ihtfp.com Computer and Internet Security Consultant

3 4

4.2.0 Unable to create VM disk from VM Portal
by Vrgotic, Marko 18 Jan '18

18 Jan '18

--_000_B536AA2E93C94EFF948B2FB02AFDD203activevideocom_ Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 RGVhciBvVmlydCBjb21tdW5pdHksDQoNCkkgaGF2ZSBmb2xsb3dpbmcgc2V0dXA6DQo0IHNlcnZl cnMgYXMgZ2x1c3RlcmZzIHN0b3JhZ2U6IHByb3ZpZGluZyB2b2x1bWUgMSBmb3IgSG9zdGVkRW5n aW5lIGFkIHZvbHVtZSAyIGZvciBWTSBTdG9yYWdlDQo3IEh5cGVydmlzb3JzDQpTZWxmSG9zdGVk IEVuZ2luZSA0LjIuMCBjbGVhbiBpbnN0YWxsIChub3QgYW4gdXBncmFkZSkNCg0KUHJvYmxlbToN Ck1pc3Npbmcgb3B0aW9uIHRvIGFkZC9jcmVhdGUgVk0gZGlzayBmcm9tIFZNIFBvcnRhbCAvIGV2 ZW4gYXMgQWRtaW4gdXNlci4NCkkgaGF2ZSB0cmllZCB3aXRoIEFkbWluIHN1ZXIgYW5kIGNyZWF0 aW5nIHVzZXIg4oCcVGVzdOKAnSB3dGggUG93ZXJVc2VyIHJvbGUuDQpJbiBib3RoIGNhc2VzIHRo ZXJlIGlzIG5vdCBvcHRpb24gdG8gY3JlYXRlIG9yIGF0dGFjaCBkaXNrLg0KDQpJcyB0aGUgYXBw cm9hY2ggY2hhbmdlZCBzaW5jZSA0LjIuMD8gQW0gSSBtaXNzaW5nIGEgc2V0dGluZz8NClBsZWFz ZSBsZXQgbWUga25vdywgSSB3aWxsIHByb3ZpZGUgbW9yZSBkYXRhIGlmIG5lZWRlZC4NCg0KS2lu ZGx5IGF3YWl0aW5nIHlvdXIgcmVwbHkuDQoNCi0tDQpNZXQgdnJpZW5kZWxpamtlIGdyb2V0IC8g QmVzdCByZWdhcmRzLA0KTWFya28gVnJnb3RpYw0KU3lzdGVtIEVuZ2luZWVyL0N1c3RvbWVyIENh cmUNCg0K --_000_B536AA2E93C94EFF948B2FB02AFDD203activevideocom_ Content-Type: text/html; charset="utf-8" Content-ID: <4BE36FEDD7A7564FA55EC61B84810D36(a)namprd08.prod.outlook.com> Content-Transfer-Encoding: base64 PGh0bWwgeG1sbnM6bz0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6b2ZmaWNlIiB4 bWxuczp3PSJ1cm46c2NoZW1hcy1taWNyb3NvZnQtY29tOm9mZmljZTp3b3JkIiB4bWxuczptPSJo dHRwOi8vc2NoZW1hcy5taWNyb3NvZnQuY29tL29mZmljZS8yMDA0LzEyL29tbWwiIHhtbG5zPSJo dHRwOi8vd3d3LnczLm9yZy9UUi9SRUMtaHRtbDQwIj4NCjxoZWFkPg0KPG1ldGEgaHR0cC1lcXVp dj0iQ29udGVudC1UeXBlIiBjb250ZW50PSJ0ZXh0L2h0bWw7IGNoYXJzZXQ9dXRmLTgiPg0KPG1l dGEgbmFtZT0iVGl0bGUiIGNvbnRlbnQ9IiI+DQo8bWV0YSBuYW1lPSJLZXl3b3JkcyIgY29udGVu dD0iIj4NCjxtZXRhIG5hbWU9IkdlbmVyYXRvciIgY29udGVudD0iTWljcm9zb2Z0IFdvcmQgMTUg KGZpbHRlcmVkIG1lZGl1bSkiPg0KPHN0eWxlPjwhLS0NCi8qIEZvbnQgRGVmaW5pdGlvbnMgKi8N CkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6IkNhbWJyaWEgTWF0aCI7DQoJcGFub3NlLTE6MiA0 IDUgMyA1IDQgNiAzIDIgNDt9DQpAZm9udC1mYWNlDQoJe2ZvbnQtZmFtaWx5OkNhbGlicmk7DQoJ cGFub3NlLTE6MiAxNSA1IDIgMiAyIDQgMyAyIDQ7fQ0KQGZvbnQtZmFjZQ0KCXtmb250LWZhbWls eTotd2Via2l0LXN0YW5kYXJkO30NCi8qIFN0eWxlIERlZmluaXRpb25zICovDQpwLk1zb05vcm1h bCwgbGkuTXNvTm9ybWFsLCBkaXYuTXNvTm9ybWFsDQoJe21hcmdpbjowY207DQoJbWFyZ2luLWJv dHRvbTouMDAwMXB0Ow0KCWZvbnQtc2l6ZToxMi4wcHQ7DQoJZm9udC1mYW1pbHk6Q2FsaWJyaTsN Cgltc28tZmFyZWFzdC1sYW5ndWFnZTpFTi1VUzt9DQphOmxpbmssIHNwYW4uTXNvSHlwZXJsaW5r DQoJe21zby1zdHlsZS1wcmlvcml0eTo5OTsNCgljb2xvcjojMDU2M0MxOw0KCXRleHQtZGVjb3Jh dGlvbjp1bmRlcmxpbmU7fQ0KYTp2aXNpdGVkLCBzcGFuLk1zb0h5cGVybGlua0ZvbGxvd2VkDQoJ e21zby1zdHlsZS1wcmlvcml0eTo5OTsNCgljb2xvcjojOTU0RjcyOw0KCXRleHQtZGVjb3JhdGlv bjp1bmRlcmxpbmU7fQ0Kc3Bhbi5FbWFpbFN0eWxlMTcNCgl7bXNvLXN0eWxlLXR5cGU6cGVyc29u YWwtY29tcG9zZTsNCglmb250LWZhbWlseTpDYWxpYnJpOw0KCWNvbG9yOndpbmRvd3RleHQ7fQ0K c3Bhbi5tc29JbnMNCgl7bXNvLXN0eWxlLXR5cGU6ZXhwb3J0LW9ubHk7DQoJbXNvLXN0eWxlLW5h bWU6IiI7DQoJdGV4dC1kZWNvcmF0aW9uOnVuZGVybGluZTsNCgljb2xvcjp0ZWFsO30NCi5Nc29D aHBEZWZhdWx0DQoJe21zby1zdHlsZS10eXBlOmV4cG9ydC1vbmx5Ow0KCWZvbnQtZmFtaWx5OkNh bGlicmk7DQoJbXNvLWZhcmVhc3QtbGFuZ3VhZ2U6RU4tVVM7fQ0KQHBhZ2UgV29yZFNlY3Rpb24x DQoJe3NpemU6NTk1LjBwdCA4NDIuMHB0Ow0KCW1hcmdpbjo3Mi4wcHQgNzIuMHB0IDcyLjBwdCA3 Mi4wcHQ7fQ0KZGl2LldvcmRTZWN0aW9uMQ0KCXtwYWdlOldvcmRTZWN0aW9uMTt9DQotLT48L3N0 eWxlPg0KPC9oZWFkPg0KPGJvZHkgYmdjb2xvcj0id2hpdGUiIGxhbmc9IkVOLUdCIiBsaW5rPSIj MDU2M0MxIiB2bGluaz0iIzk1NEY3MiI+DQo8ZGl2IGNsYXNzPSJXb3JkU2VjdGlvbjEiPg0KPHAg Y2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQiPkRlYXIgb1Zp cnQgY29tbXVuaXR5LDxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwi PjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0Ij48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48 L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdCI+ SSBoYXZlIGZvbGxvd2luZyBzZXR1cDo8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0i TXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdCI+NCBzZXJ2ZXJzIGFzIGds dXN0ZXJmcyBzdG9yYWdlOiBwcm92aWRpbmcgdm9sdW1lIDEgZm9yIEhvc3RlZEVuZ2luZSBhZCB2 b2x1bWUgMiBmb3IgVk0gU3RvcmFnZTxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJN c29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0Ij43IEh5cGVydmlzb3JzIDxv OnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJm b250LXNpemU6MTEuMHB0Ij5TZWxmSG9zdGVkIEVuZ2luZSA0LjIuMCBjbGVhbiBpbnN0YWxsIChu b3QgYW4gdXBncmFkZSk8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFs Ij48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdCI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+ PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQi PlByb2JsZW06PG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNw YW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQiPk1pc3Npbmcgb3B0aW9uIHRvIGFkZC9jcmVhdGUg Vk0gZGlzayBmcm9tIFZNIFBvcnRhbCAvIGV2ZW4gYXMgQWRtaW4gdXNlci48bzpwPjwvbzpwPjwv c3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEx LjBwdCI+SSBoYXZlIHRyaWVkIHdpdGggQWRtaW4gc3VlciBhbmQgY3JlYXRpbmcgdXNlciDigJxU ZXN04oCdIHd0aCBQb3dlclVzZXIgcm9sZS48bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFz cz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdCI+SW4gYm90aCBjYXNl cyB0aGVyZSBpcyBub3Qgb3B0aW9uIHRvIGNyZWF0ZSBvciBhdHRhY2ggZGlzay48bzpwPjwvbzpw Pjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXpl OjExLjBwdCI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1h bCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQiPklzIHRoZSBhcHByb2FjaCBjaGFuZ2Vk IHNpbmNlIDQuMi4wPyBBbSBJIG1pc3NpbmcgYSBzZXR0aW5nPzxvOnA+PC9vOnA+PC9zcGFuPjwv cD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0Ij5Q bGVhc2UgbGV0IG1lIGtub3csIEkgd2lsbCBwcm92aWRlIG1vcmUgZGF0YSBpZiBuZWVkZWQuPG86 cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZv bnQtc2l6ZToxMS4wcHQiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJN c29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0Ij5LaW5kbHkgYXdhaXRpbmcg eW91ciByZXBseS48bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48 c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdCI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9w Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gbGFuZz0iTkwiIHN0eWxlPSJmb250LXNpemU6 MTAuNXB0O2ZvbnQtZmFtaWx5Oi13ZWJraXQtc3RhbmRhcmQ7Y29sb3I6YmxhY2s7bXNvLWZhcmVh c3QtbGFuZ3VhZ2U6RU4tR0IiPi0tPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1z b05vcm1hbCI+PHNwYW4gbGFuZz0iTkwiIHN0eWxlPSJmb250LXNpemU6MTAuNXB0O2ZvbnQtZmFt aWx5Oi13ZWJraXQtc3RhbmRhcmQ7Y29sb3I6YmxhY2s7bXNvLWZhcmVhc3QtbGFuZ3VhZ2U6RU4t R0IiPk1ldCB2cmllbmRlbGlqa2UgZ3JvZXQgLyBCZXN0IHJlZ2FyZHMsPG86cD48L286cD48L3Nw YW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJm b250LXNpemU6MTAuNXB0O2ZvbnQtZmFtaWx5Oi13ZWJraXQtc3RhbmRhcmQ7Y29sb3I6YmxhY2s7 bXNvLWZhcmVhc3QtbGFuZ3VhZ2U6RU4tR0IiPk1hcmtvIFZyZ290aWM8bzpwPjwvbzpwPjwvc3Bh bj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImZv bnQtc2l6ZToxMC41cHQ7Zm9udC1mYW1pbHk6LXdlYmtpdC1zdGFuZGFyZDtjb2xvcjpibGFjaztt c28tZmFyZWFzdC1sYW5ndWFnZTpFTi1HQiI+U3lzdGVtIEVuZ2luZWVyL0N1c3RvbWVyIENhcmU8 bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwv bzpwPjwvcD4NCjwvZGl2Pg0KPC9ib2R5Pg0KPC9odG1sPg0K --_000_B536AA2E93C94EFF948B2FB02AFDD203activevideocom_--

3 5

All in one adds host failure
by Pym 18 Jan '18

18 Jan '18

------=_Part_123075_1879919599.1516186973359 Content-Type: text/plain; charset=GBK Content-Transfer-Encoding: base64 SGk6CgoKSSdtIGRvaW5nIG92aXJ0IGFsbCBpbiBvbmUuIEkgZmlyc3QgdXNlZCB0aGUgaXNvIG9m IG5vZGUgdG8gZG8gdGhlIGJhc2ljIGVudmlyb25tZW50LCBhbmQgdGhlbiBJIGRpZCB0aGUgc291 cmNlIGNvbXBpbGF0aW9uIGFuZCBpbnN0YWxsYXRpb24gb3ZpcnQgb24gdGhlIG5vZGUgZW52aXJv bm1lbnQuIFdoZW4gSSBhZGQgYSBOZXcgSG9zdCB0byB0aGUgd2ViIGludGVyZmFjZSBIb3N0LSA+ TmV3LCBpbiB0aGUgbG9nOiBBbnNpYmxlIHBsYXlib29rIGNvbW1hbmQgaGFzIGV4aXRlZCB3aXRo IHZhbHVlOiAxLCB0aGVuIEhvc3QgaW5zdGFsbGF0aW9uIGZhaWxlZC4KCgpUaGUgbG9ncyBhcmUg YXMgZm9sbG93czoKCgplbmdpbmUubG9nOgoiMjAxOC0wMS0wNSAxNDo0OTozMiw0NDUrMDggSU5G TyAgW29yZy5vdmlydC5lbmdpbmUuY29yZS5kYWwuZGJicm9rZXIuYXVkaXRsb2doYW5kbGluZy5B dWRpdExvZ0RpcmVjdG9yXSAoVmRzRGVwbG95KSBbNzhjNmE5Y2ItZWNjNS00ZTQ0LTkzNjYtMTM5 NjBlZjA0NTU5XSBFVkVOVF9JRDogVkRTX0lOU1RBTExfSU5fUFJPR1JFU1MoNTA5KSwgSW5zdGFs bGluZyBIb3N0IHRjaHlwLXRlc3QuZWNyLmNvbS4gU3RhZ2U6IFRlcm1pbmF0aW9uLgoyMDE4LTAx LTA1IDE0OjQ5OjMyLDUwNiswOCBJTkZPICBbb3JnLm92aXJ0LmVuZ2luZS5jb3JlLmRhbC5kYmJy b2tlci5hdWRpdGxvZ2hhbmRsaW5nLkF1ZGl0TG9nRGlyZWN0b3JdIChFRS1NYW5hZ2VkVGhyZWFk RmFjdG9yeS1lbmdpbmUtVGhyZWFkLTEyKSBbNzhjNmE5Y2ItZWNjNS00ZTQ0LTkzNjYtMTM5NjBl ZjA0NTU5XSBFVkVOVF9JRDogVkRTX0FOU0lCTEVfSU5TVEFMTF9TVEFSVEVEKDU2MCksIEFuc2li bGUgaG9zdC1kZXBsb3kgcGxheWJvb2sgZXhlY3V0aW9uIGhhcyBzdGFydGVkIG9uIGhvc3QgdGNo eXAtdGVzdC5lY3IuY29tLgoyMDE4LTAxLTA1IDE0OjQ5OjMyLDUwNyswOCBJTkZPICBbb3JnLm92 aXJ0LmVuZ2luZS5jb3JlLmNvbW1vbi51dGlscy5hbnNpYmxlLkFuc2libGVFeGVjdXRvcl0gKEVF LU1hbmFnZWRUaHJlYWRGYWN0b3J5LWVuZ2luZS1UaHJlYWQtMTIpIFs3OGM2YTljYi1lY2M1LTRl NDQtOTM2Ni0xMzk2MGVmMDQ1NTldIEV4ZWN1dGluZyBBbnNpYmxlIGNvbW1hbmQ6IC91c3IvYmlu L2Fuc2libGUtcGxheWJvb2sgLS1wcml2YXRlLWtleT0vaG9tZS9wYW4vb3ZpcnQtZW5naW5lL2V0 Yy9wa2kvb3ZpcnQtZW5naW5lL2tleXMvZW5naW5lX2lkX3JzYSAtLWludmVudG9yeT0vdG1wL2Fu c2libGUtaW52ZW50b3J5NDc3Nzg5MTU4Mjg1NTI2Nzg1NSAtLWV4dHJhLXZhcnM9aG9zdF9kZXBs b3lfY2x1c3Rlcl92ZXJzaW9uPTQuMiAtLWV4dHJhLXZhcnM9aG9zdF9kZXBsb3lfZ2x1c3Rlcl9l bmFibGVkPWZhbHNlIC0tZXh0cmEtdmFycz1ob3N0X2RlcGxveV92aXJ0X2VuYWJsZWQ9dHJ1ZSAt LWV4dHJhLXZhcnM9aG9zdF9kZXBsb3lfdmRzbV9wb3J0PTU0MzIxIC0tZXh0cmEtdmFycz1ob3N0 X2RlcGxveV9vdmVycmlkZV9maXJld2FsbD10cnVlIC0tZXh0cmEtdmFycz1ob3N0X2RlcGxveV9m aXJld2FsbF90eXBlPUZJUkVXQUxMRCAtLWV4dHJhLXZhcnM9YW5zaWJsZV9wb3J0PTIyIC0tZXh0 cmEtdmFycz1ob3N0X2RlcGxveV9wb3N0X3Rhc2tzPS9ob21lL3Bhbi9vdmlydC1lbmdpbmUvZXRj L292aXJ0LWVuZ2luZS9hbnNpYmxlL292aXJ0LWhvc3QtZGVwbG95LXBvc3QtdGFza3MueW1sIC0t ZXh0cmEtdmFycz1ob3N0X2RlcGxveV9vdm5fdHVubmVsaW5nX2ludGVyZmFjZT0xMjcuMC4wLjEg LS1leHRyYS12YXJzPWhvc3RfZGVwbG95X292bl9jZW50cmFsPW51bGwgL2hvbWUvcGFuL292aXJ0 LWVuZ2luZS9zaGFyZS9vdmlydC1lbmdpbmUvcGxheWJvb2tzL292aXJ0LWhvc3QtZGVwbG95Lnlt bAoyMDE4LTAxLTA1IDE0OjQ5OjMyLDc4OCswOCBJTkZPICBbb3JnLm92aXJ0LmVuZ2luZS5jb3Jl LmNvbW1vbi51dGlscy5hbnNpYmxlLkFuc2libGVFeGVjdXRvcl0gKEVFLU1hbmFnZWRUaHJlYWRG YWN0b3J5LWVuZ2luZS1UaHJlYWQtMTIpIFs3OGM2YTljYi1lY2M1LTRlNDQtOTM2Ni0xMzk2MGVm MDQ1NTldIEFuc2libGUgcGxheWJvb2sgY29tbWFuZCBoYXMgZXhpdGVkIHdpdGggdmFsdWU6IDEK MjAxOC0wMS0wNSAxNDo0OTozMiw3OTIrMDggRVJST1IgW29yZy5vdmlydC5lbmdpbmUuY29yZS5i bGwuaG9zdGRlcGxveS5JbnN0YWxsVmRzSW50ZXJuYWxDb21tYW5kXSAoRUUtTWFuYWdlZFRocmVh ZEZhY3RvcnktZW5naW5lLVRocmVhZC0xMikgWzc4YzZhOWNiLWVjYzUtNGU0NC05MzY2LTEzOTYw ZWYwNDU1OV0gSG9zdCBpbnN0YWxsYXRpb24gZmFpbGVkIGZvciBob3N0ICdkYmVmYTU4Yi0zZWUy LTQzOWItOGU5Zi03ZWNlNTcxYWJlMzInLCAndGNoeXAtdGVzdC5lY3IuY29tJzogMjUwCjIwMTgt MDEtMDUgMTQ6NDk6MzIsNzk2KzA4IElORk8gIFtvcmcub3ZpcnQuZW5naW5lLmNvcmUudmRzYnJv a2VyLlNldFZkc1N0YXR1c1ZEU0NvbW1hbmRdIChFRS1NYW5hZ2VkVGhyZWFkRmFjdG9yeS1lbmdp bmUtVGhyZWFkLTEyKSBbNzhjNmE5Y2ItZWNjNS00ZTQ0LTkzNjYtMTM5NjBlZjA0NTU5XSBTVEFS VCwgU2V0VmRzU3RhdHVzVkRTQ29tbWFuZChIb3N0TmFtZSA9IHRjaHlwLXRlc3QuZWNyLmNvbSwg U2V0VmRzU3RhdHVzVkRTQ29tbWFuZFBhcmFtZXRlcnM6e2hvc3RJZD0nZGJlZmE1OGItM2VlMi00 MzliLThlOWYtN2VjZTU3MWFiZTMyJywgc3RhdHVzPSdJbnN0YWxsRmFpbGVkJywgbm9uT3BlcmF0 aW9uYWxSZWFzb249J05PTkUnLCBzdG9wU3BtRmFpbHVyZUxvZ2dlZD0nZmFsc2UnLCBtYWludGVu YW5jZVJlYXNvbj0nbnVsbCd9KSwgbG9nIGlkOiBhZjBiOGQxCjIwMTgtMDEtMDUgMTQ6NDk6MzIs ODAzKzA4IElORk8gIFtvcmcub3ZpcnQuZW5naW5lLmNvcmUudmRzYnJva2VyLlNldFZkc1N0YXR1 c1ZEU0NvbW1hbmRdIChFRS1NYW5hZ2VkVGhyZWFkRmFjdG9yeS1lbmdpbmUtVGhyZWFkLTEyKSBb NzhjNmE5Y2ItZWNjNS00ZTQ0LTkzNjYtMTM5NjBlZjA0NTU5XSBGSU5JU0gsIFNldFZkc1N0YXR1 c1ZEU0NvbW1hbmQsIGxvZyBpZDogYWYwYjhkMQoyMDE4LTAxLTA1IDE0OjQ5OjMyLDgwOSswOCBF UlJPUiBbb3JnLm92aXJ0LmVuZ2luZS5jb3JlLmRhbC5kYmJyb2tlci5hdWRpdGxvZ2hhbmRsaW5n LkF1ZGl0TG9nRGlyZWN0b3JdIChFRS1NYW5hZ2VkVGhyZWFkRmFjdG9yeS1lbmdpbmUtVGhyZWFk LTEyKSBbNzhjNmE5Y2ItZWNjNS00ZTQ0LTkzNjYtMTM5NjBlZjA0NTU5XSBFVkVOVF9JRDogVkRT X0lOU1RBTExfRkFJTEVEKDUwNSksIEhvc3QgdGNoeXAtdGVzdC5lY3IuY29tIGluc3RhbGxhdGlv biBmYWlsZWQuIDI1MC4iCgoKb3ZpcnQtaG9zdC1kZXBsb3ktYW5zaWJsZS0yMDE4MDEwNTE0NDkz Mi0xMjcuMC4wLjEtNzhjNmE5Y2ItZWNjNS00ZTQ0LTkzNjYtMTM5NjBlZjA0NTU5LmxvZzoKIkVS Uk9SISBVbmV4cGVjdGVkIEV4Y2VwdGlvbjogW0Vycm5vIDEzXSBQZXJtaXNzaW9uIGRlbmllZAp0 byBzZWUgdGhlIGZ1bGwgdHJhY2ViYWNrLCB1c2UgLXZ2diIKCgpXaGF0IGFtIEkgZ29pbmcgdG8g ZG8gbm93IHRvIGFkZCBob3N0IHN1Y2Nlc3M/CgoKVGhhbmtzLgoK ------=_Part_123075_1879919599.1516186973359 Content-Type: text/html; charset=GBK Content-Transfer-Encoding: base64 PGRpdiBzdHlsZT0ibGluZS1oZWlnaHQ6MS43O2NvbG9yOiMwMDAwMDA7Zm9udC1zaXplOjE0cHg7 Zm9udC1mYW1pbHk6QXJpYWwiPjxkaXY+PGRpdj5IaTo8L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2 PkknbSBkb2luZyBvdmlydCBhbGwgaW4gb25lLiBJIGZpcnN0IHVzZWQgdGhlIGlzbyBvZiBub2Rl IHRvIGRvIHRoZSBiYXNpYyBlbnZpcm9ubWVudCwgYW5kIHRoZW4gSSBkaWQgdGhlIHNvdXJjZSBj b21waWxhdGlvbiBhbmQgaW5zdGFsbGF0aW9uIG92aXJ0IG9uIHRoZSBub2RlIGVudmlyb25tZW50 LiBXaGVuIEkgYWRkIGEgTmV3IEhvc3QgdG8gdGhlIHdlYiBpbnRlcmZhY2UgSG9zdC0gJmd0O05l dywgaW4gdGhlIGxvZzogQW5zaWJsZSBwbGF5Ym9vayBjb21tYW5kIGhhcyBleGl0ZWQgd2l0aCB2 YWx1ZTogMSwgdGhlbiBIb3N0IGluc3RhbGxhdGlvbiBmYWlsZWQuPC9kaXY+PGRpdj48YnI+PC9k aXY+PGRpdj5UaGUgbG9ncyBhcmUgYXMgZm9sbG93czo8L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2 PmVuZ2luZS5sb2c6PC9kaXY+PC9kaXY+PGRpdj48ZGl2PiIyMDE4LTAxLTA1IDE0OjQ5OjMyLDQ0 NSswOCBJTkZPJm5ic3A7IFtvcmcub3ZpcnQuZW5naW5lLmNvcmUuZGFsLmRiYnJva2VyLmF1ZGl0 bG9naGFuZGxpbmcuQXVkaXRMb2dEaXJlY3Rvcl0gKFZkc0RlcGxveSkgWzc4YzZhOWNiLWVjYzUt NGU0NC05MzY2LTEzOTYwZWYwNDU1OV0gRVZFTlRfSUQ6IFZEU19JTlNUQUxMX0lOX1BST0dSRVNT KDUwOSksIEluc3RhbGxpbmcgSG9zdCB0Y2h5cC10ZXN0LmVjci5jb20uIFN0YWdlOiBUZXJtaW5h dGlvbi48L2Rpdj48ZGl2PjIwMTgtMDEtMDUgMTQ6NDk6MzIsNTA2KzA4IElORk8mbmJzcDsgW29y Zy5vdmlydC5lbmdpbmUuY29yZS5kYWwuZGJicm9rZXIuYXVkaXRsb2doYW5kbGluZy5BdWRpdExv Z0RpcmVjdG9yXSAoRUUtTWFuYWdlZFRocmVhZEZhY3RvcnktZW5naW5lLVRocmVhZC0xMikgWzc4 YzZhOWNiLWVjYzUtNGU0NC05MzY2LTEzOTYwZWYwNDU1OV0gRVZFTlRfSUQ6IFZEU19BTlNJQkxF X0lOU1RBTExfU1RBUlRFRCg1NjApLCBBbnNpYmxlIGhvc3QtZGVwbG95IHBsYXlib29rIGV4ZWN1 dGlvbiBoYXMgc3RhcnRlZCBvbiBob3N0IHRjaHlwLXRlc3QuZWNyLmNvbS48L2Rpdj48ZGl2PjIw MTgtMDEtMDUgMTQ6NDk6MzIsNTA3KzA4IElORk8mbmJzcDsgW29yZy5vdmlydC5lbmdpbmUuY29y ZS5jb21tb24udXRpbHMuYW5zaWJsZS5BbnNpYmxlRXhlY3V0b3JdIChFRS1NYW5hZ2VkVGhyZWFk RmFjdG9yeS1lbmdpbmUtVGhyZWFkLTEyKSBbNzhjNmE5Y2ItZWNjNS00ZTQ0LTkzNjYtMTM5NjBl ZjA0NTU5XSBFeGVjdXRpbmcgQW5zaWJsZSBjb21tYW5kOiAvdXNyL2Jpbi9hbnNpYmxlLXBsYXli b29rIC0tcHJpdmF0ZS1rZXk9L2hvbWUvcGFuL292aXJ0LWVuZ2luZS9ldGMvcGtpL292aXJ0LWVu Z2luZS9rZXlzL2VuZ2luZV9pZF9yc2EgLS1pbnZlbnRvcnk9L3RtcC9hbnNpYmxlLWludmVudG9y eTQ3Nzc4OTE1ODI4NTUyNjc4NTUgLS1leHRyYS12YXJzPWhvc3RfZGVwbG95X2NsdXN0ZXJfdmVy c2lvbj00LjIgLS1leHRyYS12YXJzPWhvc3RfZGVwbG95X2dsdXN0ZXJfZW5hYmxlZD1mYWxzZSAt LWV4dHJhLXZhcnM9aG9zdF9kZXBsb3lfdmlydF9lbmFibGVkPXRydWUgLS1leHRyYS12YXJzPWhv c3RfZGVwbG95X3Zkc21fcG9ydD01NDMyMSAtLWV4dHJhLXZhcnM9aG9zdF9kZXBsb3lfb3ZlcnJp ZGVfZmlyZXdhbGw9dHJ1ZSAtLWV4dHJhLXZhcnM9aG9zdF9kZXBsb3lfZmlyZXdhbGxfdHlwZT1G SVJFV0FMTEQgLS1leHRyYS12YXJzPWFuc2libGVfcG9ydD0yMiAtLWV4dHJhLXZhcnM9aG9zdF9k ZXBsb3lfcG9zdF90YXNrcz0vaG9tZS9wYW4vb3ZpcnQtZW5naW5lL2V0Yy9vdmlydC1lbmdpbmUv YW5zaWJsZS9vdmlydC1ob3N0LWRlcGxveS1wb3N0LXRhc2tzLnltbCAtLWV4dHJhLXZhcnM9aG9z dF9kZXBsb3lfb3ZuX3R1bm5lbGluZ19pbnRlcmZhY2U9MTI3LjAuMC4xIC0tZXh0cmEtdmFycz1o b3N0X2RlcGxveV9vdm5fY2VudHJhbD1udWxsIC9ob21lL3Bhbi9vdmlydC1lbmdpbmUvc2hhcmUv b3ZpcnQtZW5naW5lL3BsYXlib29rcy9vdmlydC1ob3N0LWRlcGxveS55bWw8L2Rpdj48ZGl2PjIw MTgtMDEtMDUgMTQ6NDk6MzIsNzg4KzA4IElORk8mbmJzcDsgW29yZy5vdmlydC5lbmdpbmUuY29y ZS5jb21tb24udXRpbHMuYW5zaWJsZS5BbnNpYmxlRXhlY3V0b3JdIChFRS1NYW5hZ2VkVGhyZWFk RmFjdG9yeS1lbmdpbmUtVGhyZWFkLTEyKSBbNzhjNmE5Y2ItZWNjNS00ZTQ0LTkzNjYtMTM5NjBl ZjA0NTU5XSBBbnNpYmxlIHBsYXlib29rIGNvbW1hbmQgaGFzIGV4aXRlZCB3aXRoIHZhbHVlOiAx PC9kaXY+PGRpdj4yMDE4LTAxLTA1IDE0OjQ5OjMyLDc5MiswOCBFUlJPUiBbb3JnLm92aXJ0LmVu Z2luZS5jb3JlLmJsbC5ob3N0ZGVwbG95Lkluc3RhbGxWZHNJbnRlcm5hbENvbW1hbmRdIChFRS1N YW5hZ2VkVGhyZWFkRmFjdG9yeS1lbmdpbmUtVGhyZWFkLTEyKSBbNzhjNmE5Y2ItZWNjNS00ZTQ0 LTkzNjYtMTM5NjBlZjA0NTU5XSBIb3N0IGluc3RhbGxhdGlvbiBmYWlsZWQgZm9yIGhvc3QgJ2Ri ZWZhNThiLTNlZTItNDM5Yi04ZTlmLTdlY2U1NzFhYmUzMicsICd0Y2h5cC10ZXN0LmVjci5jb20n OiAyNTA8L2Rpdj48ZGl2PjIwMTgtMDEtMDUgMTQ6NDk6MzIsNzk2KzA4IElORk8mbmJzcDsgW29y Zy5vdmlydC5lbmdpbmUuY29yZS52ZHNicm9rZXIuU2V0VmRzU3RhdHVzVkRTQ29tbWFuZF0gKEVF LU1hbmFnZWRUaHJlYWRGYWN0b3J5LWVuZ2luZS1UaHJlYWQtMTIpIFs3OGM2YTljYi1lY2M1LTRl NDQtOTM2Ni0xMzk2MGVmMDQ1NTldIFNUQVJULCBTZXRWZHNTdGF0dXNWRFNDb21tYW5kKEhvc3RO YW1lID0gdGNoeXAtdGVzdC5lY3IuY29tLCBTZXRWZHNTdGF0dXNWRFNDb21tYW5kUGFyYW1ldGVy czp7aG9zdElkPSdkYmVmYTU4Yi0zZWUyLTQzOWItOGU5Zi03ZWNlNTcxYWJlMzInLCBzdGF0dXM9 J0luc3RhbGxGYWlsZWQnLCBub25PcGVyYXRpb25hbFJlYXNvbj0nTk9ORScsIHN0b3BTcG1GYWls dXJlTG9nZ2VkPSdmYWxzZScsIG1haW50ZW5hbmNlUmVhc29uPSdudWxsJ30pLCBsb2cgaWQ6IGFm MGI4ZDE8L2Rpdj48ZGl2PjIwMTgtMDEtMDUgMTQ6NDk6MzIsODAzKzA4IElORk8mbmJzcDsgW29y Zy5vdmlydC5lbmdpbmUuY29yZS52ZHNicm9rZXIuU2V0VmRzU3RhdHVzVkRTQ29tbWFuZF0gKEVF LU1hbmFnZWRUaHJlYWRGYWN0b3J5LWVuZ2luZS1UaHJlYWQtMTIpIFs3OGM2YTljYi1lY2M1LTRl NDQtOTM2Ni0xMzk2MGVmMDQ1NTldIEZJTklTSCwgU2V0VmRzU3RhdHVzVkRTQ29tbWFuZCwgbG9n IGlkOiBhZjBiOGQxPC9kaXY+PC9kaXY+PGRpdj48ZGl2PjIwMTgtMDEtMDUgMTQ6NDk6MzIsODA5 KzA4IEVSUk9SIFtvcmcub3ZpcnQuZW5naW5lLmNvcmUuZGFsLmRiYnJva2VyLmF1ZGl0bG9naGFu ZGxpbmcuQXVkaXRMb2dEaXJlY3Rvcl0gKEVFLU1hbmFnZWRUaHJlYWRGYWN0b3J5LWVuZ2luZS1U aHJlYWQtMTIpIFs3OGM2YTljYi1lY2M1LTRlNDQtOTM2Ni0xMzk2MGVmMDQ1NTldIEVWRU5UX0lE OiBWRFNfSU5TVEFMTF9GQUlMRUQoNTA1KSwgSG9zdCB0Y2h5cC10ZXN0LmVjci5jb20gaW5zdGFs bGF0aW9uIGZhaWxlZC4gMjUwLiI8L2Rpdj48L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2Pm92aXJ0 LWhvc3QtZGVwbG95LWFuc2libGUtMjAxODAxMDUxNDQ5MzItMTI3LjAuMC4xLTc4YzZhOWNiLWVj YzUtNGU0NC05MzY2LTEzOTYwZWYwNDU1OS5sb2c6PC9kaXY+PGRpdj48ZGl2PiJFUlJPUiEgVW5l eHBlY3RlZCBFeGNlcHRpb246IFtFcnJubyAxM10gUGVybWlzc2lvbiBkZW5pZWQ8L2Rpdj48ZGl2 PnRvIHNlZSB0aGUgZnVsbCB0cmFjZWJhY2ssIHVzZSAtdnZ2IjwvZGl2PjwvZGl2PjxkaXY+PGJy PjwvZGl2PjxkaXY+V2hhdCBhbSBJIGdvaW5nIHRvIGRvIG5vdyB0byBhZGQgaG9zdCBzdWNjZXNz PzwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+VGhhbmtzLjwvZGl2PjxkaXY+PGJyPjwvZGl2Pjwv ZGl2Pjxicj48YnI+PHNwYW4gdGl0bGU9Im5ldGVhc2Vmb290ZXIiPjxwPiZuYnNwOzwvcD48L3Nw YW4+ ------=_Part_123075_1879919599.1516186973359--

1 0

Ovirt 4.2 Windows guest tools
by Alex K 18 Jan '18

18 Jan '18

Hi all, Is the ISO at http://plain.resources.ovirt.org/pub/ovirt-4.2/iso/oVirt-toolsSetup/4.2-1.e… the latest guest tools? I'm interested for Windows 2016 and Windows 10. Thanx, Alex

2 2

Ovirt 4.2 Bug with Permissons on the Vm Portal?
by Thomas Fecke 17 Jan '18

17 Jan '18

--_000_29f4cad202e140c1bb149e33fd16af23DR1XEXCH02Besetcorp_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hello Guys, i recently upgrade to 4.2 We used the User Portal before. Every User could see his VM=B4s, every Admi= n just could see his VM=B4s ( User View ). After the Update: Admin=B4s can see every VM in VM Portal The Users can=B4t see Vm=B4s anymore. The Portal is Empty. Role "UserVMMana= ger" is set. We tested some Scenarios: If the User create a new Virtual Machine, it is s= hown in the VM Portal. When he log out and in again -> VM is gone in his Vi= ew Admin View: The VM was created, Permission where set. Everything seems to b= e good. Any Ideas? Thanks Guys --_000_29f4cad202e140c1bb149e33fd16af23DR1XEXCH02Besetcorp_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable <html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr= osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" = xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:= //www.w3.org/TR/REC-html40"> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Diso-8859-= 1"> <meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered medium)"> <style></style> </head> <body lang=3D"EN-US" link=3D"#0563C1" vlink=3D"#954F72"> <div class=3D"WordSection1"> <p class=3D"MsoNormal"><span lang=3D"DE">Hello Guys,<o:p></o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"DE"><o:p> </o:p></span></p> <p class=3D"MsoNormal"><span lang=3D"DE">i recently upgrade to 4.2<o:p></o:= p></span></p> <p class=3D"MsoNormal"><span lang=3D"DE"><o:p> </o:p></span></p> <p class=3D"MsoNormal">We used the User Portal before. Every User could see= his<u> </u> VM=B4s, every Admin just could see his VM=B4s ( User View ).<o:p></o:p></p> <p class=3D"MsoNormal"><o:p> </o:p></p> <p class=3D"MsoNormal">After the Update:<o:p></o:p></p> <p class=3D"MsoNormal"><o:p> </o:p></p> <p class=3D"MsoNormal">Admin=B4s can see every VM in VM Portal<o:p></o:p></= p> <p class=3D"MsoNormal"><o:p> </o:p></p> <p class=3D"MsoNormal">The Users can=B4t see Vm=B4s anymore. The Portal is = Empty. Role “UserVMManager” is set.<o:p></o:p></p> <p class=3D"MsoNormal"><o:p> </o:p></p> <p class=3D"MsoNormal">We tested some Scenarios: If the User create a new V= irtual Machine, it is shown in the VM Portal. When he log out and in again = -> VM is gone in his View<o:p></o:p></p> <p class=3D"MsoNormal"><o:p> </o:p></p> <p class=3D"MsoNormal">Admin View: The VM was created, Permission where set= . Everything seems to be good.<o:p></o:p></p> <p class=3D"MsoNormal"><o:p> </o:p></p> <p class=3D"MsoNormal">Any Ideas? Thanks Guys<u><o:p></o:p></u></p> </div> </body> </html> --_000_29f4cad202e140c1bb149e33fd16af23DR1XEXCH02Besetcorp_--

4 5

Is there a way to rename the hosted engine VM name (not hostname)
by Jayme 17 Jan '18

17 Jan '18

Our oVirt environment was originally setup by someone else. The hosted engine VM has a custom name, but it seems to me like some of the hosted-engine tools such as hosted-engine --console for example expect the domain to be "HostedEngine'. I tried renaming it in the admin interface but changes are locked for the hostedengine VM. Is there a way that I can change the domain of the VM back to HostedEngine instead of the custom name?

1 0

[ANN] oVirt 4.1.9 Second Release Candidate is now available
by Lev Veyde 17 Jan '18

17 Jan '18

The oVirt Project is pleased to announce the availability of the oVirt 4.1.9 RC2 release, as of January 17th, 2017 This update is the ninth in a series of stabilization updates to the 4.1 series. This release is available now for: * Red Hat Enterprise Linux 7.4 or later * CentOS Linux (or similar) 7.4 or later This release supports Hypervisor Hosts running: * Red Hat Enterprise Linux 7.4 or later * CentOS Linux (or similar) 7.4 or later * oVirt Node 4.1 See the release notes [1] for installation / upgrade instructions and a list of new features and bugs fixed. Notes: - oVirt Appliance is already available - oVirt Live is already available [2] - oVirt Node will be available soon [2] Additional Resources: * Read more about the oVirt 4.1.9 release highlights: http://www.ovirt.org/release/4.1.9/ * Get more oVirt Project updates on Twitter: https://twitter.com/ovirt * Check out the latest project news on the oVirt blog: http://www.ovirt.org/blog/ [1] http://www.ovirt.org/release/4.1.9/ [2] http://resources.ovirt.org/pub/ovirt-4.1-pre/iso/ -- Lev Veyde Software Engineer, RHCE | RHCVA | MCITP Red Hat Israel <https://www.redhat.com> lev(a)redhat.com | lveyde(a)redhat.com <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

1 0

[solved] Error while upgrading engine from 4.1.8 to 4.2
by Luca 'remix_tj' Lorenzetto 17 Jan '18

17 Jan '18

Hello, i just completed the upgrade of my engine appliance from 4.1.8 to 4.2.0.1. I would like to share the only error i've encountered while upgrading and running engine-setup: [ INFO ] Creating/refreshing Engine database schema [ ERROR ] schema.sh: FATAL: Cannot execute sql command: --file=/usr/share/ovirt-engine/dbscripts/upgrade/04_02_0140_add_max_memory_constraint.sql [ ERROR ] Failed to execute stage 'Misc configuration': Engine schema refresh failed After some search i found out that is an error linked to a bad configuration of one (or more) vm having "Memory Size" value set with an higher value than "Maximum Memory". In my case the vm with that error was HostedEngine. Editing the vm configuration by setting an appropriate value allowed me to run again engine-setup with success. Luca -- "E' assurdo impiegare gli uomini di intelligenza eccellente per fare calcoli che potrebbero essere affidati a chiunque se si usassero delle macchine" Gottfried Wilhelm von Leibnitz, Filosofo e Matematico (1646-1716) "Internet è la più grande biblioteca del mondo. Ma il problema è che i libri sono tutti sparsi sul pavimento" John Allen Paulos, Matematico (1945-vivente) Luca 'remix_tj' Lorenzetto, http://www.remixtj.net , <lorenzetto.luca(a)gmail.com>

3 4

oVirt 4.2: hostdev passthrough noop
by Daniel Helgenberger 17 Jan '18

17 Jan '18

Hello, yesterday I upgraded to 4.2 from 4.1.8. Now I notice I cannot assign host dev pass though any more; in the GUI the 'Pinnded to host' list is empty. The hostdev from before the upgrades are still present. I tried to remove them and got an NPE (see below). Has anyone an idea? ----------------------- > 2018-01-17 11:37:51,035+01 INFO [org.ovirt.engine.core.bll.hostdev.RemoveVmHostDevicesCommand] (default task-48) [57a8c14d-fc2c-4846-bc3d-cc4f3e8393f8] Running command: RemoveVmHostDevicesCommand internal: false. Entities affected : ID: 6132322b-e187-4a83-b8c1-0477bde10497 Type: VMAction group EDIT_ADMIN_VM_PROPERTIES with role type ADMIN > 2018-01-17 11:37:51,037+01 ERROR [org.ovirt.engine.core.bll.hostdev.RemoveVmHostDevicesCommand] (default task-48) [57a8c14d-fc2c-4846-bc3d-cc4f3e8393f8] Command 'org.ovirt.engine.core.bll.hostdev.RemoveVmHostDevicesCommand' failed: null > 2018-01-17 11:37:51,037+01 ERROR [org.ovirt.engine.core.bll.hostdev.RemoveVmHostDevicesCommand] (default task-48) [57a8c14d-fc2c-4846-bc3d-cc4f3e8393f8] Exception: java.lang.NullPointerException > at org.ovirt.engine.core.bll.hostdev.RemoveVmHostDevicesCommand.executeCommand(RemoveVmHostDevicesCommand.java:64) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1205) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1345) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1987) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:202) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInRequired(TransactionSupport.java:137) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:105) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1405) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:412) [bll.jar:] > at org.ovirt.engine.core.bll.executor.DefaultBackendActionExecutor.execute(DefaultBackendActionExecutor.java:13) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:509) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:491) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:444) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_151] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_151] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_151] > at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_151] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) > at org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:92) [wildfly-weld-ejb-11.0.0.Final.jar:11.0.0.Final] > at org.jboss.weld.interceptor.proxy.WeldInvocationContext.interceptorChainCompleted(WeldInvocationContext.java:98) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > at org.jboss.weld.interceptor.proxy.WeldInvocationContext.proceed(WeldInvocationContext.java:117) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > at org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12) [common.jar:] > at sun.reflect.GeneratedMethodAccessor67.invoke(Unknown Source) [:1.8.0_151] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_151] > at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_151] > at org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > at org.jboss.weld.interceptor.proxy.WeldInvocationContext.invokeNext(WeldInvocationContext.java:83) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > at org.jboss.weld.interceptor.proxy.WeldInvocationContext.proceed(WeldInvocationContext.java:115) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > at org.jboss.weld.bean.InterceptorImpl.intercept(InterceptorImpl.java:108) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > at org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:82) [wildfly-weld-ejb-11.0.0.Final.jar:11.0.0.Final] > at org.jboss.as.weld.interceptors.EjbComponentInterceptorSupport.delegateInterception(EjbComponentInterceptorSupport.java:60) > at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:76) > at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:88) > at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:101) > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) > at org.ovirt.engine.core.bll.interceptors.CorrelationIdTrackerInterceptor.aroundInvoke(CorrelationIdTrackerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor97.invoke(Unknown Source) [:1.8.0_151] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_151] > at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_151] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:89) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45) [wildfly-ee-11.0.0.Final.jar:11.0.0.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:40) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53) > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:264) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:379) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:244) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) > at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73) [weld-core-impl-2.4.3.Final.jar:2.4.3.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) [wildfly-ejb3-11.0.0.Final.jar:11.0.0.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438) > at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:609) > at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53) > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:81) > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view4.runAction(Unknown Source) [common.jar:] > at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runAction(GenericApiGWTServiceImpl.java:176) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_151] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_151] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_151] > at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_151] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:587) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:333) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:303) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:373) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final] > at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.service(GenericApiGWTServiceImpl.java:78) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final] > at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) > at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129) > at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] > at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) > at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) > at org.ovirt.engine.core.utils.servlet.CachingFilter.doFilter(CachingFilter.java:133) [utils.jar:] > at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) > at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) > at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:73) [branding.jar:] > at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) > at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) > at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:65) [utils.jar:] > at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) > at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) > at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84) > at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) > at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) > at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) > at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) > at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) > at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) > at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) > at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) > at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) > at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) > at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) > at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) > at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) > at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) > at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) > at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) > at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) > at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) > at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) > at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) > at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) > at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) > at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) > at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) > at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) > at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) > at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) > at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) > at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) > at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) > at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) > at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) > at io.undertow.server.Connectors.executeRootHandler(Connectors.java:326) > at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:812) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_151] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_151] > at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_151] > > 2018-01-17 11:37:51,049+01 ERROR [org.ovirt.engine.core.bll.hostdev.RemoveVmHostDevicesCommand] (default task-48) [57a8c14d-fc2c-4846-bc3d-cc4f3e8393f8] Transaction rolled-back for command 'org.ovirt.engine.core.bll.hostdev.RemoveVmHostDevicesCommand'. > 2018-01-17 11:37:51,059+01 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-48) [57a8c14d-fc2c-4846-bc3d-cc4f3e8393f8] EVENT_ID: VM_REMOVE_HOST_DEVICES(10,801), Host devices [usb_4_10] were detached from Vm license2.int.lugundtrug.net by User admin@internal-authz.

1 0

oVirt 2018 Survey
by Sandro Bonazzola 17 Jan '18

17 Jan '18

As we continue to develop oVirt 4.2 and future releases, the Development and Integration teams at Red Hat would value insights on how you are deploying the oVirt environment. Please help us to hit the mark by completing this short survey. Survey will close on February 1st. Here's the link to the survey: https://goo.gl/forms/cAKWAR8RD7rGrVhE2 Thanks, -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

5 5

oVirt 4.2.1 rc1 and upload iso to data domain test
by Gianluca Cecchi 17 Jan '18

17 Jan '18

Hello, I see in release notes this BZ 1530730 [downstream clone - 4.2.1] [RFE] Allow uploading ISO images to data domains and using them in VMs It is now possible to upload an ISO file to a data domain and attach it to a VM as a CDROM device. In order to do so the user has to upload an ISO file via the UI (which will recognize the ISO by it's header and will upload it as ISO) or via the APIs in which case the request should define the disk container "content_type" property as "iso" before the upload. Once the ISO exists on an active storage domain in the data center it will be possible to attach it to a VM as a CDROM device either through the "Edit VM" dialog or through the APIs (see example in comment #27 So I'm trying it on an HCI Gluster environment of mine for testing. I get this in image-proxy.log (Thread-39 ) INFO 2018-01-14 18:35:38,066 web:95:web:(log_start) START [192.168.150.101] PUT /images/0d852f7a-b19e-447d-82ad-966755070701 (Thread-39 ) WARNING 2018-01-14 18:35:38,067 web:112:web:(log_error) ERROR [192.168.150.101] PUT /images/0d852f7a-b19e-447d-82ad-966755070701: [401] Not authorized (0.00s) (Thread-40 ) INFO 2018-01-14 18:35:38,106 web:95:web:(log_start) START [192.168.150.101] PUT /images/0d852f7a-b19e-447d-82ad-966755070701 (Thread-40 ) WARNING 2018-01-14 18:35:38,106 web:112:web:(log_error) ERROR [192.168.150.101] PUT /images/0d852f7a-b19e-447d-82ad-966755070701: [401] Not authorized (0.00s) Does this mean the functionality is not completely ready yet or what? Any one has already tried on iSCSI and/or FC? Thanks, Gianluca

2 6

Problems with some vms
by Endre Karlson 17 Jan '18

17 Jan '18

Hi, we are getting some errors with some of our vms in a 3 node server setup. 2018-01-14 15:01:44,015+0100 INFO (libvirt/events) [virt.vm] (vmId='2c34f52d-140b-4dbe-a4bd-d2cb467b0b7c') abnormal vm stop device virtio-disk0 error eother (vm:4880) We are running glusterfs for shared storage. I have tried setting global maintenance on the first server and then issuing a 'hosted-engine --vm-start' but that leads to nowhere.

3 7

oVirt 4.x - RestAPI - Backup - How to Image Inactive Disk
by Zip 16 Jan '18

16 Jan '18

> This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. --B_3598954007_92574425 Content-type: text/plain; charset="ISO-8859-1" Content-transfer-encoding: quoted-printable Hi All, I have created a bash script that uses curl and the REST API based on the docs that I could find. I have the script working all the way up until the VM-ToBeBackedUp-Disk is attached to the VM-RunningTheBackupScript with the mention of the Snapshot. Example: <disk_attachment> <disk id=3D"33b533f8-13b1-4cc1-8091-31b07913b32a"> <snapshot id=3D"34709147-4b0c-4684-b3d9-f4892873f36f=B2/> </disk> <bootable>false</bootable> <interface>virtio</interface> </disk_attachment> Posted to :=20 https://-myFQDN-/ovirt-engine/api/vms/-UUID-OF-VM-RunningTheBackupScript/di= s kattachments/ The above results in an attached disk to my VM-RunningTheBackupScript. The question I have is how to I image that disk from the VM-RunningTheBackupScript? The VM-RunningTheBackupScript is running Debian8 and that VM cannot see the attached disk as it is Inactive. I have reviewed many of the older scripts out there and it looks trivial, however everything I have tried has no joy ;/ If anyone on the list knows of the required magic, please advise ;) * If I have to go to using python etc, I will, but trying to avoid it if possible. Either way, I need some examples of how to proceed. Some links I have reviewed: https://www.ovirt.org/develop/api/design/backup-api/ http://200.1.19.60/ovirt-engine/docs/manual/en-US/html/Administration_Guide= / sect-Backing_Up_and_Restoring_Virtual_Machines_Using_the_Backup_and_Restore= _ API.html https://markmc.fedorapeople.org/rhevm-api/en-US/html-single/index.html https://github.com/voidloop/ovirt-bash-backup/blob/master/backupvm.sh https://github.com/laravot/backuprestoreapi/blob/master/example.py Thanks Zip --B_3598954007_92574425 Content-type: text/html; charset="ISO-8859-1" Content-transfer-encoding: quoted-printable <html><head></head><body style=3D"word-wrap: break-word; -webkit-nbsp-mode: s= pace; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size:= 14px; font-family: Calibri, sans-serif;"><div>Hi All,</div><div><br></div><= div>I have created a bash script that uses curl and the REST API based on th= e docs that I could find. </div><div><br></div><div>I have the script w= orking all the way up until the VM-ToBeBackedUp-Disk is attached to the VM-R= unningTheBackupScript with the mention of the Snapshot.</div><div><br></div>= <div>Example:</div><div><br></div><div><div><disk_attachment></div><di= v>    <disk id=3D"33b533f8-13b1-4cc1-8091-31b07913b32a"></div>= <div>       <snapshot id=3D"34709147-4b0c-4684-b3d9-f48= 92873f36f”/></div><div>    </disk></div><div> =   <bootable>false</bootable></div><div>    <i= nterface>virtio</interface></div><div></disk_attachment></div= ></div><div><br></div><div>Posted to : <a href=3D"https://-myFQDN-/ovirt-= engine/api/vms/-UUID-OF-VM-RunningTheBackupScript/diskattachments">https://-= myFQDN-/ovirt-engine/api/vms/-UUID-OF-VM-RunningTheBackupScript/diskattachme= nts</a>/</div><div><br></div><div>The above results in an attached disk to m= y VM-RunningTheBackupScript.</div><div><br></div><div>The question I have is= how to I image that disk from the VM-RunningTheBackupScript?</div><div><br>= </div><div>The VM-RunningTheBackupScript is running Debian8 and that VM cann= ot see the attached disk as it is Inactive.</div><div><br></div><div>I have = reviewed many of the older scripts out there and it looks trivial, however e= verything I have tried has no joy ;/</div><div><br></div><div>If anyone on t= he list knows of the required magic, please advise ;)</div><div><br></div><d= iv>* If I have to go to using python etc, I will, but trying to avoid it if = possible. Either way, I need some examples of how to proceed.</div><div><br>= </div><div>Some links I have reviewed:</div><div><br></div><div><a href=3D"htt= ps://www.ovirt.org/develop/api/design/backup-api">https://www.ovirt.org/deve= lop/api/design/backup-api</a>/</div><div><a href=3D"http://200.1.19.60/ovirt-e= ngine/docs/manual/en-US/html/Administration_Guide/sect-Backing_Up_and_Restor= ing_Virtual_Machines_Using_the_Backup_and_Restore_API.html">http://200.1.19.= 60/ovirt-engine/docs/manual/en-US/html/Administration_Guide/sect-Backing_Up_= and_Restoring_Virtual_Machines_Using_the_Backup_and_Restore_API.html</a></di= v><div><a href=3D"https://markmc.fedorapeople.org/rhevm-api/en-US/html-single/= index.html">https://markmc.fedorapeople.org/rhevm-api/en-US/html-single/inde= x.html</a></div><div><a href=3D"https://github.com/voidloop/ovirt-bash-backup/= blob/master/backupvm.sh">https://github.com/voidloop/ovirt-bash-backup/blob/= master/backupvm.sh</a></div><div><a href=3D"https://github.com/laravot/backupr= estoreapi/blob/master/example.py">https://github.com/laravot/backuprestoreap= i/blob/master/example.py</a></div><div><br></div><div><br></div><div>Thanks<= /div><div><br></div><div>Zip</div></body></html> --B_3598954007_92574425--

1 0

Are Ovirt updates nessessary after CVE-2017-5754 CVE-2017-5753 CVE-2017-5715
by Marcel Hanke 16 Jan '18

16 Jan '18

Hi, besides the kernel and microcode updates are there also updates of ovirt- engine and vdsm nessessary and if so, is there a timeline when the patches can be expected? If there are Patches nessessary will there also be updates for ovirt 4.1 or only 4.2? Thanks Marcel

8 17

Shutdown all VM's command line
by Wesley Stewart 16 Jan '18

16 Jan '18

Is there an easy way to do this? I have a UPS connected to my freenas box, and using NUT and UPSC I can monitor the battery level on my centos 7 host. In the case of an emergency, and my UPS goes below, lets say 40% battery, I would like to make a script to shutdown all running VMs, and then reboot the host. Otherwise I am afraid of sending a straight reboot command to the host without taking care of the virtual machines first. Thanks!

6 8

ovirt-guest-agent on atomic ?
by Nathanaël Blanchet 16 Jan '18

16 Jan '18

Hi all, I didn't find any working ovirt-guest-agent for atomic7, and it is a requirement for launching atomic from vagrant-ovirt4 plugin. I tried installing ovirt-guest-agent inside, I tried installing ovirtguestagent/centos7-atomic, but I had no success Does any official project exist for it? -- Nathanaël Blanchet Supervision réseau Pôle Infrastrutures Informatiques 227 avenue Professeur-Jean-Louis-Viala 34193 MONTPELLIER CEDEX 5 Tél. 33 (0)4 67 54 84 55 Fax 33 (0)4 67 54 84 14 blanchet(a)abes.fr

2 1

Re: [ovirt-users] Configuration of FCoE in oVirt 4.2 on HP BladeSystem c7000
by Gunder Johansen 15 Jan '18

15 Jan '18

------=_Part_6439429_1730970573.1516054958003 Content-Type: multipart/alternative; boundary="----=_Part_6439428_201364203.1516054957950" ------=_Part_6439428_201364203.1516054957950 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Thanks, Fred. I have been looking at FCoE VDSM Hooks to no help. Looking at it again, I a= m still not able to see any FCoE=C2=A0 [root@ovirtengine ~]# engine-config -g UserDefinedNetworkCustomPropertiesUs= erDefinedNetworkCustomProperties:=C2=A0 version: 3.6UserDefinedNetworkCusto= mProperties:=C2=A0 version: 4.0UserDefinedNetworkCustomProperties: fcoe=3D^= ((enable|dcb|auto_vlan)=3D(yes|no),?)*$ version: 4.1UserDefinedNetworkCusto= mProperties: fcoe=3D^((enable|dcb|auto_vlan)=3D(yes|no),?)*$ version: 4.2 I finally managed to find the "custom Property" where I could set "enable= =3Dyes,dcb=3Dno" for fcoe, but when applying the change, I get unexepected = error. Yes, the host was in local maintenance mode when I applied the chang= e. I am afraid I am not understanding all the steps needed from the Virtual Co= nnect configuration in the blade rack to the network interfaces inside oVir= t. Should I add a new FCoE only network interface and should this have VLAN= /IP address in a special range compared to the internal network in the rack= ? Thanks again. =C2=A0From:=C2=A0Fred Rolland [mailto:frolland@redhat.com]=C2=A0 Sent:=C2=A013. januar 2018 14:21 To:=C2=A0Luca 'remix_tj' Lorenzetto Cc:=C2=A0Gunder Johansen; users Subject:=C2=A0Re: [ovirt-users] Configuration of FCoE in oVirt 4.2 on HP Bl= adeSystem c7000=C2=A0Take a look also at the FCoE Vdsm hook: oVirt/vdsm |=20 |=20 |=20 | | | | | |=20 | |=20 oVirt/vdsm vdsm - This is a mirror for http://gerrit.ovirt.org, for issues use http://= bugzilla.redhat.com | | | | ------=_Part_6439428_201364203.1516054957950 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <html><head></head><body><div style=3D"color:#000; background-color:#fff; f= ont-family:Courier New, courier, monaco, monospace, sans-serif;font-size:13= px"><div id=3D"yui_3_16_0_ym19_1_1516021449310_15492"><span id=3D"yui_3_16_= 0_ym19_1_1516021449310_15493">Thanks, Fred.</span></div><div id=3D"yui_3_16= _0_ym19_1_1516021449310_15494"><br clear=3D"none" id=3D"yui_3_16_0_ym19_1_1= 516021449310_15495"></div><div dir=3D"ltr" id=3D"yui_3_16_0_ym19_1_15160214= 49310_15496">I have been looking at FCoE VDSM Hooks to no help. Looking at = it again, I am still not able to see any FCoE </div><div id=3D"yui_3_1= 6_0_ym19_1_1516021449310_15497"><br clear=3D"none" id=3D"yui_3_16_0_ym19_1_= 1516021449310_15498"></div><div id=3D"yui_3_16_0_ym19_1_1516021449310_15499= ">[root@ovirtengine ~]# engine-config -g UserDefinedNetworkCustomProperties= </div><div id=3D"yui_3_16_0_ym19_1_1516021449310_15500">UserDefinedNetworkC= ustomProperties:  version: 3.6</div><div id=3D"yui_3_16_0_ym19_1_15160= 21449310_15501">UserDefinedNetworkCustomProperties:  version: 4.0</div= ><div id=3D"yui_3_16_0_ym19_1_1516021449310_15502">UserDefinedNetworkCustom= Properties: fcoe=3D^((enable|dcb|auto_vlan)=3D(yes|no),?)*$ version: 4.1</d= iv><div id=3D"yui_3_16_0_ym19_1_1516021449310_15503">UserDefinedNetworkCust= omProperties: fcoe=3D^((enable|dcb|auto_vlan)=3D(yes|no),?)*$ version: 4.2<= /div><div id=3D"yui_3_16_0_ym19_1_1516021449310_15504"><br clear=3D"none" i= d=3D"yui_3_16_0_ym19_1_1516021449310_15505"></div><div dir=3D"ltr" id=3D"yu= i_3_16_0_ym19_1_1516021449310_15506">I finally managed to find the "custom = Property" where I could set "enable=3Dyes,dcb=3Dno" for fcoe, but when appl= ying the change, I get unexepected error. Yes, the host was in local mainte= nance mode when I applied the change.</div><div dir=3D"ltr" id=3D"yui_3_16_= 0_ym19_1_1516021449310_15507"><br clear=3D"none" id=3D"yui_3_16_0_ym19_1_15= 16021449310_15508"></div><div dir=3D"ltr" id=3D"yui_3_16_0_ym19_1_151602144= 9310_15509">I am afraid I am not understanding all the steps needed from th= e Virtual Connect configuration in the blade rack to the network interfaces= inside oVirt. Should I add a new FCoE only network interface and should th= is have VLAN/IP address in a special range compared to the internal network= in the rack?</div><div dir=3D"ltr" id=3D"yui_3_16_0_ym19_1_1516021449310_1= 5510"><br clear=3D"none" id=3D"yui_3_16_0_ym19_1_1516021449310_15511"></div= ><div dir=3D"ltr" id=3D"yui_3_16_0_ym19_1_1516021449310_15512">Thanks again= .</div><div dir=3D"ltr" id=3D"yui_3_16_0_ym19_1_1516021449310_15513"><br cl= ear=3D"none" id=3D"yui_3_16_0_ym19_1_1516021449310_15514"><br clear=3D"none= " id=3D"yui_3_16_0_ym19_1_1516021449310_15515"></div><div dir=3D"ltr" id=3D= "yui_3_16_0_ym19_1_1516021449310_15516"><br clear=3D"none" id=3D"yui_3_16_0= _ym19_1_1516021449310_15517"></div><div id=3D"yui_3_16_0_ym19_1_15160214493= 10_15518"><br clear=3D"none" id=3D"yui_3_16_0_ym19_1_1516021449310_15519"><= br clear=3D"none" id=3D"yui_3_16_0_ym19_1_1516021449310_15520"></div><div i= d=3D"yui_3_16_0_ym19_1_1516021449310_15521"><div id=3D"yui_3_16_0_ym19_1_15= 16021449310_15522"><div style=3D"font-family: HelveticaNeue, "Helvetic= a Neue", Helvetica, Arial, "Lucida Grande", sans-serif; font= -size: 16px;" id=3D"yui_3_16_0_ym19_1_1516021449310_15523"><div dir=3D"ltr"= id=3D"yui_3_16_0_ym19_1_1516021449310_15524"><font size=3D"2" face=3D"Aria= l" id=3D"yui_3_16_0_ym19_1_1516021449310_15525"></font><hr size=3D"1" id=3D= "yui_3_16_0_ym19_1_1516021449310_15526"><div id=3D"yui_3_16_0_ym19_1_151602= 1449310_15527"><span style=3D"font-size: 11pt;" id=3D"yui_3_16_0_ym19_1_151= 6021449310_15528"> </span></div><div id=3D"yui_3_16_0_ym19_1_151602144= 9310_15529"><b id=3D"yui_3_16_0_ym19_1_1516021449310_15530"><span lang=3D"E= N-US" style=3D"font-size: 10pt;" id=3D"yui_3_16_0_ym19_1_1516021449310_1553= 1">From:</span></b><span lang=3D"EN-US" style=3D"font-size: 10pt;" id=3D"yu= i_3_16_0_ym19_1_1516021449310_15532"> Fred Rolland [mailto:frolland@re= dhat.com] <br clear=3D"none" id=3D"yui_3_16_0_ym19_1_1516021449310_155= 33"><b id=3D"yui_3_16_0_ym19_1_1516021449310_15534">Sent:</b> 13. janu= ar 2018 14:21<br clear=3D"none" id=3D"yui_3_16_0_ym19_1_1516021449310_15535= "><b id=3D"yui_3_16_0_ym19_1_1516021449310_15536">To:</b> Luca 'remix_= tj' Lorenzetto<br clear=3D"none" id=3D"yui_3_16_0_ym19_1_1516021449310_1553= 7"><b id=3D"yui_3_16_0_ym19_1_1516021449310_15538">Cc:</b> Gunder Joha= nsen; users<br clear=3D"none" id=3D"yui_3_16_0_ym19_1_1516021449310_15539">= <b id=3D"yui_3_16_0_ym19_1_1516021449310_15540">Subject:</b> Re: [ovir= t-users] Configuration of FCoE in oVirt 4.2 on HP BladeSystem c7000</span><= /div><div id=3D"yui_3_16_0_ym19_1_1516021449310_15541"> </div><div dir= =3D"ltr" id=3D"yui_3_16_0_ym19_1_1516021449310_15542">Take a look also at t= he FCoE Vdsm hook:<br clear=3D"none" id=3D"yui_3_16_0_ym19_1_1516021449310_= 15543"><br clear=3D"none" id=3D"yui_3_16_0_ym19_1_1516021449310_15544"><a r= el=3D"nofollow" shape=3D"rect" target=3D"_blank" href=3D"https://github.com= /oVirt/vdsm/tree/master/vdsm_hooks/fcoe" style=3D"color: rgb(25, 106, 212);= " id=3D"yui_3_16_0_ym19_1_1516021449310_15545">oVirt/vdsm</a></div><div id= =3D"yui_3_16_0_ym19_1_1516021449310_15546"><br clear=3D"none" id=3D"yui_3_1= 6_0_ym19_1_1516021449310_15547"></div><div class=3D"yiv3767805512yahoo-link= -enhancr-card yiv3767805512ymail-preserve-class yiv3767805512ymail-preserve= -style" dir=3D"ltr" data-url=3D"https://github.com/oVirt/vdsm/tree/master/v= dsm_hooks/fcoe" data-type=3D"yenhancr" data-category=3D"object" data-embed-= url=3D"" data-size=3D"medium" style=3D"overflow-x: auto; overflow-y: hidden= ; max-width: 400px; font-family: "Helvetica Neue", Helvetica, Ari= al, sans-serif;" id=3D"yui_3_16_0_ym19_1_1516021449310_15548" contenteditab= le=3D"false"><a rel=3D"nofollow" shape=3D"rect" target=3D"_blank" href=3D"h= ttps://github.com/oVirt/vdsm/tree/master/vdsm_hooks/fcoe" style=3D"color: r= gb(0, 0, 0);" id=3D"yui_3_16_0_ym19_1_1516021449310_15549"><table class=3D"= yahoo-ignore-table" cellpadding=3D"0" cellspacing=3D"0" border=3D"0" style= =3D"word-break: break-word; overflow-x: auto; max-width: 400px;" id=3D"yui_= 3_16_0_ym19_1_1516021449310_15550"><tbody style=3D"width: 400px;" id=3D"yui= _3_16_0_ym19_1_1516021449310_15551"><tr id=3D"yui_3_16_0_ym19_1_15160214493= 10_15552"><td colspan=3D"1" rowspan=3D"1" width=3D"400" id=3D"yui_3_16_0_ym= 19_1_1516021449310_15553"><table class=3D"yahoo-ignore-table" cellpadding= =3D"0" cellspacing=3D"0" border=3D"0" width=3D"100%" style=3D"word-break: b= reak-word; overflow-x: auto; max-width: 400px;" id=3D"yui_3_16_0_ym19_1_151= 6021449310_15554"><tbody style=3D"width: 400px;" id=3D"yui_3_16_0_ym19_1_15= 16021449310_15555"><tr id=3D"yui_3_16_0_ym19_1_1516021449310_15556"><td col= span=3D"1" rowspan=3D"1" valign=3D"top" background=3D"https://s.yimg.com/vv= //api/res/1.2/TRGM7nw4vAiw.NSSO6tFAA--~A/YXBwaWQ9bWFpbDtmaT1maWxsO2g9MjAwO3= c9NDAw/https://avatars1.githubusercontent.com/u/1318634?s=3D400&v=3D4.c= f.jpg" bgcolor=3D"#000000" style=3D"background: url("https://s.yimg.co= m/vv//api/res/1.2/TRGM7nw4vAiw.NSSO6tFAA--~A/YXBwaWQ9bWFpbDtmaT1maWxsO2g9Mj= AwO3c9NDAw/https://avatars1.githubusercontent.com/u/1318634?s=3D400&v= =3D4.cf.jpg") center center / cover no-repeat rgb(0, 0, 0); min-height= : 200px; position: relative;" id=3D"yui_3_16_0_ym19_1_1516021449310_15557">= <table class=3D"yahoo-ignore-table" cellpadding=3D"0" cellspacing=3D"0" bor= der=3D"0" style=3D"word-break: break-word; width: 400px;" id=3D"yui_3_16_0_= ym19_1_1516021449310_15558"><tbody style=3D"width: 400px;" id=3D"yui_3_16_0= _ym19_1_1516021449310_15559"><tr id=3D"yui_3_16_0_ym19_1_1516021449310_1556= 0"><td colspan=3D"1" rowspan=3D"1" valign=3D"top" background=3D"https://s.y= img.com/nq/storm/assets/enhancrV2/12/overlay-tile.png" bgcolor=3D"transpare= nt" style=3D"background: url("https://s.yimg.com/nq/storm/assets/enhan= crV2/12/overlay-tile.png") left top repeat transparent; min-height: 20= 0px;" id=3D"yui_3_16_0_ym19_1_1516021449310_15561"><table class=3D"yahoo-ig= nore-table" height=3D"185" style=3D"word-break: break-word; width: 400px; m= in-height: 185px;" id=3D"yui_3_16_0_ym19_1_1516021449310_15562"><tbody styl= e=3D"width: 400px;" id=3D"yui_3_16_0_ym19_1_1516021449310_15563"><tr id=3D"= yui_3_16_0_ym19_1_1516021449310_15564"><td colspan=3D"1" rowspan=3D"1" styl= e=3D"padding-top: 15px; padding-left: 15px; vertical-align: top;" id=3D"yui= _3_16_0_ym19_1_1516021449310_15565"></td><td colspan=3D"1" rowspan=3D"1" st= yle=3D"text-align: right; padding-top: 15px; padding-right: 15px; vertical-= align: top;" id=3D"yui_3_16_0_ym19_1_1516021449310_15566"><div style=3D"ver= tical-align: top;" id=3D"yui_3_16_0_ym19_1_1516021449310_15567"><button dat= a-share-url=3D"https://github.com/oVirt/vdsm/tree/master/vdsm_hooks/fcoe" s= tyle=3D"min-height: 32px; width: 32px; background-color: rgba(255, 255, 255= , 0.9); border-width: initial; border-style: none; border-color: initial; b= order-radius: 2px; box-shadow: rgba(0, 0, 0, 0.1) 0px 1px 1px 0px; padding:= 4.5px;" id=3D"yui_3_16_0_ym19_1_1516021449310_15568"><span style=3D"font-s= ize: 24px; padding: 0px;" id=3D"yui_3_16_0_ym19_1_1516021449310_15569"></sp= an></button></div></td></tr></tbody></table></td></tr></tbody></table></td>= </tr><tr id=3D"yui_3_16_0_ym19_1_1516021449310_15570"><td colspan=3D"1" row= span=3D"1" id=3D"yui_3_16_0_ym19_1_1516021449310_15571"><div id=3D"yui_3_16= _0_ym19_1_1516021449310_15572"></div><table align=3D"center" class=3D"yahoo= -ignore-table" cellpadding=3D"0" cellspacing=3D"0" border=3D"0" style=3D"ma= rgin-top: -40px; margin-right: auto; margin-left: auto; word-break: break-w= ord; background-image: initial; background-position: initial; background-si= ze: initial; background-repeat: initial; background-attachment: initial; ba= ckground-origin: initial; background-clip: initial; position: relative; z-i= ndex: 2; width: 378px; overflow-x: auto; max-width: 380px; border-width: 1p= x 1px 3px; border-style: solid; border-color: rgb(224, 228, 233) rgb(224, 2= 28, 233) rgb(1, 1, 1); border-image: initial;" id=3D"yui_3_16_0_ym19_1_1516= 021449310_15573"><tbody style=3D"width: 378px;" id=3D"yui_3_16_0_ym19_1_151= 6021449310_15574"><tr id=3D"yui_3_16_0_ym19_1_1516021449310_15575"><td cols= pan=3D"1" rowspan=3D"1" style=3D"padding-top: 16px; padding-bottom: 16px; p= adding-left: 12px; vertical-align: top;" id=3D"yui_3_16_0_ym19_1_1516021449= 310_15576"><img class=3D"yiv3767805512card-object-1 yiv3767805512yahoo-igno= re-inline-image yiv3767805512ymail-preserve-class" src=3D"https://s.yimg.co= m/nq/storm/assets/enhancrV2/23/logos/github.png" height=3D"32" style=3D"bor= der-width: 1px; border-style: solid; border-color: rgb(224, 228, 233); min-= width: 32px; margin-top: 3px;" id=3D"yui_3_16_0_ym19_1_1516021449310_15577"= data-id=3D"cb935002-08ec-9eaa-0113-1e31d0f20d58"></td><td colspan=3D"1" ro= wspan=3D"1" style=3D"vertical-align: middle; padding: 16px 12px; width: 296= px;" id=3D"yui_3_16_0_ym19_1_1516021449310_15578"><h2 style=3D"margin-top: = 0px; margin-bottom: 4px; font-size: 16px; line-height: 19px;" id=3D"yui_3_1= 6_0_ym19_1_1516021449310_15579">oVirt/vdsm</h2><div style=3D"font-size: 11p= x; line-height: 15px; color: rgb(153, 153, 153);" id=3D"yui_3_16_0_ym19_1_1= 516021449310_15580">vdsm - This is a mirror for http://gerrit.ovirt.org, fo= r issues use http://bugzilla.redhat.com</div></td><td colspan=3D"1" rowspan= =3D"1" style=3D"text-align: right; padding-top: 16px; padding-right: 12px; = padding-bottom: 16px;" id=3D"yui_3_16_0_ym19_1_1516021449310_15581"></td></= tr></tbody></table><div id=3D"yui_3_16_0_ym19_1_1516021449310_15582"></div>= </td></tr></tbody></table><div id=3D"yui_3_16_0_ym19_1_1516021449310_15583"= ></div></td></tr></tbody></table><div id=3D"yui_3_16_0_ym19_1_1516021449310= _15584"></div></a><br id=3D"yui_3_16_0_ym19_1_1516021449310_15585"><br id= =3D"yui_3_16_0_ym19_1_1516021449310_15586"></div><div id=3D"yui_3_16_0_ym19= _1_1516021449310_15587"></div></div></div></div></div></div></body></html> ------=_Part_6439428_201364203.1516054957950-- ------=_Part_6439429_1730970573.1516054958003 Content-Type: image/png Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="screenshot.1161.png" Content-ID: <18712935-6369-a66a-7979-b3478851eea3(a)yahoo.com> iVBORw0KGgoAAAANSUhEUgAAAloAAAGwCAYAAABxbMuTAAAAAXNSR0IArs4c6QAAAARnQU1BAACx jwv8YQUAAAAJcEhZcwAADsQAAA7EAZUrDhsAAHHQSURBVHhe7d0JYBTV/QfwbzjDIbeKyqXIoUkJ im0Fg4pWRVAOLwQKWlvRitSrgKIU8SyoKH9rVbQeIChVq6GCJwUkAioiwYQzCOEyIjchBEjIf36z 85K3k5m9spPsJt9P//OfmffevJldzM43b2YnCYcOHSoGEVUJK1aswLnnnmut+djL9HVZFvp6sO2J iCh0Naw5EcUJCT4qIAUSrI2qlxBlX1aTKhP2ciIiCo5BiygOOY06CT0U6XPhtKyHLJnr5apOlRER UfgYtKjKi3ZQKG9/Ttvby5zWVZkeslS5PTCpoKQmobdRZcLeTqh2in1dOJUREZE/Bi2KS6EEASHl esCQeajbCqc6PZAIpz5FOGX6MQr7MQtVpiZVZj8eN7KNaqu2F/qy3pfe3k7fhoiI3DFoVXPROGFK H8H6cavXy52W3baTACB19npVpperZXtbxd6X3i6UoCFtwjke1VbI3L7uRtWpY5J1fVLler2aO9XL XG+jz+19yLo+qTKh2hARUVn81mE1pZ8c5YQp6/a5ok6oir2Nqrdv49aH2j5U9vb2/QqnfetzYe9H OJXpAtVLndD3Y2+vr6v2wqlMhFsupM7et9O6U7lQZfZ6Yd/WPndrQ0REPhzRqibkBKhPwn5CVOVC 6vR6e50Te7lT/3o/wZbt7XXSt71e35+qd6K2k8nej14nk14vk95G6tQ+VbtA7YVqr8pUH3q5ospk bi9320aofoW+rV4u7H3at3Nqa58LfTud3oaIqLriiFYlUycjdaKzU+X6XOjLOre2gaj2TvR9BGrj tL9A/SqBttHr1LKi9+u0H7d+nDi1DUba6/2qPux9qWXFqZ0iZSLQNkLVO/Xhxq1vO6c+A+3HrU7t j4ioumPQqkRyggp2Qgr3BCdUn6p/ezt9n059qG30foRbX3qZvq0+10mZUOVqXdi3cWur96na27dT ZUqwbXRO2wt7W6ft7dval4PRtw/U3t5nOPsIhdP+Ax2bU5nOvq3MiYiqOgatGKKffOzLTtQJS7Gv 27nV6+X6PoP1F4za3j4Pl769zqkvp305lQXj1jZQ/zp7mb6dsLcXTv2IULZx2tatPyVYfSCR7s9O 2juVExFVFQxaFUydiPSTizrZOJ2kAp28VJ0+F3p7exs3+rZubYP1IVQ/ilt71Ze9T7W9Kgtln4pb n8G4bResH7f6UPcfajsRTltdKNtF2rcb1Z9bv1JORFRdMGhVEvsJSJ18Ap3wwjkhhnKSU/V622js QxfqcdjbOW0Xyv6iKdT9Reu49H5kWbj1W9HvRTDBjlc4HbPajoioqmLQqmByorGflJxOUuqk5HRy UuxtArUNVyj7jYQXx6rzql9F+heR7CMa71s8CHSsTnXqPSUiqooYtDymTirhnHwCtVVCaWMXyTZC bRfucUW6v1CFc1yK18cUqlg5Di+F+xqlPRFRVcOgVQHcTjbBTkSB6uPxRH3ffR/gmWcGWmveqqj3 pzz7icd/Qy/J+1FVZU67CaPe3oI2g5/HmyOSrVIiqg4ifmBp7lfTMHpwX1zWqxd69eqNfrc+hH8t 2mLVBrF8Fh55ZBaWW6ueyJyGm+TYbpqGTKtIyAdeL6P8pml6aaSWY9Yjj2CW7YXoJ0+1rJ9E1LLT SVZvF+gk7NRvpM6tE9q/hASlcNiPraJClgj03oUj2Ptbnv1E6xi9Fo3/xspl3njzZ7ZXr8sw8q1s q1BkYtpNZX/GA6qIzx4iIk1kQSvnHTzx8NtYvgtofW4vdE9qiqLNK/FT4clWg0COYP4ns7BgwRrs tEri1ZH5n2DWggVYY70QOXGqk6e+LNyW7QLVOQm3vZMVR8+zlgILNyjpx7Z86n3WUvgeGxv5ttEW aegId7tKDzeaaPw3Zuf0+mQ/gfdViNVvvoj3cq3VsFWdzx4iih+RBa2sLPxQaMzPH4nXnvkbnvjH 25j70fsYfWldX33uV5g2ejD6XqZGux7Df9cfMSr2YOGUW/HU/EPGcjqeMn9LHY95xpoaaRovKybr t1Wr3vi1FuOl/X2v47+Tb/H13fsGjJ72FSL+3DUdwfr/PoZb+/U299+rdz/c+th/YR6u1K7/Lx67 tR96W79R973mVjz5aQ72LJyCW5+aD/OVPCV1vcz2sXSCrCj33TfOWnJ33l3PWEvhe2jSM8Y+Kids 2U/8kYSOseOfCns7e/vxT/3HWqoa9NcX6r9th7PPRq3CFZj+8mfGJ4mLcD97lj2H62T5gf8anwSG nLcw0qy7Ck8ulAJg2XPXmT/74+dJi1x8NW00Bve9zPyZv6zvYP/PIHP0zdh2fgbeGn2D+blR+plW Knf+kxhiHONlQ57EfGPjMlcIht2JNzjsRlQlRBa0kpLwq1rGPP0l3Do5Dav2GB9AdevCF7Oy8daj D+Pt5YfQvt/tuOcP3dFw83xMGT/NHK5v2iUFbcx2HXDVPffgnnuuRGdzPUQrpuOFzKb4bc9z0Rq/ YPnbk/D6QisVOdm9DDMeeQSPWNOMZbutCp8jRmD665T5yD56Es41PuTOPekosucbZX+XD3PjN+AZ /4f52cfRvr+8ln7o3KQIjZu3lReCFN8LQYer7sEzz/iChNMJ9R//mGktVU3PPPOEteQd9f7Go0mP jraWnH0QQtB4dPQ11lLVI/+2X732kLXm7uTeI3CN8TN3cOG/MH2Z0898BJ8953dFtwZG0Q+ZWGrM 9nz7PVajOZo3P4TVazKMkkys+l4+M4yf9/PrIvutR/Hw28uxq257dO/VHe3r7jI+gx7GuH/pFy8P YdFTY/GvlYdQv0UKOp5hFVuOZEzDOOPz5ad6qbhnygO4tGUm5kx7G8tzG+L8P9yD2wdfgDaFTdDY th0RxafIglbbGzHu4cHo2vQwsj9+Dndd2x/DJ3+GLfLZt3YR/re6EGh1Jf44ahD6DZ+Av/RtDuxc hPRlzZDyu7Pgu8B4Ms7q1w/9+vVAeJ8n5+JPzz2Dv/3tGTz3Jxk9OoivlspHpItDm/DNggVYYE3f bJLfaEstWvQ/owej1z89h2f+9jc888poXGp88B5c+Cn+V/Jr8yFszcpBfoe+mPjaa7jjPKBZyu9w lnWl9OSz+plzt9GsO+8cai35O/fc8o3FVSeVNaJVEQbGcYiMhna7/osLbnnMXK5j/AzJLytq0u3Z 3wS3/OFynICdSHvxzbL3ZUX02dMdKecYvzUe+g4rlwGrNqwDzjbqOgDbMn9ATs5KZMitp2efg183 W4tF/1uNQiOI9R3zEp742xN4afIwI7YVYtPHX8DYvMSROr/G6Blz8Z93n8Mw7TfJNo3WYtxf38am eudh5NMPoU9Lq8K0BxtX56Lx+SPxzNuPYWAzq5iI4lrEN8O3vGAEnv3PB3hh7DVG4CrC1o+fxGOz s4HtP2GHNNj2b9xtDoP3wgNp8hvhbux2He93UoSi49ain/qob30ANatf35wfys83547aDMbzWtB6 frA1DGXKxk/b5RpoG3TqbHVa92ScZHw2A7uwZ0dd9Bn5MAafdyKOZn+Ml8f9EQOvuafMCWBgS9+6 02hWICtW+H3KUgCVPaJ131Rex/HK5hZXW0uA+j6e0y8tefkHUffiP+DW843fhLa8ixff+tmqsUT0 2VMX56d0Mea7sSrrA6z87hCad+iIi5LOBlZnYdWqdVht1LZJ6Yq22I6fzB2chY7ny9xwZgtfeNu9 2/9yZkp3W4jyWfavl7FCPnLQDC3aWrdaGK/6pgfvxaWn18EvX7+NSXddi/7DJ5uXFIko/kUWtHJ/ xI95stAQZ/cehWf/5PvU2bBhPXDaKThVVpJuwv+98Qbe0KZRF5rNAtr+01rfwp4NyN7mW/SXj3zr Ey17l++DtoEVuMJ3Jk45Ta6BbsG6tVanR37GTvPqYgs0kxfS8gKMeOrfSHt/Ku5OPQmFe1di7H93 maGq5cDSk38492aF++29ijC1Co8YRcMzd4X2hQEqP/lZkp8v9TO1oqX+JYyWuPqPN+LsWoVY/c50 fKUPUEf42dPsku6QPW1duwRbd9fCr84+H23P7IDm2IYlS2Q4qw0u6CER8DScYu5gDdar4avsXTA/ hZo3N6JTCFr3xW39T0etg59hymPzSu7tqtvxajz02lx88K+/4YazG+DI1o8xc671WUhEcS2yoLXi dfxxYF9cc+t9eOSR+3DLC/KpY3xAJSUBnS/CJWcb4SXrHTz1yqdYlpGBZZ/OxsuzvsaehrLxaTi1 lcy/xTv3PYJHxr2BxcbaqSeeZPQAbEp7BuMeeQT33f0qnMeHVuDVu337fWTWBmP9BFzQvbuvKgIX XdEXJxnzFa/ejftkv7c+Bblf9oSLr8AlxifnZ08Oxp2PTsOH6Zuxv1ZN8xibNWtmngRqfv1swHuz zj36pbXkryIfcxCqu6r55SuKDfJzpIcscW6u7ReTM3+PP1/fBji0BVv0Wy4j/OxBs1/jnLOBwpUr sQod0b6TUXb+2fhVrW1YtswIWs3PQRdzqK0z+vY51/gM2I25k283PqfG4fYxM7DBKDn9yt9BDXIF cur5v8ONd4/DTcZxHkx/Fo+aj6vIxJt33or7np+O/2XuRGFN+ViuhebN5JOJiOJdZEHrpPZIalkT +dkrsGDBCuyo2Q4X/OFhjLuxrVF5Jn4/aSruvrQ19i1/Gy89+yxe+k86th5ORD1z4xQMHHwpTqx7 BFtXLMAS44PlqFHarM+t+Mulp6P+4WwsXfw19nYcibuvMq/h+Wt5Oa5N3ouvF6/AVpyI8/74KO69 WA3Bh6/u+X/GUxMH47xGO7FiwQKs2FkHZ156L56+/3LzN9R69Wthy1e+1/F6+n5MMgLJmD/4RjeK fnuPOXe7f2hFnQux94t/WGtV033j/m4txbYPct1HHDsmbrWWygr1GWO6c881fuGgiKn7tAJJvunP 6F8mh0T22QO0RVf5ZkthIQpbJeNX8jGGTmjf0axEg25dS0JUy+smYurdl6L1kY1YumApNh5pgfMG P4wn/hjOQ0iN4xx/D1JPkMdVPIFpGcbxNdiNtXNex7PPvoT/rDdaXDkao/qENEZGRDEujp4ML493 eArpSMXoBY+ij1VaGey/cSvBTg5VnbrkE0yo7SpKrB0PlXL6OSMiiicR3wxfneknZpmrScqr84nB Kaw4vR9O7SqTOh6e1CuP/t7rP0ex9t8KEVG4GLTKwekkEOmJoaqe5OPpRGk/Vv2EX1n0/bstVwX6 e68vV7XXSUTVD/+odBTYTwyyrubVVVV4/eG+BhUKyvu6Q9lvPL+/ob4+IqKqgCNaUeR2AomXk0Y0 j1Peh1D6i+X3JtwgI+1D2SbYaw6lD7c28fDfWijHHsp7QEQUDxi0IiAnAacTgV6mluXk4XbSiLWT ottxRiqU/qK9z1gk/85qEvKanf7tQ/3vIVC7SN/PUPcdbWq/Mnd7X4iI4hmDVgTkZKCfEAKdHAKd PIKdFL046Tj16cV+KlNFvW92qo29rfw72/+tA/3bB9tXsP9ulED96HWyHGqf0eb2vgR7D4iI4gWD VpgiOSFFehLz4uTn1Ge4+/HiJFgZJ1a1T32uJjv9PXKqF/Y2el8yt7/Per3i1C6a1P70fYS7P/sx hyLQNvbXHEn/RESxikErTPpJwOnk4HSSCHbiiLcTSzgn5lBfm9uJNpL3RvUVbFvVTp+rSXHqQ+ql XE1O9L6kjepTtVdlqtxO79dtORi1b6HmTvuztwlG7yPYNm77ddpOykI9BiKieMFvHUaJOpHIicJp 2S5QnVcqY586L98Pp+2lTEi5vizs60qwfnT2tuGuC7e+hWrvtF0gqk9Fbav3Y19WwtlPuNz2o8ql TG9DRFQVMGhFgdPJK1JufUSjb12k/QXbLpR+pY0I53UG6lf1pwvnGPS+A/XlVCdUHzq933D61/ty 2tbej729UOs6vZ0Tta2dvk+dKtf7VOv6PBindnqfRETxjkGrnPSTjb5s53TScTvJOLVVVH2wdrpQ +ywvez9O60LKwj0mfT3Qsk7KnfpR9Hp7P6pM6MvCrU4ty1zo2wi93l6nC7Z9uNy2C9SfqrO3cSp3 ayukLBT27YiIqgoGrSgL5USk1yl6nWJfj4TaTyj9BNtfOH0Jvb9QX0u47ez7UFSdfTkcTtuF0pd9 30LWQ+kvlP5DFc2+wqVet05/D/Rjq8zjJCLyGoNWOTmdJOxlsi5COcGoMn0unLa109sLe1t7vXDr z62tvX9ZdyoXqi4Ypz4UfXu9P7dlxanMTThtKXx8f4moOmPQirJAJxVVZ28j6zrVRi0rTmWKWx96 Xzp7verTvo3bcjD2/hT7ui5QHRERUTxi0IphKnjoc+EURvQ6PbC4LQu1jbBvp9Pb2dm3U22d+tG5 7YuIiKgqSSg2WMtUxaSnpyM1NdVa81Fl9jpZF/ayQG3sfdnb6wLVERERVVUMWuQXoPRlIiIiKh8G LSIiIiKP8E/wEBEREXmEQYuIiIjIIwxaRERERB5h0CIiIiLyCIMWERERkUcYtIiIiIg8wqBFRERE 5BEGLSIiIiKPMGgREREReYRBi4iIiMgjDFpEREREHmHQIiIiIvJIyH9U+j8fpOHHTZusNSIiIiIK JuSg9fSU53DH7SOsNSIiIiIKhpcOiYiIiDzCoEVERETkEQYtIiIiIo8waBERERF5hEGLiIiIyCMM WkREREQeYdAiIiIi8giDFhEREZFHGLSIiIiIPMKgRUREROQRBi0iIiIijzBoEREREXmEQYuIiIjI IwxaRERERB5h0CIiIiLySDUKWpmYdlMv9OqlT+Mxz6qtyjKn3YSbpmVaa9Ezb3wvjK8ObyAREVGE qtmIVhsMfn4BFiywptHAU71uggcZhIiIiKiaXzrs8yieHwy8PdvbYZl546vHyBkRERH5q/b3aCX3 uABt0pf6glDmNNw0fp55qU0uLfout9kuOd40zSixzBtfWq4m27U0ubz2VHo6njLrVeCah/EBtilh 9H/TtHna/mX0zf949E1lXyV9ul0Wldeobadeqznpr83Gr53e91L9PdBHB93ft3njjXbT3LazvTfm pPbn32fp5VD7ZWEGWyIiig28GT65FdpYi6b0p/AURpuXFt8ckWwUJGPEm6WXG0e3eRu+ATDj5D57 S+mlyOcHo02qsd2jfaSyRJ9Hn8fgNqkYbW7/KPqYoeApGAVWn0b9lqdc76Ha8vbs0raj2+DtUdq2 o1ORPrs0wPR51Co3pucHb8Fse58SDEdtxSCj3jxMY33U1kEl2ywYtBWjnEKftPvqAjyv2pmvwyd9 S+uScv/RQbf3TWzB2185b5c5bTa2DH7e2k69d779zRs/ClsHlfY5aOsoX2CcNxtfXaC2kan0+IiI iCoTg1bmNuO0r2kzGKPNgKX4j5Y8lW4VO9myrST0uMpcgq8wGINKkoARSAalYstWv6MolToIJYfT pjXatLkAPfR1a1HoI1qj3rb199VT6GVmtNIQMm+p8WKMYKm26SUvzuE1SLvUQSOMIy1LLzdHB63l YO+b+3Z2W7DNPKB58B2uf59bpLJPd7R5e5RRxpEsIiKKLdU+aGUu+QpbUru7joDIKIo+WjI61aow A5KMMFkn/lFf4YLRzmGkIsilPTUSJ9Pzg/2jy5Y2g3w3/9tGrFJLRtas6c3ovAb39y2w5BGDrNAk 76v0Mbo0aEKNDJZOvlHHPnjUXO+OpeZ2DFxERBQbqnXQknAy6m1gcOnwkk0mtm0B2rRSZ3rfqIqS aVSWBpU3tUBgp0ZlDMk9cAH0y2hyCTIdbVq7j+mEQkbESvvIxJKv/Ee0zLo+jxpRrPQyZZ/uqUh/ KngoMdtplyiDC/y+BSQjjHIJ1i9IiT7onppeJij68wWu0ana+01ERFSJqlnQ2lI6AmVMo95ug9EB A1IyRowejC0ll6uWorU2UiSXvErrZHIKLckwmln7lXq5d8l8roS1zSi83Wa0Figi0+fR0dpI0FPA Bc5DSHLP2AVfjfKFLfNbl1usG/V9k+O9YtLugq8wqqRdsHAW+H0LSILoFu1ypjGpbGXe7+ZXZ91E b/tSwlPQLrcSERFVooRig7Uc0NNTnsMdt4+w1sg8uc9ujee1S23q8l15Q1P1Jd84nI3Wz2vhV74l KfeWRemSJhERUUXizfDl0aaV38nfvHxXcrmMItMGfm/hlq3YYnufiYiI4gVHtCIm36obBf3LfW0G P8/RrHLy3Tfn96b6jRoSERHFEwYtIiIiIo/w0iERERGRRxi0iIiIiDzCoEVERETkEQYtIiIiIo8w aBERERF5hEGLiIiIyCMMWkREREQeYdAiIiIi8giDFhEREZFHGLSIiIiIPMKgRUREROQRBi0iIiIi jzBoEREREXkkodhgLQf0nw/S8OOmTdYaEREREQUTctAiIiIiovDw0iERERGRRxi0iIiIiDzCoEVE RETkEQYtIiIiIo8waBERERF5hEGLiIiIyCMMWkREREQeYdAiIiIi8giDFhEREZFHGLSIiIiIPMKg RUREROQRBi0iIiIijzBoEREREXmEQYuIiIjIIwxaRERERB5h0CIiIiLyCIMWERERkUcYtIiIiIg8 wqBFRERE5BEGLSIiIiKPMGgREREReYRBi4iIiMgjDFpEREREHmHQIiIiIvIIgxYRERGRRxi0iIiI iDzCoEVERETkEQYtIiIiIo8waBERERF5hEGLiIiIyAPfr9vMoEVERETkFQYtIiIiIo8waBERERF5 hEGLiIiIyCMMWkREREQeYdAiyxyM+e1v8VuZxsyxyoiIKt6cMb9FvH0MxeMxu5PzwRjj/1M0MGjF oPB+YDPwwqDy/0DMGfM4coZPw9dff42vJ/ezSoko7swZY/zCFIWTZLT6iSr5vBuEFzKs1SjJeGEQ fjvoBaN3L8gxO/8CG85nvRxj1Qly1QuDVtzLQc5ma7Gc2rZOsZaIKD4ZJ/WZOWjXbhHSy3tS7jfZ +MVrMqrDr10pI2fj69kj4d0nYDu0y3m8XEEpJ1of9FThGLRiXcYLGGT8dMpvPuZlPZnUb15S99vH scj43+Nmnfrt0/oNyulSoPFb6iDj10HV36AXXjAvGT6+CFj0uJRZvy2av8269GEwfwMsqdd+w/Tb Tv/N03ZMxsTfzoiiLCMdX+JCjBs3HDkzbSM09p9pv88RrVx9jki5Nsrj/zNvTVb9nDHGz/oc/378 f74DfCYZ/Po2+txqlYfLfox+uynzOmXSPu/0xra28pnpK/bvX5UH1xZD5d/k8SAjhI6fn773rvQz 2nhdxue2/m+jjld/CX6jZbZ/e7/jNur8zwkOr8ncXh2PdpuJOemf8+SomGJO2ujfFI9Os1ZW/qP4 ht9o68Uri/9xw2+Kb/jHSmtV6kcXl1QbZPvflHZgti9ZTRtd/Bujv5LtLX77FCtXGlsqacWj9WMw +/Dfp8k8lhuKS7qWdjf8w+xn5T9uKLNPIoqu0p9j+bnXfhbNn2GHn9ky7TTy82z9/Jb5nNHrDOZn jrZu/4wI/plk69vvM08X4Hgd+1Ftbfu09yPblh6Q/+edTv9c9Ovf9xqDHbN8Dpbux7aNrT/zmEre U/vxyzGWtjU/X419+L8G672w92t/feb7Zj8n2La3/VtrL4GCWLF2UzFHtOLCRUgtGb9PQeqF7bA5 J8dat5uD9EXtMHyo2iAFI4dehEX6dYR2wzFuZJBB8pQUbRi9H1IvshYNc9IXGV0MLXNJISP9S2y+ aChKuu43FMON36/Tjd92Ulq3xebpMwP/NkdE5SA/++qzQn7u22L6TPUT1xZt2y3CzDJDDylo3Xaz 1i4Mm3Ogfwq1uzC19DOjX6rxqZWDrdYISKDPpDKfJympMD7iwla2n5EYNxz4Uj6AHG2G48fonHQs Mj4jSw5Xp38uRnCcKSPHYbjLJcRAn59lyWeyOv4MGJviwqEXol3OVt8ol/kajH9zqbX3a2w7+cEQ zwkyUjYiB0O1y6pt27bDIvtoKQXEoBUPrB+YkGRsNT7ejA/OEdrQrow5qx9A0bZ16YeFG9vQuXTh k4Gtxg+30/1c5j0Eix4v3e9vR2D6ZuvDwLzfIxXpVp3TBw0RlYOcXEtuIzAm81pTuvXLjRFuZn+N cXjCV6ddduo3+Wt8nZpu/cy6XNoyQ0tOad8jpqPtg/73b7ne4xnwM8np80TCn7UYMvfPJd8vpcbr Ny/dqWMwPpvaPgin7/1k+Dpy/oz0uwQnn29WecjUcZR9nwN+fjroZ/z2a4Yl63JxqhFuL7SCmbwG FXyl33ZtHd7QoOeEHMx8Yjo2284/5v1s44AnzGPkZcNQMGhVNSmtjR+Ki/CgfHtQn8K50dP8Lcb4 0Z1Wur3xC5DF9yGY4/tV1Y/8ptNOfXNRm0o/zIzfpMyyaWg7k2GLKHp8N8EP135mZXrwIv+b4s2T pNQNzcEI/R4f8xcho3xaW8x0CVtywr7owdK+Q/5ycsDPJPfPk/C491MSMozEsvmiB0v37/ICZPTd L4QoErIeh/Y6pmF4BCNvJaHV9gEY/PPTxnhdMoI1xxzOklAlVztkBG+Ob4Qr1feJL/06XgEJ4Rfu C8cZrxHTMcL+YW28htnm8Q1FzgiGrWAYtKoENUQvZEjZ+K223CmmLUp+OTSC18ySES3fb1JOlwFT 5Kd8+hMh/NBF8hsrEblSoxq2M6c56uF0mUdO0taiHzMUOZERo3ayWQQCfyaV+Tyxfd6EyqmfJ6aX Bg5zlCeUFyCXPTdPh+PVVH10Z87MCEa0fNQlxJnGe6qE8vnpFyTl0qXxrz5TC1VmH1/OxJebSz+/ pazdoplav3Mw5vFFuKj0fhQX0oeMhD6Iixa5fWNSLklbi+SKQSveGb9ZDL1IDcv7fhOVSwEPQh+C DufbMQazT+0SxBPAUP1XN+O332n6ZQQ1fCy/5UwzfvT1SwTWMdm/rfO4cYQh/0ZMRAHNmWkmirIj FOZ9PhIabN8UkxHrcTKiZP82sAzZOD3SQe6raut/+S+My0YBP5Psnyf2z5sybJchjckMATIq9yBs lzdnl9yblDJyKNpOH+G3nfPnYj9MnqZfZrTaWe/lCLV9eqo20h8u3yVEbNaSWoDPT9/7L0HSd/y+ 0CO/sG42utB+KTbDl1F2UWrpv2GZfn3PTAz987f0/ZD9+n0D/rcj8OWF47T7v8hJgtwVby0TERGV Ib8ojcgZ6n+5TS6lGWEjLh5wbN4OkYOhfs8Fk/CZjtRq8qwwqhzfr9vMES0iIgrMvD/LdqlJvuUX N8z7s7RRHmF+eYDIexzRIiKiIGT0Rx6OrJEby+Po+r9c8ir99rSQG/Sr12hWeno6UlNTyyyTd2RE i0GLiIioinIKVAxZFYeXDomIiKogCVNCApVaVvSQZa+j6OOIFhERURUSyYgVR7m8wREtIiKiKkQP TKGMVqk2DFneYdAiIiKKc06BySk82cOX3iaUYEbhY9AiIiKqAuxBySk4SbByC1QqdDFwRReDFhER URyTYGQfvXIqU4KNYrltR5Fh0CIiIopjKhjJXAUnt7Ck6u3tnAIXRQeDFhERURUR6miUHsoEA5d3 ov54h8OHD1tLRERE5KXvvvsO3bp1s9Yio/cRjf6qg3r16llLgfHxDkRERHHMLRRJYNLp6/ZlvQ+G rOhj0CIiIopjTiHKHpjcAhSDlfcYtIiIiOJUqCNSehijisWgRUREFKf0YBVqmHLahkHMOwxaRERE cUyFJAlQboHJLZCpcr2eootBi4iIKI7pIcktUClSZg9VbuGMooNBi4iIKM45hSWnUSops7e1lzF4 RReDFhERUZxzClVChaZg4Unf3q0vigwfWEpERFQFqDDFoOS9cB5YWmlBa8aMGdYSERERRdvZZ5+N 1atXW2u+dSFl9mW9XXU2bNgwaymwuAlaob4gIiIiCo+McDmNbqlyt/rqKpxcwj/BQ0REVM3pIUq/ R4shq2IxaBEREVVx9lDFkFVxGLSIiIiIPMKgRUREROQRBi0iIiIijzBoEREREXmEQYuIiIjIIwxa RERERB5h0CIiIiLyCJ8MT0RUDt9//721RESx6JxzzrGWAvPqyfAMWkRE5SBBq0ePHtYaEcWSJUuW VHrQ4qVDIiIiIo8waBERERF5hEGLiIiIyCMMWkREREQeYdAiIiIi8giDFhFRFZOenm4tuZM2wdrZ 60PZhoj8MWgREcWxQMHHrU4vt7eRdTWlpqaWlAlZV2XRZj8OoqqCQYuIKA4ECyIqHAVqZ68LFJz0 crVs315fV8uB9h+Ivj/pQ01qXc31cqEvE8UiPrCUiKgcKuqBpRIoJIyouSoTqjxSen/2vpz6DtZe qDK9Xi07tVFUvarTOZXr25L35AGg0VBRD/nlA0uJiMiVPVSodbfySNhDj84t8EiZKldzvR+ZAvXr VCZUub69motAfbn1SdEnIak8U3XDoEVEFMMkQAQKGOVl799tORjVVg9YekgSgfajrwu1rub2vhSn IObUH1Ud/6pZ25zy8vJQWFholcYuBi0iohgUr0FBP+5ovga3vqRcJj3gCVmP5v4pNkjAuvlmmNPs xk2xb9++mA9bDFpEVG18+eWX1lLsUsEhFrmNKsUK++hW0ONNG46EhOFIk3nXyciyiktkTUbXhK6Y bFRkTe6KrrIQgLRx6scsl/1Y6yVkv8PLlJZlHqfvOPylYXhCAvy60I65qhndt7a1BOzL9c137dqF goIC30qMYtAiomollsOWPWDFWrCJ5REip4Cq1p2POwuTJ6zCpMzp6N//WgzLmIW5tnCSNXcWMoZN xJgkqyCgNEwaC6SgbD8iJWUVBoQSqsowwtQEYFiKtaqYgWoCVtnLk8Zg5YddMHaoQ3D0wOPbbse8 vTPxz9zxmLLjPjy27Tbcn3Mj7tp0Ne748QrcnH0BRmRfarWOnISsAVd1RpOHW+KNN4APPwE6LVpg 1cY2Bi0iqnYqI2wlfZZgLflTAcEpDEhZecJWz549raXqS72HZd7ftEkYiyHoa4ao/hg7CZjll5Cy MHdWBoZd299aDyLtfcwwQtnKiUbImVQ2UHWZOBOTVg3wH30KygiDXScgeeZYJFslPkb50FkYkrkS xu7KcgmOXlidvxwFx/ORV3QAO47mIPfoVuQe24qdx7ZjR36OWVewpI7VOjIqZOXu2meur+3tC1lN mjRBixYtkJiYaJbHKgYtIqqWKjJsSchafbl72FJhyh6qHANCGBYvXmwtVQ9uoVS9hzJXy2nvz0DK kL5Qg1VJfYdA0lZJDsqai1kZwxBazrJGx8YajSXkzHi/zGXCVeuAMTMnYdUAh0uILrImD8WsITMd RtSSMGblygAjbf1x7bAMW3D0RmKNBth8ZK055RXtN0LWFtRNSMT+nw+iYd1GOPBzHmrXiDxo2UPW smW56P+AL2S1bNnSnNeqVcusi1UMWkQUNyQclWeycyqLNhWyhB621ElfD1NqrtjXKfDlVLf3S99G X+7SSUsqSX0xJGUG3rdSkFw2lGuBoeUsI5T5jY6twgSnm6SSxmCmURfSJcSsyRg6awhmhnbdsowz k1OQkZltrXmnW8OLcO+pz+CHL9fipfaf471OmXju9DmY13MD3urwDdJSjfchwqRRFUKWYNAiIvKI HrIUFbbUCZ9hKjzhvl96kC0NWVlYtwoY0Vy/ZJuEvkNSMMNMWr77rYb4klNQ5r1cGWORnJCABGNK HpuBjFlzjb2UlTQmlEuIaRienImJK8eUjLjFqvmbfC/kyWteNOe5R7dg4MIUFBUXoecbbVE7oQ6a pYT2cE+dhKz7n34u7kOWYNAiorhz4YUXRjTZOZVFi1PIUvSRLQpNampTayk8EqQkYKnApQe1abv9 /wRRUt8hSJHLfnK/VYoaoQpGQlkXfFhcDPlDK74pE5NcboqXQBf0EqLs3/jfACu4JSQkY2xGBsYm JwT95mNF27+hAJn53yC1UR/0eLk1WtZpg9qNaqJmQk0knuwLQscLI/8DNBdfd39chyzBoEVE1VJl hSwlWD35S0/fay2FToUoPWD5ypLQqYvcN2ULLUljMHHYDEyYsMrv/q2AzJvgr7VdYpTRMbnlyyVK qUuIsh+ryE//6Vpos4JbSgomZRZjZYiXErMzM5CSfKa15p3GneuZ3zYc+WNv1D+ttvlNw6P7ijB8 Q3fAyFe3ZPfEgTVHrNahW5jbFX//693mFM8hSzBoEVG143nIam4sfBd4Wpy+GD3r8VuB0aCPSun8 w1UpKe9/7TD8o8feMpf3pDwjw/myYcbYZGuEyZqGT8bkCTMcv5mYNGYihs2Y4Po8K/MSIjKM/3kh De/PSAn50md57N95ALUSaiN7YzYadUjEhnXZqNOoJrauykVii1rIWb0de7cetFqHbsGiBbht6hoM ffxb85uF8RqyBP+oNBHFDXXzeqRBSbaPdsiK9I9K65ewyFsStOT99g9cWUbZXixpmooee8v3GI2Y Iw84nZCMTA/u8ZI/0qz/937v5oHIyv4BNeokoLiwGAk1EnyXCo3/U5cMzz3r15jU7h1zWdj7cCJP e5cHkcpcwpU8wiGSkMU/Kk1EVIG8HMmKhip1so8hKtT6h9skpO6ehrHJw821KhN85UGmA1Zh0syK uZF+SrsP8OnvsvHxhRvwySXG/OIN5vqnl2Xj8ys3mpMeskIloaphw4bmKJbM43EkS2HQIiKqYG4n 9Spzso9RZS4hNh+B4uLp5nKVCbnyZPjiQM/YoorGoEVEVMHUSV0/uTuVUeic3stAJNTyPaeKwHu0 iChulPceLS9Eco8WR64qjoQoPVSpZX0uGLZCI/c8RUO4PzORioV7tBi0iChuVJWgJewnemFfp9AE et+c6lSosr/3VPXwZngiompITvAqAOgneLewQGWF+r6p91hNqkyn90UUbQxaREQVTJ3YZc5wFRm3 901/b9Wy3laVqzK1zH8H8gqDFhFRJYjFk7sKJvFKD1Dq/dWDlf39Vq9XtSHyAoMWEVElUCd3/WRf 2exBJN64BSkVuHSx8H5T9cCgRURUiVQIiPeQE02RhiD7dk7Bi+8zVTQGLSKiSuR28o80bFQFkYQh 9X45vW96mdv7TeQVPt6BiOJGrD7eoTyOHz+OGjVqmHPFvh5vQj3+UNrpbWRZCfR+qXZO2wkpt5dR 1cXnaBERhSgWg1a0fffdd+a8W7duJcuVQd9/OMcS6XE7bWc/Bp29XNZVe71M2LclcsLnaBERVXGB gkFFhgW1L3tgUQIdiwo6arJz21bfh2rj9H5Imb6PQEJpQ+Q1Bi0iohgRLBhEMzTo+1LLal2FGT3o 6JMedGSytxH27RW1rdDLnTjVS5lbuT4nihUMWkREMUIFEycquJQnSOhhJNC+pF5NQrVVc1Uu3I7J 3ofbXFFt9fpAx0gULxi0iIhihFv40JfDDR/69k7s5U7rgfpwKnPjtr1bH4HqiOIFgxYRUYzTw4Ye PtSyU71ebq/X50TkLQYtIqI4ZA9Kss7wRBR7GLSIiIiIPMKgRUREROQRBi0iIiIij/DJ8EQUN6ri n+AhIm/xT/AQEYUoVoNWjx49rDUiiiVLliyp9KDFS4dEREREHmHQIiIiIvIIgxYRERGRRxi0iIiI iDwSczfDP/biW9YSEcWLh/78e2vJW7wZnojCEQs3w8dk0Bp9y/XWGhHFuqdee5dBi0GLKCbxW4dE REREVRiDFhEREZFHeOmQiMqFlw556bAqkUtN5L2K+pnhPVoMWkRxj0GLQasqkRNzef89VViL1X7+ 97+lWL++ARITD6BNm0Rccsl5Zrnuu+8y8e23x802J59cB1de+Zugx7PvwEH8PPQmFB0pwGnvvIXG zZpZNf6i8R6HSvbFe7SIiIioQqxevRG7dyeiV6/W6NatAwoKGmDp0jVWrc+mTduxbRtK2hQWNizT xq7QmAqWLMOR7B9RuH0HDn33PaI6iqP5V83a5pSXl2ccm+w5tjFoEVG1oUbEqp0v70erxES0ur+a vn4y7d69Dxs27EXLlieiU6emaN/+JJxySgssX77RalHaplmzJiVtWrU60a+Nk6MHDuLQF/NRfOwo ahr/rR3+Mh2HDx60aqNHAtbNN8OcZjduin379sV82GLQIqJqpVqGrfpN0LplS5zVqrlV4G75qyMw 4tXl1loM2ToXj4x4BHO3WusUtvXrt+HIkRNwwQWtUFRUjNq1i3HOOSdi+/a9RsDaj82bd2DFirVG eKnt2MbN8eJiHP5xE/IXfYn6v0pGwx7n4+jir3AwZwuieXfS6L61rSVgX65vvmvXLhQUFPhWYhSD FhFVO9UubJ13P5Zu3ozP70yyClwc+RyvjXkFr3y53SqIHT/OeRITXvkPsg5YBXHos0Vf4ZXFyzHz +1V4Z2Um3l6xCjO+WYFXl3yD5xcvw1tpH1kt3T2+7XbM2zsT/8wdjyk77sNj227D/Tk34q5NV+OO H6/AzdkXYET2pVZrf+vWJaBPn9NRwzjz79tXYISu42Z5kyb18Nxzs9Cr12148cXPXNu4KSosxP7/ /Q9F27fjtNH34sShg3Fk3Toc/vpbFBYVWa3KR0LWgKs6o8nDLfHGG8CHnwCdFi2wamMbgxYRVUsV HbaSPkuwliL0czomD+yIJokJSEhIRJOOAzE5/Wej4gg+v6OZWXbNO2rUwVaWNtxYNrYbnmbWZk3u aqxfgpdXfY77zmuCxISumLzsf7j/1wPxonQxY4DWPguTuxrLl7yAz18eiDMayv4b4oyBL2PFipcx 8IyGZtuGJ12Gx5ep/f+Mz+/rjpPMtvo0HGkR9Jf12lW48L6vjKUMjE2W9sbxZplV3vvuPBw9etRc XPt+sjnHpod8c4OUmfVGu2B+KjyGZceOYUfN2ihu1BDNW56Elq1PQ7u2bXD6Gadj84Hgl9pW5y9H wfF85BUdwI6jOcg9uhW5x7Zi57Ht2JGfY9YVLKljtfYnh1m3bi1zuUGD2qhduwbk6l5BQSHeffcL TJgwAkuXfov33vvMsY2bg9t3oGD+AjTrdTE21khA5sEDaNzjfBxO+y/ycq2hp3JQISt31z5zfW1v X8hq0qQJWrRogcTERLM8VjFoEVHckZAUyWTnVOYFCVmrLy9P2PoRLw/+HcZ+uAUNfzMYtw7+DRpu +RBjf3cVns2qi8uu+z2aGuFq3r8/hRlNjnyCGa8ZS01vwZ8HNjV7KGsJ7utxNaasrYOWbS9B104t 0OvKrr6qC8YYJ9538e7tXXzrYsGduHx8Ls4fMhjnn3gImz68Hd263YeNvxqCweefiEO/fIGHRr2B dUbTra8Nx9VTluHUO77AmjX/xR3tpYNL8H/ZL6K/LIow+qvf+QpceJJs1BqD/884rncnofepsl4x 6mx/xJzXb+EcplR9MIeOFeKXI4fx7LpsjPphFYYuX4Gh36zA8CVLcfUXi5GfEPyUnFijATYfWWtO eUX7jZC1BXWNQL3/54NoWLcRDvych9o1nIOW2LLlIHJyDpiBa+nSDLz55n+xfPkPWLcuBzfffDV6 9z7PKPsIn3/+ndkmP/8ofvnlKIqKfCNbdkeLinDom+U4vmUral18Id6Z+xE+mD8fxT26o3CNcYyL FuNYOa4e2kPWsmW56P+AL2S1bNnSnNeq5QuPsSoug1bdunXNaf/+/fjpp5/w0PhxeP75qXj5pZex c+dOs46IKBaokCUiDlvr0vDqgiNGzrgP7305C9NmGcHxmQuMQLUcz85aZmSYm/AXI8wcmfdvfGrk qyOfvI9/S/M/D8dlrh+HR1B41iPI2L0TmzdPweVNu+CKHmf4qs7ogeuuuw7X9Tjdt25qij/PXIhZ 02bhjQd8gaPusJn4Nm0aZi18EYOkYPn3WGvMNmV9b/QOdLngUnTufBUuN7/J/wM2b28gC5bQ+zu9 hxEEzScFNEPXS4zjuu4KdHHLj17IWwX8/BbqtegG7P0COJ5vpMlnzNGszucOAPZ8YjUM7NCxY8Yr qIOfjhRgd2ECdh0vxi/HC/FTYRGw4xDyQjgjd2t4Ee499Rn88OVavNT+c7zXKRPPnT4H83puwFsd vkFaalbAM/v69fvw9dc/4ejRQjz77Nt45plpyMpahXbtTjHrX399gvHfw3Y89th0s83OnYeM8+wh 1KlT06y3yz94EAXzPkHdZs1wsHNHPD91Kt7597+xt+OZqNm4EQ6982/k5x2K6BuIVSFkibgd0ZJA JTfBjbj1Vtz6h9twyy1/wrDhw/Dev9/D1q28W5KoKpPnaEUy2TmVRZMespSIwlbOephfrr/4Nzjf LDCyUCtfKNq6VS4fdsPweyR4fYC35/6MT97/txF02uOWa1RrZ52v74suIf9e2gbtTvU17nTGWea8 c/KZMEvq1jLiQ6lf/TrVLF/y2UdYu/YjfCaPX6p7DpL03BZGf5WqWW/fPG8FTjz1LPyyw/eYg42Z n/mHLNUugG25uXjzP0YwXpQDfLIOmGv0Nc+Y/2+zEaZ3YcdBI8AFMX+T7/Lvk9e8aM5zj27BwIUp KCouQs832qJ2Qh00S3G/n6pTpybYv/+IEab2Y/78pViw4OWSSZHlb7/9zmwjN8Tn5x8z53YyxpWf tQaHPv8cjXtfjiIjbOXn55vP0zze6jQ0vux3KMhcjcPpi1F03HlEzI2ErPuffi7uQ5aIy6ClQtad I0fi+f/7B6a99k+kpqZi0uTHccufbsF7771ntSQiclYZIUsJO2y17Qgziiz8BsZp2vTjth/NeevW J5vzM64cDCNqYc5Ho/G+DGddcA+GdzOrKlzTG6fg5b4nYOM/r8ZZZ12HmTUGYNIX03FLa6tBPDn9 MaDJRb7lwxtwYlPfyb19KyMS6iNZ0i6Iqbf+AcWv/LVkQtIZpetv3If3bvuD1dLd/g0FyMz/BqmN +qDHy63Rsk4b1G5UEzUTaiLxZN+xHS90Hz86+eT6OO+8k9GwYV107twBEye+UjIpsnzeeUlmm8OH i7BnzxEcP172v9fDhw4hf9Y7qNm4MQ51+RV+yMoyAlkRjh09iswffkDR73qhZr16OPjRPBwJ4f4z Jxdfd39chywRl0FLErNcMhQJxv9andrOSN6b8cnHXxj/8IdRs4bzECcRlfV+8xPN6ZDxoRkPD/+L hsoMWUpYYavTENzVzzixb30G1104BCOGXOi7ObzuebhniDVqdcYQ3DnQaDN7BmYYOavXsH6wLgSG 7vTOML+X+J/x6D9iBIY8H+E9bF8+j9FzDyLljtfx7rtv4dUnhuKMo79gr1xPjMiZaN9Z5mvxf382 Xv+IsXhPbt6qSIOeNYLVUaDfHcBPBcAf/gtszANu+y6k0axoady5nvltw5E/9kb902qb3zQ8uq8I wzd0h1yfuyW7Jw6scX+j5eb2E06oY4SVRFxxRS+sWrUTS5euxxtvGK/HIPOvvsrA5ZdfbLbJzy/E 7t0HUVBwzKxXZHwqzwhThxcsxAkX9kTuCQ2w5Cv5woLcOF+Apenp2NOiORpdfBEOL1mGQ+vWG2Et 9FGthbld8fe/3m1O8RyyRFwGrWPHjplhS50Ucn/Kxf33329+UyXvUB5+3uULYaXWYmrPnpgqF/sr yrzb0fT2edYKUWySgKUe/vdJm3bmfY9VPWxVSMiSx1UZ599gk7QLLWydjN/PWonZtycj75u38crb 3yCvjYwSfYR7Sp7Y0BQDb7rBuvQ2ECNuiGD4qMsf8fjtp6PBoR8w55VZ+GJTZM8nOnRSF1x0MpDx zz/g+uuv902X/gqnDH7Hd7N+2OriqnFTcFFjYPvitzH93yuwy6qpEK3vA+bOQF69Osib/U/sbZiI X567CrlNGwDvPhjSaJZdwnP/8ZuHav/OA6iVUBvZG7PRqEMiNqzLRp1GNbF1VS4SW9RCzurt2LvV efRIfrbz8gqxffshNGhQC7169cSAATegX7/eRviqD3mO1m23PYmrr74EPY1zprQ5cuSYEbR+Mbb2 HyWTUau8eZ+gcO9+NL/hOuwzzss//vgjatb0DXRs3rQJ+w4ewIl/uhnHd+3GwYWLcOyI79uboViw aAFum7oGQx//1vxmYbyGLBG392gpxcb/Mlf/gPy8/LDScoVbOxU9mzZFU3Oq4NBH5ODxoSdaS6UP /9uzZ4/xwRrxsEPM8zpkiazLjU+lbqFP0j4kDTrjhheXY1+BsV1xAfat/wBjUn2XDZW6/aejoNio L/gPbtRvFjfK5cGRxdN93/lLGrPSXF85xv5crZPR/8UfkSdti/Owc4oMyyVhzEpZX4mS5lZ/pdv3 x3Rzm+nov/cdDOt6M+ZfPBtbzWM1poIsTD5PbiGbhy/D7c8qqdvlHizcV2C2K9j3OW7vZFVUlJN/ j4bt/2xOTbtOwYmp/0XLy4y0HEHIEsV3X1MyhaNLcgpyNm1BjboJyN9+FLXq1cDRA0Vm2CrYXYga tRPQ/vrSn23dzp07jKBVA506+eoLC4vwyy+b0bfv2ejf/yLzOVqdOnXCsGE3OrZR5L/YQxs3omDx UjT4TTc06P5bXNS9O0aNGoXGjRvjV7/6FW7/85/x227nIdHor9ElF+PIx5/hwM6dvg5CII9skHDV rl07tGrVKm5DlojroHW8qAhLli3BunXr8Yc//AFXXPk7c7Qr9szD7d0fRtLMvdi715iWXof3ut9u lBJVDglZ9of/nTbXd+mAqFzy9mOfkdXzV36Gz1esxZbv/4c3pk7Ev36QgbY+8D7qVm1T2n2AT3+X jY8v3IBPLjHmF28w1z+9LBufX7nRnCa1e8dq7a9fv2SsWrUBmZnbjcAqfxB9DXr0aIPk5PYYOfIG TJhwG554YgS2bdvp2EYpNM6zeUu/xrHsbDS//joUFxaiXtFxnNq4CZ58aDzGjByJM4yQVNcol7pG vS7CsS1bULBkKY6GOGIuoaphw4ZmwJJ5vIYsEZdBq3bt2qhfvz5uv+MOvPn66+ZvOC1OaoHTWrbB +PEP4dY/3ma1tPn0dmtESSZ70DHCUEmdMfWcan6t2MeoM9bnTe3pvr19xGqDVS7mzcHs5Idxdx9r vfNdeGDQbMxh0qJKoEKW/vA/CVnym2izZs34eBQqn9Y34+X3bsc5e97BLT3OQttzL8WdT2fhtJHv 4Ye3b0RFPpWB/J17bmsjtNRAfv4h876sxMQinH9+BzRv3sSY/wo333wVrrqqu2sbJf+XX5D/xf9w /OBBFO3Zg33/ScPeD9JQa/FX6F23Hs4/fBT1ln2DvUb5vg/TcGzHTzh+6BAO//cj5P8c+qhWVRGX QUtC1imnnGJet/3TiBHmtxx25u7Elm2b8Mijj5r1ZWXi4TX9fCNKMs0EhpaEKbmHayiyHl5aUj8z 6WF01++xynwYT+KV0nojKA0tqbeNWO19BXhvtlVn9L4hC0jqAPNeTssZZyVjNpMWVTB7yJJv81w6 yheyTjrpJHMez785Uiyoiw7XvoilO/PMX4JlytuZifnPXIsOzPCV7uKLO+OMMxrirbe+xIUXtke7 dq2smlLB2sjfMJS/ZZhgfFbs/OdL2PHE37H9sSdx/F9vIO/5F7B3ylQUvfK6Wb7jiUnY9cZ0yPDY obmf4NCmHKuX6iPB+CEI8QaB0MjzM0IxY8YMDBs2zFor9diLb2H0Lddba87kN255xIN883Djxo34 y1/+gs8//RzTXnsJd97xF7Rv3952n4kEqe5Y88BevKRGlcwRrCdx1tLFuAtT0bP7Gjyw9yWUVJv1 c9DPLNPaqrQkN7s/eRaWLr4LnfVlq3rt1J7ovuYB7DV2KMu3GiFtccnG/vVEFcGrkPXUa+/ioT// 3lrzlnqSe0XcaxWq77//Hj16mE/kpCpgyRJ56Bd5raJ+ZuTf85xzzrHWAnPLJU7q1XN/Vpnu+3Wb 4z9o3TZiBLZu24a3ZryF1998DU2aNMbov441TxylJGjdCryiBSU9fKFsUPILYp310GXRwpXRUdnQ JPVz+pUELXs9gxZVJAlZ8vC/he/93VyP5kgWgxaDFlGsioWgFZeXDtUDS2+5+WaMGTvWvHQoz9GS G+H/Nv5hzHxrptVSl4k1vuf7WX7EmsxknCUPmjnjLCRnrjFK7JLQoTR5uercIQnI2mBdhizLqf7H NZlINndOVHHUw/94uZCIqGLEZdBSDyw9XlyMRic0QutWrbB9x3YsXLAIu3fudn3Mg35P1NqpT2J2 8nW4QoJU5ytwXfJsPKk9c2He7UMxe1A/7VJiAH36YVDmw3iupPu1mPpk6T1aZerXTsWTs5Nxnblz Iu/pD/9jyCIiqjhxGbTUA0vFKSedgrN/1RlPPPFEScD6MSfbnPsbhJn95pR8a7D7w0mYWXKpsDPu WrwU173XvaR+aNbDWBryZb0+eGnpw8gaqr51eCvwwMNItmrL1Js3zuuXMYm89eGcDzH40a/R7/7/ oXnz5gxZREQVJC6DlqJuL5PnafXt29e8hKjW/UmQegl9+rxU8q3BvX43vgtpo+qMye9+LSMo2dtL X3qbzndhcUnfRojqY6zrQc2vXr8pn8h7cl/jySefjDZt2uDUU09lyCIiqiBxH7QWLl5o/lXwzh07 48o+l2H9hvUx+tBSosojoapBgwZmwJI5QxYRUcWIy6ClHlg6eswYTJv2Mm4aegt6XtgTw39/C/42 YTwefeQJqyURERFR5YnLxzvIH76Vbx1u2bLFnI8YMcIslz8q/e23vj9AKb+5E5H3+HiH760lIopF fI6WTTjP0ZKQJd8+lBvj9afFy42+VfkP4xLFkuoetIioauBztGwkTEmokqfAd+7c2ZyrkEVEREQU C+IyaMlolUxyeVBGseSbVDKXdVVHREREVNni+luHRERERLGMQYuIiIjIIwxaRERERB5h0CIiIiLy CIMWERERkUcYtIiIiIg8wqBFRERE5JG4eTK8PH2aiCqX288m/wQPEcUq/gkem0BB67q+l1lrRFTR 3pv7OYOWAwlaPXr0sNaIKJYsWbKk0oMWLx0SEREReYRBi4iIiMgjDFpEREREHmHQIiIiIvIIgxYR ERGRRxi0iIjiUHp6urUUGmkfyjaqnWobaDu9jZtI64iqimoZtOrVrYOWzZvgtBOb4tQoTNKP9Fff 6JeIKNokkOiTzqlOb6OWU1NTzbnQ29knRW8vy25t1VzY6xS3MqdjUuzrRPGqWj5HS0LRQ/ePxsED B5GYWBspZ5+KjZt34/Q2TbFpy97SeWtjvnWPsd6spFzandG2ubG+B+2N+bqNO7H3QD5OPfUUPDB+ InJ377P2QlS18DlaziryOVqhhg8VjNSyEmh7+zZOy8EEaxtoH8K+rVMbonDwOVqVpEFiHXy9bClW ZqzExnXfo3OT77FnxwqcecJK/3kjmX9vrPvqOxjlv2zzre/abqw3+h7bNn2Pb775Dquzssx+iWLd irM7mFN+fj6KioqsUoplEjZkkrChAkeg4KEHFrWsBxanbZ22EfpyMPa2+n5kWepVmX0fep2i2qht 9W2ockhwicZUnVTLoHW8uBiJiYkoKixC3cQG6HnlEDRv2QY9rrjBZT6oZL3pSaehx+VG+Umt0aP3 IDRqeiKOFx1DnTp1zH6JYpkErJtvhjmtPS8FBw8cYNiKQU6BQg8bajlUahu1HA32UORE31cox60f p96/Kg+2PVUMGcEtz1TdVOub4WvVqoG9+/Iw7pGZyMzKwcQnZ5eZ/6CtP2zM167fjomT/o2sdVvx 8BPvIGfLL6hdu5bVY+XZ+GofnNv5Xiy01sXCsacbZdY04GVstMqpepp1bwdrCdiXa8337sGRI0d8 KxQz9EChhw/FKWzY2wQSzvZu5aEGHn17t22kjWqn2qi5Xkck/lWztjnl5eWhsLDQKo1dVeYeraaN GqBhvURrzefwkaPYte+gtVaqVa2PrKXo21Z4lbVUgbJfxo2vrEOHNODStVNwsVEkwev6j/rj3Q9v Q3u1vuEerJjEvxdZHUnIGnBVZ+Tu2od9D/tSVo0Zb6NJ06Y4qeUpqF+/vlkWSHnv0ZL7q8p7b1V5 79GKxjHYRfMerUBBxF7nVKYLVh+KaPQRqlD3pdrJnCqeXPYr73/v5elDApaMyIs33gD6//wTmjRp glq1nAc8ZF+8RysK6tapbYas4uLjyD20Hj/u/xYHj/5ifrvw5OaNrValTjpzVMnktG7n1E4vU/Tl Up9jvBpVMqYbX11vlfvCz/j57vXCb1Sqcx+8mW1VlDC2v2odbp90pbUu1iP9ozXoO8oXskT7P92D vmnPOmxPVZ0essTa3r6Q1bBRIyNoNUPdunXNci+pgBQLYulYdCo86AEiUJgIFkqiEZCi0UeoQt2X tOs5bbc5V5Mq19nXHaUNR0LCcKTJvOtkZFnFJbImo2tCV0w2KrImd0VXWQhA2jj1Y5bLfqz1ErLf 4WVKDWkYnpBgbGOffMdi9IjJXW11ar/aMVc1o/vWtpZKR+V37dqFgoIC30qMqhJBq1EDX7L8ePMU TF5+GSZ9cyn+7/trzcBVx0i5J9T3H+kSO7OfNydFrTuHpVL27QJbjzcHjMCGv36KFWs3GdM0dHj6 br+wM3fkx7jUrDOmj+4HtHoJYveuux/vqvoXOmPqVfbLgyOAF3yjWP7OwpltrUVTW5zZ2VqkasMe spYty8XZQ30hq0WLE3GCMa9Zs6ZZ55VYDDaxdkwSCtRIjR4Q1LKahzqKE2q7eLV4RHPH12h//+z8 308jrExYhUmZ09G//7UYljELc23hJGvuLGQMm4gxSVZBQGmYNBZIQdl+RErKKgxwDFVO+mN6cTHk glPJlDnJ6GMI+pYcS4px7Fr9yjEwq5LGYOWHXTB2qENw9MDj227HvL0z8c/c8Ziy4z48tu023J9z I+7adDXu+PEK3Jx9AUZkX2q1jpyELPksa/JwS3Mk68NPgE6LFli1sa1qjGhZ90h9mvMMzuvfEudd fzJqnLsS83OfM8tlxMtOH5XyTPYCfLx2IG7/U0er4DLc/FcjEC7URq36X1kaks7shSs7r0F2jqyU HZXCpVMwpf8HmD/fWp9/L+7FNDxa5r/hjmjbaQ2mvvK5tS6h7W5MXWutULXAkBVYRR9b0mcJ1lJZ KmTJ3D7pAoUIXajt4pn+Gu2vV72Xbsy6tEkYCxVc+mPsJONnxi8hZWHurAwMu7a/tR5E2vuYYYSy lRONkDOpbKDqMnEmJq0agJCzlk2akeK6TLTCFLKRmdEFndwCoEtw9MLq/OUoOJ6PvKID2HE0B7lH tyL32FbsPLYdO/JzzLqCJeX7Rr4KWfqovIQsuWTYokUL88ttsaxKBC0ZNhVHiw/hIHbiCPai0Dh/ 7CzwBZpatpOJ06iUCl6qPCpBLGedEZc+wL3apcHrn16D9RvMJGXq2MFv2EmTg+y19lEpnw2bjNcl 92U93wnvutxzdfGkT3HXuhEl+30Q9+AujmhVGxKy7n/6uZgLWVJWnklxqgtlsnMq84KErNWXBw9b QkKCCg72AOGk6ZI7raWqJXX3NGspMD1QOQVT/T3U69Pen4GUIX2t4GL82/QdAklbJTkoay5mZQxD aDnLGh0bazSWkDPj/TKXCVetA8bMnIRVAxwuIQaTNRkTVhnB0O9YZmCAdunQP8D1x7XDMmzB0RuJ NRpg85G15pRXtN8IWVtQNyER+38+iIZ1G+HAz3moXSPyoGUPWfJZ1v8BX8hq2bJlwPuzYkWVCFpF RcfNebN6rbA3bzf27z6KYweB0xt0N8uPHvP/VoJTiFLhS5U7hTFh31ZtI1OZ9m07oWNn7dKfmkK6 IV0u9anRLX8dTu+IjQvTsH7t33F9SYgbgblWqBtvjnh1xE0flu7znYt/xMfGD1/qmWYXVE1cfN39 lRKyqJQKWcItbNkDgfAPCLutpbL29viHteStabv9Q4zX0puPsJZCp94zCVSyLHMVrvT3WObNRyxG F31IKKkvhqTMwPtWYJHLhnItMLScZYQyv9GxVZjgdJNU0hjMNOpCv4To4z+aJWyXFjMlwPnfl3Vm cgoyMr2/Kbdbw4tw76nP4Icv1+Kl9p/jvU6ZeO70OZjXcwPe6vAN0lKNg4owaVSFkCWqRNA6cuyY Ob++/VNIzPwNjq4+DWf+Mhi/O/Vus/zIUV+9okKUCkZ6QCoTlgx6O31S7Osl5FIg/o4HbTe4h6Yj Uq86C3Of1x7LIJcK0wbi0kvl5vZ5/uFt7TT0xUBMMZbLXkqUG+b/jg76ZUiq0hbmdsXf/3q3OVVW yHL6dp+UlWdSnOpCmeycyqJJD1mKU9jSw4CzEC9feWhEc7djqxwqRDm9Z3qgUpO9/ZI7e5pzVW/8 q6DvkBTMMJOW736rIaU3RAVk3suVMRbJ1uhS8tgMZMya63iPVNKYMC8hymjWjCAja0aAm1hBI1h2 8zf5XsiT17xoznOPbsHAhSkoKi5CzzfaonZCHTRLCe0bejoJWfZR+XgMWaJKBK29Bw+h6PhxdD2x L+7u/AUm/moNhp/5Mk6qfwYKjJB1qMD/OUF1T74FF/d7ERdd/U9ceNUL5pTa5/mwJ7Vtj95TMfi2 D81+/cmo0qe48qMrSi7hyeQbcQpOwtS7V6WVjlqNXIu7PnK68d2BXFos2eezOPMjpwBGVdVzr76J nnd/gW63ppmPcKiskSyvg0x5VEbIUuxhS538VeCyU/XRMq1naJflYoXT+6ICktt7JlSdCliqfWpq U/T4x+KSOtUuqe8QpMhlP7nfyu/G80AklHXBh/oIU3EmJrncFG/sJaxLiDKaFfLIWiXYv6EAmfnf ILVRH/R4uTVa1mmD2o1qomZCTSSe7AtCxwsjf4qUGpWP15AlqkTQOn682PwbgwcPHTYvE0royjfC 1Z4Dh/DL3gNWq1IH8wswcOA1uKL3lehtTFf26YNTTzvNqg1Nq1atzO1k+75XXYUeF6Qi77DTgx/9 L+HJpAKPBKl3Sm6UF762eiDyH7mah5tcL/1dhketZ2iZzrwN74S0HVVF8siG5ka4OvW0VuZzsirz cmEshq3KDFmKPWypMKCok79Qy3qZEsk9WiMWh39ZrjLp74tdZHW+BLVqXWkSMtuaI0MzMGHCKr/7 twIyb4K/1haEZHRMbvlyiVLqEqLsxypyZI1mTbR/7TEry3+0zGyX4jcCl52ZgZRk7z/4G3euZ37b cOSPvVH/tNrmNw2P7ivC8A3dASNf3ZLdEwfWhP9QZH1UPp5DlqgSQUtI2NqXl4+f9+zHjl/2Yvf+ PBw67Pxsjf1GIPvzX+7BmHHjMebBv2HsuL+Z/4ChOm4EuTPOaI8HHprg2/7BCbj9zruw39g/USyQ UCUPIZWAJfPKvicrlsJWhYSs5sbCd8EnaSftnQKUnPhLyo0TqXAKDhV1j1ZlmNYzOjf66++leg/l Zvt/9Nhbum7V9792GDIynC8bZoxNLrn53JyGT8bkCTMcv5mYNGYihs2Y4Po8K/MSIjKM/7lzH82a i6H6cSTPwpDMldpjKNLwvi14eWX/zgOolVAb2Ruz0ahDIjasy0adRjWxdVUuElvUQs7q7di7teyD w4NZsGgBbpu6BkMf/xbyzcJ4DVmiygStcMg9Wzv3HjBHwWQqNv4nJ6PGjRubUyANGjQw/7Fr1qpp DhGrPiTgyWVKInIWC2GrIo4h63LjE6Vb6JO0l5O9U4hS0vdWv78PJ0YsLl+IVOFJ5vr7ay73H4sl d/qCnN9733+68dmuhxafpDErtUuD1jR9DMasLMZ0x+t6csO6rx/ZdmWZh3ElmdsWO29s6j+92GE7 gzwry+9YbMcb1qXP8umSnIKcTVtQo24C8rcfRa16NXD0QJEZtgp2F6JG7QS0v/5Eq3Xo5JENEq7a tWtnXkGK15AlqmXQsjuQdxgPjHsIr/zrdQweMtQqLUtGBQbdOBivvvY67r7nPnNkjIjIK/bwVTLC FSXR7i9S4RxHOG3V+ydz2c5/WyPoTOyCnj2nYXeMvA9RkTUZXQeswqSZ+rcUvTOl3Qf49HfZ+PjC DfjkEmN+8QZz/dPLsvH5lRvNaVK7d6zWoZNQ1bBhQzNgyTxeQ5Zg0DLIzfKHj9fECc1ORKNGjVCj Rg0zVNkn+YeW+sYtTkZhzbrIy4/tx/4TUXyzhwp78CqvaPcXqXCOI9Jjdtyu/3QsXjwiZm80j4g5 2lV2RI4qD4OWpbCoCAfy8nFF3/74ZvkKfP3td2Wmr5Z+jasHXm+OgB0rLLK2JCIqH320RQ9XsRKE Ypk9jAq3Mnk/+f5SRWPQ0sgXUH/ZdwDZ2352naRevtVIRBRN6qTPk7+PU1hy4vR+BSqTud43g1f4 lixZUq6pumHQIiKKEeqkbw8ZoYaOqiQaocftfZO+nfqvju9zuHr06BGVqTph0CIiigFOIUstx0Mo qKjjCWU/bu+bvq0sy+QWuoiihUGLiCiGqJO+CgX2uRJOOLBv64WKCCv21+H2utyORS+XZbUu/VTE 8VP1lFAsD+GIosOHQ3vkwYwZMzBs2DBrrdRjL76F0bdcb62Veuq1d60lIqosbj+bD/3599aat778 8ktzHksPQP3+++89uRTiduLXQ4E9ILgFhlCCRHm2DVe0+4ykP9mGqj65J+ycc86x1gJzyyVO6tUL 7e83fr9uc/wELSKKTQxaFR+0hF5f3uBi3748wSWU7cp7vE6c+tTLAi0LWVfLVHXEQtDipUMiohgk J337JOwBwY2+TTAqZATbR6D+ZJtQw4rqXxfKdoHYj1nvT5b1+lCPkygaGLSIiGKYhAI1CXtocKLK VTs7+3Z6nzr7fuzr9rlw6kevd6NvZ2+vrzv1FWz/UqbaqHq9TDj1SxQNvHRIROXCS4ffW0sVS/64 vfwVC31u51aus/ehbyPLToL1qYSy/0jZ+w62L1Wvz6l64D1aNgxaRPGlugetivTdd9+hW7du1ppv 3Y20s9erMnsf+rri1rdqq7ZzmodK307Rtw+lv2Bt9L6FfX9ESrUPWvzWIVHl47cOY48eHOyBI1h5 NLgFF32f9uNw27/el1p22sbet31dOG2vl6l2RAqDlvFhfl3fy6w1Iqpo7839nEErRjgFBRUoFHvI EPZ1xa08EvqxqeVAZU51TqRO2OvD2Ua1DbQNVV9eBS1epCYiijNOIUHK9EkvU+zrilt5JFQ/9jCj Qo/QA4/OqUyROr0/odrqc31S26h6oS8TVQQGLSIiijp7KNLXVfixtxFOZW6kbbD2+n5U+2DbEEUT gxYREXlKDzZ66HELW+Fy61+fcySLKguDFhERVYpohCydPVzpor0volAxaBERERF5hEGLiIiIyCPV MmjVq1sHLZs3wWknNsWpUZikH+mvvtEvERERkVItn6Mloeih+0fj4IGDSEysjZSzT8XGzbtxepum 2LRlb+m8tTHfusdYb1ZSLu3OaNvcWN+D9sZ83cad2HsgH6eeegoeGD8Rubv3WXshqlr4HC1nlfUn eIgoNPwTPDYVEbTan3YSepz/Gxw9VogTm9TCuD+cihnzdmHQZc0x+/PdDvNmxnwPbjTWZ3y8C0N7 N8fbn+3G73u3wEv/2YmsjXno2OF0/OfDOdi4fae1F6LYtOLsDua88/IM1K1bFzVr1jTXg2HQciZB q0ePHtZafLP/oWX1B5ilTC0Ltz/AbN+eqLItWbKk0oNWtbx0eNzIlomJiSgqLELdxAboeeUQNG/Z Bj2uuMFlPqhkvelJp6HH5Ub5Sa3Ro/cgNGp6Io4XHUOdOnXMfolimYSsm2+GOa09LwUHDxxAUVGR VUvVjQpPeqBS9GAlVJ1bG4YsImfV+mb4WrVqYO++PIx7ZCYys3Iw8cnZZeY/aOsPG/O167dj4qR/ I2vdVjz8xDvI2fILateuZfVYsTa+2gfndj5dm/rgzWyr0uBXP+BlbLTKqXqada9vJEvsy7Xme/fg yJEjvhWq8sIJRvZAFU57xb4/ouqoylw6TEhIQNMTGqBunVqoVbMmjh8vxpFjx3Dg0GHzEqGuVa2P rKXo21Z4lbXkPQlSD+I5vPOnjlaJZv69OHckMGXtFFxsrErb6zfcgxWT+PciqyMJWQOu6ozcXfuw 72Ffyqox4200adoUJ7U8BfXr1zfLAuGlQ2exfunQLeCoYKTXuwUpex/STsr0OVEs4qXDKJGQJTe4 N6hX1wxZokaNBPPbhSc3a4w6thGnk84cVTI5rds5tdPLFH251OcYr4063fjqeqvcF37Gz3evFwvH ltbZR6y2bliDDqc7hCzDws8+QMe/3m6GLNH+T/egb9rHWGitU/WhhyyxtrcvZDVs1MgIWs3M+7Qo /rkFKicqICkqKEmZKrfPnUid3pdb20B9BBLpdkSxpEoErUYN6hkBqwbqzP8CTc//NVqcchKade6I ev98oaTebmf28+akqHXnsFTKvl1g6/HmgBHY8NdPsWLtJmOahg5P3+0XluaO/BiXmnXG9NH9gFYv QezedffjXVX/QmdMvepev7A0d6QWxMZ+bpWuR8462EJYW5zZ+QPMn2+tUrVgD1nLluXi7KG+kNWi xYk4wZiHejM8xTYVeOyTqlPswcheJ/R6IeWqTp/LZO9L5qpMqHK1bK8X9vZqXe9Pb0MUT6pE0Kpb p7Y5r/fUZNS8/nr5o1aoMfhG1P/iM7O8bm1fvU4flfJM9gJ8vHYgbi+5tHcZbv4r8PFCbdSq/5Ul o044sxeu7LwG2Tmysh7pH61B31G3ob1Zabh0Cqb0Lw1LF0+yApg5fYq71o3QRsTOwpltrUWqlhiy qjY9fLgFET3gCLWus/fjxN5Gn3T6ul6vltX+nQKUmgv9OPVlvT1RvKgSQUtGs0TNTZuAPXuA5cuB oiIk5OWZ5XIZUec0KqWClyqPShDLWWfEpQ9wb8mlv9Nx/dNrsH6DmaRMHTu4paEcZK91DksbNvlf XvTpiJtGDcT6jxZYN72rwEbVkYSs+59+jiGrinELI27soSSSoCL7UftSy6HsW9G3VfuWub0vNVfU serHq9rby4liWZUIWkeO+m52P9arF/DTT+aIFvbuRVGDBmZ5YdFxc644hSgVvlS5UxgT9m3VNjKV ad+2Ezp21i79qSmkG9LlUp9zWHK7L6tUR9m1LZC5Bzequi6+7n6GrDjmFib0oKHm9qASLfZ96euK 0771MhWOFFWnl9n7tFN9OLULti1RZaoiQeuYOc+/fxwKjBPJ8aVLcfTwYRwa+RezXL59qFMhSgUj PSCVCUsGvZ0+Kfb1EnIpEH/Hg7Yb3EPTEalXnYW5z2uPZZh/L+5NG4hLL5WV9dio3etl3g/2/Afo eFUv81LjxZcPxPqnXyq5n2vjq89ibuf+SD3TKqAqbWFuV/z9r3ebE0NWfFPhQiYVNvQQo9jLndqU l1ufsm87VSZze72+7rRsb69TxyBzaReoLUWffIsvGlN1UmUe73Bi00ZItO7V0slo1s+79/k9TFSe DN/9t7+GvHT18o8f9x/1CkWNGr6cWlhYiLZt27o8GV5uiL8CU9daq4a+L2zCo0ZYKvt4Bl/b7FG+ emE+luHpNb4VnIW7PpqHm8ywVLbfjn/91O9RD/7bDix51ANVffn5+di96xfkHzoE+VahfLuwvCGL j3dwFu3HO3gdHFRAqUgqHDnttzzHY99W7Ye8IyGpvP+9R6OPUMm++HiHKPll7wHsOZCH/IIjKDLC VcHRY9ifl18mZImD+QUYOPAaXNH7SvQ2piv79MGpp51m1YamVatW5nayfd+rrkKPC1KRd9jpwY8d cdOH/pcOVYhq/6d5tmdg+dqqeiFtSrdVIUuU7df+PC3/bRmyqhMJV81bnGj8d93KfE4WR7JimwoL MreHBVlXZU51FSnS/cnrcgtT9nL99QZj39ZtH1S1/KtmbXPKy8szBzpiXZUJWuKQEXR278/Djl17 zeAlDyt1+rM4+43yP//lHowZNx5jHvwbxo77G1q2bGnVBiejX2ec0R4PPDTBt/2DE3D7nXeZwY4o FkiokoeQSsCSOUOWjxoRiyUSDiRY6HNFBQ5VZg8S9vVQRLKN4rZtqMEoFLIPtR+3fqXcqS6ax0Gx SQKW+jNisxs3xb59+2I+bFWpoBUquadrpxHEcnfvM6di439yMmrcuLE5BdKgQQM0adIENWvVNC87 qj5+3rPfHEUjotgWK2FLDy1uAcYvdOyeZs5jkdvxK5EGoJLXrm0vy/r+ZF1NQn/PgkobjoSE4UiT edfJyLKKS2RNRteErphsVGRN7oqushCAtHHqxyyX/VjrJWS/w8uU+nHqM214QtDtqqLRfUtvD1J/ RmzXrl0oKCjwrcSoahm07A7kHcYD4x7CK/96HYOHDLVKy5JRgUE3Dsarr72Ou++5zxwZI6L4Ewth SwUDt7ldevMR1lL8CTn4uJDt9SBln9vLhL7sLAuTJ6zCpMzp6N//WgzLmIW5toSUNXcWMoZNxJgk qyCgNEwaC6SgbD8iJWUVBoQZjiRQJc+SPv31n56JSasGIBay1uPbbse8vTPxz9zxmLLjPjy27Tbc n3Mj7tp0Ne748QrcnH0BRmRr98NESEKWPBewycMt8cYbwIefAJ0WLbBqYxuDluFQwREcPl4TJzQ7 EY0aNTJvcpdQZZ9q1apl1jducTIKa9ZFXn5sp2giclfRYSvpM9/z/IKFASlzC1vVmf29kvfI/j6p dae6MtImYSyGoK8Zovpj7CRgll9CysLcWRkYdm1/az2ItPcxwwhlKyd2Mfoqm4C6TJwZXjhKG44B +BDFM4dYBbok9B2SghnvV37SWp2/HAXH85FXdAA7juYg9+hW5B7bip3HtmNHfo5ZV7CkjtU6Mipk 6X9GTEKWXF1q0aIFEhMTzfJYxaBlKSwqwoG8fFzRtz++Wb4CX3/7XZnpq6Vf4+qB15sjYMcKi6wt iaiiSUiKZLJzKvOChKzVl/vmegDQl4MFMLvUphX3FfmgoaWCyfHIe+T0nqm6YNLen4GUIX2NyOKT 1NcINEbaKokuWXMxK2MYQstZ1ujYWKOxjI7NeL/MZcJV64AxMydh1QCHS4hO+k9H8XT3ncvxpjjs p6Il1miAzUfWmlNe0X4jZG1B3YRE7P/5IBrWbYQDP+ehdo3Ig5Y9ZMlzAfs/4AtZcm+1zGUQJJYx aGnktvlf9h1A9rafXSepL4rgURBEVD2pkCVU2HILCGoeLNiYYWJvxXw9XoQSXCJR+pqbmvNgpu32 tbcfj3rP7O9lMF06adcEk/piSMoMqEEiuWwo1wJDy1lGKPMbHVuFCU73cyWNwUyjLtxLiI6SOqEL VmGdw24qUreGF+HeU5/BD1+uxUvtP8d7nTLx3OlzMK/nBrzV4RukpRoHGGHSqAohSzBoEVHckedo RTLZOZVFkx6yFFm/Lb9nmVAg6/oUKCyobeOdeh3p6XvNeTAjmju/bvWeqeXgsrBulbVYQr8c57vf aogvOQVl3suVMRbJCQlIMKbksRnImDXX7wZ2JWlMmJcQY9z8Tb4X8uQ1L5rz3KNbMHBhCoqKi9Dz jbaonVAHzVJCe+aUTkKW/c+IxWPIEgxaRFQtVUbIUuxhS1EjMxQZ9f6FFrbKSlKX4+R+qxQ1QhWM hLIu+NB6ALZvysQkl5viJdCFdQkxxu3fUIDM/G+Q2qgPerzcGi3rtEHtRjVRM6EmEk/2BaHjhWUf sxQq9WfE4jVkCQYtIqp2KjNkLU5fbE5OYUuW1XqkYaG6098/9/cwCZ26yH1TtiSUNAYTh83AhAmr /O7fCsi8Cf5a2yVGGR2TW75copS6hCj7sYrClrUOq9AF+tXPytC4cz3z24Yjf+yN+qfVNr9peHRf EYZv6G7ej3NLdk8cWOP0MO/A9D8jFs8hS8TVn+AhosoV73+CR7aPdsiy/wkeM2Q1t1ZcLD68GD3r 9SxZdhqF0cvKM0pTVdnfE7Vuf99cyTOsJiQjc+UY/0Al5QPksQ8r/R7rIM+zkkuCfoZNwqRVY5E5 sRhl71tPw/CECUg2+uk7tyuGYiZW+j0nIguTuyZjbJcPA970buwYXYcCM23HaR5P5sTA23rA/udz fvfRGTjjrHbI+XELGrSpg4M/HkFi81rI23oUDVvXQX7uMTRddzreGz3f2iK0P8EjT33Pzc015/Kt Qvl2YSQhKxb+BE/cBC0iik38W4eh/61DexAQKgw41VFgTu+leh+FvlyWBB0zwYT4nKxY4gtpzgHP W/aQdO/mgcjK/gE16iSguLAYCTUSfJcKjf9TlwzPPevXmNTuHXNZhBK05Gnv8iBSmUu4krAVyUhW LAQtXjokIqog+klfllUwoMg4hSwlcMgSSRgjz7xKjr97pdKG+0bCKjpkOZnS7gN8+rtsfHzhBnxy iTG/eIO5/ull2fj8yo3mpIesUEmoatiwoTmKJfN4u1yoY9AiIqpAKgDYQ4E+F4FDAgUT0vsnz6oq nh7aIxxiSP/pxRV+yZAix6BFRFSB7CFLDwTxEq5COc7Kei3295eosjFoERFVIAkBKoTIslso8Dos lCcIhXJsXh6/07HL/ior3FU3ct9TeabqJiZvhiei+MKb4UO7GV6oAOIWFoTUOS07CVZfUbw6Drd+ A+1P6oiEBDt+65CIKERVIWgpEhJUIFDLKjioZb3eTm8vArXV2bcLV6Dty9t3KJxet1qXZSJdLAQt XjokIqoEeihQYcGpzB4i1FwPG0Jv60b16ZVw+7a/Xn3diTr+YO2IYgmDFhFRDLIHChViyhOUyrOt LpRQFC45tmDHZ38PVPAiimUMWkRElcwtLOiBwh5s9PVohR6nfpzK5Lj0Y9OFcyzhhKRA+5HlcPZL VJEYtIiIKpkKChI89LmdKtPb6O3s60Ktu5UHo4cht74UKQ8nPIlAxxFoPyLcfRFVBgYtIqIYoQKE HjACBRGhhw1ZlknfRtWruR5S7Puxt9HnalnaqHbCvo3Q2zux92en6p36FuoY9H6IYhW/dUhEcSNW v3UYS44fP44aNXy/Q8uynV4ny/pcp9op9vaqTNjbOrH3L/T96stO7PWh7JNI8PEOREQhisWgFSu+ ++47a8mnW7du1pKPvV5IGylXcyXYtvp2wt6H0/Zube2c6uxl9v6JooGPdyAiIlcSPtRkp8KNqtPb yLK93k6vc2qj17nVq6CklvV2+rIeqITeXk32NkSxjCNaRBQ3OKIV+/QQZQ9UTiFKqHb29iJQHVE0 cUSLiIhiXrAwJPVqUpy2sYcyonjFoEVERJ6wByinQKVzqldlwbYlilUMWkREFHMYrKiqYNAiIiIi 8giDFhEREZFHGLSIiIiIPMKgRUREROQRPkeLiOIG/wQPEYWLf4KHiChEsRq0evToYa0RUSxZsmRJ pQctXjokIiIi8giDFhEREZFHGLSIiChupKenW0tE8YFBi4iIYpY9WKWmpvqVMXhRrGPQIiIiz0Qa hNR2erCSuUxSRhQvGLSIiKogezjRuZUpatk+V2RdnwJRoShYOzs9YKllmatlvU6E2z9RRWHQIiKK Y4EChl6nlmWuwomd1OnbqFAjVJ1at/eht3Oau7UXTts41Qv7sr0tUaxh0CIiilMqYOiBQ5VJsAkW hoTbNk5lbnO9X1Um9HW9jZByWbaXC6lTk6Iv2wWqI6psfGApEcUNPrC0bFCxz53q3LjV69uLQH0o 9r70ddWPsJc5bROsL6c+1DJ5Sx4AGg0V9TMTCw8sZdAiorhRnYOWHkDs84pgDzVu+3WrC3TMet9u 9O1Ue6GX2Zf1dhQdElzK+997NPoIFZ8MT0RUgVRQiwcSFFRwCDavCLIvfb/2EKPW3Y7JaRtF79uN Xq/ay+TUp2qrb0NVx79q1janvLw8FBYWWqWxi0GLiKqVWA5bejCQAKFChFtAqUz2EOMWavRjlzbR Dj/Sn+xD9RvL7xmVnwSsm2+GOc1u3BT79u2L+bDFoEVE1U4shi17AFHrMrfXeS2ckBKsbUUcu74P WVbBS81dpQ1HQsJwpMm862RkWcUlsiaja0JXTDYqsiZ3RVdZCEDaOPVjlst+rPUSst/hZUr9uPXp O/YEa/Ido0k75qpmdN/a1hKwL9c337VrFwoKCnwrMYpBi4iqpVgJWyoISChQoUWVBQwJHgpnv+U5 xmAhzc1u23b2fvRjct9HFiZPWIVJmdPRv/+1GJYxC3Nt4SRr7ixkDJuIMUlWQUBpmDQWSEHZfkRK yioMCBKq7NKGJyB5lvRpI2FqgBx7MeQ26+LMIZiVbAW5pDFY+WEXjB3qEM488Pi22zFv70z8M3c8 puy4D49tuw3359yIuzZdjTt+vAI3Z1+AEdmXWq0jJyFrwFWd0eThlnjjDeDDT4BOixZYtbGNQYuI 4o6EpEgmO6cyryR9lmAt+ZMgIMFAn6qDntN2R/xam9u2c+sn4PuZNgljMQR9zRDVH2MnAbP8ElIW 5s7KwLBr+1vrQaS9jxlGKFs50Qg5k8oGqi4TZ2LSqgEIOWulDccAfIjimUOsglJlAqARriYOm4EJ ahjLJTh6YXX+chQcz0de0QHsOJqD3KNbkXtsK3Ye244d+TlmXcGSOlbryKiQlbtrn7m+trcvZDVp 0gQtWrRAYmKiWR6rGLSIiDwmIWv15YHDlj6vDhaPaG4tec8pbKW9PwMpQ/qiJKv0NQKNkbZKclDW XMzKGIbQcpY1OjbWaCwhZ8b7ZS4TrloHjJk5CasGOFxCdNJ/Ooqnu+88JflMa8nnzGR93Ks/rh2W YQuO3kis0QCbj6w1p7yi/UbI2oK6CYnY//NBNKzbCAd+zkPtGpEHLXvIWrYsF/0f8IWsli1bmvNa tWqZdbGKQYuI4oY81qE8k51TWbSpkCXsYUsFAPu8OgWuytSlk3ZNMKkvhqTMwPtWCpJRI7kWGFrO MkKZ3+jYqtLRJV3SGMw06sK9hGiX1KkLMvxC4WQMHZthrfhI8MrIzLbWvNOt4UW499Rn8MOXa/FS +8/xXqdMPHf6HMzruQFvdfgGaanG+xBh0qgKIUswaBFRtVTRIUtRYUtClbpsyGDlPRVifbKwbpW1 WCIJfYekYIaZtHz3Ww3xJaegzEt5GWORbN2cnmyEnoxZcx3vkUoaE+YlRCf9pyNTApu6GX4oMHFS mTu5KsT8Tb4X8uQ1L5rz3KNbMHBhCoqKi9DzjbaonVAHzVJCe+aUTkLW/U8/F/chSzBoEVG1U1kh S1HlesjSw5Z/KKBI2MOrWg/03ib1HYIUuewn91ulqBGqYCSUdcGHclN6yZSJSS43xUugC+sSoouk MStL97eyL9bNCj0YRtP+DQXIzP8GqY36oMfLrdGyThvUblQTNRNqIvFkXxA6Xhj5c9Evvu7+uA5Z gkGLiKqVyg5ZdnLiVxNFj/5+lh0xTEKnLnLflC0JqZvKJ6zyu38rIPMm+GttlxhldExu+XKJUuoS ouzHKiqPtOHJGNvF/9uR2ZkZZe7j8kLjzvXMbxuO/LE36p9W2/ym4dF9RRi+oTtg5KtbsnviwJoj VuvQLcztir//9W5ziueQJRi0iKjaqLCQJfd5f+c8LU5f7Ft2UTYUVK6ePadZS2XF2rHq1LHJ3Cl0 9b92mOPlPbM8w3l0KGNssvXcKmsaPhmTJ8xw/GZi0piJGDZjguvzrMxLiMgw/heJLEzuWnocE5Iz bTfOp+H9GSkVMsK1f+cB1EqojeyN2WjUIREb1mWjTqOa2LoqF4ktaiFn9Xbs3XrQah26BYsW4Lap azD08W/NbxbGa8gS/FuHRETlUJ6/dWgPABzVih71furvq/9yUyOsDAVmrgzxOVlxRB5mOiEZmSvH hDYqFwb73ym8d/NAZGX/gBp1ElBcWIyEGgm+S4XG/6lLhuee9WtMaveOuSxC+VuH8rR3eRCpzCVc ySMcIglZsfC3Dhm0iIjKIZKgxYBV+eR9NwPJAODD4umhfbswHsjDTJNnYUimNwEyGn8QOhp9hCoW ghYvHRIRVQLzRG/QR1sUfZmiR72vJe+vPKuqKoUsIU+GL66Co3RxjEGLiKgS2QOXsI9wMXiVj7x/ Msn7yveSKhovHRIRlUO4lw71EKWf/J3mTgLVxYuKfA36vmSZykcuxUVDdbp0yKBFRFQOkQQtpxN+ LISneAtx6n10O2a93mmZqj7eo0VEVA2pYCBzNYlwAoAXYUEdRzyQ16+/d4r9fVH1qq3ajqiiMGgR EVUgPQjIsr6uAoA9LDip7LAQyjEq4bQNldvrV2FKJn1Z58XxELlh0CIiqmDBTvwVHaIiCR6hHqPT a41kf8HofcqxqZBlp467ot9jqr4YtIiIKoF+oref9KMdRIL1Fyh0hHosbu2kb/trDbS/SOl9qmNR ZTKXMjUJt+MlijYGLSKiSqCf6O0nfRUMIhXOtqqtfa6Eeiwq1ARr67af8nDry4t9EYWLQYuIKAbY w0CowcWJvq2+vVNfqq0KVGpdUWVO2yp6nd7WaRt9fyJQv6Gy96Ufg31/RBWNj3cgIioHebxDNB0/ fhw1atQw59Fg7yvQuiwL+771ctXe3tZpW7f+hOpHZy8LtH0o1PY66cupnKouPkeLiIjK+O6776yl srp162Yt+ZNtAtXp3NoJ1Y8+F6Fso5aFvq2dU1vh1I/OrU+1jbD3KfR+iZwwaBERVTORhINYDBSR vg5FtlV96OWKvW+n/UVyDFS9eBW0OH5KRBSjIgkGsRgmIn0darKHK1Wukzb2dvq6vT1RRWHQIiKi mKZCkj1ICVWnBymnMqLKwqBFREQxL1CgUuzrTsGMqKIxaBERUVwId4Qq3PZEXmDQIiKiuKLux+KI FcUDBi0iIoorMlKlJqJYx6BFRERE5BEGLSIiIiKPVOoDS4mIiIhiRZV6MjwRERFRPOKT4YmIiIhi AIMWERERkUcYtIiIiIg8wqBFRERE5BEGLSIiIiKPMGgREREReYRBi4iIiMgjDFpEREREHmHQIiIi IvJI1J8MT0RERER8MjwRERGRpxi0iIiIiDzCoEVERETkEQYtIiIiIo8waBERERF5hEGLiIiIyCMM WkREREQeYdAiIiIi8giDFhEREZFHGLSIiIiIPMKgRUREROQJ4P8Bg6qTdFLdMD8AAAAASUVORK5C YII= ------=_Part_6439429_1730970573.1516054958003--

1 0

ha-agent and broker continually crashing after 4.2 update
by Jayme 15 Jan '18

15 Jan '18

recently upgraded to 4.2 and had some problems with engine vm running, got that cleared up now my only remaining issue is that now it seems ovirt-ha-broker and ovirt-ha-agent are continually crashing on all three of my hosts. Everything is up and working fine otherwise, all VMs running and hosted engine VM is running along with interface etc. Jan 12 16:52:34 cultivar0 journal: vdsm storage.Dispatcher ERROR FINISH prepareImage error=Volume does not exist: (u'8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8',) Jan 12 16:52:34 cultivar0 python: detected unhandled Python exception in '/usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker' Jan 12 16:52:34 cultivar0 abrt-server: Not saving repeating crash in '/usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker' Jan 12 16:52:34 cultivar0 systemd: ovirt-ha-broker.service: main process exited, code=exited, status=1/FAILURE Jan 12 16:52:34 cultivar0 systemd: Unit ovirt-ha-broker.service entered failed state. Jan 12 16:52:34 cultivar0 systemd: ovirt-ha-broker.service failed. Jan 12 16:52:34 cultivar0 systemd: ovirt-ha-broker.service holdoff time over, scheduling restart. Jan 12 16:52:34 cultivar0 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Jan 12 16:52:34 cultivar0 systemd: Started oVirt Hosted Engine High Availability Communications Broker. Jan 12 16:52:34 cultivar0 systemd: Starting oVirt Hosted Engine High Availability Communications Broker... Jan 12 16:52:36 cultivar0 journal: vdsm storage.TaskManager.Task ERROR (Task='73141dec-9d8f-4164-9c4e-67c43a102eff') Unexpected error#012Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run#012 return fn(*args, **kargs)#012 File "<string>", line 2, in prepareImage#012 File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in method#012 ret = func(*args, **kwargs)#012 File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 3162, in prepareImage#012 raise se.VolumeDoesNotExist(leafUUID)#012VolumeDoesNotExist: Volume does not exist: (u'8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8',) Jan 12 16:52:36 cultivar0 journal: vdsm storage.Dispatcher ERROR FINISH prepareImage error=Volume does not exist: (u'8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8',) Jan 12 16:52:36 cultivar0 python: detected unhandled Python exception in '/usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker' Jan 12 16:52:36 cultivar0 abrt-server: Not saving repeating crash in '/usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker' Jan 12 16:52:36 cultivar0 systemd: ovirt-ha-broker.service: main process exited, code=exited, status=1/FAILURE Jan 12 16:52:36 cultivar0 systemd: Unit ovirt-ha-broker.service entered failed state. Jan 12 16:52:36 cultivar0 systemd: ovirt-ha-broker.service failed. Jan 12 16:52:36 cultivar0 systemd: ovirt-ha-broker.service holdoff time over, scheduling restart. Jan 12 16:52:36 cultivar0 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Jan 12 16:52:36 cultivar0 systemd: Started oVirt Hosted Engine High Availability Communications Broker. Jan 12 16:52:36 cultivar0 systemd: Starting oVirt Hosted Engine High Availability Communications Broker... Jan 12 16:52:37 cultivar0 journal: vdsm storage.TaskManager.Task ERROR (Task='bc7af1e2-0ab2-4164-ae88-d2bee03500f9') Unexpected error#012Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run#012 return fn(*args, **kargs)#012 File "<string>", line 2, in prepareImage#012 File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in method#012 ret = func(*args, **kwargs)#012 File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 3162, in prepareImage#012 raise se.VolumeDoesNotExist(leafUUID)#012VolumeDoesNotExist: Volume does not exist: (u'8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8',) Jan 12 16:52:37 cultivar0 journal: vdsm storage.Dispatcher ERROR FINISH prepareImage error=Volume does not exist: (u'8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8',) Jan 12 16:52:37 cultivar0 python: detected unhandled Python exception in '/usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker' Jan 12 16:52:38 cultivar0 abrt-server: Not saving repeating crash in '/usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker' Jan 12 16:52:38 cultivar0 systemd: ovirt-ha-broker.service: main process exited, code=exited, status=1/FAILURE Jan 12 16:52:38 cultivar0 systemd: Unit ovirt-ha-broker.service entered failed state. Jan 12 16:52:38 cultivar0 systemd: ovirt-ha-broker.service failed. Jan 12 16:52:38 cultivar0 systemd: ovirt-ha-broker.service holdoff time over, scheduling restart. Jan 12 16:52:38 cultivar0 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Jan 12 16:52:38 cultivar0 systemd: start request repeated too quickly for ovirt-ha-broker.service Jan 12 16:52:38 cultivar0 systemd: Failed to start oVirt Hosted Engine High Availability Communications Broker. Jan 12 16:52:38 cultivar0 systemd: Unit ovirt-ha-broker.service entered failed state. Jan 12 16:52:38 cultivar0 systemd: ovirt-ha-broker.service failed. Jan 12 16:52:40 cultivar0 systemd: ovirt-ha-agent.service holdoff time over, scheduling restart. Jan 12 16:52:40 cultivar0 systemd: Cannot add dependency job for unit lvm2-lvmetad.socket, ignoring: Unit is masked. Jan 12 16:52:40 cultivar0 systemd: Started oVirt Hosted Engine High Availability Communications Broker. Jan 12 16:52:40 cultivar0 systemd: Starting oVirt Hosted Engine High Availability Communications Broker... Jan 12 16:52:40 cultivar0 systemd: Started oVirt Hosted Engine High Availability Monitoring Agent. Jan 12 16:52:40 cultivar0 systemd: Starting oVirt Hosted Engine High Availability Monitoring Agent... Jan 12 16:52:41 cultivar0 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors Jan 12 16:52:41 cultivar0 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 416, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 535, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 83, in start_monitor#012 .format(type, options, e))#012RequestError: Failed to start monitor ping, options {'addr': '192.168.0.1'}: [Errno 2] No such file or directory Jan 12 16:52:41 cultivar0 journal: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent Jan 12 16:52:42 cultivar0 systemd: ovirt-ha-agent.service: main process exited, code=exited, status=157/n/a Jan 12 16:52:42 cultivar0 systemd: Unit ovirt-ha-agent.service entered failed state. Jan 12 16:52:42 cultivar0 systemd: ovirt-ha-agent.service failed.

3 2

(no subject)
by Artem Tambovskiy 15 Jan '18

15 Jan '18

Trying to fix one thing I broke another :( I fixed mnt_options for hosted engine storage domain and installed latest security patches to my hosts and hosted engine. All VM's up and running, but hosted_engine --vm-status reports about issues: [root@ovirt1 ~]# hosted-engine --vm-status --== Host 1 status ==-- conf_on_shared_storage : True Status up-to-date : False Hostname : ovirt2 Host ID : 1 Engine status : unknown stale-data Score : 0 stopped : False Local maintenance : False crc32 : 193164b8 local_conf_timestamp : 8350 Host timestamp : 8350 Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=8350 (Fri Jan 12 19:03:54 2018) host-id=1 score=0 vm_conf_refresh_time=8350 (Fri Jan 12 19:03:54 2018) conf_on_shared_storage=True maintenance=False state=EngineUnexpectedlyDown stopped=False timeout=Thu Jan 1 05:24:43 1970 --== Host 2 status ==-- conf_on_shared_storage : True Status up-to-date : False Hostname : ovirt1.telia.ru Host ID : 2 Engine status : unknown stale-data Score : 0 stopped : True Local maintenance : False crc32 : c7037c03 local_conf_timestamp : 7530 Host timestamp : 7530 Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=7530 (Fri Jan 12 16:10:12 2018) host-id=2 score=0 vm_conf_refresh_time=7530 (Fri Jan 12 16:10:12 2018) conf_on_shared_storage=True maintenance=False state=AgentStopped stopped=True [root@ovirt1 ~]# from second host situation looks a bit different: [root@ovirt2 ~]# hosted-engine --vm-status --== Host 1 status ==-- conf_on_shared_storage : True Status up-to-date : True Hostname : ovirt2 Host ID : 1 Engine status : {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"} Score : 0 stopped : False Local maintenance : False crc32 : 78eabdb6 local_conf_timestamp : 8403 Host timestamp : 8402 Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=8402 (Fri Jan 12 19:04:47 2018) host-id=1 score=0 vm_conf_refresh_time=8403 (Fri Jan 12 19:04:47 2018) conf_on_shared_storage=True maintenance=False state=EngineUnexpectedlyDown stopped=False timeout=Thu Jan 1 05:24:43 1970 --== Host 2 status ==-- conf_on_shared_storage : True Status up-to-date : False Hostname : ovirt1.telia.ru Host ID : 2 Engine status : unknown stale-data Score : 0 stopped : True Local maintenance : False crc32 : c7037c03 local_conf_timestamp : 7530 Host timestamp : 7530 Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=7530 (Fri Jan 12 16:10:12 2018) host-id=2 score=0 vm_conf_refresh_time=7530 (Fri Jan 12 16:10:12 2018) conf_on_shared_storage=True maintenance=False state=AgentStopped stopped=True WebGUI shows that engine running on host ovirt1. Gluster looks fine [root@ovirt1 ~]# gluster volume status engine Status of volume: engine Gluster process TCP Port RDMA Port Online Pid ------------------------------------------------------------------------------ Brick ovirt1.telia.ru:/oVirt/engine 49169 0 Y 3244 Brick ovirt2.telia.ru:/oVirt/engine 49179 0 Y 20372 Brick ovirt3.telia.ru:/oVirt/engine 49206 0 Y 16609 Self-heal Daemon on localhost N/A N/A Y 117868 Self-heal Daemon on ovirt2.telia.ru N/A N/A Y 20521 Self-heal Daemon on ovirt3 N/A N/A Y 25093 Task Status of Volume engine ------------------------------------------------------------------------------ There are no active volume tasks How to resolve this issue?

2 1

Troubleshooting a self-hosted engine deployment
by Николаев Алексей 15 Jan '18

15 Jan '18

2 4

ovirt-engine-webadmin-portal-debuginfo package for 4.2?
by Jayme 14 Jan '18

14 Jan '18

First, apologies for all the posts to this list lately, I've been having a heck of a time after 4.2 upgrade and you've been helpful, I appreciate that. Since 4.2 upgrade I'm experiencing a few problems that I'm trying to debug. Current status is engine and all hosts are upgraded to 4.2, and cluster and domain set to 4.2 compatibility. Hosted Engine VM is running and ui accessible etc, all VMs on hosts are running but no HA service. Web UI is giving a few errors when checking network and snapshots on the hosted engine VM only, it doesn't give errors on any of the others VMs that I spot checked. 1. HA-agent and HA-broker are continually crashing on all three hosts over and over every few seconds. I sent an email to users list with more details on this problem but unfortunately haven't heard anything back yet. The general error in the logs seems to be: VolumeDoesNotExist(leafUUID)#012VolumeDoesNotExist: Volume does not exist: (u'8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8',) -- What? Volume doesn't exist, why not? 2. Error when clicking "network interfaces" in the web gui for the hosted VM engine. 3. Similar to #2 above an error is given when clicking "snapshots" in the web gui for the hosted engine VM. The errors for #2 and #3 are generic "cannot read property 'a' of null". I've read previous postings on ovirt-mailing list that suggest you can install debug-info package to get a human readable error.. but this package does not seem to be compatible with 4.2, it expects 4.1: Requires: "ovirt-engine-webadmin-portal = 4.1.2.2-1.el7.centos" -- Perhaps this package is no longer required? I do see some additional details in the ui.log that I can post if helpful. There is obviously something odd going on here with the hosted engine VM. All three errors appear to related to a problem with it, although it is indeed up and running. I'd really like to get HA broker and agent back up and running, and fix these GUI errors related to hosted engine VM. All three problems may be connected to one common issue? Thanks in advance!

3 9

Re: [ovirt-users] ovirt-engine-webadmin-portal-debuginfo package for 4.2?
by Karli Sjöberg 14 Jan '18

14 Jan '18

1 0

Fixing unencrypted ovirt-provider-ovn password problem
by Marcin Mirecki 14 Jan '18

14 Jan '18

ovirt-provider-ovn installed using engine-setup might have it's password stored unencrypted in the engine database. The problem occurs in some pre-4.2.1 engine-setup versions. To fix the problem in existing ovirt-provider-ovn isntances, it is enough to open the network provider dialog (Administration -> Providers), choose the appropriate network provider, open it by clicking "Edit", and confirm changes by clicking the "OK" button.

1 0

Re: [ovirt-users] unable to bring up hosted engine after botched 4.2 upgrade
by Jayme 12 Jan '18

12 Jan '18

No luck I'm afraid. It's very odd that I wouldn't be able to get a console to it, if the status is up and seen by virsh. Any clue? Engine status : {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"} # virsh -r list Id Name State ---------------------------------------------------- 118 Cultivar running # hosted-engine --console The engine VM is running on this host error: failed to get domain 'HostedEngine' error: Domain not found: no domain with matching name 'HostedEngine' # hosted-engine --console 118 The engine VM is running on this host error: failed to get domain 'HostedEngine' error: Domain not found: no domain with matching name 'HostedEngine' # hosted-engine --console Cultivar The engine VM is running on this host error: failed to get domain 'HostedEngine' error: Domain not found: no domain with matching name 'HostedEngine' On Fri, Jan 12, 2018 at 2:05 PM, Martin Sivak <msivak(a)redhat.com> wrote: > Try listing the domains with > > virsh -r list > > maybe it just has some weird name... > > Martin > > On Fri, Jan 12, 2018 at 6:56 PM, Jayme <jaymef(a)gmail.com> wrote: > > I thought that it might be a good sign but unfortunately I cannot access > it > > with console :( if I could get console access to it I might be able to > fix > > the problem. But seeing is how the console is also not working leads me > to > > believe there is a bigger issue at hand here. > > > > hosted-engine --console > > The engine VM is running on this host > > error: failed to get domain 'HostedEngine' > > error: Domain not found: no domain with matching name 'HostedEngine' > > > > I really wonder if this is all a symlinking problem in some way. Is it > > possible for me to upgrade host to 4.2 RC2 without being able to upgrade > the > > engine first or should I keep everything on 4.2 as it is? > > > > On Fri, Jan 12, 2018 at 1:49 PM, Martin Sivak <msivak(a)redhat.com> wrote: > >> > >> Hi, > >> > >> the VM is up according to the status (at least for a while). You > >> should be able to use console and diagnose anything that happened > >> inside (line the need for fsck and such) now. > >> > >> Check the presence of those links again now, the metadata file content > >> is not important, but the file has to exist (agents will populate it > >> with status data). I have no new idea about what is wrong with that > >> though. > >> > >> Best regards > >> > >> Martin > >> > >> > >> > >> On Fri, Jan 12, 2018 at 5:47 PM, Jayme <jaymef(a)gmail.com> wrote: > >> > The lock space issue was an issue I needed to clear but I don't > believe > >> > it > >> > has resolved the problem. I shutdown agent and broker on all hosts > and > >> > disconnected hosted-storage then enabled broker/agent on just one host > >> > and > >> > connected storage. I started the VM and actually didn't get any > errors > >> > in > >> > the logs barely at all which was good to see, however the VM is still > >> > not > >> > running: > >> > > >> > HOST3: > >> > > >> > Engine status : {"reason": "failed liveliness > >> > check", > >> > "health": "bad", "vm": "up", "detail": "Up"} > >> > > >> > ==> /var/log/messages <== > >> > Jan 12 12:42:57 cultivar3 kernel: ovirtmgmt: port 2(vnet0) entered > >> > disabled > >> > state > >> > Jan 12 12:42:57 cultivar3 kernel: device vnet0 entered promiscuous > mode > >> > Jan 12 12:42:57 cultivar3 kernel: ovirtmgmt: port 2(vnet0) entered > >> > blocking > >> > state > >> > Jan 12 12:42:57 cultivar3 kernel: ovirtmgmt: port 2(vnet0) entered > >> > forwarding state > >> > Jan 12 12:42:57 cultivar3 lldpad: recvfrom(Event interface): No buffer > >> > space > >> > available > >> > Jan 12 12:42:57 cultivar3 systemd-machined: New machine > >> > qemu-111-Cultivar. > >> > Jan 12 12:42:57 cultivar3 systemd: Started Virtual Machine > >> > qemu-111-Cultivar. > >> > Jan 12 12:42:57 cultivar3 systemd: Starting Virtual Machine > >> > qemu-111-Cultivar. > >> > Jan 12 12:42:57 cultivar3 kvm: 3 guests now active > >> > Jan 12 12:44:38 cultivar3 libvirtd: 2018-01-12 16:44:38.737+0000: > 1535: > >> > error : qemuDomainAgentAvailable:6010 : Guest agent is not responding: > >> > QEMU > >> > guest agent is not connected > >> > > >> > Interestingly though, now I'm seeing this in the logs which may be a > new > >> > clue: > >> > > >> > > >> > ==> /var/log/vdsm/vdsm.log <== > >> > File "/usr/lib/python2.7/site-packages/vdsm/storage/nfsSD.py", line > >> > 126, > >> > in findDomain > >> > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > >> > File "/usr/lib/python2.7/site-packages/vdsm/storage/nfsSD.py", line > >> > 116, > >> > in findDomainPath > >> > raise se.StorageDomainDoesNotExist(sdUUID) > >> > StorageDomainDoesNotExist: Storage domain does not exist: > >> > (u'248f46f0-d793-4581-9810-c9d965e2f286',) > >> > jsonrpc/4::ERROR::2018-01-12 > >> > 12:40:30,380::dispatcher::82::storage.Dispatcher::(wrapper) FINISH > >> > getStorageDomainInfo error=Storage domain does not exist: > >> > (u'248f46f0-d793-4581-9810-c9d965e2f286',) > >> > periodic/42::ERROR::2018-01-12 > >> > 12:40:35,430::api::196::root::(_getHaInfo) > >> > failed to retrieve Hosted Engine HA score '[Errno 2] No such file or > >> > directory'Is the Hosted Engine setup finished? > >> > periodic/43::ERROR::2018-01-12 > >> > 12:40:50,473::api::196::root::(_getHaInfo) > >> > failed to retrieve Hosted Engine HA score '[Errno 2] No such file or > >> > directory'Is the Hosted Engine setup finished? > >> > periodic/40::ERROR::2018-01-12 > >> > 12:41:05,519::api::196::root::(_getHaInfo) > >> > failed to retrieve Hosted Engine HA score '[Errno 2] No such file or > >> > directory'Is the Hosted Engine setup finished? > >> > periodic/43::ERROR::2018-01-12 > >> > 12:41:20,566::api::196::root::(_getHaInfo) > >> > failed to retrieve Hosted Engine HA score '[Errno 2] No such file or > >> > directory'Is the Hosted Engine setup finished? > >> > > >> > ==> /var/log/ovirt-hosted-engine-ha/broker.log <== > >> > File > >> > > >> > "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ > ha/broker/storage_broker.py", > >> > line 151, in get_raw_stats > >> > f = os.open(path, direct_flag | os.O_RDONLY | os.O_SYNC) > >> > OSError: [Errno 2] No such file or directory: > >> > > >> > '/var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/8582bdfc-ef54-47af-9f1e- > f5b7ec1f1cf8' > >> > StatusStorageThread::ERROR::2018-01-12 > >> > > >> > 12:32:06,049::status_broker::92::ovirt_hosted_engine_ha. > broker.status_broker.StatusBroker.Update::(run) > >> > Failed to read state. > >> > Traceback (most recent call last): > >> > File > >> > > >> > "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ > ha/broker/status_broker.py", > >> > line 88, in run > >> > self._storage_broker.get_raw_stats() > >> > File > >> > > >> > "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ > ha/broker/storage_broker.py", > >> > line 162, in get_raw_stats > >> > .format(str(e))) > >> > RequestError: failed to read metadata: [Errno 2] No such file or > >> > directory: > >> > > >> > '/var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/8582bdfc-ef54-47af-9f1e- > f5b7ec1f1cf8' > >> > > >> > On Fri, Jan 12, 2018 at 12:02 PM, Martin Sivak <msivak(a)redhat.com> > >> > wrote: > >> >> > >> >> The lock is the issue. > >> >> > >> >> - try running sanlock client status on all hosts > >> >> - also make sure you do not have some forgotten host still connected > >> >> to the lockspace, but without ha daemons running (and with the VM) > >> >> > >> >> I need to go to our president election now, I might check the email > >> >> later tonight. > >> >> > >> >> Martin > >> >> > >> >> On Fri, Jan 12, 2018 at 4:59 PM, Jayme <jaymef(a)gmail.com> wrote: > >> >> > Here are the newest logs from me trying to start hosted vm: > >> >> > > >> >> > ==> /var/log/messages <== > >> >> > Jan 12 11:58:14 cultivar0 kernel: ovirtmgmt: port 6(vnet4) entered > >> >> > blocking > >> >> > state > >> >> > Jan 12 11:58:14 cultivar0 kernel: ovirtmgmt: port 6(vnet4) entered > >> >> > disabled > >> >> > state > >> >> > Jan 12 11:58:14 cultivar0 kernel: device vnet4 entered promiscuous > >> >> > mode > >> >> > Jan 12 11:58:14 cultivar0 kernel: ovirtmgmt: port 6(vnet4) entered > >> >> > blocking > >> >> > state > >> >> > Jan 12 11:58:14 cultivar0 kernel: ovirtmgmt: port 6(vnet4) entered > >> >> > forwarding state > >> >> > Jan 12 11:58:14 cultivar0 lldpad: recvfrom(Event interface): No > >> >> > buffer > >> >> > space > >> >> > available > >> >> > Jan 12 11:58:14 cultivar0 NetworkManager[1092]: <info> > >> >> > [1515772694.8715] > >> >> > manager: (vnet4): new Tun device > >> >> > (/org/freedesktop/NetworkManager/Devices/140) > >> >> > Jan 12 11:58:14 cultivar0 NetworkManager[1092]: <info> > >> >> > [1515772694.8795] > >> >> > device (vnet4): state change: unmanaged -> unavailable (reason > >> >> > 'connection-assumed') [10 20 41] > >> >> > > >> >> > ==> /var/log/libvirt/qemu/Cultivar.log <== > >> >> > 2018-01-12 15:58:14.879+0000: starting up libvirt version: 3.2.0, > >> >> > package: > >> >> > 14.el7_4.7 (CentOS BuildSystem <http://bugs.centos.org>, > >> >> > 2018-01-04-19:31:34, c1bm.rdu2.centos.org) qemu version: > >> >> > 2.9.0(qemu-kvm-ev-2.9.0-16.el7_4.13.1), hostname: > >> >> > cultivar0.grove.silverorange.com > >> >> > LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin > >> >> > QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm -name > >> >> > guest=Cultivar,debug-threads=on -S -object > >> >> > > >> >> > > >> >> > secret,id=masterKey0,format=raw,file=/var/lib/libvirt/ > qemu/domain-119-Cultivar/master-key.aes > >> >> > -machine pc-i440fx-rhel7.3.0,accel=kvm,usb=off,dump-guest-core=off > >> >> > -cpu > >> >> > Conroe -m 8192 -realtime mlock=off -smp > >> >> > 2,maxcpus=16,sockets=16,cores=1,threads=1 -uuid > >> >> > 4013c829-c9d7-4b72-90d5-6fe58137504c -smbios > >> >> > 'type=1,manufacturer=oVirt,product=oVirt > >> >> > > >> >> > > >> >> > Node,version=7-4.1708.el7.centos,serial=44454C4C-3300- > 1042-8031-B4C04F4B4831,uuid=4013c829-c9d7-4b72-90d5-6fe58137504c' > >> >> > -no-user-config -nodefaults -chardev > >> >> > > >> >> > > >> >> > socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain- > 119-Cultivar/monitor.sock,server,nowait > >> >> > -mon chardev=charmonitor,id=monitor,mode=control -rtc > >> >> > base=2018-01-12T15:58:14,driftfix=slew -global > >> >> > kvm-pit.lost_tick_policy=delay -no-hpet -no-reboot -boot strict=on > >> >> > -device > >> >> > piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device > >> >> > virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x4 -drive > >> >> > > >> >> > > >> >> > file=/var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/c2dde892-f978-4dfc-a421-c8e04cf387f9/23aa0a66-fa6c-4967-a1e5- > fbe47c0cd705,format=raw,if=none,id=drive-virtio-disk0, > serial=c2dde892-f978-4dfc-a421-c8e04cf387f9,cache=none, > werror=stop,rerror=stop,aio=threads > >> >> > -device > >> >> > > >> >> > > >> >> > virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive- > virtio-disk0,id=virtio-disk0,bootindex=1 > >> >> > -drive if=none,id=drive-ide0-1-0,readonly=on -device > >> >> > ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev > >> >> > tap,fd=35,id=hostnet0,vhost=on,vhostfd=38 -device > >> >> > > >> >> > > >> >> > virtio-net-pci,netdev=hostnet0,id=net0,mac=00:16:3e: > 7f:d6:83,bus=pci.0,addr=0x3 > >> >> > -chardev > >> >> > > >> >> > > >> >> > socket,id=charchannel0,path=/var/lib/libvirt/qemu/channels/ > 4013c829-c9d7-4b72-90d5-6fe58137504c.com.redhat.rhevm.vdsm,server,nowait > >> >> > -device > >> >> > > >> >> > > >> >> > virtserialport,bus=virtio-serial0.0,nr=1,chardev= > charchannel0,id=channel0,name=com.redhat.rhevm.vdsm > >> >> > -chardev > >> >> > > >> >> > > >> >> > socket,id=charchannel1,path=/var/lib/libvirt/qemu/channels/ > 4013c829-c9d7-4b72-90d5-6fe58137504c.org.qemu.guest_agent.0,server,nowait > >> >> > -device > >> >> > > >> >> > > >> >> > virtserialport,bus=virtio-serial0.0,nr=2,chardev= > charchannel1,id=channel1,name=org.qemu.guest_agent.0 > >> >> > -chardev spicevmc,id=charchannel2,name=vdagent -device > >> >> > > >> >> > > >> >> > virtserialport,bus=virtio-serial0.0,nr=3,chardev= > charchannel2,id=channel2,name=com.redhat.spice.0 > >> >> > -chardev > >> >> > > >> >> > > >> >> > socket,id=charchannel3,path=/var/lib/libvirt/qemu/channels/ > 4013c829-c9d7-4b72-90d5-6fe58137504c.org.ovirt.hosted- > engine-setup.0,server,nowait > >> >> > -device > >> >> > > >> >> > > >> >> > virtserialport,bus=virtio-serial0.0,nr=4,chardev= > charchannel3,id=channel3,name=org.ovirt.hosted-engine-setup.0 > >> >> > -chardev pty,id=charconsole0 -device > >> >> > virtconsole,chardev=charconsole0,id=console0 -spice > >> >> > > >> >> > > >> >> > tls-port=5904,addr=0,x509-dir=/etc/pki/vdsm/libvirt-spice, > tls-channel=default,seamless-migration=on > >> >> > -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -object > >> >> > rng-random,id=objrng0,filename=/dev/urandom -device > >> >> > virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x5 -msg > >> >> > timestamp=on > >> >> > > >> >> > ==> /var/log/messages <== > >> >> > Jan 12 11:58:14 cultivar0 NetworkManager[1092]: <info> > >> >> > [1515772694.8807] > >> >> > device (vnet4): state change: unavailable -> disconnected (reason > >> >> > 'none') > >> >> > [20 30 0] > >> >> > Jan 12 11:58:14 cultivar0 systemd-machined: New machine > >> >> > qemu-119-Cultivar. > >> >> > Jan 12 11:58:14 cultivar0 systemd: Started Virtual Machine > >> >> > qemu-119-Cultivar. > >> >> > Jan 12 11:58:14 cultivar0 systemd: Starting Virtual Machine > >> >> > qemu-119-Cultivar. > >> >> > > >> >> > ==> /var/log/libvirt/qemu/Cultivar.log <== > >> >> > 2018-01-12T15:58:15.094002Z qemu-kvm: -chardev pty,id=charconsole0: > >> >> > char > >> >> > device redirected to /dev/pts/1 (label charconsole0) > >> >> > > >> >> > ==> /var/log/messages <== > >> >> > Jan 12 11:58:15 cultivar0 kvm: 5 guests now active > >> >> > > >> >> > ==> /var/log/libvirt/qemu/Cultivar.log <== > >> >> > 2018-01-12 15:58:15.217+0000: shutting down, reason=failed > >> >> > > >> >> > ==> /var/log/messages <== > >> >> > Jan 12 11:58:15 cultivar0 libvirtd: 2018-01-12 15:58:15.217+0000: > >> >> > 1908: > >> >> > error : virLockManagerSanlockAcquire:1041 : resource busy: Failed > to > >> >> > acquire > >> >> > lock: Lease is held by another host > >> >> > > >> >> > ==> /var/log/libvirt/qemu/Cultivar.log <== > >> >> > 2018-01-12T15:58:15.219934Z qemu-kvm: terminating on signal 15 from > >> >> > pid > >> >> > 1773 > >> >> > (/usr/sbin/libvirtd) > >> >> > > >> >> > ==> /var/log/messages <== > >> >> > Jan 12 11:58:15 cultivar0 kernel: ovirtmgmt: port 6(vnet4) entered > >> >> > disabled > >> >> > state > >> >> > Jan 12 11:58:15 cultivar0 kernel: device vnet4 left promiscuous > mode > >> >> > Jan 12 11:58:15 cultivar0 kernel: ovirtmgmt: port 6(vnet4) entered > >> >> > disabled > >> >> > state > >> >> > Jan 12 11:58:15 cultivar0 NetworkManager[1092]: <info> > >> >> > [1515772695.2348] > >> >> > device (vnet4): state change: disconnected -> unmanaged (reason > >> >> > 'unmanaged') > >> >> > [30 10 3] > >> >> > Jan 12 11:58:15 cultivar0 NetworkManager[1092]: <info> > >> >> > [1515772695.2349] > >> >> > device (vnet4): released from master device ovirtmgmt > >> >> > Jan 12 11:58:15 cultivar0 kvm: 4 guests now active > >> >> > Jan 12 11:58:15 cultivar0 systemd-machined: Machine > qemu-119-Cultivar > >> >> > terminated. > >> >> > > >> >> > ==> /var/log/vdsm/vdsm.log <== > >> >> > vm/4013c829::ERROR::2018-01-12 > >> >> > 11:58:15,444::vm::914::virt.vm::(_startUnderlyingVm) > >> >> > (vmId='4013c829-c9d7-4b72-90d5-6fe58137504c') The vm start process > >> >> > failed > >> >> > Traceback (most recent call last): > >> >> > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line > 843, > >> >> > in > >> >> > _startUnderlyingVm > >> >> > self._run() > >> >> > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line > 2721, > >> >> > in > >> >> > _run > >> >> > dom.createWithFlags(flags) > >> >> > File "/usr/lib/python2.7/site-packages/vdsm/ > libvirtconnection.py", > >> >> > line > >> >> > 126, in wrapper > >> >> > ret = f(*args, **kwargs) > >> >> > File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line > 512, in > >> >> > wrapper > >> >> > return func(inst, *args, **kwargs) > >> >> > File "/usr/lib64/python2.7/site-packages/libvirt.py", line > 1069, in > >> >> > createWithFlags > >> >> > if ret == -1: raise libvirtError ('virDomainCreateWithFlags() > >> >> > failed', > >> >> > dom=self) > >> >> > libvirtError: resource busy: Failed to acquire lock: Lease is held > by > >> >> > another host > >> >> > jsonrpc/6::ERROR::2018-01-12 > >> >> > 11:58:16,421::__init__::611::jsonrpc.JsonRpcServer::(_ > handle_request) > >> >> > Internal server error > >> >> > Traceback (most recent call last): > >> >> > File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", > line > >> >> > 606, > >> >> > in _handle_request > >> >> > res = method(**params) > >> >> > File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line > >> >> > 201, > >> >> > in > >> >> > _dynamicMethod > >> >> > result = fn(*methodArgs) > >> >> > File "<string>", line 2, in getAllVmIoTunePolicies > >> >> > File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line > >> >> > 48, > >> >> > in > >> >> > method > >> >> > ret = func(*args, **kwargs) > >> >> > File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1354, > in > >> >> > getAllVmIoTunePolicies > >> >> > io_tune_policies_dict = self._cif.getAllVmIoTunePolicies() > >> >> > File "/usr/lib/python2.7/site-packages/vdsm/clientIF.py", line > 524, > >> >> > in > >> >> > getAllVmIoTunePolicies > >> >> > 'current_values': v.getIoTune()} > >> >> > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line > 3481, > >> >> > in > >> >> > getIoTune > >> >> > result = self.getIoTuneResponse() > >> >> > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line > 3500, > >> >> > in > >> >> > getIoTuneResponse > >> >> > res = self._dom.blockIoTune( > >> >> > File "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py", > >> >> > line > >> >> > 47, > >> >> > in __getattr__ > >> >> > % self.vmid) > >> >> > NotConnectedError: VM '4013c829-c9d7-4b72-90d5-6fe58137504c' was > not > >> >> > defined > >> >> > yet or was undefined > >> >> > > >> >> > ==> /var/log/messages <== > >> >> > Jan 12 11:58:16 cultivar0 journal: vdsm jsonrpc.JsonRpcServer ERROR > >> >> > Internal > >> >> > server error#012Traceback (most recent call last):#012 File > >> >> > "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line > 606, > >> >> > in > >> >> > _handle_request#012 res = method(**params)#012 File > >> >> > "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 201, > in > >> >> > _dynamicMethod#012 result = fn(*methodArgs)#012 File > "<string>", > >> >> > line 2, > >> >> > in getAllVmIoTunePolicies#012 File > >> >> > "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in > >> >> > method#012 ret = func(*args, **kwargs)#012 File > >> >> > "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1354, in > >> >> > getAllVmIoTunePolicies#012 io_tune_policies_dict = > >> >> > self._cif.getAllVmIoTunePolicies()#012 File > >> >> > "/usr/lib/python2.7/site-packages/vdsm/clientIF.py", line 524, in > >> >> > getAllVmIoTunePolicies#012 'current_values': v.getIoTune()}#012 > >> >> > File > >> >> > "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 3481, in > >> >> > getIoTune#012 result = self.getIoTuneResponse()#012 File > >> >> > "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 3500, in > >> >> > getIoTuneResponse#012 res = self._dom.blockIoTune(#012 File > >> >> > "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py", line > 47, > >> >> > in > >> >> > __getattr__#012 % self.vmid)#012NotConnectedError: VM > >> >> > '4013c829-c9d7-4b72-90d5-6fe58137504c' was not defined yet or was > >> >> > undefined > >> >> > > >> >> > On Fri, Jan 12, 2018 at 11:55 AM, Jayme <jaymef(a)gmail.com> wrote: > >> >> >> > >> >> >> One other tidbit I noticed is that it seems like there are less > >> >> >> errors > >> >> >> if > >> >> >> I started in paused mode: > >> >> >> > >> >> >> but still shows: Engine status : {"reason": > >> >> >> "bad > >> >> >> vm > >> >> >> status", "health": "bad", "vm": "up", "detail": "Paused"} > >> >> >> > >> >> >> ==> /var/log/messages <== > >> >> >> Jan 12 11:55:05 cultivar0 kernel: ovirtmgmt: port 6(vnet4) entered > >> >> >> blocking state > >> >> >> Jan 12 11:55:05 cultivar0 kernel: ovirtmgmt: port 6(vnet4) entered > >> >> >> disabled state > >> >> >> Jan 12 11:55:05 cultivar0 kernel: device vnet4 entered promiscuous > >> >> >> mode > >> >> >> Jan 12 11:55:05 cultivar0 kernel: ovirtmgmt: port 6(vnet4) entered > >> >> >> blocking state > >> >> >> Jan 12 11:55:05 cultivar0 kernel: ovirtmgmt: port 6(vnet4) entered > >> >> >> forwarding state > >> >> >> Jan 12 11:55:05 cultivar0 lldpad: recvfrom(Event interface): No > >> >> >> buffer > >> >> >> space available > >> >> >> Jan 12 11:55:05 cultivar0 NetworkManager[1092]: <info> > >> >> >> [1515772505.3625] > >> >> >> manager: (vnet4): new Tun device > >> >> >> (/org/freedesktop/NetworkManager/Devices/139) > >> >> >> > >> >> >> ==> /var/log/libvirt/qemu/Cultivar.log <== > >> >> >> 2018-01-12 15:55:05.370+0000: starting up libvirt version: 3.2.0, > >> >> >> package: > >> >> >> 14.el7_4.7 (CentOS BuildSystem <http://bugs.centos.org>, > >> >> >> 2018-01-04-19:31:34, c1bm.rdu2.centos.org) qemu version: > >> >> >> 2.9.0(qemu-kvm-ev-2.9.0-16.el7_4.13.1), hostname: > >> >> >> cultivar0.grove.silverorange.com > >> >> >> LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin > >> >> >> QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm -name > >> >> >> guest=Cultivar,debug-threads=on -S -object > >> >> >> > >> >> >> > >> >> >> secret,id=masterKey0,format=raw,file=/var/lib/libvirt/ > qemu/domain-118-Cultivar/master-key.aes > >> >> >> -machine pc-i440fx-rhel7.3.0,accel=kvm, > usb=off,dump-guest-core=off > >> >> >> -cpu > >> >> >> Conroe -m 8192 -realtime mlock=off -smp > >> >> >> 2,maxcpus=16,sockets=16,cores=1,threads=1 -uuid > >> >> >> 4013c829-c9d7-4b72-90d5-6fe58137504c -smbios > >> >> >> 'type=1,manufacturer=oVirt,product=oVirt > >> >> >> > >> >> >> > >> >> >> Node,version=7-4.1708.el7.centos,serial=44454C4C-3300- > 1042-8031-B4C04F4B4831,uuid=4013c829-c9d7-4b72-90d5-6fe58137504c' > >> >> >> -no-user-config -nodefaults -chardev > >> >> >> > >> >> >> > >> >> >> socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain- > 118-Cultivar/monitor.sock,server,nowait > >> >> >> -mon chardev=charmonitor,id=monitor,mode=control -rtc > >> >> >> base=2018-01-12T15:55:05,driftfix=slew -global > >> >> >> kvm-pit.lost_tick_policy=delay -no-hpet -no-reboot -boot strict=on > >> >> >> -device > >> >> >> piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device > >> >> >> virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x4 -drive > >> >> >> > >> >> >> > >> >> >> file=/var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/c2dde892-f978-4dfc-a421-c8e04cf387f9/23aa0a66-fa6c-4967-a1e5- > fbe47c0cd705,format=raw,if=none,id=drive-virtio-disk0, > serial=c2dde892-f978-4dfc-a421-c8e04cf387f9,cache=none, > werror=stop,rerror=stop,aio=threads > >> >> >> -device > >> >> >> > >> >> >> > >> >> >> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive- > virtio-disk0,id=virtio-disk0,bootindex=1 > >> >> >> -drive if=none,id=drive-ide0-1-0,readonly=on -device > >> >> >> ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev > >> >> >> tap,fd=35,id=hostnet0,vhost=on,vhostfd=38 -device > >> >> >> > >> >> >> > >> >> >> virtio-net-pci,netdev=hostnet0,id=net0,mac=00:16:3e: > 7f:d6:83,bus=pci.0,addr=0x3 > >> >> >> -chardev > >> >> >> > >> >> >> > >> >> >> socket,id=charchannel0,path=/var/lib/libvirt/qemu/channels/ > 4013c829-c9d7-4b72-90d5-6fe58137504c.com.redhat.rhevm.vdsm,server,nowait > >> >> >> -device > >> >> >> > >> >> >> > >> >> >> virtserialport,bus=virtio-serial0.0,nr=1,chardev= > charchannel0,id=channel0,name=com.redhat.rhevm.vdsm > >> >> >> -chardev > >> >> >> > >> >> >> > >> >> >> socket,id=charchannel1,path=/var/lib/libvirt/qemu/channels/ > 4013c829-c9d7-4b72-90d5-6fe58137504c.org.qemu.guest_agent.0,server,nowait > >> >> >> -device > >> >> >> > >> >> >> > >> >> >> virtserialport,bus=virtio-serial0.0,nr=2,chardev= > charchannel1,id=channel1,name=org.qemu.guest_agent.0 > >> >> >> -chardev spicevmc,id=charchannel2,name=vdagent -device > >> >> >> > >> >> >> > >> >> >> virtserialport,bus=virtio-serial0.0,nr=3,chardev= > charchannel2,id=channel2,name=com.redhat.spice.0 > >> >> >> -chardev > >> >> >> > >> >> >> > >> >> >> socket,id=charchannel3,path=/var/lib/libvirt/qemu/channels/ > 4013c829-c9d7-4b72-90d5-6fe58137504c.org.ovirt.hosted- > engine-setup.0,server,nowait > >> >> >> -device > >> >> >> > >> >> >> > >> >> >> virtserialport,bus=virtio-serial0.0,nr=4,chardev= > charchannel3,id=channel3,name=org.ovirt.hosted-engine-setup.0 > >> >> >> -chardev pty,id=charconsole0 -device > >> >> >> virtconsole,chardev=charconsole0,id=console0 -spice > >> >> >> > >> >> >> > >> >> >> tls-port=5904,addr=0,x509-dir=/etc/pki/vdsm/libvirt-spice, > tls-channel=default,seamless-migration=on > >> >> >> -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -object > >> >> >> rng-random,id=objrng0,filename=/dev/urandom -device > >> >> >> virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x5 -msg > >> >> >> timestamp=on > >> >> >> > >> >> >> ==> /var/log/messages <== > >> >> >> Jan 12 11:55:05 cultivar0 NetworkManager[1092]: <info> > >> >> >> [1515772505.3689] > >> >> >> device (vnet4): state change: unmanaged -> unavailable (reason > >> >> >> 'connection-assumed') [10 20 41] > >> >> >> Jan 12 11:55:05 cultivar0 NetworkManager[1092]: <info> > >> >> >> [1515772505.3702] > >> >> >> device (vnet4): state change: unavailable -> disconnected (reason > >> >> >> 'none') > >> >> >> [20 30 0] > >> >> >> Jan 12 11:55:05 cultivar0 systemd-machined: New machine > >> >> >> qemu-118-Cultivar. > >> >> >> Jan 12 11:55:05 cultivar0 systemd: Started Virtual Machine > >> >> >> qemu-118-Cultivar. > >> >> >> Jan 12 11:55:05 cultivar0 systemd: Starting Virtual Machine > >> >> >> qemu-118-Cultivar. > >> >> >> > >> >> >> ==> /var/log/libvirt/qemu/Cultivar.log <== > >> >> >> 2018-01-12T15:55:05.586827Z qemu-kvm: -chardev > pty,id=charconsole0: > >> >> >> char > >> >> >> device redirected to /dev/pts/1 (label charconsole0) > >> >> >> > >> >> >> ==> /var/log/messages <== > >> >> >> Jan 12 11:55:05 cultivar0 kvm: 5 guests now active > >> >> >> > >> >> >> On Fri, Jan 12, 2018 at 11:36 AM, Jayme <jaymef(a)gmail.com> wrote: > >> >> >>> > >> >> >>> Yeah I am in global maintenance: > >> >> >>> > >> >> >>> state=GlobalMaintenance > >> >> >>> > >> >> >>> host0: {"reason": "vm not running on this host", "health": > "bad", > >> >> >>> "vm": > >> >> >>> "down", "detail": "unknown"} > >> >> >>> host2: {"reason": "vm not running on this host", "health": "bad", > >> >> >>> "vm": > >> >> >>> "down", "detail": "unknown"} > >> >> >>> host3: {"reason": "vm not running on this host", "health": "bad", > >> >> >>> "vm": > >> >> >>> "down", "detail": "unknown"} > >> >> >>> > >> >> >>> I understand the lock is an issue, I'll try to make sure it is > >> >> >>> fully > >> >> >>> stopped on all three before starting but I don't think that is > the > >> >> >>> issue at > >> >> >>> hand either. What concerns me is mostly that it seems to be > >> >> >>> unable > >> >> >>> to read > >> >> >>> the meta data, I think that might be the heart of the problem but > >> >> >>> I'm > >> >> >>> not > >> >> >>> sure what is causing it. > >> >> >>> > >> >> >>> On Fri, Jan 12, 2018 at 11:33 AM, Martin Sivak < > msivak(a)redhat.com> > >> >> >>> wrote: > >> >> >>>> > >> >> >>>> > On all three hosts I ran hosted-engine --vm-shutdown; > >> >> >>>> > hosted-engine > >> >> >>>> > --vm-poweroff > >> >> >>>> > >> >> >>>> Are you in global maintenance? I think you were in one of the > >> >> >>>> previous > >> >> >>>> emails, but worth checking. > >> >> >>>> > >> >> >>>> > I started ovirt-ha-broker with systemctl as root user but it > >> >> >>>> > does > >> >> >>>> > appear to be running under vdsm: > >> >> >>>> > >> >> >>>> That is the correct behavior. > >> >> >>>> > >> >> >>>> > libvirtError: resource busy: Failed to acquire lock: Lease is > >> >> >>>> > held > >> >> >>>> > by > >> >> >>>> > another host > >> >> >>>> > >> >> >>>> sanlock seems to think the VM runs somewhere and it is possible > >> >> >>>> that > >> >> >>>> some other host tried to start the VM as well unless you are in > >> >> >>>> global > >> >> >>>> maintenance (that is why I asked the first question here). > >> >> >>>> > >> >> >>>> Martin > >> >> >>>> > >> >> >>>> On Fri, Jan 12, 2018 at 4:28 PM, Jayme <jaymef(a)gmail.com> > wrote: > >> >> >>>> > Martin, > >> >> >>>> > > >> >> >>>> > Thanks so much for keeping with me, this is driving me > crazy! I > >> >> >>>> > really do > >> >> >>>> > appreciate it, thanks again > >> >> >>>> > > >> >> >>>> > Let's go through this: > >> >> >>>> > > >> >> >>>> > HE VM is down - YES > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > HE agent fails when opening metadata using the symlink - YES > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > the symlink is there and readable by vdsm:kvm - it appears to > >> >> >>>> > be: > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > lrwxrwxrwx. 1 vdsm kvm 159 Jan 10 21:20 > >> >> >>>> > 14a20941-1b84-4b82-be8f-ace38d7c037a > >> >> >>>> > -> > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > /rhev/data-center/mnt/cultivar0.grove.silverorange. > com:_exports_hosted__engine/248f46f0-d793-4581-9810- > c9d965e2f286/images/14a20941-1b84-4b82-be8f-ace38d7c037a > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > And the files in the linked directory exist and have vdsm:kvm > >> >> >>>> > perms > >> >> >>>> > as > >> >> >>>> > well: > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > # cd > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > /rhev/data-center/mnt/cultivar0.grove.silverorange. > com:_exports_hosted__engine/248f46f0-d793-4581-9810- > c9d965e2f286/images/14a20941-1b84-4b82-be8f-ace38d7c037a > >> >> >>>> > > >> >> >>>> > [root@cultivar0 14a20941-1b84-4b82-be8f-ace38d7c037a]# ls -al > >> >> >>>> > > >> >> >>>> > total 2040 > >> >> >>>> > > >> >> >>>> > drwxr-xr-x. 2 vdsm kvm 4096 Jan 12 10:51 . > >> >> >>>> > > >> >> >>>> > drwxr-xr-x. 8 vdsm kvm 4096 Feb 3 2016 .. > >> >> >>>> > > >> >> >>>> > -rw-rw----. 1 vdsm kvm 1028096 Jan 12 11:19 > >> >> >>>> > 8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8 > >> >> >>>> > > >> >> >>>> > -rw-rw----. 1 vdsm kvm 1048576 Feb 3 2016 > >> >> >>>> > 8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8.lease > >> >> >>>> > > >> >> >>>> > -rw-r--r--. 1 vdsm kvm 283 Feb 3 2016 > >> >> >>>> > 8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8.meta > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > I started ovirt-ha-broker with systemctl as root user but it > >> >> >>>> > does > >> >> >>>> > appear to > >> >> >>>> > be running under vdsm: > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > vdsm 16928 0.6 0.0 1618244 43328 ? Ssl 10:33 > 0:18 > >> >> >>>> > /usr/bin/python > >> >> >>>> > /usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > Here is something I tried: > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > - On all three hosts I ran hosted-engine --vm-shutdown; > >> >> >>>> > hosted-engine > >> >> >>>> > --vm-poweroff > >> >> >>>> > > >> >> >>>> > - On HOST0 (cultivar0) I disconnected and reconnected storage > >> >> >>>> > using > >> >> >>>> > hosted-engine > >> >> >>>> > > >> >> >>>> > - Tried starting up the hosted VM on cultivar0 while tailing > the > >> >> >>>> > logs: > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > # hosted-engine --vm-start > >> >> >>>> > > >> >> >>>> > VM exists and is down, cleaning up and restarting > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > ==> /var/log/vdsm/vdsm.log <== > >> >> >>>> > > >> >> >>>> > jsonrpc/2::ERROR::2018-01-12 > >> >> >>>> > 11:27:27,194::vm::1766::virt.vm::(_getRunningVmStats) > >> >> >>>> > (vmId='4013c829-c9d7-4b72-90d5-6fe58137504c') Error fetching > vm > >> >> >>>> > stats > >> >> >>>> > > >> >> >>>> > Traceback (most recent call last): > >> >> >>>> > > >> >> >>>> > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", > line > >> >> >>>> > 1762, > >> >> >>>> > in > >> >> >>>> > _getRunningVmStats > >> >> >>>> > > >> >> >>>> > vm_sample.interval) > >> >> >>>> > > >> >> >>>> > File "/usr/lib/python2.7/site- > packages/vdsm/virt/vmstats.py", > >> >> >>>> > line > >> >> >>>> > 45, in > >> >> >>>> > produce > >> >> >>>> > > >> >> >>>> > networks(vm, stats, first_sample, last_sample, interval) > >> >> >>>> > > >> >> >>>> > File "/usr/lib/python2.7/site- > packages/vdsm/virt/vmstats.py", > >> >> >>>> > line > >> >> >>>> > 322, in > >> >> >>>> > networks > >> >> >>>> > > >> >> >>>> > if nic.name.startswith('hostdev'): > >> >> >>>> > > >> >> >>>> > AttributeError: name > >> >> >>>> > > >> >> >>>> > jsonrpc/3::ERROR::2018-01-12 > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > 11:27:27,221::__init__::611::jsonrpc.JsonRpcServer::(_ > handle_request) > >> >> >>>> > Internal server error > >> >> >>>> > > >> >> >>>> > Traceback (most recent call last): > >> >> >>>> > > >> >> >>>> > File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__. > py", > >> >> >>>> > line > >> >> >>>> > 606, > >> >> >>>> > in _handle_request > >> >> >>>> > > >> >> >>>> > res = method(**params) > >> >> >>>> > > >> >> >>>> > File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", > >> >> >>>> > line > >> >> >>>> > 201, in > >> >> >>>> > _dynamicMethod > >> >> >>>> > > >> >> >>>> > result = fn(*methodArgs) > >> >> >>>> > > >> >> >>>> > File "<string>", line 2, in getAllVmIoTunePolicies > >> >> >>>> > > >> >> >>>> > File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", > >> >> >>>> > line > >> >> >>>> > 48, > >> >> >>>> > in > >> >> >>>> > method > >> >> >>>> > > >> >> >>>> > ret = func(*args, **kwargs) > >> >> >>>> > > >> >> >>>> > File "/usr/lib/python2.7/site-packages/vdsm/API.py", line > >> >> >>>> > 1354, > >> >> >>>> > in > >> >> >>>> > getAllVmIoTunePolicies > >> >> >>>> > > >> >> >>>> > io_tune_policies_dict = self._cif. > getAllVmIoTunePolicies() > >> >> >>>> > > >> >> >>>> > File "/usr/lib/python2.7/site-packages/vdsm/clientIF.py", > line > >> >> >>>> > 524, > >> >> >>>> > in > >> >> >>>> > getAllVmIoTunePolicies > >> >> >>>> > > >> >> >>>> > 'current_values': v.getIoTune()} > >> >> >>>> > > >> >> >>>> > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", > line > >> >> >>>> > 3481, > >> >> >>>> > in > >> >> >>>> > getIoTune > >> >> >>>> > > >> >> >>>> > result = self.getIoTuneResponse() > >> >> >>>> > > >> >> >>>> > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", > line > >> >> >>>> > 3500, > >> >> >>>> > in > >> >> >>>> > getIoTuneResponse > >> >> >>>> > > >> >> >>>> > res = self._dom.blockIoTune( > >> >> >>>> > > >> >> >>>> > File > >> >> >>>> > "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py", > >> >> >>>> > line > >> >> >>>> > 47, > >> >> >>>> > in __getattr__ > >> >> >>>> > > >> >> >>>> > % self.vmid) > >> >> >>>> > > >> >> >>>> > NotConnectedError: VM '4013c829-c9d7-4b72-90d5-6fe58137504c' > was > >> >> >>>> > not > >> >> >>>> > defined > >> >> >>>> > yet or was undefined > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > ==> /var/log/messages <== > >> >> >>>> > > >> >> >>>> > Jan 12 11:27:27 cultivar0 journal: vdsm jsonrpc.JsonRpcServer > >> >> >>>> > ERROR > >> >> >>>> > Internal > >> >> >>>> > server error#012Traceback (most recent call last):#012 File > >> >> >>>> > "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", > line > >> >> >>>> > 606, > >> >> >>>> > in > >> >> >>>> > _handle_request#012 res = method(**params)#012 File > >> >> >>>> > "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line > 201, > >> >> >>>> > in > >> >> >>>> > _dynamicMethod#012 result = fn(*methodArgs)#012 File > >> >> >>>> > "<string>", > >> >> >>>> > line 2, > >> >> >>>> > in getAllVmIoTunePolicies#012 File > >> >> >>>> > "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line > 48, > >> >> >>>> > in > >> >> >>>> > method#012 ret = func(*args, **kwargs)#012 File > >> >> >>>> > "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1354, in > >> >> >>>> > getAllVmIoTunePolicies#012 io_tune_policies_dict = > >> >> >>>> > self._cif.getAllVmIoTunePolicies()#012 File > >> >> >>>> > "/usr/lib/python2.7/site-packages/vdsm/clientIF.py", line > 524, > >> >> >>>> > in > >> >> >>>> > getAllVmIoTunePolicies#012 'current_values': > >> >> >>>> > v.getIoTune()}#012 > >> >> >>>> > File > >> >> >>>> > "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line > 3481, > >> >> >>>> > in > >> >> >>>> > getIoTune#012 result = self.getIoTuneResponse()#012 File > >> >> >>>> > "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line > 3500, > >> >> >>>> > in > >> >> >>>> > getIoTuneResponse#012 res = self._dom.blockIoTune(#012 > File > >> >> >>>> > "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py", > line > >> >> >>>> > 47, > >> >> >>>> > in > >> >> >>>> > __getattr__#012 % self.vmid)#012NotConnectedError: VM > >> >> >>>> > '4013c829-c9d7-4b72-90d5-6fe58137504c' was not defined yet or > >> >> >>>> > was > >> >> >>>> > undefined > >> >> >>>> > > >> >> >>>> > Jan 12 11:27:27 cultivar0 kernel: ovirtmgmt: port 6(vnet4) > >> >> >>>> > entered > >> >> >>>> > blocking > >> >> >>>> > state > >> >> >>>> > > >> >> >>>> > Jan 12 11:27:27 cultivar0 kernel: ovirtmgmt: port 6(vnet4) > >> >> >>>> > entered > >> >> >>>> > disabled > >> >> >>>> > state > >> >> >>>> > > >> >> >>>> > Jan 12 11:27:27 cultivar0 kernel: device vnet4 entered > >> >> >>>> > promiscuous > >> >> >>>> > mode > >> >> >>>> > > >> >> >>>> > Jan 12 11:27:27 cultivar0 kernel: ovirtmgmt: port 6(vnet4) > >> >> >>>> > entered > >> >> >>>> > blocking > >> >> >>>> > state > >> >> >>>> > > >> >> >>>> > Jan 12 11:27:27 cultivar0 kernel: ovirtmgmt: port 6(vnet4) > >> >> >>>> > entered > >> >> >>>> > forwarding state > >> >> >>>> > > >> >> >>>> > Jan 12 11:27:27 cultivar0 lldpad: recvfrom(Event interface): > No > >> >> >>>> > buffer > >> >> >>>> > space > >> >> >>>> > available > >> >> >>>> > > >> >> >>>> > Jan 12 11:27:27 cultivar0 NetworkManager[1092]: <info> > >> >> >>>> > [1515770847.4264] > >> >> >>>> > manager: (vnet4): new Tun device > >> >> >>>> > (/org/freedesktop/NetworkManager/Devices/135) > >> >> >>>> > > >> >> >>>> > Jan 12 11:27:27 cultivar0 NetworkManager[1092]: <info> > >> >> >>>> > [1515770847.4342] > >> >> >>>> > device (vnet4): state change: unmanaged -> unavailable (reason > >> >> >>>> > 'connection-assumed') [10 20 41] > >> >> >>>> > > >> >> >>>> > Jan 12 11:27:27 cultivar0 NetworkManager[1092]: <info> > >> >> >>>> > [1515770847.4353] > >> >> >>>> > device (vnet4): state change: unavailable -> disconnected > >> >> >>>> > (reason > >> >> >>>> > 'none') > >> >> >>>> > [20 30 0] > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > ==> /var/log/libvirt/qemu/Cultivar.log <== > >> >> >>>> > > >> >> >>>> > 2018-01-12 15:27:27.435+0000: starting up libvirt version: > >> >> >>>> > 3.2.0, > >> >> >>>> > package: > >> >> >>>> > 14.el7_4.7 (CentOS BuildSystem <http://bugs.centos.org>, > >> >> >>>> > 2018-01-04-19:31:34, c1bm.rdu2.centos.org) qemu version: > >> >> >>>> > 2.9.0(qemu-kvm-ev-2.9.0-16.el7_4.13.1), hostname: > >> >> >>>> > cultivar0.grove.silverorange.com > >> >> >>>> > > >> >> >>>> > LC_ALL=C PATH=/usr/local/sbin:/usr/ > local/bin:/usr/sbin:/usr/bin > >> >> >>>> > QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm -name > >> >> >>>> > guest=Cultivar,debug-threads=on -S -object > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > secret,id=masterKey0,format=raw,file=/var/lib/libvirt/ > qemu/domain-114-Cultivar/master-key.aes > >> >> >>>> > -machine > >> >> >>>> > pc-i440fx-rhel7.3.0,accel=kvm,usb=off,dump-guest-core=off > >> >> >>>> > -cpu > >> >> >>>> > Conroe -m 8192 -realtime mlock=off -smp > >> >> >>>> > 2,maxcpus=16,sockets=16,cores=1,threads=1 -uuid > >> >> >>>> > 4013c829-c9d7-4b72-90d5-6fe58137504c -smbios > >> >> >>>> > 'type=1,manufacturer=oVirt,product=oVirt > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > Node,version=7-4.1708.el7.centos,serial=44454C4C-3300- > 1042-8031-B4C04F4B4831,uuid=4013c829-c9d7-4b72-90d5-6fe58137504c' > >> >> >>>> > -no-user-config -nodefaults -chardev > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain- > 114-Cultivar/monitor.sock,server,nowait > >> >> >>>> > -mon chardev=charmonitor,id=monitor,mode=control -rtc > >> >> >>>> > base=2018-01-12T15:27:27,driftfix=slew -global > >> >> >>>> > kvm-pit.lost_tick_policy=delay -no-hpet -no-reboot -boot > >> >> >>>> > strict=on > >> >> >>>> > -device > >> >> >>>> > piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device > >> >> >>>> > virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x4 -drive > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > file=/var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/c2dde892-f978-4dfc-a421-c8e04cf387f9/23aa0a66-fa6c-4967-a1e5- > fbe47c0cd705,format=raw,if=none,id=drive-virtio-disk0, > serial=c2dde892-f978-4dfc-a421-c8e04cf387f9,cache=none, > werror=stop,rerror=stop,aio=threads > >> >> >>>> > -device > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive- > virtio-disk0,id=virtio-disk0,bootindex=1 > >> >> >>>> > -drive if=none,id=drive-ide0-1-0,readonly=on -device > >> >> >>>> > ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 > -netdev > >> >> >>>> > tap,fd=35,id=hostnet0,vhost=on,vhostfd=38 -device > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > virtio-net-pci,netdev=hostnet0,id=net0,mac=00:16:3e: > 7f:d6:83,bus=pci.0,addr=0x3 > >> >> >>>> > -chardev > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > socket,id=charchannel0,path=/var/lib/libvirt/qemu/channels/ > 4013c829-c9d7-4b72-90d5-6fe58137504c.com.redhat.rhevm.vdsm,server,nowait > >> >> >>>> > -device > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > virtserialport,bus=virtio-serial0.0,nr=1,chardev= > charchannel0,id=channel0,name=com.redhat.rhevm.vdsm > >> >> >>>> > -chardev > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > socket,id=charchannel1,path=/var/lib/libvirt/qemu/channels/ > 4013c829-c9d7-4b72-90d5-6fe58137504c.org.qemu.guest_agent.0,server,nowait > >> >> >>>> > -device > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > virtserialport,bus=virtio-serial0.0,nr=2,chardev= > charchannel1,id=channel1,name=org.qemu.guest_agent.0 > >> >> >>>> > -chardev spicevmc,id=charchannel2,name=vdagent -device > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > virtserialport,bus=virtio-serial0.0,nr=3,chardev= > charchannel2,id=channel2,name=com.redhat.spice.0 > >> >> >>>> > -chardev > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > socket,id=charchannel3,path=/var/lib/libvirt/qemu/channels/ > 4013c829-c9d7-4b72-90d5-6fe58137504c.org.ovirt.hosted- > engine-setup.0,server,nowait > >> >> >>>> > -device > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > virtserialport,bus=virtio-serial0.0,nr=4,chardev= > charchannel3,id=channel3,name=org.ovirt.hosted-engine-setup.0 > >> >> >>>> > -chardev pty,id=charconsole0 -device > >> >> >>>> > virtconsole,chardev=charconsole0,id=console0 -spice > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > tls-port=5904,addr=0,x509-dir=/etc/pki/vdsm/libvirt-spice, > tls-channel=default,seamless-migration=on > >> >> >>>> > -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -object > >> >> >>>> > rng-random,id=objrng0,filename=/dev/urandom -device > >> >> >>>> > virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x5 -msg > >> >> >>>> > timestamp=on > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > ==> /var/log/messages <== > >> >> >>>> > > >> >> >>>> > Jan 12 11:27:27 cultivar0 systemd-machined: New machine > >> >> >>>> > qemu-114-Cultivar. > >> >> >>>> > > >> >> >>>> > Jan 12 11:27:27 cultivar0 systemd: Started Virtual Machine > >> >> >>>> > qemu-114-Cultivar. > >> >> >>>> > > >> >> >>>> > Jan 12 11:27:27 cultivar0 systemd: Starting Virtual Machine > >> >> >>>> > qemu-114-Cultivar. > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > ==> /var/log/libvirt/qemu/Cultivar.log <== > >> >> >>>> > > >> >> >>>> > 2018-01-12T15:27:27.651669Z qemu-kvm: -chardev > >> >> >>>> > pty,id=charconsole0: > >> >> >>>> > char > >> >> >>>> > device redirected to /dev/pts/2 (label charconsole0) > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > ==> /var/log/messages <== > >> >> >>>> > > >> >> >>>> > Jan 12 11:27:27 cultivar0 kvm: 5 guests now active > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > ==> /var/log/libvirt/qemu/Cultivar.log <== > >> >> >>>> > > >> >> >>>> > 2018-01-12 15:27:27.773+0000: shutting down, reason=failed > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > ==> /var/log/messages <== > >> >> >>>> > > >> >> >>>> > Jan 12 11:27:27 cultivar0 libvirtd: 2018-01-12 > >> >> >>>> > 15:27:27.773+0000: > >> >> >>>> > 1910: > >> >> >>>> > error : virLockManagerSanlockAcquire:1041 : resource busy: > >> >> >>>> > Failed > >> >> >>>> > to > >> >> >>>> > acquire > >> >> >>>> > lock: Lease is held by another host > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > ==> /var/log/libvirt/qemu/Cultivar.log <== > >> >> >>>> > > >> >> >>>> > 2018-01-12T15:27:27.776135Z qemu-kvm: terminating on signal 15 > >> >> >>>> > from > >> >> >>>> > pid 1773 > >> >> >>>> > (/usr/sbin/libvirtd) > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > ==> /var/log/messages <== > >> >> >>>> > > >> >> >>>> > Jan 12 11:27:27 cultivar0 kernel: ovirtmgmt: port 6(vnet4) > >> >> >>>> > entered > >> >> >>>> > disabled > >> >> >>>> > state > >> >> >>>> > > >> >> >>>> > Jan 12 11:27:27 cultivar0 kernel: device vnet4 left > promiscuous > >> >> >>>> > mode > >> >> >>>> > > >> >> >>>> > Jan 12 11:27:27 cultivar0 kernel: ovirtmgmt: port 6(vnet4) > >> >> >>>> > entered > >> >> >>>> > disabled > >> >> >>>> > state > >> >> >>>> > > >> >> >>>> > Jan 12 11:27:27 cultivar0 NetworkManager[1092]: <info> > >> >> >>>> > [1515770847.7989] > >> >> >>>> > device (vnet4): state change: disconnected -> unmanaged > (reason > >> >> >>>> > 'unmanaged') > >> >> >>>> > [30 10 3] > >> >> >>>> > > >> >> >>>> > Jan 12 11:27:27 cultivar0 NetworkManager[1092]: <info> > >> >> >>>> > [1515770847.7989] > >> >> >>>> > device (vnet4): released from master device ovirtmgmt > >> >> >>>> > > >> >> >>>> > Jan 12 11:27:27 cultivar0 kvm: 4 guests now active > >> >> >>>> > > >> >> >>>> > Jan 12 11:27:27 cultivar0 systemd-machined: Machine > >> >> >>>> > qemu-114-Cultivar > >> >> >>>> > terminated. > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > ==> /var/log/vdsm/vdsm.log <== > >> >> >>>> > > >> >> >>>> > vm/4013c829::ERROR::2018-01-12 > >> >> >>>> > 11:27:28,001::vm::914::virt.vm::(_startUnderlyingVm) > >> >> >>>> > (vmId='4013c829-c9d7-4b72-90d5-6fe58137504c') The vm start > >> >> >>>> > process > >> >> >>>> > failed > >> >> >>>> > > >> >> >>>> > Traceback (most recent call last): > >> >> >>>> > > >> >> >>>> > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", > line > >> >> >>>> > 843, > >> >> >>>> > in > >> >> >>>> > _startUnderlyingVm > >> >> >>>> > > >> >> >>>> > self._run() > >> >> >>>> > > >> >> >>>> > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", > line > >> >> >>>> > 2721, > >> >> >>>> > in > >> >> >>>> > _run > >> >> >>>> > > >> >> >>>> > dom.createWithFlags(flags) > >> >> >>>> > > >> >> >>>> > File > >> >> >>>> > "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", > >> >> >>>> > line > >> >> >>>> > 126, in wrapper > >> >> >>>> > > >> >> >>>> > ret = f(*args, **kwargs) > >> >> >>>> > > >> >> >>>> > File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line > >> >> >>>> > 512, > >> >> >>>> > in > >> >> >>>> > wrapper > >> >> >>>> > > >> >> >>>> > return func(inst, *args, **kwargs) > >> >> >>>> > > >> >> >>>> > File "/usr/lib64/python2.7/site-packages/libvirt.py", line > >> >> >>>> > 1069, > >> >> >>>> > in > >> >> >>>> > createWithFlags > >> >> >>>> > > >> >> >>>> > if ret == -1: raise libvirtError > >> >> >>>> > ('virDomainCreateWithFlags() > >> >> >>>> > failed', > >> >> >>>> > dom=self) > >> >> >>>> > > >> >> >>>> > libvirtError: resource busy: Failed to acquire lock: Lease is > >> >> >>>> > held > >> >> >>>> > by > >> >> >>>> > another host > >> >> >>>> > > >> >> >>>> > periodic/47::ERROR::2018-01-12 > >> >> >>>> > 11:27:32,858::periodic::215::virt.periodic.Operation::(__ > call__) > >> >> >>>> > <vdsm.virt.sampling.VMBulkstatsMonitor object at 0x3692590> > >> >> >>>> > operation > >> >> >>>> > failed > >> >> >>>> > > >> >> >>>> > Traceback (most recent call last): > >> >> >>>> > > >> >> >>>> > File "/usr/lib/python2.7/site-packages/vdsm/virt/periodic. > py", > >> >> >>>> > line > >> >> >>>> > 213, > >> >> >>>> > in __call__ > >> >> >>>> > > >> >> >>>> > self._func() > >> >> >>>> > > >> >> >>>> > File "/usr/lib/python2.7/site-packages/vdsm/virt/sampling. > py", > >> >> >>>> > line > >> >> >>>> > 522, > >> >> >>>> > in __call__ > >> >> >>>> > > >> >> >>>> > self._send_metrics() > >> >> >>>> > > >> >> >>>> > File "/usr/lib/python2.7/site-packages/vdsm/virt/sampling. > py", > >> >> >>>> > line > >> >> >>>> > 538, > >> >> >>>> > in _send_metrics > >> >> >>>> > > >> >> >>>> > vm_sample.interval) > >> >> >>>> > > >> >> >>>> > File "/usr/lib/python2.7/site- > packages/vdsm/virt/vmstats.py", > >> >> >>>> > line > >> >> >>>> > 45, in > >> >> >>>> > produce > >> >> >>>> > > >> >> >>>> > networks(vm, stats, first_sample, last_sample, interval) > >> >> >>>> > > >> >> >>>> > File "/usr/lib/python2.7/site- > packages/vdsm/virt/vmstats.py", > >> >> >>>> > line > >> >> >>>> > 322, in > >> >> >>>> > networks > >> >> >>>> > > >> >> >>>> > if nic.name.startswith('hostdev'): > >> >> >>>> > > >> >> >>>> > AttributeError: name > >> >> >>>> > > >> >> >>>> > > >> >> >>>> > On Fri, Jan 12, 2018 at 11:14 AM, Martin Sivak > >> >> >>>> > <msivak(a)redhat.com> > >> >> >>>> > wrote: > >> >> >>>> >> > >> >> >>>> >> Hmm that rules out most of NFS related permission issues. > >> >> >>>> >> > >> >> >>>> >> So the current status is (I need to sum it up to get the full > >> >> >>>> >> picture): > >> >> >>>> >> > >> >> >>>> >> - HE VM is down > >> >> >>>> >> - HE agent fails when opening metadata using the symlink > >> >> >>>> >> - the symlink is there > >> >> >>>> >> - the symlink is readable by vdsm:kvm > >> >> >>>> >> > >> >> >>>> >> Hmm can you check under which user is ovirt-ha-broker > started? > >> >> >>>> >> > >> >> >>>> >> Martin > >> >> >>>> >> > >> >> >>>> >> > >> >> >>>> >> On Fri, Jan 12, 2018 at 4:10 PM, Jayme <jaymef(a)gmail.com> > >> >> >>>> >> wrote: > >> >> >>>> >> > Same thing happens with data images of other VMs as well > >> >> >>>> >> > though, > >> >> >>>> >> > and > >> >> >>>> >> > those > >> >> >>>> >> > seem to be running ok so I'm not sure if it's the problem. > >> >> >>>> >> > > >> >> >>>> >> > On Fri, Jan 12, 2018 at 11:08 AM, Jayme <jaymef(a)gmail.com> > >> >> >>>> >> > wrote: > >> >> >>>> >> >> > >> >> >>>> >> >> Martin, > >> >> >>>> >> >> > >> >> >>>> >> >> I can as VDSM user but not as root . I get permission > denied > >> >> >>>> >> >> trying to > >> >> >>>> >> >> touch one of the files as root, is that normal? > >> >> >>>> >> >> > >> >> >>>> >> >> On Fri, Jan 12, 2018 at 11:03 AM, Martin Sivak > >> >> >>>> >> >> <msivak(a)redhat.com> > >> >> >>>> >> >> wrote: > >> >> >>>> >> >>> > >> >> >>>> >> >>> Hmm, then it might be a permission issue indeed. Can you > >> >> >>>> >> >>> touch > >> >> >>>> >> >>> the > >> >> >>>> >> >>> file? Open it? (try hexdump) Just to make sure NFS does > not > >> >> >>>> >> >>> prevent > >> >> >>>> >> >>> you from doing that. > >> >> >>>> >> >>> > >> >> >>>> >> >>> Martin > >> >> >>>> >> >>> > >> >> >>>> >> >>> On Fri, Jan 12, 2018 at 3:57 PM, Jayme <jaymef(a)gmail.com > > > >> >> >>>> >> >>> wrote: > >> >> >>>> >> >>> > Sorry, I think we got confused about the symlink, there > >> >> >>>> >> >>> > are > >> >> >>>> >> >>> > symlinks > >> >> >>>> >> >>> > in > >> >> >>>> >> >>> > /var/run that point the /rhev when I was doing an LS it > >> >> >>>> >> >>> > was > >> >> >>>> >> >>> > listing > >> >> >>>> >> >>> > the > >> >> >>>> >> >>> > files in /rhev > >> >> >>>> >> >>> > > >> >> >>>> >> >>> > > >> >> >>>> >> >>> > /var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286 > >> >> >>>> >> >>> > > >> >> >>>> >> >>> > 14a20941-1b84-4b82-be8f-ace38d7c037a -> > >> >> >>>> >> >>> > > >> >> >>>> >> >>> > > >> >> >>>> >> >>> > > >> >> >>>> >> >>> > > >> >> >>>> >> >>> > > >> >> >>>> >> >>> > /rhev/data-center/mnt/cultivar0.grove.silverorange. > com:_exports_hosted__engine/248f46f0-d793-4581-9810- > c9d965e2f286/images/14a20941-1b84-4b82-be8f-ace38d7c037a > >> >> >>>> >> >>> > > >> >> >>>> >> >>> > ls -al > >> >> >>>> >> >>> > > >> >> >>>> >> >>> > > >> >> >>>> >> >>> > > >> >> >>>> >> >>> > > >> >> >>>> >> >>> > > >> >> >>>> >> >>> > /rhev/data-center/mnt/cultivar0.grove.silverorange. > com:_exports_hosted__engine/248f46f0-d793-4581-9810- > c9d965e2f286/images/14a20941-1b84-4b82-be8f-ace38d7c037a > >> >> >>>> >> >>> > total 2040 > >> >> >>>> >> >>> > drwxr-xr-x. 2 vdsm kvm 4096 Jan 12 10:51 . > >> >> >>>> >> >>> > drwxr-xr-x. 8 vdsm kvm 4096 Feb 3 2016 .. > >> >> >>>> >> >>> > -rw-rw----. 1 vdsm kvm 1028096 Jan 12 10:56 > >> >> >>>> >> >>> > 8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8 > >> >> >>>> >> >>> > -rw-rw----. 1 vdsm kvm 1048576 Feb 3 2016 > >> >> >>>> >> >>> > 8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8.lease > >> >> >>>> >> >>> > -rw-r--r--. 1 vdsm kvm 283 Feb 3 2016 > >> >> >>>> >> >>> > 8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8.meta > >> >> >>>> >> >>> > > >> >> >>>> >> >>> > Is it possible that this is the wrong image for hosted > >> >> >>>> >> >>> > engine? > >> >> >>>> >> >>> > > >> >> >>>> >> >>> > this is all I get in vdsm log when running > hosted-engine > >> >> >>>> >> >>> > --connect-storage > >> >> >>>> >> >>> > > >> >> >>>> >> >>> > jsonrpc/4::ERROR::2018-01-12 > >> >> >>>> >> >>> > > >> >> >>>> >> >>> > > >> >> >>>> >> >>> > > >> >> >>>> >> >>> > > >> >> >>>> >> >>> > 10:52:53,019::__init__::611::jsonrpc.JsonRpcServer::(_ > handle_request) > >> >> >>>> >> >>> > Internal server error > >> >> >>>> >> >>> > Traceback (most recent call last): > >> >> >>>> >> >>> > File > >> >> >>>> >> >>> > "/usr/lib/python2.7/site-packages/yajsonrpc/__init__. > py", > >> >> >>>> >> >>> > line > >> >> >>>> >> >>> > 606, > >> >> >>>> >> >>> > in _handle_request > >> >> >>>> >> >>> > res = method(**params) > >> >> >>>> >> >>> > File > >> >> >>>> >> >>> > "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", > >> >> >>>> >> >>> > line > >> >> >>>> >> >>> > 201, > >> >> >>>> >> >>> > in > >> >> >>>> >> >>> > _dynamicMethod > >> >> >>>> >> >>> > result = fn(*methodArgs) > >> >> >>>> >> >>> > File "<string>", line 2, in getAllVmIoTunePolicies > >> >> >>>> >> >>> > File > >> >> >>>> >> >>> > "/usr/lib/python2.7/site-packages/vdsm/common/api.py", > >> >> >>>> >> >>> > line > >> >> >>>> >> >>> > 48, > >> >> >>>> >> >>> > in > >> >> >>>> >> >>> > method > >> >> >>>> >> >>> > ret = func(*args, **kwargs) > >> >> >>>> >> >>> > File "/usr/lib/python2.7/site-packages/vdsm/API.py", > >> >> >>>> >> >>> > line > >> >> >>>> >> >>> > 1354, in > >> >> >>>> >> >>> > getAllVmIoTunePolicies > >> >> >>>> >> >>> > io_tune_policies_dict = > >> >> >>>> >> >>> > self._cif.getAllVmIoTunePolicies() > >> >> >>>> >> >>> > File > >> >> >>>> >> >>> > "/usr/lib/python2.7/site-packages/vdsm/clientIF.py", > >> >> >>>> >> >>> > line > >> >> >>>> >> >>> > 524, > >> >> >>>> >> >>> > in > >> >> >>>> >> >>> > getAllVmIoTunePolicies > >> >> >>>> >> >>> > 'current_values': v.getIoTune()} > >> >> >>>> >> >>> > File > >> >> >>>> >> >>> > "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", > >> >> >>>> >> >>> > line > >> >> >>>> >> >>> > 3481, > >> >> >>>> >> >>> > in > >> >> >>>> >> >>> > getIoTune > >> >> >>>> >> >>> > result = self.getIoTuneResponse() > >> >> >>>> >> >>> > File > >> >> >>>> >> >>> > "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", > >> >> >>>> >> >>> > line > >> >> >>>> >> >>> > 3500, > >> >> >>>> >> >>> > in > >> >> >>>> >> >>> > getIoTuneResponse > >> >> >>>> >> >>> > res = self._dom.blockIoTune( > >> >> >>>> >> >>> > File > >> >> >>>> >> >>> > > >> >> >>>> >> >>> > "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain. > py", > >> >> >>>> >> >>> > line > >> >> >>>> >> >>> > 47, > >> >> >>>> >> >>> > in __getattr__ > >> >> >>>> >> >>> > % self.vmid) > >> >> >>>> >> >>> > NotConnectedError: VM > >> >> >>>> >> >>> > '4013c829-c9d7-4b72-90d5-6fe58137504c' > >> >> >>>> >> >>> > was not > >> >> >>>> >> >>> > defined > >> >> >>>> >> >>> > yet or was undefined > >> >> >>>> >> >>> > > >> >> >>>> >> >>> > On Fri, Jan 12, 2018 at 10:48 AM, Martin Sivak > >> >> >>>> >> >>> > <msivak(a)redhat.com> > >> >> >>>> >> >>> > wrote: > >> >> >>>> >> >>> >> > >> >> >>>> >> >>> >> Hi, > >> >> >>>> >> >>> >> > >> >> >>>> >> >>> >> what happens when you try hosted-engine > >> >> >>>> >> >>> >> --connect-storage? > >> >> >>>> >> >>> >> Do > >> >> >>>> >> >>> >> you > >> >> >>>> >> >>> >> see > >> >> >>>> >> >>> >> any errors in the vdsm log? > >> >> >>>> >> >>> >> > >> >> >>>> >> >>> >> Best regards > >> >> >>>> >> >>> >> > >> >> >>>> >> >>> >> Martin Sivak > >> >> >>>> >> >>> >> > >> >> >>>> >> >>> >> On Fri, Jan 12, 2018 at 3:41 PM, Jayme > >> >> >>>> >> >>> >> <jaymef(a)gmail.com> > >> >> >>>> >> >>> >> wrote: > >> >> >>>> >> >>> >> > Ok this is what I've done: > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > - All three hosts in global maintenance mode > >> >> >>>> >> >>> >> > - Ran: systemctl stop ovirt-ha-broker; systemctl > stop > >> >> >>>> >> >>> >> > ovirt-ha-broker -- > >> >> >>>> >> >>> >> > on > >> >> >>>> >> >>> >> > all three hosts > >> >> >>>> >> >>> >> > - Moved ALL files in > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > /var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/ > >> >> >>>> >> >>> >> > to > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > /var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/backup > >> >> >>>> >> >>> >> > - Ran: systemctl start ovirt-ha-broker; systemctl > >> >> >>>> >> >>> >> > start > >> >> >>>> >> >>> >> > ovirt-ha-broker > >> >> >>>> >> >>> >> > -- > >> >> >>>> >> >>> >> > on all three hosts > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > - attempt start of engine vm from HOST0 (cultivar0): > >> >> >>>> >> >>> >> > hosted-engine > >> >> >>>> >> >>> >> > --vm-start > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > Lots of errors in the logs still, it appears to be > >> >> >>>> >> >>> >> > having > >> >> >>>> >> >>> >> > problems > >> >> >>>> >> >>> >> > with > >> >> >>>> >> >>> >> > that > >> >> >>>> >> >>> >> > directory still: > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > Jan 12 10:40:13 cultivar0 journal: ovirt-ha-broker > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > ovirt_hosted_engine_ha.broker. > storage_broker.StorageBroker > >> >> >>>> >> >>> >> > ERROR > >> >> >>>> >> >>> >> > Failed > >> >> >>>> >> >>> >> > to > >> >> >>>> >> >>> >> > write metadata for host 1 to > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > /var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/8582bdfc-ef54-47af-9f1e- > f5b7ec1f1cf8#012Traceback > >> >> >>>> >> >>> >> > (most recent call last):#012 File > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > "/usr/lib/python2.7/site- > packages/ovirt_hosted_engine_ha/broker/storage_broker.py", > >> >> >>>> >> >>> >> > line 202, in put_stats#012 f = os.open(path, > >> >> >>>> >> >>> >> > direct_flag > >> >> >>>> >> >>> >> > | > >> >> >>>> >> >>> >> > os.O_WRONLY | > >> >> >>>> >> >>> >> > os.O_SYNC)#012OSError: [Errno 2] No such file or > >> >> >>>> >> >>> >> > directory: > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > '/var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/8582bdfc-ef54-47af-9f1e- > f5b7ec1f1cf8' > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > There are no new files or symlinks in > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > /var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/ > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > - Jayme > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > On Fri, Jan 12, 2018 at 10:23 AM, Martin Sivak > >> >> >>>> >> >>> >> > <msivak(a)redhat.com> > >> >> >>>> >> >>> >> > wrote: > >> >> >>>> >> >>> >> >> > >> >> >>>> >> >>> >> >> > Can you please stop all hosted engine tooling ( > >> >> >>>> >> >>> >> >> > >> >> >>>> >> >>> >> >> On all hosts I should have added. > >> >> >>>> >> >>> >> >> > >> >> >>>> >> >>> >> >> Martin > >> >> >>>> >> >>> >> >> > >> >> >>>> >> >>> >> >> On Fri, Jan 12, 2018 at 3:22 PM, Martin Sivak > >> >> >>>> >> >>> >> >> <msivak(a)redhat.com> > >> >> >>>> >> >>> >> >> wrote: > >> >> >>>> >> >>> >> >> >> RequestError: failed to read metadata: [Errno 2] > >> >> >>>> >> >>> >> >> >> No > >> >> >>>> >> >>> >> >> >> such > >> >> >>>> >> >>> >> >> >> file > >> >> >>>> >> >>> >> >> >> or > >> >> >>>> >> >>> >> >> >> directory: > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> '/var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/8582bdfc-ef54-47af-9f1e- > f5b7ec1f1cf8' > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> ls -al > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> /var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/8582bdfc-ef54-47af-9f1e- > f5b7ec1f1cf8 > >> >> >>>> >> >>> >> >> >> -rw-rw----. 1 vdsm kvm 1028096 Jan 12 09:59 > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> /var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/8582bdfc-ef54-47af-9f1e- > f5b7ec1f1cf8 > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> Is this due to the symlink problem you guys are > >> >> >>>> >> >>> >> >> >> referring to > >> >> >>>> >> >>> >> >> >> that > >> >> >>>> >> >>> >> >> >> was > >> >> >>>> >> >>> >> >> >> addressed in RC1 or something else? > >> >> >>>> >> >>> >> >> > > >> >> >>>> >> >>> >> >> > No, this file is the symlink. It should point to > >> >> >>>> >> >>> >> >> > somewhere > >> >> >>>> >> >>> >> >> > inside > >> >> >>>> >> >>> >> >> > /rhev/. I see it is a 1G file in your case. That > is > >> >> >>>> >> >>> >> >> > really > >> >> >>>> >> >>> >> >> > interesting. > >> >> >>>> >> >>> >> >> > > >> >> >>>> >> >>> >> >> > Can you please stop all hosted engine tooling > >> >> >>>> >> >>> >> >> > (ovirt-ha-agent, > >> >> >>>> >> >>> >> >> > ovirt-ha-broker), move the file (metadata file is > >> >> >>>> >> >>> >> >> > not > >> >> >>>> >> >>> >> >> > important > >> >> >>>> >> >>> >> >> > when > >> >> >>>> >> >>> >> >> > services are stopped, but better safe than sorry) > >> >> >>>> >> >>> >> >> > and > >> >> >>>> >> >>> >> >> > restart > >> >> >>>> >> >>> >> >> > all > >> >> >>>> >> >>> >> >> > services again? > >> >> >>>> >> >>> >> >> > > >> >> >>>> >> >>> >> >> >> Could there possibly be a permissions > >> >> >>>> >> >>> >> >> >> problem somewhere? > >> >> >>>> >> >>> >> >> > > >> >> >>>> >> >>> >> >> > Maybe, but the file itself looks out of the > >> >> >>>> >> >>> >> >> > ordinary. > >> >> >>>> >> >>> >> >> > I > >> >> >>>> >> >>> >> >> > wonder > >> >> >>>> >> >>> >> >> > how it > >> >> >>>> >> >>> >> >> > got there. > >> >> >>>> >> >>> >> >> > > >> >> >>>> >> >>> >> >> > Best regards > >> >> >>>> >> >>> >> >> > > >> >> >>>> >> >>> >> >> > Martin Sivak > >> >> >>>> >> >>> >> >> > > >> >> >>>> >> >>> >> >> > On Fri, Jan 12, 2018 at 3:09 PM, Jayme > >> >> >>>> >> >>> >> >> > <jaymef(a)gmail.com> > >> >> >>>> >> >>> >> >> > wrote: > >> >> >>>> >> >>> >> >> >> Thanks for the help thus far. Storage could be > >> >> >>>> >> >>> >> >> >> related > >> >> >>>> >> >>> >> >> >> but > >> >> >>>> >> >>> >> >> >> all > >> >> >>>> >> >>> >> >> >> other > >> >> >>>> >> >>> >> >> >> VMs on > >> >> >>>> >> >>> >> >> >> same storage are running ok. The storage is > >> >> >>>> >> >>> >> >> >> mounted > >> >> >>>> >> >>> >> >> >> via > >> >> >>>> >> >>> >> >> >> NFS > >> >> >>>> >> >>> >> >> >> from > >> >> >>>> >> >>> >> >> >> within one > >> >> >>>> >> >>> >> >> >> of the three hosts, I realize this is not ideal. > >> >> >>>> >> >>> >> >> >> This > >> >> >>>> >> >>> >> >> >> was > >> >> >>>> >> >>> >> >> >> setup > >> >> >>>> >> >>> >> >> >> by > >> >> >>>> >> >>> >> >> >> a > >> >> >>>> >> >>> >> >> >> previous admin more as a proof of concept and > VMs > >> >> >>>> >> >>> >> >> >> were > >> >> >>>> >> >>> >> >> >> put on > >> >> >>>> >> >>> >> >> >> there > >> >> >>>> >> >>> >> >> >> that > >> >> >>>> >> >>> >> >> >> should not have been placed in a proof of > concept > >> >> >>>> >> >>> >> >> >> environment.. > >> >> >>>> >> >>> >> >> >> it > >> >> >>>> >> >>> >> >> >> was > >> >> >>>> >> >>> >> >> >> intended to be rebuilt with proper storage down > >> >> >>>> >> >>> >> >> >> the > >> >> >>>> >> >>> >> >> >> road. > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> So the storage is on HOST0 and the other hosts > >> >> >>>> >> >>> >> >> >> mount > >> >> >>>> >> >>> >> >> >> NFS > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> cultivar0.grove.silverorange.com:/exports/data > >> >> >>>> >> >>> >> >> >> 4861742080 > >> >> >>>> >> >>> >> >> >> 1039352832 3822389248 22% > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> /rhev/data-center/mnt/ > cultivar0.grove.silverorange.com:_exports_data > >> >> >>>> >> >>> >> >> >> cultivar0.grove.silverorange.com:/exports/iso > >> >> >>>> >> >>> >> >> >> 4861742080 > >> >> >>>> >> >>> >> >> >> 1039352832 3822389248 22% > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> /rhev/data-center/mnt/ > cultivar0.grove.silverorange.com:_exports_iso > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> cultivar0.grove.silverorange. > com:/exports/import_export > >> >> >>>> >> >>> >> >> >> 4861742080 > >> >> >>>> >> >>> >> >> >> 1039352832 3822389248 22% > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> /rhev/data-center/mnt/ > cultivar0.grove.silverorange.com:_exports_import__export > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> cultivar0.grove.silverorange. > com:/exports/hosted_engine > >> >> >>>> >> >>> >> >> >> 4861742080 > >> >> >>>> >> >>> >> >> >> 1039352832 3822389248 22% > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> /rhev/data-center/mnt/ > cultivar0.grove.silverorange.com:_exports_hosted__engine > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> Like I said, the VM data storage itself seems to > >> >> >>>> >> >>> >> >> >> be > >> >> >>>> >> >>> >> >> >> working > >> >> >>>> >> >>> >> >> >> ok, > >> >> >>>> >> >>> >> >> >> as > >> >> >>>> >> >>> >> >> >> all > >> >> >>>> >> >>> >> >> >> other > >> >> >>>> >> >>> >> >> >> VMs appear to be running. > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> I'm curious why the broker log says this file is > >> >> >>>> >> >>> >> >> >> not > >> >> >>>> >> >>> >> >> >> found > >> >> >>>> >> >>> >> >> >> when > >> >> >>>> >> >>> >> >> >> it > >> >> >>>> >> >>> >> >> >> is > >> >> >>>> >> >>> >> >> >> correct and I can see the file at that path: > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> RequestError: failed to read metadata: [Errno 2] > >> >> >>>> >> >>> >> >> >> No > >> >> >>>> >> >>> >> >> >> such > >> >> >>>> >> >>> >> >> >> file > >> >> >>>> >> >>> >> >> >> or > >> >> >>>> >> >>> >> >> >> directory: > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> '/var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/8582bdfc-ef54-47af-9f1e- > f5b7ec1f1cf8' > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> ls -al > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> /var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/8582bdfc-ef54-47af-9f1e- > f5b7ec1f1cf8 > >> >> >>>> >> >>> >> >> >> -rw-rw----. 1 vdsm kvm 1028096 Jan 12 09:59 > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> /var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/8582bdfc-ef54-47af-9f1e- > f5b7ec1f1cf8 > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> Is this due to the symlink problem you guys are > >> >> >>>> >> >>> >> >> >> referring to > >> >> >>>> >> >>> >> >> >> that > >> >> >>>> >> >>> >> >> >> was > >> >> >>>> >> >>> >> >> >> addressed in RC1 or something else? Could there > >> >> >>>> >> >>> >> >> >> possibly be > >> >> >>>> >> >>> >> >> >> a > >> >> >>>> >> >>> >> >> >> permissions > >> >> >>>> >> >>> >> >> >> problem somewhere? > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> Assuming that all three hosts have 4.2 rpms > >> >> >>>> >> >>> >> >> >> installed > >> >> >>>> >> >>> >> >> >> and the > >> >> >>>> >> >>> >> >> >> host > >> >> >>>> >> >>> >> >> >> engine > >> >> >>>> >> >>> >> >> >> will not start is it safe for me to update hosts > >> >> >>>> >> >>> >> >> >> to > >> >> >>>> >> >>> >> >> >> 4.2 > >> >> >>>> >> >>> >> >> >> RC1 > >> >> >>>> >> >>> >> >> >> rpms? > >> >> >>>> >> >>> >> >> >> Or > >> >> >>>> >> >>> >> >> >> perhaps install that repo and *only* update the > >> >> >>>> >> >>> >> >> >> ovirt > >> >> >>>> >> >>> >> >> >> HA > >> >> >>>> >> >>> >> >> >> packages? > >> >> >>>> >> >>> >> >> >> Assuming that I cannot yet apply the same > updates > >> >> >>>> >> >>> >> >> >> to > >> >> >>>> >> >>> >> >> >> the > >> >> >>>> >> >>> >> >> >> inaccessible > >> >> >>>> >> >>> >> >> >> hosted > >> >> >>>> >> >>> >> >> >> engine VM. > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> I should also mention one more thing. I > >> >> >>>> >> >>> >> >> >> originally > >> >> >>>> >> >>> >> >> >> upgraded > >> >> >>>> >> >>> >> >> >> the > >> >> >>>> >> >>> >> >> >> engine > >> >> >>>> >> >>> >> >> >> VM > >> >> >>>> >> >>> >> >> >> first using new RPMS then engine-setup. It > failed > >> >> >>>> >> >>> >> >> >> due > >> >> >>>> >> >>> >> >> >> to not > >> >> >>>> >> >>> >> >> >> being > >> >> >>>> >> >>> >> >> >> in > >> >> >>>> >> >>> >> >> >> global maintenance, so I set global maintenance > >> >> >>>> >> >>> >> >> >> and > >> >> >>>> >> >>> >> >> >> ran > >> >> >>>> >> >>> >> >> >> it > >> >> >>>> >> >>> >> >> >> again, > >> >> >>>> >> >>> >> >> >> which > >> >> >>>> >> >>> >> >> >> appeared to complete as intended but never came > >> >> >>>> >> >>> >> >> >> back > >> >> >>>> >> >>> >> >> >> up > >> >> >>>> >> >>> >> >> >> after. > >> >> >>>> >> >>> >> >> >> Just > >> >> >>>> >> >>> >> >> >> in > >> >> >>>> >> >>> >> >> >> case > >> >> >>>> >> >>> >> >> >> this might have anything at all to do with what > >> >> >>>> >> >>> >> >> >> could > >> >> >>>> >> >>> >> >> >> have > >> >> >>>> >> >>> >> >> >> happened. > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> Thanks very much again, I very much appreciate > the > >> >> >>>> >> >>> >> >> >> help! > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> - Jayme > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> >> >> On Fri, Jan 12, 2018 at 8:44 AM, Simone > Tiraboschi > >> >> >>>> >> >>> >> >> >> <stirabos(a)redhat.com> > >> >> >>>> >> >>> >> >> >> wrote: > >> >> >>>> >> >>> >> >> >>> > >> >> >>>> >> >>> >> >> >>> > >> >> >>>> >> >>> >> >> >>> > >> >> >>>> >> >>> >> >> >>> On Fri, Jan 12, 2018 at 11:11 AM, Martin Sivak > >> >> >>>> >> >>> >> >> >>> <msivak(a)redhat.com> > >> >> >>>> >> >>> >> >> >>> wrote: > >> >> >>>> >> >>> >> >> >>>> > >> >> >>>> >> >>> >> >> >>>> Hi, > >> >> >>>> >> >>> >> >> >>>> > >> >> >>>> >> >>> >> >> >>>> the hosted engine agent issue might be fixed > by > >> >> >>>> >> >>> >> >> >>>> restarting > >> >> >>>> >> >>> >> >> >>>> ovirt-ha-broker or updating to newest > >> >> >>>> >> >>> >> >> >>>> ovirt-hosted-engine-ha > >> >> >>>> >> >>> >> >> >>>> and > >> >> >>>> >> >>> >> >> >>>> -setup. We improved handling of the missing > >> >> >>>> >> >>> >> >> >>>> symlink. > >> >> >>>> >> >>> >> >> >>> > >> >> >>>> >> >>> >> >> >>> > >> >> >>>> >> >>> >> >> >>> Available just in oVirt 4.2.1 RC1 > >> >> >>>> >> >>> >> >> >>> > >> >> >>>> >> >>> >> >> >>>> > >> >> >>>> >> >>> >> >> >>>> > >> >> >>>> >> >>> >> >> >>>> All the other issues seem to point to some > >> >> >>>> >> >>> >> >> >>>> storage > >> >> >>>> >> >>> >> >> >>>> problem > >> >> >>>> >> >>> >> >> >>>> I > >> >> >>>> >> >>> >> >> >>>> am > >> >> >>>> >> >>> >> >> >>>> afraid. > >> >> >>>> >> >>> >> >> >>>> > >> >> >>>> >> >>> >> >> >>>> You said you started the VM, do you see it in > >> >> >>>> >> >>> >> >> >>>> virsh > >> >> >>>> >> >>> >> >> >>>> -r > >> >> >>>> >> >>> >> >> >>>> list? > >> >> >>>> >> >>> >> >> >>>> > >> >> >>>> >> >>> >> >> >>>> Best regards > >> >> >>>> >> >>> >> >> >>>> > >> >> >>>> >> >>> >> >> >>>> Martin Sivak > >> >> >>>> >> >>> >> >> >>>> > >> >> >>>> >> >>> >> >> >>>> On Thu, Jan 11, 2018 at 10:00 PM, Jayme > >> >> >>>> >> >>> >> >> >>>> <jaymef(a)gmail.com> > >> >> >>>> >> >>> >> >> >>>> wrote: > >> >> >>>> >> >>> >> >> >>>> > Please help, I'm really not sure what else > to > >> >> >>>> >> >>> >> >> >>>> > try > >> >> >>>> >> >>> >> >> >>>> > at > >> >> >>>> >> >>> >> >> >>>> > this > >> >> >>>> >> >>> >> >> >>>> > point. > >> >> >>>> >> >>> >> >> >>>> > Thank > >> >> >>>> >> >>> >> >> >>>> > you > >> >> >>>> >> >>> >> >> >>>> > for reading! > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > I'm still working on trying to get my hosted > >> >> >>>> >> >>> >> >> >>>> > engine > >> >> >>>> >> >>> >> >> >>>> > running > >> >> >>>> >> >>> >> >> >>>> > after a > >> >> >>>> >> >>> >> >> >>>> > botched > >> >> >>>> >> >>> >> >> >>>> > upgrade to 4.2. Storage is NFS mounted from > >> >> >>>> >> >>> >> >> >>>> > within > >> >> >>>> >> >>> >> >> >>>> > one > >> >> >>>> >> >>> >> >> >>>> > of > >> >> >>>> >> >>> >> >> >>>> > the > >> >> >>>> >> >>> >> >> >>>> > hosts. > >> >> >>>> >> >>> >> >> >>>> > Right > >> >> >>>> >> >>> >> >> >>>> > now I have 3 centos7 hosts that are fully > >> >> >>>> >> >>> >> >> >>>> > updated > >> >> >>>> >> >>> >> >> >>>> > with > >> >> >>>> >> >>> >> >> >>>> > yum > >> >> >>>> >> >>> >> >> >>>> > packages > >> >> >>>> >> >>> >> >> >>>> > from > >> >> >>>> >> >>> >> >> >>>> > ovirt 4.2, the engine was fully updated with > >> >> >>>> >> >>> >> >> >>>> > yum > >> >> >>>> >> >>> >> >> >>>> > packages > >> >> >>>> >> >>> >> >> >>>> > and > >> >> >>>> >> >>> >> >> >>>> > failed to > >> >> >>>> >> >>> >> >> >>>> > come > >> >> >>>> >> >>> >> >> >>>> > up after reboot. As of right now, > everything > >> >> >>>> >> >>> >> >> >>>> > should > >> >> >>>> >> >>> >> >> >>>> > have > >> >> >>>> >> >>> >> >> >>>> > full > >> >> >>>> >> >>> >> >> >>>> > yum > >> >> >>>> >> >>> >> >> >>>> > updates > >> >> >>>> >> >>> >> >> >>>> > and all having 4.2 rpms. I have global > >> >> >>>> >> >>> >> >> >>>> > maintenance > >> >> >>>> >> >>> >> >> >>>> > mode > >> >> >>>> >> >>> >> >> >>>> > on > >> >> >>>> >> >>> >> >> >>>> > right > >> >> >>>> >> >>> >> >> >>>> > now > >> >> >>>> >> >>> >> >> >>>> > and > >> >> >>>> >> >>> >> >> >>>> > started hosted-engine on one of the three > host > >> >> >>>> >> >>> >> >> >>>> > and > >> >> >>>> >> >>> >> >> >>>> > the > >> >> >>>> >> >>> >> >> >>>> > status is > >> >> >>>> >> >>> >> >> >>>> > currently: > >> >> >>>> >> >>> >> >> >>>> > Engine status : {"reason": "failed > liveliness > >> >> >>>> >> >>> >> >> >>>> > check”; > >> >> >>>> >> >>> >> >> >>>> > "health": > >> >> >>>> >> >>> >> >> >>>> > "bad", > >> >> >>>> >> >>> >> >> >>>> > "vm": > >> >> >>>> >> >>> >> >> >>>> > "up", "detail": "Up"} > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > this is what I get when trying to enter > >> >> >>>> >> >>> >> >> >>>> > hosted-vm > >> >> >>>> >> >>> >> >> >>>> > --console > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > The engine VM is running on this host > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > error: failed to get domain 'HostedEngine' > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > error: Domain not found: no domain with > >> >> >>>> >> >>> >> >> >>>> > matching > >> >> >>>> >> >>> >> >> >>>> > name > >> >> >>>> >> >>> >> >> >>>> > 'HostedEngine' > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > Here are logs from various sources when I > >> >> >>>> >> >>> >> >> >>>> > start > >> >> >>>> >> >>> >> >> >>>> > the > >> >> >>>> >> >>> >> >> >>>> > VM on > >> >> >>>> >> >>> >> >> >>>> > HOST3: > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > hosted-engine --vm-start > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > Command VM.getStats with args {'vmID': > >> >> >>>> >> >>> >> >> >>>> > '4013c829-c9d7-4b72-90d5-6fe58137504c'} > >> >> >>>> >> >>> >> >> >>>> > failed: > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > (code=1, message=Virtual machine does not > >> >> >>>> >> >>> >> >> >>>> > exist: > >> >> >>>> >> >>> >> >> >>>> > {'vmId': > >> >> >>>> >> >>> >> >> >>>> > u'4013c829-c9d7-4b72-90d5-6fe58137504c'}) > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > Jan 11 16:55:57 cultivar3 systemd-machined: > >> >> >>>> >> >>> >> >> >>>> > New > >> >> >>>> >> >>> >> >> >>>> > machine > >> >> >>>> >> >>> >> >> >>>> > qemu-110-Cultivar. > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > Jan 11 16:55:57 cultivar3 systemd: Started > >> >> >>>> >> >>> >> >> >>>> > Virtual > >> >> >>>> >> >>> >> >> >>>> > Machine > >> >> >>>> >> >>> >> >> >>>> > qemu-110-Cultivar. > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > Jan 11 16:55:57 cultivar3 systemd: Starting > >> >> >>>> >> >>> >> >> >>>> > Virtual > >> >> >>>> >> >>> >> >> >>>> > Machine > >> >> >>>> >> >>> >> >> >>>> > qemu-110-Cultivar. > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > Jan 11 16:55:57 cultivar3 kvm: 3 guests now > >> >> >>>> >> >>> >> >> >>>> > active > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > ==> /var/log/vdsm/vdsm.log <== > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > File > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > "/usr/lib/python2.7/site- > packages/vdsm/common/api.py", > >> >> >>>> >> >>> >> >> >>>> > line > >> >> >>>> >> >>> >> >> >>>> > 48, > >> >> >>>> >> >>> >> >> >>>> > in > >> >> >>>> >> >>> >> >> >>>> > method > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > ret = func(*args, **kwargs) > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > File > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > "/usr/lib/python2.7/site- > packages/vdsm/storage/hsm.py", > >> >> >>>> >> >>> >> >> >>>> > line > >> >> >>>> >> >>> >> >> >>>> > 2718, in > >> >> >>>> >> >>> >> >> >>>> > getStorageDomainInfo > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > dom = self.validateSdUUID(sdUUID) > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > File > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > "/usr/lib/python2.7/site- > packages/vdsm/storage/hsm.py", > >> >> >>>> >> >>> >> >> >>>> > line > >> >> >>>> >> >>> >> >> >>>> > 304, in > >> >> >>>> >> >>> >> >> >>>> > validateSdUUID > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > sdDom.validate() > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > File > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > "/usr/lib/python2.7/site- > packages/vdsm/storage/fileSD.py", > >> >> >>>> >> >>> >> >> >>>> > line > >> >> >>>> >> >>> >> >> >>>> > 515, > >> >> >>>> >> >>> >> >> >>>> > in validate > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > raise > >> >> >>>> >> >>> >> >> >>>> > se.StorageDomainAccessError(self.sdUUID) > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > StorageDomainAccessError: Domain is either > >> >> >>>> >> >>> >> >> >>>> > partially > >> >> >>>> >> >>> >> >> >>>> > accessible > >> >> >>>> >> >>> >> >> >>>> > or > >> >> >>>> >> >>> >> >> >>>> > entirely > >> >> >>>> >> >>> >> >> >>>> > inaccessible: > >> >> >>>> >> >>> >> >> >>>> > (u'248f46f0-d793-4581-9810-c9d965e2f286',) > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > jsonrpc/2::ERROR::2018-01-11 > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > 16:55:16,144::dispatcher::82:: > storage.Dispatcher::(wrapper) > >> >> >>>> >> >>> >> >> >>>> > FINISH > >> >> >>>> >> >>> >> >> >>>> > getStorageDomainInfo error=Domain is either > >> >> >>>> >> >>> >> >> >>>> > partially > >> >> >>>> >> >>> >> >> >>>> > accessible > >> >> >>>> >> >>> >> >> >>>> > or > >> >> >>>> >> >>> >> >> >>>> > entirely > >> >> >>>> >> >>> >> >> >>>> > inaccessible: > >> >> >>>> >> >>> >> >> >>>> > (u'248f46f0-d793-4581-9810-c9d965e2f286',) > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > ==> /var/log/libvirt/qemu/Cultivar.log <== > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > LC_ALL=C > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > PATH=/usr/local/sbin:/usr/ > local/bin:/usr/sbin:/usr/bin > >> >> >>>> >> >>> >> >> >>>> > QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm > >> >> >>>> >> >>> >> >> >>>> > -name > >> >> >>>> >> >>> >> >> >>>> > guest=Cultivar,debug-threads=on -S -object > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > secret,id=masterKey0,format= > raw,file=/var/lib/libvirt/qemu/domain-108-Cultivar/master-key.aes > >> >> >>>> >> >>> >> >> >>>> > -machine > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > pc-i440fx-rhel7.3.0,accel=kvm, > usb=off,dump-guest-core=off > >> >> >>>> >> >>> >> >> >>>> > -cpu > >> >> >>>> >> >>> >> >> >>>> > Conroe -m 8192 -realtime mlock=off -smp > >> >> >>>> >> >>> >> >> >>>> > 2,maxcpus=16,sockets=16,cores=1,threads=1 > >> >> >>>> >> >>> >> >> >>>> > -uuid > >> >> >>>> >> >>> >> >> >>>> > 4013c829-c9d7-4b72-90d5-6fe58137504c > -smbios > >> >> >>>> >> >>> >> >> >>>> > 'type=1,manufacturer=oVirt,product=oVirt > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > Node,version=7-4.1708.el7. > centos,serial=44454C4C-4300-1034-8035-CAC04F424331,uuid= > 4013c829-c9d7-4b72-90d5-6fe58137504c' > >> >> >>>> >> >>> >> >> >>>> > -no-user-config -nodefaults -chardev > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > socket,id=charmonitor,path=/ > var/lib/libvirt/qemu/domain-108-Cultivar/monitor.sock,server,nowait > >> >> >>>> >> >>> >> >> >>>> > -mon > >> >> >>>> >> >>> >> >> >>>> > chardev=charmonitor,id=monitor,mode=control > >> >> >>>> >> >>> >> >> >>>> > -rtc > >> >> >>>> >> >>> >> >> >>>> > base=2018-01-11T20:33:19,driftfix=slew > -global > >> >> >>>> >> >>> >> >> >>>> > kvm-pit.lost_tick_policy=delay -no-hpet > >> >> >>>> >> >>> >> >> >>>> > -no-reboot > >> >> >>>> >> >>> >> >> >>>> > -boot > >> >> >>>> >> >>> >> >> >>>> > strict=on > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > piix3-usb-uhci,id=usb,bus=pci. > 0,addr=0x1.0x2 > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > virtio-serial-pci,id=virtio- > serial0,bus=pci.0,addr=0x4 > >> >> >>>> >> >>> >> >> >>>> > -drive > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > file=/var/run/vdsm/storage/ > 248f46f0-d793-4581-9810-c9d965e2f286/c2dde892-f978-4dfc-a421-c8e04cf387f9/ > 23aa0a66-fa6c-4967-a1e5-fbe47c0cd705,format=raw,if= > none,id=drive-virtio-disk0,serial=c2dde892-f978-4dfc- > a421-c8e04cf387f9,cache=none,werror=stop,rerror=stop,aio=threads > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > virtio-blk-pci,scsi=off,bus= > pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 > >> >> >>>> >> >>> >> >> >>>> > -drive if=none,id=drive-ide0-1-0, > readonly=on > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > ide-cd,bus=ide.1,unit=0,drive= > drive-ide0-1-0,id=ide0-1-0 > >> >> >>>> >> >>> >> >> >>>> > -netdev > >> >> >>>> >> >>> >> >> >>>> > tap,fd=30,id=hostnet0,vhost=on,vhostfd=32 > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > virtio-net-pci,netdev= > hostnet0,id=net0,mac=00:16:3e:7f:d6:83,bus=pci.0,addr=0x3 > >> >> >>>> >> >>> >> >> >>>> > -chardev > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > socket,id=charchannel0,path=/ > var/lib/libvirt/qemu/channels/4013c829-c9d7-4b72-90d5- > 6fe58137504c.com.redhat.rhevm.vdsm,server,nowait > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > virtserialport,bus=virtio- > serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.rhevm.vdsm > >> >> >>>> >> >>> >> >> >>>> > -chardev > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > socket,id=charchannel1,path=/ > var/lib/libvirt/qemu/channels/4013c829-c9d7-4b72-90d5- > 6fe58137504c.org.qemu.guest_agent.0,server,nowait > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > virtserialport,bus=virtio- > serial0.0,nr=2,chardev=charchannel1,id=channel1,name= > org.qemu.guest_agent.0 > >> >> >>>> >> >>> >> >> >>>> > -chardev spicevmc,id=charchannel2,name= > vdagent > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > virtserialport,bus=virtio- > serial0.0,nr=3,chardev=charchannel2,id=channel2,name=com.redhat.spice.0 > >> >> >>>> >> >>> >> >> >>>> > -chardev > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > socket,id=charchannel3,path=/ > var/lib/libvirt/qemu/channels/4013c829-c9d7-4b72-90d5- > 6fe58137504c.org.ovirt.hosted-engine-setup.0,server,nowait > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > virtserialport,bus=virtio- > serial0.0,nr=4,chardev=charchannel3,id=channel3,name= > org.ovirt.hosted-engine-setup.0 > >> >> >>>> >> >>> >> >> >>>> > -chardev pty,id=charconsole0 -device > >> >> >>>> >> >>> >> >> >>>> > virtconsole,chardev= > charconsole0,id=console0 > >> >> >>>> >> >>> >> >> >>>> > -spice > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > tls-port=5900,addr=0,x509-dir= > /etc/pki/vdsm/libvirt-spice,tls-channel=default,seamless-migration=on > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > cirrus-vga,id=video0,bus=pci.0,addr=0x2 > >> >> >>>> >> >>> >> >> >>>> > -object > >> >> >>>> >> >>> >> >> >>>> > rng-random,id=objrng0,filename=/dev/urandom > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > virtio-rng-pci,rng=objrng0,id= > rng0,bus=pci.0,addr=0x5 > >> >> >>>> >> >>> >> >> >>>> > -msg > >> >> >>>> >> >>> >> >> >>>> > timestamp=on > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > 2018-01-11T20:33:19.699999Z qemu-kvm: > -chardev > >> >> >>>> >> >>> >> >> >>>> > pty,id=charconsole0: > >> >> >>>> >> >>> >> >> >>>> > char > >> >> >>>> >> >>> >> >> >>>> > device redirected to /dev/pts/2 (label > >> >> >>>> >> >>> >> >> >>>> > charconsole0) > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > 2018-01-11 20:38:11.640+0000: shutting down, > >> >> >>>> >> >>> >> >> >>>> > reason=shutdown > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > 2018-01-11 20:39:02.122+0000: starting up > >> >> >>>> >> >>> >> >> >>>> > libvirt > >> >> >>>> >> >>> >> >> >>>> > version: > >> >> >>>> >> >>> >> >> >>>> > 3.2.0, > >> >> >>>> >> >>> >> >> >>>> > package: > >> >> >>>> >> >>> >> >> >>>> > 14.el7_4.7 (CentOS BuildSystem > >> >> >>>> >> >>> >> >> >>>> > <http://bugs.centos.org>, > >> >> >>>> >> >>> >> >> >>>> > 2018-01-04-19:31:34, c1bm.rdu2.centos.org) > >> >> >>>> >> >>> >> >> >>>> > qemu > >> >> >>>> >> >>> >> >> >>>> > version: > >> >> >>>> >> >>> >> >> >>>> > 2.9.0(qemu-kvm-ev-2.9.0-16.el7_4.13.1), > >> >> >>>> >> >>> >> >> >>>> > hostname: > >> >> >>>> >> >>> >> >> >>>> > cultivar3 > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > LC_ALL=C > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > PATH=/usr/local/sbin:/usr/ > local/bin:/usr/sbin:/usr/bin > >> >> >>>> >> >>> >> >> >>>> > QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm > >> >> >>>> >> >>> >> >> >>>> > -name > >> >> >>>> >> >>> >> >> >>>> > guest=Cultivar,debug-threads=on -S -object > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > secret,id=masterKey0,format= > raw,file=/var/lib/libvirt/qemu/domain-109-Cultivar/master-key.aes > >> >> >>>> >> >>> >> >> >>>> > -machine > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > pc-i440fx-rhel7.3.0,accel=kvm, > usb=off,dump-guest-core=off > >> >> >>>> >> >>> >> >> >>>> > -cpu > >> >> >>>> >> >>> >> >> >>>> > Conroe -m 8192 -realtime mlock=off -smp > >> >> >>>> >> >>> >> >> >>>> > 2,maxcpus=16,sockets=16,cores=1,threads=1 > >> >> >>>> >> >>> >> >> >>>> > -uuid > >> >> >>>> >> >>> >> >> >>>> > 4013c829-c9d7-4b72-90d5-6fe58137504c > -smbios > >> >> >>>> >> >>> >> >> >>>> > 'type=1,manufacturer=oVirt,product=oVirt > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > Node,version=7-4.1708.el7. > centos,serial=44454C4C-4300-1034-8035-CAC04F424331,uuid= > 4013c829-c9d7-4b72-90d5-6fe58137504c' > >> >> >>>> >> >>> >> >> >>>> > -no-user-config -nodefaults -chardev > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > socket,id=charmonitor,path=/ > var/lib/libvirt/qemu/domain-109-Cultivar/monitor.sock,server,nowait > >> >> >>>> >> >>> >> >> >>>> > -mon > >> >> >>>> >> >>> >> >> >>>> > chardev=charmonitor,id=monitor,mode=control > >> >> >>>> >> >>> >> >> >>>> > -rtc > >> >> >>>> >> >>> >> >> >>>> > base=2018-01-11T20:39:02,driftfix=slew > -global > >> >> >>>> >> >>> >> >> >>>> > kvm-pit.lost_tick_policy=delay -no-hpet > >> >> >>>> >> >>> >> >> >>>> > -no-reboot > >> >> >>>> >> >>> >> >> >>>> > -boot > >> >> >>>> >> >>> >> >> >>>> > strict=on > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > piix3-usb-uhci,id=usb,bus=pci. > 0,addr=0x1.0x2 > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > virtio-serial-pci,id=virtio- > serial0,bus=pci.0,addr=0x4 > >> >> >>>> >> >>> >> >> >>>> > -drive > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > file=/var/run/vdsm/storage/ > 248f46f0-d793-4581-9810-c9d965e2f286/c2dde892-f978-4dfc-a421-c8e04cf387f9/ > 23aa0a66-fa6c-4967-a1e5-fbe47c0cd705,format=raw,if= > none,id=drive-virtio-disk0,serial=c2dde892-f978-4dfc- > a421-c8e04cf387f9,cache=none,werror=stop,rerror=stop,aio=threads > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > virtio-blk-pci,scsi=off,bus= > pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 > >> >> >>>> >> >>> >> >> >>>> > -drive if=none,id=drive-ide0-1-0, > readonly=on > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > ide-cd,bus=ide.1,unit=0,drive= > drive-ide0-1-0,id=ide0-1-0 > >> >> >>>> >> >>> >> >> >>>> > -netdev > >> >> >>>> >> >>> >> >> >>>> > tap,fd=30,id=hostnet0,vhost=on,vhostfd=32 > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > virtio-net-pci,netdev= > hostnet0,id=net0,mac=00:16:3e:7f:d6:83,bus=pci.0,addr=0x3 > >> >> >>>> >> >>> >> >> >>>> > -chardev > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > socket,id=charchannel0,path=/ > var/lib/libvirt/qemu/channels/4013c829-c9d7-4b72-90d5- > 6fe58137504c.com.redhat.rhevm.vdsm,server,nowait > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > virtserialport,bus=virtio- > serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.rhevm.vdsm > >> >> >>>> >> >>> >> >> >>>> > -chardev > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > socket,id=charchannel1,path=/ > var/lib/libvirt/qemu/channels/4013c829-c9d7-4b72-90d5- > 6fe58137504c.org.qemu.guest_agent.0,server,nowait > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > virtserialport,bus=virtio- > serial0.0,nr=2,chardev=charchannel1,id=channel1,name= > org.qemu.guest_agent.0 > >> >> >>>> >> >>> >> >> >>>> > -chardev spicevmc,id=charchannel2,name= > vdagent > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > virtserialport,bus=virtio- > serial0.0,nr=3,chardev=charchannel2,id=channel2,name=com.redhat.spice.0 > >> >> >>>> >> >>> >> >> >>>> > -chardev > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > socket,id=charchannel3,path=/ > var/lib/libvirt/qemu/channels/4013c829-c9d7-4b72-90d5- > 6fe58137504c.org.ovirt.hosted-engine-setup.0,server,nowait > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > virtserialport,bus=virtio- > serial0.0,nr=4,chardev=charchannel3,id=channel3,name= > org.ovirt.hosted-engine-setup.0 > >> >> >>>> >> >>> >> >> >>>> > -chardev pty,id=charconsole0 -device > >> >> >>>> >> >>> >> >> >>>> > virtconsole,chardev= > charconsole0,id=console0 > >> >> >>>> >> >>> >> >> >>>> > -spice > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > tls-port=5900,addr=0,x509-dir= > /etc/pki/vdsm/libvirt-spice,tls-channel=default,seamless-migration=on > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > cirrus-vga,id=video0,bus=pci.0,addr=0x2 > >> >> >>>> >> >>> >> >> >>>> > -object > >> >> >>>> >> >>> >> >> >>>> > rng-random,id=objrng0,filename=/dev/urandom > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > virtio-rng-pci,rng=objrng0,id= > rng0,bus=pci.0,addr=0x5 > >> >> >>>> >> >>> >> >> >>>> > -msg > >> >> >>>> >> >>> >> >> >>>> > timestamp=on > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > 2018-01-11T20:39:02.380773Z qemu-kvm: > -chardev > >> >> >>>> >> >>> >> >> >>>> > pty,id=charconsole0: > >> >> >>>> >> >>> >> >> >>>> > char > >> >> >>>> >> >>> >> >> >>>> > device redirected to /dev/pts/2 (label > >> >> >>>> >> >>> >> >> >>>> > charconsole0) > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > 2018-01-11 20:53:11.407+0000: shutting down, > >> >> >>>> >> >>> >> >> >>>> > reason=shutdown > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > 2018-01-11 20:55:57.210+0000: starting up > >> >> >>>> >> >>> >> >> >>>> > libvirt > >> >> >>>> >> >>> >> >> >>>> > version: > >> >> >>>> >> >>> >> >> >>>> > 3.2.0, > >> >> >>>> >> >>> >> >> >>>> > package: > >> >> >>>> >> >>> >> >> >>>> > 14.el7_4.7 (CentOS BuildSystem > >> >> >>>> >> >>> >> >> >>>> > <http://bugs.centos.org>, > >> >> >>>> >> >>> >> >> >>>> > 2018-01-04-19:31:34, c1bm.rdu2.centos.org) > >> >> >>>> >> >>> >> >> >>>> > qemu > >> >> >>>> >> >>> >> >> >>>> > version: > >> >> >>>> >> >>> >> >> >>>> > 2.9.0(qemu-kvm-ev-2.9.0-16.el7_4.13.1), > >> >> >>>> >> >>> >> >> >>>> > hostname: > >> >> >>>> >> >>> >> >> >>>> > cultivar3.grove.silverorange.com > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > LC_ALL=C > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > PATH=/usr/local/sbin:/usr/ > local/bin:/usr/sbin:/usr/bin > >> >> >>>> >> >>> >> >> >>>> > QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm > >> >> >>>> >> >>> >> >> >>>> > -name > >> >> >>>> >> >>> >> >> >>>> > guest=Cultivar,debug-threads=on -S -object > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > secret,id=masterKey0,format= > raw,file=/var/lib/libvirt/qemu/domain-110-Cultivar/master-key.aes > >> >> >>>> >> >>> >> >> >>>> > -machine > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > pc-i440fx-rhel7.3.0,accel=kvm, > usb=off,dump-guest-core=off > >> >> >>>> >> >>> >> >> >>>> > -cpu > >> >> >>>> >> >>> >> >> >>>> > Conroe -m 8192 -realtime mlock=off -smp > >> >> >>>> >> >>> >> >> >>>> > 2,maxcpus=16,sockets=16,cores=1,threads=1 > >> >> >>>> >> >>> >> >> >>>> > -uuid > >> >> >>>> >> >>> >> >> >>>> > 4013c829-c9d7-4b72-90d5-6fe58137504c > -smbios > >> >> >>>> >> >>> >> >> >>>> > 'type=1,manufacturer=oVirt,product=oVirt > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > Node,version=7-4.1708.el7. > centos,serial=44454C4C-4300-1034-8035-CAC04F424331,uuid= > 4013c829-c9d7-4b72-90d5-6fe58137504c' > >> >> >>>> >> >>> >> >> >>>> > -no-user-config -nodefaults -chardev > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > socket,id=charmonitor,path=/ > var/lib/libvirt/qemu/domain-110-Cultivar/monitor.sock,server,nowait > >> >> >>>> >> >>> >> >> >>>> > -mon > >> >> >>>> >> >>> >> >> >>>> > chardev=charmonitor,id=monitor,mode=control > >> >> >>>> >> >>> >> >> >>>> > -rtc > >> >> >>>> >> >>> >> >> >>>> > base=2018-01-11T20:55:57,driftfix=slew > -global > >> >> >>>> >> >>> >> >> >>>> > kvm-pit.lost_tick_policy=delay -no-hpet > >> >> >>>> >> >>> >> >> >>>> > -no-reboot > >> >> >>>> >> >>> >> >> >>>> > -boot > >> >> >>>> >> >>> >> >> >>>> > strict=on > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > piix3-usb-uhci,id=usb,bus=pci. > 0,addr=0x1.0x2 > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > virtio-serial-pci,id=virtio- > serial0,bus=pci.0,addr=0x4 > >> >> >>>> >> >>> >> >> >>>> > -drive > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > file=/var/run/vdsm/storage/ > 248f46f0-d793-4581-9810-c9d965e2f286/c2dde892-f978-4dfc-a421-c8e04cf387f9/ > 23aa0a66-fa6c-4967-a1e5-fbe47c0cd705,format=raw,if= > none,id=drive-virtio-disk0,serial=c2dde892-f978-4dfc- > a421-c8e04cf387f9,cache=none,werror=stop,rerror=stop,aio=threads > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > virtio-blk-pci,scsi=off,bus= > pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 > >> >> >>>> >> >>> >> >> >>>> > -drive if=none,id=drive-ide0-1-0, > readonly=on > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > ide-cd,bus=ide.1,unit=0,drive= > drive-ide0-1-0,id=ide0-1-0 > >> >> >>>> >> >>> >> >> >>>> > -netdev > >> >> >>>> >> >>> >> >> >>>> > tap,fd=30,id=hostnet0,vhost=on,vhostfd=32 > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > virtio-net-pci,netdev= > hostnet0,id=net0,mac=00:16:3e:7f:d6:83,bus=pci.0,addr=0x3 > >> >> >>>> >> >>> >> >> >>>> > -chardev > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > socket,id=charchannel0,path=/ > var/lib/libvirt/qemu/channels/4013c829-c9d7-4b72-90d5- > 6fe58137504c.com.redhat.rhevm.vdsm,server,nowait > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > virtserialport,bus=virtio- > serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.rhevm.vdsm > >> >> >>>> >> >>> >> >> >>>> > -chardev > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > socket,id=charchannel1,path=/ > var/lib/libvirt/qemu/channels/4013c829-c9d7-4b72-90d5- > 6fe58137504c.org.qemu.guest_agent.0,server,nowait > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > virtserialport,bus=virtio- > serial0.0,nr=2,chardev=charchannel1,id=channel1,name= > org.qemu.guest_agent.0 > >> >> >>>> >> >>> >> >> >>>> > -chardev spicevmc,id=charchannel2,name= > vdagent > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > virtserialport,bus=virtio- > serial0.0,nr=3,chardev=charchannel2,id=channel2,name=com.redhat.spice.0 > >> >> >>>> >> >>> >> >> >>>> > -chardev > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > socket,id=charchannel3,path=/ > var/lib/libvirt/qemu/channels/4013c829-c9d7-4b72-90d5- > 6fe58137504c.org.ovirt.hosted-engine-setup.0,server,nowait > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > virtserialport,bus=virtio- > serial0.0,nr=4,chardev=charchannel3,id=channel3,name= > org.ovirt.hosted-engine-setup.0 > >> >> >>>> >> >>> >> >> >>>> > -chardev pty,id=charconsole0 -device > >> >> >>>> >> >>> >> >> >>>> > virtconsole,chardev= > charconsole0,id=console0 > >> >> >>>> >> >>> >> >> >>>> > -spice > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > tls-port=5900,addr=0,x509-dir= > /etc/pki/vdsm/libvirt-spice,tls-channel=default,seamless-migration=on > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > cirrus-vga,id=video0,bus=pci.0,addr=0x2 > >> >> >>>> >> >>> >> >> >>>> > -object > >> >> >>>> >> >>> >> >> >>>> > rng-random,id=objrng0,filename=/dev/urandom > >> >> >>>> >> >>> >> >> >>>> > -device > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > virtio-rng-pci,rng=objrng0,id= > rng0,bus=pci.0,addr=0x5 > >> >> >>>> >> >>> >> >> >>>> > -msg > >> >> >>>> >> >>> >> >> >>>> > timestamp=on > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > 2018-01-11T20:55:57.468037Z qemu-kvm: > -chardev > >> >> >>>> >> >>> >> >> >>>> > pty,id=charconsole0: > >> >> >>>> >> >>> >> >> >>>> > char > >> >> >>>> >> >>> >> >> >>>> > device redirected to /dev/pts/2 (label > >> >> >>>> >> >>> >> >> >>>> > charconsole0) > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > ==> /var/log/ovirt-hosted-engine- > ha/broker.log > >> >> >>>> >> >>> >> >> >>>> > <== > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > File > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > "/usr/lib/python2.7/site- > packages/ovirt_hosted_engine_ha/broker/storage_broker.py", > >> >> >>>> >> >>> >> >> >>>> > line 151, in get_raw_stats > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > f = os.open(path, direct_flag | > >> >> >>>> >> >>> >> >> >>>> > os.O_RDONLY | > >> >> >>>> >> >>> >> >> >>>> > os.O_SYNC) > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > OSError: [Errno 2] No such file or > directory: > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > '/var/run/vdsm/storage/ > 248f46f0-d793-4581-9810-c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/ > 8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8' > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > StatusStorageThread::ERROR::2018-01-11 > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > 16:55:15,761::status_broker:: > 92::ovirt_hosted_engine_ha.broker.status_broker.StatusBroker.Update::(run) > >> >> >>>> >> >>> >> >> >>>> > Failed to read state. > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > Traceback (most recent call last): > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > File > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > "/usr/lib/python2.7/site- > packages/ovirt_hosted_engine_ha/broker/status_broker.py", > >> >> >>>> >> >>> >> >> >>>> > line 88, in run > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > self._storage_broker.get_raw_stats() > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > File > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > "/usr/lib/python2.7/site- > packages/ovirt_hosted_engine_ha/broker/storage_broker.py", > >> >> >>>> >> >>> >> >> >>>> > line 162, in get_raw_stats > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > .format(str(e))) > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > RequestError: failed to read metadata: > [Errno > >> >> >>>> >> >>> >> >> >>>> > 2] > >> >> >>>> >> >>> >> >> >>>> > No > >> >> >>>> >> >>> >> >> >>>> > such > >> >> >>>> >> >>> >> >> >>>> > file or > >> >> >>>> >> >>> >> >> >>>> > directory: > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > '/var/run/vdsm/storage/ > 248f46f0-d793-4581-9810-c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/ > 8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8' > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > ==> /var/log/ovirt-hosted-engine- > ha/agent.log > >> >> >>>> >> >>> >> >> >>>> > <== > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > result = refresh_method() > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > File > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > "/usr/lib/python2.7/site- > packages/ovirt_hosted_engine_ha/env/config.py", > >> >> >>>> >> >>> >> >> >>>> > line 519, in refresh_vm_conf > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > content = > >> >> >>>> >> >>> >> >> >>>> > self._get_file_content_from_ > shared_storage(VM) > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > File > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > "/usr/lib/python2.7/site- > packages/ovirt_hosted_engine_ha/env/config.py", > >> >> >>>> >> >>> >> >> >>>> > line 484, in > >> >> >>>> >> >>> >> >> >>>> > _get_file_content_from_shared_storage > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > config_volume_path = > >> >> >>>> >> >>> >> >> >>>> > self._get_config_volume_path() > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > File > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > "/usr/lib/python2.7/site- > packages/ovirt_hosted_engine_ha/env/config.py", > >> >> >>>> >> >>> >> >> >>>> > line 188, in _get_config_volume_path > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > conf_vol_uuid > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > File > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > "/usr/lib/python2.7/site- > packages/ovirt_hosted_engine_ha/lib/heconflib.py", > >> >> >>>> >> >>> >> >> >>>> > line 358, in get_volume_path > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > root=envconst.SD_RUN_DIR, > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > RuntimeError: Path to volume > >> >> >>>> >> >>> >> >> >>>> > 4838749f-216d-406b-b245-98d0343fcf7f > >> >> >>>> >> >>> >> >> >>>> > not > >> >> >>>> >> >>> >> >> >>>> > found > >> >> >>>> >> >>> >> >> >>>> > in /run/vdsm/storag > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > ==> /var/log/vdsm/vdsm.log <== > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > periodic/42::ERROR::2018-01-11 > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > 16:56:11,446::vmstats::260:: > virt.vmstats::(send_metrics) > >> >> >>>> >> >>> >> >> >>>> > VM > >> >> >>>> >> >>> >> >> >>>> > metrics > >> >> >>>> >> >>> >> >> >>>> > collection failed > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > Traceback (most recent call last): > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > File > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > "/usr/lib/python2.7/site- > packages/vdsm/virt/vmstats.py", > >> >> >>>> >> >>> >> >> >>>> > line > >> >> >>>> >> >>> >> >> >>>> > 197, in > >> >> >>>> >> >>> >> >> >>>> > send_metrics > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > data[prefix + '.cpu.usage'] = > >> >> >>>> >> >>> >> >> >>>> > stat['cpuUsage'] > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > KeyError: 'cpuUsage' > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> > ______________________________ > _________________ > >> >> >>>> >> >>> >> >> >>>> > Users mailing list > >> >> >>>> >> >>> >> >> >>>> > Users(a)ovirt.org > >> >> >>>> >> >>> >> >> >>>> > http://lists.ovirt.org/ > mailman/listinfo/users > >> >> >>>> >> >>> >> >> >>>> > > >> >> >>>> >> >>> >> >> >>>> ______________________________ > _________________ > >> >> >>>> >> >>> >> >> >>>> Users mailing list > >> >> >>>> >> >>> >> >> >>>> Users(a)ovirt.org > >> >> >>>> >> >>> >> >> >>>> http://lists.ovirt.org/mailman/listinfo/users > >> >> >>>> >> >>> >> >> >>> > >> >> >>>> >> >>> >> >> >>> > >> >> >>>> >> >>> >> >> >> > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> >> > > >> >> >>>> >> >>> > > >> >> >>>> >> >>> > > >> >> >>>> >> >> > >> >> >>>> >> >> > >> >> >>>> >> > > >> >> >>>> > > >> >> >>>> > > >> >> >>> > >> >> >>> > >> >> >> > >> >> > > >> > > >> > > > > > >

1 2

Re: [ovirt-users] unable to bring up hosted engine after botched 4.2 upgrade
by Jayme 12 Jan '18

12 Jan '18

The lock space issue was an issue I needed to clear but I don't believe it has resolved the problem. I shutdown agent and broker on all hosts and disconnected hosted-storage then enabled broker/agent on just one host and connected storage. I started the VM and actually didn't get any errors in the logs barely at all which was good to see, however the VM is still not running: HOST3: Engine status : {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"} ==> /var/log/messages <== Jan 12 12:42:57 cultivar3 kernel: ovirtmgmt: port 2(vnet0) entered disabled state Jan 12 12:42:57 cultivar3 kernel: device vnet0 entered promiscuous mode Jan 12 12:42:57 cultivar3 kernel: ovirtmgmt: port 2(vnet0) entered blocking state Jan 12 12:42:57 cultivar3 kernel: ovirtmgmt: port 2(vnet0) entered forwarding state Jan 12 12:42:57 cultivar3 lldpad: recvfrom(Event interface): No buffer space available Jan 12 12:42:57 cultivar3 systemd-machined: New machine qemu-111-Cultivar. Jan 12 12:42:57 cultivar3 systemd: Started Virtual Machine qemu-111-Cultivar. Jan 12 12:42:57 cultivar3 systemd: Starting Virtual Machine qemu-111-Cultivar. Jan 12 12:42:57 cultivar3 kvm: 3 guests now active Jan 12 12:44:38 cultivar3 libvirtd: 2018-01-12 16:44:38.737+0000: 1535: error : qemuDomainAgentAvailable:6010 : Guest agent is not responding: QEMU guest agent is not connected Interestingly though, now I'm seeing this in the logs which may be a new clue: ==> /var/log/vdsm/vdsm.log <== File "/usr/lib/python2.7/site-packages/vdsm/storage/nfsSD.py", line 126, in findDomain return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) File "/usr/lib/python2.7/site-packages/vdsm/storage/nfsSD.py", line 116, in findDomainPath raise se.StorageDomainDoesNotExist(sdUUID) StorageDomainDoesNotExist: Storage domain does not exist: (u'248f46f0-d793-4581-9810-c9d965e2f286',) jsonrpc/4::ERROR::2018-01-12 12:40:30,380::dispatcher::82::storage.Dispatcher::(wrapper) FINISH getStorageDomainInfo error=Storage domain does not exist: (u'248f46f0-d793-4581-9810-c9d965e2f286',) periodic/42::ERROR::2018-01-12 12:40:35,430::api::196::root::(_getHaInfo) failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? periodic/43::ERROR::2018-01-12 12:40:50,473::api::196::root::(_getHaInfo) failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? periodic/40::ERROR::2018-01-12 12:41:05,519::api::196::root::(_getHaInfo) failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? periodic/43::ERROR::2018-01-12 12:41:20,566::api::196::root::(_getHaInfo) failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? ==> /var/log/ovirt-hosted-engine-ha/broker.log <== File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py", line 151, in get_raw_stats f = os.open(path, direct_flag | os.O_RDONLY | os.O_SYNC) OSError: [Errno 2] No such file or directory: '/var/run/vdsm/storage/248f46f0-d793-4581-9810-c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8' StatusStorageThread::ERROR::2018-01-12 12:32:06,049::status_broker::92::ovirt_hosted_engine_ha.broker.status_broker.StatusBroker.Update::(run) Failed to read state. Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py", line 88, in run self._storage_broker.get_raw_stats() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py", line 162, in get_raw_stats .format(str(e))) RequestError: failed to read metadata: [Errno 2] No such file or directory: '/var/run/vdsm/storage/248f46f0-d793-4581-9810-c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8' On Fri, Jan 12, 2018 at 12:02 PM, Martin Sivak <msivak(a)redhat.com> wrote: > The lock is the issue. > > - try running sanlock client status on all hosts > - also make sure you do not have some forgotten host still connected > to the lockspace, but without ha daemons running (and with the VM) > > I need to go to our president election now, I might check the email > later tonight. > > Martin > > On Fri, Jan 12, 2018 at 4:59 PM, Jayme <jaymef(a)gmail.com> wrote: > > Here are the newest logs from me trying to start hosted vm: > > > > ==> /var/log/messages <== > > Jan 12 11:58:14 cultivar0 kernel: ovirtmgmt: port 6(vnet4) entered > blocking > > state > > Jan 12 11:58:14 cultivar0 kernel: ovirtmgmt: port 6(vnet4) entered > disabled > > state > > Jan 12 11:58:14 cultivar0 kernel: device vnet4 entered promiscuous mode > > Jan 12 11:58:14 cultivar0 kernel: ovirtmgmt: port 6(vnet4) entered > blocking > > state > > Jan 12 11:58:14 cultivar0 kernel: ovirtmgmt: port 6(vnet4) entered > > forwarding state > > Jan 12 11:58:14 cultivar0 lldpad: recvfrom(Event interface): No buffer > space > > available > > Jan 12 11:58:14 cultivar0 NetworkManager[1092]: <info> [1515772694.8715] > > manager: (vnet4): new Tun device > > (/org/freedesktop/NetworkManager/Devices/140) > > Jan 12 11:58:14 cultivar0 NetworkManager[1092]: <info> [1515772694.8795] > > device (vnet4): state change: unmanaged -> unavailable (reason > > 'connection-assumed') [10 20 41] > > > > ==> /var/log/libvirt/qemu/Cultivar.log <== > > 2018-01-12 15:58:14.879+0000: starting up libvirt version: 3.2.0, > package: > > 14.el7_4.7 (CentOS BuildSystem <http://bugs.centos.org>, > > 2018-01-04-19:31:34, c1bm.rdu2.centos.org) qemu version: > > 2.9.0(qemu-kvm-ev-2.9.0-16.el7_4.13.1), hostname: > > cultivar0.grove.silverorange.com > > LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin > > QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm -name > > guest=Cultivar,debug-threads=on -S -object > > secret,id=masterKey0,format=raw,file=/var/lib/libvirt/ > qemu/domain-119-Cultivar/master-key.aes > > -machine pc-i440fx-rhel7.3.0,accel=kvm,usb=off,dump-guest-core=off -cpu > > Conroe -m 8192 -realtime mlock=off -smp > > 2,maxcpus=16,sockets=16,cores=1,threads=1 -uuid > > 4013c829-c9d7-4b72-90d5-6fe58137504c -smbios > > 'type=1,manufacturer=oVirt,product=oVirt > > Node,version=7-4.1708.el7.centos,serial=44454C4C-3300- > 1042-8031-B4C04F4B4831,uuid=4013c829-c9d7-4b72-90d5-6fe58137504c' > > -no-user-config -nodefaults -chardev > > socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain- > 119-Cultivar/monitor.sock,server,nowait > > -mon chardev=charmonitor,id=monitor,mode=control -rtc > > base=2018-01-12T15:58:14,driftfix=slew -global > > kvm-pit.lost_tick_policy=delay -no-hpet -no-reboot -boot strict=on > -device > > piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device > > virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x4 -drive > > file=/var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/c2dde892-f978-4dfc-a421-c8e04cf387f9/23aa0a66-fa6c-4967-a1e5- > fbe47c0cd705,format=raw,if=none,id=drive-virtio-disk0, > serial=c2dde892-f978-4dfc-a421-c8e04cf387f9,cache=none, > werror=stop,rerror=stop,aio=threads > > -device > > virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive- > virtio-disk0,id=virtio-disk0,bootindex=1 > > -drive if=none,id=drive-ide0-1-0,readonly=on -device > > ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev > > tap,fd=35,id=hostnet0,vhost=on,vhostfd=38 -device > > virtio-net-pci,netdev=hostnet0,id=net0,mac=00:16:3e: > 7f:d6:83,bus=pci.0,addr=0x3 > > -chardev > > socket,id=charchannel0,path=/var/lib/libvirt/qemu/channels/ > 4013c829-c9d7-4b72-90d5-6fe58137504c.com.redhat.rhevm.vdsm,server,nowait > > -device > > virtserialport,bus=virtio-serial0.0,nr=1,chardev= > charchannel0,id=channel0,name=com.redhat.rhevm.vdsm > > -chardev > > socket,id=charchannel1,path=/var/lib/libvirt/qemu/channels/ > 4013c829-c9d7-4b72-90d5-6fe58137504c.org.qemu.guest_agent.0,server,nowait > > -device > > virtserialport,bus=virtio-serial0.0,nr=2,chardev= > charchannel1,id=channel1,name=org.qemu.guest_agent.0 > > -chardev spicevmc,id=charchannel2,name=vdagent -device > > virtserialport,bus=virtio-serial0.0,nr=3,chardev= > charchannel2,id=channel2,name=com.redhat.spice.0 > > -chardev > > socket,id=charchannel3,path=/var/lib/libvirt/qemu/channels/ > 4013c829-c9d7-4b72-90d5-6fe58137504c.org.ovirt.hosted- > engine-setup.0,server,nowait > > -device > > virtserialport,bus=virtio-serial0.0,nr=4,chardev= > charchannel3,id=channel3,name=org.ovirt.hosted-engine-setup.0 > > -chardev pty,id=charconsole0 -device > > virtconsole,chardev=charconsole0,id=console0 -spice > > tls-port=5904,addr=0,x509-dir=/etc/pki/vdsm/libvirt-spice, > tls-channel=default,seamless-migration=on > > -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -object > > rng-random,id=objrng0,filename=/dev/urandom -device > > virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x5 -msg timestamp=on > > > > ==> /var/log/messages <== > > Jan 12 11:58:14 cultivar0 NetworkManager[1092]: <info> [1515772694.8807] > > device (vnet4): state change: unavailable -> disconnected (reason 'none') > > [20 30 0] > > Jan 12 11:58:14 cultivar0 systemd-machined: New machine > qemu-119-Cultivar. > > Jan 12 11:58:14 cultivar0 systemd: Started Virtual Machine > > qemu-119-Cultivar. > > Jan 12 11:58:14 cultivar0 systemd: Starting Virtual Machine > > qemu-119-Cultivar. > > > > ==> /var/log/libvirt/qemu/Cultivar.log <== > > 2018-01-12T15:58:15.094002Z qemu-kvm: -chardev pty,id=charconsole0: char > > device redirected to /dev/pts/1 (label charconsole0) > > > > ==> /var/log/messages <== > > Jan 12 11:58:15 cultivar0 kvm: 5 guests now active > > > > ==> /var/log/libvirt/qemu/Cultivar.log <== > > 2018-01-12 15:58:15.217+0000: shutting down, reason=failed > > > > ==> /var/log/messages <== > > Jan 12 11:58:15 cultivar0 libvirtd: 2018-01-12 15:58:15.217+0000: 1908: > > error : virLockManagerSanlockAcquire:1041 : resource busy: Failed to > acquire > > lock: Lease is held by another host > > > > ==> /var/log/libvirt/qemu/Cultivar.log <== > > 2018-01-12T15:58:15.219934Z qemu-kvm: terminating on signal 15 from pid > 1773 > > (/usr/sbin/libvirtd) > > > > ==> /var/log/messages <== > > Jan 12 11:58:15 cultivar0 kernel: ovirtmgmt: port 6(vnet4) entered > disabled > > state > > Jan 12 11:58:15 cultivar0 kernel: device vnet4 left promiscuous mode > > Jan 12 11:58:15 cultivar0 kernel: ovirtmgmt: port 6(vnet4) entered > disabled > > state > > Jan 12 11:58:15 cultivar0 NetworkManager[1092]: <info> [1515772695.2348] > > device (vnet4): state change: disconnected -> unmanaged (reason > 'unmanaged') > > [30 10 3] > > Jan 12 11:58:15 cultivar0 NetworkManager[1092]: <info> [1515772695.2349] > > device (vnet4): released from master device ovirtmgmt > > Jan 12 11:58:15 cultivar0 kvm: 4 guests now active > > Jan 12 11:58:15 cultivar0 systemd-machined: Machine qemu-119-Cultivar > > terminated. > > > > ==> /var/log/vdsm/vdsm.log <== > > vm/4013c829::ERROR::2018-01-12 > > 11:58:15,444::vm::914::virt.vm::(_startUnderlyingVm) > > (vmId='4013c829-c9d7-4b72-90d5-6fe58137504c') The vm start process > failed > > Traceback (most recent call last): > > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 843, in > > _startUnderlyingVm > > self._run() > > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2721, in > > _run > > dom.createWithFlags(flags) > > File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", > line > > 126, in wrapper > > ret = f(*args, **kwargs) > > File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 512, in > > wrapper > > return func(inst, *args, **kwargs) > > File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1069, in > > createWithFlags > > if ret == -1: raise libvirtError ('virDomainCreateWithFlags() > failed', > > dom=self) > > libvirtError: resource busy: Failed to acquire lock: Lease is held by > > another host > > jsonrpc/6::ERROR::2018-01-12 > > 11:58:16,421::__init__::611::jsonrpc.JsonRpcServer::(_handle_request) > > Internal server error > > Traceback (most recent call last): > > File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line > 606, > > in _handle_request > > res = method(**params) > > File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 201, > in > > _dynamicMethod > > result = fn(*methodArgs) > > File "<string>", line 2, in getAllVmIoTunePolicies > > File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, > in > > method > > ret = func(*args, **kwargs) > > File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1354, in > > getAllVmIoTunePolicies > > io_tune_policies_dict = self._cif.getAllVmIoTunePolicies() > > File "/usr/lib/python2.7/site-packages/vdsm/clientIF.py", line 524, in > > getAllVmIoTunePolicies > > 'current_values': v.getIoTune()} > > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 3481, in > > getIoTune > > result = self.getIoTuneResponse() > > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 3500, in > > getIoTuneResponse > > res = self._dom.blockIoTune( > > File "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py", line > 47, > > in __getattr__ > > % self.vmid) > > NotConnectedError: VM '4013c829-c9d7-4b72-90d5-6fe58137504c' was not > defined > > yet or was undefined > > > > ==> /var/log/messages <== > > Jan 12 11:58:16 cultivar0 journal: vdsm jsonrpc.JsonRpcServer ERROR > Internal > > server error#012Traceback (most recent call last):#012 File > > "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 606, in > > _handle_request#012 res = method(**params)#012 File > > "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 201, in > > _dynamicMethod#012 result = fn(*methodArgs)#012 File "<string>", > line 2, > > in getAllVmIoTunePolicies#012 File > > "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in > > method#012 ret = func(*args, **kwargs)#012 File > > "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1354, in > > getAllVmIoTunePolicies#012 io_tune_policies_dict = > > self._cif.getAllVmIoTunePolicies()#012 File > > "/usr/lib/python2.7/site-packages/vdsm/clientIF.py", line 524, in > > getAllVmIoTunePolicies#012 'current_values': v.getIoTune()}#012 File > > "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 3481, in > > getIoTune#012 result = self.getIoTuneResponse()#012 File > > "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 3500, in > > getIoTuneResponse#012 res = self._dom.blockIoTune(#012 File > > "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py", line 47, in > > __getattr__#012 % self.vmid)#012NotConnectedError: VM > > '4013c829-c9d7-4b72-90d5-6fe58137504c' was not defined yet or was > undefined > > > > On Fri, Jan 12, 2018 at 11:55 AM, Jayme <jaymef(a)gmail.com> wrote: > >> > >> One other tidbit I noticed is that it seems like there are less errors > if > >> I started in paused mode: > >> > >> but still shows: Engine status : {"reason": "bad vm > >> status", "health": "bad", "vm": "up", "detail": "Paused"} > >> > >> ==> /var/log/messages <== > >> Jan 12 11:55:05 cultivar0 kernel: ovirtmgmt: port 6(vnet4) entered > >> blocking state > >> Jan 12 11:55:05 cultivar0 kernel: ovirtmgmt: port 6(vnet4) entered > >> disabled state > >> Jan 12 11:55:05 cultivar0 kernel: device vnet4 entered promiscuous mode > >> Jan 12 11:55:05 cultivar0 kernel: ovirtmgmt: port 6(vnet4) entered > >> blocking state > >> Jan 12 11:55:05 cultivar0 kernel: ovirtmgmt: port 6(vnet4) entered > >> forwarding state > >> Jan 12 11:55:05 cultivar0 lldpad: recvfrom(Event interface): No buffer > >> space available > >> Jan 12 11:55:05 cultivar0 NetworkManager[1092]: <info> > [1515772505.3625] > >> manager: (vnet4): new Tun device > >> (/org/freedesktop/NetworkManager/Devices/139) > >> > >> ==> /var/log/libvirt/qemu/Cultivar.log <== > >> 2018-01-12 15:55:05.370+0000: starting up libvirt version: 3.2.0, > package: > >> 14.el7_4.7 (CentOS BuildSystem <http://bugs.centos.org>, > >> 2018-01-04-19:31:34, c1bm.rdu2.centos.org) qemu version: > >> 2.9.0(qemu-kvm-ev-2.9.0-16.el7_4.13.1), hostname: > >> cultivar0.grove.silverorange.com > >> LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin > >> QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm -name > >> guest=Cultivar,debug-threads=on -S -object > >> secret,id=masterKey0,format=raw,file=/var/lib/libvirt/ > qemu/domain-118-Cultivar/master-key.aes > >> -machine pc-i440fx-rhel7.3.0,accel=kvm,usb=off,dump-guest-core=off -cpu > >> Conroe -m 8192 -realtime mlock=off -smp > >> 2,maxcpus=16,sockets=16,cores=1,threads=1 -uuid > >> 4013c829-c9d7-4b72-90d5-6fe58137504c -smbios > >> 'type=1,manufacturer=oVirt,product=oVirt > >> Node,version=7-4.1708.el7.centos,serial=44454C4C-3300- > 1042-8031-B4C04F4B4831,uuid=4013c829-c9d7-4b72-90d5-6fe58137504c' > >> -no-user-config -nodefaults -chardev > >> socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain- > 118-Cultivar/monitor.sock,server,nowait > >> -mon chardev=charmonitor,id=monitor,mode=control -rtc > >> base=2018-01-12T15:55:05,driftfix=slew -global > >> kvm-pit.lost_tick_policy=delay -no-hpet -no-reboot -boot strict=on > -device > >> piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device > >> virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x4 -drive > >> file=/var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/c2dde892-f978-4dfc-a421-c8e04cf387f9/23aa0a66-fa6c-4967-a1e5- > fbe47c0cd705,format=raw,if=none,id=drive-virtio-disk0, > serial=c2dde892-f978-4dfc-a421-c8e04cf387f9,cache=none, > werror=stop,rerror=stop,aio=threads > >> -device > >> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive- > virtio-disk0,id=virtio-disk0,bootindex=1 > >> -drive if=none,id=drive-ide0-1-0,readonly=on -device > >> ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev > >> tap,fd=35,id=hostnet0,vhost=on,vhostfd=38 -device > >> virtio-net-pci,netdev=hostnet0,id=net0,mac=00:16:3e: > 7f:d6:83,bus=pci.0,addr=0x3 > >> -chardev > >> socket,id=charchannel0,path=/var/lib/libvirt/qemu/channels/ > 4013c829-c9d7-4b72-90d5-6fe58137504c.com.redhat.rhevm.vdsm,server,nowait > >> -device > >> virtserialport,bus=virtio-serial0.0,nr=1,chardev= > charchannel0,id=channel0,name=com.redhat.rhevm.vdsm > >> -chardev > >> socket,id=charchannel1,path=/var/lib/libvirt/qemu/channels/ > 4013c829-c9d7-4b72-90d5-6fe58137504c.org.qemu.guest_agent.0,server,nowait > >> -device > >> virtserialport,bus=virtio-serial0.0,nr=2,chardev= > charchannel1,id=channel1,name=org.qemu.guest_agent.0 > >> -chardev spicevmc,id=charchannel2,name=vdagent -device > >> virtserialport,bus=virtio-serial0.0,nr=3,chardev= > charchannel2,id=channel2,name=com.redhat.spice.0 > >> -chardev > >> socket,id=charchannel3,path=/var/lib/libvirt/qemu/channels/ > 4013c829-c9d7-4b72-90d5-6fe58137504c.org.ovirt.hosted- > engine-setup.0,server,nowait > >> -device > >> virtserialport,bus=virtio-serial0.0,nr=4,chardev= > charchannel3,id=channel3,name=org.ovirt.hosted-engine-setup.0 > >> -chardev pty,id=charconsole0 -device > >> virtconsole,chardev=charconsole0,id=console0 -spice > >> tls-port=5904,addr=0,x509-dir=/etc/pki/vdsm/libvirt-spice, > tls-channel=default,seamless-migration=on > >> -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -object > >> rng-random,id=objrng0,filename=/dev/urandom -device > >> virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x5 -msg timestamp=on > >> > >> ==> /var/log/messages <== > >> Jan 12 11:55:05 cultivar0 NetworkManager[1092]: <info> > [1515772505.3689] > >> device (vnet4): state change: unmanaged -> unavailable (reason > >> 'connection-assumed') [10 20 41] > >> Jan 12 11:55:05 cultivar0 NetworkManager[1092]: <info> > [1515772505.3702] > >> device (vnet4): state change: unavailable -> disconnected (reason > 'none') > >> [20 30 0] > >> Jan 12 11:55:05 cultivar0 systemd-machined: New machine > qemu-118-Cultivar. > >> Jan 12 11:55:05 cultivar0 systemd: Started Virtual Machine > >> qemu-118-Cultivar. > >> Jan 12 11:55:05 cultivar0 systemd: Starting Virtual Machine > >> qemu-118-Cultivar. > >> > >> ==> /var/log/libvirt/qemu/Cultivar.log <== > >> 2018-01-12T15:55:05.586827Z qemu-kvm: -chardev pty,id=charconsole0: char > >> device redirected to /dev/pts/1 (label charconsole0) > >> > >> ==> /var/log/messages <== > >> Jan 12 11:55:05 cultivar0 kvm: 5 guests now active > >> > >> On Fri, Jan 12, 2018 at 11:36 AM, Jayme <jaymef(a)gmail.com> wrote: > >>> > >>> Yeah I am in global maintenance: > >>> > >>> state=GlobalMaintenance > >>> > >>> host0: {"reason": "vm not running on this host", "health": "bad", > "vm": > >>> "down", "detail": "unknown"} > >>> host2: {"reason": "vm not running on this host", "health": "bad", "vm": > >>> "down", "detail": "unknown"} > >>> host3: {"reason": "vm not running on this host", "health": "bad", "vm": > >>> "down", "detail": "unknown"} > >>> > >>> I understand the lock is an issue, I'll try to make sure it is fully > >>> stopped on all three before starting but I don't think that is the > issue at > >>> hand either. What concerns me is mostly that it seems to be unable > to read > >>> the meta data, I think that might be the heart of the problem but I'm > not > >>> sure what is causing it. > >>> > >>> On Fri, Jan 12, 2018 at 11:33 AM, Martin Sivak <msivak(a)redhat.com> > wrote: > >>>> > >>>> > On all three hosts I ran hosted-engine --vm-shutdown; hosted-engine > >>>> > --vm-poweroff > >>>> > >>>> Are you in global maintenance? I think you were in one of the previous > >>>> emails, but worth checking. > >>>> > >>>> > I started ovirt-ha-broker with systemctl as root user but it does > >>>> > appear to be running under vdsm: > >>>> > >>>> That is the correct behavior. > >>>> > >>>> > libvirtError: resource busy: Failed to acquire lock: Lease is held > by > >>>> > another host > >>>> > >>>> sanlock seems to think the VM runs somewhere and it is possible that > >>>> some other host tried to start the VM as well unless you are in global > >>>> maintenance (that is why I asked the first question here). > >>>> > >>>> Martin > >>>> > >>>> On Fri, Jan 12, 2018 at 4:28 PM, Jayme <jaymef(a)gmail.com> wrote: > >>>> > Martin, > >>>> > > >>>> > Thanks so much for keeping with me, this is driving me crazy! I > >>>> > really do > >>>> > appreciate it, thanks again > >>>> > > >>>> > Let's go through this: > >>>> > > >>>> > HE VM is down - YES > >>>> > > >>>> > > >>>> > HE agent fails when opening metadata using the symlink - YES > >>>> > > >>>> > > >>>> > the symlink is there and readable by vdsm:kvm - it appears to be: > >>>> > > >>>> > > >>>> > lrwxrwxrwx. 1 vdsm kvm 159 Jan 10 21:20 > >>>> > 14a20941-1b84-4b82-be8f-ace38d7c037a > >>>> > -> > >>>> > > >>>> > /rhev/data-center/mnt/cultivar0.grove.silverorange. > com:_exports_hosted__engine/248f46f0-d793-4581-9810- > c9d965e2f286/images/14a20941-1b84-4b82-be8f-ace38d7c037a > >>>> > > >>>> > > >>>> > And the files in the linked directory exist and have vdsm:kvm perms > as > >>>> > well: > >>>> > > >>>> > > >>>> > # cd > >>>> > > >>>> > /rhev/data-center/mnt/cultivar0.grove.silverorange. > com:_exports_hosted__engine/248f46f0-d793-4581-9810- > c9d965e2f286/images/14a20941-1b84-4b82-be8f-ace38d7c037a > >>>> > > >>>> > [root@cultivar0 14a20941-1b84-4b82-be8f-ace38d7c037a]# ls -al > >>>> > > >>>> > total 2040 > >>>> > > >>>> > drwxr-xr-x. 2 vdsm kvm 4096 Jan 12 10:51 . > >>>> > > >>>> > drwxr-xr-x. 8 vdsm kvm 4096 Feb 3 2016 .. > >>>> > > >>>> > -rw-rw----. 1 vdsm kvm 1028096 Jan 12 11:19 > >>>> > 8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8 > >>>> > > >>>> > -rw-rw----. 1 vdsm kvm 1048576 Feb 3 2016 > >>>> > 8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8.lease > >>>> > > >>>> > -rw-r--r--. 1 vdsm kvm 283 Feb 3 2016 > >>>> > 8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8.meta > >>>> > > >>>> > > >>>> > I started ovirt-ha-broker with systemctl as root user but it does > >>>> > appear to > >>>> > be running under vdsm: > >>>> > > >>>> > > >>>> > vdsm 16928 0.6 0.0 1618244 43328 ? Ssl 10:33 0:18 > >>>> > /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker > >>>> > > >>>> > > >>>> > > >>>> > Here is something I tried: > >>>> > > >>>> > > >>>> > - On all three hosts I ran hosted-engine --vm-shutdown; > hosted-engine > >>>> > --vm-poweroff > >>>> > > >>>> > - On HOST0 (cultivar0) I disconnected and reconnected storage using > >>>> > hosted-engine > >>>> > > >>>> > - Tried starting up the hosted VM on cultivar0 while tailing the > logs: > >>>> > > >>>> > > >>>> > # hosted-engine --vm-start > >>>> > > >>>> > VM exists and is down, cleaning up and restarting > >>>> > > >>>> > > >>>> > > >>>> > ==> /var/log/vdsm/vdsm.log <== > >>>> > > >>>> > jsonrpc/2::ERROR::2018-01-12 > >>>> > 11:27:27,194::vm::1766::virt.vm::(_getRunningVmStats) > >>>> > (vmId='4013c829-c9d7-4b72-90d5-6fe58137504c') Error fetching vm > stats > >>>> > > >>>> > Traceback (most recent call last): > >>>> > > >>>> > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line > 1762, > >>>> > in > >>>> > _getRunningVmStats > >>>> > > >>>> > vm_sample.interval) > >>>> > > >>>> > File "/usr/lib/python2.7/site-packages/vdsm/virt/vmstats.py", > line > >>>> > 45, in > >>>> > produce > >>>> > > >>>> > networks(vm, stats, first_sample, last_sample, interval) > >>>> > > >>>> > File "/usr/lib/python2.7/site-packages/vdsm/virt/vmstats.py", > line > >>>> > 322, in > >>>> > networks > >>>> > > >>>> > if nic.name.startswith('hostdev'): > >>>> > > >>>> > AttributeError: name > >>>> > > >>>> > jsonrpc/3::ERROR::2018-01-12 > >>>> > 11:27:27,221::__init__::611::jsonrpc.JsonRpcServer::(_ > handle_request) > >>>> > Internal server error > >>>> > > >>>> > Traceback (most recent call last): > >>>> > > >>>> > File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", > line > >>>> > 606, > >>>> > in _handle_request > >>>> > > >>>> > res = method(**params) > >>>> > > >>>> > File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line > >>>> > 201, in > >>>> > _dynamicMethod > >>>> > > >>>> > result = fn(*methodArgs) > >>>> > > >>>> > File "<string>", line 2, in getAllVmIoTunePolicies > >>>> > > >>>> > File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line > 48, > >>>> > in > >>>> > method > >>>> > > >>>> > ret = func(*args, **kwargs) > >>>> > > >>>> > File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1354, > in > >>>> > getAllVmIoTunePolicies > >>>> > > >>>> > io_tune_policies_dict = self._cif.getAllVmIoTunePolicies() > >>>> > > >>>> > File "/usr/lib/python2.7/site-packages/vdsm/clientIF.py", line > 524, > >>>> > in > >>>> > getAllVmIoTunePolicies > >>>> > > >>>> > 'current_values': v.getIoTune()} > >>>> > > >>>> > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line > 3481, > >>>> > in > >>>> > getIoTune > >>>> > > >>>> > result = self.getIoTuneResponse() > >>>> > > >>>> > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line > 3500, > >>>> > in > >>>> > getIoTuneResponse > >>>> > > >>>> > res = self._dom.blockIoTune( > >>>> > > >>>> > File "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py", > line > >>>> > 47, > >>>> > in __getattr__ > >>>> > > >>>> > % self.vmid) > >>>> > > >>>> > NotConnectedError: VM '4013c829-c9d7-4b72-90d5-6fe58137504c' was > not > >>>> > defined > >>>> > yet or was undefined > >>>> > > >>>> > > >>>> > ==> /var/log/messages <== > >>>> > > >>>> > Jan 12 11:27:27 cultivar0 journal: vdsm jsonrpc.JsonRpcServer ERROR > >>>> > Internal > >>>> > server error#012Traceback (most recent call last):#012 File > >>>> > "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line > 606, in > >>>> > _handle_request#012 res = method(**params)#012 File > >>>> > "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 201, in > >>>> > _dynamicMethod#012 result = fn(*methodArgs)#012 File "<string>", > >>>> > line 2, > >>>> > in getAllVmIoTunePolicies#012 File > >>>> > "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in > >>>> > method#012 ret = func(*args, **kwargs)#012 File > >>>> > "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1354, in > >>>> > getAllVmIoTunePolicies#012 io_tune_policies_dict = > >>>> > self._cif.getAllVmIoTunePolicies()#012 File > >>>> > "/usr/lib/python2.7/site-packages/vdsm/clientIF.py", line 524, in > >>>> > getAllVmIoTunePolicies#012 'current_values': v.getIoTune()}#012 > >>>> > File > >>>> > "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 3481, in > >>>> > getIoTune#012 result = self.getIoTuneResponse()#012 File > >>>> > "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 3500, in > >>>> > getIoTuneResponse#012 res = self._dom.blockIoTune(#012 File > >>>> > "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py", line > 47, in > >>>> > __getattr__#012 % self.vmid)#012NotConnectedError: VM > >>>> > '4013c829-c9d7-4b72-90d5-6fe58137504c' was not defined yet or was > >>>> > undefined > >>>> > > >>>> > Jan 12 11:27:27 cultivar0 kernel: ovirtmgmt: port 6(vnet4) entered > >>>> > blocking > >>>> > state > >>>> > > >>>> > Jan 12 11:27:27 cultivar0 kernel: ovirtmgmt: port 6(vnet4) entered > >>>> > disabled > >>>> > state > >>>> > > >>>> > Jan 12 11:27:27 cultivar0 kernel: device vnet4 entered promiscuous > >>>> > mode > >>>> > > >>>> > Jan 12 11:27:27 cultivar0 kernel: ovirtmgmt: port 6(vnet4) entered > >>>> > blocking > >>>> > state > >>>> > > >>>> > Jan 12 11:27:27 cultivar0 kernel: ovirtmgmt: port 6(vnet4) entered > >>>> > forwarding state > >>>> > > >>>> > Jan 12 11:27:27 cultivar0 lldpad: recvfrom(Event interface): No > buffer > >>>> > space > >>>> > available > >>>> > > >>>> > Jan 12 11:27:27 cultivar0 NetworkManager[1092]: <info> > >>>> > [1515770847.4264] > >>>> > manager: (vnet4): new Tun device > >>>> > (/org/freedesktop/NetworkManager/Devices/135) > >>>> > > >>>> > Jan 12 11:27:27 cultivar0 NetworkManager[1092]: <info> > >>>> > [1515770847.4342] > >>>> > device (vnet4): state change: unmanaged -> unavailable (reason > >>>> > 'connection-assumed') [10 20 41] > >>>> > > >>>> > Jan 12 11:27:27 cultivar0 NetworkManager[1092]: <info> > >>>> > [1515770847.4353] > >>>> > device (vnet4): state change: unavailable -> disconnected (reason > >>>> > 'none') > >>>> > [20 30 0] > >>>> > > >>>> > > >>>> > ==> /var/log/libvirt/qemu/Cultivar.log <== > >>>> > > >>>> > 2018-01-12 15:27:27.435+0000: starting up libvirt version: 3.2.0, > >>>> > package: > >>>> > 14.el7_4.7 (CentOS BuildSystem <http://bugs.centos.org>, > >>>> > 2018-01-04-19:31:34, c1bm.rdu2.centos.org) qemu version: > >>>> > 2.9.0(qemu-kvm-ev-2.9.0-16.el7_4.13.1), hostname: > >>>> > cultivar0.grove.silverorange.com > >>>> > > >>>> > LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin > >>>> > QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm -name > >>>> > guest=Cultivar,debug-threads=on -S -object > >>>> > > >>>> > secret,id=masterKey0,format=raw,file=/var/lib/libvirt/ > qemu/domain-114-Cultivar/master-key.aes > >>>> > -machine pc-i440fx-rhel7.3.0,accel=kvm,usb=off,dump-guest-core=off > >>>> > -cpu > >>>> > Conroe -m 8192 -realtime mlock=off -smp > >>>> > 2,maxcpus=16,sockets=16,cores=1,threads=1 -uuid > >>>> > 4013c829-c9d7-4b72-90d5-6fe58137504c -smbios > >>>> > 'type=1,manufacturer=oVirt,product=oVirt > >>>> > > >>>> > Node,version=7-4.1708.el7.centos,serial=44454C4C-3300- > 1042-8031-B4C04F4B4831,uuid=4013c829-c9d7-4b72-90d5-6fe58137504c' > >>>> > -no-user-config -nodefaults -chardev > >>>> > > >>>> > socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain- > 114-Cultivar/monitor.sock,server,nowait > >>>> > -mon chardev=charmonitor,id=monitor,mode=control -rtc > >>>> > base=2018-01-12T15:27:27,driftfix=slew -global > >>>> > kvm-pit.lost_tick_policy=delay -no-hpet -no-reboot -boot strict=on > >>>> > -device > >>>> > piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device > >>>> > virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x4 -drive > >>>> > > >>>> > file=/var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/c2dde892-f978-4dfc-a421-c8e04cf387f9/23aa0a66-fa6c-4967-a1e5- > fbe47c0cd705,format=raw,if=none,id=drive-virtio-disk0, > serial=c2dde892-f978-4dfc-a421-c8e04cf387f9,cache=none, > werror=stop,rerror=stop,aio=threads > >>>> > -device > >>>> > > >>>> > virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive- > virtio-disk0,id=virtio-disk0,bootindex=1 > >>>> > -drive if=none,id=drive-ide0-1-0,readonly=on -device > >>>> > ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev > >>>> > tap,fd=35,id=hostnet0,vhost=on,vhostfd=38 -device > >>>> > > >>>> > virtio-net-pci,netdev=hostnet0,id=net0,mac=00:16:3e: > 7f:d6:83,bus=pci.0,addr=0x3 > >>>> > -chardev > >>>> > > >>>> > socket,id=charchannel0,path=/var/lib/libvirt/qemu/channels/ > 4013c829-c9d7-4b72-90d5-6fe58137504c.com.redhat.rhevm.vdsm,server,nowait > >>>> > -device > >>>> > > >>>> > virtserialport,bus=virtio-serial0.0,nr=1,chardev= > charchannel0,id=channel0,name=com.redhat.rhevm.vdsm > >>>> > -chardev > >>>> > > >>>> > socket,id=charchannel1,path=/var/lib/libvirt/qemu/channels/ > 4013c829-c9d7-4b72-90d5-6fe58137504c.org.qemu.guest_agent.0,server,nowait > >>>> > -device > >>>> > > >>>> > virtserialport,bus=virtio-serial0.0,nr=2,chardev= > charchannel1,id=channel1,name=org.qemu.guest_agent.0 > >>>> > -chardev spicevmc,id=charchannel2,name=vdagent -device > >>>> > > >>>> > virtserialport,bus=virtio-serial0.0,nr=3,chardev= > charchannel2,id=channel2,name=com.redhat.spice.0 > >>>> > -chardev > >>>> > > >>>> > socket,id=charchannel3,path=/var/lib/libvirt/qemu/channels/ > 4013c829-c9d7-4b72-90d5-6fe58137504c.org.ovirt.hosted- > engine-setup.0,server,nowait > >>>> > -device > >>>> > > >>>> > virtserialport,bus=virtio-serial0.0,nr=4,chardev= > charchannel3,id=channel3,name=org.ovirt.hosted-engine-setup.0 > >>>> > -chardev pty,id=charconsole0 -device > >>>> > virtconsole,chardev=charconsole0,id=console0 -spice > >>>> > > >>>> > tls-port=5904,addr=0,x509-dir=/etc/pki/vdsm/libvirt-spice, > tls-channel=default,seamless-migration=on > >>>> > -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -object > >>>> > rng-random,id=objrng0,filename=/dev/urandom -device > >>>> > virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x5 -msg > >>>> > timestamp=on > >>>> > > >>>> > > >>>> > ==> /var/log/messages <== > >>>> > > >>>> > Jan 12 11:27:27 cultivar0 systemd-machined: New machine > >>>> > qemu-114-Cultivar. > >>>> > > >>>> > Jan 12 11:27:27 cultivar0 systemd: Started Virtual Machine > >>>> > qemu-114-Cultivar. > >>>> > > >>>> > Jan 12 11:27:27 cultivar0 systemd: Starting Virtual Machine > >>>> > qemu-114-Cultivar. > >>>> > > >>>> > > >>>> > ==> /var/log/libvirt/qemu/Cultivar.log <== > >>>> > > >>>> > 2018-01-12T15:27:27.651669Z qemu-kvm: -chardev pty,id=charconsole0: > >>>> > char > >>>> > device redirected to /dev/pts/2 (label charconsole0) > >>>> > > >>>> > > >>>> > ==> /var/log/messages <== > >>>> > > >>>> > Jan 12 11:27:27 cultivar0 kvm: 5 guests now active > >>>> > > >>>> > > >>>> > ==> /var/log/libvirt/qemu/Cultivar.log <== > >>>> > > >>>> > 2018-01-12 15:27:27.773+0000: shutting down, reason=failed > >>>> > > >>>> > > >>>> > ==> /var/log/messages <== > >>>> > > >>>> > Jan 12 11:27:27 cultivar0 libvirtd: 2018-01-12 15:27:27.773+0000: > >>>> > 1910: > >>>> > error : virLockManagerSanlockAcquire:1041 : resource busy: Failed > to > >>>> > acquire > >>>> > lock: Lease is held by another host > >>>> > > >>>> > > >>>> > ==> /var/log/libvirt/qemu/Cultivar.log <== > >>>> > > >>>> > 2018-01-12T15:27:27.776135Z qemu-kvm: terminating on signal 15 from > >>>> > pid 1773 > >>>> > (/usr/sbin/libvirtd) > >>>> > > >>>> > > >>>> > ==> /var/log/messages <== > >>>> > > >>>> > Jan 12 11:27:27 cultivar0 kernel: ovirtmgmt: port 6(vnet4) entered > >>>> > disabled > >>>> > state > >>>> > > >>>> > Jan 12 11:27:27 cultivar0 kernel: device vnet4 left promiscuous mode > >>>> > > >>>> > Jan 12 11:27:27 cultivar0 kernel: ovirtmgmt: port 6(vnet4) entered > >>>> > disabled > >>>> > state > >>>> > > >>>> > Jan 12 11:27:27 cultivar0 NetworkManager[1092]: <info> > >>>> > [1515770847.7989] > >>>> > device (vnet4): state change: disconnected -> unmanaged (reason > >>>> > 'unmanaged') > >>>> > [30 10 3] > >>>> > > >>>> > Jan 12 11:27:27 cultivar0 NetworkManager[1092]: <info> > >>>> > [1515770847.7989] > >>>> > device (vnet4): released from master device ovirtmgmt > >>>> > > >>>> > Jan 12 11:27:27 cultivar0 kvm: 4 guests now active > >>>> > > >>>> > Jan 12 11:27:27 cultivar0 systemd-machined: Machine > qemu-114-Cultivar > >>>> > terminated. > >>>> > > >>>> > > >>>> > ==> /var/log/vdsm/vdsm.log <== > >>>> > > >>>> > vm/4013c829::ERROR::2018-01-12 > >>>> > 11:27:28,001::vm::914::virt.vm::(_startUnderlyingVm) > >>>> > (vmId='4013c829-c9d7-4b72-90d5-6fe58137504c') The vm start process > >>>> > failed > >>>> > > >>>> > Traceback (most recent call last): > >>>> > > >>>> > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line > 843, > >>>> > in > >>>> > _startUnderlyingVm > >>>> > > >>>> > self._run() > >>>> > > >>>> > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line > 2721, > >>>> > in > >>>> > _run > >>>> > > >>>> > dom.createWithFlags(flags) > >>>> > > >>>> > File "/usr/lib/python2.7/site-packages/vdsm/ > libvirtconnection.py", > >>>> > line > >>>> > 126, in wrapper > >>>> > > >>>> > ret = f(*args, **kwargs) > >>>> > > >>>> > File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 512, > in > >>>> > wrapper > >>>> > > >>>> > return func(inst, *args, **kwargs) > >>>> > > >>>> > File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1069, > in > >>>> > createWithFlags > >>>> > > >>>> > if ret == -1: raise libvirtError ('virDomainCreateWithFlags() > >>>> > failed', > >>>> > dom=self) > >>>> > > >>>> > libvirtError: resource busy: Failed to acquire lock: Lease is held > by > >>>> > another host > >>>> > > >>>> > periodic/47::ERROR::2018-01-12 > >>>> > 11:27:32,858::periodic::215::virt.periodic.Operation::(__call__) > >>>> > <vdsm.virt.sampling.VMBulkstatsMonitor object at 0x3692590> > operation > >>>> > failed > >>>> > > >>>> > Traceback (most recent call last): > >>>> > > >>>> > File "/usr/lib/python2.7/site-packages/vdsm/virt/periodic.py", > line > >>>> > 213, > >>>> > in __call__ > >>>> > > >>>> > self._func() > >>>> > > >>>> > File "/usr/lib/python2.7/site-packages/vdsm/virt/sampling.py", > line > >>>> > 522, > >>>> > in __call__ > >>>> > > >>>> > self._send_metrics() > >>>> > > >>>> > File "/usr/lib/python2.7/site-packages/vdsm/virt/sampling.py", > line > >>>> > 538, > >>>> > in _send_metrics > >>>> > > >>>> > vm_sample.interval) > >>>> > > >>>> > File "/usr/lib/python2.7/site-packages/vdsm/virt/vmstats.py", > line > >>>> > 45, in > >>>> > produce > >>>> > > >>>> > networks(vm, stats, first_sample, last_sample, interval) > >>>> > > >>>> > File "/usr/lib/python2.7/site-packages/vdsm/virt/vmstats.py", > line > >>>> > 322, in > >>>> > networks > >>>> > > >>>> > if nic.name.startswith('hostdev'): > >>>> > > >>>> > AttributeError: name > >>>> > > >>>> > > >>>> > On Fri, Jan 12, 2018 at 11:14 AM, Martin Sivak <msivak(a)redhat.com> > >>>> > wrote: > >>>> >> > >>>> >> Hmm that rules out most of NFS related permission issues. > >>>> >> > >>>> >> So the current status is (I need to sum it up to get the full > >>>> >> picture): > >>>> >> > >>>> >> - HE VM is down > >>>> >> - HE agent fails when opening metadata using the symlink > >>>> >> - the symlink is there > >>>> >> - the symlink is readable by vdsm:kvm > >>>> >> > >>>> >> Hmm can you check under which user is ovirt-ha-broker started? > >>>> >> > >>>> >> Martin > >>>> >> > >>>> >> > >>>> >> On Fri, Jan 12, 2018 at 4:10 PM, Jayme <jaymef(a)gmail.com> wrote: > >>>> >> > Same thing happens with data images of other VMs as well though, > >>>> >> > and > >>>> >> > those > >>>> >> > seem to be running ok so I'm not sure if it's the problem. > >>>> >> > > >>>> >> > On Fri, Jan 12, 2018 at 11:08 AM, Jayme <jaymef(a)gmail.com> > wrote: > >>>> >> >> > >>>> >> >> Martin, > >>>> >> >> > >>>> >> >> I can as VDSM user but not as root . I get permission denied > >>>> >> >> trying to > >>>> >> >> touch one of the files as root, is that normal? > >>>> >> >> > >>>> >> >> On Fri, Jan 12, 2018 at 11:03 AM, Martin Sivak < > msivak(a)redhat.com> > >>>> >> >> wrote: > >>>> >> >>> > >>>> >> >>> Hmm, then it might be a permission issue indeed. Can you touch > >>>> >> >>> the > >>>> >> >>> file? Open it? (try hexdump) Just to make sure NFS does not > >>>> >> >>> prevent > >>>> >> >>> you from doing that. > >>>> >> >>> > >>>> >> >>> Martin > >>>> >> >>> > >>>> >> >>> On Fri, Jan 12, 2018 at 3:57 PM, Jayme <jaymef(a)gmail.com> > wrote: > >>>> >> >>> > Sorry, I think we got confused about the symlink, there are > >>>> >> >>> > symlinks > >>>> >> >>> > in > >>>> >> >>> > /var/run that point the /rhev when I was doing an LS it was > >>>> >> >>> > listing > >>>> >> >>> > the > >>>> >> >>> > files in /rhev > >>>> >> >>> > > >>>> >> >>> > /var/run/vdsm/storage/248f46f0-d793-4581-9810-c9d965e2f286 > >>>> >> >>> > > >>>> >> >>> > 14a20941-1b84-4b82-be8f-ace38d7c037a -> > >>>> >> >>> > > >>>> >> >>> > > >>>> >> >>> > > >>>> >> >>> > /rhev/data-center/mnt/cultivar0.grove.silverorange. > com:_exports_hosted__engine/248f46f0-d793-4581-9810- > c9d965e2f286/images/14a20941-1b84-4b82-be8f-ace38d7c037a > >>>> >> >>> > > >>>> >> >>> > ls -al > >>>> >> >>> > > >>>> >> >>> > > >>>> >> >>> > > >>>> >> >>> > /rhev/data-center/mnt/cultivar0.grove.silverorange. > com:_exports_hosted__engine/248f46f0-d793-4581-9810- > c9d965e2f286/images/14a20941-1b84-4b82-be8f-ace38d7c037a > >>>> >> >>> > total 2040 > >>>> >> >>> > drwxr-xr-x. 2 vdsm kvm 4096 Jan 12 10:51 . > >>>> >> >>> > drwxr-xr-x. 8 vdsm kvm 4096 Feb 3 2016 .. > >>>> >> >>> > -rw-rw----. 1 vdsm kvm 1028096 Jan 12 10:56 > >>>> >> >>> > 8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8 > >>>> >> >>> > -rw-rw----. 1 vdsm kvm 1048576 Feb 3 2016 > >>>> >> >>> > 8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8.lease > >>>> >> >>> > -rw-r--r--. 1 vdsm kvm 283 Feb 3 2016 > >>>> >> >>> > 8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8.meta > >>>> >> >>> > > >>>> >> >>> > Is it possible that this is the wrong image for hosted > engine? > >>>> >> >>> > > >>>> >> >>> > this is all I get in vdsm log when running hosted-engine > >>>> >> >>> > --connect-storage > >>>> >> >>> > > >>>> >> >>> > jsonrpc/4::ERROR::2018-01-12 > >>>> >> >>> > > >>>> >> >>> > > >>>> >> >>> > 10:52:53,019::__init__::611::jsonrpc.JsonRpcServer::(_ > handle_request) > >>>> >> >>> > Internal server error > >>>> >> >>> > Traceback (most recent call last): > >>>> >> >>> > File > >>>> >> >>> > "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", > >>>> >> >>> > line > >>>> >> >>> > 606, > >>>> >> >>> > in _handle_request > >>>> >> >>> > res = method(**params) > >>>> >> >>> > File "/usr/lib/python2.7/site- > packages/vdsm/rpc/Bridge.py", > >>>> >> >>> > line > >>>> >> >>> > 201, > >>>> >> >>> > in > >>>> >> >>> > _dynamicMethod > >>>> >> >>> > result = fn(*methodArgs) > >>>> >> >>> > File "<string>", line 2, in getAllVmIoTunePolicies > >>>> >> >>> > File "/usr/lib/python2.7/site- > packages/vdsm/common/api.py", > >>>> >> >>> > line > >>>> >> >>> > 48, > >>>> >> >>> > in > >>>> >> >>> > method > >>>> >> >>> > ret = func(*args, **kwargs) > >>>> >> >>> > File "/usr/lib/python2.7/site-packages/vdsm/API.py", line > >>>> >> >>> > 1354, in > >>>> >> >>> > getAllVmIoTunePolicies > >>>> >> >>> > io_tune_policies_dict = self._cif. > getAllVmIoTunePolicies() > >>>> >> >>> > File "/usr/lib/python2.7/site-packages/vdsm/clientIF.py", > >>>> >> >>> > line > >>>> >> >>> > 524, > >>>> >> >>> > in > >>>> >> >>> > getAllVmIoTunePolicies > >>>> >> >>> > 'current_values': v.getIoTune()} > >>>> >> >>> > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", > line > >>>> >> >>> > 3481, > >>>> >> >>> > in > >>>> >> >>> > getIoTune > >>>> >> >>> > result = self.getIoTuneResponse() > >>>> >> >>> > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", > line > >>>> >> >>> > 3500, > >>>> >> >>> > in > >>>> >> >>> > getIoTuneResponse > >>>> >> >>> > res = self._dom.blockIoTune( > >>>> >> >>> > File > >>>> >> >>> > "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py", > >>>> >> >>> > line > >>>> >> >>> > 47, > >>>> >> >>> > in __getattr__ > >>>> >> >>> > % self.vmid) > >>>> >> >>> > NotConnectedError: VM '4013c829-c9d7-4b72-90d5-6fe58137504c' > >>>> >> >>> > was not > >>>> >> >>> > defined > >>>> >> >>> > yet or was undefined > >>>> >> >>> > > >>>> >> >>> > On Fri, Jan 12, 2018 at 10:48 AM, Martin Sivak > >>>> >> >>> > <msivak(a)redhat.com> > >>>> >> >>> > wrote: > >>>> >> >>> >> > >>>> >> >>> >> Hi, > >>>> >> >>> >> > >>>> >> >>> >> what happens when you try hosted-engine --connect-storage? > Do > >>>> >> >>> >> you > >>>> >> >>> >> see > >>>> >> >>> >> any errors in the vdsm log? > >>>> >> >>> >> > >>>> >> >>> >> Best regards > >>>> >> >>> >> > >>>> >> >>> >> Martin Sivak > >>>> >> >>> >> > >>>> >> >>> >> On Fri, Jan 12, 2018 at 3:41 PM, Jayme <jaymef(a)gmail.com> > >>>> >> >>> >> wrote: > >>>> >> >>> >> > Ok this is what I've done: > >>>> >> >>> >> > > >>>> >> >>> >> > - All three hosts in global maintenance mode > >>>> >> >>> >> > - Ran: systemctl stop ovirt-ha-broker; systemctl stop > >>>> >> >>> >> > ovirt-ha-broker -- > >>>> >> >>> >> > on > >>>> >> >>> >> > all three hosts > >>>> >> >>> >> > - Moved ALL files in > >>>> >> >>> >> > > >>>> >> >>> >> > > >>>> >> >>> >> > > >>>> >> >>> >> > > >>>> >> >>> >> > /var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/ > >>>> >> >>> >> > to > >>>> >> >>> >> > > >>>> >> >>> >> > > >>>> >> >>> >> > > >>>> >> >>> >> > > >>>> >> >>> >> > /var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/backup > >>>> >> >>> >> > - Ran: systemctl start ovirt-ha-broker; systemctl start > >>>> >> >>> >> > ovirt-ha-broker > >>>> >> >>> >> > -- > >>>> >> >>> >> > on all three hosts > >>>> >> >>> >> > > >>>> >> >>> >> > - attempt start of engine vm from HOST0 (cultivar0): > >>>> >> >>> >> > hosted-engine > >>>> >> >>> >> > --vm-start > >>>> >> >>> >> > > >>>> >> >>> >> > Lots of errors in the logs still, it appears to be having > >>>> >> >>> >> > problems > >>>> >> >>> >> > with > >>>> >> >>> >> > that > >>>> >> >>> >> > directory still: > >>>> >> >>> >> > > >>>> >> >>> >> > Jan 12 10:40:13 cultivar0 journal: ovirt-ha-broker > >>>> >> >>> >> > ovirt_hosted_engine_ha.broker. > storage_broker.StorageBroker > >>>> >> >>> >> > ERROR > >>>> >> >>> >> > Failed > >>>> >> >>> >> > to > >>>> >> >>> >> > write metadata for host 1 to > >>>> >> >>> >> > > >>>> >> >>> >> > > >>>> >> >>> >> > > >>>> >> >>> >> > > >>>> >> >>> >> > /var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/8582bdfc-ef54-47af-9f1e- > f5b7ec1f1cf8#012Traceback > >>>> >> >>> >> > (most recent call last):#012 File > >>>> >> >>> >> > > >>>> >> >>> >> > > >>>> >> >>> >> > > >>>> >> >>> >> > > >>>> >> >>> >> > "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ > ha/broker/storage_broker.py", > >>>> >> >>> >> > line 202, in put_stats#012 f = os.open(path, > direct_flag > >>>> >> >>> >> > | > >>>> >> >>> >> > os.O_WRONLY | > >>>> >> >>> >> > os.O_SYNC)#012OSError: [Errno 2] No such file or > directory: > >>>> >> >>> >> > > >>>> >> >>> >> > > >>>> >> >>> >> > > >>>> >> >>> >> > > >>>> >> >>> >> > '/var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/8582bdfc-ef54-47af-9f1e- > f5b7ec1f1cf8' > >>>> >> >>> >> > > >>>> >> >>> >> > There are no new files or symlinks in > >>>> >> >>> >> > > >>>> >> >>> >> > > >>>> >> >>> >> > > >>>> >> >>> >> > > >>>> >> >>> >> > /var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/ > >>>> >> >>> >> > > >>>> >> >>> >> > - Jayme > >>>> >> >>> >> > > >>>> >> >>> >> > > >>>> >> >>> >> > > >>>> >> >>> >> > On Fri, Jan 12, 2018 at 10:23 AM, Martin Sivak > >>>> >> >>> >> > <msivak(a)redhat.com> > >>>> >> >>> >> > wrote: > >>>> >> >>> >> >> > >>>> >> >>> >> >> > Can you please stop all hosted engine tooling ( > >>>> >> >>> >> >> > >>>> >> >>> >> >> On all hosts I should have added. > >>>> >> >>> >> >> > >>>> >> >>> >> >> Martin > >>>> >> >>> >> >> > >>>> >> >>> >> >> On Fri, Jan 12, 2018 at 3:22 PM, Martin Sivak > >>>> >> >>> >> >> <msivak(a)redhat.com> > >>>> >> >>> >> >> wrote: > >>>> >> >>> >> >> >> RequestError: failed to read metadata: [Errno 2] No > such > >>>> >> >>> >> >> >> file > >>>> >> >>> >> >> >> or > >>>> >> >>> >> >> >> directory: > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> '/var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/8582bdfc-ef54-47af-9f1e- > f5b7ec1f1cf8' > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> ls -al > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> /var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/8582bdfc-ef54-47af-9f1e- > f5b7ec1f1cf8 > >>>> >> >>> >> >> >> -rw-rw----. 1 vdsm kvm 1028096 Jan 12 09:59 > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> /var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/8582bdfc-ef54-47af-9f1e- > f5b7ec1f1cf8 > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> Is this due to the symlink problem you guys are > >>>> >> >>> >> >> >> referring to > >>>> >> >>> >> >> >> that > >>>> >> >>> >> >> >> was > >>>> >> >>> >> >> >> addressed in RC1 or something else? > >>>> >> >>> >> >> > > >>>> >> >>> >> >> > No, this file is the symlink. It should point to > >>>> >> >>> >> >> > somewhere > >>>> >> >>> >> >> > inside > >>>> >> >>> >> >> > /rhev/. I see it is a 1G file in your case. That is > >>>> >> >>> >> >> > really > >>>> >> >>> >> >> > interesting. > >>>> >> >>> >> >> > > >>>> >> >>> >> >> > Can you please stop all hosted engine tooling > >>>> >> >>> >> >> > (ovirt-ha-agent, > >>>> >> >>> >> >> > ovirt-ha-broker), move the file (metadata file is not > >>>> >> >>> >> >> > important > >>>> >> >>> >> >> > when > >>>> >> >>> >> >> > services are stopped, but better safe than sorry) and > >>>> >> >>> >> >> > restart > >>>> >> >>> >> >> > all > >>>> >> >>> >> >> > services again? > >>>> >> >>> >> >> > > >>>> >> >>> >> >> >> Could there possibly be a permissions > >>>> >> >>> >> >> >> problem somewhere? > >>>> >> >>> >> >> > > >>>> >> >>> >> >> > Maybe, but the file itself looks out of the ordinary. I > >>>> >> >>> >> >> > wonder > >>>> >> >>> >> >> > how it > >>>> >> >>> >> >> > got there. > >>>> >> >>> >> >> > > >>>> >> >>> >> >> > Best regards > >>>> >> >>> >> >> > > >>>> >> >>> >> >> > Martin Sivak > >>>> >> >>> >> >> > > >>>> >> >>> >> >> > On Fri, Jan 12, 2018 at 3:09 PM, Jayme < > jaymef(a)gmail.com> > >>>> >> >>> >> >> > wrote: > >>>> >> >>> >> >> >> Thanks for the help thus far. Storage could be > related > >>>> >> >>> >> >> >> but > >>>> >> >>> >> >> >> all > >>>> >> >>> >> >> >> other > >>>> >> >>> >> >> >> VMs on > >>>> >> >>> >> >> >> same storage are running ok. The storage is mounted > via > >>>> >> >>> >> >> >> NFS > >>>> >> >>> >> >> >> from > >>>> >> >>> >> >> >> within one > >>>> >> >>> >> >> >> of the three hosts, I realize this is not ideal. This > >>>> >> >>> >> >> >> was > >>>> >> >>> >> >> >> setup > >>>> >> >>> >> >> >> by > >>>> >> >>> >> >> >> a > >>>> >> >>> >> >> >> previous admin more as a proof of concept and VMs were > >>>> >> >>> >> >> >> put on > >>>> >> >>> >> >> >> there > >>>> >> >>> >> >> >> that > >>>> >> >>> >> >> >> should not have been placed in a proof of concept > >>>> >> >>> >> >> >> environment.. > >>>> >> >>> >> >> >> it > >>>> >> >>> >> >> >> was > >>>> >> >>> >> >> >> intended to be rebuilt with proper storage down the > >>>> >> >>> >> >> >> road. > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> So the storage is on HOST0 and the other hosts mount > NFS > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> cultivar0.grove.silverorange.com:/exports/data > >>>> >> >>> >> >> >> 4861742080 > >>>> >> >>> >> >> >> 1039352832 3822389248 22% > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> /rhev/data-center/mnt/cultivar0.grove.silverorange. > com:_exports_data > >>>> >> >>> >> >> >> cultivar0.grove.silverorange.com:/exports/iso > >>>> >> >>> >> >> >> 4861742080 > >>>> >> >>> >> >> >> 1039352832 3822389248 22% > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> /rhev/data-center/mnt/cultivar0.grove.silverorange. > com:_exports_iso > >>>> >> >>> >> >> >> cultivar0.grove.silverorange. > com:/exports/import_export > >>>> >> >>> >> >> >> 4861742080 > >>>> >> >>> >> >> >> 1039352832 3822389248 22% > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> /rhev/data-center/mnt/cultivar0.grove.silverorange. > com:_exports_import__export > >>>> >> >>> >> >> >> cultivar0.grove.silverorange. > com:/exports/hosted_engine > >>>> >> >>> >> >> >> 4861742080 > >>>> >> >>> >> >> >> 1039352832 3822389248 22% > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> /rhev/data-center/mnt/cultivar0.grove.silverorange. > com:_exports_hosted__engine > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> Like I said, the VM data storage itself seems to be > >>>> >> >>> >> >> >> working > >>>> >> >>> >> >> >> ok, > >>>> >> >>> >> >> >> as > >>>> >> >>> >> >> >> all > >>>> >> >>> >> >> >> other > >>>> >> >>> >> >> >> VMs appear to be running. > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> I'm curious why the broker log says this file is not > >>>> >> >>> >> >> >> found > >>>> >> >>> >> >> >> when > >>>> >> >>> >> >> >> it > >>>> >> >>> >> >> >> is > >>>> >> >>> >> >> >> correct and I can see the file at that path: > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> RequestError: failed to read metadata: [Errno 2] No > such > >>>> >> >>> >> >> >> file > >>>> >> >>> >> >> >> or > >>>> >> >>> >> >> >> directory: > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> '/var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/8582bdfc-ef54-47af-9f1e- > f5b7ec1f1cf8' > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> ls -al > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> /var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/8582bdfc-ef54-47af-9f1e- > f5b7ec1f1cf8 > >>>> >> >>> >> >> >> -rw-rw----. 1 vdsm kvm 1028096 Jan 12 09:59 > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> /var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/8582bdfc-ef54-47af-9f1e- > f5b7ec1f1cf8 > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> Is this due to the symlink problem you guys are > >>>> >> >>> >> >> >> referring to > >>>> >> >>> >> >> >> that > >>>> >> >>> >> >> >> was > >>>> >> >>> >> >> >> addressed in RC1 or something else? Could there > >>>> >> >>> >> >> >> possibly be > >>>> >> >>> >> >> >> a > >>>> >> >>> >> >> >> permissions > >>>> >> >>> >> >> >> problem somewhere? > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> Assuming that all three hosts have 4.2 rpms installed > >>>> >> >>> >> >> >> and the > >>>> >> >>> >> >> >> host > >>>> >> >>> >> >> >> engine > >>>> >> >>> >> >> >> will not start is it safe for me to update hosts to > 4.2 > >>>> >> >>> >> >> >> RC1 > >>>> >> >>> >> >> >> rpms? > >>>> >> >>> >> >> >> Or > >>>> >> >>> >> >> >> perhaps install that repo and *only* update the ovirt > HA > >>>> >> >>> >> >> >> packages? > >>>> >> >>> >> >> >> Assuming that I cannot yet apply the same updates to > the > >>>> >> >>> >> >> >> inaccessible > >>>> >> >>> >> >> >> hosted > >>>> >> >>> >> >> >> engine VM. > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> I should also mention one more thing. I originally > >>>> >> >>> >> >> >> upgraded > >>>> >> >>> >> >> >> the > >>>> >> >>> >> >> >> engine > >>>> >> >>> >> >> >> VM > >>>> >> >>> >> >> >> first using new RPMS then engine-setup. It failed due > >>>> >> >>> >> >> >> to not > >>>> >> >>> >> >> >> being > >>>> >> >>> >> >> >> in > >>>> >> >>> >> >> >> global maintenance, so I set global maintenance and > ran > >>>> >> >>> >> >> >> it > >>>> >> >>> >> >> >> again, > >>>> >> >>> >> >> >> which > >>>> >> >>> >> >> >> appeared to complete as intended but never came back > up > >>>> >> >>> >> >> >> after. > >>>> >> >>> >> >> >> Just > >>>> >> >>> >> >> >> in > >>>> >> >>> >> >> >> case > >>>> >> >>> >> >> >> this might have anything at all to do with what could > >>>> >> >>> >> >> >> have > >>>> >> >>> >> >> >> happened. > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> Thanks very much again, I very much appreciate the > help! > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> - Jayme > >>>> >> >>> >> >> >> > >>>> >> >>> >> >> >> On Fri, Jan 12, 2018 at 8:44 AM, Simone Tiraboschi > >>>> >> >>> >> >> >> <stirabos(a)redhat.com> > >>>> >> >>> >> >> >> wrote: > >>>> >> >>> >> >> >>> > >>>> >> >>> >> >> >>> > >>>> >> >>> >> >> >>> > >>>> >> >>> >> >> >>> On Fri, Jan 12, 2018 at 11:11 AM, Martin Sivak > >>>> >> >>> >> >> >>> <msivak(a)redhat.com> > >>>> >> >>> >> >> >>> wrote: > >>>> >> >>> >> >> >>>> > >>>> >> >>> >> >> >>>> Hi, > >>>> >> >>> >> >> >>>> > >>>> >> >>> >> >> >>>> the hosted engine agent issue might be fixed by > >>>> >> >>> >> >> >>>> restarting > >>>> >> >>> >> >> >>>> ovirt-ha-broker or updating to newest > >>>> >> >>> >> >> >>>> ovirt-hosted-engine-ha > >>>> >> >>> >> >> >>>> and > >>>> >> >>> >> >> >>>> -setup. We improved handling of the missing symlink. > >>>> >> >>> >> >> >>> > >>>> >> >>> >> >> >>> > >>>> >> >>> >> >> >>> Available just in oVirt 4.2.1 RC1 > >>>> >> >>> >> >> >>> > >>>> >> >>> >> >> >>>> > >>>> >> >>> >> >> >>>> > >>>> >> >>> >> >> >>>> All the other issues seem to point to some storage > >>>> >> >>> >> >> >>>> problem > >>>> >> >>> >> >> >>>> I > >>>> >> >>> >> >> >>>> am > >>>> >> >>> >> >> >>>> afraid. > >>>> >> >>> >> >> >>>> > >>>> >> >>> >> >> >>>> You said you started the VM, do you see it in virsh > -r > >>>> >> >>> >> >> >>>> list? > >>>> >> >>> >> >> >>>> > >>>> >> >>> >> >> >>>> Best regards > >>>> >> >>> >> >> >>>> > >>>> >> >>> >> >> >>>> Martin Sivak > >>>> >> >>> >> >> >>>> > >>>> >> >>> >> >> >>>> On Thu, Jan 11, 2018 at 10:00 PM, Jayme > >>>> >> >>> >> >> >>>> <jaymef(a)gmail.com> > >>>> >> >>> >> >> >>>> wrote: > >>>> >> >>> >> >> >>>> > Please help, I'm really not sure what else to try > at > >>>> >> >>> >> >> >>>> > this > >>>> >> >>> >> >> >>>> > point. > >>>> >> >>> >> >> >>>> > Thank > >>>> >> >>> >> >> >>>> > you > >>>> >> >>> >> >> >>>> > for reading! > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > I'm still working on trying to get my hosted > engine > >>>> >> >>> >> >> >>>> > running > >>>> >> >>> >> >> >>>> > after a > >>>> >> >>> >> >> >>>> > botched > >>>> >> >>> >> >> >>>> > upgrade to 4.2. Storage is NFS mounted from > within > >>>> >> >>> >> >> >>>> > one > >>>> >> >>> >> >> >>>> > of > >>>> >> >>> >> >> >>>> > the > >>>> >> >>> >> >> >>>> > hosts. > >>>> >> >>> >> >> >>>> > Right > >>>> >> >>> >> >> >>>> > now I have 3 centos7 hosts that are fully updated > >>>> >> >>> >> >> >>>> > with > >>>> >> >>> >> >> >>>> > yum > >>>> >> >>> >> >> >>>> > packages > >>>> >> >>> >> >> >>>> > from > >>>> >> >>> >> >> >>>> > ovirt 4.2, the engine was fully updated with yum > >>>> >> >>> >> >> >>>> > packages > >>>> >> >>> >> >> >>>> > and > >>>> >> >>> >> >> >>>> > failed to > >>>> >> >>> >> >> >>>> > come > >>>> >> >>> >> >> >>>> > up after reboot. As of right now, everything > should > >>>> >> >>> >> >> >>>> > have > >>>> >> >>> >> >> >>>> > full > >>>> >> >>> >> >> >>>> > yum > >>>> >> >>> >> >> >>>> > updates > >>>> >> >>> >> >> >>>> > and all having 4.2 rpms. I have global > maintenance > >>>> >> >>> >> >> >>>> > mode > >>>> >> >>> >> >> >>>> > on > >>>> >> >>> >> >> >>>> > right > >>>> >> >>> >> >> >>>> > now > >>>> >> >>> >> >> >>>> > and > >>>> >> >>> >> >> >>>> > started hosted-engine on one of the three host and > >>>> >> >>> >> >> >>>> > the > >>>> >> >>> >> >> >>>> > status is > >>>> >> >>> >> >> >>>> > currently: > >>>> >> >>> >> >> >>>> > Engine status : {"reason": "failed liveliness > >>>> >> >>> >> >> >>>> > check”; > >>>> >> >>> >> >> >>>> > "health": > >>>> >> >>> >> >> >>>> > "bad", > >>>> >> >>> >> >> >>>> > "vm": > >>>> >> >>> >> >> >>>> > "up", "detail": "Up"} > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > this is what I get when trying to enter hosted-vm > >>>> >> >>> >> >> >>>> > --console > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > The engine VM is running on this host > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > error: failed to get domain 'HostedEngine' > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > error: Domain not found: no domain with matching > >>>> >> >>> >> >> >>>> > name > >>>> >> >>> >> >> >>>> > 'HostedEngine' > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > Here are logs from various sources when I start > the > >>>> >> >>> >> >> >>>> > VM on > >>>> >> >>> >> >> >>>> > HOST3: > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > hosted-engine --vm-start > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > Command VM.getStats with args {'vmID': > >>>> >> >>> >> >> >>>> > '4013c829-c9d7-4b72-90d5-6fe58137504c'} failed: > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > (code=1, message=Virtual machine does not exist: > >>>> >> >>> >> >> >>>> > {'vmId': > >>>> >> >>> >> >> >>>> > u'4013c829-c9d7-4b72-90d5-6fe58137504c'}) > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > Jan 11 16:55:57 cultivar3 systemd-machined: New > >>>> >> >>> >> >> >>>> > machine > >>>> >> >>> >> >> >>>> > qemu-110-Cultivar. > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > Jan 11 16:55:57 cultivar3 systemd: Started Virtual > >>>> >> >>> >> >> >>>> > Machine > >>>> >> >>> >> >> >>>> > qemu-110-Cultivar. > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > Jan 11 16:55:57 cultivar3 systemd: Starting > Virtual > >>>> >> >>> >> >> >>>> > Machine > >>>> >> >>> >> >> >>>> > qemu-110-Cultivar. > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > Jan 11 16:55:57 cultivar3 kvm: 3 guests now active > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > ==> /var/log/vdsm/vdsm.log <== > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > File > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > "/usr/lib/python2.7/site- > packages/vdsm/common/api.py", > >>>> >> >>> >> >> >>>> > line > >>>> >> >>> >> >> >>>> > 48, > >>>> >> >>> >> >> >>>> > in > >>>> >> >>> >> >> >>>> > method > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > ret = func(*args, **kwargs) > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > File > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > "/usr/lib/python2.7/site- > packages/vdsm/storage/hsm.py", > >>>> >> >>> >> >> >>>> > line > >>>> >> >>> >> >> >>>> > 2718, in > >>>> >> >>> >> >> >>>> > getStorageDomainInfo > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > dom = self.validateSdUUID(sdUUID) > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > File > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > "/usr/lib/python2.7/site- > packages/vdsm/storage/hsm.py", > >>>> >> >>> >> >> >>>> > line > >>>> >> >>> >> >> >>>> > 304, in > >>>> >> >>> >> >> >>>> > validateSdUUID > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > sdDom.validate() > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > File > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > "/usr/lib/python2.7/site- > packages/vdsm/storage/fileSD.py", > >>>> >> >>> >> >> >>>> > line > >>>> >> >>> >> >> >>>> > 515, > >>>> >> >>> >> >> >>>> > in validate > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > raise se.StorageDomainAccessError( > self.sdUUID) > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > StorageDomainAccessError: Domain is either > partially > >>>> >> >>> >> >> >>>> > accessible > >>>> >> >>> >> >> >>>> > or > >>>> >> >>> >> >> >>>> > entirely > >>>> >> >>> >> >> >>>> > inaccessible: > >>>> >> >>> >> >> >>>> > (u'248f46f0-d793-4581-9810-c9d965e2f286',) > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > jsonrpc/2::ERROR::2018-01-11 > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > 16:55:16,144::dispatcher::82:: > storage.Dispatcher::(wrapper) > >>>> >> >>> >> >> >>>> > FINISH > >>>> >> >>> >> >> >>>> > getStorageDomainInfo error=Domain is either > >>>> >> >>> >> >> >>>> > partially > >>>> >> >>> >> >> >>>> > accessible > >>>> >> >>> >> >> >>>> > or > >>>> >> >>> >> >> >>>> > entirely > >>>> >> >>> >> >> >>>> > inaccessible: > >>>> >> >>> >> >> >>>> > (u'248f46f0-d793-4581-9810-c9d965e2f286',) > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > ==> /var/log/libvirt/qemu/Cultivar.log <== > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > LC_ALL=C > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > PATH=/usr/local/sbin:/usr/ > local/bin:/usr/sbin:/usr/bin > >>>> >> >>> >> >> >>>> > QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm -name > >>>> >> >>> >> >> >>>> > guest=Cultivar,debug-threads=on -S -object > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > secret,id=masterKey0,format= > raw,file=/var/lib/libvirt/qemu/domain-108-Cultivar/master-key.aes > >>>> >> >>> >> >> >>>> > -machine > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > pc-i440fx-rhel7.3.0,accel=kvm, > usb=off,dump-guest-core=off > >>>> >> >>> >> >> >>>> > -cpu > >>>> >> >>> >> >> >>>> > Conroe -m 8192 -realtime mlock=off -smp > >>>> >> >>> >> >> >>>> > 2,maxcpus=16,sockets=16,cores=1,threads=1 -uuid > >>>> >> >>> >> >> >>>> > 4013c829-c9d7-4b72-90d5-6fe58137504c -smbios > >>>> >> >>> >> >> >>>> > 'type=1,manufacturer=oVirt,product=oVirt > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > Node,version=7-4.1708.el7. > centos,serial=44454C4C-4300-1034-8035-CAC04F424331,uuid= > 4013c829-c9d7-4b72-90d5-6fe58137504c' > >>>> >> >>> >> >> >>>> > -no-user-config -nodefaults -chardev > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > socket,id=charmonitor,path=/ > var/lib/libvirt/qemu/domain-108-Cultivar/monitor.sock,server,nowait > >>>> >> >>> >> >> >>>> > -mon chardev=charmonitor,id=monitor,mode=control > >>>> >> >>> >> >> >>>> > -rtc > >>>> >> >>> >> >> >>>> > base=2018-01-11T20:33:19,driftfix=slew -global > >>>> >> >>> >> >> >>>> > kvm-pit.lost_tick_policy=delay -no-hpet -no-reboot > >>>> >> >>> >> >> >>>> > -boot > >>>> >> >>> >> >> >>>> > strict=on > >>>> >> >>> >> >> >>>> > -device > >>>> >> >>> >> >> >>>> > piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 > -device > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > virtio-serial-pci,id=virtio- > serial0,bus=pci.0,addr=0x4 > >>>> >> >>> >> >> >>>> > -drive > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > file=/var/run/vdsm/storage/ > 248f46f0-d793-4581-9810-c9d965e2f286/c2dde892-f978-4dfc-a421-c8e04cf387f9/ > 23aa0a66-fa6c-4967-a1e5-fbe47c0cd705,format=raw,if= > none,id=drive-virtio-disk0,serial=c2dde892-f978-4dfc- > a421-c8e04cf387f9,cache=none,werror=stop,rerror=stop,aio=threads > >>>> >> >>> >> >> >>>> > -device > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > virtio-blk-pci,scsi=off,bus= > pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 > >>>> >> >>> >> >> >>>> > -drive if=none,id=drive-ide0-1-0,readonly=on > -device > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > ide-cd,bus=ide.1,unit=0,drive= > drive-ide0-1-0,id=ide0-1-0 > >>>> >> >>> >> >> >>>> > -netdev > >>>> >> >>> >> >> >>>> > tap,fd=30,id=hostnet0,vhost=on,vhostfd=32 -device > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > virtio-net-pci,netdev= > hostnet0,id=net0,mac=00:16:3e:7f:d6:83,bus=pci.0,addr=0x3 > >>>> >> >>> >> >> >>>> > -chardev > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > socket,id=charchannel0,path=/ > var/lib/libvirt/qemu/channels/4013c829-c9d7-4b72-90d5- > 6fe58137504c.com.redhat.rhevm.vdsm,server,nowait > >>>> >> >>> >> >> >>>> > -device > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > virtserialport,bus=virtio-serial0.0,nr=1,chardev= > charchannel0,id=channel0,name=com.redhat.rhevm.vdsm > >>>> >> >>> >> >> >>>> > -chardev > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > socket,id=charchannel1,path=/ > var/lib/libvirt/qemu/channels/4013c829-c9d7-4b72-90d5- > 6fe58137504c.org.qemu.guest_agent.0,server,nowait > >>>> >> >>> >> >> >>>> > -device > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > virtserialport,bus=virtio-serial0.0,nr=2,chardev= > charchannel1,id=channel1,name=org.qemu.guest_agent.0 > >>>> >> >>> >> >> >>>> > -chardev spicevmc,id=charchannel2,name=vdagent > >>>> >> >>> >> >> >>>> > -device > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > virtserialport,bus=virtio-serial0.0,nr=3,chardev= > charchannel2,id=channel2,name=com.redhat.spice.0 > >>>> >> >>> >> >> >>>> > -chardev > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > socket,id=charchannel3,path=/ > var/lib/libvirt/qemu/channels/4013c829-c9d7-4b72-90d5- > 6fe58137504c.org.ovirt.hosted-engine-setup.0,server,nowait > >>>> >> >>> >> >> >>>> > -device > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > virtserialport,bus=virtio-serial0.0,nr=4,chardev= > charchannel3,id=channel3,name=org.ovirt.hosted-engine-setup.0 > >>>> >> >>> >> >> >>>> > -chardev pty,id=charconsole0 -device > >>>> >> >>> >> >> >>>> > virtconsole,chardev=charconsole0,id=console0 > -spice > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > tls-port=5900,addr=0,x509-dir= > /etc/pki/vdsm/libvirt-spice,tls-channel=default,seamless-migration=on > >>>> >> >>> >> >> >>>> > -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 > >>>> >> >>> >> >> >>>> > -object > >>>> >> >>> >> >> >>>> > rng-random,id=objrng0,filename=/dev/urandom > -device > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > virtio-rng-pci,rng=objrng0,id= > rng0,bus=pci.0,addr=0x5 > >>>> >> >>> >> >> >>>> > -msg > >>>> >> >>> >> >> >>>> > timestamp=on > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > 2018-01-11T20:33:19.699999Z qemu-kvm: -chardev > >>>> >> >>> >> >> >>>> > pty,id=charconsole0: > >>>> >> >>> >> >> >>>> > char > >>>> >> >>> >> >> >>>> > device redirected to /dev/pts/2 (label > charconsole0) > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > 2018-01-11 20:38:11.640+0000: shutting down, > >>>> >> >>> >> >> >>>> > reason=shutdown > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > 2018-01-11 20:39:02.122+0000: starting up libvirt > >>>> >> >>> >> >> >>>> > version: > >>>> >> >>> >> >> >>>> > 3.2.0, > >>>> >> >>> >> >> >>>> > package: > >>>> >> >>> >> >> >>>> > 14.el7_4.7 (CentOS BuildSystem > >>>> >> >>> >> >> >>>> > <http://bugs.centos.org>, > >>>> >> >>> >> >> >>>> > 2018-01-04-19:31:34, c1bm.rdu2.centos.org) qemu > >>>> >> >>> >> >> >>>> > version: > >>>> >> >>> >> >> >>>> > 2.9.0(qemu-kvm-ev-2.9.0-16.el7_4.13.1), hostname: > >>>> >> >>> >> >> >>>> > cultivar3 > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > LC_ALL=C > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > PATH=/usr/local/sbin:/usr/ > local/bin:/usr/sbin:/usr/bin > >>>> >> >>> >> >> >>>> > QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm -name > >>>> >> >>> >> >> >>>> > guest=Cultivar,debug-threads=on -S -object > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > secret,id=masterKey0,format= > raw,file=/var/lib/libvirt/qemu/domain-109-Cultivar/master-key.aes > >>>> >> >>> >> >> >>>> > -machine > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > pc-i440fx-rhel7.3.0,accel=kvm, > usb=off,dump-guest-core=off > >>>> >> >>> >> >> >>>> > -cpu > >>>> >> >>> >> >> >>>> > Conroe -m 8192 -realtime mlock=off -smp > >>>> >> >>> >> >> >>>> > 2,maxcpus=16,sockets=16,cores=1,threads=1 -uuid > >>>> >> >>> >> >> >>>> > 4013c829-c9d7-4b72-90d5-6fe58137504c -smbios > >>>> >> >>> >> >> >>>> > 'type=1,manufacturer=oVirt,product=oVirt > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > Node,version=7-4.1708.el7. > centos,serial=44454C4C-4300-1034-8035-CAC04F424331,uuid= > 4013c829-c9d7-4b72-90d5-6fe58137504c' > >>>> >> >>> >> >> >>>> > -no-user-config -nodefaults -chardev > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > socket,id=charmonitor,path=/ > var/lib/libvirt/qemu/domain-109-Cultivar/monitor.sock,server,nowait > >>>> >> >>> >> >> >>>> > -mon chardev=charmonitor,id=monitor,mode=control > >>>> >> >>> >> >> >>>> > -rtc > >>>> >> >>> >> >> >>>> > base=2018-01-11T20:39:02,driftfix=slew -global > >>>> >> >>> >> >> >>>> > kvm-pit.lost_tick_policy=delay -no-hpet -no-reboot > >>>> >> >>> >> >> >>>> > -boot > >>>> >> >>> >> >> >>>> > strict=on > >>>> >> >>> >> >> >>>> > -device > >>>> >> >>> >> >> >>>> > piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 > -device > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > virtio-serial-pci,id=virtio- > serial0,bus=pci.0,addr=0x4 > >>>> >> >>> >> >> >>>> > -drive > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > file=/var/run/vdsm/storage/ > 248f46f0-d793-4581-9810-c9d965e2f286/c2dde892-f978-4dfc-a421-c8e04cf387f9/ > 23aa0a66-fa6c-4967-a1e5-fbe47c0cd705,format=raw,if= > none,id=drive-virtio-disk0,serial=c2dde892-f978-4dfc- > a421-c8e04cf387f9,cache=none,werror=stop,rerror=stop,aio=threads > >>>> >> >>> >> >> >>>> > -device > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > virtio-blk-pci,scsi=off,bus= > pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 > >>>> >> >>> >> >> >>>> > -drive if=none,id=drive-ide0-1-0,readonly=on > -device > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > ide-cd,bus=ide.1,unit=0,drive= > drive-ide0-1-0,id=ide0-1-0 > >>>> >> >>> >> >> >>>> > -netdev > >>>> >> >>> >> >> >>>> > tap,fd=30,id=hostnet0,vhost=on,vhostfd=32 -device > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > virtio-net-pci,netdev= > hostnet0,id=net0,mac=00:16:3e:7f:d6:83,bus=pci.0,addr=0x3 > >>>> >> >>> >> >> >>>> > -chardev > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > socket,id=charchannel0,path=/ > var/lib/libvirt/qemu/channels/4013c829-c9d7-4b72-90d5- > 6fe58137504c.com.redhat.rhevm.vdsm,server,nowait > >>>> >> >>> >> >> >>>> > -device > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > virtserialport,bus=virtio-serial0.0,nr=1,chardev= > charchannel0,id=channel0,name=com.redhat.rhevm.vdsm > >>>> >> >>> >> >> >>>> > -chardev > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > socket,id=charchannel1,path=/ > var/lib/libvirt/qemu/channels/4013c829-c9d7-4b72-90d5- > 6fe58137504c.org.qemu.guest_agent.0,server,nowait > >>>> >> >>> >> >> >>>> > -device > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > virtserialport,bus=virtio-serial0.0,nr=2,chardev= > charchannel1,id=channel1,name=org.qemu.guest_agent.0 > >>>> >> >>> >> >> >>>> > -chardev spicevmc,id=charchannel2,name=vdagent > >>>> >> >>> >> >> >>>> > -device > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > virtserialport,bus=virtio-serial0.0,nr=3,chardev= > charchannel2,id=channel2,name=com.redhat.spice.0 > >>>> >> >>> >> >> >>>> > -chardev > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > socket,id=charchannel3,path=/ > var/lib/libvirt/qemu/channels/4013c829-c9d7-4b72-90d5- > 6fe58137504c.org.ovirt.hosted-engine-setup.0,server,nowait > >>>> >> >>> >> >> >>>> > -device > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > virtserialport,bus=virtio-serial0.0,nr=4,chardev= > charchannel3,id=channel3,name=org.ovirt.hosted-engine-setup.0 > >>>> >> >>> >> >> >>>> > -chardev pty,id=charconsole0 -device > >>>> >> >>> >> >> >>>> > virtconsole,chardev=charconsole0,id=console0 > -spice > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > tls-port=5900,addr=0,x509-dir= > /etc/pki/vdsm/libvirt-spice,tls-channel=default,seamless-migration=on > >>>> >> >>> >> >> >>>> > -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 > >>>> >> >>> >> >> >>>> > -object > >>>> >> >>> >> >> >>>> > rng-random,id=objrng0,filename=/dev/urandom > -device > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > virtio-rng-pci,rng=objrng0,id= > rng0,bus=pci.0,addr=0x5 > >>>> >> >>> >> >> >>>> > -msg > >>>> >> >>> >> >> >>>> > timestamp=on > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > 2018-01-11T20:39:02.380773Z qemu-kvm: -chardev > >>>> >> >>> >> >> >>>> > pty,id=charconsole0: > >>>> >> >>> >> >> >>>> > char > >>>> >> >>> >> >> >>>> > device redirected to /dev/pts/2 (label > charconsole0) > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > 2018-01-11 20:53:11.407+0000: shutting down, > >>>> >> >>> >> >> >>>> > reason=shutdown > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > 2018-01-11 20:55:57.210+0000: starting up libvirt > >>>> >> >>> >> >> >>>> > version: > >>>> >> >>> >> >> >>>> > 3.2.0, > >>>> >> >>> >> >> >>>> > package: > >>>> >> >>> >> >> >>>> > 14.el7_4.7 (CentOS BuildSystem > >>>> >> >>> >> >> >>>> > <http://bugs.centos.org>, > >>>> >> >>> >> >> >>>> > 2018-01-04-19:31:34, c1bm.rdu2.centos.org) qemu > >>>> >> >>> >> >> >>>> > version: > >>>> >> >>> >> >> >>>> > 2.9.0(qemu-kvm-ev-2.9.0-16.el7_4.13.1), hostname: > >>>> >> >>> >> >> >>>> > cultivar3.grove.silverorange.com > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > LC_ALL=C > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > PATH=/usr/local/sbin:/usr/ > local/bin:/usr/sbin:/usr/bin > >>>> >> >>> >> >> >>>> > QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm -name > >>>> >> >>> >> >> >>>> > guest=Cultivar,debug-threads=on -S -object > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > secret,id=masterKey0,format= > raw,file=/var/lib/libvirt/qemu/domain-110-Cultivar/master-key.aes > >>>> >> >>> >> >> >>>> > -machine > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > pc-i440fx-rhel7.3.0,accel=kvm, > usb=off,dump-guest-core=off > >>>> >> >>> >> >> >>>> > -cpu > >>>> >> >>> >> >> >>>> > Conroe -m 8192 -realtime mlock=off -smp > >>>> >> >>> >> >> >>>> > 2,maxcpus=16,sockets=16,cores=1,threads=1 -uuid > >>>> >> >>> >> >> >>>> > 4013c829-c9d7-4b72-90d5-6fe58137504c -smbios > >>>> >> >>> >> >> >>>> > 'type=1,manufacturer=oVirt,product=oVirt > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > Node,version=7-4.1708.el7. > centos,serial=44454C4C-4300-1034-8035-CAC04F424331,uuid= > 4013c829-c9d7-4b72-90d5-6fe58137504c' > >>>> >> >>> >> >> >>>> > -no-user-config -nodefaults -chardev > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > socket,id=charmonitor,path=/ > var/lib/libvirt/qemu/domain-110-Cultivar/monitor.sock,server,nowait > >>>> >> >>> >> >> >>>> > -mon chardev=charmonitor,id=monitor,mode=control > >>>> >> >>> >> >> >>>> > -rtc > >>>> >> >>> >> >> >>>> > base=2018-01-11T20:55:57,driftfix=slew -global > >>>> >> >>> >> >> >>>> > kvm-pit.lost_tick_policy=delay -no-hpet -no-reboot > >>>> >> >>> >> >> >>>> > -boot > >>>> >> >>> >> >> >>>> > strict=on > >>>> >> >>> >> >> >>>> > -device > >>>> >> >>> >> >> >>>> > piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 > -device > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > virtio-serial-pci,id=virtio- > serial0,bus=pci.0,addr=0x4 > >>>> >> >>> >> >> >>>> > -drive > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > file=/var/run/vdsm/storage/ > 248f46f0-d793-4581-9810-c9d965e2f286/c2dde892-f978-4dfc-a421-c8e04cf387f9/ > 23aa0a66-fa6c-4967-a1e5-fbe47c0cd705,format=raw,if= > none,id=drive-virtio-disk0,serial=c2dde892-f978-4dfc- > a421-c8e04cf387f9,cache=none,werror=stop,rerror=stop,aio=threads > >>>> >> >>> >> >> >>>> > -device > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > virtio-blk-pci,scsi=off,bus= > pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 > >>>> >> >>> >> >> >>>> > -drive if=none,id=drive-ide0-1-0,readonly=on > -device > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > ide-cd,bus=ide.1,unit=0,drive= > drive-ide0-1-0,id=ide0-1-0 > >>>> >> >>> >> >> >>>> > -netdev > >>>> >> >>> >> >> >>>> > tap,fd=30,id=hostnet0,vhost=on,vhostfd=32 -device > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > virtio-net-pci,netdev= > hostnet0,id=net0,mac=00:16:3e:7f:d6:83,bus=pci.0,addr=0x3 > >>>> >> >>> >> >> >>>> > -chardev > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > socket,id=charchannel0,path=/ > var/lib/libvirt/qemu/channels/4013c829-c9d7-4b72-90d5- > 6fe58137504c.com.redhat.rhevm.vdsm,server,nowait > >>>> >> >>> >> >> >>>> > -device > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > virtserialport,bus=virtio-serial0.0,nr=1,chardev= > charchannel0,id=channel0,name=com.redhat.rhevm.vdsm > >>>> >> >>> >> >> >>>> > -chardev > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > socket,id=charchannel1,path=/ > var/lib/libvirt/qemu/channels/4013c829-c9d7-4b72-90d5- > 6fe58137504c.org.qemu.guest_agent.0,server,nowait > >>>> >> >>> >> >> >>>> > -device > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > virtserialport,bus=virtio-serial0.0,nr=2,chardev= > charchannel1,id=channel1,name=org.qemu.guest_agent.0 > >>>> >> >>> >> >> >>>> > -chardev spicevmc,id=charchannel2,name=vdagent > >>>> >> >>> >> >> >>>> > -device > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > virtserialport,bus=virtio-serial0.0,nr=3,chardev= > charchannel2,id=channel2,name=com.redhat.spice.0 > >>>> >> >>> >> >> >>>> > -chardev > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > socket,id=charchannel3,path=/ > var/lib/libvirt/qemu/channels/4013c829-c9d7-4b72-90d5- > 6fe58137504c.org.ovirt.hosted-engine-setup.0,server,nowait > >>>> >> >>> >> >> >>>> > -device > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > virtserialport,bus=virtio-serial0.0,nr=4,chardev= > charchannel3,id=channel3,name=org.ovirt.hosted-engine-setup.0 > >>>> >> >>> >> >> >>>> > -chardev pty,id=charconsole0 -device > >>>> >> >>> >> >> >>>> > virtconsole,chardev=charconsole0,id=console0 > -spice > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > tls-port=5900,addr=0,x509-dir= > /etc/pki/vdsm/libvirt-spice,tls-channel=default,seamless-migration=on > >>>> >> >>> >> >> >>>> > -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 > >>>> >> >>> >> >> >>>> > -object > >>>> >> >>> >> >> >>>> > rng-random,id=objrng0,filename=/dev/urandom > -device > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > virtio-rng-pci,rng=objrng0,id= > rng0,bus=pci.0,addr=0x5 > >>>> >> >>> >> >> >>>> > -msg > >>>> >> >>> >> >> >>>> > timestamp=on > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > 2018-01-11T20:55:57.468037Z qemu-kvm: -chardev > >>>> >> >>> >> >> >>>> > pty,id=charconsole0: > >>>> >> >>> >> >> >>>> > char > >>>> >> >>> >> >> >>>> > device redirected to /dev/pts/2 (label > charconsole0) > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > ==> /var/log/ovirt-hosted-engine-ha/broker.log > <== > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > File > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > "/usr/lib/python2.7/site- > packages/ovirt_hosted_engine_ha/broker/storage_broker.py", > >>>> >> >>> >> >> >>>> > line 151, in get_raw_stats > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > f = os.open(path, direct_flag | os.O_RDONLY | > >>>> >> >>> >> >> >>>> > os.O_SYNC) > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > OSError: [Errno 2] No such file or directory: > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > '/var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/8582bdfc-ef54-47af-9f1e- > f5b7ec1f1cf8' > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > StatusStorageThread::ERROR::2018-01-11 > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > 16:55:15,761::status_broker:: > 92::ovirt_hosted_engine_ha.broker.status_broker.StatusBroker.Update::(run) > >>>> >> >>> >> >> >>>> > Failed to read state. > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > Traceback (most recent call last): > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > File > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > "/usr/lib/python2.7/site- > packages/ovirt_hosted_engine_ha/broker/status_broker.py", > >>>> >> >>> >> >> >>>> > line 88, in run > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > self._storage_broker.get_raw_stats() > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > File > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > "/usr/lib/python2.7/site- > packages/ovirt_hosted_engine_ha/broker/storage_broker.py", > >>>> >> >>> >> >> >>>> > line 162, in get_raw_stats > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > .format(str(e))) > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > RequestError: failed to read metadata: [Errno 2] > No > >>>> >> >>> >> >> >>>> > such > >>>> >> >>> >> >> >>>> > file or > >>>> >> >>> >> >> >>>> > directory: > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > '/var/run/vdsm/storage/248f46f0-d793-4581-9810- > c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/8582bdfc-ef54-47af-9f1e- > f5b7ec1f1cf8' > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > ==> /var/log/ovirt-hosted-engine-ha/agent.log <== > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > result = refresh_method() > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > File > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > "/usr/lib/python2.7/site- > packages/ovirt_hosted_engine_ha/env/config.py", > >>>> >> >>> >> >> >>>> > line 519, in refresh_vm_conf > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > content = > >>>> >> >>> >> >> >>>> > self._get_file_content_from_shared_storage(VM) > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > File > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > "/usr/lib/python2.7/site- > packages/ovirt_hosted_engine_ha/env/config.py", > >>>> >> >>> >> >> >>>> > line 484, in _get_file_content_from_shared_ > storage > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > config_volume_path = > >>>> >> >>> >> >> >>>> > self._get_config_volume_path() > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > File > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > "/usr/lib/python2.7/site- > packages/ovirt_hosted_engine_ha/env/config.py", > >>>> >> >>> >> >> >>>> > line 188, in _get_config_volume_path > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > conf_vol_uuid > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > File > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > "/usr/lib/python2.7/site- > packages/ovirt_hosted_engine_ha/lib/heconflib.py", > >>>> >> >>> >> >> >>>> > line 358, in get_volume_path > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > root=envconst.SD_RUN_DIR, > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > RuntimeError: Path to volume > >>>> >> >>> >> >> >>>> > 4838749f-216d-406b-b245-98d0343fcf7f > >>>> >> >>> >> >> >>>> > not > >>>> >> >>> >> >> >>>> > found > >>>> >> >>> >> >> >>>> > in /run/vdsm/storag > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > ==> /var/log/vdsm/vdsm.log <== > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > periodic/42::ERROR::2018-01-11 > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > 16:56:11,446::vmstats::260:: > virt.vmstats::(send_metrics) > >>>> >> >>> >> >> >>>> > VM > >>>> >> >>> >> >> >>>> > metrics > >>>> >> >>> >> >> >>>> > collection failed > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > Traceback (most recent call last): > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > File > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > "/usr/lib/python2.7/site- > packages/vdsm/virt/vmstats.py", > >>>> >> >>> >> >> >>>> > line > >>>> >> >>> >> >> >>>> > 197, in > >>>> >> >>> >> >> >>>> > send_metrics > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > data[prefix + '.cpu.usage'] = stat['cpuUsage'] > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > KeyError: 'cpuUsage' > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> > _______________________________________________ > >>>> >> >>> >> >> >>>> > Users mailing list > >>>> >> >>> >> >> >>>> > Users(a)ovirt.org > >>>> >> >>> >> >> >>>> > http://lists.ovirt.org/mailman/listinfo/users > >>>> >> >>> >> >> >>>> > > >>>> >> >>> >> >> >>>> _______________________________________________ > >>>> >> >>> >> >> >>>> Users mailing list > >>>> >> >>> >> >> >>>> Users(a)ovirt.org > >>>> >> >>> >> >> >>>> http://lists.ovirt.org/mailman/listinfo/users > >>>> >> >>> >> >> >>> > >>>> >> >>> >> >> >>> > >>>> >> >>> >> >> >> > >>>> >> >>> >> > > >>>> >> >>> >> > > >>>> >> >>> > > >>>> >> >>> > > >>>> >> >> > >>>> >> >> > >>>> >> > > >>>> > > >>>> > > >>> > >>> > >> > > >

2 1

oVirt NGN image customization troubles
by Giuseppe Ragusa 12 Jan '18

12 Jan '18

Hi all, I'm trying to modify the oVirt NGN image (to add RPMs, since imgbased rpmpersistence currently seems to have a bug: https://bugzilla.redhat.com/show_bug.cgi?id=1528468 ) but I'm unfortunately stuck at the very beginning: it seems that I'm unable to recreate even the standard 4.1 squashfs image. I'm following the instructions at https://gerrit.ovirt.org/gitweb?p=ovirt-node-ng.git;a=blob;f=README I'm working inside a CentOS7 fully-updated vm (hosted inside VMware, with nested virtualization enabled). I'm trying to work on the 4.1 branch, so I issued a: ./autogen.sh --with-ovirt-release-rpm-url=http://resources.ovirt.org/pub/yum-repo/ovirt-release41.rpm And after that I'm stuck in the "make squashfs" step: it never ends (keeps printing dots forever with no errors/warnings in log messages nor any apparent activity on the virtual disk image). Invoking it in debug mode and connecting to the VNC console shows the detailed Plymouth startup listing stuck (latest messages displayed: "Starting udev Wait for Complete Device Initialization..." and "Starting Device-Mapper Multipath Device Controller...") I wonder if it's actually supposed to be run only from a recent Fedora (the "dnf" reference seems a good indicator): if so, which version? I kindly ask for advice: has anyone succeeded in modifying/reproducing NGN squash images recently? If so, how? :-) Many thanks in advance, Giuseppe

3 2

mount_options for hosted_engine storage domain
by Artem Tambovskiy 12 Jan '18

12 Jan '18

Hi, I have deployed a small cluster with 2 ovirt hosts and GlusterFS cluster some time ago. And recently during software upgrade I noticed that I made some mistakes during the installation: if the host which was deployed first will be taken down for upgrade (powered off or rebooted) the engine becomes unavailable (even all VM's and hosted engine were migrated to second host in advance). I was thinking that this is due to missing mnt_options=backup-volfile--servers=host1.domain.com;host2.domain.com option for hosted engine storage domain. Is there any good way to fix this? I have tried edit /etc/ovirt-hosted-engine/hosted-engine.conf manually to add missing mnt_options but after while I noticed that those changes are gone. Any suggestions? Thanks in advance! Artem

2 2

unable to bring up hosted engine after botched 4.2 upgrade
by Jayme 12 Jan '18

12 Jan '18

Please help, I'm really not sure what else to try at this point. Thank you for reading! I'm still working on trying to get my hosted engine running after a botched upgrade to 4.2. Storage is NFS mounted from within one of the hosts. Right now I have 3 centos7 hosts that are fully updated with yum packages from ovirt 4.2, the engine was fully updated with yum packages and failed to come up after reboot. As of right now, everything should have full yum updates and all having 4.2 rpms. I have global maintenance mode on right now and started hosted-engine on one of the three host and the status is currently: Engine status : {"reason": "failed liveliness check”; "health": "bad", "vm": "up", "detail": "Up"} this is what I get when trying to enter hosted-vm --console The engine VM is running on this host error: failed to get domain 'HostedEngine' error: Domain not found: no domain with matching name 'HostedEngine' Here are logs from various sources when I start the VM on HOST3: hosted-engine --vm-start Command VM.getStats with args {'vmID': '4013c829-c9d7-4b72-90d5-6fe58137504c'} failed: (code=1, message=Virtual machine does not exist: {'vmId': u'4013c829-c9d7-4b72-90d5-6fe58137504c'}) Jan 11 16:55:57 cultivar3 systemd-machined: New machine qemu-110-Cultivar. Jan 11 16:55:57 cultivar3 systemd: Started Virtual Machine qemu-110-Cultivar. Jan 11 16:55:57 cultivar3 systemd: Starting Virtual Machine qemu-110-Cultivar. Jan 11 16:55:57 cultivar3 kvm: 3 guests now active ==> /var/log/vdsm/vdsm.log <== File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 2718, in getStorageDomainInfo dom = self.validateSdUUID(sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 304, in validateSdUUID sdDom.validate() File "/usr/lib/python2.7/site-packages/vdsm/storage/fileSD.py", line 515, in validate raise se.StorageDomainAccessError(self.sdUUID) StorageDomainAccessError: Domain is either partially accessible or entirely inaccessible: (u'248f46f0-d793-4581-9810-c9d965e2f286',) jsonrpc/2::ERROR::2018-01-11 16:55:16,144::dispatcher::82::storage.Dispatcher::(wrapper) FINISH getStorageDomainInfo error=Domain is either partially accessible or entirely inaccessible: (u'248f46f0-d793-4581-9810-c9d965e2f286',) ==> /var/log/libvirt/qemu/Cultivar.log <== LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm -name guest=Cultivar,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-108-Cultivar/master-key.aes -machine pc-i440fx-rhel7.3.0,accel=kvm,usb=off,dump-guest-core=off -cpu Conroe -m 8192 -realtime mlock=off -smp 2,maxcpus=16,sockets=16,cores=1,threads=1 -uuid 4013c829-c9d7-4b72-90d5-6fe58137504c -smbios 'type=1,manufacturer=oVirt,product=oVirt Node,version=7-4.1708.el7.centos,serial=44454C4C-4300-1034-8035-CAC04F424331,uuid=4013c829-c9d7-4b72-90d5-6fe58137504c' -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-108-Cultivar/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=2018-01-11T20:33:19,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-reboot -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x4 -drive file=/var/run/vdsm/storage/248f46f0-d793-4581-9810-c9d965e2f286/c2dde892-f978-4dfc-a421-c8e04cf387f9/23aa0a66-fa6c-4967-a1e5-fbe47c0cd705,format=raw,if=none,id=drive-virtio-disk0,serial=c2dde892-f978-4dfc-a421-c8e04cf387f9,cache=none,werror=stop,rerror=stop,aio=threads -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive if=none,id=drive-ide0-1-0,readonly=on -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=30,id=hostnet0,vhost=on,vhostfd=32 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:16:3e:7f:d6:83,bus=pci.0,addr=0x3 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channels/4013c829-c9d7-4b72-90d5-6fe58137504c.com.redhat.rhevm.vdsm,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.rhevm.vdsm -chardev socket,id=charchannel1,path=/var/lib/libvirt/qemu/channels/4013c829-c9d7-4b72-90d5-6fe58137504c.org.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel2,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=3,chardev=charchannel2,id=channel2,name=com.redhat.spice.0 -chardev socket,id=charchannel3,path=/var/lib/libvirt/qemu/channels/4013c829-c9d7-4b72-90d5-6fe58137504c.org.ovirt.hosted-engine-setup.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=4,chardev=charchannel3,id=channel3,name=org.ovirt.hosted-engine-setup.0 -chardev pty,id=charconsole0 -device virtconsole,chardev=charconsole0,id=console0 -spice tls-port=5900,addr=0,x509-dir=/etc/pki/vdsm/libvirt-spice,tls-channel=default,seamless-migration=on -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -object rng-random,id=objrng0,filename=/dev/urandom -device virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x5 -msg timestamp=on 2018-01-11T20:33:19.699999Z qemu-kvm: -chardev pty,id=charconsole0: char device redirected to /dev/pts/2 (label charconsole0) 2018-01-11 20:38:11.640+0000: shutting down, reason=shutdown 2018-01-11 20:39:02.122+0000: starting up libvirt version: 3.2.0, package: 14.el7_4.7 (CentOS BuildSystem <http://bugs.centos.org>, 2018-01-04-19:31:34, c1bm.rdu2.centos.org) qemu version: 2.9.0(qemu-kvm-ev-2.9.0-16.el7_4.13.1), hostname: cultivar3 LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm -name guest=Cultivar,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-109-Cultivar/master-key.aes -machine pc-i440fx-rhel7.3.0,accel=kvm,usb=off,dump-guest-core=off -cpu Conroe -m 8192 -realtime mlock=off -smp 2,maxcpus=16,sockets=16,cores=1,threads=1 -uuid 4013c829-c9d7-4b72-90d5-6fe58137504c -smbios 'type=1,manufacturer=oVirt,product=oVirt Node,version=7-4.1708.el7.centos,serial=44454C4C-4300-1034-8035-CAC04F424331,uuid=4013c829-c9d7-4b72-90d5-6fe58137504c' -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-109-Cultivar/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=2018-01-11T20:39:02,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-reboot -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x4 -drive file=/var/run/vdsm/storage/248f46f0-d793-4581-9810-c9d965e2f286/c2dde892-f978-4dfc-a421-c8e04cf387f9/23aa0a66-fa6c-4967-a1e5-fbe47c0cd705,format=raw,if=none,id=drive-virtio-disk0,serial=c2dde892-f978-4dfc-a421-c8e04cf387f9,cache=none,werror=stop,rerror=stop,aio=threads -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive if=none,id=drive-ide0-1-0,readonly=on -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=30,id=hostnet0,vhost=on,vhostfd=32 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:16:3e:7f:d6:83,bus=pci.0,addr=0x3 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channels/4013c829-c9d7-4b72-90d5-6fe58137504c.com.redhat.rhevm.vdsm,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.rhevm.vdsm -chardev socket,id=charchannel1,path=/var/lib/libvirt/qemu/channels/4013c829-c9d7-4b72-90d5-6fe58137504c.org.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel2,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=3,chardev=charchannel2,id=channel2,name=com.redhat.spice.0 -chardev socket,id=charchannel3,path=/var/lib/libvirt/qemu/channels/4013c829-c9d7-4b72-90d5-6fe58137504c.org.ovirt.hosted-engine-setup.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=4,chardev=charchannel3,id=channel3,name=org.ovirt.hosted-engine-setup.0 -chardev pty,id=charconsole0 -device virtconsole,chardev=charconsole0,id=console0 -spice tls-port=5900,addr=0,x509-dir=/etc/pki/vdsm/libvirt-spice,tls-channel=default,seamless-migration=on -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -object rng-random,id=objrng0,filename=/dev/urandom -device virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x5 -msg timestamp=on 2018-01-11T20:39:02.380773Z qemu-kvm: -chardev pty,id=charconsole0: char device redirected to /dev/pts/2 (label charconsole0) 2018-01-11 20:53:11.407+0000: shutting down, reason=shutdown 2018-01-11 20:55:57.210+0000: starting up libvirt version: 3.2.0, package: 14.el7_4.7 (CentOS BuildSystem <http://bugs.centos.org>, 2018-01-04-19:31:34, c1bm.rdu2.centos.org) qemu version: 2.9.0(qemu-kvm-ev-2.9.0-16.el7_4.13.1), hostname: cultivar3.grove.silverorange.com LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm -name guest=Cultivar,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-110-Cultivar/master-key.aes -machine pc-i440fx-rhel7.3.0,accel=kvm,usb=off,dump-guest-core=off -cpu Conroe -m 8192 -realtime mlock=off -smp 2,maxcpus=16,sockets=16,cores=1,threads=1 -uuid 4013c829-c9d7-4b72-90d5-6fe58137504c -smbios 'type=1,manufacturer=oVirt,product=oVirt Node,version=7-4.1708.el7.centos,serial=44454C4C-4300-1034-8035-CAC04F424331,uuid=4013c829-c9d7-4b72-90d5-6fe58137504c' -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-110-Cultivar/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=2018-01-11T20:55:57,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-reboot -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x4 -drive file=/var/run/vdsm/storage/248f46f0-d793-4581-9810-c9d965e2f286/c2dde892-f978-4dfc-a421-c8e04cf387f9/23aa0a66-fa6c-4967-a1e5-fbe47c0cd705,format=raw,if=none,id=drive-virtio-disk0,serial=c2dde892-f978-4dfc-a421-c8e04cf387f9,cache=none,werror=stop,rerror=stop,aio=threads -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive if=none,id=drive-ide0-1-0,readonly=on -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=30,id=hostnet0,vhost=on,vhostfd=32 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:16:3e:7f:d6:83,bus=pci.0,addr=0x3 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channels/4013c829-c9d7-4b72-90d5-6fe58137504c.com.redhat.rhevm.vdsm,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.rhevm.vdsm -chardev socket,id=charchannel1,path=/var/lib/libvirt/qemu/channels/4013c829-c9d7-4b72-90d5-6fe58137504c.org.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel2,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=3,chardev=charchannel2,id=channel2,name=com.redhat.spice.0 -chardev socket,id=charchannel3,path=/var/lib/libvirt/qemu/channels/4013c829-c9d7-4b72-90d5-6fe58137504c.org.ovirt.hosted-engine-setup.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=4,chardev=charchannel3,id=channel3,name=org.ovirt.hosted-engine-setup.0 -chardev pty,id=charconsole0 -device virtconsole,chardev=charconsole0,id=console0 -spice tls-port=5900,addr=0,x509-dir=/etc/pki/vdsm/libvirt-spice,tls-channel=default,seamless-migration=on -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -object rng-random,id=objrng0,filename=/dev/urandom -device virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x5 -msg timestamp=on 2018-01-11T20:55:57.468037Z qemu-kvm: -chardev pty,id=charconsole0: char device redirected to /dev/pts/2 (label charconsole0) ==> /var/log/ovirt-hosted-engine-ha/broker.log <== File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py", line 151, in get_raw_stats f = os.open(path, direct_flag | os.O_RDONLY | os.O_SYNC) OSError: [Errno 2] No such file or directory: '/var/run/vdsm/storage/248f46f0-d793-4581-9810-c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8' StatusStorageThread::ERROR::2018-01-11 16:55:15,761::status_broker::92::ovirt_hosted_engine_ha.broker.status_broker.StatusBroker.Update::(run) Failed to read state. Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py", line 88, in run self._storage_broker.get_raw_stats() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py", line 162, in get_raw_stats .format(str(e))) RequestError: failed to read metadata: [Errno 2] No such file or directory: '/var/run/vdsm/storage/248f46f0-d793-4581-9810-c9d965e2f286/14a20941-1b84-4b82-be8f-ace38d7c037a/8582bdfc-ef54-47af-9f1e-f5b7ec1f1cf8' ==> /var/log/ovirt-hosted-engine-ha/agent.log <== result = refresh_method() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 519, in refresh_vm_conf content = self._get_file_content_from_shared_storage(VM) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 484, in _get_file_content_from_shared_storage config_volume_path = self._get_config_volume_path() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/env/config.py", line 188, in _get_config_volume_path conf_vol_uuid File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/heconflib.py", line 358, in get_volume_path root=envconst.SD_RUN_DIR, RuntimeError: Path to volume 4838749f-216d-406b-b245-98d0343fcf7f not found in /run/vdsm/storag ==> /var/log/vdsm/vdsm.log <== periodic/42::ERROR::2018-01-11 16:56:11,446::vmstats::260::virt.vmstats::(send_metrics) VM metrics collection failed Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/virt/vmstats.py", line 197, in send_metrics data[prefix + '.cpu.usage'] = stat['cpuUsage'] KeyError: 'cpuUsage'

4 8

Some major problems after 4.2 upgrade, could really use some assistance
by Jayme 12 Jan '18

12 Jan '18

I performed Ovirt 4.2 upgrade on a 3 host cluster with NFS shared storage. The shared storage is mounted from one of the hosts. I upgraded the hosted engine first, downloading the 4.2 rpm, doing a yum update then engine setup which seemed to complete successfully, at the end it powered down the hosted VM but it never came back up. I was unable to start it. I proceeded to upgrade the three hosts, ovirt 4.2 rpm and a full yum update. I also rebooted each of the three hosts. After some time the hosts did come back and almost all of the VMs are running again and seem to be working ok with the exception of two: 1. The hosted VM still will not start, I've tried everything I can think of. 2. A VM that I know existed is not running and does not appear to exist, I have no idea where it is or how to start it. 1. Hosted engine >From one of the hosts I get a weird error trying to start it: # hosted-engine --vm-start Command VM.getStats with args {'vmID': '4013c829-c9d7-4b72-90d5-6fe58137504c'} failed: (code=1, message=Virtual machine does not exist: {'vmId': u'4013c829-c9d7-4b72-90d5-6fe58137504c'}) >From the two other hosts I do not get the same error as above, sometimes it appears to start but --vm-status shows errors such as: Engine status : {"reason": "failed liveliness check", "health": "bad", "vm": "up", "detail": "Up"} Seeing these errors in syslog: Jan 11 01:06:30 host0 libvirtd: 2018-01-11 05:06:30.473+0000: 1910: error : qemuOpenFileAs:3183 : Failed to open file '/var/run/vdsm/storage/248f46f0-d793-4581-9810-c9d965e2f286/c2dde892-f978-4dfc-a421-c8e04cf387f9/23aa0a66-fa6c-4967-a1e5-fbe47c0cd705': No such file or directory Jan 11 01:06:30 host0 libvirtd: 2018-01-11 05:06:30.473+0000: 1910: error : qemuDomainStorageOpenStat:11492 : cannot stat file '/var/run/vdsm/storage/248f46f0-d793-4581-9810-c9d965e2f286/c2dde892-f978-4dfc-a421-c8e04cf387f9/23aa0a66-fa6c-4967-a1e5-fbe47c0cd705': Bad file descriptor 2. Missing VM. virsh -r list on each host does not show the VM at all. I know it existed and is important. The log on one of the hosts even shows that it started it recently then stopped in 10 or so minutes later: Jan 10 18:47:17 host3 systemd-machined: New machine qemu-9-Berna. Jan 10 18:47:17 host3 systemd: Started Virtual Machine qemu-9-Berna. Jan 10 18:47:17 host3 systemd: Starting Virtual Machine qemu-9-Berna. Jan 10 18:54:45 host3 systemd-machined: Machine qemu-9-Berna terminated. How can I find out the status of the "Berna" VM and get it running again? Thanks so much!

3 2

hosted_engine
by volga629＠networklab.ca 12 Jan '18

12 Jan '18

Hello Everyone, Is possible in 4.2 migrate hosted_engine to another storage from same type. Right now I am trying migrate from old to new iscsi storage. volga629

2 1

non operational node
by Tomeu Sastre Cabanellas 12 Jan '18

12 Jan '18

hi there, i'm testing ovirt 4.2 because I want to migrate all our VMs from XenServer, I have set a engine and a node, when conecting to the node I receive a "non-operational" and I cannot set it to ON. I'm an experienced engineer, but I'm new with oVirt, any clue where do I have to start to check ? thanks a lot. [image: Inline images 1]

3 2

wheel next to the up triangle in the UI
by Nathanaël Blanchet 12 Jan '18

12 Jan '18

Hi all, I'm using vagrant ovirt4 plugin to provision some vms, and I noticed that a wheel is still present next to the up status of those vms. What does that mean? -- Nathanaël Blanchet Supervision réseau Pôle Infrastrutures Informatiques 227 avenue Professeur-Jean-Louis-Viala 34193 MONTPELLIER CEDEX 5 Tél. 33 (0)4 67 54 84 55 Fax 33 (0)4 67 54 84 14 blanchet(a)abes.fr

2 3

[ANN] oVirt 4.2.1 First Release Candidate is now available
by Sandro Bonazzola 12 Jan '18

12 Jan '18

The oVirt Project is pleased to announce the availability of the oVirt 4.2.1 First Release Candidate, as of January 12th, 2017 This update is the first in a series of stabilization updates to the 4.2 series. This release is available now for: * Red Hat Enterprise Linux 7.4 or later * CentOS Linux (or similar) 7.4 or later This release supports Hypervisor Hosts running: * Red Hat Enterprise Linux 7.4 or later * CentOS Linux (or similar) 7.4 or later * oVirt Node 4.2 See the release notes [1] for installation / upgrade instructions and a list of new features and bugs fixed. Notes: - oVirt Appliance is already available - oVirt Live is already available [2] - oVirt Node will be available soon [2] Additional Resources: * Read more about the oVirt 4.2.1 release highlights: http://www.ovirt.org/release/4.2.1/ * Get more oVirt Project updates on Twitter: https://twitter.com/ovirt * Check out the latest project news on the oVirt blog: http://www.ovirt.org/blog/ [1] http://www.ovirt.org/release/4.2.1/ [2] http://resources.ovirt.org/pub/ovirt-4.2-pre/iso/ -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

1 0

unlock_entity.sh oVirt 4.2
by Grundmann, Christian 12 Jan '18

12 Jan '18

--_000_6A17C71B52524C408E7AAF69103E9E490FFBF1A1fabamailserverf_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi, i have a VM on oVirt 4.2 which is locked after a snapshot revert. i unlocked the disk/snapshot and vm with unlock_entity.sh it now shows no locks /usr/share/ovirt-engine/setup/dbutils/unlock_entity.sh -t all -q Locked VMs: Locked templates: Locked disks: Locked snapshots: But the VM still shows "Image Locked" How can I unlock them? Thx Christian --_000_6A17C71B52524C408E7AAF69103E9E490FFBF1A1fabamailserverf_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable <html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr= osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" = xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:= //www.w3.org/TR/REC-html40"> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"= > <meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered medium)"> <style></style> </head> <body lang=3D"DE" link=3D"#0563C1" vlink=3D"#954F72"> <div class=3D"WordSection1"> <p class=3D"MsoNormal">Hi,<o:p></o:p></p> <p class=3D"MsoNormal">i have a VM on oVirt 4.2 which is locked after a sna= pshot revert.<o:p></o:p></p> <p class=3D"MsoNormal"><o:p> </o:p></p> <p class=3D"MsoNormal">i unlocked the disk/snapshot and vm with unlock_enti= ty.sh<o:p></o:p></p> <p class=3D"MsoNormal"><o:p> </o:p></p> <p class=3D"MsoNormal">it now shows no locks<o:p></o:p></p> <p class=3D"MsoNormal"><o:p> </o:p></p> <p class=3D"MsoNormal"><span style=3D"font-size:8.0pt">/usr/share/ovirt-eng= ine/setup/dbutils/unlock_entity.sh -t all -q<o:p></o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:8.0pt">Locked VMs:<o:p></o:= p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:8.0pt"><o:p> </o:p></s= pan></p> <p class=3D"MsoNormal"><span style=3D"font-size:8.0pt">Locked templates:<o:= p></o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:8.0pt"><o:p> </o:p></s= pan></p> <p class=3D"MsoNormal"><span style=3D"font-size:8.0pt">Locked disks:<o:p></= o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:8.0pt"><o:p> </o:p></s= pan></p> <p class=3D"MsoNormal"><span style=3D"font-size:8.0pt">Locked snapshots:<o:= p></o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:8.0pt"><o:p> </o:p></s= pan></p> <p class=3D"MsoNormal"><o:p> </o:p></p> <p class=3D"MsoNormal">But the VM still shows „Image Locked“<o:= p></o:p></p> <p class=3D"MsoNormal"><o:p> </o:p></p> <p class=3D"MsoNormal">How can I unlock them?<o:p></o:p></p> <p class=3D"MsoNormal"><o:p> </o:p></p> <p class=3D"MsoNormal">Thx Christian<o:p></o:p></p> </div> </body> </html> --_000_6A17C71B52524C408E7AAF69103E9E490FFBF1A1fabamailserverf_--

1 0

10Gb Networking with OVN
by Beau Sapach 12 Jan '18

12 Jan '18

Hello everyone, I see here: https://www.ovirt.org/blog/2017/09/introducing-ovirt-4.2.0/ that version 4.2 will have OVN support. Does anyone know if this will allow VMs to take advantage of 10Gb networking without needing SR-IOV? -- Beau Sapach *System Administrator | Information Technology Services | University of Alberta Libraries* *Phone: 780.492.4181 | Email: Beau.Sapach(a)ualberta.ca <Beau.Sapach(a)ualberta.ca>*

4 6

Re: [ovirt-users] Some major problems after 4.2 upgrade, could really use some assistance
by Jayme 12 Jan '18

12 Jan '18

This is becoming critical for me, does anyone have any ideas or recommendations on what I can do to recover access to hosted VM? As of right now I have three hosts that are fully updated, they have the 4.2 repo and a full yum update was performed on them, there are no new updates to apply. The hosted engine had updates as well as a full and complete engine-setup, but did not return after being shut down. There must be some way I can get the engine running again? Please On Thu, Jan 11, 2018 at 8:24 AM, Jayme <jaymef(a)gmail.com> wrote: > The hosts have all ready been fully updated with 4.2 packages though. > > ex. > > ovirt-host.x86_64 > 4.2.0-1.el7.centos @ovirt-4.2 > ovirt-host-dependencies.x86_64 > 4.2.0-1.el7.centos @ovirt-4.2 > ovirt-host-deploy.noarch > 1.7.0-1.el7.centos @ovirt-4.2 > ovirt-hosted-engine-ha.noarch > 2.2.2-1.el7.centos @ovirt-4.2 > ovirt-hosted-engine-setup.noarch > 2.2.3-1.el7.centos @ovirt-4.2 > > On Thu, Jan 11, 2018 at 8:16 AM, Martin Sivak <msivak(a)redhat.com> wrote: > >> Hi, >> >> yes, you need to upgrade the hosts. Just take the >> ovirt-hosted-engine-ha and ovirt-hosted-engine-setup packages from >> ovirt 4.2 repositories. >> >> Martin >> >> On Thu, Jan 11, 2018 at 11:40 AM, Jayme <jaymef(a)gmail.com> wrote: >> > How do I upgrade the hosted engine packages when I can't reach it or do >> you >> > mean upgrade the hosts if so how exactly do I do that. As for the >> missing VM >> > it appears that the disk image is there but it's missing XML file I >> have no >> > idea why or how to recreate it. >> > >> > On Jan 11, 2018 4:43 AM, "Martin Sivak" <msivak(a)redhat.com> wrote: >> >> >> >> Hi, >> >> >> >> you hit one known issue we already have fixes for (4.1 hosts with 4.2 >> >> engine): >> >> https://gerrit.ovirt.org/#/q/status:open+project:ovirt-hoste >> d-engine-ha+branch:v2.1.z+topic:ovf_42_for_41 >> >> >> >> You can try hotfixing it by upgrading hosted engine packages to 4.2 or >> >> applying the patches manually and installing python-lxml. >> >> >> >> I am not sure what happened to your other VM. >> >> >> >> Best regards >> >> >> >> Martin Sivak >> >> >> >> On Thu, Jan 11, 2018 at 6:15 AM, Jayme <jaymef(a)gmail.com> wrote: >> >> > I performed Ovirt 4.2 upgrade on a 3 host cluster with NFS shared >> >> > storage. >> >> > The shared storage is mounted from one of the hosts. >> >> > >> >> > I upgraded the hosted engine first, downloading the 4.2 rpm, doing a >> yum >> >> > update then engine setup which seemed to complete successfully, at >> the >> >> > end >> >> > it powered down the hosted VM but it never came back up. I was >> unable >> >> > to >> >> > start it. >> >> > >> >> > I proceeded to upgrade the three hosts, ovirt 4.2 rpm and a full yum >> >> > update. >> >> > I also rebooted each of the three hosts. >> >> > >> >> > After some time the hosts did come back and almost all of the VMs are >> >> > running again and seem to be working ok with the exception of two: >> >> > >> >> > 1. The hosted VM still will not start, I've tried everything I can >> think >> >> > of. >> >> > >> >> > 2. A VM that I know existed is not running and does not appear to >> exist, >> >> > I >> >> > have no idea where it is or how to start it. >> >> > >> >> > 1. Hosted engine >> >> > >> >> > From one of the hosts I get a weird error trying to start it: >> >> > >> >> > # hosted-engine --vm-start >> >> > Command VM.getStats with args {'vmID': >> >> > '4013c829-c9d7-4b72-90d5-6fe58137504c'} failed: >> >> > (code=1, message=Virtual machine does not exist: {'vmId': >> >> > u'4013c829-c9d7-4b72-90d5-6fe58137504c'}) >> >> > >> >> > From the two other hosts I do not get the same error as above, >> sometimes >> >> > it >> >> > appears to start but --vm-status shows errors such as: Engine status >> >> > : {"reason": "failed liveliness check", "health": "bad", "vm": "up", >> >> > "detail": "Up"} >> >> > >> >> > Seeing these errors in syslog: >> >> > >> >> > Jan 11 01:06:30 host0 libvirtd: 2018-01-11 05:06:30.473+0000: 1910: >> >> > error : >> >> > qemuOpenFileAs:3183 : Failed to open file >> >> > >> >> > '/var/run/vdsm/storage/248f46f0-d793-4581-9810-c9d965e2f286/ >> c2dde892-f978-4dfc-a421-c8e04cf387f9/23aa0a66-fa6c- >> 4967-a1e5-fbe47c0cd705': >> >> > No such file or directory >> >> > >> >> > Jan 11 01:06:30 host0 libvirtd: 2018-01-11 05:06:30.473+0000: 1910: >> >> > error : >> >> > qemuDomainStorageOpenStat:11492 : cannot stat file >> >> > >> >> > '/var/run/vdsm/storage/248f46f0-d793-4581-9810-c9d965e2f286/ >> c2dde892-f978-4dfc-a421-c8e04cf387f9/23aa0a66-fa6c- >> 4967-a1e5-fbe47c0cd705': >> >> > Bad file descriptor >> >> > >> >> > 2. Missing VM. virsh -r list on each host does not show the VM at >> all. >> >> > I >> >> > know it existed and is important. The log on one of the hosts even >> >> > shows >> >> > that it started it recently then stopped in 10 or so minutes later: >> >> > >> >> > Jan 10 18:47:17 host3 systemd-machined: New machine qemu-9-Berna. >> >> > Jan 10 18:47:17 host3 systemd: Started Virtual Machine qemu-9-Berna. >> >> > Jan 10 18:47:17 host3 systemd: Starting Virtual Machine qemu-9-Berna. >> >> > Jan 10 18:54:45 host3 systemd-machined: Machine qemu-9-Berna >> terminated. >> >> > >> >> > How can I find out the status of the "Berna" VM and get it running >> >> > again? >> >> > >> >> > Thanks so much! >> >> > >> >> > _______________________________________________ >> >> > Users mailing list >> >> > Users(a)ovirt.org >> >> > http://lists.ovirt.org/mailman/listinfo/users >> >> > >> > >

2 1

snapshotting
by Erekle Magradze 11 Jan '18

11 Jan '18

Hello, Could you please share your experience with snapshotting the VMs in oVirt 4.1 with GlusterFS as a storage? Thanks in advance Erekle

1 0

Persistence on oVirt-Nodes
by FERNANDO FREDIANI 11 Jan '18

11 Jan '18

Quick questions about Nodes (in order to not hijack the other thread). As they don't have much notion of persistence how can I: - To have a simple configuration backup in order for, if the Operating System disk fails I reinstall the Node, restore the config and everything comes back up fine ? Backing up /var /etc and /root folders would be enough or anything else ? - If I install something custom (e.g: A Zabbix Agent) it works until the host receives a new system/image Upgrade. Is there any way that can be made persistent ? - As oVirt-Nodes are images, if I install them in a SD Card or USB Stick will be there much write to disk still ? Other than possibly the logs and configuration changes what else could be written to disk ? If so a Feature Request to solve this would be to mount the logs partition in RAM as the Node boots and logrotate that. If someone needs to keep it more time can always use rsyslog Thanks Fernando

1 0

Non-RHEL RPMs gone after RHV-H upgrade
by Colin Coe 11 Jan '18

11 Jan '18

Hi all We're running RHV 4.1.6. Yesterday I upgraded the RHV-H nodes in our DEV environment to 20180102 and found that all the non-RHEL RPMs are now gone. Their associated config files in /etc are still there. The RPMs in question were from HPE SPP plus a monitoring system client (Xymon). I had thought that non-RHEL RPMs would persist after a host upgrade. Am I wrong on this? Thanks CC

2 1

oVirt Node 4.1 question, writing files to /root and RPMs
by Matt Simonsen 11 Jan '18

11 Jan '18

Hello all, I read at https://www.ovirt.org/develop/projects/node/troubleshooting/ that "Changes made from the command line are done at your own risk. Making changes has the potential to leave your system in an unusable state." It seems clear that RPMs should not be installed. Is this accurate for https://www.ovirt.org/node/ ? We have added smartctl and hpacucli in order to do disk and RAID monitoring. So far our node servers have retained changes across reboots, which is the primary reason I'm wondering if perhaps this applies to an older version of oVirt Node. If what we have been doing is not supported, what is the suggested method to do do hardware monitoring (in particular disks)? Thanks Matt

5 5

[ANN] oVirt 4.1.9 First Release Candidate is now available
by Lev Veyde 11 Jan '18

11 Jan '18

The oVirt Project is pleased to announce the availability of the oVirt 4.1.9 First Release Candidate, as of January 11th, 2017 This update is the nineth in a series of stabilization updates to the 4.1 series. This release is available now for: * Red Hat Enterprise Linux 7.4 or later * CentOS Linux (or similar) 7.4 or later This release supports Hypervisor Hosts running: * Red Hat Enterprise Linux 7.4 or later * CentOS Linux (or similar) 7.4 or later * oVirt Node 4.1 See the release notes [1] for installation / upgrade instructions and a list of new features and bugs fixed. Notes: - oVirt Appliance is already available - oVirt Live is already available [2] - oVirt Node will be available soon [2] Additional Resources: * Read more about the oVirt 4.1.9 release highlights: http://www.ovirt.org/release/4.1. <http://www.ovirt.org/release/4.1.9/>9 <http://www.ovirt.org/release/4.1.9/>/ <http://www.ovirt.org/release/4.1.9/> * Get more oVirt Project updates on Twitter: https://twitter.com/ovirt * Check out the latest project news on the oVirt blog: http://www.ovirt.org/blog/ [1] http://www.ovirt.org/release/4.1. <http://www.ovirt.org/release/4.1.9/>9 <http://www.ovirt.org/release/4.1.9/>/ <http://www.ovirt.org/release/4.1.9/> [2] http://resources.ovirt.org/pub/ovirt-4.1/iso/ -- Lev Veyde Software Engineer, RHCE | RHCVA | MCITP Red Hat Israel <https://www.redhat.com> lev(a)redhat.com | lveyde(a)redhat.com <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

1 0

Re: [ovirt-users] Rebuilding my infra..
by Karli Sjöberg 11 Jan '18

11 Jan '18

5 5

Fresh install - unable to web gui login
by Robin 11 Jan '18

11 Jan '18

This is a multi-part message in MIME format. --------------060108090708000007020704 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Hi, I'm a new user , just installed Ovirt on a Centos 7, I gotten the web ui page but clicking on Login button does nothing. There is a warning "The redirection URI for client is not registered" screen capture. https://filedn.com/l3TGy7Y83c247u0RDYa9fkp/temp/ovirt/overt1.jpg -- Robin --------------060108090708000007020704 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit <html><head> <meta http-equiv="content-type" content="text/html; charset=ISO-8859-1"></head><body text="" bgcolor=""> Hi,<br> <br> I'm a new user , just installed Ovirt on a Centos 7, I gotten the web ui page but clicking on Login button does nothing. There is a warning<br> <br> "The redirection URI for client is not registered"<br> <br> screen capture.<br> <br> <a class="moz-txt-link-freetext" href="https://filedn.com/l3TGy7Y83c247u0RDYa9fkp/temp/ovirt/overt1.jpg">https://filedn.com/l3TGy7Y83c247u0RDYa9fkp/temp/ovirt/overt1.jpg</a><br> <br> <br> <div class="moz-signature">-- <br>Robin</div> </body> </html> --------------060108090708000007020704--

2 1

user permissions
by Zhong Qiang 11 Jan '18

11 Jan '18

Hi, I tried to give a user the permissions for vms. when this user log in vm portal ,get messages "No VM available." this user was granted follow roles: UserVmManager UserRole engine logs: 2018-01-10 20:32:33,938-05 INFO [org.ovirt.engine.core.bll.AddPermissionCommand] (EE-ManagedThreadFactory-engine-Thread-7438) [9b5a405a-c956-4d69-b286-f6b22cbf3c12] Running command: AddPermissionCommand internal: false. Entities affected : ID: 1cf75959-7992-43fe-a1f7-ed6c0c48fd35 Type: VMAction group MANIPULATE_PERMISSIONS with role type USER, ID: 1cf75959-7992-43fe-a1f7-ed6c0c48fd35 Type: VMAction group ADD_USERS_AND_GROUPS_FROM_DIRECTORY with role type USER 2018-01-10 20:32:33,944-05 INFO [org.ovirt.engine.core.bll.aaa.AddUserCommand] (EE-ManagedThreadFactory-engine-Thread-7438) [787deac0] Running command: AddUserCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group MANIPULATE_USERS with role type ADMIN 2018-01-10 20:32:33,981-05 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-7438) [787deac0] EVENT_ID: USER_ADD(149), User 'zhongq(a)ctcnet.com' was added successfully to the system. 2018-01-10 20:32:34,036-05 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-7438) [787deac0] EVENT_ID: USER_ADD_PERMISSION(850), User/Group zhongq(a)ctcnet.com, Namespace DC=ctcnet,DC=com, Authorization provider: ctcnet.com-authz was granted permission for Role UserRole on VM ubuntu16.04-64, by admin@internal-authz. 2018-01-10 20:38:06,263-05 INFO [org.ovirt.engine.core.sso.utils.AuthenticationUtils] (default task-4) [] User zhongq(a)ctcnet.com successfully logged in with scopes: ovirt-app-admin ovirt-app-api ovirt-app-portal ovirt-ext=auth:sequence-priority=~ ovirt-ext=revoke:revoke-all ovirt-ext=token-info:authz-search ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate ovirt-ext=token:passwor d-access 2018-01-10 20:38:06,301-05 INFO [org.ovirt.engine.core.bll.aaa.CreateUserSessionCommand] (default task-5) [2a6c3d14] Running command: CreateUserSessionCommand internal: false. 2018-01-10 20:38:06,338-05 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-5) [2a6c3d14] EVENT_ID: USER_VDC_LOGIN(30), User zhongq@ctcnet.com@ctcnet.com-authz connecting from '10.10.19.228' using session 'z0/9HgB4mjzfDnIN4P/fe4A3fzwWIWWcR9xKDvsI/XXgHZApjRp1BCufgtSK6n3kvA/ScdP4qqGqiX01lyJHSQ==' logged in. 2018-01-10 20:38:06,956-05 ERROR [org.ovirt.engine.core.bll.GetSystemStatisticsQuery] (default task-14) [06c80cc6-ad15-4d82-a907-21ab9a5c1cc4] Query execution failed due to insufficient permissions. 2018-01-10 20:38:07,044-05 ERROR [org.ovirt.engine.core.bll.GetPermissionsForObjectQuery] (default task-20) [1b7a6564-534d-4df5-a2b7-52da214b95cd] Query execution failed due to insufficient permissions. 2018-01-10 20:38:07,045-05 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-20) [] Operation Failed: query execution failed due to insufficient permissions.

1 1

ovirt-live 4.2 - missing stable ISO? + Networking question - when to setup host & in NM or ifcfg?
by Sam McLeod 11 Jan '18

11 Jan '18

--Apple-Mail=_C8B56014-332F-4A8D-8A2A-EDDDF68CE520 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii I'm trying to find the stable / current oVirt-live ISO to download. According to the official documentation, it looks like there is only the = legacy 4.1 ISO, or nightly / unstable builds of 4.2(.2?)? SRC: https://www.ovirt.org/download/ovirt-live/ = <https://www.ovirt.org/download/ovirt-live/> --- Alternatively, if one is to deploy the 'self-hosted-engine' on top of = CentOS 7.4, the documentation doesn't make it clear what you should vs = shouldn't setup on the host prior to deploying the hosted engine. For example, some big questions pop up around networking, e.g.: - Should I be setting those up during CentOS's install using the = installer (which I believe configures them with Network Manager), or = should I be setting up the ifcfg files manually by hand without touching = Network Manager via the server's remote console after the install has = finished? - Is it OK for me to setup my first two NICs in a mode 1 (A/P) or 4 = (XOR) bond (these are connected to a 'dumb' switch and provide internet = access and later will be used for management activities such as vm = migrations). - Is it OK for me to setup my second two NICs in a LACP bond (required = as they connect to our core switches) and to add VLANs on top of that = bond, include the storage VLAN required for iSCSI access which is later = required to host the hosted-engine? SRC: = https://www.ovirt.org/documentation/self-hosted/chap-Deploying_Self-Hosted= _Engine/ = <https://www.ovirt.org/documentation/self-hosted/chap-Deploying_Self-Hoste= d_Engine/> --- I think the hardest thing is that the documentation for oVirt seems very = poorly maintained, or it's at least scattered around various links or = different guides. Perhaps this isn't obvious to people that are already familiar with the = components, terminology and setup practises of oVirt / RHEV, but for = someone like me coming from XenServer - it's confusing as anything. Example diagram of infrastructure: https://i.imgur.com/U4hCP3a.png = <https://i.imgur.com/U4hCP3a.png> -- Sam McLeod (protoporpoise on IRC) https://smcleod.net https://twitter.com/s_mcleod Words are my own opinions and do not necessarily represent those of my = employer or partners. --Apple-Mail=_C8B56014-332F-4A8D-8A2A-EDDDF68CE520 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; = charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><div = class=3D"">I'm trying to find the stable / current oVirt-live ISO to = download.</div><div class=3D""><br class=3D""></div><div = class=3D"">According to the official documentation, it looks like there = is only the legacy 4.1 ISO, or nightly / unstable builds of = 4.2(.2?)?</div><div class=3D""><br class=3D""></div><div = class=3D"">SRC: <a = href=3D"https://www.ovirt.org/download/ovirt-live/" = class=3D"">https://www.ovirt.org/download/ovirt-live/</a></div><div = class=3D""><br class=3D""></div><div class=3D"">---</div><div = class=3D""><br class=3D""></div><div class=3D"">Alternatively, if one is = to deploy the 'self-hosted-engine' on top of CentOS 7.4, the = documentation doesn't make it clear what you should vs shouldn't setup = on the host prior to deploying the hosted engine.</div><div class=3D""><br= class=3D""></div><div class=3D"">For example, some big questions pop up = around networking, e.g.:</div><div class=3D""><br class=3D""></div><div = class=3D"">- Should I be setting those up during CentOS's install using = the installer (which I believe configures them with Network Manager), or = should I be setting up the ifcfg files manually by hand without = touching Network Manager via the server's remote console after the = install has finished?</div><div class=3D""><br class=3D""></div><div = class=3D"">- Is it OK for me to setup my first two NICs in a mode 1 = (A/P) or 4 (XOR) bond (these are connected to a 'dumb' switch and = provide internet access and later will be used for management activities = such as vm migrations).</div><div class=3D""><br class=3D""></div><div = class=3D"">- Is it OK for me to setup my second two NICs in a LACP bond = (required as they connect to our core switches) and to add VLANs on top = of that bond, include the storage VLAN required for iSCSI access which = is later required to host the hosted-engine?</div><div class=3D""><br = class=3D""></div><div class=3D"">SRC: <a = href=3D"https://www.ovirt.org/documentation/self-hosted/chap-Deploying_Sel= f-Hosted_Engine/" = class=3D"">https://www.ovirt.org/documentation/self-hosted/chap-Deploying_= Self-Hosted_Engine/</a></div><div class=3D""><br class=3D""></div><div = class=3D"">---</div><div class=3D""><br class=3D""></div><div class=3D"">I= think the hardest thing is that the documentation for oVirt seems very = poorly maintained, or it's at least scattered around various links or = different guides.</div><div class=3D""><br class=3D""></div><div = class=3D"">Perhaps this isn't obvious to people that are already = familiar with the components, terminology and setup practises of oVirt / = RHEV, but for someone like me coming from XenServer - it's confusing as = anything.</div><div class=3D""><br class=3D""></div><div class=3D""><br = class=3D""></div><div class=3D"">Example diagram of = infrastructure: <a href=3D"https://i.imgur.com/U4hCP3a.png" = class=3D"">https://i.imgur.com/U4hCP3a.png</a></div><div class=3D""><br = class=3D""></div><div class=3D""><div class=3D""> <div dir=3D"auto" style=3D"color: rgb(0, 0, 0); letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: = break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" = class=3D""><div dir=3D"auto" style=3D"color: rgb(0, 0, 0); = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><div = dir=3D"auto" style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = line-break: after-white-space;" class=3D""><div style=3D"color: rgb(0, = 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; = font-variant-caps: normal; font-weight: normal; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;">--<br = class=3D"">Sam McLeod (protoporpoise on IRC)</div><div style=3D"color: = rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: = normal; font-variant-caps: normal; font-weight: normal; letter-spacing: = normal; text-align: start; text-indent: 0px; text-transform: none; = white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: = 0px;"><a href=3D"https://smcleod.net" = class=3D"">https://smcleod.net</a><br = class=3D"">https://twitter.com/s_mcleod<br class=3D""><br class=3D"">Words= are my own opinions and do not necessarily represent those of = my employer or partners.</div></div></div></div> </div> <br class=3D""></div></body></html>= --Apple-Mail=_C8B56014-332F-4A8D-8A2A-EDDDF68CE520--

2 3

Hosted Engine and network isolation
by Luca 'remix_tj' Lorenzetto 10 Jan '18

10 Jan '18

Hello, i just completed the setup of my 4.1 new cluster and i'm validating against a set of certification tests i did in the earlier version. One of the test was consisting on isolating the host where hosted-engine vm is running, setting ports on the switch down. Storage domain hosting the engine is FC. Up to 4.0.4, that was the version we was using during that tests, isolating the host was causing a restart of the engine vm on another available host with the fence of the isolated host. Now, with 4.1.8, isolating the host doesn't restart the engine on the other host, only moves the status of the host2 to EngineUnexpectedlyDown, while host1 is isolated, with the engine vm still running on it. Is this due to storage leases? What should be the right action to take in case of isolated host for moving engine away? Thank you, Luca -- "E' assurdo impiegare gli uomini di intelligenza eccellente per fare calcoli che potrebbero essere affidati a chiunque se si usassero delle macchine" Gottfried Wilhelm von Leibnitz, Filosofo e Matematico (1646-1716) "Internet è la più grande biblioteca del mondo. Ma il problema è che i libri sono tutti sparsi sul pavimento" John Allen Paulos, Matematico (1945-vivente) Luca 'remix_tj' Lorenzetto, http://www.remixtj.net , <lorenzetto.luca(a)gmail.com>

1 0

Planned restart of production services
by Evgheni Dereveanchin 10 Jan '18

10 Jan '18

Hi everyone, I will be restarting several production systems within the following hour to apply security updates. The following services may be unreachable for some period of time: - resources.ovirt.org - package repositories - gerrit.ovirt.org - code review - jenkins.ovirt.org - CI master It will not be possible to submit/review patches, clone repositories or run CI jobs during this period. Package repositories will also be unreachable for a short period of time. I will announce you once the maintenance is complete. -- Regards, Evgheni Dereveanchin

1 2

oVirt 4.2 CEPH support
by Abdurrahman A. Ibrahim 09 Jan '18

09 Jan '18

Hello, I read in RHV 4.2 Beta release note that CEPH will be supported using iSCSI. I have tried to check community documentation regarding CEPH support but there was no luck. Do we have such document? Best regards, Ab

9 11

ovirt 4.2 upgrade questions
by Peter Hudec 09 Jan '18

09 Jan '18

Hi, maybe it was already here, but I haven't found it quickly in archive ;( I upgrade the hosted engine and one hots, my notes and questions. The upgrade goes well, I only needed to manually fix the memy value in database for hosted engine [ ERROR ] schema.sh: FATAL: Cannot execute sql command: --file=/usr/share/ovirt-engine/dbscripts/upgrade/04_02_0140_add_max_memory_constraint.sql [ ERROR ] Failed to execute stage 'Misc configuration': Engine schema refresh failed 1) firewalld after upgrade the hot server, the i needed to stop firewalld. It seems, that, the rules are not generated correctly. The engine was not able to connect to the host. How do I could fix it? 2) old repo removal Could i remove the 4.1 repo? If yes, what is the best way to do that? 3) Hosted Engine HA: Hosted Engine HA on upgraded hosts is 3400, the same as on the 4.1 hosts. Is this good or bad? regards Peter -- *Peter Hudec* Infraštruktúrny architekt phudec(a)cnc.sk <mailto:phudec@cnc.sk> *CNC, a.s.* Borská 6, 841 04 Bratislava Recepcia: +421 2 35 000 100 Mobil:+421 905 997 203 *www.cnc.sk* <http:///www.cnc.sk>

3 15

Behaviour when attaching shared iSCSI storage with existing data
by Sam McLeod 09 Jan '18

09 Jan '18

--Apple-Mail=_0EC3C8C4-0EB1-4E35-A593-42CEA5D59791 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii If one was to attach a shared iSCSI LUN as 'storage' to an oVirt data = centre that contains existing data - how does oVirt behave? For example the LUN might be partitioned as LVM, then contain existing = filesystems etc... =20 - Would oVirt see that there is existing data on the LUN and simply = attach it as any other linux initiator (client) world, or would it try = to wipe the LUN clean and reinitialise it? Context: Investigating migration from XenServer to oVirt (4.2.0) All our iSCSI storage is currently attached to XenServer hosts, = XenServer formats those raw LUNs with LVM and VMs are stored within = them. If the answer to this is already out there and I should have found it by = searching, I apologise, please point me to the link and I'll RTFM. -- Sam McLeod https://smcleod.net https://twitter.com/s_mcleod --Apple-Mail=_0EC3C8C4-0EB1-4E35-A593-42CEA5D59791 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; = charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><div = class=3D"">If one was to attach a shared iSCSI LUN as 'storage' to an = oVirt data centre that contains existing data - how does oVirt = behave?</div><div class=3D""><br class=3D""></div><div class=3D"">For = example the LUN might be partitioned as LVM, then contain existing = filesystems etc...</div><div class=3D""> </div><div class=3D"">- = Would oVirt see that there is existing data on the LUN and simply attach = it as any other linux initiator (client) world, or would it try to wipe = the LUN clean and reinitialise it?</div><div class=3D""><br = class=3D""></div><div class=3D""><br class=3D""></div><div = class=3D"">Context: Investigating migration from XenServer to oVirt = (4.2.0)</div><div class=3D""><br class=3D""></div><div class=3D"">All = our iSCSI storage is currently attached to XenServer hosts, XenServer = formats those raw LUNs with LVM and VMs are stored within = them.</div><div class=3D""><br class=3D""></div><div class=3D""><br = class=3D""></div><div class=3D""><br class=3D""></div><div class=3D""><i = class=3D"">If the answer to this is already out there and I should have = found it by searching, I apologise, please point me to the link and I'll = RTFM.</i></div><div class=3D""><div class=3D""> <div dir=3D"auto" style=3D"color: rgb(0, 0, 0); letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: = break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" = class=3D""><div dir=3D"auto" style=3D"color: rgb(0, 0, 0); = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><div = dir=3D"auto" style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = line-break: after-white-space;" class=3D""><div style=3D"color: rgb(0, = 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; = font-variant-caps: normal; font-weight: normal; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><br = class=3D"">--<br class=3D"">Sam McLeod<br class=3D""><a = href=3D"https://smcleod.net" class=3D"">https://smcleod.net</a><br = class=3D"">https://twitter.com/s_mcleod</div></div></div></div> </div> <br class=3D""></div></body></html>= --Apple-Mail=_0EC3C8C4-0EB1-4E35-A593-42CEA5D59791--

3 6

Re: [ovirt-users] How power-saving schedule policy applied?
by Karli Sjöberg 09 Jan '18

09 Jan '18

3 4

Suggestions on changing hosts' network
by Gianluca Cecchi 09 Jan '18

09 Jan '18

Hello, I have a 4.1 cluster based on FC storage domain. There are 2 hosts that are oVirt node-ng The engine is external to the oVirt environment Currently the network situation for ovirtmgmt is this one: engine on network1 (eg 192.168.1.x/24) host1 and host2 on network2 (eg 192.168.2.x/24) network1 and network2 routed through a gw I have to physically move host1 and host2 so that I should change their ip with one on network1. When I added the hosts in oVirt I used as Hostname/IP field their dns hostname What is the simplest approach? I can also give total downtime without particular problems as this is a test environment. I think I can put all infra into maintenance, change dns configuration but then I have also to change hosts network configuration itself... Possibly the best thing to do is remove/add the hosts? I have some doubts about the workflow to follow for removal/add and if for an oVirt-ng node I can change its ip configuration in an easy way.. perhaps from cockpit and then reboot and connect via cockpit to the new ip Thanks in advance for any suggestion Gianluca

4 7

Rebuilding my infra..
by carl langlois 08 Jan '18

08 Jan '18

Hi all After screwing my infra with the update to 4.2 (probably a bad manipulation), i am planning a rebuild of the entire infra. First i want to replace my NFS storage with a glusterfs storage. All documentation tell me that i need 3 hosts.. but for the moment i only have 2 but planning to had more later. So does it make sense to start with 2 hosts and use glusterfs as the storage domain(lets says with a replicate of two with all its limitations). If it make sense, 1- what is the best way to do it. 2- how hard will it be to had the 3rd host when available and make it replica 2+arbiter. Also in a setup where i have 3 hosts (replica 2+arbiter) does all the 3 hosts can run users vm? Thanks for your inputs. Carl

4 4

Deploying self-hosted engine - Waiting for VDSM host to become operation + received packet with own address as source
by Sam McLeod 08 Jan '18

08 Jan '18

--Apple-Mail=_20A673E9-6970-46A2-AB1D-24C67EB2F4DB Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hello, I'm trying to setup a host as a self-hosted engine as per = https://www.ovirt.org/documentation/self-hosted/chap-Deploying_Self-Hosted= _Engine/ = <https://www.ovirt.org/documentation/self-hosted/chap-Deploying_Self-Hoste= d_Engine/>. The host is configured with two bonded network interfaces: bond0 =3D management network for hypervisors, setup as active/passive. bond1 =3D network that has VLANs for various network segments for = virtual machines to use, setup as LACP bond to upstream switches. On the host, both networks are operational and work as expected. When setting up the self-hosted engine, bond0 is selected as network to = bridge with, and a unique IP is given to the self-hosted engine VM. During the final stages of the self-hosted engine setup, the installer = gets stuck on ' Waiting for the VDSM host to become operational'. While it is repeating this every minute or so the host repeats the = message 'bond0: received packet with own address as source address', = which is odd to me as it's in active/passive bond and I'd only expect to = see this kind of message on XOR / load balanced interfaces. Host console screenshot: https://imgur.com/a/a2JLd = <https://imgur.com/a/a2JLd> Host OS: CentOS 7.4 oVirt version: 4.2.0 ip a on host while install is stuck waiting for VDSM: # ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen = 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: enp2s0f0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9000 qdisc mq = master bond1 portid 0100000000000000000000373031384833 state UP qlen = 1000 link/ether 78:e3:b5:10:74:88 brd ff:ff:ff:ff:ff:ff 3: enp2s0f1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9000 qdisc mq = master bond1 portid 0200000000000000000000373031384833 state UP qlen = 1000 link/ether 78:e3:b5:10:74:88 brd ff:ff:ff:ff:ff:ff 4: ens1f0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq = master bond0 portid 0100000000000000000000474835323543 state UP qlen = 1000 link/ether 00:9c:02:3c:49:90 brd ff:ff:ff:ff:ff:ff 5: ens1f1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq = master bond0 portid 0200000000000000000000474835323543 state UP qlen = 1000 link/ether 00:9c:02:3c:49:90 brd ff:ff:ff:ff:ff:ff 6: bond1: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 9000 qdisc = noqueue state UP qlen 1000 link/ether 78:e3:b5:10:74:88 brd ff:ff:ff:ff:ff:ff inet6 fe80::7ae3:b5ff:fe10:7488/64 scope link valid_lft forever preferred_lft forever 7: storage@bond1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc = noqueue state UP qlen 1000 link/ether 78:e3:b5:10:74:88 brd ff:ff:ff:ff:ff:ff inet 10.51.40.172/24 brd 10.51.40.255 scope global storage valid_lft forever preferred_lft forever inet6 fe80::7ae3:b5ff:fe10:7488/64 scope link valid_lft forever preferred_lft forever 9: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen = 1000 link/ether ae:c0:02:25:42:24 brd ff:ff:ff:ff:ff:ff 10: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen = 1000 link/ether be:92:5d:c3:28:4d brd ff:ff:ff:ff:ff:ff 30: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc = noqueue master ovirtmgmt state UP qlen 1000 link/ether 00:9c:02:3c:49:90 brd ff:ff:ff:ff:ff:ff 46: ovirtmgmt: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue = state UP qlen 1000 link/ether 00:9c:02:3c:49:90 brd ff:ff:ff:ff:ff:ff inet 10.51.14.112/24 brd 10.51.14.255 scope global ovirtmgmt valid_lft forever preferred_lft forever 47: ;vdsmdummy;: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN = qlen 1000 link/ether 8e:c0:25:88:40:de brd ff:ff:ff:ff:ff:ff Before the self-hosted engine install was run the following did not = exist: ;vdsmdummy; ovs-system br-int ovirtmgmt and bond0 was not a slave of ovirtmgmt. I'm now going to kick off a complete reinstall of CentOS 7 on the host = as I've since tried cleaning up the host using the = ovirt-hosted-engine-cleanup command and removing the packages which seem = to leave the network configuration a mess an doesn't actually cleanup = files on disk as expected. -- Sam McLeod (protoporpoise on IRC) https://smcleod.net https://twitter.com/s_mcleod Words are my own opinions and do not necessarily represent those of my = employer or partners. --Apple-Mail=_20A673E9-6970-46A2-AB1D-24C67EB2F4DB Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; = charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" = class=3D"">Hello,<div class=3D""><br class=3D""></div><div class=3D"">I'm = trying to setup a host as a self-hosted engine as per <a = href=3D"https://www.ovirt.org/documentation/self-hosted/chap-Deploying_Sel= f-Hosted_Engine/" = class=3D"">https://www.ovirt.org/documentation/self-hosted/chap-Deploying_= Self-Hosted_Engine/</a>.</div><div class=3D"">The host is configured = with two bonded network interfaces:</div><div class=3D""><br = class=3D""></div><div class=3D"">bond0 =3D management network for = hypervisors, setup as active/passive.</div><div class=3D"">bond1 =3D = network that has VLANs for various network segments for virtual machines = to use, setup as LACP bond to upstream switches.</div><div class=3D""><div= class=3D""><br class=3D"webkit-block-placeholder"></div><div = class=3D"">On the host, both networks are operational and work as = expected.</div><div class=3D""><br class=3D""></div><div class=3D"">When = setting up the self-hosted engine, bond0 is selected as network to = bridge with, and a unique IP is given to the self-hosted engine = VM.</div><div class=3D""><br class=3D""></div><div class=3D"">During the = final stages of the self-hosted engine setup, the installer gets stuck = on ' Waiting for the VDSM host to become operational'.</div><div = class=3D"">While it is repeating this every minute or so the host = repeats the message 'bond0: received packet with own address as source = address', which is odd to me as it's in active/passive bond and I'd only = expect to see this kind of message on XOR / load balanced = interfaces.</div><div class=3D""><br = class=3D"webkit-block-placeholder"></div><div class=3D"">Host console = screenshot: <a href=3D"https://imgur.com/a/a2JLd" = class=3D"">https://imgur.com/a/a2JLd</a></div><div class=3D"">Host OS: = CentOS 7.4</div><div class=3D"">oVirt version: 4.2.0</div><div = class=3D""><br class=3D""></div><div class=3D"">ip a on host while = install is stuck waiting for VDSM:</div><div class=3D""><br = class=3D""></div><div class=3D""><font face=3D"IBMPlexMono" class=3D""># = ip a<br class=3D"">1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc = noqueue state UNKNOWN qlen 1000<br class=3D"">    = link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00<br class=3D""> =   inet 127.0.0.1/8 scope host lo<br class=3D"">    =    valid_lft forever preferred_lft forever<br class=3D"">  =   inet6 ::1/128 scope host<br class=3D"">      =  valid_lft forever preferred_lft forever<br class=3D"">2: enp2s0f0: = <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9000 qdisc mq master = bond1 portid 0100000000000000000000373031384833 state UP qlen 1000<br = class=3D"">    link/ether 78:e3:b5:10:74:88 brd = ff:ff:ff:ff:ff:ff<br class=3D"">3: enp2s0f1: = <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9000 qdisc mq master = bond1 portid 0200000000000000000000373031384833 state UP qlen 1000<br = class=3D"">    link/ether 78:e3:b5:10:74:88 brd = ff:ff:ff:ff:ff:ff<br class=3D"">4: ens1f0: = <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master = bond0 portid 0100000000000000000000474835323543 state UP qlen 1000<br = class=3D"">    link/ether 00:9c:02:3c:49:90 brd = ff:ff:ff:ff:ff:ff<br class=3D"">5: ens1f1: = <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master = bond0 portid 0200000000000000000000474835323543 state UP qlen 1000<br = class=3D"">    link/ether 00:9c:02:3c:49:90 brd = ff:ff:ff:ff:ff:ff<br class=3D"">6: bond1: = <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 9000 qdisc noqueue = state UP qlen 1000<br class=3D"">    link/ether = 78:e3:b5:10:74:88 brd ff:ff:ff:ff:ff:ff<br class=3D"">    = inet6 fe80::7ae3:b5ff:fe10:7488/64 scope link<br class=3D"">  =      valid_lft forever preferred_lft forever<br = class=3D"">7: storage@bond1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu = 9000 qdisc noqueue state UP qlen 1000<br class=3D"">    = link/ether 78:e3:b5:10:74:88 brd ff:ff:ff:ff:ff:ff<br class=3D"">  =   inet 10.51.40.172/24 brd 10.51.40.255 scope global storage<br = class=3D"">       valid_lft forever preferred_lft = forever<br class=3D"">    inet6 fe80::7ae3:b5ff:fe10:7488/64 = scope link<br class=3D"">       valid_lft forever = preferred_lft forever<br class=3D"">9: ovs-system: = <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000<br = class=3D"">    link/ether ae:c0:02:25:42:24 brd = ff:ff:ff:ff:ff:ff<br class=3D"">10: br-int: <BROADCAST,MULTICAST> = mtu 1500 qdisc noop state DOWN qlen 1000<br class=3D"">    = link/ether be:92:5d:c3:28:4d brd ff:ff:ff:ff:ff:ff<br class=3D"">30: = bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc = noqueue master ovirtmgmt state UP qlen 1000<br class=3D"">    = link/ether 00:9c:02:3c:49:90 brd ff:ff:ff:ff:ff:ff<br class=3D"">46: = ovirtmgmt: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc = noqueue state UP qlen 1000<br class=3D"">    link/ether = 00:9c:02:3c:49:90 brd ff:ff:ff:ff:ff:ff<br class=3D"">    inet = 10.51.14.112/24 brd 10.51.14.255 scope global ovirtmgmt<br = class=3D"">       valid_lft forever preferred_lft = forever<br class=3D"">47: ;vdsmdummy;: <BROADCAST,MULTICAST> mtu = 1500 qdisc noop state DOWN qlen 1000<br class=3D"">    = link/ether 8e:c0:25:88:40:de brd ff:ff:ff:ff:ff:ff</font></div><div = class=3D""><br class=3D""></div><div class=3D""><br class=3D""></div><div = class=3D"">Before the self-hosted engine install was run the following = did not exist:</div><div class=3D""><br class=3D""></div><div = class=3D""><font face=3D"IBMPlexMono" = class=3D"">;vdsmdummy;</font></div><div class=3D""><font = face=3D"IBMPlexMono" class=3D"">ovs-system</font></div><div = class=3D""><font face=3D"IBMPlexMono" class=3D"">br-int</font></div><div = class=3D""><font face=3D"IBMPlexMono" = class=3D"">ovirtmgmt</font></div><div class=3D""><br class=3D""></div><div= class=3D"">and bond0 was <i class=3D"">not</i> a slave of = ovirtmgmt.</div><div class=3D""><br class=3D""></div><div class=3D"">I'm = now going to kick off a complete reinstall of CentOS 7 on the host as = I've since tried cleaning up the host using the = ovirt-hosted-engine-cleanup command and removing the packages which seem = to leave the network configuration a mess an doesn't actually cleanup = files on disk as expected.</div><div class=3D""><br class=3D""></div><div = class=3D""><div dir=3D"auto" style=3D"color: rgb(0, 0, 0); = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><div = dir=3D"auto" style=3D"color: rgb(0, 0, 0); letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: = break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" = class=3D""><div dir=3D"auto" style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><div = style=3D"color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = font-style: normal; font-variant-caps: normal; font-weight: normal; = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px;"><br = class=3D""></div></div></div></div></div></div><div class=3D""> <div dir=3D"auto" style=3D"color: rgb(0, 0, 0); letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: = break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" = class=3D""><div dir=3D"auto" style=3D"color: rgb(0, 0, 0); = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><div = dir=3D"auto" style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = line-break: after-white-space;" class=3D""><div style=3D"color: rgb(0, = 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; = font-variant-caps: normal; font-weight: normal; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;">--<br = class=3D"">Sam McLeod (protoporpoise on IRC)</div><div style=3D"color: = rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: = normal; font-variant-caps: normal; font-weight: normal; letter-spacing: = normal; text-align: start; text-indent: 0px; text-transform: none; = white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: = 0px;"><a href=3D"https://smcleod.net" = class=3D"">https://smcleod.net</a><br = class=3D"">https://twitter.com/s_mcleod<br class=3D""><br class=3D"">Words= are my own opinions and do not necessarily represent those of = my employer or partners.</div></div></div></div> </div> <br class=3D""></body></html>= --Apple-Mail=_20A673E9-6970-46A2-AB1D-24C67EB2F4DB--

2 2

Re: [ovirt-users] Ovirt 4.0
by Wesley Stewart 08 Jan '18

08 Jan '18

I believe you most upgrade to 4.1 first, and then to 4.2. On Jan 8, 2018 6:00 AM, "Marktvk" <marktvk(a)xs4all.nl> wrote: Hello, We have now running ovirt 4.0 for a specif reason we can not upgrade to the new version. If possible for 1 time to upgrade qemu-kvm to the new version for the fix with processors today ? I hope so. _______________________________________________ Users mailing list Users(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/users

4 3

ovirt 4.1 hosted engine: Resize or Change "hosted-engine" Data Domain
by yayo (j) 08 Jan '18

08 Jan '18

hi at all, We have the "hosted engine" Data domain on a FC LUN too big for only the hosted-engine so we want to create another little FC LUN and move the hosted-engine vm on this new LUN and destroy old one ... Is there any official workflow or how to to do this operation? Or, someone can guide me? Thank you!

3 6

oVirt 4.2 translations
by Sandro Bonazzola 08 Jan '18

08 Jan '18

Hi, oVirt project is going to finalize translations, if you have time and want to contribute translations to your native language, the project is spread over the following four links: https://translate.zanata.org/iteration/view/ovirt/ovirt-4.2 https://translate.zanata.org/iteration/view/ovirt-engine- dashboard/ovirt-dashboard-1.2 https://translate.zanata.org/iteration/view/ovirt-web-ui/1.3.0 If you have some extra time you might also consider contributing translations to Red Hat downstream: https://translate.zanata.org/iteration/view/rhevm-branding/rhvm-4.2 Thanks -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

1 0

How to simulate update of ng node from 4.1.7 to 4.1.8
by Gianluca Cecchi 08 Jan '18

08 Jan '18

Suppose I download both the isos: the fixed one for 4.1.17 here: http://resources.ovirt.org/pub/ovirt-4.1/iso/ovirt-node-ng-installer-ovirt/… and the 4.1.8 here: http://resources.ovirt.org/pub/ovirt-4.1/iso/ovirt-node-ng-installer-ovirt/… and I want to simulate install of 4.1.7 from the iso and then update of the ng node to 4.1.8 without internet access but using in some way the 4.1.8 iso (or something derived from it) on my local lan, how can I do? Thanks, Gianluca

2 1

[CORRECTED] Can't create 2nd network for VMs in DMZ zone
by Andrei V 08 Jan '18

08 Jan '18

Hi ! Made a typo in previous message, sorry. I'm having difficulty to utilize 2nd NIC on HP Proliant server in order to connect guest VM to DMZ (engine, node and all other guest VMs are on internal zone). I tried to use PCI passthrough, downed ifcfg-enp3s4f1 on node host, created vNic profile "PCI NIC Passthrough", network named "Node10-NIC2", then Virtual Machines -> Network Interfaces -> nic 2 of type PCI Passthrough and profile above. Unfortunately, guest OS (SuSE Leap 42.3) still don't recognize this NIC. This 2nd NIC is visible on Compute -> Hosts -> Network Interfaces. I can assign it network named "Node10-NIC2". However, right now I'm working from the remote, if node net setup get screwed, it will be a big problem. In the archive I found that this 2nd NIC must be up with no IP and default route. Is this correct definition of ifcfg-enp3s4f1? NetworkManager is disabled on node. TYPE=Ethernet PROXY_METHOD=none BROWSER_ONLY=no BOOTPROTO=none DEFROUTE=no IPV4_FAILURE_FATAL=no IPV6INIT=no NM_CONTROLLED=no NAME=enp3s4f1 UUID=09f96366-e4a9-4e6e-a090-7267ed102d36 DEVICE=enp3s4f1 ONBOOT=yes Thanks in advance Andrei V

2 1

How power-saving schedule policy applied?
by Alex Shen 08 Jan '18

08 Jan '18

This is a multipart message in MIME format. ------=_NextPart_000_0093_01D388B2.B43A6B40 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi, I'm wondering how to apply power-saving schedule in ovirt cluster(version i= s 4.2.0.2-1.el7.centos)? Is there any instruction or manual with UI snapshots= guidance? That will be appreciated. I've configed all host with ipmilan protocol and test ok. I suppose that power saving policy should converge VMs into hosts one by one. Something like that. From the beginning, all hosts are in 'power off' state, if a VM launch, power saving scheduling module should power on one host, and assign= VM into that host. With many VMs activated, exceeding the threshold, power saving scheduling module should turn on another host. If anyone had played power saving scheduling policy, please share me your scenarios or some hints. Thanks a lot. Alex --- Avast =E9=98=B2=E6=AF=92=E8=BD=AF=E4=BB=B6=E5=B7=B2=E5=AF=B9=E6=AD=A4=E7=94= =B5=E5=AD=90=E9=82=AE=E4=BB=B6=E6=89=A7=E8=A1=8C=E7=97=85=E6=AF=92=E6=A3=80= =E6=9F=A5=E3=80=82 https://www.avast.com/antivirus ------=_NextPart_000_0093_01D388B2.B43A6B40 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr= osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" = xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:= //www.w3.org/TR/REC-html40"><head><meta http-equiv=3DContent-Type content= =3D"text/html; charset=3DUTF-8"><meta name=3DGenerator content=3D"Microsoft= Word 15 (filtered medium)"><style></style></head><body lang=3DZH-CN link=3D"#0563C1= " vlink=3D"#954F72" style=3D'text-justify-trim:punctuation'><div class=3DWo= rdSection1><p class=3DMsoNormal><span lang=3DEN-US>Hi,<o:p></o:p></span></p= ><p class=3DMsoNormal><span lang=3DEN-US><o:p> </o:p></span></p><p cla= ss=3DMsoNormal><span lang=3DEN-US>I’m wondering how to apply power-sa= ving schedule in ovirt cluster(version is <span class=3Dgwt-inlinelabel>4.2= =2E0.2-1.el7.centos</span>)? Is there any instruction or manual with UI sna= pshots guidance? That will be appreciated. <o:p></o:p></span></p><p class= =3DMsoNormal><span lang=3DEN-US><o:p> </o:p></span></p><p class=3DMsoN= ormal><span lang=3DEN-US>I’ve configed all host with ipmilan protocol= and test ok. I suppose that power saving policy should converge VMs into h= osts one by one. Something like that. From the beginning, all hosts are in = ‘power off’ state, if a VM launch, power saving scheduling modu= le should power on one host, and assign VM into that host. With many VMs ac= tivated, exceeding the threshold, power saving scheduling module should tur= n on another host…<o:p></o:p></span></p><p class=3DMsoNormal><span la= ng=3DEN-US><o:p> </o:p></span></p><p class=3DMsoNormal><span lang=3DEN= -US>If anyone had played power saving scheduling policy, please share me yo= ur scenarios or some hints. Thanks a lot.<o:p></o:p></span></p><p class=3DM= soNormal><span lang=3DEN-US><o:p> </o:p></span></p><p class=3DMsoNorma= l><span lang=3DEN-US>Alex<o:p></o:p></span></p></div><div id=3D"DAB4FAD8-2D= D7-40BB-A1B8-4E2AA1F9FDF2"><br /> <table style=3D"border-top: 1px solid #D3= D4DE;"> <tr> <td style=3D"width: 55px; padding-top: 18px;"><a href=3D"https://www.= avast.com/sig-email?utm_medium=3Demail&utm_source=3Dlink&utm_campaign=3Dsig= -email&utm_content=3Demailclient" target=3D"_blank"><img src=3D"https://ipm= cdn.avast.com/images/icons/icon-envelope-tick-round-orange-animated-no-repe= at-v1.gif" alt=3D"" width=3D"46" height=3D"29" style=3D"width: 46px; height= : 29px;" /></a></td> <td style=3D"width: 470px; padding-top: 17px; color: #41424e; font-size: = 13px; font-family: Arial, Helvetica, sans-serif; line-height: 18px;">=E6=97= =A0=E7=97=85=E6=AF=92=E3=80=82<a href=3D"https://www.avast.com/sig-email?ut= m_medium=3Demail&utm_source=3Dlink&utm_campaign=3Dsig-email&utm_content=3De= mailclient" target=3D"_blank" style=3D"color: #4453ea;">www.avast.com</a> = </td> </tr> </table> <a href=3D"#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2" width=3D"1" height=3D"1">= </a></div></body></html> ------=_NextPart_000_0093_01D388B2.B43A6B40--

1 0

Ovirt 4.0
by Marktvk 08 Jan '18

08 Jan '18

Hello, We have now running ovirt 4.0 for a specif reason we can not upgrade to the new version. If possible for 1 time to upgrade qemu-kvm to the new version for the fix with processors today ? I hope so.

1 0

Re: [ovirt-users] bad bond name when setting up hosted engine
by Karli Sjöberg 08 Jan '18

08 Jan '18

1 0

bad bond name when setting up hosted engine
by Sam McLeod 08 Jan '18

08 Jan '18

--Apple-Mail=_0F0D4FDF-F7DC-4344-9507-87656F59712C Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii I'm having a problem where when setting up hosted engine deployment it = fails stating that the selected bond name is bad. "code=3D25, message=3Dbad bond name(s): mgmt)" - Is there a problem similar to = https://bugzilla.redhat.com/show_bug.cgi?id=3D1519807 = <https://bugzilla.redhat.com/show_bug.cgi?id=3D1519807> that's known? - If it seems to be this bug, is it preferred that I simply update the = existing, closed issue as I have done, or open a new bug? -- Sam McLeod https://smcleod.net https://twitter.com/s_mcleod --Apple-Mail=_0F0D4FDF-F7DC-4344-9507-87656F59712C Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; = charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><div = class=3D"">I'm having a problem where when setting up hosted engine = deployment it fails stating that the selected bond name is = bad.</div><div class=3D""><br class=3D""></div><div class=3D"">"code=3D25,= message=3Dbad bond name(s): mgmt)"</div><div class=3D""><br = class=3D""></div>- Is there a problem similar to <a = href=3D"https://bugzilla.redhat.com/show_bug.cgi?id=3D1519807" = class=3D"">https://bugzilla.redhat.com/show_bug.cgi?id=3D1519807</a> = that's known?<br class=3D""><div class=3D"">- If it seems to be this = bug, is it preferred that I simply update the existing, closed issue as = I have done, or open a new bug?</div><div class=3D""> <div dir=3D"auto" style=3D"color: rgb(0, 0, 0); letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: = break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" = class=3D""><div dir=3D"auto" style=3D"color: rgb(0, 0, 0); = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><div = dir=3D"auto" style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = line-break: after-white-space;" class=3D""><div style=3D"color: rgb(0, = 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; = font-variant-caps: normal; font-weight: normal; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><br = class=3D"">--<br class=3D"">Sam McLeod<br class=3D""><a = href=3D"https://smcleod.net" class=3D"">https://smcleod.net</a><br = class=3D"">https://twitter.com/s_mcleod</div></div></div></div> </div> <br class=3D""></body></html>= --Apple-Mail=_0F0D4FDF-F7DC-4344-9507-87656F59712C--

4 6