<html><head></head><body><div style="color:#000; background-color:#fff; font-family:Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:13px"><div id="yiv6639019017"><div id="yui_3_16_0_ym19_1_1499051725992_6544">
Hi Yaniv,<div id="yui_3_16_0_ym19_1_1499051725992_6543"><br clear="none"></div><div id="yui_3_16_0_ym19_1_1499051725992_6566">Thanks for your reply. Apologies for my late reply we had a long holiday here. </div><div id="yui_3_16_0_ym19_1_1499051725992_6565"><br clear="none"></div><div id="yui_3_16_0_ym19_1_1499051725992_6564">To answer you:</div><div id="yui_3_16_0_ym19_1_1499051725992_6563"><br clear="none"></div><div id="yui_3_16_0_ym19_1_1499051725992_6557">Yes the guest VM become completely frozen and non responsive as soon as its disk has any activity for example when we shutdown or do a yum update. </div><div id="yui_3_16_0_ym19_1_1499051725992_6556"><br clear="none"></div><div id="yui_3_16_0_ym19_1_1499051725992_6555"><br clear="none"></div><div id="yui_3_16_0_ym19_1_1499051725992_6546" dir="ltr"><div style="" id="yui_3_16_0_ym19_1_1499051725992_6554"><span style="" id="yui_3_16_0_ym19_1_1499051725992_6553">Versions of all the components involved - guest OS, host OS (qemu-kvm version), how do you run the VM (vdsm log would be helpful here), exact storage specification (1Gb or 10Gb link? What is the NFS version? What is it hosted on? etc.)</span></div><div style="" id="yui_3_16_0_ym19_1_1499051725992_6552"><span style=""> Y.</span></div><div style="" id="yui_3_16_0_ym19_1_1499051725992_6551"><span style=""><br clear="none"></span></div><div style="" id="yui_3_16_0_ym19_1_1499051725992_6551"><span style="" id="yui_3_16_0_ym19_1_1499051725992_9556">Some facts about our environment:</span></div><div style="" id="yui_3_16_0_ym19_1_1499051725992_6551"><span style=""><br></span></div><div style="" id="yui_3_16_0_ym19_1_1499051725992_6551"><span style="" id="yui_3_16_0_ym19_1_1499051725992_9703">1) Previously, this environment was using XEN using raw disk and we change it to Ovirt (Ovirt were able to read the VMs's disks without any conversion.) </span></div><div style="" id="yui_3_16_0_ym19_1_1499051725992_6551"><span style="" id="yui_3_16_0_ym19_1_1499051725992_9800">2) The issue we are facing is not happening for any of the existing VMs. </span></div><div style="" id="yui_3_16_0_ym19_1_1499051725992_6551"><span style="" id="yui_3_16_0_ym19_1_1499051725992_9686"><b id="yui_3_16_0_ym19_1_1499051725992_9685">3) This issue only happens for new VMs.</b></span></div><div style="" id="yui_3_16_0_ym19_1_1499051725992_6551"><span style="">4) G</span>uest (kernel v3.10) and host(kernel v4.1) OSes are both CentOS 7 minimal installation. </div><div style="" id="yui_3_16_0_ym19_1_1499051725992_6551"><b id="yui_3_16_0_ym19_1_1499051725992_9591">5) NFS version 4</b> and Using Ovirt 4.1</div><div style="" id="yui_3_16_0_ym19_1_1499051725992_6550"><span style="">6) </span><b>The network speed is 1 GB.</b></div><div style="" id="yui_3_16_0_ym19_1_1499051725992_6550"><span style="">7) </span>The output for rpm -qa | grep qemu-kvm shows:</div><div style="" id="yui_3_16_0_ym19_1_1499051725992_6550"><b id="yui_3_16_0_ym19_1_1499051725992_9222"> qemu-kvm-common-ev-2.6.0-28.e17_3.6.1.x86_64</b><br></div><div style="" id="yui_3_16_0_ym19_1_1499051725992_6550" dir="ltr"><b id="yui_3_16_0_ym19_1_1499051725992_9360"><span style="" id="yui_3_16_0_ym19_1_1499051725992_8120"> qemu-kvm-tools-ev-</span>2.6.0-28.e17_3.6.1.x86_64</b></div><div style="" id="yui_3_16_0_ym19_1_1499051725992_6550" dir="ltr"><b id="yui_3_16_0_ym19_1_1499051725992_9359"> qemu-kvm-ev-2.6.0-28.e17_3.6.1.x86_64</b></div><div style="" id="yui_3_16_0_ym19_1_1499051725992_6550" dir="ltr"><b>8) </b>The storage is from a SAN device which is connected to the NFS server using fiber channel.</div><div style="" id="yui_3_16_0_ym19_1_1499051725992_6548"><span style=""><br clear="none"></span></div><div style="" id="yui_3_16_0_ym19_1_1499051725992_6548"><span style="" id="yui_3_16_0_ym19_1_1499051725992_7076">So for example during shutdown also it froze and shows something like this in event section:</span></div><div style="" id="yui_3_16_0_ym19_1_1499051725992_6548"><br></div><div style="" id="yui_3_16_0_ym19_1_1499051725992_6548"><b id="yui_3_16_0_ym19_1_1499051725992_9195">VM ILMU_WEB has been paused due to storage I/O problem.</b></div><div style="" id="yui_3_16_0_ym19_1_1499051725992_6548"><br></div><div style="" id="yui_3_16_0_ym19_1_1499051725992_6548"><br></div>More information:</div><div id="yui_3_16_0_ym19_1_1499051725992_6546" dir="ltr"><br></div><div id="yui_3_16_0_ym19_1_1499051725992_6546" dir="ltr">VDSM log at the time of this issue (The issue happened at Jul 3, 2017 9:50:43 AM):</div><div id="yui_3_16_0_ym19_1_1499051725992_6546" dir="ltr"><br></div><div id="yui_3_16_0_ym19_1_1499051725992_6546" dir="ltr"><div dir="ltr" id="yui_3_16_0_ym19_1_1499051725992_8436"><pre style="user-select: text; font-family: "Courier New", Courier, monospace, arial, sans-serif; font-size: 14px; margin-top: 0px; margin-bottom: 0px; word-wrap: break-word;" id="yui_3_16_0_ym19_1_1499051725992_8875">2017-07-03 09:50:37,113+0800 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:515)
2017-07-03 09:50:37,897+0800 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmIoTunePolicies succeeded in 0.02 seconds (__init__:515)
2017-07-03 09:50:42,510+0800 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.00 seconds (__init__:515)
<b id="yui_3_16_0_ym19_1_1499051725992_9241">2017-07-03 09:50:43,548+0800 INFO (jsonrpc/3) [dispatcher] Run and protect: repoStats(options=None) (logUtils:51)
2017-07-03 09:50:43,548+0800 INFO (jsonrpc/3) [dispatcher] Run and protect: repoStats, Return response: {u'e01186c1-7e44-4808-b551-4722f0f8e84b': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000144822', 'lastCheck': '8.9', 'valid': True}, u'721b5233-b0ba-4722-8a7d-ba2a372190a0': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000327909', 'lastCheck': '8.9', 'valid': True}, u'94775bd3-3244-45b4-8a06-37eff8856afa': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000256425', 'lastCheck': '8.9', 'valid': True}, u'731bb771-5b73-4b5c-ac46-56499df97721': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000238159', 'lastCheck': '8.9', 'valid': True}, u'f620781f-93d4-4410-8697-eb41045cacd6': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.00022004', 'lastCheck': '8.9', 'valid': True}, u'a1a7d0a4-e3b6-4bd5-862b-96e70dae3f29': {'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay': '0.000298581', 'lastCheck': '8.8', 'valid': True}} (logUtils:54)
</b>2017-07-03 09:50:43,563+0800 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call Host.getStats succeeded in 0.01 seconds (__init__:515)
2017-07-03 09:50:46,737+0800 INFO (periodic/3) [dispatcher] Run and protect: getVolumeSize(sdUUID=u'721b5233-b0ba-4722-8a7d-ba2a372190a0', spUUID=u'b04ca6e4-2660-4eaa-acdb-c1dae4e21f2d', imgUUID=u'3c26476e-1dae-44d7-9208-531b91ae5ae1', volUUID=u'a7e789fb-6646-4d0a-9b51-f5ab8242c8d5', options=None) (logUtils:51)
2017-07-03 09:50:46,738+0800 INFO (periodic/0) [dispatcher] Run and protect: getVolumeSize(sdUUID=u'f620781f-93d4-4410-8697-eb41045cacd6', spUUID=u'b04ca6e4-2660-4eaa-acdb-c1dae4e21f2d', imgUUID=u'2158fdae-54e1-413d-a844-73da5d1bb4ca', volUUID=u'6ee0b0eb-0bba-4e18-9c00-c1539b632e8a', options=None) (logUtils:51)
2017-07-03 09:50:46,740+0800 INFO (periodic/2) [dispatcher] Run and protect: getVolumeSize(sdUUID=u'f620781f-93d4-4410-8697-eb41045cacd6', spUUID=u'b04ca6e4-2660-4eaa-acdb-c1dae4e21f2d', imgUUID=u'a967016d-a56b-41e8-b7a2-57903cbd2825', volUUID=u'784514cb-2b33-431c-b193-045f23c596d8', options=None) (logUtils:51)
2017-07-03 09:50:46,741+0800 INFO (periodic/1) [dispatcher] Run and protect: getVolumeSize(sdUUID=u'721b5233-b0ba-4722-8a7d-ba2a372190a0', spUUID=u'b04ca6e4-2660-4eaa-acdb-c1dae4e21f2d', imgUUID=u'bb35c163-f068-4f08-a1c2-28c4cb1b76d9', volUUID=u'fce7e0a0-7411-4d8c-b72c-2f46c4b4db1e', options=None) (logUtils:51)
2017-07-03 09:50:46,743+0800 INFO (periodic/0) [dispatcher] Run and protect: getVolumeSize, Return response: {'truesize': '6361276416', 'apparentsize': '107374182400'} (logUtils:54)</pre><pre style="user-select: text; font-family: "Courier New", Courier, monospace, arial, sans-serif; font-size: 14px; margin-top: 0px; margin-bottom: 0px; word-wrap: break-word;" id="yui_3_16_0_ym19_1_1499051725992_8875"><br></pre><pre style="user-select: text; font-family: "Courier New", Courier, monospace, arial, sans-serif; font-size: 14px; margin-top: 0px; margin-bottom: 0px; word-wrap: break-word;" id="yui_3_16_0_ym19_1_1499051725992_8875">......</pre><pre style="user-select: text; font-family: "Courier New", Courier, monospace, arial, sans-serif; font-size: 14px; margin-top: 0px; margin-bottom: 0px; word-wrap: break-word;" id="yui_3_16_0_ym19_1_1499051725992_8875">......</pre><pre style="user-select: text; font-family: "Courier New", Courier, monospace, arial, sans-serif; font-size: 14px; margin-top: 0px; margin-bottom: 0px; word-wrap: break-word;" id="yui_3_16_0_ym19_1_1499051725992_8875"><br></pre><pre style="user-select: text; font-family: "Courier New", Courier, monospace, arial, sans-serif; font-size: 14px; margin-top: 0px; margin-bottom: 0px; word-wrap: break-word;" id="yui_3_16_0_ym19_1_1499051725992_8875"><pre style="user-select: text; font-family: "Courier New", Courier, monospace, arial, sans-serif; margin-top: 0px; margin-bottom: 0px; word-wrap: break-word;" id="yui_3_16_0_ym19_1_1499051725992_9311"><b id="yui_3_16_0_ym19_1_1499051725992_9341">2017-07-03 09:52:16,941+0800 INFO (libvirt/events) [virt.vm] (vmId='c84f519e-398d-40a3-85b2-b7e53f3d7f67') abnormal vm stop device scsi0-0-0-0 error eio (vm:4112)
2017-07-03 09:52:16,941+0800 INFO (libvirt/events) [virt.vm] (vmId='c84f519e-398d-40a3-85b2-b7e53f3d7f67') CPU stopped: onIOError (vm:4997)
2017-07-03 09:52:16,942+0800 INFO (libvirt/events) [virt.vm] (vmId='c84f519e-398d-40a3-85b2-b7e53f3d7f67') CPU stopped: onSuspend (vm:4997)
2017-07-03 09:52:16,942+0800 INFO (libvirt/events) [virt.vm] (vmId='c84f519e-398d-40a3-85b2-b7e53f3d7f67') abnormal vm stop device scsi0-0-0-0 error eio (vm:4112)
2017-07-03 09:52:16,943+0800 INFO (libvirt/events) [virt.vm] (vmId='c84f519e-398d-40a3-85b2-b7e53f3d7f67') CPU stopped: onIOError (vm:4997)
2017-07-03 09:52:16,943+0800 INFO (libvirt/events) [virt.vm] (vmId='c84f519e-398d-40a3-85b2-b7e53f3d7f67') abnormal vm stop device scsi0-0-0-0 error eio (vm:4112)
2017-07-03 09:52:16,944+0800 INFO (libvirt/events) [virt.vm] (vmId='c84f519e-398d-40a3-85b2-b7e53f3d7f67') CPU stopped: onIOError</b></pre><pre style="user-select: text; font-family: "Courier New", Courier, monospace, arial, sans-serif; margin-top: 0px; margin-bottom: 0px; word-wrap: break-word;" id="yui_3_16_0_ym19_1_1499051725992_9311"><b><br></b></pre><pre style="user-select: text; font-family: "Courier New", Courier, monospace, arial, sans-serif; margin-top: 0px; margin-bottom: 0px; word-wrap: break-word;" id="yui_3_16_0_ym19_1_1499051725992_9311"><b><br></b></pre><pre style="user-select: text; font-family: "Courier New", Courier, monospace, arial, sans-serif; margin-top: 0px; margin-bottom: 0px; word-wrap: break-word;" id="yui_3_16_0_ym19_1_1499051725992_9311"><b><br></b></pre></pre></div><div class="qtdSeparateBR"><br><br></div><div class="yiv6639019017yqt2677119843" id="yiv6639019017yqtfd82856"><div class="yiv6639019017yahoo-quoted-begin" style="font-size:15px;color:#715FFA;padding-top:15px;margin-top:0;" id="yui_3_16_0_ym19_1_1499051725992_6545"><b id="yui_3_16_0_ym19_1_1499051725992_9344">On Thursday, June 22, 2</b>017, 2:48 PM, Yaniv Kaul <ykaul@redhat.com> wrote:</div><blockquote class="yiv6639019017iosymail" id="yui_3_16_0_ym19_1_1499051725992_6642"><div id="yiv6639019017"><div id="yui_3_16_0_ym19_1_1499051725992_6641"><div dir="ltr" id="yui_3_16_0_ym19_1_1499051725992_6640"><br clear="none"><div class="yiv6639019017gmail_extra" id="yui_3_16_0_ym19_1_1499051725992_6643"><br clear="none"><div class="yiv6639019017gmail_quote" id="yui_3_16_0_ym19_1_1499051725992_6644">On Thu, Jun 22, 2017 at 5:07 AM, M Mahboubian <span dir="ltr"><<a rel="nofollow" shape="rect" ymailto="mailto:m_mahboubian@yahoo.com" target="_blank" href="mailto:m_mahboubian@yahoo.com">m_mahboubian@yahoo.com</a>></span> wrote:<br clear="none"><blockquote class="yiv6639019017gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex;" id="yui_3_16_0_ym19_1_1499051725992_6646"><div id="yui_3_16_0_ym19_1_1499051725992_6645">
Dear all,<div id="yui_3_16_0_ym19_1_1499051725992_6647">I appreciate if anybody could possibly help with the issue I am facing.</div><div id="yui_3_16_0_ym19_1_1499051725992_6648"><br clear="none"></div><div id="yui_3_16_0_ym19_1_1499051725992_6649">In our environment we have 2 hosts 1 NFS server and 1 ovirt engine server. The NFS server provides storage to the VMs in the hosts.</div><div id="yui_3_16_0_ym19_1_1499051725992_6650"><br clear="none"></div><div id="yui_3_16_0_ym19_1_1499051725992_6651">I can create new VMs and install os but once i do something like yum update the VM freezes. I can reproduce this every single time I do yum update.</div></div></blockquote><div id="yui_3_16_0_ym19_1_1499051725992_6652"><br clear="none"></div><div id="yui_3_16_0_ym19_1_1499051725992_6653">Is it paused, or completely frozen?</div><div id="yui_3_16_0_ym19_1_1499051725992_6654"> <br clear="none"></div><blockquote class="yiv6639019017gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex;" id="yui_3_16_0_ym19_1_1499051725992_9059"><div id="yui_3_16_0_ym19_1_1499051725992_9058"><div id="yui_3_16_0_ym19_1_1499051725992_9057"><br clear="none"></div><div id="yui_3_16_0_ym19_1_1499051725992_9791">what information/log files should I provide you to trubleshoot this?</div></div></blockquote><div><br clear="none"></div><div id="yui_3_16_0_ym19_1_1499051725992_9834">Versions of all the components involved - guest OS, host OS (qemu-kvm version), how do you run the VM (vdsm log would be helpful here), exact storage specification (1Gb or 10Gb link? What is the NFS version? What is it hosted on? etc.)</div><div> Y.</div><div class="yiv6639019017yqt6606837936" id="yiv6639019017yqtfd75418"><div><br clear="none"></div></div><blockquote class="yiv6639019017gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex;" id="yui_3_16_0_ym19_1_1499051725992_9794"><div id="yui_3_16_0_ym19_1_1499051725992_9793"><div class="yiv6639019017yqt6606837936" id="yiv6639019017yqtfd33806"><div id="yui_3_16_0_ym19_1_1499051725992_9792"><br clear="none"></div><div> Regards</div></div>
</div><br clear="none">______________________________ _________________<br clear="none">
Users mailing list<br clear="none">
<a rel="nofollow" shape="rect" ymailto="mailto:Users@ovirt.org" target="_blank" href="mailto:Users@ovirt.org">Users@ovirt.org</a><br clear="none">
<a rel="nofollow" shape="rect" target="_blank" href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/ mailman/listinfo/users</a><div class="yiv6639019017yqt6606837936" id="yiv6639019017yqtfd65242"><br clear="none">
<br clear="none"></div></blockquote></div><div class="yiv6639019017yqt6606837936" id="yiv6639019017yqtfd02968"><br clear="none"></div></div></div></div></div><blockquote></blockquote></blockquote></div></div><div class="yiv6639019017yqt2677119843" id="yiv6639019017yqtfd23176">
</div></div></div></div></body></html>