<div dir="ltr"><div>Hi all, <br><br></div><div>In case you need any further logs let me know. <br></div><div>Thanx for the time. <br><br></div><div>Alex<br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Feb 8, 2018 at 9:41 AM, Alex K <span dir="ltr"><<a href="mailto:rightkicktech@gmail.com" target="_blank">rightkicktech@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div>Hi Shani, <br><br></div>Didn't notice that. <br></div>I am attaching later vdsm logs. <br><br></div>Thanx, <br></div>Alex<br></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Feb 7, 2018 at 5:31 PM, Shani Leviim <span dir="ltr"><<a href="mailto:sleviim@redhat.com" target="_blank">sleviim@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small">Hi Alex,<br></div><div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small">Sorry for the mail's delay.<br><br></div><div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small">From a brief look at your logs, I've noticed that the error you've got at the engine's log was logged at 2018-02-03 00:22:56,<br></div><div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small">while your vdsm's log ends at 2018-02-03 00:01:01.<br></div><div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small">Is there a way you can reproduce a fuller vdsm log? <br></div></div><div class="gmail_extra"><br clear="all"><div><div class="m_5148315078321314582m_-6900038685846383369gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><span style="font-family:tahoma,sans-serif"><b>Regards,<br></b></span></div><span style="font-family:tahoma,sans-serif"><b>Shani Leviim<br></b></span></div></div></div></div></div><div><div class="m_5148315078321314582h5">
<br><div class="gmail_quote">On Sat, Feb 3, 2018 at 5:41 PM, Alex K <span dir="ltr"><<a href="mailto:rightkicktech@gmail.com" target="_blank">rightkicktech@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div>Attaching vdm log from host that trigerred the error, where the Vm that was being cloned was running at that time. <br><br></div>thanx, <br></div>Alex<br></div><div class="m_5148315078321314582m_-6900038685846383369HOEnZb"><div class="m_5148315078321314582m_-6900038685846383369h5"><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Feb 3, 2018 at 5:20 PM, Yaniv Kaul <span dir="ltr"><<a href="mailto:ykaul@redhat.com" target="_blank">ykaul@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto"><span><div><br><div class="gmail_extra"><br><div class="gmail_quote">On Feb 3, 2018 3:24 PM, "Alex K" <<a href="mailto:rightkicktech@gmail.com" target="_blank">rightkicktech@gmail.com</a>> wrote:<br type="attribution"><blockquote class="m_5148315078321314582m_-6900038685846383369m_276036620754789088m_-1762826299309603000quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div>Hi All, <br><br></div>I have reproduced the backups failure. The VM that failed is named Win-FileServer and is a Windows 2016 server 64bit with 300GB of disk. <br>During the cloning step the VM went unresponsive and I had to stop/start it. <br>I am attaching the logs.I have another VM with same OS (named DC-Server within the logs) but with smaller disk (60GB) which does not give any error when it is cloned.<br></div><div>I see a line: <br><br>EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,<wbr>802), Correlation ID: null, Call Stack: null, Custom ID: null, Custom Event ID: -1, Message: VDSM v2.sitedomain command SnapshotVDS failed: Message timeout which can be caused by communication issues<br></div></div></div></div></blockquote></div></div></div><div dir="auto"><br></div></span><div dir="auto">I suggest adding relevant vdsm.log as well. </div><div dir="auto">Y. </div><div dir="auto"><br></div><div dir="auto"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="m_5148315078321314582m_-6900038685846383369m_276036620754789088m_-1762826299309603000quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="m_5148315078321314582m_-6900038685846383369m_276036620754789088h5"><div dir="ltr"><div><div><div> <br></div>I appreciate any advise why I am facing such issue with the backups. <br><br></div>thanx, <br></div>Alex<br></div><div class="m_5148315078321314582m_-6900038685846383369m_276036620754789088m_-1762826299309603000elided-text"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jan 30, 2018 at 12:49 AM, Alex K <span dir="ltr"><<a href="mailto:rightkicktech@gmail.com" target="_blank">rightkicktech@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto">Ok. I will reproduce and collect logs.<div dir="auto"><br></div><div dir="auto">Thanx,</div><div dir="auto">Alex</div></div><div class="m_5148315078321314582m_-6900038685846383369m_276036620754789088m_-1762826299309603000m_-732652084987950504HOEnZb"><div class="m_5148315078321314582m_-6900038685846383369m_276036620754789088m_-1762826299309603000m_-732652084987950504h5"><div class="gmail_extra"><br><div class="gmail_quote">On Jan 29, 2018 20:21, "Mahdi Adnan" <<a href="mailto:mahdi.adnan@outlook.com" target="_blank">mahdi.adnan@outlook.com</a>> wrote:<br type="attribution"><blockquote class="m_5148315078321314582m_-6900038685846383369m_276036620754789088m_-1762826299309603000m_-732652084987950504m_3731284340350880423quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgba(0,0,0,0)">
I have Windows VMs, both client and server.</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgba(0,0,0,0)">
if you provide the engine.log file we might have a look at it.</div><div class="m_5148315078321314582m_-6900038685846383369m_276036620754789088m_-1762826299309603000m_-732652084987950504m_3731284340350880423quoted-text">
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div id="m_5148315078321314582m_-6900038685846383369m_276036620754789088m_-1762826299309603000m_-732652084987950504m_3731284340350880423m_-8235018946273683167signature"><br>
<div class="m_5148315078321314582m_-6900038685846383369m_276036620754789088m_-1762826299309603000m_-732652084987950504m_3731284340350880423m_-8235018946273683167ecxmoz-signature">-- <br>
<br>
<font color="#3366ff"><font color="#000000">Respectfully<b><br>
</b><b>Mahdi A. Mahdi</b></font></font><font color="#3366ff"><br>
<br>
</font><font color="#3366ff"></font></div>
</div>
<hr style="display:inline-block;width:98%">
</div><div id="m_5148315078321314582m_-6900038685846383369m_276036620754789088m_-1762826299309603000m_-732652084987950504m_3731284340350880423m_-8235018946273683167divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" color="#000000" face="Calibri, sans-serif"><b>From:</b> Alex K <<a href="mailto:rightkicktech@gmail.com" target="_blank">rightkicktech@gmail.com</a>><br>
<b>Sent:</b> Monday, January 29, 2018 5:40 PM<br>
<b>To:</b> Mahdi Adnan<br>
<b>Cc:</b> users<br>
<b>Subject:</b> Re: [ovirt-users] Ovirt backups lead to unresponsive VM</font>
<div> </div>
</div><div class="m_5148315078321314582m_-6900038685846383369m_276036620754789088m_-1762826299309603000m_-732652084987950504m_3731284340350880423elided-text">
<div>
<div dir="ltr">
<div>
<div>
<div>
<div>Hi, <br>
</div>
<div><br>
</div>
<div>I have observed this logged at host when the issue occurs: <br>
</div>
<div><br>
</div>
<div>VDSM command GetStoragePoolInfoVDS failed: Connection reset by peer</div>
<div><br>
</div>
<div>or <br>
</div>
<div><br>
</div>
<div>VDSM host.domain command GetStatsVDS failed: Connection reset by peer<br>
</div>
<div><br>
</div>
<div>At engine logs have not been able to correlate. <br>
</div>
<div><br>
</div>
Are you hosting Windows 2016 server and Windows 10 VMs?<br>
</div>
The weird is that I have same setup on other clusters with no issues. <br>
<br>
</div>
Thanx, <br>
</div>
Alex<br>
</div>
<div class="m_5148315078321314582m_-6900038685846383369m_276036620754789088m_-1762826299309603000m_-732652084987950504m_3731284340350880423m_-8235018946273683167x_gmail_extra"><br>
<div class="m_5148315078321314582m_-6900038685846383369m_276036620754789088m_-1762826299309603000m_-732652084987950504m_3731284340350880423m_-8235018946273683167x_gmail_quote">On Sun, Jan 28, 2018 at 9:21 PM, Mahdi Adnan <span dir="ltr">
<<a href="mailto:mahdi.adnan@outlook.com" target="_blank">mahdi.adnan@outlook.com</a>></span> wrote:<br>
<blockquote class="m_5148315078321314582m_-6900038685846383369m_276036620754789088m_-1762826299309603000m_-732652084987950504m_3731284340350880423m_-8235018946273683167x_gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgba(0,0,0,0)">
Hi,</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgba(0,0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgba(0,0,0,0)">
We have a cluster of 17 nodes, backed by GlusterFS storage, and using this same script for backup.</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgba(0,0,0,0)">
we have no issues with it so far.</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgba(0,0,0,0)">
have you checked engine log file ?</div>
<span class="m_5148315078321314582m_-6900038685846383369m_276036620754789088m_-1762826299309603000m_-732652084987950504m_3731284340350880423m_-8235018946273683167x_HOEnZb"><font color="#888888">
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div id="m_5148315078321314582m_-6900038685846383369m_276036620754789088m_-1762826299309603000m_-732652084987950504m_3731284340350880423m_-8235018946273683167x_m_2991573719749610803signature"><br>
<div class="m_5148315078321314582m_-6900038685846383369m_276036620754789088m_-1762826299309603000m_-732652084987950504m_3731284340350880423m_-8235018946273683167x_m_2991573719749610803ecxmoz-signature">-- <br>
<br>
<font color="#3366ff"><font color="#000000">Respectfully<b><br>
</b><b>Mahdi A. Mahdi</b></font></font><font color="#3366ff"><br>
<br>
</font><font color="#3366ff"></font></div>
</div>
<hr style="display:inline-block;width:98%">
<div id="m_5148315078321314582m_-6900038685846383369m_276036620754789088m_-1762826299309603000m_-732652084987950504m_3731284340350880423m_-8235018946273683167x_m_2991573719749610803divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" color="#000000" face="Calibri, sans-serif"><b>From:</b>
<a href="mailto:users-bounces@ovirt.org" target="_blank">users-bounces@ovirt.org</a> <<a href="mailto:users-bounces@ovirt.org" target="_blank">users-bounces@ovirt.org</a>> on behalf of Alex K <<a href="mailto:rightkicktech@gmail.com" target="_blank">rightkicktech@gmail.com</a>><br>
<b>Sent:</b> Wednesday, January 24, 2018 4:18 PM<br>
<b>To:</b> users<br>
<b>Subject:</b> [ovirt-users] Ovirt backups lead to unresponsive VM</font>
<div> </div>
</div>
</font></span>
<div>
<div class="m_5148315078321314582m_-6900038685846383369m_276036620754789088m_-1762826299309603000m_-732652084987950504m_3731284340350880423m_-8235018946273683167x_h5">
<div>
<div dir="ltr">
<div>
<div>
<div>Hi all, <br>
<br>
</div>
I have a cluster with 3 nodes, using ovirt 4.1 in a self hosted setup on top glusterfs. </div>
On some VMs (especially one Windows server 2016 64bit with 500 GB of disk). Guest agents are installed at VMs. i almost always observe that during the backup of the VM the VM is rendered unresponsive (dashboard shows a question mark at the VM status and VM
does not respond to ping or to anything). <br>
<br>
</div>
For scheduled backups I use:
<p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2"><a href="https://github.com/wefixit-AT/oVirtBackup" target="_blank">https://github.com/wefixit-AT/<wbr>oVirtBackup</a></font></font></p>
<p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2">The script does the following:<br>
</font></font></p>
<p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2">1. snapshot VM (this is done ok without any failure)<br>
</font></font></p>
<p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2">2. Clone snapshot (this steps renders the VM unresponsive)<br>
</font></font></p>
<p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2">3. Export Clone</font></font></p>
<p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2">4. Delete clone</font></font></p>
<p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2">5. Delete snapshot</font></font></p>
<p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2"><br>
</font></font></p>
<p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2">Do you have any similar experience? Any suggestions to address this?</font></font></p>
<p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2">I have never seen such issue with hosted Linux VMs.<br>
</font></font></p>
<p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2">The cluster has enough storage to accommodate the clone.
<br>
</font></font></p>
<p style="margin-bottom:0in;line-height:100%"><br>
</p>
<p style="margin-bottom:0in;line-height:100%">Thanx, <br>
</p>
<p style="margin-bottom:0in;line-height:100%">Alex<br>
</p>
<p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2"><b></b></font></font></p>
<p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2"><b><br>
</b></font></font></p>
<p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2"><b><br>
</b></font></font></p>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div><br></div></div>______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/users</a><br>
<br></blockquote></div><br></div></div></div>
</blockquote></div><br></div>
</div></div><br>______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/users</a><br>
<br></blockquote></div><br></div></div></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>