<div dir="ltr"><div><div>Thank you Nir for the below. <br><br></div>I am putting some comments inline in blue. <br><br></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Feb 13, 2018 at 7:33 PM, Nir Soffer <span dir="ltr"><<a href="mailto:nsoffer@redhat.com" target="_blank">nsoffer@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><span class=""><div dir="ltr">On Wed, Jan 24, 2018 at 3:19 PM Alex K <<a href="mailto:rightkicktech@gmail.com" target="_blank">rightkicktech@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div>Hi all, <br><br></div>I have a cluster with 3 nodes, using ovirt 4.1 in a self hosted setup on top glusterfs. </div>On some VMs (especially one Windows server 2016 64bit with 500 GB of disk). Guest agents are installed at VMs. i almost always observe that during the backup of the VM the VM is rendered unresponsive (dashboard shows a question mark at the VM status and VM does not respond to ping or to anything). <br><br></div>For scheduled backups I use:
<p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2"><a href="https://github.com/wefixit-AT/oVirtBackup" target="_blank">https://github.com/wefixit-AT/<wbr>oVirtBackup</a></font></font></p><p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2">The script does the following:<br></font></font></p><p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2">1. snapshot VM (this is done ok without any failure)<br></font></font></p></div></blockquote><div><br></div></span><div>This is a very cheap operation</div><span class=""><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2"></font></font></p><p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2">2. Clone snapshot (this steps renders the VM unresponsive)<br></font></font></p></div></blockquote><div><br></div></span><div>This copy 500g of data. In gluster case, it copies 1500g of data, since in glusterfs, the client</div><div>is doing the replication.</div><div><br></div><div>Maybe your network or gluster server is too slow? Can you describe the network topology?</div><div><br></div><div>Please attach also the volume info for the gluster volume, maybe it is not configured in the </div><div>best way?</div></div></div></blockquote><div><br></div><div><span style="color:rgb(0,0,255)">The network is 1Gbit. The hosts (3 hosts) are decent ones and new hardware with each host having: 32GB RAM, 16 CPU cores and 2 TB of storage in RAID10. <br></span></div><div><span style="color:rgb(0,0,255)">The VMS hosted (7 VMs) exhibit high performance. The VMs are Windows 2016 and Windows10. <br></span></div><div><span style="color:rgb(0,0,255)">The network topology is: two networks defined at ovirt: ovirtmgmt is for the managment and access network and "storage" is a separate network, where each server is connected with two network cables at a managed switch with mode 6 load balancing. this storage network is used for gluster traffic. Attached the volume configuration. </span><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2"></font></font></p><p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2">3. Export Clone</font></font></p></div></blockquote><div><br></div><div>This copy 500g to the export domain. If the export domain is on glusterfs as well, you</div><div>copy now another 1500g of data.</div><span class=""><div> </div></span></div></div></blockquote><div><span style="color:rgb(0,0,255)">Export domain a Synology NAS with NFS share. If the cloning succeeds then export is completed ok. </span><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><span class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2">4. Delete clone</font></font></p><p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2">5. Delete snapshot</font></font></p></div></blockquote><div><br></div></span><div>Not clear why do you need to clone the vm before you export it, you can save half of </div><div>the data copies.<br></div></div></div></blockquote><div><span style="color:rgb(0,0,255)">Because I cannot export the VM while it is running. It does not provide such option. </span><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div></div><div><br></div><div>If you 4.2, you can backup the vm *while the vm is running* by:</div><div>- Take a snapshot</div><div>- Get the vm ovf from the engine api</div><div>- Download the vm disks using ovirt-imageio and store the snaphosts in your backup</div><div> storage</div><div>- Delete a snapshot</div><div><br></div><div>In this flow, you would copy 500g.</div><div><br></div></div></div></blockquote><div><span style="color:rgb(0,0,255)">I am not aware about this option. checking quickly at site this seems that it is still half implemented? Is there any script that I may use and test this? I am interested to have these backups scheduled. </span><br> <br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div></div><div><div>Daniel, please correct me if I'm wrong regarding doing this online.</div></div><div><br></div><div>Regardless, a vm should not become non-responsive while cloning. Please file a bug</div><div>for this and attach engine, vdsm, and glusterfs logs.</div><span class="HOEnZb"><font color="#888888"><div> <br></div></font></span></div></div></blockquote><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><span class="HOEnZb"><font color="#888888"><div></div><div>Nir</div><div><br></div></font></span><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class=""><div dir="ltr"><p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2">Do you have any similar experience? Any suggestions to address this?</font></font></p><p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2">I have never seen such issue with hosted Linux VMs.<br></font></font></p><p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2">The cluster has enough storage to accommodate the clone. <br></font></font></p><p style="margin-bottom:0in;line-height:100%"><br></p><p style="margin-bottom:0in;line-height:100%">Thanx, <br></p><p style="margin-bottom:0in;line-height:100%">Alex<br></p><p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2"><b></b></font></font></p><p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2"><b><br></b></font></font></p><p style="margin-bottom:0in;line-height:100%"><font face="Courier 10 Pitch"><font style="font-size:10pt" size="2"><b><br></b></font></font></p>
</div></span><span class="">
______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/<wbr>mailman/listinfo/users</a><br>
</span></blockquote></div></div>
</blockquote></div><br></div></div>