<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Thu, Sep 28, 2017 at 2:34 PM, Kevin Wolf <span dir="ltr"><<a href="mailto:kwolf@redhat.com" target="_blank">kwolf@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Am 28.09.2017 um 12:44 hat Nir Soffer geschrieben:<br>
<div><div class="gmail-h5">> On Thu, Sep 28, 2017 at 12:03 PM Gianluca Cecchi <<a href="mailto:gianluca.cecchi@gmail.com">gianluca.cecchi@gmail.com</a>><br>
> wrote:<br>
><br>
> > Hello,<br>
> > I'm on 4.1.5 and I'm cloning a snapshot of a VM with 3 disks for a total<br>
> > of about 200Gb to copy<br>
> > The target I choose is on a different domain than the source one.<br>
> > They are both FC storage domains, with the source on SSD disks and the<br>
> > target on SAS disks.<br></div></div></blockquote><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div class="gmail-h5">[snip]<br></div></div></blockquote><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div class="gmail-h5">
> ><br>
> > but despite capabilities it seems it is copying using very low system<br>
> > resources.<br>
> ><br>
><br>
> We run qemu-img convert (and other storage related commands) with:<br>
><br>
> nice -n 19 ionice -c 3 qemu-img ...<br>
><br>
> ionice should not have any effect unless you use the CFQ I/O scheduler.<br>
><br>
> The intent is to limit the effect of virtual machines.<br>
><br></div></div></blockquote><div><br></div><div>Ah, ok.</div><div>The hypervisor is ovirt node based on CentOS 7 so the default scheduler should be deadline if not customized in node.</div><div>And in fact in /sys/block/sd*/queue/scheduler I see only [deadline] contents and also for dm-* block devices where it is not none, it is deadline too</div><div> </div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div class="gmail-h5">
><br>
> > I see this both using iotop and vmstat<br>
> ><br>
> > vmstat 3 gives:<br>
> > ----io---- -system-- ------cpu-----<br>
> > bi bo in cs us sy id wa st<br>
> > 2527 698 3771 29394 1 0 89 10 0<br>
> ><br>
><br>
> us 94% also seems very high - maybe this hypervisor is overloaded with<br>
> other workloads?<br>
> wa 89% seems very high<br>
<br>
</div></div>The alignment in the table is a bit off, but us is 1%. The 94 you saw is<br>
part of cs=29394. A high percentage for wait is generally a good sign<br>
because that means that the system is busy with actual I/O work.<br>
Obviously, this I/O work is rather slow, but at least qemu-img is making<br>
requests to the kernel instead of doing other work, otherwise user would<br>
be much higher.<br>
<span class="gmail-HOEnZb"><font color="#888888"><br>
Kevin<br>
</font></span></blockquote></div><br></div><div class="gmail_extra"><br></div><div class="gmail_extra"><div>Yes, probably a misalignement of output I truncated. Actually a sampling of about 300 lines (once every 3 seconds) shows these number of lines occurrences and related percentage </div><div><br></div><div>user time</div><div><div> 195 0% </div><div> 95 1%</div></div><div><br></div><div>So user time is indeed quite low </div><div><br></div><div>wait time:</div><div><div> 105 7%</div><div> 58 8%</div><div> 33 9%</div><div> 21 6%</div><div> 17 10%</div><div> 16 5%</div><div> 16%</div><div> 12 11%<br></div><div> 9 12%</div><div> 7 14%</div><div> 6 13%</div><div> 2 15%</div><div> 1 4%</div><div> 1 16%</div><div> 1 0%<br></div></div><div><br></div><div>with wait time an average of 7%</div><div><br></div><div>AT the end the overall performance in copying has been around 30MB/s that probably is to be expected do to how the qemu-img process is run.</div><div>What about the events I reported instead?</div><div><br></div><div>Gianluca</div><div><br></div></div></div>