<html><head></head><body><div>I added log snippets from when it fails to the bug entry and how the volume is setup.</div><div><br></div><div><br></div><div>/Johan</div><div><br></div><div>On Mon, 2017-07-31 at 10:23 +0300, Benny Zlotnik wrote:</div><blockquote type="cite"><div dir="ltr"><div>Forgot to add there is a bug for this issue[1] <br></div><div>Please add your gluster mount and brick logs to the bug entry</div><div><br></div><div class="gmail_extra">[1] - <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1458846">https://bugzilla.redhat.com/show_bug.cgi?id=1458846</a></div><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Jul 30, 2017 at 3:02 PM, Johan Bernhardsson <span dir="ltr"><<a href="mailto:johan@kafit.se" target="_blank">johan@kafit.se</a>></span> wrote:<br><blockquote type="cite"><div><div><div class="gmail-m_3522047505523402941row" style="box-sizing:border-box;color:rgb(0,0,0);font-family:"Arial Unicode MS",Arial,sans-serif;font-size:small"><div class="gmail-m_3522047505523402941col-md-12" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:1681px"><div class="gmail-m_3522047505523402941row" style="box-sizing:border-box"><div class="gmail-m_3522047505523402941col-md-2" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:280.156px"><div class="gmail-m_3522047505523402941GKIIXFICNAD" id="gmail-m_3522047505523402941SubTabHostGeneralSoftwareView_formPanel_col0_row0_label" style="box-sizing:border-box;font-family:Arial,sans-serif">OS Version:</div></div><div class="gmail-m_3522047505523402941col-md-10" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:1400.83px"><span id="gmail-m_3522047505523402941SubTabHostGeneralSoftwareView_formPanel_col0_row0_value" class="gmail-m_3522047505523402941GKIIXFICOAD" style="box-sizing:border-box;color:rgb(119,119,119);font-family:Arial,sans-serif;word-wrap:break-word">RHEL - 7 - 3.1611.el7.centos</span></div></div></div></div><div class="gmail-m_3522047505523402941row" style="box-sizing:border-box;color:rgb(0,0,0);font-family:"Arial Unicode MS",Arial,sans-serif;font-size:small"><div class="gmail-m_3522047505523402941col-md-12" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:1681px"><div class="gmail-m_3522047505523402941row" style="box-sizing:border-box"><div class="gmail-m_3522047505523402941col-md-2" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:280.156px"><div class="gmail-m_3522047505523402941GKIIXFICNAD" id="gmail-m_3522047505523402941SubTabHostGeneralSoftwareView_formPanel_col0_row1_label" style="box-sizing:border-box;font-family:Arial,sans-serif">OS Description:</div></div><div class="gmail-m_3522047505523402941col-md-10" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:1400.83px"><span id="gmail-m_3522047505523402941SubTabHostGeneralSoftwareView_formPanel_col0_row1_value" class="gmail-m_3522047505523402941GKIIXFICOAD" style="box-sizing:border-box;color:rgb(119,119,119);font-family:Arial,sans-serif;word-wrap:break-word">CentOS Linux 7 (Core)</span></div></div></div></div><div class="gmail-m_3522047505523402941row" style="box-sizing:border-box;color:rgb(0,0,0);font-family:"Arial Unicode MS",Arial,sans-serif;font-size:small"><div class="gmail-m_3522047505523402941col-md-12" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:1681px"><div class="gmail-m_3522047505523402941row" style="box-sizing:border-box"><div class="gmail-m_3522047505523402941col-md-2" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:280.156px"><div class="gmail-m_3522047505523402941GKIIXFICNAD" id="gmail-m_3522047505523402941SubTabHostGeneralSoftwareView_formPanel_col0_row2_label" style="box-sizing:border-box;font-family:Arial,sans-serif">Kernel Version:</div></div><div class="gmail-m_3522047505523402941col-md-10" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:1400.83px"><span id="gmail-m_3522047505523402941SubTabHostGeneralSoftwareView_formPanel_col0_row2_value" class="gmail-m_3522047505523402941GKIIXFICOAD" style="box-sizing:border-box;color:rgb(119,119,119);font-family:Arial,sans-serif;word-wrap:break-word">3.10.0 - 514.16.1.el7.x86_64</span></div></div></div></div><div class="gmail-m_3522047505523402941row" style="box-sizing:border-box;color:rgb(0,0,0);font-family:"Arial Unicode MS",Arial,sans-serif;font-size:small"><div class="gmail-m_3522047505523402941col-md-12" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:1681px"><div class="gmail-m_3522047505523402941row" style="box-sizing:border-box"><div class="gmail-m_3522047505523402941col-md-2" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:280.156px"><div class="gmail-m_3522047505523402941GKIIXFICNAD" id="gmail-m_3522047505523402941SubTabHostGeneralSoftwareView_formPanel_col0_row3_label" style="box-sizing:border-box;font-family:Arial,sans-serif">KVM Version:</div></div><div class="gmail-m_3522047505523402941col-md-10" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:1400.83px"><span id="gmail-m_3522047505523402941SubTabHostGeneralSoftwareView_formPanel_col0_row3_value" class="gmail-m_3522047505523402941GKIIXFICOAD" style="box-sizing:border-box;color:rgb(119,119,119);font-family:Arial,sans-serif;word-wrap:break-word">2.6.0 - 28.el7_3.9.1</span></div></div></div></div><div class="gmail-m_3522047505523402941row" style="box-sizing:border-box;color:rgb(0,0,0);font-family:"Arial Unicode MS",Arial,sans-serif;font-size:small"><div class="gmail-m_3522047505523402941col-md-12" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:1681px"><div class="gmail-m_3522047505523402941row" style="box-sizing:border-box"><div class="gmail-m_3522047505523402941col-md-2" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:280.156px"><div class="gmail-m_3522047505523402941GKIIXFICNAD" id="gmail-m_3522047505523402941SubTabHostGeneralSoftwareView_formPanel_col0_row4_label" style="box-sizing:border-box;font-family:Arial,sans-serif">LIBVIRT Version:</div></div><div class="gmail-m_3522047505523402941col-md-10" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:1400.83px"><span id="gmail-m_3522047505523402941SubTabHostGeneralSoftwareView_formPanel_col0_row4_value" class="gmail-m_3522047505523402941GKIIXFICOAD" style="box-sizing:border-box;color:rgb(119,119,119);font-family:Arial,sans-serif;word-wrap:break-word">libvirt-2.0.0-10.el7_3.9</span></div></div></div></div><div class="gmail-m_3522047505523402941row" style="box-sizing:border-box;color:rgb(0,0,0);font-family:"Arial Unicode MS",Arial,sans-serif;font-size:small"><div class="gmail-m_3522047505523402941col-md-12" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:1681px"><div class="gmail-m_3522047505523402941row" style="box-sizing:border-box"><div class="gmail-m_3522047505523402941col-md-2" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:280.156px"><div class="gmail-m_3522047505523402941GKIIXFICNAD" id="gmail-m_3522047505523402941SubTabHostGeneralSoftwareView_formPanel_col0_row5_label" style="box-sizing:border-box;font-family:Arial,sans-serif">VDSM Version:</div></div><div class="gmail-m_3522047505523402941col-md-10" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:1400.83px"><span id="gmail-m_3522047505523402941SubTabHostGeneralSoftwareView_formPanel_col0_row5_value" class="gmail-m_3522047505523402941GKIIXFICOAD" style="box-sizing:border-box;color:rgb(119,119,119);font-family:Arial,sans-serif;word-wrap:break-word">vdsm-4.19.15-1.el7.centos</span></div></div></div></div><div class="gmail-m_3522047505523402941row" style="box-sizing:border-box;color:rgb(0,0,0);font-family:"Arial Unicode MS",Arial,sans-serif;font-size:small"><div class="gmail-m_3522047505523402941col-md-12" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:1681px"><div class="gmail-m_3522047505523402941row" style="box-sizing:border-box"><div class="gmail-m_3522047505523402941col-md-2" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:280.156px"><div class="gmail-m_3522047505523402941GKIIXFICNAD" id="gmail-m_3522047505523402941SubTabHostGeneralSoftwareView_formPanel_col0_row6_label" style="box-sizing:border-box;font-family:Arial,sans-serif">SPICE Version:</div></div><div class="gmail-m_3522047505523402941col-md-10" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:1400.83px"><span id="gmail-m_3522047505523402941SubTabHostGeneralSoftwareView_formPanel_col0_row6_value" class="gmail-m_3522047505523402941GKIIXFICOAD" style="box-sizing:border-box;color:rgb(119,119,119);font-family:Arial,sans-serif;word-wrap:break-word">0.12.4 - 20.el7_3</span></div></div></div></div><div class="gmail-m_3522047505523402941row" style="box-sizing:border-box;color:rgb(0,0,0);font-family:"Arial Unicode MS",Arial,sans-serif;font-size:small"><div class="gmail-m_3522047505523402941col-md-12" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:1681px"><div class="gmail-m_3522047505523402941row" style="box-sizing:border-box"><div class="gmail-m_3522047505523402941col-md-2" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:280.156px"><div class="gmail-m_3522047505523402941GKIIXFICNAD" id="gmail-m_3522047505523402941SubTabHostGeneralSoftwareView_formPanel_col0_row7_label" style="box-sizing:border-box;font-family:Arial,sans-serif">GlusterFS Version:</div></div><div class="gmail-m_3522047505523402941col-md-10" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:1400.83px"><span id="gmail-m_3522047505523402941SubTabHostGeneralSoftwareView_formPanel_col0_row7_value" class="gmail-m_3522047505523402941GKIIXFICOAD" style="box-sizing:border-box;color:rgb(119,119,119);font-family:Arial,sans-serif;word-wrap:break-word">glusterfs-3.8.11-1.el7</span></div></div></div></div><div class="gmail-m_3522047505523402941row" style="box-sizing:border-box;color:rgb(0,0,0);font-family:"Arial Unicode MS",Arial,sans-serif;font-size:small"><div class="gmail-m_3522047505523402941col-md-12" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:1681px"><div class="gmail-m_3522047505523402941row" style="box-sizing:border-box"><div class="gmail-m_3522047505523402941col-md-2" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:280.156px"><div class="gmail-m_3522047505523402941GKIIXFICNAD" id="gmail-m_3522047505523402941SubTabHostGeneralSoftwareView_formPanel_col0_row8_label" style="box-sizing:border-box;font-family:Arial,sans-serif">CEPH Version:</div></div><div class="gmail-m_3522047505523402941col-md-10" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:1400.83px"><span id="gmail-m_3522047505523402941SubTabHostGeneralSoftwareView_formPanel_col0_row8_value" class="gmail-m_3522047505523402941GKIIXFICOAD" style="box-sizing:border-box;color:rgb(119,119,119);font-family:Arial,sans-serif;word-wrap:break-word">librbd1-0.94.5-1.el7</span></div><div class="gmail-m_3522047505523402941col-md-10" style="box-sizing:border-box;min-height:1px;padding-left:20px;padding-right:20px;float:left;width:1400.83px"><span class="gmail-m_3522047505523402941GKIIXFICOAD" style="box-sizing:border-box;color:rgb(119,119,119);font-family:Arial,sans-serif;word-wrap:break-word"><br></span></div></div></div></div></div><div>qemu-img version 2.6.0 (qemu-kvm-ev-2.6.0-28.el7_3.9.<wbr>1), Copyright (c) 2004-2008 Fabrice Bellard</div><div><br></div><div>This is what i have on the hosts.</div><span class="gmail-HOEnZb"><font color="#888888"><div><br></div><div>/Johan</div></font></span><div><div class="gmail-h5"><div><br></div><div>On Sun, 2017-07-30 at 13:56 +0300, Benny Zlotnik wrote:</div><blockquote type="cite"><div dir="ltr"><div dir="ltr" style="font-size:12.8px">Hi, <div><br></div><div>Can please you provide the versions of vdsm, qemu, libvirt?</div></div><div class="gmail-m_3522047505523402941gmail-yj6qo gmail-m_3522047505523402941gmail-ajU" style="margin:2px 0px 0px;font-size:12.8px"></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Jul 30, 2017 at 1:01 PM, Johan Bernhardsson <span dir="ltr"><<a href="mailto:johan@kafit.se" target="_blank">johan@kafit.se</a>></span> wrote:<br><blockquote type="cite">Hello,<br>
<br>
We get this error message while moving or copying some of the disks on<br>
our main cluster running 4.1.2 on centos7 <br>
<br>
This is shown in the engine:<br>
VDSM vbgkvm02 command HSMGetAllTasksStatusesVDS failed: low level Image<br>
copy failed<br>
<br>
I can copy it inside the host. And i can use dd to copy. Haven't tried<br>
to run qemu-img manually yet.<br>
<br>
<br>
This is from vdsm.log on the host:<br>
2017-07-28 09:07:22,741+0200 ERROR (tasks/6) [root] Job u'c82d4c53-<br>
3eb4-405e-a2d5-c4c77519360e' failed (jobs:217)<br>
Traceback (most recent call last):<br>
File "/usr/lib/python2.7/site-packa<wbr>ges/vdsm/jobs.py", line 154, in<br>
run<br>
self._run()<br>
File "/usr/share/vdsm/storage/sdm/a<wbr>pi/copy_data.py", line 88, in _run<br>
self._operation.wait_for_c<wbr>ompletion()<br>
File "/usr/lib/python2.7/site-packa<wbr>ges/vdsm/qemuimg.py", line 329, in<br>
wait_for_completion<br>
self.poll(timeout)<br>
File "/usr/lib/python2.7/site-packa<wbr>ges/vdsm/qemuimg.py", line 324, in<br>
poll<br>
self.error)<br>
QImgError: cmd=['/usr/bin/taskset', '--cpu-list', '0-15',<br>
'/usr/bin/nice', '-n', '19', '/usr/bin/ionice', '-c', '3',<br>
'/usr/bin/qemu-img', 'convert', '-p', '-t', 'none', '-T', 'none', '-f',<br>
'raw', u'/rhev/data-center/mnt/gluste<wbr>rSD/vbgsan02:_fs02/0924ff77-<wbr>ef51-<br>
43<br>
5b-b90d-50bfbf2e8de7/images/75<wbr>0f4184-b852-4b00-94fc-<br>
476f3f5b93c7/3fe43487-3302-4b3<wbr>4-865a-07c5c6aedbf2', '-O', 'raw',<br>
u'/rhev/data-center/mnt/gluste<wbr>rSD/10.137.30.105:_fs03/5d47a2<wbr>97-a21f-<br>
4587-bb7c-dd00d52010d5/images/<wbr>750f4184-b852-4b00-94fc-<br>
476f3f5b93c7/3fe43487-3302-4b3<wbr>4-865<br>
a-07c5c6aedbf2'], ecode=1, stdout=, stderr=qemu-img: error while<br>
reading sector 12197886: No data available<br>
, message=None<br>
<br>
<br>
The storage domains are all based on gluster. The storage domains that<br>
we see this on is configured as dispersed volumes. <br>
<br>
Found a way to "fix" the problem. And that is to run dd if=/dev/vda<br>
of=/dev/null bs=1M inside the virtual guest. After that we can copy an<br>
image or use storage livemigration.<br>
<br>
Is this a gluster problem or an vdsm problem? Or could it be something<br>
with qemu-img?<br>
<br>
/Johan<br>
______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/users</a><br>
<br></blockquote></div><br></div>
</blockquote></div></div></div><br></blockquote></div><br></div></div>
</blockquote></body></html>