[ovirt-users] problem while moving/copying disks: vdsm low level image copy failed

[Johan Bernhardsson]

Hello,

We get this error message while moving or copying some of the disks on
our main cluster running 4.1.2 on centos7  

This is shown in the engine:
VDSM vbgkvm02 command HSMGetAllTasksStatusesVDS failed: low level Image
copy failed

I can copy it inside the host. And i can use dd to copy. Haven't tried
to run qemu-img manually yet.


This is from vdsm.log on the host:
2017-07-28 09:07:22,741+0200 ERROR (tasks/6) [root] Job u'c82d4c53-
3eb4-405e-a2d5-c4c77519360e' failed (jobs:217)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/jobs.py", line 154, in
run
    self._run()
  File "/usr/share/vdsm/storage/sdm/api/copy_data.py", line 88, in _run
    self._operation.wait_for_completion()
  File "/usr/lib/python2.7/site-packages/vdsm/qemuimg.py", line 329, in
wait_for_completion
    self.poll(timeout)
  File "/usr/lib/python2.7/site-packages/vdsm/qemuimg.py", line 324, in
poll
    self.error)
QImgError: cmd=['/usr/bin/taskset', '--cpu-list', '0-15',
'/usr/bin/nice', '-n', '19', '/usr/bin/ionice', '-c', '3',
'/usr/bin/qemu-img', 'convert', '-p', '-t', 'none', '-T', 'none', '-f', 
'raw', u'/rhev/data-center/mnt/glusterSD/vbgsan02:_fs02/0924ff77-ef51-
43
5b-b90d-50bfbf2e8de7/images/750f4184-b852-4b00-94fc-
476f3f5b93c7/3fe43487-3302-4b34-865a-07c5c6aedbf2', '-O', 'raw',
u'/rhev/data-center/mnt/glusterSD/10.137.30.105:_fs03/5d47a297-a21f-
4587-bb7c-dd00d52010d5/images/750f4184-b852-4b00-94fc-
476f3f5b93c7/3fe43487-3302-4b34-865
a-07c5c6aedbf2'], ecode=1, stdout=, stderr=qemu-img: error while
reading sector 12197886: No data available
, message=None


The storage domains are all based on gluster. The storage domains that
we see this on is configured as dispersed volumes. 

Found a way to "fix" the problem. And that is to run dd if=/dev/vda
of=/dev/null bs=1M  inside the virtual guest. After that we can copy an
image or use storage livemigration.

Is this a gluster problem or an vdsm problem? Or could it be something
with qemu-img?

/Johan