[JIRA] (OVIRT-2024) virt-sparsify fails silently

virt-sparsify fails silently ----------------------------- Key: OVIRT-2024 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2024 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Dafna Ron Assignee: Dafna Ron Labels: ost_failures virt-sparsify fails silently (intentionally silently). you can see the error in the host vdsm log (even on successful builds) 018-05-14 07:15:02,271-0400 ERROR (tasks/6) [root] Job u'796dd36e-0095-4adc-afb9-8b011f6b8ced' failed (jobs:221) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/jobs.py", line 157, in run self._run() File "/usr/lib/python2.7/site-packages/vdsm/storage/sdm/api/sparsify_volume.py", line 56, in _run virtsparsify.sparsify_inplace(self._vol_info.path) File "/usr/lib/python2.7/site-packages/vdsm/virtsparsify.py", line 71, in sparsify_inplace raise cmdutils.Error(cmd, rc, out, err) Error: Command ['/usr/bin/virt-sparsify', '--machine-readable', '--in-place', u'/rhev/data-center/mnt/192.168.200.4:_exports_nfs_share1/96157fda-d60a-49d1-adb8-f306e402a0cf/images/156eaf32-5219-4be7-b61e-b5104f2b6c4d/77eb6602-13ac-46aa-a4 39-af9e51e3c2cb'] failed with rc=1 out=['3/12'] err=['virt-sparsify: error: libguestfs error: guestfs_launch failed.', 'This usually means the libguestfs appliance failed to start or crashed.', 'Do:', ' export LIBGUESTFS_DEBUG=1 LIBGUEST FS_TRACE=1', 'and run the command again. For further information, read:', ' http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs';, "You can also run 'libguestfs-test-tool' and post the *complete* output", 'into a bug report or message to the libguestfs mailing list.', '', 'If reporting bugs, run virt-sparsify with debugging enabled and include the ', 'complete output:', '', ' virt-sparsify -v -x [...]'] 2018-05-14 07:15:02,273-0400 INFO (tasks/6) [root] Job u'796dd36e-0095-4adc-afb9-8b011f6b8ced' will be deleted in 3600 seconds (jobs:249) 2018-05-14 07:15:02,273-0400 DEBUG (tasks/6) [jsonrpc.Notification] Sending event {"params": {"status": "failed", "description": "sparsify_volume", "job_type": "storage", "error": {"message": "General Exception: ('Command [\\'/usr/bin/vir t-sparsify\\', \\'--machine-readable\\', \\'--in-place\\', u\\'/rhev/data-center/mnt/192.168.200.4:_exports_nfs_share1/96157fda-d60a-49d1-adb8-f306e402a0cf/images/156eaf32-5219-4be7-b61e-b5104f2b6c4d/77eb6602-13ac-46aa-a439-af9e51e3c2cb\\ '] failed with rc=1 out=[\\'3/12\\'] err=[\\'virt-sparsify: error: libguestfs error: guestfs_launch failed.\\', \\'This usually means the libguestfs appliance failed to start or crashed.\\', \\'Do:\\', \\' export LIBGUESTFS_DEBUG=1 LIBGU ESTFS_TRACE=1\\', \\'and run the command again. For further information, read:\\', \\' http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs\\';, \"You can also run \\'libguestfs-test-tool\\' and post the *complete* output\", \\' into a bug report or message to the libguestfs mailing list.\\', \\'\\', \\'If reporting bugs, run virt-sparsify with debugging enabled and include the \\', \\'complete output:\\', \\'\\', \\' virt-sparsify -v -x [...]\\']',)", "code": 1 00}, "notify_time": 4295843980, "id": "796dd36e-0095-4adc-afb9-8b011f6b8ced"}, "jsonrpc": "2.0", "method": "|jobs|status|796dd36e-0095-4adc-afb9-8b011f6b8ced"} (__init__:180) 2018-05-14 07:15:02,273-0400 DEBUG (tasks/6) [vds] Sending notification |jobs|status|796dd36e-0095-4adc-afb9-8b011f6b8ced with params {'status': 'failed', 'description': 'sparsify_volume', 'job_type': 'storage', 'error': {'message': 'Gene ral Exception: (\'Command [\\\'/usr/bin/virt-sparsify\\\', \\\'--machine-readable\\\', \\\'--in-place\\\', u\\\'/rhev/data-center/mnt/192.168.200.4:_exports_nfs_share1/96157fda-d60a-49d1-adb8-f306e402a0cf/images/156eaf32-5219-4be7-b61e-b5 104f2b6c4d/77eb6602-13ac-46aa-a439-af9e51e3c2cb\\\'] failed with rc=1 out=[\\\'3/12\\\'] err=[\\\'virt-sparsify: error: libguestfs error: guestfs_launch failed.\\\', \\\'This usually means the libguestfs appliance failed to start or crash ed.\\\', \\\'Do:\\\', \\\' export LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1\\\', \\\'and run the command again. For further information, read:\\\', \\\' http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs\\\';, "You can also run \ \\'libguestfs-test-tool\\\' and post the *complete* output", \\\'into a bug report or message to the libguestfs mailing list.\\\', \\\'\\\', \\\'If reporting bugs, run virt-sparsify with debugging enabled and include the \\\', \\\'complet e output:\\\', \\\'\\\', \\\' virt-sparsify -v -x [...]\\\']\',)', 'code': 100}, 'notify_time': 4295843980, 'id': u'796dd36e-0095-4adc-afb9-8b011f6b8ced'} (clientIF:225) 2018-05-14 07:15:02,274-0400 INFO (tasks/6) [storage.ThreadPool.WorkerThread] FINISH task 041a7423-8c05-44b2-b3c9-6d5cf8710edc (threadPool:210) 2018-05-14 07:15:04,229-0400 DEBUG (qgapoller/1) [vds] Not sending QEMU-GA command 'guest-get-users' to vm_id='e1317264-a572-47e0-98b4-d492239e285e', command is not supported (qemuguestagent:192) 2018-05-14 07:15:04,230-0400 DEBUG (qgapoller/2) [vds] Not sending QEMU-GA command 'guest-get-users' to vm_id='29a56e04-51c7-4f25-b3c8-6df5489b2445', command is not supported (qemuguestagent:192) 2018-05-14 07:15:05,186-0400 DEBUG (vmchannels) [vds] Timeout on fileno 63. (vmchannels:126) Shmuel is investigating and we need to follow up with him.