Veritas: Image Transfer Finalize Call Failure
Hi Team, Can anyone help answer this question? Best, Pavan. ---------- Forwarded message --------- From: Ketan Pachpande <Ketan.Pachpande@veritas.com> Date: Thu, Aug 9, 2018 at 9:44 AM Subject: RE: [EXTERNAL] Updated invitation: RHV- Veritas Netbackup Weekly Sync (Tentative) @ Weekly from 10am to 10:30am on Thursday (EDT) ( ketan.pachpande@veritas.com) To: pchavva@redhat.com <pchavva@redhat.com>, Abhay Marode < Abhay.Marode@veritas.com>, Suchitra Herwadkar < Suchitra.Herwadkar@veritas.com>, Mahesh Falmari <Mahesh.Falmari@veritas.com>, Sudhakar Paulzagade <Sudhakar.Paulzagade@veritas.com>, Navin Tah < Navin.Tah@veritas.com> Cc: ydary@redhat.com <ylavi@redhat.com>, adbarbos@redhat.com < adbarbos@redhat.com> Hi Pavan, I have a question regarding imagetransfer finalize call. After imagetransfer upload, when I call the finalize the transfer, I am getting Finalize Failure error. I am following these steps to upload a disk via rest API. 1. Create disk on a storage domain (POST https:// <ovirt-server>/ovirt-engine/api/disks) 2. Initiate imagetransfer and get the proxy_url and signed_ticket (https:// <ovirt-server> /ovirt-engine/api/imagetransfers) 3. Upload data using curl (to proxy URL) 1. Finalize the transfer: After that disk is getting deleted automatically. Sequence of events in events tab: Is it expected behaviour on finalizing the imagetransfer failure? If so, how to troubleshoot and get the reason of finalizing failure? Thanks, Ketan Pachpande
On Thu, Aug 9, 2018 at 6:02 PM Pavan Chavva <pchavva@redhat.com> wrote:
I have a question regarding imagetransfer finalize call.
After imagetransfer upload, when I call the finalize the transfer, I am getting Finalize Failure error.
I am following these steps to upload a disk via rest API.
1. Create disk on a storage domain (POST https:// <ovirt-server>/ovirt-engine/api/disks)
Why not use the SDK?
1. Initiate imagetransfer and get the proxy_url and signed_ticket (https:// <ovirt-server> /ovirt-engine/api/imagetransfers)
Why are you using proxy_url? The recommended way is to use transfer_url, so
you upload directly to the host. The only reason for using the proxy url is not being able to access the host from the machine doing the upload.
1. Upload data using curl (to proxy URL)
[image: image003.jpg]
curl is less efficient than imageio upload_disk.py from the SDK. Also,
using curl you are exposing the transfer id in the command line, so every process running on the machine doing the upload can steal that id and access the image. The proxy_url and transfer_url are sensitive and should not be exposed in the command line.
1. Finalize the transfer:
[image: image005.jpg]
[image: image009.jpg]
After that disk is getting deleted automatically.
Sequence of events in events tab:
[image: image010.png]
Is it expected behaviour on finalizing the imagetransfer failure?
Yes, it there was an issue with the upload. A possible issue may be mismatch between the created disk and the actual data uploaded. For example, incorrect backing file in qcow2 format, or incorrect format, like uploading raw file to qcow2 image.
If so, how to troubleshoot and get the reason of finalizing failure?
Without engine, vdsm, proxy and daemon logs we cannot tell. Please share: engine host: /var/log/ovirt-engine/engine.log /var/log/ovirt-imageio-proxy/image-proxy.log host performing the upload: /var/log/vdsm/vdsm.log /var/log/ovirt-imageio-daemon/daemon.log Make sure that all logs include the transfer id - you can find it in the proxy_url and fransfer_url: https://server:port/images/transfer-id Nir
Thanks Nir, I figured out that there was a mismatch in disk format while uploading. Corrected it and it started working as expected. Finalize call is successful now. About using SDK, as of now we are bound to use curl from C++, because of our own limitations of using python on Netbackup server. But we will definitely consider your inputs about curl being less efficient and less secure. Also, we will make use transfer_url instead of proxy_url. Thanks, Ketan Pachpande From: Nir Soffer <nsoffer@redhat.com> Sent: 09 August 2018 23:24 To: Pavan Chavva <pchavva@redhat.com> Cc: devel <devel@ovirt.org>; Adelino Barbosa <adbarbos@redhat.com>; Navin Tah <Navin.Tah@veritas.com>; Mahesh Falmari <Mahesh.Falmari@veritas.com>; Yaniv Lavi (Dary) <ylavi@redhat.com>; Sudhakar Paulzagade <Sudhakar.Paulzagade@veritas.com>; Ketan Pachpande <Ketan.Pachpande@veritas.com>; Suchitra Herwadkar <Suchitra.Herwadkar@veritas.com>; Abhay Marode <Abhay.Marode@veritas.com>; Daniel Erez <derez@redhat.com> Subject: [EXTERNAL] Re: [ovirt-devel] Veritas: Image Transfer Finalize Call Failure On Thu, Aug 9, 2018 at 6:02 PM Pavan Chavva <pchavva@redhat.com<mailto:pchavva@redhat.com>> wrote: I have a question regarding imagetransfer finalize call. After imagetransfer upload, when I call the finalize the transfer, I am getting Finalize Failure error. I am following these steps to upload a disk via rest API. 1. Create disk on a storage domain (POST https://<ovirt-server>/ovirt-engine/api/disks<https://%3covirt-server%3e/ovirt-engine/api/disks>) Why not use the SDK? 1. Initiate imagetransfer and get the proxy_url and signed_ticket (https:// <ovirt-server> /ovirt-engine/api/imagetransfers) Why are you using proxy_url? The recommended way is to use transfer_url, so you upload directly to the host. The only reason for using the proxy url is not being able to access the host from the machine doing the upload. 1. Upload data using curl (to proxy URL) [image003.jpg] curl is less efficient than imageio upload_disk.py from the SDK. Also, using curl you are exposing the transfer id in the command line, so every process running on the machine doing the upload can steal that id and access the image. The proxy_url and transfer_url are sensitive and should not be exposed in the command line. 1. Finalize the transfer: [image005.jpg] [image009.jpg] After that disk is getting deleted automatically. Sequence of events in events tab: [image010.png] Is it expected behaviour on finalizing the imagetransfer failure? Yes, it there was an issue with the upload. A possible issue may be mismatch between the created disk and the actual data uploaded. For example, incorrect backing file in qcow2 format, or incorrect format, like uploading raw file to qcow2 image. If so, how to troubleshoot and get the reason of finalizing failure? Without engine, vdsm, proxy and daemon logs we cannot tell. Please share: engine host: /var/log/ovirt-engine/engine.log /var/log/ovirt-imageio-proxy/image-proxy.log host performing the upload: /var/log/vdsm/vdsm.log /var/log/ovirt-imageio-daemon/daemon.log Make sure that all logs include the transfer id - you can find it in the proxy_url and fransfer_url: https://server:port/images/transfer-id Nir
On Thu, Aug 9, 2018 at 9:10 PM Ketan Pachpande <Ketan.Pachpande@veritas.com> wrote:
Thanks Nir,
I figured out that there was a mismatch in disk format while uploading. Corrected it and it started working as expected. Finalize call is successful now.
About using SDK, as of now we are bound to use curl from C++, because of our own limitations of using python on Netbackup server.
Maybe use libcurl? https://curl.haxx.se/libcurl/c/example.html
But we will definitely consider your inputs about curl being less efficient and less secure.
Also, we will make use transfer_url instead of proxy_url.
Thanks,
Ketan Pachpande
*From:* Nir Soffer <nsoffer@redhat.com> *Sent:* 09 August 2018 23:24 *To:* Pavan Chavva <pchavva@redhat.com> *Cc:* devel <devel@ovirt.org>; Adelino Barbosa <adbarbos@redhat.com>; Navin Tah <Navin.Tah@veritas.com>; Mahesh Falmari < Mahesh.Falmari@veritas.com>; Yaniv Lavi (Dary) <ylavi@redhat.com>; Sudhakar Paulzagade <Sudhakar.Paulzagade@veritas.com>; Ketan Pachpande < Ketan.Pachpande@veritas.com>; Suchitra Herwadkar < Suchitra.Herwadkar@veritas.com>; Abhay Marode <Abhay.Marode@veritas.com>; Daniel Erez <derez@redhat.com> *Subject:* [EXTERNAL] Re: [ovirt-devel] Veritas: Image Transfer Finalize Call Failure
On Thu, Aug 9, 2018 at 6:02 PM Pavan Chavva <pchavva@redhat.com> wrote:
I have a question regarding imagetransfer finalize call.
After imagetransfer upload, when I call the finalize the transfer, I am getting Finalize Failure error.
I am following these steps to upload a disk via rest API.
1. Create disk on a storage domain (POST https://<ovirt-server>/ovirt-engine/api/disks)
Why not use the SDK?
1. Initiate imagetransfer and get the proxy_url and signed_ticket (https:// <ovirt-server> /ovirt-engine/api/imagetransfers)
Why are you using proxy_url? The recommended way is to use transfer_url, so
you upload directly to the host. The only reason for using the proxy url is not being
able to access the host from the machine doing the upload.
1. Upload data using curl (to proxy URL)
[image: image003.jpg]
curl is less efficient than imageio upload_disk.py from the SDK. Also, using curl you are
exposing the transfer id in the command line, so every process running on the machine
doing the upload can steal that id and access the image. The proxy_url and transfer_url
are sensitive and should not be exposed in the command line.
1. Finalize the transfer:
[image: image001.jpg]
[image: image002.jpg]
After that disk is getting deleted automatically.
Sequence of events in events tab:
[image: image003.png]
Is it expected behaviour on finalizing the imagetransfer failure?
Yes, it there was an issue with the upload.
A possible issue may be mismatch between the created disk and the actual data uploaded.
For example, incorrect backing file in qcow2 format, or incorrect format, like uploading raw
file to qcow2 image.
If so, how to troubleshoot and get the reason of finalizing failure?
Without engine, vdsm, proxy and daemon logs we cannot tell.
Please share:
engine host:
/var/log/ovirt-engine/engine.log
/var/log/ovirt-imageio-proxy/image-proxy.log
host performing the upload:
/var/log/vdsm/vdsm.log
/var/log/ovirt-imageio-daemon/daemon.log
Make sure that all logs include the transfer id - you can find it in the proxy_url and fransfer_url:
https://server:port/images/transfer-id
Nir
participants (3)
-
Ketan Pachpande -
Nir Soffer -
Pavan Chavva