Hi

 

I am trying to upload a snapshot disk in chunks. Everything seems to work fine, but observed that the actual_size after upload, is much lesser than the actual_size of the original disk.

 

Here are the steps:

1.       Take a snapshot of a vm disk and download it (using Image Transfer mechanism). Save it on the file system somewhere.  This disk name is 3gbdisk. It is Raw + sparse. Resides on nfs storage. The size of this downloaded file is 3 GB.

 

  "actual_size" : "1389109248", //1 GB

  "alias" : "3gbdisk",

  "content_type" : "data",

  "format" : "raw",

  "image_id" : "8fbac55e-0c86-4c0b-911b-f5b0a6722834",

  "propagate_errors" : "false",

  "provisioned_size" : "3221225472",

  "shareable" : "false",

  "sparse" : "true",

  "status" : "ok",

  "storage_type" : "image",

  "total_size" : "0",

  "wipe_after_delete" : "false",

 

2.       Now create a new floating disk, (raw + sparse), with provisioned_size = 3221225472, or 3 GB. This disk name is vmRestoreDisk

3.       Upload to this disk using Image Transfer API, using libCurl  in chunks of 128 MB. This is done in a while loop,  sequentially reading portions of the file downloaded in step 1 and uploading these chunks via libcurl.  I Use the Transfer URL, not proxy URL.

 

Here is the trace of the first chunk. Note the Content-Range and Content-Length headers. Start offset = 0, end offset = 134217727 (or 128 MB)

 

upload request for chunk, start offset: 0, end offset: 134217727

Upload Started

Header:Content-Range: bytes 0-134217727/3221225472

Header:Content-Length: 3221225472

*   Trying 10.210.46.215...

* TCP_NODELAY set

* Connected to pnm86hpch30bl15.pne.ven.veritas.com (10.210.46.215) port 54322 (#0)

* ALPN, offering http/1.1

* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH

* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384

* ALPN, server did not agree to a protocol

* Server certificate:

*  subject: O=pne.ven.veritas.com; CN=pnm86hpch30bl15.pne.ven.veritas.com

*  start date: Oct  7 08:55:24 2018 GMT

*  expire date: Oct  7 08:55:24 2023 GMT

*  issuer: C=US; O=pne.ven.veritas.com; CN=pravauto20.pne.ven.veritas.com.59289

*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.

> PUT /images/8ebc9fa8-d322-423e-8a14-5e46ca10ed4e HTTP/1.1

Host: pnm86hpch30bl15.pne.ven.veritas.com:54322

Accept: */*

Content-Length: 134217728

Expect: 100-continue

 

* Done waiting for 100-continue

* We are completely uploaded and fine

* HTTP 1.0, assume close after body

< HTTP/1.0 200 OK

< Date: Fri, 23 Nov 2018 11:52:53 GMT

< Server: WSGIServer/0.1 Python/2.7.5

< Content-Type: application/json; charset=UTF-8

< Content-Length: 0

< 

* Closing connection 0

http response code from curl 200

Upload Finished. Return Value: 0

 

 

4.       Finalize the Image Transfer after all chunks are uploaded. Observed that the disk status goes from ‘uploading via API’ to finalizing to OK.

5.       Do a GET call on the disk (vmRestoreDisk).

  "actual_size" : "134217728", //128MB

  "alias" : "vmRestoreDisk",

  "content_type" : "data",

  "format" : "raw",

  "image_id" : "3eda3df2-514a-4e78-b999-1729216b25db",

  "propagate_errors" : "false",

  "provisioned_size" : "3221225472",

  "shareable" : "false",

  "sparse" : "true",

  "status" : "ok",

  "storage_type" : "image",

  "total_size" : "0",

  "wipe_after_delete" : "false",

 

As you can see, the actual size is just 128 MB, not 1 GB.  I have attached the logs of the upload operation. I think I may be missing something, let me know in case you need further information.

 

Thanks

Ranjit