[ovirt-users] Disk image upload via CLI?

[Nir Soffer]

On Tue, Sep 12, 2017 at 5:45 PM Matthias Leopold <
matthias.leopold at meduniwien.ac.at> wrote:

> Thanks, i tried this script and it _almost_ worked ;-)
>
> i uploaded two images i created with
> qemu-img create -f qcow2 -o preallocation=full
> and
> qemu-img create -f qcow2 -o preallocation=falloc
>
> for initial_size and provisioned_size i took the value reported by
> "qemu-img info" in "virtual size" (same as "disk size" in this case)

the upload goes to 100% and then fails with
>
> 200 OK Completed 100%
> Traceback (most recent call last):
>    File "./upload_disk.py", line 157, in <module>
>      headers=upload_headers,
>    File "/usr/lib64/python2.7/httplib.py", line 1017, in request
>      self._send_request(method, url, body, headers)
>    File "/usr/lib64/python2.7/httplib.py", line 1051, in _send_request
>      self.endheaders(body)
>    File "/usr/lib64/python2.7/httplib.py", line 1013, in endheaders
>      self._send_output(message_body)
>    File "/usr/lib64/python2.7/httplib.py", line 864, in _send_output
>      self.send(msg)
>    File "/usr/lib64/python2.7/httplib.py", line 840, in send
>      self.sock.sendall(data)
>    File "/usr/lib64/python2.7/ssl.py", line 746, in sendall
>      v = self.send(data[count:])
>    File "/usr/lib64/python2.7/ssl.py", line 712, in send
>      v = self._sslobj.write(data)
> socket.error: [Errno 104] Connection reset by peer
>
> in web GUI the disk stays in Status: "Transferring via API"
> it can only be removed when manually unlocking it (unlock_entity.sh)
>
> engine.log tells nothing interesting
>
> i attached the last lines of ovirt-imageio-proxy/image-proxy.log and
> ovirt-imageio-daemon/daemon.log (from the executing node)
>
> the HTTP status 403 in ovirt-imageio-daemon/daemon.log doesn't look too
> nice to me
>
> can you explain what happens?
>

Hi Matthias,

I think qemu-img reports the wrong size in disk size, so the disk you
created
for the upload was smaller than the file you tried to upload. This will end
in
404 forbidden error when you try to write after the end of the specified
size.

In this case we send a detailed error to the proxy but it cannot get this
response
since it tries to send the data to the daemon, but the daemon will never
read
data from invalid request.

You should get the file size using os.path.getsize() instead of qemu-img
info.

That said, our logging is not good enough:

2017-09-12 16:07:13,188 WARNING (Thread-641) [web] xxx.yyy.215.2 - PUT
/1e12aa19-f122-4f6c-bfad-ce84abe2684e 403 138 (0.00s)

This log should explain why the operation is forbidden - for example:

    Trying to write to offset x, maximum offset y

Can you file a bug for more detailed logging when upload/download request
is invalid?

Regarding the logs you submitted, please submit complete logs. For example
in this case we would see the ticket size in the log for adding the ticket
for this
upload, which makes the issue more clear.


>
> ovirt engine is 4.1.5
> ovirt node is 4.1.3 (is that a problem?)
>

No but you want to run latest code to get the all the fixes introduced
in 4.1.5.

Cheers,
Nir


>
> thx
> matthias
>
>
>
> Am 2017-09-12 um 13:15 schrieb Fred Rolland:
> > Hi,
> >
> > You can check this example:
> >
> https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/upload_disk.py
> >
> > Regards,
> > Fred
> >
> > On Tue, Sep 12, 2017 at 11:49 AM, Matthias Leopold
> > <matthias.leopold at meduniwien.ac.at
> > <mailto:matthias.leopold at meduniwien.ac.at>> wrote:
> >
> >     Hi,
> >
> >     is there a way to upload disk images (not OVF files, not ISO files)
> >     to oVirt storage domains via CLI? I need to upload a 800GB file and
> >     this is not really comfortable via browser. I looked at ovirt-shell
> >     and
> >
> https://www.ovirt.org/develop/release-management/features/storage/image-upload/
> >     <
> https://www.ovirt.org/develop/release-management/features/storage/image-upload/
> >,
> >     but i didn't find an option in either of them.
> >
> >     thx
> >     matthias
> >
> >     _______________________________________________
> >     Users mailing list
> >     Users at ovirt.org <mailto:Users at ovirt.org>
> >     http://lists.ovirt.org/mailman/listinfo/users
> >     <http://lists.ovirt.org/mailman/listinfo/users>
> >
> >
>
> --
> Matthias Leopold
> IT Systems & Communications
> Medizinische Universität Wien
> Spitalgasse 23 / BT 88 /Ebene 00
> A-1090 Wien
> Tel: +43 1 40160-21241 <+43%201%204016021241>
> Fax: +43 1 40160-921200 <+43%201%2040160921200>
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170913/3f79d9db/attachment.html>