On Thu, Feb 7, 2019 at 7:32 PM Fedor Gavrilov <fgavrilo@redhat.com> wrote:
I just did a fresh setup, applying the latest suggestions and it seems now there is ab error message in the engine log when I press "test connection" in upload image window:

What is the result of the test in the UI? I guess you get yellow warning?
 

***
192.168.111.1 is machine where iso is
192.168.111.2 is engine
192.168.111.3 is host and nfs storage
***

----------------------------------------
Exception happened during processing of request from ('192.168.111.1', 46230)
Traceback (most recent call last):
  File "/usr/lib64/python2.7/SocketServer.py", line 596, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/lib64/python2.7/SocketServer.py", line 331, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib64/python2.7/SocketServer.py", line 652, in __init__
    self.handle()
  File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 116, in handle
    self.raw_requestline = self.rfile.readline(65537)
  File "/usr/lib64/python2.7/socket.py", line 480, in readline
    data = self._sock.recv(self._rbufsize)
  File "/usr/lib64/python2.7/ssl.py", line 772, in recv
    return self.read(buflen)
  File "/usr/lib64/python2.7/ssl.py", line 659, in read
    v = self._sslobj.read(len)
SSLError: [SSL: SSLV3_ALERT_BAD_CERTIFICATE] sslv3 alert bad certificate (_ssl.c:1941)
----------------------------------------

This looks like bad proxy configuration, it does not accept the engine certificate.

Can you share:
- $ENGINE_PREFIX/etc/ovirt-imageio-proxy/image-proxy.conf?
- your engine-setup answer file

Didi, where do we keep the answer file?

Not sure what it means though. Certificate is installed in my browser, just double-checked that.
When trying to upload the file nevertheless, this is what appears in engine logs:

Trying to upload to proxy or daemon? from the UI or using upload_disk.py example?

2019-02-07 18:27:34,768+01 INFO  [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [1b6235be-02b4-446a-b486-22cce0d7a1bb] Adding image ticket to ovirt-imageio-proxy, id 00e11769-70c4-4b92-9cb9-4ff633566d8e
2019-02-07 18:27:34,820+01 ERROR [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [1b6235be-02b4-446a-b486-22cce0d7a1bb] Failed to add image ticket to ovirt-imageio-proxy: javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No subject alternative names matching IP address 192.168.111.2 found
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.8.0_191]
        at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1946) [jsse.jar:1.8.0_191]
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:316) [jsse.jar:1.8.0_191]
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310) [jsse.jar:1.8.0_191]
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639) [jsse.jar:1.8.0_191]
        at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223) [jsse.jar:1.8.0_191]
        at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037) [jsse.jar:1.8.0_191]
        at sun.security.ssl.Handshaker.process_record(Handshaker.java:965) [jsse.jar:1.8.0_191]
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064) [jsse.jar:1.8.0_191]
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367) [jsse.jar:1.8.0_191]
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395) [jsse.jar:1.8.0_191]
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379) [jsse.jar:1.8.0_191]
        at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559) [rt.jar:1.8.0_191]
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185) [rt.jar:1.8.0_191]
        at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1334) [rt.jar:1.8.0_191]
        at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1309) [rt.jar:1.8.0_191]
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:259) [rt.jar:1.8.0_191]
        at org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand.addImageTicketToProxy(TransferDiskImageCommand.java:837) [bll.jar:]
        at org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand.startImageTransferSession(TransferDiskImageCommand.java:763) [bll.jar:]
        at org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand.handleImageIsReadyForTransfer(TransferDiskImageCommand.java:452) [bll.jar:]
        at org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand.handleInitializing(TransferDiskImageCommand.java:423) [bll.jar:]
        at org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand.executeStateHandler(TransferDiskImageCommand.java:358) [bll.jar:]
        at org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand.proceedCommandExecution(TransferDiskImageCommand.java:345) [bll.jar:]
        at org.ovirt.engine.core.bll.storage.disk.image.TransferImageCommandCallback.doPolling(TransferImageCommandCallback.java:21) [bll.jar:]
        at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethodsImpl(CommandCallbacksPoller.java:175) [bll.jar:]
        at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethods(CommandCallbacksPoller.java:109) [bll.jar:]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_191]
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [rt.jar:1.8.0_191]
        at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:383) [javax.enterprise.concurrent-1.0.jar:]
        at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:534) [javax.enterprise.concurrent-1.0.jar:]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_191]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_191]
        at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_191]
        at org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250) [javax.enterprise.concurrent-1.0.jar:]
        at org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManagedThreadFactory.java:78)
Caused by: java.security.cert.CertificateException: No subject alternative names matching IP address 192.168.111.2 found
        at sun.security.util.HostnameChecker.matchIP(HostnameChecker.java:168) [rt.jar:1.8.0_191]
        at sun.security.util.HostnameChecker.match(HostnameChecker.java:94) [rt.jar:1.8.0_191]
        at sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:455) [jsse.jar:1.8.0_191]
        at sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:436) [jsse.jar:1.8.0_191]
        at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:200) [jsse.jar:1.8.0_191]
        at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124) [jsse.jar:1.8.0_191]
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621) [jsse.jar:1.8.0_191]
        ... 30 more

2019-02-07 18:27:34,830+01 ERROR [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [1b6235be-02b4-446a-b486-22cce0d7a1bb] Failed to add image ticket to ovirt-imageio-proxy

Expected when proxy will not accept engine request because of bad certificate.
 
2019-02-07 18:27:34,836+01 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [1b6235be-02b4-446a-b486-22cce0d7a1bb] EVENT_ID: TRANSFER_IMAGE_STOPPED_BY_SYSTEM_FAILED_TO_ADD_TICKET_TO_PROXY(1,070), Transfer was stopped by system. Reason: failed to add image ticket to ovirt-imageio-proxy.

I will continue looking into it tomorrow as well, but any advice is much appreciated.

Thanks,
Fedor Gavrilov

----- Original Message -----
From: "Nir Soffer" <nsoffer@redhat.com>
To: "Fedor Gavrilov" <fgavrilo@redhat.com>
Cc: "Roy Golan" <rgolan@redhat.com>, "devel" <devel@ovirt.org>, "Daniel Erez" <derez@redhat.com>
Sent: Wednesday, February 6, 2019 10:26:00 PM
Subject: Re: [ovirt-devel] Re: imageio proxy and engine dev setup

On Wed, Feb 6, 2019 at 12:24 PM Fedor Gavrilov <fgavrilo@redhat.com> wrote:

First, please keep Daniel in the CC, this is your best chance to get a help
on
this, and a good practice for most issues :-)

Thanks, Roy! I will try setting it up according to what you suggested.
> Last attempt failed indeed: according to logs, both daemon and proxy tried
> establishing a connection with each other with some 200 OK in logs, no
> error messages but nevertheless upload did not happen after all.
>

Did you restart engine after changing the config?
Did you add engine CA to the browser?
Did you check the browser console.log?
Can you share your logs?

Can you reply to these questions?

> Speaking about it, does anyone know more straightforward way to have ISO
> disk on data domain?


Uploading from the UI is the most straightforward way. But you need to get
a working setup
first.

I am not as much interested in debugging ISO upload but rather attaching it
> to VM.
>

Sad that you are not interested in this yet, but in the meantime you can
use the ovirt SDK
upload_disk.py example.

1. install first the ovirt python sdk version 4:

    dnf install python3-ovirt-engine-sdk4

2. Download the upload disk example:


https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/upload_disk.py

3. Change the configuration to match your setup (e.g. storage domain name)

4. Upload:

    python upload_disk.py --direct /path/to/disk.iso

Note that --direct goes directly to the host, this is faster compared with
going to the proxy.

I think we should have a proper command line tool that make all this much
easier. We have
this RFE:
https://bugzilla.redhat.com/show_bug.cgi?id=1626262

Maybe you can be interested in implementing this?

Nir