On Tue, Aug 18, 2020 at 7:10 PM Michael Jones <mj@mikejonesey.co.uk> wrote:
Thanks for the help, not sure if this is a top post or bottom post mail list, feel free to tell me off;
We don't have strict rules, but I think that bottom posting will be better, even better selecting the relevant parts in your reply.
it does sound as if, my problems are around the fact that i am using an all-in-one box, (host and engine all in one);
This explains how that you can install all-in-one when engine is a VM running on the single host, not as a program running on the host. How did you managed to get engine installed on the same host? I would expect that the installer would fail or at least warn about this.
I'm about to setup another server, so i'll perhaps try engine as vm setup through the cockpit installer, which i've only used on ovirt clusters so far, not on single host machines.
If that works, perhaps i'll work out a way to port the server i'm having issues with.
Taking a backup of engine database, installing engine on another host or as vm, and restoring the backup should be the easiest way to move the engine.
the download / upload function is important to me as my backup solution is dependent on this.
Until the sort this out, you should know that you can transfer images without the proxy. The proxy is needed only for the UI, for cases when engine and hosts are on different networks, so the only way to transfer images is via the engine host. To get this working on your setup, you need to restore 50-vdsm.conf to the default configuration and restart ovirt-imageio service. There is no way to disable the proxy in engine, so for every transfer engine will try to add a ticket the proxy and will fail, but this should not fail the transfer, only log the error. To transfer images, use upload_disk.py and download_disk.py from ovirt-engine-sdk. dnf install python3-ovirt-enigne-sdk4 And try: python3 /usr/share/doc/python3-ovirt-engine-sdk4/examples/upload_disk.py -h python3 /usr/share/doc/python3-ovirt-engine-sdk4/examples/download_disk.py -h There is also a backup_vm.py example that can be very useful for backup, supporting both full and incremental backup: python3 /usr/share/doc/python3-ovirt-engine-sdk4/examples/backup_vm.py -h Nir
Thanks,
Kind Regards,
Michael Jones
On 18/08/2020 16:55, Nir Soffer wrote:
On Tue, Aug 18, 2020 at 4:06 PM Michael Jones <mj@mikejonesey.co.uk> wrote:
I have image upload/download working on some older ovirt servers where it still has the split daemon/proxy...
on one newer one this feature is not working;
software in use:
CentOS 8 ovirt-engine-4.4.1.10-1.el8.noarch ovirt-imageio-client-2.0.9-1.el8.x86_64 ovirt-imageio-daemon-2.0.9-1.el8.x86_64 ovirt-imageio-common-2.0.9-1.el8.x86_64 Both host and engine running these versions?
the ui test button allowed me to work out 50-vdsm.conf was setting the wrong remote port... (was 54322, changed to 54323) 50-vdsm.conf is cannot break the connection test, since it should be installed only on the hosts. On engine you have 50-engine.conf.
The correct configuration for engine is:
[tls] enable = true key_file = /etc/pki/ovirt-engine/keys/apache.key.nopass cert_file = /etc/pki/ovirt-engine/certs/apache.cer ca_file = /etc/pki/ovirt-engine/apache-ca.pem
[remote] port = 54323
[local] enable = false
[control] transport = tcp port = 54324
The correct settings for the hosts are:
[tls] enable = true key_file = /etc/pki/vdsm/keys/vdsmkey.pem cert_file = /etc/pki/vdsm/certs/vdsmcert.pem ca_file = /etc/pki/vdsm/certs/cacert.pem
[remote] port = 54322
These files belongs to engine and vdsm and you should not change them. Your changes will be overwirtten on the next upgrade.
The top of the file explain how to change the configuration.
updated remote with;
[remote] host = 0.0.0.0 port = 54323
the test now passes, but on upload or download it still fails. Do you have 50-vdsm.conf on the engine host?!
It sounds like you have all-in-one configuration when engine host is also the single host. This configuration is not supported for 5 years or so.
Or you installed vdsm by mistake on engine host, in this case you will have both 50-vdsm.conf and 50-engine.conf, and because "vdsm" sorts after "engine" its configuration will win.
Next i changed the control to be unix socket instead of tcp port 54324 (vdsm was giving an error: Image daemon is unsupported);
I looked up the error line in the vdsm code, and found it was looking for unix socket: DAEMON_SOCK=/run/ovirt-imageio/sock
switching to sock seemed to resolve all errors in the vdsm log; Expected, using tcp for the host is not supported.
---
content of the imageio log; imagieo log on engine or host?
no errors as far as i can see:
2020-08-18 12:49:56,109 INFO (MainThread) [server] Starting (pid=2696562, version=2.0.9) 2020-08-18 12:49:56,109 DEBUG (MainThread) [services] Creating remote.service on port 54323 2020-08-18 12:49:56,111 DEBUG (MainThread) [http] Prefer IPv4: False 2020-08-18 12:49:56,111 DEBUG (MainThread) [http] Available network interfaces: [(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('0.0.0.0', 54323))] 2020-08-18 12:49:56,111 DEBUG (MainThread) [http] Creating server socket with family=AddressFamily.AF_INET and type=SocketKind.SOCK_STREAM 2020-08-18 12:49:56,111 DEBUG (MainThread) [services] Securing server (cafile=/etc/pki/vdsm/certs/cacert.pem, certfile=/etc/pki/vdsm/certs/vdsmcert.pem, keyfile=/etc/pki/vdsm/keys/vdsmkey.pem) So this is log from the host
2020-08-18 12:49:56,113 INFO (MainThread) [services] remote.service listening on ('0.0.0.0', 54323) This port is wrong, you will not be able to tansfer anything since engine assumes port 54322.
2020-08-18 12:49:56,113 DEBUG (MainThread) [services] Creating local.service on socket '\x00/org/ovirt/imageio' 2020-08-18 12:49:56,113 INFO (MainThread) [services] local.service listening on '\x00/org/ovirt/imageio' 2020-08-18 12:49:56,113 DEBUG (MainThread) [services] Creating control.service on socket '/run/ovirt-imageio/sock' 2020-08-18 12:49:56,113 DEBUG (MainThread) [uhttp] Removing socket '/run/ovirt-imageio/sock' 2020-08-18 12:49:56,113 INFO (MainThread) [services] control.service listening on '/run/ovirt-imageio/sock' 2020-08-18 12:49:56,115 DEBUG (MainThread) [server] Changing ownership of /run/ovirt-imageio to 988:984 2020-08-18 12:49:56,115 DEBUG (MainThread) [server] Changing ownership of /var/log/ovirt-imageio/daemon.log to 988:984 2020-08-18 12:49:56,115 DEBUG (MainThread) [server] Dropping root privileges, running as 988:984 2020-08-18 12:49:56,116 DEBUG (MainThread) [services] Starting remote.service 2020-08-18 12:49:56,116 DEBUG (remote.service) [services] remote.service started 2020-08-18 12:49:56,116 DEBUG (MainThread) [services] Starting local.service 2020-08-18 12:49:56,117 DEBUG (local.service) [services] local.service started 2020-08-18 12:49:56,117 DEBUG (MainThread) [services] Starting control.service 2020-08-18 12:49:56,117 DEBUG (control.service) [services] control.service started 2020-08-18 12:49:56,117 INFO (MainThread) [server] Ready for requests 2020-08-18 12:51:34,602 INFO (Thread-1) [http] OPEN client=local 2020-08-18 12:51:34,603 INFO (Thread-1) [tickets] [local] ADD ticket={'dirty': False, 'ops': ['read'], 'filename': 'debian-10.3.0-amd64-netinst.iso.raw', 'size': 351272960, 'sparse': True, 'transfer_id': 'ec7b329e-3ef1-448d-ac73-1e5e1b4c0ef9', 'uuid': '17db8f9f-3280-4e7c-8fbf-e5326e1f98be', 'timeout': 300, 'url': 'file:///rhev/data-center/mnt/_var_local_ovirt_local__data/7ee98037-126b-4c79-b921-a9ff4272fa91/images/315af697-a064-4c2c-bdf6-8144a8218526/8e709a79-4da0-4d91-803e-d6ec12ec84c3'} Looks good, this host is using local storage.
2020-08-18 12:51:34,603 DEBUG (Thread-1) [http] Empty request line, client disconnected 2020-08-18 12:51:34,603 INFO (Thread-1) [http] CLOSE client=local [connection 1 ops, 0.000842 s] [dispatch 1 ops, 0.000305 s]
---
vdsm log for transfer id: ec7b329e-3ef1-448d-ac73-1e5e1b4c0ef9
no errors as far as i can see...
2020-08-18 12:51:34,602+0000 INFO (jsonrpc/5) [vdsm.api] START add_image_ticket(ticket={'dirty': False, 'ops': ['read'], 'filename': 'debian-10.3.0-amd64-netinst.iso.raw', 'size': 351272960, 'sparse': True, 'transfer_id': 'ec7b329e-3ef1-448d-ac73-1e5e1b4c0ef9', 'uuid': '17db8f9f-3280-4e7c-8fbf-e5326e1f98be', 'timeout': 300, 'url': 'file:///rhev/data-center/mnt/_var_local_ovirt_local__data/7ee98037-126b-4c79-b921-a9ff4272fa91/images/315af697-a064-4c2c-bdf6-8144a8218526/8e709a79-4da0-4d91-803e-d6ec12ec84c3'}) from=::ffff:37.187.146.167,58882, flow_id=19204d8c-ffe6-4c45-81fe-96e07097d519, task_id=441bf5ea-1e93-4b5b-aeae-f0e52b43cf62 (api:48) 2020-08-18 12:51:34,603+0000 INFO (jsonrpc/5) [vdsm.api] FINISH add_image_ticket return=None from=::ffff:37.187.146.167,58882, flow_id=19204d8c-ffe6-4c45-81fe-96e07097d519, task_id=441bf5ea-1e93-4b5b-aeae-f0e52b43cf62 (api:54) 2020-08-18 12:51:34,603+0000 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.add_image_ticket succeeded in 0.00 seconds (__init__:312)
---
engine log:
urn [status=Status [code=0, message=Done]], log id: 7ac2f68b 2020-08-18 12:51:34,604Z INFO [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (default task-6) [19204d8c-ffe6-4c45-81fe-96e07097d519] Started transfer session with ticket id 17db8f9f-328 0-4e7c-8fbf-e5326e1f98be, timeout 300 seconds 2020-08-18 12:51:34,604Z INFO [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (default task-6) [19204d8c-ffe6-4c45-81fe-96e07097d519] Adding image ticket to ovirt-imageio-proxy, id 17db8 f9f-3280-4e7c-8fbf-e5326e1f98be 2020-08-18 12:51:34,604Z ERROR [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (default task-6) [19204d8c-ffe6-4c45-81fe-96e07097d519] Failed to add image ticket to ovirt-imageio-proxy: C onnection refused (Connection refused): java.lang.RuntimeException: Connection refused (Connection refused) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.image.ImageioClient.getConnection(ImageioClient.java:114) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.image.ImageioClient.putTicket(ImageioClient.java:65)
is there a way for me to validate where engine thinks the proxy is? the test button passed after changing remote port to 54323... The test button tests that the UI can connect ot the ovirt-imageio for transfering data, but engine connects to localhost:54324 for (control port on engine host) for adding a ticket.
Sounds like you try to use unsupported configuration, it will not work.
If we want to make this work, engine should know that engine host is a hypervisor and disable usage of the proxy, since no proxy is needed when engine host is the hypervisor. But we don't have such configuration.
Nir