ovirt-imageio : can't upload / download
Hi, I have image upload/download working on some older ovirt servers where it still has the split daemon/proxy... on one newer one this feature is not working; software in use: CentOS 8 ovirt-engine-4.4.1.10-1.el8.noarch ovirt-imageio-client-2.0.9-1.el8.x86_64 ovirt-imageio-daemon-2.0.9-1.el8.x86_64 ovirt-imageio-common-2.0.9-1.el8.x86_64 the ui test button allowed me to work out 50-vdsm.conf was setting the wrong remote port... (was 54322, changed to 54323) updated remote with; [remote] host = 0.0.0.0 port = 54323 the test now passes, but on upload or download it still fails. Next i changed the control to be unix socket instead of tcp port 54324 (vdsm was giving an error: Image daemon is unsupported); I looked up the error line in the vdsm code, and found it was looking for unix socket: DAEMON_SOCK=/run/ovirt-imageio/sock switching to sock seemed to resolve all errors in the vdsm log; --- content of the imageio log; no errors as far as i can see: 2020-08-18 12:49:56,109 INFO (MainThread) [server] Starting (pid=2696562, version=2.0.9) 2020-08-18 12:49:56,109 DEBUG (MainThread) [services] Creating remote.service on port 54323 2020-08-18 12:49:56,111 DEBUG (MainThread) [http] Prefer IPv4: False 2020-08-18 12:49:56,111 DEBUG (MainThread) [http] Available network interfaces: [(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('0.0.0.0', 54323))] 2020-08-18 12:49:56,111 DEBUG (MainThread) [http] Creating server socket with family=AddressFamily.AF_INET and type=SocketKind.SOCK_STREAM 2020-08-18 12:49:56,111 DEBUG (MainThread) [services] Securing server (cafile=/etc/pki/vdsm/certs/cacert.pem, certfile=/etc/pki/vdsm/certs/vdsmcert.pem, keyfile=/etc/pki/vdsm/keys/vdsmkey.pem) 2020-08-18 12:49:56,113 INFO (MainThread) [services] remote.service listening on ('0.0.0.0', 54323) 2020-08-18 12:49:56,113 DEBUG (MainThread) [services] Creating local.service on socket '\x00/org/ovirt/imageio' 2020-08-18 12:49:56,113 INFO (MainThread) [services] local.service listening on '\x00/org/ovirt/imageio' 2020-08-18 12:49:56,113 DEBUG (MainThread) [services] Creating control.service on socket '/run/ovirt-imageio/sock' 2020-08-18 12:49:56,113 DEBUG (MainThread) [uhttp] Removing socket '/run/ovirt-imageio/sock' 2020-08-18 12:49:56,113 INFO (MainThread) [services] control.service listening on '/run/ovirt-imageio/sock' 2020-08-18 12:49:56,115 DEBUG (MainThread) [server] Changing ownership of /run/ovirt-imageio to 988:984 2020-08-18 12:49:56,115 DEBUG (MainThread) [server] Changing ownership of /var/log/ovirt-imageio/daemon.log to 988:984 2020-08-18 12:49:56,115 DEBUG (MainThread) [server] Dropping root privileges, running as 988:984 2020-08-18 12:49:56,116 DEBUG (MainThread) [services] Starting remote.service 2020-08-18 12:49:56,116 DEBUG (remote.service) [services] remote.service started 2020-08-18 12:49:56,116 DEBUG (MainThread) [services] Starting local.service 2020-08-18 12:49:56,117 DEBUG (local.service) [services] local.service started 2020-08-18 12:49:56,117 DEBUG (MainThread) [services] Starting control.service 2020-08-18 12:49:56,117 DEBUG (control.service) [services] control.service started 2020-08-18 12:49:56,117 INFO (MainThread) [server] Ready for requests 2020-08-18 12:51:34,602 INFO (Thread-1) [http] OPEN client=local 2020-08-18 12:51:34,603 INFO (Thread-1) [tickets] [local] ADD ticket={'dirty': False, 'ops': ['read'], 'filename': 'debian-10.3.0-amd64-netinst.iso.raw', 'size': 351272960, 'sparse': True, 'transfer_id': 'ec7b329e-3ef1-448d-ac73-1e5e1b4c0ef9', 'uuid': '17db8f9f-3280-4e7c-8fbf-e5326e1f98be', 'timeout': 300, 'url': 'file:///rhev/data-center/mnt/_var_local_ovirt_local__data/7ee98037-126b-4c79-b921-a9ff4272fa91/images/315af697-a064-4c2c-bdf6-8144a8218526/8e709a79-4da0-4d91-803e-d6ec12ec84c3'} 2020-08-18 12:51:34,603 DEBUG (Thread-1) [http] Empty request line, client disconnected 2020-08-18 12:51:34,603 INFO (Thread-1) [http] CLOSE client=local [connection 1 ops, 0.000842 s] [dispatch 1 ops, 0.000305 s] --- vdsm log for transfer id: ec7b329e-3ef1-448d-ac73-1e5e1b4c0ef9 no errors as far as i can see... 2020-08-18 12:51:34,602+0000 INFO (jsonrpc/5) [vdsm.api] START add_image_ticket(ticket={'dirty': False, 'ops': ['read'], 'filename': 'debian-10.3.0-amd64-netinst.iso.raw', 'size': 351272960, 'sparse': True, 'transfer_id': 'ec7b329e-3ef1-448d-ac73-1e5e1b4c0ef9', 'uuid': '17db8f9f-3280-4e7c-8fbf-e5326e1f98be', 'timeout': 300, 'url': 'file:///rhev/data-center/mnt/_var_local_ovirt_local__data/7ee98037-126b-4c79-b921-a9ff4272fa91/images/315af697-a064-4c2c-bdf6-8144a8218526/8e709a79-4da0-4d91-803e-d6ec12ec84c3'}) from=::ffff:37.187.146.167,58882, flow_id=19204d8c-ffe6-4c45-81fe-96e07097d519, task_id=441bf5ea-1e93-4b5b-aeae-f0e52b43cf62 (api:48) 2020-08-18 12:51:34,603+0000 INFO (jsonrpc/5) [vdsm.api] FINISH add_image_ticket return=None from=::ffff:37.187.146.167,58882, flow_id=19204d8c-ffe6-4c45-81fe-96e07097d519, task_id=441bf5ea-1e93-4b5b-aeae-f0e52b43cf62 (api:54) 2020-08-18 12:51:34,603+0000 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.add_image_ticket succeeded in 0.00 seconds (__init__:312) --- engine log: urn [status=Status [code=0, message=Done]], log id: 7ac2f68b 2020-08-18 12:51:34,604Z INFO [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (default task-6) [19204d8c-ffe6-4c45-81fe-96e07097d519] Started transfer session with ticket id 17db8f9f-328 0-4e7c-8fbf-e5326e1f98be, timeout 300 seconds 2020-08-18 12:51:34,604Z INFO [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (default task-6) [19204d8c-ffe6-4c45-81fe-96e07097d519] Adding image ticket to ovirt-imageio-proxy, id 17db8 f9f-3280-4e7c-8fbf-e5326e1f98be 2020-08-18 12:51:34,604Z ERROR [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (default task-6) [19204d8c-ffe6-4c45-81fe-96e07097d519] Failed to add image ticket to ovirt-imageio-proxy: C onnection refused (Connection refused): java.lang.RuntimeException: Connection refused (Connection refused) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.image.ImageioClient.getConnection(ImageioClient.java:114) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.image.ImageioClient.putTicket(ImageioClient.java:65) is there a way for me to validate where engine thinks the proxy is? the test button passed after changing remote port to 54323... any help much appreciated. Kind Regards, Michael Jones
On Tue, Aug 18, 2020 at 4:06 PM Michael Jones <mj@mikejonesey.co.uk> wrote:
I have image upload/download working on some older ovirt servers where it still has the split daemon/proxy...
on one newer one this feature is not working;
software in use:
CentOS 8 ovirt-engine-4.4.1.10-1.el8.noarch ovirt-imageio-client-2.0.9-1.el8.x86_64 ovirt-imageio-daemon-2.0.9-1.el8.x86_64 ovirt-imageio-common-2.0.9-1.el8.x86_64
Both host and engine running these versions?
the ui test button allowed me to work out 50-vdsm.conf was setting the wrong remote port... (was 54322, changed to 54323)
50-vdsm.conf is cannot break the connection test, since it should be installed only on the hosts. On engine you have 50-engine.conf. The correct configuration for engine is: [tls] enable = true key_file = /etc/pki/ovirt-engine/keys/apache.key.nopass cert_file = /etc/pki/ovirt-engine/certs/apache.cer ca_file = /etc/pki/ovirt-engine/apache-ca.pem [remote] port = 54323 [local] enable = false [control] transport = tcp port = 54324 The correct settings for the hosts are: [tls] enable = true key_file = /etc/pki/vdsm/keys/vdsmkey.pem cert_file = /etc/pki/vdsm/certs/vdsmcert.pem ca_file = /etc/pki/vdsm/certs/cacert.pem [remote] port = 54322 These files belongs to engine and vdsm and you should not change them. Your changes will be overwirtten on the next upgrade. The top of the file explain how to change the configuration.
updated remote with;
[remote] host = 0.0.0.0 port = 54323
the test now passes, but on upload or download it still fails.
Do you have 50-vdsm.conf on the engine host?! It sounds like you have all-in-one configuration when engine host is also the single host. This configuration is not supported for 5 years or so. Or you installed vdsm by mistake on engine host, in this case you will have both 50-vdsm.conf and 50-engine.conf, and because "vdsm" sorts after "engine" its configuration will win.
Next i changed the control to be unix socket instead of tcp port 54324 (vdsm was giving an error: Image daemon is unsupported);
I looked up the error line in the vdsm code, and found it was looking for unix socket: DAEMON_SOCK=/run/ovirt-imageio/sock
switching to sock seemed to resolve all errors in the vdsm log;
Expected, using tcp for the host is not supported.
---
content of the imageio log;
imagieo log on engine or host?
no errors as far as i can see:
2020-08-18 12:49:56,109 INFO (MainThread) [server] Starting (pid=2696562, version=2.0.9) 2020-08-18 12:49:56,109 DEBUG (MainThread) [services] Creating remote.service on port 54323 2020-08-18 12:49:56,111 DEBUG (MainThread) [http] Prefer IPv4: False 2020-08-18 12:49:56,111 DEBUG (MainThread) [http] Available network interfaces: [(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('0.0.0.0', 54323))] 2020-08-18 12:49:56,111 DEBUG (MainThread) [http] Creating server socket with family=AddressFamily.AF_INET and type=SocketKind.SOCK_STREAM 2020-08-18 12:49:56,111 DEBUG (MainThread) [services] Securing server (cafile=/etc/pki/vdsm/certs/cacert.pem, certfile=/etc/pki/vdsm/certs/vdsmcert.pem, keyfile=/etc/pki/vdsm/keys/vdsmkey.pem)
So this is log from the host
2020-08-18 12:49:56,113 INFO (MainThread) [services] remote.service listening on ('0.0.0.0', 54323)
This port is wrong, you will not be able to tansfer anything since engine assumes port 54322.
2020-08-18 12:49:56,113 DEBUG (MainThread) [services] Creating local.service on socket '\x00/org/ovirt/imageio' 2020-08-18 12:49:56,113 INFO (MainThread) [services] local.service listening on '\x00/org/ovirt/imageio' 2020-08-18 12:49:56,113 DEBUG (MainThread) [services] Creating control.service on socket '/run/ovirt-imageio/sock' 2020-08-18 12:49:56,113 DEBUG (MainThread) [uhttp] Removing socket '/run/ovirt-imageio/sock' 2020-08-18 12:49:56,113 INFO (MainThread) [services] control.service listening on '/run/ovirt-imageio/sock' 2020-08-18 12:49:56,115 DEBUG (MainThread) [server] Changing ownership of /run/ovirt-imageio to 988:984 2020-08-18 12:49:56,115 DEBUG (MainThread) [server] Changing ownership of /var/log/ovirt-imageio/daemon.log to 988:984 2020-08-18 12:49:56,115 DEBUG (MainThread) [server] Dropping root privileges, running as 988:984 2020-08-18 12:49:56,116 DEBUG (MainThread) [services] Starting remote.service 2020-08-18 12:49:56,116 DEBUG (remote.service) [services] remote.service started 2020-08-18 12:49:56,116 DEBUG (MainThread) [services] Starting local.service 2020-08-18 12:49:56,117 DEBUG (local.service) [services] local.service started 2020-08-18 12:49:56,117 DEBUG (MainThread) [services] Starting control.service 2020-08-18 12:49:56,117 DEBUG (control.service) [services] control.service started 2020-08-18 12:49:56,117 INFO (MainThread) [server] Ready for requests 2020-08-18 12:51:34,602 INFO (Thread-1) [http] OPEN client=local 2020-08-18 12:51:34,603 INFO (Thread-1) [tickets] [local] ADD ticket={'dirty': False, 'ops': ['read'], 'filename': 'debian-10.3.0-amd64-netinst.iso.raw', 'size': 351272960, 'sparse': True, 'transfer_id': 'ec7b329e-3ef1-448d-ac73-1e5e1b4c0ef9', 'uuid': '17db8f9f-3280-4e7c-8fbf-e5326e1f98be', 'timeout': 300, 'url': 'file:///rhev/data-center/mnt/_var_local_ovirt_local__data/7ee98037-126b-4c79-b921-a9ff4272fa91/images/315af697-a064-4c2c-bdf6-8144a8218526/8e709a79-4da0-4d91-803e-d6ec12ec84c3'}
Looks good, this host is using local storage.
2020-08-18 12:51:34,603 DEBUG (Thread-1) [http] Empty request line, client disconnected 2020-08-18 12:51:34,603 INFO (Thread-1) [http] CLOSE client=local [connection 1 ops, 0.000842 s] [dispatch 1 ops, 0.000305 s]
---
vdsm log for transfer id: ec7b329e-3ef1-448d-ac73-1e5e1b4c0ef9
no errors as far as i can see...
2020-08-18 12:51:34,602+0000 INFO (jsonrpc/5) [vdsm.api] START add_image_ticket(ticket={'dirty': False, 'ops': ['read'], 'filename': 'debian-10.3.0-amd64-netinst.iso.raw', 'size': 351272960, 'sparse': True, 'transfer_id': 'ec7b329e-3ef1-448d-ac73-1e5e1b4c0ef9', 'uuid': '17db8f9f-3280-4e7c-8fbf-e5326e1f98be', 'timeout': 300, 'url': 'file:///rhev/data-center/mnt/_var_local_ovirt_local__data/7ee98037-126b-4c79-b921-a9ff4272fa91/images/315af697-a064-4c2c-bdf6-8144a8218526/8e709a79-4da0-4d91-803e-d6ec12ec84c3'}) from=::ffff:37.187.146.167,58882, flow_id=19204d8c-ffe6-4c45-81fe-96e07097d519, task_id=441bf5ea-1e93-4b5b-aeae-f0e52b43cf62 (api:48) 2020-08-18 12:51:34,603+0000 INFO (jsonrpc/5) [vdsm.api] FINISH add_image_ticket return=None from=::ffff:37.187.146.167,58882, flow_id=19204d8c-ffe6-4c45-81fe-96e07097d519, task_id=441bf5ea-1e93-4b5b-aeae-f0e52b43cf62 (api:54) 2020-08-18 12:51:34,603+0000 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.add_image_ticket succeeded in 0.00 seconds (__init__:312)
---
engine log:
urn [status=Status [code=0, message=Done]], log id: 7ac2f68b 2020-08-18 12:51:34,604Z INFO [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (default task-6) [19204d8c-ffe6-4c45-81fe-96e07097d519] Started transfer session with ticket id 17db8f9f-328 0-4e7c-8fbf-e5326e1f98be, timeout 300 seconds 2020-08-18 12:51:34,604Z INFO [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (default task-6) [19204d8c-ffe6-4c45-81fe-96e07097d519] Adding image ticket to ovirt-imageio-proxy, id 17db8 f9f-3280-4e7c-8fbf-e5326e1f98be 2020-08-18 12:51:34,604Z ERROR [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (default task-6) [19204d8c-ffe6-4c45-81fe-96e07097d519] Failed to add image ticket to ovirt-imageio-proxy: C onnection refused (Connection refused): java.lang.RuntimeException: Connection refused (Connection refused) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.image.ImageioClient.getConnection(ImageioClient.java:114) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.image.ImageioClient.putTicket(ImageioClient.java:65)
is there a way for me to validate where engine thinks the proxy is? the test button passed after changing remote port to 54323...
The test button tests that the UI can connect ot the ovirt-imageio for transfering data, but engine connects to localhost:54324 for (control port on engine host) for adding a ticket. Sounds like you try to use unsupported configuration, it will not work. If we want to make this work, engine should know that engine host is a hypervisor and disable usage of the proxy, since no proxy is needed when engine host is the hypervisor. But we don't have such configuration. Nir
Hi Nir, Thanks for the help, not sure if this is a top post or bottom post mail list, feel free to tell me off; it does sound as if, my problems are around the fact that i am using an all-in-one box, (host and engine all in one); https://www.ovirt.org/download/alternate_downloads.html I'm about to setup another server, so i'll perhaps try engine as vm setup through the cockpit installer, which i've only used on ovirt clusters so far, not on single host machines. If that works, perhaps i'll work out a way to port the server i'm having issues with. the download / upload function is important to me as my backup solution is dependent on this. Thanks, Kind Regards, Michael Jones On 18/08/2020 16:55, Nir Soffer wrote:
On Tue, Aug 18, 2020 at 4:06 PM Michael Jones <mj@mikejonesey.co.uk> wrote:
I have image upload/download working on some older ovirt servers where it still has the split daemon/proxy...
on one newer one this feature is not working;
software in use:
CentOS 8 ovirt-engine-4.4.1.10-1.el8.noarch ovirt-imageio-client-2.0.9-1.el8.x86_64 ovirt-imageio-daemon-2.0.9-1.el8.x86_64 ovirt-imageio-common-2.0.9-1.el8.x86_64 Both host and engine running these versions?
the ui test button allowed me to work out 50-vdsm.conf was setting the wrong remote port... (was 54322, changed to 54323) 50-vdsm.conf is cannot break the connection test, since it should be installed only on the hosts. On engine you have 50-engine.conf.
The correct configuration for engine is:
[tls] enable = true key_file = /etc/pki/ovirt-engine/keys/apache.key.nopass cert_file = /etc/pki/ovirt-engine/certs/apache.cer ca_file = /etc/pki/ovirt-engine/apache-ca.pem
[remote] port = 54323
[local] enable = false
[control] transport = tcp port = 54324
The correct settings for the hosts are:
[tls] enable = true key_file = /etc/pki/vdsm/keys/vdsmkey.pem cert_file = /etc/pki/vdsm/certs/vdsmcert.pem ca_file = /etc/pki/vdsm/certs/cacert.pem
[remote] port = 54322
These files belongs to engine and vdsm and you should not change them. Your changes will be overwirtten on the next upgrade.
The top of the file explain how to change the configuration.
updated remote with;
[remote] host = 0.0.0.0 port = 54323
the test now passes, but on upload or download it still fails. Do you have 50-vdsm.conf on the engine host?!
It sounds like you have all-in-one configuration when engine host is also the single host. This configuration is not supported for 5 years or so.
Or you installed vdsm by mistake on engine host, in this case you will have both 50-vdsm.conf and 50-engine.conf, and because "vdsm" sorts after "engine" its configuration will win.
Next i changed the control to be unix socket instead of tcp port 54324 (vdsm was giving an error: Image daemon is unsupported);
I looked up the error line in the vdsm code, and found it was looking for unix socket: DAEMON_SOCK=/run/ovirt-imageio/sock
switching to sock seemed to resolve all errors in the vdsm log; Expected, using tcp for the host is not supported.
---
content of the imageio log; imagieo log on engine or host?
no errors as far as i can see:
2020-08-18 12:49:56,109 INFO (MainThread) [server] Starting (pid=2696562, version=2.0.9) 2020-08-18 12:49:56,109 DEBUG (MainThread) [services] Creating remote.service on port 54323 2020-08-18 12:49:56,111 DEBUG (MainThread) [http] Prefer IPv4: False 2020-08-18 12:49:56,111 DEBUG (MainThread) [http] Available network interfaces: [(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('0.0.0.0', 54323))] 2020-08-18 12:49:56,111 DEBUG (MainThread) [http] Creating server socket with family=AddressFamily.AF_INET and type=SocketKind.SOCK_STREAM 2020-08-18 12:49:56,111 DEBUG (MainThread) [services] Securing server (cafile=/etc/pki/vdsm/certs/cacert.pem, certfile=/etc/pki/vdsm/certs/vdsmcert.pem, keyfile=/etc/pki/vdsm/keys/vdsmkey.pem) So this is log from the host
2020-08-18 12:49:56,113 INFO (MainThread) [services] remote.service listening on ('0.0.0.0', 54323) This port is wrong, you will not be able to tansfer anything since engine assumes port 54322.
2020-08-18 12:49:56,113 DEBUG (MainThread) [services] Creating local.service on socket '\x00/org/ovirt/imageio' 2020-08-18 12:49:56,113 INFO (MainThread) [services] local.service listening on '\x00/org/ovirt/imageio' 2020-08-18 12:49:56,113 DEBUG (MainThread) [services] Creating control.service on socket '/run/ovirt-imageio/sock' 2020-08-18 12:49:56,113 DEBUG (MainThread) [uhttp] Removing socket '/run/ovirt-imageio/sock' 2020-08-18 12:49:56,113 INFO (MainThread) [services] control.service listening on '/run/ovirt-imageio/sock' 2020-08-18 12:49:56,115 DEBUG (MainThread) [server] Changing ownership of /run/ovirt-imageio to 988:984 2020-08-18 12:49:56,115 DEBUG (MainThread) [server] Changing ownership of /var/log/ovirt-imageio/daemon.log to 988:984 2020-08-18 12:49:56,115 DEBUG (MainThread) [server] Dropping root privileges, running as 988:984 2020-08-18 12:49:56,116 DEBUG (MainThread) [services] Starting remote.service 2020-08-18 12:49:56,116 DEBUG (remote.service) [services] remote.service started 2020-08-18 12:49:56,116 DEBUG (MainThread) [services] Starting local.service 2020-08-18 12:49:56,117 DEBUG (local.service) [services] local.service started 2020-08-18 12:49:56,117 DEBUG (MainThread) [services] Starting control.service 2020-08-18 12:49:56,117 DEBUG (control.service) [services] control.service started 2020-08-18 12:49:56,117 INFO (MainThread) [server] Ready for requests 2020-08-18 12:51:34,602 INFO (Thread-1) [http] OPEN client=local 2020-08-18 12:51:34,603 INFO (Thread-1) [tickets] [local] ADD ticket={'dirty': False, 'ops': ['read'], 'filename': 'debian-10.3.0-amd64-netinst.iso.raw', 'size': 351272960, 'sparse': True, 'transfer_id': 'ec7b329e-3ef1-448d-ac73-1e5e1b4c0ef9', 'uuid': '17db8f9f-3280-4e7c-8fbf-e5326e1f98be', 'timeout': 300, 'url': 'file:///rhev/data-center/mnt/_var_local_ovirt_local__data/7ee98037-126b-4c79-b921-a9ff4272fa91/images/315af697-a064-4c2c-bdf6-8144a8218526/8e709a79-4da0-4d91-803e-d6ec12ec84c3'} Looks good, this host is using local storage.
2020-08-18 12:51:34,603 DEBUG (Thread-1) [http] Empty request line, client disconnected 2020-08-18 12:51:34,603 INFO (Thread-1) [http] CLOSE client=local [connection 1 ops, 0.000842 s] [dispatch 1 ops, 0.000305 s]
---
vdsm log for transfer id: ec7b329e-3ef1-448d-ac73-1e5e1b4c0ef9
no errors as far as i can see...
2020-08-18 12:51:34,602+0000 INFO (jsonrpc/5) [vdsm.api] START add_image_ticket(ticket={'dirty': False, 'ops': ['read'], 'filename': 'debian-10.3.0-amd64-netinst.iso.raw', 'size': 351272960, 'sparse': True, 'transfer_id': 'ec7b329e-3ef1-448d-ac73-1e5e1b4c0ef9', 'uuid': '17db8f9f-3280-4e7c-8fbf-e5326e1f98be', 'timeout': 300, 'url': 'file:///rhev/data-center/mnt/_var_local_ovirt_local__data/7ee98037-126b-4c79-b921-a9ff4272fa91/images/315af697-a064-4c2c-bdf6-8144a8218526/8e709a79-4da0-4d91-803e-d6ec12ec84c3'}) from=::ffff:37.187.146.167,58882, flow_id=19204d8c-ffe6-4c45-81fe-96e07097d519, task_id=441bf5ea-1e93-4b5b-aeae-f0e52b43cf62 (api:48) 2020-08-18 12:51:34,603+0000 INFO (jsonrpc/5) [vdsm.api] FINISH add_image_ticket return=None from=::ffff:37.187.146.167,58882, flow_id=19204d8c-ffe6-4c45-81fe-96e07097d519, task_id=441bf5ea-1e93-4b5b-aeae-f0e52b43cf62 (api:54) 2020-08-18 12:51:34,603+0000 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.add_image_ticket succeeded in 0.00 seconds (__init__:312)
---
engine log:
urn [status=Status [code=0, message=Done]], log id: 7ac2f68b 2020-08-18 12:51:34,604Z INFO [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (default task-6) [19204d8c-ffe6-4c45-81fe-96e07097d519] Started transfer session with ticket id 17db8f9f-328 0-4e7c-8fbf-e5326e1f98be, timeout 300 seconds 2020-08-18 12:51:34,604Z INFO [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (default task-6) [19204d8c-ffe6-4c45-81fe-96e07097d519] Adding image ticket to ovirt-imageio-proxy, id 17db8 f9f-3280-4e7c-8fbf-e5326e1f98be 2020-08-18 12:51:34,604Z ERROR [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (default task-6) [19204d8c-ffe6-4c45-81fe-96e07097d519] Failed to add image ticket to ovirt-imageio-proxy: C onnection refused (Connection refused): java.lang.RuntimeException: Connection refused (Connection refused) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.image.ImageioClient.getConnection(ImageioClient.java:114) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.image.ImageioClient.putTicket(ImageioClient.java:65)
is there a way for me to validate where engine thinks the proxy is? the test button passed after changing remote port to 54323... The test button tests that the UI can connect ot the ovirt-imageio for transfering data, but engine connects to localhost:54324 for (control port on engine host) for adding a ticket.
Sounds like you try to use unsupported configuration, it will not work.
If we want to make this work, engine should know that engine host is a hypervisor and disable usage of the proxy, since no proxy is needed when engine host is the hypervisor. But we don't have such configuration.
Nir
On Tue, Aug 18, 2020 at 7:10 PM Michael Jones <mj@mikejonesey.co.uk> wrote:
Thanks for the help, not sure if this is a top post or bottom post mail list, feel free to tell me off;
We don't have strict rules, but I think that bottom posting will be better, even better selecting the relevant parts in your reply.
it does sound as if, my problems are around the fact that i am using an all-in-one box, (host and engine all in one);
This explains how that you can install all-in-one when engine is a VM running on the single host, not as a program running on the host. How did you managed to get engine installed on the same host? I would expect that the installer would fail or at least warn about this.
I'm about to setup another server, so i'll perhaps try engine as vm setup through the cockpit installer, which i've only used on ovirt clusters so far, not on single host machines.
If that works, perhaps i'll work out a way to port the server i'm having issues with.
Taking a backup of engine database, installing engine on another host or as vm, and restoring the backup should be the easiest way to move the engine.
the download / upload function is important to me as my backup solution is dependent on this.
Until the sort this out, you should know that you can transfer images without the proxy. The proxy is needed only for the UI, for cases when engine and hosts are on different networks, so the only way to transfer images is via the engine host. To get this working on your setup, you need to restore 50-vdsm.conf to the default configuration and restart ovirt-imageio service. There is no way to disable the proxy in engine, so for every transfer engine will try to add a ticket the proxy and will fail, but this should not fail the transfer, only log the error. To transfer images, use upload_disk.py and download_disk.py from ovirt-engine-sdk. dnf install python3-ovirt-enigne-sdk4 And try: python3 /usr/share/doc/python3-ovirt-engine-sdk4/examples/upload_disk.py -h python3 /usr/share/doc/python3-ovirt-engine-sdk4/examples/download_disk.py -h There is also a backup_vm.py example that can be very useful for backup, supporting both full and incremental backup: python3 /usr/share/doc/python3-ovirt-engine-sdk4/examples/backup_vm.py -h Nir
Thanks,
Kind Regards,
Michael Jones
On 18/08/2020 16:55, Nir Soffer wrote:
On Tue, Aug 18, 2020 at 4:06 PM Michael Jones <mj@mikejonesey.co.uk> wrote:
I have image upload/download working on some older ovirt servers where it still has the split daemon/proxy...
on one newer one this feature is not working;
software in use:
CentOS 8 ovirt-engine-4.4.1.10-1.el8.noarch ovirt-imageio-client-2.0.9-1.el8.x86_64 ovirt-imageio-daemon-2.0.9-1.el8.x86_64 ovirt-imageio-common-2.0.9-1.el8.x86_64 Both host and engine running these versions?
the ui test button allowed me to work out 50-vdsm.conf was setting the wrong remote port... (was 54322, changed to 54323) 50-vdsm.conf is cannot break the connection test, since it should be installed only on the hosts. On engine you have 50-engine.conf.
The correct configuration for engine is:
[tls] enable = true key_file = /etc/pki/ovirt-engine/keys/apache.key.nopass cert_file = /etc/pki/ovirt-engine/certs/apache.cer ca_file = /etc/pki/ovirt-engine/apache-ca.pem
[remote] port = 54323
[local] enable = false
[control] transport = tcp port = 54324
The correct settings for the hosts are:
[tls] enable = true key_file = /etc/pki/vdsm/keys/vdsmkey.pem cert_file = /etc/pki/vdsm/certs/vdsmcert.pem ca_file = /etc/pki/vdsm/certs/cacert.pem
[remote] port = 54322
These files belongs to engine and vdsm and you should not change them. Your changes will be overwirtten on the next upgrade.
The top of the file explain how to change the configuration.
updated remote with;
[remote] host = 0.0.0.0 port = 54323
the test now passes, but on upload or download it still fails. Do you have 50-vdsm.conf on the engine host?!
It sounds like you have all-in-one configuration when engine host is also the single host. This configuration is not supported for 5 years or so.
Or you installed vdsm by mistake on engine host, in this case you will have both 50-vdsm.conf and 50-engine.conf, and because "vdsm" sorts after "engine" its configuration will win.
Next i changed the control to be unix socket instead of tcp port 54324 (vdsm was giving an error: Image daemon is unsupported);
I looked up the error line in the vdsm code, and found it was looking for unix socket: DAEMON_SOCK=/run/ovirt-imageio/sock
switching to sock seemed to resolve all errors in the vdsm log; Expected, using tcp for the host is not supported.
---
content of the imageio log; imagieo log on engine or host?
no errors as far as i can see:
2020-08-18 12:49:56,109 INFO (MainThread) [server] Starting (pid=2696562, version=2.0.9) 2020-08-18 12:49:56,109 DEBUG (MainThread) [services] Creating remote.service on port 54323 2020-08-18 12:49:56,111 DEBUG (MainThread) [http] Prefer IPv4: False 2020-08-18 12:49:56,111 DEBUG (MainThread) [http] Available network interfaces: [(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('0.0.0.0', 54323))] 2020-08-18 12:49:56,111 DEBUG (MainThread) [http] Creating server socket with family=AddressFamily.AF_INET and type=SocketKind.SOCK_STREAM 2020-08-18 12:49:56,111 DEBUG (MainThread) [services] Securing server (cafile=/etc/pki/vdsm/certs/cacert.pem, certfile=/etc/pki/vdsm/certs/vdsmcert.pem, keyfile=/etc/pki/vdsm/keys/vdsmkey.pem) So this is log from the host
2020-08-18 12:49:56,113 INFO (MainThread) [services] remote.service listening on ('0.0.0.0', 54323) This port is wrong, you will not be able to tansfer anything since engine assumes port 54322.
2020-08-18 12:49:56,113 DEBUG (MainThread) [services] Creating local.service on socket '\x00/org/ovirt/imageio' 2020-08-18 12:49:56,113 INFO (MainThread) [services] local.service listening on '\x00/org/ovirt/imageio' 2020-08-18 12:49:56,113 DEBUG (MainThread) [services] Creating control.service on socket '/run/ovirt-imageio/sock' 2020-08-18 12:49:56,113 DEBUG (MainThread) [uhttp] Removing socket '/run/ovirt-imageio/sock' 2020-08-18 12:49:56,113 INFO (MainThread) [services] control.service listening on '/run/ovirt-imageio/sock' 2020-08-18 12:49:56,115 DEBUG (MainThread) [server] Changing ownership of /run/ovirt-imageio to 988:984 2020-08-18 12:49:56,115 DEBUG (MainThread) [server] Changing ownership of /var/log/ovirt-imageio/daemon.log to 988:984 2020-08-18 12:49:56,115 DEBUG (MainThread) [server] Dropping root privileges, running as 988:984 2020-08-18 12:49:56,116 DEBUG (MainThread) [services] Starting remote.service 2020-08-18 12:49:56,116 DEBUG (remote.service) [services] remote.service started 2020-08-18 12:49:56,116 DEBUG (MainThread) [services] Starting local.service 2020-08-18 12:49:56,117 DEBUG (local.service) [services] local.service started 2020-08-18 12:49:56,117 DEBUG (MainThread) [services] Starting control.service 2020-08-18 12:49:56,117 DEBUG (control.service) [services] control.service started 2020-08-18 12:49:56,117 INFO (MainThread) [server] Ready for requests 2020-08-18 12:51:34,602 INFO (Thread-1) [http] OPEN client=local 2020-08-18 12:51:34,603 INFO (Thread-1) [tickets] [local] ADD ticket={'dirty': False, 'ops': ['read'], 'filename': 'debian-10.3.0-amd64-netinst.iso.raw', 'size': 351272960, 'sparse': True, 'transfer_id': 'ec7b329e-3ef1-448d-ac73-1e5e1b4c0ef9', 'uuid': '17db8f9f-3280-4e7c-8fbf-e5326e1f98be', 'timeout': 300, 'url': 'file:///rhev/data-center/mnt/_var_local_ovirt_local__data/7ee98037-126b-4c79-b921-a9ff4272fa91/images/315af697-a064-4c2c-bdf6-8144a8218526/8e709a79-4da0-4d91-803e-d6ec12ec84c3'} Looks good, this host is using local storage.
2020-08-18 12:51:34,603 DEBUG (Thread-1) [http] Empty request line, client disconnected 2020-08-18 12:51:34,603 INFO (Thread-1) [http] CLOSE client=local [connection 1 ops, 0.000842 s] [dispatch 1 ops, 0.000305 s]
---
vdsm log for transfer id: ec7b329e-3ef1-448d-ac73-1e5e1b4c0ef9
no errors as far as i can see...
2020-08-18 12:51:34,602+0000 INFO (jsonrpc/5) [vdsm.api] START add_image_ticket(ticket={'dirty': False, 'ops': ['read'], 'filename': 'debian-10.3.0-amd64-netinst.iso.raw', 'size': 351272960, 'sparse': True, 'transfer_id': 'ec7b329e-3ef1-448d-ac73-1e5e1b4c0ef9', 'uuid': '17db8f9f-3280-4e7c-8fbf-e5326e1f98be', 'timeout': 300, 'url': 'file:///rhev/data-center/mnt/_var_local_ovirt_local__data/7ee98037-126b-4c79-b921-a9ff4272fa91/images/315af697-a064-4c2c-bdf6-8144a8218526/8e709a79-4da0-4d91-803e-d6ec12ec84c3'}) from=::ffff:37.187.146.167,58882, flow_id=19204d8c-ffe6-4c45-81fe-96e07097d519, task_id=441bf5ea-1e93-4b5b-aeae-f0e52b43cf62 (api:48) 2020-08-18 12:51:34,603+0000 INFO (jsonrpc/5) [vdsm.api] FINISH add_image_ticket return=None from=::ffff:37.187.146.167,58882, flow_id=19204d8c-ffe6-4c45-81fe-96e07097d519, task_id=441bf5ea-1e93-4b5b-aeae-f0e52b43cf62 (api:54) 2020-08-18 12:51:34,603+0000 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.add_image_ticket succeeded in 0.00 seconds (__init__:312)
---
engine log:
urn [status=Status [code=0, message=Done]], log id: 7ac2f68b 2020-08-18 12:51:34,604Z INFO [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (default task-6) [19204d8c-ffe6-4c45-81fe-96e07097d519] Started transfer session with ticket id 17db8f9f-328 0-4e7c-8fbf-e5326e1f98be, timeout 300 seconds 2020-08-18 12:51:34,604Z INFO [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (default task-6) [19204d8c-ffe6-4c45-81fe-96e07097d519] Adding image ticket to ovirt-imageio-proxy, id 17db8 f9f-3280-4e7c-8fbf-e5326e1f98be 2020-08-18 12:51:34,604Z ERROR [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (default task-6) [19204d8c-ffe6-4c45-81fe-96e07097d519] Failed to add image ticket to ovirt-imageio-proxy: C onnection refused (Connection refused): java.lang.RuntimeException: Connection refused (Connection refused) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.image.ImageioClient.getConnection(ImageioClient.java:114) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.image.ImageioClient.putTicket(ImageioClient.java:65)
is there a way for me to validate where engine thinks the proxy is? the test button passed after changing remote port to 54323... The test button tests that the UI can connect ot the ovirt-imageio for transfering data, but engine connects to localhost:54324 for (control port on engine host) for adding a ticket.
Sounds like you try to use unsupported configuration, it will not work.
If we want to make this work, engine should know that engine host is a hypervisor and disable usage of the proxy, since no proxy is needed when engine host is the hypervisor. But we don't have such configuration.
Nir
On 18/08/2020 17:58, Nir Soffer wrote:
it does sound as if, my problems are around the fact that i am using an all-in-one box, (host and engine all in one);
https://www.ovirt.org/download/alternate_downloads.html This explains how that you can install all-in-one when engine is a VM running on the single host, not as a program running on the host.
How did you managed to get engine installed on the same host?
I would expect that the installer would fail or at least warn about this.
The alternate install method is essentially, install packages and run engine-setup, which doesn't setup the hosted vm. The default installer via cockpit always installs engine as vm. perhaps a warning is needed on the alternate installer page for epel8. at the moment there is only a recommendation to use the normal installer. On 18/08/2020 17:58, Nir Soffer wrote:
the download / upload function is important to me as my backup solution is dependent on this. Until the sort this out, you should know that you can transfer images without the proxy. The proxy is needed only for the UI, for cases when engine and hosts are on different networks, so the only way to transfer images is via the engine host.
I use vprotect which i think is dependent on the proxy, but I'll definitely checkout the sdk/scripts you linked. I'll post an update once the new host is setup the normal way / working. Thanks again. Michael Jones
On Tue, Aug 18, 2020 at 8:51 PM Michael Jones <mj@mikejonesey.co.uk> wrote:
On 18/08/2020 17:58, Nir Soffer wrote:
it does sound as if, my problems are around the fact that i am using an all-in-one box, (host and engine all in one);
https://www.ovirt.org/download/alternate_downloads.html
This explains how that you can install all-in-one when engine is a VM running on the single host, not as a program running on the host.
Let's clarify the terminology first. I admit we are not always super-clear about this. - Standalone engine - Engine is running on some machine, which it does not manage by itself. Normally, this is a physical machine, but can be a VM managed by something else (virsh/virt-manager, another ovirt engine, vmware/virtualbox/hyperv/xen, etc.). - Hosted-engine - An engine that is running in a VM, that runs inside a host, that this engine manages. If it sounds like a chicken-and-egg problem, it indeed is... See documentation and some presentation slides on the website for the architecture, if interested. - All-In-One - a Standalone engine that also manages, as a host, the machine on which it runs. This used to have official support in the past, in terms of code helping to implement it (in engine-setup): https://www.ovirt.org/develop/release-management/features/integration/allino... As above page states, this is long gone. However, over the years, people did report successes in doing this manually - install and setup an engine, then add it to itself. I agree it would be nice to keep it working, and the discussion below indeed clarifies that it's currently broken, but this is definitely very low priority IMO. The official answer to the question "How can I setup oVirt on a single machine?" is: Use Hosted-engine with gluster, a.k.a HCI.
How did you managed to get engine installed on the same host?
I would expect that the installer would fail or at least warn about this.
The alternate install method is essentially, install packages and run engine-setup, which doesn't setup the hosted vm. The default installer via cockpit always installs engine as vm.
perhaps a warning is needed on the alternate installer page for epel8.
at the moment there is only a recommendation to use the normal installer.
On 18/08/2020 17:58, Nir Soffer wrote:
the download / upload function is important to me as my backup solution is dependent on this.
Until the sort this out, you should know that you can transfer images without the proxy. The proxy is needed only for the UI, for cases when engine and hosts are on different networks, so the only way to transfer images is via the engine host.
I use vprotect which i think is dependent on the proxy, but I'll definitely checkout the sdk/scripts you linked.
I'll post an update once the new host is setup the normal way / working.
Thanks again.
Michael Jones
-- Didi
On 19/08/2020 06:52, Yedidyah Bar David wrote:
On Tue, Aug 18, 2020 at 8:51 PM Michael Jones <mj@mikejonesey.co.uk> wrote:
On 18/08/2020 17:58, Nir Soffer wrote:
it does sound as if, my problems are around the fact that i am using an all-in-one box, (host and engine all in one);
https://www.ovirt.org/download/alternate_downloads.html
This explains how that you can install all-in-one when engine is a VM running on the single host, not as a program running on the host. Let's clarify the terminology first. I admit we are not always super-clear about this.
- Standalone engine - Engine is running on some machine, which it does not manage by itself. Normally, this is a physical machine, but can be a VM managed by something else (virsh/virt-manager, another ovirt engine, vmware/virtualbox/hyperv/xen, etc.).
- Hosted-engine - An engine that is running in a VM, that runs inside a host, that this engine manages. If it sounds like a chicken-and-egg problem, it indeed is... See documentation and some presentation slides on the website for the architecture, if interested.
- All-In-One - a Standalone engine that also manages, as a host, the machine on which it runs. This used to have official support in the past, in terms of code helping to implement it (in engine-setup):
https://www.ovirt.org/develop/release-management/features/integration/allino...
As above page states, this is long gone. However, over the years, people did report successes in doing this manually - install and setup an engine, then add it to itself. I agree it would be nice to keep it working, and the discussion below indeed clarifies that it's currently broken, but this is definitely very low priority IMO. The official answer to the question "How can I setup oVirt on a single machine?" is: Use Hosted-engine with gluster, a.k.a HCI.
from the link you shared, the status is; Current status * Included since 3.1 * Deprecated since 3.6.0 * Removed since 4.0.0 I think there should be a warning that the install is deprecated, traversing; - https://www.ovirt.org/ - https://www.ovirt.org/download/ (download button) - https://www.ovirt.org/download/alternate_downloads.html (Alternate download options <https://www.ovirt.org/download/alternate_downloads.html>) There is no warning the method was deprecated and will be missing functionality. The steps detailed on the alt install page are for the all-in-one running engine-setup. It's also worth noting this works fine in; Version 4.3.1.1-1.el7 but not in; Version 4.4.1.10-1.el8 (el8 has the change in imageio daemons) The alternate install method is still useful to have, but i think a red warning about all-in-one on el8 on that page would be good. Kind Regards, Michael Jones
On Wed, Aug 19, 2020 at 12:29 PM Michael Jones <mj@mikejonesey.co.uk> wrote:
On 19/08/2020 06:52, Yedidyah Bar David wrote:
On Tue, Aug 18, 2020 at 8:51 PM Michael Jones <mj@mikejonesey.co.uk> wrote:
On 18/08/2020 17:58, Nir Soffer wrote:
it does sound as if, my problems are around the fact that i am using an all-in-one box, (host and engine all in one);
https://www.ovirt.org/download/alternate_downloads.html
This explains how that you can install all-in-one when engine is a VM running on the single host, not as a program running on the host.
Let's clarify the terminology first. I admit we are not always super-clear about this.
- Standalone engine - Engine is running on some machine, which it does not manage by itself. Normally, this is a physical machine, but can be a VM managed by something else (virsh/virt-manager, another ovirt engine, vmware/virtualbox/hyperv/xen, etc.).
- Hosted-engine - An engine that is running in a VM, that runs inside a host, that this engine manages. If it sounds like a chicken-and-egg problem, it indeed is... See documentation and some presentation slides on the website for the architecture, if interested.
- All-In-One - a Standalone engine that also manages, as a host, the machine on which it runs. This used to have official support in the past, in terms of code helping to implement it (in engine-setup):
https://www.ovirt.org/develop/release-management/features/integration/allino...
As above page states, this is long gone. However, over the years, people did report successes in doing this manually - install and setup an engine, then add it to itself. I agree it would be nice to keep it working, and the discussion below indeed clarifies that it's currently broken, but this is definitely very low priority IMO. The official answer to the question "How can I setup oVirt on a single machine?" is: Use Hosted-engine with gluster, a.k.a HCI.
from the link you shared, the status is;
Current status
Included since 3.1 Deprecated since 3.6.0 Removed since 4.0.0
I think there should be a warning that the install is deprecated, traversing;
- https://www.ovirt.org/ - https://www.ovirt.org/download/ (download button) - https://www.ovirt.org/download/alternate_downloads.html (Alternate download options)
There is no warning the method was deprecated and will be missing functionality.
The steps detailed on the alt install page are for the all-in-one running engine-setup.
It's also worth noting this works fine in;
Version 4.3.1.1-1.el7
but not in;
Version 4.4.1.10-1.el8
(el8 has the change in imageio daemons)
The alternate install method is still useful to have, but i think a red warning about all-in-one on el8 on that page would be good.
Kind Regards, Michael Jones
Micheal, can you file a bug for this? If you have a good use case for all-in-one deployment (not using hosted engine), please explain it in the bug. Personally I think simple all-in-one deployment without the complexity of hosted engine is better, and we should keep it, but for this we need to teach engine to handle the case when the proxy and the daemon are the same server. In this case engine will not try to setup a proxy ticket, and image transfer would work directly with the host daemon. I'm not very optimistic that we will support this again, since this feature is not needed for RHV customers, but for oVirt this makes sense. Nir
On 19/08/2020 10:41, Nir Soffer wrote:
There is no warning the method was deprecated and will be missing functionality.
The steps detailed on the alt install page are for the all-in-one running engine-setup.
It's also worth noting this works fine in;
Version 4.3.1.1-1.el7
but not in;
Version 4.4.1.10-1.el8
(el8 has the change in imageio daemons)
The alternate install method is still useful to have, but i think a red warning about all-in-one on el8 on that page would be good.
Kind Regards, Michael Jones Micheal, can you file a bug for this?
If you have a good use case for all-in-one deployment (not using hosted engine), please explain it in the bug.
Personally I think simple all-in-one deployment without the complexity of hosted engine is better, and we should keep it, but for this we need to teach engine to handle the case when the proxy and the daemon are the same server.
In this case engine will not try to setup a proxy ticket, and image transfer would work directly with the host daemon.
I'm not very optimistic that we will support this again, since this feature is not needed for RHV customers, but for oVirt this makes sense.
Nir
Yes, I can file a bug, The main usage / setup's I have are; on-prem installs: - hosted engine - gluster - high availiblity - internal ip address - easy great... dedicated host provider for example OVH single machine: - alternate install - all-in-one The main reason for the separation is that using the cockpit install / hosted engine install causes problems with ip allocations; cockpit method requires 1x ip for host, and 1x ip for engine vm, and both ip must be in the same subnet... applying internal ip would cut off access, and to make it even harder, getting public ip blocks didn't work as the box main ip wouldn't be in the same subnet, adding nic alias ip doesn't work either (fail on install due to failing to setup ovirtmgmt network). atm, i'll struggle with changing the machine's main ip to be one of the same subnet with the engine one... (currently causes host to be taken offline due to hosting provider, health checks) provided i can change the host primary ip to be one of the OVH failover ip allocated in a block... i will be able to install using the cockpit. and after the install i can setup internal ip with the network tag. Kind Regards, Mike
On 19/08/2020 12:12, Michael Jones wrote:
On 19/08/2020 10:41, Nir Soffer wrote:
There is no warning the method was deprecated and will be missing functionality.
The steps detailed on the alt install page are for the all-in-one running engine-setup.
It's also worth noting this works fine in;
Version 4.3.1.1-1.el7
but not in;
Version 4.4.1.10-1.el8
(el8 has the change in imageio daemons)
The alternate install method is still useful to have, but i think a red warning about all-in-one on el8 on that page would be good.
Kind Regards, Michael Jones Micheal, can you file a bug for this?
If you have a good use case for all-in-one deployment (not using hosted engine), please explain it in the bug.
Personally I think simple all-in-one deployment without the complexity of hosted engine is better, and we should keep it, but for this we need to teach engine to handle the case when the proxy and the daemon are the same server.
In this case engine will not try to setup a proxy ticket, and image transfer would work directly with the host daemon.
I'm not very optimistic that we will support this again, since this feature is not needed for RHV customers, but for oVirt this makes sense.
Nir
Yes, I can file a bug,
The main usage / setup's I have are;
on-prem installs:
- hosted engine - gluster - high availiblity - internal ip address - easy great...
dedicated host provider for example OVH single machine:
- alternate install - all-in-one
The main reason for the separation is that using the cockpit install / hosted engine install causes problems with ip allocations;
cockpit method requires 1x ip for host, and 1x ip for engine vm, and both ip must be in the same subnet...
applying internal ip would cut off access, and to make it even harder, getting public ip blocks didn't work as the box main ip wouldn't be in the same subnet, adding nic alias ip doesn't work either (fail on install due to failing to setup ovirtmgmt network).
atm, i'll struggle with changing the machine's main ip to be one of the same subnet with the engine one... (currently causes host to be taken offline due to hosting provider, health checks)
provided i can change the host primary ip to be one of the OVH failover ip allocated in a block... i will be able to install using the cockpit.
and after the install i can setup internal ip with the network tag.
Kind Regards,
Mike
despite managing to get OVH to disable monitoring (ping to the main ip, and rebooting host) and getting the host in the same ip range as the engine vm... ie: host ip: 158.x.x.13/32 = not used anymore new subnet: 54.x.x.x/28 and reserving; host = 54.x.x.16 engine = 54.x.x.17 [ ERROR ] The Engine VM (54.x.x.17/28) and the default gateway (158.x.x.254) will not be in the same IP subnet. the hosted engine installer crashes due to the gw being in a different subnet, so all three; - host - engine - gateway must be in the same subnet... this rules out an install on ovh dedicated server. unless... I can install the all-in-one again (this bit works), and then install the engine vm in an existing all-in-one setup... essentially the cockpit installation is not compatible with this infra setup.
On 19/08/2020 14:48, Michael Jones wrote:
On 19/08/2020 12:12, Michael Jones wrote:
On 19/08/2020 10:41, Nir Soffer wrote:
There is no warning the method was deprecated and will be missing functionality.
The steps detailed on the alt install page are for the all-in-one running engine-setup.
It's also worth noting this works fine in;
Version 4.3.1.1-1.el7
but not in;
Version 4.4.1.10-1.el8
(el8 has the change in imageio daemons)
The alternate install method is still useful to have, but i think a red warning about all-in-one on el8 on that page would be good.
Kind Regards, Michael Jones Micheal, can you file a bug for this?
If you have a good use case for all-in-one deployment (not using hosted engine), please explain it in the bug.
Personally I think simple all-in-one deployment without the complexity of hosted engine is better, and we should keep it, but for this we need to teach engine to handle the case when the proxy and the daemon are the same server.
In this case engine will not try to setup a proxy ticket, and image transfer would work directly with the host daemon.
I'm not very optimistic that we will support this again, since this feature is not needed for RHV customers, but for oVirt this makes sense.
Nir
Yes, I can file a bug,
The main usage / setup's I have are;
on-prem installs:
- hosted engine - gluster - high availiblity - internal ip address - easy great...
dedicated host provider for example OVH single machine:
- alternate install - all-in-one
The main reason for the separation is that using the cockpit install / hosted engine install causes problems with ip allocations;
cockpit method requires 1x ip for host, and 1x ip for engine vm, and both ip must be in the same subnet...
applying internal ip would cut off access, and to make it even harder, getting public ip blocks didn't work as the box main ip wouldn't be in the same subnet, adding nic alias ip doesn't work either (fail on install due to failing to setup ovirtmgmt network).
atm, i'll struggle with changing the machine's main ip to be one of the same subnet with the engine one... (currently causes host to be taken offline due to hosting provider, health checks)
provided i can change the host primary ip to be one of the OVH failover ip allocated in a block... i will be able to install using the cockpit.
and after the install i can setup internal ip with the network tag.
Kind Regards,
Mike
despite managing to get OVH to disable monitoring (ping to the main ip, and rebooting host) and getting the host in the same ip range as the engine vm...
ie:
host ip: 158.x.x.13/32 = not used anymore
new subnet: 54.x.x.x/28
and reserving; host = 54.x.x.16 engine = 54.x.x.17
[ ERROR ] The Engine VM (54.x.x.17/28) and the default gateway (158.x.x.254) will not be in the same IP subnet.
the hosted engine installer crashes due to the gw being in a different subnet, so all three;
- host - engine - gateway
must be in the same subnet...
this rules out an install on ovh dedicated server.
unless... I can install the all-in-one again (this bit works), and then install the engine vm in an existing all-in-one setup...
essentially the cockpit installation is not compatible with this infra setup.
After going through the documentation again, I understand the best way to approach this would be to have a remote manager, ie; self hosted engine (on-prem) > host/2nd DC/Cluster (remote/ovh) standalone manager (on-prem) > host/2nd DC/Cluster (remote/ovh) That way resolves the ip issues (only need host ip, just don't install the manager on the remote server) outstanding... i need to workout the security implications of this. shame all-in-one is gone, but the above does work, and even means the remote host can again use local storage. I'll raise the bug report now i've finished testing, as I think stand alone, all-in-one, dedicated hosts are affordable and open ovirt to a wider user base (keeping hardware requirements minimal). Thanks again, Mike
On 20/08/2020 20:55, Michael Jones wrote:
On 19/08/2020 14:48, Michael Jones wrote:
On 19/08/2020 12:12, Michael Jones wrote:
On 19/08/2020 10:41, Nir Soffer wrote:
There is no warning the method was deprecated and will be missing functionality.
The steps detailed on the alt install page are for the all-in-one running engine-setup.
It's also worth noting this works fine in;
Version 4.3.1.1-1.el7
but not in;
Version 4.4.1.10-1.el8
(el8 has the change in imageio daemons)
The alternate install method is still useful to have, but i think a red warning about all-in-one on el8 on that page would be good.
Kind Regards, Michael Jones Micheal, can you file a bug for this?
If you have a good use case for all-in-one deployment (not using hosted engine), please explain it in the bug.
Personally I think simple all-in-one deployment without the complexity of hosted engine is better, and we should keep it, but for this we need to teach engine to handle the case when the proxy and the daemon are the same server.
In this case engine will not try to setup a proxy ticket, and image transfer would work directly with the host daemon.
I'm not very optimistic that we will support this again, since this feature is not needed for RHV customers, but for oVirt this makes sense.
Nir
Yes, I can file a bug,
The main usage / setup's I have are;
on-prem installs:
- hosted engine - gluster - high availiblity - internal ip address - easy great...
dedicated host provider for example OVH single machine:
- alternate install - all-in-one
The main reason for the separation is that using the cockpit install / hosted engine install causes problems with ip allocations;
cockpit method requires 1x ip for host, and 1x ip for engine vm, and both ip must be in the same subnet...
applying internal ip would cut off access, and to make it even harder, getting public ip blocks didn't work as the box main ip wouldn't be in the same subnet, adding nic alias ip doesn't work either (fail on install due to failing to setup ovirtmgmt network).
atm, i'll struggle with changing the machine's main ip to be one of the same subnet with the engine one... (currently causes host to be taken offline due to hosting provider, health checks)
provided i can change the host primary ip to be one of the OVH failover ip allocated in a block... i will be able to install using the cockpit.
and after the install i can setup internal ip with the network tag.
Kind Regards,
Mike
despite managing to get OVH to disable monitoring (ping to the main ip, and rebooting host) and getting the host in the same ip range as the engine vm...
ie:
host ip: 158.x.x.13/32 = not used anymore
new subnet: 54.x.x.x/28
and reserving; host = 54.x.x.16 engine = 54.x.x.17
[ ERROR ] The Engine VM (54.x.x.17/28) and the default gateway (158.x.x.254) will not be in the same IP subnet.
the hosted engine installer crashes due to the gw being in a different subnet, so all three;
- host - engine - gateway
must be in the same subnet...
this rules out an install on ovh dedicated server.
unless... I can install the all-in-one again (this bit works), and then install the engine vm in an existing all-in-one setup...
essentially the cockpit installation is not compatible with this infra setup.
After going through the documentation again, I understand the best way to approach this would be to have a remote manager, ie;
self hosted engine (on-prem) > host/2nd DC/Cluster (remote/ovh)
standalone manager (on-prem) > host/2nd DC/Cluster (remote/ovh)
That way resolves the ip issues (only need host ip, just don't install the manager on the remote server)
outstanding... i need to workout the security implications of this.
shame all-in-one is gone, but the above does work, and even means the remote host can again use local storage.
I'll raise the bug report now i've finished testing, as I think stand alone, all-in-one, dedicated hosts are affordable and open ovirt to a wider user base (keeping hardware requirements minimal).
Thanks again,
Mike
Bug raised: https://bugzilla.redhat.com/show_bug.cgi?id=1871348 There was also a comment that vprotect shouldn't be using the proxy, I'll check that and raise a separate bug with them. Kind Regards, Mike
On Sat, Aug 22, 2020 at 6:26 PM Michael Jones <mj@mikejonesey.co.uk> wrote:
On 20/08/2020 20:55, Michael Jones wrote:
On 19/08/2020 14:48, Michael Jones wrote:
On 19/08/2020 12:12, Michael Jones wrote:
On 19/08/2020 10:41, Nir Soffer wrote:
There is no warning the method was deprecated and will be missing functionality.
The steps detailed on the alt install page are for the all-in-one running engine-setup.
It's also worth noting this works fine in;
Version 4.3.1.1-1.el7
but not in;
Version 4.4.1.10-1.el8
(el8 has the change in imageio daemons)
The alternate install method is still useful to have, but i think a red warning about all-in-one on el8 on that page would be good.
Kind Regards, Michael Jones
Micheal, can you file a bug for this?
If you have a good use case for all-in-one deployment (not using hosted engine), please explain it in the bug.
Personally I think simple all-in-one deployment without the complexity of hosted engine is better, and we should keep it, but for this we need to teach engine to handle the case when the proxy and the daemon are the same server.
In this case engine will not try to setup a proxy ticket, and image transfer would work directly with the host daemon.
I'm not very optimistic that we will support this again, since this feature is not needed for RHV customers, but for oVirt this makes sense.
Nir
Yes, I can file a bug,
The main usage / setup's I have are;
on-prem installs:
- hosted engine - gluster - high availiblity - internal ip address - easy great...
dedicated host provider for example OVH single machine:
- alternate install - all-in-one
The main reason for the separation is that using the cockpit install / hosted engine install causes problems with ip allocations;
cockpit method requires 1x ip for host, and 1x ip for engine vm, and both ip must be in the same subnet...
applying internal ip would cut off access, and to make it even harder, getting public ip blocks didn't work as the box main ip wouldn't be in the same subnet, adding nic alias ip doesn't work either (fail on install due to failing to setup ovirtmgmt network).
atm, i'll struggle with changing the machine's main ip to be one of the same subnet with the engine one... (currently causes host to be taken offline due to hosting provider, health checks)
provided i can change the host primary ip to be one of the OVH failover ip allocated in a block... i will be able to install using the cockpit.
and after the install i can setup internal ip with the network tag.
Kind Regards,
Mike
despite managing to get OVH to disable monitoring (ping to the main ip, and rebooting host) and getting the host in the same ip range as the engine vm...
ie:
host ip: 158.x.x.13/32 = not used anymore
new subnet: 54.x.x.x/28
and reserving; host = 54.x.x.16 engine = 54.x.x.17
[ ERROR ] The Engine VM (54.x.x.17/28) and the default gateway (158.x.x.254) will not be in the same IP subnet.
the hosted engine installer crashes due to the gw being in a different subnet, so all three;
- host - engine - gateway
must be in the same subnet...
this rules out an install on ovh dedicated server.
unless... I can install the all-in-one again (this bit works), and then install the engine vm in an existing all-in-one setup...
essentially the cockpit installation is not compatible with this infra setup.
After going through the documentation again, I understand the best way to approach this would be to have a remote manager, ie;
self hosted engine (on-prem) > host/2nd DC/Cluster (remote/ovh)
standalone manager (on-prem) > host/2nd DC/Cluster (remote/ovh)
That way resolves the ip issues (only need host ip, just don't install the manager on the remote server)
outstanding... i need to workout the security implications of this.
shame all-in-one is gone, but the above does work, and even means the remote host can again use local storage.
I'll raise the bug report now i've finished testing, as I think stand alone, all-in-one, dedicated hosts are affordable and open ovirt to a wider user base (keeping hardware requirements minimal).
Thanks again,
Mike
Bug raised:
Thanks, but we need more info why you cannot use the recommended deployment. See my questions in the bug.
On 22/08/2020 23:57, Nir Soffer wrote:
Bug raised:
https://bugzilla.redhat.com/show_bug.cgi?id=1871348 Thanks, but we need more info why you cannot use the recommended deployment. See my questions in the bug.
updated, thanks. Mike
On Tue, Aug 18, 2020 at 8:51 PM Michael Jones <mj@mikejonesey.co.uk> wrote:
On 18/08/2020 17:58, Nir Soffer wrote:
it does sound as if, my problems are around the fact that i am using an all-in-one box, (host and engine all in one);
https://www.ovirt.org/download/alternate_downloads.html
This explains how that you can install all-in-one when engine is a VM running on the single host, not as a program running on the host.
How did you managed to get engine installed on the same host?
I would expect that the installer would fail or at least warn about this.
The alternate install method is essentially, install packages and run engine-setup, which doesn't setup the hosted vm. The default installer via cockpit always installs engine as vm.
perhaps a warning is needed on the alternate installer page for epel8.
at the moment there is only a recommendation to use the normal installer.
On 18/08/2020 17:58, Nir Soffer wrote:
the download / upload function is important to me as my backup solution is dependent on this.
Until the sort this out, you should know that you can transfer images without the proxy. The proxy is needed only for the UI, for cases when engine and hosts are on different networks, so the only way to transfer images is via the engine host.
I use vprotect which i think is dependent on the proxy,
This worth a bug for vprotect, it should never use a proxy if it can access the host directly. This is very bad for backup use case, specially with the old proxy.
but I'll definitely checkout the sdk/scripts you linked.
I'll post an update once the new host is setup the normal way / working.
Thanks again.
Michael Jones
participants (3)
-
Michael Jones -
Nir Soffer -
Yedidyah Bar David