On Mon, Apr 26, 2021 at 7:46 PM Don Dupuis <dondster(a)gmail.com&gt; wrote: > > I am installing Ovirt 4.4.6.5-1 engine on CentOS Stream 8 on a dedicated vm. Towards the end of the engine-setup session I have a failure: > > [ INFO ] Stage: Transaction setup > [ INFO ] Stopping engine service > [ INFO ] Stopping ovirt-fence-kdump-listener service > [ INFO ] Stopping dwh service > [ INFO ] Stopping vmconsole-proxy service > [ INFO ] Stopping websocket-proxy service > [ INFO ] Stage: Misc configuration (early) > [ INFO ] Stage: Package installation > [ INFO ] Stage: Misc configuration > [ INFO ] Upgrading CA > [ INFO ] Initializing PostgreSQL > [ INFO ] Creating PostgreSQL 'engine' database > [ INFO ] Configuring PostgreSQL > [ INFO ] Creating PostgreSQL 'ovirt_engine_history' database > [ INFO ] Configuring PostgreSQL > [ INFO ] Creating CA: /etc/pki/ovirt-engine/ca.pem > [ INFO ] Creating CA: /etc/pki/ovirt-engine/qemu-ca.pem > [ INFO ] Updating OVN SSL configuration > [ INFO ] Updating OVN timeout configuration > [ INFO ] Creating/refreshing DWH database schema > [ INFO ] Setting up ovirt-vmconsole proxy helper PKI artifacts > [ INFO ] Setting up ovirt-vmconsole SSH PKI artifacts > [ INFO ] Configuring WebSocket Proxy > [ INFO ] Creating/refreshing Engine database schema > [ INFO ] Creating a user for Grafana > [ INFO ] Creating/refreshing Engine 'internal' domain database schema > [ INFO ] Creating default mac pool range > [ INFO ] Adding default OVN provider to database > [ INFO ] Adding OVN provider secret to database > [ INFO ] Setting a password for internal user admin > [ INFO ] Install selinux module /usr/share/ovirt-engine/selinux/ansible-runner-service.cil > [ INFO ] Generating post install configuration file '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf' > [ INFO ] Stage: Transaction commit > [ INFO ] Stage: Closing up > [ INFO ] Starting engine service > [ INFO ] Starting dwh service > [ INFO ] Starting Grafana service > [ ERROR ] Failed to execute stage 'Closing up': Failed to start service 'ovirt-imageio' > [ INFO ] Stage: Clean up > Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20210426111225-g31wa5.log > [ INFO ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20210426111501-setup.conf' > [ INFO ] Stage: Pre-termination > [ INFO ] Stage: Termination > [ ERROR ] Execution of setup failed > > Below is output of /var/log/ovirt-imageio/daemon.log: > > 2021-04-26 11:15:01,481 ERROR (MainThread) [server] Server failed > Traceback (most recent call last): > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", line 46, in main > server = Server(cfg) > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", line 110, in __init__ > self.remote_service = services.RemoteService(self.config, self.auth) > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/services.py", line 73, in __init__ > self._server = http.Server((config.remote.host, port), http.Connection) > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line 91, in __init__ > self.create_socket(prefer_ipv4) > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line 123, in create_socket > self.socket = socket.socket(self.address_family, self.socket_type) > File "/usr/lib64/python3.6/socket.py", line 144, in __init__ > _socket.socket.__init__(self, family, type, proto, fileno) > OSError: [Errno 97] Address family not supported by protocol > > I am not seeing errors in /var/log/ovirt-engine/setup/setup.log. What other information do you need that is revelant to this issue? Can you share a complete log? /var/log/ovirt-imageio/daemon.log Nir

On Mon, Apr 26, 2021 at 10:58 PM Don Dupuis <dondster(a)gmail.com&gt; wrote: > > Nir > It just repeats. Here you go Thanks, it seems that we need more work on logging. Can you try to change the log level to DEBUG and start the ovirt-imageio service? Add this file: $ cat /etc/ovirt-imageio/conf.d/99-local.conf [logger_root] level = DEBUG And start the service: $ systemctl start ovirt-imageio I hope we will have more info in the log explaining this issue. And to make sure we have correct configuration, please share output of: $ ovirt-imageio --show-config > 2021-04-26 13:50:49,600 INFO (MainThread) [server] Starting (hostname=manager2 pid=60513, version=2.1.1) > 2021-04-26 13:50:49,609 ERROR (MainThread) [server] Server failed > Traceback (most recent call last): > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", line 46, in main > server = Server(cfg) > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", line 110, in __init__ > self.remote_service = services.RemoteService(self.config, self.auth) > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/services.py", line 73, in __init__ > self._server = http.Server((config.remote.host, port), http.Connection) > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line 91, in __init__ > self.create_socket(prefer_ipv4) > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line 123, in create_socket > self.socket = socket.socket(self.address_family, self.socket_type) > File "/usr/lib64/python3.6/socket.py", line 144, in __init__ > _socket.socket.__init__(self, family, type, proto, fileno) > OSError: [Errno 97] Address family not supported by protocol > 2021-04-26 13:50:49,954 INFO (MainThread) [server] Starting (hostname=manager2 pid=60530, version=2.1.1) > 2021-04-26 13:50:49,967 ERROR (MainThread) [server] Server failed > Traceback (most recent call last): > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", line 46, in main > server = Server(cfg) > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", line 110, in __init__ > self.remote_service = services.RemoteService(self.config, self.auth) > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/services.py", line 73, in __init__ > self._server = http.Server((config.remote.host, port), http.Connection) > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line 91, in __init__ > self.create_socket(prefer_ipv4) > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line 123, in create_socket > self.socket = socket.socket(self.address_family, self.socket_type) > File "/usr/lib64/python3.6/socket.py", line 144, in __init__ > _socket.socket.__init__(self, family, type, proto, fileno) > OSError: [Errno 97] Address family not supported by protocol > 2021-04-26 13:50:50,199 INFO (MainThread) [server] Starting (hostname=manager2 pid=60567, version=2.1.1) > 2021-04-26 13:50:50,203 ERROR (MainThread) [server] Server failed > Traceback (most recent call last): > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", line 46, in main > server = Server(cfg) > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", line 110, in __init__ > self.remote_service = services.RemoteService(self.config, self.auth) > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/services.py", line 73, in __init__ > self._server = http.Server((config.remote.host, port), http.Connection) > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line 91, in __init__ > self.create_socket(prefer_ipv4) > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line 123, in create_socket > self.socket = socket.socket(self.address_family, self.socket_type) > File "/usr/lib64/python3.6/socket.py", line 144, in __init__ > _socket.socket.__init__(self, family, type, proto, fileno) > OSError: [Errno 97] Address family not supported by protocol > > Don > > On Mon, Apr 26, 2021 at 2:34 PM Nir Soffer <nsoffer(a)redhat.com&gt; wrote: >> >> On Mon, Apr 26, 2021 at 7:46 PM Don Dupuis <dondster(a)gmail.com&gt; wrote: >> > >> > I am installing Ovirt 4.4.6.5-1 engine on CentOS Stream 8 on a dedicated vm. Towards the end of the engine-setup session I have a failure: >> > >> > [ INFO ] Stage: Transaction setup >> > [ INFO ] Stopping engine service >> > [ INFO ] Stopping ovirt-fence-kdump-listener service >> > [ INFO ] Stopping dwh service >> > [ INFO ] Stopping vmconsole-proxy service >> > [ INFO ] Stopping websocket-proxy service >> > [ INFO ] Stage: Misc configuration (early) >> > [ INFO ] Stage: Package installation >> > [ INFO ] Stage: Misc configuration >> > [ INFO ] Upgrading CA >> > [ INFO ] Initializing PostgreSQL >> > [ INFO ] Creating PostgreSQL 'engine' database >> > [ INFO ] Configuring PostgreSQL >> > [ INFO ] Creating PostgreSQL 'ovirt_engine_history' database >> > [ INFO ] Configuring PostgreSQL >> > [ INFO ] Creating CA: /etc/pki/ovirt-engine/ca.pem >> > [ INFO ] Creating CA: /etc/pki/ovirt-engine/qemu-ca.pem >> > [ INFO ] Updating OVN SSL configuration >> > [ INFO ] Updating OVN timeout configuration >> > [ INFO ] Creating/refreshing DWH database schema >> > [ INFO ] Setting up ovirt-vmconsole proxy helper PKI artifacts >> > [ INFO ] Setting up ovirt-vmconsole SSH PKI artifacts >> > [ INFO ] Configuring WebSocket Proxy >> > [ INFO ] Creating/refreshing Engine database schema >> > [ INFO ] Creating a user for Grafana >> > [ INFO ] Creating/refreshing Engine 'internal' domain database schema >> > [ INFO ] Creating default mac pool range >> > [ INFO ] Adding default OVN provider to database >> > [ INFO ] Adding OVN provider secret to database >> > [ INFO ] Setting a password for internal user admin >> > [ INFO ] Install selinux module /usr/share/ovirt-engine/selinux/ansible-runner-service.cil >> > [ INFO ] Generating post install configuration file '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf' >> > [ INFO ] Stage: Transaction commit >> > [ INFO ] Stage: Closing up >> > [ INFO ] Starting engine service >> > [ INFO ] Starting dwh service >> > [ INFO ] Starting Grafana service >> > [ ERROR ] Failed to execute stage 'Closing up': Failed to start service 'ovirt-imageio' >> > [ INFO ] Stage: Clean up >> > Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20210426111225-g31wa5.log >> > [ INFO ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20210426111501-setup.conf' >> > [ INFO ] Stage: Pre-termination >> > [ INFO ] Stage: Termination >> > [ ERROR ] Execution of setup failed >> > >> > Below is output of /var/log/ovirt-imageio/daemon.log: >> > >> > 2021-04-26 11:15:01,481 ERROR (MainThread) [server] Server failed >> > Traceback (most recent call last): >> > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", line 46, in main >> > server = Server(cfg) >> > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", line 110, in __init__ >> > self.remote_service = services.RemoteService(self.config, self.auth) >> > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/services.py", line 73, in __init__ >> > self._server = http.Server((config.remote.host, port), http.Connection) >> > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line 91, in __init__ >> > self.create_socket(prefer_ipv4) >> > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line 123, in create_socket >> > self.socket = socket.socket(self.address_family, self.socket_type) >> > File "/usr/lib64/python3.6/socket.py", line 144, in __init__ >> > _socket.socket.__init__(self, family, type, proto, fileno) >> > OSError: [Errno 97] Address family not supported by protocol >> > >> > I am not seeing errors in /var/log/ovirt-engine/setup/setup.log. What other information do you need that is revelant to this issue? >> >> Can you share a complete log? >> >> /var/log/ovirt-imageio/daemon.log >> >> Nir >>

Nir After looking at the debug output of daemon.log, it looks like the code is expecting ipv6 to be enabled and I have explicitly disable ipv6. When I enable it and reboot, it now WORKS!!.

Where I work we don't use ipv6, is there another option to set so that this works on ipv4 only setups?

Thanks Don On Mon, Apr 26, 2021 at 3:32 PM Don Dupuis <dondster(a)gmail.com&gt; wrote: > > 2021-04-26 15:29:50,728 INFO (MainThread) [server] Starting (hostname=rvsmanager2 pid=13203, version=2.1.1) > 2021-04-26 15:29:50,729 DEBUG (MainThread) [services] Creating remote.service on port 54323 > 2021-04-26 15:29:50,730 DEBUG (MainThread) [http] Prefer IPv4: False > 2021-04-26 15:29:50,730 DEBUG (MainThread) [http] Available network interfaces: [(<AddressFamily.AF_INET6: 10>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('::', 54323, 0, 0))] > 2021-04-26 15:29:50,730 DEBUG (MainThread) [http] Creating server socket with family=AddressFamily.AF_INET6 and type=SocketKind.SOCK_STREAM > 2021-04-26 15:29:50,732 ERROR (MainThread) [server] Server failed > Traceback (most recent call last): > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", line 46, in main > server = Server(cfg) > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", line 110, in __init__ > self.remote_service = services.RemoteService(self.config, self.auth) > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/services.py", line 73, in __init__ > self._server = http.Server((config.remote.host, port), http.Connection) > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line 91, in __init__ > self.create_socket(prefer_ipv4) > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line 123, in create_socket > self.socket = socket.socket(self.address_family, self.socket_type) > File "/usr/lib64/python3.6/socket.py", line 144, in __init__ > _socket.socket.__init__(self, family, type, proto, fileno) > OSError: [Errno 97] Address family not supported by protocol > 2021-04-26 15:29:50,919 INFO (MainThread) [server] Starting (hostname=rvsmanager2 pid=13208, version=2.1.1) > 2021-04-26 15:29:50,919 DEBUG (MainThread) [services] Creating remote.service on port 54323 > 2021-04-26 15:29:50,920 DEBUG (MainThread) [http] Prefer IPv4: False > 2021-04-26 15:29:50,920 DEBUG (MainThread) [http] Available network interfaces: [(<AddressFamily.AF_INET6: 10>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('::', 54323, 0, 0))] > 2021-04-26 15:29:50,920 DEBUG (MainThread) [http] Creating server socket with family=AddressFamily.AF_INET6 and type=SocketKind.SOCK_STREAM > 2021-04-26 15:29:50,922 ERROR (MainThread) [server] Server failed > Traceback (most recent call last): > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", line 46, in main > server = Server(cfg) > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", line 110, in __init__ > self.remote_service = services.RemoteService(self.config, self.auth) > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/services.py", line 73, in __init__ > self._server = http.Server((config.remote.host, port), http.Connection) > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line 91, in __init__ > self.create_socket(prefer_ipv4) > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line 123, in create_socket > self.socket = socket.socket(self.address_family, self.socket_type) > File "/usr/lib64/python3.6/socket.py", line 144, in __init__ > _socket.socket.__init__(self, family, type, proto, fileno) > OSError: [Errno 97] Address family not supported by protocol > > # ovirt-imageio --show-config > { > "backend_file": { > "buffer_size": 8388608 > }, > "backend_http": { > "buffer_size": 8388608, > "ca_file": "/etc/pki/ovirt-engine/ca.pem" > }, > "backend_nbd": { > "buffer_size": 8388608 > }, > "control": { > "port": 54324, > "prefer_ipv4": true, > "remove_timeout": 60, > "socket": "/run/ovirt-imageio/sock", > "transport": "tcp" > }, > "daemon": { > "drop_privileges": true, > "group_name": "ovirtimg", > "max_connections": 8, > "poll_interval": 1.0, > "run_dir": "/run/ovirt-imageio", > "user_name": "ovirtimg" > }, > "formatter_long": { > "format": "%(asctime)s %(levelname)-7s (%(threadName)s) [%(name)s] %(message)s" > }, > "formatters": { > "keys": "long" > }, > "handler_logfile": { > "args": "(\"/var/log/ovirt-imageio/daemon.log\",)", > "formatter": "long", > "class": "logging.handlers.RotatingFileHandler", > "kwargs": "{\"maxBytes\": 20971520, \"backupCount\": 10}", > "level": "DEBUG" > }, > "handler_stderr": { > "formatter": "long", > "class": "logging.StreamHandler", > "level": "DEBUG" > }, > "handlers": { > "keys": "logfile" > }, > "local": { > "enable": false, > "socket": "\u0000/org/ovirt/imageio" > }, > "logger_root": { > "handlers": "logfile", > "level": "DEBUG", > "propagate": 0 > }, > "loggers": { > "keys": "root" > }, > "profile": { > "filename": "/run/ovirt-imageio/profile" > }, > "remote": { > "host": "::", > "port": 54323 > }, > "tls": { > "ca_file": "/etc/pki/ovirt-engine/apache-ca.pem", > "cert_file": "/etc/pki/ovirt-engine/certs/apache.cer", > "enable": true, > "enable_tls1_1": false, > "key_file": "/etc/pki/ovirt-engine/keys/apache.key.nopass" > } > } > > > On Mon, Apr 26, 2021 at 3:22 PM Nir Soffer <nsoffer(a)redhat.com&gt; wrote: >> >> On Mon, Apr 26, 2021 at 10:58 PM Don Dupuis <dondster(a)gmail.com&gt; wrote: >> > >> > Nir >> > It just repeats. Here you go >> >> Thanks, it seems that we need more work on logging. >> >> Can you try to change the log level to DEBUG and start the >> ovirt-imageio service? >> >> Add this file: >> >> $ cat /etc/ovirt-imageio/conf.d/99-local.conf >> [logger_root] >> level = DEBUG >> >> And start the service: >> >> $ systemctl start ovirt-imageio >> >> I hope we will have more info in the log explaining this issue. >> >> And to make sure we have correct configuration, please share output of: >> >> $ ovirt-imageio --show-config >> >> > 2021-04-26 13:50:49,600 INFO (MainThread) [server] Starting (hostname=manager2 pid=60513, version=2.1.1) >> > 2021-04-26 13:50:49,609 ERROR (MainThread) [server] Server failed >> > Traceback (most recent call last): >> > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", line 46, in main >> > server = Server(cfg) >> > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", line 110, in __init__ >> > self.remote_service = services.RemoteService(self.config, self.auth) >> > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/services.py", line 73, in __init__ >> > self._server = http.Server((config.remote.host, port), http.Connection) >> > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line 91, in __init__ >> > self.create_socket(prefer_ipv4) >> > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line 123, in create_socket >> > self.socket = socket.socket(self.address_family, self.socket_type) >> > File "/usr/lib64/python3.6/socket.py", line 144, in __init__ >> > _socket.socket.__init__(self, family, type, proto, fileno) >> > OSError: [Errno 97] Address family not supported by protocol >> > 2021-04-26 13:50:49,954 INFO (MainThread) [server] Starting (hostname=manager2 pid=60530, version=2.1.1) >> > 2021-04-26 13:50:49,967 ERROR (MainThread) [server] Server failed >> > Traceback (most recent call last): >> > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", line 46, in main >> > server = Server(cfg) >> > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", line 110, in __init__ >> > self.remote_service = services.RemoteService(self.config, self.auth) >> > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/services.py", line 73, in __init__ >> > self._server = http.Server((config.remote.host, port), http.Connection) >> > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line 91, in __init__ >> > self.create_socket(prefer_ipv4) >> > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line 123, in create_socket >> > self.socket = socket.socket(self.address_family, self.socket_type) >> > File "/usr/lib64/python3.6/socket.py", line 144, in __init__ >> > _socket.socket.__init__(self, family, type, proto, fileno) >> > OSError: [Errno 97] Address family not supported by protocol >> > 2021-04-26 13:50:50,199 INFO (MainThread) [server] Starting (hostname=manager2 pid=60567, version=2.1.1) >> > 2021-04-26 13:50:50,203 ERROR (MainThread) [server] Server failed >> > Traceback (most recent call last): >> > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", line 46, in main >> > server = Server(cfg) >> > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", line 110, in __init__ >> > self.remote_service = services.RemoteService(self.config, self.auth) >> > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/services.py", line 73, in __init__ >> > self._server = http.Server((config.remote.host, port), http.Connection) >> > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line 91, in __init__ >> > self.create_socket(prefer_ipv4) >> > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line 123, in create_socket >> > self.socket = socket.socket(self.address_family, self.socket_type) >> > File "/usr/lib64/python3.6/socket.py", line 144, in __init__ >> > _socket.socket.__init__(self, family, type, proto, fileno) >> > OSError: [Errno 97] Address family not supported by protocol >> > >> > Don >> > >> > On Mon, Apr 26, 2021 at 2:34 PM Nir Soffer <nsoffer(a)redhat.com&gt; wrote: >> >> >> >> On Mon, Apr 26, 2021 at 7:46 PM Don Dupuis <dondster(a)gmail.com&gt; wrote: >> >> > >> >> > I am installing Ovirt 4.4.6.5-1 engine on CentOS Stream 8 on a dedicated vm. Towards the end of the engine-setup session I have a failure: >> >> > >> >> > [ INFO ] Stage: Transaction setup >> >> > [ INFO ] Stopping engine service >> >> > [ INFO ] Stopping ovirt-fence-kdump-listener service >> >> > [ INFO ] Stopping dwh service >> >> > [ INFO ] Stopping vmconsole-proxy service >> >> > [ INFO ] Stopping websocket-proxy service >> >> > [ INFO ] Stage: Misc configuration (early) >> >> > [ INFO ] Stage: Package installation >> >> > [ INFO ] Stage: Misc configuration >> >> > [ INFO ] Upgrading CA >> >> > [ INFO ] Initializing PostgreSQL >> >> > [ INFO ] Creating PostgreSQL 'engine' database >> >> > [ INFO ] Configuring PostgreSQL >> >> > [ INFO ] Creating PostgreSQL 'ovirt_engine_history' database >> >> > [ INFO ] Configuring PostgreSQL >> >> > [ INFO ] Creating CA: /etc/pki/ovirt-engine/ca.pem >> >> > [ INFO ] Creating CA: /etc/pki/ovirt-engine/qemu-ca.pem >> >> > [ INFO ] Updating OVN SSL configuration >> >> > [ INFO ] Updating OVN timeout configuration >> >> > [ INFO ] Creating/refreshing DWH database schema >> >> > [ INFO ] Setting up ovirt-vmconsole proxy helper PKI artifacts >> >> > [ INFO ] Setting up ovirt-vmconsole SSH PKI artifacts >> >> > [ INFO ] Configuring WebSocket Proxy >> >> > [ INFO ] Creating/refreshing Engine database schema >> >> > [ INFO ] Creating a user for Grafana >> >> > [ INFO ] Creating/refreshing Engine 'internal' domain database schema >> >> > [ INFO ] Creating default mac pool range >> >> > [ INFO ] Adding default OVN provider to database >> >> > [ INFO ] Adding OVN provider secret to database >> >> > [ INFO ] Setting a password for internal user admin >> >> > [ INFO ] Install selinux module /usr/share/ovirt-engine/selinux/ansible-runner-service.cil >> >> > [ INFO ] Generating post install configuration file '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf' >> >> > [ INFO ] Stage: Transaction commit >> >> > [ INFO ] Stage: Closing up >> >> > [ INFO ] Starting engine service >> >> > [ INFO ] Starting dwh service >> >> > [ INFO ] Starting Grafana service >> >> > [ ERROR ] Failed to execute stage 'Closing up': Failed to start service 'ovirt-imageio' >> >> > [ INFO ] Stage: Clean up >> >> > Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20210426111225-g31wa5.log >> >> > [ INFO ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20210426111501-setup.conf' >> >> > [ INFO ] Stage: Pre-termination >> >> > [ INFO ] Stage: Termination >> >> > [ ERROR ] Execution of setup failed >> >> > >> >> > Below is output of /var/log/ovirt-imageio/daemon.log: >> >> > >> >> > 2021-04-26 11:15:01,481 ERROR (MainThread) [server] Server failed >> >> > Traceback (most recent call last): >> >> > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", line 46, in main >> >> > server = Server(cfg) >> >> > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", line 110, in __init__ >> >> > self.remote_service = services.RemoteService(self.config, self.auth) >> >> > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/services.py", line 73, in __init__ >> >> > self._server = http.Server((config.remote.host, port), http.Connection) >> >> > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line 91, in __init__ >> >> > self.create_socket(prefer_ipv4) >> >> > File "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line 123, in create_socket >> >> > self.socket = socket.socket(self.address_family, self.socket_type) >> >> > File "/usr/lib64/python3.6/socket.py", line 144, in __init__ >> >> > _socket.socket.__init__(self, family, type, proto, fileno) >> >> > OSError: [Errno 97] Address family not supported by protocol >> >> > >> >> > I am not seeing errors in /var/log/ovirt-engine/setup/setup.log. What other information do you need that is revelant to this issue? >> >> >> >> Can you share a complete log? >> >> >> >> /var/log/ovirt-imageio/daemon.log >> >> >> >> Nir >> >> >>

Nir Thanks for the info and explanation. Don On Mon, Apr 26, 2021 at 4:27 PM Nir Soffer <nsoffer(a)redhat.com&gt; wrote: > On Mon, Apr 26, 2021 at 11:56 PM Don Dupuis <dondster(a)gmail.com&gt; wrote: > > > > Nir > > After looking at the debug output of daemon.log, it looks like the code > is expecting ipv6 to be enabled and I have explicitly disable ipv6. When I > enable it and reboot, it now WORKS!!. > > Yes, we assume that ipv6 is enabled on the engine and hypervisors. > You will have the same issue with the hypervisors, since vdsm is also > listening on the same address ("::"). > > > Where I work we don't use ipv6, is there another option to set so that > this works on ipv4 only setups? > > We use the special "::" address so we listen to both ipv4 and ipv6. > This works for > people who use ipv4 or ipv6. > > If you must disable ipv6 on the engine host, you can try to listen to > an ipv4 address: > > # cat /etc/ovirt-imageio/conf.d/99-local.conf > [remote] > host = 0.0.0.0 > > Similar change is needed for vdsm: > > # cat /etc/vdsm/vdsm.conf.d/99-local.conf > [addresses] > management_ip = 0.0.0.0 > > I'm not sure that disabling ipv6 is tested, you may find other issues > on setup with > ipv6 disabled. > > Ales may have more insight on disabling ipv6. > > Nir > > > > > Thanks > > Don > > > > On Mon, Apr 26, 2021 at 3:32 PM Don Dupuis <dondster(a)gmail.com&gt; wrote: > >> > >> 2021-04-26 15:29:50,728 INFO (MainThread) [server] Starting > (hostname=rvsmanager2 pid=13203, version=2.1.1) > >> 2021-04-26 15:29:50,729 DEBUG (MainThread) [services] Creating > remote.service on port 54323 > >> 2021-04-26 15:29:50,730 DEBUG (MainThread) [http] Prefer IPv4: False > >> 2021-04-26 15:29:50,730 DEBUG (MainThread) [http] Available network > interfaces: [(<AddressFamily.AF_INET6: 10>, <SocketKind.SOCK_STREAM: 1>, 6, > '', ('::', 54323, 0, 0))] > >> 2021-04-26 15:29:50,730 DEBUG (MainThread) [http] Creating server > socket with family=AddressFamily.AF_INET6 and type=SocketKind.SOCK_STREAM > >> 2021-04-26 15:29:50,732 ERROR (MainThread) [server] Server failed > >> Traceback (most recent call last): > >> File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", > line 46, in main > >> server = Server(cfg) > >> File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", > line 110, in __init__ > >> self.remote_service = services.RemoteService(self.config, > self.auth) > >> File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/services.py", > line 73, in __init__ > >> self._server = http.Server((config.remote.host, port), > http.Connection) > >> File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line > 91, in __init__ > >> self.create_socket(prefer_ipv4) > >> File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line > 123, in create_socket > >> self.socket = socket.socket(self.address_family, self.socket_type) > >> File "/usr/lib64/python3.6/socket.py", line 144, in __init__ > >> _socket.socket.__init__(self, family, type, proto, fileno) > >> OSError: [Errno 97] Address family not supported by protocol > >> 2021-04-26 15:29:50,919 INFO (MainThread) [server] Starting > (hostname=rvsmanager2 pid=13208, version=2.1.1) > >> 2021-04-26 15:29:50,919 DEBUG (MainThread) [services] Creating > remote.service on port 54323 > >> 2021-04-26 15:29:50,920 DEBUG (MainThread) [http] Prefer IPv4: False > >> 2021-04-26 15:29:50,920 DEBUG (MainThread) [http] Available network > interfaces: [(<AddressFamily.AF_INET6: 10>, <SocketKind.SOCK_STREAM: 1>, 6, > '', ('::', 54323, 0, 0))] > >> 2021-04-26 15:29:50,920 DEBUG (MainThread) [http] Creating server > socket with family=AddressFamily.AF_INET6 and type=SocketKind.SOCK_STREAM > >> 2021-04-26 15:29:50,922 ERROR (MainThread) [server] Server failed > >> Traceback (most recent call last): > >> File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", > line 46, in main > >> server = Server(cfg) > >> File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", > line 110, in __init__ > >> self.remote_service = services.RemoteService(self.config, > self.auth) > >> File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/services.py", > line 73, in __init__ > >> self._server = http.Server((config.remote.host, port), > http.Connection) > >> File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line > 91, in __init__ > >> self.create_socket(prefer_ipv4) > >> File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line > 123, in create_socket > >> self.socket = socket.socket(self.address_family, self.socket_type) > >> File "/usr/lib64/python3.6/socket.py", line 144, in __init__ > >> _socket.socket.__init__(self, family, type, proto, fileno) > >> OSError: [Errno 97] Address family not supported by protocol > >> > >> # ovirt-imageio --show-config > >> { > >> "backend_file": { > >> "buffer_size": 8388608 > >> }, > >> "backend_http": { > >> "buffer_size": 8388608, > >> "ca_file": "/etc/pki/ovirt-engine/ca.pem" > >> }, > >> "backend_nbd": { > >> "buffer_size": 8388608 > >> }, > >> "control": { > >> "port": 54324, > >> "prefer_ipv4": true, > >> "remove_timeout": 60, > >> "socket": "/run/ovirt-imageio/sock", > >> "transport": "tcp" > >> }, > >> "daemon": { > >> "drop_privileges": true, > >> "group_name": "ovirtimg", > >> "max_connections": 8, > >> "poll_interval": 1.0, > >> "run_dir": "/run/ovirt-imageio", > >> "user_name": "ovirtimg" > >> }, > >> "formatter_long": { > >> "format": "%(asctime)s %(levelname)-7s (%(threadName)s) > [%(name)s] %(message)s" > >> }, > >> "formatters": { > >> "keys": "long" > >> }, > >> "handler_logfile": { > >> "args": "(\"/var/log/ovirt-imageio/daemon.log\",)", > >> "formatter": "long", > >> "class": "logging.handlers.RotatingFileHandler", > >> "kwargs": "{\"maxBytes\": 20971520, \"backupCount\": 10}", > >> "level": "DEBUG" > >> }, > >> "handler_stderr": { > >> "formatter": "long", > >> "class": "logging.StreamHandler", > >> "level": "DEBUG" > >> }, > >> "handlers": { > >> "keys": "logfile" > >> }, > >> "local": { > >> "enable": false, > >> "socket": "\u0000/org/ovirt/imageio" > >> }, > >> "logger_root": { > >> "handlers": "logfile", > >> "level": "DEBUG", > >> "propagate": 0 > >> }, > >> "loggers": { > >> "keys": "root" > >> }, > >> "profile": { > >> "filename": "/run/ovirt-imageio/profile" > >> }, > >> "remote": { > >> "host": "::", > >> "port": 54323 > >> }, > >> "tls": { > >> "ca_file": "/etc/pki/ovirt-engine/apache-ca.pem", > >> "cert_file": "/etc/pki/ovirt-engine/certs/apache.cer", > >> "enable": true, > >> "enable_tls1_1": false, > >> "key_file": "/etc/pki/ovirt-engine/keys/apache.key.nopass" > >> } > >> } > >> > >> > >> On Mon, Apr 26, 2021 at 3:22 PM Nir Soffer <nsoffer(a)redhat.com&gt; wrote: > >>> > >>> On Mon, Apr 26, 2021 at 10:58 PM Don Dupuis <dondster(a)gmail.com&gt; > wrote: > >>> > > >>> > Nir > >>> > It just repeats. Here you go > >>> > >>> Thanks, it seems that we need more work on logging. > >>> > >>> Can you try to change the log level to DEBUG and start the > >>> ovirt-imageio service? > >>> > >>> Add this file: > >>> > >>> $ cat /etc/ovirt-imageio/conf.d/99-local.conf > >>> [logger_root] > >>> level = DEBUG > >>> > >>> And start the service: > >>> > >>> $ systemctl start ovirt-imageio > >>> > >>> I hope we will have more info in the log explaining this issue. > >>> > >>> And to make sure we have correct configuration, please share output > of: > >>> > >>> $ ovirt-imageio --show-config > >>> > >>> > 2021-04-26 13:50:49,600 INFO (MainThread) [server] Starting > (hostname=manager2 pid=60513, version=2.1.1) > >>> > 2021-04-26 13:50:49,609 ERROR (MainThread) [server] Server failed > >>> > Traceback (most recent call last): > >>> > File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", > line 46, in main > >>> > server = Server(cfg) > >>> > File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", > line 110, in __init__ > >>> > self.remote_service = services.RemoteService(self.config, > self.auth) > >>> > File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/services.py", > line 73, in __init__ > >>> > self._server = http.Server((config.remote.host, port), > http.Connection) > >>> > File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line > 91, in __init__ > >>> > self.create_socket(prefer_ipv4) > >>> > File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line > 123, in create_socket > >>> > self.socket = socket.socket(self.address_family, > self.socket_type) > >>> > File "/usr/lib64/python3.6/socket.py", line 144, in __init__ > >>> > _socket.socket.__init__(self, family, type, proto, fileno) > >>> > OSError: [Errno 97] Address family not supported by protocol > >>> > 2021-04-26 13:50:49,954 INFO (MainThread) [server] Starting > (hostname=manager2 pid=60530, version=2.1.1) > >>> > 2021-04-26 13:50:49,967 ERROR (MainThread) [server] Server failed > >>> > Traceback (most recent call last): > >>> > File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", > line 46, in main > >>> > server = Server(cfg) > >>> > File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", > line 110, in __init__ > >>> > self.remote_service = services.RemoteService(self.config, > self.auth) > >>> > File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/services.py", > line 73, in __init__ > >>> > self._server = http.Server((config.remote.host, port), > http.Connection) > >>> > File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line > 91, in __init__ > >>> > self.create_socket(prefer_ipv4) > >>> > File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line > 123, in create_socket > >>> > self.socket = socket.socket(self.address_family, > self.socket_type) > >>> > File "/usr/lib64/python3.6/socket.py", line 144, in __init__ > >>> > _socket.socket.__init__(self, family, type, proto, fileno) > >>> > OSError: [Errno 97] Address family not supported by protocol > >>> > 2021-04-26 13:50:50,199 INFO (MainThread) [server] Starting > (hostname=manager2 pid=60567, version=2.1.1) > >>> > 2021-04-26 13:50:50,203 ERROR (MainThread) [server] Server failed > >>> > Traceback (most recent call last): > >>> > File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", > line 46, in main > >>> > server = Server(cfg) > >>> > File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", > line 110, in __init__ > >>> > self.remote_service = services.RemoteService(self.config, > self.auth) > >>> > File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/services.py", > line 73, in __init__ > >>> > self._server = http.Server((config.remote.host, port), > http.Connection) > >>> > File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line > 91, in __init__ > >>> > self.create_socket(prefer_ipv4) > >>> > File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line > 123, in create_socket > >>> > self.socket = socket.socket(self.address_family, > self.socket_type) > >>> > File "/usr/lib64/python3.6/socket.py", line 144, in __init__ > >>> > _socket.socket.__init__(self, family, type, proto, fileno) > >>> > OSError: [Errno 97] Address family not supported by protocol > >>> > > >>> > Don > >>> > > >>> > On Mon, Apr 26, 2021 at 2:34 PM Nir Soffer <nsoffer(a)redhat.com&gt; > wrote: > >>> >> > >>> >> On Mon, Apr 26, 2021 at 7:46 PM Don Dupuis <dondster(a)gmail.com&gt; > wrote: > >>> >> > > >>> >> > I am installing Ovirt 4.4.6.5-1 engine on CentOS Stream 8 on a > dedicated vm. Towards the end of the engine-setup session I have a failure: > >>> >> > > >>> >> > [ INFO ] Stage: Transaction setup > >>> >> > [ INFO ] Stopping engine service > >>> >> > [ INFO ] Stopping ovirt-fence-kdump-listener service > >>> >> > [ INFO ] Stopping dwh service > >>> >> > [ INFO ] Stopping vmconsole-proxy service > >>> >> > [ INFO ] Stopping websocket-proxy service > >>> >> > [ INFO ] Stage: Misc configuration (early) > >>> >> > [ INFO ] Stage: Package installation > >>> >> > [ INFO ] Stage: Misc configuration > >>> >> > [ INFO ] Upgrading CA > >>> >> > [ INFO ] Initializing PostgreSQL > >>> >> > [ INFO ] Creating PostgreSQL 'engine' database > >>> >> > [ INFO ] Configuring PostgreSQL > >>> >> > [ INFO ] Creating PostgreSQL 'ovirt_engine_history' database > >>> >> > [ INFO ] Configuring PostgreSQL > >>> >> > [ INFO ] Creating CA: /etc/pki/ovirt-engine/ca.pem > >>> >> > [ INFO ] Creating CA: /etc/pki/ovirt-engine/qemu-ca.pem > >>> >> > [ INFO ] Updating OVN SSL configuration > >>> >> > [ INFO ] Updating OVN timeout configuration > >>> >> > [ INFO ] Creating/refreshing DWH database schema > >>> >> > [ INFO ] Setting up ovirt-vmconsole proxy helper PKI artifacts > >>> >> > [ INFO ] Setting up ovirt-vmconsole SSH PKI artifacts > >>> >> > [ INFO ] Configuring WebSocket Proxy > >>> >> > [ INFO ] Creating/refreshing Engine database schema > >>> >> > [ INFO ] Creating a user for Grafana > >>> >> > [ INFO ] Creating/refreshing Engine 'internal' domain database > schema > >>> >> > [ INFO ] Creating default mac pool range > >>> >> > [ INFO ] Adding default OVN provider to database > >>> >> > [ INFO ] Adding OVN provider secret to database > >>> >> > [ INFO ] Setting a password for internal user admin > >>> >> > [ INFO ] Install selinux module > /usr/share/ovirt-engine/selinux/ansible-runner-service.cil > >>> >> > [ INFO ] Generating post install configuration file > '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf' > >>> >> > [ INFO ] Stage: Transaction commit > >>> >> > [ INFO ] Stage: Closing up > >>> >> > [ INFO ] Starting engine service > >>> >> > [ INFO ] Starting dwh service > >>> >> > [ INFO ] Starting Grafana service > >>> >> > [ ERROR ] Failed to execute stage 'Closing up': Failed to start > service 'ovirt-imageio' > >>> >> > [ INFO ] Stage: Clean up > >>> >> > Log file is located at > /var/log/ovirt-engine/setup/ovirt-engine-setup-20210426111225-g31wa5.log > >>> >> > [ INFO ] Generating answer file > '/var/lib/ovirt-engine/setup/answers/20210426111501-setup.conf' > >>> >> > [ INFO ] Stage: Pre-termination > >>> >> > [ INFO ] Stage: Termination > >>> >> > [ ERROR ] Execution of setup failed > >>> >> > > >>> >> > Below is output of /var/log/ovirt-imageio/daemon.log: > >>> >> > > >>> >> > 2021-04-26 11:15:01,481 ERROR (MainThread) [server] Server > failed > >>> >> > Traceback (most recent call last): > >>> >> > File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", > line 46, in main > >>> >> > server = Server(cfg) > >>> >> > File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py", > line 110, in __init__ > >>> >> > self.remote_service = services.RemoteService(self.config, > self.auth) > >>> >> > File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/services.py", > line 73, in __init__ > >>> >> > self._server = http.Server((config.remote.host, port), > http.Connection) > >>> >> > File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line > 91, in __init__ > >>> >> > self.create_socket(prefer_ipv4) > >>> >> > File > "/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line > 123, in create_socket > >>> >> > self.socket = socket.socket(self.address_family, > self.socket_type) > >>> >> > File "/usr/lib64/python3.6/socket.py", line 144, in __init__ > >>> >> > _socket.socket.__init__(self, family, type, proto, fileno) > >>> >> > OSError: [Errno 97] Address family not supported by protocol > >>> >> > > >>> >> > I am not seeing errors in /var/log/ovirt-engine/setup/setup.log. > What other information do you need that is revelant to this issue? > >>> >> > >>> >> Can you share a complete log? > >>> >> > >>> >> /var/log/ovirt-imageio/daemon.log > >>> >> > >>> >> Nir > >>> >> > >>> > >