9:46 a.m.
I have an odd situation:
When I go to
https://ovengine/manager-engine.com/webadmin/?locale=en_US
after authentication passes...
it shows the top banner of
oVirt OPEN VIRTUALIZATION MANAGER
and the
Loading ...
in the center. but never gets past that.
I also followed the thread and tried opening the admin portal in the private/incognito
mode but I still face the issue.
I rebooted the hosted engine as well but I am still facing the same issue.
When I trigger the above url, in the engine.log, following logs are created:
2023-04-19 12:09:37,535+05 INFO [org.ovirt.engine.core.sso.service.AuthenticationService]
(default task-14) [] User admin@internal-authz with profile [internal] successfully logged
in with scopes: ovirt-app-admin ovirt-app-api ovirt-app-portal
ovirt-ext=auth:sequence-priority=~ ovirt-ext=revoke:revoke-all
ovirt-ext=token-info:authz-search ovirt-ext=token-info:public-authz-search
ovirt-ext=token-info:validate ovirt-ext=token:password-access
2023-04-19 12:09:37,581+05 INFO [org.ovirt.engine.core.bll.aaa.CreateUserSessionCommand]
(default task-14) [4593247b] Running command: CreateUserSessionCommand internal: false.
2023-04-19 12:09:37,655+05 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-14)
[4593247b] EVENT_ID: USER_VDC_LOGIN(30), User admin@internal-authz connecting from
'10.0.1.2' using session
'tmdEg3zM+kPrJudpmQr+ITRboP8CWKVDlKP4XmYiB2GDqgWfSRj2H/kpNd96JfEXOuB+CjBNz72FkLx5//77kg=='
logged in.
2023-04-19 12:11:01,457+05 INFO
[org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-60) [1e895f1c] Lock
Acquired to object
'EngineLock:{exclusiveLocks='[7a49bb81-3c73-4119-8cb0-f3f6d88fafb9=PROVIDER]',
sharedLocks=''}'
2023-04-19 12:11:01,463+05 INFO
[org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-60) [1e895f1c]
Running command: SyncNetworkProviderCommand internal: true.
2023-04-19 12:11:01,564+05 INFO [org.ovirt.engine.core.sso.service.AuthenticationService]
(default task-6) [] User admin@internal-authz with profile [internal] successfully logged
in with scopes: ovirt-app-api ovirt-ext=token-info:authz-search
ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate
ovirt-ext=token:password-access
2023-04-19 12:11:01,647+05 INFO
[org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-60) [1e895f1c] Lock
freed to object
'EngineLock:{exclusiveLocks='[7a49bb81-3c73-4119-8cb0-f3f6d88fafb9=PROVIDER]',
sharedLocks=''}'
As you can see there are no error logs.
But at the same time I am able to see some prior errors in the engine.log file:
ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-96) [] Command
'HSMGetAllTasksStatusesVDSCommand(HostName = ovirt-host,
VdsIdVDSCommandParametersBase:{hostId='b2137069-b4e7-4c16-91e1-7f889c62ab88'})'
execution failed: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not
SPM: ()
ERROR [org.ovirt.engine.core.bll.pm.FenceProxyLocator]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-47) [1e63b6f1] Can
not run fence action on host 'ovirt-host', no suitable proxy host was found.
WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedScheduledExecutorService-engineSch eduledThreadPool-Thread-14)
[14a0ecc4] EVENT_ID: SYSTEM_CHANGE_STORAGE_POOL_STATUS_PROBLEMATIC_WITH_ERROR(987),
Invalid status on Data Center Default . Setting Data Center status to Non
Responsive (On host ovirt-host, Error: Network error during communication with the
Host.).
2023-04-18 19:15:13,838+05 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient]
(SSL Stomp Reactor) [] Connection timeout for host 'ovirt-hos t', last
response arrived 22501 ms ago.
ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring]
(EE-ManagedScheduledExecutorService-engineScheduledThrea dPool-Thread-45) [] Unable
to GetStats: VDSNetworkException: VDSGenericException: VDSNetworkException: Broken pipe
Also I can see that there is a connection between the ovirt-host and the hosted-engine:
==========================Hosted
Engine=====================================================
[root@manager-engine ovirt-engine]# netstat -anlp | grep 54321
tcp6 0 0 10.0.1.103:39944 10.0.1.101:54321 ESTABLISHED
2971/ovirt-engine
tcp6 0 0 10.0.1.103:44234 10.0.1.101:54321 ESTABLISHED
2971/ovirt-engine
[root@manager-engine ovirt-engine]#
===========================Ovirt-host========================================================
[root@ovirt-host ~]# netstat -anlp | grep 54321
tcp6 0 0 :::54321 :::* LISTEN
114608/python3
tcp6 0 0 ::1:54321 ::1:37428 ESTABLISHED
114608/python3
tcp6 0 0 ::1:54321 ::1:37414 ESTABLISHED
114608/python3
tcp6 0 0 10.0.1.101:54321 10.0.1.103:44234 ESTABLISHED
114608/python3
tcp6 0 0 ::1:37428 ::1:54321 ESTABLISHED
105702/platform-pyt
tcp6 0 0 ::1:54321 ::1:37416 ESTABLISHED
114608/python3
tcp6 0 0 ::1:37416 ::1:54321 ESTABLISHED
114861/platform-pyt
tcp6 0 0 10.0.1.101:54321 10.0.1.103:39944 ESTABLISHED
114608/python3
tcp6 0 0 ::1:37414 ::1:54321 ESTABLISHED
105491/platform-pyt
[root@ovirt-host ~]# netstat -anlp | grep vdsm
unix 2 [ ACC ] STREAM LISTENING 770058 96907/python3
/run/vdsm/svdsm.sock
unix 2 [ ACC ] STREAM LISTENING 1223139 114861/platform-pyt
/run/vdsm/mom-vdsm.sock
unix 3 [ ] STREAM CONNECTED 1226522 96907/python3
/run/vdsm/svdsm.sock
unix 3 [ ] STREAM CONNECTED 6903654 96907/python3
/run/vdsm/svdsm.sock
unix 3 [ ] STREAM CONNECTED 1230271 96907/python3
/run/vdsm/svdsm.sock
unix 3 [ ] STREAM CONNECTED 1230065 96907/python3
/run/vdsm/svdsm.sock
unix 3 [ ] STREAM CONNECTED 6823534 96907/python3
/run/vdsm/svdsm.sock
unix 3 [ ] STREAM CONNECTED 6838191 96907/python3
/run/vdsm/svdsm.sock
unix 3 [ ] STREAM CONNECTED 1224904 96907/python3
/run/vdsm/svdsm.sock
unix 3 [ ] STREAM CONNECTED 1225672 96907/python3
/run/vdsm/svdsm.sock
unix 3 [ ] STREAM CONNECTED 1214103 96907/python3
/run/vdsm/svdsm.sock
[root@ovirt-host ~]#
[root@ovirt-host ~]# systemctl status vdsmd.service
? vdsmd.service - Virtual Desktop Server Manager
Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled; vendor preset:
disabled)
Active: active (running) since Tue 2023-04-18 19:15:08 IST; 16h ago
Process: 114535 ExecStartPre=/usr/libexec/vdsm/vdsmd_init_common.sh --pre-start
(code=exited, status=0/SUCCESS)
Main PID: 114608 (vdsmd)
Tasks: 63 (limit: 48695)
Memory: 122.2M
CGroup: /system.slice/vdsmd.service
tq114608 /usr/bin/python3 /usr/share/vdsm/vdsmd
mq114826 /usr/libexec/ioprocess --read-pipe-fd 50 --write-pipe-fd 49
--max-threads 10 --max-queued-requests 10
Apr 19 09:03:35 ovirt-host vdsm[114608]: WARN Attempting to remove a non existing network:
ovirtmgmt/dcaa480a-068c-4030-ba9b-2baddc6de8c0
Apr 19 09:03:35 ovirt-host vdsm[114608]: WARN Attempting to remove a non existing net
user: ovirtmgmt/dcaa480a-068c-4030-ba9b-2baddc6de8c0
Apr 19 09:03:35 ovirt-host vdsm[114608]: WARN Attempting to remove a non existing network:
ovirtmgmt/dcaa480a-068c-4030-ba9b-2baddc6de8c0
Apr 19 09:03:35 ovirt-host vdsm[114608]: WARN Attempting to remove a non existing net
user: ovirtmgmt/dcaa480a-068c-4030-ba9b-2baddc6de8c0
Apr 19 09:13:34 ovirt-host vdsm[114608]: WARN Attempting to add an existing net user:
ovirtmgmt/dcaa480a-068c-4030-ba9b-2baddc6de8c0
Apr 19 11:18:27 ovirt-host vdsm[114608]: ERROR ssl handshake: socket error, address:
::ffff:10.0.1.103
Warning: Journal has been rotated since unit was started. Log output is incomplete or
unavailable.
[root@ovirt-host ~]#
Current netwoking:
[root@ovirt-host ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group
default qlen 1000
link/ether 48:4d:7e:a8:d8:73 brd ff:ff:ff:ff:ff:ff
3: enp0s20f0u6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP
group default qlen 1000
link/ether 80:3f:5d:08:4f:7e brd ff:ff:ff:ff:ff:ff
4: enp0s20f0u6.408@enp0s20f0u6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
noqueue master ovirtmgmt state UP group default qlen 1000
link/ether 80:3f:5d:08:4f:7e brd ff:ff:ff:ff:ff:ff
5: enp0s20f0u6.400@enp0s20f0u6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
noqueue state UP group default qlen 1000
link/ether 80:3f:5d:08:4f:7e brd ff:ff:ff:ff:ff:ff
inet6 abcd:abcd:abcd::17/64 scope global noprefixroute
valid_lft forever preferred_lft forever
inet6 fe80::1a7:ccb1:ae9b:9b8f/64 scope link noprefixroute
valid_lft forever preferred_lft forever
38: ;vdsmdummy;: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default
qlen 1000
link/ether de:b4:76:ba:04:82 brd ff:ff:ff:ff:ff:ff
39: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default
qlen 1000
link/ether 6e:94:55:f5:51:12 brd ff:ff:ff:ff:ff:ff
40: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen
1000
link/ether b6:96:23:82:d2:48 brd ff:ff:ff:ff:ff:ff
73: ovirtmgmt: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
group default qlen 1000
link/ether 80:3f:5d:08:4f:7e brd ff:ff:ff:ff:ff:ff
inet 10.0.1.101/24 brd 10.0.1.255 scope global noprefixroute ovirtmgmt
valid_lft forever preferred_lft forever
90: vnet2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master ovirtmgmt
state UNKNOWN group default qlen 1000
link/ether fe:16:3e:0d:0f:b9 brd ff:ff:ff:ff:ff:ff
inet6 fe80::fc16:3eff:fe0d:fb9/64 scope link
valid_lft forever preferred_lft forever
[root@ovirt-host ~]#
Could anyone please help me with this? I have tried re-installation as well but still no
luck.
Thanks and Regards
Kushagra Gupta