Thanks for your answer, I checked but I am still stuck :

I confirm that the servlet can be reached, according to your recommended test (Method Not Allowed.):

[root@vs-inf-prd-ovt-fr-501 ~]# wget https://localhost:443/ovirt-engine/services/vmconsole-proxy --no-check-certificate
--2022-06-13 10:30:11--  https://localhost/ovirt-engine/services/vmconsole-proxy
Resolving localhost (localhost)... ::1, 127.0.0.1
Connecting to localhost (localhost)|::1|:443... connected.
The certificate's owner does not match hostname 'localhost'
HTTP request sent, awaiting response... 405 Method Not Allowed
2022-06-13 10:30:11 ERROR 405: Method Not Allowed.

I retried ovirt-vmconsole-list.py with "--debug", and looked at the logs : 

[root@vs-inf-prd-ovt-fr-501 ~]# /usr/libexec/ovirt-vmconsole-proxy-helper/ovirt-vmconsole-list.py --debug --version "1" keys
[root@vs-inf-prd-ovt-fr-501 ~]# 
[root@vs-inf-prd-ovt-fr-501 ~]# grep vmconsole /var/log/messages
Jun 13 10:35:41 vs-inf-prd-ovt-fr-501 journal[3112274]: 2022-06-13 10:35:41,222+0200 ovirt-vmconsole-list: ERROR main:265 Error: HTTP Error 403: Forbidden

To be noted,
We did change the engine's CA certificate at some point by following this procedure https://ovirt.org/documentation/administration_guide/index.html#Replacing_the_Manager_CA_Certificate
We also renewed the certificates during a standard engine --setup upgrade to 4.4.10



Guillaume Pavese
Ingénieur Système et Réseau
Interactiv-Group


On Mon, Jun 13, 2022 at 4:47 PM Radoslaw Szwajkowski <rszwajko@redhat.com> wrote:
Hi,
the first thing to check is the firewall:  check with wget if the
servlet can be reached (method not allow error means you have
connected)

wget localhost:8080/ovirt-engine/services/vmconsole-proxy
HTTP request sent, awaiting response... 405 Method Not Allowed
2022-06-13 09:36:48 ERROR 405: Method Not Allowed.

When using the ovirt-vmconsole-list you can also check system log[1]
i.e. if the server cannot be reached you should see sth like this

grep vmconsole  /var/log/messages
Jun 13 08:58:02 developer journal[2972]: 2022-06-13 08:58:02,992+0200
ovirt-vmconsole-list: ERROR main:265 Error: <urlopen error [Errno 111]
Connection refused>

Note also that you can increase the log level by passing "--debug"
param or just look inside the script.

best regards,
radek

[1] https://github.com/oVirt/ovirt-vmconsole#problem-determination

On Mon, Jun 13, 2022 at 8:22 AM Guillaume Pavese
<guillaume.pavese@interactiv-group.com> wrote:
>
> Hello everyone,
>
> We have the same problem on our oVirt 4.4.10 Production server.
> ssh connection to vmconsole@engine was previously working in 4.4.6. but it stopped working at some point, maybe since upgraded to 4.4.10
>
> contrary to a working test environment that was directly installed on 4.4.10,
> And as for Nathanaël,
> the following returns nothing : ovirt-vmconsole-list.py --version "1" keys
>
> [root@vs-inf-prd-ovt-fr-501 ~]# /usr/libexec/ovirt-vmconsole-proxy-helper/ovirt-vmconsole-list.py --version "1" keys
> [root@vs-inf-prd-ovt-fr-501 ~]#
>
> I have verified that the keys stills appear on users' Option -> "User's Public Key" in the engine's UI
>
> What can I try to fix this?
>
>
> Guillaume Pavese
> Ingénieur Système et Réseau
> Interactiv-Group
>
>
> On Mon, May 10, 2021 at 9:47 PM Nathanaël Blanchet <blanchet@abes.fr> wrote:
>>
>> Hi,
>>
>> I can't still connect to my vms with vmconsole proxy on my production engine (other test and dev engine are OK).
>>
>> the ssh key for the wanted user is available in the the API:
>>
>> <ssh_public_keys>
>> <ssh_public_key href="/ovirt-engine/api/users/64b7f3bf-9d43-4508-af93-63ad77652be3/sshpublickeys/aaace8d4-08d3-4452-ac91-df4b491bd899" id="aaace8d4-08d3-4452-ac91-df4b491bd899">
>> <content>
>> ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyfrDI84RWtSvFOUvpb9DkbnIuEfZEQAt4ZCXDHNXcmRwa9iXfPbj69gkOJyj7Jhj9RinJn9at4NgJtrO/rRRgT+SzYUWpdO2KWHgRM5v1rpYcw820ZDdAZk+yxCjQsy6kd49q/q6B+Uzg8Kpth+CAV1ubRrBYqFiuT/qQe9y+0N1TkNdASWL38oZH9K0rzbDb4WlU2Er2BCXzoLF2NBk7iyaS3+Y65DqWPPHHdh89nilC6k5N7SCUkSOayrjh7NnErkBAKZ6PPaarZqZhZPrCbHZnu0oqA0XQXKLcYpwuhNwcK8e4ZWsDwMmArnNcmS6JFxnPIrGYxxmv01K6VXVvw==
>> </content>
>> <user href="/ovirt-engine/api/users/64b7f3bf-9d43-4508-af93-63ad77652be3" id="64b7f3bf-9d43-4508-af93-63ad77652be3"/>
>> </ssh_public_key>
>> </ssh_public_keys>
>>
>> But /usr/libexec/ovirt-vmconsole-proxy-helper/ovirt-vmconsole-list.py  --version "1" keys still returns nothing.
>>
>> On the engine:
>>
>> [root@air ~]# systemctl status ovirt-vmconsole-proxy-sshd.service
>> ● ovirt-vmconsole-proxy-sshd.service - oVirt VM Console SSH server daemon
>>    Loaded: loaded (/usr/lib/systemd/system/ovirt-vmconsole-proxy-sshd.service; enabled; vendor preset: disabled)
>>    Active: active (running) since Mon 2021-05-10 14:16:55 CEST; 22min ago
>>  Main PID: 3649210 (sshd)
>>     Tasks: 1 (limit: 204594)
>>    Memory: 2.7M
>>    CGroup: /system.slice/ovirt-vmconsole-proxy-sshd.service
>>            └─3649210 /usr/sbin/sshd -f /usr/share/ovirt-vmconsole/ovirt-vmconsole-proxy/ovirt-vmconsole-proxy-sshd/sshd_config -D
>>
>> mai 10 14:16:55 air.v100.abes.fr systemd[1]: Started oVirt VM Console SSH server daemon.
>> mai 10 14:16:55 air.v100.abes.fr sshd[3649210]: Server listening on 0.0.0.0 port 2222.
>> mai 10 14:16:55 air.v100.abes.fr sshd[3649210]: Server listening on :: port 2222.
>> mai 10 14:17:01 air.v100.abes.fr ovirt-vmconsole-proxy-keys[3649214]: ERROR '"keys"'
>> mai 10 14:17:01 air.v100.abes.fr sshd[3649212]: AuthorizedKeysCommand /usr/libexec/ovirt-vmconsole-proxy-keys ovirt-vmconsole failed, status 1
>> mai 10 14:17:02 air.v100.abes.fr ovirt-vmconsole-proxy-keys[3649218]: ERROR '"keys"'
>> mai 10 14:17:02 air.v100.abes.fr sshd[3649212]: AuthorizedKeysCommand /usr/libexec/ovirt-vmconsole-proxy-keys ovirt-vmconsole failed, status 1
>> mai 10 14:17:02 air.v100.abes.fr sshd[3649212]: Connection closed by authenticating user ovirt-vmconsole 10.34.100.131 port 46874 [preauth]
>>
>> I tried to execute /usr/libexec/ovirt-vmconsole-proxy-keys ovirt-vmconsole but it gives an internal ERROR (as on the other working engine, so it may be not relevant)
>>
>> What can I test more?
>>
>> Le 18/04/2021 à 15:59, Sharon Gratch a écrit :
>>
>> Hi,
>>
>> Please follow the instructions mentioned here:
>> https://www.ovirt.org/documentation/virtual_machine_management_guide/#Logging_in_to_a_virtual_machine_using_SPICE  - > " Opening a Serial Console to a Virtual Machine".
>>
>> It seems that something is wrong with the user permissions/keys.
>> Is the 4.4.5 oVirt installation an upgraded or a new installation?
>> You mentioned that it's working with your other engines? Do they all use the 4.4.5 version?
>>
>> Thanks,
>> Sharon
>>
>>
>> On Fri, Apr 16, 2021 at 1:31 PM Nathanaël Blanchet <blanchet@abes.fr> wrote:
>>>
>>> I removed the user and created an other time. Now, I have this
>>>
>>> The key seems to be present in the DB
>>>
>>> engine=# SELECT users.username, user_profiles.property_content::text
>>> FROM user_profiles
>>> JOIN users ON users.user_id = user_profiles.user_id
>>> WHERE user_profiles.property_type= 'SSH_PUBLIC_KEY';
>>>           username |
>>> property_content
>>>
>>> --------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------
>>> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>>> -------------------------------
>>>   sblanchet@levant.abes.fr | "ssh-rsa
>>> AAAAB3NzaC1yc2EAAAABIwAAAQEAyfrDI84RWtSvFOUvpb9DkbnIuEfZEQAt4ZCXDHNXcmRwa9iXfPbj69gkOJyj7Jhj9RinJn9at4NgJtrO/rRRgT+SzYUWpdO2KWHgRM5v1rpYcw820ZDdAZk+yxCjQ
>>> sy6kd49q/q6B+Uzg8Kpth+CAV1ubRrBYqFiuT/qQe9y+0N1TkNdASWL38oZH9K0rzbDb4WlU2Er2BCXzoLF2NBk7iyaS3+Y65DqWPPHHdh89nilC6k5N7SCUkSOayrjh7NnErkBAKZ6PPaarZqZhZPrCbHZnu0oqA0XQXKLcYpwuhNwcK8e4ZWsDwMmArn
>>> NcmS6JFxnPIrGYxxmv01K6VXVvw=="
>>> (1 row)
>>>
>>> and now in the api
>>>
>>> <ssh_public_keys>
>>> <ssh_public_key
>>> href="/ovirt-engine/api/users/64b7f3bf-9d43-4508-af93-63ad77652be3/sshpublickeys/70850a0e-1b20-4dd5-9fcd-4f64303509d1"
>>> id="70850a0e-1b20-4dd5-9fcd-4f64303509d1">
>>> <content>
>>> ssh-rsa
>>> AAAAB3NzaC1yc2EAAAABIwAAAQEAyfrDI84RWtSvFOUvpb9DkbnIuEfZEQAt4ZCXDHNXcmRwa9iXfPbj69gkOJyj7Jhj9RinJn9at4NgJtrO/rRRgT+SzYUWpdO2KWHgRM5v1rpYcw820ZDdAZk+yxCjQsy6kd49q/q6B+Uzg8Kpth+CAV1ubRrBYqFiuT/qQe9y+0N1TkNdASWL38oZH9K0rzbDb4WlU2Er2BCXzoLF2NBk7iyaS3+Y65DqWPPHHdh89nilC6k5N7SCUkSOayrjh7NnErkBAKZ6PPaarZqZhZPrCbHZnu0oqA0XQXKLcYpwuhNwcK8e4ZWsDwMmArnNcmS6JFxnPIrGYxxmv01K6VXVvw==
>>> </content>
>>> <user
>>> href="/ovirt-engine/api/users/64b7f3bf-9d43-4508-af93-63ad77652be3"
>>> id="64b7f3bf-9d43-4508-af93-63ad77652be3"/>
>>> </ssh_public_key>
>>> </ssh_public_keys>
>>>
>>> but I still can't connect
>>>
>>> $ ssh -t -p 2222  ovirt-vmconsole@air.v100.abes.fr connect
>>> ovirt-vmconsole@air.v100.abes.fr: Permission denied (publickey).
>>>
>>> and
>>>
>>> [root@air ~]#
>>> /usr/libexec/ovirt-vmconsole-proxy-helper/ovirt-vmconsole-list.py
>>> --version "1" keys
>>>
>>> still returns empty string...
>>>
>>>
>>> Le 16/04/2021 à 11:07, Nathanaël Blanchet a écrit :
>>> >
>>> > Le 16/04/2021 à 10:31, Radoslaw Szwajkowski a écrit :
>>> >>> [root@air-dev ~]#
>>> >>> /usr/libexec/ovirt-vmconsole-proxy-helper/ovirt-vmconsole-list.py
>>> >>> --version "1" keys
>>> >>> {"keys": [{"entityid": "d5e69fa0-96a0-4aae-952d-18fe36940248",
>>> >>> "entity":
>>> >>> "sblanchet@levant.abes.fr@abes.fr-authz", "key": "ssh-rsa
>>> >>> AAAAB3NzaC1yc2EAAAABIwAAAQEAyfrDI84RWtSvFOUvpb9DkbnIuEfZEQAt4ZCXDHNXcmRwa9iXfPbj69gkOJyj7Jhj9RinJn9at4NgJtrO/rRRgT+SzYUWpdO2KWHgRM5v1rpYcw820ZDdAZk+yxCjQsy6kd49q/q6B+Uzg8Kpth+CAV1ubRrBYqFiuT/qQe9y+0N1TkNdASWL38oZH9K0rzbDb4WlU2Er2BCXzoLF2NBk7iyaS3+Y65DqWPPHHdh89nilC6k5N7SCUkSOayrjh7NnErkBAKZ6PPaarZqZhZPrCbHZnu0oqA0XQXKLcYpwuhNwcK8e4ZWsDwMmArnNcmS6JFxnPIrGYxxmv01K6VXVvw=="}],
>>> >>>
>>> >>> "version": 1, "content": "key_list"}
>>> >>>
>>> >>> but the same command on the main  engine returns empty
>>> >>>
>>> >>> [root@air ~]#
>>> >>> /usr/libexec/ovirt-vmconsole-proxy-helper/ovirt-vmconsole-list.py
>>> >>> --version "1" keys
>>> >>>
>>> >> Empty list (no keys) should look similar to: {"keys": [], "version":
>>> >> 1, "content": "key_list"}
>>> >> In your case it seems that VMConsoleProxyServlet is not responding
>>> >> i.e. on my dev env I get a similar result (empty output,error code 1)
>>> >> when server is down.
>>> >
>>> > it is up
>>> >
>>> >
>>> > ● ovirt-vmconsole-proxy-sshd.service - oVirt VM Console SSH server daemon
>>> >    Loaded: loaded
>>> > (/usr/lib/systemd/system/ovirt-vmconsole-proxy-sshd.service; enabled;
>>> > vendor preset: disabled)
>>> >    Active: active (running) since Fri 2021-04-16 10:50:41 CEST; 1min
>>> > 27s ago
>>> >  Main PID: 1914370 (sshd)
>>> >     Tasks: 1 (limit: 204594)
>>> >    Memory: 3.5M
>>> >    CGroup: /system.slice/ovirt-vmconsole-proxy-sshd.service
>>> >            └─1914370 /usr/sbin/sshd -f
>>> > /usr/share/ovirt-vmconsole/ovirt-vmconsole-proxy/ovirt-vmconsole-proxy-sshd/sshd_config
>>> > -D
>>> >
>>> > avril 16 10:50:41 air.v100.abes.fr systemd[1]: Started oVirt VM
>>> > Console SSH server daemon.
>>> > avril 16 10:50:41 air.v100.abes.fr sshd[1914370]: Server listening on
>>> > 0.0.0.0 port 2222.
>>> > avril 16 10:50:41 air.v100.abes.fr sshd[1914370]: Server listening on
>>> > :: port 2222.
>>> > avril 16 10:52:02 air.v100.abes.fr ovirt-vmconsole[1914540]:
>>> > 2021-04-16 10:52:02,241+0200 ovirt-vmconsole-list: ERROR main:265
>>> > Error: HTTP Error 403: Forbidden
>>> > avril 16 10:52:02 air.v100.abes.fr
>>> > ovirt-vmconsole-proxy-keys[1914536]: ERROR Key list execution failed rc=1
>>> > avril 16 10:52:02 air.v100.abes.fr sshd[1914534]:
>>> > AuthorizedKeysCommand /usr/libexec/ovirt-vmconsole-proxy-keys
>>> > ovirt-vmconsole failed, status 1
>>> > avril 16 10:52:02 air.v100.abes.fr ovirt-vmconsole[1914547]:
>>> > 2021-04-16 10:52:02,806+0200 ovirt-vmconsole-list: ERROR main:265
>>> > Error: HTTP Error 403: Forbidden
>>> > avril 16 10:52:02 air.v100.abes.fr
>>> > ovirt-vmconsole-proxy-keys[1914543]: ERROR Key list execution failed rc=1
>>> > avril 16 10:52:02 air.v100.abes.fr sshd[1914534]:
>>> > AuthorizedKeysCommand /usr/libexec/ovirt-vmconsole-proxy-keys
>>> > ovirt-vmconsole failed, status 1
>>> > avril 16 10:52:03 air.v100.abes.fr sshd[1914534]: Connection closed by
>>> > authenticating user ovirt-vmconsole 10.34.100.131 port 53674 [preauth]
>>> >
>>> >>
>>> >> However you can check if DB contains the right data (key is encoded as
>>> >> JSON string - enclosed in double quotes):
>>> >> SELECT users.username, user_profiles.property_content::text
>>> >> FROM user_profiles
>>> >> JOIN users ON users.user_id = user_profiles.user_id
>>> >> WHERE user_profiles.property_type= 'SSH_PUBLIC_KEY';
>>> >
>>> > https://air.v100.abes.fr//ovirt-engine/api/users/1bb90486-d431-4554-a6a1-37631d8c16d4/sshpublickeys
>>> >
>>> >
>>> > <ssh_public_keys/>
>>> >
>>> > is empty
>>> >
>>> > while
>>> >
>>> > https://air-dev.v100.abes.fr/ovirt-engine/api/users/d5e69fa0-96a0-4aae-952d-18fe36940248/sshpublickeys
>>> >
>>> >
>>> > returns
>>> >
>>> > <ssh_public_keys>
>>> > <ssh_public_key
>>> > href="/ovirt-engine/api/users/d5e69fa0-96a0-4aae-952d-18fe36940248/sshpublickeys/1fa3fcaf-7475-4c72-9565-b32425d3c8fd"
>>> > id="1fa3fcaf-7475-4c72-9565-b32425d3c8fd">
>>> > <content>
>>> > ssh-rsa
>>> > AAAAB3NzaC1yc2EAAAABIwAAAQEAyfrDI84RWtSvFOUvpb9DkbnIuEfZEQAt4ZCXDHNXcmRwa9iXfPbj69gkOJyj7Jhj9RinJn9at4NgJtrO/rRRgT+SzYUWpdO2KWHgRM5v1rpYcw820ZDdAZk+yxCjQsy6kd49q/q6B+Uzg8Kpth+CAV1ubRrBYqFiuT/qQe9y+0N1TkNdASWL38oZH9K0rzbDb4WlU2Er2BCXzoLF2NBk7iyaS3+Y65DqWPPHHdh89nilC6k5N7SCUkSOayrjh7NnErkBAKZ6PPaarZqZhZPrCbHZnu0oqA0XQXKLcYpwuhNwcK8e4ZWsDwMmArnNcmS6JFxnPIrGYxxmv01K6VXVvw==
>>> > </content>
>>> > <user
>>> > href="/ovirt-engine/api/users/d5e69fa0-96a0-4aae-952d-18fe36940248"
>>> > id="d5e69fa0-96a0-4aae-952d-18fe36940248"/>
>>> > </ssh_public_key>
>>> > </ssh_public_keys>
>>> >
>>> >>
>>> >> best regards,
>>> >> Radek
>>> >>
>>> --
>>> Nathanaël Blanchet
>>>
>>> Supervision réseau
>>> SIRE
>>> 227 avenue Professeur-Jean-Louis-Viala
>>> 34193 MONTPELLIER CEDEX 5
>>> Tél. 33 (0)4 67 54 84 55
>>> Fax  33 (0)4 67 54 84 14
>>> blanchet@abes.fr
>>> _______________________________________________
>>> Users mailing list -- users@ovirt.org
>>> To unsubscribe send an email to users-leave@ovirt.org
>>> Privacy Statement: https://www.ovirt.org/privacy-policy.html
>>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
>>> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/TUHJA7C32NPJ5K5ITX4YGXEKNOZCXVHF/
>>
>> --
>> Nathanaël Blanchet
>>
>> Supervision réseau
>> SIRE
>> 227 avenue Professeur-Jean-Louis-Viala
>> 34193 MONTPELLIER CEDEX 5
>> Tél. 33 (0)4 67 54 84 55
>> Fax  33 (0)4 67 54 84 14
>> blanchet@abes.fr
>>
>> _______________________________________________
>> Users mailing list -- users@ovirt.org
>> To unsubscribe send an email to users-leave@ovirt.org
>> Privacy Statement: https://www.ovirt.org/privacy-policy.html
>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
>> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/OZNT3FINAXWFKQZ3TSZWYWGQVIWJ7KPI/
>
>
> Ce message et toutes les pièces jointes (ci-après le “message”) sont établis à l’intention exclusive de ses destinataires et sont confidentiels. Si vous recevez ce message par erreur, merci de le détruire et d’en avertir immédiatement l’expéditeur. Toute utilisation de ce message non conforme a sa destination, toute diffusion ou toute publication, totale ou partielle, est interdite, sauf autorisation expresse. L’internet ne permettant pas d’assurer l’intégrité de ce message . Interactiv-group (et ses filiales) décline(nt) toute responsabilité au titre de ce message, dans l’hypothèse ou il aurait été modifié. IT, ES, UK.


Ce message et toutes les pièces jointes (ci-après le “message”) sont établis à l’intention exclusive de ses destinataires et sont confidentiels. Si vous recevez ce message par erreur, merci de le détruire et d’en avertir immédiatement l’expéditeur. Toute utilisation de ce message non conforme a sa destination, toute diffusion ou toute publication, totale ou partielle, est interdite, sauf autorisation expresse. L’internet ne permettant pas d’assurer l’intégrité de ce message . Interactiv-group (et ses filiales) décline(nt) toute responsabilité au titre de ce message, dans l’hypothèse ou il aurait été modifié. IT, ES, UK.