5:11 a.m.
Hello,
due to missing megaraid_sas kernel module in RH EL 8 and CentOS 8, I'm
deploying a new oVirt host using CentOS 8 and elrepo kernel driver and not
ovirt node ng.
Based on installation guide:
- install CentOS 8.2 ("Server" chosen as base environment)
- yum install https://resources.ovirt.org/pub/yum-repo/ovirt-release44.rpm
- yum install cockpit-ovirt-dashboard
- yum update
- reboot
Try to add host from engine web admin gui, I get:
Host ov200 installation failed. Failed to execute Ansible host-deploy role:
Failed to execute call to start playbook. . Please check logs for more
details: /var/log/ovirt-engine/ansible-runner-service.log.
Inside the log file above on engine:
2020-10-08 11:58:43,389 - runner_service.controllers.hosts - DEBUG -
Request received, content-type :None
2020-10-08 11:58:43,390 - runner_service.controllers.hosts - INFO -
127.0.0.1 - GET /api/v1/hosts/ov200
2020-10-08 11:58:43,398 - runner_service.controllers.playbooks - DEBUG -
Request received, content-type :application/json; charset=UTF-8
2020-10-08 11:58:43,398 - runner_service.controllers.playbooks - INFO -
127.0.0.1 - POST /api/v1/playbooks/ovirt-host-deploy.yml
Do I have to enable any module or pre-install anything else before adding
it?
BTW: on host
[root@ov200 ~]# rpm -q ansible
ansible-2.9.13-2.el8.noarch
[root@ov200 ~]#
Thanks,
Gianluca
9:55 a.m.
Hi Gianluca,
Please execute the following command on your engine, save the output into a
file and attach it:
sudo journalctl -u ansible-runner-service
Dana
On Thu, Oct 8, 2020 at 1:13 PM Gianluca Cecchi <gianluca.cecchi(a)gmail.com>
wrote:
Hello,
due to missing megaraid_sas kernel module in RH EL 8 and CentOS 8, I'm
deploying a new oVirt host using CentOS 8 and elrepo kernel driver and not
ovirt node ng.
Based on installation guide:
- install CentOS 8.2 ("Server" chosen as base environment)
- yum install https://resources.ovirt.org/pub/yum-repo/ovirt-release44.rpm
- yum install cockpit-ovirt-dashboard
- yum update
- reboot
Try to add host from engine web admin gui, I get:
Host ov200 installation failed. Failed to execute Ansible host-deploy
role: Failed to execute call to start playbook. . Please check logs for
more details: /var/log/ovirt-engine/ansible-runner-service.log.
Inside the log file above on engine:
2020-10-08 11:58:43,389 - runner_service.controllers.hosts - DEBUG -
Request received, content-type :None
2020-10-08 11:58:43,390 - runner_service.controllers.hosts - INFO -
127.0.0.1 - GET /api/v1/hosts/ov200
2020-10-08 11:58:43,398 - runner_service.controllers.playbooks - DEBUG -
Request received, content-type :application/json; charset=UTF-8
2020-10-08 11:58:43,398 - runner_service.controllers.playbooks - INFO -
127.0.0.1 - POST /api/v1/playbooks/ovirt-host-deploy.yml
Do I have to enable any module or pre-install anything else before adding
it?
BTW: on host
[root@ov200 ~]# rpm -q ansible
ansible-2.9.13-2.el8.noarch
[root@ov200 ~]#
Thanks,
Gianluca
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/FT4S5MRJGJJ...
9:59 a.m.
And also please attach the content of the file found at:
/etc/ansible-runner-service/config.yaml
On Thu, Oct 8, 2020 at 5:55 PM Dana Elfassy <delfassy(a)redhat.com> wrote:
Hi Gianluca,
Please execute the following command on your engine, save the output into
a file and attach it:
sudo journalctl -u ansible-runner-service
Dana
On Thu, Oct 8, 2020 at 1:13 PM Gianluca Cecchi <gianluca.cecchi(a)gmail.com>
wrote:
> Hello,
> due to missing megaraid_sas kernel module in RH EL 8 and CentOS 8, I'm
> deploying a new oVirt host using CentOS 8 and elrepo kernel driver and not
> ovirt node ng.
>
> Based on installation guide:
>
> - install CentOS 8.2 ("Server" chosen as base environment)
> - yum install
> https://resources.ovirt.org/pub/yum-repo/ovirt-release44.rpm
> - yum install cockpit-ovirt-dashboard
> - yum update
> - reboot
>
> Try to add host from engine web admin gui, I get:
> Host ov200 installation failed. Failed to execute Ansible host-deploy
> role: Failed to execute call to start playbook. . Please check logs for
> more details: /var/log/ovirt-engine/ansible-runner-service.log.
>
> Inside the log file above on engine:
> 2020-10-08 11:58:43,389 - runner_service.controllers.hosts - DEBUG -
> Request received, content-type :None
> 2020-10-08 11:58:43,390 - runner_service.controllers.hosts - INFO -
> 127.0.0.1 - GET /api/v1/hosts/ov200
> 2020-10-08 11:58:43,398 - runner_service.controllers.playbooks - DEBUG -
> Request received, content-type :application/json; charset=UTF-8
> 2020-10-08 11:58:43,398 - runner_service.controllers.playbooks - INFO -
> 127.0.0.1 - POST /api/v1/playbooks/ovirt-host-deploy.yml
>
> Do I have to enable any module or pre-install anything else before adding
> it?
>
> BTW: on host
>
> [root@ov200 ~]# rpm -q ansible
> ansible-2.9.13-2.el8.noarch
> [root@ov200 ~]#
>
> Thanks,
> Gianluca
>
> _______________________________________________
> Users mailing list -- users(a)ovirt.org
> To unsubscribe send an email to users-leave(a)ovirt.org
> Privacy Statement: https://www.ovirt.org/privacy-policy.html
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
>
https://lists.ovirt.org/archives/list/users@ovirt.org/message/FT4S5MRJGJJ...
>
10:08 a.m.
On Thu, Oct 8, 2020 at 4:59 PM Dana Elfassy <delfassy(a)redhat.com> wrote:
And also please attach the content of the file found at:
/etc/ansible-runner-service/config.yaml
On Thu, Oct 8, 2020 at 5:55 PM Dana Elfassy <delfassy(a)redhat.com> wrote:
> Hi Gianluca,
> Please execute the following command on your engine, save the output into
> a file and attach it:
> sudo journalctl -u ansible-runner-service
> Dana
>
>
Thanks for answering, Dana.
[root@ovmgr1 ansible-runner-service]# sudo journalctl -u
ansible-runner-service
-- Logs begin at Tue 2020-10-06 11:12:46 CEST, end at Thu 2020-10-08
17:02:25 CEST. --
-- No entries --
[root@ovmgr1 ansible-runner-service]#
[root@ovmgr1 ansible-runner-service]# cat
/etc/ansible-runner-service/config.yaml
version: 1
playbooks_root_dir: '/usr/share/ovirt-engine/ansible-runner-service-project'
ssh_private_key: '/etc/pki/ovirt-engine/keys/engine_id_rsa'
port: 50001
target_user: root
log_path: '/var/log/ovirt-engine'
[root@ovmgr1 ansible-runner-service]#
I noticed that both on engine and on host the "ansible-runner" package is
not installed. Is it correct and only ansible-runner-service package to be
installed only on the engine?
Also, does the "service" in the name imply that I should have any systemd
or other kind of related service on engine?
Finally, I have to use a proxy for dnf/yum.
To be able to run "engine-setup" on engine I had to set http_proxy and
https_proxy eng variables inside the shell session, because it seems that
engine-setup was not able to leverage the global configuration. Could it be
something similar due to the host having to use a proxy too (that I already
setup in /etc/dnf/dnf.conf)? Just a guess.
Gianluca
10:13 a.m.
On Thu, Oct 8, 2020 at 5:08 PM Gianluca Cecchi <gianluca.cecchi(a)gmail.com>
wrote:
On Thu, Oct 8, 2020 at 4:59 PM Dana Elfassy
<delfassy(a)redhat.com> wrote:
> And also please attach the content of the file found at:
> /etc/ansible-runner-service/config.yaml
>
> On Thu, Oct 8, 2020 at 5:55 PM Dana Elfassy <delfassy(a)redhat.com> wrote:
>
>> Hi Gianluca,
>> Please execute the following command on your engine, save the output
>> into a file and attach it:
>> sudo journalctl -u ansible-runner-service
>> Dana
>>
>>
Thanks for answering, Dana.
[root@ovmgr1 ansible-runner-service]# sudo journalctl -u
ansible-runner-service
-- Logs begin at Tue 2020-10-06 11:12:46 CEST, end at Thu 2020-10-08
17:02:25 CEST. --
-- No entries --
[root@ovmgr1 ansible-runner-service]#
[root@ovmgr1 ansible-runner-service]# cat
/etc/ansible-runner-service/config.yaml
version: 1
playbooks_root_dir:
'/usr/share/ovirt-engine/ansible-runner-service-project'
ssh_private_key: '/etc/pki/ovirt-engine/keys/engine_id_rsa'
port: 50001
target_user: root
log_path: '/var/log/ovirt-engine'
[root@ovmgr1 ansible-runner-service]#
I noticed that both on engine and on host the "ansible-runner" package is
not installed. Is it correct and only ansible-runner-service package to be
installed only on the engine?
Also, does the "service" in the name imply that I should have any systemd
or other kind of related service on engine?
Finally, I have to use a proxy for dnf/yum.
To be able to run "engine-setup" on engine I had to set http_proxy and
https_proxy eng variables inside the shell session, because it seems that
engine-setup was not able to leverage the global configuration. Could it be
something similar due to the host having to use a proxy too (that I already
setup in /etc/dnf/dnf.conf)? Just a guess.
Gianluca
Also, the host already existed in 4.3. I upgraded the standalone engine
from 4.3.10 to 4.4.2 following the guide.
Now to update my hosts I put a host into maintenance, removed the host from
the gui, reinstalled the server in CentOS 8.2 with same network parameters,
and then add new host with the same name/hostname as before. Could it be a
problem to reuse the host?
Gianluca
9:43 a.m.
On Thu, Oct 8, 2020 at 5:13 PM Gianluca Cecchi <gianluca.cecchi(a)gmail.com>
wrote:
On Thu, Oct 8, 2020 at 5:08 PM Gianluca Cecchi <gianluca.cecchi(a)gmail.com>
wrote:
> On Thu, Oct 8, 2020 at 4:59 PM Dana Elfassy <delfassy(a)redhat.com> wrote:
>
>> And also please attach the content of the file found at:
>> /etc/ansible-runner-service/config.yaml
>>
>> On Thu, Oct 8, 2020 at 5:55 PM Dana Elfassy <delfassy(a)redhat.com> wrote:
>>
>>> Hi Gianluca,
>>> Please execute the following command on your engine, save the output
>>> into a file and attach it:
>>> sudo journalctl -u ansible-runner-service
>>> Dana
>>>
>>>
> Thanks for answering, Dana.
>
> [root@ovmgr1 ansible-runner-service]# sudo journalctl -u
> ansible-runner-service
> -- Logs begin at Tue 2020-10-06 11:12:46 CEST, end at Thu 2020-10-08
> 17:02:25 CEST. --
> -- No entries --
> [root@ovmgr1 ansible-runner-service]#
>
>
> [root@ovmgr1 ansible-runner-service]# cat
> /etc/ansible-runner-service/config.yaml
>
> version: 1
> playbooks_root_dir:
> '/usr/share/ovirt-engine/ansible-runner-service-project'
> ssh_private_key: '/etc/pki/ovirt-engine/keys/engine_id_rsa'
> port: 50001
> target_user: root
> log_path: '/var/log/ovirt-engine'
> [root@ovmgr1 ansible-runner-service]#
>
> I noticed that both on engine and on host the "ansible-runner" package is
> not installed. Is it correct and only ansible-runner-service package to be
> installed only on the engine?
> Also, does the "service" in the name imply that I should have any systemd
> or other kind of related service on engine?
> Finally, I have to use a proxy for dnf/yum.
> To be able to run "engine-setup" on engine I had to set http_proxy and
> https_proxy eng variables inside the shell session, because it seems that
> engine-setup was not able to leverage the global configuration. Could it be
> something similar due to the host having to use a proxy too (that I already
> setup in /etc/dnf/dnf.conf)? Just a guess.
>
> Gianluca
>
Also, the host already existed in 4.3. I upgraded the standalone engine
from 4.3.10 to 4.4.2 following the guide.
Now to update my hosts I put a host into maintenance, removed the host
from the gui, reinstalled the server in CentOS 8.2 with same network
parameters, and then add new host with the same name/hostname as before.
Could it be a problem to reuse the host?
Gianluca
Any other thing to check to be able to provision a node in 4.4.2 using
plain CentOS 8.2 host?
Thanks,
Gianluca
9:57 a.m.
Hi Gianluca,
could you please check selinux context of
/var/log/ovirt-engine/ansible-runner-service.log to see if you are not
affected by https://bugzilla.redhat.com/show_bug.cgi?id=1880171#c5 ?
Thanks,
Martin
On Fri, Oct 9, 2020 at 4:45 PM Gianluca Cecchi <gianluca.cecchi(a)gmail.com>
wrote:
On Thu, Oct 8, 2020 at 5:13 PM Gianluca Cecchi
<gianluca.cecchi(a)gmail.com>
wrote:
>
>
> On Thu, Oct 8, 2020 at 5:08 PM Gianluca Cecchi <gianluca.cecchi(a)gmail.com>
> wrote:
>
>> On Thu, Oct 8, 2020 at 4:59 PM Dana Elfassy <delfassy(a)redhat.com> wrote:
>>
>>> And also please attach the content of the file found at:
>>> /etc/ansible-runner-service/config.yaml
>>>
>>> On Thu, Oct 8, 2020 at 5:55 PM Dana Elfassy <delfassy(a)redhat.com>
>>> wrote:
>>>
>>>> Hi Gianluca,
>>>> Please execute the following command on your engine, save the output
>>>> into a file and attach it:
>>>> sudo journalctl -u ansible-runner-service
>>>> Dana
>>>>
>>>>
>> Thanks for answering, Dana.
>>
>> [root@ovmgr1 ansible-runner-service]# sudo journalctl -u
>> ansible-runner-service
>> -- Logs begin at Tue 2020-10-06 11:12:46 CEST, end at Thu 2020-10-08
>> 17:02:25 CEST. --
>> -- No entries --
>> [root@ovmgr1 ansible-runner-service]#
>>
>>
>> [root@ovmgr1 ansible-runner-service]# cat
>> /etc/ansible-runner-service/config.yaml
>>
>> version: 1
>> playbooks_root_dir:
>> '/usr/share/ovirt-engine/ansible-runner-service-project'
>> ssh_private_key: '/etc/pki/ovirt-engine/keys/engine_id_rsa'
>> port: 50001
>> target_user: root
>> log_path: '/var/log/ovirt-engine'
>> [root@ovmgr1 ansible-runner-service]#
>>
>> I noticed that both on engine and on host the "ansible-runner" package
>> is not installed. Is it correct and only ansible-runner-service package to
>> be installed only on the engine?
>> Also, does the "service" in the name imply that I should have any
>> systemd or other kind of related service on engine?
>> Finally, I have to use a proxy for dnf/yum.
>> To be able to run "engine-setup" on engine I had to set http_proxy and
>> https_proxy eng variables inside the shell session, because it seems that
>> engine-setup was not able to leverage the global configuration. Could it be
>> something similar due to the host having to use a proxy too (that I already
>> setup in /etc/dnf/dnf.conf)? Just a guess.
>>
>> Gianluca
>>
>
> Also, the host already existed in 4.3. I upgraded the standalone engine
> from 4.3.10 to 4.4.2 following the guide.
> Now to update my hosts I put a host into maintenance, removed the host
> from the gui, reinstalled the server in CentOS 8.2 with same network
> parameters, and then add new host with the same name/hostname as before.
> Could it be a problem to reuse the host?
>
> Gianluca
>
Any other thing to check to be able to provision a node in 4.4.2 using
plain CentOS 8.2 host?
Thanks,
Gianluca
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/AJN3ENCAXNC...
--
Martin Perina
Manager, Software Engineering
Red Hat Czech s.r.o.
10:54 a.m.
On Fri, Oct 9, 2020 at 4:58 PM Martin Perina <mperina(a)redhat.com> wrote:
Hi Gianluca,
could you please check selinux context of
/var/log/ovirt-engine/ansible-runner-service.log to see if you are not
affected by https://bugzilla.redhat.com/show_bug.cgi?id=1880171#c5 ?
Thanks,
Martin
Thanks for answering.
It seems ok. On the engine:
[root@ovmgr1 ~]# ls -Z /var/log/ovirt-engine/ansible-runner-service.log
system_u:object_r:httpd_log_t:s0
/var/log/ovirt-engine/ansible-runner-service.log
[root@ovmgr1 ~]#
Gianluca
11:29 a.m.
On Fri, Oct 9, 2020 at 5:54 PM Gianluca Cecchi <gianluca.cecchi(a)gmail.com>
wrote:
On Fri, Oct 9, 2020 at 4:58 PM Martin Perina
<mperina(a)redhat.com> wrote:
> Hi Gianluca,
>
> could you please check selinux context of
> /var/log/ovirt-engine/ansible-runner-service.log to see if you are not
> affected by https://bugzilla.redhat.com/show_bug.cgi?id=1880171#c5 ?
>
> Thanks,
> Martin
>
Thanks for answering.
It seems ok. On the engine:
[root@ovmgr1 ~]# ls -Z /var/log/ovirt-engine/ansible-runner-service.log
system_u:object_r:httpd_log_t:s0
/var/log/ovirt-engine/ansible-runner-service.log
[root@ovmgr1 ~]#
Gianluca
OK, so could you please apply the workaround mentioned in
https://bugzilla.redhat.com/show_bug.cgi?id=1880171#c5 to resolve the issue
until oVirt 4.4.3 is released?
--
Martin Perina
Manager, Software Engineering
Red Hat Czech s.r.o.
11:46 a.m.
On Fri, Oct 9, 2020 at 6:29 PM Martin Perina <mperina(a)redhat.com> wrote:
On Fri, Oct 9, 2020 at 5:54 PM Gianluca Cecchi <gianluca.cecchi(a)gmail.com>
wrote:
> On Fri, Oct 9, 2020 at 4:58 PM Martin Perina <mperina(a)redhat.com> wrote:
>
>> Hi Gianluca,
>>
>> could you please check selinux context of
>> /var/log/ovirt-engine/ansible-runner-service.log to see if you are not
>> affected by https://bugzilla.redhat.com/show_bug.cgi?id=1880171#c5 ?
>>
>> Thanks,
>> Martin
>>
>
> Thanks for answering.
> It seems ok. On the engine:
> [root@ovmgr1 ~]# ls -Z /var/log/ovirt-engine/ansible-runner-service.log
> system_u:object_r:httpd_log_t:s0
> /var/log/ovirt-engine/ansible-runner-service.log
> [root@ovmgr1 ~]#
>
> Gianluca
>
OK, so could you please apply the workaround mentioned in
https://bugzilla.redhat.com/show_bug.cgi?id=1880171#c5 to resolve the
issue until oVirt 4.4.3 is released?
Sorry, but isn't it already ok? The SELinux security context for the file
is already httpd_log_t, so I don't have to apply anything.
I also applied the more brutal workaround described in
https://bugzilla.redhat.com/show_bug.cgi?id=1880171#c4 without any effect,
so I'm not in this bugzilla context.
Do I have to apply also for the directory /var/log/ovirt-engine itself,
that currently has a context of var_log_t? I don't think so...
Gianluca
12:11 p.m.
On Fri, Oct 9, 2020 at 6:47 PM Gianluca Cecchi <gianluca.cecchi(a)gmail.com>
wrote:
On Fri, Oct 9, 2020 at 6:29 PM Martin Perina <mperina(a)redhat.com> wrote:
>
>
> On Fri, Oct 9, 2020 at 5:54 PM Gianluca Cecchi <gianluca.cecchi(a)gmail.com>
> wrote:
>
>> On Fri, Oct 9, 2020 at 4:58 PM Martin Perina <mperina(a)redhat.com> wrote:
>>
>>> Hi Gianluca,
>>>
>>> could you please check selinux context of
>>> /var/log/ovirt-engine/ansible-runner-service.log to see if you are not
>>> affected by https://bugzilla.redhat.com/show_bug.cgi?id=1880171#c5 ?
>>>
>>> Thanks,
>>> Martin
>>>
>>
>> Thanks for answering.
>> It seems ok. On the engine:
>> [root@ovmgr1 ~]# ls -Z /var/log/ovirt-engine/ansible-runner-service.log
>> system_u:object_r:httpd_log_t:s0
>> /var/log/ovirt-engine/ansible-runner-service.log
>> [root@ovmgr1 ~]#
>>
>> Gianluca
>>
>
> OK, so could you please apply the workaround mentioned in
> https://bugzilla.redhat.com/show_bug.cgi?id=1880171#c5 to resolve the
> issue until oVirt 4.4.3 is released?
>
>
Sorry, but isn't it already ok? The SELinux security context for the file
is already httpd_log_t, so I don't have to apply anything.
I also applied the more brutal workaround described in
https://bugzilla.redhat.com/show_bug.cgi?id=1880171#c4 without any
effect, so I'm not in this bugzilla context.
Do I have to apply also for the directory /var/log/ovirt-engine itself,
that currently has a context of var_log_t? I don't think so...
Ahh, sorry, I've misunderstood your reply, I thought you replied you are
affected.
Could you please share with us all logs from engine gathered by
logcollector? We will try to find out any clue what's wrong in your env ...
Thanks,
Martin
Gianluca
--
Martin Perina
Manager, Software Engineering
Red Hat Czech s.r.o.
6:24 p.m.
On Fri, Oct 9, 2020 at 7:12 PM Martin Perina <mperina(a)redhat.com> wrote:
Could you please share with us all logs from engine gathered by
logcollector? We will try to find out any clue what's wrong in your env ...
Thanks,
Martin
I will try to collect.
In the mean time I've found that SSH could be in some way involved
When I add the host and get the immediate failure and apparently nothing
happens at all, I see these two lines in /var/log/ovirt-engine/server.log
2020-10-09 18:15:09,369+02 WARN
[org.apache.sshd.client.session.ClientConnectionService]
(sshd-SshClient[7cb54873]-nio2-thread-1)
globalRequest(ClientConnectionService[ClientSessionImpl[root@ov200
/10.4.192.32:22]])[hostkeys-00@openssh.com, want-reply=false] failed
(SshException) to process: EdDSA provider not supported
2020-10-09 18:15:09,699+02 WARN
[org.apache.sshd.client.session.ClientConnectionService]
(sshd-SshClient[2cbceeab]-nio2-thread-1)
globalRequest(ClientConnectionService[ClientSessionImpl[root@ov200
/10.4.192.32:22]])[hostkeys-00@openssh.com, want-reply=false] failed
(SshException) to process: EdDSA provider not supported
could it be that the ssh client embedded is not able to connect to the
CentOS 8.2 for some reason?
On host at the moment when I try to add it I see again two sessions opened
and immediately closed (tried several times), eg in the timeframe above I
have:
Oct 9 18:15:09 ov200 systemd-logind[1237]: New session 41 of user root.
Oct 9 18:15:09 ov200 systemd[1]: Started Session 41 of user root.
Oct 9 18:15:09 ov200 systemd-logind[1237]: Session 41 logged out. Waiting
for processes to exit.
Oct 9 18:15:09 ov200 systemd-logind[1237]: Removed session 41.
Oct 9 18:15:09 ov200 systemd-logind[1237]: New session 42 of user root.
Oct 9 18:15:09 ov200 systemd[1]: Started Session 42 of user root.
Oct 9 18:15:09 ov200 systemd-logind[1237]: Session 42 logged out. Waiting
for processes to exit.
Oct 9 18:15:09 ov200 systemd-logind[1237]: Removed session 42.
anyway at sshd service level it seems it is ok om the host:
journalctl -u sshd.service has
Oct 09 18:15:09 ov200 sshd[13379]: Accepted password for root from
10.4.192.43 port 46008 ssh2
Oct 09 18:15:09 ov200 sshd[13379]: pam_unix(sshd:session): session opened
for user root by (uid=0)
Oct 09 18:15:09 ov200 sshd[13379]: pam_unix(sshd:session): session closed
for user root
Oct 09 18:15:09 ov200 sshd[13398]: Accepted password for root from
10.4.192.43 port 46014 ssh2
Oct 09 18:15:09 ov200 sshd[13398]: pam_unix(sshd:session): session opened
for user root by (uid=0)
Oct 09 18:15:09 ov200 sshd[13398]: pam_unix(sshd:session): session closed
for user root
On the host I have not customized anything ssh related:
[root@ov200 ssh]# ps -ef|grep sshd
root 1274 1 0 Oct08 ? 00:00:00 /usr/sbin/sshd -D
-oCiphers=aes256-gcm@openssh.com,chacha20-poly1305(a)openssh.com
,aes256-ctr,aes256-cbc,aes128-gcm(a)openssh.com,aes128-ctr,aes128-cbc -oMACs=
hmac-sha2-256-etm@openssh.com,hmac-sha1-etm(a)openssh.com,
umac-128-etm@openssh.com,hmac-sha2-512-etm(a)openssh.com
,hmac-sha2-256,hmac-sha1,umac-128(a)openssh.com,hmac-sha2-512
-oGSSAPIKexAlgorithms=gss-gex-sha1-,gss-group14-sha1-
-oKexAlgorithms=curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1
-oHostKeyAlgorithms=rsa-sha2-256,rsa-sha2-256-cert-v01(a)openssh.com
,ecdsa-sha2-nistp256,ecdsa-sha2-nistp256-cert-v01(a)openssh.com
,ecdsa-sha2-nistp384,ecdsa-sha2-nistp384-cert-v01(a)openssh.com,rsa-sha2-512,
rsa-sha2-512-cert-v01(a)openssh.com,ecdsa-sha2-nistp521,
ecdsa-sha2-nistp521-cert-v01(a)openssh.com,ssh-ed25519,
ssh-ed25519-cert-v01@openssh.com,ssh-rsa,ssh-rsa-cert-v01(a)openssh.com
-oPubkeyAcceptedKeyTypes=rsa-sha2-256,rsa-sha2-256-cert-v01(a)openssh.com
,ecdsa-sha2-nistp256,ecdsa-sha2-nistp256-cert-v01(a)openssh.com
,ecdsa-sha2-nistp384,ecdsa-sha2-nistp384-cert-v01(a)openssh.com,rsa-sha2-512,
rsa-sha2-512-cert-v01(a)openssh.com,ecdsa-sha2-nistp521,
ecdsa-sha2-nistp521-cert-v01(a)openssh.com,ssh-ed25519,
ssh-ed25519-cert-v01@openssh.com,ssh-rsa,ssh-rsa-cert-v01(a)openssh.com
-oCASignatureAlgorithms=rsa-sha2-256,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,rsa-sha2-512,ecdsa-sha2-nistp521,ssh-ed25519,ssh-rsa
and in sshd_config
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key
Can I replicate the command that the engine would run on host through ssh?
Gianluca
1:35 a.m.
New subject: problems installing standard Linux as nodes in 4.4
I guess you tried to ssh to the HostedEngine and then ssh to the host , right ?
Best Regards,
Strahil Nikolov
В събота, 10 октомври 2020 г., 02:28:35 Гринуич+3, Gianluca Cecchi
<gianluca.cecchi(a)gmail.com> написа:
On Fri, Oct 9, 2020 at 7:12 PM Martin Perina <mperina(a)redhat.com> wrote:
Could you please share with us all logs from engine gathered by logcollector? We will try
to find out any clue what's wrong in your env ...
Thanks,
Martin
I will try to collect.
In the mean time I've found that SSH could be in some way involved
When I add the host and get the immediate failure and apparently nothing happens at all,
I see these two lines in /var/log/ovirt-engine/server.log
2020-10-09 18:15:09,369+02 WARN [org.apache.sshd.client.session.ClientConnectionService]
(sshd-SshClient[7cb54873]-nio2-thread-1)
globalRequest(ClientConnectionService[ClientSessionImpl[root@ov200/10.4.192.32:22]])[hostkeys-00@openssh.com,
want-reply=false] failed (SshException) to process: EdDSA provider not supported
2020-10-09 18:15:09,699+02 WARN [org.apache.sshd.client.session.ClientConnectionService]
(sshd-SshClient[2cbceeab]-nio2-thread-1)
globalRequest(ClientConnectionService[ClientSessionImpl[root@ov200/10.4.192.32:22]])[hostkeys-00@openssh.com,
want-reply=false] failed (SshException) to process: EdDSA provider not supported
could it be that the ssh client embedded is not able to connect to the CentOS 8.2 for some
reason?
On host at the moment when I try to add it I see again two sessions opened and immediately
closed (tried several times), eg in the timeframe above I have:
Oct 9 18:15:09 ov200 systemd-logind[1237]: New session 41 of user root.
Oct 9 18:15:09 ov200 systemd[1]: Started Session 41 of user root.
Oct 9 18:15:09 ov200 systemd-logind[1237]: Session 41 logged out. Waiting for processes
to exit.
Oct 9 18:15:09 ov200 systemd-logind[1237]: Removed session 41.
Oct 9 18:15:09 ov200 systemd-logind[1237]: New session 42 of user root.
Oct 9 18:15:09 ov200 systemd[1]: Started Session 42 of user root.
Oct 9 18:15:09 ov200 systemd-logind[1237]: Session 42 logged out. Waiting for processes
to exit.
Oct 9 18:15:09 ov200 systemd-logind[1237]: Removed session 42.
anyway at sshd service level it seems it is ok om the host:
journalctl -u sshd.service has
Oct 09 18:15:09 ov200 sshd[13379]: Accepted password for root from 10.4.192.43 port 46008
ssh2
Oct 09 18:15:09 ov200 sshd[13379]: pam_unix(sshd:session): session opened for user root by
(uid=0)
Oct 09 18:15:09 ov200 sshd[13379]: pam_unix(sshd:session): session closed for user root
Oct 09 18:15:09 ov200 sshd[13398]: Accepted password for root from 10.4.192.43 port 46014
ssh2
Oct 09 18:15:09 ov200 sshd[13398]: pam_unix(sshd:session): session opened for user root by
(uid=0)
Oct 09 18:15:09 ov200 sshd[13398]: pam_unix(sshd:session): session closed for user root
On the host I have not customized anything ssh related:
[root@ov200 ssh]# ps -ef|grep sshd
root 1274 1 0 Oct08 ? 00:00:00 /usr/sbin/sshd -D
-oCiphers=aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes256-cbc,aes128-gcm(a)openssh.com,aes128-ctr,aes128-cbc
-oMACs=hmac-sha2-256-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha1,umac-128(a)openssh.com,hmac-sha2-512
-oGSSAPIKexAlgorithms=gss-gex-sha1-,gss-group14-sha1-
-oKexAlgorithms=curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1
-oHostKeyAlgorithms=rsa-sha2-256,rsa-sha2-256-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384,ecdsa-sha2-nistp384-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-512-cert-v01@openssh.com,ecdsa-sha2-nistp521,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519,ssh-ed25519-cert-v01@openssh.com,ssh-rsa,ssh-rsa-cert-v01(a)openssh.com
-oPubkeyAcceptedKeyTypes=rsa-sha2-256,rsa-sha2-256-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384,ecdsa-sha2-nistp384-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-512-cert-v01@openssh.com,ecdsa-sha2-nistp521,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519,ssh-ed25519-cert-v01@openssh.com,ssh-rsa,ssh-rsa-cert-v01(a)openssh.com
-oCASignatureAlgorithms=rsa-sha2-256,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,rsa-sha2-512,ecdsa-sha2-nistp521,ssh-ed25519,ssh-rsa
and in sshd_config
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key
Can I replicate the command that the engine would run on host through ssh?
Gianluca
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/QNZP6PA6XWF...
3:13 a.m.
On Sat, 10 Oct 2020, 01:24 Gianluca Cecchi, <gianluca.cecchi(a)gmail.com>
wrote:
On Fri, Oct 9, 2020 at 7:12 PM Martin Perina
<mperina(a)redhat.com> wrote:
>
>
> Could you please share with us all logs from engine gathered by
> logcollector? We will try to find out any clue what's wrong in your env ...
>
> Thanks,
> Martin
>
>
I will try to collect.
In the mean time I've found that SSH could be in some way involved
When I add the host and get the immediate failure and apparently nothing
happens at all, I see these two lines in /var/log/ovirt-engine/server.log
2020-10-09 18:15:09,369+02 WARN
[org.apache.sshd.client.session.ClientConnectionService]
(sshd-SshClient[7cb54873]-nio2-thread-1)
globalRequest(ClientConnectionService[ClientSessionImpl[root@ov200
/10.4.192.32:22]])[hostkeys-00@openssh.com, want-reply=false] failed
(SshException) to process: EdDSA provider not supported
2020-10-09 18:15:09,699+02 WARN
[org.apache.sshd.client.session.ClientConnectionService]
(sshd-SshClient[2cbceeab]-nio2-thread-1)
globalRequest(ClientConnectionService[ClientSessionImpl[root@ov200
/10.4.192.32:22]])[hostkeys-00@openssh.com, want-reply=false] failed
(SshException) to process: EdDSA provider not supported
This harmless, AFAIK EdDSA is not supported by default in OpenJDK 11 and
engine uses only ssh-rsa and ssh-rsa2 anyway
could it be that the ssh client embedded is not able to connect to
the
CentOS 8.2 for some reason?
If that's the case we should see an error either in engine.log or
ansible-runner-service.log
On host at the moment when I try to add it I see again two sessions
opened
and immediately closed (tried several times), eg in the timeframe above I
have:
Oct 9 18:15:09 ov200 systemd-logind[1237]: New session 41 of user root.
Oct 9 18:15:09 ov200 systemd[1]: Started Session 41 of user root.
Oct 9 18:15:09 ov200 systemd-logind[1237]: Session 41 logged out. Waiting
for processes to exit.
Oct 9 18:15:09 ov200 systemd-logind[1237]: Removed session 41.
Oct 9 18:15:09 ov200 systemd-logind[1237]: New session 42 of user root.
Oct 9 18:15:09 ov200 systemd[1]: Started Session 42 of user root.
Oct 9 18:15:09 ov200 systemd-logind[1237]: Session 42 logged out. Waiting
for processes to exit.
Oct 9 18:15:09 ov200 systemd-logind[1237]: Removed session 42.
anyway at sshd service level it seems it is ok om the host:
journalctl -u sshd.service has
Oct 09 18:15:09 ov200 sshd[13379]: Accepted password for root from
10.4.192.43 port 46008 ssh2
Oct 09 18:15:09 ov200 sshd[13379]: pam_unix(sshd:session): session opened
for user root by (uid=0)
Oct 09 18:15:09 ov200 sshd[13379]: pam_unix(sshd:session): session closed
for user root
Oct 09 18:15:09 ov200 sshd[13398]: Accepted password for root from
10.4.192.43 port 46014 ssh2
Oct 09 18:15:09 ov200 sshd[13398]: pam_unix(sshd:session): session opened
for user root by (uid=0)
Oct 09 18:15:09 ov200 sshd[13398]: pam_unix(sshd:session): session closed
for user root
On the host I have not customized anything ssh related:
[root@ov200 ssh]# ps -ef|grep sshd
root 1274 1 0 Oct08 ? 00:00:00 /usr/sbin/sshd -D
-oCiphers=aes256-gcm@openssh.com,chacha20-poly1305(a)openssh.com
,aes256-ctr,aes256-cbc,aes128-gcm(a)openssh.com,aes128-ctr,aes128-cbc
-oMACs=hmac-sha2-256-etm@openssh.com,hmac-sha1-etm(a)openssh.com,
umac-128-etm@openssh.com,hmac-sha2-512-etm(a)openssh.com
,hmac-sha2-256,hmac-sha1,umac-128(a)openssh.com,hmac-sha2-512
-oGSSAPIKexAlgorithms=gss-gex-sha1-,gss-group14-sha1-
-oKexAlgorithms=curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1
-oHostKeyAlgorithms=rsa-sha2-256,rsa-sha2-256-cert-v01(a)openssh.com
,ecdsa-sha2-nistp256,ecdsa-sha2-nistp256-cert-v01(a)openssh.com
,ecdsa-sha2-nistp384,ecdsa-sha2-nistp384-cert-v01(a)openssh.com
,rsa-sha2-512,rsa-sha2-512-cert-v01(a)openssh.com,ecdsa-sha2-nistp521,
ecdsa-sha2-nistp521-cert-v01(a)openssh.com,ssh-ed25519,
ssh-ed25519-cert-v01@openssh.com,ssh-rsa,ssh-rsa-cert-v01(a)openssh.com
-oPubkeyAcceptedKeyTypes=rsa-sha2-256,rsa-sha2-256-cert-v01(a)openssh.com
,ecdsa-sha2-nistp256,ecdsa-sha2-nistp256-cert-v01(a)openssh.com
,ecdsa-sha2-nistp384,ecdsa-sha2-nistp384-cert-v01(a)openssh.com
,rsa-sha2-512,rsa-sha2-512-cert-v01(a)openssh.com,ecdsa-sha2-nistp521,
ecdsa-sha2-nistp521-cert-v01(a)openssh.com,ssh-ed25519,
ssh-ed25519-cert-v01@openssh.com,ssh-rsa,ssh-rsa-cert-v01(a)openssh.com
-oCASignatureAlgorithms=rsa-sha2-256,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,rsa-sha2-512,ecdsa-sha2-nistp521,ssh-ed25519,ssh-rsa
and in sshd_config
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key
That looks good
Can I replicate the command that the engine would run on host through
ssh?
I don't think so there is an easy way to do it
Let's see what else we can get from the logs...
Martin
Gianluca
5:06 a.m.
On Sat, Oct 10, 2020 at 10:13 AM Martin Perina <mperina(a)redhat.com> wrote:
[snip]
> Can I replicate the command that the engine would run on host
through ssh?
>
I don't think so there is an easy way to do it
Let's see what else we can get from the logs...
Martin
Hi,
I've run on engine the command
ovirt-log-collector --no-hypervisors
but potentially there is much sensitive information (like the dump of the
database).
Is there any particular file you are more interested in that archive I can
share?
BTW: can I put engine in debug for the time I'm trying to add the host so
that we can see if more messages are shown?
In that case how can I do?
Another information I have noticed is that when the new host command from
web admin GUI suddenly fails, anyway the ov200 host is now present in the
host list, with the down icon and "Install failed" info.
If I click on it and go in General subtab, in the section "Action Items" I
see 3 items with exclamation mark in front of them:
1) Power Management is not configured for this Host.
Enable Power Management
--> OK, I skipped it
2) Host has no default route.
---> I don't know why it says this.
[root@ov200 log]# ip route show
default via 10.4.192.254 dev bond0.68 proto static metric 400
10.4.192.0/24 dev bond0.68 proto kernel scope link src 10.4.192.32 metric
400
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1
linkdown
[root@ov200 log]#
On the still in CentOS 7 active host I have:
[root@ov300 ~]# ip route show
default via 10.4.192.254 dev ovirtmgmntZ2Z3
10.4.187.0/24 dev p1p2.187 proto kernel scope link src 10.4.187.100
10.4.192.0/24 dev ovirtmgmntZ2Z3 proto kernel scope link src 10.4.192.33
10.10.100.0/24 dev p1p2 proto kernel scope link src 10.10.100.88
10.10.100.0/24 dev p1p1.100 proto kernel scope link src 10.10.100.87
[root@ov300 ~]#
[root@ov300 ~]# brctl show ovirtmgmntZ2Z3
bridge name bridge id STP enabled interfaces
ovirtmgmntZ2Z3 8000.1803730ba369 no bond0.68
[root@ov300 ~]#
Could it be the fact that for historical reasons my mgmt network has not
the name ovirtmgmt but ovirtmgmntZ2Z3 that confuses the installer that
expects ovirtmgmt to setup? And erroneously reports the no default route
message?
3) The host CPU does not match the Cluster CPU Type and is running in a
degraded mode. It is missing the following CPU flags: vmx, ssbd, nx,
model_Westmere, aes, spec_ctrl. Please update the host CPU microcode or
change the Cluster CPU Type.
The cluster is set as "Intel Westmere IBRS SSBD Family".
all the hosts are the same hw Dell PE M610, with same processor
Host installed in CentOS 8:
[root@ov200 log]# cat /proc/cpuinfo | grep "model name" | sort -u
model name : Intel(R) Xeon(R) CPU X5690 @ 3.47GHz
[root@ov200 log]#
Host still in CentOS 7:
[root@ov300 ~]# cat /proc/cpuinfo | grep "model name" | sort -u
model name : Intel(R) Xeon(R) CPU X5690 @ 3.47GHz
[root@ov300 ~]#
If I compare the cpu flags inside the OS I see:
CentOS 8:
[root@ov200 log]# cat /proc/cpuinfo | grep flags | sort -u
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat
pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb
rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology
nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx
est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 popcnt aes lahf_lm pti
ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid dtherm ida arat
flush_l1d
[root@ov200 log]#
CentOS 7:
[root@ov300 ~]# cat /proc/cpuinfo | grep flags | sort -u
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat
pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb
rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology
nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx
est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 popcnt aes lahf_lm ssbd
ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid dtherm ida arat
spec_ctrl intel_stibp flush_l1d
[root@ov300 ~]#
When still in CentOS 7, ov200 had the same flags as ov300
ov200 has this more now:
cpuid pti
ov200 has these less now:
eagerfpu spec_ctrl intel_stibp
Gianluca
3:41 a.m.
On Tue, Oct 13, 2020 at 12:06 PM Gianluca Cecchi <gianluca.cecchi(a)gmail.com>
wrote:
On Sat, Oct 10, 2020 at 10:13 AM Martin Perina
<mperina(a)redhat.com> wrote:
[snip]
>> Can I replicate the command that the engine would run on host through
>> ssh?
>>
>
> I don't think so there is an easy way to do it
> Let's see what else we can get from the logs...
>
> Martin
>
>
Hi,
I've run on engine the command
ovirt-log-collector --no-hypervisors
but potentially there is much sensitive information (like the dump of the
database).
Is there any particular file you are more interested in that archive I can
share?
BTW: can I put engine in debug for the time I'm trying to add the host so
that we can see if more messages are shown?
In that case how can I do?
Another information I have noticed is that when the new host command from
web admin GUI suddenly fails, anyway the ov200 host is now present in the
host list, with the down icon and "Install failed" info.
If I click on it and go in General subtab, in the section "Action Items" I
see 3 items with exclamation mark in front of them:
1) Power Management is not configured for this Host.
Enable Power Management
--> OK, I skipped it
2) Host has no default route.
---> I don't know why it says this.
[root@ov200 log]# ip route show
default via 10.4.192.254 dev bond0.68 proto static metric 400
10.4.192.0/24 dev bond0.68 proto kernel scope link src 10.4.192.32 metric
400
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1
linkdown
[root@ov200 log]#
On the still in CentOS 7 active host I have:
[root@ov300 ~]# ip route show
default via 10.4.192.254 dev ovirtmgmntZ2Z3
10.4.187.0/24 dev p1p2.187 proto kernel scope link src 10.4.187.100
10.4.192.0/24 dev ovirtmgmntZ2Z3 proto kernel scope link src 10.4.192.33
10.10.100.0/24 dev p1p2 proto kernel scope link src 10.10.100.88
10.10.100.0/24 dev p1p1.100 proto kernel scope link src 10.10.100.87
[root@ov300 ~]#
[root@ov300 ~]# brctl show ovirtmgmntZ2Z3
bridge name bridge id STP enabled interfaces
ovirtmgmntZ2Z3 8000.1803730ba369 no bond0.68
[root@ov300 ~]#
Could it be the fact that for historical reasons my mgmt network has not
the name ovirtmgmt but ovirtmgmntZ2Z3 that confuses the installer that
expects ovirtmgmt to setup? And erroneously reports the no default route
message?
3) The host CPU does not match the Cluster CPU Type and is running in a
degraded mode. It is missing the following CPU flags: vmx, ssbd, nx,
model_Westmere, aes, spec_ctrl. Please update the host CPU microcode or
change the Cluster CPU Type.
The cluster is set as "Intel Westmere IBRS SSBD Family".
all the hosts are the same hw Dell PE M610, with same processor
Host installed in CentOS 8:
[root@ov200 log]# cat /proc/cpuinfo | grep "model name" | sort -u
model name : Intel(R) Xeon(R) CPU X5690 @ 3.47GHz
[root@ov200 log]#
Host still in CentOS 7:
[root@ov300 ~]# cat /proc/cpuinfo | grep "model name" | sort -u
model name : Intel(R) Xeon(R) CPU X5690 @ 3.47GHz
[root@ov300 ~]#
If I compare the cpu flags inside the OS I see:
CentOS 8:
[root@ov200 log]# cat /proc/cpuinfo | grep flags | sort -u
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat
pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb
rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology
nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx
est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 popcnt aes lahf_lm pti
ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid dtherm ida arat
flush_l1d
[root@ov200 log]#
CentOS 7:
[root@ov300 ~]# cat /proc/cpuinfo | grep flags | sort -u
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat
pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb
rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology
nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx
est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 popcnt aes lahf_lm ssbd
ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid dtherm ida arat
spec_ctrl intel_stibp flush_l1d
[root@ov300 ~]#
When still in CentOS 7, ov200 had the same flags as ov300
ov200 has this more now:
cpuid pti
ov200 has these less now:
eagerfpu spec_ctrl intel_stibp
Gianluca
Any feedback on my latest comments?
In the meantime here:
https://drive.google.com/file/d/1iN37znRtCo2vgyGTH_ymLhBJfs-2pWDr/view?us...
you can find inside the sosreport in tar.gz format, where I have modified
some file names and context in respect of hostnames.
The only file I have not put inside is the dump of the database, but I can
run any query you like in case.
Gianluca
5:25 a.m.
On Thu, Oct 15, 2020 at 10:41 AM Gianluca Cecchi <gianluca.cecchi(a)gmail.com>
wrote:
Any feedback on my latest comments?
In the meantime here:
https://drive.google.com/file/d/1iN37znRtCo2vgyGTH_ymLhBJfs-2pWDr/view?us...
you can find inside the sosreport in tar.gz format, where I have modified
some file names and context in respect of hostnames.
The only file I have not put inside is the dump of the database, but I can
run any query you like in case.
Gianluca
I have also tried to put debug into the engine.
Method used base on this link:
https://www.ovirt.org/develop/developer-guide/engine/engine-development-e...
and I used engine.core as the package
[root@ovmgr1 ~]# diff ovirt-engine.xml.in ovirt-engine.xml.in.debug
118c118
< <level name="ALL"/>
---
<level name="DEBUG"/>
197a198,200
> <logger category="org.ovirt.engine.core">
<level name="DEBUG"/>
>
</logger>
[root@ovmgr1 ~]#
When the install fails I get this in engine.log now:
2020-10-15 12:16:15,394+02 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engine-Thread-31)
[c439aded-ade3-4474-a5f1-2f074ed5d920] EVENT_ID:
VDS_ANSIBLE_INSTALL_STARTED(560), Ansible host-deploy playbook execution
has started on host ov200.
2020-10-15 12:16:15,412+02 ERROR
[org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor]
(EE-ManagedThreadFactory-engine-Thread-31)
[c439aded-ade3-4474-a5f1-2f074ed5d920] Exception: Failed to execute call to
start playbook.
2020-10-15 12:16:15,412+02 DEBUG
[org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor]
(EE-ManagedThreadFactory-engine-Thread-31)
[c439aded-ade3-4474-a5f1-2f074ed5d920] Exception: :
org.ovirt.engine.core.common.utils.ansible.AnsibleRunnerCallException:
Failed to execute call to start playbook.
at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.common.utils.ansible.AnsibleRunnerHTTPClient.runPlaybook(AnsibleRunnerHTTPClient.java:153)
at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor.runCommand(AnsibleExecutor.java:113)
at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor.runCommand(AnsibleExecutor.java:78)
at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand.runAnsibleHostDeployPlaybook(InstallVdsInternalCommand.java:281)
at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand.executeCommand(InstallVdsInternalCommand.java:145)
at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1169)
at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1327)
at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:2003)
at
org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:140)
at
org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:79)
at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1387)
at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:419)
at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.PrevalidatingMultipleActionsRunner.executeValidatedCommand(PrevalidatingMultipleActionsRunner.java:204)
at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.PrevalidatingMultipleActionsRunner.runCommands(PrevalidatingMultipleActionsRunner.java:176)
at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.PrevalidatingMultipleActionsRunner.lambda$invokeCommands$3(PrevalidatingMultipleActionsRunner.java:182)
at
org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:96)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
at
org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:227)
2020-10-15 12:16:15,412+02 ERROR
[org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand]
(EE-ManagedThreadFactory-engine-Thread-31)
[c439aded-ade3-4474-a5f1-2f074ed5d920] Host installation failed for host
'79da834f-d03a-4abc-b89e-8ad0186c173c', 'ov200': Failed to execute
Ansible
host-deploy role: Failed to execute call to start playbook. . Please check
logs for more details: /var/log/ovirt-engine/ansible-runner-service.log
Any insight?
Gianluca
1:21 p.m.
On Thu, Oct 15, 2020 at 12:25 PM Gianluca Cecchi <gianluca.cecchi(a)gmail.com>
wrote:
On Thu, Oct 15, 2020 at 10:41 AM Gianluca Cecchi <
gianluca.cecchi(a)gmail.com> wrote:
>
>
> Any feedback on my latest comments?
> In the meantime here:
>
> https://drive.google.com/file/d/1iN37znRtCo2vgyGTH_ymLhBJfs-2pWDr/view?us...
> you can find inside the sosreport in tar.gz format, where I have modified
> some file names and context in respect of hostnames.
> The only file I have not put inside is the dump of the database, but I
> can run any query you like in case.
>
> Gianluca
>
>
I have also tried to put debug into the engine.
So after huge debugging work with Dana Elfassy and Martin Necas (thank you
very much to both!) and coordination of Sandro we found the culprit!
Inside firewall custom rules of my engine I had this (note the double
quotes for the comment about Nagios):
[root@ovmgr1 ovirt-engine]# engine-config -g IPTablesConfigSiteCustom
IPTablesConfigSiteCustom: -A INPUT -p tcp --dport 5666 -s 10.4.5.99/32 -m
comment --comment "Nagios NRPE daemon" -j ACCEPT version: general
[root@ovmgr1 ovirt-engine]#
So those double quotes caused a wrong formatted json block that
ansible-runner-service was not able to manage in the http post phase
After changing with single quotes, with this command:
engine-config -s IPTablesConfigSiteCustom="-A INPUT -p tcp --dport 5666 -s
10.4.5.99/32 -m comment --comment 'Nagios NRPE daemon' -j ACCEPT"
and restarting the engine so that now I have
[root@ovmgr1 ovirt-engine]# engine-config -g IPTablesConfigSiteCustom
IPTablesConfigSiteCustom: -A INPUT -p tcp --dport 5666 -s 10.4.5.99/32 -m
comment --comment 'Nagios NRPE daemon' -j ACCEPT version: general
[root@ovmgr1 ovirt-engine]#
I was able to add the CentOS 8.2 host.
So mind if you have the double quotes in any engine-config key before
upgrading from 4.3 to 4.4.
What a nasty thing to detect...
Thanks again guys for your help
Gianluca
2:31 a.m.
Hi Gianluca,
happy to hear that your issue was fixed!
Just please be aware that iptables support for hosts has been deprecated
and it's completely unsupported for cluster levels 4.4 and up. So unless
you switch your cluster to firewalld, you will not be able to upgrade your
cluster to 4.4 version. You can take a look at documentation how to prepare
custom firewall rules for firewalld:
https://www.ovirt.org/documentation/administration_guide/#Configuring_Hos...
Regards,
Martin
On Mon, Oct 26, 2020 at 7:22 PM Gianluca Cecchi <gianluca.cecchi(a)gmail.com>
wrote:
On Thu, Oct 15, 2020 at 12:25 PM Gianluca Cecchi <
gianluca.cecchi(a)gmail.com> wrote:
> On Thu, Oct 15, 2020 at 10:41 AM Gianluca Cecchi <
> gianluca.cecchi(a)gmail.com> wrote:
>
>>
>>
>> Any feedback on my latest comments?
>> In the meantime here:
>>
>>
https://drive.google.com/file/d/1iN37znRtCo2vgyGTH_ymLhBJfs-2pWDr/view?us...
>> you can find inside the sosreport in tar.gz format, where I have
>> modified some file names and context in respect of hostnames.
>> The only file I have not put inside is the dump of the database, but I
>> can run any query you like in case.
>>
>> Gianluca
>>
>>
>
> I have also tried to put debug into the engine.
>
>
So after huge debugging work with Dana Elfassy and Martin Necas (thank you
very much to both!) and coordination of Sandro we found the culprit!
Inside firewall custom rules of my engine I had this (note the double
quotes for the comment about Nagios):
[root@ovmgr1 ovirt-engine]# engine-config -g IPTablesConfigSiteCustom
IPTablesConfigSiteCustom: -A INPUT -p tcp --dport 5666 -s 10.4.5.99/32 -m
comment --comment "Nagios NRPE daemon" -j ACCEPT version: general
[root@ovmgr1 ovirt-engine]#
So those double quotes caused a wrong formatted json block that
ansible-runner-service was not able to manage in the http post phase
After changing with single quotes, with this command:
engine-config -s IPTablesConfigSiteCustom="-A INPUT -p tcp --dport 5666 -s
10.4.5.99/32 -m comment --comment 'Nagios NRPE daemon' -j ACCEPT"
and restarting the engine so that now I have
[root@ovmgr1 ovirt-engine]# engine-config -g IPTablesConfigSiteCustom
IPTablesConfigSiteCustom: -A INPUT -p tcp --dport 5666 -s 10.4.5.99/32 -m
comment --comment 'Nagios NRPE daemon' -j ACCEPT version: general
[root@ovmgr1 ovirt-engine]#
I was able to add the CentOS 8.2 host.
So mind if you have the double quotes in any engine-config key before
upgrading from 4.3 to 4.4.
What a nasty thing to detect...
Thanks again guys for your help
Gianluca
--
Martin Perina
Manager, Software Engineering
Red Hat Czech s.r.o.
1270
days inactive
1289
days old
18 comments
4 participants
participants (4)
-
Dana Elfassy -
Gianluca Cecchi -
Martin Perina -
Strahil Nikolov