On Sun, Jan 26, 2020 at 9:07 PM Marcin Sobczyk <msobczyk@redhat.com> wrote:
On 1/26/20 12:17 PM, Marcin Sobczyk wrote:
Hi,
I recently posted and merged [1], which makes supervdsmd depend on libvirt's socket units. Not sure how the machine ended up masked libvirt sockets though. Could you please share with me the whole deployment logs off list?
Looking at ansible deployment log I can see that it started around 10:39:
2020-01-26 10:39:10 IST - TASK [Gathering Facts] *********************************************************
2020-01-26 10:39:16 IST - ok: [didi-centos8-host.lab.eng.tlv2.redhat.com]
but 'systemctl show libvirtd-tls.socket' on the host gives me:
StateChangeTimestamp=Sun 2020-01-26 08:43:36 IST
so I think that libvirt sockets were masked even before the deployment process started. The uptime for the host is ~7 days - maybe it was contaminated before?
Can you reproduce this in a clean env?
Searched, and eventually managed to find: I saw that /etc/systemd/system/libvirtd-tls.socket's timestamp is from when I ran 'dnf update'. So I checked 'rpm -q --scripts' on all the packages I updated, and found this, in libvirt-daemon: posttrans scriptlet (using /bin/sh): if [ -f /var/lib/rpm-state/libvirt/restart ]; then # See if user has previously modified their install to # tell libvirtd to use --listen grep -E '^LIBVIRTD_ARGS=.*--listen' /etc/sysconfig/libvirtd 1>/dev/null 2>&1 if test $? = 0 then # Then lets keep honouring --listen and *not* use # systemd socket activation, because switching things # might confuse mgmt tool like puppet/ansible that # expect the old style libvirtd /bin/systemctl mask libvirtd.socket >/dev/null 2>&1 || : /bin/systemctl mask libvirtd-ro.socket >/dev/null 2>&1 || : /bin/systemctl mask libvirtd-admin.socket >/dev/null 2>&1 || : /bin/systemctl mask libvirtd-tls.socket >/dev/null 2>&1 || : /bin/systemctl mask libvirtd-tcp.socket >/dev/null 2>&1 || : else So the flow is, more-or-less: Install vdsm Somehow start it. I think it's configured to start automatically, so it's enough to reboot dnf update libvirt-daemon This masks libvirtd-tls.socket. Please handle :-), thanks!
Regards, Marcin
Thanks, Marcin
[1] https://gerrit.ovirt.org/#/c/105334/
On 1/26/20 11:53 AM, Yedidyah Bar David wrote:
Hi all,
Tried now 'hosted-engine --deploy' on fully updated CentOS 8/ovirt-master-snapshot machine. It failed during adding the host to the engine. engine.log has:
2020-01-26 10:41:47,825+02 ERROR [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand] (EE-ManagedThreadFactory-engine-Thread-1) [11ba00a7] Host installation failed for host 'efd6cb8a-935d-4812-b35c-3fbde5651b5a', 'didi-centos8-host.lab.eng.tlv2.redhat.com': Task Start and enable services failed to execute: 2020-01-26 10:41:47,836+02 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-1) [11ba00a7] START, SetVdsStatusVDSCommand(HostName = didi-centos8-host.lab.eng.tlv2.redhat.com, SetVdsStatusVDSCommandParameters:{hostId='efd6cb8a-935d-4812-b35c-3fbde5651b5a',
status='InstallFailed', nonOperationalReason='NONE', stopSpmFailureLogged='false', maintenanceReason='null'}), log id: 4f107d5d 2020-01-26 10:41:47,901+02 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-1) [11ba00a7] FINISH, SetVdsStatusVDSCommand, return: , log id: 4f107d5d 2020-01-26 10:41:48,002+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-1) [11ba00a7] EVENT_ID: VDS_INSTALL_FAILED(505), Host didi-centos8-host.lab.eng.tlv2.redhat.com installation failed. Task Start and enable services failed to execute: .
The code emitting this error seems to be in backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/common/utils/ansible/AnsibleRunnerHTTPClient.java
:
String.format("Task %1$s failed to execute: %2$s", task, "") // stdout, stderr?
Seems like someone considered logging also stdout/stderr but failed to make up their minds. Is this tracked somewhere?
ovirt-host-deploy-ansible-20200126103902-didi-centos8-host.lab.eng.tlv2.redhat.com-11ba00a7.log
has:
2020-01-26 10:41:38 IST - TASK [ovirt-host-deploy-vdsm : Start and enable services] ********************** 2020-01-26 10:41:47 IST - 2020-01-26 10:41:47 IST - { "status" : "OK", "msg" : "", "data" : { "event" : "runner_on_failed", ... "msg" : "Unable to start service vdsmd.service: Failed to start vdsmd.service: Unit libvirtd-tcp.socket is masked.\n", "_ansible_item_label" : "vdsmd.service"
systemctl status libvirtd-tcp.socket indeed still says it's masked. Package is:
# rpm -qif /usr/lib/systemd/system/libvirtd-tcp.socket Name : libvirt-daemon Version : 5.6.0 Release : 6.el8 Architecture: x86_64 Install Date: Mon 20 Jan 2020 08:23:12 AM IST Group : Unspecified Size : 1320922 License : LGPLv2+ Signature : RSA/SHA1, Wed 08 Jan 2020 11:06:38 AM IST, Key ID 695b5f7eff3e3445 Source RPM : libvirt-5.6.0-6.el8.src.rpm Build Date : Wed 08 Jan 2020 11:06:04 AM IST Build Host : copr-builder-156909441.novalocal Relocations : (not relocatable) URL : https://libvirt.org/ Summary : Server side daemon and supporting files for libvirt library Description : Server side daemon required to manage the virtualization capabilities of recent versions of Linux. Requires a hypervisor specific sub-RPM for specific drivers.
Known issue?
Thanks and best regards,
-- Didi