vdsmd.service stuck in state: activating

= Noob questions = A) First time here. I couldn't find the rulebook/netiquette for oVirt mailing list, could you give me a link? B) What do you use here, top or bottom posting? C) Which is the line length limit? I've written this email manually entering new line after 80 chars, is it fine? D) Which is the log length limit? E) When should I use pastebin or similar websites? = /Noob questions = = Issue questions = TL;DR self host engine deployment fails, vdsmd.service has this error I can't find on Google: 'sysctl: cannot stat /proc/sys/ssl: No such file or directory' Long description: I've been trying for a while to deploy an oVirt self hosted engine but it failed all the time. After every failed deployment, node status resulted in 'FAIL' (or 'DEGRADED', I don't remember) and to restore it to 'OK' status I'd to run 'vdsm-tool configure --force'. vdsm-tool configuration never went through fully successful: < code > abrt is not configured for vdsm lvm is configured for vdsm libvirt is not configured for vdsm yet FAILED: conflicting vdsm and libvirt-qemu tls configuration. vdsm.conf with ssl=True requires the following changes: libvirtd.conf: listen_tcp=0, auth_tcp="sasl", listen_tls=1 qemu.conf: spice_tls=1. Current revision of multipath.conf detected, preserving < /code > I set 'auth_tcp = "sasl"' (in "/etc/libvirtd.conf") and 'spice_tls=1' (in "/etc/libvirt/qemu.conf") and restart vdsdmd.service, but it still fails: < code > Starting Virtual Desktop Server Manager... _init_common.sh[2839]: vdsm: Running mkdirs _init_common.sh[2839]: vdsm: Running configure_coredump _init_common.sh[2839]: vdsm: Running configure_vdsm_logs _init_common.sh[2839]: vdsm: Running wait_for_network _init_common.sh[2839]: vdsm: Running run_init_hooks _init_common.sh[2839]: vdsm: Running check_is_configured _init_common.sh[2839]: abrt is already configured for vdsm _init_common.sh[2839]: lvm is configured for vdsm _init_common.sh[2839]: libvirt is already configured for vdsm _init_common.sh[2839]: Current revision of multipath.conf detected, preserving _init_common.sh[2839]: vdsm: Running validate_configuration _init_common.sh[2839]: SUCCESS: ssl configured to true. No conflicts _init_common.sh[2839]: vdsm: Running prepare_transient_repository _init_common.sh[2839]: vdsm: Running syslog_available _init_common.sh[2839]: vdsm: Running nwfilter _init_common.sh[2839]: vdsm: Running dummybr _init_common.sh[2839]: vdsm: Running tune_system _init_common.sh[2839]: sysctl: cannot stat /proc/sys/ssl: No such file or directory md[1]: vdsmd.service: control process exited, code=exited status=1 md[1]: Failed to start Virtual Desktop Server Manager. md[1]: Unit vdsmd.service entered failed state. md[1]: vdsmd.service failed. md[1]: vdsmd.service holdoff time over, scheduling restart. < /code > I've no idea about how to investigate, solve or troubleshoot this issue. I've updated yesterday to the last version but nothing changed: ovirt-release-master-4.3.0-0.1.master.20180820000052.gitdd598f0.el7.noarch = /Issue questions =

Hi, I don't want to bump, but I've other stuff to post so I'd like to receive answers to my noob questions, as I've wall of logs incoming.

Hi there and welcome aboard! = Noob questions =
A) First time here. I couldn't find the rulebook/netiquette for oVirt mailing
list, could you give me a link?
Asking questions regarding to ovirt is a part of this mailing porpuses, so I don't think there's a rulebook. It was meant to help you and others :) For my opinion, just try to be as imformative as you can. If there's some more needed information, you'll probably be asked to supply it :) B) What do you use here, top or bottom posting?
Depends on you.
C) Which is the line length limit? I've written this email manually entering new line after 80 chars, is it fine?
Once again - your call :)
D) Which is the log length limit?
It's better to keep it focused on your issue, but not too short - basically it should contain the operation you're trying to do. If some more logs are needed, you'll be asked to add them. E) When should I use pastebin or similar websites?
As much as I know, attaching the logs as mail attachments are more common. It's worth checking if the data you've pasted on patebin will be expired eventually, so I personally think that attaching the logs here is will be more durable. = /Noob questions =
= Issue questions = TL;DR self host engine deployment fails, vdsmd.service has this error I can't find on Google: 'sysctl: cannot stat /proc/sys/ssl: No such file or directory'
Long description: I've been trying for a while to deploy an oVirt self hosted engine but it failed all the time. After every failed deployment, node status resulted in 'FAIL' (or 'DEGRADED', I don't remember) and to restore it to 'OK' status I'd to run 'vdsm-tool configure --force'.
vdsm-tool configuration never went through fully successful: < code > abrt is not configured for vdsm lvm is configured for vdsm libvirt is not configured for vdsm yet FAILED: conflicting vdsm and libvirt-qemu tls configuration. vdsm.conf with ssl=True requires the following changes: libvirtd.conf: listen_tcp=0, auth_tcp="sasl", listen_tls=1 qemu.conf: spice_tls=1. Current revision of multipath.conf detected, preserving < /code >
I set 'auth_tcp = "sasl"' (in "/etc/libvirtd.conf") and 'spice_tls=1' (in "/etc/libvirt/qemu.conf") and restart vdsdmd.service, but it still fails: < code > Starting Virtual Desktop Server Manager... _init_common.sh[2839]: vdsm: Running mkdirs _init_common.sh[2839]: vdsm: Running configure_coredump _init_common.sh[2839]: vdsm: Running configure_vdsm_logs _init_common.sh[2839]: vdsm: Running wait_for_network _init_common.sh[2839]: vdsm: Running run_init_hooks _init_common.sh[2839]: vdsm: Running check_is_configured _init_common.sh[2839]: abrt is already configured for vdsm _init_common.sh[2839]: lvm is configured for vdsm _init_common.sh[2839]: libvirt is already configured for vdsm _init_common.sh[2839]: Current revision of multipath.conf detected, preserving _init_common.sh[2839]: vdsm: Running validate_configuration _init_common.sh[2839]: SUCCESS: ssl configured to true. No conflicts _init_common.sh[2839]: vdsm: Running prepare_transient_repository _init_common.sh[2839]: vdsm: Running syslog_available _init_common.sh[2839]: vdsm: Running nwfilter _init_common.sh[2839]: vdsm: Running dummybr _init_common.sh[2839]: vdsm: Running tune_system _init_common.sh[2839]: sysctl: cannot stat /proc/sys/ssl: No such file or directory md[1]: vdsmd.service: control process exited, code=exited status=1 md[1]: Failed to start Virtual Desktop Server Manager. md[1]: Unit vdsmd.service entered failed state. md[1]: vdsmd.service failed. md[1]: vdsmd.service holdoff time over, scheduling restart. < /code >
I've no idea about how to investigate, solve or troubleshoot this issue.
I've updated yesterday to the last version but nothing changed: ovirt-release-master-4.3.0-0.1.master.20180820000052.gitdd598f0.el7.noarch = /Issue questions =
I think that this one can help : https://www.ovirt.org/develop/developer-guide/vdsm/installing-vdsm-from-rpm/... Make sure you've installed libvirt from the rpm as appeared on the link there. _______________________________________________
Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/4K6M7V5ELD5XBY...
*Regards,* *Shani Leviim*

On Wed, Oct 31, 2018 at 11:48 AM Shani Leviim <sleviim@redhat.com> wrote:
B) What do you use here, top or bottom posting?
Depends on you.
But if you want to get help from other users or developers reading this list, you should not use top posting.
C) Which is the line length limit? I've written this email manually
entering new line after 80 chars, is it fine?
Once again - your call :)
Keeping 60-70 characters per line will help others to read on a computer. However this does not people reading mobile phones. D) Which is the log length limit?
It's better to keep it focused on your issue, but not too short - basically it should contain the operation you're trying to do. If some more logs are needed, you'll be asked to add them.
When posting logs please trim the log so it starts at the start of a flow, and ends after the end of the flow. Always compress logs, I think xz format give the best compression these days. Nir

Thank you all for the replies, I'm sticking to bottom posting with 70 columns wide line, with the exception of log lines. On Wed, Oct 31, 2018 at 11:48 AM Shani Leviim <sleviim@redhat.com> wrote:
I think that this one can help : https://www.ovirt.org/develop/developer-guide/vdsm/installing-vdsm-from-rpm/... Make sure you've installed libvirt from the rpm as appeared on the link there.
I've followed your link, from my notes I see I've already tested it in the beginning of my messing. I've set 'ssl = false' in /etc/vdsm/vdsm.conf and run 'vdsm-tool configure --force'. Since it has just commented out 'spice_tls=1', I've forced it to 'spice_tls=0' (though it should give the same result as commenting out) and run once again 'vdsm-tool configure --force'. There was no warning report about non-configured stuff. Vdsm still fails to start. I've attached 2 logs, compressed in one archive 'vdsm_logs.xz'. These logs are: 1) vdsmd_journalctl.0.txt contains the result of 'journalctl -xe -u vdsmd' 2) vsdm_messages_err.1.txt contains the result of 'grep -i vdsm /var/log/messages' Quick summary with what I think are the relevant lines from the logs: # journalctl -xe -u vdsmd [...] Nov 06 14:18:28 onode1.ifac.cnr.it vdsmd_init_common.sh[18732]: vdsm: Running tune_system Nov 06 14:18:28 onode1.ifac.cnr.it vdsmd_init_common.sh[18732]: sysctl: cannot stat /proc/sys/ssl: No such file or directory Nov 06 14:18:28 onode1.ifac.cnr.it systemd[1]: vdsmd.service: control process exited, code=exited status=1 Nov 06 14:18:28 onode1.ifac.cnr.it systemd[1]: Failed to start Virtual Desktop Server Manager. [...] # grep -i vdsm /var/log/messages [...] Nov 6 14:24:59 onode1 vdsmd_init_common.sh: vdsm: Running tune_system Nov 6 14:24:59 onode1 vdsmd_init_common.sh: sysctl: cannot stat /proc/sys/ssl: No such file or directory Nov 6 14:24:59 onode1 vdsmd_init_common.sh: vdsm: stopped during execute tune_system task (task returned with error code 255). Nov 6 14:24:59 onode1 systemd: vdsmd.service: control process exited, code=exited status=1 Nov 6 14:24:59 onode1 systemd: Dependency failed for MOM instance configured for VDSM purposes. Nov 6 14:24:59 onode1 systemd: Job mom-vdsm.service/start failed with result 'dependency'. Nov 6 14:24:59 onode1 systemd: Unit vdsmd.service entered failed state. Nov 6 14:24:59 onode1 systemd: vdsmd.service failed. Nov 6 14:25:00 onode1 systemd: vdsmd.service holdoff time over, scheduling restart. [...] I can't find anything on Google and in oVirt documentation regarding these issues.

On Tue, Nov 6, 2018 at 3:58 PM Giannino Stoppani <sceglimilano@gmail.com> wrote:
Thank you all for the replies, I'm sticking to bottom posting with 70 columns wide line, with the exception of log lines.
On Wed, Oct 31, 2018 at 11:48 AM Shani Leviim <sleviim@redhat.com> wrote:
I think that this one can help : https://www.ovirt.org/develop/developer-guide/vdsm/installing-vdsm-from-rpm/... Make sure you've installed libvirt from the rpm as appeared on the link there.
I've followed your link, from my notes I see I've already tested it in the beginning of my messing. I've set 'ssl = false' in /etc/vdsm/vdsm.conf
Disabling ssl is not supported configuration. If can be used for debugging in development environment but it should not be used in production. This configuration is also not tested, so it is likely to break. Even if you disable ssl in vdsm, other parts of the system require ssl, like ovirt-imageio. Nir

I've always used ovirt with enabled ssl until now, and I've done so only because it's instructed by oVirt doc: https://www.ovirt.org/develop/developer-guide/vdsm/installing-vdsm-from-rpm/... Having ssl enabled, still lead to what I think are the same errors, as you can see from the following and the attached logs. # vi /etc/vdsm/vdsm.conf [vars] ssl = true Restarting vdsmd will lead to failure (expectedly, as we've just switched ssl mode): # systemctl restart vdsmd Job for vdsmd.service failed because the control process exited with error code. See "systemctl status vdsmd.service" and "journalctl -xe" for details. So I'll reconfigure it: # vdsm-tool configure --force Checking configuration status... abrt is already configured for vdsm lvm is configured for vdsm libvirt is already configured for vdsm FAILED: conflicting vdsm and libvirt-qemu tls configuration. vdsm.conf with ssl=True requires the following changes: libvirtd.conf: listen_tcp=0, auth_tcp="sasl", listen_tls=1 qemu.conf: spice_tls=1. Current revision of multipath.conf detected, preserving Running configure... Reconfiguration of abrt is done. Reconfiguration of passwd is done. Reconfiguration of libvirt is done. Done configuring modules to VDSM. As a quick check to see if everything has been configured correctly, I run again the configuration and this time there won't be anything to be done: # vdsm-tool configure --force Checking configuration status... abrt is already configured for vdsm lvm is configured for vdsm libvirt is already configured for vdsm SUCCESS: ssl configured to true. No conflicts Current revision of multipath.conf detected, preserving Running configure... Reconfiguration of abrt is done. Reconfiguration of passwd is done. Reconfiguration of libvirt is done. Done configuring modules to VDSM. Starting vdsmd still fails: # systemctl start vdsmd Job for vdsmd.service failed because the control process exited with error code. See "systemctl status vdsmd.service" and "journalctl -xe" for details. I've attached new logs for ssl enabled as xz archive. Il giorno mer 7 nov 2018 alle ore 12:07 Nir Soffer <nsoffer@redhat.com> ha scritto:
On Tue, Nov 6, 2018 at 3:58 PM Giannino Stoppani <sceglimilano@gmail.com> wrote:
Thank you all for the replies, I'm sticking to bottom posting with 70 columns wide line, with the exception of log lines.
On Wed, Oct 31, 2018 at 11:48 AM Shani Leviim <sleviim@redhat.com> wrote:
I think that this one can help : https://www.ovirt.org/develop/developer-guide/vdsm/installing-vdsm-from-rpm/... Make sure you've installed libvirt from the rpm as appeared on the link there.
I've followed your link, from my notes I see I've already tested it in the beginning of my messing. I've set 'ssl = false' in /etc/vdsm/vdsm.conf
Disabling ssl is not supported configuration. If can be used for debugging in development environment but it should not be used in production. This configuration is also not tested, so it is likely to break.
Even if you disable ssl in vdsm, other parts of the system require ssl, like ovirt-imageio.
Nir

On Mon, Nov 12, 2018 at 4:38 PM Giannino Stoppani <sceglimilano@gmail.com> wrote:
I've always used ovirt with enabled ssl until now, and I've done so only because it's instructed by oVirt doc:
https://www.ovirt.org/develop/developer-guide/vdsm/installing-vdsm-from-rpm/...
Having ssl enabled, still lead to what I think are the same errors, as you can see from the following and the attached logs.
# vi /etc/vdsm/vdsm.conf [vars] ssl = true
Restarting vdsmd will lead to failure (expectedly, as we've just switched ssl mode): # systemctl restart vdsmd Job for vdsmd.service failed because the control process exited with error code. See "systemctl status vdsmd.service" and "journalctl -xe" for details.
So I'll reconfigure it: # vdsm-tool configure --force Checking configuration status...
abrt is already configured for vdsm lvm is configured for vdsm libvirt is already configured for vdsm FAILED: conflicting vdsm and libvirt-qemu tls configuration. vdsm.conf with ssl=True requires the following changes: libvirtd.conf: listen_tcp=0, auth_tcp="sasl", listen_tls=1 qemu.conf: spice_tls=1. Current revision of multipath.conf detected, preserving
Running configure... Reconfiguration of abrt is done. Reconfiguration of passwd is done. Reconfiguration of libvirt is done.
Done configuring modules to VDSM.
Looks fine...
As a quick check to see if everything has been configured correctly, I run again the configuration and this time there won't be anything to be done: # vdsm-tool configure --force
You can do this instead: vdsm-tool is-configured
Checking configuration status...
abrt is already configured for vdsm lvm is configured for vdsm libvirt is already configured for vdsm SUCCESS: ssl configured to true. No conflicts Current revision of multipath.conf detected, preserving
Running configure... Reconfiguration of abrt is done. Reconfiguration of passwd is done. Reconfiguration of libvirt is done.
Done configuring modules to VDSM.
Still ok...
Starting vdsmd still fails: # systemctl start vdsmd Job for vdsmd.service failed because the control process exited with error code. See "systemctl status vdsmd.service" and "journalctl -xe" for details.
I've attached new logs for ssl enabled as xz archive.
Good. What we see is: Nov 12 14:44:50 ovirtone.test.one vdsmd_init_common.sh[26855]: sysctl: cannot stat /proc/sys/ssl: No s Nov 12 14:44:50 ovirtone.test.one systemd[1]: vdsmd.service: control process exited, code=exited statu Nov 12 14:44:50 ovirtone.test.one systemd[1]: Failed to start Virtual Desktop Server Manager. Looks like a bug, can you file urgent bug for this with the attached logs? Nir
Il giorno mer 7 nov 2018 alle ore 12:07 Nir Soffer <nsoffer@redhat.com> ha scritto:
On Tue, Nov 6, 2018 at 3:58 PM Giannino Stoppani <sceglimilano@gmail.com> wrote:
Thank you all for the replies, I'm sticking to bottom posting with 70 columns wide line, with the exception of log lines.
On Wed, Oct 31, 2018 at 11:48 AM Shani Leviim <sleviim@redhat.com> wrote:
I think that this one can help : https://www.ovirt.org/develop/developer-guide/vdsm/installing-vdsm-from-rpm/... Make sure you've installed libvirt from the rpm as appeared on the link there.
I've followed your link, from my notes I see I've already tested it in the beginning of my messing. I've set 'ssl = false' in /etc/vdsm/vdsm.conf
Disabling ssl is not supported configuration. If can be used for debugging in development environment but it should not be used in production. This configuration is also not tested, so it is likely to break.
Even if you disable ssl in vdsm, other parts of the system require ssl, like ovirt-imageio.
Nir

Thanks, I've created this bug report, being my first one, would you all please check it and let me know if I did anything wrong or if I've missed something? https://bugzilla.redhat.com/show_bug.cgi?id=1649268
participants (4)
-
Giannino Stoppani
-
Nir Soffer
-
sceglimilano@gmail.com
-
Shani Leviim