Upgrade from 4.4.3 to 4.4.4 (oVirt Node) - vdsmd.service/start failed with result 'dependency'
Hi all, I have just upgraded one of my oVirt nodes from 4.4.3 to 4.4.4. After the reboot, the 4.4.4 image is correctly loaded but vdsmd is not starting due to this error: vdsmd.service: Job vdsmd.service/start failed with result 'dependency'. Looks like it has a dependency on mom-vdsm, and this as well has a dependency issue: mom-vdsm.service: Job mom-vdsm.service/start failed with result 'dependency'. After some investigation looks like mom-vdsm has a dependency on ovsdb-server, and this is the unit creating the problem: ovs-delete-transient-ports.service: Starting requested but asserts failed. Assertion failed for Open vSwitch Delete Transient Ports Failed to start Open vSwitch Database Unit. Details below: -- Unit ovsdb-server.service has begun starting up. Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net chown[13658]: /usr/bin/chown: cannot access '/var/run/openvswitch': No such file or directory Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net ovs-ctl[13667]: /etc/openvswitch/conf.db does not exist ... (warning). Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net ovsdb-tool[13714]: ovs|00001|lockfile|WARN|/etc/openvswitch/.conf.db.~lock~: failed to open lock file: Permission denied Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net ovs-ctl[13667]: Creating empty database /etc/openvswitch/conf.db ovsdb-tool: I/O error: /etc/openvswitch/conf.db: failed to lock lockfile (Resource temporarily unavailable) Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net ovsdb-tool[13714]: ovs|00002|lockfile|WARN|/etc/openvswitch/.conf.db.~lock~: failed to lock file: Resource temporarily unavailable Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net ovs-ctl[13667]: [FAILED] Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net systemd[1]: ovsdb-server.service: Control process exited, code=exited status=1 Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net systemd[1]: ovsdb-server.service: Failed with result 'exit-code'. -- Subject: Unit failed Any suggestions? Thanks, Marco
Hi, just a quick correction on the below -- the dependency chain is vdsmd --> vdsm-network --> openvswitch --> ovsdb-server (failed) The conf.db file and /var/run/openvswitch dir are in 4.4.3 but not in 4.4.4 In 4.4.3 looks like the vdsm-network service does not depend from openvswitch: *4.4.3* -- [Unit] Description=Virtual Desktop Server Manager network restoration Wants=network.target Requires=libvirtd.service After=libvirtd.service *4.4.4* -- [Unit] Description=Virtual Desktop Server Manager network restoration Wants=network.target Requires=libvirtd.service openvswitch.service NetworkManager.service After=libvirtd.service openvswitch.service NetworkManager.service Regards, Marco On Thu, 24 Dec 2020 at 12:29, Marco Fais <evilmf@gmail.com> wrote:
Hi all,
I have just upgraded one of my oVirt nodes from 4.4.3 to 4.4.4.
After the reboot, the 4.4.4 image is correctly loaded but vdsmd is not starting due to this error:
vdsmd.service: Job vdsmd.service/start failed with result 'dependency'.
Looks like it has a dependency on mom-vdsm, and this as well has a dependency issue:
mom-vdsm.service: Job mom-vdsm.service/start failed with result 'dependency'.
After some investigation looks like mom-vdsm has a dependency on ovsdb-server, and this is the unit creating the problem:
ovs-delete-transient-ports.service: Starting requested but asserts failed. Assertion failed for Open vSwitch Delete Transient Ports Failed to start Open vSwitch Database Unit.
Details below: -- Unit ovsdb-server.service has begun starting up. Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net chown[13658]: /usr/bin/chown: cannot access '/var/run/openvswitch': No such file or directory Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net ovs-ctl[13667]: /etc/openvswitch/conf.db does not exist ... (warning). Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net ovsdb-tool[13714]: ovs|00001|lockfile|WARN|/etc/openvswitch/.conf.db.~lock~: failed to open lock file: Permission denied Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net ovs-ctl[13667]: Creating empty database /etc/openvswitch/conf.db ovsdb-tool: I/O error: /etc/openvswitch/conf.db: failed to lock lockfile (Resource temporarily unavailable) Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net ovsdb-tool[13714]: ovs|00002|lockfile|WARN|/etc/openvswitch/.conf.db.~lock~: failed to lock file: Resource temporarily unavailable Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net ovs-ctl[13667]: [FAILED] Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net systemd[1]: ovsdb-server.service: Control process exited, code=exited status=1 Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net systemd[1]: ovsdb-server.service: Failed with result 'exit-code'. -- Subject: Unit failed
Any suggestions?
Thanks, Marco
On Thu, Dec 24, 2020 at 5:14 PM Marco Fais <evilmf@gmail.com> wrote:
Hi,
just a quick correction on the below -- the dependency chain is vdsmd --> vdsm-network --> openvswitch --> ovsdb-server (failed)
The conf.db file and /var/run/openvswitch dir are in 4.4.3 but not in 4.4.4 In 4.4.3 looks like the vdsm-network service does not depend from openvswitch:
*4.4.3* -- [Unit] Description=Virtual Desktop Server Manager network restoration Wants=network.target Requires=libvirtd.service After=libvirtd.service
*4.4.4* -- [Unit] Description=Virtual Desktop Server Manager network restoration Wants=network.target Requires=libvirtd.service openvswitch.service NetworkManager.service After=libvirtd.service openvswitch.service NetworkManager.service
Regards, Marco
On Thu, 24 Dec 2020 at 12:29, Marco Fais <evilmf@gmail.com> wrote:
Hi all,
I have just upgraded one of my oVirt nodes from 4.4.3 to 4.4.4.
After the reboot, the 4.4.4 image is correctly loaded but vdsmd is not starting due to this error:
vdsmd.service: Job vdsmd.service/start failed with result 'dependency'.
Looks like it has a dependency on mom-vdsm, and this as well has a dependency issue:
mom-vdsm.service: Job mom-vdsm.service/start failed with result 'dependency'.
After some investigation looks like mom-vdsm has a dependency on ovsdb-server, and this is the unit creating the problem:
ovs-delete-transient-ports.service: Starting requested but asserts failed. Assertion failed for Open vSwitch Delete Transient Ports Failed to start Open vSwitch Database Unit.
Details below: -- Unit ovsdb-server.service has begun starting up. Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net chown[13658]: /usr/bin/chown: cannot access '/var/run/openvswitch': No such file or directory Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net ovs-ctl[13667]: /etc/openvswitch/conf.db does not exist ... (warning). Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net ovsdb-tool[13714]: ovs|00001|lockfile|WARN|/etc/openvswitch/.conf.db.~lock~: failed to open lock file: Permission denied Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net ovs-ctl[13667]: Creating empty database /etc/openvswitch/conf.db ovsdb-tool: I/O error: /etc/openvswitch/conf.db: failed to lock lockfile (Resource temporarily unavailable) Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net ovsdb-tool[13714]: ovs|00002|lockfile|WARN|/etc/openvswitch/.conf.db.~lock~: failed to lock file: Resource temporarily unavailable Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net ovs-ctl[13667]: [FAILED] Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net systemd[1]: ovsdb-server.service: Control process exited, code=exited status=1 Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net systemd[1]: ovsdb-server.service: Failed with result 'exit-code'. -- Subject: Unit failed
Any suggestions?
Thanks, Marco
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/Z7XM6QW2GP3FQK...
Hi, the dependency on openvswitch comes from 4.4.4 this is correct. However I am not entirely sure why openvswitch would refuse to start. Do you have some special OvS configuration in place on the affected host? Thank you. Regards, Ales -- Ales Musil Software Engineer - RHV Network Red Hat EMEA <https://www.redhat.com> amusil@redhat.com IM: amusil <https://red.ht/sig>
Hello Marco, can you please check the owner and groups of /etc/openvswitch/ and /var/run/openvswitch and the files in these directories ? Are there any hints if this issue might be related to *Bug 1909782* <https://bugzilla.redhat.com/show_bug.cgi?id=1909782> - /etc/openvswitch permissions broken after upgrade ? Dominik On Mon, Jan 4, 2021 at 7:29 AM Ales Musil <amusil@redhat.com> wrote:
On Thu, Dec 24, 2020 at 5:14 PM Marco Fais <evilmf@gmail.com> wrote:
Hi,
just a quick correction on the below -- the dependency chain is vdsmd --> vdsm-network --> openvswitch --> ovsdb-server (failed)
The conf.db file and /var/run/openvswitch dir are in 4.4.3 but not in 4.4.4 In 4.4.3 looks like the vdsm-network service does not depend from openvswitch:
*4.4.3* -- [Unit] Description=Virtual Desktop Server Manager network restoration Wants=network.target Requires=libvirtd.service After=libvirtd.service
*4.4.4* -- [Unit] Description=Virtual Desktop Server Manager network restoration Wants=network.target Requires=libvirtd.service openvswitch.service NetworkManager.service After=libvirtd.service openvswitch.service NetworkManager.service
Regards, Marco
On Thu, 24 Dec 2020 at 12:29, Marco Fais <evilmf@gmail.com> wrote:
Hi all,
I have just upgraded one of my oVirt nodes from 4.4.3 to 4.4.4.
After the reboot, the 4.4.4 image is correctly loaded but vdsmd is not starting due to this error:
vdsmd.service: Job vdsmd.service/start failed with result 'dependency'.
Looks like it has a dependency on mom-vdsm, and this as well has a dependency issue:
mom-vdsm.service: Job mom-vdsm.service/start failed with result 'dependency'.
After some investigation looks like mom-vdsm has a dependency on ovsdb-server, and this is the unit creating the problem:
ovs-delete-transient-ports.service: Starting requested but asserts failed. Assertion failed for Open vSwitch Delete Transient Ports Failed to start Open vSwitch Database Unit.
Details below: -- Unit ovsdb-server.service has begun starting up. Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net chown[13658]: /usr/bin/chown: cannot access '/var/run/openvswitch': No such file or directory Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net ovs-ctl[13667]: /etc/openvswitch/conf.db does not exist ... (warning). Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net ovsdb-tool[13714]: ovs|00001|lockfile|WARN|/etc/openvswitch/.conf.db.~lock~: failed to open lock file: Permission denied Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net ovs-ctl[13667]: Creating empty database /etc/openvswitch/conf.db ovsdb-tool: I/O error: /etc/openvswitch/conf.db: failed to lock lockfile (Resource temporarily unavailable) Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net ovsdb-tool[13714]: ovs|00002|lockfile|WARN|/etc/openvswitch/.conf.db.~lock~: failed to lock file: Resource temporarily unavailable Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net ovs-ctl[13667]: [FAILED] Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net systemd[1]: ovsdb-server.service: Control process exited, code=exited status=1 Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net systemd[1]: ovsdb-server.service: Failed with result 'exit-code'. -- Subject: Unit failed
Any suggestions?
Thanks, Marco
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/Z7XM6QW2GP3FQK...
Hi,
the dependency on openvswitch comes from 4.4.4 this is correct. However I am not entirely sure why openvswitch would refuse to start. Do you have some special OvS configuration in place on the affected host?
Thank you. Regards, Ales
--
Ales Musil
Software Engineer - RHV Network
Red Hat EMEA <https://www.redhat.com>
amusil@redhat.com IM: amusil <https://red.ht/sig> _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/KHG6ISFOUARQ2K...
Dominik, Ales, it was indeed the effect of the bug you have highlighted. Changing the ownership of /etc/openvswitch solved the problem. Interestingly it only happened to a portion of the ~20 nodes I have upgraded -- I believe I had different behaviour depending on the specific version used for installation and upgrade path taken, but not 100% sure. Thanks Marco On Mon, 4 Jan 2021 at 10:34, Dominik Holler <dholler@redhat.com> wrote:
Hello Marco, can you please check the owner and groups of /etc/openvswitch/ and /var/run/openvswitch and the files in these directories ?
Are there any hints if this issue might be related to *Bug 1909782* <https://bugzilla.redhat.com/show_bug.cgi?id=1909782> - /etc/openvswitch permissions broken after upgrade ?
Dominik
On Mon, Jan 4, 2021 at 7:29 AM Ales Musil <amusil@redhat.com> wrote:
On Thu, Dec 24, 2020 at 5:14 PM Marco Fais <evilmf@gmail.com> wrote:
Hi,
just a quick correction on the below -- the dependency chain is vdsmd --> vdsm-network --> openvswitch --> ovsdb-server (failed)
The conf.db file and /var/run/openvswitch dir are in 4.4.3 but not in 4.4.4 In 4.4.3 looks like the vdsm-network service does not depend from openvswitch:
*4.4.3* -- [Unit] Description=Virtual Desktop Server Manager network restoration Wants=network.target Requires=libvirtd.service After=libvirtd.service
*4.4.4* -- [Unit] Description=Virtual Desktop Server Manager network restoration Wants=network.target Requires=libvirtd.service openvswitch.service NetworkManager.service After=libvirtd.service openvswitch.service NetworkManager.service
Regards, Marco
On Thu, 24 Dec 2020 at 12:29, Marco Fais <evilmf@gmail.com> wrote:
Hi all,
I have just upgraded one of my oVirt nodes from 4.4.3 to 4.4.4.
After the reboot, the 4.4.4 image is correctly loaded but vdsmd is not starting due to this error:
vdsmd.service: Job vdsmd.service/start failed with result 'dependency'.
Looks like it has a dependency on mom-vdsm, and this as well has a dependency issue:
mom-vdsm.service: Job mom-vdsm.service/start failed with result 'dependency'.
After some investigation looks like mom-vdsm has a dependency on ovsdb-server, and this is the unit creating the problem:
ovs-delete-transient-ports.service: Starting requested but asserts failed. Assertion failed for Open vSwitch Delete Transient Ports Failed to start Open vSwitch Database Unit.
Details below: -- Unit ovsdb-server.service has begun starting up. Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net chown[13658]: /usr/bin/chown: cannot access '/var/run/openvswitch': No such file or directory Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net ovs-ctl[13667]: /etc/openvswitch/conf.db does not exist ... (warning). Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net ovsdb-tool[13714]: ovs|00001|lockfile|WARN|/etc/openvswitch/.conf.db.~lock~: failed to open lock file: Permission denied Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net ovs-ctl[13667]: Creating empty database /etc/openvswitch/conf.db ovsdb-tool: I/O error: /etc/openvswitch/conf.db: failed to lock lockfile (Resource temporarily unavailable) Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net ovsdb-tool[13714]: ovs|00002|lockfile|WARN|/etc/openvswitch/.conf.db.~lock~: failed to lock file: Resource temporarily unavailable Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net ovs-ctl[13667]: [FAILED] Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net systemd[1]: ovsdb-server.service: Control process exited, code=exited status=1 Dec 24 12:21:57 LAB-CNVirt-H04.ngv.eircom.net systemd[1]: ovsdb-server.service: Failed with result 'exit-code'. -- Subject: Unit failed
Any suggestions?
Thanks, Marco
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/Z7XM6QW2GP3FQK...
Hi,
the dependency on openvswitch comes from 4.4.4 this is correct. However I am not entirely sure why openvswitch would refuse to start. Do you have some special OvS configuration in place on the affected host?
Thank you. Regards, Ales
--
Ales Musil
Software Engineer - RHV Network
Red Hat EMEA <https://www.redhat.com>
amusil@redhat.com IM: amusil <https://red.ht/sig> _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/KHG6ISFOUARQ2K...
Hi all, i have the same error. I have 2 nodes. The first I updated it and everything ok! The second after the upgrade and after reboot and display the following error: Host XX1 installation failed. Task Start and enable services failed to execute. Please check logs for more details: /var/log/ovirt-engine/host-deploy/ovirt-host-deploy-ansible-20210201145717-192.168.197.2-002dc9a6-fdf8-47b9-95a8-eaa2d5b1066b.log. The error is due to openvswitch failing to restart I checked the permissions and they look fine .... I tried to reinstall it several times but still the same error. I was thinking of trying to reinstall it via the iso image. Any suggestions? Thanks so much nikkognt
Hi, I add that in the first host in the directory / etc / openvswitch there are files but in the second host that has problems in the same directory is empty. How can I do? Thanks so much
participants (4)
-
Ales Musil -
Dominik Holler -
Marco Fais -
nikkognt＠gmail.com