[ovirt-users] VDSM service won't start

Fri Jan 13 21:46:46 UTC 2017

Oh, I stumbled onto something relevant.

I noticed on the host that was working correctly that the ifcfg-enp6s0 
file included a line for "BRIDGE=ovirtmgmt", and the other two didn't 
have that line. When I added that line to the other two hosts, and 
restarted networking, I was able to get those hosts in a status of "UP".

That file is autogenerated by VDSM, so I wondered if it would survive a 
reboot. When I rebooted, the line had been removed again by VDSM.

So, I guess the final question then is how to get persistence in keeping 
this BRIDGE line from getting removed across reboots?

On 1/13/2017 2:54 PM, Nir Soffer wrote:
> On Fri, Jan 13, 2017 at 9:24 PM, paul.greene.va
> <paul.greene.va at verizon.net> wrote:
>> Output below ...
>>
>>
>>
>> On 1/13/2017 1:47 PM, Nir Soffer wrote:
>>> On Fri, Jan 13, 2017 at 5:45 PM, paul.greene.va
>>> <paul.greene.va at verizon.net> wrote:
>>>> All,
>>>>
>>>> I'm having an issue with the vdsmd service refusing to start on a fresh
>>>> install of RHEL 7.2, RHEV version 4.0.
>>>>
>>>> It initially came up correctly, and the command "ip a" showed a
>>>> "vdsmdummy"
>>>> interface and a "ovirtmgmt" interface. However after a couple of reboots,
>>>> those interfaces disappeared, and running "systemctl status vdsmd"
>>>> generated
>>>> the message "Dependency failed for Virtual Desktop Service Manager/Job
>>>> vdsmd.service/start failed with result 'dependency'". Didn't say what
>>>> dependency though
>>>>
>>>> I have 3 hosts where this happening on 2 out of 3 hosts. For some odd
>>>> reason, the one host isn't having any problems.
>>>>
>>>> In a Google search I found an instance where system clock timing was out
>>>> of
>>>> sync, and that messed it up. I checked all three hosts, as well as the
>>>> RHEV
>>>> manager and they all had chronyd running and the clocks appeared to be in
>>>> sync.
>>>>
>>>> After a reboot the virtual interfaces usually initially come up, but go
>>>> down
>>>> again within a few minutes.
>>>>
>>>> Running journalctl -xe gives these three messages:
>>>>
>>>> "failed to start Virtual Desktop Server Manager network restoration"
>>>>
>>>> "Dependency failed for Virtual Desktop Server Manager"  (but it doesn't
>>>> say
>>>> which dependency failed"
>>>>
>>>> "Dependency failed for MOM instance configured for VDSM purposes"
>>>> (again,
>>>> doesn't way which dependency)
>>>>
>>>> Any suggestions?
>>> Can you share the output of:
>>>
>>> systemctl status vdsmd
>>> systemctl status mom
>>> systemctl status libvirtd
>>> journalctl -xe
>>>
>>> Nir
>>>
>> Sure, here you go ....
>>
>>
>>
>> [root at rhevh03 vdsm]# systemctl status vdsmd
>> ā vdsmd.service - Virtual Desktop Server Manager
>>     Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled; vendor
>> preset: enabled)
>>     Active: inactive (dead)
>>
>> Jan 13 12:01:53 rhevh03 systemd[1]: Dependency failed for Virtual Desktop
>> Server Manager.
>> Jan 13 12:01:53 rhevh03 systemd[1]: Job vdsmd.service/start failed with
>> result 'dependency'.
>> Jan 13 13:51:50 rhevh03 systemd[1]: Dependency failed for Virtual Desktop
>> Server Manager.
>> Jan 13 13:51:50 rhevh03 systemd[1]: Job vdsmd.service/start failed with
>> result 'dependency'.
>> Jan 13 13:55:15 rhevh03 systemd[1]: Dependency failed for Virtual Desktop
>> Server Manager.
>> Jan 13 13:55:15 rhevh03 systemd[1]: Job vdsmd.service/start failed with
>> result 'dependency'.
>>
>>
>>
>> [root at rhevh03 vdsm]# systemctl status momd
>> ā momd.service - Memory Overcommitment Manager Daemon
>>     Loaded: loaded (/usr/lib/systemd/system/momd.service; static; vendor
>> preset: disabled)
>>     Active: inactive (dead) since Fri 2017-01-13 13:53:09 EST; 2min 26s ago
>>    Process: 28294 ExecStart=/usr/sbin/momd -c /etc/momd.conf -d --pid-file
>> /var/run/momd.pid (code=exited, status=0/SUCCESS)
>>   Main PID: 28298 (code=exited, status=0/SUCCESS)
>>
>> Jan 13 13:53:09 rhevh03 systemd[1]: Starting Memory Overcommitment Manager
>> Daemon...
>> Jan 13 13:53:09 rhevh03 systemd[1]: momd.service: Supervising process 28298
>> which is not our child. We'll most likely not notice when it exits.
>> Jan 13 13:53:09 rhevh03 systemd[1]: Started Memory Overcommitment Manager
>> Daemon.
>> Jan 13 13:53:09 rhevh03 python[28298]: No worthy mechs found
>>
>>
>>
>> [root at rhevh03 vdsm]# systemctl status libvirtd
>> ā libvirtd.service - Virtualization daemon
>>     Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor
>> preset: enabled)
>>    Drop-In: /etc/systemd/system/libvirtd.service.d
>>             āāunlimited-core.conf
>>     Active: active (running) since Fri 2017-01-13 13:50:47 EST; 8min ago
>>       Docs: man:libvirtd(8)
>>             http://libvirt.org
>>   Main PID: 27964 (libvirtd)
>>     CGroup: /system.slice/libvirtd.service
>>             āā27964 /usr/sbin/libvirtd --listen
>>
>> Jan 13 13:50:47 rhevh03 systemd[1]: Starting Virtualization daemon...
>> Jan 13 13:50:47 rhevh03 systemd[1]: Started Virtualization daemon.
>> Jan 13 13:53:09 rhevh03 libvirtd[27964]: libvirt version: 2.0.0, package:
>> 10.el7_3.2 (Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>,
>> 2016-11-10-04:43:57, x86-034.build.eng.bos.redhat.com)
>> Jan 13 13:53:09 rhevh03 libvirtd[27964]: hostname: rhevh03
>> Jan 13 13:53:09 rhevh03 libvirtd[27964]: End of file while reading data:
>> Input/output error
>>
>>
>> [root at rhevh03 vdsm]# journalctl -xe
>> Jan 13 13:55:15 rhevh03 vdsm-tool[28334]: File
>> "/usr/lib/python2.7/site-packages/vdsm/network/configurators/ifcfg.py", line
>> 951, in _exec_ifup
>> Jan 13 13:55:15 rhevh03 vdsm-tool[28334]: _exec_ifup_by_name(iface.name,
>> cgroup)
>> Jan 13 13:55:15 rhevh03 vdsm-tool[28334]: File
>> "/usr/lib/python2.7/site-packages/vdsm/network/configurators/ifcfg.py", line
>> 937, in _exec_ifup_by_name
>> Jan 13 13:55:15 rhevh03 vdsm-tool[28334]: raise
>> ConfigNetworkError(ERR_FAILED_IFUP, out[-1] if out else '')
>> Jan 13 13:55:15 rhevh03 vdsm-tool[28334]:
>> vdsm.network.errors.ConfigNetworkError: (29, 'Determining IP information for
>> ovirtmgmt... failed.')
>> Jan 13 13:55:15 rhevh03 vdsm-tool[28334]: Traceback (most recent call last):
>> Jan 13 13:55:15 rhevh03 vdsm-tool[28334]: File "/usr/bin/vdsm-tool", line
>> 219, in main
>> Jan 13 13:55:15 rhevh03 vdsm-tool[28334]: return
>> tool_command[cmd]["command"](*args)
>> Jan 13 13:55:15 rhevh03 vdsm-tool[28334]: File
>> "/usr/lib/python2.7/site-packages/vdsm/tool/restore_nets.py", line 41, in
>> restore_command
>> Jan 13 13:55:15 rhevh03 vdsm-tool[28334]: exec_restore(cmd)
>> Jan 13 13:55:15 rhevh03 vdsm-tool[28334]: File
>> "/usr/lib/python2.7/site-packages/vdsm/tool/restore_nets.py", line 54, in
>> exec_restore
>> Jan 13 13:55:15 rhevh03 vdsm-tool[28334]: raise EnvironmentError('Failed to
>> restore the persisted networks')
>> Jan 13 13:55:15 rhevh03 vdsm-tool[28334]: EnvironmentError: Failed to
>> restore the persisted networks
>> Jan 13 13:55:15 rhevh03 postfix/local[28598]: 05920804620D:
>> to=<root at localhost.localdomain>, orig_to=<root at localhost>, relay=local,
>> delay=0.03, delays=0.02/0.01/0/0, dsn=2.0.0, status=sent (delivered to
>> Jan 13 13:55:15 rhevh03 postfix/qmgr[3522]: 05920804620D: removed
>> Jan 13 13:55:15 rhevh03 systemd[1]: vdsm-network.service: main process
>> exited, code=exited, status=1/FAILURE
>> Jan 13 13:55:15 rhevh03 systemd[1]: Failed to start Virtual Desktop Server
>> Manager network restoration.
>> -- Subject: Unit vdsm-network.service has failed
>> -- Defined-By: systemd
>> -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
>> --
>> -- Unit vdsm-network.service has failed.
> Seems that you have a networking issue, adding Dan.
>
>> --
>> -- The result is failed.
>> Jan 13 13:55:15 rhevh03 systemd[1]: Dependency failed for Virtual Desktop
>> Server Manager.
>> -- Subject: Unit vdsmd.service has failed
>> -- Defined-By: systemd
>> -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
>> --
>> -- Unit vdsmd.service has failed.
>> --
>> -- The result is dependency.
>> Jan 13 13:55:15 rhevh03 systemd[1]: Dependency failed for MOM instance
>> configured for VDSM purposes.
>> -- Subject: Unit mom-vdsm.service has failed
>> -- Defined-By: systemd
>> -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
>> --
>> -- Unit mom-vdsm.service has failed.
>> --
>> -- The result is dependency.
>> Jan 13 13:55:15 rhevh03 systemd[1]: Job mom-vdsm.service/start failed with
>> result 'dependency'.
>> Jan 13 13:55:15 rhevh03 systemd[1]: Job vdsmd.service/start failed with
>> result 'dependency'.
>> Jan 13 13:55:15 rhevh03 systemd[1]: Unit vdsm-network.service entered failed
>> state.
>> Jan 13 13:55:15 rhevh03 systemd[1]: vdsm-network.service failed.
>> Jan 13 13:55:15 rhevh03 polkitd[1675]: Unregistered Authentication Agent for
>> unix-process:28317:681858 (system bus name :1.121, object path
>> /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.
>> Jan 13 14:00:01 rhevh03 systemd[1]: Started Session 16 of user root.
>> -- Subject: Unit session-16.scope has finished start-up
>> -- Defined-By: systemd
>> -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
>> --
>> -- Unit session-16.scope has finished starting up.
>> --
>> -- The start-up result is done.
>> Jan 13 14:00:01 rhevh03 systemd[1]: Starting Session 16 of user root.
>> -- Subject: Unit session-16.scope has begun start-up
>> -- Defined-By: systemd
>> -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
>> --
>> -- Unit session-16.scope has begun starting up.
>> Jan 13 14:00:01 rhevh03 CROND[28607]: (root) CMD (/usr/lib64/sa/sa1 1 1)
>>