[ovirt-users] Network config
Ondřej Svoboda
ondrej at svobodasoft.cz
Tue Jan 27 18:10:07 UTC 2015
From the first log (journal): VDSM managed to start, only it is being
run too early (it should be waiting patiently for its supervdsm
component to start responding).
Are there no "RTNETLINK answers: File exists" errors anymore?
Second log: This is normal operation.
So the host stopped responding? It should have come up, assuming that it
obtained the same address from DHCP (which it should, but please check).
Could you try using static network configuration first? And when it
succeeds you can try DHCP again.
===
About the GUI: Is your host's status now "Not Operational"? In that case
oVirt cannot tell anything about the status of the host's interfaces, it
may only reflect the status before communication was lost.
Please file a bug if this is really the case I described :-) It would
then be a matter of discussion if oVirt should display a yellow
"unknown" status of the interfaces, or keep displaying the last known
one. (The first solution would make it obvious that we really lost the
host and go looking for the host's status to discover it is "Not
Operational". The other is informative as well, just in a different way.)
On 27.1.2015 16:31, Jorick Astrego wrote:
>
> Done, maybe this helps:
>
> Jan 27 16:20:26 test2 vdsmd_init_common.sh: vdsm: Running
> unified_network_persistence_upgrade
> Jan 27 16:20:26 test2 vdsmd_init_common.sh: vdsm: Running restore_nets
> Jan 27 16:21:47 test2 systemd-logind: New session 5 of user root.
> Jan 27 16:21:47 test2 systemd: Starting Session 5 of user root.
> Jan 27 16:21:47 test2 systemd: Started Session 5 of user root.
> Jan 27 16:21:55 test2 systemd: vdsmd.service operation timed out.
> Terminating.
> Jan 27 16:21:55 test2 journal: End of file while reading data:
> Input/output error
> Jan 27 16:21:55 test2 systemd: Failed to start Virtual Desktop
> Server Manager.
> Jan 27 16:21:55 test2 systemd: Unit vdsmd.service entered failed
> state.
> Jan 27 16:21:55 test2 systemd: vdsmd.service holdoff time over,
> scheduling restart.
> Jan 27 16:21:55 test2 systemd: Stopping Virtual Desktop Server
> Manager...
> Jan 27 16:21:55 test2 systemd: Starting Virtual Desktop Server
> Manager...
> Jan 27 16:21:55 test2 vdsmd_init_common.sh: vdsm: Running mkdirs
>
> Still:
>
> MainThread::INFO::2015-01-27
> 16:21:56,964::netconfpersistence::158::root::(_clearDisk) Clearing
> /var/run/vdsm/netconf/nets/ and /var/run/vdsm/netconf/bonds/
> MainThread::DEBUG::2015-01-27
> 16:21:56,964::netconfpersistence::166::root::(_clearDisk) No
> existent config to clear.
> MainThread::INFO::2015-01-27
> 16:21:56,964::netconfpersistence::182::root::(save) Saved new
> config RunningConfig({}, {}) to /var/run/vdsm/netconf/nets/ and
> /var/run/vdsm/netconf/bonds/
> MainThread::DEBUG::2015-01-27
> 16:21:56,965::netconfpersistence::134::root::(_getConfigs)
> Non-existing config set.
>
> And when I want to reapply setup networks in ovirt I get:
>
>
> Error while executing action Commit Network changes: Network error
> during communication with the Host.
>
> Every interface and bond is still green and visible in the webgui by
> the way. That is also strange, every interface of the host is bonded
> and up while in reality the host has lost it's network config and
> doesn't respond anymore.
>
> Shouldn't they all be red on communication failure to the host?
>
> Cheers,
>
> Jorick
>
> On 01/27/2015 04:01 PM, Ondřej Svoboda wrote:
>> This is weird, like there were multiple dhclients on the bond. Can
>> you stop VDSM (systemctl stop vdsmd supervdsmd) and maybe `killall
>> -SIGKILL dhclient` and restart VDSM afterwards?
>>
>> On 27.1.2015 14:43, Jorick Astrego wrote:
>>> Ah yes, I forgot to disable it as I switched to foreman provisioning.
>>>
>>> Maybe it can be auto detected and disabled in the "install host"
>>> scripts when ovirt installs vdsm and sets up the networking. For
>>> people who forget to disable it.....
>>>
>>> But now I get this:
>>>
>>> sourceRoute::DEBUG::2015-01-27
>>> 14:33:57,416::utils::739::root::(execCmd) /sbin/ip -4 route add
>>> 0.0.0.0/0 via ***.***.***.*** dev bond0 table 174351414 (cwd None)
>>> sourceRoute::DEBUG::2015-01-27
>>> 14:33:57,420::utils::759::root::(execCmd) FAILED: <err> =
>>> 'RTNETLINK answers: File exists\n'; <rc> = 2
>>> sourceRoute::ERROR::2015-01-27
>>> 14:33:57,420::sourceroute::88::root::(configure) ip binary
>>> failed during source route configuration: ['RTNETLINK answers:
>>> File exists']
>>> sourceRoute::WARNING::2015-01-27
>>> 14:33:57,420::utils::129::root::(rmFile) File:
>>> /var/run/vdsm/trackedInterfaces/bond0 already removed
>>>
>>> Jan 27 14:38:19 test3.netbulae.test network[3042]: RTNETLINK
>>> answers: File exists
>>> Jan 27 14:38:19 test3.netbulae.test systemd[1]: network.service:
>>> control process exited, code=exited status=1
>>> Jan 27 14:38:19 test3.netbulae.test systemd[1]: Failed to start
>>> LSB: Bring up/down networking.
>>> -- Subject: Unit network.service has failed
>>> -- Defined-By: systemd
>>> -- Support:
>>> http://lists.freedesktop.org/mailman/listinfo/systemd-devel
>>> --
>>> -- Unit network.service has failed.
>>> --
>>> -- The result is failed.
>>> Jan 27 14:38:19 test3.netbulae.test systemd[1]: Unit
>>> network.service entered failed state.
>>>
>>>
>>>
>>> Bond0 is still there though
>>>
>>> sourceRoute::DEBUG::2015-01-27
>>> 14:33:57,410::sourceroutethread::39::root::(process_IN_CLOSE_WRITE_filePath)
>>> Responding to DHCP response in /var/run/vdsm/sourceRoutes/1422365637
>>> sourceRoute::INFO::2015-01-27
>>> 14:33:57,416::sourceroute::78::root::(configure) Configuring
>>> gateway - ip: **.***.**.**, network: 10.100.100.0/24, subnet:
>>> 255.255.255.0, gateway: **.***.**.**, table: 174351414, device:
>>> bond0
>>> sourceRoute::DEBUG::2015-01-27
>>> 14:33:57,416::utils::739::root::(execCmd) /sbin/ip -4 route add
>>> 0.0.0.0/0 via **.***.**.** dev bond0 table 174351414 (cwd None)
>>> sourceRoute::DEBUG::2015-01-27
>>> 14:33:57,420::utils::759::root::(execCmd) FAILED: <err> =
>>> 'RTNETLINK answers: File exists\n'; <rc> = 2
>>> sourceRoute::ERROR::2015-01-27
>>> 14:33:57,420::sourceroute::88::root::(configure) ip binary
>>> failed during source route configuration: ['RTNETLINK answers:
>>> File exists']
>>> sourceRoute::WARNING::2015-01-27
>>> 14:33:57,420::utils::129::root::(rmFile) File:
>>> /var/run/vdsm/trackedInterfaces/bond0 already removed
>>> sourceRoute::DEBUG::2015-01-27
>>> 14:37:52,085::sourceroutethread::39::root::(process_IN_CLOSE_WRITE_filePath)
>>> Responding to DHCP response in /var/run/vdsm/sourceRoutes/1422365872
>>> sourceRoute::INFO::2015-01-27
>>> 14:37:52,090::sourceroute::78::root::(configure) Configuring
>>> gateway - ip: **.***.**.**, network: 10.100.100.0/24, subnet:
>>> 255.255.255.0, gateway: **.***.**.**, table: 174351414, device:
>>> bond0
>>> sourceRoute::DEBUG::2015-01-27
>>> 14:37:52,090::utils::739::root::(execCmd) /sbin/ip -4 route add
>>> 0.0.0.0/0 via **.***.**.** dev bond0 table 174351414 (cwd None)
>>> sourceRoute::DEBUG::2015-01-27
>>> 14:37:52,094::utils::759::root::(execCmd) FAILED: <err> =
>>> 'RTNETLINK answers: File exists\n'; <rc> = 2
>>> sourceRoute::ERROR::2015-01-27
>>> 14:37:52,094::sourceroute::88::root::(configure) ip binary
>>> failed during source route configuration: ['RTNETLINK answers:
>>> File exists']
>>> sourceRoute::WARNING::2015-01-27
>>> 14:37:52,094::utils::129::root::(rmFile) File:
>>> /var/run/vdsm/trackedInterfaces/bond0 already removed
>>>
>>>
>>> If you need more info, let me know.
>>>
>>>
>>> On 01/27/2015 02:20 PM, Ondřej Svoboda wrote:
>>>> Jorick,
>>>>
>>>> just what comes to my mind: do you have NetworkManager running, by
>>>> any chance? VDSM and NetworkManager don't mix well currently so
>>>> please try
>>>>
>>>> # systemctl stop NetworkManager
>>>> # systemctl disable NetworkManager
>>>>
>>>> Regards,
>>>> Ondra Svoboda
>>>>
>>>> On 27.1.2015 14:10, Jorick Astrego wrote:
>>>>>
>>>>> On 01/27/2015 01:56 PM, Koen Vanoppen wrote:
>>>>>> Hello everybody,
>>>>>>
>>>>>> Just wanted to know if anybody else has the problem, that when
>>>>>> you restart your network configuration of a hypervisor, all the
>>>>>> settings concerning the bonding are gone...
>>>>>> Do I file a bug report for this or is this already a known issue?
>>>>>>
>>>>>> Kind regards,
>>>>>>
>>>>>> Koen
>>>>>>
>>>>> Hmm,
>>>>>
>>>>> I cannot restart the network service from the host for some reason:
>>>>>
>>>>> Jan 27 14:00:27 test2 systemd: network.service: control
>>>>> process exited, code=exited status=1
>>>>> Jan 27 14:00:27 test2 systemd: Failed to start LSB: Bring
>>>>> up/down networking.
>>>>>
>>>>>
>>>>> systemctl status network.service
>>>>> network.service - LSB: Bring up/down networking
>>>>> Loaded: loaded (/etc/rc.d/init.d/network)
>>>>> Active: failed (Result: exit-code) since Tue 2015-01-27
>>>>> 14:00:27 CET; 1min 56s ago
>>>>> Process: 4113 ExecStart=/etc/rc.d/init.d/network start
>>>>> (code=exited, status=1/FAILURE)
>>>>>
>>>>> Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
>>>>> answers: File exists
>>>>> Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
>>>>> answers: File exists
>>>>> Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
>>>>> answers: File exists
>>>>> Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
>>>>> answers: File exists
>>>>> Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
>>>>> answers: File exists
>>>>> Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
>>>>> answers: File exists
>>>>> Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
>>>>> answers: File exists
>>>>> Jan 27 14:00:27 test2.netbulae.test systemd[1]:
>>>>> network.service: control process exited, code=exited status=1
>>>>> Jan 27 14:00:27 test2.netbulae.test systemd[1]: Failed to
>>>>> start LSB: Bring up/down networking.
>>>>> Jan 27 14:00:27 test2.netbulae.test systemd[1]: Unit
>>>>> network.service entered failed state.
>>>>>
>>>>> -- Subject: Unit network.service has failed
>>>>> -- Defined-By: systemd
>>>>> -- Support:
>>>>> http://lists.freedesktop.org/mailman/listinfo/systemd-devel
>>>>> --
>>>>> -- Unit network.service has failed.
>>>>> --
>>>>> -- The result is failed.
>>>>> Jan 27 14:00:27 test2.netbulae.test systemd[1]: Unit
>>>>> network.service entered failed state.
>>>>>
>>>>>
>>>>> In supervdsm I find lots and lots of these:
>>>>>
>>>>>
>>>>> sourceRoute::DEBUG::2015-01-27
>>>>> 14:02:40,802::utils::739::root::(execCmd) /sbin/ip -4 route
>>>>> add 0.0.0.0/0 via **.***.**.** dev bond0 table 174351414 (cwd
>>>>> None)
>>>>> sourceRoute::DEBUG::2015-01-27
>>>>> 14:02:40,806::utils::759::root::(execCmd) FAILED: <err> =
>>>>> 'RTNETLINK answers: File exists\n'; <rc> = 2
>>>>> sourceRoute::ERROR::2015-01-27
>>>>> 14:02:40,806::sourceroute::88::root::(configure) ip binary
>>>>> failed during source route configuration: ['RTNETLINK answers:
>>>>> File exists']
>>>>> sourceRoute::WARNING::2015-01-27
>>>>> 14:02:40,807::utils::129::root::(rmFile) File:
>>>>> /var/run/vdsm/trackedInterfaces/bond0 already removed
>>>>> sourceRoute::DEBUG::2015-01-27
>>>>> 14:07:09,047::sourceroutethread::39::root::(process_IN_CLOSE_WRITE_filePath)
>>>>> Responding to DHCP response in
>>>>> /var/run/vdsm/sourceRoutes/1422364029
>>>>> sourceRoute::INFO::2015-01-27
>>>>> 14:07:09,052::sourceroute::78::root::(configure) Configuring
>>>>> gateway - ip: **.***.**.***, network: **.***.**.**0/24,
>>>>> subnet: 255.255.255.0, gateway: **.***.**.**, table:
>>>>> 174351414, device: bond0
>>>>> sourceRoute::DEBUG::2015-01-27
>>>>> 14:07:09,052::utils::739::root::(execCmd) /sbin/ip -4 route
>>>>> add 0.0.0.0/0 via **.***.**.** dev bond0 table 174351414 (cwd
>>>>> None)
>>>>> sourceRoute::DEBUG::2015-01-27
>>>>> 14:07:09,057::utils::759::root::(execCmd) FAILED: <err> =
>>>>> 'RTNETLINK answers: File exists\n'; <rc> = 2
>>>>> sourceRoute::ERROR::2015-01-27
>>>>> 14:07:09,057::sourceroute::88::root::(configure) ip binary
>>>>> failed during source route configuration: ['RTNETLINK answers:
>>>>> File exists']
>>>>> sourceRoute::WARNING::2015-01-27
>>>>> 14:07:09,057::utils::129::root::(rmFile) File:
>>>>> /var/run/vdsm/trackedInterfaces/bond0 already removed
>>>>>
>>>>>
>>>>> The hosts are all Centos 7 freshly provisioned. With these vdsm
>>>>> packages:
>>>>>
>>>>> vdsm-python-zombiereaper-4.16.10-8.gitc937927.el7.noarch
>>>>> vdsm-python-4.16.10-8.gitc937927.el7.noarch
>>>>> vdsm-yajsonrpc-4.16.10-8.gitc937927.el7.noarch
>>>>> vdsm-4.16.10-8.gitc937927.el7.x86_64
>>>>> vdsm-xmlrpc-4.16.10-8.gitc937927.el7.noarch
>>>>> vdsm-jsonrpc-4.16.10-8.gitc937927.el7.noarch
>>>>> vdsm-cli-4.16.10-8.gitc937927.el7.noarch
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Met vriendelijke groet, With kind regards,
>>>>>
>>>>> Jorick Astrego*
>>>>>
>>>>> Netbulae Virtualization Experts *
>>>>> ------------------------------------------------------------------------
>>>>> Tel: 053 20 30 270 info at netbulae.eu Staalsteden 4-3A KvK 08198180
>>>>> Fax: 053 20 30 271 www.netbulae.eu 7547 TA Enschede BTW
>>>>> NL821234584B01
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Users mailing list
>>>>> Users at ovirt.org
>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>
>>>
>>>
>>>
>>> Met vriendelijke groet, With kind regards,
>>>
>>> Jorick Astrego*
>>>
>>> Netbulae Virtualization Experts *
>>> ------------------------------------------------------------------------
>>> Tel: 053 20 30 270 info at netbulae.eu Staalsteden 4-3A KvK 08198180
>>> Fax: 053 20 30 271 www.netbulae.eu 7547 TA Enschede BTW
>>> NL821234584B01
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/users
>>
>>
>>
>> _______________________________________________
>> Users mailing list
>> Users at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>
>
>
>
> Met vriendelijke groet, With kind regards,
>
> Jorick Astrego*
>
> Netbulae Virtualization Experts *
> ------------------------------------------------------------------------
> Tel: 053 20 30 270 info at netbulae.eu Staalsteden 4-3A KvK 08198180
> Fax: 053 20 30 271 www.netbulae.eu 7547 TA Enschede BTW NL821234584B01
>
>
> ------------------------------------------------------------------------
>
>
>
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20150127/6a06a756/attachment-0001.html>
More information about the Users
mailing list