[ovirt-users] Network config

Ondřej Svoboda ondrej at svobodasoft.cz
Tue Jan 27 18:15:41 UTC 2015


On 27.1.2015 19:10, Ondřej Svoboda wrote:
> From the first log (journal): VDSM managed to start, only it is being 
> run too early (it should be waiting patiently for its supervdsm 
> component to start responding).
>
> Are there no "RTNETLINK answers: File exists" errors anymore?
>
> Second log: This is normal operation.
>
> So the host stopped responding? It should have come up, assuming that 
> it obtained the same address from DHCP (which it should, but please 
> check).
>
> Could you try using static network configuration first? And when it 
> succeeds you can try DHCP again.
>
> ===
>
> About the GUI: Is your host's status now "Not Operational"? In that 
> case oVirt cannot tell anything about the status of the host's 
> interfaces, it may only reflect the status before communication was lost.
>
> Please file a bug if this is really the case I described :-) It would 
> then be a matter of discussion if oVirt should display a yellow 
> "unknown" status of the interfaces, or keep displaying the last known 
> one. (The first solution would make it obvious that we really lost the 
> host and go looking for the host's status to discover it is "Not 
> Operational". The other is informative as well, just in a different way.)
>
I should add that the "yellow, unknown" state is just a suggestion, it 
need not exist yet.

> On 27.1.2015 16:31, Jorick Astrego wrote:
>>
>> Done, maybe this helps:
>>
>>     Jan 27 16:20:26 test2 vdsmd_init_common.sh: vdsm: Running
>>     unified_network_persistence_upgrade
>>     Jan 27 16:20:26 test2 vdsmd_init_common.sh: vdsm: Running
>>     restore_nets
>>     Jan 27 16:21:47 test2 systemd-logind: New session 5 of user root.
>>     Jan 27 16:21:47 test2 systemd: Starting Session 5 of user root.
>>     Jan 27 16:21:47 test2 systemd: Started Session 5 of user root.
>>     Jan 27 16:21:55 test2 systemd: vdsmd.service operation timed out.
>>     Terminating.
>>     Jan 27 16:21:55 test2 journal: End of file while reading data:
>>     Input/output error
>>     Jan 27 16:21:55 test2 systemd: Failed to start Virtual Desktop
>>     Server Manager.
>>     Jan 27 16:21:55 test2 systemd: Unit vdsmd.service entered failed
>>     state.
>>     Jan 27 16:21:55 test2 systemd: vdsmd.service holdoff time over,
>>     scheduling restart.
>>     Jan 27 16:21:55 test2 systemd: Stopping Virtual Desktop Server
>>     Manager...
>>     Jan 27 16:21:55 test2 systemd: Starting Virtual Desktop Server
>>     Manager...
>>     Jan 27 16:21:55 test2 vdsmd_init_common.sh: vdsm: Running mkdirs
>>
>> Still:
>>
>>     MainThread::INFO::2015-01-27
>>     16:21:56,964::netconfpersistence::158::root::(_clearDisk)
>>     Clearing /var/run/vdsm/netconf/nets/ and /var/run/vdsm/netconf/bonds/
>>     MainThread::DEBUG::2015-01-27
>>     16:21:56,964::netconfpersistence::166::root::(_clearDisk) No
>>     existent config to clear.
>>     MainThread::INFO::2015-01-27
>>     16:21:56,964::netconfpersistence::182::root::(save) Saved new
>>     config RunningConfig({}, {}) to /var/run/vdsm/netconf/nets/ and
>>     /var/run/vdsm/netconf/bonds/
>>     MainThread::DEBUG::2015-01-27
>>     16:21:56,965::netconfpersistence::134::root::(_getConfigs)
>>     Non-existing config set.
>>
>> And when I want to reapply setup networks in ovirt I get:
>>
>>
>>     Error while executing action Commit Network changes: Network
>>     error during communication with the Host.
>>
>> Every interface and bond is still green and visible in the webgui by 
>> the way. That is also strange, every interface of the host is bonded 
>> and up while in reality the host has lost it's network config and 
>> doesn't respond anymore.
>>
>> Shouldn't they all be red on communication failure to the host?
>>
>> Cheers,
>>
>> Jorick
>>
>> On 01/27/2015 04:01 PM, Ondřej Svoboda wrote:
>>> This is weird, like there were multiple dhclients on the bond. Can 
>>> you stop VDSM (systemctl stop vdsmd supervdsmd) and maybe `killall 
>>> -SIGKILL dhclient` and restart VDSM afterwards?
>>>
>>> On 27.1.2015 14:43, Jorick Astrego wrote:
>>>> Ah yes, I forgot to disable it as I switched to foreman provisioning.
>>>>
>>>> Maybe it can be auto detected and disabled in the "install host" 
>>>> scripts when ovirt installs vdsm and sets up the networking. For 
>>>> people who forget to disable it.....
>>>>
>>>> But now I get this:
>>>>
>>>>     sourceRoute::DEBUG::2015-01-27
>>>>     14:33:57,416::utils::739::root::(execCmd) /sbin/ip -4 route add
>>>>     0.0.0.0/0 via ***.***.***.*** dev bond0 table 174351414 (cwd None)
>>>>     sourceRoute::DEBUG::2015-01-27
>>>>     14:33:57,420::utils::759::root::(execCmd) FAILED: <err> =
>>>>     'RTNETLINK answers: File exists\n'; <rc> = 2
>>>>     sourceRoute::ERROR::2015-01-27
>>>>     14:33:57,420::sourceroute::88::root::(configure) ip binary
>>>>     failed during source route configuration: ['RTNETLINK answers:
>>>>     File exists']
>>>>     sourceRoute::WARNING::2015-01-27
>>>>     14:33:57,420::utils::129::root::(rmFile) File:
>>>>     /var/run/vdsm/trackedInterfaces/bond0 already removed
>>>>
>>>>     Jan 27 14:38:19 test3.netbulae.test network[3042]: RTNETLINK
>>>>     answers: File exists
>>>>     Jan 27 14:38:19 test3.netbulae.test systemd[1]:
>>>>     network.service: control process exited, code=exited status=1
>>>>     Jan 27 14:38:19 test3.netbulae.test systemd[1]: Failed to start
>>>>     LSB: Bring up/down networking.
>>>>     -- Subject: Unit network.service has failed
>>>>     -- Defined-By: systemd
>>>>     -- Support:
>>>>     http://lists.freedesktop.org/mailman/listinfo/systemd-devel
>>>>     -- 
>>>>     -- Unit network.service has failed.
>>>>     -- 
>>>>     -- The result is failed.
>>>>     Jan 27 14:38:19 test3.netbulae.test systemd[1]: Unit
>>>>     network.service entered failed state.
>>>>
>>>>
>>>>
>>>> Bond0 is still there though
>>>>
>>>>     sourceRoute::DEBUG::2015-01-27
>>>>     14:33:57,410::sourceroutethread::39::root::(process_IN_CLOSE_WRITE_filePath)
>>>>     Responding to DHCP response in
>>>>     /var/run/vdsm/sourceRoutes/1422365637
>>>>     sourceRoute::INFO::2015-01-27
>>>>     14:33:57,416::sourceroute::78::root::(configure) Configuring
>>>>     gateway - ip: **.***.**.**, network: 10.100.100.0/24, subnet:
>>>>     255.255.255.0, gateway: **.***.**.**, table: 174351414, device:
>>>>     bond0
>>>>     sourceRoute::DEBUG::2015-01-27
>>>>     14:33:57,416::utils::739::root::(execCmd) /sbin/ip -4 route add
>>>>     0.0.0.0/0 via **.***.**.** dev bond0 table 174351414 (cwd None)
>>>>     sourceRoute::DEBUG::2015-01-27
>>>>     14:33:57,420::utils::759::root::(execCmd) FAILED: <err> =
>>>>     'RTNETLINK answers: File exists\n'; <rc> = 2
>>>>     sourceRoute::ERROR::2015-01-27
>>>>     14:33:57,420::sourceroute::88::root::(configure) ip binary
>>>>     failed during source route configuration: ['RTNETLINK answers:
>>>>     File exists']
>>>>     sourceRoute::WARNING::2015-01-27
>>>>     14:33:57,420::utils::129::root::(rmFile) File:
>>>>     /var/run/vdsm/trackedInterfaces/bond0 already removed
>>>>     sourceRoute::DEBUG::2015-01-27
>>>>     14:37:52,085::sourceroutethread::39::root::(process_IN_CLOSE_WRITE_filePath)
>>>>     Responding to DHCP response in
>>>>     /var/run/vdsm/sourceRoutes/1422365872
>>>>     sourceRoute::INFO::2015-01-27
>>>>     14:37:52,090::sourceroute::78::root::(configure) Configuring
>>>>     gateway - ip: **.***.**.**, network: 10.100.100.0/24, subnet:
>>>>     255.255.255.0, gateway: **.***.**.**, table: 174351414, device:
>>>>     bond0
>>>>     sourceRoute::DEBUG::2015-01-27
>>>>     14:37:52,090::utils::739::root::(execCmd) /sbin/ip -4 route add
>>>>     0.0.0.0/0 via **.***.**.** dev bond0 table 174351414 (cwd None)
>>>>     sourceRoute::DEBUG::2015-01-27
>>>>     14:37:52,094::utils::759::root::(execCmd) FAILED: <err> =
>>>>     'RTNETLINK answers: File exists\n'; <rc> = 2
>>>>     sourceRoute::ERROR::2015-01-27
>>>>     14:37:52,094::sourceroute::88::root::(configure) ip binary
>>>>     failed during source route configuration: ['RTNETLINK answers:
>>>>     File exists']
>>>>     sourceRoute::WARNING::2015-01-27
>>>>     14:37:52,094::utils::129::root::(rmFile) File:
>>>>     /var/run/vdsm/trackedInterfaces/bond0 already removed
>>>>
>>>>
>>>> If you need more info, let me know.
>>>>
>>>>
>>>> On 01/27/2015 02:20 PM, Ondřej Svoboda wrote:
>>>>> Jorick,
>>>>>
>>>>> just what comes to my mind: do you have NetworkManager running, by 
>>>>> any chance? VDSM and NetworkManager don't mix well currently so 
>>>>> please try
>>>>>
>>>>> # systemctl stop NetworkManager
>>>>> # systemctl disable NetworkManager
>>>>>
>>>>> Regards,
>>>>> Ondra Svoboda
>>>>>
>>>>> On 27.1.2015 14:10, Jorick Astrego wrote:
>>>>>>
>>>>>> On 01/27/2015 01:56 PM, Koen Vanoppen wrote:
>>>>>>> Hello everybody,
>>>>>>>
>>>>>>> Just wanted to know if anybody else has the problem, that when 
>>>>>>> you restart your network configuration of a hypervisor, all the 
>>>>>>> settings concerning the bonding are gone...
>>>>>>> Do I file a bug report for this or is this already a known issue?
>>>>>>>
>>>>>>> Kind regards,
>>>>>>>
>>>>>>> Koen
>>>>>>>
>>>>>> Hmm,
>>>>>>
>>>>>> I cannot restart the network service from the host for some reason:
>>>>>>
>>>>>>     Jan 27 14:00:27 test2 systemd: network.service: control
>>>>>>     process exited, code=exited status=1
>>>>>>     Jan 27 14:00:27 test2 systemd: Failed to start LSB: Bring
>>>>>>     up/down networking.
>>>>>>
>>>>>>
>>>>>>     systemctl status network.service
>>>>>>     network.service - LSB: Bring up/down networking
>>>>>>        Loaded: loaded (/etc/rc.d/init.d/network)
>>>>>>        Active: failed (Result: exit-code) since Tue 2015-01-27
>>>>>>     14:00:27 CET; 1min 56s ago
>>>>>>       Process: 4113 ExecStart=/etc/rc.d/init.d/network start
>>>>>>     (code=exited, status=1/FAILURE)
>>>>>>
>>>>>>     Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
>>>>>>     answers: File exists
>>>>>>     Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
>>>>>>     answers: File exists
>>>>>>     Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
>>>>>>     answers: File exists
>>>>>>     Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
>>>>>>     answers: File exists
>>>>>>     Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
>>>>>>     answers: File exists
>>>>>>     Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
>>>>>>     answers: File exists
>>>>>>     Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
>>>>>>     answers: File exists
>>>>>>     Jan 27 14:00:27 test2.netbulae.test systemd[1]:
>>>>>>     network.service: control process exited, code=exited status=1
>>>>>>     Jan 27 14:00:27 test2.netbulae.test systemd[1]: Failed to
>>>>>>     start LSB: Bring up/down networking.
>>>>>>     Jan 27 14:00:27 test2.netbulae.test systemd[1]: Unit
>>>>>>     network.service entered failed state.
>>>>>>
>>>>>>     -- Subject: Unit network.service has failed
>>>>>>     -- Defined-By: systemd
>>>>>>     -- Support:
>>>>>>     http://lists.freedesktop.org/mailman/listinfo/systemd-devel
>>>>>>     -- 
>>>>>>     -- Unit network.service has failed.
>>>>>>     -- 
>>>>>>     -- The result is failed.
>>>>>>     Jan 27 14:00:27 test2.netbulae.test systemd[1]: Unit
>>>>>>     network.service entered failed state.
>>>>>>
>>>>>>
>>>>>>     In supervdsm I find lots and lots of these:
>>>>>>
>>>>>>
>>>>>>     sourceRoute::DEBUG::2015-01-27
>>>>>>     14:02:40,802::utils::739::root::(execCmd) /sbin/ip -4 route
>>>>>>     add 0.0.0.0/0 via **.***.**.** dev bond0 table 174351414 (cwd
>>>>>>     None)
>>>>>>     sourceRoute::DEBUG::2015-01-27
>>>>>>     14:02:40,806::utils::759::root::(execCmd) FAILED: <err> =
>>>>>>     'RTNETLINK answers: File exists\n'; <rc> = 2
>>>>>>     sourceRoute::ERROR::2015-01-27
>>>>>>     14:02:40,806::sourceroute::88::root::(configure) ip binary
>>>>>>     failed during source route configuration: ['RTNETLINK
>>>>>>     answers: File exists']
>>>>>>     sourceRoute::WARNING::2015-01-27
>>>>>>     14:02:40,807::utils::129::root::(rmFile) File:
>>>>>>     /var/run/vdsm/trackedInterfaces/bond0 already removed
>>>>>>     sourceRoute::DEBUG::2015-01-27
>>>>>>     14:07:09,047::sourceroutethread::39::root::(process_IN_CLOSE_WRITE_filePath)
>>>>>>     Responding to DHCP response in
>>>>>>     /var/run/vdsm/sourceRoutes/1422364029
>>>>>>     sourceRoute::INFO::2015-01-27
>>>>>>     14:07:09,052::sourceroute::78::root::(configure) Configuring
>>>>>>     gateway - ip: **.***.**.***, network: **.***.**.**0/24,
>>>>>>     subnet: 255.255.255.0, gateway: **.***.**.**, table:
>>>>>>     174351414, device: bond0
>>>>>>     sourceRoute::DEBUG::2015-01-27
>>>>>>     14:07:09,052::utils::739::root::(execCmd) /sbin/ip -4 route
>>>>>>     add 0.0.0.0/0 via **.***.**.** dev bond0 table 174351414 (cwd
>>>>>>     None)
>>>>>>     sourceRoute::DEBUG::2015-01-27
>>>>>>     14:07:09,057::utils::759::root::(execCmd) FAILED: <err> =
>>>>>>     'RTNETLINK answers: File exists\n'; <rc> = 2
>>>>>>     sourceRoute::ERROR::2015-01-27
>>>>>>     14:07:09,057::sourceroute::88::root::(configure) ip binary
>>>>>>     failed during source route configuration: ['RTNETLINK
>>>>>>     answers: File exists']
>>>>>>     sourceRoute::WARNING::2015-01-27
>>>>>>     14:07:09,057::utils::129::root::(rmFile) File:
>>>>>>     /var/run/vdsm/trackedInterfaces/bond0 already removed
>>>>>>
>>>>>>
>>>>>> The hosts are all Centos 7 freshly provisioned. With these vdsm 
>>>>>> packages:
>>>>>>
>>>>>>     vdsm-python-zombiereaper-4.16.10-8.gitc937927.el7.noarch
>>>>>>     vdsm-python-4.16.10-8.gitc937927.el7.noarch
>>>>>>     vdsm-yajsonrpc-4.16.10-8.gitc937927.el7.noarch
>>>>>>     vdsm-4.16.10-8.gitc937927.el7.x86_64
>>>>>>     vdsm-xmlrpc-4.16.10-8.gitc937927.el7.noarch
>>>>>>     vdsm-jsonrpc-4.16.10-8.gitc937927.el7.noarch
>>>>>>     vdsm-cli-4.16.10-8.gitc937927.el7.noarch
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Met vriendelijke groet, With kind regards,
>>>>>>
>>>>>> Jorick Astrego*
>>>>>>
>>>>>> Netbulae Virtualization Experts *
>>>>>> ------------------------------------------------------------------------
>>>>>> Tel: 053 20 30 270 	info at netbulae.eu 	Staalsteden 4-3A 	KvK 08198180
>>>>>> Fax: 053 20 30 271 	www.netbulae.eu 	7547 TA Enschede 	BTW 
>>>>>> NL821234584B01
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Users mailing list
>>>>>> Users at ovirt.org
>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Users mailing list
>>>>> Users at ovirt.org
>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>
>>>>
>>>>
>>>>
>>>> Met vriendelijke groet, With kind regards,
>>>>
>>>> Jorick Astrego*
>>>>
>>>> Netbulae Virtualization Experts *
>>>> ------------------------------------------------------------------------
>>>> Tel: 053 20 30 270 	info at netbulae.eu 	Staalsteden 4-3A 	KvK 08198180
>>>> Fax: 053 20 30 271 	www.netbulae.eu 	7547 TA Enschede 	BTW 
>>>> NL821234584B01
>>>>
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>
>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/users
>>
>>
>>
>>
>> Met vriendelijke groet, With kind regards,
>>
>> Jorick Astrego*
>>
>> Netbulae Virtualization Experts *
>> ------------------------------------------------------------------------
>> Tel: 053 20 30 270 	info at netbulae.eu 	Staalsteden 4-3A 	KvK 08198180
>> Fax: 053 20 30 271 	www.netbulae.eu 	7547 TA Enschede 	BTW 
>> NL821234584B01
>>
>>
>> ------------------------------------------------------------------------
>>
>>
>>
>> _______________________________________________
>> Users mailing list
>> Users at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>
>
>
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20150127/e703fc55/attachment-0001.html>


More information about the Users mailing list