[ovirt-users] Network config
Ondřej Svoboda
ondrej at svobodasoft.cz
Tue Jan 27 18:15:41 UTC 2015
On 27.1.2015 19:10, Ondřej Svoboda wrote:
> From the first log (journal): VDSM managed to start, only it is being
> run too early (it should be waiting patiently for its supervdsm
> component to start responding).
>
> Are there no "RTNETLINK answers: File exists" errors anymore?
>
> Second log: This is normal operation.
>
> So the host stopped responding? It should have come up, assuming that
> it obtained the same address from DHCP (which it should, but please
> check).
>
> Could you try using static network configuration first? And when it
> succeeds you can try DHCP again.
>
> ===
>
> About the GUI: Is your host's status now "Not Operational"? In that
> case oVirt cannot tell anything about the status of the host's
> interfaces, it may only reflect the status before communication was lost.
>
> Please file a bug if this is really the case I described :-) It would
> then be a matter of discussion if oVirt should display a yellow
> "unknown" status of the interfaces, or keep displaying the last known
> one. (The first solution would make it obvious that we really lost the
> host and go looking for the host's status to discover it is "Not
> Operational". The other is informative as well, just in a different way.)
>
I should add that the "yellow, unknown" state is just a suggestion, it
need not exist yet.
> On 27.1.2015 16:31, Jorick Astrego wrote:
>>
>> Done, maybe this helps:
>>
>> Jan 27 16:20:26 test2 vdsmd_init_common.sh: vdsm: Running
>> unified_network_persistence_upgrade
>> Jan 27 16:20:26 test2 vdsmd_init_common.sh: vdsm: Running
>> restore_nets
>> Jan 27 16:21:47 test2 systemd-logind: New session 5 of user root.
>> Jan 27 16:21:47 test2 systemd: Starting Session 5 of user root.
>> Jan 27 16:21:47 test2 systemd: Started Session 5 of user root.
>> Jan 27 16:21:55 test2 systemd: vdsmd.service operation timed out.
>> Terminating.
>> Jan 27 16:21:55 test2 journal: End of file while reading data:
>> Input/output error
>> Jan 27 16:21:55 test2 systemd: Failed to start Virtual Desktop
>> Server Manager.
>> Jan 27 16:21:55 test2 systemd: Unit vdsmd.service entered failed
>> state.
>> Jan 27 16:21:55 test2 systemd: vdsmd.service holdoff time over,
>> scheduling restart.
>> Jan 27 16:21:55 test2 systemd: Stopping Virtual Desktop Server
>> Manager...
>> Jan 27 16:21:55 test2 systemd: Starting Virtual Desktop Server
>> Manager...
>> Jan 27 16:21:55 test2 vdsmd_init_common.sh: vdsm: Running mkdirs
>>
>> Still:
>>
>> MainThread::INFO::2015-01-27
>> 16:21:56,964::netconfpersistence::158::root::(_clearDisk)
>> Clearing /var/run/vdsm/netconf/nets/ and /var/run/vdsm/netconf/bonds/
>> MainThread::DEBUG::2015-01-27
>> 16:21:56,964::netconfpersistence::166::root::(_clearDisk) No
>> existent config to clear.
>> MainThread::INFO::2015-01-27
>> 16:21:56,964::netconfpersistence::182::root::(save) Saved new
>> config RunningConfig({}, {}) to /var/run/vdsm/netconf/nets/ and
>> /var/run/vdsm/netconf/bonds/
>> MainThread::DEBUG::2015-01-27
>> 16:21:56,965::netconfpersistence::134::root::(_getConfigs)
>> Non-existing config set.
>>
>> And when I want to reapply setup networks in ovirt I get:
>>
>>
>> Error while executing action Commit Network changes: Network
>> error during communication with the Host.
>>
>> Every interface and bond is still green and visible in the webgui by
>> the way. That is also strange, every interface of the host is bonded
>> and up while in reality the host has lost it's network config and
>> doesn't respond anymore.
>>
>> Shouldn't they all be red on communication failure to the host?
>>
>> Cheers,
>>
>> Jorick
>>
>> On 01/27/2015 04:01 PM, Ondřej Svoboda wrote:
>>> This is weird, like there were multiple dhclients on the bond. Can
>>> you stop VDSM (systemctl stop vdsmd supervdsmd) and maybe `killall
>>> -SIGKILL dhclient` and restart VDSM afterwards?
>>>
>>> On 27.1.2015 14:43, Jorick Astrego wrote:
>>>> Ah yes, I forgot to disable it as I switched to foreman provisioning.
>>>>
>>>> Maybe it can be auto detected and disabled in the "install host"
>>>> scripts when ovirt installs vdsm and sets up the networking. For
>>>> people who forget to disable it.....
>>>>
>>>> But now I get this:
>>>>
>>>> sourceRoute::DEBUG::2015-01-27
>>>> 14:33:57,416::utils::739::root::(execCmd) /sbin/ip -4 route add
>>>> 0.0.0.0/0 via ***.***.***.*** dev bond0 table 174351414 (cwd None)
>>>> sourceRoute::DEBUG::2015-01-27
>>>> 14:33:57,420::utils::759::root::(execCmd) FAILED: <err> =
>>>> 'RTNETLINK answers: File exists\n'; <rc> = 2
>>>> sourceRoute::ERROR::2015-01-27
>>>> 14:33:57,420::sourceroute::88::root::(configure) ip binary
>>>> failed during source route configuration: ['RTNETLINK answers:
>>>> File exists']
>>>> sourceRoute::WARNING::2015-01-27
>>>> 14:33:57,420::utils::129::root::(rmFile) File:
>>>> /var/run/vdsm/trackedInterfaces/bond0 already removed
>>>>
>>>> Jan 27 14:38:19 test3.netbulae.test network[3042]: RTNETLINK
>>>> answers: File exists
>>>> Jan 27 14:38:19 test3.netbulae.test systemd[1]:
>>>> network.service: control process exited, code=exited status=1
>>>> Jan 27 14:38:19 test3.netbulae.test systemd[1]: Failed to start
>>>> LSB: Bring up/down networking.
>>>> -- Subject: Unit network.service has failed
>>>> -- Defined-By: systemd
>>>> -- Support:
>>>> http://lists.freedesktop.org/mailman/listinfo/systemd-devel
>>>> --
>>>> -- Unit network.service has failed.
>>>> --
>>>> -- The result is failed.
>>>> Jan 27 14:38:19 test3.netbulae.test systemd[1]: Unit
>>>> network.service entered failed state.
>>>>
>>>>
>>>>
>>>> Bond0 is still there though
>>>>
>>>> sourceRoute::DEBUG::2015-01-27
>>>> 14:33:57,410::sourceroutethread::39::root::(process_IN_CLOSE_WRITE_filePath)
>>>> Responding to DHCP response in
>>>> /var/run/vdsm/sourceRoutes/1422365637
>>>> sourceRoute::INFO::2015-01-27
>>>> 14:33:57,416::sourceroute::78::root::(configure) Configuring
>>>> gateway - ip: **.***.**.**, network: 10.100.100.0/24, subnet:
>>>> 255.255.255.0, gateway: **.***.**.**, table: 174351414, device:
>>>> bond0
>>>> sourceRoute::DEBUG::2015-01-27
>>>> 14:33:57,416::utils::739::root::(execCmd) /sbin/ip -4 route add
>>>> 0.0.0.0/0 via **.***.**.** dev bond0 table 174351414 (cwd None)
>>>> sourceRoute::DEBUG::2015-01-27
>>>> 14:33:57,420::utils::759::root::(execCmd) FAILED: <err> =
>>>> 'RTNETLINK answers: File exists\n'; <rc> = 2
>>>> sourceRoute::ERROR::2015-01-27
>>>> 14:33:57,420::sourceroute::88::root::(configure) ip binary
>>>> failed during source route configuration: ['RTNETLINK answers:
>>>> File exists']
>>>> sourceRoute::WARNING::2015-01-27
>>>> 14:33:57,420::utils::129::root::(rmFile) File:
>>>> /var/run/vdsm/trackedInterfaces/bond0 already removed
>>>> sourceRoute::DEBUG::2015-01-27
>>>> 14:37:52,085::sourceroutethread::39::root::(process_IN_CLOSE_WRITE_filePath)
>>>> Responding to DHCP response in
>>>> /var/run/vdsm/sourceRoutes/1422365872
>>>> sourceRoute::INFO::2015-01-27
>>>> 14:37:52,090::sourceroute::78::root::(configure) Configuring
>>>> gateway - ip: **.***.**.**, network: 10.100.100.0/24, subnet:
>>>> 255.255.255.0, gateway: **.***.**.**, table: 174351414, device:
>>>> bond0
>>>> sourceRoute::DEBUG::2015-01-27
>>>> 14:37:52,090::utils::739::root::(execCmd) /sbin/ip -4 route add
>>>> 0.0.0.0/0 via **.***.**.** dev bond0 table 174351414 (cwd None)
>>>> sourceRoute::DEBUG::2015-01-27
>>>> 14:37:52,094::utils::759::root::(execCmd) FAILED: <err> =
>>>> 'RTNETLINK answers: File exists\n'; <rc> = 2
>>>> sourceRoute::ERROR::2015-01-27
>>>> 14:37:52,094::sourceroute::88::root::(configure) ip binary
>>>> failed during source route configuration: ['RTNETLINK answers:
>>>> File exists']
>>>> sourceRoute::WARNING::2015-01-27
>>>> 14:37:52,094::utils::129::root::(rmFile) File:
>>>> /var/run/vdsm/trackedInterfaces/bond0 already removed
>>>>
>>>>
>>>> If you need more info, let me know.
>>>>
>>>>
>>>> On 01/27/2015 02:20 PM, Ondřej Svoboda wrote:
>>>>> Jorick,
>>>>>
>>>>> just what comes to my mind: do you have NetworkManager running, by
>>>>> any chance? VDSM and NetworkManager don't mix well currently so
>>>>> please try
>>>>>
>>>>> # systemctl stop NetworkManager
>>>>> # systemctl disable NetworkManager
>>>>>
>>>>> Regards,
>>>>> Ondra Svoboda
>>>>>
>>>>> On 27.1.2015 14:10, Jorick Astrego wrote:
>>>>>>
>>>>>> On 01/27/2015 01:56 PM, Koen Vanoppen wrote:
>>>>>>> Hello everybody,
>>>>>>>
>>>>>>> Just wanted to know if anybody else has the problem, that when
>>>>>>> you restart your network configuration of a hypervisor, all the
>>>>>>> settings concerning the bonding are gone...
>>>>>>> Do I file a bug report for this or is this already a known issue?
>>>>>>>
>>>>>>> Kind regards,
>>>>>>>
>>>>>>> Koen
>>>>>>>
>>>>>> Hmm,
>>>>>>
>>>>>> I cannot restart the network service from the host for some reason:
>>>>>>
>>>>>> Jan 27 14:00:27 test2 systemd: network.service: control
>>>>>> process exited, code=exited status=1
>>>>>> Jan 27 14:00:27 test2 systemd: Failed to start LSB: Bring
>>>>>> up/down networking.
>>>>>>
>>>>>>
>>>>>> systemctl status network.service
>>>>>> network.service - LSB: Bring up/down networking
>>>>>> Loaded: loaded (/etc/rc.d/init.d/network)
>>>>>> Active: failed (Result: exit-code) since Tue 2015-01-27
>>>>>> 14:00:27 CET; 1min 56s ago
>>>>>> Process: 4113 ExecStart=/etc/rc.d/init.d/network start
>>>>>> (code=exited, status=1/FAILURE)
>>>>>>
>>>>>> Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
>>>>>> answers: File exists
>>>>>> Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
>>>>>> answers: File exists
>>>>>> Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
>>>>>> answers: File exists
>>>>>> Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
>>>>>> answers: File exists
>>>>>> Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
>>>>>> answers: File exists
>>>>>> Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
>>>>>> answers: File exists
>>>>>> Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
>>>>>> answers: File exists
>>>>>> Jan 27 14:00:27 test2.netbulae.test systemd[1]:
>>>>>> network.service: control process exited, code=exited status=1
>>>>>> Jan 27 14:00:27 test2.netbulae.test systemd[1]: Failed to
>>>>>> start LSB: Bring up/down networking.
>>>>>> Jan 27 14:00:27 test2.netbulae.test systemd[1]: Unit
>>>>>> network.service entered failed state.
>>>>>>
>>>>>> -- Subject: Unit network.service has failed
>>>>>> -- Defined-By: systemd
>>>>>> -- Support:
>>>>>> http://lists.freedesktop.org/mailman/listinfo/systemd-devel
>>>>>> --
>>>>>> -- Unit network.service has failed.
>>>>>> --
>>>>>> -- The result is failed.
>>>>>> Jan 27 14:00:27 test2.netbulae.test systemd[1]: Unit
>>>>>> network.service entered failed state.
>>>>>>
>>>>>>
>>>>>> In supervdsm I find lots and lots of these:
>>>>>>
>>>>>>
>>>>>> sourceRoute::DEBUG::2015-01-27
>>>>>> 14:02:40,802::utils::739::root::(execCmd) /sbin/ip -4 route
>>>>>> add 0.0.0.0/0 via **.***.**.** dev bond0 table 174351414 (cwd
>>>>>> None)
>>>>>> sourceRoute::DEBUG::2015-01-27
>>>>>> 14:02:40,806::utils::759::root::(execCmd) FAILED: <err> =
>>>>>> 'RTNETLINK answers: File exists\n'; <rc> = 2
>>>>>> sourceRoute::ERROR::2015-01-27
>>>>>> 14:02:40,806::sourceroute::88::root::(configure) ip binary
>>>>>> failed during source route configuration: ['RTNETLINK
>>>>>> answers: File exists']
>>>>>> sourceRoute::WARNING::2015-01-27
>>>>>> 14:02:40,807::utils::129::root::(rmFile) File:
>>>>>> /var/run/vdsm/trackedInterfaces/bond0 already removed
>>>>>> sourceRoute::DEBUG::2015-01-27
>>>>>> 14:07:09,047::sourceroutethread::39::root::(process_IN_CLOSE_WRITE_filePath)
>>>>>> Responding to DHCP response in
>>>>>> /var/run/vdsm/sourceRoutes/1422364029
>>>>>> sourceRoute::INFO::2015-01-27
>>>>>> 14:07:09,052::sourceroute::78::root::(configure) Configuring
>>>>>> gateway - ip: **.***.**.***, network: **.***.**.**0/24,
>>>>>> subnet: 255.255.255.0, gateway: **.***.**.**, table:
>>>>>> 174351414, device: bond0
>>>>>> sourceRoute::DEBUG::2015-01-27
>>>>>> 14:07:09,052::utils::739::root::(execCmd) /sbin/ip -4 route
>>>>>> add 0.0.0.0/0 via **.***.**.** dev bond0 table 174351414 (cwd
>>>>>> None)
>>>>>> sourceRoute::DEBUG::2015-01-27
>>>>>> 14:07:09,057::utils::759::root::(execCmd) FAILED: <err> =
>>>>>> 'RTNETLINK answers: File exists\n'; <rc> = 2
>>>>>> sourceRoute::ERROR::2015-01-27
>>>>>> 14:07:09,057::sourceroute::88::root::(configure) ip binary
>>>>>> failed during source route configuration: ['RTNETLINK
>>>>>> answers: File exists']
>>>>>> sourceRoute::WARNING::2015-01-27
>>>>>> 14:07:09,057::utils::129::root::(rmFile) File:
>>>>>> /var/run/vdsm/trackedInterfaces/bond0 already removed
>>>>>>
>>>>>>
>>>>>> The hosts are all Centos 7 freshly provisioned. With these vdsm
>>>>>> packages:
>>>>>>
>>>>>> vdsm-python-zombiereaper-4.16.10-8.gitc937927.el7.noarch
>>>>>> vdsm-python-4.16.10-8.gitc937927.el7.noarch
>>>>>> vdsm-yajsonrpc-4.16.10-8.gitc937927.el7.noarch
>>>>>> vdsm-4.16.10-8.gitc937927.el7.x86_64
>>>>>> vdsm-xmlrpc-4.16.10-8.gitc937927.el7.noarch
>>>>>> vdsm-jsonrpc-4.16.10-8.gitc937927.el7.noarch
>>>>>> vdsm-cli-4.16.10-8.gitc937927.el7.noarch
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Met vriendelijke groet, With kind regards,
>>>>>>
>>>>>> Jorick Astrego*
>>>>>>
>>>>>> Netbulae Virtualization Experts *
>>>>>> ------------------------------------------------------------------------
>>>>>> Tel: 053 20 30 270 info at netbulae.eu Staalsteden 4-3A KvK 08198180
>>>>>> Fax: 053 20 30 271 www.netbulae.eu 7547 TA Enschede BTW
>>>>>> NL821234584B01
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Users mailing list
>>>>>> Users at ovirt.org
>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Users mailing list
>>>>> Users at ovirt.org
>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>
>>>>
>>>>
>>>>
>>>> Met vriendelijke groet, With kind regards,
>>>>
>>>> Jorick Astrego*
>>>>
>>>> Netbulae Virtualization Experts *
>>>> ------------------------------------------------------------------------
>>>> Tel: 053 20 30 270 info at netbulae.eu Staalsteden 4-3A KvK 08198180
>>>> Fax: 053 20 30 271 www.netbulae.eu 7547 TA Enschede BTW
>>>> NL821234584B01
>>>>
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>
>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/users
>>
>>
>>
>>
>> Met vriendelijke groet, With kind regards,
>>
>> Jorick Astrego*
>>
>> Netbulae Virtualization Experts *
>> ------------------------------------------------------------------------
>> Tel: 053 20 30 270 info at netbulae.eu Staalsteden 4-3A KvK 08198180
>> Fax: 053 20 30 271 www.netbulae.eu 7547 TA Enschede BTW
>> NL821234584B01
>>
>>
>> ------------------------------------------------------------------------
>>
>>
>>
>> _______________________________________________
>> Users mailing list
>> Users at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>
>
>
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20150127/e703fc55/attachment-0001.html>
More information about the Users
mailing list