<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
From the first log (journal): VDSM managed to start, only it is
being run too early (it should be waiting patiently for its
supervdsm component to start responding).<br>
<br>
Are there no "RTNETLINK answers: File exists" errors anymore?<br>
<br>
Second log: This is normal operation.<br>
<br>
So the host stopped responding? It should have come up, assuming
that it obtained the same address from DHCP (which it should, but
please check).<br>
<br>
Could you try using static network configuration first? And when it
succeeds you can try DHCP again.<br>
<br>
===<br>
<br>
About the GUI: Is your host's status now "Not Operational"? In that
case oVirt cannot tell anything about the status of the host's
interfaces, it may only reflect the status before communication was
lost.<br>
<br>
Please file a bug if this is really the case I described :-) It
would then be a matter of discussion if oVirt should display a
yellow "unknown" status of the interfaces, or keep displaying the
last known one. (The first solution would make it obvious that we
really lost the host and go looking for the host's status to
discover it is "Not Operational". The other is informative as well,
just in a different way.)<br>
<br>
<div class="moz-cite-prefix">On 27.1.2015 16:31, Jorick Astrego
wrote:<br>
</div>
<blockquote cite="mid:54C7AF59.5090305@netbulae.eu" type="cite">
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
<br>
Done, maybe this helps:<br>
<blockquote>Jan 27 16:20:26 test2 vdsmd_init_common.sh: vdsm:
Running unified_network_persistence_upgrade<br>
Jan 27 16:20:26 test2 vdsmd_init_common.sh: vdsm: Running
restore_nets<br>
Jan 27 16:21:47 test2 systemd-logind: New session 5 of user
root.<br>
Jan 27 16:21:47 test2 systemd: Starting Session 5 of user root.<br>
Jan 27 16:21:47 test2 systemd: Started Session 5 of user root.<br>
Jan 27 16:21:55 test2 systemd: vdsmd.service operation timed
out. Terminating.<br>
Jan 27 16:21:55 test2 journal: End of file while reading data:
Input/output error<br>
Jan 27 16:21:55 test2 systemd: Failed to start Virtual Desktop
Server Manager.<br>
Jan 27 16:21:55 test2 systemd: Unit vdsmd.service entered failed
state.<br>
Jan 27 16:21:55 test2 systemd: vdsmd.service holdoff time over,
scheduling restart.<br>
Jan 27 16:21:55 test2 systemd: Stopping Virtual Desktop Server
Manager...<br>
Jan 27 16:21:55 test2 systemd: Starting Virtual Desktop Server
Manager...<br>
Jan 27 16:21:55 test2 vdsmd_init_common.sh: vdsm: Running mkdirs<br>
</blockquote>
Still:<br>
<br>
<blockquote>MainThread::<a moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-01-27">INFO::2015-01-27</a>
16:21:56,964::netconfpersistence::158::root::(_clearDisk)
Clearing /var/run/vdsm/netconf/nets/ and
/var/run/vdsm/netconf/bonds/<br>
MainThread::DEBUG::2015-01-27
16:21:56,964::netconfpersistence::166::root::(_clearDisk) No
existent config to clear.<br>
MainThread::<a moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-01-27">INFO::2015-01-27</a>
16:21:56,964::netconfpersistence::182::root::(save) Saved new
config RunningConfig({}, {}) to /var/run/vdsm/netconf/nets/ and
/var/run/vdsm/netconf/bonds/<br>
MainThread::DEBUG::2015-01-27
16:21:56,965::netconfpersistence::134::root::(_getConfigs)
Non-existing config set.<br>
<br>
</blockquote>
And when I want to reapply setup networks in ovirt I get:<br>
<br>
<br>
<blockquote>Error while executing action Commit Network changes:
Network error during communication with the Host.<br>
<br>
</blockquote>
Every interface and bond is still green and visible in the webgui
by the way. That is also strange, every interface of the host is
bonded and up while in reality the host has lost it's network
config and doesn't respond anymore.<br>
<br>
Shouldn't they all be red on communication failure to the host?<br>
<br>
Cheers,<br>
<br>
Jorick<br>
<br>
<div class="moz-cite-prefix">On 01/27/2015 04:01 PM, Ondřej
Svoboda wrote:<br>
</div>
<blockquote cite="mid:54C7A830.2040109@svobodasoft.cz" type="cite">
<meta content="text/html; charset=UTF-8"
http-equiv="Content-Type">
This is weird, like there were multiple dhclients on the bond.
Can you stop VDSM (systemctl stop vdsmd supervdsmd) and maybe
`killall -SIGKILL dhclient` and restart VDSM afterwards?<br>
<br>
<div class="moz-cite-prefix">On 27.1.2015 14:43, Jorick Astrego
wrote:<br>
</div>
<blockquote cite="mid:54C7960B.4080103@netbulae.eu" type="cite">
<meta content="text/html; charset=UTF-8"
http-equiv="Content-Type">
Ah yes, I forgot to disable it as I switched to foreman
provisioning.<br>
<br>
Maybe it can be auto detected and disabled in the "install
host" scripts when ovirt installs vdsm and sets up the
networking. For people who forget to disable it.....<br>
<br>
But now I get this:<br>
<br>
<blockquote>sourceRoute::DEBUG::2015-01-27
14:33:57,416::utils::739::root::(execCmd) /sbin/ip -4 route
add 0.0.0.0/0 via ***.***.***.*** dev bond0 table 174351414
(cwd None)<br>
sourceRoute::DEBUG::2015-01-27
14:33:57,420::utils::759::root::(execCmd) FAILED:
<err> = 'RTNETLINK answers: File exists\n'; <rc>
= 2<br>
sourceRoute::ERROR::2015-01-27
14:33:57,420::sourceroute::88::root::(configure) ip binary
failed during source route configuration: ['RTNETLINK
answers: File exists']<br>
sourceRoute::WARNING::2015-01-27
14:33:57,420::utils::129::root::(rmFile) File:
/var/run/vdsm/trackedInterfaces/bond0 already removed<br>
<br>
Jan 27 14:38:19 test3.netbulae.test network[3042]: RTNETLINK
answers: File exists<br>
Jan 27 14:38:19 test3.netbulae.test systemd[1]:
network.service: control process exited, code=exited
status=1<br>
Jan 27 14:38:19 test3.netbulae.test systemd[1]: Failed to
start LSB: Bring up/down networking.<br>
-- Subject: Unit network.service has failed<br>
-- Defined-By: systemd<br>
-- Support: <a moz-do-not-send="true"
class="moz-txt-link-freetext"
href="http://lists.freedesktop.org/mailman/listinfo/systemd-devel">http://lists.freedesktop.org/mailman/listinfo/systemd-devel</a><br>
-- <br>
-- Unit network.service has failed.<br>
-- <br>
-- The result is failed.<br>
Jan 27 14:38:19 test3.netbulae.test systemd[1]: Unit
network.service entered failed state.<br>
<br>
<br>
<br>
</blockquote>
Bond0 is still there though<br>
<br>
<blockquote>sourceRoute::DEBUG::2015-01-27
14:33:57,410::sourceroutethread::39::root::(process_IN_CLOSE_WRITE_filePath)
Responding to DHCP response in
/var/run/vdsm/sourceRoutes/1422365637<br>
sourceRoute::<a moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-01-27">INFO::2015-01-27</a>
14:33:57,416::sourceroute::78::root::(configure) Configuring
gateway - ip: **.***.**.**, network: 10.100.100.0/24,
subnet: 255.255.255.0, gateway: **.***.**.**, table:
174351414, device: bond0<br>
sourceRoute::DEBUG::2015-01-27
14:33:57,416::utils::739::root::(execCmd) /sbin/ip -4 route
add 0.0.0.0/0 via **.***.**.** dev bond0 table 174351414
(cwd None)<br>
sourceRoute::DEBUG::2015-01-27
14:33:57,420::utils::759::root::(execCmd) FAILED:
<err> = 'RTNETLINK answers: File exists\n'; <rc>
= 2<br>
sourceRoute::ERROR::2015-01-27
14:33:57,420::sourceroute::88::root::(configure) ip binary
failed during source route configuration: ['RTNETLINK
answers: File exists']<br>
sourceRoute::WARNING::2015-01-27
14:33:57,420::utils::129::root::(rmFile) File:
/var/run/vdsm/trackedInterfaces/bond0 already removed<br>
sourceRoute::DEBUG::2015-01-27
14:37:52,085::sourceroutethread::39::root::(process_IN_CLOSE_WRITE_filePath)
Responding to DHCP response in
/var/run/vdsm/sourceRoutes/1422365872<br>
sourceRoute::<a moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-01-27">INFO::2015-01-27</a>
14:37:52,090::sourceroute::78::root::(configure) Configuring
gateway - ip: **.***.**.**, network: 10.100.100.0/24,
subnet: 255.255.255.0, gateway: **.***.**.**, table:
174351414, device: bond0<br>
sourceRoute::DEBUG::2015-01-27
14:37:52,090::utils::739::root::(execCmd) /sbin/ip -4 route
add 0.0.0.0/0 via **.***.**.** dev bond0 table 174351414
(cwd None)<br>
sourceRoute::DEBUG::2015-01-27
14:37:52,094::utils::759::root::(execCmd) FAILED:
<err> = 'RTNETLINK answers: File exists\n'; <rc>
= 2<br>
sourceRoute::ERROR::2015-01-27
14:37:52,094::sourceroute::88::root::(configure) ip binary
failed during source route configuration: ['RTNETLINK
answers: File exists']<br>
sourceRoute::WARNING::2015-01-27
14:37:52,094::utils::129::root::(rmFile) File:
/var/run/vdsm/trackedInterfaces/bond0 already removed<br>
<br>
<br>
</blockquote>
If you need more info, let me know.<br>
<br>
<br>
<div class="moz-cite-prefix">On 01/27/2015 02:20 PM, Ondřej
Svoboda wrote:<br>
</div>
<blockquote cite="mid:54C7909B.9050103@svobodasoft.cz"
type="cite">
<meta content="text/html; charset=UTF-8"
http-equiv="Content-Type">
Jorick,<br>
<br>
just what comes to my mind: do you have NetworkManager
running, by any chance? VDSM and NetworkManager don't mix
well currently so please try<br>
<br>
# systemctl stop NetworkManager<br>
# systemctl disable NetworkManager<br>
<br>
Regards,<br>
Ondra Svoboda<br>
<br>
<div class="moz-cite-prefix">On 27.1.2015 14:10, Jorick
Astrego wrote:<br>
</div>
<blockquote cite="mid:54C78E52.20305@netbulae.eu"
type="cite">
<meta content="text/html; charset=UTF-8"
http-equiv="Content-Type">
<br>
<div class="moz-cite-prefix">On 01/27/2015 01:56 PM, Koen
Vanoppen wrote:<br>
</div>
<blockquote
cite="mid:CACfY+MZ4zSzYnqbf3aM5wnHiNVMrm2bwsj9YwAQAeWO53ppdDw@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>
<div>Hello everybody,<br>
<br>
</div>
Just wanted to know if anybody else has the
problem, that when you restart your network
configuration of a hypervisor, all the settings
concerning the bonding are gone...<br>
</div>
Do I file a bug report for this or is this already a
known issue?<br>
<br>
</div>
Kind regards,<br>
<br>
Koen<br>
</div>
<br>
</blockquote>
Hmm,<br>
<br>
I cannot restart the network service from the host for
some reason:<br>
<br>
<blockquote>Jan 27 14:00:27 test2 systemd:
network.service: control process exited, code=exited
status=1<br>
Jan 27 14:00:27 test2 systemd: Failed to start LSB:
Bring up/down networking.<br>
<br>
<br>
systemctl status network.service<br>
network.service - LSB: Bring up/down networking<br>
Loaded: loaded (/etc/rc.d/init.d/network)<br>
Active: failed (Result: exit-code) since Tue
2015-01-27 14:00:27 CET; 1min 56s ago<br>
Process: 4113 ExecStart=/etc/rc.d/init.d/network start
(code=exited, status=1/FAILURE)<br>
<br>
Jan 27 14:00:27 test2.netbulae.test network[4113]:
RTNETLINK answers: File exists<br>
Jan 27 14:00:27 test2.netbulae.test network[4113]:
RTNETLINK answers: File exists<br>
Jan 27 14:00:27 test2.netbulae.test network[4113]:
RTNETLINK answers: File exists<br>
Jan 27 14:00:27 test2.netbulae.test network[4113]:
RTNETLINK answers: File exists<br>
Jan 27 14:00:27 test2.netbulae.test network[4113]:
RTNETLINK answers: File exists<br>
Jan 27 14:00:27 test2.netbulae.test network[4113]:
RTNETLINK answers: File exists<br>
Jan 27 14:00:27 test2.netbulae.test network[4113]:
RTNETLINK answers: File exists<br>
Jan 27 14:00:27 test2.netbulae.test systemd[1]:
network.service: control process exited, code=exited
status=1<br>
Jan 27 14:00:27 test2.netbulae.test systemd[1]: Failed
to start LSB: Bring up/down networking.<br>
Jan 27 14:00:27 test2.netbulae.test systemd[1]: Unit
network.service entered failed state.<br>
</blockquote>
<blockquote>-- Subject: Unit network.service has failed<br>
-- Defined-By: systemd<br>
-- Support: <a moz-do-not-send="true"
class="moz-txt-link-freetext"
href="http://lists.freedesktop.org/mailman/listinfo/systemd-devel">http://lists.freedesktop.org/mailman/listinfo/systemd-devel</a><br>
-- <br>
-- Unit network.service has failed.<br>
-- <br>
-- The result is failed.<br>
Jan 27 14:00:27 test2.netbulae.test systemd[1]: Unit
network.service entered failed state.<br>
<br>
<br>
In supervdsm I find lots and lots of these:<br>
<br>
<br>
sourceRoute::DEBUG::2015-01-27
14:02:40,802::utils::739::root::(execCmd) /sbin/ip -4
route add 0.0.0.0/0 via **.***.**.** dev bond0 table
174351414 (cwd None)<br>
sourceRoute::DEBUG::2015-01-27
14:02:40,806::utils::759::root::(execCmd) FAILED:
<err> = 'RTNETLINK answers: File exists\n';
<rc> = 2<br>
sourceRoute::ERROR::2015-01-27
14:02:40,806::sourceroute::88::root::(configure) ip
binary failed during source route configuration:
['RTNETLINK answers: File exists']<br>
sourceRoute::WARNING::2015-01-27
14:02:40,807::utils::129::root::(rmFile) File:
/var/run/vdsm/trackedInterfaces/bond0 already removed<br>
sourceRoute::DEBUG::2015-01-27
14:07:09,047::sourceroutethread::39::root::(process_IN_CLOSE_WRITE_filePath)
Responding to DHCP response in
/var/run/vdsm/sourceRoutes/1422364029<br>
sourceRoute::<a moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-01-27">INFO::2015-01-27</a>
14:07:09,052::sourceroute::78::root::(configure)
Configuring gateway - ip: **.***.**.***, network:
**.***.**.**0/24, subnet: 255.255.255.0, gateway:
**.***.**.**, table: 174351414, device: bond0<br>
sourceRoute::DEBUG::2015-01-27
14:07:09,052::utils::739::root::(execCmd) /sbin/ip -4
route add 0.0.0.0/0 via **.***.**.** dev bond0 table
174351414 (cwd None)<br>
sourceRoute::DEBUG::2015-01-27
14:07:09,057::utils::759::root::(execCmd) FAILED:
<err> = 'RTNETLINK answers: File exists\n';
<rc> = 2<br>
sourceRoute::ERROR::2015-01-27
14:07:09,057::sourceroute::88::root::(configure) ip
binary failed during source route configuration:
['RTNETLINK answers: File exists']<br>
sourceRoute::WARNING::2015-01-27
14:07:09,057::utils::129::root::(rmFile) File:
/var/run/vdsm/trackedInterfaces/bond0 already removed<br>
<br>
<br>
</blockquote>
The hosts are all Centos 7 freshly provisioned. With these
vdsm packages:<br>
<br>
<blockquote>vdsm-python-zombiereaper-4.16.10-8.gitc937927.el7.noarch<br>
vdsm-python-4.16.10-8.gitc937927.el7.noarch<br>
vdsm-yajsonrpc-4.16.10-8.gitc937927.el7.noarch<br>
vdsm-4.16.10-8.gitc937927.el7.x86_64<br>
vdsm-xmlrpc-4.16.10-8.gitc937927.el7.noarch<br>
vdsm-jsonrpc-4.16.10-8.gitc937927.el7.noarch<br>
vdsm-cli-4.16.10-8.gitc937927.el7.noarch<br>
</blockquote>
<br>
<br>
<br>
<br>
<span style="color:#604c78;"><font color="000000"><span
style="mso-fareast-language:en-gb;" lang="NL">Met
vriendelijke groet, With kind regards,<br>
<br>
</span>Jorick Astrego</font></span><b
style="color:#604c78"><br>
<br>
Netbulae Virtualization Experts </b><br>
<hr style="border:none;border-top:1px solid #ccc;">
<table style="width: 522px">
<tbody>
<tr>
<td style="width: 130px;font-size: 10px">Tel: 053 20
30 270</td>
<td style="width: 130px;font-size: 10px"><a
moz-do-not-send="true"
class="moz-txt-link-abbreviated"
href="mailto:info@netbulae.eu">info@netbulae.eu</a></td>
<td style="width: 130px;font-size: 10px">Staalsteden
4-3A</td>
<td style="width: 130px;font-size: 10px">KvK
08198180</td>
</tr>
<tr>
<td style="width: 130px;font-size: 10px">Fax: 053 20
30 271</td>
<td style="width: 130px;font-size: 10px"><a
moz-do-not-send="true"
class="moz-txt-link-abbreviated"
href="http://www.netbulae.eu">www.netbulae.eu</a></td>
<td style="width: 130px;font-size: 10px">7547 TA
Enschede</td>
<td style="width: 130px;font-size: 10px">BTW
NL821234584B01</td>
</tr>
</tbody>
</table>
<br>
<hr style="border:none;border-top:1px solid #ccc;"><br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Users mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a>
</pre>
</blockquote>
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Users mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a>
</pre>
</blockquote>
<br>
<br>
<br>
<br>
<span style="color:#604c78;"><font color="000000"><span
style="mso-fareast-language:en-gb;" lang="NL">Met
vriendelijke groet, With kind regards,<br>
<br>
</span>Jorick Astrego</font></span><b
style="color:#604c78"><br>
<br>
Netbulae Virtualization Experts </b><br>
<hr style="border:none;border-top:1px solid #ccc;">
<table style="width: 522px">
<tbody>
<tr>
<td style="width: 130px;font-size: 10px">Tel: 053 20 30
270</td>
<td style="width: 130px;font-size: 10px"><a
moz-do-not-send="true"
class="moz-txt-link-abbreviated"
href="mailto:info@netbulae.eu">info@netbulae.eu</a></td>
<td style="width: 130px;font-size: 10px">Staalsteden
4-3A</td>
<td style="width: 130px;font-size: 10px">KvK 08198180</td>
</tr>
<tr>
<td style="width: 130px;font-size: 10px">Fax: 053 20 30
271</td>
<td style="width: 130px;font-size: 10px"><a
moz-do-not-send="true"
class="moz-txt-link-abbreviated"
href="http://www.netbulae.eu">www.netbulae.eu</a></td>
<td style="width: 130px;font-size: 10px">7547 TA
Enschede</td>
<td style="width: 130px;font-size: 10px">BTW
NL821234584B01</td>
</tr>
</tbody>
</table>
<br>
<hr style="border:none;border-top:1px solid #ccc;"><br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Users mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a>
</pre>
</blockquote>
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Users mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a>
</pre>
</blockquote>
<br>
<br>
<br>
<br>
<span style="color:#604c78;"><font color="000000"><span
style="mso-fareast-language:en-gb;" lang="NL">Met
vriendelijke groet, With kind regards,<br>
<br>
</span>Jorick Astrego</font></span><b style="color:#604c78"><br>
<br>
Netbulae Virtualization Experts </b><br>
<hr style="border:none;border-top:1px solid #ccc;">
<table style="width: 522px">
<tbody>
<tr>
<td style="width: 130px;font-size: 10px">Tel: 053 20 30 270</td>
<td style="width: 130px;font-size: 10px"><a class="moz-txt-link-abbreviated" href="mailto:info@netbulae.eu">info@netbulae.eu</a></td>
<td style="width: 130px;font-size: 10px">Staalsteden 4-3A</td>
<td style="width: 130px;font-size: 10px">KvK 08198180</td>
</tr>
<tr>
<td style="width: 130px;font-size: 10px">Fax: 053 20 30 271</td>
<td style="width: 130px;font-size: 10px"><a class="moz-txt-link-abbreviated" href="http://www.netbulae.eu">www.netbulae.eu</a></td>
<td style="width: 130px;font-size: 10px">7547 TA Enschede</td>
<td style="width: 130px;font-size: 10px">BTW NL821234584B01</td>
</tr>
</tbody>
</table>
<br>
<hr style="border:none;border-top:1px solid #ccc;"><br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a>
<a class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a>
</pre>
</blockquote>
<br>
</body>
</html>