[ovirt-users] 3.5.1 net config persistence

[Nicolas Ecarnot]

Le 20/03/2015 14:40, Dan Kenigsberg a écrit :
> On Fri, Mar 20, 2015 at 10:14:54AM +0100, Nicolas Ecarnot wrote:
>> Hello,
>>
>> There are a number of bugs [1] reported these days about the issue aourd
>> network config of the hosts, when dealing with interfaces manually
>> configured, with bonding and VLANs.
>> These /etc/sysconfig/network-scripts/ifcfg.* files are wiped by vdsm after
>> rebooting.
>>
>> I see that there are people at Redhat working on these, and some cases were
>> reproduced in lab conditions - and some were not.
>>
>> I upgraded 3 DC from 3.4.? to 3.5.1, and faced this issue (lost of every
>> network files) in an non-consistent manner.
>>
>> I finally thought I coped with this problem by adding
>> net_persistence = ifcfg
>> to /etc/vdsm/vdsm.conf
>> and indeed, when restarting vdsmd and the network, files were conserved.
>>
>> It was before I observed that some action [2] lead to /etc/vdsm/vdsm.conf
>> being renamed into /etc/vdsm/vdsm.conf.some_timestamp and the original one
>> replaced by a very short file with no netcfg persistence at all.
>>
>> I didn't identified [2]. That could be :
>> - some actions made by me through the Web UI ?
>> - service vdsmd restart ?
>> - reboots ?
>>
>> I'm sure that some Redhat people know what could be responsible for renaming
>> /etc/vdsm/vdsm.conf into /etc/vdsm/vdsm.conf.some_timestamp, and I wish they
>> are working closely with Dan Kenigsberg and Michael Burman who helped a lot
>> on these issues (or maybe, THEY are the coders responsible for this ?)
>>
>> [2] :
>> - https://bugzilla.redhat.com/show_bug.cgi?id=1154399
>> - https://bugzilla.redhat.com/show_bug.cgi?id=1188251
>> - and more or less related :
>> https://bugzilla.redhat.com/show_bug.cgi?id=1134346
>
> Thanks for reporting this issue. We are well aware of it, and working
> hard to fix it. Unfortunately, there were several bugs on the process of
> upgrading ifcfg-based network configuration to vdsm's own "unified
> persistence" that sits under /var/lib/vdsm/persistence/netconf.
>
> Would you share which platform are you using? el6? el7? ovirt-node, or
> plain install?

We are using centos 6.6 on all our hosts, minimal install.
Idem on the manager, bare metal stand alone, not hosted.

> There is a recent report that ovirt-node may be restarting networking
> while vdsm starts up, which may well explain the problem and its
> inpredictability. Is this the case with you?

We are not using ovirt-nodes since 3 years, for some reasons.

> Regarding /etc/vdsm/vdsm.conf: vdsm never rename it. Could it be rpm's
> new behavior (replacing vdsm.conf.rpmsave) ? Or could it be the node,
> Fabian?

Let us stay prudent : I indeed did some yum upgrade, BUT :
- I made every step in a very modular way : first upgrade the manager
- then put one host in maintenance
- add the 3.5.1 repo on the host
- then web-gui-reinstall it (upgrading the useful packages)
- then put it up, migrate some VM on it, well, test it.
- then put it back into maintenance
- then yum upgrade it
- then reboot it
- then blah blah blah

well you see, I won't explain every step, but I did that in a very 
cautious way, taking time for each of them, and repeating this whole 
process more than 20 times.

I don't get why it is working like a charm on most of them, and facing 
the issues mentioned above on a portion of them.


To answer to the renaming comment : yes Dan, some package upgrade 
renamed vdsm.conf into rpmsave, BUT I was explicitly talking about an 
additional renaming into vdsm.conf.201503191220 something, and I never 
saw a package upgrade do that.


Just a final word : though I sound grumpy and find this issue a real 
pain, I am actually absolutely amazed by all the work done by all the 
oVirt community and the Redhat people :)

-- 
Nicolas Ecarnot