[Users] [vdsm] ovirtmgmt vanishes after reboot

[Antoni Segura Puimedon]

----- Original Message -----
> From: "Dan Kenigsberg" <danken at redhat.com>
> To: "Francisco Pérez" <fperez.x at gmail.com>
> Cc: "Fabian Deutsch" <fabiand at redhat.com>, amuller at redhat.com, asegurap at redhat.com, "VDSM Project Development"
> <vdsm-devel at lists.fedorahosted.org>, users at ovirt.org
> Sent: Tuesday, January 28, 2014 10:47:17 PM
> Subject: Re: [vdsm] [Users] ovirtmgmt vanishes after reboot
> 
> On Tue, Jan 28, 2014 at 11:27:21AM -0300, Francisco Pérez wrote:
> >  Im setting up two networks. ovirtmgmt and a LAN. Both of them are gone
> > after reboot on the node, its starts only with eth0 initial configuration.
> > 
> > Antoni, here are the outputs you requested:
> > 
> > [root at ovirth1 ~]# /etc/init.d/vdsmd status
> > VDS daemon server is running
> > [root at ovirth1 ~]# service vdsmd status
> > VDS daemon server is running
> > [root at ovirth1 ~]# virsh -r net-list
> > setlocale: No such file or directory
> > Name                 State      Autostart     Persistent
> > --------------------------------------------------
> > ;vdsmdummy;          active     no            no
> > 
> > [root at ovirth1 ~]# ls /etc/sysconfig/network-scripts/
> > ifcfg-eth0   ifdown-eth   ifdown-post    ifdown-tunnel  ifup-eth
> > ifup-plip   ifup-routes    init.ipv6-global        route-LAN
> > ifcfg-lo     ifdown-ippp  ifdown-ppp     ifup           ifup-ippp
> >  ifup-plusb  ifup-sit       net.hotplug             route-ovirtmgmt
> > ifdown       ifdown-ipv6  ifdown-routes  ifup-aliases   ifup-ipv6
> >  ifup-post   ifup-tunnel    network-functions       rule-LAN
> > ifdown-bnep  ifdown-isdn  ifdown-sit     ifup-bnep      ifup-isdn  ifup-ppp
> >    ifup-wireless  network-functions-ipv6  rule-ovirtmgmt
> > 
> > Dan those command are excecuted where? on the manager or the node? Im new
> > at this and the logs are a little overwhelming so if you help me to
> > interpret them i'll apreciate them. I have attached the whole log directory
> > /var/log/vdsm/
> > >
> > > Could you share vdsm.log and supervdsm.log? When you set up ovirtmgmt,
> > > as setupNetworks command should be visible there. Later, on "save", a
> > > call to setSafeNetworkConfig is sent, which is expected to persist your
> > > network configuration.
> > >
> > > Is it visible? Is it successful?
> 
> Your supervdsmd has
> 
>     MainProcess|Thread-26::DEBUG::2014-01-27
>     15:51:48,866::configNetwork::561::Storage.Misc.excCmd::(setSafeNetworkConfig)
>     '/usr/share/vdsm/vdsm-store-net-config' (cwd None)
> 
> which reports success, too.
> 
> > >
> > > What do you see in the logs post boot? Can you tell whether
> > > the vdsm-restore-net-config.init server was run? (its job is to revert
> > > unsaved network config changes on non-ovirt-nodes).
> 
> Maybe you can debug the issue:
> Setup up a network but do not save it yet.
> Log into your ovirt-node.
> 
> You should see /etc/sysconfig/network-scripts/ifcfg-<YOURNET>
> and /var/lib/vdsm/netconfback/ifcfg-<YOURNET>, the
> latter meaning that it's not yet persisted.
> 
> Now run locally
>   vdsClient -s 0 setSafeNetworkConfig
> 
> You should see /config/etc/sysconfig/network-scripts/ifcfg-<YOURNET>
> created,
> and /var/lib/vdsm/netconfback/ifcfg-<YOURNET> gone.

/var/lib/vdsm/netconfback/ifcfg-<YOURNET> is not removed on oVirt Node
as /var/lib/vdsm/netconfback is sitting on a tmpfs so there was no
need to call it for the reboot case.

We made some more tests with Francisco on #ovirt today and we saw that
for some unknown reason, when rebooting the machine the ifcfg-<YOURNET>
files disappear while the rule-<YOURNET> and route-<YOURNET> stay. That
happens even when the both ifcfg, rule and route have the correct entry
in /config/files and are bound to /config/etc/sysconfig/

After reboot it returns to eth0 having the connectivity as it was defined
with the TUI.

For libvirt networks a bit of the same as for ifcfg files. Before reboot
/etc/libvirt/qemu/networks is present in /config/files and findmnt reports
that /etc/libvirt/qemu/networks is a bound mount of /config/etc/libvirt/qemu/networks.
After reboot it is not the case anymore.

@Fabian: Do you think there is something on the reboot that restores the
network conf to the TUI settings?

> 
> Now please run
>     /usr/share/vdsm/vdsm-restore-net-config
> which should do absolutely nothing.
> 
> If the process above breaks, it's a Vdsm bug. If not, it most probably
> something in ovirt-node.
>