[ovirt-users] Unable to reinstall hosts after network removal.

Tue Dec 30 12:21:26 UTC 2014

Ok I found the conflict. The ip rule list  does not have any record on IB0
interface, all other networks are updated correctly but not IB one.
Probably this is not a good idea to have an ISCSI storage devices with the
same interface as VM network.
the network update operation trying to put interface up and down in that
point all iscsi devices going to IO/failure this stops to update IB
interface data.

Maybe one need to put whole cluster to maintenance mode in order to change
Storage dependent network setup?

this is the main trouble which prevents to fix/change any network setup on
IB0 interface.
tail -f  /var/log/vdsm/vdsm.log
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/hsm.py", line 2430, in connectStorageServer
    conObj.connect()
  File "/usr/share/vdsm/storage/storageServer.py", line 367, in connect
    iscsi.addIscsiNode(self._iface, self._target, self._cred)
  File "/usr/share/vdsm/storage/iscsi.py", line 164, in addIscsiNode
    iscsiadm.node_login(iface.name, portalStr, target.iqn)
  File "/usr/share/vdsm/storage/iscsiadm.py", line 295, in node_login
    raise IscsiNodeError(rc, out, err)
IscsiNodeError: (8, ['Logging in to [iface: default, target:
iqn.2003-01.org.linux-iscsi.data01.x8664:sn.f5c9b017e103, portal:
10.10.10.31,3260] (multiple)'], ['iscsiadm: Could not login to [iface:
default, target: iqn.2003-01.org.linux-iscsi.data01.x8664:sn.f5c9b017e103,
portal: 10.10.10.31,3260].', 'iscsiadm: initiator reported error (8 -
connection timed out)', 'iscsiadm: Could not log into all portals'])

PS
"save network configuration"  is always checked.

***********************************************************

Dr. Arman Khalatyan eScience -SuperComputing Leibniz-Institut für
Astrophysik Potsdam (AIP) An der Sternwarte 16, 14482 Potsdam, Germany

***********************************************************

On Tue, Dec 30, 2014 at 7:05 AM, Moti Asayag <masayag at redhat.com> wrote:

>
>
> ----- Original Message -----
> > From: "Arman Khalatyan" <arm2arm at gmail.com>
> > To: "Moti Asayag" <masayag at redhat.com>
> > Cc: "users" <users at ovirt.org>
> > Sent: Monday, December 29, 2014 3:59:46 PM
> > Subject: Re: [ovirt-users] Unable to reinstall hosts after network
> removal.
> >
> > My setup  has 3 networks: 1xIB,1x10Gbit+1Gbit for ovirt-management.
> > The ovirt network does not have any trouble, it is always there.
> > I was trying to rename or remove my IB network which was used for VM
> > migrations.
> > I was using web-GUI, which was removing IB0 network w/o problem. After
> > removal the hosts where ok.
>
> Have you checked the option of "save network configuration" in the setup
> networks
> dialog (iirc should have been checked by default) ?
>
> Could you also attach the /var/log/ovirt-engine/engine.log from the engine
> server
> and /var/log/vdsm/vdsm.log and /var/log/vdsm/supervdsm.log from the node so
> we can see which request was sent to vdsm and its result ?
>
> > Then I put them to maintenance mode. To refresh
> > iptables rules I did reinstall. then reinstall was failing with message
> > that IB0 not attached to any interface. But IB0 interface is not possible
> > to attach it is already deleted and not visible in any network dialog.
> > After creating interface with the same name everything is online now.
> > My current interface list is following:
> >  virsh -r net-list
> > Name                 State      Autostart     Persistent
> > --------------------------------------------------
> > ;vdsmdummy;          active     no            no
> > vdsm-cls10G          active     yes           yes
> > vdsm-IB0             active     yes           yes
> > vdsm-ovirtmgmt       active     yes           yes
> >
> > On this host the IB0 is not atached to any interface, I wondered, if it
> > should show up  in the net list?
> >
> > I think the GUI does not rename/remove the interface from the DB. Some
> > constrain keeps still IB0 in DB.
> >
> >
> >
> > ***********************************************************
> >
> > Dr. Arman Khalatyan eScience -SuperComputing Leibniz-Institut für
> > Astrophysik Potsdam (AIP) An der Sternwarte 16, 14482 Potsdam, Germany
> >
> > ***********************************************************
> >
> >
> > On Sun, Dec 28, 2014 at 10:31 AM, Moti Asayag <masayag at redhat.com>
> wrote:
> >
> > >
> > >
> > > ----- Original Message -----
> > > > From: "Arman Khalatyan" <arm2arm at gmail.com>
> > > > To: "users" <users at ovirt.org>
> > > > Sent: Wednesday, December 24, 2014 1:22:43 PM
> > > > Subject: [ovirt-users] Unable to reinstall hosts after network
> removal.
> > > >
> > > > Hello,
> > > > I have a little trouble with ovirt 3.5 on CentOS6.6:
> > > > I was removing all networks from all hosts.
> > >
> > > Did you use the setup networks dialog from the UI in order to remove
> those
> > > networks ?
> > > Or have you removed those networks from the host directly (where you
> > > should used the:
> > > 1. virsh net-destroy 'the-network-name'
> > > 2. virsh net-undefine 'the-network-name'
> > > )
> > >
> > > can you report the output of 'virsh -r net-list'  ?
> > >
> > > > Then after removing network from data center the hosts went to
> unusable.
> > >
> > > What was the host's status prior to removing its networks ? Was it up ?
> > >
> > > > Every time after reinstall the host claims that the network is not
> > > > configured, but it s already removed from network tab in DC.
> > >
> > > What is the missing network name ? Is it 'ovirtmgmt' ?
> > >
> > > > Where from it gets the old configuration? the old interfaces also
> > > restored
> > > > every time on the reinstalled hosts.
> > >
> > > The hosts via vdsm reports their network configuration via the
> > > 'getCapabilities' verb
> > > of vdsm. You can try running it on the host:
> > >
> > > vdsClient -s 0 getVdsCaps
> > >
> > > and examine the nics / neworks / bridges / vlans / bonds elements.
> > >
> > > > Which DB table is in charge of dc-networks?
> > > >
> > >
> > > The retrieved information from vdsm is reported to 'vds_interace'
> table.
> > > The dc networks are stored in 'networks' table and networks attached to
> > > clusters are stored in network_cluster table.
> > >
> > > I wouldn't recommend on deleting entries from the tables directly.
> There
> > > are
> > > certain constraints which shouldn't be violated, i.e. the management
> > > network
> > > 'ovirtmgnt' is blocked for removal from the engine.
> > >
> > > > Thanks,
> > > > Arman.
> > > >
> > > >
> > > > ***********************************************************
> > > > Dr. Arman Khalatyan eScience -SuperComputing Leibniz-Institut für
> > > Astrophysik
> > > > Potsdam (AIP) An der Sternwarte 16, 14482 Potsdam, Germany
> > > > ***********************************************************
> > > >
> > > > _______________________________________________
> > > > Users mailing list
> > > > Users at ovirt.org
> > > > http://lists.ovirt.org/mailman/listinfo/users
> > > >
> > >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20141230/c2b25d1b/attachment-0001.html>