[ovirt-users] Re: major network changes

On Thu, Jul 18, 2019 at 2:50 PM Miguel Duarte de Mora Barroso <mdbarroso(a)redhat.com&gt; wrote: > > On Thu, Jul 18, 2019 at 1:57 PM carl langlois <crl.langlois(a)gmail.com&gt; wrote: > > > > Hi Miguel, > > > > I have managed to change the config for the ovn-controler. > > with those commands > > ovs-vsctl set Open_vSwitch . external-ids:ovn-remote=ssl: 10.16.248.74:6642 > > ovs-vsctl set Open_vSwitch . external-ids:ovn-encap-ip=10.16.248.65 > > and restating the services > > Yes, that's what the script is supposed to do, check [0]. > > Not sure why running vdsm-tool didn't work for you. > > > > > But even with this i still have the "fail for liveliness check" when starting the ovirt engine. But one thing i notice with our new network is that the reverse DNS does not work(IP -> hostname). The forward is working fine. I am trying to see with our IT why it is not working. > > Do you guys use OVN? If not, you could disable the provider, install > the hosted-engine VM, then, if needed, re-add / re-activate it . I'm assuming it fails for the same reason you've stated initially - i.e. ovn-controller is involved; if it is not, disregard this msg :) > > [0] - https://github.com/oVirt/ovirt-provider-ovn/blob/master/driver/scripts/se... > > > > > Regards. > > Carl > > > > On Thu, Jul 18, 2019 at 4:03 AM Miguel Duarte de Mora Barroso < mdbarroso(a)redhat.com&gt; wrote: > >> > >> On Wed, Jul 17, 2019 at 7:07 PM carl langlois <crl.langlois(a)gmail.com&gt; wrote: > >> > > >> > Hi > >> > Here is the output of the command > >> > > >> > [root@ovhost1 ~]# vdsm-tool --vvverbose ovn-config 10.16.248.74 ovirtmgmt > >> > MainThread::DEBUG::2019-07-17 13:02:52,581::cmdutils::150::root::(exec_cmd) lshw -json -disable usb -disable pcmcia -disable isapnp -disable ide -disable scsi -disable dmi -disable memory -disable cpuinfo (cwd None) > >> > MainThread::DEBUG::2019-07-17 13:02:52,738::cmdutils::158::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 > >> > MainThread::DEBUG::2019-07-17 13:02:52,741::routes::109::root::(get_gateway) The gateway 10.16.248.1 is duplicated for the device ovirtmgmt > >> > MainThread::DEBUG::2019-07-17 13:02:52,742::routes::109::root::(get_gateway) The gateway 10.16.248.1 is duplicated for the device ovirtmgmt > >> > MainThread::DEBUG::2019-07-17 13:02:52,742::cmdutils::150::root::(exec_cmd) /sbin/tc qdisc show (cwd None) > >> > MainThread::DEBUG::2019-07-17 13:02:52,744::cmdutils::158::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 > >> > MainThread::DEBUG::2019-07-17 13:02:52,745::cmdutils::150::root::(exec_cmd) /sbin/tc class show dev enp2s0f1 classid 0:1388 (cwd None) > >> > MainThread::DEBUG::2019-07-17 13:02:52,747::cmdutils::158::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 > >> > MainThread::DEBUG::2019-07-17 13:02:52,766::cmdutils::150::root::(exec_cmd) /usr/share/openvswitch/scripts/ovs-ctl status (cwd None) > >> > MainThread::DEBUG::2019-07-17 13:02:52,777::cmdutils::158::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 > >> > MainThread::DEBUG::2019-07-17 13:02:52,778::vsctl::67::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface > >> > MainThread::DEBUG::2019-07-17 13:02:52,778::cmdutils::150::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface (cwd None) > >> > MainThread::DEBUG::2019-07-17 13:02:52,799::cmdutils::158::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 > >> > netlink/events::DEBUG::2019-07-17 13:02:52,802::concurrent::192::root::(run) START thread <Thread(netlink/events, started daemon 140299323660032)> (func=<bound method Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at 0x7f99fb618c90>>, args=(), kwargs={}) > >> > netlink/events::DEBUG::2019-07-17 13:02:54,805::concurrent::195::root::(run) FINISH thread <Thread(netlink/events, started daemon 140299323660032)> > >> > Using default PKI files > >> > > >> > I do not see any indication of the config?? > >> > >> And afterwards when you execute "ovs-vsctl list Open_vSwitch" does it > >> reflect the updated value ? > >> > >> This command would have to be performed in the node where hosted > >> engine will be hosted - not sure if it's possible to determine before > >> hand which one it will be. If not, you should run it in all the nodes > >> in the cluster, to be sure. > >> > >> > > >> > Regards > >> > Carl > >> > > >> > On Wed, Jul 17, 2019 at 11:40 AM carl langlois < crl.langlois(a)gmail.com&gt; wrote: > >> >> > >> >> Hi > >> >> > >> >> I have open a bug https://bugzilla.redhat.com/show_bug.cgi?id=1730776 > >> >> > >> >> I have try this command "vdsm-tool ovn-config 10.16.248.74 ovirtmgmt" on one of the host but nothing changed. After a restart of the ovn-controler i still get > >> >> > >> >> 2019-07-17T15:38:52.572Z|00033|reconnect|INFO|ssl: 10.8.236.244:6642: waiting 8 seconds before reconnect > >> >> 2019-07-17T15:39:00.578Z|00034|reconnect|INFO|ssl: 10.8.236.244:6642: connecting... > >> >> 2019-07-17T15:39:05.720Z|00035|fatal_signal|WARN|terminating with signal 15 (Terminated) > >> >> 2019-07-17T15:39:05.863Z|00001|vlog|INFO|opened log file /var/log/openvswitch/ovn-controller.log > >> >> 2019-07-17T15:39:05.864Z|00002|reconnect|INFO|unix:/var/run/openvswitch/db.sock: connecting... > >> >> 2019-07-17T15:39:05.864Z|00003|reconnect|INFO|unix:/var/run/openvswitch/db.sock: connected > >> >> 2019-07-17T15:39:05.865Z|00004|reconnect|INFO|ssl: 10.8.236.244:6642: connecting... > >> >> 2019-07-17T15:39:06.865Z|00005|reconnect|INFO|ssl: 10.8.236.244:6642: connection attempt timed out > >> >> 2019-07-17T15:39:06.865Z|00006|reconnect|INFO|ssl: 10.8.236.244:6642: waiting 1 seconds before reconnect > >> >> 2019-07-17T15:39:07.867Z|00007|reconnect|INFO|ssl: 10.8.236.244:6642: connecting... > >> >> 2019-07-17T15:39:08.867Z|00008|reconnect|INFO|ssl: 10.8.236.244:6642: connection attempt timed out > >> >> 2019-07-17T15:39:08.868Z|00009|reconnect|INFO|ssl: 10.8.236.244:6642: waiting 2 seconds before reconnect > >> >> 2019-07-17T15:39:10.870Z|00010|reconnect|INFO|ssl: 10.8.236.244:6642: connecting... > >> >> 2019-07-17T15:39:12.872Z|00011|reconnect|INFO|ssl: 10.8.236.244:6642: connection attempt timed out > >> >> 2019-07-17T15:39:12.872Z|00012|reconnect|INFO|ssl: 10.8.236.244:6642: waiting 4 seconds before reconnect > >> >> > >> >> > >> >> > >> >> On Wed, Jul 17, 2019 at 10:56 AM Miguel Duarte de Mora Barroso < mdbarroso(a)redhat.com&gt; wrote: > >> >>> > >> >>> On Wed, Jul 17, 2019 at 3:01 PM carl langlois < crl.langlois(a)gmail.com&gt; wrote: > >> >>> > > >> >>> > Hi Miguel > >> >>> > > >> >>> > if i do ovs-vsctl list Open_vSwitch i get > >> >>> > > >> >>> > uuid : ce94c4b1-7eb2-42e3-8bfd-96e1dec40dea > >> >>> > bridges : [9b0738ee-594d-4a87-8967-049a8b1a5774] > >> >>> > cur_cfg : 1 > >> >>> > datapath_types : [netdev, system] > >> >>> > db_version : "7.14.0" > >> >>> > external_ids : {hostname="ovhost2", ovn-bridge-mappings="", ovn-encap-ip="10.8.236.150", ovn-encap-type=geneve, ovn-remote="ssl:10.8.236.244:6642", system-id="7c39d07b-1d54-417b-bf56-7a0f1a07f832"} > >> >>> > iface_types : [geneve, gre, internal, lisp, patch, stt, system, tap, vxlan] > >> >>> > manager_options : [] > >> >>> > next_cfg : 1 > >> >>> > other_config : {} > >> >>> > ovs_version : "2.7.3" > >> >>> > ssl : [] > >> >>> > statistics : {} > >> >>> > system_type : centos > >> >>> > system_version : "7" > >> >>> > > >> >>> > I can see two addresses that are on the old network.. > >> >>> > >> >>> Yes, those are it. > >> >>> > >> >>> Use the tool I mentioned to update that to the correct addresses on > >> >>> the network, and re-try. > >> >>> > >> >>> vdsm-tool ovn-config <engine_ip_on_net> <name of the management network> > >> >>> > >> >>> > Regards > >> >>> > Carl > >> >>> > > >> >>> > > >> >>> > On Wed, Jul 17, 2019 at 8:21 AM carl langlois < crl.langlois(a)gmail.com&gt; wrote: > >> >>> >> > >> >>> >> Hi Miguel, > >> >>> >> > >> >>> >> I will surely open a bugs, any specific ovirt componenent to select when openeing the bug? > >> >>> > >> >>> ovirt-engine > >> >>> > >> >>> >> > >> >>> >> When you say that the hosted-engine should have trigger a the update. Do you mean is was suppose to trigger the update and did not work or it is something missing? > >> >>> > >> >>> I sincerely do not know. @Dominik Holler, could you shed some light into this ? > >> >>> > >> >>> >> Could i have missed a step when switching the network? > >> >>> >> > >> >>> >> Also if i try to do ovs-vsctl list . The list command require a Table name. Not sure what table to use? > >> >>> >> > >> >>> >> Regards > >> >>> >> Carl > >> >>> >> > >> >>> >> > >> >>> >> > >> >>> >> On Wed, Jul 17, 2019 at 4:21 AM Miguel Duarte de Mora Barroso < mdbarroso(a)redhat.com&gt; wrote: > >> >>> >>> > >> >>> >>> On Tue, Jul 16, 2019 at 8:48 PM carl langlois < crl.langlois(a)gmail.com&gt; wrote: > >> >>> >>> > > >> >>> >>> > Hi > >> >>> >>> > > >> >>> >>> > We are in a process of changing our network connection. Our current network is using 10.8.256.x and we will change to 10.16.248.x. We have a HA ovirt cluster (around 10 nodes) currently configure on the 10.8.256.x. So my question is is it possible to relocate the ovirt cluster to the 10.16.248.x. We have tried to move everything to the new network without success. All the node seem to boot up properly, our gluster storage also work properly. > >> >>> >>> > When we try to start the hosted-engine it goes up but fail the liveliness check. We have notice in the /var/log/openvswitch/ovn-controller.log that he is triying to connect to the hold ip address of the hosted-engine vm. > >> >>> >>> > 019-07-16T18:41:29.483Z|01992|reconnect|INFO|ssl: 10.8.236.244:6642: waiting 8 seconds before reconnect > >> >>> >>> > 2019-07-16T18:41:37.489Z|01993|reconnect|INFO|ssl: 10.8.236.244:6642: connecting... > >> >>> >>> > 2019-07-16T18:41:45.497Z|01994|reconnect|INFO|ssl: 10.8.236.244:6642: connection attempt timed out > >> >>> >>> > > >> >>> >>> > So my question is were is the 10.8.236.244 come from. > >> >>> >>> > >> >>> >>> Looks like the ovn controllers were not updated during the network change. > >> >>> >>> > >> >>> >>> The wrong IP is configured within openvswitch, you can see it in the > >> >>> >>> (offending) nodes through "ovs-vsctl list . ". It'll be a key in the > >> >>> >>> 'external_ids' column called 'ovn-remote' . > >> >>> >>> > >> >>> >>> This is not the solution, but a work-around; you could try to > >> >>> >>> configure the ovn controllers via: > >> >>> >>> vdsm-tool ovn-config <engine_ip_on_net> <name of the management network> > >> >>> >>> > >> >>> >>> Despite the provided work-around, I really think the hosted engine > >> >>> >>> should have triggered the ansible role that in turn triggers this > >> >>> >>> reconfiguration. > >> >>> >>> > >> >>> >>> Would you open a bug with this information ? > >> >>> >>> > >> >>> >>> > >> >>> >>> > > >> >>> >>> > The routing table for one of our host look like this > >> >>> >>> > > >> >>> >>> > estination Gateway Genmask Flags Metric Ref Use Iface > >> >>> >>> > default gateway 0.0.0.0 UG 0 0 0 ovirtmgmt > >> >>> >>> > 10.16.248.0 0.0.0.0 255.255.255.0 U 0 0 0 ovirtmgmt > >> >>> >>> > link-local 0.0.0.0 255.255.0.0 U 1002 0 0 eno1 > >> >>> >>> > link-local 0.0.0.0 255.255.0.0 U 1003 0 0 eno2 > >> >>> >>> > link-local 0.0.0.0 255.255.0.0 U 1025 0 0 ovirtmgmt > >> >>> >>> > > >> >>> >>> > Any help would be really appreciated. > >> >>> >>> > > >> >>> >>> > Regards > >> >>> >>> > Carl > >> >>> >>> > > >> >>> >>> > > >> >>> >>> > > >> >>> >>> > > >> >>> >>> > _______________________________________________ > >> >>> >>> > Users mailing list -- users(a)ovirt.org > >> >>> >>> > To unsubscribe send an email to users-leave(a)ovirt.org > >> >>> >>> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > >> >>> >>> > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > >> >>> >>> > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/DBQUWEPPDK2...