On Tue, Jul 16, 2019 at 8:48 PM carl langlois <crl.langlois(a)gmail.com&gt; wrote: > > Hi > > We are in a process of changing our network connection. Our current network is using 10.8.256.x and we will change to 10.16.248.x. We have a HA ovirt cluster (around 10 nodes) currently configure on the 10.8.256.x. So my question is is it possible to relocate the ovirt cluster to the 10.16.248.x. We have tried to move everything to the new network without success. All the node seem to boot up properly, our gluster storage also work properly. > When we try to start the hosted-engine it goes up but fail the liveliness check. We have notice in the /var/log/openvswitch/ovn-controller.log that he is triying to connect to the hold ip address of the hosted-engine vm. > 019-07-16T18:41:29.483Z|01992|reconnect|INFO|ssl:10.8.236.244:6642: waiting 8 seconds before reconnect > 2019-07-16T18:41:37.489Z|01993|reconnect|INFO|ssl:10.8.236.244:6642: connecting... > 2019-07-16T18:41:45.497Z|01994|reconnect|INFO|ssl:10.8.236.244:6642: connection attempt timed out > > So my question is were is the 10.8.236.244 come from. Looks like the ovn controllers were not updated during the network change. The wrong IP is configured within openvswitch, you can see it in the (offending) nodes through "ovs-vsctl list . ". It'll be a key in the 'external_ids' column called 'ovn-remote' . This is not the solution, but a work-around; you could try to configure the ovn controllers via: vdsm-tool ovn-config <engine_ip_on_net> <name of the management network> Despite the provided work-around, I really think the hosted engine should have triggered the ansible role that in turn triggers this reconfiguration. Would you open a bug with this information ? > > The routing table for one of our host look like this > > estination Gateway Genmask Flags Metric Ref Use Iface > default gateway 0.0.0.0 UG 0 0 0 ovirtmgmt > 10.16.248.0 0.0.0.0 255.255.255.0 U 0 0 0 ovirtmgmt > link-local 0.0.0.0 255.255.0.0 U 1002 0 0 eno1 > link-local 0.0.0.0 255.255.0.0 U 1003 0 0 eno2 > link-local 0.0.0.0 255.255.0.0 U 1025 0 0 ovirtmgmt > > Any help would be really appreciated. > > Regards > Carl > > > > > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/DBQUWEPPDK2...

Hi Miguel if i do ovs-vsctl list Open_vSwitch i get uuid : ce94c4b1-7eb2-42e3-8bfd-96e1dec40dea bridges : [9b0738ee-594d-4a87-8967-049a8b1a5774] cur_cfg : 1 datapath_types : [netdev, system] db_version : "7.14.0" external_ids : {hostname="ovhost2", ovn-bridge-mappings="", ovn-encap-ip="10.8.236.150", ovn-encap-type=geneve, ovn-remote="ssl:10.8.236.244:6642", system-id="7c39d07b-1d54-417b-bf56-7a0f1a07f832"} iface_types : [geneve, gre, internal, lisp, patch, stt, system, tap, vxlan] manager_options : [] next_cfg : 1 other_config : {} ovs_version : "2.7.3" ssl : [] statistics : {} system_type : centos system_version : "7" I can see two addresses that are on the old network..

Regards Carl On Wed, Jul 17, 2019 at 8:21 AM carl langlois <crl.langlois(a)gmail.com&gt; wrote: > > Hi Miguel, > > I will surely open a bugs, any specific ovirt componenent to select when openeing the bug?

> > When you say that the hosted-engine should have trigger a the update. Do you mean is was suppose to trigger the update and did not work or it is something missing?

Hi I have open a bug https://bugzilla.redhat.com/show_bug.cgi?id=1730776 I have try this command "vdsm-tool ovn-config 10.16.248.74 ovirtmgmt" on one of the host but nothing changed. After a restart of the ovn-controler i still get 2019-07-17T15:38:52.572Z|00033|reconnect|INFO|ssl:10.8.236.244:6642: waiting 8 seconds before reconnect 2019-07-17T15:39:00.578Z|00034|reconnect|INFO|ssl:10.8.236.244:6642: connecting... 2019-07-17T15:39:05.720Z|00035|fatal_signal|WARN|terminating with signal 15 (Terminated) 2019-07-17T15:39:05.863Z|00001|vlog|INFO|opened log file /var/log/openvswitch/ovn-controller.log 2019-07-17T15:39:05.864Z|00002|reconnect|INFO|unix:/var/run/openvswitch/db.sock: connecting... 2019-07-17T15:39:05.864Z|00003|reconnect|INFO|unix:/var/run/openvswitch/db.sock: connected 2019-07-17T15:39:05.865Z|00004|reconnect|INFO|ssl:10.8.236.244:6642: connecting... 2019-07-17T15:39:06.865Z|00005|reconnect|INFO|ssl:10.8.236.244:6642: connection attempt timed out 2019-07-17T15:39:06.865Z|00006|reconnect|INFO|ssl:10.8.236.244:6642: waiting 1 seconds before reconnect 2019-07-17T15:39:07.867Z|00007|reconnect|INFO|ssl:10.8.236.244:6642: connecting... 2019-07-17T15:39:08.867Z|00008|reconnect|INFO|ssl:10.8.236.244:6642: connection attempt timed out 2019-07-17T15:39:08.868Z|00009|reconnect|INFO|ssl:10.8.236.244:6642: waiting 2 seconds before reconnect 2019-07-17T15:39:10.870Z|00010|reconnect|INFO|ssl:10.8.236.244:6642: connecting... 2019-07-17T15:39:12.872Z|00011|reconnect|INFO|ssl:10.8.236.244:6642: connection attempt timed out 2019-07-17T15:39:12.872Z|00012|reconnect|INFO|ssl:10.8.236.244:6642: waiting 4 seconds before reconnect On Wed, Jul 17, 2019 at 10:56 AM Miguel Duarte de Mora Barroso < mdbarroso(a)redhat.com&gt; wrote: > On Wed, Jul 17, 2019 at 3:01 PM carl langlois <crl.langlois(a)gmail.com&gt; > wrote: > > > > Hi Miguel > > > > if i do ovs-vsctl list Open_vSwitch i get > > > > uuid : ce94c4b1-7eb2-42e3-8bfd-96e1dec40dea > > bridges : [9b0738ee-594d-4a87-8967-049a8b1a5774] > > cur_cfg : 1 > > datapath_types : [netdev, system] > > db_version : "7.14.0" > > external_ids : {hostname="ovhost2", ovn-bridge-mappings="", > ovn-encap-ip="10.8.236.150", ovn-encap-type=geneve, ovn-remote="ssl: > 10.8.236.244:6642", system-id="7c39d07b-1d54-417b-bf56-7a0f1a07f832"} > > iface_types : [geneve, gre, internal, lisp, patch, stt, system, > tap, vxlan] > > manager_options : [] > > next_cfg : 1 > > other_config : {} > > ovs_version : "2.7.3" > > ssl : [] > > statistics : {} > > system_type : centos > > system_version : "7" > > > > I can see two addresses that are on the old network.. > > Yes, those are it. > > Use the tool I mentioned to update that to the correct addresses on > the network, and re-try. > > vdsm-tool ovn-config <engine_ip_on_net> <name of the management network> > > > Regards > > Carl > > > > > > On Wed, Jul 17, 2019 at 8:21 AM carl langlois <crl.langlois(a)gmail.com&gt; > wrote: > >> > >> Hi Miguel, > >> > >> I will surely open a bugs, any specific ovirt componenent to select > when openeing the bug? > > ovirt-engine > > >> > >> When you say that the hosted-engine should have trigger a the update. > Do you mean is was suppose to trigger the update and did not work or it is > something missing? > > I sincerely do not know. @Dominik Holler, could you shed some light into > this ? > > >> Could i have missed a step when switching the network? > >> > >> Also if i try to do ovs-vsctl list . The list command require a Table > name. Not sure what table to use? > >> > >> Regards > >> Carl > >> > >> > >> > >> On Wed, Jul 17, 2019 at 4:21 AM Miguel Duarte de Mora Barroso < > mdbarroso(a)redhat.com&gt; wrote: > >>> > >>> On Tue, Jul 16, 2019 at 8:48 PM carl langlois <crl.langlois(a)gmail.com&gt; > wrote: > >>> > > >>> > Hi > >>> > > >>> > We are in a process of changing our network connection. Our current > network is using 10.8.256.x and we will change to 10.16.248.x. We have a HA > ovirt cluster (around 10 nodes) currently configure on the 10.8.256.x. So > my question is is it possible to relocate the ovirt cluster to the > 10.16.248.x. We have tried to move everything to the new network without > success. All the node seem to boot up properly, our gluster storage also > work properly. > >>> > When we try to start the hosted-engine it goes up but fail the > liveliness check. We have notice in the > /var/log/openvswitch/ovn-controller.log that he is triying to connect to > the hold ip address of the hosted-engine vm. > >>> > 019-07-16T18:41:29.483Z|01992|reconnect|INFO|ssl:10.8.236.244:6642: > waiting 8 seconds before reconnect > >>> > 2019-07-16T18:41:37.489Z|01993|reconnect|INFO|ssl:10.8.236.244:6642: > connecting... > >>> > 2019-07-16T18:41:45.497Z|01994|reconnect|INFO|ssl:10.8.236.244:6642: > connection attempt timed out > >>> > > >>> > So my question is were is the 10.8.236.244 come from. > >>> > >>> Looks like the ovn controllers were not updated during the network > change. > >>> > >>> The wrong IP is configured within openvswitch, you can see it in the > >>> (offending) nodes through "ovs-vsctl list . ". It'll be a key in the > >>> 'external_ids' column called 'ovn-remote' . > >>> > >>> This is not the solution, but a work-around; you could try to > >>> configure the ovn controllers via: > >>> vdsm-tool ovn-config <engine_ip_on_net> <name of the management > network> > >>> > >>> Despite the provided work-around, I really think the hosted engine > >>> should have triggered the ansible role that in turn triggers this > >>> reconfiguration. > >>> > >>> Would you open a bug with this information ? > >>> > >>> > >>> > > >>> > The routing table for one of our host look like this > >>> > > >>> > estination Gateway Genmask Flags Metric Ref > Use Iface > >>> > default gateway 0.0.0.0 UG 0 0 > 0 ovirtmgmt > >>> > 10.16.248.0 0.0.0.0 255.255.255.0 U 0 0 > 0 ovirtmgmt > >>> > link-local 0.0.0.0 255.255.0.0 U 1002 0 > 0 eno1 > >>> > link-local 0.0.0.0 255.255.0.0 U 1003 0 > 0 eno2 > >>> > link-local 0.0.0.0 255.255.0.0 U 1025 0 > 0 ovirtmgmt > >>> > > >>> > Any help would be really appreciated. > >>> > > >>> > Regards > >>> > Carl > >>> > > >>> > > >>> > > >>> > > >>> > _______________________________________________ > >>> > Users mailing list -- users(a)ovirt.org > >>> > To unsubscribe send an email to users-leave(a)ovirt.org > >>> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > >>> > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > >>> > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/DBQUWEPPDK2... >

On Wed, Jul 17, 2019 at 7:07 PM carl langlois <crl.langlois(a)gmail.com&gt; wrote: > > Hi > Here is the output of the command > > [root@ovhost1 ~]# vdsm-tool --vvverbose ovn-config 10.16.248.74 ovirtmgmt > MainThread::DEBUG::2019-07-17 13:02:52,581::cmdutils::150::root::(exec_cmd) lshw -json -disable usb -disable pcmcia -disable isapnp -disable ide -disable scsi -disable dmi -disable memory -disable cpuinfo (cwd None) > MainThread::DEBUG::2019-07-17 13:02:52,738::cmdutils::158::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 > MainThread::DEBUG::2019-07-17 13:02:52,741::routes::109::root::(get_gateway) The gateway 10.16.248.1 is duplicated for the device ovirtmgmt > MainThread::DEBUG::2019-07-17 13:02:52,742::routes::109::root::(get_gateway) The gateway 10.16.248.1 is duplicated for the device ovirtmgmt > MainThread::DEBUG::2019-07-17 13:02:52,742::cmdutils::150::root::(exec_cmd) /sbin/tc qdisc show (cwd None) > MainThread::DEBUG::2019-07-17 13:02:52,744::cmdutils::158::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 > MainThread::DEBUG::2019-07-17 13:02:52,745::cmdutils::150::root::(exec_cmd) /sbin/tc class show dev enp2s0f1 classid 0:1388 (cwd None) > MainThread::DEBUG::2019-07-17 13:02:52,747::cmdutils::158::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 > MainThread::DEBUG::2019-07-17 13:02:52,766::cmdutils::150::root::(exec_cmd) /usr/share/openvswitch/scripts/ovs-ctl status (cwd None) > MainThread::DEBUG::2019-07-17 13:02:52,777::cmdutils::158::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 > MainThread::DEBUG::2019-07-17 13:02:52,778::vsctl::67::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface > MainThread::DEBUG::2019-07-17 13:02:52,778::cmdutils::150::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface (cwd None) > MainThread::DEBUG::2019-07-17 13:02:52,799::cmdutils::158::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 > netlink/events::DEBUG::2019-07-17 13:02:52,802::concurrent::192::root::(run) START thread <Thread(netlink/events, started daemon 140299323660032)> (func=<bound method Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at 0x7f99fb618c90>>, args=(), kwargs={}) > netlink/events::DEBUG::2019-07-17 13:02:54,805::concurrent::195::root::(run) FINISH thread <Thread(netlink/events, started daemon 140299323660032)> > Using default PKI files > > I do not see any indication of the config?? And afterwards when you execute "ovs-vsctl list Open_vSwitch" does it reflect the updated value ? This command would have to be performed in the node where hosted engine will be hosted - not sure if it's possible to determine before hand which one it will be. If not, you should run it in all the nodes in the cluster, to be sure. > > Regards > Carl > > On Wed, Jul 17, 2019 at 11:40 AM carl langlois <crl.langlois(a)gmail.com&gt; wrote: >> >> Hi >> >> I have open a bug https://bugzilla.redhat.com/show_bug.cgi?id=1730776 >> >> I have try this command "vdsm-tool ovn-config 10.16.248.74 ovirtmgmt" on one of the host but nothing changed. After a restart of the ovn-controler i still get >> >> 2019-07-17T15:38:52.572Z|00033|reconnect|INFO|ssl:10.8.236.244:6642: waiting 8 seconds before reconnect >> 2019-07-17T15:39:00.578Z|00034|reconnect|INFO|ssl:10.8.236.244:6642: connecting... >> 2019-07-17T15:39:05.720Z|00035|fatal_signal|WARN|terminating with signal 15 (Terminated) >> 2019-07-17T15:39:05.863Z|00001|vlog|INFO|opened log file /var/log/openvswitch/ovn-controller.log >> 2019-07-17T15:39:05.864Z|00002|reconnect|INFO|unix:/var/run/openvswitch/db.sock: connecting... >> 2019-07-17T15:39:05.864Z|00003|reconnect|INFO|unix:/var/run/openvswitch/db.sock: connected >> 2019-07-17T15:39:05.865Z|00004|reconnect|INFO|ssl:10.8.236.244:6642: connecting... >> 2019-07-17T15:39:06.865Z|00005|reconnect|INFO|ssl:10.8.236.244:6642: connection attempt timed out >> 2019-07-17T15:39:06.865Z|00006|reconnect|INFO|ssl:10.8.236.244:6642: waiting 1 seconds before reconnect >> 2019-07-17T15:39:07.867Z|00007|reconnect|INFO|ssl:10.8.236.244:6642: connecting... >> 2019-07-17T15:39:08.867Z|00008|reconnect|INFO|ssl:10.8.236.244:6642: connection attempt timed out >> 2019-07-17T15:39:08.868Z|00009|reconnect|INFO|ssl:10.8.236.244:6642: waiting 2 seconds before reconnect >> 2019-07-17T15:39:10.870Z|00010|reconnect|INFO|ssl:10.8.236.244:6642: connecting... >> 2019-07-17T15:39:12.872Z|00011|reconnect|INFO|ssl:10.8.236.244:6642: connection attempt timed out >> 2019-07-17T15:39:12.872Z|00012|reconnect|INFO|ssl:10.8.236.244:6642: waiting 4 seconds before reconnect >> >> >> >> On Wed, Jul 17, 2019 at 10:56 AM Miguel Duarte de Mora Barroso < mdbarroso(a)redhat.com&gt; wrote: >>> >>> On Wed, Jul 17, 2019 at 3:01 PM carl langlois <crl.langlois(a)gmail.com&gt; wrote: >>> > >>> > Hi Miguel >>> > >>> > if i do ovs-vsctl list Open_vSwitch i get >>> > >>> > uuid : ce94c4b1-7eb2-42e3-8bfd-96e1dec40dea >>> > bridges : [9b0738ee-594d-4a87-8967-049a8b1a5774] >>> > cur_cfg : 1 >>> > datapath_types : [netdev, system] >>> > db_version : "7.14.0" >>> > external_ids : {hostname="ovhost2", ovn-bridge-mappings="", ovn-encap-ip="10.8.236.150", ovn-encap-type=geneve, ovn-remote="ssl: 10.8.236.244:6642", system-id="7c39d07b-1d54-417b-bf56-7a0f1a07f832"} >>> > iface_types : [geneve, gre, internal, lisp, patch, stt, system, tap, vxlan] >>> > manager_options : [] >>> > next_cfg : 1 >>> > other_config : {} >>> > ovs_version : "2.7.3" >>> > ssl : [] >>> > statistics : {} >>> > system_type : centos >>> > system_version : "7" >>> > >>> > I can see two addresses that are on the old network.. >>> >>> Yes, those are it. >>> >>> Use the tool I mentioned to update that to the correct addresses on >>> the network, and re-try. >>> >>> vdsm-tool ovn-config <engine_ip_on_net> <name of the management network> >>> >>> > Regards >>> > Carl >>> > >>> > >>> > On Wed, Jul 17, 2019 at 8:21 AM carl langlois < crl.langlois(a)gmail.com&gt; wrote: >>> >> >>> >> Hi Miguel, >>> >> >>> >> I will surely open a bugs, any specific ovirt componenent to select when openeing the bug? >>> >>> ovirt-engine >>> >>> >> >>> >> When you say that the hosted-engine should have trigger a the update. Do you mean is was suppose to trigger the update and did not work or it is something missing? >>> >>> I sincerely do not know. @Dominik Holler, could you shed some light into this ? >>> >>> >> Could i have missed a step when switching the network? >>> >> >>> >> Also if i try to do ovs-vsctl list . The list command require a Table name. Not sure what table to use? >>> >> >>> >> Regards >>> >> Carl >>> >> >>> >> >>> >> >>> >> On Wed, Jul 17, 2019 at 4:21 AM Miguel Duarte de Mora Barroso < mdbarroso(a)redhat.com&gt; wrote: >>> >>> >>> >>> On Tue, Jul 16, 2019 at 8:48 PM carl langlois < crl.langlois(a)gmail.com&gt; wrote: >>> >>> > >>> >>> > Hi >>> >>> > >>> >>> > We are in a process of changing our network connection. Our current network is using 10.8.256.x and we will change to 10.16.248.x. We have a HA ovirt cluster (around 10 nodes) currently configure on the 10.8.256.x. So my question is is it possible to relocate the ovirt cluster to the 10.16.248.x. We have tried to move everything to the new network without success. All the node seem to boot up properly, our gluster storage also work properly. >>> >>> > When we try to start the hosted-engine it goes up but fail the liveliness check. We have notice in the /var/log/openvswitch/ovn-controller.log that he is triying to connect to the hold ip address of the hosted-engine vm. >>> >>> > 019-07-16T18:41:29.483Z|01992|reconnect|INFO|ssl: 10.8.236.244:6642: waiting 8 seconds before reconnect >>> >>> > 2019-07-16T18:41:37.489Z|01993|reconnect|INFO|ssl: 10.8.236.244:6642: connecting... >>> >>> > 2019-07-16T18:41:45.497Z|01994|reconnect|INFO|ssl: 10.8.236.244:6642: connection attempt timed out >>> >>> > >>> >>> > So my question is were is the 10.8.236.244 come from. >>> >>> >>> >>> Looks like the ovn controllers were not updated during the network change. >>> >>> >>> >>> The wrong IP is configured within openvswitch, you can see it in the >>> >>> (offending) nodes through "ovs-vsctl list . ". It'll be a key in the >>> >>> 'external_ids' column called 'ovn-remote' . >>> >>> >>> >>> This is not the solution, but a work-around; you could try to >>> >>> configure the ovn controllers via: >>> >>> vdsm-tool ovn-config <engine_ip_on_net> <name of the management network> >>> >>> >>> >>> Despite the provided work-around, I really think the hosted engine >>> >>> should have triggered the ansible role that in turn triggers this >>> >>> reconfiguration. >>> >>> >>> >>> Would you open a bug with this information ? >>> >>> >>> >>> >>> >>> > >>> >>> > The routing table for one of our host look like this >>> >>> > >>> >>> > estination Gateway Genmask Flags Metric Ref Use Iface >>> >>> > default gateway 0.0.0.0 UG 0 0 0 ovirtmgmt >>> >>> > 10.16.248.0 0.0.0.0 255.255.255.0 U 0 0 0 ovirtmgmt >>> >>> > link-local 0.0.0.0 255.255.0.0 U 1002 0 0 eno1 >>> >>> > link-local 0.0.0.0 255.255.0.0 U 1003 0 0 eno2 >>> >>> > link-local 0.0.0.0 255.255.0.0 U 1025 0 0 ovirtmgmt >>> >>> > >>> >>> > Any help would be really appreciated. >>> >>> > >>> >>> > Regards >>> >>> > Carl >>> >>> > >>> >>> > >>> >>> > >>> >>> > >>> >>> > _______________________________________________ >>> >>> > Users mailing list -- users(a)ovirt.org >>> >>> > To unsubscribe send an email to users-leave(a)ovirt.org >>> >>> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> >>> > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>> >>> > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/DBQUWEPPDK2...

On Thu, Jul 18, 2019 at 1:57 PM carl langlois <crl.langlois(a)gmail.com&gt; wrote: > > Hi Miguel, > > I have managed to change the config for the ovn-controler. > with those commands > ovs-vsctl set Open_vSwitch . external-ids:ovn-remote=ssl:10.16.248.74:6642 > ovs-vsctl set Open_vSwitch . external-ids:ovn-encap-ip=10.16.248.65 > and restating the services Yes, that's what the script is supposed to do, check [0]. Not sure why running vdsm-tool didn't work for you. > > But even with this i still have the "fail for liveliness check" when starting the ovirt engine. But one thing i notice with our new network is that the reverse DNS does not work(IP -> hostname). The forward is working fine. I am trying to see with our IT why it is not working. Do you guys use OVN? If not, you could disable the provider, install the hosted-engine VM, then, if needed, re-add / re-activate it .

Hi, We have managed to stabilize the DNS udpate in out network. Now the current situation is. I have 3 hosts that can run the engine (hosted-engine). They were all in the 10.8.236.x. Now i have moved one of them in the 10.16.248.x. If i boot the engine on one of the host that is in the 10.8.236.x the engine is going up with status "good". I can access the engine UI. I can see all my hosts even the one in the 10.16.248.x network. But if i boot the engine on the hosted-engine host that was switch to the 10.16.248.x the engine is booting. I can ssh to it but the status is always " fail for liveliness check". The main difference is that when i boot on the host that is in the 10.16.248.x network the engine gets a address in the 248.x network. On the engine i have this in the /var/log/ovirt-engine-dwh/ovirt-engine-dwhd.log 019-07-23 09:05:30|MFzehi|YYTDiS|jTq2w8|OVIRT_ENGINE_DWH|SampleTimeKeepingJob|Default|5|tWarn|tWarn_1|Can not sample data, oVirt Engine is not updating the statistics. Please check your oVirt Engine status.|9704 the engine.log seems okey. So i need to understand what this " liveliness check" do(or try to do) so i can investigate why the engine status is not becoming good. The initial deployment was done in the 10.8.236.x network. Maybe is as something to do with that. Thanks & Regards Carl On Thu, Jul 18, 2019 at 8:53 AM Miguel Duarte de Mora Barroso < mdbarroso(a)redhat.com&gt; wrote: > On Thu, Jul 18, 2019 at 2:50 PM Miguel Duarte de Mora Barroso > <mdbarroso(a)redhat.com&gt; wrote: > > > > On Thu, Jul 18, 2019 at 1:57 PM carl langlois <crl.langlois(a)gmail.com&gt; > wrote: > > > > > > Hi Miguel, > > > > > > I have managed to change the config for the ovn-controler. > > > with those commands > > > ovs-vsctl set Open_vSwitch . external-ids:ovn-remote=ssl: > 10.16.248.74:6642 > > > ovs-vsctl set Open_vSwitch . external-ids:ovn-encap-ip=10.16.248.65 > > > and restating the services > > > > Yes, that's what the script is supposed to do, check [0]. > > > > Not sure why running vdsm-tool didn't work for you. > > > > > > > > But even with this i still have the "fail for liveliness check" when > starting the ovirt engine. But one thing i notice with our new network > is > that the reverse DNS does not work(IP -> hostname). The forward is > working > fine. I am trying to see with our IT why it is not working. > > > > Do you guys use OVN? If not, you could disable the provider, install > > the hosted-engine VM, then, if needed, re-add / re-activate it . > > I'm assuming it fails for the same reason you've stated initially - > i.e. ovn-controller is involved; if it is not, disregard this msg :) > > > > [0] - > https://github.com/oVirt/ovirt-provider-ovn/blob/master/driver/scripts/se... > > > > > > > > Regards. > > > Carl > > > > > > On Thu, Jul 18, 2019 at 4:03 AM Miguel Duarte de Mora Barroso < > mdbarroso(a)redhat.com&gt; wrote: > > >> > > >> On Wed, Jul 17, 2019 at 7:07 PM carl langlois > <crl.langlois(a)gmail.com&gt; > wrote: > > >> > > > >> > Hi > > >> > Here is the output of the command > > >> > > > >> > [root@ovhost1 ~]# vdsm-tool --vvverbose ovn-config 10.16.248.74 > ovirtmgmt > > >> > MainThread::DEBUG::2019-07-17 > 13:02:52,581::cmdutils::150::root::(exec_cmd) lshw -json -disable usb > -disable pcmcia -disable isapnp -disable ide -disable scsi -disable dmi > -disable memory -disable cpuinfo (cwd None) > > >> > MainThread::DEBUG::2019-07-17 > 13:02:52,738::cmdutils::158::root::(exec_cmd) SUCCESS: <err> = ''; <rc> > = 0 > > >> > MainThread::DEBUG::2019-07-17 > 13:02:52,741::routes::109::root::(get_gateway) The gateway 10.16.248.1 > is > duplicated for the device ovirtmgmt > > >> > MainThread::DEBUG::2019-07-17 > 13:02:52,742::routes::109::root::(get_gateway) The gateway 10.16.248.1 > is > duplicated for the device ovirtmgmt > > >> > MainThread::DEBUG::2019-07-17 > 13:02:52,742::cmdutils::150::root::(exec_cmd) /sbin/tc qdisc show (cwd > None) > > >> > MainThread::DEBUG::2019-07-17 > 13:02:52,744::cmdutils::158::root::(exec_cmd) SUCCESS: <err> = ''; <rc> > = 0 > > >> > MainThread::DEBUG::2019-07-17 > 13:02:52,745::cmdutils::150::root::(exec_cmd) /sbin/tc class show dev > enp2s0f1 classid 0:1388 (cwd None) > > >> > MainThread::DEBUG::2019-07-17 > 13:02:52,747::cmdutils::158::root::(exec_cmd) SUCCESS: <err> = ''; <rc> > = 0 > > >> > MainThread::DEBUG::2019-07-17 > 13:02:52,766::cmdutils::150::root::(exec_cmd) > /usr/share/openvswitch/scripts/ovs-ctl status (cwd None) > > >> > MainThread::DEBUG::2019-07-17 > 13:02:52,777::cmdutils::158::root::(exec_cmd) SUCCESS: <err> = ''; <rc> > = 0 > > >> > MainThread::DEBUG::2019-07-17 > 13:02:52,778::vsctl::67::root::(commit) Executing commands: > /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- > list Port -- list Interface > > >> > MainThread::DEBUG::2019-07-17 > 13:02:52,778::cmdutils::150::root::(exec_cmd) /usr/bin/ovs-vsctl > --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list > Interface (cwd None) > > >> > MainThread::DEBUG::2019-07-17 > 13:02:52,799::cmdutils::158::root::(exec_cmd) SUCCESS: <err> = ''; <rc> > = 0 > > >> > netlink/events::DEBUG::2019-07-17 > 13:02:52,802::concurrent::192::root::(run) START thread > <Thread(netlink/events, started daemon 140299323660032)> (func=<bound > method Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at > 0x7f99fb618c90>>, args=(), kwargs={}) > > >> > netlink/events::DEBUG::2019-07-17 > 13:02:54,805::concurrent::195::root::(run) FINISH thread > <Thread(netlink/events, started daemon 140299323660032)> > > >> > Using default PKI files > > >> > > > >> > I do not see any indication of the config?? > > >> > > >> And afterwards when you execute "ovs-vsctl list Open_vSwitch" does > it > > >> reflect the updated value ? > > >> > > >> This command would have to be performed in the node where hosted > > >> engine will be hosted - not sure if it's possible to determine > before > > >> hand which one it will be. If not, you should run it in all the > nodes > > >> in the cluster, to be sure. > > >> > > >> > > > >> > Regards > > >> > Carl > > >> > > > >> > On Wed, Jul 17, 2019 at 11:40 AM carl langlois < > crl.langlois(a)gmail.com&gt; wrote: > > >> >> > > >> >> Hi > > >> >> > > >> >> I have open a bug > https://bugzilla.redhat.com/show_bug.cgi?id=1730776 > > >> >> > > >> >> I have try this command "vdsm-tool ovn-config 10.16.248.74 > ovirtmgmt" on one of the host but nothing changed. After a restart of > the > ovn-controler i still get > > >> >> > > >> >> 2019-07-17T15:38:52.572Z|00033|reconnect|INFO|ssl: > 10.8.236.244:6642: waiting 8 seconds before reconnect > > >> >> 2019-07-17T15:39:00.578Z|00034|reconnect|INFO|ssl: > 10.8.236.244:6642: connecting... > > >> >> 2019-07-17T15:39:05.720Z|00035|fatal_signal|WARN|terminating > with > signal 15 (Terminated) > > >> >> 2019-07-17T15:39:05.863Z|00001|vlog|INFO|opened log file > /var/log/openvswitch/ovn-controller.log > > >> >> > 2019-07-17T15:39:05.864Z|00002|reconnect|INFO|unix:/var/run/openvswitch/db.sock: > connecting... > > >> >> > 2019-07-17T15:39:05.864Z|00003|reconnect|INFO|unix:/var/run/openvswitch/db.sock: > connected > > >> >> 2019-07-17T15:39:05.865Z|00004|reconnect|INFO|ssl: > 10.8.236.244:6642: connecting... > > >> >> 2019-07-17T15:39:06.865Z|00005|reconnect|INFO|ssl: > 10.8.236.244:6642: connection attempt timed out > > >> >> 2019-07-17T15:39:06.865Z|00006|reconnect|INFO|ssl: > 10.8.236.244:6642: waiting 1 seconds before reconnect > > >> >> 2019-07-17T15:39:07.867Z|00007|reconnect|INFO|ssl: > 10.8.236.244:6642: connecting... > > >> >> 2019-07-17T15:39:08.867Z|00008|reconnect|INFO|ssl: > 10.8.236.244:6642: connection attempt timed out > > >> >> 2019-07-17T15:39:08.868Z|00009|reconnect|INFO|ssl: > 10.8.236.244:6642: waiting 2 seconds before reconnect > > >> >> 2019-07-17T15:39:10.870Z|00010|reconnect|INFO|ssl: > 10.8.236.244:6642: connecting... > > >> >> 2019-07-17T15:39:12.872Z|00011|reconnect|INFO|ssl: > 10.8.236.244:6642: connection attempt timed out > > >> >> 2019-07-17T15:39:12.872Z|00012|reconnect|INFO|ssl: > 10.8.236.244:6642: waiting 4 seconds before reconnect > > >> >> > > >> >> > > >> >> > > >> >> On Wed, Jul 17, 2019 at 10:56 AM Miguel Duarte de Mora Barroso < > mdbarroso(a)redhat.com&gt; wrote: > > >> >>> > > >> >>> On Wed, Jul 17, 2019 at 3:01 PM carl langlois < > crl.langlois(a)gmail.com&gt; wrote: > > >> >>> > > > >> >>> > Hi Miguel > > >> >>> > > > >> >>> > if i do ovs-vsctl list Open_vSwitch i get > > >> >>> > > > >> >>> > uuid : ce94c4b1-7eb2-42e3-8bfd-96e1dec40dea > > >> >>> > bridges : [9b0738ee-594d-4a87-8967-049a8b1a5774] > > >> >>> > cur_cfg : 1 > > >> >>> > datapath_types : [netdev, system] > > >> >>> > db_version : "7.14.0" > > >> >>> > external_ids : {hostname="ovhost2", > ovn-bridge-mappings="", ovn-encap-ip="10.8.236.150", > ovn-encap-type=geneve, > ovn-remote="ssl:10.8.236.244:6642", > system-id="7c39d07b-1d54-417b-bf56-7a0f1a07f832"} > > >> >>> > iface_types : [geneve, gre, internal, lisp, patch, > stt, > system, tap, vxlan] > > >> >>> > manager_options : [] > > >> >>> > next_cfg : 1 > > >> >>> > other_config : {} > > >> >>> > ovs_version : "2.7.3" > > >> >>> > ssl : [] > > >> >>> > statistics : {} > > >> >>> > system_type : centos > > >> >>> > system_version : "7" > > >> >>> > > > >> >>> > I can see two addresses that are on the old network.. > > >> >>> > > >> >>> Yes, those are it. > > >> >>> > > >> >>> Use the tool I mentioned to update that to the correct > addresses > on > > >> >>> the network, and re-try. > > >> >>> > > >> >>> vdsm-tool ovn-config <engine_ip_on_net> <name of the management > network> > > >> >>> > > >> >>> > Regards > > >> >>> > Carl > > >> >>> > > > >> >>> > > > >> >>> > On Wed, Jul 17, 2019 at 8:21 AM carl langlois < > crl.langlois(a)gmail.com&gt; wrote: > > >> >>> >> > > >> >>> >> Hi Miguel, > > >> >>> >> > > >> >>> >> I will surely open a bugs, any specific ovirt componenent to > select when openeing the bug? > > >> >>> > > >> >>> ovirt-engine > > >> >>> > > >> >>> >> > > >> >>> >> When you say that the hosted-engine should have trigger a > the > update. Do you mean is was suppose to trigger the update and did not > work > or it is something missing? > > >> >>> > > >> >>> I sincerely do not know. @Dominik Holler, could you shed some > light into this ? > > >> >>> > > >> >>> >> Could i have missed a step when switching the network? > > >> >>> >> > > >> >>> >> Also if i try to do ovs-vsctl list . The list command > require > a Table name. Not sure what table to use? > > >> >>> >> > > >> >>> >> Regards > > >> >>> >> Carl > > >> >>> >> > > >> >>> >> > > >> >>> >> > > >> >>> >> On Wed, Jul 17, 2019 at 4:21 AM Miguel Duarte de Mora > Barroso < > mdbarroso(a)redhat.com&gt; wrote: > > >> >>> >>> > > >> >>> >>> On Tue, Jul 16, 2019 at 8:48 PM carl langlois < > crl.langlois(a)gmail.com&gt; wrote: > > >> >>> >>> > > > >> >>> >>> > Hi > > >> >>> >>> > > > >> >>> >>> > We are in a process of changing our network connection. > Our > current network is using 10.8.256.x and we will change to 10.16.248.x. > We > have a HA ovirt cluster (around 10 nodes) currently configure on the > 10.8.256.x. So my question is is it possible to relocate the ovirt > cluster > to the 10.16.248.x. We have tried to move everything to the new network > without success. All the node seem to boot up properly, our gluster > storage > also work properly. > > >> >>> >>> > When we try to start the hosted-engine it goes up but > fail > the liveliness check. We have notice in the > /var/log/openvswitch/ovn-controller.log that he is triying to connect to > the hold ip address of the hosted-engine vm. > > >> >>> >>> > 019-07-16T18:41:29.483Z|01992|reconnect|INFO|ssl: > 10.8.236.244:6642: waiting 8 seconds before reconnect > > >> >>> >>> > 2019-07-16T18:41:37.489Z|01993|reconnect|INFO|ssl: > 10.8.236.244:6642: connecting... > > >> >>> >>> > 2019-07-16T18:41:45.497Z|01994|reconnect|INFO|ssl: > 10.8.236.244:6642: connection attempt timed out > > >> >>> >>> > > > >> >>> >>> > So my question is were is the 10.8.236.244 come from. > > >> >>> >>> > > >> >>> >>> Looks like the ovn controllers were not updated during the > network change. > > >> >>> >>> > > >> >>> >>> The wrong IP is configured within openvswitch, you can see > it > in the > > >> >>> >>> (offending) nodes through "ovs-vsctl list . ". It'll be a > key > in the > > >> >>> >>> 'external_ids' column called 'ovn-remote' . > > >> >>> >>> > > >> >>> >>> This is not the solution, but a work-around; you could try > to > > >> >>> >>> configure the ovn controllers via: > > >> >>> >>> vdsm-tool ovn-config <engine_ip_on_net> <name of the > management network> > > >> >>> >>> > > >> >>> >>> Despite the provided work-around, I really think the hosted > engine > > >> >>> >>> should have triggered the ansible role that in turn > triggers > this > > >> >>> >>> reconfiguration. > > >> >>> >>> > > >> >>> >>> Would you open a bug with this information ? > > >> >>> >>> > > >> >>> >>> > > >> >>> >>> > > > >> >>> >>> > The routing table for one of our host look like this > > >> >>> >>> > > > >> >>> >>> > estination Gateway Genmask Flags > Metric > Ref Use Iface > > >> >>> >>> > default gateway 0.0.0.0 UG 0 > 0 0 ovirtmgmt > > >> >>> >>> > 10.16.248.0 0.0.0.0 255.255.255.0 U 0 > 0 0 ovirtmgmt > > >> >>> >>> > link-local 0.0.0.0 255.255.0.0 U > 1002 > 0 0 eno1 > > >> >>> >>> > link-local 0.0.0.0 255.255.0.0 U > 1003 > 0 0 eno2 > > >> >>> >>> > link-local 0.0.0.0 255.255.0.0 U > 1025 > 0 0 ovirtmgmt > > >> >>> >>> > > > >> >>> >>> > Any help would be really appreciated. > > >> >>> >>> > > > >> >>> >>> > Regards > > >> >>> >>> > Carl > > >> >>> >>> > > > >> >>> >>> > > > >> >>> >>> > > > >> >>> >>> > > > >> >>> >>> > _______________________________________________ > > >> >>> >>> > Users mailing list -- users(a)ovirt.org > > >> >>> >>> > To unsubscribe send an email to users-leave(a)ovirt.org > > >> >>> >>> > Privacy Statement: > https://www.ovirt.org/site/privacy-policy/ > > >> >>> >>> > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > > >> >>> >>> > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/DBQUWEPPDK2... > _______________________________________________ Users mailing list -- users(a)ovirt.org To unsubscribe send an email to users-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/UB72PHIP2FO...