New oVirt Node 4.0.6.1 network problem with bond and many vlans - Network.service timeout failed
Hi oVirt admins, Yesterday, i install a new node 4.0.6.1 and after config network bond (1) + vlans ( about 50 ), network.service failed with timeout as above. All my old and working fine oVirt nodes run 4.0.5 and have the same 5 min timeout on systemd. When i try to run vms on this new server, some networks are ok and some not. I install oVirt Node with 4.0.3 on same server and this problem do not occur. How can i workaround or change timeout to network.service ? In time, there are any way to update oVirt Node to 4.0.5 from 4.0.3 install iso ? [root@prd-rbs-ovirt-kvm21-poa ~]# systemctl status network.service â— network.service - LSB: Bring up/down networking Loaded: loaded (/etc/rc.d/init.d/network; bad; vendor preset: disabled) Active: failed (Result: timeout) since Thu 2017-05-04 11:29:43 BRT; 3h 12min ago Docs: man:systemd-sysv-generator(8) Process: 7574 ExecStart=/etc/rc.d/init.d/network start (code=killed, signal=TERM) May 04 11:29:12 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[7574]: Bringing up interface o_p_s_a_lb_758: [ OK ] May 04 11:29:18 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[7574]: Bringing up interface o_p_sites_a_755: [ OK ] May 04 11:29:23 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[7574]: Bringing up interface o_prod_app_757: [ OK ] May 04 11:29:28 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[7574]: Bringing up interface o_prod_db_756: [ OK ] May 04 11:29:38 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[7574]: Bringing up interface ovirtmgmt: [ OK ] *May 04 11:29:43 prd-rbs-ovirt-kvm21-poa.rbs.com.br <http://prd-rbs-ovirt-kvm21-poa.rbs.com.br> systemd[1]: network.service start operation timed out. Terminating.* May 04 11:29:43 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]: Failed to start LSB: Bring up/down networking. May 04 11:29:43 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]: Unit network.service entered failed state. May 04 11:29:43 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]: network.service failed. May 04 11:29:43 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[7574]: Bringing up interface paas_1305: [root@prd-rbs-ovirt-kvm21-poa ~]# [root@prd-rbs-ovirt-kvm21-poa ~]# systemctl restart network.service Job for network.service failed because a timeout was exceeded. See "systemctl status network.service" and "journalctl -xe" for details. [root@prd-rbs-ovirt-kvm21-poa ~]# systemctl status network.service â— network.service - LSB: Bring up/down networking Loaded: loaded (/etc/rc.d/init.d/network; bad; vendor preset: disabled) Active: failed (Result: timeout) since Thu 2017-05-04 14:55:34 BRT; 4min 46s ago Docs: man:systemd-sysv-generator(8) Process: 40810 ExecStart=/etc/rc.d/init.d/network start (code=killed, signal=TERM) May 04 14:55:21 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[40810]: [ OK ] May 04 14:55:23 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[40810]: Bringing up interface o_hlg_db_766: RTNETLINK answers: File exists May 04 14:55:26 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[40810]: [ OK ] May 04 14:55:28 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[40810]: Bringing up interface o_p_s_a_lb_758: RTNETLINK answers: File exists May 04 14:55:31 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[40810]: [ OK ] May 04 14:55:33 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[40810]: Bringing up interface o_p_sites_a_755: RTNETLINK answers: File exists May 04 14:55:34 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]: network.service start operation timed out. Terminating. May 04 14:55:34 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]: Failed to start LSB: Bring up/down networking. May 04 14:55:34 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]: Unit network.service entered failed state. May 04 14:55:34 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]: network.service failed. [root@prd-rbs-ovirt-kvm21-poa ~]#
Hi again oVirt Admins, After dig some time, i adjust network.service (/etc/rc.d/init.d/network) timeout to zero creating a custom timeout.conf file for systemctl. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=818676 https://www.turnkeylinux.org/blog/debugging-systemd-sysv-init-compat https://unix.stackexchange.com/questions/233468/how-does-systemd-use-etc-ini... [root@prd-rbs-ovirt-kvm21-poa system]# systemctl show network.service | grep -i timeout TimeoutStartUSec=5min TimeoutStopUSec=5min Result=timeout JobTimeoutUSec=0 JobTimeoutAction=none [root@prd-rbs-ovirt-kvm21-poa system]# pwd /usr/lib/systemd/system [root@prd-rbs-ovirt-kvm21-poa system]# mkdir network.service.d [root@prd-rbs-ovirt-kvm21-poa system]# cd network.service.d/ [root@prd-rbs-ovirt-kvm21-poa network.service.d]# vi timeout.conf [Service] TimeoutSec=0 TimeoutStartUSec=0 [root@prd-rbs-ovirt-kvm21-poa system]# systemctl daemon-reload [root@prd-rbs-ovirt-kvm21-poa system]# systemctl show network.service | grep -i timeout TimeoutStartUSec=0 TimeoutStopUSec=0 Result=timeout DropInPaths=/usr/lib/systemd/system/network.service.d/timeout.conf JobTimeoutUSec=0 JobTimeoutAction=none [root@prd-rbs-ovirt-kvm21-poa system]# [root@prd-rbs-ovirt-kvm21-poa ~]# systemctl restart network.service [root@prd-rbs-ovirt-kvm21-poa ~]# systemctl status network.service â— network.service - LSB: Bring up/down networking Loaded: loaded (/etc/rc.d/init.d/network; bad; vendor preset: disabled) Drop-In: /usr/lib/systemd/system/network.service.d └─timeout.conf Active: active (exited) since Thu 2017-05-04 15:22:12 BRT; 5min ago Docs: man:systemd-sysv-generator(8) Process: 345 ExecStart=/etc/rc.d/init.d/network start (code=exited, status=0/SUCCESS) May 04 15:21:39 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[345]: Bringing up interface rede_5: [ OK ] May 04 15:21:44 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[345]: Bringing up interface rede_6: [ OK ] May 04 15:21:48 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[345]: Bringing up interface server_lb_7: [ OK ] May 04 15:21:50 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[345]: Bringing up interface srv_1056: RTNETLINK answers: File exists May 04 15:21:53 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[345]: [ OK ] May 04 15:21:58 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[345]: Bringing up interface streaming_106: [ OK ] May 04 15:22:02 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[345]: Bringing up interface tef_100: [ OK ] May 04 15:22:07 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[345]: Bringing up interface telecom_25: [ OK ] May 04 15:22:12 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[345]: Bringing up interface zh_32: [ OK ] May 04 15:22:12 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]: Started LSB: Bring up/down networking. [root@prd-rbs-ovirt-kvm21-poa ~]# Em qui, 4 de mai de 2017 às 15:01, Rogério Ceni Coelho < rogeriocenicoelho@gmail.com> escreveu:
Hi oVirt admins,
Yesterday, i install a new node 4.0.6.1 and after config network bond (1) + vlans ( about 50 ), network.service failed with timeout as above. All my old and working fine oVirt nodes run 4.0.5 and have the same 5 min timeout on systemd.
When i try to run vms on this new server, some networks are ok and some not.
I install oVirt Node with 4.0.3 on same server and this problem do not occur.
How can i workaround or change timeout to network.service ?
In time, there are any way to update oVirt Node to 4.0.5 from 4.0.3 install iso ?
[root@prd-rbs-ovirt-kvm21-poa ~]# systemctl status network.service â— network.service - LSB: Bring up/down networking Loaded: loaded (/etc/rc.d/init.d/network; bad; vendor preset: disabled) Active: failed (Result: timeout) since Thu 2017-05-04 11:29:43 BRT; 3h 12min ago Docs: man:systemd-sysv-generator(8) Process: 7574 ExecStart=/etc/rc.d/init.d/network start (code=killed, signal=TERM)
May 04 11:29:12 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[7574]: Bringing up interface o_p_s_a_lb_758: [ OK ] May 04 11:29:18 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[7574]: Bringing up interface o_p_sites_a_755: [ OK ] May 04 11:29:23 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[7574]: Bringing up interface o_prod_app_757: [ OK ] May 04 11:29:28 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[7574]: Bringing up interface o_prod_db_756: [ OK ] May 04 11:29:38 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[7574]: Bringing up interface ovirtmgmt: [ OK ] *May 04 11:29:43 prd-rbs-ovirt-kvm21-poa.rbs.com.br <http://prd-rbs-ovirt-kvm21-poa.rbs.com.br> systemd[1]: network.service start operation timed out. Terminating.* May 04 11:29:43 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]: Failed to start LSB: Bring up/down networking. May 04 11:29:43 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]: Unit network.service entered failed state. May 04 11:29:43 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]: network.service failed. May 04 11:29:43 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[7574]: Bringing up interface paas_1305: [root@prd-rbs-ovirt-kvm21-poa ~]#
[root@prd-rbs-ovirt-kvm21-poa ~]# systemctl restart network.service Job for network.service failed because a timeout was exceeded. See "systemctl status network.service" and "journalctl -xe" for details. [root@prd-rbs-ovirt-kvm21-poa ~]# systemctl status network.service â— network.service - LSB: Bring up/down networking Loaded: loaded (/etc/rc.d/init.d/network; bad; vendor preset: disabled) Active: failed (Result: timeout) since Thu 2017-05-04 14:55:34 BRT; 4min 46s ago Docs: man:systemd-sysv-generator(8) Process: 40810 ExecStart=/etc/rc.d/init.d/network start (code=killed, signal=TERM)
May 04 14:55:21 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[40810]: [ OK ] May 04 14:55:23 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[40810]: Bringing up interface o_hlg_db_766: RTNETLINK answers: File exists May 04 14:55:26 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[40810]: [ OK ] May 04 14:55:28 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[40810]: Bringing up interface o_p_s_a_lb_758: RTNETLINK answers: File exists May 04 14:55:31 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[40810]: [ OK ] May 04 14:55:33 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[40810]: Bringing up interface o_p_sites_a_755: RTNETLINK answers: File exists May 04 14:55:34 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]: network.service start operation timed out. Terminating. May 04 14:55:34 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]: Failed to start LSB: Bring up/down networking. May 04 14:55:34 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]: Unit network.service entered failed state. May 04 14:55:34 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]: network.service failed. [root@prd-rbs-ovirt-kvm21-poa ~]#
After reboot : [root@prd-rbs-ovirt-kvm21-poa ~]# systemctl --failed 0 loaded units listed. Pass --all to see loaded but inactive units, too. To show all installed unit files use 'systemctl list-unit-files'. [root@prd-rbs-ovirt-kvm21-poa ~]# systemctl status network.service ◠network.service - LSB: Bring up/down networking Loaded: loaded (/etc/rc.d/init.d/network; bad; vendor preset: disabled) Drop-In: /usr/lib/systemd/system/network.service.d └─timeout.conf Active: active (exited) since Thu 2017-05-04 15:41:34 BRT; 1min 50s ago Docs: man:systemd-sysv-generator(8) Process: 6886 ExecStart=/etc/rc.d/init.d/network start (code=exited, status=0/SUCCESS) May 04 15:40:47 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[6886]: Bringing up interface rbs_farm_200: [ OK ] May 04 15:40:53 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[6886]: Bringing up interface rede_5: [ OK ] May 04 15:40:58 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[6886]: Bringing up interface rede_6: [ OK ] May 04 15:41:04 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[6886]: Bringing up interface server_lb_7: [ OK ] May 04 15:41:13 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[6886]: Bringing up interface srv_1056: [ OK ] May 04 15:41:18 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[6886]: Bringing up interface streaming_106: [ OK ] May 04 15:41:23 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[6886]: Bringing up interface tef_100: [ OK ] May 04 15:41:28 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[6886]: Bringing up interface telecom_25: [ OK ] May 04 15:41:34 prd-rbs-ovirt-kvm21-poa.rbs.com.br network[6886]: Bringing up interface zh_32: [ OK ] May 04 15:41:34 prd-rbs-ovirt-kvm21-poa.rbs.com.br systemd[1]: Started LSB: Bring up/down networking. [root@prd-rbs-ovirt-kvm21-poa ~]#
participants (1)
-
Rogério Ceni Coelho