April 2022 - Users - oVirt List Archives

API how to increase extend resize disk VM
by ovirt.org＠nevim.eu 25 Jun '22

25 Jun '22

Hello, please how to increase extend resize disk of VM? I can working with ansible or REST API. Ansible working is here, but i not found manual for update size: https://docs.ansible.com/ansible/2.3/ovirt_disks_module.html On official ovirt documentation i cant found how to update. I found only old manual in KB page, but not working on lastest: https://www.ovirt.org/develop/release-management/features/storage/online-vi… PUT /api/vms/{VM_ID}/disks/{DISK_ID} HTTP/1.1 Accept: application/xml Content-type: application/xml <disk> <size>{NEW_SIZE_IN_BYTES}</size> </disk> Thanks for advice.

5 13

failed to mount hosted engine gluster storage - how to debug?
by diego.ercolani＠ssis.sm 21 Jun '22

21 Jun '22

Hello, I have an issue probably related to my particular implementation but I think some controls are missing Here is the story. I have a cluster of two nodes 4.4.10.3 with an upgraded kernel as the cpu (Ryzen 5) suffer from an incompatibility issue with the kernel provided by 4.4.10.x series. On each node there are three glusterfs "partitions" in replica mode, one for the hosted_engine, the other two are for user usage. The third node was an old i3 workstation only used to provide the arbiter partition to the glusterfs cluster. I installed a new server (ryzen processor) with 4.5.0 and successfully installed glusterfs 10.1 and inserted the arbiter bricks implemented on glusterfs 10.1 while the replica bricks are 8.6 after removing the old i3 provided bricks. I successfully imported the new node in the ovirt engine (after updating the engine to 4.5) The problem is that the ovirt-ha-broker doesn't start complaining that is not possible to connect the storage. (I suppose the hosted_engine storage) so I did some digs that I'm going to show here: #### 1. The node seem to be correctly configured: [root@ovirt-node3 devices]# vdsm-tool validate-config SUCCESS: ssl configured to true. No conflicts [root@ovirt-node3 devices]# vdsm-tool configure Checking configuration status... libvirt is already configured for vdsm SUCCESS: ssl configured to true. No conflicts sanlock is configured for vdsm Managed volume database is already configured lvm is configured for vdsm Current revision of multipath.conf detected, preserving Running configure... Done configuring modules to VDSM. [root@ovirt-node3 devices]# vdsm-tool validate-config SUCCESS: ssl configured to true. No conflicts #### 2. the node refuses to mount via hosted-engine (same error in broker.log) [root@ovirt-node3 devices]# hosted-engine --connect-storage Traceback (most recent call last): File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_setup/connect_storage_server.py", line 30, in <module> timeout=ohostedcons.Const.STORAGE_SERVER_TIMEOUT, File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/client/client.py", line 312, in connect_storage_server sserver.connect_storage_server(timeout=timeout) File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/storage_server.py", line 451, in connect_storage_server 'Connection to storage server failed' RuntimeError: Connection to storage server failed ##### 3. manually mount of glusterfs work correctly [root@ovirt-node3 devices]# grep storage /etc/ovirt-hosted-engine/hosted-engine.conf storage=ovirt-node2.ovirt:/gveng # The following are used only for iSCSI storage [root@ovirt-node3 devices]# [root@ovirt-node3 devices]# mount -t glusterfs ovirt-node2.ovirt:/gveng /mnt/tmp/ [root@ovirt-node3 devices]# ls -l /mnt/tmp total 0 drwxr-xr-x. 6 vdsm kvm 64 Dec 15 19:04 7b8f1cc9-e3de-401f-b97f-8c281ca30482 What else should I control? Thank you and sorry for the long message Diego

5 9

After attaching the Storage domain, the VMs are disappeared from the VM import
by aminur.rahman＠iongroup.com 20 Jun '22

20 Jun '22

Hi, We're noticing some weird issue while re-attaching the storage domain. After re-attach the storage domain, some VMs are completely missing from the VM Import. Before detaching the storage domain, all the VMs were shutdown gracefully. I also noticed some disks are exists with no Alias under the disk import on the storage domain and I can't import those disks. Its failed to register the disk with <UNKONOWN> error. We're using Ovirt 4.2 with multiple Dell hosts in the cluster and Compellent SAN with iSCSI volumes. Please kindly advise if I am missing anything before detach the storage domain. Thanks

4 4

Cannot log into oVirt Manager - certificate issue
by Diggy Mc 18 Jun '22

18 Jun '22

I cannot log into oVirt Manager. My browser gave me a warning that the site's certificate has expired. Then when I try to log in, I receive the following error message: "PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed" How can I fix this problem? In advance, thank you for your help. hosted-engine: v4.4.8.6 hosts: oVirt Node v4.4.8.3

7 24

can't use vmconsole anymore
by Nathanaël Blanchet 13 Jun '22

13 Jun '22

Hi, I was used to use the vmconsole proxy, but since a while, I'm getting this issue (currently 4.4.5): # ssh -t -p 2222 ovirt-vmconsole(a)air.v100.abes.fr connect ovirt-vmconsole(a)air.v100.abes.fr: Permission denied (publickey). I found following in the engine.log 2021-04-15 17:55:43,094+02 ERROR [org.ovirt.engine.core.services.VMConsoleProxyServlet] (default task-4) [] Error validating ticket: : sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141) at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126) at java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297) at org.ovirt.engine.core.uutils//org.ovirt.engine.core.uutils.crypto.CertificateChain.buildCertPath(CertificateChain.java:128) at org.ovirt.engine.core.uutils//org.ovirt.engine.core.uutils.crypto.ticket.TicketDecoder.decode(TicketDecoder.java:89) at deployment.engine.ear.services.war//org.ovirt.engine.core.services.VMConsoleProxyServlet.validateTicket(VMConsoleProxyServlet.java:175) at deployment.engine.ear.services.war//org.ovirt.engine.core.services.VMConsoleProxyServlet.doPost(VMConsoleProxyServlet.java:225) The user key is the good one, I use the same with my other engines and I can successfully connect to vm consoles. Thank you for helping -- Nathanaël Blanchet Supervision réseau SIRE 227 avenue Professeur-Jean-Louis-Viala 34193 MONTPELLIER CEDEX 5 Tél. 33 (0)4 67 54 84 55 Fax 33 (0)4 67 54 84 14 blanchet(a)abes.fr

5 16

looking for some ISV backup software which integrates the backup API
by Nathanaël Blanchet 08 Jun '22

08 Jun '22

Hello, We are about to change our backup provider, and I find it is a great chance to choose a full supported ovirt backup solution. I currently use this python script vm-backup-scheduler (https://github.com/wefixit-AT/oVirtBackup) but it is not the workflow officially suggested by the community (https://www.ovirt.org/develop/release-management/features/storage/backup-re…) I've been looking for a long time an ISV who supports such an API, but the only one I found is this one : Acronis Backup Advanced suggested here https://access.redhat.com/ecosystem/search/#/ecosystem/Red%20Hat%20Enterpri… I ran the trial version, but it doesn't seem to do better than the vm-backup-scheduler script, and it doesn't seem to use the backup API (attach a clone as a disk to an existing vm). Can you suggest me some other ISV solutions, if they ever exist... or share me your backup experience?

5 4

VM HostedEngine is down with error
by souvaliotimaria＠mail.com 26 May '22

26 May '22

Hello everyone, I have a replica 2 + arbiter installation and this morning the Hosted Engine gave the following error on the UI and resumed on a different node (node3) than the one it was originally running(node1). (The original node has more memory than the one it ended up, but it had a better memory usage percentage at the time). Also, the only way I discovered the migration had happened and there was an Error in Events, was because I logged in the web interface of ovirt for a routine inspection. Βesides that, everything was working properly and still is. The error that popped is the following: VM HostedEngine is down with error. Exit message: internal error: qemu unexpectedly closed the monitor: 2020-09-01T06:49:20.749126Z qemu-kvm: warning: All CPU(s) up to maxcpus should be described in NUMA config, ability to start up with partial NUMA mappings is obsoleted and will be removed in future 2020-09-01T06:49:20.927274Z qemu-kvm: -device virtio-blk-pci,iothread=iothread1,scsi=off,bus=pci.0,addr=0x7,drive=drive-ua-d5de54b6-9f8e-4fba-819b-ebf6780757d2,id=ua-d5de54b6-9f8e-4fba-819b-ebf6780757d2,bootindex=1,write-cache=on: Failed to get "write" lock Is another process using the image?. Which from what I could gather concerns the following snippet from the HostedEngine.xml and it's the virtio disk of the Hosted Engine: <disk type='file' device='disk' snapshot='no'> <driver name='qemu' type='raw' cache='none' error_policy='stop' io='threads' iothread='1'/> <source file='/var/run/vdsm/storage/80f6e393-9718-4738-a14a-64cf43c3d8c2/d5de54b6-9f8e-4fba-819b-ebf6780757d2/a48555f4-be23-4467-8a54-400ae7baf9d7'> <seclabel model='dac' relabel='no'/> </source> <target dev='vda' bus='virtio'/> <serial>d5de54b6-9f8e-4fba-819b-ebf6780757d2</serial> <alias name='ua-d5de54b6-9f8e-4fba-819b-ebf6780757d2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </disk> I've tried looking into the logs and the sar command but I couldn't find anything to relate with the above errors and determining the reason for it to happen. Is this a Gluster or a QEMU problem? The Hosted Engine was manually migrated five days before on node1. Is there a standard practice I could follow to determine what happened and secure my system? Thank you very much for your time, Maria Souvalioti

4 6

LACP across multiple switches
by Jorge Visentini 25 May '22

25 May '22

Hi all. Is it possible to configure oVirt for work with two NICs in bond/LACP across two switches, according to the image below? [image: LACP_Across_Two_Switchs.png] Thank you all. You guys do a wonderful job. -- Att, Jorge Visentini +55 55 98432-9868

5 5

new host addition to OVN cluster fails with Connectivity check failed, rolling back
by ravi k 19 May '22

19 May '22

Hello, I have a a oVirt 4.3 installation with two clusters. One of the cluster has switch type as OVS. I'm trying to add a second host to this cluster. I did a clean install of the OS, configured bond0 and bond0.2306 as the VLAN interface. I was able to add the host to the cluster. When I go to setup networks and drag ovirtmgmt onto bond0, I notice in the host that it was able to create the vdsm and br_int bridges, create ovirtmgmt interface and assign the IP on top of it. However I also notice that the bond0.2306 interface also exists with the ip assigned. Then it rolls back the config removing the bridges. I checked the supervdsm log and see that it's rolling back because "connectivity::48::root::(check) Connectivity check failed, rolling back" I'm pasting the relevant lines from supervdsm below MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:53,381::supervdsm_server::106::SuperVdsm.ServerCallback::(wrapper) return network_caps with {'bridges': {}, 'bondings': {'bond0': {'ipv4addrs': [], 'active_slave': '', 'ad_aggregator_id': '1', 'netmask': '', 'ad_partner_mac': '44:38:39:ff:01:33', 'hwaddr': '7c:d3:0a:60:e9:48', 'speed': 20000, 'gateway': '', 'ipv6autoconf': True, 'addr': '', 'dhcpv6': False, 'ipv6addrs': [], 'mtu': '9000', 'dhcpv4': False, 'switch': 'legacy', 'ipv4defaultroute': False, 'slaves': ['eno1', 'eno2'], 'ipv6gateway': '::', 'opts': {'mode': '4'}}}, 'nameservers': ['10.222.0.6', '10.333.0.6'], 'nics': {'eno1': {'permhwaddr': '7c:d3:0a:60:e9:48', 'ipv6autoconf': True, 'addr': '', 'speed': 10000, 'dhcpv6': False, 'ipv6addrs': [], 'ad_aggregator_id': '1', 'dhcpv4': False, 'netmask': '', 'ipv4defaultroute': False, 'ipv4addrs': [], 'hwaddr': '7c:d3:0a:60:e9:48', 'mtu': '9000', 'ipv6gateway': '::', 'gateway': ''}, 'eno2': {'permhwaddr': '7c:d3:0a:60:e9:49', 'ipv6autoco nf': True, 'addr': '', 'speed': 10000, 'dhcpv6': False, 'ipv6addrs': [], 'ad_aggregator_id': '1', 'dhcpv4': False, 'netmask': '', 'ipv4defaultroute': False, 'ipv4addrs': [], 'hwaddr': '7c:d3:0a:60:e9:48', 'mtu': '9000', 'ipv6gateway': '::', 'gateway': ''}, 'eno3': {'ipv6autoconf': True, 'addr': '', 'speed': 10000, 'dhcpv6': False, 'ipv6addrs': [], 'mtu': '1500', 'dhcpv4': False, 'netmask': '', 'ipv4defaultroute': False, 'ipv4addrs': [], 'hwaddr': '7c:d3:0a:60:e9:4a', 'ipv6gateway': '::', 'gateway': ''}, 'eno4': {'ipv6autoconf': True, 'addr': '', 'speed': 10000, 'dhcpv6': False, 'ipv6addrs': [], 'mtu': '1500', 'dhcpv4': False, 'netmask': '', 'ipv4defaultroute': False, 'ipv4addrs': [], 'hwaddr': '7c:d3:0a:60:e9:4b', 'ipv6gateway': '::', 'gateway': ''}, 'enp0s20f0u1u6': {'ipv6autoconf': True, 'addr': '', 'speed': 0, 'dhcpv6': False, 'ipv6addrs': [], 'mtu': '1500', 'dhcpv4': False, 'netmask': '', 'ipv4defaultroute': False, 'ipv4addrs': [], 'hwaddr': '7e:d3:0a:60:e9:4f', 'ipv6gateway': ' ::', 'gateway': ''}}, 'supportsIPv6': True, 'vlans': {'bond0.2306': {'iface': 'bond0', 'ipv6autoconf': True, 'addr': '10.119.6.237', 'dhcpv6': False, 'ipv6addrs': [], 'vlanid': 2306, 'mtu': '9000', 'dhcpv4': False, 'netmask': '255.255.255.0', 'ipv4defaultroute': True, 'ipv4addrs': ['10.119.6.237/24'], 'ipv6gateway': '::', 'gateway': '10.119.6.1'}}, 'networks': {}} MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,243::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper) call get_pti with () {} MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,243::supervdsm_server::106::SuperVdsm.ServerCallback::(wrapper) return get_pti with -1 MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,244::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper) call get_retp with () {} MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,244::supervdsm_server::106::SuperVdsm.ServerCallback::(wrapper) return get_retp with -1 MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,244::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper) call get_ibrs with () {} MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,245::supervdsm_server::106::SuperVdsm.ServerCallback::(wrapper) return get_ibrs with 1 MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,245::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper) call get_ssbd with () {} MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,245::supervdsm_server::106::SuperVdsm.ServerCallback::(wrapper) return get_ssbd with -1 MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,246::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper) call check_qemu_conf_contains with ('vnc_tls', '1') {} MainProcess|jsonrpc/1::DEBUG::2022-04-21 11:26:54,250::supervdsm_server::106::SuperVdsm.ServerCallback::(wrapper) return check_qemu_conf_contains with True MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,304::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper) call setupNetworks with ({u'ovirtmgmt': {u'ipv6autoconf': True, u'vlan': u'2306', u'ipaddr': u'10.119.6.237', u'switch': u'ovs', u'mtu': 9000, u'bonding': u'bond0', u'dhcpv6': False, u'STP': u'no', u'bridged': u'true', u'netmask': u'255.255.255.0', u'gateway': u'10.119.6.1', u'defaultRoute': True}}, {}, {u'connectivityCheck': u'true', u'connectivityTimeout': 120, u'commitOnSuccess': False}) {} MainProcess|jsonrpc/2::INFO::2022-04-21 11:26:55,305::api::211::root::(setupNetworks) Setting up network according to configuration: networks:{u'ovirtmgmt': {u'ipv6autoconf': True, u'vlan': u'2306', u'ipaddr': u'10.119.6.237', u'bonding': u'bond0', u'mtu': 9000, u'switch': u'ovs', u'dhcpv6': False, u'STP': u'no', u'bridged': u'true', u'netmask': u'255.255.255.0', u'gateway': u'10.119.6.1', u'defaultRoute': True}}, bondings:{}, options:{u'connectivityCheck': u'true', u'connectivityTimeout': 120, u'commitOnSuccess': False} MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,323::cmdutils::133::root::(exec_cmd) /sbin/tc qdisc show (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,342::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,408::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,408::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,445::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::INFO::2022-04-21 11:26:55,459::netconfpersistence::69::root::(setBonding) Adding bond0({'nics': ['eno1', 'eno2'], 'switch': 'legacy', 'options': 'mode=4'}) MainProcess|jsonrpc/2::INFO::2022-04-21 11:26:55,460::netconfpersistence::58::root::(setNetwork) Adding network ovirtmgmt({u'ipv6autoconf': True, 'nameservers': ['10.222.0.6', '10.333.0.6'], u'vlan': 2306, u'ipaddr': u'10.119.6.237', u'switch': u'ovs', u'mtu': 9000, u'bonding': u'bond0', u'dhcpv6': False, 'stp': False, u'bridged': True, u'netmask': u'255.255.255.0', u'gateway': u'10.119.6.1', u'defaultRoute': True, 'bootproto': 'none'}) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,462::commands::198::root::(execCmd) /usr/bin/taskset --cpu-list 0-95 /usr/libexec/vdsm/hooks/before_network_setup/50_fcoe (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,909::commands::219::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::INFO::2022-04-21 11:26:55,909::hooks::114::root::(_runHooksDir) /usr/libexec/vdsm/hooks/before_network_setup/50_fcoe: rc=0 err= MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,911::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,911::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,947::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,953::configurator::265::root::(_remove_networks) Removing networks: [] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,955::setup::41::root::(remove_bonds) Removing bonds: [] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,958::ifacquire::70::root::(acquire) Acquiring ifaces: set([u'eno1', u'eno2', u'bond0']) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:55,959::cmdutils::133::root::(exec_cmd) /sbin/ifdown eno1 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:56,606::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:56,609::cmdutils::133::root::(exec_cmd) /sbin/ifdown eno2 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:57,268::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:57,271::cmdutils::133::root::(exec_cmd) /sbin/ifdown bond0 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:58,940::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:58,940::setup::61::root::(edit_bonds) Editing bonds: [u'bond0'] MainProcess|jsonrpc/2::INFO::2022-04-21 11:26:58,946::netconfpersistence::69::root::(setBonding) Adding bond0({'nics': [], 'switch': u'ovs', 'options': {u'mode': '4'}}) MainProcess|jsonrpc/2::INFO::2022-04-21 11:26:58,949::sysfs_driver::104::root::(set_options) Bond bond0 options set: {u'mode': u'4'}. MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:58,950::cmdutils::133::root::(exec_cmd) /sbin/ip link set dev eno1 down (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:58,956::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::INFO::2022-04-21 11:26:59,000::sysfs_driver::78::root::(add_slaves) Slave eno1 has been added to bond bond0. MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,001::cmdutils::133::root::(exec_cmd) /sbin/ip link set dev eno2 down (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,009::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::INFO::2022-04-21 11:26:59,053::sysfs_driver::78::root::(add_slaves) Slave eno2 has been added to bond bond0. MainProcess|jsonrpc/2::INFO::2022-04-21 11:26:59,053::netconfpersistence::69::root::(setBonding) Adding bond0({u'hwaddr': u'7c:d3:0a:60:e9:48', u'nics': [u'eno1', u'eno2'], u'switch': u'ovs', u'options': u'mode=4'}) netlink/events::DEBUG::2022-04-21 11:26:59,055::concurrent::258::root::(run) START thread <Thread(netlink/events, started daemon 140363790243584)> (func=<bound method Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at 0x7fa8fe64d610>>, args=(), kwargs={}) netlink/events::DEBUG::2022-04-21 11:26:59,057::concurrent::258::root::(run) START thread <Thread(netlink/events, started daemon 140363781850880)> (func=<bound method Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at 0x7fa8fe5e9550>>, args=(), kwargs={}) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,058::cmdutils::133::root::(exec_cmd) /sbin/ip link set dev bond0 up (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,067::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 netlink/events::DEBUG::2022-04-21 11:26:59,071::concurrent::261::root::(run) FINISH thread <Thread(netlink/events, stopped daemon 140363781850880)> netlink/events::DEBUG::2022-04-21 11:26:59,073::concurrent::258::root::(run) START thread <Thread(netlink/events, started daemon 140363781850880)> (func=<bound method Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at 0x7fa8fe5e91d0>>, args=(), kwargs={}) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,074::cmdutils::133::root::(exec_cmd) /sbin/ip link set dev eno1 up (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,081::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 netlink/events::DEBUG::2022-04-21 11:26:59,083::concurrent::261::root::(run) FINISH thread <Thread(netlink/events, stopped daemon 140363781850880)> netlink/events::DEBUG::2022-04-21 11:26:59,084::concurrent::258::root::(run) START thread <Thread(netlink/events, started daemon 140363781850880)> (func=<bound method Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at 0x7fa8fe5e91d0>>, args=(), kwargs={}) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,085::cmdutils::133::root::(exec_cmd) /sbin/ip link set dev eno2 up (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,090::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 netlink/events::DEBUG::2022-04-21 11:26:59,092::concurrent::261::root::(run) FINISH thread <Thread(netlink/events, stopped daemon 140363781850880)> netlink/events::DEBUG::2022-04-21 11:26:59,093::concurrent::261::root::(run) FINISH thread <Thread(netlink/events, stopped daemon 140363790243584)> MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,219::cmdutils::133::root::(exec_cmd) /sbin/ip addr flush dev eno1 scope global (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,225::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,340::cmdutils::133::root::(exec_cmd) /sbin/ip addr flush dev eno2 scope global (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,346::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,346::setup::92::root::(add_bonds) Creating bonds: [] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,346::configurator::274::root::(_add_networks) Adding networks: [u'ovirtmgmt'] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,347::ifacquire::70::root::(acquire) Acquiring ifaces: set([u'bond0']) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:26:59,348::cmdutils::133::root::(exec_cmd) /sbin/ifdown eno1 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:00,004::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:00,006::cmdutils::133::root::(exec_cmd) /sbin/ifdown eno2 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:00,657::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:00,660::cmdutils::133::root::(exec_cmd) /sbin/ifdown bond0 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,360::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,361::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- add-br vdsmbr_TtLDFxN1 -- set Bridge vdsmbr_TtLDFxN1 other-config:hwaddr="7a:b2:a6:5b:96:78" -- add-port vdsmbr_TtLDFxN1 bond0 -- set Port bond0 other_config:vdsm_level=southbound -- add-port vdsmbr_TtLDFxN1 ovirtmgmt -- set Port ovirtmgmt other_config:vdsm_level=northbound -- set Interface ovirtmgmt type=internal -- set Port ovirtmgmt tag=2306 -- set Interface ovirtmgmt mtu_request=9000 -- set Interface bond0 mtu_request=9000 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,361::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- add-br vdsmbr_TtLDFxN1 -- set Bridge vdsmbr_TtLDFxN1 'other-config:hwaddr="7a:b2:a6:5b:96:78"' -- add-port vdsmbr_TtLDFxN1 bond0 -- set Port bond0 other_config:vdsm_level=southbound -- add-port vdsmbr_TtLDFxN1 ovirtmgmt -- set Port ovirtmgmt other_config:vdsm_level=northbound -- set Interface ovirtmgmt type=internal -- set Port ovirtmgmt tag=2306 -- set Interface ovirtmgmt mtu_request=9000 -- set Interface bond0 mtu_request=9000 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,704::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::INFO::2022-04-21 11:27:02,705::netconfpersistence::58::root::(setNetwork) Adding network ovirtmgmt({u'ipv6autoconf': True, u'nameservers': [u'10.129.0.60', u'10.229.0.60'], u'vlan': 2306, u'ipaddr': u'10.119.6.237', u'bonding': u'bond0', u'mtu': 9000, u'switch': u'ovs', u'dhcpv6': False, u'stp': False, u'bridged': True, u'netmask': u'255.255.255.0', u'gateway': u'10.119.6.1', u'defaultRoute': True, u'bootproto': u'none'}) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,705::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,706::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,745::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,753::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- set open . external-ids:ovn-bridge-mappings="ovirtmgmt:vdsmbr_TtLDFxN1" MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,753::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- set open . 'external-ids:ovn-bridge-mappings="ovirtmgmt:vdsmbr_TtLDFxN1"' (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,794::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 netlink/events::DEBUG::2022-04-21 11:27:02,796::concurrent::258::root::(run) START thread <Thread(netlink/events, started daemon 140363790243584)> (func=<bound method Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at 0x7fa8fe64d590>>, args=(), kwargs={}) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,797::cmdutils::133::root::(exec_cmd) /sbin/ip link set dev ovirtmgmt up (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,805::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 netlink/events::DEBUG::2022-04-21 11:27:02,807::concurrent::261::root::(run) FINISH thread <Thread(netlink/events, stopped daemon 140363790243584)> netlink/events::DEBUG::2022-04-21 11:27:02,808::concurrent::258::root::(run) START thread <Thread(netlink/events, started daemon 140363790243584)> (func=<bound method Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at 0x7fa8fe64d590>>, args=(), kwargs={}) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,809::cmdutils::133::root::(exec_cmd) /sbin/ip link set dev bond0 up (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,816::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 netlink/events::DEBUG::2022-04-21 11:27:02,819::concurrent::261::root::(run) FINISH thread <Thread(netlink/events, stopped daemon 140363790243584)> netlink/events::DEBUG::2022-04-21 11:27:02,820::concurrent::258::root::(run) START thread <Thread(netlink/events, started daemon 140363790243584)> (func=<bound method Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at 0x7fa8fe64d590>>, args=(), kwargs={}) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,821::cmdutils::133::root::(exec_cmd) /sbin/ip link set dev eno1 up (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,867::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 netlink/events::DEBUG::2022-04-21 11:27:02,870::concurrent::261::root::(run) FINISH thread <Thread(netlink/events, stopped daemon 140363790243584)> netlink/events::DEBUG::2022-04-21 11:27:02,871::concurrent::258::root::(run) START thread <Thread(netlink/events, started daemon 140363790243584)> (func=<bound method Monitor._scan of <vdsm.network.netlink.monitor.Monitor object at 0x7fa8fe64d590>>, args=(), kwargs={}) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,872::cmdutils::133::root::(exec_cmd) /sbin/ip link set dev eno2 up (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,920::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 netlink/events::DEBUG::2022-04-21 11:27:02,922::concurrent::261::root::(run) FINISH thread <Thread(netlink/events, stopped daemon 140363790243584)> MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,923::cmdutils::133::root::(exec_cmd) /sbin/ip addr flush dev ovirtmgmt scope global (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,930::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,931::cmdutils::133::root::(exec_cmd) /sbin/ip -4 addr add dev ovirtmgmt 10.119.6.237/255.255.255.0 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,938::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,938::cmdutils::133::root::(exec_cmd) /sbin/ip -4 route add default via 10.119.6.1 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,945::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:27:02,947::connectivity::46::root::(check) Checking connectivity... MainProcess|jsonrpc/3::DEBUG::2022-04-21 11:27:10,538::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper) call ksmTune with ({u'run': 0, u'merge_across_nodes': 1},) {} MainProcess|jsonrpc/3::DEBUG::2022-04-21 11:27:10,538::supervdsm_server::106::SuperVdsm.ServerCallback::(wrapper) return ksmTune with None MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:04,075::connectivity::48::root::(check) Connectivity check failed, rolling back MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:04,075::logutils::319::root::(_report_stats) ThreadedHandler is ok in the last 136 seconds (max pending: 3) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:04,075::ifacquire::76::root::(_rollback) Acquiring transaction failed, reverting ifaces: [u'eno1', u'eno2', u'bond0'] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:04,077::cmdutils::133::root::(exec_cmd) /sbin/ifup eno1 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:04,485::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:04,488::cmdutils::133::root::(exec_cmd) /sbin/ifup eno2 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:04,897::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:04,900::cmdutils::133::root::(exec_cmd) /sbin/ifup bond0 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,045::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = './network-functions: line 119: .: /etc/sysconfig/network-scripts/: is a directory\n'; <rc> = 0 MainProcess|jsonrpc/2::WARNING::2022-04-21 11:29:06,046::netconfpersistence::294::root::(__exit__) Failed setup transaction,reverting to last known good network. Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/network/netswitch/configurator.py", line 249, in _setup_ovs connectivity.check(options) File "/usr/lib/python2.7/site-packages/vdsm/network/connectivity.py", line 50, in check 'connectivity check failed') ConfigNetworkError: (10, 'connectivity check failed') MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:06,047::api::211::root::(setupNetworks) Setting up network according to configuration: networks:{u'ovirtmgmt': {'remove': True}}, bondings:{u'bond0': {'remove': True}}, options:{'connectivityCheck': 0, 'inRollback': True} MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,061::cmdutils::133::root::(exec_cmd) /sbin/tc qdisc show (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,079::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,153::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,153::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,190::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:06,326::netconfpersistence::58::root::(setNetwork) Adding network ovirtmgmt({'bonding': u'bond0', 'ipv6autoconf': True, 'nameservers': [], 'vlan': 2306, 'ipaddr': '10.119.6.237', 'switch': 'ovs', 'mtu': 9000, 'netmask': '255.255.255.0', 'dhcpv6': False, 'stp': False, 'bridged': True, 'defaultRoute': False, 'bootproto': 'none'}) MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:06,327::netconfpersistence::69::root::(setBonding) Adding bond0({'nics': ['eno1', 'eno2'], 'switch': 'legacy', 'options': 'mode=4'}) MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:06,327::netconfpersistence::63::root::(removeNetwork) Removing network ovirtmgmt MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,334::commands::198::root::(execCmd) /usr/bin/taskset --cpu-list 0-95 /usr/libexec/vdsm/hooks/before_network_setup/50_fcoe (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,911::commands::219::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:06,911::hooks::114::root::(_runHooksDir) /usr/libexec/vdsm/hooks/before_network_setup/50_fcoe: rc=0 err= MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,918::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,918::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:06,956::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:07,091::configurator::265::root::(_remove_networks) Removing networks: [u'ovirtmgmt'] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:07,219::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- set Interface bond0 mtu_request=1500 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:07,220::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- set Interface bond0 mtu_request=1500 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:07,425::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:07,426::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- del-port ovirtmgmt -- del-port bond0 -- del-br vdsmbr_TtLDFxN1 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:07,426::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- del-port ovirtmgmt -- del-port bond0 -- del-br vdsmbr_TtLDFxN1 (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:07,949::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:07,950::netconfpersistence::65::root::(removeNetwork) Network ovirtmgmt not found for removal MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:07,951::setup::41::root::(remove_bonds) Removing bonds: [u'bond0'] MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:08,165::sysfs_driver::69::root::(destroy) Bond bond0 has been destroyed. MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,166::netconfpersistence::76::root::(removeBonding) bond0 not found for removal MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,172::ifacquire::70::root::(acquire) Acquiring ifaces: set([]) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,172::setup::61::root::(edit_bonds) Editing bonds: [] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,172::setup::92::root::(add_bonds) Creating bonds: [] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,172::configurator::274::root::(_add_networks) Adding networks: [] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,172::ifacquire::70::root::(acquire) Acquiring ifaces: set([]) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,173::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,173::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list Bridge -- list Port -- list Interface (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,211::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,217::vsctl::68::root::(commit) Executing commands: /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- set open . external-ids:ovn-bridge-mappings="" MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,217::cmdutils::133::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- set open . 'external-ids:ovn-bridge-mappings=""' (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,247::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:08,249::netconfpersistence::231::root::(_clearDisk) Clearing netconf: /var/lib/vdsm/staging/netconf MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:08,253::netconfpersistence::181::root::(save) Saved new config RunningConfig({}, {}, {}) to [/var/lib/vdsm/staging/netconf/nets,/var/lib/vdsm/staging/netconf/bonds,/var/lib/vdsm/staging/netconf/devices] MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,256::commands::198::root::(execCmd) /usr/bin/taskset --cpu-list 0-95 /usr/libexec/vdsm/hooks/after_network_setup/30_ethtool_options (cwd None) MainProcess|jsonrpc/2::DEBUG::2022-04-21 11:29:08,434::commands::219::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 MainProcess|jsonrpc/2::INFO::2022-04-21 11:29:08,434::hooks::114::root::(_runHooksDir) /usr/libexec/vdsm/hooks/after_network_setup/30_ethtool_options: rc=0 err= MainProcess|jsonrpc/2::ERROR::2022-04-21 11:29:08,436::supervdsm_server::103::SuperVdsm.ServerCallback::(wrapper) Error in setupNetworks Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/supervdsm_server.py", line 101, in wrapper res = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/network/api.py", line 228, in setupNetworks _setup_networks(networks, bondings, options, net_info) File "/usr/lib/python2.7/site-packages/vdsm/network/api.py", line 249, in _setup_networks networks, bondings, options, net_info, in_rollback) File "/usr/lib64/python2.7/contextlib.py", line 35, in __exit__ self.gen.throw(type, value, traceback) File "/usr/lib/python2.7/site-packages/vdsm/network/api.py", line 157, in _rollback six.reraise(excType, value, tb) File "/usr/lib/python2.7/site-packages/vdsm/network/api.py", line 140, in _rollback yield File "/usr/lib/python2.7/site-packages/vdsm/network/api.py", line 249, in _setup_networks networks, bondings, options, net_info, in_rollback) File "/usr/lib/python2.7/site-packages/vdsm/network/netswitch/configurator.py", line 139, in setup _setup(networks, bondings, options, in_rollback, net_info) File "/usr/lib/python2.7/site-packages/vdsm/network/netswitch/configurator.py", line 159, in _setup _setup_ovs(ovs_nets, ovs_bonds, options, net_info, in_rollback) File "/usr/lib/python2.7/site-packages/vdsm/network/netswitch/configurator.py", line 249, in _setup_ovs connectivity.check(options) File "/usr/lib/python2.7/site-packages/vdsm/network/netconfpersistence.py", line 295, in __exit__ raise ne.RollbackIncomplete(config_diff, ex_type, ex_value) ConfigNetworkError: (10, 'connectivity check failed') MainProcess|jsonrpc/5::DEBUG::2022-04-21 12:01:20,062::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper) call network_caps with () {} MainProcess|jsonrpc/5::DEBUG::2022-04-21 12:01:20,062::logutils::319::root::(_report_stats) ThreadedHandler is ok in the last 1935 seconds (max pending: 3) MainProcess|jsonrpc/5::DEBUG::2022-04-21 12:01:20,076::cmdutils::133::root::(exec_cmd) /sbin/tc qdisc show (cwd None) MainProcess|jsonrpc/5::DEBUG::2022-04-21 12:01:20,098::cmdutils::141::root::(exec_cmd) SUCCESS: <err> = ''; <rc> = 0 Regards, Ravi

2 5

Deployment suddenly fails at engine check
by Harry O 16 May '22

16 May '22

Hi, After the new update, my deployment fails at engine check. What can I do to debug? [ INFO ] skipping: [localhost] [ INFO ] TASK [ovirt.ovirt.engine_setup : Check if Engine health page is up] [ ERROR ] fatal: [localhost -> 192.168.222.12]: FAILED! => {"attempts": 30, "changed": false, "connection": "close", "content": "<html><head><title>Error</title></head><body>500 - Internal Server Error</body></html>", "content_encoding": "identity", "content_length": "86", "content_type": "text/html; charset=UTF-8", "date": "Fri, 22 Apr 2022 16:02:04 GMT", "elapsed": 0, "msg": "Status code was 500 and not [200]: HTTP Error 500: Internal Server Error", "redirected": false, "server": "Apache/2.4.37 (centos) OpenSSL/1.1.1k mod_auth_gssapi/1.6.1 mod_wsgi/4.6.4 Python/3.6", "status": 500, "url": "http://localhost/ovirt-engine/services/health"} [ INFO ] TASK [ovirt.ovirt.engine_setup : Clean temporary files] [ INFO ] changed: [localhost -> 192.168.222.12] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Sync on engine machine] [ INFO ] changed: [localhost -> 192.168.222.12] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Set destination directory path] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Create destination directory] [ INFO ] changed: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : include_tasks] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Find the local appliance image] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Set local_vm_disk_path] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Give the vm time to flush dirty buffers] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Copy engine logs] [ INFO ] changed: [localhost] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Notify the user about a failure] [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "There was a failure deploying the engine on the local engine VM. The system may not be provisioned according to the playbook results: please check the logs for the issue, fix accordingly or re-deploy from scratch.\n"}

3 7