[ovirt-users] Re: Issue upgrading 4.4 to 4.5 Gluster HCG

On Tue, Apr 26, 2022 at 12:47 PM Alessandro De Salvo <Alessandro.DeSalvo(a)roma1.infn.it&gt; wrote: > > Hi, > > the error with XML and gluster is the same I reported with a > possible fix in vdsm in another thread. > > The following fix worked for me, i.e. replacing the following line > in /usr/lib/python3.6/site-packages/vdsm/gluster/cli.y > > 429c429 > <         if (el.find('stripeCount')): value['stripeCount'] = > el.find('stripeCount').text > > --- > >         value['stripeCount'] = el.find('stripeCount').text > > In this way, after restarting vdsmd and supervdsmd, I was able to > connect to gluster 10 volumes. I can file a bug if someone could > please point me where to file it :-) Someone already filed a bug: https://github.com/oVirt/vdsm/issues/155 You can send a pull request with this fix: https://github.com/oVirt/vdsm/pulls Nir > > Cheers, > > >     Alessandro > > > Il 26/04/22 10:55, Sandro Bonazzola ha scritto: > > @Gobinda Das can you please have a look? > > Il giorno mar 26 apr 2022 alle ore 06:47 Abe E <aellahib(a)gmail.com&gt; > ha scritto: > > > > Hey All, > > > > I am having an issue upgrading from 4.4 to 4.5. > > My setup > > 3 Node Gluster (Cluster 1) + 3 Node Cluster (Cluster 2) > > > > If i recall the process correctly, the process I did last week: > > > > On all my Nodes: > > dnf install -y centos-release-ovirt45 --enablerepo=extras > > > > On Ovirt Engine: > > dnf install -y centos-release-ovirt45 > > dnf update -y --nobest > > engine-setup > > > > Once the engine was upgraded successfully I ran the upgrade from > > the GUI on the Cluster 2 Nodes one by one although when they came > > back, they complained of "Host failed to attach one of the > > Storage Domains attached to it." which is the "hosted_storage", > > "data" (gluster). > > > > I thought maybe its due to the fact that 4.5 brings an update to > > the glusterfs version, so I decided to upgrade Node 3 in my > > Gluster Cluster and it booted to emergency mode after the install > > "succeeded". > > > > I feel like I did something wrong, aside from my bravery of > > upgrading so much before realizing somethings not right. > > > > My VDSM Logs from one of the nodes that fails to connect to > > storage (FYI I have 2 Networks, one for Mgmt and 1 for storage > > that are up): > > > > [root@ovirt-4 ~]# tail -f /var/log/vdsm/vdsm.log > > 2022-04-25 22:41:31,584-0600 INFO  (jsonrpc/3) [vdsm.api] FINISH > > repoStats return={} from=::ffff:172.17.117.80,38712, > > task_id=8370855e-dea6-4168-870a-d6235d9044e9 (api:54) > > 2022-04-25 22:41:31,584-0600 INFO  (jsonrpc/3) [vdsm.api] START > > multipath_health() from=::ffff:172.17.117.80,38712, > > task_id=14eb199a-7fbf-4638-a6bf-a384dfbb9d2c (api:48) > > 2022-04-25 22:41:31,584-0600 INFO  (jsonrpc/3) [vdsm.api] FINISH > > multipath_health return={} from=::ffff:172.17.117.80,38712, > > task_id=14eb199a-7fbf-4638-a6bf-a384dfbb9d2c (api:54) > > 2022-04-25 22:41:31,602-0600 INFO  (periodic/1) [vdsm.api] START > > repoStats(domains=()) from=internal, task_id=08a5c00b-1f66-493f- > > a408-d4006ddaa959 (api:48) > > 2022-04-25 22:41:31,603-0600 INFO  (periodic/1) [vdsm.api] FINISH > > repoStats return={} from=internal, task_id=08a5c00b-1f66-493f- > > a408-d4006ddaa959 (api:54) > > 2022-04-25 22:41:31,606-0600 INFO  (jsonrpc/3) [api.host] FINISH > > getStats return={'status': {'code': 0, 'message': 'Done'}, > > 'info': (suppressed)} from=::ffff:172.17.117.80,38712 (api:54) > > 2022-04-25 22:41:35,393-0600 INFO  (jsonrpc/5) [api.host] START > > getAllVmStats() from=::ffff:172.17.117.80,38712 (api:48) > > 2022-04-25 22:41:35,393-0600 INFO  (jsonrpc/5) [api.host] FINISH > > getAllVmStats return={'status': {'code': 0, 'message': 'Done'}, > > 'statsList': (suppressed)} from=::ffff:172.17.117.80,38712 > > (api:54) > > 2022-04-25 22:41:39,366-0600 INFO  (jsonrpc/2) [api.host] START > > getAllVmStats() from=::1,53634 (api:48) > > 2022-04-25 22:41:39,366-0600 INFO  (jsonrpc/2) [api.host] FINISH > > getAllVmStats return={'status': {'code': 0, 'message': 'Done'}, > > 'statsList': (suppressed)} from=::1,53634 (api:54) > > 2022-04-25 22:41:46,530-0600 INFO  (jsonrpc/1) [api.host] START > > getStats() from=::ffff:172.17.117.80,38712 (api:48) > > 2022-04-25 22:41:46,568-0600 INFO  (jsonrpc/1) [vdsm.api] START > > repoStats(domains=()) from=::ffff:172.17.117.80,38712, > > task_id=30404767-9761-4f8c-884a-5561dd0d82fe (api:48) > > 2022-04-25 22:41:46,568-0600 INFO  (jsonrpc/1) [vdsm.api] FINISH > > repoStats return={} from=::ffff:172.17.117.80,38712, > > task_id=30404767-9761-4f8c-884a-5561dd0d82fe (api:54) > > 2022-04-25 22:41:46,569-0600 INFO  (jsonrpc/1) [vdsm.api] START > > multipath_health() from=::ffff:172.17.117.80,38712, > > task_id=8dbfa47f-e1b7-408c-a060-8d45012f0b90 (api:48) > > 2022-04-25 22:41:46,569-0600 INFO  (jsonrpc/1) [vdsm.api] FINISH > > multipath_health return={} from=::ffff:172.17.117.80,38712, > > task_id=8dbfa47f-e1b7-408c-a060-8d45012f0b90 (api:54) > > 2022-04-25 22:41:46,574-0600 INFO  (jsonrpc/1) [api.host] FINISH > > getStats return={'status': {'code': 0, 'message': 'Done'}, > > 'info': (suppressed)} from=::ffff:172.17.117.80,38712 (api:54) > > 2022-04-25 22:41:46,651-0600 INFO  (periodic/0) [vdsm.api] START > > repoStats(domains=()) from=internal, task_id=92c69020-d0b1-4813- > > 8610-3f3e1892c20b (api:48) > > 2022-04-25 22:41:46,652-0600 INFO  (periodic/0) [vdsm.api] FINISH > > repoStats return={} from=internal, task_id=92c69020-d0b1-4813- > > 8610-3f3e1892c20b (api:54) > > 2022-04-25 22:41:50,397-0600 INFO  (jsonrpc/6) [api.host] START > > getAllVmStats() from=::ffff:172.17.117.80,38712 (api:48) > > 2022-04-25 22:41:50,397-0600 INFO  (jsonrpc/6) [api.host] FINISH > > getAllVmStats return={'status': {'code': 0, 'message': 'Done'}, > > 'statsList': (suppressed)} from=::ffff:172.17.117.80,38712 > > (api:54) > > 2022-04-25 22:41:52,533-0600 INFO  (jsonrpc/4) [api.host] START > > getCapabilities() from=::ffff:172.17.117.80,38712 (api:48) > > 2022-04-25 22:41:54,382-0600 INFO  (jsonrpc/0) [api.host] START > > getAllVmStats() from=::1,53634 (api:48) > > 2022-04-25 22:41:54,382-0600 INFO  (jsonrpc/0) [api.host] FINISH > > getAllVmStats return={'status': {'code': 0, 'message': 'Done'}, > > 'statsList': (suppressed)} from=::1,53634 (api:54) > > 2022-04-25 22:41:55,037-0600 INFO  (jsonrpc/4) [root] > > /usr/libexec/vdsm/hooks/after_get_caps/ovirt_provider_ovn_hook: > > rc=0 err=b'' (hooks:123) > > 2022-04-25 22:41:55,039-0600 INFO  (jsonrpc/4) [api.host] FINISH > > getCapabilities return={'status': {'code': 0, 'message': 'Done'}, > > 'info': {'kvmEnabled': 'true', 'cpuCores': '6', 'cpuThreads': > > '12', 'cpuSockets': '1', 'onlineCpus': > > '0,1,2,3,4,5,6,7,8,9,10,11', 'cpuTopology': [{'cpu_id': 0, > > 'numa_cell_id': 0, 'socket_id': 0, 'die_id': 0, 'core_id': 0}, > > {'cpu_id': 1, 'numa_cell_id': 0, 'socket_id': 0, 'die_id': 0, > > 'core_id': 1}, {'cpu_id': 2, 'numa_cell_id': 0, 'socket_id': 0, > > 'die_id': 0, 'core_id': 2}, {'cpu_id': 3, 'numa_cell_id': 0, > > 'socket_id': 0, 'die_id': 0, 'core_id': 3}, {'cpu_id': 4, > > 'numa_cell_id': 0, 'socket_id': 0, 'die_id': 0, 'core_id': 4}, > > {'cpu_id': 5, 'numa_cell_id': 0, 'socket_id': 0, 'die_id': 0, > > 'core_id': 5}, {'cpu_id': 6, 'numa_cell_id': 0, 'socket_id': 0, > > 'die_id': 0, 'core_id': 0}, {'cpu_id': 7, 'numa_cell_id': 0, > > 'socket_id': 0, 'die_id': 0, 'core_id': 1}, {'cpu_id': 8, > > 'numa_cell_id': 0, 'socket_id': 0, 'die_id': 0, 'core_id': 2}, > > {'cpu_id': 9, 'numa_cell_id' > >  : 0, 'socket_id': 0, 'die_id': 0, 'core_id': 3}, {'cpu_id': 10, > > 'numa_cell_id': 0, 'socket_id': 0, 'die_id': 0, 'core_id': 4}, > > {'cpu_id': 11, 'numa_cell_id': 0, 'socket_id': 0, 'die_id': 0, > > 'core_id': 5}], 'cpuSpeed': '2500.000', 'cpuModel': 'Intel(R) > > Xeon(R) CPU E5-2620 0 @ 2.00GHz', 'cpuFlags': > > 'pdcm,xsaveopt,dtes64,xtpr,clflush,de,ibpb,popcnt,cpuid,ida,monit > > or,amd- > > stibp,x2apic,lm,arat,pse36,tsc_deadline_timer,fxsr,ht,skip-l1dfl- > > vmentry,est,pcid,aperfmperf,nopl,apic,mce,xsave,ibrs,flush_l1d,dt > > herm,dts,flexpriority,pse,pdpe1gb,pni,sse2,pge,cx16,pschange-mc- > > no,bts,rdtscp,dca,avx,hypervisor,tsc,tsc_adjust,nx,mmx,pebs,ss,um > > ip,xtopology,vnmi,arch- > > capabilities,pae,pclmulqdq,tm,aes,invtsc,md_clear,ssse3,amd- > > ssbd,ssbd,sse4_1,smx,rep_good,vmx,cx8,sse,arch_perfmon,msr,stibp, > > nonstop_tsc,pti,ds_cpl,mca,cmov,md- > > clear,fpu,lahf_lm,tm2,sep,tpr_shadow,constant_tsc,pbe,pat,syscall > > ,sse4_2,pln,acpi,mtrr,pts,vme,ept,vpid,spec_ctrl,model_pentium,mo > > del_Nehalem,model_486,model_SandyBridge,model_pentium2, > >  model_Opteron_G1,model_Nehalem- > > IBRS,model_qemu32,model_kvm32,model_coreduo,model_Westmere,model_ > > SandyBridge-IBRS,model_Westmere- > > IBRS,model_Penryn,model_pentium3,model_qemu64,model_Conroe,model_ > > kvm64,model_core2duo,model_Opteron_G2', 'vdsmToCpusAffinity': > > [1], 'version_name': 'Snow Man', 'software_version': '4.50.0.13', > > 'software_revision': '1', 'supportedENGINEs': ['4.2', '4.3', > > '4.4', '4.5'], 'clusterLevels': ['4.2', '4.3', '4.4', '4.5', > > '4.6', '4.7'], 'networks': {'ovirtmgmt': {'ports': ['bond0'], > > 'stp': 'off', 'iface': 'ovirtmgmt', 'bridged': True, 'addr': > > '172.17.117.74', 'netmask': '255.255.255.0', 'ipv4addrs': > > ['172.17.117.74/24'], 'ipv6addrs': [], 'ipv6autoconf': True, > > 'gateway': '172.17.117.1', 'ipv6gateway': '::', > > 'ipv4defaultroute': True, 'mtu': '1500', 'switch': 'legacy', > > 'southbound': 'bond0', 'dhcpv4': False, 'dhcpv6': True}, 'LabNet- > > v106': {'ports': ['bond0.106'], 'stp': 'off', 'iface': 'LabNet- > > v106', 'bridged': True, 'addr': '', 'netmask': '', 'ipv4addrs': > > [], 'ipv6ad > >  drs': [], 'ipv6autoconf': False, 'gateway': '', 'ipv6gateway': > > '::', 'ipv4defaultroute': False, 'mtu': '1500', 'switch': > > 'legacy', 'southbound': 'bond0.106', 'vlanid': 106, 'dhcpv4': > > False, 'dhcpv6': False}, 'PIP_V991': {'ports': ['bond0.991'], > > 'stp': 'off', 'iface': 'PIP_V991', 'bridged': True, 'addr': '', > > 'netmask': '', 'ipv4addrs': [], 'ipv6addrs': [], 'ipv6autoconf': > > False, 'gateway': '', 'ipv6gateway': '::', 'ipv4defaultroute': > > False, 'mtu': '1500', 'switch': 'legacy', 'southbound': > > 'bond0.991', 'vlanid': 991, 'dhcpv4': False, 'dhcpv6': False}, > > 'NetEng-V3101': {'ports': ['bond0.3101'], 'stp': 'off', 'iface': > > 'NetEng-V3101', 'bridged': True, 'addr': '', 'netmask': '', > > 'ipv4addrs': [], 'ipv6addrs': [], 'ipv6autoconf': False, > > 'gateway': '', 'ipv6gateway': '::', 'ipv4defaultroute': False, > > 'mtu': '1500', 'switch': 'legacy', 'southbound': 'bond0.3101', > > 'vlanid': 3101, 'dhcpv4': False, 'dhcpv6': False}, 'OVIRT-VMs': > > {'ports': ['bond0.177'], 'stp': 'off', 'iface': 'OVIRT-VMs', > > 'bridged > >  ': True, 'addr': '', 'netmask': '', 'ipv4addrs': [], > > 'ipv6addrs': [], 'ipv6autoconf': False, 'gateway': '', > > 'ipv6gateway': '::', 'ipv4defaultroute': False, 'mtu': '1500', > > 'switch': 'legacy', 'southbound': 'bond0.177', 'vlanid': 177, > > 'dhcpv4': False, 'dhcpv6': False}, 'Gluster_Net': {'ports': > > ['bond1'], 'stp': 'off', 'iface': 'Gluster_Net', 'bridged': True, > > 'addr': '172.17.181.13', 'netmask': '255.255.255.0', 'ipv4addrs': > > ['172.17.181.13/24'], 'ipv6addrs': [], 'ipv6autoconf': False, > > 'gateway': '172.17.181.1', 'ipv6gateway': '::', > > 'ipv4defaultroute': False, 'mtu': '1500', 'switch': 'legacy', > > 'southbound': 'bond1', 'dhcpv4': False, 'dhcpv6': False}}, > > 'bondings': {'bond0': {'hwaddr': 'c8:1f:66:f6:e5:48', 'slaves': > > ['eno1', 'eno2'], 'active_slave': '', 'opts': {'mode': '4', > > 'xmit_hash_policy': '2'}, 'ad_aggregator_id': '1', > > 'ad_partner_mac': '4c:4e:35:26:2c:00', 'switch': 'legacy', > > 'addr': '', 'ipv4addrs': [], 'ipv6addrs': [], 'ipv6autoconf': > > False, 'gateway': '', 'ipv6gateway': '::', 'm > >  tu': '1500', 'netmask': '', 'ipv4defaultroute': False, 'dhcpv4': > > False, 'dhcpv6': False, 'speed': 2000}, 'bond1': {'hwaddr': > > 'c8:1f:66:f6:e5:4a', 'slaves': ['eno4', 'eno3'], 'active_slave': > > '', 'opts': {'mode': '4', 'xmit_hash_policy': '2'}, > > 'ad_aggregator_id': '1', 'ad_partner_mac': '4c:4e:35:26:2c:00', > > 'switch': 'legacy', 'addr': '', 'ipv4addrs': [], 'ipv6addrs': [], > > 'ipv6autoconf': False, 'gateway': '', 'ipv6gateway': '::', 'mtu': > > '1500', 'netmask': '', 'ipv4defaultroute': False, 'dhcpv4': > > False, 'dhcpv6': False, 'speed': 2000}}, 'bridges': > > {'Gluster_Net': {'ports': ['bond1'], 'stp': 'off', 'addr': > > '172.17.181.13', 'ipv4addrs': ['172.17.181.13/24'], 'ipv6addrs': > > [], 'ipv6autoconf': False, 'gateway': '172.17.181.1', > > 'ipv6gateway': '::', 'mtu': '1500', 'netmask': '255.255.255.0', > > 'ipv4defaultroute': False, 'dhcpv4': False, 'dhcpv6': False, > > 'opts': {'ageing_time': '30000', 'multicast_query_use_ifaddr': > > '0', 'gc_timer': '148', 'hello_time': '200', 'multicast_router': > > '1', 'nf_call_ip > >  tables': '0', 'group_addr': '01:80:c2:00:00:00', > > 'group_fwd_mask': '0x0', 'multicast_querier': '0', 'hash_max': > > '4096', 'tcn_timer': '0', 'vlan_protocol': '0x8100', > > 'vlan_stats_per_port': '0', 'root_port': '0', 'vlan_filtering': > > '0', 'multicast_query_interval': '12500', 'bridge_id': > > '8000.c81f66f6e54a', 'max_age': '2000', 'nf_call_arptables': '0', > > 'multicast_startup_query_interval': '3125', > > 'multicast_stats_enabled': '0', > > 'multicast_query_response_interval': '1000', 'topology_change': > > '0', 'priority': '32768', 'multicast_mld_version': '1', > > 'hash_elasticity': '16', 'hello_timer': '0', 'default_pvid': '1', > > 'root_path_cost': '0', 'multicast_igmp_version': '2', > > 'stp_state': '0', 'multicast_startup_query_count': '2', > > 'topology_change_detected': '0', > > 'multicast_last_member_interval': '100', 'topology_change_timer': > > '0', 'root_id': '8000.c81f66f6e54a', 'no_linklocal_learn': '0', > > 'forward_delay': '1500', 'multicast_membership_interval': > > '26000', 'multicast_querier_interval': '25500', 'vlan_ > >  stats_enabled': '0', 'multicast_snooping': '1', > > 'nf_call_ip6tables': '0', 'multicast_last_member_count': '2'}}, > > 'ovirtmgmt': {'ports': ['bond0'], 'stp': 'off', 'addr': > > '172.17.117.74', 'ipv4addrs': ['172.17.117.74/24'], 'ipv6addrs': > > [], 'ipv6autoconf': True, 'gateway': '172.17.117.1', > > 'ipv6gateway': '::', 'mtu': '1500', 'netmask': '255.255.255.0', > > 'ipv4defaultroute': True, 'dhcpv4': False, 'dhcpv6': True, > > 'opts': {'ageing_time': '30000', 'multicast_query_use_ifaddr': > > '0', 'gc_timer': '148', 'hello_time': '200', 'multicast_router': > > '1', 'nf_call_iptables': '0', 'group_addr': '01:80:c2:00:00:00', > > 'group_fwd_mask': '0x0', 'multicast_querier': '0', 'hash_max': > > '4096', 'tcn_timer': '0', 'vlan_protocol': '0x8100', > > 'vlan_stats_per_port': '0', 'root_port': '0', 'vlan_filtering': > > '0', 'multicast_query_interval': '12500', 'bridge_id': > > '8000.c81f66f6e548', 'max_age': '2000', 'nf_call_arptables': '0', > > 'multicast_startup_query_interval': '3125', > > 'multicast_stats_enabled': '0', 'multicast_query_r > >  esponse_interval': '1000', 'topology_change': '0', 'priority': > > '32768', 'multicast_mld_version': '1', 'hash_elasticity': '16', > > 'hello_timer': '0', 'default_pvid': '1', 'root_path_cost': '0', > > 'multicast_igmp_version': '2', 'stp_state': '0', > > 'multicast_startup_query_count': '2', 'topology_change_detected': > > '0', 'multicast_last_member_interval': '100', > > 'topology_change_timer': '0', 'root_id': '8000.c81f66f6e548', > > 'no_linklocal_learn': '0', 'forward_delay': '1500', > > 'multicast_membership_interval': '26000', > > 'multicast_querier_interval': '25500', 'vlan_stats_enabled': '0', > > 'multicast_snooping': '1', 'nf_call_ip6tables': '0', > > 'multicast_last_member_count': '2'}}, 'LabNet-v106': {'ports': > > ['bond0.106'], 'stp': 'off', 'addr': '', 'ipv4addrs': [], > > 'ipv6addrs': [], 'ipv6autoconf': False, 'gateway': '', > > 'ipv6gateway': '::', 'mtu': '1500', 'netmask': '', > > 'ipv4defaultroute': False, 'dhcpv4': False, 'dhcpv6': False, > > 'opts': {'ageing_time': '30000', 'multicast_query_use_ifaddr': > > '0', 'gc_timer': '1 > >  8247', 'hello_time': '200', 'multicast_router': '1', > > 'nf_call_iptables': '0', 'group_addr': '01:80:c2:00:00:00', > > 'group_fwd_mask': '0x0', 'multicast_querier': '0', 'hash_max': > > '4096', 'tcn_timer': '0', 'vlan_protocol': '0x8100', > > 'vlan_stats_per_port': '0', 'root_port': '0', 'vlan_filtering': > > '0', 'multicast_query_interval': '12500', 'bridge_id': > > '8000.c81f66f6e548', 'max_age': '2000', 'nf_call_arptables': '0', > > 'multicast_startup_query_interval': '3125', > > 'multicast_stats_enabled': '0', > > 'multicast_query_response_interval': '1000', 'topology_change': > > '0', 'priority': '32768', 'multicast_mld_version': '1', > > 'hash_elasticity': '16', 'hello_timer': '0', 'default_pvid': '1', > > 'root_path_cost': '0', 'multicast_igmp_version': '2', > > 'stp_state': '0', 'multicast_startup_query_count': '2', > > 'topology_change_detected': '0', > > 'multicast_last_member_interval': '100', 'topology_change_timer': > > '0', 'root_id': '8000.c81f66f6e548', 'no_linklocal_learn': '0', > > 'forward_delay': '1500', 'multicast_membership_i > >  nterval': '26000', 'multicast_querier_interval': '25500', > > 'vlan_stats_enabled': '0', 'multicast_snooping': '1', > > 'nf_call_ip6tables': '0', 'multicast_last_member_count': '2'}}, > > 'NetEng-V3101': {'ports': ['bond0.3101'], 'stp': 'off', 'addr': > > '', 'ipv4addrs': [], 'ipv6addrs': [], 'ipv6autoconf': False, > > 'gateway': '', 'ipv6gateway': '::', 'mtu': '1500', 'netmask': '', > > 'ipv4defaultroute': False, 'dhcpv4': False, 'dhcpv6': False, > > 'opts': {'ageing_time': '30000', 'multicast_query_use_ifaddr': > > '0', 'gc_timer': '6208', 'hello_time': '200', 'multicast_router': > > '1', 'nf_call_iptables': '0', 'group_addr': '01:80:c2:00:00:00', > > 'group_fwd_mask': '0x0', 'multicast_querier': '0', 'hash_max': > > '4096', 'tcn_timer': '0', 'vlan_protocol': '0x8100', > > 'vlan_stats_per_port': '0', 'root_port': '0', 'vlan_filtering': > > '0', 'multicast_query_interval': '12500', 'bridge_id': > > '8000.c81f66f6e548', 'max_age': '2000', 'nf_call_arptables': '0', > > 'multicast_startup_query_interval': '3125', > > 'multicast_stats_enabled': '0' > >  , 'multicast_query_response_interval': '1000', > > 'topology_change': '0', 'priority': '32768', > > 'multicast_mld_version': '1', 'hash_elasticity': '16', > > 'hello_timer': '0', 'default_pvid': '1', 'root_path_cost': '0', > > 'multicast_igmp_version': '2', 'stp_state': '0', > > 'multicast_startup_query_count': '2', 'topology_change_detected': > > '0', 'multicast_last_member_interval': '100', > > 'topology_change_timer': '0', 'root_id': '8000.c81f66f6e548', > > 'no_linklocal_learn': '0', 'forward_delay': '1500', > > 'multicast_membership_interval': '26000', > > 'multicast_querier_interval': '25500', 'vlan_stats_enabled': '0', > > 'multicast_snooping': '1', 'nf_call_ip6tables': '0', > > 'multicast_last_member_count': '2'}}, 'PIP_V991': {'ports': > > ['bond0.991'], 'stp': 'off', 'addr': '', 'ipv4addrs': [], > > 'ipv6addrs': [], 'ipv6autoconf': False, 'gateway': '', > > 'ipv6gateway': '::', 'mtu': '1500', 'netmask': '', > > 'ipv4defaultroute': False, 'dhcpv4': False, 'dhcpv6': False, > > 'opts': {'ageing_time': '30000', 'multicast_query_use_ifaddr': '0 > >  ', 'gc_timer': '15686', 'hello_time': '200', 'multicast_router': > > '1', 'nf_call_iptables': '0', 'group_addr': '01:80:c2:00:00:00', > > 'group_fwd_mask': '0x0', 'multicast_querier': '0', 'hash_max': > > '4096', 'tcn_timer': '0', 'vlan_protocol': '0x8100', > > 'vlan_stats_per_port': '0', 'root_port': '0', 'vlan_filtering': > > '0', 'multicast_query_interval': '12500', 'bridge_id': > > '8000.c81f66f6e548', 'max_age': '2000', 'nf_call_arptables': '0', > > 'multicast_startup_query_interval': '3125', > > 'multicast_stats_enabled': '0', > > 'multicast_query_response_interval': '1000', 'topology_change': > > '0', 'priority': '32768', 'multicast_mld_version': '1', > > 'hash_elasticity': '16', 'hello_timer': '0', 'default_pvid': '1', > > 'root_path_cost': '0', 'multicast_igmp_version': '2', > > 'stp_state': '0', 'multicast_startup_query_count': '2', > > 'topology_change_detected': '0', > > 'multicast_last_member_interval': '100', 'topology_change_timer': > > '0', 'root_id': '8000.c81f66f6e548', 'no_linklocal_learn': '0', > > 'forward_delay': '1500', 'multi > >  cast_membership_interval': '26000', > > 'multicast_querier_interval': '25500', 'vlan_stats_enabled': '0', > > 'multicast_snooping': '1', 'nf_call_ip6tables': '0', > > 'multicast_last_member_count': '2'}}, 'OVIRT-VMs': {'ports': > > ['bond0.177'], 'stp': 'off', 'addr': '', 'ipv4addrs': [], > > 'ipv6addrs': [], 'ipv6autoconf': False, 'gateway': '', > > 'ipv6gateway': '::', 'mtu': '1500', 'netmask': '', > > 'ipv4defaultroute': False, 'dhcpv4': False, 'dhcpv6': False, > > 'opts': {'ageing_time': '30000', 'multicast_query_use_ifaddr': > > '0', 'gc_timer': '20761', 'hello_time': '200', > > 'multicast_router': '1', 'nf_call_iptables': '0', 'group_addr': > > '01:80:c2:00:00:00', 'group_fwd_mask': '0x0', > > 'multicast_querier': '0', 'hash_max': '4096', 'tcn_timer': '0', > > 'vlan_protocol': '0x8100', 'vlan_stats_per_port': '0', > > 'root_port': '0', 'vlan_filtering': '0', > > 'multicast_query_interval': '12500', 'bridge_id': > > '8000.c81f66f6e548', 'max_age': '2000', 'nf_call_arptables': '0', > > 'multicast_startup_query_interval': '3125', 'multicast_stats > >  _enabled': '0', 'multicast_query_response_interval': '1000', > > 'topology_change': '0', 'priority': '32768', > > 'multicast_mld_version': '1', 'hash_elasticity': '16', > > 'hello_timer': '0', 'default_pvid': '1', 'root_path_cost': '0', > > 'multicast_igmp_version': '2', 'stp_state': '0', > > 'multicast_startup_query_count': '2', 'topology_change_detected': > > '0', 'multicast_last_member_interval': '100', > > 'topology_change_timer': '0', 'root_id': '8000.c81f66f6e548', > > 'no_linklocal_learn': '0', 'forward_delay': '1500', > > 'multicast_membership_interval': '26000', > > 'multicast_querier_interval': '25500', 'vlan_stats_enabled': '0', > > 'multicast_snooping': '1', 'nf_call_ip6tables': '0', > > 'multicast_last_member_count': '2'}}}, 'nics': {'eno3': > > {'hwaddr': 'c8:1f:66:f6:e5:4a', 'ad_aggregator_id': '1', 'addr': > > '', 'ipv4addrs': [], 'ipv6addrs': [], 'ipv6autoconf': False, > > 'gateway': '', 'ipv6gateway': '::', 'mtu': '1500', 'netmask': '', > > 'ipv4defaultroute': False, 'permhwaddr': 'c8:1f:66:f6:e5:4a', > > 'dhcpv4': False, 'dhcpv6': > >   False, 'speed': 1000}, 'eno4': {'hwaddr': 'c8:1f:66:f6:e5:4a', > > 'ad_aggregator_id': '1', 'addr': '', 'ipv4addrs': [], > > 'ipv6addrs': [], 'ipv6autoconf': False, 'gateway': '', > > 'ipv6gateway': '::', 'mtu': '1500', 'netmask': '', > > 'ipv4defaultroute': False, 'permhwaddr': 'c8:1f:66:f6:e5:4b', > > 'dhcpv4': False, 'dhcpv6': False, 'speed': 1000}, 'eno1': > > {'hwaddr': 'c8:1f:66:f6:e5:48', 'ad_aggregator_id': '1', 'addr': > > '', 'ipv4addrs': [], 'ipv6addrs': [], 'ipv6autoconf': False, > > 'gateway': '', 'ipv6gateway': '::', 'mtu': '1500', 'netmask': '', > > 'ipv4defaultroute': False, 'permhwaddr': 'c8:1f:66:f6:e5:48', > > 'dhcpv4': False, 'dhcpv6': False, 'speed': 1000}, 'eno2': > > {'hwaddr': 'c8:1f:66:f6:e5:48', 'ad_aggregator_id': '1', 'addr': > > '', 'ipv4addrs': [], 'ipv6addrs': [], 'ipv6autoconf': False, > > 'gateway': '', 'ipv6gateway': '::', 'mtu': '1500', 'netmask': '', > > 'ipv4defaultroute': False, 'permhwaddr': 'c8:1f:66:f6:e5:49', > > 'dhcpv4': False, 'dhcpv6': False, 'speed': 1000}}, 'vlans': > > {'bond0.106': {'iface': 'bo > >  nd0', 'vlanid': 106, 'addr': '', 'ipv4addrs': [], 'ipv6addrs': > > [], 'ipv6autoconf': False, 'gateway': '', 'ipv6gateway': '::', > > 'mtu': '1500', 'netmask': '', 'ipv4defaultroute': False, > > 'dhcpv4': False, 'dhcpv6': False}, 'bond0.991': {'iface': > > 'bond0', 'vlanid': 991, 'addr': '', 'ipv4addrs': [], 'ipv6addrs': > > [], 'ipv6autoconf': False, 'gateway': '', 'ipv6gateway': '::', > > 'mtu': '1500', 'netmask': '', 'ipv4defaultroute': False, > > 'dhcpv4': False, 'dhcpv6': False}, 'bond0.177': {'iface': > > 'bond0', 'vlanid': 177, 'addr': '', 'ipv4addrs': [], 'ipv6addrs': > > [], 'ipv6autoconf': False, 'gateway': '', 'ipv6gateway': '::', > > 'mtu': '1500', 'netmask': '', 'ipv4defaultroute': False, > > 'dhcpv4': False, 'dhcpv6': False}, 'bond0.3101': {'iface': > > 'bond0', 'vlanid': 3101, 'addr': '', 'ipv4addrs': [], > > 'ipv6addrs': [], 'ipv6autoconf': False, 'gateway': '', > > 'ipv6gateway': '::', 'mtu': '1500', 'netmask': '', > > 'ipv4defaultroute': False, 'dhcpv4': False, 'dhcpv6': False}}, > > 'nameservers': ['8.8.8.8'], 'supportsIPv6': > >  True, 'ovnConfigured': False, 'hooks': {'before_vm_start': > > {'50_hostedengine': {'checksum': > > 'e5f5262cf22e06cd34e227afb27647e479351266876019a64210dbcbd2a43830 > > '}}, 'after_get_caps': {'ovirt_provider_ovn_hook': {'checksum': > > 'a2bdefca38b96c8ddab39822cc8282bf3f67d875c4879003ffc9661826c92421 > > '}}, 'before_device_create': {'10_ovirt_provider_ovn_hook': > > {'checksum': > > 'b9d67afb41bd86a521ada2121e3505297b3b6dfd478275ce0bd9501fcda6dafc > > '}}, 'before_nic_hotplug': {'10_ovirt_provider_ovn_hook': > > {'checksum': > > 'b9d67afb41bd86a521ada2121e3505297b3b6dfd478275ce0bd9501fcda6dafc > > '}}}, 'operatingSystem': {'release': '1.el8', 'version': > > '8.6.2203.0', 'name': 'RHEL', 'pretty_name': 'oVirt Node 4.5.0'}, > > 'uuid': '4c4c4544-0053-5810-8052-c8c04f563132', 'packages2': > > {'kernel': {'version': '4.18.0', 'release': '373.el8.x86_64'}, > > 'glusterfs-cli': {'version': '10.1', 'release': '1.el8s'}, > > 'librbd1': {'version': '16.2.7', 'release': '1.el8s'}, 'libvirt': > > {'version': '8.0.0', 'release': '2.module_el8.6.0+1087+b42c8331'} > >  , 'mom': {'version': '0.6.2', 'release': '1.el8'}, 'ovirt- > > hosted-engine-ha': {'version': '2.5.0', 'release': '1.el8'}, > > 'openvswitch': {'version': '2.15', 'release': '3.el8'}, > > 'nmstate': {'version': '1.2.1', 'release': '1.el8'}, 'qemu-img': > > {'version': '6.2.0', 'release': > > '5.module_el8.6.0+1087+b42c8331'}, 'qemu-kvm': {'version': > > '6.2.0', 'release': '5.module_el8.6.0+1087+b42c8331'}, 'spice- > > server': {'version': '0.14.3', 'release': '4.el8'}, 'vdsm': > > {'version': '4.50.0.13', 'release': '1.el8'}, 'glusterfs': > > {'version': '10.1', 'release': '1.el8s'}, 'glusterfs-fuse': > > {'version': '10.1', 'release': '1.el8s'}, 'glusterfs-geo- > > replication': {'version': '10.1', 'release': '1.el8s'}, > > 'glusterfs-server': {'version': '10.1', 'release': '1.el8s'}}, > > 'realtimeKernel': False, 'kernelArgs': > > 'BOOT_IMAGE=(hd0,msdos1)//ovirt-node-ng-4.5.0- > > 0.20220420.0+1/vmlinuz-4.18.0-373.el8.x86_64 crashkernel=auto > > resume=/dev/mapper/onn-swap rd.lvm.lv=onn/ovirt-node-ng-4.5.0- > > 0.20220420.0+1 rd.lvm.lv=onn/swap rhgb q > >  uiet kvm-intel.nested=1 root=/dev/onn/ovirt-node-ng-4.5.0- > > 0.20220420.0+1 boot=UUID=adb2035d-5047-471d-8b51-206e0afb39f4 > > rootflags=discard img.bootid=ovirt-node-ng-4.5.0-0.20220420.0+1', > > 'nestedVirtualization': True, 'emulatedMachines': ['pc-q35- > > rhel8.6.0', 'pc-i440fx-rhel7.1.0', 'pc-q35-rhel8.2.0', 'pc-q35- > > rhel7.6.0', 'pc-i440fx-rhel7.3.0', 'pc-i440fx-rhel7.6.0', 'pc- > > q35-rhel8.5.0', 'pc-q35-rhel8.0.0', 'pc-i440fx-rhel7.2.0', 'pc', > > 'pc-q35-rhel7.3.0', 'pc-i440fx-rhel7.4.0', 'q35', 'pc-i440fx- > > 2.11', 'pc-q35-rhel7.4.0', 'pc-i440fx-rhel7.5.0', 'pc-i440fx- > > rhel7.0.0', 'pc-q35-rhel7.5.0', 'pc-i440fx-4.2', 'pc-q35- > > rhel8.3.0', 'pc-q35-rhel8.1.0', 'pc-q35-rhel8.4.0'], > > 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:ea2a5da196cb', > > 'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994- > > 05.com.redhat:ea2a5da196cb'}], 'FC': []}, 'vmTypes': ['kvm'], > > 'memSize': '43996', 'reservedMem': '321', 'guestOverhead': '65', > > 'rngSources': ['random', 'hwrng'], 'numaNodes': {'0': > > {'totalMemory': '43996', 'hugepag > >  es': {'4': {'totalPages': '11263209'}, '2048': {'totalPages': > > '0'}, '1048576': {'totalPages': '0'}}, 'cpus': [0, 1, 2, 3, 4, 5, > > 6, 7, 8, 9, 10, 11]}}, 'numaNodeDistance': {'0': [10]}, > > 'autoNumaBalancing': 2, 'selinux': {'mode': '1'}, 'liveSnapshot': > > 'true', 'liveMerge': 'true', 'kdumpStatus': 0, > > 'deferred_preallocation': True, 'hostdevPassthrough': 'false', > > 'additionalFeatures': ['libgfapi_supported', 'GLUSTER_SNAPSHOT', > > 'GLUSTER_GEO_REPLICATION', 'GLUSTER_BRICK_MANAGEMENT'], > > 'hostedEngineDeployed': False, 'hugepages': [2048, 1048576], > > 'kernelFeatures': {'SPECTRE_V2': '(Mitigation: Full generic > > retpoline, IBPB: conditional, IBRS_FW, STIBP: conditional, RSB > > filling)', 'ITLB_MULTIHIT': '(KVM: Mitigation: VMX disabled)', > > 'MDS': '(Mitigation: Clear CPU buffers; SMT vulnerable)', 'L1TF': > > '(Mitigation: PTE Inversion; VMX: conditional cache flushes, SMT > > vulnerable)', 'SPEC_STORE_BYPASS': '(Mitigation: Speculative > > Store Bypass disabled via prctl and seccomp)', 'TSX_ASYNC_ABORT': > > '(Not affec > >  ted)', 'SPECTRE_V1': '(Mitigation: usercopy/swapgs barriers and > > __user pointer sanitization)', 'SRBDS': '(Not affected)', > > 'MELTDOWN': '(Mitigation: PTI)'}, 'vncEncrypted': True, > > 'backupEnabled': True, 'coldBackupEnabled': True, > > 'clearBitmapsEnabled': True, 'fipsEnabled': False, 'boot_uuid': > > 'adb2035d-5047-471d-8b51-206e0afb39f4', 'tscFrequency': > > '1999999000', 'tscScaling': False, 'connector_info': {'platform': > > 'x86_64', 'os_type': 'linux', 'ip': None, 'host': 'ovirt- > > 4.[removed].com', 'multipath': True, 'initiator': 'iqn.1994- > > 05.com.redhat:ea2a5da196cb', 'do_local_attach': False, 'uuid': > > '215601b1-e536-4258-ad35-d1f869afa0f8', 'system uuid': '4c4c4544- > > 0053-5810-8052-c8c04f563132', 'nqn': 'nqn.2014- > > 08.org.nvmexpress:uuid:00000000-0000-0000-0000-000000000000', > > 'found_dsc': ''}, 'domain_versions': [0, 2, 3, 4, 5], > > 'supported_block_size': {'FCP': [512], 'GLUSTERFS': [0, 512, > > 4096], 'ISCSI': [512], 'LOCALFS': [0, 512, 4096], 'NFS': [512], > > 'POSIXFS': [512]}, 'cd_change_pdiv': True, 'refres > >  h_disk_supported': True, 'replicate_extend': True, > > 'measure_subchain': True, 'measure_active': True, > > 'mailbox_events': True, 'netConfigDirty': 'False', > > 'openstack_binding_host_ids': {'OVIRT_PROVIDER_OVN': 'eaa82268- > > bd08-453f-9953-b4aad4c4c307'}, 'lastClientIface': 'ovirtmgmt'}} > > from=::ffff:172.17.117.80,38712 (api:54) > > 2022-04-25 22:41:55,046-0600 INFO  (jsonrpc/4) > > [jsonrpc.JsonRpcServer] RPC call Host.getCapabilities took more > > than 1.00 seconds to succeed: 2.51 (__init__:316) > > 2022-04-25 22:41:55,058-0600 INFO  (jsonrpc/7) [api.host] START > > getHardwareInfo() from=::ffff:172.17.117.80,38712 (api:48) > > 2022-04-25 22:41:55,058-0600 INFO  (jsonrpc/7) [api.host] FINISH > > getHardwareInfo return={'status': {'code': 0, 'message': 'Done'}, > > 'info': {'systemManufacturer': ' ', 'systemProductName': ' ', > > 'systemVersion': '', 'systemSerialNumber': 'HSXRV12', > > 'systemUUID': '4C4C4544-0053-5810-8052-C8C04F563132', > > 'systemFamily': ''}} from=::ffff:172.17.117.80,38712 (api:54) > > 2022-04-25 22:41:55,121-0600 INFO  (jsonrpc/3) [api.host] START > > getStats() from=::ffff:172.17.117.80,38712, flow_id=610b2f2d > > (api:48) > > 2022-04-25 22:41:55,160-0600 INFO  (jsonrpc/3) [vdsm.api] START > > repoStats(domains=()) from=::ffff:172.17.117.80,38712, > > flow_id=610b2f2d, task_id=de30e54a-21c6-46ff-a669-230145f93ae2 > > (api:48) > > 2022-04-25 22:41:55,160-0600 INFO  (jsonrpc/3) [vdsm.api] FINISH > > repoStats return={} from=::ffff:172.17.117.80,38712, > > flow_id=610b2f2d, task_id=de30e54a-21c6-46ff-a669-230145f93ae2 > > (api:54) > > 2022-04-25 22:41:55,161-0600 INFO  (jsonrpc/3) [vdsm.api] START > > multipath_health() from=::ffff:172.17.117.80,38712, > > flow_id=610b2f2d, task_id=b751b6d0-bc99-457b-9620-4ffaaaa37c91 > > (api:48) > > 2022-04-25 22:41:55,161-0600 INFO  (jsonrpc/3) [vdsm.api] FINISH > > multipath_health return={} from=::ffff:172.17.117.80,38712, > > flow_id=610b2f2d, task_id=b751b6d0-bc99-457b-9620-4ffaaaa37c91 > > (api:54) > > 2022-04-25 22:41:55,166-0600 INFO  (jsonrpc/3) [api.host] FINISH > > getStats return={'status': {'code': 0, 'message': 'Done'}, > > 'info': (suppressed)} from=::ffff:172.17.117.80,38712, > > flow_id=610b2f2d (api:54) > > 2022-04-25 22:41:55,346-0600 INFO  (jsonrpc/5) [vdsm.api] START > > connectStorageServer(domType=7, spUUID='3115b136-9981-11ec-90ae- > > 00163e6f31f1', conList=[{'password': '********', 'vfs_type': > > 'glusterfs', 'port': '', 'mnt_options': 'backup-volfile- > > servers=gluster-2.[removed].com:gluster-3.[removed].com', 'iqn': > > '', 'connection': 'gluster-1.[removed].com:/engine', > > 'ipv6_enabled': 'false', 'id': '98ea63c0-e8c4-4857-8ef7- > > 3cb256d45997', 'user': '', 'tpgt': '1'}]) > > from=::ffff:172.17.117.80,38712, flow_id=77e22c1a, > > task_id=6a175be8-fdd0-4d2a-8b79-1866aff21159 (api:48) > > 2022-04-25 22:41:55,478-0600 ERROR (jsonrpc/5) > > [storage.storageServer] Could not connect to storage server > > (storageServer:92) > > Traceback (most recent call last): > >   File "/usr/lib/python3.6/site- > > packages/vdsm/storage/storageServer.py", line 90, in connect_all > >     con.connect() > >   File "/usr/lib/python3.6/site- > > packages/vdsm/storage/storageServer.py", line 233, in connect > >     self.validate() > >   File "/usr/lib/python3.6/site- > > packages/vdsm/storage/storageServer.py", line 365, in validate > >     if not self.volinfo: > >   File "/usr/lib/python3.6/site- > > packages/vdsm/storage/storageServer.py", line 352, in volinfo > >     self._volinfo = self._get_gluster_volinfo() > >   File "/usr/lib/python3.6/site- > > packages/vdsm/storage/storageServer.py", line 405, in > > _get_gluster_volinfo > >     self._volfileserver) > >   File "/usr/lib/python3.6/site- > > packages/vdsm/common/supervdsm.py", line 56, in __call__ > >     return callMethod() > >   File "/usr/lib/python3.6/site- > > packages/vdsm/common/supervdsm.py", line 54, in <lambda> > >     **kwargs) > >   File "<string>", line 2, in glusterVolumeInfo > >   File "/usr/lib64/python3.6/multiprocessing/managers.py", line > > 772, in _callmethod > >     raise convert_to_error(kind, result) > > vdsm.gluster.exception.GlusterXmlErrorException: XML error: rc=0 > > out=() err=[b'<cliOutput>\n  <opRet>0</opRet>\n  > > <opErrno>0</opErrno>\n  <opErrstr />\n  <volInfo>\n    > > <volumes>\n      <volume>\n        <name>engine</name>\n        > > <id>51bb4ddb-dfbc-4376-85cd-d7070e287946</id>\n        > > <status>1</status>\n        > > <statusStr>Started</statusStr>\n        > > <snapshotCount>0</snapshotCount>\n        > > <brickCount>3</brickCount>\n        > > <distCount>1</distCount>\n        > > <replicaCount>3</replicaCount>\n        > > <arbiterCount>0</arbiterCount>\n        > > <disperseCount>0</disperseCount>\n        > > <redundancyCount>0</redundancyCount>\n        > > <type>2</type>\n        <typeStr>Replicate</typeStr>\n        > > <transport>0</transport>\n        <bricks>\n          <brick > > uuid="1679e76e-938c-4da4-b7f7-5161c5badcd3">gluster- > > 1.[removed].com:/gluster_bricks/engine/engine<name>gluster- > > 1.[removed].com:/gluster_bricks/engine/engine</name><hostUuid>167 > > 9e76e-938c-4da4-b7f7-5161c5badcd3</hostUuid><isArbiter>0</isArb > >  iter></brick>\n          <brick uuid="530faa98-b564-45d5-8495- > > 8e4006a7628c">gluster- > > 2.[removed].com:/gluster_bricks/engine/engine<name>gluster- > > 2.[removed].com:/gluster_bricks/engine/engine</name><hostUuid>530 > > faa98-b564-45d5-8495- > > 8e4006a7628c</hostUuid><isArbiter>0</isArbiter></brick>\n         > >   <brick uuid="06c45945-8a79-476b-9a02-483113191e69">gluster- > > 3.[removed].com:/gluster_bricks/engine/engine<name>gluster- > > 3.[removed].com:/gluster_bricks/engine/engine</name><hostUuid>06c > > 45945-8a79-476b-9a02- > > 483113191e69</hostUuid><isArbiter>0</isArbiter></brick>\n        > > </bricks>\n        <optCount>31</optCount>\n        > > <options>\n          <option>\n            > > <name>cluster.granular-entry-heal</name>\n            > > <value>enable</value>\n          </option>\n          > > <option>\n            <name>storage.owner-gid</name>\n            > > <value>36</value>\n          </option>\n          > > <option>\n            <name>storage.owner-uid</name>\n            > > <value>36</value>\n          </option>\n > >     <option>\n            <name>cluster.lookup- > > optimize</name>\n            <value>off</value>\n          > > </option>\n          <option>\n            > > <name>server.keepalive-count</name>\n            > > <value>5</value>\n          </option>\n          > > <option>\n            <name>server.keepalive- > > interval</name>\n            <value>2</value>\n          > > </option>\n          <option>\n            > > <name>server.keepalive-time</name>\n            > > <value>10</value>\n          </option>\n          > > <option>\n            <name>server.tcp-user- > > timeout</name>\n            <value>20</value>\n          > > </option>\n          <option>\n            <name>network.ping- > > timeout</name>\n            <value>30</value>\n          > > </option>\n          <option>\n            <name>server.event- > > threads</name>\n            <value>4</value>\n          > > </option>\n          <option>\n            <name>client.event- > > threads</name>\n            <value>4</value>\n          > > </option>\n          <option>\n            <name>clu > >  ster.choose-local</name>\n            > > <value>off</value>\n          </option>\n          > > <option>\n            <name>user.cifs</name>\n            > > <value>off</value>\n          </option>\n          > > <option>\n            <name>features.shard</name>\n            > > <value>on</value>\n          </option>\n          > > <option>\n            <name>cluster.shd-wait- > > qlength</name>\n            <value>10000</value>\n          > > </option>\n          <option>\n            <name>cluster.shd-max- > > threads</name>\n            <value>8</value>\n          > > </option>\n          <option>\n            <name>cluster.locking- > > scheme</name>\n            <value>granular</value>\n          > > </option>\n          <option>\n            <name>cluster.data- > > self-heal-algorithm</name>\n            > > <value>full</value>\n          </option>\n          > > <option>\n            <name>cluster.server-quorum- > > type</name>\n            <value>server</value>\n          > > </option>\n          <option>\n            <name>cluster.quorum- > > type</n > >  ame>\n            <value>auto</value>\n          > > </option>\n          <option>\n            <name>cluster.eager- > > lock</name>\n            <value>enable</value>\n          > > </option>\n          <option>\n            > > <name>performance.strict-o-direct</name>\n            > > <value>on</value>\n          </option>\n          > > <option>\n            <name>network.remote- > > dio</name>\n            <value>off</value>\n          > > </option>\n          <option>\n            <name>performance.low- > > prio-threads</name>\n            <value>32</value>\n          > > </option>\n          <option>\n            <name>performance.io- > > cache</name>\n            <value>off</value>\n          > > </option>\n          <option>\n            > > <name>performance.read-ahead</name>\n            > > <value>off</value>\n          </option>\n          > > <option>\n            <name>performance.quick- > > read</name>\n            <value>off</value>\n          > > </option>\n          <option>\n            <name>storage.fips- > > mode-rchecksum</name>\n > >       <value>on</value>\n          </option>\n          > > <option>\n            <name>transport.address- > > family</name>\n            <value>inet</value>\n          > > </option>\n          <option>\n            > > <name>nfs.disable</name>\n            > > <value>on</value>\n          </option>\n          > > <option>\n            <name>performance.client-io- > > threads</name>\n            <value>on</value>\n          > > </option>\n        </options>\n      </volume>\n      > > <count>1</count>\n    </volumes>\n  </volInfo>\n</cliOutput>'] > > 2022-04-25 22:41:55,478-0600 INFO  (jsonrpc/5) > > [storage.storagedomaincache] Invalidating storage domain cache > > (sdc:74) > > 2022-04-25 22:41:55,478-0600 INFO  (jsonrpc/5) [vdsm.api] FINISH > > connectStorageServer return={'statuslist': [{'id': '98ea63c0- > > e8c4-4857-8ef7-3cb256d45997', 'status': 4106}]} > > from=::ffff:172.17.117.80,38712, flow_id=77e22c1a, > > task_id=6a175be8-fdd0-4d2a-8b79-1866aff21159 (api:54) > > 2022-04-25 22:41:55,515-0600 INFO  (jsonrpc/2) [vdsm.api] START > > connectStorageServer(domType=7, spUUID='3115b136-9981-11ec-90ae- > > 00163e6f31f1', conList=[{'password': '********', 'vfs_type': > > 'glusterfs', 'port': '', 'mnt_options': 'backup-volfile- > > servers=gluster-2.[removed].com:gluster-3.[removed].com', 'iqn': > > '', 'connection': 'gluster-1.[removed].com:/data', > > 'ipv6_enabled': 'false', 'id': '170e1dda-af02-4ff3-808b- > > 16dc814e750a', 'user': '', 'tpgt': '1'}]) > > from=::ffff:172.17.117.80,38712, flow_id=77e22c1a, > > task_id=9d0ded6d-e19e-41f8-85b4-52b18bd3380d (api:48) > > 2022-04-25 22:41:55,647-0600 ERROR (jsonrpc/2) > > [storage.storageServer] Could not connect to storage server > > (storageServer:92) > > Traceback (most recent call last): > >   File "/usr/lib/python3.6/site- > > packages/vdsm/storage/storageServer.py", line 90, in connect_all > >     con.connect() > >   File "/usr/lib/python3.6/site- > > packages/vdsm/storage/storageServer.py", line 233, in connect > >     self.validate() > >   File "/usr/lib/python3.6/site- > > packages/vdsm/storage/storageServer.py", line 365, in validate > >     if not self.volinfo: > >   File "/usr/lib/python3.6/site- > > packages/vdsm/storage/storageServer.py", line 352, in volinfo > >     self._volinfo = self._get_gluster_volinfo() > >   File "/usr/lib/python3.6/site- > > packages/vdsm/storage/storageServer.py", line 405, in > > _get_gluster_volinfo > >     self._volfileserver) > >   File "/usr/lib/python3.6/site- > > packages/vdsm/common/supervdsm.py", line 56, in __call__ > >     return callMethod() > >   File "/usr/lib/python3.6/site- > > packages/vdsm/common/supervdsm.py", line 54, in <lambda> > >     **kwargs) > >   File "<string>", line 2, in glusterVolumeInfo > >   File "/usr/lib64/python3.6/multiprocessing/managers.py", line > > 772, in _callmethod > >     raise convert_to_error(kind, result) > > vdsm.gluster.exception.GlusterXmlErrorException: XML error: rc=0 > > out=() err=[b'<cliOutput>\n  <opRet>0</opRet>\n  > > <opErrno>0</opErrno>\n  <opErrstr />\n  <volInfo>\n    > > <volumes>\n      <volume>\n        <name>data</name>\n        > > <id>06ce0d34-b4b4-472c-9cec-24ffe934ed05</id>\n        > > <status>1</status>\n        > > <statusStr>Started</statusStr>\n        > > <snapshotCount>0</snapshotCount>\n        > > <brickCount>3</brickCount>\n        > > <distCount>1</distCount>\n        > > <replicaCount>3</replicaCount>\n        > > <arbiterCount>0</arbiterCount>\n        > > <disperseCount>0</disperseCount>\n        > > <redundancyCount>0</redundancyCount>\n        > > <type>2</type>\n        <typeStr>Replicate</typeStr>\n        > > <transport>0</transport>\n        <bricks>\n          <brick > > uuid="1679e76e-938c-4da4-b7f7-5161c5badcd3">gluster- > > 1.[removed].com:/gluster_bricks/data/data<name>gluster- > > 1.[removed].com:/gluster_bricks/data/data</name><hostUuid>1679e76 > > e-938c-4da4-b7f7- > > 5161c5badcd3</hostUuid><isArbiter>0</isArbiter></bri > >  ck>\n          <brick uuid="530faa98-b564-45d5-8495- > > 8e4006a7628c">gluster- > > 2.[removed].com:/gluster_bricks/data/data<name>gluster- > > 2.[removed].com:/gluster_bricks/data/data</name><hostUuid>530faa9 > > 8-b564-45d5-8495- > > 8e4006a7628c</hostUuid><isArbiter>0</isArbiter></brick>\n         > >   <brick uuid="06c45945-8a79-476b-9a02-483113191e69">gluster- > > 3.[removed].com:/gluster_bricks/data/data<name>gluster- > > 3.[removed].com:/gluster_bricks/data/data</name><hostUuid>06c4594 > > 5-8a79-476b-9a02- > > 483113191e69</hostUuid><isArbiter>0</isArbiter></brick>\n        > > </bricks>\n        <optCount>31</optCount>\n        > > <options>\n          <option>\n            > > <name>cluster.granular-entry-heal</name>\n            > > <value>enable</value>\n          </option>\n          > > <option>\n            <name>storage.owner-gid</name>\n            > > <value>36</value>\n          </option>\n          > > <option>\n            <name>storage.owner-uid</name>\n            > > <value>36</value>\n          </option>\n          > > <option>\n            < > >  name>cluster.lookup-optimize</name>\n            > > <value>off</value>\n          </option>\n          > > <option>\n            <name>server.keepalive- > > count</name>\n            <value>5</value>\n          > > </option>\n          <option>\n            > > <name>server.keepalive-interval</name>\n            > > <value>2</value>\n          </option>\n          > > <option>\n            <name>server.keepalive- > > time</name>\n            <value>10</value>\n          > > </option>\n          <option>\n            <name>server.tcp-user- > > timeout</name>\n            <value>20</value>\n          > > </option>\n          <option>\n            <name>network.ping- > > timeout</name>\n            <value>30</value>\n          > > </option>\n          <option>\n            <name>server.event- > > threads</name>\n            <value>4</value>\n          > > </option>\n          <option>\n            <name>client.event- > > threads</name>\n            <value>4</value>\n          > > </option>\n          <option>\n            <name>cluster.choose- > > local</name>\n > >              <value>off</value>\n          </option>\n          > > <option>\n            <name>user.cifs</name>\n            > > <value>off</value>\n          </option>\n          > > <option>\n            <name>features.shard</name>\n            > > <value>on</value>\n          </option>\n          > > <option>\n            <name>cluster.shd-wait- > > qlength</name>\n            <value>10000</value>\n          > > </option>\n          <option>\n            <name>cluster.shd-max- > > threads</name>\n            <value>8</value>\n          > > </option>\n          <option>\n            <name>cluster.locking- > > scheme</name>\n            <value>granular</value>\n          > > </option>\n          <option>\n            <name>cluster.data- > > self-heal-algorithm</name>\n            > > <value>full</value>\n          </option>\n          > > <option>\n            <name>cluster.server-quorum- > > type</name>\n            <value>server</value>\n          > > </option>\n          <option>\n            <name>cluster.quorum- > > type</name>\n            <value>a > >  uto</value>\n          </option>\n          > > <option>\n            <name>cluster.eager- > > lock</name>\n            <value>enable</value>\n          > > </option>\n          <option>\n            > > <name>performance.strict-o-direct</name>\n            > > <value>on</value>\n          </option>\n          > > <option>\n            <name>network.remote- > > dio</name>\n            <value>off</value>\n          > > </option>\n          <option>\n            <name>performance.low- > > prio-threads</name>\n            <value>32</value>\n          > > </option>\n          <option>\n            <name>performance.io- > > cache</name>\n            <value>off</value>\n          > > </option>\n          <option>\n            > > <name>performance.read-ahead</name>\n            > > <value>off</value>\n          </option>\n          > > <option>\n            <name>performance.quick- > > read</name>\n            <value>off</value>\n          > > </option>\n          <option>\n            <name>storage.fips- > > mode-rchecksum</name>\n            <value>on</value>\n > >          </option>\n          <option>\n            > > <name>transport.address-family</name>\n            > > <value>inet</value>\n          </option>\n          > > <option>\n            <name>nfs.disable</name>\n            > > <value>on</value>\n          </option>\n          > > <option>\n            <name>performance.client-io- > > threads</name>\n            <value>on</value>\n          > > </option>\n        </options>\n      </volume>\n      > > <count>1</count>\n    </volumes>\n  </volInfo>\n</cliOutput>'] > > 2022-04-25 22:41:55,647-0600 INFO  (jsonrpc/2) > > [storage.storagedomaincache] Invalidating storage domain cache > > (sdc:74) > > 2022-04-25 22:41:55,647-0600 INFO  (jsonrpc/2) [vdsm.api] FINISH > > connectStorageServer return={'statuslist': [{'id': '170e1dda- > > af02-4ff3-808b-16dc814e750a', 'status': 4106}]} > > from=::ffff:172.17.117.80,38712, flow_id=77e22c1a, > > task_id=9d0ded6d-e19e-41f8-85b4-52b18bd3380d (api:54) > > 2022-04-25 22:41:55,682-0600 INFO  (jsonrpc/1) [vdsm.api] START > > connectStorageServer(domType=1, spUUID='3115b136-9981-11ec-90ae- > > 00163e6f31f1', conList=[{'password': '********', > > 'protocol_version': 'auto', 'port': '', 'iqn': '', 'connection': > > 'ovirt-2.[removed].com:/nfs2/data', 'ipv6_enabled': 'false', > > 'id': '311165ae-bfbf-4f51-994f-051aef56d94f', 'user': '', 'tpgt': > > '1'}, {'password': '********', 'protocol_version': 'auto', > > 'port': '', 'iqn': '', 'connection': 'ovirt- > > 1.[removed].com:/nfs1/data', 'ipv6_enabled': 'false', 'id': > > 'ec52f74c-a041-4e3a-9aae-5f1c6629d77f', 'user': '', 'tpgt': > > '1'}]) from=::ffff:172.17.117.80,38712, flow_id=77e22c1a, > > task_id=bc135c71-2b43-4627-b3d3-2eb0a4d25227 (api:48) > > 2022-04-25 22:41:55,688-0600 INFO  (jsonrpc/1) > > [storage.storagedomaincache] Removing domain ddeb67aa-9ec8-488b- > > 9632-5cc19a244815 from storage domain cache (sdc:211) > > 2022-04-25 22:41:55,689-0600 INFO  (jsonrpc/1) > > [storage.storagedomaincache] Removing domain e1ae9b1a-7aa4-4072- > > b92e-5e967f5a2ee7 from storage domain cache (sdc:211) > > 2022-04-25 22:41:55,689-0600 INFO  (jsonrpc/1) > > [storage.storagedomaincache] Invalidating storage domain cache > > (sdc:74) > > 2022-04-25 22:41:55,689-0600 INFO  (jsonrpc/1) [vdsm.api] FINISH > > connectStorageServer return={'statuslist': [{'id': '311165ae- > > bfbf-4f51-994f-051aef56d94f', 'status': 0}, {'id': 'ec52f74c- > > a041-4e3a-9aae-5f1c6629d77f', 'status': 0}]} > > from=::ffff:172.17.117.80,38712, flow_id=77e22c1a, > > task_id=bc135c71-2b43-4627-b3d3-2eb0a4d25227 (api:54) > > 2022-04-25 22:42:01,696-0600 INFO  (periodic/0) [vdsm.api] START > > repoStats(domains=()) from=internal, task_id=9b42900c-cfd0-4e56- > > b6f8-07a340497993 (api:48) > > 2022-04-25 22:42:01,697-0600 INFO  (periodic/0) [vdsm.api] FINISH > > repoStats return={} from=internal, task_id=9b42900c-cfd0-4e56- > > b6f8-07a340497993 (api:54) > > 2022-04-25 22:42:04,539-0600 INFO  (jsonrpc/6) [api.host] START > > getStats() from=::ffff:172.17.117.80,38712 (api:48) > > 2022-04-25 22:42:04,577-0600 INFO  (jsonrpc/6) [vdsm.api] START > > repoStats(domains=()) from=::ffff:172.17.117.80,38712, > > task_id=a97abe37-ce3b-4745-8cfc-89d0d01bfc74 (api:48) > > 2022-04-25 22:42:04,577-0600 INFO  (jsonrpc/6) [vdsm.api] FINISH > > repoStats return={} from=::ffff:172.17.117.80,38712, > > task_id=a97abe37-ce3b-4745-8cfc-89d0d01bfc74 (api:54) > > 2022-04-25 22:42:04,578-0600 INFO  (jsonrpc/6) [vdsm.api] START > > multipath_health() from=::ffff:172.17.117.80,38712, > > task_id=b054a35e-e4ea-4402-963b-5b0416a30692 (api:48) > > 2022-04-25 22:42:04,578-0600 INFO  (jsonrpc/6) [vdsm.api] FINISH > > multipath_health return={} from=::ffff:172.17.117.80,38712, > > task_id=b054a35e-e4ea-4402-963b-5b0416a30692 (api:54) > > 2022-04-25 22:42:04,583-0600 INFO  (jsonrpc/6) [api.host] FINISH > > getStats return={'status': {'code': 0, 'message': 'Done'}, > > 'info': (suppressed)} from=::ffff:172.17.117.80,38712 (api:54) > > _______________________________________________ > > Users mailing list -- users(a)ovirt.org > > To unsubscribe send an email to users-leave(a)ovirt.org > > Privacy Statement: https://www.ovirt.org/privacy-policy.html > > oVirt Code of Conduct: > > https://www.ovirt.org/community/about/community-guidelines/ > > List Archives: > > https://lists.ovirt.org/archives/list/users@ovirt.org/message/X625FWXPUML... > > > > -- > > Sandro Bonazzola > > MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV > > Red Hat EMEA > > sbonazzo(a)redhat.com > > > Red Hat respects your work life balance. Therefore there is no need > to answer this email out of your office hours. > > > > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/J2VAJDLG7E5... > > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/6Y3HUNG2VAQ... _______________________________________________ Users mailing list -- users(a)ovirt.org To unsubscribe send an email to users-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/7XMJY44ZGBK...