Re: [ovirt-devel] [VDSM] New network test failure - ctypes.ArgumentError: argument 3: <class 'TypeError'>: wrong type

I think we should get rid of the ctypes code, it is very hard to understand, unsafe, and slow. If we need to call C we should use proper python extensions.

On Thu, Sep 22, 2016 at 5:34 PM, Piotr Kliczewski <pkliczew(a)redhat.com&gt; wrote: > > > On Thu, Sep 22, 2016 at 9:58 AM, Dan Kenigsberg <danken(a)redhat.com&gt; wrote: > >> On Wed, Sep 21, 2016 at 12:18:37AM +0300, Nir Soffer wrote: >> > On Tue, Sep 20, 2016 at 8:18 PM, Nir Soffer <nsoffer(a)redhat.com&gt; wrote: >> > > Failed only one patch when building 4, so this looks like >> > > a random failure, but the error is disturbing. >> >> Petr, could you look into it? Could it be that libnl3 changed their API? >> Or that a slave is pulling an ancient lib? Or broken kernel? Nothing >> makes sense. >> >> > > >> > > See http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc24-x8 >> 6_64/2255/console >> > > >> > > 17:00:31 ============================================================ >> ========== >> > > 17:00:31 ERROR: test_bond_create_failure_on_slave_add >> > > (network.link_bond_test.LinkBondTests) >> > > 17:00:31 ------------------------------------------------------------ >> ---------- >> > > 17:00:31 Traceback (most recent call last): >> > > 17:00:31 File >> > > "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64 >> /vdsm/tests/network/link_bond_test.py", >> > > line 83, in test_bond_create_failure_on_slave_add >> > > 17:00:31 base_bond.add_slaves((nic1, nic2)) >> > > 17:00:31 File >> > > "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64 >> /vdsm/lib/vdsm/network/link/bond.py", >> > > line 153, in add_slaves >> > > 17:00:31 with _preserve_iface_state(slave): >> > > 17:00:31 File "/usr/lib64/python3.5/contextlib.py", line 59, in >> __enter__ >> > > 17:00:31 return next(self.gen) >> > > 17:00:31 File >> > > "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64 >> /vdsm/lib/vdsm/network/link/bond.py", >> > > line 214, in _preserve_iface_state >> > > 17:00:31 dev_was_up = iface.is_up(dev) >> > > 17:00:31 File >> > > "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64 >> /vdsm/lib/vdsm/network/link/iface.py", >> > > line 54, in is_up >> > > 17:00:31 return is_admin_up(dev) >> > > 17:00:31 File >> > > "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64 >> /vdsm/lib/vdsm/network/link/iface.py", >> > > line 58, in is_admin_up >> > > 17:00:31 return is_link_up(get_link(dev)['flags'], >> check_oper_status=False) >> > > 17:00:31 File >> > > "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64 >> /vdsm/lib/vdsm/network/netlink/link.py", >> > > line 56, in get_link >> > > 17:00:31 with _get_link(name=name, sock=sock) as link: >> > > 17:00:31 File "/usr/lib64/python3.5/contextlib.py", line 59, in >> __enter__ >> > > 17:00:31 return next(self.gen) >> > > 17:00:31 File >> > > "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64 >> /vdsm/lib/vdsm/network/netlink/link.py", >> > > line 199, in _get_link >> > > 17:00:31 err = _rtnl_link_get_kernel(sock, index, name, >> byref(link)) >> > > 17:00:31 ctypes.ArgumentError: argument 3: <class 'TypeError'>: wrong >> type >> > > 17:00:31 -------------------- >> begin captured logging << >> -------------------- >> > > 17:00:31 2016-09-20 17:00:12,586 WARNING [py.warnings] (MainThread) >> > > /usr/lib/python3.5/site-packages/nose/util.py:453: >> DeprecationWarning: >> > > inspect.getargspec() is deprecated, use inspect.signature() instead >> > > 17:00:31 inspect.getargspec(func) >> > > 17:00:31 (inspect:1041) >> >> Piotr, the DeprecationWarning is unrelated to our code but we are using >> this API, too. >> >> > Please take a look at https://gerrit.ovirt.org/64313 > > >> > > 17:00:31 2016-09-20 17:00:12,587 INFO [root] (MainThread) Bond >> > > check_7lzvb has been created. (bond:142) >> > > 17:00:31 2016-09-20 17:00:12,598 INFO [root] (MainThread) Bond >> > > check_7lzvb has been destroyed. (bond:149) >> > > 17:00:31 2016-09-20 17:00:12,598 DEBUG [root] (MainThread) >> > > /usr/bin/taskset --cpu-list 0-15 /sbin/ip link add name dummy_Dmwc5 >> > > type dummy (cwd None) (commands:69) >> > > 17:00:31 2016-09-20 17:00:12,607 DEBUG [root] (MainThread) SUCCESS: >> > > <err> = b''; <rc> = 0 (commands:93) >> > > 17:00:31 2016-09-20 17:00:12,608 DEBUG [root] (MainThread) >> > > /usr/bin/taskset --cpu-list 0-15 /sbin/ip link add name dummy_xIK5C >> > > type dummy (cwd None) (commands:69) >> > > 17:00:31 2016-09-20 17:00:12,615 DEBUG [root] (MainThread) SUCCESS: >> > > <err> = b''; <rc> = 0 (commands:93) >> > > 17:00:31 2016-09-20 17:00:12,617 INFO [root] (MainThread) Bond >> > > bond_rwTTFu has been created. (bond:142) >> > > 17:00:31 2016-09-20 17:00:12,634 INFO [root] (MainThread) Bond >> > > bond_rwTTFu has been destroyed. (bond:149) >> > > 17:00:31 2016-09-20 17:00:12,634 DEBUG [root] (MainThread) >> > > /usr/bin/taskset --cpu-list 0-15 /sbin/ip link del dev dummy_Dmwc5 >> > > (cwd None) (commands:69) >> > > 17:00:31 2016-09-20 17:00:12,658 DEBUG [root] (MainThread) SUCCESS: >> > > <err> = b''; <rc> = 0 (commands:93) >> > > 17:00:31 2016-09-20 17:00:12,659 DEBUG [root] (MainThread) >> > > /usr/bin/taskset --cpu-list 0-15 /sbin/ip link del dev dummy_xIK5C >> > > (cwd None) (commands:69) >> > > 17:00:31 2016-09-20 17:00:12,676 DEBUG [root] (MainThread) SUCCESS: >> > > <err> = b''; <rc> = 0 (commands:93) >> > > 17:00:31 --------------------- >> end captured logging << >> --------------------- >> > >> > Another instance, again only one patch failed when building 5 patches >> series. >> > >> > This time with new log format for the tests. >> > >> > 18:56:15 ============================================================ >> ========== >> > 18:56:15 ERROR: test_bond_create_failure_on_slave_add >> > (network.link_bond_test.LinkBondTests) >> > 18:56:15 ------------------------------------------------------------ >> ---------- >> > 18:56:15 Traceback (most recent call last): >> > 18:56:15 File >> > "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64 >> /vdsm/tests/network/link_bond_test.py", >> > line 83, in test_bond_create_failure_on_slave_add >> > 18:56:15 base_bond.add_slaves((nic1, nic2)) >> > 18:56:15 File >> > "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64 >> /vdsm/lib/vdsm/network/link/bond.py", >> > line 153, in add_slaves >> > 18:56:15 with _preserve_iface_state(slave): >> > 18:56:15 File "/usr/lib64/python3.5/contextlib.py", line 59, in >> __enter__ >> > 18:56:15 return next(self.gen) >> > 18:56:15 File >> > "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64 >> /vdsm/lib/vdsm/network/link/bond.py", >> > line 214, in _preserve_iface_state >> > 18:56:15 dev_was_up = iface.is_up(dev) >> > 18:56:15 File >> > "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64 >> /vdsm/lib/vdsm/network/link/iface.py", >> > line 54, in is_up >> > 18:56:15 return is_admin_up(dev) >> > 18:56:15 File >> > "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64 >> /vdsm/lib/vdsm/network/link/iface.py", >> > line 58, in is_admin_up >> > 18:56:15 return is_link_up(get_link(dev)['flags'], >> check_oper_status=False) >> > 18:56:15 File >> > "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64 >> /vdsm/lib/vdsm/network/netlink/link.py", >> > line 56, in get_link >> > 18:56:15 with _get_link(name=name, sock=sock) as link: >> > 18:56:15 File "/usr/lib64/python3.5/contextlib.py", line 59, in >> __enter__ >> > 18:56:15 return next(self.gen) >> > 18:56:15 File >> > "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64 >> /vdsm/lib/vdsm/network/netlink/link.py", >> > line 199, in _get_link >> > 18:56:15 err = _rtnl_link_get_kernel(sock, index, name, byref(link)) >> > 18:56:15 ctypes.ArgumentError: argument 3: <class 'TypeError'>: wrong >> type >> > 18:56:15 -------------------- >> begin captured logging << >> -------------------- >> > 18:56:15 2016-09-20 18:55:55,220 WARNING [py.warnings] (MainThread) >> > /usr/lib/python3.5/site-packages/nose/util.py:453: DeprecationWarning: >> > inspect.getargspec() is deprecated, use inspect.signature() instead >> > 18:56:15 inspect.getargspec(func) >> > 18:56:15 (inspect:1041) >> > 18:56:15 2016-09-20 18:55:55,222 INFO [root] (MainThread) Bond >> > check_pkopm has been created. (bond:142) >> > 18:56:15 2016-09-20 18:55:55,233 INFO [root] (MainThread) Bond >> > check_pkopm has been destroyed. (bond:149) >> > 18:56:15 2016-09-20 18:55:55,234 DEBUG [root] (MainThread) >> > /usr/bin/taskset --cpu-list 0-3 /sbin/ip link add name dummy_gmayN >> > type dummy (cwd None) (commands:69) >> > 18:56:15 2016-09-20 18:55:55,242 DEBUG [root] (MainThread) SUCCESS: >> > <err> = b''; <rc> = 0 (commands:93) >> > 18:56:15 2016-09-20 18:55:55,242 DEBUG [root] (MainThread) >> > /usr/bin/taskset --cpu-list 0-3 /sbin/ip link add name dummy_p3yZR >> > type dummy (cwd None) (commands:69) >> > 18:56:15 2016-09-20 18:55:55,250 DEBUG [root] (MainThread) SUCCESS: >> > <err> = b''; <rc> = 0 (commands:93) >> > 18:56:15 2016-09-20 18:55:55,258 INFO [root] (MainThread) Bond >> > bond_Siiakr has been created. (bond:142) >> > 18:56:15 2016-09-20 18:55:55,265 INFO [root] (MainThread) Bond >> > bond_Siiakr has been destroyed. (bond:149) >> > 18:56:15 2016-09-20 18:55:55,265 DEBUG [root] (MainThread) >> > /usr/bin/taskset --cpu-list 0-3 /sbin/ip link del dev dummy_gmayN (cwd >> > None) (commands:69) >> > 18:56:15 2016-09-20 18:55:55,277 DEBUG [root] (MainThread) SUCCESS: >> > <err> = b''; <rc> = 0 (commands:93) >> > 18:56:15 2016-09-20 18:55:55,278 DEBUG [root] (MainThread) >> > /usr/bin/taskset --cpu-list 0-3 /sbin/ip link del dev dummy_p3yZR (cwd >> > None) (commands:69) >> > 18:56:15 2016-09-20 18:55:55,289 DEBUG [root] (MainThread) SUCCESS: >> > <err> = b''; <rc> = 0 (commands:93) >> > 18:56:15 --------------------- >> end captured logging << >> --------------------- >> > >