[VDSM] network test failure - failing all builds
I'm seeing now this failure on multiple untreated patches. Seems to be related to these patches merged today: https://gerrit.ovirt.org/#/q/topic:ip_refactoring+is:merged *21:04:36* ======================================================================*21:04:36* FAIL: test_add_delete_and_read_rule (network.ip_rule_test.TestIpRule)*21:04:36* ----------------------------------------------------------------------*21:04:36* Traceback (most recent call last):*21:04:36* File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/network/ip_rule_test.py", line 47, in test_add_delete_and_read_rule*21:04:36* self.assertEqual(1, len(rules))*21:04:36* AssertionError: 1 != 2*21:04:36* -------------------- >> begin captured logging << --------------------*21:04:36* 2017-05-12 21:04:18,381 DEBUG (MainThread) [root] /usr/bin/taskset --cpu-list 0-1 /sbin/ip rule add from all to 192.168.99.1 dev lo table main (cwd None) (commands:69)*21:04:36* 2017-05-12 21:04:18,393 DEBUG (MainThread) [root] SUCCESS: <err> = ''; <rc> = 0 (commands:93)*21:04:36* 2017-05-12 21:04:18,393 DEBUG (MainThread) [root] /usr/bin/taskset --cpu-list 0-1 /sbin/ip rule (cwd None) (commands:69)*21:04:36* 2017-05-12 21:04:18,404 DEBUG (MainThread) [root] SUCCESS: <err> = ''; <rc> = 0 (commands:93)*21:04:36* 2017-05-12 21:04:18,406 DEBUG (MainThread) [root] /usr/bin/taskset --cpu-list 0-1 /sbin/ip rule del from all to 192.168.99.1 dev lo table main (cwd None) (commands:69)*21:04:36* 2017-05-12 21:04:18,417 DEBUG (MainThread) [root] SUCCESS: <err> = ''; <rc> = 0 (commands:93)*21:04:36* --------------------- >> end captured logging << ---------------------
Test trends: http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc25-x86_64/buildTimeTr... http://jenkins.ovirt.org/job/vdsm_master_check-patch-el7-x86_64/buildTimeTre... <http://jenkins.ovirt.org/view/01%20Unstable%20Jobs%20(ALL)/job/vdsm_master_check-patch-el7-x86_64/buildTimeTrend> On Sat, May 13, 2017 at 12:16 AM Nir Soffer <nsoffer@redhat.com> wrote:
I'm seeing now this failure on multiple untreated patches.
Seems to be related to these patches merged today: https://gerrit.ovirt.org/#/q/topic:ip_refactoring+is:merged
*21:04:36* ======================================================================*21:04:36* FAIL: test_add_delete_and_read_rule (network.ip_rule_test.TestIpRule)*21:04:36* ----------------------------------------------------------------------*21:04:36* Traceback (most recent call last):*21:04:36* File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/network/ip_rule_test.py", line 47, in test_add_delete_and_read_rule*21:04:36* self.assertEqual(1, len(rules))*21:04:36* AssertionError: 1 != 2*21:04:36* -------------------- >> begin captured logging << --------------------*21:04:36* 2017-05-12 21:04:18,381 DEBUG (MainThread) [root] /usr/bin/taskset --cpu-list 0-1 /sbin/ip rule add from all to 192.168.99.1 dev lo table main (cwd None) (commands:69)*21:04:36* 2017-05-12 21:04:18,393 DEBUG (MainThread) [root] SUCCESS: <err> = ''; <rc> = 0 (commands:93)*21:04:36* 2017-05-12 21:04:18,393 DEBUG (MainThread) [root] /usr/bin/taskset --cpu-list 0-1 /sbin/ip rule (cwd None) (commands:69)*21:04:36* 2017-05-12 21:04:18,404 DEBUG (MainThread) [root] SUCCESS: <err> = ''; <rc> = 0 (commands:93)*21:04:36* 2017-05-12 21:04:18,406 DEBUG (MainThread) [root] /usr/bin/taskset --cpu-list 0-1 /sbin/ip rule del from all to 192.168.99.1 dev lo table main (cwd None) (commands:69)*21:04:36* 2017-05-12 21:04:18,417 DEBUG (MainThread) [root] SUCCESS: <err> = ''; <rc> = 0 (commands:93)*21:04:36* --------------------- >> end captured logging << ---------------------
Very strange (and very late), trying to simulate locally. On Sat, May 13, 2017 at 12:22 AM, Nir Soffer <nsoffer@redhat.com> wrote:
Test trends: http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc25- x86_64/buildTimeTrend http://jenkins.ovirt.org/job/vdsm_master_check-patch-el7- x86_64/buildTimeTrend <http://jenkins.ovirt.org/view/01%20Unstable%20Jobs%20(ALL)/job/vdsm_master_check-patch-el7-x86_64/buildTimeTrend>
On Sat, May 13, 2017 at 12:16 AM Nir Soffer <nsoffer@redhat.com> wrote:
I'm seeing now this failure on multiple untreated patches.
Seems to be related to these patches merged today: https://gerrit.ovirt.org/#/q/topic:ip_refactoring+is:merged
*21:04:36* ======================================================================*21:04:36* FAIL: test_add_delete_and_read_rule (network.ip_rule_test.TestIpRule)*21:04:36* ----------------------------------------------------------------------*21:04:36* Traceback (most recent call last):*21:04:36* File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/network/ip_rule_test.py", line 47, in test_add_delete_and_read_rule*21:04:36* self.assertEqual(1, len(rules))*21:04:36* AssertionError: 1 != 2*21:04:36* -------------------- >> begin captured logging << --------------------*21:04:36* 2017-05-12 21:04:18,381 DEBUG (MainThread) [root] /usr/bin/taskset --cpu-list 0-1 /sbin/ip rule add from all to 192.168.99.1 dev lo table main (cwd None) (commands:69)*21:04:36* 2017-05-12 21:04:18,393 DEBUG (MainThread) [root] SUCCESS: <err> = ''; <rc> = 0 (commands:93)*21:04:36* 2017-05-12 21:04:18,393 DEBUG (MainThread) [root] /usr/bin/taskset --cpu-list 0-1 /sbin/ip rule (cwd None) (commands:69)*21:04:36* 2017-05-12 21:04:18,404 DEBUG (MainThread) [root] SUCCESS: <err> = ''; <rc> = 0 (commands:93)*21:04:36* 2017-05-12 21:04:18,406 DEBUG (MainThread) [root] /usr/bin/taskset --cpu-list 0-1 /sbin/ip rule del from all to 192.168.99.1 dev lo table main (cwd None) (commands:69)*21:04:36* 2017-05-12 21:04:18,417 DEBUG (MainThread) [root] SUCCESS: <err> = ''; <rc> = 0 (commands:93)*21:04:36* --------------------- >> end captured logging << ---------------------
It may have been a patch (not particular one that got merged) that messed up the ci hosts. Looks better now, recent checks passed. Will try to monitor it later on. Thanks, Edy. On Sat, May 13, 2017 at 12:45 AM, Edward Haas <ehaas@redhat.com> wrote:
Very strange (and very late), trying to simulate locally.
On Sat, May 13, 2017 at 12:22 AM, Nir Soffer <nsoffer@redhat.com> wrote:
Test trends: http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc25-x8 6_64/buildTimeTrend http://jenkins.ovirt.org/job/vdsm_master_check-patch-el7-x86 _64/buildTimeTrend <http://jenkins.ovirt.org/view/01%20Unstable%20Jobs%20(ALL)/job/vdsm_master_check-patch-el7-x86_64/buildTimeTrend>
On Sat, May 13, 2017 at 12:16 AM Nir Soffer <nsoffer@redhat.com> wrote:
I'm seeing now this failure on multiple untreated patches.
Seems to be related to these patches merged today: https://gerrit.ovirt.org/#/q/topic:ip_refactoring+is:merged
*21:04:36* ======================================================================*21:04:36* FAIL: test_add_delete_and_read_rule (network.ip_rule_test.TestIpRule)*21:04:36* ----------------------------------------------------------------------*21:04:36* Traceback (most recent call last):*21:04:36* File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/network/ip_rule_test.py", line 47, in test_add_delete_and_read_rule*21:04:36* self.assertEqual(1, len(rules))*21:04:36* AssertionError: 1 != 2*21:04:36* -------------------- >> begin captured logging << --------------------*21:04:36* 2017-05-12 21:04:18,381 DEBUG (MainThread) [root] /usr/bin/taskset --cpu-list 0-1 /sbin/ip rule add from all to 192.168.99.1 dev lo table main (cwd None) (commands:69)*21:04:36* 2017-05-12 21:04:18,393 DEBUG (MainThread) [root] SUCCESS: <err> = ''; <rc> = 0 (commands:93)*21:04:36* 2017-05-12 21:04:18,393 DEBUG (MainThread) [root] /usr/bin/taskset --cpu-list 0-1 /sbin/ip rule (cwd None) (commands:69)*21:04:36* 2017-05-12 21:04:18,404 DEBUG (MainThread) [root] SUCCESS: <err> = ''; <rc> = 0 (commands:93)*21:04:36* 2017-05-12 21:04:18,406 DEBUG (MainThread) [root] /usr/bin/taskset --cpu-list 0-1 /sbin/ip rule del from all to 192.168.99.1 dev lo table main (cwd None) (commands:69)*21:04:36* 2017-05-12 21:04:18,417 DEBUG (MainThread) [root] SUCCESS: <err> = ''; <rc> = 0 (commands:93)*21:04:36* --------------------- >> end captured logging << ---------------------
Tried to handle this using: https://gerrit.ovirt.org/#/c/76781 Thanks, Edy. On Sat, May 13, 2017 at 1:02 AM, Edward Haas <ehaas@redhat.com> wrote:
It may have been a patch (not particular one that got merged) that messed up the ci hosts. Looks better now, recent checks passed.
Will try to monitor it later on.
Thanks, Edy.
On Sat, May 13, 2017 at 12:45 AM, Edward Haas <ehaas@redhat.com> wrote:
Very strange (and very late), trying to simulate locally.
On Sat, May 13, 2017 at 12:22 AM, Nir Soffer <nsoffer@redhat.com> wrote:
Test trends: http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc25-x8 6_64/buildTimeTrend http://jenkins.ovirt.org/job/vdsm_master_check-patch-el7-x86 _64/buildTimeTrend <http://jenkins.ovirt.org/view/01%20Unstable%20Jobs%20(ALL)/job/vdsm_master_check-patch-el7-x86_64/buildTimeTrend>
On Sat, May 13, 2017 at 12:16 AM Nir Soffer <nsoffer@redhat.com> wrote:
I'm seeing now this failure on multiple untreated patches.
Seems to be related to these patches merged today: https://gerrit.ovirt.org/#/q/topic:ip_refactoring+is:merged
*21:04:36* ======================================================================*21:04:36* FAIL: test_add_delete_and_read_rule (network.ip_rule_test.TestIpRule)*21:04:36* ----------------------------------------------------------------------*21:04:36* Traceback (most recent call last):*21:04:36* File "/home/jenkins/workspace/vdsm_master_check-patch-el7-x86_64/vdsm/tests/network/ip_rule_test.py", line 47, in test_add_delete_and_read_rule*21:04:36* self.assertEqual(1, len(rules))*21:04:36* AssertionError: 1 != 2*21:04:36* -------------------- >> begin captured logging << --------------------*21:04:36* 2017-05-12 21:04:18,381 DEBUG (MainThread) [root] /usr/bin/taskset --cpu-list 0-1 /sbin/ip rule add from all to 192.168.99.1 dev lo table main (cwd None) (commands:69)*21:04:36* 2017-05-12 21:04:18,393 DEBUG (MainThread) [root] SUCCESS: <err> = ''; <rc> = 0 (commands:93)*21:04:36* 2017-05-12 21:04:18,393 DEBUG (MainThread) [root] /usr/bin/taskset --cpu-list 0-1 /sbin/ip rule (cwd None) (commands:69)*21:04:36* 2017-05-12 21:04:18,404 DEBUG (MainThread) [root] SUCCESS: <err> = ''; <rc> = 0 (commands:93)*21:04:36* 2017-05-12 21:04:18,406 DEBUG (MainThread) [root] /usr/bin/taskset --cpu-list 0-1 /sbin/ip rule del from all to 192.168.99.1 dev lo table main (cwd None) (commands:69)*21:04:36* 2017-05-12 21:04:18,417 DEBUG (MainThread) [root] SUCCESS: <err> = ''; <rc> = 0 (commands:93)*21:04:36* --------------------- >> end captured logging << ---------------------
participants (2)
-
Edward Haas -
Nir Soffer