Re: [ovirt-devel] URGENT! OST broken! please stop merging patches to ovirt-engine!
On Aug 30, 2017 8:46 AM, "Maor Lipchuk" <mlipchuk@redhat.com> wrote: Yes, there is already an acked patch that solves it. The weird thing is that the OST ran successfully when running with "ci please build" although it seems that the artifact was not added to it. That doesn't run full OST. It only runs build-artifacts :) On Wed, Aug 30, 2017 at 3:17 PM, Greg Sheremeta <gshereme@redhat.com> wrote:
On Wed, Aug 30, 2017 at 7:13 AM, Barak Korren <bkorren@redhat.com> wrote:
We have two unresolved issues that are currently causing all master OST runs to fail and preventing us from effectively finding regressions.
The 1st issue, which was already reported on Sunday, is a regression that is causing vm_run to fail and was probably introduced by this patch: https://gerrit.ovirt.org/#/c/79033/41
cc'd Shubham and Maor. Is anyone looking into this?
Here is a recent run that failed with this: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2147/
Logs are here:
tester/2147/artifact/exported-artifacts/basic-suit-master- el7/test_logs/basic-suite-master/post-002_bootstrap.py/
The 2nd issue is a new one, that seems to be causing add-host failures and I've just send another email about it. The gist of this is that I'm siing the following failures in the supervdsm logs, and vdsm is not loading:
MainThread::ERROR::2017-08-30 05:55:59,476::initializer::53::root::(_lldp_init) Failed to enable LLDP on eth1 Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/network/initializer.py", line 51, in _lldp_init Lldp.enable_lldp_on_iface(device) File "/usr/lib/python2.7/site-packages/vdsm/network/lldp/lldpad.py", line 30, in enable_lldp_on_iface lldptool.enable_lldp_on_iface(iface, rx_only) File "/usr/lib/python2.7/site-packages/vdsm/network/lldpad/
lldptool.py",
line 46, in enable_lldp_on_iface raise EnableLldpError(rc, out, err, iface) EnableLldpError: (1, "timeout\n'M00000001C3040000000c04eth1000badminStatus0002rx' command timed out.\n", '', 'eth1') MainThread::DEBUG::2017-08-30 05:55:59,477::cmdutils::133::root::(exec_cmd) /usr/sbin/lldptool get-lldp -i eth0 adminStatus (cwd None)
This failure does not seem to have been introduced by the platform and not oVirt code, because it also happens on code that already passed OST a few days ago.
Please avoid merging any patches except to the purpose of resolving these issues. Nothing is making it into the 'tested' and nightly snapshot repos anyway ATM.
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
I also ran the OST, see http://jenkins.ovirt.org/job/ovirt-system-tests_manual/1014/parameters/ Any how the fixes patch was merged: https://gerrit.ovirt.org/#/c/81198/ On Wed, Aug 30, 2017 at 4:01 PM, Greg Sheremeta <gshereme@redhat.com> wrote:
On Aug 30, 2017 8:46 AM, "Maor Lipchuk" <mlipchuk@redhat.com> wrote:
Yes, there is already an acked patch that solves it. The weird thing is that the OST ran successfully when running with "ci please build" although it seems that the artifact was not added to it.
That doesn't run full OST. It only runs build-artifacts :)
On Wed, Aug 30, 2017 at 3:17 PM, Greg Sheremeta <gshereme@redhat.com> wrote:
On Wed, Aug 30, 2017 at 7:13 AM, Barak Korren <bkorren@redhat.com> wrote:
We have two unresolved issues that are currently causing all master OST runs to fail and preventing us from effectively finding regressions.
The 1st issue, which was already reported on Sunday, is a regression that is causing vm_run to fail and was probably introduced by this patch: https://gerrit.ovirt.org/#/c/79033/41
cc'd Shubham and Maor. Is anyone looking into this?
Here is a recent run that failed with this: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2147/
Logs are here:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2147/artifact/...
The 2nd issue is a new one, that seems to be causing add-host failures and I've just send another email about it. The gist of this is that I'm siing the following failures in the supervdsm logs, and vdsm is not loading:
MainThread::ERROR::2017-08-30 05:55:59,476::initializer::53::root::(_lldp_init) Failed to enable LLDP on eth1 Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/network/initializer.py", line 51, in _lldp_init Lldp.enable_lldp_on_iface(device) File "/usr/lib/python2.7/site-packages/vdsm/network/lldp/lldpad.py", line 30, in enable_lldp_on_iface lldptool.enable_lldp_on_iface(iface, rx_only) File "/usr/lib/python2.7/site-packages/vdsm/network/lldpad/lldptool.py", line 46, in enable_lldp_on_iface raise EnableLldpError(rc, out, err, iface) EnableLldpError: (1, "timeout\n'M00000001C3040000000c04eth1000badminStatus0002rx' command timed out.\n", '', 'eth1') MainThread::DEBUG::2017-08-30 05:55:59,477::cmdutils::133::root::(exec_cmd) /usr/sbin/lldptool get-lldp -i eth0 adminStatus (cwd None)
This failure does not seem to have been introduced by the platform and not oVirt code, because it also happens on code that already passed OST a few days ago.
Please avoid merging any patches except to the purpose of resolving these issues. Nothing is making it into the 'tested' and nightly snapshot repos anyway ATM.
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
participants (2)
-
Greg Sheremeta -
Maor Lipchuk