[Lago][HC] tuned service fails to start on one of HC hosts

One of the host fails to install with Failed to install Host lago-hc-basic-suite-master-host2. Failed to execute stage 'Misc configuration': Failed to start service 'tuned' The installation on other 2 hosts have been successful - all hosts are created from same template/repo. So this is strange. On Wed, Jun 28, 2017 at 8:58 AM, <jenkins@jenkins.phx.ovirt.org> wrote:
Project: http://jenkins.ovirt.org/job/ovirt_master_hc-system-tests/ Build: http://jenkins.ovirt.org/job/ovirt_master_hc-system-tests/154/ Build Number: 154 Build Status: Failure Triggered By: Started by timer
------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #154 [Daniel Belenky] Exclude packages from ovirt-master.repo
[Barak Korren] Filter builds sent to change queues by version
[Barak Korren] Make change queue invoke OST basic suit
[Barak Korren] Various UX improvements in change-queue jobs
[Barak Korren] Add a job for direct deployments to 'tested'
[Barak Korren] Use new job to update 'tested' from experimental
[Barak Korren] Use new tested deploy job in timed builders
[Eyal Edri] update build retention policy for big artifacts
----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 002_bootstrap.add_hosts
Error Message: Host lago-hc-basic-suite-master-host2 failed to install -------------------- >> begin captured logging << -------------------- ovirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_up at 0x3f11cf8> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 217, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt_master_hc-system-tests/ ovirt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 151, in _host_is_up raise RuntimeError('Host %s failed to install' % host.name()) RuntimeError: Host lago-hc-basic-suite-master-host2 failed to install --------------------- >> end captured logging << ---------------------
Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt_master_hc-system-tests/ ovirt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 164, in add_hosts testlib.assert_true_within(_host_is_up, timeout=15 * 60) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 256, in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 217, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt_master_hc-system-tests/ ovirt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 151, in _host_is_up raise RuntimeError('Host %s failed to install' % host.name()) Host lago-hc-basic-suite-master-host2 failed to install -------------------- >> begin captured logging << -------------------- ovirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_up at 0x3f11cf8> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 217, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt_master_hc-system-tests/ ovirt-system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 151, in _host_is_up raise RuntimeError('Host %s failed to install' % host.name()) RuntimeError: Host lago-hc-basic-suite-master-host2 failed to install --------------------- >> end captured logging << ---------------------

On Wed, Jun 28, 2017 at 8:22 AM, Sahina Bose <sabose@redhat.com> wrote:
One of the host fails to install with
Failed to install Host lago-hc-basic-suite-master-host2. Failed to execute stage 'Misc configuration': Failed to start service 'tuned'
The installation on other 2 hosts have been successful - all hosts are created from same template/repo. So this is strange.
https://bugzilla.redhat.com/show_bug.cgi?id=1258868#c5 Please escalate if need, so we'll get it fixed. Y.
On Wed, Jun 28, 2017 at 8:58 AM, <jenkins@jenkins.phx.ovirt.org> wrote:
Project: http://jenkins.ovirt.org/job/ovirt_master_hc-system-tests/ Build: http://jenkins.ovirt.org/job/ovirt_master_hc-system-tests/154/ Build Number: 154 Build Status: Failure Triggered By: Started by timer
------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #154 [Daniel Belenky] Exclude packages from ovirt-master.repo
[Barak Korren] Filter builds sent to change queues by version
[Barak Korren] Make change queue invoke OST basic suit
[Barak Korren] Various UX improvements in change-queue jobs
[Barak Korren] Add a job for direct deployments to 'tested'
[Barak Korren] Use new job to update 'tested' from experimental
[Barak Korren] Use new tested deploy job in timed builders
[Eyal Edri] update build retention policy for big artifacts
----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 002_bootstrap.add_hosts
Error Message: Host lago-hc-basic-suite-master-host2 failed to install -------------------- >> begin captured logging << -------------------- ovirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_up at 0x3f11cf8> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 217, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt- system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 151, in _host_is_up raise RuntimeError('Host %s failed to install' % host.name()) RuntimeError: Host lago-hc-basic-suite-master-host2 failed to install --------------------- >> end captured logging << ---------------------
Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt- system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 164, in add_hosts testlib.assert_true_within(_host_is_up, timeout=15 * 60) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 256, in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 217, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt- system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 151, in _host_is_up raise RuntimeError('Host %s failed to install' % host.name()) Host lago-hc-basic-suite-master-host2 failed to install -------------------- >> begin captured logging << -------------------- ovirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_up at 0x3f11cf8> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 217, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt- system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 151, in _host_is_up raise RuntimeError('Host %s failed to install' % host.name()) RuntimeError: Host lago-hc-basic-suite-master-host2 failed to install --------------------- >> end captured logging << ---------------------
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel

On Wed, Jun 28, 2017 at 12:20 PM, Yaniv Kaul <ykaul@redhat.com> wrote:
On Wed, Jun 28, 2017 at 8:22 AM, Sahina Bose <sabose@redhat.com> wrote:
One of the host fails to install with
Failed to install Host lago-hc-basic-suite-master-host2. Failed to execute stage 'Misc configuration': Failed to start service 'tuned'
The installation on other 2 hosts have been successful - all hosts are created from same template/repo. So this is strange.
https://bugzilla.redhat.com/show_bug.cgi?id=1258868#c5
Please escalate if need, so we'll get it fixed.
Is this a race? The HC OST test ran successfully in the last build.
Y.
On Wed, Jun 28, 2017 at 8:58 AM, <jenkins@jenkins.phx.ovirt.org> wrote:
Project: http://jenkins.ovirt.org/job/ovirt_master_hc-system-tests/ Build: http://jenkins.ovirt.org/job/ovirt_master_hc-system-tests/154/ Build Number: 154 Build Status: Failure Triggered By: Started by timer
------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #154 [Daniel Belenky] Exclude packages from ovirt-master.repo
[Barak Korren] Filter builds sent to change queues by version
[Barak Korren] Make change queue invoke OST basic suit
[Barak Korren] Various UX improvements in change-queue jobs
[Barak Korren] Add a job for direct deployments to 'tested'
[Barak Korren] Use new job to update 'tested' from experimental
[Barak Korren] Use new tested deploy job in timed builders
[Eyal Edri] update build retention policy for big artifacts
----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 002_bootstrap.add_hosts
Error Message: Host lago-hc-basic-suite-master-host2 failed to install -------------------- >> begin captured logging << -------------------- ovirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_up at 0x3f11cf8> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 217, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt- system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 151, in _host_is_up raise RuntimeError('Host %s failed to install' % host.name()) RuntimeError: Host lago-hc-basic-suite-master-host2 failed to install --------------------- >> end captured logging << ---------------------
Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt- system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 164, in add_hosts testlib.assert_true_within(_host_is_up, timeout=15 * 60) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 256, in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 217, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt- system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 151, in _host_is_up raise RuntimeError('Host %s failed to install' % host.name()) Host lago-hc-basic-suite-master-host2 failed to install -------------------- >> begin captured logging << -------------------- ovirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_up at 0x3f11cf8> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 217, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt- system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 151, in _host_is_up raise RuntimeError('Host %s failed to install' % host.name()) RuntimeError: Host lago-hc-basic-suite-master-host2 failed to install --------------------- >> end captured logging << ---------------------
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel

On Thu, Jun 29, 2017 at 9:04 AM, Sahina Bose <sabose@redhat.com> wrote:
On Wed, Jun 28, 2017 at 12:20 PM, Yaniv Kaul <ykaul@redhat.com> wrote:
On Wed, Jun 28, 2017 at 8:22 AM, Sahina Bose <sabose@redhat.com> wrote:
One of the host fails to install with
Failed to install Host lago-hc-basic-suite-master-host2. Failed to execute stage 'Misc configuration': Failed to start service 'tuned'
The installation on other 2 hosts have been successful - all hosts are created from same template/repo. So this is strange.
https://bugzilla.redhat.com/show_bug.cgi?id=1258868#c5
Please escalate if need, so we'll get it fixed.
Is this a race? The HC OST test ran successfully in the last build.
Looks like it - especially as other hosts passed and it's not highly reproducible... Y.
Y.
On Wed, Jun 28, 2017 at 8:58 AM, <jenkins@jenkins.phx.ovirt.org> wrote:
Project: http://jenkins.ovirt.org/job/ovirt_master_hc-system-tests/ Build: http://jenkins.ovirt.org/job/ovirt_master_hc-system-tests/154/ Build Number: 154 Build Status: Failure Triggered By: Started by timer
------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #154 [Daniel Belenky] Exclude packages from ovirt-master.repo
[Barak Korren] Filter builds sent to change queues by version
[Barak Korren] Make change queue invoke OST basic suit
[Barak Korren] Various UX improvements in change-queue jobs
[Barak Korren] Add a job for direct deployments to 'tested'
[Barak Korren] Use new job to update 'tested' from experimental
[Barak Korren] Use new tested deploy job in timed builders
[Eyal Edri] update build retention policy for big artifacts
----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 002_bootstrap.add_hosts
Error Message: Host lago-hc-basic-suite-master-host2 failed to install -------------------- >> begin captured logging << -------------------- ovirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_up at 0x3f11cf8> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 217, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt- system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 151, in _host_is_up raise RuntimeError('Host %s failed to install' % host.name()) RuntimeError: Host lago-hc-basic-suite-master-host2 failed to install --------------------- >> end captured logging << ---------------------
Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt- system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 164, in add_hosts testlib.assert_true_within(_host_is_up, timeout=15 * 60) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 256, in assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 217, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt- system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 151, in _host_is_up raise RuntimeError('Host %s failed to install' % host.name()) Host lago-hc-basic-suite-master-host2 failed to install -------------------- >> begin captured logging << -------------------- ovirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_up at 0x3f11cf8> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 217, in assert_equals_within res = func() File "/home/jenkins/workspace/ovirt_master_hc-system-tests/ovirt- system-tests/hc-basic-suite-master/test-scenarios/002_bootstrap.py", line 151, in _host_is_up raise RuntimeError('Host %s failed to install' % host.name()) RuntimeError: Host lago-hc-basic-suite-master-host2 failed to install --------------------- >> end captured logging << ---------------------
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
participants (2)
-
Sahina Bose
-
Yaniv Kaul