5 a.m.
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/
Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_night...
Build Number: 1246
Build Status: Failure
Triggered By: Started by timer
-------------------------------------
Changes Since Last Success:
-------------------------------------
Changes for Build #1246
[Shani Leviim] engine_utils: verify job status by correlation id
-----------------
Failed Tests:
-----------------
1 tests failed.
FAILED:
basic-suite-master.test-scenarios.test_007_sd_reattach.test_deactivate_storage_domain
Error Message:
AssertionError: set() != {<JobStatus.FINISHED: 'finished'>} after 180
seconds
Stack Trace:
engine_api = <ovirtsdk4.Connection object at 0x7f522564b5c0>
@order_by(_TEST_LIST)
def test_deactivate_storage_domain(engine_api):
# TODO: uncomment once VnicSetup checks are fixed.
# TODO: this also seems to leave running tasks behind which break the
deactivation.
# TODO: it should be tested in multiple runs or properly waited for.
# VnicSetup.vnic_setup().init(engine_api.system_service(),
# VM2_NAME, DC_NAME, CLUSTER_NAME)
engine = engine_api.system_service()
dc = test_utils.data_center_service(engine_api.system_service(), DC_NAME)
correlation_id = 'deactivate_storage_domain'
def _deactivate_with_running_ovf_update_task():
try:
test_utils.get_attached_storage_domain(
dc, SD_SECOND_NFS_NAME, service=True).deactivate(
query={'correlation_id': correlation_id})
return True
except ovirtsdk4.Error as err:
# The storage domain's deactivation may fail if it has running tasks.
# In case of updating ovf_store disks task (UploadStream),
# ignore. Otherwise, raise the exception.
if not (('UploadStream' in err.args[0]) or ('OVF' in
err.args[0])):
raise
return False
assertions.assert_true_within_short(
_deactivate_with_running_ovf_update_task)
# Wait for the storage deactivation to be finished.
assertions.assert_equals_within_short(
lambda:
engine_utils.get_jobs_statuses(engine, correlation_id),
{ovirtsdk4.types.JobStatus.FINISHED}
)
basic-suite-master/test-scenarios/test_007_sd_reattach.py:85:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
ost_utils/ost_utils/assertions.py:74: in assert_equals_within_short
error_message=error_message
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
func = <function test_deactivate_storage_domain.<locals>.<lambda> at
0x7f522576e0d0>
value = {<JobStatus.FINISHED: 'finished'>}, timeout = 180
allowed_exceptions = [], initial_wait = 0
error_message = "set() != {<JobStatus.FINISHED: 'finished'>} after 180
seconds"
def assert_equals_within(
func, value, timeout, allowed_exceptions=None, initial_wait=10,
error_message=None
):
allowed_exceptions = allowed_exceptions or []
res = '<no-result-obtained>'
with _EggTimer(timeout) as timer:
while not timer.elapsed():
try:
res = func()
if res == value:
return
except Exception as exc:
if _instance_of_any(exc, allowed_exceptions):
time.sleep(3)
continue
LOGGER.exception("Unhandled exception in %s", func)
raise
if initial_wait == 0:
time.sleep(3)
else:
time.sleep(initial_wait)
initial_wait = 0
try:
if error_message is None:
error_message = '%s != %s after %s seconds' % (res, value,
timeout)
raise AssertionError(error_message)
E
AssertionError: set() != {<JobStatus.FINISHED: 'finished'>} after 180
seconds
ost_utils/ost_utils/assertions.py:59: AssertionError
8:37 a.m.
New subject: [oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly -
Build # 1247 - Still Failing!
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/
Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_night...
Build Number: 1247
Build Status: Still Failing
Triggered By: Started by timer
-------------------------------------
Changes Since Last Success:
-------------------------------------
Changes for Build #1246
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1247
[Shani Leviim] engine_utils: verify job status by correlation id
-----------------
Failed Tests:
-----------------
1 tests failed.
FAILED: basic-suite-master.test-scenarios.test_004_basic_sanity.test_hotplug_memory
Error Message:
AssertionError: False != True after 600 seconds
Stack Trace:
engine = <ovirtsdk4.services.SystemService object at 0x7f6e602b17b8>
event_id = [2039], timeout = 600
@contextlib.contextmanager
def wait_for_event(engine, event_id, timeout=assertions.LONG_TIMEOUT):
'''
event_id could either be an int - a single
event ID or a list - multiple event IDs
that all will be checked
'''
events = engine.events_service()
last_event = int(events.list(max=2)[0].id)
try:
yield
ost_utils/ost_utils/engine_utils.py:36:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
assert_vm_is_alive = <function assert_vm_is_alive.<locals>.is_alive at
0x7f6e602ef598>
engine_api = <ovirtsdk4.Connection object at 0x7f6e602b1550>
get_vm_libvirt_memory_amount = <function
get_vm_libvirt_memory_amount.<locals>.mem_amount at 0x7f6e602ef1e0>
hotplug_mem_amount = 268435456
@order_by(_TEST_LIST)
def test_hotplug_memory(assert_vm_is_alive, engine_api,
get_vm_libvirt_memory_amount, hotplug_mem_amount):
engine = engine_api.system_service()
vm_service = test_utils.get_vm_service(engine, VM0_NAME)
new_memory = vm_service.get().memory + hotplug_mem_amount
with engine_utils.wait_for_event(engine, 2039): # HOT_SET_MEMORY(2,039)
vm_service.update(
vm=types.Vm(
memory=new_memory,
# Need to avoid OOM scenario where ballooning would immediately try to
claim some memory.
# CirrOS is lacking memory onlining rules so the guest memory
doesn't really increase and
# balloon inflation just crashes the guest instead. Balloon gets
inflated because MOM
# does not know that guest size didn't increase and just assumes
it did, and the host
# OST VM is likely under memory pressure, there's not much free
RAM in OST environment.
# Setting minimum guaranteed to new memory size keeps MOM from
inflating balloon.
memory_policy=types.MemoryPolicy(
guaranteed=new_memory,
)
)
)
assert vm_service.get().memory == new_memory
E
AssertionError: assert 100663296 == 369098752
E +100663296
E -369098752
basic-suite-master/test-scenarios/test_004_basic_sanity.py:1423: AssertionError
During handling of the above exception, another exception occurred:
assert_vm_is_alive = <function assert_vm_is_alive.<locals>.is_alive at
0x7f6e602ef598>
engine_api = <ovirtsdk4.Connection object at 0x7f6e602b1550>
get_vm_libvirt_memory_amount = <function
get_vm_libvirt_memory_amount.<locals>.mem_amount at 0x7f6e602ef1e0>
hotplug_mem_amount = 268435456
@order_by(_TEST_LIST)
def test_hotplug_memory(assert_vm_is_alive, engine_api,
get_vm_libvirt_memory_amount, hotplug_mem_amount):
engine = engine_api.system_service()
vm_service = test_utils.get_vm_service(engine, VM0_NAME)
new_memory = vm_service.get().memory + hotplug_mem_amount
with engine_utils.wait_for_event(engine, 2039): # HOT_SET_MEMORY(2,039)
vm_service.update(
vm=types.Vm(
memory=new_memory,
# Need to avoid OOM scenario where ballooning would immediately try to
claim some memory.
# CirrOS is lacking memory onlining rules so the guest memory
doesn't really increase and
# balloon inflation just crashes the guest instead. Balloon gets
inflated because MOM
# does not know that guest size didn't increase and just assumes
it did, and the host
# OST VM is likely under memory pressure, there's not much free
RAM in OST environment.
# Setting minimum guaranteed to new memory size keeps MOM from
inflating balloon.
memory_policy=types.MemoryPolicy(
guaranteed=new_memory,
)
)
)
assert vm_service.get().memory == new_memory
basic-suite-master/test-scenarios/test_004_basic_sanity.py:1423:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.6/contextlib.py:99: in __exit__
self.gen.throw(type, value, traceback)
ost_utils/ost_utils/engine_utils.py:44: in wait_for_event
timeout
ost_utils/ost_utils/assertions.py:86: in assert_true_within
assert_equals_within(func, True, timeout, allowed_exceptions)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
func = <function wait_for_event.<locals>.<lambda> at 0x7f6e6e02c0d0>
value = True, timeout = 600, allowed_exceptions = [], initial_wait = 0
error_message = 'False != True after 600 seconds'
def assert_equals_within(
func, value, timeout, allowed_exceptions=None, initial_wait=10,
error_message=None
):
allowed_exceptions = allowed_exceptions or []
res = '<no-result-obtained>'
with _EggTimer(timeout) as timer:
while not timer.elapsed():
try:
res = func()
if res == value:
return
except Exception as exc:
if _instance_of_any(exc, allowed_exceptions):
time.sleep(3)
continue
LOGGER.exception("Unhandled exception in %s", func)
raise
if initial_wait == 0:
time.sleep(3)
else:
time.sleep(initial_wait)
initial_wait = 0
try:
if error_message is None:
error_message = '%s != %s after %s seconds' % (res, value,
timeout)
raise AssertionError(error_message)
E
AssertionError: False != True after 600 seconds
ost_utils/ost_utils/assertions.py:59: AssertionError
9:21 a.m.
New subject: [oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly -
Build # 1248 - Still Failing!
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/
Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_night...
Build Number: 1248
Build Status: Still Failing
Triggered By: Started by timer
-------------------------------------
Changes Since Last Success:
-------------------------------------
Changes for Build #1246
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1247
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1248
[Shani Leviim] engine_utils: verify job status by correlation id
-----------------
Failed Tests:
-----------------
1 tests failed.
FAILED: basic-suite-master.test-scenarios.test_004_basic_sanity.test_hotplug_memory
Error Message:
AssertionError: False != True after 600 seconds
Stack Trace:
engine = <ovirtsdk4.services.SystemService object at 0x7f126dccd518>
event_id = [2039], timeout = 600
@contextlib.contextmanager
def wait_for_event(engine, event_id, timeout=assertions.LONG_TIMEOUT):
'''
event_id could either be an int - a single
event ID or a list - multiple event IDs
that all will be checked
'''
events = engine.events_service()
last_event = int(events.list(max=2)[0].id)
try:
yield
ost_utils/ost_utils/engine_utils.py:36:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
assert_vm_is_alive = <function assert_vm_is_alive.<locals>.is_alive at
0x7f126dd4f598>
engine_api = <ovirtsdk4.Connection object at 0x7f126dccd898>
get_vm_libvirt_memory_amount = <function
get_vm_libvirt_memory_amount.<locals>.mem_amount at 0x7f126dbc56a8>
hotplug_mem_amount = 268435456
@order_by(_TEST_LIST)
def test_hotplug_memory(assert_vm_is_alive, engine_api,
get_vm_libvirt_memory_amount, hotplug_mem_amount):
engine = engine_api.system_service()
vm_service = test_utils.get_vm_service(engine, VM0_NAME)
new_memory = vm_service.get().memory + hotplug_mem_amount
with engine_utils.wait_for_event(engine, 2039): # HOT_SET_MEMORY(2,039)
vm_service.update(
vm=types.Vm(
memory=new_memory,
# Need to avoid OOM scenario where ballooning would immediately try to
claim some memory.
# CirrOS is lacking memory onlining rules so the guest memory
doesn't really increase and
# balloon inflation just crashes the guest instead. Balloon gets
inflated because MOM
# does not know that guest size didn't increase and just assumes
it did, and the host
# OST VM is likely under memory pressure, there's not much free
RAM in OST environment.
# Setting minimum guaranteed to new memory size keeps MOM from
inflating balloon.
memory_policy=types.MemoryPolicy(
guaranteed=new_memory,
)
)
)
assert vm_service.get().memory == new_memory
E
AssertionError: assert 100663296 == 369098752
E +100663296
E -369098752
basic-suite-master/test-scenarios/test_004_basic_sanity.py:1423: AssertionError
During handling of the above exception, another exception occurred:
assert_vm_is_alive = <function assert_vm_is_alive.<locals>.is_alive at
0x7f126dd4f598>
engine_api = <ovirtsdk4.Connection object at 0x7f126dccd898>
get_vm_libvirt_memory_amount = <function
get_vm_libvirt_memory_amount.<locals>.mem_amount at 0x7f126dbc56a8>
hotplug_mem_amount = 268435456
@order_by(_TEST_LIST)
def test_hotplug_memory(assert_vm_is_alive, engine_api,
get_vm_libvirt_memory_amount, hotplug_mem_amount):
engine = engine_api.system_service()
vm_service = test_utils.get_vm_service(engine, VM0_NAME)
new_memory = vm_service.get().memory + hotplug_mem_amount
with engine_utils.wait_for_event(engine, 2039): # HOT_SET_MEMORY(2,039)
vm_service.update(
vm=types.Vm(
memory=new_memory,
# Need to avoid OOM scenario where ballooning would immediately try to
claim some memory.
# CirrOS is lacking memory onlining rules so the guest memory
doesn't really increase and
# balloon inflation just crashes the guest instead. Balloon gets
inflated because MOM
# does not know that guest size didn't increase and just assumes
it did, and the host
# OST VM is likely under memory pressure, there's not much free
RAM in OST environment.
# Setting minimum guaranteed to new memory size keeps MOM from
inflating balloon.
memory_policy=types.MemoryPolicy(
guaranteed=new_memory,
)
)
)
assert vm_service.get().memory == new_memory
basic-suite-master/test-scenarios/test_004_basic_sanity.py:1423:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.6/contextlib.py:99: in __exit__
self.gen.throw(type, value, traceback)
ost_utils/ost_utils/engine_utils.py:44: in wait_for_event
timeout
ost_utils/ost_utils/assertions.py:86: in assert_true_within
assert_equals_within(func, True, timeout, allowed_exceptions)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
func = <function wait_for_event.<locals>.<lambda> at 0x7f127b99e0d0>
value = True, timeout = 600, allowed_exceptions = [], initial_wait = 0
error_message = 'False != True after 600 seconds'
def assert_equals_within(
func, value, timeout, allowed_exceptions=None, initial_wait=10,
error_message=None
):
allowed_exceptions = allowed_exceptions or []
res = '<no-result-obtained>'
with _EggTimer(timeout) as timer:
while not timer.elapsed():
try:
res = func()
if res == value:
return
except Exception as exc:
if _instance_of_any(exc, allowed_exceptions):
time.sleep(3)
continue
LOGGER.exception("Unhandled exception in %s", func)
raise
if initial_wait == 0:
time.sleep(3)
else:
time.sleep(initial_wait)
initial_wait = 0
try:
if error_message is None:
error_message = '%s != %s after %s seconds' % (res, value,
timeout)
raise AssertionError(error_message)
E
AssertionError: False != True after 600 seconds
ost_utils/ost_utils/assertions.py:59: AssertionError
4:44 a.m.
New subject: [oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly -
Build # 1249 - Still Failing!
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/
Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_night...
Build Number: 1249
Build Status: Still Failing
Triggered By: Started by timer
-------------------------------------
Changes Since Last Success:
-------------------------------------
Changes for Build #1246
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1247
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1248
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1249
[Shani Leviim] engine_utils: verify job status by correlation id
-----------------
Failed Tests:
-----------------
1 tests failed.
FAILED: basic-suite-master.test-scenarios.test_004_basic_sanity.test_hotplug_memory
Error Message:
AssertionError: False != True after 600 seconds
Stack Trace:
engine = <ovirtsdk4.services.SystemService object at 0x7f8e7f731550>
event_id = [2039], timeout = 600
@contextlib.contextmanager
def wait_for_event(engine, event_id, timeout=assertions.LONG_TIMEOUT):
'''
event_id could either be an int - a single
event ID or a list - multiple event IDs
that all will be checked
'''
events = engine.events_service()
last_event = int(events.list(max=2)[0].id)
try:
yield
ost_utils/ost_utils/engine_utils.py:36:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
assert_vm_is_alive = <function assert_vm_is_alive.<locals>.is_alive at
0x7f8e7f6e1f28>
engine_api = <ovirtsdk4.Connection object at 0x7f8e7f711f98>
get_vm_libvirt_memory_amount = <function
get_vm_libvirt_memory_amount.<locals>.mem_amount at 0x7f8e7f6e1598>
hotplug_mem_amount = 268435456
@order_by(_TEST_LIST)
def test_hotplug_memory(assert_vm_is_alive, engine_api,
get_vm_libvirt_memory_amount, hotplug_mem_amount):
engine = engine_api.system_service()
vm_service = test_utils.get_vm_service(engine, VM0_NAME)
new_memory = vm_service.get().memory + hotplug_mem_amount
with engine_utils.wait_for_event(engine, 2039): # HOT_SET_MEMORY(2,039)
vm_service.update(
vm=types.Vm(
memory=new_memory,
# Need to avoid OOM scenario where ballooning would immediately try to
claim some memory.
# CirrOS is lacking memory onlining rules so the guest memory
doesn't really increase and
# balloon inflation just crashes the guest instead. Balloon gets
inflated because MOM
# does not know that guest size didn't increase and just assumes
it did, and the host
# OST VM is likely under memory pressure, there's not much free
RAM in OST environment.
# Setting minimum guaranteed to new memory size keeps MOM from
inflating balloon.
memory_policy=types.MemoryPolicy(
guaranteed=new_memory,
)
)
)
assert vm_service.get().memory == new_memory
E
AssertionError: assert 100663296 == 369098752
E +100663296
E -369098752
basic-suite-master/test-scenarios/test_004_basic_sanity.py:1423: AssertionError
During handling of the above exception, another exception occurred:
assert_vm_is_alive = <function assert_vm_is_alive.<locals>.is_alive at
0x7f8e7f6e1f28>
engine_api = <ovirtsdk4.Connection object at 0x7f8e7f711f98>
get_vm_libvirt_memory_amount = <function
get_vm_libvirt_memory_amount.<locals>.mem_amount at 0x7f8e7f6e1598>
hotplug_mem_amount = 268435456
@order_by(_TEST_LIST)
def test_hotplug_memory(assert_vm_is_alive, engine_api,
get_vm_libvirt_memory_amount, hotplug_mem_amount):
engine = engine_api.system_service()
vm_service = test_utils.get_vm_service(engine, VM0_NAME)
new_memory = vm_service.get().memory + hotplug_mem_amount
with engine_utils.wait_for_event(engine, 2039): # HOT_SET_MEMORY(2,039)
vm_service.update(
vm=types.Vm(
memory=new_memory,
# Need to avoid OOM scenario where ballooning would immediately try to
claim some memory.
# CirrOS is lacking memory onlining rules so the guest memory
doesn't really increase and
# balloon inflation just crashes the guest instead. Balloon gets
inflated because MOM
# does not know that guest size didn't increase and just assumes
it did, and the host
# OST VM is likely under memory pressure, there's not much free
RAM in OST environment.
# Setting minimum guaranteed to new memory size keeps MOM from
inflating balloon.
memory_policy=types.MemoryPolicy(
guaranteed=new_memory,
)
)
)
assert vm_service.get().memory == new_memory
basic-suite-master/test-scenarios/test_004_basic_sanity.py:1423:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.6/contextlib.py:99: in __exit__
self.gen.throw(type, value, traceback)
ost_utils/ost_utils/engine_utils.py:44: in wait_for_event
timeout
ost_utils/ost_utils/assertions.py:86: in assert_true_within
assert_equals_within(func, True, timeout, allowed_exceptions)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
func = <function wait_for_event.<locals>.<lambda> at 0x7f8e914900d0>
value = True, timeout = 600, allowed_exceptions = [], initial_wait = 0
error_message = 'False != True after 600 seconds'
def assert_equals_within(
func, value, timeout, allowed_exceptions=None, initial_wait=10,
error_message=None
):
allowed_exceptions = allowed_exceptions or []
res = '<no-result-obtained>'
with _EggTimer(timeout) as timer:
while not timer.elapsed():
try:
res = func()
if res == value:
return
except Exception as exc:
if _instance_of_any(exc, allowed_exceptions):
time.sleep(3)
continue
LOGGER.exception("Unhandled exception in %s", func)
raise
if initial_wait == 0:
time.sleep(3)
else:
time.sleep(initial_wait)
initial_wait = 0
try:
if error_message is None:
error_message = '%s != %s after %s seconds' % (res, value,
timeout)
raise AssertionError(error_message)
E
AssertionError: False != True after 600 seconds
ost_utils/ost_utils/assertions.py:59: AssertionError
8:07 a.m.
New subject: [oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly -
Build # 1250 - Still Failing!
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/
Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_night...
Build Number: 1250
Build Status: Still Failing
Triggered By: Started by timer
-------------------------------------
Changes Since Last Success:
-------------------------------------
Changes for Build #1246
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1247
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1248
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1249
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1250
[Shani Leviim] engine_utils: verify job status by correlation id
-----------------
Failed Tests:
-----------------
1 tests failed.
FAILED: basic-suite-master.test-scenarios.test_004_basic_sanity.test_hotplug_memory
Error Message:
AssertionError: False != True after 600 seconds
Stack Trace:
engine = <ovirtsdk4.services.SystemService object at 0x7fc331fa27b8>
event_id = [2039], timeout = 600
@contextlib.contextmanager
def wait_for_event(engine, event_id, timeout=assertions.LONG_TIMEOUT):
'''
event_id could either be an int - a single
event ID or a list - multiple event IDs
that all will be checked
'''
events = engine.events_service()
last_event = int(events.list(max=2)[0].id)
try:
yield
ost_utils/ost_utils/engine_utils.py:36:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
assert_vm_is_alive = <function assert_vm_is_alive.<locals>.is_alive at
0x7fc331f8b598>
engine_api = <ovirtsdk4.Connection object at 0x7fc331fa2550>
get_vm_libvirt_memory_amount = <function
get_vm_libvirt_memory_amount.<locals>.mem_amount at 0x7fc331f8b0d0>
hotplug_mem_amount = 268435456
@order_by(_TEST_LIST)
def test_hotplug_memory(assert_vm_is_alive, engine_api,
get_vm_libvirt_memory_amount, hotplug_mem_amount):
engine = engine_api.system_service()
vm_service = test_utils.get_vm_service(engine, VM0_NAME)
new_memory = vm_service.get().memory + hotplug_mem_amount
with engine_utils.wait_for_event(engine, 2039): # HOT_SET_MEMORY(2,039)
vm_service.update(
vm=types.Vm(
memory=new_memory,
# Need to avoid OOM scenario where ballooning would immediately try to
claim some memory.
# CirrOS is lacking memory onlining rules so the guest memory
doesn't really increase and
# balloon inflation just crashes the guest instead. Balloon gets
inflated because MOM
# does not know that guest size didn't increase and just assumes
it did, and the host
# OST VM is likely under memory pressure, there's not much free
RAM in OST environment.
# Setting minimum guaranteed to new memory size keeps MOM from
inflating balloon.
memory_policy=types.MemoryPolicy(
guaranteed=new_memory,
)
)
)
assert vm_service.get().memory == new_memory
E
AssertionError: assert 100663296 == 369098752
E +100663296
E -369098752
basic-suite-master/test-scenarios/test_004_basic_sanity.py:1423: AssertionError
During handling of the above exception, another exception occurred:
assert_vm_is_alive = <function assert_vm_is_alive.<locals>.is_alive at
0x7fc331f8b598>
engine_api = <ovirtsdk4.Connection object at 0x7fc331fa2550>
get_vm_libvirt_memory_amount = <function
get_vm_libvirt_memory_amount.<locals>.mem_amount at 0x7fc331f8b0d0>
hotplug_mem_amount = 268435456
@order_by(_TEST_LIST)
def test_hotplug_memory(assert_vm_is_alive, engine_api,
get_vm_libvirt_memory_amount, hotplug_mem_amount):
engine = engine_api.system_service()
vm_service = test_utils.get_vm_service(engine, VM0_NAME)
new_memory = vm_service.get().memory + hotplug_mem_amount
with engine_utils.wait_for_event(engine, 2039): # HOT_SET_MEMORY(2,039)
vm_service.update(
vm=types.Vm(
memory=new_memory,
# Need to avoid OOM scenario where ballooning would immediately try to
claim some memory.
# CirrOS is lacking memory onlining rules so the guest memory
doesn't really increase and
# balloon inflation just crashes the guest instead. Balloon gets
inflated because MOM
# does not know that guest size didn't increase and just assumes
it did, and the host
# OST VM is likely under memory pressure, there's not much free
RAM in OST environment.
# Setting minimum guaranteed to new memory size keeps MOM from
inflating balloon.
memory_policy=types.MemoryPolicy(
guaranteed=new_memory,
)
)
)
assert vm_service.get().memory == new_memory
basic-suite-master/test-scenarios/test_004_basic_sanity.py:1423:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.6/contextlib.py:99: in __exit__
self.gen.throw(type, value, traceback)
ost_utils/ost_utils/engine_utils.py:44: in wait_for_event
timeout
ost_utils/ost_utils/assertions.py:86: in assert_true_within
assert_equals_within(func, True, timeout, allowed_exceptions)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
func = <function wait_for_event.<locals>.<lambda> at 0x7fc33fc5b0d0>
value = True, timeout = 600, allowed_exceptions = [], initial_wait = 0
error_message = 'False != True after 600 seconds'
def assert_equals_within(
func, value, timeout, allowed_exceptions=None, initial_wait=10,
error_message=None
):
allowed_exceptions = allowed_exceptions or []
res = '<no-result-obtained>'
with _EggTimer(timeout) as timer:
while not timer.elapsed():
try:
res = func()
if res == value:
return
except Exception as exc:
if _instance_of_any(exc, allowed_exceptions):
time.sleep(3)
continue
LOGGER.exception("Unhandled exception in %s", func)
raise
if initial_wait == 0:
time.sleep(3)
else:
time.sleep(initial_wait)
initial_wait = 0
try:
if error_message is None:
error_message = '%s != %s after %s seconds' % (res, value,
timeout)
raise AssertionError(error_message)
E
AssertionError: False != True after 600 seconds
ost_utils/ost_utils/assertions.py:59: AssertionError
8:50 a.m.
New subject: [oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly -
Build # 1251 - Still Failing!
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/
Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_night...
Build Number: 1251
Build Status: Still Failing
Triggered By: Started by timer
-------------------------------------
Changes Since Last Success:
-------------------------------------
Changes for Build #1246
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1247
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1248
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1249
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1250
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1251
[Shani Leviim] engine_utils: verify job status by correlation id
-----------------
Failed Tests:
-----------------
1 tests failed.
FAILED: basic-suite-master.test-scenarios.test_004_basic_sanity.test_hotplug_memory
Error Message:
AssertionError: False != True after 600 seconds
Stack Trace:
engine = <ovirtsdk4.services.SystemService object at 0x7f702395c160>
event_id = [2039], timeout = 600
@contextlib.contextmanager
def wait_for_event(engine, event_id, timeout=assertions.LONG_TIMEOUT):
'''
event_id could either be an int - a single
event ID or a list - multiple event IDs
that all will be checked
'''
events = engine.events_service()
last_event = int(events.list(max=2)[0].id)
try:
yield
ost_utils/ost_utils/engine_utils.py:36:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
assert_vm_is_alive = <function assert_vm_is_alive.<locals>.is_alive at
0x7f7023b88598>
engine_api = <ovirtsdk4.Connection object at 0x7f702395cf28>
get_vm_libvirt_memory_amount = <function
get_vm_libvirt_memory_amount.<locals>.mem_amount at 0x7f70238fb6a8>
hotplug_mem_amount = 268435456
@order_by(_TEST_LIST)
def test_hotplug_memory(assert_vm_is_alive, engine_api,
get_vm_libvirt_memory_amount, hotplug_mem_amount):
engine = engine_api.system_service()
vm_service = test_utils.get_vm_service(engine, VM0_NAME)
new_memory = vm_service.get().memory + hotplug_mem_amount
with engine_utils.wait_for_event(engine, 2039): # HOT_SET_MEMORY(2,039)
vm_service.update(
vm=types.Vm(
memory=new_memory,
# Need to avoid OOM scenario where ballooning would immediately try to
claim some memory.
# CirrOS is lacking memory onlining rules so the guest memory
doesn't really increase and
# balloon inflation just crashes the guest instead. Balloon gets
inflated because MOM
# does not know that guest size didn't increase and just assumes
it did, and the host
# OST VM is likely under memory pressure, there's not much free
RAM in OST environment.
# Setting minimum guaranteed to new memory size keeps MOM from
inflating balloon.
memory_policy=types.MemoryPolicy(
guaranteed=new_memory,
)
)
)
assert vm_service.get().memory == new_memory
E
AssertionError: assert 100663296 == 369098752
E +100663296
E -369098752
basic-suite-master/test-scenarios/test_004_basic_sanity.py:1423: AssertionError
During handling of the above exception, another exception occurred:
assert_vm_is_alive = <function assert_vm_is_alive.<locals>.is_alive at
0x7f7023b88598>
engine_api = <ovirtsdk4.Connection object at 0x7f702395cf28>
get_vm_libvirt_memory_amount = <function
get_vm_libvirt_memory_amount.<locals>.mem_amount at 0x7f70238fb6a8>
hotplug_mem_amount = 268435456
@order_by(_TEST_LIST)
def test_hotplug_memory(assert_vm_is_alive, engine_api,
get_vm_libvirt_memory_amount, hotplug_mem_amount):
engine = engine_api.system_service()
vm_service = test_utils.get_vm_service(engine, VM0_NAME)
new_memory = vm_service.get().memory + hotplug_mem_amount
with engine_utils.wait_for_event(engine, 2039): # HOT_SET_MEMORY(2,039)
vm_service.update(
vm=types.Vm(
memory=new_memory,
# Need to avoid OOM scenario where ballooning would immediately try to
claim some memory.
# CirrOS is lacking memory onlining rules so the guest memory
doesn't really increase and
# balloon inflation just crashes the guest instead. Balloon gets
inflated because MOM
# does not know that guest size didn't increase and just assumes
it did, and the host
# OST VM is likely under memory pressure, there's not much free
RAM in OST environment.
# Setting minimum guaranteed to new memory size keeps MOM from
inflating balloon.
memory_policy=types.MemoryPolicy(
guaranteed=new_memory,
)
)
)
assert vm_service.get().memory == new_memory
basic-suite-master/test-scenarios/test_004_basic_sanity.py:1423:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.6/contextlib.py:99: in __exit__
self.gen.throw(type, value, traceback)
ost_utils/ost_utils/engine_utils.py:44: in wait_for_event
timeout
ost_utils/ost_utils/assertions.py:86: in assert_true_within
assert_equals_within(func, True, timeout, allowed_exceptions)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
func = <function wait_for_event.<locals>.<lambda> at 0x7f703d70e0d0>
value = True, timeout = 600, allowed_exceptions = [], initial_wait = 0
error_message = 'False != True after 600 seconds'
def assert_equals_within(
func, value, timeout, allowed_exceptions=None, initial_wait=10,
error_message=None
):
allowed_exceptions = allowed_exceptions or []
res = '<no-result-obtained>'
with _EggTimer(timeout) as timer:
while not timer.elapsed():
try:
res = func()
if res == value:
return
except Exception as exc:
if _instance_of_any(exc, allowed_exceptions):
time.sleep(3)
continue
LOGGER.exception("Unhandled exception in %s", func)
raise
if initial_wait == 0:
time.sleep(3)
else:
time.sleep(initial_wait)
initial_wait = 0
try:
if error_message is None:
error_message = '%s != %s after %s seconds' % (res, value,
timeout)
raise AssertionError(error_message)
E
AssertionError: False != True after 600 seconds
ost_utils/ost_utils/assertions.py:59: AssertionError
4:44 a.m.
New subject: [oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly -
Build # 1252 - Still Failing!
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/
Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_night...
Build Number: 1252
Build Status: Still Failing
Triggered By: Started by timer
-------------------------------------
Changes Since Last Success:
-------------------------------------
Changes for Build #1246
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1247
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1248
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1249
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1250
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1251
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1252
[Shani Leviim] engine_utils: verify job status by correlation id
-----------------
Failed Tests:
-----------------
1 tests failed.
FAILED: basic-suite-master.test-scenarios.test_004_basic_sanity.test_hotplug_memory
Error Message:
AssertionError: False != True after 600 seconds
Stack Trace:
engine = <ovirtsdk4.services.SystemService object at 0x7f471b2f97b8>
event_id = [2039], timeout = 600
@contextlib.contextmanager
def wait_for_event(engine, event_id, timeout=assertions.LONG_TIMEOUT):
'''
event_id could either be an int - a single
event ID or a list - multiple event IDs
that all will be checked
'''
events = engine.events_service()
last_event = int(events.list(max=2)[0].id)
try:
yield
ost_utils/ost_utils/engine_utils.py:36:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
assert_vm_is_alive = <function assert_vm_is_alive.<locals>.is_alive at
0x7f471b2b9598>
engine_api = <ovirtsdk4.Connection object at 0x7f471b2f9550>
get_vm_libvirt_memory_amount = <function
get_vm_libvirt_memory_amount.<locals>.mem_amount at 0x7f471b2596a8>
hotplug_mem_amount = 268435456
@order_by(_TEST_LIST)
def test_hotplug_memory(assert_vm_is_alive, engine_api,
get_vm_libvirt_memory_amount, hotplug_mem_amount):
engine = engine_api.system_service()
vm_service = test_utils.get_vm_service(engine, VM0_NAME)
new_memory = vm_service.get().memory + hotplug_mem_amount
with engine_utils.wait_for_event(engine, 2039): # HOT_SET_MEMORY(2,039)
vm_service.update(
vm=types.Vm(
memory=new_memory,
# Need to avoid OOM scenario where ballooning would immediately try to
claim some memory.
# CirrOS is lacking memory onlining rules so the guest memory
doesn't really increase and
# balloon inflation just crashes the guest instead. Balloon gets
inflated because MOM
# does not know that guest size didn't increase and just assumes
it did, and the host
# OST VM is likely under memory pressure, there's not much free
RAM in OST environment.
# Setting minimum guaranteed to new memory size keeps MOM from
inflating balloon.
memory_policy=types.MemoryPolicy(
guaranteed=new_memory,
)
)
)
assert vm_service.get().memory == new_memory
E
AssertionError: assert 100663296 == 369098752
E +100663296
E -369098752
basic-suite-master/test-scenarios/test_004_basic_sanity.py:1423: AssertionError
During handling of the above exception, another exception occurred:
assert_vm_is_alive = <function assert_vm_is_alive.<locals>.is_alive at
0x7f471b2b9598>
engine_api = <ovirtsdk4.Connection object at 0x7f471b2f9550>
get_vm_libvirt_memory_amount = <function
get_vm_libvirt_memory_amount.<locals>.mem_amount at 0x7f471b2596a8>
hotplug_mem_amount = 268435456
@order_by(_TEST_LIST)
def test_hotplug_memory(assert_vm_is_alive, engine_api,
get_vm_libvirt_memory_amount, hotplug_mem_amount):
engine = engine_api.system_service()
vm_service = test_utils.get_vm_service(engine, VM0_NAME)
new_memory = vm_service.get().memory + hotplug_mem_amount
with engine_utils.wait_for_event(engine, 2039): # HOT_SET_MEMORY(2,039)
vm_service.update(
vm=types.Vm(
memory=new_memory,
# Need to avoid OOM scenario where ballooning would immediately try to
claim some memory.
# CirrOS is lacking memory onlining rules so the guest memory
doesn't really increase and
# balloon inflation just crashes the guest instead. Balloon gets
inflated because MOM
# does not know that guest size didn't increase and just assumes
it did, and the host
# OST VM is likely under memory pressure, there's not much free
RAM in OST environment.
# Setting minimum guaranteed to new memory size keeps MOM from
inflating balloon.
memory_policy=types.MemoryPolicy(
guaranteed=new_memory,
)
)
)
assert vm_service.get().memory == new_memory
basic-suite-master/test-scenarios/test_004_basic_sanity.py:1423:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.6/contextlib.py:99: in __exit__
self.gen.throw(type, value, traceback)
ost_utils/ost_utils/engine_utils.py:44: in wait_for_event
timeout
ost_utils/ost_utils/assertions.py:86: in assert_true_within
assert_equals_within(func, True, timeout, allowed_exceptions)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
func = <function wait_for_event.<locals>.<lambda> at 0x7f472d0340d0>
value = True, timeout = 600, allowed_exceptions = [], initial_wait = 0
error_message = 'False != True after 600 seconds'
def assert_equals_within(
func, value, timeout, allowed_exceptions=None, initial_wait=10,
error_message=None
):
allowed_exceptions = allowed_exceptions or []
res = '<no-result-obtained>'
with _EggTimer(timeout) as timer:
while not timer.elapsed():
try:
res = func()
if res == value:
return
except Exception as exc:
if _instance_of_any(exc, allowed_exceptions):
time.sleep(3)
continue
LOGGER.exception("Unhandled exception in %s", func)
raise
if initial_wait == 0:
time.sleep(3)
else:
time.sleep(initial_wait)
initial_wait = 0
try:
if error_message is None:
error_message = '%s != %s after %s seconds' % (res, value,
timeout)
raise AssertionError(error_message)
E
AssertionError: False != True after 600 seconds
ost_utils/ost_utils/assertions.py:59: AssertionError
8:02 a.m.
New subject: [oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly -
Build # 1253 - Still Failing!
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/
Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_night...
Build Number: 1253
Build Status: Still Failing
Triggered By: Started by timer
-------------------------------------
Changes Since Last Success:
-------------------------------------
Changes for Build #1246
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1247
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1248
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1249
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1250
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1251
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1252
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1253
[Shani Leviim] engine_utils: verify job status by correlation id
-----------------
Failed Tests:
-----------------
1 tests failed.
FAILED: basic-suite-master.test-scenarios.test_004_basic_sanity.test_hotplug_memory
Error Message:
AssertionError: False != True after 600 seconds
Stack Trace:
engine = <ovirtsdk4.services.SystemService object at 0x7fc37fd2f2e8>
event_id = [2039], timeout = 600
@contextlib.contextmanager
def wait_for_event(engine, event_id, timeout=assertions.LONG_TIMEOUT):
'''
event_id could either be an int - a single
event ID or a list - multiple event IDs
that all will be checked
'''
events = engine.events_service()
last_event = int(events.list(max=2)[0].id)
try:
yield
ost_utils/ost_utils/engine_utils.py:36:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
assert_vm_is_alive = <function assert_vm_is_alive.<locals>.is_alive at
0x7fc37fd53598>
engine_api = <ovirtsdk4.Connection object at 0x7fc37fd2fc50>
get_vm_libvirt_memory_amount = <function
get_vm_libvirt_memory_amount.<locals>.mem_amount at 0x7fc37fd530d0>
hotplug_mem_amount = 268435456
@order_by(_TEST_LIST)
def test_hotplug_memory(assert_vm_is_alive, engine_api,
get_vm_libvirt_memory_amount, hotplug_mem_amount):
engine = engine_api.system_service()
vm_service = test_utils.get_vm_service(engine, VM0_NAME)
new_memory = vm_service.get().memory + hotplug_mem_amount
with engine_utils.wait_for_event(engine, 2039): # HOT_SET_MEMORY(2,039)
vm_service.update(
vm=types.Vm(
memory=new_memory,
# Need to avoid OOM scenario where ballooning would immediately try to
claim some memory.
# CirrOS is lacking memory onlining rules so the guest memory
doesn't really increase and
# balloon inflation just crashes the guest instead. Balloon gets
inflated because MOM
# does not know that guest size didn't increase and just assumes
it did, and the host
# OST VM is likely under memory pressure, there's not much free
RAM in OST environment.
# Setting minimum guaranteed to new memory size keeps MOM from
inflating balloon.
memory_policy=types.MemoryPolicy(
guaranteed=new_memory,
)
)
)
assert vm_service.get().memory == new_memory
E
AssertionError: assert 100663296 == 369098752
E +100663296
E -369098752
basic-suite-master/test-scenarios/test_004_basic_sanity.py:1423: AssertionError
During handling of the above exception, another exception occurred:
assert_vm_is_alive = <function assert_vm_is_alive.<locals>.is_alive at
0x7fc37fd53598>
engine_api = <ovirtsdk4.Connection object at 0x7fc37fd2fc50>
get_vm_libvirt_memory_amount = <function
get_vm_libvirt_memory_amount.<locals>.mem_amount at 0x7fc37fd530d0>
hotplug_mem_amount = 268435456
@order_by(_TEST_LIST)
def test_hotplug_memory(assert_vm_is_alive, engine_api,
get_vm_libvirt_memory_amount, hotplug_mem_amount):
engine = engine_api.system_service()
vm_service = test_utils.get_vm_service(engine, VM0_NAME)
new_memory = vm_service.get().memory + hotplug_mem_amount
with engine_utils.wait_for_event(engine, 2039): # HOT_SET_MEMORY(2,039)
vm_service.update(
vm=types.Vm(
memory=new_memory,
# Need to avoid OOM scenario where ballooning would immediately try to
claim some memory.
# CirrOS is lacking memory onlining rules so the guest memory
doesn't really increase and
# balloon inflation just crashes the guest instead. Balloon gets
inflated because MOM
# does not know that guest size didn't increase and just assumes
it did, and the host
# OST VM is likely under memory pressure, there's not much free
RAM in OST environment.
# Setting minimum guaranteed to new memory size keeps MOM from
inflating balloon.
memory_policy=types.MemoryPolicy(
guaranteed=new_memory,
)
)
)
assert vm_service.get().memory == new_memory
basic-suite-master/test-scenarios/test_004_basic_sanity.py:1423:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.6/contextlib.py:99: in __exit__
self.gen.throw(type, value, traceback)
ost_utils/ost_utils/engine_utils.py:44: in wait_for_event
timeout
ost_utils/ost_utils/assertions.py:86: in assert_true_within
assert_equals_within(func, True, timeout, allowed_exceptions)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
func = <function wait_for_event.<locals>.<lambda> at 0x7fc399a320d0>
value = True, timeout = 600, allowed_exceptions = [], initial_wait = 0
error_message = 'False != True after 600 seconds'
def assert_equals_within(
func, value, timeout, allowed_exceptions=None, initial_wait=10,
error_message=None
):
allowed_exceptions = allowed_exceptions or []
res = '<no-result-obtained>'
with _EggTimer(timeout) as timer:
while not timer.elapsed():
try:
res = func()
if res == value:
return
except Exception as exc:
if _instance_of_any(exc, allowed_exceptions):
time.sleep(3)
continue
LOGGER.exception("Unhandled exception in %s", func)
raise
if initial_wait == 0:
time.sleep(3)
else:
time.sleep(initial_wait)
initial_wait = 0
try:
if error_message is None:
error_message = '%s != %s after %s seconds' % (res, value,
timeout)
raise AssertionError(error_message)
E
AssertionError: False != True after 600 seconds
ost_utils/ost_utils/assertions.py:59: AssertionError
8:46 a.m.
New subject: [oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly -
Build # 1254 - Still Failing!
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/
Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_night...
Build Number: 1254
Build Status: Still Failing
Triggered By: Started by timer
-------------------------------------
Changes Since Last Success:
-------------------------------------
Changes for Build #1246
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1247
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1248
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1249
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1250
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1251
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1252
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1253
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1254
[Shani Leviim] engine_utils: verify job status by correlation id
-----------------
Failed Tests:
-----------------
1 tests failed.
FAILED: basic-suite-master.test-scenarios.test_004_basic_sanity.test_hotplug_memory
Error Message:
AssertionError: False != True after 600 seconds
Stack Trace:
engine = <ovirtsdk4.services.SystemService object at 0x7fb723c682e8>
event_id = [2039], timeout = 600
@contextlib.contextmanager
def wait_for_event(engine, event_id, timeout=assertions.LONG_TIMEOUT):
'''
event_id could either be an int - a single
event ID or a list - multiple event IDs
that all will be checked
'''
events = engine.events_service()
last_event = int(events.list(max=2)[0].id)
try:
yield
ost_utils/ost_utils/engine_utils.py:36:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
assert_vm_is_alive = <function assert_vm_is_alive.<locals>.is_alive at
0x7fb723cd4598>
engine_api = <ovirtsdk4.Connection object at 0x7fb723c68c50>
get_vm_libvirt_memory_amount = <function
get_vm_libvirt_memory_amount.<locals>.mem_amount at 0x7fb723cd40d0>
hotplug_mem_amount = 268435456
@order_by(_TEST_LIST)
def test_hotplug_memory(assert_vm_is_alive, engine_api,
get_vm_libvirt_memory_amount, hotplug_mem_amount):
engine = engine_api.system_service()
vm_service = test_utils.get_vm_service(engine, VM0_NAME)
new_memory = vm_service.get().memory + hotplug_mem_amount
with engine_utils.wait_for_event(engine, 2039): # HOT_SET_MEMORY(2,039)
vm_service.update(
vm=types.Vm(
memory=new_memory,
# Need to avoid OOM scenario where ballooning would immediately try to
claim some memory.
# CirrOS is lacking memory onlining rules so the guest memory
doesn't really increase and
# balloon inflation just crashes the guest instead. Balloon gets
inflated because MOM
# does not know that guest size didn't increase and just assumes
it did, and the host
# OST VM is likely under memory pressure, there's not much free
RAM in OST environment.
# Setting minimum guaranteed to new memory size keeps MOM from
inflating balloon.
memory_policy=types.MemoryPolicy(
guaranteed=new_memory,
)
)
)
assert vm_service.get().memory == new_memory
E
AssertionError: assert 100663296 == 369098752
E +100663296
E -369098752
basic-suite-master/test-scenarios/test_004_basic_sanity.py:1423: AssertionError
During handling of the above exception, another exception occurred:
assert_vm_is_alive = <function assert_vm_is_alive.<locals>.is_alive at
0x7fb723cd4598>
engine_api = <ovirtsdk4.Connection object at 0x7fb723c68c50>
get_vm_libvirt_memory_amount = <function
get_vm_libvirt_memory_amount.<locals>.mem_amount at 0x7fb723cd40d0>
hotplug_mem_amount = 268435456
@order_by(_TEST_LIST)
def test_hotplug_memory(assert_vm_is_alive, engine_api,
get_vm_libvirt_memory_amount, hotplug_mem_amount):
engine = engine_api.system_service()
vm_service = test_utils.get_vm_service(engine, VM0_NAME)
new_memory = vm_service.get().memory + hotplug_mem_amount
with engine_utils.wait_for_event(engine, 2039): # HOT_SET_MEMORY(2,039)
vm_service.update(
vm=types.Vm(
memory=new_memory,
# Need to avoid OOM scenario where ballooning would immediately try to
claim some memory.
# CirrOS is lacking memory onlining rules so the guest memory
doesn't really increase and
# balloon inflation just crashes the guest instead. Balloon gets
inflated because MOM
# does not know that guest size didn't increase and just assumes
it did, and the host
# OST VM is likely under memory pressure, there's not much free
RAM in OST environment.
# Setting minimum guaranteed to new memory size keeps MOM from
inflating balloon.
memory_policy=types.MemoryPolicy(
guaranteed=new_memory,
)
)
)
assert vm_service.get().memory == new_memory
basic-suite-master/test-scenarios/test_004_basic_sanity.py:1423:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.6/contextlib.py:99: in __exit__
self.gen.throw(type, value, traceback)
ost_utils/ost_utils/engine_utils.py:44: in wait_for_event
timeout
ost_utils/ost_utils/assertions.py:86: in assert_true_within
assert_equals_within(func, True, timeout, allowed_exceptions)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
func = <function wait_for_event.<locals>.<lambda> at 0x7fb73d9a60d0>
value = True, timeout = 600, allowed_exceptions = [], initial_wait = 0
error_message = 'False != True after 600 seconds'
def assert_equals_within(
func, value, timeout, allowed_exceptions=None, initial_wait=10,
error_message=None
):
allowed_exceptions = allowed_exceptions or []
res = '<no-result-obtained>'
with _EggTimer(timeout) as timer:
while not timer.elapsed():
try:
res = func()
if res == value:
return
except Exception as exc:
if _instance_of_any(exc, allowed_exceptions):
time.sleep(3)
continue
LOGGER.exception("Unhandled exception in %s", func)
raise
if initial_wait == 0:
time.sleep(3)
else:
time.sleep(initial_wait)
initial_wait = 0
try:
if error_message is None:
error_message = '%s != %s after %s seconds' % (res, value,
timeout)
raise AssertionError(error_message)
E
AssertionError: False != True after 600 seconds
ost_utils/ost_utils/assertions.py:59: AssertionError
12:46 p.m.
New subject: [oVirt Jenkins] ovirt-system-tests_basic-suite-master_nightly -
Build # 1255 - Fixed!
Project: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_nightly/
Build: https://jenkins.ovirt.org/job/ovirt-system-tests_basic-suite-master_night...
Build Number: 1255
Build Status: Fixed
Triggered By: Started by timer, Started by timer, Started by timer
-------------------------------------
Changes Since Last Success:
-------------------------------------
Changes for Build #1246
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1247
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1248
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1249
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1250
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1251
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1252
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1253
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1254
[Shani Leviim] engine_utils: verify job status by correlation id
Changes for Build #1255
[Shani Leviim] 004_basic_sanity: remove xfail from test_live_storage_migration
-----------------
Failed Tests:
-----------------
All tests passed
1245
days inactive
1248
days old
9 comments
1 participants
participants (1)
-
jenkins＠jenkins.phx.ovirt.org