[ovirt-devel] [ OST Failure Report ] [ oVirt 4.1 && oVirt master ] [ 20-03-2017 ] [ 004_basic_sanity.snapshots_merge ]
Pavel Zhukov
pzhukov at redhat.com
Mon Mar 27 07:32:38 UTC 2017
Hi all,
Happened again
http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6017/console
On Wed, Mar 22, 2017 at 4:48 PM, Gil Shinar <gshinar at redhat.com> wrote:
> Looks like the actuall error is in VDSM log:
>
> 2017-03-22 11:01:22,410-0400 ERROR (jsonrpc/7) [storage.TaskManager.Task] (Task='179424c1-b23d-40aa-a091-3c197971f420') Unexpected error (task:871)
> Traceback (most recent call last):
> File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 878, in _run
> return fn(*args, **kargs)
> File "/usr/lib/python2.7/site-packages/vdsm/logUtils.py", line 52, in wrapper
> res = f(*args, **kwargs)
> File "/usr/share/vdsm/storage/hsm.py", line 3060, in getVolumeInfo
> volUUID=volUUID).getInfo()
> File "/usr/share/vdsm/storage/sd.py", line 748, in produceVolume
> volUUID)
> File "/usr/share/vdsm/storage/fileVolume.py", line 361, in __init__
> manifest = self.manifestClass(repoPath, sdUUID, imgUUID, volUUID)
> File "/usr/share/vdsm/storage/fileVolume.py", line 61, in __init__
> volUUID)
> File "/usr/share/vdsm/storage/volume.py", line 86, in __init__
> self.validate()
> File "/usr/share/vdsm/storage/volume.py", line 109, in validate
> self.validateVolumePath()
> File "/usr/share/vdsm/storage/fileVolume.py", line 120, in validateVolumePath
> raise se.VolumeDoesNotExist(self.volUUID)
> VolumeDoesNotExist: Volume does not exist: (u'5133a146-d4bc-49a1-8358-7e14522ffc4b',)
> 2017-03-22 11:01:22,413-0400 INFO (jsonrpc/7) [storage.TaskManager.Task] (Task='179424c1-b23d-40aa-a091-3c197971f420') aborting: Task is aborted: 'Volume does not exist' - code 201 (task:1176)
> 2017-03-22 11:01:22,414-0400 ERROR (jsonrpc/7) [storage.Dispatcher] {'status': {'message': "Volume does not exist: (u'5133a146-d4bc-49a1-8358-7e14522ffc4b',)", 'code': 201}} (dispatcher:78)
>
>
> On Wed, Mar 22, 2017 at 10:57 AM, Gil Shinar <gshinar at redhat.com> wrote:
>
>> It has happened again:
>> http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_4.
>> 1/1033/artifact/exported-artifacts/basic-suit-4.1-el7/test_
>> logs/basic-suite-4.1/post-004_basic_sanity.py/
>>
>> Any news concerning this issue?
>>
>> Thanks
>> Gil
>>
>> On Mon, Mar 20, 2017 at 5:53 PM, Alexander Wels <awels at redhat.com> wrote:
>>
>>> On Monday, March 20, 2017 11:01:21 AM EDT Yaniv Kaul wrote:
>>> > On Mon, Mar 20, 2017 at 4:18 PM, Shlomo Ben David <sbendavi at redhat.com
>>> >
>>> >
>>> > wrote:
>>> > > Hi,
>>> > >
>>> > >
>>> > > Test failed: [ 004_basic_sanity.snapshots_merge ]
>>> > >
>>> > > Link to suspected patches: N/A
>>> > >
>>> > > Link to Job:
>>> > > 1. http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_4.
>>> 1/1007
>>> > > 2. http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_
>>> > > master/5918
>>> >
>>> > They look similar, but master has an additional failure (unrelated to
>>> the
>>> > snap merge failure).
>>> > Allon/Tal - can you have someone from storage look at the failures?
>>> > Alexander - can you take a look at the below?
>>> >
>>> >
>>> > 2017-03-20 09:30:10,283-04 DEBUG
>>> > [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$
>>> PostgresSimpleJd
>>> > bcCall] (ServerService Thread Pool -- 48) [] SqlCall for procedure
>>> > [GetAllMacPoolRangesByMacPoolId] compiled
>>> > 2017-03-20 09:30:10,283-04 INFO [org.ovirt.engine.core.bll.Tag
>>> sDirector]
>>> > (ServerService Thread Pool -- 41) [] Finished initializing TagsDirector
>>> > 2017-03-20 09:30:10,275-04 ERROR
>>> > [org.ovirt.engine.ui.frontend.server.dashboard.DashboardData
>>> Servlet.CacheUpd
>>> > ate.Utilization] (EE-ManagedThreadFactory-default-Thread-2) [] Could
>>> not
>>> > update the Utilization Cache: Error while running SQL query:
>>> > org.ovirt.engine.ui.frontend.server.dashboard.DashboardDataException:
>>> Error
>>> > while running SQL query
>>> > at
>>> > org.ovirt.engine.ui.frontend.server.dashboard.dao.BaseDao.ru
>>> nQuery(BaseDao.j
>>> > ava:60) [frontend.jar:]
>>> > at
>>> > org.ovirt.engine.ui.frontend.server.dashboard.dao.HostDwhDao
>>> .getTotalCpuMemC
>>> > ount(HostDwhDao.java:78) [frontend.jar:]
>>> > at
>>> > org.ovirt.engine.ui.frontend.server.dashboard.HourlySummaryH
>>> elper.getTotalCp
>>> > uMemCount(HourlySummaryHelper.java:43) [frontend.jar:]
>>> > at
>>> > org.ovirt.engine.ui.frontend.server.dashboard.HourlySummaryH
>>> elper.getCpuMemS
>>> > ummary(HourlySummaryHelper.java:21) [frontend.jar:]
>>> > at
>>> > org.ovirt.engine.ui.frontend.server.dashboard.DashboardDataS
>>> ervlet.lookupGlo
>>> > balUtilization(DashboardDataServlet.java:287) [frontend.jar:]
>>> > at
>>> > org.ovirt.engine.ui.frontend.server.dashboard.DashboardDataS
>>> ervlet.getDashbo
>>> > ard(DashboardDataServlet.java:261) [frontend.jar:]
>>> >
>>>
>>> So it looks like a communication failure with the DWH. But when I look
>>> at the
>>> associated logs, the error is not in there, nor do the time stamps
>>> match. Is
>>> this error from the same run or a previous one? I was trying to get more
>>> information than a partial stack trace.
>>>
>>> > > 1.
>>> > >
>>> > > Link to all logs:
>>> > > 1. http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_
>>> > > 4.1/1007/artifact/exported-artifacts/basic-suit-4.1-el7/
>>> > > test_logs/basic-suite-4.1/post-004_basic_sanity.py/
>>> > > <http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_4
>>> .1/1007/art
>>> > > ifact/exported-artifacts/basic-suit-4.1-el7/test_logs/basic-
>>> suite-4.1/p
>>> > > ost-004_basic_sanity.py/> 2.
>>> > > http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_
>>> > > master/5918/artifact/exported-artifacts/basic-suit-master-
>>> > > el7/test_logs/basic-suite-master/post-004_basic_sanity.py/
>>> > > <http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_m
>>> aster/5918/
>>> > > artifact/exported-artifacts/basic-suit-master-el7/test_logs/
>>> basic-suite
>>> > > -master/post-004_basic_sanity.py/>>
>>> > > Error snippet from the log:
>>> > >
>>> > > <error>
>>> > >
>>> > > ovirtlago.testlib: ERROR: Unhandled exception in <function <lambda>
>>> at
>>> > > 0x446a758>
>>> > >
>>> > > Traceback (most recent call last):
>>> > > File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py",
>>> line 217,
>>> > >
>>> > > in assert_equals_within
>>> > >
>>> > > res = func()
>>> > >
>>> > > File "/home/jenkins/workspace/test-repo_ovirt_experimental_4.1/
>>> > >
>>> > > ovirt-system-tests/basic-suite-4.1/test-scenarios/004_basic_
>>> sanity.py",
>>> > > line 449, in <lambda>
>>> > >
>>> > > api.vms.get(VM0_NAME).disks.get(disk_name).status.state == 'ok'
>>> > >
>>> > > AttributeError: 'NoneType' object has no attribute 'state'
>>> > > lago.utils: ERROR: Error while running thread
>>> > >
>>> > > Traceback (most recent call last):
>>> > > File "/usr/lib/python2.7/site-packages/lago/utils.py", line 57, in
>>> > >
>>> > > _ret_via_queue
>>> > >
>>> > > queue.put({'return': func()})
>>> > >
>>> > > File "/home/jenkins/workspace/test-repo_ovirt_experimental_4.1/
>>> > >
>>> > > ovirt-system-tests/basic-suite-4.1/test-scenarios/004_basic_
>>> sanity.py",
>>> > > line 448, in snapshot_live_merge
>>> > >
>>> > > lambda:
>>> > > File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py",
>>> line 264,
>>> > >
>>> > > in assert_true_within_long
>>> > >
>>> > > assert_equals_within_long(func, True, allowed_exceptions)
>>> > >
>>> > > File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py",
>>> line 251,
>>> > >
>>> > > in assert_equals_within_long
>>> > >
>>> > > func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions
>>> > >
>>> > > File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py",
>>> line 217,
>>> > >
>>> > > in assert_equals_within
>>> > >
>>> > > res = func()
>>> > >
>>> > > File "/home/jenkins/workspace/test-repo_ovirt_experimental_4.1/
>>> > >
>>> > > ovirt-system-tests/basic-suite-4.1/test-scenarios/004_basic_
>>> sanity.py",
>>> > > line 449, in <lambda>
>>> > >
>>> > > api.vms.get(VM0_NAME).disks.get(disk_name).status.state == 'ok'
>>> > >
>>> > > AttributeError: 'NoneType' object has no attribute 'state'
>>> > >
>>> > > </error>
>>> > >
>>> > >
>>> > > Best Regards,
>>> > >
>>> > > Shlomi Ben-David | Software Engineer | Red Hat ISRAEL
>>> > > RHCSA | RHCVA | RHCE
>>> > > IRC: shlomibendavid (on #rhev-integ, #rhev-dev, #rhev-ci)
>>> > >
>>> > > OPEN SOURCE - 1 4 011 && 011 4 1
>>> > >
>>> > >
>>> > > _______________________________________________
>>> > > Devel mailing list
>>> > > Devel at ovirt.org
>>> > > http://lists.ovirt.org/mailman/listinfo/devel
>>>
>>>
>>> _______________________________________________
>>> Devel mailing list
>>> Devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>
>>
>>
>
> _______________________________________________
> Devel mailing list
> Devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>
--
Pavel Zhukov
Software Engineer
RHEV Devops
IRC: landgraf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/devel/attachments/20170327/807b8899/attachment-0001.html>
More information about the Devel
mailing list