----- Original Message -----
From: "Dan Kenigsberg" <danken@redhat.com> To: "Vered Volansky" <vered@redhat.com>, alitke@redhat.com Cc: "infra" <infra@ovirt.org> Sent: Tuesday, June 17, 2014 3:35:42 PM Subject: Re: u/s CI job failure on infra issue
On Tue, Jun 17, 2014 at 03:02:07AM -0400, Vered Volansky wrote:
Hi,
http://jenkins.ovirt.org/job/vdsm_master_storage_functional_tests_localfs_ge... failed on an infra issue. There are more patches that fail like this. Is there indeed an infra issue? If so, what is it? If not, what should be done in the job configuration as to not inform on an infra issue?
All I can tell is that _attachStorageDomain failed with
AssertionError: error code: 16, message: Unexpected exception
I wish I could get a grip of the vdsm.log during this test. I am not sure that http://jenkins.ovirt.org/job/vdsm_master_storage_functional_tests_localfs_ge... is the whole thing, but it includes very interesting tracebacks that should be better understood.
This is most likely a bug in Vdsm code: Exactly, this is what it looked like to me, but the infra message is misleading and I was wondering how to remove (assuming that there isn't an infra issue on top the vdsm code obvious issue).
Thread-11::ERROR::2014-06-16 15:22:42,157::storageServer::704::Storage.ConnectionMonitor::(_monitorConnections) Monitoring failed Traceback (most recent call last): File "/usr/share/vdsm/storage/storageServer.py", line 696, in _monitorConnections self._checkConnections() File "/usr/share/vdsm/storage/storageServer.py", line 670, in _checkConnections for conId, con in self._conDict.iteritems(): RuntimeError: dictionary changed size during iteration
And this one looks like a mom bug: Thread-113::ERROR::2014-06-16 15:22:34,115::BindingXMLRPC::1126::vds::(wrapper) unexpected error Traceback (most recent call last): File "/usr/share/vdsm/rpc/BindingXMLRPC.py", line 1110, in wrapper res = f(*args, **kwargs) File "/usr/share/vdsm/rpc/BindingXMLRPC.py", line 455, in getStats return api.getStats() File "/usr/share/vdsm/API.py", line 1268, in getStats stats.update(self._cif.mom.getKsmStats()) File "/usr/share/vdsm/momIF.py", line 60, in getKsmStats stats = self._mom.getStatistics()['host'] File "/usr/lib/python2.7/site-packages/mom/MOMFuncs.py", line 74, in getStatistics host_stats = self.threads['host_monitor'].interrogate().statistics[-1] AttributeError: 'NoneType' object has no attribute 'statistics'