[ovirt-devel] [VDSM] Test failure test_reject_concurrency(True) (protocoldetectorTests.AcceptorTests)

[Nir Soffer]

test_reject_concurrency(True) (protocoldetectorTests.AcceptorTests)
fails once every few month.

Looks like a real issue in the actual code, doing double close.
This is a known issue in asyncore that we already fixed in few places.

Nir

----

11:26:48 ======================================================================
11:26:48 ERROR: test_reject_concurrency(True)
(protocoldetectorTests.AcceptorTests)
11:26:48 ----------------------------------------------------------------------
11:26:48 Traceback (most recent call last):
11:26:48   File
"/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64/vdsm/tests/protocoldetectorTests.py",
line 113, in tearDown
11:26:48     self.acceptor.stop()
11:26:48   File
"/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64/vdsm/lib/vdsm/protocoldetector.py",
line 211, in stop
11:26:48     self._acceptor.close()
11:26:48   File "/usr/lib64/python2.7/asyncore.py", line 407, in close
11:26:48     self.del_channel()
11:26:48   File
"/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py",
line 137, in del_channel
11:26:48     asyncore.dispatcher.del_channel(self, map)
11:26:48   File "/usr/lib64/python2.7/asyncore.py", line 292, in del_channel
11:26:48     del map[fd]
11:26:48 KeyError: 63
11:26:48 -------------------- >> begin captured logging << --------------------
11:26:48 2016-09-11 11:24:27,399 INFO    [vds.MultiProtocolAcceptor]
(MainThread) Listening at 127.0.0.1:46615
11:26:48 2016-09-11 11:24:27,399 DEBUG   [vds.MultiProtocolAcceptor]
(MainThread) Adding detector <protocoldetectorTests.Echo object at
0x7f03ee0b01d0>
11:26:48 2016-09-11 11:24:27,399 DEBUG   [vds.MultiProtocolAcceptor]
(MainThread) Adding detector <protocoldetectorTests.Uppercase object
at 0x7f03ee0b0c50>
11:26:48 2016-09-11 11:24:27,402 INFO
[ProtocolDetector.AcceptorImpl] (Thread-191) Accepting connection from
127.0.0.1:38136
11:26:48 2016-09-11 11:24:27,407 INFO
[ProtocolDetector.AcceptorImpl] (Thread-191) Accepting connection from
127.0.0.1:38138
11:26:48 2016-09-11 11:24:27,411 INFO
[ProtocolDetector.AcceptorImpl] (Thread-191) Accepting connection from
127.0.0.1:38140
11:26:48 2016-09-11 11:24:27,415 INFO
[ProtocolDetector.AcceptorImpl] (Thread-191) Accepting connection from
127.0.0.1:38142
11:26:48 2016-09-11 11:24:27,416 DEBUG   [ProtocolDetector.Detector]
(Thread-191) Using required_size=9
11:26:48 2016-09-11 11:24:27,416 WARNING [vds.dispatcher] (Thread-191)
unhandled write event
11:26:48 2016-09-11 11:24:27,420 INFO
[ProtocolDetector.AcceptorImpl] (Thread-191) Accepting connection from
127.0.0.1:38144
11:26:48 2016-09-11 11:24:27,421 WARNING [ProtocolDetector.Detector]
(Thread-191) Unrecognized protocol: 'no such p'
11:26:48 2016-09-11 11:24:27,427 DEBUG   [ProtocolDetector.Detector]
(Thread-191) Using required_size=9
11:26:48 2016-09-11 11:24:27,427 WARNING [vds.dispatcher] (Thread-191)
unhandled write event
11:26:48 2016-09-11 11:24:27,428 DEBUG   [ProtocolDetector.Detector]
(Thread-191) Using required_size=9
11:26:48 2016-09-11 11:24:27,428 WARNING [vds.dispatcher] (Thread-191)
unhandled write event
11:26:48 2016-09-11 11:24:27,429 WARNING [ProtocolDetector.Detector]
(Thread-191) Unrecognized protocol: 'no such p'
11:26:48 2016-09-11 11:24:27,430 WARNING [ProtocolDetector.Detector]
(Thread-191) Unrecognized protocol: 'no such p'
11:26:48 2016-09-11 11:24:27,430 DEBUG   [ProtocolDetector.Detector]
(Thread-191) Using required_size=9
11:26:48 2016-09-11 11:24:27,431 WARNING [ProtocolDetector.Detector]
(Thread-191) Unrecognized protocol: 'no such p'
11:26:48 2016-09-11 11:24:27,433 DEBUG   [ProtocolDetector.Detector]
(Thread-191) Using required_size=9
11:26:48 2016-09-11 11:24:27,433 WARNING [ProtocolDetector.Detector]
(Thread-191) Unrecognized protocol: 'no such p'
11:26:48 2016-09-11 11:24:27,434 DEBUG   [vds.MultiProtocolAcceptor]
(MainThread) Stopping Acceptor
11:26:48 --------------------- >> end captured logging << ---------------------