test_reject_concurrency(True) (protocoldetectorTests.AcceptorTests) fails once every few month. Looks like a real issue in the actual code, doing double close. This is a known issue in asyncore that we already fixed in few places. Nir ---- 11:26:48 ====================================================================== 11:26:48 ERROR: test_reject_concurrency(True) (protocoldetectorTests.AcceptorTests) 11:26:48 ---------------------------------------------------------------------- 11:26:48 Traceback (most recent call last): 11:26:48 File "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64/vdsm/tests/protocoldetectorTests.py", line 113, in tearDown 11:26:48 self.acceptor.stop() 11:26:48 File "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64/vdsm/lib/vdsm/protocoldetector.py", line 211, in stop 11:26:48 self._acceptor.close() 11:26:48 File "/usr/lib64/python2.7/asyncore.py", line 407, in close 11:26:48 self.del_channel() 11:26:48 File "/home/jenkins/workspace/vdsm_master_check-patch-fc24-x86_64/vdsm/lib/yajsonrpc/betterAsyncore.py", line 137, in del_channel 11:26:48 asyncore.dispatcher.del_channel(self, map) 11:26:48 File "/usr/lib64/python2.7/asyncore.py", line 292, in del_channel 11:26:48 del map[fd] 11:26:48 KeyError: 63 11:26:48 -------------------- >> begin captured logging << -------------------- 11:26:48 2016-09-11 11:24:27,399 INFO [vds.MultiProtocolAcceptor] (MainThread) Listening at 127.0.0.1:46615 11:26:48 2016-09-11 11:24:27,399 DEBUG [vds.MultiProtocolAcceptor] (MainThread) Adding detector <protocoldetectorTests.Echo object at 0x7f03ee0b01d0> 11:26:48 2016-09-11 11:24:27,399 DEBUG [vds.MultiProtocolAcceptor] (MainThread) Adding detector <protocoldetectorTests.Uppercase object at 0x7f03ee0b0c50> 11:26:48 2016-09-11 11:24:27,402 INFO [ProtocolDetector.AcceptorImpl] (Thread-191) Accepting connection from 127.0.0.1:38136 11:26:48 2016-09-11 11:24:27,407 INFO [ProtocolDetector.AcceptorImpl] (Thread-191) Accepting connection from 127.0.0.1:38138 11:26:48 2016-09-11 11:24:27,411 INFO [ProtocolDetector.AcceptorImpl] (Thread-191) Accepting connection from 127.0.0.1:38140 11:26:48 2016-09-11 11:24:27,415 INFO [ProtocolDetector.AcceptorImpl] (Thread-191) Accepting connection from 127.0.0.1:38142 11:26:48 2016-09-11 11:24:27,416 DEBUG [ProtocolDetector.Detector] (Thread-191) Using required_size=9 11:26:48 2016-09-11 11:24:27,416 WARNING [vds.dispatcher] (Thread-191) unhandled write event 11:26:48 2016-09-11 11:24:27,420 INFO [ProtocolDetector.AcceptorImpl] (Thread-191) Accepting connection from 127.0.0.1:38144 11:26:48 2016-09-11 11:24:27,421 WARNING [ProtocolDetector.Detector] (Thread-191) Unrecognized protocol: 'no such p' 11:26:48 2016-09-11 11:24:27,427 DEBUG [ProtocolDetector.Detector] (Thread-191) Using required_size=9 11:26:48 2016-09-11 11:24:27,427 WARNING [vds.dispatcher] (Thread-191) unhandled write event 11:26:48 2016-09-11 11:24:27,428 DEBUG [ProtocolDetector.Detector] (Thread-191) Using required_size=9 11:26:48 2016-09-11 11:24:27,428 WARNING [vds.dispatcher] (Thread-191) unhandled write event 11:26:48 2016-09-11 11:24:27,429 WARNING [ProtocolDetector.Detector] (Thread-191) Unrecognized protocol: 'no such p' 11:26:48 2016-09-11 11:24:27,430 WARNING [ProtocolDetector.Detector] (Thread-191) Unrecognized protocol: 'no such p' 11:26:48 2016-09-11 11:24:27,430 DEBUG [ProtocolDetector.Detector] (Thread-191) Using required_size=9 11:26:48 2016-09-11 11:24:27,431 WARNING [ProtocolDetector.Detector] (Thread-191) Unrecognized protocol: 'no such p' 11:26:48 2016-09-11 11:24:27,433 DEBUG [ProtocolDetector.Detector] (Thread-191) Using required_size=9 11:26:48 2016-09-11 11:24:27,433 WARNING [ProtocolDetector.Detector] (Thread-191) Unrecognized protocol: 'no such p' 11:26:48 2016-09-11 11:24:27,434 DEBUG [vds.MultiProtocolAcceptor] (MainThread) Stopping Acceptor 11:26:48 --------------------- >> end captured logging << ---------------------