On Thu, Mar 12, 2020 at 9:10 PM Nir Soffer <nsoffer(a)redhat.com&gt; wrote: Another random failure today: ________________ JsonRpcServerTests.testMethodCallArgList(True) ________________ self = <jsonrpcserver_test.JsonRpcServerTests testMethod=testMethodCallArgList(True)> use_ssl = True @permutations(USE_SSL) def testMethodCallArgList(self, use_ssl): data = dummyTextGenerator(1024) ssl_ctx = self.ssl_ctx if use_ssl else None bridge = _DummyBridge() with constructClient(self.log, bridge, ssl_ctx) as clientFactory: with self._client(clientFactory) as client: self.log.info("Calling 'echo'") self.assertEqual(self._callTimeout(client, "echo", lib/yajsonrpc/jsonrpcserver_test.py:166: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <jsonrpcserver_test.JsonRpcServerTests testMethod=testMethodCallArgList(True)> client = <yajsonrpc.jsonrpcclient.JsonRpcClient object at 0x7f07715ba6d8> methodName = 'echo' params = ('Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna...lor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut en',) rid = '2c8134fd-7dd4-4cfc-b7f8-6b7549399cb6', timeout = None def _callTimeout(self, client, methodName, params=None, rid=None, timeout=None): responses = client.call(JsonRpcRequest(methodName, params, rid), timeout=CALL_TIMEOUT) if not responses: E yajsonrpc.exception.JsonRpcNoResponseError: No response for JSON-RPC request: {'method': 'echo'} lib/yajsonrpc/jsonrpcserver_test.py:123: JsonRpcNoResponseError ------------------------------ Captured log call ------------------------------- WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event ERROR vds.dispatcher:betterAsyncore.py:179 uncaptured python exception, closing channel <yajsonrpc.betterAsyncore.Dispatcher ('::1', 34046, 0, 0) at 0x7f07715bafd0> (<class 'ValueError'>:'b'ept-version:1.2'' contains illegal character ':' [/usr/lib64/python3.6/asyncore.py|readwrite|108] [/usr/lib64/python3.6/asyncore.py|handle_read_event|423] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py|handle_read|71] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py|_delegate_call|168] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/protocoldetector.py|handle_read|129] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stompserver.py|handle_socket|413] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/rpc/bindingjsonrpc.py|add_socket|54] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stompserver.py|createListener|379] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stompserver.py|StompListener|345] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py|__init__|47] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py|switch_implementation|86] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stompserver.py|init|363] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/rpc/bindingjsonrpc.py|_onAccept|57] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py|set_message_handler|645] [/usr/lib64/python3.6/asyncore.py|handle_read_event|423] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py|handle_read|71] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py|_delegate_call|168] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py|handle_read|421] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py|parse|323] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py|_parse_command|245] [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py|decode_value|167]) WARNING vds.dispatcher:betterAsyncore.py:179 unhandled close event =============================== warnings summary =============================== /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.tox/lib/lib/python3.6/site-packages/_pytest/mark/structures.py:334 /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.tox/lib/lib/python3.6/site-packages/_pytest/mark/structures.py:334: PytestUnknownMarkWarning: Unknown pytest.mark.stress - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html PytestUnknownMarkWarning, /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.tox/lib/lib/python3.6/site-packages/_pytest/mark/structures.py:334 /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.tox/lib/lib/python3.6/site-packages/_pytest/mark/structures.py:334: PytestUnknownMarkWarning: Unknown pytest.mark.slow - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html PytestUnknownMarkWarning, /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/v2v.py:79 /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/v2v.py:79: DeprecationWarning: invalid escape sequence \d _SSH_AUTH_RE = b'(SSH_AUTH_SOCK)=([^;]+).*;\nSSH_AGENT_PID=(\d+)' /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/v2v.py:1421 /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/v2v.py:1421: DeprecationWarning: invalid escape sequence \^ '(?P<m2_base>[0-9]+){sp}\^{sp}(?P<m2_exp>{exp}))'.format( /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.tox/lib/lib/python3.6/site-packages/_pytest/mark/structures.py:334 /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.tox/lib/lib/python3.6/site-packages/_pytest/mark/structures.py:334: PytestUnknownMarkWarning: Unknown pytest.mark.xpass - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html PytestUnknownMarkWarning, tests/lib/osinfo_test.py::test_package_versions /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/osinfo.py:284: UnicodeWarning: decode() called on unicode string, see https://bugzilla.redhat.com/show_bug.cgi?id=1693751 'version': mi['version'].decode('utf-8'), tests/lib/osinfo_test.py::test_package_versions /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/osinfo.py:285: UnicodeWarning: decode() called on unicode string, see https://bugzilla.redhat.com/show_bug.cgi?id=1693751 'release': mi['release'].decode('utf-8'), -- Docs: https://docs.pytest.org/en/latest/warnings.html ----------- coverage: platform linux, python 3.6.8-final-0 ----------- Coverage HTML written to dir htmlcov-lib ========================== slowest 10 test durations =========================== 18.17s call tests/lib/yajsonrpc/jsonrpcserver_test.py::JsonRpcServerTests::testMethodCallArgList(True) 3.00s call tests/lib/protocoldetector_test.py::AcceptorTests::test_reject_very_slow_client(True) 2.80s call tests/lib/protocoldetector_test.py::AcceptorTests::test_reject_very_slow_client_concurrency(True) 2.79s call tests/lib/protocoldetector_test.py::AcceptorTests::test_reject_very_slow_client_concurrency(False) 2.67s call tests/lib/protocoldetector_test.py::AcceptorTests::test_reject_very_slow_client(False) 1.97s call tests/pywatch_test.py::TestPyWatch::test_timeout_output 1.90s call tests/pywatch_test.py::TestPyWatch::test_timeout_backtrace 1.35s call tests/lib/protocoldetector_test.py::AcceptorTests::test_detect_slow_client(True) 1.35s call tests/lib/utils_test.py::TestPickleCopy::test_picklecopy_faster 1.29s call tests/lib/protocoldetector_test.py::AcceptorTests::test_detect_slow_client_concurrency(False) > > > _________________ JsonRpcServerTests.testClientSubscribe(True) > _________________ > > self = <jsonrpcserver_test.JsonRpcServerTests > testMethod=testClientSubscribe(True)> > use_ssl = True > > @permutations(USE_SSL) > def testClientSubscribe(self, use_ssl): > ssl_ctx = self.ssl_ctx if use_ssl else None > bridge = _DummyBridge() > > with constructClient(self.log, bridge, ssl_ctx) as clientFactory: > with self._client(clientFactory) as client: > event_queue = queue.Queue() > sub = client.subscribe(EVENT_TOPIC, event_queue) > > res = self._callTimeout(client, "send_event", [], > > CALL_ID) > > lib/yajsonrpc/jsonrpcserver_test.py:264: > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ _ _ > > self = <jsonrpcserver_test.JsonRpcServerTests > testMethod=testClientSubscribe(True)> > client = <yajsonrpc.jsonrpcclient.JsonRpcClient object at 0x7fc58ed1b410> > methodName = 'send_event', params = [] > rid = '2c8134fd-7dd4-4cfc-b7f8-6b7549399cb6', timeout = None > > def _callTimeout(self, client, methodName, params=None, rid=None, > timeout=None): > responses = client.call(JsonRpcRequest(methodName, params, rid), > timeout=CALL_TIMEOUT) > if not responses: > > E yajsonrpc.exception.JsonRpcNoResponseError: No response > for JSON-RPC request: {'method': 'send_event'} > > lib/yajsonrpc/jsonrpcserver_test.py:123: JsonRpcNoResponseError > ------------------------------ Captured log call > ------------------------------- > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > > We are missing handle_write in some dispatcher. > > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > ERROR vds.dispatcher:betterAsyncore.py:179 uncaptured python > exception, closing channel <yajsonrpc.betterAsyncore.Dispatcher > ('::1', 56210, 0, 0) at 0x7fc58ed1b790> (<class > 'ValueError'>:'b'ept-version:1.2'' contains illegal character ':' > [/usr/lib64/python3.7/asyncore.py|readwrite|108] > [/usr/lib64/python3.7/asyncore.py|handle_read_event|422] > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py|handle_read|71] > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py|_delegate_call|168] > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/protocoldetector.py|handle_read|129] > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stompserver.py|handle_socket|413] > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/rpc/bindingjsonrpc.py|add_socket|54] > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stompserver.py|createListener|379] > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stompserver.py|StompListener|345] > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py|__init__|47] > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py|switch_implementation|86] > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stompserver.py|init|363] > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/rpc/bindingjsonrpc.py|_onAccept|57] > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py|set_message_handler|645] > [/usr/lib64/python3.7/asyncore.py|handle_read_event|422] > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py|handle_read|71] > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py|_delegate_call|168] > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py|handle_read|421] > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py|parse|323] > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py|_parse_command|245] > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py|decode_value|167]) > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled close event > ERROR root:concurrent.py:267 FINISH thread <Thread(JsonRpc > (StompReactor), started daemon 140486081763072)> failed > Traceback (most recent call last): > File > "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/common/concurrent.py", > line 260, in run > ret = func(*args, **kwargs) > File > "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stompserver.py", > line 393, in process_requests > self._reactor.process_requests() > File > "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py", > line 238, in process_requests > timeout=self._get_timeout(self._map), > File > "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py", > line 253, in _get_timeout > interval = disp.next_check_interval() > File > "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py", > line 99, in next_check_interval > return getattr(self.__impl, "next_check_interval", default_func)() > File > "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py", > line 486, in next_check_interval > self.handle_timeout() > File > "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py", > line 451, in handle_timeout > self._frame_handler.handle_timeout(self) > File > "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stompclient.py", > line 130, in handle_timeout > dispatcher._on_timeout) > File > "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py", > line 630, in reconnect > AsyncDispatcher(self, self._async_client, count=count)) > File > "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py", > line 271, in reconnect > dispatcher.create_socket(address, sslctx) > File > "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py", > line 112, in create_socket > sock = sslctx.wrapSocket(sock) > File > "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/sslutils.py", > line 113, in wrapSocket > ca_certs=self.ca_certs) > File "/usr/lib64/python3.7/ssl.py", line 1230, in wrap_socket > context.load_verify_locations(ca_certs) > FileNotFoundError: [Errno 2] No such file or directory > > On Thu, Mar 12, 2020 at 5:34 PM Nir Soffer <nsoffer(a)redhat.com&gt; wrote: > > > > I had 2 of these unrelated failures today. Would be nice to mark this > > test as broken on CI. > > > > > https://jenkins.ovirt.org/job/vdsm_standard-check-patch/19353//artifact/c... > > > https://jenkins.ovirt.org/job/vdsm_standard-check-patch/19354//artifact/c... > > > > =================================== FAILURES > =================================== > > ______________________ VdsmClientTests.test_event_handler > ______________________ > > > > self = <stomprpcclient_test.VdsmClientTests > testMethod=test_event_handler> > > > > def test_event_handler(self): > > with self._create_client() as client: > > event_queue = queue.Queue() > > > > sub_id = client.subscribe(EVENT_TOPIC, event_queue) > > > client.Test.sendEvent() > > > > lib/yajsonrpc/stomprpcclient_test.py:215: > > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ _ _ _ > > > > self = <stomprpcclient_test._MockedClient object at 0x7fe19f6db1d0> > > namespace = 'Test', method_name = 'sendEvent', kwargs = {} > > method = 'Test.sendEvent', timeout = 3 > > req = {'jsonrpc': '2.0', 'method': 'Test.sendEvent', 'params': {}, > > 'id': 'c3985771-6f77-4431-9ed6-15082063a2d5'} > > responses = None > > > > def _call(self, namespace, method_name, **kwargs): > > """ > > Client call method, executes a given command > > > > Args: > > namespace (string): namespace name > > method_name (string): method name > > **kwargs: Arbitrary keyword arguments > > > > Returns: > > method result > > > > Raises: > > ClientError: in case of an error in the protocol. > > TimeoutError: if there is no response after a pre configured > time. > > ServerError: in case of an error while executing the command > > """ > > method = namespace + "." + method_name > > timeout = kwargs.pop("_timeout", self._default_timeout) > > > > req = yajsonrpc.JsonRpcRequest( > > method, kwargs, reqId=str(uuid.uuid4())) > > > > try: > > responses = self._client.call( > > req, timeout=timeout, flow_id=self._flow_id) > > except EnvironmentError as e: > > raise ClientError(method, kwargs, e) > > > > if not responses: > > > raise TimeoutError(method, kwargs, timeout) > > E vdsm.client.TimeoutError: Request Test.sendEvent with args > > {} timed out after 3 seconds > > > > ../lib/vdsm/client.py:294: TimeoutError > > ------------------------------ Captured log call > ------------------------------- > > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled read event > > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled write event > > ERROR vds.dispatcher:betterAsyncore.py:179 uncaptured python > > exception, closing channel <yajsonrpc.betterAsyncore.Dispatcher > > ('::1', 36398, 0, 0) at 0x7fe19f6db208> (<class > > 'ValueError'>:'b'ept-version:1.2'' contains illegal character ':' > > [/usr/lib64/python3.6/asyncore.py|readwrite|108] > > [/usr/lib64/python3.6/asyncore.py|handle_read_event|423] > > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py|handle_read|71] > > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py|_delegate_call|168] > > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/protocoldetector.py|handle_read|129] > > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stompserver.py|handle_socket|413] > > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/rpc/bindingjsonrpc.py|add_socket|54] > > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stompserver.py|createListener|379] > > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stompserver.py|StompListener|345] > > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py|__init__|47] > > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py|switch_implementation|86] > > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stompserver.py|init|363] > > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/rpc/bindingjsonrpc.py|_onAccept|57] > > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py|set_message_handler|645] > > [/usr/lib64/python3.6/asyncore.py|handle_read_event|423] > > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py|handle_read|71] > > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py|_delegate_call|168] > > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py|handle_read|421] > > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py|parse|323] > > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py|_parse_command|245] > > > [/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py|decode_value|167]) > > WARNING vds.dispatcher:betterAsyncore.py:179 unhandled close event > > ERROR root:concurrent.py:267 FINISH thread <Thread(JsonRpc > > (StompReactor), started daemon 140606575712000)> failed > > Traceback (most recent call last): > > File > "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/common/concurrent.py", > > line 260, in run > > ret = func(*args, **kwargs) > > File > "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stompserver.py", > > line 393, in process_requests > > self._reactor.process_requests() > > File > "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py", > > line 238, in process_requests > > timeout=self._get_timeout(self._map), > > File > "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py", > > line 253, in _get_timeout > > interval = disp.next_check_interval() > > File > "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py", > > line 99, in next_check_interval > > return getattr(self.__impl, "next_check_interval", default_func)() > > File > "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py", > > line 486, in next_check_interval > > self.handle_timeout() > > File > "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py", > > line 451, in handle_timeout > > self._frame_handler.handle_timeout(self) > > File > "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stompclient.py", > > line 130, in handle_timeout > > dispatcher._on_timeout) > > File > "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/stomp.py", > > line 630, in reconnect > > AsyncDispatcher(self, self._async_client, count=count)) > > File > "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py", > > line 271, in reconnect > > dispatcher.create_socket(address, sslctx) > > File > "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/yajsonrpc/betterAsyncore.py", > > line 112, in create_socket > > sock = sslctx.wrapSocket(sock) > > File > "/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/sslutils.py", > > line 113, in wrapSocket > > ca_certs=self.ca_certs) > > File "/usr/lib64/python3.6/ssl.py", line 1114, in wrap_socket > > ciphers=ciphers) > > File "/usr/lib64/python3.6/ssl.py", line 704, in __init__ > > self._context.load_verify_locations(ca_certs) > > FileNotFoundError: [Errno 2] No such file or directory > > =============================== warnings summary > =============================== > > > /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.tox/lib/lib/python3.6/site-packages/_pytest/mark/structures.py:334 > > > /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.tox/lib/lib/python3.6/site-packages/_pytest/mark/structures.py:334: > > PytestUnknownMarkWarning: Unknown pytest.mark.stress - is this a typo? > > You can register custom marks to avoid this warning - for details, > > see https://docs.pytest.org/en/latest/mark.html > > PytestUnknownMarkWarning, > > > > > /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.tox/lib/lib/python3.6/site-packages/_pytest/mark/structures.py:334 > > > /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.tox/lib/lib/python3.6/site-packages/_pytest/mark/structures.py:334: > > PytestUnknownMarkWarning: Unknown pytest.mark.slow - is this a typo? > > You can register custom marks to avoid this warning - for details, see > > https://docs.pytest.org/en/latest/mark.html > > PytestUnknownMarkWarning, > > > > /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/v2v.py:79 > > > /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/v2v.py:79: > > DeprecationWarning: invalid escape sequence \d > > _SSH_AUTH_RE = b'(SSH_AUTH_SOCK)=([^;]+).*;\nSSH_AGENT_PID=(\d+)' > > > > > /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/v2v.py:1421 > > > /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/v2v.py:1421: > > DeprecationWarning: invalid escape sequence \^ > > '(?P<m2_base>[0-9]+){sp}\^{sp}(?P<m2_exp>{exp}))'.format( > > > > > /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.tox/lib/lib/python3.6/site-packages/_pytest/mark/structures.py:334 > > > /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.tox/lib/lib/python3.6/site-packages/_pytest/mark/structures.py:334: > > PytestUnknownMarkWarning: Unknown pytest.mark.xpass - is this a typo? > > You can register custom marks to avoid this warning - for details, see > > https://docs.pytest.org/en/latest/mark.html > > PytestUnknownMarkWarning, > > > > tests/lib/osinfo_test.py::test_package_versions > > > /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/osinfo.py:284: > > UnicodeWarning: decode() called on unicode string, see > > https://bugzilla.redhat.com/show_bug.cgi?id=1693751 > > 'version': mi['version'].decode('utf-8'), > > > > tests/lib/osinfo_test.py::test_package_versions > > > /home/jenkins/workspace/vdsm_standard-check-patch/vdsm/lib/vdsm/osinfo.py:285: > > UnicodeWarning: decode() called on unicode string, see > > https://bugzilla.redhat.com/show_bug.cgi?id=1693751 > > 'release': mi['release'].decode('utf-8'), > > > > -- Docs: https://docs.pytest.org/en/latest/warnings.html > > > > ----------- coverage: platform linux, python 3.6.8-final-0 ----------- > > Coverage HTML written to dir htmlcov-lib > > > > ========================== slowest 10 test durations > =========================== > > 18.29s call > > > tests/lib/yajsonrpc/stomprpcclient_test.py::VdsmClientTests::test_event_handler > > 2.96s call > tests/lib/protocoldetector_test.py::AcceptorTests::test_reject_very_slow_client_concurrency(True) > > 2.90s call > tests/lib/protocoldetector_test.py::AcceptorTests::test_reject_very_slow_client_concurrency(False) > > 2.75s call > tests/lib/protocoldetector_test.py::AcceptorTests::test_reject_very_slow_client(False) > > 2.63s call > tests/lib/protocoldetector_test.py::AcceptorTests::test_reject_very_slow_client(True) > > 1.90s call tests/pywatch_test.py::TestPyWatch::test_timeout_backtrace > > 1.44s call tests/pywatch_test.py::TestPyWatch::test_timeout_output > > 1.19s call tests/pywatch_test.py::TestPyWatch::test_kill_grandkids > > 0.93s call > tests/lib/protocoldetector_test.py::AcceptorTests::test_detect_slow_client(True) > > 0.90s call > tests/lib/yajsonrpc/stomp_test.py::StompTests::test_echo(16384, > > False) > > =========================== short test summary info > ============================ >

Hi, sorry for the delay. I've posted [1] to disable all the mentioned tests in CI. Marcin [1] https://gerrit.ovirt.org/#/c/107711/ On 3/17/20 1:15 PM, Nir Soffer wrote:

On 3/23/20 4:32 PM, Nir Soffer wrote: Posted: https://gerrit.ovirt.org/#/c/107866/ That would add quite a lot of boilerplate I guess. Also, given that we don't have gerrit-travis integration, it's jenkins that is the source of truth for us. I would rather leave things as is.

On Tue, Mar 24, 2020 at 12:20 PM Martin Perina <mperina(a)redhat.com&gt; wrote: Not really, some tests cannot run on jenkins or are flaky, but run fine on travis. For storage successful build on travis is required to get your patches merged. Current CI in jenkins does not work for us. Lot of storage tests are broken and skipped on jenkins but works fine on Travis. On Travis we have better environment, running each build in a new VM, and using containers for testing different distributions, so we control our test environment better. Look at typical run on jenkins: https://jenkins.ovirt.org/job/vdsm_standard-check-patch/19605/ Took 24 minutes. Expected failures: $ curl -s https://jenkins.ovirt.org/job/vdsm_standard-check-patch/19605//artifact/c... | grep XFAIL | wc -l 166 Same build on Travis: https://travis-ci.org/github/oVirt/vdsm/builds/666254744 Took 9 min 38 sec Expected failures: $ curl -s https://api.travis-ci.org/v3/job/666254745/log.txt | grep XFAIL | wc -l 22 You can see that travis builds are much faster. Also maintaining our travis build is much easier - basically single file, and some dockerfiles: $ find docker/ | wc -l 7 $ cat .travis.yml docker/* | wc -l 282 Compared with 57 files in automation/ and 1189 lines: $ find automation | wc -l 57 $ cat automation/* | wc -l 1189 It makes sense for the reason above and others. For example qemu is using both Travis and other CIs enviromnets. Sanlock has also unofficial CI on Travis (and no other CI). We need Travis in gerrit, or move projects to github, or maybe gitlab. The recommended workflow for vsm and ovirt-imageio is to run the tests on Travis, and upload a patch to gerrit only when your Travis build is working. This can save a lot of time and avoids unwanted notifications from jenkins to reviewers when your build fails.