[ OST Failure Report ] [ oVirt master ] [ 27-04-2017 ] [add_hosts]
Test failed: 002_bootstrap/add_hosts Link to suspected patches: https://gerrit.ovirt.org/76107 - ssl: change default library Link to job: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6491/ VDSM log: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6491/artifa... Error snippet from VDSM log, this repeats on each connection attempt from Engine side: <error> 2017-04-27 06:39:27,768-0400 INFO (Reactor thread) [ProtocolDetector.AcceptorImpl] Accepted connection from ::ffff: 192.168.201.3:49530 (protocoldetector:74) 2017-04-27 06:39:27,898-0400 ERROR (Reactor thread) [vds.dispatcher] uncaptured python exception, closing channel <yajsonrpc.betterAsyncore.Dispatcher connected ('::ffff:192.168.201.3', 49530, 0, 0) at 0x1cc3b00> (<class 'socket.error'>:Address family not supported by protocol [/usr/lib64/python2.7/asyncore.py|readwrite|110] [/usr/lib64/python2.7/asyncore.py|handle_write_event|468] [/usr/lib/python2.7/site-packages/yajsonrpc/betterAsyncore.py|handle_write|70] [/usr/lib/python2.7/site-packages/yajsonrpc/betterAsyncore.py|_delegate_call|149] [/usr/lib/python2.7/site-packages/vdsm/sslutils.py|handle_write|213] [/usr/lib/python2.7/site-packages/vdsm/sslutils.py|_handle_io|223] [/usr/lib/python2.7/site-packages/vdsm/sslutils.py|_verify_host|237] [/usr/lib/python2.7/site-packages/vdsm/sslutils.py|compare_names|249]) (betterAsyncore:160) </error> -- Regards, Evgheni Dereveanchin
On Thu, Apr 27, 2017 at 3:13 PM, Evgheni Dereveanchin <ederevea@redhat.com> wrote:
Test failed: 002_bootstrap/add_hosts
Link to suspected patches: https://gerrit.ovirt.org/76107 - ssl: change default library
Link to job: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6491/
VDSM log: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_ master/6491/artifact/exported-artifacts/basic-suit-master- el7/test_logs/basic-suite-master/post-002_bootstrap.py/ lago-basic-suite-master-host0/_var_log/vdsm/vdsm.log
Error snippet from VDSM log, this repeats on each connection attempt from Engine side:
<error>
2017-04-27 06:39:27,768-0400 INFO (Reactor thread) [ProtocolDetector.AcceptorImpl] Accepted connection from ::ffff:192.168.201.3:49530 (protocoldetector:74) 2017-04-27 06:39:27,898-0400 ERROR (Reactor thread) [vds.dispatcher] uncaptured python exception, closing channel <yajsonrpc.betterAsyncore.Dispatcher connected ('::ffff:192.168.201.3', 49530, 0, 0) at 0x1cc3b00> (<class 'socket.error'>:Address family not supported by protocol [/usr/lib64/python2.7/asyncore.py|readwrite|110] [/usr/lib64/python2.7/ asyncore.py|handle_write_event|468] [/usr/lib/python2.7/site- packages/yajsonrpc/betterAsyncore.py|handle_write|70] [/usr/lib/python2.7/site-packages/yajsonrpc/betterAsyncore.py|_delegate_call|149] [/usr/lib/python2.7/site-packages/vdsm/sslutils.py|handle_write|213] [/usr/lib/python2.7/site-packages/vdsm/sslutils.py|_handle_io|223] [/usr/lib/python2.7/site-packages/vdsm/sslutils.py|_verify_host|237] [/usr/lib/python2.7/site-packages/vdsm/sslutils.py|compare_names|249]) (betterAsyncore:160)
</error>
This means that what we have in the certificate do not match the source address we get. I suspect that we issue the certificate for 192.168.201.3 but when we get ::ffff:192.168.201.3. The change was verified in the env when ipv4 is used. I pushed a revert [1] for now so we can work on fixing the issue. [1] https://gerrit.ovirt.org/#/c/76160 <http://192.168.201.3:49530>
-- Regards, Evgheni Dereveanchin
I started to investigate the issue [1] and it seems like there is an issue in Lago setup we use. During handshake we have a step to verify whether client certificate was issued for a specific host (no such functionality in m2crytpo code base). It works fine when using either ip addresses or fqdns but in this particular setup we use mixed. When added logging I see that in engine certificate we use 'engine' name which is not resolvable on the host side and the check fails. I posted a patch [2] which fixes IPv4 mapped addresses issue but we need to fix the setup issue. Thanks, Piotr [1] http://jenkins.ovirt.org/job/ovirt-system-tests_manual/326/ [2] https://gerrit.ovirt.org/#/c/76197/ On Thu, Apr 27, 2017 at 3:39 PM, Piotr Kliczewski <pkliczew@redhat.com> wrote:
On Thu, Apr 27, 2017 at 3:13 PM, Evgheni Dereveanchin <ederevea@redhat.com
wrote:
Test failed: 002_bootstrap/add_hosts
Link to suspected patches: https://gerrit.ovirt.org/76107 - ssl: change default library
Link to job: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6491/
VDSM log: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_m aster/6491/artifact/exported-artifacts/basic-suit-master-el7 /test_logs/basic-suite-master/post-002_bootstrap.py/lago- basic-suite-master-host0/_var_log/vdsm/vdsm.log
Error snippet from VDSM log, this repeats on each connection attempt from Engine side:
<error>
2017-04-27 06:39:27,768-0400 INFO (Reactor thread) [ProtocolDetector.AcceptorImpl] Accepted connection from ::ffff: 192.168.201.3:49530 (protocoldetector:74) 2017-04-27 06:39:27,898-0400 ERROR (Reactor thread) [vds.dispatcher] uncaptured python exception, closing channel <yajsonrpc.betterAsyncore.Dispatcher connected ('::ffff:192.168.201.3', 49530, 0, 0) at 0x1cc3b00> (<class 'socket.error'>:Address family not supported by protocol [/usr/lib64/python2.7/asyncore.py|readwrite|110] [/usr/lib64/python2.7/asyncore.py|handle_write_event|468] [/usr/lib/python2.7/site-packages/yajsonrpc/betterAsyncore.py|handle_write|70] [/usr/lib/python2.7/site-packages/yajsonrpc/betterAsyncore.py|_delegate_call|149] [/usr/lib/python2.7/site-packages/vdsm/sslutils.py|handle_write|213] [/usr/lib/python2.7/site-packages/vdsm/sslutils.py|_handle_io|223] [/usr/lib/python2.7/site-packages/vdsm/sslutils.py|_verify_host|237] [/usr/lib/python2.7/site-packages/vdsm/sslutils.py|compare_names|249]) (betterAsyncore:160)
</error>
This means that what we have in the certificate do not match the source address we get. I suspect that we issue the certificate for 192.168.201.3 but when we get ::ffff:192.168.201.3. The change was verified in the env when ipv4 is used. I pushed a revert [1] for now so we can work on fixing the issue.
[1] https://gerrit.ovirt.org/#/c/76160 <http://192.168.201.3:49530>
-- Regards, Evgheni Dereveanchin
Hi Piotr, Can you clarify what you noticed is not resolvable - the 'engine' FQDN from host0? Thanks, Nadav. On Fri, Apr 28, 2017 at 4:15 PM, Piotr Kliczewski <pkliczew@redhat.com> wrote:
I started to investigate the issue [1] and it seems like there is an issue in Lago setup we use.
During handshake we have a step to verify whether client certificate was issued for a specific host (no such functionality in m2crytpo code base). It works fine when using either ip addresses or fqdns but in this particular setup we use mixed.
When added logging I see that in engine certificate we use 'engine' name which is not resolvable on the host side and the check fails. I posted a patch [2] which fixes IPv4 mapped addresses issue but we need to fix the setup issue.
Thanks, Piotr
[1] http://jenkins.ovirt.org/job/ovirt-system-tests_manual/326/ [2] https://gerrit.ovirt.org/#/c/76197/
On Thu, Apr 27, 2017 at 3:39 PM, Piotr Kliczewski <pkliczew@redhat.com> wrote:
On Thu, Apr 27, 2017 at 3:13 PM, Evgheni Dereveanchin <ederevea@redhat.com> wrote:
Test failed: 002_bootstrap/add_hosts
Link to suspected patches: https://gerrit.ovirt.org/76107 - ssl: change default library
Link to job: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6491/
VDSM log:
http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6491/artifa...
Error snippet from VDSM log, this repeats on each connection attempt from Engine side:
<error>
2017-04-27 06:39:27,768-0400 INFO (Reactor thread) [ProtocolDetector.AcceptorImpl] Accepted connection from ::ffff:192.168.201.3:49530 (protocoldetector:74) 2017-04-27 06:39:27,898-0400 ERROR (Reactor thread) [vds.dispatcher] uncaptured python exception, closing channel <yajsonrpc.betterAsyncore.Dispatcher connected ('::ffff:192.168.201.3', 49530, 0, 0) at 0x1cc3b00> (<class 'socket.error'>:Address family not supported by protocol [/usr/lib64/python2.7/asyncore.py|readwrite|110] [/usr/lib64/python2.7/asyncore.py|handle_write_event|468] [/usr/lib/python2.7/site-packages/yajsonrpc/betterAsyncore.py|handle_write|70] [/usr/lib/python2.7/site-packages/yajsonrpc/betterAsyncore.py|_delegate_call|149] [/usr/lib/python2.7/site-packages/vdsm/sslutils.py|handle_write|213] [/usr/lib/python2.7/site-packages/vdsm/sslutils.py|_handle_io|223] [/usr/lib/python2.7/site-packages/vdsm/sslutils.py|_verify_host|237] [/usr/lib/python2.7/site-packages/vdsm/sslutils.py|compare_names|249]) (betterAsyncore:160)
</error>
This means that what we have in the certificate do not match the source address we get. I suspect that we issue the certificate for 192.168.201.3 but when we get ::ffff:192.168.201.3. The change was verified in the env when ipv4 is used. I pushed a revert [1] for now so we can work on fixing the issue.
[1] https://gerrit.ovirt.org/#/c/76160
-- Regards, Evgheni Dereveanchin
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
participants (3)
-
Evgheni Dereveanchin -
Nadav Goldin -
Piotr Kliczewski