Hi, I need some help/pointers to solve the issues I'm running into while running the gluster HC tests - https://gerrit.ovirt.org/57283 1. Once I run the suite, initialization completes successfully, but further tests fail. Engine is running as per "hosted-engine --vm-status" on the first host (lago_basic_suite_hc_host0) and I can ping the engine from the host running the test File "/usr/lib/python2.7/site-packages/ovirtlago/virt.py", line 189, in get_api return self.get_api_v3() File "/usr/lib/python2.7/site-packages/ovirtlago/virt.py", line 195, in get_api_v3 self._api_v3 = self._get_api(api_ver=3) File "/usr/lib/python2.7/site-packages/ovirtlago/virt.py", line 180, in _get_api raise RuntimeError('Failed to connect to the engine') RuntimeError: Failed to connect to the engine Which logs should I look at to debug this issue? 2. Cannot access the web url of engine: I've setup tunnelling: ssh -L hc-engine.lago.local:8443:hc-engine.lago.local:443 root@rhsdev-grafton1.lab.eng.blr.redhat.com But continue getting the error: The client is not authorized to request an authorization. It's required to access the system using FQDN. 3. Once the vms are left running for a while, connectivity is lost between the vms and cannot connect to some of the VMs via lago shell . [root@lago_basic_suite_hc_host0 ~]# ping lago_basic_suite_hc_host1 PING lago_basic_suite_hc_host1 (192.168.200.2) 56(84) bytes of data.
From lago-basic-suite-hc-host0.lago.local (192.168.200.3) icmp_seq=1 Destination Host Unreachable
Looks like issue with network in VM from console of VM deployment-basic_suite_hc]# virsh console a2051d60-lago_basic_suite_hc_host1 Connected to domain a2051d60-lago_basic_suite_hc_host1 [root@lago_basic_suite_hc_host1 ~]# ip addr show 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 54:52:c0:a8:c8:02 brd ff:ff:ff:ff:ff:ff inet6 fe80::5652:c0ff:fea8:c802/64 scope link valid_lft forever preferred_lft forever Any pointers are much appreciated. thanks sahina
Adding also ovirt devel list On Tue, Nov 8, 2016 at 10:25 AM, Sahina Bose <sabose@redhat.com> wrote:
Hi,
I need some help/pointers to solve the issues I'm running into while running the gluster HC tests - https://gerrit.ovirt.org/57283
1. Once I run the suite, initialization completes successfully, but further tests fail. Engine is running as per "hosted-engine --vm-status" on the first host (lago_basic_suite_hc_host0) and I can ping the engine from the host running the test
File "/usr/lib/python2.7/site-packages/ovirtlago/virt.py", line 189, in get_api return self.get_api_v3() File "/usr/lib/python2.7/site-packages/ovirtlago/virt.py", line 195, in get_api_v3 self._api_v3 = self._get_api(api_ver=3) File "/usr/lib/python2.7/site-packages/ovirtlago/virt.py", line 180, in _get_api raise RuntimeError('Failed to connect to the engine') RuntimeError: Failed to connect to the engine
Which logs should I look at to debug this issue?
2. Cannot access the web url of engine: I've setup tunnelling: ssh -L hc-engine.lago.local:8443:hc-engine.lago.local:443 root@rhsdev-grafton1.lab.eng.blr.redhat.com But continue getting the error: The client is not authorized to request an authorization. It's required to access the system using FQDN.
3. Once the vms are left running for a while, connectivity is lost between the vms and cannot connect to some of the VMs via lago shell .
[root@lago_basic_suite_hc_host0 ~]# ping lago_basic_suite_hc_host1 PING lago_basic_suite_hc_host1 (192.168.200.2) 56(84) bytes of data. From lago-basic-suite-hc-host0.lago.local (192.168.200.3) icmp_seq=1 Destination Host Unreachable
Looks like issue with network in VM from console of VM deployment-basic_suite_hc]# virsh console a2051d60-lago_basic_suite_hc_ host1 Connected to domain a2051d60-lago_basic_suite_hc_host1
[root@lago_basic_suite_hc_host1 ~]# ip addr show 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 54:52:c0:a8:c8:02 brd ff:ff:ff:ff:ff:ff inet6 fe80::5652:c0ff:fea8:c802/64 scope link valid_lft forever preferred_lft forever
Any pointers are much appreciated.
thanks sahina
_______________________________________________ lago-devel mailing list lago-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/lago-devel
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
On Tue, Nov 8, 2016 at 3:01 PM, Nicolas Ecarnot <nicolas@ecarnot.net> wrote:
Le 08/11/2016 à 09:25, Sahina Bose a écrit :
2. Cannot access the web url of engine: I've setup tunnelling: ssh -L hc-engine.lago.local:8443:hc-engine.lago.local:443 root@rhsdev-grafton1.lab.eng.blr.redhat.com But continue getting the error: The client is not authorized to request an authorization. It's required to access the system using FQDN.
Would this be relevant?
http://lists.ovirt.org/pipermail/lago-devel/Week-of- Mon-20160926/000244.html
(Especially the part about /etc/hosts)
I did follow this thread - on my laptop, which is where I'm trying the browser /etc/hosts 127.0.0.1 hc-engine.lago.local on the host running the engine i.e rhsdev-grafton1.lab.eng.blr.redhat.com <root@rhsdev-grafton1.lab.eng.blr.redhat.com> /etc/hosts 192.168.200.99 hc-engine.lago.local hostname on the engine VM = hc-engine.lago.local What am I missing? --
Nicolas ECARNOT
_______________________________________________ lago-devel mailing list lago-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/lago-devel
Hi Sahina, the entries in /etc/hosts need to match the ones given in the answer file[1], maybe you configured something else?, this is from the master suite:
OVESETUP_CONFIG/fqdn=str:engine OVESETUP_ENGINE_CONFIG/fqdn=str:engine
[1] https://gerrit.ovirt.org/gitweb?p=ovirt-system-tests.git;a=blob;f=common/ans... On Tue, Nov 8, 2016 at 11:47 AM, Sahina Bose <sabose@redhat.com> wrote:
On Tue, Nov 8, 2016 at 3:01 PM, Nicolas Ecarnot <nicolas@ecarnot.net> wrote:
Le 08/11/2016 à 09:25, Sahina Bose a écrit :
2. Cannot access the web url of engine: I've setup tunnelling: ssh -L hc-engine.lago.local:8443:hc-engine.lago.local:443 root@rhsdev-grafton1.lab.eng.blr.redhat.com But continue getting the error: The client is not authorized to request an authorization. It's required to access the system using FQDN.
Would this be relevant?
http://lists.ovirt.org/pipermail/lago-devel/Week-of-Mon-20160926/000244.html
(Especially the part about /etc/hosts)
I did follow this thread - on my laptop, which is where I'm trying the browser /etc/hosts
127.0.0.1 hc-engine.lago.local
on the host running the engine i.e rhsdev-grafton1.lab.eng.blr.redhat.com /etc/hosts 192.168.200.99 hc-engine.lago.local
hostname on the engine VM = hc-engine.lago.local
What am I missing?
-- Nicolas ECARNOT
_______________________________________________ lago-devel mailing list lago-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/lago-devel
_______________________________________________ lago-devel mailing list lago-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/lago-devel
On Tue, Nov 8, 2016 at 3:26 PM, Nadav Goldin <ngoldin@redhat.com> wrote:
Hi Sahina, the entries in /etc/hosts need to match the ones given in the answer file[1], maybe you configured something else?, this is from the master suite:
OVESETUP_CONFIG/fqdn=str:engine OVESETUP_ENGINE_CONFIG/fqdn=str:engine
[1] https://gerrit.ovirt.org/gitweb?p=ovirt-system-tests. git;a=blob;f=common/answer-files/el7_master.conf;h= 8b7bd0b5aca905e270df9ce679998f7ae7d111b5;hb=HEAD
[root@rhsdev-grafton1 deployment-basic_suite_hc]# grep ENGINE_FQDN current/test_logs/002_bootstrap.add_dc_quota-20161108124733/lago_basic_suite_hc_engine/_var_log_ovirt-engine/engine.log 2016-11-08 07:12:05,737 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) [] Value of property 'ENGINE_FQDN' is 'hc-engine.lago.local'. And in https://gerrit.ovirt.org/#/c/57283/4/basic_suite_hc/generate-hc-answerfile.s... - 'hc-engine.lago.local' is passed to substitute fqdn value in answer file.
On Tue, Nov 8, 2016 at 11:47 AM, Sahina Bose <sabose@redhat.com> wrote:
On Tue, Nov 8, 2016 at 3:01 PM, Nicolas Ecarnot <nicolas@ecarnot.net>
wrote:
Le 08/11/2016 à 09:25, Sahina Bose a écrit :
2. Cannot access the web url of engine: I've setup tunnelling: ssh -L hc-engine.lago.local:8443:hc-engine.lago.local:443 root@rhsdev-grafton1.lab.eng.blr.redhat.com But continue getting the error: The client is not authorized to request an authorization. It's required
to
access the system using FQDN.
Would this be relevant?
http://lists.ovirt.org/pipermail/lago-devel/Week-of- Mon-20160926/000244.html
(Especially the part about /etc/hosts)
I did follow this thread - on my laptop, which is where I'm trying the browser /etc/hosts
127.0.0.1 hc-engine.lago.local
on the host running the engine i.e rhsdev-grafton1.lab.eng.blr. redhat.com /etc/hosts 192.168.200.99 hc-engine.lago.local
hostname on the engine VM = hc-engine.lago.local
What am I missing?
-- Nicolas ECARNOT
_______________________________________________ lago-devel mailing list lago-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/lago-devel
_______________________________________________ lago-devel mailing list lago-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/lago-devel
Try looking at the httpd logs on the engine VM: /var/lib/httpd/ssl_access_log while attempting to log and check it actually gets the proper fqdn, it should be something like(here with "engine" as the fqdn): 192.168.200.1 - - [08/Nov/2016:10:46:41 -0500] "GET /ovirt-engine/sso/oauth/authorize?client_id=ovirt-engine-core&response_type=code&app_url=https%3A%2F%2Fengine%2Fovirt-engine%2Fwebadmin%2F%3Flocale%3Den_US&engine_url=https%3A%2F%2Fengine%3A443%2Fovirt-engine&redirect_uri=https%3A%2F%2Fengine%3A443%2Fovirt-engine%2Fwebadmin%2Fsso%2Foauth2-callback&scope=ovirt-app-admin+ovirt-app-portal+ovirt-ext%3Dauth%3Asequence-priority%3D%7E HTTP/1.1" 3 and here with a wrong fqdn(127.0.0.1): 192.168.200.1 - - [08/Nov/2016:10:50:56 -0500] "GET /ovirt-engine/sso/oauth/authorize?client_id=ovirt-engine-core&response_type=code&app_url=https%3A%2F%2F127.0.0.1%2Fovirt-engine%2Fwebadmin%2F%3Flocale%3Den_US&engine_url=https%3A%2F%2F127.0.0.1%3A443%2Fovirt-engine&redirect_uri=https%3A%2F%2F127.0.0.1%3A443%2Fovirt-engine%2Fwebadmin%2Fsso%2Foauth2-callback&scope=ovirt-app-admin+ovirt-app-portal+ovirt-ext%3Dauth%3Asequence-priority%3D%7E HTTP/1.1" 302 - On Tue, Nov 8, 2016 at 12:07 PM, Sahina Bose <sabose@redhat.com> wrote:
On Tue, Nov 8, 2016 at 3:26 PM, Nadav Goldin <ngoldin@redhat.com> wrote:
Hi Sahina, the entries in /etc/hosts need to match the ones given in the answer file[1], maybe you configured something else?, this is from the master suite:
OVESETUP_CONFIG/fqdn=str:engine OVESETUP_ENGINE_CONFIG/fqdn=str:engine
[1] https://gerrit.ovirt.org/gitweb?p=ovirt-system-tests.git;a=blob;f=common/ans...
[root@rhsdev-grafton1 deployment-basic_suite_hc]# grep ENGINE_FQDN current/test_logs/002_bootstrap.add_dc_quota-20161108124733/lago_basic_suite_hc_engine/_var_log_ovirt-engine/engine.log 2016-11-08 07:12:05,737 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) [] Value of property 'ENGINE_FQDN' is 'hc-engine.lago.local'.
And in https://gerrit.ovirt.org/#/c/57283/4/basic_suite_hc/generate-hc-answerfile.s... - 'hc-engine.lago.local' is passed to substitute fqdn value in answer file.
On Tue, Nov 8, 2016 at 11:47 AM, Sahina Bose <sabose@redhat.com> wrote:
On Tue, Nov 8, 2016 at 3:01 PM, Nicolas Ecarnot <nicolas@ecarnot.net> wrote:
Le 08/11/2016 à 09:25, Sahina Bose a écrit :
2. Cannot access the web url of engine: I've setup tunnelling: ssh -L hc-engine.lago.local:8443:hc-engine.lago.local:443 root@rhsdev-grafton1.lab.eng.blr.redhat.com But continue getting the error: The client is not authorized to request an authorization. It's required to access the system using FQDN.
Would this be relevant?
http://lists.ovirt.org/pipermail/lago-devel/Week-of-Mon-20160926/000244.html
(Especially the part about /etc/hosts)
I did follow this thread - on my laptop, which is where I'm trying the browser /etc/hosts
127.0.0.1 hc-engine.lago.local
on the host running the engine i.e rhsdev-grafton1.lab.eng.blr.redhat.com /etc/hosts 192.168.200.99 hc-engine.lago.local
hostname on the engine VM = hc-engine.lago.local
What am I missing?
-- Nicolas ECARNOT
_______________________________________________ lago-devel mailing list lago-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/lago-devel
_______________________________________________ lago-devel mailing list lago-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/lago-devel
192.168.200.1 - - [09/Nov/2016:05:45:35 +0000] "GET /ovirt-engine/login?scope=ovirt-ext%3Dauth %3Aidentity HTTP/1.1" 302 - 192.168.200.1 - - [09/Nov/2016:05:45:36 +0000] "GET /ovirt-engine/sso/oauth/authorize?client_id=ovirt-engine-core&response_type=code&engine_url=https%3A%2F%2Fhc-engine.lago.local%3A8443%2Fovirt-engine&redirect_uri=https%3A%2F%2Fhc-engine.lago.local%3A8443%2Fovirt-engine%2Foauth2-callback&scope=ovirt-ext%3Dauth%3Aidentity&locale=en_US HTTP/1.1" 302 - 192.168.200.1 - - [09/Nov/2016:05:45:36 +0000] "GET /ovirt-engine/oauth2-callback?error_code=server_error&error=The+client+is+not+authorized+to+request+an+authorization.+It%27s+required+to+access+the+system+using+FQDN. HTTP/1.1" 302 - 127.0.0.1 - - [09/Nov/2016:05:45:36 +0000] "POST /ovirt-engine/sso/status HTTP/1.1" 200 76 Thanks, Nadav! so the local port 8443 was the issue. I would have thought the engine-url would be https%3A%2F%2Fhc-engine.lago.local%3A443 not https%3A%2F%2Fhc-engine.lago.local%3A8443 On Tue, Nov 8, 2016 at 9:23 PM, Nadav Goldin <ngoldin@redhat.com> wrote:
Try looking at the httpd logs on the engine VM: /var/lib/httpd/ssl_access_log while attempting to log and check it actually gets the proper fqdn, it should be something like(here with "engine" as the fqdn):
192.168.200.1 - - [08/Nov/2016:10:46:41 -0500] "GET /ovirt-engine/sso/oauth/authorize?client_id=ovirt- engine-core&response_type=code&app_url=https%3A%2F% 2Fengine%2Fovirt-engine%2Fwebadmin%2F%3Flocale%3Den_ US&engine_url=https%3A%2F%2Fengine%3A443%2Fovirt-engine& redirect_uri=https%3A%2F%2Fengine%3A443%2Fovirt-engine% 2Fwebadmin%2Fsso%2Foauth2-callback&scope=ovirt-app- admin+ovirt-app-portal+ovirt-ext%3Dauth%3Asequence-priority%3D%7E HTTP/1.1" 3
and here with a wrong fqdn(127.0.0.1): 192.168.200.1 - - [08/Nov/2016:10:50:56 -0500] "GET /ovirt-engine/sso/oauth/authorize?client_id=ovirt- engine-core&response_type=code&app_url=https%3A%2F% 2F127.0.0.1%2Fovirt-engine%2Fwebadmin%2F%3Flocale%3Den_ US&engine_url=https%3A%2F%2F127.0.0.1%3A443%2Fovirt- engine&redirect_uri=https%3A%2F%2F127.0.0.1%3A443%2Fovirt- engine%2Fwebadmin%2Fsso%2Foauth2-callback&scope=ovirt- app-admin+ovirt-app-portal+ovirt-ext%3Dauth%3Asequence-priority%3D%7E HTTP/1.1" 302 -
On Tue, Nov 8, 2016 at 12:07 PM, Sahina Bose <sabose@redhat.com> wrote:
On Tue, Nov 8, 2016 at 3:26 PM, Nadav Goldin <ngoldin@redhat.com> wrote:
Hi Sahina, the entries in /etc/hosts need to match the ones given in the answer file[1], maybe you configured something else?, this is from the master suite:
OVESETUP_CONFIG/fqdn=str:engine OVESETUP_ENGINE_CONFIG/fqdn=str:engine
git;a=blob;f=common/answer-files/el7_master.conf;h= 8b7bd0b5aca905e270df9ce679998f7ae7d111b5;hb=HEAD
[root@rhsdev-grafton1 deployment-basic_suite_hc]# grep ENGINE_FQDN current/test_logs/002_bootstrap.add_dc_quota-20161108124733/lago_basic_
suite_hc_engine/_var_log_ovirt-engine/engine.log
2016-11-08 07:12:05,737 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 45) [] Value of property 'ENGINE_FQDN' is 'hc-engine.lago.local'.
And in https://gerrit.ovirt.org/#/c/57283/4/basic_suite_hc/ generate-hc-answerfile.sh - 'hc-engine.lago.local' is passed to substitute fqdn value in answer file.
On Tue, Nov 8, 2016 at 11:47 AM, Sahina Bose <sabose@redhat.com> wrote:
On Tue, Nov 8, 2016 at 3:01 PM, Nicolas Ecarnot <nicolas@ecarnot.net> wrote:
Le 08/11/2016 à 09:25, Sahina Bose a écrit :
2. Cannot access the web url of engine: I've setup tunnelling: ssh -L hc-engine.lago.local:8443:hc-engine.lago.local:443 root@rhsdev-grafton1.lab.eng.blr.redhat.com But continue getting the error: The client is not authorized to request an authorization. It's
required
to access the system using FQDN.
Would this be relevant?
http://lists.ovirt.org/pipermail/lago-devel/Week-of- Mon-20160926/000244.html
(Especially the part about /etc/hosts)
I did follow this thread - on my laptop, which is where I'm trying the browser /etc/hosts
127.0.0.1 hc-engine.lago.local
on the host running the engine i.e rhsdev-grafton1.lab.eng.blr.redhat.com /etc/hosts 192.168.200.99 hc-engine.lago.local
hostname on the engine VM = hc-engine.lago.local
What am I missing?
-- Nicolas ECARNOT
_______________________________________________ lago-devel mailing list lago-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/lago-devel
_______________________________________________ lago-devel mailing list lago-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/lago-devel
participants (4)
-
Eyal Edri -
Nadav Goldin -
Nicolas Ecarnot -
Sahina Bose