[JIRA] (OVIRT-2538) ovirt-engine master failure on test
008_basic_ui_sanity.initialize_chrome
by Greg Sheremeta (oVirt JIRA)
[ https://ovirt-jira.atlassian.net/browse/OVIRT-2538?page=com.atlassian.jir... ]
Greg Sheremeta commented on OVIRT-2538:
---------------------------------------
It depends on how often it happens.
There are 2 things, generally, that can go wrong in the test. 1 gwt doesn't load at all, and 2 there are timeouts when waiting for gwt to change pages when menu items are clicked. we are pretty resilient when it comes to problem 2, but maybe .05% of time the graceful handling of this could fail. we don't deal with 1 other than silently fail the test at the beginning, and you have to look at the screenshot and take a guess that that's what happened. [I believe that's what happened above.]
So, the test could be more graceful. It can be upgraded to perform detection of this condition (gwt doesn't load) and do further inspection (is the gwt script there? is it valid javascript? can we try again? etc.)
> ovirt-engine master failure on test 008_basic_ui_sanity.initialize_chrome
> ---------------------------------------------------------------------------
>
> Key: OVIRT-2538
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2538
> Project: oVirt - virtualization made easy
> Issue Type: Bug
> Reporter: Dafna Ron
> Assignee: infra
> Priority: High
> Labels: ost_failures, ost_race
>
> I have seen random failures of test 008_basic_ui_sanity.initialize_chrome which are not related to the changes.
> Here is the latest failure which I confirmed with didi is not a real failure and is not related to the change
> https://gerrit.ovirt.org/#/c/89572/
> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10601/
> I cannot see anything in the http logs and in engine logs.
> I can see a warn on ssl and some issues which seems to be resolved later on domain names:
> '[Mon Oct 15 02:56:26.041204 2018] [ssl:warn] [pid 18661] AH01909: RSA certificate configured for 192.168.201.4:443 does NOT include an ID which matches the server name'
> I think the issue is the test since the only errors I can see are in the lago logs.
> I saved logs to allow debugging of the test failure.
> [~gbenhaim(a)redhat.com]
> [~gshereme(a)redhat.com]
> Can you please check this issue?
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100096)
5 years, 11 months
[JIRA] (OVIRT-2632) No space left on device
by Eyal Edri (oVirt JIRA)
[ https://ovirt-jira.atlassian.net/browse/OVIRT-2632?page=com.atlassian.jir... ]
Eyal Edri reassigned OVIRT-2632:
--------------------------------
Assignee: Galit Rosenthal (was: infra)
[~ederevea] do we have monitoring/cleanup on the Jenkins VM slaves that can alert us if space is running our on the filesystem? if not, can we add it? maybe to the global setup.sh? [~bkorren(a)redhat.com]
[~grosenth(a)redhat.com] I see a newer build is failing on timeout downloading from the fedora mirror, few options to check:
1. increase timeout for the CURL command
2. See if we can mirror/proxy these images on our infra
3. Check with [~ederevea] if we have any known network issues between Fedora to PHX DC.
> No space left on device
> -----------------------
>
> Key: OVIRT-2632
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2632
> Project: oVirt - virtualization made easy
> Issue Type: By-EMAIL
> Reporter: sbonazzo
> Assignee: Galit Rosenthal
>
> Hi,
> looks like on some slave there's not enough space for handling the jobs:
> https://jenkins.ovirt.org/job/ovirt-appliance_master_build-artifacts-fc28...
> Can you please check if we need larger disks or if it's a case of cleanup
> failing?
> In both cases, please take action on fixing the issue.
> --
> SANDRO BONAZZOLA
> MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
> Red Hat EMEA <https://www.redhat.com/>
> sbonazzo(a)redhat.com
> <https://red.ht/sig>
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100096)
5 years, 11 months
[JIRA] (OVIRT-2202)
006_migrations.prepare_migration_attachments_ipv4 for
ovirt-engine-dashboard
by Eyal Edri (oVirt JIRA)
[ https://ovirt-jira.atlassian.net/browse/OVIRT-2202?page=com.atlassian.jir... ]
Eyal Edri updated OVIRT-2202:
-----------------------------
Resolution: Fixed
Status: Done (was: To Do)
from the JIRA ticket seems this might need a bz on RHV, closing as we don't see this anymore,
if it will keep failing again, we'll reconsult with network team how to handle it.
> 006_migrations.prepare_migration_attachments_ipv4 for ovirt-engine-dashboard
> ----------------------------------------------------------------------------
>
> Key: OVIRT-2202
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2202
> Project: oVirt - virtualization made easy
> Issue Type: Bug
> Reporter: Dafna Ron
> Assignee: infra
> Labels: ost_failures, ost_race
>
> There was a failed test 006_migrations.prepare_migration_attachments_ipv4
> Error: The response content type 'text/html; charset=iso-8859-1' isn't the expected XML
> Build: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/8208/
> patch reported: https://gerrit.ovirt.org/#/c/92238/1 - add resolutions section to force jquery and bootstrap versions
> The actual error is: Unable to process messages Broken pipe: java.io.IOException: Broken pipe
> this would be the error in vdsm:
> '2018-06-14 04:05:34,698-0400 ERROR (jsonrpc/5) [vds] connectivity check failed (API:1537)
> Traceback (most recent call last):
> File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1534, in setupNetworks
> supervdsm.getProxy().setupNetworks(networks, bondings, options)
> File "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 56, in __call__
> return callMethod()
> File "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 54, in <lambda>
> **kwargs)
> File "<string>", line 2, in setupNetworks
> File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in _callmethod
> raise convert_to_error(kind, result)
> ConfigNetworkError: (10, 'connectivity check failed')
> 2018-06-14 04:05:34,699-0400 INFO (jsonrpc/5) [api.network] FINISH setupNetworks return={'status': {'message': 'connectivity check failed', 'code': 10}} from=::ffff:192.168.201.4,50952 (api:53)
> 2018-06-14 04:05:34,699-0400 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.setupNetworks failed (error 10) in 127.20 seconds (__init__:311)'
> The patch may be related so I will as Greg to take a look
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100096)
5 years, 11 months
[JIRA] (OVIRT-2538) ovirt-engine master failure on test
008_basic_ui_sanity.initialize_chrome
by Eyal Edri (oVirt JIRA)
[ https://ovirt-jira.atlassian.net/browse/OVIRT-2538?page=com.atlassian.jir... ]
Eyal Edri commented on OVIRT-2538:
----------------------------------
[~gshereme(a)redhat.com] [~dron] is this still an issue worth investigating? race maybe?
> ovirt-engine master failure on test 008_basic_ui_sanity.initialize_chrome
> ---------------------------------------------------------------------------
>
> Key: OVIRT-2538
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2538
> Project: oVirt - virtualization made easy
> Issue Type: Bug
> Reporter: Dafna Ron
> Assignee: infra
> Priority: High
> Labels: ost_failures, ost_race
>
> I have seen random failures of test 008_basic_ui_sanity.initialize_chrome which are not related to the changes.
> Here is the latest failure which I confirmed with didi is not a real failure and is not related to the change
> https://gerrit.ovirt.org/#/c/89572/
> https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10601/
> I cannot see anything in the http logs and in engine logs.
> I can see a warn on ssl and some issues which seems to be resolved later on domain names:
> '[Mon Oct 15 02:56:26.041204 2018] [ssl:warn] [pid 18661] AH01909: RSA certificate configured for 192.168.201.4:443 does NOT include an ID which matches the server name'
> I think the issue is the test since the only errors I can see are in the lago logs.
> I saved logs to allow debugging of the test failure.
> [~gbenhaim(a)redhat.com]
> [~gshereme(a)redhat.com]
> Can you please check this issue?
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100096)
5 years, 11 months
Build failed in Jenkins:
system-sync_mirrors-fedora-base-fc29-x86_64 #142
by jenkins@jenkins.phx.ovirt.org
See <http://jenkins.ovirt.org/job/system-sync_mirrors-fedora-base-fc29-x86_64/...>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on mirrors.phx.ovirt.org (mirrors) in workspace <http://jenkins.ovirt.org/job/system-sync_mirrors-fedora-base-fc29-x86_64/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url http://gerrit.ovirt.org/jenkins.git # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
Pruning obsolete local branches
Fetching upstream changes from http://gerrit.ovirt.org/jenkins.git
> git --version # timeout=10
> git fetch --tags --progress http://gerrit.ovirt.org/jenkins.git +refs/heads/*:refs/remotes/origin/* --prune
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision cfe4d8cfee724d0179eb657e00980116f4c34414 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f cfe4d8cfee724d0179eb657e00980116f4c34414
Commit message: "Print out node information"
> git rev-list --no-walk cfe4d8cfee724d0179eb657e00980116f4c34414 # timeout=10
[system-sync_mirrors-fedora-base-fc29-x86_64] $ /bin/bash -xe /tmp/jenkins230601372153969823.sh
+ jenkins/scripts/mirror_mgr.sh resync_yum_mirror fedora-base-fc29 x86_64 jenkins/data/mirrors-reposync.conf
Checking if mirror needs a resync
Traceback (most recent call last):
File "/usr/bin/reposync", line 343, in <module>
main()
File "/usr/bin/reposync", line 175, in main
my.doRepoSetup()
File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 681, in doRepoSetup
return self._getRepos(thisrepo, True)
File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 721, in _getRepos
self._repos.doSetup(thisrepo)
File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup
self.retrieveAllMD()
File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88, in retrieveAllMD
dl = repo._async and repo._commonLoadRepoXML(repo)
File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1465, in _commonLoadRepoXML
local = self.cachedir + '/repomd.xml'
File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 774, in <lambda>
cachedir = property(lambda self: self._dirGetAttr('cachedir'))
File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 757, in _dirGetAttr
self.dirSetup()
File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 735, in dirSetup
self._dirSetupMkdir_p(dir)
File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 712, in _dirSetupMkdir_p
raise Errors.RepoError, msg
yum.Errors.RepoError: Error making cache directory: /home/jenkins/mirrors_cache/centos-kvm-common-el7 error was: [Errno 17] File exists: '/home/jenkins/mirrors_cache/centos-kvm-common-el7'
Build step 'Execute shell' marked build as failure
5 years, 11 months