[JIRA] (OVIRT-869) Investigate proxy errors from proxy.phx.ovirt.org
by Gil Shinar (oVirt JIRA)
[ https://ovirt-jira.atlassian.net/browse/OVIRT-869?page=com.atlassian.jira... ]
Gil Shinar commented on OVIRT-869:
----------------------------------
In the log I'm seeing the following error messages:
*ConnectionError: HTTPConnectionPool(host='ftp.linux.ncsu.edu', port=80): Max retries exceeded with url: /pub/fedora/linux/updates/23/x86_64/// (Caused by <class 'socket.error'>: [Errno 110] Connection timed out)
ConnectionError: HTTPConnectionPool(host='mirror.evolvedservers.com', port=80): Max retries exceeded with url: /CentOS/7.1.1503/os/x86_64//repodata/repomd.xml (Caused by <class 'socket.error'>: [Errno 113] No route to host)*
and
*ConnectionError: HTTPConnectionPool(host='mirror.evolvedservers.com', port=80): Max retries exceeded with url: /CentOS/7.1.1503/os/x86_64//repodata/repomd.xml (Caused by <class 'socket.error'>: [Errno 101] Network is unreachable)*
This one as well:
*ConnectionError: HTTPConnectionPool(host='mirror.symnds.com', port=80): Max retries exceeded with url: /CentOS/7.2.1511/os/x86_64//repodata/repomd.xml (Caused by <class 'httplib.BadStatusLine'>: '')*
After restart looks like they have gone but we need to find the core of the problem
> Investigate proxy errors from proxy.phx.ovirt.org
> -------------------------------------------------
>
> Key: OVIRT-869
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-869
> Project: oVirt - virtualization made easy
> Issue Type: Bug
> Reporter: eyal edri [Administrator]
> Assignee: infra
> Priority: Highest
>
> We need to make sure jobs don't fail on proxy like [1].
> Either use original repos if proxy is down or add watchdog / alerts for the proxy service.
> http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/3693/art...
--
This message was sent by Atlassian JIRA
(v1000.571.2#100021)
7 years, 11 months
[JIRA] (OVIRT-872) Re: [ovirt-devel] Gerrit headers are not added to commits in vdsm repo
by Barak Korren (oVirt JIRA)
[ https://ovirt-jira.atlassian.net/browse/OVIRT-872?page=com.atlassian.jira... ]
Barak Korren updated OVIRT-872:
-------------------------------
Resolution: Won't Fix
Status: Done (was: To Do)
This has to do with the maintainers` chosen working mode for the Gerrit repo - as I replied in the mailing thread.
The maintainers need to the decide if they want to change it, and understand the consequences.
> Re: [ovirt-devel] Gerrit headers are not added to commits in vdsm repo
> ----------------------------------------------------------------------
>
> Key: OVIRT-872
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-872
> Project: oVirt - virtualization made easy
> Issue Type: By-EMAIL
> Reporter: eyal edri [Administrator]
> Assignee: infra
>
> I don't see any options to control this from project config, it will
> require more investigating to see if its a config option or only available
> via cherry-pick.
> opening a ticket to track this.
> On Sun, Nov 27, 2016 at 1:38 PM, Dan Kenigsberg <danken(a)redhat.com> wrote:
> > On Sun, Nov 27, 2016 at 12:31:21PM +0200, Eyal Edri wrote:
> > > Not sure I understand what do you mean by Gerrit Headers.
> > > Can you give examples?
> > >
> > > On Fri, Nov 25, 2016 at 4:57 PM, Nir Soffer <nsoffer(a)redhat.com> wrote:
> > >
> > > > On Fri, Nov 25, 2016 at 4:45 PM, Tomáš Golembiovský <
> > tgolembi(a)redhat.com>
> > > > wrote:
> > > > > Hi,
> > > > >
> > > > > I've noticed that in vdsm repo the merged commits do not contain the
> > > > > info headers added by Gerrit any more (Reviewed-by/Reviewed-on/etc.)
> > .
> > > > >
> > > > > Is that intentional? If yes, what was the motivation behind this?
> > > > >
> > > > > The change seem to have happened about 4 days ago. Sometime between
> > the
> > > > > following two commits:
> > > > >
> > > > > * 505f5da API: Introduce getQemuImageInfo API. [Maor Lipchuk]
> > > > > * 1c4a39c protocoldetector: Avoid unneeded getpeername() [Nir
> > Soffer]
> > > >
> > > > We switched vdsm to fast-forward 4 days ago, maybe this was unintended
> > > > side effect of this change?
> > > >
> > > > The gerrit headers are very useful, please add back.
> >
> >
> > https://gerrit.ovirt.org/#/c/66295/ is the last one which had them:
> >
> > Reviewed-on: https://gerrit.ovirt.org/66295
> > Reviewed-by: Nir Soffer <nsoffer(a)redhat.com>
> > Continuous-Integration: Jenkins CI
> >
> > they are added to the commit message during cherry-pick, and I find them
> > very useful.
> >
> --
> Eyal Edri
> Associate Manager
> RHV DevOps
> EMEA ENG Virtualization R&D
> Red Hat Israel
> phone: +972-9-7692018
> irc: eedri (on #tlv #rhev-dev #rhev-integ)
--
This message was sent by Atlassian JIRA
(v1000.571.2#100021)
7 years, 11 months
[JIRA] (OVIRT-872) Re: [ovirt-devel] Gerrit headers are not added to commits in vdsm repo
by eyal edri [Administrator] (oVirt JIRA)
eyal edri [Administrator] created OVIRT-872:
-----------------------------------------------
Summary: Re: [ovirt-devel] Gerrit headers are not added to commits in vdsm repo
Key: OVIRT-872
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-872
Project: oVirt - virtualization made easy
Issue Type: By-EMAIL
Reporter: eyal edri [Administrator]
Assignee: infra
I don't see any options to control this from project config, it will
require more investigating to see if its a config option or only available
via cherry-pick.
opening a ticket to track this.
On Sun, Nov 27, 2016 at 1:38 PM, Dan Kenigsberg <danken(a)redhat.com> wrote:
> On Sun, Nov 27, 2016 at 12:31:21PM +0200, Eyal Edri wrote:
> > Not sure I understand what do you mean by Gerrit Headers.
> > Can you give examples?
> >
> > On Fri, Nov 25, 2016 at 4:57 PM, Nir Soffer <nsoffer(a)redhat.com> wrote:
> >
> > > On Fri, Nov 25, 2016 at 4:45 PM, Tomáš Golembiovský <
> tgolembi(a)redhat.com>
> > > wrote:
> > > > Hi,
> > > >
> > > > I've noticed that in vdsm repo the merged commits do not contain the
> > > > info headers added by Gerrit any more (Reviewed-by/Reviewed-on/etc.)
> .
> > > >
> > > > Is that intentional? If yes, what was the motivation behind this?
> > > >
> > > > The change seem to have happened about 4 days ago. Sometime between
> the
> > > > following two commits:
> > > >
> > > > * 505f5da API: Introduce getQemuImageInfo API. [Maor Lipchuk]
> > > > * 1c4a39c protocoldetector: Avoid unneeded getpeername() [Nir
> Soffer]
> > >
> > > We switched vdsm to fast-forward 4 days ago, maybe this was unintended
> > > side effect of this change?
> > >
> > > The gerrit headers are very useful, please add back.
>
>
> https://gerrit.ovirt.org/#/c/66295/ is the last one which had them:
>
> Reviewed-on: https://gerrit.ovirt.org/66295
> Reviewed-by: Nir Soffer <nsoffer(a)redhat.com>
> Continuous-Integration: Jenkins CI
>
> they are added to the commit message during cherry-pick, and I find them
> very useful.
>
--
Eyal Edri
Associate Manager
RHV DevOps
EMEA ENG Virtualization R&D
Red Hat Israel
phone: +972-9-7692018
irc: eedri (on #tlv #rhev-dev #rhev-integ)
--
This message was sent by Atlassian JIRA
(v1000.571.2#100021)
7 years, 11 months
Build failed in Jenkins: ovirt_3.6_he-system-tests #734
by jenkins＠jenkins.phx.ovirt.org
See <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/734/changes>
Changes:
[Eyal Edri] adding missing python-simplejson to epel repo
------------------------------------------
[...truncated 874 lines...]
## took 1345 seconds
## rc = 1
##########################################################
##! ERROR vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
##! Last 20 log entries: logs/mocker-epel-7-x86_64.el7.he_basic_suite_3.6.sh/he_basic_suite_3.6.sh.log
##!
+ env_cleanup
+ echo '#########################'
#########################
+ local res=0
+ local uuid
+ echo '======== Cleaning up'
======== Cleaning up
+ [[ -e <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-te...> ]]
+ echo '----------- Cleaning with lago'
----------- Cleaning with lago
+ lago --workdir <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-te...> destroy --yes --all-prefixes
+ echo '----------- Cleaning with lago done'
----------- Cleaning with lago done
+ [[ 0 != \0 ]]
+ echo '======== Cleanup done'
======== Cleanup done
+ exit 0
+ exit
Took 1204 seconds
===================================
##!
##! ERROR ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
##!########################################################
##########################################################
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for :.* : True
Logical operation result is TRUE
Running script : #!/bin/bash -xe
echo 'shell_scripts/system_tests.collect_logs.sh'
#
# Required jjb vars:
# version
#
VERSION=3.6
SUITE_TYPE=
WORKSPACE="$PWD"
OVIRT_SUITE="$SUITE_TYPE_suite_$VERSION"
TESTS_LOGS="$WORKSPACE/ovirt-system-tests/exported-artifacts"
rm -rf "$WORKSPACE/exported-artifacts"
mkdir -p "$WORKSPACE/exported-artifacts"
if [[ -d "$TESTS_LOGS" ]]; then
mv "$TESTS_LOGS/"* "$WORKSPACE/exported-artifacts/"
fi
[ovirt_3.6_he-system-tests] $ /bin/bash -xe /tmp/hudson4013391311264500255.sh
+ echo shell_scripts/system_tests.collect_logs.sh
shell_scripts/system_tests.collect_logs.sh
+ VERSION=3.6
+ SUITE_TYPE=
+ WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/>
+ OVIRT_SUITE=3.6
+ TESTS_LOGS=<http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-te...>
+ rm -rf <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/734/artifact/expor...>
+ mkdir -p <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/734/artifact/expor...>
+ [[ -d <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-te...> ]]
+ mv <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-te...> <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/734/artifact/expor...>
POST BUILD TASK : SUCCESS
END OF POST BUILD TASK : 0
Match found for :.* : True
Logical operation result is TRUE
Running script : #!/bin/bash -xe
echo "shell-scripts/mock_cleanup.sh"
shopt -s nullglob
WORKSPACE="$PWD"
# Make clear this is the cleanup, helps reading the jenkins logs
cat <<EOC
_______________________________________________________________________
#######################################################################
# #
# CLEANUP #
# #
#######################################################################
EOC
# Archive the logs, we want them anyway
logs=(
./*log
./*/logs
)
if [[ "$logs" ]]; then
for log in "${logs[@]}"
do
echo "Copying ${log} to exported-artifacts"
mv $log exported-artifacts/
done
fi
# stop any processes running inside the chroot
failed=false
mock_confs=("$WORKSPACE"/*/mocker*)
# Clean current jobs mockroot if any
for mock_conf_file in "${mock_confs[@]}"; do
[[ "$mock_conf_file" ]] || continue
echo "Cleaning up mock $mock_conf"
mock_root="${mock_conf_file##*/}"
mock_root="${mock_root%.*}"
my_mock="/usr/bin/mock"
my_mock+=" --configdir=${mock_conf_file%/*}"
my_mock+=" --root=${mock_root}"
my_mock+=" --resultdir=$WORKSPACE"
#TODO: investigate why mock --clean fails to umount certain dirs sometimes,
#so we can use it instead of manually doing all this.
echo "Killing all mock orphan processes, if any."
$my_mock \
--orphanskill \
|| {
echo "ERROR: Failed to kill orphans on $chroot."
failed=true
}
mock_root="$(\
grep \
-Po "(?<=config_opts\['root'\] = ')[^']*" \
"$mock_conf_file" \
)" || :
[[ "$mock_root" ]] || continue
mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || :
if [[ "$mounts" ]]; then
echo "Found mounted dirs inside the chroot $chroot. Trying to umount."
fi
for mount in "${mounts[@]}"; do
sudo umount --lazy "$mount" \
|| {
echo "ERROR: Failed to umount $mount."
failed=true
}
done
done
# Clean any leftover chroot from other jobs
for mock_root in /var/lib/mock/*; do
this_chroot_failed=false
mounts=($(mount | awk '{print $3}' | grep "$mock_root")) || :
if [[ "$mounts" ]]; then
echo "Found mounted dirs inside the chroot $mock_root." \
"Trying to umount."
fi
for mount in "${mounts[@]}"; do
sudo umount --lazy "$mount" \
|| {
echo "ERROR: Failed to umount $mount."
failed=true
this_chroot_failed=true
}
done
if ! $this_chroot_failed; then
sudo rm -rf "$mock_root"
fi
done
if $failed; then
echo "Aborting."
exit 1
fi
# remove mock system cache, we will setup proxies to do the caching and this
# takes lots of space between runs
shopt -u nullglob
sudo rm -Rf /var/cache/mock/*
# restore the permissions in the working dir, as sometimes it leaves files
# owned by root and then the 'cleanup workspace' from jenkins job fails to
# clean and breaks the jobs
sudo chown -R "$USER" "$WORKSPACE"
# Drop all left over libvirt domains
for UUID in $(virsh list --all --uuid); do
virsh destroy $UUID || :
sleep 2
virsh undefine --remove-all-storage --storage vda --snapshots-metadata $UUID || :
done
[ovirt_3.6_he-system-tests] $ /bin/bash -xe /tmp/hudson6850545221930021592.sh
+ echo shell-scripts/mock_cleanup.sh
shell-scripts/mock_cleanup.sh
+ shopt -s nullglob
+ WORKSPACE=<http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/>
+ cat
_______________________________________________________________________
#######################################################################
# #
# CLEANUP #
# #
#######################################################################
+ logs=(./*log ./*/logs)
+ [[ -n ./ovirt-system-tests/logs ]]
+ for log in '"${logs[@]}"'
+ echo 'Copying ./ovirt-system-tests/logs to exported-artifacts'
Copying ./ovirt-system-tests/logs to exported-artifacts
+ mv ./ovirt-system-tests/logs exported-artifacts/
+ failed=false
+ mock_confs=("$WORKSPACE"/*/mocker*)
+ for mock_conf_file in '"${mock_confs[@]}"'
+ [[ -n <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-te...> ]]
+ echo 'Cleaning up mock '
Cleaning up mock
+ mock_root=mocker-epel-7-x86_64.el7.cfg
+ mock_root=mocker-epel-7-x86_64.el7
+ my_mock=/usr/bin/mock
+ my_mock+=' --configdir=<http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-tests'>
+ my_mock+=' --root=mocker-epel-7-x86_64.el7'
+ my_mock+=' --resultdir=<http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/'>
+ echo 'Killing all mock orphan processes, if any.'
Killing all mock orphan processes, if any.
+ /usr/bin/mock --configdir=<http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-tests> --root=mocker-epel-7-x86_64.el7 --resultdir=<http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/> --orphanskill
WARNING: Could not find required logging config file: <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-te....> Using default...
INFO: mock.py version 1.2.21 starting (python version = 3.4.3)...
Start: init plugins
INFO: selinux enabled
Finish: init plugins
Start: run
Finish: run
++ grep -Po '(?<=config_opts\['\''root'\''\] = '\'')[^'\'']*' <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/ovirt-system-te...>
+ mock_root=epel-7-x86_64-235bec7d0621e95d1cae73d7cf9dc27c
+ [[ -n epel-7-x86_64-235bec7d0621e95d1cae73d7cf9dc27c ]]
+ mounts=($(mount | awk '{print $3}' | grep "$mock_root"))
++ mount
++ awk '{print $3}'
++ grep epel-7-x86_64-235bec7d0621e95d1cae73d7cf9dc27c
+ :
+ [[ -n '' ]]
+ false
+ shopt -u nullglob
+ sudo rm -Rf /var/cache/mock/epel-7-x86_64-235bec7d0621e95d1cae73d7cf9dc27c
+ sudo chown -R jenkins <http://jenkins.ovirt.org/job/ovirt_3.6_he-system-tests/ws/>
++ virsh list --all --uuid
POST BUILD TASK : SUCCESS
END OF POST BUILD TASK : 1
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Archiving artifacts
7 years, 11 months
[JIRA] (OVIRT-871) unrelated CI failure
by danken (oVirt JIRA)
danken created OVIRT-871:
----------------------------
Summary: unrelated CI failure
Key: OVIRT-871
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-871
Project: oVirt - virtualization made easy
Issue Type: By-EMAIL
Reporter: danken
Assignee: infra
Change https://gerrit.ovirt.org/#/c/67101/ just failed on
http://jenkins.ovirt.org/job/ovirt-engine_master_check-patch-fc24-x86_64/...
which I believe is utterly unrelated to the patch. please resolved the underlying problem.
[ERROR] Failed to execute goal on project restapi-definition: Could not resolve dependencies for project org.ovirt.engine.api:restapi-definition:jar:4.1.0-SNAPSHOT: Failed to collect dependencies at org.ovirt.engine.api:metamodel-runtime:jar:1.1.8: Failed to read artifact descriptor for org.ovirt.engine.api:metamodel-runtime:jar:1.1.8: Could not transfer artifact org.ovirt.engine.api:metamodel-runtime:pom:1.1.8 from/to ovirt-maven-repository (http://artifactory.ovirt.org/artifactory/ovirt-mirror): Failed to transfer file: http://artifactory.ovirt.org/artifactory/ovirt-mirror/org/ovirt/engine/ap.... Return code is: 502 , ReasonPhrase:Proxy Error. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionExce...
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :restapi-definition
Makefile:256: recipe for target 'maven' failed
make[1]: *** [maven] Error 1
make[1]: Leaving directory '/home/jenkins/workspace/ovirt-engine_master_check-patch-fc24-x86_64/ovirt-engine/rpmbuild/BUILD/ovirt-engine-4.1.0'
Makefile:263: recipe for target 'tmp.built' failed
make: *** [tmp.built] Error 2
error: Bad exit status from /var/tmp/rpm-tmp.2ykGhi (%build)
--
This message was sent by Atlassian JIRA
(v1000.571.2#100021)
7 years, 11 months