Indeed, these were in the build queue when I was restarting Jenkins and the
jobs tried to re-run before it was fully up. I also see the following error
when publishing el7:
Temporary repodata directory
/srv/resources/repos/ovirt/tested/./master/rpm/el7/.repodata/ already
exists! (Another createrepo process is running?)
Looks like it's also caused by me cancelling a job that left behind a
.repodata directory. Will clean it up now to unblock el7 publishing.
On Wed, Jun 5, 2019 at 8:07 AM Emil Natan <ena(a)redhat.com> wrote:
A bunch of deploy to tested jobs failed as result of the maintenance
restarts earlier today. I'm re-triggering those now.
On Wed, Jun 5, 2019 at 1:38 AM <jenkins(a)jenkins.phx.ovirt.org> wrote:
> See <
>
http://jenkins.ovirt.org/job/deploy-to_ovirt-master_tested/7776/display/r...
> >
>
> ------------------------------------------
> Started by upstream project "ovirt-master_change-queue-tester" build
> number 14434
> originally caused by:
> Started by upstream project "ovirt-master_change-queue" build number
> 42136
> originally caused by:
> Started by upstream project "vdsm_standard-on-merge" build number 795
> originally caused by:
> Triggered by Gerrit:
https://gerrit.ovirt.org/100463
> [EnvInject] - Loading node environment variables.
> Building remotely on
vm0010.workers-phx.ovirt.org (el7 phx nested) in
> workspace <
http://jenkins.ovirt.org/job/deploy-to_ovirt-master_tested/ws/
> >
> [ssh-agent] Looking for ssh-agent implementation...
> [ssh-agent] Exec ssh-agent (binary ssh-agent on a remote machine)
> $ ssh-agent
> SSH_AUTH_SOCK=/tmp/ssh-epHRQo8YDSQE/agent.19927
> SSH_AGENT_PID=19929
> [ssh-agent] Started.
> Running ssh-add (command line suppressed)
> Identity added: <
>
http://jenkins.ovirt.org/job/deploy-to_ovirt-master_tested/ws/@tmp/privat...
> (<
>
http://jenkins.ovirt.org/job/deploy-to_ovirt-master_tested/ws/@tmp/privat...
> )>
> [ssh-agent] Using credentials deploy-ovirt-experimental (SSH key for
> deploying to the tested repo)
> [deploy-to_ovirt-master_tested] $ /bin/bash -xe
> /tmp/jenkins7284571172790321491.sh
> + [[
jenkins:http://jenkins.ovirt.org/job/vdsm_standard-on-merge/795/ ==
> '' ]]
> + queue_name=ovirt-master
> + ssh -o StrictHostKeyChecking=no
> deploy-ovirt-experimental(a)resources.ovirt.org
> + echo repo-extra-dir:master
> + echo
jenkins:http://jenkins.ovirt.org/job/vdsm_standard-on-merge/795/
> Pseudo-terminal
> <
http://jenkins.ovirt.org/job/vdsm_standard-on-merge/795/Pseudo-terminal>
> will not be allocated because stdin is not a terminal.
> + BASE_DIR=/srv/resources/repos/ovirt/tested
> + PUBLISH_MD_COPIES=50
> + main
> + local tmp_dir
> + mkdir -p /srv/resources/repos/ovirt/tested
> ++ mktemp -d /srv/resources/repos/ovirt/tested/.deploy.XXXXXXXXXX
> Collecting packages
> + tmp_dir=/srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v
> + trap 'rm -rf
>
'\''/srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v'\'''
EXIT HUP
> + echo 'Collecting packages'
> + collect_packages /srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v
> + local repoman_dst=/srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v
> + repoman --temp-dir generate-in-repo --option
> main.allowed_repo_paths=/srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v
> --option main.on_empty_source=warn
> /srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v add conf:stdin
> 2019-06-04 22:37:56,171::INFO::repoman.cmd::
> 2019-06-04 22:37:56,171::INFO::repoman.cmd::Adding artifacts to the repo
> /srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v
> 2019-06-04 22:37:56,171::INFO::repoman.common.repo::Adding repo extra dir
> master
> 2019-06-04 22:37:56,173::INFO::repoman.common.stores.RPM::Loading repo
> /srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v/master
> 2019-06-04 22:37:56,173::INFO::repoman.common.stores.RPM::Repo
> /srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v/master loaded
> 2019-06-04 22:37:56,174::INFO::repoman.common.stores.iso::Loading repo
> /srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v/master
> 2019-06-04 22:37:56,175::INFO::repoman.common.stores.iso::Repo
> /srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v/master loaded
> 2019-06-04 22:37:56,183::INFO::repoman.common.repo::Resolving artifact
> source
jenkins:http://jenkins.ovirt.org/job/vdsm_standard-on-merge/795/
> 2019-06-04
> <
http://jenkins.ovirt.org/job/vdsm_standard-on-merge/795/2019-06-04>
> 22:37:56,308::ERROR::repoman.common.sources.jenkins::URL:
>
http://jenkins.ovirt.org/job/vdsm_standard-on-merge/795//api/json?depth=3
> Status: 503
> Reason: Service Unavailable
> Headers: {'cache-control': 'no-cache,no-store,must-revalidate',
> 'connection': 'close',
> 'content-encoding': 'gzip',
> 'content-length': '456',
> 'content-type': 'text/html;charset=utf-8',
> 'date': 'Tue, 04 Jun 2019 22:37:54 GMT',
> 'expires': '0',
> 'server': 'Jetty(9.4.z-SNAPSHOT)',
> 'via': '1.1 jenkins.phx.ovirt.org',
> 'x-content-type-options': 'nosniff'}
> Body:
>
>
> <!DOCTYPE html><html lang="en-US"><head
resURL="/static/539a36f3"
> data-rooturl=""
data-resurl="/static/539a36f3"><title>Starting
> Jenkins</title><meta name="ROBOTS" content="NOFOLLOW"
/><link
> rel="stylesheet" href="/static/539a36f3/css/simple-page.css"
> type="text/css" /><link rel="stylesheet"
> href="/static/539a36f3/css/simple-page.theme.css" type="text/css"
/><link
> rel="stylesheet" href="/static/539a36f3/css/loading.css"
type="text/css"
> /></head><body><div class="simple-page"
role="main"><div class="modal
> signup"><div class="signupIntroDefault"><div
class="logo"></div><h1
> class="loading">
> Please wait while Jenkins is getting ready to
> work
>
>
<span>.</span><span>.</span><span>.</span></h1><p>Your
browser will reload
> automatically when Jenkins is
ready.</p></div></div></div><script
> src="/static/539a36f3/scripts/loading.js"
>
type="text/javascript"></script><script>safeRedirector(window.location.href);</script></body></html>
>
> 2019-06-04 22:37:58,360::ERROR::repoman.common.sources.jenkins::URL:
>
http://jenkins.ovirt.org/job/vdsm_standard-on-merge/795//api/json?depth=3
> Status: 503
> Reason: Service Unavailable
> Headers: {'cache-control': 'no-cache,no-store,must-revalidate',
> 'connection': 'close',
> 'content-encoding': 'gzip',
> 'content-length': '456',
> 'content-type': 'text/html;charset=utf-8',
> 'date': 'Tue, 04 Jun 2019 22:37:56 GMT',
> 'expires': '0',
> 'server': 'Jetty(9.4.z-SNAPSHOT)',
> 'via': '1.1 jenkins.phx.ovirt.org',
> 'x-content-type-options': 'nosniff'}
> Body:
>
>
> <!DOCTYPE html><html lang="en-US"><head
resURL="/static/539a36f3"
> data-rooturl=""
data-resurl="/static/539a36f3"><title>Starting
> Jenkins</title><meta name="ROBOTS" content="NOFOLLOW"
/><link
> rel="stylesheet" href="/static/539a36f3/css/simple-page.css"
> type="text/css" /><link rel="stylesheet"
> href="/static/539a36f3/css/simple-page.theme.css" type="text/css"
/><link
> rel="stylesheet" href="/static/539a36f3/css/loading.css"
type="text/css"
> /></head><body><div class="simple-page"
role="main"><div class="modal
> signup"><div class="signupIntroDefault"><div
class="logo"></div><h1
> class="loading">
> Please wait while Jenkins is getting ready to
> work
>
>
<span>.</span><span>.</span><span>.</span></h1><p>Your
browser will reload
> automatically when Jenkins is
ready.</p></div></div></div><script
> src="/static/539a36f3/scripts/loading.js"
>
type="text/javascript"></script><script>safeRedirector(window.location.href);</script></body></html>
>
> 2019-06-04 22:38:00,436::ERROR::repoman.common.sources.jenkins::URL:
>
http://jenkins.ovirt.org/job/vdsm_standard-on-merge/795//api/json?depth=3
> Status: 503
> Reason: Service Unavailable
> Headers: {'cache-control': 'no-cache,no-store,must-revalidate',
> 'connection': 'close',
> 'content-encoding': 'gzip',
> 'content-length': '456',
> 'content-type': 'text/html;charset=utf-8',
> 'date': 'Tue, 04 Jun 2019 22:37:58 GMT',
> 'expires': '0',
> 'server': 'Jetty(9.4.z-SNAPSHOT)',
> 'via': '1.1 jenkins.phx.ovirt.org',
> 'x-content-type-options': 'nosniff'}
> Body:
>
>
> <!DOCTYPE html><html lang="en-US"><head
resURL="/static/539a36f3"
> data-rooturl=""
data-resurl="/static/539a36f3"><title>Starting
> Jenkins</title><meta name="ROBOTS" content="NOFOLLOW"
/><link
> rel="stylesheet" href="/static/539a36f3/css/simple-page.css"
> type="text/css" /><link rel="stylesheet"
> href="/static/539a36f3/css/simple-page.theme.css" type="text/css"
/><link
> rel="stylesheet" href="/static/539a36f3/css/loading.css"
type="text/css"
> /></head><body><div class="simple-page"
role="main"><div class="modal
> signup"><div class="signupIntroDefault"><div
class="logo"></div><h1
> class="loading">
> Please wait while Jenkins is getting ready to
> work
>
>
<span>.</span><span>.</span><span>.</span></h1><p>Your
browser will reload
> automatically when Jenkins is
ready.</p></div></div></div><script
> src="/static/539a36f3/scripts/loading.js"
>
type="text/javascript"></script><script>safeRedirector(window.location.href);</script></body></html>
>
> 2019-06-04 22:38:02,514::ERROR::repoman.common.sources.jenkins::URL:
>
http://jenkins.ovirt.org/job/vdsm_standard-on-merge/795//api/json?depth=3
> Status: 503
> Reason: Service Unavailable
> Headers: {'cache-control': 'no-cache,no-store,must-revalidate',
> 'connection': 'close',
> 'content-encoding': 'gzip',
> 'content-length': '456',
> 'content-type': 'text/html;charset=utf-8',
> 'date': 'Tue, 04 Jun 2019 22:38:01 GMT',
> 'expires': '0',
> 'server': 'Jetty(9.4.z-SNAPSHOT)',
> 'via': '1.1 jenkins.phx.ovirt.org',
> 'x-content-type-options': 'nosniff'}
> Body:
>
>
> <!DOCTYPE html><html lang="en-US"><head
resURL="/static/539a36f3"
> data-rooturl=""
data-resurl="/static/539a36f3"><title>Starting
> Jenkins</title><meta name="ROBOTS" content="NOFOLLOW"
/><link
> rel="stylesheet" href="/static/539a36f3/css/simple-page.css"
> type="text/css" /><link rel="stylesheet"
> href="/static/539a36f3/css/simple-page.theme.css" type="text/css"
/><link
> rel="stylesheet" href="/static/539a36f3/css/loading.css"
type="text/css"
> /></head><body><div class="simple-page"
role="main"><div class="modal
> signup"><div class="signupIntroDefault"><div
class="logo"></div><h1
> class="loading">
> Please wait while Jenkins is getting ready to
> work
>
>
<span>.</span><span>.</span><span>.</span></h1><p>Your
browser will reload
> automatically when Jenkins is
ready.</p></div></div></div><script
> src="/static/539a36f3/scripts/loading.js"
>
type="text/javascript"></script><script>safeRedirector(window.location.href);</script></body></html>
>
> 2019-06-04 22:38:04,516::ERROR::repoman.cmd::
> 2019-06-04 22:38:04,516::ERROR::repoman.cmd::Error while adding conf:stdin
> 2019-06-04 22:38:04,518::INFO::repoman.common.repo::Cleaning up temporary
> dir
> /srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v/.lago_tmp/tmpQ4NzXM/tmpgqw_Ee
> 2019-06-04 22:38:04,519::INFO::repoman.common.repo::Cleaning up temporary
> dir /srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v/.lago_tmp/tmpQ4NzXM
> + rm -rf /srv/resources/repos/ovirt/tested/.deploy.5Xqeujnn6v
> Build step 'Execute shell' marked build as failure
> $ ssh-agent -k
> unset SSH_AUTH_SOCK;
> unset SSH_AGENT_PID;
> echo Agent pid 19929 killed;
> [ssh-agent] Stopped.
> _______________________________________________
> Infra mailing list -- infra(a)ovirt.org
> To unsubscribe send an email to infra-leave(a)ovirt.org
> Privacy Statement:
https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct:
>
https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
>
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/ETOIPTHC5I4...
>
--
Emil Natan
RHV/CNV DevOps
_______________________________________________
Infra mailing list -- infra(a)ovirt.org
To unsubscribe send an email to infra-leave(a)ovirt.org
Privacy Statement:
https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/B5TI3BQQFN5...