December 2016 - Infra - oVirt List Archives

[JIRA] (OVIRT-729) Jenkins leaves behind connections in ESTABLISHED state
by eyal edri [Administrator] (oVirt JIRA) 08 Dec '16

08 Dec '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-729?page=com.atlassian.jira.p… ] eyal edri [Administrator] reassigned OVIRT-729: ----------------------------------------------- Assignee: Evgheni Dereveanchin (was: infra) see if this is still relevant. > Jenkins leaves behind connections in ESTABLISHED state > ------------------------------------------------------ > > Key: OVIRT-729 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-729 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Evgheni Dereveanchin > Assignee: Evgheni Dereveanchin > Priority: Low > > Throughout the last days I noticed several times when a slave got non-responsive and after rebooting it never recovered until being removed/re-added to Jenkins. > Further investigation on el7-vm06.phx.ovirt.org revealed that on Jenkins the connection is still marked as "ESTABLISHED" hours after the slave crashed and rebooted: > {quote} > tcp6 0 0 66.187.230.92:33374 66.187.230.77:22 ESTABLISHED 14625/java > {quote} > Due to this new SSH connections are not opened and Jenkins can't reconnect to the slave. > I didn't see this behavior before, maybe it is related to the latest security upgrade of jenkins. -- This message was sent by Atlassian JIRA (v1000.620.0#100023)

1 0

[JIRA] (OVIRT-453) Gerrit: Issue 3885: Failed to index change
by eyal edri [Administrator] (oVirt JIRA) 08 Dec '16

08 Dec '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-453?page=com.atlassian.jira.p… ] eyal edri [Administrator] updated OVIRT-453: -------------------------------------------- Resolution: Cannot Reproduce Status: Done (was: To Do) Havn't seen this another time since the upgrade. We'll do a periodic upgrade to 2.13/2.14 which will anyway include a fix for this. the upgrade will be monitored on another ticket. > Gerrit: Issue 3885: Failed to index change > ------------------------------------------ > > Key: OVIRT-453 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-453 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: eyal edri [Administrator] > Assignee: Shlomo Ben David > > Its an open issue [1] in Gerrit, but still no fix available. > We'll track it and see if there is a pending fix, cc'ing infra-support to > open a ticket to track this. > [1] > https://code.google.com/p/gerrit/issues/detail?can=2&start=0&num=100&q=&col… > On Sun, Mar 27, 2016 at 7:03 AM, Nadav Goldin <ngoldin(a)redhat.com> wrote: > > Hi, > > gerrit started throwing weird errors this morning, such as: > > > >> internal server error: com.google.gerrit.server.git.UpdateException: > >> java.io.IOException: java.util.concurrent.ExecutionException: > >> org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed) > >> > > > > restarting it now. > > > > > > Nadav. > > > > > > _______________________________________________ > > Infra mailing list > > Infra(a)ovirt.org > > http://lists.ovirt.org/mailman/listinfo/infra > > > > > -- > Eyal Edri > Associate Manager > RHEV DevOps > EMEA ENG Virtualization R&D > Red Hat Israel > phone: +972-9-7692018 > irc: eedri (on #tlv #rhev-dev #rhev-integ) -- This message was sent by Atlassian JIRA (v1000.620.0#100023)

1 0

[JIRA] (OVIRT-296) [jenkins] take offline faulty bad slaves
by eyal edri [Administrator] (oVirt JIRA) 08 Dec '16

08 Dec '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-296?page=com.atlassian.jira.p… ] eyal edri [Administrator] updated OVIRT-296: -------------------------------------------- Priority: Medium (was: Highest) > [jenkins] take offline faulty bad slaves > ---------------------------------------- > > Key: OVIRT-296 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-296 > Project: oVirt - virtualization made easy > Issue Type: Task > Components: Jenkins > Affects Versions: Test > Reporter: eyal edri [Administrator] > Assignee: infra > Labels: jenkins, monitoring, > > it seems that quite often we hit an issue with a specific slave on phx, due to various reasons (out of space/git/network/etc..). > which leads to multiple jobs trying to run on it and failing. > we need an automated way of finding this. > proposal: > add post groovy build to jobs that will take a slave offline if it's misbehaves using: > manager.build.getBuiltOn().toComputer.setTemporarilyOffline(true) > the trick is to find such a slave and to be able to know if it failed consistently in the past X hours to justify it's disable. > we need some sort of counter or service to track slaves and thier error state and according to it take offline a specific slave. > for example: > if a slave was failing x jobs in Y time and runtime was < Z min , it might indicate such a problem. > (e.g 10 jobs were failing on the same slave in a timeframe of 5 min and job runtime was less than a 1 min.. ) > the post script should email infra(a)ovirt.org that it disabled a slave and we should look into it. -- This message was sent by Atlassian JIRA (v1000.620.0#100023)

1 0

[JIRA] (OVIRT-837) artifactory /opt dir isn't monitored for space
by eyal edri [Administrator] (oVirt JIRA) 08 Dec '16

08 Dec '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-837?page=com.atlassian.jira.p… ] eyal edri [Administrator] updated OVIRT-837: -------------------------------------------- Priority: Highest (was: High) > artifactory /opt dir isn't monitored for space > ---------------------------------------------- > > Key: OVIRT-837 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-837 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: eyal edri [Administrator] > Assignee: infra > Priority: Highest > > Artifactory application is running on /opt and if the disk is full the application is failing to provide jars and gets very slow ( but errors on out of space are seen in the logs ). -- This message was sent by Atlassian JIRA (v1000.620.0#100023)

1 0

[JIRA] (OVIRT-837) artifactory /opt dir isn't monitored for space
by eyal edri [Administrator] (oVirt JIRA) 08 Dec '16

08 Dec '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-837?page=com.atlassian.jira.p… ] eyal edri [Administrator] reassigned OVIRT-837: ----------------------------------------------- Assignee: Evgheni Dereveanchin (was: infra) > artifactory /opt dir isn't monitored for space > ---------------------------------------------- > > Key: OVIRT-837 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-837 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: eyal edri [Administrator] > Assignee: Evgheni Dereveanchin > Priority: Highest > > Artifactory application is running on /opt and if the disk is full the application is failing to provide jars and gets very slow ( but errors on out of space are seen in the logs ). -- This message was sent by Atlassian JIRA (v1000.620.0#100023)

1 0

[JIRA] (OVIRT-896) Re: Jenkins issues
by eyal edri [Administrator] (oVirt JIRA) 08 Dec '16

08 Dec '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-896?page=com.atlassian.jira.p… ] eyal edri [Administrator] updated OVIRT-896: -------------------------------------------- Resolution: Duplicate Status: Done (was: To Do) duplicate of #OVIRT-909 > Re: Jenkins issues > ------------------ > > Key: OVIRT-896 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-896 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: eyal edri [Administrator] > Assignee: Shlomo Ben David > > better to send such info to infra-support to document as a ticket. ( adding > infra-support ) > On Thu, Dec 1, 2016 at 12:22 PM, Fabian Deutsch <fdeutsch(a)redhat.com> wrote: > > Hey, > > > > so - I have an issue > > this job: > > is often picking a wrong commit! > > http://jenkins.ovirt.org/user/fabiand/my-views/view/ovirt- > > node-ng/job/ovirt-node-ng_master_build-artifacts-el7- > > x86_64/219/consoleFull > > > > It says it resetted to master, but master is actually a different commit. > > I observed this a few times in the recent days. > > > > Does somebody have an idea why this could be happening? > > > > 00:00:02.156 Checking out Revision > > c1970c6f7a6404e8ab030147eec81233905345cf (origin/master) > > > > on my local host: > > $ git rev-parse FETCH_HEAD^{commit} # timeout=10 > > 9123c1fd3ce15b02f2e69efdc367ae1477193aef > > > > c1970c6f7a6404e8ab030147eec81233905345cf -- does actually no exist (in > > my history) > > > > - fabian > > > -- > Eyal Edri > Associate Manager > RHV DevOps > EMEA ENG Virtualization R&D > Red Hat Israel > phone: +972-9-7692018 > irc: eedri (on #tlv #rhev-dev #rhev-integ) -- This message was sent by Atlassian JIRA (v1000.620.0#100023)

1 0

[JIRA] (OVIRT-922) Add new project to standard ci from github
by Ondra Machacek (oVirt JIRA) 08 Dec '16

08 Dec '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-922?page=com.atlassian.jira.p… ] Ondra Machacek updated OVIRT-922: --------------------------------- Blocked By: OVIRT-874 Status: Blocked (was: To Do) > Add new project to standard ci from github > ------------------------------------------ > > Key: OVIRT-922 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-922 > Project: oVirt - virtualization made easy > Issue Type: New Feature > Components: Jenkins > Reporter: Ondra Machacek > Assignee: infra > > I would like to request a jenkins machine for Ansible modules tests. > Those tests needs host with nested virtualization as it will create a few virtual machines for tests. > Actually, what I would need is similar machine which OST are using, but with preinstalled vagrant and ansible. > Then I would need some possibility to run this job from a hook from github, when new commit is pushed. > The tests are described here for an idea: > https://github.com/machacekondra/ovirt-tests > Would that be possible? -- This message was sent by Atlassian JIRA (v1000.620.0#100023)

1 0

[JIRA] (OVIRT-886) Yum install does not throw error on missing package
by Barak Korren (oVirt JIRA) 08 Dec '16

08 Dec '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-886?page=com.atlassian.jira.p… ] Barak Korren commented on OVIRT-886: ------------------------------------ Need to check if recent changes (see OVIRT-902) resolve this. > Yum install does not throw error on missing package > --------------------------------------------------- > > Key: OVIRT-886 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-886 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Gil Shinar > Assignee: infra > > When running el7 mock on fc24 (not necessary the issue) and one of the required packages is missing because a repository in .repos hadn't been added, yum will not fail and the package will not be installed -- This message was sent by Atlassian JIRA (v1000.620.0#100023)

1 0

[JIRA] (OVIRT-884) check workspace cleanup task
by Barak Korren (oVirt JIRA) 08 Dec '16

08 Dec '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-884?page=com.atlassian.jira.p… ] Barak Korren commented on OVIRT-884: ------------------------------------ Isn't this a duplicate of OVIRT-909 ? > check workspace cleanup task > ---------------------------- > > Key: OVIRT-884 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-884 > Project: oVirt - virtualization made easy > Issue Type: Task > Components: Jenkins > Reporter: Evgheni Dereveanchin > Assignee: infra > > Recently we get lots of failures due to dirty workspaces on Jenkins slaves. > The cleanup task is reporting success, yet its runtime is usually below 1 second, so it may not be running properly and causing the failures: > http://jenkins.ovirt.org/job/system_cleanup-workspaces/5261/console > Need to check this job to ensure it actually cleans up workspaces. -- This message was sent by Atlassian JIRA (v1000.620.0#100023)

1 0

[JIRA] (OVIRT-922) Add new project to standard ci from github
by eyal edri [Administrator] (oVirt JIRA) 08 Dec '16

08 Dec '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-922?page=com.atlassian.jira.p… ] eyal edri [Administrator] updated OVIRT-922: -------------------------------------------- Epic Link: OVIRT-400 > Add new project to standard ci from github > ------------------------------------------ > > Key: OVIRT-922 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-922 > Project: oVirt - virtualization made easy > Issue Type: New Feature > Components: Jenkins > Reporter: Ondra Machacek > Assignee: infra > > I would like to request a jenkins machine for Ansible modules tests. > Those tests needs host with nested virtualization as it will create a few virtual machines for tests. > Actually, what I would need is similar machine which OST are using, but with preinstalled vagrant and ansible. > Then I would need some possibility to run this job from a hook from github, when new commit is pushed. > The tests are described here for an idea: > https://github.com/machacekondra/ovirt-tests > Would that be possible? -- This message was sent by Atlassian JIRA (v1000.620.0#100023)

1 0