Stuck job for oVirt Engine CI?
Hi infra, I've observed over the weekend and today that sometimes ovirt-engine's CI seems to be stuck. E.g., take a look at [1]. The first out of four jobs started about 90 mintues(!) ago, and nothing seems to have progressed since. Can someone please take a look? Thanks, Allon [1] https://gerrit.ovirt.org/#/c/48155/
there seems to be a queue now in progress, so that's the reason it takes time. also, we're investigating an issue with the proxy from phx DC, that is down and affecting the jobs. but regadless, i wonder if we can't enable concurrent jobs for http://jenkins.ovirt.org/job/ovirt-engine_master_check-patch-el7-x86_64/ for e.g? right now its only running 1 instance at a time.. this goes for all 'check-patch' jobs. On Sun, Nov 15, 2015 at 2:52 PM, Allon Mureinik <amureini@redhat.com> wrote:
Hi infra,
I've observed over the weekend and today that sometimes ovirt-engine's CI seems to be stuck.
E.g., take a look at [1]. The first out of four jobs started about 90 mintues(!) ago, and nothing seems to have progressed since.
Can someone please take a look?
Thanks, Allon
[1] https://gerrit.ovirt.org/#/c/48155/
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
-- Eyal Edri Supervisor, RHEV CI EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
joining the rant. This is not the first time we experience that. On Sun, Nov 15, 2015 at 3:11 PM, Eyal Edri <eedri@redhat.com> wrote:
there seems to be a queue now in progress, so that's the reason it takes time. also, we're investigating an issue with the proxy from phx DC, that is down and affecting the jobs.
but regadless, i wonder if we can't enable concurrent jobs for http://jenkins.ovirt.org/job/ovirt-engine_master_check-patch-el7-x86_64/ for e.g? right now its only running 1 instance at a time..
this goes for all 'check-patch' jobs.
On Sun, Nov 15, 2015 at 2:52 PM, Allon Mureinik <amureini@redhat.com> wrote:
Hi infra,
I've observed over the weekend and today that sometimes ovirt-engine's CI seems to be stuck.
E.g., take a look at [1]. The first out of four jobs started about 90 mintues(!) ago, and nothing seems to have progressed since.
Can someone please take a look?
Thanks, Allon
[1] https://gerrit.ovirt.org/#/c/48155/
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
-- Eyal Edri Supervisor, RHEV CI EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
thanks for reporting roy, do you have an example for a patch that is waiting? the more reports we'll get on issues with the infra, the more chance there will be to fix it, we don't monitor the per patch jobs, its not possible with the amount, so we rely on reports from developers in order to fix it. On Sun, Nov 15, 2015 at 5:52 PM, Roy Golan <rgolan@redhat.com> wrote:
joining the rant. This is not the first time we experience that.
On Sun, Nov 15, 2015 at 3:11 PM, Eyal Edri <eedri@redhat.com> wrote:
there seems to be a queue now in progress, so that's the reason it takes time. also, we're investigating an issue with the proxy from phx DC, that is down and affecting the jobs.
but regadless, i wonder if we can't enable concurrent jobs for http://jenkins.ovirt.org/job/ovirt-engine_master_check-patch-el7-x86_64/ for e.g? right now its only running 1 instance at a time..
this goes for all 'check-patch' jobs.
On Sun, Nov 15, 2015 at 2:52 PM, Allon Mureinik <amureini@redhat.com> wrote:
Hi infra,
I've observed over the weekend and today that sometimes ovirt-engine's CI seems to be stuck.
E.g., take a look at [1]. The first out of four jobs started about 90 mintues(!) ago, and nothing seems to have progressed since.
Can someone please take a look?
Thanks, Allon
[1] https://gerrit.ovirt.org/#/c/48155/
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
-- Eyal Edri Supervisor, RHEV CI EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
-- Eyal Edri Supervisor, RHEV CI EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
I post to infra usually. I will report on thread when I see more of these. But why rely on developers report? Can't we measure how much time a job takes to trigger and if it exceeds it, report to infra automatically. On Sun, Nov 15, 2015 at 5:55 PM, Eyal Edri <eedri@redhat.com> wrote:
thanks for reporting roy, do you have an example for a patch that is waiting? the more reports we'll get on issues with the infra, the more chance there will be to fix it, we don't monitor the per patch jobs, its not possible with the amount, so we rely on reports from developers in order to fix it.
On Sun, Nov 15, 2015 at 5:52 PM, Roy Golan <rgolan@redhat.com> wrote:
joining the rant. This is not the first time we experience that.
On Sun, Nov 15, 2015 at 3:11 PM, Eyal Edri <eedri@redhat.com> wrote:
there seems to be a queue now in progress, so that's the reason it takes time. also, we're investigating an issue with the proxy from phx DC, that is down and affecting the jobs.
but regadless, i wonder if we can't enable concurrent jobs for http://jenkins.ovirt.org/job/ovirt-engine_master_check-patch-el7-x86_64/ for e.g? right now its only running 1 instance at a time..
this goes for all 'check-patch' jobs.
On Sun, Nov 15, 2015 at 2:52 PM, Allon Mureinik <amureini@redhat.com> wrote:
Hi infra,
I've observed over the weekend and today that sometimes ovirt-engine's CI seems to be stuck.
E.g., take a look at [1]. The first out of four jobs started about 90 mintues(!) ago, and nothing seems to have progressed since.
Can someone please take a look?
Thanks, Allon
[1] https://gerrit.ovirt.org/#/c/48155/
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
-- Eyal Edri Supervisor, RHEV CI EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
-- Eyal Edri Supervisor, RHEV CI EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
participants (3)
-
Allon Mureinik -
Eyal Edri -
Roy Golan