[JIRA] (OVIRT-682) Improve CI logging
by eyal edri [Administrator] (oVirt JIRA)
[ https://ovirt-jira.atlassian.net/browse/OVIRT-682?page=com.atlassian.jira... ]
eyal edri [Administrator] reassigned OVIRT-682:
-----------------------------------------------
Assignee: infra (was: eyal edri [Administrator])
> Improve CI logging
> ------------------
>
> Key: OVIRT-682
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-682
> Project: oVirt - virtualization made easy
> Issue Type: By-EMAIL
> Reporter: eyal edri [Administrator]
> Assignee: infra
>
> On Fri, Aug 12, 2016 at 8:08 PM, Nir Soffer <nsoffer(a)redhat.com> wrote:
> > Hi all,
> >
> > Lately vdsm builds are failing in the install packages stage with
> > this unhelpful error:
> >
> > 13:59:42 INFO: installing package(s): autoconf automake gdb ...
> > 13:59:53 ERROR: Command failed. See logs for output.
> >
> > I downloaded the logs.tgz file from
> > http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc24-
> > x86_64/1154/artifact/exported-artifacts/logs.tgz
> >
> > And indeed in /./vdsm/logs/mocker-fedora-24-x86_64.fc24.install_packages/
> > root.log
> > I found found this error:
> >
> > DEBUG util.py:421: Error: Package:
> > python-ioprocess-0.17.0-1.201608111609.gitbd272f2.fc24.noarch
> > (ovirt-snapshot)
> > DEBUG util.py:421: Requires: ioprocess =
> > 0.17.0-1.201608111609.gitbd272f2.fc24
> > DEBUG util.py:421: Available:
> > ioprocess-0.15.1-1.fc24.x86_64 (fedora)
> > DEBUG util.py:421: ioprocess = 0.15.1-1.fc24
> > DEBUG util.py:421: Available:
> > ioprocess-0.16.1-1.fc24.x86_64 (updates)
> > DEBUG util.py:421: ioprocess = 0.16.1-1.fc24
> > DEBUG util.py:421: Available:
> > ioprocess-0.17.0-1.201607121058.gitbd272f2.fc24.x86_64
> > (ovirt-snapshot)
> > DEBUG util.py:421: ioprocess =
> > 0.17.0-1.201607121058.gitbd272f2.fc24
> > DEBUG util.py:421: Available:
> > ioprocess-0.17.0-1.201608111129.gitbd272f2.fc24.x86_64
> > (ovirt-snapshot)
> > DEBUG util.py:421: ioprocess =
> > 0.17.0-1.201608111129.gitbd272f2.fc24
> >
> > So we have 2 issues here:
> >
> > 1. We need *all* errors in the console
> > http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc24-
> > x86_64/1154/console
> Not sure its possible or easy to do, these errors usually come from mock
> and are not relevant to the code running on the console.
> I've cc'd infra-support and updated topic to track it and check our options.
> Going foward, we might want to install a log collector like logstash and
> then searching errors there should be much easier.
> >
> >
> > 2. Someone need to fix the ovirt-snapshot repository - it should have
> > the missing ioprocess
> > package.
> > Maybe the project xml is not correct?
> >
> > Nir
> >
> --
> Eyal Edri
> Associate Manager
> RHV DevOps
> EMEA ENG Virtualization R&D
> Red Hat Israel
> phone: +972-9-7692018
> irc: eedri (on #tlv #rhev-dev #rhev-integ)
--
This message was sent by Atlassian JIRA
(v1000.695.1#100025)
7 years, 11 months
[JIRA] (OVIRT-1054) PHX hosted engine VM restarted
by Evgheni Dereveanchin (oVirt JIRA)
[ https://ovirt-jira.atlassian.net/browse/OVIRT-1054?page=com.atlassian.jir... ]
Evgheni Dereveanchin reassigned OVIRT-1054:
-------------------------------------------
Assignee: Evgheni Dereveanchin (was: infra)
> PHX hosted engine VM restarted
> ------------------------------
>
> Key: OVIRT-1054
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1054
> Project: oVirt - virtualization made easy
> Issue Type: Bug
> Reporter: Evgheni Dereveanchin
> Assignee: Evgheni Dereveanchin
>
> After updating PHX oVirt to 4.0 as part of OVIRT-779 today I noticed that the engine VM I was working on suddenly went for shutdown.
> I saw the folowing in HA agent logs:
> MainThread::INFO::2017-01-19 13:54:08,837::hosted_engine::1187::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_stop_engine_vm) Shutting down vm using `/usr/sbin/hosted-engine --vm-shutdown`
> MainThread::INFO::2017-01-19 13:54:09,128::hosted_engine::1192::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_stop_engine_vm) stdout: Machine shutting down
> MainThread::INFO::2017-01-19 13:54:09,128::hosted_engine::1193::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_stop_engine_vm) stderr:
> MainThread::ERROR::2017-01-19 13:54:09,128::hosted_engine::1201::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_stop_engine_vm) Engine VM stopped on localhost
> Then the VM started again on the same host. Opening this ticket to track if this happens in the future and to better understand the cause as "hosted-engine --vm-status" was reporting correct Engine status while it was shutting down.
--
This message was sent by Atlassian JIRA
(v1000.695.1#100025)
7 years, 11 months
[JIRA] (OVIRT-1054) PHX hosted engine VM restarted
by Evgheni Dereveanchin (oVirt JIRA)
Evgheni Dereveanchin created OVIRT-1054:
-------------------------------------------
Summary: PHX hosted engine VM restarted
Key: OVIRT-1054
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1054
Project: oVirt - virtualization made easy
Issue Type: Bug
Reporter: Evgheni Dereveanchin
Assignee: infra
After updating PHX oVirt to 4.0 as part of OVIRT-779 today I noticed that the engine VM I was working on suddenly went for shutdown.
I saw the folowing in HA agent logs:
MainThread::INFO::2017-01-19 13:54:08,837::hosted_engine::1187::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_stop_engine_vm) Shutting down vm using `/usr/sbin/hosted-engine --vm-shutdown`
MainThread::INFO::2017-01-19 13:54:09,128::hosted_engine::1192::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_stop_engine_vm) stdout: Machine shutting down
MainThread::INFO::2017-01-19 13:54:09,128::hosted_engine::1193::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_stop_engine_vm) stderr:
MainThread::ERROR::2017-01-19 13:54:09,128::hosted_engine::1201::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_stop_engine_vm) Engine VM stopped on localhost
Then the VM started again on the same host. Opening this ticket to track if this happens in the future and to better understand the cause as "hosted-engine --vm-status" was reporting correct Engine status while it was shutting down.
--
This message was sent by Atlassian JIRA
(v1000.695.1#100025)
7 years, 11 months
[JIRA] (OVIRT-1053) ppc64le - qemu-kvm: Failed to allocate KVM HPT of order 33
by Evgheni Dereveanchin (oVirt JIRA)
[ https://ovirt-jira.atlassian.net/browse/OVIRT-1053?page=com.atlassian.jir... ]
Evgheni Dereveanchin reassigned OVIRT-1053:
-------------------------------------------
Assignee: Evgheni Dereveanchin (was: infra)
> ppc64le - qemu-kvm: Failed to allocate KVM HPT of order 33
> ----------------------------------------------------------
>
> Key: OVIRT-1053
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1053
> Project: oVirt - virtualization made easy
> Issue Type: Bug
> Reporter: Evgheni Dereveanchin
> Assignee: Evgheni Dereveanchin
>
> After updating PHX oVirt to 4.0 as part of OVIRT-779 the cluster level was updated on the ppc64le data center, yet VMs do not start any more with error:
> qemu-kvm: Failed to allocate KVM HPT of order 33 (try smaller maxmem?): Cannot allocate memory
> This looks like a known bug, opening this to track workarounds/solutions.
--
This message was sent by Atlassian JIRA
(v1000.695.1#100025)
7 years, 11 months
[JIRA] (OVIRT-1053) ppc64le - qemu-kvm: Failed to allocate KVM HPT of order 33
by Evgheni Dereveanchin (oVirt JIRA)
Evgheni Dereveanchin created OVIRT-1053:
-------------------------------------------
Summary: ppc64le - qemu-kvm: Failed to allocate KVM HPT of order 33
Key: OVIRT-1053
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1053
Project: oVirt - virtualization made easy
Issue Type: Bug
Reporter: Evgheni Dereveanchin
Assignee: infra
After updating PHX oVirt to 4.0 as part of OVIRT-779 the cluster level was updated on the ppc64le data center, yet VMs do not start any more with error:
qemu-kvm: Failed to allocate KVM HPT of order 33 (try smaller maxmem?): Cannot allocate memory
This looks like a known bug, opening this to track workarounds/solutions.
--
This message was sent by Atlassian JIRA
(v1000.695.1#100025)
7 years, 11 months