[JIRA] (OVIRT-842) Artifactory service is restart by puppet in the loop
by Pavel Zhukov (oVirt JIRA)
Pavel Zhukov created OVIRT-842:
----------------------------------
Summary: Artifactory service is restart by puppet in the loop
Key: OVIRT-842
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-842
Project: oVirt - virtualization made easy
Issue Type: Epic
Reporter: Pavel Zhukov
Assignee: infra
Priority: High
For some reasons systemd reports artifactory as failed state while it works normally. This cause service to be restarted in the loop and DoS as the result.
As a workaround I've disabled puppet service to prevent artifactory from restarts and start it manually. Looks like PID file is not created.
As far as I know we had problem with free space on this server
--
This message was sent by Atlassian JIRA
(v1000.526.2#100018)
8 years, 1 month
vdsm_master_build-artifacts-fc24-ppc64le job keeps failing
by Irit Goihman
Hi,
It seems like there's a problem with this job.
All the jobs fail because of this error:
Start: yum install*15:11:34* ERROR: Command failed. See logs for output.
Can you please check?
Thanks,
--
Irit Goihman
Software Engineer
Red Hat Israel Ltd.
8 years, 1 month
OST failing for master - might be related to change in 'remove vm guest agent ...'
by Eyal Edri
Failing after this patch was merged:
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-fc24-x86...
[0][1]
https://gerrit.ovirt.org/#/c/65860/
[0]
http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/3329/con...
[1]
Lago log:
16-11-14 13:54:35,485::ssh.py::ssh::55::lago.ssh::DEBUG::Running
e0703cf0 on lago-basic-suite-master-host0: /usr/bin/systemctl status
--lines=0 qemu-guest-agent
2016-11-14 13:54:35,528::ssh.py::ssh::80::lago.ssh::DEBUG::Command
e0703cf0 on lago-basic-suite-master-host0 returned with 0
2016-11-14 13:54:35,528::ssh.py::ssh::88::lago.ssh::DEBUG::Command
e0703cf0 on lago-basic-suite-master-host0 output:
â— qemu-guest-agent.service - QEMU Guest Agent
Loaded: loaded (/usr/lib/systemd/system/qemu-guest-agent.service;
enabled; vendor preset: enabled)
Active: active (running) since Mon 2016-11-14 08:29:16 EST; 25min ago
Main PID: 476 (qemu-ga)
CGroup: /system.slice/qemu-guest-agent.service
└─476 /usr/bin/qemu-ga --method=virtio-serial
--path=/dev/virtio-ports/org.qemu.guest_agent.0
--blacklist=guest-file-open guest-file-close guest-file-read
guest-file-write guest-file-seek guest-file-flush
-F/etc/qemu-ga/fsfreeze-hook
2016-11-14 13:54:37,086::log_utils.py::end_log_task::649::root::ERROR::
# [Thread-2] lago-basic-suite-master-engine: [31mERROR [0m (in
0:00:02)
2016-11-14 13:54:37,087::utils.py::_ret_via_queue::57::lago.utils::ERROR::Error
while running thread
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/lago/utils.py", line 55, in
_ret_via_queue
queue.put({'return': func()})
File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1117,
in _collect_artifacts
vm.collect_artifacts(path)
File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
456, in collect_artifacts
) for guest_path in self._artifact_paths()
File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line
302, in extract_paths
return self.provider.extract_paths(paths, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/lago/vm.py", line 194, in extract_paths
self._extract_paths_live(paths=paths)
File "/usr/lib/python2.7/site-packages/lago/vm.py", line 409, in
_extract_paths_live
self._extract_paths_dead(paths=paths)
File "/usr/lib/python2.7/site-packages/lago/vm.py", line 423, in
_extract_paths_dead
gfs_cli.launch()
File "/usr/lib64/python2.7/site-packages/guestfs.py", line 4731, in launch
r = libguestfsmod.launch (self._o)
RuntimeError: guestfs_launch failed.
This usually means the libguestfs appliance failed to start or crashed.
See http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs
or run 'libguestfs-test-tool' and post the *complete* output into a
bug report or message to the libguestfs mailing list.
Exception from test:
*13:54:37* [36m # [Thread-2] lago-basic-suite-master-engine:
[31mERROR [0m (in 0:00:02) [0m*13:54:37* [31mError while running
thread*13:54:37* Traceback (most recent call last):*13:54:37* File
"/usr/lib/python2.7/site-packages/lago/utils.py", line 55, in
_ret_via_queue*13:54:37* queue.put({'return': func()})*13:54:37*
File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1117, in
_collect_artifacts*13:54:37* vm.collect_artifacts(path)*13:54:37*
File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 456,
in collect_artifacts*13:54:37* ) for guest_path in
self._artifact_paths()*13:54:37* File
"/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 302, in
extract_paths*13:54:37* return self.provider.extract_paths(paths,
*args, **kwargs)*13:54:37* File
"/usr/lib/python2.7/site-packages/lago/vm.py", line 194, in
extract_paths*13:54:37*
self._extract_paths_live(paths=paths)*13:54:37* File
"/usr/lib/python2.7/site-packages/lago/vm.py", line 409, in
_extract_paths_live*13:54:37*
self._extract_paths_dead(paths=paths)*13:54:37* File
"/usr/lib/python2.7/site-packages/lago/vm.py", line 423, in
_extract_paths_dead*13:54:37* gfs_cli.launch()*13:54:37* File
"/usr/lib64/python2.7/site-packages/guestfs.py", line 4731, in
launch*13:54:37* r = libguestfsmod.launch (self._o)*13:54:37*
RuntimeError: guestfs_launch failed.*13:54:37* This usually means the
libguestfs appliance failed to start or crashed.*13:54:37* See
http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs*13:54:37*
or run 'libguestfs-test-tool' and post the *complete* output into
a*13:54:37* bug report or message to the libguestfs mailing list.
[0m*13:54:47* [36m # [Thread-3] lago-basic-suite-master-host1:
[32mSuccess [0m (in 0:00:11) [0m*13:54:47* [36m # [Thread-1]
lago-basic-suite-master-host0: [32mSuccess [0m (in 0:00:11)
[0m*13:54:47* [36m@ Collect artifacts: [31mERROR [0m (in 0:00:11)
[0m*13:54:47* [31mError occured, aborting*13:54:47* Traceback (most
recent call last):*13:54:47* File
"/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 259, in
do_run*13:54:47*
self.cli_plugins[args.ovirtverb].do_run(args)*13:54:47* File
"/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 180, in
do_run*13:54:47* self._do_run(**vars(args))*13:54:47* File
"/usr/lib/python2.7/site-packages/lago/utils.py", line 490, in
wrapper*13:54:47* return func(*args, **kwargs)*13:54:47* File
"/usr/lib/python2.7/site-packages/lago/utils.py", line 501, in
wrapper*13:54:47* return func(*args, prefix=prefix,
**kwargs)*13:54:47* File
"/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 228, in
do_ovirt_collect*13:54:47*
prefix.collect_artifacts(output)*13:54:47* File
"/usr/lib/python2.7/site-packages/lago/log_utils.py", line 615, in
wrapper*13:54:47* return func(*args, **kwargs)*13:54:47* File
"/usr/lib/python2.7/site-packages/lago/prefix.py", line 1121, in
collect_artifacts*13:54:47*
self.virt_env.get_vms().values(),*13:54:47* File
"/usr/lib/python2.7/site-packages/lago/utils.py", line 97, in
invoke_in_parallel*13:54:47* vt.join_all()*13:54:47* File
"/usr/lib/python2.7/site-packages/lago/utils.py", line 55, in
_ret_via_queue*13:54:47* queue.put({'return': func()})*13:54:47*
File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1117, in
_collect_artifacts*13:54:47* vm.collect_artifacts(path)*13:54:47*
File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 456,
in collect_artifacts*13:54:47* ) for guest_path in
self._artifact_paths()*13:54:47* File
"/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 302, in
extract_paths*13:54:47* return self.provider.extract_paths(paths,
*args, **kwargs)*13:54:47* File
"/usr/lib/python2.7/site-packages/lago/vm.py", line 194, in
extract_paths*13:54:47*
self._extract_paths_live(paths=paths)*13:54:47* File
"/usr/lib/python2.7/site-packages/lago/vm.py", line 409, in
_extract_paths_live*13:54:47*
self._extract_paths_dead(paths=paths)*13:54:47* File
"/usr/lib/python2.7/site-packages/lago/vm.py", line 423, in
_extract_paths_dead*13:54:47* gfs_cli.launch()*13:54:47* File
"/usr/lib64/python2.7/site-packages/guestfs.py", line 4731, in
launch*13:54:47* r = libguestfsmod.launch (self._o)*13:54:47*
RuntimeError: guestfs_launch failed.*13:54:47* This usually means the
libguestfs appliance failed to start or crashed.*13:54:47* See
http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs*13:54:47*
or run 'libguestfs-test-tool' and post the *complete* output into
a*13:54:47* bug report or message to the libguestfs mailing list. [0m
On Mon, Nov 14, 2016 at 3:55 PM, <jenkins(a)jenkins.phx.ovirt.org> wrote:
> Build: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_
> master/3329/,
> Build Number: 3329,
> Build Status: FAILURE
> _______________________________________________
> Infra mailing list
> Infra(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/infra
>
>
--
Eyal Edri
Associate Manager
RHV DevOps
EMEA ENG Virtualization R&D
Red Hat Israel
phone: +972-9-7692018
irc: eedri (on #tlv #rhev-dev #rhev-integ)
8 years, 1 month