February 2019 - Infra - oVirt List Archives

[ OST Failure Report ] [ oVirt Master (ovirt-engine) ] [ 17-02-2019 ] [ 001_initialize_engine.test_initialize_engine ]
by Dafna Ron 17 Feb '19

17 Feb '19

Hi, we are failing upgrade from release suite on ovirt-engine master. CQ points below patch as root cause: https://gerrit.ovirt.org/#/c/97719/ - webadmin: don't allow trimming whitespace for the kernel cmdline Unfortunately the process logs are not collected and I have no way of seeing why the processes failed. I can try to debug this further in the morning ERROR: setup log: 2019-02-17 10:17:24,507-0500 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:926 execute-output: ('/usr/bin/systemctl', 'start', 'rh-postgresql95-postgresql.service') stderr: Job for rh-postgresql95-postgresql.service failed because the control process exited with error code. See "systemctl status rh-postgresql95-postgresql.service" and "journalctl -xe" for details. 2019-02-17 10:17:24,508-0500 DEBUG otopi.transaction transaction.abort:119 aborting 'File transaction for '/var/opt/rh/rh-postgresql95/lib/pgsql/data/pg_hba.conf'' 2019-02-17 10:17:24,509-0500 DEBUG otopi.context context._executeMethod:143 method exception Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/otopi/context.py", line 133, in _executeMethod method['method']() File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/provisioning/postgres.py", line 201, in _misc self._provisioning.provision() File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine_common/postgres.py", line 498, in provision self.restartPG() File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine_common/postgres.py", line 399, in restartPG state=state, File "/usr/share/otopi/plugins/otopi/services/systemd.py", line 141, in state service=name, RuntimeError: Failed to start service 'rh-postgresql95-postgresql' 2019-02-17 10:17:24,511-0500 ERROR otopi.context context._executeMethod:152 Failed to execute stage 'Misc configuration': Failed to start service 'rh-postgresql95-postgresql' 2019-02-17 10:17:24,556-0500 DEBUG otopi.plugins.otopi.debug.debug_failure.debug_failure debug_failure._notification:100 tcp connections: id uid local foreign state pid exe 0: 0 0.0.0.0:111 0.0.0.0:0 LISTEN 4191 /usr/sbin/rpcbind 1: 29 0.0.0.0:662 0.0.0.0:0 LISTEN 4225 /usr/sbin/rpc.statd 2: 0 0.0.0.0:22 0.0.0.0:0 LISTEN 3361 /usr/sbin/sshd 3: 0 0.0.0.0:892 0.0.0.0:0 LISTEN 4266 /usr/sbin/rpc.mountd 4: 0 0.0.0.0:2049 0.0.0.0:0 LISTEN UnknownPID UnknownEXE 5: 0 0.0.0.0:32803 0.0.0.0:0 LISTEN UnknownPID UnknownEXE 6: 0 192.168.201.2:22 192.168.201.1:54800 ESTABLISHED 8189 /usr/sbin/sshd 2019-02-17 10:17:24,557-0500 DEBUG otopi.transaction transaction.abort:119 aborting 'Yum Transaction' 2019-02-17 10:17:24,558-0500 INFO otopi.plugins.otopi.packagers.yumpackager yumpackager.info:80 Yum Performing yum transaction rollback Loaded plugins: fastestmirror, versionlock

2 4

[CQ]: 96550, 33 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 17 Feb '19

17 Feb '19

A system test invoked by the "ovirt-master" change queue including change 96550,33 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 97719,3 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 97719,3 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/96550/33 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/97719/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/12924/

1 0

[CQ]: 96549, 33 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 17 Feb '19

17 Feb '19

A system test invoked by the "ovirt-master" change queue including change 96549,33 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 97719,3 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 97719,3 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/96549/33 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/97719/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/12922/

1 0

[JENKINS] Failed to setup proejct kubevirt_kubevirt_standard-check-pr
by jenkins＠jenkins.phx.ovirt.org 17 Feb '19

17 Feb '19

Failed to run project_setup.sh for: #3616 kubevirt [check-patch]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-kubevirt-org-2lxh8.

1 0

[JENKINS] Failed to setup proejct kubevirt_kubevirt_standard-check-pr
by jenkins＠jenkins.phx.ovirt.org 17 Feb '19

17 Feb '19

Failed to run project_setup.sh for: #3615 kubevirt [check-patch]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-kubevirt-org-90rnw.

1 0

[JIRA] (OVIRT-2681) please move ovirt-openshift-extensions to container based slaves
by Roy Golan (oVirt JIRA) 17 Feb '19

17 Feb '19

Roy Golan created OVIRT-2681: -------------------------------- Summary: please move ovirt-openshift-extensions to container based slaves Key: OVIRT-2681 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2681 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: Roy Golan Assignee: infra I'd like to use multi-stage container builds, please adjust ovirt/ovirt-openshift-extensions project accordingly. Thanks, Roy -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100098)

1 0

[JENKINS] Failed to setup proejct kubevirt_kubevirt_standard-check-pr
by jenkins＠jenkins.phx.ovirt.org 17 Feb '19

17 Feb '19

Failed to run project_setup.sh for: #3611 kubevirt [check-patch]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-kubevirt-org-qwvsx.

1 0

[JENKINS] Failed to setup proejct kubevirt_kubevirt_standard-check-pr
by jenkins＠jenkins.phx.ovirt.org 17 Feb '19

17 Feb '19

Failed to run project_setup.sh for: #3610 kubevirt [check-patch]. It probably means that docker_cleanup.py failed. This step doesn't fail the job, but we do collect data about such failures to find the root cause. Infra owner, ensure that we're not running out of disk space on openshift-kubevirt-org-vn818.

1 0

[CQ]: 97715, 3 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 17 Feb '19

17 Feb '19

A system test invoked by the "ovirt-master" change queue including change 97715,3 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 97719,3 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 97719,3 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/97715/3 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/97719/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/12920/

1 0

[JIRA] (OVIRT-2680) puppetdb on foreman.ovirt.org OOM
by Emil Natan (oVirt JIRA) 17 Feb '19

17 Feb '19

Emil Natan created OVIRT-2680: --------------------------------- Summary: puppetdb on foreman.ovirt.org OOM Key: OVIRT-2680 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2680 Project: oVirt - virtualization made easy Issue Type: Task Reporter: Emil Natan Assignee: infra Pupperdb on forman.ovirt.org went down, it seems because of OOM. Below is the message from puppetdb-daemon.log. Not sure if this is the first time this happens, if yes then maybe just increase memory and swap. java.lang.OutOfMemoryError: Java heap space Dumping heap to /var/log/puppetdb/puppetdb-oom.hprof ... Heap dump file created [285263302 bytes in 2.953 secs] # # java.lang.OutOfMemoryError: Java heap space # -XX:OnOutOfMemoryError="kill -9 %p" # Executing /bin/sh -c "kill -9 6929"... -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100098)

1 0