[ovirt-devel] [OST Failure Report][oVirt master][2017-04-11] add_secondary_storage_domains
Sandro Bonazzola
sbonazzo at redhat.com
Wed Apr 12 07:17:58 UTC 2017
On Wed, Apr 12, 2017 at 9:03 AM, Michal Skrivanek <
michal.skrivanek at redhat.com> wrote:
>
> On 12 Apr 2017, at 08:37, Yaniv Kaul <ykaul at redhat.com> wrote:
>
>
>
> On Wed, Apr 12, 2017 at 9:24 AM, Barak Korren <bkorren at redhat.com> wrote:
>
>> This seems to have been the Secodary SD issue I reported on Sunday:
>> http://lists.ovirt.org/pipermail/devel/2017-April/030139.html
>>
>> That was up until build #6307 where AddHost started failing:
>> http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6307
>>
>
> That's an entirely different game:
> *2017-04-11 11:24:36,783-04 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand]
> (org.ovirt.thread.pool-7-thread-1) [3497aa05] Error:
> java.lang.NoClassDefFoundError: org/apache/commons/lang/StringUtils*
> 2017-04-11 11:24:36,783-04 ERROR [org.ovirt.engine.core.
> vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-7-thread-1)
> [3497aa05] Exception: java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError:
>
>
>
> this one started in http://jenkins.ovirt.org/job/test-repo_ovirt_
> experimental_master/6307/
>
> there are 2 patches linkded, one is in vdsm-jsonrpc-java, which is what
> seems to be mismatched
> Piotr/Sandro, are all artifacts published correctly?
>
Please define published correctly.
Since the publisher job is not used anymore to publish artifacts I've no
list of what gets published.
I can only check repository content:
cockpit-ovirt-dashboard-0:0.11.0-0.0.2.el7.centos.noarch
cockpit-ovirt-uiplugin-0:0.11.0-0.0.2.el7.centos.noarch
imgbased-0:0.9.999-0.201704061527gita27110c.el7.centos.noarch
ioprocess-0:0.17.0-1.201702282323.gite82786d.el7.centos.x86_64
ioprocess-debuginfo-0:0.17.0-1.201702282323.gite82786d.el7.centos.x86_64
java-ovirt-engine-sdk4-0:4.1.0-0.1.20170331gitdcf3f4d.el7.centos.noarch
libcacard-devel-ev-10:2.3.0-31.el7_2.21.1.x86_64
libcacard-ev-10:2.3.0-31.el7_2.21.1.x86_64
libcacard-tools-ev-10:2.3.0-31.el7_2.21.1.x86_64
mom-0:0.5.9-0.0.master.el7.centos.noarch
nodectl-0:4.1-999.20160902.0.el7.noarch
otopi-0:1.7.0-0.0.master.20170405141623.git0166f14.el7.centos.noarch
otopi-debug-plugins-0:1.7.0-0.0.master.20170405141623.git0166f14.el7.centos.noarch
otopi-devtools-0:1.7.0-0.0.master.20170405141623.git0166f14.el7.centos.noarch
otopi-java-0:1.7.0-0.0.master.20170405141623.git0166f14.el7.centos.noarch
otopi-javadoc-0:1.7.0-0.0.master.20170405141623.git0166f14.el7.centos.noarch
ovirt-engine-0:4.2.0-0.0.master.20170408115943.gitea6bd62.el7.centos.noarch
ovirt-engine-appliance-0:4.2-20170408.1.el7.centos.noarch
ovirt-engine-backend-0:4.2.0-0.0.master.20170408115943.gitea6bd62.el7.centos.noarch
ovirt-engine-cli-0:3.6.9.3-0.1.20170116.gitad0561a.el7.centos.noarch
ovirt-engine-dashboard-0:1.2.0-0.5.20170331git8568fc5.el7.centos.noarch
ovirt-engine-dbscripts-0:4.2.0-0.0.master.20170408115943.gitea6bd62.el7.centos.noarch
ovirt-engine-dwh-0:4.2.0-0.0.master.20170403125608.el7.centos.noarch
ovirt-engine-dwh-setup-0:4.2.0-0.0.master.20170403125608.el7.centos.noarch
ovirt-engine-extension-aaa-jdbc-0:1.1.5-0.0.master.git2a03d42.el7.centos.noarch
ovirt-engine-extension-aaa-ldap-0:1.3.2-0.0.master.gitabe7725.el7.centos.noarch
ovirt-engine-extension-aaa-ldap-setup-0:1.3.2-0.0.master.gitabe7725.el7.centos.noarch
ovirt-engine-extension-aaa-misc-0:1.0.2-0.0.master.git9430be4.el7.centos.noarch
ovirt-engine-extension-logger-log4j-0:1.0.3-0.0.master.el7.centos.noarch
ovirt-engine-extensions-api-impl-0:4.2.0-0.0.master.20170408115943.gitea6bd62.el7.centos.noarch
ovirt-engine-extensions-api-impl-javadoc-0:4.2.0-0.0.master.20170408115943.gitea6bd62.el7.centos.noarch
ovirt-engine-hosts-ansible-inventory-0:4.2.0-0.0.master.20170302072348.git4674298.el7.centos.noarch
ovirt-engine-lib-0:4.2.0-0.0.master.20170408115943.gitea6bd62.el7.centos.noarch
ovirt-engine-metrics-0:1.0.3-0.1.master.20170406102316.el7.centos.noarch
ovirt-engine-nodejs-0:6.9.4-5.el7.centos.x86_64
ovirt-engine-nodejs-modules-0:1.0.4-5.el7.centos.x86_64
ovirt-engine-nodejs-modules-0:1.0.7-1.el7.centos.noarch
ovirt-engine-phantomjs-0:2.1.1-1.el7.centos.x86_64
ovirt-engine-phantomjs-debuginfo-0:2.1.1-1.el7.centos.x86_64
ovirt-engine-restapi-0:4.2.0-0.0.master.20170408115943.gitea6bd62.el7.centos.noarch
ovirt-engine-sdk-java-0:3.6.10.0-1.20170215git67dcbcd.el7.centos.noarch
ovirt-engine-sdk-java-javadoc-0:3.6.10.0-1.20170215git67dcbcd.el7.centos.noarch
ovirt-engine-sdk-python-0:3.6.9.2-0.1.20161204.gite99bbd1.el7.centos.noarch
ovirt-engine-setup-0:4.2.0-0.0.master.20170408115943.gitea6bd62.el7.centos.noarch
ovirt-engine-setup-base-0:4.2.0-0.0.master.20170408115943.gitea6bd62.el7.centos.noarch
ovirt-engine-setup-plugin-dockerc-0:4.2.0-0.0.master.20170408115943.gitea6bd62.el7.centos.noarch
ovirt-engine-setup-plugin-live-0:4.2.0-0.0.master.20170331075633.git8b425a3.el7.centos.noarch
ovirt-engine-setup-plugin-ovirt-engine-0:4.2.0-0.0.master.20170408115943.gitea6bd62.el7.centos.noarch
ovirt-engine-setup-plugin-ovirt-engine-common-0:4.2.0-0.0.master.20170408115943.gitea6bd62.el7.centos.noarch
ovirt-engine-setup-plugin-vmconsole-proxy-helper-0:4.2.0-0.0.master.20170408115943.gitea6bd62.el7.centos.noarch
ovirt-engine-setup-plugin-websocket-proxy-0:4.2.0-0.0.master.20170408115943.gitea6bd62.el7.centos.noarch
ovirt-engine-tools-0:4.2.0-0.0.master.20170408115943.gitea6bd62.el7.centos.noarch
ovirt-engine-tools-backup-0:4.2.0-0.0.master.20170408115943.gitea6bd62.el7.centos.noarch
ovirt-engine-userportal-0:4.2.0-0.0.master.20170408115943.gitea6bd62.el7.centos.noarch
ovirt-engine-userportal-debuginfo-0:4.2.0-0.0.master.20170408115943.gitea6bd62.el7.centos.noarch
ovirt-engine-vmconsole-proxy-helper-0:4.2.0-0.0.master.20170408115943.gitea6bd62.el7.centos.noarch
ovirt-engine-webadmin-portal-0:4.2.0-0.0.master.20170408115943.gitea6bd62.el7.centos.noarch
ovirt-engine-webadmin-portal-debuginfo-0:4.2.0-0.0.master.20170408115943.gitea6bd62.el7.centos.noarch
ovirt-engine-websocket-proxy-0:4.2.0-0.0.master.20170408115943.gitea6bd62.el7.centos.noarch
ovirt-engine-wildfly-0:10.1.0-1.el7.x86_64
ovirt-engine-wildfly-overlay-0:10.0.0-1.el7.noarch
ovirt-engine-yarn-0:0.19.1-2.el7.centos.x86_64
ovirt-engine-yarn-0:0.19.1-3.el7.centos.noarch
ovirt-guest-agent-common-0:1.0.13-1.el7.centos.noarch
ovirt-guest-agent-debuginfo-0:1.0.13-1.el7.centos.x86_64
ovirt-guest-agent-gdm-plugin-0:1.0.13-1.el7.centos.noarch
ovirt-guest-agent-pam-module-0:1.0.13-1.el7.centos.x86_64
ovirt-host-deploy-0:1.7.0-0.0.master.20170407114427.gitb41f5b9.el7.centos.noarch
ovirt-host-deploy-java-0:1.7.0-0.0.master.20170407114427.gitb41f5b9.el7.centos.noarch
ovirt-host-deploy-javadoc-0:1.7.0-0.0.master.20170407114427.gitb41f5b9.el7.centos.noarch
ovirt-hosted-engine-ha-0:2.2.0-0.0.master.20170404202238.20170404202234.git68258f1.el7.centos.noarch
ovirt-hosted-engine-setup-0:2.2.0-0.0.master.20170330072130.git8a05e04.el7.centos.noarch
ovirt-image-uploader-0:4.1.0-0.0.master.20160830110830.gitad0be34.el7.centos.noarch
ovirt-imageio-common-0:1.0.0-0.201703211442.gitfa6a59d.el7.centos.noarch
ovirt-imageio-daemon-0:1.0.0-0.201703211442.gitfa6a59d.el7.centos.noarch
ovirt-imageio-proxy-0:1.0.0-0.201703211442.gitfa6a59d.el7.centos.noarch
ovirt-imageio-proxy-setup-0:1.0.0-0.201703211442.gitfa6a59d.el7.centos.noarch
ovirt-iso-uploader-0:4.1.0-0.0.master.20160909154152.git14502bd.el7.centos.noarch
ovirt-jboss-modules-maven-plugin-0:2.0.2-0.1.20160715.gitec01507.el7.centos.noarch
ovirt-jboss-modules-maven-plugin-javadoc-0:2.0.2-0.1.20160715.gitec01507.el7.centos.noarch
ovirt-js-dependencies-0:0.0.3-2.el7.centos.x86_64
ovirt-js-dependencies-0:0.0.3-4.el7.centos.noarch
ovirt-live-artwork-0:1.1.0-0.0.master.20170331075633.git8b425a3.el7.centos.noarch
ovirt-live-artwork-gnome-0:1.1.0-0.0.master.20170331075633.git8b425a3.el7.centos.noarch
ovirt-live-artwork-mate-0:1.1.0-0.0.master.20170331075633.git8b425a3.el7.centos.noarch
ovirt-log-collector-0:4.1.0-0.0.master.20170405134355.gitd0eaad4.el7.centos.noarch
ovirt-log-collector-analyzer-0:4.1.0-0.0.master.20170405134355.gitd0eaad4.el7.centos.noarch
ovirt-node-ng-docs-0:4.1.0-0.20170408.0.el7.noarch
ovirt-node-ng-image-0:4.2.0-0.3.master.20170406003036.git2f43a3c.el7.centos.noarch
ovirt-node-ng-image-update-0:4.2.0-0.3.master.20170406003036.git2f43a3c.el7.centos.noarch
ovirt-node-ng-image-update-placeholder-0:4.2.0-0.3.master.20170408002716.git2f43a3c.el7.centos.noarch
ovirt-node-ng-nodectl-0:4.1.0-0.20170408.0.el7.noarch
ovirt-node-ng-tools-0:4.1.0-0.20170408.0.el7.noarch
ovirt-optimizer-0:0.15-0.176.201702021258.el7.centos.noarch
ovirt-optimizer-dependencies-0:0.15-0.176.201702021258.el7.centos.noarch
ovirt-optimizer-jboss-0:0.15-0.176.201702021258.el7.centos.noarch
ovirt-optimizer-ui-0:0.15-0.176.201702021258.el7.centos.noarch
ovirt-provider-ovn-0:1.1-2.20170407114314.git8f0d7ee.el7.centos.noarch
ovirt-provider-ovn-driver-0:1.1-2.20170407114314.git8f0d7ee.el7.centos.noarch
ovirt-release-host-node-0:4.2.0-0.3.master.20170408002716.git2f43a3c.el7.centos.noarch
ovirt-release-master-0:4.2.0-0.3.master.20170408002716.git2f43a3c.el7.centos.noarch
ovirt-release-master-experimental-0:4.2.0-0.3.master.20170408002716.git2f43a3c.el7.centos.noarch
ovirt-setup-lib-0:1.1.0-1.20161219151842.git402fec4.el7.centos.noarch
ovirt-vmconsole-0:1.0.4-0.0.master.20170301155209.git50f0014.el7.centos.noarch
ovirt-vmconsole-host-0:1.0.4-0.0.master.20170301155209.git50f0014.el7.centos.noarch
ovirt-vmconsole-proxy-0:1.0.4-0.0.master.20170301155209.git50f0014.el7.centos.noarch
ovirt-web-ui-0:0.1.2-1.el7.centos.x86_64
python-imgbased-0:0.9.999-0.201704061527gita27110c.el7.centos.noarch
python-ioprocess-0:0.17.0-1.201702282324.gite82786d.el7.centos.noarch
python-ovirt-engine-sdk4-0:4.2.0-1.a0.20170331gitd6b662b.el7.centos.x86_64
python-ovirt-engine-sdk4-debuginfo-0:4.2.0-1.a0.20170331gitd6b662b.el7.centos.x86_64
qemu-img-ev-10:2.6.0-28.el7_3.6.1.x86_64
qemu-kvm-common-ev-10:2.6.0-28.el7_3.6.1.x86_64
qemu-kvm-ev-10:2.6.0-28.el7_3.6.1.x86_64
qemu-kvm-ev-debuginfo-10:2.6.0-28.el7_3.6.1.x86_64
qemu-kvm-tools-ev-10:2.6.0-28.el7_3.6.1.x86_64
rubygem-ovirt-engine-sdk4-0:4.2.0-1.alpha0.20170407gite402de7.el7.centos.x86_64
rubygem-ovirt-engine-sdk4-debuginfo-0:4.2.0-1.alpha0.20170407gite402de7.el7.centos.x86_64
rubygem-ovirt-engine-sdk4-doc-0:4.2.0-1.alpha0.20170407gite402de7.el7.centos.x86_64
unboundid-ldapsdk-0:3.2.0-1.el7.noarch
unboundid-ldapsdk-javadoc-0:3.2.0-1.el7.noarch
vdsm-0:4.20.0-612.git378d837.el7.centos.x86_64
vdsm-api-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-client-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-gluster-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-allocate_net-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-checkimages-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-checkips-0:4.20.0-612.git378d837.el7.centos.x86_64
vdsm-hook-diskunmap-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-ethtool-options-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-extnet-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-extra-ipv4-addrs-0:4.20.0-612.git378d837.el7.centos.x86_64
vdsm-hook-fakesriov-0:4.20.0-612.git378d837.el7.centos.x86_64
vdsm-hook-fakevmstats-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-faqemu-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-fcoe-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-fileinject-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-floppy-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-hostusb-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-httpsisoboot-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-hugepages-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-ipv6-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-isolatedprivatevlan-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-localdisk-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-macbind-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-macspoof-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-nestedvt-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-noipspoof-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-numa-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-openstacknet-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-pincpu-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-promisc-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-qemucmdline-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-qos-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-scratchpad-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-smbios-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-spiceoptions-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-vhostmd-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-vmdisk-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-vmfex-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-hook-vmfex-dev-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-infra-0:4.18.999-212.git2556c05.el7.centos.noarch
vdsm-jsonrpc-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-jsonrpc-java-0:1.4.5-1.20170330074257.gitbef06c7.el7.centos.noarch
vdsm-jsonrpc-java-javadoc-0:1.4.5-1.20170330074257.gitbef06c7.el7.centos.noarch
vdsm-python-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-tests-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-xmlrpc-0:4.20.0-612.git378d837.el7.centos.noarch
vdsm-yajsonrpc-0:4.20.0-612.git378d837.el7.centos.noarch
>
> Thanks,
> michal
>
> org/apache/commons/lang/StringUtils
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> [rt.jar:1.8.0_121]
> at java.util.concurrent.FutureTask.get(FutureTask.java:206)
> [rt.jar:1.8.0_121]
> at org.ovirt.engine.core.vdsbroker.vdsbroker.FutureVDSCommand.get(FutureVDSCommand.java:69)
> [vdsbroker.jar:]
> at org.ovirt.engine.core.bll.network.host.HostSetupNetworkPoller.getValue(
> HostSetupNetworkPoller.java:56) [bll.jar:]
> at org.ovirt.engine.core.bll.network.host.HostSetupNetworkPoller.poll(
> HostSetupNetworkPoller.java:41) [bll.jar:]
>
> ...
> Caused by: java.lang.NoClassDefFoundError: org/apache/commons/lang/
> StringUtils
> at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.impl.Message.
> withCorrelationId(Message.java:75) [vdsm-jsonrpc-java-client.jar:]
> at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.SSLStompClient.
> sendMessage(SSLStompClient.java:85) [vdsm-jsonrpc-java-client.jar:]
> at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:83)
> [vdsm-jsonrpc-java-client.jar:]
> at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:91)
> [vdsbroker.jar:]
> at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.lambda$
> timeBoundPoll$1(JsonRpcVdsServer.java:965) [vdsbroker.jar:]
> at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$
> FutureCallable.call(JsonRpcVdsServer.java:484) [vdsbroker.jar:]
> at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$
> FutureCallable.call(JsonRpcVdsServer.java:473) [vdsbroker.jar:]
>
>
>
>>
>>
>>
>> On 11 April 2017 at 18:12, Piotr Kliczewski <piotr.kliczewski at gmail.com>
>> wrote:
>>
>>> I looks like SD activate failed on the engine side and the spm
>>> connection was gently closed by the engine:
>>>
>>> 2017-04-11 09:10:24,807-04 INFO [org.ovirt.engine.core.bll.st
>>> orage.domain.ActivateStorageDomainCommand] (default task-29) [2b66a42b]
>>> ActivateStorage Domain. After Connect all hosts to pool. Time: Tue Apr 11
>>> 09:10:24 EDT 2017
>>> 2017-04-11 09:10:24,842-04 ERROR [org.ovirt.engine.core.bll.sto
>>> rage.pool.RefreshPoolSingleAsyncOperation]
>>> (org.ovirt.thread.pool-7-thread-29) [2b66a42b] Could not connect vds
>>> 'lago-basic-suite-master-host0' to pool 'test-dc' - moving host to
>>> non-operational: null
>>> 2017-04-11 09:10:24,842-04 ERROR [org.ovirt.engine.core.bll.sto
>>> rage.pool.RefreshPoolSingleAsyncOperation]
>>> (org.ovirt.thread.pool-7-thread-28) [2b66a42b] Could not connect vds
>>> 'lago-basic-suite-master-host1' to pool 'test-dc' - moving host to
>>> non-operational: null
>>> 2017-04-11 09:10:24,904-04 INFO [org.ovirt.engine.core.bll.Se
>>> <http://org.ovirt.engine.core.bll.se/>tNonOperationalVdsCommand]
>>> (default task-29) [2bc83b1c] Running command: SetNonOperationalVdsCommand
>>> internal: true. Entities affected : ID: e1b611f2-67fb-4acb-8261-07133134e0e7
>>> Type: VDS
>>> 2017-04-11 09:10:24,909-04 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand]
>>> (default task-29) [2bc83b1c] START, SetVdsStatusVDSCommand(HostName =
>>> lago-basic-suite-master-host0, SetVdsStatusVDSCommandParameters:{runAsync='true',
>>> hostId='e1b611f2-67fb-4acb-8261-07133134e0e7', status='NonOperational',
>>> nonOperationalReason='STORAGE_DOMAIN_UNREACHABLE',
>>> stopSpmFailureLogged='false', maintenanceReason='null'}), log id: 5c45239
>>> 2017-04-11 09:10:24,919-04 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand]
>>> (default task-29) [2bc83b1c] FINISH, SetVdsStatusVDSCommand, log id: 5c45239
>>> 2017-04-11 09:10:24,922-04 DEBUG [org.ovirt.engine.core.dal.dbb
>>> roker.PostgresDbEngineDialect$PostgresSimpleJdbcCall] (default task-29)
>>> [2bc83b1c] Compiled stored procedure. Call string is [{call
>>> getvmsrunningonvds(?)}]
>>> 2017-04-11 09:10:24,922-04 DEBUG [org.ovirt.engine.core.dal.dbb
>>> roker.PostgresDbEngineDialect$PostgresSimpleJdbcCall] (default task-29)
>>> [2bc83b1c] SqlCall for procedure [GetVmsRunningOnVds] compiled
>>> 2017-04-11 09:10:24,985-04 WARN [org.ovirt.engine.core.dal.db
>>> broker.auditloghandling.AuditLogDirector] (default task-29) [2bc83b1c]
>>> EVENT_ID: VDS_SET_NONOPERATIONAL_DOMAIN(522), Correlation ID: 2bc83b1c,
>>> Job ID: d6455a45-48ad-42cd-9815-e9826944b9a0, Call Stack: null, Custom
>>> Event ID: -1, Message: Host lago-basic-suite-master-host0 cannot access the
>>> Storage Domain(s) iscsi attached to the Data Center test-dc. Setting Host
>>> state to Non-Operational.
>>> 2017-04-11 09:10:24,985-04 DEBUG [org.ovirt.engine.core.utils.timer.FixedDelayJobListener]
>>> (DefaultQuartzScheduler4) [] Rescheduling DEFAULT.org.ovirt.engine.core.
>>> vdsbroker.VdsManager.onTimer#-9223372036854775741 as there is no
>>> unfired trigger.
>>> 2017-04-11 09:10:25,036-04 INFO [org.ovirt.engine.core.bll.Se
>>> <http://org.ovirt.engine.core.bll.se/>tNonOperationalVdsCommand]
>>> (default task-29) [2ffa5981] Running command: SetNonOperationalVdsCommand
>>> internal: true. Entities affected : ID: 87a5f691-ea5c-4692-a4ab-52dad0dafe34
>>> Type: VDS
>>> 2017-04-11 09:10:25,039-04 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand]
>>> (default task-29) [2ffa5981] START, SetVdsStatusVDSCommand(HostName =
>>> lago-basic-suite-master-host1, SetVdsStatusVDSCommandParameters:{runAsync='true',
>>> hostId='87a5f691-ea5c-4692-a4ab-52dad0dafe34', status='NonOperational',
>>> nonOperationalReason='STORAGE_DOMAIN_UNREACHABLE',
>>> stopSpmFailureLogged='false', maintenanceReason='null'}), log id: 706ffc3b
>>>
>>> .....
>>>
>>> 2017-04-11 09:10:25,366-04 INFO [org.ovirt.engine.core.vdsbro
>>> ker.vdsbroker.SpmStopVDSCommand] (default task-29) [2ffa5981] FINISH,
>>> SpmStopVDSCommand, log id: 62f82cfb
>>> 2017-04-11 09:10:25,366-04 DEBUG [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.impl.Message]
>>> (default task-29) [2ffa5981] UNSUBSCRIBE
>>> id:4cd59c67-1b57-4456-9a16-52a72ac6f689
>>>
>>> \00
>>> 2017-04-11 09:10:25,367-04 DEBUG [org.ovirt.vdsm.jsonrpc.client
>>> .reactors.stomp.StompCommonClient] (default task-29) [2ffa5981] Message
>>> sent: UNSUBSCRIBE
>>>
>>>
>>>
>>> On Tue, Apr 11, 2017 at 4:49 PM, Sandro Bonazzola <sbonazzo at redhat.com>
>>> wrote:
>>>
>>>> Link to Job: http://jenkins.ovirt.org/job/t
>>>> est-repo_ovirt_experimental_master/6303
>>>>
>>>>
>>>> *OST is failing in the last 50 builds*
>>>>
>>>>
>>>>
>>>> *00:18:22.090* [basic_suit_el7] # add_secondary_storage_domains: *00:18:22.090* [basic_suit_el7] Error while running thread*00:18:22.090* [basic_suit_el7] Traceback (most recent call last):*00:18:22.090* [basic_suit_el7] File "/usr/lib/python2.7/site-packages/lago/utils.py", line 57, in _ret_via_queue*00:18:22.090* [basic_suit_el7] queue.put({'return': func()})*00:18:22.090* [basic_suit_el7] File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 427, in add_nfs_storage_domain*00:18:22.090* [basic_suit_el7] add_generic_nfs_storage_domain(prefix, SD_NFS_NAME, SD_NFS_HOST_NAME, SD_NFS_PATH, nfs_version='v4_2')*00:18:22.091* [basic_suit_el7] File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 441, in add_generic_nfs_storage_domain*00:18:22.091* [basic_suit_el7] add_generic_nfs_storage_domain_4(prefix, sd_nfs_name, nfs_host_name, mount_path, sd_format, sd_type, nfs_version)*00:18:22.091* [basic_suit_el7] File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 490, in add_generic_nfs_storage_domain_4*00:18:22.091* [basic_suit_el7] host=_random_host_from_dc_4(api, DC_NAME),*00:18:22.091* [basic_suit_el7] File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 106, in _random_host_from_dc_4*00:18:22.091* [basic_suit_el7] return random.choice(_hosts_in_dc_4(api, dc_name))*00:18:22.091* [basic_suit_el7] File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 100, in _hosts_in_dc_4*00:18:22.091* [basic_suit_el7] raise RuntimeError('Could not find hosts that are up in DC %s' % dc_name)*00:18:22.092* [basic_suit_el7] RuntimeError: Could not find hosts that are up in DC test-dc*00:18:22.092* [basic_suit_el7] Error while running thread*00:18:22.092* [basic_suit_el7] Traceback (most recent call last):*00:18:22.092* [basic_suit_el7] File "/usr/lib/python2.7/site-packages/lago/utils.py", line 57, in _ret_via_queue*00:18:22.092* [basic_suit_el7] queue.put({'return': func()})*00:18:22.092* [basic_suit_el7] File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 583, in add_iso_storage_domain*00:18:22.092* [basic_suit_el7] add_generic_nfs_storage_domain(prefix, SD_ISO_NAME, SD_ISO_HOST_NAME, SD_ISO_PATH, sd_format='v1', sd_type='iso', nfs_version='v3')*00:18:22.092* [basic_suit_el7] File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 441, in add_generic_nfs_storage_domain*00:18:22.092* [basic_suit_el7] add_generic_nfs_storage_domain_4(prefix, sd_nfs_name, nfs_host_name, mount_path, sd_format, sd_type, nfs_version)*00:18:22.093* [basic_suit_el7] File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 490, in add_generic_nfs_storage_domain_4*00:18:22.093* [basic_suit_el7] host=_random_host_from_dc_4(api, DC_NAME),*00:18:22.093* [basic_suit_el7] File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 106, in _random_host_from_dc_4*00:18:22.093* [basic_suit_el7] return random.choice(_hosts_in_dc_4(api, dc_name))*00:18:22.093* [basic_suit_el7] File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 100, in _hosts_in_dc_4*00:18:22.093* [basic_suit_el7] raise RuntimeError('Could not find hosts that are up in DC %s' % dc_name)*00:18:22.093* [basic_suit_el7] RuntimeError: Could not find hosts that are up in DC test-dc*00:18:22.094* [basic_suit_el7] Error while running thread*00:18:22.094* [basic_suit_el7] Traceback (most recent call last):*00:18:22.094* [basic_suit_el7] File "/usr/lib/python2.7/site-packages/lago/utils.py", line 57, in _ret_via_queue*00:18:22.094* [basic_suit_el7] queue.put({'return': func()})*00:18:22.094* [basic_suit_el7] File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 587, in add_templates_storage_domain*00:18:22.094* [basic_suit_el7] add_generic_nfs_storage_domain(prefix, SD_TEMPLATES_NAME, SD_TEMPLATES_HOST_NAME, SD_TEMPLATES_PATH, sd_format='v1', sd_type='export', nfs_version='v4_1')*00:18:22.094* [basic_suit_el7] File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 441, in add_generic_nfs_storage_domain*00:18:22.094* [basic_suit_el7] add_generic_nfs_storage_domain_4(prefix, sd_nfs_name, nfs_host_name, mount_path, sd_format, sd_type, nfs_version)*00:18:22.094* [basic_suit_el7] File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 490, in add_generic_nfs_storage_domain_4*00:18:22.095* [basic_suit_el7] host=_random_host_from_dc_4(api, DC_NAME),*00:18:22.095* [basic_suit_el7] File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 106, in _random_host_from_dc_4*00:18:22.095* [basic_suit_el7] return random.choice(_hosts_in_dc_4(api, dc_name))*00:18:22.095* [basic_suit_el7] File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 100, in _hosts_in_dc_4*00:18:22.095* [basic_suit_el7] raise RuntimeError('Could not find hosts that are up in DC %s' % dc_name)*00:18:22.095* [basic_suit_el7] RuntimeError: Could not find hosts that are up in DC test-dc*00:18:22.095* [basic_suit_el7] Error while running thread*00:18:22.095* [basic_suit_el7] Traceback (most recent call last):*00:18:22.096* [basic_suit_el7] File "/usr/lib/python2.7/site-packages/lago/utils.py", line 57, in _ret_via_queue*00:18:22.096* [basic_suit_el7] queue.put({'return': func()})*00:18:22.096* [basic_suit_el7] File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 436, in add_second_nfs_storage_domain*00:18:22.096* [basic_suit_el7] SD_NFS_HOST_NAME, SD_SECOND_NFS_PATH)*00:18:22.096* [basic_suit_el7] File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 441, in add_generic_nfs_storage_domain*00:18:22.096* [basic_suit_el7] add_generic_nfs_storage_domain_4(prefix, sd_nfs_name, nfs_host_name, mount_path, sd_format, sd_type, nfs_version)*00:18:22.096* [basic_suit_el7] File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 490, in add_generic_nfs_storage_domain_4*00:18:22.096* [basic_suit_el7] host=_random_host_from_dc_4(api, DC_NAME),*00:18:22.096* [basic_suit_el7] File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 106, in _random_host_from_dc_4*00:18:22.097* [basic_suit_el7] return random.choice(_hosts_in_dc_4(api, dc_name))*00:18:22.097* [basic_suit_el7] File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 100, in _hosts_in_dc_4*00:18:22.097* [basic_suit_el7] raise RuntimeError('Could not find hosts that are up in DC %s' % dc_name)*00:18:22.097* [basic_suit_el7] RuntimeError: Could not find hosts that are up in DC test-dc*00:18:22.097* [basic_suit_el7] Error while running thread*00:18:22.097* [basic_suit_el7] Traceback (most recent call last):*00:18:22.097* [basic_suit_el7] File "/usr/lib/python2.7/site-packages/lago/utils.py", line 57, in _ret_via_queue*00:18:22.098* [basic_suit_el7] queue.put({'return': func()})*00:18:22.098* [basic_suit_el7] File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 803, in import_template_from_glance*00:18:22.098* [basic_suit_el7] generic_import_from_glance(api, image_name=CIRROS_IMAGE_NAME, image_ext='_glance_template', as_template=True)*00:18:22.098* [basic_suit_el7] File "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 637, in generic_import_from_glance*00:18:22.098* [basic_suit_el7] target_image.import_image(import_action)*00:18:22.098* [basic_suit_el7] File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", line 26017, in import_image*00:18:22.098* [basic_suit_el7] headers={"Correlation-Id":correlation_id}*00:18:22.098* [basic_suit_el7] File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", line 122, in request*00:18:22.098* [basic_suit_el7] persistent_auth=self.__persistent_auth*00:18:22.099* [basic_suit_el7] File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 79, in do_request*00:18:22.099* [basic_suit_el7] persistent_auth)*00:18:22.099* [basic_suit_el7] File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.py", line 162, in __do_request*00:18:22.099* [basic_suit_el7] raise errors.RequestError(response_code, response_reason, response_body)*00:18:22.099* [basic_suit_el7] RequestError: *00:18:22.099* [basic_suit_el7] status: 400*00:18:22.099* [basic_suit_el7] reason: Bad Request*00:18:22.099* [basic_suit_el7] detail: Cannot import Virtual Disk: Storage Domain cannot be accessed.*00:18:22.099* [basic_suit_el7] -Please check that at least one Host is operational and Data Center state is up.
>>>>
>>>>
>>>> --
>>>> SANDRO BONAZZOLA
>>>>
>>>> ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D
>>>> Red Hat EMEA <https://www.redhat.com/>
>>>> <https://red.ht/sig>
>>>> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
>>>>
>>>> _______________________________________________
>>>> Devel mailing list
>>>> Devel at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>>
>>>
>>>
>>> _______________________________________________
>>> Devel mailing list
>>> Devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>
>>
>>
>>
>> --
>> Barak Korren
>> bkorren at redhat.com
>> RHCE, RHCi, RHV-DevOps Team
>> https://ifireball.wordpress.com/
>>
>> _______________________________________________
>> Devel mailing list
>> Devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/devel
>>
>
> _______________________________________________
> Devel mailing list
> Devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>
>
>
--
SANDRO BONAZZOLA
ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D
Red Hat EMEA <https://www.redhat.com/>
<https://red.ht/sig>
TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/devel/attachments/20170412/977e898f/attachment-0001.html>
More information about the Devel
mailing list