Re: [ovirt-devel] [OST Failure Report][oVirt master][2017-04-11] add_secondary_storage_domains

--Apple-Mail=_A73F9645-3BD9-4104-A39D-D19AEC539032 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 <mailto:bkorren@redhat.com>> wrote: <http://lists.ovirt.org/pipermail/devel/2017-April/030139.html> <http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6307> [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] = (org.ovirt.thread.pool-7-thread-1) [3497aa05] Error: = java.lang.NoClassDefFoundError: org/apache/commons/lang/StringUtils [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] = (org.ovirt.thread.pool-7-thread-1) [3497aa05] Exception: = java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: this one started in = http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6307/ = <http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6307/> there are 2 patches linkded, one is in vdsm-jsonrpc-java, which is what = seems to be mismatched Piotr/Sandro, are all artifacts published correctly? Thanks, michal [rt.jar:1.8.0_121] [rt.jar:1.8.0_121] org.ovirt.engine.core.vdsbroker.vdsbroker.FutureVDSCommand.get(FutureVDSCo= mmand.java:69) [vdsbroker.jar:] org.ovirt.engine.core.bll.network.host.HostSetupNetworkPoller.getValue(Hos= tSetupNetworkPoller.java:56) [bll.jar:] org.ovirt.engine.core.bll.network.host.HostSetupNetworkPoller.poll(HostSet= upNetworkPoller.java:41) [bll.jar:] org/apache/commons/lang/StringUtils org.ovirt.vdsm.jsonrpc.client.reactors.stomp.impl.Message.withCorrelationI= d(Message.java:75) [vdsm-jsonrpc-java-client.jar:] org.ovirt.vdsm.jsonrpc.client.reactors.stomp.SSLStompClient.sendMessage(SS= LStompClient.java:85) [vdsm-jsonrpc-java-client.jar:] org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:83) = [vdsm-jsonrpc-java-client.jar:] org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:91= ) [vdsbroker.jar:] org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.lambda$timeBoundP= oll$1(JsonRpcVdsServer.java:965) [vdsbroker.jar:] org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$FutureCallable.ca= ll(JsonRpcVdsServer.java:484) [vdsbroker.jar:] org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$FutureCallable.ca= ll(JsonRpcVdsServer.java:473) [vdsbroker.jar:] <piotr.kliczewski(a)gmail.com <mailto:piotr.kliczewski@gmail.com>> wrote: connection was gently closed by the engine: <http://org.ovirt.engine.core.bll.st/>orage.domain.ActivateStorageDomai... mmand] (default task-29) [2b66a42b] ActivateStorage Domain. After = Connect all hosts to pool. Time: Tue Apr 11 09:10:24 EDT 2017 [org.ovirt.engine.core.bll.storage.pool.RefreshPoolSingleAsyncOperation] = (org.ovirt.thread.pool-7-thread-29) [2b66a42b] Could not connect vds = 'lago-basic-suite-master-host0' to pool 'test-dc' - moving host to = non-operational: null [org.ovirt.engine.core.bll.storage.pool.RefreshPoolSingleAsyncOperation] = (org.ovirt.thread.pool-7-thread-28) [2b66a42b] Could not connect vds = 'lago-basic-suite-master-host1' to pool 'test-dc' - moving host to = non-operational: null <http://org.ovirt.engine.core.bll.se/>tNonOperationalVdsCommand] = (default task-29) [2bc83b1c] Running command: = SetNonOperationalVdsCommand internal: true. Entities affected : ID: = e1b611f2-67fb-4acb-8261-07133134e0e7 Type: VDS [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (default = task-29) [2bc83b1c] START, SetVdsStatusVDSCommand(HostName =3D = lago-basic-suite-master-host0, = SetVdsStatusVDSCommandParameters:{runAsync=3D'true', = hostId=3D'e1b611f2-67fb-4acb-8261-07133134e0e7', = status=3D'NonOperational', = nonOperationalReason=3D'STORAGE_DOMAIN_UNREACHABLE', = stopSpmFailureLogged=3D'false', maintenanceReason=3D'null'}), log id: = 5c45239 [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (default = task-29) [2bc83b1c] FINISH, SetVdsStatusVDSCommand, log id: 5c45239 [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimple= JdbcCall] (default task-29) [2bc83b1c] Compiled stored procedure. Call = string is [{call getvmsrunningonvds(?)}] [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimple= JdbcCall] (default task-29) [2bc83b1c] SqlCall for procedure = [GetVmsRunningOnVds] compiled [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] = (default task-29) [2bc83b1c] EVENT_ID: = VDS_SET_NONOPERATIONAL_DOMAIN(522), Correlation ID: 2bc83b1c, Job ID: = d6455a45-48ad-42cd-9815-e9826944b9a0, Call Stack: null, Custom Event ID: = -1, Message: Host lago-basic-suite-master-host0 cannot access the = Storage Domain(s) iscsi attached to the Data Center test-dc. Setting = Host state to Non-Operational. [org.ovirt.engine.core.utils.timer.FixedDelayJobListener] = (DefaultQuartzScheduler4) [] Rescheduling = DEFAULT.org.ovirt.engine.core.vdsbroker.VdsManager.onTimer#-92233720368547= 75741 as there is no unfired trigger. <http://org.ovirt.engine.core.bll.se/>tNonOperationalVdsCommand] = (default task-29) [2ffa5981] Running command: = SetNonOperationalVdsCommand internal: true. Entities affected : ID: = 87a5f691-ea5c-4692-a4ab-52dad0dafe34 Type: VDS [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (default = task-29) [2ffa5981] START, SetVdsStatusVDSCommand(HostName =3D = lago-basic-suite-master-host1, = SetVdsStatusVDSCommandParameters:{runAsync=3D'true', = hostId=3D'87a5f691-ea5c-4692-a4ab-52dad0dafe34', = status=3D'NonOperational', = nonOperationalReason=3D'STORAGE_DOMAIN_UNREACHABLE', = stopSpmFailureLogged=3D'false', maintenanceReason=3D'null'}), log id: = 706ffc3b [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default = task-29) [2ffa5981] FINISH, SpmStopVDSCommand, log id: 62f82cfb [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.impl.Message] (default = task-29) [2ffa5981] UNSUBSCRIBE [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] = (default task-29) [2ffa5981] Message sent: UNSUBSCRIBE <mailto:sbonazzo@redhat.com>> wrote: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6303 = <http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/6303> "/usr/lib/python2.7/site-packages/lago/utils.py", line 57, in = _ret_via_queue "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-= tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 427, in = add_nfs_storage_domain add_generic_nfs_storage_domain(prefix, SD_NFS_NAME, SD_NFS_HOST_NAME, = SD_NFS_PATH, nfs_version=3D'v4_2') "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-= tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 441, in = add_generic_nfs_storage_domain add_generic_nfs_storage_domain_4(prefix, sd_nfs_name, nfs_host_name, = mount_path, sd_format, sd_type, nfs_version) "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-= tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 490, in = add_generic_nfs_storage_domain_4 DC_NAME), "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-= tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 106, in = _random_host_from_dc_4 random.choice(_hosts_in_dc_4(api, dc_name)) "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-= tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 100, in = _hosts_in_dc_4 hosts that are up in DC %s' % dc_name) are up in DC test-dc "/usr/lib/python2.7/site-packages/lago/utils.py", line 57, in = _ret_via_queue "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-= tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 583, in = add_iso_storage_domain add_generic_nfs_storage_domain(prefix, SD_ISO_NAME, SD_ISO_HOST_NAME, = SD_ISO_PATH, sd_format=3D'v1', sd_type=3D'iso', nfs_version=3D'v3') "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-= tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 441, in = add_generic_nfs_storage_domain add_generic_nfs_storage_domain_4(prefix, sd_nfs_name, nfs_host_name, = mount_path, sd_format, sd_type, nfs_version) "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-= tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 490, in = add_generic_nfs_storage_domain_4 DC_NAME), "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-= tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 106, in = _random_host_from_dc_4 random.choice(_hosts_in_dc_4(api, dc_name)) "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-= tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 100, in = _hosts_in_dc_4 hosts that are up in DC %s' % dc_name) are up in DC test-dc "/usr/lib/python2.7/site-packages/lago/utils.py", line 57, in = _ret_via_queue "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-= tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 587, in = add_templates_storage_domain add_generic_nfs_storage_domain(prefix, SD_TEMPLATES_NAME, = SD_TEMPLATES_HOST_NAME, SD_TEMPLATES_PATH, sd_format=3D'v1', = sd_type=3D'export', nfs_version=3D'v4_1') "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-= tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 441, in = add_generic_nfs_storage_domain add_generic_nfs_storage_domain_4(prefix, sd_nfs_name, nfs_host_name, = mount_path, sd_format, sd_type, nfs_version) "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-= tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 490, in = add_generic_nfs_storage_domain_4 DC_NAME), "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-= tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 106, in = _random_host_from_dc_4 random.choice(_hosts_in_dc_4(api, dc_name)) "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-= tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 100, in = _hosts_in_dc_4 hosts that are up in DC %s' % dc_name) are up in DC test-dc "/usr/lib/python2.7/site-packages/lago/utils.py", line 57, in = _ret_via_queue "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-= tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 436, in = add_second_nfs_storage_domain SD_SECOND_NFS_PATH) "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-= tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 441, in = add_generic_nfs_storage_domain add_generic_nfs_storage_domain_4(prefix, sd_nfs_name, nfs_host_name, = mount_path, sd_format, sd_type, nfs_version) "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-= tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 490, in = add_generic_nfs_storage_domain_4 DC_NAME), "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-= tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 106, in = _random_host_from_dc_4 random.choice(_hosts_in_dc_4(api, dc_name)) "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-= tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 100, in = _hosts_in_dc_4 hosts that are up in DC %s' % dc_name) are up in DC test-dc > 00:18:22.097 [basic_suit_el7] Error while running thread > 00:18:22.097 [basic_suit_el7] Traceback (most recent call last): "/usr/lib/python2.7/site-packages/lago/utils.py", line 57, in = _ret_via_queue "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-= tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 803, in = import_template_from_glance image_name=3DCIRROS_IMAGE_NAME, image_ext=3D'_glance_template', = as_template=3DTrue) "/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-= tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 637, in = generic_import_from_glance target_image.import_image(import_action) "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py", = line 26017, in import_image headers=3D{"Correlation-Id":correlation_id} "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py", = line 122, in request persistent_auth=3Dself.__persistent_auth "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.= py", line 79, in do_request > 00:18:22.099 [basic_suit_el7] persistent_auth) "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connectionspool.= py", line 162, in __do_request errors.RequestError(response_code, response_reason, response_body) Storage Domain cannot be accessed. operational and Data Center state is up. <http://lists.ovirt.org/mailman/listinfo/devel> <http://lists.ovirt.org/mailman/listinfo/devel> <http://lists.ovirt.org/mailman/listinfo/devel> --Apple-Mail=_A73F9645-3BD9-4104-A39D-D19AEC539032 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html = charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" = class=3D""><br class=3D""><div><blockquote type=3D"cite" class=3D""><div = class=3D"">On 12 Apr 2017, at 08:37, Yaniv Kaul &lt;<a = href=3D"mailto:ykaul@redhat.com" class=3D&quot;&quot;&gt;ykaul(a)redhat.com&lt;/a&gt;&amp;gt; = wrote:</div><br class=3D"Apple-interchange-newline"><div class=3D""><div = dir=3D"ltr" class=3D""><br class=3D""><div class=3D"gmail_extra"><br = class=3D""><div class=3D"gmail_quote">On Wed, Apr 12, 2017 at 9:24 AM, = Barak Korren <span dir=3D"ltr" class=3D"">&lt;<a = href=3D"mailto:bkorren@redhat.com" target=3D"_blank" = class=3D&quot;&quot;&gt;bkorren(a)redhat.com&lt;/a&gt;&amp;gt;&lt;/span&gt; wrote:<br = class=3D""><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px = 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div = dir=3D"ltr" class=3D""><div class=3D"">This seems to have been the = Secodary SD issue I reported on Sunday:<br class=3D""><a = href=3D"http://lists.ovirt.org/pipermail/devel/2017-April/030139.htm... = target=3D"_blank" class=3D"">http://lists.ovirt.org/<wbr = class=3D"">pipermail/devel/2017-April/<wbr class=3D"">030139.html</a><br = class=3D""><br class=3D""></div>That was up until build #6307 where = AddHost started failing:<br class=3D""><a = href=3D"http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_ma... 307" target=3D"_blank" class=3D"">http://jenkins.ovirt.org/job/<wbr = class=3D"">test-repo_ovirt_experimental_<wbr = class=3D"">master/6307</a></div></blockquote><div class=3D""><br = class=3D""></div><div class=3D"">That's an entirely different = game:</div><div class=3D""><b class=3D"">2017-04-11 11:24:36,783-04 = ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] = (org.ovirt.thread.pool-7-thread-1) [3497aa05] Error: = java.lang.NoClassDefFoundError: = org/apache/commons/lang/StringUtils</b></div><div class=3D"">2017-04-11 = 11:24:36,783-04 ERROR = [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] = (org.ovirt.thread.pool-7-thread-1) [3497aa05] Exception: = java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: = </div></div></div></div></div></blockquote><div><br class=3D""></div>this = one started in&nbsp;<a = href=3D"http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_ma... 307/" = class=3D"">http://jenkins.ovirt.org/job/test-repo_ovirt_expe... r/6307/</a></div><div><br class=3D""></div><div>there are 2 patches = linkded, one is in vdsm-jsonrpc-java, which is what seems to be = mismatched</div><div>Piotr/Sandro, are all artifacts published = correctly?</div><div><br = class=3D""></div><div>Thanks,</div><div>michal</div><div><br = class=3D""><blockquote type=3D"cite" class=3D""><div class=3D""><div = dir=3D"ltr" class=3D""><div class=3D"gmail_extra"><div = class=3D"gmail_quote"><div = class=3D"">org/apache/commons/lang/StringUtils</div><div class=3D""><span = class=3D"gmail-Apple-tab-span" style=3D"white-space:pre"> = </span>at java.util.concurrent.FutureTask.report(FutureTask.java:122) = [rt.jar:1.8.0_121]</div><div class=3D""><span = class=3D"gmail-Apple-tab-span" style=3D"white-space:pre"> = </span>at java.util.concurrent.FutureTask.get(FutureTask.java:206) = [rt.jar:1.8.0_121]</div><div class=3D""><span = class=3D"gmail-Apple-tab-span" style=3D"white-space:pre"> = </span>at = org.ovirt.engine.core.vdsbroker.vdsbroker.FutureVDSCommand.get(FutureVDSCo= mmand.java:69) [vdsbroker.jar:]</div><div class=3D""><span = class=3D"gmail-Apple-tab-span" style=3D"white-space:pre"> = </span>at = org.ovirt.engine.core.bll.network.host.HostSetupNetworkPoller.getValue(Hos= tSetupNetworkPoller.java:56) [bll.jar:]</div><div class=3D""><span = class=3D"gmail-Apple-tab-span" style=3D"white-space:pre"> = </span>at = org.ovirt.engine.core.bll.network.host.HostSetupNetworkPoller.poll(HostSet= upNetworkPoller.java:41) [bll.jar:]</div><div class=3D""><br = class=3D""></div><div class=3D"">...</div><div class=3D"">Caused by: = java.lang.NoClassDefFoundError: = org/apache/commons/lang/StringUtils</div><div class=3D""><span = class=3D"gmail-Apple-tab-span" style=3D"white-space:pre"> = </span>at = org.ovirt.vdsm.jsonrpc.client.reactors.stomp.impl.Message.withCorrelationI= d(Message.java:75) [vdsm-jsonrpc-java-client.jar:]</div><div = class=3D""><span class=3D"gmail-Apple-tab-span" style=3D"white-space:pre">= </span>at = org.ovirt.vdsm.jsonrpc.client.reactors.stomp.SSLStompClient.sendMessage(SS= LStompClient.java:85) [vdsm-jsonrpc-java-client.jar:]</div><div = class=3D""><span class=3D"gmail-Apple-tab-span" style=3D"white-space:pre">= </span>at = org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:83) = [vdsm-jsonrpc-java-client.jar:]</div><div class=3D""><span = class=3D"gmail-Apple-tab-span" style=3D"white-space:pre"> = </span>at = org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.&lt;init&gt;(FutureMap.j= ava:91) [vdsbroker.jar:]</div><div class=3D""><span = class=3D"gmail-Apple-tab-span" style=3D"white-space:pre"> = </span>at = org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.lambda$timeBoundP= oll$1(JsonRpcVdsServer.java:965) [vdsbroker.jar:]</div><div = class=3D""><span class=3D"gmail-Apple-tab-span" style=3D"white-space:pre">= </span>at = org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$FutureCallable.ca= ll(JsonRpcVdsServer.java:484) [vdsbroker.jar:]</div><div class=3D""><span = class=3D"gmail-Apple-tab-span" style=3D"white-space:pre"> = </span>at = org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$FutureCallable.ca= ll(JsonRpcVdsServer.java:473) [vdsbroker.jar:]</div><div class=3D""><br = class=3D""></div><div class=3D"">&nbsp;</div><blockquote = class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px = solid rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr" class=3D""><br = class=3D""><br class=3D""><div class=3D""><div class=3D""><div = class=3D"gmail_extra"><div class=3D""><div class=3D"gmail-h5"><br = class=3D""><div class=3D"gmail_quote">On 11 April 2017 at 18:12, Piotr = Kliczewski <span dir=3D"ltr" class=3D"">&lt;<a = href=3D"mailto:piotr.kliczewski@gmail.com" target=3D"_blank" = class=3D&quot;&quot;&gt;piotr.kliczewski(a)gmail.com&lt;/a&gt;&amp;gt;&lt;/span&gt; wrote:<br = class=3D""><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px = 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div = dir=3D"ltr" class=3D"">I looks like SD activate failed on the engine = side and the spm connection was gently closed by the engine:<div = class=3D""><br class=3D""></div><div class=3D""><div class=3D"">2017-04-11= 09:10:24,807-04 INFO &nbsp;[<a = href=3D"http://org.ovirt.engine.core.bll.st/" = class=3D"">org.ovirt.engine.core.bll.st</a><wbr = class=3D"">orage.domain.ActivateStorageDo<wbr class=3D"">mainCommand] = (default task-29) [2b66a42b] ActivateStorage Domain. After Connect all = hosts to pool. Time: Tue Apr 11 09:10:24 EDT 2017</div><div = class=3D"">2017-04-11 09:10:24,842-04 ERROR = [org.ovirt.engine.core.bll.sto<wbr = class=3D"">rage.pool.RefreshPoolSingleAsy<wbr class=3D"">ncOperation] = (org.ovirt.thread.pool-7-threa<wbr class=3D"">d-29) [2b66a42b] Could not = connect vds 'lago-basic-suite-master-host0<wbr class=3D"">' to pool = 'test-dc' - moving host to non-operational: null</div><div = class=3D"">2017-04-11 09:10:24,842-04 ERROR = [org.ovirt.engine.core.bll.sto<wbr = class=3D"">rage.pool.RefreshPoolSingleAsy<wbr class=3D"">ncOperation] = (org.ovirt.thread.pool-7-threa<wbr class=3D"">d-28) [2b66a42b] Could not = connect vds 'lago-basic-suite-master-host1<wbr class=3D"">' to pool = 'test-dc' - moving host to non-operational: null</div><div = class=3D"">2017-04-11 09:10:24,904-04 INFO &nbsp;[<a = href=3D"http://org.ovirt.engine.core.bll.se/" = class=3D"">org.ovirt.engine.core.bll.Se</a><wbr = class=3D"">tNonOperationalVdsCommand] (default task-29) [2bc83b1c] = Running command: SetNonOperationalVdsCommand internal: true. Entities = affected : &nbsp;ID: e1b611f2-67fb-4acb-8261-071331<wbr class=3D"">34e0e7 = Type: VDS</div><div class=3D"">2017-04-11 09:10:24,909-04 INFO = &nbsp;[org.ovirt.engine.core.vdsbro<wbr = class=3D"">ker.SetVdsStatusVDSCommand] (default task-29) [2bc83b1c] = START, SetVdsStatusVDSCommand(HostNam<wbr class=3D"">e =3D = lago-basic-suite-master-host0, SetVdsStatusVDSCommandParamete<wbr = class=3D"">rs:{runAsync=3D'true', hostId=3D'e1b611f2-67fb-4acb-826<wbr = class=3D"">1-07133134e0e7', status=3D'NonOperational', = nonOperationalReason=3D'STORAGE_<wbr class=3D"">DOMAIN_UNREACHABLE', = stopSpmFailureLogged=3D'false', maintenanceReason=3D'null'}), log id: = 5c45239</div><div class=3D"">2017-04-11 09:10:24,919-04 INFO = &nbsp;[org.ovirt.engine.core.vdsbro<wbr = class=3D"">ker.SetVdsStatusVDSCommand] (default task-29) [2bc83b1c] = FINISH, SetVdsStatusVDSCommand, log id: 5c45239</div><div = class=3D"">2017-04-11 09:10:24,922-04 DEBUG = [org.ovirt.engine.core.dal.dbb<wbr = class=3D"">roker.PostgresDbEngineDialect$<wbr = class=3D"">PostgresSimpleJdbcCall] (default task-29) [2bc83b1c] Compiled = stored procedure. Call string is [{call = getvmsrunningonvds(?)}]</div><div class=3D"">2017-04-11 09:10:24,922-04 = DEBUG [org.ovirt.engine.core.dal.dbb<wbr = class=3D"">roker.PostgresDbEngineDialect$<wbr = class=3D"">PostgresSimpleJdbcCall] (default task-29) [2bc83b1c] SqlCall = for procedure [GetVmsRunningOnVds] compiled</div><div = class=3D"">2017-04-11 09:10:24,985-04 WARN = &nbsp;[org.ovirt.engine.core.dal.db<wbr = class=3D"">broker.auditloghandling.AuditL<wbr class=3D"">ogDirector] = (default task-29) [2bc83b1c] EVENT_ID: = VDS_SET_NONOPERATIONAL_DOMAIN(<wbr class=3D"">522), Correlation ID: = 2bc83b1c, Job ID: d6455a45-48ad-42cd-9815-e98269<wbr class=3D"">44b9a0, = Call Stack: null, Custom Event ID: -1, Message: Host = lago-basic-suite-master-host0 cannot access the Storage Domain(s) iscsi = attached to the Data Center test-dc. Setting Host state to = Non-Operational.</div><div class=3D"">2017-04-11 09:10:24,985-04 DEBUG = [org.ovirt.engine.core.utils.t<wbr class=3D"">imer.FixedDelayJobListener] = (DefaultQuartzScheduler4) [] Rescheduling <a href=3D"http://DEFAULT.org" = class=3D"">DEFAULT.org</a>.ovirt.engine.core.<wbr = class=3D"">vdsbroker.VdsManager.onTimer#-<wbr = class=3D"">9223372036854775741 as there is no unfired trigger.</div><div = class=3D"">2017-04-11 09:10:25,036-04 INFO &nbsp;[<a = href=3D"http://org.ovirt.engine.core.bll.se/" = class=3D"">org.ovirt.engine.core.bll.Se</a><wbr = class=3D"">tNonOperationalVdsCommand] (default task-29) [2ffa5981] = Running command: SetNonOperationalVdsCommand internal: true. Entities = affected : &nbsp;ID: 87a5f691-ea5c-4692-a4ab-52dad0<wbr class=3D"">dafe34 = Type: VDS</div><div class=3D"">2017-04-11 09:10:25,039-04 INFO = &nbsp;[org.ovirt.engine.core.vdsbro<wbr = class=3D"">ker.SetVdsStatusVDSCommand] (default task-29) [2ffa5981] = START, SetVdsStatusVDSCommand(HostNam<wbr class=3D"">e =3D = lago-basic-suite-master-host1, SetVdsStatusVDSCommandParamete<wbr = class=3D"">rs:{runAsync=3D'true', hostId=3D'87a5f691-ea5c-4692-a4a<wbr = class=3D"">b-52dad0dafe34', status=3D'NonOperational', = nonOperationalReason=3D'STORAGE_<wbr class=3D"">DOMAIN_UNREACHABLE', = stopSpmFailureLogged=3D'false', maintenanceReason=3D'null'}), log id: = 706ffc3b</div></div><div class=3D""><br class=3D""></div><div = class=3D"">.....</div><div class=3D""><br class=3D""></div><div = class=3D""><div class=3D"">2017-04-11 09:10:25,366-04 INFO = &nbsp;[org.ovirt.engine.core.vdsbro<wbr = class=3D"">ker.vdsbroker.SpmStopVDSComman<wbr class=3D"">d] (default = task-29) [2ffa5981] FINISH, SpmStopVDSCommand, log id: = 62f82cfb</div><div class=3D"">2017-04-11 09:10:25,366-04 DEBUG = [org.ovirt.vdsm.jsonrpc.client<wbr = class=3D"">.reactors.stomp.impl.Message] (default task-29) [2ffa5981] = UNSUBSCRIBE</div><div class=3D"">id:4cd59c67-1b57-4456-9a16-52a<wbr = class=3D"">72ac6f689</div><div class=3D""><br class=3D""></div><div = class=3D"">\00</div><div class=3D"">2017-04-11 09:10:25,367-04 DEBUG = [org.ovirt.vdsm.jsonrpc.client<wbr = class=3D"">.reactors.stomp.StompCommonCli<wbr class=3D"">ent] (default = task-29) [2ffa5981] Message sent: UNSUBSCRIBE</div></div><div = class=3D""><br class=3D""></div><div class=3D""><br = class=3D""></div></div><div class=3D"gmail_extra"><br class=3D""><div = class=3D"gmail_quote"><div class=3D""><div = class=3D"gmail-m_5006786018485841575gmail-h5">On Tue, Apr 11, 2017 at = 4:49 PM, Sandro Bonazzola <span dir=3D"ltr" class=3D"">&lt;<a = href=3D"mailto:sbonazzo@redhat.com" target=3D"_blank" = class=3D&quot;&quot;&gt;sbonazzo(a)redhat.com&lt;/a&gt;&amp;gt;&lt;/span&gt; wrote:<br = class=3D""></div></div><blockquote class=3D"gmail_quote" = style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid = rgb(204,204,204);padding-left:1ex"><div class=3D""><div = class=3D"gmail-m_5006786018485841575gmail-h5"><div dir=3D"ltr" = class=3D"">Link to Job: <a = href=3D"http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_ma... 303" target=3D"_blank" class=3D"">http://jenkins.ovirt.org/job/t<wbr = class=3D"">est-repo_ovirt_experimental_ma<wbr class=3D"">ster/6303</a><pre= = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-console-output" = style=3D"box-sizing:border-box;white-space:pre-wrap;margin-top:0px;margin-= bottom:0px;color:rgb(51,51,51);font-size:14px"><span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D""><br = class=3D""></b></span></pre><pre = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-console-output" = style=3D"box-sizing:border-box;white-space:pre-wrap;margin-top:0px;margin-= bottom:0px;color:rgb(51,51,51);font-size:14px"><span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">OST is failing in the last 50 = builds</b></span></pre><pre = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-console-output" = style=3D"box-sizing:border-box;white-space:pre-wrap;margin-top:0px;margin-= bottom:0px;color:rgb(51,51,51);font-size:14px"><span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D""><br = class=3D""></b></span></pre><pre = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-console-output" = style=3D"box-sizing:border-box;white-space:pre-wrap;margin-top:0px;margin-= bottom:0px;color:rgb(51,51,51);font-size:14px"><br class=3D""></pre><pre = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-console-output" = style=3D"box-sizing:border-box;white-space:pre-wrap;margin-top:0px;margin-= bottom:0px;color:rgb(51,51,51);font-size:14px"><span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.090</b> = </span>[basic_suit_el7] <span = style=3D"box-sizing:border-box;color:rgb(0,205,205)" class=3D""> # = add_secondary_storage_domains: </span> <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.090</b> = </span>[basic_suit_el7] <span = style=3D"box-sizing:border-box;color:rgb(205,0,0)" class=3D"">Error = while running thread <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.090</b> = </span>[basic_suit_el7] Traceback (most recent call last): <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.090</b> = </span>[basic_suit_el7] File "/usr/lib/python2.7/site-packa<wbr = class=3D"">ges/lago/utils.py", line 57, in _ret_via_queue <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.090</b> = </span>[basic_suit_el7] queue.put({'return': func()}) <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.090</b> = </span>[basic_suit_el7] File "/home/jenkins/workspace/test-<wbr = class=3D"">repo_ovirt_experimental_master<wbr = class=3D"">/ovirt-system-tests/basic-suit<wbr = class=3D"">e-master/test-scenarios/002_<wbr class=3D"">bootstrap.py", = line 427, in add_nfs_storage_domain <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.090</b> = </span>[basic_suit_el7] add_generic_nfs_storage_domain<wbr = class=3D"">(prefix, SD_NFS_NAME, SD_NFS_HOST_NAME, SD_NFS_PATH, = nfs_version=3D'v4_2') <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.091</b> = </span>[basic_suit_el7] File "/home/jenkins/workspace/test-<wbr = class=3D"">repo_ovirt_experimental_master<wbr = class=3D"">/ovirt-system-tests/basic-suit<wbr = class=3D"">e-master/test-scenarios/002_<wbr class=3D"">bootstrap.py", = line 441, in add_generic_nfs_storage_domain <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.091</b> = </span>[basic_suit_el7] add_generic_nfs_storage_domain<wbr = class=3D"">_4(prefix, sd_nfs_name, nfs_host_name, mount_path, sd_format, = sd_type, nfs_version) <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.091</b> = </span>[basic_suit_el7] File "/home/jenkins/workspace/test-<wbr = class=3D"">repo_ovirt_experimental_master<wbr = class=3D"">/ovirt-system-tests/basic-suit<wbr = class=3D"">e-master/test-scenarios/002_<wbr class=3D"">bootstrap.py", = line 490, in add_generic_nfs_storage_domain<wbr class=3D"">_4 <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.091</b> = </span>[basic_suit_el7] host=3D_random_host_from_dc_4(ap<wbr = class=3D"">i, DC_NAME), <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.091</b> = </span>[basic_suit_el7] File "/home/jenkins/workspace/test-<wbr = class=3D"">repo_ovirt_experimental_master<wbr = class=3D"">/ovirt-system-tests/basic-suit<wbr = class=3D"">e-master/test-scenarios/002_<wbr class=3D"">bootstrap.py", = line 106, in _random_host_from_dc_4 <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.091</b> = </span>[basic_suit_el7] return random.choice(_hosts_in_dc_4(a<wbr = class=3D"">pi, dc_name)) <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.091</b> = </span>[basic_suit_el7] File "/home/jenkins/workspace/test-<wbr = class=3D"">repo_ovirt_experimental_master<wbr = class=3D"">/ovirt-system-tests/basic-suit<wbr = class=3D"">e-master/test-scenarios/002_<wbr class=3D"">bootstrap.py", = line 100, in _hosts_in_dc_4 <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.091</b> = </span>[basic_suit_el7] raise RuntimeError('Could not find hosts = that are up in DC %s' % dc_name) <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.092</b> = </span>[basic_suit_el7] RuntimeError: Could not find hosts that are up = in DC test-dc</span> <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.092</b> = </span>[basic_suit_el7] <span = style=3D"box-sizing:border-box;color:rgb(205,0,0)" class=3D"">Error = while running thread <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.092</b> = </span>[basic_suit_el7] Traceback (most recent call last): <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.092</b> = </span>[basic_suit_el7] File "/usr/lib/python2.7/site-packa<wbr = class=3D"">ges/lago/utils.py", line 57, in _ret_via_queue <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.092</b> = </span>[basic_suit_el7] queue.put({'return': func()}) <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.092</b> = </span>[basic_suit_el7] File "/home/jenkins/workspace/test-<wbr = class=3D"">repo_ovirt_experimental_master<wbr = class=3D"">/ovirt-system-tests/basic-suit<wbr = class=3D"">e-master/test-scenarios/002_<wbr class=3D"">bootstrap.py", = line 583, in add_iso_storage_domain <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.092</b> = </span>[basic_suit_el7] add_generic_nfs_storage_domain<wbr = class=3D"">(prefix, SD_ISO_NAME, SD_ISO_HOST_NAME, SD_ISO_PATH, = sd_format=3D'v1', sd_type=3D'iso', nfs_version=3D'v3') <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.092</b> = </span>[basic_suit_el7] File "/home/jenkins/workspace/test-<wbr = class=3D"">repo_ovirt_experimental_master<wbr = class=3D"">/ovirt-system-tests/basic-suit<wbr = class=3D"">e-master/test-scenarios/002_<wbr class=3D"">bootstrap.py", = line 441, in add_generic_nfs_storage_domain <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.092</b> = </span>[basic_suit_el7] add_generic_nfs_storage_domain<wbr = class=3D"">_4(prefix, sd_nfs_name, nfs_host_name, mount_path, sd_format, = sd_type, nfs_version) <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.093</b> = </span>[basic_suit_el7] File "/home/jenkins/workspace/test-<wbr = class=3D"">repo_ovirt_experimental_master<wbr = class=3D"">/ovirt-system-tests/basic-suit<wbr = class=3D"">e-master/test-scenarios/002_<wbr class=3D"">bootstrap.py", = line 490, in add_generic_nfs_storage_domain<wbr class=3D"">_4 <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.093</b> = </span>[basic_suit_el7] host=3D_random_host_from_dc_4(ap<wbr = class=3D"">i, DC_NAME), <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.093</b> = </span>[basic_suit_el7] File "/home/jenkins/workspace/test-<wbr = class=3D"">repo_ovirt_experimental_master<wbr = class=3D"">/ovirt-system-tests/basic-suit<wbr = class=3D"">e-master/test-scenarios/002_<wbr class=3D"">bootstrap.py", = line 106, in _random_host_from_dc_4 <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.093</b> = </span>[basic_suit_el7] return random.choice(_hosts_in_dc_4(a<wbr = class=3D"">pi, dc_name)) <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.093</b> = </span>[basic_suit_el7] File "/home/jenkins/workspace/test-<wbr = class=3D"">repo_ovirt_experimental_master<wbr = class=3D"">/ovirt-system-tests/basic-suit<wbr = class=3D"">e-master/test-scenarios/002_<wbr class=3D"">bootstrap.py", = line 100, in _hosts_in_dc_4 <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.093</b> = </span>[basic_suit_el7] raise RuntimeError('Could not find hosts = that are up in DC %s' % dc_name) <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.093</b> = </span>[basic_suit_el7] RuntimeError: Could not find hosts that are up = in DC test-dc</span> <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.094</b> = </span>[basic_suit_el7] <span = style=3D"box-sizing:border-box;color:rgb(205,0,0)" class=3D"">Error = while running thread <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.094</b> = </span>[basic_suit_el7] Traceback (most recent call last): <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.094</b> = </span>[basic_suit_el7] File "/usr/lib/python2.7/site-packa<wbr = class=3D"">ges/lago/utils.py", line 57, in _ret_via_queue <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.094</b> = </span>[basic_suit_el7] queue.put({'return': func()}) <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.094</b> = </span>[basic_suit_el7] File "/home/jenkins/workspace/test-<wbr = class=3D"">repo_ovirt_experimental_master<wbr = class=3D"">/ovirt-system-tests/basic-suit<wbr = class=3D"">e-master/test-scenarios/002_<wbr class=3D"">bootstrap.py", = line 587, in add_templates_storage_domain <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.094</b> = </span>[basic_suit_el7] add_generic_nfs_storage_domain<wbr = class=3D"">(prefix, SD_TEMPLATES_NAME, SD_TEMPLATES_HOST_NAME, = SD_TEMPLATES_PATH, sd_format=3D'v1', sd_type=3D'export', = nfs_version=3D'v4_1') <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.094</b> = </span>[basic_suit_el7] File "/home/jenkins/workspace/test-<wbr = class=3D"">repo_ovirt_experimental_master<wbr = class=3D"">/ovirt-system-tests/basic-suit<wbr = class=3D"">e-master/test-scenarios/002_<wbr class=3D"">bootstrap.py", = line 441, in add_generic_nfs_storage_domain <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.094</b> = </span>[basic_suit_el7] add_generic_nfs_storage_domain<wbr = class=3D"">_4(prefix, sd_nfs_name, nfs_host_name, mount_path, sd_format, = sd_type, nfs_version) <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.094</b> = </span>[basic_suit_el7] File "/home/jenkins/workspace/test-<wbr = class=3D"">repo_ovirt_experimental_master<wbr = class=3D"">/ovirt-system-tests/basic-suit<wbr = class=3D"">e-master/test-scenarios/002_<wbr class=3D"">bootstrap.py", = line 490, in add_generic_nfs_storage_domain<wbr class=3D"">_4 <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.095</b> = </span>[basic_suit_el7] host=3D_random_host_from_dc_4(ap<wbr = class=3D"">i, DC_NAME), <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.095</b> = </span>[basic_suit_el7] File "/home/jenkins/workspace/test-<wbr = class=3D"">repo_ovirt_experimental_master<wbr = class=3D"">/ovirt-system-tests/basic-suit<wbr = class=3D"">e-master/test-scenarios/002_<wbr class=3D"">bootstrap.py", = line 106, in _random_host_from_dc_4 <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.095</b> = </span>[basic_suit_el7] return random.choice(_hosts_in_dc_4(a<wbr = class=3D"">pi, dc_name)) <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.095</b> = </span>[basic_suit_el7] File "/home/jenkins/workspace/test-<wbr = class=3D"">repo_ovirt_experimental_master<wbr = class=3D"">/ovirt-system-tests/basic-suit<wbr = class=3D"">e-master/test-scenarios/002_<wbr class=3D"">bootstrap.py", = line 100, in _hosts_in_dc_4 <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.095</b> = </span>[basic_suit_el7] raise RuntimeError('Could not find hosts = that are up in DC %s' % dc_name) <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.095</b> = </span>[basic_suit_el7] RuntimeError: Could not find hosts that are up = in DC test-dc</span> <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.095</b> = </span>[basic_suit_el7] <span = style=3D"box-sizing:border-box;color:rgb(205,0,0)" class=3D"">Error = while running thread <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.095</b> = </span>[basic_suit_el7] Traceback (most recent call last): <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.096</b> = </span>[basic_suit_el7] File "/usr/lib/python2.7/site-packa<wbr = class=3D"">ges/lago/utils.py", line 57, in _ret_via_queue <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.096</b> = </span>[basic_suit_el7] queue.put({'return': func()}) <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.096</b> = </span>[basic_suit_el7] File "/home/jenkins/workspace/test-<wbr = class=3D"">repo_ovirt_experimental_master<wbr = class=3D"">/ovirt-system-tests/basic-suit<wbr = class=3D"">e-master/test-scenarios/002_<wbr class=3D"">bootstrap.py", = line 436, in add_second_nfs_storage_domain <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.096</b> = </span>[basic_suit_el7] SD_NFS_HOST_NAME, SD_SECOND_NFS_PATH) <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.096</b> = </span>[basic_suit_el7] File "/home/jenkins/workspace/test-<wbr = class=3D"">repo_ovirt_experimental_master<wbr = class=3D"">/ovirt-system-tests/basic-suit<wbr = class=3D"">e-master/test-scenarios/002_<wbr class=3D"">bootstrap.py", = line 441, in add_generic_nfs_storage_domain <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.096</b> = </span>[basic_suit_el7] add_generic_nfs_storage_domain<wbr = class=3D"">_4(prefix, sd_nfs_name, nfs_host_name, mount_path, sd_format, = sd_type, nfs_version) <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.096</b> = </span>[basic_suit_el7] File "/home/jenkins/workspace/test-<wbr = class=3D"">repo_ovirt_experimental_master<wbr = class=3D"">/ovirt-system-tests/basic-suit<wbr = class=3D"">e-master/test-scenarios/002_<wbr class=3D"">bootstrap.py", = line 490, in add_generic_nfs_storage_domain<wbr class=3D"">_4 <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.096</b> = </span>[basic_suit_el7] host=3D_random_host_from_dc_4(ap<wbr = class=3D"">i, DC_NAME), <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.096</b> = </span>[basic_suit_el7] File "/home/jenkins/workspace/test-<wbr = class=3D"">repo_ovirt_experimental_master<wbr = class=3D"">/ovirt-system-tests/basic-suit<wbr = class=3D"">e-master/test-scenarios/002_<wbr class=3D"">bootstrap.py", = line 106, in _random_host_from_dc_4 <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.097</b> = </span>[basic_suit_el7] return random.choice(_hosts_in_dc_4(a<wbr = class=3D"">pi, dc_name)) <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.097</b> = </span>[basic_suit_el7] File "/home/jenkins/workspace/test-<wbr = class=3D"">repo_ovirt_experimental_master<wbr = class=3D"">/ovirt-system-tests/basic-suit<wbr = class=3D"">e-master/test-scenarios/002_<wbr class=3D"">bootstrap.py", = line 100, in _hosts_in_dc_4 <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.097</b> = </span>[basic_suit_el7] raise RuntimeError('Could not find hosts = that are up in DC %s' % dc_name) <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.097</b> = </span>[basic_suit_el7] RuntimeError: Could not find hosts that are up = in DC test-dc</span> <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.097</b> = </span>[basic_suit_el7] <span = style=3D"box-sizing:border-box;color:rgb(205,0,0)" class=3D"">Error = while running thread <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.097</b> = </span>[basic_suit_el7] Traceback (most recent call last): <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.097</b> = </span>[basic_suit_el7] File "/usr/lib/python2.7/site-packa<wbr = class=3D"">ges/lago/utils.py", line 57, in _ret_via_queue <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.098</b> = </span>[basic_suit_el7] queue.put({'return': func()}) <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.098</b> = </span>[basic_suit_el7] File "/home/jenkins/workspace/test-<wbr = class=3D"">repo_ovirt_experimental_master<wbr = class=3D"">/ovirt-system-tests/basic-suit<wbr = class=3D"">e-master/test-scenarios/002_<wbr class=3D"">bootstrap.py", = line 803, in import_template_from_glance <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.098</b> = </span>[basic_suit_el7] generic_import_from_glance(api<wbr = class=3D"">, image_name=3DCIRROS_IMAGE_NAME, = image_ext=3D'_glance_template', as_template=3DTrue) <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.098</b> = </span>[basic_suit_el7] File "/home/jenkins/workspace/test-<wbr = class=3D"">repo_ovirt_experimental_master<wbr = class=3D"">/ovirt-system-tests/basic-suit<wbr = class=3D"">e-master/test-scenarios/002_<wbr class=3D"">bootstrap.py", = line 637, in generic_import_from_glance <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.098</b> = </span>[basic_suit_el7] target_image.import_image(impo<wbr = class=3D"">rt_action) <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.098</b> = </span>[basic_suit_el7] File "/usr/lib/python2.7/site-packa<wbr = class=3D"">ges/ovirtsdk/infrastructure/br<wbr class=3D"">okers.py", line = 26017, in import_image <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.098</b> = </span>[basic_suit_el7] headers=3D{"Correlation-Id":corr<wbr = class=3D"">elation_id} <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.098</b> = </span>[basic_suit_el7] File "/usr/lib/python2.7/site-packa<wbr = class=3D"">ges/ovirtsdk/infrastructure/pr<wbr class=3D"">oxy.py", line = 122, in request <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.098</b> = </span>[basic_suit_el7] persistent_auth=3Dself.__persist<wbr = class=3D"">ent_auth <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.099</b> = </span>[basic_suit_el7] File "/usr/lib/python2.7/site-packa<wbr = class=3D"">ges/ovirtsdk/infrastructure/co<wbr = class=3D"">nnectionspool.py", line 79, in do_request <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.099</b> = </span>[basic_suit_el7] persistent_auth) <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.099</b> = </span>[basic_suit_el7] File "/usr/lib/python2.7/site-packa<wbr = class=3D"">ges/ovirtsdk/infrastructure/co<wbr = class=3D"">nnectionspool.py", line 162, in __do_request <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.099</b> = </span>[basic_suit_el7] raise errors.RequestError(response_c<wbr = class=3D"">ode, response_reason, response_body) <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.099</b> = </span>[basic_suit_el7] RequestError:=20 <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.099</b> = </span>[basic_suit_el7] status: 400 <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.099</b> = </span>[basic_suit_el7] reason: Bad Request <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.099</b> = </span>[basic_suit_el7] detail: Cannot import Virtual Disk: Storage = Domain cannot be accessed. <span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail-timestamp" style=3D"box-sizing:border-box"><b = style=3D"box-sizing:border-box" class=3D"">00:18:22.099</b> = </span>[basic_suit_el7] -Please check that at least one Host is = operational and Data Center state is up.</span></pre><span = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735HOEnZb"><fo= nt color=3D"#888888" class=3D""><div class=3D""><br class=3D""></div>-- = <br class=3D""><div = class=3D"gmail-m_5006786018485841575gmail-m_3046267183852181735m_525458228= 0271388056gmail_signature"><div dir=3D"ltr" class=3D""><div = class=3D""><div dir=3D"ltr" class=3D""><div dir=3D"ltr" class=3D""><div = dir=3D"ltr" class=3D""><div dir=3D"ltr" class=3D""><div = style=3D"font-family: overpass, sans-serif; font-weight: bold; margin: = 0px; padding: 0px; font-size: 14px; text-transform: uppercase;" = class=3D""><span class=3D"">SANDRO</span>&nbsp;<span = class=3D"">BONAZZOLA</span></div><p style=3D"font-family: overpass, = sans-serif; font-size: 10px; margin: 0px 0px 4px; text-transform: = uppercase;" class=3D""><span class=3D"">ASSOCIATE MANAGER, SOFTWARE = ENGINEERING, EMEA ENG VIRTUALIZATION R&amp;D</span></p><div = style=3D"font-family: overpass, sans-serif; margin: 0px; font-size: = 10px; color: rgb(153, 153, 153);" class=3D""><a = href=3D"https://www.redhat.com/" style=3D"color:rgb(0,136,206);margin:0px"= target=3D"_blank" class=3D"">Red Hat&nbsp;<span = class=3D"">EMEA</span></a></div><table style=3D"font-family: overpass, = sans-serif; font-size: inherit;" border=3D"0" class=3D""><tbody = class=3D""><tr class=3D""><td width=3D"100px" class=3D""><a = href=3D"https://red.ht/sig" target=3D"_blank" class=3D""><img = src=3D"https://www.redhat.com/profiles/rh/themes/redhatdotcom/img/lo... -hat-black.png" width=3D"90" height=3D"auto" class=3D""></a></td><td = style=3D"font-size:10px" class=3D""><div class=3D""><a = href=3D"https://redhat.com/trusted" = style=3D"color:rgb(204,0,0);font-weight:bold" target=3D"_blank" = class=3D"">TRIED. TESTED. = TRUSTED.</a></div></td></tr></tbody></table></div></div></div></div></div>= </div></div> </font></span></div> <br class=3D""></div></div>______________________________<wbr = class=3D"">_________________<br class=3D""> Devel mailing list<br class=3D""> <a href=3D"mailto:Devel@ovirt.org" target=3D"_blank" = class=3D&quot;&quot;&gt;Devel(a)ovirt.org&lt;/a&gt;&lt;br class=3D""> <a href=3D"http://lists.ovirt.org/mailman/listinfo/devel" = rel=3D"noreferrer" target=3D"_blank" = class=3D"">http://lists.ovirt.org/mailman<wbr = class=3D"">/listinfo/devel</a><br class=3D""></blockquote></div><br = class=3D""></div> <br class=3D"">______________________________<wbr = class=3D"">_________________<br class=3D""> Devel mailing list<br class=3D""> <a href=3D"mailto:Devel@ovirt.org" target=3D"_blank" = class=3D&quot;&quot;&gt;Devel(a)ovirt.org&lt;/a&gt;&lt;br class=3D""> <a href=3D"http://lists.ovirt.org/mailman/listinfo/devel" = rel=3D"noreferrer" target=3D"_blank" = class=3D"">http://lists.ovirt.org/mailman<wbr = class=3D"">/listinfo/devel</a><br class=3D""></blockquote></div><br = class=3D""><br clear=3D"all" class=3D""><br class=3D""></div></div><span = class=3D"gmail-HOEnZb"><font color=3D"#888888" class=3D"">-- <br = class=3D""><div class=3D"gmail-m_5006786018485841575gmail_signature">Barak= Korren<br class=3D""><a href=3D"mailto:bkorren@redhat.com" = target=3D"_blank" class=3D&quot;&quot;&gt;bkorren(a)redhat.com&lt;/a&gt;&lt;br class=3D"">RHCE, = RHCi, RHV-DevOps Team<br class=3D""><a = href=3D"https://ifireball.wordpress.com/" target=3D"_blank" = class=3D"">https://ifireball.wordpress.<wbr class=3D"">com/</a></div> </font></span></div></div></div></div> <br class=3D"">______________________________<wbr = class=3D"">_________________<br class=3D""> Devel mailing list<br class=3D""> <a href=3D"mailto:Devel@ovirt.org" class=3D&quot;&quot;&gt;Devel(a)ovirt.org&lt;/a&gt;&lt;br = class=3D""> <a href=3D"http://lists.ovirt.org/mailman/listinfo/devel" = rel=3D"noreferrer" target=3D"_blank" = class=3D"">http://lists.ovirt.org/<wbr = class=3D"">mailman/listinfo/devel</a><br class=3D""></blockquote></div><br= class=3D""></div></div> _______________________________________________<br class=3D"">Devel = mailing list<br class=3D""><a href=3D"mailto:Devel@ovirt.org" = class=3D&quot;&quot;&gt;Devel(a)ovirt.org&lt;/a&gt;&lt;br = class=3D"">http://lists.ovirt.org/mailman/listinfo/devel<... --Apple-Mail=_A73F9645-3BD9-4104-A39D-D19AEC539032--