The failure is on migrate_vm

logs:
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/13327/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-master/post-006_migrations.py/


engine

2019-03-08 12:30:03,062-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [] EVENT_ID: VDS_
BROKER_COMMAND_FAILURE(10,802), VDSM lago-basic-suite-master-host-0 command Get Host Capabilities failed: Recovering from crash or Initializing
2019-03-08 12:30:03,063-05 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [] Unable to RefreshCapabili
ties: VDSRecoveringException: Recovering from crash or Initializing
2019-03-08 12:30:03,063-05 DEBUG [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [] Exception: org.ovirt.engi
ne.core.vdsbroker.vdsbroker.VDSRecoveringException: Recovering from crash or Initializing
        at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:40) [vdsbroker.jar:]
        at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand$GetCapabilitiesVDSCommandCallback.onFailure(GetCapabilitiesAsyncVDSCommand.java:58) [vdsb
roker.jar:]
        at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.lambda$processResponse$0(JsonRpcClient.java:186) [vdsm-jsonrpc-java-client.jar:]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_201]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_201]
        at org.glassfish.enterprise.concurrent.internal.ManagedFutureTask.run(ManagedFutureTask.java:141) [javax.enterprise.concurrent-1.0.jar:]
        at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$101(ManagedScheduledThreadPoolExecutor.java:383)
 [javax.enterprise.concurrent-1.0.jar:]
        at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:532) [javax
.enterprise.concurrent-1.0.jar:]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_201]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_201]
        at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_201]
        at org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250) [javax.enterprise.concurrent-1.0.jar:]
        at org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManagedThreadFactory.java:78)

vdsm:

2019-03-08 12:31:46,401-0500 ERROR (jsonrpc/1) [api] FINISH create error=Error creating the requested VM (api:131)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 124, in method
    ret = func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 231, in create
    "Host runs in FIPS mode but SASL authentication for VNC "
CannotCreateVM: Error creating the requested VM


On Fri, Mar 8, 2019 at 6:59 PM Dafna Ron <dron@redhat.com> wrote:
the patch itself failed CQ: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/13327/



On Fri, Mar 8, 2019 at 11:43 AM Dafna Ron <dron@redhat.com> wrote:
thanks.
I'll monitor 

On Fri, Mar 8, 2019 at 11:39 AM Tomasz Baranski <tbaransk@redhat.com> wrote:
Fix: https://gerrit.ovirt.org/98371

T.

On 19/03/08 10:21, Dafna Ron wrote:
> Thanks Tomasz :)
>
> On Fri, Mar 8, 2019 at 10:19 AM Tomasz Baranski <tbaransk@redhat.com> wrote:
>
> > Hmm...
> > Looking into it...
> >
> > Tomo
> >
> > On 19/03/08 10:05, Dafna Ron wrote:
> > > Hi,
> > >
> > > We are failing vdsm project on master branch with test
> > > 002_bootstrap.verify_add_hosts
> > >
> > > CQ identified this patch as the root cause:
> > >
> > > https://gerrit.ovirt.org/#/c/97381/ - virt: In FIPS mode, use SASL auth
> > > instead of qemu passwords
> > >
> > > Can you please take a look?
> > >
> > > build logs:
> > >
> > https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/13317/artifact/basic-suite.el7.x86_64/test_logs/basic-suite-master/post-002_bootstrap.py/
> > >
> > > <error>
> > >
> > > host:
> > >
> > > 2019-03-07 17:37:38,839-0500 ERROR (MainThread) [vds] Exception raised
> > > (vdsmd:159)
> > > Traceback (most recent call last):
> > >   File "/usr/lib/python2.7/site-packages/vdsm/vdsmd.py", line 157, in run
> > >     serve_clients(log)
> > >   File "/usr/lib/python2.7/site-packages/vdsm/vdsmd.py", line 103, in
> > > serve_clients
> > >     from vdsm.clientIF import clientIF  # must import after config is
> > read
> > >   File "/usr/lib/python2.7/site-packages/vdsm/clientIF.py", line 64, in
> > > <module>
> > >     from vdsm.virt import vm
> > >   File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 81, in
> > > <module>
> > >     from vdsm.virt import saslpasswd2
> > > ImportError: cannot import name saslpasswd2
> > > 2019-03-07 17:37:38,841-0500 INFO  (MainThread) [vds] Stopping threads
> > > (vdsmd:161)
> > > 2019-03-07 17:37:38,841-0500 INFO  (MainThread) [vds] Stopping
> > > <MonitorObserver(mpathlistener, started daemon 139815311881984)>
> > (vdsmd:164)
> > >
> > > engine:
> > >
> > > 2019-03-07 17:37:47,780-05 ERROR
> > >
> > [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand]
> > > (EE-ManagedThreadFactory-engineScheduled-Thread-28) [] Command 'G
> > > etCapabilitiesAsyncVDSCommand(HostName = lago-basic-suite-master-host-1,
> > >
> > VdsIdAndVdsVDSCommandParametersBase:{hostId='5d685bce-ada7-4730-975c-45d30afc0a8b',
> > > vds='Host[lago-b
> > > asic-suite-master-host-1,5d685bce-ada7-4730-975c-45d30afc0a8b]'})'
> > > execution failed: java.net.ConnectException: Connection refused
> > > 2019-03-07 17:37:47,780-05 DEBUG
> > >
> > [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand]
> > > (EE-ManagedThreadFactory-engineScheduled-Thread-28) [] Exception:
> > >  org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException:
> > > java.net.ConnectException: Connection refused
> > >         at
> > >
> > org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:172)
> > > [vdsbroker.jar:]
> > >         at
> > >
> > org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVdsCommandWithNetworkEvent(VdsBrokerCommand.java:135)
> > > [vdsbroker.jar:]
> > >         at
> > >
> > org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:111)
> > > [vdsbroker.jar:]
> > >         at
> > >
> > org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:65)
> > > [vdsbroker.jar:]
> > >         at
> > > org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31)
> > > [dal.jar:]
> > >         at
> > >
> > org.ovirt.engine.core.vdsbroker.vdsbroker.DefaultVdsCommandExecutor.execute(DefaultVdsCommandExecutor.java:14)
> > > [vdsbroker.jar:]
> > >         at
> > >
> > org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:396)
> > > [vdsbroker.jar:]
> > >         at
> > >
> > org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand$$super(Unknown
> > > Source) [vdsbroker.jar:]
> > >         at sun.reflect.GeneratedMethodAccessor252.invoke(Unknown Source)
> > > [:1.8.0_201]
> > >         at
> > >
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > [rt.jar:1.8.0_201]
> > >         at java.lang.reflect.Method.invoke(Method.java:498)
> > > [rt.jar:1.8.0_201]
> > >         at
> > >
> > org.jboss.weld.interceptor.proxy.TerminalAroundInvokeInvocationContext.proceedInternal(TerminalAroundInvokeInvocationContext.java:51)
> > > [weld-core-impl-3.0.5.Final.
> > > jar:3.0.5.Final]
> > >         at
> > >
> > org.jboss.weld.interceptor.proxy.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:78)
> > > [weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
> > >         at
> > >
> > org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12)
> > > [common.jar:]
> > >         at sun.reflect.GeneratedMethodAccessor61.invoke(Unknown Source)
> > > [:1.8.0_201]
> > >         at
> > >
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > [rt.jar:1.8.0_201]
> > >         at java.lang.reflect.Method.invoke(Method.java:498)
> > > [rt.jar:1.8.0_201]
> > >         at
> > >
> > org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73)
> > > [weld-core-impl-3.0.5.Final.jar:3
> > > .0.5.Final]
> > >         at
> > >
> > org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeAroundInvoke(InterceptorMethodHandler.java:84)
> > > [weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
> > >         at
> > >
> > org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeInterception(InterceptorMethodHandler.java:72)
> > > [weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
> > >         at
> > >
> > org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.invoke(InterceptorMethodHandler.java:56)
> > > [weld-core-impl-3.0.5.Final.jar:3.0.5.Final]
> > >         at
> > >
> > org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:79)
> > > [weld-core-impl-3.0.
> > > 5.Final.jar:3.0.5.Final]
> > >         at
> > >
> > org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:68)
> > > [weld-core-impl-3.0.
> > > 5.Final.jar:3.0.5.Final]
> > >         at
> > >
> > org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand(Unknown
> > > Source) [vdsbroker.jar:]
> > >         at
> > >
> > org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:779)
> > > [vdsbroker.jar:]
> > >         at
> > >
> > org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring.refreshCapabilities(HostMonitoring.java:189)
> > > [vdsbroker.jar:]
> > >
> > > </error>
> >