2020-06-01 16:44:23,209-05 INFO [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-65) [aa50e9bf-ab1c-42da-9e8c-760262a33b70] Lock Acquired to object 'EngineLock:{exclusiveLocks='[9b58ebc6-fe0d-4b9c-ad0a-a8f3999ada03=HOST_STORAGE_DEVICES]', sharedLocks=''}' 2020-06-01 16:44:23,225-05 INFO [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-65) [aa50e9bf-ab1c-42da-9e8c-760262a33b70] Running command: CreateBrickCommand internal: false. Entities affected : ID: a45f92bf-a0ea-4398-aef8-2e0d73096a9d Type: VDSAction group MANIPULATE_HOST with role type ADMIN 2020-06-01 16:44:23,625-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (default task-68) [b4dbd2c2-a926-448f-bacc-dcd1ddb3070b] START, GlusterServersListVDSCommand(HostName = hypervisor1.ovirt2.packet.mx, VdsIdVDSCommandParametersBase:{hostId='a45f92bf-a0ea-4398-aef8-2e0d73096a9d'}), log id: 51cde49 2020-06-01 16:44:23,913-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (default task-68) [b4dbd2c2-a926-448f-bacc-dcd1ddb3070b] FINISH, GlusterServersListVDSCommand, return: [172.25.209.11/24:CONNECTED, hypervisor3.ovirt2.packet.mx:CONNECTED, hypervisor2.ovirt2.packet.mx:CONNECTED], log id: 51cde49 2020-06-01 16:44:25,599-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-70) [] START, GlusterServersListVDSCommand(HostName = hypervisor1.ovirt2.packet.mx, VdsIdVDSCommandParametersBase:{hostId='a45f92bf-a0ea-4398-aef8-2e0d73096a9d'}), log id: 20779981 2020-06-01 16:44:25,886-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-70) [] FINISH, GlusterServersListVDSCommand, return: [172.25.209.11/24:CONNECTED, hypervisor3.ovirt2.packet.mx:CONNECTED, hypervisor2.ovirt2.packet.mx:CONNECTED], log id: 20779981 2020-06-01 16:44:25,890-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-70) [] START, GlusterVolumesListVDSCommand(HostName = hypervisor1.ovirt2.packet.mx, GlusterVolumesListVDSParameters:{hostId='a45f92bf-a0ea-4398-aef8-2e0d73096a9d'}), log id: 3c53c735 2020-06-01 16:44:26,042-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-70) [] FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 3c53c735 2020-06-01 16:44:28,820-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (default task-68) [6f22b600-c3c5-4d29-a973-a3fc77eaba79] START, GlusterServersListVDSCommand(HostName = hypervisor1.ovirt2.packet.mx, VdsIdVDSCommandParametersBase:{hostId='a45f92bf-a0ea-4398-aef8-2e0d73096a9d'}), log id: 398d9c3d 2020-06-01 16:44:29,102-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (default task-68) [6f22b600-c3c5-4d29-a973-a3fc77eaba79] FINISH, GlusterServersListVDSCommand, return: [172.25.209.11/24:CONNECTED, hypervisor3.ovirt2.packet.mx:CONNECTED, hypervisor2.ovirt2.packet.mx:CONNECTED], log id: 398d9c3d 2020-06-01 16:44:32,283-05 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-65) [aa50e9bf-ab1c-42da-9e8c-760262a33b70] EVENT_ID: ANSIBLE_RUNNER_EVENT_NOTIFICATION(559), Create Brick. Clean up filesystem signature. 2020-06-01 16:44:32,292-05 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-65) [aa50e9bf-ab1c-42da-9e8c-760262a33b70] EVENT_ID: ANSIBLE_RUNNER_EVENT_NOTIFICATION(559), Create Brick. Set PV data alignment for JBOD. 2020-06-01 16:44:33,851-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (default task-68) [7b8c43ec-337a-4891-832f-6c1fce553c8a] START, GlusterServersListVDSCommand(HostName = hypervisor1.ovirt2.packet.mx, VdsIdVDSCommandParametersBase:{hostId='a45f92bf-a0ea-4398-aef8-2e0d73096a9d'}), log id: 37725e8c 2020-06-01 16:44:34,039-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (default task-70) [8f5ff0d0-7f48-4a4c-83fd-1e6a9d19f668] START, GlusterServersListVDSCommand(HostName = hypervisor1.ovirt2.packet.mx, VdsIdVDSCommandParametersBase:{hostId='a45f92bf-a0ea-4398-aef8-2e0d73096a9d'}), log id: 42bd6565 2020-06-01 16:44:34,136-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (default task-68) [7b8c43ec-337a-4891-832f-6c1fce553c8a] FINISH, GlusterServersListVDSCommand, return: [172.25.209.11/24:CONNECTED, hypervisor3.ovirt2.packet.mx:CONNECTED, hypervisor2.ovirt2.packet.mx:CONNECTED], log id: 37725e8c 2020-06-01 16:44:34,322-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (default task-70) [8f5ff0d0-7f48-4a4c-83fd-1e6a9d19f668] FINISH, GlusterServersListVDSCommand, return: [172.25.209.11/24:CONNECTED, hypervisor3.ovirt2.packet.mx:CONNECTED, hypervisor2.ovirt2.packet.mx:CONNECTED], log id: 42bd6565 2020-06-01 16:44:38,318-05 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-65) [aa50e9bf-ab1c-42da-9e8c-760262a33b70] EVENT_ID: ANSIBLE_RUNNER_EVENT_NOTIFICATION(559), Create Brick. Create volume groups. 2020-06-01 16:44:38,334-05 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-65) [aa50e9bf-ab1c-42da-9e8c-760262a33b70] EVENT_ID: ANSIBLE_RUNNER_EVENT_NOTIFICATION(559), Create Brick. Set chunksize for JBOD. 2020-06-01 16:44:39,012-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (default task-70) [16e38247-e4e6-45ff-a753-c9262a301a42] START, GlusterServersListVDSCommand(HostName = hypervisor1.ovirt2.packet.mx, VdsIdVDSCommandParametersBase:{hostId='a45f92bf-a0ea-4398-aef8-2e0d73096a9d'}), log id: 15e32acf 2020-06-01 16:44:39,303-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (default task-70) [16e38247-e4e6-45ff-a753-c9262a301a42] FINISH, GlusterServersListVDSCommand, return: [172.25.209.11/24:CONNECTED, hypervisor3.ovirt2.packet.mx:CONNECTED, hypervisor2.ovirt2.packet.mx:CONNECTED], log id: 15e32acf 2020-06-01 16:44:39,312-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (default task-70) [16e38247-e4e6-45ff-a753-c9262a301a42] START, GlusterServersListVDSCommand(HostName = hypervisor1.ovirt2.packet.mx, VdsIdVDSCommandParametersBase:{hostId='a45f92bf-a0ea-4398-aef8-2e0d73096a9d'}), log id: 548f11a3 2020-06-01 16:44:39,595-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (default task-70) [16e38247-e4e6-45ff-a753-c9262a301a42] FINISH, GlusterServersListVDSCommand, return: [172.25.209.11/24:CONNECTED, hypervisor3.ovirt2.packet.mx:CONNECTED, hypervisor2.ovirt2.packet.mx:CONNECTED], log id: 548f11a3 2020-06-01 16:44:41,049-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-17) [] START, GlusterServersListVDSCommand(HostName = hypervisor1.ovirt2.packet.mx, VdsIdVDSCommandParametersBase:{hostId='a45f92bf-a0ea-4398-aef8-2e0d73096a9d'}), log id: 55811c1a 2020-06-01 16:44:41,347-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-17) [] FINISH, GlusterServersListVDSCommand, return: [172.25.209.11/24:CONNECTED, hypervisor3.ovirt2.packet.mx:CONNECTED, hypervisor2.ovirt2.packet.mx:CONNECTED], log id: 55811c1a 2020-06-01 16:44:41,350-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-17) [] START, GlusterVolumesListVDSCommand(HostName = hypervisor1.ovirt2.packet.mx, GlusterVolumesListVDSParameters:{hostId='a45f92bf-a0ea-4398-aef8-2e0d73096a9d'}), log id: 14ec16d5 2020-06-01 16:44:41,368-05 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-65) [aa50e9bf-ab1c-42da-9e8c-760262a33b70] EVENT_ID: ANSIBLE_RUNNER_EVENT_NOTIFICATION(559), Create Brick. Create a LV thinpool. 2020-06-01 16:44:41,495-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-17) [] FINISH, GlusterVolumesListVDSCommand, return: {}, log id: 14ec16d5 2020-06-01 16:44:44,008-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (default task-68) [728a6c91-f499-419a-a92e-82b9943e233f] START, GlusterServersListVDSCommand(HostName = hypervisor1.ovirt2.packet.mx, VdsIdVDSCommandParametersBase:{hostId='a45f92bf-a0ea-4398-aef8-2e0d73096a9d'}), log id: 714f953e 2020-06-01 16:44:44,301-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (default task-68) [728a6c91-f499-419a-a92e-82b9943e233f] FINISH, GlusterServersListVDSCommand, return: [172.25.209.11/24:CONNECTED, hypervisor3.ovirt2.packet.mx:CONNECTED, hypervisor2.ovirt2.packet.mx:CONNECTED], log id: 714f953e 2020-06-01 16:44:44,473-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (default task-68) [b412ff2f-9d0d-4621-90a0-bc88b4140085] START, GlusterServersListVDSCommand(HostName = hypervisor1.ovirt2.packet.mx, VdsIdVDSCommandParametersBase:{hostId='a45f92bf-a0ea-4398-aef8-2e0d73096a9d'}), log id: 62a07883 2020-06-01 16:44:44,667-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeSnapshotInfoVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] START, GetGlusterVolumeSnapshotInfoVDSCommand(HostName = hypervisor1.ovirt2.packet.mx, GlusterVolumeSnapshotVDSParameters:{hostId='a45f92bf-a0ea-4398-aef8-2e0d73096a9d', volumeName='null'}), log id: 16d9ef61 2020-06-01 16:44:44,763-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (default task-68) [b412ff2f-9d0d-4621-90a0-bc88b4140085] FINISH, GlusterServersListVDSCommand, return: [172.25.209.11/24:CONNECTED, hypervisor3.ovirt2.packet.mx:CONNECTED, hypervisor2.ovirt2.packet.mx:CONNECTED], log id: 62a07883 2020-06-01 16:44:44,817-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeSnapshotInfoVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] FINISH, GetGlusterVolumeSnapshotInfoVDSCommand, return: [], log id: 16d9ef61 2020-06-01 16:44:44,822-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeSnapshotConfigInfoVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] START, GetGlusterVolumeSnapshotConfigInfoVDSCommand(HostName = hypervisor1.ovirt2.packet.mx, GlusterVolumeSnapshotVDSParameters:{hostId='a45f92bf-a0ea-4398-aef8-2e0d73096a9d', volumeName='null'}), log id: 56da6779 2020-06-01 16:44:44,960-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeSnapshotConfigInfoVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-25) [] FINISH, GetGlusterVolumeSnapshotConfigInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.gluster.GlusterSnapshotConfigInfo@7a582908, log id: 56da6779 2020-06-01 16:44:48,783-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (default task-71) [97954f37-f2a1-446f-a7d9-4a5be29365cf] START, GlusterServersListVDSCommand(HostName = hypervisor1.ovirt2.packet.mx, VdsIdVDSCommandParametersBase:{hostId='a45f92bf-a0ea-4398-aef8-2e0d73096a9d'}), log id: 3c84ebd3 2020-06-01 16:44:49,073-05 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand] (default task-71) [97954f37-f2a1-446f-a7d9-4a5be29365cf] FINISH, GlusterServersListVDSCommand, return: [172.25.209.11/24:CONNECTED, hypervisor3.ovirt2.packet.mx:CONNECTED, hypervisor2.ovirt2.packet.mx:CONNECTED], log id: 3c84ebd3 2020-06-01 16:44:50,409-05 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-65) [aa50e9bf-ab1c-42da-9e8c-760262a33b70] EVENT_ID: ANSIBLE_RUNNER_EVENT_NOTIFICATION(559), Create Brick. Create thin logical volume. 2020-06-01 16:44:50,418-05 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-65) [aa50e9bf-ab1c-42da-9e8c-760262a33b70] EVENT_ID: ANSIBLE_RUNNER_EVENT_NOTIFICATION(559), Create Brick. include_tasks. 2020-06-01 16:44:53,434-05 ERROR [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-65) [aa50e9bf-ab1c-42da-9e8c-760262a33b70] Command 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand' failed: Task Setup SSD for caching | Extend the Volume Group failed to execute: 2020-06-01 16:44:53,434-05 ERROR [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-65) [aa50e9bf-ab1c-42da-9e8c-760262a33b70] Exception: org.ovirt.engine.core.common.utils.ansible.AnsibleRunnerCallException: Task Setup SSD for caching | Extend the Volume Group failed to execute: at deployment.engine.ear.bll.jar//org.ovirt.engine.core.common.utils.ansible.AnsibleRunnerHTTPClient.processEvents(AnsibleRunnerHTTPClient.java:239) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor.runCommand(AnsibleExecutor.java:170) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor.runCommand(AnsibleExecutor.java:59) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor.runCommand(AnsibleExecutor.java:46) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.gluster.CreateBrickCommand.runAnsibleCreateBrickPlaybook(CreateBrickCommand.java:192) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.gluster.CreateBrickCommand.executeCommand(CreateBrickCommand.java:109) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1169) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1327) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:2003) at org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:181) at org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInRequired(TransactionSupport.java:113) at org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:81) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1387) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:419) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.executor.DefaultBackendActionExecutor.execute(DefaultBackendActionExecutor.java:13) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.Backend.runAction(Backend.java:442) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:424) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.Backend.runAction(Backend.java:379) at jdk.internal.reflect.GeneratedMethodAccessor247.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.jboss.as.ee@18.0.1.Final//org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at org.jboss.as.weld.ejb@18.0.1.Final//org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:92) at org.jboss.weld.core@3.1.2.Final//org.jboss.weld.interceptor.proxy.WeldInvocationContextImpl.interceptorChainCompleted(WeldInvocationContextImpl.java:107) at org.jboss.weld.core@3.1.2.Final//org.jboss.weld.interceptor.proxy.WeldInvocationContextImpl.proceed(WeldInvocationContextImpl.java:126) at org.ovirt.engine.core.common//org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12) at jdk.internal.reflect.GeneratedMethodAccessor57.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.jboss.weld.core@3.1.2.Final//org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73) at org.jboss.weld.core@3.1.2.Final//org.jboss.weld.interceptor.proxy.WeldInvocationContextImpl.invokeNext(WeldInvocationContextImpl.java:92) at org.jboss.weld.core@3.1.2.Final//org.jboss.weld.interceptor.proxy.WeldInvocationContextImpl.proceed(WeldInvocationContextImpl.java:124) at org.jboss.weld.core@3.1.2.Final//org.jboss.weld.bean.InterceptorImpl.intercept(InterceptorImpl.java:105) at org.jboss.as.weld.ejb@18.0.1.Final//org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:82) at org.jboss.as.weld.ejb@18.0.1.Final//org.jboss.as.weld.interceptors.EjbComponentInterceptorSupport.delegateInterception(EjbComponentInterceptorSupport.java:60) at org.jboss.as.weld.common@18.0.1.Final//org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:77) at org.jboss.as.weld.common@18.0.1.Final//org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:89) at org.jboss.as.weld.common@18.0.1.Final//org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:102) at org.jboss.as.ee@18.0.1.Final//org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.interceptors.CorrelationIdTrackerInterceptor.aroundInvoke(CorrelationIdTrackerInterceptor.java:13) at jdk.internal.reflect.GeneratedMethodAccessor62.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.jboss.as.ee@18.0.1.Final//org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:89) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@18.0.1.Final//org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ee@18.0.1.Final//org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:40) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53) at org.jboss.as.ee@18.0.1.Final//org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@18.0.1.Final//org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@18.0.1.Final//org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:216) at org.jboss.as.ejb3@18.0.1.Final//org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:418) at org.jboss.as.ejb3@18.0.1.Final//org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:148) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at org.jboss.weld.core@3.1.2.Final//org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:81) at org.jboss.as.weld.common@18.0.1.Final//org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@18.0.1.Final//org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@18.0.1.Final//org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@18.0.1.Final//org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@18.0.1.Final//org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@18.0.1.Final//org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@18.0.1.Final//org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ee@18.0.1.Final//org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438) at org.wildfly.security.elytron-private@1.10.4.Final//org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:627) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53) at org.jboss.as.ee@18.0.1.Final//org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee@18.0.1.Final//org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.as.ee@18.0.1.Final//org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:81) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.common.interfaces.BackendLocal$$$view4.runAction(Unknown Source) at deployment.engine.ear.webadmin.war//org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runAction(GenericApiGWTServiceImpl.java:176) at jdk.internal.reflect.GeneratedMethodAccessor253.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at deployment.engine.ear.webadmin.war//com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:587) at deployment.engine.ear.webadmin.war//com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:333) at deployment.engine.ear.webadmin.war//com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:303) at deployment.engine.ear.webadmin.war//com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:373) at deployment.engine.ear.webadmin.war//com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) at javax.servlet.api@2.0.0.Final//javax.servlet.http.HttpServlet.service(HttpServlet.java:523) at deployment.engine.ear.webadmin.war//org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.service(GenericApiGWTServiceImpl.java:78) at javax.servlet.api@2.0.0.Final//javax.servlet.http.HttpServlet.service(HttpServlet.java:590) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129) at org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.servlet.CachingFilter.doFilter(CachingFilter.java:133) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at deployment.engine.ear.webadmin.war//org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:73) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:65) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow@18.0.1.Final//org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.core@2.0.27.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:68) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:132) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.core@2.0.27.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.core@2.0.27.Final//io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.core@2.0.27.Final//io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.core@2.0.27.Final//io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.core@2.0.27.Final//io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.core@2.0.27.Final//io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.core@2.0.27.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow@18.0.1.Final//org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.core@2.0.27.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.core@2.0.27.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:269) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:78) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:133) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:130) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at org.wildfly.extension.undertow@18.0.1.Final//org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) at org.wildfly.extension.undertow@18.0.1.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1504) at org.wildfly.extension.undertow@18.0.1.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1504) at org.wildfly.extension.undertow@18.0.1.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1504) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:249) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:78) at io.undertow.servlet@2.0.27.Final//io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:99) at io.undertow.core@2.0.27.Final//io.undertow.server.Connectors.executeRootHandler(Connectors.java:376) at io.undertow.core@2.0.27.Final//io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) at org.jboss.threads@2.3.3.Final//org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads@2.3.3.Final//org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads@2.3.3.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads@2.3.3.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.base/java.lang.Thread.run(Thread.java:834) 2020-06-01 16:44:53,436-05 ERROR [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-65) [aa50e9bf-ab1c-42da-9e8c-760262a33b70] Transaction rolled-back for command 'org.ovirt.engine.core.bll.gluster.CreateBrickCommand'. 2020-06-01 16:44:53,443-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-65) [aa50e9bf-ab1c-42da-9e8c-760262a33b70] EVENT_ID: CREATE_GLUSTER_BRICK_FAILED(4,139), Failed to create brick vmsa on host hypervisor1.ovirt2.packet.mx of cluster Default. 2020-06-01 16:44:53,451-05 INFO [org.ovirt.engine.core.bll.gluster.CreateBrickCommand] (default task-65) [aa50e9bf-ab1c-42da-9e8c-760262a33b70] Lock freed to object 'EngineLock:{exclusiveLocks='[9b58ebc6-fe0d-4b9c-ad0a-a8f3999ada03=HOST_STORAGE_DEVICES]', sharedLocks=''}'