Cannot remove Snapshot. Related operation is currently in progress. Please try again later.

Wednesday, 13 April 2022

Hello,

my storage got disconnected during the image migration, resulting is stuck migration. Task
had been cleaned with restart of vdsdm service, but lead to stuck disk image. The VM got
powered down in the meantime, and currently I cannot do anything with it, I always get a
message like "Cannot.... Related operation is currently in progress."  I already
used the unlock_entity.sh script, it cleaned something, but did not help me. I cannot
delete the VM, delete the disk, delete the snapshots. I even deleted the wrong snapshot
from disk. All I need is it get rid of the VM and its disk, as I can restore it from the
backup. Could you please somebody try to navigate me how to solve the problem? Engine log
keeps filling up with messages which indicate there is an orphaned task, however I am not
sure what to delete from database:

2022-04-13 22:26:41,822+02 INFO  [org.ovirt.engine.core.bll.StorageJobCallback]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-87)
[e546bb30-7d18-4348-9c28-9dbbf4ab1893] Command CopyData id:
'62b1976b-729a-41ae-972c-a42de750ad1d': couldn't get the status of job
'6ef3cb28-14cf-486a-ad71-96484c1a596e' on host 'kvm2.finamis.com' (id:
'f880cfbe-5827-4853-a4fc-11199c439f79'), assuming it's still running
2022-04-13 22:26:41,846+02 INFO 
[org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-87)
[e546bb30-7d18-4348-9c28-9dbbf4ab1893] Command 'LiveMigrateDisk' (id:
'3cd698fd-08a2-419e-bdd8-6b3516bb3127') waiting on child command id:
'2e9ad954-8f4e-41ed-9c75-015afeed337d' type:'CopyImageGroupVolumesData' to
complete
2022-04-13 22:26:41,847+02 INFO  [org.ovirt.engine.core.bll.StorageJobCallback]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-87)
[e546bb30-7d18-4348-9c28-9dbbf4ab1893] Command FenceVolumeJob id:
'831828a3-2e0f-4989-965b-91761fa9601f': job
'9dc88836-db04-49ae-b01e-18a56e24c91a' wasn't executed on any host,
considering the job status as failed
2022-04-13 22:26:41,847+02 INFO  [org.ovirt.engine.core.bll.StorageJobCallback]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-87)
[e546bb30-7d18-4348-9c28-9dbbf4ab1893] Command FenceVolumeJob id:
'831828a3-2e0f-4989-965b-91761fa9601f': execution was completed, the command
status is 'FAILED'
2022-04-13 22:26:42,850+02 ERROR [org.ovirt.engine.core.bll.job.JobRepositoryImpl]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-6)
[e546bb30-7d18-4348-9c28-9dbbf4ab1893] Failed to save step
'499b0207-61ba-4998-8cb8-2cb93b89422b', 'FINALIZING':
CallableStatementCallback; SQL [{call insertstep(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
?)}ERROR: insert or update on table "step" violates foreign key constraint
"fk_step_job"
  Detail: Key (job_id)=(ed10f993-cfd2-4a8b-806f-3fbd823101ef) is not present in table
"job".
  Where: SQL statement "INSERT INTO step (
        step_id,
        parent_step_id,
        job_id,
        step_type,
        description,
        step_number,
        status,
        progress,
        start_time,
        end_time,
        correlation_id,
        external_id,
        external_system_type,
        is_external
        )
    VALUES (
        v_step_id,
        v_parent_step_id,
        v_job_id,
        v_step_type,
        v_description,
        v_step_number,
        v_status,
        v_progress,
        v_start_time,
        v_end_time,
        v_correlation_id,
        v_external_id,
        v_external_system_type,
        v_is_external
        )"
PL/pgSQL function insertstep(uuid,uuid,uuid,character varying,text,integer,character
varying,smallint,timestamp with time zone,timestamp with time zone,character
varying,uuid,character varying,boolean) line 3 at SQL statement; nested exception is
org.postgresql.util.PSQLException: ERROR: insert or update on table "step"
violates foreign key constraint "fk_step_job"
  Detail: Key (job_id)=(ed10f993-cfd2-4a8b-806f-3fbd823101ef) is not present in table
"job".
  Where: SQL statement "INSERT INTO step (
        step_id,
        parent_step_id,
        job_id,
        step_type,
        description,
        step_number,
        status,
        progress,
        start_time,
        end_time,
        correlation_id,
        external_id,
        external_system_type,
        is_external
        )
    VALUES (
        v_step_id,
        v_parent_step_id,
        v_job_id,
        v_step_type,
        v_description,
        v_step_number,
        v_status,
        v_progress,
        v_start_time,
        v_end_time,
        v_correlation_id,
        v_external_id,
        v_external_system_type,
        v_is_external
        )"
PL/pgSQL function insertstep(uuid,uuid,uuid,character varying,text,integer,character
varying,smallint,timestamp with time zone,timestamp with time zone,character
varying,uuid,character varying,boolean) line 3 at SQL statement
2022-04-13 22:26:42,851+02 ERROR
[org.ovirt.engine.core.bll.storage.disk.image.FenceVolumeJobCommand]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-6)
[e546bb30-7d18-4348-9c28-9dbbf4ab1893] Ending command
'org.ovirt.engine.core.bll.storage.disk.image.FenceVolumeJobCommand' with
failure.
2022-04-13 22:26:51,855+02 INFO 
[org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-76)
[e546bb30-7d18-4348-9c28-9dbbf4ab1893] Command 'CopyImageGroupVolumesData' (id:
'2e9ad954-8f4e-41ed-9c75-015afeed337d') waiting on child command id:
'62b1976b-729a-41ae-972c-a42de750ad1d' type:'CopyData' to complete
2022-04-13 22:26:51,858+02 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetHostJobsVDSCommand]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-76)
[e546bb30-7d18-4348-9c28-9dbbf4ab1893] START, GetHostJobsVDSCommand(HostName =
kvm2.finamis.com,
GetHostJobsVDSCommandParameters:{hostId='f880cfbe-5827-4853-a4fc-11199c439f79',
type='storage', jobIds='[6ef3cb28-14cf-486a-ad71-96484c1a596e]'}), log id:
480d931c
2022-04-13 22:26:51,862+02 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetHostJobsVDSCommand]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-76)
[e546bb30-7d18-4348-9c28-9dbbf4ab1893] FINISH, GetHostJobsVDSCommand, return: {}, log id:
480d931c
2022-04-13 22:26:51,862+02 INFO  [org.ovirt.engine.core.bll.StorageJobCallback]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-76)
[e546bb30-7d18-4348-9c28-9dbbf4ab1893] Command CopyData id:
'62b1976b-729a-41ae-972c-a42de750ad1d': attempting to determine the job status by
polling the entity.
2022-04-13 22:26:51,862+02 ERROR [org.ovirt.engine.core.bll.StorageJobCallback]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-76)
[e546bb30-7d18-4348-9c28-9dbbf4ab1893] Command CopyData id:
'62b1976b-729a-41ae-972c-a42de750ad1d': failed to poll the command entity
2022-04-13 22:26:51,862+02 INFO 
[org.ovirt.engine.core.bll.storage.disk.image.CopyDataCommand]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-76)
[e546bb30-7d18-4348-9c28-9dbbf4ab1893] Command CopyData id:
'62b1976b-729a-41ae-972c-a42de750ad1d': attempting to fence job
ed10f993-cfd2-4a8b-806f-3fbd823101ef
2022-04-13 22:26:51,868+02 INFO 
[org.ovirt.engine.core.bll.storage.disk.image.FenceVolumeJobCommand]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-76)
[e546bb30-7d18-4348-9c28-9dbbf4ab1893] Running command: FenceVolumeJobCommand internal:
true.
2022-04-13 22:26:51,868+02 ERROR
[org.ovirt.engine.core.bll.storage.disk.image.FenceVolumeJobCommand]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-76)
[e546bb30-7d18-4348-9c28-9dbbf4ab1893] Command
'org.ovirt.engine.core.bll.storage.disk.image.FenceVolumeJobCommand' failed: null
2022-04-13 22:26:51,868+02 ERROR
[org.ovirt.engine.core.bll.storage.disk.image.FenceVolumeJobCommand]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-76)
[e546bb30-7d18-4348-9c28-9dbbf4ab1893] Exception: java.lang.NullPointerException
        at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.image.FenceVolumeJobCommand.executeCommand(FenceVolumeJobCommand.java:31)
        at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1174)
        at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1332)
        at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:2008)
        at
org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:140)
        at
org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:79)
        at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1392)
        at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:424)
        at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.executor.DefaultBackendActionExecutor.execute(DefaultBackendActionExecutor.java:13)
        at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.Backend.runAction(Backend.java:450)
        at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:432)
        at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:638)
        at jdk.internal.reflect.GeneratedMethodAccessor273.invoke(Unknown Source)
        at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at
org.jboss.as.ee@23.0.2.Final//org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52)
        at
org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
        at
org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
        at
org.jboss.as.weld.common@23.0.2.Final//org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:79)
        at
org.jboss.as.weld.common@23.0.2.Final//org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:89)
        at
org.jboss.as.weld.common@23.0.2.Final//org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:102)
        at
org.jboss.as.ee@23.0.2.Final//org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63)
        at
org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
        at
org.jboss.as.ejb3@23.0.2.Final//org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43)
        at
org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
        at
org.jboss.as.ee@23.0.2.Final//org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45)
        at
org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
        at
org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:40)
        at
org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
        at
org.jboss.invocation@1.6.0.Final//org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53)
        at
org.jboss.as.ee@23.0.2.Final//org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52)
        at
org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
        at
org.jboss.as.ejb3@23.0.2.Final//org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53)
        at
org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
        at
org.jboss.as.ejb3@23.0.2.Final//org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:232)
        at
org.jboss.as.ejb3@23.0.2.Final//org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:446)
        at
org.jboss.as.ejb3@23.0.2.Final//org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:164)
        at
org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
        at
org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
        at
org.jboss.weld.core@3.1.6.Final//org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:81)
        at
org.jboss.as.weld.common@23.0.2.Final//org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89)
        at
org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
        at
org.jboss.as.ejb3@23.0.2.Final//org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
        at
org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
        at
org.jboss.as.ejb3@23.0.2.Final//org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47)
        at
org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
        at
org.jboss.as.ejb3@23.0.2.Final//org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100)
        at
org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
        at
org.jboss.as.ejb3@23.0.2.Final//org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22)
        at
org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
        at
org.jboss.as.ejb3@23.0.2.Final//org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64)
        at
org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
        at
org.jboss.as.ejb3@23.0.2.Final//org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67)
        at
org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
        at
org.jboss.as.ee@23.0.2.Final//org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
        at
org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
        at
org.jboss.invocation@1.6.0.Final//org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60)
        at
org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
        at
org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438)
        at
org.wildfly.security.elytron-private@1.15.3.Final//org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:633)
        at
org.jboss.invocation@1.6.0.Final//org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57)
        at
org.jboss.invocation@1.6.0.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
        at
org.jboss.invocation@1.6.0.Final//org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53)
        at
org.jboss.as.ee@23.0.2.Final//org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198)
        at
org.jboss.as.ee@23.0.2.Final//org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:191)
        at
org.jboss.as.ee@23.0.2.Final//org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:81)
        at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view3.runInternalAction(Unknown
Source)
        at jdk.internal.reflect.GeneratedMethodAccessor272.invoke(Unknown Source)
        at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at
org.jboss.weld.core@3.1.6.Final//org.jboss.weld.util.reflection.Reflections.invokeAndUnwrap(Reflections.java:410)
        at
org.jboss.weld.core@3.1.6.Final//org.jboss.weld.module.ejb.EnterpriseBeanProxyMethodHandler.invoke(EnterpriseBeanProxyMethodHandler.java:134)
        at
org.jboss.weld.core@3.1.6.Final//org.jboss.weld.bean.proxy.EnterpriseTargetBeanInstance.invoke(EnterpriseTargetBeanInstance.java:56)
        at
org.jboss.weld.core@3.1.6.Final//org.jboss.weld.module.ejb.InjectionPointPropagatingEnterpriseTargetBeanInstance.invoke(InjectionPointPropagatingEnterpriseTargetBeanInstance.java:68)
        at
org.jboss.weld.core@3.1.6.Final//org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:106)
        at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.BackendCommandObjectsHandler$BackendInternal$BackendLocal2049259618$Proxy$_$$_Weld$EnterpriseProxy$.runInternalAction(Unknown
Source)
        at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.runInternalAction(CommandBase.java:2386)
        at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.runInternalActionWithTasksContext(CommandBase.java:2411)
        at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.runInternalActionWithTasksContext(CommandBase.java:2406)
        at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.image.CopyDataCommand.attemptToFenceJob(CopyDataCommand.java:144)
        at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.HostJobCallback.handleUndeterminedJobStatus(HostJobCallback.java:196)
        at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.HostJobCallback.childCommandsExecutionEnded(HostJobCallback.java:84)
        at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.ChildCommandsCallbackBase.doPolling(ChildCommandsCallbackBase.java:80)
        at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethodsImpl(CommandCallbacksPoller.java:181)
        at
deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethods(CommandCallbacksPoller.java:109)
        at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
        at
org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:360)
        at
org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:511)
        at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
        at
org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:227)

2022-04-13 22:26:51,872+02 INFO  [org.ovirt.engine.core.bll.StorageJobCallback]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-76)
[e546bb30-7d18-4348-9c28-9dbbf4ab1893] Command CopyData id:
'62b1976b-729a-41ae-972c-a42de750ad1d': couldn't get the status of job
'6ef3cb28-14cf-486a-ad71-96484c1a596e' on host 'kvm2.finamis.com' (id:
'f880cfbe-5827-4853-a4fc-11199c439f79'), assuming it's still running
2022-04-13 22:26:51,899+02 INFO 
[org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-76)
[e546bb30-7d18-4348-9c28-9dbbf4ab1893] Command 'LiveMigrateDisk' (id:
'3cd698fd-08a2-419e-bdd8-6b3516bb3127') waiting on child command id:
'2e9ad954-8f4e-41ed-9c75-015afeed337d' type:'CopyImageGroupVolumesData' to
complete
2022-04-13 22:26:51,900+02 INFO  [org.ovirt.engine.core.bll.StorageJobCallback]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-76)
[e546bb30-7d18-4348-9c28-9dbbf4ab1893] Command FenceVolumeJob id:
'3c8b0471-c405-4b7f-ac54-43732407d3cb': job
'3c38e07a-44f3-46b7-97f9-56b6218e69b4' wasn't executed on any host,
considering the job status as failed
2022-04-13 22:26:51,900+02 INFO  [org.ovirt.engine.core.bll.StorageJobCallback]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-76)
[e546bb30-7d18-4348-9c28-9dbbf4ab1893] Command FenceVolumeJob id:
'3c8b0471-c405-4b7f-ac54-43732407d3cb': execution was completed, the command
status is 'FAILED'
2022-04-13 22:26:52,903+02 ERROR [org.ovirt.engine.core.bll.job.JobRepositoryImpl]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-14)
[e546bb30-7d18-4348-9c28-9dbbf4ab1893] Failed to save step
'ad184ef6-e708-41f3-acff-962d39f01c30', 'FINALIZING':
CallableStatementCallback; SQL [{call insertstep(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
?)}ERROR: insert or update on table "step" violates foreign key constraint
"fk_step_job"
  Detail: Key (job_id)=(ed10f993-cfd2-4a8b-806f-3fbd823101ef) is not present in table
"job".
  Where: SQL statement "INSERT INTO step (
        step_id,
        parent_step_id,
        job_id,
        step_type,
        description,
        step_number,
        status,
        progress,
        start_time,
        end_time,
        correlation_id,
        external_id,
        external_system_type,
        is_external
        )
    VALUES (
        v_step_id,
        v_parent_step_id,
        v_job_id,
        v_step_type,
        v_description,
        v_step_number,
        v_status,
        v_progress,
        v_start_time,
        v_end_time,
        v_correlation_id,
        v_external_id,
        v_external_system_type,
        v_is_external
        )"
PL/pgSQL function insertstep(uuid,uuid,uuid,character varying,text,integer,character
varying,smallint,timestamp with time zone,timestamp with time zone,character
varying,uuid,character varying,boolean) line 3 at SQL statement; nested exception is
org.postgresql.util.PSQLException: ERROR: insert or update on table "step"
violates foreign key constraint "fk_step_job"
  Detail: Key (job_id)=(ed10f993-cfd2-4a8b-806f-3fbd823101ef) is not present in table
"job".
  Where: SQL statement "INSERT INTO step (
        step_id,
        parent_step_id,
        job_id,
        step_type,
        description,
        step_number,
        status,
        progress,
        start_time,
        end_time,
        correlation_id,
        external_id,
        external_system_type,
        is_external
        )
    VALUES (
        v_step_id,
        v_parent_step_id,
        v_job_id,
        v_step_type,
        v_description,
        v_step_number,
        v_status,
        v_progress,
        v_start_time,
        v_end_time,
        v_correlation_id,
        v_external_id,
        v_external_system_type,
        v_is_external
        )"
PL/pgSQL function insertstep(uuid,uuid,uuid,character varying,text,integer,character
varying,smallint,timestamp with time zone,timestamp with time zone,character
varying,uuid,character varying,boolean) line 3 at SQL statement
2022-04-13 22:26:52,904+02 ERROR
[org.ovirt.engine.core.bll.storage.disk.image.FenceVolumeJobCommand]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-14)
[e546bb30-7d18-4348-9c28-9dbbf4ab1893] Ending command
'org.ovirt.engine.core.bll.storage.disk.image.FenceVolumeJobCommand' with
failure.
^C

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011