Re: [External] : Really weird ! Image IO / browser upload only working on 1 cluster (out of many) - anyone got any advice ? (network access, certs seem fine.) - Users

6 Mar 2025

      Hi.

Sorry for the delay in response..

still have the issue

Its is not cert related as I can use image io for one cluster/dc (the first one) and that wouldn't work if cert was the issue..

i.e we have 

cluster1: app1+app2  - app1 sd (this works)
cluster2: db1+db2 - db1 sd 
cluster3: dmz1+dmz2 - dmz1 sd 
, and so on (we have 8 clusters each in own dc) each cluster has access to their own section of san (storage domain)

the issue is that if I go to storage - domains -  and enter any storage domain and 'upload' the hosts showing in ALL clusters/storage domains in the drop down in the upload page shows the same hosts  'app1 and app2..'

So I can upload to the storage domain app1+app2 uses (only) and only if I use the hosts app1 and app2 (cluster1) - but all storage domains only show app1 and app2 as hosts (in the upload section)  - in the storage domains used by db1 +db2 they should show those hosts (but I only see in drop down app1 + app2)..  Really odd..  - 

test connection always works (because I do have access to app1/app2 ) - but app1+app2 only have access to their sd ..

Here are logs when attempting to use imageio (through browser ) 

From engine - engine.log
-----------------------------------------------------------------------------------------------------------------------------------------
2025-03-06 17:16:48,356Z INFO  [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (default task-385) [70460fc8-7d40-4153-a387-f18a06641f6c] Lock Acquired to object 'EngineLock:{exclusiveLocks='[]', sharedLocks='[]'}'
2025-03-06 17:16:48,505Z INFO  [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (default task-385) [70460fc8-7d40-4153-a387-f18a06641f6c] Running command: TransferDiskImageCommand internal: false. Entities affected :  ID: 5fda4199-06c3-4d36-a6f2-be9791972c3b Type: StorageAction group CREATE_DISK with role type USER
2025-03-06 17:16:48,505Z INFO  [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (default task-385) [70460fc8-7d40-4153-a387-f18a06641f6c] Creating ImageTransfer entity for command '0e3822a0-68e8-4c51-ac86-b6a433649183', proxyEnabled: true
2025-03-06 17:16:48,535Z INFO  [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (default task-385) [70460fc8-7d40-4153-a387-f18a06641f6c] Starting image transfer: ImageTransfer:{id='0e3822a0-68e8-4c51-ac86-b6a433649183', phase='Initializing', type='Upload', active='false', lastUpdated='Thu Mar 06 17:16:48 UTC 2025', message='null', vdsId='null', diskId='null', imagedTicketId='null', proxyUri='null', bytesSent='null', bytesTotal='113', clientInactivityTimeout='60', timeoutPolicy='legacy', imageFormat='RAW', transferClientType='Transfer via browser', shallow='false'}
2025-03-06 17:16:48,535Z INFO  [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (default task-385) [70460fc8-7d40-4153-a387-f18a06641f6c] Creating disk image
2025-03-06 17:16:48,579Z INFO  [org.ovirt.engine.core.bll.storage.disk.AddDiskCommand] (default task-385) [70460fc8-7d40-4153-a387-f18a06641f6c] Running command: AddDiskCommand internal: true. Entities affected :  ID: 5fda4199-06c3-4d36-a6f2-be9791972c3b Type: StorageAction group CREATE_DISK with role type USER
2025-03-06 17:16:48,646Z INFO  [org.ovirt.engine.core.bll.storage.disk.image.AddImageFromScratchCommand] (default task-385) [70460fc8-7d40-4153-a387-f18a06641f6c] Running command: AddImageFromScratchCommand internal: true. Entities affected :  ID: 5fda4199-06c3-4d36-a6f2-be9791972c3b Type: Storage
2025-03-06 17:16:48,733Z INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.CreateVolumeVDSCommand] (default task-385) [70460fc8-7d40-4153-a387-f18a06641f6c] START, CreateVolumeVDSCommand( CreateVolumeVDSCommandParameters:{storagePoolId='5c316ba3-d868-4314-917c-da0cc53b577e', ignoreFailoverLimit='false', storageDomainId='5fda4199-06c3-4d36-a6f2-be9791972c3b', imageGroupId='22372bb7-9d9b-4db2-927d-1820917fd6c5', imageSizeInBytes='113', volumeFormat='COW', newImageId='79424cad-773d-4f29-ab88-b14ce0cfaa72', imageType='Sparse', newImageDescription='{"DiskAlias":"test","DiskDescription":""}', imageInitialSizeInBytes='113', imageId='00000000-0000-0000-0000-000000000000', sourceImageGroupId='00000000-0000-0000-0000-000000000000', shouldAddBitmaps='false', legal='true', sequenceNumber='1', bitmap='null'}), log id: 590d036d
2025-03-06 17:16:48,841Z INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.CreateVolumeVDSCommand] (default task-385) [70460fc8-7d40-4153-a387-f18a06641f6c] FINISH, CreateVolumeVDSCommand, return: 79424cad-773d-4f29-ab88-b14ce0cfaa72, log id: 590d036d
2025-03-06 17:16:48,841Z INFO  [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (default task-385) [70460fc8-7d40-4153-a387-f18a06641f6c] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command '3e07f385-e2b9-46ee-822f-a466e2215347'
2025-03-06 17:16:48,841Z INFO  [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (default task-385) [70460fc8-7d40-4153-a387-f18a06641f6c] CommandMultiAsyncTasks::attachTask: Attaching task '10c19ce5-cdac-4a0c-b1d2-75c34c5e2c27' to command '3e07f385-e2b9-46ee-822f-a466e2215347'.
2025-03-06 17:16:49,139Z INFO  [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (default task-385) [70460fc8-7d40-4153-a387-f18a06641f6c] Adding task '10c19ce5-cdac-4a0c-b1d2-75c34c5e2c27' (Parent Command 'AddImageFromScratch', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet..
2025-03-06 17:16:49,172Z INFO  [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (default task-385) [70460fc8-7d40-4153-a387-f18a06641f6c] BaseAsyncTask::startPollingTask: Starting to poll task '10c19ce5-cdac-4a0c-b1d2-75c34c5e2c27'.
2025-03-06 17:16:49,256Z INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-385) [70460fc8-7d40-4153-a387-f18a06641f6c] EVENT_ID: ADD_DISK_INTERNAL(2,036), Add-Disk operation of 'test' was initiated by the system.
2025-03-06 17:16:49,371Z INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-385) [70460fc8-7d40-4153-a387-f18a06641f6c] EVENT_ID: TRANSFER_IMAGE_INITIATED(1,031), Image Upload with disk test was initiated by mcox@internal-authz.
2025-03-06 17:16:50,682Z INFO  [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-46) [70460fc8-7d40-4153-a387-f18a06641f6c] Waiting for disk to be added for image transfer '0e3822a0-68e8-4c51-ac86-b6a433649183'
2025-03-06 17:16:50,682Z INFO  [org.ovirt.engine.core.bll.storage.disk.AddDiskCommandCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-46) [70460fc8-7d40-4153-a387-f18a06641f6c] Command 'AddDisk' (id: '53dfac83-aa6f-4b6e-8941-af27a4dc0e0b') waiting on child command id: '3e07f385-e2b9-46ee-822f-a466e2215347' type:'AddImageFromScratch' to complete
2025-03-06 17:16:52,584Z INFO  [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-4) [] Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now
2025-03-06 17:16:52,591Z INFO  [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-4) [] SPMAsyncTask::PollTask: Polling task '10c19ce5-cdac-4a0c-b1d2-75c34c5e2c27' (Parent Command 'AddImageFromScratch', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'success'.
2025-03-06 17:16:52,591Z INFO  [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-4) [] BaseAsyncTask::onTaskEndSuccess: Task '10c19ce5-cdac-4a0c-b1d2-75c34c5e2c27' (Parent Command 'AddImageFromScratch', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully.
2025-03-06 17:16:52,591Z INFO  [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-4) [] CommandAsyncTask::endActionIfNecessary: All tasks of command '3e07f385-e2b9-46ee-822f-a466e2215347' has ended -> executing 'endAction'
2025-03-06 17:16:52,591Z INFO  [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-4) [] CommandAsyncTask::endAction: Ending action for '1' tasks (command ID: '3e07f385-e2b9-46ee-822f-a466e2215347'): calling endAction '.
2025-03-06 17:16:52,592Z INFO  [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-65196) [] CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction 'AddImageFromScratch',
2025-03-06 17:16:52,648Z INFO  [org.ovirt.engine.core.bll.storage.disk.image.AddImageFromScratchCommand] (EE-ManagedThreadFactory-engine-Thread-65196) [70460fc8-7d40-4153-a387-f18a06641f6c] Command [id=3e07f385-e2b9-46ee-822f-a466e2215347]: Updating status to 'SUCCEEDED', The command end method logic will be executed by one of its parent commands.
2025-03-06 17:16:52,648Z INFO  [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-65196) [70460fc8-7d40-4153-a387-f18a06641f6c] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'AddImageFromScratch' completed, handling the result.
2025-03-06 17:16:52,648Z INFO  [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-65196) [70460fc8-7d40-4153-a387-f18a06641f6c] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'AddImageFromScratch' succeeded, clearing tasks.
2025-03-06 17:16:52,648Z INFO  [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engine-Thread-65196) [70460fc8-7d40-4153-a387-f18a06641f6c] SPMAsyncTask::ClearAsyncTask: Attempting to clear task '10c19ce5-cdac-4a0c-b1d2-75c34c5e2c27'
2025-03-06 17:16:52,649Z INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-65196) [70460fc8-7d40-4153-a387-f18a06641f6c] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{storagePoolId='5c316ba3-d868-4314-917c-da0cc53b577e', ignoreFailoverLimit='false', taskId='10c19ce5-cdac-4a0c-b1d2-75c34c5e2c27'}), log id: 3adf882e
2025-03-06 17:16:52,649Z INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-65196) [70460fc8-7d40-4153-a387-f18a06641f6c] START, HSMClearTaskVDSCommand(HostName = ng2-olvm-stbdb2, HSMTaskGuidBaseVDSCommandParameters:{hostId='52ad3b6b-8bd8-4cb4-a6dc-432ca3c15f26', taskId='10c19ce5-cdac-4a0c-b1d2-75c34c5e2c27'}), log id: 6432d6de
2025-03-06 17:16:52,666Z INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-65196) [70460fc8-7d40-4153-a387-f18a06641f6c] FINISH, HSMClearTaskVDSCommand, return: , log id: 6432d6de
2025-03-06 17:16:52,666Z INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-65196) [70460fc8-7d40-4153-a387-f18a06641f6c] FINISH, SPMClearTaskVDSCommand, return: , log id: 3adf882e
2025-03-06 17:16:52,673Z INFO  [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engine-Thread-65196) [70460fc8-7d40-4153-a387-f18a06641f6c] BaseAsyncTask::removeTaskFromDB: Removed task '10c19ce5-cdac-4a0c-b1d2-75c34c5e2c27' from DataBase
2025-03-06 17:16:52,673Z INFO  [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-65196) [70460fc8-7d40-4153-a387-f18a06641f6c] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity '3e07f385-e2b9-46ee-822f-a466e2215347'
2025-03-06 17:16:53,553Z INFO  [org.ovirt.engine.core.bll.storage.disk.image.TransferImageStatusCommand] (default task-385) [268debf2-769a-4562-be29-65fa4b11e4c2] Running command: TransferImageStatusCommand internal: false. Entities affected :  ID: 5fda4199-06c3-4d36-a6f2-be9791972c3b Type: SystemAction group CREATE_DISK with role type USER
2025-03-06 17:16:54,683Z INFO  [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-84) [70460fc8-7d40-4153-a387-f18a06641f6c] Waiting for disk to be added for image transfer '0e3822a0-68e8-4c51-ac86-b6a433649183'
2025-03-06 17:16:54,684Z INFO  [org.ovirt.engine.core.bll.storage.disk.AddDiskCommandCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-84) [70460fc8-7d40-4153-a387-f18a06641f6c] Getting volume info for image '22372bb7-9d9b-4db2-927d-1820917fd6c5/79424cad-773d-4f29-ab88-b14ce0cfaa72'
2025-03-06 17:16:54,686Z INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVolumeInfoVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-84) [70460fc8-7d40-4153-a387-f18a06641f6c] START, GetVolumeInfoVDSCommand(HostName = ng2-olvm-stbdb2, GetVolumeInfoVDSCommandParameters:{hostId='52ad3b6b-8bd8-4cb4-a6dc-432ca3c15f26', storagePoolId='5c316ba3-d868-4314-917c-da0cc53b577e', storageDomainId='5fda4199-06c3-4d36-a6f2-be9791972c3b', imageGroupId='22372bb7-9d9b-4db2-927d-1820917fd6c5', imageId='79424cad-773d-4f29-ab88-b14ce0cfaa72'}), log id: 4c62db44
2025-03-06 17:16:54,861Z INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVolumeInfoVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-84) [70460fc8-7d40-4153-a387-f18a06641f6c] FINISH, GetVolumeInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@7f011b06, log id: 4c62db44
2025-03-06 17:16:54,861Z INFO  [org.ovirt.engine.core.bll.storage.disk.AddDiskCommandCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-84) [70460fc8-7d40-4153-a387-f18a06641f6c] Updating size from '113' to '4096'
2025-03-06 17:16:54,874Z INFO  [org.ovirt.engine.core.bll.storage.disk.AddDiskCommandCallback] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-84) [70460fc8-7d40-4153-a387-f18a06641f6c] Command 'AddDisk' id: '53dfac83-aa6f-4b6e-8941-af27a4dc0e0b' child commands '[3e07f385-e2b9-46ee-822f-a466e2215347]' executions were completed, status 'SUCCEEDED'
2025-03-06 17:16:55,883Z INFO  [org.ovirt.engine.core.bll.storage.disk.AddDiskCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-22) [70460fc8-7d40-4153-a387-f18a06641f6c] Ending command 'org.ovirt.engine.core.bll.storage.disk.AddDiskCommand' successfully.
2025-03-06 17:16:55,964Z INFO  [org.ovirt.engine.core.bll.storage.disk.image.AddImageFromScratchCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-22) [70460fc8-7d40-4153-a387-f18a06641f6c] Ending command 'org.ovirt.engine.core.bll.storage.disk.image.AddImageFromScratchCommand' successfully.
2025-03-06 17:16:55,970Z INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-22) [70460fc8-7d40-4153-a387-f18a06641f6c] START, GetImageInfoVDSCommand( GetImageInfoVDSCommandParameters:{storagePoolId='5c316ba3-d868-4314-917c-da0cc53b577e', ignoreFailoverLimit='false', storageDomainId='5fda4199-06c3-4d36-a6f2-be9791972c3b', imageGroupId='22372bb7-9d9b-4db2-927d-1820917fd6c5', imageId='79424cad-773d-4f29-ab88-b14ce0cfaa72'}), log id: 4125fc8d
2025-03-06 17:16:55,970Z INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVolumeInfoVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-22) [70460fc8-7d40-4153-a387-f18a06641f6c] START, GetVolumeInfoVDSCommand(HostName = ng2-olvm-stbdb2, GetVolumeInfoVDSCommandParameters:{hostId='52ad3b6b-8bd8-4cb4-a6dc-432ca3c15f26', storagePoolId='5c316ba3-d868-4314-917c-da0cc53b577e', storageDomainId='5fda4199-06c3-4d36-a6f2-be9791972c3b', imageGroupId='22372bb7-9d9b-4db2-927d-1820917fd6c5', imageId='79424cad-773d-4f29-ab88-b14ce0cfaa72'}), log id: 6a92bb72
2025-03-06 17:16:56,015Z INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVolumeInfoVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-22) [70460fc8-7d40-4153-a387-f18a06641f6c] FINISH, GetVolumeInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@7f011b06, log id: 6a92bb72
2025-03-06 17:16:56,015Z INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-22) [70460fc8-7d40-4153-a387-f18a06641f6c] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@7f011b06, log id: 4125fc8d
2025-03-06 17:16:56,057Z INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.PrepareImageVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-22) [70460fc8-7d40-4153-a387-f18a06641f6c] START, PrepareImageVDSCommand(HostName = ng2-olvm-stbdb1, PrepareImageVDSCommandParameters:{hostId='681f0da5-9d7c-4a62-b7b8-df833e43ff29'}), log id: 1f67388c
2025-03-06 17:16:56,889Z INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.PrepareImageVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-22) [70460fc8-7d40-4153-a387-f18a06641f6c] FINISH, PrepareImageVDSCommand, return: PrepareImageReturn:{status='Status [code=0, message=Done]'}, log id: 1f67388c
2025-03-06 17:16:56,891Z INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.GetQemuImageInfoVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-22) [70460fc8-7d40-4153-a387-f18a06641f6c] START, GetQemuImageInfoVDSCommand(HostName = ng2-olvm-stbdb1, GetVolumeInfoVDSCommandParameters:{hostId='681f0da5-9d7c-4a62-b7b8-df833e43ff29', storagePoolId='5c316ba3-d868-4314-917c-da0cc53b577e', storageDomainId='5fda4199-06c3-4d36-a6f2-be9791972c3b', imageGroupId='22372bb7-9d9b-4db2-927d-1820917fd6c5', imageId='79424cad-773d-4f29-ab88-b14ce0cfaa72'}), log id: 1a568567
2025-03-06 17:16:56,935Z INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.GetQemuImageInfoVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-22) [70460fc8-7d40-4153-a387-f18a06641f6c] FINISH, GetQemuImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.QemuImageInfo@6ab4f0e9, log id: 1a568567
2025-03-06 17:16:56,936Z INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.TeardownImageVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-22) [70460fc8-7d40-4153-a387-f18a06641f6c] START, TeardownImageVDSCommand(HostName = ng2-olvm-stbdb1, ImageActionsVDSCommandParameters:{hostId='681f0da5-9d7c-4a62-b7b8-df833e43ff29'}), log id: 15dea1c0
2025-03-06 17:16:57,253Z INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.TeardownImageVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-22) [70460fc8-7d40-4153-a387-f18a06641f6c] FINISH, TeardownImageVDSCommand, return: StatusReturn:{status='Status [code=0, message=Done]'}, log id: 15dea1c0
2025-03-06 17:16:57,268Z WARN  [org.ovirt.engine.core.bll.storage.disk.AddDiskCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-22) [] VM is null - no unlocking
2025-03-06 17:16:57,301Z INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-22) [] EVENT_ID: USER_ADD_DISK_FINISHED_SUCCESS(2,021), The disk 'test' was successfully added.
2025-03-06 17:16:57,552Z INFO  [org.ovirt.engine.core.bll.storage.disk.image.TransferImageStatusCommand] (default task-385) [c31320c9-f078-4aa8-b17f-802907bf5124] Running command: TransferImageStatusCommand internal: false. Entities affected :  ID: 5fda4199-06c3-4d36-a6f2-be9791972c3b Type: SystemAction group CREATE_DISK with role type USER
2025-03-06 17:16:58,376Z INFO  [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-34) [70460fc8-7d40-4153-a387-f18a06641f6c] Successfully added Upload disk 'test' (disk id: '22372bb7-9d9b-4db2-927d-1820917fd6c5', image id: '79424cad-773d-4f29-ab88-b14ce0cfaa72') for image transfer '0e3822a0-68e8-4c51-ac86-b6a433649183'
2025-03-06 17:16:58,398Z INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.PrepareImageVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-34) [70460fc8-7d40-4153-a387-f18a06641f6c] START, PrepareImageVDSCommand(HostName = ng2-olvm-app1, PrepareImageVDSCommandParameters:{hostId='a2500912-390f-4216-88b0-34f4e62c4dff'}), log id: 570232b4
2025-03-06 17:16:58,402Z ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PrepareImageVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-34) [70460fc8-7d40-4153-a387-f18a06641f6c] Failed in 'PrepareImageVDS' method, for vds: 'ng2-olvm-app1'; host: '10.168.72.201': null
2025-03-06 17:16:58,402Z ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PrepareImageVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-34) [70460fc8-7d40-4153-a387-f18a06641f6c] Command 'PrepareImageVDSCommand(HostName = ng2-olvm-app1, PrepareImageVDSCommandParameters:{hostId='a2500912-390f-4216-88b0-34f4e62c4dff'})' execution failed: null
2025-03-06 17:16:58,402Z INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.PrepareImageVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-34) [70460fc8-7d40-4153-a387-f18a06641f6c] FINISH, PrepareImageVDSCommand, return: , log id: 570232b4
2025-03-06 17:16:58,402Z ERROR [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-34) [70460fc8-7d40-4153-a387-f18a06641f6c] Failed to prepare image for image transfer '0e3822a0-68e8-4c51-ac86-b6a433649183': {}: org.ovirt.engine.core.common.errors.EngineException: EngineException: java.lang.NullPointerException (Failed with error ENGINE and code 5001)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:114)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.runVdsCommand(VDSBrokerFrontendImpl.java:33)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.runVdsCommand(CommandBase.java:2121)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand.prepareImage(TransferDiskImageCommand.java:187)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand.startImageTransferSession(TransferDiskImageCommand.java:1063)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand.handleImageIsReadyForTransfer(TransferDiskImageCommand.java:680)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand.handleInitializing(TransferDiskImageCommand.java:653)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand.executeStateHandler(TransferDiskImageCommand.java:586)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand.proceedCommandExecution(TransferDiskImageCommand.java:573)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.storage.disk.image.TransferImageCommandCallback.doPolling(TransferImageCommandCallback.java:21)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethodsImpl(CommandCallbacksPoller.java:175)
        at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethods(CommandCallbacksPoller.java:109)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
        at org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:360)
        at org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:511)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
        at org.glassfish.javax.enterprise.concurrent//org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:227)
Caused by: java.lang.NullPointerException
        at deployment.engine.ear//org.ovirt.engine.core.vdsbroker.vdsbroker.PrepareImageReturn.<init>(PrepareImageReturn.java:15)
        at deployment.engine.ear//org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.prepareImage(JsonRpcVdsServer.java:1947)
        at deployment.engine.ear//org.ovirt.engine.core.vdsbroker.vdsbroker.PrepareImageVDSCommand.executeImageActionVdsBrokerCommand(PrepareImageVDSCommand.java:18)
        at deployment.engine.ear//org.ovirt.engine.core.vdsbroker.vdsbroker.PrepareImageVDSCommand.executeImageActionVdsBrokerCommand(PrepareImageVDSCommand.java:5)
        at deployment.engine.ear//org.ovirt.engine.core.vdsbroker.vdsbroker.ImageActionsVDSCommandBase.executeVdsBrokerCommand(ImageActionsVDSCommandBase.java:14)
        at deployment.engine.ear//org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVdsCommandWithNetworkEvent(VdsBrokerCommand.java:123)
        at deployment.engine.ear//org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:111)
        at deployment.engine.ear//org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:65)
        at org.ovirt.engine.core.dal//org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31)
        at deployment.engine.ear//org.ovirt.engine.core.vdsbroker.vdsbroker.DefaultVdsCommandExecutor.execute(DefaultVdsCommandExecutor.java:14)
        at deployment.engine.ear//org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:410)
        at deployment.engine.ear//org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand$$super(Unknown Source)
        at jdk.internal.reflect.GeneratedMethodAccessor70.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.jboss.weld.core@3.1.7.SP1//org.jboss.weld.interceptor.proxy.TerminalAroundInvokeInvocationContext.proceedInternal(TerminalAroundInvokeInvocationContext.java:51)
        at org.jboss.weld.core@3.1.7.SP1//org.jboss.weld.interceptor.proxy.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:78)
        at org.ovirt.engine.core.common//org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12)
        at jdk.internal.reflect.GeneratedMethodAccessor65.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.jboss.weld.core@3.1.7.SP1//org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73)
        at org.jboss.weld.core@3.1.7.SP1//org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeAroundInvoke(InterceptorMethodHandler.java:84)
        at org.jboss.weld.core@3.1.7.SP1//org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeInterception(InterceptorMethodHandler.java:72)
        at org.jboss.weld.core@3.1.7.SP1//org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.invoke(InterceptorMethodHandler.java:56)
        at org.jboss.weld.core@3.1.7.SP1//org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:79)
        at org.jboss.weld.core@3.1.7.SP1//org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:68)
        at deployment.engine.ear//org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand(Unknown Source)
        ... 19 more

2025-03-06 17:16:59,427Z ERROR [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [70460fc8-7d40-4153-a387-f18a06641f6c] Ending command 'org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand' with failure.
2025-03-06 17:16:59,427Z ERROR [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [70460fc8-7d40-4153-a387-f18a06641f6c] Failed to transfer disk '00000000-0000-0000-0000-000000000000' for image transfer '0e3822a0-68e8-4c51-ac86-b6a433649183'
2025-03-06 17:16:59,459Z INFO  [org.ovirt.engine.core.bll.storage.disk.RemoveDiskCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [45a61a6a] Running command: RemoveDiskCommand internal: true. Entities affected :  ID: 22372bb7-9d9b-4db2-927d-1820917fd6c5 Type: DiskAction group DELETE_DISK with role type USER
2025-03-06 17:16:59,505Z INFO  [org.ovirt.engine.core.bll.storage.disk.image.RemoveImageCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [45a61a6a] Running command: RemoveImageCommand internal: true. Entities affected :  ID: 5fda4199-06c3-4d36-a6f2-be9791972c3b Type: Storage
2025-03-06 17:16:59,586Z INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-90) [45a61a6a] START, DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{storagePoolId='5c316ba3-d868-4314-917c-da0cc53b577e', ignoreFailoverLimit='false', storageDomainId='5fda4199-06c3-4d36-a6f2-be9791972c3b', imageGroupId='22372bb7-9d9b-4db2-927d-1820917fd6c5', postZeros='false', discard='false', forceDelete='false'}), log id: 5e1bb690
2025-03-06 17:16:59,958Z INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-T

-----------------------------------------------------------------------------------------------------------------------------------------

And from image io log

25-03-06 17:14:03,483 INFO    (Thread-2) [http] OPEN connection=2 client=::ffff:10.88.1.105
2025-03-06 17:15:03,511 WARNING (Thread-2) [http] Timeout reading or writing to socket: The read operation timed out
2025-03-06 17:15:03,511 INFO    (Thread-2) [http] CLOSE connection=2 client=::ffff:10.88.1.105 [connection 1 ops, 60.028099 s] [dispatch 1 ops, 0.000126 s]
2025-03-06 17:16:48,303 INFO    (Thread-3) [http] OPEN connection=3 client=::ffff:10.88.1.105
2025-03-06 17:17:48,306 WARNING (Thread-3) [http] Timeout reading or writing to socket: The read operation timed out
2025-03-06 17:17:48,307 INFO    (Thread-3) [http] CLOSE connection=3 client=::ffff:10.88.1.105 [connection 1 ops, 60.003327 s] [dispatch 1 ops, 0.000124 s]
-----------------------------------------------------------------------------------------------------------------

Has anyone got any advice - pretty sure this used to work on all domains ..

Re: [External] : Really weird ! Image IO / browser upload only working on 1 cluster (out of many) - anyone got any advice ? (network access, certs seem fine.)

morgan cox

tags

participants (1)