9:23 a.m.
I'm having trouble to create a storage ISO Domain and attach it to a Datacenter. It
just give me this error message:
Error while executing action Attach Storage Domain: Could not obtain lock
Also the oVirt's Engine log files show this error message: "setsid: failed to
execute /usr/bin/ionice: Permission denied", but I was unable to identify what
exactly it's trying to do to get this permission denied.
2019-11-14 16:46:07,779-03 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engine-Thread-7388) [86161370-2aaa-4eff-9aab-c184bdf5bb98]
EVENT_ID: IRS_BROKER_COMMAND_FAILURE(10,803), VDSM command AttachStorageDomainVDS failed:
Cannot obtain lock: u"id=e6b34c42-0ca6-41f4-be3e-3c9b2af1747b, rc=1, out=[],
err=['setsid: failed to execute /usr/bin/ionice: Permission denied']"
This behavior just happens on ISO Domains, while Data Domains works fine. I have read
oVirt documentation and searched everywhere but I was unable to find the solution for this
issue.
I'm using CentOS 7 with last update of all packages (oVirt version 4.3.6.7). Please
help!
Thanks,
Ivan de Gusmão Apolonio
3:26 a.m.
Hi Ivan,
can you please provide ovirt engine log and vdsm logs?
On Sun, Nov 17, 2019 at 9:19 AM Ivan de Gusmão Apolonio <
ivan(a)apolonio.com.br> wrote:
--
Ahmad Khiet
Red Hat <https://www.redhat.com/>
akhiet(a)redhat.com
M: +972-54-6225629
<https://red.ht/sig>
12:08 p.m.
When trying to attach ISO Domain to Datacenter, the following lines are populated on file
/var/log/ovirt-engine/engine.log:
2019-11-17 15:02:44,269-03 INFO
[org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand] (default
task-15) [f3795b36-ac1b-4115-9cfd-69e1b13783c9] Lock Acquired to object
'EngineLock:{exclusiveLocks='[e6b34c42-0ca6-41f4-be3e-3c9b2af1747b=STORAGE]',
sharedLocks=''}'
2019-11-17 15:02:44,301-03 INFO
[org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand]
(EE-ManagedThreadFactory-engine-Thread-138287) [f3795b36-ac1b-4115-9cfd-69e1b13783c9]
Running command: AttachStorageDomainToPoolCommand internal: false. Entities affected :
ID: e6b34c42-0ca6-41f4-be3e-3c9b2af1747b Type: StorageAction group
MANIPULATE_STORAGE_DOMAIN with role type ADMIN, ID: dd2a11c0-f450-11e9-8e3b-0050568ac2b9
Type: StoragePoolAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN
2019-11-17 15:02:44,315-03 INFO
[org.ovirt.engine.core.bll.storage.connection.ConnectStorageToVdsCommand]
(EE-ManagedThreadFactory-engine-Thread-138289) [2c0665e4] Running command:
ConnectStorageToVdsCommand internal: true. Entities affected : ID:
aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with
role type ADMIN
2019-11-17 15:02:44,315-03 INFO
[org.ovirt.engine.core.bll.storage.connection.ConnectStorageToVdsCommand]
(EE-ManagedThreadFactory-engine-Thread-138291) [40c539bc] Running command:
ConnectStorageToVdsCommand internal: true. Entities affected : ID:
aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with
role type ADMIN
2019-11-17 15:02:44,318-03 INFO
[org.ovirt.engine.core.bll.storage.connection.ConnectStorageToVdsCommand]
(EE-ManagedThreadFactory-engine-Thread-138290) [7b359087] Running command:
ConnectStorageToVdsCommand internal: true. Entities affected : ID:
aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with
role type ADMIN
2019-11-17 15:02:44,319-03 INFO
[org.ovirt.engine.core.bll.storage.connection.ConnectStorageToVdsCommand]
(EE-ManagedThreadFactory-engine-Thread-138288) [505dd498] Running command:
ConnectStorageToVdsCommand internal: true. Entities affected : ID:
aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with
role type ADMIN
2019-11-17 15:02:44,321-03 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-138291) [40c539bc] START,
ConnectStorageServerVDSCommand(HostName = ZeLele,
StorageServerConnectionManagementVDSParameters:{hostId='86198321-38ff-4596-8c24-2f9442928009',
storagePoolId='00000000-0000-0000-0000-000000000000', storageType='NFS',
connectionList='[StorageServerConnections:{id='ff0b050c-1b20-41fb-8373-32d9991a327e',
connection='nholau:/storage/iso', iqn='null', vfsType='null',
mountOptions='null', nfsVersion='AUTO', nfsRetrans='null',
nfsTimeo='null', iface='null', netIfaceName='null'}]',
sendNetworkEventOnFailure='true'}), log id: 742a0d4d
2019-11-17 15:02:44,338-03 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-138289) [2c0665e4] START,
ConnectStorageServerVDSCommand(HostName = ChicoBento,
StorageServerConnectionManagementVDSParameters:{hostId='c5018534-42b6-497a-8107-91493193d088',
storagePoolId='00000000-0000-0000-0000-000000000000', storageType='NFS',
connectionList='[StorageServerConnections:{id='ff0b050c-1b20-41fb-8373-32d9991a327e',
connection='nholau:/storage/iso', iqn='null', vfsType='null',
mountOptions='null', nfsVersion='AUTO', nfsRetrans='null',
nfsTimeo='null', iface='null', netIfaceName='null'}]',
sendNetworkEventOnFailure='true'}), log id: 65f576b9
2019-11-17 15:02:44,339-03 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-138290) [7b359087] START,
ConnectStorageServerVDSCommand(HostName = NhoLau,
StorageServerConnectionManagementVDSParameters:{hostId='0fc23e51-3e84-46df-9fd6-e97d83d32040',
storagePoolId='00000000-0000-0000-0000-000000000000', storageType='NFS',
connectionList='[StorageServerConnections:{id='ff0b050c-1b20-41fb-8373-32d9991a327e',
connection='nholau:/storage/iso', iqn='null', vfsType='null',
mountOptions='null', nfsVersion='AUTO', nfsRetrans='null',
nfsTimeo='null', iface='null', netIfaceName='null'}]',
sendNetworkEventOnFailure='true'}), log id: 6ed52cef
2019-11-17 15:02:44,344-03 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-138288) [505dd498] START,
ConnectStorageServerVDSCommand(HostName = Rosinha,
StorageServerConnectionManagementVDSParameters:{hostId='a838a3d5-8f52-44cd-a293-e530692814a3',
storagePoolId='00000000-0000-0000-0000-000000000000', storageType='NFS',
connectionList='[StorageServerConnections:{id='ff0b050c-1b20-41fb-8373-32d9991a327e',
connection='nholau:/storage/iso', iqn='null', vfsType='null',
mountOptions='null', nfsVersion='AUTO', nfsRetrans='null',
nfsTimeo='null', iface='null', netIfaceName='null'}]',
sendNetworkEventOnFailure='true'}), log id: 796b2168
2019-11-17 15:02:44,348-03 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-138291) [40c539bc] FINISH,
ConnectStorageServerVDSCommand, return: {ff0b050c-1b20-41fb-8373-32d9991a327e=0}, log id:
742a0d4d
2019-11-17 15:02:44,363-03 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-138289) [2c0665e4] FINISH,
ConnectStorageServerVDSCommand, return: {ff0b050c-1b20-41fb-8373-32d9991a327e=0}, log id:
65f576b9
2019-11-17 15:02:44,365-03 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-138290) [7b359087] FINISH,
ConnectStorageServerVDSCommand, return: {ff0b050c-1b20-41fb-8373-32d9991a327e=0}, log id:
6ed52cef
2019-11-17 15:02:44,368-03 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-138288) [505dd498] FINISH,
ConnectStorageServerVDSCommand, return: {ff0b050c-1b20-41fb-8373-32d9991a327e=0}, log id:
796b2168
2019-11-17 15:02:44,371-03 INFO
[org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-138287) [f3795b36-ac1b-4115-9cfd-69e1b13783c9]
START, AttachStorageDomainVDSCommand(
AttachStorageDomainVDSCommandParameters:{storagePoolId='dd2a11c0-f450-11e9-8e3b-0050568ac2b9',
ignoreFailoverLimit='false',
storageDomainId='e6b34c42-0ca6-41f4-be3e-3c9b2af1747b'}), log id: 133801f9
2019-11-17 15:02:44,766-03 ERROR
[org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-138287) [f3795b36-ac1b-4115-9cfd-69e1b13783c9]
Failed in 'AttachStorageDomainVDS' method
2019-11-17 15:02:44,813-03 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engine-Thread-138287) [f3795b36-ac1b-4115-9cfd-69e1b13783c9]
EVENT_ID: IRS_BROKER_COMMAND_FAILURE(10,803), VDSM command AttachStorageDomainVDS failed:
Cannot obtain lock: u"id=e6b34c42-0ca6-41f4-be3e-3c9b2af1747b, rc=1, out=[],
err=['setsid: failed to execute /usr/bin/ionice: Permission denied']"
2019-11-17 15:02:44,813-03 ERROR
[org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-138287) [f3795b36-ac1b-4115-9cfd-69e1b13783c9]
Command 'AttachStorageDomainVDSCommand(
AttachStorageDomainVDSCommandParameters:{storagePoolId='dd2a11c0-f450-11e9-8e3b-0050568ac2b9',
ignoreFailoverLimit='false',
storageDomainId='e6b34c42-0ca6-41f4-be3e-3c9b2af1747b'})' execution failed:
IRSGenericException: IRSErrorException: Failed to AttachStorageDomainVDS, error = Cannot
obtain lock: u"id=e6b34c42-0ca6-41f4-be3e-3c9b2af1747b, rc=1, out=[],
err=['setsid: failed to execute /usr/bin/ionice: Permission denied']", code =
651
2019-11-17 15:02:44,813-03 INFO
[org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-138287) [f3795b36-ac1b-4115-9cfd-69e1b13783c9]
FINISH, AttachStorageDomainVDSCommand, return: , log id: 133801f9
2019-11-17 15:02:44,813-03 ERROR
[org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand]
(EE-ManagedThreadFactory-engine-Thread-138287) [f3795b36-ac1b-4115-9cfd-69e1b13783c9]
Command
'org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand'
failed: EngineException:
org.ovirt.engine.core.vdsbroker.irsbroker.IrsOperationFailedNoFailoverException:
IRSGenericException: IRSErrorException: Failed to AttachStorageDomainVDS, error = Cannot
obtain lock: u"id=e6b34c42-0ca6-41f4-be3e-3c9b2af1747b, rc=1, out=[],
err=['setsid: failed to execute /usr/bin/ionice: Permission denied']", code =
651 (Failed with error AcquireLockFailure and code 651)
2019-11-17 15:02:44,815-03 INFO [org.ovirt.engine.core.bll.CommandCompensator]
(EE-ManagedThreadFactory-engine-Thread-138287) [f3795b36-ac1b-4115-9cfd-69e1b13783c9]
Command [id=37e3914f-d199-4578-8ba5-278adfd0f06f]: Compensating NEW_ENTITY_ID of
org.ovirt.engine.core.common.businessentities.StoragePoolIsoMap; snapshot:
StoragePoolIsoMapId:{storagePoolId='dd2a11c0-f450-11e9-8e3b-0050568ac2b9',
storageId='e6b34c42-0ca6-41f4-be3e-3c9b2af1747b'}.
2019-11-17 15:02:44,819-03 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engine-Thread-138287) [f3795b36-ac1b-4115-9cfd-69e1b13783c9]
EVENT_ID: USER_ATTACH_STORAGE_DOMAIN_TO_POOL_FAILED(963), Failed to attach Storage Domain
iso-wise to Data Center Wise. (User: ivan@wct.local(a)wct.local-authz)
2019-11-17 15:02:44,823-03 INFO
[org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand]
(EE-ManagedThreadFactory-engine-Thread-138287) [f3795b36-ac1b-4115-9cfd-69e1b13783c9] Lock
freed to object
'EngineLock:{exclusiveLocks='[e6b34c42-0ca6-41f4-be3e-3c9b2af1747b=STORAGE]',
sharedLocks=''}'
About /var/log/vdsm.vdsm.log, nothing special is logged besides these lines:
2019-11-17 15:04:37,304-0300 INFO (periodic/1) [vdsm.api] START repoStats(domains=())
from=internal, task_id=b298b588-c81e-4c50-804d-3ae24fc74380 (api:48)
2019-11-17 15:04:37,304-0300 INFO (periodic/1) [vdsm.api] FINISH repoStats return={}
from=internal, task_id=b298b588-c81e-4c50-804d-3ae24fc74380 (api:54)
2019-11-17 15:04:38,949-0300 INFO (jsonrpc/6) [api.host] START getAllVmStats()
from=::ffff:127.0.0.1,46370 (api:48)
2019-11-17 15:04:38,949-0300 INFO (jsonrpc/6) [api.host] FINISH getAllVmStats
return={'status': {'message': 'Done', 'code': 0},
'statsList': (suppressed)} from=::ffff:127.0.0.1,46370 (api:54)
10:36 a.m.
Follow the content of file /var/log/ovirt-engine/engine.log when I'm trying to attach
ISO Domain to Datacenter:
2019-11-18 13:32:16,146-03 INFO
[org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand] (default
task-26) [f5555798-4334-40ca-ba43-e97e4be396db] Lock Acquired to object
'EngineLock:{exclusiveLocks='[e6b34c42-0ca6-41f4-be3e-3c9b2af1747b=STORAGE]',
sharedLocks=''}'
2019-11-18 13:32:16,241-03 INFO
[org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand]
(EE-ManagedThreadFactory-engine-Thread-178210) [f5555798-4334-40ca-ba43-e97e4be396db]
Running command: AttachStorageDomainToPoolCommand internal: false. Entities affected :
ID: e6b34c42-0ca6-41f4-be3e-3c9b2af1747b Type: StorageAction group
MANIPULATE_STORAGE_DOMAIN with role type ADMIN, ID: dd2a11c0-f450-11e9-8e3b-0050568ac2b9
Type: StoragePoolAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN
2019-11-18 13:32:16,261-03 INFO
[org.ovirt.engine.core.bll.storage.connection.ConnectStorageToVdsCommand]
(EE-ManagedThreadFactory-engine-Thread-178211) [20e44128] Running command:
ConnectStorageToVdsCommand internal: true. Entities affected : ID:
aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with
role type ADMIN
2019-11-18 13:32:16,261-03 INFO
[org.ovirt.engine.core.bll.storage.connection.ConnectStorageToVdsCommand]
(EE-ManagedThreadFactory-engine-Thread-178212) [a7f5e93] Running command:
ConnectStorageToVdsCommand internal: true. Entities affected : ID:
aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with
role type ADMIN
2019-11-18 13:32:16,266-03 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-178212) [a7f5e93] START,
ConnectStorageServerVDSCommand(HostName = ZeLele,
StorageServerConnectionManagementVDSParameters:{hostId='86198321-38ff-4596-8c24-2f9442928009',
storagePoolId='00000000-0000-0000-0000-000000000000', storageType='NFS',
connectionList='[StorageServerConnections:{id='ff0b050c-1b20-41fb-8373-32d9991a327e',
connection='nholau:/storage/iso', iqn='null', vfsType='null',
mountOptions='null', nfsVersion='AUTO', nfsRetrans='null',
nfsTimeo='null', iface='null', netIfaceName='null'}]',
sendNetworkEventOnFailure='true'}), log id: 2c81e6d5
2019-11-18 13:32:16,266-03 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-178211) [20e44128] START,
ConnectStorageServerVDSCommand(HostName = Rosinha,
StorageServerConnectionManagementVDSParameters:{hostId='a838a3d5-8f52-44cd-a293-e530692814a3',
storagePoolId='00000000-0000-0000-0000-000000000000', storageType='NFS',
connectionList='[StorageServerConnections:{id='ff0b050c-1b20-41fb-8373-32d9991a327e',
connection='nholau:/storage/iso', iqn='null', vfsType='null',
mountOptions='null', nfsVersion='AUTO', nfsRetrans='null',
nfsTimeo='null', iface='null', netIfaceName='null'}]',
sendNetworkEventOnFailure='true'}), log id: 482f56e6
2019-11-18 13:32:16,385-03 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-178211) [20e44128] FINISH,
ConnectStorageServerVDSCommand, return: {ff0b050c-1b20-41fb-8373-32d9991a327e=0}, log id:
482f56e6
2019-11-18 13:32:16,385-03 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-178212) [a7f5e93] FINISH,
ConnectStorageServerVDSCommand, return: {ff0b050c-1b20-41fb-8373-32d9991a327e=0}, log id:
2c81e6d5
2019-11-18 13:32:16,387-03 INFO
[org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-178210) [f5555798-4334-40ca-ba43-e97e4be396db]
START, AttachStorageDomainVDSCommand(
AttachStorageDomainVDSCommandParameters:{storagePoolId='dd2a11c0-f450-11e9-8e3b-0050568ac2b9',
ignoreFailoverLimit='false',
storageDomainId='e6b34c42-0ca6-41f4-be3e-3c9b2af1747b'}), log id: 2263565b
2019-11-18 13:32:16,801-03 ERROR
[org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-178210) [f5555798-4334-40ca-ba43-e97e4be396db]
Failed in 'AttachStorageDomainVDS' method
2019-11-18 13:32:16,838-03 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engine-Thread-178210) [f5555798-4334-40ca-ba43-e97e4be396db]
EVENT_ID: IRS_BROKER_COMMAND_FAILURE(10,803), VDSM command AttachStorageDomainVDS failed:
Cannot obtain lock: u"id=e6b34c42-0ca6-41f4-be3e-3c9b2af1747b, rc=1, out=[],
err=['setsid: failed to execute /usr/bin/ionice: Permission denied']"
2019-11-18 13:32:16,838-03 ERROR
[org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-178210) [f5555798-4334-40ca-ba43-e97e4be396db]
Command 'AttachStorageDomainVDSCommand(
AttachStorageDomainVDSCommandParameters:{storagePoolId='dd2a11c0-f450-11e9-8e3b-0050568ac2b9',
ignoreFailoverLimit='false',
storageDomainId='e6b34c42-0ca6-41f4-be3e-3c9b2af1747b'})' execution failed:
IRSGenericException: IRSErrorException: Failed to AttachStorageDomainVDS, error = Cannot
obtain lock: u"id=e6b34c42-0ca6-41f4-be3e-3c9b2af1747b, rc=1, out=[],
err=['setsid: failed to execute /usr/bin/ionice: Permission denied']", code =
651
2019-11-18 13:32:16,838-03 INFO
[org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-178210) [f5555798-4334-40ca-ba43-e97e4be396db]
FINISH, AttachStorageDomainVDSCommand, return: , log id: 2263565b
2019-11-18 13:32:16,838-03 ERROR
[org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand]
(EE-ManagedThreadFactory-engine-Thread-178210) [f5555798-4334-40ca-ba43-e97e4be396db]
Command
'org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand'
failed: EngineException:
org.ovirt.engine.core.vdsbroker.irsbroker.IrsOperationFailedNoFailoverException:
IRSGenericException: IRSErrorException: Failed to AttachStorageDomainVDS, error = Cannot
obtain lock: u"id=e6b34c42-0ca6-41f4-be3e-3c9b2af1747b, rc=1, out=[],
err=['setsid: failed to execute /usr/bin/ionice: Permission denied']", code =
651 (Failed with error AcquireLockFailure and code 651)
2019-11-18 13:32:16,839-03 INFO [org.ovirt.engine.core.bll.CommandCompensator]
(EE-ManagedThreadFactory-engine-Thread-178210) [f5555798-4334-40ca-ba43-e97e4be396db]
Command [id=e38e591b-f48d-4aff-959e-e3470d73cead]: Compensating NEW_ENTITY_ID of
org.ovirt.engine.core.common.businessentities.StoragePoolIsoMap; snapshot:
StoragePoolIsoMapId:{storagePoolId='dd2a11c0-f450-11e9-8e3b-0050568ac2b9',
storageId='e6b34c42-0ca6-41f4-be3e-3c9b2af1747b'}.
2019-11-18 13:32:16,844-03 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engine-Thread-178210) [f5555798-4334-40ca-ba43-e97e4be396db]
EVENT_ID: USER_ATTACH_STORAGE_DOMAIN_TO_POOL_FAILED(963), Failed to attach Storage Domain
iso-wise to Data Center Wise. (User: ivan@wct.local(a)wct.local-authz)
2019-11-18 13:32:16,847-03 INFO
[org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand]
(EE-ManagedThreadFactory-engine-Thread-178210) [f5555798-4334-40ca-ba43-e97e4be396db] Lock
freed to object
'EngineLock:{exclusiveLocks='[e6b34c42-0ca6-41f4-be3e-3c9b2af1747b=STORAGE]',
sharedLocks=''}'
In other hand, file /var/log/vdsm/vdsm.log didn't show anything related to this
issue:
2019-11-18 13:32:15,014-0300 INFO (periodic/0) [vdsm.api] START repoStats(domains=())
from=internal, task_id=73ae4a01-397e-4b66-a481-a422ccd570b4 (api:48)
2019-11-18 13:32:15,014-0300 INFO (periodic/0) [vdsm.api] FINISH repoStats return={}
from=internal, task_id=73ae4a01-397e-4b66-a481-a422ccd570b4 (api:54)
2019-11-18 13:32:20,604-0300 INFO (jsonrpc/7) [api.host] START getAllVmStats()
from=::ffff:127.0.0.1,46370 (api:48)
2019-11-18 13:32:20,605-0300 INFO (jsonrpc/7) [api.host] FINISH getAllVmStats
return={'status': {'message': 'Done', 'code': 0},
'statsList': (suppressed)} from=::ffff:127.0.0.1,46370 (api:54)
2019-11-18 13:32:20,605-0300 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call
Host.getAllVmStats succeeded in 0.00 seconds (__init__:312)
6:51 a.m.
I've posted the logs asked. Do you have any clue about what's wrong? I tried
Downloading the official Ovirt-Node ISO and the same thing happens :-(
7:20 a.m.
Further system information:
oVirt installed packets:
$ rpm -qa '*ovirt*'
ovirt-engine-restapi-4.3.6.7-1.el7.noarch
ovirt-web-ui-1.6.0-1.el7.noarch
ovirt-vmconsole-proxy-1.0.7-2.el7.noarch
python2-ovirt-host-deploy-1.8.4-1.el7.noarch
ovirt-host-4.3.5-1.el7.x86_64
ovirt-engine-dwh-4.3.6-1.el7.noarch
cockpit-machines-ovirt-195.1-1.el7.centos.0.1.noarch
ovirt-ansible-repositories-1.1.5-1.el7.noarch
ovirt-ansible-roles-1.1.7-1.el7.noarch
python2-ovirt-engine-lib-4.3.7.2-1.el7.noarch
ovirt-provider-ovn-1.2.27-1.el7.noarch
ovirt-engine-setup-plugin-ovirt-engine-common-4.3.7.2-1.el7.noarch
ovirt-engine-setup-4.3.7.2-1.el7.noarch
ovirt-ansible-cluster-upgrade-1.1.14-1.el7.noarch
ovirt-cockpit-sso-0.1.1-1.el7.noarch
ovirt-engine-ui-extensions-1.0.10-1.el7.noarch
ovirt-engine-4.3.6.7-1.el7.noarch
ovirt-hosted-engine-setup-2.3.12-1.el7.noarch
ovirt-ansible-vm-infra-1.1.22-1.el7.noarch
ovirt-engine-setup-plugin-cinderlib-4.3.7.2-1.el7.noarch
python-ovirt-engine-sdk4-4.3.2-2.el7.x86_64
ovirt-release43-4.3.7-1.el7.noarch
python2-ovirt-setup-lib-1.2.0-1.el7.noarch
ovirt-engine-wildfly-overlay-17.0.1-1.el7.noarch
ovirt-imageio-proxy-1.5.2-0.el7.noarch
ovirt-engine-sdk-python-3.6.9.1-1.el7.noarch
ovirt-vmconsole-1.0.7-2.el7.noarch
ovirt-engine-wildfly-17.0.1-1.el7.x86_64
ovirt-ansible-image-template-1.1.12-1.el7.noarch
ovirt-ansible-manageiq-1.1.14-1.el7.noarch
ovirt-engine-tools-4.3.6.7-1.el7.noarch
ovirt-engine-dbscripts-4.3.6.7-1.el7.noarch
cockpit-ovirt-dashboard-0.13.8-1.el7.noarch
ovirt-engine-extension-aaa-ldap-1.3.10-1.el7.noarch
ovirt-engine-setup-plugin-ovirt-engine-4.3.7.2-1.el7.noarch
ovirt-host-deploy-java-1.8.4-1.el7.noarch
ovirt-hosted-engine-ha-2.3.6-1.el7.noarch
ovirt-engine-metrics-1.3.5.1-1.el7.noarch
ovirt-engine-extensions-api-impl-4.3.7.2-1.el7.noarch
ovirt-engine-dwh-setup-4.3.6-1.el7.noarch
ovirt-engine-api-explorer-0.0.5-1.el7.noarch
ovirt-host-deploy-common-1.8.4-1.el7.noarch
ovirt-engine-setup-plugin-vmconsole-proxy-helper-4.3.7.2-1.el7.noarch
ovirt-engine-vmconsole-proxy-helper-4.3.7.2-1.el7.noarch
ovirt-engine-tools-backup-4.3.6.7-1.el7.noarch
ovirt-ansible-engine-setup-1.1.9-1.el7.noarch
ovirt-ansible-shutdown-env-1.0.3-1.el7.noarch
ovirt-engine-webadmin-portal-4.3.6.7-1.el7.noarch
ovirt-imageio-daemon-1.5.2-0.el7.noarch
ovirt-vmconsole-host-1.0.7-2.el7.noarch
ovirt-engine-setup-plugin-websocket-proxy-4.3.7.2-1.el7.noarch
ovirt-host-dependencies-4.3.5-1.el7.x86_64
ovirt-ansible-hosted-engine-setup-1.0.32-1.el7.noarch
ovirt-imageio-proxy-setup-1.5.2-0.el7.noarch
ovirt-iso-uploader-4.3.2-1.el7.noarch
ovirt-ansible-disaster-recovery-1.2.0-1.el7.noarch
ovirt-engine-extension-aaa-jdbc-1.1.10-1.el7.noarch
ovirt-engine-backend-4.3.6.7-1.el7.noarch
ovirt-engine-extension-aaa-ldap-setup-1.3.10-1.el7.noarch
ovirt-imageio-common-1.5.2-0.el7.x86_64
ovirt-engine-cli-3.6.9.2-1.el7.noarch
ovirt-provider-ovn-driver-1.2.27-1.el7.noarch
ovirt-engine-setup-base-4.3.7.2-1.el7.noarch
ovirt-engine-websocket-proxy-4.3.7.2-1.el7.noarch
ovirt-ansible-infra-1.1.13-1.el7.noarch
Kernel Version:
$ uname -r
3.10.0-1062.4.3.el7.x86_64
RedHat Version:
$ cat /etc/redhat-release
CentOS Linux release 7.7.1908 (Core)
6:49 a.m.
I'm unable to start VMs via command line without ISO domain. Please Help.
============= ERROR ============
status: 409
reason: Conflict
detail: Cannot run VM with attached CD without an active ISO domain.
Alternatives:
- Attach an ISO Domain (to enable CD operations).
- Change the boot sequence, using the Edit VM command (Boot Sub-Tab).
- Select a different boot device (using the 'Run Once' command).
2:56 p.m.
Can you edit the VM ?If yes, remove the DVD and try again.
В петък, 22 ноември 2019 г., 14:50:22 ч. Гринуич+2, Ivan Apolonio
<ivan(a)apolonio.com.br> написа:
I'm unable to start VMs via command line without ISO domain. Please Help.
============= ERROR ============
status: 409
reason: Conflict
detail: Cannot run VM with attached CD without an active ISO domain.
Alternatives:
- Attach an ISO Domain (to enable CD operations).
- Change the boot sequence, using the Edit VM command (Boot Sub-Tab).
- Select a different boot device (using the 'Run Once' command).
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/35EN6KIFPBM...
1:38 p.m.
On Sun, Nov 17, 2019 at 9:19 AM Ivan de Gusmão Apolonio <
ivan(a)apolonio.com.br> wrote:
ionice command runs in an elevated suid permissions by vdsm,
see that your host has /etc/sudoers.d/50_vdsm file with ionice command
entry and that /etc/sudoers has the #includedir /etc/sudoers.d directive.
2:07 p.m.
On Mon, Dec 2, 2019 at 9:38 PM Amit Bawer <abawer(a)redhat.com> wrote:
also make sure the nfs share used for the ISO domain has proper mount
options (no "no exec" option, with "no_root_squash" ) and
ownership/permission as you have probably set [1]
[1]
https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.2/...
8:15 p.m.
Hello Amit. Thanks for you reply.
This is the content of /etc/sudoers.d/50_vdsm file (it's the default generated by
ovirt install):
Cmnd_Alias VDSM_LIFECYCLE = \
/usr/sbin/dmidecode -s system-uuid
Cmnd_Alias VDSM_STORAGE = \
/usr/sbin/fsck -p *, \
/usr/sbin/tune2fs -j *, \
/usr/sbin/mkfs -q -j *, \
/usr/bin/kill, \
/usr/bin/chown vdsm\:qemu *, \
/usr/bin/chown vdsm\:kvm *, \
/usr/sbin/iscsiadm *, \
/usr/sbin/lvm, \
/usr/bin/setsid /usr/bin/ionice -c ? -n ? /usr/bin/su vdsm -s /bin/sh -c
/usr/libexec/vdsm/spmprotect.sh*, \
/usr/sbin/service vdsmd *, \
/usr/sbin/reboot -f
vdsm ALL=(ALL) NOPASSWD: VDSM_LIFECYCLE, VDSM_STORAGE
Defaults:vdsm !requiretty
Defaults:vdsm !syslog
I was pretty curious about the format of the line "/usr/bin/setsid /usr/bin/ionice -c
? -n ? /usr/bin/su vdsm -s /bin/sh -c /usr/libexec/vdsm/spmprotect.sh*", but looking
at source code (https://github.com/oVirt/vdsm/blob/master/static/etc/sudoers.d/50_vdsm.in)
it looks to be just like that. If I need to change anything on this file, it looks that
there's some bug on vdsm package.
In other hand, I watched the /var/log/secure file while I was trying to attach a
Datacenter to ISO Domain and it didn't showed anything new, meaning that the referred
"ionice" command was not executed via sudo by vdsm. If it's is true, that
could explain the "permission denied" error.
About the NFS export, it is exactly the same as parameters as Data Domain exports (which
works perfectly):
exportfs -v
/storage/vm
172.31.17.0/24(sync,wdelay,hide,no_subtree_check,sec=sys,rw,secure,root_squash,no_all_squash)
/storage/vm
172.31.48.0/24(sync,wdelay,hide,no_subtree_check,sec=sys,rw,secure,root_squash,no_all_squash)
/storage/iso
<world>(sync,wdelay,hide,no_subtree_check,sec=sys,rw,secure,root_squash,no_all_squash)
What else do I need to check?
Thanks
Ivan
11:33 p.m.
On Tuesday, December 3, 2019, Ivan Apolonio <ivan(a)apolonio.com.br> wrote:
This line shuts logging, worth to comment it out during check. Plus, do you
have an #includedir setting in your /etc/sudoers file?
The vdsm.log snippet seems later than the error in the engine.log, could
you provide one covering the failing attempt?
8:58 a.m.
Hello.
I tried to reinstall my whole environment using oficial oVirt ISO image and the same
problem occurs. I also tried to give full access to vdsm user in sudoers file and the
results are exactly the same. This is the error on showed on oVirt Manager:
VDSM command AttachStorageDomainVDS failed: Cannot obtain lock:
u"id=e65b4f0f-faa1-4cdc-9d2a-ab4fccfec0b0, rc=1, out=[], err=['setsid: failed to
execute /usr/bin/ionice: Permission denied']"
In other hand, the node server's vdsm.log file shows no errors.
The engine.log still shows the same errors as my previous posts with logs. I don't
know what else can I do.
Please help.
Thanks,
Ivan
10:39 a.m.
Hello, Amit.
It looks that commenting out that last line (Defaults:vdsm !syslog) did the trick to help
identify the problem. According to /var/log/secure log file, vdsm uid is being blocked to
sudo due to pam requirements:
Dec 4 10:53:36 Rosinha sudo: pam_unix(sudo:auth): authentication failure; logname=root
uid=36 euid=0 tty=/dev/pts/0 ruser=vdsm rhost= user=vdsm
Dec 4 10:53:36 Rosinha sudo: pam_succeed_if(sudo:auth): requirement "uid >=
1000" not met by user "vdsm"
Dec 4 10:58:38 Rosinha sudo: pam_unix(sudo:auth): conversation failed
Dec 4 10:58:38 Rosinha sudo: pam_unix(sudo:auth): auth could not identify password for
[vdsm]
Dec 4 10:58:38 Rosinha sudo: pam_succeed_if(sudo:auth): requirement "uid >=
1000" not met by user "vdsm"
This "uid >= 1000" requirement is the CentOS 7 default. What is the best way
to work around it? I'm asking that because if I just comment this rule on pam
configuration files, it is going to allow other system users to sudo, which would lead to
security issues.
Thanks,
Ivan
10:55 a.m.
I've tried to commenting out this line on /etc/pam.d/password-auth:
auth requisite pam_succeed_if.so uid >= 1000 quiet_success
However, although it has stopped complaining about "requirement uid >= 1000 not
met by user vdsm" on /var/log/secure , the error trying to attach ISO Domain to
Datacenter still persists, showing the same error message, so it looks that this was not
the cause of the problem :-(
Going back to stage zero.
11:43 a.m.
I was finally able to attach ISO Domain to Datacenter after moving all Datacenter's
Storage Domains and Hosts to MAINTAINANCE except on the Host where I was creating the ISO
Domain.
It looks like a bug on oVirt that didn't allow to simply add a ISO Domain after all
other hosts and Data Domains has been previously added to my Datacenter. However, I'm
glad I was finally able to attach the ISO Domain to my Datacenter.
Thanks for everyone assistance
2007
days inactive
2034
days old
18 comments
6 participants
participants (6)
-
Ahmad Khiet -
Amit Bawer -
Ivan Apolonio -
Ivan de Gusmão Apolonio
-
ivan＠apolonio.com.br
-
Strahil Nikolov