[ovirt-users] Node issues

Sun Nov 30 14:47:27 UTC 2014

Hi Juan.
it seems as you have connectivity issues with the storage:

> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (DefaultQuartzScheduler_Worker-79) [be7203c] Correlation ID: null, Call
> Stack: null, Custom Event ID: -1, Message: Failed to connect Host node3 to
> the Storage Domains node1_image.

And-

> 2014-11-29 23:40:05,691 ERROR
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (DefaultQuartzScheduler_Worker-79) [be7203c] Correlation ID: null, Call
> Stack: null, Custom Event ID: -1, Message: The error message for connection
> node1.monalisa.com.py:/nfs/image returned by VDSM was: Problem while trying
> to mount target
> 2014-11-29 23:40:05,695 ERROR
> [org.ovirt.engine.core.bll.storage.NFSStorageHelper]
> (DefaultQuartzScheduler_Worker-79) [be7203c] The connection with details
> node1.monalisa.com.py:/nfs/image failed because of error code 477 and error
> message is: problem while trying to mount target

You should verify your configuration first. If it looks fine consider selinux
issues as well.

Doron

----- Original Message -----
> From: "Juan Carlos YJ. Lin" <lin at unisoft.com.py>
> To: users at ovirt.org
> Sent: Sunday, November 30, 2014 4:57:30 AM
> Subject: [ovirt-users] Node issues
> 
> OVirt 3.5 hosted, installed new node, centos 6.5 minumal, updated, ovirt3.5
> repos,
> 
> with ovirt interface, create on default datacenter try to add second host
> the system installs and then occurs error
> 
> the host cannot be up
> 
> engine.log
> 
> 2014-11-29 23:37:57,719 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand]
> (org.ovirt.thread.pool-8-thread-48) Activate finished. Lock released.
> Monitoring can run now for host node3 from data-center Default
> 2014-11-29 23:37:57,730 INFO
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (org.ovirt.thread.pool-8-thread-48) Correlation ID: 7cf0e41a, Job ID:
> f4e3d0cb-e7f7-4c91-8fcd-f3323848d53a, Call Stack: null, Custom Event ID: -1,
> Message: Host node3 was activated by admin.
> 2014-11-29 23:37:57,762 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand]
> (org.ovirt.thread.pool-8-thread-48) Lock freed to object EngineLock
> [exclusiveLocks= key: 447f5911-07b7-4b10-9eb6-15a5e06e535a value: VDS
> , sharedLocks= ]
> 2014-11-29 23:38:00,369 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand]
> (DefaultQuartzScheduler_Worker-79) START, GetHardwareInfoVDSCommand(HostName
> = node3, HostId = 447f5911-07b7-4b10-9eb6-15a5e06e535a,
> vds=Host[node3,447f5911-07b7-4b10-9eb6-15a5e06e535a]), log id: 737e2224
> 2014-11-29 23:38:00,382 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand]
> (DefaultQuartzScheduler_Worker-79) FINISH, GetHardwareInfoVDSCommand, log
> id: 737e2224
> 2014-11-29 23:38:00,417 INFO
> [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand]
> (DefaultQuartzScheduler_Worker-79) [d948749] Running command:
> HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected :
> ID: 447f5911-07b7-4b10-9eb6-15a5e06e535a Type: VDS
> 2014-11-29 23:38:00,431 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand]
> (DefaultQuartzScheduler_Worker-79) [2abcada9] Running command:
> InitVdsOnUpCommand internal: true. Entities affected : ID:
> 00000002-0002-0002-0002-000000000101 Type: StoragePool
> 2014-11-29 23:38:00,445 INFO
> [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand]
> (DefaultQuartzScheduler_Worker-79) [be7203c] Running command:
> ConnectHostToStoragePoolServersCommand internal: true. Entities affected :
> ID: 00000002-0002-0002-0002-000000000101 Type: StoragePool
> 2014-11-29 23:38:00,453 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]
> (DefaultQuartzScheduler_Worker-79) [be7203c] START,
> ConnectStorageServerVDSCommand(HostName = node3, HostId =
> 447f5911-07b7-4b10-9eb6-15a5e06e535a, storagePoolId =
> 00000002-0002-0002-0002-000000000101, storageType = NFS, connectionList = [{
> id: 2fc02401-37e2-4169-83e4-c302ac1e0eb2, connection:
> 192.168.1.90:/mnt/storage1/ovirt_share/exports, iqn: null, vfsType: null,
> mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{
> id: 4454ad4c-be93-49e6-ab77-87befc141b21, connection:
> node1.monalisa.com.py:/nfs/image, iqn: null, vfsType: null, mountOptions:
> null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id:
> 653cab22
> 2014-11-29 23:40:05,676 ERROR
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (DefaultQuartzScheduler_Worker-79) [be7203c] Correlation ID: null, Call
> Stack: null, Custom Event ID: -1, Message: Failed to connect Host node3 to
> the Storage Domains node1_image.
> 2014-11-29 23:40:05,678 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]
> (DefaultQuartzScheduler_Worker-79) [be7203c] FINISH,
> ConnectStorageServerVDSCommand, return:
> {4454ad4c-be93-49e6-ab77-87befc141b21=477,
> 2fc02401-37e2-4169-83e4-c302ac1e0eb2=0}, log id: 653cab22
> 2014-11-29 23:40:05,691 ERROR
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (DefaultQuartzScheduler_Worker-79) [be7203c] Correlation ID: null, Call
> Stack: null, Custom Event ID: -1, Message: The error message for connection
> node1.monalisa.com.py:/nfs/image returned by VDSM was: Problem while trying
> to mount target
> 2014-11-29 23:40:05,695 ERROR
> [org.ovirt.engine.core.bll.storage.NFSStorageHelper]
> (DefaultQuartzScheduler_Worker-79) [be7203c] The connection with details
> node1.monalisa.com.py:/nfs/image failed because of error code 477 and error
> message is: problem while trying to mount target
> 2014-11-29 23:40:05,697 INFO
> [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand]
> (DefaultQuartzScheduler_Worker-79) [be7203c] Host node3 storage connection
> was failed
> 2014-11-29 23:40:05,708 WARN
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (DefaultQuartzScheduler_Worker-79) [be7203c] Correlation ID: be7203c, Call
> Stack: null, Custom Event ID: -1, Message: Failed to connect Host node3 to
> Storage Servers
> 2014-11-29 23:40:05,725 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
> (org.ovirt.thread.pool-8-thread-19) START,
> ConnectStoragePoolVDSCommand(HostName = node3, HostId =
> 447f5911-07b7-4b10-9eb6-15a5e06e535a, vdsId =
> 447f5911-07b7-4b10-9eb6-15a5e06e535a, storagePoolId =
> 00000002-0002-0002-0002-000000000101, masterVersion = 1), log id: 596dfe5
> 2014-11-29 23:40:06,415 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
> (org.ovirt.thread.pool-8-thread-19) Command
> org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand
> return value
> StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=304,
> mMessage=Cannot find master domain:
> u'spUUID=00000002-0002-0002-0002-000000000101,
> msdUUID=e81ff04f-f960-4e0e-b43c-e8ba63bfe1e6']]
> 2014-11-29 23:40:06,418 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
> (org.ovirt.thread.pool-8-thread-19) HostName = node3
> 2014-11-29 23:40:06,421 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
> (org.ovirt.thread.pool-8-thread-19) Command
> ConnectStoragePoolVDSCommand(HostName = node3, HostId =
> 447f5911-07b7-4b10-9eb6-15a5e06e535a, vdsId =
> 447f5911-07b7-4b10-9eb6-15a5e06e535a, storagePoolId =
> 00000002-0002-0002-0002-000000000101, masterVersion = 1) execution failed.
> Exception: IRSNoMasterDomainException: IRSGenericException:
> IRSErrorException: IRSNoMasterDomainException: Cannot find master domain:
> u'spUUID=00000002-0002-0002-0002-000000000101,
> msdUUID=e81ff04f-f960-4e0e-b43c-e8ba63bfe1e6'
> 2014-11-29 23:40:06,435 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
> (org.ovirt.thread.pool-8-thread-19) FINISH, ConnectStoragePoolVDSCommand,
> log id: 596dfe5
> 2014-11-29 23:40:06,440 ERROR [org.ovirt.engine.core.bll.InitVdsOnUpCommand]
> (org.ovirt.thread.pool-8-thread-19) Could not connect host node3 to pool
> Default with the message: null
> 2014-11-29 23:40:06,478 INFO
> [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand]
> (DefaultQuartzScheduler_Worker-79) [50557c8e] Running command:
> SetNonOperationalVdsCommand internal: true. Entities affected : ID:
> 447f5911-07b7-4b10-9eb6-15a5e06e535a Type: VDS
> 2014-11-29 23:40:06,483 INFO
> [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand]
> (DefaultQuartzScheduler_Worker-79) [50557c8e] START,
> SetVdsStatusVDSCommand(HostName = node3, HostId =
> 447f5911-07b7-4b10-9eb6-15a5e06e535a, status=NonOperational,
> nonOperationalReason=STORAGE_DOMAIN_UNREACHABLE,
> stopSpmFailureLogged=false), log id: 71bcd327
> 2014-11-29 23:40:06,494 INFO
> [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand]
> (DefaultQuartzScheduler_Worker-79) [50557c8e] FINISH,
> SetVdsStatusVDSCommand, log id: 71bcd327
> 2014-11-29 23:40:06,508 WARN
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (DefaultQuartzScheduler_Worker-79) [50557c8e] Correlation ID: 50557c8e, Job
> ID: 114e8b35-4943-4865-819b-bbc53cca4401, Call Stack: null, Custom Event ID:
> -1, Message: Host node3 cannot access the Storage Domain(s) <UNKNOWN>
> attached to the Data Center Default. Setting Host state to Non-Operational.
> 2014-11-29 23:40:06,525 WARN
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (DefaultQuartzScheduler_Worker-79) [50557c8e] Correlation ID: null, Call
> Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management
> configuration for Host node3.
> 2014-11-29 23:40:06,533 WARN
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (DefaultQuartzScheduler_Worker-79) [50557c8e] Correlation ID: 2abcada9, Call
> Stack: null, Custom Event ID: -1, Message: Failed to connect Host node3 to
> Storage Pool Default
> 2014-11-29 23:40:06,554 INFO
> [org.ovirt.engine.core.bll.HandleVdsVersionCommand]
> (DefaultQuartzScheduler_Worker-79) [488b3d18] Running command:
> HandleVdsVersionCommand internal: true. Entities affected : ID:
> 447f5911-07b7-4b10-9eb6-15a5e06e535a Type: VDS
> 2014-11-29 23:40:06,562 INFO
> [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]
> (DefaultQuartzScheduler_Worker-79) [488b3d18] Host
> 447f5911-07b7-4b10-9eb6-15a5e06e535a : node3 is already in NonOperational
> status for reason STORAGE_DOMAIN_UNREACHABLE. SetNonOperationalVds command
> is skipped.
> 2014-11-29 23:45:00,018 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager]
> (DefaultQuartzScheduler_Worker-33) [3e873048] Autorecovering 1 hosts
> 2014-11-29 23:45:00,022 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager]
> (DefaultQuartzScheduler_Worker-33) [3e873048] Autorecovering hosts id:
> 447f5911-07b7-4b10-9eb6-15a5e06e535a, name : node3
> 2014-11-29 23:45:00,031 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand]
> (DefaultQuartzScheduler_Worker-33) [5cad5ff8] Lock Acquired to object
> EngineLock [exclusiveLocks= key: 447f5911-07b7-4b10-9eb6-15a5e06e535a value:
> VDS
> , sharedLocks= ]
> 2014-11-29 23:45:00,037 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand]
> (DefaultQuartzScheduler_Worker-33) [5cad5ff8] Running command:
> ActivateVdsCommand internal: true. Entities affected : ID:
> 447f5911-07b7-4b10-9eb6-15a5e06e535a Type: VDSAction group MANIPULATE_HOST
> with role type ADMIN
> 2014-11-29 23:45:00,045 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand]
> (DefaultQuartzScheduler_Worker-33) [5cad5ff8] Before acquiring lock in order
> to prevent monitoring for host node3 from data-center Default
> 2014-11-29 23:45:00,050 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand]
> (DefaultQuartzScheduler_Worker-33) [5cad5ff8] Lock acquired, from now a
> monitoring of host will be skipped for host node3 from data-center Default
> 2014-11-29 23:45:00,062 INFO
> [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand]
> (DefaultQuartzScheduler_Worker-33) [5cad5ff8] START,
> SetVdsStatusVDSCommand(HostName = node3, HostId =
> 447f5911-07b7-4b10-9eb6-15a5e06e535a, status=Unassigned,
> nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 4707f81f
> 2014-11-29 23:45:00,085 INFO
> [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand]
> (DefaultQuartzScheduler_Worker-33) [5cad5ff8] FINISH,
> SetVdsStatusVDSCommand, log id: 4707f81f
> 2014-11-29 23:45:00,136 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand]
> (DefaultQuartzScheduler_Worker-33) Activate finished. Lock released.
> Monitoring can run now for host node3 from data-center Default
> 2014-11-29 23:45:00,139 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand]
> (DefaultQuartzScheduler_Worker-33) Lock freed to object EngineLock
> [exclusiveLocks= key: 447f5911-07b7-4b10-9eb6-15a5e06e535a value: VDS
> , sharedLocks= ]
> 2014-11-29 23:45:01,072 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand]
> (DefaultQuartzScheduler_Worker-59) START, GetHardwareInfoVDSCommand(HostName
> = node3, HostId = 447f5911-07b7-4b10-9eb6-15a5e06e535a,
> vds=Host[node3,447f5911-07b7-4b10-9eb6-15a5e06e535a]), log id: 6ad31104
> 2014-11-29 23:45:01,085 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand]
> (DefaultQuartzScheduler_Worker-59) FINISH, GetHardwareInfoVDSCommand, log
> id: 6ad31104
> 2014-11-29 23:45:01,127 INFO
> [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand]
> (DefaultQuartzScheduler_Worker-59) [40186252] Running command:
> HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected :
> ID: 447f5911-07b7-4b10-9eb6-15a5e06e535a Type: VDS
> 2014-11-29 23:45:01,146 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand]
> (DefaultQuartzScheduler_Worker-59) [2f699171] Running command:
> InitVdsOnUpCommand internal: true. Entities affected : ID:
> 00000002-0002-0002-0002-000000000101 Type: StoragePool
> 2014-11-29 23:45:01,165 INFO
> [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand]
> (DefaultQuartzScheduler_Worker-59) [216861d0] Running command:
> ConnectHostToStoragePoolServersCommand internal: true. Entities affected :
> ID: 00000002-0002-0002-0002-000000000101 Type: StoragePool
> 2014-11-29 23:45:01,179 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]
> (DefaultQuartzScheduler_Worker-59) [216861d0] START,
> ConnectStorageServerVDSCommand(HostName = node3, HostId =
> 447f5911-07b7-4b10-9eb6-15a5e06e535a, storagePoolId =
> 00000002-0002-0002-0002-000000000101, storageType = NFS, connectionList = [{
> id: 2fc02401-37e2-4169-83e4-c302ac1e0eb2, connection:
> 192.168.1.90:/mnt/storage1/ovirt_share/exports, iqn: null, vfsType: null,
> mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{
> id: 4454ad4c-be93-49e6-ab77-87befc141b21, connection:
> node1.monalisa.com.py:/nfs/image, iqn: null, vfsType: null, mountOptions:
> null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id:
> 37185939
> 
> 
> Juan Carlos Lin
> Unisoft S.A.
> +595-993-288330
> 
> 
> System-wide Disclaimer ---------------------------------------------------
> "Antes de imprimir, recuérdese de su compromiso con el Medio Ambiente"
> "Aviso: Este mensaje es dirigido para su destinatario y contiene
> informaciones que no pueden ser usadas por otras personas que no sean su(s)
> destinatario(s). La retransmisión del contenido no está autorizada fuera del
> contexto de su envío y a quien corresponde. El uso no autorizado de la
> información en este mensaje se halla penado por las leyes vigentes en todo
> el mundo. Si ha recibido este mensaje por error, por favor bórrala y
> notifique al remitente en la brevedad posible. El contenido de este mensaje
> no es responsabilidad de la Empresa y debe ser atribuido siempre a su autor.
> Gracias."
> 
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
> 