4:09 a.m.
Hi,
After reboot my Hypervisior host can not activate again in the cluster and
failed with the following error :-
Gluster command [<UNKNOWN>] failed on server...
Engine logs :-
2014-11-24 18:05:28,397 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
(DefaultQuartzScheduler_Worker-64) START,
GlusterVolumesListVDSCommand(HostName = Compute4, HostId =
33648a90-200c-45ca-89d5-1ce305d79a6a), log id: 5f251c90
2014-11-24 18:05:30,609 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
(DefaultQuartzScheduler_Worker-64) FINISH, GlusterVolumesListVDSCommand,
return:
{26ae1672-ee09-4a38-8fd2-72dd9974cc2b=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@d95203e0},
log id: 5f251c90
2014-11-24 18:05:33,768 INFO
[org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp--127.0.0.1-8702-8)
[287d570d] Lock Acquired to object EngineLock [exclusiveLocks= key:
0bf6b00f-7947-4411-b55a-cc5eea2b381a value: VDS
, sharedLocks= ]
2014-11-24 18:05:33,795 INFO
[org.ovirt.engine.core.bll.ActivateVdsCommand]
(org.ovirt.thread.pool-8-thread-45) [287d570d] Running command:
ActivateVdsCommand internal: false. Entities affected : ID:
0bf6b00f-7947-4411-b55a-cc5eea2b381a Type: VDSAction group MANIPULATE_HOST
with role type ADMIN
2014-11-24 18:05:33,796 INFO
[org.ovirt.engine.core.bll.ActivateVdsCommand]
(org.ovirt.thread.pool-8-thread-45) [287d570d] Before acquiring lock in
order to prevent monitoring for host Compute5 from data-center SV_WTC
2014-11-24 18:05:33,797 INFO
[org.ovirt.engine.core.bll.ActivateVdsCommand]
(org.ovirt.thread.pool-8-thread-45) [287d570d] Lock acquired, from now a
monitoring of host will be skipped for host Compute5 from data-center SV_WTC
2014-11-24 18:05:33,817 INFO
[org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand]
(org.ovirt.thread.pool-8-thread-45) [287d570d] START,
SetVdsStatusVDSCommand(HostName = Compute5, HostId =
0bf6b00f-7947-4411-b55a-cc5eea2b381a, status=Unassigned,
nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 1cbc7311
2014-11-24 18:05:33,820 INFO
[org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand]
(org.ovirt.thread.pool-8-thread-45) [287d570d] FINISH,
SetVdsStatusVDSCommand, log id: 1cbc7311
2014-11-24 18:05:34,086 INFO
[org.ovirt.engine.core.bll.ActivateVdsCommand]
(org.ovirt.thread.pool-8-thread-45) Activate finished. Lock released.
Monitoring can run now for host Compute5 from data-center SV_WTC
2014-11-24 18:05:34,088 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(org.ovirt.thread.pool-8-thread-45) Correlation ID: 287d570d, Job ID:
5ef8e4d6-b2bc-469e-8e81-7ef74b2a001a, Call Stack: null, Custom Event ID:
-1, Message: Host Compute5 was activated by admin.
2014-11-24 18:05:34,090 INFO
[org.ovirt.engine.core.bll.ActivateVdsCommand]
(org.ovirt.thread.pool-8-thread-45) Lock freed to object EngineLock
[exclusiveLocks= key: 0bf6b00f-7947-4411-b55a-cc5eea2b381a value: VDS
, sharedLocks= ]
2014-11-24 18:05:35,792 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
(DefaultQuartzScheduler_Worker-55) [3706e836] START,
GlusterVolumesListVDSCommand(HostName = Compute4, HostId =
33648a90-200c-45ca-89d5-1ce305d79a6a), log id: 48a0c832
2014-11-24 18:05:37,064 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand]
(DefaultQuartzScheduler_Worker-69) START,
GetHardwareInfoVDSCommand(HostName = Compute5, HostId =
0bf6b00f-7947-4411-b55a-cc5eea2b381a,
vds=Host[Compute5,0bf6b00f-7947-4411-b55a-cc5eea2b381a]), log id: 6d560cc2
2014-11-24 18:05:37,074 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand]
(DefaultQuartzScheduler_Worker-69) FINISH, GetHardwareInfoVDSCommand, log
id: 6d560cc2
2014-11-24 18:05:37,093 WARN [org.ovirt.engine.core.vdsbroker.VdsManager]
(DefaultQuartzScheduler_Worker-69) Host Compute5 is running with disabled
SELinux.
2014-11-24 18:05:37,127 INFO
[org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand]
(DefaultQuartzScheduler_Worker-69) [2b4a51cf] Running command:
HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected
: ID: 0bf6b00f-7947-4411-b55a-cc5eea2b381a Type: VDS
2014-11-24 18:05:37,147 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand]
(DefaultQuartzScheduler_Worker-69) [2b4a51cf] START,
GlusterServersListVDSCommand(HostName = Compute5, HostId =
0bf6b00f-7947-4411-b55a-cc5eea2b381a), log id: 4faed87
2014-11-24 18:05:37,164 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterServersListVDSCommand]
(DefaultQuartzScheduler_Worker-69) [2b4a51cf] FINISH,
GlusterServersListVDSCommand, log id: 4faed87
2014-11-24 18:05:37,189 INFO
[org.ovirt.engine.core.bll.SetNonOperationalVdsCommand]
(DefaultQuartzScheduler_Worker-69) [4a84c4e5] Running command:
SetNonOperationalVdsCommand internal: true. Entities affected : ID:
0bf6b00f-7947-4411-b55a-cc5eea2b381a Type: VDS
2014-11-24 18:05:37,206 INFO
[org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand]
(DefaultQuartzScheduler_Worker-69) [4a84c4e5] START,
SetVdsStatusVDSCommand(HostName = Compute5, HostId =
0bf6b00f-7947-4411-b55a-cc5eea2b381a, status=NonOperational,
nonOperationalReason=GLUSTER_COMMAND_FAILED, stopSpmFailureLogged=false),
log id: fed5617
2014-11-24 18:05:37,209 INFO
[org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand]
(DefaultQuartzScheduler_Worker-69) [4a84c4e5] FINISH,
SetVdsStatusVDSCommand, log id: fed5617
2014-11-24 18:05:37,223 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(DefaultQuartzScheduler_Worker-69) [4a84c4e5] Correlation ID: 4a84c4e5, Job
ID: 4bfd4a6d-c3ef-468f-a40e-a3a6ca13011b, Call Stack: null, Custom Event
ID: -1, Message: Gluster command [<UNKNOWN>] failed on server Compute5.
2014-11-24 18:05:37,243 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(DefaultQuartzScheduler_Worker-69) [4a84c4e5] Correlation ID: null, Call
Stack: null, Custom Event ID: -1, Message: Status of host Compute5 was set
to NonOperational.
2014-11-24 18:05:37,272 INFO
[org.ovirt.engine.core.bll.HandleVdsVersionCommand]
(DefaultQuartzScheduler_Worker-69) [a0c8a7f] Running command:
HandleVdsVersionCommand internal: true. Entities affected : ID:
0bf6b00f-7947-4411-b55a-cc5eea2b381a Type: VDS
2014-11-24 18:05:37,274 INFO
[org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo]
(DefaultQuartzScheduler_Worker-69) [a0c8a7f] Host
0bf6b00f-7947-4411-b55a-cc5eea2b381a : Compute5 is already in
NonOperational status for reason GLUSTER_COMMAND_FAILED.
SetNonOperationalVds command is skipped.
2014-11-24 18:05:38,065 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
(DefaultQuartzScheduler_Worker-55) [3706e836] FINISH,
GlusterVolumesListVDSCommand, return:
{26ae1672-ee09-4a38-8fd2-72dd9974cc2b=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@4e72a1b1},
log id: 48a0c832
2014-11-24 18:05:43,243 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
(DefaultQuartzScheduler_Worker-35) START,
GlusterVolumesListVDSCommand(HostName = Compute4, HostId =
33648a90-200c-45ca-89d5-1ce305d79a6a), log id: 3ce13ebc
^C
[root@ccr01 ~]#
Thanks,
Punit