2:26 a.m.
--------------DDA22B530AEEAC5C11E37F1E
Content-Type: text/plain; charset="utf-8"; format=flowed
Content-Transfer-Encoding: 7bit
I'm adding 3 hardware nodes to our cluster. All 3 same type of server
and software, HP DL360 G8, centos 7.
One fails the fence agent test.
The one I'm having problems with has a newer version of ilo firmware,
not sure if related.
Troublemaker: 2.54
Others: 2.53
ovirt-engine-4.1.0.4-1.el7.centos.noarch
vdsm-4.19.4-1.el7.centos.x86_64
2017-07-26 15:14:30,215-07 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-52) [4d72d360-2b92-43f6-b2df-e80ee305c622] EVENT_ID:
FENCE_OPERATION_USING_AGENT_AND_PROXY_STARTED(9,020), Correlation ID:
null, Call Stack: null, Custom Event ID: -1, Message: Executing power
management status on Host ovirt6.j2noc.com using Proxy Host
ovirt1.j2noc.com and Fence Agent ilo4:10.144.254.89.
2017-07-26 15:14:30,216-07 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (default
task-52) [4d72d360-2b92-43f6-b2df-e80ee305c622] START,
FenceVdsVDSCommand(HostName = ovirt1.j2noc.com,
FenceVdsVDSCommandParameters:{runAsync='true',
hostId='23d2c0ab-5dd1-43af-9db3-2a426a539faf',
targetVdsId='00000000-0000-0000-0000-000000000000', action='STATUS',
agent='FenceAgent:{id='null', hostId='null', order='1',
type='ilo4',
ip='10.144.254.89', port='null', user='Administrator',
password='***',
encryptOptions='false', options='power_wait=4'}',
policy='null'}), log
id: 1498b3c4
*2017-07-26 15:14:30,414-07 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (default
task-52) [4d72d360-2b92-43f6-b2df-e80ee305c622] FINISH,
FenceVdsVDSCommand, return: FenceOperationResult:{status='ERROR',
powerStatus='UNKNOWN', message='[Failed: Unable to obtain correct plug
status or plug is not available, , ]'}, log id: 1498b3c4*
2017-07-26 15:14:30,420-07 WARN
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-52) [4d72d360-2b92-43f6-b2df-e80ee305c622] EVENT_ID:
FENCE_OPERATION_USING_AGENT_AND_PROXY_FAILED(9,021), Correlation ID:
null, Call Stack: null, Custom Event ID: -1, Message: Execution of power
management status on Host ovirt6.j2noc.com using Proxy Host
ovirt1.j2noc.com and Fence Agent ilo4:10.144.254.89 failed.
2017-07-26 15:14:30,420-07 WARN
[org.ovirt.engine.core.bll.pm.FenceAgentExecutor] (default task-52)
[4d72d360-2b92-43f6-b2df-e80ee305c622] Fence action failed using proxy
host '10.144.110.99', trying another proxy
2017-07-26 15:14:30,740-07 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-52) [4d72d360-2b92-43f6-b2df-e80ee305c622] EVENT_ID:
FENCE_OPERATION_USING_AGENT_AND_PROXY_STARTED(9,020), Correlation ID:
null, Call Stack: null, Custom Event ID: -1, Message: Executing power
management status on Host ovirt6.j2noc.com using Proxy Host
ovirt2.j2noc.com and Fence Agent ilo4:10.144.254.89.
2017-07-26 15:14:30,741-07 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (default
task-52) [4d72d360-2b92-43f6-b2df-e80ee305c622] START,
FenceVdsVDSCommand(HostName = ovirt2.j2noc.com,
FenceVdsVDSCommandParameters:{runAsync='true',
hostId='91d8fa70-fd24-4530-90f7-982ff068230b',
targetVdsId='00000000-0000-0000-0000-000000000000', action='STATUS',
agent='FenceAgent:{id='null', hostId='null', order='1',
type='ilo4',
ip='10.144.254.89', port='null', user='Administrator',
password='***',
encryptOptions='false', options='power_wait=4'}',
policy='null'}), log
id: 67d837da
2017-07-26 15:14:30,898-07 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (default
task-52) [4d72d360-2b92-43f6-b2df-e80ee305c622] FINISH,
FenceVdsVDSCommand, return: FenceOperationResult:{status='ERROR',
powerStatus='UNKNOWN', message='[Failed: Unable to obtain correct plug
status or plug is not available, , ]'}, log id: 67d837da
2017-07-26 15:14:30,903-07 WARN
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-52) [4d72d360-2b92-43f6-b2df-e80ee305c622] EVENT_ID:
FENCE_OPERATION_USING_AGENT_AND_PROXY_FAILED(9,021), Correlation ID:
null, Call Stack: null, Custom Event ID: -1, Message: Execution of power
management status on Host ovirt6.j2noc.com using Proxy Host
ovirt2.j2noc.com and Fence Agent ilo4:10.144.254.89 failed.
I'm not sure the right syntax for fence_ipmilan since even a "good" host
fails:
[root@ovirt4 prod vdsm]# fence_ipmilan -a 10.144.254.87 -P -l
Administrator -p *** -o status -v chassis power status
Executing: /usr/bin/ipmitool -I lanplus -H 10.144.254.87 -U
Administrator -P [set] -p 623 -L ADMINISTRATOR chassis power status
1 Error: Unable to establish IPMI v2 / RMCP+ session
Failed: Unable to obtain correct plug status or plug is not available
Any ideas on what the issue is?
--------------DDA22B530AEEAC5C11E37F1E
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta http-equiv="content-type" content="text/html;
charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
I'm adding 3 hardware nodes to our cluster. All 3 same type of
server and software, HP DL360 G8, centos 7.<br>
One fails the fence agent test.<br>
The one I'm having problems with has a newer version of ilo
firmware, not sure if related.<br>
Troublemaker: 2.54<br>
Others: 2.53<br>
<br>
ovirt-engine-4.1.0.4-1.el7.centos.noarch<br>
vdsm-4.19.4-1.el7.centos.x86_64<br>
<br>
<br>
2017-07-26 15:14:30,215-07 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-52) [4d72d360-2b92-43f6-b2df-e80ee305c622] EVENT_ID:
FENCE_OPERATION_USING_AGENT_AND_PROXY_STARTED(9,020), Correlation
ID: null, Call Stack: null, Custom Event ID: -1, Message: Executing
power management status on Host ovirt6.j2noc.com using Proxy Host
ovirt1.j2noc.com and Fence Agent ilo4:10.144.254.89.<br>
2017-07-26 15:14:30,216-07 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand]
(default task-52) [4d72d360-2b92-43f6-b2df-e80ee305c622] START,
FenceVdsVDSCommand(HostName = ovirt1.j2noc.com,
FenceVdsVDSCommandParameters:{runAsync='true',
hostId='23d2c0ab-5dd1-43af-9db3-2a426a539faf',
targetVdsId='00000000-0000-0000-0000-000000000000', action='STATUS',
agent='FenceAgent:{id='null', hostId='null', order='1',
type='ilo4',
ip='10.144.254.89', port='null', user='Administrator',
password='***', encryptOptions='false',
options='power_wait=4'}',
policy='null'}), log id: 1498b3c4<br>
<b>2017-07-26 15:14:30,414-07 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand]
(default task-52) [4d72d360-2b92-43f6-b2df-e80ee305c622] FINISH,
FenceVdsVDSCommand, return: FenceOperationResult:{status='ERROR',
powerStatus='UNKNOWN', message='[Failed: Unable to obtain correct
plug status or plug is not available, , ]'}, log id:
1498b3c4</b><br>
2017-07-26 15:14:30,420-07 WARN
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-52) [4d72d360-2b92-43f6-b2df-e80ee305c622] EVENT_ID:
FENCE_OPERATION_USING_AGENT_AND_PROXY_FAILED(9,021), Correlation ID:
null, Call Stack: null, Custom Event ID: -1, Message: Execution of
power management status on Host ovirt6.j2noc.com using Proxy Host
ovirt1.j2noc.com and Fence Agent ilo4:10.144.254.89 failed.<br>
2017-07-26 15:14:30,420-07 WARN
[org.ovirt.engine.core.bll.pm.FenceAgentExecutor] (default task-52)
[4d72d360-2b92-43f6-b2df-e80ee305c622] Fence action failed using
proxy host '10.144.110.99', trying another proxy<br>
2017-07-26 15:14:30,740-07 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-52) [4d72d360-2b92-43f6-b2df-e80ee305c622] EVENT_ID:
FENCE_OPERATION_USING_AGENT_AND_PROXY_STARTED(9,020), Correlation
ID: null, Call Stack: null, Custom Event ID: -1, Message: Executing
power management status on Host ovirt6.j2noc.com using Proxy Host
ovirt2.j2noc.com and Fence Agent ilo4:10.144.254.89.<br>
2017-07-26 15:14:30,741-07 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand]
(default task-52) [4d72d360-2b92-43f6-b2df-e80ee305c622] START,
FenceVdsVDSCommand(HostName = ovirt2.j2noc.com,
FenceVdsVDSCommandParameters:{runAsync='true',
hostId='91d8fa70-fd24-4530-90f7-982ff068230b',
targetVdsId='00000000-0000-0000-0000-000000000000', action='STATUS',
agent='FenceAgent:{id='null', hostId='null', order='1',
type='ilo4',
ip='10.144.254.89', port='null', user='Administrator',
password='***', encryptOptions='false',
options='power_wait=4'}',
policy='null'}), log id: 67d837da<br>
2017-07-26 15:14:30,898-07 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand]
(default task-52) [4d72d360-2b92-43f6-b2df-e80ee305c622] FINISH,
FenceVdsVDSCommand, return: FenceOperationResult:{status='ERROR',
powerStatus='UNKNOWN', message='[Failed: Unable to obtain correct
plug status or plug is not available, , ]'}, log id: 67d837da<br>
2017-07-26 15:14:30,903-07 WARN
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-52) [4d72d360-2b92-43f6-b2df-e80ee305c622] EVENT_ID:
FENCE_OPERATION_USING_AGENT_AND_PROXY_FAILED(9,021), Correlation ID:
null, Call Stack: null, Custom Event ID: -1, Message: Execution of
power management status on Host ovirt6.j2noc.com using Proxy Host
ovirt2.j2noc.com and Fence Agent ilo4:10.144.254.89 failed.<br>
<br>
<br>
<br>
I'm not sure the right syntax for fence_ipmilan since even a "good"
host fails:<br>
<br>
<br>
[root@ovirt4 prod vdsm]# fence_ipmilan -a 10.144.254.87 -P -l
Administrator -p *** -o status -v chassis power status <br>
Executing: /usr/bin/ipmitool -I lanplus -H 10.144.254.87 -U
Administrator -P [set] -p 623 -L ADMINISTRATOR chassis power status<br>
<br>
1 Error: Unable to establish IPMI v2 / RMCP+ session<br>
<br>
<br>
Failed: Unable to obtain correct plug status or plug is not
available<br>
<br>
<br>
<br>
<br>
Any ideas on what the issue is?<br>
<br>
</body>
</html>
--------------DDA22B530AEEAC5C11E37F1E--