Re: [ovirt-users] [Centos7.1x64] [Ovirt 3.5.2] SOLVED: Test fence : Power management test failed for Host hosted_engine1 Done

From: "wodel youchi" <wodel.youchi(a)gmail.com&gt; To: "Martin Perina" <mperina(a)redhat.com&gt; Cc: "users" <Users(a)ovirt.org&gt;, "Eli Mesika" <emesika(a)redhat.com&gt; Sent: Wednesday, June 17, 2015 12:00:57 AM Subject: Re: [ovirt-users] [Centos7.1x64] [Ovirt 3.5.2] SOLVED: Test fence : Power management test failed for Host hosted_engine1 Done Hi, and thanks for your replies. The problem was the *quotation marks* in fence options, the lanplus="1" and power_wait="60" were misinterpreted, the correct form is lanplus=1, power_wait=60 After changing that the power management was configured and the test passed with : success, on.

thanks. 2015-06-11 12:11 GMT+01:00 Martin Perina <mperina(a)redhat.com&gt;: > Hi, > > I have HP DL160 G6 with ILO 2.0 and Firmware 4.23, but I'm really > don't have an idea what is your issue right now with power management. > Because oVirt just executes fence_ipmi on selected proxy host with > parameters you could see in your vdsm log. And those parameters are > identical to your command line options that you used when testing it > with fence_ipmi directly (which worked fine). Very strange :-( > > Also I can't see any difference from oVirt power management code point > of view in using Centos 6 or 7. Just be warned that in oVirt 3.6 not > all features will be supported on Centos 6, so using Centos 7 is IMO > a better option for now. > > But anyway could you please file a bug for oVirt 3.5.2 with the description > of your issue and your logs attached? We will try to reproduce this issue, > but so for I wasn't able to reproduce it. > > About your issue with hp watch dog, I cannot give any specifi advice, just > that you can try to solve the issue by updating BIOS/firmware to latest > version and/or try to contact HP support. > > Thanks a lot > > Martin Perina > > > ----- Original Message ----- > > From: "wodel youchi" <wodel.youchi(a)gmail.com&gt; > > To: "Martin Perina" <mperina(a)redhat.com&gt; > > Cc: "users" <Users(a)ovirt.org&gt;, "Eli Mesika" <emesika(a)redhat.com&gt; > > Sent: Thursday, June 11, 2015 12:56:46 PM > > Subject: Re: [ovirt-users] [Centos7.1x64] [Ovirt 3.5.2] Test fence : > Power management test failed for Host > > hosted_engine1 Done > > > > Hi Martin, > > > > Could you please tell me the version of the ILO firmware you are using? > > > > I did upgrade mine from 1.40 to 2.10 but nothing changed, I did also > > upgrade the smart array p420i card from 5.10 to 6.34 without luck so far. > > > > I checked again all parameters, I can't find the error. > > > > I did all the updates for Centos and oVirt > > > > I have another problem when rebooting any hypervisor, the hypervisor > hangs, > > the problem is with hpwtd (hp watch dog) > > "hpwdt unexpected close not stopping watchdog" > > > > I added this to kernel parameters "intremap=no_x2apic_optout" but it > didn't > > change any thing. > > > > I am thinking to test with the latest kernel available to see if it's a > > kernel problem. > > > > and I am going to reinstall the platform with Centos 6 to see if there > will > > be any differences. > > > > > > > > > > 2015-06-10 12:00 GMT+01:00 wodel youchi <wodel.youchi(a)gmail.com&gt;: > > > > > Hi, > > > > > > engine log is already in debug mode > > > > > > here it is: > > > 2015-06-10 11:48:23,653 INFO > > > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > > > (ajp--127.0.0.1-8702-12) Correlation ID: null, Call Stack: null, Custom > > > Event ID: -1, Message: Host hosted_engine_2 from cluster Default was > chosen > > > as a proxy to execute Status command on Host hosted_engine_1. > > > 2015-06-10 11:48:23,653 INFO [org.ovirt.engine.core.bll.FenceExecutor] > > > (ajp--127.0.0.1-8702-12) Using Host hosted_engine_2 from cluster > Default as > > > proxy to execute Status command on Host > > > 2015-06-10 11:48:23,673 INFO [org.ovirt.engine.core.bll.FenceExecutor] > > > (ajp--127.0.0.1-8702-12) Executing <Status> Power Management command, > Proxy > > > Host:hosted_engine_2, Agent:ipmilan, Target Host:, Management > > > IP:192.168.2.2, User:Administrator, Options: > power_wait="60",lanplus="1", > > > Fencing policy:null > > > 2015-06-10 11:48:23,703 INFO > > > *[org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] > > > (ajp--127.0.0.1-8702-12) START, FenceVdsVDSCommand(HostName = > > > hosted_engine_2, HostId = 0192d1ac-b905-4660-b149-4bef578985dd, > targetVdsId > > > = cf2d1260-7bb3-451a-9cd7-80e6a0ede52a, action = Status, ip = > 192.168.2.2, > > > port = , type = ipmilan, user = Administrator, password = ******, > options = > > > ' power_wait="60",lanplus="1"', policy = 'null'), log id: > > > 2bda01bd2015-06-10 11:48:23,892 WARN > > > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > > > (ajp--127.0.0.1-8702-12) Correlation ID: null, Call Stack: null, Custom > > > Event ID: -1, Message: Power Management test failed for Host > > > hosted_engine_1.Done* > > > 2015-06-10 11:48:23,892 INFO > > > [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] > > > (ajp--127.0.0.1-8702-12) FINISH, FenceVdsVDSCommand, return: > > > *Test Succeeded, unknown, log id: 2bda01bd2015-06-10 11:48:23,897 WARN > > > [org.ovirt.engine.core.bll.FenceExecutor] (ajp--127.0.0.1-8702-12) > Fencing > > > operation failed with proxy host 0192d1ac-b905-4660-*b149-4bef578985dd, > > > trying another proxy... > > > 2015-06-10 11:48:24,039 ERROR [org.ovirt.engine.core.bll.FenceExecutor] > > > (ajp--127.0.0.1-8702-12) Failed to run Power Management command on > Host , > > > no running proxy Host was found. > > > 2015-06-10 11:48:24,039 WARN [org.ovirt.engine.core.bll.FenceExecutor] > > > (ajp--127.0.0.1-8702-12) Failed to find other proxy to re-run failed > fence > > > operation, retrying with the same proxy... > > > 2015-06-10 11:48:24,143 INFO > > > *[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > > > (ajp--127.0.0.1-8702-12) Correlation ID: null, Call Stack: null, Custom > > > Event ID: -1, Message: Host hosted_engine_2 from cluster Default was > chosen > > > as a proxy to execute Status command on Host hosted_engine_1.* > > > 2015-06-10 11:48:24,143 INFO [org.ovirt.engine.core.bll.FenceExecutor] > > > (ajp--127.0.0.1-8702-12) Using Host hosted_engine_2 from cluster > Default as > > > proxy to execute Status command on Host > > > > > > *2015-06-10 11:48:24,148 INFO > [org.ovirt.engine.core.bll.FenceExecutor] > > > (ajp--127.0.0.1-8702-12) Executing <Status> Power Management command, > Proxy > > > Host:hosted_engine_2, Agent:ipmilan, Target Host:, Management > > > IP:192.168.2.2, User:Administrator, Options: > power_wait="60",lanplus="1", > > > Fencing policy:null2015-06-10 11:48:24,165 INFO > > > *[org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] > > > (ajp--127.0.0.1-8702-12) START, FenceVdsVDSCommand(HostName = > > > hosted_engine_2, HostId = 0192d1ac-b905-4660-b149-4bef578985dd, > targetVdsId > > > = cf2d1260-7bb3-451a-9cd7-80e6a0ede52a, action = Status, ip = > 192.168.2.2, > > > port = , type = ipmilan, user = Administrator, password = ******, > options = > > > ' power_wait="60",lanplus="1"', policy = 'null'), log id: 7e7f2726 > > > 2015-06-10 11:48:24,360 WARN > > > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > > > (ajp--127.0.0.1-8702-12) Correlation ID: null, Call Stack: null, Custom > > > Event ID: -1, Message: Power Management test failed for Host > > > hosted_engine_1.Done > > > 2015-06-10 11:48:24,360 INFO > > > *[org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] > > > (ajp--127.0.0.1-8702-12) FINISH, FenceVdsVDSCommand, return: Test > > > Succeeded, unknown, log id: 7e7f2726* > > > > > > > > > VDSM log from hosted_engine_2 > > > > > > JsonRpcServer::DEBUG::2015-06-10 > > > 11:48:23,640::__init__::506::jsonrpc.JsonRpcServer::(serve_requests) > > > Waiting for request > > > Thread-2201::DEBUG::2015-06-10 > 11:48:23,642::API::1209::vds::(fenceNode) > > > > *fenceNode(addr=192.168.2.2,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=False,options= > > > power_wait="60"lanplus="1",policy=None)* > > > Thread-2201::DEBUG::2015-06-10 > 11:48:23,642::utils::739::root::(execCmd) > > > /usr/sbin/fence_ipmilan (cwd None) > > > Thread-2201::DEBUG::2015-06-10 > 11:48:23,709::utils::759::root::(execCmd) > > > *FAILED: > > > <err> = 'Failed: Unable to obtain correct plug status or plug is not > > > available\n\n\n*'; <rc> = 1 > > > Thread-2201::DEBUG::2015-06-10 11:48:23,710::API::1164::vds::(fence) > rc 1 > > > inp agent=fence_ipmilan > > > ipaddr=192.168.2.2 > > > login=Administrator > > > action=status > > > passwd=XXXX > > > power_wait="60" > > > lanplus="1" out [] err ['Failed: Unable to obtain correct plug status > or > > > plug is not available', '', ''] > > > Thread-2201::DEBUG::2015-06-10 > 11:48:23,710::API::1235::vds::(fenceNode) > > > rc 1 in agent=fence_ipmilan > > > ipaddr=192.168.2.2 > > > login=Administrator > > > action=status > > > passwd=XXXX > > > power_wait="60" > > > lanplus="1" out [] err ['Failed: Unable to obtain correct plug status > or > > > plug is not available', '', ''] > > > Thread-2201::DEBUG::2015-06-10 > > > 11:48:23,710::stompReactor::163::yajsonrpc.StompServer::(send) Sending > > > response > > > JsonRpc (StompReactor)::DEBUG::2015-06-10 > > > 11:48:23,712::stompReactor::98::Broker.StompAdapter::(handle_frame) > > > Handling message <StompFrame command='SEND'> > > > JsonRpcServer::DEBUG::2015-06-10 > > > 11:48:23,713::__init__::506::jsonrpc.JsonRpcServer::(serve_requests) > > > Waiting for request > > > Thread-2202::DEBUG::2015-06-10 > 11:48:23,715::API::1209::vds::(fenceNode) > > > > fenceNode(addr=192.168.2.2,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=False,options= > > > power_wait="60" > > > lanplus="1",policy=None) > > > Thread-2202::DEBUG::2015-06-10 > 11:48:23,715::utils::739::root::(execCmd) > > > /usr/sbin/fence_ipmilan (cwd None) > > > Thread-2202::DEBUG::2015-06-10 > 11:48:23,781::utils::759::root::(execCmd) > > > FAILED: <err> = 'Failed: Unable to obtain correct plug status or plug > is > > > not available\n\n\n'; <rc> = 1 > > > Thread-2202::DEBUG::2015-06-10 11:48:23,781::API::1164::vds::(fence) > rc 1 > > > inp agent=fence_ipmilan > > > ipaddr=192.168.2.2 > > > login=Administrator > > > action=status > > > passwd=XXXX > > > power_wait="60" > > > lanplus="1" out [] err ['Failed: Unable to obtain correct plug status > or > > > plug is not available', '', ''] > > > > > > > > > > > > I triple checked, I used the correct IPs and login password, the test > in > > > console works. > > > > > > 2015-06-10 10:31 GMT+01:00 Martin Perina <mperina(a)redhat.com&gt;: > > > > > >> Hi, > > >> > > >> I just install engine 3.5.2 on Centos 7.1, added 2 Centos 7.1 hosts > (both > > >> with ipmilan fence devices) and everything worked fine. I also tried > to > > >> add > > >> options > > >> > > >> lanplus="1", power_wait="60" > > >> > > >> and even with them getting power status of hosts worked fine. > > >> > > >> So could you please check again settings of your hosts in webadmin? > > >> > > >> hosted_engine1 > > >> PM address: IP address of ILO4 interface of the host hosted_engine1 > > >> > > >> > > >> hosted_engine2 > > >> PM address: IP address of ILO4 interface of the host hosted_engine2 > > >> > > >> If the IP addresses are entered correctly, please allow DEBUG log for > > >> engine, > > >> execute test of PM settings for one host and attach logs from engine > and > > >> VDSM logs from both hosts. > > >> > > >> Thanks > > >> > > >> Martin Perina > > >> > > >> > > >> ----- Original Message ----- > > >> > From: "wodel youchi" <wodel.youchi(a)gmail.com&gt; > > >> > To: "users" <Users(a)ovirt.org&gt; > > >> > Sent: Tuesday, June 9, 2015 2:41:02 PM > > >> > Subject: [ovirt-users] [Centos7.1x64] [Ovirt 3.5.2] Test fence : > Power > > >> management test failed for Host hosted_engine1 > > >> > Done > > >> > > > >> > Hi, > > >> > > > >> > I have a weird problem with fencing > > >> > > > >> > I have a cluster of two HP DL380p G8 (ILO4) > > >> > > > >> > Centos7.1x64 and oVirt 3.5.2 ALL UPDATED > > >> > > > >> > I configured fencing first with ilo4 then ipmilan > > >> > > > >> > When testing fence from the engine I get : Succeeded, Unknown > > >> > > > >> > And in alerts tab I get : Power management test failed for Host > > >> > hosted_engine1 Done (the same for host2) > > >> > > > >> > I tested with fence_ilo4 and fence_ipmilan and they report the > result > > >> > correctly > > >> > > > >> > # fence_ipmilan -P -a 192.168.2.2 -o status -l Administrator -p > ertyuiop > > >> > -vExecuting: /usr/bin/ipmitool -I lanplus -H 192.168.2.2 -U > > >> Administrator -P > > >> > ertyuiop -p 623 -L ADMINISTRATOR chassis power status > > >> > > > >> > 0 Chassis Power is on > > >> > > > >> > > > >> > Status: ON > > >> > > > >> > > > >> > # fence_ilo4 -l Administrator -p ertyuiop -a 192.168.2.2 -o status > -v > > >> > Executing: /usr/bin/ipmitool -I lanplus -H 192.168.2.2 -U > Administrator > > >> -P > > >> > ertyuiop -p 623 -L ADMINISTRATOR chassis power status > > >> > > > >> > 0 Chassis Power is on > > >> > > > >> > > > >> > Status: ON > > >> > > > >> > ---------------------------------- > > >> > These are the options passed to fence_ipmilan (I tested with the > > >> options and > > >> > without them) > > >> > > > >> > lanplus="1", power_wait="60" > > >> > > > >> > > > >> > This is the engine log: > > >> > > > >> > 2015-06-09 13:35:29,287 INFO > [org.ovirt.engine.core.bll.FenceExecutor] > > >> > (ajp--127.0.0.1-8702-7) Using Host hosted_engine_2 from cluster > Default > > >> as > > >> > proxy to execute Status command on Host > > >> > 2015-06-09 13:35:29,289 INFO > [org.ovirt.engine.core.bll.FenceExecutor] > > >> > (ajp--127.0.0.1-8702-7) Executing <Status> Power Management command, > > >> Proxy > > >> > Host:hosted_engine_2, Agent:ipmilan, Target Host:, Management > > >> > IP:192.168.2.2, User:Administrator, Options: > > >> power_wait="60",lanplus="1", > > >> > Fencing policy:null > > >> > 2015-06-09 13:35:29,306 INFO > > >> > [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] > > >> > (ajp--127.0.0.1-8702-7) START, FenceVdsVDSCommand( > > >> > HostName = hosted_engine_2, > > >> > HostId = 0192d1ac-b905-4660-b149-4bef578985dd, > > >> > targetVdsId = cf2d1260-7bb3-451a-9cd7-80e6a0ede52a, > > >> > action = Status, > > >> > ip = 192.168.2.2, > > >> > port = , > > >> > type = ipmilan, > > >> > user = Administrator, > > >> > password = ******, > > >> > options = ' power_wait="60",lanplus="1"', > > >> > policy = 'null'), log id: 24ce6206 > > >> > 2015-06-09 13:35:29,516 WARN > > >> > > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > > >> > (ajp--127.0.0.1-8702-7) Correlation ID: null, Call Stack: null, > Custom > > >> Event > > >> > ID: -1, Message: Power Management test failed for Host > > >> hosted_engine_1.Done > > >> > 2015-06-09 13:35:29,516 INFO > > >> > [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] > > >> > (ajp--127.0.0.1-8702-7) FINISH, FenceVdsVDSCommand, return: Test > > >> Succeeded, > > >> > unknown , log id: 24ce6206 > > >> > > > >> > > > >> > and here the vdsm log from the proxy > > >> > > > >> > JsonRpcServer::DEBUG::2015-06-09 > > >> > 13:37:52,461::__init__::506::jsonrpc.JsonRpcServer::(serve_requests) > > >> Waiting > > >> > for request > > >> > Thread-131907::DEBUG::2015-06-09 > > >> 13:37:52,463::API::1209::vds::(fenceNode) > > >> > > > >> > fenceNode(addr=192.168.2.2,port=,agent=ipmilan,user=Administrator,passwd=XXXX,action=status,secure=False,options= > > >> > power_wait="60" > > >> > lanplus="1",policy=None) > > >> > Thread-131907::DEBUG::2015-06-09 > > >> 13:37:52,463::utils::739::root::(execCmd) > > >> > /usr/sbin/fence_ipmilan (cwd None) > > >> > Thread-131907::DEBUG::2015-06-09 > > >> 13:37:52,533::utils::759::root::(execCmd) > > >> > FAILED: <err> = 'Failed: Unable to obtain correct plug status or > plug > > >> is not > > >> > available\n\n\n'; <rc> = 1 > > >> > Thread-131907::DEBUG::2015-06-09 > 13:37:52,533::API::1164::vds::(fence) > > >> rc 1 > > >> > inp agent=fence_ipmilan > > >> > ipaddr=192.168.2.2 > > >> > login=Administrator > > >> > action=status > > >> > passwd=XXXX > > >> > power_wait="60" > > >> > lanplus="1" out [] err ['Failed: Unable to obtain correct plug > status > > >> or plug > > >> > is not available', '', ''] > > >> > Thread-131907::DEBUG::2015-06-09 > > >> 13:37:52,533::API::1235::vds::(fenceNode) rc > > >> > 1 in agent=fence_ipmilan > > >> > ipaddr=192.168.2.2 > > >> > login=Administrator > > >> > action=status > > >> > passwd=XXXX > > >> > power_wait="60" > > >> > lanplus="1" out [] err [' Failed: Unable to obtain correct plug > status > > >> or > > >> > plug is not available ', '', ''] > > >> > Thread-131907::DEBUG::2015-06-09 > > >> > 13:37:52,534::stompReactor::163::yajsonrpc.StompServer::(send) > Sending > > >> > response > > >> > Detector thread::DEBUG::2015-06-09 > > >> > > > >> > 13:37:53,670::protocoldetector::187::vds.MultiProtocolAcceptor::(_add_connection) > > >> > Adding connection from 127.0.0.1:55761 > > >> > > > >> > > > >> > VDSM rpms > > >> > # rpm -qa | grep vdsm > > >> > vdsm-cli-4.16.14-0.el7.noarch > > >> > vdsm-python-zombiereaper-4.16.14-0.el7.noarch > > >> > vdsm-xmlrpc-4.16.14-0.el7.noarch > > >> > vdsm-yajsonrpc-4.16.14-0.el7.noarch > > >> > vdsm-4.16.14-0.el7.x86_64 > > >> > vdsm-python-4.16.14-0.el7.noarch > > >> > vdsm-jsonrpc-4.16.14-0.el7.noarch > > >> > > > >> > any idea? > > >> > > > >> > Thanks in advance. > > >> > > > >> > _______________________________________________ > > >> > Users mailing list > > >> > Users(a)ovirt.org > > >> > http://lists.ovirt.org/mailman/listinfo/users > > >> > > > >> > > > > > > > > >