[Users] Issues with power management through fence_ipmilan
I have three machines; one manage and two hypervisors. I've configured power management for the two hypervisor nodes to use ipmilan as the power management type, with the options "power_wait=4,lanplus=yes". When I press the test button, the test fails. The error message on the manager is: 2012-04-12 10:48:54,610 INFO [org.ovirt.engine.core.bll.FencingExecutor] (http--0.0.0.0-8443-8) Executing <Status> Power Management command, Proxy Host:rhevh2.jfa.unibet.com, Agent:ipmilan, Target Host:, Management IP:10.0.102.150, User:root, Options:power_wait=4,lanplus=yes 2012-04-12 10:48:54,615 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (http--0.0.0.0-8443-8) START, FenceVdsVDSCommand(vdsId = 242132ee-83ab-11e1-8e45-2c768aac9498, targetVdsId = cd224c12-83aa-11e1-a177-2c768aac9498, action = Status, ip = 10.0.102.150, port = , type = ipmilan, user = root, password = ******, options = 'power_wait=4,lanplus=yes '), log id: 643e7825 2012-04-12 10:48:56,824 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (http--0.0.0.0-8443-8) FINISH, FenceVdsVDSCommand, return: Test Failed, Host Status is: unknown. The fence-agent script reported the following error: Getting status of IPMI:10.0.102.150...Chassis power = Unknown On the proxy hypervisor, I get this message: Thread-48608::DEBUG::2012-04-12 10:58:32,009::API::954::vds::(fenceNode) fenceNode(addr=10.0.102.150,port=,agent=ipmilan,user=root,passwd=XXXX,action=status,secure=,options=) Thread-48608::DEBUG::2012-04-12 10:58:33,029::API::980::vds::(fenceNode) rc 1 in agent=fence_ipmilan ipaddr=10.0.102.150 login=root option=status passwd=XXXX out Getting status of IPMI:10.0.102.150...Chassis power = Unknown Failed err Thread-48610::DEBUG::2012-04-12 10:58:33,110::API::954::vds::(fenceNode) fenceNode(addr=10.0.102.150,port=,agent=ipmilan,user=root,passwd=XXXX,action=status,secure=,options=) Thread-48610::DEBUG::2012-04-12 10:58:34,129::API::980::vds::(fenceNode) rc 1 in agent=fence_ipmilan ipaddr=10.0.102.150 login=root option=status passwd=XXXX out Getting status of IPMI:10.0.102.150...Chassis power = Unknown Failed err It looks to me like the options are dropped, especially since the fenceNode line from the hypervisor says "options=". What's going on here? When I try to save the power management configuration, the Options-field is empty the next time I go to Edit Host. Running fence_ipmilan manually works fine, both from the hypervisors and the manager. I also can't find the "ovirt-manage-domains" command, so I'm starting to think this might be a problem with the CentOS build of oVirt I'm using ( http://www.dreyou.org/ovirt/).
On 04/12/2012 01:01 PM, Martin Hovmöller wrote:
It looks to me like the options are dropped, especially since the fenceNode line from the hypervisor says "options=". What's going on here? When I try to save the power management configuration, the Options-field is empty the next time I go to Edit Host. Running fence_ipmilan manually works fine, both from the hypervisors and the manager.
sounds like a bug in the UI. please check from api to rule out an engine bug. btw, from this bug, it looks like it was only added to intended 3.1 release of ovirt in the ui? Bug 782703 - [webadmin] No power management options in GUI which was fixed mid-march this also seem in same area (could be fixed by now, but you are not using latest devel branch), so again, i'd check from api first: Bug 784790 - [ovirt][engine][webadmin] - power management - slot information is not being saved (bladecenter)
On Thu, Apr 12, 2012 at 12:41 PM, Itamar Heim <iheim@redhat.com> wrote:
On 04/12/2012 01:01 PM, Martin Hovmöller wrote:
It looks to me like the options are dropped, especially since the fenceNode line from the hypervisor says "options=". What's going on here? When I try to save the power management configuration, the Options-field is empty the next time I go to Edit Host. Running fence_ipmilan manually works fine, both from the hypervisors and the manager.
sounds like a bug in the UI. please check from api to rule out an engine bug. btw, from this bug, it looks like it was only added to intended 3.1 release of ovirt in the ui? Bug 782703 - [webadmin] No power management options in GUI which was fixed mid-march
this also seem in same area (could be fixed by now, but you are not using latest devel branch), so again, i'd check from api first: Bug 784790 - [ovirt][engine][webadmin] - power management - slot information is not being saved (bladecenter)
It does indeed seem to be an engine bug. The cli gives me the following info: oVirt shell (connected)]# show host rhevh1.domain id : 4e6ccb2c-84c7-11e1-aecc-2c768aac9498 name : rhevh1.domain address : 10.23.30.110 cluster-id : 99408929-82cf-4dc7-a532-9d998063fa95 cpu-name : AMD Opteron(tm) Processor 6174 cpu-speed : 2200.0 cpu-topology-cores : 6 cpu-topology-sockets : 1 iscsi-initiator : iqn.1994-05.com.redhat:a85815241216 ksm-enabled : True memory : 25194135552 port : 54321 power_management-address : 10.0.102.150 power_management-enabled : True power_management-options-option-name : power_wait power_management-options-option-value: 4 power_management-options-option-name : lanplus power_management-options-option-value: yes power_management-type : ipmilan power_management-username : root status-state : up storage_manager-priority : 5 storage_manager-valueOf : false summary-active : 0 summary-migrating : 0 summary-total : 0 transparent_hugepages-enabled : True type : rhel So apparently the options are saved. They are lost if I edit the host and don't manually add the options again before clicking Ok, so it seems to be saving the options correctly, but not re-reading them. Manual fencing by klicking Power Management -> Restart fails as well, even though the options seem to be passwd correctly when I do that. The fact that this option exists means that the first bug you pointed out doesn't match, but the second one seems to be the same one I'm seeing. 2012-04-12 20:48:19,307 INFO [org.ovirt.engine.core.bll.FencingExecutor] (http--0.0.0.0-8443-6) Executing <Status> Power Management command, Proxy Host:rhevh2.domain, Agent:ipmilan, Target Host:rhevh1.domain, Management IP:10.0.102.150, User:root, Options:power_wait=4,lanplus=yes 2012-04-12 20:48:19,309 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (http--0.0.0.0-8443-6) START, FenceVdsVDSCommand(vdsId = 6fdbf336-84c8-11e1-82ef-2c768aac9498, targetVdsId = 4e6ccb2c-84c7-11e1-aecc-2c768aac9498, action = Status, ip = 10.0.102.150, port = , type = ipmilan, user = root, password = ******, options = 'power_wait=4,lanplus=yes'), log id: 11a7cfb9 2012-04-12 20:48:21,529 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (http--0.0.0.0-8443-6) FINISH, FenceVdsVDSCommand, return: Test Failed, Host Status is: unknown. The fence-agent script reported the following error: Getting status of IPMI:10.0.102.150...Chassis power = Unknown Failed , log id: 11a7cfb9 2012-04-12 20:48:21,529 ERROR [org.ovirt.engine.core.bll.RestartVdsCommand] (http--0.0.0.0-8443-6) Failed to run RestartVdsCommand on vds :rhevh1.domain 2012-04-12 20:48:21,530 WARN [org.ovirt.engine.core.bll.RestartVdsCommand] (http--0.0.0.0-8443-6) CanDoAction of action RestartVds failed. Reasons:VDS_FAILED_FENCE_VIA_PROXY_CONNECTION,VDS_FENCING_OPERATION_FAILED,VAR__TYPE__HOST,VAR__ACTION__RESTART I'm trying to figure out how to try to fence a machine from the cli, but I can't figure out the correct syntax. The help seems to be a bit broken: [oVirt shell (connected)]# action host rhevh1.domain fence error: status: 400 reason: Bad Request detail: Action [fenceType] required for fence Why do I need to specify fenceType when it's already stored for the host (and how do I specify it for that matter)?
On 04/13/2012 11:09 AM, Martin Hovmöller wrote:
On Thu, Apr 12, 2012 at 12:41 PM, Itamar Heim <iheim@redhat.com <mailto:iheim@redhat.com>> wrote:
On 04/12/2012 01:01 PM, Martin Hovmöller wrote:
It looks to me like the options are dropped, especially since the fenceNode line from the hypervisor says "options=". What's going on here? When I try to save the power management configuration, the Options-field is empty the next time I go to Edit Host. Running fence_ipmilan manually works fine, both from the hypervisors and the manager.
sounds like a bug in the UI. please check from api to rule out an engine bug. btw, from this bug, it looks like it was only added to intended 3.1 release of ovirt in the ui? Bug 782703 - [webadmin] No power management options in GUI which was fixed mid-march
this also seem in same area (could be fixed by now, but you are not using latest devel branch), so again, i'd check from api first: Bug 784790 - [ovirt][engine][webadmin] - power management - slot information is not being saved (bladecenter)
It does indeed seem to be an engine bug. The cli gives me the following info: oVirt shell (connected)]# show host rhevh1.domain
id : 4e6ccb2c-84c7-11e1-aecc-2c768aac9498 name : rhevh1.domain address : 10.23.30.110 cluster-id : 99408929-82cf-4dc7-a532-9d998063fa95 cpu-name : AMD Opteron(tm) Processor 6174 cpu-speed : 2200.0 cpu-topology-cores : 6 cpu-topology-sockets : 1 iscsi-initiator : iqn.1994-05.com.redhat:a85815241216 ksm-enabled : True memory : 25194135552 port : 54321 power_management-address : 10.0.102.150 power_management-enabled : True power_management-options-option-name : power_wait power_management-options-option-value: 4 power_management-options-option-name : lanplus power_management-options-option-value: yes power_management-type : ipmilan power_management-username : root status-state : up storage_manager-priority : 5 storage_manager-valueOf : false summary-active : 0 summary-migrating : 0 summary-total : 0 transparent_hugepages-enabled : True type : rhel
So apparently the options are saved. They are lost if I edit the host and don't manually add the options again before clicking Ok, so it seems to be saving the options correctly, but not re-reading them. Manual fencing by klicking Power Management -> Restart fails as well, even though the options seem to be passwd correctly when I do that. The fact that this option exists means that the first bug you pointed out doesn't match, but the second one seems to be the same one I'm seeing.
2012-04-12 20:48:19,307 INFO [org.ovirt.engine.core.bll.FencingExecutor] (http--0.0.0.0-8443-6) Executing <Status> Power Management command, Proxy Host:rhevh2.domain, Agent:ipmilan, Target Host:rhevh1.domain, Management IP:10.0.102.150, User:root, Options:power_wait=4,lanplus=yes 2012-04-12 20:48:19,309 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (http--0.0.0.0-8443-6) START, FenceVdsVDSCommand(vdsId = 6fdbf336-84c8-11e1-82ef-2c768aac9498, targetVdsId = 4e6ccb2c-84c7-11e1-aecc-2c768aac9498, action = Status, ip = 10.0.102.150, port = , type = ipmilan, user = root, password = ******, options = 'power_wait=4,lanplus=yes'), log id: 11a7cfb9 2012-04-12 20:48:21,529 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (http--0.0.0.0-8443-6) FINISH, FenceVdsVDSCommand, return: Test Failed, Host Status is: unknown. The fence-agent script reported the following error: Getting status of IPMI:10.0.102.150...Chassis power = Unknown Failed , log id: 11a7cfb9 2012-04-12 20:48:21,529 ERROR [org.ovirt.engine.core.bll.RestartVdsCommand] (http--0.0.0.0-8443-6) Failed to run RestartVdsCommand on vds :rhevh1.domain 2012-04-12 20:48:21,530 WARN [org.ovirt.engine.core.bll.RestartVdsCommand] (http--0.0.0.0-8443-6) CanDoAction of action RestartVds failed. Reasons:VDS_FAILED_FENCE_VIA_PROXY_CONNECTION,VDS_FENCING_OPERATION_FAILED,VAR__TYPE__HOST,VAR__ACTION__RESTART
I'm trying to figure out how to try to fence a machine from the cli, but I can't figure out the correct syntax. The help seems to be a bit broken: [oVirt shell (connected)]# action host rhevh1.domain fence error: status: 400 reason: Bad Request detail: Action [fenceType] required for fence
Why do I need to specify fenceType when it's already stored for the host (and how do I specify it for that matter)?
from: https://xxx/api/capabilities <fence_types> <fence_type>manual</fence_type> <fence_type>restart</fence_type> <fence_type>start</fence_type> fence_type>stop</fence_type> <fence_type>status</fence_type> </fence_types>
participants (2)
-
Itamar Heim -
Martin Hovmöller