[ovirt-users] Options not being passed fence_ipmilan, Ovirt3.5 on Centos 7.1 hosts

Wed Apr 22 07:00:29 UTC 2015


----- Original Message -----
> From: "Mike Lindsay" <mike.lindsay at cbc.ca>
> To: users at ovirt.org
> Sent: Tuesday, April 21, 2015 8:16:18 PM
> Subject: [ovirt-users] Options not being passed fence_ipmilan,	Ovirt3.5 on Centos 7.1 hosts
> 
> Hi All,
> 
> I have a bit of an issue with a new install of Ovirt 3.5 (our 3.4 cluster is
> working fine) in a 4 node cluster.
> 
> When I test fencing (or cause a kernal panic triggering a fence) the fencing
> fails. On investigation it appears that the fencing options are not being
> passed to the fencing script (fence_ipmilan in this case):
> 
> Fence options under GUI(as entered in the gui): lanplus, ipport=623,
> power_wait=4, privlvl=operator
> 
> from vdsm.log on the fence proxy node:
> 
> Thread-818296::DEBUG::2015-04-21 12:39:39,136::API::1209::vds::(fenceNode)
> fenceNode(addr=x.x.x.x,port=,agent=ipmilan,user=stonith,passwd=XXXX,action=status,secure=False,options=
> power_wait=4
> Thread-818296::DEBUG::2015-04-21 12:39:39,137::utils::739::root::(execCmd)
> /usr/sbin/fence_ipmilan (cwd None)
> Thread-818296::DEBUG::2015-04-21 12:39:39,295::utils::759::root::(execCmd)
> FAILED: <err> = 'Failed: Unable to obtain correct plug status or plug is not
> available\n\n\n'; <rc> = 1
> Thread-818296::DEBUG::2015-04-21 12:39:39,296::API::1164::vds::(fence) rc 1
> inp agent=fence_ipmilan
> Thread-818296::DEBUG::2015-04-21 12:39:39,296::API::1235::vds::(fenceNode) rc
> 1 in agent=fence_ipmilan
> Thread-818296::DEBUG::2015-04-21
> 12:39:39,297::stompReactor::163::yajsonrpc.StompServer::(send) Sending
> response
> 
> 
> from engine.log on the engine:
> 2015-04-21 12:39:38,843 INFO
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (ajp--127.0.0.1-8702-4) Correlation ID: null, Call Stack: null, Custom Event
> ID: -1, Message: Host mpc-ovirt-node03 from cluster Default was chosen as a
> proxy to execute Status command on Host mpc-ovirt-node04.
> 2015-04-21 12:39:38,845 INFO [org.ovirt.engine.core.bll.FenceExecutor]
> (ajp--127.0.0.1-8702-4) Using Host mpc-ovirt-node03 from cluster Default as
> proxy to execute Status command on Host
> 2015-04-21 12:39:38,885 INFO [org.ovirt.engine.core.bll.FenceExecutor]
> (ajp--127.0.0.1-8702-4) Executing <Status> Power Management command, Proxy
> Host:mpc-ovirt-node03, Agent:ipmilan, Target Host:, Management IP:x.x.x.x,
> User:stonith, Options: power_wait=4, ipport=623, privlvl=operator,lanplus,
> Fencing policy:null
> 2015-04-21 12:39:38,921 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand]
> (ajp--127.0.0.1-8702-4) START, FenceVdsVDSCommand(HostName =
> mpc-ovirt-node03, HostId = 5613a489-589d-4e89-ab01-3642795eedb8, targetVdsId
> = dbfa4e85-3e97-4324-b222-bf40a491db08, action = Status, ip = x.x.x.x, port
> = , type = ipmilan, user = stonith, password = ******, options = '
> power_wait=4, ipport=623, privlvl=operator,lanplus', policy = 'null'), log
> id: 774f328
> 2015-04-21 12:39:39,338 WARN
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (ajp--127.0.0.1-8702-4) Correlation ID: null, Call Stack: null, Custom Event
> ID: -1, Message: Power Management test failed for Host mpc-ovirt-node04.Done
> 2015-04-21 12:39:39,339 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand]
> (ajp--127.0.0.1-8702-4) FINISH, FenceVdsVDSCommand, return: Test Succeeded,
> unknown, log id: 774f328
> 2015-04-21 12:39:39,340 WARN [org.ovirt.engine.core.bll.FenceExecutor]
> (ajp--127.0.0.1-8702-4) Fencing operation failed with proxy host
> 5613a489-589d-4e89-ab01-3642795eedb8, trying another proxy...
> 2015-04-21 12:39:39,594 INFO
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (ajp--127.0.0.1-8702-4) Correlation ID: null, Call Stack: null, Custom Event
> ID: -1, Message: Host mpc-ovirt-node01 from cluster Default was chosen as a
> proxy to execute Status command on Host mpc-ovirt-node04.
> 2015-04-21 12:39:39,595 INFO [org.ovirt.engine.core.bll.FenceExecutor]
> (ajp--127.0.0.1-8702-4) Using Host mpc-ovirt-node01 from cluster Default as
> proxy to execute Status command on Host
> 2015-04-21 12:39:39,598 INFO [org.ovirt.engine.core.bll.FenceExecutor]
> (ajp--127.0.0.1-8702-4) Executing <Status> Power Management command, Proxy
> Host:mpc-ovirt-node01, Agent:ipmilan, Target Host:, Management IP:x.x.x.x,
> User:stonith, Options: power_wait=4, ipport=623, privlvl=operator,lanplus,
> Fencing policy:null
> 2015-04-21 12:39:39,634 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand]
> (ajp--127.0.0.1-8702-4) START, FenceVdsVDSCommand(HostName =
> mpc-ovirt-node01, HostId = c3e8be6e-ac54-4861-b774-17ba5cc66dc6, targetVdsId
> = dbfa4e85-3e97-4324-b222-bf40a491db08, action = Status, ip = x.x.x.x, port
> = , type = ipmilan, user = stonith, password = ******, options = '
> power_wait=4, ipport=623, privlvl=operator,lanplus', policy = 'null'), log
> id: 6369eb1
> 2015-04-21 12:39:40,056 WARN
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (ajp--127.0.0.1-8702-4) Correlation ID: null, Call Stack: null, Custom Event
> ID: -1, Message: Power Management test failed for Host mpc-ovirt-node04.Done
> 2015-04-21 12:39:40,057 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand]
> (ajp--127.0.0.1-8702-4) FINISH, FenceVdsVDSCommand, return: Test Succeeded,
> unknown, log id: 6369eb1
> 
> 
> For verification I temporarily replaced /usr/sbin/fence_ipmilan with a shell
> script that dumps the env plus any cli args passed into a log file:
> 
> -------------------------- Tue Apr 21 12:39:39 EDT 2015
> ----------------------------
> ENV DUMP:
> LC_ALL=C
> USER=vdsm
> PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
> PWD=/
> LANG=en_CA.UTF-8
> LIBVIRT_LOG_FILTERS=
> SHLVL=1
> HOME=/var/lib/vdsm
> LOGNAME=vdsm
> LIBVIRT_LOG_OUTPUTS=
> _=/usr/bin/env
> 
> ------------------------------
> CLI DUMP:
> 
> <this is where the cli args should be listed>
> 
> 
> Version info:
> libvirt version: 1.2.8, package: 16.el7_1.2 (CentOS BuildSystem <
> http://bugs.centos.org >, 2015-03-26-23:17:42, worker1.bsys.centos.org )
> fence_ipmilan: 4.0.11 (built Mon Apr 13 13:22:18 UTC 2015)
> vdsm.x86_64: 4.16.10-8.gitc937927.el7
> ovirt-engine.noarch: 3.5.1.1-1.el6
> 
> Engine os: Centos 6.6
> Host os: Centos 7.1.1503
> 
> I've found some old post from 2012 that describe the same problem. Has anyone
> else run into this?
> 
> Any thought or suggestions would be appreciated.

Hi Mike

Seems from the logs above that the oVirt engine got the options correctly but somehow it is ignored by VDSM
Can you please share installed VDSM RPM version on the host that served as a proxy for this operation 
Please also provide the cluster version of the cluster that contains the proxy host as well

Thanks 

Eli


> 
> Cheers,
> Mike
> 
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>