- Users - oVirt List Archives

Re: [ovirt-users] hosted engine health check issues
by Jiri Moskovcak 17 Apr '14

17 Apr '14

On 04/17/2014 09:34 AM, René Koch wrote: > On 04/15/2014 04:53 PM, Jiri Moskovcak wrote: >> On 04/14/2014 10:50 AM, René Koch wrote: >>> Hi, >>> >>> I have some issues with hosted engine status. >>> >>> oVirt hosts think that hosted engine is down because it seems that hosts >>> can't write to hosted-engine.lockspace due to glusterfs issues (or at >>> least I think so). >>> >>> Here's the output of vm-status: >>> >>> # hosted-engine --vm-status >>> >>> >>> --== Host 1 status ==-- >>> >>> Status up-to-date : False >>> Hostname : 10.0.200.102 >>> Host ID : 1 >>> Engine status : unknown stale-data >>> Score : 2400 >>> Local maintenance : False >>> Host timestamp : 1397035677 >>> Extra metadata (valid at timestamp): >>> metadata_parse_version=1 >>> metadata_feature_version=1 >>> timestamp=1397035677 (Wed Apr 9 11:27:57 2014) >>> host-id=1 >>> score=2400 >>> maintenance=False >>> state=EngineUp >>> >>> >>> --== Host 2 status ==-- >>> >>> Status up-to-date : True >>> Hostname : 10.0.200.101 >>> Host ID : 2 >>> Engine status : {'reason': 'vm not running on this >>> host', 'health': 'bad', 'vm': 'down', 'detail': 'unknown'} >>> Score : 0 >>> Local maintenance : False >>> Host timestamp : 1397464031 >>> Extra metadata (valid at timestamp): >>> metadata_parse_version=1 >>> metadata_feature_version=1 >>> timestamp=1397464031 (Mon Apr 14 10:27:11 2014) >>> host-id=2 >>> score=0 >>> maintenance=False >>> state=EngineUnexpectedlyDown >>> timeout=Mon Apr 14 10:35:05 2014 >>> >>> oVirt engine is sending me 2 emails every 10 minutes with the following >>> subjects: >>> - ovirt-hosted-engine state transition EngineDown-EngineStart >>> - ovirt-hosted-engine state transition EngineStart-EngineUp >>> >>> In oVirt webadmin I can see the following message: >>> VM HostedEngine is down. Exit message: internal error Failed to acquire >>> lock: error -243. >>> >>> These messages are really annoying as oVirt isn't doing anything with >>> hosted engine - I have an uptime of 9 days in my engine vm. >>> >>> So my questions are now: >>> Is it intended to send out these messages and detect that ovirt engine >>> is down (which is false anyway), but not to restart the vm? >>> >>> How can I disable notifications? I'm planning to write a Nagios plugin >>> which parses the output of hosted-engine --vm-status and only Nagios >>> should notify me, not hosted-engine script. >>> >>> Is is possible or planned to make the whole ha feature optional? I >>> really really really hate cluster software as it causes more troubles >>> then standalone machines and in my case the hosted-engine ha feature >>> really causes troubles (and I didn't had a hardware or network outage >>> yet only issues with hosted-engine ha agent). I don't need any ha >>> feature for hosted engine. I just want to run engine virtualized on >>> oVirt and if engine vm fails (e.g. because of issues with a host) I'll >>> restart it on another node. >> >> Hi, you can: >> 1. edit /etc/ovirt-hosted-engine-ha/{agent,broker}-log.conf and tweak >> the logger as you like >> 2. or kill ovirt-ha-broker & ovirt-ha-agent services > > Thanks for the information. > So engine is able to run when ovirt-ha-broker and ovirt-ha-agent isn't > running? > - yes, it might cause some problems if you set up another host for hosted engine and run the agent on the other host, but as long as you don't have the agent running anywhere or you don't need to migrate the engine vm, you should be fine. --Jirka > > Regards, > René > >> >> --Jirka >>> >>> Thanks, >>> René >>> >>> >>

2 1

Nagios/Icinga monitoring plugin check_rhev3 1.4 released
by René Koch 17 Apr '14

17 Apr '14

I'm happy to announce version 1.4 of check_rhev3. check_rhev3 is a monitoring plugin for Icinga/Nagios and it's forks, which is used to monitor datacenters, clusters, hosts, vms, vm pools and storage domains of Red Hat Enterprise Virtualization (RHEV) and oVirt virtualization environments. Download this plugin from: https://github.com/ovido/check_rhev3/releases/check_rhev3-1.4 For further information on how to install this plugin visit: https://github.com/ovido/check_rhev3/wiki/Installation-Documentation A detailed usage documentation can be found here: https://github.com/ovido/check_rhev3/wiki/Usage-Documentation Changelog: * New features: - Get status of vms on a specific host (Issue #8) - Proxy/No Proxy environment variable support (Issue #12) - Removed RHEV password from debug output and added plugin version number (Issue #13) - Default port changed to 443 (Issue #15) - Improved source code formatting (Issue #16) - Default warning and critical thresholds for CPU load based on number of cores (Issue #17) - More detailed status information for datacenters, hosts, vms and storagedomains (Issue #18) - PNP4Nagios templates for datacenter, hosts and vms stati (Issue #23) * Bugs fixed: - Interface traffic in Mbytes/s instead of Bytes/s (Issue #14) - Authentication fails if password contains equal sign (Issue #21) - Initialization of variable in check_rhev3.pl - thanks Administratoor (Issue #22) - Memory usage of vms in hypervisor set to 0 if negative (Issue #24) Please note! - Default port of API has change to 443. If you're using a different port please update your Nagios command definition to use your port (add option -p). - Performance data for datacenter, host, vm and storagedomain status now provide more detailed information and therefor use a new format. Please convert or remove your old PNP4Nagios RRD databases in order to be able to use the new feature. - Network traffic calculation is fixed, so please remove your old PNP4Nagios RRD databases for the network graphs. If you have any questions or ideas, please drop me an email: rkoch(a)linuxland.at. Thank you for using check_rhev3. -- Best Regards René Koch Senior Solution Architect ============================================ LIS-Linuxland GmbH Brünner Straße 163, A-1210 Vienna Phone: +43 1 236 91 60 Mobile: +43 660 / 512 21 31 E-Mail: rkoch(a)linuxland.at ============================================

3 4

vdsm host hooks
by Maurice James 17 Apr '14

17 Apr '14

------=_Part_266_1298254940.1397693635990 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Does anyone know of a good reference/how-to for vdsm host hooks? ------=_Part_266_1298254940.1397693635990 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 7bit <html><body><div style="font-family: times new roman, new york, times, serif; font-size: 12pt; color: #000000"><div>Does anyone know of a good reference/how-to for vdsm host hooks?</div><div><span name="x"></span><div><span style="font-family: 'trebuchet ms', sans-serif;" data-mce-style="font-family: 'trebuchet ms', sans-serif;"></span></div></div></div></body></html> ------=_Part_266_1298254940.1397693635990--

3 2

HA question
by Maurice James 16 Apr '14

16 Apr '14

------=_Part_869_900338456.1397586237056 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit If I lose the host with the engine manager, will HA still function? ------=_Part_869_900338456.1397586237056 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 7bit <html><body><div style="font-family: times new roman, new york, times, serif; font-size: 12pt; color: #000000"><div>If I lose the host with the engine manager, will HA still function?<br data-mce-bogus="1"></div></div></body></html> ------=_Part_869_900338456.1397586237056--

4 6

[Users] Instructions to add a remote controlled power strip not in the native list?
by David Smith 16 Apr '14

16 Apr '14

We use Raritan / Dominion PX remote power blocks, is there a way to easily add support for these? On a feature request side, seems like adding support for generic wouldn't be too hard; just ask for what the login/password prompts look like, what the main prompt is, and the command sequence to turn on and off a port as well as logout.

6 40

Export domain. Same LUN but different NFS server : update SQL table?
by Nicolas Ecarnot 16 Apr '14

16 Apr '14

Hi, Our NFS storage domain is stored as a LUN, and served via a NFS server. I have to change this NFS server, but I don't need to change the LUN. After setting this NFS export domain in maintenance and detached, is it safe to just update the database table "storage_server_connections" and restart the engine? Regards, PS : I read no answer to this : http://lists.ovirt.org/pipermail/users/2014-March/022610.html -- Nicolas Ecarnot

1 2

off topic = newbie with bugzilla, gerrit and ovirt VDSM 4.14
by Tamer Lima 16 Apr '14

16 Apr '14

Hi, I have a problem with an automatic yum vdsm upgrade (4.13 to 4.14) on ovirt 3.3 : Host x.x.x.x is installed with VDSM version (4.14) and cannot join cluster Default which is compatible with VDSM versions [4.13,4.9,4.11,4.12,4.10]. I am searching the solution and one of them (from bz Bug 1083008 ) points to gerrit site(http://gerrit.ovirt.org/#/c/23456/) I do not have any experience in solving bugs, neither bz, gerrit, github.... How to initialize in this world and solve the ovirt problem ? another thing : does downgrade or remove vdms package excludes the installed virtual machines ? I know about ovirt 3.4 is the new release, but I want to learn how to solve some bugs when new releases is not launched; (off course I am afraid to upgrade to ovirt 3.4 too :) ) Tamer

4 6

Re: [ovirt-users] SPM error
by Maurice James 16 Apr '14

16 Apr '14

What did you do to try to fix the sanlock? Anything is better than nothing at this point ----- Original Message ----- From: "Ted Miller" <tmiller(a)hcjb.org> To: "Maurice James" <mjames(a)media-node.com> Sent: Friday, April 11, 2014 7:27:24 PM Subject: Re: [ovirt-users] SPM error I did receive some help on one stage of rebuilding my sanlock, but there were too many other things wrong to get it started again. Only advice I have is -- look at your sanlock logs, and see if you can find anything there that is helpful. On 4/11/2014 7:23 PM, Maurice James wrote: > Nooooooo. > > > Sent from my Galaxy S®III > > -------- Original message -------- > From: Ted Miller <tmiller(a)hcjb.org> > Date:04/11/2014 7:08 PM (GMT-05:00) > To: Maurice James <mjames(a)media-node.com> > Subject: Re: [ovirt-users] SPM error > > > > I didn't, really. I did something wrong along the way, and ended up having > to rebuild the engine and hosts. (My problems were due to a glusterfs > split-brain.) > Ted Miller > > On 4/11/2014 6:03 PM, Maurice James wrote: >> How did you fix it? >> >> >> Sent from my Galaxy S®III >> >> -------- Original message -------- >> From: Ted Miller <tmiller(a)hcjb.org> >> Date:04/11/2014 6:00 PM (GMT-05:00) >> To: users(a)ovirt.org >> Subject: Re: [ovirt-users] SPM error >> >> >> >> On 4/11/2014 2:05 PM, Maurice James wrote: >>> I have an error trying to bring the master DC back online. After several >>> reboots, no luck. I took the other cluster members offline to try to >>> troubleshoot. The remaining host is constantly in contention with itself >>> for SPM >>> >>> >>> ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] >>> (DefaultQuartzScheduler_Worker-40) [38d400ea] >>> IrsBroker::Failed::GetStoragePoolInfoVDS due to: >>> IrsSpmStartFailedException: IRSGenericException: IRSErrorException: >>> SpmStart failed >>> >> I'm no expert, but the last time I beat my head on that rock, something was >> wrong with my sanlock storage. YMMV >> Ted Miller >> Elkhart, IN, USA >> -- "He is no fool who gives what he cannot keep, to gain what he cannot lose." - - Jim Elliot For more information about Jim Elliot and his unusual life, see http://www.christianliteratureandliving.com/march2003/carolyn.html. Ted Miller Design Engineer HCJB Global Technology Center, a ministry of Reach Beyond 2830 South 17th St Elkhart, IN 46517 574--970-4272 my desk 574--970-4252 receptionist

5 13

[Users] Otopi pre-seeded answers and firewall settings
by Giuseppe Ragusa 16 Apr '14

16 Apr '14

--_a5ec309a-8526-4910-aaa4-2d74d77ba60d_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi all=2C I'm trying to automate as much as possible of ovirt-hosted-engine-setup and= engine-setup by means of otopi answer files passed in using "--config-appe= nd=3Dfilename.conf". I succeded in forcing engine-setup to leave my iptables settings alone with= : OVESETUP_CONFIG/firewallManager=3Dstr:iptables OVESETUP_CONFIG/updateFirewall=3Dbool:False but ovirt-hosted-engine-setup still modified my iptables settings even with= the following options: OVEHOSTED_NETWORK/firewallManager=3Dstr:iptables OVEHOSTED_NETWORK/iptablesEnable=3Dbool:False Maybe I used the wrong option (deduced by looking inside source code). Does anybody have any hint/suggestion? Many thanks in advance=2C Giuseppe = --_a5ec309a-8526-4910-aaa4-2d74d77ba60d_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable <html> <head> <style></style></head> <body class=3D'hmmessage'><div dir=3D'ltr'>Hi all=2C<br>I'm trying to autom= ate as much as possible of ovirt-hosted-engine-setup and engine-setup by me= ans of otopi answer files passed in using "--config-append=3Dfilename.conf"= .<br><br>I succeded in forcing engine-setup to leave my iptables settings a= lone with:<br><br>OVESETUP_CONFIG/firewallManager=3Dstr:iptables<br>OVESETU= P_CONFIG/updateFirewall=3Dbool:False<br><br>but ovirt-hosted-engine-setup s= till modified my iptables settings even with the following options:<br><br>= OVEHOSTED_NETWORK/firewallManager=3Dstr:iptables<br>OVEHOSTED_NETWORK/iptab= lesEnable=3Dbool:False<br><br>Maybe I used the wrong option (deduced by loo= king inside source code).<br><br>Does anybody have any hint/suggestion?<br>= <br>Many thanks in advance=2C<br>Giuseppe<br> </div></body> </html>= --_a5ec309a-8526-4910-aaa4-2d74d77ba60d_--

5 13

VM samba4 is down. Exit message: 'int' object has no attribute 'replace'.
by René Koch 16 Apr '14

16 Apr '14

Hi, I configured Network QoS in my oVirt 3.4 setup, but can't start vms. Here are the steps I did: - Create QoS in Data Centers - Network QoS tab (Name: testing, limits: 10/10/100, 10/10/100) - Add QoS to vNIC Profile using Networks - vNIC Profile - edit When I try to start a vm the following error is shown in vdsm.log: Thread-837844::ERROR::2014-04-14 12:45:51,822::vm::2289::vm.Vm::(_startUnderlyingVm) vmId=`d7034850-c274-48f7-ac0f-b3d4cc57c255`::The vm start process failed Traceback (most recent call last): File "/usr/share/vdsm/vm.py", line 2249, in _startUnderlyingVm self._run() File "/usr/share/vdsm/vm.py", line 3135, in _run domxml = hooks.before_vm_start(self._buildCmdLine(), self.conf) File "/usr/share/vdsm/vm.py", line 2965, in _buildCmdLine self._appendDevices(domxml) File "/usr/share/vdsm/vm.py", line 2922, in _appendDevices deviceXML = dev.getXML().toxml(encoding='utf-8') File "/usr/lib64/python2.6/xml/dom/minidom.py", line 45, in toxml return self.toprettyxml("", "", encoding) File "/usr/lib64/python2.6/xml/dom/minidom.py", line 59, in toprettyxml self.writexml(writer, "", indent, newl) File "/usr/share/vdsm/vm.py", line 5268, in __hacked_writexml node.writexml(writer, indent + addindent, addindent, newl) File "/usr/share/vdsm/vm.py", line 5268, in __hacked_writexml node.writexml(writer, indent + addindent, addindent, newl) File "/usr/share/vdsm/vm.py", line 5256, in __hacked_writexml xml.dom.minidom._write_data(writer, attrs[a_name].value) File "/usr/lib64/python2.6/xml/dom/minidom.py", line 301, in _write_data data = data.replace("&", "&").replace("<", "<") AttributeError: 'int' object has no attribute 'replace' Thread-837844::DEBUG::2014-04-14 12:45:51,826::vm::2731::vm.Vm::(setDownStatus) vmId=`d7034850-c274-48f7-ac0f-b3d4cc57c255`::Changed state to Down: 'int' object has no attribute 'replace' When removing the QoS (or better say put it to unlimited) my vm is starting fine. My host is running CentOS 6.5 with latest updates. -- Best Regards René Koch Senior Solution Architect ============================================ LIS-Linuxland GmbH Brünner Straße 163, A-1210 Vienna Phone: +43 1 236 91 60 Mobile: +43 660 / 512 21 31 E-Mail: rkoch(a)linuxland.at ============================================

5 5