Re: [ovirt-users] high number of IOPS from ovirt hosts

Hi, just a quick update on the issue. The high amount of IOPS was indeed due to having the hosted-engine on the EQL storage; we worked around the issue by moving the hosted-engine to a gluster storage instead.

Andrea On 13/02/2017 13:24, Nir Soffer wrote: On Mon, Feb 13, 2017 at 12:32 PM, Andrea Fagiani < andrea.fagiani(a)immobiliare.it&gt; wrote: > Hi Nir, > > the amount of IOPS is pretty much constant, attaching the logs from the > last hour or so. > Andrea, can you send also logs from ovirt-hosted-engine-ha? (I'm not sure about the name of the service or the location of the logs) > Andrea > > On 13/02/2017 10:13, Nir Soffer wrote: > > On Mon, Feb 13, 2017 at 11:05 AM, Andrea Fagiani < > andrea.fagiani(a)immobiliare.it&gt; wrote: > >> Thanks for the heads up, I'll take a look at Mark's thread. >> We are presenting 2 storage LUNs to a single ovirt cluster and yes, all >> hosts are hosted engine hosts. >> >> Thanks, >> Andrea > > > I think we already have hosted engine bug about doing to many calls to > vdsm > that cause reloads of lvm data; this can cause increased io. > > Can you attach vdsm and hosted engine agent logs showing the timeframe > when > you have lot of iops on the server? > > Nir > > >> >> >> >> On 12/02/2017 18:27, Andrea Ghelardi wrote: >> >>> Andrea, >>> I don't want to point any fingers but Mark Greenall faced some issues >>> with Ovirt 4.06/4.1 linked to equallogic as well. >>> You may want to search posts in this ML with subject " Ovirt 4.0.6 >>> guests 'Not Responding' " to see if you find any similarities. >>> How many storages are you presenting to how many clusters? >>> Also, are your nodes all Hosted Engine Hosts? In my old configuration, >>> normal hosts read IOPS ranged from 50 to 500 while HE ones peaked even ~3000 >>> Cheers >>> AG >>> >>> >>> -----Original Message----- >>> From: Andrea Fagiani [mailto:andrea.fagiani@immobiliare.it] >>> Sent: Friday, February 10, 2017 5:56 PM >>> To: Andrea Ghelardi <a.ghelardi(a)iontrading.com>; users(a)ovirt.org >>> Subject: Re: [ovirt-users] high number of IOPS from ovirt hosts >>> >>> Hi Andrea, >>> >>> thanks for the insight; no I wasn't able to pinpoint the issue to a >>> single host, the load seems to be evenly spread. >>> We have another cluster running 4.0.6 on a compellent as well and we >>> haven't experienced any similar issues. >>> >>> Thanks, >>> Andrea >>> >>> >>> On 10/02/2017 17:17, Andrea Ghelardi wrote: >>> >>>> Hello Andrea, >>>> I saw that happening on my HE hosts connected to a COMPELLENT SC040 >>>> when I was running Ovirt 3.5 >>>> >>>> All HE hosts were hammering the SAN with 2~4k IOPS each >>>> >>>> Upgrading to 4.0.5 solved my case. >>>> Are you able to check if IOPS are generated from a particular host or >>>> rather spread among all nodes? >>>> >>>> Cheers >>>> AG >>>> >>>> >>>> -----Original Message----- >>>> From: users-bounces(a)ovirt.org [mailto:users-bounces@ovirt.org] On >>>> Behalf Of Andrea Fagiani >>>> Sent: Friday, February 10, 2017 12:47 PM >>>> To: users(a)ovirt.org >>>> Subject: [ovirt-users] high number of IOPS from ovirt hosts >>>> >>>> Hi, >>>> >>>> We've got a 8-host cluster running oVirt 4.1; each host is connected >>>> via iscsi to a EQL 4100 storage (2x). >>>> Most of the VMs use barely any IOPS, however on our storage controller >>>> we've noticed a constant steam of reads, about 2000 IOPS; any idea what >>>> might cause the ovirt hosts to hammer on the storage so much? >>>> >>>> Looking at iotop, it seems that systemd-udevd is being activated every >>>> few seconds (and its IO% is usually fairly high) so I assume this might be >>>> somehow related. >>>> >>>> Thanks, >>>> Andrea >>>> _______________________________________________ >>>> Users mailing list >>>> Users(a)ovirt.org >>>> http://lists.ovirt.org/mailman/listinfo/users >>>> >>> >> _______________________________________________ >> Users mailing list >> Users(a)ovirt.org >> http://lists.ovirt.org/mailman/listinfo/users >> > > > _______________________________________________ Users mailing list Users(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/users