[ovirt-users] high number of IOPS from ovirt hosts
Andrea Fagiani
andrea.fagiani at immobiliare.it
Tue Feb 14 15:08:57 UTC 2017
Correct,
coincidentally, there no instances of systemd-udevd going haywire either.
Andrea
On 14/02/2017 15:20, Yaniv Kaul wrote:
>
>
> On Tue, Feb 14, 2017 at 2:15 PM, Andrea Fagiani
> <andrea.fagiani at immobiliare.it <mailto:andrea.fagiani at immobiliare.it>>
> wrote:
>
> Hi,
>
> just a quick update on the issue. The high amount of IOPS was
> indeed due to having the hosted-engine on the EQL storage; we
> worked around the issue by moving the hosted-engine to a gluster
> storage instead.
>
>
> And now there are no high amount of IOPS on the Gluster storage?
> Y.
>
> Thanks to everyone involved,
> Andrea
>
>
> On 13/02/2017 13:24, Nir Soffer wrote:
>> On Mon, Feb 13, 2017 at 12:32 PM, Andrea Fagiani
>> <andrea.fagiani at immobiliare.it
>> <mailto:andrea.fagiani at immobiliare.it>> wrote:
>>
>> Hi Nir,
>>
>> the amount of IOPS is pretty much constant, attaching the
>> logs from the last hour or so.
>>
>>
>> Andrea, can you send also logs from ovirt-hosted-engine-ha?
>> (I'm not sure about the name of the service or the location of
>> the logs)
>>
>> Andrea
>>
>>
>> On 13/02/2017 10:13, Nir Soffer wrote:
>>> On Mon, Feb 13, 2017 at 11:05 AM, Andrea Fagiani
>>> <andrea.fagiani at immobiliare.it
>>> <mailto:andrea.fagiani at immobiliare.it>> wrote:
>>>
>>> Thanks for the heads up, I'll take a look at Mark's thread.
>>> We are presenting 2 storage LUNs to a single ovirt
>>> cluster and yes, all hosts are hosted engine hosts.
>>>
>>> Thanks,
>>> Andrea
>>>
>>>
>>> I think we already have hosted engine bug about doing to
>>> many calls to vdsm
>>> that cause reloads of lvm data; this can cause increased io.
>>>
>>> Can you attach vdsm and hosted engine agent logs showing the
>>> timeframe when
>>> you have lot of iops on the server?
>>>
>>> Nir
>>>
>>>
>>>
>>>
>>> On 12/02/2017 18:27, Andrea Ghelardi wrote:
>>>
>>> Andrea,
>>> I don't want to point any fingers but Mark Greenall
>>> faced some issues with Ovirt 4.06/4.1 linked to
>>> equallogic as well.
>>> You may want to search posts in this ML with subject
>>> " Ovirt 4.0.6 guests 'Not Responding' " to see if
>>> you find any similarities.
>>> How many storages are you presenting to how many
>>> clusters?
>>> Also, are your nodes all Hosted Engine Hosts? In my
>>> old configuration, normal hosts read IOPS ranged
>>> from 50 to 500 while HE ones peaked even ~3000
>>> Cheers
>>> AG
>>>
>>>
>>> -----Original Message-----
>>> From: Andrea Fagiani
>>> [mailto:andrea.fagiani at immobiliare.it
>>> <mailto:andrea.fagiani at immobiliare.it>]
>>> Sent: Friday, February 10, 2017 5:56 PM
>>> To: Andrea Ghelardi <a.ghelardi at iontrading.com
>>> <mailto:a.ghelardi at iontrading.com>>; users at ovirt.org
>>> <mailto:users at ovirt.org>
>>> Subject: Re: [ovirt-users] high number of IOPS from
>>> ovirt hosts
>>>
>>> Hi Andrea,
>>>
>>> thanks for the insight; no I wasn't able to pinpoint
>>> the issue to a single host, the load seems to be
>>> evenly spread.
>>> We have another cluster running 4.0.6 on a
>>> compellent as well and we haven't experienced any
>>> similar issues.
>>>
>>> Thanks,
>>> Andrea
>>>
>>>
>>> On 10/02/2017 17:17, Andrea Ghelardi wrote:
>>>
>>> Hello Andrea,
>>> I saw that happening on my HE hosts connected to
>>> a COMPELLENT SC040
>>> when I was running Ovirt 3.5
>>>
>>> All HE hosts were hammering the SAN with 2~4k
>>> IOPS each
>>>
>>> Upgrading to 4.0.5 solved my case.
>>> Are you able to check if IOPS are generated from
>>> a particular host or rather spread among all nodes?
>>>
>>> Cheers
>>> AG
>>>
>>>
>>> -----Original Message-----
>>> From: users-bounces at ovirt.org
>>> <mailto:users-bounces at ovirt.org>
>>> [mailto:users-bounces at ovirt.org
>>> <mailto:users-bounces at ovirt.org>] On
>>> Behalf Of Andrea Fagiani
>>> Sent: Friday, February 10, 2017 12:47 PM
>>> To: users at ovirt.org <mailto:users at ovirt.org>
>>> Subject: [ovirt-users] high number of IOPS from
>>> ovirt hosts
>>>
>>> Hi,
>>>
>>> We've got a 8-host cluster running oVirt 4.1;
>>> each host is connected via iscsi to a EQL 4100
>>> storage (2x).
>>> Most of the VMs use barely any IOPS, however on
>>> our storage controller we've noticed a constant
>>> steam of reads, about 2000 IOPS; any idea what
>>> might cause the ovirt hosts to hammer on the
>>> storage so much?
>>>
>>> Looking at iotop, it seems that systemd-udevd is
>>> being activated every few seconds (and its IO%
>>> is usually fairly high) so I assume this might
>>> be somehow related.
>>>
>>> Thanks,
>>> Andrea
>>> _______________________________________________
>>> Users mailing list
>>> Users at ovirt.org <mailto:Users at ovirt.org>
>>> http://lists.ovirt.org/mailman/listinfo/users
>>> <http://lists.ovirt.org/mailman/listinfo/users>
>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at ovirt.org <mailto:Users at ovirt.org>
>>> http://lists.ovirt.org/mailman/listinfo/users
>>> <http://lists.ovirt.org/mailman/listinfo/users>
>>>
>>>
>>
>>
>
>
> _______________________________________________
> Users mailing list
> Users at ovirt.org <mailto:Users at ovirt.org>
> http://lists.ovirt.org/mailman/listinfo/users
> <http://lists.ovirt.org/mailman/listinfo/users>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170214/6de1c7c3/attachment-0001.html>
More information about the Users
mailing list