Re: [ovirt-devel] scale & hooks
On Jun 30, 2014, at 14:18 , Dan Kenigsberg <danken(a)redhat.com> wrote:
On Mon, Jun 30, 2014 at 01:04:09PM +0200, Michal Skrivanek wrote:
> Hi,
> just thinking about the recent stats hook[1] by Dima and scale,
> It is a very nice hook and quite useful for the scale testing - ( though when focus
is on engine I'd suggest to use FakeVDSM instead to eliminate all the bottlenecks on
VDSM side.)
>
> One issue I see - since this is a very high-profile call, every 15s time number of
VMs - is with the hook invocation. Currently the hook mechanism always go through the
hooks dit and does stat() call to see script is there or not…
> it certainly doesn't matter on small scale but when the system is on the edge
every IO on local FS cost something….
> Would be nice if the hook mechanism is enhanced in general, figure out which hooks
are relevant only at the startup and don't waste time passing domainXML every single
time when the hook is not there
But a caching mechanism for hook scripts costs "something", too (code
size, bugs, simplicty of testing new script). Before complicating the
mechanism, I'd rather see numbers: how much would be saved? I would
guess that filesystem caching already shaves costs of stat()s. Writing
domxml to a memory-map file before starting a vm may be insignificant,
too.
I don't expect it's a big impact, but I'm worried about effects of filesystem
access issues (say, very stressed environment with high IO), we used to have issues with
vdsm persistent files;
I think doing the scan only on startup is easier and may even make the code simpler (yes,
ignoring the use case of adding a new hook on the fly, but I don't think it matters
much and in some cases I'd even prefer to control the deployment of my new fancy
hook(s) and trigger them only on vdsmd restart)
high level comparison is always tricky. On one hand you won't see anything like
"it makes a CPU usage drop by 1/2", however every single thing adds up; and as I
said, it may only be ever relevant for things which are executed very often, like's
the case of *_get_vm_stats.
Of course to eliminate the number of calls from engine would be much much more important
Dan.