[ovirt-devel] scale & hooks

[Michal Skrivanek]

On Jun 30, 2014, at 14:18 , Dan Kenigsberg <danken at redhat.com> wrote:

> On Mon, Jun 30, 2014 at 01:04:09PM +0200, Michal Skrivanek wrote:
>> Hi,
>> just thinking about the recent stats hook[1] by Dima and scale, 
>> It is a very nice hook and quite useful for the scale testing - ( though when focus is on engine I'd suggest to use FakeVDSM instead to eliminate all the bottlenecks on VDSM side.)
>> 
>> One issue I see -  since this is a very high-profile call, every 15s time number of VMs - is with the hook invocation. Currently the hook mechanism always go through the hooks dit and does stat() call to see script is there or not…
>> it certainly doesn't matter on small scale but when the system is on the edge every IO on local FS cost something….
> 
>> Would be nice if the hook mechanism is enhanced in general, figure out which hooks are relevant only at the startup and don't waste time passing domainXML every single time when the hook is not there
> 
> But a caching mechanism for hook scripts costs "something", too (code
> size, bugs, simplicty of testing new script). Before complicating the
> mechanism, I'd rather see numbers: how much would be saved? I would
> guess that filesystem caching already shaves costs of stat()s. Writing
> domxml to a memory-map file before starting a vm may be insignificant,
> too.

I don't expect it's a big impact, but I'm worried about effects of filesystem access issues (say, very stressed environment with high IO), we used to have issues with vdsm persistent files; 
I think doing the scan only on startup is easier and may even make the code simpler (yes, ignoring the use case of adding a new hook on the fly, but I don't think it matters much and in some cases I'd even prefer to control the deployment of my new fancy hook(s) and trigger them only on vdsmd restart)


high level comparison is always tricky. On one hand you won't see anything like "it makes a CPU usage drop by 1/2", however every single thing adds up; and as I said, it may only be ever relevant for things which are executed very often, like's the case of *_get_vm_stats.
Of course to eliminate the number of calls from engine would be much much more important

> 
> Dan.